Web Site Error Logging and Reporting System

C#Today
Welcome

13 Categories:

Application Development
C# Language
Web Services
XML
Performance
Security
Data Access, ADO.NET
SOAP, Data Transfer
Graphics, Games
Site Design
Mobile
E-Commerce
Other Tech

(Some articles have more than one themes, they appear in serveal categories.)
Brought to you by ownSky!
C#Today - © 2002 Wrox Press Limited


Web Site Error Logging and Reporting System

Application Development

Application Development (Sort by Name)

.NET Metadata and Custom Attributes in C#
.NET Web Data ToolKit
A .NET Data Provider
A Grid Layout Manager for your Windows Forms
A Multi-threaded search engine in C#
An In-Depth Look at the DateTimeFormatInfo Class
An In-Depth Look at the NumberFormatInfo Class
Application of the Cache Subsystem for Web Applications
ASP.NET Server Control Creation
Building a Forms Generator using Code DOM

Building a List Server
Building a Moderated Discussion Forum in C#
Building a special purpose search engine
Building a Type Viewer with System.Reflection
Building an Online Shopping Cart using C# and ASP.NET Part 1
Building an Online Shopping Cart Using C# Part 2
Creating a .NET assembly in C#
Creating a custom DataGridColumnStyle in C#
Creating a Distributed Application using Enterprise Templates
Creating a flexible sorting procedure using IComparators and the QuickSort Algorithm
Creating a Windows Service in C# - a Network Status Monitor

Creating an ISAPI FIlter HTTP Packet Analyzer
Creating Data-Driven Enterprise Applications using Cs and SQL Server
Creating International-Ready Applications
Customized Context Menus for your C# Forms
Data Transformation Engine
Demonstrating Attributes and Reflection with a Bug Tracker and Database Manager
Developing A Rich Client Forum Reader Using Windows Form and Web Service
Extending Application Functionality
Generating Design Patterns Using CodeDOM
Intranet Based Project Management Suite
Introducing Smart Device Extensions and the _NET Compact Framework

Localization using .NET and C#
Mail Tracking and Analysis System
MDI with C#
Migrating a Windows DNA Application to .NET Architecture
Modularizing Object Collections With The Collections API
Monitoring the File System for Tampering using the FileSystemWatcher Class
Most Recently Used File List Helper In C#
Multi-threading In C#
Network Programming in C#
Notification Framework with C#
Programmatically Managing Windows Services and Device Drivers with C#

Programming Internet Explorer in C#
Runtime Reverse Engineering with System.Diagnostics
Sharpening Your C# Unit Testing and Debugging Skills
Threads, Delegates and Queues
timeTrack An ASP.NET Contract Management System
'Trickle' download and deployment of smart-client Windows Forms applications over the web
Using the TreeView Control
Visual Studio .NET Add-Ins
Visual Studio .NET Add-Ins Part 2
Visual Studio .NET Custom Wizards

Web Site Error Logging and Reporting System
Windows Explorer in C#
Windows Forms User Interface Design with C#
WMI for .NET
Writing a Messenger Application Using C# - Part 1
Writing a Messenger Application using C# - Part 2


Creating a Distributed Application using Enterprise Templates

Creating a Distributed Application using Enterprise Templates
http://www.csharptoday.com/content.asp?id=2060
Article Rating: not yet rated
Published on December 30, 2002
Abstract
Today’s software applications are expected to deliver a wide variety of solutions to address different kinds of customer and business needs. Before .NET, the Windows DNA was the platform of choice for developing scalable and extensible applications for the Windows platform. With the arrival of .NET, things have not changed much in terms of application design or the reusable application framework; N-Tier application design is still the right way to go about developing enterprise class applications. When we were developing using the Windows DNA methodology, we used a pre-defined application framework to build applications. Every time we developed a new application, we had to start from scratch and add our application framework to it, and then go about developing the application. To avoid the repetitive work, Microsoft created a set of pre-defined templates known as Enterprise Templates that provided the base framework for developing N-Tier enterprise applications using the .NET Framework and Visual Studio.NET. These templates can be used as the basis for developing more complex applications. In this case study, Thiru Thangarathinam looks at how to leverage the features of the Distributed Enterprise Application Templates to create an airline reservation e-commerce site that allows customers to search for flights and book tickets. While discussing them, we will also build an application framework that consists of various application building blocks such as a data access application block, and an exception management building block.
Case Study

Today's software applications are expected to deliver a wide variety of solutions to address different kinds of customer and business needs. Before.NET, the Windows DNA was the platform of choice for developing scalable and extensible applications for the Windows platform. With the arrival of .NET, things have not changed much in terms of application design or the reusable application framework; N-Tier application design is still the right way to go about developing enterprise class applications. When we were developing using the Windows DNA methodology, we used a pre-defined application framework to build applications. Every time we developed a new application, we had to start from scratch and add our application framework to it, and then go about developing the application.

To avoid the repetitive work, Microsoft created a set of pre-defined templates known as Enterprise Templates that provided the base framework for developing N-Tier enterprise applications using the .NET Framework and Visual Studio.NET. These templates can be used as the basis for developing more complex applications. In this case study, we will see how to leverage the features of the Distributed Enterprise Application Templates to create an airline reservation e-commerce site that allows customers to search for flights and book tickets. While discussing them, we will also build an application framework that consists of various application building blocks such as a data access application block, and an exception management building block.

Introduction

One of the compelling features of Visual Studio .NET is the feature-set it provides which makes it possible to design and develop effective applications using a variety of mechanisms. One such feature is the Enterprise Templates that are available with the Enterprise Developer and Enterprise Architect versions of Visual Studio.NET. The templates in Visual Studio .NET contain an initial project structure component, which is essentially the starting point from which a software architect can begin building an application. The component can include reusable software components, classes, and other pieces of technology. Templates also contain the policy associated with the project, in which an architect can outline which technologies a developer should use when creating the applications.

In VS.NET, Enterprise Templates consist of two major components:

  • An application template, which defines the initial project structure (base files, BCL assemblies that are included, etc.). You can draw analogies between the application template and the Active Template Library (ATL) from VS6. When you create a project in ATL, the environment gives you some basic boilerplate files (the IDL, co-class files, .h files, etc.). Similarly, you can define what boilerplate files a user gets when they create a new project based on your template.
  • A policy file, which allows you to customize the VS.NET environment to a significant degree. For example, you can institute a policy that disables the managed C++ compiler and refuses to import certain BCL libraries such as System.Remoting . You can also configure certain compiler options (e.g. disallow unsafe code).

It is also possible to customize the enterprise templates to have a common application framework. The frameworks are reusable components that can greatly simplify programming tasks by including pre-written code to handle procedures such as application management, exception handling, and so on. Together, the templates and frameworks enable companies to define, build, and manage applications based on best practices .

Role of Enterprise Templates in creating Distributed Applications

Developers have been faced with ever-increasing complexity as technologies and opportunities have proliferated. Enterprise templates and the Template Description Language will help to reduce this complexity, and will increase the ease of collaboration across the development team. The benefits provided by the enterprise templates can be summarized as follows. The enterprise templates.

  • Allows us to define the initial structure of the application.
  • Allows architects to provide architectural and technological guidance. Also makes it possible to define the reusable application building blocks for the application upfront.
  • Reduces complexity for developers by providing a set of clearly defined application development paths.

Let us demonstrate how to implement the features of the enterprise templates when developing our application.

Architecture of the System

This case study is based on a web site called AirlineReservation , which provides a onestop shop for consumers who want to find out information about flights, deals, and the latest travel news. It also provides consumers with the ability to book tickets for flights.

The architecture of our application is split into several layers. By calling them layers, we emphasize logical distribution rather than the physical distribution .

The application consists of the following tiers:

  • ASP.NET Web GUI layer: This layer consists of all the ASP.NET pages and user controls that are used to generate the user interface for our application.
  • ASP.NET Web Services layer: This layer consists of web service methods that allow us to expose the functionality of the AirlineReservation application to external consumers over HTTP and SOAP.
  • Business Façade layer: This layer consists of Visual C# class library components that allow us to provide a consistent interface into the business logic layer. This layer depends on the business logic layer for all of its functionalities.
  • Business Services layer: This layer again consists of Visual C# class library components that contain the core business logic for our application.
  • Data Services layer: This layer consists of the ADO.NET code to invoke the stored procedures in SQL Server 2000. The data services layer will leverage the data access application block from Microsoft for executing queries and stored procedures against the SQL Server database.
  • System Framework projects: These set of components provide the core building blocks for our application and they will be shared by all the layers of the application.
  • SQL Server Database: The SQL Server database contains all the tables and stored procedures.

Business Processes

Clearly the scope of the case study is to show how to develop an enterprise class application using Enterprise templates provided by Visual Studio.NET. But before we do that, we will detail the business processes that our application will consist of. The business processes we are going to discuss are as follows:

  • Login Process - Registered users are given the option to login to our site and perform operations such as searching for flights, booking tickets for flights, and so on. The login system allows the users to identify themselves to the system. The user must provide an existing user id and a valid password to be allowed to enter the system.
  • New user Registration process - New users are given the opportunity to become members of the site by filling out our online form. In this step, the user is asked to create a unique user id, which is used to identify the user to the system. The user can also fill out details like name and address that can be used by the site administrator to contact the user when necessary. After the user has entered all the details and once the data has been validated, then the user's profile is stored in the database for later retrieval.
  • Flight search process - Users can search for flights based on pre-defined search parameters like destination, departure location, arrival date, departure date, and so on.
  • Deals and News display process - In this process, the consumers can find out information about the hot deals and latest news.
  • Logout process - Allows the user to log out from the site, which internally nulls out the cookie, thereby terminating the session

Implementation

Now that we have had an introduction to the application architecture and business processes, let us construct the individual building blocks that are required for implementing this solution. We will split our discussion into the following sections.

  • Database Design
  • Creation of distributed application template - This template will provide the application framework on top of which all the application layers are constructed.
  • Implementation of System Framework components - This includes reusable building blocks (that provide the core foundation for the application framework) such as data access application block, exception management block, configuration sections handling block, and so on
  • Implementation of the data access layer component
  • Implementation of the business logic layer component
  • Implementation of the business façade layer component
  • Implementation of ASP.NET Web GUI application that provides the user interface for the airline reservation system

We will start our discussion by looking at the database design that is required to support our application.

Database Design

As we can see from the architecture of our system, we use SQL Server to store and provide information to our application. Since the main aim of the case study is to lay emphasis on .NET enterprise application development using Enterprise templates, the databases we are going to use will have the minimum number of tables required to implement this solution.

Firstly, we will consider the design of the AirlineReservation database.

AirlineReservation database design

This database is basically used to store and retrieve information about the list of users registered on our site. The database diagram for the AirlineReservation database is as follows:

The SQL scripts required for creating the database can be downloaded along with the support material of this case study. Let us briefly take a look at the stored procedures used by the application.

The UserLogIn stored procedure returns 1 if a record with the supplied user name and password is found in the Users table, otherwise it returns a value of -1. It uses output parameters to pass the results back to the caller.

CREATE Procedure UserLogin
(
@UserID varchar(20),
@Password varchar(10),
@RetValue int OUTPUT
)
As
BEGIN
SET NOCOUNT ON
SELECT *
FROM Users
WHERE UserID = @UserID
AND Password = @Password

IF @@Rowcount < 1
SELECT @RetValue = -1
ELSE
SELECT @RetValue = 1
RETURN 0
END
GO

As the name suggests, the AddUser procedure inserts a record into the users table using the supplied parameters.

Create proc AddUser(
@UserID char(20),
@Password char(10),
@Name varchar(128),
@Address varchar(128),
@City varchar(50),
@State char(2),
@Zip char(5))

as
Insert into Users(UserID,Password,Name,Address,City,State,Zip)

Values(@UserID,@Password,@Name,@Address,@City,@State,@Zip)
GO

The SearchFlight stored procedure allows us to search for flights based on the search parameters such as starting location, destination, arrival date, departure date, and travel class id. As a result, the stored procedure returns the list of flights that match the search criteria.

CREATE procedure dbo.SearchFlight
@sStartingFrom varchar(50)
,@sDestination varchar(50)
,@dArrivalDate datetime
,@dDepartureDate datetime
,@iTravelClassID int
as

begin
set nocount on
select F.*,TC.travel_class_id
from flights F inner join travel_class_seat_capacity TCSC
on TCSC.flight_id = F.flight_id
inner join travel_class TC
on TC.travel_class_id = TCSC.travel_class_id
where F.starting_from = @sStartingFrom
and F.destination = @sDestination
and F.arrival_date = @dArrivalDate
and F.departure_date = @dDepartureDate
and TC.travel_class_id = @iTravelClassID
and TCSC.number_of_seats > 0
end
GO

The AddBooking stored procedure allows us to save the details of a specific booking in the bookings table. After saving the supplied values in the table, it also returns the identify value generated by the booking id column.

CREATE procedure dbo.AddBooking
@iFlightID int
,@sPassengerID varchar(20)
,@iTravelClassID int
,@iBookingID int output
as

begin
set nocount on

Insert into Bookings
(flight_id,passenger_id,travel_class_id)
values
(@iFlightID,@sPassengerID,@iTravelClassID)
select @iBookingID = @@identity
end
GO

The GetInformationByTypeID stored procedure allows us to get information on things such as deals or news, based on the information type id .

CREATE procedure GetInformationByTypeID
@iInformationTypeID int
as
begin

set nocount on
select convert(varchar(12),info_date) as info_date,
[description] from Information
where information_type_id = @iInformationTypeID
return 0
end
GO

Creation of distributed application template

In this section, we will see how to use the Distributed Application template to create the base structure of our application. We will start by creating a new Visual C# Distributed Application , as shown in the following screenshot:

The Visual C# Distributed Application template allows us to create the standard distributed application solution using C# as the development language of choice. While creating the project, you will also be prompted to enter the location where your web UI projects and web service projects will be created.

Once the project is created, your solution explorer should look similar to the following:

As you can see from above, by default, Visual Studio.NET created seven different layers (or building blocks) as part of the solution. Within each layer, it is possible to include projects from different programming languages. Let us briefly look at each of the building blocks and understand its functionality.

Business Façade projects

The business façade layer allows us to provide a consistent interface into the business logic that is contained in the business logic layer objects. The following list summarizes the characteristics of the business façade layer objects:

  • It depends on the business logic layer for all of its functionalities
  • Invoked either by the WinUI or the WebUI layer
  • Returns the results returned by the business logic layer to the WinUI or WebUI layer
  • Maintains state across multiple calls to the business logic layer

Business Rules projects

The business rules layer contains the business objects that provide the business logic for the application. Most of the business rules of the system are encapsulated within these objects.

Some of the characteristics of the business rules layer objects are:

  • Receives requests from the business façade layer, applies the corresponding business rules and passes the results back to the business façade layer
  • Depends on the data access layer for the retrieval of data from the database
  • Stateless design, thereby increasing scalability

Data Access projects

The data access layer performs operations such as fetching data, executing stored procedures, or sending data to the back end store such as SQL Server. The data access layer is generally designed to have the following characteristics:

  • Receives requests from the business rules layer, to fetch or send data from/to the back end store
  • Mainly depends on high performance stored procedures for executing queries

System Framework projects

These set of projects are designed to provide core system services to the application. For example, in our application we might decide to handle the exceptions (that may occur in the application) in a standardized way. That piece of functionality can be implemented as part of the system framework and shared by the all of the layers within the application. In our application, we will implement the data access application block, exception management block, and custom configuration file handling classes as part of the system framework.

Web Service projects

As the name suggests, these set of projects allow us to provide public SOAP based interfaces to our application, which are accessible to web service clients that use SOAP, HTTP and XML.

Web UI projects

The Web UI projects allow us to create web applications that serve up HTML pages to the web clients.

The Web UI layer is generally designed to have the following characteristics:

  • Renders appropriate HTML content based on the type of the browser that is making the request
  • Also performs validation on the data entered by the user
  • Invokes the methods of the business façade layer and displays the returned results

Win UI projects

The Win UI projects are similar to Web UI projects except that they render the output in rich window forms.

Now that we have created the application template and the base structure for our application, let us look at the creation of the different components that make up the system framework projects of our application.

Implementation of system framework

In this section, we will create the system framework components that are shared by all the layers of the application. As we already mentioned, we will cover the following four components:

  • Data Access Application block
  • Exception Management Application block
  • Custom Configuration file section handling classes
  • Common objects - this consists of all the common objects

Let us look at the creation of data access application block.

Data Access Application Block

When you are developing the data access layer for your application, most of the time, you will be writing the same data access plumbing code again and again. You might want to wrap the common data access code in helper functions that allow you to call the stored procedure in a single line. With .NET, Microsoft themselves have created an application block named the data access application block that contains optimized data access code that will help you call stored procedures and issue SQL statements against a SQL Server database.

We can use it as one of the building blocks in our .NET application and reduce the amount of custom code that we need to write, test, and maintain. The data access application block also encapsulates performance and resource management best practices for accessing SQL Server database. Using the data access block, we can perform the following operations:

  • Invoke stored procedures or execute SQL commands
  • Specify parameter details
  • Return SqlDataReader , DataSet , or XmlReader objects

The data access block contains a class named SqlHelper that provides a set of static helper methods that can be used to execute a variety of different command types against an SQL Server database. It also contains a class named SqlHelperParameterCache that provides command parameter caching functionality that can be very useful in improving the performance of the application. The SqlHelper class provides five shared methods named ExecuteNonQuery , ExecuteDataset , ExecuteReader , ExecuteScalar , and ExecuteXmlReader . Each of the above methods provides a consistent set of overloads. This provides a well-defined pattern for executing a command by using the SqlHelper class.

For example, we can execute a stored procedure named GetInformationByTypeID that takes an informationTypeID as an argument using the following lines of code:

DataSet ds = SqlHelper.ExecuteDataset( connectionString, CommandType.StoredProcedure,
"GetInformationByTypeID",new SqlParameter("@iInformationTypeID", informationTypeID));

As you can see from the above, the ExecuteDataset method executes the stored procedure and returns a dataset as a result. For more information on the data access application block, please consult the following link from Microsoft: Microsoft Application Blocks for .NET.

For our application, we will download the data access application block from the link above and add it to our solution under system framework projects. Once the project is added to the solution, it can easily be referenced from other projects in the solution using the Project References option. Once the reference is added, we can start using the SqlHelper class (that is part of the data access application block) methods to execute stored procedures and issue SQL commands.

Exception Management Application Block

In an ideal world, all the code you write would always run without error. But the reality is that no matter how carefully you write your code, errors can and will occur. For that reason, it is good to have an efficient error handling routine in place that can handle the errors in a graceful manner. It is also important to realize that the effectiveness of any exception handling solution is gauged by the way it impacts the user's experience when working with the application. Having seen the amount of time and effort spent by the developers in creating exception management systems, Microsoft have created a powerful and extensible exception management application block that can be reused in any .NET application.

By using the exception management block, with a single line of application code you can easily log exception information to the event Log or extend it by creating your own components that log exception details to other data sources. As mentioned before, the exception management application block can easily be used as a building block in any .NET application. The exception management block provides the following benefits:

  • Allows us to manage exceptions in our applications in an efficient and consistent way
  • Makes it possible to clearly isolate exception handling code from the application logic
  • Allows for handling and logging of exceptions with the minimal amount of custom code

To use exception management in our application, we need to perform the following steps:

  • Download the Exception Management Application block from the following link: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnbda/html/emab-rm.asp&WROXEMPTOKEN=1041080Z1d0HHcdVW0lYDz16T3
  • Add the exception management and exception management interfaces projects to our solution.
  • Add a reference to the exception management using the Project References option.
  • Add a using statement to reference the exception management block from your custom class.
  • Now you can publish exceptions by calling the Publish method of the ExceptionManager class as shown in the following line of code:

catch ( Exception ex
            )
{
ExceptionManager.Publish( ex );
}

Most of the time, you will be satisfied with the functionality of the default publisher, which writes exception details to the event log. However, there are times where you may want to extend the exception management block by writing your own custom publishers which may log exception information to alternate locations. To accomplish this, you need to create a custom class that implements either the IExceptionPublisher or the IExceptionXmlPublisher interface defined within the Microsoft.ApplicationBlocks.ExceptionManagement.Interfaces.dll assembly.

For more information on the exception management block, please consult the Exception Management in .NET Architecture Guide.

Custom Configuration section handling classes

Before the arrival of .NET, we used to store global configuration settings such as connection strings in a centralized location like the registry. In the .NET world, all these global settings are stored in the application configuration file that is normally deployed along with the application code. For an ASP.NET web application, it will be the web.config file and for a WinUI application, it will be app.config .

For example, the following appSettings entry in the web.config file can be used to store the connection string:

<configuration>
<appSettings>
<add key="connectionstring" value="server=localhost;database=ASPToday;uid=sa;pwd=;"/>
</appSettings>
</configuration>

We can read the above connection string from our code by using the following statement:

string connString = new ConfigurationSettings.AppSettings["connectionString"];

This simple key/value mechanism (stored in the appSettings section) is perfect for many common needs, such as storing database connection strings at application scope; but it's not flexible enough for more complex data. As you can see, the data that is stored in the appSetttings section is in plain text, which renders it unfit for storing settings such as connection strings to the database and so on.

Fortunately, Microsoft also built in a mechanism for creating custom configuration data. Rather than reading a hard-coded list of tags recognized only via code within a specific application, the.NET framework can also read one or more configSections sections, which define the tag names the framework should expect to find in the remainder of the file. Before we look at the example application that implements the custom configuration section handling classes, let us understand the theory behind custom configuration sections. Custom configuration sections are placed into two main areas in the configuration file:

  • A configuration section declaration area: this is identified by the configSections element. For example, to create a configuration section named appConfigSection, we need to place the following settings within the web.config file:

<configuration>
<configSections>
<section name="appConfigSection" type="DistributedAppConfigHandler.ApplConfigSectionHandler,DistributedAppConfigHandler">
</section>
</configSections>
..
..
</configuration>

As you can see from the above, we use the type attribute to specify the section handler class (that implements the IConfigurationSectionHandler interface) that will be automatically invoked when the application tries to read the configuration information using the System.Configuration.ConfigurationSettings.GetConfig method. We will see the implementation of the ApplConfigSectionHandler class in a moment.

  • Configuration section settings area: this is the area where the actual configuration settings are stored. For example, in the following declaration we define the section settings under the appConfigSection element:

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
..
..

<appConfigSection>
<appSection name="connectionStringSection">
<appSetting key="airlinesConnectionString" value="server=localhost;uid=sa;pwd=thiru;database=AirlineReservation">
</appSetting>
</appSection>
</appConfigSection>
</configuration>

Now that we understand the theory behind the custom configuration classes, let us see how to add custom configuration section handling functionality to our application.

We will start by creating a new project named DistributedAppConfigHandler under the SystemFrameworksProjects folder. You can do this by right clicking on the SystemFrameworksProjects folder and selecting Add->New Project . In the new project dialog box, select Visual C# project from the project types and select Empty project from the list of templates. Once the project is created, remember to change the output type of the project from Console Application to Class Library by right clicking on the project and selecting Properties->General . Now that the project is created, add a new class named ApplConfigSectionHandler and add the following lines of code to it.

using System;
using System.Xml;
using
            System.Collections;
using
            System.Collections.Specialized;
using
            System.Configuration;

namespace
            DistributedAppConfigHandler
{

public class
            ApplConfigSectionHandler :
            IConfigurationSectionHandler
{
public
            ApplConfigSectionHandler()
{

}

public Object
            Create(Object parent, Object input, XmlNode
            node)
{
XmlNodeList appSectionNodes;
XmlNodeList
            appSettings;
NameValueCollection arrAppSetting;
ArrayList
            arrSettings = new ArrayList();
string name;
string
            sname;
string svalue;

appSectionNodes =
            node.SelectNodes("appSection");
//Loop through all the AppSection
            nodes and add them to the ArrayList
foreach(XmlNode
            appSectionNode in appSectionNodes)
{
arrAppSetting = new
            NameValueCollection();
name =
            appSectionNode.Attributes.GetNamedItem("name").Value;
appSettings
            = appSectionNode.SelectNodes("appSetting");
if (appSettings !=
            null)
{
//Loop through all the appSetting elements and add
            them to the arraylist
foreach(XmlNode appSetting in
            appSettings)
{
sname =
            appSetting.Attributes["key"].Value;
svalue =
            appSetting.Attributes["value"].Value;
arrAppSetting.Add(sname,
            svalue);
}
}
arrSettings.Add(new AppSection(name,
            arrAppSetting));
}
//Return the constructed object to the
            caller
return arrSettings;
}
}
}

Let us walk through the above lines of code. We start by implementing the IConfigurationSectionHandler interface. The IConfigurationSectionHandler interface provides a method named Create that is used to parse the XML of the configuration section. The object returned by the Create method is added to the configuration collection and is typically accessed by the GetConfig method of the System.Configuration.ConfigurationSettings class.

public class ApplConfigSectionHandler : IConfigurationSectionHandler
{

public Object Create(Object parent, Object input, XmlNode node)
{
XmlNodeList appSectionNodes;
XmlNodeList appSettings;
NameValueCollection arrAppSetting;
ArrayList arrSettings = new ArrayList();
string name;
string sname;
string svalue;

In the following line, we get references to all the appSection nodes present in the configuration file. We will represent each and every appSection node in the configuration file in a separate class named AppSection , which we will see in a moment.

appSectionNodes = node.SelectNodes("appSection");
//Loop through all the AppSection nodes and add them to the ArrayList

We then loop through all the elements contained in the collection of appSection nodes and retrieve the appSettings nodes contained in them.

         foreach(XmlNode
            appSectionNode in appSectionNodes)
{
arrAppSetting = new
            NameValueCollection();
name =
            appSectionNode.Attributes.GetNamedItem("name").Value;
appSettings
            = appSectionNode.SelectNodes("appSetting");
if (appSettings !=
            null)
{
//Loop through all the appSetting elements and add
            them to the arraylist

In the following lines of code, we retrieve the entire appSetting nodes (contained in the appSection node) and add them to a NameValueCollection object.

               foreach(XmlNode
            appSetting in appSettings)
{
sname =
            appSetting.Attributes["key"].Value;
svalue =
            appSetting.Attributes["value"].Value;
arrAppSetting.Add(sname,
            svalue);
}
}

Here we create an instance of the AppSection class (that represents each appSection node in the configuration sections) and pass the name of the appSection and the NameValueCollection object to its constructor. We also add the newly created AppSection object to another NameValueCollection object that acts as a container for all the appSection nodes in the configuration file.

arrSettings.Add(new AppSection(name,
            arrAppSetting));
}
//Return the constructed object to the
            caller
return arrSettings;
}
}

Now that we have had a look at the implementation of the ApplConfigSectionHandler class, let us see the implementation of the AppSection class.

AppSection class

As we already mentioned, this class represents an instance of the appSection node in the configuration section.

namespace
            DistributedAppConfigHandler
{

public class
            AppSection
{
private string _name;
private
            NameValueCollection _appSettings;

public AppSection(string
            name, NameValueCollection appSettings)
{
_name =
            name;
_appSettings = appSettings;
}

public string
            Name
{
get
{
return _name;
}
}
public string
            GetValue(string key)
{
string value = null;
if (
            _appSettings !=null)
{
//Get the corresponding value from the
            NameValueCollection
value = _appSettings[key.Trim()];
}
return value;
}
}
}

The AppSection class exposes a read only property named Name that is used to return the name of the appSection . It also exposes a method named GetValue that can return the value of the specific appSetting element based on the name of the appSetting element.

So far, we have implemented classes that can not only parse the configuration section, but also hold an instance of an appSection node. However, there is no way easy way for the consumers to get to the configuration sections. To this end, we will implement a helper class named ConfigHelper that will allow the consumers to get to the desired appSection in a seamless way. Let us look at the implementation of this class:

using System;
using System.Collections;
using System.Xml;
using System.Collections.Specialized;

Since we don't want anyone to inherit the ConfigHelper class, we will prefix this class with the sealed keyword.

namespace
            DistributedAppConfigHandler
{
public sealed class
            ConfigHelper
{

private ConfigHelper()
{
//
//
            TODO: Add constructor logic here
//
}

public static
            AppSection GetAppSectionByName(string name)
{
ArrayList
            appConfigSection = new ArrayList();
AppSection returnSection =
            null;
//Get reference to the root configuration
            section
appConfigSection = (ArrayList)
            System.Configuration.ConfigurationSettings.GetConfig("appConfigSection");
if
            (appConfigSection != null)
{
foreach(object obj in
            appConfigSection)
{
AppSection section =
            (AppSection)obj;
if (section.Name == name)
//Return the
            matching section to the caller
returnSection = section;
}
}
return returnSection;
}

}
}

As you can see from the above, the ConfigHelper class exposes one static method named GetAppSectionByName . As the name suggests, this method takes the name of an appSection node and returns the corresponding AppSection object. Once we get the reference to an AppSection object, we can then easily retrieve the individual appSetting values by invoking the GetValue method of the AppSection object.

Common Objects

As the name suggests, this project will be made up of the core objects and collections that are used in our application. Any time we invoke the business façade layer method to retrieve some data; we will always get the return value in the form of a strongly typed collection object. All of these collection objects and the individual objects that make up these collection objects are defined in the common objects project. For example, to store information about a user, we can create a UserObject , and a set of UserObjects can make up a UserCollection object.

Let us start by creating a new class named UserObject and add the following lines of code to it:

using System;
using System.Data;
using
            System.Data.SqlClient;
using
            System.Configuration;

namespace
            CommonObjects
{
[Serializable]
public class
            UserObject
{
public UserObject()
{

}

string
            userID;
string passWord;
string name;
string
            address;
string city;
string state;
string
            zip;

public string UserID
{
get
{
return
            userID;
}
set
{
userID = value;
}
}

public
            string PassWord
{
get
{
return
            passWord;
}
set
{
passWord =
            value;
}
}

public string
            Name
{
get
{
return name;
}
set
{
name =
            value;
}
}

public string
            Address
{
get
{
return
            address;
}
set
{
address =
            value;
}
}

public string
            City
{
get
{
return city;
}
set
{
city =
            value;
}
}

public string
            State
{
get
{
return state;
}
set
{
state =
            value;
}
}

public string Zip
{
get
{
return
            zip;
}
set
{
zip =
value;
}
}
}
}

In the above lines of code, we decorate our class with the attribute Serializable to allow our class to be serializable. This attribute makes it possible to convert the UserObject into a linear sequence of bytes for either storage or transmission to another location. Apart from the constructor, we have also declared a number of public get/set properties.

Now that we understand the UserObject class, let us look at how the UserCollection object is defined:

using System;
using
            System.Collections;
using System.IO;

namespace
            CommonObjects
{
[Serializable]
public class UserCollection
            : IEnumerator, ICollection
{
// Private member variable that
            holds the underlying Collection.
private ArrayList
            userArray;

// Private member variable that holds the index of
            the current UserCollection.
private int _index =
            -1;

public UserCollection()
{
userArray = new
            ArrayList();
}

public UserCollection(ArrayList
            theArrayList)
{
userArray = theArrayList;
}

public
            int Count
{
get
{
return
            userArray.Count;
}
}

public void Add(UserObject
            obj)
{
userArray.Add(obj);
}

public void
            Remove(UserObject
            obj)
{
userArray.Remove(obj);
}

public UserObject
            this[int indexer]
{
get
{
// Check if the indexer falls
            within the allowable range.
if ((indexer >= 0) &&
            (indexer < userArray.Count))
return
            (UserObject)userArray[indexer];
else
throw new
            System.IndexOutOfRangeException("Index must be between 0 and " +
userArray.Count.ToString() + ".");
}
}

public
            IEnumerator GetEnumerator()
{
return (IEnumerator) new
            UserCollection(userArray);
}

public object
            Current
{
get
{
return
            userArray[_index];
}
}

public bool
            MoveNext()
{
_index++;
return _index <
            userArray.Count;
}

public void Reset()
{
_index =
            -1;
}

public ArrayList GetArrayList()
{
return
            userArray;
}

public object SyncRoot
{
get{return
            this;}
}

public bool IsSynchronized
{
get{return
            false;}
}

public void CopyTo(Array a, int
            index)
{
userArray.CopyTo(a,index);
}
}
}

We start by deriving our collection class from the IEnumerator and ICollection interfaces. After that, we implement various properties and methods such as Count , IsSynchronized , SyncRoot , Current , MoveNext and Reset that are defined in the ICollection and IEnumerator interfaces. For more information on the methods of the ICollection interface, please refer to the ICollection Overview on MSDN.

The code for the remaining classes can be found inside the support material for this cases study.

Implementation of data access layer

In this section, we will see how to implement the data access layer for our application. As discussed before, this layer contains all the classes required for establishing connections with the databases, and executing stored procedures and SQL statements against the database.

We will start by navigating to the DataAccess class library project that is present under the DataAccessProjects folder in our solution explorer. To the DataAccess project, we will add a new class named UserDB and name the class file as UserDB.cs . Since we need to reference the classes present in the different system framework assemblies such as the data access block, exception management block, configuration section handling project, and the CommonObjects , we will need to add references to them. There are different ways by which you can reference one project from another project:

  • File based assembly reference: this is done by right clicking on the project in the solution explorer and selecting Add Reference->Browse and then navigating to the path of the assembly. This is normally used when you are trying to reference a third party assembly from your project; especially when the source code for the assembly is not available.
  • Relative reference using Project Reference: this technique allows us to reference one project from another project using relative references; but to be able to use this option, we need to make sure that all the related projects are contained within the same solution. If you are working in a team development environment, you will find this option very useful.

In our case, since we need to add references to other projects (that are contained within the same solution) from the DataAccess project, we will use the project references option. To perform this, we will right click on the DataAccess project and select Add Reference . In the Add Reference dialog box, we will navigate to the Projects tab and add the Microsoft.ApplicationBlocks.Data , Microsoft.ApplicationBlocks.ExceptionManagement , DistributedAppConfigHandler , and CommonObjects projects. After adding the reference, the dialog box should look like the following:

Once the reference is added, we can start implementing the methods in the UserDB class. Let us look at the methods of the UserDB class.

We will start by importing the different namespaces that we will be requiring in our application.

using System;
using System.Data;
using System.Data.SqlClient;
using System.Configuration;
using CommonObjects;
using Microsoft.ApplicationBlocks.Data;
using Microsoft.ApplicationBlocks.ExceptionManagement;
using DistributedAppConfigHandler;

namespace DataAccess
{

We declare the class as sealed to prevent this class from being inherited.

   public sealed class
            UserDB
{
private UserDB()
{
//
// TODO: Add
            constructor logic here
//
}

Here we declare a static method CheckLogin , that executes a stored procedure named UserLogIn . If the supplied login is valid, this method returns true; otherwise it raises an exception named InValidLoginException .

public static bool CheckLogIn(string userName, string passWord)
{

In the following line, we get a reference to the connectionString section in the web.config using the configuration helper classes that we created earlier.

//Get the reference to the connection string section
AppSection section = ConfigHelper.GetAppSectionByName("connectionStringSection");

Once we get the reference to the specific section, we can then retrieve the individual appSetting value in that section by calling the GetValue method.

//Get the connection string
string connString = section.GetValue("airlinesConnectionString");

try
{

In the following lines of code, we create an array of SqlParameter objects and set the contents of the array to appropriate values.

            SqlParameter[]
            arrParams = new SqlParameter[3];
//Add the User name
            parameter
arrParams[0]= new SqlParameter("@UserID",
            SqlDbType.VarChar, 20);
arrParams[0].Direction =
            ParameterDirection.Input;
arrParams[0].Value = userName;

//Add the password parameter
arrParams[1]= new
            SqlParameter("@Password", SqlDbType.VarChar,
            10);
arrParams[0].Direction =
            ParameterDirection.Input;
arrParams[1].Value =
            passWord;

//Add the return value
            parameter
arrParams[2] = new SqlParameter("@RetValue",
            SqlDbType.Int, 4);
arrParams[2].Direction =
            ParameterDirection.Output;

Once we create the required SqlParameter array, we then invoke the ExecuteNonQuery method of the SqlHelper class, passing in the connection string, the name of the stored procedure to execute, and the SqlParameter array.

// We pass in database connection string, command type, stored procedure name and an array of SqlParameter objects
SqlHelper.ExecuteNonQuery(connString, CommandType.StoredProcedure, "UserLogIn", arrParams);

After executing the stored procedure, we check the output parameter to see if it returns -1. If it returns -1, we then throw an exception to the caller; otherwise we return true to indicate that the supplied login is valid.

//Obtain the return value of the stored
            procedure into a variable
int retValue =
            (int)arrParams[2].Value;

if (retValue == -1)
{
throw
            new InValidLoginException("Invalid Login");
}
else
{
//Return true to indicate that the login is
            successful
return true;
}

}

If any exception occurs during the execution of any of the above statements, we then log the exception using the exception management application block's ExceptionManager class. After logging the exception, we also raise the exception back to the caller by using the throw keyword.

         catch
            (Exception ex)
{
//Publish the exception using the Exception
            Manager
ExceptionManager.Publish(ex);
throw
            ex;
}

}

The SaveUserInfo method is similar to the CheckLogin method except that in this case we execute a different stored procedure.

      public
            static bool SaveUserInfo(UserObject userInfo)
{
//Get the
            reference to the connection string section
AppSection section =
            ConfigHelper.GetAppSectionByName("connectionStringSection");
//Get
            the connection string
string connString =
            section.GetValue("airlinesConnectionString");

try
{

//Add the User name parameter
SqlParameter[] arrParams =
            new SqlParameter[7];
arrParams[0]= new SqlParameter("@UserID",
            SqlDbType.VarChar, 20);
arrParams[0].Value =
            userInfo.UserID;

//Add the password
            parameter
arrParams[1] = new SqlParameter("@Password",
            SqlDbType.VarChar, 10);
arrParams[1].Value = userInfo.PassWord;

//Add the Name parameter
arrParams[2] = new
            SqlParameter("@Name", SqlDbType.VarChar, 128);
arrParams[2].Value
            = userInfo.Name;

//Add the Address parameter
arrParams[3]
            = new SqlParameter("@Address", SqlDbType.VarChar,
            128);
arrParams[3].Value = userInfo.Address;

//Add the
            City parameter
arrParams[4] = new SqlParameter("@City",
            SqlDbType.VarChar, 50);
arrParams[4].Value = userInfo.City;

//Add the State parameter
arrParams[5] = new
            SqlParameter("@State", SqlDbType.Char, 2);
arrParams[5].Value =
            userInfo.State;

//Add the Zip parameter
arrParams[6] = new
            SqlParameter("@Zip", SqlDbType.Char, 5);
arrParams[6].Value =
            userInfo.Zip;

SqlHelper.ExecuteNonQuery(connString,
            CommandType.StoredProcedure,
"AddUser",arrParams);

return
            true;

}

catch (Exception ex)
{
//Publish the
            exception using the Exception
            Manager
ExceptionManager.Publish(ex);
throw ex;
}
}
}

}

Apart from the UserDB class, we also need to implement other classes in the data access layer. Since the focus of the case study is not on creating the data access layer, we will not be looking at all of them in detail. However you can download the code for the rest of the classes in the support material of this case study.

Implementation of business logic layer

In this section, we will demonstrate how to abstract the business logic for our application in a separate layer that acts as a bridge between the data access layer and the business façade layer. For this implementation, we will add classes to the BusinessRules project that is contained within the BusinessRulesProjects folder.

We will start by adding a new class named UserBiz to the BusinessRules project. Once the class is added, we will then add the following lines of code. As before, we import the required namespaces through using statements.

using System;
using System.Data;
using
            DataAccess;
using
            Microsoft.ApplicationBlocks.ExceptionManagement;
using
            CommonObjects;

namespace BusinessRules
{

public
            class UserBiz
{
public UserBiz()
{
}

Here we declare a method named CheckLogIn that invokes the CheckLogin method of the data access layer class ( UserDB class).

      public
            bool CheckLogIn(string userName, string passWord)
{
try
{
return UserDB.CheckLogIn(userName,passWord);
}

If any exception is thrown during the execution of the above statement, we catch the exception and log it using the Publish method of the ExceptionManager class. We also throw the exception back to the caller using the throw keyword.

         catch
            (Exception ex)
{
//Publish the exception using the Exception
            Manager
ExceptionManager.Publish(ex);
throw
            ex;
}

}

The SaveUserInfo method is implemented similar to CheckLogIn method except that it invokes a different method in the UserDB class.

      public
            bool SaveUserInfo(UserObject userInfo)
{
try
{
return
            UserDB.SaveUserInfo(userInfo);
}

catch (Exception
            ex)
{
//Publish the exception using the Exception
            Manager
ExceptionManager.Publish(ex);
throw
            ex;
}
}
}
}

Similar to the data access layer, the business logic layer also contains a number of classes and methods. For reasons of brevity, we will not examining all of them here.

Implementation of the business façade layer

In this section, we will look at the implementation of the business façade layer. As mentioned before, the business façade layer provides a wrapper around the business logic layer and provides a consistent interface into the business logic implementation. For this implementation, we will add classes to the BusinessFacade project that is contained within the BusinessFacadeProjects folder.

We will start by adding a new class named InformationBF to the BusinessFacade project. To the newly added class, we will add the following statements:

using System;
using System.Data;
using
            BusinessRules;
using CommonObjects;
using
            Microsoft.ApplicationBlocks.ExceptionManagement;

namespace
            BusinessFacade
{
public class InformationBF
{
public
            InformationBF()
{
}

Here we declare a method named GetInformation that takes the InformationTypeID as an argument and returns an object of type InformationCollection .

public InformationCollection
            GetInformation(int informationTypeID)
{
try
{
DataSet
            information;
InformationCollection col = new
            InformationCollection();

Now we create an instance of the business logic layer class named InformationBiz .

InformationBiz objInformation = new InformationBiz();

After creating an instance of the object, we then invoke the GetInformation method, passing in the informationTypeID as an argument.

information = objInformation.GetInformation(informationTypeID);

//Get reference to the first table in the dataset
DataTable dt = information.Tables[0];
//Convert the dataset into a collection object

Here we loop through all the rows in the DataTable object, and assign its values to an object of type InformationObject .

            foreach
            (DataRow row in dt.Rows)
{
DateTime defaultDateTime = new
            DateTime();
// Create and populate a new
            object.
InformationObject obj = new InformationObject();
obj.InfoDate = row["info_date"] != System.DBNull.Value ?
            (string)row["info_date"] : "" ;
obj.Description =
            row["description"] != System.DBNull.Value ?
            (string)row["description"] : "" ;

Then we add the InformationObject to the InformationCollection by invoking the Add method of the InformationCollection object.

// Add the object to the
            collection.
col.Add(obj);
}
return col;
}

Here again, we log the exceptions that may arise during the execution of the statements above, using the exception management application block.

         catch
            (Exception ex)
{
//Publish the exception using the Exception
            Manager
ExceptionManager.Publish(ex);
throw ex;
}

}

}
}

Since the other business façade layer classes are also implemented in the same fashion, we will not be examining all of them here.

Implementation of ASP.NET Web Application

Now that we have constructed the core building parts of the application, let us see how to implement the ASP.NET Web UI project that will leverage all of the building blocks we've created so far. For the Web UI, we will add a new Web UI project named AirlineReservation under the WebUIProjects folder. We will look at the implementation of the AirlineReservation web application by considering the different processes involved. Before we get into that discussion, let us go over the code of the user controls that we are going to use.

A Look at User Controls

User controls are very powerful and flexible in that they allow us to define our own controls using the same programming techniques that we use for writing web forms. For our application, we are going to create three self-contained reusable user controls. They are as follows:

  • The Header control ( Header.ascx ) allows us to provide a standard and consistent header for all of the web pages across our site.
  • The Left Navigation Bar control ( LeftNavBar.ascx ) gives the standard left hand side navigation hyperlinks for all the pages.
  • The control named InformationDisplayControl.ascx is used to display the latest news and travel deals available on our site. This control displays different information (either news or deals information) based on the values we set for the property named InformationTypeID . We will review the implementation of this control later, when we discuss the web form InformationDisplay.aspx that contains this user control.

Before we discuss the underlying code behind the header user control and the left navigation bar control, let us identify the steps to be followed while creating user-defined controls.

  • Create a user control and save it with the extension .ascx. Any file with the extension .ascx is identified as a user control and it cannot be executed independently as a standalone web page.
  • Include the user control in the container web form using the Register directive.

<%@ Register TagPrefix=" AirlineReservation " TagName="Header" Src="Header.ascx" %>

In the above directive, the TagPrefix indicates the unique namespace for the user control and the TagName is used to give a unique name for the control. The Src attribute is used to identify the relative virtual path to the user control.

  • Finally place the user control in the web form using the following line of code.

<AirlineReservation:Header id="Header1" runat="server">
</ AirlineReservation:Header>

Once we place the control using the directive above, we can invoke its properties and methods using the name that is provided in the id attribute.

Now that we have seen the steps involved in creating a user-defined control, let us create the header user control which is going to be embedded in all the pages of AirlineReservation application. The code for the user control Header.ascx looks like the following:

<%@ Control Language="c#" AutoEventWireup="false"
Codebehind="Header.ascx.cs" Inherits="AirlineReservation.Header" %>
<asp:Table id="tblTop" bgColor="maroon" style="Z-INDEX: 101; LEFT: 1px; POSITION: absolute;
TOP: 3px" runat="server" Width="819px" Height="110px" ForeColor="blue"
BackColor="#C04000">
<asp:TableRow>
<asp:TableCell>
<img src='Images\wroxlogo.gif'>
</asp:TableCell>
<asp:TableCell>
<img align='right' src='Images\csharptoday.gif'>
</asp:TableCell>
</asp:TableRow>
<asp:TableRow HorizontalAlign="Center">
<asp:TableCell ColumnSpan="2">
<asp:Label runat="server" ForeColor="White" ID="Label1" Font-Size="Medium">
Online Reservation System
</asp:Label>
</asp:TableCell>
</asp:TableRow>
</asp:Table>

The code involved in the creation of the header control above is pretty straightforward. The left navigation user control is similar to the above control except that it provides hyperlinks for performing different operations such as searching for flights, logging in to the site, logging out of the site, creating new memberships, and so on.

Login Process

In our site, the user must be logged in to perform tasks such as searching for the flights and booking the tickets. We will design the login page in such a way that it authenticates the customer's user name and password against the AirlineReservation database. Once validated, the user is then redirected to the search flights page. If the user does not have a valid login, he can opt to create one by clicking the hyperlink New Users Click here in the login page. Clicking this hyperlink takes the user to the registration page where the user provides all the necessary details for completing the registration. The login page ( Login.aspx ) looks like the following:

The Login page provides customers with a forms-based login authentication mechanism through which they can authenticate themselves to the AirlineReservation application. This form-based authentication technology depends on cookies to store the authentication information for the current user. Once the user is authenticated using the information present in the Users table, cookies are used to store and maintain session information, thereby allowing us to identify the current user.

We enable forms-based authentication in our application by making the following entry in the web.config file.

<authentication mode="Forms">
<forms name=" AirlineReservationAuth" loginUrl="Login.aspx" path="/">
</forms>
</authentication>

The loginUrl attribute specifies the name of the login page that we want the users to be redirected to, any time they access a resource that does not allow anonymous access.

In our application, since we do not want to allow anonymous users to browse the site, we will secure the following pages using entries in the web.config file:

  • Flights Search page ( SearchFlights.aspx )
  • Flights Search Results page ( SearchFlightsResults.aspx )
  • Book tickets page ( BookTickets.aspx )
  • Information Display Page ( InformationDisplay.aspx )

For example, to set restrictions on authenticated user access for a page called SearchFlights.aspx , we add the following entry directly under the configuration tag of the web.config file:

<location path="SearchFlights.aspx">
<system.web>
<authorization>
<deny users="?" />
</authorization>
</system.web>
</location>

Due to the above entry, when a user attempts to access the SearchFlights.aspx page, the ASP.NET forms-based security system will automatically redirect them to the Login.aspx page, and will continue to prevent them from accessing it until they have successfully validated their user name and password credentials to the AirlineReservation application. Similarly, we protect the other secured pages by adding corresponding entries in the web.config file.

Now let us see how to integrate this built-in form based authentication mechanism in our code. The code behind file ( login.aspx.cs ) of the login page encapsulates the actions to be performed when the user clicks on the Login button .

private void btnLogin_Click(object sender,
            System.EventArgs e)
{
try
{
UserBF user = new UserBF();
if (user.CheckLogIn(txtUserName.Text,txtPassword.Text))
{
FormsAuthentication.SetAuthCookie(txtUserName.Text,true);
//Set
            the UserID in the session variable
Session["UserID"] =
            txtUserName.Text;
Response.Redirect("SearchFlights.aspx");
}
}
catch(InValidLoginException ex)
{
lblMessage.Visible
            = true;
lblMessage.Text = ex.Message;;
}
}

In the above lines of code, we first invoke the FormsAuthentication.SetAuthCookie method to create the authentication cookie, and then invoke the Resonse.Redirect method to redirect the user to the search page. However, it is important to note that the same functionality can be accomplished by using the FormsAuthentication.RedirectFromLoginPage , which not only creates the authentication cookie but also redirects the user to the next page in a single step. Please refer to the Duwamish 7.0 MSDN Sample for more information on forms authentication security.

Here, we instantiate an object of type UserBF and assign its reference to the local variable user.

UserBF user = new UserBF();

As we already discussed, the CheckLogin method returns true if the login is successful, else it throws an exception of type InvalidLoginException .

if (user.CheckLogIn(txtUserName.Text,txtPassword.Text) == true)

If the login is successful, we then invoke the SetAuthCookie method to generate an authentication ticket for the authenticated user name and password and attach it to the cookies collection of the outgoing response. Once the cookie is generated, it is used to maintain information about the session information for every user that logs in to our site.

FormsAuthentication.SetAuthCookie(txtUserName.Text, true);

Now we store the valid user id in a session variable for later retrieval. In the next line, we perform a redirect to the SearchFlights.aspx page that allows the users to search for flights.

Session["UserID"] = txtUserName.Text;
Response.Redirect("SearchFlights.aspx");

Whenever the exception InValidLoginException occurs, we catch that in the catch block and display the exception message in the label control lblMessage . The label control named lblMessage , by default, has its visibility set to false and is used as a placeholder to display custom messages to the user.

catch(InValidLoginException ex)
{
lblMessage.Visible = true;
lblMessage.Text = ex.Message;;
}

Registration Process

The registration page Registration.aspx allows users that want to take advantage of AirlineReservation to register themselves as members. The registration page looks something like the following:

In the above screen, the user will enter all the necessary details and then click Save Details to save the information. Let us take a look at the code necessary to save the details entered by the user to the database.

private void btnSave_Click(object sender, System.EventArgs e)
{
UserBF userRegister = new UserBF();
CommonObjects.UserObject userInfo = new CommonObjects.UserObject();
//Assign values to the UserDetails object
userInfo.UserID = txtUserName.Text;
userInfo.PassWord= txtPassWord.Text;
userInfo.Name = txtName.Text;
userInfo.Address = txtAddress.Text;
userInfo.City = txtCity.Text;
userInfo.State = txtState.Text;
userInfo.Zip = txtZip.Text;
//Invoke the SaveUserInfo method to save the user details to the database
userRegister.SaveUserInfo(userInfo);
//Redirect the user to the Confirmation page
Server.Transfer("Confirmation.aspx");
}

We start the implementation of the registration process by declaring and instantiating an object of type User . Then we create an instance of object type UserInfo that represents a unique user in our site. Once we set the members of the UserInfo object to appropriate values, we then invoke the SaveUserInfo method of the User object. Finally we redirect the users to the confirmation page to provide the users with the confirmation message that they have been successfully registered.

The confirmation page looks like the following:

When the user clicks the Click here to log on to the site button, we invoke the Transfer method of the Server object to redirect the user back to the login page. This is accomplished using the following line of code.

private void btnLogin_Click(object sender, System.EventArgs e)
{
Server.Transfer("Login.aspx");
}

SearchFlights Process

This involves searching for flights based on parameters such as arrival date, start date, starting location, and so on.

When the user enters all the values and clicks Search, we transfer the control to the web form SearchFlightsResults.aspx . To the Server.Transfer method, we pass true in the second argument to ensure that the currently submitted form is preserved and made available to the SearchFlightsResults.aspx also.

private void btnSearch_Click(object sender, System.EventArgs e)
{
Server.Transfer("SearchFlightsResults.aspx",true);
}

Once the control is transferred to the SearchFlightsResults.aspx web form, it performs a sequence of operations to accomplish the desired functionality. Before we examine the code, let us take a look at the layout of the search flight results web form. It basically uses a data grid control to display the details of the search result. This control is bound directly to the collection returned by the business façade layer method. The DataGrid control displays the public properties of the object that is contained in the collection as columns in a table.

The DataGrid named grdSearchResults is defined as follows in the SearchFlightsResults.aspx file:

<asp:DataGrid id="grdSearchResults"
            style="Z-INDEX: 101; LEFT: 232px; POSITION: absolute; TOP: 127px"
            runat="server"
Height="363px" width="633px" BorderColor="Black"
            cellpadding="4" Font-Name="Verdana" Font-Size="8pt"
HeaderStyle-CssClass="CartListHead"
            FooterStyle-CssClass="cartlistfooter"
            ItemStyle-CssClass="CartListItem"
AlternatingItemStyle-CssClass="CartListItemAlt"
            AutoGenerateColumns="False"
ShowFooter="True"
            Font-Names="Verdana">

<FooterStyle ForeColor="Control"
            CssClass="cartlistfooter"
            BackColor="ActiveCaptionText"></FooterStyle>
<HeaderStyle
            CssClass="CartListHead"
            BackColor="#00AAAA"></HeaderStyle>
<AlternatingItemStyle
            CssClass="CartListItemAlt"></AlternatingItemStyle>
<ItemStyle
            CssClass="CartListItem"
            BackColor="Control"></ItemStyle>

The number of columns that we specify between the opening and closing <Columns> tags determines the number of columns that appear in the data grid control. The DataField property of BoundColumn allows us to specify the name of the column in the data view that we want to bind to.

<Columns>
<asp:BoundColumn DataField="StartingFrom" HeaderText="Starting From">
<HeaderStyle Width="300px"></HeaderStyle>
</asp:BoundColumn>
<asp:BoundColumn DataField="Destination" HeaderText="Destination">
<HeaderStyle Width="300px"></HeaderStyle>
</asp:BoundColumn>
<asp:BoundColumn DataField="DepartureDate" HeaderText="Departure Date">
<HeaderStyle Width="300px"></HeaderStyle>
</asp:BoundColumn>
<asp:BoundColumn DataField="ArrivalDate" HeaderText="Arrival Date">
<HeaderStyle Width="300px"></HeaderStyle>
</asp:BoundColumn>
<asp:BoundColumn DataField="FlightNumber" HeaderText="Flight Number">
<HeaderStyle Width="300px"></HeaderStyle>
</asp:BoundColumn>
<asp:BoundColumn DataField="Price" HeaderText="Price">
<HeaderStyle Width="300px"></HeaderStyle>
</asp:BoundColumn>
<asp:HyperLinkColumn Text="Click here to book the ticket" DataNavigateUrlField="FlightID"
DataNavigateUrlFormatString="BookTickets.aspx?FlightID={0}" HeaderText="Book Tickets">
<HeaderStyle Width="300px"></HeaderStyle>
</asp:HyperLinkColumn>
</Columns>

As you can make out from the name of the tag, asp:hyperlinkcolumn allows us to have a column in the DataGrid that can act as a hyperlink. The DataNavigateUrlFormatString attribute permits us to specify the URL to navigate to when the hyperlink is clicked. We also utilize the DataNavigateUrlField attribute to pass a parameter to the BookTickets.aspx file in the form of a query string.

The following screenshot shows the output generated by the DataGrid control:

Now that we understand the layout of the web form, let us figure out the code required to implement the functionality. As you can see in the following code, the bulk of the work is done in the Page_Load event:

private void Page_Load(object sender, System.EventArgs e)
{
// Put user code to initialize the page here
int travelClassID = Convert.ToInt32(Request.Form["lstTravelClassCode"]);
//Store the TravelClassID in the Session variable for later retrieval
Session["TravelClassID"] = travelClassID;

FlightBF flight = new FlightBF();
FlightDetailsCollection col = flight.SearchFlight(Request.Form["lstStartingFrom"],Request.Form["lstDestination"],
Convert.ToDateTime(Request.Form["txtDepartureDate"]),Convert.ToDateTime(Request.Form["txtArrivalDate"]),travelClassID);
grdSearchResults.DataSource = col;
grdSearchResults.DataBind();
}

In the code above, we start by retrieving the value of the control lstTravelClassCode contained in the submitted form collection into the local variable. We also store the travel class id in a session level variable for later retrieval. We use this stored session travel class id in the BookTickets.aspx page to book tickets for a particular flight. We then invoke the SearchFlight method of the business façade layer method and pass in all the required parameters. We then directly bind the returned collection object to the DataGrid by setting the DataSource property of the DataGrid to the returned collection object. In the Search flights results page, when the user clicks the hyperlink Click here to book the ticket , we take the user to the BookTickets.aspx , which is the central point of our next discussion.

Booking Tickets Process

In this section, we will see the steps involved in booking a ticket in our AirlineReservation application. As we already mentioned, when the user clicks the link to book the ticket in the search results page, we redirect him to the BookTickets.aspx file. We execute the following lines of code in the Page_Load event of this web form to save the booking details to the database.

private void Page_Load(object sender, System.EventArgs e)
{
//Execute this code only if the page is not posting back to itself
if (!Page.IsPostBack)
{
int flightID = Convert.ToInt32(Request.QueryString["flightid"]);
string userID = Session["UserID"].ToString();
int travelClassID = Convert.ToInt32(Session["TravelClassID"]);

FlightBF bookingService = new FlightBF();
//Invoke the web service method and store the returned booking id into the local variable
int bookingID = bookingService.AddBooking(flightID,userID,travelClassID);
//Show the booking confirmation number in the label named lblConfirmation
lblConfirmationNumber.Text = bookingID.ToString();
}
}

In the code shown above, we invoke the AddBooking method of the business façade layer method to save the booking details to the database. Finally we display the returned booking id to the user by setting the Text property of the label to the returned booking id.

The output generated by the web form BookTickets.aspx looks like this:

News and Deals Display Process

You might have already seen in the left side navigation bar that we have two links that will show us the list of all the deals available in the site and the latest news depending on which hyperlink is clicked. The execution logic to achieve the above functionality is encapsulated in the web form named InformationDisplay.aspx , which in turn depends on the user control called InformationDisplayControl.ascx to display appropriate information to the users.

Let us start our discussion by considering the user control. The user control we are going to create encapsulates all the code required for retrieving the pertinent information. It exposes a public property called informationTypeID . By setting the informationTypeID property to an appropriate value, we can vary the kind of information that gets displayed.

Clicking the link News in the left side navigation bar results in the following output being generated:

Let us explore the code behind file ( InformationDisplayControl.ascx.cs ) of the user control to understand the implementation of user control.

We start by declaring a member named informationTypeID with public scope. It is the value of this variable that determines the kind of information the user control displays. So the web form that hosts the user control needs to set this member variable to the proper value to get the desired result.

public int informationTypeID;

In the Page_Load event, we invoke the GetInformation method of the InformationBF class and pass the value of the variable informationTypeID . We then bind the repeater control to the collection object returned as the result of the method execution.

private void Page_Load(object sender, System.EventArgs e)
{

InformationBF information = new InformationBF();
//Invoke the web service's GetInformation method passing to it the type of information we want to display
InformationCollection col = information.GetInformation(informationTypeID);
informationRepeater.DataSource = col;
informationRepeater.DataBind();
}

Since the user control encapsulates all the logic for retrieving information based on the InformationTypeID value, the container web form can easily leverage the functionalities of the user control by setting the InformationTypeID property to an appropriate value.

Handling of Exceptions

As we write applications that are more and more complex and depend on a large number of external resources to perform its operations, it is in our best interests to assume that at some point in time things might go wrong. For that reason, it is good to have an efficient error handling routine in place that may handle the errors in a graceful manner. For our AirlineReservation application, we will take advantage of the excellent inbuilt exception handling mechanism that the ASP.NET runtime provides. As discussed earlier, whenever an exception occurs in any of the layers of the application, it is logged using the exception management application block and thrown back to the caller. Due to the above design, all of the exceptions will propagate up through the call stack and reach the ASP.NET layer, where we need to display appropriate message to the user depending on the severity of the exception.

To take advantage of this exception handling plumbing that is provided by ASP.NET, we need to add the following entry in the web.config file:

<customErrors defaultRedirect="ErrorPage.aspx"
mode="RemoteOnly">
</customErrors>

The defaultRedirect attribute allows us to specify the URL to direct the browser to any time an error occurs in our site. The attribute mode indicates whether custom errors are enabled, disabled, or displayed only to remote clients. It can take any one of the following values:

  • On  Indicates that custom errors are enabled
  • Off - Specifies that custom errors are disabled
  • RemoteOnly - Indicates that the custom errors are shown only to the remote clients

The above entry in the web.config file indicates that we want the remote users to be redirected to the page ErrorPage.aspx any time an error occurs in our AirlineReservation application.

Deploying the application

There are two ways by which we can deploy the above application:

  • Using the Copy Project feature in Visual Studio.NET- To accomplish this, select the AirlineReservation project from the Solution Explorer and click the Copy Project icon in the solution explorer toolbar.

Using the above dialog, we can easily deploy our AirlineReservation application in a remote server. By default, this will copy all the files required for running our application (including all the dependent assemblies) to the remote server.

It is also possible to create a setup installer for our web application. This can be accomplished by right clicking our solution in the solution explorer and selecting Add ->New Project from the context menu. In the New Project dialog box, we select Setup and Deployment Projects from the project types and Web Setup Project from the list of templates. Once we add the necessary files to the setup project, we can create a Windows installer msi file that can be executed in the remote server to install the application.

Putting it all together

Now that we have constructed the different parts of the application, let us test the application by going through the following steps:

  • First, we start the application by entering the location of the web application in the browser. At the login screen, if we enter a valid login and click Login command button, it will log us into the system. It is also possible to create a new valid login by clicking the New Member hyperlink in the left navigation bar and entering all the required details.
  • Once logged in, we can then search for flights by entering all the required parameters and then clicking the Search command button. The system will display the details of the flights that match the search criteria.
  • In the search results screen, we can book the ticket by navigating to the hyperlink Click here to book the ticket .
  • From the left navigation bar, it is also possible to bring up deals and news information by clicking either the Deals or News hyperlinks.
  • We can log out of the system at any time by clicking on the Log Out hyperlink.

Conclusion

In this article, we have seen the following:

  • How to use enterprise template projects to create application starting points by specifying an initial application structure, including any reusable or standard components and technologies, design documents, and models
  • How to leverage the features provided by the enterprise templates for creating distributed applications
  • How to create the application framework that consists of reusable components that can greatly simplify programming tasks by including pre-written code to handle procedures such as application management and exception handling techniques

Although the application we created was simple in functionality, it should provide a solid foundation for understanding how to build high performance, scalable, flexible and reliable web applications using the enterprise application templates and reusable application frameworks.

C#Today - ©2002 Wrox Press Limited


Sharpening Your C# Unit Testing and Debugging Skills

Sharpening Your C# Unit Testing and Debugging Skills
http://www.csharptoday.com/content.asp?id=2059
Article Rating: 5
Published on December 23, 2002
Abstract
In this case study, James A Brannan takes us through several techniques that can be used to unit test your code. First, we discusses NUnit, an automated testing framework used to debug and test .NET applications. Next, we take a look at black box testing techniques: specification oriented testing, equivalence partitioning, boundary value analysis, and state transition testing. Finally we explore the white box testing techniques: branch testing, conditional testing, and data definition and use testing.
Case Study

In this case study, we will look at several techniques that can be used to unit test your code. First, we discuss NUnit, an automated testing framework used to debug and test .NET applications. Next, we will take a look at black box testing techniques: specification oriented testing, equivalence partitioning, boundary value analysis, and state transition testing. Finally we will explore the white box testing techniques: branch testing, conditional testing, and data definition and use testing.

Introduction

How do you make your C# components better? Learn more C# programming techniques? Perhaps, but countless studies by software engineers show the single biggest thing you can do to make your components better is unit test them. Unit testing is the testing of an individual component. For example, when using C# after coding a class, you would test it. In this case study, I illustrate several unit testing techniques - black box and white box - you can use to unit test C# components. By the end of this case study, you should understand unit testing C# components using the tool NUnit, and how to use the following testing techniques: specification oriented testing, equivalence partitioning, boundary value analysis, state transition testing, branch testing, conditional testing, and data definition and use testing.

The Problem

The deadline. The closer it gets, the more panicked you become. The more panicked you become, the faster you code. The faster you code, the sloppier you become. The sloppier you become, the harder testing becomes. The harder testing becomes, the more time it takes. The deadline gets closer. And so you forgo unit testing to save time.

Forgoing unit testing just waylays the problem, though. For example, on a large project, when the code makes it to the testing team, now they must find the bugs in the unit. On a small project, or in-house project, the bug might make it into the release. Now, even worse then having testers find the bugs, the users must find the unit's bugs - bugs that might be hidden deep inside an application.

When the tester or user finds a bug, the developer responsible for the unit must usually fix it. So you must now fix the bug you did not fix during development. The bug might now take you days to fix; however, had you found and fixed it when you were coding, the bug might have taken hours or minutes. Forgoing unit testing did not save time.

The Solution

You can save time by unit testing and coding concurrently and purposefully. Unit testing and coding concurrently helps you debug components quicker and smoother. Unit testing purposefully, no matter how informal, is more effective then unit testing haphazardly.

Why unit test concurrently? Unit testing concurrently with development helps prevent you from forgoing unit testing. This is converse to conventional software engineering thinking, which maintains that you should complete the entire unit (for example an assembly) and then unit test the component. Unit testing and developing concurrently helps you find and fix errors immediately, before another interesting feature captures your interest

How do you unit test purposefully? Write the unit test's specifications before writing the unit test. If you are fortunate enough to have a requirements document, or Use Cases, use them to help guide your testing. But, do not aimlessly step through your code in a debugger and consider the first time it runs without exceptions as code complete.

Incrementally debug, unit test, and code. Begin by planning how you intend to implement a component. Then incrementally develop the component, feature by feature. During each increment, first code the feature. Second, plan how you intend to unit test the feature by writing down a few unit tests. Third, write the unit tests and use them to debug and unit test the feature. Repeat these three steps for each feature in the component. Finally, when the component is complete, consider any areas you might have overlooked testing and be sure to unit test them. The following figure illustrates this process.

Note that what I am calling concurrent coding and unit testing, I directly derived from Extreme Programming's (XP) test-driven development philosophy. When practicing XP's test-driven development, testing literally drives development. You can distill test-driven development to three main tenets. First, you code a feature's test(s) before coding the feature. You write the unit test, code the feature, and then run the unit tests on it. When the feature passes the test, the feature is completed. You code no more and no less then that required in passing the test(s). Second, you maintain a testing suite, testing every feature of every component in an application. Ideally, you should automate the test suite. Third, you release no code - even minor enhancements or bug fixes - unless it has associated unit tests that pass when run. Like test-driven development, when concurrently coding and unit testing, you incrementally unit test and code you applications, feature by feature.

Unlike test-driven development, in this case study I write unit tests after writing a feature and not before as XP dictates. If you prefer writing a feature's test before writing the feature, note that the techniques shown here are still valid. Whether you write the test before or after writing a feature does not seem important to me. However, you should note that Extreme Programming enthusiasts emphatically maintain that you write a feature's test before coding the feature. Regardless of whether you write your test immediately before coding a feature or immediately after coding a feature, what seems important to me is that you unit test it thoroughly before coding another feature or enhancing the current feature.

System Requirements

To follow along, you need NUnit 2.0, available at http://www.nunit.org/?WROXEMPTOKEN=981098ZFI2gYTm0YyOIQ6Szs2X. Be sure that you download version 2.0, because it is a total rewrite from version 1.0. You need C# knowledge; I use C# examples that contain parameters, attributes, events, and unsafe code. In this case study, I assume you are already familiar with these C# concepts. You also need some basic familiarity with C pointers and arrays. However, even if you do not fully understand all the C# concepts presented in the code samples, the unit testing concepts should still be clear.

Automated Unit Testing using NUnit

Strict adherence to a regime of coding and unit testing concurrently will do wonders in improving your code. However, regression testing is still problematic. Regression testing tests that new features and modifications to existing features do not break code that passed earlier tests. When practicing the incremental development method in this case study, regression testing could quickly become unwieldy. Every time you incrementally added a new feature, you would have to unit test it and rerun the unit tests for all earlier coded features. What you need is an automated way to test your code.

Driver

One way you can automate testing your code is by using a driver. A driver is code designed to execute a component or function that ordinarily would not be executable on its own. You cannot test an assembly that lacks a main method, for instance.

Consider the following example. Typically, after coding the Tested class you would create a driver to run it.

using System;
namespace ToTest{
class
            Tested
{
public void
            Run()
{
Console.WriteLine("Running.....");
}
}
}

using
            System;
using ToTest;

class MainClass
{
public
            static void Main(string[] args)
{
Tested objTest = new
            Tested();
objTest.Run();
}
}

You would compile both preceding classes and then open them in a debugger (Microsoft's CLR Debugger for example). Next, you would set the executable as the program to debug, set breakpoints, and then click Start on the Debug menu.

You might even add assertions to your driver using System.Diagnostics.Debug.Assert (covered in more detail later in this case study). For example, to the preceding code you might add a read only Name property to Tested and a debug statement to ensure it worked as expected.

private String strName = "tester";
public
            String Name{
get{return this.strName;}
}

System.Diagnostics.Debug.Assert(objTest.Name.Equals("incorrect"),
"Tested class name did not equal ''incorrect.''");

After defining the DEBUG symbol, you would then compile MainClass.

csc /out:test.exe /d:DEBUG /t:exe /r:totest.dll MainClass.cs

Because the DEBUG symbol enables debugging, when MainClass reaches a line containing an assertion, it throws an exception, and displays the Assertion Failed dialog box.

You can sufficiently unit test your code using a simple driver with assertions. If you prefer to write your own drivers, go ahead. The black box and white box unit testing techniques in this case study remain valid. Skip the following section on NUnit and proceed directly to the black box testing section. But before you do, you might want to at least briefly glance at the section on NUnit. A testing framework like NUnit really does make testing your .NET code easier and more robust.

NUnit

NUnit is an open-source testing framework for unit testing .NET applications that makes automated testing easier. Think of NUnit as a framework for writing drivers and tests for C# assemblies.

Test Fixtures and Test

The fundamental two NUnit constructs are the test fixture and the test. A test fixture is a class that contains tests. The only restriction on a test fixture is that it does not override its default constructor. You mark a class as a test fixture via the TestFixture attribute.

[TestFixture]
public class ExampleOneTest{

A test is a method inside a test fixture that tests something. You mark a method as a test via the Test attribute.

[Test]
public void TestGetGreeting()

Consider the following class and associated NUnit unit test.

namespace Examples {
using
            System;

public class ExampleOne {
public String
            GetGreeting(){
return "Hello
            Turkey.";
}
}
}

namespace Examples {
using
            NUnit.Framework;

[TestFixture]
public class
            ExampleOneTest{

[Test]
public void
            TestGetGreeting(){
ExampleOne objExampleOne = new
            ExampleOne();
Assertion.AssertEquals("Hello Turkey.",
            objExampleOne.GetGreeting());
}
}
}

ExampleOne contains one method, GetGreeting. ExampleOneTest is a test fixture that contains one test, TestGetGreeting. TestGetGreeting tests ExampleOne by creating a new ExampleOne instance, calling the GetGreeting method, and ensuring that it equals Hello Turkey.

NUnit GUI Program

NUnit comes with a command-line program and an easy-to-use GUI. To better understand the GUI, let's run ExampleOneTest in the NUnit GUI.

Copy and paste, or re-write, ExampleOne and ExampleOnTest from the preceding code and compile both classes. Be sure to include the NUnit framework assembly (nunit.framework.dll) as a reference. Compile the two classes into one DLL using the SDK C# command-line compiler or your IDE's equivalent.

csc /out:exampleone.dll /t:library /r:nunit.framework.dll ExampleOne.cs ExampleOneTest.cs

After compiling, start the NUnit GUI program. On the File menu, click Open, and then open exampleone.dll. NUnit loads the assembly and creates a hierarchical tree from the fully qualified file name, namespace, test fixtures, and tests.

NUnit places the assembly's file name at the top of the hierarchy, followed by the Examples namespace, the ExampleOneTest test fixture, and the TestGetGreeting test. Select the Examples node in the Tree View, click Run, and NUnit runs all tests in all the fixtures in the Examples namespace, albeit there is only one fixture and one test in this example.

You can include as many tests as desired in a test fixture and as many fixtures as desired in a namespace. If compiled in the same assembly, NUnit creates a suite of tests automatically by examining the assembly's namespace and identifying all test fixtures and tests in that assembly. For example, add the following new class and associated test fixture to your exampleone.dll assembly.

namespace Examples {
using
            System;

public class ExampleTwo {
private String
            strUserName;

public String GetGreeting(){
return "Hello
            Turkey.";
}

public String UserName{
get{return
            this.strUserName;}
set{this.strUserName =
            value;}
}
}
}

namespace Examples {
using
            System;
using NUnit.Framework;

[TestFixture]
public
            class ExampleTwoTest{

ExampleTwo objExampleTwo = new
            ExampleTwo();

[Test]
public void
            TestGetGreeting()
{
Assertion.AssertEquals("Hello Turkey.",
            objExampleTwo.GetGreeting());
}

[Test]
public void
            TestUserName(){
String strName =
            "James";
objExampleTwo.UserName =
            "James";
Assertion.AssertEquals(strName,
            objExampleTwo.UserName);
}
}
}

After recompiling, reload exampleone.dll in NUnit. The ExampleTwoTest test fixture and its two tests, TestGetGreeting and TestUserName, appear in the Tree View. Note that after selecting any node in the hierarchy, clicking Run tests child nodes to the node selected, other nodes do not run.

When NUnit tests fail, the progress bar changes from green (signifying tests passed) to red (signifying one or more tests failed). For example, in the following code, EchoName throws a DivideByZero exception. The ExpectedException attribute indicates that the EchoName test expects an ArgumentException. When you run the tests, the EchoName test fails, and the progress bar turns red, as does the Examples node and the EchoName node.

namespace Examples {
using
            System;
using NUnit.Framework;

[TestFixture]
public
            class ExampleThreeTest{

ExampleThree objExampleThree = new
            ExampleThree();

[Test]
[ExpectedException(typeof(System.ArgumentException))]
public
            void EchoName(){
int test = 0;
int intWillFail =
            4/test;
}

}
}

When a fixture's test fails, both the fixture's node and the failing test's node turn red. Multiple fixtures and multiple tests might fail - in this situation, all failing fixtures and tests turn red.

Attributes

NUnit uses .NET custom attributes to identify NUnit specific functionality. An attribute allows you to define metadata that the CLR retrieves at runtime using reflection. NUnit defines the TestFixture, Test, SetUp, TearDown, ExpectedException, and Ignore attributes. The TestFixture attribute identifies classes that contain NUnit tests. The Test attribute identifies NUnit test methods. SetUp marks the first method run before a test fixture runs any tests, while TearDown marks a method run after a test fixture finishes running tests. Note that both SetUp and TearDown are optional. The Ignore attribute allows you to skip a test when running NUnit tests. Although you could simply remove the Test attribute, Ignore helps you remember that you are not running the test. For example, when running the tests in the NUnit GUI, removing the Test attribute makes the test nonexistent as far as NUnit is concerned. However, when marking the test with Ignore, NUnit changes the ignored node to yellow, making it explicit that you are temporarily skipping the test.

The following code illustrates NUnit's attributes:

namespace Examples {
using
            System;

public class ExampleThree {

public String
            EchoName(String Name){

return
Name;
}
}
}

namespace Examples {
using System;
using NUnit.Framework;

// ---------------------------------------------------------------------
// TestFixture attribute: identifies an NUnit class that contains one or
// more tests.
// ---------------------------------------------------------------------

[TestFixture]
public class ExampleFourTest{
ExampleThree objExampleThree;

// ---------------------------------------------------------------------
// SetUp attribute: identifies the method run when constructing a test
// fixture. This method is run before any tests are run.
//----------------------------------------------------------------------

[SetUp]
public void Init(){
objExampleThree = new ExampleThree();
}

// ---------------------------------------------------------------------
// Test attribute: identifies NUnit test methods.
// ---------------------------------------------------------------------

[Test]
[ExpectedException(typeof(System.DivideByZeroException))]
public void EchoName(){
int test = 0;
int intWillFail = 4/test;
}

// ---------------------------------------------------------------------
// Ignore attribute: notifies NUnit to skip the test.
// ExpectedException attribute: notifies NUnit that the test should
// throw an exception of the specified type. If the test does not
// throw an exception of specified type, it is a failure
// ---------------------------------------------------------------------

[Test]
[Ignore("Don't feel like testing")]
[ExpectedException(typeof(System.ArgumentException))]
public void EchoNameToo(){
int test = 0;
int intWillFail = 4/test;
}

// ---------------------------------------------------------------------
// TearDown attribute: identifies the method run when completing running
// a test fixture. This method is run after all tests are run.
//----------------------------------------------------------------------

[TearDown]
public void Dispose(){
objExampleThree.EchoName("John");
objExampleThree = null;
}
}
}

NUnit's Assertion Class

Inside a test method, you test some functionality using NUnit's Assertion class. You expect a result from exercising some functionality in the feature under test. For example, when NUnit got to the test MyTest, it would then execute the assertion to ensure it was true. If not, NUnit would record the failure.

[Test]
public void MyTest(){
Assertion.Assert("1 does not equal 2.", 1==2);
}

NUnit's Assertion class contains the static methods you use to test expected results. For full coverage of the Assertion class member functions, refer to NUnit's documentation. However, we use several Assertion functions in this case study, and so I will cover them briefly here.

Assert

Assert asserts something is true. For example, the following code fragment asserts that MyFunction returns true.

Assertion.Assert(objMyObject.MyFunction(double dblMathQuestionAnswer))

You can also provide a failure message to Assert that is echoed to the console when the assertion fails.

Assertion.Assert("The test failed.", objMyObject.MyFunction.IsCorrect(double dblMathQuestionAnswer))

AssertEquals

AssertEquals asserts that two values are equal. The following code fragment tests two boolean values, the expected result (parameter one) and the actual result (parameter two).

Assertion.AssertEquals(false, objMyObject.MyFunction.IsCorrect(double
dblMathQuestionAnswer))

You can overload AssertEquals to accept integers, doubles, strings, floats, and objects. You can also provide a failure message to AssertEquals.

Assertion.AssertEquals("The test failed.", false,
objMyObject.MyFunction.IsCorrect(double dblMathQuestionAnswer))

AssertNotNull and AssertNull

AssertNull and AssertNotNull test whether an object is or is not null.

MyObject objMyObject =
            null;
Assertion.AssertNull(objMyObject);
objMyObject = new
            MyObject();
Assertion.AssertNotNull(objMyObject);

As with Assert and AssertEquals, you can also provide a failure message to these functions.

MyObject objMyObject =
            null;
Assertion.AssertNull("Its not null!",
            objMyObject);
objMyObject = new
            MyObject();
Assertion.AssertNotNull("Its null!",
            objMyObject);

AssertSame

AssertSame tests if two object references refer to the same underlying object.

MyObject objMyFirstObject = new MyObject();
MyObject objMySecondObject = objMyFirstObject;
Assertion.AssertSame("Not the same!", objMyFirstObject, objMySecondObject);

NUnit Summary

You now have all the information you need to start using NUnit to unit test your code. Notice that you can incrementally add new test methods to a test fixture without destroying the old tests. You automatically regression test every time you test a new feature. Consider what NUnit does not do, though. NUnit does not provide testing techniques to test your code. Unit testing, though in many respects intuitive, has many different available techniques. These techniques are as important to coding better, if not more, than understanding how to use NUnit. NUnit after all, is only a tool.

Black Box Testing Techniques

Black box testing test expected outputs from known inputs and treats a component's internals as a black box. Black box testing verifies that software meets its specification. However, black box testing does not necessarily validate that a unit functions correctly. For validating the code's functionality, you need white box testing as well.

Strictly speaking, when unit testing concurrently with developing, you are not exclusively black box or white box testing, but, a little of both. For this reason, some testing resources call this testing gray box testing. For example, in the upcoming examples, we know the unit's internals; we are writing the internals as we test, after all. Thus, the dichotomy between black box and white box testing is somewhat false in the upcoming examples.

When unit testing concurrently with development, or following XP's Test Driven development, you employ black box techniques and white box techniques simultaneously. However, for presentation clarity, in this case study I separate the two techniques. This separation is logical, though. When black box testing, you examine inputs and outputs to a component, not the component's internals. Of course, you have an idea, based on the internals, of what the outputs should be, but you are not examining the internals.

Many proven techniques for black box testing exist; in this case study I illustrate four of them:

  • specification oriented testing,
  • equivalence partitioning,
  • boundary value analysis,
  • state transition testing.

Specification oriented testing is the testing you are probably most familiar with; it tests that inputs to a unit yields the expected outputs. For example, if a function is supposed to square a number, then if you give the function the number two as input, the function should return four as output. Equivalence partitioning is a technique used to ensure adequate testing coverage by considering a range of values equal for testing purposes - the goal is to test one or more values from each range of values. Boundary value analysis tests values at a partition's boundaries. Errors are more likely to occur at a boundary, and so you pay special attention to boundary values. Finally, state transition testing derives tests from a component's change in state; the test provides a known event and then tests that the object enters the expected state. Let us consider each testing technique in more detail.

Specification Oriented Testing

Specification oriented testing shows that a feature implements its specification. For example, suppose your boss told you the following.

  • You need to develop a component that returns the greeting "Never pay taxes again <person's name>" or "We will help lower your taxes <person's name>". The component is equally likely to return either greeting.

The following code implements this informal specification:

namespace Greetings {
using
            System;

public class Greeting {

private Random
            objRandom = new Random();

public String GetGreeting(String
            Name){
double dblRandValue =
            this.objRandom.NextDouble();
String strToReturn;
if
            (dblRandValue < .5){
strToReturn = "Never pay taxes again " +
            Name;
}
else
strToReturn = "We will help lower your taxes
            " + Name;

return strToReturn;
}
}
}

The following test fixture tests the preceding component:

namespace Greetings {
using
            System;
using NUnit.Framework;

[TestFixture]
public
            class GreetingTest{

Greeting objGreeting = new
            Greeting();

// Specification oriented test example
            //

[Test]
public void GetGreetingTest()
{
String
            strTheGreeting
            =
objGreeting.GetGreeting("Jerry");

String[]
            objFullGreetings = {"Never pay taxes again Jerry",
"We will help
            lower your taxes
            Jerry"};

Assertion.Assert(strTheGreeting.Equals(objFullGreetings[0])
||
            strTheGreeting.Equals(objFullGreetings[1]));
}
}
}

GetGreetingTest tests that when you provide Jerry as input, it receives either Never pay taxes again Jerry or We will help lower your taxes Jerry. GetGreetingTest tests that the feature (GetGreeting) implements its specification.

As an aside, before continuing, take a moment to reflect on what we did not test. The test ensures the component returns one of two messages - it does not test that GetGreeting is equally likely to return either message. Because dblRandValue is a local variable, we have no easy way to see its value in GetGreetingTest. To test either message's probability, we need to use white box testing techniques, which I cover later.

Equivalence Partitioning

Specification oriented testing ensures a feature fulfills its specification - expected inputs yield expected outputs. However, note that these tests usually only test the simplest cases; sometimes you need more then a single test to adequately test a feature. Functions often accept a range of values as parameters. To adequately test a function like this, you need to test with several values. However, testing all possible values is impractical, if not impossible. Consider the following example:

namespace Greetings {
using
            System;

public class Greeting {

public enum Theme
            {GetMoreBack,PayNothing,PayLess};

// [ GetGreeting excluded
            for brevity ] //

public Theme GetTheme(double
            dblTaxesPaid){
Theme enumTheme = new Theme();

// grouping
            one: -infinity to zero (inclusive) //

if (dblTaxesPaid <=
            0)
enumTheme = Theme.GetMoreBack;

// grouping two:
            (exclusive) zero to 1000 (inclusive) //

if (dblTaxesPaid >
            0 && dblTaxesPaid <= 1000)
enumTheme =
            Theme.PayNothing;

// grouping three: (exclusive) 1000 to +
            infinity //

if (dblTaxesPaid > 1000)
enumTheme =
            Theme.PayLess;

return enumTheme;
}
}
}

Depending upon the tax paid, GetTheme returns GetMoreBack, PayNothing, or PayLess. The range of values GetTheme accepts for dblTaxesPaid ranges from negative infinity to positive infinity. This makes testing all values impossible. Although testing all possible values is impossible, you can test a robust subset of those values using equivalence partitioning.

Equivalence partitioning groups data with similar values in groups, considering all values in a grouping equivalent for testing purposes. For example, in the preceding GetTheme method, there are three natural groupings; values less then or equal to zero, values greater then zero and less then or equal to 1000, and values greater then 1000. We can derive the following tests from the partitions:

  • Test one: enter -20 as input, expected to receive Theme.GetMoreBack.
  • Test two: enter 100 as input, expected to receive Theme.PayNothing.
  • Test three: enter 5000 as input, expected to receive Theme.PayLess.

We write the NUnit tests, adding the GetThemeTestGetMoreBack, GetThemeTestPayNothing, and the GetThemeTestPayLess test methods to GreetingTest.

namespace Greetings {
using
            System;
using NUnit.Framework;

[TestFixture]
public
            class GreetingTest{

Greeting objGreeting = new
            Greeting();

// Test one: enter -20 as input, expected to
            receive
// Theme.GetMoreBack //

[Test]
public void
            GetThemeTestGetMoreBack(){
Assertion.AssertEquals(Greeting.Theme.GetMoreBack,
objGreeting.GetTheme(-20));
}

//
            Test two: enter 100 as input, expected to receive Theme.PayNothing
            //

[Test]
public void
            GetThemeTestPayNothing(){
Assertion.AssertEquals(Greeting.Theme.PayNothing,
objGreeting.GetTheme(100));
}

//
            Test three: enter 5000 as input, expected to receive Theme.PayLess
            //

[Test]
public void
            GetThemeTestPayLess(){
Assertion.AssertEquals(Greeting.Theme.PayLess,
objGreeting.GetTheme(5000));
}

//
            [Specification oriented test example omitted for brevity ]
            //

}
}

GetThemTestGetMoreBack tests that GetTheme returns Theme.GetMoreBack for values less then zero, GetThemeTestPayNothing tests that GetTheme returns Theme.PayNothing for values from zero to 5000, and GetThemeTestPayLess tests that GetTheme returns Theme.PayLess for values greater then 5000.

Boundary Value Analysis

Equivalence partitioning helps ensure adequate test coverage by testing one or more values in each partition. However, errors often occur at a partition's boundary. Boundary value analysis analyzes a partition's boundary values for those errors.

For example, the preceding test cases randomly picked test values to test in each partition. The tests passed; however, these tests overlooked values at the boundary. Suppose we modified the original specification so that values up to 1000, but not including 1000, return Theme.PayNothing, while values over 999 return Theme.PayLess. Although the previous tests still pass, GetTheme has two errors. The first error, the second condition should be all values from zero to 999 not zero to 1000.

if (dblTaxesPaid > 0 && dblTaxesPaid <= 1000) enumTheme = Theme.PayNothing;

The second error, the third condition mistakenly excludes 1000.

if (dblTaxesPaid > 1000) enumTheme = Theme.PayLess;

Tests one, two, and three miss these two errors - these three tests do not use the value 1000 (the boundary value). By testing each partition's boundaries (in this example, the values 0 and 1000) you can catch these errors.

Boundary value analysis involves three steps. First, determine the boundary values. Second, write the test specifications based upon those values.

  • Test four: enter zero as input, expected to receive Theme.GetMoreBack.
  • Test five: enter 1000 as input, expected to receive Theme.PayLess.

Third, write the NUnit tests that test the boundary values:

namespace Greetings {
using
            System;
using NUnit.Framework;

[TestFixture]
public
            class GreetingTest{

Greeting objGreeting = new
            Greeting();

// boundary value analysis tests example
            //

[Test]
public void
            GetThemeTestGetMoreBackBoundary(){
Assertion.AssertEquals(Greeting.Theme.GetMoreBack,
objGreeting.GetTheme(0));
}

[Test]
public
            void
            GetThemeTestPayLessBoundary(){
Assertion.AssertEquals(Greeting.Theme.PayLess,
objGreeting.GetTheme(1000));
}

.
            . .

}
}

The GetThemeTestPayLessBoundary test fails. But after fixing the second condition and third condition in GetTheme, as in the following code, the test passes.

    public Theme
            GetTheme(double dblTaxesPaid){
Theme enumTheme = new
            Theme();

// grouping one: -infinity to zero (inclusive)
            //

if (dblTaxesPaid <= 0)
enumTheme =
            Theme.GetMoreBack;

// grouping two: (exclusive) zero to 1000
            (exclusive) //

if (dblTaxesPaid > 0 &&
            dblTaxesPaid < 1000)
enumTheme = Theme.PayNothing;

//
            grouping three: (exclusive) 1000 to + infinity //

if
            (dblTaxesPaid >= 1000)
enumTheme =
            Theme.PayLess;

return enumTheme;
}

Boundary value analysis does not always have to test explicit partitions as in the last example. Data partitions can occur in less obvious places. For example, in the following method, there are two partitions - dates greater then the current date and dates less then or equal to the current date. Suppose we added the following function to the Greeting class.

    public bool
            ValidDate(DateTime objDate){

// note: this code is
            incorrect
bool blnValid = false;
if
            (!(DateTime.Compare(objDate, System.DateTime.Now) >
            -1))
blnValid = true;

return blnValid;
}

ValidDate determines if a date is valid. If the date is before the system date, the function returns true. Conversely, if the date is on or after the system date, the function returns false.

Please note a subtle potential bug in the code: if the DateTime passed to ValidDate is initialized using a date with no time, and that date is the same date as System.DateTime, then the method returns false. Although technically correct, this is not the intended behavior - we want two dates with the same date to be equal. This false value occurs because System.DateTime.Now initializes to the current time on the current date while the DateTime passed to the function initializes to midnight.

Using specification oriented testing or equivalence partitioning would probably miss this bug, because we would test one value less than System.DateTime.Now and one value greater then System.DateTime.Now. The following test passes, for example.

[Test]
public void ValidDateTest(){
Assertion.Assert(objGreeting.ValidDate(new DateTime(1965,10,02)));
}

However, if we consider the boundary, where both dates are equal, the following test fails.

[Test]
public void ValidDateTest (){

// should fail so negate to true to pass
// Note: change to current system date if following along at home.

Assertion.Assert(!(objGreeting.ValidDate(new DateTime(2002,10,26))));
}

Now suppose that after running the preceding test, we caught our mistake and changed ValidDate to the following:

    public bool
            ValidDate(DateTime objDate){
bool blnValid = false;
if
            (!(DateTime.Compare(objDate.Date, System.DateTime.Now.Date) >
            -1))
blnValid = true;

return blnValid;
}

ValidDate is now comparing the date parts of two DateTime classes and not the date and time.

State Transition Testing

Sometimes you can modeling a feature's behavior using a state transition diagram. A state transition diagram models a finite state machine. A finite state machine is an idealized model of a system where the system can usually only be in one state at a time. For example, a light can be on or it can be off, but not both. Suppose we needed to add a new feature to Greeting that allows a user to win when the class receives an external event signifying Greeting should allow winners. The following diagram models the necessary behavior.

One way we can implement the preceding requirement is by using an event. The following code adds an event handler to the Greeting class:

namespace Greetings {
using
            System;

public class Greeting {

private bool blnWinner
            = false;

// property indicating if user is a winner
            //

public bool Winner{
get{return
            this.blnWinner;}
}

// method that captures a let win event
            //

public void LetWinEventNotifier(Object sender, bool
            blnWin){
this.blnWinner =
            blnWin;
System.Console.WriteLine("called");
}

. .
            .

}
}

To test the event handler, we add a delegate, an event handler, and a method to register the event handler and test the handler in the Greeting class:

namespace Greetings {
using
            System;
using NUnit.Framework;

public delegate void
            LetWinEventHandler (Object source, bool
            blnWin);

[TestFixture]
public class
            GreetingTest{

Greeting objGreeting = new Greeting();

.
            . .

public event LetWinEventHandler
            LetWinEvent;

[Test]
public void
            LetWinHandledTest(){
this.LetWinEvent +=
            new
LetWinEventHandler(objGreeting.LetWinEventNotifier);
Assertion.Assert(!objGreeting.Winner);
LetWinEvent(this,true);
Assertion.Assert(objGreeting.Winner);
LetWinEvent(this,false);
Assertion.Assert(!objGreeting.Winner);
}
}
}

After declaring a delegate with the same method signature as LetWinEventNotifier in the Greeting class, the GreetingTest fixture declares a LetWinEvent event. The LetWinHandledTest registers the Greeting class's LetWinEventNotifier and then tests that it receives the events and reacts correctly.

White Box Testing Techniques

White box testing tests a unit's internals, focusing on conditional statements, looping, and other structural concerns. While black box testing ignores a unit's internals, white box testing uses this knowledge of a unit's internals to further test a component. When black box testing, you were testing that inputs yielded expected outputs, now, you are testing that the component's internals function correctly.

Assertions and Debug Statements

By using assertions and debugging statements, you can add code that is executed when you compile your application in debug mode, but not when you compile it for release. When writing .NET applications, the Debug class implements both these concepts. Although there are many different ways you can use the Debug class, in this case study we use it to write debugging to a StringWriter. We then test the StringWriter for certain sub-strings in our NUnit test fixture. This use should become clearer after the examples in the next two sections.

Assertions

C# supports assertions for debugging your code through the Assert method in the System.Diagnostics.Debug class. The Assert method tests conditions that should be true. If an assertion is not true then it opens an Assertion Failed dialog box.

When using assertions, the compiler only enables assertions when you compile the code in debug mode. When compiled in release mode, the compiler disables assertion statements. You must compile the code with the \d:debug switch.

Having an Assertion Failed dialog box open when running automated unit tests is unacceptable (someone has to click Ok, Abort, or Cancel). We need to obtain the information another way. We can change this behavior by replacing the Debug class's listeners with our own. For more on the Listener class, refer to the .NET SDK documentation. Consider the following example:

using System;
using ToTest;
using
            System.Diagnostics;
using System.IO;

public class
            MainClass
{
public static void Main(string[] args)
{
//
            create a stringwriter and trace listener to write debug messages
            to

StringWriter objWriter = new
            StringWriter();
TextWriterTraceListener myTextListener =
            new
TextWriterTraceListener(objWriter);
Debug.Listeners.Clear();
Debug.Listeners.Add(myTextListener);
Tested
            objTest = new
            Tested();

Debug.Assert(objTest.Name.Equals("incorrect"),
"Assertion failed:Main.cs:line
            17");

Console.WriteLine(objWriter.ToString());
objWriter.Close();
myTextListener.Close();
}
}

The preceding code replaces the Debug class's DefaultTraceListener with a TextWriterTraceListener that writes to a StringWriter, which then echoes the string to the console.

Fail:Assertion failed:Main.cs:line 17

Instead of echoing to a string, though, we can return values to a NUnit test, allowing the tests to see inside the component. Consider the following component and associated NUnit test fixture.

namespace AssertsAndDebug{
using
            System;

public class TestedClass {
public bool
            ReturnFalse(){
System.Diagnostics.Debug.Assert(false,
"Assertion
            Failed:TestClass.cs:line 7");
return
            false;
}
}
}

namespace AssertsAndDebug {
using
            System;
using NUnit.Framework;
using
            System.Diagnostics;
using
            System.IO;

[TestFixture]
public class TestedClassTest
            {

private TestedClass objTestedClass = new
            TestedClass();
private StringWriter objWriter = new
            StringWriter();
private TextWriterTraceListener
            objTextListener;

[SetUp]
public void
            Init(){
objTextListener = new
            TextWriterTraceListener(objWriter);
Debug.Listeners.Clear();
Debug.Listeners.Add(objTextListener);
}

private
            void ReInitializeWriters(){
objWriter.Close();
objWriter = new
            StringWriter();
objTextListener.Writer =
            objWriter;
}

[Test]
public void
            TestReturn(){
this.ReInitializeWriters();
objTestedClass.ReturnFalse();

if
            (objWriter.ToString().Trim().Length >
            0)
Assertion.Fail(objWriter.ToString().Trim());
}

[Test]
public
            void
            TestReturnTwo(){
this.ReInitializeWriters();
objTestedClass.ReturnFalse();
if
            (objWriter.ToString().Trim().Length >
            0)
Assertion.Fail(objWriter.ToString().Trim());
}

[TearDown]
public
            void
            Dispose(){
objWriter.Close();
objTextListener.Close();
}
}
}

When we load the assembly in the NUnit GUI, the tests fail. The reason they fail is that an assertion fails in the AssertsAndDebug class. The StringWriter returns the string from the StringWriter and Debug.Assert in TestedClass fails.

Now before you object, yes this is a "low-rent" way to test a unit's internals. Writing debugging output to a string and then doing substring comparisons to test for certain values is not a high-tech solution. But it works and you can implement it easily.

Certainly there are other ways to obtain the same results as assertions and debugging statements; if you have a better way to do the same thing, use that technique. Better yet, post it to this article's discussion board. Use whatever technique you prefer; however, what is important is that you internally record what is happening inside your component and then test what happened in your NUnit tests.

Branch Testing

When branch testing, you examine the paths through a component. For example, consider the following component. The paths are marked A, B, C, and D.

namespace Greetings {
using
            System;
using System.Diagnostics;

public class Greeting
            {

private bool blnIsEighteen = false;
private static int
            WINNUMBER = 7;

public double GetCashOut(int winningNumber,
            bool tax, bool is18){
this.blnIsEighteen = is18;
double
            winnings = 0;
if (this.blnIsEighteen){
if
            (IsWinner(winningNumber)){
A. if (tax) winnings =
            ReturnCashOutTaxed();
B. else winnings =
            ReturnCashOutNotTaxed();
}
C. else winnings =
            ReturnCashDeducted();
}
D. else throw new Exception("Must be
            18.:line 23:Greeting.cs");

return
            winnings;
}

private bool IsWinner(int
            winningNumber){
if (winningNumber == WINNUMBER)
            {

return true;
}
return false;
}

A.
            private double
            ReturnCashOutTaxed(){
Debug.Assert(this.blnIsEighteen, "Not
            eighteen:line
            41:Greeting.cs");
Debug.WriteLine("in_ReturnCashOutTaxed");
return
            DetermineWinning() * .3;
}

B. private double
            ReturnCashOutNotTaxed(){
Debug.Assert(this.blnIsEighteen, "Not
            eighteen:line
            47:Greeting.cs");
Debug.WriteLine("in_ReturnCashOutNotTaxed");
return
            DetermineWinning();
}

C. private double
            ReturnCashDeducted(){
Debug.Assert(this.blnIsEighteen, "Not
            eighteen: line
            53:Greeting.cs");
Debug.WriteLine("in_ReturnCashDeducted");
return
            -10;
}

private double DetermineWinning(){
Random
            objRandom = new Random();
double dblRandom =
            objRandom.NextDouble();
Debug.WriteLine("<random>" +
            dblRandom + "</random>" );
return dblRandom *
            10000;
}
}
}

When the user has the winning number and must pay taxes, Greeting follows path A. When the user has the winning number and is not required to pay taxes, Greeting follows path B. When the user does not have the winning number, Greeting follows path C. Finally, if the user is under eighteen, Greeting follows path D. We can glean the following four tests from these paths:

  • Test one: Call GetCashOut(7,true,true), expected to not receive any output from failed assertions. Expect to call ReturnCashOutTaxed method in Greeting. Expect the random number multiplied by 10000 to equal the returned number from GetCashOut.
  • Test two: Call GetCashOut(7,false,true), expected to not receive any output from failed assertions. Expect to call ReturnCashOutNotTaxed method in Greeting. Expect the random number multiplied by 10000 to equal the returned number from GetCashOut.
  • Test three: Call GetCashOut(6,false,true), expected to not receive any output from failed assertions. Note that 6 is just an arbitrarily chosen number, 7 is the winning number. Expect to call ReturnCashDeducted method in Greeting. Expect number returned from GetCashOut to equal -10.
  • Test four: Call GetCashOut(6,false,false), expected to receive an exception.

Greeting only writes "line" to the listener when an assertion evaluates to false. So in the tests, if a test finds "line", we know an assertion evaluated to false, and the test fails.

Assertion.Assert(strDebugOut, strDebugOut.IndexOf("line") < 0);

Notice that we cannot test the returned value by using black box testing techniques. Testing the returned value requires knowing the internal random value. Instead, in the preceding code, we remedy this problem by using Debug.WriteLine.

Debug.WriteLine("<random>" + dblRandom + "</random>" );

Then, when writing the test, you can write code similar to the following to obtain an internal value:

String strWinning =
            strDebugOut.Substring(
strDebugOut.IndexOf("<random>")+8,
strDebugOut.IndexOf("</random>") -
(strDebugOut.IndexOf("<random>")+8) );

Finally, note that you can also test the branch taken in a component by using Debug.WriteLine.

Debug.WriteLine("in_ReturnCashDeducted");

Then, assert that the string is in the debug output in your tests.

Assertion.Assert("Path wrong?",
strDebugOut.IndexOf("in_ReturnedCashOutTaxed") > 0);

The following code illustrates these techniques by testing the Greeting class:

namespace Greetings {
using
            System;
using NUnit.Framework;
using System.IO;
using
            System.Diagnostics;

[TestFixture]
public class
            classGreetingTest{
Greeting objGreeting = new
            Greeting();

//
            ----------------------------------------------------- //
// Test
            Path A - Test one //
//
            -----------------------------------------------------
            //

[Test]
public void
            GetCashOutTestPathA(){

StringWriter objWriter = new
            StringWriter();
TextWriterTraceListener objListener =
            new
TextWriterTraceListener(objWriter);

Debug.Listeners.Clear();
Debug.Listeners.Add(objListener);

Assertion.Assert(objGreeting.GetCashOut(7,true,true)
            > 0);

String strDebugOut = objWriter.ToString();

//
            Test that no Debug.Assert statements were false
            //

Assertion.Assert(strDebugOut, strDebugOut.IndexOf("line")
            < 0);
// Test path A taken //

Assertion.Assert("Path
            wrong?",
strDebugOut.IndexOf("in_ReturnedCashOutTaxed") >
            0);

objWriter.Close();
objListener.Close();
}

[Test]
public
            void GetCashOutTestTheAmountPathA(){

StringWriter objWriter =
            new StringWriter();
TextWriterTraceListener objListener =
            new
TextWriterTraceListener(objWriter);

Debug.Listeners.Clear();
Debug.Listeners.Add(objListener);

//
            Get winnings - user is winner, pays tax, and is over 18
            //

double amount =
            objGreeting.GetCashOut(7,true,true);

String strDebugOut =
            objWriter.ToString();

// get the random number from Debug
            //

String strWinning =
            strDebugOut.Substring(
strDebugOut.IndexOf("<random>")+8,
strDebugOut.IndexOf("</random>") -
(strDebugOut.IndexOf("<random>")+8) );

// Test that
            the amount returned is what was expected
            //

Assertion.AssertEquals("amount not equal to what was
            expected",
System.Convert.ToDouble(strWinning) * 1000,
            amount);
}

//
            ----------------------------------------------------- //
// Test
            Path B - Test two //
//
            -----------------------------------------------------
            //

[Test]
public void
            GetCashOutTestPathB(){

StringWriter objWriter = new
            StringWriter();
TextWriterTraceListener objListener =
            new
TextWriterTraceListener(objWriter);

Debug.Listeners.Clear();
Debug.Listeners.Add(objListener);

Assertion.Assert(objGreeting.GetCashOut(7,false,true)
            > 0);

String strDebugOut = objWriter.ToString();

//
            Test that no Debug.Assert statements were false
            //

Assertion.Assert(strDebugOut, strDebugOut.IndexOf("line")
            < 0);

// Test path B taken
            //

Assertion.Assert("Path
            wrong?",
strDebugOut.IndexOf("in_ReturnedCashOutNotTaxed") >
            0);

objWriter.Close();
objListener.Close();
}

[Test]
public
            void GetCashOutTestTheAmountPathB(){

StringWriter objWriter =
            new StringWriter();
TextWriterTraceListener objListener =
            new
TextWriterTraceListener(objWriter);

Debug.Listeners.Clear();
Debug.Listeners.Add(objListener);

//
            Get winnings - user is winner, doesn't pay tax, and is over 18
            //

double amount =
            objGreeting.GetCashOut(7,false,true);

String strDebugOut =
            objWriter.ToString();

// get the random number from Debug
            //

String strWinning =
            strDebugOut.Substring(
strDebugOut.IndexOf("<random>")+8,
strDebugOut.IndexOf("</random>") -
(strDebugOut.IndexOf("<random>")+8) );

// Test that
            the amount returned is what was expected
            //

Assertion.AssertEquals("amount not equal to what was
            expected",
System.Convert.ToDouble(strWinning) * 1000,
            amount);
}

//
            ----------------------------------------------------- //
// Test
            Path C - Test three //
//
            -----------------------------------------------------
            //

[Test]
public void
            GetCashOutTestPathC(){

StringWriter objWriter = new
            StringWriter();
TextWriterTraceListener objListener =
            new
TextWriterTraceListener(objWriter);

Debug.Listeners.Clear();
Debug.Listeners.Add(objListener);

Assertion.Assert(objGreeting.GetCashOut(6,false,true)
            > 0);

// Test path C taken
            //

Assertion.Assert("Path
            wrong?",
strDebugOut.IndexOf("in_ReturnedCashDeducted") >
            0);

String strDebugOut = objWriter.ToString();

// Test
            that no Debug.Assert statements were false
            //

Assertion.Assert(strDebugOut, strDebugOut.IndexOf("line")
            <
            0);

objWriter.Close();
objListener.Close();
}

[Test]
public
            void GetCashOutTestTheAmountPathC(){

StringWriter objWriter =
            new StringWriter();
TextWriterTraceListener objListener =
            new
TextWriterTraceListener(objWriter);

Debug.Listeners.Clear();
Debug.Listeners.Add(objListener);

//
            Get winnings - user is not winner, doesn't pay tax, is over 18
            //

double amount =
            objGreeting.GetCashOut(6,false,true);

// Test that the amount
            returned is what was expected
            //

Assertion.AssertEquals("amount not equal to what was
            expected", -10,
amount);
}

//
            ----------------------------------------------------- //
// Test
            Path D - Test four //
//
            -----------------------------------------------------
            //

[Test]
[ExpectedException(typeof(System.Exception))]
public
            void Under18Test(){
double amount =
            objGreeting.GetCashOut(6,false,false);
}

}
}

Conditional Testing

While branch testing focuses on ensuring a program follows a particular path, conditional testing focuses on the condition that might have led to the path taken. You use conditional testing to test complex logic. For example, suppose we had to develop a simple component that simulated a baby's decision making when deciding to eat food, play with the food, or throw the food:

  • If food is bad, baby is angry, and mom looks like she might give a spanking if baby is bad - refuse food.
  • If food is bad, baby is angry, and mom does not look like she might give a spanking if baby is bad - throw bowl.
  • If food is bad, baby is mischievous, and mom looks like she might give a spanking if baby is bad - eat food.
  • If food is bad, baby is mischievous, and mom does not look like she might give a spanking if baby is bad - refuse food.
  • If food is bad and baby is tired - sleep.
  • If food is good - eat food.

We can model these specifications using a decision tree:

We can also model the logical conditions using a truth table.

 | Mood:Angry | Mood:Mischievous | Mood:Tired | Delicious | Mom will spank
Throw Bowl | true | false | false | false | false
Refuse Food | true | false | false | false | true
Refuse Food | false | true | false | false | false
Sleep | false | false | true | false | na
Eat | false | true | false | false | true
Eat | na | na | na | true | na

If baby throws his or her bowl, then we know the following is true:

enumBabyReaction == BabyReaction.ThrowBowl && mood == BabyMood.Angry && !blnFoodGood && !blnMomSpank

If the baby refuses food, then we know the following is true:

enumBabyReaction == BabyReaction.RefuseFood
            && ((mood == BabyMood.Angry &&
blnMomSpank) ||
            (mood == BabyMood.Mischievous && !blnMomSpank))
            &&
!blnFoodGood

If baby eats food, then we know the following is true:

enumBabyReaction == BabyReaction.EatFood && ( blnFoodGood || (mood ==
BabyMood.Mischievous && blnMomSpank))

The complete Baby class follows:

namespace AssertsAndDebug {
using System;
using System.Diagnostics;

public class Baby {

public enum BabyReaction{ThrowBowl,EatFood,RefuseFood,Sleep};
public enum BabyMood{Angry,Mischievous,Tired};

private BabyMood mood;

public BabyMood Mood{
set{mood = value;}
}

private bool blnMomSpank;

public bool MomSpank{
set{blnMomSpank = value;}
}

// Determine baby's reaction based upon several factors //

public BabyReaction GetFood(bool blnFoodGood){

BabyReaction enumBabyReaction = BabyReaction.EatFood;

if (blnFoodGood) enumBabyReaction = BabyReaction.EatFood;
else{

switch (mood) {
case BabyMood.Angry :
if (blnMomSpank) enumBabyReaction = BabyReaction.RefuseFood;
else enumBabyReaction = BabyReaction.ThrowBowl;
break;
case BabyMood.Mischievous :
if (!blnMomSpank) enumBabyReaction = BabyReaction.RefuseFood;
break;
case BabyMood.Tired :
enumBabyReaction = BabyReaction.Sleep;
break;
}

// note: using fact that this code is only executed when in
// debug mode. be carefull for side-effects

Debug.Assert(this.BabyReactionTest(enumBabyReaction,
blnFoodGood),"Failed BabyReactionTest Baby.cs line 38");

return enumBabyReaction;
}

// Test that ensures the conditions are correct for baby's reaction //

private bool BabyReactionTest(BabyReaction enumBabyReaction,
bool blnFoodGood){

bool passed = false;

if (enumBabyReaction == BabyReaction.ThrowBowl
&& mood == BabyMood.Angry
&& !blnFoodGood
&& !blnMomSpank
)
passed = true;
else if (
enumBabyReaction == BabyReaction.RefuseFood
&& (
(mood == BabyMood.Angry && blnMomSpank)
||
(mood == BabyMood.Mischievous && !blnMomSpank)
)
&& !blnFoodGood
)
passed = true;
else if (enumBabyReaction == BabyReaction.EatFood
&& ( blnFoodGood
||
(mood == BabyMood.Mischievous && blnMomSpank)
)
)
passed = true;
else if (enumBabyReaction == BabyReaction.Sleep
&& mood == BabyMood.Tired
)
passed = true;

return passed;
}

}
}

When unit testing the GetFood method, you need to ensure every logical possible action is invoked. To do this, we need to understand what causes different reactions. For example, when the food is bad, mommy will not spank, and baby is angry, GetFood returns ThrowBowl. The following test initializes the baby component so that GetFood returns ThrowBowl :

[Test]
public void
            TestThrowBowl(){
this.ReInitializeWriters();
this.objBaby.MomSpank
            = false;
this.objBaby.Mood =
            Baby.BabyMood.Angry;
this.objBaby.GetFood(false);

String
            test = objWriter.ToString();
if
            (objWriter.ToString().Trim().Length >
            0){
Assertion.Fail(objWriter.ToString().Trim());
}
}

The following code tests every possible Reaction from GetFood :

namespace AssertsAndDebug {
using
            System;
using NUnit.Framework;
using
            System.Diagnostics;
using
            System.IO;

[TestFixture]
public class BabyTest
            {

private Baby objBaby = new Baby();
private StringWriter
            objWriter = new StringWriter();
private TextWriterTraceListener
            objTextListener;

[SetUp]
public void
            Init(){
objTextListener = new
            TextWriterTraceListener(objWriter);
Debug.Listeners.Clear();
Debug.Listeners.Add(objTextListener);
}

private
            void ReInitializeWriters(){
objWriter.Close();
objWriter = new
            StringWriter();
objTextListener.Writer =
            objWriter;
}

[Test]
public void
            TestThrowBowl(){
this.ReInitializeWriters();
this.objBaby.MomSpank
            = false;
this.objBaby.Mood =
            Baby.BabyMood.Angry;
this.objBaby.GetFood(false);

String
            test = objWriter.ToString();
if
            (objWriter.ToString().Trim().Length >
            0)
{
Assertion.Fail(objWriter.ToString().Trim());
}
}

[Test]
public
            void
            TestRefuseFood(){
this.ReInitializeWriters();

this.objBaby.MomSpank
            = true;
this.objBaby.Mood =
            Baby.BabyMood.Angry;
this.objBaby.GetFood(false);

this.objBaby.MomSpank
            = false;
this.objBaby.Mood =
            Baby.BabyMood.Mischievous;
this.objBaby.GetFood(false);

String
            test = objWriter.ToString();
if
            (objWriter.ToString().Trim().Length >
            0)
{
Assertion.Fail(objWriter.ToString().Trim());
}
}

[Test]
public
            void
            TestEat(){
this.ReInitializeWriters();

this.objBaby.GetFood(true);

String
            test = objWriter.ToString();
if
            (objWriter.ToString().Trim().Length >
            0)
{
Assertion.Fail(objWriter.ToString().Trim());
}
}

[Test]
public
            void
            TestSleep(){
this.ReInitializeWriters();

this.objBaby.MomSpank
            = false;
this.objBaby.Mood =
            Baby.BabyMood.Tired;
this.objBaby.GetFood(false);

String
            test = objWriter.ToString();
if
            (objWriter.ToString().Trim().Length >
            0)
{
Assertion.Fail(objWriter.ToString().Trim());
}
}

[TearDown]
public
            void
            Dispose(){
objWriter.Close();
objTextListener.Close();
}
}
}

Looping constructs like while loops also use logical conditions to determine when to stop looping. Suppose you have a component that allows you to drain a bucket. However, you can only drain it to a minimum level of 25 units and you only have 100 "cycles" that you can drain the bucket.

this.level > intDesiredLevel && this.level > MINACCEPTABLE && intCycles < MAXCYCLES

Thus, we know that after starting the drain action, it will only stop when the following condition evaluates to true.

(this.level == intDesiredLevel)|| (this.level == MINACCEPTABLE) || intCycles == MAXCYCLES)

Knowing this limitation, we can write an assertion that tests if this condition is true when draining stops.

namespace Examples {
using
            System;
using System.Diagnostics;

public class Drainer
            {

private int level;
private static int MAXCYCLES =
            100;
private static int MINACCEPTABLE = 25;

public
            Drainer(int level){
if (level < MINACCEPTABLE)
this.level =
            MINACCEPTABLE;
else
this.level = level;
}

public
            void Drain(int intDesiredLevel){
int intCycles =
            0;
Console.WriteLine("Draining");
while(this.level >
            intDesiredLevel &&
this.level > MINACCEPTABLE
            &&
intCycles <
            MAXCYCLES)
{
this.level--;
intCycles++;
Console.Write(".");
}
Debug.Assert((this.level
            == intDesiredLevel)||
(this.level == MINACCEPTABLE) ||
            (intCycles == MAXCYCLES),
"Assertion did not hold. Drainer.cs
            line 26.");

Debug.Write("<level>" + this.level +
            "</level>");
Debug.Write("<cycles>" + intCycles +
            "</cycles>");
}
}
}

We can then write the test. Using the same technique as before, we write a unit test that tests the debug output for the "line" string. To adequately cover the while loop, we must consider three conditions, or, the following three test cases:

  • Test one: initialize level to 900, set drain level to 400, expect the cycles to equal 100.
  • Test two: initialize level to 100, set drain level to 96, expect the level to equal 96.
  • Test three: initialize level to 27, set drain level to 12, expect the level to equal 25.

DrainTest implements these tests in the following NUnit test fixture:

namespace Examples {
using
            System.Diagnostics;
using System.IO;
using
            NUnit.Framework;

[TestFixture]
public class
            DrainerTest{

StringWriter
            objWriter;
TextWriterTraceListener
            objTextWriterTraceListener;

[SetUp]
public void
            Init(){
objWriter = new
            StringWriter();
objTextWriterTraceListener =
new
            TextWriterTraceListener(objWriter);

Debug.Listeners.Clear();
Debug.Listeners.Add(objTextWriterTraceListener);
}

private
            void ReInitialize(){
objWriter = new
            StringWriter();
objTextWriterTraceListener.Writer =
            objWriter;
}

// Test exit from maximum cycles being
            reached //

[Test]
public void
            TestDrainMaxCycles()
{
Drainer objDrainer = new
            Drainer(900);
objDrainer.Drain(400);
Assertion.Assert(objWriter.ToString().Trim(),
objWriter.ToString().IndexOf("line")
            <
            0);
Assertion.Assert(objWriter.ToString().Trim(),
objWriter.ToString().IndexOf("<cycles>100</cycles>")
            > -1);

}

// Test exit from desired level being
            reached //

[Test]
public void
            TestDesiredLevel(){
this.ReInitialize();
Drainer objDrainer =
            new
            Drainer(100);
objDrainer.Drain(96);
Assertion.Assert(objWriter.ToString().Trim(),
objWriter.ToString().IndexOf("line")
            <
            0);
Assertion.Assert(objWriter.ToString().Trim(),
objWriter.ToString().IndexOf("<level>96</level>")
            > -1);
}

// Test exit from minimum level being reached
            //

[Test]
public void
            TestMinLevel(){
this.ReInitialize();
Drainer objDrainer = new
            Drainer(27);
objDrainer.Drain(12);
Assertion.Assert(objWriter.ToString().Trim(),
objWriter.ToString().IndexOf("line")
            <
            0);
Assertion.Assert(objWriter.ToString().Trim(),
objWriter.ToString().IndexOf("<level>25</level>")
            > -1);
}
}
}

Data Definition and Use Testing

The data definition and use testing test is a white box testing technique that tests a data's definition and its later use. Using a watch in a debugger, for example, is an example of data definition and use testing. You test the variable's initialization and then you test its different values at different points in the application. The goal is writing tests that drive the program to initialize and then use data in certain ways. You are consciously testing how your program uses different variables.

Now, before we continue with an example, no, you would never want to program like this at home. Yes, using unsafe code, character arrays, and pointers like this in C# is irresponsible, at best. And no, the "fixes" described below are not really fixes at all. But the following example illustrates data definition and use testing, and why it is important, so suspend disbelief. Consider the following unsafe code example.

namespace Examples {
using
            System;
using System.Diagnostics;
using
            System.IO;

public class UnsafeExample {

// copy
            characters from one character array to another
// note that if
            destination is longer then source, and
// destination has
            characters already in it, those chars
// remain. i.e.
// src:
            test
// dest: hey now
// dest after copying:
            testnow

private static unsafe void DoSomething(char[] src,
            char[] dst){
int srcLen = src.Length;
int dstLen =
            dst.Length;

fixed (char* pSrc = src, pDst = dst){
char*
            ps = pSrc;
char* pd = pDst;

for (int i = 0; i < srcLen;
            i++){
*pd =
            *ps;
pd++;
ps++;
}

}

}

public static
            StringWriter DoIT (){

char[] initial = {'Y','O','U', '
            ','T','U','R','K','E','Y'};;
char[] outputMessage = {'Y','O','U',
            ' ','A','R','E',' ','1','!',
'
            ','W','R','I','T','E','R'};

DoSomething(initial,outputMessage);

StringWriter
            objWriter = new StringWriter();

// note: error here - the
            length of the string
// in the stringwriter is 17 "YOU
            TURKEY\0\0\0\0\0\0\0"
// note that \0 is a null char in C
            language

for(int i=0; i < outputMessage.Length; ++i)
objWriter.Write(outputMessage[i]);

return
            objWriter;
}

}
}

The UnsafeExample has two methods, DoSomething and DoIT. DoSomething takes two character arrays as parameters. The first array is the source, while the second is the destination. DoIt initializes two character arrays and then passes them to DoSomething. DoSomething uses pointers to copy the source to the destination. DoIT then copies the destination character array to a StringWriter and returns the StringWriter.

But note the glaring error in the preceeding code - src only overwrites the first ten characters of dst, leaving the remaining seven characters intact. Instead of YOU TURKEY, we obtain YOU TURKEY WRITER. We need to ensure that dst only contains the characters from src. One way we can do this is by checking that src always ends with null.

private static unsafe void
            DoSomething(char[] src, char[] dst){
int srcLen =
            src.Length;
int dstLen = dst.Length;

// assert that the
            last character of src is null string
            terminator

Debug.Assert(src[src.Length-1] == '\0',
"Last
            char not a null. UnsafeExample.cs:line 13");

// we would add
            exception here //

fixed (char* pSrc = src, pDst =
            dst){
char* ps = pSrc;
char* pd = pDst;

for (int i = 0;
            i < srcLen; i++){
*pd = *ps;
pd++;
ps++;
}
}

However, even after fixing DoSomething, there remains an error in DoIT. The outputMessage character array is 17 characters long. If initial is terminated with a \0, then when initial overwrites outputMessage in DoSomething, the first ten characters are overwritten while the remaining seven characters are null. A NUnit assertion like the following would fail.

Assertion.AssertEquals("YOU TURKEY", strMessage);

The string is YOU TURKEY\0\0\0\0\0\0\0 not YOU TURKEY. To fix this problem, only copy outputMessage to the first null.

public static StringWriter DoIT (){
char[] initial = {'Y','O','U', ' ','T','U','R','K','E','Y','\0'};;
char[] outputMessage = {'Y','O','U', ' ','A','R','E',' ','1','!',
' ','W','R','I','T','E','R'};

DoSomething(initial,outputMessage);

StringWriter objWriter = new StringWriter();

// only copy up to the first null

for(int i=0; outputMessage[i] != '\0'; ++i)
objWriter.Write(outputMessage[i]);

return objWriter;
}

We can now write the NUnit test and it passes, albeit the code it is testing is something you would never want to do in the first place.

namespace Examples {
using
            System.Diagnostics;
using System.IO;
using System;
using
            NUnit.Framework;

[TestFixture]
public class
            UnsafeExampleTest{

StringWriter
            objWriter;
TextWriterTraceListener
            objTextWriterTraceListener;

[SetUp]
public void
            Init(){
objWriter = new
            StringWriter();
objTextWriterTraceListener = new
            TextWriterTraceListener(objWriter);

Debug.Listeners.Clear();
Debug.Listeners.Add(objTextWriterTraceListener);
}

private
            void ReInitialize(){
objWriter = new
            StringWriter();
objTextWriterTraceListener.Writer =
            objWriter;
}

[Test]
public void UnsafeTest
            (){
String strMessage =
            UnsafeExample.DoIT().ToString().Trim();
Assertion.Assert(objWriter.ToString().Trim(),
objWriter.ToString().IndexOf("line")
            < 0);
Assertion.AssertEquals("YOU TURKEY",
            strMessage);
}
}
}

In the preceding code, notice how we analyzed and fixed the problems. We first noticed that the code did not initialize initial (which became src) with a \0 as the last character. We next found that when the code used dst and src, it did so incorrectly. Instead of overwriting dst with all of src, it only overwrote the first ten characters. After fixing this problem, we then noticed that the remaining characters in dst were overwritten with nulls. We did not obtain src, but instead, obtained src with trailing nulls. We fixed that problem and the function worked as expected.

Summary

Concurrent unit testing and coding will help you write robust components. A tool like NUnit facilitates writing new unit tests and automating regression testing. Although the distinction between black box testing and white box testing is unclear when discussing unit testing, there are several black box and white box testing techniques you can employ to unit test your code. When black box testing; specification oriented testing, equivalence partitioning, boundary value analysis, and state transition testing are just four of many available testing techniques. When white box testing; branch testing, conditional testing, and data definition and use testing are three of many available testing techniques.

Unit testing your code using proven techniques is more productive then random unit testing. Black box tests test that for given inputs you obtain expected outputs. When the output does not match the value expected, the test fails. White box testing tests that a unit's internals function correctly. By adding assertions and debugging statements, you test that certain conditions are true as the unit runs. There are many ways you can use assertions and debugging statements. This case study does not portend to be a discussion of the best way to use assertions and debugging statements. However, it does illustrate that you should use them so that their output is available to NUnit tests if you want to automate your testing.

C#Today - ©2002 Wrox Press Limited


Creating International-Ready Applications

Creating International-Ready Applications
http://www.csharptoday.com/content.asp?id=1996
Article Rating: 4.5
Published on November 25, 2002
Abstract
It is essential to build nimble software that can quickly adapt to new challenges. One of these challenges is in building applications that can respond to an increasingly global economy. While modern development languages have greatly eased the task of creating software for any locale, the challenge is still in creating software that can simultaneously support multiple locales. The .NET platform assists with many of these aspects, but there are still many design aspects to consider.In this case study, Damon Allison and Ben Hyrman discuss the design of international applications, and introduce you to the .NET Framework’s built-in support for internationalization.
Case Study

It is essential to build nimble software that can quickly adapt to new challenges. One of these challenges is in building applications that can respond to an increasingly global economy. While modern development languages have greatly eased the task of creating software for any locale, the challenge is still in creating software that can simultaneously support multiple locales. The .NET platform assists with many of these aspects, but there are still many design aspects to consider. In this case study, we will discuss designing of international applications, and introduce you to the .NET Framework's built-in support for internationalization.

Introduction

Why Internationalization?

Companies today must quickly adapt to change. The national brand of today is the international corporation of tomorrow. Businesses need software that can support this growth. Trade agreements open new commerce opportunities, and provide a new consumer base. It is a given that nimble companies will grow and rigid companies will fail. The same can be said of software.

The simple fact is that the world marketplace is becoming unified; this is especially true with regards to software, as computers are ingratiated worldwide. The case for creating internationalized software is a simple one. An application developer should be concerned with creating software that reaches as wide a base of customers as possible, and this increasingly means looking to other countries as well. An IT developer has to be ready to respond to company needs, which increasingly involves expansion into foreign markets and reusing core systems wherever possible.

There are many issues related to adapting software to new markets. Some of these issues include:

  • UI design considerations. Color meanings differ by culture, text widths can vary greatly, screen flow is not always left to right, and input fields may need to be larger. For example, red is often used to indicate danger or warning in the US. In fact, you may often see red used to denote data entry errors. However, red means happiness in China. The same consideration must be given to logos, icons, and images in general. An icon that seems harmless in the US, such as a thumbs-up gesture on a dialog box, is considered offensive and obscene in Japan.
  • Translation. This is one of the most time-intensive parts of localization. Every piece of text that a user will see must be changed, including menus, labels, and errors. Even elements that may not spring to mind, such as wording embedded in a graphic, must be translated.
  • Formatting differences. Everything from currency, to date and time, to weights and measures must be formatted for each locale.
  • Database design. Planning a database layout can be a challenge. Planning for a layout that can support multiple locales can seem almost impossible, at first.
  • Code segmentation and versioning. Code should be partitioned to separate the internationalized core from the localized components. Testing and versioning can become a large issue if not properly addressed.

In this case study, we will begin by discussing the concepts behind internationalization. We will point out where design mistakes are often made. We will also provide best practices and insight into planning and designing to handle internationalization. We will then focus on .NET's abilities to ease internationalization, and how to design applications to take advantage of the Common Language Runtime (CLR)'s built-in multi-cultural support for localization.

For better understanding of internationalization, it is important to discuss common terminology. Let's turn our attention to the concepts and terminology used in creating international apps.

Key Concepts in Internationalization

Globalization: This is the entire scope of effort defined here, including creating an internationalized core, localized extensions, and locale-specific translations.

Internationalization: Internationalization is the act of generalizing code so that it can support multiple locales with a minimal amount of redesign and redevelopment. Internationalized code is independent of the language, culture, and character set. Internationalization does not include translation, cultural design considerations for a specific locale, or any work related to localizing an application.

Internationalization is abbreviated as I18N. It may seem cryptic and mystical at first, but this abbreviation likely came about out of laziness, or ingenuity. If you will notice, there are eighteen letters between the first 'I' and the last 'n' in internationalization.

Localization: Localization is the exercise of customizing the application to meet specific requirements for deployment to a locale. Localization generally covers areas such as translation, formatting, and user interface redesign. Localization is often referred to as L10N.

Typically, a localization effort will include the help of a subject matter expert for the given locale. In other words, if you're US-based, developing for France, and the only thing you know about the culture is that they have an accent, then you should engage an expert to assist with UI design, locale formatting, and translation.

In addition, this effort will employ the use of graphics artists to localize images, translators to tackle all of the text in the application, and programmers will be responsible for any code changes to support a specific locale.

Locale: A locale defines a user's environment, including information specific to country, language, and code set. For example, date formatting, numeric formatting, currency type, time zone, and language are part of what make up a user's locale. This becomes crucial in determining both how to store and display data. In the US, we might state that an appointment is on 07/04/2002; while in Germany we would display the date as 04.07.2002.

Translation: Translation is the linguistic component of localization. Translation involves the conversion of thoughts, ideas, and concepts from one language to another. This may be the single largest effort of localization because it applies to all text a user will see, including user interfaces, help files, and documentation.

One thing that we should point out here is that translation is still largely a manual process. While there are many automatic translation services available, ranging from free to not-so-free, these are still not quite to the level of being able to be fully automated features of any application.

System Requirements

This case study assumes a basic understanding of C# and ASP.NET. In order to run the code you will need the ASP.NET V1.0 Framework installed and access to a SQL-aware database. The SQL make table scripts are included. They may need to be slightly modified to work with your database. SQL Server 2000 was used for this piece.

Defining The Problem

Design Challenge

The challenge with internationalizing applications is determining an effective design. A good design allows the application to add locales without the need to change database, code, or application logic. In the ideal scenario, adding additional locales to the application would simply be a matter of creating locale specific data, putting the data into the proper data stores, and testing the application. For example, someone would translate the user interface components such as text, dialog boxes, and graphics and give the translated data to the developer. They would add the data and the application would be off and running in a new locale. While this seems far-fetched and difficult, it isn't. If the application is designed with internationalization in mind (the application is "internationalized"), it is possible to rapidly speed up rolling out changes to an application to support new languages, currencies, and locales. Of course, testing must still happen, but many parts of development are made much easier.

The application created in this case study will pay particular attention to the design considerations and choices developers need to consider in order to create a well designed internationalized application. As a developer, you know the challenges of changing applications mid-lifecycle and how much more difficult it can be than creating an application from scratch. Internationalization is a far-reaching 'change' and needs to be considered early to avoid extensive recoding.

While designing for internationalization may seem blatantly obvious, many subtleties are often overlooked in the design stage. For example, if images have embedded text, they must be modified for each culture.

System Upgrades

There is a point mentioned in the last section that deserves to be said again: a more difficult challenge internationalization poses is determining how to implement locales into a current, non-internationalized application. It is important to understand how much work is involved in upgrading systems. Not only does the international design need to be developed, the current logic must be preserved and the implementation must be transparent to the user. Imagine an application designed to work with ASCII data that now must support Unicode. Serious changes could be needed, which could have easily been avoided if Unicode had been considered from the start.

Pre-.NET Landscape

Prior to .NET, internationalization was not always easy to implement in Microsoft systems, especially with Web-based applications. While VB and C++ applications had some support for resource files and could use API calls to retrieve locale information, there was little inherent support for internationalization that didn't have to be customized and tweaked. The landscape was even worse for ASP developers. This led to many different solutions of varying levels of sophistication and reliability.

When attempting to create international-ready ASP applications, programmers were often forced to buy third party products or implement custom database-driven applications to return localized information. Some crafty developers created XML file based solutions with a small API to retrieve localized data from the file. However, these solutions were all one-offs, there was no single means for internationalization, and these solutions took time to build. Plus, in the end, parsing an XML document or issuing SQL calls brings a significant overhead. What was needed was a single API, built into the runtime, which would take much of the burden off of the developer.

Microsoft's choice to implement the .NET framework with internationalization features further eases the development cycle. The .NET framework incorporates many of the common features found in internationalized applications (resource lookup and formatting in particular) directly into the framework.

Our Case Study

Our fictitious client, ComputerReviews, is a computer and technology retailer based in the US. ComputerReviews is developing a website to accept product reviews online. The owner of ComputerReviews would like to start by offering a German and English interface to their site. Consideration must be given for additional languages to be implemented in the future with little to no recoding.

The possibilities for an international website are infinite. Perhaps a site that you have developed would be a candidate for a multicultural website. Perhaps your business would have a competitive advantage if you could offer a Spanish or French translation. Our case study goal is to present a detailed look at how to design an internationalized application using the technologies available with the .NET framework.

Case Study Structure

In this case study, we will be creating an internationalized website from scratch to support English and German languages using the .NET framework. Since perhaps the most important part of internationalizing an application occurs in the design phase, we will begin by discussing design considerations. Once we have an application design, we will implement the database, component layers, and user interface. The outline is as follows:

  • Planning for Internationalization
  • Code Structure
  • Best Practices
  • Designing for Internationalization
  • Database Design and Structure
  • Component Design and Development
  • UI Design and Localization
  • Satellite Assemblies versus Resource Files
  • Next Steps
  • Related Information and considerations for improvements

Planning for Internationalization

Planning and designing a system for internationalization will take more time and effort than designing an application for a single locale. However, the effort spent at the start will reap enormous benefits the first time the application is expanded to support a second locale, or even a second language in the same locale.

While it may not seem too obvious to a normal web user, there is a huge amount of text and images that are used when building a website. All the labels, links, menus, images, colors, error messages and text must be translated and reviewed from a translator. Anything that is presented to the user is subject to be different across locales, and incorporating all the data into your application requires significant initial investment.

Code Structure

An internationalized application should be designed to maximize reuse across locales. On the surface this may sound obvious, but in practice it is not always followed. Code that will span multiple locales, such as business logic, should be placed in the application core - separated from locale specific data. Services that are specific to a locale, such as tax calculation, should be exposed to the core through a façade if needed, and placed into the localized extensions.

Quite simply, a façade is a means to hide a specific implementation from other components. For example, a tax component is generally specific to each locale. However, it wouldn't be wise to directly tie the application core to each specific tax implementation. A façade can provide a common interface between the application core and the various tax components. Graphically, this would like this:

In this example, we've created a core to the application that houses the bulk of the application logic. The localized extension should contain only information specific to a given locale. Over time, the application should be refactored to move functionality duplicated across locales into the core. We've also shown a service whose concrete implementation is specific to a locale. A tax engine is a good example of a utility that is location specific. We've hidden the implementation to keep from tying the core to any single locale. Of course, this is very easy to do on paper, but will require quite a bit of thought for any real application.

User interface design is a bit more difficult, since unlike the component layer, the user interface is drastically affected by culture. For example, as the culture changes, so does the need for larger or smaller sized labels and text controls. A 20 character string in English can translate into a 60 character string in another language. One option to deal with the interface differences across cultures is to create a separate UI for each locale. While we have total control of the interface, we lose the ability to create and manage a single code base. Plus, maintenance and support across locales becomes difficult as we have the same user interface logic repeated many times over.

Another option is to make everything from colors, to layout, to labels and messages dynamic in order to support all locales. The advantage that there is only one code base to maintain. As you will see, the .NET Framework has made this option much easier and more attractive. Even with the improvements, however, internationalization requires careful thought and design.

A possible trade-off between the two options, especially taking into account your first target locales, is to build a UI for logical groups of locales. For example, you may want to create one UI for Latin-based languages and another for each graphical-based language such as Bengali and Kanji.

Again, the goal is to push the bulk of the application into the core and have lightweight extensions per locale, as described in the diagram below. This will greatly ease development for future locales.

One more note, concerning testing. It should be obvious, but changes to the core must be tested across all locales, while changes to the extension will generally only require testing between that locale and the core. This is a trade-off to the recommended approach of maximizing the core in that the frequency of changes, and therefore testing with all locales, is increased. However, the benefits in reuse and leveraging a single code base greatly outweigh this drawback. Again, a solid design up front will further assist with minimizing the need for changes later on.

Internationalization Best Practices

There are many facets of internationalization that are very straightforward. For example, many Americans do not use the metric system for measurement, but you would be hard-pressed to not support metric measurements in Europe. When designing an application, it becomes important to identify and account for these differences. While not an exhaustive list, here are a few examples of how cultures differ and how to design for them in your application.

  • Currency
  • Do not store numeric separators with the currency value
  • Do not store currency symbols with the currency value
  • Ensure the field size is larger enough to support multiple locales
  • Ensure it is possible to identify currency locale, especially if it must be converted for multiple locales
  • Language
  • Store strings in Unicode
  • Store literals, labels, and messages in resource files or a database; never hard-code strings
  • Always use the Framework for sorting and comparing strings
  • Store user input in the original language entered

Measure

If storing multiple units of measure, such as both English Imperial and Metric:

  • Store the measure as well as unit of measure (i.e., store both "50" and "lbs", not just "50")

Or

  • Store only the measure but ensure that storage in a single format is understood across the application. In the case of measures, this is generally the preferred approach, as converting measures is a fairly static operation.

Measures are a bit unique in that conversions from one unit to another are constant, the formulas are well known, and the overhead for conversion is small. The key in design is to provide a consistent approach.

  • Date and Time
  • Always store date in a 24-hour format (HH:MM:SS)
  • Store time as Greenwich Mean Time (GMT) + offset. For example, US Central Standard Time is GMT-6
  • Be ready to handle daylight savings time. If not handled correctly, this can pose problems, especially where timestamps are used to ensure proper ordering
  • Store date in ISO standard format, YYYY-MM-DD; see ISO 8601 for details
  • Graphics
  • Graphics should not contain text. For static graphics, the text would need to be re-edited for every locale. For dynamically created graphics, the text may not fit properly for every language.
  • Graphics should be easily replaceable. Colors may need to be changed for graphics in other cultures. Icons may need to be replaced to offer a clear meaning in other cultures. These graphics should be replaceable with minimal impact to the application.

Database Design and Structure

Go ahead and open a database you've designed in the past. Is it well designed? Sure, although you probably see some things now that you'd change. Now, can it handle multiple locales, or even multiple languages? Would you know what records belong to which locale? Regardless if your database is designed for internationalization or not, there are certain principles that all databases can use to improve their support for internationalization. Let's first take a look at a typical, one language database design and consider the impact of changing the structure to support internationalization.

Typical Database Design

In a typical database design, the data is normalized into logical relationships in a manner that minimizes data redundancy. However, thought is rarely given to supporting multiple locales. Take a look at our first design for the product catalog and reviews.

We have two tables joined with a foreign key. The 'Products' table represents a list of products for sale and the 'Reviews' table represents a user review for a particular product. While this design certainly meets our needs, it presents some problems when we discuss expansions into even our second locale. We'll likely want to carry the same product in all locales. However, our current design dictates that we must duplicate product information for each language if we want to display the product description in the user's language.

Let's examine the 'Products' table. How would you add a German record in the database? How would you know the record is German when retrieving for display? At the very least, we need to specify a locale to retrieve this information. Assume for arguments sake that we added a field and called it locale. We would have the ability to determine which products are German, but we would run into another problem. The ProductID for German is different from the ProductID in English. We would need to change the primary key to be a combination of ProductID and Locale.

Even in our simple two table example, designing for internationalization is beginning to require quite a bit of database change. Notice how drastic the changes are becoming and think of how those changes will impact deployed code. All the code that relied on ProductID to be the primary key would now need to be edited. Stored procedures would need to be edited. At the object layer, the design would need to make room for locale. With a more complicated database, we could only imagine the change necessary.

Internationalized Database Design

We've expanded the database design to take into account multiple locales. At first the design may seem to be much more complex. However, the benefits should become evident rather quickly. We've further segmented the design to group data common to all locales and languages apart from the data that must change for each language or locale.

So why did we make such changes? There is some information in the Products table that is culture neutral that we can separate into its own table. The new 'Products' table contains data that does not vary over cultures. For our example, Manufacturer (Say IBM) does not translate any differently in German than it does in English. The same holds true for model number.

The design that you choose for your database may be different than ours. Since we don't show any information from the Products table to the user (we use ProductDetails), we can abstract this information into its own table. Products is used internally by our company and does not need to be taken into consideration when internationalizing. We do need it, however, for our internal uses.

If this data would be presented to the user, you would want to consider locale. For example, Chinese may require a different model number and manufacturer. Design the system as it meets your needs.

No matter how you design the system, there is one principal that should be considered: data that is common to all locales and languages should be placed in base tables. Data that must be customized or translated for each locale or language should be separated into extensions of the base tables. Candidates for the base table may not always be obvious. For example, measurement and weight would likely be best placed in the base table. The conversions for these units are simple, and a single value will be much easier to manage than values spread across all entries in the extension. In our example, Products is the base table, ProductDetails is the extension.

NOTE: We have made this database available as a SQL script in the source download. The source for this application can be found by clicking the appropriate tab on the toolbar at the top of the screen.

Component Design and Coding

Components are where the application's logic and rules are applied. A well-designed system will have its logic separated from the user interface into a set of components. In traditional COM, DLLs might be written in C++ or Visual Basic to provide the logic needed for the application to run. In .NET, assemblies have replaced COM objects and are a much more versatile unit of deployment than an unmanaged DLL.

Have you ever thought about how the design of a component changes with the introduction of multi-language sites? For example, can your application support Unicode or ASCII? What languages are the error messages written in? These design considerations that were taken for granted with one language now become important.

Dealing with Interfaces

How much of your component design, in particular the interface design, is impacted by internationalization? If designed properly, not much! The interfaces that are exposed by the components will need to expose parameters or functionality for setting the locale information used for retrieving data, but the majority of the interface will remain similar to your current designs.

The component layer is often more complicated than what we are presenting in this small application (we will see the code shortly). But no matter how complicated, we can generalize topics that impact internationalization at the component layer.

Watch out for all data points and avoid hard coding strings and values that will be presented to the user. Internal data and data structures can hold and use strings internally written in one language. For example, we should not be concerned with using foreign keys in one common language for access into tables or using English constants to represent database names and system values. Since these values are not returned to the user, they are language neutral. However, the text for the accompanying error message cannot be hard coded as it is specific to each locale and must be returned to the user. .

Let's take a look at the component code used to retrieve information from our internationalized database above. Pay particular attention to the use of Locale in the functions. Here is the code in its entirety. You don't need to completely understand the code in its entirety as we will discuss in more detail shortly.

/***********************************************************************
* ProductRetrieval.Retrieve
* Purpose: Provides data access (middle tier)
* layer to retreve/save data to/from the
* data store. This abstraction allows us to
* separate data access from UI.
*
* Improvements:
* Passed in culture is not verified.
* Perhaps create a table with supported cultures.
* This example uses inline sql for brevity.
* A more robust approach would be to call Stored
* Procedures maintained within the database.
* CommandBehavior.CloseConnection - closes connection when data reader is closed.
***********************************************************************/
//csc /t:library /r:System.Data.dll /r:System.Xml.dll ProductRetrieval.cs

using System;
using System.Data;
using System.Data.SqlClient;
namespace ProductRetrieval
{
/// <summary>Provides data retrieval procedures.</summary>
public class Retrieve
{
private string CONN;
private string locale;
public Retrieve(string LocaleCode)
{
//NOTE: Change conn string for your environment.
//Possible improvement: use a configuration file to avoid hardcoding
CONN = "Data Source=localhost;Initial Catalog=Products;Integrated Security=sspi";
locale = LocaleCode;

}
/// <summary>
/// Returns the product list in the specified locale code.
/// The ProductDetails table in the product database must have a
/// product record for each product and each locale.
/// </summary>
/// <returns>A SqlDataReader of all avaliable products for a given locale</returns>
public SqlDataReader GetProducts()
{
SqlConnection cn = new SqlConnection(CONN);
SqlCommand cm = new SqlCommand("Select * from ProductDetails where LocaleCode = '" +
locale + "'", cn);
cn.Open();
SqlDataReader dr = cm.ExecuteReader(CommandBehavior.CloseConnection);
return dr;
}

/// <summary>
/// Returns specific product information on a given product/locale combination
/// </summary>
/// <param name="ProductID">Product ID to retrieve</param>
/// <returns>SqlDataReader with 1 record containing product information</returns>
public SqlDataReader GetProductDetails(int ProductID)
{
SqlConnection cn = new SqlConnection(CONN);
SqlCommand cm = new SqlCommand("Select * from ProductDetails Where LocaleCode = '" +
locale + "' and ProductID = " + ProductID, cn);
cn.Open();
SqlDataReader dr = cm.ExecuteReader(CommandBehavior.CloseConnection);
return dr;
}
/// <summary>
/// Returns review information for a product. The user
/// will only see the reviews specified in their locale.
/// </summary>
/// <param name="ProductID">ProductID to retrieve reviews from</param>
/// <returns>SqlDataReader containing the product reviews. 0->n rows</returns>
public SqlDataReader GetReviews(int ProductID)
{
SqlConnection cn = new SqlConnection(CONN);
SqlCommand cm = new SqlCommand("Select * from Reviews Where LocaleCode = '" +
locale + "' and ProductID = " + ProductID + " order by ReviewID DESC", cn);
cn.Open();
SqlDataReader dr = cm.ExecuteReader(CommandBehavior.CloseConnection);
return dr;
}

/// <summary>
/// Add a review to a specific product.
/// Possible improvements include validating data before inserting into database.
/// This procedure could insert a blank record.
/// </summary>
/// <param name="ProductID">ProductID to add review</param>
/// <param name="Name">Reviewer Name</param>
/// <param name="Rating">Reviewer Rating</param>
/// <param name="Title">Title of reviewer's comments.</param>
/// <param name="Review">Review text supplied by reviewer.</param>
public void AddReview(int ProductID, string Name, int Rating,
string Title, string Review)
{
SqlConnection cn = new SqlConnection(CONN);
SqlCommand cm = new SqlCommand("INSERT INTO Reviews (ProductID, ReviewerName, LocaleCode, " +
"DatePosted, Rating, Title, Review) VALUES (@ProductID, @ReviewerName, @LocaleCode, " +
"@DatePosted, @Rating, @Title, " +
"@Review)", cn);
cm.Parameters.Add("@ProductID", ProductID);
cm.Parameters.Add("@ReviewerName", Name);
cm.Parameters.Add("@LocaleCode", locale);
cm.Parameters.Add("@DatePosted", DateTime.Now);
cm.Parameters.Add("@Rating", Rating);
cm.Parameters.Add("@Title", Title);
cm.Parameters.Add("@Review", Review);
cn.Open();
cm.ExecuteNonQuery();
cn.Close();
}
}
}

The code here is very simple. It performs no logic. It simply retrieves and saves data into the database. What is important in terms of internationalization is the inclusion of LocaleCode as a parameter to the constructor. The locale parameter could have been passed into each function, but since an object is created every time it is needed, we are assuming the locale will be the same from the time the object is created until it is destroyed. Let's examine the important function:

public Retrieve(string LocaleCode)
{
//NOTE: Change conn string for your environment.
//Possible improvement: use a configuration file to avoid hardcoding
CONN = "Data Source=localhost;Initial Catalog=Products;Integrated Security=sspi";
locale = LocaleCode;
}

In this constructor, locale is passed into the constructor and used throughout the other functions for storing the locale into the database. The other functions simply issue a SQL statement to the database and retrieve a SqlDataReader or write a record into the database. This code will allow us to retrieve from the database and save product reviews. These four functions are the extent of our component layer. To compile this code, save the file as ProductRetrieval.cs and use the following statement at a command prompt:

> csc /t:library /r:System.Data.dll /r:System.Xml.dll ProductRetrieval.cs

csc.exe is, of course, the command-line compiler for C#

NOTE: You will want to edit the connection string to pertain to your environment. You may also want to create a test project and test the four functions out. They are quite self-explanatory and can be tested very easily.

At this point, it becomes quite obvious that we are not designing a true enterprise application, but rather an example to show internationalization. There are many improvements that can be made to this code for security, scalability, and performance. Some of the more obvious improvements would be moving the database connection string into a configuration file, using stored procedures to execute the SQL, and proper error handling. We will show how to retrieve locale specific strings when we discuss UI, just understand that locale specific error messages should be considered.

This example illustrates including locale at the component level and also how the middle tier is largely language agnostic. Adding locale at the constructor level or at the function level enables us to vary query results dependent on language. If this component were to handle logic, the locale should be largely transparent. For example, if we are adding dates or subtracting dates, the base class libraries should be used to handle the calculation.

The base class libraries are designed with culture in mind. For example, if you pass in a currency, it will be returned in a culture specific format. Since this functionality is built into the base class libraries, it becomes important to understand the formatting options presented in the base class libraries and avoid recreating similar functionality. When we discuss UI, you will see examples of the base class libraries in action.

UI Design and Coding

So far we have the database designed and the component layer designed without performing any translation. This is not uncommon since the bulk of the work in internationalization surrounds the presentation layer. Internationalization is largely about presentation. The application logic and storage is relatively similar across all cultures.

Very briefly, let's brainstorm what needs to happen at the user interface level to support multiple languages.

First, strings, captions, images, colors, and images need to be retrieved from a data store. We want the ability to have one .aspx page to use across cultures to avoid repetition. In order to achieve this goal, we need to store locale specific strings in some form of data store. Much of this data is not dynamic in nature; it is just different across cultures. If it were dynamic, it would change frequently and is better implemented in a database. The UI elements such as captions, copyrights, and slogans are not generally very dynamic in nature.

Second, the data retrieval must be very fast. Since potentially thousands of strings and culture specific data will be retrieved with each request, we need to make sure we do not hinder performance drastically. Storing perhaps a thousand resource strings into a database can hinder performance, drastically if not properly cached.

Third, we want the process of interacting with the culture specific data to involve as little effort as possible. We do not want to spend time and write large amounts of code to retrieve and display data. For example, we do not want to code if blocks or switch statements around the different cultures. It is obvious this code will not scale if additional cultures are added.

if(culture != "en-US")
myString = "Willkommen";
else
myString = "Welcome";

Lastly, we need the ability to plug in another culture without rewriting any code. We want to be able to insert the strings into our data store and the locale will be ready to go! With .NET, there is an entire namespace devoted to using resource information: System.Resources.

Resource Files

.NET continues the same concept of a resource file to provide resources to an application. At a high level, a resource file is a set of culture specific data that is compiled and read by the System.Resources.ResourceManager class. String data is entered in a resource file and saved with an appropriate filename to signify what culture the resources belong to. A resource file is created for every language and the user interface designer is free from needing to store the data into the database or into the interface directly. The resource file is similar to a configuration file, where parameters are retrieved and used when needed.

The vast majority (if not all) applications use textual resources. A resource is simply a piece of data that is used by the application when executing. The data itself is not code and cannot be executed; it is simply read into the application at runtime.

The simplest form of a resource file is a .txt file containing name value pairs. Let's take a look at an example of a resource file:

;example resource file
;comment lines start with ;

;General strings
Welcome=Welcome to my website:

;Product
ProductName=Take a look at our product list!

Resource files have a special file naming requirement. Each file is named with the following format:

<basename>.<culture>.txt

Basename is the name used when referencing the application. The culture is the name of the locale these resources are saved as. To create a set of 'strings' resources, the following names would be used for the American English (en-US) and German (de-DE) locales:

strings.en-US.txt
strings.de-DE.txt
strings.txt

As an aside, a resource for England would be en-GB Australia would be en-AU.

When a user enters the site with an en-US locale, the en-US resource file is used. Likewise, when the user enters with a de-DE locale, the de-DE resource file is used. The file strings.txt is called a fallback resource file. When a user enters the application with an unrecognized locale (not en-US or de-DE in our example), the fallback resource file is used for resources. Fallback resources are important to implement, especially on the web, where you are not certain what locales will be used. If a user enters the site with an unrecognized locale and the site does not implement fallback resources, an error will be thrown. Resources should not be a source of application errors.

Once we have the resource files built for our application, we will compile them with the resgen.exe tool. The command to compile a resource file is:

> resgen strings.en-US.txt

The resgen.exe tool will create a file called strings.en-US.resources. All resource files, once compiled, are given the .resources extension. The ResourceManager class (discussed shortly) is programmed to use the .resources extension when looking for resource files. If you deviate from the expected filename, the resources will not be recognized. The resource files are compiled into a more compact format

Resource files must be created prior to creating the user interface. At this point, it becomes difficult to recognize the importance of the resource files. If we created the user interface at this point, we would not be able to internationalize the application. Now that they are created, we can begin to construct the user interface. Let's look at the English and German resource files used in our case study. The basename of our resource files is ui.

English version :

;ui.en-US.txt

;Banner User Control
BannerHeader=Product Display
;Product.aspx
Welcome=Welcome to the product reviews website. This page is designed to provide product information and user reviews.
Click on one of the following products for more information.
ProductTableHeader=User reviews and comments
CultureSelect=Select a culture
;ProductInfo
WriteReview=Want to write a review? Now's your chance!
ReviewTableHeader=Product reviews
ReviewerName=Your Name:
ReviewerRating=Rating (0 to 5):
ReviewerTitle=Title:
ReviewerReview=Enter your review:
Submit=Submit
ReturnLink=Return to products page

German Version :

;ui.de-DE.txt
;Banner User Control
BannerHeader=Produkt-Anzeige
;Product.aspx
Welcome=Willkommen zum Produkt wiederholt website. Diese Seite wird entworfen, um Produktinformation und Benutzerberichte
zur Verfügung zu stellen. Klicken Sie an eins der folgenden Produkte zu mehr Information.
ProductTableHeader=Produkte
CultureSelect=Wählen Sie eine Kultur vor
;ProductInfo
WriteReview=Möchten einen Bericht schreiben? Jetzt Ihre Wahrscheinlichkeit!
ReviewTableHeader=Produkt-Berichte
ReviewerName=Ihr Name:
ReviewerRating=Veranschlagen (0 bis 5):
ReviewerTitle=Titel:
ReviewerReview=Tragen Sie Ihren Bericht ein:
Submit=Reichen Sie ein
ReturnLink=Gehen Sie zur Produktseite zurück

It is interesting to note how fast the resource file can fill up. Every color, every string, every data point that is culture specific comes from the resource file. It becomes critical to keep a detailed illustration of the site and the resource fields used from page to page in order to reduce confusion. Maintenance is also critical. If you remove controls or UI from the application without deleting from the resource file, you will have unnecessary data in your resource file. Over time, this will make managing and maintaining the resource files slightly more difficult. So, as a best practice, always revisit the resource file when making changes.

Using Resource Files From ASP.NET

Once the resource files are compiled, they can be used by the .aspx application. Our application has two pages. The first page is used to present a list of products to the user. For the purposes of demonstration, we allow the user to select which locale they want displayed. In a production application, you would retrieve the locale information directly from the browser using the Request.UserLanguages property. In the HTTP stream, language information is sent to the web server and can be read via the Request object. The page's name is products.aspx(and can be found in the support material for this case study) and has the following presentation (shown with German locale):

The user is presented with a list of products, and can select one for more information. Even in this small example, notice how many data points are needed. If this site had any serious UI, the amount of resources will be quite large. If we wanted to improve on this design, we would also have made the languages German and English select box items resource items also. We left these hard coded as English strings.

The second page contains the product details and a list of reviews submitted for each product. The second page is productInfo.aspx :

Here, resources are needed for the banner and the product reviews ("Produkt-Berichte") labels. The rest of the text is retrieved from the database via the component layer. The entire code for these two pages is available via download. We want to focus on how to retrieve the resources from the recently compiled .resources files.

Importing Namespaces

There are a few key namespaces that should be imported into the application. Here are the namespaces relevant to internationalization:

using System.Threading;
using System.Globalization;
using ProductRetrieval;
//System.Resources imports namespace for ResourceManager
using System.Resources;

System.Threading is used to get access to the current thread. The current thread holds the locale information the runtime uses for the request. For numbers, dates, currencies, and whenever the culture is needed, the runtime uses the current threads culture by default. System.Globalization contains the CultureInfo object. CultureInfo represents the actual culture used by the thread. It specifies the formatting rules, calendaring rules, and culture-specific data for the culture. ProductRetrieval is the namespace for our component. System.Resources is the namespace used to do the actual resource lookup.

We have a lot of new material here. Let's discuss each namespace a bit further by looking at the code that makes internationalization a reality in our application. Let's look at the code in productInfo.aspx that is responsible for working with these namespaces.

private void Page_Load(object sender, System.EventArgs e)
{
//Set culture
Thread.CurrentThread.CurrentCulture = CultureInfo.CreateSpecificCulture(Session["selectedLocale"].ToString());
Thread.CurrentThread.CurrentUICulture = Thread.CurrentThread.CurrentCulture;

//Retrieve product information to populate controls
Retrieve r = new Retrieve(Session["selectedLocale"].ToString());

//Set product ID
if(!IsPostBack)
Session["prodID"] = Request.QueryString["prodID"];
else
{
//Record payment
r.AddReview(Int32.Parse(Session["prodID"].ToString()), NameText.Text, Int32.Parse(RatingText.Text),
TitleText.Text, ReviewText.Text);

}
//Retrieve product details into UI labels
SqlDataReader dr = r.GetProductDetails(Int32.Parse(Session["prodID"].ToString()));
dr.Read();
ProductName.Text = dr["ProductName"].ToString();
Description.Text = dr["Description"].ToString();
EditorComments.Text = dr["EditorComments"].ToString();
Price.Text = Double.Parse(dr["Price"].ToString()).ToString("c", Thread.CurrentThread.CurrentCulture);

//Bind reviews
SqlDataReader drReviews = r.GetReviews(Int32.Parse(Session["prodID"].ToString()));
ReviewList.DataSource = drReviews;
ReviewList.DataBind();

//Set international labels using resources in app's main directory)

ResourceManager rm =
ResourceManager.CreateFileBasedResourceManager
("ui", Server.MapPath(".")
+ System.IO.Path.DirectorySeparatorChar, null);
WriteReview.Text = rm.GetString("WriteReview");
ReviewTableHeader.Text = rm.GetString("ReviewTableHeader");
ReviewerName.Text = rm.GetString("ReviewerName");
ReviewerRating.Text = rm.GetString("ReviewerRating");
ReviewerTitle.Text = rm.GetString("ReviewerTitle");
ReviewerReview.Text = rm.GetString("ReviewerReview");
ReturnLink.Text = rm.GetString("ReturnLink");
Submit.Text = rm.GetString("Submit");
}

Let's examine the code in more detail.

//Set culture
Thread.CurrentThread.CurrentCulture = CultureInfo.CreateSpecificCulture(Session["selectedLocale"].ToString());
Thread.CurrentThread.CurrentUICulture = Thread.CurrentThread.CurrentCulture;

Since each request is stateless, each time the user enters our site, we need to set the current thread locale. In products.aspx, the user selected a locale. We saved that locale string (en-US or de-DE) into session. The CurrentThread object's CurrentCulture and CurrentUICulture are set to the proper locale. The CurrentCulture determines the culture settings for formatting, date, and number presentation. The CurrentUICulture represents the current user interface culture. Setting them both ensures that the base class libraries will format to the specific culture when appropriate.

//Retrieve product information to populate controls
Retrieve r = new Retrieve(Session["selectedLocale"].ToString());

Here we create an instance of our component and prepare for data retrieval.

ResourceManager rm =
ResourceManager.CreateFileBasedResourceManager ("ui", Server.MapPath(".")
+ System.IO.Path.DirectorySeparatorChar, null);
WriteReview.Text = rm.GetString("WriteReview");
ReviewTableHeader.Text = rm.GetString("ReviewTableHeader");
ReviewerName.Text = rm.GetString("ReviewerName");
ReviewerRating.Text = rm.GetString("ReviewerRating");
ReviewerTitle.Text = rm.GetString("ReviewerTitle");
ReviewerReview.Text = rm.GetString("ReviewerReview");
ReturnLink.Text = rm.GetString("ReturnLink");
Submit.Text = rm.GetString("Submit");

Here, we create an instance of the ResourceManager object. ResourceManager is responsible for retrieving the appropriate set of resources from the .resources files. It uses the current locale and retrieves the resources from the file matching the current locale. (Remember the specific naming conventions used for resource files? The ResourceManager needs the files to have the proper naming conventions.) The .resources in this application are maintained in the root directory where the .aspx pages reside.

The ResourceManager is very easy to use. The process of determining what file to access, how to parse the file, and what strings to retrieve is handled by this class. All you need to do is set the locale and the ResourceManager takes care of making sure the correct resource file is used. As you can imagine, keeping track of what resources are stored in the files is much more time consuming than creating the code to create the ResourceManager.

Limitations of resource files

Resource files are nice because they allow us a quick way to retrieve culture specific data fast. However they are not the most ideal case in the ASP.NET internationalized scenario. Two useful features introduced in ASP.NET are xcopy deployment and assembly versioning.

Resource files are just that, files. When a file is in use, the operating system locks the file. If you are in the process of deploying changes to resource files and the file is locked, the xcopy command will error. Assembly versioning allows us to host multiple versions of applications side by side. Applications can be configured to use only a particular version of an assembly. When new versions are created, they can be linked to updated assemblies. Both copies of the application can be executed simultaneously without conflict. The benefits of assembly versioning are very well documented. Unfortunately resource files cannot be versioned alone. In order to be versioned, they need to be compiled into assemblies. Assemblies containing only resources are called satellite assemblies in .NET.

Satellite Assemblies

Satellite assemblies contain no code, only resources. Satellite assemblies are the preferred way to deploy resource files. You retain the benefits of resource files and obtain the benefits of assemblies, such as versioning. In order to create satellite assemblies, there is an extra compilation step to convert .resources files into .dll assemblies.

The assembly linker tool, al.exe, is used to convert .resources into assemblies. The following batch script was created with this application to create the resource files from the .txt files and create the .dll files from the .resources. The batch file is called deploy.bat and contains the following console commands:

resgen ui.en-US.txt
resgen ui.de-DE.txt
resgen ui.txt
al /t:library /embed:ui.resources /out:bin/ui.dll
al /t:library /embed:ui.en-US.resources /culture:en-US /template:bin/ui.dll /out:bin/en-US/ui.resources.dll
al /t:library /embed:ui.de-DE.resources /culture:de-DE /template:bin/ui.dll /out:bin/de-DE/ui.resources.dll

The resgen lines we saw previously simply create the .resources files. Once the resources are created, the al.exe tool is responsible for making the assemblies and copying them into the appropriate directory. al.exe generates the .dll manifest from the resource files and embeds the resources into the assembly. The /embed switch copies the contents of the resources file into the assembly. The /culture switch tells the assembly loader what culture to associate the final assembly with. The /template switch tells al.exe to inherit the metadata from the template assembly. In this case, the template is the fallback resource.

The /out switch is rather interesting. Notice that each .dll file is named the same (ui.resources.dll). In general, the assemblies used as satellite assemblies must be the same name and put into a directory that matches their culture. Just like resources had required filenames, so do assemblies. With resource files, they must be named according to their culture and installed into the same directory. Assemblies must be named the same and installed into a directory according to their culture. To use locally in the application (as opposed to globally, which we'll touch on shortly), we need to install the assemblies under the ASP.NET application's \bin directory.

Here is what the \bin directory looks like when the assemblies are deployed. At the root of the \bin directory, ui.dll, the fallback resource, is installed. The fallback resource is culture-independent and does not need a unique directory. The en-US and de-DE cultures both contain a ui.resources.dll as well. When the ResourceManager class needs to use resources, it first checks for the existence of the appropriate culture's assembly. If the assembly cannot be located, the fallback ui.resources.dll must be used. Note that the assembly filename must end in .resouces.dll. The standard convention for assembly filenames is:

<basename>.resources.dll

Here is an image of the \bin folder's contents. The i18l.dll assembly is the assembly created for the ASP.NET project. ProductRetrieval.dll is the component we developed earlier.

Using Satellite Assemblies

When we retrieved resources from resource files, we used ResourceManager.CreateFileBasedResourceManager. CreateFileBasedResourceManager tells the resource manager to search for resources in the specific directory (in our case the application's root, ".") instead of using the assembly manifest. When using satellite assemblies, the ResourceManager is passed an instance of our base (fallback) resource assembly. The ResouceManager is using that assembly as its base for locating the culture specific resources. Here is the Page_Load function from products.aspx. The assembly ui.resources is passed to the ResourceManager constructor along with the base name of the resource files. With those two parameters, the ResourceManager has enough information to retrieve resources for any culture implemented.

private void Page_Load(object sender, System.EventArgs e)
{
//Set locale
Session["selectedLocale"] = Locale.SelectedItem.Value;
Thread.CurrentThread.CurrentCulture = CultureInfo.CreateSpecificCulture(Session["selectedLocale"].ToString());
Thread.CurrentThread.CurrentUICulture = Thread.CurrentThread.CurrentCulture;

//Bind reviews
Retrieve r = new Retrieve(Session["selectedLocale"].ToString());
ProductList.DataSource = r.GetProducts();
ProductList.DataBind();

//Load resources
System.Reflection.Assembly a = System.Reflection.Assembly.Load("ui.resources");
ResourceManager rm = new ResourceManager("ui", a);

CultureSelect.Text = rm.GetString("CultureSelect");
ProductTableHeader.Text = rm.GetString("ProductTableHeader");
Welcome.Text = rm.GetString("Welcome");
}

Global Assemblies

Satellite assemblies can be made global to the machine by compiling the assembly (using al.exe) with a strong name. The sn.exe utility is used to create a unique keyfile that can be applied to the assembly. Once compiled, use the gacutil.exe utility to install the files into the GAC. The makes the resource files global to all applications running on the current machine. It is usually sufficient to copy directories into the bin directory; however the GAC gives assemblies additional advantages, such as versioning.

The Sample Application

Since the application contains a database, middle tier, and UI, running the sample application takes a little bit of preparation and configuration. IIS, .NET, Visual Studio (optionally), and SQL Server are needed. Complete the following steps to install the code:

1. Download the application source

2. Extract the .zip file to any location. We suggest a subdirectory under the IIS WWWRoot directory.

3. Create a virtual directory named i18l in IIS pointing to the i18l directory under your root.

4. Launch the i18l.sln file within Visual Studio.

Points of interest will be:

Products.aspx - User interface for the products page.

Products.aspx.cs - Resource retrieval for products.aspx

ProductInfo.aspx - Product details page

ProductInfo.aspx - Resource retrieval and UI for an individual products page with reviews

ProductInfo.aspx.cs - Resource Retrieval and Page_Load for individual products page

Deploy.bat - build script for resource files

Ui.txt - Fallback resource file

Ui.en-US.txt - en-US resource file

Ui.de-DE.txt - de-DE resource file

ProductsSQL2000.bak - database backup file. You should restore this as the 'Products' database in your instance of SQL 2000.

Case Study Review

Internationalization is an important and often underestimated design topic. Since internationalization affects the heart of an application, it cannot be easily added to an application without initial design planning. If designed properly, the effort of adding another language to an existing application can be drastically reduced.

At the database level, internationalized applications determine what fields of data are dependent on culture and which are not. Data dependent on culture is separated from culture neutral data. In the case study example, the product description and editors comments were deemed culture specific and separated out from the Products table. This allowed us to create a product detail record for each culture.

The component layer of the application is largely culture neutral. The case study component was made aware of the culture at the constructor level. The rest of the interface remained similar to a non-internationalized application. Because of the stateless nature of the web, each request from a user can potentially come over a different thread. Each request must set the culture, which means we want to generally avoid storing state at the component level.

The user interface is where we begin to see internationalization come to life. Resource files and satellite assemblies provide access to resource information and the ResourceManager handles all the work of retrieving the appropriate resources for the application to use.

With the introduction of .NET, many of the common tasks have been absorbed into the Common Language Runtime and the Base Class Libraries. For example, assembly resolution, resource file location and retrieval, data formatting, and culture are natively built into the framework. Knowing how to use the tools to create, compile, and distribute resources represent the extent of the learning curve. The vast majority of the time spent in internationalizing code involves determining resource names and the actual translation involved with the user interface's data points.

Limitations or Future Work

The resource file presented in this case study was all string, name=value pair, based. .NET also offers another form of resource file called a .resx resource (.resx is the file extension). Resx resources are similar to those presented in this case study, however they are capable of holding more complex data. For example, the .resx file, which is xml based and human readable, can contain binary data or serialized objects. The default resource creator in Visual Studio.NET creates resx files by default. For more information on .resx files, refer to the .NET SDK.

The System.Resources namespace provides the developer programmatic access to create resource files. The System.Resources.ResourceWriter class can be used to dynamically create resource files. This class would come in handy if you want to create an application the language translators can use to create resource files. It would also come in handy if you have a large system and would like to automate resource file creation.

The application presented did not take security, performance, or proper logical code design into consideration. Many improvements can be made to this code. For example, the resource manager class could be created once and stored in session if desired, stored procedures could be used instead of inline SQL, and the default culture could be determined by the browser, not requiring the user to select the culture manually.

This case study was presented to a technical audience and thus was focused around .NET's ability to assist the programmer with internationalization. Another aspect of internationalization that is more cultural in nature is the process of translation. For example, the color red is a sign of rage in the United States, but could be pleasing to foreign cultures. Slogans or text could be misinterpreted and become derogatory or offensive to another culture. The slogans and text in this application were converted to German by Babelfish provided by AltaVista. Babelfish can be found at http://babelfish.altavista.com/?WROXEMPTOKEN=618658Zezi9SpDFSZgQps5Xv6u.

If you want to learn more about Internationalization, why not take a look at the new C# Text Manipulation Handbook from Wrox?

The .NET Framework has a variety of string manipulation features, and incorrect or ill-considered use of these methods could drastically increase your memory and processor time consumption. This book teaches you how to make use of regular expressions to match and replace text, and how to manipulate text using the various string matching, manipulation, and replacement classes. Issues such as Localization and data conversion will also be investigated.

For interviews with authors and editors, sample chapters, and news on the other C# handbooks, please see our Handbooks page.


Network Programming in C#

Network Programming in C#
http://www.csharptoday.com/content.asp?id=1994
Article Rating: 4.5
Published on November 18, 2002
Abstract
This case study by Irfan Rizvi focuses on network programming in C# using sockets. Sockets provide a medium of communication between two applications, whether those applications reside on the same computer, two different computers connected to each other on local network, or two computers connected to the internet.
Case Study

This case study focuses on network programming in C# using sockets. Sockets provide a medium of communication between two applications, whether those applications reside on the same computer, two different computers connected to each other on local network, or two computers connected to the internet.

A Brief Introduction To Networks

There are two main protocols that are used in network related applications; both have their own advantages and disadvantages. TCP (Transmission Control Protocol) is a highly reliable connection-oriented protocol, but sends acknowledgements for each request and response and therefore is more time consuming. TCP is the most widely used protocol because of its reliability; it is the standard protocol of the Internet.

The other protocol is UDP (User Datagram Protocol), which is a connectionless protocol - this means no acknowledgements are sent after request or responses, and hence it is faster then TCP but less reliable. UDP is used in applications in which time is the critical factor.

If you are new to sockets, you might be interesting to know that almost all servers, browsers, and almost everything in a TCP/IP network environment are using socket technology in the background. The theory behind TCP/IP network communication is that every machine on a network has a unique address (its IP address). You might be wondering that if each computer has but a single address, how can they differentiate which data is for which application when multiple applications are running on a machine? For example, if you have two browser windows open, each requesting information from a different website, then how does the computer know which browser to send what data to? The answer to this is addressing, which is done on two levels.

The first level is NSAP (Network Service Access Point), usually called IP addressing, which identifies the machine. When the data reaches the correct machine, we need to find the correct SAP (Service Access Point). This is the second level, which is level addressing. Service Access Points are also called port addresses, and these uniquely identify each application. Sockets are made up of a unique combination of an IP address and a port address, allowing the correct port address on the correct machine to be found.

The main aim of this case study is to demonstrate network programming in C# using sockets and TCP. .NET provides base classes for TCP server and clients in its System.Net namespace, and there are several classes available to deal with various requirements of network applications.

Introduction

In this case study we will be using a simple client/server application to demonstrate how socket programming works. This application uses TCP for the communication.

In this application, the server lets multiple clients to connect to it. Once the client is connected, the server can view all of its information like name, IP address, user name, the port at which it is connected, and the protocol it is using.

The server can also request the list of processes running on the client, and can view information related to each of these processes such as process name, start time, etc. Similarly, the server can also request the list of services that are available on the client, and information relating to these. These services are the same ones that you can manually access via Control Panel.

The server can perform several operations on the clients such as pausing, resuming or stopping a service. Through the Stop Client operation, the server can also disconnect the client - that is, the client will no longer be connected to the network.

By reading this case study, you will gain valuable knowledge regarding sockets, TCP, and the knowledge required to start programming client server applications.

Networks In More Detail: Theory and Technology

In this section we will take a detailed look at what networks are, how they work, and how the .NET architecture supports network programming.

OSI Reference Model

The OSI (Open Systems Interconnections) model is based on the proposal developed by the International Standards Organization (ISO) as a first step towards an international standardization of the protocols used. This model is called the Open System Interconnections Reference Model because it deals with connecting open systems, that is, systems that are open for communication with other systems.

The OSI Model has seven layers which are described briefly as follows:

Physical Layer: This layer handles the transmission of bits over the network. It provides the physical connection for the transmission of data among the network devices. The physical layer is responsible for making sure that data is read the same way on the destination device as it is sent from the source device. For example, the physical layer specifications include the amount of voltage on a cable, how a signal changes to signify a 1 or a 0 being sent, and in what order a signal is sent.

Data Link Layer: The main purpose of this layer is to provide a reliable method of transmitting data across the physical media. This layer breaks the input data into frames, transmits the frames sequentially, and processes the frames when they reach are received. It adds a header and trailer to the frames it creates - this allows the destination device to see when a frame begins or ends.

Network Layer: This layer is responsible for routing information from one network device to another; the network layer decides what path data will take if the destination device is located on another network. It accepts messages from the source, converts them into packets, and make sure that the packets are directed towards the destination. It is also responsible for deciding on the best route the packet should take through the network

Transport Layer: The transport layer is the fourth layer of the OSI model. It provides a transport service between the session layer and the network layer. This service takes information from the session layer and splits it up into packets if necessary, then passes this information to the network layer and checks to make sure the information arrived at the destination device successfully. The transport layer adds a header that includes error detection (to detect corruption of data), sequencing (as the message is split into several packets, this allows it to be re-assembled correctly), and flow control messages (which are used to make sure the destination is receiving information correctly).

Session Layer: This layer lets users establish a connection (called a "Session") between devices. Once the connection has been established, the session layer can manage the dialog. To establish a session, a user must provide the remote address they want to connect to. These addresses are not like the network addresses; they are intended for the user, and are easier to remember.

Presentation Layer: The presentation layer establishes the format in which data is exchanged. This layer is responsible for character set or numeric translations needed. It is also responsible for data compression to reduce the amount of data transmitted, as well as encryption.

Application Layer: The top layer of the OSI model is the application layer. This layer is the interface between the user's applications and the network. It allows the application the user sees to transfer files, send emails, and to do anything else it needs to on the network.

TCP/IP Protocol Suite

The Transmission Control Protocol/Internet Protocol (TCP/IP) suite, also known as the Internet Protocols, is a collection of industry-standard protocols. The TCP/IP suite is made up of many protocols, not just TCP and IP, and has a broad feature set due to its large number of open-standard protocols. Over the years, the individual components have evolved to handle almost any need a network user may have.

The original designs for TCP/IP (Transmission Control Protocol/ Internet Protocol) were stated long before the OSI model was developed; instead of the OSI's seven layers, TCP/IP has just four layers. The four layers can be loosely matched to the OSI model as follows:

Network Access Layer: This layer corresponds to the physical and data link layers of the OSI model.

Internet Layer: This layer roughly matches up with the network layer of the OSI model. Both of these layers are responsible for moving data to other devices on the network.

Host-to-Host Layer: This one is similar to the transport layer of the OSI model. The job of both of these layers is to communicate between peers on the network. As a result, almost all devices on a TCP/IP network are considered host, whether they are workstations, servers, or network-attached printers.

Process/Application Layer: This fourth layer does the same job as the top three layers of the OSI model, which is to provide network services.

TCP (Transmission Control Protocol)

In client-server applications, the server will provide some service such as processing database queries or sending out current stock prices. The client uses the service provided by the server, in these cases to either display database query results to the user or to make stock purchase recommendations to an investor. The communication that occurs between the client and the server must be reliable. That is, no data can be dropped, and it must arrive at the client side in the same order in which the server sent it.

TCP provides a reliable, point-to-point communication channel that client-server applications on the Internet use to communicate with each other. TCP was specifically designed to provide a reliable end-to-end byte stream over an unreliable internetwork. An internetwork differs from a single network because different parts may have widely different topologies, bandwidths, delays, packet sizes, and other parameters. TCP was designed to dynamically adapt to properties and to be robust in the face of many kinds of failures.

To communicate over TCP, a client program and a server program establish a connection to one another. Each program binds a socket to its end of the connection. To communicate, the client and the server each reads from and writes to the socket bound to the connection. The client and server send an IP packet to each other; this packet contains the TCP header that includes flow control, sequencing, and error checking (as touched upon earlier).

Each client is given the same port number of the server so that all the data being sent reaches the correct virtual connection. A port is similar to a mailbox. When data is delivered to a computer via the network, it must be sent to a process on the computer. A computer may have multiple processes running on it. Each service that expects data from the network will register itself onto a port number. For example, an internet web server uses port 80, while a mail server uses port 25. Inside the header of the network packet is a port number for which the data is intended, similar to a postal address for a letter.

Sequence Numbers: TCP uses a 32-bit sequence number that counts bytes in the data stream. Each TCP packet contains the starting sequence number of the data in that packet, and the sequence number (called the acknowledgment number) of the last byte received from the remote peer. Forward and reverse sequence numbers are completely independent, and each TCP peer must track both its own sequence numbering and the numbering being used by the remote peer.

Window Size and Buffering: Each endpoint of a TCP connection will have a buffer for storing data that is transmitted over the network, waiting until the application is ready to read the data. This lets network transfers take place while applications are busy with other processing, which improves overall performance. To avoid buffer overflow, TCP sets a Window Size field in each packet it transmits. This field contains the amount of data that may be transmitted into the buffer. If this number falls to zero, the remote TCP can send no more data. It must wait until buffer space becomes available and it receives a packet announcing a non-zero window size.

What is an IP Address?

In order for two applications to make a connection, they need to be able to identify each other. Computers on networks have a unique identifier called an IP address which is represented in dot-notation, an example would be 10.20.120.127. No two computers on the same network can have the same IP address. As we discussed earlier, the IP address consists of a network address and a host address. The network address represents the network to which this machine belongs, while the host address is the address of the machine itself. IP addresses can either be dynamic or static. There are some IP addresses that are used for special proposes, such as 127.0.0.1, which is known as the loop back address, etc.

What are Ports?

IP addresses are network addresses which identify a particular machine on a network. However, there are many applications running on a single machine, many of them communicating over the network, so each of these applications need an address to uniquely identify them. The solution to this problem is ports. Ports consist of an integer address which ranges from 1 to 65536, and each application on a machine has a port address. Servers usually have standard port addresses which are used to handle clients. For example, all web servers have a default port address of 80.

What are Sockets?

A socket is a unique combination of an IP address and a port number, identifying a destination that data can be sent to.

Normally, a server runs on a specific computer and has a socket that is bound to a specific port number. The server just waits, listening to the socket for a client to make a connection request. The client knows the hostname or the address of the machine on which the server is running, and the port number to which the server is connected. To make a connection request, the client tries to rendezvous with the server on the server's machine and port. If everything goes well, the server accepts the connection. Upon acceptance, the server gets a new socket, bound to a different port. It needs a new socket (and consequently a different port number) so that it can continue to listen to the original socket for connection requests while tending to the needs of the connected client.

On the client side, if the connection is accepted, a socket is successfully created and the client can use the socket to communicate with the server. Note that the socket on the client side is not bound to the port number used to rendezvous with the server. Rather, the client is assigned a port number local to the machine on which the client is running. The client and server can now communicate by writing to, or reading from, their sockets.

Socket Programming in C#

Network programming in Windows is possible using socket communication. A socket is like a handle to a file. You can use socket programming to have two applications communicate with each other. The applications are typically on different computers, but they can be on the same computer, or even the same process or thread of the same application.

For the two applications to talk to each using sockets, either on the same or different computers, one application is generally a server that keeps listening to the incoming requests and the other application acts as a client and makes the connection to the server application. The server application can either accept or reject the connection. If the server accepts the connection, a dialog can begin between the client and the server. Once the client is done with whatever it needs to do, it can close the connection with the server. Connections are expensive in the sense that servers allow finite connections to occur. This is because there are 65536 available ports on a machine, and another reason is that usually servers create a thread for each client, and as the number of client grows, large number of threads are running simultaneously, which will consume the server's resources.

During the time the client has an active connection, it can send data to the server and/or receive data. The complexity begins here. When either side (client or server) sends data, the other side is supposed to read the data. But how will the other side know when data has arrived? There are two options - either the application needs to poll for the data at regular intervals, or there needs to be some sort of mechanism that would enable applications to get notifications when new data arrives. Well, Windows is an event driven system, so the notification system is the obvious best choice.

The System.Net namespace

The .NET Framework provides a wide range of excellent features that developers can take advantage of while building networking applications. The System.Net classes are similar to Microsoft's WinInet API (used for internet protocol programming) in that they allow applications to get and send data using Internet protocols. The .NET classes are quite different in design and function however, the main difference being that the .NET classes are designed with an emphasis on standing up to a high-stress server environment and present an object based API model, while the WinInet was a functional API.

The System.Net namespace provides a simple programming interface for many of the protocols used on networks today. The WebRequest and WebResponse classes form the basis of what are called pluggable protocols, an implementation of network services that enables you to develop applications that use Internet resources without worrying about the specific details of the individual protocols.

The System.Net namespace contains a number of classes that can be used to communicate with resources on the Internet.

  • Socket programming can be done through System.Net.Sockets.Socket class.
  • Web resources can be downloaded by using the System.Net.WebRequest class.
  • Use System.Net.IPAddress to represent an IP address. You can use System.Net.Dns to resolve the IP address to a name ( System.Net.IPHostEntry ) or vice versa.
  • System.Net.WebProxy objects can be used to pass requests through a proxy server.

Since our case study only focuses on the System.Net.Sockets.Socket class, we will discuss only this class here.

System.Net.Sockets.Socket

The Socket class creates a managed version of an Internet transport service. Once the Socket is created, the Socket is bound to a specific endpoint through the Bind method, and the connection to that endpoint is established through the connect method. Data is sent to the Socket using the Send or SendTo methods, and data is read from the Socket using the Receive or RecieveFrom methods. After you are done with the Socket, the ShutDown method is used to disable the Socket, and the Close method to close the Socket.

The Socket class is used by the Microsoft .NET Framework to provide Internet connections to TcpClient, UdpClient, WebRequest, and the derived classes. We will only be discussing TcpListener and TcpClient here.

TcpListener: The TcpListener class builds upon the Socket class to provide TCP services at a higher level of abstraction. You can use a TcpListener to listen for connections from TCP clients. Application level protocols such as FTP and HTTP are built on the TcpListener class. You can create an instance of the TcpListener class by providing one of the following constructor argument lists: an IPEndPoint containing your local IP address and port number, your IP address and port number, or just your port number (in this case, the default network interface will be used).

TcpClient: The TcpClient class builds upon the Socket class to provide TCP services at a higher level of abstraction. TcpClient provides simple methods for connecting, sending, and receiving data over a network. You can connect with a remote host using one of the following two ways:

  • Create an instance of the TcpClient class, using either no parameters or using a local IPEndPoint, and then calling one of the three available TcpClient Connect methods.
  • Create an instance of the TcpClient class using the host name and port number of the device to which you intend to connect. This constructor will automatically establish a connection.

To send and receive data, a GetStream method is used that obtain a NetworkStream that can send and receive data on the underlying connected Socket. After utilizing the write and read methods available through the NetworkStream, the Close method is used to release all resources associated with the TcpClient.

I/O Stream

Streams in C# usually involve reading and writing bytes to and from the console, the filesystem, or the network. In C#, the stream paradigm is used more generally whenever a program needs to move or operate on a group of bytes.

To read and write to and from the console, C# provides a single class called System.IO.Stream. Instead of having two objects, one for reading and one for writing, a C# Stream object needs to be tested for its capabilities via the CanRead and the CanWrite properties. This class has methods to operate on one byte at a time, along with methods that operate on an array of bytes.

NetworkStream: This implements the standard .NET Framework stream mechanism to send and receive data through network sockets. NetworkStream supports both synchronous and asynchronous access to the network data stream. NetworkStream does not support random access to the network data stream.

Applying Our Knowledge

We have now taken a detailed look at the theory behind networks, and you should now have a good knowledge of why and how they are used. We will spend the rest of this case study using this theoretical knowledge to practically implement our client\server application, showing how all its features work as we go.

A Note About Loopback

Please note before we begin that the application that we will be creating in this case study is network related, and so it is better that client and server should run on separate machines - but this is not absolutely necessary. You can run both server and client on the same machine using the Loopback address. This address is a special IP number (127.0.0.1) that is designated for the software loopback interface of a machine. The loopback interface has no hardware associated with it, and it is not physically connected to a network.

An Overview Of The Final Application

The application we will be creating looks like the following:

From this screen (on the server), the administrator will be able to view all the clients that are connected to the server (in the left hand box). Selecting a client will cause the relevant details (such as IP Address, Username, the communication protocol being used, etc) to be shown in the top right hand collection of boxes. In the bottom right hand box, the administrator can view either the list of processes or services currently being run on that client. By using the buttons in the toolbar at the top, the administrator can manipulate these, for example pausing, resuming or stopping a process.

Now that we have seen what the final application will look like, and seen what it will actually do, let's get down to the hard work of putting together the code for this project. By closely following the following code and explanation, you will gain a good understanding of how sockets work in C#, what namespaces are important, and some of the issues involved, such as dealing with byte streams.

Component Design and Coding

Server Description

The above diagram represents the class architecture of the server. The main class is Server, which is the start point of this application and handles the User Interface; it also starts a thread of serverListener class which will run in the background and handle the incoming client connection requests. Whenever a new client arrives, this thread will make an object of ClientInfo class with the information about that client, and pass this object to Server where it will be added to its collection. The serverListenerArgs class acts as a wrapper class for transporting ClientInfo instances from the serverListener class to the Server class. The ProcessObject class and ServiceObject class are representatives of a Process and a Service.

Now we are going to discuss each class individually:

Class Description: Server

The Server class actually handles the User Interface via events. Most of the code is generated by Visual Studio itself - we will not describe this generated code, as it is out of the scope of this case study.

Attributes

Name | Type | Access Modifier | Description
alClientInformation | ArrayList | Private | Container for all clients
slListener | serverListener | Private | Listener object
GUI object | System.Windows.Form.* | Private | All other form objects

Methods

private void
            InitializeComponent()
{
this.slListener = new
            serverListener(1024);
this.slListener.OnClientSignal += new
            serverListener.ClientSignal(this.ClientConnectionHandler);
}

Although this method is Visual Studio generated, some of it is worth mentioning:

this.slListener = new serverListener(1024);

This statement starts the server's listener thread by making an instance of the serverListener class, while

this.slListener.OnClientSignal += new serverListener.ClientSignal(this.ClientConnectionHandler);

registers the ClientSignal event with the ClientConnectionHandler method.

private void ClientConnectionHandler (object sender , serverListenerArgs args)
{
ClientInfo ciClient = args.ClientInfo;
alClientInformation.Add (ciClient);

lstClients.Items.Add(ciClient.MachineName);
}

This is the event handler of a new client connection. Whenever a new client request arrives, the listener throws a notification about this request, with all the information about that client in an object of serverListenerArgs which is inherited from EventArgs class. One thing worth mentioning here in the architecture of event driven programming is that all the event handlers in .NET receive two and only two arguments; the first one is of object type which describes the sender from whom the notification is coming, and the second is the one through which all the information about that particular event is sent. This second argument must be an object of EventArgs class or any class inherited from EventArgs. We will describe the serverListenerArgs class later.

Getting back on track, this method or event handler will be initiated whenever a client arrives. It receives the information about that client (via the parameters) and adds it to its ClientInformation container. The client's machine name is added to the list box to show which client has been connected.

private void lstClients_SelectedIndexChanged(object sender, System.EventArgs args)
{
ClientInfo ciClient = (ClientInfo) alClientInformation [lstClients.SelectedIndex];

tbMachineName.Text = ciClient.MachineName;
tbUserName.Text = ciClient.UserName;
tbIPAddress.Text = ciClient.StrIPAddress;
tbPort.Text = ciClient.PortNumber.ToString() ;
tbProtocol.Text = ciClient.SConnectedSocket.ProtocolType.ToString();

if(rdbProcesses.Checked)
{
lstServices.Items.Clear();
IEnumerator en = ciClient.ProcessList.GetEnumerator();
while (en.MoveNext())
{
ProcessObject processObject = (ProcessObject) en.Current;
lstServices.Items.Add(processObject.StrProcessName);
}
}
else if (rdbServices.Checked)
{
lstServices.Items.Clear();
IEnumerator en = ciClient.ServiceList.GetEnumerator();

while (en.MoveNext())
{
ServiceObject serviceObject = (ServiceObject) en.Current;
lstServices.Items.Add (serviceObject.DisplayName);
}
}
}

This is the lstClients event handler. Whenever the server admin selects a client's machine name, this event handler is executed and it displays that machine's information.

private void lstServices_SelectedIndexChanged (object sender, System.EventArgs args)
{
ClientInfo ciClient = (ClientInfo) alClientInformation[lstClients.SelectedIndex];

if (rdbProcesses.Checked)
{

ProcessObject processObject = (ProcessObject) ciClient.ProcessList[lstServices.SelectedIndex];
tbServiceName.Text = processObject.StrProcessName;
tbServiceType.Text = processObject.ProcessStartTime;
}
else
{
ServiceObject serviceObject = (ServiceObject) ciClient.ServiceList [lstServices.SelectedIndex];
tbServiceName.Text = serviceObject.DisplayName;
tbServiceType.Text = serviceObject.ServiceType;
tbServiceStatus.Text = serviceObject.Status;
}
}

This event handler executes when the user selects a particular process or service from the list box. It get the relevant information and displays it on screen.

private void
            rdbProcesses_CheckedChanged(object sender, System.EventArgs
            e)
{
if (rdbProcesses.Checked)
{
lbllist.Text = "List
            of Process";
lblname.Text = "Process Name";
lbltype.Text =
            "Process Type";

tbServiceStatus.Visible =
            false;
lblstatus.Visible = false;

if
            (lstClients.SelectedIndex >= 0)
{
ClientInfo ciClient =
            (ClientInfo) alClientInformation
            [lstClients.SelectedIndex];

lstServices.Items.Clear();
IEnumerator
            en = ciClient.ProcessList.GetEnumerator();

while
            (en.MoveNext())
{
ProcessObject processObject =
            (ProcessObject)
            en.Current;
lstServices.Items.Add(processObject.StrProcessName);
}
}
}
}

This event handler switches the on-screen display from service mode into process mode. So, if the user selects the "process" radio button, information about the highlighted machine's process are then shown on the screen.

private void
            rdbServices_CheckedChanged(object sender, System.EventArgs
            e)
{
if (rdbServices.Checked)
{
lbllist.Text = "List of
            Service";
lblname.Text = "Service Name";
lbltype.Text =
            "Service Type";
tbServiceStatus.Visible =
            true;
lblstatus.Visible = true;
if (lstClients.SelectedIndex
            >= 0)
{
ClientInfo ciClient = (ClientInfo)
            alClientInformation[lstClients.SelectedIndex];

lstServices.Items.Clear();
IEnumerator
            en = ciClient.ServiceList.GetEnumerator();
while
            (en.MoveNext())
{
ServiceObject serviceObject =
            (ServiceObject)
            en.Current;
lstServices.Items.Add(serviceObject.DisplayName);
}
}
}
}

This event handler is the opposite of rdbProcesses_CheckedChanged, in that it switches the view from process mode into service mode, as oppose to switching from service to process.

private void tlbClicked(object sender,
            System.Windows.Forms.ToolBarButtonClickEventArgs args)
{
if
            (args.Button.Equals(this.tlbOptions.Buttons[0]))
{
MessageBox.Show("stop
            client");
}
else if
            (args.Button.Equals(this.tlbOptions.Buttons[2]))
{
MessageBox.Show("Pause
            Service");
}
else if
            (args.Button.Equals(this.tlbOptions.Buttons[3]))
{
MessageBox.Show("Resume
            Service");
}
else if
            (args.Button.Equals(this.tlbOptions.Buttons[4]))
{
if
            (selectedClient != null && selectedService !=
            null)
{
bool result = selectedClient.stopService
            selectedService.DisplayName);

if
            (result)
{
selectedService.Status="Stopped";
lstServices_SelectedIndexChanged(sender,
            e);
}
else
MessageBox.Show("Operation Failed","Error
            Message");
}
else
{
MessageBox.Show("No Service
            selected","Alert");
}
}
else if
            (e.Button.Equals(this.tlbOptions.Buttons[6]))
{
MessageBox.Show("Stop
            Process");
}
}

The above method is the event handler for the toolbar. We are only going to discuss one operation here ( stopService ), as this will be enough to give you a good understanding of what is happening.

if (selectedClient != null && selectedService != null)
{
bool result = selectedClient.stopService(selectedService.DisplayName);
if (result)
{
selectedService.Status="Stopped";
lstServices_SelectedIndexChanged(sender, e);
}
else
MessageBox.Show("Operation Failed","Error Message");
}
else
{
MessageBox.Show("No Service selected","Alert");
}
}

Whenever the stopService button is pressed, the code above is executed. It calls the stopService method of the selected client and passes the selected service's name to it. What this stopService method actually does will be discussed in more detail later, for now we are just assuming that if the operation is performed successfully then true is returned, else false is returned.

Class Description: ServerListener

Attributes

Name | Type | Access Modifier | Description
ClientSignal | delegate | Public | Delegate for the new client connection event
OnClientSignal | <<Event>> ClientSignal | Public | Represents the event shown when a new client arrives
Server_Socket | TcpListener | Private | Listens for connections from TCP network clients
iPort | Int | Private | Specifies the port where the Listener is listening
bRunning | Bool | Private | Its used for starting and stopping the Listener

Methods

public serverListener(int
            prt)
{
try
{
iPort=prt;
Server_Socket=new
            TcpListener(iPort);
Server_Socket.Start();

Thread thr=new
            Thread(new
            ThreadStart(server_listen));
thr.Start();
}
catch(Exception
            e)
{
throw new Exception ("Error Occurred in
            Listener");
}

}

This initiates the Listener object and starts it through

Server_Socket.Start();

The difference between initializing and starting the TcpListener's object is that when initializing, the port is initialized/opened, but no operation can be performed at that moment. Actually the TcpListener's object starts working when its start method is called. The start method acts as an entry point for its object. This method also starts a Listener thread which continuously listens for any new clients while working in the background.

public void
            server_listen()
{
while(bRunning)
{
try
{
Socket
            sock=Server_Socket.AcceptSocket ();
ClientInfo clt=new
            ClientInfo(sock);
OnClientSignal(this , new
            serverListenerArgs(clt));
}
catch(Exception e)
{
throw
            new Exception("Error Occurred while listening for clients ");
}
}
}

This method accepts requests from the clients which arise as objects of the socket class.

Socket sock=Server_Socket.AcceptSocket();

In the statement above, the thread goes into a wait mode. This means that this statement will not get completely executed until a client socket arrives. When a new client does arrive, its socket is received into a local variable named sock, and then an object of ClientInfo class is instantiated by passing this socket object. After that, an event is thrown to notify the server class that a new client has arrived.

OnClientSignal(this , new serverListenerArgs(clt));

You may be wondering why we created a serverListenerArgs object while we had a ClientInfo object that we could have passed? The answer to this question is that when throwing an event, the first parameter represents the object which is throwing this event (in this case that would be the serverListener class), and the second parameter must be an object of EventArgs (or any class inherited from the EventArgs ). So to achieve our requirement we created another class called serverListenerArgs and inherited it from EventArgs. We'll explain the serverListenerArgs class later.

public void
            close()
{
bRunning=false;
Server_Socket.Stop();
}

This method is simply used to close the Listener thread.

Class Description: ClientInfo

Attributes

Name | Type | Access Modifier | Description
sConnectedSocket | Socket | Private | This object creates a managed version of an Internet transport service
ClientThread | Thread | Private | it is used for each of the client that arrives
strMachineName | string | Private | The name of the machine of each client
strIPAddress | string | Private | The IP address of each client
iPortNumber | int | Private | Its used for starting and stopping the Listener
strUserName | String | Private | The username of each client
alServiceList | ArrayList | Private |
alProcessList | ArrayList | Private |
bRunning | Bool | Private | Its used for starting and stopping the Listener

Methods

public ClientInfo(Socket
            sock)
{
try
{
sConnectedSocket=sock;

string
            temp = sock.RemoteEndPoint.ToString();
StrIPAddress =
            temp.Substring(0,temp.IndexOf(':'));
iPortNumber =
            Convert.ToInt32(temp.Substring(temp.IndexOf(':')+1));

getRemoteMachineName();
getRemoteUserName();

alServiceList = new ArrayList();
alProcessList = new
            ArrayList();

getProcessList();
getServiceList();

}
catch(Exception
            e)
{
MessageBox.Show(e.Message,"Information");
}
}

This constructor gets all the information from the remote client which this object is handling. For example

getRemoteMachineName();

This method gets the machine name from the client. This constructor also initiates a thread to listen to all the requests that this particular client sends. This means that each and every client which is connected to the server has its own thread which is monitoring its activities. This multithreaded approach to handle clients is used so that we can handle all the clients concurrently.

public void
            run_Thread()
{
try
{
Byte[] bReceive = new
            Byte[1024];

while(bRunning)
{
int i =
            sConnectedSocket.Receive( bReceive);
string
            sBuffer=Encoding.ASCII.GetString(bReceive);
}
}
catch(Exception
            e)
{
MessageBox.Show(e.Message);
}

finally
{
sConnectedSocket.Close();
}
}

This is the entry point for a particular client's thread, and it continuously listens for data coming from the client side.

int i = sConnectedSocket.Receive( bReceive);

Notice that every socket has the ability to send and receive data, to or from the machine which it is representing.

The socket class provides methods to achieve this. The Receive method used in the code above waits until the data arrives and then moves the data into a byte array. There are several overloads present for the Receive method. The integer value that it is returning is the number of bytes that it has received.

string sBuffer=Encoding.ASCII.GetString(bReceive);

We use the above statement to convert the byte stream into a string.

public void
            getRemoteMachineName()
{
string strMachineName = "<command
            type=\"Machine_Name\">";
strMachineName +=
            "</command>$";

byte[] bBuffer = new
            byte[1024];
bBuffer =
            Encoding.ASCII.GetBytes(strMachineName);
sConnectedSocket.Send(bBuffer);

sConnectedSocket.Receive
            (bBuffer);
string s =
            Encoding.ASCII.GetString(bBuffer);

while(sConnectedSocket.Available
            >0 )
{
sConnectedSocket.Receive (bBuffer);
s +=
            Encoding.ASCII.GetString(bBuffer);
}

string
            strResultXML = s.Substring(0,s.IndexOf('$'));

XmlDocument
            doc = new
            XmlDocument();
doc.LoadXml(strResultXML);

XmlNode node
            =doc.FirstChild;
if (node.Attributes.GetNamedItem("type").Value
            == "Machine_Name")
{
XmlNode childNode =
            node.FirstChild;
this.strMachineName =
            childNode.Attributes.GetNamedItem("name").Value;
}
else
throw new Exception("Wrong Packet Recieved");

}

This is the first method where actual client server communication is happening. There are a few things you will need to understand before we go into more detail; in socket communication, stream of bytes are passed because the TCP protocol is unaware of the type of data that is passed. To TCP, any type of data, whether it is an object or any predefined type (such as int, string, etc), is simply a collection of bytes. This is the reason why all the send and receive methods in socket programming uses byte arrays as a buffer to store or retrieve data. Therefore we have used string objects to send and receive, as they can easily be converted into byte arrays. The best medium to produce self-describing data in string format is XML.

In recent years, XML has become the standard way to describe data. Although we could have just passed simple strings and still have achieved all the required results, using XML provides the following benefits:

  • Easy interpretation of data using the System.xml classes.
  • Data is self describing, which means that anyone can easily understand what a particular string is representing. Without the use of XML, only the author of the code would understand the particular data.

Now I that we have explained why XML will be used in our application, let's go back to the getRemoteMachineName method.

string strMachineName = "<command type=\"Machine_Name\">";
strMachineName += "</command>$";

This method is preparing a command, in the form of an XML string, to get a machine name. The character $ acts as an end-of-data identifier. Here the string which we have sent acts as a command for the client, ordering it to reply with its machine name.

bBuffer = Encoding.ASCII.GetBytes(strMachineName);
sConnectedSocket.Send(bBuffer);

These statements convert the XML string into byte form. The send method of the socket object sends the data to the application which this socket is representing.

sConnectedSocket.Receive (bBuffer);
string s = Encoding.ASCII.GetString(bBuffer);

The logic here is that whenever the server sends a command to a client, it expects a response. The response is either the data that was requested, or a notification that the operation requested was performed or not. In this case, the server has sent a command for the machine name - it is now waiting for the client to respond to the request and send back its machine name. How the client responds to this command will be explained in the client code description. The receive method receives the clients response.

while(sConnectedSocket.Available >0 )
{
sConnectedSocket.Receive (bBuffer);
s += Encoding.ASCII.GetString(bBuffer);
}

The limitation with the receive method is that it doesn't receive all the data in one go; instead, it obtains it in chunks. Usually the receive method receives 40 bytes at a time. To overcome this problem, we continue receiving this data until it has all arrived. The Available property of the socket object tells us the amount of data that has yet to be received, in other words, the number of bytes available on the stream.

string strResultXML = s.Substring(0,s.IndexOf('$'));

The line above removes the end-of-string identifier from the string received.

XmlDocument doc = new
            XmlDocument();
doc.LoadXml(strResultXML);

Now we come to the response interpretation part. The string received from the client is also in XML form because of the reasons described earlier. The lines above make an XMLDocument object and load the client's response into it. As we know the format of the response, we can easily parse it:

XmlNode node =doc.FirstChild;
if (node.Attributes.GetNamedItem("type").Value == "Machine_Name")
{
XmlNode childNode = node.FirstChild;
this.strMachineName = childNode.Attributes.GetNamedItem("name").Value;
}
else
throw new Exception("Wrong Packet Recieved");

In the above code, we traverse to the first node and check the type. If it is valid, then it moves to its child node and gets its value, which will be the name of the machine. Otherwise, an exception is thrown.

The description of the getRemoteMachineName method explains in detail how we have achieved client server communication. There are several other ways of doing what we have done, but this approach is much more error-safe and close to standards.

public void
            getRemoteUserName()
{
string strUserName = "<command
            type=\"User_Name\">";
strUserName +=
            "</command>$";

byte[] bBuffer = new
            byte[1024];
bBuffer =
            Encoding.ASCII.GetBytes(strUserName);
sConnectedSocket.Send(bBuffer);

sConnectedSocket.Receive (bBuffer);
string s =
            Encoding.ASCII.GetString(bBuffer);

while(sConnectedSocket.Available
            >0 )
{
sConnectedSocket.Receive (bBuffer);
s +=
            Encoding.ASCII.GetString(bBuffer);
}

string strResultXML =
            s.Substring(0,s.IndexOf('$'));

XmlDocument doc = new
            XmlDocument();
doc.LoadXml(strResultXML);

XmlNode node
            =doc.FirstChild;
if (node.Attributes.GetNamedItem("type").Value
            == "User_Name")
{
XmlNode childNode =
            node.FirstChild;
this.strUserName =
            childNode.Attributes.GetNamedItem("name").Value;
}
else
throw new Exception("Wrong Packet Recieved");
}

The working of this getRemoteUserName method is very similar to that of getRemoteMachineName, so we will not go into its details. This method gets the name of the user who is logged into the client's machine.

public void getProcessList()
{
string
            strProcesslist = "<command
            type=\"Process_List\">";
strProcessList +=
            "</command>$";

byte[] bBuffer = new
            byte[3000];
bBuffer =
            Encoding.ASCII.GetBytes(strProcessList);
sConnectedSocket.Send(bBuffer);

sConnectedSocket.Receive (bBuffer);
string s =
            Encoding.ASCII.GetString(bBuffer);

while(sConnectedSocket.Available
            >0 )
{
sConnectedSocket.Receive (bBuffer);
s +=
            Encoding.ASCII.GetString(bBuffer);
}

string strResultXML =
            s.Substring(0,s.IndexOf('$'));

XmlDocument doc = new
            XmlDocument();
doc.LoadXml(strResultXML);

XmlNode node
            =doc.FirstChild;
if (node.Attributes.GetNamedItem("type").Value
            == "Process_List")
{
XmlNodeList nodeList =
            node.ChildNodes;
IEnumerator en =
            nodeList.GetEnumerator();

while
            (en.MoveNext())
{

XmlNode childNode = (XmlNode)
            en.Current;
string pname=
            childNode.Attributes.GetNamedItem("name").Value;
string ptime =
            childNode.Attributes.GetNamedItem("time").Value;
ProcessObject
            processObject = new ProcessObject(pname,ptime);
strProcessList.Add(processObject);
}
}
else
throw
            new Exception("Wrong Packet Recieved");
}

This getProcessList method obtains the list of processes of the particular client. The command and receive part of this method is the same as in the previous methods, but it differs in the way the response is interpreted. In the other two methods discussed above, the result was a single string such as machine name or username, but in this case the response is a list of processes with several properties. We have defined a class named ProcessObject to represent a process which will be discussed later.

XmlNodeList nodeList = node.ChildNodes;

The above statement gets a list of all the child nodes of the resulting parent node. Each child node describes a single process.

      IEnumerator
            en = nodeList.GetEnumerator();
while
            (en.MoveNext())
{
XmlNode childNode = (XmlNode) en.Current;
string pname=
            childNode.Attributes.GetNamedItem("name").Value;

string ptime
            = childNode.Attributes.GetNamedItem("time").Value;
ProcessObject
            processObject = new ProcessObject(pname,ptime);
strProcessList.Add(processObject);
}

We traverse through each child node and obtain two values. The first one is the name of process and the second one is the time at which the process was started. Once we get both these values we create an object of ProcessObject :

new ProcessObject(pname,ptime);

We then add this newly created abject to the container strProcessList.

public void getServiceList()
{
string
            strServiceList = "<command
            type=\"Service_List\">";
strServiceList +=
            "</command>$";

byte[] bBuffer = new
            byte[3000];
bBuffer =
            Encoding.ASCII.GetBytes(strServiceList);
sConnectedSocket.Send(bBuffer);

sConnectedSocket.Receive (bBuffer);
string s =
            Encoding.ASCII.GetString(bBuffer);

while(sConnectedSocket.Available
            >0 )
{
sConnectedSocket.Receive (bBuffer);
s +=
            Encoding.ASCII.GetString(bBuffer);
}

string strResultXML =
            s.Substring(0,s.IndexOf('$'));

XmlDocument doc = new
            XmlDocument();
doc.LoadXml(strResultXML);

XmlNode node
            =doc.FirstChild;
if (node.Attributes.GetNamedItem("type").Value
            == "Service_List")
{
XmlNodeList nodeList =
            node.ChildNodes;
IEnumerator en =
            nodeList.GetEnumerator();

while
            (en.MoveNext())
{
XmlNode childNode = (XmlNode) en.Current;
string sname=
            childNode.Attributes.GetNamedItem("name").Value;
string stype =
            childNode.Attributes.GetNamedItem("type").Value;
string sstatus =
            childNode.Attributes.GetNamedItem("status").Value;
ServiceObject
            serviceObject = new ServiceObject(sname,stype,sstatus);
strServiceList.Add(serviceObject);
}
}
else
throw
            new Exception("Wrong Packet Recieved");
}

This method is similar to getProcessList; it receives the list of all the windows services running on the client machine. Each service is represented by a ServiceObject class which will be discussed later.

public bool stopService(string
            servicename)
{
string strServiceString = "<command
            type=\"Stop_Service\">";
strServiceString += "<param
            name=\""+servicename+"\"></param>";
strServiceString +=
            "</command>$";

byte[] bBuffer = new
            byte[1024];
bBuffer =
            Encoding.ASCII.GetBytes(strServiceString);
sConnectedSocket.Send(bBuffer);

sConnectedSocket.Receive
            (bBuffer);
string s =
            Encoding.ASCII.GetString(bBuffer);

while(sConnectedSocket.Available
            >0 )
{
sConnectedSocket.Receive (bBuffer);
s +=
            Encoding.ASCII.GetString(bBuffer);
}

string strResultXML =
            s.Substring(0,s.IndexOf('$'));
XmlDocument doc = new
            XmlDocument();
doc.LoadXml(strResultXML);

XmlNode node
            =doc.FirstChild;
if (node.Attributes.GetNamedItem("type").Value
            == "Stop_Service")
{
XmlNode childNode =
            node.FirstChild;
string a =
            childNode.Attributes.GetNamedItem("result").Value;
if (a ==
            "Success")
return true;
else
return
            false;
}
else
{
throw new Exception("Wrong Packet
            Recieved");
return false;
}
}

This method is different from the other methods in the way that it is performed due to the request of a user. Whenever the user needs to stop any service that is running on its machine, it sends the request along with the service name to this method. The sending and receiving part of this method are similar to the previous ones.

XmlNode node =doc.FirstChild;
if (node.Attributes.GetNamedItem("type").Value == "Stop_Service")
{
XmlNode childNode = node.FirstChild;
string a = childNode.Attributes.GetNamedItem("result").Value;
if (a == "Success")
return true;
else
return false;
}
else
{
throw new Exception("Wrong Packet Recieved");
return false;
}

Since this method is not requesting any information, but is asking the client to perform an operation, the response will not be any information but just an acknowledgement which will tell the server whether the requested operation was successfully performed or not.

Class Description: serverListenerArgs

This class is inherited from the EventArgs class. The reason behind using this class is that in event driven programming, when an event is throw,n the parameters passed to the event handler must be through an object of EventArgs class or any of its derived classes. Hence the object of this class is instantiated whenever a listener thread receives a new client and it throws an OnClientSignal event. This class has only one attribute, a ClientInfo object. In short, serverListenerArgs provides a transportation medium for ClientInfo objects to be sent from listener to server class.

Attributes

Name | Type | Access Modifier | Description
ClientInfo | ClientInfo | Private | The client information object which will be passed to the server class.

Class Description: ProcessObject

This class just acts as a user defined type for representing processes. An object of the process class itself has many attributes and methods, but of two of them are of use to us. As we have discussed above, data transferring in Socket programming is restricted in that only byte streams can be sent, so we cannot send objects. What we have done is fetched the information from the Process class object (such as process name and process start time) and sent it through the stream. Since this information wasn't complete enough to make a new Process instance, we have made our own class which is in a sense representative of the process class.

Attributes

Name | Type | Access Modifier | Description
StrStrProcessName | String | Private | Specifies the name of that particular process
StrProcessStartTime | String | Private | Specifies the time the process started

Class Description: ServiceObject

Like the ProcessObject class, ServiceObject is a user defined type to represent Services.

Attributes

Name | Type | Access Modifier | Description
StrDisplayName | String | Private | Specifies the name of that particular service
strServiceType | String | Private | Specifies the type of the service.
strStatus | String | Private | Status of service (Stopped, Started, Paused, etc.)

Client Description

Class Description: netClient

Attributes

Name | Type | Access Modifier | Description
ClientSocket | TcpClient | Private | The main object that connects the client to the server
iServerPort | Int | Private | Specifies the port of the server where the client is to be connected
sServerIP | String | Private | Specifies the IP Address of the server
sMachineName | String | Private | The client's machine name
sUserName | String | Private | The name of the user who has logged onto the client's machine
NetworkStream | NetworkStream | Private | Provides the underlying stream of data for the network access.
Services | ServiceController [] | Private | List of ServiceController instances, each representing a single service available on the clients machine
Processes | Prosess[] | Private | List of processes running on the client's machine
bRunning | Bool | Private | It is used for starting and stopping the client

Methods

public netclient(string ipaddr, int prt)
{
iServerPort=prt;
sServerIP=ipaddr;

Services = new ServiceController[50];
refreshServiceList();

Processes = new Process[20];
refreshProcessList();

sMachineName = SystemInformation.ComputerName;
sUserName = SystemInformation.SUserName;

ClientSocket=new TcpClient();
ClientSocket.Connect(sServerIP,iServerPort);
Console.WriteLine("Client Connected to Server");

ClientSocket.ReceiveBufferSize = 3000;
ClientSocket.SendBufferSize = 3000;
NetworkStream = ClientSocket.GetStream();

if(NetworkStream.CanWrite && NetworkStream.CanRead)
{
try
{
while (bRunning)
{
Byte[] recBytes = new Byte[1024];
int a= NetworkStream.Read(recBytes,0,1024);
string sBuffer = Encoding.ASCII.GetString(recBytes);
sBuffer = sBuffer.Substring(0,sBuffer.IndexOf('$'));

XmlDocument doc = new XmlDocument();
doc.LoadXml(sBuffer);

XmlNode node = doc.FirstChild;
XmlNode childNode;

string sCommandType = node.Attributes.GetNamedItem("type").Value;

switch (sCommandType)
{
case "User_Name":
sendUserName();
break;
case "Machine_Name":

sendMachineName();
break;
case "Service_List":
SendServiceList();
break;
case "Process_List":

SendProcessList();
break;
case "Restart_Client":
restartClient();
break;
case "Shutdown_Client":
shutdownClient();
break;
case "Pause_Service":
childNode = node.FirstChild;

pauseService(childNode.Attributes.GetNamedItem("name").Value);
break;
case "Stop_Service":
childNode = node.FirstChild;

StopService(childNode.Attributes.GetNamedItem("name").Value);
break;
case "Resume_Service":
childNode = node.FirstChild;
resumeService(childNode.Attributes.GetNamedItem("name").Value);
break;
}
}
}
catch(Exception e)
{
Console.WriteLine(e.Message);
}
}
Console.WriteLine("Press any key to continue..");
Console.Read();
}

The netclient method comprises the main functionality of the client. It gathers all the client information, receives all the servers commands, perform actions in accordance to those commands, etc. Now we'll discuss this method in a modular fashion.

iServerPort=prt;
sServerIP=ipaddr;

Services = new ServiceController[50];
refreshServiceList();

Processes = new Process[20];
refreshProcessList();

sMachineName = SystemInformation.ComputerName;
sUserName = SystemInformation.SUserName;

The first thing this method does is to gather all the information about itself and the servers where it has to be connected. The functionality of refreshServiceList and refreshProcessList will be discussed later. SystemInformation is a class from the System.Windows.Forms namespace; it gives all the low level information about the computer. A detailed description of this class is unnecessary with respect to our case study. We have used this class to get the machine and user name.

       ClientSocket=new
            TcpClient();
ClientSocket.Connect(sServerIP,iServerPort);

The above lines are the main part of any client. The first line creates an instance of TcpClient's class. The connect method of this class has several overloads, but the one we have used takes two parameters. The first one is the IP Address of the server, and the second one is the port at which the server is listening for clients. These two parameters must be known before any connection request is sent. The connect method prepares the socket object of the client with information such as IP Address, port number at which the client application is running, the protocol type which is being used for the transmission, and some other information. It makes a packet of them, and sends it through the network to the server whose IP was specified in the parameters, and the application running on that server whose port number was specified.

ClientSocket.ReceiveBufferSize = 3000;
ClientSocket.SendBufferSize = 3000;

Once the client is connected to the server, a few things are done before actually sending or receiving data. The code above sets the size of the send and receive buffer of the client. Every TcpClient instance has two buffers associated with it, one where all the data arrives and the other from where all the data is sent. Different computers have different send and receive rates, and because of this, buffers are used. Buffer sizes are application dependent; large sized buffers are used for the applications in which the amount of data is large.

NetworkStream = ClientSocket.GetStream();

The GetStream method gets an instance of the NetworkStream class. Each TcpClient has its own stream, through which it can send and receive data. By providing separate streams for each TcpClient instance, developers have enforced a level of security, so that if two or more clients are running on same machine each of them get their own data, although all the data is coming from the same medium.

Byte[] recBytes = new Byte[1024];
int a= NetworkStream.Read(recBytes,0,1024);
string sBuffer = Encoding.ASCII.GetString(recBytes);
sBuffer = sBuffer.Substring(0,sBuffer.IndexOf('$'));

As the NetworkStream class is inherited from the Stream class, like all other NetworkStream instances it has the ability to read and write. The above code is where the client listens for commands from the server. The Read method used here waits until data arrives from serve. It has several overloads, but the one we have used has three parameters: the first one is the buffer in which data will be received, the second one is the offset telling it from which position in the byte array it has to start writing data, and the third is the number of bytes to be read. The last two statements of the code perform the same function as in the server i.e. of converting byte array intro string and removing the end-of-string character.

XmlDocument doc = new XmlDocument();
doc.LoadXml(sBuffer);

XmlNode node = doc.FirstChild;
XmlNode childNode;

string sCommandType = node.Attributes.GetNamedItem("type").Value

Once the data has arrived it has to be interpreted - again the same method is applied by parsing the XML string, and getting the type of the command. The last statement gets the type of the command sent.

switch (sCommandType)
{
case "User_Name":
sendUserName();
break;

case "Machine_Name":
sendMachineName();
break;

//Rest of the code
}

Once we have received the type of command, we take the appropriate action. On each type of command we are calling a method to perform the requested action and send a response back to the server.

public void sendMachineName()
{
string
            sMachineNameXML;
sMachineNameXML = "<result
            type=\"Machine_Name\">";
sMachineNameXML += "<params
            name=\"" + sMachineName + "\" />";
sMachineNameXML +=
            "</result>$";

byte[] bBuffer = new
            byte[1024];
bBuffer = Encoding.ASCII.GetBytes(sMachineNameXML);
NetworkStream.Write(bBuffer,0,machineNameXML.Length);
}

The above sendMachineName method is called whenever a request for the Client's machine name arrives. It prepares an XML string with the requested machine name in parameters, and sends this response back to the server. We are just going to take a brief look at the write method. Like the read method, write is also inherited from the stream class, and it also has several overloads available. The one we have used has three parameters: the first one is the reference to the buffer from which data has to be sent, the second one is the offset in the buffer, and the third is the number of bytes to be sent.

public void sendUserName()
{
string
            sUserNameXML ;
sUserNameXML = "<result
            type=\"User_Name\">";
sUserNameXML += "<params name=\"" +
            sUserName + "\" />";
sUserNameXML +=
            "</result>$";
byte[] bBuffer = new byte[1024];
bBuffer =
            Encoding.ASCII.GetBytes(sUserNameXML);
NetworkStream.Write(bBuffer,0,sUserNameXML.Length);
}

This sendUserName method is used to retrieve the user name from the client. Whenever a request for the Client's user name arrives, an XML string is generated that include the requested user name in parameters, and sends this response back to the server.

public void sendProcessList()
{
string
            sProcessXML = "";
sProcessXML += "<result
            type=\"Process_List\">";
IEnumerator en =
            Processes.GetEnumerator();
while (en.MoveNext())
{
Process
            process = (Process) en.Current;
sProcessXML += "<params
            name=\""+process.StrProcessName+"\"
            time=\""+process.StartTime.ToString()+"\"
            />";
}
sProcessXML += "</result>$";
byte[]
            bBuffer = new byte[3000];
bBuffer =
            Encoding.ASCII.GetBytes(sProcessXML);
NetworkStream.Write(bBuffer,0,sProcessXML.Length);

}

The sendProcessList method sends the list of the processes running on the client to server. This method differs to the other methods in the way the response is generated. Since in this method we have to generate a list of processes with their attributes, the XML generating code is a little different. Once a complete response is generated, it is sent back to the server.

public void SendServiceList()
{
string
            sServiceXML = "";
sServiceXML += "<result
            type=\"Service_List\">";
IEnumerator en =
            Services.GetEnumerator();
while
            (en.MoveNext())
{
ServiceController service =
            (ServiceController) en.Current;
sServiceXML += "<params
            name=\""+service.DisplayName+"\"
type=\""+service.ServiceType.ToString()+"\"
status=\""+service.Status.ToString()+"\"
            />";
}
sServiceXML += "</result>$";
byte[] bBuffer
            = new byte[3000];
bBuffer = Encoding.ASCII.GetBytes(sServiceXML);
NetworkStream.Write(bBuffer,0,sServiceXML.Length);
}

The sendServiceList method is more or less like the sendProcessList, the only difference is that instead of the process list, the requested service list of that particular client is sent to the server.

 public bool stopService(string
            serviceName)
{
try
{
IEnumerator en =
            Services.GetEnumerator();
while
            (en.MoveNext())
{
ServiceController service =
            (ServiceController) en.Current;
if (service.DisplayName ==
            serviceName)
service.Stop();
}
refreshServiceList();
string strResultXML;
strResultXML = "<result
            type=\"Stop_Service\">";
strResultXML += "<params
            result=\"Success\"/>";
strResultXML +=
            "</result>$";
byte[] bBuffer = new byte[3000];
bBuffer =
            Encoding.ASCII.GetBytes(strResultXML);
NetworkStream.Write(bBuffer,0,strResultXML.Length);
return
            true;
}
catch(Exception e)
{
string strResultXML =
            "<result type=\"Stop_Service\">";
strResultXML +=
            "<params result=\"Failure\"/>";
strResultXML +=
            "</result>$";
byte[] bBuffer = new byte[3000];
bBuffer =
            Encoding.ASCII.GetBytes(strResultXML);
NetworkStream.Write(bBuffer,0,strResultXML.Length);
return
            false;
}
}

The above method is used to stop a particular service of the client. The name of the service is received in the parameters of the stopService method. The ServiceController class is used here to stop the service, this class is used to connect to and control the existing windows services. Every instance of the service controller class has a Stop method, which stops the service. If the service cannot be stopped, it throws an InvalidOperationException, so in our case, if the service is stopped successfully, we prepare a result message with success as its parameter, else a failure response is sent back to the server.

Designing The User Interface

Our main goal in this application was to demonstrate network programming in C# using sockets, therefore the Graphical User Interface of this application was not the main focus. The server side comprises of a UI, as it is going to perform all the operations, while the client side does not have any UI.

Now we will be discussing each screen, as the operations are performed.

Main Screen

Above is the main screen when the server is executed. This screen contains information on each client that has been logged on. This is the only screen, so all the operations will be performed here. You can find this form in the code download that accompanies this case study. Let's discuss each section.

Name | Type | Description
lbClientsConnected | List Box | This list box contains all the clients that have been connected to the server.
tbComputerName | Text Box | When we select a client from the client connected list, the name of that particular client is displayed here
tbIPAddress | Text Box | When we select a client from the client connected list, the IP Address of the client is displayed here
tbUserName | Text Box | The name of the user of that client is displayed here
tbServiceAccessPoint | Text Box | The port of the server at which the client is connected
tbCommunicationProtocol | Text Box | The protocol that client is using e.g TCP, UDP
lbListofProcesses | List Box | It contain the name of the list of processes that are running on the clients computer
lbListofServices | List Box | It contain the name of the list of services that are available on the clients computer
tbStrProcessName | Text Box | When the process from the List of Process is selected, the name of that process goes here
tbProcessStartTime | Text Box | When the process from the List of Process is selected, the time the process started goes here

Whenever the client is connected to the server, its name is displayed in the Clients Connected list. The server administrator can then observe all the information regarding that client on the screen. The Process List shows all the processes that are running on the client's side, and similarly, the Service List shows all the services available on the client side.

The server can stop the client by pressing the Stop Client button; this button will disconnect the client from the server. The server can also perform operations on the services available from the client, that is, it can pause, resume, or stop the service. This can be done by selecting a service from the Service List and then pressing the respective button. For example, the service can be stopped by selecting it from the Server List and then pressing the Stop Service button. Selecting the process from the process List and then pressing the Stop Process button can only kill the process.

Now well show how a connection is established, and how a service is killed.

The above screen shows the client connected to the server.

The above screen demonstrates what happens when the client is selected. As you can see, all the information described earlier is displayed in the respective boxes. In this screen, the Process list is selected, which means that all the processes running on that particular client are shown.

The above screen is similar to the previous one, the only difference is that the Service List is selected here, which means that all the services available on the client are shown.

In this screen shot, one of the operations is demonstrated. By selecting any service from this list and then pressing any button at the top of the screen, the respective operation is performed. As can be seen, the ASP. NET State Service 's status is set to be stopped (highlighted).

Conclusion

While using .NET, I realized a need to demonstrate the networking concepts in C#. One can certainly read lots about these concepts, but what I wanted was a real application that shows the basic concepts of networking, like client server connection for example, in C#. For this reason, I made up a simple client server application using Sockets and TCP protocol. This application uses the System.Net namespace that includes base classes for TCP server and clients provided by .NET.

The case study illustrates some basic knowledge about .NET technologies, as well as some other technologies such as threads, XML, etc. As the key objective of the case study was to demonstrate Socket programming, a whole section is devoted to the Sockets and TCP concepts. It shows the networking concepts and describes in detail the OSI model, mapping of TCP over OSI model, Sockets, Socket programming, the System.Net namespace, etc. After going through all the technologies, the case study describes how Sockets can be used in C# by creating a small application.

The application is a simple client/server application in which multiple clients connect to a single server through sockets. This application uses TCP for the communication; it is a connection oriented protocol which is highly reliable as it sends acknowledgements for each request and response. The application allows the server to view all the client's information such as its name, its IP Address, the protocol it is using, the port it is connected to, the user's name, etc. The server can also view the processes and services running on the client side. It has the ability to make changes on the client's machine, such as to make the client stop, make the service stop, pause, or resume, or make the process stop.

C#Today - ©2002 Wrox Press Limited


Programming Internet Explorer in C#

Programming Internet Explorer in C#
http://www.csharptoday.com/content.asp?id=1980
Article Rating: 4.5
Published on November 11, 2002
Abstract
Internet Explorer is a powerful platform that can be used to develop presentation layers for any program that needs to have a rich formatted document interface. The.NET platform makes it possible to access all the power of IE without going too deeply into the details of COM interfaces implementations. However, because the actual version (6.0) is still a pure COM application and the only .NET libraries available for it are simple “interop? wrappers, it can be difficult to write client-side programs that use Internet Explorer, without having some knowledge of particular programming issues. These issues are actually very poorly documented at the moment. To try to fill up this gap, this case study by Nikit Zykov introduces the main starting points that make it possible to effectively access to the power of the IE platform using C#. You can also consider this case study to be about COM to .NET interoperability.
Case Study

Internet Explorer is a powerful platform that can be used to develop presentation layers for any program that needs to have a rich formatted document interface. The.NET platform makes it possible to access all the power of IE without going too deeply into details of COM interfaces implementations. However, because the actual version (6.0) is still a pure COM application and the only .NET libraries available for it are simple "interop" wrappers, it can be difficult to write client-side programs that use Internet Explorer, without having some knowledge of particular programming issues. These issues are actually very poorly documented at the moment. To try to fill up this gap, this case study introduces the main starting points that make it possible to effectively access to the power of the IE platform using C#. You can also consider this case study to be about COM to .NET interoperability.

Introduction

In the last few years, the success of Internet technologies has turned HTML into a widely recognized standard of data presentation. However, its use goes far beyond displaying simple hyperlinked documents. Market demand pushed the companies developing Internet browsers to include into their systems a wide range of formatting, navigation, and programming functionalities. This made it possible for software companies to create complex and fully functional products using HTML browsers as intelligent client presentation platforms.

The best known application paradigm that uses HTML-based technologies at the presentation layer is actually ASP : not Active Server Pages especially, but more generally as Application Service Providing. The idea is to host application software, or to be more precise its data and business layers, on a central Web server, sending the user interface data as a HTML flow to the user's browser, and then exchange data between the client and server via the HTTP protocol. However, there have been some serious problems in developing this model, and as a result it is not yet (at least in Europe) a clear market success.

The first problem is purely marketing: the end-users need some time to get used to this new distribution model, and for the time being, there is not yet enough confidence to it, especially for a non-computing audience. The second reason is technological: the commonly used "page at once" HTTP model makes a really dynamic interface creation difficult, and the fact the programmer has no real control over the browser being used client-side adds extra problems.

Despite these problems (and others that we haven't mentioned), the technologies are constantly in development and we can foresee them becoming overwhelming important in the market in a few years, especially with the probable growth of real-world client-server solutions based on the SOAP exchange protocol. Meanwhile, most programmers still have to work on traditional, more or less "stand-alone", client software.

Naturally, when writing stand-alone programs, we don't want to lose the advantages the HTML presentation model gives us. And fortunately, we don't have to: Internet Explorer (IE), the HTML browser market leader, is not simply a program allowing the user to view some online Web content, but essentially a full-featured programming component that can be integrated into any Windows-based software solution. Here we're going far beyond the traditional "browser-based" models: even if we still have access to the browser's communication model allowing us to manipulate content resources at any location, this part of the functionality remains secondary. The power of Internet Explorer model is the possibility for it to be integrated into any presentation logic, including that of "traditional" Windows-based systems.

There are many advantages to directly programming the Internet Explorer interface. The extended DHTML object model grants the programmer full and real-time access to any presentation element's behavior. This allows us to fully control the entire information flow to and from the user, and to create the richest possible dynamic and intelligent interface for the software. Also, directly accessing IE eliminates the need to have a specific platform for the "server" part of the application, avoiding the need for client-server data exchange and reducing technical support costs. Nevertheless, this programming technique doesn't restrict you to just the world of "stand-alone" applications; you can still use the server-based solutions, simply delegating some of the processing to the client platform, meaning you can still enjoy all of the power of the presentation and communication model Internet Explorer gives you. In fact, this gives your programs a fully scaled presentation layer, concentrating all the presentation logic on one platform, as opposed to the "classic" ASP that delegates only the simple data display to the client, and processes all the presentation management server-side.

Before the .NET platform release, there were basically two main ways to program Internet Explorer. These were:

  • The scripting model
  • Treating IE as a COM component

The scripting model was given by IE itself, while treating IE as a COM component meant programming it directly in C++.

Note : There also was an intermediate way, in that automation-capable languages (those allowing access to IDispatch -type COM interfaces, like VB6 and J++) offered some access to the Internet Explorer object model. However, there were some serious limitations concerning the implementation of advanced interfaces, such as binary behaviors for example, which meant this way was far from perfect, and so we will not explore it in this case study.

Scripting was a good way for those who wanted to easily and quickly write interface code with a minimum knowledge about the platform's functional details. Also, scripting client-side code is generally the best solution when creating "light client" deployment systems when the only coherence between the software users' systems should be the Internet browser version.

At the same time, scripting imposes important limitations on our access to system functionalities, including, in the case of the Internet Explorer, limits on some features of the scripting platform itself. Also, scripting languages, even advanced ones like JavaScript, often take ease of use over rigorousness of the structure. Execution speed is not a strong point of script engines.

C++ programming was a good alternative, because it allowed access to all the platform features and a complete control of the underlying system's functionality. At the same time, this solution required highly qualified developers, and also often meant lots of debugging was necessary.

.NET, and C# in particular, allow us to combine the advantages of both solutions without having to experience their disadvantages. The relative ease of use goes together with a complete access to the operating system and its components. There is of course also the possibility of integrating Internet Explorer features with any .NET software too.

Nevertheless, in releasing .NET, Microsoft has not transformed Internet Explorer into a true .NET system. It's true that since version 5.01 there has been some preparatory work, but the platform access is still a pure COM. The only effort made was to compile a wrapper assembly (named mshtml.dll) that allows us to use the IE COM interfaces from any .NET language.

Looking into this assembly, a programmer can easily understand why the Internet Explorer .NET "new-look programming" has still not really been done. There are countless functions in many hundreds of interface classes, and the classes' functionalities are often redundant. Worse, there is no real hierarchy in these classes: the COM paradigm made an interface immutable - every time there are some new features added to a COM object, a new interface is exported, and generally this interface includes only the new features, without the big "inheritance" effort.

The MSHTML library follows this paradigm, simply giving access to the majority of interfaces exposed by Internet Explorer. A little effort has been done by mechanically combining interfaces relative to any particular underlying COM object into some kind of .NET class, without really trying to organize all that into a more logical manner. In conclusion, the assembly is not really easy to use, especially because there is practically no accessible documentation!

However, programmatically accessing Internet Explorer platform remains a relatively simple task. Most of the effort is essentially organizational. All you need to know is how to access some basic techniques, then to encapsulate this functionality to a consistent class library, and then IE programming using C# or any other .NET language becomes as easy as a traditional DHTML scripting.

Some programmers will ask why they should need to use an Internet Explorer interface when they now have easy access to Windows Forms, which is an extremely powerful presentation platform. In fact, the choice of whether or not to use an IE DHTML presentation for the user interface depends on the type of information the programmer has to show to the final user.

Windows Forms are perfect if you know in advance the exact structure and format of data you have to display. This library is designed to give the programmer full control over every individual object displayed, its characteristics, contents, and position. However, this flexibility has its cost: while it is very easy to work with any number of single objects, the task becomes much more difficult when dealing with complex data flows, as it is the case for any document management system, for example.

This is exactly the situation where Internet Explorer shows its worth. HTML was created because of a need to have a common data format that everyone can read, but it has grown due to a need to have a rich presentation for this data. If you want to give a new skin to the data your client provides in some structured format (which actually often means XML), using Windows Forms, you'll need to write your own parser. This is interesting, but it means very high development costs. With Internet Explorer used as a presentation component, it's straightforward; you just have to add a style sheet. And if you want to add interactivity to this data management, you'll need to access the IE programming interfaces (which is precisely the subject of this case study), thus integrating it de facto to Windows Forms system as one more component, probably the most complex, but also giving powerful rich interactive data presentation.

The best use of the Internet Explorer's "browser" component is integrating it into a more complex system, including other Widows Forms elements, that can offer more power for some particular task. Remember the presentation of rich-formatted documents (especially if this presentation is often changing, don't forget that Internet Explorer understands CSS too) and document-driven data is much easier when using the IE component.

In this case study, we won't be especially considering Internet Explorer as a web browser, i.e. as a tool used to access remote data. Writing server-based and client-independent applications is another situation that has its own advantages and constraints; we'll not analyze it now. Here, we're taking about a means of presentation layer development for applications that are not especially networked, but that do have requirements of high-quality presentation of big volumes of human-readable data. We'll also be considering how this helps reduce the presentation layer development costs.

This case study gives you a starting point; it unveils the techniques needed to effectively access the power of the platform by presenting some basic examples that are easy to understand. It also gives an idea of the way to follow to create an easy-to-use library to access the platform. And for the curious people it also explains how to go further to explore the advanced features of the Internet Explorer.

Requirements

To get real benefit from this case study, the reader must have a good understanding of C# and of object-oriented programming in general. A good knowledge of the DHTML object model would be very helpful too. There is no real need to deeply understand the COM model, but an understanding of its basics would be advantageous.

The examples are meant to be used with the final release of Visual Studio.NET (2002) on machines with Internet Explorer 6 (or more recent) installed.

Definitions and structure

This is not a traditional "one task" case study. Its goal is not to guide the programmer through the steps of developing one particular application, but rather to introduce some essential techniques allowing the use of a particular presentation platform. Each section will talk about its own little bit of code in order to demonstrate the issues being discussed.

The main sections of this case study are:

1 - Internet Explorer as a programming component

This section shows how to integrate an Internet Explorer interface into a C# application created using Visual Studio .NET. It explains how to call its methods and respond to its events.

2 - Accessing DHTML object model

This section explains how, having a document loaded in the IE component, to obtain access to its DHTML structure. We take the page loading procedure from the previous section's example and as an example we extract the HTML code of each page loaded. We also discuss here the interoperability details needed to completely understand the underlying mechanisms.

3 - Introduction to the DHTML document structure

Without going too deeply into the concepts of DHTML, this section presents the notions of "document" and "element" in the DHTML object model. First, loading a blank document to the component, we show that even a blank document has its own DHTML structure. We then programmatically create a text label, a button, and an input zone on the blank page. We obtain interfaces for each of these elements. On the basis of this example, we study some of the basic ideas of encapsulation of the document-related objects into a consistent class structure.

4 - DHTML events treatment

This section discusses the way the Microsoft wrapper treats the DHTML events in the .NET code. It also explains why this method can only by applied to certain cases, and then develops an original method of implementing event handlers. This example also demonstrates some advanced interoperability techniques.

5 - Launching scripts from C#

In this section, we'll show how to call any JavaScript function implemented on the DHTML page's namespace. It also discusses some problems related to parameters and return values, touching on some basic aspects of marshalling data between .NET and COM applications.

6 - Extending the DHTML model

window.external is a scripting interface object that when implemented gives the script author the opportunity to call additional methods defined by the developer of the application hosting the Internet Explorer component. We explore this situation in this section.

1 - Internet Explorer as a programming component

When using the .NET environment, you can consider an Internet Explorer browser window to be a simple Windows Forms component. You can integrate it into a form, call methods and use properties it exports, and attach its events to the handlers you develop. However, all this requires a little preparatory work: don't forget Internet Explorer is not a native .NET application. Fortunately, the new Microsoft platform includes easy to use interoperability functionalities that allow integration to any COM/ActiveX object as if it were a native .NET component.

Let's see now how to effectively do it. Open Visual Studio .NET and create a new C# Windows Application project - this creates an application template with an empty form as a main window. We now have to add Internet Explorer component to this form. To do it, we have first to integrate the Microsoft Web Browser component to the environment as a Windows Forms control.

Go to the Toolbox, right-click it, and choose Customize Toolbox in the context menu.

In the dialog box shown, choose the COM Components tab, and then check the Microsoft Web Browser component. Please note that if your operating system or Internet Explorer version are not English, this component's name will probably have been translated. For example, it is called "Navigateur Web Microsoft" for French systems. After closing the dialog box, you'll find the new Explorer component at the end of the Windows Forms section of your toolbox.

Now drag and drop this component onto your form. This has two effects. First, it automatically creates and adds two interoperability assemblies to the bin/Debug directory of your application (assuming you compiled it in debug mode). These are Interop.SHDocVw.dll and AxInterop.SHDocVw.dll, both referencing the COM shdocvw.dll library that implements the Web Browser control. The first library represents a complete "interop" view of the underlying COM library, giving access to its interfaces and events that allows the programmer to "fine-tune" the component use. The second, used by the Windows Forms integration shell, is an "ActiveX" view of the same library, giving a VB-style access to the Web Browser component implemented as an application class.

The references to these assemblies are added to your projects. This allows you to directly use their classes and interfaces in your code. The second effect of adding the Explorer control to the form is, as with any Windows Forms component, to generate the C# code to effectively add the component to the form at execution time. Look at the Windows Form Designer generated code region in your code to see details of the component integration, and notice it's slightly different compared to ordinary .NET component integration. This difference is transparent for you now, but it can be important if you need to create your code without using Visual Studio.NET.

Normally, the first Web Browser component created will be referenced in your code as the axWebBrowser1 variable of the type AxSHDocVw.AxWebBrowser, which is in fact a wrapper for a Visual Basic-style ActiveX class generated around the IWebBrowser2 interface of the component. This class is our entry point, allowing us to manipulate the IE Web Browser object and to use its events.

The first thing we can try to do is to navigate to a web page of our choice when the program initializes. This functionality can be accessed through the Navigate method of the Web Browser class. The method takes five arguments. We'll only use the first, specifying the URL to navigate to. The four other arguments allow us to specify the navigation details, for example the target frame or POST data.

A small problem is that this method is a direct wrapper to the correspondent C++ specification, so its arguments follow exactly the standard type conversion rules; in particular, when there is a Variant pointer declared in C++, we get a ref object in C#. So, even for passing zero or empty string parameters, we absolutely have to create intermediate (useless) variables. Considering all that, the navigation code (using http://www.csharptoday.com/?WROXEMPTOKEN=618658Zezi9SpDFSZgQps5Xv6u as an example navigation target) will be the following (insert it into the form constructor, after the InitializeComponent() call):

object zero=0;
object nl="";
axWebBrowser1.Navigate("http://www.csharptoday.com",ref zero, ref nl, ref nl, ref nl);

Launch your application, and if you are connected to the Internet, you'll see the C# Today homepage displayed in your navigator component:

Now, adding some basic functionality, we are already able to program the simplest-possible custom Internet browsing program. This program will have an address bar showing the loaded page's URL, a Navigate button allowing the user to go to the address entered in the address bar when it is changed, and a Home button that takes us to a hard-coded homepage.

First go to the form designer and add one text box and two buttons to your form. Label the first button "Go" and the second "Home". Now, by double-clicking the "Home" button, define its activation (mouse click) event; it'll be button2_Click in your code. Move the navigation code actually entered in the Form1 constructor code to this handler. After adding a line that sets a cursor to the "wait" state when the page is loading, our handler will look like this:

object zero=0;
object nl="";
axWebBrowser1.Navigate("http://www.csharptoday.com",ref zero,ref nl,ref nl,ref nl);
this.Cursor=Cursors.WaitCursor;

Navigating to an Internet page in IE is of course an asynchronous process, so we need to handle the event that fires when the page load is complete. This is done by defining a handler to the DocumentComplete event, fired by the Explorer COM object.

Return to the form designer, select the Explorer component, then the Events tab in the Properties window. After that, double-click on the DocumentComplete event. This adds to your code the axWebBrowser1_DocumentComplete handler that is called when the corresponding event fires. If you look at the Windows Forms Designer generated code section, you'll see that DocumentComplete is exposed by the AxWebBrowser class as a C# event and your event handler simply implements the delegate type DWebBrowserEvents2_DocumentCompleteEventHandler. So basically, implementing event handlers for the Explorer object is as easy as for any native .NET object.

Place the following code in the event handler:

textBox1.Text=axWebBrowser1.LocationURL;
this.Cursor=Cursors.Default;

With this code in place, when the page is finished loading, you'll find its complete URL in your "address bar" and the mouse cursor will return to its normal state. Take care however, this works fine for single-frame windows; when you go multi-frame, the DocumentComplete event is fired on every frame load completion, and so if you plan to place some important initialization code here, you'll have to detect which part of the HTML document fires this.

The last thing we have to do is creating an event handler for the "Go" button; the only difference compared to the "Home" handler will be that the URL to navigate to will be taken from our "address bar", and not hard coded:

object zero=0;
object nl="";
axWebBrowser1.Navigate(textBox1.Text,ref zero,ref nl,ref nl,ref nl);
this.Cursor=Cursors.WaitCursor;

That's all; you can now use your program as an Internet browser!

At this point, we get to look at the first bit of interesting encapsulation code: notice that the Navigate method is not really easy to use as is. Boxing parameters makes you waste lines of code, but it is not the only problem in our case: we can't easily emulate the HTTP posting of string data, because even if there is a data conversion (marshaling) between .NET and COM (for instance, a reference to an array in .NET is directly translated to the pointer to SAFEARRAY COM type), the mandatory array data type for the "post data" parameter is not really convenient for everyday use.

The other important issue in the case of this method is the fact that in the most cases we'll really only use one, or a maximum of two, of its five parameters. So, the best idea would be to create a method with a variable number of parameters. C# gives us such a possibility by providing a params keyword. Using it, the Navigate encapsulation should look like this:

public void Navigate(params object[]
            data){
object url= (data.Length>0 && data[0] is
            string) ? data[0] : "about:blank";
object target=
            (data.Length>1 && data[1] is string) ? data[1] :
            "";
object headers= (data.Length>2 && data[2] is
            string) ? data[2] : "";
object
            post="";
if(data.Length>3){
object post=data[3];
if(post
            is
            string){
if(((string)headers).IndexOf("Content-Type:")<0)
headers+="Content-Type:
            application/x-www-form-urlencoded\r\n" ;
string
            tmp=(string)(data[3]) ;
post=new byte[tmp.Length];
for(int
            i=0;i<tmp.Length;i++)
((byte[])post)[i]=(byte)tmp[i];
}
}
object
            flags= data.Length>4 ? data[4] :
            0;
axWebBrowser1.Navigate(url,ref flags,ref target,ref post,ref
            headers);
}

This method changes the order of arguments you pass to the method to put the most frequently used first. The first argument is (as for the original Navigate) the page URL, the second is the target (for example, "_top" for navigating in the topmost frame, just like in the target attribute of the <a href.> HTML tag), the third is a string containing the CrLf-separated HTTP headers sent, the fourth is the post data, and the fifth are the flags modifying the method behavior.

(Headers are the set of technical information the browser sends to the server when requesting a page. It is in general a plain ASCII text containing one parameter per line in the " name: value " form. That can be used, like in the example above, to specify the format of the data posted to the server or, for instance, to specify, for multi-host Web servers, a virtual service URI.)

The post data treatment requires some special work in the function. First, it can be passed as a string, or alternatively, as an array of any integer value type. The array version gives the maximum freedom to the programmer, allowing us to pass any amount of any data to the given address. The string version can be used, for example, to simulate a form data submit. In this case, we need, as it was written before, to first convert the string to the byte array (no, HTTP doesn't always use Unicode) and to add, if it is not already done, a header specifying that we are sending data to a form.

Now, sending two form fields to a dumb ASP script can be done like this:

Navigate("http://www.zykov.com/dumb.asp","","","test=essai&test2=proba");

.and simply requesting a page is just:

Navigate("http://www.wrox.com");

2 - Accessing DHTML object model

Until now, we have had almost nothing to program: Visual Studio automatically did most of the work. The ActiveX interfaces of the Web Browser object give the interoperability system enough information to naturally incorporate this component and to give you all the "visual" programming facilities you have with any .NET component, including IntelliSense, which allows you to easily discover the object's functionalities.

All that gives you is access to manipulate the browser window itself; what is much more interesting is the contents manipulation, i.e. the possibility for us to view and change the structure of the HTML documents loaded into the Internet Explorer browser window. This first requires some supplemental work.

The AxWebBrowser type gives us access to the Document property that represents the object managing the DHTML document that is loaded. This object can be considered as an (extended) equivalent of the document object in the DHTML scripting model. In C#, our problem is that this property is of type object, which doesn't really help us to manipulate it.

If we look into the C++ headers declaring "low-level" the equivalent interface type, we'll see that the Document property is exported as an IDispatch interface pointer, which is effectively represented by an object type after marshalling to .NET. In COM, when we need to use such an object, we have to obtain one or more interfaces that indirectly give you access to the class's methods and properties. The main difference between COM and .NET interface concepts is that in COM the interface is the only possibility to use an object, any other access is (in theory) forbidden. So, if we want to use the object behind the Document property, we first need to obtain interfaces giving access to it.

Fortunately, the relatively complex process of querying a COM interface is hidden in .NET behind a simple type casting procedure: it's enough to explicitly convert the object's type to the interface type you need in order to automatically obtain the pointer giving you access to all the properties and methods declared in the interface. This also has an interesting side effect: it's enough to apply the C# is operator to a .NET reference of the COM object to know if the interface having its type as the other operator's argument is exported.

But, where can we find declarations of the interfaces we need? The ShDocVw libraries we got when integrating the Explorer component into our code don't give us any information on the underlying HTML documents. This is the point when we need to use MSHTML, the library we've briefly spoken about in this case study's introduction. We need to explicitly reference this library before using it.

Go to the Visual Studio Solution Explorer with your project open. Right-click the References folder and choose Add Reference.

In the dialog box that appears, be sure to have the .NET tab selected, and then find the Microsoft.mshtml component in the list. Double-click on it (it must appear in the Selected Components box), and then click OK. Now we simply have to add the using mshtml declaration at the beginning of the C# code file where we need to access it, and we can begin work.

The MSHTML interfaces we can use to access the DHTML object model are actually IHTMLDocument, IHTMLDocument2, IHTMLDocument3, IHTMLDocument4 and IHTMLDocument5. Why are there five interfaces to access the same object? The reason is that in COM, once the interface is declared and uniquely identified, it can't change at all. So, when you add some functionality to your COM class and you want to give the class users access to these new methods and properties, you need to create a new interface that generally declares only new methods. There is no real inheritance mechanism behind all that, so frequently (and that is the case with the Document object) each individual interface contains only the new methods of the object's versions corresponding to the moment the interface was created. In our case, IHTMLDocument is a heritage of "prehistoric" versions of Internet Explorer, with the other four exports being the new functions of versions 4.0, 5.0, 5.5 and 6.0 of the platform respectively.

The MSHTML.NET library that is in fact a wrapper for the corresponding COM library, does a little more by combining all these interfaces into one HTMLDocumentClass.NET class. However, it is preferable to use interface classes directly when creating your Internet Explorer C# applications. The reasons are too complex to be discussed here; we can simply say that you can program easier by creating your own class hierarchies that fit better to your project's style and generally to the .NET class libraries' ideology.

Now, let's try to obtain one of these document interfaces and use it, for instance, to access the loaded page's DHTML code. As we won't be directly working with the form components' behavior from now on, it's better to create a new class and to encapsulate all the DHTML document treatment into it. So, create a new class file in the Solution Explorer, naming it DHTMLDoc (don't forget to add " using mshtml " to the class file's headings!), and add the following "skeleton" code to it:

public class DHTMLDoc{
public
            DHTMLDoc(object doc){
if(!(doc is IHTMLDocument3))throw new
            Exception("This is not a DHTML
            document!");
document=doc;
doc2=(IHTMLDocument2)doc;
doc3=(IHTMLDocument3)doc;
}

object
            document;
IHTMLDocument2 doc2;
IHTMLDocument3
            doc3;
}

This makes you a constructor that, when obtaining the reference of the COM object's wrapper, first confirms that it's really a DHTML document object reference (we can't use the Internet Explorer versions older that 5.01 with the .NET Framework anyway, IHTMLDocument3 is the minimum requirement), and then obtains from it the two most used interfaces: IHTMLDocument2 and IHTMLDocument3. These two are actually declared in the MSHTML library as common .NET interfaces, so you can use them as is and explore them through IntelliSense in the Visual Studio .NET environment.

The last important thing to note before implementing our "HTML code view" is the fact that you can (of course) only use the Document property of the Explorer object when there is an HTML document completely loaded to it. But you already know how to control this: there is the DocumentComplete event in this object that fires exactly when any document is completely loaded. Let's use it.

First add to your axWebBrowser1_DocumentComplete handler the following line:

DHTMLDoc doc=new DHTMLDoc(axWebBrowser1.Document);

This instantiates the document encapsulation class you recently created. We'll now use it to obtain the document's "source code". Add to the DHTMLDoc class the following method:

public override string ToString(){
return doc3!=null ? doc3.documentElement.outerHTML : "";
}

This function overrides the standard ToString function of the Object class, thus we consider here the DHTML document's contents as its string representation from the C# point of view. We use the instance of the IHTMLDocument3 document's interface to obtain its root element (i.e. the container for all the document code), exposed as a documentElement. This element is already declared as an IHTMLElement, that is the default (but not the only) interface it exports, so we can directly use it to get, just like in script DHTML model, the outerHTML property returning the underlying HTML code, as it is parsed by Internet Explorer.

Now, the only thing we have to do is to show the result to the user. Go to the Form designer, add a new text box to your form (named textBox2), and declare it as multi-line, read-only, and using both scroll bars. Now add just one more line to your DocumentComplete event handler:

textBox2.Text=doc.ToString();

When navigating to any page with your new application, you'll now see the page in the navigator window and its HTML code in the text box at the same time:

One little bug fix to end this section: you'll probably have some difficulties effectively resizing the Explorer control you're using. More precisely, resize will be reported as effective (and reflected in the source code), but when your program launches, you'll see the control returning to its initial size. This problem is common to many "VB-style" ActiveX controls: it comes from the fact the component's size is automatically kept in the "OCX state" bag and restored (by default) every time the program is launched. To fix this, after visually modifying the component's size, but before compiling and launching the program, you just have to look into the "Windows Form Generated Code" section in your form code and to find a string containing " OcxState ", and comment it. After that, compile and launch your program, then immediately close it. When that's done, the new size is recorded in the state bag, and you can uncomment the line again.

In conclusion, technically the COM to .NET interoperability is relatively easy to use, but philosophically, it remains a "thing in itself".

3 - Introduction to the DHTML document structure

In the previous section, we introduced the concept of the document. This is the central point of the DHTML object structure. There is one other class of objects, closely related to the document paradigm, a window. It relates to the same underlying object, but looks more at the presentation aspects of the framework than the structural details. These objects often share (especially when seen from the MSHTML point of view, which is more detailed that the script DHTML model) the same functionalities, like script management or frames manipulation for example. In the most cases, you can consider that one document object corresponds exactly to one window object. It seems logical when implementing an encapsulating .NET library, to mix the both functionalities in one generalized class, giving a starting point for manipulating any aspect of the DHTML data flow.

These two object's classes give us our initial framework; now let's see how we can manipulate the inner structure. The primary concept behind HTML means that almost everything you can encounter in a "document" is an element, i.e. object representations of tags and their contents. It is the fact that you can manipulate these objects and change their presentation properties in real time that adds the "D" (dynamic) to the DHTML acronym. Note that the element objects export, just like documents, multiple interfaces. These are actually IHTMLElement, IHTMLElement2, IHTMLElement3 and IHTMLElement4.

Elements are basically organized as a tree, so elements can contain other elements and the document can be considered as a root container. The HTML standard, and especially its Internet Explorer version, forces any document to have some minimum structure. In the program you created in the previous section, if you navigate to about:blank (which is the equivalent of loading an empty HTML page), you'll see there is the following code in the corresponding text box:

<HTML><HEAD></HEAD>
<BODY></BODY></HTML>

This simply means that the document object created from the empty code still gives you access to the visual document presentation contained in the BODY section. This is represented by the body property of the IHTMLDocument2 interface, and its management details that are normally listed in the HEAD section of an HTML file and are accessible through multiple document's and window's properties and collections.

We'll use this by implementing an example of dynamically inserting some HTML into the empty document that is loaded. Change the DocumentComplete handler by inserting (before the string showing the document code, because we want to see the insertion results both in the browser presentation and in the code) a call to the Test method of our doc object, conditioned by the fact the URL of the page loaded is about:blank. The handler's code should now look like this:

private void
            axWebBrowser1_DocumentComplete(object sender,
AxSHDocVw.DWebBrowserEvents2_DocumentCompleteEvent
            e)
{
textBox1.Text=axWebBrowser1.LocationURL;
this.Cursor=Cursors.Default;

DHTMLDoc
            doc=new
            DHTMLDoc(axWebBrowser1.Document);
if(axWebBrowser1.LocationURL=="about:blank")doc.Test();
textBox2.Text=doc.ToString();
}

Yes, you're right; the Test method is not yet implemented in the DHTMLDoc object. In this method, we'll simply fill the innerHTML read-write property of the document's body (which is of course an element object). If you already programmed scripts for Internet Explorer 4+, you probably know this technique:

public void
            Test()
{
doc2.body.innerHTML="<input type=\"text\"
            id=\"inp1\"/><hr/>
<button
            id=\"but1\">Button</button><hr/>
<span
            id=\"text1\">Text zone</span>";
}

Now, once you've compiled and launched your program, and then navigating to the about:blank URL, you'll get the following results:

Note that the HTML code we put into the document's body is a well-formed XML fragment; the result we're seeing in the Browser's document object is not. The Internet Explorer hypertext markup processor converts the code it reads to make it fit to the internal object structure that is actually not true XML, even if some efforts were done in the last version by introducing the DHTML DOM paradigm.

Just as you have done with the body, you can access and manipulate any of the document's elements. To demonstrate it, we'll develop an extension to our program, allowing the user to change the background color of any element.

First, we'll need some preparatory work. We'll have to add some new interface elements (we're talking about Windows Forms here, don't confuse it with DHTML!) to our program to access the DHTML document from their event handlers. So, we first need to grant access to the document object from anywhere in the class implementing our main application's form. For that, just move the doc variable declaration from the DocumentComplete event handler to the Form1 class's body. To be sure we're always using a valid document object, initialize it to null before every Navigate call. Now, go to the form designer and add to the form the following: one combo box (it'll list our document's elements; declare it with a DropDownList type), one label (it'll "ask" for the element's background color), one more text box (we'll specify the color here; we suppose its name is textBox3 in the C# code), and finally one button (we'll use it to apply the changes to the document).

Now we have to prepare three methods to give us the new functionality. The first will list (by specifying tag names and id attributes, if present) all the elements actually present in the DHTML document loaded in our Browser component, returning the result as an object array. The second will extract a specific element's actual background color (the element being identified by its ordinal number in the document's all collection containing every element present). The third will effectively change the background color style property of such an element. Because the first method uses the dynamic array list, we'll also need to add the using System.Collections declaration to the class file heading. The methods' code follows:

public object[] GetAll()
{
ArrayList
            retval=new ArrayList();
IHTMLElementCollection
            alles=doc2.all;
foreach(IHTMLElement elm in
            alles)
retval.Add(elm.tagName+(elm.id==null?"":"
            id="+elm.id));
return retval.ToArray();
}

public string
            GetColor(int en)
{
IHTMLElement
            elm=(IHTMLElement)doc2.all.item(en,0);
if(elm.style==null)return
            null;
return
            (string)elm.style.backgroundColor;
}

public void
            SetColor(int en,string col)
{
IHTMLElement
            elm=(IHTMLElement)doc2.all.item(en,0);
if(elm.style!=null)
elm.style.backgroundColor=col;
}

We have to fill the combo box with the elements' list every time the new document is loaded to the Browser object. To do so, add the following code to your axWebBrowser1_DocumentComplete handler before the line displaying the document code:

object[] elms=doc.GetAll();
foreach(string e in elms)
comboBox1.Items.Add(e);

We just have to code the other two events now. The first, the combo box element selection, will show the selected element's actual background color value. The second, fired when the "Change" button is pressed, will change that color. The handlers code is as follows:

private void comboBox1_SelectedIndexChanged(object sender, System.EventArgs e)
{
string col=doc.GetColor(comboBox1.SelectedIndex);
if(col!=null)textBox3.Text=col;
else textBox3.Text="";
}

private void button3_Click(object sender, System.EventArgs e)
{
try
{
doc.SetColor(comboBox1.SelectedIndex,textBox3.Text);
textBox2.Text=doc.ToString();
}
catch(Exception ex)
{
MessageBox.Show("Color change error:\n"+ex.Message,"Error");
}
}

Notice the second handler's exception handling code. It is needed because the user is free to put any value into the "color" text box, and the DHTML processor does not accept invalid values. Compile and launch your program; test it, taking the previously generated HTML code as an example:

This section's examples show you how to analyze and change everything in any HTML document you load to the Browser component. Don't forget that when you get an object from one of the MSHTML methods or properties which has some particular interface as a declared type (IHTMLElement for IHTMLDocument2.body for example), you can often also obtain the interfaces that were implemented later for the same object. For example, if obj is declared as IHTMLElement, you can almost be sure that (IHTMLElement2)obj and (IHTMLElement3)obj conversions will be valid.

A good idea to optimize the DHTML integration in your projects would be to create a class hierarchy that has two main base classes: document, that combines DHTML document and window features; and element that encapsulates all the methods and properties exported through different interfaces of the underlying COM object. You can then inherit all the classes implementing specific functionality concerning the particular element types (like an input box or an anchor). For complex systems, it can also be interesting to encapsulate the Web Browser object itself in order to give your programmers a complete and consistent class hierarchy.

4 - DHTML events treatment

The features we've studied so far give us full access to DHTML content management, but the interaction with Internet Explorer-side remains completely passive. We still need to use standard Windows Forms components to deal with user input. Nevertheless, the DHTML object model exposes the large event model, allowing us to develop a full-featured user interface just by using Internet Explorer.

Of course, we can manage these events from a C# program. Microsoft developers integrated the full set of .NET events and delegates encapsulating the DHTML events functionality. To give you an example of this encapsulation use, we'll first try to treat the mouse click event for the HTML button from the previous section's example.

If you've already programmed DHTML scripts, you will know that such an event actually uses the onclick handler. In the MSHTML library, a particular event's interface and a full set of corresponding delegates are associated with every particular type of element, like buttons or input boxes. In our case, all the events are managed by the HTMLButtonElementEvents2_Event interface (that we need to explicitly obtain from the element object), and the onclick event is described by the HTMLButtonElementEvents2_onclickEventHandler. Let's put all that together. We'll first implement an easy way to access to a document's element from its id attribute value. For that, we'll implement in our DHTMLDoc class a little indexer method:

public object this[string id]
{
get{
            return doc2.all.item(id,0); }
}

Using this, we'll just need a couple of lines of code to attach the event to the handler function we'll develop in our form class, following the delegate prototype like this:

private bool OnDHTMLClick(IHTMLEventObj e)
{
MessageBox.Show("DHTML button click detected","C# event handler");
return true;
}

The code we have to put at the end of the DocumentComplete event handler follows:

if(doc["but1"] is HTMLButtonElementEvents2_Event)
((HTMLButtonElementEvents2_Event)doc["but1"]).onclick+=new
HTMLButtonElementEvents2_onclickEventHandler(OnDHTMLClick);

The only peculiarity in this two-line code snippet is the fact that we verify if the interface we're interested in is really exported before we actually use it. After compiling and launching the program, you can verify that your C# code is really invoked when you click the HTML button:

It is relatively easy to write this: we really need only two lines of standard C# event code to attach the handler we need. However, it is only easy if we remember the (forty-character) name of the event and that of the corresponding delegate! Also, don't forget that the delegate prototype is not the same for all events! This last point is not easy to understand: the DHTML object treats all its events in a strictly identical manner, there is one event object shared by all the elements through the DHTML window object.

That would not be such a serious problem (the library may not be intuitive, but it can still be usable) if it could be easily implemented, but it is not. If you try to handle the onchange event for our text input box, you'll see that, even after attaching and correctly implementing the HTMLInputTextElementEvents2_onchangeEventHandler delegate for the onchange event of the HTMLInputTextElementEvents2_Event interface there is no reaction from the event handler when the event is fired. A forgotten implementation, probably.

So, if we really want to have access to any DHTML event and do it easily, without having to verify the delegate type at every event attachment, we have to find another way to deal with events. The solution exists, of course: .NET events and delegates are not the native mechanism for dealing with events in the IE COM platform, they are just wrappers. Internet Explorer events handlers are in fact ordinary interface pointers, which you can set to an object of your choice. But what kind of objects is it? The MSHTML C++ documentation says it must be an IDispatch -type COM objects having the event handler as its default method. There is no problem with IDispatch, because any .NET object, when marshalled to COM, implements this basic interface, but what's a "default method"? There is no such notion in C#. In COM, any IDispatch interface method has a numeric identifier, a " dispid " - the default method is simply one that has this identifier set to zero. The .NET platform allows you to force the interoperability engine by setting the dispid of a particular method to any value through the attributes mechanism: so, it is enough to give your method the [DispId(0)] attribute to make it default from the COM point of view.

That means we can create a class that implements the event handler through its "COM-default" method and assign an object of this class to one of the IHTMLElement* interfaces' event handler properties, like onchange or ondblclick. Note these properties are not the .NET-style events; they are interface pointers, declared in .NET, after standard marshalling as an object type.

The last thing we have to do before implementing such an event-handler class is to make it universal. That means that we don't want to create a new class every time we want to handle an event. To do this, we can simply create a delegate and manage a variable of this delegate type; our handlers will simply be attached to this variable. For our first example, we'll use the simplest possible implementation, with nothing dealing with the event information object, our handler will simply respond to the event fired without trying to deepen its contextual knowledge.

Create a new class named DumbDHTMLEventHandler. Put the following code in the class file:

using System;
using
            System.Runtime.InteropServices;

namespace
            IEExample
{
public delegate void
            DumbDHTMLEvent();

public class
            DumbDHTMLEventHandler
{
[DispId(0)]
public void
            Call()
{
Handler();
}

public DumbDHTMLEvent
            Handler;
}
}

Note the use of the System.Runtime.InteropServices namespace. We need it to access the implementation of the DispId attribute. Note also the DumbDHTMLEvent delegate is declared namespace-wide, we'll use it to declare event handlers in our code. Let's implement the onclick event for the input box generated in our example. Place the following code at the end of the DocumentComplete event handler:

if(doc["inp1"] is
            IHTMLInputTextElement)
{
DumbDHTMLEventHandler h=new
            DumbDHTMLEventHandler();
h.Handler+=new
            DumbDHTMLEvent(OnDHTMLText);
((IHTMLInputTextElement)doc["inp1"]).onchange=h;
}

Note we need to obtain the IHTMLInputTextElement interface from our element reference in order to be able to change the onchange property: this event is not common to all the DHTML elements, so we need to deal with it in a particular way; this is a case when using multiple interfaces for one class is completely appropriate.

We can implement the OnDHTMLText handler in our Form1 class like this:

private void OnDHTMLText()
{
MessageBox.Show("DHTML text change detected:
"+(((IHTMLInputTextElement)(doc["inp1"])).value),"C# event handler");
}

After compiling our application, launching it, loading the about:blank page (which launches our DHTML generation code), changing the HTML input box's text, quitting that box (by pushing the Tab key, for example), we can see our event handler work successfully:

This works effectively; however, it would be more interesting to make it a true .NET-style event handler in order to use it in C# with normal event - delegate pairs, without losing the advantages of our implementation simplicity of course. Fortunately, C# allows us to change the manner the event handlers are managed through add and remove modifiers. First, to make our event fully-functional, we once again need to do a little preparatory work.

First, let's create a handler class able to give us access to the DHTML event object interface that we need in order to access to some supplemental information about the event we're managing, like mouse click coordinates or the event source object. As we have already said, the event object is unique for a DHTML document; it is updated every time an event is fired. So, to obtain this object, just like we do in DHTML script code, we can get the event property of the window object attached to the document. We'll extend our DHTMLDoc class to make this operation easy by adding a new Event property to it (note the "@" character preceding a marshalled event property, it's a syntax trick avoiding a keyword rules violation):

public IHTMLEventObj Event
{
get{
            return doc2.parentWindow.@event; }
}

Now we can create a new fully functional DHTMLEventHandler class and a corresponding delegate (don't forget to also declare our use of the mshtml and System.Runtime.InteropServices namespaces):

public delegate void
            DHTMLEvent(IHTMLEventObj e);

public class
            DHTMLEventHandler
{
public DHTMLEventHandler(DHTMLDoc
            doc)
{
document=doc;
}

[DispId(0)]
public void
            Call()
{
Handler(document.Event);
}

public
            DHTMLEvent Handler;
DHTMLDoc document;
}

Now we'll create a new DHTMLElement class taking as a model the DHTMLDoc we developed earlier. This class will allow us to encapsulate the generic DHTML element functionality. The first bit of functionality we'll manage will be event handling. In the code below, we'll implement the onclick event template:

public class DHTMLElement
{
public
            DHTMLElement(DHTMLDoc d,object e)
{
if(!(e is
            IHTMLElement))throw new Exception("This is not a DHTML
            element!");
doc=d;
elm=(IHTMLElement)e;
}

public
            event DHTMLEvent onClick
{
add
{
object
            old=elm.onclick;
DHTMLEventHandler
            h;
if(old.GetType()==typeof(DHTMLEventHandler))h=((DHTMLEventHandler)old);
else
            elm.onclick=h=new
            DHTMLEventHandler(doc);
h.Handler+=value;
}
remove
{
object
            old=elm.onclick;
if(old.GetType()==typeof(DHTMLEventHandler))
((DHTMLEventHandler)old).Handler-=value;
}
}

IHTMLElement
            elm;
DHTMLDoc doc;
}

The code for adding and removing event handlers uses the fact that in C# (and in .NET generally) we can compare object types at runtime. So, if we see the DHTML property used as event handler entry point is already set to an object of our class type, we simply use this object, otherwise we create a new one. You can do copy-and-paste to create other events handlers; the only additional work you'll probably have to do concerns querying for special interfaces for the events that don't concern all the DHTML element types (like onclick).

Once this work is done, you can add an event handler to any DHTML element by using only two lines of code, like with DHTML except that in our case, you are sure it works and you can share one delegate type for all your events. To give you an example, we'll create the following handler in our Form1 class:

private void DHTMLEvtClick(IHTMLEventObj e)
{
MessageBox.Show("Click detected at: "+e.x+"/"+e.y,"C# event handler");
}

To attach this to the event, just add these two lines to your DocumentComplete handler:

DHTMLElement elm=new
            DHTMLElement(doc,doc["text1"]);
elm.onClick+=new
            DHTMLEvent(DHTMLEvtClick);

It works! You can now see the coordinates of your mouse pointer when you click on the text zone you attached your handler to:

5 - Launching scripts from C#

If you're interested in Internet Explorer programming, you've probably already written scripts for this platform. You probably even have a library of script code you use in your everyday work. Of course, it's better to translate these scripts in C#: their extensibility and execution speed will improve. However, this kind of work is longwinded and dull, so you should ask yourself how to directly interact with this script code from C#.

Your first idea may be to extract the script code from the page it's written on and execute it using standard Microsoft scripting engine object. The problem is that in this case you lose access to the page's DHTML object model that is integrated when you execute scripts from Internet Explorer, and you don't even have access to the common variables namespace of the page. So, if you want your code executed correctly, you have to directly access the IE internal script engine, and even more precisely, to the instance of that engine attached to one concrete page, because the variable namespaces in Internet Explorer are attached to every particular document object.

DHTML document interfaces give you direct access to the script engine through the Script property. This one can be accessed through the IHTMLDocument and IHTMLDocument2 interfaces (the second inherits from the first, which is rare in this kind of library). The fact that this property is the only thing exported by IHTMLDocument seems to indicate (it's just a guess) that in the "pre-historic" times (i.e. before IE version 4) all the non-script access to the pre-DHTML object model passed by this property, that is to say that the IE contents was only manipulated as underlying scripting objects.

But what kind of object is that? As is often the case in the MSHTML library, its marshaled type is object and the underlying COM is an IDispatch pointer. When we explore this a bit more precisely, we discover that this "automation" COM object has its list of callable methods modified every time a function is declared in the code it interprets. This means that every "function" (if we use JavaScript terminology) can be seen from outside the script engine as an ordinary COM method of the engine object. Thanks to marshalling, we can call it from .NET too!

How do we call a method of an object without knowing in advance which methods this object exports? Doesn't this mean we are unable to declare it in C#? To get around this, we'll use the interoperability technique called "late binding". For any object, you can call the GetType method to return the type object. In the case of objects marshaled from COM, this object contains all the information you need to access any underlying object's method or property at runtime. This is done by simply giving its literal name.

This information should be sufficient to write a little example. First create a simple HTML file with the following contents:

<html>
<head><script>
function Bonjour(){
alert("Hello world");
}
</script></head>
<body>
Test file
</body>
</html>

Save this file in the same directory as your executable (it should be the bin/Debug subdirectory of your project folder) as test.htm. In our C# main form example, change the code of the handler that navigates to our pre-defined home page (normally button2_click). You'll need to load our test HTML file at this point. To do it, you have to modify your Navigate call like this:

Navigate("file:///"+Application.StartupPath+"\\test.htm");

Now, once again, we have to modify our DocumentComplete handler in order to launch a script call when the test file is loaded (you'll also need to add the using System.Reflection directive to your form code file at this point):

if(axWebBrowser1.LocationURL.EndsWith("test.htm"))
{
object
            script=doc.Script;
script.GetType().InvokeMember("Bonjour",BindingFlags.InvokeMethod,null,script,null);
}

In this code, we first get the script object from our document. To do that, the script property in our DHTMLDoc object can be implemented like this:

public object Script
{
get{ return doc2.Script; }
}

The second line is the script call itself, which uses the InvokeMethod method of the Type class. In our case, it is translated to the COM Invoke method call, which in turn finds the method with the same name as the first parameter that was passed in the underlying script object, and then calls it:

Let's look at the InvokeMember function a little more closely. As we've already seen, its first parameter is the name of the method (or property) we are looking for. The second parameter says what we want to do with the named object found. In our case, we consider it is a method, and we call it. We don't really need the third parameter at the moment, and the fourth is the reference to the object on which we are making the call (don't forget InvokeMember is a function of the Type class, not the object 's). Most interesting to us is the fifth parameter. It allows passing parameters to the function we are calling. Its marshalled type is object[], which is equivalent to VARIANT SAFEARRAY in COM. SAFEARRAY is a COM standard type representing a resizable array that is capable of handling its size and the types of its elements.

So, when we need to pass parameters of some type to a script function, we just have to box the values in the object s and put these objects in an array. The function's return value can be retrieved as an InvokeMember 's return object value.

What about parameter and return types? C# is a strictly-typed language, and JavaScript, for example, has no types at all. More precisely it has one data type, the VARIANT. This type, marshalled as an object to .NET, is a sort of union structure to which we can write almost any data type. The condition is to always specify the type of the data that the variable holds. The mechanism is very similar to .NET boxing, that's why it allows a real equivalence between VARIANT COM variables and .NET object s.

An integer value in script will be translated to .NET as Int32, and a string as a String object. We can clearly show this by calling a simple JavaScript function which we'll add to our test.htm file:

function Sum(a,b){
return a+b;
}

We'll call this function twice to observe its behaviour when treating different variable types:

string show="";
object
            retval=script.GetType().InvokeMember("Sum",BindingFlags.InvokeMethod,null,
script,new
            object[]{ 2, 2 });
show+="Value : "+retval+", type :
            "+retval.GetType().ToString();
retval=script.GetType().InvokeMember("Sum",BindingFlags.InvokeMethod,
null,script,new
            object[]{ "2", "2" });
show+="\r\nValue : "+retval+", type :
            "+retval.GetType().ToString();
MessageBox.Show(show,"C# script
            call result");

The call result is self-explanatory:

That works fine with standard variable types. However, don't try to simply treat a JavaScript array as an object[]. It is not one, because in COM it is not directly encapsulated in a Safearray, which is equivalent to a .NET array. Marshalling arrays and other structured object types is a complex subject; it would take another case study entirely to properly explore it correctly.

The last thing we can do is encapsulate the script call to an easy-to-use function integrated in our simple object model. In C#, such function writing is straightforward; you just need to add the following lines to your DHTMLDoc class (after adding using System.Reflection to the .cs file headings, of course):

public object ScriptCall(string method,params object[] prm)
{
return Script.GetType().InvokeMember(method,BindingFlags.InvokeMethod,null,Script,prm);
}

We can use this function to implement a scripting functionality that we'll probably need: until now, we've always called functions, without trying to execute true code fragments. This last task is very easy, because we can directly access any standard script functions just like those we've written ourselves. Amongst these standard functions there is one called eval that simply executes the code passed as its only string parameter. We simply add one more function to the DHTMLDoc class to do it:

public object ScriptEval(string
            expr)
{
return ScriptCall("eval",expr);
}

That's all! If you now want to implement a more complex scripting model, including variables, complex objects management and more, it could be better to move the scripting functionality from the DHTML document wrapper to a separate encapsulating object, which can be, for instance, returned as a new value of the Script property of the DHTMLDoc class.

6 - Extending the DHTML model

In the previous section, we've seen how to use script code from C#. Now we'll try to go the other way, calling C# code from script. You might need it, for example, to give your script development staff (if they've not yet migrated to .NET) direct access to some system functionalities you implemented.

If you don't want to be subjected to the security constraints and other organizational issues related to the use of components directly integrated to HTML pages, the only "legal" way to extend the DHTML object model your environment uses is to implement the window.external object.

When a DHTML document is loaded in Internet Explorer, it exports a special interface called ICustomDoc. This exposes the SetHandler method, getting a reference to a "container" object as a parameter.

What kind of "container"? It is a COM object communicating with the outer world through the IDocHostUIHandler interface. There are a number of browser's behaviors, like a reaction to a shortcut key press or a contextual menu display that can be redefined through this interface. Before performing such an action, the Internet Explorer engine checks for the existence of an object set by a SetHandler call and by implementing IDocHostUIHandler, then calls the method that corresponds to the behavior in question. One of these customizable processing properties is the reference to the object that implements methods we can put behind the window.external scripting object. The Explorer object asks for this property by calling the GetExternal method of the IDocHostUIHandler interface.

The big problem is that the ICustomDoc and IDocHostUIHandler interfaces are not implemented in the MSHTML .NET library, so there is no way to directly treat them as .NET interfaces. That means we have to look at the interfaces' COM implementations and convert them to C#.

First, we'll try implementing and using the ICustomDoc interface; it seems to be a little simpler than the other one. Searching in the C++ COM headers (MsHtmlHst.h) we find the following definition:

MIDL_INTERFACE("3050f3f0-98b5-11cf-bb82-00aa00bdce0b")
ICustomDoc : public IUnknown
{
public:
virtual HRESULT STDMETHODCALLTYPE SetUIHandler(
/* [in] */ IDocHostUIHandler *pUIHandler) = 0;

};

We'll simply transcribe the same thing in C#, applying interoperability rules. C# attributes let us specify three important things that we can observe when reading a C++ definition. First, the interface we declare is a COM interface, so we need to use the interoperability engine to deal with the classes that implement it. Second, it is an IUnknown -style object, it is not an automation (IDispatch -style) interface. Third, and most important for us, we give the GUID (globally unique identifier) to identify this interface.

Create a new C# class file, name it UIHandler.cs, and declare in its headers the use of the System.Runtime.InteropServices, System.Windows.Forms and mshtml namespaces (we'll also use this class file for the other classes of this section). First make the following interface declaration, which is the C# transcription of the earlier C++ statement:

[ComImport(),
            InterfaceType(ComInterfaceType.InterfaceIsIUnknown),
GuidAttribute("3050f3f0-98b5-11cf-bb82-00aa00bdce0b")]
public
            interface ICustomDoc
{
[PreserveSig]
uint
            SetUIHandler(IDocHostUIHandler pUIHandler);
}

We saw earlier that this interface is exported by the DHTML document object. In .NET, querying for an exported interface of an object simply means converting this object type to this interface type. So, if we want to obtain this interface from our document, we should simply put the following line at the beginning of the DocumentComplete header:

ICustomDoc cust=(ICustomDoc)(axWebBrowser1.Document);

How can we convert the object to a type which has no connection from the point of view of the C# language declarations? As the type is not referenced in the .NET libraries managing Internet Explorer functionalities, there should be a casting error at this point. In reality, there isn't. Don't forget we're working with interoperability libraries. So, our ICustomDoc interface is rightly considered to be related to a COM object (remember, we put the ComImport attribute in its declaration).

That means querying this interface is simply translated to a COM QueryInterface call (asking the underlying COM object for an appropriate interface) for which the only important thing is the interface's GUID. But we already declared this GUID in the C# interface's attributes! So, when we're casting this interface type, the interoperability engine connects through a COM interface identified to the underlying object and maps its methods to the corresponding .NET interface signature.

You can declare and use any COM object accessible via its interfaces in this way. You can even create your own definitions for the DHTML objects you'll use after reading this case study. Sometimes, you'll even need to do it: not all the interfaces in MSHTML are well implemented. Be aware however, the GUID corresponding to every interface is the most important thing, but when describing the interface itself, you also must follow (even if you use attribute modifying marshalling rules) the order of property and method declarations, and you must preserve the general interface signature, i.e. the names of the interface members and at least the size of their arguments and return values types.

We can now describe the IDocHostUIHandler interface which is relatively more complex, but the declaration work in itself is straightforward. We just have to translate the definitions between C++ (the same MsHtmlHst.h header we used to analyze ICustomDoc) and C#. We first declare the enumerations and structures the interface uses in order to preserve the "COM style" (i.e. the enumerations values and secondary type names), when implementing the interface:

public enum HRESULT : uint
{
S_OK = 0,
E_UNEXPECTED = 0x8000ffff,
E_NOTIMPL = 0x80004001,
E_OUTOFMEMORY = 0x8007000e,
E_INVALIDARG = 0x80070057,
E_NOINTERFACE = 0x80004002,
E_POINTER = 0x80004003,
E_HANDLE = 0x80070006,
E_ABORT = 0x80004004,
E_FAIL = 0x80004005,
E_ACCESSDENIED = 0x80074005
}

[ StructLayout( LayoutKind.Sequential )]
public struct DOCHOSTUIINFO
{
uint cbSize;
uint dwFlags;
uint dwDoubleClick;
[MarshalAs(UnmanagedType.BStr)] string pchHostCss;
[MarshalAs(UnmanagedType.BStr)] string pchHostNS;
}

[StructLayout( LayoutKind.Sequential )]
public struct tagMSG
{
IntPtr hwnd;
int lParam;
uint message;
tagPOINT pt;
uint time;
uint wParam;
}

Now, let's declare the interface itself. The only method we'll have to directly deal with now is GetExternal ; however, as it was shown earlier, the interface signature absolutely has to be preserved, so we need to declare all the methods that can be implemented for this interface. Note the use of the PreserveSig attribute for every interface member. This is very important, because it is the only possibility we can use in order to have a direct correspondence between COM and .NET method declarations. Without this attribute, the .NET return value is translated, if it isn't void, to the last argument of the marshaled COM function, this argument being simply referenced as retval -style.

In our case, we need to return a real COM return value, which is of the HRESULT enumeration type and which normally indicates the success of function call. This is assumed to always be successful when dealing with .NET interoperability without specifying PreserveSig.

The complete interface declaration is as follows:

[ComImport(), InterfaceType(ComInterfaceType.InterfaceIsIUnknown),
GuidAttribute("bd3f23c0-d43e-11cf-893b-00aa00bdce1a")]
public interface IDocHostUIHandler
{
[PreserveSig]
HRESULT ShowContextMenu(uint dwID, ref tagPOINT ppt,
[MarshalAs(UnmanagedType.IUnknown)] object pcmdtReserved,
[MarshalAs(UnmanagedType.IDispatch)]object pdispReserved);
[PreserveSig]
HRESULT GetHostInfo(ref DOCHOSTUIINFO pInfo);
[PreserveSig]
HRESULT ShowUI(uint dwID, [MarshalAs(UnmanagedType.Interface)] IntPtr
pActiveObject, [MarshalAs(UnmanagedType.Interface)] IntPtr
pCommandTarget, [MarshalAs(UnmanagedType.Interface)] IntPtr pFrame,
[MarshalAs(UnmanagedType.Interface)] IntPtr pDoc);
[PreserveSig]
HRESULT HideUI();
[PreserveSig]
HRESULT UpdateUI();
[PreserveSig]
HRESULT EnableModeless(int fEnable);
[PreserveSig]
HRESULT OnDocWindowActivate(int fActivate);
[PreserveSig]
HRESULT OnFrameWindowActivate(int fActivate);
[PreserveSig]
HRESULT ResizeBorder(ref tagRECT prcBorder,
[MarshalAs(UnmanagedType.Interface)] IntPtr pUIWindow, int
fRameWindow);
[PreserveSig]
HRESULT TranslateAccelerator(ref tagMSG lpmsg, ref Guid pguidCmdGroup,
uint nCmdID);
[PreserveSig]
HRESULT GetOptionKeyPath(ref string pchKey, uint dw);
[PreserveSig]
HRESULT GetDropTarget([MarshalAs(UnmanagedType.Interface)] IntPtr
pDropTarget, [MarshalAs(UnmanagedType.Interface)] ref IntPtr
ppDropTarget);
[PreserveSig]
HRESULT GetExternal([Out,MarshalAs(UnmanagedType.IDispatch)] out object ppDispatch);
[PreserveSig]
HRESULT TranslateUrl(uint dwTranslate, ref ushort pchURLIn, IntPtr
ppchURLOut);
[PreserveSig]
HRESULT FilterDataObject(IDataObject pDO, ref IDataObject ppDORet);
}

Once the interface is declared, we have to implement it. The only method we really need to implement now is GetExternal. However, if we want to have a class we can effectively instantiate as a result, we have to implement all the interface's methods.

This is exactly the reason for preserving the COM HRESULT return values of the interface's methods. For the methods implementing the behaviors we don't want to change, we have to specify to the caller that these are not customized, by returning the HRESULT.E_NOTIMPL value. Otherwise, Internet Explorer will think that our interface has already done all the work needed and will also block its default behaviors, for example by showing no context menu at all when the user right-clicks the browser window.

If you now look at the GetExternal method declaration, you'll see that the only thing it is intended to do is set its unique argument's value to the pointer to an object (IDispatch, which normally means any .NET object marshaled to COM) implementing the methods we'll access later from script through the window.external reference. Of course, if we have no such pointer, the return value must indicate that the behavior is not implemented. So, if we integrate the pointer, we need to return GetExternal to the class construction logic. Our implementation should look like this:

public class UIHandler : IDocHostUIHandler
{
protected IExternal ext;

public UIHandler(){ ext=null; }

public UIHandler(IExternal e){ ext=e; }

public virtual HRESULT ShowContextMenu(uint dwID, ref tagPOINT ppt,
[MarshalAs(UnmanagedType.IUnknown)] object pcmdtReserved,
[MarshalAs(UnmanagedType.IDispatch)]object pdispReserved)
{
return HRESULT.E_NOTIMPL;
}

public virtual HRESULT GetHostInfo(ref DOCHOSTUIINFO pInfo)
{
return HRESULT.E_NOTIMPL;
}

public virtual HRESULT ShowUI(uint dwID, [MarshalAs(UnmanagedType.Interface)]
IntPtr pActiveObject, [MarshalAs(UnmanagedType.Interface)] IntPtr
pCommandTarget, [MarshalAs(UnmanagedType.Interface)] IntPtr pFrame,
[MarshalAs(UnmanagedType.Interface)] IntPtr pDoc)
{
return HRESULT.E_NOTIMPL;
}

public virtual HRESULT HideUI()
{
return HRESULT.E_NOTIMPL;
}

public virtual HRESULT UpdateUI()
{
return HRESULT.E_NOTIMPL;
}

public virtual HRESULT EnableModeless(int fEnable)
{
return HRESULT.E_NOTIMPL;
}

public virtual HRESULT OnDocWindowActivate(int fActivate)
{
return HRESULT.E_NOTIMPL;
}

public virtual HRESULT OnFrameWindowActivate(int fActivate)
{
return HRESULT.E_NOTIMPL;
}

public virtual HRESULT ResizeBorder(ref tagRECT prcBorder,
[MarshalAs(UnmanagedType.Interface)] IntPtr pUIWindow, int
fRameWindow)
{
return HRESULT.E_NOTIMPL;
}

public virtual HRESULT TranslateAccelerator(ref tagMSG lpmsg, ref Guid
pguidCmdGroup, uint nCmdID)
{
return HRESULT.E_NOTIMPL;
}

public virtual HRESULT GetOptionKeyPath(ref string pchKey, uint dw)
{
return HRESULT.E_NOTIMPL;
}

public virtual HRESULT GetDropTarget([MarshalAs(UnmanagedType.Interface)]
IntPtr pDropTarget, [MarshalAs(UnmanagedType.Interface)] ref IntPtr
ppDropTarget)
{
return HRESULT.E_NOTIMPL;
}

public virtual HRESULT GetExternal([Out,MarshalAs(UnmanagedType.IDispatch)] out object ppDispatch)
{
if(ext==null)
{
ppDispatch=null;
return HRESULT.E_NOTIMPL;
}
ppDispatch=ext;
return HRESULT.S_OK;
}

public virtual HRESULT TranslateUrl(uint dwTranslate, ref ushort pchURLIn, IntPtr ppchURLOut)
{
return HRESULT.E_NOTIMPL;
}
public virtual HRESULT FilterDataObject(IDataObject pDO, ref IDataObject ppDORet)
{
return HRESULT.E_NOTIMPL;
}
}

The last thing we have to do is to implement the object we'll access through window.external and to connect it to the browser by using our new UIHandler object.

The C++ documentation concerning the GetExternal method indicates that the object whose pointer it returns has simply to implement the IDispatch interface to be usable. However, normally, all the C# classes, when marshaled to COM, are IDispatch type. Does it mean we can pass any C# object pointer to the constructor of the UIHandler class to make its methods and properties accessible through window.external ? It seems logical, but it doesn't work. The problem is that if we want an object behaving as a true automation object, it's not enough to implement IDispatch, it's also necessary to implement it well. It means we have to force our object to export an interface with all its methods and properties declared.

There is a trick in C# that allows us to easily do it. It's enough to declare an empty interface, declare it as IDispatch -type, and implement it in any class. This effectively makes all this class's methods exported and accessible to COM. The interface declaration may look like this:

[InterfaceType(ComInterfaceType.InterfaceIsIDispatch)]
public interface IExternal{}

Any class implementing this interface will be considered, at the COM point of view, as a fully-featured automation interface. This means the interoperability engine will be forced to analyze the class's signature and to add, at the COM side, an automatic implementation of everything an automation client needs to access the methods.

We need all this because the IE script engine uses the 'late binding" method; the relation between a concrete property or method name and its implementation's address is established at run time, so we must force the interoperability engine to avoid any shortcuts in the name translations.

We can now declare any class (even our main application's form) as implementing this interface, and then use all its methods from the script. As an example, we'll implement a simple class with one method implementing a message box, allowing the script user to define text for both the box and its header:

public class CExternal : IExternal
{
public void Alert(string text,string head)
{
MessageBox.Show(text,head);
}
}

This class implements our empty interface declared as InterfaceIsIDispatch, so when marshaled to COM, it is seen as a new automation object interface. The function asking at runtime for its member's addresses returns, when it receives the "Alert" string, the entry point allowing it to call the right function.

Note that we have absolute freedom, as there are no methods or properties declared in the interface implemented; we could do everything we want in the implementing class, every method or property we declare will be marshaled and its name will be immediately available to use for any automation client.

To connect the interface handler to the browser, we have to add the following code after the line requesting the ICustomDoc interface we wrote at the beginning of this section:

cust.SetUIHandler(new UIHandler(new CExternal()));

The constructor of the UIHandler class (the class that implements IDocHostUIHandler) receives as an argument an address of a new instance of the class designed to be used in the GetExternal function. The result of all this is passed to the appropriate method of the ICustomDoc interface which declares our new object as a new "container" authorized to modify the browser user interface's behaviour.

Now we'll change our test.htm file to test our implementation. We add the following DHTML code to its body:

<hr/>
<button onclick="window.external.Alert('Extending DHTML','window.external')">
Test window.external</button>

You can now compile your application and run a test by pushing the button on our "home" page:

As you noticed when looking at the IDocHostUIHandler interface, implementing extensions for the DHTML object model is only one of the numerous functionalities you can modify by using it. So, all the work done will serve you in the future as a starting point for writing complex applications using all the power of the Internet Explorer programming platform.

Case study review

As was stated at the beginning of this case study, the subjects covered here are just intended to give you a starting point to begin a serious study of the Internet Explorer browser platform from the C# point of view. The possibilities offered by the platform are endless. The fact that the (D)HTML platform is very popular thanks to the general success of the Web makes this platform very attractive to user interface designers, so the programmers who have to manage application presentations layers should consider this platform as a serious alternative to traditional Windows-based interface development. Deep knowledge of the leading DHTML presentation platform is important for both standalone and online applications programming; the better we know the client interface capabilities, the better the final user's experience and the wider the success of your software.

Programming Internet Explorer is an extremely wide topic; it can't really be covered by a single case study. However, this piece explains the main starting points you need to explore first. Looking at the browser as a .NET component, we observed it's most important functionalities related to internet navigation. Going further in this direction, and studying all the methods and properties exported by the component, we can create a personalized framework implementing almost all the functionality accessible through the standard Internet Explorer interface, and even make it more flexible \ better adapted to your particular tasks.

The true advantage of the platform is the access it gives to the DHTML object model. Many years of development at the front edge of new technologies has made it perfectly adapted to developing any kind of user interface, and especially those concerning document-based systems. Having read this case study you have learned how to access this model; you followed the development of basic classes encapsulating its most important objects in the .NET manner. Now, you can continue its exploration, extending these classes and using them: you'll be surprised how easy is it to write DHTML code in C# once some basic interfaces and properties have been encapsulated. By putting together this case study's information about accessing DHTML, and reading the widely accessible documentation about DHTML scripting, you can combine the ease of the Internet Explorer presentation model with the power of the .NET platform in your applications.

Finally, the example of the DHTML model extension is just the first step into the world of advanced Internet Explorer interfaces. Almost everything in the IE-related objects' behaviour can be personalized. Now, .NET gives access to this power to all the industrial programmers who don't have the time to write and debug low-level C++ code, but who wish to create really powerful interfaces

Further work

If you're courageous enough to explore a library containing hundreds of interfaces, you can open the Object Browser window in a project which references the MSHTML library. Open the microsoft.mshtml/mshtml node and take a look:

Sometimes, by clicking on an interface or on an interface's function you can get a small piece of helpful text on it; however, the true .NET documentation on the subject is actually non-existent! You'll be better off using the corresponding C++ help. All the interfaces and function names are the same.

There are many other interesting domains to explore. It could be the markup interfaces system, giving you direct access to the engine Internet Explorer uses to interpret and display HTML code, or it could be the elements' behaviors, Internet Explorer automation, style sheets, helpers, or web accessories. The possibilities of the platform are endless. Enjoy it!

C#Today - ©2002 Wrox Press Limited


Creating Data-Driven Enterprise Applications using C# and SQL Server

Creating Data-Driven Enterprise Applications using C# and SQL Server
http://www.csharptoday.com/content.asp?id=1955
Article Rating: 5
Published on October 28, 2002
Abstract
This case study by Mark Plummer describes how to go about designing and implementing the data tier of an enterprise application using C# and SQL Server. It concentrates on a single database server scenario (although multiple server implementation is also looked at along the way), and bring together the new features of .NET with proven methodologies to create a high performance, scalable transactional data tier.
Case Study

This case study describes how to go about designing and implementing the data tier of an enterprise application using C# and SQL Server. It will concentrate on a single database server scenario (although multiple server implementation is also looked at along the way), and bring together the new features of .NET with proven methodologies to create a high performance, scalable transactional data tier.

Introduction

Most enterprise applications interact with a database. Very few of these enterprise applications are the same, but the database interaction is strikingly similar in each case. Yet many of these applications suffer from poor performance, low scalability, inability to use transactions correctly, and generally poor design. This case study will set out to show the reader what developers have been doing wrong, and what can be done to remedy these mistakes.

Several methodologies have emerged in recent years, but it is only fairly recently that one has emerged on top. Unfortunately, this is only described in terms of complex VC++, so I would like to 'convert' the relevant ideas to C# and the .NET environment.

What the reader will gain from this piece is an easy-to-implement methodology that can be applied over and over to reap the rewards of performance, scalability, and transactional support, and exert a sense of uniformity over his or her data tier.

System Requirements

SQL Server 2000 and the .NET framework are mandatory. A good C# editor like Visual Studio .NET is highly recommended. A good understanding of ADO or ADO.NET would help, although a lot will be explained as we go along. As this is a C# case study, an intermediary knowledge of C# is required. A general understanding of databases and database theory is highly recommended.

Defining The Problem

Almost every data tier that I have looked at has been designed and implemented differently, and that is from a methodology that is almost eight years old. With the recent emergence of .NET and its new way of approaching data access, the amount of permutations of data tier design and implementation is likely to be extremely high. These permutations will lead to inconsistencies in performance, scalability, transactional support, and integration with other systems

The Business

When a firm wants to create, extend or update just about anything in IT, they will generally make use of a business analyst. The business analyst's role is to extract the requirements of the business and business users and present them in a structured and logical manner. Although a lot of them are, a business analyst is not required to be highly technical. Add to that the fact that most business users are not technical experts and do not think about their job in a technical or pedantic manner like techies do, and you create a potential situation where you have a logical business document going straight to the programmers.

Add to this the fact that many concepts common to business people are either unknown or incomplete to the technical people. For example, assume the business states that: "All employee's addresses are to be recorded". Does this mean that an employee can have more than one address? How many fields does the business want in the address? What are their names? Is the post code a required field? If so, what happens if an employee does not know his post code? Can he save the rest of the details in the interim? As you can see, even a simple concept such as an address can have many questions attached to it from a technical perspective that can easily go unnoticed from a business perspective. Thus:

"All employee's addresses are to be recorded",

can easily turn into (probably after many versions and iterations):

Definition: Address

An address records a unique place in the world and consists of the following fields:

Address Name (50 character Unicode string)
Flat Number (10 character nullable Unicode string - [E.G. 52A])
House or Flat Name (50 character nullable Unicode string)
Street Number (10 character nullable Unicode string - [E.G. 52A])
Street Name (50 character Unicode string)
Locality (50 character nullable Unicode string)
City (50 character Unicode string)
County (50 character nullable Unicode string)
Country (50 character Unicode string)
Post Code (10 character Unicode string)

Rule: Employee addresses

Employee's addresses are to be captured as soon as their postcode and street name
are known. An employee may have zero, one, or more addresses and each must be named.
(e.g Residential Address, Postal Address, etc)

What is often lacking in these situations is a technical analyst - someone who can take the business analyst's documents and translate it into technical documents for the IT department. This technical analyst can save many round trips from the IT department to the business users and back again by accompanying the business analyst where appropriate. Better still is to have a person who can play both roles.

What I am trying to demonstrate here is the need to be pedantic and thorough during the design phase as it will not only save you money, but it will save you time as well.

The Different ways of Storing and Retrieving Data

Communication with a database can be broken down into two steps. Firstly, you send some sort of request to the database, and secondly, you optionally get some data returned to you. The data that is returned (apart from any error codes) can be a combination of a return value, a set of zero or more output parameters, and a set of zero or more result sets (also known as record sets, record arrays, data sets, or tables).

In the past you had to explicitly manage a raw memory buffer using C/C++ via the ODBC and later the OLEDB libraries to get information to and from the database. Microsoft has, during the years, provided wrappers for this in the form of DAO, RDO, ADO, and now ADO.NET, but in essence they are merely handling the buffers behind the scenes. You achieve ease of use in return for a minimal loss of performance.

The fastest way to get data back from a database is by using output parameters and/or the return value. The return value is simply a mandatory integer output parameter that accompanies each and every stored procedure. It has historically been used to return error codes to the caller, or the number of rows affected by the last database operation. It is a highly underused parameter; it is either being ignored by developers as an error code, or it is being ignored when returning the number of rows affected by the last database operation. (If you wish to increase performance you can tell SQL Server not to keep returning the number of rows affected - and hence make use of the return value yourself - by using SET NOCOUNT ON. To reverse this, use SET NOCOUNT OFF.)

Since the return value is a 'built-in' parameter, it should offer a slight performance advantage over an output parameter. Otherwise, the return value and output parameters of a stored procedure can be compared to the return value and output parameters of a normal function in terms of performance. The disadvantage of using the return value, however, is that it is not plainly obvious to spot it in the data tier or the header of the stored procedure.

Returning data in the form of a result set is considerably slower than using output parameters. (I have seen people use multiple result sets to get back two integer values, and they wondered why their application was slow!) Later in this case study I will provide some metrics (from MSDN) to show the performance comparisons of the various ways of retrieving data.

If you need to perform several operations and only have those operations saved permanently if all of them succeed, you will need to use a transaction. Using a transaction, you can modify the database as much as you want, and as soon as something goes wrong, you can tell the database to undo everything that you have done. Transactions help to maintain the integrity of the data, and help to adhere to the well-known ACID test (Atomicity, Consistency, Isolation and Durability).

Generally, Atomicity refers to the fact that you are attempting an atomic unit of work, which means that from a business point of view the transaction's work cannot and should not be able to be broken down.

Consistency means that each transaction will leave the data in a consistent state and so data integrity is preserved.

Isolation refers to the fact that one transaction should not and will not be affected by another transaction happening at the same time. They are logically ignorant of each other.

Finally, Durability refers to the fact that if a transaction succeeds, it is guaranteed to persist itself, even if the machine crashes. Special log files are used for this.

Transactions are a must for enterprise applications as data integrity is essential for accounting, legal, and service reasons.

The History of the Data Tier

In the Beginning...

Data tiers never used to exist. Instead, there was just a mammoth application that contained the data store, the data retrieval code, the business logic, and the presentation code. With the advent of ODBC, some application developers began separating the database from the application. Even then, a lot of application developers kept vast amounts of business logic embedded in the database.

Tiers Emerge

With the emergence of Microsoft DNA (i.e. n-tier architecture) the application was divided into a presentation tier, a business logic tier, a data tier, and the database. This methodology has largely stuck, albeit with a few minor changes, which we will look at shortly.

With the database now not guaranteed to be in the same process, nor on the same machine as the data tier, a connection is required. This is essentially an object residing on the data tier machine loaded from the database library (located on the data tier machine) that can locate the database via a connection string and talk to it over the network (or internet).

Connections are expensive, both in resources and in monetary terms. Having one connection per client for thousands of internet users connecting to your database is not advisable from a performance or budgetary perspective. Additionally, having the connection wait around while the user is idle is simply a waste of resources. For these reasons, Microsoft implemented connection pooling, whereby closing a connection simply returned that connection to a pool. When you then create a new connection, the pool will be searched first, looking for a match against your connection string (an exact character match is required!). Developers were then encouraged to close connections as soon as possible to gain performance and scalability.

MTS is born

With regards to transactions, originally there was limited library support and the onus was on the developer to create their own complex frameworks. The arrival of Microsoft Transaction Server (MTS) was heralded as the best thing since sliced bread, and many people jumped on the bandwagon. MTS was easy to use in the sense that there was no complex parameter passing required to keep track of transactions. You merely called a method on a global object and effectively set none, one, or both of two bits: a 'done' and a 'happy' bit. You set the 'happy' bit if you deemed that your method had succeeded, and set the 'done' bit to say that you were finished. MTS took care of the rest. When the 'root object' (the object at the beginning of the call chain) exited, MTS would intercept it, check the 'done' and 'happy' bits, and commit or abort (rollback) the transaction accordingly.

COM+ takes over

COM+ extended MTS by using attributes (yes, attributes). From the Component Services Explorer (CSE), you as the administrator (or developer) could set the transactional requirements of a class, and could also set whether the 'done' bit is set on return of a method.

It is important to note that these transactions used by COM+ and MTS are distributed transactions and not local transactions. This means that they require a 'transaction coordinator', like Microsoft's Distributed Transaction Coordinator (MS DTC) to coordinate the transaction between several different databases (or other XA compliant devices), even if there is only one database involved in the transaction.

COM+ also completed the unfinished parts of MTS, namely object pooling. With object pooling, COM+ stores a pool of objects rather than destroying and recreating them. When you need a new object, COM+ will first look in the pool for one. It was discovered by Microsoft that using object pooling with objects that contained a connection was twice as fast as using connection pooling, so a lot of C++ programmers now use object pooling for their connections (note that object pooling is not available in VB6). The only 'extra' to watch out for is that you will have to manually enlist your newly activated object in a transaction if you require its participation, as COM+ cannot do this for you automatically. However, COM+ does store deactivated objects that were in a transaction in their own 'mini pool' and will hand you one of these first if the same transaction requires a new object of that type. (Thank you to Joe Long at Microsoft for pointing that one out to me.)

.NET Arrives

The .NET framework library lets you use all of the COM+ Services by providing attributes and the classes in the EnterpriseServices namespace. It also provides OLE DB support as well as native SQL Server classes via the OledbClient and SqlClient namespaces respectively (part of ADO.NET).

SqlClient talks directly to SQL Server through dbnetlib.dll using the TDS protocol (Tabular Data Stream) and is the fastest (and recommended) way to communicate with SQL Server. It is handy to remember that ADO.NET datasets are completely disconnected, whereas its predecessor's recordsets generally were not, or were tricky to implement. It is useful to note that the .NET framework library does not seem to support manual enlistment into distributed transactions. For that, one would have to use PInvoke or COM InterOp.

SqlClient In-Depth

Armed with Microsoft's IL Disassembler (ILDasm.exe), I decided to forage around the guts of the SqlClient namespace in System.Data.dll. To summarize my findings: SqlClient basically uses the SqlConnection class to connect to the database and to begin and end transactions, the SqlCommand class to capture the user's requests for data in a structured manner, and the SqlTransaction class to handle committing or rolling back the transaction. Several other classes exist (like the SqlDataReader class) to turn an internal buffer of returned data into an easily used object.

When you begin a transaction using the SqlConnection class' BeginTransaction method, all that happens under the covers is a ' SET ISOLATION LEVEL XXX; BEGIN TRANSACTION ' T-SQL statement is sent to SQL Server, where XXX is the isolation level that you have set, or 'Read Committed' as the default. (The method does make sure that the connection is still open, that no existing SqlDataReader classes are attached (as they block the connection), and that no old or existing transaction exists).

When you call any of the ExecXXX methods on the SqlCommand class, they all internally call ExecuteReader, which uses an internal class called TdsParser to turn all the nicely structured information that you have placed into the SqlCommand object into a buffer to send to SQL Server. If data is returned, it is then placed into another buffer to be used by one of the SqlClient data container classes if required. It is useful to note that output parameters and return values are placed at the end of the buffer, so if any result sets are returned, they must be read first before these values can become available (they will appear as null until this time).

The SqlTransaction class merely emits ' COMMIT TRANSACTION ' and ' ROLLBACK TRANSACTION ' T-SQL statements to SQL Server when the Commit or RollBack methods are called respectively. Since other parts of the SqlClient namespace might roll the transaction back internally if an error occurs, this Rollback method could throw an exception if you call it after the transaction has been rolled back internally. If this occurs, you are guaranteed that the transaction has been rolled back.

If you wish to participate in a transaction, you will need a reference to the SqlTransaction object that you got from calling BeginTransaction on the SqlConnection object, and you will need a reference to that SqlConnection class. Armed with these, you can use a SqlCommand object (a new or existing one) by setting its Connection and Transaction properties to the two references you obtained above.

The Past Mistakes of the Data Tier

Before distributed programming took off, there was sparse information, methodology, or metrics available about the implications the design of a data tier had on the performance and scalability of the application. Good object orientated (OO) application programmers were often given the task of designing n-tier distributed applications, and were put in charge of, amongst other things, the data tier. Having no previous commercial experience of databases, and armed only with their years of server development experience, they set off to design an elegant data tier. They were treading on new ground, and so a few (now well recognized) common mistakes emerged:

Get Next

Linked lists are a common structure in programming; they are a structure that has a pointer to both its current data and next element. With that, one could write an elegant loop that would do something with the data, get the next element in the list, and start again. Some programmers used this technique as an elegant looking solution to the data tier and designed the database tables in the same way, not realizing that the databases had been designed, and hence optimized, for sets and set operations. The result was an exponential increase in the amount of trips to the database, which in turn had a huge negative effect on performance.

Sticking with strict OO

At the time, many OO programmers held OO above all else. They lived by it, and they would not compromise. For example, if they had n database queries to call in a row, and none of the queries returned any results, a batch job could be created in one string to send to the database. This would mean one trip to the database. The OO programmers, on the other hand, would stick to their guns that they must maintain their structure and set each one of the queries up as a separate class each with its own method to run itself. The result would be n trips to the database instead of one.

Using a bazooka to kill a fly

When MTS (and later COM+) came out, it was seen as the way to handle transactions. It could handle transactions involving any number of different databases (and other devices too) using a device called a DTC (Distributed Transaction Coordinator). This was fine if you were using more than one database, but most businesses weren't. So what was the impact of DTC on an application using only one database? It actually decreased performance!

DTC uses a methodology called 'Two-Phase-Commit' where each participant records a log of what it would actually do were it to commit. If all the participants succeed, then DTC tells each participant to commit what they have logged. With an application consisting of only one database, this is effectively like doing the transaction twice, consuming time and resources along the way!

Failing to keep order

One mistake made by data tier implementers is to treat each stored procedure as a separate entity, and not look at them as a whole. The result is that resources (tables, etc) get acquired in many different orders. This can lead to deadlocks where stored procedure A has acquired resource X and is waiting on resource Y, because stored procedure B has acquired resource Y and is waiting on resource X. The database then has to intervene and destroy one of the transactions. In the mean time, unnecessary resources have been held for a long time period, decreasing performance and scalability.

Isolation Level versus Performance

The default isolation level for MTS and COM+ is Serializable. The added guarantee that this level offers over the others is that it will not allow Phantoms. In other words, this isolation level will not allow the situation where the transaction reads a set of records twice and gets different results because another transaction inserted or deleted records from this set.

SQL server uses a concept called 'Key Range' locks to enforce this level. It effectively takes the largest and smallest primary key values of your set and locks all the possible records in between, whether or not they are in your set or even exist!

If your primary key is an identity column (which is merely an internal id and has no real business bearing) and a transaction brought back ID numbers 1 and 999, then all records with ids greater than zero and less than 1000 would be locked. This would have a severe impact on the performance and scalability of your application, as this is not what you intended in the first place.

A caveat

A caveat to this is the fact that reading data often requires a higher degree of isolation than writing data. Citing an example from Tim Ewald's Transactional COM+ (pg 260):

You have two tables participating in a parent-child relationship. The parent table stores order details and the child table stores line items for the order. A foreign key constraint is in place to maintain data integrity.

You have a transaction (A) that:

  • Deletes all the line items for an order
  • Deletes that order

The order of deletion must be so because of the foreign key constraint. Next you have another transaction (B) that:

  • Reads all the line items for an order
  • Reads that order

The order of reading must be so to avoid deadlocks.

Now if, for the same order, transaction B executes step 1, then transaction A executes step 1 and step 2, then transaction B executes 2, you will end up with transaction B returning line items but no order. What you actually need to do is to wrap transaction B in a Repeatable Read Isolation Level. Many developers have failed to realize this.

Identity Columns and Clustered Indexes

Index design, which some see more as an art than a science, was largely left until complaints came in about performance. Many developers simply stuck with the defaults or made all their identity columns the primary key using a clustered index. What this means is that the physical data on disk gets stored in that order. Since identity columns are simply auto incrementing numbers, every new row will get inserted at the end of the table. Although this sounds like a good idea, it can actually be quite harmful with respect to performance. If one client is trying to insert a few rows at a time, all this activity is taking place at the same place in the physical table, and SQL Server would then try to optimize by locking the entire page for the client. Any other client would then be blocked, which can have a snowball effect on the server.

Long Running Transactions

When a transaction acquires a lock on a resource (be it a row, page, table, etc) it effectively blocks all other transactions from using it, causing them to wait. The number of permutations of waiting dependencies that even a small number of transactions can have can easily grow exponentially. It is therefore crucial not to hold onto a resource for any more time than is absolutely necessary. Yet many developers have had to wait for transactions while the application is waiting on user input (which in CPU time is almost an eternity). If the user went to lunch, the backlog of transactions waiting on transactions waiting on transaction would surely cause the entire system to grind to a halt, if not crash. The best that could happen is that the transaction would timeout and be rolled back.

Suggestions and Improvements to Date

Through the years, many developers, authors, and vendors have come up with a set of suggestions and metrics to try and guide us in the direction of performance and scalability. We will now take a look through a few of the more common suggestions.

Stored Procedures are good

To keep in with interface programming (the current recommended way to go), as well as for performance reasons, all data access should be made via stored procedures. SQL Server pre-compiles the stored procedures and creates an execution plan the first time they are run. This execution plan is then used on subsequent requests.

Also, if you know that all of your database access is done through the stored procedures, then that is the only place you need to update if you change your table structure. If you make sure that the stored procedure names, the parameter names and order, and the result set's column names (aliases) and order are maintained, you should not break any existing code. Once you have created a stored procedure, test it and try to optimize it for peak performance. Also look at it in conjunction with ALL the other stored procedures that you have created. Is the order of resource acquisition the same as any other references to these resources?

COM+ Declarative Transactions are slow

For single database systems, do not use COM+ declarative transactions. Either write the transactional support into your stored procedures, or, if that will become too complex, use some kind of code framework to start, commit, or rollback the transactions (this is what we are going to be doing!). This should increase your performance and help scalability. It is interesting to note how this has gone full circle in recent years.

Make chunky not chatty calls and minimize round trips

If you don't require any return values, group database calls together and submit them as one batch. Remember that databases are designed, and hence optimized, for sets and set operations, so design your tables and stored procedures with that in mind. Making one call to the database to get 100 rows is exponentially better than making 100 calls to each get one row, or even to return output parameters.

Pass Data by Value

In the data server, you basically have to get the data into and out of a buffer, so having further marshalling involved to get each piece of data is wasteful. Pass the data into the data tier by value, and then into the buffer by value. Optimally you would want to do this in one step.

What I prefer to do is to create a struct that mimics the return value, the input and output parameters, and the output columns, and pass that in and out of my methods on the data tier. That way, if someone else wants to add another field to that struct (and we are using the same stuct for business reasons) it will not affect my code. It also has the added bonus that if I were to want to add a similar field, I would go to the definition of the struct and see that it has already been added. This would lead me to look at code using the new field to see if I could use the existing code, rather than duplicate it somewhere else.

Also, if your COM+ data tier classes can be loaded in the same context as your business tier classes, then they should be. This can be done in COM+ by marking your data tier assembly as a Library Application instead of a Server Application. This will eliminate unnecessary marshalling. If a parameter only returns data, mark it as an out parameter in C#, as this will eliminate that parameter having to be marshaled into the method.

Pool Connections or, if possible, objects

Object pooling has been demonstrated by Microsoft to be twice as fast as connection pooling. Unfortunately, this functionality deals with distributed transactions, where multiple connections participate in the same transaction. As we will not be using distributed transactions, we will be implementing a different (and new) approach.

Optimize your Indexes

One of the major factors in database performance is the index. An index can mean the difference between 0.001 seconds and 100 seconds. Index design is a complex topic, and some people would rate it as more of an art than a science, but it is essential for your application to perform well.

Indexes in SQL Server are divided into clustered and non-clustered indexes. Clustered indexes actually store the physical data in the table in that order, hence there can only be one clustered index per table. A non-clustered index is simply another internal table holding the index (in order) with an internal row id of the table.

If you will be retrieving single rows by their primary key, then it is best not to make that key a clustered index. A clustered index serves its purpose well if you will be searching on a range of values or will be returning a set of values ordered in the same way as the clustered index.

Acquire resources in order

If you try to follow the pattern of acquiring resources in the same order in different transactions, you will minimize the probability of getting deadlocks.

Isolation is a matter of taste

Choose the isolation level that is correct for the transaction it will be applied to. Keep in mind that reading often requires a higher degree of isolation.

Retrieve Data in the correct way

Microsoft recommends using a data reader over a dataset if you do not need the data lying around (MSDN Article - Performance Tips and Tricks in .NET Applications) and using SqlClient over ODBC. The following performance metrics (higher numbers are better) show this:

 | ADO (ODBC) | SQL
DataSet | 801 | 2507
DataReader | 1083 | 4585

Microsoft also recommends (only if you are using some form of connection pooling - which we are) using output parameters over returning result sets (MSDN Article - .NET Data Access Architecture Guide), as the former can outperform the latter by up to 30% in high-stress conditions (200+ simultaneous connections).

The Proposed Solution

What is propose is a set of .NET classes that will interact with each other to provide a data tier that will adhere to all the guidelines suggested in this case study. It must be structured enough to allow maximum code re-use, but not at the cost of performance nor scalability.

The Solution

Our enterprise application consists of a set of employees, each of whom can have one or more addresses. We will need to capture new employees details, as well as update or delete employees. We will also input some test data for a demonstration of our 'framework'.

Database Design and Structure

The Database

We will need to refer to this database later in the code, so we will name it now.

CREATE DATABASE data_tier
GO
USE data_tier
GO

The Tables

Let us start with the address table. An address will have, in addition to the logical description at the beginning of this case study, a unique internal id to distinguish between addresses.

CREATE TABLE [dbo].[address] (
[id] [int] IDENTITY (1, 1) NOT NULL,
[name] [nvarchar] (50) NOT NULL,
[flat_number] [nvarchar] (10) NULL,
[flat_house_name] [nvarchar] (50) NULL,
[street_number] [nvarchar] (10) NULL,
[street_name] [nvarchar] (50) NULL,
[locality] [nvarchar] (50) NULL,
[city] [nvarchar] (50) NULL,
[county] [nvarchar] (50) NULL,
[country] [nvarchar] (50) NOT NULL,
[post_code] [nvarchar] (10) NOT NULL
) ON [PRIMARY]
GO
CREATE CLUSTERED INDEX [IX_address] ON
[dbo].[address]([post_code]) ON [PRIMARY]
GO
ALTER TABLE [dbo].[address] WITH NOCHECK ADD
CONSTRAINT [PK_address] PRIMARY KEY NONCLUSTERED
([id]) ON [PRIMARY]
GO

A clustered index has been placed on the post_code column. This forces the data in the table to be stored in this order, making searches on partial post codes extremely fast. The primary key has been made a non-clustered index on the id column, as generally only single rows will be retrieved using this primary key.

Next we have the employee table. This will record the fist name, last name, and the date of birth of the employee, as well as an internal id to distinguish between employees.

CREATE TABLE [dbo].[employee] (
[id] [int] IDENTITY (1, 1) NOT NULL,
[first_name] [nvarchar] (50) NOT NULL,
[last_name] [nvarchar] (50) NOT NULL,
[date_of_birth] [datetime] NOT NULL
) ON [PRIMARY]
GO

CREATE CLUSTERED INDEX [IX_employee] ON
[dbo].[customer]([last_name]) ON [PRIMARY]
GO

ALTER TABLE [dbo].[employee] WITH NOCHECK ADD
CONSTRAINT [PK_employee] PRIMARY KEY NONCLUSTERED
([id]) ON [PRIMARY]
GO

A clustered index has been placed on the last_name column. This will sort the data by that column and be fast for employee searches. The primary key has been made a non-clustered index on the id column, as generally only single rows will be retrieved using this primary key.

Finally, we need a table to record which addresses belong to which employee.

CREATE TABLE [dbo].[employee_address] (
[employee_id] [int] NOT NULL,
[address_id] [int] NOT NULL
) ON [PRIMARY]
GO

ALTER TABLE [dbo].[employee_address] WITH NOCHECK ADD
CONSTRAINT [PK_employee_address] PRIMARY KEY CLUSTERED
([employee_id],[address_id]) ON [PRIMARY]
GO

A clustered index (which is also the primary key) has simply been placed on both columns. This has been done with the employee_id first to physically sort the data by employee_id and then address_id. The reason for placing both columns into the index is that once you have found the value in the index, you already have all the information that you need, and do not have to make an extra traversal over the table to retrieve the address_id.

The Stored Procedures

To access these tables, we will define a set of stored procedures. We will need to be able to create a new employee, update an existing one, and delete an existing one. We also want to provide the functionality to create, update or delete an employee's address. Finally, we will provide a search mechanism.

CREATE PROCEDURE usp_get_employee
(@id int,
@first_name nvarchar(50) OUT,
@last_name nvarchar(50) OUT,
@date_of_birth datetime OUT)
AS
BEGIN
SELECT id as id,
name as name,
flat_number as flat_number,
flat_house_name as flat_house_name,
street_number as street_number,
street_name as street_name,
locality as locality,
city as city,
county as county,
country as country,
post_code as post_code
FROM address a
JOIN employee_address ea ON a.id = ea.address_id
WHERE employee_id = @id

SELECT @first_name = first_name,
@last_name = last_name,
@date_of_birth = date_of_birth
FROM employee
WHERE id = @id
END
GO

This stored procedure assumes that the addresses are always required when retrieving an employee. Although this may not always be the case, it has been included here to demonstrate several points.

Firstly, it has eliminated round trips to the database. Some OO programmers would still prefer to get an array of address ids, pass each one to an address object, and get the address object to populate itself from the database. Secondly, it shows that combining output parameters and result sets should not be shied away from. Thirdly, it shows the strict use of interface programming. Everything coming from the database is named using the 'as' keyword, including the column names. This leave the database developer free to change his internal structure as his sees fit. He could rename the post_code column to zip_code, and all he would need to do is to correct the stored procedure. No data tier code would need to be changed. Finally, it shows the order of selection. We select the addresses first and then the employee, as this is the order in which we would delete them. Doing this minimizes the chances of a deadlock.

The other stored procedures to be used by the application are presented below.

CREATE PROCEDURE usp_create_employee
(@first_name nvarchar(50),
@last_name nvarchar(50),
@date_of_birth datetime,
@id int OUT)
AS
BEGIN
INSERT INTO employee (first_name, last_name, date_of_birth)
VALUES(@first_name, @last_name, @date_of_birth)
SET @id = @@identity
END
GO

CREATE PROCEDURE usp_update_employee
(@id int,
@first_name nvarchar(50),
@last_name nvarchar(50),
@date_of_birth datetime)
AS
BEGIN
UPDATE employee
SET first_name = @first_name,
last_name = @last_name,
date_of_birth = @date_of_birth
WHERE id = @id
END
GO

CREATE PROCEDURE usp_delete_employee
(@id int)
AS
BEGIN
-- 1. Delete all the employees adresses
DELETE address FROM address a
JOIN employee_address ea ON a.id = ea.address_id
WHERE ea.employee_id = @id
-- 2. Delete the Mapping
DELETE employee_address
WHERE employee_id = @id
-- 3. Delete the Employee
DELETE employee
WHERE id = @id
END
GO

An employee can be created, updated, or deleted. Creating an employee returns an id back to the caller.

CREATE PROCEDURE usp_create_employee_address
(@employee_id int,
@address_name nvarchar(50),
@flat_number nvarchar(10),
@flat_house_name nvarchar(50),
@street_number nvarchar(10),
@street_name nvarchar(50),
@locality nvarchar(50),
@city nvarchar(50),
@county nvarchar(50),
@country nvarchar(50),
@post_code nvarchar(10),
@id int OUT)
AS
BEGIN
INSERT INTO address (name, flat_number, flat_house_name,
street_number, street_name, locality,
city, county, country, post_code)
VALUES (@address_name, @flat_number, @flat_house_name,
@street_number, @street_name, @locality,
@city, @county, @country, @post_code)
SET @id = @@Identity
INSERT INTO employee_address (employee_id, address_id)
VALUES (@employee_id, @id)
END
GO

CREATE PROCEDURE usp_update_employee_address
(@id int,
@address_name nvarchar(50),
@flat_number nvarchar(10),
@flat_house_name nvarchar(50),
@street_number nvarchar(10),
@street_name nvarchar(50),
@locality nvarchar(50),
@city nvarchar(50),
@county nvarchar(50),
@country nvarchar(50),
@post_code nvarchar(10))
AS
BEGIN
UPDATE address
SET name = @address_name,
flat_number = @flat_number,
flat_house_name = @flat_house_name,
street_number = @street_number,
street_name = @street_name,
locality = @locality,
city = @city,
county = @county,
country = @country,
post_code = @post_code
WHERE id = @id
END
GO

CREATE PROCEDURE usp_delete_employee_addresses
(@id int)
AS
BEGIN
DELETE address FROM address a
JOIN employee_address ea ON a.id = ea.address_id
WHERE ea.employee_id = @id
DELETE employee_address
WHERE employee_id = @id
END
GO

One interesting thing to note is that the addresses of the employee are deleted in the stored procedure that deletes the employee. Some people may disagree with this approach, claiming that these are business rules. I would argue that these are data-centric rules, as an employee address record cannot exist in the database without an employee, making this a table relationship rule.

The Test Data

DECLARE @employee_id int
DECLARE @address_id int
EXEC usp_create_employee 'Fred','Smith','1972-12-05 12:12:12.007',
@employee_id OUT
EXEC usp_create_employee_address @employee_id, 'Personal1', '1',
'Tennis Court', '1', 'Wimbledon Ave', 'Wimbledon', 'London', 'London',
'UK', 'SW19', @address_id OUT
EXEC usp_create_employee_address @employee_id, 'Personal2', '1',
'Tennis Court', '1', 'Wimbledon Ave', 'Wimbledon', 'London', 'London',
'UK', 'SW19', @address_id OUT
EXEC usp_create_employee_address @employee_id, 'Personal3', '1',
'Tennis Court', '1', 'Wimbledon Ave', 'Wimbledon', 'London', 'London',
'ZZ', 'SW19', @address_id OUT
EXEC usp_create_employee_address @employee_id, 'Personal4', '1',
'Tennis Court', '1', 'Wimbledon Ave', 'Wimbledon', 'London', 'London',
'ZZ', 'SW19', @address_id OUT

The test data is used in conjunction with the business tier to show the benefits of using this methodology.

Data Tier Design and Coding

For this demonstration, we will be creating a C# class library (dll file). We shall call the class library (and its namespace) DataTier.

Of the COM+ Component Services, we will be using a connection strings and object pooling. Using COM+ Component Services will require us to reference the System.EnterpriseServices library, and will require our assembly to be Strong Named. Therefore we will require a Strong Name Key (.snk) file.

You can either use the .snk file that is supplied, or you can create your own with Microsoft's sn.exe utility, which comes with the Framework SDK (found in the bin folder). From the command prompt, type: sn -k <file name> where file name is the name you wish to give to your strong name key file.

The Assembly Info

Once you have a .snk file, you need to reference it. In the AsseblyInfo.cs file, you will need to insert the following:

[assembly: AssemblyKeyFileAttribute(@"yourDirectory\youFile.snk")]

where yourDirectory is the directory the .snk file is in, and yourFile is the name of it.

Since we require the data tier to be in the same context as the business tier, we must mark it as being a Library Application. This is done by placing the following line in the AssemblyInfo.cs file:

[assembly: ApplicationActivation(ActivationOption.Library)]

We will also need to provide the COM+ Application name and ID that we wish to belong to, and hence be registered for. This is done by inserting the following in the AssemblyInfo.cs file:

[assembly: AssemblyDelaySign(false)]
[assembly: ApplicationName("DataTier")]
[assembly: ApplicationID("78BF86F2-E008-44c7-B37F-762FC1A27909")]

For this to compile, you will need to place the following at the top of the AssemblyInfo.cs file with the other using directives.

using System.EnterpriseServices;

The Data Types

We start off by defining the internal structures that we will be using. They will entirely consist of primitive types in order to act as a bridge between the business tier and the database.

using System;

namespace
            DataTier
{
public struct InternalEmployee
{
public Int32
            ID;
public string FirstName;
public string LastName;
public
            DateTime DateOfBith;
}
public struct
            InternalAddress
{
public Int32 ID;
public string
            Name;
public string FlatNumber;
public string
            FlatHouseName;
public string StreetNumber;
public string
            StreetName;
public string Locality;
public string
            City;
public string County;
public string Country;
public
            string PostCode;
}
}

The DataServer Class

We start the file off with the using directives:

using System;
using System.Data;
using System.Data.SqlClient;
using System.EnterpriseServices; // ServicedComponent and Attributes
using System.Text; // StringBuilder

namespace DataTier
{
[ObjectPooling(true)]
[MustRunInClientContext(true)]
[ConstructionEnabled(Default="initial catalog=data_tier;data
source=localhost;User Id=sa")]
public class DataServer : ServicedComponent
{
public DataServer(){}

Note that we are referencing the System.EnterpriseServices library as we are deriving our DataServer class from ServicedComponent and using various attributes from that namespace. We will be using the StringBuilder class and hence need to reference the System.Text namespace.

The ObjectPooling attribute declares our intention to use object pooling, as we have supplied it with a value of true. The minimum and maximum pool sizes could also be declared here for fine grain control. The minimum pool size is useful if you require COM+ to pre-populate a number of objects on startup.

The MustRunInClientContext attribute is used in conjunction with the attribute we declare in the AssemblyInfo.cs file to load the assembly as a Library Application. The ConstructionEnabled attribute is used to declare our intention to use construction strings and to provide a default construction string in case one is not set in the CSE. I have used the default 'sa' database user for our application. For a commercial version, it is much more secure to remove all security from the connection string and use the CSE to let the component run under a particular Windows User. SQL Server can then be set up to accept this user and assign privileges.

Our class finally derives from Serviced Component to ensure that all the necessary COM+ plumbing will be wired up for us.

Next we declare our private member variables:

private Select m_rSelect;
private Update m_rUpdate;
private Processor m_rProcessor;
private string m_sCnn;
private SqlConnection m_rCnn;
private SqlCommand m_rCmd;
private SqlCommand m_rInternalCommand;
private object m_rRoot;
private TransactionStatus m_eStatus;
private StringBuilder m_sbParsedCommands;
private IsolationLevel m_eIsolationLevel;

We will address the purpose of each of these member variables when they become relevant. For now we will press on.

The Static and Constant Helper Functions

We will now define several 'Helper' functions and constants for use with the Data Tier:

public const Int32 NameTextSize = 50;
public const Int32 NumberTextSize = 10;
public static string GetNVarChar(string Input)
{
if(Input==null)
return "NULL";
Input = Input.Replace("'","''");
return "'" + Input + "'";
}
public static string GetDateTime(DateTime Input)
{
if(Input==DateTime.MinValue)
return "NULL";
// More accurate than SqlClient
return "'" + Input.ToString("yyyy-MM-dd hh:mm:ss.fff") + "'";
}
public enum TransactionStatus
{
Nothing = 0,
BeginTransaction = 1,
InTransaction = 2
}

The NameTextSize and NumberTextSize help us store, in one place, the lengths of our database strings. As we will be referencing them many times in our Data Tier code, if we need to change any of them, only one change will be needed.

Part of our design is the 'flattening out' of our commands, i.e. creating a stream of commands in a string. We will need a function to make sure that text fields are returned in the proper format. We will also need to convert DateTime variables into strings. Currently SqlClient chops off the seconds and fractions of seconds when doing this. I prefer to leave them in. Our GetNVarChar and GetDateTime methods fulfill these requirements.

We will also need to record the intention of using a transaction so I have created an enumeration to store the 'logical' status of the transaction. This is crucial in order to start a transaction as late as possible to minimize the time that the transaction is actually running.

COM+ Services

The first COM+ Service that we have declared our support of is the construction string. We therefore need to override the Construct method, which COM+ guarantees will be called immediately after the constructor and will be passed in the construction string in the CSE, or the default declared in the attribute (ConstructionEnabled) if none exists.

protected override void Construct(string
            constructString)
{
m_sCnn = constructString;
m_rCnn = new
            SqlConnection(m_sCnn);
m_rCnn.Open();
m_rCmd = new
            SqlCommand();
m_rCmd.CommandType =
            CommandType.StoredProcedure;
m_rCmd.Connection =
            m_rCnn;
m_rInternalCommand = new
            SqlCommand();
m_rInternalCommand.CommandType =
            CommandType.Text;
m_rInternalCommand.Connection =
            m_rCnn;
m_rSelect= new Select(this, m_rCmd);
m_rUpdate = new
            Update(this, m_rCmd);
m_rProcessor = new Processor(this,
            m_rCmd);
}

Here we stored the construction string (you will see why later) and a connection that we open. We then create two command objects pointing to the same connection. One is a public command used for commands that return data to the client by means of a stored procedure, and one is an internal command used to execute buffered commands that do not return data. Although the documentation in MSDN states that this is a public method, it is in fact a protected method and will not compile unless set as such.

If an error occurs, it is wise to log it somewhere (as users will not remember technical error information) and to throw the error up to be handled by the business tier. I have removed all error handling that is not part of the logic I wish to demonstrate, for the sake of brevity.

The next COM+ Service that we have declared our support for is object pooling. When an instance of the DataServer class is requested, COM+ will first look in the object pool to see if one already exists that is not in use. If there is an instance of DataServer in the pool, COM+ removes it from the pool, calls Activate on it, and releases it to the client. If the pool is empty, COM+ will create a new DataServer object, and will still call the activate method before releasing it to the client.

protected override void
            Activate()
{
m_eStatus =
            TransactionStatus.Nothing;
m_sbParsedCommands = new
            StringBuilder();
if(m_rCnn.State!=ConnectionState.Open)
{
m_rCnn.Close();
this.Construct(m_sCnn);
}
else
            if(m_rCmd.Transaction!=null)
{
this.Rollback();
}
}

When a DataServer object is activated, it will reset its internal transaction status to nothing and create a new stringbuilder object to store the buffered commands. It will also make sure that the connection has not become corrupt and that no lingering transaction exists.

When an object that implements IObjectControl (i.e. when it declares its support for object pooling with the ObjectPooling attribute) is about to be disposed, COM+ guarantees to call the Deactivate method on it.

protected override void
            Deactivate()
{
this.Rollback();
}

Here we simply call our RollBack method, which, as we will learn later, will Rollback any lingering transactions, if they exist.

After COM+ has called the Deactivate method it calls the CanBePooled method. If this returns true, then COM+ will place this object on the pool, otherwise it will let the object die gracefully.

Internal Helper Methods

The DataServer class has several internal helper methods to get the infrastructure to work smoothly. I will discuss each in turn.

internal bool InTransaction()
{
return
            m_rCmd!=null &&
m_rCmd.Transaction!=null
            &&
m_rCmd.Transaction.Connection!=null;
}

The InTransaction method checks to see whether the DataServer is currently inside an actual transaction. This is done in SqlClient by checking the Connection property of the Transaction object.

internal void BeginTransaction(object
            Caller, IsolationLevel Isolation)
{
if(m_eStatus ==
            TransactionStatus.Nothing)
{
m_rRoot=Caller;
m_eStatus =
            TransactionStatus.BeginTransaction;
}
if(m_eIsolationLevel<Isolation)
m_eIsolationLevel
            = Isolation;
}

The BeginTransaction is used to record the intention of beginning a transaction, without beginning one. This is done to start the transaction as late as possible, to minimize its lifetime. It also records the root caller to know when to commit, as well as the intended isolation level, which it will check against previous requests, and select the highest.

internal void AppendCommand(object Caller, string SQL,
IsolationLevel Isolation)
{
m_sbParsedCommands.Append(" " + SQL + " \n");
this.BeginTransaction(Caller, Isolation);
}

The AppendCommand method is used for commands that do not return data. These commands can be stored (in a stream or string) and sent to the database as one batch, thus minimizing round-trips. It is assumed that any operation not returning data must be doing something to the database and hence will require a transaction. The isolation level can be specified.

internal void
            FlushBuffer()
{
if(m_sbParsedCommands.Length>0)
{
if(m_eStatus!=TransactionStatus.InTransaction)
{
m_rCmd.Transaction
            =
m_rCnn.BeginTransaction(m_eIsolationLevel);
m_eStatus=TransactionStatus.InTransaction;
}
m_rInternalCommand.CommandText
            = m_sbParsedCommands.ToString();
m_rInternalCommand.Transaction =
            m_rCmd.Transaction;
m_sbParsedCommands.Remove(0,
            m_sbParsedCommands.Length-1);
m_rInternalCommand.ExecuteNonQuery();
}
}

The FlushBuffer method is used for sending the buffered commands, if any, to the database. This will be done either when the root object is about to commit, or when a command that returns data needs to be executed. This method creates a physical transaction at the appropriate isolation level, marks the DataServer object as being in a transaction, executes the buffered commands on the internal command object, and clears the buffer.

internal void
            Rollback()
{
try
{
if(InTransaction())
m_rCmd.Transaction.Rollback();
}
catch
{
/*
            Microsoft could have already rolled back */
/* Could have done a
            IF @@TRANCOUNT > 0 ROLLBACK TRANS*/
}
finally{m_eStatus =
            TransactionStatus.Nothing;}
}

The Rollback method is used, surprisingly, for rolling back transactions. It will first check to see if the DataServer object is in a transaction, and if so, it will roll it back. It is interesting to note that I have placed this logic inside a try block. This is done because several of the SqlClient classes will actually rollback the transaction for you. Thus calling Rollback will result in a SQL error as there is no transaction to rollback. One suggestion that I have seen is to send an "IF @@TRANCOUNT > 0 ROLLBACK TRANS " command down the wire. While this will prevent an error, it will not do all the things that SqlConnection.Rollback does behind the scenes for you. Microsoft suggest that you merely catch any error and ignore it. The finally block makes sure that the status is reset.

internal void
            Commit()
{
try
{
this.FlushBuffer();
if(InTransaction())
m_rCmd.Transaction.Commit();
}
catch
{
Rollback();
throw
            new Exception("Transaction Aborted");
}
finally{m_eStatus =
            TransactionStatus.Nothing;}
}

The Commit method is used, surprisingly, for committing our transaction. It will first flush the buffer of any pending commands. If an error occurs, it will attempt to roll the transaction back, and then raise an exception to the client. The finally block makes sure that the status is reset.

internal void ExecuteNonQuery(object Caller,
            bool WithTransaction,IsolationLevel
            Isolation)
{
if(WithTransaction)
this.BeginTransaction(Caller,
            Isolation);
FlushBuffer();
if(WithTransaction &&
            m_eStatus!=TransactionStatus.InTransaction)
{
m_rCmd.Transaction
            =
m_rCnn.BeginTransaction(m_eIsolationLevel);
m_eStatus=TransactionStatus.InTransaction;
}
m_rCmd.ExecuteNonQuery();
}

The ExecuteNonQuery method is used to execute a command that does not return a dataset, but cannot be placed in the buffer as it returns either output parameters or a return value. A transaction is optionally started (if one does not exist), the buffer is flushed, and the command is executed.

internal SqlDataReader ExecuteReader(object Caller, bool WithTransaction, IsolationLevel Isolation)
{
if(WithTransaction)
this.BeginTransaction(Caller, Isolation);
FlushBuffer();
if(WithTransaction && m_eStatus!=TransactionStatus.InTransaction)
{
m_rCmd.Transaction =
m_rCnn.BeginTransaction(m_eIsolationLevel);
m_eStatus=TransactionStatus.InTransaction;
}
return m_rCmd.ExecuteReader();
}

The ExecuteReader method is used to execute a command that returns a DataReader. A transaction is started if necessary, the buffer is flushed, and the command is executed.

The Public Methods and Properties

public void
            SetAbort()
{
this.Rollback();
}
public void
            SetComplete(object Caller)
{
if(object.ReferenceEquals(Caller,
            m_rRoot))
this.Commit();
}
public Select Select{get{return
            m_rSelect;}}
public Update Update{get{return
            m_rUpdate;}}
public Processor Processor{get{return
            m_rProcessor;}}

The SetAbort method is provided for the client to abort the transaction. A database transaction may not even have started physically, but this method guarantees to rollback any database activity. No exception will be thrown by SetAbort (this is guaranteed), and the code calling SetAbort would already know that an error has occurred, so would not want to handle an exception anyway.

The SetComplete method will commit the transaction if the caller is the root caller.

The Select, Update and Processor properties simply provide access to our objects containing the data access code.

Well, that is it for our DataServer class. We will now see how our data accessing classes will interact with this class.

The Processor Class

The Processor class is the class in which we will store all data access code that will not be returning any values.

using System;
using System.Data;
using
            System.Data.SqlClient;
using System.Text;

namespace
            DataTier
{
public class Processor
{
private SqlCommand
            m_rCmd;
private DataServer m_rParent;
internal
            Processor(DataServer Parent, SqlCommand SqlCommand)
{
m_rCmd =
            SqlCommand;
m_rParent = Parent;
}

We will be making use of the StringBuilder class so we have referenced the System.Text namespace. We hold a reference to the DataServer class (our parent) and mark the constructor as internal (similar to VB's PublicNotCreateable class attribute) to make sure that only this assembly can create instances of this class. The SqlCommand object that we are holding a reference to is the public SqlCommand object of the DataServer class.

public void UpdateEmployee(object Caller, InternalEmployee Employee)
{
StringBuilder sb = new StringBuilder("EXEC usp_update_employee ");
sb.Append(Employee.ID);
sb.Append("," + DataServer.GetNVarChar(Employee.FirstName));
sb.Append("," + DataServer.GetNVarChar(Employee.LastName));
sb.Append("," + DataServer.GetDateTime(Employee.DateOfBith));
m_rParent.AppendCommand(Caller, sb.ToString(),
IsolationLevel.ReadCommitted);
}

The UpdateEmployee method takes a vtEmployee structure and the caller and creates an SQL command that it sends to the DataServer 's buffer. It uses the static GetNVarChar method to encode the text and the static GetDateTime method to encode the date. This is where the isolation level decision is made.

public void DeleteEmployee(object Caller, Int32 EmployeeID)
{
StringBuilder sb = new StringBuilder("EXEC usp_delete_employee ");
sb.Append(EmployeeID);
m_rParent.AppendCommand(Caller, sb.ToString(),
IsolationLevel.ReadCommitted);
}
public void UpdateAddress(object Caller, InternalAddress Address)
{
StringBuilder sb =
new StringBuilder("EXEC usp_update_employee_address ");
sb.Append(Address.ID);
sb.Append("," + DataServer.GetNVarChar(Address.Name));
sb.Append("," + DataServer.GetNVarChar(Address.FlatNumber));
sb.Append("," + DataServer.GetNVarChar(Address.FlatHouseName));
sb.Append("," + DataServer.GetNVarChar(Address.StreetNumber));
sb.Append("," + DataServer.GetNVarChar(Address.StreetName));
sb.Append("," + DataServer.GetNVarChar(Address.Locality));
sb.Append("," + DataServer.GetNVarChar(Address.City));
sb.Append("," + DataServer.GetNVarChar(Address.County));
sb.Append("," + DataServer.GetNVarChar(Address.Country));
sb.Append("," + DataServer.GetNVarChar(Address.PostCode));
m_rParent.AppendCommand(Caller, sb.ToString(),
IsolationLevel.ReadCommitted);
}
public void DeleteAddress(object Caller, Int32 AddressID)
{
StringBuilder sb =
new StringBuilder("EXEC usp_delete_employee_addresses ");
sb.Append(AddressID);
m_rParent.AppendCommand(Caller, sb.ToString(),
IsolationLevel.ReadCommitted);
}

The DeleteEmployee, UpdateAddress and DeleteAddress methods use the same methodology to complete their tasks.

The Update Class

The Update class is the class in which we will store all data access code that will modify the database and return data as well.

using System;
using System.Data;
using System.Data.SqlClient;
using System.Data.SqlTypes; // SqlDateTime.Null

namespace DataTier
{
public class Update
{
private SqlCommand m_rCmd;
private DataServer m_rParent;
internal Update(DataServer Parent, SqlCommand SqlCommand)
{
m_rCmd = SqlCommand;
m_rParent = Parent;
}

We will be making use of the SqlDateTime.Null value for DateTime fields so we have referenced the System.Data.SqlTypes namespace. We hold a reference to the DataServer class (our parent) and mark the constructor as internal. The SqlCommand object that we are holding a reference to is the public SqlCommand object of the DataServer class.

public Int32 CreateEmployee(object Caller, InternalEmployee Employee)
{
m_rCmd.Parameters.Clear();
m_rCmd.CommandText = "usp_create_employee";
m_rCmd.Parameters.Add("@first_name", SqlDbType.NVarChar,
DataServer.NameTextSize).Value =
Employee.FirstName;
m_rCmd.Parameters.Add("@last_name", SqlDbType.NVarChar,
DataServer.NameTextSize).Value =
Employee.LastName;
m_rCmd.Parameters.Add("@date_of_birth", SqlDbType.DateTime).Value =
Employee.DateOfBith==DateTime.MinValue?
SqlDateTime.Null:Employee.DateOfBith;
m_rCmd.Parameters.Add("@id", SqlDbType.Int).Direction =
ParameterDirection.Output;
m_rParent.ExecuteNonQuery(Caller, true, IsolationLevel.ReadCommitted);
return m_rCmd.Parameters["@id"].Value==DBNull.Value?
0:(Int32)m_rCmd.Parameters["@id"].Value;
}

The CreateEmployee method takes a vtEmployee structure and the caller and builds the command on the DataServer 's public command object. It then calls ExecuteNonQuery on the DataServer, passing in the caller, the desire for a transaction, and the isolation level. Finally it extracts the output parameter and returns it.

public Int32 CreateAddress(object Caller, Int32 EmployeeID, InternalAddress Address)
{
m_rCmd.Parameters.Clear();
m_rCmd.CommandText = "usp_create_employee_address";
m_rCmd.Parameters.Add("@employee_id",
SqlDbType.Int).Value = EmployeeID;
m_rCmd.Parameters.Add("@address_name",
SqlDbType.NVarChar,
DataServer.NameTextSize).Value = Address.Name;
m_rCmd.Parameters.Add("@flat_number",
SqlDbType.NVarChar,
DataServer.NumberTextSize).Value =
Address.FlatNumber;
m_rCmd.Parameters.Add("@flat_house_name",
SqlDbType.NVarChar,
DataServer.NameTextSize).Value =
Address.FlatHouseName;
m_rCmd.Parameters.Add("@street_number",
SqlDbType.NVarChar,
DataServer.NumberTextSize).Value =
Address.StreetNumber;
m_rCmd.Parameters.Add("@street_name",
SqlDbType.NVarChar,
DataServer.NameTextSize).Value =
Address.StreetName;
m_rCmd.Parameters.Add("@locality",
SqlDbType.NVarChar,
DataServer.NameTextSize).Value =
Address.Locality;
m_rCmd.Parameters.Add("@city",
SqlDbType.NVarChar,
DataServer.NameTextSize).Value =
Address.sCity;
m_rCmd.Parameters.Add("@county",
SqlDbType.NVarChar,
DataServer.NameTextSize).Value =
Address.County;
m_rCmd.Parameters.Add("@country",
SqlDbType.NVarChar,
DataServer.NameTextSize).Value =
Address.Country;
m_rCmd.Parameters.Add("@post_code",
SqlDbType.NVarChar,
DataServer.NumberTextSize).Value =
Address.PostCode;
m_rCmd.Parameters.Add("@id", SqlDbType.Int).Direction =
ParameterDirection.Output;
m_rParent.ExecuteNonQuery(Caller, true, IsolationLevel.ReadCommitted);
return m_rCmd.Parameters["@id"].Value==DBNull.Value?
0:(Int32)m_rCmd.Parameters["@id"].Value;
}

The CreateAddress method works in exactly the same manner.

The Select Class

The Select class is the class in which we will store all data access code that will return data.

using System;
using System.Data;
using
            System.Data.SqlClient;
using System.Collections;

namespace
            DataTier
{
public class Select
{
private SqlCommand
            m_rCmd;
private DataServer m_rParent;
internal
            Select(DataServer Parent, SqlCommand SqlCommand)
{
m_rCmd =
            SqlCommand;
m_rParent = Parent;
}

We will be making use of the ArrayList class for our collections so we have referenced the System.Collections namespace. We hold a reference to the DataServer class (our parent) and mark the constructor as internal. The SqlCommand object that we are holding a reference to is the public SqlCommand object of the DataServer class.

public vtEmployee GetEmployee(object Caller, Int32 EmployeeID, out ArrayList Addresses)
{
InternalEmployee vEmployee = new InternalEmployee();
vEmployee.ID = EmployeeID;
m_rCmd.Parameters.Clear();
m_rCmd.CommandText = "usp_get_employee";
m_rCmd.Parameters.Add("@id", SqlDbType.Int).Value = vEmployee.ID;
m_rCmd.Parameters.Add("@first_name",
SqlDbType.NVarChar,
DataServer.NameTextSize).Direction =
ParameterDirection.Output;
m_rCmd.Parameters.Add("@last_name",
SqlDbType.NVarChar,
DataServer.NameTextSize).Direction =
ParameterDirection.Output;
m_rCmd.Parameters.Add("@date_of_birth",
SqlDbType.DateTime).Direction =
ParameterDirection.Output;
// We must read the data BEFORE we can get the Output Parameters
System.Data.SqlClient.SqlDataReader rReader =
m_rParent.ExecuteReader(Caller, false,
IsolationLevel.RepeatableRead);
Addresses = new ArrayList();
while(rReader.Read())
{
InternalAddress vAddress = new InternalAddress();
vAddress.ID = rReader.GetInt32(0);
vAddress.Name = rReader.GetString(1);
if(!rReader.IsDBNull(2))
vAddress.FlatNumber = rReader.GetString(2);
if(!rReader.IsDBNull(3))
vAddress.FlatHouseName = rReader.GetString(3);
if(!rReader.IsDBNull(4))
vAddress.StreetNumber = rReader.GetString(4);
if(!rReader.IsDBNull(5))
vAddress.StreetName = rReader.GetString(5);
if(!rReader.IsDBNull(6))
vAddress.Locality = rReader.GetString(6);
if(!rReader.IsDBNull(7))
vAddress.City = rReader.GetString(7);
if(!rReader.IsDBNull(8))
vAddress.County= rReader.GetString(8);
vAddress.Country = rReader.GetString(9);
vAddress.PostCode = rReader.GetString(10);
Addresses.Add(vAddress);
}
// Close the Reader to free up the Connection
rReader.Close();
// Output Parameters are now available
vEmployee.FirstName =
(m_rCmd.Parameters["@first_name"].Value != DBNull.Value)?
(string)m_rCmd.Parameters["@first_name"].Value : "" ;
vEmployee.LastName =
(m_rCmd.Parameters["@last_name"].Value != DBNull.Value)?
(string)m_rCmd.Parameters["@last_name"].Value : "" ;
vEmployee.DateOfBith =
(m_rCmd.Parameters["@date_of_birth"].Value != DBNull.Value)?
(DateTime)m_rCmd.Parameters["@date_of_birth"].Value :
DateTime.MinValue;
return vEmployee;
}

The GetEmployee method takes the Employee's ID and the caller and builds the command on the DataServer 's public command object. It then calls ExecuteReader on the DataServer, passing in the caller, the desire for a transaction, and the isolation level. It then reads the list of addresses into an ArrayList and then extracts the output parameters. It then closes the reader to free up the connection and returns.

Business Tier Design and Coding

For this demonstration, we will be creating a C# console application (.exe file) to mimic the business tier. We shall call the console application (and its namespace) DataTierClient. We will create a class called Utility to mimic a typical business tier class and let the main procedure mimic a client.

using System;
using System.Collections;
using DataTier;

namespace DataTierClient
{
public class Utility
{
public Utility(){}

Nothing new here! We are referencing the DataTier and System.Collections namespaces.

public void CleanAddresses(Int32
            EmployeeID)
{
DataServer Data = new DataServer();
ArrayList
            aAddresses = null;
InternalEmployee vEmployee
            =
Data.Select.GetEmployee(this, EmployeeID, out
            aAddresses);
for(int
            i=0;i<aAddresses.Count;++i)
{
InternalAddress vAddress =
            (InternalAddress)aAddresses[i];
if(vAddress.Country.StartsWith("Z"))
Data.Processor.DeleteAddress(this,
            vAddress.ID);
else
            if(vAddress.Country=="UK")
{
vAddress.Country = "United
            Kingdom";
Data.Processor.UpdateAddress(this,
            vAddress);
}
}
}

The CleanAddresses method is used to implement the following business task: given an employee ID, iterate through all of his or her addresses and either delete it if the country begins with a Z or change the country to "United Kingdom" if it currently reads "UK".

using System;

namespace
            DataTierClient
{
class StartClass
{
///
            <summary>
/// The main entry point for the
            application.
/// </summary>
[STAThread]
static void
            Main(string[] args)
{
Utility rUtility = new
            Utility();
rUtility.CleanAddresses(1);
}
}
}

The Main method is used to mimic a client request coming in for our pretend business server.

The Sample Application

There is not much to see visually with this application. It is, after all, a Data Tier application. However, that said, one can gain a great deal out of running a trace against the database using the Profiler tool that ships with SQL server. That should demonstrate what is happening.

Application Setup

After running the SQL Script file to set up the database, simply create two instances of VS.NET and load the DataTierClient and DataTier solutions respectively. Then simply step through the DataTierClient application in debug mode (it is assumed that the database is residing on the same machine as the VS.NET projects and that the 'sa' password is blank. If not, please refer to the ReadMe.txt file).

Case Study Review

This case study has set out to suggest a methodology to use when implementing a single-server data tier. It has investigated past mistakes and technologies and tried to amalgamate the information into a technical framework.

The various methods of retrieving data are discussed. These include using output parameters (including the return value) and result sets, as well as the two methods of reading the result set; namely by using a data reader or populating a data set. Microsoft's metrics are provided to compare these methods.

Next, past mistakes made by developers were investigated and explained, and the most commonly accepted solutions and suggestions followed on from this. Examples included not applying database theory and principles (like using indexes and set operations), letting in network latency (round-tripping) and not understanding locking and blocking principles. A solution was then suggested. This solution incorporated the technologies and avoided the mistakes already discussed.

We then created a database and some tables and stored procedures to demonstrate our solution. Several interesting and important peripheral facts are discussed and demonstrated here with respect to the choice of indexes and the layout of the stored procedures, for example we look at the reason why we have chosen a non-clustered over a clustered index. While not directly related to our framework, they are crucial to a good data tier and are included for completeness.

Of the COM+ Services, we used construction strings, object pooling, and client context loading, as the former is useful for administration and the latter two will increase our performance and scalability.

The DataServer class that we create is nothing by itself, and needs to interact with the Select, Update and Processor classes. We use communication by internal methods to create a flexible framework that will minimize roundtrips to the database, yet control transaction logic.

The business tier can then use this DataServer class whilst being relatively ignorant of the data logic below. This approach, I feel, will give the business tier programmer flexibility whilst not reducing performance nor scalability. It can actually enhance it in some cases.

What the reader (hopefully) is left with is a good methodology and the information of SqlClient to help him or her utilize it efficiently.

Any Limitations or Further Work

There are a few limitations of this approach. One limitation is the fact that the DataServer class needs to be passed around, which is cumbersome and inelegant. A better approach would be to 'hide' it in the context or call sink (that would be another article in itself!).

Another limitation is that the root object cannot call methods on itself, otherwise it would commit transactions prematurely. It would be nice to have the root object be known by the call chain. An observation on this methodology is the fact that once a transaction has begun, the isolation level cannot be changed, but this would be the case anyway if one were to code it by hand.

Further work on this would be to hide the DataServer object and let the other data tier classes use attributes to indicate their transactional requirements (like declarative transactions in COM+ Enterprise Services). Interception would then be used to interact and control the transaction(s).

Another piece of performance-enhancing work would be to somehow map a structure that you wish the output data to reside in to the output buffer in the internal TdsParser object. This would eliminate an extra copy operation for ALL the data returned. This approach would be similar to the OLE DB Consumer templates provided in ATL VC++. This again would probably be an article in itself!

C#Today - ©2002 Wrox Press Limited


Web Site Error Logging and Reporting System

Web Site Error Logging and Reporting System
http://www.csharptoday.com/content.asp?id=1945
Article Rating: 5
Published on October 21, 2002
Abstract
Constantly monitoring a web site can be both a time-consuming and tedious job. Sometimes this task needs to be done manually, and even then it might not be possible to log all the details relating to errors that occur in a web application. In this case study, Vinod Kumar showcases a way to log and report the errors that occur in a web application, taking us through all the stages in the development of an application.
Case Study

Constantly monitoring a web site can be both a time-consuming and tedious job. Sometimes this task needs to be done manually, and even then it might not be possible to log all the details relating to errors that occur in a web application. In this case study, I would like to showcase a way to log and report the errors that occur in a web application.

Introduction

We will be developing an error logging and reporting system which will be used to log all the errors that occur in a web application, including details such as the time that the error occurred, the URL, the server name, the user agent, the user IP, the user host name, the user's authentication, the error message, the source, the target web site, etc. The errors can be logged to:

  • Text File
  • OLEDB Supported database
  • Event Log
  • Sending the mail of the Error Details as and when it's happens.
  • Sending SMS of the Error

The web site can be configured to select any or all of the reporting systems listed above to be used for keeping track of the errors in the system. Before we look any further into this, let's see the system requirements for using this software, and the knowledge we'll be assuming in this case study.

System Requirement

Before continuing, please take note of the following system requirements for the purposes of this discussion, as well as for working with the sample code that accompanies the case study.

The web site error logging and reporting system is designed to log the errors that happen in the ASPX file within a Web Application. The other requirements are:

  • Windows NT, Windows 2000 or XP with IIS
  • .NET Framework Version 1
  • Visual Studio .NET RTM Version 1
  • SQL Server 7.0 or 2000

Even though we'll be using SQLServer 2000, this application has been tested with both Oracle and Access. Any other OleDb-compliant databases can also be used. This case study requires some familiarity with the Microsoft .Net Framework, C#, and XML. Familiarity with Microsoft SQL Server is not necessary, but would be helpful.

Error Logging System Design Structure

Now let's see how the error logging and reporting system works. Generally, web sites cannot realistically be monitored constantly, and even if they could, it's not possible to log all the errors that occur in a web application along with their details. Logging error information is extremely helpful, as being able to analyze the cause of errors is of course the easiest way to identify the fault and come up with a solution. We will design a system that will log both the handled and unhandled exceptions, with a further option to send an e-mail and SMS as and when the errors happen, in order to notify the owner of the system.

The web application error logging and reporting system consists of three modules which can be seen as three VS.NET projects. Let's now briefly look at these three modules:

  • WebAlert
  • WebLog
  • WebException

WebAlert

The WebAlert module has an AlertManager class that uses the System.Web.Mail namespace to send e-mails containing custom error information. This module also provides the necessary functionality to send the error information as an SMS message.

WebLog

In the WebLog module we have a LogManager class that gives us the functionality to store error information in an event log. The data can then be used for future reference and analysis. As well as logging the error information into an event log, we also store it in database. This allows for easier and better analysis of the information. Another main advantage of storing the information in a database is that it means the information can be ported to other systems or databases without too many problems.

WebException

The core of this case study is the WebException module. The ErrorHandler class of this module uses the AlertManager and LogManager classes mentioned in the previous sections to log and report the errors.

Before going any further, let's briefly see how the system uses the above-mentioned class. Whenever an unhandled exception occurs, the Application_onError event gets triggered. This event uses the ErrorHandle class in the WebException module. This ErrorHandle class uses the LogManager and AlertManager classes to log the information and alert the web site administrator by sending an e-mail and SMS message. A similar approach is used for handled exceptions. We use the ErrorHandler class within a try catch block of the handled exception to log the error.

Now let's look at each of these modules in detail. At the end of the case study we will see how to integrate the entire web logging and reporting feature into a web site, and we will see how to use the configuration file to set the logging error information and error alert options.

The screenshot below shows the web application error logging and reporting system's structure:

Component Design - WebAlert

The WebAlert module contains an AlertManager class. This class uses the System.Web.Mail namespace to send the error information. The class is designed in such a way that as well as sending error information as an e-mail, it can also be used for sending mails.

Now let's examine this class in more detail. This class imports the following namespaces: System.Web, System.Web.Mail, System.Collections, System.Xml, System.IO, and System.Text. This class contains a set of private variables that are used to hold mail information. Some of these variables are exposed in the form of properties - this design decision allows us to exercise a greater degree of control over the values that are assigned to the variables. mmMsg is set to the MailMessage class, which is present in the System.Web.Mail namespace.

The diagram below shows all the members in the AlertManager class:

MailMessage has properties and methods for constructing an e-mail message.

using System;
using System.Web.Mail;
using System.Xml;
using System.IO;
using System.Text;
using System.Web;
namespace ErrorLog.WebAlert
{
/// <summary>
/// Class used for sending mails.
/// </summary>
public class AlertManager
{
// Global message object used for sending the mail
MailMessage mmMsg = new MailMessage();
// Private variables for holding the property values.
string strSMTPServer; //Holds SMTPServer Name
string strTo; //Receiver Email ID
string strFrom; // Sender Mail ID
string strSubject; // Mail Subject
string strBody; // Mail Body
string strCC; // CC to Mail
string strBCC; // BCC to Mail

//Property to Set and Get MailFormat
// HTML or Text Format
public MessageFormat Format
{
get{return enuMsgFormat;}
set{enuMsgFormat = value;}
}
//Property to Set and Get SMTPServer
public string SMTPServer
{
get{return strSMTPServer;}
set{strSMTPServer = value;}
}
//Property to Set and Get Senders Email ID
public string From
{
get{return strFrom;}
set{strFrom = value;}
}
//Property to Set and Get Receivers Email ID
public string To
{
get{return strTo;}
set{strTo = value;}
}
//Property to Set and Get Mail Subject
public string Subject
{
get{return strSubject;}
set{strSubject = value;}
}
//Property to Set and Get Mail Body
public string Body
{
get{return strBody;}
set{strBody = value;}
}
//Property to Set and Get Mail Carbon Copy
public string CC
{
get{return strCC;}
set{strCC = value;}
}
//Property to Set and Get Mail BCC
public string BCC
{
get{return strBCC;}
set{strBCC = value;}
}

A private variable called enuMsgFormat is used to store the MailFormat information:

private MessageFormat enuMsgFormat;
/// <summary>
/// Message Format
/// </summary>
public enum MessageFormat
{
Text,
Html
};

The AlertManager class has a default constructor that is used for initializing the class:

      public
            AlertManager() // Default Constructor.
{

}

Now let's create an overloaded version for the constructor of the AlertManager. This overloaded version gets and stores the email message attributes (From, To, Subject, Body, Attachments, etc). We set the SMTP Server property to the local server if it's not specified. The code below is the overload version of the AlertManager constructor:

/// <summary>
/// Gets and Store
            the mail attributes
/// </summary>
/// <param
            name="strFrom">From EMail</param>
/// <param
            name="strTo">To Email</param>
/// <param
            name="strSubject">Mail Subject</param>
/// <param
            name="strBody">Mail Body</param>
/// <param
            name="strCC">CC Email</param>
/// <param
            name="strBCC">BCC EMail</param>
/// <param
            name="strSMTPServer">SMTP Server</param>
/// <param
            name="Attachments">Attachment</param>
public
            AlertManager(string strFrom,string strTo,string strSubject,string
            strBody,
string strCC, string strBCC,string strSMTPServer, string
            Attachments)
// Custom Constructor.
{
if
            (null!=strFrom)
this.From = strFrom;

if
            (null!=strTo)
this.To = strTo;

if
            (null!=strBody)
this.Body = strBody;

if
            (null!=strCC)
this.CC = strCC;

if
            (null!=strBCC)
this.BCC = strBCC;

if
            (strSubject.Equals(null))
this.Subject="None";
else
this.Subject
            = strSubject;

if (null!=strSMTPServer)
this.SMTPServer =
            strSMTPServer;
else

this.SMTPServer =
            "127.0.0.1";

this.Format = (MessageFormat)MailFormat.Html;
this.AddAttachment(Attachments.ToString());
}

In the AlertManager class, the AddAttachment method is used to store the mail attachments. This method in turn uses the MailAttachment class in the System.Web.Mail namespace to send the attachment along with the mail. Before sending the attachment file across the network we need to convert it into byte format, so we set the MailEncoding enumerator of the MailAttachment class (System.Web.Mail Namespace) to Base64 in the MailAttachment method.

/// <summary>
/// ADD the File
            attachments
/// </summary>
/// <param
            name="strFilename">Attachment File Path</param>
public
            void AddAttachment(string strFilename)
{
// Code needed to
            implement ";" for file seperator .
//Setting MailEncoder to
            Base64
mmMsg.Attachments.Add(new
MailAttachment(strFilename,MailEncoding.Base64));
}

The SendMail method is used to send mails after building the mail message by setting the various mail properties. This method uses the SmtpMail class in the System.Web.Mail namespace to send the mail. This SmtpMail class uses the Collaboration Data Objects for Windows 2000 (CDOSYS) message component for sending mails. The mail message is delivered using the SMTP mail service built into Microsoft Windows 2000 or through an external SMTP server.

Important note : If the SmtpServer property is not set, then it uses the SmartHost feature from the service configuration parameters to locate the SMTP Server in the network. If this fails, then the messages will be queued. Also, if the name of the SMTP server is not set, the name of the local SMTP server is used.

The next method is the AlertManager class that is used to send SMS. This method uses the web service provided by kSMSWebService. The kSMS web service is used to send free SMS text messages to mobile phones around the world. Unlike other SMS services, one can send any number of messages using this web service. This service is absolutely free. The WSDL file of this WebService is located at http://www22.brinkster.com/smsonweb/ksms.asmx?WSDL&WROXEMPTOKEN=618658Zezi9SpDFSZgQps5Xv6u. To learn more about using this, please take a look at the articles listed in the links section of this case study.

The SendSMS method takes the SendersName, MobileNumber and Message as arguments. This method calls the web method SendSMS of the KSMS WebService to send the message to the mobile device.

public void SendSMS(string SendersName,string MobileNumber,string Message)
{
WebAlert.com.brinkster.www22.kSMS objSMS = new
WebAlert.com.brinkster.www22.kSMS ();
objSMS.SendSMS(SendersName,MobileNumber,Message);
}

Component Design - WebLog

Before exploring the WebLog module, let's take a detailed look at the database tables that are used to store the log information.

Database Design

As we mention earlier, storing the error information into a database table helps us to better analyze the errors that are raised from the web application. In order to help achieve this, we create two tables, LogManager and LogError. The LogManager table stores the log configuration information, and the LogError table logs the error. Now let's see each of them in detail.

The screenshot below show the list of tables that are added to the database:

LogManager Table

The LogManager table is used to store the configuration setting information that is used to manage the error information in the LogError table. This table has three fields:

ManageType Field

This column holds the type of log management. The column can have the following management types:

  • No Management (N) : If the ManageType is set to "N", then the error log will not be stored in the database.
  • Temporary Management (T) : If the ManageType is set to "T", then the error information will only be stored in the LogError table temporarily. The administrator of the web site can define the time period, or the life span of the logs stored in the database. After the specified period of time has elapsed, the logs that are stored in the database are deleted.
  • Permanent Management (P) : In the ManageType is set to "P", then the logs are not deleted automatically. They remain in the database until the administrator manually deletes them from the table.

NumberOfDays Field

This field specifies the total number of days that the log messages are to remain in the database. This can be set only if the ManageType is set to "T" (Temporary Management).

DateLastDelete Field

This field stores the date on which the records in the LogError table were last deleted.

LogError Table

The LogError table logs the exceptions that are raised. Some of the information it records is ServerName, UserAgent, UserIP, Authentication, UserName, Error Message, Error Source, Time of Error, and Error page. Let's now examine the LogManager class.

The LogManager class has three methods: ConnectDatabase, WriteLog, and WriteEventLog. In this class we import the following namespaces: System, System.Data, System.Data.Common, System.Data.SqlClient, System.Data.OleDb, System.Web, and System.Diagnostics.

The diagram below shows all the members in the LogManager class:

using System;
using
            System.Data;
using System.Data.Common;
using
            System.Data.SqlClient;
using System.Data.OleDb;
using
            System.Text;
using System.Web ;
using System.IO;
namespace
            ErrorLog.WebLog{
/// <summary>
/// Summary description
            for WebLog.
/// </summary>
public class
            LogManager
{
/// <summary>
/// LogManager Helps to
            Manage and Config the Log Information.
///
            </summary>
public LogManager()
{
}
protected
            HttpContext HttpInfo = HttpContext.Current;
protected
            IDbConnection conPublic =null;
public enum DbType
            {Others,SQLServer};

ConnectDataBase

This method is used to connect to the database. It is capable of connecting to any OleDb compliant database server specified by the user. This is achieved by using the IDbConnection class in the System.Data namespace to create the connection.

The .NET data provider has four key classes. They are implemented from the IDbConnection, IDbCommand, IDataReader, and IDbDataAdapter interfaces which are found in the System.Data namespace. We use this class to connect to the database and manipulate the data held within.

These four classes are manifested in their SQL Server or OLE DB providers as follows: SqlConnection and OledbConnection for the Connection class; SqlCommand and OledbCommand for the Command class; SqlDataReader and OledbDataReader for the DataReader class; SqlDataAdapte and OledbDataAdapter for the DataAdapter class. We can use them to support any type of data source that is compliant with OleDb.

Note : We could simply use the managed OleDb provider that supports all databases, including SQL Server. However, considering the performance advantages, we will treat SQL Server and other databases differently.

/// <summary>
/// Used for
            connecting to the database
/// </summary>
/// <param
            name="ConnectionString">Database Connection
String</param>
/// <param
            name="DataBaseType">Database Type Support
for Oracle and SQL
            Server</param>
/// <returns>Returns
            IDbConnection</returns>
public IDbConnection
            ConnectDataBase(string ConnectionString,
DbType
            DataBaseType)
{
//Initializating the Connection
            object
//depending upon the datasourse

if
            (DataBaseType.Equals(DbType.Others))
{

// Oledb example
            Oracle , Access

conPublic = new
            OleDbConnection(ConnectionString);
}
else if
            (DataBaseType.Equals(DbType.SQLServer))
{

//SQL
            Server

conPublic = new
            SqlConnection(ConnectionString);
}
HttpInfo.Session["ConnString"]=ConnectionString;
HttpInfo.Session["DataBaseType"]=DataBaseType;
IDataReader
            idRCFL =null;

// Opening the
            Connection

conPublic.Open();
string strSQL = "Select *
            from LogManager";
IDbCommand idbCFL=null;
idbCFL=
            conPublic.CreateCommand();
idbCFL.CommandText
            =strSQL;
idbCFL.Connection = conPublic;
idRCFL=
            idbCFL.ExecuteReader(CommandBehavior.CloseConnection);

//
            checking for data in LogManager table using
// DataReader read
            method

if (idRCFL.Read())
{
//if management type is T-
            Temporary

if (idRCFL.GetString(0).Equals("T"))
{
if
            (DataBaseType.Equals(DbType.Others))
{
//Oracle Supported SQL
            query to delet data //from logError table

strSQL = "Delete
            from LogError where TimeofError Like
SysDate - " +
            idRCFL.GetInt32(1).ToString();
}
else if
            (DataBaseType.Equals(DbType.SQLServer))
{
//SQLServer
            Supported

strSQL = "Delete from LogError where TimeofError =
            DATEADD ( day ,
" + idRCFL.GetInt32(1).ToString() + " ,
'" +
            DateTime.Today.ToShortDateString() + "' ) ";
}

//if
            Connection open then close it and open
//else open the
            connection

if (conPublic.State == ConnectionState.Open )
{
conPublic.Close
            ();
conPublic.Open();
}
else
{
conPublic.Open();
}
IDbCommand
            idbC=null;
idbC= conPublic.CreateCommand();
idbC.CommandText
            =strSQL;
idbC.Connection = conPublic;

// Executing the
            query using CommandObject

int
            intDelCount=idbC.ExecuteNonQuery();

//if any record deleted
            the DateLastDeleted field
// is updated to the current
            date

if (intDelCount!=0)
{
//if Connection open then
            close it and open
//else open the connection

if
            (conPublic.State == ConnectionState.Open )
{
conPublic.Close
            ();
conPublic.Open();
}
else
{
conPublic.Open();
}
IDbCommand
            idbCidRUpdate=null;
idbCidRUpdate=
            conPublic.CreateCommand();
idbCidRUpdate.CommandText ="Update
            LogManager set DateLastDeleted =
'" +
            DateTime.Today.ToShortDateString() + "'"
            ;
idbCidRUpdate.Connection =
            conPublic;
idbCidRUpdate.ExecuteNonQuery();
}
}
}
return conPublic;
}

Apart from the connection to the database, this method also checks for the ManageType field in the LogManager table. If the ManageType is set to "T" (Temporary Management) then it deletes all the records in the LogError table that have expired. The expiry date is found by taking the NumberOfDays field in the LogManger table. After deleting the record, this method helps to update the DateLastDelete field with the current date.

WriteLog

The WriteLog method is used to write the error information to the LogError database table. This method takes the Connection String, Data Source type, and the Exception as the input arguments, and returns an integer value. The ManageType status in the LogManager table is checked by this method, and if it is set as either "P" or "T" then the date is inserted into the log.

/// <summary>
/// Writes the Log to
            the database specified in the connection string
///
            </summary>
/// <param name="ConnectionString">Writes
            the Log to the database
specified in the connection
            string</param>
/// <param
            name="DataBaseType">Database Type Support for Oracle and
SQL
            Server</param>
public int WriteLog(Exception
            CurrException,string ConnectionString,DbType
            DataBaseType)
{
try
{
string strServerName ="";
string strUserAgent ="";
string strUserIP="";
string
            strAuthentication="";
string strUserName="";
string
            strMessage="";
string strSource="";
string
            strTargetCode="";
string strTimeOfError=
            DateTime.Now.ToString();
string strStackTrace="";
string
            strURL="";
string strLastDeleteDate ="";
string strSQL="";
string strMType= "";

StringBuilder sbSQL = new
            StringBuilder();
IDbConnection idConn=null;
//Using
            ServerVariable to get Server Name from HTTP Header
strServerName
            = HttpInfo.Request.ServerVariables["SERVER_NAME"] ;

//Using
            Request object to get UserAgent from HTTP Header
strUserAgent=
            HttpInfo.Request.UserAgent;

//Using Request object to get
            UserAgent from HTTP Header
strUserIP=
            HttpInfo.Request.UserHostAddress ;

//Using HttpContext class
            User property to get
//security information for the current HTTP
            request.
strAuthentication=
            HttpInfo.User.Identity.IsAuthenticated.ToString() ;

//to get
            user identity
strUserName= HttpInfo.User.Identity.Name
            ;

//Get Error Message
strMessage= CurrException.Message
            ;

//Error Source
strSource= CurrException.Source
            ;

//Gets the method that throws the current
            exception
strTargetCode=CurrException.TargetSite.ToString();

//Gets a string representation of the frames on the
//call stack at the time the current exception was
            thrown.
strStackTrace= CurrException.StackTrace ;

//URL of
            the Error
strURL = HttpInfo.Request.Url.ToString()
            ;

IDataReader idRCFL =null;

//using
            ConnectDataBase to get the Connection
            Object
idConn=ConnectDataBase(ConnectionString,DataBaseType);

//if
            Connection open then close it and open
//else open the
            connection
if (conPublic.State == ConnectionState.Open )
{
idConn.Close();
idConn.Open();
}
else
{
idConn.Open();
}
strSQL
            = "Select * from LogManager";
IDbCommand idbCFL=null;
idbCFL=
            idConn.CreateCommand();
idbCFL.CommandText
            =strSQL;
idbCFL.Connection = idConn;
idRCFL=
            idbCFL.ExecuteReader(CommandBehavior.CloseConnection);

//
            checking for data in LogManager table using
// DataReader read
            method
if (idRCFL.Read())
{
// get Management
            Type
strMType = idRCFL.GetString(0).Trim();

//insert the
            Error Log information if the
//Management Type is onlt T-
            Temporaty and P- //Permanent
if
            (strMType.Equals("T")||strMType.Equals("P"))
{

//Building
            SQL insert query using StringBuilder
sbSQL.Append("Insert into
            LogError values('" + strServerName + "','" );
sbSQL.Append (
            strUserAgent + "','" + strUserIP + "','" +
            strAuthentication);
sbSQL.Append ("','" + strUserName + "','" +
            strMessage + "','" + strSource);
sbSQL.Append ("','" +
            strTargetCode + "','" + strStackTrace + "','" );
sbSQL.Append
            (strTimeOfError + "','" + strURL + "')");
IDataReader idR=
            null;
if (conPublic.State == ConnectionState.Open )
{
idConn.Close
            ();
idConn.Open();
}
else
{
idConn.Open();
}
IDbCommand
            idbC=null;
idbC= idConn.CreateCommand();
idbC.CommandText
            =sbSQL.ToString();
idbC.Connection = idConn;

//Executing
            the Insert Query Using IDBCommand
            //Object
idR=idbC.ExecuteReader(CommandBehavior.CloseConnection);
}
}
return
            0;
}
catch
{
return 1;
}
}

This method gets error information like Error Source, Error message, Error Code, and StackTrace from the exception thrown by the application which is currently executing. The HttpContext class is used to get the HTTP-specific information for the requested page that threw the error.

Event logging provides a standard, centralized way for applications to record important software and hardware events. Windows supplies a standard user interface for viewing the logs, the Event Viewer. By using the common language run-time's EventLog component, we can easily connect to the existing event logs on both local and remote computers and write entries to these logs. We can also read entries from existing logs and create our own custom event logs. The code below uses the CreateEventSource method in the EventLog class of the System.Diagnostics namespace to create an event source, if it does not already exist, and logs the error information.

public void WriteEventLog(String
            eventLog)
{
// Create the source, if it does not already
            exist.
if(!EventLog.SourceExists("ASPX
            Error"))
{
EventLog.CreateEventSource("ASPX Error", "WebSite
            Error");
}
// Create an EventLog instance and assign its
            source.
EventLog elError = new EventLog();
elError.Source =
            "ASPX Error";

// Write an informational entry to the event
            log.
elError.WriteEntry(eventLog);
}

Component Design - Web Exception

The WebException module has an ErrorHandler class that is used to get the error information and alert the administrator of the site of the error via e-mail, or to log the error information either in the text file or in the database table using the AlertManager and LogManager classes. This class has various methods to perform this activity.

The diagram below shows all the members of the ErrorHandler class:

using System;
using System.Web;
using System.Data;
using System.Text;
using
            System.IO;
using ErrorLog.WebLog;
using
            ErrorLog.WebAlert;
using System.Xml;
namespace
            ErrorLog.WebException
{
public class ErrorHandler
{
//To
            Build Error Information
private StringBuilder sbFile;
// To
            Build ASPX Error Page
private StringBuilder sbASPX;
// Get
            Receiver Email ID
private string strTo = null;
// Get Sender
            Email ID
private string strFrom = null;
//Get Mail
            Subject
private string strSubject = null;
//Get SMTP
            Server
private string strSmtpServer = null;
// Ger ConfigFile
            Path
private string strConfigFile = null;

HandleException

The HandleException method is used to get the last error from the web application. This method takes the configuration file path as the input argument and passes it, along with the exception, to the overloaded version of the HandleException.

/// <summary>
/// Method for
            getting the Last Error from the WebServer.
///
            </summary>
/// <param name="ConfigFilePath">XML
Configration File </param>
public void
            HandleException(String ConfigFilePath)
{
// Get's last error
            to the HTTP output stream.
Exception e =
            HttpContext.Current.Server.GetLastError();
if (e ==
            null)
return;
//Get's the root cause exceptions of one or
            more subsequent
//exceptions.
e = e.GetBaseException();
if (e != null)
//calling the method
            HandleException
HandleException(e);
}

HandleException - Overload Method

The overloaded version of HandleException gets the configuration settings using the ConfigurationSettings method from the configuration file and, depending upon the configuration setting, calls the methods WriteToFile, SendMail, LogEvent, SendSMS, and WriteEventLog. The error information is formatted using the method FormatExceptionDescription before sending it as an e-mail or writing it to the text file. Once this is done, we redirect to the Errorpage.

/// <summary>
/// Overloaded Method
            for Handling the Exception with parameter as Exception.
///
            </summary>
/// <param
            name="e">Exception</param>
/// <param
            name="ConfigFilePath">XML Configration File</param>
public void HandleException(Exception e)
{
string
            sExceptionDescription = FormatExceptionDescription(e);
// Get
            the FileNotify Option from Config file
if (
            ConfigurationSettings("FileNotify").ToUpper().Equals
            ("TRUE"))
{
// call the method to write in text
            file
WriteToFile(sExceptionDescription);
}
if
            (ConfigurationSettings("EmailNotify").ToUpper().Equals
            ("TRUE"))
{
// call the method to send
            email
SendEmail(sExceptionDescription);
}
if
            (ConfigurationSettings("LogNotify").ToUpper().Equals
            ("TRUE"))
{
//call the method to log in
            database
LogEvent(e);
}
if
            (ConfigurationSettings("SMSNotify").ToUpper().Equals
            ("TRUE"))
{
//call the method to send SMS
SendSMS(e);
}
if (ConfigurationSettings("EventLogNotify").ToUpper().Equals
            ("TRUE"))
{
//call the method to log in
            eventLog
WriteEventLog(sExceptionDescription);
}

//Creating
            the Error Page
FileInfo finfo = new
FileInfo(HttpContext.Current.Request.PhysicalApplicationPath +
            "ErrPage.aspx");

//Calling CreateErrorPage method to create
            ASPX Error
// File

CreateErrorPage();

//Directing
            the page to Just created Error
            page
HttpContext.Current.Server.Transfer("ErrPage.aspx");
}

Note : The configuration file is an XML file which is used to get information like error log preference, the connection string, the data source type, SMTP server, the e-mail ID of "From", the e-mail ID of "To", etc.

The ConfigurationSettings method is used to get the configuration information. This method gets the tag name as an input parameter and returns the value using the method GetElementsByTagName.

FormatExceptionDescription

The FormatExceptionDescription method takes the exception as an argument and formats the exception using the StringBuilder class in the System.Text namespace. The StringBuilder stores the error information like Error Source, Error message, Error Code, and StackTrace from the exception that was passed in. The HttpContext class is used to get the HTTP-specific information for the requested page which threw the error. This information is appended along with the error information, and is returned as a string. This method builds the error information in text and HTML format. The HTML formatted information is stored in a text file called ErrHTML.txt. The information in the ErrHTML.txt file is displayed in the error page generated using the CreateErrorPage method.

protected virtual string FormatExceptionDescription(Exception e)
{
// to Build Error Information to log in to TextFile
sbFile = new StringBuilder();
// to Build Error Information in HTML format to show in error page
sbASPX = new StringBuilder();
HttpContext context = HttpContext.Current;

//Building the Error Information for Text File
sbFile.Append("Time of Error: " + DateTime.Now.ToString("g") + Environment.NewLine);
sbFile.Append("URL : " + context.Request.Url + Environment.NewLine);
sbFile.Append("Form : " + context.Request.Form.ToString() + Environment.NewLine);
sbFile.Append("QueryString : " + context.Request.QueryString.ToString()
+ Environment.NewLine);
sbFile.Append("Server Name : " + context.Request.ServerVariables["SERVER_NAME"]
+ Environment.NewLine);
sbFile.Append("User Agent : " + context.Request.UserAgent
+ Environment.NewLine);
sbFile.Append("User IP : " + context.Request.UserHostAddress
+ Environment.NewLine);
sbFile.Append("User Host Name : " + context.Request.UserHostName
+ Environment.NewLine);
sbFile.Append("User is Authenticated : "
+ context.User.Identity.IsAuthenticated.ToString() + Environment.NewLine);
sbFile.Append("User Name : " + context.User.Identity.Name + Environment.NewLine);
sbFile.Append("Message : " + e.Message + Environment.NewLine);
sbFile.Append("Source : " + e.Source+ Environment.NewLine);
sbFile.Append("TargetSite : " + e.TargetSite + Environment.NewLine);
sbFile.Append("StackTrace : " + e.StackTrace + Environment.NewLine);
sbFile.Append("XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX");
sbFile.Append(Environment.NewLine + Environment.NewLine );

//Similarly Building Error information in HTML Format

sbASPX.Append("<table><tr><td>Time of Error: </td><td>" + DateTime.Now.ToString("g")
+ "</td></tr>");
sbASPX.Append("<tr><td>URL : </td><td>" + context.Request.Url + "</td></tr>");
sbASPX.Append("<tr><td>Form : </td><td>" + context.Request.Form.ToString() + "</td></tr>");
sbASPX.Append("<tr><td>QueryString : </td><td>" + context.Request.QueryString.ToString()
+ "</td></tr>");
sbASPX.Append("<tr><td>Server Name :</td><td> "
+ context.Request.ServerVariables["SERVER_NAME"]
+ "</td></tr>");
sbASPX.Append("<tr><td>User Agent : </td><td>" + context.Request.UserAgent + "</td></tr>");
sbASPX.Append("<tr><td>User IP : </td><td>" + context.Request.UserHostAddress
+ "</td></tr>");
sbASPX.Append("<tr><td>User Host Name : </td><td>" + context.Request.UserHostName
+ "</td></tr>");
sbASPX.Append("<tr><td>User is Authenticated : </td><td>"
+ context.User.Identity.IsAuthenticated.ToString()
+ "</td></tr>");
sbASPX.Append("<tr><td>User Name : </td><td>" + context.User.Identity.Name
+ "</td></tr>");

while (e != null)
{
sbASPX.Append("<tr><td>Message : </td><td>" + e.Message + "</td></tr>");
sbASPX.Append("<tr><td>Source : </td><td>" + e.Source + "</td></tr>");
sbASPX.Append("<tr><td>TargetSite : </td><td>" + e.TargetSite + "</td></tr>");
sbASPX.Append("<tr><td>StackTrace : </td><td>" + e.StackTrace + "</td></tr>");
e = e.InnerException;
}
sbASPX.Append("</table>");
//Saving the HTML part in the Text file.
string sPath = HttpContext.Current.Server.MapPath("ErrASPX.txt");
try
{
FileStream fs = new FileStream(sPath, FileMode.Create, FileAccess.Write);
StreamWriter sWriter = new StreamWriter(fs);
sWriter.Write(sbASPX.ToString());
sWriter.Close();
fs.Close();
}
catch (FieldAccessException)
{

}
//returns the Error Information build for text file
return sbFile.ToString();
}

Once the error information is built, it looks like the one shown below:

Time of Error: 8/27/2002 2:15 PM
URL : http://localhost/ErrorReportTest/WebForm1.aspx
Form :
QueryString :
Server Name : localhost
User Agent : Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1;
.NET CLR 1.0.3705)
User IP : 127.0.0.1
User Host Name : 127.0.0.1
User is Authenticated : False
User Name :
Message : Attempted to divide by zero.
Source : ErrorReportTest
TargetSite : Void Page_Load(System.Object, System.EventArgs)
StackTrace : at ErrorReportTest.WebForm1.Page_Load(Object sender,
EventArgs e)
in d:\inetpub\wwwroot\errorreporttest\webform1.aspx.cs:line 25
at System.Web.UI.Control.OnLoad(EventArgs e)
at System.Web.UI.Control.LoadRecursive()
at System.Web.UI.Page.ProcessRequestMain()
======================================================================

WriteToFile

The WriteToFile method takes the formatted error information as an input parameter and writes the information into a text file using the FileStream object.

/// <summary>
/// Method for
            writing the Exception to a file.
/// </summary>
///
            <param name="sText">String To write in file</param>
void WriteToFile(string sText)
{
// gets the Errtxt file
            path to write the error info
string sPath =
            HttpContext.Current.Server.MapPath("Errtxt.txt");
try
{
//
            writing the information in to the file
// using file stream
            object
FileStream fs = new FileStream(sPath, FileMode.Append,
            FileAccess.Write);
StreamWriter sWriter = new
            StreamWriter(fs);
sWriter.Write(sText);
sWriter.Close();
fs.Close();
}
catch
            (Exception )
{
}
}

SendMail

The SendMail method helps to send the error information to the e-mail addresses stored in the configuration file using the AlertManager class. The EmailNotification function is called within this method, and is used to get and store the mail information from the configuration file. This information is then used for framing the e-mail.

/// <summary>
/// Method for
            mailing the Exception.
/// </summary>
/// <param
            name="sBody">String Body to Send in Mail</param>
void
            SendEmail(string sBody)
{
// calling the EmailNotification to
            set the email
//property
EmailNotification();
//initilizating the AlertManager class
AlertManager objMail =
            new AlertManager();
objMail.From = strFrom;// setting sender
            ID
objMail.To =strTo;// setting receiver id
objMail.SMTPServer
            = strSmtpServer; // SMTP Server
objMail.Subject = strSubject; //
            Mail Subject
// Mail Format HTML Format
objMail.Format=
            AlertManager.MessageFormat.Html ;
// Error information as Mail
            Body
objMail.Body = sBody;
// sending mail using SendMail
            method
objMail.SendMail();
}

LogEvent

The LogEvent method is used to log the exception into the database using the ConnectDataBase and WriteLog methods in the LogManager class. The LogEvent method gets the connection string and data source type from the configuration file and passes these values as input parameters to the ConnectDatabase method. After connecting to the database, the WriteLog method is called which takes the exception as an argument.

/// <summary>
/// Method for
            logging the Exception to the DataBase Log.
///
            </summary>
/// <param
            name="Ex">Exception</param>
void LogEvent(Exception
            Ex)
{
string strConn="";
// getting the connection string
            from Config File
strConn =
            ConfigurationSettings("ConnectionString");
////initilizating the
            LogManager class
LogManager objLog= new
            LogManager();
//Checking the database type in config file using
//ConfigurationSettings method
if
            (ConfigurationSettings(@"DatabaseType").ToLower().Equals("Others"))
{
//using
            the WriteLog method to write in to
            db
objLog.WriteLog(Ex,strConn,LogManager.DbType.Others);
}
else
{
//objLog.ConnectDataBase(strConn,LogManager.DbType.SQLServer);
objLog.WriteLog(Ex,strConn,LogManager.DbType.SQLServer);
}
}

The WriteEventLog method is used to log the exception into the system EventLog using the method WriteEventLog in the LogManager class.

void WriteEventLog(string
            sText)
{
////Initilizating the LogManager class
LogManager
            lmEventLog = new LogManager();
lmEventLog.WriteEventLog(sText);
}

The screenshot below shows the error log information in the system Event Handler:

The SendSMS method is used to send the error information as an SMS to a mobile phone. This method gets the exception as an argument, builds the message, and sends it using SendSMS method in the AlertManager class.

void SendSMS(Exception
            Ex)
{
//Initilizating the AlertManager class
AlertManager
            amSMS = new AlertManager();
HttpContext context =
            HttpContext.Current;
string strMSG =null;
// Building the
            error message
strMSG = "Error in Web App: Time " +
            DateTime.Now.ToString("g") + " Error:"
+ Ex.Message + " URL :" +
            context.Request.Url ;
//sending the SMS
amSMS.SendSMS
            (ConfigurationSettings("SMSSenderName"),
ConfigurationSettings("SMSMobileNo"),
            strMSG);
}

User Interface

Until now we have just seen the components that are used for logging and reporting the errors. Now we shall see the user interfaces that are used to configure and report the errors. We create two ASPX web pages:

  • ManageConfig.aspx
  • ManageLog.aspx

Now let's look at both of these two web pages in detail.

ManageConfig

The ManageConfig.aspx page is used to set the values in the database table LogManager. This WebForm has a DropDownList control cboMType that lists the management types that we saw earlier (where "P" stands for Permanent, "T" for Temporary, and "N" for No Management), and a TextBox control txtDays, which is used to get the time period for deleting records. The WebForm also has a Submit button that is used to update the changes to the database.

<html>
<head>
<title>ManageConfig</title>
</head>
<body ms_positioning="GridLayout">
<form id="ManageConfig" method="post" runat="server">
<h2>Manage Config</h2>
<table>
<tbody>
<tr>
<td>
<asp:Label id="lblMType" runat="server" CssClass="Label">Management Type</asp:Label>
</td>
<td>
<asp:DropDownList id="cboMType" runat="server" AutoPostBack="True"
OnSelectedIndexChanged="cboMType_SelectedIndexChanged">
<asp:ListItem Value="N">No Management</asp:ListItem>
<asp:ListItem Value="T">Temporary Management</asp:ListItem>
<asp:ListItem Value="P">Permanent Management</asp:ListItem>
</asp:DropDownList>
<td>
</tr>

<tr>
<td>
<asp:Label id="lblDay" runat="server">No of Days</asp:Label>
</td>
<td>
<asp:TextBox id="txtDays" runat="server"></asp:TextBox>
</td>
</tr>

<tr>
<td>
<asp:Label id="lblLUpdate" runat="server">Date of LastUpdate</asp:Label>
</td>
<td>
<asp:Label id="lblLUpdateValue" runat="server"></asp:Label>
</td>
</tr>

<tr>
<td align="middle">
<asp:Button id="cmdSave" runat="server" Text="Save" onClick="cmdSave_Click">
</asp:Button>
</td>

<td align="middle">
<asp:Button id="cmdClose" runat="server" Text="Close"></asp:Button>
</td>
</tr>

</tbody>
</table>
</form>
</body>
</html>

The ManageConfig.aspx page has three events and one method. The event handlers are Page_Load, cmdSave_Click, and cboMType_SelectedIndexChanged, and the method is GetFromDB. A Page_load () event is triggered every time a page gets loaded. If the page is not a post back, the page_load() event handler calls the method GetFromDB. Since the GetFromDB() method is called once, we must switch on the View state of the page. All the code is written inside the server script block of the ASPX page.

private void Page_Load(object sender,
            System.EventArgs e)
{
// Triggers when the page load
// for
            first time
if (!Page.IsPostBack)
{
GetFromDB();
}
}

GetFromDB

The GetFromDB method is used to get the configuration information from the database table LogManager and assigns the value to the fields in the WebForm. This method uses the ConnectDatabase method from the LogManager class to connect to the database.

private void GetFromDB()
{
//
            Initializating the LogManager Class
LogManager logMan=new
            LogManager();
IDbConnection conPublic =null;
string strSQL="";
IDbConnection idl=null;
IDataReader idRCFL =null;
//
            getting the database type from config file
if
            (ConfigurationSettings("DatabaseType").ToLower().Equals("Others"))
{
//connecting
            to Oledb database using ConnectDataBase
conPublic =
            logMan.ConnectDataBase(ConfigurationSettings("ConnectionString"),
LogManager.DbType.Others);
}
else
{
//connecting
            to SQLServer database using ConnectDataBase
conPublic =
            logMan.ConnectDataBase(ConfigurationSettings("ConnectionString"),
LogManager.DbType.SQLServer);
}
//if
            Connection open then close it and open
//else open the
            connection

if
            (conPublic.State.ToString().Equals("Open"))
{
conPublic.Close
            ();
conPublic.Open();
}
else
{
conPublic.Open();
}
strSQL
            = "Select * from LogManager";
IDbCommand idbCFL=null;
idbCFL=
            conPublic .CreateCommand();
idbCFL.CommandText =strSQL;
idRCFL= idbCFL.ExecuteReader(CommandBehavior.CloseConnection);
// Getting the Config Information from LogManager
// table to
            show in the Web Form
if (idRCFL.Read())
{
//setting last
            deleted date in the label field
lblLUpdateValue.Text =
            idRCFL.GetDateTime(2).ToString();
//setting the noofdays value in
            the textbox
txtDays.Text
            =idRCFL.GetInt32(1).ToString();
//assigning the Management Type
            in dropdownlist
if
            (idRCFL.GetString(0).Trim().Equals("N"))
{
//if managementype
            is N-No Management
//disable the textbox so no updates can be
            made
cboMType.Items[0].Selected =true;
txtDays.Enabled
            =false;
}
else if
            (idRCFL.GetString(0).Trim().Equals("T"))
{
cboMType.Items[1].Selected
            =true;
txtDays.Enabled =true;
}
else if
            (idRCFL.GetString(0).Trim().Equals("P"))
{
//if managementype
            is P- Permanent
//disable the textbox so no updates can be
            made
cboMType.Items[2].Selected =true;
txtDays.Enabled
            =false;
}
}
}

The cboMType_SelectedIndexChanged event enables the textbox txtDays if the user selects the "T" (temporary) management type. If any other management type is selected, then we do not need txtDays to be active.

private void
            cboMType_SelectedIndexChanged(object sender, System.EventArgs
            e)
{
if
            (cboMType.SelectedItem.Value.Equals("P")||cboMType.SelectedItem.Value.Equals("N")
            )
{
txtDays.Enabled=false
            ;
}
else
{
txtDays.Enabled=true ;
}
}

The cmdSave_Click() event handler is triggered when the user clicks on the Save submit button after changing the configuration. This event uses the ConnectDatabase method of the LogManger class to connect to the database, which returns a connection object depending upon the data source type. This connection object is used to update the configuration setting in the LogManger table using IDbCommand object. The GetFromDB method is called after updating.

private void cmdSave_Click(object sender,
            System.EventArgs e)
{
if (Page.IsPostBack)
{
//
            Initializating the LogManager Class
LogManager logMan=new
            LogManager();
IDbConnection idlUpdate=null;
// getting the
            database type from config file
if
            (ConfigurationSettings(@"DatabaseType").ToLower().Equals("Others"))
{
//connecting
            to Oledb database using ConnectDataBase

idlUpdate =
            logMan.ConnectDataBase(ConfigurationSettings("ConnectionString"),
LogManager.DbType.Others);
}
else
{
//connecting
            to SQL Server database using ConnectDataBase

idlUpdate =
            logMan.ConnectDataBase(ConfigurationSettings("ConnectionString"),
LogManager.DbType.SQLServer);
}
//building
            the Update Query
string strSQL = "Update LogManager set
            ManageType='"
+ cboMType.SelectedItem.Value
+ "',
            Numberofdays=" + txtDays.Text.Trim();
if
            (idlUpdate.State.ToString().Equals("Open"))
{
idlUpdate.Close
            ();
idlUpdate.Open();
}
else
{
idlUpdate.Open();
}
IDbCommand
            idbCUpdate=null;
idbCUpdate=
            idlUpdate.CreateCommand();
idbCUpdate.CommandText =strSQL;
idbCUpdate.ExecuteReader(CommandBehavior.CloseConnection);
// Executing the Query and calling the Method
            GetFromDB()
//to bind the data to the grid
GetFromDB();
}
}

ManageLog

The ManageLog.aspx page is used to view the log information that is stored in the database. In this page we also provide a search mechanism that helps to list the error information logged between two specified dates. The ManageLog.aspx page also has a delete option, which we can use to permanently delete error information from the database.

In the ManageLog web form, we place an ASP.NET DataGrid server control. This DataGrid control displays the error information from the LogError table. We place two TextBox control and a Submit button (cmdSearch) on the page. This control is used for the Search feature that is incorporated into the WebForm. Below the grid we place a Submit button that is used to delete the error log information from the database.

The figure below shows how the ManageLog.aspx page should look:

The ManageLog WebForm has three events and three methods. The event handlers are Page_Load, cmdDelete_Click, and cmdSearch_Click. The methods are CreateDSource(), GridDataBind(), and DeleteRow().

A Page_load () event is triggered every time a page gets loaded. If the page is not a post back, the page_load() event handler calls the method GridDataBind(). When the cmdClose button is clicked, we run a JavaScript command to close the window.

protected bool SearchFor;

private
            void Page_Load(object sender, System.EventArgs e)
{
// To Run
            the javascript comman to close the form
cmdClose.Attributes.Add("OnClick", "window.close();");
//
            Initilizating the LogManager class
LogManager logMan=new
            LogManager();
// Triggers when the page load
// for first time
if (!Page.IsPostBack)
{
//disabling the
            SearchFeature
SearchFor =false;
//calling the GridDataBind()
            method
// to bind the data in to the grid
GridDataBind();
}
}

The GridDataBind() binds the record in the DataGrid dgInfo from the table LogError using the method CreateSource().

public void GridDataBind()
{
//Binds
            the data to grid if CreateSource()
//
            return more then 0
if
            (CreateDSource().Count!=0)
{
dgLogInfo.DataSource =
            CreateDSource();
dgLogInfo.DataBind();
}
else
{
dgLogInfo.Visible
            =false;
}
}

The CreateSource() method returns a DataTable, which is built from the data in the LogError table. This method uses the ConnectDatabase method of the LogManager class to connect to the database.

ICollection CreateDSource()
{
string
            strSQL="";
IDbConnection idl=null;
IDataReader idRCFL
            =null;
//Initilizating the LogManager class
LogManager
            logMan=new LogManager();
//Initilizating the DataTable to
            store
// the data
DataTable dtLog = new
            DataTable("LOG");
DataRow drNewRow;
//Creating datatable
            columns
dtLog.Columns.Add("LogID");
dtLog.Columns.Add("ServerName");
dtLog.Columns.Add("Message");
dtLog.Columns.Add("Source");
dtLog.Columns.Add("TimeofError");
dtLog.Columns.Add("URL");

//
            getting the database type from config file
if
            (ConfigurationSettings("DatabaseType").ToLower().Equals("Others"))
{
//connecting
            to Oledb database using ConnectDataBase
idl =
            logMan.ConnectDataBase(ConfigurationSettings("ConnectionString"),
LogManager.DbType.Others);
}
else
{
//connecting
            to SQLServer database using ConnectDataBase
idl =
            logMan.ConnectDataBase(ConfigurationSettings("ConnectionString"),
LogManager.DbType.SQLServer);
}
//if
            Connection open then close it and open
//else open the
            connection

if (idl.State== ConnectionState.Open
            )
{
idl.Close
            ();
idl.Open();
}
else
{
idl.Open();
}

//
            Building SQL Query depending upon searchFor bool

if
            (SearchFor==true)
{
//Enabling search by filtering the record
            with respect to
// date
strSQL = "Select
            LogID,ServerName,Message,Source,TimeOfError,
URL from LogError
            where TimeofError between '"
+ txtFrom.Text.Trim() + "' and '" +
            txtToDate.Text.Trim() + "'";
}
else
{
strSQL = "Select
            LogID,ServerName,Message,Source,TimeOfError,
URL from
            LogError";
}
IDbCommand idbCFL=null;
idbCFL=
            idl.CreateCommand();
idbCFL.CommandText
            =strSQL;
idbCFL.Connection = idl;
//executing the sql query
            using ExecuteReader
idRCFL=
            idbCFL.ExecuteReader(CommandBehavior.CloseConnection);
//Reading
            the Record
while (idRCFL.Read())
{
//stroing the record in
            Datatable
drNewRow=
            dtLog.NewRow();
drNewRow[0]=idRCFL.GetInt32(0);
drNewRow[1]=idRCFL.GetString(1);
drNewRow[2]=idRCFL.GetString(2);
drNewRow[3]=idRCFL.GetString(3);
drNewRow[4]=idRCFL.GetDateTime(4);
drNewRow[5]=idRCFL.GetString(5);
dtLog.Rows.Add(drNewRow);
}
DataView dv = new
            DataView(dtLog);
return dv;
}

The cmdDelete_Click() event is raised every time the user clicks the cmdDelete button after selecting the rows in the DataGrid for deletion. This event calls the function DeleteRow(), which takes the LogID as an input parameter and deletes the corresponding rows from the LogError table.

private void cmdDelete_Click(object sender,
            System.EventArgs e)
{
CheckBox
            chkBoxCurrent;
HtmlInputHidden txtHID;
for (int i=0; i <=
            dgLogInfo.Items.Count -1 ; i++)
{
//assigning the CheckBox in
            the Grid Row to the
// Checkbox initilized
chkBoxCurrent =
            (CheckBox)dgLogInfo.Items[i].FindControl("chkCodRow")
            ;

//assigning the HiddenText box in the Grid Row to the
// HtmlInputHidden initilized
txtHID=
            (HtmlInputHidden)dgLogInfo.Items[i].FindControl("myHid") ;

if
            (chkBoxCurrent!=null)
{
//checking if the check box in the row
            is checked
if (chkBoxCurrent.Checked)
{
//calling the
            method DeleteRow to delete
// the record in the database for the
            value
// in the
            Hiddenfield
DeleteRow(Int32.Parse(txtHID.Value));
}
}
}
SearchFor
            =false;
// Binding the update records in to the
            grid
GridDataBind();
}

The cmdSave_Click() event sets the SearchFor variable to True, which enables the search feature in the method CreateSource. This event calls the method GridDataBind().

private void cmdSearch_Click(object sender, System.EventArgs e)
{
SearchFor =true;
GridDataBind();
}

We have now built the component and the user interfaces of the web site error logging and reporting system. Now we will see how to integrate it into a web application.

Integrating Web Site Error Logging and Reporting System

The first task in integrating the web site error logging and reporting system is to create the configuration file, which stores the information necessary to run the reporting system.

The configuration file is an XML file which shall be named ErrorSetting.config.

<ErrorInfo>
<SMTPServer>SVK</SMTPServer>
<EmailFrom>errorreporter@dotnetforce.com</EmailFrom>
<EmailTo>admin@dotnetforce</EmailTo>
<EmailSubject>DotNetForce</EmailSubject>
<ConnectionString>data source=svk;initial catalog=WebSiteErrorReporting;
persist security info=False;user id=sa;workstation id=SVK;packet size=4096
</ConnectionString>
<DatabaseType>SQLServer</DatabaseType>
<FileNotify>True</FileNotify>
<EmailNotify>True</EmailNotify>
<LogNotify>True</LogNotify>
<SMSNotify>False</SMSNotify>
<EventLogNotify>True</EventLogNotify>
<SMSSenderName>Vinod</SMSSenderName>
<SMSMobileNo>910994022310</SMSMobileNo>
</ErrorInfo>

TagName | Purpose
SMTPServer | Stores the SMTP server name
EmailFrom | Sender's e-mail address
EmailTo | Receiver's e-mail address. We can specify any number of e-mail IDs by giving a semi-colon as a separator
EmailSubject | Subject of the e-mail
ConnectionString | Database connection string
DatabaseType | DataBase type can be either SQLServer or "others"
FileNotify | Preference to log the error information into a text file. The value is either True or False.
EmailNotify | Preference to send the error iInformation as an e-mail. The value is either True or False.
LogNotify | Preference to log the error information into the database. The value is either True or False.
SMSNotify | Preference to send the error information as an SMS. The value is either True or False.
SMSSenderName | SMS sender name
SMSMobileNo | Mobile number to send SMS

We place this configuration file in the root directory of the web application. The next step is to import the ErrorLog.WebException namespace into the Global.asax file.

using System;
using System.Collections;
using System.ComponentModel;
using System.Web;
using System.Web.SessionState;
using ErrorLog.WebException;

After importing the namespace we initialize the ErrorHandler class and call the method HandleException() from the Application_onError event in the Global.asax file. This event triggers whenever an unhandled exception is thrown.

protected void Application_Error(Object
            sender, EventArgs e)
{
// Fires when an error occurs
//
            Initializating the ErrorHandler class
ErrorHandler eh= new
            ErrorHandler();
eh.HandleException();
}

We can directly call the HandleException() overloaded method in any try catch block within a procedure.

try
{
string strConfigFile=
            HttpContext.Current.Server.MapPath
("./ErrorSetting.Config");
XmlDocument xdConfig = new
            XmlDocument();
XmlNodeList xnlList =null;
xdConfig.Load
            (strConfigFile);
}
catch(XmlException
            xEx)
{
ErrorHandler eh = new
            ErrorHandler();
eh.HandleException(xEx);
}

Testing the Error Reporting System

To test this system we can write some simple code that throws a "divide by zero" exception:

private void Page_Load(object sender,
            System.EventArgs e)
{
int i=0;
int j=1;
int
            k=j/i;
}

Since this code is unhandled, the Application_Error event in the Global.asax file is triggered and the event calls the method HandleException in the ErrorHandler class. This HandleException method logs the error and redirects it to an error page.

The screenshot below shows the error information logged to a text file:

The screenshot below shows the error page:

Live Demo

A live demo of the error logging and reporting system is available at the DotNetForce site. You can access this by clicking the link http://www.dotnetforce.com/CsharpCS.aspx?WROXEMPTOKEN=618658Zezi9SpDFSZgQps5Xv6u.

Summary

In this case study we have developed a real time error logging and reporting system. We have developed three components in this case study: the first of which helps to alert the system administrator by sending the error information by SMS or e-mail; the second component logs the errors into a database so we can analyze it at a later date; and the third one integrates the other two components. This system has been tested on a few live web sites like http://www.dotnetforce.com/?WROXEMPTOKEN=618658Zezi9SpDFSZgQps5Xv6u, and I would like to mention that this system helps in checking out the errors that occur on that site as and when they happen. Integration of this system is simple and straightforward, so I hope this will be a useful system to you.


Intranet Based Project Management Suite

Intranet Based Project Management Suite
http://www.csharptoday.com/content.asp?id=1933
Article Rating: 4
Published on October 14, 2002
Abstract
This case study looks at the creation of an intranet based project management suite. Hammad Rajjoub worked as part of a team who created this product in respose to a company's need for a system to improve team efficiency, improve communictation, and to reduce the implementation time of their solutions, and he uses this case study to show how this project was completed successfully. This case study demonstrates database programming using .NET, plus several different techniques like file uploading, emails, using assemblies, and effective usage of server side controls.
Case Study

Recently, Global Securities (a leading security firm in Pakistan) identified a need to increase their team efficiency, improve communications, and reduce the implementation time of their solutions. To achieve this, they knew that a collaborative project management solution was necessary. They turned to Creative Chaos (a web enabling firm, where I recently worked) to provide them with a cost effective customized project collaboration solution. This case study demonstrates the solution that was created, an intranet based project management suite.

We completed the entire solution within three months, and deployed it successfully on their intranet.

The complete product was created using Microsoft .NET technology. On the technical side, this case study demonstrates database programming using .NET, plus several different techniques like file uploading, emails, using assemblies, and effective usage of server side controls.

The project management suite consists of five different modules:

  • Users: This module will handle all the tasks related to users, such as user management, user invitation, user addition, etc.
  • Messages: This module will be responsible for the provision of the user's messaging needs. The functionality of this module will include sending message, receiving message, deleting message, etc.
  • Tasks: This module will provide project managers a tool for assigning tasks to other team members.
  • Documents: Project management also means document management. The documents module will help users of the application to upload/download documents to/from the central repository.
  • Calendar: This module will actually support the task and document modules. We use the calendar module to provide a time-wise view of tasks.

The Project module will also come under discussion. This module will deal with project addition, editing and deletion.

This project management suite can handle unlimited projects with unlimited users working in each project. Microsoft SQL Server 7.0 is used as a database server during this case study. C# is used as the main development language in conjunction with ASP.NET (to provide a browser based interface).

This case study will be beneficial to those who want to gain an insight into C#, ADO.NET and ASP.NET, and want to develop their database applications for a browser based environment.

In this case study we will be learning how to create web-based applications using C# and ASP.NET. We will be discussing, in detail, how ADO.NET comes into play in .NETs database development. We will also be discussing some of the design issues that came up during this project.

What we will be doing in this case-study:

In this case study we will be looking at:

  1. Database design and development.
  2. Database accessibility through ADO.NET
  3. Creating C# classes
  4. Making C# Assemblies
  5. ASP.NET based web development
  6. Consuming C# assemblies from ASP.NET applications

Figure 1. Structure of the intranet based project management suite

When we talk about project management, we are talking about messaging, task assignment, user management, document repository, etc. All this information needs to be stored somewhere, so we will take a look at database design. We will look at the data dictionary, E-R diagrams, and also some of the design decisions that were made. After going through the database design section, you will be able to develop this type of application from the scratch.

Accessing the database with ease, efficiency, and security is very important for any application. In our project management suite, we will see that how ADO.NET can be used in the .NET environment to store, retrieve, and modify data.

We will be writing classes in C# that will provide access to the database through ADO.NET. These C# classes will not only provide access to the database through ADO.NET, but also implement the application logic as well. These classes will be converted to C# assembles which will in turn act as components for the front end (i.e. ASP.NET).

This project management suite is a browser-based application. I have used ASP.NET exclusively for the front-end development. We will see how to create ASP.NET pages that can use C# classes/components. We will also be discussing two ways by which an ASP.NET page can imply code and content separation - Code Inline Method and Code Behind Method.

The Project

As we saw earlier, this project management suite consists of five different modules: users, messages, documents, tasks and calendar. There is also another module called Project that contains the other five modules.

Figure 2: Intranet Based Project Management Suite - five modules

Details of Individual Modules

The User Module

The project management suite is designed to address the users' project management needs. Whenever we talk about project management, we are talking about resource management. In fact, team members for a given project are the most important resource. This Users module is designed to help organize the team members to make use of them in the most efficient manner possible. The module will provide the following functionality:

Adding User: we add users by specifying their details explicitly. Adding a new user means that the person that has been added is now the part of the project team. While adding the user, his role can also be specified.

Inviting User: The other way to add users to a project team is by inviting him/her. Using the project management suite, users can be invited by sending them an invitation mail. This email contains an invitation message and a follow up URL. All the user needs to do is to specify the email address of the person to be added. Once the email address is provided, the invitation mail is sent. In response, if the invitee wants to accept the invitation then he can click on the link that will lead to a form where he can fill in his personal details. If the user rejects the invitation, then he need not reply to the mail.

Editing User: Once the user is added to the project, there is always a possibility that one will need to edit the user details, for example if the details given originally become out of date, or the project manager wishes to change an employee's role.

Deleting User: Occasionally we may need to delete users, such as when the project ends.

Pending User: Pending users are those who have been invited, but haven't responded yet. Project mangers might delete pending users if they don't respond within a given time.

The Message Module

Communication plays an important role in any organization, but its importance is even greater in software development. The communication from project manager to team leaders, and from team leaders to developers is a very important part of the development environment. This is exactly the idea behind this module - it will cater for the communication needs of the project management team. The main features of this module are:

Private Messaging: Allows the user to send a private mail to a team or individual.

Public Messaging: The user can send a message that if declared public, is visible to all his team mates.

Message Notification: Whenever a new message is received, the user is notified.

The Documents Module

As the name specifies, this module will be the central repository of all the documents that need to be shared at any level in the project development phase. Using the module the user will be able to create new folders, which can be hierarchical so that they can be nested. Folders can also have their permissions set to either "public" or "restricted".

The Tasks Module

In project development, without settings goals and milestones, it is virtually impossible to get jobs done on time. Tasks are for this purpose - project manager can set tasks for team leaders, and team leaders can set tasks for team members.

Whenever a user logs on, he will be shown the status of his current responsibilities. Tasks are added using information like:

  • Name
  • Assignee
  • Deadline
  • Description
  • Notification
  • Restriction (public/private)

The functionality the tasks module provides is:

Adding Tasks: Tasks can be added by project managers. For example, the project manager can add the task for his team that the design and analysis phase of the current project should be over by 16th September 2002.

Editing Tasks: Once the task is assigned there is often a need to edit it. For example a project manager might extend the deadline for the project completion.

Deleting Tasks: A project manager might also be interested in deleting a task, for example when the task is completed.

Summary

Figure 3: How the module fits together.

In short, the intranet based project management suite is based on easing communication needs and increasing the efficiency of project development by providing a true collaborative environment.

Technology

Before we get into actually developing the module and merging them together, we will take a look of the technology that we will be using in this application.

Database Accessibility Through ADO.NET

When you are programming in .NET and interested in database accessibility, then ADO.NET is the obvious choice. This is because ADO.NET is very much powerful and provides functionality and ease of use. The intranet based project management suite does not utilize all the main features of ADO.NET, simply because it doesn't need to, but does use some of them.

In the old days (not so old actually, just older!) of windows programming, one could connect to a database using OLE DB or through ADO. In .NET's managed environment things have changed. Here we talk about managed code, and thus the connection provider needs to comply with managed code. As a result, we currently have quite a few managed data providers for the .NET Framework. They are:

  1. System.Data.SqlClient
  2. System.Data.OracleClient
  3. System.Data.OleDb.OleDbConnection

In this case study we have used SQL Server, and so System.Data.SqlClient is the namespace that we will be using throughout this case study. I will be discussing the System.Data.SqlClient namespace in detail later in this section. However, before we go any further in our discussion of database connectivity and SqlClient details, I think we should take a short tour of ADO.NET.

ADO.NET

ADO.NET is a strategic application level interface to data access in the Microsoft Platform. ADO.NET provides improved and optimized access for SQL Server, and an OleDb connection can be used whenever needed (provided the database is not Sql Server). Recently, Oracle launched its managed connection for .NET, and in the months (or maybe years) to come we might see some more managed and optimized connections for databases other than SQL Server. ADO.NET is not a wrapper class on ADO - it is much more powerful than its predecessor, thanks to the concepts of disconnected databases (in the form of datasets).

Some major features of ADO.NET are:

  • Connectivity to a database
  • Transactions on a database
  • Writing commands to be executed on a database and then executing them
  • Accessing disconnected data sets as off-line containers of data. Data sets are unique and very powerful in the sense that they can contain data from multiple data tables and we can also define relationships between those data tables
  • Creating procedures for a database
  • Using and creating data relationships
  • Implementing data constraints
  • Generating and fetching data from XML Schema. This is another very strong feature of ADO.NET, and one which differentiates it from ADO
  • Populating data using data adapters

Let's now expand on some of those points:

Connectivity

A data provider in .NET is a bridge between a front end (application, web interface, etc) and a back end (the target data base). In .NET Enterprise release version 1.0 we had two data providers, SQL Server Data Provider and OLE DB Data Provider, but recently some more Data Providers have been introduced, including Oracle. The data providers in .NET are shown in the table below:

Data Provider | Description
SQL Server .NET Data Provider | For Microsoft® SQL ServerT version 7.0 or later.
OLE DB .NET Data Provider | For data sources exposed using OLE DB.
Oracle Client | For Oracle® 9i and later.

Connection String

Although we will only be working on SqlClient, I would still like us to take a quick look at the other client (i.e. OleDbClient) as well for completeness. Now, let's see how to build a connection string for SQL Client and OleDbClient, both for local and remote machines:

SQL Version

Below is a code snippet that provides a way to create a query string:

//for local host using Windows NT
            Authentication
//for SQL server you don't need to provide :
            Database providers attribute (usr
//id and pwd are required if
            SQL server
//authentication mode is used to access
//database.)

string m_strSource = "Data
            Source=localhost;Integrated Security=SSPI;Initial
            Catalog=northwind";

// for remote host connection string
            should be like this:
//string m_strSource="user id=sa;initial
            catalog=check;data
//source=SqlServerName;";

SqlConnection m_oConn = new
            SqlConnection(m_strSource);
m_oConn.Open(); //Connecting to the
            data b

OleDb Version

Now we will see how to create an OleDb query string:

//for local host using Windows NT
            Authentication
//for SQL server you don't need to provide :
            Database providers attribute
//(usr id and pwd are required if
            SQL server authentication mode is used to
//access
            database).

string m_strSource="Provider=SQLOLEDB;Data
            Source=localhost;Integrated Security=SSPI;Initial
            Catalog=northwind";

// or connection string could like this
            for Access Connection
//string strSource =
            "Provider=Microsoft.Jet.OLEDB.4.0;Data
//Source=BugTypes.MDB";

OleDbConnection m_oConn = new
            OleDbConnection(strSource);
m_oConn.Open(); //Connecting to the
            data base

Transactions

There will be times in your database development using C# when you will be interested in running consecutive queries in a specified sequence. That's when you'd be interested in using transactions.

A transaction in ADO.NET begins by calling BeginTransaction () :

string m_strSource = "Data
            Source=localhost;Integrated Security=SSPI;Initial
            Catalog=northwind";
SqlConnection m_oConn = new
            SqlConnection(m_strSource);
m_oConn.Open();//Open connecion
//Create a Transaction - begin transaction
SqlTransaction
            m_oTxn = m_oConn.BeginTransaction();

//Execute some updats
            here..
......//some more
            here.

m_oTxn.Commit();
m_oConn.Close();

Commands

Whenever you need to get some SQL queries running for a specified connection, commands can be used. Commands are SQL statements with a reference to a valid open connection. For an SQL connection you use SqlCommand object, and for an OleDB connection you would use OleDbCommand object.

You can also specify the command type of a SqlCommand object by simply setting its CommandType property. The different command types could be plain SQL commands, stored procedures, etc.

Commands are executed by simply calling one of execute methods available to connection objects:

  • ExecuteScalar: Executes the command and returns a single result.
  • ExecuteReader: Executes the command and returns a reader object of type IDataReader
  • ExecuteNonQuery: Execute the command and returns the number of rows effected
  • ExecuteResultSet: The implementation details of this method are not available at the time of writing; they are reserved for future use.
  • ExecuteXMLReader: Executes the command and returns a reader of type XMLReader

For this project, we have extensively used SqlReaders whenever the result of a query was to be retuned to the front-end, and just used the plain ExecuteNonQuery whenever we were interested in finding out the number of resulting rows as in the case of user validation (discussed in the next section). The following are excerpts from our application showing where some of the methods we have discussed are used:

ExecuteScalar

Execute Scalar is used whenever a SQL query is supposed to return a single value.

// id is a local variable within the
            scope
string strQ= "Select count(*) from Users where
            usr_LoginId='"+id+"'";

//conn is an existing Connection
            Object that is live
comm = new
            SqlCommand(strQ,con);
//Executing ExecuteScalar() that returns
            result of type
//Object.
int i =
            int.Parse(comm.ExecuteScalar().ToString());

ExecuteNonQuery

ExecuteNonQuery is used whenever a Sql command is supposed to do some data manipulation. Examples include inserting, updating, or deleting data from the database.

//Adding some user in table
            Users
strQ="insert into Project_User_Group values
            ("+prjId+","+grpId+",'"+id+"',"+isPrjMgr+")";

//conn is an
            existing Connection Object that is live
comm = new
            SqlCommand(str,con);
//Execute Query doesn't return any
            thing.
comm.ExecuteNonQuery();

ExecuteReader

ExecuteReader is very important as it returns rows of the database based on the command criteria.

// Selecting a particular user on the basis
            of his groups
//and userId
String strQuery = "select
            .usr_FirstName, .usr_LastName, .usr_Description from Users
            a,Project_User_Group b
where a.usr_LoginId = b.usr_LoginId and
            b.prj_Id ="+prjid;

//conn is an existing Connection Object
            that is live
this.comm = new
            SqlCommand(strQuery,con);
//Execute reader returns a
            SqlDataReader object
SqlDataReader rdr
            =comm.ExecuteReader();
//now with this reader we can navigate
            through the fetched data
while(rdr.read())
{
// do some
            thing with data
// . like display data etc.
}

Now that you have seen the basics of ADO.NET, it's time to move onto the implementation stage!

Implementation

Here we will implement a class that will be used as a utility to provide open and live database connection.

This class will include the following namespaces for implementation purposes:

using System;
using System.Data;
using System.Data.SqlClient;

The database connection class implements a default as well as a parameterized constructor. In this constructor we set the connection string and create an SqlConnection object.

l_source="user id=sa;initial
            catalog=check;data source=Server;";
m_con = new
            SqlConnection(l_source);

This class also provides methods with which the connection can be opened and closed. Whenever the request is made to get a connection (via the getCon() method), the state of the connection object is checked (through the openCon() method) - if the connection is not closed then the connection is opened and is returned to the requesting object.

if (opencon()== true) return m_con;

This brief introduction to database connectivity and ADO.NET architecture should have provided you with enough information to get you started working on ADO.NET using C#.

C# Assemblies - Component Based Development

This project management suite project is actually a three tier application. At the back end is SQL Server, at the front are ASP.NET pages with C#, and at the middle tier I have created C# classes to act as a bridge between the front and back-end while implementing business rules. These middle tier classes are developed as assemblies, which are .NET's way to break free from DLL Hell.

I won't be discussing the way that I have implemented these components, rather I would be giving a brief into to assemblies, similar to the way that we discussed ADO.NET in the previous section.

Assemblies

Assemblies are the basic unit of programming in NET Framework. An assembly is a collection of files that appears to the user as a single dynamic link library (DLL) or executable (EXE). DLLs are collections of classes and methods that are linked into your program at run time, only when they are needed.

Assemblies are a fundamental part of programming with .NET Framework and give us a break from the tedious traditional way of looking at re-usability, e.g. normal DLL's exposing global methods, COM DLLs exposing COM classes, etc. All this was too much for developers to tackle, and even for Microsoft, who introduced the term "DLL Hell" to represent the troubles that a programmer had to face in the past.

In our intranet based project management suite we have developed components in the form of assemblies, as assemblies are easier to design, build, and deploy. In this case study we have created modules as DLLs and they are the constituent piece of the assemblies. As mentioned earlier, assemblies are loaded on demand and will not be loaded unless needed.

Firstly I would like to mention some of the strong points of assemblies in the .NET Framework, and then I will proceed to look at the way in which assemblies can be created and deployed. I will be giving an example of a module that can be converted into an assembly. First let us look at the strong points of assemblies:

  • Metadata: Assemblies are self describing because metadata fully describes the contents of each module. Metadata plays an important role in the usability of .NET assemblies, and they can potentially save programmers time and effort by eliminating some concerns from the programmer's task list - registry keys for example. The metadata include types exported from the assembly and a manifest.
  • Security Boundary: Assemblies form security as well as type boundaries. An assembly is the unit at which permissions are requested and granted.
  • Versioning: Assemblies do form a version boundary as well. Each assembly has a version number, it is in fact the smallest version-able unit, and all the types in the assembly are given the same version number. So, a version number can only refer to the contents of a single assembly. The assembly's manifest describes the version dependencies you specify for any dependent assemblies
  • Manifest: Each and every assembly, whether it is static or dynamic, has a manifest as a part of metadata. This metadata contains all the information about that assembly. It also contains a collection of data that describes how the elements in the assembly relate to each other. The assembly manifest can be stored in either a portable executable (PE) file (an .exe or .dll) with Microsoft intermediate language (MSIL) code, or in a standalone PE file that contains only assembly manifest information.

Making Assemblies

There are two ways to create assemblies:

  1. VS.NET way.
  2. Command Line

VS.NET way

Creating assemblies using VS.NET is very easy. All you need to do is to create a new project of type "Class Library"! Below is the screen shot of VS.NET:

In VS.NET, if the type of project to be created is set to Class Library then VS.NET creates an assembly for you with the name set to that of your project.

Using Command Prompt

You can convert your CS file to an Assembly (DLL) by using following C# compiler option:

C:\WINDOWS\Microsoft.NET\Framework\v1.0.3705>csc /t:library c:\prj\Users.cs

The compiler will show warnings or errors if there are any, else a successful dll (assembly) will be generated in the C:\WINDOWS\Microsoft.NET\Framework\v1.0.3705\ directory.

In the project management suite application we have used the command prompt way of creating assemblies. There will be more discussion on this when we move on to creating the modules of this project.

Consuming C# assemblies from the ASP.NET application

We will be developing the front-end of our application using ASP.NET, and we will need to access C# assemblies from here.

ASP.NET is a powerful language available for the .NET framework. It is much more powerful than its predecessor, ASP 3.0. It is an event driven and compiled language. In previous versions of ASP there were less, or no means at all, of separating code from content in a given .asp page. The .NET framework now provides two ways to engineer a separation between code and content:

  • Code Inline
  • Code Behind

Code Inline

The code inline technique separates code from the content; although they remain on the same physical file, there are logical boundaries. Code is places in the code section and content is present in the content section. Coding can be done in either one of the two .NET languages.

Code Behind

In the code behind technique, code is physically separated from content - so the code is placed in a completely different file to the content. This implies a clear separation between code and content. This is exactly the way with which you do ASP.NET programming in Visual Studio.NET.

In this case study we will be using Code Inline technique, which is simpler to use. Another reason for using code inline is that it does not require Visual Studo.NET. We have used Web Matrix for ASP.NET development. Web Matrix is ASP.NET development tool available for free at http://www.asp.net/?WROXEMPTOKEN=618658Zezi9SpDFSZgQps5Xv6u.

Using Assemblies from ASP.NET pages

If someone is interested in accessing assemblies from ASP.NET pages, the code inline technique makes this very easy. ASP.NET provides a unique way to access assemblies, by simply importing their namespaces and placing all the required assemblies in the /bin directory, i.e. where your ASP.NET pages are kept. That's literally all you have to do to call your assembly using ASP.NET. The format is <%@ import Namespace ="Name Of Name Space" %>.

The Solution

We will build this application in five different modules, and then we will combine them to make a fully functional application.

User Module

Database Design

We will be creating a user table. The first step is to identify all the attributes of the user entity, which are shown in the table below:

Entity Attribute | Description
Usr_LoginID | It's the primary key used to uniquely identify each user.
Usr_FirstName | First name of user
Usr_LastName | Last name of user.
Usr_Description | Additional details about the user.
Usr_Address | Home/Office Address of user.
Usr_Email | Email Address of user.
Usr_PhoneNumber | Home/Office Phone Number of user.
Usr_Password | Secret password used during authentication.

Dependencies

This user entity is very important, and thus is linked to a number of other entities like projects, tasks, messages, and documents.

Entity Dependency | Description
Project_User_Group | Used to relate user with project and group.
Message_User | Used to show the users who will receive a particular message.
Document_User | Documents are only viewable to users related here.
Task_User | Used to show the users who will be assigned this task.
Project | Linked through prj_creator showing the creator of the project.
Message | Linked through msg_postedby showing the creator of the message.
Task | Linked through tsk_AssignedBy showing the assignee of the task
Document | Linked through doc_CreatedBy showing the owner of the Document.
Folder | Linked through fld_CreatedBy showing the owner of the Folder.

Class Diagram

In the following class diagram, you will notice that I have used two classes for a single User entity. Although I could have used a single class for the single User entity, I have chosen to use two because it is better to have a separate class that represents an attribute level get/set abstraction over a database entity, and a separate class for functionality provision on the same entity. Although this approach might ask you to create extra classes, it is really a better approach in terms of functionality. This is because at the interface (GUI) level, we are satisfied with get/set abstraction of a database entity i.e. when either reading/writing values from/to interface (GUI). At the component level (middle tier) we will be interested in applying functionality over those collected get/set abstractions of the database entity.

So in the user module, the User class is used as the get/set abstraction for the database user entity and the Users class is used as the class that provides functionality over User type objects. This is clearly shown in the diagram below:

Class Functionality

Below is the class diagram for the User module. This module, as described in the earlier section, will provide the utilities for user management.

Use Case for Users Module

Implementation

We will now take a look at the functionality that the user module provides. This is the functionality described in the Users class above. We will see more of this when we will build the application in the last section of this case study. Below are some of the important functions of the Users class.

doVerifyuser: Verifies whether the user mentioned exists, and if their password is correct or not. If all conditions are met it returns true, otherwise it returns false. User verification is very important, and key to the security of the application. Whenever users log into the application he is asked to provide a valid user ID and password. If he is a new user than he can sign up, although if he is an existing user then he must supply a valid User ID and password.

This method takes the user ID and password as input parameters. The third parameter that this function takes is actually a variable that specifies the state of query execution. This state is actually defined as an enum in the namespace projectuser.

Enum were used in this application as a sort of flag, that will tell us about the state of the previous task execution. This value will always be passed to a method that queries the database (whether it be creation, updating, or even deletion of tables) and will always be passed out as public enum ECODE {NOERROR,NULLRET,OTHER}.

The query used for user verification is:

string strQ="select count(*) from users
            where usr_LoginId ='" +uid+ "' and
            usr_Password='"+pwd+"'";

When the query executes, if execute scalar returns one then it means that there is exactly one record with the specified attributes (user id and password).

if(int.Parse(comm.ExecuteScalar().ToString())==1)
{
e=0;
return
            true;
}

Here, e= 0 implies that the query has been executed smoothly without any errors.

For a commercial application, we would not store the user id and password as a plain text, since there would always be a security risk. There will be a performance overhead for applying encryption/ decryption algorithms to the user ID and password. For this case study, we have used plain text storage for the sake of simplicity

LoadAllUsersByProjectId: This method returns a record set containing all the users who are working on the project specified by the Project ID:

string strQuery = "select a.usr_LoginID,a.usr_FirstName,a.usr_LastName,
a.usr_Description from Users a,Project_User_Group b where a.usr_LoginId = b.usr_LoginId and b.prj_Id
="+prjid;

Now after creating this query text we can use execute reader:

this.comm = new
            SqlCommand(strQuery,con);
rdr=comm.ExecuteReader();
e=0;

After executing this reader, it is returned to the callee:

return rdr;

LoadUsersById: This returns the information for the user (as a record set) whose user id matches that of the parameter, and who is working on the project represented by the project ID passed in as a parameter. This is the query for loading user record:

string str ="select * from Users where usr_LoginId='"+id+"' ";

The command is executed and a reader is returned. Also, the enum variable is set so that successful execution is implied:

rdr = comm.ExecuteReader();
e=0;
return (rdr);

AddUser: This method adds a new user into the database, but only if a user with same user ID doesn't already exists in the database. It returns true or false depending on the success of the method. This method is called when a user signs up on invitation. This is the query for insertion of record into the database:

string str = "insert into Users
            (usr_LoginID,usr_Password,usr_FirstName,usr_LastName,usr_Description,usr_Address,usr_Email,usr_PhoneNumber)";
str+="values('"+id+"','"+pwd+"','"+fname+"','"+lname+"','"+desc+"','"+address+"','"+email+"','"+phone+"')";
comm
            = new SqlCommand(str,m_oCon);
comm.ExecuteNonQuery();

Once the record has been added, the addition must be reflected in the tables where user dependency exists. One such dependency exists in the Project_Group_User table. The query is:

strQ="insert into Project_User_Group values ("+grpId+","+prjId+",'"+id+"',"+isPrjMgr+")";

Now after setting the command text to the query above, we can execute the Sql command:

comm.ExecuteNonQuery();

EditUser: As we have already discussed, once the user has been added, the record might need updating. EditUser is a method in the Users class that provides this functionality. Below is the query that is used in the EditUser method of the Users class:

string str = "Update Users set
            usr_Password='"+pwd+"' , usr_FirstName='"+fname+"',
            usr_LastName='"+lname+"',";
str+="usr_Description='"+desc+"',usr_Address='"+address+"',usr_Email='"+email+"',usr_PhoneNumber='"+phone+"'
            ";
str+=" where (usr_loginId='"+id+"')";

Once the user record has been updated, all the dependencies might also need to be updated. The following query updates the Project_User_Group table, just as we saw with the AddUser method earlier:

str="update project_user_group set
            prj_isManager="+isman+" where usr_loginId='"+id+"' and
            prj_id="+pid;

For both of the above sql queries we have ExecuteNonQuery.

isManager: This method verifies whether the user is project manager in the project specified. The query for this task is:

"select count(*) from project_user_group where prj_id= " + p_id + "
and lower (usr_loginid) = '" + userid.ToLower() + "' and prj_ismanager = 1"

Then we check whether a person exists with this id, who is also the project manager. We have used the ExecuteScalar method to find this out:

if (int.Parse(Com.ExecuteScalar().ToString())>0)
{
return true;
}

If ExecuteScalar returns a value greater than 0 than we return true to the callee function, otherwise false is returned.

inviteUser: This method is called when a project manager invites a user to join the project team. An email will be sent to that person using the mail server passed as parameter, and the user will be added to the pending table, where he will remain until either that user signs up or the project manager deletes him.

string str = "insert into pending
            values("+pid+",'"+email+"')";
comm = new
            SqlCommand(str,con);
comm.ExecuteNonQuery();

To send email we will use the Smtp.Mail class. Here we have created the message body first, and then we have assigned an SmptServer object to the mail so that the mail could be sent using that smtp server.

SmtpMail.SmtpServer = mailserver;
SmtpMail.Send(msg);

Here msg is an object of type MailMessage.

UI Design and Coding

Design Considerations

The UI design for this application is simple and can be easily understood. Since the application is browser-based, ASP.NET is used for the front-end. With ASP.NET, interfaces are developed (screen shots are presented in the following section) and every interface uses some component logic at the back-end. To support this back-end logic, C# components are written.

In this project, ASP.NET has been used solely for front-end development, and we haven't tried anything fancy (although ASP.NET is itself very powerful).

Key to the design of the UI for this application is the use of assemblies that are in turn C# based components. These provide easy XCOPY installation, and a programmer doesn't have to worry much about version control, and registering the components repeatedly for each modification and recompilation made. So, all you have to do is to press refresh on your browser after placing the modified assembly in the root directory, or at the specified path.

User Module

We have common code for all the interfaces in the User module. This code contains a way to include references (of assemblies/components) that the application needs:

<%@ Page Language="C#" Debug="true" %>
<%@ import Namespace="System.Data" %>
<%@ import Namespace="projectdata" %>
<%@ import Namespace="System.Data.SqlTypes" %>
<%@ import Namespace="System.Data.SqlClient" %>
<%@ import Namespace="Project.DataBases" %>
<%@ import Namespace="Project.User" %>
<%@ import Namespace="projectuser" %>
<script runat="server">

Since this application is component based, there has to be some way of accessing the component from the front end. ASP.NET provides a unique way to access assemblies, by simply importing their namespaces and placing all the components (in dll format) in the /bin directory. The format is <%@ import Namespace ="Name Of NameSpace" %>.

Login Screen

Details:

Input Field | Type | Validation
User ID: | Alphanumeric | Yes
Password: | Alphanumeric | Yes

Fields:

User ID: Specifies the ID assigned to the user.
Password: The password used for verification.

Description: The login screen is the first screen that the user gets to see. This process of user authentication is accomplished by posting this data inputted by the user to COM through Users.cs, which is the file where all the implementation about users is implemented. This queries the database and thus fulfills the criteria for a true 3-tier application. We will show all the code for this process later in this case study, but for the sake of making this explanation more understandable, some of the more important sections of code are as follows:

Onbtn_LoginClick does the main job this page:

if (usrs.doVerifyUser(UserName.Text,UserPass.Text,out e_)==true)
{
Session["id"]=UserName.Text;
Response.Redirect("mainpage.aspx");
}

This method creates an object of Users. If the user's name and password are valid then there is a call to a method to verify User. If the user is verified, doVerifyUser returns true and the value of the out variable is set to a non-error code. Otherwise, it returns false, and the value of the out variable e is set to an error code. If the result is true, then this page sets some session variables to their appropriate values, and the user is redirected to mainpage.aspx, otherwise the user stays at the same page. If the user opts to sign in, then they are redirected to the user sign up page.

Session Variables: We have used session variables instead of cookies to keep the user state information because session variables have a couple of advantages. Firstly, session variables are stored on the server which means that clients have no access to them, thus increasing security. Secondly, the lifetime of sessions are controlled by server configuration, which enable us to change their duration without changing the code and recompiling it. In cookies, the lifetime has to be hard-coded.

In the Page_Load event, we have set all the session variables to default values:

     void
            Page_load(Object s,EventArgs args)
{
con=new
            projectdata.datacon("
            ");
con.opencon();
Session["pid"]="";
Session["id"]="";
Session["flid"]="";
Session["did"]="";
Session["tid"]="";
Session["msg_id"]="";
}

User verification is done as soon as the user presses the login button. This is when the value from the control is passed to the doVerifyUser method, available in the Users assembly.

Sign Up Screen

Input Fields

Field Name | Type | Validation
User ID: | Alphanumeric | Yes
Password: | Alphanumeric | Yes
First Name: | Alphanumeric | No
Last Name: | Alphanumeric | No
Address: | Alphanumeric | No
Phone #: | Alphanumeric | No
E-mail Address: | Alphanumeric | No
Descriptions: | Alphanumeric | No

Details

User ID: A unique user ID assigned to the new user.
Password: Secret password for the verification.
First Name: first name of the new user
Last Name: Last name of the new user.
Address: Postal Address of the new user
Phone #: Phone number of the new user
E-Mail Address: E-mail address of the new user.
Descriptions: Description of the new user

Description: Users are asked to enter some details. On submission of these details, all the data is posted to a component assembly that inserts this data in database. There is an error if the Login ID chosen by the user already exists in the database, in which case the user is prompted to enter a different ID.

On page initialization, we set up a live and open connection. This utility comes from the database utility class:

void Page_init(Object s,EventArgs
            args)
{
con=new projectdata.datacon("
            ");
con.opencon();
}

If user is added successfully, then the value of the out variable o is set to 0, otherwise it is set to 1 and a message "User ID Already Exist" is displayed on the screen.

usrs.AddUser(this.txtFName.Text,this.txtLName.Text,this.txtDesc.Text,this.txtAddress.Text,this.txtPhone.Text,
this.txtPwd.Text,this.txtLoginId.Text,this.txtEmail.Text,out o);

Now we can check the value of o to decide the appropriate course of action.

User Management Screen

Options

Add User: Take the project manager to the page where he can add new users.

Pending User: The project manager can see the users who haven't responded to invitations.

Description: This page shows all the users that exist in the project, and the project manager can either view or delete these users from the project. Selecting to view pending members will take the user to a page where the list of users in the project that are yet to be included in the project is displayed. On page load, a database connection variable is set to an open and live connection so that tasks can be performed, and also some session variables are initialized to appropriate values.

        string
            proj_id=Session["pid"].ToString();
string
            uid=Session["id"].ToString();
con=new projectdata.datacon("
            ");
con.opencon();
SqlConnection sCon =
            con.getcon();

User Add Screen

Input Fields

Field Name | Type | Validation
User ID: | Alphanumeric | Yes
Password: | Alphanumeric | Yes
First Name: | Alphanumeric | No
Last Name: | Alphanumeric | No
Address: | Alphanumeric | No
Phone #: | Alphanumeric | No
E-mail Address: | Alphanumeric | No
Descriptions: | Alphanumeric | No

Details

Login Id: A unique user ID assigned to the new user.
Password: Secret password for the verification.
First Name: first name of the new user.
Last Name: Last name of the new user.
*Project Manager: Select if the user being added is a project manager
Address: Postal Address of the new user.
Phone #: Phone number of the new user.
E-mail: e-mail address of the new user.
Description: description of the new user.

Description: From the user management screen, the project manager can add a new user. Adding new users will bring up this screen, and from here the project manager will add the user by providing all the information related to the user. As in previous screens, the database connection is prepared and session variables are set at the page_init event.

User Edit Screen

Input Fields

Field Name | Type | Validation
Password: | Alphanumeric | Yes
First Name: | Alphanumeric | No
Last Name: | Alphanumeric | No
Address: | Alphanumeric | No
Phone #: | Alphanumeric | No
E-mail Address: | Alphanumeric | No
Descriptions: | Alphanumeric | No

Details

Password: Secret password for the verification.
First Name: first name of the new user.
Last Name: Last name of the new user.
Project Manager: Select if the user being added is a project manager
Address: Postal Address of the new user.
Phone #: Phone number of the new user.
E-mail: e-mail address of the new user.
Description: description of the existing user.

Description: In the user edit screen, the project manager can edit the details of the user. However, the project manager cannot change the Login Id of the user. If the project manager chooses to save the details then, after validation, the data is saved to the back-end database through the Users component. If the project manager chooses to cancel this, then he is redirected to the user management page.

What is important here is the way in which user details are given to the ASP.NET page in edit mode, and then when user presses "done", the new details are saved back to the database.

rdr = usrs.LoadUser(long.Parse(proj_id),uid,out b);

If the reader is not null then we can assign appropriate values to the controls fetched from the database.

while(rdr.Read())

When the user presses "done", the editUser method is called from Users component, and the response is redirected.

Response.Redirect("UserManagment.aspx");

User Profile Edit Screen

Input Fields

Field Name | Type | Validation
Password: | Alphanumeric | Yes
First Name: | Alphanumeric | No
Last Name: | Alphanumeric | No
Address: | Alphanumeric | No
Phone #: | Alphanumeric | No
E-mail Address: | Alphanumeric | No
Descriptions: | Alphanumeric | No

Details

Password: Secret password for the verification.
First Name: first name of the new user.
Last Name: Last name of the new user.
Project Manager: Select if the user being added is a project manager
Address: Postal Address of the new user.
Phone #: Phone number of the new user.
E-mail: e-mail address of the new user.
Description: Description of the user.

Description: If someone is interested in changing his profile then he can from this page, but he can never change his name.

          rdr
            =
            usrs.LoadUserById(uid);
while(rdr.Read())
{
txtFName.Text=rdr["usr_FirstName"].ToString();
txtLName.Text=rdr["usr_LastName"].ToString();
//so
            on and so forth for the rest if the fields

When the user comes to this page, his ID is retrieved from the session and then a User type object is created and all its fields are set according to the data present in the database. The setting of values is done by calling a method from the Users class, loadUserById(), providing it with the retrieved ID from the session. This method returns an object of type User, which is then used to set the controls on the screen.

If the user opts to save the details, then all this information is saved to the back-end via the component, by calling doEditUser() and providing appropriate parameters. If the user opts to cancel, then the response is redirected to the main page.

View Pending User Screen

Options

Delete: Select and delete any (or multiple) pending users.

Cancel: Go back to the User Management screen

Description: On this page, project managers can view pending users and can also delete any user if he wants. Pending users are those users that have been invited but haven't responded yet. On page initialization, all the pending users are loaded into the data reader that is in turn used to print the details of these pending users.

while(rdr.Read())
{
user.Add(rdr[1]);
}

Message Module

Database Design

Entity Attribute | Description
Msg_ID | It's the primary key used to uniquely identify each Message.
Msg_PostedBy | UserID of the person who created the message.
Msg_Subject | Subject of the message
Msg_Name | Title of the Message such as (imp, edu, etc.)
Msg_Description | Body of the Message
Msg_PostedOn | Date on which it was sent
Msg_NotifyAll | Set when message was sent to all users of the project
Msg_NotifyAll | Set when the creator also receives the message

Dependencies

Entity Dependency | Description
Message_User | Used to show the relationship of the message with project and users

Class Diagram

As we saw in the Users module, we have two classes, one for get/set abstraction and the other one for functionality purpose. But here in this module we have used only a single class to represent the project entity. This class is basically an implementation class and provides functionality over the project entity.

Class Functionality

Users can create, send, view, or delete messages. This is shown in the following diagram.

Use Case for Message module.

Implementation

CreateMessage: Creating a message means adding a message into the database. Whenever the user creates a message and sends it to the specified users, an entry specific to that message is made in a message table. Below is the query that the CreateMessage() method executes:

string strSql = "INSERT INTO Message (msg_Id,prj_id,msg_PostedBy, msg_Name, msg_Description, msg_Subject,msg_PostedOn, msg_NotifyAll,msg_NotifyMe)";
strSql+="VALUES( " + i + ","+pid+",'"+user+"','" + name +"','" + desc + "','" + subject + "','"+postedon+"'," +notme+","+notall+" )";

The variable "i" is used to assign an ID to a specific message record. In fact, this message ID is also a primary key for this table. This variable i is generated on the fly, in the following manner:

string strQ= "Select max(msg_Id) from
            Message";
comm = new
            SqlCommand(strQ,m_oCon);
i=int.Parse(comm.ExecuteScalar().ToString());

We then check if i > 0, and then we move onto adding the message record to the database.

SendMessage: This method uses the project ID to make sure that only members of that project can read the message. The query string for this task is as follows:

strQuery = "insert into message_user (msg_Id,prj_Id, usr_LoginId, msg_isviewdby) values
(" + mid + "," + pid + ",'" + user[cnt] + "', 1)"

deleteMessage: If the user wants, then he can delete the message. Deleting a message does not mean that the message will be deleted from the Message table of the database. In fact, only a record from the Message_User table will be deleted, so that the message remains available to the rest of the users.

strQuery = "delete from message_user where
            prj_Id = "+ p_id +" and
msg_id = "+ mid +" and usr_loginid = '"
            + user +"'"

UI Design and Coding

Main Message Screen

Options

Create Message: Go to the screen where a new message can be created.

View Message: Select any message and view it on a Message View page

Description: On this page a user can view all the messages that he has received for this project. New messages are highlighted, while the old ones remain plain. The user can also create new messages and then send them to selected users. As mentioned in the design specifications, every UI depends on some relevant assemblies, and needs to access those assemblies by referring to them. We can include their namespaces in an ASP.NET page as follows:

<%@ Page Language="C#" Debug="true" %>
<%@ import Namespace="projectdata" %>
<%@ import Namespace="projectmessage" %>
<%@ import Namespace="System.Data" %>
<%@ import Namespace="System.Data.SqlClient" %>

Here we have created a class called msgret, which acts as a get/set abstraction of the database entity Message. We could have created a separate assembly, but since the Message entity only has a few attributes, it didn't actually required a separate class - and hence a UI level class is used for the purpose of Message entity abstraction.

With the help of user ID and project ID we have called a method from the Messages class, View_short, that returns message type objects. We have filled an Array List with those objects, which will be used to display messages on the page.

If the user opts to view all the messages by clicking the "view all" link, all the messages are loaded:

SqlDataReader rdr=obj2.View_all(Session["id"].ToString(),long.Parse(Session["pid"].ToString()),out b);
if(b==0)// no error occurred in executing specified query

Message View Screen

Options

Delete: Delete the message being viewed.

Cancel: Select cancel and go back to the main message screen.

On this screen, the user can see the message that he selected from the main message screen. The viewed message contains the following fields:

  • Message Title: Title of the message.
  • Message Subject: Subject of the message.
  • Message Description: Message text.
  • Message Is Posted By: The user Id of the person who posted the message.
  • Message Is Posted On: The Date Time when the message was posted.

Description: The user here has got two options. He can either delete that message, or he can cancel and go back to the message view screen.

void LinkButton2_Click(Object sender, EventArgs e)
{
Response.Redirect("messagemain.aspx");
}

When the user opts to delete the message, the delete method is called from the Message class. After deletion, the user is redirected to the main message page:

    void
            LinkButton1_Click(Object sender, EventArgs e)
{
Messages obj=
            new Messages(con.getcon());
int
            b=0;
obj.Delete(Session["id"].ToString(),long.Parse(Session["pid"].ToString()),long.Parse(Session["mid"].ToString()),out
            b);
Response.Redirect("messagemain.aspx");
}

Add Messages

Input Fields

Field Name | Type | Validation
Message Title | Alphanumeric | No
Message Subject | Alphanumeric | No
Message Description | Alphanumeric | No
Notify Me | Check Box |
Notify All | Check Box |
Select User | User Id | Yes

Options

  • Send
  • Cancel

Description: This screen lets the user of this application create and send messages. By creating a new message, the user can actually specify the following attributes of the message:

  • Message Title: Title of the message goes here
  • Message Subject: Subject Of message.
  • Message Description: Message text goes here.
  • Notify Me: Select if you want to be notified
  • Notify All: Select this if message is public and is for everyone.
  • Select Users (Select different users working on the same project whom you want to send this message to)

All this functionality is implemented in an assembly by the name of Messages. Once the attributes are specified, then the user can just click on the send button to send the message. Or, he can click cancel to go back to previous screen.

When the user opts to create a message, he can create and send new messages to specified users or to everyone:

long
            newid=obj.createMessage(long.Parse(Session["pid"].ToString()),
Session["id"].ToString(),msg_name.Text,msg_sub.Text,msg_desc.Text,DateTime.Now.ToString(),n,a,out
            b);
if(a==1 )
{
string[] selusers=new
            string[arr.Count];
for(int
            i=0;i<arr.Count;i++)
selusers[i]=arr[i].ToString();
obj.sendMessage(newid,long.Parse(Session["pid"].ToString()),
selusers,out
            b);
}

Documents Module

In this application, folders are created in a virtual directory of the web server.

Database Design

Entity Attribute | Description
Doc_ID | It's the primary key used to uniquely identify each Document.
Doc_Name | Name of the Document (Same as File Name)
Doc_Type | Type of Document (File Extension)
Doc_CreatedOn | Date on which document was uploaded
Doc_IsPublic | Tells whether the document is viewable for all or not
Doc_CreatedBy | UserID of the person who uploaded the document

Dependencies

Entity Dependency | Description
Document_User | Used to show the relationship of the document with project, users and folders in which the document is placed.
Folder | All the documents are stored in folders. So each documents stores with its own folder id.

Class Diagram

The logic for this class implementation remains the same as for the message module.

Class Functionality

Since the documents module is responsible for both document management and also folder management, this class provides functionality both for documents and for folders. The following diagram provides a good idea of what sort of functionality this module provides.

Use Case diagram for the Message module.

Implementation

AddDocument: Users can add documents to the specified folder. These folders are created in the virtual directory of the web server. The query for inserting a record into the document table is simple. It is:

strQ="INSERT INTO Document (doc_Id,prj_id, fld_ID, doc_Name, doc_Type, doc_Description,
doc_CreatedOn, doc_isPublic, doc_CreatedBy)";

strQ+="VALUES ( "+i+","+pid+","+folderId+" , '"+docName+"', '"+docType+"', '"+docDescription+"','"+docCreatedOn+"', "+isPublic+", '"+uId+"')";

Every document is supposed to reside in some folder. That's why whenever we add a record for a document in the document table, we will be specifying the folder id as well.

DeleteDocument: Deleting a document is also very simple. We have already seen that the document entity is dependent on the document_user entity (which is a bridge between the document and user entity). Whenever we will be deleting a record from the document entity, we have to make sure that the dependencies reflect the deletion of the record as well. That's why deleting a document also implies the deletion of a specific record from the document_user table. The following query show how the document_user record is deleted, and only then is the document table altered.

///delete from document_user
strQuery
            ="delete document_user where doc_id="+d_id;
oCm = new
            SqlCommand(strQuery,m_oCon);
oCm.ExecuteNonQuery();

Once the specified record is deleted from document_user table, we can proceed to delete a record from the document table.

///delete from document
strQuery ="delete
            document where doc_id="+d_id;
oCm = new
            SqlCommand(strQuery,m_oCon);
oCm.ExecuteNonQuery();

DocSendTo: This sends the document to specified users. Here sending isn't meant literally, sending just means notifying other users about the addition of new documents. So, whenever a user logs in, he is notified. A document (notification) could be sent to multiple users at a time, and if so, then we will have to notify all of the selected users. This functionality is implemented using the following for loop, which iterates through all the users. The rest is a simple query:

for( int cnt=0; cnt<
            user.Length;cnt++)
{
string strQuery="insert into
            document_user(doc_Id,prj_Id,usr_LoginId,fld_id)
            values("+d_id+","+p_id+",'"+user[cnt]+"',"+f_id+")";
SqlCommand
            oCm=new
            SqlCommand(strQuery,m_oCon);
oCm.ExecuteNonQuery();
}

LoadSubFolder: Loading a subfolder if another important function, as since a folder can be nested one needs to know about folders within folders. The following query is used:

string strQuery ="Select fld_ID,fld_ID,fld_ID, fld_Name,fld_ID, fld_Decription,fld_ID,
fld_ChildOf,fld_createdby from folder where fld_id<>"+f_id+" and fld_ChildOf="+f_id;

SqlCommand oCm = new SqlCommand(strQuery,m_oCon);
SqlDataReader or =oCm.ExecuteReader();

Once the query is executed, we have got a data reader that can iterate through the results.

UI Design and Coding

Document Main

Options

New Document: Select this option and go to the screen where new documents can be added to the current folder.

New Folder: Create a new folder within the current folder.

Delete: Delete the selected folder/document.

Rename: Rename the selected document/folder.

Description: On this screen, the user can view the documents that are in the current folder. The user can select a document and can then view it on the Document View screen. The user can, on this screen, rename documents/folders and can also add new documents/folders by clicking "New Document" and "New Folder" respectively. Likewise, the user can also delete documents/folders by selecting appropriate links from this page. Once again, as in the previous section, there wasn't a big class required for the get/set abstraction of the database entity; we have created the docret class that provides this functionality.

On the page initialization event we have set some session variables and extracted some values from session:

     string
            proj_id=Session["pid"].ToString();
string
            uid=Session["id"].ToString();
curfld=Session["flid"].ToString();
Session["curflid"]=curfld;
string
            temvar=Request.Params["fl_id"];

We get a live and open connection:

     con=new
            projectdata.datacon(" ");
con.opencon();

We create a document type variable by providing it a live connection, and then creating a data reader that contains a subfolder within the current folder. If the loadsubfolder method of the document class executes without any error (i.e. enum value = 0) then a collection is filled with the extracted document from data reader:

     SqlDataReader
            rdr=obj2.loadsubfolder(long.Parse(fldid),out
            b);
if(b==0)
{
while(rdr.Read())
{
doc.Add(new
            docret(long.Parse(rdr[0].ToString()),rdr[3].ToString(),rdr[8].ToString(),"Dir",DateTime.Now,true));
}
rdr.Close();
}

After loading the folders, we now load the documents. This is done by providing parameters to the loadDocs method:

     rdr=obj2.loadDocs(uid,long.Parse(fldid),out
            b);
if(b==0)
{
while(rdr.Read())
{
doc.Add(new
            docret(long.Parse(rdr[0].ToString()),rdr[3].ToString(),rdr[8].ToString(),rdr[4].ToString(),
(DateTime)rdr[6],false));
}
rdr.Close();
}
pname.Text="Document
            Console";
}

If the current user is the creator of the current folder, then he can delete the document. For this purpose, we have created a method called isfoldercreator that returns true if the user is the creator of the current folder, otherwise it returns false. If it returns false, then all the links that allows editing and deletion of documents are set to be inactive so that the user cannot alter or delete the document or folder.

if(obj.isfoldercreator(long.Parse(Session["flid"].ToString()),Session["id"].ToString(),out
            b))
{
obj.DeleteFolder(long.Parse(Session["flid"].ToString()),long.Parse(Session["pid"].ToString()),
Session["id"].ToString(),out
            b);
if(b==0)
{
Response.Redirect("docmain.aspx");
}
else
{
error.Text="Cannot
            delete some other users
            folder";
}
}
else
{
error.Text="You are not the
            Folder Creator";
}
}

Similarly, editing of folder and document name is checked.

Document Add

Options

Document Name: The fully qualified name of the document, i.e. the name with the physical path of the document.

Browse: While adding documents, the user can browse his local hard disk so as to select the desired document.

Document Description: Describe the document being added

Document Type: Specify the document type.

Select Users: Select the users for whom this document should be available.

Description: The user can add new documents to the current folder by browsing the local hard drive and selecting a particular document. He then has to fill in details like description, type, and the users for whom this message will be available. After doing this, the user can now press the send button or he can press cancel and navigate back to Document Main screen. On page initialization, we perform a few tasks, such as extracting information from session variables:

             string
            proj_id=Session["pid"].ToString();
string
            uid=Session["id"].ToString();
string
            fldid=Session["flid"].ToString();
con=new projectdata.datacon("
            ");
con.opencon();
int b=0;

Now the users are loaded into a local collection. We load all the users from the users object and fill a local collection of type ArrayList:

           SqlDataReader
            sdr=obj.LoadAllUsers(out
            b);
while(sdr.Read())
{
arr.Add(sdr[0]);
}
sdr.Close();

When the user has entered all the relevant information and pressed the add button, then screen level validation is done. On successful validation, the AddDocument method is called from the class Document. The document is then sent to the specified users:

long
            newid=obj.AddDocument(long.Parse(Session["pid"].ToString()),temp,long.Parse(Session["flid"].ToString()),
type.Text,desc.Text,DateTime.Now.ToString(),1,Session["id"].ToString(),out
            b);
string
            spliter=Request.Form["sel"];
if(spliter!=null)
{
string[]
            selusers=spliter.Split(',');
obj.docSendTo(selusers,newid,long.Parse(Session["flid"].ToString()),
long.Parse(Session["pid"].ToString()),out
            b);
}

Document Edit

Options

Selected Users: Select/unselect the users for the selected document to assign them view rights for this document.

Description: The user (creator of the document) can actually specify which of the users will be allowed to view the document. He can add some new users to assign them the view rights for this document or can deselect the previously selected users so that the current documents wont be available for them to view.

void Page_init(Object S, EventArgs args)

We have used the event above to perform all the initialization operations, because page_init is the first method to get called. We need to check whether the current user is the creator of the document or not:

document obj= new
            document(con.getcon());
if(obj.isdoccreator(long.Parse(doc_id),uid,out
            b))
{
SqlDataReader
            sdr=obj.View_user(long.Parse(doc_id),long.Parse(proj_id),out
            b);

If the result is true then the reader is iterated and an array type collection is filled.

while(sdr.Read())
{
arr.Add(sdr[0]);
}
sdr.Close();

Otherwise the response is redirected:

Response.Redirect("docview.aspx");

Once the different users are selected, the document is sent to the specified users by calling the docSendTo method that takes the selected users as a parameter. If the selection is empty then the response is redirected to the main document page, else, the documents are sent and then the response is redirected:

if(spliter!=null)
{
string[]
            selusers=Spliter.Split(',');
obj.docSendTo(selusers,long.Parse(Session["did"].ToString()),long.Parse(Session["flid"].ToString()),
long.Parse(Session["pid"].ToString()),out
            b);
}
Response.Redirect("docmain.aspx");

Document View

Options:

Delete: Delete the document being viewed.

Cancel: Select cancel and go back to the main document screen.

Description: The user can see the document that he selected on the main document screen, with the following fields dislplayed:

  • Document Name: Name of document
  • Document Description: Description of document.
  • Document Type: Type of the document.
  • Download Document: Download the document to your local hard drive..

If the user is creator of the document then he can also delete this document, otherwise the document is only viewable for those to whom it was specified at the time of creation. On page initialization, all the relevant information about the document is loaded, and the controls are filled:

name.Text=rdr[3].ToString();
desc.Text=rdr[5].ToString();
type.Text=rdr[4].ToString();
cron.Text=rdr[6].ToString();
createdby.Text=rdr[8].ToString();

If the user opts to delete the document for some reason, then first of all the user's rights are checked. If the user turns out to be the creator of the document then he is given the permission to delete the document and the DeleteDocument method is called, else the response is redirected to a new page and an appropriate error message is called.

Add Folder

Options

Save: Create and save the newly created folder as per details provided by the user.

Cancel: Cancel the current activity and go back to the main document page, without saving anything.

Description: The user can create a new folder by specifying the name of the new folder as it would appear on the screen, and the user can also provide a description of this folder (e.g. the user might specify the folders as a "document" folder, a "code" folder, etc).

int b=0;
long
            newid=obj.AddFolder(name.Text,desc.Text,long.Parse(Session["flid"].ToString()),
Session["id"].ToString(),out
            b);
Response.Redirect("docmain.aspx");

Task Module

The task module primarily deals with the Task entity of database. Tasks are very important in project management. Once a task is assigned, a notification is sent to all of the assigned users.

Database Design

The task entity contains attributes like task ID (for identification purpose), description of task, assigned date, due date, etc.

Entity Attribute | Description
Tsk_ID | It's the primary key used to uniquely identify each Task
Tsk_Name | Name of the Task
Tsk_Description | Additional details about the Task
Tsk_AssigneOn | Date on which task was assigned
Tsk_DueDate | Date at which Task is expected to be completed
Tsk_Status | Shows Current Staus of the task
Tsk_AssignedBy | UserID of the person who assigned the task.

Dependencies

Entity Dependency | Description
Task_User | Used to show the relationship of task with project and users

Class Diagram

The class diagram remains quite similar to that of the Message and Document modules.

Class Functionality

The following diagram shows the important functionalities that this Tasks class provides:

Use Case Diagram for Tasks Module

Implementation

Iscreator: This method verifies that whether the user is the creator of the task specified by Task ID in the project specified. It returns true if he is, false otherwise. The query is very simple, and goes like this:

"select count(*) from task where prj_id = "
            + p_id + " and lower (tsk_assignedby) = '" + userid.ToLower() + "'
and tsk_id = " + tid

This query is executed by the ExecuteScalar method so that the count could be checked. If count is equal to one, then the user is the creator of the task and is allowed to do the modification, else his role is restricted.

Add: This method adds a new task into database and assigns it a Task ID. As we did with the messages and documents, we will generate the id for the task first, and then we will create a new task with the calculated ID.

TaskSendTo: This method associates the task with the correct users.

string strQuery="insert into task_user (tsk_Id, prj_Id, usr_LoginId, tsk_viewed) values
(" + t_id + "," + p_id + ",'" + user[cnt] + "', 1 )"

Delete: This deletes the task with the Task ID taken from database, but only if the user ID passed in as a parameter is the assignee of this task.

"Delete task_user where prj_Id="+p_id+" and tsk_id="+t_id

Initially the record from the dependent table is removed, and only then is the record from task deleted.

"Delete Task where prj_Id="+p_id+" and tsk_id="+t_id

show_events: This method is used in the Calendar module in which all the tasks of the specified user, in the specified project, with the specified due date are returned. The view mode specifies that the result will be the tasks of a day, week, or whole month.

"Select a.tsk_Id, a.prj_id, tsk_Name, tsk_Description, tsk_AssignedOn, tsk_DueDate, tsk_Status,
tsk_AssignedBy from task a, task_user b where a.tsk_id = b.tsk_id and b.prj_id = " + pid + " and b.usr_loginid
= '" + uid + "' and convert (datetime, tsk_duedate, 103) between '" + dte + "' and
dateadd (day, " + view + ",'" + dte + "')"

After the execution of the query, a reader is returned which contains all the records that fulfill criteria specified in the query.

View_user: This method is used when selecting users to whom this task will be assigned, and it will return all those users who haven't been assigned this task yet.

Update: This method updates the task details.

UpdateStatus: This method updates the task status to new status specified in parameters.

"update task set tsk_status ='" + status + "'
where prj_Id = " + p_id + " and tsk_id = " + t_id

UI Design and Coding

Task Main

Options

New Task: The project manager can create a new task, to be assigned to number of project members.

Description: The user can view tasks that are assigned to him. If the task is new then that task will be highlighted, else it will remain dim. A class tskret is created to provide get/set abstraction for the task entity. All the tasks that are currently available/assigned to the current user are loaded in a ArrayList collection.

tsk.Add(new tskret(rdr[2].ToString(),rdr[6].ToString(),rdr[7].ToString(),long.Parse(rdr[0].ToString()),
(DateTime)rdr[5],"New"));

If the user opts to view all the tasks, then they are loaded into the page. If the View_all method of the class Task executes without any error, then all the tasks are added to a local collection named tsk.

string nchk="
            ";
if(int.Parse(rdr[8].ToString())==1)
nchk="New";
tsk.Add(new
            tskret(rdr[2].ToString(),rdr[6].ToString(),rdr[7].ToString(),long.Parse(rdr[0].ToString()),
(DateTime)rdr[5],nchk));

If the View_all method returns some error code (enum values other than 0), then the View_short method is called.

tsk.Add(new tskret(rdr[2].ToString(),rdr[6].ToString(),rdr[7].ToString(),long.Parse(rdr[0].ToString()),
(DateTime)rdr[5],"New"));

Task Add

Input Fields

Field Name | Type | Validation
Task Name | Alphanumeric | Yes
Task Description | Alphanumeric | No
Assigned Date | Date Time | Yes
Due Tame | Date Time | Yes
Status |  |
Selected User | User Id | Yes

Options

Save: Once all the fields specified on this page are entered then the project manager can click save to send the task to selected Users.

Cancel: By clicking cancel, the project manager can go back to the Task Main screen where he came from to add a new task.

Description: The project manager can add a new task for the members of his project. While the project manager can assign new tasks to the project members, members can only respond to the project manager by message. Non-project members can not assign tasks. For their messaging requirements, the message module can be used.

When the user opts to assign a task, first all the screen level validation is done. Then we check whether the current user is a project manager or not. If the current user is a project manager, then the task is added, else an error message is shown. Here, screen level validation includes one very important thing, and that is checking that the task assigned date is always less than the due date:

if( syr<=dyr && smn<=dmn && sdy<=ddy)
{

If there are no problems, then the task is added:

if(obj.isManager(long.Parse(Session["pid"].ToString()),Session["id"].ToString(),out b))
{
string start=smon.SelectedItem.Value+"/"+sday.SelectedItem.Value+"/"
+syear.SelectedItem.Value;
string due=dmon.SelectedItem.Value+"/"+dday.SelectedItem.Value+"/"
+dyear.SelectedItem.Value;
long newid=obj.Add(Session["id"].ToString(),long.Parse(Session["pid"].ToString()),
tsk_name.Text,tsk_desc.Text,start,due,out b);

The task is then assigned to the selected users. For this purpose, a splitting technique is used, and once the selected users are identified, the TaskSendTo method is called. If there is no user to whom this task is to be sent, then the response is redirected to the main page, otherwise the task is sent to specified users first and then the response is redirected.

Response.Redirect("taskmain.aspx");

Task Edit

Input Fields

Field Name | Type | Validation
Task Name | Alphanumeric | Yes
Task Description | Alphanumeric | No
Assigned Date | Date Time | Yes
Due Tame | Date Time | Yes
Status |  |
Selected User | User Id | Yes

Options

Save: The project manager can create a new task, to be assigned to a number of project members

Cancel: By selecting the cancel option, the details of the task remain unchanged.

Details: Whenever a project manager assigns a task to a project member, the only way for a member to respond to this task is by replying to the manager with a message. It is then the manager's responsibility to alter the task status accordingly. If the task has been accomplished by the assigned people, then the manager should change the status of the task from "in progress" to "completed", or even change the details of the task.

Suppose if the initial task assigned was to construct a GUI design specification for some project. The project manager can alter that task detail, perhaps by adding more users to the task, or even extending the due date.

When the page loads, we check whether the current user is the creator (or assignee) of the task. If the he is then the task object is loaded into the page and the controls are filled with specified values (from the data reader).

if(obj.iscreator(long.Parse(Session["tid"].ToString()),
long.Parse(Session["pid"].ToString()),Session["id"].ToString(),out
            b))
{
SqlDataReader sdr=obj.View(long.Parse(taskid),out
            b);
if(sdr.Read())
{

If the reader contains a record then we can assign the values to the controls.

If the user is not the creator (or assignee) of the task, then the response is redirected to the task main page. If the user is the creator of task, and he opts to change the values of the task, then the new values are stored back into the database through the Task assembly.

Validation is then done - if validation is successful, then the new values are stored back into the database, otherwise an appropriate error message is shown to the user.

error.Text="Please make sure that selected due date is not less than start date..";

TaskView

Options

Cancel: The user can cancel and go back to the main task screen.

Delete: Only the project manager can delete the task, for everyone else the link will remain inactive.

Edit Task: Only the project manager can edit the task whenever he wants, the links will not be active for anyone else.

Descriptions: The Project manager can alter the message details by editing the message and can also delete the task. For other users, these links won't be active, so they cannot take any action. When the page initializes, the task is loaded and displayed on the page.

        taskform
            obj=new taskform(con.getcon());
SqlDataReader
            rdr=obj.View(long.Parse(taskid.ToString()),out b);
if(b==0)// if
            no error
{
if(rdr.Read())
{

Once the task has been viewed, it's status is set to "viewed".

obj.UpdateViewed(long.Parse(taskid.ToString()),long.Parse(proj_id.ToString()),uid,out b);

If the current user is the creator of the task, then he may also delete the task, otherwise the delete link would be inactive.

error.Text="You dont have delete rights";

Project Module

Database Design

Entity Attribute | Description
Prj_ID | It's the primary key used to uniquely identify each project.
Prj_Name | Name of the project
Prj_Description | Additional details about the project
Prj_StartDate | Starting Date of Project
Prj_DueDate | Deadline of Project
Prj_Comments | Any Comments about the project
Prj_Url | Location of the project where it is being deployed
Prj_Creator | UserID of the person who created the project.

Dependencies

Since the project entity is the central entity, and this application is about project management, it's natural so find most of the entities being dependent on it.

Entity Dependency | Description
Project_User_Group | Used to relate project with users and groups in it.
Message_User | Used to show the project in which the particular message is visible
Document_User | Documents are only accessible within project related here.
Task_User | Used to show the project in which this task.has been assigned

Class Functionality

Project managers can add, edit, and delete projects.

Use Case for Project module.

Calendar Module

The calendar module doesn't actually have a database entity. As a matter of fact, it supports other modules like tasks, projects, etc. The calendar module provides very important functionality to this whole application. It provides a way to view tasks. Tasks can be viewed in the calendar view, categorized as monthly view, weekly view, daily view, etc.

Use Case for Calendar module

In this section we will be seeing a screen shot of the calendar module with some code snippets to explain what actually is going on at the back end.

Options

View By day: View the tasks by Day.

View By Week: View the tasks by Week.

View By Month: View the tasks by Month.

Description: In this module, the user can view tasks on a daily, weekly or monthly scale. When the page is loaded, the tasks for the current day are shown (if any). If the user opts to view the tasks in a daily format, then he can navigate the calendar on a daily basis to view further tasks, and likewise for the weekly and monthly formats. Whenever a view mode is selected, the show_events method is called to display the corresponding tasks:

SqlDataReader rdr=obj1.show_events(uid,proj_id,cal.SelectedDate.ToShortDateString(),viewtype,

Get the application running

Building the solution

In this section, we will build the whole application. We will be seeing how to tie all the modules that we built in the previous section together.

Interaction between different modules

To build this application you will need to use either Visual Studio.NET or ASP.NET Web Matrix. Although we have used ASP.NET Web Matrix for this application, and we will be discussing the reason for this choice in a short while, you might well want to use Visual Studio .NET.

Building Application Using Visual Studio.NET

To build this project management suite using Visual Studio, we will need to create assemblies. We can create a single assembly containing all the class definitions that we have developed in the previous section. The advantage of using this approach is that it will be easier to build the application, and also it will require less effort and will provide better integration. At the same time, a single assembly approach could expose a few problems - for example, if after you've deployed the project you want to make some changes to a particular module, then the whole assembly would require alteration. On the other hand, using a single assembly per module makes it easier for the developer to make alterations.

Now, to build this intranet based project management suite you will need to create a new project, let's call it IntranetBasedProjectManagement, selecting Class Library as the template type. Once the project has been loaded, you will need to add existing items (the code for the modules) to this project.

Building Intranet Based Project Management Suite Using Visual Studio .NET

By doing this you should be able to add all the .cs files to the IntranetBasedProjectManagement project. Once all the items are added, you can build the application, which will yield a single assembly in the debug folder of the current project.

One assembly per module could easily be created using the command line C# compiler (using the method discussed at the start of the case study). After doing all this, you are left just needing to create interfaces (ASP.NET pages), that you can either create using the Code Behind or Code Inline methods.

Building Application Using ASP.NET Web Matrix

We chose ASP.NET Web Matrix as our development tool because of the cost of Visual Studio. Besides being a free tool, it is also quite easy to use, and at the same time its size is minimal. Another big plus of using ASP.NET Web Matrix is that its size is smaller. For more details you can visit http://www.asp.net/?WROXEMPTOKEN=618658Zezi9SpDFSZgQps5Xv6u.

Building the intranet based project management suite using ASP.NET web matrix requires you to create assemblies before developing your ASP.NET pages. Once you have created your assemblies, you can access them from your ASP.NET pages by simply placing them in the /bin directory, under your current directory containing the ASP.NET pages. You can refer to those assemblies by simply using a Namespace directive in an ASP.NET page.

To start doing it all, after creating assemblies, you will need to create an ASP.NET file in ASP.NET Web Matrix, specifying C# as a coding language.

Now you can develop your ASP.NET pages using this easy-to-use tool.

Running The Application

One of the most powerful features of this application is that it provides an XCOPY installation. Even for a server, all you need to do to get your web server running is IIS 5.0 or later, configured for ASP.NET runtime or ASP.NET web matrix server. For a client, all the client needs is a web browser and a connection to the server.

One important thing is that database server should be configured, and the connection string property of the database connection class should be set up properly. There are more notes on installation in a setup guide available with the code download for this case study.

Further Work

In this application, we have tried to keep things simple and at the same time fully functional - we haven't tried any thing fancy in this application.

In the coming versions of this application: we might provide a mobile interface to this application; we might utilize some important features of .NET like disconnected databases in the form of Data Sets; ASP.NET form authentication; Web Farms scenario could be considered; user id / password encryption; we can also expand the hierarchy of users and groups, etc.

Conclusion

In this case study we have learned how .NET can be used to deploy an enterprise solution in a 3-tier application. Although we haven't dug deep enough to get to talk about and implement Data Sets and Data Binding controls, etc., we still have covered a lot of ground. We started with the .NET architecture, what CLR is, what the .NET base classes are, and how powerful C# is as a language. We further noticed that at the front-end, ASP.NET provides an Ideal solution for web interfaces. We have also seen what assemblies are, and how they give us a break from DLL Hell. I hope this case study will help .NET developers when developing their own web-based applications.


Developing A Rich Client Forum Reader Using Windows Form and Web Service

Developing A Rich Client Forum Reader Using Windows Form and Web Service
http://www.csharptoday.com/content.asp?id=1902
Article Rating: 4
Published on September 23, 2002
Abstract
In this case study, Steve Shi demonstrates how to develop a set of XML Web Services to expose the contents and functionalities of a traditional discussion forum website, as well as a rich-client application that would consume these web services. By interacting with the supporting Web Services, the rich-client application provides such basic functionalities as browsing forum contents, posting new topics, and replying to existing topics - hence the name rich-client forum reader. Furthermore, the most notable feature of the rich-client forum reader is the ability to cache all forum contents retrieved from the supporting Web Services in an internal local database, thereby providing an offline forum browsing capability. The forum reader is implemented as a full-fledged Windows Forms application under the .NET platform, with its underlying component architecture strictly based on classic object oriented analysis and design principles. It sports an intuitive GUI that loosely resembles that of Microsoft Outlook, with the underling data access components fulfilling their tasks of retrieving and replying to online forum messages by interacting with the supporting web services on the forum server.
Case Study

In this case study, Steve Shi demonstrates how to develop a set of XML Web Services to expose the contents and functionalities of a traditional discussion forum website, as well as a rich-client application that would consume these web services. By interacting with the supporting Web Services, the rich-client application provides such basic functionalities as browsing forum contents, posting new topics, and replying to existing topics - hence the name rich-client forum reader. Furthermore, the most notable feature of the rich-client forum reader is the ability to cache all forum contents retrieved from the supporting Web Services in an internal local database, thereby providing an offline forum browsing capability. The forum reader is implemented as a full-fledged Windows Forms application under the .NET platform, with its underlying component architecture strictly based on classic object oriented analysis and design principles. It sports an intuitive GUI that loosely resembles that of Microsoft Outlook, with the underling data access components fulfilling their tasks of retrieving and replying to online forum messages by interacting with the supporting web services on the forum server.

Introduction

A Brief Look at the Forum Reader Application

What is a forum reader application? What do we use it for? We'll briefly answer these questions now, and a more in-depth discussion will be presented in the sections that follow.

The so-called rich-client forum reader that we've developed is a client-side windows application that provides the same functionalities as a discussion forum website, together with some enhancements. Practically, it is developed as an extension to a traditional forum website rather than a replacement of it. Typically, companies running forum websites would be interested in developing such software and distributing it among their forum users. As a forum user, other than using a web browser to visit the forum website, you can alternatively use the forum reader to access and interact with the same online forum contents. For example, you can use the forum reader to browse online forum contents, post new topics, and reply to existing topics. Besides these functionalities, the forum reader can automatically save all the online forum contents that you have viewed in its internal local database, which makes for easy offline reference in the future. This is a unique feature of the forum reader, as by contrast, it is unavailable in the browser-based client.

However, in order to support the forum reader to perform its functions, a set of Web Services must be developed and installed on the forum web server in the first place. The purpose of these Web Services is to provide programmatic access to the backend database of the forum website. Or, to put it in another way, these Web Services serve to expose the online forum contents and functionalities that are intended to be accessed by the rich-client forum reader. Therefore, by interacting with these supporting Web Services, the forum reader could perform such basic tasks as retrieving forum topics from, or posting new topics to, the online forum database. As I mentioned earlier, the retrieved forum contents are not only displayed in the user interface but also saved in an internal local database of the forum reader. We will explain this feature in detail a little later.

What do we mean by "rich-client"?

As you have seen, when we talk about the forum reader application, we often refer to it as "rich-client" forum reader. Why do we qualify the forum reader application with the term "rich-client"? Literally, the term "rich-client" has a twofold meaning, which is "both rich and client-side". However, this is only a vague interpretation. Precisely, in .NET terminology, the term "rich-client" is typically used to refer to windows application that performs the role as the client-side entity in a distributed client/server architecture. This term is specifically coined to distinguish windows-based client-side application from browser-based client (i.e., the web browser). The word "rich" in this term both implies and stresses the point that windows applications can take advantage of a rich set of resources and features that are provided by the underlying operating system and the .NET platform. However, by contrast, these resources and features are not accessible from a browser-based client.

The Perspective, Scope, and Merits of this Case Study

With the advent of the .NET platform and the shipment of Visual Studio.NET, the development of traditional Windows applications that have built-in support for consuming Web Services has been dramatically simplified. With this case study, you will have the chance to walk through the design and development of a complete application implemented with Windows Forms and Web Service technology. Moreover, in contrast to other kinds of articles that drill down into one specific .NET programming technique or a particular piece of .NET technology, this case study aims to depict the big picture of the component architecture of a sample .NET application and examine the key design considerations as well as the implementation details related with each component. By and large, the focus of this case study is more on the architectural model than on specific coding details and techniques.

What's the merit of this case study for the readers?

In a sense, the forum reader application that we'll examine in this case study is a full-fledged one in terms of both design consideration and implementation. It has incorporated a bit of almost everything as far as a moderately complex Windows Forms application is concerned. For example, at the design stage, all four major components that make up the entire object model of a typical windows application have been addressed. These include the user interface component, the data access component, the business logic component and the external system interaction component. On the other hand, at the implementation stage, it employs quite a handful of .NET-centric technologies and techniques, such as Windows Forms GUI widgets, object oriented programming in C#, multithreaded components, ASP.NET Web Services, data access with ADO.NET and stored procedures, etc.

So by reading through this case study, you will have a bird's eye view of the various architectural components of an entire Windows Forms application as well as a few relevant close-ups of some coding practices.

System Requirements

You should have a set of necessary software in place in order to experiment with the source code in the download package accompanying this article. These include:

  • Microsoft .NET platform 1.0 (with service pack 1 or above)
  • Visual Studio.NET (Visual C#)
  • SQL Server 2000
  • IIS 5 or above

As regards the level of expertise assumed in this case study, you should have:

  • A solid handle on the C# programming language
  • Familiarity with database design and implementation using SQL Server and stored procedures
  • Familiarity with ADO.NET
  • A working knowledge of windows programming using .NET Windows Forms
  • Basic knowledge of Web Service design with ASP.NET
  • A basic understand of Unified Modeling Language (UML) notation

Case Study Structure

In this case study, we'll walk through the design and development of the forum reader application and examine the inner workings of its major components. Here in the list below, we briefly summarize each section that follows so that you could easily get your bearings during the walkthrough.

Defining The Problem

This section talks about the specific business problem that the forum reader application intends to solve.

Defining The Solution

  • Solution Overview
  • The Forum Reader Application In Context
  • Defining the Functionalities of the Forum Reader
  • The Local Caching Mechanism of the Forum Reader

Database Design and Structure

All the tables, fields, constraints and stored procedures are covered in this section:

  • Online Forum Database
  • Local Forum Database

Web Service Design

This section discusses the development of the supporting Web Services:

  • The Data Access Layer
  • The Façade Layer

Componentization of the Forum Reader

This section describes the component architecture of the forum reader application:

  • Physical Component Packages
  • Functional Sequence

Architectural Overview of the Overall Class Model

This section takes a detailed look at the entire class model of the forum reader application.

User Interface Design

  • Laying out the GUI widgets
  • Other UI-related classes
  • Examining GUI behaviors
  • Marshalling Cross-Thread Function Call

Remote Data Access Component

This section briefly talked about the Web Service proxies used in the application.

Local Data Access Component

This section explains the component that accesses local forum database.

The Business Logic Component

This section describes the details of each of the business logic related classes.

  • CategoryManager Class
  • TopicsManager Class
  • RepliesManager Class
  • AccountManager Class

A Symphony of Objects Interaction

This section demonstrates how the various objects collaborate with each other to deliver a particular functionality as a whole.

Application Setup

This section gives step-by-step installation instructions for the download source files.

Defining The Problem

Our problem is concerned with online discussion forum websites, especially those with considerable amount of forum contents. Let's set the scene:

The chances are, as a programmer, you often go to various technology-centric forum websites to seek answers to your technical questions or to look for solutions to the problems that plague you. Whenever you find a particular topic of your interest, an intuitive approach you take to reserve it for future reference is simply saving the web page to your local hard disk. As the volume of topic and reply swells up on the forum website, your local directory tends to become bloated if you're a frequent forum visitor. Moreover, your locally stored topic pages easily become out of sync with the original online topics if there're new replies posted, and your further attempts to re-sync your locally stored forum web pages with their online counterparts will result in more files being saved to your local disk. The upshot of it all is you end up having a whole slew of unordered and discrete files in your local directory. It wouldn't be a pleasant experience if you had to sift through these files for useful information in the future. In this circumstance, you start to ask whether it is possible to have some sort of software utility that could facilitate the process of retrieving and viewing online forum contents as well as saving them locally for future reference.

So this story depicts and also defines our problem. In the following sections, we'll introduce the solution that we've devised.

Defining The Solution

Solution Overview

Figure 1: The solution includes two major functional components

Broadly speaking, the solution that we devised includes two major functional components: a rich-client forum reader application and the supporting forum Web Services. So we naturally took a two-step approach to implement this solution.

First, we designed a set of Web Services that expose the forum contents by directly interacting with the online forum database. At one end, these Web Services invoke the same set of stored procedures that are also used by the forum website to store and retrieve forums contents. At the other end, these Web Services expose the forum contents as well as the functions to manipulate them through public interfaces (i.e. Web Methods) that could potentially be consumed by any Web Service enabled client application. In effect, these Web Services serve to bypass both the presentation layer and the business logic layer of the traditional forum website, and expose its data access layer through a public programming interface that wraps a direct call to the database. Specifically, these Web Services perform three primary tasks:

  • To return the requested forum contents such as the categories, forums, topics or replies to the client.
  • To accept the information posted from the client, such as new topics, replies, or account registration requests, and then add this data to the relevant forum tables.
  • To authenticate the user through a logon process and maintain the user's session state in the server using native ASP.NET session management facilities.

Secondly, we built a traditional windows application called the rich-client forum reader. By interacting with the Web Services implemented at the forum server, the forum reader application encapsulates a number of essential forum interaction functionalities that correspond to those of its forum website counterpart, together with additional add-on enhancements. Specifically, the basic functionalities of the forum reader include navigating different forums, browsing through the topics of a given forum, viewing replies, posting new topics, replying to existing topics, registering membership, login/logout, etc. The main task of this development was to extract the business logic layer and presentation layer of the forum website and rebuilt them into a self-contained rich-client forum reader application. The forum reader application is in effect a client-side replica of the forum website but in a different incarnation. In addition, the forum reader has implemented a local caching mechanism to save retrieved forum contents in an internal local database, which takes advantage of the local processing power and local data storage capabilities that are unavailable with the web browser. This particular feature represents the most notable enhancement to the forum reader that makes it more functional than the forum website. A more detailed description of this feature will be given a little later.

The Forum Reader Application In Context

When this solution is brought to life in a real world production environment, the forum website owner will want to distribute his own custom-built forum reader application among the forum users. It goes without saying that if there are a thousand different users who have actually installed and used the forum reader application in their own computers, then there will potentially be a thousand distinct running instances of the forum reader application that all access the same online forum database. Below is a diagram illustrating the functional role of the forum reader application in relation to other components of the overall solution, including the forum Web Services, the original discussion forum website, and the database server. With this diagram, you should have a good picture of the forum reader application in the context of the real world scenario. Additionally, the diagram clearly illustrates the fact that whether a user views the forum contents using a web browser or using the forum reader, he ultimately accesses the same online forum database.

Figure 2: The forum reader application in production environment

Defining the Functionalities of the Forum Reader

Practically, we could extract as many functionalities as possible from the forum website and inject them into the design of our forum reader application. However, we've only covered a subset of them in the sample application that is essential for forum interactions. Specifically, the forum website functionalities that we choose to reconstruct in the forum reader application are listed below:

  • Allowing the users to retrieve a list of available categories and forums from the remote forum Web Services, and viewing the result in a tree view hierarchy for forum navigation.
  • Allowing the users to retrieve a selected page of topics in a particular forum from the forum Web Service, and then viewing the result in a list view.
  • Allowing the users to retrieve the replies to a particular topic, and then view the result in a rich textbox .
  • Allowing the users to register a new account, login, or logout from the forum with his account.
  • Allowing the users to post new topics or send replies to an existing topic against the forum Web Service. However, users must be logged on to the forum before they can do so.
  • Saving all the forum contents retrieved from the forum Web Services in a local database and allowing the users to browse through these saved forum contents.

The functionalities listed above serve as an outline of the project specification for the development of the forum reader application. As you have seen, all the functionalities list above, except for the last one, merely echo the online forum website counterpart. However, considering the last one in the above list, we have implemented a local caching mechanism in the forum reader application that is a unique feature compared with the forum website. We won't go into details about this local caching mechanism now, because a more detailed description will be given in the next section.

Below is a USE CASE diagram illustrating all the functionalities mentioned above, which we have implemented in the forum reader application.

Figure 3: Forum Reader Application Functionalities

The Local Caching Mechanism of the Forum Reader

As we've mentioned in the preceding sections, the capability to save online forum contents locally is the most notable feature of the forum reader. Now, let us take an in-depth look at the so-called local caching mechanism.

What is the local caching mechanism?

You can use the forum reader to view online forum contents. Here, by "forum contents", we mean the forum topics and replies as well as the list of available categories and forums. Obviously, in order to display the specific forum contents that you request to view, the forum reader must first retrieve the requested forum contents from the remote forum Web Services. After the requested forum contents have been retrieved, they are not only displayed in the user interface, but also saved in an internal local database of the forum reader. As you browse through the forum contents, a series of requests to retrieve the relevant forum contents are sent to the forum Web Services one by one. Consequently, the forum contents corresponding to each request are then retrieved, and displayed in the user interface. More importantly, during this process, the retrieved forum contents corresponding to each request are also saved in an internal local database of the forum reader, one piece at a time. More precisely, the saving of each piece of retrieved forum contents happens immediately before they are displayed in the user interface. This is what the local caching mechanism of the forum reader all about.

With each installation of the forum reader, a local database is created and maintained in the local directory where the forum reader is installed. This local database is only for the internal use of the forum reader and it is implemented using a MS Access database. The local caching mechanism is implemented in such a way that whenever the forum reader is used by a given user to view online forums, it automatically save whatever forum contents that the user has viewed during each session into this internal local forum database. In other words, the forum reader automatically maintains a local forum repository by accumulating all the forum contents that the user has viewed in a private local database.

Note that the term "local forum database" is used frequently throughout this case study and it all refers to the client-side forum database maintained by the forum reader application. Additionally, one point you should be clear about is that each particular installation of the forum reader maintains one and only one local forum database. This implies that if the forum reader has been installed on one thousand different client computers, then there will be precisely one and only one local forum database existing in each of these one thousand client computers.

What exactly has been saved in the local database?

Let us use an example to answer this question. Suppose you are the user of a particular forum website that has implemented the aforementioned Web Services and provides a custom-built forum reader for public download. You download the forum reader from this website and install it on your computer. Then, instead of using your web browser to visit the forum website, you start to view the online forum contents using your newly installed forum reader. The fact is, from then on, whatever online forum contents (i.e. topics, replies, etc.) you have ever viewed through this forum reader will all be saved in the internal local database of the forum reader. On the other hand, the online forum contents that you haven't bothered to view through your forum reader will not make their way into your local forum database. In effect, the so-called local forum database maintained by the forum reader is a partial copy of the online forum database and specifically, it only contains the specific forum contents that you have previously viewed through this forum reader. One point to note is that the local forum database does NOT contain a complete duplicate of the entire online forum contents unless in an extreme case you have manually traversed and viewed all the contents of the online forums using the forum reader. The local forum database serves the purpose of storing only those forum contents (i.e., topics, replies, etc.) that you have actually viewed through the forum reader. Put in another way, the functional role of the local forum database is to aggregate all the online forum contents that you have actually viewed using the forum reader application.

What exactly has NOT been saved in the local database?

This is an odd question. However, knowing a definite answer to this question helps you to fully understand the local caching mechanism. Practically, it's very likely that they are some, if not many online forum contents that you haven't retrieved and viewed through the forum reader. The online forum contents that you haven't retrieved and viewed will NOT be saved in the local database of your forum reader. Put in another way, you won't find in the local database any online forum contents that you haven't previously retrieved and viewed.

What will happen if some newly retrieved forum contents already exist in the local database?

Suppose, at an earlier time, you retrieved the replies to a certain topic from the forum Web Service and as a result, these replies were saved locally. Now, if you retrieve the replies to the same topic again in order to check if there are new replies posted, will the currently retrieved replies overwrite the previously saved ones? Naturally, if the currently retrieved forum contents are more up-to-date, then they will overwrite existing ones. Otherwise, the forum reader will not bother to save the currently retrieved forum contents because they are identical to the existing data. This is the logic used in the local caching mechanism to determine whether newly retrieved forum contents should overwrite existing ones in the local database.

How can you control the local caching mechanism?

The fact is, you don't have much control over the local caching mechanism. The local caching happens whenever you retrieve and view some online forum contents. The saving of the retrieved forum contents is an automatic process, which happens behind the scenes. The only way you can order the forum reader to save some specific online forum contents locally is simply to view them. This means the local caching mechanism that we implement in the forum reader is a kind of passive caching. You view some online forum contents and they will be saved locally; if don't view anything, nothing will be saved. The local caching mechanism implemented in the forum reader doesn't have the capability to grab an entire online forum (including all of its topics and replies) in one go. If you want to achieve this effect, you have to manually traverse the forum by viewing every topic and reply therein.

On a related note, after the online forum contents have been saved locally, they will persist in the local database from that point on. For the sample application, we haven't implemented the feature to delete forum contents stored in the local database. However, it shouldn't be a difficult task to add this feature if you so desire.

The Benefit of the Local Caching Mechanism

After all, what good is it maintaining such a local forum database or implementing such a local caching mechanism? As you can imagine, the purpose of it is to support offline reading. Given the fact that a local forum database contains all the forum contents a given user has ever viewed, a new functionality based on this feature could be implemented in the forum reader application that allows the users to browse through the locally stored forum contents without connecting to the internet. If you simply want to revisit a previously viewed forum topic or reply, you just need to browse through the locally stored forum contents, thereby avoiding the hassle of getting on the Internet and browsing through the online forums. In short, this feature makes for quick and easy offline reference without the need to wait for the slow Internet response. On the other hand, as an additional bonus, a certain amount of internet traffic is reduced on the forum server because of this offline browsing feature. By and large, the forum reader, in conjunction with the supporting web services, make an excellent compliment to traditional discussion forum websites.

Database Design and Structure

In essence, our forum reader application reflects a variation of the traditional client/server solution architecture with a modern day Web Services technology. By virtue of this nature, our database design issues are concerned with two functionally interrelated but physically discrete databases, namely an online forum database and a local forum database.

Online Forum Database

The so-called online forum database is simply a backend database server that stores forum data and contents. This database server is originally installed to support a front-end discussion forum website. It is against this existing forum database server that we've built a new interface using ASP.NET Web Services to both expose and interact with its contents. In other words, we simply built a new data access layer to an existing forum database server using Web Services, rather than adding a new dedicated database server to solely service our forum reader application. Because an existing forum website's database schema should have already been properly designed with all necessary tables and stored procedures in place, so the Web Services we built simply access these existing tables and stored procedures. Usually, when implementing such a solution in a real world scenario, there is no actual work concerned with designing the online forum database. With the online forum database here, our only concern is only to examine its schema and get familiar with its stored procedures.

In our sample application, we choose Microsoft SQL Server as the database server for the online forum. The reason for this choice is simply because the .NET framework provides a set of more optimized classes to support SQL Server. Specifically, these classes are grouped under the namespace System.Data.SqlClient, and they directly communicate with Microsoft SQL Server using a low-level proprietary protocol, which bypasses both OLE DB and ODBC. Therefore, by using these classes, we get dramatic performance benefits.

Online Forum Database Schema

Because the main purpose of this case study is to demonstrate the design and development of the entire forum reader application, as a trade-off I purposely choose the online forum database schema to be reasonably simple and straightforward for the sake of clear illustration. As for the sample online forum database, it only incorporates just enough tables and fields to support a bare minimum of essential discussion forum functionalities. When you have a clear picture of the entire application architecture, you should have no problem to adapt it to work with any sophisticated forum database structures.

The following diagram shows the full online forum database schema. As regards the column naming convention in the database schema definition, we use Pascal Casing for all primary key and foreign key columns. For all non-key columns, Camel Casing is used. Even though Pascal Casing and Camel Casing are typically recommended in the .NET class naming conventions, here I also use them for the naming of the database table columns. For those of you who happen to be unfamiliar with these two capitalization styles for identifiers, here is a brief definition:

  • In Pascal Casing, the first letter in the identifier and the first letter of each subsequent concatenated word are capitalized.
  • In Camel Casing, the first letter of an identifier is lowercase and the first letter of each subsequent concatenated word is capitalized.

Figure 4: Online Forum Database Schema

One-to-Many Relationship

From the above diagram, you can easily identify a cascading one-to-many relationship existing between the forum tables. Specifically, a particular forum category could have many forums grouped together where each one in turn could possibly contain many discussion topics where each topic tends to receive many replies. This coherent model of one-to-many relationship is enforced by first installing a primary key in each table and then establishing foreign key constraints between the related tables to reference the primary keys.

Additionally, if you have experience of developing a forum website, you may also notice another design consideration from the above diagram. That is, the topic and reply entities are separated into two different tables instead of being combined into a single one.

While the above diagram is already self-explanatory as regards the meaning of each table and field, a brief summary of their exact definitions are presented here for your reference:

Categories Table

The Categories table stores the information describing the categories by which the forums are grouped together. An auto-numbering identity field named CategoryID is defined as the primary key of this table, which is also used for foreign key reference from the Forums table. The categoryName field stores the name of categories. There is a unique constraint placed on the categoryName field to ensure that no duplication of category name among different categories is ever allowed.

Forums Table

The Forums table stores the information about the subject focus and a brief description of the forums in the forumName field and the description field respectively. The identity field ForumID serves as the primary key, whereas the CategoryID field refers to the category to which a specific forum belongs. The CategoryID field is also used to create a foreign key constraint, which references the same name primary key field of the Category table. In addition, there is a unique constraint applied to the forumName field to ensure that all forum names must be unique.

Topics Table

The Topics table stores the topics that are the initial messages posted against a particular forum. Conceptually, every topic in this table is intended to represent the subject for a new thread of discussion, which will aggregate the relevant replies in the Replies table.

The TopicID field is the primary key identifying each topic. The ForumID field is a foreign key that refers to the forum to which a specific topic is posted.

The subject and message fields represent the caption and inner contents of the topic, while datePosted represents the date when a topic is posted.

The Originator field represents the user who posted a specific topic. This is a foreign key field referencing the relevant user account data stored in the Users table.

The viewCount and replyCount fields are used to record statistical information about a particular topic, which reflects the level of popularity of each thread of discussion. The lastPost field indicates the time at which the latest reply pertaining to a topic is posted. Specifically, whenever a given topic is viewed either through the forum website or from our forum reader application, its viewCount should be incremented. Whenever a new reply to a given topic is posted to the Replies table, the replyCount of this topic should be augmented together with the lastPost value updated in accordance with the date of this reply.

The Revision field is used to identify whether a topic has been changed since the last time we looked. To give an example, when a new topic is posted to the Topics table, its Revision field initially takes the value of zero. From that point on, whenever any new replies targeting this topic are added to the Replies table, the Revision value of this very topic should be incremented by 1. By employing this small technique, it will enable the forum reader application to keep track of whether a particular topic has had any new replies since it was cached previously.

Replies Table

The Replies table stores the replies to existing topics in the Topics table. The ReplyID field is the primary key identifying each reply, and the TopicID field is a foreign key referencing the target topic. The PostedBy field stores the user account ID of the sender of a particular reply. It is also a foreign key referencing the relevant user account in the Users table. The message and datePosted fields have identical meaning to that of the corresponding fields in the Topics table. However, the Revision field is simply a reserved field for future use and not currently used in this sample application. The forum reader doesn't have the functionality to modify replies. However, if you want to implement this functionality, you can use the Revision field of the Replies table to keep track of the revision number of each reply.

Users Table

The Users table stores the detail of user accounts, which are created by the users and used to authenticate the users' session when they are posting topics or replies. Note that in this particular application we only authenticate the users upon topic posting and replying actions, whereas we don't authenticate in the case of viewing or replying to topics. The UserID field is the primary key identifying each user. The userName and userPassword represent the users' login name and password. The userRealName and userEmail fields are simply some additional information about the user.

Stored Procedures

All data in the online forum database are accessed through a set of stored procedures. Here's a brief summary of them:

  • getCategories : returns all category records in the Categories table.
  • getForums : returns all forum records in the Forums table.
  • getOnePageOfTopics : returns a specified page of topics.
  • getReplies : takes a TopicID value as an input parameter and returns all the replies to this topic.
  • getTopic : takes a TopicID value as input parameter and returns the entire topic record.
  • addTopic : adds a new topic to the Topics table.
  • addReply : adds a reply record to the Replies table.
  • createAccount : creates a user account record in the Users table.
  • login : authenticates the user against the user account data stored in the Users table.

Local Forum Database

The local forum database refers to the one that is inherently an integral part of the rich-client forum reader application. It's an MS Access 2000 database residing in the same directory in the client computer as the forum reader application is installed. The forum reader uses ad hoc SQL to access this local database via ADO.NET, because MS Access database doesn't support standard stored procedures. Functionally, the local forum database serves as a local cache used by the forum reader to store the forum contents grabbed from the remote online forum database via the forum Web Services. The cached forum data in the local database can persist as long as you don't delete them directly through MS Access.

One point to note is that the local database is only meant to serve the internal use of the forum reader. Deleting records from the local database directly through MS Access or by any other means would of course cause certain locally cached forum contents to go away. However, this will not harm the functionalities of the forum reader as long as you don't modify the schema or the structure of the local database.

On a related note, even though here we have chosen MS Access to implement the local database of the forum reader application, but this shouldn't be the only choice. Theoretically, you can choose whatever SQL-92 compliant databases that are supported by ADO.NET. Possible choices include Microsoft Visual FoxPro, dBase, Paradox, etc. If you want to modify the forum reader application to work with these databases, you have to change the relevant codes of the data access component to accommodate the new database you choose.

Local Forum Database Schema

Figure 5: The complete schema of the local forum database

Because the purpose of the local database is to save the retrieved forum data locally, for the most part the local database schema mirrors the online database schema only with a small number of variations. Here we only concentrate on these variations.

No primary keys in the four tables are auto numbered because their values are already predetermined by the server-side online forum database.

The Users table that originally exists in the online forum database is excluded from the local forum database. You absolutely don't want every installation of the forum reader application to have a local copy of the entire Users table. Consequently, in the local database, the Originator and the PostedBy fields of the Topics and Replies table have been redefined to store the actual user real name rather than the UserID as a foreign key. The reason is obvious - when the users browse through the locally cached forum topics and replies, we certainly want to display the real names of the people who posted the topics and replies. However, as we mentioned, the Users table doesn't exist in the local database, therefore we must store the real names along with each topic and reply.

Three new fields have been added to the Category, Topics and Replies table in the local database. They are the isObsolete field of both the Category and Topics tables, and the isViewed field of the Topics Table. Sometimes the administrator of the forum website may decide to remove a whole forum or even an entire category from the online forum database for whatever reason. However, this forum or category that is to be deleted from the online forum database may have already been cached in the local database of the forum reader application. More often than not, the user would prefer to maintain this particular local forum or category for later reference if they contain content is of interest to them. In this circumstance, we set a flag on the isObsolete field of that forum or category to indicate that its counterpart on the server-side forum database has been deleted.

Besides these variations, all other aspects of the local forum database schema are virtually identical to the online forum database.

Web Service Design

Figure 6: Web Service Architecture

The Web Service component exposes the forum contents and functionalities to the rich-client forum reader. This component has a three-layer architecture. The data layer is simply the forum database hosted on SQL Server. The data access layer consists of a data access object together with a set of stored procedures inside the forum database. The business façade layer has two separate Web Service modules, namely the forum service and the account service. These two Web Services use SOAP as the transport protocol to interact with all the forum reader clients and serve their own particular business purposes.

All the source code files of the Web Service component are contained in a single Visual Studio.NET project named ForumService.

The Data Access Layer

You can find the source codes for the data access class from the file named dataaccess.cs in the download package. The data access class (dataaccess.cs) performs two different tasks. One is to create a database connection object. The other is to build a thin layer that wraps around each store procedure and abstracts all accesses to the database.

If you look at the source codes, you will find that there is a method named GetSqlConnection() that reads the connection string from the configuration file and creates a connection object. Each of the other methods in this class simply wraps a specific stored procedure and creates a SqlCommand object with the SqlParameter properly configured.

The main purpose of the data access class is to isolate the objects belonging to the business logic layer from the details of the database access information. We don't want the Business Logic Layer to include any information on what was necessary to access the database. To keep the layers of the architecture distinct and the design clean, we would need to wrap access to the stored procedures.

The Business Logic Layer

The logic layer represents the public interface that the forum reader application interacts with. It consists of two Web Services, namely the forum service and the account service.

Forum Service

The codes for the forum service are contained in the file called Services.asmx in the download package. You can find it in the ForumService project.

The purpose of the forum service module (Services.asmx) is to actually expose the forum contents and communicate them to the rich-client forum reader. Let's first take a look at one of its Web Methods:

[WebMethod (Description="Fetch a given page
            of topics in the specified forum", EnableSession=true)]
public
            DataSet GetOnePageOfTopics(short forumID,short pageNumber, short
            pageSize,ref int totalTopics,ref int returnCode)
{
//
            Validating input parameters
if ((forumID < 1) || (pageNumber
            < 1) || (pageSize <1))
{
//return adhoc error code if
            parameters are invalid
//check the file ServiceErrors.txt for
            error description
returnCode = 1007;
return
            null;
}
//Parameters are valid.
else
{
DataSet ds =
            new DataSet();
SqlConnection conn =
            dataAccess.GetSqlConnection();
try
{
//Retrieve the topics
            from DB based on the parameters
SqlCommand cmd =
            dataAccess.GetOnePageOfTopics(conn);
cmd.Parameters["@ForumID"].Value
            = forumID;
cmd.Parameters["@PageNumber"].Value =
            pageNumber;
cmd.Parameters["@PageSize"].Value =
            pageSize;
conn.Open();
SqlDataAdapter sda = new
            SqlDataAdapter(cmd);
sda.Fill(ds,"Topics");
conn.Close();

//Here,
            returnCode will always be 0 indicating no error.
returnCode =
            Convert.ToInt32(cmd.Parameters["@RETURN_VALUE"].Value);

//return
            total number of topics in this forum
totalTopics =
            Convert.ToInt32(cmd.Parameters["@TotalTopics"].Value);
}
catch(Exception)
{
//throw
            SoapException for all unexpected errors.
throw new
            SoapException("It was not possible to retrieve the topics.",
SoapException.ServerFaultCode);
}
finally
{
//make
            sure DB connections are closed
if (conn.State ==
            ConnectionState.Open)
{
conn.Close();
}
}
//return
            the topics data to the calling client.
return
            ds;
}
}

This Web Method returns a DataSet object containing a page of topics within a particular forum that are specified by the input parameter forumID. Because topics are returned to the client in a page-by-page manner, the parameter pageNumber is used to specify which page of topics to fetch and the pageSize parameter specifies the maximum topics allowed in a page. This Web Method also returns the total number of the topics that currently exist in that particular forum, and this value is returned through the ref parameter totalTopics.

While this is all quite straightforward, there are a few points to note:

  • We have enabled session support for the Web Method with the attribute declaration [WebMethod(EnableSession=true)]. This is also the case with all other public Web Methods of the two Web Services. This is necessary because we decided to enlist ASP.NET session management to maintain state for the authenticated user. We'll examine the authentication scheme as well as the issues concerned with enabling session support for Web Services in the next section.
  • Input parameters are always validated at the beginning of the Web Method.
  • All identifiable error conditions are signified by returning ad hoc error codes to the calling client through a " ref " parameter. The meanings of these error codes are documented in the text file named ServiceErrors.txt. At the client side, when the forum reader receives such an error code after calling the forum Web Services, it identifies the error condition by checking the error code and then displays meaningful messages accordingly..
  • As for all the web methods that return a particular kind of forum data to the client, the returned data are always in the form of a .NET DataSet.

The above points represent some general principles that we conformed to when coding each Web Method of the two Web Services. Here we list all the other methods in the forum service module together with a description of their respective purposes:

  • GetCategoriesForums : returns all the records in the Category and Forums table.
  • GetOneTopicWithItsReplies : returns the details of a specific topic in the Topics table and all of its relevant replies in the Replies table.
  • AddTopic : adds a new topic to the Topics Table.
  • AddReply : adds a reply record to the Replies table.
  • IsUserAuthenticated : returns the UserID value that was previously saved in the session state when the user was authenticated.

You may have noticed that we haven't provided any Web Methods to remove or alter the forum contents. If you need these functionalities when you develop your own version of the forum reader application, you can add them to the forum service.

Account Service

The account service module deals with user account creation and handles the login/logout process. It has three Web Methods:

  • The Login method handles the user login process and authenticates the user session.
  • The CreateNewLogin method takes the user's account registration detail and creates a user account for the user. Note that after the account is created, the user is automatically logged in.
  • The Logout method let the server explicitly abandon the user's session state data, which effectively returns the user back to the unauthenticated state.

A Simple Authentication Scheme

Now we will go into more detail about how to implement a simple authentication scheme for the Web Method calls.

Whenever a particular user posts a new topic or sends a reply to the forum service, the forum service must first obtain the identity of the user who is sending this request. Then, the identity of the user (UserID) in conjunction with the new topic or reply that the user has posted are saved to the Topics or Replies table as a whole. Any unauthenticated posting request from the forum reader should be denied. This is the reason why we have to implement some sort of authentication scheme.

Because ASP.NET provides a very flexible, easy to use, and scalable session state management mechanism, we simply based our user authentication scheme on the intrinsic ASP.NET session state management facility.

The default ASP.NET session scheme relies on HTTP cookies to track user session state. A cookie is typically used to track session state for browser-based clients. In that case, an HTTP cookie is sent along with the HTTP response to the client and placed in the client browser whenever a user makes the first HTTP request during a particular session. From that point on, ASP.NET will check for the presence of this cookie in every HTTP request sent by the browser so as to identify the browser and maintain the state for it at the server. However, cookies can also be used with ASP.NET Web Services. In fact, ASP.NET will also place session cookies in every Web Service client whenever the client makes the first Web Method invocation during a session. This means that for every active Web Service client, its specific state information is also automatically maintained by ASP.NET at the server. What this means to our authentication scheme for Web Method invocations is that we could store the user identity data (such as UserID value) in the session state whenever the user has successfully completed the logon process. The presence of this user identity data in its session state could indicate that the current session of that particular user has been authenticated. Otherwise, if this identity token (UserID) doesn't exist in the user's session state, it simply means the user hasn't logged in. By checking this identity token (UserID) against the Users table, our Web Service could identify which user is making the current invocation. Now, let's take a look at the code of the Login method that handles the user login process:

[WebMethod (Description="Authenticate the user session", EnableSession=true)]
public void Login(string name, string password, ref int returnCode)
{
//Validate input parameters.
if ((name == null) || (password == null) || (name.Length > 100) ||
(name.Length ==0) || (password.Length > 100) )
{
//return adhoc error code if parameters are invalid
//check the file ServiceErrors.txt for error description
returnCode = 1018;
return;
}
SqlConnection conn = dataAccess.GetSqlConnection();
try
{
//Authenticate the user
SqlCommand cmd = dataAccess.Login(conn);
cmd.Parameters["@Name"].Value = name;
cmd.Parameters["@Password"].Value = password;
conn.Open();
cmd.ExecuteNonQuery();
conn.Close();

//Pick up the return code from the stored procedure "login",
//Possible values are 0, 1013, 1014. 0 indicating successful
//authentication. Other values indicate errors.
//check the file ServiceErrors.txt for error description.
returnCode = Convert.ToInt32(cmd.Parameters["@RETURN_VALUE"].Value);

//Authenticated. (account name and password both correct).
if (returnCode == 0)
{
//Store the UserID in the session, as a token to
//indicate the user has been authenticated.
this.Session["UserID"] = cmd.Parameters["@UserID"].Value;
}
}
catch(Exception)
{
//throw SoapException for all unexpected errors.
throw new SoapException("It was not possible to login at the moment.",
SoapException.ServerFaultCode);
}
finally
{
if (conn.State == ConnectionState.Open)
{
conn.Close();
}
}
}

This code shows that the user supplied name and password are checked against the Users table by executing a SqlCommand that wraps around the relevant stored procedure. If the name and password are correct then a value of zero together with the UserID value will be returned from the stored procedure. Upon discovering that a value of zero has been returned from the stored procedure, the UserID value is then saved to the session state. If the AddTopic or AddReply Web Method of the forum service needs to obtain a user's identity, they can simply check whether session.["UserID"] exists in the current user's session state.

By means of the simple code described above, our Web Service authentication scheme is implemented. The upside is that the code is very neat and clean thanks to the ASP.NET session state management mechanics. Moreover, this authentication scheme is also secure enough because apart from a cookie containing the ASP.NET SessionID, no actual user identity data is ever sent across the wire to the client. In addition, the cookie is only valid for the current session and will be discarded by the server when it times out. We could boldly claim that our Web Service authentication scheme is as secure as ASP.NET session management. However, because this scheme is heavily dependent upon cookie-based session management, one pitfall that we must be aware of is that we must implement the cookie container in the rich-client forum reader application and associate the cookie container with every Web Service call. If the client-side cookie container is not implemented, our entire authentication scheme simply won't work at all. In that event, to make matters worse, a new session is created on every client request, which occupies a bit of server resources until it's timed out. After all, the good news is that with .NET, implementing a cookie container is a snap. We'll examine how to implement a cookie container in the forum reader a little later. .

As a side note, if you're interested in using a cookieless mechanism for supporting sessions with Web Services, you can find an article on this topic in MSDN website, which is entitled "Using ASP.NET Session State in a Web Service". The link to this article is provided in the links section of this case study.

Enter the Forum Reader Application Design

Now that we have looked at the supporting Web Services in some detail, we can begin to dive into the centerpiece of this case study: the design of the rich-client forum reader.

Componentization of the Forum Reader

Figure 7: Component Architecture of the Forum Reader

The forum reader application is implemented as a traditional windows application using the .NET Windows Forms technology. From an object-oriented point of view, the application is composed of four distinct logical components, namely the user interface component, the business logic component, the (local) data access component and the external system integration component.

The user interface component only deals with the user interaction aspect of the application. On the one hand, it receives user requests (such as retrieving topics or sending a reply) from the user but never attempts to process these requests by itself - instead it delegates these requests to the business logic component to actually fulfill them. On the other hand, the user interface component receives the forum data that has been processed by the business logic component and then display them in the various GUI widgets.

The business logic component plays the role of an intermediary between the user interface and the underlying local data access component or the Web Service proxies. It receives various requests from the user interface and then enlists the Web Service proxies and/or the local data access component to fulfill the requests.

The local data access component abstracts access to the local database. It encapsulates data access logic of both saving data to and retrieving data from the local database.

The forum service proxy and account service proxy together comprise the external system integration component for the forum reader application. Here we label the Web Service proxies as the external system integration component in that from the vantage point inside the rich-client forum reader, the remote forum Web Service could be viewed as an external system indeed. Apart from this object-oriented methodology concept, the good news is that there's no coding required on our part to create this component because Visual Studio.NET could automatically generate the plumbing codes for any Web Service proxy.

Now with the above description, we have a general overview of the high level components of the forum reader application. The separation of the entire application logic into these four specialized components is based on classic object-oriented analysis and design methodology. This typical pattern of componentization allows us to isolate problems and group objects according to problem domain or technology area. In a nutshell, doing so enables a higher degree of object reuse and a higher level of system/application maintainability over the course of its life.

Physical Component Packages

If you look at the source code in the download package accompanying this case study, you could find that except for the external system integration component (the Web Service proxies), each of the other three components is contained in its own Visual Studio.NET project. Specifically, the user interface component is contained in the ForumClient project. The business logic component is contained in the BusinessRule project. The local data access component is contained in the LocalDataAccess project. The two Web Service proxies reside in the BusinessRule project. The ForumClient project is dependent on the BusinessRule project in order to be successfully compiled. Likewise, the BusinessRule project is dependent on the LocalDataAccess project in order to be successfully compiled.

Functional Sequence

Now let's take a brief look at an example of the functional sequence of actions that takes place among the different components. In a typical situation, the business logic component would use the forum Web Service proxies to retrieve a page of topics from the remote forum service at the request of the user interface component. After it completes the retrieval of the specified topics, it uses the local data access component to store the retrieved topics in the local database. Finally, the business logic component notifies the user interface component of the retrieved topics by firing an event. The user interface component is pre-configured to subscribe to such events sent from the business logic component. Thereby the user interface component extracts the topics data from the event argument and display them in the user interface. Below is the UML sequence diagram illustrating this process:

Figure 8: A sample of functional sequence between these components

All the data retrieval behaviors in the forum reader application follow the same pattern as shown in the above diagram. In addition, they are all implemented in the same vein. We'll cover this process in depth in the section about the business logic component.

Architectural Overview of the Overall Class Model

Here we illustrate the big picture of the overall class model of the forum reader application:

Figure 9: Overall class model of the forum reader application

Our class model design directly reflects the component architecture that is described above. From the class model diagram, we can easily distinguish to which component a particular class belongs. For example, the MainForm class, the PostNewTopicForm class, the ReplyForm class, the LoginForm class, and the CreateAccountForm class altogether comprise the entire user interface component, while the the CategoryManager class, the TopicsManager class, the RepliesManager class, and the AccountManager class form the entire business logic component. The LocalDataAccess class alone constitutes the local data access component.

Given our high level component design outlined in the preceding section, deriving such an overall class model should be intuitive. However, there're some design considerations that we should be aware of:

  • The business logic objects are factored out into four distinct classes, namely the CategoryManager class, the TopicsManager class, the RepliesManager class and the AccountManager class. We come up with such a factorization simply because we want each business logic class to deal with only one specific kind of forum data. In other words, we want all the actions relating to a particular kind of forum data be encapsulated in a single class. The name of these classes implies the specific data that these classes process. For instance, the CategoryManager class deals with retrieving the categories and forums information from the remote forum service as well as saving them to the local database.
  • The MainForm is the main user interface that is populated with the various GUI controls for the user to interact with. The MainForm also performs the role of central management in that it creates all the business logic objects and maintains references to them throughout the lifetime of the application. It delegates tasks to the specific business logic objects by calling their methods directly through these references. However, in the opposite direction, the business logic components have no direct reference to the MainForm or any other GUI objects. The business logic components have to communicate data or information to the MainForm only through firing events. Another point worth noting is that all the business logic components are singletons. There is only one instance for each business logic object existing through the lifetime of the MainForm or the application. All the business logic objects are created by the MainForm upon application start-up, thereafter they all have their lifetimes in parallel with that of the MainForm.
  • There are a few other helper classes in the application that are not included in the above class model diagram for the sake of clear illustration.

User Interface Design

Figure 10: The main user interface of the forum reader application

Laying out the GUI widgets

The forum reader application has a main user interface that loosely resembles the one used in Microsoft Outlook. At first sight, we could see from the above screenshot that the main form features an OutlookBar control containing a TreeView control on the left side and the right side containing a ListView control above a RichTextBox control. The OutlookBar control is a third party offering, not from Microsoft. Here I must give credit to its developer Mr. Carlos H. Perez for providing such a fantastic control. Now let's examine how these various controls are laid out on the main form in this fashion. The technique for it is to partition the screen real estate into several panes using Panel controls and then place Splitter controls on the border between the adjacent panes that needs resizing. The Splitter controls allow independent resizing of the other bordering controls.

A long and narrow Panel named topPanel is docked horizontally against the top edge of the main form. Immediately under it, a Panel named outlookBarPanel is docked on the left side. Another Panel called rightPanel fills the entire right side. The topPanel contains a Label control to display the information about the particular forum being viewed. The outlookBarPanel is filled with the outlookBar control, and this outlookBar control in turn hosts two TreeView controls that are used to display the categories and forums information for forum navigation. The TreeView control named onlineForumTreeView is used to display the categories and forums information directly retrieved from the forum Web Service, while the TreeView control named localForumTreeView displays the same kind of data retrieved from the local database. A Splitter control is fixed between the outlookBarPanel and the rightPanel. The rightPanel contains a child panel called listViewPanel that is docked on the upper part within the rightPanel. The bottom area of the rightPanel is filled with a RichTextBox control named repliesRichTextBox. The repliesRichTextBox is used to display all the replies to a particular topic. A Splitter control is slotted between the listViewPanel and the repliesRichTextBox. Finally, a ToolBar control (topicNavToolBar) together with a ListView control (topicListView) on top of it is positioned inside the listViewPanel. The ListView control is used to display the topic list and the ToolBar below it is used to navigate topics in a page-by-page fashion. This is all it takes to craft our Microsoft Outlook-like user interface.

Other UI-related classes

Besides the main user interface, there are several other standalone forms that are displayed on an ad hoc basis:

  • The LoginForm is used to input the account name and password for the login process.
  • The CreateAccount form is used to input user account registration information.
  • The NewTopicForm and ReplyForm are used to input new topics and reply to existing topics respectively.

Another important aspect of the user interface is the context menu. Our forum reader application relies heavily upon contextual menus for smooth user interaction. For example, the onlineForumTreeView, the localForumTreeView and the topicListView all have their own specific contextual menus for performing the relevant tasks.

Examining GUI behaviors

Now let's give the forum reader a test run to get a taste of its GUI behaviors. When the application is launched, by default the left side TreeView control displays the available categories and forums cached in the local database. Right clicking anywhere in the TreeView control brings up a context menu containing the menu item Refresh Category. Click this menu item and after a few seconds, the available categories and forums are displayed in the TreeView. Right Click on the TreeView control again, this time choose Latest Topics and in a moment a list of topics are displayed in the ListView control. Right click any topic in the ListView control and choose GetReplies in the context menu, then the relevant replies are displayed in the RichTextBox control. These are some typical behaviors of the main user interface. I leave the rest of them for you to explore.

Marshalling Cross-Thread Function Call

A principle of Windows Programming dictates that all function calls to a GUI control must occur in its creation thread. This implies we should not operate a window control from anywhere other than its creation thread. The reason why this principle should be complied with is to do with the inner workings of native W32 windows, especially the single-threaded apartment memory model. However, this is a very complex issue and explaining the rationale behind it with just a few words is almost impossible here. Typically, it requires at least a few dedicated book chapters to fully explain the related concepts. So here, we just take it for granted that this principle represents a doctrine or dogma of windows programming. For further information about this issue, you can follow the relevant links at the bottom of this case study. You can also visit the MSDN website to get more information about threading and windows programming.

Usually, multithreaded programming is associated with user interface-based applications that need to perform a time-consuming operation without affecting the end user. In our case, the forum reader calls the remote Web Services across the internet to retrieve forum contents. This is, by nature, a time-consuming operation. So, in the implementation of the business logic classes, we create new worker threads to perform the Web Service calls. . When these business logic objects finish retrieving data from the remote Web Services, they would fire events to send data or message to the Mainform for display. However, when the MainForm picks up the data or message from the event argument in order to display them in a particular control, it has no knowledge that the event is actually fired from a worker thread that is different from the main GUI thread. In this case, due to the above-mentioned windows programming principle, the event handler must marshal the function call to any GUI control from the worker thread to the main UI thread.

To give an example to demonstrate this trick, suppose we have a helper method that displays text in a StatusBarPanel like this:

//This method is not thread-safe when making cross-thread call.
Private void ShowStatusBarInfo(StatusBarPanel statusBarPanel,string info)
{
statusBarPanel.text = info;
}

However, if this method is to be called from another method running on a different thread, then according to the above-mentioned windows programming principle, it cannot be called directly without the necessary marshalling. We must transform the original method to incorporate the necessary marshalling codes. The following revised code does this work with the help of a delegate:

//
delegate void ShowStatusBarInfoDelegate(StatusBarPanel statusBarPanel,string info);

private void ShowStatusBarInfo(StatusBarPanel statusBarPanel,string info)
{
//The caller is from another thread that is different from
// the main UI thread. ('this' refers to the mainForm,which is
// of course residing on the main UI thread.
if (this.InvokeRequired == true)
{
//Create the delegate that represents this method itself.
ShowStatusBarInfoDelegate showStatusBarInfo =
new ShowStatusBarInfoDelegate(ShowStatusBarInfo);
//Calling this method itself again through the delegate and
//marshal the call to the main UI thread.
this.Invoke(showStatusBarInfo,new object[] {statusBarPanel,info});
}
//The caller is from the main UI thread.
else
{
statusBarPanel.Text = info;
}
}

Each UI class in Windows Forms (that is, every class that ultimately derives from System.Windows.Forms.Control) has a property that you can safely use from any thread to detect whether a call to the UI class is from a different thread. The name of the property is InvokeRequired, which returns true if the calling thread needs to pass control over to the creating thread before calling a method on that object. Also, every UI class has a method called Invoke that could be used to marshal the call to its own method from any other thread to its own creation thread. Put it in another way, if an object existing on a worker thread calls the Invoke method of a UI class, then the Invoke method will be executed on the main UI thread rather than on the worker thread. This is exactly the effect of marshalling a function call from a worker thread to the main UI thread. Because the invoke method requires a delegate in its parameter list, that's why we define a delegate called ShowStatusBarInfoDelegate. This delegate has exactly the same signature as the method to be called.

If you look at the source code of the MainForm class, you could see that we actually marshaled all the event handlers to the main UI thread.

Remote Data Access Component

In our forum reader application, the remote data access component consists of two Web Service proxies as shown in the figure below:

Figure 11: Web Service Proxies

Visual Studio.NET greatly simplifies the process of creating Web Service proxy classes. We just need to add the web references that point to the URLs of the Web Services to be called and Visual Studio.NET will handle the creation of the Web Service proxy classes automatically. As shown in the above diagram, we've added two web references to the BusinessRule project, namely the ForumServiceProxy and the accountServiceProxy. The ForumServiceProxy contains the class that encapsulates method calls to the forum service, whereas the accountServiceProxy encapsulates method calls to the account service. These Web Service proxies are used by the business logic objects in order to communicate with the remote Web Services.

Local Data Access Component

Figure 12: LocalDataAccess project

The Local Data Access component has only one class called LocalDataAccess, which is contained in the Visual Studio.NET project named LocalDataAccess. The LocalDataAccess class serves the purpose of accessing the local forum database. It encapsulates all the logic concerned with saving data to the local forum database as well as extracting data from it, thereby insulating the business logic classes from the details of accessing local database. For example, after the business logic objects retrieve certain forum contents from the forum Web Service, they immediately call the LocalDataAccess object to save the retrieved forum contents to the local database.

The LocalDataAccess class uses ad hoc SQL to interact with the local database via ADO.NET. Three methods in this class handle the retrieval of different kinds of forum data from the local database. They are GetCategoriesForums, GetTopics and GetReplies, which retrieve the data of categories, topics, and replies respectively. All the other methods in this class deal with saving different kinds of forum data to the local database, such as categories, topics and replies. One point worthy of note is that these methods all employ a bit of decision logic in the saving process. This decision logic is illustrated in the diagram below:

Figure 13: The decision logic employed when saving data to the local database

The idea is to check whether the data to be saved already exists in the local table. If it doesn't, then we insert it into the table. Otherwise, we only need to update the existing data if its revision value is out of sync with the data to be saved.

The Business Logic Component

In the forum reader application, the business logic component refers to the one that sits in between the data access component (either remote or local) and the user interface component. It consists of a set of classes that take orders from the GUI and delegate them to the data access components, or at other times, they transfer the results retrieved from the data access components to the GUI. In this section, we'll examine the inner workings of these business logic classes.

All the classes belonging to the business logic component are contained in the Visual Studio.NET project named "BusinessRule". You can find them in the download package.

CategoryManager Class

Figure 14: CategoryManager Class

The source code for the CategoryManager class is contained in the file named CategoryManager.cs. The responsibility of the CategoryManger class includes fetching categories and forums either from the forum Web Service (GetOnlineCategoriesForums method) or from the local database (GetLocalCategoriesForums method). The public field IsFetching is a flag used to indicate that the CategoryManager object is currently in the process of fetching data from the remote service.

Multithreaded Programming

Calling remote Web Services to fetch data can be a long-running operation. If we directly use the main UI thread to perform this task, then the user interface will temporarily freeze up while waiting for the Web Service call to complete. This is because the main UI thread has been held up and couldn't respond to user interaction until the Web Service call returns. We can get around this bad user experience by using multithreaded programming techniques. Specifically, we can call the remote Web Service in a multithreaded way in order to free up the main UI thread and keep it alive for continuous user interaction.

When making multithreaded calls to Web Services, there are two choices open to us. One is to create a new worker thread and call the Web Service synchronously on that thread. The other is to call the Web Service asynchronously. In fact, both methods are suitable for our purpose. Here, we choose the former one, which is to call the Web Service synchronously on a new thread, and it turns out this method works for our forum reader without a hitch.

Now let's look at the code for the GetOnlineCategoriesForums method and the startFetchData method of the CatetegoryManger class, which implements the multithreaded Web Service call. You can find this code in the file named CategoryManager.cs in the download package:

public void
            GetOnlineCategoriesForums()
{
//Create a new thread and
            specify startFetchData method as the code
//to be executed on the
            thread.
fetchDataThread = new Thread(new
            ThreadStart(startFetchData));

//Name the thread for debugging
            purpose.
fetchDataThread.Name="Fetch Categories";

//Check
            if the user has requested to cancel the retrieving of
//categories and forums.
if (this.AbortRequested ==
            true)
{
//Dispose of the thread.
fetchDataThread =
            null;
//Set AbortRequested to indicate no
            cancellation.
this.AbortRequested = false;

//Check if
            there are subscribers hooked up with the event.
if
            (FetchCategoriesCompleted != null)
{
//Raise the event
            indicating user's cancellation.
FetchCategoriesCompleted(this,new
            FetchEventArgs(FetchStates.Aborted,null,null));
}
//Set the
            flag to indicate the CategoryManager object is free
//to service
            new fetching request.
this.IsFetching = false;
}
//The user
            didn't request to cancel the fetching.
else
{
//Triggar the
            thread to
            execute.
fetchDataThread.Start();
}
}

private
            void startFetchData()
{
try
{
//Create the Web Service
            proxy object
ForumServiceProxy.ForumService remoteService = new
            ForumServiceProxy.ForumService();

//Attach the cookie
            container to maintain session.
remoteService.CookieContainer =
            this.CookieContainer;

//Check if user has requested to cancel
            the fetching.
if (this.AbortRequested ==
            true)
{
this.AbortRequested = false;
if
            (FetchCategoriesCompleted !=
            null)
{
FetchCategoriesCompleted(this,new
            FetchEventArgs(FetchStates.Aborted,null,null));
}
this.IsFetching
            = false;
return;
}

//Retrieve the Categories and forums
            from the Web Service.
DataSet categories =
            remoteService.GetCategoriesForums();

//Check if user has
            requested to cancel the fetching.
if (this.AbortRequested ==
            true)
{
this.AbortRequested = false;
if
            (FetchCategoriesCompleted !=
            null)
{
FetchCategoriesCompleted(this,new
            FetchEventArgs(FetchStates.Aborted,null,null));
}
this.IsFetching
            = false;
return;
}
//Save the retrieved data in the local
            database.
LocalDataAccess localDB = new
            LocalDataAccess();
localDB.SaveCategoriesForums(categories);

this.AbortRequested
            = false;
if (FetchCategoriesCompleted != null)
{
//Fire the
            event to send the retrieved data to the event
            handler
FetchCategoriesCompleted(this,new
            FetchEventArgs(FetchStates.Success,categories,null));
}
this.IsFetching
            = false;
}

catch(Exception ex)
{
this.AbortRequested
            = false;
if (FetchCategoriesCompleted != null)
            FetchCategoriesCompleted(this,
new
            FetchEventArgs(FetchStates.Exception,null,ex));
this.IsFetching =
            false;
}
}

The code might appear a bit confusing because we've included additional logic to cancel the fetching operation at the user's request. First, let's see how multithreading is put to work in this code.

The UI component calls the GetOnlineCategoriesForums method of the CategoryManager class whenever the user requests to retrieve the categories and the forums. Inside this method, what the code does is to create a new thread to perform this task. The new thread executes the startFetchData method to perform the Web Service call. Upon completion of the Web Service call, the startFetchData method fires an event encapsulating the retrieved data in the event argument. By creating new thread to do the work, the call from UI component quickly returns from the GetOnlineCategoriesForums method, without having to wait for the actual Web Service call to be completed. The MainForm is configured to subscribe to the event that is fired from the startFetchData. When the event is fired, the MainForm picks up the retrieved categories and forms from the event argument. Here we could see that the event is fired from the new thread that we created in the GetOnlineCategoriesForums method. This is the reason why cross-thread marshalling is necessary when the MainForm responds to this event.

After we get the idea of how multithreading works in the code, let's see how to implement the functionality that allows the user to cancel a pending request.

A user request in our application typically requires two steps to complete. The first step is to retrieve the data from the remote Web Service. The second step is to save the retrieved data to the local database. Because these two steps are all executed on the same thread, we would naturally think that by terminating this running thread we could effectively cancel the user request. Terminating a thread only requires the one line of code that is to call the abort method on that thread. However doing it this way has potential pitfalls. Specifically, we have no control over the exact point of execution at which the .NET runtime terminates our thread. In fact, if we call the abort method on a thread, the thread could be terminated at any point during the execution. What this means is that aborting a thread has a fair chance of leaving the running context of our application in an unstable state, for example if the internals of the .NET runtime had not performed its usual housekeeping work for our application at the time of the thread being aborted. In this event, you might encounter all sorts of strange exceptions relating to the internals of .NET runtime, such as errors with ADO.NET or errors on SOAP messaging to name but a few. Because these errors occur in the internals of .NET, we have no way to get it right from the code in our own application. The upshot of it all is that our application simply couldn't function properly after the thread has been aborted unless we restart the program again.

So attempting to cancel a user request by means of aborting its running thread is simply the wrong approach. We simply couldn't rely on .NET runtime to abort our thread safely. As a rule of thumb, after a thread is started, we have to let it run its course till it reaches its normal exit point and returns.

Due to the lack of a safe way to terminate a thread, I have to make do with a checkpoint approach. I add a flag called AbortRequested in the CategoryManager class. When the user requests to cancel the retrieval of the categories and forums, I set this flag with the value of true. Inside the GetOnlineCategoriesForums and startFetchData method, I insert the following code snippet to probe the value of AbortRequested at certain points:

if (this.AbortRequested == true)
{
fetchDataThread = null;
this.AbortRequested = false;
if (FetchCategoriesCompleted != null)
{
FetchCategoriesCompleted(this,new FetchEventArgs(FetchStates.Aborted,null,null));
}
this.IsFetching = false;
}

Whenever the AbortRequested is detected to have the value of true, we immediately fire an event indicating this situation and then exit the thread. So this code snippet serves as a checkpoint, and we scatter it in several different points in the code in order to capture the user's cancellation request as early as possible. For example, if after the data has been retrieved from the remote service the code detects that the user had actually requested to cancel this operation, then the code would immediately return without bothering to save the retrieved data to the local database.

After all, the idea of this approach is to exit the thread as quickly as possible instead of aborting a thread. While this is not an elegant and ideal approach, it at least fits the bill in our application.

TopicsManager Class

Figure 15: TopicsManager Class

The source code for the TopicManager class is contained in the file named TopicsManager.cs. The TopicsManger class has the following major responsibilities:

  • The GetOnlineTopics method retrieves forum topics from the forum Web Service.
  • The GetLocalTopics method retrieves forum topics from the local database.
  • The PostNewTopic method sends a new topic to the forum Web Service in order to be inserted in the Topics table of the online forum database.
  • The TopicManager object fires an event called FetchTopicsComplete when it finishes retrieving the topic data from the forum service. The topic data retrieved are encapsulated in the event argument, which is of type FetchEventArgs.

RepliesManager Class

Figure 16: RepliesManager Class

The source code for the RepliesManager class is contained in the file named RepliesManager.cs.

The RepliesManager class has the following major responsibilities:

  • The GetOnlineReplies method retrieves the replies to a particular topic from the forum Web Service.
  • The GetLocalReplies method retrieves the replies to a particular topic from the local database.
  • The ReplyToTopic method sends a reply message to the forum Web Service in order to be inserted in the Replies table of the online forum database.
  • The RepliesManager object fires an event called FetchRepliesComplete when it finishes retrieving the reply data. The reply data retrieved from the forum service is encapsulated in the event argument, which is of type FetchRepliesEventArgs. As a side note, the FetchRepliesEventArgs class derives from the FetchEventArgs class.

Again, the RepliesManager class makes use of the same multithreading technique as the CategoryManger class.

AccountManager Class

Figure 17: AccountManager Class

The source code for the AccountManager class is contained in the file named AccountManager.cs.

The AccountManger class has the following major responsibilities: The Login method sends the user- supplied login name and password to the Account Web Service in order to get its session authenticated by the service. If the login process is completed successfully, we set the flag IsLoggedIn to true. Other objects in the application could ascertain whether the current session has been authenticated or not by checking the IsLoggedIn flag of this class.

A subtle point worthy of note is that we need to implement the cookie container (System.Net.CookieContainer) and use it with every Web Service call, because we use cookie based session state management for our Web Service authentication scheme. Therefore, we have added a CookieContainer reference to each of the four business logic objects, and in the MainForm class there is a method named InitializeCookieContainers that is used to actually create a new CookieContainer object and assign its reference to each of the four business logic objects. Now that the business logic objects have the CookieContainer, they attach it to every Web Service call they make so that the user session is maintained across the various Web Service calls in the application.

The Logout method does the opposite thing in that it asks the Account Web Service to destroy the user's current session. Any Web Service call after logout will be processed by the server in the context of a new and unauthenticated session until the user logins again. There is also a subtle point associated with the logout mechanism that we should be aware of. The user session may be timed out by the server, however in that event the forum reader is not informed of this situation immediately, so the IsLoggedIn flag still has the value of true - which is not correct. This is not a problem, because next time when the user post a new topic or send a reply, he will be alerted and at that point the IsLoggedIn flag will be set to false.

Lastly, the CreateAccount method sends the user registration request to the Account Web Service in order to create an account for the user. Immediately after the account is created, the user's current session is automatically authenticated by the Web Service.

Object Interaction

At this point, we've covered all four logical components of the forum reader application. In this section, we'll bring these various components together and put them in action. We'll demonstrate how these various components collaborate with each other to deliver a particular functionality as a whole.

From an object oriented viewpoint, a specific functionality or task of a given application is fulfilled by the underlying sequences of object interactions. Here we take a close-up at one of the most noticeable and most significant sequence of object interactions inside the forum reader application, one that fulfils the task of grabbing a page of forum topics from the forum Web Service and displaying them in the ListView control. This one is a roundtrip in that it starts with the user hitting the Latest Topics menu item to trigger the sequence and ends with the requested forum topics eventually being displayed to the user.

Below is the UML sequence diagram that illustrates this process:

Figure 18: Sequence Diagram for retrieving the latest topics from the web service

First, let's set the scene by retrieving the categories and forms from the forum Web Service. When the forum reader is initially launched, by default, the available categories and forums are retrieved from the local database and displayed in the left-side TreeView. Now if you want to check whether there are new categories or forums added to the online forum, you can right click on the TreeView under the caption Online Forum to bring up its context menu.

Figure 19: Click Refresh Categories in the context menu of the TreeView control

Next, click the Refresh Categories menu item in the context menu. After a short period, the available categories and forums in the online forum database are displayed in the TreeView. Naturally, the forums are displayed as leaf nodes in the TreeView.

Depending on the situation, each leaf node could be in any of three distinctive states: unselected, selected or highlighted. A leaf node is in selected state when you left click on it. When you right click on a leaf node, it will turn into highlighted state. We differentiate between these two states because the users tend to expect that the actions triggered by the context menu items are normally orientated at the highlighted leaf node rather than the selected one. Because the TreeView control already has a property called SelectedNode, we just need to add the highlightedNode field to the MainForm class to track the currently highlighted node. Besides these node state differentiations, the class type of the TreeNode in the TreeView has also been tweaked to support our functional purpose. Specifically, we derived our own type of category TreeNode called CategoryTreeNode in order to attach the CategoryID and CategoryName data to the TreeNode. Similarly, we derive our own ForumTreeNode type to include the ForumID and ForumName data.

Additionally, two public fields in the TopicsManager object, namely CurrentTopicsPage and TopicsPageToFetch, play an important role in the topics retrieving procedure. They are all of type TopicsPageInfo, which encapsulates information about a particular page of topics. The TopicsPageToFetch is used to provide a temporary record of the particular page of topics that is to be fetched. The CurrentTopicsPage is used to store the information about the page of topics that is currently displayed in the ListView.

Now that we've set the scene, let's dive in a bit further.

Right click a leaf node representing a forum in the TreeView and then click Latest Topics in the context menu that follows:

Figure 20: Click Latest Topics in the Context menu of the TreeView

At this moment, the onlineTopicsMenuItem_Click event handler is invoked and it in turn calls the getOnlineLatestTopics method. Inside the getOnlineLatestTopics method we first make sure that the topicsManager object is not already engaged in an ongoing fetching operation. After that, we disable all the buttons on the forum navigation toolbar except the stop button. By doing this, we ensure that the user cannot send more than one request at a time to retrieve topics from the Web Service. Next, we extract the ForumID, ForumName , CategoryID and CategoryName values from the highlighted forum node and its parent category node. With this data, we create the TopicsPageInfo object to denote the particular topic page that we intend to fetch. Here we want to fetch the Latest Topics that is page number one, so we assigned the value 1 to the pageNumber parameter in the constructor of TopicsPageInfo. Next, we delegate the request to the topicsManager object by calling its GetOnlineTopics method and thus the control is transferred.

Inside the GetOnlineTopics method, we first check whether the user has requested to cancel the request during this short moment. This is a bit fussy but we still put this checkpoint code snippet there just to play safe. Next, we spin up a new thread called Fetch Topics and execute the startFetchTopics method on that thread.

Within the startFetchTopics method, we create the forum Web Service proxy object and call its GetOnePageOfTopics method with the parameter values obtained from the TopicsPageToFetch object that we created in the previous step. The forum Web Service proxy then routes the call to the remote forum Web Service and fetches the topic data from there. The response from the forum Web Service includes the topic data that we requested together with the total number of topics that currently exist in that particular forum. As a partial result, we save the value of this total number in the TotalTopics property of the TopicsPageToFetch object. Next, we create the LocalDataAccess object and call its SaveTopics method to save the retrieved topic data to the local database. After that, we fire the event FetchTopicsCompleted with the topic data embedded in its event argument. At this point, the control of this thread leaves the TopicsManager object and passes back to the MainForm again. (Also, at this point the Web Service proxy object and the LocalDataAccess object that we created in the startFetchTopics method will go out of scope and will be garbage collected by the .NET runtime.)

The MainForm responds to the event FetchTopicsCompleted with the event handler called OnFetchOnlineTopicsCompleted. Inside this event handler, we first check if this is an event indicating the user's cancellation of a previous request. If it is, we display a note indicating this situation in the status bar. Otherwise, we invoke a helper method called FillListView to display the fetched topics in the ListView control. Finally, we restore the buttons on the forum navigation toolbar to their appropriate state in accordance with the current page being fetched.

After reading through the information above, you should have a clear view of how the functionality of retrieving topics actually works under the hood. You should also be in a good position to explore the inner workings of other functionalities in the forum reader application on your own.

Application Setup

All the source code associated with this case study is contained in a single download file named forumreader_steveshi.zip (please find in the downloads section of this case study). It is assumed that you will typically install both the forum reader application and the supporting Web Service modules on the same box for ease of experiment. The following installation procedures are described with this assumption in mind. Of course, you may just as well install them separately on two computers to simulate a real world client/server environment.

Step by Step Installation Instructions

There are three major steps to complete the installation of the entire forum reader application:

  • Install the SQL Server Database
  • Install the Web Service Modules
  • Install the Rich-Client Forum Reader

Please note before proceeding:

  • To create the database you will need administrator privileges for the local SQL Server.
  • To create the Web site you will need administrator privileges on local machine.

Unzip the Download Package

Unzip the files in the download package to a directory on your local hard disk. We'll refer to the subdirectory named sourcecode as the installation directory in the following steps.

Install the SQL Server Database

Run the forums.sql file in the databasescript subdirectory with SQL Query Analyzer. This will create the WroxForums database including all the tables, stored procedures, primary keys, foreign key constraints, unique constraints, and some sample forum data.

Install the Forum Web Services

  • Create a virtual directory named forumservice under your default website and make this virtual directory point to the forumservice subdirectory under the installation directory. Make sure this virtual directory is created as an IIS application.
  • Open the file Webconfig.web in the sub-directory forumservice with NotePad. Locate the <appSettings> section and specify the DataConnectionString key with your own SQL Server database connection information.
  • Give it a test run by pointing your web browser to http://localhost/forumservice/service.asmx and http://localhost/forumservice/accountservice.asmx. You should see the default Web Service test page that ASP.NET provides. This indicates you have successfully completed the installation of the Web Service module.

Install the Rich-Client Forum Reader

The source code of the rich-client forum reader is right there in the installation directory, just click the solution file to view it in Visual Studio.NET.

Case Study Review

In this case study, we've walked through the major design and development aspects of a rich-client forum reader application together with its supporting Web Services. Here we summarize what we have covered during the walkthrough.

We began by identifying a potential problem with traditional discussion forum websites, especially those with extremely large volumes of forum contents, and followed up with an overview of the solution that was devised. The solution entails the development of two Web Service modules and a client-side windows application called the rich-client forum reader. The Web Service modules expose online forum contents and functionalities for the rich-client forum reader to consume. The rich-client forum reader allows the end user to view online forum contents and interact with the forum functionalities by calling forum Web Services. It also stores the retrieved forum contents in a local database for offline browsing. So broadly speaking, through this case study, you learned how to expose server database contents through Web Services and how to develop rich-client windows application to consume these services.

Next, we defined all the specific functionalities that we decided to implement in the rich-client forum reader application. These functionalities include viewing online forum categories, topics, and replies as well as login/logout the forum, posting new topics and replying to existing topics. In addition, the application must enable the user to view the locally stored forum contents.

When we started to explore the actual design and development, the first thing we saw was the database schema of both the online forum database and the client side local forum database. We explained the purpose and meaning of each table and field as well as the foreign key constraints, unique name constraints and the stored procedures. So, you were presented with a precise view of the data that the application deals with.

Next, we looked at the design of the supporting Web Services. The Web Services were designed to conform to a typical three-layer architecture, with the forum database being the data layer and the Web Service module itself containing a data access layer and a business façade layer. As this is a widely adopted architecture for Web Service design, so you can apply it to your own Web Service development projects. We also took a detailed look at the authentication scheme used in the Web Services. We leveraged intrinsic ASP.NET session state management to authenticate Web Method calls.

Next, we drilled down into the details of the design and development of the forum reader application. Our design is both component oriented and object oriented. We illustrated how the application logic is separated into four specialized components, each dealing with one particular aspect of the application. Specifically, these components are the local data access component, the business logic component, the user interface component, and the external system integration component. We then took a detailed look at the overall class model of the application. As you have seen, the class model design directly reflects the model of the component architecture. By examining this classic method of componentization and the overall class model, you could strength your knowledge about object oriented design and analysis methodology.

Next, we explained how the various GUI widgets in the main user interface are laid out. As you have seen, we used several panel controls and splitter controls to model our user interface after the style of Microsoft Outlook. We then looked at a specific UI-related coding technique that is used to marshal cross-thread function call.

We then took a look at the local data access class and illustrated the decision logic that is used when saving forum data to the local database. We use MS Access database to implement the local cache of forum data and use ADO.NET to access it.

Next, we peeked inside the four classes that make up the business logic component. We looked at how and why the entire business logic is factored out into these four separated classes, namely the CategoryManager class, the TopicsManager class, the RepliesManager class, and the AccountManager class. We then described the functional purposes of these classes. We also examined how to take advantage of multithreaded programming techniques to make Web Service calls. We explained the pitfalls with aborting a running thread and introduced a feasible workaround. As .NET simplifies multithreaded programming, so you could leverage it in your own development project to provide the end user with a more responsive application.

Finally, we wrapped up this case study by examining the detail procedure that involves a set of various objects interacting with each other in order to fulfill a particular piece of functionality.

Overall, with this case study you have a good chance to look at quite a lot of topics in the realm of Windows Forms and Web Service programming under .NET. You also have the chance to get your hands dirty with the entire design and development phase of a small yet complete application.

Further Work

Last but not least, this sample forum reader application represents a generic solution paradigm, in that the server exposes data and functionality using XML web services, whereas the rich-client application retrieves the exposed data and then consumes, stores, presents and does whatever it sees fit on the data in collaboration with the Web Services. Conceptually, any system that has its data being retrieved and locally consumed by multiple clients fits into this pattern.

Practically, this typical solution pattern could be applied to many similar problem domains and business scenarios of this nature. Take yourself as being a member of CSharpToday.com for example. I believe it is more than likely that when you find an appealing case study on CSharpToday.com, you would first hit "save-as" to save it to your local hard disk and then read it at your leisure. But, what happens when your CSharpToday collection of case studies and articles reaches into the thousands and your local directory become equally bloated accordingly? I guess at that moment you would thank them if the CSharpToday team gives you a kind of rich-client case study reader to help both aggregate your discrete, unordered files and sort them out into the original categories and shapes. This is just one case where a rich-client reader utility makes perfect sense. There's more examples along the same line, especially concerned with various online contents providers. What about a MSDN client reader hooked up with the MSDN website, a campus digital library reader.? Hopefully this case study lets you see the potential niche area of such applications.


Introducing Smart Device Extensions and the .NET Compact Framework

Introducing Smart Device Extensions and the .NET Compact Framework
http://www.csharptoday.com/content.asp?id=1884
Article Rating: 3.5
Published on September 16, 2002
Abstract
The Smart Device Extensions for Visual Studio .NET provides an environment for the development of .NET applications targeted at Pocket PC, Windows CE .NET, and other smart devices. In this case study, Tony Loton provides advice on installing and configuring the Smart Device Extensions Beta 1 and discusses the relationship between these Visual Studio .NET extensions and the .NET Compact Framework APIs. He offers a simple – but interesting – first application that may be deployed to a real device or to the SDE emulator and introduces some ideas for novel speech-based applications ideally suited to the smart device platform.
Case Study

The Smart Device Extensions for Visual Studio .NET provides an environment for the development of .NET applications targeted at Pocket PC, Windows CE .NET, and other smart devices. In this case study, Tony Loton provides advice on installing and configuring the Smart Device Extensions Beta 1 and discusses the relationship between these Visual Studio .NET extensions and the .NET Compact Framework APIs. He offers a simple - but interesting - first application that may be deployed to a real device or to the SDE emulator and introduces some ideas for novel speech-based applications ideally suited to the smart device platform.

Introduction

In mid-2002, Microsoft released the Smart Device Extensions for Visual Studio .NET Beta 1, incorporating the .NET Compact Framework. "Great!" I hear you say, but what are they?

  • The Smart Device Extensions for Visual Studio .NET enables developers to develop, debug, and deploy applications for devices running the .NET Compact Framework. In a nutshell, it adds smart-device-specific features to the Visual Studio .NET IDE.
  • The .NET Compact Framework is a subset of the .NET Framework that is designed to run on a smart device, providing support for managed code and XML web services. So it's a cut-down set of .NET classes biased in favor of the client-side features.

In this introductory case study we'll discuss:

  • Installation and configuration of the Smart Device Extensions Beta 1. It's a logical starting point, and you can take advantage of the fact that I've already overcome many of the difficulties that you may face.
  • How to test the installation and how to resolve deployment problems.
  • The implementation of a first example application that accesses a native device function in order to play sampled sounds.
  • My ideas for future speech-based applications targeted at the Pocket PC platform.

Before moving on I'll just state up-front how developer-friendly the Smart Device Extensions environment is in comparison with the Symbian SDK that I have experimented with earlier. For me, it's now the environment of choice for smart device application development. Keep that positive statement in your mind as motivation while you work through the installation and configuration tips presented later.

What is a Smart Device?

The term smart device refers to Personal Digital Assistants and handheld computers, like the Psion Series 5 or the Compaq iPaq. Within the last year or so, these kinds of devices have become even smarter with the incorporation of an integral mobile phone. I'm thinking here of the Nokia 9210, which I used to own, and the O2 XDA of which I am now the proud owner.

These devices fall roughly into two camps, the Symbian camp (comprising Psion, Nokia, and some other mobile phone manufacturers) and the Microsoft camp. For the purposes of this piece, you can disregard the Psion Series 5 and Nokia 9210 completely as they're Symbian devices. You can rule in any of those devices that come labeled as Pocket PC, Pocket PC 2002, Windows CE, and Pocket PC Phone Edition.

The main thing to understand about smart devices - in contrast to desktop / laptop PCs - is that they are constrained devices: constrained in terms of memory (typically 32Mb-64Mb for RAM and permanent storage), screen size (typically 320 x 240 pixels), and bandwidth (usually connected via a mobile phone, integral or external, at 9600 bps GSM or 56 kbps GPRS). These constraints are important factors to take into account when designing applications for smart devices.

Oh, and it's because of device constraints that I decided to switch my development effort from the Nokia 9210 / Symbian SDK combination to the Pocket PC / VS.NET SDE combination. The Symbian SDK that I used was the Java variant, which suffered from the unfortunate side effect that by the time the Java Virtual Machine had been loaded into the Nokia's limited memory there was no memory left to run my applications! So far so good with the .NET Compact Framework though, which seems to load onto a device with plenty of room to spare.

What's in the .NET Compact Framework?

The compact framework shares with the full .NET framework a common language runtime, assemblies and file formats (but not binary compatibility), multi language support (but currently only Visual C# and Visual Basic .NET), a common type system, and a threading model. More information can be found at ms-help://MS.VSCC/MS.MSDNVS/ms.embeddedtoolssdk/evtuv/html/etconsimilaritieswithnetframework.htm.

It does not share support for application domains, ASP .NET, COM interoperability and callback functions (but you can use platform invoke as described later), asynchronous delegates, application profiling, binary serialization, classes from the System.Data.OleDb namespace, XML schema validation and XPath queries. More information can be found at ms-help://MS.VSCC/MS.MSDNVS/ms.embeddedtoolssdk/evtuv/html/etcondifferenceswithnetframework.htm.

On the whole the compact framework is constrained in order to cope with the low memory and low bandwidth characteristics of smart devices. As I understand it, the XML schema validation is not supported due to memory and processor speed limitations. The features that have been retained from the full .NET framework are those that help to position smart devices as a client platform, thus data access will typically be via a server-side web service rather than via direct communication with the database.

The .NET Compact Framework class library supports a subset of the .NET Framework class library. It contains additional classes exclusively designed for the .NET Compact Framework which are limited (in the Beta 1 at least) to Infrared Data Association (IrDA) classes and SQL Server CE Classes. The former may be found in the System.Net namespace, and the latter in the System.Data.SqlServerCe namespace. You can find more information about the exclusive classes in help topic ms-help://MS.VSCC/MS.MSDNVS/ms.embeddedtoolssdk/evtuv/html/etconClassesExclusiveToNETCompactFramework.htm.

System Requirements

For the remainder of this case study you will need the Visual Studio .NET IDE plus the Smart Device Extensions for Visual Studio .NET Beta 1. The latter can be found included in the July 2002 MSDN CD distribution and - at the time of writing - is available by registering for the beta program at URL http://msdn.microsoft.com/vstudio/device/sdebeta.asp?WROXEMPTOKEN=618658Zezi9SpDFSZgQps5Xv6u.

Optionally you will need a real device, Pocket PC or Windows CE .NET, on which to deploy your applications. That's not mandatory because the Smart Device Extensions (from now on SDE) includes a very effective device emulator, so in theory you could write your first smart device applications without a smart device. Everything in this case study has been tested using the emulator and a real device, the device being an O2 XDA (http://www.o2.co.uk/business/productsservices/xda?WROXEMPTOKEN=618658Zezi9SpDFSZgQps5Xv6u) which generically is a Pocket PC Phone Edition.

Defining the Problem

For this case study, our initial problem statement will be:

"How to install and configure the Visual Studio .NET Smart Device Extensions, thereby allowing us to write a first smart device application using the .NET Compact Framework."

The problem will subsequently expand to incorporate:

"How do we access native device functions that are otherwise not (yet) encapsulated with .NET Compact Framework classes?"

Case Study Structure

In the Defining the Solution section I will offer advice on:

  • Installing the Smart Device Extensions and .NET Compact Framework
  • Testing the Installation
  • Resolving Deployment Problems
  • Closing the Emulator and Redeployment

In The Sample Application section I will discuss:

  • Component Design for the application, incorporating platform Native API Access
  • Component Coding of a class to play sample sounds through the device speaker
  • UI Design and UI Code for a simple form that responds to button clicks
  • Application Set-up
  • Debugging
  • Limitations and Further Work for this application, including speech synthesis and problems of playing back of .wav files

I review the case study and present some ideas for future applications in my Conclusion at the end.

Defining the Solution

There will be two aspects to my solution:

First we will look at a set of tips on installing, configuring, and testing the installation of the Smart Device Extensions. Here you will benefit from my own experiences of trying, failing, and finally succeeding in getting the whole thing up-and-running.

Second, a fun sample application that shows in particular how you can step outside the limitations of the current .NET Compact Framework by calling native device functions directly.

Installing the Smart Device Extensions and .NET Compact Framework

A single installation executable is provided with the SDE distribution, along with a README.HTML file that helps you resolve any installation and configuration problems.

I predict that your installation will either go very smoothly, or very badly. I've been in both situations, so I am able to distill my experiences - good and bad - into a list of tips that should help you to avoid the potential pitfalls. It looks more frightening than it is, and once it's all up-and-running you can trust me that it's quite effortless in use. The ordering of this list is important!

  • If you have a spare PC that is not crucial to your operations, install there first. My first installation attempts ground my PC to a halt consistently at the same point, and from then on each reboot prompted for the installation of a new hardware device - some kind of Virtual PC Application as far as I can remember - which also caused the PC to hang. Admittedly I had an odd combination of the .NET command line SDK plus Visual Studio .NET, plus a not very convincing ActiveSync installation (see later), but nothing that was not resolved by reinstalling the entire machine (including Windows XP) from scratch. Sounds a bit drastic, I know.
  • Install the Microsoft TCP Loopback adapter. This is vital if your PC is not networked and you want to deploy to the emulator as well as a real device. Instructions for installing this adapter are included in the README.HTML file, and it's not necessary if your PC is on a network. even a dial-up network!
  • Disable the Internet Connection Firewall. The README.HTML file suggests that deployment problems may occur if the Internet Connection Firewall is enabled. I can confirm that it's true, and that's probably a good reason to run on a disconnected PC using the loopback adapter.
  • Install Microsoft ActiveSync. This should be on the CD that came with your device, or you can download it from http://www.microsoft.com/mobile/pocketpc/downloads/activesync35.asp?WROXEMPTOKEN=618658Zezi9SpDFSZgQps5Xv6u. If you have a real device, make sure you can connect to it now using ActiveSync.
  • Install the Smart Device Extensions and .NET Compact Framework. This will modify your Visual Studio .NET installation.

Testing the Installation

You can check that the installation has been successful by launching the New Project dialog. You will see a new kind of project called a Smart Device Application, as shown below, but we won't create one just yet.

You will also see that the help content includes a new category called Smart Device Extensions, with various new topics.

To test the installation fully we will build, deploy, and run the Calculator application shown in the previous figure. If you click the Calculator Sample document you will find instructions on how to locate the relevant Visual Studio solution. I located the Pocket PC / C# version of the solution in directory C:\Program Files\Microsoft Visual Studio .NET\CompactFrameworkSDK\v1.0.3300\Samples\PocketPC\VC#\Calculator.

Once you've located a version of the solution - in Visual Basic if you prefer, or for Windows CE .NET - you can choose the Debug / Start without Debugging menu option to compile, deploy, and run it. A dialog will appear asking you whether to deploy to your real device (if you have one) or to the emulator:

Let's assume you'll choose the second option, Pocket PC Emulator, as you might prefer during the development phase. At this point you should see some reassuring messages in the Visual Studio .NET output window like this:

------ Build started: Project: Calculator, Configuration: Debug Pocket PC ------

Preparing resources...
Updating references...
Performing main compilation...

Build complete -- 0 errors, 0 warnings
Building satellite assemblies...
Visual Studio is ready to deploy Calculator

------ Deploy started: Project: Calculator, Configuration: Debug Pocket PC ------

Deploying to Pocket PC Emulator using Emulation
Connected to Pocket PC Emulator (Pocket PC) running on X86.
Copying files from 'C:\Program Files\Microsoft Visual Studio .NET\CompactFrameworkSDK\v1.0.3300\
Samples\PocketPC\VC#\Calculator\bin\Debug' to '\Windows'
Copying Calculator.exe
Copying netcf.cjk.ppc3.x86.cab
Launching device installation of 'netcf.cjk.ppc3.x86.cab'. Please check device screen
for further instructions...................

---------------------- Done ----------------------

Build: 1 succeeded, 0 failed, 0 skipped
Deploy: 1 succeeded, 0 failed, 0 skipped

Launching the Application

The emulator will be launched in a separate window and you will see the progress of the deployment on the device itself:

Finally you will be rewarded with the Calculator form:

Did it all work out for you? If so, you can skip the next section.

Resolving Deployment Problems

My real device is an O2 XDA, which generically is a Pocket PC Phone Edition. Deployment to that device works pretty consistently as long as it's connected. However, deployment to the emulator can be a more hit-and-miss affair and in my experience it's usually a networking problem.

Here are some tips:

  • If you previously had a network connection, connect to a network once again. even if that means establishing a dial-up connection to the Internet. Try the deployment while connected.
  • If you've never had a network connection, make sure the Microsoft TCP Lookback adapter is enabled.
  • In all cases, disable the Internet Connection Firewall.

If my tips don't help then you're on your own and it's time to search the MSDN news groups. I found an interesting posting along the lines of.

".and deployment failed until I disabled the Microsoft TCP Loopback adapter."

Strange when you consider what I've already said, but probably true in some situations, so best to try it both ways.

Closing the Emulator and Redeployment

After deploying and testing an application on the emulator it will be tempting to close down the emulator by clicking the red X in the top-right corner of the emulator window. In that case you will see the following dialog that invites you to Save Emulator State or Turn off Emulator :

Note that if you're repeatedly redeploying an application that you are developing, you do not strictly need to close down the emulator and restart it each time. I have often closed down just the application-under-test by clicking the OK button on the application form. Subsequently redeploying the application with the emulator still running seems to work fine, and it's much quicker.

The Sample Application

Having got the Smart Device Extensions environment installed and configured, we'll now create a sample application that will give you a feel for development using the .NET Compact Framework, as well as demonstrating how to break out of the limited .NET Compact Framework functionality by invoking native device functions directly.

My original idea was to create a BeatBox application that would allow rhythms to be tapped out on the Pocket PC screen, by coupling up sampled (.wav) sound effects to on-screen buttons. It's probably not a good idea to tap frantically on the screen, so I've decided to change the name from BeatBox to SoundBox. We'll still have some drum sounds, but also a range of fun sound effects like screams and gun shots. Think of it as an executive stress toy, or as something to annoy your friends and family.

Component Design

There is a serious side to this application, which stems from the fact that we cannot play sounds on the device via the .NET Compact Framework classes themselves. So to achieve this feat we'll need to access a capability of the underlying device platform.

Native API Access

The Smart Device Extensions documentation (section ms-help://MS.VSCC/MS.MSDNVS/ms.embeddedtoolssdk/evtuv/html/etconnativeapiaccess.htm in Visual Studio .NET) states:

The .NET Compact Framework provides limited support for interoperating with native APIs. It supports accessing functions in native DLLs, but does not support creating COM objects in managed code (COM interop) or interoperating with ActiveX controls.

It goes on to say:

You can, however, call an unmanaged DLL by using platform invoke (PInvoke) with . the DLLImport attribute in Visual C#. PInvoke allows managed code to call unmanaged code and accesses APIs through standard DLL entry points.

That's how we'll solve the problem of playing sounds on the device. We'll create a class to hold a DLL function, thereby exposing an entry point to a COM object. Our class, called Sound, will be defined in the next section.

I won't go into much more detail about accessing functions in native DLLs because for the purposes of this case study it was just a means to an end. I just wanted a way to play sampled sounds in my demonstration application, with a view ultimately to realizing my aim of developing a rudimentary Pocket PC speech synthesizer (mentioned later).

In any case this technique is not limited to smart device development, so the hardcore platform SDK programmers amongst you can read more about the technicalities in the .NET Framework Developer's Guide "Consuming Unmanaged DLL Functions" at ms-help://MS.VSCC/MS.MSDNVS/cpguide/html/cpconconsumingunmanageddllfunctions.htm.

Component Coding (for class "Sound")

The Smart Device Extensions documentation (section ms-help://MS.VSCC/MS.MSDNVS/ms.embeddedtoolssdk/evtuv/html/etconPlayingSoundsUsingPInvoke.htm in Visual Studio .NET help) provides some example code (but not a complete sample application) under the heading "Playing Sounds Using PInvoke". I took that example as a starting point and rationalized the two sample classes - Helpers and Sound - to the one class that follows:

using System;

We must import the following System.Runtime.InteropServices namespace:

using System.Runtime.InteropServices;

namespace SoundBox
{

public class Sound
{
public Sound()
{
}

The original example shows how we can define a set of flags to control how the sound will be played. We will combine these flags using a bitwise OR.

[Flags]
public enum SoundFlags : int
{
SND_SYNC = 0x0000,
SND_ASYNC = 0x0001,
SND_NODEFAULT = 0x0002,
SND_MEMORY = 0x0004,
SND_LOOP = 0x0008,
SND_NOSTOP = 0x0010,
SND_NOWAIT = 0x00002000,
SND_ALIAS = 0x00010000,
SND_ALIAS_ID = 0x00110000,
SND_FILENAME = 0x00020000,
SND_RESOURCE = 0x00040004
}

Here is the DllImport attribute that specifies the DLL containing our target native function:

[DllImport("coredll")]

and this is our .NET wrapper for that function:

public static extern bool PlaySound( string szSound, IntPtr hMod, SoundFlags flags );

Finally we'll allow other classes to play a sound by making a method call of the form Sound.play("mySound.wav") :

public static void play(string fileName)
{
PlaySound( fileName, IntPtr.Zero, SoundFlags.SND_FILENAME | SoundFlags.SND_ASYNC );
}

}

}

Notice how we've combined the SoundFlags according to how the sound should be played. In a nutshell, we want to play the sound directly from a .wav file (SoundFlags.SND_FILENAME) and we want the playback to be asynchronous so that the application remains responsive (SoundFlags.SND_ASYNC). Feel free to experiment with other combinations of flags.

Now we need a user interface.

User Interface Design

When creating a new Smart Device Application project a dialog appears to prompt for the kind of application you want. This was to be a Windows Application.

You get a default form, sized to the Pocket PC (or other device) screen, onto which you can drag components from the Device Controls toolbar shown next. No real difference here compared with designing forms for any .NET application, apart from the limited form size and the limited set of device controls.

For my application I dragged Button controls onto the form so that the resulting application would look like this:

That's all there is to the UI design, and now we need to connect those buttons to the sound effects they represent.

User Interface Code

As with any other forms based .NET application, Visual Studio .NET does the hard work of coupling up components to their event handlers. Just double-click a button in the design view and you'll be transported into an event handling method for the button click. Here's my implementation for the screamButton_Click() method that plays the scream.wav sampled sound (located relative to the application deployment location) via our earlier Sound class:

private void screamButton_Click(object sender, System.EventArgs e)
{
Sound.play("scream.wav");
}

Each button of the final application has a _Click() method almost identical to that one. I know that hard coding sounds to buttons is not necessarily the best way to do it, but it will suffice for our first application.

Application Setup

I've included a complete solution so that you don't have to build it yourself from scratch. It includes some sample sound effects that I've recorded. Just load the SoundBox solution into Visual Studio .NET and in the Solution Explorer you'll see something resembling the screen shot below. Besides the two source files - Sound.cs and SoundForm.cs - you can see a collection of .wav files which I had included originally by right-clicking the SoundBox project and choosing Add / Add Existing.

To build, deploy, and run the application choose the Debug / Start Without Debugging menu option. You can specify whether to deploy to a real device (if you have one) or to the emulator. I'm sure you need no further instruction on how to use this application.

It's interesting enough to run it in the emulator, but not half as much fun as walking around with a real Pocket PC directing screams and gunshots at unsuspecting friends!

Debugging

I've just invited you to try out the sample application by choosing the Debug / Start Without Debugging menu option. Previously I suggested that you test the SDE installation by loading up the Calculator application and running it in the same way.

No doubt you'll now be wondering what happens if you run a Smart Device application using the Debug / Start menu option instead. Answer: it runs the application in full debugging mode, which doesn't sound that remarkable until you discover that it should work equally well whether you choose to deploy the application to the emulator or to a real device. In my experience it does work equally well either way.

To try it out I set a breakpoint on the following line of code:

Sound.play("tom3.wav");

I then deployed the application and pressed the various buttons. Upon pressing the other buttons, the appropriate sound was played without delay, but when the button marked Tom 3 was pressed, no sound ensued. The application was temporarily suspended on the device and control transferred to the breakpoint within the Visual Studio .NET IDE, and there it stayed until I stepped over the line of code.

Limitations and Further Work

In testing the SoundBox application I've discovered an interesting anomaly relating to the playback of .wav files. I've also thought up a really interesting idea for where to take this next, by developing a low-memory Pocket PC speech synthesizer and text-to-speech engine as the basis of a new set of killer applications.

Playback of .wav files

I've noticed that when deployed on a real device - my Pocket PC - the sampled sounds are played in response to button presses regardless of the origin of the .wav files. When deployed to the emulator for the first time however, some .wav files played OK and some (for example, my speech samples) resulted in no sound whatsoever. No error message, just silence.

My limited investigations have shown that this depends on the application I used to record the samples, or more particularly on the exact format in which the samples were saved. Should it be PCM 8.000 kHz, 8 bit, Mono or CCIT A-law 22.050 kHz, 8 bit, Stereo or one of the other myriad choices? Ultimately I would guess that it depends on the sound CODECs installed on your machine, but the success of playback in the emulator seems unrelated to the success of playback in the Windows Media Player, and unrelated to the success on the device itself.

Conclusion? If my samples don't work in your emulator then try substituting some of your own. And if they don't play on your real device it's best to check first that you have program sounds enabled via this dialog:

Yes many is the time that I have turned off program sounds in a crowded office, only to be frustrated later upon discovering that the SoundBox application is not working properly.

Speech Synthesis

This is where it gets really interesting for me.

In a previous article for Java World, "Talking Java" at http://www.javaworld.com/javaworld/jw-08-2001/jw-0817-javatalk.html?WROXEMPTOKEN=618658Zezi9SpDFSZgQps5Xv6u I described the implementation of a speech synthesizer that worked by concatenating individual speech samples - called allophones - to form whole spoken words. That was really a vehicle for showing off the Java 2 Sound APIs, but it led me on to implementing a Java 1.1 version that would work even in applets and even on small Java-enabled devices such as my Nokia 9210 phone / PDA. With limited bandwidth (in the case of a speech applet) or limited memory (in the case of a Java-enabled device) there was no possibility of basing the speech synthesis on samples of whole words or sentences, so concatenation - in various combinations - of about 60 shorter allophone samples was the only viable solution.

Having proven the technique with Java, I'd now like to try the same thing on the Pocket PC. I've made a start by including a Speak button in this case study's sample application. The method for the Speak button's click event is as follows:

private void speakButton_Click(object
            sender, System.EventArgs e)
{
Sound.play("s.wav");
Sound.play("m.wav");
Sound.play("ar.wav");
Sound.play("tt.wav");
Sound.play("10ms.wav");
Sound.play("dd.wav");
Sound.play("e.wav");
Sound.play("v.wav");
Sound.play("ii.wav");
Sound.play("s.wav");
}

It shouldn't be too difficult to figure out that when you press the button the phrase " Smart Device " is spoken, with a 10ms silence between the two words. But not until you change one of the lines of code within the Sound class to read:

PlaySound( fileName, IntPtr.Zero, SoundFlags.SND_FILENAME | SoundFlags.SND_SYNC);

Rather than the original:

PlaySound( fileName, IntPtr.Zero, SoundFlags.SND_FILENAME | SoundFlags.SND_ASYNC);

The difference is that we want the speech samples to be played out synchronously, one-by-one in strict sequence, with each new sound (allophone) not interrupting the previous one. So, this time we use the SoundFlags.SND_SYNC flag. Conversely, for the sound effects and drum samples we wanted a much more responsive application that allows a button to be pressed, and a new sound played, before the old sound have finished. So previously we used the SoundFlags.SND_ASYNC flag.

To really appreciate the suitability of the allophone technique for implementing a speech capability on a constrained device I invite you to play a game. See how many new words and phrases you can make by rearranging the allophones of the phrase "Smart Device". To get you started how about "Dart", "Site", and even "Smart Mice" ?

Now try to figure out how many kilobytes would be required to store all of your new phrases in entirety as separate .wav files. I bet it's considerably more than the 63 kb combined size of the allophone .wav files.

Text-to-Speech Conversion

It's all well and good that we have a way to assemble spoken sentences from a limited set of allophones, but it's a bit cumbersome to figure out the correct allophone sequences in advance. In my conclusion, I'll discuss some novel applications for smart devices that would take information from various sources - the device calendar, incoming email messages, web pages - and speak that information to the user. Since we won't know in advance what the text will be, some form of automatic text-to-speech conversion will be required.

I can't yet provide a complete solution for that, but I can offer a starting point that you can take further if you wish. The challenge here is to take some ordinary text:

"SMART DEVICE"

And convert that text into a sequence of allophones (below) that may be replayed via our Sound.play() method:

"s|m|ar|tt|10ms|dd|e|v|ii|s|"

Simply replacing the individual letters - or groups of letters - of the input text with allophone names in the output text is a technique that has worked surprisingly well for me in the past. So, on a letter-by-letter basis, we would do some replacements like this:

text = Regex.Replace(text,
            @"\s","10ms|");

text = Regex.Replace(text,
            @"A","a|");
text = Regex.Replace(text, @"C","k|");
text =
            Regex.Replace(text, @"D","dd|");
text = Regex.Replace(text,
            @"E","e|");
text = Regex.Replace(text, @"I","i|");
text =
            Regex.Replace(text, @"M","m|");
text = Regex.Replace(text,
            @"R","r|");
text = Regex.Replace(text, @"S","s|");
text =
            Regex.Replace(text, @"T","tt|");
text = Regex.Replace(text,
            @"V","v|");

I'm using simple regular expressions there to perform the replacements. So, you'll be pleased to hear that the .NET Compact Framework supports regular expressions providing you import the System.Text.RegularExpressions namespace and include a reference to the System.Text.RegularExpressions.dll in your project. As I'm not including a tutorial on regular expressions here you'll also be pleased to hear that have I provided a separate tutorial on that subject at C#Today, "Introducing .NET Regular Expressions with C#" at http://www.csharptoday.com/content/articles/20020201.asp?WROXEMPTOKEN=618658Zezi9SpDFSZgQps5Xv6u.

With just those letter-to-allophone replacements the result would be:

"s|m|a|r|tt|10ms|dd|e|v|i|k|e"

This might not look too bad, apart for the "k" allophone. However, by common convention the "a" allophone (file a.wav) in a complete set would be sounded as in "cat", and the "i" allophone (file i.wav) would be sounded as in "pig", which is not at all what we want. To fix that problem we need two additional replacement rules that take precedence over the individual letter replacements. These new rules are:

text = Regex.Replace(text,
            @"ICE","ii|s|");
text = Regex.Replace(text,
            @"AR","ar|");

As I said, I've used this simple technique to remarkable effect in a previous project. Of course a complete text-to-speech conversion requires many more replacement rules, including some for whole words, and the ordering of those rules is vital. To get you started I've built the starter set of rules into a TextToSpeech class included in the sample project. Here is the code listing for that class:

namespace SoundBox
{

public class
            TextToSpeech
{

public static void speak( string
            text)
{
string allophones=textToSpeech(text);

Regex
            allophoneRegex=new Regex(@"(?<allophone>\w+)\|");
Match
            allophoneMatch=allophoneRegex.Match(allophones);

for ( ;
            allophoneMatch.Success; allophoneMatch =
            allophoneMatch.NextMatch())
{
Sound.play(allophoneMatch.Groups["allophone"]+".wav");
}
}

public
            static string textToSpeech(string text)
{
// Compound
            replacements

text = Regex.Replace(text,
            @"ICE","ii|s|");
text = Regex.Replace(text,
            @"AR","ar|");

// Single letter replacements
text =
            Regex.Replace(text, @"\s","10ms|");

text =
            Regex.Replace(text, @"A","a|");
text = Regex.Replace(text,
            @"C","k|");
text = Regex.Replace(text, @"D","dd|");
text =
            Regex.Replace(text, @"E","e|");
text = Regex.Replace(text,
            @"I","i|");
text = Regex.Replace(text, @"M","m|");
text =
            Regex.Replace(text, @"R","r|");
text = Regex.Replace(text,
            @"S","s|");
text = Regex.Replace(text, @"T","tt|");
text =
            Regex.Replace(text, @"V","v|");

return
            text;
}
}
}

To try out that class you just need to modify the SoundForm.speakButton_Click() method to read:

private void speakButton_Click(object sender, System.EventArgs e)
{
TextToSpeech.speak("SMART DEVICE");
}

If that idea turns you on, don't forget to check out the separate regular expressions tutorial that I've mentioned.

.NET Compact Framework Wish List

I'm really interested in writing smart device applications that can take advantage of the integrated phone capabilities on my device. Though you can issue HTTP requests and collect the responses using classes of the System.Net namespace, there appears to be no obvious way of sending and receiving SMS messages, sending and receiving email messages, or simply placing telephone calls using the .NET Compact Framework classes.

From what I've read, it should be possible to achieve those things using the Pocket PC 2002 SDK - if I can motivate myself to write some C++ code. Alternatively, I suspect that the platform invoke technique introduced in this case study could be used, providing I can identify the appropriate DLL and the right function within it. In that respect, the POOM application (discussed in the next section) might be a good starting point.

But what I'd really like to see is a .NET equivalent of - for example - the JavaMail API that I've used previously.

In the spirit of the non-device-specific nature of the .NET framework these omissions may appear to be quite reasonable. After all, not all devices have mobile communication capabilities. If that's what you're thinking, my challenge is for you to find a smart device that doesn't have phone capabilities in a year's time.

Conclusion

In this introductory case study I wanted to focus on the Smart Device Extensions installation and configuration issues that I had encountered, the resolutions of which will be immediately useful to you.

To test the installation, and as a vehicle for discussing some of the possible deployment problems, I introduced you to the Calculator sample that comes bundled with the SDE distribution. The other bundled applications are worth looking at if you want to learn more about various aspects of the .NET Compact Framework.

Bundled Sample Applications

All of the bundled sample applications can be found in directory C:\Program Files\Microsoft Visual Studio .NET\CompactFrameworkSDK\v1.0.3300\Samples. At that location you will find them subdivided into directories for Pocket PC and Windows CE platforms, and further subdivided for Visual Basic and Visual C# samples. The full path to the Bubble sample for Pocket PC written in C# is therefore: C:\Program Files\Microsoft Visual Studio .NET\CompactFrameworkSDK\v1.0.3300\Samples\PocketPC\VC#\Scribbler.

You can load up any one of the samples into Visual Studio .NET by double-clicking the solution (.sln) file, and then choose Debug / Start Without Debugging to run it.

Bubble - This application demonstrates some drawing and animation features that might be a good starting point for those of you looking to write games for the platform.

Scribbler - Demonstrates a very rudimentary paint box-type application that demonstrates how to capture mouse clicks and drags.

WaitCursor - Shows how to change the state of the on-screen cursor.

StockQuote - Shows how the device can act as a client to a remote web service, which could prove to be the most common model for smart device applications. This example requires a lot more set-up than the others in order to simulate a web service provider, and it requires IIS to be running on your computer.

SoundBox Sample Application

My own sample application introduced the idea of accessing a native device function from .NET, and I only scratched the surface in terms of the technicalities and in terms of the uses for that technique. In my search for other examples, I found an interesting sample application on the GotDotNet web site at http://www.gotdotnet.com/team/netcf/samples.aspx?WROXEMPTOKEN=618658Zezi9SpDFSZgQps5Xv6u. This sample called POOM (Pocket Outlook Object Mode) " .lets you view all of the contacts on your Pocket PC, add new ones, and delete existing ones. " From a technical perspective it " .consists of a managed application built on top of the .NET Compact Framework and a native DLL written in C++ using Embedded Visual C++ 3.0 " So rather than accessing a function within an existing device DLL - as we have done in this case study - the POOM sample accesses a function in a custom-written DLL which in turn interrogates one of the device on-board applications.

Why do I find that POOM example so interesting? Well, imagine that we extracted information from the on-board calendar application and combined that with my speech synthesis idea so that up-coming appointments are spoken to the user like this:

"Tony, you have an appointment with Mr. Smith and 10 o'clock"

Now imagine that we gain access to the device InBox so that incoming email (or SMS) messages are spoken like this:

"Tony, you have a new SMS message from phone number 0123456. The message reads."

Towards the end of this case study I mentioned some messaging and telephone features that would make the .NET Compact Framework truly useful, providing for applications that make full use of the integrated phone features of many of the new devices. I'm surprised that this functionality is not yet included, given the enormous potential of messaging applications; but not too surprised because I suffered exactly the same frustrations previously in trying to develop Java messaging / phone applications for my Nokia 9210 PDA-and-phone using the Symbian SDK. Once again, no access to SMS, email, or the integrated phone.

But we do have a web access capability thanks to the System.Net classes, so finally imagine an application that extracts selected information from particular web sites and speaks that information to you. As I'm driving along with the headset plugged into my Pocket PC Phone Edition I need never take my eyes off the road or my hands off the wheel in order to discover that:

"The latest stock price for XYX industries is $100. Maybe you should sell out now!"

I'll be doing a great deal of thinking about that application in the coming months, and so might you. You could start by reading my previous article "Working with Web Data in C#" (http://www.csharptoday.com/content/articles/20020128.asp?WROXEMPTOKEN=618658Zezi9SpDFSZgQps5Xv6u) which describes how to extract data - such as stock quotes -- from live web pages. Then on the speech synthesis side you'll need a complete set of allophones and a more complete text-to-speech engine. I'm working on it.


Threads, Delegates and Queues

Threads, Delegates and Queues
http://www.csharptoday.com/content.asp?id=1865
Article Rating: 5
Published on September 9, 2002
Abstract
Multithreaded applications are tough to design and even tougher to debug, but the rewards are definitely worth the effort. In .NET we now have a new set of thread classes which change the landscape somewhat for programming veterans, and gives novices an easier footing. This easier ground still has traps for the unwary. Veterans who are accustomed to the Win32 design of posting messages to threads will find they have some work to transition that model into .NET; novices will find creating threads is easy – but threading issues such as data access conflicts or deadlocks are still present and need to be avoided. In this case study by Scot Boyd we will create a simple multi-threaded application, demonstrating different techniques of creating and using threads, and explore the design choices intended to avoid some of the problems a multi-threaded application can encounter. We will study how one thread can send messages to another, going from simple to robust techniques; first using delegates, then by using queues from the .NET Framework Classes, before finally exploring how to use the Microsoft Message Queue product.
Case Study

One of the many issues that programmers of multi-threaded applications have to address is the communication of data and events between threads. This case study will demonstrate how delegates can be used to send a simple message from thread to thread, briefly examine some of the problems you may encounter using this technique under heavy loads, and show you how to use System.Collection.Queue and the Microsoft Message Queue Service (MSMQ) to alleviate some of those problems.

Introduction

Multithreaded applications are tough to design and even tougher to debug, but the rewards are definitely worth the effort. In .NET we now have a new set of thread classes which change the landscape somewhat for programming veterans, and gives novices an easier footing. This easier ground still has traps for the unwary. Veterans who are accustomed to the Win32 design of posting messages to threads will find they have some work to transition that model into .NET; novices will find creating threads is easy – but threading issues such as data access conflicts or deadlocks are still present and need to be avoided.

In this case study we will create a simple multi-threaded application, demonstrating different techniques of creating and using threads, and explore the design choices intended to avoid some of the problems a multi-threaded application can encounter. We will study how one thread can send messages to another, going from simple to robust techniques; first using delegates, then by using queues from the .NET Framework Classes, before finally exploring how to use the Microsoft Message Queue product.

System Requirements

To get the most out of this case study, the reader should already be familiar with the concepts of asynchronous communication and multithreading. We will introduce delegates and queues in .NET so prior familiarity is not required. The reader should also be aware of basic .NET concepts and the C# language.

Defining The Problem

Using multiple threads in an application can provide many benefits to the programmer. On multi-CPU machines, an application using threads will see them spread across the system CPUs, increasing overall performance. Even on a single-CPU machine, multithreading has considerable benefits - it allows you to run a lengthy section of code in the background while allowing access to the rest of your application, or to setup code which waits for the right time to act. Multithreading can keep your UI responsive and snappy, making your application respond to the user’s wishes like a sports car instead of a dump truck. Multithreading, however, is not a universal panacea; if your application exists solely to calculate pi to the nth place, then one thread will do.

Programmers who want to take advantage of multiple threads are faced with the troublesome task of managing communications between those threads. It’s inadvisable to simply call into a class being used by another thread as if the class belonged to you. Problems spring up immediately when two threads try to access the same data, or else get blocked waiting for a previous thread to finish. When two threads compete to access the same data, corruption is usually the result, and blocking a client thread for any length of time quickly degrades the performance benefits of multithreading.

The most efficient methods for communication between threads revolve around the concept of asynchronous communication, or simply messaging. By quickly sending a message to another thread (or a class being run by another thread) the client threads can get on with their business and the server thread can process its messages without waiting for class properties or methods to become unblocked. For encapsulation purists, messaging also provides a method of communication where knowledge of the destination class is unnecessary.

Case Study Structure

This case study has three major parts:

  • Delegates – How to define, create, and invoke delegates between threads to send a message.
  • Queues – Creating a managed queue and using it to send and receive messages.
  • MSMQ – Accessing the MSMQ service and using their queues to send and receive messages.

Defining The Solution

In this case study we will explore three ways to send messages: delegates, Queues from the .NET Framework, and Microsoft Message Queue Service. A delegate-based solution does not provide true messaging so much as a message-like abstraction that delivers quite a few benefits for a very low overhead. Message queues are the professional strength way to trade more overhead for the benefits of passing true messages between threads, and we will demonstrate how to use them.

For our example I’ve chosen to imitate a program used in the Interactive Television world up until fairly recently. Interactive Television (ITV) is a fusion of web and television content, which can be viewed by some set-top boxes today (including Microsoft’s WebTV and UltimateTV products). The most common use of ITV in the US is sending URLs of specially designed Web pages (called ‘triggers’) in unused areas of the video signal. The set-top box retrieves these pages from the Internet and displays them along with the video content. In order to make this happen, the broadcaster needs a program designed to send these triggers at specific times using a trigger inserter, running on the broadcaster’s local network. My sample stops short of actually contacting the inserter equipment.

Component Design and Coding

This program can be viewed as the interaction of three different types of threads. Each running thread in this application is closely linked to a particular class instance. First, we have the Form1 class, our basic UI. It is run on a thread created by the Application.Run() call found in the Main() function. This thread spawns the other two types of threads in our application, using the ServerThread and ClientThread classes. ServerThread is our main ‘worker’ thread, i.e. it does the work. ClientThread is used as a delay timer, taking data intended for ServerThread and waiting for the right time to deliver that data.

In this case study we only begin to explore threading in .NET using the Thread class, and basic synchronization using the lock () statement. There are more sophisticated structures a programmer comfortable with threading basics will want to explore in .NET - the ThreadPool class which allows you to spawn multiple threads and manages their performance on your behalf, and synchronization classes including the Monitor, Mutex, and ReaderWriterLock classes. All of the above classes reside in the System.Threading namespace.

Version 1 – Delegate Based Communication Between Threads

The following sections describe how we design an initial version of our ITV Trigger application using a delegate to communicate between ClientThread and ServerThread. Delegates provide a quick, easily encapsulated, and strongly typed way to pass messages between classes unknown to each other, and only incur a low overhead. This structure uses the same form as version 2, and is outlined in the UI Design and Coding section.

Defining ServerThread

Let’s look at the definition of ServerThread.DoWork(), where the main work of the ServerThread class is accomplished. All the code shown in this case study is included with the sample projects available for download.

public void
            DoWork()
{
while(bGo)
{
Console.Write(DateTime.Now.Second);

//
            check trigger state
if (aCount >= 0)
{
string
            trig;
lock(this) // secure the data for this thread
{
trig
            = aTriggers[aCount--];
}
// send latest trigger if
            possible
Console.WriteLine(trig);
}

// sleep a bit -
            simulate transmission delay between app and ITV
            inserter
System.Threading.Thread.Sleep(1000);
}
}

Because DoWork() is called on a new thread, we need some mechanism to ‘trap’ the thread into continuing to work until we’re ready to shut down. When the thread exits the procedure, it will die. So, the first order of business is a while loop we can easily control. In this case, the thread continues looping as long as bGo, a private property in our class, is true. A public Stop() method is used to set bGo to false when we want to shut down this thread.

The Console.Write call at the top of the loop is a simple diagnostic to show that our server thread is ticking over. The next two lines are where the real action occurs. aTriggers is an array of strings designed to hold pending ITV triggers. aCount is the number of waiting triggers minus one, so it can do double duty as an index into the array. If a trigger is waiting, we put in a temporary variable, then show it to the console. This is where a real ITV trigger inserter would use Windows Sockets to activate a connection with the inserter hardware and broadcast the trigger. That would take time, so we use a call to Sleep() to simulate one second of delay. One second is the time it takes to contact an ITV trigger inserter mechanism on the broadcaster’s local network and insert the trigger into the video data. After we wake up, we take another turn around the block, unless bGo has been changed to false.

Note that DoWork() is not a static function. This is because ServerThread holds its own working data, and the simplest way to do that is to create properties to hold the data. Where do we get the data? ServerThread has a function named SendTrigger, which is called by other threads:

public int SendTrigger(String sTrigger)
{
if (aCount >= 9)
return 10; // Not throwing exception because it's not the caller's fault. Probably

lock(this)
{
aTriggers[++aCount] = sTrigger;
}
return aCount;
}

Our array of triggers is only ten elements wide. This is because it takes a second to send a trigger, so if we have ten triggers waiting we are ten seconds behind in transmitting them. So our first step is to make sure we don’t overflow the array, by checking aCount. You could throw an exception here, but I like to use them only when absolutely necessary since they are very disruptive to program flow and this problem was not really caused by the calling thread. If we have room we quickly store the data and return.

Why don’t we just use the calling thread to contact the ITV inserter? Why are we caching the data and releasing the thread so quickly? The answer to these two questions illustrates the first problem a multi-threaded application is likely to run into: blocking calling threads.

You may have noticed a block in both of these methods within a lock () statement. Lock () is used for synchronized access to objects. Synchronized access simply means that only one thread at a time can access the protected data. Because the thread in both cases is using an instance of a class, we use this to close off access to all members of the ServerThread instance. Access is restricted to the current thread until the end of the code block.

Without these locks you run the risk of encountering a class of bug called a race condition. A race condition occurs when two threads access the same data at the same time - causing a ‘race’ to see which thread wins – and usually you end up with data corruption. Race conditions are the front-runner for the Worst Bug Ever Award. They are often intermittent, making them hard to detect in a controlled test; the effect of a single race condition can range from mild to disastrous in any given encounter; and if you can’t find the bug, the only remedy is to rewrite sections of code hoping to fix it by accident.

A lock (any data synchronization) can cause two other classes of bugs: a blocked thread or a deadlock. A thread becomes blocked when it’s waiting for another thread to release a lock before it can proceed. This kind of bug doesn’t cause data corruption, but it induces a drag on performance that cancels out some of the benefits of using a thread in the first place. In the worst case, a block can last indefinitely, preventing your application from finishing. A deadlock is actually deadly – two threads become blocked indefinitely by each other, and neither can continue.

public string strA;
public string
            strB;

public void
            Forward()
{
lock(strA)
{
Console.WriteLine("Forward has
            A");
Thread.Sleep(1000);
lock(strB)
{
Console.WriteLine("Forward has
            B");
}
}
}

public void
            Reverse()
{
lock(strB)
{
Console.WriteLine("Reverse has
            B");
Thread.Sleep(1000);
lock(strA)
{
Console.WriteLine("Reverse has
            A");
}
}
}

In the sample code above, I’ve contrived a situation where a deadlock is sure to occur. In a situation where one thread calls Forward within one second of another thread calling Reverse, the two threads will run headlong into each other. This is because they’ve each grabbed a lock the other one needs in order to continue.

The advice on how to fix this situation bring to mind the joke where a man tells a doctor "It hurts when I do this", and the doctor replies "So don’t do that." There are a couple of ways to fix this situation: don’t nest lock () statements, or take your locks in the same order. Another approach is to try and take locks one a time (this isn’t a useful option if you’re handling transactions) or to put your thread-accessible data in one class and then lock that ‘master’ object instead of all the data objects separately. As a final option, you can roll your own locks using the System.Threading.Monitor class.

Above all, to avoid blocked threads and deadlocks, try to keep your lock () statements as short and as fast as possible. You still want them to encompass all the data accesses that are related to each other – transaction programmers will recognize this as the concept of Atomicity.

Creating ServerThread

// The next two lines create an instance of ServerThread (so we can store pent-up Triggers)
// and create a Thread for that instance
instance = new ServerThread();
Thread workThread = new Thread(new ThreadStart(instance.DoWork));

// Now we start our thread before popping up the Form
workThread.Start();

Calling a new thread is a three step process. First you create a ThreadStart object. The ThreadStart object needs a function when you construct it. This is the function that will eventually be called when the thread is started. Next you create a thread, passing your ThreadStart object in the constructor. Finally, when you are ready to kick off your thread, you call Start () on the Thread object you’ve created.

It’s important to note that the function you pass to ThreadStart cannot have any parameters. This isn’t a problem for ServerThread, but it will become a problem later.

Shutting Down ServerThread

In order to provide for the orderly shutdown of the ServerThread, I have added a public method Stop () to the class. This function sets an internal Boolean value bGo to false; every time the thread in the DoWork () method finishes the while loop, it checks the status of bGo to see if it should continue.

public bool Stop()
{
bool retval =
            bGo;
bGo = false;
return retval;
}

For this function, we don’t use the lock on bGo because even in a race condition, both threads will be setting it to false and both threads want the value to be false. Retval is local to the calling thread’s memory, so a second calling thread won’t switch another thread’s return value before the function returns. It would be possible for two threads, calling Stop () at the same time, to both get return values of true. This indicates to both that the thread was running, which is true.

Another way to stop a thread in progress is to call the Abort () function on the Thread class. This raises a ThreadAbortException in the working thread. By implementing Stop () rather than expecting clients to call Abort (), I decide when the thread stops executing and can provide for a more orderly shutdown than an exception handler might be able to provide. If the Thread class was not declared as sealed (cannot be derived) I would probably have used Thread as my base class and overridden Abort () to provide an orderly shutdown.

Calling ServerThread.SendTrigger()

Delegates (defined as System.Delegate) are an interesting feature of .NET. One way to describe them is as ‘object oriented function pointers.’ Another description is ‘type safe callbacks’. They are classes that are declared by programmers to call a function of a particular type, and only that type. In a sense, when you declare a delegate you are really giving a type definition to a function call. Once a delegate has been defined in this way, an instance of it can be created which refers to a function of the proper type. Consider this definition:

public delegate int SendTriggerDelegate(String sTrigger);

Now take a look at the definition of SendTrigger above. I’ll wait.

Even though the delegate is named after the procedure I intend to use it with, a delegate with this definition will bind to any procedure that takes a String as its only parameter and returns an integer. Creating the delegate is straightforward:

ServerThread instance = new
            ServerThread();
SendTriggerDelegate d;
d = new
            SendTriggerDelegate(instance.SendTrigger);

Now, any time I want to call SendTrigger () on this instance of ServerThread, I can write the following code:

d("http://www.wrox.com/show/title.htm");

More importantly, I have declared a simple short type that can be used to send triggers to any destination object I create. I can give this object to anything that understands the SendTriggerDelegate type, and they can call ServerThread even if they don’t understand the ServerThread type. Only the class that creates the new delegate needs to know the type of the destination class.

ClientThread accepts a delegate rather than a pointer to the ServerThread class for this reason. It gives me a layer of indirection between the two classes that is similar to messaging.

Delegates can serve many purposes in .NET. For instance, delegates are used to indicate functions to be run by the ThreadPool class, and multicast delegates are used to deliver local events (the EventHandler class is a delegate class). Our use of delegates to isolate the ServerThread class is just a single example of the utility of delegates.

Defining ClientThread

Once ClientThread is started, the work of the thread is incredibly simple:

public static void SendTriggerAtTime(SendTriggerDelegate funcd, DateTime dtTime, String sTrigger)
{
// Sleep until the trigger comes due
Thread.Sleep(dtTime - DateTime.Now);
func(sTrigger);
}

ClientThread takes a SendTriggerDelegate and waits until the time indicated by dtTime. Then it makes a call the delegate with the given trigger string. Unlike ServerThread.DoWork(), this function is marked static, so that it doesn’t require an instance of the ClientThread class to run. ClientThread doesn’t really need instance data in this case as long as I can pass parameters, and because there is no ceiling on the number of ClientThread threads this saves a little overhead.

Starting ClientThread

Now we hit a snag. Thread.Start() calls a procedure given to the ThreadStart object’s constructor – and that procedure can have no parameters. So what do I do when I want to start a function with parameters? It turns out the folks at Microsoft have a shell game to get around this limitation. Let’s play along and see if we can keep our eye on the ball.

First, how do we give parameters to a procedure that can’t accept parameters? We can create an object, set a few properties, and call a parameterless function that will refer back to those properties. Let’s look at this example:

// This class wraps the paramaters the form
            will be sending to the client thread
private class
            ClientThreadParams
{
// These three values become parameters
            to SendTriggerAtTime
public SendTriggerDelegate d1;
public
            DateTime dtTime;
public String sTrigger;

// This method is
            called by ThreadStart(), and calls the client thread
            SendTriggerAtTime in turn
public void CallClientThread()
{
ClientThread.SendTriggerAtTime(d1, dtTime,
            sTrigger);
}
}

Each parameter that SendTriggerAtTime() requires is represented by a property, and CallClientThread() uses those properties to call SendTriggerAtTime(). Using properties in this manner is impure from an object-oriented programming standpoint, mostly because the code doesn’t make it obvious where the property data is coming from.

Second, in order to start the client thread we need to create a new instance of ClientThreadParams and pass a reference to CallClientThread() to the ThreadStart constructor. For convenience, I’ve wrapped this sequence within another static function named BeginThread() :

public static void
            BeginThread(SendTriggerDelegate d, DateTime dtTime, String
            sTrigger)
{
ClientThreadParams param = new
            ClientThreadParams();
param.d1 = d;
param.dtTime =
            dtTime;
param.sTrigger = sTrigger;

Thread t = new
            Thread(new
            ThreadStart(param.CallClientThread));
t.Start();
}

Now whenever I want a thread to spawn and wait for a given time before submitting a trigger to the SendTrigger() function, I just call BeginThread() with a SendTriggerDelegate. Since the thread only manipulates private data I don’t have any synchronization worries.

Pros & Cons in this version:

  • Pros: A single function for thread access limits the number of ways we can get into trouble.
  • Pros: Locking access to aTriggers and aCount protects us from race conditions.
  • Pros: Only one lock() protects us from a deadlock.
  • Con: Use of an array for incoming data. The manner in which we use the aTriggers array makes it a LIFO (last in, first out) store. A collection type with FIFO (first in, first out) access would be preferable.
  • Con: Use of locks: Synchronizing our own data members is a fairly hardcore approach. It would be nice if we didn’t have to worry about synchronizing client threads.
  • Con: Instantiating a thread with parameters: ClientThreadParams is a little sloppy – there might be a better way to structure that class.
  • Con: Trigger data is not encapsulated enough: Right now we are using raw strings for triggers. If we want to expand the type of data we use for triggers it will require a lot of class updates to accommodate that change.

Version 2 – Using a System.Collections.Queue as a Message Receipt Mechanism

The next three sections show how a publicly exposed data member of the type System.Collections.Queue can be used as a messaging destination for a Thread. A Queue is a FIFO (first in, first out) data store that is better suited to our application, and it provides its own synchronization mechanism for clients to use. Arrays can be used in a FIFO manner, but we would have to write our own synchronized access functions that the Queue class provides for free.

This structure uses the same form as version 1, and is outlined in the UI Design and Coding section.

Adding a Queue to ServerThread

Instead of the string array (aTriggers) and array size variable (aCount) we used earlier, we declare a variable named MsgQueue of the Systems.Collection.Queue type. Note that unlike an array, the Queue does not need to be declared in conjunction with the type it is designed to hold.

public Queue MsgQueue; // holds TriggerMsg objects

Modifying DoWork () is relatively simple. Inside the while loop, instead of checking the value of aCount to see if we have messages waiting, we check the Count property of our MsgQueue object. Note that we do this without synchronizing thread access. Since we are the only thread dequeuing messages, the only race condition we have to worry about is if Count is being incremented as we are accessing it. In that case, if we get a zero result we will miss the new message until we come around the loop again – that doesn’t present a problem. If we do get a positive count, then we will still be okay as long as we call Dequeue () in a thread safe manner. Providing, of course, that the other thread plays by the rules and uses a synchronized copy of the queue when calling Enqueue ().

In the previous version we synchronized access to our object by passing our this pointer to lock (). The Queue class provides it’s own synchronized implementation, freeing us from needing to worry about lock () ourselves. Instead we get a safe copy of our MsgQueue object by calling the Synchronized () method. Using a synchronized version of the queue is safe for every operation except enumeration (see the definition of Queue.SyncRoot in the .NET SDK for more information on enumeration and using lock () with queues).

if (MsgQueue.Count > 0)
{
// get a
            handle to a securable queue
Queue SyncQueue =
            Queue.Synchronized(MsgQueue);
TriggerMsg msg = (TriggerMsg)
            SyncQueue.Dequeue();
Console.WriteLine(msg.Trigger);
};

One last thing: because Dequeue returns an object, we have to cast that object to our message type. TriggerMsg is a simple class with a string property named, surprisingly enough, Trigger:

public class TriggerMsg
{
public String Trigger;
}

Putting Messages into the Queue : ClientThread

With SendTriggerAtTime, we swap the SendTriggerDelegate for a pointer to our destination queue, and the simple string trigger for a TriggerMsg. The message class gives us a little more type encapsulation for the parameters, and using a Queue maintains the encapsulation from ServerThread that using a delegate gives us.

public static void SendTriggerAtTime(Queue destQueue, DateTime dtTime, TriggerMsg msg)
{
// Sleep until the right time
Thread.Sleep(dtTime - DateTime.Now);

// Get a thread-safe handle to the queue before adding our message to it
Queue SyncQueue = Queue.Synchronized(destQueue);
SyncQueue.Enqueue(msg);
}

One important change is that because the ServerThread implementation handled synchronization in its SendTrigger method, the responsibility for synchronization now falls to ClientThread as we move to a queue implementation. For this reason we grab a synchronized copy of the queue before calling Enqueue () and passing the message we were given. In this case, we are relying on our clients to synchronize along with us; this is fine if we are writing the client code ourselves, but isn’t a good idea if you are writing a library or a reusable component. In that case, you’ll want to wrap Enqueue () and make sure it’s synchronized, similar to the way we wrapped the array access in the first version.

Cleaning up ClientThreadParams

In the previous version we invoked ClientThread using a class called ClientThreadParams. ClientThreadParams had public properties which had to be set before CallClientThread () was called. This isn’t absolutely the best coding practice – even great programmers can forget a detail like this. We can fix it by marking the properties private, and creating a mandatory constructor. This ensures that a programmer won’t forget to set the properties, but not that they won’t pass garbage data. Verifying the parameters is an exercise left to the reader.

private class ClientThreadParams
{
public ClientThreadParams(Queue destination, DateTime time, TriggerMsg message)
{
destQueue = destination;
dtTime = time;
msg = message;
}

// These three values become parameters to SendTriggerAtTime
private Queue destQueue;
private DateTime dtTime;
private TriggerMsg msg;

// This method is called by ThreadStart(), and calls the client thread SendTriggerAtTime in turn
public void CallClientThread()
{
ClientThread.SendTriggerAtTime(destQueue, dtTime, msg);
}
}

Oh, and I updated it to pass a Queue and a TriggerMsg instead of a delegate and a string. BeginThread (not shown) is similarly modified.

Pros & Cons in this version:

  • Pro: By using a Queue for incoming messages we service the triggers in the same order they are submitted (first in, first out).
  • Pro: By using a Queue we don’t have to worry about Thread Synchronization in ServerThread except to call Synchronize to get a safe copy of the queue.
  • Pro: By wrapping the data in the TriggerMsg class we are able to change our data format without revisiting ClientThread ’s code (aside from a recompile).
  • Pro: Removing the public data members from ClientThreadParams and providing a mandatory parameterized constructor for the class gives us a guarantee that the values will be populated before CallClientThread() is executed.
  • Con: ServerThread is trusting the client to synchronize queue access.
  • Con: We are limited to the scalability of the Queue class, which is an X factor..

Version 3 – Using a Microsoft Message Queue (MSMQ) as a Message Receipt Mechanism

These final sections detail how to transform this application to use the MSMQ product via the objects in the System.Messaging namespace. MSMQ is a robust system providing for remote access, offline storage of messages, bridging to heterogeneous networks (including UNIX, MVS, OS/2, AS/400, Unisys, and VMS systems), message priorities and a host of other features.

This structure uses an updated version of the form from the previous sections, and is outlined in the UI Design and Coding section. Be sure to check the Application Setup section for installation instructions for the Microsoft Message Queuing Services.

Messaging and MSMQ: A Brief Overview

Messaging is a communications paradigm and a programming methodology at the same time. A Message is, in its purest form, a self-contained object (or data package) sent from one source to a recipient. Most messages are sent for one of two reasons: either to advise the recipient of an event or action, or to request an action or data from the recipient. Messages should be designed to be as small as possible while still being complete. Completeness comes before size - you want to make sure the recipient receives all the information they need so that your message doesn’t require them to send other messages before taking action. Smaller messages are processed more quickly and less likely to encounter transmission errors. A message should also be independent – understandable as a single entity, so that the recipient doesn’t need to know a previous context or message in order to act on it. Lastly, a message recipient should be designed to handle messages in a random order. Just as a message should be independent, the response to any message should be independent of what messages have already been received.

Like most rules, these can be broken and the world will still keep turning. Some messages will necessarily break these ‘rules’ (especially size-wise). If you can manage to follow these rules the design and operation of your system becomes easier. When the messages are complete and the actions taken in response are independent, the system handles uncertainty surprisingly well.

Sometimes the rules must be broken, such as when you have to send data sequentially in multiple messages. Professional messaging systems (BizTalk, IBM MQ Series, etc) can compensate for this by offering Qualities of Service. A Quality of Service (QoS) is a guarantee by the messaging system about the way it will handle messages. When sequential messages are used, the system may request a queue with a QoS guaranteeing that messages will be delivered in the same order they are submitted. MSMQ offers other qualities of service, including persistence (storing the message on disk), security (e.g. authentication), and priorities.

We won’t be exploring advanced features of MSMQ in this case study. Instead, we’re just going to learn how to use a basic MSMQ queue. In order to accomplish this, we’ll explore a few of the System.Messaging classes, including the MessageQueue class which represents a local or remote MSMQ queue, and the Message class which represents the individual messages you will send or receive. Accessing the MSMQ queues are easy, but there is a cost associated with them. The queues are maintained by an unmanaged service in unmanaged memory, so your messages will be delayed as they are copied ("thunked") across the managed/unmanaged boundary. This delay might be noticeable if you use the queue heavily in a single managed process, but will not be a factor if you use the queue to send messages between processes or machines.

Enqueuing an MSMQ Message: ClientThread

In this version the SendTriggerAtTime function in ClientThread grows substantially although the basic job is pretty simple: put a TriggerMsg into the designated queue. The first step is to initialize a System.Messaging.MessageQueue object to use the queue named in the destQueue parameter. If we don’t make sure the queue exists first, we’ll get an exception (MSMQ defines over 100 reasons to throw an exception in this API, so you will encounter them quite often). That makes this piece of code our first order of business:

try
{
if
            (!MessageQueue.Exists(destQueue))
{
// Create the queue if it
            does not exist.
MessageQueue myNewPublicQueue =
MessageQueue.Create(destQueue);
}
}

The catch block isn’t shown - for this application, I just popup an error dialog (except for one handler that we’ll see later). This code is pretty straightforward - if the queue doesn’t exist, we create it. Queue names follow a particular nomenclature, depending on the type of queue you’re trying to use. There are two major queue types you will be dealing with. Public queues, which are accessible to MSMQ installations operating in a domain with an MSMQ server, follow the syntax of MachineName\QueueName. Public queues are replicated across the domain. Private queues use the syntax MachineName\Private$\QueueName. They are available on both domain and workgroup (single-machine or non-Active Directory) installations, but if they are on a domain they are not replicated. The MachineName can be specified as .\ for the local machine, or omitted entirely.

In MSMQ it doesn’t matter who creates the queue initially, as long as both the sender and receiver of messages are using the same queue address. It’s entirely possible for a sender to create a queue, put messages into the queue, and shutdown before the receiver even starts. That receiver can then pull the messages off the queue even though the client is shutdown - they are stored by MSMQ until retrieved. Messages can be set to expire while waiting for retrieval - you should check the SDK documentation for the MessageQueue.DefaultPropertiesToSend.TimeToBeReceived property for more information. By default messages never expire. Administrators can remove messages manually if necessary.

The next bit of code is pretty straightforward. Now that the queue definitely exists, we can create a new MessageQueue object and point it to our destination queue (destQueue).

try
{
// The XML formatter is default
            here
MessageQueue q = new
            MessageQueue(destQueue);
q.Send(msg);
}

There are four basic ways to determine the binary form a message will take when it’s in the queue. There are three formatters that are provided automatically in the System.Messaging namespace: XMLMessageFormatter, ActiveXMessageFormatter, and BinaryMessageFormatter. The XMLMessageFormatter is the default for a newly created MessageQueue object. It will take any managed object passed to Send () and serialize it to XML before putting it into the queue. Receive () is not so generous, as we will see later.

ActiveXMessageFormatter is used for interoperability with the ActiveX version of the MSMQ API, which serializes messages based on the IPersistStream interface from COM. The BinaryMessageFormatter renders messages to binary format instead of XML in order to provide a performance boost. The fourth method is the tried and tested do-it-yourself model where you create your own implementation of the IMessageFormatter interface.

Receiving an MSMQ Message : ServerThread

DoWork () has to undergo a few changes in order to accommodate using MSMQ. First, we have to setup the XMLMessageFormatter to receive TriggerMsg objects. While MessageQueue.Send () with the XMLMessageFormatter will serialize any type it comes in contact with, Receive () has to be primed with either the type declarations or type names of any class you expect to receive when you dequeue a message.

try
{
if (!MessageQueue.Exists(QueueName))
MessageQueue.Create(QueueName);
MsgQueue = new MessageQueue(QueueName);
XmlMessageFormatter xmlmf = new XmlMessageFormatter(new Type[]{typeof(TriggerMsg)});
MsgQueue.Formatter = xmlmf;
}

In the code above we use types directly instead of type names, specifying the TriggerMsg class as the only message payload we expect. In MSMQ, the Message is made up of dozens of properties, the data received by Send () is serialized into the Body property. The XMLMessageFormatter will automatically deserialize the Body to whatever type or types we tell it to – in this case, TriggerMsg. Note that the XMLMessageFormatter constructor takes an array of types even though we only use one in this example.

You will want to take caution using this formatter to serialize objects. If the message class is derived from many other classes, either directly or through ‘deep’ inheritance (class D inherits C inherits B, etc) your message could be much fatter than necessary. Declaring your message class separately is a safe bet in this regard.

The next major revision comes from within the while loop used to dequeue messages.

try
{
System.Messaging.Message
            TriggerMessage = MsgQueue.Receive(new
            TimeSpan(10000));
TriggerMsg msg = (TriggerMsg)
            TriggerMessage.Body;
StatusBar.Text = msg.Trigger;
}
catch
            (MessageQueueException e)
{
// Handle no message arriving in
            the queue.
if (e.MessageQueueErrorCode ==
            MessageQueueErrorCode.IOTimeout)
{
// do nothing, this is not
            fatal
}

…
}

MessageQueue.Receive () is a synchronous call but it will take a TimeSpan as a maximum amount of time to wait before returning. If Receive () does exceed its allotted time span, it will throw an exception instead of returning normally – this means Receive () pretty much needs its own try block in which to work. BeginReceive () provides an asynchronous method of retrieving Messages. This would be preferable if we were receiving messages on the Forms thread, but since we have already dedicated a thread to dequeuing messages our thread has nothing better to do than to give Receive () a second to wait. We catch the exception inside the while loop so that we can come back around and try again.

When Receive () does return properly it gives us the entire MSMQ Message and not just the payload. We access the payload through the Body property, casting it to one of the types we passed to the XMLMessageFormatter earlier. In the Version 3 form we show the trigger text on a new StatusBar provided when ServerThread is constructed, instead of the console.

Once we leave the while loop we go ahead and delete the queue as our last action:

try
{
if
            (MessageQueue.Exists(QueueName))
MessageQueue.Delete(QueueName);
}

This is not strictly necessary, it’s even inadvisable in a real-world application since you will delete any waiting messages. For the purposes of this sample it is better to clean up behind ourselves so that we don’t end up with a proliferation of dead queues. Under normal circumstances Queue names are fixed at design-time so orphaned queues are rare. Delete () will throw an exception if the queue doesn’t exist already, so we need to check that before calling Delete () to ensure an orderly shutdown.

Pros & Cons in this version:

  • Pro: Using MSMQ preserves FIFO order of delivered messages.
  • Pro: MSMQ continues to handle Thread Synchronization for us.
  • Pro: MSMQ accepts .NET classes as message types quite painlessly.
  • Pro: Using MSMQ gives us options to increase our program’s scalability beyond the capabilities of the Queue collection class.
  • Con: Using MSMQ demands careful exception handling.
  • Con: Higher program overhead accessing MSMQ queues.
  • Con: MSMQ has to be installed by System Administrators on any machine that will utilize it, and will require some configuration for a Domain.

UI Design and Coding

The form used in this case study is mainly designed to spawn the client and server threads. It will load a file as a list of triggers and the interval to wait before sending the trigger. The intervals do not have to be in order. When the Send Triggers button is pushed, the form spawns a client thread for each trigger, which then waits for the appropriate time to send that trigger to the server thread.

In the second major version of the form (the third version of our application) some controls have been added to set an MSMQ queue name and to switch between operating modes.

Form Structure, Version 1 & 2

Our initial form has three major controls: a Text box, a File Open button (marked as an ellipsis), and a Send Triggers button. The File Open button pops up a standard forms dialog that allows the user to select a file. This file is read directly into the edit box, and should consist of a number of seconds before a trigger is needed, then the trigger itself with a tab in between. Triggers and their counts are separated by newlines. Send Triggers reads the data from the edit box and spawns a thread for each to wait for the right time to send the given trigger.

SendTriggers_Click

This function takes the contents of the text box and spawns multiple client threads using the ClientThread.BeginThread() function. The delegate points to an instance of ServerThread -the delegate itself is reused since the garbage collector .NET uses will dispose of it automatically when all the client threads finish. A DateTime object is created using the static DateTime.Now() and DateTime.AddSeconds() methods together with a string-to-double conversion using the Convert class. Note that the for loop I used here iterates twice within the loop, since Triggers.GetLength() is expected to return an even number of strings (seconds and triggers). The string format is <seconds>\t<trigger>\n (e.g. "10 http://www.wrox.com/show.htm"). For more details see the Application Setup section.

private void SendTriggers_Click(object
            sender, System.EventArgs e)
{
// This creates an array of
            numbers of triggers, alternately
String[] Triggers =
            TriggerList.Text.Split(new Char[] {'\t', '\n'});

// We'll
            reuse this delegate; it will die when all the treads drop their
            references
SendTriggerDelegate d = new
            SendTriggerDelegate(instance.SendTrigger);

for (int i = 0; i
            < Triggers.GetLength(0);)
{
// Send the delegate, the time
            index from now, and the trigger
ClientThread.BeginThread(d,
            DateTime.Now.AddSeconds(Convert.ToDouble(Triggers[i++])),
            Triggers[i++]);
}
}

In Version 2, this final call to BeginThread changes slightly as we replace the delegate d with the destination queue:

ClientThread.BeginThread(destQueue, dt, msg);

Form Structure, Version 3

This new form adds three major new elements. First, it allows us to switch modes: "Client Only" mode allows you to load trigger files and send triggers to the specified queue; "Server Only" mode dequeues triggers from the specified queue; and the default "Single App" mode performs both functions just as the Version 1 & Version 2 forms did.

The next major element is the combination of the edit box (labeled "Queue:") and the Set button. The box remains read-only until the Set button is clicked, when the edit box is enabled so you can edit the name of the queue you are using. Clicking Set again resets the ServerThread, if it is running, and any subsequent ClientThread s will use this queue name as well.

Lastly I have added a Status Bar that is updated by ServerThread when a trigger is dequeued. This was an easy way to see what’s going on without the Console.

The Sample Application

The simplest way to start TDQ is to use the "Shortcut to TDQ.exe" shortcut located in the TDQ Project top level folder of the included files. This shortcut invokes the executable from the Release directory of the TDQv3 project. Any number of instances of this application can run simultaneously in Single App mode (although they shouldn’t reference the same queue when they do so!) or in a mixture of Client Only and Server Only mode. Multiple Client Only instances can send triggers to a single Server Only instance that uses the same queue.

Single App mode

The default operating mode of this application is "Single App" (single application) mode. In this mode the application runs a server thread dequeuing messages from the given queue, and as many client threads as necessary when the Send Triggers button is pressed. In order to use Send Triggers you first need to load a file specifying one or more triggers. Files are loaded by pressing the button marked with an ellipsis (…), and two files are provided: triggers.txt and stresstriggers.txt. Once a file has been loaded, the Send Triggers button becomes available. When that button is pressed, one client thread per trigger is created and that thread waits until the time for the trigger has elapsed. At that time, a message is deposited into the MSMQ queue specified. The server thread will remove messages from the same queue each second and post the trigger name to the status bar at the bottom.

The name of the queue can be changed by pressing the ‘Set’ button, modifying the queue name, and pressing ‘Set’ when finished. At this point the server thread will reset and begin dequeuing the new queue.

Client/Server Only modes

In Client Only mode, files can be loaded and the Send Triggers button will operate normally. Triggers will accumulate in the MSMQ queue unless another version of this application is running in either Single App or Server Only mode.

In Server Only mode, the ellipsis button and the Send Triggers button are disabled. At startup, a server thread is created to dequeue messages from the specified MSMQ queue. The queue name can be changed using the Set button. If the queue is changed, the server thread will reset and begin dequeuing from the new queue. When a message is dequeued, the trigger string will be displayed in the status bar.

Using the MSMQ Queue Manager

There are two simple ways you can monitor the contents of an MSMQ queue. If you are running the Visual Studio environment you may have a Server Explorer interface available (it is not included with the Standard editions of Visual Studio). Under this interface you can examine various aspects of servers you are connected into, including events, performance info, services, SQL processes, and MSMQ queues. You can select a queue to monitor by navigating to it and clicking on it in this interface.

A similar interface is available from the Control Panel. Inside the Control Panel, open the Computer Management applet and navigate through the ‘Service and Applications’ section to the ‘Message Queuing’ section, and then to the queue you want to monitor.

If you run the application in Client Only mode, load a file and press Send Trigger, you can watch messages as they accumulate in the queue using either of the methods above.

Application Setup

The sample code for all three versions of TDQ is provided in the zip file named TDQ.zip. Under the ‘TDQ Project’ directory each version of the project is provided in its own subdirectory. For convenience, a shortcut to the TDQv3 project is also provided. That project also has links to the two previous versions.

In order for TDQ version 3 to work, you will need to have MSMQ and the Microsoft .NET Framework installed and running.

Installing the Microsoft .NET Framework

These samples require the Microsoft .NET Framework to operate. To install the Microsoft .NET Framework navigate to the following URL: http://msdn.microsoft.com/downloads/default.asp?url=/downloads/sample.asp?url=/msdn-files/027/001/829/msdncompositedoc.xml&WROXEMPTOKEN=618658Zezi9SpDFSZgQps5Xv6u. You should also install the Service Pack 2 available on this same page.

Installing MSMQ

MSMQ is included with Windows 2000 and Windows XP. To install, login with Administrator credentials and then go to Control Panel. In Control Panel, click Add/Remove Programs and select the Add/Remove Windows Components button. In the Windows Components Wizard scroll down the Components box to reveal "Message Queuing Services". Check the box next to "Message Queuing Services" if it is not already checked. Click ‘Next’ and follow any instructions Setup presents. To install MSMQ 1.0 on Windows NT 4.0, Windows 98, or Windows ME, use the Windows NT Server 4.0 Option Pack: http://www.microsoft.com/ntserver/nts/downloads/recommended/NT4OptPk/default.asp.

During installation you may be presented with the choice between installing an MSMQ server, independent client, or dependant client. Servers are MSMQ controllers for an entire domain, and require Active Directory. Independent clients host their own queues and work if connected to a domain or separate. Dependant clients require an MSMQ server on the domain, and access queues stored on other machines. If in doubt, select Independent Client during setup.

NOTE: If you have MSMQ installed properly on your Windows domain, you will be able to specify queues on other machines as the destination queue in TDQv3. This scenario has not been tested.

Trigger file format

There are two trigger files included in this sample: triggers.txt and stresstriggers.txt. Triggers.txt shows a normal progression of ITV triggers, although slightly accelerated. Stresstriggers.txt fires four triggers at once in order to create a race condition for testing. The format of these files is: <#seconds><tab><trigger><newline> on each line of the file. ‘#seconds’ indicates how long until the trigger is supposed to fire, and ‘trigger’ is the text to be sent (an URL to a properly formatted website in the ITV world, although for the purpose of this sample any text will do).

Case Study Review

Programmers use threads for a number of reasons. A thread dedicated to the user interface makes that interface responsive while the real work happens on another thread. Long processes or intense calculations can benefit from their own thread as long as your application can perform other useful tasks in the meantime.

These benefits carry a price. Two or more threads accessing the same data at the same moment will cause unpredictable behaviors - bugs ranging from minor to serious. This is the worse kind of threading bug – a race condition. Sometimes impossible to duplicate, a programmer faced with fixing these bugs will begin to wonder if the benefits of threading are worth the pain of debugging.

There are two major ways for a programmer to inoculate themselves against these bugs: synchronizing data manually via the lock() command, or running objects in their own synchronization context, where the .NET Execution Engine takes over synchronization tasks at the cost of locking your entire object as any thread enters it.

In this example we see how to lock data and how to use synchronizable objects (in this case, System.Collections.Queue) safely with multiple threads. We have to keep in mind that locking up data causes another potential threading bug – the deadlock. In a deadlock situation, Thread A takes a lock while accessing data, then waits for data locked by Thread B. The problem arises if, under these circumstances, Thread B is waiting on the data that Thread A has locked. In this case both threads freeze and cannot resume (hence DEADlock). Being aware that kind of bug exists is the best way to prevent it.

One method of keeping threads from interfering with each other directly is to use messaging to communicate between them. By encapsulating data in messages you can keep threads from fighting over that data to some extent. Using messaging properly is also a good way to reduce the complexity of an application - threads that don’t care in what order messages are sent become more resilient in unpredictable situations.

What is Messaging, exactly? A Message, you’ll recall, is a self-contained object or data package sent as indication of an event or a command request. Ideally a message is as small as possible, has all the data the recipient needs, and is not dependant on knowledge of some prior state or sequence.

In our first thread example, rather than creating a message object and a receptacle for that object, we opt instead for a method abstracted by a delegate. A delegate is an object-oriented function pointer that allows you to create an object (the delegate) representing a method or static function. This delegate can be linked to any method fitting the defined profile: once attached to a method it can be passed to another object without revealing the source of the final method. This encapsulation gives you a way to mimic messaging to some extent. If the destination method is relatively stateless (making each method invocation work independently of other invocations) and the data passed with the delegate is complete, you can get a lot of the benefits of messaging with little overhead. Each declared delegate type can be treated as a message type and each instance of a delegate can be treated as the destination of a message.

If you want to move beyond delegates, the best receptacle for messages is a queue. In our second example, we defined a message type and used a System.Collections.Queue object as our message destination. Passing the queue as a destination still allows us to encapsulate the destination from the message sources. Queues also provide their own synchronization mechanism, preventing the scourge of deadlocks, and will automatically expand as message traffic increases. Sending messages to a thread using a queue is also a technique that Win32 programmers should be familiar with (ah, the glory days of PostMessage()).

Professional strength messaging offers many benefits over a managed queue. Most, but not all, of these benefits revolve around the idea of Quality of Service. A Quality of Service (QoS) is a guarantee from the messaging system about how it will handle your message. First-in first-out (FIFO) access to a queue can be considered a Quality of Service. Violating FIFO in order to service messages with a higher priority first is also a Quality of Service.

Microsoft Message Queue Service (MSMQ) is a professional messaging product included with Windows 2000 and Windows XP, accessible via the classes defined in the System.Messaging namespace. MSMQ offers message routing across a domain, priorities for messages, persistence of messages (the queue can exist even when the programs using it do not) and other qualities of service.

Using MSMQ does cost some overhead: the service runs in an unmanaged space so each call is going to cause some thunks (managed/unmanaged transitions). You will also need to pay some attention to how managed data is formatted going into and out of the queue – in our last example we used the built-in XMLFormatter class to handle this task for us as we demonstrated how to send and receive messages using an MSMQ queue.

Any Limitations or Further Work

Turning this application into a real trigger inserter would require a few extra steps. There are actually two types of triggers; Transport A (closed captioned) and Transport B (bit-streamed). For a real world application the following steps would have to be taken:

  • Transport A triggers require a computed checksum.
  • Transport A triggers allow for a display title and other optional instructions.
  • A new file format and error handling in case the file is corrupt.
  • Support for Transport B.
  • A UI for specifying an inserter to call.
  • A feedback UI for the client would be optional.


Data Transformation Engine

Data Transformation Engine
http://www.csharptoday.com/content.asp?id=1853
Article Rating: 5
Published on August 12, 2002
Abstract
Numerous software applications require data to be imported from an external source and translated into a format that the application knows how to deal with. In this case study, Catalin Tomescu explains the problems he encountered while working on a real-world project some time ago. The application was the direct need of an insurance company to import service contracts from different clients into their database. The format of the data sent by these clients was very different depending on the system they had implemented at their end. Some clients sent the data in XML format, others in CSV file, etc. Because of this, Catalin's team was forced to create a system that would allow them to import the data into the database. In this case study he’ll show you the solution to this problem. We’ll look at the framework for data transformations that was built which is capable of taking data from various sources and transforming it into a wide range of types.
Case Study

At some point in its life cycle, all software applications will need to process some kind of data. Data can represent any information that the application needs to process in order to fulfill the goal it was built for. Unless this data is internally generated by the software application itself, it must be imported from an external source and translated into a format that the application knows how to deal with.

To be more specific, we'll explain the problems we encountered while working on a project some time ago. The application we were building was a direct need of an insurance company to import service contracts from different clients into their database. The format of the data sent by these clients was very different depending on the system they had implemented at their end. Some clients agreed to send the data in XML format, others in CSV file or fixed width files, others using files with a proprietary format. Because of this, we were forced to create a system that would allow us to import the data into our database. We started to use DTS, but soon realized that as the number of clients grew, so would the range of data formats we would have to deal with. DTS was far from the perfect solution, so we had to search for another solution.

In this case study we'll show you a solution to this problem. We'll show you a framework for data transformations that we built and which is capable of taking data from various sources and transforming it into a wide range of types.

Introduction

Data transformation involves a source and one or more destinations. The format of the data in the source and destinations can be a text file (CSV, fixed width, or proprietary format), a file containing records with different format, an XML file, data existing in a database, or anything else. Turning the source data to the specific format that an application needs it in can be a painful job sometimes. Often this task requires developers to write a transformer that will make the translation from the source format to the destination format.

This is not a new problem, and this is not the first attempt to solve it. Let's examine one of the existing solutions to the problem. With version 7.0 SQL Server comes a package called Data Transformation Services that was specifically created to solve these kinds of problems. Data Transformation Services (DTS) provides a set of tools that let you extract, transform, and consolidate data from disparate sources into single or multiple destinations supported by DTS connectivity. By using DTS tools to graphically build DTS packages, or by programming a package with the DTS object model, you can create custom data movement solutions tailored to the specialized business needs of your organization. A DTS package is an organized collection of connections, DTS tasks, DTS transformations, and workflow constraints assembled either with a DTS tool or programmatically.

For most situations, DTS is sufficient, but there are situations where the problem is much more complex and DTS is not able to address it. For example, one problem is that there are no connectors to XML files or files with formats other than CSV or fixed width.

To overcome the situations when DTS can't be used, we designed and build a Data Transformation Engine that will address existing issues.

System Requirements

To compile and run the code included in this article you need:

  • Visual Studio .NET Professional or any other edition
  • SQL Server 7.0 or 2000 can be used if you want to test the database connector. You can also choose the lighter version of SQL Server, which is MSDE. MSDE is distributed free by the Microsoft Corporation and can be downloaded from their web site.
  • Any XML editor or a simple text editor for creating XML files

The case study also assumes you have a fairly good understanding of the C# language and .NET Framework. Experience with XML, OO principles and design patterns are a plus in understanding this piece.

Defining The Problem

Whether we are talking about financing solutions or engineering packages, both processes need some kind of data to be able to produce results. The driving source of information is financial statements, accounting statements, or sensor readings. Very often this data it's not available in exactly the desired format and requires some kind of transformation. Financial statements can be transmitted via the Internet in comma-separated text files, it can be extracted from a database, or it can be sent as an XML file through a web service.

As I mentioned earlier, a possible solution for this problem would be to use Data Transformation Services available as a package in SQL Server 7.0 and above. We could use DTS in this way:

  • Create a package in SQL Server that will import the specified source data into the specified destination. Each DTS package contains one or more steps that are executed when the package is run. When executed, the package connects to the correct data sources, copies data and database objects, transforms data, and notifies other users or processes of events.
  • Run the package manually or automate this task through SQL Task Scheduler.

DTS is a good choice for solving simple and medium difficulty problems. For example, the number of formats available in DTS for describing data sources and data destinations does not cover some of the new technologies like XML. Another drawback is that DTS requires SQL Server to be present on the computer in order to store and schedule DTS packages. Earlier versions of DTS were even more restrictive in respect to the tasks that were available to use in a package.

Defining The Solution

Data Transformation Engine, the framework that we designed, expands the list of possible data types for describing sources and destinations available in most of the existing solutions. In addition to the standard list (CVS, fixed width files, databases, Excel) DTE is capable of creating and reading XML files, files with multiple types of records.

What is more important about DTE is that it was designed to allow new sources and destinations to be created and plugged into the system very easily. This allows us, or any other developer, to add data sources as they are needed.

Component Design and Coding

This section will focus on explaining how the DTE framework was designed, as well as going through implementation details. The implementation is entirely done using C# language, but because .NET is a cross language environment you can choose to rewrite parts of the application in your language of choice. Similarly, extending this framework can be done in any of .NET's supported languages.

The DTE solution contains four projects:

The DTE project contains the controlling objects in the framework.

The DTECommon project contains interface definitions. We chose to separate the interface definitions from the rest of the framework implementation so that it can be used in extending the set of data sources and destinations. This way we can keep the size of the library very small. More details about this will be given in the next sections.

The DTEStdDataFactory project contains the implementation for most used data sources and destinations. More implementation details about this will be given in the next sections.

The Test project contains the source code of the example application included in this case study.

Design Decisions

When we designed the DTE we started with analyzing a transformation process. Basically what happens is that source records of data are transformed into destination records and pushed to the data destination. This is the first design decision we made for our framework:

Data sources and destination will treat data as records composed of fields.

We used this approach because the practice has proved that in most of the cases the data will be seen as records of information that will be accessed sequentially for either reading or writing.

This process is captured in the following figure:

We designed DTE starting with this simple idea. The central part of the DTE is the processing engine. The processing engine controls the flow of transformation tasks and has the following major tasks:

1.Read transformation descriptor file

2.Create source and destination(s) objects based on the information in the descriptor

3.Open source and destination(s)

4.While records exist in source

...4.1Decrypt raw data and create source record

...4.2Choose destination record

...4.3Perform data transformation using field mapping matrix

...4.4Write destination record

5.Close data source and data destination(s)

Though the overall design is good, it comes with a limitation: if a client application needs to process only a subset of the data in the source storage, DTE will still parse the whole set of source data. Implementing a filter mechanism is relatively easy with databases, but becomes more complicated with text files. DTE allows client application to select a subset of records from the source database but has no provision for filters when it comes to text files.

We mentioned in the previous sections that the DTE framework is able to deal with a wide range of sources and destinations. To achieve this, we used a key element in the design of DTE. In the following section you'll see how we made use of design patterns to achieve a flexible, extensible, and reusable set of components.

Managing Sources and Destinations

The second design decision we made was:

Use the Abstract Factory pattern to manage sources and destinations.

This allows us to maintain the same interface in the processing engine while extending the types of sources and destinations available to users. The Abstract Pattern allows the DT Engine to create instances of objects from different families of sources and destinations without specifying their concrete classes.

The architecture of the DT Engine with respect to this topic is outlined in the following UML class diagram:

The implementation of the Abstract Factory pattern uses interfaces and inheritance. Of course, each programming language has its own ways to create or simulate interfaces and deal with inheritance.

As you can see from the above figure, we defined a set of interfaces for data source, data destination and corresponding records.

From these interfaces we'll derive concrete classes for each type of source/destination we decide to implement. By default, the DTE framework comes with the following sets of hierarchies: CVS, Fixed Width Fields, Database, XML and Record Based Files. Each of these packages is contained in a separate C# namespace to be manipulated easily. We'll discuss details of their implementation later.

The breakdown in physical packages is shown in the following UML structure diagram:

The DTEApplication object is designed to use the IDataFactory interface to create data sources, data destinations, and record objects. To be able to better understand how this works, let's get into the implementation details of the DTEApplication object.

DTE application

DTEApplication is the main controlling object of the framework and it is located in the DTE project. The main purpose of this object is to control the flow of the transformation process. The transformation process can basically be broken down into three phases: initialization, transformation process, and cleanup. This can easily be seen by looking at the StartTrxProcess method of the object.

The Initialization phase consist of:

  • Read transformation descriptor file
  • Create data source and destinations
  • Assign initialization properties to source and each destination object

Client applications describe a transformation process by writing a descriptor file, which must be in XML format, and have following structure:

<?xml version="1.0"?>
<transformation>
<source mode="" name="">
<property name="Data Source"> </property>
<property name="Configuration File"> </property>
</source>

As you may already have guessed, source element describes the source of data used in the transformation process. The source element contains all the attributes that are necessary to locate and use a file, database table, or any other type of source. Mode and name attributes are mandatory and the framework will generate an error if they are not present:

  • The Mode attribute defines the type of data source. Valid values for this attribute are defined in the DataFactories section of the DTE settings file. Here is the current list: CHAR_SEPARATED_VALUES, FIXED_WIDTH, XML, DATABASE, and RECORD_BASED.
  • The Name attribute must be uniquely defined among source and destination names. This is just an identifier and doesn't need to be tied to the actual source.

The Property element defines a specific property for the data source. There are two mandatory properties for any data source: Data Source and Configuration File.

  • Data Source property can be:
  • The name and path to the source file. You can have something like this: \\machinename\folder\mysource.txt or c:\myfiles\source.txt.
  • A SELECT SQL statement that gathers records from one or more tables.
  • Anything else that describes the source of data in your specific data type. For example, I can imagine someone writing a connector to access registry information in which case this property contains a registry key.
  • Configuration File property represents the path to a file that contains the descriptor for specified data source. We chose not to include the descriptor in the same XML file so that you can reuse the source descriptor file somewhere else. The descriptor is specific for each source and we'll discuss them later.

The software allows a programmer to extend this list of properties with additional items as they are needed by a data source. Such an example is the Database connector, which defines a new property called "Connection String" to capture the information required to access data in a database.

<destination mode="" name="">
<property name="Data Source"> </property>
<property name="Configuration File"> </property>
</destination>

A destination element is used to describe a data destination. A unique feature of DTE is that a transformation process may contain multiple destinations. Imagine the following example: a file with POS sales and returns that needs to be imported in a database. Sales must go to the Sales table and returns must go to the Restock table. This process will contain two destination elements, one for each table. All attributes and property tags have the same meaning, as for the source.

<client>
<assembly_name>MyClient.dll</assembly_name>
<object_name>MyClient.MyProcesserEvents</object_name>
</client>

The Client element is used to indicate a notification object. This notification object is called during the transformation process to give a chance for the client to dynamically make decisions at run-time. The notification object must implement the IDTENotifications interface that is defined as follows:

public interface IDTENotifications
{
bool CheckProperties (string sDataName, ref Hashtable colProperties);
string GetSrcRecordIDFromRawData (object RawData);
string GetDestRecordID(ISourceRecord srcRecord);
bool BeforeApplyTransformation (ISourceRecord srcRecord);
bool BeforeComit(ref IDestinationRecord destRecord);
}

The CheckProperties method allows client objects to modify properties before they are applied to the source or destination objects.

The GetSrcRecordIDFFromRawData method is called when the source supports multiple record types and the data source is not capable of automatically determining which record type to use. This method is defined for future versions because current versions of the framework support only sources with records that can be automatically determined.

The GetDestRecordID method has the same meaning as above, but for data destinations.

The BeforeApplyTransformation method is called just before the DTE populates the destination columns from the source record. Calling this method gives the client object a chance to modify the source fields, perhaps to apply more transformations.

The BeforeCommit method is called after applying transformations, but before persisting the destination record.

<mappings source_rec_id=" " dest_rec_id="
            " dest_name=" ">
<map destination_column_index="0"
            source_column_index="0"/>
<map destination_column_index="1"
            source_column_index="2"/>
<map destination_column_index="2"
            source_column_index="1"/>
<map destination_column_index="3"
            source_column_index="4"/>
<map destination_column_index="4"
            source_column_index="3"/>
</mappings>
</transformation>

The Mappings element describes the transformations that need to be applied between source record and destination record. There could be more than one mappings element depending on how many destination records exist in each destination:

  • Source_Rec_ID attribute represents the id of the source record and this should match one of the record ids in the data source descriptor file.
  • Dest_Rec_Id attribute represents which destination record to use when making the mapping.
  • Dest_Name represents the name of a destination defined in a Destination element.

A Mappings element contains a list of Map elements that indicates a field-to-field correspondence. The source and destination elements can be indicated using indexes or names. However, the use of names comes with a performance trade-off, as there is an additional function call to determine the index of specified column name in the array of filed definitions. The function that accomplishes that is named GetFieldIndexByName and you'll find it included in each record implementation.

Here is the source code of this function:

private int GetFieldIndexByName (string
            sFieldName)
{
for (int i=0; i<_fieldInfo.Length;
            i++)
{
if (_fieldInfo[i].Name.Equals(sFieldName))
return
            i;
}

return -1;
}

The DTEApplication object uses the DTETrxDescriptorReader object to read a transformation descriptor. The Load method takes a file name as a parameter and searches for each key element in the XML document. To process mapping elements, the Load methods uses a FieldMappingReader object to create a FieldMappingCollection object that is the internal representation of a XML mappings element.

The implementation of these two classes is fairly straightforward. We use the System.Xml namespace to parse an XML file and use XSLT to find a particular XML node or collection of nodes. To achieve this, we use the XmlDocument.SelectSingleNode and XmlDocument.SelectNodes methods respectively. Both methods take an XSLT search path as a parameter.

Here are few code excerpts:

XmlNodeList list = root.SelectNodes("destination");
XmlNodeList mapList = root.SelectNodes("mappings");
node = root.SelectSingleNode("client");

Now that we've seen what the initialization step involves, let's move to the next phase: create data sources and destinations. Once the descriptor is parsed, DTE knows what type of data sources and destinations are involved in the process and can create an instance of each.

As we mentioned earlier, DTE uses the IDataFactory interface to create the source and destination objects from specific data factories. The first step is to obtain a reference to the data factory corresponding to each source and destination. To do that DTE calls a private method GetDataFactoryFromMode that takes a string parameter representing the type source or destination to retrieve the factory for. A DTE configuration file maintains a list with active data factories and it can be updated with new data factories as they are available to DTE. The structure of this configuration file is:

<DataFactories>
<DataFactory mode="">
<Assembly> </Assembly>
<Class> </Class>
</DataFactory>
</DataFactories>

One or multiple DataFactory elements describe an active factory by defining a corresponding assembly and class. When a DTEApplication object instance is created, it loads the information in this file and makes it available to the transformation descriptor reader object.

Here is the source code for the GetDataFactoryFromMode method:

private IDataFactory GetDataFactoryFromMode
            (string mode)
{
IDataFactory factory = null;
FactoryInfo
            factoryInfo = null;

try
{
factoryInfo =
            _factoryReader.GetFactoryInfoByMode(mode);
_assembly =
            Assembly.Load(factoryInfo.Assembly);
factory =
            (IDataFactory)_assembly.CreateInstance(factoryInfo.Class);
}
catch (Exception e)
{
Trace.WriteLine(e);
factory =
            null;
}

return factory;
}

After DTE creates data sources and destinations, the next step is to assign properties read from the descriptor to each source and destination. DTE first notifies the client object, if it exists, that it is about to assign properties, and if any changes are made to the properties they are captured before.

Here is the source code from the private TrxInitialization method:

Hashtable tmp;
if (_descriptorReader.NotificationInterface != null)
{
for (int i = 0; i < _descriptorReader.DestinationDescriptors.Count; i++)
{
tmp =
((DataTrxDescriptor)_descriptorReader.DestinationDescriptors[i]).GetAllProperties();

// if user changes any of the properties send new properties back to
// the descriptor to be processed.
if (_descriptorReader.NotificationInterface.CheckProperties(
((DataTrxDescriptor)_descriptorReader.DestinationDescriptors[i]).GetName(),
ref tmp))
((IDataDestination)_descriptorReader.Destinations[((DataTrxDescriptor)_descriptorReader.
DestinationDescriptors[i]).GetName()]).SetProperties(tmp);
}

tmp = ((DataTrxDescriptor)_descriptorReader.SourceDescriptor).GetAllProperties();

if (_descriptorReader.NotificationInterface.CheckProperties(
((DataTrxDescriptor)_descriptorReader.SourceDescriptor).GetName(), ref tmp))
_descriptorReader.Source.SetProperties(tmp);
}

This was the last step in the initialization phase, and DTE has now got all the information necessary to step into the second phase: the actual transformation process. But before going into this second phase, we'll walk you through the design and implementation of standard types of data sources and destination.

Standard Data Sources and Destinations

DTE is shipped with a set of the most-used types of data sources and destinations: CSV files, fixed width files, XML files, database (only SQL Server and MSDE are supported), and a record base file. In DTE terminology we call them "connectors". We chose these connectors because:

  • CSV and fixed width are the most used types of text files.
  • XML is a new technology that has gained a lot of popularity in the industry, and so there is a very good chance that you'll need to work with such files.
  • Database is probably the best choice when it comes to manipulating large amounts of information.
  • Record based files are also a very popular choice mostly in legacy systems like mainframe and AS/400.

Of course we could have added a connector for Excel and other popular Office types of files, but this is a relatively easy task given the standard set.

Text based connectors like CSV, Fixed width and Record Based File are designed using a basic type of file that is capable of handling records of information stored in fields or columns. In this section we'll show you the design for a single connector (CSV Connector) and walk you through the implementation of the others. The design for all other connectors is identical and should not pose too many difficulties in this case study.

The architecture of the CSV Connector is described in the following UML Class Diagram:

Let's analyze for a second this architecture. As you can see, we use the interfaces defined earlier, IDataSource and IDataDestiantion, to create sources and destinations, and ISourceRecord and IDestinationRecord for corresponding records. If we take a closer look at IDataSource and IDataDestination, one may say they are almost identical so we could have put all the code in the same class and spare a class definition. That might be true, but we thought that in 99% of the cases the data source and destination are different types so, loading a class instance with both implementations when we only need one part, source or destination, doesn't make sense.

This separation is also beneficial from another perspective. If a client application doesn't need, let's say, a destination (e.g. in a scenario with a mainframe source where we only export data), the developer will choose to implement only a MainframeSource class to only extract data from mainframe. The same consideration applies in the design of records, where we separate the source record and destination record classes.

The BaseFile is a class that prolongs the abstraction of a text file in our architecture. This design was preferred over the use of .NET's TextReader and TextWrites classes because we might want to change the foundation of handling files in the future. The implementation is simple enough and does not require any additional comments.

The BaseFileWithFields class is designed as a placeholder for a collection of fields with string values. This class is primarily used by corresponding record objects in text file connectors and is necessary to reduce the duplication of code in each concrete record object. The implementation is simple and includes methods to load and get the value of a field or for a collection of fields.

CSV Connector

The CSVSource class implements the IDataSource interface and derives from the BaseFile class. Let's analyze the source code of this class:

/// <summary>
/// CSVSource is the
            implementation of IDataSource for CVS connector.
///
            </summary>
public class CSVSource : BaseFile, IDisposable,
            IDataSource
{
private string _sFileName,
            _sConfigFileName;
private ISourceRecord _record;
private
            string _recordID = null;
private bool _eof = false;

public
            CSVSource()
{
_record = new
            CSVSourceRecord();
}

The class constructor creates a new instance of the CSVSourceRecord that will be used to handle records from a file with character separated value records.

Below is the implementation for methods in the IDataSource interface:

// IDataSource methods
public bool MultipleRecordTypesSupported ()
{
return false;
}

Because this type of file has only one type of record type, the MultipleRecordTypesSupported method returns false.

public bool LoadDescriptor (string DescriptorFile)
{
return _record.LoadDescriptor(DescriptorFile);
}

The LoadDescriptor method delegates the call to the record object that in fact is able to digest the information in the descriptor file. Some of the information might be useful to the source object, but most of it helps the record object to parse raw data.

Let's analyze a sample descriptor file:

<?xml version="1.0"?>
<descriptor type="CHAR_SEPARATED_VALUES" separation_char="|" rec_id="ReturnDst">
<column name="TRANSACTION_ID"/>
<column name="SKU_NUMBER"/>
<column name="QUANTITY"/>
</descriptor>

As you may notice, the type of the descriptor is set to CHAR_SEPARATED_VALUES, which indicates that this is a CSV descriptor. The separation_char attribute indicates the character that will be used to split the raw data of information, a pipe ("|") character in this example. The record_id attribute defines this record's identifier. This attribute is very important as it plays a critical role in defining field mappings. The three column elements define the fields in this sample CSV file:

public bool MoveNext()
{
bool ret =
            false;

if (!_eof && base.ReadLine())
ret =
            _record.ParseFieldsFromRawData(base.line);
else
_eof =
            true;

return ret;
}

MoveNext as the name suggests, is used to move to the next record in the data source. In this case we read lines from the source file as long as the end of the file is not reached. To advance to the next line we call the ReadLine method on the base class BaseFile. If we get a positive result, we call the ParseFieldsFromRawData method on the record object, which populates each field in the source record with the value in the file.

public new bool Close()
{
return base.Close();
}

Close delegates the call to the BaseFile. As in the case of LoadDescriptor we use the base keyword to call methods on the base class. The Base keyword is a new feature in .NET's C# language where only one base class is allowed per class declaration. This restriction makes it safe to identify the base class in the current inheritance operation.

public bool Open()
{
bool ret =
            true;

if (LoadDescriptor(_sConfigFileName) == false)
ret =
            false;
else
{
ret = Open(_sFileName,
            OpenTypeEnum.OT_READ);
if (ret)
_recordID =
            _record.GetRecordID();
}

_eof = !ret;

return
            ret;
}

The Open method is a good place to load the data source descriptor. If the call is successful, we use the Open method on the BaseFile class. If this operation succeeds, the record name is stored in a local variable.

public void SetProperties (Hashtable props)
{
_sFileName =
            (string)props[DTEStaticPropertiesEnum.DTE_PROP_DATASOURCE];
//
            file name
_sConfigFileName =
            (string)props[CSVConnectorPropertiesEnum.CSV_PROP_CONFIG_FILE];
// configuration file
}

If you remember when we discussed data properties, the final step in the process of assigning properties to source and destinations was to call the SetProperties method to achieve this goal. The implementation stored the value of two important properties in local variables for later use.

public string GetCurrentRecordID
            ()
{
return _recordID;
}

public void
            SetCurrentRecordID (string recordID)
{
_recordID =
            recordID;
}

public ISourceRecord
            GetCurrentRecord()
{
return (_eof ? _null :
            record);
}
}

The GetCurrentRecord function returns the instance of ISourceRecord stored in the class when the file has not reached the end.

As you can see, implementing the source part of the CSV connector is not very complicated and is made easy using supporting classes. Let's now examine the implementation of the CSVSourceRecord class:

namespace DTEConnectorFactory.CSV
{
/// <summary>
/// Implementation of ISourceRecord for CVS connector.
/// </summary>
public class CSVSourceRecord : BaseFileWithFields, ISourceRecord
{
private char _separator;
private string _recordID = null;
private string _rawData = null;

// ISourceRecord methods

//code ommited

The ParseFieldFromRawData is the method that transforms raw data, in our case a text line in the source file, into corresponding fields of the CSV connector by splitting the string with the character indicated in the source descriptor file.

public bool ParseFieldsFromRawData (object sLine)
{
try
{
_rawData = (string)sLine;

base.FieldValue = _rawData.Split(_separator,
base.FieldValue.Length);

return true;
}
catch (Exception e)
{
Trace.WriteLine(e);
return false;
}
}

The LoadDescriptor method has two definitions, one that takes a string as a parameter and second one that takes an XmlElement as a parameter. The second method is useful, as we'll see later, for record-based connectors. Both methods use an instance of the CSVDescriptorReader class to read the CSV source descriptor and retrieve some of the information that is needed in this class like, such as information about fields, field separator, and record name.

public bool LoadDescriptor (string
            DescriptorFile)
{
CSVDescriptorReader configReader = new
            CSVDescriptorReader();

if
            (!configReader.Load(DescriptorFile))
return
            false;

_separator = new
            char[configReader.SeparationChar.Length];
configReader.SeparationChar.CopyTo(0,
            _separator, 0,
configReader.SeparationChar.Length);

_recordID =
            configReader.RecordID;
base.InitializeNoOfFields(configReader.Columns.Length);

return
            true;
}

The CSVDescriptorReader class is designed to parse a descriptor for a CSV connector. Each descriptor is unique because it contains information of a particular data source and therefore it is not possible to write a generic reader. In this particular case, the CSV descriptor contains a list of columns in a record and the character used as field separator. More details about each descriptor are presented in the next section.

Next we'll present the CSVDestination and CSVDestinationRecord classes. Destinations implement the IDataDestination interface, and CSVDestination is no exception. The primary goal of a destination is to save a record to the specified location. This is accomplished with the WriteRecord method, which takes as a parameter an object that implements the IDestinationRecord interface. The implementation of WriteRecord is particular to each connector and we'll outline the differences as we move on.

Let's take a look at this method in CSVDestination.

public bool WriteRecord (IDestinationRecord record)
{
base.line = (string)record.CollateRecordFromColumns();
return base.WriteLine();
}

We designed the method this way keeping in mind that records in different destinations have different formats. For example a record for a CSV file is a list of columns joined using a separator character, a record in an XML file is an Xml element, and a record in a database can be created using an INSERT SQL statement.

Therefore, the CollateRecordFromColumns method returns an Object value. In .NET all classes derive from the Object class and we can take advantage of this and rely on this lowest common denominator to achieve a more generic code. Using a cast in the connector we can quickly convert the Object value to the original format.

CSVDestination converts the returned value into a string representing the line of text to be written in the CSV destination file, and performs a write operation using the WriteLine method in BaseFile class.

Let's pause for a moment to analyze the CollateRecordFromColumns method. In the case of the CSV connector, the line of text that needs to be written is built simply by calling String.Join. The String class has a rich set of methods and one of them is Join, which is exactly what we need to build the CSV record.

public object CollateRecordFromColumns()
{
return String.Join(_separator, base.FieldValue);
}

The rest of the methods in CSVDestination and CSVDestinationRecord are simple enough to be omitted from this presentation.

The next text-based connector is for text files with fixed width fields, which is probably the next most used text file format.

Fixed Width Connector

Let's start by analyzing a sample descriptor for this type of connector:

<?xml
            version="1.0"?>
<descriptor type="FIXED_WIDTH"
            rec_id="POSRecord">
<column name="StoreID" start="0"
            width="8" paddingType="none"/>
<column name="SKUNumber"
            start="8" width="6" paddingType="left" paddingChar="
            "/>
<column name="PurchaseDateTime" start="14" width="10"
            paddingType="none"/>
<column name="ProductDesc" start="24"
            width="30" paddingType="none"/>
<column name="SalePrice"
            start="54" width="10" paddingType="right"
            paddingChar="0"/>
</descriptor>

The type attribute contains the identifier for the fixed width connector, FIXED_WIDTH. The record identifier is contained in the rec_id attribute. Column elements contain information about fields in the file like name starting position (zero based), width of each field, and padding information: type and character. The starting position of the columns don't need to be in sequence, and it doesn't have to cover 100% a physical record. For example, given the above example, we might decide to omit one of the fields from the sequence. Let's say we don't need the product description in our application, so we decide to delete this column element from definition. This will not cause problems for DTE, and it will simply ignore fields that are not described in a column element.

The implementation of FWFSource and FWFDestination are very similar to FWFSourceRecord and FWFDestinationRecord respectively with CSV, except with the way that fields are parsed from raw source data and the way a record is built from fields. The implementation for these two different methods is provided below.

The ParseFieldsFromRawData method uses the information from the descriptor, where each field starts, and how many characters it has, to get the value of each field.

public bool ParseFieldsFromRawData (object sLine)
{
_rawData = (string)sLine;

try
{
for (int i=0; i < _fieldInfo.Length; i++)
base.FieldValue[i] =
_rawData.Substring(_fieldInfo[i].Start, _fieldInfo[i].Width);
}
catch (Exception e)
{
Trace.WriteLine(e);
return false;
}

return true;
}

For this to work correctly it is very important to have correct information in the descriptor file. If the raw data information is not exactly matching the pattern in the descriptor, the function returns false.

In the FWFDestinationRecord class we have two methods that help to build the record data. In the case of a file with Fixed Width Fields the record is a string.

public object CollateRecordFromColumns()
{
string tmpLine = System.String.Empty;
int tmp = 0;

for (int i=0; i < _fieldInfo.Length; i++)
{
if (tmp < _fieldInfo[i].Start)
{
tmpLine += new String(' ', _fieldInfo[i].Start - tmp);
}
tmpLine += MakeFieldWidth(i);
tmp += _fieldInfo[i].Width;
}

return tmpLine;
}

// private functions
private string MakeFieldWidth (int index)
{
FWFField _info = _fieldInfo[index];

if (FieldValue[index] == null)
return new String(_info.PaddingChar[0], _info.Width);

if (FieldValue[index].Length > _info.Width)
return (FieldValue[index]).Substring(0, _info.Width);
else if (FieldValue[index].Length < _info.Width)
if (_info.PaddingType == PaddingTypeEnum.PADDING_TYPE_RIGHT)
return (FieldValue[index].PadRight(_info.Width, _info.PaddingChar[0]));
else if (_info.PaddingType == PaddingTypeEnum.PADDING_TYPE_LEFT)
return (FieldValue[index].PadLeft(_info.Width, _info.PaddingChar[0]));
else if (_info.PaddingType == PaddingTypeEnum.PADDING_TYPE_NONE)
return (FieldValue[index].PadRight(_info.Width, ' '));

return FieldValue[index];
}

A helper function MakeFieldWidth is implemented to bring a field to the correct length. This is done using the information in the descriptor file. As we'll see in the next section the descriptor contains all the necessary information about fields. Again we make use of the built-in functions of the String class to left or right pad a string with given characters.

Next we'll take a look at the record based connector. This type of connector is very useful when we have to read or create a source file with multiple types of records. The connector is built to recognize the type of record in the source file by looking at a specific sequence of characters, or by interrogating the notification client object. A limitation of the current version is that only two types of record types are allowed: CSV and FWF.

In the RecordBasedSource class the most interesting piece of code is the MoveNext method:

public bool MoveNext()
{
bool ret = true;
string tmpRawData = null;
RecordIdentifierValueToIDMapping tmpMap;

try
{

We use BaseFile methods to read the next line of text from the file. If the operation succeeds, the next step is to identity what record type we just read. For this, we first get the sequence of characters as indicated in the descriptor file and lookup in the map to find the correct record name. All the information is provided in the descriptor file.

if (!_eof && !base.ReadLine())
{
ret = false;
_eof = true;
}
else
{
// get record ID
_currRecordID =
base.line.Substring(_recID.Start, _recID.Length);

tmpMap = (RecordIdentifierValueToIDMapping)_recIdValueRecIDMappings[_currRecordID];
_currRecordID = tmpMap.RecordID;

// get record object from list
_currRecord = (ISourceRecord)_records[_currRecordID];

After we have the record name we can move to parsing the fields from the raw data. There is one more step before we can do this, which is to eliminate the record identification sequence from the raw data. We call ParseFieldsFromRawData on the identified record object instance and if the source record needs to store the value of the identifier in a specific column we put the value in place.

            //
            parse fields from data source
tmpRawData = (_recID.Start > 0 ?
            base.line.Substring(0, _recID.Start - 1) :
String.Empty) +
            base.line.Substring(_recID.Start + _recID.Length);

ret =
            _currRecord.ParseFieldsFromRawData(tmpRawData);

if (ret
            && tmpMap.DestColumnIndex >
            -1)
_currRecord.SetColumnValue(tmpMap.DestColumnIndex,
            tmpMap.Identifier);
}
}
catch (Exception
            e)
{
Trace.WriteLine(e);
ret = false;
}

return
            ret;
}

You might have already noticed that this connector namespace has no Source and Destination records defined, because we use either CVS's or FWF's for corresponding records.

XML Connector

Next, we'll take a look at the XML connector, which is a little bit more complex than previous ones. When we designed the XML connector we began by analyzing many types of files to observe a common pattern. What we noticed is that in most situations the elements with significant information are located immediately under the root element or somewhere in the heart of the file. Also, the structure of the record can be simple or complex, meaning that the elements of the record are located immediately under record tag or there is an additional XSL path to it.

The following diagram depicts a sample file:

<?xml version="1.0"?>
<x>
<x1>
<y> <!- Record starts here -->
<z>
<a>field 11</a>
<b>field 12</b>
<c>
<d>field 13</d>
</c>
</z>
</y> <!- Records ends here -->
<y> <!- Record starts here -->
<z>
<a>field 21</a>
<b>field 22</b>
<c>
<d>field 23</d>
</c>
</z>
</y> <!- Records ends here -->
<y> <!- Records starts here -->
<z>
<a>field 31</a>
<b>field 22</b>
<c>
<d>field 33</d>
</c>
</z>
</y> <!- Records ends here -->
</x1>
</x>

As you can see in this example, the path to the list of records is x/x1, and each record has three fields. The third field has a more complex structure with an outer element c and inner one d. Also the record has two levels to actual field elements: y/z.

Let's now examine the descriptor file to better understand how it works:

<?xml version="1.0"?>
<descriptor type="XML" rec_id="XMLRecord">
<record_path>//x/x1</record_path>
<record_tag>y/z</record_tag>
<columns>
<column name="DP_STREAM_NO">a</column>
<column name="CONTRACT">b</column>
<column name="PURCHASE_DATE">c/d</column>
</columns>
</descriptor>

The Record_Path element describes the XPath to the list of records. In our example, records are enclosed in element y which is located under element x1, so the XPath to y is //x/x1. The Record_Tag element contains the XPath inside the record structure to the element where the field nodes start. In our sample, record fields are located inside z element, and the XPath relative to a record root is y/z. A series of column elements describe the XPath of each field relative to the root element in the report_tag element. In our example, the record has three fields described in nodes a, b and d. The node d is located under c so the relative XPath is c/d.

The XML connector we built can create and read these kind of XML files. The structure of the XML file we want to read can be more complex that this, and we intend to extend the capabilities of this connector, but in the majority of cases this is good enough.

The basic idea behind XmlSource is to create a list with source elements and then fetch them as the client application calls the MoveNext method. Let's see the Open method:

public bool Open ()
{
bool ret =
            true;

try
{
if (LoadDescriptor(_sConfigFileName) ==
            false)
return false;

_recordID =
            _record.GetRecordID();
_index = -1;

_doc = new
            XmlDocument();
_doc.Load(_sFileName);
_root =
            _doc.DocumentElement;
_lines =
            _root.SelectNodes(((XmlSourceRecord)_record).RecordPath + "/"
+
            ((XmlSourceRecord)_record).RecordTag);

if (_lines.Count ==
            0)
_eof = true;
else
_eof = false;
}
catch
            (Exception e)
{
Trace.WriteLine(e);
ret =
            false;
}

return ret;
}

After loading the descriptor file we can open the source XML file, and using XSLT we grab the list of elements with records that we are interested in. The information about where these records are located is maintained in the descriptor file. The list of XML elements with records is maintained and we'll fetch them when requested.

The MoveNext method is shown next:

public bool MoveNext ()
{
if (_eof
            && _index + 1 > _lines.Count - 1)
{
_eof =
            true;
return (!_eof);
}
else
return
            _record.ParseFieldsFromRawData(_lines.Item(++_index));
}

We use a variable to maintain the current position in the list, very much like a cursor pointer in a database recordset. When there are no more elements we flag the EOF variable.

Moving to the XmlSourceRecord class we analyze the ParseFieldsFromRawData method:

public bool ParseFieldsFromRawData (object
            oRawData)
{
XmlNode _tmp = null;
XmlNode aNode =
            (XmlNode)oRawData;
bool ret = true;

try
{
for (int
            i=0; i < _fieldInfo.Length; i++)
{
_tmp =
            aNode.SelectSingleNode(_fieldInfo[i].ColumnTag);
if (_tmp !=
            null)
base.SetField(i, _tmp.InnerText);
}
}
catch
            (Exception e)
{
Trace.WriteLine(e);
ret =
            false;
}

return ret;
}

Raw data in this case is an XmlNode object that is passed from the MoveNext method. We then use the information in the descriptor about the record structure and parse each field value.

Moving forward to the XmlDestination and XmlSourceDestination classes, notice how we build the record object. In our case, this is the XmlNode containing all fields. Building the XmlNode requires us to build the whole record structure while we populate field values.

public object
            CollateRecordFromColumns()
{
string[] elem;
XmlNode node =
            null, rec = null, newRecord = null;
int i = 0;

// create
            record
elem = _recordTag.Split(new char[] {'/'});
node =
            _recs;

for (int idx = 0 ; idx < elem.Length; idx++)
{
node =
            node.AppendChild(_doc.CreateElement(elem[idx]));
if (i ==
            0)
newRecord = node.Clone();
}

// for safety reasons
            we'll check if new records was created
if (newRecord ==
            null)
newRecord = node.Clone();

rec = node;

// add
            record fields
for (int idx = 0 ; idx < _fieldInfo.Length;
            idx++)
{
elem = _fieldInfo[idx].ColumnTag.Split(new char[]
            {'/'});
i = 0;
node = rec;
while (i < elem.Length)
node =
            node.AppendChild(_doc.CreateElement(elem[i++]));

node.InnerText
            = _file.GetField(idx);
}

return
newRecord;
}

After the record is built, we return the XmlNode object to the WriteMethod function that adds it to the existing list of records. The XmlDestination object maintains an instance of XmlDocument with all the current records until the Close method is called, when it saves it on disk.

The so-called header part of the XML file, x/x1 in our sample file, is created when the destination file is opened. Notice the Open method:

public bool Open()
{
if (LoadDescriptor(_sConfigFileName) == false)
return false;
else
_recordID = _record.GetRecordID();

// create the XmlPath to the records
string[] elem = ((XmlDestinationRecord)_record).RecordPath.Split(new char[] {'/'});
string sXML = "<?xml version=\"1.0\"?>";

Stack stack = new Stack(elem.Length);

As you can see, we've made use of Stack class. Stack is a structure that follows the rule FILO, First In Last Out. This means that elements are stacked in the same order they are pushed in, and extracted in reverse order. You can think of it as a box where books are stored. To reach the first stored book in the box you must first extract all the books in reverse order.

In .NET the stack is implemented as a collection with two important methods, Push and Pop. Push adds a new element to the stack and Pop extracts last added element. The Stack object can be created with an initial limit, but the capacity grows as needed.

// create the structure to the record
int
            i = -1;
while (++i < elem.Length)
if
            (!elem[i].Equals(String.Empty))
{
sXML += "<" + elem[i] +
            ">";
stack.Push("</" + elem[i] + ">");
}

while
            (stack.Count>0)
sXML += (string)stack.Pop();

_doc =
            new XmlDocument();
_doc.LoadXml(sXML);
_root =
            _doc.DocumentElement;
_recs =
            _root.SelectSingleNode(((XmlDestinationRecord)_record).RecordPath);
((XmlDestinationRecord)_record).SetOutterElements(_doc,
            _recs);

return true;
}

We loop through the list of outer elements and create each one of them. The last one created, the most inner one, serves as a base for adding record elements and we store it in a variable _recs.

Database Connector

The last connector on our list, but probably the most used one, is Database.

Let's examine a descriptor example:

<?xml version="1.0"
            encoding="UTF-8"?>
<descriptor type="DATABASE"
            rec_id="POSSaleDBRec">
<column name="TransactionID"
            dataType="String" length="20"/>
<column name="ProductID"
            dataType="String" length="10"/>
<column name="Quantity"
            dataType="Int32"/>
<column name="SaleDate"
            dataType="DateTime"/>
</descriptor>

The column elements describe columns in the source recordset. The dataType and length attributes help you describe the type of fields that the source recordset contains.

Following the same pattern as before, we implemented DBSource and DBDestination classes just like we did with the DBSourceRecord and DBDestinationRecord classes. The only difference in the Source is that we use the System.SqlClient namespace to get a set of records from the specified database and store them in a SqlDataReader object. The source record is implemented as a collection that holds objects of type Object, so we cannot use the BaseFile class any more.

Here is the implementation for the ParseFieldsFromRawData method:

public bool ParseFieldsFromRawData (object
            rawData)
{
try
{
_rawData =
            (SqlDataReader)rawData;

// read fields from data reader based
            on descriptor
for (int i=0; i < _dbFieldValues.Length;
            i++)
_dbFieldValues[i] = _rawData.GetValue(i);

return
            true;
}
catch (Exception e)
{
Trace.WriteLine(e);
return false;
}
}

The implementation for the Destination object is also quite simple because persisting a record involves nothing more than executing an INSERT SQL statement. The INSERT SQL is returned by the DBDestinationRecord object. Here is the implementation of WriteRecord :

public bool WriteRecord (IDestinationRecord record)
{
string sInsertSQL = (string)_record.CollateRecordFromColumns();
_dbHelper.RunSQL(sInsertSQL);
return true;
}

The implementation for CollateRecordFromColumns is listed below:

public object
            CollateRecordFromColumns()
{
string sInsertSQL = "INSERT INTO
            " + _sTableName + "(";
string sValue = null;

for (int i=0;
            i < _fieldInfo.Length; i++)
sInsertSQL += (i==0 ? String.Empty
            : ",") + _fieldInfo[i].Name;

sInsertSQL += ")
            VALUES(";

for (int i=0; i < _fieldInfo.Length;
            i++)
{
sValue = Convert.ToString(_dbFieldValues[i]);
if
            (_fieldInfo[i].Length != -1 && sValue.Length >
            _fieldInfo[i].Length)
sValue = sValue.Substring(0,
            _fieldInfo[i].Length);

if (_fieldInfo[i].DataType ==
            FieldDataType.FIELD_TYPE_STRING ||
_fieldInfo[i].DataType ==
            FieldDataType.FIELD_TYPE_DATETIME)
sValue = "'" + sValue +
            "'";

sInsertSQL += (i==0 ? String.Empty : ",") +
            sValue;
}

sInsertSQL += ")";

return
            sInsertSQL;
}

We basically loop through the list of fields and build an SQL statement like this: INSERT INTO (columns) VALUES (values). We use the information in the descriptor to check the type of each column because string and date fields need to be enclosed between ' and ' characters.

The DBDestinationRecord class has a slightly more complicated implementation for the SetColumnValue function. Because the database can hold multiple types of data, we need to convert incoming values to destination data type. This is accomplished by using the System.Reflection namespace to dynamically invoke a function in a class.

The Convert class has a set of static functions that perform conversions from one type to another. If the type of passed parameter is different from the destination type we call the appropriate function in the Convert class. We use the Invoke method of the Type class, making sure to set the correct binding parameters. In this case, BindingFlags.Public, BindingFlags.InvokeMethod and BindingFlags.Static are required.

public void SetColumnValue (int index, object colValue)
{
string sDataType = colValue.GetType().FullName;
object result = null;

// need to translate to destination data type
Type type = typeof (Convert);

if (!sDataType.EndsWith(_fieldInfo[index].DataType))
{
result = type.InvokeMember("To" + _fieldInfo[index].DataType,
BindingFlags.Public | BindingFlags.InvokeMethod | BindingFlags.Static,
null, null, new Object[] { colValue });
_dbFieldValues[index] = result;
}
else
_dbFieldValues[index] = colValue;
}

All database related operations like, open connection, execute SQL statement, create a command object, create and load a set of records, are separated in the DatabaseHelper class.

The following diagram depicts the relationship between transformation descriptor elements, record descriptor elements, and DTE to help you better understands the existing relations between key elements.

This figure helps you remember key elements in the descriptors and the relationship between them:

  • The name attribute in the source and destination elements is used in the mappings element. They help identify a particular destination and of course the source.
  • The rec_id attribute is also a key attribute in the mappings element as it helps identify the record that is used in a mapping relation.
  • The properties attached to source and destination elements are also important for the DTE as they identify the location of files, database connection string or SELECT statements. Remember that these properties can be modified at run-time through DTE notifications.

Source and Destination descriptors

This section is important because it explains the structure of each descriptor so that you'll be able to create your own descriptors. In previous sections we mentioned that critical information about each connector is contained in descriptor files.

There are few general rules that apply to all descriptors:

  • Type attribute in the root element contains the description of record type. The value of this attribute is predefined in the factory settings file.
  • Rec_id attribute in the root element contains the name of the record

Both these elements are mandatory for all descriptors.

CSV Descriptor

<?xml
            version="1.0"?>
<descriptor type="CHAR_SEPARATED_VALUES"
            separation_char="," rec_id=" ">
<column
            name="field1"/>
<column name="field2"/>
<column
            name="field3"/>
</descriptor>

The separation_char attribute indicates which character is going to be used when parsing source records, and also when building destination records from individual fields. A collection of column elements indicates the order of fields and the name of each field.

FWF Descriptor

<?xml
            version="1.0"?>
<descriptor type="FIXED_WIDTH"
            rec_id="POSRecord">
<column name="StoreID" start="0"
            width="8" paddingType="none"/>
<column name="SKUNumber"
            start="8" width="6" paddingType="left" paddingChar="
            "/>
<column name="PurchaseDateTime" start="14" width="10"
            paddingType="none"/>
<column name="ProductDesc" start="24"
            width="30" paddingType="none"/>
<column name="SalePrice"
            start="54" width="10" paddingType="right"
            paddingChar="0"/>
</descriptor>

There are no additional attributes in the root element. A collection of column elements defines each field. For this descriptor, the order is not important because each field is very well defined within the collection. Each column element can have following attributes:

  • Name is mandatory and represents the name of the field.
  • Start is mandatory and represents the start character position for this field. Valid values are greater or equal to zero.
  • Width is mandatory and represents the maximum number of characters the field can contain.
  • PaddingType is optional and contains the type of padding to be applied to the current field. This attribute is used only for destination records. Valid values for this attribute are: right, left, none. If the attribute is not present, the value none is assumed by default.
  • PaddingChar is mandatory if PaddingType is different than none. It represents the character to be used in the padding operation. As for PaddingType, this attribute is used only for destination records.

Xml Descriptor

<?xml version="1.0"?>
<descriptor type="XML" rec_id="XMLRecord">
<record_path>//x/x1</record_path>
<record_tag>y/z</record_tag>
<columns>
<column name="DP_STREAM_NO">a</column>
<column name="CONTRACT">b</column>
<column name="PURCHASE_DATE">c/d</column>
</columns>
</descriptor>

There are no additional attributes in the root element.

The Record_Path element defines a XSL path to the record tag. This element can be empty if the record tag nodes are child elements of the root XML document.

Record_Tag represents a XSL path to the element that contains the list of fields inside the record, plus the name of record element. This element cannot be empty, but can be only the name of the record tag without an XSL path.

The Columns collection contains a list of columns in a record. The order is not important because each element is described as an XSL path. Column elements contain a name attribute representing the name of the field, and the value of column element is the XSL path to the field element.

Database connector

<?xml version="1.0"
            encoding="UTF-8"?>
<descriptor type="DATABASE"
            rec_id="ShoppingCartRec">
<column name="shcCartID"
            dataType="String" length="30"/>
<column name="shcItemID"
            dataType="String" length="30"/>
<column name="shcQuantity"
            dataType="Int32"/>
</descriptor>

The Column elements define a column returned by the SQL statement in a DBSource object, or fields that need to be filled in for a DBDestination object. The order is important only for the source end.

  • Name attribute is the name of the column as defined in the database.
  • DataType attribute specifies the type of field. Valid values for this field are: String, Int32, Int64, Int16, Long, DateTime, etc. In general any data type that can be represented in a database and which has a .NET object defined.

Some additional attributes may appear as they are required by the DataType attribute. For example, String data type requires a Length attribute.

RecordBased descriptor

The structure of this descriptor is a little bit different because of the complexity associated with the connector.

<?xml version="1.0"?>
<descriptor type="RECORD_BASED">
<record_identifier type="AUTO" start_pos="0" length="3" />

<!-- Describes the mapping between record name and record identifier -->
<record_identifier_type rec_id="Rec1" identifier="ID" dest_column_index="1"/>

<!-- for now only FW and CSV record types are supported -->
<record type="CHAR_SEPARATED_VALUES" separation_char="," rec_id="Rec1" import="">
<column name="field1"/>
<column name="field2"/>
</record>
</descriptor>

The Type attribute in the descriptor element identifies the type of connector and should always be RECORD_BASED. This attribute is mandatory and an error is raised if the attribute is not present.

The Record_Identifier element describes the starting position and the number of chars where the record identifier is located. In most cases this element is located at the beginning of record.

The Record_Identifier_Type elements define a map between the record name and the value of the record identifier. As we mentioned before, each record will be identified after a sequence of chars in a specified position in the source file. There must be defined a number of such elements equal with the number of possible values for record identifier field. The Rec_Id attribute defines the name of the record that must be defined later in the file. The Identifier attribute contains the value of the record identifier.

Each record element defines a record that must correspond with a Rec_id attribute in a Record_Identifier_Type element. The structure of this record is identical with that of the descriptors mentioned above. If you want to avoid including the record descriptor in this XML file you can specify in the import attribute a path to the file that contains the descriptor. Import attribute is optional, but if present, takes precedence over column elements, in which case column elements are ignored.

The Sample Application

The application that we'll show you comes to exemplify the use of DTE in a real world scenario. A retail store registers POS sales (Point Of Sale) in a text file with following structure:

  • The file can contain two types of records: sales and customer returned products
  • The record identifier is three characters, starting at position 0. Valid values are: SAL for sale and RET for return.

The record structure for each record type is:

A Sale record has the following structure: the record Identifier is three characters, "SAL" in this example, and then a list of CSV fields containing sale attributes.

SALTransactionID,SKU,Quantity,UnitRetailPrice

A return record has the following structure: the record identifier is three characters, "RET" in this example, and then a list of CSV fields containing return attributes.

RETTransaction,SKU,Quantity

SAL and RET values are record identifiers that tells DTE which record to choose when decoding a line in the source file.

We need to import sales and returns in two different tables in a database. For simplicity, and because not everybody is able to install a database, I changed the requirement with: "We need to separate the file in two different CSV files: sales and returns. The destination files will contain the same number of fields in the corresponding source record. In addition the sale file will contain another field with the record's identifier value".

To accomplish this, we create a file with test data. This file is called POSSales_In_All.txt. The second step is to define the descriptor for the RecordBase connector. The file name is RecordBasedFile1.xml and contains the following elements:

  • Record_identifier is defined as the first three characters in the record (SAL and RET). Notice that for the SAL record we have the dest_column_index attribute defined because we need to copy the identifier in the destination field number four.
  • Two record_identifier_type elements to map each record identifier with a record description. SAL is mapped to SaleSrc record and RET is mapped to ReturnSrc record.
  • Two record definitions one for each destination file: Sales and Returns. They contain the fields as they were defined in the test data file. An additional field RECORD_ID has been added to the SAL record to capture record_id value.

The complete source for RecordBasedFile1.xml is:

<?xml version="1.0"?>
<descriptor type="RECORD_BASED">
<record_identifier type="AUTO" start_pos="0" length="3" />

<!-- Describes the mapping between record name and record identifier -->
<record_identifier_type rec_id="SaleSrc" identifier="SAL" dest_column_index="4"/>
<record_identifier_type rec_id="ReturnSrc" identifier="RET" />

<!-- for now only FW and CSV record types are supported -->
<record type="CHAR_SEPARATED_VALUES" separation_char="," rec_id="SaleSrc">
<column name="TRANSACTION_ID"/>
<column name="SKU_NUMBER"/>
<column name="QUANTITY"/>
<column name="PRICE"/>
<column name="RECORD_ID" />
</record>
<record type="CHAR_SEPARATED_VALUES" separation_char="," rec_id="ReturnSrc">
<column name="TRANSACTION_ID"/>
<column name="SKU_NUMBER"/>
<column name="QUANTITY"/>
</record>
</descriptor>

Next, we must create the description files for each destination. The first destination description file is SalesOnly.xml :

<?xml version="1.0"?>
<descriptor type="CHAR_SEPARATED_VALUES" separation_char="|" rec_id="SaleDst">
<column name="TRANSACTION_ID"/>
<column name="SKU_NUMBER"/>
<column name="QUANTITY"/>
<column name="PRICE"/>
<column name="RECORD_ID" />
</descriptor>

We change the separation char into "|" (pipe), whereas in the source file we have "," (comma). This is just to show that we can define a very flexible scenario. The record contains all the fields from the source file plus another field, RECORD_ID, which will be populated with the record_identifier value, SAL.

The second destination descriptor file is ReturnsOnly.xml :

<?xml version="1.0"?>
<descriptor type="CHAR_SEPARATED_VALUES" separation_char="|" rec_id="ReturnDst">
<column name="TRANSACTION_ID"/>
<column name="SKU_NUMBER"/>
<column name="QUANTITY"/>
</descriptor>

We make the same change in the separation char, and for this record we don't add the additional field. This is not required because we don't need to copy the record identifier value, RET, to this record.

Next we define the transformation descriptor in the file TrxTest.xml :

<?xml version="1.0"?>
<transformation>

Source is defined as RECORD_BASED and we name it "POSAll". The two required properties are:

  • "Data Source" containing the path to the test data file.
  • "Configuration File" which contains the path to the RecordBasedFile1.xml file.

<source mode="RECORD_BASED" name="POSAll">
<property name="Data Source">E:\projects\DTE\misc\test_data\POSSales_In_All.txt</property>
<property name="Configuration File">E:\projects\DTE\misc\test_data\RecordBasedFile1.xml</property>
</source>

The two destinations for Sales and Returns are defined as CHAR_SEPARATED_VALUES and we name them: "SalesOnly" and "ReturnsOnly".

Each of them has the two mandatory properties defined as follows:

  • "Data Source" as SalesOnly_Out.txt and respectively ReturnsOnly_Out.txt
  • "Configuration File" as SalesOnly_Out.xml and respectively ReturnsOnly_Out.xml

<destination mode="CHAR_SEPARATED_VALUES" name="SalesOnly">
<property name="Data Source">E:\projects\DTE\misc\test_data\SalesOnly_Out.txt</property>
<property name="Configuration File">E:\projects\DTE\misc\test_data\SalesOnly.xml</property>
</destination>
<destination mode="CHAR_SEPARETED_VALUES" name="ReturnsOnly">
<property name="Data Source">E:\projects\DTE\misc\test_data\ReturnsOnly_Out.txt</property>
<property name="Configuration File">E:\projects\DTE\misc\test_data\ReturnsOnly.xml</property>
</destination>

We do not define any client object for this test application.

The mappings elements define the transformation of source fields into destination fields, which in this case is symmetrical. Each source field goes to the corresponding destination field:

<mappings source_rec_id="SaleSrc"
            dest_rec_id="SaleDst" dest_name="SalesOnly">
<map
            destination_column_index="0" source_column_index="0"/>
<map
            destination_column_index="1" source_column_index="1"/>
<map
            destination_column_index="2" source_column_index="2"/>
<map
            destination_column_index="3" source_column_index="3"/>
<map
            destination_column_index="4"
            source_column_index="4"/>
</mappings>
<mappings
            source_rec_id="ReturnSrc" dest_rec_id="ReturnDst"
            dest_name="ReturnsOnly">
<map destination_column_index="0"
            source_column_index="0"/>
<map destination_column_index="1"
            source_column_index="1"/>
<map destination_column_index="2"
            source_column_index="2"/>
</mappings>
</transformation>

The last step is to build a C# console application in our solution to test the transformation process. In the solution that comes with this case study, you already have this project created. Open the Test project class TestDTE and you'll see following source code:

using System;
using DTE;

namespace
            TestDTE
{
/// <summary>
/// Summary description for
            TestDTE.
/// </summary>
class TestDTE
{
///
            <summary>
/// The main entry point for the
            application.
/// </summary>
[STAThread]
static void
            Main(string[] args)
{
TestDTE tstApp = new
            TestDTE();
tstApp.Test1();
}

public void
            Test1()
{
DTEApplication appDTE = new
            DTEApplication();

if
            (appDTE.StartTrxProcess("E:\\projects\\DTE\\misc\\test_data\\TrxTest7.xml"))
Console.WriteLine("Operation
            successed.");
else
Console.WriteLine("Operation
            failed.");
}
}
}

This is a very simple console application that uses the DTE namespace. The main function creates a new DTEApplication object and calls the StartTrxProcess method with a parameter containing the path to the transformation descriptor file.

After the application terminates you should be able to see the two files created with the following content:

SalesOnly.txt
10045|256MbRAM|3|67.99|SAL
10046|DELLP4+PRT|1|1867.95|SAL
10047|SCAN16x12|2|78.99|SAL

ReturnsOnly.txt

10045|256MbRAM|1
10047|SCAN16x12|1

Application Setup

To setup the DTE framework you need to download the source code provided with the case study and unpack the zip file to a folder. Open the DTE.sln file which will start Visual Studio .NET and open all projects in this solution.

In order to be able to test the framework you need to modify following files:

DTEApplication.cs in project DTE, line 36:

_factoryReader.Load("E:\\projects\\DTE\\testdata\\DataFactory.config.xml");

Change the path to where you saved the file DataFactory.config.xml. We'll change this in the next version to read this information from the application configuration file.

Change the path to the folder where you unpacked the zip file in the following files:

  • TrxTest.xml
  • RecordBased1.xml
  • SalesOnly.xml
  • ReturnsOnly.xml

To execute the application you must move to the Test project and hit F5 key. This will start the application in debug mode and you'll be able to see if any error messages are displayed during execution.

Case Study Review

Now that you have read through this case study you've seen why Data Transformation Engine represents one of the solutions to a problem that is critical to data processing applications or any other application that needs to easily be able to gather information from diverse sources. This might seem like an old problem with all the standards that we see today in the industry regarding data manipulation and data transfer, like EDI, SOAP or XML, however, there are still many companies and organizations that prefer to stick with current setup and millions of applications still produce data in proprietary formats.

This case study gave you the opportunity to observe how this application was designed and how we made use of design patterns to create reusable code and make room for extensibility. Design patterns are perhaps the most powerful thing not used enough in software development today, and software engineers should perhaps use them more often.

DTE also gave us the opportunity to show you how to make use of new features in .NET Framework, like dynamic loading of assemblies and dynamic invocation of methods on object instances. These are very powerful features previously available only to Java developers. The adoption of metadata and self described classes and assemblies in .NET make it possible for us developers to design better applications and implement a large variety of features not possible before, like the one we've seen in DTE. Imagine how cumbersome it would have been to create a mechanism for object instantiation at run-time knowing only the name of the class and possibly the DLL in which it is defined.

Further Work

The DTE framework presented in this case study is only the first version, and we have in mind a few features that will be added in future versions, such as:

  • Implement the piece of source code that allows the client observer object to determine the type of record in the data source based on raw information. We have designed the interface and the method but we must enable the source code in the DTEApplication class.
  • Build a user interface that allows clients to easily build descriptors and configure DTE settings.
  • Extend the list of connectors with a very interesting one: an Object that can act as source or destination. In this case, the client application will either pass a parameter to the process indicating an instance of the object or instruct the DTE to get/create an instance of the object. This will give DTE the capability to act not only as transformer but also as data source or data consumer.
  • Extend the features for record based connector to allow:
  • Developers to specify a different column where record identifier value goes.

Case Study Information
Author | Catalin Tomescu
Editor | Adam Ryland
Reviewers | Bill Wilson, Matthew MacDonald, David M Williams

If you have any questions or comments about this case study, or suggestions for further pieces, please contact the technical editor. If you wish to talk about the case study with other subscribers, please visit our forum here.


Multi-threading In C#

Multi-threading In C#
http://www.csharptoday.com/content.asp?id=1841
Article Rating: 4
Published on May 20, 2002
Abstract
C# and the .NET platform offer developers unprecedented support for threaded applications targeted to run on Windows. This opens up a variety of opportunities and problems for developers who are interested in exploiting the power of threads. In this case study, Ramon Arjona addresses several of these opportunities, and outlines solutions to many of these problems by examining an implementation of a Quote of The Day (QOTD) server. We discuss the appropriate use of threads in an application, and touch on the important differences between threading in C#, Java, and traditional Microsoft Foundation Class (MFC) development.
Case Study

Editor's Note: Please remember that we are always open to feedback from our subscribers! If you have comments on the direction of C#, the content of our case studies, suggestions for future themes, etc, please don't hesitate to mail me at here.

C# and the .NET platform offer developers unprecedented support for threaded applications targeted to run on Windows. This opens up a variety of opportunities and problems for developers who are interested in exploiting the power of threads. This case study addresses several of these opportunities, and outlines solutions to many of these problems by examining an implementation of a Quote of The Day (QOTD) server, which should be familiar to us from its many popular incarnations targeted to the UNIX and Linux platforms. We discuss the appropriate use of threads in an application, and touch on the important differences between threading in C#, Java, and traditional Microsoft Foundation Class (MFC) development.

Introduction

Everybody needs to multi-task in today's fast paced technical environment. For instance, while I am writing this case study, I am also microwaving a gourmet burrito dinner, listening to blues music, and checking the clock to see if I'll actually make my deadline. My computer, likewise, is doing several things at once. It is accepting my keyboard input, playing my blues CD, and monitoring my Internet connection. If I - or my computer - could only manage one task at a time, tasks would pile up and life would rapidly grind to a halt while I waited for each individual task to complete.

The need to do multiple things at once is what makes multi-threaded programming techniques necessary. You will often hear threads referred to as lightweight processes (LWPs), but I find that it's useful to think of a thread as a "unit of work." For instance, the task "microwave dinner" runs on one thread, while the task "listen to blues" runs on another. At the moment, I'm doing about as many things at once as I can - I've got tasks running on every available thread. If my wife comes home and asks me to do something, I'll need to interrupt one of these threads (i.e., put my dinner down) or she will have to wait until one of the threads I'm running dies or goes to sleep (i.e. my CD stops playing).

Most modern desktop operating systems support multi-threading. Even Mac OS X, with its POSIX-based kernel, has native thread support. Developers targeting operating systems that do not support threads natively could still leverage the power of multi-threading - after a fashion - if there is a Java Virtual Machine (JVM) for that OS, because the JVM can implement a kind of logical threading model that is divorced from the underlying OS and hardware, called "green threads". (You don't need to know anything about green threads for this case study, and I sincerely hope you never need to know anything about them, ever. My point is that multi-threaded programming is possible, though perhaps ill-advised, even on operating systems that don't have native thread support.) On the Windows platform, support for threads has been available since the days of Windows 95 and Windows NT.

So why haven't all Windows developers been busily developing multi-threaded applications over the past six years? Some of us have, but many of us have been content to continue to program in a "synchronous" model, waiting for one task to complete before moving on to the next one.

Partly this is because of the cryptic implementation of threading in COM and MFC. In COM, in order to program a multi-threaded object, you would sometimes have to do something called "aggregating the free-threaded marshaller." You would also have to understand the differences between the various apartment models in COM threading: single threaded apartments, multi-threaded apartments, and neutral apartments.

Doesn't that sound complicated? Even for seasoned coders, it could get frustrating. And don't forget that in order to implement a multi-threaded program on Windows, you had to code it using C++. Visual Basic developers, until the release of Visual Basic.NET, had no easy way to implement a multi-threaded program. Of course, you could always have resorted to implementing your multi-threaded application in the Java language, and some of us did - only to find that integrating Java with COM could be a pain and that Java threads have, well, some room for improvement. (There is an excellent article detailing many of the problems with Java threads and what could be done to fix them. I won't repeat any of it here, but if you're interested the link can be found at the end of this case study.)

Now, with the advent of .NET and the magic of the Foundation Class Library, all of this has changed. Windows developers now have a straightforward, easy to understand idiom for implementing threaded applications, and we don't even have to tie ourselves to a particular language to use it - any language that targets the Common Language Runtime (CLR) can leverage .NET threads (even poor old VB).

The problem I've been faced with is a relatively straightforward one: build a server that implements the same behavior as the QOTD daemon on UNIX, which powers the familiar fortune game on UNIX and Linux. The QOTD service needs to maintain a list of pithy quotes, and spit them out randomly over a TCP channel to anyone who connects to the appropriate port. While QOTD is entertaining, it is definitely not "just for fun". From the work that goes into QOTD comes the infrastructure necessary to implement chat daemons, mail daemons, and all other network services that must support large numbers of concurrent users. In fact, I started working on QOTD as a sidetrack while building an irc daemon from scratch in C#.

What Are Threads Good For?

Certain kinds of problems obviously lend themselves to multi-threaded solutions. For instance, if we have an application that will run on a multi-processor box, we can use threads to take advantage of parallel processing. While one of our application threads is executing on Processor 1, another thread can be executing on Processor 2. This allows us to make efficient use of the entire machine and can buy us significant performance gains.

Threads are also useful when the responsiveness of an application is an issue. For instance, if we have a long task that we want to execute, we likely don't want our application to sit idly by while it waits for the long task to complete. Instead, we can spawn a new thread on which the long task can run, and allow it do its work while our application gets on with other tasks. Meanwhile, this long running task is processed asynchronously, and lets us know when it's done via a callback method.

Threading can also be useful when we want to share resources between users. Web servers, for instance, wouldn't be too useful if only one user could request a given HTML page at a time.

However, when making the decision to use threads or not, it is important to weigh any potential benefits against the cost of implementing and debugging a multithreaded solution. Multi-threading raises a host of issues - like concurrency management, contention issues, and deadlocks. Synchronous solutions can simply pretend that these issues don't exist.

Many other writers have said it, but I'll say it again: multi-threading is not a magic bullet. I'm reminded of a recent post to the comp.programming.threads newsgroup, in which the poster was asking for help refactoring threads out of his application. Weigh your options carefully, and choose your solution set with care.

System Requirements

The code in this case study requires Windows 2000 Server and the released version of the .NET Framework SDK. You do not need to have a network connection, but you do need to have a network card or modem and TCP/IP installed correctly for this to work. You should have a good grasp of C# and of .NET Intermediate Language (IL). C# does not require you to have a particularly low level knowledge of threads or threading, and neither will this case study. If you're interested in the implementation of threads at the kernel level, look at the links at the bottom of this case study.

Defining The Problem

The problem is a straightforward one: we need to build a service that will run on Windows 2000 and support a large number of concurrent users who will connect to the service via TCP/IP. This presents us with several problems: How do we manage concurrent read/write access to our data? How do we deal with organizing and queuing multiple user requests? How can we debug the server, and how can we profile its performance once we're done?

Case Study Structure

The case study is broken into three parts:

  • The first part will examine the initial design and implementation of the application, using a command console
  • The second part will describe the test driver for the application, and changes that were made once test results were derived
  • The third part will show how the application is installed as a service.

Database Design and Structure

To implement a QOTD service, first we need a place to store our quotes. My initial thought was to replicate the file-based storage mechanism that the UNIX-based application uses. In this implementation, quotes are organized into files based on topic. For instance, all of the quotes that are drawn from C#Today would be put into a file of the same name. Quotes are separated from each other by a line containing only a percent (%) character.

I abandoned this approach early for two reasons. First, it's 2002 and there are better ways for storing data than flat files. Second, the quote files I had access to are part of a Linux distribution. Like the rest of the Linux distribution, they are covered under the GNU Public License (GPL).

Instead, I chose to use an ADO.NET dataset, which would live in memory for the lifetime of the service, and be persisted to disk as needed.

If the dataset was created previously and persisted to disk, the application loads the dataset. If the dataset has not been created before, the application creates it and immediately serializes it. It is important that we always serialize the schema as well as the data, so ADO.NET understands the type of data it is looking at, and the relationship between the datatables. The dataset has the following schema, which is automatically generated for us by ADO.NET:

<xs:schema id="NewDataSet" xmlns="" xmlns:xs="http://www.w3.org/2001/XMLSchema"
xmlns:msdata="urn:schemas-microsoft-com:xml-msdata">
<xs:element name="NewDataSet" msdata:IsDataSet="true">
<xs:complexType>
<xs:choice maxOccurs="unbounded">
<xs:element name="QuoteType">
<xs:complexType>
<xs:sequence>
<xs:element name="TypeID" msdata:AutoIncrement="true" type="xs:long" />
<xs:element name="TypeName" type="xs:string" minOccurs="0" />
</xs:sequence>
</xs:complexType>
</xs:element>
<xs:element name="Quote">
<xs:complexType>
<xs:sequence>
<xs:element name="TypeID" type="xs:long" />
<xs:element name="Quote" type="xs:string" />
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:choice>
</xs:complexType>
<xs:unique name="PK_QuoteType" msdata:PrimaryKey="true">
<xs:selector xpath=".//QuoteType" />
<xs:field xpath="TypeID" />
</xs:unique>
<xs:unique name="PK_Quote" msdata:PrimaryKey="true">
<xs:selector xpath=".//Quote" />
<xs:field xpath="TypeID" />
<xs:field xpath="Quote" />
</xs:unique>
<xs:keyref name="FK_QuoteTypeID" refer="PK_QuoteType" msdata:ConstraintOnly="true">
<xs:selector xpath=".//Quote" />
<xs:field xpath="TypeID" />
</xs:keyref>
</xs:element>
</xs:schema>

Note that I use two data tables, one for quote type and one for quotes. This is inspired by the Linux implementation, which organizes quotes into files according to type, and is largely an administrative convenience. I could probably have gotten away with a single datatable, just for quotes - this would have become difficult for a human to read, however, as the set of quotes grows larger. In the code that creates the dataset, I specify that each quote type must be unique. I require each quote to have a corresponding quote type, and I also require that a quote/quote type combination be unique. The UniqueConstraint and ForeignKeyConstraint objects available in ADO.NET make this a straightforward task, and obviate the need for an external database engine, like MS SQL Server or Access.

Now that we have a structure to store our data, the question arises: where does our data come from? Again, my first thought was to use the quote files on the Linux implementation - I could easily have written a script to parse the files, extra the data, and add them the dataset. However this seemed like cheating to me, because of my concerns about the GPL that covers these files as I mentioned above.

I could have tried filling the dataset myself, with my favorite pieces of literature and witty quips I've collected over the years. However, as much as I like to think I've got a sharp mind and a keen grasp of literature, I doubt that I'd be able to come up with a significant volume of interesting material.

Finally, I decided that I'd allow users to connect to an administrative service and add quotes themselves. Writing the administrative service, I decided, would be much more rewarding - and much less work - than trying to think of interesting things to include on my own. (This, perhaps, is why I am a programmer and not a politician.) The implementation of the administrative service appears in the section below on component design and coding.

Component Design and Coding

QuoteLibrary Object

The QuoteLibrary object is where our quotes actually reside. It derives from System.Data.DataSet, because the ADO.NET dataset object already addresses much of the problem domain surrounding the storage of our quotes, including the creation of referential integrity constraints, identity columns that autogenerate new ids as we add rows to a table, and serialization.

An instance of the QuoteLibrary object is shared by the QuoteEngine object and the QuoteAdmin object. The QuoteEngine object only needs to read from this resource, but the QuoteAdmin object needs to read from it and write to it. The QuoteLibrary object itself doesn't know anything about the objects that are calling it - but it does not trust that it's callers will guarantee that they are making their calls in a thread-safe fashion. Therefore, the QuoteLibrary object takes it upon itself to ensure that its data is used in a thread safe fashion by locking itself during every read and write operation.

At this point, we should take a moment to define what we mean by "thread-safe", and explain why it's a good idea for the QuoteLibrary to manage its own thread-safety.

A thread, as we said above, is a "logical unit of work." Many of these "logical units of work" can be active in a given application domain at the same time. Each thread maintains its own stack, which contains data that is local to that particular thread. All of the threads in an application domain, however, share resources like memory, file handles, and database connections. When we say that our application is "thread-safe", we mean that the threads share these resources nicely - without, for instance, clobbering data out from under each other.

Let's pretend that my wife and I are two threads in an application domain. Think of the application domain as our "house." We have our own data and our own local stack (for instance, my "To Do" list and her "To Do" list), but we share other resources (for instance, our kitchen and our checkbook). Sharing the resources in our house works out nicely most of the time, as it would become prohibitively expensive to build two kitchens - but we have to share nicely.

Essentially, the problem is that there is no way to guarantee when a particular thread will do something to a shared resource. The scheduling of threads is largely up to the CLR, though we can influence the amount of time the CLR grants to a running thread by increasing it's priority. Each thread is granted a slice of processor time to do its work, after which the CLR allows other threads a chance to execute. For instance, Thread A might read data from a file in preparation for some long analytical operation. The CLR then gives Thread B a chance to execute, and Thread B writes some data to the end of the shared file. Thread A is then allowed to resume executing and, because it knows nothing about the work Thread B just did, it over-writes the Thread B's changes as it is writing back to the file.

We need, therefore, to provide a means for our threads to synchronize their actions when dealing with shared resources. Otherwise, concurrency bugs will appear and chaos will ensue as threads trip over each other while they do their work. When we deal with our shared resources in a way that avoids this chaos by using proper synchronization, we can say that we are being "thread safe."

The .NET runtime provides a variety of synchronization mechanisms, some of which we will deal with in this case study. The most easily implemented synchronization mechanism in C# is the lock keyword:

lock(this)
{
this.Tables["QuoteType"].Rows.Add(row);
this.AcceptChanges();
this.Write();
}

This keyword places a lock on an object. Only one thread can hold the lock at a time. If a thread tries to acquire this lock while another thread already holds it, this first thread goes into a sleeping state while it waits for the lock to be released. The lock lasts for the duration of the code in the curly braces following the lock statement.

Now, we don't have to implement locking in the QuoteLibrary object itself. We could, instead, choose to trust the callers of the QuoteLibrary object to implement locking properly - but this approach is naive and should be avoided. The callers of QuoteLibrary should not have to know how the object manages its resources.

Details such as locking, and the implementation of locking, should be opaque to objects that use QuoteLibrary. The objects that use QuoteLibrary, after all, should only care about the public methods and properties that it exposes. They should not care about private implementation details, such as the way QuoteLibrary chooses to manage its data. Allowing implementation details like this to "bleed" into calling classes violates the principle of encapsulation and therefore undermines an object-oriented design methodology. This in turn makes changes and optimizations to the QuoteLibrary object more difficult to implement.

For instance, in the section below on profiling the application, we will see that the lock keyword is a suboptimal solution for our problem. If we had relied on the calling objects to implement locking, we would have had to make code changes in each of the callers to alter our method of synchronization. Since each caller would essentially duplicate the synchronization code, we'd have to track down every place where this code appears and make our change - an annoying and error prone process.

We're much better off allowing the objects that encapsulate our shared resources to manage their own synchronization whenever possible.

An example of this is illustrated by the code snippet below, taken from the QuoteLibrary object, illustrating the way this object manages its own synchronization by placing a lock around the this pointer:

private void
            Write()
{
lock(this)
{
this.WriteXml("quotelib.xml",
            XmlWriteMode.WriteSchema);
}
}

QuoteEngine object

The QuoteEngine object is the heart of our QOTD service. This object implements our core functionality, listening on a TCP port and serving pithy quotes to all the clients who care to connect.

These days, writing a service using C# and .NET is much easier and more straightforward than it has been in the past. However, there is still a level of rigmarole surrounding the creation and installation of a service. During the early phases of my development, I chose to implement the QuoteEngine inside of a command console. This made it easy to start, stop, and recompile the application as I experimented with a series of minor changes. Transitioning from a command console to a service is straightforward, and is detailed below. I use the precompiler directive feature in C# to control the debugging messages that are output to the command console, but that I want to suppress them when the QuoteEngine is running as a service:

#if Console
Console.WriteLine("Current QuoteCount: " + quoteCount.ToString());
#endif

The QuoteEngine object exposes a DoStart method. The signature of this method is wrapped in a ThreadStart delegate, and passed into the constructor of a Thread object.

When we first create an instance of a Thread, its ThreadState property is set to Unstarted. This means that the Thread instance exists, but isn't doing anything yet. We call the Start method of the Thread to set it about doing its work, setting its status to Running. The function we passed to it in the ThreadStart will now start executing on this thread.

In the DoStart method of the QuoteEngine object, I first test to see if the caller has set the Name property of the thread. A thread's name property is "write once", meaning that this property can be set once and only once. Any further opportunity to change the name of the thread causes an exception to be thrown. I find that it's useful to name the main threads in an application, as it can greatly simplify debugging. However, there is no way the QuoteEngine object can guarantee that the caller who spawned the thread has not already set that threads name. Therefore, it is necessary to test this property before trying to write to it, or be prepared to catch the InvalidOperationException that will be thrown if the property has already been set.

Once this is done, we create an instance of the QuoteLibrary object. Then we create an instance of the QuoteAdmin object, which encapsulates the functionality that allows users to add quotes to our QOTD server, and we pass a reference to the QuoteLibrary object to our QuoteAdmin object. The QuoteEngine and QuoteAdmin object now share the same QuoteLibrary. QuoteEngine will read from it, while QuoteAdmin will attempt to read from it and write to it.

Next, we start a TcpListener on the port that was supplied with our start-up arguments, or on port 6060 if no port was supplied. We enter an infinite loop, waiting for incoming user requests. If we have no pending clients on the socket, we send the thread to sleep for 1000 milliseconds since we have nothing to do and can afford to yield the processor to other threads that might have work that needs to get done.

If we do have a pending client, we wrap a reference to the QuoteEngine's DoWork method in a WaitCallBack delegate, and pass this delegate into the QueueUserWorkItem method of the ThreadPool object.

ThreadPool is a very useful object. We don't, after all, want to spawn a new thread for every user who connects to our application. If we did, we could rapidly have more threads going at once than the CLR could comfortably handle. We'd see our system spending more time just managing the threads we've spawned, and less time doing actual work.

ThreadPool helps us avoid this problem and manage our threads more efficiently by providing us with a pool of threads that are managed for us by the system. When we first reference ThreadPool in a process, a pool of threads is created for us with a default size of 25 threads per available processor. We add work items to the ThreadPool's queue - in this case, the work item is the output of a pithy quote to our clients - and the ThreadPool sees to it that the work item is serviced once it has a thread available in the pool.

ThreadPool is an implementation of what in threading literature is frequently called the Producer/Consumer design pattern. If the Foundation Class Library did not provide us with a basic implementation of this pattern, we'd probably all spend a lot of time rolling our own. Of course, in some very specific circumstances you may still want to roll your own producer/consumer implementation - but these circumstances should probably not occur very often. The ThreadPool object should be more than sufficient for most cases when you'll need a producer/consumer implementation.

The DoWork method itself is fairly trivial. It gets a quote from the QuoteLibrary object and returns it to the TCP client. If we lose the client for some reason, we just ignore it. If they really want to hear something from QOTD, they'll be back anyway. We close the client connection as soon as we've sent the quote, to keep our use of network resources to a minimum.

QuoteAdmin object

The core of the QuoteAdmin object is essentially the same as the QuoteEngine. There is a DoStart method that handles the initial start-up of our object, establishing a TcpListener on port 6061, and queuing work items to the ThreadPool.

The DoWork method of QuoteAdmin, however, is more substantial because we need to implement a means for users to input information. We therefore implement a simple protocol, with a small set of commands, and we parse the user's input as it is entered. For now, we presume that the user is connected to our service via a simple telnet session, so our input buffer is only 4 bytes wide - just wide enough to receive a single character, which we then append to a StringBuilder object. Once we've received a newline character, we know that the user has finished telling us what they wanted to tell us and we can process their command.

This method is prepared to deal with two kinds of exceptions - DataExceptions raised when a user tries to add information that violates the referential integrity constraints we've established in our ADO.NET data set, and Format Exceptions that occur when the user attempts to send us a malformed string. In both cases, the user is simply apprised of the error and we move happily on.

The Test Driver

Once I finished the QOTD service itself, I needed a way to test it. I built a simple, console-based application that attempts to connect to the QOTD service over and over. If you're going to run the test application yourself, it is best to run it on a separate machine than is running the QOTD server. Otherwise the QOTD service will have to contend with the test driver for processor time, which is not quite what we want for our test case.

The test driver spawns a large number of individual threads. These threads attempt to connect to the QOTD server, and then go to sleep for a random number of milliseconds between 200 and 1000 before trying to connect again. While running this test application, I used the performance monitor and the counters for CLR threads to monitor what was going on with the server.

One thing that might be of some interest in the test driver is the use of Interlocked.Increment. The Interlocked object supports a set of static methods that provide for atomic operations on variables that multiple threads share. In other words, an increment or decrement operation happens with a single operation, and is guaranteed to be unaffected by context switching between threads. The familiar post-fix increment operator (i.e. ++i), for instance, actually performs a fetch operation, then an increment operation, which could lead to strange behavior if the variable being incremented is shared amongst several threads, all of which are subject to context switching by the operating system.

I would not, however, recommend using the Interlocked object the way I do in this test driver. Based on the high amount of contention I noticed while watching the test driver itself in the performance monitor, it's clear that I am spawning more threads than is optimal, and dealing with my resources in a less-than-ideal way. It's good enough for our purposes, as it is a simple test driver meant to give our service a workout, but I do not want to leave you with the impression that this is a good way to implement production code that you intend to distribute.

The Performance Metrics

After running my first test, I was disappointed to see that contention was occurring on the server. Since the test driver only does read operations, I'd prefer that there not be any contention occurring at all. After all readers aren't trying to do anything with the data, they only want to look at it. I'd prefer that they not experience any contention for shared resources unless there is a write operation taking place on the shared resource through the QuoteAdmin object.

This led me to examine my use of the lock keyword. While it is handy that lock is built into the C# language, it is important to remember that it is not the ideal synchronization mechanism for every situation. At the IL level, the lock keyword is equivalent to implementing a Monitor object. The following output from ildasm run against the original version of the QOTDConsole application bears this out:

IL_0025: /* 28 | (0A)00000B */ call
void [mscorlib/* 23000002 */]System.Threading.Monitor/* 0100000F
*/::Enter(object) /* 0A00000B */
.try
{
IL_002a: /* 02 | */ ldarg.0
IL_002b: /* 28 | (0A)000007 */ call
instance class [System.Data/* 23000001 */]System.Data.DataTableCollection/* 0100000C */
[System.Data/* 23000001 */]System.Data.DataSet/* 01000001 */::get_Tables() /* 0A000007 */
IL_0030: /* 72 | (70)0000A7 */ ldstr "QuoteType" /* 700000A7 */
IL_0035: /* 6F | (0A)000008 */ callvirt
instance class [System.Data/* 23000001 */]System.Data.DataTable/* 0100000D */ [System.Data/*
23000001 */]System.Data.DataTableCollection/* 0100000C */::get_Item(string) /* 0A000008 */
IL_003a: /* 6F | (0A)00000C */ callvirt
instance class [System.Data/* 23000001 */]System.Data.DataRowCollection/* 01000003 */
[System.Data/* 23000001 */]System.Data.DataTable/* 0100000D */::get_Rows() /* 0A00000C */
IL_003f: /* 06 | */ ldloc.0
IL_0040: /* 6F | (0A)00000D */ callvirt
instance void [System.Data/* 23000001 */]System.Data.DataRowCollection/* 01000003
*/::Add(class [System.Data/* 23000001 */]System.Data.DataRow/* 0100000E */) /* 0A00000D */
IL_0045: /* 02 | */ ldarg.0
IL_0046: /* 28 | (0A)00000E */ call
instance void [System.Data/* 23000001 */]System.Data.DataSet/* 01000001 */::AcceptChanges() /*
0A00000E */
IL_004b: /* 02 | */ ldarg.0
IL_004c: /* 28 | (06)000003 */ call
instance void QOTD.NET.QuoteLibrary/* 02000002 */::Write() /* 06000003 */
IL_0051: /* DE | 07 */ leave.s IL_005a

} // end .try
finally
{
IL_0053: /* 07 | */ ldloc.1
IL_0054: /* 28 | (0A)00000F */ call
void [mscorlib/* 23000002 */]System.Threading.Monitor/* 0100000F */::Exit(object) /* 0A00000F
*/
IL_0059: /* DC | */ endfinally
} // end handler

The Monitor object does just what we expect: it only lets one thread into the critical section of code at a time. But for our purposes in the QOTD, this isn't exactly what we want. We'd like an object that lets as many reader threads into the critical sections of code as possible, but that also gives us a means of synchronizing our read operations with our write operations. What is needed here is a ReaderWriterLock object.

The ReaderWriterLock object is similar to the Monitor object, which the lock keyword so conveniently stands in for, but is ideal for multiple-reader scenarios.

It was easy to change the QuoteLibrary object to support this. I added a ReaderWriterLock object to the class as a private field. Then, I altered the implementation of the methods that perform read operations to use the ReaderWriterLock object as follows:

public string
            GetQuote()
{
quoteLock.AcquireReaderLock(Timeout.Infinite);
try
{
int
            quoteCount = this.Tables["Quote"].Rows.Count;
#if
            Console
Console.WriteLine("Current QuoteCount: " +
            quoteCount.ToString());
#endif
if(quoteCount ==
            0)
{
return "This server has nothing to say. Have someone add
            quotes to the quote file.";
} else {

Random r = new
            Random();
int rand = r.Next(quoteCount);
DataRow row =
            this.Tables["Quote"].Rows[rand];
#if
            Console
Console.WriteLine("Getting Quote: " +
            row["Quote"].ToString());
#endif
return
            row["Quote"].ToString();
}
}
finally
{
quoteLock.ReleaseReaderLock();
}
}

The methods that perform write operations were changed in a similar fashion, invoking AcquireWriterLock(Timeout.Infinite) at the top of their method body. The call to AcquireReaderLock will only block if a different thread owns the writer lock, while the call to AcquireWriterLock will block if another thread owns the reader lock. Once a writer lock is requested, however, no new readers are allowed until the writer lock is released. The documentation claims that this implementation attempts to be "fair", ensuring that neither writers nor readers are perpetually denied access.

We put the call to AcquireReaderLock and ReleaseWriterLock in the finally block, to ensure that we always let go of our locks properly. Otherwise, we might end up holding onto the lock forever and causing undesirable behavior.

This small change eliminated contention from the server while running the test driver, and also reduced the amount of memory QOTD consumed.

A Word about Thread.Abort

I have run across some controversy and confusion surrounding the Thread.Abort method, so I would like to spend some additional time talking about what it is and what it is not.

First, as I mentioned above, the Thread.Abort method signals that a Thread object is being aborted by throwing an unchecked ThreadAbortException. Some newsgroup posters feel that throwing an exception this way is "inelegant." The gist of their argument, as I understand it, is this: since the stopping of a Thread is an expected case, rather than an error case, we shouldn't use an exception to report it. This, in my opinion, stems from a commonly held misperception about exceptions and exception handling, and shouldn't influence the way we code our multithreaded applications.

Simply put, an exception is not the same as an error. An exception might be used to report an error condition, but it can also be used to report a condition that we expect, but that violates some basic assumption that is intrinsic to the operation we're executing at the time.

If we're building a library that other developers will use to perform HTTP client operations, for instance, we expect a variety of cases that violate our intrinsic assumptions. Web servers, for instance, will occasionally return a header indicating that a resource has moved. This goes against what should be one of our intrinsic assumptions - that when we request a resource over the Web, we'll get a meaningful stream of bits back representing that resource. However, what does the application code need to do about it?

Well, as the library developers, we don't know what the application code needs to do about it, because we're not necessarily the ones writing the application that calls into our library. Some applications may want to follow the resource moved header to its destination, hopefully finding the resource we wanted in the first place at the end of it. Other applications may want to give up in despair after the first try. We need to tell the calling application that there's a condition that violates our assumptions, and then let that application do whatever it needs to do about the condition based on whatever the application's purpose is - so we throw an exception. (We could return a cryptic error code, like an HRESULT, but exceptions are much cleaner and easier to debug.)

An in depth discussion of the theory and practice of exceptions is beyond the scope of this case study. My only point is that there is nothing inherently ugly about using an unchecked exception to notify the system that a Thread is being stopped. For a more detailed view of exceptions and exception handling in .NET and C#, you might want to consult Jeffrey Richter's "Applied Microsoft .NET Framework Programming". There are also a number of good books published by Wrox that cover this topic.

Second, I have frequently heard the Thread.Abort method compared to the deprecated stop() method that appears in the Java language. The similarities between these two methods are completely superficial, and in my opinion comparisons between them are not useful.

Thread.Abort and Java's stop() both throw unchecked exceptions to notify the system that a thread is being stopped. (In Java, the type of this exception is ThreadDeath.) That's pretty much all they have in common.

Java's stop() is very difficult to use, and very brutal in its execution. For instance, if we call stop() on a Java thread while executing a finally block, the rest of that finally block won't execute. If we call stop() on a Java thread while that thread is doing something through the Java Native Interface (JNI), we have no guarantee that it will actually exit from that JNI call before it ends. Simply put, Java's stop() is a key ingredient in a recipe for disaster. That's why it's been deprecated.

Thread.Abort, according to the specification and according to my experience so far, suffers from none of these problems. Thread.Abort politely lets finally blocks execute, and while this might result in a Thread never actually dying if we, for instance, create an infinite loop in our finally block (which would be a Bad Thing in any case, with or without multi-threading), it is certainly better than arbitrarily breaking out of a finally block that we were counting on. (The point of finally, after all, is that it always runs, whether there was an exception or not.) As we mentioned above, Thread.Abort also waits politely while we exit from unmanaged code. As with the finally block, this could create a condition in which a Thread is never given an chance to stop - but it is preferable to exiting three-quarters of the way through a JNI call, leaving memory mangled and important resources unreleased.

My point here is: do not be afraid of Thread.Abort, even if you are a Java coder who has been traumatized by stop(). They are not the same thing. Thread.Abort is much easier to use, and in my opinion much less likely to result in strange errors or dire frustration than Java's stop().

That Old Black Magic

Since I've already digressed a little to talk about Thread.Abort, it won't hurt if I digress a little further and talk about the .NET runtime's internal implementation of Threads. If you're anything like me, you're probably curious about how threading is implemented in .NET, and how it relates to the threading library in the underlying platform.

Well, if you decompile MSCorlib.dll using ILDASM and go looking for the interesting bits of the Threading implementation, you'll see something like this:

.method public hidebysig instance void Abort() cil managed
{
// Code size 7 (0x7)
.maxstack 8
IL_0000: ldarg.0
IL_0001: call instance void System.Threading.Thread::AbortInternal()
IL_0006: ret
} // end of method Thread::Abort

.method private hidebysig instance void
AbortInternal() cil managed internalcall
{
} // end of method Thread::AbortInternal

Doesn't look like there's a whole lot going on, does it?

The internalcall keyword is what's important here. It tells the compiler that this is a "manually managed" method. Essentially, the method is built into the .NET runtime itself, an implementation detail that is hidden from our immediate view, implemented in native code. I have yet to find any documentation about internalcall and "manually managed" methods outside of unofficial admonitions to avoid them as difficult to implement sources of bugs, that lead developers to wail and gnash their teeth. So, at least for the time being, we must be content with trusting the system and knowing that a "miracle" occurs, and allows our threads to work.

Mutexes

Before we go on to build our service, there is another synchronization mechanism we ought to touch on: the Mutex. The idea of a Mutex is common to most threading implementations. Much like the monitor object, it allows us to restrict access to a segment of code to one thread at a time. The name "mutex" is derived by squishing together the words "mutually exclusive".

When would you want to use a Mutex instead of a monitor? Well, if you need to synchronize access to a piece of code across process boundaries, or ApplicationDomain boundaries for that matter, you should think about using the Mutex object. Since it derives from System.MarshalByRefObject it is appropriate for solutions that need to support remoting because by default it is passed by reference. The Monitor object, on the other hand, by default is passed by value. This means that when we hand a Mutex object off to a function, we are handing that function a reference which points back to our original Mutex object. This is important because we want our function to request ownership of this particular Mutex before attempting to enter a critical section of code. If the Mutex were passed by value, essentially creating a copy of the original Mutex object, it wouldn't help us at all - we'd have two different synchronization objects going on at the same time when we need to have one, and we'd quickly start tripping over our own feet. Think of it as an intersection with a traffic light. There needs to be a single, synchronized set of traffic lights at the intersection in order to avoid collision. If there are three traffic lights that operate independently, with no knowledge of one another, then a disaster is going to occur.

To create a Mutex object, we call one of its constructors as follows:

System.Threading.Mutex quotemutex = new System.Threading.Mutex(false,"QuoteMutex")

The first parameter is a boolean value, indicating whether the thread that is creating the mutex should also be set to the initial owner. By specifying false, we have created a Mutex that is initially unowned, or signaled. (When a Mutex object is unowned, it is said to be in the signaled state. When a thread owns the Mutex object, it is said to be in the unsignaled state.) The second parameter allows us to specify a string that is a name for our Mutex.

Then, to use the Mutex object, we pass it to the Thread that is doing our work, for instance via a public property:

QuoteAdmin qa = new QuoteAdmin();
//the admin app shares the library.
qa.quotelib = quotelib;
//the admin app also shares the Mutex object.
//the QuoteEngine and the QuoteAdmin both now have a reference
//to the same mutex object and can coordinate their actions through it.
qa.quotemutex = quotemutex;

The code in this function, for instance, will not execute until it has ownership of the Mutex:

public void AddType(string
            s)
{
quotemutex.WaitOne();
try
{
DataRow row =
            this.Tables["QuoteType"].NewRow();
row["TypeName"] =
            s;
this.Tables["QuoteType"].Rows.Add(row);
this.AcceptChanges();
this.Write();
}
finally
{
quotemutex.ReleaseMutex();
}
}

This function calls the WaitOne method, which the Mutex class inherits from the WaitHandle class. With no parameters, WaitOne will cause this thread to sit patiently forever while it waits to get ownership of the Mutex before proceeding with the rest of the code. We could, instead, specify a timeout value in milliseconds or using a Timespan object, if we wanted to give up after a while. These methods also require that we provide a boolean flag, indicating whether we want to leave the synchronization domain before the wait.

Our thread can call the WaitOne method multiple times without causing itself any ill effect. However, we should call the ReleaseMutex method once for every time we call WaitOne, to properly release ownership of the Mutex object. If our thread terminates normally while it owns the Mutex, the runtime will take care of signaling the Mutex and giving ownership to the next waiting thread. It's still a good idea to call ReleaseMutex, rather than waiting for our thread to terminate and then waiting for the runtime to take care of the Mutex ownership details for us. This is simple politeness, as it allows waiting threads to get on with their work as soon as possible.

Using the Mutex object, it is also possible for us to wait for more than one object to enter a signaled state before taking action. We can wait for one object in an array of WaitHandle objects to become signaled, or we can wait for all of the objects in an array of WaitHandle objects to become signaled. The array must not contain references to the same WaitHandle object, or an exception will be thrown.

The following is an example of the WaitAny method:

//create an array of mutexes
System.Threading.Mutex[] mutexArray = new System.Threading.Mutex[2];

//pretend that our quote application has more than one mutex,
//and load them into the array
mutexArray[0] = this.quotemutex;
mutexArray[1] = this.otherquotemutex;

//wait for either mutex to enter a signalled state. Note that WaitAny
//and WaitAll are both static methods, while WaitOne is an instance method
int indexOfSignalledMutex = Mutex.WaitAny(mutexArray);

//now, print the array of the object that entered the signalled state,
//allowing us to proceed, before moving on to our real work
#if Console
System.Console.WriteLine("The index was " + int.ToString());
#endif

The Mutex inherits most of its useful methods from the WaitHandle object, a virtual class that implements the advanced functionality available in a specific operating system's synchronization functions. WaitHandle, and the classes that derive from it, are not as portable as the Monitor class, since Monitor is completely managed. WaitHandle, on the other hand, depends heavily on the Win32 operating system. Code that we write targeting the .NET CLR on, for instance FreeBSD, may not have precisely the same implentation of the WaitHandle class, if it has one at all.

The .NET mutex object is largely the same as the Win32 mutex object, in essentially the same way that the .NET ThreadPool is the same as the thread pools in Windows NT, so an understanding of Win32 mutexes can definitely be helpful here, though it is not necessary.

A detailed discussion of remoting in .NET is beyond the scope of this case study. Multi-threading with remoted objects, however, is essentially the same as multi-threading within a single application domain or process. The only real caveat is that we must use the right synchronization objects, for instance the Mutex, and we must use them correctly.

Building The Service

Once I was satisfied with the QOTD implementation, I went on to set it up to run as a service on my Windows 2000 machine.

I created a class derived from System.ServiceProcess.ServiceBase, which consumes the QOTD objects in essentially the same way that the command line application did. Then I created a ServiceInstaller for the service, which provides the necessary hooks for the install utility to install the service.

The only appreciable difference in the service implementation is that I use the DoStop method of the QuoteEngine object. This method calls the abort method of the currently running thread, causing a ThreadAbortException to be thrown. This exception causes the thread to exit at the earliest possible point, but allows the application to finish processing finally blocks and also waits until the application returns to managed code if it has stepped into unmanaged code. As the documentation points out, it is therefore possible to create a situation in which a thread will never exit because of something we're doing in a finally block. This makes it very important to avoid infinite loops and long processes inside of finally blocks.

After calling the Abort method, we call the Join method, sending the thread into the WaitSleepJoin state while we wait for the abort request to be processed. This ensures that the caller of the thread will block until the thread terminates.

The Sample Application

When run as a console based application, the application prints information to the console. We remove this from the application when it is run as a service by toggling the precompiler directive. In any case, this application is intended to be used by simple telnet clients as seen below:

Installing the service

Running the code as a console application is straightforward - simply launch the executable you're interested in.

Running the code as a service is only a little more complicated, because we have to install the service. This requires that we run InstalUtil.exe on QOTDService.exe like this:

InstallUtil QOTDService.exe

I prefer to put this off for as long as possible, as I really hate the tedium of starting, stopping, installing, uninstalling and reinstalling the service while I'm making small changes to it during development.

Any Limitations or Further Work

There is still a decent amount of work that remains to be done in the QOTD application. For instance, the service was designed with the basic assumption that there would be many readers, but very few writers. It would probably be useful to refactor this design somewhat, allowing for a larger number of individuals to contribute their wisdom to the QOTD database. It would also be useful to constrain the list of people who are able to use the admin function, likely by integrating with Active Directory and providing a basic login/password prompt for the Admin UI.

The code also presumes that human beings are going to be the only ones interested in adding or reading quotes. It would be useful to rethink this somewhat, and change the code to allow for bots to have easier access to both the reading and the writing functions of the service.

Conclusion

This case study has covered some of the basic concepts of multi-threaded programming in C#, using an implementation of a Quote of the Day service as our case study. We have demonstrated the ThreadPool class and showm how to manage contention between threads.

We have, however, barely scratched the surface. Entire books are devoted to the topic of multi-threading, and there is a decent amount of material that could not be treated comfortably within the scope of this case study. There is, for instance, a straightforward means for working with threads that are created in unmanaged code. There is a Timer class that allows us to schedule threads to run at specific intervals. There are other derivatives of the WaitHandle class, the AutoResetEvent and ManualResetEvent, which expose other advanced synchronization mechanisms.

C# and .NET provide developers with unprecedented support for multi-threaded programming. This will allow us to create better, more efficient programs in less time, so long as we take care to use threads correctly and pay attention to the basic concepts behind multi-threaded programming, such as concurrency management.

Case Study Information
Author | Ramon Arjona
Technical Editor | Adam Ryland
Project Manager | Helen Cuthill
Reviewers | Sean Schade, Susan Connery, Matthew MacDonald

If you have any questions or comments about this case study, or suggestions for further pieces, please contact the technical editor. If you wish to talk about the case study with other subscribers, please visit our forum here.


timeTrack : An ASP.NET Contract Management System

timeTrack : An ASP.NET Contract Management System
http://www.csharptoday.com/content.asp?id=1840
Article Rating: 4
Published on May 13, 2002
Abstract
ASP.NET has brought with it a huge leap forward in the way that web applications are designed and developed. The following case study by Brian Boyce takes you from business requirements to completed application, focusing on the various technologies and techniques that the .NET Framework provides to simplify implementation and to dramatically reduce the development lifecycle.
Case Study

ASP.NET has brought with it a huge leap forward in the way that web applications are designed and developed. The following case study by Brian Boyce takes you from business requirements to completed application, focusing on the various technologies and techniques that the .NET Framework provides to simplify implementation and to dramatically reduce the development lifecycle.

Introduction

The case study revolves around a simple and prevalent problem in many organizations; accurate measurement and charging of resources. The application presented in this case study is a rewrite of a portion of a larger resource management system. In order to distinguish the two, the application presented in this case study has been called timeTrack Demo. The material presented in the case study relates exclusively to the demo version, with the exception of the Defining the Solution section which contains a little background on the applications history. The application aims to solve the following problems:

  • How much is my internal development costing me?
  • Which contracts are most profitable?
  • Ability to support all service oriented charging (accountability).

This case study will walk through the development of an ASP.NET application in C# that satisfies these requirements. The case study presents a simple application architecture that can be re-used for rapid development and prototyping of departmental and simple line-of-business applications. The technologies discussed in the development of the application include:

  • Web Forms
  • Writing User Controls
  • Custom Forms based Security
  • Implementing role based security
  • Creation of a site Navigation Control
  • ASP.NET Validation Controls and regular expressions
  • The use of HttpContext

These will be presented at the appropriate juncture in the development process.

System Requirements

The article assumes a basic familiarity with C#, ASP.NET and Visual Studio. In order to run the timeTrack Demo application you will need to have the ASP.NET V1.0 Framework installed, IIS 5 or later, and either SQL Server 2000 or MSDE 2000.

Defining the Problem

The client is a fictional software house, ABC Consulting, who provides business consultancy services, off-the-shelf products and consultancy lead development of bespoke software for a number of clients throughout the world.

As their business has grown and the number of contracts increased, they are finding it increasingly difficult to track the amount of time consultants are working on each contract. In addition, they believe that while many of their consultants are logging a single day, they are in fact working through lunch breaks and often into the evening. All of this additional time is going unrecorded and is therefore not being charged to their clients resulting in lost revenue for the company. In addition, the company is unable to monitor (and reward) its highest revenue generating consultants.

The defining requirements are as follows:

  • ABC Consultants must be able to log their time remotely
  • The system must be simple to use and require minimal input from each user
  • The application must enable the logging of time against contract for profitability tracking, enabling ABC to charge for days that are currently slipping through as part days. A combination of lost half hours across 40 consultants could amount to significant lost revenue.
  • The application will also enable ABC as a company to monitor the cost of different internal development and project work; which in turn will help them focus on truly profitable work streams.

The specific functional requirements will be explained as I describe the data model underpinning timeTrack.

Defining the Solution

As mentioned earlier, the timeTrack Demo application presented here is a simplified subset of a larger corporate intranet application. The original application was written in C# using the .NET Framework Preview Release and upgraded through Beta 1, Beta 2, and Version 1. The choice to implement the timeTrack Demo application in ASP.NET followed on from this. I now want to discuss some of the alternative implementation tools considered and what lead to me choosing to implement the application with ASP.NET.

The nature of the application was ideally suited to a web application; as such, the initial focus was on UltraDev and ASP. However, shortly after working with the ASP+ Preview (later called ASP.NET) I saw the enormous potential this technology had for shortening the development cycle of web and windows applications. The key features that drove the decision to use ASP.NET on this particular project were:

  • The separation of function from form (code behind, user controls and custom controls).
  • A clean and consistent object oriented platform.
  • The Framework classes which provided an enormous functional resource, making the grunt work an integration exercise rather than a large development effort.

The separation of function from form was a key factor in the decision; not only did it allow the developer to focus on the business functionality through the support of the framework classes but through the use of web controls the developer did not need to understand the nuances of cross browser HTML and scripting (though this does become a distinct advantage in more complex projects). The separation of UI also allowed the development to be split between graphic designers and developers in a way that had been almost impossible in ASP. The more development I did with ASP.NET, the more I realized that these were the most obvious advantages, but by no means the most compelling.

Database Design

The design implemented can be broken into two distinct areas:

  • Security
  • Contract Management

Diagram 1 shows the table structure.

Diagram 1: timeTrack Database Diagram

Database structure

Security

Both windows and forms mode authentication are well suited to this application, and I had initially decided on windows authentication. However, as I began to work through the application design, it became clear that forms based authentication better suited timeTrack for the following reasons:

  • The application is suited to small software and consultancy companies, many of whom will not have a dedicated web server (often subscribing to a shared hosting package). They are therefore not able to manage NT users and groups as easily.
  • The obvious extension of this application is to enable clients to submit requests for resources and view invoices on-line. In general, use the system as a means of simplifying and reducing the overall cost of communicating with your clients.

This case study presents the implementation using a custom forms based security model. The model presented allows us to store user and role information within the database. The four key roles are as follows (a user can be a member of 0 or more roles):

Role | Description
Administrator | The Administrator is responsible for maintenance of users and role membership.
ContractAdmin | The ContractAdmin is more of a business role, and involves maintaining user profiles, contract, client, and country definitions.
Consultant | Consultants are all those users who are able to record their time against a Contract.
Analyst | The Analyst has access to reports and historical analysis but is not able to alter the definition of the business entities nor record time against a Contract.

The Users table is used to store information about each of the registered application users. The Resources table is used to store information about resources that can be charged to a contract. The decision to separate the Users information from the Resources table was taken for three main reasons. Firstly, it separates the security from the functional entities, making it easier to re-use both the security model and contract data model in other web applications. Secondly, and more importantly, there may be a need for users who cannot log time against a contract, such as analysts and administrators (or in the future clients and customers) logging onto the system in order to generate invoices, requests or reports. The final reason is that although Resources in the current business model are people, in the future a requirement may arise to track the use of other resources and bill for their usage, e.g. Conference facilities, etc. These reasons also help explain why there is no reference between the Users and Resources tables, i.e. Resources are not necessarily people (e.g. Conference Facility), and Users are not necessarily Resources (e.g. Administrator).

The Roles for each user are defined in the Roles table and the UserRoles table is used to represent a many-to-many relationship that exists between Users and Roles. The only notable setting here is the definition of delete referential integrity on the relationship between the Users and UserRoles tables and the Roles and UserRoles tables. This setting ensures that any deletion of records from the primary key table in the relationship results in the deletion of the associated records in the foreign key table. See figure 1, which shows the Roles table relationship property window.

Figure 1: Configuring Delete Referential Integrity

The stored procedures for these tables are fairly straightforward; those pertinent to particular core functionality will be discussed in the component design section.

Defining a Contract

The Country, Category and Client tables are simple informational tables used to provide additional information about these entities. The primary purpose for the normalization of these tables is to ensure that analysis of ChargeableTime information can be done by Country, Category and Client in a consistent and reliable way. As such, their use and structure is self-explanatory. The code has been included in each table in order to simplify the creation of dimension tables for the creation of OLAP Cubes, if required.

The Country table is used to store information about the country a contract relates to.

The Category table can be used by ABC Consulting to categorize their contracts in any way they like. This table was included to provide a further user-definable level of grouping to assist their analysis.

The Client table is used to store information about the client the contract relates to.

The interesting tables are Contract, Resource and ChargeableTime. The Contract table, as you might expect, contains a single record for each Contract. Each of the fields in this table, its purpose and use are defined in table 1 below.

Table 1: Defining a Contract

Field | Use
Code | User-definable account code, which can be used to uniquely identify a Contract in dimension tables.
Description | Contract title or description for reports
CategoryID | Foreign key reference to the Category table.
PurchaseOrderNum | The purchase order number associated with a Contract. This is included to allow for direct interfacing with an accounts package or automatic generation of client invoices.
CountryID | Foreign key reference to the Country table.
Period | The period is used to identify how the contract is charged. The acceptable values are: H - Hourly D - Daily FP - Fixed Price The hourly and daily setting for this field are used in conjunction with StandardRate to determine how much to charge the client Per Hour (Period of 'H') or Per Day (Period of 'D').
IsExternal | Is this an internal or an external contract. This allows ABC Consulting to track the cost of internal work such as research, development of off-the-shelf products, etc.
FixedPrice | The fixed price as stipulated in the contract. Note time is still recorded against the contract to enable ABC to track how profitable and how accurate their estimates are for Fixed Price work.
StandardRate | Is the monetary rate per period to charge the client.
OutOfHoursMultiplier | This multiplier can be used to reflect any special rates negotiated in the contract for overtime or unsociable hours etc. A value of 1.5 results in the client being charged time-and-a-half for any out of hours work undertaken by the ABC Resources.
WeekendMultiplier | Is similar to OutOfHoursMultiplier and can be used to reflect any special weekend rates applicable to the contract.
ContractDay | This is used to reflect the length of a work day in hours as negotiated in the contract. This allows you to accurately charge for different contracts where the number of hours in a work day is different. ABC can use this to offer discounted rates for 7.5 hour days etc.
InitialDays | The number of agreed days in the contract. This field can be used to track man-days bought by a client, and allows the project managers to track the burn rate of contract days and renegotiate or extend the contract with the client before the total number of days is exhausted. See the discussion of highlighting DataRows in ContractAdmin.ascx user control.
StartDate | The start date of the term of the contract.
EndDate | The end date of the term of the contract.
ClientID | Foreign key reference to the client details relating to this contract.
IsDeleted | Is used to flag a Contract as inactive. It controls whether or not the Contract is available in the drop down list box presented to Consultants when they log their time.

The stored procedures that surround the Contract table perform the usual SelectAll, SelectSingle, Insert, Delete and Update. The SelectAll stored procedure called GetContracts returns not only the details of the Contract definition, but also the number of Days remaining on the Contract (based on the information in ChargeableTime and the initial days field). The provided implementation of this calculation applies the multiplier to the hours, e.g. 19:00 - 20:00 on a Saturday is calculated as 1 * weekend mult. * out of hours mult. This field could alternatively have been implemented by a series of triggers on the ChargeableTime table, however the calculation of the information only when it is requested rather than on each entry to the ChargeableTime is far more efficient.

The DeleteContractItem stored procedure does not delete the record from the table (though I have included the necessary SQL to do this in the stored proc as a comment). This feature was included to ensure that the Contract definition would not be deleted before the final invoice had been sent to the client. It is also important to persist historical Contract information for fixed price and internal contracts in order to monitor the profitability of these contracts and adjust future quotes according to the proven profitability of previous similar contracts.

Modeling a Resource

The Resource table models the Consultant's standard working day. The detail involved in this table is a direct result of the business requirement that specifies minimal user interaction in order to log time. To meet this requirement, the timeTrack system stores a profile of each individual resource's working pattern. This enables the consultant to record a typical day with a single click (see figure 2).

Figure 2: The Time Log Screen

By choosing 'Add a Day (based on profile)', timeTrack will take the user to the data entry screen with all of the fields already filled in according to their individual profile settings. This helps the organization capture the additional half hours, etc, that some employees work in a typical day.

Don't Forget the Data

The ChargeableTime table stores all of the information logged by each of the Resources / Users. The majority of the information stored within this table is self-explanatory. The description fields can be used to provide additional details about the work that was undertaken by the resource in the specified period. This enables ABC Consulting to provide an audit trail of all charged time to any clients who request it.

The IsLoadedInCube bit field is worthy of some additional discussion. Although the implementation of an OLAP Cube against the timeTrack data is out of the scope of this Case Study, I have included some of the control fields in the DB design to enable me to highlight some of the considerations and solutions to the management of generating dependable Analysis Services feeds. For those not familiar, Analysis Services is the OLAP engine that comes with SQL Server 2000 (it was called OLAP Services in SQL 7). This bit field is set to a default of 0. This field is only ever explicitly set in the UpdateContractData stored procedure, which sets the field to 0 to ensure that any fact table (the central table in a star or snowflake schema used for population of an OLAP cube; in simpler terms the table with the raw line-item or transactional data rather than a table describing business entities such as country, site, or resource) construction is aware of the change to the possibly historical data item. The field is only set to 1 during the fact table construction to indicate that the line item data has been included in the fact table information. Another useful means of using this type of control flag is to effectively lock records from update after an invoice has been generated, to ensure consistency of historical reporting.

A critical requirement of all charging related and time tracking systems is to ensure that a single resource has not recorded the same time period against multiple tasks. In timeTrack, the procedures AddContractData and UpdateContractData check before adding a record that the Resource has not already allocated this time period using the SQL ,shown below:

IF EXISTS (
SELECT StartTime, EndTime
FROM ChargeableTime
WHERE ResourceID = @ResourceID
AND [Date] = @Date
AND ((@StartTime > StartTime and @StartTime < EndTime)
OR (@StartTime < StartTime and @EndTime > StartTime)
OR (@StartTime < EndTime and @EndTime > EndTime)
OR (@StartTime = StartTime and @EndTime > StartTime)
))
BEGIN
RAISERROR ('The times specified overlap other already allocated times!', 16, 1)
END

The RAISERROR function allows me to pass an error back through to the appropriate User Control for display to the user.

Component Design

The high level component design for the demo version of timeTrack has been simplified to enable me to focus on the techniques used to implement each component.

The timeTrack application uses a SQL Server database with all table access controlled through stored procedures rather than in-line SQL: this improves performance, standardizes access, and simplifies server side security implications. It is worth noting that the use of stored procedures, and indeed the System.Data.SqlClient namespace, tie the application to SQL Server (see resources for article on problems with stored procedures). The alternative is to use a database independent DAL. I have seen applications which use an XML file (certainly better than placing the SQL in-line throughout the application) to store the SQL, and use either the System.Data.SqlClient or System.Data.OleDb namspaces to execute the SQL depending on a configuration setting. This is a more versatile solution, but does suffer from the additional hit on query plan determination at run-time. I chose to use SQL Server for the timeTrack case study because the .NET Framework installs the MSDE, and so all readers will be able to run the accompanying code. One point I would like to stress is that any business logic should not be implemented in the SQL stored procedures without very good reason. The database design has already been discussed above.

The data access layer contains a series of database access wrapper classes. The majority of the classes are stateless, acting only as efficient .NET wrappers on the stored procedure calls. As such, the methods are static.

The final logical application layer comprises the user control components, the security configuration, navigation control, and overall site presentation. The user controls are used to modularize the application functionality in order to simplify the final site implementation, and are discussed in detail in the Component Coding section.

Component Coding

DAL : The Data Access Layer

The data access layer provides a series of .NET objects which can be used to interact with the underlying database. The main DAL classes in timeTrack are ContractDB and ProfileDB. Both of these classes exist only to execute stored procedure calls and return the results. Since this task requires no state knowledge, the methods are implemented as static methods, enabling them to be called without first instantiating an instance of ContractDB or ProfileDB. This is very common in DAL's. The classes provide the following set of methods for each controllable table in the database:

GetAll - Returns all records from the table

GetSingle - Returns a single record from the table

Add / Insert - Adds a new record to the table, returning the identity values of the record inserted.

Update - Updates a specific record in the table with some new values

Delete - Deletes a specific record from the table

Due to the repetitive nature of DAL coding, there are several tools available which will generate a skeleton DAL wrapper class for a table. Some will even generate the SQL stored procedures (see resources at the end of this article for a link to ".Net Enterprise Development in C#". This book contains the Wrox Enterprise Objects tool). The SQL Server information schema views are very useful for providing a complete list of all parameters and their properties for any databases stored procedure. See listing 1 for some sample SQL.

Listing 1 : Interrogating the SQL Information Schema Views

SELECT SPECIFIC_NAME, PARAMETER_MODE, PARAMETER_NAME, DATA_TYPE, CHARACTER_MAXIMUM_LENGTH
FROM INFORMATION_SCHEMA.PARAMETERS
WHERE SPECIFIC_NAME NOT LIKE 'dt%'
ORDER BY SPECIFIC_NAME, ORDINAL_POSITION

As ever, when dealing with databases it is important to ensure that connections are properly closed. There has been a great deal of discussion on this topic (see http://www.asplists.com/asplists/aspngdata.asp?WROXEMPTOKEN=618658Zezi9SpDFSZgQps5Xv6u). The connection used to populate a DataSet can and should be closed immediately after the Fill. The behaviour of the SqlDataReader is a little more complex. In general, code like the following from AdminDB.GetUsers() will close the connection only after the SqlDataReader itself is explicitly closed.

// Open the database connection and execute the command
myConnection.Open();
SqlDataReader dr = myCommand.ExecuteReader(CommandBehavior.CloseConnection);

If you are binding the SqlDataReader, the close method is handled for you. However, when you are instead iterating through the data reader you must explicitly call the SqlDataReader.Close() method to ensure that the underlying connection object is closed.

Security and the Login Control

The application security settings are controlled by the <authentication> and <authorization> tags in the web.config file. Authentication is the process of associating and validating a users identity, whereas authorization is used to control the access granted to anonymous and authenticated users to specific resources.

ASP.NET provides four standard authentication modes: Windows, Forms, Passport and None. All of these are easily configurable through the authentication tag in the web.config file. Windows is most commonly used in an Intranet environment, though it can be used with IIS Basic, Digest and Integrated security configurations. Specifying windows authentication is as simple as adding the following tag to your web.config file.

<authentication mode="Windows" />

The authentication sub tag <forms> is used to configure custom forms-based authentication. This tag allows you to configure the name of the cookie used (name attribute). This name should be unique to your application in order to prevent conflicts with other applications residing on the same IIS server. It also allows you to configure the encryption and validation options for the cookie (see .NET help on the <forms> element for full details of the encryption and validation options), the default is to use both cookie encryption and validation and is the recommended setting. The timeTrack application's authentication tag is:

      <authentication
            mode="Forms">
<forms name="ContractCS"
            loginUrl="Login.aspx" />
</authentication>

The loginUrl attribute allows you to specify a page that all unauthorized users will be directed to. This simple mechanism enables you to ensure that no-one can access any ASP.NET controlled resource in your application without first being authorized. The controlled resources I refer to are those file types that are associated, within IIS, with the ASP.NET ISAPI filter (e.g. .aspx, ascx, .config, etc). While it is possible to add other file types to this association, thereby bringing them under the control of ASP.NET authentication, this approach is not recommended because of the performance overhead. A better solution to this problem, though only applicable to Windows mode authentication, is to use ACL's to protect the non-ASP.NET files and allow impersonation (by specifying <identity impersonate="true"> in web.config). Impersonation causes the ASP.NET code to execute in the context of the users security profile. So, any user who requests default.aspx will be redirected to Login.aspx unless they are authorized (in the case of timeTrack this requires them to have been authenticated). Once the user signs in by providing a valid username and password they are then redirected to the page they originally requested.

The authorization tag can be used to determine whether a specified identity should be granted access to a specific resource. The URLAuthorization module, which controls URL authorization, implements allow and deny assertions on resource access, allowing you to specify the access to specific parts of the URI namespace at the role or identity (HttpContext.User.Identity) level (a set of roles or identities can be specified by use of a comma-separated list). The following authorization tag demonstrates this:

<authorization>
<allow roles="ABCConsulting\Consultants, ABCConsulting\Management">
<allow users="ABCConsulting\DGoodwin">
<deny users="*">
</authorization>

The above authorization tag allows members of the roles Consultants and Management to access the site, as well as a specific user DGoodwin. All identities (indicated by "*") are denied, including anonymous users which could have been explicitly denied using <deny users="?">. The default ASP.NET configuration contains an <allow users="*"> element, authorizing all users. You can also specify the HTTP verbs (GET, POST, or HEAD) allowed by each user by entering tags such as <allow verb="POST" roles="ABCConsulting\Consultants">.

Figure 3: The timeTrack Login Control

The login Control (Login.ascx, see figure 3) works by asking the user to enter an e-mail address and password. These values are then checked against the Users table, and if the values match a users definition in the Users table, the code creates a cookie for the user using:

// Use security system to set the UserID within a client-side Cookie
FormsAuthentication.SetAuthCookie(email.Text, RememberCheckbox.Checked);

Given that the login control is only available on the login page, it may seem strange to have chosen to implement it as a control at all. This is, to some degree, personal preference; I find later restructuring of code and interface to be simpler when dealing with controls (custom or user).

The FormsAuthentication class provides static helper utilities for the manipulation of authentication tickets. The SetAuthCookie retrieves the encrypted values from the cookie as an HttpCookie instance. If the user asked to persist the cookie by checking the RememberCheckbox checkbox, the SetAuthCookie creates an authentication ticket, encrypts it, and stores it within the cookie on the client machine. Listing 2 shows the code in the timeTrack Demo used to control the persistence and subsequent retrieval of the cookie authentication ticket.

Listing 2: Cookie Creation in Global.asax

protected void Application_AuthenticateRequest(Object sender, EventArgs e)
{
if (Request.IsAuthenticated == true)
{
String[] roles;
// Create the roles cookie if it doesn't exist yet for this session.
if ((Request.Cookies["ContractCSroles"] == null) ||
(Request.Cookies["ContractCSroles"].Value == ""))
{
// Get roles from UserRoles table, and add to cookie
UsersDB user = new UsersDB();
roles = user.GetRoles(User.Identity.Name);

// Create a string to persist the roles
String roleStr = "";
foreach (String role in roles)
{
roleStr += role;
roleStr += ";";
}

// Create a cookie authentication ticket.
FormsAuthenticationTicket ticket = new FormsAuthenticationTicket(
1, // version
Context.User.Identity.Name, // user name
DateTime.Now, // issue time
DateTime.Now.AddHours(1), // expires every hour
false, // don't persist cookie
roleStr // roles
);

// Encrypt the ticket
String cookieStr = FormsAuthentication.Encrypt(ticket);

// Send the cookie to the client
Response.Cookies["ContractCSroles"].Value = cookieStr;
Response.Cookies["ContractCSroles"].Path = "/";
Response.Cookies["ContractCSroles"].Expires =
DateTime.Now.AddMinutes(1);
}
else
{
// Get roles from roles cookie
FormsAuthenticationTicket ticket =
FormsAuthentication.Decrypt(Context.Request.
Cookies["ContractCSroles"].Value);

// convert the string representation of the role data
// into a string array
ArrayList userRoles = new ArrayList();
foreach (String role in ticket.UserData.Split( new char[] {';'} ))
{
userRoles.Add(role);
}

roles = (String[]) userRoles.ToArray(typeof(String));
}

// Add our own custom principal to the request containing the
// roles in the auth ticket
Context.User = new GenericPrincipal(Context.User.Identity, roles);
}
}

This code instantiates a GenericPrincipal object which implements IPrincipal and exposes an IIdentity interface through the identity property, enabling us to check role membership (IsInRole), Name (Name), and Authentication (.IsAuthenticated).

Site Navigation

One of the main reasons for implementing a custom role based authentication model was to enable me to vary the accessibility and user interface presentation based on a user's security settings. I decided to implement the navigation control as a user control (SiteBanner.ascx, see figure 4) in order to encapsulate as much of the logic as possible. I considered several approaches to this control and hit quite a few problems in its implementation, which I will now discuss.

Figure 4: The timeTrack SiteBanner Control

The first and most obvious question I faced was whether or not to create a custom control or a user control. I decided to use a user control mainly because of the fact that the control is a composite control and had no special design time, rendering, or deployment requirements (custom controls are the better option for creating controls that will be distributed to be used in multiple applications). As an aside, I also believe that the development of a truly flexible and re-usable navigation control is a suitable subject for a whole case study and would only cloud the core subject matter of this case study (see resources for links on Custom Control Creation).

The most obvious and simplest solution would have been to dynamically create a single row table of hyperlinks, based on the users roles. However, I wanted to be able to maintain the state navigation control, displaying the selected tab differently from the unselected. The first problem I hit in the creation of this control was my inability to trap the SelectedIndexChanged event of the BaseDataList. Since this event is only triggered when the selected index changes between server postbacks, I could not use this event (due to the nature of the navigation control, when a new item was selected a new page was requested). Eventually, I realized I could use a LinkButton and persist the SelectedItemIndex by means of the dlNavBar_ItemCommand event. The ItemCommand event allows you to pass a command string back to your server side code using the CommandName property. The CommandName property can be used to pass a string that indicates the type of command to be performed. An example from the timeTrack AdminRoles.ascx user control is shown in Listing 3 below:

Listing 3: Use of the CommandName property to pass the action type back to server side code

<asp:DataList id="rolesList" DataKeyField="RoleID" runat="server">
<ItemTemplate>
<asp:ImageButton ImageUrl="~/images/edit.gif" CommandName="edit" AlternateText="Edit this item"
runat="server" ID="Imagebutton1" NAME="Imagebutton1" />
<asp:ImageButton ImageUrl="~/images/delete.gif" CommandName="delete" AlternateText="Delete this item"
runat="server" ID="Imagebutton2" NAME="Imagebutton2" />

From the AdminRoles.ascx.cs rolesList_ItemCommand event handler:

if (e.CommandName == "edit")
{
//
            Set editable list item index if "edit" button clicked next to the
            item
rolesList.EditItemIndex = e.Item.ItemIndex;

//
            Repopulate the datalist control
BindData();
}
else if
            (e.CommandName == "apply")
{
// Apply changes
String
            _roleName = ((TextBox)
            e.Item.FindControl("roleName")).Text;
..

This allows you to add several ImageButton and LinkButton controls to a control item's template and identify the action to take based on the CommandName string property. Each control can specify an optional CommandArgument string which can be used to pass parameters or additional information to supplement the command action.

Now I was able to at least set the Selected index of the horizontal DataList control. This still didn't solve the problem of persisting the index setting across requests. While viewstate and ASP.NET handles the persistence across postbacks it does not persist the control settings across requests for different pages. Initially I thought of creating a protected int variable on each page called NavId to hold the Selected index value. I then added a NavId parameter to the hyperlink stored in each tabs NavBar control, as follows:

authorizedLinks.Add(new NavBar("Contract Admin", "~/ContractAdmin.aspx?NavId=1", "Contract Admin"));

This approach was flawed in that the index of the Tab in the navigation control changes depending on the user's role membership. For example, user A, a Consultant, would sign on and see links for Home (index 0) and Log Time (index 1), while and User B, a ContractAdmin and Consultant, would sign on and see Home (0), Contract Admin (1) and Log Time (2). This one aspect of the problem could have been resolved by passing the name of the Tab and iterating through the navigation list (or performing a direct read on a hashtable) to find the matching entry before setting the SelectedIndex property. This solution is more efficient than enabling sessionState for the sole purpose of storing the navigation index, but the timeTrack application is likely to be used not as a sole web application but rather as part of a larger company intranet/internet application. I am also averse to putting unnecessary information in the URL (especially since in the above scenario User A sending User B a hyperlink could generate an incorrect rendering of the navigation control), preferring instead to handle storage state information in session, application, or cache objects. The code accompanying this case study uses the HttpSessionState class through Context.Session to store the currently selected tab index.

The HttpSessionState object allows you to persist information relating to a particular Session for the entire duration of their visit to the site. It works by assigning a unique session ID for a user on their first visit to the site, the ID is then associated with the user until the session times out. This is configurable at both page level through the Timeout property

Session.Timeout = 3 // in minutes

Or through the web.config file:

    <sessionState
mode="InProc"
stateConnectionString="tcpip=127.0.0.1:42424"
sqlConnectionString="data
            source=127.0.0.1;user id=sa;password="
cookieless="false"
timeout="20"
/>

The sessionState tag above shows the default settings for sessionState. This configuration will cause ASP.NET to track the session id using cookies on the client machine. For the purposes of our demo this is adequate, although timeout could be dropped dramatically since the NavId is reset on each click on the Navigation control. The available modes for session state are:

sessionState mode | Description
Off | SessionState is not enabled
InProc | SessionState is stored locally
StateServer | SessionState is stored on a remote server.
SQLServer | SessionState is stored in a SQL Server (note this is only available in SQL Server 7 and above)

In-Process session state is the fastest, and ideal for small amounts of non-critical, volatile information. The drawback of in-process sessionsState, however, is that an IIS restart or ASP.NET process crash results in the loss of the session information by virtue of the fact that the session state is managed within the IIS process. The advantages of a separate out-of-process session management is that the session state information is retained throughout an IIS restart or process crash, and that the sessionState is not inextricably linked to any one machine or server. While this loss is not critical in an application like timeTrack, in an e-commerce site where shopping cart information is held in sessionState, this kind of data loss has a direct impact on revenue and customer confidence.

In order to provide an alternative out-of-process solution to state management, ASP.NET provides SQL Server and State Server modes. StateServer mode allows you to start a separate state server on a remote machine using net start aspnet_state, and enter a stateConnectionString value in web.config. To configure SessionState to store its information in SQL Server, you must run a special SQL script (InstallSqlState.sql) against your DB. Fortunately MS have provided this script with the .NET Framework, it is located in the <%WINPATH%>\Microsoft.NET\Framework\<version>\ folder by default. This creates a new ASPState database and some stored procedures to manage state. Once the script has run successfully and the new database has been created, all you need to do to configure SQL Server StateManagement is alter the web.config, specifying mode=SQLServer and providing a connection string in the sqlConnectionString attribute:

<configuration>
<system.web>
<sessionState mode="SQLServer" sqlConnectionString="data source=127.0.0.1;user id=sa;password=" />
<system.web>
</configuration>

The decision to use in-process or out-of-process state management depends, to a large degree, on the information being persisted. In-process state management also offers a performance benefit over out-of-process management.

By using session state in the SiteBanner.ascx, I am able to pass the selected item index across to the next page, allowing it to render the navigation control correctly. The removal of the NavId parameter from the query string also means that a hyperlink to timeTrack used by different operators will not result in an incorrectly rendered selected tab setting.

Defining the Contracts, Assigning the Resource

While the navigation and login controls were the most deeply entrenched in ASP.NET configuration, their function is structural rather than functional in the sense that they do not help solve the business problem at hand. What follows is a simple model for quickly putting together prototypes and departmental line of business systems that are easy to maintain.

The reason for demonstrating this approach is to show how you can build valuable customer ASP.NET applications with a relatively narrow field of knowledge with regards to ASP.NET. I have already shown how easily the security can be configured for your application, all that remains is to provide a clean usable user interface to the function. In my opinion, this is the beauty of .NET: it allows you to focus on the application specific function by providing libraries of standard functions and simple reliable configuration mechanisms.

Each of the core business entities Contract, Client, Country, Category and Resource are handled in essentially the same way. Once the requisite informational structure has been defined in the database table I write several standard stored procedures that perform the basic expected interactions: Select All (returning a DataSet or SqlDataReader), Select Single instance (returning a DataSet, DataRow or SqlDataReader), Insert (returning the identity value of the item just added), Update and Delete (both having void return type). The next stage is to provide a .NET wrapper to the stored procedure call. I chose to implement each method by establishing a connection to the database, using the web.config file to store the connection string:

<configuration>
<appSettings>
<add key="ConnectionString" value="server=localhost;uid=sa;pwd=;database=ContractCS" />
.
</appSettings>
.
</configuration>

and

ConfigurationSettings.AppSettings["connectionString"]

to retrieve the value.

The stored procedure call is built up by adding to the parameters collection of the command or adapter object. In general, for this case study I have used DataSet return types as I do when prototyping since this allows greater flexibility in the manipulation of the front end user interface, allowing me to present to the client a number of different front end interface concepts based on the same backend DAL components. In actual fact, more recently I have replaced this habit firstly with a SqlDataReader to SqlDataSet conversion function in prototypes, with the full strongly typed entity and entity collection wrappers being used for second stage development (see resources for some resources on the construction of the entity wrappers).

Next I create a composite user control to display the records in the table and provide a means of editing, adding, and deleting new items. A typical examples is the ContractProfile.ascx (see figure 5)

Figure 5: The timeTrack User Profiles Control

The ContractProfile control makes use of the ASP.NET DataGrid's built-in paging functionality. This allows you to configure the DataGrid to display a subset of the underlying records and provide the user with pager controls to move through the information. ASP.NET provides standard and custom paging options. The standard paging option is very simple to set-up and only requires you to set the AllowPaging property of the DataGrid to true (the default page size is 10) and set the PagerStyle objects mode property in the PagerStyle template.

<ASP:DataGrid id="dgProfile" AllowPaging="true" . runat="server">
...
<PagerStyle Mode=NextPrev></PagerStyle>
</ASP:DataGrid>

The two modes for the PagerStyle object are NextPrev, which defaults to a display of '< >', or NumericPages which displays a row of page numbers. DataGrid provides a vast number of style based properties for configuring the appearance of the PagerStyle object. One important point to note about the standard ASP.NET paging mechanism is that it retrieves all records on each page request and should not, therefore, be used with a very large numbers of records. If you do have to deal with paging through large numbers of records, then the back-end approach is very similar to how you would do it in traditional ASP, by creating a stored procedure that creates a temporary table with an identity column and selecting a specified page based on input parameters of @pagesize, @CurrentMinID and @currentMaxID (the minimum and maximum identity values from the page currently being displayed). With this information you can use

SELECT TOP @pagesize * FROM <##table> WHERE ID > @currentMaxID

To return the Next page of records, and

SELECT TOP @pagesize * FROM <##table> WHERE ID < @currentMinID

To return the Previous page of records. Your stored procedure would also contain bounds checking.

As you can see from figure 5, the user can page through the available resources, edit any resource by clicking on the arrow to the left of the resource, and add new resource profile definitions. The add, update and delete is handled by an EditContractProfile.aspx page. Each of the core functional .ascx controls has an associated web form that handles the edits. I make use of this fact in the user control by using the ItemTemplate to define a hyperlink that links to a URL to edit the current record as follows:

<asp:HyperLink ImageUrl="~/images/edit.gif" NavigateUrl='<%# "EditProfile.aspx?ItemID="
+ DataBinder.Eval(Container.DataItem,"cres_id") %>'

The edit pages contain role based security checks in the Page_Load to ensure that a user cannot gain unauthorized edit access by editing the URL. The code to implement this simply checks the current users HttpContext IsInRole member:

// Verify that the user has access to the
            current page
if
            (Context.User.IsInRole("ContractAdmin".ToString()) ==
            false)
{
//
            Response.Redirect("~/AccessDenied.aspx");
Response.Redirect(Configuration.ApplicationPath
            + @"\AccessDenied.aspx");
}

The above snippet should contain a slight cautionary note on the use of '~'. In C#, the '~' operator can be used as a shortcut to specifying the Request.ApplicationPath. In virtual directories this works fine, however, if the application is at the root domain level it can result in the generation of invalid URL's. It should be noted that HttpRequest.ApplicationPath also returns '/' at the root domain level. I have not been able to find an elegant solution to this problem. In an attempt to enable you to run the application at either domain or virtual directory level, I created a small helper class (Configuration.cs) with a single static property ApplicationPath, which returns String.Empty if the HttpRequest.ApplicationPath returns '/'. The code accompanying this article does not use this method throughout; rather it is demonstrated in SiteBanner.ascx control. The simple reason for this is that I could not find an elegant way of replacing every ~ within the application while retaining full VS.NET designer support, short of specifying every link URL within the code.

The ContractAdmin.ascx also uses the ItemCreated event of the DataGrid to color code specific values to give the user visual cues to the state of the Contracts. The code that performs this uses FindControl to locate the particular control in the templated data bound grid and change its forecolor. See listing 3.

Listing 3: Providing Visual Cues through Color

if
            (((Label)e.Item.FindControl("lblActiveField")).Text ==
            Boolean.FalseString)
{
e.Item.ForeColor =
            System.Drawing.Color.DarkGray;
}
else
{
decimal
            DaysRemaining =
            System.Decimal.Parse(((Label)e.Item.FindControl("lblDaysRemaining")).Text);
if
            (DaysRemaining < 0)
{
((Label)e.Item.FindControl("lblDaysRemaining")).ForeColor =
            System.Drawing.Color.Red;
}
else
{
if (DaysRemaining
            > 0 && DaysRemaining <
            10)
{
((Label)e.Item.FindControl("lblDaysRemaining")).ForeColor
            = System.Drawing.Color.Blue;
}

The FindControl method is enormously powerful and avoids the maintenance headaches caused by hardcoding indexes in control collections to refer to objects, such as:

((Label)e.Item.Cells[8].Controls[1]).Text

which works only as long as another column is not added to the data grid columns template.

The Client, Country and Category controls have fragment caching defined at the control level since the data in these controls is not subject to frequent change. The Contract and Contract data controls are transactional by nature or require accurate up to the minute provision of figures. Fragment caching can be achieved by using the OutputCache page directive:

<%@ OutputCache Duration="5" VaryByParam="none" %>

VaryByParam allows you to cache a different version of the control based on a parameter value, VaryByParam="*", which will cache a different page fragment whenever any parameter changes. Another very useful attribute for caching controls is VaryByControl, which allows you to vary the output cache based on a semi-colon separated list of user control properties. Caching is one of my favourite features of ASP.NET, and I cannot stress its importance in the development of heavy-use web applications enough; my advice is to make caching the next thing you learn about in ASP.NET if you don't already know it!

Figure 7: Component Development Model

Figure 7 shows the simple model used in the timeTrack application to wrap all interaction with the database by means of a Data Access Layer which calls the SQL stored procedures. The biggest advantage gained from this simple model of development is that the wrapper around each business entity forms a distinct module that is reliable, consistent, and easy to maintain. In small systems such as the timeTrack demonstrated here, there is increasing pressure on developers to be able to develop, deploy and maintain these applications quickly and consistently. Often when change requests are raised, they are in reaction to a business process change with immediate impact, and as such require very rapid resolution and implementation. A simple model as outlined above minimizes the impact on the application of table alterations and user interface changes. I am not suggesting that this approach be taken for large scale enterprise systems, but do believe that this approach is very valuable in being able to provide the business with solutions quickly and cheaply. The secondary advantage is that the similarity among the controls allows them to be cloned very quickly, enabling you to add a whole new business entity with full user interaction to the system in a matter of minutes or hours rather than days.

UI Design

The importance of user interface design can never be underestimated. Through the development of countless systems for clients I have come to realize that an attractive and usable user interface is critical right from pre-sales presentations through to implementation. Often the user interface will bear the brunt of more scrutiny and form the basis of more opinions than the underlying functionality: sad but true.

The use of Edit pages rather than using the DataGrids EditItemTemplate provides the user with a fuller and more user friendly data entry screen. Although the EditItemTemplate can contain quite an extensive set of controls for the update of the record, it is difficult to provide a clean and consistent visual appearance across a whole site. The edit page also provides me with sufficient white space to provide detailed error text for my validation controls, and in my opinion leads to a better user experience. I believe each has its role, and use EditItemTemplate in my applications where appropriate.

One of the main business requirements for this application was that the recording of information requires minimal effort and time on the part of the consultant. In order to achieve this I considered various options. The idea of storing individual work profiles for each resource allowed the user to record a typical day with a single click on the Add a Day (based on Profile) link. Initially, the application simply created a new instance for the current day without taking the user to the EditContractData page. However, as I was testing the application, I realized that on days where the user has left 15 minutes early or stayed a little later but otherwise worked a typical day, the convenience of clicking on a link to fill in all of the default properties would be appreciated.

The ContractData control also displays a couple of links that allow the user to vary the period of time shown in the DataGrid. This allows them to perform checks on the information they've entered and also to look back and see what they were working on at any given date.

The focus of the application from a Contract Admin user's perspective is on the number of days remaining in a contract. For this reason, I implemented colour highlighting of key values in the DataItem to show contracts that were running low on agreed dates, etc.

The user interface should be as clean and consistent as possible. The use of a standard approach to all our controls ensured a consistent user experience throughout the application usage, and a single cascading stylesheet (ContractCS.css) was used throughout to provide a consistent visual format. When using VS.NET, it is often tempting to set style attributes to specific fonts or colours within the control properties. This leads to fragmented and often inconsistent settings. Cascading Stylesheets provide a way to separate content from style by allowing for the definition of named collections of style attributes. These can be associated with HTML tags as follows:

A.CommandButton:link {
text-decoration:
            underline;
color: #FF702B;
}

A.CommandButton:visited
            {
text-decoration: underline;
color: #FF702B;
}

The use of a stylesheet in this way enables you to easily alter the appearance of the entire site in a consistent manner, which can be a very powerful tool in facilitating customer specific branding of ASP.NET and other web applications.

UI Code

Due to the critical nature of Roles within the application, any attempt to delete a role prompts the user for confirmation. This is achieved by creating invisible Yes/No link buttons in the DataList template. I could have used JavaScript on the delete button (see Listing 4) to create a pop-up confirmation dialog box, but felt that the link buttons were more in keeping with the look and feel of the application.

Listing 4 : Implementing a JavaScript Confirmation Dialog Box

ImageButton2.Attributes.Add("onClick","javascript: return confirm('Are you sure?');");

Another valuable addition to the developers kitbag introduced in ASP.NET is the validation controls. These controls allow you to provide the user with meaningful feedback on incorrect data entry values and help ensure that any input information is valid and correctly formed. Within timeTrack demo I have used these where appropriate to improve the user experience. The EditContractData user control, for example, contains several RequiredFieldValidator controls (to ensure that data is entered) and several RegularExpressionValidator controls (to ensure that the time supplied by the user is a valid time). To add a validation control to your web form or user control simply drag the appropriate validation control onto the form and set the properties as outlined in the table below:

Property | Description
ControlToValidate | The name of the control you wish to validate
Text | The text you want to appear in the validation control if the validation fails
ErrorMessage | The text you want to appear in any ValidationSummary control on the page if validation fails
Display | None - the control will never appear in-line. A message will only be output to the validation summary on failure. Static - Space is allocated to the control Dynamic - Space is only allocated to the control when validation fails.

Application Setup

The following steps describe the process of installing the timeTrackDemo application:

  1. Restore the supplied ContractCS database to your SQL Server 2000 or MSDE database.
  2. In IIS create a new Virtual Directory and extract the files from the timeTrackCode.zip file into the folder used for this virtual directory.
  3. Edit the .webinfo file in the folder you have just created so that it points to your virtual directory. This will enable VS.NET to connect to the appropriate virtual directory.

    <appSettings>
    <add key="ConnectionString" value="server=localhost;uid=sa;pwd=;database=ContractCS" />
    <!-- Mail Server MUST be changed to your IIS_configured Server name -->
    <add key="MailServer" value="mymail.myserver.com" />
    </appSettings>

  4. Edit the web.config file, changing the connection string to suit your database server name and password. The code supplied will connect to localhost as 'sa' with blank password. If you have set-up SMTP mail on your IIS Server then edit the MailServer value (don't worry if you don't, this is only required to enable the mail me my password feature and will not affect the overall running of the application in any other way).

You should now be able to login as admin@timetrack.com with password 'admin'. The admin@timetrack.com user is a member of every role and has full access to the application.

The Sample Application

The following screen shots show sample usage scenarios for different roles.

All Users

Users login to the application. Use admin@timeTrack.com and password admin for full access to the demo.

Figure 7: Logging In

Consultant

A Consultant can then see what time they logged in the last x weeks.

Figure 8: Users personal time sheet summary

from here the Consultant can log time against a contract using either Add New Task or Add Profile Day, or edit an existing entry.

Figure 9: Recording Time

ContractAdmin

A ContractAdmin is able to define Contracts, monitor their status and create or edit user profiles. At the moment, the ContractAdmin is not able to restrict the contracts a user can log time against, so a user can log time against any contract (that has not been deleted).

Figure(s) 10: Contract Definition Screens

Analyst

The timeTrackDemo provides a sample query interface to show members of the Analyst role all time logged for any specified day.

Figure 11: Sample Enquiry Function

Administrator

The Administrator is responsible for setting up users and maintaining roles.

Figure 12: Account Admin and User Management

Any Limitations or Further Work

The timeTrack Demo is a Case Study implementation, and while it contains the framework and base functionality, there are a few enhancements and extensions that could be made. The following is a list of the ones that occurred to me

  • The application does not allow you to record a time period that spans midnight. This is in part intentional, to ensure that all weekend and out of hours work is correctly accounted for and charged.
  • On the EditContractData screen, you could give ContractAdmins the ability to log time for any user, by means of a drop down list box. All other users would still be presented with a read only textbox.
  • The case study does not include reports. I have however discussed some of the issues surrounding the creation of OLAP cubes from the contract data.
  • A Mobile interface would allow Consultants to login and record time from their PDA or phone.
  • The Role based security is too tightly coupled to the Navigation control. Extending the application to use permissions within roles would improve the flexibility of the model.
  • The User and Profile tables are not directly linked. I had considered using triggers to automatically create a default Profile when a New User was registered. However, I felt that the assignment of default contracts was the responsibility of a delivery manager or project supervisor rather than a security or network support function, and so left the two distinct.

Hopefully, the ideas presented above and the timeTrack Demo application have given you a better understanding of how an ASP.NET application hangs together. If you do develop the application in any significant way please drop me an e-mail as I'd be very interested to see how you extended it.

Conclusion

In this case study I have shown how to go from a set of business requirements to a full-featured application by following a simple user control based development model. Throughout the discussion I have highlighted various features of ASP.NET, including custom forms validation, user control creation, fragment caching, the validation controls and state management.

The purpose of this article was to provide an end to end example of an ASP.NET application for those new to ASP.NET and to introduce some of the powerful new features .NET brings to web application development. Please feel free to e-mail me with any comments, queries or suggestions.

Case Study Information
Author | Brian Boyce
Technical Editor | Adam Ryland
Project Manager | Helen Cuthill
Reviewers | David Williams, Daniel Cazzulino, Slavomir Furman

If you have any questions or comments about this case study, or suggestions for further pieces, please contact the technical editor. If you wish to talk about the case study with other subscribers, please visit our forum here.


Runtime Reverse Engineering with System.Diagnostics

Runtime Reverse Engineering with System.Diagnostics
http://www.csharptoday.com/content.asp?id=1775
Article Rating: not yet rated
Published on May 27, 2002
Abstract
In this case study, Tony Loton looks at how you can trace the execution of a .NET application to find out which methods are invoked, on which objects, by which objects, and in what order; so you'll discover how your application really behaves at runtime. You'll learn how to turn tracing on and off, and how to configure the tracing behavior outside of the application. As the logical conclusion to this work we'll build a UML sequence diagram from an application trace, which could be used for comparison against an original dynamic model design or simply as an aesthetic form of presentation.
Case Study

In this case study we'll look at how you can trace the execution of a .NET application to find out which methods are invoked, on which objects, by which objects, and in what order; so you'll discover how your application really behaves at runtime.

You'll learn how to turn tracing on and off, and how to configure the tracing behavior outside of the application. Though we'll be using C# as the implementation language, our use of the System.Diagnostics classes should be equally applicable to any .NET language.

As the logical conclusion to this work we'll build a UML sequence diagram from an application trace, which could be used for comparison against an original dynamic model design or simply as an aesthetic form of presentation.

Introduction

In a previous case study, "C# to UML Reverse Engineering" (http://www.csharptoday.com/content.asp?id=1742&WROXEMPTOKEN=618658Zezi9SpDFSZgQps5Xv6u), I discussed techniques for reverse engineering a C# source file or any .NET assembly into a UML class diagram. I ended that case study by saying this.

"As implemented in many modeling tools, the term reverse engineering can be taken in a limited sense to mean static reverse engineering, and that's what we've dealt with in this case study. Start with a non-executing binary or source code file and end up with a static UML class diagram.

UML is not limited to static modeling and the UML notation supports dynamic modeling of the execution behavior of applications. I'm thinking here of sequence diagrams and collaboration diagrams, which you normally draw up-front to describe how you expect the application to behave. Wouldn't it be good if you could reverse engineer those diagrams too, to show how the application really does behave when you run it?"

In the discussion that follows we'll look at doing exactly that, and if you're not a UML fan don't tune out just yet. While a UML sequence diagram is the logical conclusion to this work, I'm sure you'll pick up something useful along the way in terms application profiling in general. no UML required.

Judging by the responses to a Java article that I wrote on the same subject some time ago, this topic should be interesting to a wide audience, so here goes.

System Requirements

You'll need the .NET Framework SDK that's available for free from the Microsoft web site at http://msdn.microsoft.com/netframework/?WROXEMPTOKEN=618658Zezi9SpDFSZgQps5Xv6u, or the Visual Studio .NET that's not. Either way, you'll need to be able to launch a command window that's aware of the .NET environment, which VS.NET users can do by choosing the Microsoft Visual Studio .NET / Visual Studio .NET Tools / Visual Studio .NET Command Prompt program from the Windows Start menu.

If you'd like to take advantage of the Rational Rose script that I'll offer for rendering the results in UML, you'll need a copy of Rational Rose '98 or later. You could try picking up a trial copy from http://www.rational.com/tryit/rose/index.jsp?WROXEMPTOKEN=618658Zezi9SpDFSZgQps5Xv6u.

Defining The Problem

The problem I've set myself is that of capturing information from a running application so that we can trace which operations have been called, on which objects, in what order. That is, to come up with a trace like this:

which when (optionally) rendered as UML would be a sequence diagram like this:

Regardless of the representation used, what this information tells us is how the application really behaves at run time, so that you can compare that behavior with your original dynamic model. You did produce one at design time, didn't you? Alternatively it could prove to be a very handy technique for debugging.

Case Study Structure

  • In the "Defining the Solution" section we'll define the solution as comprising some instrumentation code, a compilation flag, and an application configuration file. Plus an optional Rational Rose script to build the UML sequence diagram.
  • In the "Component Design and Coding" section we'll step through the code of a program to insert instrumentation automatically into any application, and we'll look at the code for the Rose script.
  • In the "Sample Application" section I'll offer a simple Banking program, which we'll trace and for which I'll present the trace results.

Defining The Solution

It's not the first time I've looked at this kind of problem. The first time resulted in an article for Java Developer's Journal (March 2001) entitled "Using the Java Platform Debugger Architecture" and the second time resulted in chapter 10 "Servlet Debugging Techniques" of the Wrox Press "Professional Java Servlets 2.3" book (http://www.wrox.com/ACON11.asp?WROXEMPTOKEN=141193ZxsCGPbA0ntBAQn4Emvg&ISBN=186100561X). In the first case, the technology employed was the Java Platform Debugger Architecture (JPDA), and in the second case it was Servlet Filters and Event Listeners.

For the .NET solution we'll use two classes from the .NET Framework System.Diagnostics namespace: System.Diagnostics.Trace and System.Diagnostics.StackTrace. We'll instrument an application by inserting a few lines of code at the start of each method:

// ** START INSTRUMENTATION **

Get a stack trace that tells us where we are in the running application:

StackTrace stackTrace=new StackTrace();

The top stack frame will tell us which object / method is executing, and the next stack frame will tell us which (other) object invoked the method:

StackFrame topFrame=stackTrace.GetFrame(0);
StackFrame nextFrame=stackTrace.GetFrame(1);

Assume no calling object unless there really is a second stack frame, in which case we'll record the class (declaring type) of the calling object:

String __callingObject=":";
if (nextFrame!=null) __callee=":"+nextFrame.GetMethod().DeclaringType;

Now write out the calling object, the method that was called (i.e. the method into which this code has been inserted), and the class of the current object:

System.Diagnostics.Trace.Write(__callee);
System.Diagnostics.Trace.Write("|"+topFrame.GetMethod());
System.Diagnostics.Trace.WriteLine("|:"+topFrame.GetMethod().DeclaringType);

//
            ** END INSTRUMENTATION **

The result will be output as follows, with pipe-separated entries for the calling class (none on the first line), the method invoked, and the receiving class. Don't worry about the colon ":" characters just yet, which are there to allow for a future enhancement in which we would specify each object as instanceID:ClassName rather than just :ClassName.

:|Void Main(System.String[])|:Banking.Bank
:Banking.CurrentAccount|Void .ctor()|:Banking.Account
:Banking.Bank|Void .ctor()|:Banking.CurrentAccount
:Banking.CurrentAccount|Void .ctor()|:Banking.Account
:Banking.Bank|Void .ctor()|:Banking.CurrentAccount
:Banking.Bank|Double directdebit(Double)|:Banking.CurrentAccount
:Banking.SavingsAccount|Void .ctor()|:Banking.Account
:Banking.Bank|Void .ctor()|:Banking.SavingsAccount
:Banking.Bank|Double applyInterest(Double)|:Banking.SavingsAccount

But we won't get that at all unless we compile the instrumented application with the /d:TRACE option with this command:

> csc /d:TRACE Banking.cs

.and have a supporting application configuration file (named Banking.exe.config) with the following content:

<configuration>
<system.diagnostics>
<trace autoflush="true" indentsize="4">
<listeners>
<add name="myListener"
type="System.Diagnostics.TextWriterTraceListener"
initializeData=".\myListener.log" />
<remove type="System.Diagnostics.DefaultTraceListener"/>
</listeners>
</trace>
</system.diagnostics>
</configuration>

In that file we're telling the .NET Runtime to add a new listener for trace events, of type System.Diagnostics.TextWriterTraceListener, with the output destination specified as the file myListener.log. This listener will replace the default listener, which we've asked to be removed above.

So that's the basis of our solution comprising some instrumentation code, compilation with the d:/TRACE flag, and a supporting configuration file. Before moving on to the coding and demonstration I'll tell you a little more about each of those and about the optional Rational Rose scripting.

Instrumentation with System.Diagnostics classes

The "Introduction to Instrumentation and Tracing" within the "Visual Basic and Visual C# Concepts" section of the .NET documentation defines instrumentation as the ability for an application to incorporate:

  • Code tracing
  • Debugging
  • Peformance counters
  • Event logs

Those kinds of instrumentation are supported by the classes of the System.Diagnostics namespace. We'll be concentrating on code tracing and therefore the System.Diagnostics.Trace class. That said, the techniques discussed here could be extended to debugging, performance counters, and event logs ; also thanks to the Trace class, or by incorporating the Debug class from the same namespace.

The Trace and Debug classes have various static methods that allow you to write out messages to the output window, a text log file, or some other destination. Some of these methods are conditional, such as Trace.Assert(), and some are unconditional such as Trace.Write(). The code listed in the previous section uses the latter.

Trace output is not written out directly, but is intercepted by one or more trace listeners. You can add trace listeners into the Listeners collection, to supplement the DefaultTraceListener or to replace it completely. In code you could replace the default listener with new listener like this:

Trace.Listeners.Clear();
Trace.Listeners.Add(new TextWriterTraceListener(Console.Out));

In our solution we're not doing that because it would couple the tracing behavior too closely to the application code. Instead we're configuring the trace listener(s) via an application configuration file.

Application Configuration Files

Configuration files are XML files that can be used to change settings without recompiling applications. For an executable-hosted application (which is what my sample Banking application is) the configuration file is located in the same directory as the application. It has the same name plus a .config extension.

Banking.exe is the application.

Banking.exe.config is the corresponding configuration file.

As background information I'll tell you that configuration files for ASP.NET hosted applications follow a different scheme. In that case there will be a set of configuration files located at various levels in the URL path of the application. For an Internet Explorer application, the relevant configuration file will be specified by the <link> tag. You can read more about the various kinds of configuration files, for the various kinds of applications, at ms-help://MS.VSCC/MS.MSDNVS/cpguide/html/cpconapplicationconfigurationfiles.htm.

Because it is a standard XML file, an application configuration file has a schema to which the contents must conform. For Debug and Trace Settings, the schema elements (from ms-help://MS.VSCC/MS.MSDNVS/cpgenref/html/gngrftracedebugsettingsschema.htm in the .NET help documentation) are:

Element | Description
<add> element for the <listeners> element | Adds a listener to the Listeners collection.
<add> element for the <switches> element | Specifies the level where a trace switch is set.
<assert> | Specifies whether to display a message box when you call the Debug.Assert method; also specifies the name of the file to write messages to.
<listeners> | Specifies a listener that collects, stores, and routes messages.
<remove> | Removes a listener from the Listeners collection.
<swithces> | Contains trace switches and the level where the trace switches are set.
<system.diagnostics> | Specifies trace listeners that collect, store, and route messages and the level where a trace switch is set.
<trace> | Contains listeners that collect, store, and route tracing messages.

In our solution we're making use of the <system.diagnostics>, <trace>, <listeners>, <add> and <remove> elements. We're not using <switches> in our solution, so for completeness I'll just make you aware of what a trace switch is.

The System.Diagnostics.TraceSwitch class provides a multilevel switch to control tracing and debug output without recompiling your code, and you can use the trace levels to filter out messages based on their importance. Switches may be added and removed in the application configuration file, as with listeners.

In summary, the purpose of an application configuration file is to allow certain behaviors - including tracing - to be changed without requiring the application to be recompiled.

Compiling with the /d:TRACE flag

To enable tracing in a C# application you can either add a preprocessor directive #define TRACE to the top of the source file or add the d:/TRACE flag to the compiler command line. In this case study we're doing the latter, and you might like to try compiling with and without that flag to see the effect.

If you're working with Visual Basic you can enable tracing in a similar way by adding the /d:TRACE=True flag to the compiler command line.

If you preference is to compile from Visual Studio .NET itself, rather than from the command line, it's worth keeping in mind that by default the IDE compilation has tracing enabled. So unless you change it, you're released application could well be pumping out tracing information that you don't want it to. You can change this behavior via the Project Properties dialog, in the Configuration Properties / Build / Code Generation entry.

Rational Rose Scripting

Rational Rose has been one of the most widely used Object Modeling CASE tools in recent years, and as far back as I can remember it has had a scripting language that provides programmatic access to the Rose application and the model(s) housed within it.

The scripting language is called Summit BasicScript, and if you're familiar with early versions of Visual Basic (many visitors to this site, I suppose) you'll be quite comfortable with it. Access to the UML model (actually the meta-model) is provided by the Rose Extensibility Interface (REI) classes that represent the model elements - classes, methods, and so on - as well as the UML diagrams.

We'll make use of those classes via the scripting language, but the same set of classes is available for manipulation from your favorite Microsoft language via the OLE/COM objects of Rose Automation. If that's your preference, it shouldn't be too difficult to convert the script to a VB program, or for that matter even a C# program.

You can find more information about the scripting language and the extensibility classes in the Rose Entensibility Interface / Summit BasicScript Reference and Rose Extensibility Interface / Rose Extensibility Reference Rational Rose help topics.

Component Design and Coding

As described above, the key to our solution will be a few lines of instrumentation code inserted at the start of each method in the original program. Of course, you don't want to go through your program pasting that code into each and every method do you? So we'll automate it with a C# program.

We'll then implement a Rational Rose script to import the trace results and to build a UML sequence diagram reflecting the sequence of interactions between objects - i.e. the scenario - of the running program. That part's not compulsory, but quite interesting.

Automating Insertion of the Instrumentation Code

The following code of the InsertInstrumentation class will be used to read in any C# program and write it out again with instrumentation code included.

using System;

We'll make use of System.IO so as to read the original source file, and System.Text.RegularExpressions to help with parsing of that file. If you've read other articles that I've written you'll have been expecting regular expressions to appear somewhere, and - if not - you can get up to speed with them by reading my article "Introducing .NET Regular Expressions with C#" at http://www.csharptoday.com/content.asp?id=1786&WROXEMPTOKEN=618658Zezi9SpDFSZgQps5Xv6u.

using System.IO;
using System.Text.RegularExpressions;

Our class will be called InsertInstrumentation, housed within the reverse namespace:

namespace reverse
{
public class InsertInstrumentation
{

In the main method we open the original source file and call the instrument() method to insert the appropriate instrumentation:

        public
            static void Main(string[] args)
{
String
            inputString="";
String inputLine;

StreamReader sr =
            File.OpenText(args[0]);

while
            ((inputLine=sr.ReadLine())!=null)
inputString=inputString+"\n"+inputLine;

sr.Close();

instrument(inputString);
}

The instrument() method

The instrument() method is where we read the text of the input file, then write it out again with the instrumentation code inserted into each method:

private static void instrument(String text)
{

Our output file will contain modified source code with instrumentation. That output will only compile if we start by importing the System.Diagnostics namespace classes:

System.Console.WriteLine("using System.Diagnostics;");

As we go, we'll be keeping track of the previous match of the regular expression:

Match previousMatch=null;

Here we set up the regular expression itself. In a nutshell it matches each method declaration, which it reads into a named group (think of it as a variable) called "<method>", and any preceding code which is read into a named group "<priorcode>".

string regex=@"(?s)(?<priorcode>.*?)"
+@"(?<method>(public|private)"
+@"\s+[\w\s]*\w+\s*\(.*?\)\s*\{)";
Regex methodRegex=new Regex(regex);

That regular expression is not perfect because it handles only those methods that are declared explicitly as public or private. Once you become familiar with regular expressions - perhaps via my earlier article - you may wish to revisit this to account for protected and internal methods, plus those with no explicit access modifier.

Now we'll match the regular expression against the input text (the original source code) and step through the matching portions:

            Match
            methodMatch = methodRegex.Match(text);

for ( ;
            methodMatch.Success;
methodMatch =
            methodMatch.NextMatch())
{

For each match, we'll pick out the priorcode and the method declaration, and we'll write out the priorcode:

string priorcode=""+methodMatch.Groups["priorcode"];
string method=""+methodMatch.Groups["method"];
System.Console.WriteLine(priorcode);

Next we write out the method declaration along with the instrumentation code, via a separate method named writeInstrumentation(). We record this match as previousMatch and go round again:

writeInstrumentation(method);
previousMatch=methodMatch;
}

Because of how the regular expression works, we'll be left with some code at the end that we've not yet written out. Here we tidy up by writing the remaining code:

            if
            (previousMatch!=null && previousMatch.Success)
{
int
            remainingCodeStart=previousMatch.Index
+previousMatch.Length;
int
            remainingCodeLength=text.Length
-remainingCodeStart;
System.Console.WriteLine(text.Substring(remainingCodeStart
,remainingCodeLength));
}
}

You will have noticed in the above code that - upon discovering each match of the regular expression - we call a writeInstrumentation() method to actually write out the instrumentation code.

The writeInstrumentation() method

Here is that method, which takes in the method signature text as its sole parameter:

private static void writeInstrumentation(string method)
{

We write out the method signature itself as part of the modified source code, along with a comment marking the start of the instrumentation:

System.Console.WriteLine(method);
System.Console.WriteLine("\n// ** START INSTRUMENTATION **");

Now we write the instrumentation code that was explained previously in the Defining the Solution section:

System.Console.WriteLine(
"StackTrace stackTrace=new StackTrace();");
System.Console.WriteLine(
"StackFrame topFrame=stackTrace.GetFrame(0);");
System.Console.WriteLine(
"StackFrame nextFrame=stackTrace.GetFrame(1);");

System.Console.WriteLine("String __callingObject=\":\";");
System.Console.WriteLine(
"if (nextFrame!=null) "
+"__callingObject=\":\"+nextFrame.GetMethod().DeclaringType;");

System.Console.WriteLine(
"System.Diagnostics.Trace.Write(__callingObject);");
System.Console.WriteLine( "System.Diagnostics.Trace.Write(\"|\"+topFrame.GetMethod());");
System.Console.WriteLine(
"System.Diagnostics.Trace.WriteLine(\"|:\"+topFrame.GetMethod().DeclaringType);");

And we mark the end of the instrumentation with a comment:

            System.Console.WriteLine("\n//
            ** END INSTRUMENTATION **");
}

}
}

Remember that the result will be the original source file unchanged, save for the inclusion of instrumentation code at the start of every method (including constructors) like this:

public double applyInterest(double
            rate)
{

// ** START INSTRUMENTATION **
StackTrace
            stackTrace=new StackTrace();
StackFrame
            topFrame=stackTrace.GetFrame(0);
StackFrame
            nextFrame=stackTrace.GetFrame(1);
String
            __callingObject=":";
if (nextFrame!=null)
            __callingObject=":"+nextFrame.GetMethod().DeclaringType;
System.Diagnostics.Trace.Write(__callingObject);
System.Diagnostics.Trace.Write("|"+topFrame.GetMethod());
System.Diagnostics.Trace.WriteLine("|:"+topFrame.GetMethod().DeclaringType);
//
            ** END INSTRUMENTATION **

Balance=Balance*rate;
return
            Balance;
}
}

When we compile and run the instrumented application, as I'll show later, we'll get the logged output shown previously in Defining the Solution. That application trace may be further processed or displayed in a number of ways, one of which is as a UML sequence diagram in Rational Rose.

The Rational Rose Script (importScenario.ebs)

If you're using Rational Rose you'll no doubt find this script very interesting, and - if not - you may skip this discussion. But if you're working with another UML modeling tool you might just want to look through it anyway as it will give you some ideas about what you could do via the scripting language of your own tool.

In the Main subroutine we ask for an input file (the log file produced by the instrumentation) via a file dialog. We then pass the file name into the ImportScenario subroutine:

Sub Main
FileName$ = OpenFileName$ ("Import", "All files:*.*")
If FileName$ <> "" Then ImportScenario FileName$
End Sub

The ImportScenario subroutine follows. I've used the term scenario because, in UML terminology, a scenario is what we'll be presenting as a sequence diagram. You can think of a scenario as a usage pattern for application under investigation:

Sub ImportScenario (FileName As String)

Here we get hold of the current Rational Rose model, so that we can add to it:

Dim MyModel As Model
Set MyModel = RoseApp.CurrentModel

We'll be creating a new scenario (sequence) diagram with the same name as the log file containing the application trace, e.g. myListener.log or - more meaningfully - Banking.log, but only if that scenario does not already exist:

scenarioID = MyModel.RootCategory.ScenarioDiagrams.FindFirst (FileName)
Dim sDiagram As ScenarioDiagram
If scenarioID <> 0 Then
Set sDiagram = MyModel.RootCategory.ScenarioDiagrams.GetAt (scenarioID)
Else
Set sDiagram = MyModel.RootCategory.AddScenarioDiagram (FileName,1)
End If

Our sequence diagram will comprise object instances that exchange messages, with each instance having a unique ID. These variables will be used to keep track of the instances.

Dim instanceIDs() As String
ReDim instanceIDs(1)
instanceIDs(1)=""
Dim theInstances As New ObjectInstanceCollection

Note that our output log file supports the notion of sender and receiver objects of the form instanceID:ClassName, but in this case study we're actually not recording unique instance IDs for each object. The log file contents will always refer to objects as :ClassName and the resulting sequence diagram will be as shown earlier, with anonymous instances as preferred by many object modelers. If we improve the instrumentation in future to include instance IDs, this script will be ready.

Each message in our sequence diagram will have an ordinal sequence number, so here we start with number 1:

seq%=1

We open the viewport window to report the progress of the script and we open the input file for read-only access:

Viewport.Open
Open FileName$ For Input Access Read As #1

Now we step through the file, reading each line as we go:

While Not EOF (1)
Line Input #1, inString$

Each entry in the file will contain three pipe-separate fields: the sender (caller), the message (method), and the receiver (callee):

caller$=Item$(inString$,1,1,"|")
callee$=Item$(inString$,3,3,"|")
method$=Item$(inString$,2,2,"|")

The caller and callee objects could be identified by an instance ID and a class name, separated by a colon. Remember that for this case study, the instance IDs will actually be blank, but the script will take them if they're there:

callerID$=Item$(caller$,1,1,":")
callerClass$=Item$(caller$,2,2,":")
calleeID$=Item$(callee$,1,1,":")
calleeClass$=Item$(callee$,2,2,":")

For simplicity we'll put the two classes into the root category (root package) of the model, if they don't already exist there. A more advanced script could build a category (package) hierarchy reflecting the namespaces of the classes:

Dim thisCat As Category
Set thisCat=MyModel.RootCategory

If thisCat.GetAllClasses().FindFirst(callerClass$) = 0 Then
Set thisClass=thisCat.AddClass (callerClass$)
End If

If thisCat.GetAllClasses().FindFirst(calleeClass$) = 0 Then
Set thisClass=thisCat.AddClass (calleeClass$)
End If

Each object instance will be uniquely identified by a combination of its ID (not present in our log file) and its class, plus we'll use two variables to hold the caller and callee instances respectively.

callerInstanceID$=callerID$+":"+callerClass$
calleeInstanceID$=calleeID$+":"+calleeClass$
Dim callerInstance As ObjectInstance
Dim calleeInstance As ObjectInstance

We'll look to see if the caller instance has been recorded in the instanceIDs list and is therefore already on the diagram. We don't want to add a new object timeline every time this instance is the sender of a message:

foundCallerPos%=0
size%=UBound(instanceIDs)
For pos%=1 To size%
If instanceIDs(pos%)=callerInstanceID$ Then
foundCallerPos%=pos%
End If
Next pos%

If the caller instance is not already on the diagram, we add it to the diagram and also to the list of known instances:

If foundCallerPos% = 0 Then
Set callerInstance=sDiagram.addInstance(callerID$,callerClass$)
ReDim Preserve instanceIDs(size%+1)
instanceIDs(size%+1)=callerInstanceID$
foundCallerPos%=size%+1
theInstances.Add callerInstance
End If

Same again for the callee instance. Add it to the diagram if it's not already there:

foundCalleePos%=0
size%=UBound(instanceIDs)
For pos%=1 To size%
If instanceIDs(pos%)=calleeInstanceID$ Then
foundCalleePos%=pos%
End If
Next pos%
If foundCalleePos% = 0 Then
Set calleeInstance=sDiagram.addInstance(calleeID$,calleeClass$)
ReDim Preserve instanceIDs(size%+1)
instanceIDs(size%+1)=calleeInstanceID$
foundCalleePos%=size%+1
theInstances.Add calleeInstance
End If

Make sure we have a handle on the sequence diagram representations of the caller - and callee - instances, then add a message (the method signature) from one to the other:

Set callerInstance=theInstances.GetAt(foundCallerPos%-1)
Set calleeInstance=theInstances.GetAt(foundCalleePos%-1)

Dim theMessage As Message
Set theMessage=sDiagram.CreateMessage(method$,callerInstance, calleeInstance, seq%)

Give the user some feedback in the viewport window, increase the sequence number for next time around, and complete the loop:

Print callerClass$,"---",method$,"-->",calleeClass$
seq%=seq%+1
Wend

Finally, close the input file nicely and end the subroutine:

Close #1
End Sub

Sample Application

In this section we'll take a sample application, instrument it, and view the trace results as a simple list. Then we'll reverse engineer a sequence diagram representation using the Rose script.

You'll find everything you need in the download, which contains the following files. Some of them will be obvious immediately and the purpose of the others will become clear as you read on.

Download Contents

File Banking.cs contains the source code for the sample application (see below).

File Banking.exe is a compiled version of the sample application.

File Banking.save is a spare copy of the Banking.cs source code, just in case.

File importScenario.ebs is the Rational Rose script that builds the UML sequence diagram.

File InsertInstrumentation.cs is the source code of the program to add instrumentation.

File InsertInstrumentation.exe is the compiled version.

File instrumented\Banking.cs will be the instrumented version of the sample application.

File instrumented\Banking.exe is the compiled version.

File instrumented\Banking.exe.config is the application configuration file for the sample application.

File instrumented\myListener.log is where the trace results will be written.

File build.bat contains the command csc /d:TRACE Banking.cs to compile the instrumented sample application.

File sampletraces\BankingTrace.mdl represents the sample trace as a Rational Rose model.

File sampletraces\BankingTrace.xls represents the sample trace as an Excel speadsheet.

File sampletrace\BankingTrace.xml represents the sample trace as an XML file, for Internet Explorer.

Banking Application Code

If you've read my previous (Static) Reverse Engineering case study at http://www.csharptoday.com/content.asp?id=1742&WROXEMPTOKEN=618658Zezi9SpDFSZgQps5Xv6u you'll be familiar with my sample Banking program. I've changed it slightly for this case study and the latest version is provided in file Banking.cs of the code download.

I wont give a blow-by-blow account of every single line of code because that's not important, but here is the complete code with a few useful pointers.

using System;

namespace Banking
{

The Account class will be a base class for various types of account, such as CurrentAccount and SavingsAccount. We won't be concerned with the member variables and properties but the constructor will be instrumented:

    public class
            Account
{
private static int
            _nextAccountNumber;

private int _accountNumber;
private
            double _balance;

public
            Account()
{
_accountNumber=_nextAccountNumber;
_nextAccountNumber++;
}

public
            int AccountNumber
{
get { return _accountNumber;
            }
}

public double Balance
{
get { return _balance;
            }
set { _balance=value; }
}
}

The CurrentAccount class extends Account. It will have its constructor and its directdebit() method instrumented:

    public class
            CurrentAccount : Account
{
public
            CurrentAccount()
{
}

public double directdebit(double
            amount)
{
Balance=Balance-amount;
return
            Balance;
}
}

The SavingsAccount class extends Account. It will have its constructor and its applyInterest() method instrumented:

    public class
            SavingsAccount : Account
{
public
            SavingsAccount()
{
}

public double applyInterest(double
            rate)
{
Balance=Balance*rate;
return
            Balance;
}
}

The Bank class has a Main() method to drive the other classes through a typical sequence of events, i.e. a scenario:

public class Bank
{
public static void Main(string[] args)
{
// Current Account #0
CurrentAccount currentAccount=new CurrentAccount();
currentAccount.Balance=100;
System.Console.WriteLine("Current Account "
+currentAccount.AccountNumber
+" has balance "+currentAccount.Balance);

// Current Account #1
currentAccount=new CurrentAccount();
currentAccount.Balance=100;
currentAccount.directdebit(50);
System.Console.WriteLine("Current Account "
+currentAccount.AccountNumber
+" has balance "+currentAccount.Balance);

// Savings Account #1
SavingsAccount savingsAccount=new SavingsAccount();
savingsAccount.Balance=1000;
savingsAccount.applyInterest(1.1);
System.Console.WriteLine("Savings Account "
+savingsAccount.AccountNumber
+" has balance "+savingsAccount.Balance);
}
}
}

Just to clarify, the sequence of events is:

  • Create a CurrentAccount.
  • Create another CurrentAccount and call its directdebit() method.
  • Create a SavingsAccount and call its applyInterest() method.

Instrumenting the application

If you don't have an InsertInstrumentation.exe file, create one by compiling the InsertInstrumentation.cs C# program:

> csc InsertInstrumentation

Now run the InsertInstrumentation program against the Banking.cs source file, taking care to divert the output into the \instrumented directory.

> InsertInstrumentation Banking.cs > instrumented\Banking.cs

For convenience, if you accidentally overwrite the original Banking.cs file with the instrumented version (yes, I've done that!) you should be able to recover by renaming file Banking.save to be Banking.cs.

You might like to take a look at the instrumented version of the application source code before moving on.

Compiling the application

Compile the application in the \instrumented directory with tracing enabled, like this:

> cd instrumented
> csc /d:TRACE Banking.cs

At this point it's worth noting that the \instrumented directory contains a file called Banking.exe.config, which was not generated by the compilation but written in advance by me. You can use my file as the starting point of a configuration file of your own applications by simply renaming it to YourApplication.exe.config. I explained the contents of the file earlier, but as a reminder here it is again:

<configuration>
<system.diagnostics>
<trace autoflush="true" indentsize="4">
<listeners>
<add name="myListener"
type="System.Diagnostics.TextWriterTraceListener"
initializeData=".\myListener.log" />
<remove type="System.Diagnostics.DefaultTraceListener"/>
</listeners>
</trace>
</system.diagnostics>
</configuration>

Note that this application configuration file specified the file myListener.log to take the output of the TextWriterTraceListener.

Running the application

Now we can run the instrumented application and see its output:

> Banking
Current Account 0 has balance 100
Current Account 1 has balance 50
Savings Account 2 has balance 1100

For this case study we're not interested in the output from the program itself, but in the contents of the myListener.log file which are:

:|Void Main(System.String[])|:Banking.Bank
:Banking.CurrentAccount|Void .ctor()|:Banking.Account
:Banking.Bank|Void .ctor()|:Banking.CurrentAccount
:Banking.CurrentAccount|Void .ctor()|:Banking.Account
:Banking.Bank|Void .ctor()|:Banking.CurrentAccount
:Banking.Bank|Double directdebit(Double)|:Banking.CurrentAccount
:Banking.SavingsAccount|Void .ctor()|:Banking.Account
:Banking.Bank|Void .ctor()|:Banking.SavingsAccount
:Banking.Bank|Double applyInterest(Double)|:Banking.SavingsAccount

That shows a complete trace of the constructors and other methods that have been invoked during the execution of the application. Each line shows the calling object (blank on the first line), the message that was sent (or method invoked), and the receiving object.

It's not the most eye catching representation, so to make it clearer I simply opened the log file in Microsoft Excel and applied a little formatting to the columns to give:

In the Case Study Review section I'll offer some more thoughts on presenting and analysing the application trace.

Building the UML Sequence Diagram

If you have access to Rational Rose you can use my script to build a real UML sequence diagram for the scenario represented by the application trace.

First, launch Rose and use the Tools / Open Script menu option to open the importScenario script; or simply double-click the script file to launch Rose with the script already loaded. You can then run the script by pressing the green "Start" button.

Choose the file that contains the logged trace results, i.e. myListener.log:

Then watch the messages that appear in the Viewport window:

Finally you will discover that the Rose model browser contains a set of classes, and a new sequence diagram representing the application trace, as shown here:

That completes the sample application, but did you notice anything odd about the sequence of events? According to that trace it looks as though the Banking.CurrentAccount calls its superclass Banking.Account constructor before it has had its own constructor invoked. This pattern is repeated for every subclass / super class construction and there's not much we can do about it.

To understand what's going on here you have to imagine that the subclass constructor as an implicit first line of code that invokes the constructor of the superclass, and which --in effect - causes the superclass instrumentation to be executed before its own instrumentation. This following diagram might help:

Apart from that, the sequence of events is ordered as you would expect for the sample application.

Case Study Review

In this case study we've investigated a technique for instrumenting a C# application, which allows us to capture the runtime - i.e. dynamic - behavior of the application. The resulting application trace may be useful as a teaching aid (to see demonstrate objects really interact), and for debugging or performance monitoring (if we include method timings in the instrumentation code).

Ultimately it allows us to reverse engineer a UML sequence diagram to complement the reverse engineered class diagrams demonstrated in my previous case study http://www.csharptoday.com/content.asp?id=1742&WROXEMPTOKEN=618658Zezi9SpDFSZgQps5Xv6u. In a typical project you may well draw sequence diagrams for your applications up-front to describe the intended behavior, and now we have a way of validating those diagrams against the actual behavior during testing.

For the instrumentation we used mainly the System.Diagnostics.Trace class, and the System.Diagnotics.StackTrace class to discover the calling (caller) and called (callee) objects. We automated the insertion of instrumentation code into every method by implementing the InsertInstrumentation program. Through that program I demonstrated a practical use for regular expressions.

We looked at how tracing could be enabled and disabled with the /d:TRACE compiler flag and how the tracing behavior could be configured outside of the target application in an application configuration file.

With the help of a Rational Rose script I showed how the end results really could be rendered in UML, and finally I used the sample sequence diagram to highlight an interesting feature about the order of class construction.

During the review phase for this case study, one of the reviewers made a very interesting point that this approach should be a method of last resort. Surely it would be better to insert trace statements as you build an application rather than as an afterthought? My own opinion is that it would be less flexible, i.e. hard-coded, and possibly inconsistent across the application, but it's certainly something you should be thinking about as part of the design.

Notwithstanding the recommendations I've given in the next section, the important messages to take away from this case study are:

  • You can capture information about the behavior of your application as it runs, i.e. you can capture an application trace, using the classes of the System.Diagnostics namespace and the techniques described here. But there may be other ways, mentioned in "Non-invasive Tracing" below.
  • The information you capture may be useful for debugging purposes, for measuring performance (if we included counters in the instrumentation), or simply as a learning tool. Remember the order-of-construction effect that was highlighted in the example?
  • Ultimately you can reverse engineer the runtime (dynamic) behavior of an application and present it as UML; to complement any static reverse engineeringyou may do, or to validate the application designer's original dynamic model.

Remember that although we've used C# in this case study, in .NET all languages are equal. So it shouldn't be too difficult to adapt this approach to work with Visual Basic, C++ or even J# applications.

Limitations and Further Work

In this case study we've made a good start on the problem of reverse engineering the behavior of an application as it runs. In principle it works, and there are a few improvements that I can think of that you might like to work on.

Intermediate File Format

I kept the intermediate file format quite simple, mainly because my Rose scripting skills are not up to parsing a more complex structure such as XML. But I do think that XML would be a sensible choice for the intermediate results, so its something you might like to think about. I envisage an XML file like this:

<trace>
<interaction seq="1">
<sender>:</sender>
<message>Void Main(System.String[])</message>
<receiver>:Banking.Bank</receiver>
</interaction>

<interaction seq="2">
<sender>:Banking.CurrentAccount</sender>
<message>Void .ctor()</message>
<receiver>:Banking.Account</receiver>
</interaction>
</trace>

The rendering in Internet Explorer would be:

Processing the Results

We've processed the trace results by loading them into an Excel spreadsheet and by constructing a UML sequence diagram in Rational Rose. By writing out the trace as XML (above) you could open them up to a wider range of analysis or presentation tools, or maybe write a dedicated trace viewer yourself.

Remember that we've decoupled the trace handling from the application itself through the use of an application configuration file. In that file we specified a TextWriterTraceListener for writing the results to a log file, which replaced the default trace listener that would write the results as console output.

You can write your own TraceListener classes and add them in the same way, thus opening up a whole new set of possibilities. What about a trace listener that sends out each trace event as a SOAP message to a remote real-time debugging or profiling tool?

Alternative Instrumentation

As implemented here we've made use of the System.Diagnostics.StackTrace class to obtain information on the calling class, the called class, and the method that was invoked, and we've written that information out via the System.Diagnostics.Trace.Write() method. In a typical method, the instrumentation code looks like this:

public double applyInterest(double
            rate)
{

// ** START INSTRUMENTATION **
StackTrace
            stackTrace=new StackTrace();
StackFrame
            topFrame=stackTrace.GetFrame(0);
StackFrame
            nextFrame=stackTrace.GetFrame(1);
String
            __callingObject=":";
if (nextFrame!=null)
            __callingObject=":"+nextFrame.GetMethod().DeclaringType;
System.Diagnostics.Trace.Write(__callingObject);
System.Diagnostics.Trace.Write("|"+topFrame.GetMethod());
System.Diagnostics.Trace.WriteLine("|:"+topFrame.GetMethod().DeclaringType);
//
            ** END INSTRUMENTATION **

In fact, the instrumentation code looks exactly like that, regardless of the method it's in.

Now think about an alternative instrumentation that is method-specific and which therefore sends out some method related information. like the parameter values. I'm thinking about a line of code like this inserted into the method above.

System.Diagnostics.Trace.WriteLine(
"|applyInterest( rate="+rate+" )");
);

That information would be quite useful and you might like to think about inserting such lines of code via the InsertInstrumentation program. I think the contents of the regular expression's <method> group will tell you everything you need to know about the parameter names.

Finally in terms of improved instrumentation, you could think about how to identify the actual object instance on which the method has been invoked; not just the class. Remember that the Rational Rose script can cope with individually-identifiable instances when building the sequence diagram.

Non-Invasive Tracing

The approach I've described decouples the tracing behavior from the application code thanks to the application configuration file. The tracing can be enabled or disabled at the flick of a compiler switch, and a lot of typing is saved by the automated insertion of the instrumentation code.

All that said, this approach is essentially invasive; which means we must modify the original application code. In contrast, the runtime reverse engineering solutions that I devised using Java were non-invasive, using the Java Platform Debugger Architecture (JPDA) for applications in general and using sevlet filters and event listeners for web applications specifically.

Non-invasive techniques for tracing of .NET applications are worth investigating. You could try driving the Machine Debug Manager using DCOM, or maybe take advantage of Windows Management Instrumentation (WMI). I'm no expert on the latter, but it looks promising and you can read Richard Conway's article on that technology at http://www.csharptoday.com/content.asp?id=1807&WROXEMPTOKEN=618658Zezi9SpDFSZgQps5Xv6u. Either way, my past experiences tell me that by using these non-invasive techniques it could be much more difficult to capture the method-specific information that I described above.

Case Study Information
Author | Tony Loton
Technical Editor | Adam Ryland
Project Manager | Helen Cuthill
Reviewers | Andrew Krowczyk, Saurabh Nandu, John Hodgson, Adam Ryland

If you have any questions or comments about this case study, or suggestions for further pieces, please contact the technical editor. If you wish to talk about the case study with other subscribers, please visit our forum here.


Migrating a Windows DNA Application to .NET Architecture

Migrating a Windows DNA Application to .NET Architecture
http://www.csharptoday.com/content.asp?id=1847
Article Rating: 5
Published on June 24, 2002
Abstract
Many companies want to migrate their applications to the .NET Framework. Considering the paradigm shift in the application development in the .NET Framework, in most cases, it is not feasible to take the entire application and migrate it in a single go. It is also not possible to throw away the existing investments in the COM based code just to take advantage of the new .NET platform. To make matters worse, in most of the cases, our timeline may not allow a complete migration of the entire application. In this case study, Thiru Thangarathinam considers an example windows DNA application and then demonstrates how to migrate this application to the .NET platform in a phased manner.
Case Study

With the release of the .NET Framework, many companies want to migrate their applications to this new architecture to take advantage of its compelling features. Considering the paradigm shift in the application development in the .NET Framework, in most of the cases, it is not feasible to take the entire application and migrate it in a single go. Also it is not possible to throw away the existing investments in the COM based code, just to take advantage of the new .NET platform. To make matters worse, in most of the cases, our timeline may not allow a complete migration of the entire application. In this case study, we will consider an example windows DNA application and then demonstrate how to migrate this application to the .NET platform in a phased manner.

Introduction

As we already mentioned, a lot of companies have made tremendous amounts of investment in COM based code that can't simply be thrown away due to the fact that we want to migrate the application to the .NET platform. However there are many new compelling features available in .NET that can only be leveraged by migrating the application to the .NET platform. So, we need to strike a compromise between these two factors and decide how best to take advantage of the new platform without throwing away our existing COM based code investments. The recommended approach in this case is to migrate only portions of the application to start with, and bring the application up and running. The rest of the application can then be slowly migrated to the .NET platform. In this case study, we will consider a Windows DNA application and demonstrate how this application can be migrated to the .NET platform in a phased approach. We will also demonstrate how to increase the business value of the application by migrating to the .NET platform and taking advantages of its new features.

Introduction to the Case Study

This case study is based on a web site called NorthwindApp, which provides a one-stop shop for consumers that want to find out information such as the products that are on sale, availability in different stores, comparison of price across different stores. It also provides consumers with the product information and reviews of them, and highlights stores within a specified mileage radius that feature the specific item. NorthwindApp also supplies the consumers with access to an online guide that provides the best local values and product information.

As with any other enterprise solution using the windows platform, the NorthwindApp was built using the Windows DNA methodology using technologies such as ASP, VB COM Components, Sql Server, COM+, and so on. Now to take advantage of the new features of the .NET Framework, the creators of NorthwindApp want to migrate the application to the .NET Framework. Due to their business restrictions, they don't want to migrate the entire application, throwing away their existing COM based code, but some of the features of the .NET Framework such as XML Web Services are so compelling that they want to be able to implement those features in order to be able to offer a superior service to their customers.

After having considered all the possible approaches, finally they decide to adopt a phase-by-phase approach, meaning that they start the process by migrating only certain parts of the application to .NET managed code, allowing them to communicate with the remaining parts of the application (that are COM based) using COM Interop. An approach of this nature allows them to realize the benefits of the .NET Framework without forcing them to make a complete migration, which is very costly most of the time.

Business Processes

Clearly the scope of the case study is to show how to migrate an N Tier windows DNA application to the .NET architecture. But before we do that, we will see how this application is implemented in the Windows DNA architecture. Once we have understood the individual pieces of the application, we will then focus on how to upgrade the specific functionality to the .NET architecture.

Since the focus of this case study is on migrating the application to the .NET Framework, we will restrict ourselves to discussing only the following business processes:

  • Catalog browsing process - Users can browse through the list of categories available on the site, and they can also look at all the individual products in the category by clicking the category name hyperlink.
  • Products listing process - In this process, the list of products available in a particular category is shown with a hyperlink that takes the users to more information about a product.
  • Products details process - When the user clicks the hyperlink of a particular product, the user is taken to the page where detailed explanation about the particular product is shown

Implementation

Now that we have understood the business processes involved, let us construct the individual building blocks that are required for implementing this solution. We will split our discussion of the remaining parts of the case study into the following sections:

  • Database Design
  • Implementation of Phase 1 Architecture that uses windows DNA Architecture
  • Implementation of Phase 2 Architecture that uses a combination of managed code and unmanaged code. In this phase, we create an ASP.NET web services layer that acts as a managed wrapper around the existing COM objects. We will also rewrite the entire presentation layer in ASP.NET that talks to the ASP.NET web service layer.
  • Implementation of Phase 3 Architecture that has the entire application migrated to the .NET platform. In this phase, we rewrite the underlying COM objects (wrapped around by the ASP.NET web services layer) in managed code.

We will discuss the specifics of the different phases of the architecture as we move into the each phase of the implementation. We will start our discussion of the implementation by looking at the database design that is required to support our application.

Database Design

As we can see from the architecture of our system, we will be using the Northwind database for all of the information that we display on the web application.

Northwind database design

This database is basically used to retrieve information about the categories and products available in the Northwind database. It consists of Categories and Products tables. The entity relationship diagram for the Northwind database is as follows:

The above diagram shows that the ProductID column acts as the primary key for the Products table, and CategoryID column is used as the primary key for the Categories table. It also shows that the referential integrity is enforced between the Products and Categories tables through the use of the CategoryID column.

The Categories table has the following structure.

Name | DataType | Length | AllowNull | Description
CategoryID | int | 4 | No | Indicates the category id
CategoryName | nvarchar | 15 | No | Represents the category name
Description | ntext | 16 | Yes | Provides a description for the category
Picture | image | 16 | Yes | Category Image

The table structure for the Products table is as follows:

Name | DataType | Length | AllowNull | Description
ProductID | int | 4 | No | Represents the product ID and acts as the key
ProductName | nvarchar | 40 | No | Represents the product name
SupplierID | int | 4 | Yes | Represents the supplier ID
CategoryID | int | 4 | Yes | Indicated the category ID
QuantityPerUnit | nvarchar | 20 | Yes | Products per unit
UnitPrice | money | 8 | Yes | Price of the product
UnitsInStick | smallint | 2 | Yes | Available units in stock
UnitsOnOrder | smallint | 2 | Yes | Units on order
ReorderLevel | smallint | 2 | Yes | Reorder quantity
Discontinued | bit | 1 | No | Specifies whether the product is discontinued

Let us consider the stored procedures in the Northwind database that we will be using in our application. The stored procedure GetCategories returns all the categories present in the Categories table:

CREATE procedure GetCategories
as
begin
set nocount on
select * from Categories
end
GO

As the name suggests, the GetProductsByCategoryID stored procedure is used to return all the products that belong to a particular category.

CREATE procedure GetProductsByCategoryID
@lCategoryID int
as
begin
set nocount on
select * from Products
where CategoryID = @lCategoryID
end
GO

The GetProductByProductID procedure returns the details of a particular product based on the passed product id.

CREATE procedure GetProductsByCategoryID
@lCategoryID int
as
begin
set nocount on
select * from Products
where CategoryID = @lCategoryID
end
GO

Implementation of Phase 1 Architecture using Windows DNA

In this section, we will look at how the phase-1 architecture of the NorthwindApp is implemented using Windows DNA methodology.

As you can see from the above screenshot, the NorthwindWebApp uses Windows DNA based N-Tier architecture. The architecture of the web application is split into several layers called Tiers. By calling them as Tiers, we emphasize logical distribution rather than the physical distribution.

The Phase 1 architecture of the NorthwindWebApp consists of the following layers:

  • Presentation layer:

This layer consists of ASP Pages that provide the presentation for the entire application.

  • Business Logic layer:

This layer consists of COM objects written in VB and running in Component Services in Windows 2000. All the Business logic is implemented in this layer.

  • Data Services layer:

This layer consists of ADO API Code to invoke the Stored procedures in SQL Server 2000. Again this layer is encapsulated in a set of VB COM objects that are designed to run in component services.

  • SQL Server Database:

All the stored procedures return data to the calling Data services layer, and this layer is completely retained even after migrating the entire application to the .NET architecture.

Now that we have an understanding of the different layers of the application, let us start our implementation by looking at the data access layer.

Implementation of Data Access Tier

As we already said, this layer consists of VB code accessing ADO objects that invoke the Sql server stored procedures and returns the values back to the caller. We will create a new VB6 ActiveX dll named Northwind_DB and rename the default class from class1 to clsCategory.

clsCategory class

We change the MTSTransactionMode property to UsesTransaction. By setting this to UsesTransaction, we can ensure that the callers of this object can make this object become part of the same transaction, if they are running in one. Add the following lines of code to the clsCategory class:

Option Explicit
Implements IObjectConstruct
Private mstrConnectionString As String

Public Function GetCategories() As ADODB.Recordset

On Error GoTo Err_GetCategories

Set GetCategories = RunSPReturnRS(mstrConnectionString, _
"GetCategories")

Exit_GetCategories:
GetObjectContext.SetComplete
Exit Function
Err_GetCategories:
GetObjectContext.SetAbort
Err.Raise Err.Number, Err.Source & ".GetCategories", Err.Description
End Function

Private Sub IObjectConstruct_Construct(ByVal pCtorObj As Object)
'Assign the Connection String to the module level variable
mstrConnectionString = pCtorObj.ConstructString
End Sub

Option Explicit

We start by implementing the IObjectConstruct interface. By implementing the IObjectContruct interface, we can pass parameters (in this case, we are passing the connection string) to the clsCategory class from the Component Services Explorer. As you can see in the following screenshot, we store the connection string to the database in the Component Services Explorer:

Since we have implemented the Construct method of the IObjectConstruct interface, when any one of the methods in the clsCategory class is called, the COM+ runtime automatically invokes the Construct method and passes the connection string as an argument to it:

Implements IObjectConstruct
Private mstrConnectionString As String

Public Function GetCategories() As ADODB.Recordset

On Error GoTo Err_GetCategories

In this method, we invoke the utility method named RunSPReturnRS to execute the stored procedure. We will look at the code of the RunSPReturnRS method in a moment.

Set GetCategories = RunSPReturnRS(mstrConnectionString, _
"GetCategories")

Exit_GetCategories:

If the method is executed successfully, we then invoke the SetComplete method of the ObjectContext interface to not only commit the transaction, but also to deactivate the object.

GetObjectContext.SetComplete
Exit Function
Err_GetCategories:

If an error occurs during the execution of the above lines of code, we invoke SetAbort to rollback the transaction.

GetObjectContext.SetAbort
Err.Raise Err.Number, Err.Source & ".GetCategories", Err.Description
End Function

In the Construct method, we take in the connection string and store it in a module level variable. This module level variable (that holds the connection string) will be used by the actual methods to establish a connection with the database.

Private Sub IObjectConstruct_Construct(ByVal pCtorObj As Object)
'Assign the Connection String to the module level variable
mstrConnectionString = pCtorObj.ConstructString
End Sub

clsProduct Class

This class is very similar to the clsCategory class, except that the methods in this class are used to deal with the product related operations. In this section, we will only discuss the methods that are different from clsCategory class:

Option Explicit
Implements IObjectConstruct
Private mstrConnectionString As String

Public Function GetProductsByCategoryID(ByVal lngCategoryID As Long) As ADODB.Recordset

On Error GoTo Err_GetProductsByCategoryID

Set GetProductsByCategoryID = RunSPReturnRS(mstrConnectionString, _
"GetProductsByCategoryID", _
Array("@lCategoryID", adInteger, gc_intLEN_INTEGER, lngCategoryID))

Exit_GetProductsByCategoryID:
GetObjectContext.SetComplete
Exit Function
Err_GetProductsByCategoryID:
GetObjectContext.SetAbort
Err.Raise Err.Number, Err.Source & ".GetProductsByCategoryID", Err.Description
End Function

Public Function GetProductByProductID(ByVal lngProductID As Long) As ADODB.Recordset

On Error GoTo Err_GetProductByProductID

Set GetProductByProductID = RunSPReturnRS(mstrConnectionString, _
"GetProductByProductID", _
Array("@lProductID", adInteger, gc_intLEN_INTEGER, lngProductID))

Exit_GetProductByProductID:
GetObjectContext.SetComplete
Exit Function
Err_GetProductByProductID:
GetObjectContext.SetAbort
Err.Raise Err.Number, Err.Source & ".GetProductByProductID", Err.Description
End Function

Private Sub IObjectConstruct_Construct(ByVal pCtorObj As Object)
'Assign the Connection String to the module level variable
mstrConnectionString = pCtorObj.ConstructString
End Sub

Now that we have looked at the classes that are part of the data access layer, let us look at the code of the utility method RunSPReturnRS.

The RunSPReturnRS method

In this method, we have the plumbing code that is used to execute a stored procedure. This method internally calls another method named CollectParams to add parameters to the Command object that is used to execute the stored procedure.

Public Function RunSPReturnRS(ByVal strConnectionString As String, _
ByVal strSP As String, _
ParamArray a_vntParams() As Variant) As ADODB.Recordset
On Error GoTo ErrorHandler

' Create the ADO objects
Dim rs As ADODB.Recordset
Dim cmd As ADODB.Command
Set rs = CreateObject("ADODB.Recordset")
Set cmd = CreateObject("ADODB.Command")

' Init the ADO objects & the stored proc parameters
cmd.ActiveConnection = strConnectionString
cmd.CommandText = strSP
cmd.CommandType = adCmdStoredProc
CollectParams cmd, a_vntParams

' Execute the query for readonly
rs.CursorLocation = adUseClient
rs.Open cmd, , adOpenForwardOnly, adLockReadOnly

' Disconnect the recordset
Set cmd.ActiveConnection = Nothing
Set cmd = Nothing
Set rs.ActiveConnection = Nothing

' Return the resultant recordset
Set RunSPReturnRS = rs
Exit Function

ErrorHandler:
Set rs = Nothing
Set cmd = Nothing
Set RunSPReturnRS = Nothing
Err.Raise Err.Number, Err.Source & " .RunSPReturnRS ", Err.Description
End Function

Now that we have looked at the classes that are contained in the data access layer of the application, let us turn our attention to the Business services layer.

Implementation of Business logic Layer

In this section, we will create a new VB6 Active-X dll named Northwind_Bus. Once the project is created, we will rename the default class from class1 to clsCategory. Let us look at the implementation of the clsCategory class's methods.

clsCategory class

We will start by setting the MTSTransactionMode property of the class to Requires Transaction. By setting this value, the business object can call different methods present in the different objects (mostly data access layer objects) and still have them all part of the single transaction.

Add the following lines of code to the class:

Option Explicit

Public Function GetCategories() As ADODB.Recordset

On Error GoTo Err_GetCategories
Dim objCategory As Northwind_DB.clsCategory
Set objCategory = CreateObject("Northwind_DB.clsCategory")
Set GetCategories = objCategory.GetCategories()

Set objCategory = Nothing

Exit_GetCategories:
GetObjectContext.SetComplete
Exit Function
Err_GetCategories:
GetObjectContext.SetAbort
Err.Raise Err.Number, Err.Source & ".GetCategories", Err.Description
End Function

Let us walk through the above lines of code:

Public Function GetCategories() As ADODB.Recordset

On Error GoTo Err_GetCategories
Dim objCategory As Northwind_DB.clsCategory

In this line, we create an instance of the clsCategory class that is present in the data access layer:

Set objCategory = CreateObject("Northwind_DB.clsCategory")

Here, we invoke the GetCategories method of the clsCategory object and return the resulting recordset to the caller of this method:

Set GetCategories = objCategory.GetCategories()

Set objCategory = Nothing

Exit_GetCategories:

Like the other methods, here we also invoke the SetComplete method to control the object activation and transaction controlling behaviors:

GetObjectContext.SetComplete
Exit Function
Err_GetCategories:
GetObjectContext.SetAbort
Err.Raise Err.Number, Err.Source & ".GetCategories", Err.Description
End Function

clsProduct class

In this section, we will take a brief look at the methods contained in the clsProduct class. The methods in this class are similar to the ones contained in the clsCategory, except that they invoke a different set of methods in the data access layer:

Option Explicit

Public Function GetProductsByCategoryID(ByVal lngCategoryID As Long) As ADODB.Recordset

On Error GoTo Err_GetProductsByCategoryID
Dim objProduct As Northwind_DB.clsProduct
Set objProduct = CreateObject("Northwind_DB.clsProduct")
'Invoke the Data access layer method to get the list of products for the specified category id
Set GetProductsByCategoryID = objProduct.GetProductsByCategoryID(lngCategoryID)

Set objProduct = Nothing
Exit_GetProductsByCategoryID:
GetObjectContext.SetComplete
Exit Function
Err_GetProductsByCategoryID:
GetObjectContext.SetAbort
Err.Raise Err.Number, Err.Source & ".GetProductsByCategoryID", Err.Description
End Function

Public Function GetProductByProductID(ByVal lngProductID As Long) As ADODB.Recordset

On Error GoTo Err_GetProductByProductID
Dim objProduct As Northwind_DB.clsProduct
Set objProduct = CreateObject("Northwind_DB.clsProduct")
'Invoke the Data access layer method to get the product details of the specified product id
Set GetProductByProductID = objProduct.GetProductByProductID(lngProductID)

Set objProduct = Nothing
Exit_GetProductByProductID:
GetObjectContext.SetComplete
Exit Function
Err_GetProductByProductID:
GetObjectContext.SetAbort
Err.Raise Err.Number, Err.Source & ".GetProductByProductID", Err.Description
End Function

Implementation of Presentation Layer

This layer provides the presentation for the application. As you can expect, this layer depends on the business layer to perform all of its functionalities. In this phase-1, this layer consists of ASP pages that have server-side scripting code intermixed with HTML presentation. In this case study, we will concentrate on three ASP pages that perform the following operations:

  • Displaying the list of available categories (Categories.asp)
  • Displaying the list of products contained in a specific category (ProductListing.asp)
  • Displaying the details of a specific product (ProductDetails.asp)

We will start by looking at the Categories.asp page that allows us to view all the categories available in the system.

Categories List

As mentioned, this functionality is contained in the Categories.asp file, which looks like the following:

<%@ Language=VBScript %>
<% Option Explicit %>

In the server-side script, we create an instance of the business layer object and then invoke its GetCategories method to get the list of available categories in the form of a recordset.

<%
Dim objCategories
Dim rstCategories
Set objCategories = Server.CreateObject("Northwind_Bus.clsCategory")
Set rstCategories = objCategories.GetCategories()
Set objCategories = Nothing%>
<HTML>
<HEAD>
<TITLE> Northwind Web Application</TITLE>
<LINK rel='Stylesheet' type='text/css' href='Style.css'/>
</HEAD>
<body>
<table class='tableStyle' cellSpacing='0' cellPadding='0' width='600' border='0'>
<tr>
<td class='accentColor'><span class='largeBold'>Click the Category to get more details</span></td>
</tr>
<tr>
<td class='lightAccent'><IMG height='10' src='images/spacer.gif' width='1'/></td>
</tr>
<tr>
<td class='accentColor'><IMG height='1' src='images/spacer.gif' width='1'/></td>
</tr>
<tr>
<td class='largeColor' align='middle' vAlign='top'>List Of Categories<br></br>
<table height='311' cellSpacing='0' cellPadding='2' width='722' border='0'>
<tr>
<td width='631' colSpan='3' height='21'>
<hr class='hr'/>
</td>
</tr>

Now that we have the recordset that contains all the categories, it is very easy to loop through the contents and display them as part of the table:

<%Do While Not rstCategories.EOF%>
<tr>
<td class='largeColor' align='middle' width='631' colSpan='1' height='15'>
<p>
<a href="ProductListing.asp?CategoryID=<%=rstCategories("CategoryID")%>">
<%=rstCategories("CategoryName")%>
</a>
</p>
</td>
<td class='largeColor' width='631' height='15' colspan="2">
<%=rstCategories("Description")%>
</td>
</tr>
<tr>
<td colspan="3">
&nbsp;
</td>
</tr>
<%
rstCategories.MoveNext
Loop
rstCategories.Close()
Set rstCategories = Nothing
%>
</table>
</td>
</tr>
</table>
</body>
</HTML>

Products listing page

The ASP page that displays the listing of all the products that belong to a specific category is ProductListing.asp. Here is the code of the ProductListing.asp page:

<%@ Language=VBScript %>
<% Option Explicit %>

Similar to the Categories.asp page, this page also creates an instance of the business layer object (which is clsProduct in this case) and invokes its GetProductsByCategoryID method. The CategoryID that is passed as an argument to the GetProductsByCategoryID method is obtained from the query string. As a result of the method invocation, it gets a list of products in the form of recordset. This returned recordset is then looped through to display the list of products:

<%
Dim objProducts
Dim rstProducts
Dim lngCategoryID
lngCategoryID = CLng(Request.QueryString("CategoryID"))
Set objProducts = Server.CreateObject("Northwind_Bus.clsProduct")
Set rstProducts = objProducts.GetProductsByCategoryID(lngCategoryID)
Set objProducts = Nothing
%>

<HTML>

<HEAD>
<TITLE> Northwind Web Application</TITLE>
<LINK rel='Stylesheet' type='text/css' href='Style.css'/>

</HEAD>
<body>
<table class='tableStyle' cellSpacing='0' cellPadding='0' width='600' border='0'>
<tr>
<td class='accentColor'><span class='largeBold'>Click the Product to get more details</span></td>
</tr>
<tr>
<td class='lightAccent'><IMG height='10' src='images/spacer.gif' width='1'/></td>
</tr>
<tr>
<td class='accentColor'><IMG height='1' src='images/spacer.gif' width='1'/></td>
</tr>
<tr>
<td class='largeColor' align='middle' vAlign='top'>List Of Products<br></br>
<table height='311' cellSpacing='0' cellPadding='2' width='722' border='0'>
<tr>
<td width='631' colSpan='3' height='21'>
<hr class='hr'/>
</td>
</tr>

<%
Do While Not rstProducts.EOF
%>

<tr>
<td class='largeColor' align='middle' width='631' colSpan='1' height='15'>
<p>
<a href="ProductDetails.asp?ProductID=<%=rstProducts("ProductID")%>">
<%=rstProducts("ProductName")%>
</a>
</p>
</td>
<td class='largeColor' width='631' height='15' colspan="2">
<%=rstProducts("QuantityPerUnit")%>
</td>
</tr>
<tr>
<td colspan="3">
&nbsp;
</td>
</tr>

<%
rstProducts.MoveNext
Loop
rstProducts.Close()
Set rstProducts = Nothing
%>

</table>

</td>
</tr>
</table>

</body>
</HTML>

Product Details page

As the name suggests, this page allows us to display the details of a specific product:

<%@ Language=VBScript %>
<% Option Explicit %>
<%
Dim objProduct
Dim rstProduct
Dim lngProductID
'Get the ProductID from the querystring
lngProductID = CLng(Request.QueryString("ProductID"))
Set objProduct = Server.CreateObject("Northwind_Bus.clsProduct")
Set rstProduct = objProduct.GetProductByProductID(lngProductID)

Set objProduct = Nothing
%>

<HTML>
<HEAD>
<TITLE>Northwind Web Application</TITLE>
<LINK rel='Stylesheet' type='text/css' href='Style.css'/>
</HEAD>
<body>
<table class='tableStyle' cellSpacing='0' cellPadding='0' width='600' border='0'>
<tr>
<td class='accentColor'><span class='largeBold'>Product Details</span></td>

</tr>
<tr>
<td class='lightAccent'><IMG height='10' src='images/spacer.gif' width='1'/></td>
</tr>
<tr>
<td class='accentColor'><IMG height='1' src='images/spacer.gif' width='1'/></td>
</tr>
<tr>
<td class='largeColor' align='middle' vAlign='top'>Product Details<br></br>

<table height='311' cellSpacing='0' cellPadding='2' width='722' border='0'>
<tr>
<td width='631' colSpan='3' height='21'><hr class='hr'/></td>
</tr>

<tr>
<td class='largeColor' align='middle' width='631' colSpan='3' height='21'>
<p> ProductName : <%=rstProduct("ProductName") %></p>
</td>
</tr>

<tr>
<td class='largeColor' align='middle' width='631' colSpan='3' height='21'>
<p>Price : <%=rstProduct("UnitPrice") %> </p>

</td>
</tr>

<tr>
<td class='largeColor' align='middle' width='631' colSpan='3' height='21'>
<p> Description : <%=rstProduct("QuantityPerUnit") %></p>
</td>
</tr>

<tr>
<td class='largeColor' align='middle' width='631' colSpan='3' height='21'>
<p>Units In Stock : <%=rstProduct("UnitsInStock") %> </p>
</td>
</tr>
<tr>
<td class='largeColor' align='middle' width='631' colSpan='3' height='21'>
<p>Reorder Level : <%=rstProduct("ReorderLevel") %> </p> </td>
</tr>

<tr>
<td class='largeColor' align='middle' width='631' colSpan='3' height='21'>
<p>Units On Order : <%=rstProduct("UnitsOnOrder") %> </p>
</td>
</tr>

<tr>
<td align='right' width='208' height='24'></td>
<td align='left' width='345' height='24'></td>
</tr>

</table>
</td>
</tr>
</table>
</body>
</HTML>

Putting it all together

Let us see how the application looks like by navigating to the Categories.asp page. In this page, we display the list of all the categories available. The page that displays the list of categories looks like this:

When the user clicks on any one of the above categories, the user is then taken to the products listing page that looks like the screen shot below. We pass in the categoryid to this page, as part of the querystring:

Clicking on any of the products in the above screenshot brings us to the following screen in which details about the specified product is displayed:

Now that we have looked at the application and understood how the application works, let us move onto the next phase and see how this application can be migrated to the .NET platform.

Implementation of Phase 2 (Interim) Architecture using managed and unmanaged code

In this section, we will look at how the phase-2 architecture of the NorthwindApp is implemented using a combination of managed and unmanaged code.

As we already said, this interim architecture uses a combination of managed and unmanaged objects to accomplish the same set of requirements. The managed layer and unmanaged layers are represented using a solid block, and the managed layer calls into the unmanaged layer using the COM Interop services provided by Runtime Callable Wrapper (RCW).

As you can see from the above diagram, the NorthwindWebApp uses managed code (that is created using an ASP.NET web application) to provide the user interface for the application. We also have a new ASP.NET web services layer in the diagram. This layer basically wraps around the existing COM objects and provides a consistent managed interface to the ASP.NET web application. With this ASP.NET web services layer in place, exposing some of the business logic (as a service) to other applications only requires us to have the clients call into the application using the web services layer.

The business logic layer and the data access layer used in the Phase-1 architecture are still used in this phase.

The Phase-2 architecture of the NorthwindWebApp consists of the following tiers:

  • Presentation Tier:

This layer consists of ASP.NET web forms that provide presentation for the entire application.

  • ASP.NET web services Tier:

This layer is written using a ASP.NET web service (using C#) and basically it wraps around the existing COM objects and exposes the business logic to any kind of client application. In our case, the client (or consumer) of the web service layer is the ASP.NET web application. However, the client can be any type of application written using any of the languages such as VB6, Java, or VBScript (that are running on any platform) and so on. There are two important benefits to using this approach: firstly an abstraction of this kind allows us to change the underlying business logic layer without affecting the client; and secondly, the very nature of the web services concept makes it possible to expose this functionality to any type of client.

  • Business Logic Tier:

This layer consists of COM objects written in VB and running in Component Services in Windows 2000. All the Business logic is implemented in this layer. The consumer of this COM based business logic layer is the ASP.NET web service layer.

  • Data Services Tier:

This layer consists of ADO code to invoke the Stored procedures in SQL Server 2000. Again this layer is encapsulated in a set of VB COM objects that are designed to run in component services.

  • SQL Server Database:

All the stored procedures return data to the calling Data services tier and they remain the same as the previous phase.

Now that we have talked about the different pieces of the architecture, let us consider the managed portion of the application. Since the COM based business logic layer and the data access layer objects are the same as the previous phase, we will not talk about them. Let us start by considering the implementation of the ASPNET XML Web Service layer of the application.

Implementation of ASP.NET Web Service Layer

As explained before, this layer basically wraps around the existing COM based unmanaged objects and exposes them to the managed clients in a consistent fashion. By encapsulating the COM based business logic with a web service layer, we can easily expose the existing COM based business logic to the Internet clients without having to do any modifications to the code. It also allows us to complete the migration in a very seamless manner. This is possible due to the fact that when we migrate the existing COM based business logic to .NET based assemblies (which we perform in the next phase), we just need to change the web service layer to invoke the new .NET assemblies.

We will get started by creating a new ASP.NET Web Service project using Visual C#. Name the project as NorthwindWebSevice as shown in the following screenshot:

Once the project is created, change the default web service class name from Service1 to CategoriesService. Since we are trying to access the business layer objects, we need to reference those COM objects. To add a reference to a COM object, right click on the NorthwindWebService project from the Solution Explorer and then select Add Reference from the context menu. Next, navigate to the COM tab and select Northwind_Bus from the list of COM objects.

Clicking OK on the above dialog box creates a new COM interop assembly, places that assembly in the output directory, and adds a reference to the assembly. Now that we have added a reference to the business object, we can now import the COM object and start writing code against it.

We will get started by adding the following lines of using statements to the CategoriesService.asmx.cs file:

using System;
using System.Collections;
using System.ComponentModel;
using System.Data;
using System.Diagnostics;
using System.Web;
using System.Web.Services;
using Northwind_Bus;
using ADODB;
using System.Data.OleDb;

As you can see from the above list, we have also added a reference to the ADODB object library. This is primarily done due to the fact that the Northwind_Bus COM object returns a recordset as a result for most of the method invocations. Since we will be using the functionalities exposed by the OleDbDataAdapter class (that is part of the System.Data.OleDb namespace) to convert a recordset into a dataset, we add a reference to that namespace as well.

public class CategoriesService :
            System.Web.Services.WebService
{
[WebMethod]
public
            DataSet GetCategories()
{
clsCategoryClass category = new
            clsCategoryClass();
_Recordset rstCategories =
            category.GetCategories();
DataSet categoriesDataset = new
            DataSet();
OleDbDataAdapter adapter = new
            OleDbDataAdapter();
adapter.Fill(categoriesDataset,rstCategories,"Categories");
return
            categoriesDataset;
}
}
}

Let us walk through the above lines of code.

To start with, we inherit our class from the System.Web.Services.WebService class by adding the statement below. By inheriting our CatgoriesService class from the WebService class, we can easily get access to the ASP.NET intrinsic objects such as Application, Session and so on.

public class CategoriesService : System.Web.Services.WebService
{

The attribute [WebMethod] indicates that we want to expose the GetCategories method as a web service. When this web service is deployed, Visual Studio .NET runtime provides all the plumbing required to make this method discoverable and usable to the client.

[WebMethod]
public DataSet GetCategories()
{

Here we create an instance of the clsCategory class from the Northwind_Bus object and then invoke its GetCategories method to get the available categories in the form of an ADO recordset.

clsCategoryClass category = new clsCategoryClass();
_Recordset rstCategories = category.GetCategories();

In the following lines of code, we create instances of the Dataset and OleDbDataAdapater objects respectively:

      DataSet
            categoriesDataset = new DataSet();
OleDbDataAdapter adapter = new
            OleDbDataAdapter();

Here we convert the returned recordset into a dataset by invoking the Fill method of the OleDbDataAdapter object. We also name the table in the dataset as Categories by passing Categories as the third parameter.

adapter.Fill(categoriesDataset,rstCategories,"Categories");

Finally we return the dataset as the return value back to the caller:

      return
            categoriesDataset;
}
}

Now let us move onto the next web service class. We will name this class as ProductsService. As you might have guessed, this web service performs the task of creating an instance of the clsProduct class (of Northwind_Bus COM object) and invokes its methods to get the product related information.

We will add the following lines of code to the ProductsService.asmx.cs (code-behind file for the web service) file.

using System;
using
            System.Collections;
using System.ComponentModel;
using
            System.Data;
using System.Diagnostics;
using
            System.Web;
using System.Web.Services;
using
            System.Data.OleDb;
using Northwind_Bus;
using
            ADODB;

namespace NorthwindWebService
{
public class
            ProductsService :
            System.Web.Services.WebService
{

[WebMethod]
public
            DataSet GetProductsByCategoryID(int
            categoryID)
{
clsProductClass product = new clsProductClass();
_Recordset rstProducts =
            product.GetProductsByCategoryID(categoryID);
DataSet
            productsDataset = new DataSet();
OleDbDataAdapter adapter = new
            OleDbDataAdapter();
adapter.Fill(productsDataset,rstProducts,"Products");
return
            productsDataset;
}

[WebMethod]
public DataSet
            GetProductByProductID(int productID)
{
clsProductClass product
            = new clsProductClass();
_Recordset rstProduct =
            product.GetProductByProductID(productID);
DataSet productDataset
            = new DataSet();
OleDbDataAdapter adapter = new
            OleDbDataAdapter();
adapter.Fill(productDataset,rstProduct,"Product");
return
            productDataset;
}

}
}

Since the above set of statements is similar to the statements in the CategoriesService class, we will not look at them in detail. Now that we have created the web service classes, let us test the web services by using the test harness that is provided by Visual Studio .NET. To do this, build the web service project and then right click on the CategoriesService.asmx file from the solution explorer and select View in Browser. You will be presented with a test page that is similar to the following:

When you click the GetCategories method, you will see the following screen:

If you click Invoke in the above link, it internally invokes the GetCategories method and presents us with the returned value of the invocation as shown in the following screen:

Now that we have created the web service, let us move onto the ASP.NET web application client.

Implementation of ASP.NET Web Application

In this section, we will create a new ASP.NET web application named NorthwindWebApp as shown in the following screenshot:

As shown in the architecture diagram, the ASP.NET web application client depends on the web service layer for all of its operations. So, we will need to add a reference to the web service. We will do this by using the Add Web Reference option available in Visual Studio .NET. To add a web reference to a web service project, right click on the project in the Solution Explorer and select Add Web Reference from the context menu.

In the address bar, we will enter the path to the categories web service as shown:

If you click Add Reference in the above dialog box, Visual Studio .NET automatically creates a proxy class for the web service under the Web References folder in the solution explorer. We will rename the proxy class to CategorySvcProxy.

Now we will add a web reference to the products web service by using the following screen:

Once the proxy is created, we will rename the proxy class to ProductSvcProxy. After adding both of the above web references, the Solution Explorer looks like this:

Now that we have added references to all the dependencies, we will get started by looking at the Categories.aspx page that allows us to view all the categories available in the system.

Categories List

This functionality is contained in the Categories.aspx file, which looks like the following:

<%@ Page language="c#" Codebehind="Categories.aspx.cs"
AutoEventWireup="false" Inherits="NorthwindWebApp.WebForm1" %>
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" >
<HTML>
<HEAD>
<title>WebForm1</title>
<meta name="GENERATOR" Content="Microsoft Visual Studio 7.0">
<meta name="CODE_LANGUAGE" Content="C#">
<meta name="vs_defaultClientScript" content="JavaScript">
<meta name="vs_targetSchema" content="http://schemas.microsoft.com/intellisense/ie5">
<LINK rel='Stylesheet' type='text/css' href='Style.css'>
</HEAD>
<body MS_POSITIONING="GridLayout">
<form id="Form1" method="post" runat="server">
<table class='tableStyle' cellSpacing='0' cellPadding='0' width='600' border='0'>
<tr>
<td class='accentColor'><span class='largeBold'>Click the Category to get more details</span></td>
</tr>
<tr>
<td class='lightAccent'><IMG height='10' src='images/spacer.gif' width='1'></td>
</tr>
<tr>
<td class='accentColor'><IMG height='1' src='images/spacer.gif' width='1'></td>
</tr>
<tr>
<td class='largeColor' align='middle' vAlign='top'>List Of Categories<br>
<BR>
<table height='311' cellSpacing='0' cellPadding='2' width='722' border='0'>
<tr>
<td width='631' colSpan='3' height='21'>
<hr class='hr'>
</td>
</tr>
<DIV style="Z-INDEX: 101; LEFT: 15px; WIDTH: 722px; POSITION: absolute; TOP: 108px; HEIGHT: 50px"
ms_positioning="FlowLayout">
<asp:Literal id="placeHolderLiteral" runat="server"></asp:Literal></DIV>
<DIV></DIV>
</table>
</td>
</tr>
</table>
</form>
</body>
</HTML>

We add the following lines of code to the code-behind file (Categories.aspx.cs):

using System;
using
            System.Collections;
using System.ComponentModel;
using
            System.Data;
using System.Drawing;
using System.Web;
using
            System.Web.SessionState;
using System.Web.UI;
using
            System.Web.UI.WebControls;
using
            System.Web.UI.HtmlControls;
using System.Text;

We have imported the category web service proxy class namespace by using the following line:

using
            NorthwindWebApp.CategorySvcProxy;

private void
            Page_Load(object sender, System.EventArgs e)
{
if (!
            Page.IsPostBack)
{
CategoriesService categoriesSvc = new
            CategoriesService();
DataSet categoriesDataset = new
            DataSet();
//Call the GetCategories method to return the
            dataset
categoriesDataset = categoriesSvc.GetCategories();
DataTable categoriesTable =
            categoriesDataset.Tables["Categories"];
StringBuilder builder =
            new StringBuilder();
foreach(DataRow row in
            categoriesTable.Rows)
{
builder.Append ("<tr><td
            class='largeColor' align='middle' width='631' colSpan='1'
            height='15'>
<p>" + "<a
            href='ProductListing.aspx?CategoryID=" +
            row["CategoryID"].ToString()
+ "'>" +
            row["CategoryName"].ToString() + "
            </a>"+
"</p></td><td class='largeColor'
            width='631' height='15' colspan='2'>" +
            row["Description"].ToString() +
"</td></tr><tr><td
            colspan='3'>&nbsp;</td></tr>");

}
//
            Put user code to initialize the page here
placeHolderLiteral.Text
            = builder.ToString();
}
}

Let us understand the above statements by considering the important pieces of code. We start by checking if the page is posting back to itself by checking the IsPostBack property:

if (! Page.IsPostBack)
{

Here we create an instance of the category proxy class and then invoke its GetCategories method:

CategoriesService categoriesSvc = new CategoriesService();
DataSet categoriesDataset = new DataSet();

Here we create an instance of the category proxy class and then invoke its GetCategories method:

//Call the GetCategories method to return the dataset
categoriesDataset = categoriesSvc.GetCategories();

After we get the dataset, we then get the Categories table (that is part of the dataset) into an object of type DataTable:

DataTable categoriesTable = categoriesDataset.Tables["Categories"];
StringBuilder builder = new StringBuilder();

Once we have all the categories available in a datatable, we can then easily loop through them and append its contents to a StringBuilder object:

      foreach(DataRow
            row in
            categoriesTable.Rows)
{
builder.Append("<tr><td
            class='largeColor' align='middle' width='631' colSpan='1'
            height='15'>
<p>" + "<a
            href='ProductListing.aspx?CategoryID="
+
            row["CategoryID"].ToString() + "'>" +
            row["CategoryName"].ToString() + "
            </a>"+
"</p></td><td class='largeColor'
            width='631' height='15' colspan='2'>" +
            row["Description"].ToString() +
"</td></tr><tr><td
            colspan='3'>&nbsp;</td></tr>");
}

We finally display the contents of the StringBuilder object in a placeholder:

// Put user code to initialize the page here
placeHolderLiteral.Text = builder.ToString();
}

Products listing page

The ASP.NET web form that displays the listing of all the products that belong to a specific category is ProductListing.aspx. Here is the code listing for the ProductListing.aspx web form:

<%@ Page language="c#" Codebehind="ProductListing.aspx.cs"
AutoEventWireup="false" Inherits="NorthwindWebApp.ProductListing" %>
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" >
<HTML>
<HEAD>
<title>ProductListing</title>
<meta name="GENERATOR" Content="Microsoft Visual Studio 7.0">
<meta name="CODE_LANGUAGE" Content="C#">
<meta name="vs_defaultClientScript" content="JavaScript">
<meta name="vs_targetSchema" content="http://schemas.microsoft.com/intellisense/ie5">
<LINK rel='Stylesheet' type='text/css' href='Style.css'>
</HEAD>
<body MS_POSITIONING="GridLayout">
<form id="Form1" method="post" runat="server">
<table class='tableStyle' cellSpacing='0' cellPadding='0' width='600' border='0'>
<tr>
<td class='accentColor'><span class='largeBold'>Click the Product to get more details</span></td>
</tr>
<tr>
<td class='lightAccent'><IMG height='10' src='images/spacer.gif' width='1'></td>
</tr>
<tr>
<td class='accentColor'><IMG height='1' src='images/spacer.gif' width='1'></td>
</tr>
<tr>
<td class='largeColor' align='middle' vAlign='top'>List Of Products<br>
<BR>
<table height='311' cellSpacing='0' cellPadding='2' width='722' border='0'>
<tr>
<td width='631' colSpan='3' height='21'>
<hr class='hr'>
</td>
</tr>
<DIV style="Z-INDEX: 101; LEFT: 15px; WIDTH: 722px; POSITION: absolute; TOP: 108px; HEIGHT: 50px"
ms_positioning="FlowLayout">
<asp:Literal id="placeHolderLiteral" runat="server"></asp:Literal></DIV>
<DIV></DIV>
</table>
</td>
</tr>
</table>
</form>
</body>
</HTML>

Let us look at the code behind file for the ProductListing.aspx web form.

private void Page_Load(object sender, System.EventArgs e)
{
// Put user code to initialize the page here
if (! Page.IsPostBack)
{
//Get the CategoryID from the Querystring
int categoryID = Convert.ToInt32(Request.QueryString["CategoryID"].ToString());
ProductsService productsSvc = new ProductsService();
DataSet productsDataset = new DataSet();
//Call the GetProductsByCategoryID method to return the dataset
productsDataset = productsSvc.GetProductsByCategoryID(categoryID);
//Get the Products table contents in the form of Datatable
DataTable productsTable = productsDataset.Tables["Products"];
StringBuilder builder = new StringBuilder();
foreach(DataRow row in productsTable.Rows)
{
builder.Append("<tr><td class='largeColor' align='middle' width='631' colSpan='1' height='15'>
<p>" + "<a href='ProductDetails.aspx?ProductID=" + row["ProductID"].ToString()
+ "'>" + row["ProductName"].ToString() + " </a>"+
"</p></td><td class='largeColor' width='631' height='15' colspan='2'>" + row["QuantityPerUnit"].ToString() +
"</td></tr><tr><td colspan='3'>&nbsp;</td></tr>");
}
// Put user code to initialize the page here
placeHolderLiteral.Text = builder.ToString();
}
}

Like the Categories.aspx.cs page, this page also creates an instance of the web service object (which is Product in this case) and invokes its GetProductsByCategoryID method. The CategoryID that is passed as an argument to the GetProductsByCategoryID method is obtained from the query string. As a result of the method invocation, it gets a list of products in the form of dataset. This returned dataset is then looped through to display the list of products.

Product Details page

As the name suggests, this page allows us to display the details of a specific product. Let us look at the web form that provides this functionality:

<%@ Page language="c#" Codebehind="ProductDetails.aspx.cs" AutoEventWireup="false" Inherits="NorthwindWebApp.ProductDetails" %>
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" >
<HTML>
<HEAD>
<title>ProductDetails</title>
<meta name="GENERATOR" Content="Microsoft Visual Studio 7.0">
<meta name="CODE_LANGUAGE" Content="C#">
<meta name="vs_defaultClientScript" content="JavaScript">
<meta name="vs_targetSchema" content="http://schemas.microsoft.com/intellisense/ie5">
<LINK rel='Stylesheet' type='text/css' href='Style.css'>
</HEAD>
<body MS_POSITIONING="GridLayout">
<form id="ProductDetails" method="post" runat="server">
<table class='tableStyle' cellSpacing='0' cellPadding='0' width='600' border='0'>
<tr>
<td class='accentColor'><span class='largeBold'>Product Details</span></td>
</tr>
<tr>
<td class='lightAccent'><IMG height='10' src='images/spacer.gif' width='1'></td>
</tr>
<tr>
<td class='accentColor'><IMG height='1' src='images/spacer.gif' width='1'></td>
</tr>
<tr>
<td class='largeColor' align='middle' vAlign='top'>Product Details<br>
<BR>
<table height='311' cellSpacing='0' cellPadding='2' width='722' border='0'>
<tr>
<td width='631' colSpan='3' height='21'><hr class='hr'>
</td>
</tr>
<tr>
<td class='largeColor' align='middle' width='631' colSpan='3' height='21'>
<p>ProductName :
<asp:Label ID="lblProductName" Runat="server"></asp:Label>
</p>
</td>
</tr>
<tr>
<td class='largeColor' align='middle' width='631' colSpan='3' height='21'>
<p>Price :
<asp:Label ID="lblUnitPrice" Runat="server"></asp:Label></p>
</td>
</tr>
<tr>
<td class='largeColor' align='middle' width='631' colSpan='3' height='21'>
<p>Description :
<asp:Label ID="lblQuantityPerUnit" Runat="server"></asp:Label></p></td>
</tr>
<tr>
<td class='largeColor' align='middle' width='631' colSpan='3' height='21'>
<p>Units In Stock :
<asp:Label ID="lblUnitsInStock" Runat="server"></asp:Label></p>
</td>
</tr>
<tr>
<td class='largeColor' align='middle' width='631' colSpan='3' height='21'>
<p>Reorder Level :
<asp:Label ID="lblReorderLevel" Runat="server"></asp:Label></p>
</td>
</tr>
<tr>
<td class='largeColor' align='middle' width='631' colSpan='3' height='21'>
<p>Units On Order :
<asp:Label ID="lblUnitsOnOrder" Runat="server"></asp:Label>
</p>
</td>
</tr>
<tr>
<td align='right' width='208' height='24'></td>
<td align='left' width='345' height='24'></td>
</tr>
</table>
</td>
</tr>
</table>
</form>
</body>
</HTML>

Here is the code behind file for the ProductDetails web form:

private void Page_Load(object sender, System.EventArgs e)
{
// Put user code to initialize the page here
if (! Page.IsPostBack)
{
//Get the CategoryID from the Querystring
int productID = Convert.ToInt32(Request.QueryString["productID"].ToString());
ProductsService productsSvc = new ProductsService();
DataSet productsDataset = new DataSet();
//Call the GetProductsByCategoryID method to return the dataset
productsDataset = productsSvc.GetProductByProductID(productID);
//Get the Products table contents in the form of Datatable
DataTable productsTable = productsDataset.Tables["Product"];
//Check if the number of rows is greater than 0
lblProductName.Text = productsTable.Rows.Count.ToString();
if (productsTable.Rows.Count > 0)
{
DataRow row = productsTable.Rows[0];
//Set the values of the label controls
lblProductName.Text = row["ProductName"].ToString();
lblUnitPrice.Text = row["UnitPrice"].ToString();
lblQuantityPerUnit.Text = row["QuantityPerUnit"].ToString();
lblUnitsInStock.Text = row["UnitsInStock"].ToString();
lblReorderLevel.Text = row["ReorderLevel"].ToString();
lblUnitsOnOrder.Text = row["UnitsOnOrder"].ToString();
}

}
}

Now that we have completed the implementation of the entire ASP.NET web application, we will see what the application looks like by navigating to the Categories.aspx page:

Clicking on any one of the categories in the above page brings us to the following page in which we display the list of products present in that category:

If you click on any one of the products in the above list, you will get the following screen in which we display details about a specific product:

As you can see from the above set of pages, the functionality offered by the ASP.NET pages is exactly the same as that of the ASP application.

Implementation of Phase 3 Architecture using managed and unmanaged code

In this section, we will look at how the phase-3 architecture of the NorthwindApp is implemented.

As you can see from the above diagram, the above architecture is similar to the interim phase-2 architecture except that in this phase we have completely replaced the COM based unmanaged code layers (that includes the business logic layer and the data access layer) with the managed code written using C# class libraries. With this changes in place, we have migrated the entire application to the .NET architecture.

The Phase 3 architecture of the NorthwindWebApp consists of the following tiers:

  • Presentation Tier:

This layer consists of ASP.NET web forms that provide the presentation for the entire application.

  • ASP.NET web services Tier:

This layer is written using an ASP.NET web service (using C#) and basically it wraps around the managed business C# class library. In our case, the client (or consumer) of the web service layer is the ASP.NET web application. However, the client can be any type of application written using any of the languages such as VB6, Java, or VBScript client and so on.

  • Business Logic Tier:

This layer consists of C# Class Library objects, and these objects are designed to take advantage of the COM+ services (referred to as Enterprise services in .NET) provided by the System.EnterpriseServices namespace. Since this layer is also designed to run in Component Services, we can take advantage of the transaction processing capabilities and object pooling support provided by Component services.

All the Business logic is implemented in this layer. The consumer of this business logic layer is the ASP.NET web service layer.

  • Data Services Tier:

This layer consists of ADO.NET code to invoke the Stored procedures in SQL Server 2000. Again this layer is encapsulated using C# Class library objects that are designed to run in Component services.

  • SQL Server Database:

All the stored procedures return data to the calling Data services tier.

Let us start this phase of implementation by taking a look at the data access layer that is completely written in managed code.

Implementation of data access layer

As we already mentioned, we are going to implement this in managed code. Firstly, we will create a new Visual C# Class library project by selecting File-> New Project as shown in the following screenshot:

Once the project is created, we will rename the default class from class1 to Category.cs. Since we are going to use the EnterpriseServices provided by COM+, we need to add a reference to the System.EnterpriseServices assembly. Since we will be hosting the assembly in Enterprise services, we also need to sign our assembly with a key file that can be created using the sn.exe (Strong Name) utility.

To create a key file named Northwind_DB.snk using the Strong Name utility, we need to execute the following command from the command prompt.

sn -k Northwind_DB.snk

Once the file is created, add this key file to the Northwind_DB project by right clicking on the project from the Solution Explorer and selecting Add-> Add Existing Item from the context menu.

Now we need to add the following EnterpriseServices related settings to the AssemblyInfo.cs file.

The ApplicationName attribute indicates the name of the COM+ application that will be automatically created when a client application tries to execute this assembly for the first time. As you might have guessed, the Description attribute allows us to provide a brief description for the application:

[assembly: ApplicationName("Northwind_DB_App")]
[assembly: Description("Northwind Data Access Layer Application")]

The ApplicationActivation attribute indicates the type of activation (library or server activation) the application will be using:

[assembly: ApplicationActivation(ActivationOption.Library)]

In this line, we instruct the assembly to use the Northwind_DB.snk file as the key file:

[assembly: AssemblyKeyFile("..\\..\\Northwind_DB.snk")]

Now add the following statements to the Category.cs file:

using System;
using System.Data;
using
            System.Data.SqlClient;
using
            System.EnterpriseServices;

namespace
            Northwind_DB
{

[Transaction(TransactionOption.Supported)]
[ObjectPooling(true)]
[ConstructionEnabled(Default="server=localhost;uid=sa;pwd=;database=Northwind")]
public
            class Category : ServicedComponent
{
string
            connString;

protected override void Construct(string
            constructString)
{
//Assign the passed construct string to the
            module level variable
connString =
            constructString;
}

public DataSet
            GetCategories()
{
SqlConnection sqlConnection = new
            SqlConnection(connString);
try
{
DataSet categoriesDataset
            = new DataSet();
//Pass in the name of the stored procedure to be
            executed and the SqlConnection object as the argument
SqlDataAdapter adapter = new
            SqlDataAdapter("GetCategories",sqlConnection);
//Fill the Dataset
            with the return value from the stored
            procedure
adapter.Fill(categoriesDataset,"Categories");
//Call
            SetComplete to deactivate the object and commit the
            transaction
ContextUtil.SetComplete();
return
            categoriesDataset;
}
catch (Exception ex)
{
ContextUtil.SetAbort();
throw ex;
}
finally
{
if
            (sqlConnection.State ==
            ConnectionState.Open)
{
sqlConnection.Close();
}
}
}

}
}

Let us consider the above lines of code.

Here, we decorate the Category class with the Transaction attribute Supported, which indicates if the caller of this object runs in a Transaction, it will be inherited by this object, otherwise it will run without any transaction.

[Transaction(TransactionOption.Supported)]

The ObjectPooling attribute allows us to specify that we want to enable object pooling for the application:

[ObjectPooling(true)]

The ConstructionEnabled attribute allows us to specify a default construction string that will be passed to the object when the client invokes the methods of the object. This construction string is passed to the object during the creation of the object through the Construct method. It is important to note that all the attributes that are specified here in the code are automatically set in the component, when the component is registered with COM+. By using this approach, we can change global settings such as connection strings without having to change the application code.

[ConstructionEnabled(Default="server=localhost;uid=sa;pwd=;database=Northwind")]

In this line, we derive the Category class from the ServicedComponent, which allows the Category object to make use of the services provided by COM+ Enterprise services. Serviced Components are generally hosted in COM+ services as in-process servers (library applications).

public class Category : ServicedComponent
{

Here, we declare a module level variable named connString to store the connection string:

string connString;

As we already discussed, the Construct method is called during the creation of the object and this method is passed a string that is specified either programmatically or through the COM+ explorer. In the Construct method, we capture the connection string that is passed into a module level variable so that it can be used by the GetCategories method for establishing connection with the database:

protected override void Construct(string constructString)
{
//Assign the passed construct string to the module level variable
connString = constructString;
}

public DataSet GetCategories()
{

Now we create an instance of the SqlConnection object, passing the module level variable connString as an argument:

SqlConnection sqlConnection = new SqlConnection(connString);

We enclose the following statements in a try..catch block to catch any errors that may occur during the execution of those statements:

try
{

DataSet categoriesDataset = new DataSet();
//Pass in the name of the stored procedure to be executed and the SqlConnection object as the argument

Here we create an instance of the SqlDataAdapter object, passing to its constructor the name of the stored procedure to be executed, as well as the SqlConnection object:

SqlDataAdapter adapter = new SqlDataAdapter("GetCategories",sqlConnection);

Here we fill the dataset with the value that is obtained by executing the stored procedure.

//Fill the Dataset with the return value from the stored procedure
adapter.Fill(categoriesDataset,"Categories");

Using the ContextUtil.SetComplete method, we can specify that we want to commit all the operations performed by this object:

//Call SetComplete to deactivate the object and commit the transaction
ContextUtil.SetComplete();
return categoriesDataset;
}
catch (Exception ex)
{

Using the ContextUtil.SetAbort method, we can specify that we want to rollback all the operations performed by this object:

ContextUtil.SetAbort();
throw ex;
}

The statements in the finally block are always executed regardless of whether the statements in the try block are executed successfully or not. Here we inspect the state property of the SqlConnection object to check whether the connection is open. If the Connection is open, we close it by invoking the Close method:

         finally
{
if
            (sqlConnection.State ==
            ConnectionState.Open)
{
sqlConnection.Close();
}
}
}

In the next section, we will take a look at the Product.cs class that deals with all of the product related operations.

Product class

Let us add one more class, named Product.cs, to the project. After adding the class, add the following lines of code:

using System;
using
            System.EnterpriseServices;
using System.Data;
using
            System.Data.SqlClient;

namespace
            Northwind_DB
{
[Transaction(TransactionOption.Supported)]
[ObjectPooling(true)]
[ConstructionEnabled(Default="server=localhost;uid=sa;pwd=;database=Northwind")]
public
            class Product : ServicedComponent
{
string
            connString;

public DataSet GetProductsByCategoryID(int
            categoryID)
{
SqlConnection sqlConnection = new
            SqlConnection(connString);
try
{
DataSet productsDataset =
            new DataSet();
//Pass in the name of the stored procedure to be
            executed and the SqlConnection object as the argument
SqlDataAdapter adapter = new SqlDataAdapter();
SqlCommand
            command = new
            SqlCommand("GetProductsByCategoryID",sqlConnection);
//Set the
            SqlCommand object properties
command.CommandType =
            CommandType.StoredProcedure;

adapter.SelectCommand = command;

//Create the CategoryID parameter
SqlParameter
            paramCategory =
            command.Parameters.Add("@lCategoryID",SqlDbType.Int);
//Set the
            value of the CategoryID parameter
paramCategory.Value =
            categoryID;

//Fill the Dataset with the return value from the
            stored procedure
adapter.Fill(productsDataset,"Products"
            );
//Call SetComplete to deactivate the object and commit the
            transaction
//ContextUtil.SetComplete();
return
            productsDataset ;
}
catch (Exception ex)
{
//ContextUtil.SetAbort();
throw
            ex;
}
finally
{
if (sqlConnection.State ==
            ConnectionState.Open)
{
sqlConnection.Close();
}
}
}

protected
            override void Construct(string constructString)
{
//Assign the
            passed construct string to the module level variable
connString =
            constructString;
}

public DataSet
            GetProductByProductID(int productID)
{
SqlConnection
            sqlConnection = new
            SqlConnection(connString);
try
{
DataSet productsDataset =
            new DataSet("Product");
//Pass in the name of the stored
            procedure to be executed and the SqlConnection object as the
            argument
SqlDataAdapter adapter = new
            SqlDataAdapter();
SqlCommand command = new
            SqlCommand("GetProductByProductID",sqlConnection);
//Set the
            SqlCommand object properties
command.CommandType =
            CommandType.StoredProcedure;

adapter.SelectCommand = command;

//Create the CategoryID parameter
SqlParameter
            paramProduct=
            command.Parameters.Add("@lProductID",SqlDbType.Int);
//Set the
            value of the CategoryID parameter
paramProduct.Value =
            productID;

//Fill the Dataset with the return value from the
            stored procedure
adapter.Fill(productsDataset, "Product"
            );
//Call SetComplete to deactivate the object and commit the
            transaction
//ContextUtil.SetComplete();
return
            productsDataset ;
}
catch (Exception ex)
{
//ContextUtil.SetAbort();
throw
            ex;
}
finally
{
if (sqlConnection.State ==
            ConnectionState.Open)
{
sqlConnection.Close();
}
}
}
}
}

Since the Product class is similar to the Category class, we will not look at the implementation of the product class in detail. Now that we have created the data access layer, let us move onto the construction of the business layer of the application.

Implementation of business layer

We will start by creating a new Visual C# Class library project named Northwind_Bus by using File->New Project. Once the project is created, we will rename the default class from Class1 to Category.cs, and then we will create a new key file named Northwind_Bus.snk and add it to the project. Since the Northwind_Bus assembly depends on the data access layer (Northwind_DB) for all of its functionalities, we will need to add a reference to the Northwind_DB assembly.

Like the Northwind_DB project, we also add the following COM+ related settings to the AssemblyInfo.cs file:

[assembly: ApplicationName("Northwind_Bus_App")]
[assembly: ApplicationActivation(ActivationOption.Library)]
[assembly: Description("Northwind Business Layer Application")]
[assembly: AssemblyKeyFile("..\\..\\Northwind_Bus.snk")]

Now open up the Category.cs file and add the following lines of code to it:

using System;
using System.Data;
using System.EnterpriseServices;

namespace Northwind_Bus
{

Here, we decorate the Category class with the Transaction attribute Required, which indicates that if the caller of this object runs in a Transaction, it will be inherited by this object, otherwise it will create a new Transaction for itself and run within the scope of that transaction:

[Transaction(TransactionOption.Required)]
[ObjectPooling(true)]
public class Category : ServicedComponent
{
public DataSet GetCategories()
{
try
{
DataSet categoriesDataset;

Here, we create an instance of the data access layer object and then invoke its GetCategories method:

Northwind_DB.Category categoryObj = new
            Northwind_DB.Category();
categoriesDataset =
            categoryObj.GetCategories();
//Call SetComplete to deactivate the
            object and commit the
            transaction
ContextUtil.SetComplete();
return
            categoriesDataset;
}
catch (Exception ex)
{
ContextUtil.SetAbort();
throw ex;
}
}
}
}

Let us take a look at the implementation of the Product class.

Product class

To the Northwind_Bus, let us add a new class named Product.cs and add the following implementation to it:

using System;
using System.Data;
using
            System.EnterpriseServices;

namespace
            Northwind_Bus
{

[Transaction(TransactionOption.Required)]
[ObjectPooling(true)]
public
            class Product: ServicedComponent
{
public DataSet
            GetProductsByCategoryID(int categoryID)
{
try
{
DataSet
            productsDataset;
Northwind_DB.Product productObj = new
            Northwind_DB.Product();
productsDataset =
            productObj.GetProductsByCategoryID(categoryID);
//Call
            SetComplete to deactivate the object and commit the
            transaction
ContextUtil.SetComplete();
return productsDataset;
}
catch (Exception ex)
{
ContextUtil.SetAbort();
throw ex;
}
}

public
            DataSet GetProductByProductID(int productID)
{
try
{
DataSet productsDataset;
Northwind_DB.Product
            productObj = new Northwind_DB.Product();
productsDataset =
            productObj.GetProductByProductID(productID);
//Call SetComplete
            to deactivate the object and commit the
            transaction
ContextUtil.SetComplete();
return productsDataset;
}
catch (Exception ex)
{
ContextUtil.SetAbort();
throw ex;
}
}

Now that we have completed the business logic layer construction, let us move on to the ASP.NET web service layer.

Implementation of ASP.NET Web Service layer

We will start by creating a new ASP.NET Web Service project named NorthwindWebServiceNew. Once the project is created, we will change the name of the default web service name from Service1.asmx to CategoriesService.asmx.

In this phase, we will have the ASP.NET web service layer invoke the managed Northwind_Bus assembly to perform its operations. To accomplish this, let us add a reference to the Northwind_Bus assembly by using the Add Reference option.

Now open up CategoriesService.asmx file and add the following code:

using System;
using
            System.Collections;
using System.ComponentModel;
using
            System.Data;
using System.Diagnostics;
using
            System.Web;
using System.Web.Services;
using
            Northwind_Bus;

namespace
            NorthwindWebServiceNew
{

public class CategoriesService :
            System.Web.Services.WebService
{

[WebMethod]
public
            DataSet GetCategories()
{
Category categoryObj = new
            Category();
DataSet categoriesDataset;
categoriesDataset =
            categoryObj.GetCategories();
return categoriesDataset;
}
}
}

As you can see from the above statements, it is straightforward. We simply create an instance of the business layer object, invoke its methods, and then return the result to the caller.

Let us look at the code that is required for the product web service class.

ProductsService web service class

We will add a new web service to the project and name it ProductsService. To that class, we add the following code:

using System;
using
            System.Collections;
using System.ComponentModel;
using
            System.Data;
using System.Diagnostics;
using
            System.Web;
using System.Web.Services;
using
            Northwind_Bus;

namespace
            NorthwindWebServiceNew
{
public class ProductsService :
            System.Web.Services.WebService
{

[WebMethod]
public
            DataSet GetProductsByCategoryID(int categoryID)
{
Product
            productObj = new Product();
DataSet
            productsDataset;
productsDataset =
            productObj.GetProductsByCategoryID(categoryID);
return
            productsDataset;
}

[WebMethod]
public DataSet
            GetProductByProductID(int productID)
{
Product productObj =
            new Product();
DataSet productsDataset;
productsDataset =
            productObj.GetProductByProductID(productID);
return
            productsDataset;
}

}
}

Implementation ASP.NET Web Application

The implementation for this layer will be the same as that of the phase 2, except that now we will need to reference the new web service named NorthwindWebServiceNew. For this demonstration, we will create a new ASP.NET web application project named NorthwindWebAppNew. Once the project is created, we will then add a reference to the new web service (NorthwindWebServiceNew) using the Add Web Reference option. Once the reference is added, the rest of the implementation is exactly the same as that of phase 2. You can download the support material for this article (including the code) from at the end of this case study.

Conclusion

In this case study, we have seen the following:

  • How to architect a Windows DNA application that can be easily migrated to the .NET platform.
  • How to add business value to the application by incorporating XML Web Services into the application without having to change the application completely.
  • How to migrate the application in a phased manner by migrating only the individual layers of the application.
  • How to use COM Interop to leverage the existing investments in COM based middle tier objects

Although the application we created was simple in functionality, it should provide a solid foundation for understanding how to migrate a Windows DNA application to the .NET platform in a phased manner that makes the entire migration process seamless.

Case Study Information
Author | Thiru Thangarathinam
Editor | Adam Ryland
Reviewers | David Schultz, Matthew MacDonald, Venkatraman

If you have any questions or comments about this case study, or suggestions for further pieces, please contact the technical editor. If you wish to talk about the case study with other subscribers, please visit our forum here.

Fast Track Books from Wrox Press
If you want a C# book that is compact, fast-paced but still provides the quality of information that you have come to expect from a Wrox book - you want Fast Track C#, newly available from Wrox Press.

This book will help you to quickly learn about the following:
  • Understand what C# is, and how it fits into the .NET Framework
  • A concise introduction to all the main features of the C# language, with Java and C++ comparisons noted
  • Using the .NET base classes
  • Writing and deploying Windows applications with C# and Visual Studio .NET
  • Creating and configuring .NET assemblies
  • Accessing databases with ADO.NET, and using XML in C#
  • Integrating your existing COM components with the .NET Framework
  • COM+ Services in the .NET Framework
  • Building web applications with ASP.NET pages
  • Creating web services in C#


Mail Tracking and Analysis System

Mail Tracking and Analysis System
http://www.csharptoday.com/content.asp?id=1845
Article Rating: 4.5
Published on June 10, 2002
Abstract
One of the easiest, fastest and most reliable ways of communication is through e-mail. Many people carry out business with the help of e-mail, as e-mail is one of the primary means of communication in eBusiness. In this case study, Vinod Kumar looks at a Mail Client software solution that uses XML Webservices to store mails which have been sent or received in a centralized database for future analysis. A mail analyzing solution is also discussed.
Case Study

The easiest, fastest and most reliable method of communication is via e-mail. Many people carry out business with the help of e-mail. This kind of business communication can be classified into two main categories:

  • Inter-office communication, which is the communication within an organization between various departments. In this situation, the organization uses its own mail server for communication, or can use an external one.
  • Communication with external clients, where there is little control over how the user sends\deals with mails.

In the latter scenario, the tracking of the mails sent and received is tedious and difficult for to the following reasons:

  • Each client may use a different mail server for the purpose of sending and receiving mails; it would be difficult or highly impossible to keep track of the mails.
  • Suppose if each of them have their own mail account information, it would be difficult to get all the information and perform analysis.

To solve this kind of problems, we develop mail client software, called WebMail, which stores the mailing information sent and received in a centralized database. This stored information is used for mail analysis, which is discussed in the later part of this case study.

System Requirement

Before continuing, please take note of the following system requirements for the purposes of this discussion, as well as for working with the sample code accompanying the case study:

  • Windows 2000 or XP with IIS
  • . Net framework
  • Visual Studio .NET RTM
  • SQL Server 7.0 or 2000

We can use any external mail server that supports SMTP and POP for sending and receiving the mails using this tool. This case study requires some familiarity with the Microsoft .Net Framework, C#, and XML. Familiarity with SMTP and POP protocols is not necessary, but would be helpful. For a good background on these topics, please refer to:

POP3 standards

RFC 1939

ftp://ftp.isi.edu/in-notes/rfc1939.txt?WROXEMPTOKEN=618658Zezi9SpDFSZgQps5Xv6u

RFC1730

ftp://ftp.isi.edu/in-notes/rfc1730.txt?WROXEMPTOKEN=618658Zezi9SpDFSZgQps5Xv6u

A POP3 Client and C#

http://www.csharptoday.com/content.asp?id=1681&WROXEMPTOKEN=618658Zezi9SpDFSZgQps5Xv6u

TcpClientClass

http://msdn.microsoft.com/library/en-us/cpref/html/frlrfSystemNetSocketsTcpClientClassTopic.asp?frame=true&WROXEMPTOKEN=618658Zezi9SpDFSZgQps5Xv6u

NetworkStreamClass

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/cpref/html/frlrfSystemNetSocketsNetworkStreamClassTopic.asp&WROXEMPTOKEN=618658Zezi9SpDFSZgQps5Xv6u

EncodingClass

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/cpref/html/frlrfsystemtextencodingclasstopic.asp&WROXEMPTOKEN=618658Zezi9SpDFSZgQps5Xv6u

The WebMail System

Now let’s see how the WebMail works. This software consists of two different applications, namely a windows (WebMailClient) application and a Web Service (WebMailWS). The Windows application is used to send and receive the mails using any mail server that supports SMTP and POP protocol. The WebMailClient application also helps to update the mail sent and received information to a remote database using the WebMailWS Web service after storing the information in the client place in an XML format. Since a remote database can be situated anywhere in the world, we use WebServer for storing the Mail Information in a heterogeneous network. Once the information is stored, it can be used for mail analysis. Since we are using XML to store a copy of the mail information in the client location before storing it in the remote location, a client can get access to the mail information without a connection to the remote database.

Each time a mail is sent using this software, a unique identity number is generated and is appended to the subject (the WebMailClient tool hides the Unique ID from the user) along with the reference MailID (if any) and department code. This unique number is used for further references to the mail. The ClientWebMail application receives and stores only the mails that have this unique ID.

Database Design and Structure

As we see from the architecture of the system, two data stores are used to store and provide information for our application. An XML data source is used to store the information locally in the client, whereas an SQL Server database is used to store the information in the remote location.

Mail Tracking and Analysis System’s Database design

The database structure consists of five tables. Here is a list of tables that are added to the database.

The AccessInfo table sets the Access privilege for a user (through EMailID) to access others email information. This stores the EmailID, which is mapped to the AccessID.

The Department table stores the Department code, along with the name.

The AddressBook table stores the EmailID and password, along with the department code and AccessID.

The Send Mail table stores sent Mail information like Mail ID, DeptCode, Reference ID, Sender Email ID, sent date, subject, and message.

The ReceivedMailInfo table stores the mail receivers information along with the mail’s received date and Mail Read status.

The diagram below shows the table relationship:

Business Processes

The scope of this case study is to show how to manage the mails sent and received between a organization, which includes storing the mail information in a centralized database. The case study is split into three parts for better understanding.

In the first part, we build the Mail component, which helps us send and receive mails. The SendMail class sends mails using SMTP standard, and the POP3Client Class helps to receive mails, which supports the POP protocol.

In the second part, we discuss the web service (WebMailWS) that is used to perform login authentication and update the remote database with the sent and received mail’s information.

In the third part we build a client WinForm application (WebMailClient), which act as an interface to send and receive mails and also helps in analyzing the mails.

Component Design

The Mail component solution has four components. One is used to send mail, and the other three are used to receive mail.

The SendMail Class

Let’s examine the SendMail class that is used for sending mails. This class contains a set of private variables that are used to hold mail information. Some of these variables are exposed in the form of properties, since it allows us to exercise a greater degree of control over the values that are assigned to the variables. The other variables like m_Date and m_MessageID are used to set the sent date of the mail and the unique mail Message ID. m_SMTP is set to the TCPClient class, which is present in the System.Net.Sockets namespace. The TcpClient is used to establish a connection with the remote host. Application-level protocols such as FTP and HTTP are built on this class, which in turn is built upon the Socket class to provide TCP services at a higher level of abstraction.

The diagram below shows all the members in SendMail class:

public class
            SendMail
{
protected string m_MessageID=null;
protected
            string m_SMTPServer=null;
protected string
            m_From=null;
protected string m_FromName=null;
protected
            string m_To=null;
protected string m_Subject=null;
protected
            string m_Body=null;
protected string m_Date=
            DateTime.Today.ToString ();
protected TcpClient m_SMTP = null;
public SendMail()
{
}
//property to get and set From
            EmailID
public string From
{
get
{
return
            m_From;
}
set
{
m_From = value;
}
}
//property
            to get and set From Name
public string
            FromName
{
get
{
return
            m_FromName;
}
set
{
m_FromName =
            value;

}
}
//property to get and set To
            EmailID's
public string To
{
get
{
return
            m_To;
}
set
{
m_To = value;

}
}
//property
            to get and set Subject of the Mail
public string
            Subject
{
get
{
return
            m_Subject;
}
set
{
m_Subject =
            value;

}
}
//property to get and set Body of the
            Mail
public string Body
{
get
{
return
            m_Body;
}
set
{
m_Body =
            value;

}
}
//property to get and set SMTP
            ServerName
public string SMTPServer
{
get
{
return
            m_SMTPServer;
}
set
{
m_SMTPServer =
            value;

}
}

The Send method is used to send mails to the remote mail server. This method establishes a connection to the remote server using the TCPClient class, which returns a NetworkStream object that is used to read and write data to the remote host. This method uses two other methods, namely IsValidReply and SendMessage, to read and write data to the remote server. The IsValidReply method is used to read and validate the data as commands sent from the remote server. SendMessage sends the message as a command to remote server:

public string Send()
{
try
{
m_SMTP= new TcpClient(m_SMTPServer,25);
NetworkStream strm =
            m_SMTP.GetStream();
string reply = '';
// Server must replay
            220 - Server OK
reply = IsValidReply(strm);
if(reply !=
            'OK')
{
return reply;
}
// Send Hello to
            server
SendMessage('EHLO ' + m_SMTPServer +
            '\r\n',strm);
reply = IsValidReply(strm);
if(reply !=
            'OK')
{
return reply;
}
// Send From Mail ID to the
            server
SendMessage('MAIL FROM: <' + m_From +
            '>\r\n',strm);
reply = IsValidReply(strm);
if(reply !=
            'OK')
{
return reply;
}
// sending TO mailID to
            server
//Regular expression is used to split the
//to emailid
            which are seperated by comma delimiter
Regex RgxMail=new
            Regex(',');
string[] strMailSplit =
            RgxMail.Split(m_To);
foreach(string t in strMailSplit)
{
SendMessage('RCPT TO: <' + t + '>\r\n',strm);
reply =
            IsValidReply(strm);
if(reply != 'OK')
{
return
            reply;
}
}
SendMessage('DATA \r\n',strm);
reply =
            IsValidReply(strm);
if(reply != 'OK')
{
return reply;
}

The code below is used to send the message body data to the remote server, and then disconnects. This message body is built using a function buildMessageBody.

// Sending Message Body
byte[] bMime
            =
            Encoding.Default.GetBytes(buildMessageBody());
strm.Write(bMime,0,bMime.Length);
//
            Notify End of Data
SendMessage('\r\n.\r\n',strm);
reply =
            IsValidReply(strm);
if(reply != 'OK')
{
return reply;
}
// disconnecting from the
            Server
SendMessage('QUIT\r\n',strm);
return
            'OK';
}
catch(Exception x)
{
return 'ERR ' +
            x.Message;
}
}

Note: '\r\n” should be appended to each command sent. This is because SMTP specification indicates that a CRLF pair terminates all commands. SendMessage has two arguments, Data (of type string) and Stream (of type NetworkStream). Stream is a reference type variable, and Data is a value type variable. The Data argument receives SMTP commands and returns values such as 250, 220, and 354 and so on (signifying if a particular command sent to the server is carried out correctly or has failed). Before writing to the Network stream, the string has to be converted to byte format. To achieve this conversion, we use the Encoding.ASCII property of the Encoding class (System.Text Namespace) and the GetBytes function, which returns bytes out of characters.

private void SendMessage(string data,NetworkStream stream)
{
byte[] byteData = Encoding.ASCII.GetBytes(data.ToCharArray());
stream.Write(byteData,0, byteData.Length);
}

The IsValidReply method is used to read and validate the data as commands sent from the remote server. This method uses stream reader to read the data. This data, which is in byte format, is converted to a string type using the Encoding.ASCII property of the Encoding class and the GetString function. A waiting mechanism is deployed, which waits for the remote server to reply within a specific time frame before throwing an error code as a return value.

private string
            IsValidReply(NetworkStream strm)
{
string serverReply =
            'ERR';
int counter = 0;
while(true)
{
//checking for
            dada avaliability
if(strm.DataAvailable)
{
//setting byte
            size
byte[] buffer = new byte[1024];
int bytes =
            strm.Read(buffer,0,1024);
serverReply =
            Encoding.ASCII.GetString(buffer,0,bytes);
serverReply.Trim();
serverReply
            =
            serverReply.Substring(0,serverReply.Length-2);
break;
}
//waiting
            for the server replay
Thread.Sleep(300);
counter++;
if(counter > 50)
{
break;
}
}
// 250 -
            Command was Sucessful, 354 - Start inputing Data, 220 - Server is
            ready
if(serverReply.Substring(0,3) == '250' ||
            serverReply.Substring(0,3) == '354' || serverReply.Substring(0,3) ==
            '220')
{
return 'OK';
}
else
{
return
            serverReply;
}
}

The buildMessageBody() method is used to build the mail information before sending it using the SendMessage method. This method uses the BuildBody and SplitTo functions to build the mail message:

private string buildMessageBody()
{
m_MessageID = Guid.NewGuid().ToString();
string
            MailBuilder = '';
MailBuilder += 'From: \'' + m_FromName + '\'
            <' + m_From + '>\n';
MailBuilder += 'To: ' + SplitTo() +
            '\n';
MailBuilder += 'Subject: ' + m_Subject +
            '\n';
MailBuilder += 'Date: ' + m_Date + '\n';
MailBuilder +=
            'MIME-Version: 1.0\n';
MailBuilder += 'Content-Type:
            multipart/mixed;\n';
MailBuilder +=
            '\tboundary=\'----=_NextPart_' + m_MessageID +
            '\'\n';
MailBuilder += BuildBody();
return
            MailBuilder;
}

The BuildBody function is used to build the body of the message, and the SplitTo function is used to split and format the receiver’s e-mail in POP standard, for sending data using the Regular Expression RegEx class (in the System.Text.RegularExpression namespace). A comma delimiter separates the EMailID.

private string BuildBody()
{
string retVal=””;
retVal += '------=_NextPart_' + m_MessageID
            + '\n';
retVal += '\tcharset=\'iso-8859-1\'\n';
retVal +=
            'Content-Transfer-Encoding: 8bit\n\n';
retVal += 'STARTBODY\n\n';
retVal += m_Body + '\n';
retVal += 'ENDBODY\n\n';
retVal
            += '\n';
return retVal;
}
private string SplitTo()
{
string retVal = '';
Regex Regex=new Regex(',');
string[]
            strSplitParts = Regex.Split(m_To);
foreach(string t in
            strSplitParts)
{
retVal += '<' + t + '>';
if(i <
            strSplitParts.Length)
{
retVal +=
            ',\n\t';
}
else
{
retVal += '\n';
}
}
return
            retVal;
}

Now let’s look in more detail at the three classes that are used for receiving mails from the mail server.

POP3Client Class

The POP3Client class is used to send and receive POP3 commands to/from the POP3 email server. Using the POP3Client class, the user can connect to the POP3 server, read messages, and so on.

The diagram below show all the members in the POP3Client class:

This class has a set of private variables that are used to hold POP server information. Let’s look at some of these variables in the form of properties. A variable to hold the POP3MessagesInfo class instance is also declared.

public class Pop3Client
{
private
            TcpClient m_pPop3_Client = null;
private string m_Pop3Server =
            '';
private string m_UserName = '';
private string m_Password
            = '';
private bool m_LoggedIn = false;
private
            Pop3MessagesInfo m_MessagesInfo = null;
public string Pop3Server
{
get{ return m_Pop3Server; }
}
public string
            UserName
{
get{ return m_UserName; }
}
public string
            Password
{
get{ return m_Password; }
}

The LogIn method is used to validate the user with his username and password after establishing the connection to the remote server using the TCPClient class by the POP3Client method. The LogIn Method uses two functions, IsValidReply and SendMessage, to read and write the data to the remote server.

public bool LogIn(string userName,string
            password)
{
////POP Command USER - Sending USER Name for Mail
            Server
///Authentication
byte[] reply = SendMessage ('USER '
            + userName + '\r\n');
// Check if userName is
            ok.
if(IsValidReply(reply)){
////POP Command PASS - Sending
            Password for Mail Server
///Authentication
reply =
            SendMessage ('PASS ' + password + '\r\n');
// Check if password
            is ok.
if(IsValidReply (reply)){
m_LoggedIn =
            true;
m_UserName = userName;
m_Password = password;
return
            true;
}
}
return false;
}

SendMessage has two arguments, Data (of type string) and Terminator. This method is used to send the data, and waits until it gets the reply from the server for the data sent:

private byte[] SendMessage (string
            data,string terminator)
{
byte[] retVal =
            null;
try
{
NetworkStream strm =
            m_pPop3_Client.GetStream();
if(data.Length > 0){
byte[]
            byte_data =
            Encoding.ASCII.GetBytes(data.ToCharArray());
strm.Write(byte_data,0,byte_data.Length);
}
MemoryStream memStrm = new MemoryStream();
// Create
            loop
int counter = 0; // For command time out.
while(true){
if(strm.DataAvailable){
byte[] receivedData =
            new byte[m_pPop3_Client.ReceiveBufferSize];
int size =
            strm.Read(receivedData,0,m_pPop3_Client.ReceiveBufferSize);
memStrm.Seek(0,
            SeekOrigin.End);
memStrm.Write(receivedData,0,size);
// Create
            buffer for Compare eg. '\r\n.\r\n'.
byte[] bufComp = new
            byte[10];
// Read last 10 symbols from result
            stream
memStrm.Position = memStrm.Length -
            7;
memStrm.Read(bufComp,0,10);
string strCompare =
            Encoding.ASCII.GetString(bufComp);
// Check if message ends with
            terminator
if(strCompare.IndexOf(terminator) > -1){
break;
}
counter = 0;
}
else
{
System.Threading.Thread.Sleep(100);
}
if(counter
            > 100){
// ('time
            out');
break;
}
counter++;
}
retVal =
            memStrm.ToArray();
memStrm.Close();
}
catch(Exception
            x)
{
//('erro:' + x.Message);
}
return
            retVal;
}

The IsValidReply method is similar to the IsValidReply method in the SendMail class. This method uses stream reader to read the data, and this data, which is in the byte format, is converted to a string type using the Encoding.ASCII class and the GetString function. After converting the data to string, the data is validated.

private bool IsValidReply(byte[] reply)
{
if(reply != null){
string str = Encoding.ASCII.GetString
            (reply);
str =
            str.TrimStart();
if(str.StartsWith('+OK')){
return
            true;
}
}
return false;
}

The getMessagesList method is a public method. It is used to get the list of mails from the remote mail server. This method invokes the POP3MessageInfo class, which in turn gets the list of mails as an arraylist.

public Pop3MessagesInfo getMessagesList
            ()
{
// If not loggedIn, throw
            exception.
if(!m_LoggedIn){
throw new Exception('You must
            login before using this system !');
}
//POP Command LIST -
            Will list the mail info
byte[] reply = SendMessage
            ('LIST\r\n','.\r\n');
m_MessagesInfo = new
            Pop3MessagesInfo(reply);
return m_MessagesInfo;
}

Pop3MessagesInfo Class

The POP3MessageInfo class is used to list the number of mails for a user in the mail server, along with the size of each mail in an ArrayList. This class gets all the messages data in byte format and converts the byte data into a string and then splits the messages using the Split function:

private ArrayList m_MsgList = null;
public Pop3MessagesInfo(byte[] replay)
{
m_MsgList = new
            ArrayList();
string str =
            System.Text.Encoding.ASCII.GetString(replay);
string[] lines =
            str.Split(new Char[]{'\n'});
string[] msgs =
            null;
if(lines.Length > 3){
msgs = new
            string[lines.Length-3];
for(int
            i=1;i<lines.Length-2;i++){
msgs[i-1] = lines[i];
}
foreach(string s in msgs){
//
            System.Windows.Forms.MessageBox.Show(s);
string[] msgInf =
            s.Split(new char[]{' '});
if(msgInf.Length > 1){
string
            msgNr = msgInf[0];
int size =
            Convert.ToInt32(msgInf[1]);
m_MsgList.Add(new
            Pop3Message(msgNr,size));
}
else{
string msgNr =
            msgInf[0];
m_MsgList.Add(new
            Pop3Message(msgNr,-1));
}
}
}
}

The GetMessage() method is used to retrieve the mail message from the server after receiving the mail number as an argument. This method sends the POP command 'RETR' along with the mail number, using the SendDataWaitReplay function to get the message.

public byte[] getMessage(string
            messageNo)
{
// If not loggedIn, throw
            exception.
if(!m_LoggedIn){
throw new Exception('LogIn before
            using this function !');
}
//POP Command RETR - Will Retrive
            the the mails
byte[] replay = SendDataWaitRelpay('RETR ' +
            messageNo + '\r\n','.\r\n');
// Check that message is
            complete
if(EndsWith(replay,'.\r\n')){
return
            replay;
}
else{
return null;
}
}

The LogOff Method sends the POP command “QUIT” to logout from the Mail Server, but only after checking the logged in status.

public void LogOff()
{
// If not
            loggedIn, throw exception.
if(!m_LoggedIn){
throw new
            Exception('You must logged in to use this function
            ');
}
////POP Command QUIT - Will Disconnect from the
            Server.
SendDataWaitRelpay('QUIT
            \r\n');
m_pPop3_Client.Close();
}

WebMailWS - WebService

Now we are in the second part of our case study. Here we build a WebService that is used for storing the mail sent and received information. This webservice is also used to provide other information like MailDetails, Address Book, and Department List.

The WebMail Web service exposes seven methods to Internet clients:

  • IsUserValid
  • SendMailInfo
  • ReceiveUpdateInfo
  • getMailDetails
  • getAddressBook
  • getDeptDetails
  • getUserNames

Let's see each of these methods in detail.

We declare a class WebMailService that extends the WebService class from the System.Web.Services namespace. This namespace consists of the classes that enable us to build and use the Web Service.

public class WebMailService : System.Web.Services.WebService
{
private string m_UserName=null;
private string m_Password =null;
private string connStr=ConfigurationSettings.AppSettings['connStr'];

The WebService class is the base class of the WebmailService class. There can be any number of public or private classes. However, if a class is to expose some or all of its methods as part of the web service, the class and the method should be declared public.

The Web Service Attribute [WebService()] is used to specify the name of the web service and is also used in describing and providing help pages for the WebService.

IsUserValid WebMethod

IsUserValid is one of the public methods in the class WebMailService. This method validates a remote client by taking the UserName and Password as arguments. This validation allows the remote client to access various other services in the WebMailService class. Here we use the SqlDataReader object to access the database, since SqlDataReader provides a means of reading a forward-only stream of rows from a database in a much faster way. In this web service method, SqlDataReader is built using the SqlCommand object by executing the SQL statement using the ExecuteReader method. The user authentication fails if the reader returns a null value.

public bool isUserValid (string
            UserName,string Password)
{
try
{
string
            strSQL='';
m_UserName
            =UserName;
m_Password=Password;
SqlConnection sqlConn=new
            SqlConnection(connStr);
strSQL+=' Select emailid,password from
            Addressbook ';
strSQL+= ' where emailid='' + m_UserName + '' and
            ';
strSQL += ' password='' + m_Password + ''';
SqlCommand
            sqlComm = new
            SqlCommand(strSQL,sqlConn);
sqlComm.Connection.Open();
SqlDataReader
            dbReader =sqlComm.ExecuteReader(CommandBehavior.CloseConnection
            );
if (dbReader.Read())
{
//valid user
return
            true;
}
else
{
// authentication failed
return
            false;
}
}
catch
{
return false;
}
}

The SendMailInfo web method is used to store the mail send information of the remote client in the database. This web method takes MailID, Receivers Email ID, Mail Body, Mail Subject, etc as arguments and updates the database once the user has been validated. This method uses SQLConnection, SQLCommand and SQLDataReader objects for database update.

public string SendMailInfo(string
            UserName,string Password,string MailId,
string RefId,string
            ReceiverEMails,string DeptCode, string subject,string
            Message)
{
try
{
string SQL=null;
StringBuilder
            strSQL= new StringBuilder ();
m_UserName =UserName;
m_Password
            =Password;
if(IsUserValid(m_UserName,m_Password))
{
strSQL.Append (
            'Insert into SendMail values ('' + MailId + ''');
strSQL.Append
            (' ,'' + RefId + '',' + DeptCode + ','' + UserName +
            ''');
strSQL.Append (' ,'' + DateTime.Today + '','' + subject +
            '','' + Message + '')');
SqlConnection sqlConn=new
            SqlConnection(connStr);
SqlCommand sqlComm = new
            SqlCommand();
sqlComm.CommandText=strSQL.ToString();
sqlComm.Connection
            = sqlConn;
sqlComm.Connection.Open
            ();
sqlComm.ExecuteNonQuery();
sqlComm.Connection.Close();
Regex
            RgxEmail=new Regex(',');
string[] RecEmails =
            RgxEmail.Split(ReceiverEMails);
SqlCommand sqlCommUpdate= new
            SqlCommand();
foreach(string RecEmail in
            RecEmails)
{
SQL='';
SQL='insert into ReceiveMail values
            ('' + MailId + '','' + RecEmail + '','N','' + DateTime.Today +
            '','N')';
sqlCommUpdate.CommandText=SQL;
sqlCommUpdate.Connection
            =
            sqlConn;
sqlCommUpdate.Connection.Open();
sqlCommUpdate.ExecuteNonQuery();
sqlCommUpdate.Connection.Close();
}
return
            'Mail Send Successfully' ;
}
else
{
return 'User Not
            Valid';
}
}
catch(Exception e)
{
return
            e.ToString();
}
}

ReceivedDBUpdate WebMethod

The ReceivedDBUpdate WebMethod is used to set the flag “Y” in the Received column of the ReceiveMail table. This WebMethod gets the UserName, Password (as type string) and Received Mail information of a remote client as a dataset. The UserName and Password are used for the remote client authentication, and the dataset, which holds the information about the mail received by the remote user, is used to update the Mails Received status for all the MailID's in the Dataset. This method is called when a remote client receives a new mail or a set of new mails. These mails are the ones sent by any of the other clients using the WebMailClient software.

public string ReceivedDBUpdate(string
            UserName,string Password,DataSet MailUpdate)
{
try
{
string strSQL= null;
m_UserName =UserName;
m_Password
            =Password;
SqlConnection sqlConn=new SqlConnection(connStr);
if(isUserValid(m_UserName,m_Password))
{
SqlCommand
            sqlCommUpdate= new SqlCommand();
DataSet dsMailUpdate= new
            DataSet();
dsMailUpdate = MailUpdate;
//gets no of rows in the
            dataset
for (int i=0; i < dsMailUpdate.Tables[0].Rows.Count;
            i++)
{
strSQL='';
strSQL +='Update ReceiveMail set
            Received='Y',MailRead='Y',';
strSQL += ' RecDate='' +
            DateTime.Today + ''';
strSQL += ' where REMail='' + UserName + ''
            and ';
//get the mail id from the daatset
strSQL += '
            MailID='' +
            dsMailUpdate.Tables[0].Rows[i].ItemArray.GetValue(0).ToString() +
            ''';
sqlCommUpdate.CommandText=strSQL.ToString();
sqlCommUpdate.Connection
            =
            sqlConn;
sqlCommUpdate.Connection.Open();
sqlCommUpdate.ExecuteNonQuery();
sqlCommUpdate.Connection.Close();
}
}
return
            'Updated';
}
catch(Exception e)
{
return
            e.ToString();
}
}

MailDetails WebMethod

The MailDetails WebMethod takes the UserName and Password as arguments and returns mail information as a dataset. This method calls the function MailInfo for returning the dataset. This method not only gets the mail information of logged-in users, but also mail information for the other users that the logged-in user has permission to view.

public DataSet MailDetails(string
            UserName,string Password)
{
DataSet ds=new
            DataSet('SendMail');
m_UserName =UserName;
m_Password
            =Password;
if(IsUserValid
            (m_UserName,m_Password))
{
return MailInfo();
}
else
{
return ds;
}
}

SendMailInfo Method

The SendMailInfo function is used to get all the information from the SendMail table along with its corresponding information from the ReceiveMail table. The code starts out by filling a dataset with the SendMail table. The SQL query populates only the user’s mail information, which can be accessed by the logged-in user. The dataset is again filled with the ReceiveMail table. The relationship between these two tables in the dataset is created using the DataRelation object. This relationship enables us to see what links information from one table to another. The essential elements of a DataRelation are the name of the relationship, the two tables being related, and the related columns in each table. Here we relate the MailID columns of the two tables and keep the relationship name as Receiver Details.

public DataSet SendMailInfo()
{
DataSet ds;
StringBuilder strSQL = new
            StringBuilder();
SqlConnection myConnection = new
            SqlConnection(connStr);
// Create a
            SqlDataAdapter.
SqlDataAdapter myAdapter = new
            SqlDataAdapter();
myAdapter.TableMappings.Add('Table',
            'SendMail');
myConnection.Open();
strSQL.Append ('select *
            from sendmail where sender in');
strSQL.Append (' (select emailid
            from accessinfo where accessid=');
strSQL.Append (' (select
            accessid from addressbook where ');
strSQL.Append (' emailid=''+
            m_UserName + ''))');
SqlCommand myCommand = new
            SqlCommand(strSQL.ToString(),myConnection);
myCommand.CommandType
            = CommandType.Text;
myAdapter.SelectCommand = myCommand;
ds
            = new DataSet('Mail');
myAdapter.Fill(ds);
// Create a second
            Adapter and Command.
SqlDataAdapter adpReceive = new
            SqlDataAdapter();
adpReceive.TableMappings.Add('Table',
            'ReceiveMail');
strSQL.Remove(0,strSQL.Length);
strSQL.Append
            ('select * from ReceiveMail where REmail in');
strSQL.Append ('
            (select emailid from accessinfo where accessid=');
strSQL.Append
            (' (select accessid from addressbook where ');
strSQL.Append ('
            emailid=''+ m_UserName + ''))');
SqlCommand cmdReceive = new
            SqlCommand(strSQL.ToString(),myConnection);
adpReceive.SelectCommand
            = cmdReceive;
adpReceive.Fill(ds);
myConnection.Close();
// Create DataRelation.
System.Data.DataRelation
            dr;
System.Data.DataColumn dc1;
System.Data.DataColumn dc2;
dc1 = ds.Tables['SendMail'].Columns['MailID'];
dc2 =
            ds.Tables['ReceiveMail'].Columns['MailID'];
//setting the
            relation
dr = new System.Data.DataRelation('Receiver Details',
            dc1, dc2);
// Add the relation to the
            DataSet
ds.Relations.Add(dr);
return ds;
}

As well as these four methods, we also have another three web methods, which are all basically used to get general information like DepartmentList, AddressBook and UserName.

WebMethod Name | Purpose
getAddressBook | This web method is used to get all the email ids from the address book.
getDeptDetails | This web method is used to get the available department name along with the department code.
getUserName | This web method is used to get the EmailID along with the department code and department name.

Below is the screen shot of the web page rendered by the .NET Framework:

WebMailClient

Now we move to the last part of this case study. Here we build a user interface, which helps to send and receive e-mails. This interface is also used to keep track of all the mails sent using this software.

WebMailClient has a single WinForm. This does four different activities. For each of these activities we have a separate panel control. Before going further, let’s take a look at the four different activities:

  • Login to the System
  • Sending Mails
  • Receiving Mails
  • Mail Analysis

The ClientWebMail class contains a set of private variables that are used to hold information throughout the application.

public class ClientWebMail :
            System.Windows.Forms.Form
{
private string m_UserName =
            '';
private string m_Password = '';
private string m_DeptCode
            = '';
private bool m_LoggedIn = false;
private string
            XMLFilePath= Application.StartupPath + @'\';
// to store
            Addressbook info
DataSet dsAddress=new DataSet();
// to store
            departmentlist info
DataSet dsDept= new DataSet();
// to store
            username with DeptCode
DataSet dsUserName= new DataSet();
//
            to store MailInfo for analysis
DataSet dsMailInfo= new
            DataSet();
// creating a instant of the
            webservice
WebMailService WebMailWS= new WebMailService
            ();

Login to the System

The remote web server has to authenticate the user before he can access the system.

A panelControl panelDefault is used as a container to get the UserName and Password for authentication.

cmdLogin_Click Event

The cmdLogin_Click event is triggered when the user clicks the cmdLogin button after entering the UserName and Password. This event calls the IsUserValid WebMethod for authenticating the user. Once the user is authenticated, the event sets the variables m_UserName and m_Password to the username and password respectively, and then checks for the existences of the AddressBook XML file of that user. The AddressBook XML file is the one that stores the EmailID of the users using this Software. If the XML file does not exist, then a new one is created and will store the AddressBook data using the WebMethod getAddressBook. The data in the XML file is used to populate the EmailID’s in the listbox lstAddress. After populating the lstAddress, the cmdLogin_Click event populates the dataGrid with the mail information from the XML file that stores the downloaded mail information. If the XML file does not exist, then it will create a new one using the WebMethod MailDetails, which returns all the mail information from the remote database. We can even synchronize the XML file with updated information from the remote database. The cmdLogin_Click event also populates the cboDept with department codes using another WebMethod, getDeptDetails. The getUserName WebMethod is called to get the Department Code. Since most of the information is gathered from the remote WebService, we need not connect to the remote WebService regularly.

private void cmdLogin_Click(object
            sender, System.EventArgs e)
{
// checking for the textbox -
            empty
if (txtLoginID.Text==''
            ||txtLoginPassword.Text=='')
{
MessageBox.Show ('Enter
            UserName and
            Password');
}
else
{
statusBar.Panels[1].Text
            ='Connecting to WebService for User Validation';
//calling the
            remote webmethod
if
            (WebMailWS.IsUserValid(txtLoginID.Text.Trim(),txtLoginPassword.Text.Trim()))
{
statusBar.Panels[1].Text = 'Login
            Succeded';
statusBar.Panels[0].Text = txtLoginID.Text.Trim() + '
            is Logged In.';
// assigning the value
m_LoggedIn
            =true;
m_UserName=txtLoginID.Text.Trim();
m_Password=txtLoginPassword.Text.Trim();
string
            fileXML= XMLFilePath + txtLoginID.Text.Trim() + '_AddressBook.xml';
statusBar.Panels[1].Text = 'Checking for Address Book XML
            file..';
// checking for addressbook XML exist
FileInfo fi =
            new FileInfo (fileXML);
if
            (!fi.Exists)
{
statusBar.Panels[1].Text = 'File Does Not
            exist';
statusBar.Panels[1].Text = 'Getting the AddressBook
            information from WebService';
// If file does not exist
            Addressbook info
// got using the Webservice
DataSet DsAddXML=
            WebMailWS.getAddressBook ();
XmlDataDocument XD=new
            XmlDataDocument(DsAddXML);
statusBar.Panels[1].Text = 'Creating
            the XML File..';
// saving the addressbook XML
            file
XD.Save(fileXML);
}
statusBar.Panels[1].Text =
            'Popualting the Address Book';
statusBar.Panels[1].Text
            ='Reading the Address Book XML File';
dsAddress.ReadXml(fileXML);
statusBar.Panels[1].Text ='Populating the Address Book In
            ListBox';
lstAddressBook.DataSource
            =dsAddress;
lstAddressBook.DisplayMember='EMails.EMailID';
lstAddressBook.ValueMember
            ='EMails.EMailID';
fileXML= XMLFilePath + txtLoginID.Text.Trim()
            + '_MailInfo.xml';
FileInfo fiWS = new FileInfo (fileXML);
if
            (!fiWS.Exists)
{
statusBar.Panels[1].Text = 'Connecting to the
            WebService..';
WebMailWS.Credentials
            =System.Net.CredentialCache.DefaultCredentials;
statusBar.Panels[1].Text = 'It may take some time to download
            the
            MailInfo..';
dsMailInfo.Merge(WebMailWS.MailDetails(m_UserName,m_Password));
statusBar.Panels[1].Text = 'Getting the Mail
            Information..';
dsMailInfo.WriteXml(fileXML,XmlWriteMode.WriteSchema
            );
statusBar.Panels[1].Text = 'Populating the Information in the
            Grid';
//populating the mail info in
            DataGrid
dataGrid1.SetDataBinding(dsMailInfo,'SendMail');
}
else
{
statusBar.Panels[1].Text
            = 'Getting the Mail Information from Local
            Database';
dsMailInfo.ReadXml(fileXML,XmlReadMode.ReadSchema
            );
dataGrid1.SetDataBinding(dsMailInfo,'SendMail');
}
if
            (cboDept.Items.Count==0)
{
dsDept=WebMailWS.getDeptDetails();
dsUserName=WebMailWS.getUserNames
            ();
DataViewManager dvmDept = new
            DataViewManager();
dvmDept.DataSet =
            dsDept;
dvmDept.DataViewSettings[0].Sort =
            'Deptcode';
dvmDept.DataViewSettings[0].RowFilter='Distinct
            Deptcode';
//populating the DeptCode in combo
            using
//GetDeptDetails WebMethod
cboDept.DataSource
            =dvmDept;
cboDept.DisplayMember='Department.DeptName';
cboDept.ValueMember
            ='Department.DeptCode';
DataViewManager dvmDeptCode = new
            DataViewManager();
dvmDeptCode.DataSet =dsUserName;
dvmDept.DataViewSettings[0].RowFilter= 'EmailID='' + m_UserName
            + ''';
// getting the DeptCode of the loggedin
            user.
m_DeptCode =
            dvmDept.DataViewSettings[0].Table.Rows[0].ItemArray.GetValue(0).ToString
            ();
}
statusBar.Panels[1].Text ='';
panelDefault.Visible
            =false;
panelAnalysis.Visible =true;
Form.ActiveForm.Text
            ='WebMailClient -
            Analysis';
}
else
{
statusBar.Panels[1].Text = 'User
            Validation Fails Enter Correct UserName and
            Password';
}
}
}

Sending Mails

The next activity that is performed using the ClientWebMail class is sending mails. The PanelControl panelWrite is used as a container for other controls to get the mail information.

cmdSendMail_Click Event

The cmdSendMail_Click event is triggered when the user clicks the cmdSendMail button after entering the mail information. An instance of the SendMail class is created for sending the mail. After sending the mail, the mail information is stored in the remote database using the WebMethod SendMailInfo. Finally, the mail information is also stored in the XML file using the XmlDataDocument.

private void cmdSendMail_Click(object
            sender, System.EventArgs e)
{
try
{
XmlDataDocument
            XD=new XmlDataDocument();
string XMLBuilder = '';
if
            (txtUserName.Text==''
            ||txtHost.Text=='')
{MessageBox.Show('Information as to be
            Filled');}
else
{
//sending mail to remote
            server
statusBar.Panels[1].Text = 'Preparing to Send
            Mail..';
SendMail SendMail=new
            SendMail();
SendMail.SMTPServer=txtHost.Text.Trim();
SendMail.FromName
            =txtUserName.Text.Trim();
SendMail.From=txtUserName.Text.Trim() +
            '@' + txtHost.Text.Trim() ;
SendMail.To
            =lstAddressBook.Text.Trim()
            ;
SendMail.Subject=txtMailID.Text.Trim() + '#' + m_DeptCode + '#'
            + txtRefID.Text.Trim() + '#'
+
            txtSubject.Text.Trim();
SendMail.Body=txtBody.Text.Trim();
statusBar.Panels[1].Text
            = 'Sending Mail..';
SendMail.Send();
statusBar.Panels[1].Text
            = 'Mail has been Sent';
statusBar.Panels[1].Text = 'Preparing to
            Update the Mail info in WebService';
//updating the mail
            information n remote
            db
WM.SendMailInfo(m_UserName,m_Password,txtMailID.Text.Trim(),txtRefID.Text.Trim(),
lstAddressBook.Text.Trim(),m_DeptCode,txtSubject.Text.Trim
            (),txtBody.Text.Trim());
statusBar.Panels[1].Text = 'Updated the
            WebServie..';
string fileXML= XMLFilePath +
            txtLoginID.Text.Trim() + '_SendMail.xml';
statusBar.Panels[1].Text = 'Checking for Address Book XML
            file..';
// checking for addressbook XML exist
FileInfo fi =
            new FileInfo (fileXML);
if (!fi.Exists)
{
XD.LoadXml('');
XD.Save(fileXML);
}
statusBar.Panels[1].Text =
            'Writing the information in Local XML file.';
//creating the XML
            Structure
XMLBuilder += '';
XMLBuilder += ''
            + txtMailID.Text.Trim() + '';
XMLBuilder += '' +
            txtRefID.Text.Trim() + '';
XMLBuilder += '' +
            m_DeptCode + '';
XMLBuilder += '';
Regex
            Regex=new Regex(',');
string[] parts =
            Regex.Split(lstAddressBook.Text.Trim());
foreach(string t in
            parts)
{
XMLBuilder += '' + t +
            '';
}
XMLBuilder += '';
XMLBuilder
            += '' + txtSubject.Text.Trim() +
            '';
XMLBuilder += '' + txtBody.Text.Trim() +
            '';
XMLBuilder += '';
XmlNode currNode;
//Writing to the XML file
XmlDataDocument XDWriteMail = new
            XmlDataDocument ();
XDWriteMail.Load(fileXML);
XmlDocumentFragment docFrag =
            XDWriteMail.CreateDocumentFragment();
docFrag.InnerXml=
            XMLBuilder;
currNode =
            XDWriteMail.DocumentElement;
currNode.InsertBefore(docFrag,
            currNode.FirstChild );
//save the output to a
            file
statusBar.Panels[1].Text = 'Saving the XML
            File..';
XDWriteMail.Save(fileXML);
txtMailID.Text ='MID' +
            Guid.NewGuid().ToString();
statusBar.Panels[1].Text = 'New mail
            Id is generated you can send new mail
            now..';
}
}
catch(Exception
            ex)
{
statusBar.Panels[1].Text
            =ex.ToString();
}
}

cmdUpdateAddress_Click Event

The panel control panelWrite contains a command button cmdUpdateAddress. On clicking this, the cmdUpdateAddress_Click event gets triggered. This event helps to update the EMailID listed in the listbox lstAddress (lstAddress gets populated once the user is authenticated), with the latest one available in the remote database using the getAddressBook web method. This event also gives an option to update the AddressBook XML file with the latest Email Address from the remote database.

private void
            cmdUpdateAddress_Click(object sender, System.EventArgs e)
{
//calling the WebMethod
dsAddress=
            WM.GetAddressBook();
string message = 'Do you want to update the
            Client AdressBook?';
string caption = 'Update Client AddressBook
            ';
MessageBoxButtons buttons =
            MessageBoxButtons.YesNo;
DialogResult result;
result =
            MessageBox.Show(this, message, caption,
            buttons,
MessageBoxIcon.Question,
            MessageBoxDefaultButton.Button1,
MessageBoxOptions.RightAlign);
if(result
            == DialogResult.Yes)
{
//saving the latest adddressbook in XML
            file
string fileXML= XMLFilePath + txtLoginID.Text.Trim() +
            '_AddressBook.xml';
XmlDataDocument XD=new
            XmlDataDocument(dsAddress);
XD.Save(fileXML);
}
//populating the lstAddressBook with latest
            AddressBook
lstAddressBook.BindingContext['dsAddress'].SuspendBinding();
lstAddressBook.DataSource
            =dsAddress;
lstAddressBook.DisplayMember='EMails.EMailID';
lstAddressBook.ValueMember
            ='EMails.EMailID';
lstAddressBook.Refresh();
}

Receive Mail

The next activity that can be done using the ClientWebMail class is to check for new mail. The panel control panelChecMail is used as a container for showing the mail received.

menuCheckMail_Click Event

The menuCheckMail_Click event helps to get the e-mail from the remote mail server, update the received information in the remote database, and also updates the local XML file with the received mail information.

The code below creates an instant of the POP3Client class and builds a DataTable dtMailBox to store the received mail information.

private void menuCheckMail_Click(object
            sender, System.EventArgs e)
{
if
            (m_LoggedIn)
{
Pop3Client clnt = new
            Pop3Client(m_UserName,110);
string MailID=null;
string
            RefID=null;
string Date=null;
string From=null;
string
            Dept=null;
string Subject=null;
string Body=null;
string
            fileXML= XMLFilePath + txtLoginID.Text.Trim() + '_MailReceived.xml';
Regex Regex =new Regex('#');
string[]
            subjectArray;
DataSet dsOffLineMailinfo =new
            DataSet('MailInfo');
DataSet dsMailinfo =new
            DataSet('MailInfo');
DataTable dtMailBox=new
            DataTable('MailBox');
DataRow
            newRow;
dtMailBox.Columns.Add('MailID');
dtMailBox.Columns.Add('RefID');
dtMailBox.Columns.Add('DeptCode');
dtMailBox.Columns.Add('Sender');
dtMailBox.Columns.Add('MailSize');
dtMailBox.Columns.Add('Date');
dtMailBox.Columns.Add('Subject');
dtMailBox.Columns.Add('Body');

Next we get all the mail messages from the mail server using the getMessageList and getMessage methods of the POP3Client class. The received messages are split into MailID, RefID, Subject, Body, etc, by the getString function and are stored in the DataTable dtMailBox.

if(clnt.LogIn(m_UserName ,m_Password))
{
// Get messages list from server
Pop3MessagesInfo
            msgsInfo = clnt.GetMessagesList();
// Loop all messages and show
            them
foreach(Pop3Message m in
            msgsInfo.MessageInfo)
{
byte[] mime =
            clnt.GetMessage(m.MessageNr);
string
            str=System.Text.Encoding.ASCII.GetString(mime);
From =getString
            (str,'From:','To:');
Subject=getString
            (str,'Subject:','SendDate:');
subjectArray =
            Regex.Split(Subject);
if (subjectArray.GetUpperBound(0)>=
            3)
{
MailID= (string)subjectArray.GetValue(0);
Dept=
            (string)subjectArray.GetValue(1);
RefID=
            (string)subjectArray.GetValue(2);
Subject =
            (string)subjectArray.GetValue(3);
}
else
{
continue;
}
Date=getString
            (str,'SendDate:','MIME-Version:');
Body=getString
            (str,'STARTBODY','ENDBODY');
if
            (isMailReceived(MailID.Trim()))
{
newRow=
            dtMailBox.NewRow();
newRow['MailID']=MailID.Trim();
newRow['RefID']=RefID.Trim();
newRow['DeptCode']=Dept.Trim();
newRow['Sender']=From.Trim();
newRow['MailSize']
            =
            m.Size.ToString();
newRow['Date']=Date.Trim();
newRow['Subject']=Subject.Trim();
newRow['Body']=Body.Trim();
dtMailBox.Rows.Add(newRow);
}
}
clnt.LogOff();
int
            intMessageCount=dtMailBox.Rows.Count ;

Next the newly-received mail stored in the datatable dtMailBox are used to populate the listview lstReceiveMail control. To find the new mails we compare the MailId that is gotten from the received mails with the MailID's already stored in the XML file using the function IsMailReceived. The remote database is also updated with the new mail information using the WebMethod ReceivedDBUpdate.

for (int i=0; i <
            dtMailBox.Rows.Count; i++)
{
lstReceiveMail.Items.Add(dtMailBox.Rows[i].ItemArray.GetValue(0).ToString(),0);
lstReceiveMail.Items[i].BackColor=System.Drawing.Color.LightGray
            ;
lstReceiveMail.Items[i].SubItems.Add(dtMailBox.Rows[i].ItemArray.GetValue(1).ToString());
lstReceiveMail.Items[i].SubItems.Add(dtMailBox.Rows[i].ItemArray.GetValue(2).ToString());
lstReceiveMail.Items[i].SubItems.Add(dtMailBox.Rows[i].ItemArray.GetValue(3).ToString());
lstReceiveMail.Items[i].SubItems.Add(dtMailBox.Rows[i].ItemArray.GetValue(4).ToString());
lstReceiveMail.Items[i].SubItems.Add(dtMailBox.Rows[i].ItemArray.GetValue(5).ToString());
lstReceiveMail.Items[i].SubItems.Add(dtMailBox.Rows[i].ItemArray.GetValue(6).ToString());
}
DataSet
            dsWS = new
            DataSet();
dsWS.Tables.Add(dtMailBox);
WebMailWS.ReceivedDBUpdate(m_UserName
            ,m_Password,dsWS)
public bool IsMailReceived(string MailID )
{
string fileXML= XMLFilePath + txtLoginID.Text.Trim() +
            '_MailReceived.xml';
FileInfo fi=new FileInfo(fileXML);
if
            (fi.Exists)
{
XmlDocument doc = new
            XmlDocument();
doc.Load(fileXML);
XmlNodeList elemList =
            doc.GetElementsByTagName('MailID');
for (int i=0; i <
            elemList.Count; i++)
{
if
            (MailID.Equals(elemList[i].InnerXml.Trim()))
{
return
            false;
}
}
return true;
}
else
{
return
            true;
}
}

Now we again populate the Listview with the mail information stored previously in the XML file and then update the new mail information in the XML file.

FileInfo fi=new FileInfo(fileXML);
if(fi.Exists)
{
dsOffLineMailinfo.ReadXml(fileXML);
}
if (dsOffLineMailinfo.Tables.Count > 0 )
{
for (int
            j=0; j < dsOffLineMailinfo.Tables[0].Rows.Count ;
            j++)
{
lstReceiveMail.Items.Add(dsOffLineMailinfo.Tables[0].Rows[j].ItemArray.GetValue(0).ToString(),0);
lstReceiveMail.Items[j].SubItems.Add(dsOffLineMailinfo.Tables[0].Rows[j].ItemArray.GetValue(1).ToString());
lstReceiveMail.Items[j].SubItems.Add(dsOffLineMailinfo.Tables[0].Rows[j].ItemArray.GetValue(2).ToString());
lstReceiveMail.Items[j].SubItems.Add(dsOffLineMailinfo.Tables[0].Rows[j].ItemArray.GetValue(3).ToString());
lstReceiveMail.Items[j].SubItems.Add(dsOffLineMailinfo.Tables[0].Rows[j].ItemArray.GetValue(4).ToString());
lstReceiveMail.Items[j].SubItems.Add(dsOffLineMailinfo.Tables[0].Rows[j].ItemArray.GetValue(5).ToString());
lstReceiveMail.Items[j].SubItems.Add(dsOffLineMailinfo.Tables[0].Rows[j].ItemArray.GetValue(6).ToString());
}
}
dsOffLineMailinfo.Merge(dtMailBox);
dsOffLineMailinfo.WriteXml(fileXML);

Mail Analysis

The final activity that is done using the ClientWebMail class is analyzing the mail received. The panel control panelAnalysis is used as a container for showing the mail information that is populated in a dataGrid by the time the user has logged in to the system. The mail information taken from the remote database using the web method GetMailDetails is stored in a dataset dsMailInfo. A search criterion is provided to refine the mail information with respect to department code, Username, and by date using the DataView searchDV object.

private void cmdSearch_Click(object
            sender, System.EventArgs e)
{
DataView searchDV = new
            DataView();
if
            (cboDept.Items.Count!=0)
{
searchDV=dsMailInfo.Tables['SendMail'].DefaultView
            ;
searchDV.RowFilter ='DeptCode = '' + cboDept.SelectedValue +
            ''';
if(!cboUName.Text.Equals (''))
{
searchDV.RowFilter
            ='Sender = '' + cboUName.SelectedValue +
            ''';
}
searchDV.RowFilter ='SendDate >= '' + dtpFrom.Value
            + '' and SendDate <='' + dtpTo.Value + ''';
dataGrid1.SetDataBinding( searchDV ,'');
}
}

Scope for Improving the WebMail

We have now developed a WebMail mail client. Even though most features have been provided, we can still improve both the design and functionality of the WebMail. We can:

  • Provide an option to delete the mails both in the Mail server as well in the remote database.
  • Provide an option to send an attachment with the mail could be provided.
  • Since the mail information can be stored in the local disk, more analyzing options could be provided.
  • We could provide an option to get part of the mail information from the remote server, thus avoided bulk download of mail information from remote server.
  • We could improve the interface of the WebMailClient by adding HTML support.
  • We could increase the security of the XML file stored in the local disk by encrypting the file.

Summary

In this case study we have tried to develop a real time application, which helps to track and analyze mail. We also have developed a send and receive mail component that can be easily integrated with the other application.

As we have seen, this case study has been divided into three parts for better understanding; the first part explains the process involved to build the send and receive mail component, the second part explains the process involved for login authentication and for updating the mail information in a remote database using a WebService, while the third part explains the interface used to send and receive mails and to do mail analysis.

Case Study Information
Author | Vinod Kumar
Editor | Adam Ryland
Reviewers | Andrew Krowczyk, Saurabh Nandu, Shefali Kulkarni

If you have any questions or comments about this case study, or suggestions for further pieces, please contact the technical editor. If you wish to talk about the case study with other subscribers, please visit our forum here.


Generating Design Patterns Using CodeDOM

Generating Design Patterns Using CodeDOM
http://www.csharptoday.com/content.asp?id=1785
Article Rating: 4
Published on July 22, 2002
Abstract
The Code Document Model (CodeDOM), which provides the ability to generate source code in a number of the managed languages, is a very powerful .NET facility and has attracted much interest. Within Visual Studio .NET, a complementary though unrelated feature is provided by the CodeModel object, which allows programmatic access to hand-written code. Together these two tools provide a platform from which it should be possible to develop a VS.NET add-in to streamline the production of common designs in code. This case study follows the development of such a tool, which attempts to automate the generation of a widespread design pattern in an existing C# or VB.NET project.
Case Study

The Code Document Model (CodeDOM), which provides the ability to generate source code in a number of the managed languages, is a very powerful .NET facility and has attracted much interest. Within Visual Studio .NET, a complementary though unrelated feature is provided by the CodeModel object, which allows programmatic access to hand-written code. Together these two tools provide a platform from which it should be possible to develop a VS.NET add-in to streamline the production of common designs in code. This case study follows the development of such a tool, which attempts to automate the generation of a widespread design pattern in an existing C# or VB.NET project.

Introduction

The .NET framework, together with the Visual Studio .NET development environment, offers numerous tools that can be employed to improve productivity, enhance quality or enforce standards. At present though, there are large areas of untapped potential here. The MFC and ATL Class Wizards used in C++ projects by earlier versions of Visual Studio arguably allowed the developer to see his emerging code in the context of an existing class hierarchy, and did a useful job of simplifying the implementation of method overrides and COM interfaces, generating a good deal of boilerplate code along the way.

All of this is still available for C++ projects under VS.NET, albeit via a slightly different interface, but similar features for C# projects are, in my opinion, not yet as rich as they could be. It's true that the Windows Forms designer - which itself uses CodeDOM - makes the development of the user interface much simpler, but how easy is it to add an override for a virtual method defined in a base class? Is it as easy to examine events and implement event handlers for classes you've defined in your own project? Is there any way we can simplify the coding of classes that are designed to work together so we can minimize the manual effort involved? In particular, for an OO language like C#, it would be useful to have the ability to generate code based on a selected design pattern.

Design Patterns

Anyone familiar with object-oriented development will have come across design patterns. More than likely, this will be via the landmark publication "Design Patterns: Elements of Reusable Object-Oriented Software" by Gamma, Helm, Johnson and Vlissides (the so-called "Gang of Four" or "GoF"). This book describes 23 of the most common design patterns - which capture expertise in the field of OO design - and presents these in a format which allows a developer to implement his own solutions based on a tried and tested design. The OO community is now awash with patterns and there are numerous web sites and newsgroups that debate these issues widely.

The purpose of this case study is not to delve deeper into the use of patterns as design aids. This has already been covered on C# Today (see - Introduction to Design Patterns), but, as patterns in this sense raise the abstraction level at which systems are designed, they do offer an ideal opportunity for a code-generating wizard to assist the developer in producing a correct implementation.

Now this may sound a really worthwhile task to attempt, but it should be mentioned that the idea of code generation from design patterns is not a new one, and it's not as simple and straightforward as one might first think. In addition, patterns are evolving continuously and, whilst in no way language or platform dependent, may be adapted to suit particular variants in the problem domain or the implementation platform - particularly the latter - as we'll see.

The .NET framework itself employs many of the design patterns described by the GoF as well as variations thereof and completely new patterns too. In fact it's a variation of the Observer Pattern that we'll be implementing in this case study. Microsoft has dubbed this the Event Pattern, and this really has a two-fold purpose. The first covers the design aspects - the way delegates, events and related methods interact to implement the classic Observer Pattern described by Gamma et al. The second covers the naming conventions that Microsoft recommends are adopted by all applications that employ events. This latter aspect is intended to assist developers in understanding code written by others but, as a pure naming convention, is not enforced by either the CLR or the C# compiler.

One of the advantages of providing an add-in to generate code for this pattern is that, in addition to ensuring that the pattern is implemented correctly, we can encourage the use of the recommended naming convention by generating the relevant member names automatically based on minimal input from the user.

System Requirements

This system requires Visual Studio .NET V1.0 or later, running on Windows 2000 or Windows XP. The case study assumes that the reader has a good understanding of .NET in general and C# in particular. An understanding of OO design would be beneficial, particularly the application of Design Patterns.

Defining The Problem

We're going to keep the problem as simple as possible while still producing a useful application. So, the add-in we're going to develop will specifically target just the Observer Pattern, following the Microsoft recommended naming conventions. A more fully featured add-in would probably allow selection from a number of common design patterns but the principles will be the same. The main requirements we need to try and fulfil are as follows:

  • The add-in should be sensitive to the language of the host project and generate C# or VB.NET code as appropriate.
  • The rules for implementing the Observer Pattern should be encapsulated within the add-in, with minimal prompts for additional information from the user.
  • The add-in should be able to recognise potential participant classes already defined in the project and allow these to be incorporated into the design.

The first point suggests CodeDOM as an obvious candidate to be involved in the solution. This allows us to generate coding constructs in a language-neutral fashion and then decide the actual language to be generated at the last minute. If it were not for the fact that we'll be using events then we could also have included JScript as an option. CodeDOM has been covered in a previous case study (see - Building a Forms Generator Using Code DOM), so if you haven't come across this feature before, I recommend you review this article before you continue further. While we're at it, if you are not familiar with the Microsoft variation of the Observer Pattern, then stop now and read Exploring the Observer Design Pattern on MSDN.

The second point is more of a design choice than a requirement. I mentioned before that patterns are evolving and being created all the time so it may make sense to adopt a more flexible (though perhaps more complex) solution, which could allow new patterns to be included as they appear - perhaps some form of template. This would require a method of formal specification for a pattern rather than the more descriptive versions contained in the GoF book, and I have to admit I'm not entirely sure how feasible this would be. Even if possible, it's certainly outside of the scope of this case study so we'll make a conscious decision to encapsulate all of our knowledge of the pattern within the logic of the add-in. At least this way we can exercise more control over the way the design is implemented.

The third point is crucial if we are to provide a tool that is really useful to a developer. The ability to recognise classes that the developer has hand-coded as being potential participants in a design not only speeds development, but also encourages the application of coding standards in areas over which we would otherwise have little or no control. In the case of the Observer Pattern, if a developer has already hand-coded a subject class and now wishes to produce a corresponding observer, the ability to do so with no more than a few mouse clicks would be very desirable. However, this implies that we need to be able to inspect and understand existing source code in a project. Certainly in a compiled assembly, we would be able to use Reflection to great advantage, and this has also been covered before on C# Today (see - C# to UML Reverse Engineering). However, for uncompiled (and possibly invalid or incomplete) source code, the only tool we have is the CodeModel object available from within a VS.NET add-in. As we'll see, this only goes part of the way to providing the solution we need.

Case Study Structure

First, to put our requirements in perspective, we'll look at the overall design of the sample application. Next we'll skip fairly quickly through the steps needed to create a VS.NET Add-In and also the detail of the UI design for the wizard. Then we'll go a little deeper, as the individual steps in the wizard are explained in depth, and finally we'll review what we've learned and see what limitations we have to deal with to move the application forward.

The case study is structured like this:

  • Solution Design
  • Sample Application
  • Creating an Add-In
  • UI Design
  • Wizard Navigation
  • Step 1 - Choose Language
  • Step 2 - Define Subject Class
  • Step 3 - Define Observable Properties
  • Step 4 - Define Observer Class
  • Generating Code
  • Generating the Subject Class
  • Generating the EventArgs Descendant Class
  • Generating the Observer Class
  • Sample Outputs
  • Limitations
  • Case Study Review

Solution Design

As I said before, the idea is to keep the user interface as simple as possible. The add-in will be launched from a menu option and will present a number of prompts in the typical wizard fashion allowing navigation back and forward through the various steps. To extend this to cater for more than one pattern, we would need an initial dialog that allowed the user to select the pattern, then a wizard-like series of prompts for each pattern - implemented by separate forms. In this example, we're only looking at the Observer Pattern, so we'll have no pattern selection phase and will design just one form for the Observer Pattern wizard. In doing so, the main things we need to know from the user are:

  • The definition of the subject class
  • The definition of the observer class

We'll also need some more general information, like what language should be generated and in which file the generated code should be stored. We will also require the user to specify a namespace for the generated classes, but we'll keep these more mundane options aside for a moment. Looking at the first of the items above, we want to have two ways to reach a definition of the subject class:

  • The ability to generate a new class from scratch
  • The ability to select an existing class in the project

Each of these options will be treated differently but should end with the same result - a list of events to which our observer class should have the ability to subscribe. We'll present this list to the user in the final wizard step to allow them to select just those events they're interested in.

So we'll break these prompts down into four steps. This will follow a logical progression as far as the user is concerned and also keep the amount of information required at each step as small as possible. I've rather arbitrarily chosen to have the user choose the required language as the first step in the process although this could just as logically be done in the last step. I've taken my cue from the Add-In Wizard, in which this is the very first question too. The four steps are as follows:

Step 1 - Choose the required implementation language

Step 2 - Choose an existing subject or create a new one

Step 3 - If creating a new subject, define some events therein

Step 4 - Define the observer and collect any remaining housekeeping info

Notice that the third step is only required if we are creating a subject class from scratch. To keep things tidy, we'll only display this prompt if we need to. This means we need to keep track of our position in the wizard carefully so that we can take the correct action when/if the user clicks the Back and Next buttons. I'll go into this in more detail later.

At the end of the fourth step we will have all of the information we need to generate some code. At this point, we will close the UI and return the user to their project with the additional code displayed in the editor.

Figure 1 shows this schematically in an activity diagram. The structure of the UI part of the add-in will be based around the five activities shown. For the first four of these (relating to each step in the wizard), there is a preparatory stage - where user input from the previous step(s) is evaluated - followed by the display for the next logical step.

Figure 1 - Activity Diagram

Sample Application

The development of Visual Studio .NET Add-Ins has already been covered in detail on this site (see - Visual Studio .NET Add-Ins) so I'll skip over this part quite quickly. Then I'll briefly show details of the UI design and the method of navigation for the wizard before describing each of the steps in more depth.

Creating an Add-In

Create a new project from VS .NET, selecting Other Projects / Extensibility Projects as the Project Type and using the Visual Studio .NET Add-In template. Name the project DesignPattern (Figure 2).

Figure 2 - Creating a new Add-in

On the first page of the add-in wizard, select C# as the programming language then click Next. On the next page, select just Visual Studio .NET as the Application Host, then enter a brief name and description on the third page as shown in Figure 3.

Figure 3 - Enter Add-in Description

On the fourth page, tick only the following check boxes:

  • Yes, create a 'Tools' menu item.
  • I would like my add-in to load when the host application starts
  • My add-in should be available to all users of the computer .

See Figure 4 for an example. This will make sure that the necessary code is generated to load the add-in when VS .NET starts and will place a new menu item on the 'Tools' menu. We'll tweak this generated code a little to make sure that the user can only launch the add-in when there is a project open.

Take the default options through the remaining wizard pages then build the project to make sure all is well.

Figure 4 - Choose Add-in Options

If the build was successful (it should be!) then press F5 to test the general operation of the add-in. When you press F5, a new instance of Visual Studio .NET will be launched. When this has loaded, there should be an additional option visible at the top of the Tools menu as shown in Figure 5. If all is ok, close the second instance of VS.NET and return to the development environment.

Figure 5 - the Tools menu

The source code generated by the add-in wizard will be in the file Connect.cs. This contains the class Connect which implements the interfaces expected of a VS.NET add-in.

Within this definition, there are two methods that we need to change. The first of these - QueryStatus() - is called by VS.NET automatically whenever the host application needs to know the status of the add-in. For example, this method is called when the user clicks on the Tools menu, just before the menu items are displayed. This is where we can choose to make the design pattern add-in unavailable if there is no current project. Doing this avoids the need for any special handling of this situation in the add-in itself - after all, it just doesn't make sense to invoke a code-generating wizard if we have no project in which to generate code!

The default implementation of QueryStatus() provided for us by the add-in wizard always makes the add-in visible and available by returning a combination of bit-flags from the vsCommandStatus enumeration. All we need to do is to alter the default behaviour depending on whether or not there is a project open. We can find this out via the DTE reference in applicationObject by looking at the number of projects contained in the array ActiveSolutionProjects.

The amended version of QueryStatus() looks like this:

public void QueryStatus(string commandName,
EnvDTE.vsCommandStatusTextWanted neededText,
ref EnvDTE.vsCommandStatus status, ref object commandText)
{
if(neededText == EnvDTE.vsCommandStatusTextWanted.vsCommandStatusTextWantedNone)
{
if(commandName == "DesignPattern.Connect.DesignPattern")
{
// enable addin if a project is open
Array projects = (Array)applicationObject.ActiveSolutionProjects;
if (projects.Length > 0)
status = (vsCommandStatus)vsCommandStatus.vsCommandStatusSupported|vsCommandStatus.vsCommandStatusEnabled;
else
status = (vsCommandStatus)vsCommandStatus.vsCommandStatusInvisible;
}
}
}

So if there are no projects in the active solution, the DesignPattern option on the Tools menu will not be selectable.

The next method we need to change in Connect.cs is Exec(). This is the method that gets called when the user selects the new menu item from the Tools menu. For our application, we want to display a Windows Form to prompt the user through the remainder of the process, so add a project reference to System.Windows.Forms.dll and add the following line to Connect.cs :

using System.Windows.Forms;

Then add a new Windows Form item to the project, naming it MainForm, and amend the Exec() method to create and display the form. Note that I have added an argument to the form's constructor so we can pass a reference to the DTE object from the add-in to allow the form class to access the VS.NET automation model. The Exec() method looks like:

public void Exec(string commandName, EnvDTE.vsCommandExecOption executeOption,
ref object varIn, ref object varOut, ref bool handled)
{
handled = false;
if(executeOption == EnvDTE.vsCommandExecOption.vsCommandExecOptionDoDefault)
{
if(commandName == "DesignPattern.Connect.DesignPattern")
{
MainForm mainform = new MainForm(applicationObject);
mainform.Show();
handled = true;
return;
}
}
}

This is all we need to do to get the add-in launched at the appropriate time. We can now turn our attention to the design of the MainForm class, which will perform all of the processing described in the outline solution earlier.

As mentioned above, the MainForm class needs to reference the VS.NET automation model and is passed a reference to this from the Connect class. So first we'll add a few imports for the namespaces we'll be working with:

using EnvDTE;
using VSLangProj;
using System.CodeDom;
using System.CodeDom.Compiler;
using Microsoft.CSharp;
using Microsoft.VisualBasic;

And some fields to hold references to the automation model and the current project:

private _DTE applicationObject;
private Project currentProject;

Then we can rewrite the MainForm constructor to populate these fields as follows:

public MainForm(_DTE application)
{
// Required for Windows Form Designer support
InitializeComponent();

// save dte
applicationObject = application;

// get reference to current project
Array projects = (Array)applicationObject.ActiveSolutionProjects;
currentProject = (Project)projects.GetValue(0);
}

We'll also add a constant, which tells us the maximum number of steps in the wizard. That way, we can tell when the user has reached the final prompt and we need to start doing something:

private const int MAX_STEPS = 4;

Now we can design the user interface for each step of the wizard.

UI Design

The user interface is designed to look like a conventional wizard. The form uses the FixedDialog border style and consists of three panels. The top panel shows a heading and sub-heading describing the current step, the middle panel contains the UI elements required for each step (there are four such panels - one for each step), and the bottom panel contains the navigation buttons.

I won't labour the details of form design as this should be straightforward. The following screenshots show the appearance of the form for each of the steps, together with the names of any controls used.

The first step requests the programming language to be used when generating code. Only two choices are available, which are indicated by radio buttons (see Figure 6).

Name | Type | Description
panelstep1 | Panel | User interface for step 1 - containing.
langCSharp | RadioButton | Selects C#
langVB | RadioButton | Select VB.NET

The top panel contains two labels and an icon in a PictureBox. The text of the labels is changed as each step is displayed.

Name | Type | Description
toppanel | Panel | Title panel for all steps - containing.
heading | Label | Displays title for the current step
subheading | Label | Displays a prompt for the current step
pictureBox1 | PictureBox | Displays a static icon

Figure 6 - Step 1 (Choose Language)

The bottom panel contains the navigation buttons. These control movement through the wizard and are used to show the relevant panel for each step. There is also a cancel button to allow the user to close the form at any time.

Name | Type | Description
bottompanel | Panel | Navigation panel for all steps - containing.
cancelbutton | Button | Closes the form
backbutton | Button | Navigates to the previous step
nextbutton | Button | Navigates to the next step

The second step is where the user opts to create a new subject class or to select an existing one (see Figure 7).

Name | Type | Description
panelstep2 | Panel | User interface for step 2 - containing.
createsubject | RadioButton | Indicates a new subject class is required
existingsubject | RadioButton | Indicates an existing subject class will be used
txtsubject | TextBox | The name for the new subject class
subjectcombo | ComboBox | A list of existing classes in the project

Before this step is displayed, the combo box needs to be populated with a list of classes already defined in the project. This is described in the detailed processing later.

Figure 7 - Step 2 (Define Subject Class)

As with all steps, the UI controls are enabled and disabled in a logical manner to make sure that the user is guided correctly through the wizard. For example, in the above step, if the option is selected to create a new subject class, the Next button is disabled until a name has been entered in the corresponding text box. This avoids unnecessary validation in the wizard.

The third step is the optional one. It only needs to be displayed if the user has opted to create a new subject class. It is changes in the internal state of the subject that trigger the events to which any observers subscribe. In this application, we will represent the internal state of the created subject class using properties. We can simplify the user interface for collecting this information if we assume that all properties will be implemented as objects (i.e. of type System.Object), then we just provide a way for the user to specify one or more property names.

This is handled by the user interface for step three of the wizard, shown in Figure 8.

Name | Type | Description
panelstep3 | Panel | User interface for step 3 - containing.
txtproperty | TextBox | Name of property to be added to the list
addproperty | Button | Adds a property to the list
lstproperties | ListBox | The list of properties to be created

The fourth and final step is to collect the information we need to generate the observer class. In this sample application, this class will always be generated. To make this a really useful wizard it would be nice to have the ability to add the observing functionality to an existing class but this throws up a whole heap of complications. We'll discuss these later as there's more to this than meets the eye, but for now we'll follow the same premise as before and keep things as simple as possible.

So, by the time we get to this final step, we will need to have built a list of the events that are exposed by the chosen subject class. These will either be pre-existing events (where we've chosen an existing class) or events which we have created ourselves (from the list of properties collected above).

Figure 8 - Step 3 (Define Observable Properties)

We'll allow the user to select which of these events the observer class should be able to monitor, so the user interface for this step will present a CheckedListBox of events together with a few text boxes to collect the remaining miscellaneous information needed.

Figure 9 - Step 4 (Define Observer Class)

Figure 9 shows a screenshot of this step with some sample input. Note that in the final step we'll change the caption on the Next button to "Finish" to make sure the user knows we're nearly done.

That's all there is of the user interface. From this last step, we can start generating code once the user clicks Finish. We'll cover all of these steps in detail in a minute, but first a quick word about the method we'll be using to navigate between the various steps.

Wizard Navigation

We need to allow the user to move back through the wizard steps. Normally we would be able to just maintain a variable containing the current step number then increment or decrement when the user clicks Next or Back respectively. However, as one of our wizard steps is optional, we need a slightly more intelligent navigation method and we can handle this using a stack.

The FCL includes a Stack class, which allows us to push the history of steps one by one onto the stack and then pop them off again when we need to retrace our steps. This means we are free to direct the forward path through the wizard however we like - we can add steps, skip steps, fork off on a different path, etc. all depending on the user's choices, and then step logically back the way we came with no problems.

So we define an integer field, which holds the current wizard step, and a stack for maintaining the history of previous steps.

private Stack stephistory;
private int currentStep;

These can be initialised when the form is loaded so add an event handler for the OnLoad event.

private void OnLoad(object sender,
            System.EventArgs e)
{
// initialise wizard
            steps
stephistory = new Stack();
currentStep =
            0;
showNextStep();
}

The showNextStep() method referred to above will be used to handle all forward movement through the wizard. In general, this will do the following:

  • Push the current step onto the history stack
  • Decide what the next step should be
  • Prepare the UI for the next step
  • Display the UI for the next step

Something like:

private void showNextStep()
{
// save
            current step for
            backtrack
stephistory.Push(currentStep);

// prepare ui
            content before display
switch(currentStep)
{
case 0:
//
            show step 1 next
currentStep = 1;
// TODO: prepare UI for step
            1
break;
}

// show ui for the next
            step
showStep(currentStep);
}

The preparation of the UI for each of the steps could be broken out into individual methods - and would be if the wizard were larger - but for this application, they're compact enough to keep within a switch statement in the one method. I'll cover these options in detail in just a moment.

The showStep() method used in the above just shows the relevant panel for the required step, sets the headings and adjusts the properties of the Back and Next buttons at the limits of the wizard.

private void showStep (int step)
{
// show ui for given wizard step
string[] head = {"Select a Programming Language",
"Select or Create the Subject Class",
"Create Observable Properties",
"Create the Observer Class"};

string[] subhead = {"Generating code in different languages ...",
"Select or create the class which will...",
"Create observable properties in the Subject...",
"Create an Observer class and specify which..."};

// set heading and subheading
heading.Text = head[step-1];
subheading.Text = subhead[step-1];

// show correct UI for this step
panelstep1.Visible = (step == 1);
panelstep2.Visible = (step == 2);
panelstep3.Visible = (step == 3);
panelstep4.Visible = (step == 4);

// enable/disable back button
backbutton.Enabled = (step > 1);

// set caption for next button
if (step == MAX_STEPS)
nextbutton.Text = "&Finish";
else
nextbutton.Text = "&Next >";
}

Then the Next button only needs to check if it has reached the last step or not.

private void nextclick(object sender, System.EventArgs e)
{
// show next step or finish
if (currentStep < MAX_STEPS)
showNextStep();
else
generate();
}

And the Back button just pops the last step off the stack and redisplays it.

private void backclick(object sender,
            System.EventArgs e)
{
// show previous step
currentStep =
            (int)stephistory.Pop();
nextbutton.Enabled =
            true;
showStep(currentStep);
}

So all that remains for the prompting phase of the wizard is to prepare the user interface for each of the four steps by completing the switch statement in showNextStep(). I'll cover each of these separately as some are more involved than others.

Step 1 - Choose Language

This is the very first step. All we can really do here is to set the default selection depending on the language of the current project. Remember that when we get here, the current step is zero because that's how we initialised it when the form loaded.

case 0:
// show step 1 next
currentStep = 1;

// set language default before step 1
if (currentProject.Kind == PrjKind.prjKindCSharpProject)
langCSharp.Checked = true;
else
langVB.Checked = true;
break;

Step 2 - Define Subject Class

After the language has been chosen, we will always display step 2 - to allow selection or creation of the subject class. So to prepare for step 2, we must populate the combo box with the names of any classes already defined in the project. We can do this by checking the CodeElements collection held by the project's CodeModel. As this collection may contain namespace definitions, we need to do this recursively to make sure we find any classes nested within the namespace. I've created a recursive method - populateClasses() - to do this, so the switch statement in showNextStep() for this part appears as follows.

   case 1:
// show step 2
            next
currentStep = 2;

// populate combo with existing
            classes
classList = new
            ArrayList();
populateClasses(currentProject.CodeModel.CodeElements);
subjectcombo.DataSource
            = classList;

// set default options for step
            2
createsubject.Checked = true;
nextbutton.Enabled =
            false;
txtsubject.Text =
            "";
txtsubject.Focus();
break;

And the method populateClasses() is defined as:

private void populateClasses(CodeElements
            elements)
{
// populate combo with classes defined in
            project
foreach(CodeElement el in elements)
{
// recurse
            into namespaces
if (el.Kind ==
            vsCMElement.vsCMElementNamespace)
{
EnvDTE.CodeNamespace ns =
            (EnvDTE.CodeNamespace)el;
populateClasses(ns.Members);
}

//
            add class names to combo if defined in project
if (el.Kind ==
            vsCMElement.vsCMElementClass &&
el.InfoLocation ==
            vsCMInfoLocation.vsCMInfoLocationProject)
{
classList.Add(new
            ClassItem(el));
}
}
}

Notice that I've used an ArrayList called classList to hold the list of classes found by this method. This serves two purposes. First, by defining a helper object - ClassItem - to store each item found, I can retain a reference to the CodeElement describing the class so I don't need to hunt for it later if the user selects it as the subject class. Secondly, by overriding the ToString() method of ClassItem, I can use the ArrayList as the data source of the combo box to display the list of class names without any further processing.

Therefore, the list of classes is held in a member field:

private ArrayList classList;

Each entry being an instance of a ClassItem :

public class ClassItem
{
public
            CodeClass ClassRef;

public ClassItem(CodeElement
            element)
{
// save code class
ClassRef =
            (CodeClass)element;
}

public override string
            ToString()
{
// return class name
return
            ClassRef.Name;
}
}

Note that the CodeElement relating to the class definition in the user's project is recast to a CodeClass when stored by this helper class. This simplifies the process of extracting any events defined within the class when we're building the display for step 4, part of which we'll do next.

Step 3 - Define Observable Properties

This is the optional step. If the user has chosen to create a new subject class then we will display step 3 and allow them to define one or more properties that will be taken to represent the internal state of the subject. If the user has chosen an existing class as the subject, then we don't need to display step 3 but we do need to populate the list of events to be displayed in step 4. The switch statement in showNextStep() for this step is:

case 2:
// if creating the subject class, show step 3
if (createsubject.Checked)
{
// clear observable properties and events before step 3
currentStep = 3;
lstproperties.Items.Clear();
lstevents.Items.Clear();
txtproperty.Text = "";
nextbutton.Enabled = false;
}
else
{
// get events in selected class
ClassItem item = (ClassItem)subjectcombo.SelectedItem;
nextbutton.Enabled = false;
populateEvents(item.ClassRef);
goto case 3;
}
break;

The first part of this is straightforward. If we're creating the subject class then all we need to do is clear the relevant UI controls then go ahead and display step 3. Otherwise, we retrieve the selected ClassItem from the combo and use the new method populateEvents() to find out what (if any) events are defined in the CodeClass. This is where things start to get a little messy.

Let's take the easy case first. What we want to achieve in both cases is to populate the CheckedListBox lstevents with a list of event names. In a similar manner to the ClassItems we used above, we'll define a helper class - EventItem - to hold a reference to an event. This is defined as:

public class EventItem
{
public string
            EventName;
public CodeMemberMethod DelegateType;

public
            EventItem(string eventname, CodeMemberMethod
            delegatetype)
{
// save details
EventName =
            eventname;
DelegateType = delegatetype;
}

public
            override string ToString()
{
// return event name
return
            EventName;
}
}

As well as holding the name of each event, we also store a CodeDOM definition of the delegate type. This eases the generation of code in the observer class and you'll see how this works later.

If you refer back to Figure 8, you'll see that step 3 presents a text box, a list box and a button. This allows the user to add individual properties to the new subject class. At the time this is done, we can automatically generate an EventItem and add it to lstevents simply by following Microsoft's guidelines for naming delegates and events. For instance, if the user adds a property called PropertyOne, then we can generate an event called PropertyOneChanged with a delegate named PropertyOneChangedHandler. The delegate will define two parameters, a System.Object named sender and a PropertyOneChangedEventArgs named e. The latter will descend from System.EventArgs and we'll also generate this automatically, but the point right now is that, at the time the user clicks the Add button, we have all the information we need to create an EventItem and store it in the list. Like this:

private void addClicked(object sender,
            System.EventArgs e)
{
// add a property name to the
            list
lstproperties.Items.Add(txtproperty.Text);

//
            generate event name and delegate type definition
string eventname
            = txtproperty.Text + "Changed";
string delegatename = eventname +
            "Handler";
string argsname = eventname +
            "EventArgs";

CodeMemberMethod meth = new
            CodeMemberMethod();
meth.Name = delegatename;
meth.Attributes
            = MemberAttributes.Public |
            MemberAttributes.Final;
meth.ReturnType = new
            CodeTypeReference(typeof(void));

CodeParameterDeclarationExpression
            parm1 =
new CodeParameterDeclarationExpression(typeof(object),
            "sender");
CodeParameterDeclarationExpression parm2 =
new
            CodeParameterDeclarationExpression(argsname,
            "e");

meth.Parameters.Add(parm1);
meth.Parameters.Add(parm2);

// add event item to list
lstevents.Items.Add(new
            EventItem(eventname, meth), true);

// set focus for another
            property
txtproperty.Text = "";
txtproperty.Focus();

//
            allow next step when at least one property
            exists
nextbutton.Enabled = true;
}

Apart from the long-winded CodeDOM syntax required to define a member method, this is all pretty simple. We end up with a list box of event names from which we can easily generate a suitable method declaration matching the corresponding delegate. This is going to make the generation of the observer class much easier to code.

Problems, Problems.

Right, that was the easy bit (easy bit!?), because we have everything under our control so we are creating the subject class according to our strictly defined rules. Now what about the case where we are using an existing class as our subject? Well, the actions we need to take here are easy to state:

  • Find all events defined by the class
  • For each event find the matching delegate
  • Using the above, populate the list box with EventItems

This is what we want our method populateEvents() to do. We referenced it earlier from showNextStep() and we passed it a CodeClass object for the class we want to examine. So the first task above seems easy enough - we can look through the CodeElements defined in the given class and find any events.

private void populateEvents(CodeClass
            classRef)
{
// populate list box with public events from given
            class
lstevents.Items.Clear();

foreach(CodeElement el in
            classRef.Members)
{
// only add events
if (el.Kind ==
            vsCMElement.vsCMElementEvent)
{
// get event name
string
            eventname = el.Name;

So far, so good. Here's the problem - having found an event, there is no way via the CodeModel to find out the name or signature of the corresponding delegate. Remember that we could recast a CodeElement to a CodeClass to get information specific to a class? Well, there is no similar class (maybe CodeEvent?) that will give more information about an event. All we have are the properties defined by the CodeElement itself! Furthermore, while we can also find a list of all the delegates defined within a CodeClass, and we can recast these to CodeDelegate 's, we can't find any properties that link these to their matching events. I've queried this on the MSDN Managed newsgroups and received a brief reply from Microsoft that suggests that this is indeed the case.

This sort of stops us in our tracks.

We can find all the events, we can find all the delegates, but we can't match them up! We might start by assuming that the user has adhered strictly to the Microsoft naming suggestions and make an educated guess, but there would be no way of making sure. This is not good. I can only assume that the CodeModel and it's related classes are still undergoing development. Maybe we can expect all manner of code generating wizards to appear in VS.NET sometime in the future? I don't know but I'd sorely like to find a way round these problems to build my own tools right now.

Workarounds, Workarounds.

The best way I came up with to jump this last hurdle was to attempt to parse the source code myself to extract the name of the delegate from the declaration of the event. This is not such a huge task as it sounds. The CodeElement describing the event declaration exposes properties (StartPoint and EndPoint) that allow us to retrieve the text of the declaration. Then we can use Regular Expressions (see Introducing .NET Regular Expressions with C#) to extract the bit we need. So add the following line to ease access to the Regex classes:

using System.Text.RegularExpressions;

Now to show you my way round this problem, the populateEvents() method we started above finishes off as follows:

         //
            get text of declaration
EditPoint ep =
            el.StartPoint.CreateEditPoint();
string decl =
            ep.GetText(el.EndPoint.CreateEditPoint());

// find matching
            delegate
CodeMemberMethod meth = findDelegate(decl,
            classRef);

// add to list if all ok
if (meth !=
            null)
{
// enable Finish button when entry
            added
lstevents.Items.Add(new EventItem(eventname, meth),
            true);
nextbutton.Enabled = true;
}
}
}
}

I've introduced a separate method - findDelegate() - to do the now-more-complex-job of finding the delegate and building a matching CodeParameter. This takes the text of the source code declaration - which might be C# or VB.NET - and parses it to get the name of the delegate. Then it searches the delegates defined in the CodeClass to find a match, and builds a CodeMemberMethod if it succeeds. Here goes:

private CodeMemberMethod findDelegate(string eventdecl, CodeClass classRef)
{
// find delegate for given event and return as a method
string delegatename = null;
string pattern = null;

// set regex pattern for vb or c# source code
if (currentProject.Kind == PrjKind.prjKindCSharpProject)
pattern = @"event\s+(\w+)\s+";
else
pattern = @"As\s+(\w+)\s*$";

// use regular expression to parse source code
Regex r = new Regex(pattern);
Match m = r.Match(eventdecl);
if (m.Groups.Count > 1) delegatename = m.Groups[1].Value;

This gets us a delegate name from the event declaration. There are limitations of this particular implementation which I'll spell out later but for now we can just search for this delegate in the CodeClass:

   // search for a definition
            of the delegate in the class
foreach(CodeElement el in
            classRef.Members)
{
// check all delegates
if (el.Kind ==
            vsCMElement.vsCMElementDelegate
            &&
el.Name.Equals(delegatename))
{
// found it, so
            build a matching method declaration

And, when(if) we find it, we can recast it to a CodeDelegate and extract all the information we need to build a matching CodeMemberMethod using CodeDOM. This can finally be stored in the list box as an EventItem in the same way that we did earlier when creating a new subject class.

The findDelegate() method continues as follows:

         CodeDelegate
            delg = (CodeDelegate)el;
CodeMemberMethod meth = new
            CodeMemberMethod();
meth.Name = delegatename;
meth.Attributes
            = MemberAttributes.Public
            |
MemberAttributes.Final;
meth.ReturnType = new
            CodeTypeReference(typeof(void));

// build parameter
            list
foreach(CodeElement elm in delg.Parameters)
{
//
            recast CodeElement to get type and name
CodeParameter mparm =
            (CodeParameter)elm;

// add parameter to the method
            declaration
CodeParameterDeclarationExpression parm =
new
            CodeParameterDeclarationExpression(mparm.Type.AsFullName,
            mparm.Name);
meth.Parameters.Add(parm);
}

// return to
            caller
return meth;
}
}

// return null if none
            found
return null;
}

This does seem like an awfully long-winded way to get the job done. I can't help thinking that there should be some sort of integration between the CodeModel view of a project and the System.CodeDom namespace classes. There are a lot of overlaps in the two class hierarchies - for example the CodeParameter (from CodeModel) and CodeParameterDeclarationExpression (from System.CodeDom namespace) used above. There is also an ICodeParser interface defined within System.CodeDom namespace to allow compiler writers to expose code parsing facilities to designers, but unfortunately this interface is not supported by either the CSharpCodeProvider or VBCodeProvider classes. If it were, it might be possible to do away with the CodeModel altogether and parse the user's source code directly into CodeDOM, where a more comprehensive breakdown of all code structures would be possible.

In any case, we've finally come through the most complex step of the wizard so far. By now we should have populated the list of events to be displayed in step 4, so all we need to do is to set up the remaining user interface elements before displaying this final step to the user.

Step 4 - Define Observer Class

The last step of the wizard now needs little further preparation. Here we can set a default filename for the generated code depending on the language selection made in step 1:

case 3:
// show step 4 next
currentStep = 4;

// set default filename for output
if (langCSharp.Checked)
txtfilename.Text = "Observer.cs";
else
txtfilename.Text = "Observer.vb";
break;

Having got to this stage, once the user clicks the Next button, all that remains is to generate the code for the subject and/or observer classes and we're done.

Generating Code

If you remember back when we defined the click handler for the Next button, the method we call after the last wizard step is generate(). This generates the required code then closes the form.

We'll define two more private fields in the main form to assist this generation process. The first will contain the CodeDOM namespace within which all of our declarations will reside. The second is a string variable holding the name of the subject class. This is used at various points in the generation and by defining a member field, it saves us passing it to and fro between methods.

private System.CodeDom.CodeNamespace codenamespace;
private string subjectname;

Then the definition of the generate() method begins as follows:

private void generate()
{
// set code
            provider for chosen language
CodeDomProvider codeProvider;
if
            (langCSharp.Checked)
codeProvider = new
            CSharpCodeProvider();
else
codeProvider = new
            VBCodeProvider();

// initialise code
            generator
ICodeGenerator codeGenerator =
            codeProvider.CreateGenerator();

// create a
            namespace
codenamespace = new
            System.CodeDom.CodeNamespace(txtnamespace.Text);

Here we create a CodeDOM code provider for the chosen language and create a code generator. Then we create the namespace to receive the remaining declarations. The method continues with the definition of the subject class (if we're creating it), which we'll delegate to another private method:

   // set subject class
            name
if (createsubject.Checked)
{
// add "subject" class to
            the namespace
subjectname =
            txtsubject.Text;
generateSubjectClass(subjectname);
}
else
subjectname
            = subjectcombo.Text;

Then in a similar fashion, we generate the observer class:

// add "observer" class to the namespace
generateObserverClass(txtobserver.Text);

All of these generated definitions are added to the codenamespace field so when we reach this stage we can use CodeDOM to generate the code for us from the namespace:

// generate code to a string
StringWriter sw = new StringWriter();
codeGenerator.GenerateCodeFromNamespace(codenamespace, sw, null);

This gives us a neatly formatted chunk of source code in the StringWriter object.

Before I go into the detail of the CodeDOM calls for generating the subject and observer classes, I'll just finish off the generate() method by adding a new item to the current project and inserting our generated code. The method finishes like this:

// add an item to the project
ProjectItem projItem =
applicationObject.ItemOperations.AddNewItem(@"Local Project Items\Text File", txtfilename.Text);

// get edit point in new item and write code
TextDocument textDoc = (TextDocument)projItem.Document.Object("TextDocument");
EditPoint ep = textDoc.StartPoint.CreateEditPoint();
ep.Insert(sw.ToString());

// close dialog when done
Close();
}

Generating the Subject Class

The subject class is the more involved of the two classes we're generating. In addition to the properties, events and delegates needed in the subject itself, we also need to define an EventArgs descendant for each event. We simplified things earlier by assuming all of our subject's observable properties would be of type System.Object but, even for only a single property - say PropertyOne - we need to generate the following code.

An EventArgs class, taking a reference to the changed value in the constructor and exposing the same via a read-only property:

public class PropertyOneChangedEventArgs :
            System.EventArgs
{
private object _PropertyOne;

public
            PropertyOneChangedEventArgs(object PropertyOne)
{
_PropertyOne = PropertyOne;
}

private object
            PropertyOne
{
get
{
return
            _PropertyOne;
}
}
}

Then the subject class itself, containing a private member field exposed via a property, an event declaration, a delegate declaration, and a protected method to allow derived classes to catch and/or raise the event:

public class Subject
{
private object _PropertyOne = null;

public delegate void PropertyOneChangedHandler(object sender,
PropertyOneChangedEventArgs e);

public event PropertyOneChangedHandler PropertyOneChanged;

protected void OnPropertyOneChanged()
{
if (PropertyOneChanged != null)
PropertyOneChanged(this, new PropertyOneChangedEventArgs(_PropertyOne));
}

private object PropertyOne
{
get
{
return _PropertyOne;
}
set
{
_PropertyOne = value;
OnPropertyOneChanged();
}
}
}

If you read the MSDN article Exploring the Observer Design Pattern, all of this will make sense. The use of a protected method to raise the event indirectly is especially useful. This allows derived classes to monitor events in their base class (i.e. the subject class) and choose to invoke any observers registered with the base class.

So the process of generating the subject class begins like this:

private void generateSubjectClass(string
            classname)
{
// generate a type declaration for the
            "Subject"
CodeTypeDeclaration subjClass =
new
            CodeTypeDeclaration(classname);

// process property
            definitions
foreach(object prop in
            lstproperties.Items)
{
// make names
string fieldname =
            prop.ToString();
string internalfieldname = "_" +
            fieldname;
string eventname = fieldname + "Changed";
string
            delegatename = eventname + "Handler";
string fireevent = "On" +
            eventname;
string argsname = eventname + "EventArgs";

//
            create a custom EventArgs class
generateEventArgsClass(fieldname,
            internalfieldname);

Here, we create a declaration for the class itself and then loop through all of the properties defined by the user in step 3 of the wizard. For each one, we apply the recommended naming conventions to the root property name to derive all of the names we'll need during this process. Then we create the EventArgs class for the property using the helper method generateEventArgsClass(), which I'll describe in a moment.

Next, we'll create a delegate. We already know what this should be called and we know from the Microsoft recommendations what parameters it should take. So:

// create a delegate in the subject class
CodeTypeDelegate delg = new CodeTypeDelegate(delegatename);
delg.Attributes = MemberAttributes.Public;
delg.ReturnType = new CodeTypeReference(typeof(void));
CodeParameterDeclarationExpression parm1 =
new CodeParameterDeclarationExpression(typeof(object), "sender");
CodeParameterDeclarationExpression parm2 =
new CodeParameterDeclarationExpression(argsname, "e");
delg.Parameters.Add(parm1);
delg.Parameters.Add(parm2);
subjClass.Members.Add(delg);

Now we create the event and use the name of the delegate we just created to define the event type. Note that we could have created these declarations in any order - you'll see when we actually generate the code that the code provider has its own logic for choosing in what order to spit out the source code. Anyway, the event is declared like this:

// create an event in the subject
            class
CodeMemberEvent newevent = new
            CodeMemberEvent();
newevent.Attributes =
            MemberAttributes.Public;
newevent.Type = new
            CodeTypeReference(delegatename);
newevent.Name =
            eventname;
subjClass.Members.Add(newevent);

Creating the protected method that fires the event suffers from the usual unwieldy CodeDOM syntax although there's nothing essentially difficult. Note here that we can add comments as statements within the code we generate. This can (and should) be used to inform the user where they need to add their own logic into the generated code:

// create protected event firing method in
            the subject class
CodeMemberMethod onfire = new
            CodeMemberMethod();
onfire.Attributes = MemberAttributes.Family |
            MemberAttributes.Final;
onfire.Name = fireevent;

//
            prepare an "if" statement
CodeConditionStatement ifcond = new
            CodeConditionStatement();

// prepare the condition - i.e.
            (eventname != null)
CodeBinaryOperatorExpression notnull = new
            CodeBinaryOperatorExpression();
notnull.Left = new
            CodeVariableReferenceExpression(eventname);
notnull.Operator =
            CodeBinaryOperatorType.IdentityInequality;
notnull.Right = new
            CodePrimitiveExpression(null);

// add the condition to the
            "if" statement
ifcond.Condition = notnull;

// prepare the
            statement that invokes the delegate
CodeDelegateInvokeExpression
            invoke = new CodeDelegateInvokeExpression();
invoke.TargetObject
            =new CodeEventReferenceExpression(null,
            eventname);
invoke.Parameters.Add(new
            CodeThisReferenceExpression());
CodeObjectCreateExpression parm =
            new CodeObjectCreateExpression();
parm.CreateType = new
            CodeTypeReference(argsname);
parm.Parameters.Add(new
            CodeVariableReferenceExpression(internalfieldname));
invoke.Parameters.Add(parm);

//
            add the delegate invoke to the "true" part of the
            "if"
ifcond.TrueStatements.Add(new CodeCommentStatement("fire
            event"));
ifcond.TrueStatements.Add(invoke);

// add the
            "if" to the method's
            statements
onfire.Statements.Add(ifcond);

// finally, add
            the method to the subject
            class
subjClass.Members.Add(onfire);

After that, creating a private field to hold the property value seems almost trivial:

// create a private member field
CodeMemberField field =
new CodeMemberField(typeof(object), internalfieldname);
field.Attributes = MemberAttributes.Private;
field.InitExpression = new CodePrimitiveExpression(null);
subjClass.Members.Add(field);

Then, back to the long-winded syntax to describe the property itself. I said earlier that I don't intend to describe the CodeDOM syntax in detail and I'm sticking to that- you can always refer to Matt Reynolds earlier article for a detailed treatment - but this code does deserve a careful perusal to make sure you understand why everything needs to be described in such detail in terms of the CodeDOM classes. Only by doing it this way can we successfully generate code in the language of our choice (within certain limits).

The property definition is the last part of the subject class, so the generateSubjectClass() method finishes like this:

// add a property
CodeMemberProperty newprop = new CodeMemberProperty();
newprop.Attributes = MemberAttributes.Public | MemberAttributes.Final;
newprop.Type = new CodeTypeReference(typeof(object));
newprop.Name = fieldname;
CodeVariableReferenceExpression privatevar =
new CodeVariableReferenceExpression(internalfieldname);
CodeMethodReturnStatement returnstate =
new CodeMethodReturnStatement(privatevar);
newprop.GetStatements.Add(
new CodeCommentStatement("return internal state"));
newprop.GetStatements.Add(returnstate);
CodePropertySetValueReferenceExpression valuevar =
new CodePropertySetValueReferenceExpression();
CodeAssignStatement assign =
new CodeAssignStatement(privatevar, valuevar);
newprop.SetStatements.Add(
new CodeCommentStatement("TODO: Add validation if required"));
newprop.SetStatements.Add(assign);
CodeMethodReferenceExpression meth =
new CodeMethodReferenceExpression(null, fireevent);
CodeExpression[] params1 = new CodeExpression[0];
CodeMethodInvokeExpression methinv =
new CodeMethodInvokeExpression(meth, params1);
newprop.SetStatements.Add(
new CodeCommentStatement("fire event when property changes"));
newprop.SetStatements.Add(methinv);
subjClass.Members.Add(newprop);
}

// add the completed declaration to the namespace
codenamespace.Types.Add(subjClass);
}

We'll just fill in the missing gap here and cover the generation of the EventArgs descendant, one of which is generated for each observable property defined in the subject class.

Generating the EventArgs Descendant Class

When generating the subject class, we invoked the helper method generateEventArgsClass() passing the property name and internal field name as parameters. This method does much as you'd expect. First, create a CodeTypeDeclaration for the class - setting the base class to System.EventArgs - using the recommended naming convention to derive the class name:

private void generateEventArgsClass(string fieldname,
string internalfieldname)
{
// generate a type definition for a custom EventArgs class
string classname = fieldname + "ChangedEventArgs";

// create the class
CodeTypeDeclaration argsClass =
new CodeTypeDeclaration(classname);
argsClass.BaseTypes.Add(typeof(EventArgs));

Then, like we did for the subject class, we add a private field to hold the value of the changed property:

// add a private field to hold the new value
CodeMemberField field =
new CodeMemberField(typeof(object), internalfieldname);
field.Attributes = MemberAttributes.Private;
argsClass.Members.Add(field);

Next, we build a constructor:

// build the constructor
CodeConstructor cons = new CodeConstructor();
cons.Attributes = MemberAttributes.Public;
CodeParameterDeclarationExpression parm =
new CodeParameterDeclarationExpression(typeof(object), fieldname);
cons.Parameters.Add(parm);
CodeVariableReferenceExpression left =
new CodeVariableReferenceExpression(internalfieldname);
CodeVariableReferenceExpression right =
new CodeVariableReferenceExpression(fieldname);
CodeAssignStatement assign =
new CodeAssignStatement(left, right);
cons.Statements.Add(
new CodeCommentStatement("TODO: Add additional fields if required"));
cons.Statements.Add(assign);
argsClass.Members.Add(cons);

Then we add a read-only property to allow access to the changed value. The only difference between a read-only property and a normal one is that the read-only property has no set statements:

   // add a read-only
            property
CodeMemberProperty prop = new
            CodeMemberProperty();
prop.Attributes = MemberAttributes.Public |
            MemberAttributes.Final;
prop.Type = new
            CodeTypeReference(typeof(object));
prop.Name =
            fieldname;
CodeMethodReturnStatement returnstate =
new
            CodeMethodReturnStatement(left);
prop.GetStatements.Add(returnstate);
argsClass.Members.Add(prop);

And, finally, we add the generated class definition to the CodeDOM namespace:

// add the completed declaration to the namespace
codenamespace.Types.Add(argsClass);
}

The last thing to do now is to generate the observer class.

Generating the Observer Class

Generating the observer class is actually quite simple. We did all the hard work earlier by generating CodeMemberMethod definitions for event handlers based on the corresponding delegates. These were stored in EventItem instances in the CheckedListBox displayed in step 4 of the wizard.

To make it a touch more useful, we'll generate a method - Register() - in the observer class, which will take a subject class as a parameter and register the observer with all of the chosen events. We don't have to do this, but it does nicely demonstrate how we can generate the very different syntax used by C# and VB.NET from the same CodeDOM definitions.

So the generateObserverClass() starts predictably with a CodeTypeDeclaration defining the class name:

private void generateObserverClass(string classname)
{
// generate a type declaration for the "Observer"
CodeTypeDeclaration obsClass =
new CodeTypeDeclaration(classname);

Then we define the Register() method, which takes an instance of the subject class as a parameter:

   // create register method
            in observer class
CodeMemberMethod obsreg = new
            CodeMemberMethod();
obsreg.Name =
            "Register";
obsreg.Attributes = MemberAttributes.Public |
            MemberAttributes.Final;
obsreg.ReturnType = new
            CodeTypeReference(typeof(void));
CodeParameterDeclarationExpression
            obsparm =
new CodeParameterDeclarationExpression(subjectname,
            "_subject");
obsreg.Parameters.Add(obsparm);
obsreg.Statements.Add(
new
            CodeCommentStatement("add observer delegate(s) for
            event(s)"));
obsClass.Members.Add(obsreg);

Before we loop through the checked items in the step 4 CheckedListBox, we'll define a variable reference to the incoming subject parameter as we need this inside the loop:

// create variable reference for subject
CodeVariableReferenceExpression subjvar =
new CodeVariableReferenceExpression("_subject");

Finally, we can iterate through all the events the user has selected. From each EventItem we can retrieve the pre-built CodeMemberMethod that matches the corresponding delegate, add this to the observer class, and then add a statement to the Register() method that will allow the observer to monitor this event:

// process all required observable events
foreach(EventItem item in lstevents.CheckedItems)
{
// add a method for required delegate
obsClass.Members.Add(item.DelegateType);

// add attach in observer's register method
CodeTypeReference delgtype =
new CodeTypeReference(subjectname + "." + item.DelegateType.Name);
CodeThisReferenceExpression thisref = new CodeThisReferenceExpression();
CodeDelegateCreateExpression listener =
new CodeDelegateCreateExpression(delgtype, thisref,
item.DelegateType.Name);
CodeAttachEventStatement attach =
new CodeAttachEventStatement(subjvar, item.EventName, listener);
obsreg.Statements.Add(attach);
}

// add the completed declaration to the namespace
codenamespace.Types.Add(obsClass);
}

That completes the add-in. Let's try it out.

Sample Outputs

From the VS.NET development environment press F5 to build and run the DesignPattern add-in. When the second instance of VS.NET starts, create a new project - a C# Class Library will do - then choose Tools / DesignPattern from the menu. This will display the first step of our wizard.

Choose C# as the language and select the option to create a new subject class (call it Subject). Add just one property (PropertyOne), enter a namespace of Wrox.CSharpToday then accept the defaults for the remaining entries and click Finish. This will generate the following code:

namespace Wrox.CSharpToday {

public
            class PropertyOneChangedEventArgs : System.EventArgs
            {

private object _PropertyOne;

public
            PropertyOneChangedEventArgs(object PropertyOne) {
// TODO: Add
            additional fields if required
_PropertyOne =
            PropertyOne;
}

public object PropertyOne {
get
            {
return _PropertyOne;
}
}
}

public class Subject
            {

private object _PropertyOne = null;

public object
            PropertyOne {
get {
// return internal state
return
            _PropertyOne;
}
set {
// TODO: Add validation if
            required
_PropertyOne = value;
// fire event when property
            changes
OnPropertyOneChanged();
}
}

public event
            PropertyOneChangedHandler PropertyOneChanged;

protected void
            OnPropertyOneChanged() {
if ((PropertyOneChanged != null) {
//
            fire event
PropertyOneChanged(this, new
            PropertyOneChangedEventArgs(_PropertyOne));
}
}

public
            delegate void PropertyOneChangedHandler(object sender,
            PropertyOneChangedEventArgs e);
}

public class Observer
            {

public void Register(Subject _subject) {
// add observer
            delegate(s) for subject's event(s)
_subject.PropertyOneChanged +=
            new
            Subject.PropertyOneChangedHandler(this.PropertyOneChangedHandler);
}

public
            void PropertyOneChangedHandler(object sender,
            PropertyOneChangedEventArgs e) {
}
}
}

You have to admit that this was a lot easier than coding these definitions by hand!

Now launch the add-in again from the Tools menu and go through the same procedure, except this time select VB.NET as the language. When you click Finish, you should see the Visual Basic equivalent of the above C# code:

Namespace Wrox.CsharpToday

Public Class PropertyOneChangedEventArgs
Inherits System.EventArgs

Private _PropertyOne As Object

Public Sub New(ByVal PropertyOne As Object)
MyBase.New
'TODO: Add additional fields if required
_PropertyOne = PropertyOne
End Sub

Public ReadOnly Property PropertyOne As Object
Get
Return _PropertyOne
End Get
End Property
End Class

Public Class Subject

Private _PropertyOne As Object = Nothing

Public Property PropertyOne As Object
Get
'return internal state
Return _PropertyOne
End Get
Set
'TODO: Add validation if required
_PropertyOne = value
'fire event when property changes
OnPropertyOneChanged
End Set
End Property

Public Event PropertyOneChanged As PropertyOneChangedHandler

Protected Sub OnPropertyOneChanged()
If (Not (PropertyOneChanged) Is Nothing) Then
'fire event
RaiseEvent PropertyOneChanged(Me, New PropertyOneChangedEventArgs(_PropertyOne))
End If
End Sub

Public Delegate Sub PropertyOneChangedHandler(ByVal sender As Object, ByVal e As PropertyOneChangedEventArgs)
End Class

Public Class Observer

Public Sub Register(ByVal _subject As Subject)
'add observer delegate(s) for subject's event(s)
AddHandler _subject.PropertyOneChanged, AddressOf Me.PropertyOneChangedHandler
End Sub

Public Sub PropertyOneChangedHandler(ByVal sender As Object, ByVal e As PropertyOneChangedEventArgs)
End Sub
End Class
End Namespace

Notice how the long-winded CodeDOM syntax is now paying dividends? We can flick from one language to another and generate exactly the code we need without having to worry about differences in the syntax. In particular, you can see that the VB implementation of the constructor in the PropertyOneChangedEventArgs class has automatically added a call to the base class constructor. We didn't need to do any more than use the correct CodeDOM class to define this particular method as a constructor. Similarly, note the different syntax generated in the observer's Register() method, and the protected OnPropertyOneChanged() method.

Surely this has much more scope than just producing code-generating wizards? What about interactive tutorials? Demonstrate a particular coding technique then have the user click a button to see how it's implemented in C#, VB, or even JScript. If you've run through the .NET Quickstart tutorials you may see some parallels here. If it were easier to parse source code than it is using CodeModel, surely we could easily translate between languages? If we already have a parser for a language, maybe an aging legacy system or even a scripting or macro language, we could theoretically write our own tool to help port this code to the .NET environment. Perhaps this is naïve as it is optimistic - after all, CodeDOM is really only suitable for OO languages - but there's plenty of scope for a little experimentation and personally I find this idea quite appealing.

Now, we need to test that we can select an existing subject class and still get the same result. One easy way to do this is to manually delete the definition of the observer class in the code we generated above and then run the wizard again. This time, we can select the option to use an existing class as the subject - our previously-generated class should be in the list - and we'll see that the wizard steps straight to the final step where we can define the new observer. If you follow these steps, you'll find that the observer class definition gets recreated exactly like it was before (remember to select a different filename for the output or you'll overwrite the previous definitions!).

We'll just do one more test. Having generated some code, we really ought to check that it will compile and run as we expect! So -

  • Create a New Project - a C# Console Application - and call it DPTester
  • Run the add-in from the Tools menu and choose C# as the language
  • Choose to create a new subject class called TestSubject
  • Add two properties - PropertyOne and PropertyTwo
  • Use namespace DPTester and choose to create an observer called TestObserver
  • Click Finish to generate the code then save and build the project

That tests that the generated code compiles ok, so now add a statement to the TestObserver class PropertyOneChangedHandler() as follows:

System.Console.WriteLine("PropertyOne has changed");

And add a similar statement to the PropertyTwoChangedHandler() method:

System.Console.WriteLine("PropertyTwo has changed");

Then change the Main() entry point of the application as follows:

[STAThread]
static void Main(string[]
            args)
{
// create subject
Console.WriteLine("Creating
            subject");
TestSubject subj = new TestSubject();

// create
            observer
Console.WriteLine("Creating observer");
TestObserver
            obs = new TestObserver();

// test events with no
            observer
Console.WriteLine("Testing events with no
            observer");
subj.PropertyOne = 1;
subj.PropertyTwo =
            2;

// register observer
Console.WriteLine("Registering
            observer");
obs.Register(subj);

// test events with
            observer
Console.WriteLine("Testing events with
            observer");
subj.PropertyOne = 3;
subj.PropertyTwo =
            4;
}

This will check that we can create our subject and observer ok, we can fire events in the subject when we have no registered observers, the Register() method in the observer works correctly, and that the events are trapped as expected.

Build the project and run it. You should receive the following output showing that everything is working just as we would hope:

Creating subject
Creating observer
Testing events with no observer
Registering observer
Testing events with observer
PropertyOne has changed
PropertyTwo has changed

Application Setup

If you want to try this out for yourself, download the support material for this case study from the C# Today web site and follow the instructions in the ReadMe.txt file included. Distributing add-ins between systems is a little more involved than for normal projects as the necessary registry entries are only set up automatically on the original development machine.

The download contains the DesignPatternSetup project which recreates the necessary registry entries to allow you both to use the add-in and to amend the source code to make your own modifications.

Limitations

While we have created a useful application, there are some obvious and not-so-obvious limitations. The obvious limitations are those resulting from the incomplete CodeModel classes available from the VS.NET automation model. We covered a workaround using Regular Expressions that worked well but won't cope with two possible situations:

First, it's not mandatory to declare a new delegate for every event. It's perfectly allowable for an event to use the FCL class System.EventHandler as a delegate, in which case the add-in would be unable to find a definition in the source code and so would not present this event as an option in step 4.

Secondly, in VB.NET, there are two ways of declaring an event. The first way is similar to C# and this is covered by the findDelegate() workaround method. The second is to use an implied delegate (created by the VB compiler) by using syntax similar to the following:

Public Event SomethingChanged(sender As Object, e As EventArgs)

This would need a much more detailed treatment to parse the parameter list from the VB.NET source code and is not recognised by our current workaround.

Other limitations are:

The add-in does not detect events exposed by base classes. This is a little bit tricky as the base class could also be declared in the project in question or it could be declared in a referenced assembly. In the former case we'd need to get access to this via CodeModel, but in the latter it would be via Reflection. We suffer the same situation all the way back up a class hierarchy and this would make our add-in much more complex - though also much more useful!

We also don't allow an existing class to be selected as the observer. I thought long and hard about this as it would have been useful to have analysed an existing class to see what events in the subject it is currently able to observe. The big sticking point here is that there's no way we can get this information by looking at the syntax of the source code. The observer may well define a method with the same signature as a delegate in the subject class, but we can't find out whether or not the observer uses it for this purpose or not without completely analysing the bodies of all methods! Perhaps this is something that we could control if we hid all of the application design behind code-generators (cf 4GL) - then we could introduce meta-data to keep track of the design at the lower level.

As a half-way solution to the above, I considered employing custom attributes in the generated code to give some degree of visibility of the more abstract level of design in a compiled assembly. Maybe I'll follow this up further sometime but it needs sleeping on for a week or two first!

One last thing to be wary of when using CodeDOM is the possible use of data types not compliant with the Common Language Specification (CLS). If you're examining a C# project through the CodeModel, and come across a data type like uint, which is not CLS-compliant, then any attempt to generate VB source code from CodeDOM definitions containing the uint reference will result in an exception.

Case Study Review

This case study started by highlighting a few gaps in the potential of the VS.NET development environment and outlining a method whereby we might be able to do something about this with the facilities already available.

As we progressed through the development, we perhaps found out a few reasons for these gaps.

As ever, the problem is not as easy as it looks at first sight. This is true not only in a technical sense but also logically. To allow us to hook our design-generating capabilities into the development environment in the most effective manner, we really need to be much closer to the parser and compiler end of the environment that we could ever be via the more simplistic CodeModel. This implies that these sort of tools should really be more tightly integrated into the core development environment and not just bolted on using an add-in. Certainly, the ability to ask the user a few questions and then spit out a whole heap of ready-made code is relatively simple but the real benefits of this can only go so far. The big gains in development terms will come when we can abstract elements of the design from an existing project and provide a method to generate additional code that fits that design. This would open all sorts of doors. I've never been quite sure where the transition between design and coding is best drawn - over the years I have seen many CASE-type tools which allow a high level design to be decomposed to the point where it is possible to generate code, while from the "bottom-up" view we can reverse engineer class hierarchies and other structures, but what's the best way (if at all) to marry up these ideas? If you can offer an opinion or any insights then why not take part in the technical discussion by clicking the link at the end of this article?

On the other hand, looking back at our add-in, it's difficult to say that we actually failed in our attempt or that there is no merit in using what leverage we can gain from CodeModel and CodeDOM to push things as far as we can. We definitely came up with a genuinely useful tool that not only generates a correct design in the correct language syntax, but also encourages standards and conventions that are more difficult to enforce when you don't have the compiler on your side! Perhaps there is some scope for using these techniques to assist the standardisation of code at a departmental or organisational level. What do you think?

Overall, I really like the fact that CodeModel and CodeDOM are there - even if they seem incomplete or imperfect. I've not used any other development environment where it was possible to access or generate source code in such a manner and I can only hope that these features will be improved and enhanced with future releases of Visual Studio .NET. Certainly, if we can already use these tools to ease the burden on the developer, enforce standards, and encourage good design, then how can they not be a good thing?

Acknowledgements

I am pleased to offer thanks to Stephen Collyer of Netspinner for his valuable insights into Regular Expressions. Never before having used anything more than a few basic elements of this most powerful language, I couldn't have arrived at my workaround without his help.

Case Study Information
Author | Roger Rowland
Editor | Adam Ryland
Reviewers | Andrew Krowczyk, Bill Wilson, Sean Schade

If you have any questions or comments about this case study, or suggestions for further pieces, please contact the technical editor. If you wish to talk about the case study with other subscribers, please visit our forum here.

Fast Track Books from Wrox Press
If you want a C# book that is compact, fast-paced but still provides the quality of information that you have come to expect from a Wrox book - you want Fast Track C#, newly available from Wrox Press.

This book will help you to quickly learn about the following:
  • Understand what C# is, and how it fits into the .NET Framework
  • A concise introduction to all the main features of the C# language, with Java and C++ comparisons noted
  • Using the .NET base classes
  • Writing and deploying Windows applications with C# and Visual Studio .NET
  • Creating and configuring .NET assemblies
  • Accessing databases with ADO.NET, and using XML in C#
  • Integrating your existing COM components with the .NET Framework
  • COM+ Services in the .NET Framework
  • Building web applications with ASP.NET pages
  • Creating web services in C#


Extending Application Functionality

Extending Application Functionality
http://www.csharptoday.com/content.asp?id=1843
Article Rating: 3.5
Published on June 3, 2002
Abstract
Most of today’s commercial software applications support some way through which the application can be extended and customized. This allows external software vendors and developers to add functionality that is not part of the core product itself, or customize the application to form part of a bigger solution. Some companies even develop an entire solution by extending and customizing a combination of various off the shelf products. An example of this may be where a corporate customer develops an in-house solution by extending and customizing Microsoft Office. In this case study, Jerrie Pelser will take a brief look at the various techniques available to application developers to open their applications up for customization by external developers. We will then develop a basic add-in framework and develop an application which views the metadata of a .NET assembly using this framework.
Case Study

Most of today's commercial software applications support some way through which the application can be extended and customized. This allows external software vendors and developers to add functionality that is not part of the core product itself, or customize the application to form part of a bigger solution. Some companies even develop an entire solution by extending and customizing a combination of various off the shelf products. An example of this may be where a corporate customer develops an in-house solution by extending and customizing Microsoft Office.

In this case study, we will take a brief look at the various techniques available to application developers to open their applications up for customization by external developers. We will then develop a basic add-in framework and develop an application which views the metadata of a .NET assembly using this framework.

Before we continue please note the following software requirements for this case study as well as the accompanying source code:

  • Microsoft .NET Framework Version 1.0
  • Microsoft Visual Studio.NET

The various customisation mechanisms

There are various methods through which software developers can make their application more extensible:

  • Add-ins
  • Scripting
  • Exposing application functionality through an object model.
  • Exposing application functionality through exported DLL functions.

Add-ins

Add-ins allow software developers to distribute application functionality over various modules which can be added and removed from the main application. This can be done without having to recompile the entire application. The advantages of add-ins include:

  • Software developers can provide advanced application features as separate modules. This allows users of the application to pay for only the modules that they are really interested in using. Once these modules are installed on the user's machine, the extra features will typically surface in the main application via added menus or toolbar buttons.
  • Software developers can ship upgrades to their end users without having to redistribute the entire application. Only the add-in to which the changes were made need to be sent out to users.
  • If a user of the application has a need for some specific functionality, an add-in can be developed, managed and distributed to that user independently of the main application.

Add-ins are available in most modern commercial software products, including Microsoft Visual Studio .NET. Prior to .Net, applications typically took advantage of COM by implementing a specific interface through which the main application would communicate with the add-in. As stated earlier, once an add-in is loaded into the application, it will make itself typically available by appending extra menu items or toolbar buttons to the main application user interface. Other add-in technologies, such as Microsoft Management Console (MMC) snap-ins, will add extra nodes to the scope pane on the left-hand side of the main MMC user interface.

Scripting

Scripting allows users of our application to enhance its functionality by writing small pieces of software code called scripts that manipulate our application in some way, or change its standard behaviour. The most common example of scripting is found in Microsoft Office. The various applications in Microsoft Office allow end users to record macros, or develop them from scratch using VBA.

These macros are simply small pieces of software code which will perform certain tasks when they are executed. These macros are written using VBA and will manipulate the main application by performing actions on some of the objects which are exposed by the application.

Exposing the Object Model

Contemporary applications expose their functionality to other applications. In the COM world this is done through COM Automation. Automation was originally developed as a way for applications to expose their functionality to other applications, including scripting languages. The goal here is to provide a way for external client applications to access properties and call methods of the automation server. In most cases this is the method through which the previous two methods (namely Add-ins and Scripting) manipulate the main application.

Exporting DLL functions

One of the classic ways of exposing application functionality to external applications is to export DLL functions. An application developer would typically export a function from a DLL either by its name or by its ordinal value so it can be used in external applications. When an application developer wanted to use these exported functions, he would import them so they would be available to be called from his application. When importing a function, the developer would typically need to declare the name of the function, the data types of parameters it expects, as well as the name of the DLL in which it is defined. The syntax used to export and import functions from a DLL differ between programming languages and are beyond the scope of this document. For more information on how this is done under C++, please refer to the section of Importing and Exporting DLLs under the Visual C++ documentation which accompanies Visual Studio .NET.

Developing an extensible application with add-ins

During the remainder of this case study we will develop a sample application which can be extended by the use of add-ins. The sample application which we will be developing will be a .NET assembly browser. This application will allow us to view all the data types and resources which are defined in a .NET assembly.

Reflection in .NET

Before we continue discussing the .NET assembly browser in more detail, let us first have a quick look at assemblies, data types and reflection in the .NET Framework.

.NET Assemblies

Assembly Overview

A .NET assembly is a physical file which consist of one ore more portable executable (PE) files generated by a .NET compiler. An assembly consists of a manifest, one or more modules, and optionally some resources. For more information on assemblies please refer to "Simplifying Deployment and Solving DLL Hell with the .NET Framework" (http://msdn.microsoft.com/library/en-us/dndotnet/html/dplywithnet.asp?frame=true&WROXEMPTOKEN=618658Zezi9SpDFSZgQps5Xv6u).

Assemblies can be either static or dynamic. A static assembly is stored on disk in PE files. Dynamic assemblies are created in memory and executed directly without saving to disk. Dynamic assemblies are created using the classes in the System.Reflection.Emit namespace, but further discussion about this is beyond the scope of this article. For more information on dynamic assemblies please refer to the articles entitled "Dynamic assemblies in .NET framework" (http://www.csharptoday.com/content.asp?id=1725&WROXEMPTOKEN=618658Zezi9SpDFSZgQps5Xv6u) and "Emitting Dynamic Assemblies on the fly using Reflection.Emit" (http://www.csharptoday.com/content.asp?id=1784&WROXEMPTOKEN=618658Zezi9SpDFSZgQps5Xv6u).

The Assembly Manifest

The assembly manifest is the part of the assembly which makes it self-describing. The following information gets stored in the assembly manifest:

  • Assembly name. This is a text string which specifies the assembly's name.
  • Version Information. The version number consists of four parts, namely a major and minor version number, as well as a revision and build number. The Common Language Runtime (CLR) uses this information to enforce the version policy.
  • Strong name information. Contains the public key from the publisher of the assembly, which is used to give an assembly a strong name.
  • List of files. A hash of each file contained in the assembly and a file name.
  • Type information. This is the list of all the types in the assembly with a mapping to the module which contains the type. This information in used by the Reflection API to discover type information.
  • Referenced assemblies. This is a list of all external assemblies which are statically referenced by an assembly.
  • Culture Information. Defines the culture or language information for a satellite assembly. A satellite assembly contains culture or language specific resources.

Assembly Packaging

An assembly can be packaged as either a single-file assembly or as a multi-file assembly. A single-file assembly contains only one associated file. In this case the manifest is incorporated into the PE file to form a single-file assembly. A multi-file assembly contains more than one associated file. In this case the manifest can exist either as a standalone entity, or be incorporated into one of the PE files in the assembly. The diagram below illustrates this:

.NET Types

In .NET all types can be classified in two basic categories, namely value types and reference types.

Value Types

A variable of a value type contains the actual data. Every time you declare a variable of a value type, you have allocated the number of bytes associated with that type on the stack. Whenever you pass a variable that is a value type, the variable value is passed and not a reference to the underlying object.

Reference Types

Reference types are stored on the heap, and may only be accessed via a reference to its position in memory. Samples of value types are classes, interfaces, delegates, and arrays. A variable to a reference type will always contain a reference to a value of that type, or a null reference.

Everything is an object

All types in .NET are ultimately derived from System.Object, even value types. This ensures that all types have a minimum set of capabilities. Whenever you treat a value type such as an int or double as an object it goes through a process on boxing and unboxing.

Boxing is the process of converting a value type to a reference type (in other words to an object on the garbage collected heap). Boxing creates an object instance and copies the value into the new object.

Unboxing is the process of converting a reference type to a value type. All of this may seem confusing, but what is important to us is that all types in .NET ultimately inherit from System.Object and therefore has some base functionality.

Querying Metadata with Reflection

The metadata of a .NET assembly can be interrogated using the classes contained in the System.Reflection namespace. These classes enable you to logically traverse assembly and type information. If we look at the System.Reflection namespace you will notice the System.Reflection.Assembly class. This class contains information about the assembly such as the list of modules that make up the assembly, the assembly's version information, assembly dependencies, and other resources that make up the assembly.

Beneath the Assembly class we will find the System.Reflection.Module class. A module represents a single PE file consisting of one or more types. The System.Type class represents a single type declaration: a class type, interface type, array type or value type. Looking at the System.Type class we notice that it consists of various members each of which is represented by a type of System.Reflection.MemberInfo. A member can basically be a constructor, field, property, method, event or other nested types. These are all represented more specifically by classes inheriting from System.Reflection.MemberInfo, namely the ConstructorInfo, FieldInfo, PropertyInfo, MethodInfo, EventInfo, and Type classes. For constructors and methods there is one more level to the metadata. Constructors and methods may have parameters which are represented by the System.Reflection.ParameterInfo class.

The complete metadata hierarchy is displayed in the diagram below.

For more information on reflection in .NET please refer to ".NET Metadata and Custom Attributes in C#" (http://www.csharptoday.com/content.asp?id=1576&WROXEMPTOKEN=618658Zezi9SpDFSZgQps5Xv6u)

The Assembly Browser

Overview of the Assembly Browser

In the remaining part of this case study we are going to develop an application that allows us to view all the metadata for any given assembly. This Assembly Browser will be architected in such a way that it will allow other people to extend the functionality of the browser by writing a separate add-in which implements extended functionality. The Assembly Browser will consist of three main parts:

  • The Assembly Browser application. This is the main application which will be executed by the end users. It also serves as a host application for the add-ins which we will develop.
  • The extensibility module. This module is where we will define the interfaces and classes which our host application and the various add-ins will use to communicate with one another.
  • The various add-ins. For the purpose of this article we are going to develop three different add-ins:
  • A Module browser which will allow us to view the different modules which are defined in the assembly, as well as the different types which are defined in each module.
  • A Resource browser which will allow us to view information about the various resources which are included in the assembly.
  • A References browser which will allow us to view information about the other assemblies which are referenced by the specific assembly.

Below is a screenshot of what the final Assembly Browser looks like:

The user interface of the Assembly Browser is divided into the following sections:

The Explorer Pane.

The explorer pane contains the explorer tree view. The nodes that make up this tree are added by the various add-ins.

The Property Pane.

Once a node is selected in the explorer pane, the property pane contains information about the specific node which was selected. The property pane contains property sheets which displays information on the node which is selected.

The add-in framework

The add-in framework is at the heart of our Assembly Browser. It is here that we specify the interfaces that add-in developers will need to implement in order to develop an add-in for our application.

There are three ways in which an add-in could extend the Assembly Browser user interface:

  • The first way would be to register a tree node to automatically be added as a child node to another type of node.
  • The second way would be to register a callback procedure which will execute when a certain type of node is expanded.
  • The third and final way is to develop custom property sheets which will display in the property pane when a node is selected in the explorer pane.

Let us continue by looking at the various classes and interfaces which make up the add-in framework. The complete source code of all these can be found in the Extensibility project.

IAddin interface

The IAddin is implemented by add-in modules. This interface allows us to notify an add-in of events such as when an add-in gets loaded or unloaded.

Below is the definition of the IAddin interface:

public interface IAddin
{
void
            OnUnload();
void OnLoad();
}

The OnLoad() method gets executed whenever the host application loads the add-in. This allows an add-in to register the nodes and property sheets which will extend the Assembly Browser. The OnUnload() method gets executed whenever the host application unloads an add-in from memory. This will allow the add-in to perform any cleanup procedures that may be required.

ExplorerNode

The ExplorerNode class acts as the base class for all nodes which will appear in the Explorer tree view. This class has functionality which allows us to handle the expanding of nodes, as well as display popup menus.

Let's have a more detailed look at the various parts which makes up the ExplorerNode class:

protected bool HasChildren
{
get {
            return hasChildren; }
set
{
if (hasChildren !=
            value)
{
hasChildren = value;
if
            (hasChildren)
{
AddDummyNode();
}
else
{
Nodes.Clear();
}
}
}
}

The HasChildren property dictates whether a node has child nodes. Since child nodes are loaded on demand we need to specify upfront whether a node has any. If it does, we need to add a "dummy" node so that a + sign appears next to the node. Once the node is expanded the first time, the "dummy" node is removed and the actual child nodes are added to the node.

private void
            AddDummyNode()
{
Nodes.Clear();
Nodes.Add(new
            AssemblyBrowser.Extensibility.DummyExplorerNode());
}

The AddDummyNode method appends the actual dummy or placeholder node to the tree view. The first thing we do is to delete all existing child nodes from the node. Once that is done we add a new instance of the DummyExplorerNode class. This is a special class which inherits from ExplorerNode and is used only for the "dummy" node.

private bool LoadedOnce
{
get { return loadedOnce; }
}

Since child nodes are loaded on demand, we only want to call the procedure which loads child nodes once. The LoadedOnce property is used to indicate whether the nodes child nodes have already been loaded.

public void LoadNodes()
{
if
            (!LoadedOnce)
{
TreeView.Cursor =
            System.Windows.Forms.Cursors.WaitCursor;
TreeView.BeginUpdate();
try
{
RemoveDummyNode();
PerformLoadNodes();
}
finally
{
TreeView.EndUpdate();
TreeView.Cursor
            = System.Windows.Forms.Cursors.Default;
}
loadedOnce =
            true;
}
}

The LoadNodes method is called by the host application whenever the end user expands a tree node. The first task is to check whether the child nodes for this node have already been loaded. If it has been loaded we do nothing. If it has not been loaded, the first thing we do is remove our "dummy" node. Remember that this node acts as a sort of placeholder node until the actual child nodes for the node have been loaded. Next we call the PerformLoadNodes method. This is a virtual method which must be overridden by inheriting nodes to perform the actual loading of the child nodes.

While loading the child nodes we change the screen cursor to the busy cursor to give the end user an indication that something is happening. We also call the BeginUpdate method to disable redrawing of the tree view while we are loading the child nodes. This also improves the performance when adding items to the tree view. Once we have finished loading all the child nodes, we call the EndUpdate method to tell the tree view that it can resume painting.

The final thing to do is to set the private loadedOnce field. We only allow inherited classes read-only access to this field via the LoadedOnce property.

private void
            RemoveDummyNode()
{
foreach(System.Windows.Forms.TreeNode node
            in Nodes)
{
if (node is
            AssemblyBrowser.Extensibility.DummyExplorerNode)
Nodes.Remove(node);
}
}

The RemoveDummyNode method iterates through the list of child nodes, and removes all instances of the DummyExplorerNode.

private void
            Reset()
{
Nodes.Clear();

loadedOnce = false;

if
            (HasChildren)
AddDummyNode();
}

The Reset method is used to initialise an ExplorerNode to its default state. During the Reset method all the child nodes are deleted and the loadedOnce flag is cleared. If the node has child nodes, a dummy child node is added as well.

public void
            Refresh()
{
TreeView.BeginUpdate();
try
{
bool
            wasExpanded =
            IsExpanded;
this.Collapse();

Reset();

if
            (wasExpanded)
this.Expand();
}
finally
{
TreeView.EndUpdate();
}
}

The Refresh method is used to reload all child nodes. To refresh a node we call the Reset method which will restore the node to its initial state. An internal flag is maintained called wasExpanded, which is used to remember if the node was in an expanded state when the Refresh method was called. After the node is reset to its initial state, we re-expand the node if it was expanded to start off with. You will also note that one of the first things we do is to collapse the node. If we do not explicitly collapse the node, the IsExpanded property of the tree node does not get set to false, and the node does not expand correctly if we try to re-expand it. Once again all of this code is enclosed by calls to the BeginUpdate and EndUpdate methods of the tree view to prevent it from repainting while child nodes are deleted and added during the Refresh method.

protected bool HasDeleteMenu
{
get {
            return hasDeleteMenu; }
set { hasDeleteMenu = value;
            }
}

protected bool HasRefreshMenu
{
get { return
            hasRefreshMenu; }
set { hasRefreshMenu = value;
}
}

The HasDelete and HasRefresh properties are used to indicate if the context menu of the node must contain Delete and Refresh menu items. When the context menu for the node gets created, these two menu items may automatically get added based on the values of the HasRefreshMenu and HasDeleteMenu properties.

public void ShowMenu(System.Windows.Forms.Control control,
System.Drawing.Point pos)
{
System.Windows.Forms.ContextMenu contextMenu = CreateMenu();

if (contextMenu == null)
contextMenu = new System.Windows.Forms.ContextMenu();

if (HasDeleteMenu || HasRefreshMenu)
{
if (contextMenu.MenuItems.Count > 0)
{
contextMenu.MenuItems.Add("-");
}

if (HasDeleteMenu)
{
System.Windows.Forms.MenuItem deleteMenuItem = new
System.Windows.Forms.MenuItem(
"Delete",
new System.EventHandler(DeleteMenuClick));

contextMenu.MenuItems.Add(deleteMenuItem);
}

if (HasRefreshMenu)
{
System.Windows.Forms.MenuItem refreshMenuItem = new
System.Windows.Forms.MenuItem(
"Refresh",
new System.EventHandler(RefreshMenuClick));

contextMenu.MenuItems.Add(refreshMenuItem);
}
}

if (contextMenu.MenuItems.Count > 0)
contextMenu.Show(control, pos);
}

The ShowMenu method is called by the host application whenever the user right-clicks on a tree node. The application makes a call to the virtual CreateMenu method. Inherited nodes can override this method to create their own context specific menu items. Now check if the inherited nodes actually returned a context menu. If not, we create one. We check the HasDeleteMenu and HasRefreshMenu properties as discussed above, and create the Delete and Refresh menu items as appropriate. Additionally we check if other menu items already exist before we create the Delete and Refresh menu item. If there are other menu items we create a separator menu item.

You will notice that the OnClick event handlers of the Delete and Refresh menu items point to the DeleteMenuClick and RefreshMenuClick methods respectively. These will be discussed below. Lastly we check if any menu items were added to the context menu after all of this, and if so, display the context menu.

private void DeleteMenuClick(object sender,
            System.EventArgs e)
{
if
            (PerformDelete())
this.Remove();
}

protected virtual
            bool PerformDelete()
{
return false;
}

The DeleteMenuClick method is set as the OnClick event handler of the Delete menu item of the context menu. We call the virtual PerformDelete method that allows inherited nodes to perform their own deletion of the data that they represent. If the PerformDelete method returns true, we delete the actual tree node. By default the PerformDelete method returns false.

private void RefreshMenuClick(object sender,
            System.EventArgs e)
{
PerformRefresh();
}

protected
            virtual void
            PerformRefresh()
{
this.Refresh();
}

The RefreshMenuClick method is set as the OnClick event handler of the Refresh menu item of the context menu. It calls the virtual PerformRefresh method, which allows inherited nodes to perform their own processing when we refresh it. By default, the PerformRefresh method calls the Refresh method of the node.

public virtual System.Drawing.Icon
            Image
{
get { return null; }
}

public virtual
            System.Drawing.Icon SelectedImage
{
get { return null;
            }
}

The Image and SelectedImage properties are used to get the image to display next to a node. By default, these return a null value, but inherited nodes can override them to specify their images.

The IPropertySheet Interface

The IPropertySheet interface needs to be implemented by all property sheets.

public interface IPropertySheet
{
string Caption
{
get;
}

AssemblyBrowser.Extensibility.ExplorerNode ExplorerNode
{
get;
set;
}

void LoadData();
}

The Caption property must return the caption for a property sheet. This caption will be displayed as the text for the tab page on which the property sheet is displayed. The ExplorerNode property will be used to specify the node whose data the property sheet must display. Finally the LoadData method will be called to notify the property sheet that it must display its data.

The BaseAssemblyNode Class

The BaseAssemblyNode acts as the base class from which all explorer nodes in our assembly browser will inherit. It handles the displaying of nodes that were registered to add themselves automatically to the tree view. It also handles the execution of the callback methods which were registered by add-ins. We could have actually combined the ExplorerNode and BaseAssemblyNode functionality into one class for the purpose of this case study. Putting most of the functionality in the ExplorerNode class allows us to better re-use it in the future for other programs.

public System.Reflection.Assembly
            Assembly
{
get { return assembly; }
set
{
assembly
            = value;
InitializeNode();
}
}

The Assembly property allows us to keep track of a reference to the currently open assembly. Once a value is assigned to the Assembly property we call the virtual InitializeNode method to execute some initialization routines.

protected virtual void
            InitializeNode()
{
if (this.TreeView != null &&
            this.TreeView.ImageList != null)
{
int nodeIndex = -1;
int
            selectedNodeIndex = -1;

System.Drawing.Icon nodeImage =
            this.Image;
if (nodeImage !=
            null)
{
this.TreeView.ImageList.Images.Add(nodeImage);
nodeIndex
            = this.TreeView.ImageList.Images.Count - 1;
}
this.ImageIndex
            = nodeIndex;

System.Drawing.Icon nodeSelectedImage =
            this.SelectedImage;
if (nodeSelectedImage !=
            null)
{
this.TreeView.ImageList.Images.Add(nodeSelectedImage);
selectedNodeIndex
            = this.TreeView.ImageList.Images.Count - 1;
}
if
            (selectedNodeIndex >= 0)
this.SelectedImageIndex =
            selectedNodeIndex;
else
this.SelectedImageIndex =
            nodeIndex;
}
}

The InitializeNode method allows inherited nodes to perform their own initialization. By default, the InitializeNode method gets the images for the selected and unselected state of the node from the Image and SelectedImage properties. It adds these images to the ImageList component of the tree view, and sets the ImageIndex and SelectedImageIndex properties of the tree node to the indexes of the newly added images in the image list.

Please note: This routine is not optimized for efficient use of resources! As you can see it will just keep on adding images to the image list indefinitely. A future improvement might include a way to get the images for a node based on the node type, and not for each and every instance of a node.

protected System.Drawing.Icon GetImageFromResource(
System.Reflection.Assembly resourceAssembly, string resourceName)
{
System.IO.Stream iconStream = resourceAssembly.GetManifestResourceStream(
resourceName);

if (iconStream != null)
{
System.Drawing.Icon nodeIcon = new System.Drawing.Icon(iconStream);
return nodeIcon;
}
return null;
}

GetImageFromResource is a helper method to load the specified manifest resource from the assembly which is passed in as a parameter. The GetManifestResourceStream method of the assembly gives us the ability to return a data stream from which the icon is created.

protected override void PerformLoadNodes()
{
System.Type[] autoChildNodeTypes =
NodeRegistrationUtil.GetNodeTypes(this);
if (autoChildNodeTypes != null)
{
foreach(System.Type autoChildNodeType in autoChildNodeTypes)
{
if (autoChildNodeType.IsSubclassOf(
typeof(AssemblyBrowser.Extensibility.BaseAssemblyNode)))
{
AssemblyBrowser.Extensibility.BaseAssemblyNode autoNode =
(AssemblyBrowser.Extensibility.BaseAssemblyNode)
System.Activator.CreateInstance(autoChildNodeType);
this.Nodes.Add(autoNode);

autoNode.Assembly = this.Assembly;
}
}
}

AssemblyBrowser.Extensibility.NodeLoadEventHandler[] nodeLoadEventHandlers
= NodeRegistrationUtil.GetNodeLoadEventHandlers(this);
if (nodeLoadEventHandlers != null)
{
foreach(AssemblyBrowser.Extensibility.NodeLoadEventHandler
eventHandler in nodeLoadEventHandlers)
{
AssemblyBrowser.Extensibility.NodeLoadEventArgs eventArgs =
new AssemblyBrowser.Extensibility.NodeLoadEventArgs(this);
eventHandler(this, eventArgs);
}
}
}

The application overrides the base PerformLoadNodes method to allow for loading of registered nodes and the execution of node load event handlers.

A call to the GetNodeTypes method of the NodeRegistrationUtil class returns an array of System.Type for all the node types which we registered to load themselves under this node. We now iterate through the returned array and ensure that each of the types is a subclass of BaseAssemblyNode and call the CreateInstance method of the Activator class. This class is part of the Reflection API and the CreateInstance method is used to create an instance of a type. Once the new node is created, assign its Assembly property.

Now call the GetNodeLoadEventHandlers method of the NoderegistrationUtil class which will return an array of event handlers to execute when this node expands. Iterate through the returned array and execute each of the event handlers.

The AssemblyNode Class

Figure 1: An Assembly Node

The AssemblyNode class represents a specific assembly. This class does not really do much by itself, but represents the assembly which is currently loaded. Typically, all of our add-ins will register themselves to attach to this node.

protected override void
            InitializeNode()
{
base.InitializeNode();

HasChildren =
            true;
this.Text = Assembly.GetName().Name;
}

In the InitializeNode method of the AssemblyNode class, call the base InitializeNode method. We also set the HasChildren property to true, to specify that this node will have children. Finally set the Text property of the node to the name of the assembly.

public override System.Drawing.Icon
            Image
{
get
{
return
            GetImageFromResource(this.GetType().Assembly,
"AssemblyBrowser.Extensibility.Images.Assembly.ico");
}
}

public
            override System.Drawing.Icon SelectedImage
{
get
{
return GetImageFromResource(this.GetType().Assembly,
"AssemblyBrowser.Extensibility.Images.Assembly_Selected.ico");
}
}

Override the Image and SelectedImage properties to specify the icons display when the node is in the unselected and selected state.

The PropertyGridSheet Class

Figure 2: A property grid

The PropertyGridSheet class acts as the base class of all property sheets that display information about an object in a PropertyGrid. This class is basically a form which contains a PropertyGrid component and implements the IPropertySheet interface. For more information on the PropertyGrid control please refer to "The .NET Property Grid Control" (http://www.csharptoday.com/content.asp?id=1646&WROXEMPTOKEN=618658Zezi9SpDFSZgQps5Xv6u).

Besides the implementation of the IPropertySheet interface, it introduces a virtual method called GetPropertyGridObject. Derived classes or subclasses must override this method to supply the object that must be displayed in the property grid.

Developing the host application

Before we go into much more detail on each of the add-ins, let us first take a look the host application. The source code for the host application can be found in the AssemblyBrowser project.

Loading the add-ins

When the host application first starts up, the add-ins are loaded. The list of add-ins to load and their locations are specified in the application's configuration file. The structure of the configuration file is discussed later in the document under the section "The Application Configuration File".

private void LoadAddins()
{
try
{
System.Collections.IDictionary addinLocations =
(System.Collections.IDictionary)
System.Configuration.ConfigurationSettings.GetConfig("AddIns");

if (addinLocations != null)
{
foreach(System.Collections.DictionaryEntry addinLocation in
addinLocations)
{
LoadAddin((string)addinLocation.Value);
}
}
}
catch(Exception ex)
{
throw new System.ApplicationException(
"An error occurred while loading the addins", ex);
}
}

To get the list of add-ins from the application configuration file, we use the GetConfig method of the System.Configuration.ConfigurationSettings class. This will return a reference to an IDictionary which contains the settings for the configuration section which we specified. Now iterate through the list of items in the returned IDictionary, and call LoadAddin for each of the values.

private void LoadAddin(string location)
{
System.Reflection.Assembly addinAssembly =
System.Reflection.Assembly.LoadFrom(location);

if (addinAssembly != null)
{
System.Type[] addinTypes = addinAssembly.GetTypes();
foreach(System.Type addinType in addinTypes)
{
System.Type interfaceType =
addinType.GetInterface(
typeof(AssemblyBrowser.Extensibility.IAddin).FullName);
if (interfaceType != null)
{
AssemblyBrowser.Extensibility.IAddin addinInstance =
(AssemblyBrowser.Extensibility.IAddin)
System.Activator.CreateInstance(addinType);

addinInstance.OnLoad();

addins.Add(addinInstance);
}
}
}
}

The LoadAddin method loads the assembly for the add-in at the specified location. It then iterates through all the types in the assembly, and inspects each type to determine if it implements the IAddin interface. For each type that implements the IAddin interface, it creates a new instance of the type using the CreateInstance method of the Activator class. Finally it calls the OnLoad method of the add-in, and adds it to the list of add-ins which are loaded.

Opening an assembly

Figure 3: The Open menu item

public void OpenAssembly(string filename)
{
CloseAssembly();

currentAssembly = System.Reflection.Assembly.LoadFrom(filename);

AssemblyBrowser.Extensibility.AssemblyNode assemblyNode =
new AssemblyBrowser.Extensibility.AssemblyNode();
tvwBrowser.Nodes.Add(assemblyNode);
assemblyNode.Assembly = currentAssembly;

tvwBrowser.SelectedNode = assemblyNode;
}

The OpenAssembly method is used to open a new assembly. This code will be executed when the user selects the File | New menu item from the Assembly Browser user interface. When this option is selected, the application checks if it an assembly is open and if so it closes it. The selected assembly is then loaded by calling the static LoadFrom method of the System.Reflection.Assembly class. The LoadFrom method takes the name or path of the assembly to open as a parameter.

A new AssemblyNode is created and added to the explorer tree view and given focus.

Closing an assembly

Figure 4: The Close menu item

public void CloseAssembly()
{
if
            (currentAssembly !=
            null)
{
ClearPropertySheets();

tvwBrowser.Nodes.Clear();

imageList.Images.Clear();

currentAssembly
            = null;
}
}

The CloseAssembly method is executed when the user selects the File | Close menu item from the assembly user interface. This method clears (or deletes) all the property sheets and clears all the nodes from the tree view. Furthermore, it will clear all images from the image list. Finally we set the reference to the currently opened assembly to null.

Expanding a tree node

private void tvwBrowser_BeforeExpand(object sender,
System.Windows.Forms.TreeViewCancelEventArgs e)
{
if (e.Node is AssemblyBrowser.Extensibility.BaseAssemblyNode)
((AssemblyBrowser.Extensibility.BaseAssemblyNode)
e.Node).LoadNodes();
}

When the user expands a tree node, check to see if the node inherits from BaseAssemblyNode or one if its descendants. If it does, cast the node to a BaseAssemblyNode and call the LoadNodes method.

Displaying the property sheets for a node

private void tvwBrowser_AfterSelect(object sender,
System.Windows.Forms.TreeViewEventArgs e)
{
if (e.Node is AssemblyBrowser.Extensibility.BaseAssemblyNode)
LoadPropertySheets(
(AssemblyBrowser.Extensibility.BaseAssemblyNode)e.Node);
}

When the user selects a node in the explorer tree view, check to see if the node inherits from BaseAssemblyNode. If it does, call the LoadPropertySheets method and pass the node as a parameter.

public void LoadPropertySheets(
AssemblyBrowser.Extensibility.BaseAssemblyNode assemblyNode)
{
System.Windows.Forms.TabPage tabPage;

ClearPropertySheets();

System.Type[] propertySheetTypes =
AssemblyBrowser.Extensibility.NodeRegistrationUtil.
GetNodePropertySheets(assemblyNode);

if (propertySheetTypes != null)
{
foreach(System.Type propertySheetType in propertySheetTypes)
{
if (propertySheetType.GetInterface(
typeof(AssemblyBrowser.Extensibility.IPropertySheet).
FullName) != null)
{
AssemblyBrowser.Extensibility.IPropertySheet propertySheet =
(AssemblyBrowser.Extensibility.IPropertySheet)
System.Activator.CreateInstance(propertySheetType);

tabPage = new System.Windows.Forms.TabPage(propertySheet.Caption);
tabPropertySheets.TabPages.Add(tabPage);

if (propertySheet is Form)
{
((Form)propertySheet).FormBorderStyle = FormBorderStyle.None;
((Form)propertySheet).TopLevel = false;
((Form)propertySheet).Show();
}

tabPage.Controls.Add((Control)propertySheet);
((Control)propertySheet).Dock = DockStyle.Fill;

propertySheet.ExplorerNode = assemblyNode;
propertySheet.LoadData();
}
}
}

tabPropertySheets.Visible = (tabPropertySheets.TabPages.Count > 0);
}

The LoadPropertySheets method gets the list of property sheets for the node by calling the GetNodePropertySheets method of the NodeRegistrationUtil class. This will return an array of System.Type for the types of the property sheets to display for the node.

Iterate through the list of property sheet types to ensure that they implement the IPropertySheet interface. If it does, create a new instance of the property sheet. Then create a new tab page for the tab control and set the Text of the tab page to the Caption of the property sheet.

If the property sheet is a form (which it will be most of the time), set the BorderStyle to None, and set the TopLevel property to false. We need to set TopLevel to false, else we will not be able to add the property sheet to the controls collection of the tab page. Finally we call the Show method so the form can display itself.

The property sheet is then appended to the list of controls which is contained by the tab page, and its Dock property is set to DockStyle.Fill to fill the entire client area of the tab page. Assign the ExplorerNode property of the property sheet and call its LoadData method to allow it to display the data for the node.

Finally check if the selected node does indeed contain any property sheets and if it doesn't, simply hide the tab control by setting its Visible property to false.

Displaying the context menu of a node

private void tvwBrowser_MouseDown(object
            sender,
System.Windows.Forms.MouseEventArgs e)
{
if
            (e.Button == MouseButtons.Right)
{
// Get the node at the
            specified point
TreeNode node = tvwBrowser.GetNodeAt(e.X,
            e.Y);

if (node != null)
{
// Select the
            node
tvwBrowser.SelectedNode = node;

// Display the
            context menu
if (node is
            AssemblyBrowser.Extensibility.BaseAssemblyNode)
{
((AssemblyBrowser.Extensibility.BaseAssemblyNode)node).
ShowMenu(tvwBrowser,
            new System.Drawing.Point(e.X, e.Y));
}
}
}
}

When the user clicks on a node in the explorer tree view, check to see if it was the right mouse button which was clicked. If this is true, we get the tree node at the specified X and Y coordinates, and select the node. Next, check to see if the node inherits from BaseAssemblyNode. If it does, call the ShowMenu method of the node and pass the tree view and coordinates as parameters.

The application configuration file

As mentioned above, the list of add-ins are specified in the application configuration file. The configuration file is located in the same directory as the host application and has the same name, but with a .config extension added to the filename. In our case, this file is called AssemblyBrowser.exe.config. An example of this file can be found with the accompanying source code and executable application. We specify the list of add-ins under the Addins section of the configuration file. You can have a look at the files which accompany this case study to look at the contents of the configuration file, and set the locations of the add-ins to the proper locations as per your own directory structure on your computer.

For more information on application configuration files, please refer to the .NET Framework documentation.

The Modules Add-in

The Modules add-in displays a list of all the modules which are contained in an assembly, and all the types which are defined in each module. All the source code for the Modules add-in can be found in the Modules assembly.

ModuleNode

Figure 5: A module node

The ModuleNode class represents a module which is contained in the assembly.

public System.Reflection.Module
            Module
{
get { return module; }
set
{
module =
            value;
this.Text = module.Name;
}
}

The Module property contains the reference to the module which is represented by this node. When a value is assigned to the Module property, we set the Text property of the node to the name of the module.

protected override void
            InitializeNode()
{
base.InitializeNode();

HasChildren =
            true;
HasRefreshMenu = true;
}

In the InitializeNode method of the Module class we indicate that the node has children and that is has a Refresh menu item.

protected override void
            PerformLoadNodes()
{
base.PerformLoadNodes();

System.Type[]
            moduleTypes = module.GetTypes();
foreach(System.Type moduleType
            in moduleTypes)
{
TypeNode typeNode =
            TypeNodeBuilder.CreateTypeNode(moduleType);
if (typeNode !=
            null)
{
this.Nodes.Add(typeNode);
typeNode.Assembly =
            this.Assembly;
}
}
}

The PerformLoadNodes method of the Module class loads all the types which are contained in the module. These types can either be a class, interface, struct, enum, or delegate. To get the list of types contained in the module, we call the GetTypes method of the module field which contains a reference to an instance of the System.Reflection.Module class. This will return an array of System.Type objects, which contains a list of the types in the module. We iterate through the list of types to add them to the ModuleNode.

Each of the types are represented by a node which inherits from TypeNode. In order to create the correct TypeNode, we call the CreateTypeNode method of the TypeNodeBuilder class. We then append the TypeNode we just created to list child nodes for this node, and assign its Assembly property.

The TypeNodeBuilder Class

The TypeNodeBuilder class is just a helper class which we use to create the correct descendant class of the TypeNode class depending on the actual type. This class only has one static method, called CreateTypeNode, which return the correct TypeNode based on the type which is passed to it.

public static TypeNode
            CreateTypeNode(System.Type type)
{
if
            (type.IsClass)
{
if (type.BaseType ==
            typeof(System.MulticastDelegate))
{
DelegateTypeNode
            delegateTypeNode = new
            DelegateTypeNode();
delegateTypeNode.MemberInfo =
            type;

return
            delegateTypeNode;
}
else
{
ClassTypeNode classTypeNode =
            new ClassTypeNode();
classTypeNode.MemberInfo =
            type;

return classTypeNode;
}
}
else if
            (type.IsEnum)
{
EnumTypeNode enumTypeNode = new
            EnumTypeNode();
enumTypeNode.MemberInfo = type;

return
            enumTypeNode;
}
else if
            (type.IsInterface)
{
InterfaceTypeNode interfaceTypeNode = new
            InterfaceTypeNode();
interfaceTypeNode.MemberInfo =
            type;

return interfaceTypeNode;
}
else if
            (type.IsValueType)
{
StructureTypeNode structTypeNode = new
            StructureTypeNode();
structTypeNode.MemberInfo =
            type;

return
            structTypeNode;
}
else
{
UnknownTypeNode unknownTypeNode
            = new UnknownTypeNode();
unknownTypeNode.MemberInfo =
            type;

return unknownTypeNode;
}
}

To create the correct TypeNode we interrogate the attributes of the type which was passed in. If it is a class (the IsClass property of the type is true) and it inherits from MultiCastDelegate, we can safely assume that this is a delegate and create a DelegateTypeNode. Otherwise, if it is a class, we create a ClassTypeNode.

If the IsEnum property of the type is true, we create an EnumTypeNode. If the IsInterface property of the type is set to true, we create an InterfaceTypeNode. If the IsValueType property of the type is true, we can assume that this is a struct, as all other value types are accounted for already, so we create a StructTypeNode. If none of the above cases were true, it is a type which we do not cater for, so we create a special type of node called an UnknownTypeNode to indicate to the user that it is an unknown type.

The MemberInfoNode Class

The MemberInfoNode class represents a System.Reflection.MemberInfo class. All the tree nodes which represents a type node inherit from MemberInfoNode. Below is the class hierarchy of all the type nodes along with the reflection class that it maps to.

Based on the hierarchy above, lets have a quick look at the source code for each of these classes.

The TypeNode Class

The TypeNode class acts as a base class for all the classes which represents a descendant of System.Type. This includes classes, interfaces, structs, delegates and enums.

public class TypeNode : MemberInfoNode
{
public System.Type Type
{
get { return MemberInfo as System.Type; }
}
}

The TypeNode class has only one property called Type. This returns a reference to the System.Type that is represented by the class.

The ContainerTypeNode Class

The ContainerTypeNode class acts as the base class for all classes which contains other members, such a fields, methods, or properties. The ContainerTypeNode overrides the PerformLoadNodes method which implements the ability to load members. Classes which inherit from ContainerTypeNode are ClassTypeNode, InterfaceTypeNode, and StructureTypeNode.

protected override void PerformLoadNodes()
{
base.PerformLoadNodes();

System.Reflection.MemberInfo[] fields = Type.GetMembers(
System.Reflection.BindingFlags.Instance |
System.Reflection.BindingFlags.DeclaredOnly |
System.Reflection.BindingFlags.NonPublic |
System.Reflection.BindingFlags.Public |
System.Reflection.BindingFlags.Static
);

foreach(System.Reflection.MemberInfo memberInfo in fields)
{
if (memberInfo is System.Reflection.FieldInfo)
{
System.Reflection.FieldInfo fieldInfo =
(System.Reflection.FieldInfo)memberInfo;

FieldNode fieldNode = new FieldNode();
this.Nodes.Add(fieldNode);

fieldNode.Assembly = this.Assembly;
fieldNode.MemberInfo = fieldInfo;
}
else if (memberInfo is System.Reflection.PropertyInfo)
{
System.Reflection.PropertyInfo propertyInfo =
(System.Reflection.PropertyInfo)memberInfo;

PropertyNode propertyNode = new PropertyNode();
this.Nodes.Add(propertyNode);

propertyNode.Assembly = this.Assembly;
propertyNode.MemberInfo = propertyInfo;
}
else if (memberInfo is System.Reflection.MethodBase)
{
System.Reflection.MethodBase methodInfo =
(System.Reflection.MethodBase)memberInfo;

if (!methodInfo.IsSpecialName || methodInfo.IsConstructor)
{
MethodNode methodNode = new MethodNode();
this.Nodes.Add(methodNode);

methodNode.Assembly = this.Assembly;
methodNode.MemberInfo = methodInfo;
}
}
}
}

The PerformLoadNodes method gets a list of all the members for the underlying type. We do this by calling the GetMembers method of System.Type, which will return an array of MemberInfo objects for the specified type. Once a list of all the members is returned, iterate through it and create its appropriate node accordingly. For fields we create a FieldNode, for properties we create a PropertyNode, and for methods we create a MethodNode. What is important is that we exclude methods where the IsSpecialName property of the MemberInfo object is true. An example of the special methods that we exclude is the get and set methods of properties.

The ClassTypeNode, InterfaceTypeNode and StructureTypeNode Classes

As can be seen in the diagram above the ClassTypeNode, InterfaceTypeNode and StructureTypeNode all inherit from ContainerTypeNode and therefore have the ability to contain fields, methods and properties. The only property which we override in these classes is the Image property, which returns the correct icon for the node.

The remaining nodes in the Modules add-in

None of the rest of the nodes in the modules add-in adds much more functionality. They all only override the Image property which returns the correct icon for the node.

The ModulePropertySheet Class

The ModulePropertySheet class is inherited from PropertyGridSheet, and overrides the GetPropertyGridObject method.

protected override object
            GetPropertyGridObject()
{
if (ExplorerNode is
            ModuleNode)
{
return
            ((ModuleNode)ExplorerNode).Module;
}

return
            null;
}

Since the ModulePropertySheet class is defined as the property sheet class of the ModuleNode, the GetPropertyGridObject method which the ModulePropertySheet class overrides checks to see if the object referenced by the ExplorerNode property inherits from ModuleNode. If it does, it returns the Module property of the ModuleNode object.

MemberInfoPropertySheet

The MemberInfoPropertySheet class is inherited from PropertyGridSheet, and overrides the GetPropertyGridObject method.

protected override object
            GetPropertyGridObject()
{
if (ExplorerNode is
            MemberInfoNode)
{
return
            ((MemberInfoNode)ExplorerNode).MemberInfo;
}

return
            null;
}

The GetPropertyGridObject method which the MemberInfoPropertySheet class overrides checks to see if the object referenced by the ExplorerNode property inherits from MemberInfoNode. If it does, it returns the MemberInfo property of the MemberInfoNode object.

MethodParametersPropertySheet

The MethodParametersPropertySheet class displays the parameters of a method in a grid. The property sheet contains a DataGrid component in which the name and type of all of the parameters of the method are displayed.

void AssemblyBrowser.Extensibility.IPropertySheet.LoadData()
{
if (explorerNode is MethodNode)
{
System.Collections.ArrayList parameterList =
new System.Collections.ArrayList();

System.Reflection.MethodBase methodInfo =
((MethodNode)explorerNode).MethodInfo;
System.Reflection.ParameterInfo[] parameters =
methodInfo.GetParameters();

foreach(System.Reflection.ParameterInfo parameter in parameters)
{
parameterList.Add(new MethodParameter(
parameter.Name,
parameter.ParameterType.ToString()));
}

dataGrid.DataSource = parameterList;
}
}

The LoadData method first checks to see if the ExplorerNode inherits from MethodNode. If this is true, retrieve the parameters of the method by calling the GetParameters method of the MemberNode's MethodInfo property. This returns an array of ParameterInfo objects. Next we iterate through the array of parameters. For each parameter we create an instance of the internal MethodParameter class and add it to the internal array list which contains the parameters. Finally we set the DataSource property of the DataGrid to point to the array list containing the MethodParameters objects. This will display all the parameters in the grid.

The Modules add-in class

The ModulesAddin class implements the IAddin interface. This class is responsible for registering the nodes and property sheets. In the IAddin.OnLoad method of the ModulesAddin class, we register a node load event handler for the AssemblyNode class called LoadModuleNodes. This means that when the user expands a tree node that is an instance of the AssemblyNode class, the event handler which we specified is executed.

public void LoadModuleNodes(object sender, NodeLoadEventArgs e)
{
if (e.ParentNode is AssemblyBrowser.Extensibility.AssemblyNode)
{
AssemblyBrowser.Extensibility.AssemblyNode assemblyNode =
(AssemblyBrowser.Extensibility.AssemblyNode)e.ParentNode;
System.Reflection.Module[] assemblyModules =
assemblyNode.Assembly.GetModules(false);

foreach(System.Reflection.Module assemblyModule in assemblyModules)
{
ModuleNode moduleNode = new ModuleNode();
assemblyNode.Nodes.Add(moduleNode);

moduleNode.Assembly = assemblyNode.Assembly;
moduleNode.Module = assemblyModule;
}
}
}

The LoadModuleNodes method gets a list of all the modules in the assembly. It does this by executing the GetModules method of the Assembly class. This returns an array containing Module objects representing the modules in the assembly. Next we iterate through the array of modules and create ModuleNode objects for each of them.

The rest of the ModulesAddin class implements the remaining methods and properties which are specified by the IAddin interface.

The References Add-in

The References add-in displays a list of all the assemblies which are referenced by the assembly. All the source code for the References add-in can be found in the References project.

The ReferencesNode Class

Figure 6: A References node

The ReferencesNode is a container node below which all the referenced assemblies are displayed.

protected override void
            InitializeNode()
{
base.InitializeNode();

HasChildren =
            true;
Text = "References";
}

The InitializeNode method of the ReferencesNode class specifies that the node will have child nodes and it sets the Text property of the node to display "References".

protected override void
            PerformLoadNodes()
{
base.PerformLoadNodes();

System.Reflection.AssemblyName[]
            assemblyNames =
Assembly.GetReferencedAssemblies();
foreach(System.Reflection.AssemblyName
            assemblyName in assemblyNames)
{
ReferencedAssemblyNode
            referencedAssemblyNode = new
ReferencedAssemblyNode();
Nodes.Add(referencedAssemblyNode);

referencedAssemblyNode.Assembly
            = Assembly;
referencedAssemblyNode.AssemblyName =
            assemblyName;
}
}

The PerformLoadNodes method retrieves a list of all the assemblies which are referenced by the assembly. It does this by calling the GetReferencedAssemblies method of the Assembly class. This returns an array of AssemblyName objects, containing the information about each referenced assembly. Next we iterate through the array of AssemblyName objects, and create a ReferencedAssemblyNode class for each of them.

The ReferencedAssemblyNode Class

Figure 7: A referenced assembly node

The ReferencedAssemblyNode class represents an AssemblyName object for an assembly which is referenced by the assembly.

public System.Reflection.AssemblyName
            AssemblyName
{
get { return assemblyName; }
set
{
assemblyName = value;
Text = assemblyName.Name;
}
}

The AssemblyName property allows us to specify the AssemblyName object which is referenced by the ReferencedAssemblyNode. When we assign the AssemblyName property, we set the Text property of the node to the Name property of the AssemblyName object.

The ReferencePropertySheet class

The ReferencePropertySheet class is inherited from PropertyGridSheet, and overrides the GetPropertyGridObject method.

protected override object
            GetPropertyGridObject()
{
if (ExplorerNode is
            ReferencedAssemblyNode)
{
return
            ((ReferencedAssemblyNode)ExplorerNode).AssemblyName;
}

return
            null;
}

The GetPropertyGridObject method which the ReferencePropertySheet class overrides checks to see if the object referenced by the ExplorerNode property inherits from ReferencedAssemblyNode. If it does, it returns the AssemblyName property of the ReferencedAssemblyNode object.

The ReferencesAddin Class

The ReferencesAddin class implements the IAddin interface. This class is responsible for registering the nodes and property sheets.

void IAddin.OnLoad()
{
NodeRegistrationUtil.RegisterNodeType(
typeof(ReferencesNode),
typeof(AssemblyBrowser.Extensibility.AssemblyNode));

NodeRegistrationUtil.RegisterNodePropertySheet(
typeof(ReferencedAssemblyNode),
typeof(ReferencePropertySheet));
}

The IAddin.OnLoad method of the ReferencesAddin class registers the ReferencesNode to automatically be displayed below the AssemblyNode class when it is expanded. It also registers the ReferencePropertySheet class as a property sheet for the ReferencedAssemblyNode class.

The Resources Add-in

The Resources add-in displays a list of all the resources contained by the assembly. All the source code for the Resources add-in can be found in the Resources assembly.

The ResourcesNode Class

Figure 8: A resources node

The ResourcesNode acts as a container node below which all the resources contained in the assembly are displayed.

protected override void
            InitializeNode()
{
base.InitializeNode();

HasChildren =
            true;
HasRefreshMenu = true;
Text =
            "Resources";
}

The InitializeNode method specifies that the node has child nodes and the context menu contains a Refresh menu item. It also sets the Text property of the node to display "Resources".

protected override void
            PerformLoadNodes()
{
base.PerformLoadNodes();

string[]
            resourceNames =
            Assembly.GetManifestResourceNames();
foreach(string resourceName
            in resourceNames)
{
ResourceNode resourceNode = new
            ResourceNode();
Nodes.Add(resourceNode);

resourceNode.Assembly
            = Assembly;
resourceNode.ResourceName =
            resourceName;
}
}

The PerformLoadNodes method loads all the resources contained in the assembly. To get the list of resources we call the GetManifestResourceNames method of the assembly. This returns an array of strings with the names of all of the resources. We iterate through the array of names and create a ResourceNode for each resource.

The ResourceNode Class

Figure 9: A resource node

The ResourceNode class represents a resource which is contained in the assembly.

public System.Reflection.ManifestResourceInfo ResourceInfo
{
get
{
if (Assembly != null)
return Assembly.GetManifestResourceInfo(ResourceName);

return null;
}
}

The ResourceInfo property returns a ManifestResourceInfo object which contains manifest resource topology information. We get this information by calling the GetManifestResourceInfo method of the assembly.

protected override System.Windows.Forms.ContextMenu CreateMenu()
{
System.Windows.Forms.ContextMenu contextMenu =
new System.Windows.Forms.ContextMenu();

// Add the 'Export' menu item
contextMenu.MenuItems.Add("Export...");

return contextMenu;
}

The CreateMenu method is overridden in order to illustrate how easy it is to add a menu item to the context menu of a node. To do this we create a new ContextMenu object. Then we add a menu item to the context menu which displays the text "Export...".

The ResourcesAddin Class

The ResourcesAddin class implements the IAddin interface. This class is responsible for registering the nodes and property sheets.

void IAddin.OnLoad()
{
NodeRegistrationUtil.RegisterNodeType(
typeof(ResourcesNode),
typeof(AssemblyBrowser.Extensibility.AssemblyNode));

NodeRegistrationUtil.RegisterNodePropertySheet(
typeof(ResourceNode),
typeof(ResourcePropertySheet));
}

The IAddin.OnLoad method of the ResourcesAddin class registers the ResourcesNode to automatically be displayed below the AssemblyNode class when it is expanded. It also registers the ResourcePropertySheet class as a property sheet for the ResourceNode class.

Case Study Review

In this case study we covered the following:

  • Background information about the different customization methods which are available to developers if they want to make their applications extensible.
  • Background information about the format and structure of .NET assemblies and how we can use reflection to query the metadata of an assembly.
  • The development of an add-in framework which can be implemented in applications and will enable applications to be extended through the utilization of add-ins.
  • Putting together all of the information above, we developed an Assembly Browser application which allows users to view the metadata of any .NET assembly.

During the discussion of the development of the add-in framework and the Assembly Browser, I touched on all of the most important aspects of the source code. It is important for developers to understand the methods which they can use to open up their applications for customization by external developers. I would urge you however to explore the source code which accompanies this article in detail, so that you can gain a better understanding of the underlying technologies and hopefully adapt it so you can implement it in your own projects.

Case Study Information
Author | Jerrie Pelser
Technical Editor | Adam Ryland
Project Manager | Helen Cuthill
Reviewers | Dinar Dalvi, John Hodgson, Shefali Kulkarni

If you have any questions or comments about this case study, or suggestions for further pieces, please contact the technical editor. If you wish to talk about the case study with other subscribers, please visit our forum here.


Building a List Server

Building a List Server
http://www.csharptoday.com/content.asp?id=1763
Article Rating: not yet rated
Published on April 8, 2002
Abstract
In this case study, Matthew Reynolds takes a look at how we can build a simple list server. This list server is designed to handle “jobs” of several thousand e-mails, and handle returned and bounced e-mails. We will see how to build a single Windows Service application that we can put somewhere on our network and allow several computers on the network to queue messages for distribution. For message distribution, we’ll use Microsoft SMTP Service. However, we’ll actually design the system so that we can use a different service for message distribution. For example, we’ll create an “engine” that rather than connecting to the Microsoft SMTP Service through the System.Web.Mail namespace, uses Exchange or perhaps some third-party component.
Case Study

In this case study, I'm going to be taking a look at how we can build a simple list server. This list server is designed to handle "jobs" of several thousand e-mails, and handle returned and bounced e-mails.

What we want to be able to do is build a single Windows Service application that we can put somewhere on our network and allow several computers on the network (or perhaps from outside the firewall on another network or the Internet) to queue messages for distribution. We're not going to concern ourselves with maintaining a database containing the potential recipients of the e-mails, or tools to query this database and ultimately send e-mails to them - we'll simply create an endpoint that we can send the messages to and have our application send them.

For message distribution, we'll use Microsoft SMTP Service that comes as standard with various flavors of Windows 2000. However, we'll actually design the system so that we can use a different service for message distribution. For example, we'll create an "engine" that rather than connecting to the Microsoft SMTP Service through the System.Web.Mail namespace, uses Exchange or perhaps some third-party component like the ones that have been in common use by ASP developers for a while now, like ASPEmail and JMail.

For message return, we have a number of options. Typically, should a message be rejected, a message will be sent to the address specified as the sender in the originating message, for example, bounce@wrox.com. We can pick this message up using a POP3 component, and we could either write one of these ourselves, or reuse one. In this particular example, we're going to be using a third-party component. (The one we're going to use offers a free trial, although you'll find a mix of free and paid components on the market. We'll discuss this in more detail when we get to the point in the exercise where we need to handle incoming e-mails).

In terms of designing the actual service itself, as we've already said, what we need is a known endpoint that we can send messages to where they will be queued and ultimately sent on their way to the final destination. Those of you thinking in a typical .NET fashion may at this point think "Web service". What I'm keen to do is to skirt around this a little and make our approach capable of supporting a Web service, but actually design it from a perspective of using Remoting rather than Web services.

Remoting vs. Web Services

While Web services is a cool technology that should (in theory) provide ultimate platform interoperability - meaning in this case that UNIX clients could queue messages to our Windows list server - that interoperability does come at a price. Having to package every call in SOAP, which involved numerous type conversions that are particularly wasteful when turning integer values into text values and back again, and having to work over HTTP, while quick is not as efficient as it could be. If we're confident that both our client and server are both going to be implemented on Windows, or specifically .NET on Windows, we can do this more efficiently.

This is where Remoting comes in. Remoting and Web services share a common code base in the Framework, and you can think of Remoting as being an extensible, proprietary object distribution technology that does a similar thing to Web services. You expose an object on the network and clients can connect over that network to call methods on that object.

Remoting, however, is not tied to the "SOAP over HTTP" paradigm that Web services are. We can, if we want, send a stream of binary data over a TCP socket. This is substantially more efficient than Web services because the fact we're sending binary data means we don't have to convert the values in order to get them into an XML document, and the fact that we're using TCP means we don't get the overhead of HTTP. What we do lose with "Binary over TCP" is interoperability. Sending binary data in this way assumes that the client machine represents data internally in the same manner that the server machine does. This may not be true, and is in fact why Web services works so well.

Imagine you have a number represented internally by Computer A. This computer will take a certain amount of memory (say two, four, eight bytes, whatever really) to represent the number and write a string of binary digits into this block. If you take this "pattern" from Computer A and move it to Computer B that is the same platform (e.g. two Windows computers), the number represented remains constant. However, if Computer C is a different platform, the pattern may not represent the same number. To get around this problem, SOAP calls for the numbers to be converted to text, for which common standards exist. Or rather, the common standard doesn't matter, as each platform will choose its own internal data representation standard for its own ends.

What's important here is that although we're saying that we're not going to use Web services, we actually can. Web services are a superb technology for gluing an alternative access layer on top of an existing layer. What we can do is create a separate Web service that sits on top of the Remoting architecture that enables other platforms to use our system.

Here's a diagram that represents where we're trying to get to:

The actual message transfer and delivery outside of the private network through the firewall isn't of any concern to us. We'll rely on other products to actually do this for us - i.e. Microsoft SMTP Service to distribute the messages to the remote SMTP server; the remote SMTP server will deal with distributing the messages to the recipients, or return it to the bounce mailbox; the bounce mailbox will be managed by another mail server application, and so on.

The Object Model

What we're looking to do is build a single class library that we can include in our client applications that has the capability of connecting to the remote service (although we say "remote", the service could be on the same computer). Each client application will be able to create a new "job" (which is a collection of outbound messages - such as one customer service e-mail, or 10,000 newsletter messages), and once the job has been created that application will be able to queue messages to that job. At some point in the future, once all of the messages have been queued, either the client application or another application will be able to say, "OK, process the job", at which point all of the queued messages will be sent on their way.

Here's a list of some of the classes that we'll need:

  • Service -one instance of this class will exist inside of the Windows Service process (when we're debugging, we'll be using a Console Application, not a Windows Service). This class is responsible for setting up Remoting and managing jobs.
  • ServiceConnector - this is the Remoting endpoint that the client applications will connect to. It enables them to create new jobs.
  • Job - this class will be responsible for managing a job list. Among others, it contains a method called Enqueue that is used to queue the message.
  • Message - this class will contain the message that will be sent.
  • Recipient - this class represents a name and e-mail address pair.
  • Configuration - this class contains the configuration of the service.
  • SelfSerializer - a helper class that enables a class to serialize itself to and de-serialize itself from an XML document. We'll use this to persist a couple of the other classes (specifically Message and Configuration) to disk.

Queue Structure

What we haven't looked at so far is how we will actually store the messages. There are a few approaches that we can use for this. The method that I'm going to suggest involves creating a file and folder structure on disk. In your case, it might be appropriate to use a database like SQL Server as the message store, or even Microsoft Exchange Server.

When a new job is created, a new folder will be created to contain messages for that job. As messages are queued, an XML document representing the message will be saved to the job folder.

Building the Service

To kick off, we'll build the ListServer project. This project will contain all of the classes that we need to support the service, and also to connect to the service from a client application.

The first thing we need to do is get the file structure on disk configured, and also to get the ServiceConnector object made available over Remoting. This information will be stored in a configuration file, which not surprisingly we'll manage through a class called Configuration. This class will contain fields to hold the basic Remoting information and some other configuration information.

We're going to hold this information in an XML document. What's more, we're going to give the Configuration class the capability of saving and loading this document whenever it's appropriate to do so. A pattern that I recommend for doing this is to create a new class capable of performing serialization and de-serialization and extending that class with specific "document" classes. In this application, the base class is called SelfSerializer, and we'll derive Configuration from this. The pattern also calls for a static Load method (which you can consider to be similar to a constructor, but one that takes a filename. Using a separate method called Load is more intuitive to developers consuming your class than having a curious looking constructor that appears to take a filename) and an instance Save method.

This class also needs to be decorated with the System.SerializableAttribute attribute:

[Serializable()] public class
            SelfSerializer
{
public SelfSerializer()
{
}

SelfSerializer will use System.Xml.Serialization.XmlSerializer to do its magic. We'll also need the System.IO namespace to provide file I/O functionality, so add these namespaces to SelfSerializer:

using System;
using System.IO;
using System.Xml.Serialization;

The Save method is pretty straightforward. We've created two separate versions of the method : one that takes a filename and one that takes a System.IO.Stream object. In the filename version, we need to create a System.IO.FileStream and pass the new stream to the alternate version of the method:

// Save - save the file to disk...
public
            void Save(string filename)
{
// file...
FileStream stream =
            null;
try
{
// open...
stream = new FileStream(filename,
            FileMode.Create);
Save(stream);
}
finally
{
if(stream
            != null)
stream.Close();
}
}

XmlSerializer really is very easy to use! All we need is a System.Type object representing whatever it is we're trying to serialize and then call the Serialize method:

public void Save(Stream stream)
{
// serialize it...
XmlSerializer serializer = new XmlSerializer(this.GetType());
serializer.Serialize(stream, this);
}

The Load method is a little more complicated. When we create XmlSerializer we need to provide a Type object that tells it how to interpret the file. However, from inside a static method, we don't have access to the GetType method as we do in Save. What this means is that we need to create protected methods that take the Type that we want to use and create a static method inside of the derived class. Here are the Load methods:

// Load - we don't know the class, so we
            need that...
protected static object Load(Type type, string
            filename)
{
// file...
FileStream stream =
            null;
try
{
// open the stream...
stream = new
            FileStream(filename, FileMode.Open);
return Load(type,
            stream);
}
finally
{
if(stream !=
            null)
stream.Close();
}
}

protected static object
            Load(Type type, Stream stream)
{
// deserialize
            it...
XmlSerializer serializer = new
            XmlSerializer(type);
return
            serializer.Deserialize(stream);
}

Create the new Configuration class and derive it from SelfSerializer. Add fields to support the list owner and the Remoting details (ports and "application name"). Don't worry too much about the port numbers that I've chosen here. They're arbitrary, but be advised that ports are "free for all" on a particular computer, so some other service may already be running on that port, in which case you'll need to select another.

public class Configuration : SelfSerializer
{
// fields...
public string ListOwnerAddress = "bounce@wrox.com";
public int TcpPort = 8000;
public int HttpPort = 8001;
public string ApplicationName = "WroxListServer";

When we call the Load method, we'll give it the name of a config.xml file. For our purposes, we're going to assume that this file always exists in the c:\Wrox ListServer folder. If we attempt to load this file and the folder doesn't exist, we'll need to create it. Likewise, if the file doesn't exist, we'll need to create a default version of it - which is why we have default values stored against each of the members in the Configuration file.

Here's the implementation of Load that needs to be added to Configuration:

// Load - de-serialize the configuration
            from disk...
public static Configuration Load(string
            filename)
{
Configuration config = null;

// do we have
            this file?
FileInfo info = new
            FileInfo(filename);
if(info.Directory.Exists ==
            false)
info.Directory.Create();
if(info.Exists ==
            false)
{
// create a new configuration file...
config = new
            Configuration();

// save the
            file...
config.Save(filename);
}
else
config =
            (Configuration)Load(typeof(Configuration),
filename);

//
            return it...
return config;
}

Inside of the actual service process, we're going to have one instance of Service running. Primarily, the responsibility of this object is to manage the queues, but it also performs some ancillary functions such as spinning up the threads to support bounced e-mail collections. The constructor for Service needs to accept a Configuration instance. In addition to this, we also need to be able to get at the current, active instance of Service from anywhere else inside the process, which we'll do through a static property called CurrentService.

public class Service
{
// fields...
private static Service _currentService;
private Configuration _configuration;

public Service(Configuration configuration)
{
// store...
_currentService = this;
_configuration = configuration;
}

public static Service CurrentService
{
get
{
return _currentService;
}
}

public Configuration Configuration
{
get
{
return _configuration;
}
}
}

Exposing "ServiceConnector" Over Remoting

When we want to start the service running, so that we can create jobs and ultimately queue and distribute messages, we'll load the configuration file from disk, create a new instance of Service from this, and call the StartService method. At this stage, all we want this to do is establish the Remoting channels and make the ServiceConnector class available.

Before we do that, let's go into a little more depth with Remoting. For Remoting, we need two things: a channel and a formatter. A channel defines how the Remoting message will be transferred (this is a different "message" from the e-mail messages that we've been talking about so far. Think of it as a package containing the details of the remote method and the parameters). We've already mentioned "HTTP" and "TCP" in our earlier discussions. These are two examples of channels. A formatter defines how the message will be structured. We've already spoken about these too - we have "SOAP" and "Binary" provided as standard with the Framework.

The other consideration that we have is that of object creation and lifetime. When we're using objects with Remoting, there are three different kinds of object creation: singleton, single call and client activated. Client-activated objects are used in situations where you need to specifically create an instance of an object on a remote machine, which if you bear with me is not what we actually need to do.

The type of Remoting object that we want is "singleton". This object is designed for highly scalable use. What happens is that the first inbound request for a singleton object causes an instance of the object to be created. This single object instance is shared across all requests from that point until some point in the future when the object is closed down. If 10,000 clients all make a request to methods on that object, only one instance of the object is ever created. This is great from a scalability perspective. It's important that this kind of object does not hold state, as this can cause all sorts of scalability and thread synchronization problems.

This is why we have a separate ServiceConnector object. It's this object that is actually registered with Remoting and it's responsible for finding and deferring the call onto a method on the single instance of Service that actually controls the service. If you can imagine, we don't want to control the lifetime of this singleton object - we'd rather Remoting does this for us. When an instance is created to support an inbound call, we don't care what instance is actually used to do this. What we do care about is finding the all-important Service instance.

Single-call is the final type of Remoting object. This type of object exists for a "single call" - i.e. it is created to support an inbound call and it is then disposed of after that call. Not as good from a scalability perspective as we either have zero or some unknown number of objects taking up memory, rather than a singleton approach where we either have zero or one.

For both singleton and single-call objects, we must inherit from System.MarshalByRefObject. This is a special class that tells Remoting that the object should never move outside of the application domain that it was created in. Imagine we have a method on a remote object and as the return value from the object we pass an instance of a class called Customer. By default, Remoting will attempt to serialize that object and pass it back down the wire to the caller, whereupon it's reconstituted into an object and used as normal. However, if this object extends MarshalByRefObject, that object will not be moved. Instead, the object will remain executing in the remote process space and what will be returned will be a reference to that object. Subsequent calls will be passed down the wire back to the newly created remote object. This is an important point, which we'll cover in more detail later.

When it comes to registering the object with Remoting, we need a URI (Uniform Resource Identifier). This URI will be requested by the remote client as an endpoint to connect to. A good approach to use is to combine an arbitrary application name with the name of the type followed by the identifier ".remote", for example:

ApplicationName/NamespaceName.TypeName.remote

So, if we have an application name of MyApplication, and a type name of MyNamespace.MyObject, we'd get this:

MyApplication/MyNamespace.MyObject.remote

To access that object, we need to know the protocol, the hostname of the server, and the port. So, if we have an HTTP channel on port 8001 on CORRADO (the name of my file server here in the office), the full URL would be:

http://corrado:8001/MyApplication/MyNamespace.MyObject.remote

However, when we pass the URI to System.Runtime.Remoting.RemotingConfiguration when we want to actually register the object, we don't need to specify the port, protocol or hostname as this is inferred from the settings on the machine, or particular in the channel configuration. Remember, when we come to register the channels we'll provide the ports and the protocols that we need.

Before adapting Service so that it can configure the Remoting settings, create the new ServiceConnector class. This class has to extend System.MarshalByRefObject.

public class ServiceConnector :
            MarshalByRefObject
{
public
            ServiceConnector()
{
}
}

In Service, we need a fairly broad set of namespaces to be available. Add these to the Service class:

using System;
using System.IO;
using System.Threading;
using System.Collections;
using System.Runtime.Remoting;
using System.Runtime.Remoting.Channels;
using System.Runtime.Remoting.Channels.Tcp;
using System.Runtime.Remoting.Channels.Http;

In addition to this, you'll also need to add a reference to the System.Runtime.Remoting.dll assembly otherwise you won't have access to all of the namespaces.

Inside the StartService method, the first thing we need to do is to register the channels. The ports from these will be read from the configuration file, and are made available through the TcpPort and HttpPort fields of the Configuration object available through the Configuration property. This is just an issue of creating new TcpChannel and HttpChannel services and passing them to ChannelServices for registration.

// StartService - run the
            service...
public void StartService()
{
// create
            channels...
Console.WriteLine("TCP channel listening on port
            {0}",
Configuration.TcpPort);
IChannel channel = new
            TcpChannel(Configuration.TcpPort);
ChannelServices.RegisterChannel(channel);
Console.WriteLine("HTTP
            channel listening on port {0}",
Configuration.HttpPort);
channel = new
            HttpChannel(Configuration.HttpPort);
ChannelServices.RegisterChannel(channel);

Next, we need to register ServiceConnector with the Remote service:

// register a Remoting end-point...
Type type = typeof(ServiceConnector);
string uri = string.Format("{0}/{1}.remote",
Configuration.ApplicationName, type.ToString());
Console.WriteLine("Listening on {0}", uri);
RemotingConfiguration.RegisterWellKnownServiceType(type, uri,
WellKnownObjectMode.Singleton);

At this point, remote objects can attempt to connect to the remote ServiceConnector object.

Creating the Folder Structure

What we've missed so far is that we don't have any code to make sure that the folder structure is properly configured to manage the lists. Although we've created c:\Wrox ListServer on demand, inside this folder we need three folders: Jobs, Bounce and Unsubscribe.

We'll change the Configuration object so that it handles all of this for us. Firstly, add these three constants to Configuration:

public class Configuration : SelfSerializer
{
// fields...
public string ListOwnerAddress = "bounce@wrox.com";
public int TcpPort = 8000;
public int HttpPort = 8001;
public string ApplicationName = "WroxListServer";

// const...
public const string JobsFolderBaseName = "Jobs";
public const string BounceFolderBaseName = "Bounce";
public const string UnsubscribeFolderBaseName = "Unsubscribe";

In addition, add the private _baseFolder field as well:

public class Configuration : SelfSerializer
{
// fields...
private DirectoryInfo _baseFolder;
public string ListOwnerAddress = "bounce@wrox.com";

Inside the Load method, we need to set the BaseFolder property:

// Load - de-serialize the configuration
            from disk...
public static Configuration Load(string
            filename)
{
Configuration config = null;

// do we have
            this file?
FileInfo info = new
            FileInfo(filename);
if(info.Directory.Exists ==
            false)
info.Directory.Create();
if(info.Exists ==
            false)
{
// create a new configuration file...
config = new
            Configuration();

// save the
            file...
config.Save(filename);
}
else
config =
            (Configuration)Load(typeof(Configuration),
filename);

//
            configure the base folder...
config.BaseFolder =
            info.Directory;

// return it...
return
            config;
}

The BaseFolder property needs to make sure that all of the ancillary folders exist. This is done through a call to CheckFoldersExist. In addition, this property needs to be decorated with System.Xml.Serialization.XmlIgnore because we don't want to save its details when we serialize the document.

[System.Xml.Serialization.XmlIgnore()]
public
            DirectoryInfo BaseFolder
{
get
{
return
            _baseFolder;
}
set
{
_baseFolder =
            value;
CheckFoldersExist();
}
}

Before we build CheckFoldersExist, we need three properties that return System.IO.DirectoryInfo objects for each of the folders. Each of these properties needs to be decorated with XmlIgnore:

[System.Xml.Serialization.XmlIgnore()]
public DirectoryInfo JobsFolder
{
get
{
return new DirectoryInfo(BaseFolder.FullName + "\\" +
JobsFolderBaseName);
}
}

[System.Xml.Serialization.XmlIgnore()]
public DirectoryInfo BounceFolder
{
get
{
return new DirectoryInfo(BaseFolder.FullName + "\\" +
BounceFolderBaseName);
}
}

[System.Xml.Serialization.XmlIgnore()]
public DirectoryInfo UnsubscribeFolder
{
get
{
return new DirectoryInfo(BaseFolder.FullName + "\\" +
UnsubscribeFolderBaseName);
}
}

All that CheckFoldersExist has to do is go through each of the folders making sure that they are in place. We've already checked that BaseFolder exists, but for completeness we'll do this again in case this method is used from a derived class. The name implies that all folders are checked:

protected void
            CheckFoldersExist()
{
if(BaseFolder.Exists ==
            false)
BaseFolder.Create();
if(JobsFolder.Exists ==
            false)
JobsFolder.Create();
if(BounceFolder.Exists ==
            false)
BounceFolder.Create();
if(UnsubscribeFolder.Exists ==
            false)
UnsubscribeFolder.Create();
}

We're still at a point where we can't run the project, but we'll solve this now. Create a new Console Application called ListServerHost inside the existing solution. You can do this by right-clicking on the ListServer solution in Solution Explorer and selecting Add - New Project.

You'll also need to add a reference to the ListServer project. In addition, using Solution Explorer right-click on ListServerHost and select Set as Start-up Project.

Inside the Class1 class, add a namespace import directive to ListServer:

using System;
using ListServer;

namespace ListServerHost

Once you've done that, you can load up Configuration and also start the service running:

[STAThread]
static void Main(string[]
            args)
{
// load the configuration...
Configuration config =
            Configuration.Load(
"c:\\Wrox ListServer\\config.xml");

//
            start up the service...
Service service = new
            Service(config);
service.StartService();

//
            wait...
Console.ReadLine();
}

When the service is running, you'll see this:

At this point, there isn't any functionality in the service that we can use. However, if you look at the c:\Wrox ListServer server folder, at least we can see that we have created the folders and the config.xml file automatically.

Here's what the contents of the config.xml file looks like:

<?xml version="1.0" ?>
<Configuration xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<ListOwnerAddress>bounce@wrox.com</ListOwnerAddress>
<TcpPort>8000</TcpPort>
<HttpPort>8001</HttpPort>
<ApplicationName>WroxListServer</ApplicationName>
</Configuration>

Let's move on and get to a point where we can create jobs.

Creating Jobs

Although we've got a lot of magical plumbing going on, ideally we want to get to a point where the application that actually wants to use the service doesn't have to do very much work at all to consume the service. There's no reason why the application developer needs to consider the fact that the service is actually running remotely. This level of abstraction is a common approach in server-based development. Consider a SQL Server connection string - you don't care if the server is local or remote. It's up to the application to do the abstraction. What's cool here is that all this Remoting magic makes life easy for the server-side developer too, and we don't have to do much to make life typically easy for the client-side developer.

Before when we introduced Remoting, I mentioned briefly about this object called System.MarshalByRefObject that had the effect of pinning an object into a given application domain and prevented it from being serialized and passed down the wire to the caller. We can use this to great effect when working with jobs.

When an application wants to send messages, it needs a "job". Typically it will create this job as part of whatever process involves walking the list of recipients for the message. In order to create a job, the application needs to know:

  • The hostname of the server,
  • The protocol and port that should be used,
  • The application name.

These three items are combined to form the URL of the remote ServiceConnector object. Typically when using Remoting, we'll use the GetObject method on System.Activator to get hold of a proxy capable at connecting to the remote object. However, in our case, what we're going to do is add a static method to ServiceConnector that knows how to form and how to connect to the remote object. First, add this enumeration to ServiceConnector:

public class ServiceConnector : MarshalByRefObject
{
// enum...
public enum ServiceProtocol { Tcp, Http };

Next, add this method to ServiceConnector:

// Connect - connect to the remote service...
public static ServiceConnector Connect(ServiceProtocol protocol,
int port, string hostname, string applicationName)
{
// url...
Type type = typeof(ServiceConnector);
string url = string.Format("{0}://{1}:{2}/{3}/{4}.remote",
protocol.ToString(), hostname, port.ToString(),
applicationName, type.ToString());
Console.WriteLine("Connecting to {0}", url);

// connect...
return (ServiceConnector)Activator.GetObject(type, url);
}

When Connect is called, it looks like the caller will be returned a ServiceConnector object. Instead, the caller will actually be given a "transparent proxy" that supports the same methods and properties as the remote object. The remote object remains pinned into the application domain of the remote service. As in this case the remote object is a singleton, only one instance of this object will ever be created. Subsequent calls to Connect will return a new proxy instance that connects to the same remote instance of the object.

Creating the job is a relatively involved process so let's go through it step-by-step.

The "Job" Class

The first thing we need is a class that represents the job. Imagine we want to send out a newsletter. The first step is to create the job, then populate the job with outbound messages, and then process the job so that the outbound messages are sent. Among other things, this class will need to hold the name of the job and the name of the folder on disk into which new messages will be queued. Folders will not be recycled, so they'll be used for a single job. It's up to the administrator to clear out old jobs (the reason why we do this is because a copy of the sent mail is kept around on disk after it has been sent. From personal experience, I've found it useful to have a copies of these documents should someone need to audit the newsletter batch). A job also needs to keep track of its own state - and this is used to record whether the job is open and is receiving messages, whether it is sending messages, or whether it has closed. Eventually, this class will be able to spin up its own thread in order to send messages on to their final destination, but we're not going to worry about this for now.

The Job class needs to extend MarshalByRefObject - for reasons that I'll explain in a little while. For now, add this code to Job:

using System;
using System.IO;
using
            System.Threading;
using System.Runtime.Serialization;
using
            System.Runtime.Serialization.Formatters.Binary;

namespace
            ListServer
{
public class Job : MarshalByRefObject
{
//
            fields...
private DirectoryInfo _folder;

public
            Job(DirectoryInfo folder)
{
// store...
_folder =
            folder;
}

public DirectoryInfo
            Folder
{
get
{
return _folder;
}
}

public
            string Name
{
get
{
return
            Folder.Name;
}
}
}
}

The "JobCollection" Class

The Service class will need to hold a collection of Job objects. We can do this pretty easily by inheriting from System.Collections.CollectionBase and adding our own implementations for Add and Remove, and by adding our own indexer.

public class JobCollection : System.Collections.CollectionBase
{
public JobCollection()
{
}

public void Add(Job job)
{
List.Add(job);
}

public void Remove(Job job)
{
List.Remove(job);
}

public Job this[int index]
{
get
{
return (Job)List[index];
}
set
{
List[index] = value;
}
}
}

In addition to these methods, it's useful to add methods to find a Job object given the name ("Find") and to quickly discover if the collection already contains a Job with the given name ("Contains").

public Job Find(string jobName)
{
//
            loop...
foreach(Job job in
            InnerList)
{
if(string.Compare(job.Name, jobName, true) ==
            0)
return job;
}
return null;
}

public bool
            Contains(string jobName)
{
if(Find(jobName) != null)
return
            true;
else
return false;
}

Creating new Jobs

To create a job, the application ultimately needs to get hold of an instance of ServiceConnector and call the CreateJob method. We've already seen how the caller can use the static Connect method on this class to get hold of a reference to the remote object. Once the caller has done that, he or she can use the CreateJob method. This method needs to defer to a new method with the same name on the instance of Service that is actually managing the application. This instance is retrieved using the static CurrentService property, like this method, which should be added to ServiceConnector:

public Job CreateJob(string jobName)
{
return Service.CurrentService.CreateJob(jobName);
}

To hold a list of jobs, Service needs a field that holds an instance of a JobCollection object. However, because this service may be being used by multiple applications simultaneously (imagine multiple clients on the network all calling into it), it's imperative that we properly manage thread concurrency issues.

To the Service class, add the _jobs and _jobsLock fields:

public class Service
{
// fields...
private static Service _currentService;
private Configuration _configuration;
private JobCollection _jobs = null;
private ReaderWriterLock _jobsLock = new ReaderWriterLock();

The CreateJob method (this time on Service, rather than on ServiceConnector) accepts a name. Each job exists in its own folder within the c:\Wrox ListServer\Jobs folder. As no two sibling folders can have the same name, no two jobs can have the same name either. For this reason, when we're given the name of a job, we have to find the next available name for it. For example, if MyMailingList is already taken, we'll try MyMailingList_2. If that's taken, we'll try MyMailingList_3, and so on.

To properly handle the thread synchronization, we'll only allow a single thread to do this at a time. We'll obtain a writer lock before we start the process, and we'll release that lock when we've finished, or when an exception has been thrown.

// CreateJob - create a new job...
public
            Job CreateJob(string jobName)
{
Console.WriteLine("Creating
            job {0}", jobName);

// lock...
try
{
//
            lock...
_jobsLock.AcquireWriterLock(-1);

// can't have two
            jobs with the same name, so loop...
int n = 1;
DirectoryInfo
            newFolder;
while(true)
{
// try a name...
string
            useName;
if(n == 1)
useName = jobName;
else
useName =
            string.Format("{0}_{1}", jobName,
n.ToString());

// try
            it...
newFolder = new
DirectoryInfo(string.Format("{0}\\{1}",
Configuration.JobsFolder.FullName,
useName));
if(newFolder.Exists == false)
break;

//
            next...
n++;
}

// create the new
            job...
Console.WriteLine("Created job {0}",
newFolder.FullName);
Job job = new Job(newFolder);

//
            add it...
_jobs.Add(job);

// return...
return
            job;
}
finally
{
//
            unlock...
_jobsLock.ReleaseWriterLock();
}
}

The trick now is when we actually return the new Job object to the client. Because Job extends MarshalByRefObject, it never moves out of the application domain in which it has been created - or to put it more simply, it never moves from the server to the client. What this means is that any calls we make to this object in effect become remote calls. Therefore, if we have a method on Job called Enqueue and as a parameter this takes the details of a message that we want to send, what in effect happens is that the message is automatically transferred over the network without us having to actually do anything special to make this happen.

Loading Existing Jobs

Before we can try this, there's just one thing that we have to do. After we've configured the ServiceConnector object for Remoting inside of StartService, we need to look at the c:\Wrox ListServer\Jobs folder to see if there are any existing jobs. If there are, we need to load this into the collection too. Eventually, we want to get to a point where if the service stops for some reason when it's processing a job, we want to be able to restart that job when the service is restarted.

Add this code to StartService in Service:

// StartService - run the
            service...
public void StartService()
{
// create
            channels...
Console.WriteLine("TCP channel listening on port
            {0}",
Configuration.TcpPort);
IChannel channel = new
            TcpChannel(Configuration.TcpPort);
ChannelServices.RegisterChannel(channel);
Console.WriteLine("HTTP
            channel listening on port {0}",
Configuration.HttpPort);
channel = new
            HttpChannel(Configuration.HttpPort);
ChannelServices.RegisterChannel(channel);

//
            register a Remoting end-point...
Type type =
            typeof(ServiceConnector);
string uri =
            string.Format("{0}/{1}.remote",
Configuration.ApplicationName,
            type.ToString());
Console.WriteLine("Listening on {0}",
            uri);
RemotingConfiguration.RegisterWellKnownServiceType(type,
            uri,
WellKnownObjectMode.Singleton);

// load the existing
            jobs...
try
{
//
            lock...
_jobsLock.AcquireWriterLock(-1);

// create a new
            job list...
_jobs = new JobCollection();

//
            walk...
foreach(DirectoryInfo folder in
Configuration.JobsFolder.GetDirectories())
{
// create the
            new job...
Job job = new
            Job(folder);
_jobs.Add(job);
}
}
finally
{
//
            unlock...
_jobsLock.ReleaseWriterLock();
}
}

Adding another "CreateJob"

The final method that we need to add is to the Job class itself. This static method will make it a little easier to create the jobs. Rather than having to connect to a remote ServiceController instance and then call CreateJob, we'll make it so that we can do that in a single call.

Add this method to Job:

public static Job CreateJob(ServiceConnector.ServiceProtocol
protocol, int port, string hostname, string
applicationName, string jobName)
{
ServiceConnector connector =
ServiceConnector.Connect(protocol, port, hostname,
applicationName);
return connector.CreateJob(jobName);
}

Testing

To test what we've done, ideally we want a separate client application to connect to the server application. Visual Studio provides a great technique for debugging both client and server projects simultaneously, which I'll now introduce.

To the existing solution, add a new Console Application project. Call it ListServerClient. Add a reference to the ListServer project.

Right-click on the ListServer Solution in Solution Explorer and select Properties. Check on Multiple Startup Projects and set ListServerHost and ListServerClient to Start.

Now if we select Debug - Start from the menu, both the client and the server need will be started, and both will share the same ListServer project as the source of the list server application's objects.

Before we do that, we need to add some code to Class1 in ListServerClient. As the server will take a short while to start, we'll prompt the user to hit the Return key before the client attempts to connect to the server.

[STAThread]
static void Main(string[] args)
{
// wait...
Console.WriteLine("Press Return to test the server...");
Console.ReadLine();
Console.WriteLine();

// create a new job...
Job job = Job.CreateJob(ServiceConnector.ServiceProtocol.Tcp,
8000, "localhost", "WroxListServer", "MyMailingList");
Console.WriteLine("Created job {0}", job.Name);

// wait...
Console.ReadLine();
}

Run the project now and wait until the server has finished starting - i.e. wait until the "Listening on" message appears. Flip over to the client application and hit Return. The new job will be created. Here's the debug output from the server:

...and here's the debug output from the client:

You'll notice that on the client side, in the debugging information we make a call into the Name property of the newly returned Job object. Because Job is inherited from MarshalByRefObject, the object never moves. Therefore in this case what's actually happening is that a remote call is being made.

While this is great from a plumbing perspective (all we have to do to expose remote functionality to a client is add new methods to Job), it's worth keeping an eye on this from a performance perspective. Every call that's made has to go over the wire, which may be a staggeringly bad idea depending on what it is that you're trying to do as you could potentially flood your network with traffic. For example, imagine having an indexer on this object and attempting to retrieve every one of, say, 1,000 items accessible through the indexer. In this example, 1,000 individual network requests would be serviced, where it might actually be more useful to transfer all 1,000 items in one block and work through it once it's in place (transferring data to work on in a block fashion is often more efficient than downloading it piecemeal).

At this point, our new job has been created and a new folder has been created to service it:

If you attempt to run the client application again, a new job will be created, but as we mentioned before it will have a unique name. For example:

Queuing Messages

So we can create new jobs. How do we actually send messages?

All we need to do is build some classes that can represent a message. Once we've done that, we'll be able to instantiate instances and pass them over to the remote server by calling a method on the Job object.

The first class we require is Recipient. This will be able to hold the e-mail address of the recipient of an e-mail message, and can also optionally hold the name of the recipient. The class looks like this:

[Serializable()] public class
            Recipient
{
// fields...
private string _email;
private
            string _name;

public Recipient()
{
}

public
            Recipient(string email)
{
_email = email;
}

public
            Recipient(string email, string name)
{
_email =
            email;
_name = name;
}

public string
            Email
{
get
{
return _email;
}
set
{
_email
            = value;
}
}

public string
            Name
{
get
{
return _name;
}
set
{
_name =
            value;
}
}
}

Notice the fact that the class is marked with the Serializable attribute, because we'll want to send it over the wire.

We'll override ToString so that it returns a string representation of the recipient. If a name has been specified, this string needs to take the form:

name <emailaddress>

whereas, if no name is specified, it just contains the e-mail address. Here's the code:

public override string
            ToString()
{
if(Name != string.Empty)
return
            string.Format("{0} <{1}>", Name, Email);
else
return
            Email;
}

On occasion we may want to hold a collection of Recipient objects. For example, we may want a single message to go to a group of recipients. Although you may say "That's the point of a list server!", remember that in most cases we don't want to have a single message containing our entire newsletter database. Each message ideally has to be individually crafted and sent. Add a new RecipientCollection class. Notice how we inherit from CollectionBase and again mark it as Serializable. We'll also create alternate versions of Add to make it easier for the caller to add recipients:

[Serializable()] public class
            RecipientCollection :
System.Collections.CollectionBase
{
public
            RecipientCollection()
{
}

public void Add(Recipient
            rec)
{
List.Add(rec);
}

public void Add(string
            email)
{
Add(new Recipient(email));
}

public void
            Add(string email, string name)
{
Add(new Recipient(email,
            name));
}

public void Remove(Recipient
            rec)
{
List.Remove(rec);
}

public Recipient this[int
            index]
{
get
{
return
            (Recipient)List[index];
}
set
{
List[index] =
            value;
}
}
}

On those occasions where we do need to pass a list of recipients over to Microsoft SMTP Service, the set format for this is to separate each e-mail address with a semicolon. We'll override ToString again such that it goes through each recipient in the collection and adds each one to a string. We'll use the overridden ToString method on Recipient to get the address in the correct format:

public override string ToString()
{
//
            loop...
System.Text.StringBuilder builder =
new
            System.Text.StringBuilder();
for(int n = 0; n < Count;
            n++)
{
if(n !=
            0)
builder.Append(";");
builder.Append(this[n].ToString());
}

//
            return...
return builder.ToString();
}

Now we can move on to the Message class itself. This class simply requires a bundle of fields to support the various recipient lists, the subject of the message, and the body of the message. We'll also store another field called SendProblemDescription, which we'll talk about when we come to actually send the message. This class extends SelfSerializer:

[Serializable()] public class Message : SelfSerializer
{
// fields...
private Guid _id = Guid.Empty;
public Recipient From = new Recipient();
public RecipientCollection To = new RecipientCollection();
public RecipientCollection Cc = new RecipientCollection();
public RecipientCollection Bcc = new RecipientCollection();
public string Subject;
public string Body;
public string SendProblemDescription;

Notice as well how one of those fields is a System.Guid. For those of you who are unfamiliar with these, a GUID ("Globally Unique Identifier") is a great way of creating an ID for something that's guaranteed to be unique, even across different machines. It's actually a vast number (128-bits compared to the 32 bits of your average integer variable), that's made up by combining several unique system characteristics (including a unique value in your network card) with random numbers. No two GUIDs can ever be the same (while not technically true, this is practically true). Using a GUID against a message is a great way of uniquely identifying the messages no matter where they are created or where they eventually end up.

We'll on-demand create one of these using the Id property. We'll use it when we come to serialize the message:

public Guid Id
{
get
{
if(_id ==
            Guid.Empty)
_id = Guid.NewGuid();
return
            _id;
}
set
{
_id = value;
}
}

Finally, we need a Load method that defers to the Load method in the base SelfSerializer class:

public static Message Load(string filename)
{
return (Message)Load(typeof(Message), filename);
}

Now we can turn our attention to queuing the message. Once we've constructed the message and configured its properties, we'll pass it over to Job through the Enqueue method. Because the Job instance is actually running remotely, the effect of the call will be to transfer the Message object across the network to the server. However, before we do that we need to look at the "job state".

Job State

We mentioned before that a job can be in a certain "state". Specifically, a job can either be "open" (i.e. accepting messages), "running" (i.e. sending queued messages out to the recipients), "closed" (i.e. the send process has completed), or it can be in an "error" state. This state will be held in a file called state.dat in the specific job folder.

To start with, create a new enumeration in Job that can hold the state of the job:

public class Job :
            MarshalByRefObject
{
// fields...
private DirectoryInfo
            _folder;
private JobState _state;
private ReaderWriterLock
            _stateLock = new ReaderWriterLock();
private Thread
            _thread;
private bool _threadCancel;

// enum...
public
            enum JobState { Open, Running, Closed, Error };

We want the job to be able to report its state, but we don't want the consumer to be able to specifically set the state. Instead, we want methods on the Job object to do this for us as appropriate. Here's the state method:

public JobState
            State
{
get
{
return _state;
}
}

To determine the location of the state.dat file, we just need the full name of the folder, and the name of the state file:

public string StateFilename
{
get
{
return Folder.FullName + "\\state.dat";
}
}

When it's time to change the state of the job, we'll use BinaryFormatter to just write the value of the internal _state member out to the file:

protected void SetState(JobState
            newState)
{
FileStream stream = null;
try
{
//
            lock...
_stateLock.AcquireWriterLock(-1);

// set
            it...
_state = newState;

// save it to disk...
stream =
            new FileStream(StateFilename, FileMode.Create);
IFormatter
            formatter = new BinaryFormatter();
formatter.Serialize(stream,
            _state);
}
finally
{
//
            unlock...
_stateLock.ReleaseWriterLock();

//
            close...
if(stream !=
            null)
stream.Close();
}
}

"Enqueue"

Now that we have a Message class, and we have set up the job so that we know what state it's in, we can implement the Enqueue method.

All this method has to do is check the state of the job to make sure that it is "open" and, if it is, serializes the message to the job's folder. To get the name of the file, it uses the Id property (which returns a GUID) and tacks .xml onto the end.

Here's the Enqueue method that should be added to Job:

public void Enqueue(Message message)
{
Console.WriteLine("Queue message {0}", message.Id.ToString());

// check the state...
switch(State)
{
case JobState.Open:

// save it...
string filename = string.Format("{0}\\{1}.xml",
Folder.FullName, message.Id.ToString());
message.Save(filename);
break;

default:
throw new InvalidOperationException();
}
}

Notice how we use the Save method of the base SelfSerializer class to write the file to disk.

With all of that in place, we should be in a position where we can try sending a message. Open the Class1 class inside of the ListServerClient project and add code to create a new message object and queue it:

[STAThread]
static void Main(string[]
            args)
{
// wait...
Console.WriteLine("Press Return to test
            the
            server...");
Console.ReadLine();
Console.WriteLine();

//
            create a new job...
Job job =
            Job.CreateJob(ServiceConnector.ServiceProtocol.Tcp,
8000,
            "localhost", "WroxListServer",
            "MyMailingList");
Console.WriteLine("Created job {0}",
            job.Name);

// create the message...
Message message = new
            Message();
message.From.Email =
            "server@wrox.com";
message.To.Add("auser@wrox.com");
message.Subject
            = "Test message";
message.Body = "This is a test
            message!";

// queue...
Console.WriteLine("Queueing message
            {0}",
message.Id.ToString());
job.Enqueue(message);

//
            wait...
Console.ReadLine();
}

Run the project and you'll see something like this on the client:

If you look in the appropriate job folder, you'll see the message:

And if you open the XML file, you'll see the document representing the message:

<?xml version="1.0" ?>
<Message xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<From>
<Email>server@wrox.com</Email>
</From>
<To>
<Recipient>
<Email>auser@wrox.com</Email>
</Recipient>
</To>
<Cc />
<Bcc />
<Subject>Test message</Subject>
<Body>This is a test message!</Body>
<Id>ced330b3-4c7d-467e-a016-8af1e556161f</Id>
</Message>

If you run the client again, another message will be queued and so on, ad infinitum.

Sending Messages

Now that we can queue messages, we can turn out attention to how you actually send messages. Rather than tie our application in so that it has to use Microsoft SMTP Service, we'll create an extensible architecture so that we can plug in new "mail engine" classes.

One central premise of object orientation is the idea of "polymorphism". We should already be very familiar with how this works - it's the idea that by extending classes, we can add new functionality or alter existing behavior, whilst at the same time use those classes with methods and properties that understand the base classes.

In .NET, we can achieve polymorphism in two ways. One way is that we can create a base class that we expect people to extend from in order to add their own functionality. For example, imagine a base class called MailEngine that implements the method Send. MicrosoftSmtpMailEngine could extend this class and override Send such that it uses the Microsoft SMTP Service. Likewise, we could create AnotherMailEngine that also extends MailEngine, but provides a specific implementation of Send.

The other way to achieve polymorphism is to use interfaces. In this method, you sketch out a collection of methods and properties related to a common function. In our case, our IMailEngine interface (interfaces typically start with a capital "I") would implement a single method: Send. Any class that implements this interface effectively enters into a contract that guarantees that each method or property defined on the interface has an appropriate implementation on the implementing class.

The limit with the first inheritance approach is that in .NET, a class can only extend a single class, which means we can run into some limitations. If we already have a class capable of communicating with the mail server that already extends an existing class, we have to jump through some hoops to get the polymorphism working. Interfaces on the other hand can be retrofitted to most classes, so we'll use that approach.

In the ListServer project, create a new class called IMailEngine. By default, VS .NET will create this class with the class directive, whereas we actually want to use the interface directive.

public interface IMailEngine
{
void Send(Message message);
}

Now we can create a new class that implements IMailEngine. Create a new class called MicrosoftSmtpMailEngine, make it implement IMailEngine, and add a stub implementation for Send.

public class MicrosoftSmtpMailEngine :
            IMailEngine
{
public
            MicrosoftSmtpMailEngine()
{
}

public void Send(Message
            message)
{
// stub implementation...
}
}

To send messages, we're going to use the System.Web.Mail namespace. This contains a number of classes, the most relevant here being MailMessage and SmtpMail. MailMessage is a class for holding an e-mail message, and is conceptually similar to our Message class. We've written our own because we want superior handling of recipients (MailMessage does not support an easy way of providing a name alongside an e-mail address), and we also want to be able to control serialization. SmtpMail provides access to the Microsoft SMTP Service itself.

To access System.Web.Mail, you'll need to add the System.Web.dll assembly to your project.

Our Send method therefore has to construct a new MailMessage object and pass it to SmtpMail so that it will be queued once again within the Microsoft SMTP Service before it's sent on its way to its final destination.

Here is the implementation of Send:

public void Send(Message message)
{
//
            build the message...
MailMessage mail = new
            MailMessage();

// basic stuff...
mail.From =
            message.From.ToString();
mail.Subject =
            message.Subject;
mail.Body = message.Body;

// to, cc,
            bcc...
mail.To = message.To.ToString();
mail.Cc =
            message.Cc.ToString();
mail.Bcc =
            message.Bcc.ToString();

// send
            it...
SmtpMail.Send(mail);
}

Notice here how we're using the ToString method of RecipientCollection to return a properly formatted string back. Remember, we need to separate each recipient with a semi-colon and use the format name <emailaddress> if a name has been specified for the recipient.

We'll use the same mail engine irrespective of the job that we're processing. Eventually we'll want to provide a way of specifying an alternative mail engine rather than the built in MicrosoftSmtpMailEngine, and we'll do this through Configuration. Add this method to Configuration:

public IMailEngine CreateMailEngine()
{
return new MicrosoftSmtpMailEngine();
}

Sending Mail

To process a job, all we have to do is walk through each message in the job folder one by one, de-serialize it from XML, and pass it to the IMailEngine -implementing object for processing. This is a classic example of a process that's best executed in a thread. We may want to process multiple jobs simultaneously, and so we can run each one in its own thread - or at the very least we want to ensure that any threads attempting to queue messages into new jobs will not run into blocking problems.

Job, then, needs to support a thread. We also need a method that we can use for a controlled shutdown of the thread should the service be stopping when we're trying to process the thread. To Job, add the new _thread and  _cancelThread members:

public class Job : MarshalByRefObject
{
// fields...
private DirectoryInfo _folder;
private JobState _state;
private ReaderWriterLock _stateLock = new ReaderWriterLock();
private Thread _thread;
private bool _threadCancel;

When we want to run the job, we'll call the Run method. This method is responsible for checking the state of the job using the State property and spinning up the thread.

This part is easy. All we need to do is create a new System.Threading.Thread object and call start. The only valid state in which Run can be called is "open".

public void Run()
{
// check our
            state...
switch(State)
{
case JobState.Open:

//
            create and start the new thread...
_threadCancel =
            false;
_thread = new Thread(new
ThreadStart(ThreadEntryPoint));
_thread.Start();
break;

default:
throw
            new InvalidOperationException();
}
}

Inside the thread, things have the potential to get a little more complicated. What we want to do is have two folders inside the job folder. One ("Sent") will be used to store a copy of the message once it has been sent. The other ("Problem") will be used when the mail engine reports an exception in sending the message.

Create constants in Job to store the basic names of these two folders:

public class Job : MarshalByRefObject
{
// fields...
private DirectoryInfo _folder;
private JobState _state;
private ReaderWriterLock _stateLock = new ReaderWriterLock();
private Thread _thread;
private bool _threadCancel;

// enum...
public enum JobState { Open, Running, Closed, Error };

// const...
public const string StateBaseFilename = "state.dat";
public const string SentBaseFolderName = "Sent";
public const string ProblemBaseFolderName = "Problem";

Then, create new properties for each folder:

public DirectoryInfo SentFolder
{
get
{
return new DirectoryInfo(Folder.FullName + "\\" +
SentBaseFolderName);
}
}

public DirectoryInfo ProblemFolder
{
get
{
return new DirectoryInfo(Folder.FullName + "\\" +
ProblemBaseFolderName);
}
}

We need three methods to help us control the lifetime of the thread. If the service needs to stop, we need to stop sending messages and resume the queue when the service restarts. We need a Cancel method to allow the Service controller close to tell us to stop processing:

public void Cancel()
{
_threadCancel = true;
}

We also need an IsCancelled method that we can use to determine our own state:

public bool IsCancelled()
{
return _threadCancel;
}

Finally, we need a Join method that returns only when the thread itself has been finished. This method should not be called from within the thread itself otherwise we'll enter a deadlock. We'll give the thread 30 seconds to close naturally, and if this expires, we'll abort it as something would have gone wrong if we don't naturally drop out of the thread loop:

public void Join()
{
// do we have a
            thread?
if(_thread !=
            null)
{
Cancel();
if(_thread.Join(new TimeSpan(0, 0, 30))
            == false)
_thread.Abort();
_thread =
            null;
}
}

Now the thread code itself. The first thing we need to do is change our state to "running":

// ThreadEntryPoint...
private void ThreadEntryPoint()
{
Console.WriteLine("Running job {0}", Name);

// update the state...
SetState(JobState.Running);

After we've done that, we can ask Configuration to give us the mail engine that we're supposed to use:

// get the engine...
IMailEngine engine =
Service.CurrentService.Configuration.CreateMailEngine();

The next step is to make sure that the Sent and Problem folders exist. If they do not, we'll need to create them:

// check that the folders exist...
if(SentFolder.Exists == false)
SentFolder.Create();
if(ProblemFolder.Exists == false)
ProblemFolder.Create();

Now we can look at how to process the messages. We use the Folder property to return a DirectoryInfo object back, and we call the GetFiles method to get a list of all of the .xml files in the folder:

// get a list of the files...
FileInfo[] messageFiles = Folder.GetFiles("*.xml");

We loop through the messages one by one, calling IsCancelled each time. If we've been cancelled, we'll drop out of the loop:

// walk each message...
int messageIndex
            = 0;
while(IsCancelled() == false)
{
Message message =
            null;
try
{

Per iteration of the loop, we load the next message and give it to the mail engine for sending:

// process it...
message =
Message.Load(messageFiles[messageIndex].FullName);

// send it...
engine.Send(message);

If we get to this point, no exceptions have been thrown, and we can assume that the message was properly processed. We put the message into the Sent folder:

// move it...
string newFilename =
string.Format("{0}\\{1}.xml",
SentFolder.FullName,
message.Id.ToString());
messageFiles[messageIndex].MoveTo(newFilename);
}

If anything does go wrong, we want to save the message into the Problem folder. As well as this, we'll take the description of the exception and store it in the SendProblemDescription property of the message:

catch(Exception ex)
{
// ignore any
            problems that happen here...
try
{
// do we have a
            message...
if(message != null)
{
// save the
            details...
message.SendProblemDescription =
ex.GetType().ToString() + ":" +
ex.Message;

message.Save(messageFiles[messageIndex].FullName);

//
            move it...
string newFilename =
string.Format("{0}\\{1}.xml",
ProblemFolder.FullName,
message.Id.ToString());
messageFiles[messageIndex].MoveTo(newFilename);
}
}
catch
{
}
}

Finally after all that, we can move onto the next message:

// next...
messageIndex++;
if(messageIndex == messageFiles.Length)
break;
}

After we've walked all of the messages, we can set our state to "closed":

// finished...
SetState(JobState.Closed);
Console.WriteLine("Job {0} finished", Name);
}

To test this, call Run from within the client application after queuing a message:

// queue...
Console.WriteLine("Queueing
            message {0}",
message.Id.ToString());
job.Enqueue(message);

//
            run...
job.Run();

Run the project now and the server will report that the job has been executed:

In addition, you'll notice that the message file has been saved to the Sent folder:

Providing that you've changed the sample code to use a valid e-mail address, rather than the dummy e-mail address in the source code download, eventually the message will come through to your Inbox:

Handling Bounces, Returned Mail and "Unsubscribes"

Sending messages is just one part of the battle. The problems really start when messages come back! If for any reason the remote server cannot process the inbound message, the message will be returned along with an explanation to the "from" address specified in the outbound message.

Messages will be returned for a number of reasons. In some cases, the e-mail address that you've specified will be invalid, or there will be a problem with the recipient's account. For example (and this often happens with Hotmail), the recipient may not have enough free storage available to receive the message. Or the account may not exist or have been closed.

If the server doesn't reply to the message to tell you something's wrong, the actual recipient might. This can happen for, again, a number of reasons. A classic with mailing lists is the, "I'm out of the office at the moment" response. A recipient's e-mail client in response to any inbound message automatically sends these.

Another common response is the "unsubscribe" response. Traditionally, mailing lists allow people to opt out of future mailings by responding with the word UNSUBSCRIBE as the subject. We need to scan the subjects and if they do appear to be "unsubscribe" messages, we need to put them into a separate bucket for processing by another application.

In this implementation, we're not going to give the service the capability of connecting to a database and marking the messages. For example, if the user appears to want to unsubscribe, we don't want to send them messages again - otherwise we run the risk of upsetting them. Likewise, if the server is returning a message to us, we want to mark the account with a "bounce count". Once this bounce count gets high enough we stop sending them e-mail. For example, "three strikes, you're out!"

The problem we have to overcome is that while the Framework comes with support for sending e-mail, it does not come with support for receiving e-mail. To get around this problem, we need to use a third-party component.

The market for third-party COM components is well developed, and the "POP3 Download" sector of this market contains a number of products, some of them free. A great resource for finding new ASP/COM components is Aspin, which you can find at http://www.aspin.com/?WROXEMPTOKEN=618658Zezi9SpDFSZgQps5Xv6u.

The component that I'm going to use is DynuPOP3. I've used this component in the past, and I've personally never experienced problems with it (Disclaimer: Your mileage may vary!) You can find it at http://www.dynu.com/?WROXEMPTOKEN=618658Zezi9SpDFSZgQps5Xv6u.

Using the Component

Although COM has largely been replaced by a number of new technology innovations in .NET, Microsoft recognizes that the industry's investment in COM is huge and therefore, COM is by no means "dead". We can use "interop" to call down into a COM component that we like. What's more, we can also use our .NET components as if they were COM components - although we won't see how to do that in this article.

To use our DynuPOP3 component in our application, use Solution Explorer to add a new reference to the project. Change to the COM tab and find the Dynu 1.0 Type Library. Double-click it to add it to the Selected Components list.

Once you press OK, Visual Studio will create a set of wrapper classes that enable this COM component to be used from within your .NET application!

You can use the component just like a normal .NET object. However, there are a few things to be aware of.

Firstly, Microsoft have done a lot of work in evangelizing the way that components are supposed to be constructed and used in .NET, mainly to get over a lot of the confusion with COM components. .NET objects are supposed to stick to strict naming conventions, including case conventions that make it easier to move from one non-case sensitive language (like VB .NET) to another case-sensitive language (like C#). One classic place that we find this is in acronyms that are treated as normal works, e.g. System.Xml.XmlDocument is correct, whereas System.XML.XMLDocument is not.

Secondly, building rich object models is easier in .NET than it was in COM - although there's nothing new or special about .NET in this regard. It's simply easier to build new classes and also to build rich collection classes in VS .NET than it was in VS6.

Thirdly, there's no exception handling built into COM components, which means that error trapping and checking gets a bit more (frankly) primitive. Keep an eye on the state of the object and check return values.

Fourthly, we're using COM, which means we've potentially returned to the world of DLL Hell. If you are deploying an application that uses COM components through interop, be advised that the application may get broken should the user deploy a different version of the component than the one you've designed it to work with.

What these facts mean is that if you haven't done any COM work for a while and then move into doing some interop work like we're about to do, you will need to slip out of the .NET mindset and into the mindset of whatever the component manufacturer feels is logic.

Case in point - when using a .NET component, if you call a method like RetrieveMessage, you'd expect that method to return a new object containing the message. However, with DynuPOP3, this call actually sets properties on the same object that contains the method. No big deal, but something to be aware of.

Bounce Accounts

To handle inbound e-mails, we need Service to set up another thread that can potentially monitor multiple accounts and take one of three actions:

  • If the subject is "UNSUBSCRIBE", this message needs to be copied into the Unsubscribe folder.
  • If the subject appears to be one of a number of standard subjects used by mail servers, the message is either ignored (because it's a delay, or "warning") message, or copied into the Bounce folder.
  • If the subject is not understood, the message needs to be forwarded to the list owner. This will be the case if the recipient replies to the message.

We'll store a list of the accounts in a new BounceAccounts collection in Configuration. This collection will contain a list of BounceAccount objects, each one containng a username, password and server name that will be used to connect to the POP server. The BounceHandler class will walk each of these accounts, download and process any e-mail, and then wait for a set period of time before starting again.

First off, here's BounceAccount. This class needs to be serializable:

[Serializable()] public class
            BounceAccount
{
// fields...
public string
            Username;
public string Password;
public string
            ServerName;
public int Port = Pop3Port;

//
            const...
public const int Pop3Port = 110;

public
            BounceAccount()
{
}
}

Notice there how we've included a constant containing the default port for POP3, which happens to be 110.

We then need a custom collection class to hold a list of BounceAccount objects. This class also needs to be serializable:

[Serializable()] public class
            BounceAccountCollection :
System.Collections.CollectionBase
{
public void
            Add(BounceAccount acc)
{
List.Add(acc);
}

public
            void Remove(BounceAccount
            acc)
{
List.Remove(acc);
}

public BounceAccount
            this[int index]
{
get
{
return
            (BounceAccount)List[index];
}
set
{
List[index] =
            value;
}
}
}

To add this collection to Configuration, simply add a new public field called BounceAccounts.

[Serializable()] public class Configuration : SelfSerializer
{
// fields...
private DirectoryInfo _baseFolder;
public string ListOwnerAddress = "bounce@wrox.com";
public int TcpPort = 8000;
public int HttpPort = 8001;
public string ApplicationName = "WroxListServer";
public BounceAccountCollection BounceAccounts =
new BounceAccountCollection();

The issue we have now is that we need to get the details of the accounts into the configuration XML file. This is always a bit "chicken and egg", as it's silly to try and guess the potential structure of the XML file, but without having data it's hard to generate the file. What I've done here is in the server code, I find out if BounceCollections is empty and, if it is, I add a new account and save the configuration file:

[STAThread]
static void Main(string[]
            args)
{
// load the configuration...
string configFilename
            = "c:\\Wrox ListServer\\config.xml";
Configuration config =
            Configuration.Load(configFilename);

// do we have any bounce
            accounts configured?
if(config.BounceAccounts.Count ==
            0)
{
// create one...
BounceAccount acc = new
            BounceAccount();
acc.Username = "bounce";
acc.Password =
            "bounce";
acc.ServerName =
            "mail.mymailserver.com";
config.BounceAccounts.Add(acc);
config.Save(configFilename);
}

//
            start up the service...
Service service = new
            Service(config);
service.StartService();

//
            wait...
Console.ReadLine();
}

You'll need one bounce account per mailbox that you configure to handle returned e-mails. In this case, I've used a fake account. If you can imagine, if my messages are set to come from bounce@dotnet247.com, the mailbox I need to use is the one that I would use to pick up messages from this account.

Run the server and you'll find that the configuration file has been updated.

Creating "BounceHandler"

The next step is to build a new class that can handle downloading the mails. Create a new class called BounceHandler and add these namespace import declarations:

using System;
using System.IO;
using System.Web.Mail;
using System.Threading;

Controlling the lifetime of this thread is a little more complex than controlling the lifetime of the job thread. The issue here is that we want to design from the perspective of having quite a long period of time when the thread is actually doing nothing. This is the timeout period between checking mails - much like you already have your e-mail client configured to only check mail periodically, rather than constantly.

If the thread is working, we want to use the "cancel" pattern that we used before. As we're walking the e-mails, at some point we'll be able to check to see if we are cancelled and, if so, drop out of the thread. If, however, we're sleeping we could be sleeping for ten minutes, in which case we want to abort the thread.

Add fields to BounceHandler to support a Thread object, a _cancelThread flag, a _isSleeping flag, and a TimeSpan that will hold the timeout period at 10 minutes by default.

public class BounceHandler
{
// fields...
private Thread _thread;
private bool _cancelThread;
private bool _isSleeping;
public TimeSpan Timeout = new TimeSpan(0,10,0);

Spinning up the thread is easy. If the thread hasn't already been started, we create a new Thread object and start it.

// SpinUp - start the thread...
public
            void SpinUp()
{
// do we already have a thread?
if(_thread
            != null)
throw new InvalidOperationException();

// run
            it...
_cancelThread = false;
_isSleeping = false;
_thread =
            new Thread(new
            ThreadStart(ThreadEntryPoint));
_thread.Start();
}

Cancel, IsCancelled and IsSleeping are all self-explanatory:

public void Cancel()
{
// flag
            it...
_cancelThread = true;
}

public bool
            IsCancelled()
{
return _cancelThread;
}

public bool
            IsSleeping()
{
return _isSleeping;
}

The Sleep method uses the static Sleep method available on Thread, but it changes the flag before and after the Sleep call. It also won't drop into a sleep state when the thread has been cancelled.

protected void Sleep(TimeSpan
            timeout)
{
// if we're cancelled, stop...
if(IsCancelled()
            == true)
return;

// flag, sleep, wait...
_isSleeping =
            true;
Thread.Sleep(timeout);
_isSleeping =
            false;
}

Join works in a similar fashion to the Join method on Job. However, this one checks to see if the thread is sleeping and, if it is, issues the Abort call instantly, rather than waiting 30 seconds for the thread to finish naturally:

public void Join()
{
//
            thread?
if(_thread == null)
return;

//
            cancel...
Cancel();

// are we sleeping? if so, instantly
            abort...
if(IsSleeping() ==
            true)
_thread.Abort();
else
{
// try it
            gently...
if(_thread.Join(new TimeSpan(0,0,30)) ==
            false)
_thread.Abort();
}

//
            join...
_thread.Join();
_thread = null;
}

Now we can turn our attention to the thread itself. Before we do that though, let's take a quick look at the logic.

As we mentioned before, messages will either come through with the subject as "UNSUBSCRIBE", in which case we store them in the Unsubscribe bucket where they can be picked up by whatever application is actually going to handle updating their information nt the database, or the messages are going to come through with a failure message, or a warning message, or we won't understand what to do with them.

This gives us four states that we can express in an enumeration in BounceHandler:

public class BounceHandler
{
// enum...
public enum InboundMessageType { Unknown, BounceWarning,
BounceError, Unsubscribe };

How we tell whether we have a failure or warning message is a little trial and error. Each mail server will do it slightly differently, although what we will do is use the two most common replies, which come from Microsoft Exchange Server, or from Sendmail, a very popular e-mail server.

If Exchange cannot process a message, we'll receive a message with the original subject prefixed with "Undeliverable:", like this:

Undeliverable: The Mailing List Subject

If Sendmail cannot process a message straightaway we'll get a delay message, which we'll ignore:

Delivery Status Notification (Delay)

If after a period of time the message still cannot be processed, we'll get this back:

Delivery Status Notification (Failure)

These are the only cases we're going to handle in this implementation. Anything that's not understood will be forwarded directly to the list owner, which is specified in the ListOwnerAddress property of Configuration.

Here's the bounce handler thread. The first thing we do is to create a mail engine. This will be used to forward unprocessed messages onto the list owner:

// ThreadEntryPoint...
private void ThreadEntryPoint()
{
Console.WriteLine("Bounce handler started");

// create a mail engine...
IMailEngine engine =
Service.CurrentService.Configuration.CreateMailEngine();

Then, we keep processing until the thread is cancelled:

// loop...
while(IsCancelled() == false)
{

Per iteration, we go through every bounce account specified in the configuration file:

// go through each account...
foreach(BounceAccount acc in
Service.CurrentService.Configuration.BounceAccounts)
{

Each time, we use DynuPOP3 to connect to the remote server:

// connect...
DYNULib.POP3Class pop = new DYNULib.POP3Class();
pop.Connect(acc.ServerName, acc.Username, acc.Password,
acc.Port);

To find out how many messages are on the server, we use GetStatistics:

// how many messages?
pop.GetStatistics();

Next, we walk each message, provided that we haven't been cancelled. DynuPOP3 numbers the messages from 1, not 0.

// walk the messages...
for(int
            messageIndex = 1; messageIndex <= pop.NumberOfMessages
&& IsCancelled() == false;
            messageIndex++)
{

RetrieveMessage downloads the message from the server. The second parameter indicates if you want to delete the message from the server or not. 1 specifies that you do. When debugging, you might find it useful to set this to 0.

// get the message and delete it...
pop.RetrieveMessage(messageIndex, 1);

We then create a new Message object based on the details of the message on the server:

// create a new message object...
Message message = new Message();
message.From.Email = pop.From;
message.From.Name = pop.FromName;
message.Subject = pop.Subject;
message.Body = pop.Body;

Now we can examine the subject of the message. First off, we check to see if it is UNSUBSCRIBE:

// examine the subject...
InboundMessageType messageType =
InboundMessageType.Unknown;
if(string.Compare(message.Subject, "unsubscribe", true)
== 0)
messageType = InboundMessageType.Unsubscribe;
else
{

If it isn't, we compare it with our list of known subjects. If it starts with "undeliverable:" we know that it is an error:

// is it one of a known set?
string subject = message.Subject.ToLower();
if(subject.StartsWith("undeliverable:") == true)
messageType = InboundMessageType.BounceError;
else
{

Otherwise, we compare it with a list of known subjects:

switch(subject)
{
case
"delivery status notification (failure)":
messageType =
InboundMessageType.BounceError;
break;

case
"delivery status notification (delay)":
messageType =
InboundMessageType.BounceWarning;
break;
}
}
}

Once we know what the message is, it's time to decide what to do with it:

// now what?
string newFilename =
            string.Empty;
switch(messageType)
{
case
            InboundMessageType.Unsubscribe:

// save it to the unsubscribe
            folder...
newFilename = string.Format("{0}\\{1}.xml",
Service.CurrentService.Configuration.UnsubscribeFolder.FullName,
            message.Id);
break;

case
            InboundMessageType.BounceWarning:
// ignore it if it's just a
            warning...
break;

case
            InboundMessageType.BounceError:

// save it to the bounce
            folder...
newFilename = string.Format("{0}\\{1}.xml",
Service.CurrentService.Configuration.BounceFolder.FullName,
            message.Id);
break;

default:

// forward it to the
            list
            owner...
message.To.Add(Service.CurrentService.Configuration.ListOwnerAddress);
engine.Send(message);
break;
}

//
            save it?
if(newFilename !=
            string.Empty)
message.Save(newFilename);
}

//
            cancel?
if(IsCancelled() == true)
break;
}

After processing all of the accounts, we sleep until the timeout has expired:

// wait for the
            timeout...
Sleep(Timeout);
}
}

If you want to try this, you'll need to send some messages to one of the mailboxes that you can specify in the bounce accounts list. After a period of time, check the Unsubscribe and Bounce folders and you should find the messages sorted appropriately. If you sent a message without an understood subject, you should receive this in your Inbox (providing that ListOwnerAddress is properly configured), appearing as if it's come from the original recipient:

Summary

In this article, we took a detailed look at how to build a list server for .NET. We provided a mechanism whereby we could expose the functionality of the list server over Remoting and access it from any machine on the network whereby messages could be queued and ultimately sent on to their chosen recipient.

Case Study Information
Author | Matthew Reynolds
Technical Editor | Adam Ryland
Project Manager | Helen Cuthill
Reviewers | Andrew Krowczyk, Saurabh Nandu, David Schultz, Adam Ryland

If you have any questions or comments about this case study, or suggestions for further pieces, please contact the technical editor. If you wish to talk about the case study with other subscribers, please visit our forum here.


Building a Forms Generator using Code DOM

Building a Forms Generator using Code DOM
http://www.csharptoday.com/content.asp?id=1836
Article Rating: 4
Published on April 22, 2002
Abstract
In this case study, Matt Reynolds looks at one of the most powerful features of .NET – Code DOM. Code DOM is short for “Code Document Object Model”, and it has been designed to generate code in any managed language, although support from language to language may vary. Core Microsoft languages will typically support Code DOM (C# and VB .NET both do, obviously). In this case study, we’re going to look at how to build a Forms Generator using Code DOM. We’re going to put together a project that can take a DataTable and from that generate a new C# source code file containing a class that extends UserControl that also has a set of TextBox, Label and CheckBox controls that allow editing of that DataTable.
Case Study

In this case study, we're going to look at one of the most powerful features of .NET - Code DOM.

Code DOM is short for "Code Document Object Model", and it has been designed to generate code in any managed language, although support from language to language may vary. Core Microsoft languages will typically support Code DOM (C# and VB .NET both do, obviously). Some of the other languages coming out of commercial ventures or out of academia may or may not support Code DOM code generation.

In designing .NET, Microsoft wanted to give developers the ability to generate, compile and execute new, automatically generated software code from hand crafted software code. Step one of this was to integrate the language compilers into the standard Framework installation. Basically, along with the Framework install that gives you the CLI and all the associated bits and pieces, you also get the C# and VB .NET compilers.

Step two was to build two features into the Framework. One of them is Code DOM, which allows you to build up classes and methods using a very rich (if rather verbose) object model. It's designed to generate (or "spit") managed language source code that can be used in your own projects, or compiled and executed from code.

The other technology, which we won't be talking about in this case study, is implemented in the System.Reflection.Emit namespace. This does not generate managed language source code, but instead is used to generate complete assemblies of MSIL for execution.

In this case study, we're going to look at how to build a Forms Generator using Code DOM. We're going to put together a project that can take a DataTable and from that generate a new C# source code file containing a class that extends UserControl that also has a set of TextBox, Label and CheckBox controls that allow editing of that DataTable.

We're going to build the application from the perspective of allowing a developer to load the source code file into VS .NET for editing (we'll include Designer support for the control). However, using the same technology, there's no reason why we can't automatically compile the class into a temporary assembly on disk and create instances of the UserControl class.

More about Code DOM

The smart thing about Code DOM is that it is language neutral. This is what gives it its power. Because .NET supports this concept of language agnosticism - where in virtually all cases one language in .NET is as capable as any other language - thanks to the power of the CLI, generating code is now a little trickier than it was pre-.NET.

If, before .NET, I wanted to write a utility that took a stored procedure and generated code that I could use in my application that called that stored procedure, all I'd have to do is come up with some scheme that streams C++ code or VB code down to a file. The code that I'd have wouldn't look too different from the actual target code. For example:

WriteLine("public class MyClass")
WriteLine("{")
...

With Code DOM, things are very different. What we have to do is describe a class and its fields, methods and properties without using any language specific syntax. For example, consider:

Public Class MyClass
...
End Class

public class MyClass
{
...
}

These two code snippets do exactly the same thing, but the way that we express ourselves in VB .NET is different to how we express ourselves in C#. Code DOM gets around this problem by creating an abstraction layer that we describe the classes and members to. We create a "compile unit" and fill it full of classes, and fill the classes full of members, and fill the members full of statements, and then hand over to a separate code provider class that knows how to translate that Code DOM compile unit into physical source code.

The issue with Code DOM is that it is incredibly verbose! It can take ten lines of code to express a single method call. The flexibility of being able to go out to many languages means the level of abstraction of the Code DOM model has to be so high that you have to be very explicit in the way you express the code layout. It doesn't help that the object names in System.CodeDom are all very long, e.g. CodeThisReferenceExpression, CodeMethodReturnStatement, CodePropertyReferenceExpression, etc.

In this specific example, we're going to see how to generate code in C#. We'll use the Microsoft.CSharp.CSharpCodeProvider class to generate the code for compilation. If we wanted to use VB .NET, we'd use Microsoft.VisualBasic.VBCodeProvider. The actual Code DOM generation doesn't vary from code generator to code generator - we build the same language neutral model irrespective of the required target language. We simply switch the code provider just before generating the source code output from one language to another.

Visual Studio .NET and the Framework make heavy use of Code DOM. The Forms Designer uses it (and, in fact, we're going to be duplicating some of the work that VS .NET does here), and if you've ever generated typed DataSets or XML Web Service proxies, you've used Code DOM.

Another curious advantage of Code DOM is that although it's designed to generate source code, because the C# and VB .NET compilers are now included in the Framework install we can deploy code to client machines that use Code DOM to generate the source code and then use the compilers to generate assemblies. Once we have an assembly, we can load it into our application, create an instance of the newly generated class and then run methods on that instance. So we have a full solution from generate, to compile, to execution, all from code.

Describing the Form

The data for this sample application will be taken from the authors table of the sample pubs database supplied with SQL Server. The form will ultimately look like this:

We're going to consider the form in terms of bands - specifically, each band will be made up of a Label control and an "editable" control. The editable control will depend on the type of data being edited. Textual (including numeric) data will be edited using a TextBox control. Boolean data will be edited using a CheckBox control. Although we won't be doing this here, you can extend this model to include things like a date/time picker to modify a date, drop down lists to select from a lookup table of known values, etc.

In this example, we're going to assume that we'll always be extending UserControl to create our "container" classes. However, we're not limited to this, and can generate standalone Form classes. It's typically more useful to extend UserControl as this gives us the freedom to drop the container class into the actual forms used in our application, which gives us a few more UI options (common toolbars, status bars, menus, etc.).

Our FormsGenerator project will contain a FormTemplate class. This class defines:

  • The type of container to be generated,
  • The namespace name and the class name of the class to be generated,
  • A collection of ControlBand classes (see later),
  • The size of the control and other metrics bits and pieces.

We'll use ControlBand classes to build up the template. This class defines:

  • The name of the band (used to set the Text property of the label and the names of the controls),
  • The type of editable control to be created (TextBox or CheckBox),
  • A collection of properties defined in PropertyItem classes,
  • The name of the property on the editable control to bind to.

Finally, we'll need a Generator class that contains the method to generate C# code from the template.

Creating the Project

Create a new Visual C# - Class Library project now. Call it FormsGenerator.

The first class we want to build is PropertyItem. This class simply defines a name and value pair that will be a property ultimately set against the newly created editable control. Here's the class:

   public class
            PropertyItem
{
// fields...
public string Name;
public
            object Value;

public PropertyItem()
{
}

public
            PropertyItem(string name, object val)
{
Name = name;
Value
            = val;
}
}

We'll also need a strongly typed collection object called PropertyItemCollection :

   public class
            PropertyItemCollection
:
            System.Collections.CollectionBase
{
public
            PropertyItemCollection()
{
}

public void
            Add(PropertyItem item)
{
List.Add(item);
}

public
            void Add(string name, object val)
{
Add(new PropertyItem(name,
            val));
}

public void Remove(PropertyItem
            item)
{
List.Remove(item);
}

public PropertyItem
            this[int index]
{
get
{
return
            (PropertyItem)List[index];
}
set
{
List[index] =
            value;
}
}
}

ControlBand needs:

  • A Name property to hold the name of the band,
  • A ControlType property to hold the type of the editable control.
  • A PropertyItems property to hold a collection of PropertyItem objects,
  • A DataBindPropertyName property to hold the name of the bind-able property on the control (Text on the TextBox, Checked on the CheckBox.)

[Serializable()] public class ControlBand
{
// fields...
public string Name = string.Empty;
public Type ControlType = null;
public PropertyItemCollection PropertyItems =
new PropertyItemCollection();
public string DataBindPropertyName = string.Empty;
}

As you may have guessed, we also need ControlBandCollection :

[Serializable()] public class ControlBandCollection
: System.Collections.CollectionBase
{
public ControlBandCollection()
{
}

public void Add(ControlBand band)
{
List.Add(band);
}

public void Remote(ControlBand band)
{
List.Remove(band);
}

public ControlBand this[int index]
{
get
{
return (ControlBand)List[index];
}
set
{
List[index] = value;
}
}
}

FormTemplate needs these properties:

  • ClassName - holds the name of the generated class,
  • NamespaceName - holds the name of the namespace containing the generated control class,
  • Controls - holds a collection of ControlBand objects,
  • ContainerSize - holds a Size object representing the size of the newly generated control class,
  • ContainerType - holds a Type object representing the type of the required control class (typically Form or UserControl),
  • Padding - holds a Size object representing padding between controls or the edges of the form,
  • LabelWidth - holds an integer representing the width of new Label controls.

public class FormTemplate
{
// fields...
public string ClassName = "MyDynamicControl";
public string NamespaceName = "DynamicForms";
private ControlBandCollection _controls =
new ControlBandCollection();
public Size ContainerSize = new Size(300, 200);
public Type ContainerType = typeof(UserControl);
public Size Padding = new Size(5, 5);
public int LabelWidth = 100;

public FormTemplate()
{
}

public ControlBandCollection Controls
{
get
{
return _controls;
}
}
}

To build up a form, all we have to do is create a new FormTemplate class and add bands to it. Now we'll take a quick look at a method that can take a DataTable object and generate a new template from it.

The "CreateFromDataTable" Method

The CreateFromDataTable method is actually very straightforward. We take a DataTable control and iterate through its Columns collection adding new bands for each column that we find. We need to look at the type of data held in the control to determine if we want to create a TextBox control or a Label control in the band.

Here's the method that should be added to FormTemplate :

      public
            void CreateFromDataTable(DataTable table)
{
// create
            bands...
foreach(DataColumn column in table.Columns)
{
//
            create and add the band...
ControlBand band = new
            ControlBand();
band.Name =
            column.ColumnName;
Controls.Add(band);

// what's the
            type?
switch(column.DataType.ToString())
{
case
            "System.String":

// create the control...
band.ControlType
            = typeof(TextBox);

// binding...
band.DataBindPropertyName
            = "Text";

// finish...
break;

case
            "System.Boolean":

// create the
            control...
band.ControlType = typeof(CheckBox);

//
            binding...
band.DataBindPropertyName = "Checked";

//
            finish...
break;

default:

//
            text...
band.ControlType =
            typeof(Label);
band.PropertyItems.Add("Text",
string.Format("Unknown type: {0}",
column.DataType.ToString()));

//
            finish...
break;
}
}
}

Generating C# Code

Now that we have our set of template classes that can describe a form, we can turn our attention to looking at how to generate C# code from the template. We'll be able to take the generated code and paste it into a separate VS .NET project (or we can compile it from the command line) where we can test it.

Generating "Designable" Forms and Controls

In this specific example, we have two options. We can either generate .cs files and use them as a design-time solution - for example, we can generate the base form from some data and customize it before we deploy. Alternatively, we can take the generated code, compile it, and execute it at run time. These classes will extend System.Windows.Forms.UserControl, so we can take instances of these classes and load them onto a form at run-time for later use.

I'm going to concentrate on a design-time solution, i.e. we're going to see how to generate classes based on a set of data stored in a DataTable and edit that form using the VS .NET Forms Designer. The run-time solution is a little trickier as more code has to be rolled into the generated form to make it do anything useful. You can most certainly do this, but not in the space that I have allocated for this tutorial!

The Designer needs quite specific code to be made available in the .cs file containing the code. Specifically, it needs a method called InitializeComponent. In newly generated Form or Control files, this method is typically the first thing called by the constructor:

   public
            Form1()
{
//
// Required for Windows Form Designer
            support
//
InitializeComponent();

//
// TODO: Add
            any constructor code after InitializeComponent
            call
//
}

The problem that we have when generating code is that VS .NET believes that it "owns" everything in that InitializeComponent method. More specifically, it believes that any code in there that it doesn't understand can be blasted out of the way by its own code. This means that potentially any code that we write into this method, if not crafted properly, can be deleted by the Designer. What we have to do in order to trick VS .NET into thinking that code we generated is in fact code that it itself generated is exactly mimic the layout of the code that VS .NET would normally generate.

In this code snippet, I've added a single button called button1 to the form:

      #region
            Windows Form Designer generated code
/// <summary>
///
            Required method for Designer support - do not modify
/// the
            contents of this method with the code editor.
///
            </summary>
private void
            InitializeComponent()
{
this.button1 = new
            System.Windows.Forms.Button();
this.SuspendLayout();
//
//
            button1
//
this.button1.Location = new
            System.Drawing.Point(36, 60);
this.button1.Name =
            "button1";
this.button1.Size = new System.Drawing.Size(88,
            32);
this.button1.TabIndex = 0;
this.button1.Text =
            "button1";
//
// Form2
//
this.ClientSize = new
            System.Drawing.Size(292, 273);
this.Controls.AddRange(new
            System.Windows.Forms.Control[] {
this.button1});
this.Name =
            "Form2";
this.Text =
            "Form2";
this.ResumeLayout(false);

}
#endregion

First of all, InitializeComponent creates the instances of the control objects that it needs on the container control. The Designer has also created private fields on the control, the name of each field being the same as the Name property of the control. After the controls have been created, SuspendLayout is called. When the form is created, this stops .NET from sending around the various events required that support layout of the form.

Next, we go through each of the controls in turn. The properties are programmatically set. Changes to the Properties window in the Designer in turn causes new lines to be added here, or removed, or values changed.

After each control has been configured, we configure the control itself. Perhaps the most important call here is the call to the AddRange method of the Controls collection. This tells Windows Forms which child controls are included in the control layout. Finally, we call ResumeLayout, which tells .NET to start sending round the layout events once more.

You'll notice that IntializeComponent makes heavy use of the this keyword. This is a classic way of identifying Code DOM generated code. To ensure that the code remains as unambiguous as possible, Code DOM always puts in this keywords where a human developer would probably miss them out. Technically neither answer is "right", but omitting this does tend to make code more readable. This can help you understand a little about how Code DOM objects are put together. For example, if you want to refer to a method on the current object, you always have to specifically tell it that you're talking about the current object.

To recap then, all we have to do is generate code that looks like this and we'll be able to generate the basic control class from a new class that we'll call to the FormsGenerator project called Generator.

Generating the Base Class

First off, let's do a little housekeeping to get everything configured properly.

To the existing solution, add a new Visual C# - Windows Application project. Call it FormsGeneratorHost. Right-click on the new project in Solution Explorer and select Set as StartUp Project. Add a new reference to the FormsGenerator project too.

To the new Form1 form, add a Button control and a TextBox control. Call the Button control buttonGenerate and call the TextBox control textCode. The TextBox control needs to be quite large, so set the MultiLine property to True, the ScrollBars property to Both, and the WordWrap property to False. Finally, set the Anchor property to Top, Left, Bottom, Right.

Eventually, we'll fill up that TextBox with newly generated code. Before we can do that, though, we need to start building the Generator class. This class will return a System.CodeDom.CompileUnit object. As its name implies, this object represents a "unit" that can be compiled. In our case, this is going to be a single class that describes a control that we want to load into the VS .NET Designer.

To use Code DOM, we need to add a reference to the System.CodeDom namespace. In addition to this, it's helpful to be able to refer to classes that we want in the generated code directly. So, for example, if we want to create a System.Drawing.Point instance in the generated code, we should include the System.Drawing namespace.

In fact, you don't need to do this. You can refer to a type by name, rather than through a Type object (if we want to use the typeof operator, we actually need to be able to refer to the class instance otherwise we'll get a compile error). So, we could give Code DOM "System.Drawing.Point" as a literal string, rather than giving it "typeof(Location)", should we have System.Drawing imported into the class file. The advantage to the latter method is that it removes the problem of typos in the literal strings, e.g. System.Drwing.Point. The code here would be generated properly, but the resulting code would not compile.

If you haven't done so already, create a new Generator class in the FormsGenerator project. Add these namespaces:

using System;
using System.Data;
using System.Drawing;
using System.CodeDom;
using System.Collections;
using System.Windows.Forms;

To the new Generator class, we'll add the CreateCompileUnit method. This method will create the new CodeCompileUnit object and add a single namespace and a class definition. Eventually, we'll add members to this class. However, at this point we're just interested in showing that we can generate and compile an empty class.

Here's the CreateCompileUnit code. The first thing we do is to create a new CodeCompileUnit object:

public CodeCompileUnit CreateCompileUnit(FormTemplate template)
{
// create a new compile unit...
CodeCompileUnit unit = new CodeCompileUnit();

After generating the unit, we create new namespace and type ("class") definitions. We use the names defined in the template to do this:

// create a new class definition...
CodeNamespace namespaceDef =
new CodeNamespace(template.NamespaceName);
CodeTypeDeclaration classDef =
new CodeTypeDeclaration(template.ClassName);

The next move is to add the class to the namespace and to add the namespace to the unit:

// add the type to the namespace, namespace to the unit...
namespaceDef.Types.Add(classDef);
unit.Namespaces.Add(namespaceDef);

Finally, we need to inherit the class from whatever container type is defined in the template. This will, by default, be System.Windows.Forms.UserControl, although you can see here how you can use any class you like. As an aside, this BaseTypes collection is also where you define the interfaces that the base class can take.

// what does the class inherit from?
classDef.BaseTypes.Add(template.ContainerType);

As we work through this example, we'll add method calls in here to add various members to the control. For now though we'll just return the unit to the caller:

// return...
return unit;
}

The final bit of housework we need to do involves creating the template and generating some C# code from it. To the Form1 class of FormsGeneratorHost, add namespace import declarations for:

using System.Data.SqlClient;
using System.CodeDom;
using System.CodeDom.Compiler;
using Microsoft.CSharp;
using FormsGenerator;

To create the template that we'll be generating the code from, we'll use the pubs database that automatically gets installed along with SQL Server. We'll select the top row from the authors table and pass this to the CreateFromDataTable method of FormTemplate. Add this method to Form1 :

      public
            FormTemplate CreateTestTemplate()
{
// create the new
            template...
FormTemplate template = new
            FormTemplate();
template.ContainerSize.Width =
            500;
template.ContainerSize.Height = 400;

// create a
            dataset...
SqlConnection connection = new
            SqlConnection(
"integrated security=sspi;data
            source=(local);initial catalog=pubs");
Data = new
            DataTable();
SqlDataAdapter adapter = new
            SqlDataAdapter(
"select top 1 * from authors",
            connection);
adapter.Fill(Data);
adapter.Dispose();
connection.Close();

//
            create the
            controls...
template.CreateFromDataTable(Data);

//
            return...
return template;
}

By default, Code DOM wants to send its newly generated code to a stream class. I'm going to use a System.IO.MemoryStream for this, as rather than generating a file I actually want to display the generated code in the textCode control. So, we'll get CSharpCodeProvider to write the new code to a MemoryStream, we'll then rewind this stream to the beginning and read its contents out using a System.IO.StreamReader. This is a very common design pattern in .NET, and is extremely useful. If you haven't had time to commit it to memory, do so now!

Using the Designer, double-click on buttonGenerate to create a new Click event handler. The first thing we need to do is call CreateTestTemplate to create the FormTemplate that we'll generate the code from:

private void buttonGenerate_Click(object sender, System.EventArgs e)
{
// create the template...
FormTemplate template = CreateTestTemplate();

Once we have the template, we can create a new Generator and ask it to create a new CodeCompileUnit. After this call, the unit will be completely populated with the class and all of its members:

// generator...
Generator generator = new Generator();
CodeCompileUnit unit = generator.CreateCompileUnit(template);

Once we have the compile unit, we can generate the final source code. To do this, we create a new CSharpCodeProvider and ask it to give us an object that implements ICodeGenerator (at this point, if we wanted VB .NET code, we could create a new VBCodeProvider).

// show it...
CSharpCodeProvider cs = new CSharpCodeProvider();
ICodeGenerator csGen = cs.CreateGenerator();

The generator will want to write the code to a stream. We create a new System.IO.MemoryStream to do this, and then wrap a System.IO.StreamWriter around it:

         //
            create a stream...
MemoryStream stream = new
            MemoryStream();
StreamWriter writer = new
            StreamWriter(stream);
try
{

The code generation process may throw an exception, so we handle that - ultimately to present a message detailing the exception. Before we can generate the code, we can optionally provide some options. Here, I've told it to generate "C" style code. What this means is that the braces will appear in a classic C style, rather than a more contemporary C++ style where the braces appear to the right of the code expression, rather than underneath it.

// run it...
CodeGeneratorOptions options =
new CodeGeneratorOptions();
options.BracingStyle = "C";
csGen.GenerateCodeFromCompileUnit(unit, writer,
options);

The GenerateCodeFromCompileUnit is the call that writes the code into the stream. After this call, we can flush and rewind the stream, wrap a StreamReader around it, and read out all of the code for display.

            //
            rewind...
writer.Flush();
stream.Seek(0,
            SeekOrigin.Begin);
StreamReader reader = new
            StreamReader(stream);
textCode.Text =
            reader.ReadToEnd();
}
catch(Exception ex)
{
//
            loop...
StringBuilder builder = new StringBuilder();
while(ex
            !=
            null)
{
builder.Append(ex.GetType().ToString());
builder.Append(":");
builder.Append(ex.Message);
builder.Append("\r\n");
builder.Append(ex.StackTrace);
builder.Append("\r\n\r\n");
ex
            =
            ex.InnerException;
}
MessageBox.Show(builder.ToString());
}
finally
{
//
            close...
if(stream !=
            null)
stream.Close();
}
}

Run the project now and click the Generate Code button. The new class file will be generated.

So now we've proven that we can generate C# code from our application. The next step is to start populating the class with the members needed to generate the form.

Adding Fields

The first stage is to add the controls to the form for each of the bands defined in the template. Each band will need two controls, one for the label and another for the editable control itself. We'll define a naming convention of labelFieldName for the label control and textBoxFieldName or checkBoxFieldName for the editable controls. For the editable control, ideally we need to have a property that provides access to this control from outside of the class itself.

For example, for band with a name of au_lname editable through a TextBox, we'll have:

  • Field: _textBoxau_lname
  • Property: textBoxau_lname
  • Label field: labelau_lname

We'll add properties to the ControlBand class to control these names so that they are easily accessible from our code generation methods. The LabelFieldName property is very straightforward:

public string LabelFieldName
{
get
{
return "label" + Name;
}
}

PropertyName gets a little more complex as we have to take the type of the control used for editing and mangle the string. We'll get a fully qualified name back - for example, System.Windows.Forms.TextBox - so we need to strip off the namespace. We do this using the Split method to break the name apart into an array of strings. We also drop the case of the first character of the name so that the name is properly camel cased:

      public
            string PropertyName
{
get
{
string[] parts =
            ControlType.ToString().Split('.');
string controlName =
            parts[parts.Length - 1];
return controlName.Substring(0,
            1).ToLower() +
controlName.Substring(1) +
            Name;
}
}

To get the field name, all we need to do is prefix the property name with an underscore:

public string FieldName
{
get
{
return "_" + PropertyName;
}
}

Now we get to the part of the project where things start to get a little complicated, as we turn our attention to using Code DOM to build up the members of the class. As I mentioned before, this can get very verbose.

The first method we'll look at is AddControlField. This method should be added to the Generator class.

Most of the Code DOM methods are pretty lengthy, but apart from one or two exceptions, the naming of the methods is fairly logical. To add members to a class, you create objects that represent different kinds of members (fields, methods, and properties) and add them to the Members collection of the CodeTypeDeclaration. In this case, because we want a field, we create a CodeMemberField instance:

protected void AddControlField(Type type, string name,
CodeTypeDeclaration classDef)
{
// create the field...
CodeMemberField field = new CodeMemberField();

Most of the classes have a number of overloaded constructors. However, in this case we're using the empty constructor. We can then set properties on the new object to describe exactly how we want it to appear in code.

Things aren't looking too verbose so far, but let's take a look at how we give Code DOM a type. We don't just give it a System.Type object, as you might expect. Instead, we give it a CodeTypeReference object. You can construct one of these from a Type (if you have it), or from a string. As I mentioned before, if you do this from a string, there's a chance that a typo might cause your generated code not to compile properly.So, if you do have a way of explicitly referring to a class using a Type object, that is the best approach.

field.Type = new CodeTypeReference(type);

After providing the type, we supply the name. This is passed in as an argument to the AddControlField method.

field.Name = name;

By default, members are created as "private, instance" members. We can use the Attributes property to change the member's protection level, or to change them from instance to static types. In this case, I've explicitly set it to "private, instance".

field.Attributes = MemberAttributes.Private;

Fields can be given an initial value - and in this case we want to set the initial value to be null. To remove ambiguity, we have to wrap the "null" in a CodePrimitiveExpression object. If we didn't do this, Code DOM wouldn't know if a "null" value set against InitExpression meant, "there is no initial expression", or "set it to an initial expression of 'null'". This kind of move reduces ambiguity and you'll find it commonly used.

CodePrimitiveExpression itself is quite interesting. This class is used to insert literal values into the code. When generating C# code, the resulting code from this object gets turned into "null". In VB .NET, this turns into "Nothing". Another language may have a different interpretation. Alternatively, if we provide a string value to CodePrimitiveExpression, this will be properly wrapped in quotes.

field.InitExpression = new CodePrimitiveExpression(null);

The final thing we do with this method is to add it to the Members collection of the class definition. Technically, we don't have to do this here. We can do this at any point before the class is actually generated.

classDef.Members.Add(field);
}

The general purpose AddControlField method is used from within a new method called AddBandFieldsAndProperties. This method creates the two private fields (for the Label control and for the editable control) and the property to access the editable control. First off, we set up a loop to go through the collection of bands and then call AddControlField for each one:

protected void AddBandFieldsAndProperties(FormTemplate template,
CodeTypeDeclaration classDef)
{
// create fields for each band...
foreach(ControlBand band in template.Controls)
{
// add the fields...
AddControlField(band.ControlType, band.FieldName,
classDef);
AddControlField(typeof(Label), band.LabelFieldName,
classDef);

Now we need to add a new property. Rather than creating a new CodeMemberField instance, we create a new CodeMemberProperty instance. Notice how we set the Name, Type and Attributes properties as we did before, and how we add the new property to the class definition:

            //
            add a property for each...
CodeMemberProperty prop = new
            CodeMemberProperty();
prop.Name = band.PropertyName;
prop.Type
            = new CodeTypeReference(band.ControlType);
prop.Attributes =
            MemberAttributes.Public;
classDef.Members.Add(prop);

Now we need to look at our first example of injecting code into our members. In this case, the code is very simple - we just need to return the field back to the caller. This involves three things:

  • A reference to "this",
  • A reference to the field,
  • A return statement.

Although we'll only generate a single line to do this, we'll create three separate Code DOM objects, one to represent each part of this operation. To refer to the field, we need both the name of the field, and a reference to the object containing the field. As we mentioned before, the carefully non-ambiguous nature of Code DOM means that we always have to explicitly provide the this reference. This is done by using a CodeThisReferenceExpression object. In C#, this object generates the "this" keyword. In VB .NET, this object generates the "Me" keyword.

We express the complete field reference (this and the field name) using a CodeFieldReferenceExpression object. We then create a CodeMethodReturnStatement and pass in this new CodeFieldReferenceExpression as the expression to be returned.

The GetStatements collection is used as repository for the methods for the get portion of the property. If we wanted to be able to set values in this property, we can use the SetStatements property. When the code is generated, this collection is walked from top to bottom, in the order in which the statements are added.

// add the get method...
prop.GetStatements.Add(
new CodeMethodReturnStatement(
new CodeFieldReferenceExpression(
new CodeThisReferenceExpression(),
band.FieldName)));
}
}

Complex? Yes! Before we can run the code, we need to add a call to CreateBandFieldsAndProperties to CreateCompileUnit :

public CodeCompileUnit CreateCompileUnit(FormTemplate template)
{
// create a new compile unit...
CodeCompileUnit unit = new CodeCompileUnit();

// create a new class definition...
CodeNamespace namespaceDef =
new CodeNamespace(template.NamespaceName);
CodeTypeDeclaration classDef =
new CodeTypeDeclaration(template.ClassName);

// what does the class inherit from?
classDef.BaseTypes.Add(template.ContainerType);

// add the type to the namespace, namespace to the unit...
namespaceDef.Types.Add(classDef);
unit.Namespaces.Add(namespaceDef);

// add the stuff...
AddBandFieldsAndProperties(template, classDef);

// return...
return unit;
}

Run the code and you'll discover that the fields and properties are created. Here are some of the fields:

public class MyDynamicControl : System.Windows.Forms.UserControl
{

private System.Windows.Forms.TextBox _textBoxau_id = null;

private System.Windows.Forms.Label labelau_id = null;

private System.Windows.Forms.TextBox _textBoxau_lname = null;

private System.Windows.Forms.Label labelau_lname = null;

private System.Windows.Forms.TextBox _textBoxau_fname = null;

private System.Windows.Forms.Label labelau_fname = null;

private System.Windows.Forms.TextBox _textBoxphone = null;

The properties look like this:

public virtual System.Windows.Forms.TextBox textBoxau_id
{
get
{
return this._textBoxau_id;
}
}

public virtual System.Windows.Forms.TextBox textBoxau_lname
{
get
{
return this._textBoxau_lname;
}
}

Adding the Constructor

The constructor's job in our class is to call InitializeComponent, and of course the developer generating the class has the option here of adding his or her own configuration.

To define a constructor, we need to create a new CodeConstructor instance (this is an example of a Code DOM class that has a slightly odd naming convention. To be in-line with the rest of the classes, this should more properly be called CodeMemberConstructor).

Code DOM assumes that constructors do not have a name - and this is certainly true of C# and VB .NET. To start defining the constructor, we create an instance of the constructor, set its protection level using the Attributes property (here, we define it to be "public") and add it to the class' members:

protected void AddConstructor(FormTemplate template,
CodeTypeDeclaration classDef)
{
// create the constructor...
CodeConstructor cons = new CodeConstructor();
cons.Attributes = MemberAttributes.Public;
classDef.Members.Add(cons);

We can now queue the statements for the constructor. The first statement we add is a comment statement. This is simply done with a new CodeCommentStatement instance. The generator automatically prefixes the comment directive - "//" for C#, "'" for VB .NET.

// call the init component method...
cons.Statements.Add(new CodeCommentStatement(
"controls initialized here..."));

Invoking methods using Code DOM is usually quite a lengthy process (unfortunate, since it's done so often!). Typically for a method you need a reference to a method and a set of parameters to pass into the method. In this case, we have no parameters. To call a method, we first create a new CodeMethodInvokeExpression that will ultimately be queued to the constructor's Statements collection:

CodeMethodInvokeExpression invoke =
new CodeMethodInvokeExpression();

To refer to the method, we have the same issue of requiring a "this" reference. We combine this with the name of the method:

invoke.Method =
new CodeMethodReferenceExpression(
new CodeThisReferenceExpression(),
"InitializeComponent");

As this method has no parameters, we queue the invoke expression now:

cons.Statements.Add(invoke);

Most developers realize that one key to making code more readable is to include white space. This, oddly enough, is actually quite counter-intuitive to do in Code DOM. To get around this, I've created a new CodeSnippetStatement method and given it a blank string. CodeSnippetStatement is used in situations where you simply want to insert specific code into the generated code stream. It should rarely be used, as it breaks the language neutrality of Code DOM, and frankly if you don't want language neutrality, you may as well use a StringBuilder and just write whatever language code you require out as a block of text. Using it here should be fine as it's safe to assume that no language would choke when attempting to compile a blank line.

// another comment...
cons.Statements.Add(new CodeSnippetStatement(""));

Finally in this constructor, we add another comment telling the developer that she should add her own customization at the end of the constructor - as is the recommendation when working with initialize component.

cons.Statements.Add(new CodeCommentStatement(
"add your own customization here..."));
}

Before running the code, add a call to AddConstructor to the CreateCompileUnit method of Generate :

// add the stuff...
AddBandFieldsAndProperties(template, classDef);
AddConstructor(template, classDef);

Run the project and you'll find the constructor in the newly generated code:

        public
            MyDynamicControl()
{
// controls initialized
            here...
this.InitializeComponent();

// add your own
            customization here...
}

Adding "InitializeComponent"

It's at this point that we start using Code DOM in anger. The vast majority of our generated code will be implemented in the InitializeComponent method.

As we mentioned before, this method works in three phases:

  • The controls are created,
  • The controls are configured through programmatic manipulation of the properties,
  • The main control container (Form, UserControl, etc.) is configured. This includes adding the newly configured controls to the Controls collection.

The code to create InitializeComponent will be pretty lengthy, so we'll break it up into a number of methods to make it more readable. The first of these is AddCreateControlCall. This method creates a new control (it will be reused to create both the Label controls and the specific editable control defined in the control band) and assigns it to the specific field.

Creating an object instance is done through the CodeObjectCreateExpression object (personally, I think this is a curious name given the names of the methods on System.Activator. I think a better name here would be CodeCreateInstanceExpression. I mention this as I find this a hard one to remember, because of the curious name). The constructor for this object can take a Type object, or a CodeTypeReference object (which classes can and can't take a Type object is another inconsistency in the Code DOM API). This class also defines a Parameters collection - but as we're calling a parameter-less constructor in this instance, we won't use it.

To assign the newly created control object to the desired field, we use a CodeAssignStatement object. This object defines Left and Right properties, which as their name implies, provides a location to provide expressions for both sides of the assignment operation. In this case, the Right side of the assignment is the new object, the Left side of the assignment is the field defined on the current object instance ("this").

      protected
            void AddCreateControlCall(string controlName,
Type controlType,
            CodeMemberMethod initMethod)
{
//
            initialize...
CodeAssignStatement assign = new
            CodeAssignStatement();
assign.Left = new
            CodeFieldReferenceExpression(
new CodeThisReferenceExpression(),
            controlName);
assign.Right = new
            CodeObjectCreateExpression(controlType);
initMethod.Statements.Add(assign);
}

The second method we require is AddLayoutControlCall. This method is responsible for the second phase of the InitializeComponent operation - i.e. defining the properties on the control.

Every control requires basic settings, namely the name, the location, the size and the tab index of the control. In addition, we provide a parameter to allow this method to automatically anchor the control to the top, left and right edges of the container control. This means that should the control be stretched in width, the editable control will automatically size with the form.

It might be the case that we want to provide additional properties to the control. As another parameter to AddLayoutControlCall, we provide a Hashtable that contains the name of the property to be set in the key, and against the key a CodeExpression object containing the value. This value could be a CodePrimitiveExpression (for literal values), or some other complex expression.

The first thing we need to do inside of AddLayoutControlCall is to create comments that allow InitializeComponent to delimit where the control configuration code for a particular control starts:

protected void AddLayoutControlCall(string controlName,
Point location, Size size, int tabIndex,
bool anchorRight, Hashtable otherProps,
CodeMemberMethod initMethod)
{
// comment...
initMethod.Statements.Add(new CodeCommentStatement(""));
initMethod.Statements.Add(new
CodeCommentStatement(controlName));
initMethod.Statements.Add(new CodeCommentStatement(""));

Through this method, we'll always be working with the same control. fieldRef is defined to be the field containing the code throughout the method:

// field...
CodeFieldReferenceExpression fieldRef =
new CodeFieldReferenceExpression(
new CodeThisReferenceExpression(), controlName);

After defining fieldRef, we set the Name property for the control using a CodePrimitiveExpression. Notice how we use a CodePropertyReferenceExpression to point to a property, just as we used to use a CodeFieldReferenceExpression to point to a field:

         //
            set the name...
CodeAssignStatement assign = new
            CodeAssignStatement();
assign.Left = new
            CodePropertyReferenceExpression(fieldRef,
"Name");
assign.Right = new
            CodePrimitiveExpression(controlName);
initMethod.Statements.Add(assign);

Next we set the Location property. This property takes a new System.Drawing.Point instance, which we create using a CodeObjectCreateExpression instance. This time, however, we need to provide parameters to the constructor. We create a new array of CodeExpression objects of length "2". Into the first we write a new CodePrimitiveExpression instance containing the X location of the control, and into the second we write another. Finally we combine the whole lot with a CodeAssignStatement to set the property:

         //
            set the location...
assign = new
            CodeAssignStatement();
assign.Left = new
            CodePropertyReferenceExpression(fieldRef,
"Location");
CodeExpression[] args = new
            CodeExpression[2];
args[0] = new
            CodePrimitiveExpression(location.X);
args[1] = new
            CodePrimitiveExpression(location.Y);
assign.Right = new
            CodeObjectCreateExpression(typeof(Point),
args);
initMethod.Statements.Add(assign);

To set the Size, we do a similar trick, but we create a System.Drawing.Size object, as opposed to a Location object:

         //
            set the size...
assign = new
            CodeAssignStatement();
assign.Left = new
            CodePropertyReferenceExpression(fieldRef,
"Size");
args[0] =
            new CodePrimitiveExpression(size.Width);
args[1] = new
            CodePrimitiveExpression(size.Height);
assign.Right = new
            CodeObjectCreateExpression(typeof(Size),
args);
initMethod.Statements.Add(assign);

The next part is a classic example of how Code DOM can make things very complicated. In order to make the form stretch in-line with the container, we need to set the Anchor property. In C#, we'd express the "attachment" to the top, left and right edges of the form as: AnchorStyles.Left | AnchorStyles.Top | AnchorStyles.Right. However, in VB .NET, we'd express this as AnchorStyles.Left Or AnchorStyles.Top Or AnchorStyles.Right.

We can get at each value in the enumeration by using the CodePropertyReferenceExpression object and giving it a reference to the enumeration type (System.Windows.Forms.AnchorStyles) and the name of the value in the enumeration that we wish to use. To combine the values together we use the CodeBinaryOperatorExpression class and give it the CodeBinaryOperatorType.BitwiseOr enumeration.

The first move is to set up a CodeAssignStatement and give it the "left" hand side of the assignment as a reference to the Anchor property.

// do we need to anchor right?
if(anchorRight == true)
{
// set anchor...
assign = new CodeAssignStatement();
assign.Left = new
CodePropertyReferenceExpression(fieldRef,
"Anchor");

We then create new CodePropertyReferenceExpression objects for each of the three members of the enumeration that we intend to use:

CodePropertyReferenceExpression anchorLeftRef =
new CodePropertyReferenceExpression(new
CodeTypeReferenceExpression(typeof(AnchorStyles)),
"Left");
CodePropertyReferenceExpression anchorTopRef =
new CodePropertyReferenceExpression(new
CodeTypeReferenceExpression(typeof(AnchorStyles)),
"Top");
CodePropertyReferenceExpression anchorRightRef =
new CodePropertyReferenceExpression(new
CodeTypeReferenceExpression(typeof(AnchorStyles)),
"Right");

Finally, we set the "right" hand side of the assignment as a combination of the three members:

assign.Right = new CodeBinaryOperatorExpression(
new CodeBinaryOperatorExpression(anchorLeftRef,
CodeBinaryOperatorType.BitwiseOr, anchorTopRef),
CodeBinaryOperatorType.BitwiseOr, anchorRightRef);
initMethod.Statements.Add(assign);
}

Next, we set the TabIndex property:

         //
            tab index...
assign = new CodeAssignStatement();
assign.Left =
            new CodePropertyReferenceExpression(fieldRef,
"TabIndex");
assign.Right = new
            CodePrimitiveExpression(tabIndex);
initMethod.Statements.Add(assign);

Finally, if we're given a set of properties through the otherProps, we have to walk the list and create new CodeAssignStatements for each one we find. We'll assume that each of the property values stored in the Hashtable is an object that extends CodeExpression. Obviously, if it doesn't, an exception will be thrown:

         //
            do we have other props?
if(otherProps != null)
{
// go
            through the other props
foreach(string propName in
            otherProps.Keys)
{
assign = new
            CodeAssignStatement();
assign.Left =
new
            CodePropertyReferenceExpression(
fieldRef,
            propName);
assign.Right =
(CodeExpression)otherProps[propName];
initMethod.Statements.Add(assign);
}
}
}

Now that we have the two helper methods in place, we can build AddInitializeComponentMethod. After a couple of local variable definitions, we define the method itself. We use a CodeMemberMethod to do this. This time, we want to mark the method so that it is a private instance method, and we also want to indicate that we cannot override the method.

The naming in MemberAttributes can be a bit strange, seemingly having no association with C#, C++, or VB .NET. To mark an object as "non-overridable", we use the Final flag. Although we're not doing it here, if we wanted to mark a member as "protected", we would use the Family flag. Use MSDN to help you discover what each of the flags do.

protected void AddInitializeComponentMethod(FormTemplate template,
CodeTypeDeclaration classDef)
{
CodeAssignStatement assign = null;
CodeExpression[] args = null;

// create the initialize component method...
CodeMemberMethod method = new CodeMemberMethod();
method.Name = "InitializeComponent";
method.Attributes = MemberAttributes.Private |
MemberAttributes.Final;
classDef.Members.Add(method);

For each of the bands in the template, we call AddCreateControlCall twice - once to create the Label control, and once more to create the actual editable control.

// create each new control...
foreach(ControlBand band in template.Controls)
{
AddCreateControlCall(band.FieldName, band.ControlType,
method);
AddCreateControlCall(band.LabelFieldName, typeof(Label),
method);
}

Before we can set the properties on the control, we have to call SuspendLayout. This method is used to suppress multiple layout events while consecutive layout operations are performed. For example, if you set the Location and then the Size property of a control, the control would undergo two "move" operations. If we SuspendLayout, the changes are made in memory, but the actual UI view is not changed until ResumeLayout is called, at which point the user will be able to see the changes:

// call suspend layout...
CodeMethodInvokeExpression callSuspend =
new CodeMethodInvokeExpression();
callSuspend.Method =
new CodeMethodReferenceExpression(
new CodeThisReferenceExpression(),
"SuspendLayout");
method.Statements.Add(callSuspend);

Now we can actually layout the controls themselves. This will involve multiple calls to AddLayoutControlCall.

The controls will be laid out in bands across the page. We'll start in the top-left hand corner of the form and position the Label control. We'll then move a little to the right and position the editable control. We'll then move back to the left-hand side of the form and position the next Label control under the previous label control, and so on. We'll use local variables x and y as a kind of cursor, keeping track of where each control should be positioned. Although this is a relatively simple layout algorithm, you're of course free to develop ones as complex as you wish. If nothing else, Code DOM is very flexible.

One other wrinkle in the layout code is that we need to insert padding between controls otherwise they'll all be bundled together and the form will look un-compelling. As part of this, we need to keep track of clientWidth, which is the adjust width of the client area of the container taking into account this padding value:

// go through each control, create and layout...
int tabIndex = 0;
int x = template.Padding.Width;
int y = template.Padding.Height;
int clientWidth = template.ContainerSize.Width -
(2 * template.Padding.Width);
foreach(ControlBand band in template.Controls)
{
// how tall is the band?
int bandHeight = 20;

The first control to add is the Label control. As well as the Location, Size, TabIndex and Name properties that AddLayoutControlCall adds by default, we also have to set the Text property. We do this by creating a new Hashtable and adding a new CodePrimitiveExpression object containing the required Text value to the Hashtable against the Text key:

// layout the label control...
Hashtable props = new Hashtable();
props.Add("Text",
new CodePrimitiveExpression(
string.Format("{0}:", band.Name)));
AddLayoutControlCall(band.LabelFieldName,
new Point(x, y),
new Size(template.LabelWidth, bandHeight),
tabIndex, false, props, method);

For the editable control, the extra properties will be defined in the PropertyItems collections of the band itself. Here, we want to give the caller the flexibility to provide basic data types, or to provide a type that extends CodeExpression (that way, the developer doesn't necessarily have to understand Code DOM in order to generate the code). We'll look for basic types of System.String or System.Int32 and if we find them, we'll instantiate new CodePrimitiveExpression objects containing the required values:

// configure properties for the field...
props = new Hashtable();
foreach(PropertyItem propItem in band.PropertyItems)
{
// create a new expression...
CodeExpression propValue =
new CodePrimitiveExpression(null);
if(propItem.Value is int)
propValue =
new CodePrimitiveExpression(propItem.Value);
if(propItem.Value is string)
propValue =
new CodePrimitiveExpression(propItem.Value);
if(propItem.Value is CodeExpression)
propValue = (CodeExpression)propItem.Value;

If we don't understand how to convert the property to a CodeExpression, we'll set the property value to a string:

// null?
if(propValue == null)
propValue =
new CodePrimitiveExpression(
string.Format("Cannot understand type {0}",
propItem.Value.GetType().ToString()));

// add...
props.Add(propItem.Name, propValue);
}

The x-position of the control will basically be set to whatever x was (the left edge of the form, plus padding) plus the width of the label. The width of the control will be whatever space is left:

            //
            layout the actual field control...
int controlX = x +
            template.LabelWidth +
template.Padding.Width;
int
            controlWidth = template.ContainerSize.Width -
controlX -
            template.Padding.Width;
AddLayoutControlCall(band.FieldName,
new Point(controlX, y),
new Size(controlWidth, bandHeight),
tabIndex, true, props, method);

After each band has been added, we move y down to the next band position:

            //
            move...
y += (bandHeight +
            template.Padding.Height);
}

That brings us to the end of phase two of InitializeComponent. Now we need to layout the form itself. The first thing we need to do is to add the comments indicating that this is where the form layout code starts.

// configure the form itself...
method.Statements.Add(new CodeCommentStatement(""));
method.Statements.Add(
new CodeCommentStatement(classDef.Name));
method.Statements.Add(new CodeCommentStatement(""));

The next thing we do is set the size of the form. We do this by setting the ClientSize property. This allows us to define the size of the control, but the actual size of the non-client area doesn't matter. The non-client area defines the captions and borders around a control. The size of the non-client area may well change from system-to-system as its user definable, plus the size may change from one Windows version to another. (Compare the appearance of windows in Windows XP compared to Windows 2000.) For this reason, it's a good idea to let Windows worry about this sort of thing. We tell it how big we want the client area and it works out how big the window needs to be to give us exactly the size client that we require.

         //
            set the client size...
assign = new
            CodeAssignStatement();
assign.Left = new
            CodePropertyReferenceExpression(
new
            CodeThisReferenceExpression(), "ClientSize");
args = new
            CodeExpression[2];
args[0] = new
CodePrimitiveExpression(template.ContainerSize.Width);
args[1]
            = new
CodePrimitiveExpression(template.ContainerSize.Height);
assign.Right
            =
new CodeObjectCreateExpression(typeof(Size),
            args);
method.Statements.Add(assign);

Now we need to create an array of controls to add to the form. InitializeComponent expects to see a single call to Controls.AddRange with an array of Control -derived objects as a parameter, rather than multiple Controls.Add calls.

To create an array, we use the CodeArrayCreateExpression object, passing in the type of the array that we want to create. Here, I've used a Type object to specify the type that I require, although of course I could have used a CodeTypeReferenceExpression if I didn't have direct access to the Type.

To Initialize that array, we call the Add method of its Initializers collection property. Add expects to see a CodeExpression, and here I'm passing in a reference to the editable control first, then the Label control:

// create an array of the controls...
CodeArrayCreateExpression controls =
new CodeArrayCreateExpression(typeof(Control));
foreach(ControlBand band in template.Controls)
{
controls.Initializers.Add(new
CodeFieldReferenceExpression(
new CodeThisReferenceExpression(),
band.FieldName));
controls.Initializers.Add(new
CodeFieldReferenceExpression(
new CodeThisReferenceExpression(),
band.LabelFieldName));
}

Once we've built the array, we need to call the AddRange method on the Controls property. This is the first time we've seen CodeMethodInvokeExpression used with parameters. Parameters are queued using the Parameters collection, and here we're passing in the array of Control objects that we just created.

Notice how we don't create a local variable and assign the control array to it. We want to create and populate the array inline, inside of the actual AddRange call. This is what InitializeComponent expects to see:

// invoke addrange...
CodeMethodInvokeExpression invoke =
new CodeMethodInvokeExpression();
invoke.Method = new CodeMethodReferenceExpression();
invoke.Method.TargetObject =
new CodePropertyReferenceExpression(
new CodeThisReferenceExpression(),
"Controls");
invoke.Method.MethodName = "AddRange";
invoke.Parameters.Add(controls);
method.Statements.Add(invoke);

After calling AddRange, we set the Name and Text properties of the container control to be the same as the name of the class:

         //
            set the name...
assign = new
            CodeAssignStatement();
assign.Left = new
            CodePropertyReferenceExpression(
new
            CodeThisReferenceExpression(), "Name");
assign.Right = new
            CodePrimitiveExpression(classDef.Name);
method.Statements.Add(assign);

//
            set the text...
assign = new
            CodeAssignStatement();
assign.Left = new
            CodePropertyReferenceExpression(
new
            CodeThisReferenceExpression(), "Text");
assign.Right = new
            CodePrimitiveExpression(classDef.Name);
method.Statements.Add(assign);

Finally, we call ResumeLayout :

// resume layout...
CodeMethodInvokeExpression callResume =
new CodeMethodInvokeExpression();
callResume.Method =
new CodeMethodReferenceExpression(
new CodeThisReferenceExpression(), "ResumeLayout");
method.Statements.Add(callResume);
}

Before running the project, add a call to AddInitializeComponentMethod to CreateCompileUnit in Generator. Run the project and click Generate Code. Here's some of the InitializeComponent method (I've removed portions to make the code more readable):

        private
            void InitializeComponent()
{
this._textBoxau_id = new
            System.Windows.Forms.TextBox();
this.labelau_id = new
            System.Windows.Forms.Label();
this._textBoxau_lname = new
            System.Windows.Forms.TextBox();
this.labelau_lname = new
            System.Windows.Forms.Label();
.
this.SuspendLayout();
//
// labelau_id
//
this.labelau_id.Name =
            "labelau_id";
this.labelau_id.Location = new
            System.Drawing.Point(5, 5);
this.labelau_id.Size = new
            System.Drawing.Size(100, 20);
this.labelau_id.TabIndex =
            0;
this.labelau_id.Text = "au_id:";
//
//
            _textBoxau_id
//
this._textBoxau_id.Name =
            "_textBoxau_id";
this._textBoxau_id.Location = new
            System.Drawing.Point(110, 5);
this._textBoxau_id.Size = new
            System.Drawing.Size(385, 20);
this._textBoxau_id.Anchor =
            ((System.Windows.Forms.AnchorStyles.Left
|
            System.Windows.Forms.AnchorStyles.Top)
|
            System.Windows.Forms.AnchorStyles.Right);
this._textBoxau_id.TabIndex
            = 0;
.
//
// MyDynamicControl
//
this.ClientSize =
            new System.Drawing.Size(500, 400);
this.Controls.AddRange(new
            System.Windows.Forms.Control[]
            {
this._textBoxau_id,
this.labelau_id,
this._textBoxau_lname,
this.labelau_lname,
this._textBoxau_fname,
.
this.labelcontract});
this.Name
            = "MyDynamicControl";
this.Text =
            "MyDynamicControl";
this.ResumeLayout();
}
}

To test out the code, we need a new Visual C# - Windows Application project. Create the new project now with a name of DynamicForms.

To the new project, create a new User Control called MyDynamicControl. Open the code editor for this new control class and copy the generated code from Forms Generator to the code editor. Replace all of the code in MyDynamicControl.

Build the project. You should see no errors.

Open the Designer for the Form1 in the DynamicForms project. In the Toolbox, you should see a MyDynamicControl component:

Put one of these controls onto Form1. You'll see the fields are properly positioned on the control.

In addition, if you open the Designer for MyDynamicControl, you'll see the same thing. If you look in the Properties window, you'll also notice that all of the dynamic controls are in place:

The proof of the pudding, though, is in whether or not we can edit the form. Add a Button control to the Designer, close and save the Designer and the code editor windows. When you open the Designer again, everything should be as you left it. If something was wrong with the InitializeComponent code, VS .NET would trash the contents and start again - effectively, all of our dynamic controls would have been deleted.

Adding Data Binding Support

Well, that's pretty cute, but we haven't really done anything that justifies all of the effort that we've put into this little project. What we need to do is add more functionality to the control to make the control more useful to the developer.

What we'll do in this section is add data binding support to the control. We'll add a DataSource property to the control that accepts a DataTable object containing the data to be bound. We'll also build a Position property that enables the caller, or the control, to change the position of the bound row in the DataTable. (For example, if we have ten rows in the table, setting Position to 5 will display the sixth row.)

The ultimate aim is to be able to set this DataSource property to a DataTable object and automatically have all of the editable controls display the appropriate value from the current row. We'll also be able to page through the rows.

We'll also need some protected helper methods and properties. CurrencyManager will return the System.Windows.Forms.CurrencyManager object that coordinates the binding between the controls, and provides a mechanism for us to change the current row. ClearBindings will remove any data bindings on the controls, and ConfigureBindings will bind the relevant property on the editable control to the appropriate column in the row. Specifically, for TextBox controls we'll bind to the Text property. For CheckBox controls we'll bind to the Checked property.

The first thing we'll do is to build the DataSource property and the associated _dataSource field that will ultimately contain the DataTable that we wish to bind to. The code to generate this property is a little complex, so here is the resulting code that we require (the complexity comes from the if statement, but I'll walk you through this as we build the code):

        public
            virtual System.Data.DataTable DataSource
{
get
{
return
            this._dataSource;
}
set
{
this._dataSource =
            value;
this.ClearBindings();
if ((this._dataSource !=
            null))
{
this.ConfigureBindings();
}
}
}

The first thing that AddDataSourceProperty has to do is to create the field that will ultimately contain the DataTable. This field will be called _dataSource, and we'll use the AddControlField helper method to add this member to the generated class:

protected void AddDataSourceProperty(FormTemplate template,
CodeTypeDeclaration classDef)
{
// create the field...
string dataSourceFieldName = "_dataSource";
AddControlField(typeof(DataTable),
dataSourceFieldName, classDef);

For use within the property, we'll need to refer to this field:

CodeFieldReferenceExpression field =
new CodeFieldReferenceExpression(
new CodeThisReferenceExpression(), dataSourceFieldName);

Defining the public DataSource property is simply a matter of creating a new CodeMemberProperty and adding it to the class' Members collection:

// create the property...
CodeMemberProperty prop = new CodeMemberProperty();
prop.Attributes = MemberAttributes.Public;
prop.Name = "DataSource";
prop.Type = new CodeTypeReference(typeof(DataTable));
classDef.Members.Add(prop);

This property, however, will be readable and writeable, so we need to queue statements into the GetStatements collection and into the SetStatements collection. Here's a statement that returns the _dataSource field back to the caller.

// get method...
prop.GetStatements.Add(new CodeMethodReturnStatement(field));

That's all we have to do with the "get" statements. The rest of this method creates the "set" portion. The first job is to set the _dataSource field to be equal to the value provided through the value argument. To reference an argument, we use the CodeArgumentReferenceExpression object, passing in the name of the argument as a string:

         //
            assign...
CodeAssignStatement assign = new
            CodeAssignStatement();
assign.Left = field;
assign.Right = new
            CodeArgumentReferenceExpression("value");
prop.SetStatements.Add(assign);

The next move is to call the ClearBindings method. If we set DataSource to null, this call will clear all of the editable controls so that the bound data does not appear:

         //
            clear the bindings...
CodeMethodInvokeExpression invoke =
new
            CodeMethodInvokeExpression();
invoke.Method = new
            CodeMethodReferenceExpression(
new CodeThisReferenceExpression(),
            "ClearBindings");
prop.SetStatements.Add(invoke);

"if" statements can be a little complicated to define. We define if statements by creating CodeConditionStatement objects. They have a number of parts that need to be configured. First off, they need a condition statement, supplied by setting the Condition property. This is usually a CodeBinaryOperatorExpression that can test for all of the standard operations, returning a Boolean result to the caller. This object itself takes a Left and Right property (the expressions that are part of the evaluation) and an Operator property.

Once the condition has been tested, the if statement will then either branch "true" or "false", depending on the result. The TrueStatements collection contains a queue of statements that will be executed if the condition is true, otherwise the statements queued in FalseStatements will be executed.

In our case, if the condition is true, we want to execute our ConfigureBindings method:

// do we have data?
CodeConditionStatement ifStatement =
new CodeConditionStatement();
CodeBinaryOperatorExpression ifCondition =
new CodeBinaryOperatorExpression();
ifCondition.Left = field;
ifCondition.Operator =
CodeBinaryOperatorType.IdentityInequality;
ifCondition.Right = new CodePrimitiveExpression(null);
ifStatement.Condition = ifCondition;
prop.SetStatements.Add(ifStatement);

// if we do, invoke...
invoke = new CodeMethodInvokeExpression();
invoke.Method =
new CodeMethodReferenceExpression(
new CodeThisReferenceExpression(), "ConfigureBindings");
ifStatement.TrueStatements.Add(invoke);
}

The ClearBindings method goes through each of the editable controls and calls the Clear method on the DataBindings collection. As the DataBindings collection is implemented on the Control base class, all controls have a DataBindings collection.

The wrinkle here is that the editable control defined in the band might not be data bound. If this is the case, the DataBindPropertyName property of the ControlBand object will be empty.

protected void AddClearBindingsMethod(FormTemplate template,
CodeTypeDeclaration classDef)
{
// create the method...
CodeMemberMethod method = new CodeMemberMethod();
method.Attributes = MemberAttributes.Family;
method.Name = "ClearBindings";
classDef.Members.Add(method);

// go through each band...
foreach(ControlBand band in template.Controls)
{
// bound...
if(band.DataBindPropertyName != string.Empty)
{
// get the databindings property...
CodePropertyReferenceExpression databindings =
new CodePropertyReferenceExpression();
databindings.TargetObject =
new CodeFieldReferenceExpression(
new CodeThisReferenceExpression(),
band.FieldName);
databindings.PropertyName = "DataBindings";

// clear...
CodeMethodInvokeExpression invoke =
new CodeMethodInvokeExpression();
invoke.Method =
new CodeMethodReferenceExpression(databindings,
"Clear");
method.Statements.Add(invoke);
}
}
}

ConfigureBindings is a very similar method. This time, however, we need to call the Add method of the DataBindings collection, not the Clear method. The Add method requires three parameters:

  • The name of the property to be bound to (set in the DataBindPropertyName property of the ControlBand object),
  • The data source, which in our case will be the DataSource property of the container control itself,
  • The column name, which will be the name of band itself.

These three items are queued into the Parameters collection of the CodeMethodInvokeExpression object:

protected void AddConfigureBindingsMethod(FormTemplate template,
CodeTypeDeclaration classDef)
{
// create the method...
CodeMemberMethod method = new CodeMemberMethod();
method.Attributes = MemberAttributes.Family;
method.Name = "ConfigureBindings";
classDef.Members.Add(method);

// go through each band...
foreach(ControlBand band in template.Controls)
{
// bound?
if(band.DataBindPropertyName != string.Empty)
{
// get the databindings property...
CodePropertyReferenceExpression databindings =
new CodePropertyReferenceExpression();
databindings.TargetObject =
new CodeFieldReferenceExpression(
new CodeThisReferenceExpression(),
band.FieldName);
databindings.PropertyName = "DataBindings";

// call...
CodeMethodInvokeExpression invoke =
new CodeMethodInvokeExpression();
invoke.Method =
new CodeMethodReferenceExpression(databindings,
"Add");
invoke.Parameters.Add(
new CodePrimitiveExpression(
band.DataBindPropertyName));
invoke.Parameters.Add(
new CodePropertyReferenceExpression(
new CodeThisReferenceExpression(),
"DataSource"));
invoke.Parameters.Add(
new CodePrimitiveExpression(band.Name));
method.Statements.Add(invoke);
}
}
}

The last thing to do so far as the navigation methods are concerned is to implement the CurrencyManager and Position properties.

CurrencyManager uses the BindingContext property of the container control to return a System.Windows.Forms.CurrencyManager object associated with the DataTable that the editable controls are bound to. CurrencyManager, among other things, implements a Position property. When this property is changed, the controls are re-bound to the new row.

First off, we define CurrencyManager as a "protected" property, using the MemberAttributes.Family flag. We don't want this accessible to a general consumer of the class, but an extending class may well need access to this.

protected void AddNavigationMethods(FormTemplate template,
CodeTypeDeclaration classDef)
{
// add the currency manager property...
CodeMemberProperty currency = new CodeMemberProperty();
currency.Name = "CurrencyManager";
currency.Type =
new CodeTypeReference(typeof(CurrencyManager));
currency.Attributes = MemberAttributes.Family;
classDef.Members.Add(currency);

We then create a reference to the BindingContext property of the container control:

// get hold of the binding context...
CodePropertyReferenceExpression bindingContext =
new CodePropertyReferenceExpression();
bindingContext.TargetObject =
new CodeThisReferenceExpression();
bindingContext.PropertyName = "BindingContext";

This property is accessed through an indexer. The parameter we pass into the indexer is the object that we're data bound to. In this case, we pass in the value of the DataSource property. The editable controls use this technique too, so the upshot of this is that we'll get the same BindingManager -derived class (specifically, a CurrencyManager) as the editable controls. We use a CodeIndexerExpression to access values in an array or indexer. This resolves to square bracket notation in C#, and parentheses in VB .NET:

// datasource index...
CodeIndexerExpression indexer = new CodeIndexerExpression();
indexer.TargetObject = bindingContext;
indexer.Indices.Add(new CodePropertyReferenceExpression(
new CodeThisReferenceExpression(), "DataSource"));

This property only has a single get statement, which returns the value accessed through the indexer:

// add the method to the property...
currency.GetStatements.Add(new CodeMethodReturnStatement(
new CodeCastExpression(typeof(CurrencyManager),
indexer)));

The next property we need is Position. This property will be publicly accessible:

         //
            create the property...
CodeMemberProperty property = new
            CodeMemberProperty();
property.Name =
            "Position";
property.Type = new
            CodeTypeReference(typeof(int));
property.Attributes =
            MemberAttributes.Public;
classDef.Members.Add(property);

This property simply has to abstract the Position property of the CurrencyManager object returned through the new CurrencyManager property:

// position ref...
CodePropertyReferenceExpression positionRef =
new CodePropertyReferenceExpression();
positionRef.TargetObject =
new CodePropertyReferenceExpression(
new CodeThisReferenceExpression(), "CurrencyManager");
positionRef.PropertyName = "Position";

// get/set...
property.GetStatements.Add(
new CodeMethodReturnStatement(positionRef));
property.SetStatements.Add(
new CodeAssignStatement(positionRef,
new CodeArgumentReferenceExpression("value")));
}

Using the Control

To call these methods, edit Generator.CreateCompileUnit and add calls to AddDataSourceProperty, AddClearBindingsMethod, AddConfigureBindingMethods and AddNavigationMethods :

// add the stuff...
AddBandFieldsAndProperties(template, classDef);
AddConstructor(template, classDef);
AddInitializeComponentMethod(template, classDef);
AddDataSourceProperty(template, classDef);
AddClearBindingsMethod(template, classDef);
AddConfigureBindingsMethod(template, classDef);
AddNavigationMethods(template, classDef);

Run the project, generate the code, and copy it into the DynamicForms project ready for use. You should see no change to the layout of the control - except that any customizations that you make will be lost when you copy in the new code.

To test this, we now need to load some data into a DataTable and set the DataSource property on the MyDynamicControl object.

To Form1, add a TrackBar control.

Double-click on the TrackBar control to create a new Scroll event handler. Add this code:

private void trackBar1_Scroll(object sender, System.EventArgs e)
{
myDynamicControl1.Position = trackBar1.Value;
}

To the Load event of the form, add this code. You'll also need to add a reference to System.Data.SqlClient at the top of the code editor:

      private
            void Form1_Load(object sender, System.EventArgs e)
{
// create
            a datatable...
SqlConnection connection = new
            SqlConnection(
"integrated security=sspi;data
            source=(local);initial catalog=pubs");
DataTable table = new
            DataTable();
SqlDataAdapter adapter =
new
            SqlDataAdapter("select * from authors",
            connection);
adapter.Fill(table);
adapter.Dispose();
connection.Close();

//
            set...
myDynamicControl1.DataSource = table;
trackBar1.Maximum
            = table.Rows.Count;
}

Run the project and the data will "magically" appear in the fields. You'll be able to scroll through the fields using the TrackBar control.

Generating the Control Dynamically

In this case study, we've concentrated on creating the form for editing and customization in VS .NET. As we mentioned, it's possible to use the same solution to generate the source code, compile and instantiate generated objects at run time.

We're running out of space in this case study, so I won't show you how this works. The source code download contains code to do this. Look for a method called CreateInstance in the Generator class. This takes a template, compiles it and providing the compiler returns no errors, a new Control -derived class is instantiated and returned to the user.

Where Next?

In this case study, we didn't look too hard at adding cool functionality to the generated forms. Ideally, for production purposes, the generator should be adapted such that in a single method call to Generator, you get back a class capable of communicating with the business tier (or directly to the data access layer, depending on your particular preference), loading data, and saving data back. In addition, richer controls could be implemented - for example LinkButton controls that brought up child windows displaying graphics, Web pages, or other data and so on.

The example here is relatively limited. As we've mentioned about a zillion times, Code DOM is very powerful and very verbose. I've filled almost 12,000 words just showing you how to create a basic form. However, once you get the hang of Code DOM and learn where the relevant expressions and statements are in its own object model, creating really rich method and property sets on your generated classes becomes much easier.

One thing we haven't mentioned is the potential of using a base class that implements some of this other functionality. Although we've inherited from UserControl (and in the code sample, you'll see that we can inherit from Form), you can inherit from any class you create, so long as they themselves inherit from UserControl or from Form. This would certainly alleviate the need to generate properties like CurrencyManager and the DataSource property each time.

Summary

In this case study, we took a look at how to generate a new managed code source file containing a control class that could be edited in the Visual Studio .NET Forms Designer. Although we looked at this solution from the perspective of generating a source code file for design-time compilation into an application, the same Code DOM technology could be used to generate, compile, and execute on the fly from running code.

After building classes capable of representing a basic form template, we looked at how we can walk through the template creating a new class. Most of the work was done in a method called InitializeComponent. This is a method that VS .NET expects to see, and we were careful to exactly mimic the code that VS .NET itself would generate. When we pasted the code into a new project, we were able to see the controls on the Designer, and also at run time,

To round off the discussion, we added various data binding properties to the new control, including DataSource.

Case Study Information
Author | Matthew Reynolds
Technical Editor | Adam Ryland
Project Manager | Helen Cuthill
Reviewers | Andrew Krowczyk, Saurabh Nandu, Daniel Cazzulino, Adam Ryland

If you have any questions or comments about this case study, or suggestions for further pieces, please contact the technical editor. If you wish to talk about the case study with other subscribers, please visit our forum here.


A .NET Data Provider

A .NET Data Provider
http://www.csharptoday.com/content.asp?id=1760
Article Rating: not yet rated
Published on March 25, 2002
Abstract
In this case study, Todd Clemeston covers what a .NET data provider is and examines the reasons why you might consider writing one. Todd also summarizes the guidelines that a custom provider needs to follow in order to “play nicely” in the .NET framework. Once the basics are covered, Todd will walk you through the implementation of a basic data provider that supports querying of the Windows Management Instrumentation (WMI) API. To round out the piece, Todd will demonstrate the use of the custom data provider by covering a simple.NET Windows application that displays the results of WMI queries in a ListView.
Case Study

In this case study, Todd Clemetson covers what a .NET data provider is and examines the reasons why you might consider writing one. Todd also summarizes the guidelines that a custom provider needs to follow in order to "play nicely" in the .NET framework. Once the basics are covered, Todd will walk you through the implementation of a basic data provider that supports querying of the Windows Management Instrumentation (WMI) API. To round out the case study, Todd will demonstrate the use of the custom data provider by covering a simple.NET Windows application that displays the results of WMI queries in a ListView.

Before continuing please take note of the following environmental requirements for the purposes of this discussion, as well as for working with the sample code accompanying the case study:

  • .NET RTM
  • Visual Studio .NET RTM (or a text editor)

What is a .NET Data Provider?

A .NET data provider is a set of specialized classes that implement standard ADO.NET interfaces and serve as a bridge between a data source and .NET. Two data providers are included with the .NET framework: SQL Server and OLE DB. The SQL Server data provider connects to SQL Server Versions 7.0 and 2000, and the OLE DB data provider connects to OLE DB data sources. Additionally, Microsoft has made an ODBC data provider available that you may download. As its name implies, this data provider connects to ODBC based data sources. If you wish to download this provider, there is a link at the end of this case study.

Why would you ever need to write your own data provider within .NET? With the addition of the ODBC data provider mentioned above and the various XML and File IO classes that .NET provides, it is uncommon to find yourself in a situation where writing a custom data provider is the only solution. One major reason you might be forced into writing a custom data provider is if the .NET framework does not supply a data provider for a particular data source. In this case, the underlying data source technology may be old or so new that one hasn't been provided (maybe you are the designer of the data source!) A more interesting reason for writing your own data provider might be that you want all data access to go through a set of standard interfaces (i.e. ADO.NET interfaces). Writing a .NET Data Provider to encapsulate a data source allows your data access code to work with standard interfaces and not be the least bit concerned that the underlying data source is Joe's 3rd party data store and not SQL Server or Oracle. Along the same lines, having a standard set of interfaces also plays nicely for those .NET GUI components that integrate with ADO.NET providers (the DataGrid for example). Finally, creating a .NET Data Provider for a data source also helps you integrate with 3rd party products that use ADO.NET such as Crystal Reports for .NET, etc.

Guidelines for a .NET Data Provider

There really is no comprehensive specification on creating data providers within .NET. The help that ships with the RTM (see "Getting Started with a .NET Data Provider Implementation within VS.NET") defines the ADO.NET interfaces, offers suggestions for overloaded constructors, and generally gets you off to a good start. It also provides source code that implements a basic provider that may help you get the gist of what each method and property is intended to do. I have found one other article on the internet that describes how to write a custom .NET data provider. You may find the link to it at the end of this case study. Additionally, Wrox has published a book entitled "Professional ADO.NET" that does a good job covering .NET data provider construction. My recommendation is to take the "shotgun" approach and read as much material on the subject as possible. Study the help files that ship within .NET, visit the links at the end of this case study, and of course read this case study!

In the next few sections, I will attempt to provide some general guidelines to follow when writing .NET data providers. The intent of these sections is not to regurgitate the help that ships with .NET. Instead these sections highlight some guidelines that are of the most use. When we get down to the details of implementing a custom provider, I will go into specifics that you probably won't glean from the .NET help.

Namespaces

Let's start off with a relatively simple guideline. Actually, this should really be a requirement: data providers should use a unique namespace. In other words, don't put your custom .NET data provider in another data provider's namespace. For reference, the ADO.NET namespaces are listed below:

Namespace | Description
System.Data | Defines common interfaces for ADO.NET
System. Data. Common | Classes that are common across the .NET data providers
System. Data. SqlClient | Classes specific to the SQL Server data provider
System. Data. OleDb | Classes specific to the OLE DB data provider

Additionally, namespaces should follow the CompanyName.TechnologyName namespace documented in the .NET help. The WMI data provider we will write in this case study will use the CsharpToday.DataProviders.WmiData namespace.

Minimal Interfaces

The minimum set of interfaces that a .NET data provider should implement are listed below. We will cover each one of these interfaces in detail when we get into the source code that accompanies this case study.

Interface Name | Description
IDbConnection | A connection to a data source.
IDbCommand | A command or query used against a data source.
IDataReader | Represents forward - only/read-only access to data from the underlying data source.
IDbDataAdapter | Provides the link between a data source and a .NET Dataset.

Interfaces play an important role in flexibility. Writing code against interfaces gives programmers the ability to swap out chunks of code with little or no ill-effects to the source code using those interfaces. The power and flexibility of interfaces is what drives the requirement that .NET data providers implement standard ADO.NET interfaces. This requirement affords developers using your provider the choice of writing code against the standard ADO.NET interfaces listed above, or against the concrete classes that a particular data provider offers (SqlConnection, SqlCommand, etc in the case of the SQL Server Data Provider). In cases where the data access layer within a framework must support a number of current and future data source technologies, it makes sense for the layer to work with the interfaces supplied by ADO. NET. Other projects may not require this much flexibility. The nature of the application may dictate a single database platform, and the programmer may be safe to code against concrete classes. In summary, the requirement that all data providers implement a standard set of interfaces is important in that it leaves the decision of flexibility up to the developer using your provider.

Interface "No-ops"

You are not forced to implement every property and method that an ADO.NET interface specifies. For instance, if the implementation of your data provider will not support the concept of a Transaction, it is acceptable to throw a NotSupportedException. It is up to the author of the data provider to implement methods that make sense and to take appropriate action when methods and properties don't apply. The bottom line: use common sense ! As you will see in subsequent sections, the data provider that we will examine in this case study throws exceptions for areas that don't apply to the WMI API. When we encounter these areas, I will provide you with an explanation of what you would have to do to implement the specific functionality.

Compiling

We will close the guideline section with a relatively simple one. Each Data Provider should exist in its own assembly. Additionally, in order to implement the required interfaces, each assembly should reference the System.Data.dll. If you use VS.NET, simply using the Class library wizard helps you adhere to this guideline as it references this assembly.

Creating a Custom Provider

So far we have covered the basics of .NET data providers. You now know what type of data providers are included in the .NET framework, you understand the two major reasons for writing a custom data provider, and you had a quick overview of the major guidelines to keep in mind when implementing a data provider. With the basics out of the way, we are almost ready to take a look at a sample data provider. Before doing so, let's examine the process of writing a provider from a very high level by breaking the process into the following steps:

  1. Select Interfaces and determine applicability of Functionality - Aside from the standard interfaces we covered earlier, there a few other interfaces you might wish to implement (e.g. IDbDataParameter). Examine your requirements and determine the interfaces you need to implement. Within each interface, highlight methods applicable to your provider and note methods or properties that do not apply.
  2. Determine a namespace - Determine what namespace your data provider will occupy.
  3. Create the Assembly - This is somewhat of a no-brainer. Simply use the VS.NET IDE to create a class library. By default the wizard includes a reference to System.Data.dll.
  4. Implement the interfaces - In this step, you write the necessary code to implement the interfaces you selected in step 2. A few words of wisdom follow. If you are the type of programmer who compiles often and can't stand compiler errors as you code, consider providing "thin" implementations of all the interfaces before you start any hardcore coding. This suppresses compiler errors, for instance, when implementing the IDbCommand class which makes reference to the Connection class. Also, after providing a "thin" implementation of each interface, it generally makes sense to implement the interfaces in the following order: IDbConnection, IDbCommand, IDataReader, IDbDataAdapter, however it really doesn't make a difference which order you implement them in.
  5. Test - Had to throw this one in here. Be sure to test your data provider extensively. I recommend installing NUnit which is a freely available unit-testing framework from SourceForge. I provide a link to NUnit at the end of the case study.

The remainder of this case study focuses on the creation of a custom data provider that wraps a piece of the WMI API for querying and retrieving WMI results. Once we cover implementing the provider, we will look at a sample Windows application that uses the WMI provider to query a server. I kept a few things in mind when choosing a data source technology for this case study's custom data provider: I wanted to make sure I didn't use an example that another author has used, and I wanted to stick with a data source technology that was relatively simple to understand and didn't take away the focus of the case study. The WMI API fits both of these requirements. One of the disadvantages of choosing WMI as a data source is that it may make some aspects of creating an ADO.NET data provider seem simpler than they really are. I will speak more on this issue when we implement the IDataReader interface. One last note: the WMI provider we will create requires a reference to the Microsoft WMI Scripting V1.1 Library. If you don't have this COM object on your machine, you may download it from Microsoft's web site. A link to downloading the WMI SDK is provided at the end of this case study.

What is WMI?

In short, WMI is a management architecture that lets you remotely manage computer systems. Some of the things you can do with WMI that pertain to the WMI provider we will examine include listing all of the processes running on a remote machine or listing all the NTFS partitions on a server that have a specific percentage of free disk space. You may also perform operations such as starting/stopping services or rebooting machines. There is much more to the WMI API than I would care to go into at this point. If you would like more details on WMI, follow the WMI link at the bottom of this case study. The case study describes WMI and its architecture. Alternatively, you can read the case study "WMI for .NET" on C#Today (http://www.csharptoday.com/content.asp?id=1807&WROXEMPTOKEN=618658Zezi9SpDFSZgQps5Xv6u). Now that you have a basic understanding of WMI, let's examine some fictitious criteria that our WMI provider must meet:

  • Connect - The WMI data provider must be able to connect to a server that supports WMI. We will create a class named WmiConnection that manages this connection. This class will implement the ADO.NET IDbConnection interface.
  • Query - The custom provider must also support querying a server for WMI data. We will create a class named WmiCommand that encapsulates a WMI Query. This class will implement the ADO.NET IDbCommand interface.
  • Moving through Results - We also need a mechanism for calling programs to iterate through the result set that is returned from WMI. We will create a WmiDataReader class to handle this functionality. This class will implement the ADO.NET IDataReader interface.
  • Must support the .NET Dataset - The provider must support populating a dataset with the results of a WMI query. The WmiDataAdapter class will handle this requirement. This class will implement the ADO.NET IDataAdapter interface.
  • Starting/Stopping a Service - For WMI queries against Services running on a WIN32 server (i.e. SELECT * FROM WIN32_SERVICE ) we want users of the provider to be able to change the State of the service and effectively stop or start a service. The WmiDataAdapter class will handle this piece of functionality through the standard Update method exposed within the IDataAdapter interface.

Now that we have defined the functionality for the WMI provider, let's dive into some code. The first interface we will implement is the IDbConnection interface.

WmiConnection

As mentioned earlier, the IDbConnection interface defines a connection to a data source. We will implement this interface through a class called WmiConnection. The WmiConnection class manages the connection to a server that supports WMI. The details of the IDbConnection interface are depicted below. As we implement each method and property, I will provide a description and walk you through the accompanying source code.

In the guidelines section at the beginning of this case study, I mentioned that it is safe to throw exceptions when interface methods are not applicable. We will see this guideline in action within the WmiConnection class because it doesn't have the concept of a timeout, a database, or transactions. As outlined earlier, we will handle these situations by throwing a NotSupportedException. In cases where properties are not applicable, we will perform some "magic" by returning a consistent value and implementing an empty Set method.

Let's take a look at a partial class definition for WmiConnection:

public class WmiConnection : IDbConnection
{
/// <summary>
/// Represents the state of the WmiConnection.
/// Possible values are: Open, Closed, Broken,
/// Connecting, Executing and Fetching.
/// </summary>
private ConnectionState _connectionState;
/// <summary>
/// The connection string used to connect to
/// the server. Although I named this
/// connection string, in our case it is the name of the server
/// that we will be connecting to in order
/// to issue WMI queries.
/// </summary>
private string _connectionString;
/// <summary>
/// Used to connect to a server that can
/// process WMI requests.
/// </summary>
private WbemScripting.SWbemLocator _locator;
/// <summary>
/// WMI object that represents an authenticated
/// connection to a namespace. This object
/// is exposed as an internal property for
/// Reader and Adapter classes to use for
/// the purpose of executing WMI queries.
/// </summary>
private WbemScripting.ISWbemServices _services;
.
.
.

}

Note that the definition of WmiConnection includes implementing the IDbConnection interface. For a description of each member variable used within the Connection class, examine the comments that precede each variable declaration. Let's examine the properties that the WmiConnection class contains.

Properties

This section documents the properties that make up the WmiConnection class. Properties that have IDbConnection in parenthesis indicate that the IDbConnection interface dictates the specific implementation.

State (IDbConnection) - The State property describes the current state of the connection through the ConnectionState enumeration. The different states in this enumeration are listed below:

Value | Description
Broken | The underlying connection is broken and may only occur once the connection is open.
Closed | The underlying connection to the data source is closed.
Connecting | The connection to the data source is in progress.
Executing | The connection to the data source is in the process of executing a command.
Fetching | The connection is in the process of retrieving data from the data source.
Open | The underlying connection to the data source is open.

Note that this property is read-only as it only implements the get function. The underlying _connectionState variable is set internally by the connection object only. We certainly don't want code outside of the connection class modifying the state of the connection.

public ConnectionState State
{
get
{
return _connectionState;
}
}

ConnectionString (IDbConnection) - The ConnectionString property defines how to
            connect to a particular data source. If you have written code to
            access a RDBMS such as SQL server or Oracle, you most likely have
            had to specify a connection string in order to connection to the
            database server. I chose to keep the connection string simple for
            the WMI data provider. It is simply used to specify the name of the
            server to connect to. Note that we could extend this property to
            support a semi-colon separated list of parameters which also
            includes the user name and password. In this case we would have then
            had to parse the connection string when a calling program invoked
            the connection's Open method. Note
            that this property is exposed as a read/write property.

public string
            ConnectionString
{
get
{
return
            _connectionString;
}
set
{
_connectionString =
            value;
}
}

ConnectionTimeout (IDbConnection) - The ConnectionTimeout property represents the
            amount of time that expires before a connection attempt fails and an
            error is generated. Although the ConnectionTimeout property does not apply to
            the WmiConnection connection class, we
            still must provide a body for it. Here we are just returning 0 when
            the value is requested. The set implementation below has an empty
            body and does not maintain any state.

public int
            ConnectionTimeout
{
get
{
return 0;
}
set
{
}
}

Database (IDbConnection) -
            The Database property is the name of the database being referenced
            by the connection. The IDbConnection
            interface defines this property as read-only. This property is
            applicable to relational databases, but is not applicable to the WMI
            provider. The implementation will simply return an empty string.

public string
            Database
{
get
{
return "";
}
}

WmiServices - The WmiServices property is not part of the
            IDbConnection interface, but needs to
            be exposed by the WmiConnection class
            so that other WMI data provider classes (namely the WmiCommand and WmiDataAdapter) can leverage it to query for
            WMI data. The WmiServices property is declared as the ISWbemServices interface. This interface
            exposes a method called ExecQuery that
            takes a WMI query and returns a result set. We will cover this
            method and the result set in more detail later on. Note that this
            property is marked as internal within the scope of the WmiConnection class. The internal access
            modifier makes a member publicly available to all code within an
            assembly, but hides it from classes outside of the assembly. We
            don't want the connection class making the underlying WMI API
            available.

internal ISWbemServices
            WmiServices
{
get
{
return
            _services;
}
}

Methods

The following methods define the behavior of the WmiConnection class. In the case of the
            properties above, methods marked with the IDbConnection interface in parenthesis imply
            that the method is a requirement of the IDbConnection interface.

CreateCommand (IDbConnection) - The CreateCommand method is essentially a
            factory method that creates the appropriate Command object associated with a provider
            connection class. A factory method is a creational design pattern
            that is responsible for creating and returning objects that derive
            from a common abstract type. The WMI provider's CreateCommand method creates and returns
            objects that implement the IDbCommand
            interface. Even more important is the fact that the CreateCommand
            method returns an instance of a WmiCommand object, the concrete command
            object that the WMI provider knows how to work with. We will dive
            into the details of the WmiCommand
            class in the next major section. Here is the CreateCommand implementation for the WMI
            provider:

public IDbCommand
            CreateCommand()
{
return new WmiCommand();
}

Open (IDbConnection) - The
            IDbConnection Open method is responsible for opening up a
            connection to a data source given the ConnectionString property of the Connection object. The WmiConnection. Open () method below connects to a server
            that will respond to WMI requests. I kept the call to the WMI
            locator object method ConnectServer
            simple and only pass in the name of server by passing the entire
            connection string. The implementation for the Open method is shown below:

public void Open()
{
if(this.State ==
            ConnectionState.Open)
{
return;
}
if(ConnectionString !=
            "")
{
_services =
_locator.ConnectServer(ConnectionString,
null, null, null, null, null,
0, null);
this._connectionState =
            ConnectionState.Open;
}
else
{
throw new
            Exception("Invalid Connection String.");
}
}

This method first checks the state of the connection before
            opening it. If the calling program inadvertently tried opening a
            connection that is already open, we don't want to run through the
            process of re-connecting. Depending upon your implementation, you
            may wish to throw an exception, refresh the connection by simply
            closing and opening the connection, or do nothing at all since it is
            already open. Once the Open method
            verifies that the connection string is valid, it connects to the
            server and sets the state of the connection to open. Here is the
            signature to the ConnectServer
            method:

ConnectServer(Server, Namespace, User,
            Password, Locale, Authority, SecurityFlags,
            NamedValueSet)

Argument | Description
Server | Specifies the name of the computer to connect to.
Namespace | Specifies the namespace with which you log on.
User | The user name to use when connecting to the server.
Password | The password to use when connecting to the machine.
Locale | Specifies the locale from which information should be retrieved.
Authority | Specifies the authentication to be used. Should only be used with connections to remote WMI servers.
SecurityFlags | Allows you to pass flag values to ConnectServer. Passing 0 tells the ConnectServer method to return only after a connection to the server has been made.

Close (IDbConnection) - The
            IDbConnection.Close method must
            release resources that are associated with the connection. The WMI
            provider's simple implementation sets the services and locator
            objects to null in addition to setting the state of the connection
            to Closed.

public void Close()
{
_services =
            null;
_locator = null;
this._connectionState =
            ConnectionState.Closed;
}

Constructors -At a minimum, the Connection class should implement a default
            constructor in addition to an overloaded one that takes a string
            that represents the connection string of the connection. Both
            constructors should also be sure to initialize the ConnectionState to Closed. Here is the implementations for
            WmiConnection 's two constructors:

public WmiConnection
            ()
{
_connectionState =
            ConnectionState.Closed;
}

public WmiConnection(string
            strConnectionString)
{
_connectionState =
            ConnectionState.Closed;
this.ConnectionString =
            strConnectionString;
}

Unsupported Methods - The WMI provider will not be
            supporting transactions or the concept of changing a database with
            them. Because of this, it is important to throw NotSupportException's for these methods to
            let programs calling the provider know this is the case. I will,
            however, cover the intent of these methods below.

The BeginTransaction method is
            responsible for creating and returning a data provider specific
            transaction object for starting a database transaction. The
            transaction object that gets returned from the BeginTransaction method must support the
            IDbTransaction interface shown
            below:

If you need to support Transactions in your provider, you would
            also need to provide a provider specific implementation of the IDbTransaction interface. The Connection property maintains a reference to
            the data provider specific connection object. The Isolation level is
            an enumeration that describes how locking behavior will occur within
            the transaction. For the different types of isolation levels, look
            up IsolationLevel in the .NET help files. As you probably guessed,
            the Commit method commits the
            transaction and the Rollback method
            rolls the transaction back. If you need to have support in your
            custom provider, you will also need to provide an implementation for
            the Transaction property of your command object.

The other method that the WmiConnection class will not support is the
            ChangeDatabase method; WMI does not
            have the concept of a database. The purpose of the ChangeDatabase method is to change the
            database for a connection object that already has an open
            connection. Although this method really doesn't apply, we could
            extend the WMI provider and equate the ChangeDatabase method to changing the server
            name that we connect to. Under this scenario, the provider would
            have to actually disconnect from the existing server and
            re-establish its connection to the new server.

WmiCommand

Now that we have a method for connecting to a server, we will
            move onto the next requirement which calls for querying a server
            using the WMI Query language. We will implement the WMI query
            functionality by implementing a class called WmiCommand. This class implements the IDbCommand interface. A Command in ADO.NET
            represents a request to a data source. The SqlClient data provider
            in .NET, for instance, has a SqlCommand class that encapsulates the
            concept of a SQL command. The WmiCommand object will encapsulate a WMI SQL
            command. The IDbCommand interface is
            shown in the picture below:

Here is a partial class definition of the WmiCommand that shows the member variables
            of the class. The comments within the class definition describe what
            each member does.

public class WmiCommand :
            IDbCommand
{
/// <summary>
/// The WMI SQL used to
            obtain data.
/// </summary>
private string
            _wmiCommand;
/// <summary>
/// This represents how the
            results
/// of a command will be applied to a DataRow
///
            when the Update method of the DataAdapter
/// is invoked.
///
            </summary>
private UpdateRowSource _updRowSource;
///
            <summary>
/// Represents a connection to a server
///
            that serves up WMI data.
/// </summary>
private
            WmiConnection _connection;
.
.
.
}

Properties

The following is a list of the properties that the WmiCommand class contains.

CommandText (IDbCommand) -
            The CommandText property represents
            the actual query string that gets executed against the data source.
            In the case of the SQL Server and OLE DB data providers, this
            property represents the SQL to be executed against a particular
            RDBMS. The CommandText property for
            the WMI provider is the actual WMI SQL query that will be executed
            against the server.

public string
            CommandText
{
get
{
return
            _wmiCommand;
}
set
{
_wmiCommand =
            value;
}
}

CommandTimeout (IDbCommand)
            - Represents the amount of time to wait before stopping the
            execution of a command. We are not going to support the concept of a
            timeout, so we just simply return 0 to the caller and provide an
            empty set function.

public int
            CommandTimeout
{
get
{
return
            0;
}
set
{
}
}

CommandType (IDbCommand) -
            The CommandType property represents
            the type of command to be executed through the CommandType enumeration. The CommandType enumeration defines the
            following possible types: StoredProcedure, TableDirect, and Text. The data providers in the .NET
            framework support all three of these enumerations. Setting this
            property to StoredProcedure specifies
            that the CommandText property is the
            name of a stored procedure. Setting this property to TableDirect specifies that the CommandText property is the name of a table
            in a database. Setting the property to Text specifies that the CommandText is a query (e.g. SQL or WQL in
            the case of the WMI provider). Because we are only supporting the
            Text option, the implementation below always returns text and only
            supports the setting of text.

public CommandType
            CommandType
{
get
{
return
            CommandType.Text;
}
set
{
if(value !=
            CommandType.Text)
{
throw new NotSupportedException("This
            provider only supports text commands.");
}
}
}

Connection (IDbCommand) -
            The Connection property is simply the
            connection that the Command object is
            associated with. Its implementation is below:

public IDbConnection
            Connection
{
get
{
return
            _connection;
}
set
{
_connection =
            (WmiConnection)value;
}
}

Parameters - The Parameters represents a collection of
            IDataParameter objects used to specify
            the parameters of a SQL statement or stored procedure. The WMI
            custom provider will not support parameters, so it just simply
            returns null in its implementation. If you need to implement
            parameters, you must create a ParameterCollection class that implements
            the IDataParameterCollection. You will
            also need to create your own DataParameter class implementing the IDataParameter interface.

public IDataParameterCollection
            Parameters
{
get
{
return null;
}
}

Transaction (IDbCommand) -
            The Transaction property represents the transaction which the
            command will execute under. The WmiCommand does not support transactions, so
            we simply return null when asked for a transaction and throw an
            exception when the caller attempts to specify a transaction.

public IDbTransaction
            Transaction
{
get
{
return
            null;
}
set
{
throw
new NotSupportedException("The
            WmiCommand object doesn't " +
"support
            transactions.");
}
}

UpdateRowSource (IDbCommand)
            - Lets a calling program specify the method with which values
            returned from a data source are applied to a row within an ADO.NET
            Dataset. This property comes into play within the Update method of the DataAdapter. The WMI
            provider doesn't support return (or output) values in its Adapter's
            Update method, so the enumeration is
            set to None in
            the constructor and may not be changed. If a calling program
            attempts to change this to any other value, the provider will throw
            an exception.

For the curious, the following table describes the different
            values of the UpdateRowSource
            enumeration, and how they affect the behavior of a command used with
            a DataAdapter:

Name | Description
Both | The output parameters and the first row returned get mapped to the changed row in the Dataset.
FirstReturnedRecord | Data in the first returned row is mapped to the changed row in the DataSet.
None | Returned parameters or rows are ignored.
Output Parameters | Output parameters are mapped to the changed row in the DataSet.

public UpdateRowSource
            UpdatedRowSource
{
get
{
return
            _updRowSource;
}
set
{
if(value !=
            UpdateRowSource.None)
{
throw
new
            NotSupportedException("The WmiCommand " +
"object only supports
            None as an " +
"UpdateRowSource
            type.");
}
}
}

WmiCommand Methods

There are really only two methods of importance for the WmiCommand class. The rest of the methods
            don't apply to the WMI provider, although I will go over them so you
            understand what each means.

ExecuteReader (IDbCommand) -
            ExecuteReader is a factory method that
            returns a DataReader associated with
            the Command class. Remember than an
            IDataReader provides forward-only
            access to data from a data source. The ExecuteReader implementation is shown
            below:

public IDataReader
            ExecuteReader()
{
if(this.Connection == null)
{
throw
            new Exception("Cannot execute reader. The WmiConnection "
            +
"object is not valid.");
}
if(this.Connection.State !=
            ConnectionState.Open)
{
throw new Exception("Cannot execute
            reader. The connection " +
"to the server is
            invalid.");
}
// create a reader
WmiDataReader reader =

new WmiDataReader(this._connection);
// the object set that
            is returned as a result of the
// query.
ISWbemObjectSet
            ObjSet = null;
// execute the query as WMI Query Language

ObjSet =
            _connection.WmiServices.ExecQuery(_queryCommand,
"WQL", 0x10,
            null);
// set the readers data to the object set
reader.Data =
            ObjSet;
// return the reader to the caller
return
            reader;

}

The above code is not overly complex. After verifying that the
            connection is valid, the method creates a WmiDataReader object, executes the WMI
            Query, and sets the WmiDataReader.Data
            property to the result set of the WMI query. We will cover
            the details of the DataReader
            implementation in the next major section, but essentially the DataReader will use the Data property to iterate through the result
            set.

The WMI ExecQuery method
            highlighted in the code above takes the following four
arguments:

  1. WMIQuery - The first specifies the WMI query to execute. The
              syntax of this query follows WQL (WMI SQL), which is a subset of
              the ANSI standard SQL language with WMI-specific extensions.

  2. QueryLanguage - The second argument is the query language to
              use, and must always be "WQL" if anything is specified.

  3. QueryBehavior - The third argument specifies the behavior of
              the query and determines if this call returns immediately. We have
              specified WbemFlagReturnImmediately
              (0x10) for this option.

  4. NamedValueSet - The final argument is NamedValueSet. We have left this as null,
              but you can pass a SwbemNamedValueSet object to give
              contextual information that can be used by a provider servicing
              the request. If a provider supports this argument, its
              documentation will have specified the values, data types, allowed
              values, and semantics.

I mentioned earlier that using WMI as the data source made some
            aspects of authoring a data provider seem simpler than they normally
            are. This is indeed the case for the ExecuteReader() method. When implementing
            the ExecuteReader method, it is common
            to have to write somewhat complex parsing logic to break a query
            apart. In the implementation above, we simply forward the query to
            the WMI COM object.

ExecuteScalar (IDbCommand) -
            The IDbCommand. ExecuteScalar method is meant to be an
            efficient means to access data that is in the form of a single value
            within a single row. A good example of a situation where this method
            is helpful is in the case where a record count from a SQL select
            statement is required. The result of a SELECT
            COUNT(*) FROM Table1 SQL query returns a single column and
            single value. The ExecuteScalar method
            is meant to provide quick access to these types of queries. WMI does
            not support a count type query, but for purposes of this case study,
            I have provided an implementation that invokes the ExecuteReader method above, and returns the
            first column of the first row. Admittedly, this is inefficient, but
            it illustrates the intent of the method.

public object
            ExecuteScalar()
{
if(this.Connection == null)
{
throw
            new Exception("Cannot execute reader. The WmiConnection "
            +
"object is not valid.");
}
if(this.Connection.State !=
            ConnectionState.Open)
{
throw new Exception("Cannot execute
            reader. The connection " +
"to the server is
            invalid.");
}
// execute the reader
IDataReader dr =
            this.ExecuteReader();
dr.Read();
// return the first column
            for the first
// row in the resultset.
return
            dr[0];
}

The following methods throws NotSupportExceptions because WMI doesn't
            support the particular functionality:

  • Cancel - You would provide an implementation for this method
              if you wanted to give a calling program the ability to cancel the
              execution of a command.

  • CreateParameter - If your data provider supports Data
              parameters, you would override this method to create a data
              provider specific instance of an IDataParameter object.

  • ExecuteNonQuery - You would provide an implementation of this
              method if your data provider supported adding, updating, and/or
              deleting of data from your data source. The SQL and OLE DB data
              providers, for instance, use this method to execute INSERT and
              UPDATE SQL statements.

  • Prepare - This method only applies when the CommandType property is set to StoredProcedure. If your data provider
              supports stored procedures, you would write the necessary code to
              compile the command for your data source.

WmiDataReader

The purpose of an ADO.NET DataReader is to provide efficient
            forward-only/read-only access to the results of a command. The DataReader is also responsible for mapping
            the data types of the source data to the Common Language Runtime
            (CLR) data types. The WMI provider's implementation of the
            DataReader is accomplished via the WmiDataReader class. The WmiDataReader class provides access to the
            result set of a WMI query command. Recall that the WmiCommand. ExecuteReader() method executes a WMI Query,
            creates a WmiDataReader, and sets the
            internal Data property of the WmiDataReader to the result set of the
            query. The WmiDataReader implements
            interface methods allowing a calling program to iterate through the
            WMI result set. Note that the WmiDataReader class implements the IDataReader interface. This interface in
            turn implements the IDataRecord
            interface. The IDataReader interface
            is shown below. It consists of a few properties and methods.

The IDataRecord interface (which
            the IDataReader interface implements)
            is comprised of a very large number of "Getter" functions, aimed at
            retrieving individual column values from the current record in the
            result set. A partial interface definition is shown below to give
            you an idea of what this interface looks like:

Here is a partial class definition for the WmiDataReader class, with descriptions for
            each member variable:

public class WmiDataReader:
            IDataReader
{
/// <summary>
/// The
            connection associated with the reader
///
            </summary>
private WmiConnection _connection;
///
            <summary>
/// Holds the number of rows retrieved from the
            last query.
/// </summary>
private int _numRows;
///
            <summary>
/// The resultset of the WMI query.
///
            </summary>
ISWbemObjectSet _data = null;
///
            <summary>
/// The enumerator of the object set.
///
            </summary>
IEnumerator _wmiEnumerator;
///
            <summary>
/// used to map the names of the columns to
            indexes. WMI supports
/// looking up by names, but we need an
            efficient way of looking
/// up column values by index. This
            array list holds this lookup.
/// </summary>
ArrayList
            _indexLookup = new ArrayList();
/// <summary>
///
            reference to the current record in the WMI resultset.
///
            </summary>
ISWbemObject _currentObject = null;
///
            <summary>
/// reference to the property set (i.e collection
            of columns)
/// </summary>
ISWbemPropertySet
            _currentPropertySet = null;
.
.
.
}

Properties

Let's first take a look at the properties of the WmiDataReader class. They are listed
            below.

Data - You got a sneak preview of this property when we explored
            the WmiCommand.ExecuteReader() method.
            This property is marked internal so that the WmiCommand object can set its value after
            executing the WMI query. Recall that the Data property is of the
            type ISWbemObjectSet. You can think of
            the ISWbemObjectSet interface as a
            result set. This object exposes an enumerator that WmiDataReader will utilize to move forward
            through the result set. The implementation of the Data property in the next code block sets
            the private _ data member variable to
            the value passed in. It then sets the _ wmiEnumerator member variable to the ISWbemObjectSet's enumerator followed by
            setting the _ numRows member variable
            to the number of rows in the result set. This private variable will
            be accessed via the RecordsAffected
            property of the DataReader.

internal ISWbemObjectSet
            Data
{
set
{
_data = value;
// we need to keep track
            of the enumerator
// for the object set.
this._wmiEnumerator =
            _data.GetEnumerator();
// set the number of rows to the
            count.
this._numRows = _data.Count;
// build the index lookup
            table.
this.BuildIndexLookup();
}
}

Accessing actual column values within ISWbemObjectSet is accomplished by
            specifying a column name. The IDataReader
            interface, however, must provide access to these values by
            name as well as and by an integer-based index. To support looking up
            values through an index, the WMI provider creates an ArrayList named _ indexLookup that contains the ordered names
            of the columns in the WMI result set. The BuildIndexLookup method that gets invoked
            below builds this list. Methods within the WmiDataReader class will leverage this ArrayList by passing in an index to get a
            column name, and using the column name to get at the data. Here is
            the code for the BuildIndexLookup()
            method:

private void
            BuildIndexLookup()
{
ISWbemObject WbemObj =
            null;
ISWbemPropertySet WbemPropSet = null;

WbemObj =
            this.GetFirstObjectInResultSet();
WbemPropSet =
            WbemObj.Properties_;
// grab the enumerator for the
            properties
IEnumerator e = WbemPropSet.GetEnumerator();
//
            Advance the enumerator's cursor until we reach
// the last
            item
while (e.MoveNext())
{
// grab the current
            property
ISWbemProperty ob = (ISWbemProperty)e.Current;
// add
            the property name to the
            index
_indexLookup.Add(ob.Name);
}

}

The code above takes the first result, grabs the property
            collection, and adds an entry to an array that simply stores the
            name of the property.

RecordsAffected (IDataReader) - Returns the number of rows
            affected by the Command. The number of
            rows may be the result of a SELECT, UPDATE, DELETE, or INSERT query.
            With regards to the WMI provider, this property will be the number
            of records in the result set as represented by the _numRows member variable.

public int
            RecordsAffected
{
get
{
return
            _numRows;
}
set
{
}
}

FieldCount (IDataReader) -
            Returns the number of fields that are in the result set. The WMI
            provider implementation of this property simply forwards the call to
            the WMI PropertySet object's count, as
            shown below:

public int
            FieldCount
{
get
{
return
            _currentPropertySet.Count;
}
}

Depth (IDataReader) - This
            is the nesting depth for the current row. This property applies to
            data providers where queries return multidimensional or hierarchical
            results. The WmiDataReader does not
            support this concept, so the implementation of the Depth property returns 0, and contains no
            logic in the "setter" function.

public int Depth
{
get
{
return
            0;
}
set
{
}
}

IsClosed (IDataReader) -
            Returns a Boolean specifying if the DataReader is closed. The WMI provider in
            this example does not support the concept of being closed so it
            always returns false.

public bool
            IsClosed
{
get
{
return
            false;
}
set
{
}
}

Methods

Constructors - The WmiDataReader contains two constructors.
            Both initialize the _ numRows member
            variable to 0. The overloaded constructor takes a WmiConnection as a parameter and sets the
            internal connection property. Note that both constructors are marked
            internal. Users must not be able to construct readers. Only the
            Command object must create and return
            a reader. Here are the constructor definitions:

internal WmiDataReader()
{
_numRows =
            0;
}

internal WmiDataReader(WmiConnection
            connection)
{
_connection = connection;
_numRows =
            0;
}

Read (IDataReader) - The
            Read method is responsible for moving
            the DataReader to the next record in
            the result set. It returns false once it reaches the end of the
            result set, and true otherwise. The implementation below advances
            the enumerator of the result set. If it's not at the end of the
            results, it sets the _ currentObject
            and _ currentPropertySet variables. If
            it has reached the end, it sets the objects to null.

Here is the implementation:

public bool Read()
{
// move to the
            next result
bool bRet = _wmiEnumerator.MoveNext();
if(bRet ==
            true)
{
// make sure the current object points
// to the
            current item on the enumerator
_currentObject = (ISWbemObject)
            _wmiEnumerator.Current;
// also set the propertyset member
            variable
_currentPropertySet = _currentObject.Properties_;
//
            return the result of the movenext
            operation
}
else
{
_currentObject =
            null;
_currentPropertySet = null;
}
return
            bRet;
}

GetValue (IDataReader) - The
            GetValue method is part of the IDataRecord interface definition. It takes
            an integer as a parameter, and returns the value of a column for the
            given record as a .NET object. The WMI provider's implementation
            verifies that the result set contains data, verifies that the index
            is valid, looks up the column name based on the index, and invokes a
            custom method called GetCurrentValueByName().

public Object GetValue(int
            i)
{
if(_numRows == 0)
{
throw new
            IndexOutOfRangeException("Must first call read");
}
if(i <
            0 || i > this._currentPropertySet.Count)
{
throw new
            IndexOutOfRangeException("No such column");
}
string
            strFieldName =
(string)_indexLookup[i];

return
            this.GetCurrentValueByName(strFieldName);
}

The GetCurrentValueByName method
            simply returns the column value given the name passed to it. Its
            definition is shown below:

private string
            GetCurrentValueByName(string strName)
{
return
            _currentPropertySet.Item(strName,
            0).get_Value().ToString();
}

GetXXX Functions (IDataRecord) -Users of a DataReader may also obtain strongly typed
            values from the result set by invoking one of the many GetXXX methods that make up the IDataRecord interface. Recall that the WmiDataReader class implements the IDataReader interface, which also implements
            the IDataRecord interface. Listing the
            implementations of all of the IDataRecord methods would be a bit mundane,
            but I have provided two implementations of the GetXXX methods below. In general the GetXXX methods invoke the GetValue method we implemented earlier and
            cast the return value to the appropriate type.

public string GetString(int
            i)
{
return this.GetValue(i).ToString();
}

public
            bool GetBoolean(int i)
{
return
            (bool)this.GetValue(i);
}

this[parameter] (IDataReader) -The IDataReader interface also defines two
            separate indexers which allows the implementing class to be treated
            as if it were a collection. There are two implementations of the
            indexer. The first provides access to column values by name; the
            second by an integer index. Implementations for the WMI provider of
            these two indexers follow:

public object this[string
            name]
{
get
{
return
            this.GetCurrentValueByName(name);
}
}

Here is the integer based implementation of the indexer:

public object this[int
            i]
{
get
{
return
this.GetValue(i);
}
}

GetFieldType (IDataReader) -
            The GetFieldType method returns the
            CLR type of the data item at the specified index within the current
            record. WMI uses Common Information Model (CIM) data types to
            represent its data. For more information on CIM, navigate to the CIM
            link found at the end of this case study. It is up to the WmiDataReader to translate CIM to CLR data
            types. After grabbing the property at the specific index, the method
            invokes the ConvertCIMToCLR method
            whose implementation follows:

public Type GetFieldType(int
            i)
{
ISWbemProperty p =
            _currentPropertySet.Item(_indexLookup[i].ToString(),0);

return
            ConvertCIMToCLR(p.CIMType);
}

To simplify converting types, the WmiDataReader simply converts all data types
            to the CLR string data type as shown below. If we wanted to
            elaborate on this further, we could create a switch statement that
            provided a mapping for each CIM type.

private Type ConvertCIMToCLR(WbemCimtypeEnum
            en)
{
return typeof(string);
}

GetName (IDataReader) - The
            GetName method returns the name of the
            field within the result set given the index passed in. Its
            implementation is shown below. Note that it must use the _ indexLookup ArrayList because the WMI PropertySet object looks up column names by
            name. Remember that all the _ indexLookup array does is allow for
            efficient lookup of column names by index.

public string GetName(int i)
{
// grab
            the object given the index passed in
ISWbemProperty ob =

_currentPropertySet.Item(_indexLookup[i].ToString(),0);
//
            return the property name
return ob.Name;
}

GetValues (IDataReader) -
            The GetValues method populates an
            array of objects passed into the method with the data from the
            current record, where each column value represents an object. Note
            that the caller of this method must allocate enough space in the
            object array to satisfy the number of columns. The WMI provider's
            implementation for this method is shown below:

public int GetValues(object[]
            values)
{
// make sure we have a valid
// number of
            properties
if(_currentPropertySet.Count >
            0)
{
if(values.Length < this.FieldCount)
{
throw new
            Exception("You must allocate the " +
"object array with enough
            room to " +
"hold the number of fields in the " +
"row
            set.");
}

int iIndex = 0;
// iterate through the list
            of properties
foreach(ISWbemProperty p in
            _currentPropertySet)
{
// set the value objects to
// the
            value
values[iIndex] = p.get_Value();
}
// return the
            number of values
return values.Length;
}
else
{
// if
            we go here, there weren't any
// properties
values =
            null;
return 0;
}
}

GetOrdinal (IDataReader) -
            The GetOrdinal method takes a name of
            a column and returns the index at which the column name is located.
            The WmiDataReader implementation uses
            the IndexOf method of the string class to provide this functionality
            as shown below:

public int GetOrdinal(string
            name)
{
return _indexLookup.IndexOf(name);
}

Close (IDataReader) - The
            WMI implementation does not have a concept of closing a DataReader. The SQL Server and OLE DB
            providers within .NET only allow one DataReader to be open for each connection,
            and the DataReader must be explicitly
            closed after the programmer is done.

GetSchemaTable (IDataReader)
            - GetSchemaTable returns a DataTable populated with metadata about the
            columns of the reader. This method is useful in that users may
            generically execute and display the results of a query by populating
            a grid full of columns not known at design time. We will create a
            Windows application that utilizes this method to populate the
            headings of a ListView control with column names. The GetSchemaTable implementation below creates
            a table called WMIMetaData, defines a
            ColumnName and DataType column within the table, and
            creates a row for each column in the result set.

public DataTable GetSchemaTable()
{
//
            create a datatable
DataTable dt = new
            DataTable("WMIMetaData");
// add the schema
            columns
dt.Columns.Add("ColumnName",
            typeof(string));
dt.Columns.Add("DataType", typeof(Type));
//
            iterate through the list of properties
foreach(ISWbemProperty p
            in _currentPropertySet)
{
// add a new row
DataRow dr =
            dt.NewRow();
// set the name of the colum and
// the data type
            properties
dr["ColumnName"] = p.Name;
dr["DataType"] =
            this.ConvertCIMToCLR(p.CIMType);
// add the row to the
            table
dt.Rows.Add(dr);
}
return dt;
}

GetDataTypeName (IDataReader) - This method returns the name
            of the data type for the specified field index. The implementation
            below looks up the property object based upon the index, converts
            the data type to a CLR type, and returns the string representation
            of the type.

GetDataTypeName(int i)
{
// grab the
            object given the index passed in
ISWbemProperty ob =
_currentPropertySet.Item(_indexLookup[i].ToString(),0);
//
            determine the CLR type.
Type t =
            ConvertCIMToCLR(ob.CIMType);
// return the name of the
            type
return t.ToString();
}

NextResult (IDataReader) -
            NextResult is applicable to those data
            providers that support multiple result sets. The SQL Client data
            provider supports multiple result sets when SQL statements are
            separated by a semi-colon. Invoking this method tells the DataReader to point to the next result set.
            The WMI provider will not support this concept, so this method
            throws a NotSupportedException.

GetData (IDataReader) - The
            GetData method is used to expose
            nested tables and other hierarchical data. Both data providers
            within .NET do not support this method. We are not implementing this
            method either.

WmiDataAdapter

A .NET DataAdapter provides the
            link between a data source and a .NET Dataset. I assume you have a
            basic understanding of what a .NET Dataset is. If you need further
            details, please see the following article on CSharpToday: http://www.csharptoday.com/content.asp?id=1568&WROXEMPTOKEN=618658Zezi9SpDFSZgQps5Xv6u.
            The SQL Server and OLE DB implementations of the DataAdapter within the .NET framework
            support populating a Dataset with data
            from a data source in addition to updating a data source given
            changes to the Dataset. Updating of
            the data source is accomplished via the DataAdapter's Update method. This is a relatively easy
            concept to grasp in regards to a database. A typical sequence of
            events might be:

  • Populate a dataset using an adapter

  • Make modifications to a data set by modifying, adding, or
              deleting rows, and

  • invoke the DataAdaper's Update
              method to synchronize the changes back to the data source.

However, WMI's WQL only supports read-only access to the WMI
            data, so we are a limited in what we can do in this regard. However,
            the WMI provider will support the concept of Updating WMI data by
            adding logic to the Adapter's update
            method that inspects the State of a service (e.g., Stopped or
            Started) and stops or starts the service.

The WmiDataAdapter class implements
            the IDbDataAdapter interface. This
            interface derives from the IDataAdapter interface to create the
            following interface definition:

Properties

SelectCommand (IDataAdapter)
            - This is the command that the DataAdapter uses to populate the Dataset. The WMI provider's implementation
            represents this property as an instance of a WmiCommand object:

public WmiCommand
            Command
{
get
{
return
            _command;
}
set
{
_command =
value;
}
}

UpdateCommand, DeleteCommand and InsertCommand (IDataAdapter) - The WMI provider only
            supports the concept of the SelectCommand so we have empty
            implementations of these properties to satisfy the interface. This
            interface is typically geared toward a RDBMS type data provider. I
            could have chosen to just implement the IDataAdapter interface which would remove
            these four commands, but I wanted to illustrate the various commands
            associated with an adapter.

TableMappings (IDataAdapter)
            - This property specifies how a table in the data source maps to a
            DataTable within a DataSet. This allows a program to
            essentially hide the underlying physical table names and exposes
            names within a DataSet that are more "user friendly" to a user or
            calling program. It allows for a physical table named COMP_SALES to
            be represented in a DataSet as Company Sales. Additionally, each
            TableMapping instance contains a
            reference to a collection of DataColumnMapping objects that offer a
            similar service. With a DataColumnMapping, a physical column named
            COMPANY_ID could be represented in a DataSet as Company ID. If you
            need to support TableMappings in a
            provider, you must examine them when you populate a DataSet and when
            callers issue the Update method. We
            will cover these methods later.

MissingMappingAction (IDataAdapter) - This property is an
            enumeration that defines the action taken when a specified table or
            column mapping (e.g. DataTableMapping,
            DataColumnMapping) is missing from the source table or source
            column. Recall our discussion in the previous section and assume
            that a calling program attempts to populate a DataSet. In setting up
            the DataAdapter, the calling program has specified mappings one of
            which maps the COMP_SALES physical table to the Company Sales table
            in a DataSet. If the COMP_SALES table doesn't exist in the database,
            the data provider needs to know what action to take. The MissingMappingAction property specifies this
            action through the MissingMappingAction enumerator which
            provides for the following actions:

  • Error - Throwing an exception

  • Ignore - Ignoring the source tables and columns that failed to
              map. This results in some source data not making it in the Dataset

  • Passthrough - Creating the DataTable and DataColumn that failed to map.

Since the WMI provider does not support mappings, the MissingMappingAction always returns Passthrough (which creates the tables and
            columns in the dataset). The set operation throws an exception for
            any other type but Passthrough.

public MissingMappingAction
            MissingMappingAction
{
get
{
return
            MissingMappingAction.Passthrough;
}
set
{
}
}

MissingSchemaAction (IDataAdapter)
            - While the MissingMappingAction
            specifies the action to be taken when an expected table or column is
            missing from a physical data source, the MissingSchemaAction property specifies the
            action to be taken when a DataTable or
            DataColumn is missing from the schema
            within a Dataset. Possible actions
            include:

  • Add - Adding the necessary columns to the schema

  • AddWithKey - Adding the columns with primary key information

  • Error - Throwing an exception

  • Ignore Ignoring the extra columns

Our implementation only supports the Add member of the enumeration.

public MissingSchemaAction
            MissingSchemaAction
{
get
{
return
            MissingSchemaAction.Add;
}
set
{
}
}

Methods

Constructor - Aside from the default constructor, there
            are three additional constructors that make up the adapter. Their
            signatures and descriptions are below:

// sets the Command property of the
            adapter
public WmiDataAdapter(WmiCommand comm)
// sets the
            Command property of the adapter and also sets the WmiCommand's //
            Connection property.
public WmiDataAdapter(WmiCommand comm,
            WmiConnection conn)
// creates a new WmiCommand and WmiConnection
            objects.
public WmiDataAdapter(string strCommand, string
            strConnection)

Fill (IDbDataAdapter) - This
            method Inserts or Updates rows in the Dataset to match those within the data
            source. The WmiDataAdapter.Fill method
            is shown in the next code block. Its first job is to make sure the
            command object is valid. Note that this method also keeps track of
            the ConnectionState. If the connection
            is closed, it must open the connection and set a flag (bMustCloseConnection) so that the connection
            will be left in its initial state once the Fill operation is complete. It then proceeds
            to fill the schema in the Dataset and
            grabs a reference to the table based upon the table name. It then
            creates a WmiDataReader and iterates
            through it adding a new row to the table for each row in the reader.
            Note the call to AcceptChanges at the
            end of the Fill method. The call to
            AcceptChanges is vital in order to
            keep the state of each row within the DataSet in a consistent state.
            Internally, the DataSet keeps state information for columns within
            each row. It does this through the RowState property associated with each Row.
            The RowState property returns the
            DataRowState which specifies if a row has been Added, Deleted, Detached, Modified, or
            Unchanged. As the fill method adds records to the DataSet,
            each row's RowState is set to the
            Added state. The AcceptChanges method highlighted in the code
            below accepts these inserted rows and changes all of the row's
            RowState to Unchanged. Why is this important? If we removed
            this line and a program was to make modifications to rows inside the
            DataSet, the state will remain marked as Added. Now assume a program
            wants to invoke the Adapter's Update
            method to update the data source with their modifications. The Update method would query the DataSet to
            find out the state of its Data and erroneously determine that a
            bunch of rows have been inserted (because the RowState is set to Added). If the WMI provider supported adding
            rows in this scenario, it would end up duplicating all of the rows
            that the Fill method added. The AcceptChanges puts all rows in a "pristine"
            state by setting the RowState to Unchanged. The importance of this method
            will become clearer when we cover the Update method. Here is the source code to
            the Fill method:

public int Fill(DataSet ds)
{
// make
            sure the command is good
if(_command == null ||
            _command.CommandText.Trim() == "")
{
throw new
            Exception("Cannot fill without receive command");
}
// if the
            connection is closed, we must open it.
// We also need to make
            sure we close it before
// leaving this method.
bool
            bMustCloseConnection = false;
if(_command.Connection.State !=
            ConnectionState.Open)
{
_command.Connection.Open();
bMustCloseConnection
            = true;
}
// fill the schema for this
            query
this.FillSchema(ds, SchemaType.Mapped);
// create the
            table
DataTable dt = ds.Tables[this.CommandTableName];
//
            create a data reader
WmiDataReader dr =
(WmiDataReader)_command.ExecuteReader();

// iterate
            through the reader
while(dr.Read())
{
// create a new
            row
DataRow newRow = dt.NewRow();
// set all of the
            fields
for(int i = 0; i < dr.FieldCount;
            i++)
{
newRow[i] = dr[i];
}
// add the row to the
            table
dt.Rows.Add(newRow);
}
// accept all of the
            changes
ds.AcceptChanges();
// if we had to open up the
            connection to
// perform this operation then we must
// close
            it.
if(bMustCloseConnection)
{
_command.Connection.Close();
}
//
            return the number of rows affected
return
            dr.RecordsAffected;
}

FillSchema (IDbDataAdapter)
            -This method is responsible for creating a table in the DataSet and configuring the schema to match
            the source table based on the SchemaType passed in. The SchemaType may specify Mapped or Source. Mapped directs the adapter to use
            table mappings when creating the schema. WmiDataAdapter doesn't inspect the SchemaType, and implements the SchemaType.Source method that uses the
            source data to specifying the schema. The result is that the table
            and columns names inside the Dataset
            will map directly to the table and column names in the data source.

In the implementation below, the FillSchema method creates a table that maps
            to the source table (removing it if it already exists). It then
            iterates through the set of properties adding a column for each
            column in the result set.

public DataTable[] FillSchema(DataSet ds,
            SchemaType schemaType)
{
// need the connection for this
            adapter
WmiConnection conn =
            (WmiConnection)_command.Connection;
// we need to object
            set
ISWbemObjectSet ObjSet = null;
ISWbemObject WbemObj =
            null;
ISWbemPropertySet WbemPropSet = null;
// unfortunately
            we need to execute the query
// to get the metadata information
            necessary
// to fillin the schema
ObjSet = conn
            .WmiServices.ExecQuery(_command.CommandText,
"WQL", 0x10,
            null);
// create 1 table (for this version we will
// only
            support single select statements
DataTable []dt = new
            DataTable[1];
// new and create the table with the table
            name
string strTableName = this.CommandTableName;
// if the
            dataset contains a definition for this
// table name, remove
            it.
if(ds.Tables.Contains(strTableName))
{
ds.Tables.Remove(strTableName);
}

dt[0]
            = new DataTable(strTableName);
// grab the enumerator for the
            object set
IEnumerator obEnum = ObjSet.GetEnumerator();
//
            grab the current object
obEnum.MoveNext();
WbemObj =
            (ISWbemObject)obEnum.Current;
// grab the
            properties
WbemPropSet = WbemObj.Properties_;
// grab the
            property enumerator
IEnumerator ePropSet =
            WbemPropSet.GetEnumerator();
// Advance the enumerator's cursor
            until we reach
// the last item
while (ePropSet.MoveNext())
{
DataColumn dc = new DataColumn();
// Cast the Current
            item to the proper type
ISWbemProperty ob =
            (ISWbemProperty)ePropSet.Current;
// set the column
            name
dc.ColumnName = ob.Name;
dc.DataType =
            ConvertCIMToCLR(ob.CIMType);
// add this column to the
            table
dt[0].Columns.Add(dc);
}

// add this table to the
            dataset
ds.Tables.Add(dt[0]);
// return the table
return
            dt;

}

Update (IDbDataAdapter) -The
            Update method is responsible for
            updating a physical data source with changes made to the DataSet
            that was populated by the adapter. The WMI implementation of this
            method will meet the final requirement of the data provider by
            supporting stopping and starting of Window's services. The sequence
            of events might unfold as follows:

  1. A calling program uses the WMI adapter to fill the Dataset
              with the following WMI query: SELECT * FROM WIN32_SERVICE

  2. The adapter obliges this request and creates a DataTable
              called WIN32_SERVICE within the DataSet and populates it with a
              list of Windows services

  3. The calling program uses the DataSet select command and
              obtains the row containing the World Wide Web Publishing Service
              (IIS) and sets the State field from "Running" to "Stopped."

  4. The calling program invokes the Adapter's Update method.

  5. Finally the Adapter examines the changed State column for the
              World Wide Web Publishing service and uses the .NET
              ServiceController class to stop the service.

Now that you have an understanding of what the WMI provider's
            update method will do, let's look at its implementation:

public int Update(DataSet
            dataSet)
{
// as a special case to demonstrate
// updating
            functionality, we
// are going to demonstrate what
//
            happens when someone
// updates the state of a
            service
if(this.CommandTableName.ToUpper()
==
            "WIN32_SERVICE")
{
// this call gets a list of rows that
// were modified. These rows
// contain current values
            (after changes)
DataRow[] modifiedRows =
dataSet.Tables[CommandTableName].
Select("","",DataViewRowState.ModifiedCurrent);

//
            iterate through each modified row
foreach(DataRow dr in
            modifiedRows)
{
string previousServiceState =
dr["State",
            DataRowVersion.Original].ToString();

// make sure there was a
            change in the states. Someone
// could have made modifications to
            Data column values
// besides the State column. This line makes
            sure
// that the State column for the service row
// has
            indeed changed.
if(dr["State"].ToString() !=
            previousServiceState)
{
// create the Service controller
// associated with the
// service and system
            name
ServiceController sc =
new
            ServiceController(dr["Caption"].ToString(),
dr["SystemName"].ToString());

// this logic checks the
            modified rows
// state and either stops
// or starts the
            service.
if(dr["State"].ToString() ==
            "Stopped")
{
sc.Stop();
}
else if(dr["State"].ToString()
            == "Running")
{
sc.Start();
}
}
}
// return the
            number of modified rows
return
            modifiedRows.Length;
}
else
{
return
            0;
}
}

Since the provider only keys in on updates for WIN32_SERVICE
            objects, the method first verifies that the WMI Table name is
            WIN32_SERVICE . It then proceeds to grab all of the modified rows
            into a DataRow collection called modifiedRows by utilizing the
            DataTable's Select method and
            specifying the ModifiedCurrent DataViewRowState. The DataViewRowState ModifiedCurrent value informs the Select method to only return a list of
            current records that have been modified. Once the method has a list
            of rows that were modified, it iterates through each row, grabbing
            the previous value of the State column (remember this column
            indicates whether the service is running or not). If the calling
            program changed the value, the Update
            method then sets up the ServiceController object the correct
            information and starts or stops the service.

Using The Custom WMI Provider

Now that we have covered all of the properties and methods for
            the four interfaces, let's examine a quick application that tests it
            out. Included with the source code of the data provider is a Windows
            based application that allows you to type in a WMI query and
            populate a ListView control with the results. The screenshot of the
            application below shows the results from a "SELECT * FROM
            WIN32_PROCESS" query on my local machine.

The Server TextBox in the screen
            shot above allows a user to specify a server to execute queries
            against. The WmiQuery TextBox takes
            the specific WMI SQL query. Once the user has completed these two
            TextBoxes they may click the Execute
            button to execute the WMI query. The Stop/Start a Service button pertains to
            WIN32_SERVICE WMI queries. Once a WIN32_SERVICE query has been
            executed (i.e. SELECT * FROM WIN32_SERVICE), the user may select a
            service from the results grid and change the service's status from
            Running to Stopped and vice versa. The Clear button clears the Results
ListView.

There are quite a few WMI queries to choose from. Here are a few
            more for you to play around with:

QUERY | Description
SELECT * FROM Win32_service | Obtains a list of services running on the target machine.
SELECT * FROM Win32_LogicalMemoryConfiguration | Obtains one record listing details of the logical memory configuration from the target machine.
SELECT * from win32_LogicalProgramGroup | Retrieves a listing of the logical program groups on the target machine.
SELECT * from win32_operatingsystem | Obtains information about the operating system of the target machine.

You may also replace the * from the SELECT queries above with
            specific column names if you wish.

When you click on the Execute
            button, the button_click event is
            triggered, as shown in the next code block:

private void btnExecutDS_Click(object
            sender, System.EventArgs e)
{
// make sure the UI is completed
            correctly
if(tbWmiQuery.Text.Trim() ==
            "")
{
MessageBox.Show("Query is
            invalid.");
return;
}
if(tbServer.Text.Trim() ==
            "")
{
MessageBox.Show("Server must be
            specified.");
return;
}
try
{
this.ClearListView();
//
            create a WMI Connection with the
// connection
            string
WmiConnection conn = new
            WmiConnection();
conn.ConnectionString =
            tbServer.Text;
conn.Open();
// make sure the connection is
            open
if(conn.State !=
            ConnectionState.Open)
{
MessageBox.Show("Could not connect to
            {0}",
tbServer.Text);
return;
}
// create a new command
            with the WmiQuery
WmiCommand comm = new
            WmiCommand();
comm.Connection = conn;
comm.CommandText =
            tbWmiQuery.Text;
// Execute the command. We will be working
            with
// a reader
WmiDataAdapter wda = new
            WmiDataAdapter();
wda.Command = comm;
DataSet ds = new
            DataSet();

wda.Fill(ds);

this.CreateResultColumns(ds.Tables[0]);
this.PopulateListView(ds.Tables[0]);

}
catch(Exception
            ex)
{
MessageBox.Show(ex.Message);
}

}

After verifying the UI controls the code, it clears the ListView
            control, creates a connection to the server, and creates a WmiCommand and WmiDataAdapter object. It then populates a
            Dataset using the Adapter. Once the
            Dataset is populated, the method uses
            the DataSet to populate the ListView
            with columns and rows.

The CreateResultColumns method
            shown below creates a ListView column for each DataColumn in the DataTable:

private void CreateResultColumns(DataTable
            dt)
{
// the schema returned us the name and
// type of
            each of the columns.
// this routine creates ListView
            Columns
// for each column returned
foreach(DataColumn dc in
            dt.Columns)
{
lvResults.Columns.Add(dc.ColumnName,
100,
HorizontalAlignment.Left);
}
}

The PopulateListView method
            iterates through each row in the table and adds each column within
            the row to the result set.

private void PopulateListView(DataTable
            dt)
{
foreach(DataRow dr in dt.Rows)
{
ListViewItem lvi
            = new ListViewItem();
for(int i = 0; i < dt.Columns.Count;
            i++)
{
// itearate through each of the fields
// in the
            result set to continue
// updating the ListViewItem
if(i ==
            0)
{
lvi.Text =
            dr[i].ToString();
}
else
{
lvi.SubItems.Add(
            dr[i].ToString());
}
// add the results to the
//
            ListView
}
lvResults.Items.Add(lvi);
}

}

Additionally, the program shown in the screen shot above
            exercises the DataAdapter's update which allows you to start or stop
            a Windows service. The Windows application code that exercises this
            functionality is shown below and assumes the DataSet contains the
            results of a WIN32_SERVICE result set. After issuing a WIN32_SERVICE
            query, the user may select a service and click the Start/Stop a Service button. The following
            code gets executed when the user clicks the Stop/Start a Service button:

string strServiceName =
            this.GetCaptionItemText();

if(strServiceName ==
            "")
{
MessageBox.Show("Operation failed. Was your last "
            +
"query against the WIN32_SERVICE object? " +
"You must
            execute this query and then " +
"click the Start/Stop Service
            button.");
return;

}

if(_wmiDataSet != null
            && _wmiDataSet.Tables.Count > 0)
{
// look for the
            service with the correct caption

DataRow []dr =
            _wmiDataSet.Tables[0].
Select("Caption='" + strServiceName +
            "'",
"", DataViewRowState.CurrentRows);

// make sure we
            found a row
if(dr != null && dr.Length == 1)
{
//
            editing of the row begins
dr[0].BeginEdit();
// set the state

if(dr[0]["State"].ToString() ==
            "Running")
{
dr[0]["State"] =
            "Stopped";
}
else
{
dr[0]["State"] =
            "Running";
}
// editing of the row
            ends
dr[0].EndEdit();
// let the adapter handle the stopping
            or starting
// of the
            service
_wmiDataAdapter.Update(_wmiDataSet);

}
else
{
MessageBox.Show("Operation
            failed. Was your last " +
"query against the WIN32_SERVICE
            object? " +
"You must execute this query and then " +
"click
            the Start/Stop Service
            button.");

}
}
else
{
MessageBox.Show("Operation
            failed. Was your last " +
"query against the WIN32_SERVICE
            object? " +
"You must execute this query and then " +
"click
            the Start/Stop Service button.");
}

The code above first retrieves the value of the caption column
            within the ListView by invoking the GetCaptionItemText method. We will examine
            the code for this method next. The value of the caption column
            represents the name of the Service that the user selected. After
            retrieving the Service name, the code then uses the DataTable.Select method to locate the
            correct row in the DataSet. Once it has a valid row, the code then
            edits the row, toggles the State of the service and invokes the
            adapter's Update method. The Adapter
            takes over and executes our Update
            code which in turn starts or stops the service.

Here is the code for the GetCaptionItemText method:

private string
            GetCaptionItemText()
{
if(lvResults.SelectedItems.Count >
            0)
{
foreach(ColumnHeader ch in
            lvResults.Columns)
{
if(ch.Text == "Caption")
{
return
            lvResults.SelectedItems[0].
SubItems[ch.Index].Text;
}
}
}
return "";
}

Case Study Review

This case study covered quite a bit of material. We examined what
            a .NET data provider is, and explored the major reasons you might
            create one. We also touched on some of the guidelines to keep in
            mind when writing a provider. After a quick introduction to WMI, we
            proceeded to cover the implementation of the four major interfaces
            for ADO.NET providers. We completed the case study by walking
            through a simple Windows application that used the custom WMI data
            provider to retrieve and Update WMI data.

Case Study Information
Author | Todd Clemeston
Technical Editor | Adam Ryland
Project Manager | Helen Cuthill
Reviewers | Andrew Krowczyk, Saurabh Nandu, Slavomir Furman, Adam Ryland

If you have any questions or comments about this case study,
            or suggestions for further pieces, please contact the technical
            editor. If you wish to talk about the case study with other
            subscribers, please visit our forum here.

     Brought to you by ownSky!
 C#Today - © 2002 Wrox Press
Limited


WMI for .NET

WMI for .NET
http://www.csharptoday.com/content.asp?id=1757
Article Rating: not yet rated
Published on March 18, 2002
Abstract
The .NET initiative has sparked a new era in the adoption of Windows technologies. Windows developers have got used to an environment of comfort thanks to Visual Studio and the COM programming model. WMI is one of these technologies, one that requires some work to understand, and some thinking as to how it can improve an application. This case study by Richard Conway looks at WMI. During this piece, he explains the classes we can use to manage the Win32 environment, and examines WMI object instances and events which will be used in the context of managing a web application. This will show the degree of real-time information feed we can have using WMI. Finally we will use an example windows application to manage another application with a WMI implementation.
Case Study

FEEDBACK: We would love to hear what you think of this case study, and what topics you would like to see covered in future. Please take a few moments to mail the Editor to give us your opinions.

WMI (Windows Management Instrumentation) is one of the most fantastic innovations in the convergence of standards. It represents Microsoft's implementation of the WBEM (Web Based Enterprise Management) which is the result of work undertaken by Cisco, Microsoft, et al. It was later taken over by the DMTF (Distributed Management Task Force - a standards body) and the result was a common framework (The Common Information Model), which WMI complies with. The Common Information Model is based on object oriented principles, and it is effectively a repository of abstract classes representing devices, networks, etc. These abstract classes can be implemented through WMI to provide a framework for management of applications (and core Windows systems). The model follows a similar nature to that of standard OO principles, each base described above has specific classes which derive from it and may be manufacturer specific: these classes will have one or more representative instances. As we'll see, the extensibility of this system ensures a great deal of hardware, software, and Operating System control as well as the ability to extend the model to include support for anything from applications to newer system devices.

The case study will be broken into three parts:

  • The first will explain the classes we can use to manage the Win32 environment.
  • The second part will examine WMI object instances and events which will be used in the context of managing a web application. This will show the degree of real-time information feed we can have using WMI.
  • The final part will use an example windows application to manage another application with a WMI implementation. I've chosen SQL server as it would be the most coherent real-world example of this facet of WMI, and will indicate to the reader the purpose of application management.

The .NET initiative has sparked a new era in the adoption of Windows technologies. Windows developers have got used to an environment of comfort (and Intellisense!) thanks to Visual Studio and the COM programming model. We would always shy away from these technologies which demanded time, effort, and research and usually involved understanding complex APIs (there is always something that would do the job produced by a third party and designed with dual COM interfaces). WMI is one of these technologies that requires some work in understanding and some thinking as to how it can improve an application. The convergence of all Windows technologies in the .NET Framework has meant that developers no longer have to install supplementary SDKs and refer to distinct sets of documents since (as in the case of cryptography support), these once separate APIs have all been integrated into the .NET Framework making there adoption much easier. In this way, WMI has been exposed to a far wider audience, one that will immediately see the benefits of implementing many of its features.

The integration of WMI into the .NET Framework SDK has meant that developers will no longer have to rely on the VBScript/ASP model, WSH/COM and a series of discrete WMI locators and management objects to implement WMI. The new model through .NET has extended the former model to allow the use of WMI events and Instance classes (which will be explained and demonstrated later). This has made it very easy to include extra support into an application to allow monitoring of all application instance details.

Contents

Section
What has WMI brought to .NET?
WMI explained
Basic WMI In .NET
Application development with WMI
WMI Application Control
Other Providers, other uses
Conclusion

Using the downloaded code

After unzipping and installing the code download as described in the readme.txt document, ensure that the following are installed:

  • IIS 5.0 or above
  • Visual Studio .NET v1
  • SQL Server 2000
  • WMI Provider for SQL Server (instructions for installation shown later in this case study)

What has WMI brought to .NET?

Of the numerous uses of WMI within the .NET Framework, the following examples show the power and control with which our system can be managed:

Schemas

  • What parts of this application are exposed to management?
  • What can I find out about the Network Adapter?

As mentioned earlier, a fantastic innovation which we will see later is the idea that we can build WMI support into an application. We can also probe the schema information of a third party application (which has exposed management interfaces in the ways described below) to find out what is configurable and readable about an application instance.

It may also be necessary to find out basic information on a piece of hardware such as the network adapter. Similarly, it's possible to probe the schema to find out what management data has been exposed.

Events

  • Notify me when the service I have written is stopped

Through the use of standard .NET language features (such as delegates) we can subscribe to events so that our program can receive an asynchronous response. This response will indicate that a certain threshold has been reached, be it a certain percentage use of the CPU or as pointed out above whether a service (e.g. w3svc) has been stopped. A monitoring application will thus be able to handle this and respond.

Queries

  • Which services are running? Which are stopped?

Queries allow criteria to be specified in an SQL like syntax (WQL - WMI Query Language). This will enable us to set filters on the types of objects we wish to return (from our query), therefore we can target a particular instance of a class by a name or an id or, as in the above example, we can check the state of a class instance and return a collection corresponding to that state (e.g. all running services).

WMI explained

WMI Providers and the CIM

In the introduction I mentioned the CIM (Common Information Model) which was ratified by the DMTF. The CIM has two components in the WMI framework: the CIMOM (CIM Object Manager) and the CIM Object Repository.

It is the CIMOM that all WMI Providers communicate with via COM or DCOM (on remote machines). WMI Providers are nothing more than COM objects which interface with core objects, be they system objects such as Win32 or Directory Services, or with application objects such as SQL Server. The CIMOM is a broker between a management application and a WMI Provider, it controls everything that is placed in the CIM Repository.

The names for various components of the WMI infrastructure have now been superceded. The CIMOM is identified by the broad term of Management Services and the CIM more generally by the Management Schema. The CIM Repository is now simply called the Catalog.

Diagram showing data flow from a Management Application

There are many providers which cover different aspects of the Windows environment. These are some of the key ones:

  • Win32 provider
  • Perfmon provider
  • Event log provider
  • SNMP provider

The most easily exploitable, and the one that will be the most useful, is Win32 Provider. The Win32 provider exposes some 600+ classes to WMI which allow management and querying of many windows services and subsystems. Data can be obtained on anything from the graphics card to the disk drive. Within the software layer we can also control and obtain details on each running process.

The Perfmon provider will allow us to access data directly from the performance monitor. This is available, as is the Win32 provider, as part of core WMI. The other providers mentioned are installed as part of the WMI SDK. The WMI SDK can be downloaded from the MSDN site (http://msdn.microsoft.com/downloads/default.asp?url=/downloads/sample.asp?url=/MSDN-FILES/027/001/552/msdncompositedoc.xml&WROXEMPTOKEN=618658Zezi9SpDFSZgQps5Xv6u). It won't be covered here, but to be able to access certain services through WMI it is invaluable. It contains providers for such things as ODBC and Active Directory. This signifies that rather than communicating with the Active Directory directly through ADSI, we can alternatively use WMI to broker calls to the Active Directory WMI exposed management interfaces. The SDK provides a series of tools which can be used to analyse WMI object instances and event subscriptions by inspecting the CIM repository (two are referred to below). These are listed below for thoroughness, but this case study won't focus on the use of these tools for implementing WMI solutions.

  • WMI Object Browser
  • WMI CIM Studio

These tools are HTML pages with embedded ActiveX controls allowing object instances, class definitions, and events to be seen in the browser window. The Object Browser allows class instances within the CIM repository to be accessed. It displays properties and methods of each instance and abstract class, and allows values to be changed (for a class instance). This is a very useful tool to get to know many of the WMI providers and classes (and to begin working with WMI namespaces - described below). The CIM studio allows class instances to be deleted, created, and modified. This can be a very useful tool for testing WMI events (described in the first two sample applications). Instances can be modified in any way, and any application subscribing to any one of the three events described above will immediately be notified. The CIM Studio also allows very granular management of WMI classes and objects, giving descriptions of relationships through visual relationships shown in diagrams.

Coffee Break

(1) What controls everything that is placed in the CIM Repository?
(2) What does the Perfmon allow us to do?

WMI Consumers

The descriptive term for something that consumes the services of a WMI provider is a WMI consumer. The parallel here for developers is the use of OLEDB providers and consumers. Providers provide access and standard interfaces which consumers can consume to access data from a database. Remove the words "data" and "database" from the above sentence and replace with the terms "management" and "WMI" and the relationship between the two becomes obvious!

Basic WMI In .NET

History of Implementation

To illustrate the ease of WMI implementation in .NET and many of the concepts I've described above (and have yet to describe), I've provided a sample application which uses some of the classes exposed by the core Win32 provider to allow the controlling of processes and services.

It used to be necessary to use one of two approaches in scripting WSH or ASP (or within a desktop or distributed application) to create a handle to an instance of a WMI managed object. Either:

Set locator = CreateObject("WbemScripting.SWbemLocator")

Or

Set handle = GetObject("winmgmts://localhost")

The first approach uses a COM object to locate the correct WMI class, and the second uses the winmgmts moniker string to connect to the WMI moniker. As the class instance exists, GetObject is used to obtain a handle to it. This has been made infinitely easier with .NET. Follow the steps below to enable WMI in a VS.NET Project:

  • The initial step is to set a reference to the WMI assembly.
  • By default this is not referenced in the default project so it has to be done manually. From the Visual Studio menu select Project, then Add Reference.
  • On the first exposed tab (.NET Components) select the System.Management assembly under the Component Name section.
  • Now this has been done we can reference a fully qualified System.Management.xxx to create different WMI management or locator objects, or we can use the using directive as below.

using System.Management;
using System.Management.Instrumentation;

The first namespace is relevant to this example and provides the core access for virtually everything we want to accomplish with respect to managing and locating existing class instances, subscribing to events, and querying. The Instrumentation namespace we'll cover in a later section. This will provide all of the features we need to use WMI in our own applications.

The project WMIFeedback is a C# Windows form project. The project will manage and monitor Windows Services and Processes through WMI. It is comprised of a Windows Form where the main body of code is, and two classes, Win32Process and Win32Service. Each of these corresponds to the collection of windows services or processes, and exposes functions of services and processes collectively. When the application is started we'll the see the following:

Into the Application

The application enters the entry point called after all the UI components are initialised within the forms constructor. The method InitWMI() is used to create the initial state of the form, listing all the services and binding WMI events to particular event handlers within the form. This method is written to initialise everything on the form, it is called from the forms constructor method (in file Form1.cs). We'll look at the code in InitWMI() sequentially to understand what it is doing:

A basic UML Diagram showing the structure of the classes and their relationships to one another.

A Win32Service object is created here to manage the listing of the service. The implementation for this class will be shown after this section of code:

w32Service = new Win32Service();

To populate the combobox we databind it with an ArrayList :

cmbServices.DataSource = w32Service.aMgrService;

The DataSource property of the ComboBox named cmbServices is set to an ArrayList. The ArrayList is a member variable of the Win32Service class which contains all the query results from the requesting of the names of all services registered with the Service Control Manager. Setting the DataSource property of a list control is a very fast way of binding a collection to the control. The alternative would be to enumerate through the collection members and add each one to the list, which inevitably means writing more code!

w32Service.selectedService = (string)cmbServices.Items[0];

This sets the initial service in the list to the first item in the ArrayList collection.

ServiceStatus(w32Service.ServiceState());

This method is important, as every time the ComboBox selected value changes we should update the label below the ComboBox to display the state of the newly selected service.

MonitorStopStartEvent();

The method above is used to initially set the event subscription so that whenever a change to a service occurs it is captured by the application. In this way, the display can be changed if, for example, a service is started or stopped. The label containing the service state would then be updated.

The Win32Service class contains the following body of code in the constructor:

SelectQuery sqService = new
            SelectQuery("Win32_Service");
ManagementObjectSearcher
            mgrService32 = new
            ManagementObjectSearcher(sqService);
foreach(ManagementObject
            mgrSingleService in
            mgrService32.Get())
{
aMgrService.Add(mgrSingleService["Name"]);
}

Up until this point we haven't needed to know anything about the namespace of a WMI class. Every managed object has a management path which can be used to locate the object and obtain a handle to an instance. By default, the namespace for the core WMI is root\CIMV2. This doesn't have to be spelt out every time we wish to obtain a handle to an object instance within this namespace. As this is a default, it need not be specified. We can use the ManagementPath class to obtain a handle to a particular object instance:

ManagementObject mgrService = new ManagementObject(new
ManagementPath("\\\\.\\root\\CIMV2:Win32_Service.Name=\"IISADMIN\""), null);

Or

ManagementObject mgrService = new ManagementObject(new
ManagementPath("Win32_Service.Name=\"IISADMIN\""), null);

The variable mgrService will now contain a reference to IISAdmin service through this WMI call. The fully qualified, albeit unnecessary, namespace demonstrates how the Win32_Service class is found within the CIM schema.

In an earlier code segment above we create a SelectQuery. This SelectQuery can be any expression relating to a WMI object collection or single instance. In this case, the query requests everything in the Win32_Service class space, i.e. all Win32 services registered with the Service Control Manager. The ManagementObjectSearcher will find this within the CIM schema. Calling the Get() method will return a ManagementObjectCollection containing all the respective services. As each service instance is an instance of ManagementObject, we can enumerate the collection and cast each object to that of a ManagementObject. We can add the name of each service to the ArrayList that keeps track of the service names (a member variable of the Win32Service class). The unfortunate consequence of ease of use means that the service has to be late bound. In essence, there is no service class which will allow us to invoke methods and use accessor function directly. However, there are various methods which allow the getting and setting of properties (and the invocation of methods) which can be used. For brevity, I've used the indexer in the above example to get the names of each service.

The ServiceState() method reveals a similar mechanism:

public string ServiceState()
{
//Just
            set a blank return string
string temp = String.Empty;
//Object
            searcher
ManagementObjectSearcher mgrService =
            FindService();
//Iterate through the results but just take the
            first one
foreach(ManagementObject mgrSingleService in
            mgrService.Get())
{
temp =
            (string)mgrSingleService["State"];
//Exit
break;
}
return temp;
}

The state of each service is accessed using an indexer. This method invokes the FindService() method (the implementation of which is below):

//Format the query
string query =
            String.Format("Select * From Win32_Service Where Name='{0}'",
            selectedService);
//Create the Wql object
WqlObjectQuery wq =
            new WqlObjectQuery(query);
//Execute the query
return new
            ManagementObjectSearcher(wq);

The query variable holds the WQL query. We'll gloss over WQL later in the case study - in the meantime, a good introduction can be found at http://msdn.microsoft.com/library/default.asp?url=/library/en-us/wmisdk/us_query_84oc.asp&WROXEMPTOKEN=618658Zezi9SpDFSZgQps5Xv6u. Suffice it to say that the syntax is virtually identical to SQL. There are new keywords we'll introduce later, but their meaning is apparent to the context in which they're being used. In the above example, we return a handle to the service instance with a name matching the name of the selected service. The selected service is a member variable of the Win32Service class and is populated each time the ComboBox selected value is changed. The WqlObjectQuery class encapsulates the string query and is executed through the ManagementObjectSearcher.

public void StartService()
{
//Find
            the service to stop
ManagementObjectSearcher mgrService =
            FindService();
//Get the service
foreach(ManagementObject
            mgrSingleService in mgrService.Get())
{
//Resume the service
if((string)mgrSingleService["State"] == "Paused")
            mgrSingleService.InvokeMethod("ResumeService", null);
//or start
            the service
else mgrSingleService.InvokeMethod("StartService",
            null);
//Exit
break;
}
}

This method shows an example of method invocation on the Win32_Service class. The object is of type ManagementObject and thus doesn't correctly represent the service. We have to know a method name to be able to invoke it. We can request all the property name-pair values by using the properties property. In this way we can normally enumerate all the properties at runtime. The methods are called using the InvokeMethod() method, which also takes a list of parameters following the method name. The marshalling and type conversion (to respective unmanaged types) .NET and WMI will handle. The methods used in this example are:

  • StartService
  • ResumeService
  • StopService
  • PauseService

To use ResumeService, the service has to be paused. These methods are "wired-up" to the button controls on the form so that a click event will start, stop, or pause the service.

The Win32Process class is similar to the Win32Service class just described. It maintains similar states, and uses ComboBoxes in the same way. However, on the form there is a multi-line textbox used to display additional data retrieved from the process.

w32Process.CollectDetails();
ListProcessDetailsSelection(w32Process.selectedProcessDetails);

The member variable selectedProcessDetails represents an ArrayList containing supplementary information on the process currently selected in the ComboBox. This is then displayed in the textbox. The two methods shown above facilitate this action.

selectedProcessDetails.Add("Name: "+Convert.ToString(mgrSingleService["Name"]));
selectedProcessDetails.Add("ProcessID: "+Convert.ToString(mgrSingleService["ProcessID"]));
selectedProcessDetails.Add("Termination Date: "+Convert.ToString(mgrSingleService["TerminationDate"]));
selectedProcessDetails.Add("Thread Count: "+Convert.ToString(mgrSingleService["ThreadCount"]));
try
{
//Other specific user details
ManagementBaseObject mbo = (ManagementBaseObject)mgrSingleService.InvokeMethod("GetOwner", null, null);
selectedProcessDetails.Add("Process Owner: "+Convert.ToString(mbo["User"]));
selectedProcessDetails.Add("Owner Domain: "+Convert.ToString(mbo["Domain"]));
}

Above, in code, there are some details being collected from each process: Name, ProcessID, TerminationDate and ThreadCount. The method invocation "GetOwner", which returns a ManagementBaseObject, indexes two properties: User and Domain.

System.Timers.Timer tProcessorCheck = new
            System.Timers.Timer(500);
tProcessorCheck.Elapsed += new
            System.Timers.ElapsedEventHandler(this.Timer_Elapsed);
tProcessorCheck.Start();

In order to routinely check the processor usage, a timer object is created and at five second intervals calls the Timer_Elapsed method, which will update the StatusBar display allowing the percentage usage to be obtained by:

SelectQuery sqProcess = new SelectQuery("Win32_Processor");

A Win32_Processor class allows all the processor information to be returned and viewed.

ManagementObjectSearcher mgrProcess32 = new
            ManagementObjectSearcher(sqProcess);

foreach(ManagementObject
            mgrSingleProcess in mgrProcess32.Get())
{
pUsage = new
            Processor(mgrSingleProcess);
break;
}
statusCPU.Text =
            "CPU use at "+pUsage.LoadPercentage+"%";

In the VS.NET project (downloadable with this case study) you'll notice the file processor.cs on the Solution Explorer pane. This file has been auto generated using a .NET Framework tool. One of the key improvements to Visual Studio .NET over its predecessor, is the use of several command line apps which auto generate code. From an XML document one can create an XSD schema and a C# class which will have all the correct serialization code needed to serialize and de-serialize the XML file, using the xsd.exe tool. WSDL can be generated using wsdl.exe, and for UDDI, disco.exe. The foundation of VS.NET appears to be automatically generated code to support use of Early Binding. WMI is no exception. There is a Framework SDK tool called MgmtClassGen.exe which will create strongly typed classes for WMI classes. Included in the download is a processor sample to illustrate its use. To create the class, the following must be typed at the command prompt (use the prompt from the VS.NET tools start menu - this loads all the environment variables necessary to reference the .NET utility applications):

>mgmtclassgen Win32_Processor /P C:\processor.cs

This will create a Processor class in the above file and directory. If this file is then added to a project and a class instance of Processor is created, all of the class information from the Processor will thus be encapsulated. This, as you can imagine, will enable us to early bind to a WMI Win32_Processor object. There are multiple constructors. Some can be used with a ManagementPath to enable the correct instance to be narrowed. I have foregone that and narrowed to the correct instance using a SelectQuery and a subsequent ManagementObject handle which can also be passed directly into a constructor.

This is the auto generated constructor I've used:

public
            Processor(System.Management.ManagementObject theObject) {
if
            ((CheckIfProperClass(theObject.Scope, theObject.Path,
            theObject.Options) == true)) {
PrivateLateBoundObject =
            theObject;
PrivateSystemProperties = new
            ManagementSystemProperties(PrivateLateBoundObject);
curObj =
            PrivateLateBoundObject;
}
else {
throw new
            System.ArgumentException("Class name does not
            match.");
}
}

It checks the validity of the object by doing a String comparison on the ManagementPath and the name of the expected class. If both are equal, then the constructor will return without generating an exception.

 public class ProcessorEnumerator :
            object, System.Collections.IEnumerator {

private
            ManagementObjectCollection.ManagementObjectEnumerator
            ObjectEnumerator;
public
            ProcessorEnumerator(ManagementObjectCollection.ManagementObjectEnumerator
            objEnum) {
ObjectEnumerator = objEnum;
}

public object
            Current {
get {
return new
            Processor(((System.Management.ManagementObject)(ObjectEnumerator.Current)));
}
}
public bool MoveNext() {
return
            ObjectEnumerator.MoveNext();
}
public void Reset()
            {
ObjectEnumerator.Reset();
}
}

Similarly, and rather interestingly, there is a ProcessorEnumerator class for completeness. The code generation mechanism has thought of all the variables. After all, we may be running our application on a dual or quad processor system.

[DesignerSerializationVisibility(DesignerSerializationVisibility.Hidden)]
[Description(@"The ProcessorId property contains processor-specific information that describes
the processor's features. For x86 class CPUs, the field's format depends on the processor's support of
the CPUID instruction. If the instruction is supported, the ProcessorId property contains two DWORD-formatted
values. The first (offsets 08h-0Bh) is the EAX value returned by a CPUID instruction with input EAX set to 1.
The second (offsets 0Ch-0Fh) is the EDX value returned by that instruction. Only the first two bytes of the
ProcessorID property are significant (all others are set to 0) and contain (in WORD-format) the contents of
the DX register at CPU reset.")]
public string ProcessorId {
get {
return ((string)(curObj["ProcessorId"]));
}
}

To use this new class within our project we could reference it is as follows:

using ROOT.CIMV2.Win32;

This is the default namespace given to the generated class.

You can see from the above that the Processor class wraps up the call to the indexer property in the same way as we would normally do it. Using the Mgmtclassgen.exe utility can be very convenient as you can see. The code is far more readable (and includes a complete descriptive commentary in attributes) and clearer, leaving us to worry about the functional behaviour of our application rather then the implementation details of WMI.

Interestingly, core .NET includes the System.ServiceProcess assembly which contains classes that can do everything that has been demonstrated above with respect to services. The modicum of control that VB developers had over the Windows environment was dwarfed by the capabilities of the experienced Visual C++ developer. However, with the ease of use of WMI and integrated Framework assemblies (such as the System.ServiceProcess), the situation has now been reversed and developers using any .NET language can implement complex solutions in a relatively small time window.

The final part of this example application, and perhaps the most useful, is the implementation of the WMI events mechanism. This is perhaps the most interesting adaptation of WMI. It shows the flexibility necessary in the approach to WMI application development. In the application, I wanted to know when the state of a service had changed (e.g. had the service been stopped?). Any change of object modification, deletion, or creation within the CIM repository can generate a notification. In the case of WMI programming, the client has to subscribe to the event.

This body of code exists in the MonitorStartStopEvent() method:

//Set the wql event query
WqlEventQuery
            wqServiceOp = new WqlEventQuery("__InstanceModificationEvent", new
            TimeSpan(0, 0, 3),
"TargetInstance ISA \"Win32_Service\" And
            TargetInstance.State = \"Stopped\"");
//pass it to the event
            watcher
ManagementEventWatcher mgrServiceWatch = new
            ManagementEventWatcher(wqServiceOp);
//Create the handler
            class
ServiceStopStartEventHandler ssEventHandler = new
            ServiceStopStartEventHandler();
//Chain the
            event
mgrServiceWatch.EventArrived += new
            EventArrivedEventHandler(ssEventHandler.NotificationArrived);
//Set
            the window handle so we can change this
            window
ServiceStopStartEventHandler.WindowHandle =
            Handle;
//Start the monitor running
mgrServiceWatch.Start();

We met the WqlObjectQuery earlier in this case study. The equivalent Event Subscription object is the WqlEventQuery : this takes a WQL query as an input parameter. It is used within the constructor of the ManagementEventWatcher on the line below. The WqlEventQuery parameters are interesting; the first parameter can be any of these values:

  • __InstanceModificationEvent
  • __InstanceDeletionEvent
  • __InstanceCreationEvent

Each of these corresponds to the aforementioned Events of a particular object instance being deleted, created, or modified. The TimeSpan (the second parameter) is set to wait for three seconds to trigger the EventHandler code. A value must be set here - it will not function if this is left null. This event will trigger when the state of the service changes, e.g. from stopped to running. WQL specifies the ISA keyword so as to narrow down the search. In this query, we state that the target instance has to be a Win32_Service and the state property has to equal "stopped" (the change being from running to a stopped state). The first parameter of the WqlEventQuery can take several other values as well. If a class is deleted from the CIM schema, __ClassDeletedEvent will allow an event notification to be subscribed to on this basis.

It should be pointed out for completeness that rather than continuing with programming and waiting for an asynchronous response, it is possible to get the program to halt execution and the current thread to wait until the next event occurs. Using the method mgrServiceWatch.WaitForNextEvent() will do just this, and wait until the WMI event occurs before continuing execution of the application. I've created a second class to be used to handle the event ServiceStopStartEventHandler. The method NotificationArrived is used to process the asynchronous response. As this is outside of the form, for it to arrive at the correct application instance the window handle is passed to it:

public static IntPtr WindowHandle;
public void NotificationArrived(object sender, EventArrivedEventArgs e)
{
//Find the target instance
ManagementBaseObject mgrBaseService = (ManagementBaseObject)(e.NewEvent["TargetInstance"]);
//Check to see if it matches the one in the dropdown list
if((string)(((frmWmiAccess)frmWmiAccess.FromHandle(WindowHandle)).cmbServices.SelectedValue) ==
(string)mgrBaseService["Name"]) ServiceStatus((string)mgrBaseService["State"]);
}

The WindowHandle is stored locally in an IntPtr. When the thread enters the event handler it casts an object to a ManagementBaseObject. This is retrieved from the EventArrivedEventArgs. Specifically, e.NewEvent["TargetInstance"] will return the handle of the instance that has been modified (the service that has been modified). From this, we can publicly access labels and combo boxes to check whether it is being displayed on the form. If it is, then we can update the display text in the label.

Coffee Break

From an XML document one can create an XSD schema and a C# class which will have all the correct serialization
code needed to serialize and de-serialize the XML file, using the xsd.exe tool.

(1) What is the equivalent tool for WSDL?
(2) What is the equivalent tool for UDDI?

Basic WQL Syntax

WMI provides support for data queries, event queries, and schema queries. The first two we've come across in earlier sections and code listing. The third represents the idea of the holistic nature of WMI. Some of the operators described below return schema information i.e. class definitions rather than class instances, association operators can be used to provide an entire "map" of the WMI CIM schema.

WQL is a very logical means of obtaining information from the CIM repository - the use of schema queries is discussed, but is somewhat beyond the scope of this case study. SQL syntax seems a very intuitive way to do many things within a windows environment, and indeed this what WQL is modelled on. SQL Selects are normally restricted to being a way to filter data based on necessary criteria, but until recently this has been applicable solely to database use. Windows developers may have seen this model extended to other areas of information storage, such as spreadsheets (Excel) and text files, and also to Active Directory through ADSI. It follows that as the CIM repository is an information storage model, that a variant of SQL should be used to act as the necessary data filter.

The basic format for WQL is as follows:

SELECT property1, property2 FROM class1

In this case, class1 is an instrumented class. Following on from the previous application we could then write:

SELECT * FROM Win32_Service

Or

SELECT Name, State FROM Win32_Service

The former would return all of the Win32_Service properties, and the latter only the Name and State of each Win32_Service. We could narrow this down to a particular instance as follows:

SELECT State FROM Win32_Service WHERE Name="IISADMIN"

These queries will all return data on a particular Instrumented class. Syntactically, they are very similar to SQL. In the section earlier we witnessed the use of an event query too. When we have decided on the type of notification we're interested in (as described above, deletion, creation or modification) we can supply a WQL query to filter the response generated within the notification:

SELECT TargetInstance.State FROM __InstanceModificationEvent WHERE TargetInstance ISA "Win32_Service"

The above can then be extended to use a polling mechanism to produce consistent result within a time span:

SELECT TargetInstance.State FROM __InstanceModificationEvent WHERE TargetInstance
ISA "Win32_Service" WITHIN 300

Every five minutes the state of class Win32_Service instances will be polled for changes (this is the effect the keyword WITHIN 300 will have). It's also possible to:

SELECT TargetInstance.Name FROM __InstanceModificationEvent WHERE TargetInstance
ISA "Win32_Service" GROUP WITHIN 300 BY TargetInstance.State

This is a classic SQL GROUP BY clause at work within WQL. It will group all of the target instances together by the state of the service. Developers familiar with SQL can extend this to produce a WQL statement which can have a condition attached to the GROUP BY clause. The HAVING keyword is used to set a condition for the grouping of related data items.

Two other useful statements regarding data queries are the ASSOCIATORS OF and the REFERENCES OF clauses. The former will find any associated class instance of a statement. E.g.:

ASSOCIATORS OF {Win32_Service.Name="IISADMIN"}

This will return instances of Win32_ComputerSystem (which is the root dependency) and possibly a collection of related services which are dependent on this service (like w3svc). The latter clause, REFERENCES OF, will produce an instance of Win32_SystemService and one or more instances of Win32_DependentService so that a reference count can be kept of dependencies:

REFERENCES OF {Win32_Service.Name="IISADMIN"}

Application development with WMI

General

Until this point we have discussed how to work with existing classes and instances. This will help us to manage the Windows environment, but will not be that much help if we wish to write WMI into our own applications. This part of WMI has seen the most noticeable improvement with .NET integration. Prior to the introduction of .NET, classes were written using MOF (Management Object Format). This approach can seem very disconnected from the application writing process. Class files are written in a similar vein to the way classes are composed in IDL. Normal data types are ratified and supported by the DMTF, e.g. strings, uints, ints, arrays, booleans, objects, etc. Methods are also declared in MOF. Namespaces are declared through #pragma, and each individual instance of the class is declared through MOF. Mofcomp.exe is a tool that comes with WMI SDK, and is used to compile these classes and register them with the CIM repository. In fact event consumption as we saw in the last application has to be registered with WMI through the MOF Compiler references above. This is a very clunky mechanism all in all, especially for one of the most useful features of WMI. The integration with .NET has meant that this is now yesterday's news. The example web application we'll discuss will monitor generated instances (or events) for consumption.

Events (and class instance data) can be consumed by a remote machine, however, I'll only gloss over the implementation of this. The easiest way would be to retrieve website data through a connection to the machine that could authenticate the request through DCOM. This unfortunately could pose a problem over the internet with firewall configuration and other security measures. Another way to do this would be to authenticate via a web service and retrieve event and instance data via polling a queue. The implementation can be as basic or advanced as necessary, but will need to fit the practical requirements of the system in question.

Windows XP users and developers can take stock in a greater core WMI implementation, which involves the installation of providers which in previous versions of windows would not be considered core - these providers would normally have to be installed separately using the WMI SDK. From Control Panel, choose Computer Management followed by the left hand menu item WMI control. This will connect to the CIMOM after several moments, and relay all of the registered namespaces with WMI. It should like this when the properties right-hand menu item is selected.

This shows a wealth of providers which until recently were never part of the WMI build for the operating system. Instantly you'll see providers for SNMP and for the .NET Framework installed by default. We'll be investigating some of these later in this case study.

It is interesting to note the appearance of a .NET Framework provider. Inadvertently, this is used by all ASP.NET developers. Its presence shows Microsoft's commitment to WMI as an application management technology. All configuration files within the .NET Framework are essentially converted from XML into native WMI which will configure the ASP.NET runtime/.NET Framework through the provider. Developers using the RTM release of Visual Studio will have permission problems using this example, earlier versions of the .NET Framework runtime won't have the high level of security in place as the RTM so should be able to run these examples without configuration alteration. I'll clarify a workaround at the end of this case study, but for more information on new ASP.NET restrictive access see http://www.asp.net/?WROXEMPTOKEN=618658Zezi9SpDFSZgQps5Xv6u. This is a very simple example site application of WMI to track logged in users. We'll return to using some of the principles we've covered in previous sections, but this time with our own implementation of WMI instances and events.

I've created a simple model whereby it's possible to log into a system as a Manager or an Employee. Each of these is a User. A username and password is an access requirement. These are not being checked against a list of users to validate the users authenticity, but slight modifications to the code will allow this to be done in a number of ways, e.g. using an XML file to authenticate users through web.config settings, or using a custom authentication routine storing usernames and passwords in either a database or file. The implementation in this case is quite simple. Several WMI user classes will be registered and details will be checked against WMI instances of users. In effect, part of the application is a user management database, and the web front end can query WMI, check for new instance creation, etc.

How to

WMI for .NET uses the System.Management.Instrumentation namespace to allow classes to be created which can be registered with WMI and the CIM. In the model we'll look at abstract and instance class notation. One of the new improvements to the .NET Framework over the original Windows DNA programming model is the ability to embed attributes into code segments at method, class, or property level. Attributes allow metadata to be embedded in the code. When the compiler sees an attribute it will make an association with a class of type System.Attribute.

This example highlights the way that attributes can be used to aid common programming problems. In this example, multiple inheritance would need to be used to allow a class hierarchy to be put in place in conjunction with the WMI inheritance model (to allow publication of a WMI class instance or a WMI event class to be fired). Ordinarily, implementation inheritance could have provided a solution to this problem in other languages or frameworks. This would apply to such languages that don't support multiple inheritance (e.g. java, .NET). However, any .NET language can use attributes to simplify framework design and code in the way described above.

[InstrumentationClass(InstrumentationType.Abstract)]
public
            class User
{
public string m_Username =
            String.Empty;
public string m_Password = String.Empty;
public
            User(string username, string password)
{
m_Username =
            username;
m_Password = password;
}
}

The attribute above the class definition allows us to define that this class should be abstract. In this way it would be treated like the Win32_Service class itself, i.e. defining all the properties and methods of this class but unable to be instantiated itself. All the classes which inherit from this will inherit its implementation.

[InstrumentationClass(InstrumentationType.Instance)]
public
            class Manager : User
{
public Manager(string username, string
            password) : base(username, password)
            {
Instrumentation.Publish(this);
}
}

Manager, however will represent an instance of the above class. It will inherit its implementation through standard inheritance, and through attribution the idea of an instanced WMI class will be defined. As you have probably guessed, when this is first created and registered with the CIM it will be available for a client to listen for by subscribing to the __InstanceCreationEvent which was discussed earlier. The code in the Manager constructor invokes the base class constructor passing it the username and password. A static method which is called via the Instrumentation class can be used on all classes marked with attributes taking any valid enum value (i.e. an instance or event). If the class above had inherited from Instance it would have invoked a non-static publish method rather than passing the this reference to a static method. It would have been implemented as below:

this.Published = true;

Either way it is constructed is just as valid. The implementation of the Employee class is exactly the same as this. Any client which tries to detect an instance creation of the User class will throw an InvalidClassException. None of the information in the assembly is published to WMI until the classes themselves are published to the CIM repository. This is normally done through the installutil.exe utility which will inspect the classes and check the attributes on a default installer class and publish all the marked classes to the CIM repository.

In the Visual Studio Tools start menu item there is a command prompt with all the environment variables set for execution of command line tools. When started, the installutil program can be executed directly. This executable will inspect the assembly looking for a loader class (described later). Finding this, it will continue to introspect attributes and classes within the assembly. The repercussion being that attributed classes or WMI derived classes will be published to the CIM repository.

The example referred to uses WMI instances of registered classes, but it could equally use WMI events (this is very useful if actual data isn't needed, but some kind of asynchronous callback is needed instead). The most common use of WMI events is for error reporting, i.e. building a client to monitor real-time error info for a website/application.

It should be noted that an assembly reference needs to be included in the top of the file. I've left this in the main C# file (class1.cs), but alternatively it can be included in the self-generated AssemblyInfo.cs file. This file contains all the other assembly references such as name and version number. The using System.Management.Instrumentation directive is the only prerequisite. This allows any inspection tools to identify that the assembly has classes instrumented within WMI. As is evident, it also contains the new namespace:

[assembly:Instrumented("root/richardc")]

The installutil application will publish the classes present and marked to the CIM repository only if there is an install class present. The install class will look like this:

[System.ComponentModel.RunInstaller(true)]
public
            class installWMIUserStuff : DefaultManagementProjectInstaller
{
static void Main(string[] args)
{
User Ben = new
            Manager("ben", "password");
User Thelma = new Employee("thelma",
            "password");
User Trip = new Employee("trip",
            "password");
}
}

The install class should be attributed so that the installutil application can identify the class, and it should also inherit from the DefaultManagementProjectInstaller as above. In this way it will install and create and publish the necessary classes to WMI. The process is slightly more complicated if the project (and class) already uses another installer. In this case there must be a body of code in the main method which will add the installer to an installer collection. The documentation that comes as standard with VS.NET covers this in detail, see ms-help://MS.VSCC/MS.MSDNVS/cptools/html/cpconinstallerutilityinstallutilexe.htm. The application can then be started, as illustrated in the command window screenshot below:

The output should be similar to that above. The executable file (WMITrigger.exe) should be run once to create instances of the classes to be used with the web client. The trace is very comprehensive and operates within the context of a transaction. If there is a problem with the published classes then a rollback will take place. The quickest way to see the results of this is to use the CIM Studio product (described briefly earlier) that forms part of the WMI SDK. In the bitmap below it is evident that the abstract nature of the User class and the hierarchy of the Employee and Manager classes have been maintained. Also evident is the adoption of the two properties Username and Password to equivalent WMI class properties. This can be found by connecting to the root namespace (although I have created my own namespace root/richardc in the actual application, any namespace can be used - in the way illustrated above) when a copy of CIM studio is fired up.

The button's click event is handled by the following WMI consumer code:

private void btnLogin_Click(object sender,
            System.EventArgs e)
{
try
{
string query = "SELECT *
            FROM User Where
            Username='"+txtUsername.Text+"'";
ManagementObjectSearcher m =
            new ManagementObjectSearcher(new
            ManagementScope("\\root\\richardc"),
new
            WqlObjectQuery(query));
foreach(ManagementObject o in
            m.Get())
{
if(rbEmployee.Checked&&(o["__class"]=="Employee"))
{
goto CheckPassword;
}
else
            if(rbManager.Checked&&(o["__class"]=="Manager"))
{
goto
            CheckPassword;
}
else
{
throw new Exception("Login
            failure!");
}
break;
}
CheckPassword:
lblPasswordVal.Text
            = "Login Successful!";
}
catch(Exception
            e)
{
lblPasswordVal.Text = e.Message;
}
}

This code is fairly simple. It begins by creating a WQL query requesting an object of type User with a Username of whatever the user entered into the textbox. The query is executed under the root/richardc namespace where the WMITrigger application is currently running and has published User objects (the code therein should formerly use the Instrumentation.Revoke(this) method before it terminates - but doesn't. When the program terminates, WMI "garbage collects" these instances). The __Class property lets us know the name of the class so that it can be typed properly. In this way, if somebody claims to be a Manager we can check whether the class actually matches this claim.

public class LoginFailure :
            BaseEvent
{
public
            LoginFailure()
{
return;
}
}

I've identified the above simply for completeness. I haven't implemented it but it's very self explanatory. Events are registered in the same way using the installer class as described above. Attribution may be used instead of the inheritance shown above. In any case, events are actually perfect for a real-time web model where events such as purchases, exceptions and logins need to be recorded in numbers without data. In the same way as the published attribute above is used to publish a class instance to the WMI repository, so is the Instrumentation.Fired(this) or the this.Fire() method used when it is necessary to fire an event. In the first half of the case study we saw how easy it was to subscribe to events and in the same way, as described earlier, the only point to remember is that the WQL used to query above cannot be used (as it not a data query), instead a WqlEventQuery object is used to detect a new instance of the event class.

WMI Application Control

SQL Server Provider Installation

WMI providers are present with both installations of SQL Server and SQL Server 2000. They are not installed by default, but come on the installation CDs. In this section we'll look at how WMI can be used to control SQL Server in the same way that SQL-DMO (Distributed Management Objects used by application developers to gain control of SQL Server objects and instances) can be used directly (see Wrox book Professional SQL Server 7.0 Development using SQL-DMO, SQL-NS and DTS, by Frank Miller and Rachelle Reese, ISBN: 1861002807). There are many applications that allow WMI control of exposed objects, but as SQL is probably the most useful we'll focus on this. Others are Windows Installer, SMS, Microsoft Operations Manager and Microsoft Application Center. There are many third party applications that expose WMI control too.

In order to install the SQL provider:

  • browse to x86\other\wmi
  • run the Windows Installer package SQLWMI80.msi. This will install the provider.
  • Check, by using CIM Studio, and connecting to the SQL Server namespace MicrosoftSQLServer or as described and shown in an earlier bitmap
  • Find the WMI control menu option in the Computer Management snap-in
  • right-click on properties and check the security tab
  • Under root the aforementioned namespace for SQL Server should be present.

Using the SQL Server Provider

In this section I plan to demonstrate a simple SQL Server inspection application for reading database objects. All of the techniques covered here have been introduced in previous sections of this case study so it will be more apt to introduce the SQL Server WMI classes used.

To populate the main combobox listing each database, the following code is used:

string query = "SELECT * FROM
            MSSQL_Database";
ManagementObjectSearcher m = new
            ManagementObjectSearcher(new
            ManagementScope("\\root\\MicrosoftSqlServer"),
new
            WqlObjectQuery(query));
foreach(ManagementObject mo in
            m.Get())
{
dbNames.Add(mo["Name"]);
}

One of the top-level classes in the SQL Server class hierarchy is obviously the MSSQL_Database class. Each instance of this represents a valid database object. The name of each is added to the ArrayList to populate the combobox. This application is meant to be a simple illustration of the relationships between all WMI SQL Server objects. However, real-world examples can easily be extrapolated. One of the many cool example uses (which I along with many other developers have done in the past with DMO) is to access all stored procedures in a database, and dynamically write the data access layer code of a DNA application in C++ or VB. The same thing can be done using WMI allowing C# code to be written using SqlDataReader classes and SqlParameters.

This application will take two values, "table" or "sp", depending on the user's selection of showing Tables or Stored Procedures. In the event that a table is selected, then the table design will be shown in text. In the event that a stored procedure is selected, the stored procedure code will be shown in the textbox. For a fuller explanation of all things attainable through the use of the WMI provider refer to the WMI Provider Schema Documentation in the WMI subsection of the SQL Server start menu item. There are however endless possibilities for using this technology - it supports dynamic database backups and recovery, database creation, index rebuilding, file reallocation, and log analysis. It can therefore be a very powerful tool for application monitoring and automation of SQL Server.

public void ListObjects(string
            objectType)
{
string tempObjectType =
            String.Empty;
switch(objectType)
{
case
            "table":
tempObjectType = "MSSQL_Table";
break;
case
            "sp":
tempObjectType =
            "MSSQL_StoredProcedure";
break;
}

objectNames.Clear();
string
            query = "SELECT * FROM "+tempObjectType;
ManagementObjectSearcher mos = new
            ManagementObjectSearcher(new
ManagementScope("\\root\\MicrosoftSqlServer:MSSQL_Database.Name=\""+selectedDatabase+"\""),
            new WqlObjectQuery(query));
foreach(ManagementObject m in
            mos.Get())
{
objectNames.Add(m["Name"]);
}
}

The query above is slightly different to previous WQL queries that have been discussed so far. In the case of tables, the query should read SELECT * FROM MSSQL_Table, but an instance handle needs to be obtained to the correct database instance. This is done through ManagementScope which uses the SQL Server namespace in conjunction with the database name filter to obtain an exact instance handle. The tables can then be enumerated and added to the combobox. The selection (as in the first example) takes place on the combobox selectedindexchanged event.

try
{
string temp =
            String.Empty;
string checkThis =
            (string)cmbObject.SelectedItem;
switch(checkThis.Trim())
{
case
            "Stored Procedures":
temp = "sp";
break;
case
            "Tables":
temp =
            "table";
break;
}
if(!temp.Equals(String.Empty))
{
sqlWMI.ListObjects(temp);
ShowObjects();
}
}
catch(Exception)
{
///
}

Above is some boilerplate code in the application that switches between using stored procedures and passing an "sp" parameter, and tables and passing a "table" parameter to the ListObjects method above. In this case, it will inevitably choose whether to use the WMI table object MSSQL_Table or MSSQL_StoredProcedure.

These features were created using the generated classes - it is self-explanatory what these two classes do and in what way they are being used in the database form. I've included the MSSQL_Table class and the MSSQL_StoredProcedure class in the project, they have been written using the mgmtclassgen utility described previously. Perusing the auto-generated C# proxy class files is a good way of developing an understanding of the capabilities of the SQL Server WMI classes. The MSSQL_Table class has a wealth of methods which enable it to truncate data, and import and export data into the table object. The class will allow a table collection to be populated with all tables in the database (or server). Indexes can be rebuilt and new optimisations achieved. There are methods to create new tables assigning basic SQL options (such as column names and additional column options). As I've shown in the brief example above, it's possible to return the number of rows in each table as well as description of the SQL that created the table. The data in the table can be exported using the ExportData method and using a FileStream object a CSV file or some other file type can be created to back up the table to disk. This can effectively be done to all SQL Server objects.

The StoredProcedure class (the auto-generated C# proxy for the MSSQL_StoredProcedure WMI class), although less useful then the table class (since there is a vast quantity of methods and properties which allow complete control of table objects), still contains several useful features. The text property, as indicated in the code, will return the description of the Stored Procedure so that the code used to generate it can be displayed.

A screenshot of the Database Analysis application.

Using the self-generated classes encapsulates WMI objects nicely so that common operations can be done using .NET classes (and intellisense). This makes the code a good deal easier to read. However, without the use of the self-generated classes, we could just as easily construct a WQL data query as follows:

string query = "SELECT * FROM "+tempObjectType+" WHERE DATABASENAME=\""+selectedDatabase+"\"";

This will return all objects depending on the selected database or the object type, so it will query the MSSQL_Table class or the MSSQL_StoredProcedure class. To display either of these, the following queries can be used to populate the textbox as shown above:

string query = "SELECT * FROM MSSQL_Table
            WHERE DATABASENAME=\""+databaseName+"\" AND
            NAME=\""+objectName+"\"";

Obviously in the event of selecting a stored procedure as shown above, the selected class will need changing to MSSQL_StoredProcedure.

Other Providers, other uses

Using System.Diagnostics

I thought this was worth mentioning, as some of the key uses of WMI have been included in the .NET Framework already. We discussed earlier the use of performance counters. C++ programmers may be familiar with performance counter data access (ref. Programming Server-Side applications for Windows 2000 by MSPress, Richter, Clark). This model has been simplified a great deal through the use of WMI and directly through the use of the System.Diagnostics assembly. Applications can now use performance counters which measure different values of services and executables. This can be very useful for generating live or recorded trace information breaking down usage statistics. Indeed, ASP.NET comes complete with several performance counters to monitor all things from average page requests to the number of applications or sessions currently running. Below is a screenshot of the Performance Monitor application (a standard windows application) with ASP.NET counters added to the monitoring control. There is also a wealth of .NET CLR counters which enable developers to see everything that the CLR is currently involved in, from memory management to garbage collection. This feature of the .NET Framework becomes incredibly useful when building intelligent applications which respond to the current performance usage. In this way, ASP.NET applications can be built to serve minimal content to avoid congestion on the server.

As well as monitoring system and application behaviour through the use of this namespace, developers can also take advantage of building performance counters themselves. There are facilities to use the Event Logs (as opposed to using the WMI provider directly) and also to monitor process activity.

Earlier I mentioned the use of the .NET Framework WMI provider. This can be used to configure almost all aspect of the .NET environment. It can be seen visually through an MMC snap-in which will load if the .NET Configuration menu item is selected in the Administrative Tools menu. In this case, the snap-in uses the WMI provider to configure all aspects of the environment. The provider can alternatively be used directly within an application so that configuration "on the fly" can occur depending on the needs of the server and the changing nature of the environment, e.g. to dynamically configure machine remoting entries in machine.config for dynamic discovery of network remoting services, etc. (the machine.config file resides in the C:\WINDOWS\Microsoft.NET\Framework\v1.0.3705\CONFIG and is responsible for system-wide configuration of the .NET Framework - this approach is commonly used with other runtime environments such as the J2EE).

Conclusion

I have tried to demonstrate some uses of WMI in this case study as well as to elaborate on how WMI can be built into your own applications. The .NET Framework has revolutionised access to WMI, simplifying things that were previously very complicated. With the understanding of the use of all of the base classes within the System.Management namespace, WMI monitoring or management can be built into any application very quickly and easily. I have used simple Win32 classes to demonstrate basic principles of management, but I would encourage readers to experiment using these principles with all classes - in the Win32 namespace alone there are over 600 classes! Classes such as Win32_ParallelPort can be used to collect information on the parallel port, other classes can be used to retrieve and set network adapter information and printer information, and other classes can mimic the at schedule and add to the Windows timed jobs. As more applications are written using WMI, it will appreciably become the unique Windows standard for application management and exposed interfaces.

Case Study Information
Author | Richard Conway
Technical Editor | Adam Ryland
Project Manager | Helen Cuthill
Reviewers | Shefali Kulkarni, Sean Schade, David Schultz, Adam Ryland

If you have any questions or comments about this case study, or suggestions for further pieces, please contact the technical editor. If you wish to talk about the case study with other subscribers, please visit our forum here.


.NET Web Data ToolKit

.NET Web Data ToolKit
http://www.csharptoday.com/content.asp?id=1831
Article Rating: 4
Published on March 11, 2002
Abstract
Think about how useful it would be to have a toolkit that allows you to SELECT nuggets of information FROM live web pages as though selecting from tables in a relational database…using industry-standard SQL. Imagine if you could capture the results into an ADO.NET DataTable, allowing you to treat web data like any other data source. This Case Study by Tony Loton describes the problems encountered - and solutions devised - during the design and implementation of just such a toolkit.
Case Study

FEEDBACK: We would love to hear what you think of this case study, and what topics you would like to see covered in future. Please take a few moments to mail the Editor to give us your opinions.

Think about how useful it would be to have a toolkit that allows you to SELECT nuggets of information FROM live web pages as though selecting from tables in a relational database.using industry-standard SQL. Imagine if you could capture the results into an ADO.NET DataTable, allowing you to treat web data like any other data source. This Case Study describes the problems encountered -- and solutions devised -- during the design and implementation of just such a toolkit.

So that you know where we're heading, the application we'll be discussing takes a web page like this:

...and processes SQL statements like this:

SELECT .table[1].tr[0].td[1].table[1].tr[1].td[0].table[0].tr[*].td[0].a[0].text[0] AS title,
.table[1].tr[0].td[1].table[1].tr[1].td[0].table[0].tr[*].td[1].text[0] AS author,
.table[1].tr[0].td[1].table[1].tr[1].td[0].table[0].tr[*].td[2].text[0] AS isbn
FROM http://www.wrox.com/Books/Books.asp?section=1_1
WHERE title LIKE 'Professional*'
ORDER BY isbn ASC

...to produce an ADO.NET DataTable like this:

As a taste of what is to come, notice that in the SQL statement we're selecting sets of HTML tagged elements as though selecting fields of a relational table, and we're taking the data from a web page URL in the same way that we would take data from a table.

There will come a time - no doubt - when we won't need to do this because every online information provider will have a full web service using XML and SOAP. That day has not yet come, so let's press on.

Contents

Section
Scope Of The Case Study
Challenge 1: Retrieving Web Data via HTTP Submissions
Challenge 2: Parsing HTML and / or XML Elements
Challenge 3: Selecting Elements
Challenge 4: Flattening the Tag Hierarchy to form a Relational Table
Challenge 5: Supporting Structured Query Language
Case Study Review
Potential Applications

Background to the idea

It all started with my original article "Accessing the World's Biggest Database ", published on the JavaWorld web site in March 2001 (http://www.javaworld.com/javaworld/jw-03-2001/jw-0316-webdb.html?WROXEMPTOKEN=620568Zhiox25QOrrsnTPfm53p). That idea manifested itself in the WebDataKit commercial implementation for the Java 2 platform, an early version of which was written up as a case study in "Professional Java Data" (Wrox Press, ISBN 1-8610041-0-9).

With the Java implementation quite stable, I then turned my attention to the Microsoft Platform, and first to the web query support provided out-of-the-box with Microsoft Excel. I mentioned the Excel approach in my "Working with Web Data in C# " article (http://www.csharptoday.com/content.asp?id=1782&WROXEMPTOKEN=620568Zhiox25QOrrsnTPfm53p). If you're looking for a really easy way to get web data into a spreadsheet for further processing, it's well worth a look. In that article, I also introduced the basic mechanisms for extracting web data into a C# program on the .NET framework.

At the time of writing, the commercial WebDataKit for .NET is under development as a beta product. You can find out more about that, and try it out, at http://www.lotontech.com/wdkDotNet.html?WROXEMPTOKEN=620568Zhiox25QOrrsnTPfm53p.

Now that you know where the idea came from, and what progress has been made so far, let me now tell you about what we'll cover in the case study.

Scope of the Case Study

It's not practical to give a line-by-line account of the complete code, and in any case, it was written as the basis of a commercial product rather than as a teaching aid. and so is not too pretty!!! The underlying ideas are more important than the code itself, so think of this case as a recipe - rather than a blueprint -- for creating a Web Data Toolkit. We'll discuss some of the key decisions that were made during the design and implementation, and in that context you'll find out how I tackled the problems of how to...

  • Extract data from web pages stored on live web servers or as files on disk.
  • Make individual HTML and XML elements uniquely addressable.
  • Select elements using wildcards and regular expressions, to pick out related sets.
  • Flatten a hierarchic HTML or XML structure to a flat relational structure.
  • Parse SQL statements and use SQL to query web pages a though they were relational database tables.

From a programming point of view, you'll see some interesting uses for the WebRequest, Regex, and DataTable classes of the .NET framework, plus other classes from these namespaces:

  • System.Net
  • System.IO
  • System.Text.RegularExpressions
  • System.Data

At the end I'll provide some code that you can run so you can see that it really does work.

Design and Implementation Decisions

I'll present the design and implementation decisions as a series of challenges that were faced during the development of the Web Data Toolkit. For each one, I'll present the essence of the algorithmic solution or .NET technical solution that met the challenge.

I've provided the complete source code as it stands at the time of this case study. It's warts-and-all source code, included for reference only and not intended for further distribution. Where appropriate in the discussion that follows, I've included some pointers into the source files.

Challenge 1: Retrieving Web Data via HTTP Submissions

Fundamental to the rest of the toolkit is the ability to fetch the text content from a web URL (beginning http://), or a file URL (beginner file://). For web URLs, we might need to pass some parameters in order to specify exactly the response we want, and such parameters will generally be appended to the end of the URL like this.

http://www.wrox.com/Books/Books.asp?section=1_1

In this case, we're assuming that the web site will be expecting requests via the HTTP GET method. For sites expecting requests via the POST method, the URL would be.

http://www.wrox.com/Books/Books.asp

and we'd send the parameter section=1_1 as a separate stream of data.

Our solution must support one of those submission methods, or it could support both, because it is possible to append parameters to a URL and post some data at the same time.

The following method, which you could easily incorporate into your own programs, makes use of the .NET System.Net.WebRequest and System.Net.WebResponse classes to issue a HTTP submission and to capture the entire response as a single text string.

First, pass in a URL (which might have parameters appended), some additional text to post (which is optional), and a HTTP method (which may be "GET" or "POST"):

public static String getTextFromURL (String url, String postText, String method)
{

Create a WebRequest from the URL and set its method:

WebRequest request = WebRequest.Create
            (url);
request.Method = method;

Enter this section of code only if the method is "POST", and only if it is a web-based URL, capable of being cast to HttpWebRequest because it begins with http://:

    if
            (method.Equals("POST") && (request is
            HttpWebRequest))
{
request = (HttpWebRequest)
            request;
request.ContentType =
            "application/x-www-form-urlencoded";

byte [] bytes =
            null;
bytes = System.Text.Encoding.ASCII.GetBytes
            (postText);
request.ContentLength = bytes.Length;
Stream
            outputStream = request.GetRequestStream ();
outputStream.Write
            (bytes, 0,
            bytes.Length);

outputStream.Close();
}

For all GET requests, and requests to file URLs (beginning file://), we would not be able to post extra text using the code above, and attempting to do so would raise an exception. You should also be aware that for the text to be posted correctly, the ContentType of the WebRequest must be set as shown.

Whether we post additional text or not, we will need to get hold of the HTTP response:

WebResponse response = request.GetResponse();
Stream responseStream = response.GetResponseStream();

...and read the text:

StreamReader reader = new StreamReader (responseStream,System.Text.Encoding.Default);

String text=reader.ReadToEnd ();

Finally, we'll close and return the text.

reader.Close();

return text;
}

For further reading you are invited to look at the real getTextFromUrl(.) method in the WebDataEngine.cs source file.

That's quite a useful method, and not too different from my original Java solution that used the java.net.URLConnection and java.io.InputStream classes.

Challenge 2: Parsing HTML and / or XML Elements

In order to pick out nuggets of information from web pages, using structured queries or otherwise, those elements must be made uniquely addressable. For each piece of text enclosed within HTML or XML tags, I'd like to label the position of that text within the tag hierarchy. Taking as a basis the web screenshot shown earlier, I'd like to come up with something like this:

Notice how each piece of text on the page is listed in the form:

KEY (unique position) = CONTENT (the text)

In order to support an algorithm that traverses the hierarchy of tags and builds element keys by counting the tags at each level, we need an underlying mechanism for picking out (parsing) the tag- and text- tokens in a string such as:

"<HTML><HEAD><TITLE>Sample HTML
Table</TITLE></HEAD><BODY><TABLE><TR><TD>row0,col0</TD><TD>row0,col1</TD></TR>
<TR><TD>row1,col0</TD><TD>row1,col1</TD></TR></TABLE></BODY></HTML>"

At this point I made a major departure from the original Java implementation. In that version, I had taken advantage of classes from the javax.swing.text.html.HTMLEditorKit and org.xml.sax packages to parse the HTML or XML tags. At first glance I saw no equivalent parser classes in the .NET framework, so I wrote a very basic tag parser (and I mean very basic!) myself.

I then decided it was a much better approach anyway because it runs quicker, is not dependent on any third-party (e.g. SAX) class library, and is useful for parsing any kind of text embedded within <starttag> and </endtag> pairs. For this application we don't even need to ensure the validity of the HTML or XML structure, as long as there is some kind of structure. However, it's not completely problem-free though as you'll see at the end.

The Tag Parser has a method named getTagOrText(.) that simply returns the next tag or inter-tag text from the input string provided by the getTextFromURL(.) method shown earlier. It's called in a loop like this.

while (
            (tagOrText=getTagOrText(taggedText,pos))!=null)
{
// Do
            something with the tag or the text
// then set the parsing
            position for next time
            around.

pos=pos+tagOrText.Length;
}

In a nutshell, the getTagOrText(.) method takes anything between < and > as a tag, and anything between > and < as text.

The comment // Do something with the tag or the text is where we would keep track of our position within the embedded tag hierarchy (by maintaining a tag path of the form .table[0].tr[1].td[3]) as we meet each new tag, or where we would add a new text element at the current position.

That description should suffice to paint a picture of what happens. The code itself is somewhat convoluted and distracting, and shows off nothing in terms of the .NET framework, so we won't dig too deep except to point out how we deal with tag attributes and non-structural HTML tags.

Tag Attributes

If you look back at the figure shown earlier, you will see that besides capturing the text content from between tags we're also capturing the values of attributes that appear within the tags themselves. Attribute entries appear like this with an @ symbol denoting the name of the attribute:

.table[1].tr[0].td[1].table[1].tr[1].td[0].table[0].tr[2].@bgColor = white
.table[1].tr[0].td[1].table[1].tr[1].td[0].table[0].tr[2].@height = 15

In this case, there are two attributes with name bgColor / value white and name height / value 15 respectively, extracted from a tag like this:

<TR bgColor=white height=15>

Given a tag like that one, the following code segment picks out the attribute name / value pairs using the .NET Regular Expressions.

Regex attributeRegex=new Regex("(?<name>\\w+)\\s*=\\s*(?<val>(?<q>\"?).+?\\k<q>)(?=(\\s+|\\s*>))");

Match attributeMatch = attributeRegex.Match(fullTagOrText);

for ( ; attributeMatch.Success; attributeMatch = attributeMatch.NextMatch())
{
String name=""+attributeMatch.Groups["name"];
String val=""+attributeMatch.Groups["val"];

// Now that we have the attribute name and value, so we would do
// something with them.
}

In that code we first created an instance of the Regex class to hold our regular expression pattern (the awesome looking text string). Note that the regular expression engine interprets certain escape sequences - such as \s for space character - and we must prepend an additional \ (to give \\s) to prevent the C# compiler from attempting to interpret those escape sequences itself.

We use an instance of the Match class to match our regular expressions against the tag text, and we step through the matches.of which there may be more than one. Within the loop we pick out the text segment that our regular expression matched as an attribute name (look for "<name>" in the regular expression string), and the text segment that our regular expression matched as the attribute value (look for "<value>" in the regular expression string).

Now you'll be wondering about the regular expression string itself. A regular expression is a text pattern that is superimposed onto an input string to see if any segments of the string match the pattern. I'll explain the regular expression used in code here, but if you want to know more about regular expressions in general it's worth taking a look at my article "Introducing .NET Regular Expression with C# " at http://www.csharptoday.com/content.asp?id=1786&WROXEMPTOKEN=620568Zhiox25QOrrsnTPfm53p.

The following table describes the pattern we're looking for. You should be able to relate each entry back to the original regular expression string.

Regular Expression | Explanation | ...in plain English
(?<name>\\w+) | Match one of more word \\w+ characters captured into a group called <name> | The attributed name
\\s*=\\s* | Followed by zero or more spaces \\s* followed by an equals sign = then zero or more spaces \\s | An equals sign (with spaces either side)
(?<val> | Followed by a group called <val> that captures... |
(?<q>\"?) | A quote \" character that's optional ? and which is captured into a group called <q> | An optional quote character
.+? | Followed by any character(s) . repeated at least once but as few times as possible +? | The attribute value
\\k<q> | Followed by the next character matching the back referenced \\k character that was previously captured into group <q> | A closing quote if there was an opening one
) | end of <val> group |
(?=(\\s+|\\s*>)) | Followed by a look ahead group of one or more spaces \\s+ or | zero or more spaces and an end tag brace \\s*> | Some spaces or the tag's closing brace up ahead

To make it a little clearer, here's how it maps twice onto a sample tag with attributes.

Actually, for that sample tag, the regular expression could be made somewhat simpler because there are no quotes surrounding the attribute values and no spaces either side of the equals signs. But, we need to allow for possible variations in syntax, and in that respect we could have made it even more complex to handle -- for example -- attribute values enclosed within single quotes.

If you think it looks complicated, rest assured that it's easy when you know how, and it's certainly more compact - and adaptable - than writing pure C# code to look for the same patterns. As I said before, you can look at my earlier article if you want to learn more about Regular Expressions.

For your information, in the original Java implementation I could not take advantage of the JDK1.4 regular expression classes, as they had not yet arrived. Fortunately, the Java API for XML Processing (JAXP) provided a callback method that was invoked for each tag encountered, and which included a list of attribute name / value pairs as a parameter. So I simply had to write an implementation for this method signature:

public void handleStartTag(HTML.Tag tag, MutableAttributeSet attributes, int pos)

Non-Structural HTML Tags

Now let's think again about the keys allocated to individual text elements according to their positions within the source page. In the following example, you'll notice that I've chosen to ignore certain HTML tags when forming the key.

.table[1].tr[0].td[1].table[1].tr[1].td[0].table[0].tr[2].text[0]

The full path to that element would really be something like this:

.html[0].body[0].table[1].tr[0].td[1].table[1].tr[1].td[0].table[0].tr[2].b[0].text[0]

The bold text shows which tags I've chosen to ignore because they increase the size of the key unnecessarily while at the same time.

  • not adding to its uniqueness, in the case of .html[0].body[0] that would be shared by all elements
  • or splitting text elements unnecessarily, as would non-structural tags like the <b> (i.e. .b[0] ) bold tag.

To reduce the length of the keys and to ensure only structural HTML tags are included, I pass two arrays into the TagParser, one for matched HTML tags (those that have a matching end tag) and the other for empty - or simple -- tags (i.e. those with no closing tag). Any tag found in the input text that is not in one of my lists will not be added to an element's key. That's possible because HTML has a finite set of tags and we know what they are. For XML files we'll have no special knowledge, so all tags will be used in forming keys.

Here is the code segment that sets up the arrays of structural HTML tags for the TagParser constructor.

String[] matchedHTMLTags=new String[] {"<table>","<tr>","<th>","<td>","<a>"};

String[] emptyHTMLTags=new String[] {"<img>"};

TagParser tagParser=new TagParser(matchedHTMLTags,emptyHTMLTags);

At this point it's worth noting that although the examples in this case study concentrate on HTML tables, the inclusion of the <a> and <img> tags in those lists means that even in a table-free page you could still pick out all the links (.a[*].@href for the URLs, and .a[*].text[0] for the link texts) or images (.img[*]) as column data. By adding more tags to the lists, you could pick out elements from between other tag combinations, like <div> and <span>.

We run the parser via its getElements(.) method, passing in the source text from the web page, and we return the result to whatever called this code.

return tagParser.getElements(taggedText);

The end result is an ArrayList containing instances of a TaggedElement class, having two read-only properties -- a key and content - plus a read/write property selectSpec.

public class TaggedElement
{
private
            String elementKey;
private String elementContent;
private
            String elementSelectSpec;

public TaggedElement(String key,
            String
            content)
{
this.elementKey=key;
this.elementContent=content;
}

public
            string key
{
get { return elementKey; }
}

public
            string content
{
get { return elementContent;
            }
}

public string selectSpec
{
get { return
            elementSelectSpec; }
set { elementSelectSpec=value;
            }
}
}

The purpose of the key and content properties should be obvious, but you might be wondering about the selectSpec property. All will become clear as we move on.

For further reading you are invited to look at the real getTagOrText(.) and getElements(.) methods of the TagParser.cs source file.

Challenge 3: Selecting Elements

Having cracked the problem of retrieving a set of uniquely addressable elements comprising the source web page, we then needed a way of picking out a subset of elements that we're interested in. From the book listing on the Wrox web site we might be interested only in the book titles.

The following screen shot shows a breakdown of the elements at the Wrox site. You'll notice from the URL that for convenience we're actually looking at a saved version of the page, which serves to demonstrate the support for the file:// protocol in addition to the http:// protocol (see Challenge 1). That saved version has been included in the download bundle so that you'll be able to try it out exactly as I have.

Now look what happens if we place some filter text into the Key Filter box, to pick out only book titles:

The format of the elements keys, and the format of the key filter used to select them, should be intuitive enough if you're used to hand-coding HTML. Later you'll see how our filters will be given alias names to make them even more meaningful. XML tags will generally have semantically significant names rather than the syntactically significant names of the HTML tags, so when dealing with XML source data you'll see more meaningful element keys like .PLAY[0].ACT[0].TITLE[0].

Implementing that functionality meant matching the Key Filter against the key property of each TaggedElement in the list. Another job for regular expressions, but the problem is that specifying a key filter using full regular expression notation would require a less-than-friendly string like this:

\.table\[1\]\.tr\[0\]\.td\[1\]\.table\[1\]\.tr\[1\]\.td\[0\]\.table\[0\]\.tr\[\d+\]\.td\[0\]\.a\[0\]\.text\[0\]

The backslash \ character preceding each dot, open square brace, and close square brace, would be necessary for the Regex engine to treat those as literal characters rather than escape sequences with special meanings. And the \d+ sequence in that regular expression, meaning "match one or more decimal digits", would be used in place of the simple wildcard * characters.

Clearly users would not want to enter such a convoluted string, so we use the simpler syntax shown in the screenshots. The first step in our method to pick out matching elements therefore converts the simpler syntax (the Key Filter, passed as the selectSpec parameter in code) into the full regular expression syntax:

public static ArrayList getSelectedElements(ArrayList originalElements, String selectSpec)
{
String filter=selectSpec;

filter = filter.Replace("[", @"\[");
filter = filter.Replace("]", @"\]");
filter = filter.Replace(@"\[*\]", @"\[\d+\]");
filter = filter.Replace(".", @"\.");

filter = "^"+filter+"$";

At this point we have a fully formed regular expression, with a ^ prepended and a $ appended to ensure that the pattern is matched against the whole of the key property from start to finish (otherwise a shorter selectSpec could match the front portion or end portion of an element's key, or any portion in between).

Now we create an ArrayList to hold the subset of selected elements, we step through the complete set of tagged elements, and we add each element to the list only if the regular expression matches the key property. We also set the selectSpec property of each tagged element so that later we'll know what key filter - in simple non-regex syntax - was used to select it.

ArrayList filteredElements=new
            ArrayList();

Regex filterRegex = new Regex(filter);

foreach (TaggedElement element in originalElements)
{
Match theMatch=null;

theMatch =
            filterRegex.Match(element.key);

if
            (theMatch.Success)
{
filteredElements.Add(element);
element.selectSpec=selectSpec;
}
}

return
            filteredElements;
}

For further reading you are invited to look at the real getSelectedElements(.) method in the WebDataEngine.cs source file.

Another good use for regular expressions, and now we have the basis of the useful GUI application shown in the screenshots. The WebFilterViewer allows us to experiment by picking out various groups of elements using key filters (or selectSpecs, if you prefer).

Challenge 4: Flattening the Tag Hierarchy to form a Relational Table

We now have the ability to select a set of elements from a web page into a list, or - to put in another way - into a column. The next challenge was to select several columns of data into the rows of a table.

At first glance this is quite straightforward. We could easily devise key filters to pick out the Author and ISBN details from the Wrox page, in addition to the Title elements just demonstrated. And because all of those elements are contained within a regular HTML table, the three lists (or columns) will be identical in length so easy to combine. To remind you, here's what the source page looks like:

But not all tables are regular, and it's even possible to have tables within tables as you can see in this simple page.

The WebFilterViewer (shown next) shows how the elements are arranged in the HTML hierarchy, and it allows us to formulate the selectSpecs to pick out data in each column.

Use selectSpec .table[0].tr[*].td[0].text[0] for author elements...

T. Loton

A. Nother

Use selectSpec .table[0].tr[*].td[1].table[0].tr[*].td[0].text[0] for title elements...

Professional Java Data

Professional Java Servlets

Professional Java Things

Use selectSpec .table[0].tr[*].td[1].table[0].tr[*].td[1].text[0] for ISBN elements...

ISBN 1234567

ISBN 1234568

ISBN 1234569

Selecting author elements would return two entries, whereas selecting title or ISBN elements would each return three entries. In such cases we need to flatten the hierarchy, repeating elements as necessary to give a relational table. like:

T. Loton | Professional Java Data | ISBN 1234567
T. Loton | Professional Java Servlets | ISBN 1234568
A. Nother | Professional Java Things | ISBN 1234569

To show how we solve the flattening problem, take a look at the following table in which I've listed the elements selected into the first (author) column combined in every possible combination with the elements selected into the second (title) column. For simplicity I've left out the final (ISBN) column.

Author selected with .table[0].tr[*].td[0].text[0] gives | Title selected with .table[0].tr[*].td[1].table[0].tr[*].td[0].text[0] gives
T. Loton with key .table[0].tr[0].td[0].text[0] | Professional Java Data with key .table[0].tr[0].td[1].table[0].tr[0].td[0].text[0]
T. Loton with key .table[0].tr[0].td[0].text[0] | Professional Java Servlets with key .table[0].tr[0].td[1].table[0].tr[1].td[0].text[0]
T. Loton with key .table[0].tr[0].td[0].text[0] | Professional Java Things with key .table[0].tr[1].td[1].table[0].tr[0].td[0].text[0]
A. Nother with key .table[0].tr[1].td[0].text[0] | Professional Java Data with key .table[0].tr[0].td[1].table[0].tr[0].td[0].text[0]
A. Nother with key .table[0].tr[1].td[0].text[0] | Professional Java Servlets with key .table[0].tr[0].td[1].table[0].tr[1].td[0].text[0]
A. Nother with key .table[0].tr[1].td[0].text[0] | Professional Java Things with key .table[0].tr[1].td[1].table[0].tr[0].td[0].text[0]

Clearly not all of those candidate rows make sense, for example "A. Nother" combined with "Professional Java Data", so the next task is to take out the ones that don't make sense. In the following table I've replaced any concrete indexes (between square braces) in the elements keys with * characters if those indexes were fixed in the original selectSpec. Conversely, I've retained any indexes in the element keys if there was a corresponding wildcard * character in the selectSpec.

Author selected with .table[0].tr[*].td[0].text[0] gives | Title selected with .table[0].tr[*].td[1].table[0].tr[*].td[0].text[0] gives
T. Loton with key .table[*].tr[0].td[*] | Professional Java Data with key .table[*].tr[0].td[*] .table[*].tr[0].td[*]
T. Loton with key .table[*].tr[0].td[*] | Professional Java Servlets with key .table[*].tr[0].td[*] .table[*].tr[1].td[*]
T. Loton with key .table[*].tr[0].td[*] | Professional Java Things with key .table[*].tr[1].td[*] .table[*].tr[0].td[*]
A. Nother with key .table[*].tr[1].td[*] | Professional Java Data with key .table[*].tr[0].td[*] .table[*].tr[0].td[*]
A. Nother with key .table[*].tr[1].td[*] | Professional Java Servlets with key .table[*].tr[0].td[*] .table[*].tr[1].td[*]
A. Nother with key .table[*].tr[1].td[*] | Professional Java Things with key .table[*].tr[1].td[*] .table[*].tr[0].td[*]

You will see that, up to a point, the keys on the left now match the keys on the right. Except that some of them don't because the indexes do not match up as I've shown in bold. Removing those non-matches leaves the following table, which is exactly what we want.

T. Loton with key .table[*].tr[0].td[*] | Professional Java Data with key .table[*].tr[0].td[*] .table[*].tr[0].td[*]
T. Loton with key .table[*].tr[0].td[*] | Professional Java Servlets with key .table[*].tr[0].td[*] .table[*].tr[1].td[*]
A. Nother with key .table[*].tr[1].td[*] | Professional Java Things with key .table[*].tr[1].td[*] .table[*].tr[0].td[*]

Retained elements are said to be in the same context, the context being defined by the modified versions of the keys that we compared.

I implemented that flattening algorithm in code and extended it to handle any number of selected columns. The code's not too pretty but essentially all it does is.

  • Form a cross-product of every element selected into column 1, combined with every element selected into column 2, combined with every element selected into column 3. and so on.
  • Check that the context of every element in each row is consistent with the context of every other element in the same row.

Determining the context of each selected element depends on knowing the selectSpec that was used to select the element in the first place. That's why the TaggedElement class had a selectSpec property that was set in Challenge 3.

Returning rows as a .NET DataTable

As we're building a relational table, it made sense to deliver the end result as a .NET System.Data.DataTable instance, thus making the data extracted from web pages accessible in the same way as any other .NET data source, and - as you'll see later - to facilitate querying of the web data using conventional Structured Query Language.

The following getSelectedRows(.) method takes in a list of names, one for each column, which will ultimately be derived from the column aliases specified in a SQL statement. The separate ArrayLists of elements for each column, as returned by the getSelectedElements(.) method in Challenge 3, become items of a containing ArrayList passed in as a second parameter named disjointColumns.

public static DataTable getSelectedRows(ArrayList columnNames, ArrayList disjointColumns)
{

Just in case the content of the second parameter is not clear, I'll tell you that each entry in the disjointColumns list will itself be an ArrayList of elements selected with one of the selectSpecs. So in our example, disjointColumns[0] would be an ArrayList containing ["T. Loton", "A. Nother"], and disjointColumns[1] would be an ArrayList containing ["Professional Java Data", "Professional Java Servlets", "Professional Java Things"].

Now we create a .NET DataTable to hold the final results and added several column definitions to it:

int
            columnCount=disjointColumns.Count;

DataTable finalTable=new
            DataTable();

foreach (String columnName in
            columnNames)
{
DataColumn newColumn = new DataColumn();
newColumn.DataType =
            System.Type.GetType("WebDataToolKit.TaggedElement");
newColumn.AllowDBNull = true;
newColumn.Caption =
            ""+columnName;
newColumn.ColumnName =
            ""+columnName;
finalTable.Columns.Add(newColumn);
}

We then run through the algorithm described earlier to combine column elements in all combinations, omitting the combinations that don't make sense.

For further reading you are invited to look at the real getSelectedRows(.) method in the WebDataEngine.cs source file.

If you're not inspired to look through the source code for that complex algorithm - and I don't blame you - you might nonetheless be interested to know that the general procedure for adding a (valid) row to the DataTable is.

DataRow newRow = finalTable.NewRow();
newRow[0] = someData
newRow[1] = someMoreData
finalTable.Rows.Add(newRow);

Setting the column DataTypes

You will have noticed that initially we set the type of each column to be our own class TaggedElement. We're taking input as ArrayList s of TaggedElement s and we need the elements in that form throughout the algorithm so that we have access to each element's key, content and selectSpec. As a final step, we copy the whole table to a new table with more appropriate data types.

HTML is a text markup language, thus every element will be extracted as a text string like "T. Loton", "13.99", or "01-01-2002" and the default type for each column would be string. Though there are no explicit data types in HTML, we can infer that the first of those really is a text string, the second is a float or double, and the third is a date. Thus when we form the final table, we set each column's data type to:

  • System.DateTime if all entries in the column represent a date
  • System.Double if all entries in the column represent a real (floating point) number
  • String otherwise

We determine the implicit type of each element using a series of try.catch blocks as follows:

bool isDate=false;
bool
            isDouble=false;

DateTime dateValue=new DateTime();
Double
            doubleValue=Double.NaN;

try
{
dateValue=Convert.ToDateTime(thisContent);
isDate=true;
}
catch(Exception
            ex){}

try
{
doubleValue=Convert.ToDouble(thisContent);
isDouble=true;
}
catch(Exception
            ex){}

At this point we'll know whether the element is isDate or isDouble (or else default String). If all the other elements in the same column come out the same, we reset the type of that column to that type. The end result is a representation of the original web data in the form of a DataTable with appropriately typed columns, indistinguishable from any other DataTable of whatever source.

You might be concerned about the performance implications of the code just shown, and you'd be right, but the benefit in doing this will become clear when we consider SQL in the next challenge. Look out for the "Data Types Revisited" section!

Uses for the DataTable representation

As with any other DataTable, we can bind it to a System.Windows.Forms.DataGrid form element with code like this:

System.Windows.Forms.DataGrid sqlDataGrid
= new System.Windows.Forms.DataGrid();
sqlDataGrid.DataSource=ourDataTable;

So very little effort is required to visualize the data thus:

For further reading you are invited to look at the SqlResultsForm.cs source file.

Furthermore, it would be possible to construct a .NET DataSet by combining one or more DataTables originating from different web sources. Thus we really could treat the World Wide Web as a huge database in which the individual web pages are the tables. What makes the .NET DataSet particularly interesting is the fact that we could convert easily to an XML representation with a full in-line schema, using this single line of code:

string xmlWithSchema = ourDataSet.GetXmlSchema();

For those XML gurus amongst you, how about using that representation for XPATH queries or for transformations using XSLT?

Challenge 5: Supporting Structured Query Language

It's no accident that the previous screenshot was titled "SQL Results" as that's what we'll consider next. Remember what I said in the introduction?

Think about how useful it would be to have a toolkit that allows you to SELECT nuggets of information FROM live web pages as though selecting from tables in a relational database.using industry-standard SQL.

At this point in the development, I was well on the way to achieving that aim because we have the ability to select elements of a web page into columns of data, to be combined with other columns of data in a complete relational table. In a nutshell we now have the ability to run an SQL statement of the form:

SELECT .table[0].tr[*].td[0].text[0] AS author
, .table[0].tr[*].td[1].table[0].tr[*].td[0].text[0] AS title
, .table[0].tr[*].td[1].table[0].tr[*].td[1].text[0] AS isbn
FROM file://C:/XML/WroxEmbedTables.html

Each part of the SELECT clause comprises a key filter -- called a selectSpec in code, remember? -- to pass to the getSelectedElements(.) method, along with a column alias to pass through to the getSelectedRows(.) method. The FROM clause has a URL to pass into the getTextFromURL(.) method. Of course we'd have to parse the text of the SQL statement to pick out the various parts, and you won't be surprised to hear that for that job I once again used regular expressions!

To support a fuller SQL syntax we need to allow statements that include WHERE and ORDER BY clauses. Here is such a statement that refers to the Wrox New Releases web page:

SELECT .table[1].tr[0].td[1].table[1].tr[1].td[0].table[0].tr[*].td[0].a[0].text[0] AS title
, .table[1].tr[0].td[1].table[1].tr[1].td[0].table[0].tr[*].td[1].text[0] AS author
, .table[1].tr[0].td[1].table[1].tr[1].td[0].table[0].tr[*].td[2].text[0] AS isbn
FROM file://C:/XML/WroxNewReleases.htm
WHERE title LIKE 'Professional*'
ORDER BY isbn ASC

With that functionality in place we'll have completed the toolkit, so here's how I did it.

Supporting WHERE and ORDER BY

In the original Java implementation, I implemented from scratch the functionality to parse out WHERE and ORDER BY clauses, to compare each row against the WHERE clause, and to sort according to the ORDER BY clause. Not as simple as it sounds because, typically, WHERE clauses support AND / OR / NOT operators and ORDER BY clauses allow sorting by any combination of columns in ascending or descending order.

The .NET framework provides a very useful shortcut here thanks to the fact that we've already assembled our results in a DataTable. This class has a Select(.) method that allows a SQL WHERE clause to be passed straight in, along with an ORDER BY clause. So for our example, if we set a couple of variables as follows.

String whereClause = "title LIKE
            'Professional*'";
String whereClause = "isbn ASC";

We can then call the following method to get a new version of our DataTable, filtered and sorted as appropriate:

public static DataTable
            filterAndSort(DataTable results, String whereClause, String
            orderClause)
{
DataTable
            newResults=results.Clone();
newResults.Clear();

DataRow[]
            rows=results.Select(whereClause,orderClause);
foreach (DataRow
            row in rows)
{
DataRow
            newRow=newResults.NewRow();
newRow.ItemArray=row.ItemArray;

newResults.Rows.Add(newRow);
}

return
            newResults;
}

That shows off a really neat feature of the .NET DataTable, and it saves on a lot of code compared with the Java implementation, especially when you consider that we could be dealing with a SQL statement ending thus:

WHERE (title LIKE 'Professional*' OR author LIKE 'Matthew*') AND isbn >='1861005'
ORDER BY author ASC, title DESC

No matter how complicated our WHERE and ORDER BY clauses become, it's all handled effortlessly by the DataTable itself.

Data Types Revisited

Remember that when building the DataTable in the first place we took care to set the data types of the columns to more appropriate types than the default String type? That was to make the results of WHERE and ORDER BY clauses more intuitive.

When comparing values of - for example - a "percentage change" column, we'd like a comparison 20 > 100 to evaluate to false, as it will if we set the column a numeric type like System.Double. If we left the column type as System.String the same expression would evaluate to true as a consequence of a lexical comparison. A similar situation exists with dates, which would compare differently as strings than as true dates.

For further reading you are invited to look at the real filterAndSort(.) method in the SqlEngine.cs source file.

That completes the discussion of the design and implementation decisions that were taken, and we can now take the toolkit for a test run!

Proof of the Pudding

I'll give you some software to try out, so that you can see how these ideas really do hang together. I've provided an early version of part of the WebDataKit for .NET beta release that was compiled and tested using the .NET Framework final release.

Compiling the Source Code

As stated earlier, I've supplied - for reference only -- the complete source code as it stands for this case study. I've also included saved versions of some sample HTML and XML pages used in the previous discussion.

You can compile the code using this command:

csc WebFilterViewer.cs SqlResultsForm.cs WebDataEngine.cs SqlEngine.cs TagParser.cs

An executable file WebFilterViewer.exe will be produced as output, and don't worry if during the compilation you get a few warnings of the kind:

WebDataEngine.cs(219,9): warning CS0168: The variable 'ex' is declared but never used

Example use of the WebFilterViewer

In the following screenshot you will see that I've launched the WebFilterViewer, now enhanced with a text field to take an SQL statement. I've entered a URL, in this case file-based, into the URL textbox and I've submitted the HTTP request by pressing the Submit button.

This ListBox directly below the Submit button lists each HTML element of the page in the form key = content, and the element keys can be used to build a SQL statement as shown in the lower text box. For convenience, if you click an item in the list its full key is appended to the text of the SQL statement. Saves a lot of typing, believe me!

In that example, I've picked out a line referring to a book title and I've modified its key to include wildcard *s that pick out all similar items -- i.e. all book titles - as my first selectSpec. For the second selectSpec, I'm picking out author names from a higher level of the HTML hierarchy, again with a strategically placed wildcard *. Thanks to the flattening technique described in Challenge 4, each book title will be combined with the correct author as you'll see next.

Upon pressing the Run SQL button the following form - with an embedded DataGrid - is displayed.

With that example showing how straightforward it is to use the toolkit, you are invited to try it out for yourself using any http:// or file:// based HTML or XML file as a data source. When you've experimented with a few HTML files you could try it out with an XML file like the "Two Gentlemen of Verona" sample file. Here's an example.

The SQL Statement.

The Results..

Thanks to the simple TagParser, XML is supported solely because it has text embedded between <starttag> and </endtag> combinations. This means that no DTD, Schema, or other supporting information is needed and all we need is the .xml file itself.

Case Study Review

Now let's look back over the case study to review the design and implementation issues that have been discussed in relation to the .NET Web Data Toolkit. At the outset I told you that we would look at how to:

  • Extract data from live web pages or files on disk.
  • Make individual HTML and XML elements uniquely addressable.
  • Select elements using wildcards and regular expressions.
  • Flatten a hierarchic HTML or XML structure to a flat relational structure.
  • Parse SQL statements and use SQL to query web pages a though they were relational database tables.

Those issues were presented as a series of challenges to be addressed. I'll list those challenges again and remind you of the solution for each one.

Challenge 1: Retrieving Web Data via HTTP Submissions

The ability to fetch source data from web sites by simulating HTTP GET or POST submissions was the foundation on which everything else was built. We met that challenge with the help of the .NET System.Net.WebRequest and System.Net.WebResponse classes, and I listed a getTextFromURL(.) method that you could use as the basis of any application that needs to fetch web data.

Challenge 2: Parsing HTML and / or XML Elements

The next challenge was to parse the resulting page to make each HTML, XML, or in fact any <starttag> </endtag> delimited element, uniquely addressable via a key value. For that task I used a custom-built TagParser class that will work with any kind of tagged input, but which has a limited knowledge of HTML tags. The use of regular expressions was demonstrated as a mechanism for picking out attribute name / value pairs from within a tag, and the end result of the whole process - as implemented by a getElements(.) method - was a series of TaggedElement instances having key and content properties of the form:

KEY | CONTENT
.table[1].tr[0].td[1].table[1].tr[1].td[0].table[0].tr[2].@bgColor | White

Challenge 3: Selecting Elements

Having a set of individually addressable elements comprising the whole page, we next wanted to pick out sets of elements with similar keys; for example to pick out all book titles with keys matching the pattern .table[1].tr[0].td[1].table[1].tr[1].td[0].table[0].tr[*].td[0].a[0].text[0]. For that pattern-matching task we once again used regular expressions, having first converted the simpler pattern syntax to the full regular expression syntax thus:

\.table\[1\]\.tr\[0\]\.td\[1\]\.table\[1\]\.tr\[1\]\.td\[0\]\.table\[0\]\.tr\[\d+\]\.td\[0\]\.a\[0\]\.text\[0\]

In preparation for SQL support we termed the pattern a selectSpec and passed this in to a getSelectedElements(.) method along with the full list of tagged elements. That method returned a subset of elements that matched the selectSpec.

Challenge 4: Flattening the Tag Hierarchy to form a Relational Table

Combining results from separate selectSpec s into a single relational table would allow us to treat the web data in the same way as any other data source, and present it as such, so that was our next challenge. We adopted a flattening technique to ensure that sets of selected elements were combined into table rows in sensible combinations, even in cases where the source data was not presented originally in the form of a regular table.

We returned the results of the getSelectedRows(.) method as a .NET System.Data.DataTable that could be displayed directly in a System.Windows.Forms.DataGrid.

Challenge 5: Supporting Structured Query Language

The completion of Challenge 4 provided support for SQL statements to SELECT tagged elements FROM web pages URLs as though selecting column data from traditional relational tables, with the results being returned in the form of a relational table. To support SQL syntax more fully we needed to handle WHERE clauses and ORDER BY clauses, which we did by taking advantage of the DataTable.Select(.) method that we wrapped up in our own filterAndSort(.) method.

Limitations and Future Enhancements

As described in this case study, the key challenges have been met and the fundamental pieces are in place for the beta version of the .NET WebDataKit. However, it's not yet industrial-strength and has a few shortcomings compared with the Java implementation. Here I'll describe some of the limitations of the case study software and some of the work yet to be done for the beta release.

Parsing non-well-formed HTML and XML

I'd been using a saved version of the Wrox New Releases web page for some time as a basis for my testing, and when I returned to the live online version I noticed some odd behavior that manifested itself in element keys of the form:

.table[3].tr[0].td[1].table[0].tr[1].td[0].table[0].tr[0].td[0].table[0].tr[2].td[0].table[0].tr[0].td[0].
table[0].tr[1].tr[0].tr[0].td[9].text[0]

I've highlighted the offending portion, showing that what's wrong is the repeated occurrence of tr[] entries in the key. That should not be possible with a well-formed HTML file and indeed the problem lay in the fact that certain portions of the page were not well-formed to the extent that some of the <tr> tags were not closed off by a corresponding </tr> tag as you can see in this HTML fragment taken from the page.

<tr class="conBgColorLightGrey">
<td class="conHXXSBlack"><a class="conHXXSBlack"
href="ACON11.asp&ISBN=1861006152">Beginning ASP.NET using C#</span>&nbsp;</td>
<td class="conHXXSBlack">David Sussman et al&nbsp;</td>
<td><span class="conHXXSBlack" style="text-decoration:none;">1861006152</span></td>
<td><a class="conCodeDownloadLink"
href="ACON12.asp&ISBN=1861006152">Download</a></td>
<tr>
<td class="conHXXSBlack"><a class="conHXXSBlack"
href="ACON11.asp&ISBN=1861005555">Beginning Visual Basic .NET
Databases</span>&nbsp;</td>
<td class="conHXXSBlack">Matthew Reynolds et al&nbsp;</td>
<td><span class="conHXXSBlack" style="text-decoration:none;">1861005555</span></td>
<td><a class="conCodeDownloadLink"
href="ACON12.asp&ISBN=1861005555">Download</a></td>

Mid way through that HTML source a <tr> tag follows directly from a </td> tag where as we would expect to see a </tr> tag in between. Thus the TagParser was fooled into thinking that a new row had started before the previous row had ended.

Clearly we need to give the home-grown TagParser the intelligence to cope with not-so-well-formed HTML, perhaps by inserting a phantom </tr> into the text where we would normally expect to find one. For the record, the original Java implementation was unaffected by such problems because the parsing was performed by the HTML parser provided by the Java Runtime itself, which was designed to make allowances for sloppy HTML.

Impact of web site changes

The previous limitation came to light because the format of the Wrox web page had changed, and you might already be thinking about how this approach is affected by web site changes. In my experience of working with these tools, web pages are typically good for about six months before they change in format and hence break the SQL. A reworking every six months might be fine for your application, but what if you distribute applications based on this technology to other people? Well that's the whole point of using SQL as it allows you to store the page-format-dependent queries separately from the code itself, possibly in a text file. Whenever a page changes, you re-write the SQL to return the same number of columns, then simply distribute the new statement as a "patch" with no new code and no recompilation required.

Selections from multiple pages

This implementation is limited to the selection of data from one page at a time. In principle, it's possible to select data from multiple pages as you would select from multiple tables, using SQL joins. Indeed that's what the Java implementation allows through SQL statements of the form:

SELECT allbooks.table[1].tr[0].td[1].table[1].tr[1].td[0].table[0].tr[*]
.td[0].a[0].text[0] AS allbooks.title
, allbooks.table[1].tr[0].td[1].table[1].tr[1].td[0].table[0].tr[*].td[1]
.text[0] AS author
, topten.table[1].tr[0].td[2].table[0].tr[8].td[0].table[0].tr[*].td[1]
.a[0].text[0] AS topten.title

FROM http://www.wrox.com/Books/Books.asp?section=11_2 AS allbooks
, http://www.wrox.com/ AS topten

WHERE allbooks.title=topten.title

In that statement you can see how multiple pages may be specified in the FROM clause, with each page being aliased to distinguish the elements in the SELECT clause. That functionality will be added into the beta release to bring it in line with the Java implementation.

If you'd like to think about it yourself, I can tell you that elements from the first page should be combined with elements from the second page in a full cross-product as though they automatically belong in the same context. The sensible combinations would then be retained solely as a result of the WHERE clause that equates a value in the first table with a corresponding value in the second table.akin to a primary key / foreign key relationship. That's exactly how a traditional relational join works, and it depends on there being some values in common between the two pages to be joined.

Stateful Pages and Authentication

In this case study we've dealt with selecting data from a static page or from a dynamic page resulting from a single HTTP GET or POST submission. In theory, the idea could be extended to a solution for driving any online application - comprising multiple pages -- automatically. I'm thinking here of writing a program (if you dare) to drive the internet service provided by your bank.

For any such application, we'd need to submit multiple HTTP requests to step through the various pages, taking care that the state was maintained as required by the application through cookies, URL re-writing, or some other technique. We might need also to pass through an authentication stage before gaining access to the application at all. Something for you to think about if you'd like to take this to its logical conclusion, but I'll admit to not being able to help you out just yet.

What I can tell you is that as far as SSL goes, it's possible to submit requests to https:// data sources with no additional code whatsoever, which was not the case with Java until the arrival of the JDK 1.4. Try it out by typing the following URL into the WebFilterViewer: https://www.verisign.com/?WROXEMPTOKEN=620568Zhiox25QOrrsnTPfm53p

Potential Applications

As I've taken a single example throughout, I might have left you with the impression that this toolkit is only useful for obtaining a list of books from the Wrox web site. So let me give you some more food for thought.

The majority of developers using the original Java WebDataKit are working with financial web sites such as Yahoo!Finance at http://finance.yahoo.com/?WROXEMPTOKEN=620568Zhiox25QOrrsnTPfm53p and FT MarketWatch at http://mwprices.ft.com/custom/ft-com/quotechartnews.asp?WROXEMPTOKEN=620568Zhiox25QOrrsnTPfm53p, to extract and interpret up-to-the-minute share price information. In fact, I developed the toolkit originally to automate aspects of my amateur investing hobby. Subsequently I've used the software for sending SMS messages to mobile phones via a web site that provides such a service, and for querying of on-line timetables as you can see from this screenshot of the original Java tool.

Notice how in that SQL statement I could change the posted parameters (highlighted) to capture data from a journey between any two UK stations on any date.

What each of those applications has in common is the need to send some parameter data to a web site using the GET or POST method, then to capture the response - a simple SMS acknowledgement, complete share price information, or a set of train times as HTML or XML -- for further processing or local storage. For some of you the phrase screen scraping might spring to mind, and as such you can think of any web site as a service provider and this toolkit as a mechanism for automating the use of those online services. no SOAP required.

One final thought is that you might like to think about gathering up-to-the-minute information from a range of web sites to be combined into a single cockpit application, or even your own online portal.

Case Study Information
Author | Tony Loton
Technical Editor | Adam Ryland
Project Manager | Helen Cuthill
Reviewers | Andrew Krowczyk, Saurabh Nandu, Adam Ryland

If you have any questions or comments about this case study, or suggestions for further pieces, please contact the technical editor. If you wish to talk about the case study with other subscribers, please visit our forum here.


An In-Depth Look at the NumberFormatInfo Class

An In-Depth Look at the NumberFormatInfo Class
http://www.csharptoday.com/content.asp?id=1754
Article Rating: not yet rated
Published on March 4, 2002
Abstract
NumberFormatInfo (NFI) is the class in System.Globalization that provides culture-sensitive formatting for currency, numbers, and percentages. Next to confusing short dates (is 11-08-90 the 8th of November or the 11th of August?), numeric values displayed without regard to user expectations can cause annoyance and misinterpretation. What does 2,103 really mean? Is it two thousand one hundred and three, or two and one hundred and three thousandths? In financial or medical software the answer can be crucial. In this article, Bill Hall explores the methods and properties of NFI to discover how it can help us to create displays that meet user requirements for a given world region.
Article

NumberFormatInfo (NFI) is the class in System.Globalization that provides culture-sensitive formatting for currency, numbers, and percentages. Next to confusing short dates (is 11-08-90 the 8th of November or the 11th of August?), numeric values displayed without regard to user expectations can cause annoyance and misinterpretation. What does 2,103 really mean? Is it two thousand one hundred and three, or two and one hundred and three thousandths? In financial or medical software the answer can be crucial. In this article, we will explore the methods and properties of NFI to discover how it can help us to create displays that meet user requirements for a given world region. Assisting us is a form written in C# that shows most of the properties and methods of the class. The program is interactive; you can change specific properties, including separators, grouping, signs, and precision, and view the results in any .NET-supported culture.

The implementation of NFI has its roots in two Win32 functions, GetNumberFormat and GetCurrencyFormat. Although useful and versatile, these two API's are not particularly convenient to apply, especially when you want to change a parameter from the standard locale offering. For example, users like to see an integer with many digits with group separators, as its value is less likely to be interpreted incorrectly. The number 210398398750 is hard to read, but 210,398,398,750 (or if you prefer 201.398.398.750) is much easier to sort out visually. In Win32, a straightforward application of GetNumberFormat groups the whole part correctly according to the locale setting, but always tacks on a fractional component as well. To eliminate the fractional part, the programmer has to fill out six fields in a NUMBERFMT structure by repeated calls to GetLocaleInfo using LCTYPES that correspond to the NUMBERFMT fields, and then set the fractional digits to zero. Using NFI, a simple property setting, NumberDecimalDigits = 0, does the trick.

An instance of NFI is a property of a specific culture instance of System.Globalization.CultureInfo. A specific culture is determined by both language and region. A parent culture, which references language but not region, contains only a null value for its NFI. The reason is that the features of NFI are dependent on a particular world region as well as language. For example, here are two rather different number formats from two countries bordering on each other and sharing a common language:

French (Switzerland) | 123'456'789'876'543'210.99
French (France) | 123 456 789 876 543 210,99

Before reading this article you may find it helpful to review our previous one about .NET's CultureInfo class in Internationalization With The .NET CultureInfo Class & C#. You may also want to look at An In-Depth Look at the DateTimeFormatInfo Class, as the two classes have similar uses.

Simple Number Formatting

It is easy to control the format of a number. Just use the variable's ToString method with one of the eight formatting letters given in Table 1. Not all data types will work with all formatting strings; for example, you cannot use the hex formatter with a decimal type. If in doubt, use a try-catch block.

The typical code is val.ToString("<formatstring>"); where val is a numeric data type such as double, integer, decimal, float, etc. Unfortunately, not all of the possible format strings are completely documented in the .NET 1.0 Help. For example, the percentage formatters "p" and "P" are missing. Also, there is no explanation of why lower and upper case forms are provided, although apparently the reason is to case certain letters in the output (see, for example, the scientific and hexadecimal results) in Table 1.

Type | Format string | Output (lower case) | Output (upper case)
Currency | C, c | $123,456,789.99 | $123,456,789.99
Integer | D, d | 123456789 | 123456789
Scientific | E, e | 1.234568e+008 | 1.234568E+008
Fixed point | F, f | 123456789.99 | 123456789.99
General | G, g | 123456789.987654 | 123456789.987654
Number | N, n | 123,456,789.99 | 123,456,789.99
Round Trip | R, r | 123456789.98765433 | 123456789.98765433
Percent | P, p | 98.77 % | 98.77 %
Hexadecimal | X, x | 75bcd15 | 75BCD15

Table 1: Formatting Characters for Numbers

Of these, only Currency, Number, and Percentage are controlled by NFI. For example, in the table above, the thread culture just happened to be en-US when the results were recorded. However, we can create a new culture, say Swiss (French), and then augment the formatting by referencing the CultureInfo's implicit NFI as follows:

CultureInfo ci = CultureInfo.CreateSpecificCulture(de-CH);
String result = Val.ToSTring("<character>", ci);

In this case, three of the outputs above now appear as shown in Table 2; the rest remain the same:

Currency | SFr. 123'456'789.99 | SFr. 123'456'789.99
Number | 123'456'789.99 | 123'456'789.99
Percent | 98.77% | 98.77%

Table 2: Swiss-French Number Formats

So, most of the time, you don't have to explicitly reference an NFI instance if the current thread's CultureInfo matches the user's requirements since CultureInfo itself contains an instance of NFI. However, if you want to obtain information unique to NFI or modify the output, then you will have to obtain one explicitly. This is very easy to do with the line of code NumberFormatInfo nfi = ci.NumberFormat; where ci is a specific (not parent) CultureInfo object. Now let's take a look at the sample application that accompanies this article to see what is provided.

NumberFormatInfo Details

Figure 1 shows the demo program with the culture set to Russian (Russia). The form itself is divided into three sections, reflecting the partition of properties and methods for Currency, Number, and Percentage. Within each section the properties themselves are shown. Also displayed are the resulting formats for number, currency, and percentage. At the bottom are some additional pieces of information about how to correctly render "not a number" and negative and positive infinity in the selected culture.

You should be aware that although the formats tend to be somewhat similar for currency, number, and percent, each should be regarded as a separate formatting task. Often, programmers tend to think of currency formatting as a special kind of number, but that is not the case. You cannot just drop in a currency symbol in front of a number expression and expect to have a culturally correct result. Rounding rules can be very different between currencies and numbers. Numbers can have fractional parts but the currency may have none (Japanese Yen, Indonesian Rupees). Placement of the currency symbol in relationship to the number is another issue -front or back - spaced or not - from the actual value. The problem becomes even more acute when working with negative values. Not all cultures use a minus sign, but if present, positioning becomes even more complicated when considering relative order of symbol, sign, and value. If you run through the listed cultures you can find sixteen ways of arranging debit currency and five ways for showing negative numbers. Even countries using the Euro have their own individual styles: sometimes the Euro symbol is in front (Italy, Austria) and sometimes behind the value (Portugal, Greece, Luxembourg, Germany, Spain, Finland, France).

Another interesting problem is how to separate whole part grouping of numbers. You may have noticed the row of nine NumericUpDown controls in the currency and number subareas (we could have also done the same for percentage as well). Many Indian cultures actually separate whole parts not simply by three's but twos and threes such as this example from Hindi -1,23,45,67,89,87,65,43,210.99. In NFI, you can actually create up to 9 possible group sizes and in any order. In the demo program you can experiment with the various possibilities. This feature is probably overkill, both in the class implementation in .NET as well as in the demo program, but it was probably just as easy to implement the general case as it was to meet the specific requirements of the Indian locales. It does, however, make writing the demo program harder!

Figure 1: NumberFormatInfo Demo

By the way, you may notice that for some cultures, the text may not look quite right, especially for the fields at the bottom of the form. In this case, you might want to add additional language support. You can do this using the Regional Settings dialog and your Windows installation CD. Just click on the additional language settings you need and then insert the CD when requested.

NumberFormatInfo Properties

The main properties that you can change are described next. Note they are different properties for each number, currency, and percent and are not shared. For example, you will find separate properties for the number of decimal digits used in formatting currency, number, and percentage, namely CurrencyDecimalDigits, NumberDecimalDigits, and PercentDecimalDigits, respectively. For simplicity in the description below, we have grouped them together by category and used a descriptive name for each. Not all are present for each number type.

Decimal Separator | The string separating whole and fractional parts
Group Separator | The string separating groups of whole part digits
Decimal Digits | An integer representing the number of fractional digits (0-99)
Currency Symbol | The string representing the currency symbol
Positive Sign | A string that denotes that a number is positive
Negative Sign | A string that denotes that a number is negative
Percent Symbol | A string that denotes a percentage value
Per Mille Symbol | A string that denotes a per mille (thousandth) value
Positive Pattern | A digit representing positioning of symbol, value, and spacing. Ranges are (0-3) for currencies and (0-2) for percentages. For example, the possible orderings are for credit currency are $n, n$, $ n, and n $, where $ represents the monetary string.
Negative Pattern | A digit representing possible positioning of symbol, sign, value, and spacing. Ranges are (0-15) for currencies, (0-4) for numbers, and (0-2) for percentages. For example, negative percentages could have the forms -98.77 %, -98.77%, or -%98.77.
Group Sizes | An array of 9 values, each ranging from 1 to 9, that describes how many elements to group for each whole part segment. The last element can be zero to indicate repetition of the next to last value.

Changing the Default Formats

NumberFormatInfo gives you a set of useful defaults formats. By using the properties of a suitable NFI you can modify the output. There may be many reasons for doing this. Let's give an example. Many European countries now use the Euro, but historical data might have to be rendered using a pre-Euro format. Here is an example from Portugal using the Euro.

Adjusting the decimal separator to ($) and the currency symbol to Escudos provides the older format:

We have already mentioned the example of providing a readable format to an integer just by adjusting the number of decimal digits. In this example, we have adjusted the number of decimal digits in a number format to none.

If we want to go wild with group sizes, try this one. Just for fun, we also changed the negative pattern so that the negation sign is at the back and spaced away from the value. We could have also changed the separators and signs. These examples should give you a good idea of what NFI can be made to do for you.

By the way, after you have completely messed us the settings in the demo you can reset them to the culture's defaults by using the Reset button!

Code Samples

There is absolutely nothing difficult about working with NFI. As for the demo form itself, the only complicated part is managing the Group Sizes display, and that is not particularly interesting, just tedious. The details of generating a specific CultureInfo list have already been discussed in An In-Depth Look at the DateTimeFormatInfo Class.

Here are some details about the test values used in the program and the code for rendering the currency, number, and monetary displays. Notice that we alternated use of upper and lower case formatting characters, but found no differences in results for the currently supported cultures. We also used the NFI explicitly in the ToString methods, i. E. curci.NumberFormat, but the code works the same if just curci is used. Here, curci is just a writeable copy of the currently selected specific culture.

private CultureInfo curci;
private
            decimal d = 123456789876543210.987654321M;
private decimal n =
            -123456789876543210.987654321M;
private double p =
            1234.987654321;
private double pn =
            -1234.987654321;

private void ShowCurrency()
            {
txtCurrency.Text = d.ToString("c",
            curci.NumberFormat);
txtNegativeCurrency.Text = n.ToString("C",
            curci.NumberFormat);
}

private void ShowNumber()
            {
txtNumber.Text = d.ToString("n",
            curci.NumberFormat);
txtNegativeNumber.Text = n.ToString("N",
            curci.NumberFormat);
}

private void ShowPercent()
            {
txtPercent.Text = p.ToString("p",
            curci.NumberFormat);
txtNegativePercent.Text = pn.ToString("P",
            curci.NumberFormat);
}

The rest of the code manages the events that occur when the user alters the selected culture or modifies the fields; most is as easy as setting or reading a property.

Conclusion

Respect your clients and customers preferences by making good use of the formatting (NumberFormatInfo, DateTimeFormatInfo) classes of CultureInfo. These classes will give you culturally correct results for your thread of execution almost without intervention or modification on your part. Use these classes right from the beginning of project development, and you'll be able to get to market without the wasteful and expensive step of an internationalization rewrite.

If you would like to read some background on the issues of internationalizing a product take a look at the two articles located here: http://www.mlmassociates.cc/thoughts_on_internationalization.htm?WROXEMPTOKEN=620568Zhiox25QOrrsnTPfm53p. For more information you should consult http://www.microsoft.com/globaldev?WROXEMPTOKEN=620568Zhiox25QOrrsnTPfm53p.

Article Information
Author | Bill Hall
Technical Editor | Adam Ryland
Project Manager | Helen Cuthill
Reviewers | Andrew Krowczyk, Saurabh Nandu

If you have any questions or comments about this article, please contact the technical editor.


Building a Type Viewer with System.Reflection

Building a Type Viewer with System.Reflection
http://www.csharptoday.com/content.asp?id=1765
Article Rating: 5
Published on March 3, 2002
Abstract
.NET comes with a sophisticated piece of functionality called Reflection that enables the consumer to learn more about the properties and the methods available on a given type, and also to learn what types are available in a specific assembly. In this article, we’ll take a look at how to build a type viewer tool, similar in nature (but having nowhere near the functionality of) the popular Reflector tool by Lutz Roeder.
Article

.NET comes with a sophisticated piece of functionality called Reflection that enables the consumer to learn more about the properties and the methods available on a given type, and also to learn what types are available in a specific assembly.

In this article, we'll take a look at how to build a type viewer tool, similar in nature (but having no where near the functionality of) the popular Reflector tool by Lutz Roeder. You can find this tool at this URL: http://www.aisto.com/roeder/dotnet/?WROXEMPTOKEN=620568Zhiox25QOrrsnTPfm53p

The Application

Create a new Visual C# - Windows Application project. Call it TypeViewer . Like Lutz Roeder's Reflector application, we're going to present the types and their members using a TreeView control. We'll add a region to the bottom of the form in which the nodes can display the information that they need to about the member.

To layout Form1 , add a Panel control and set its Dock property to Top . Then add a Splitter control and set its Dock property to Top as well. Finally, add another Panel control and set its Dock property to Fill . This will give you a basic, resizable form layout with a splitter.

To the topmost panel add a TreeView control. Set its Name property to treeviewTypes , and its Dock property to Fill . To the bottommost panel, add a TextBox control. Set its Name property to textNodeInfo and its Dock property to Fill . In addition, set its Multiline property to True and its ScrollBars property to Vertical . You'll see this:

When working with the TreeView, my preferred design pattern is that for each different kind of node we need to display, I like to create a new class that extends TreeNode . For example, if we have a tree that looks like this:

Assembly
Namespace
Type
Member
Member
Member
Type
Member
Member
Member

We'll create classes like these:

AssemblyNode
NamespaceNode
TypeNode
MemberNode
MemberNode
MemberNode
TypeNode
MemberNode
MemberNode
MemberNode

Each node will then be responsible for providing its child nodes and displaying data in the textNodeInfo control whenever the node is displayed.

Building "BaseTreeNode"

The System.Windows.Forms.TreeNode class supplied with the Framework has a couple of shortcomings that we can deal with by building a new class called BaseTreeNode . One of my least favorite things is that TreeNode is incapable of responding to events on the TreeView like AfterSelect and DoubleClick . Microsoft appears to be of the opinion that it's down to the person responsible for hosting the TreeView control to write the code to handle events that are more properly considered to be events of the nodes themselves. I prefer to create overload-able methods on BaseTreeNode that are called in response to the TreeView events themselves. All of our own nodes will be inherited from BaseTreeNode .

In addition, because the nodes themselves need to be able to set data on the textNodeInfo TextBox control, we also want to make Form1 available to each node. In the first instance, we'll implement OnClick and OnBeforeExpand methods on BaseTreeNode :

public class BaseTreeNode :
            System.Windows.Forms.TreeNode
{
// members...
private Form1
            _main;

public BaseTreeNode(Form1 main)
{
//
            set...
_main = main;
}

public Form1
            Main
{
get
{
return _main;
}
}

//
            OnClick...
public virtual void OnClick()
{
}

//
            OnBeforeExpand...
public virtual void
            OnBeforeExpand()
{
}
}

"AssemblyNode"

The first node that we're going to build is AssemblyNode . When we load an assembly into the type viewer, we'll create one of the nodes and add it to the tree. This node itself will be responsible for enumerating the types in the assembly.

When we build the tree, one mistake that people commonly make is to populate the entire tree whenever the top-level nodes are added. A better approach is to populate child nodes specifically when the user wants to see what's inside them. For example, here's a classic view that Windows Explorer shows us:

When we see this view, Windows hasn't populated all of the sub-nodes under things like My Network Places and the drives. It only does so if you actually drill down into the node. This design pattern is a good one because it reduces the amount of memory that the tree takes (rather than starting off with a tree that's guaranteed to be big, it grows as the user uses it) and also because the startup time is greatly reduced. On my C: drive, I have several thousand files and folders. I don't want Explorer walking all of those to build a tree unless I specifically need it to.

The TreeView control appears to be designed without this design pattern in mind (which is strange, because it's so common), so what we need to do is look at building a node that extends BaseTreeNode , that will understand how to do this demand-load functionality. For example, when we add an AssemblyNode to the tree, we won't scan the types until it is expanded.

We'll call this new node DemandLoadTreeNode .

The wrinkle is that the TreeView is smart enough not to add the little plus/minus icon to the left of the node if it doesn't have any children. When we create a DemandLoadTreeNode , we need to add a dummy node, so that the TreeView will display the plus sign. We'll remove this node and give the extending class an opportunity to add its own nodes when it is expanded.

What will happen here is that the plus sign will always appear even when the node itself shouldn't have any children. You can see this problem in Windows Explorer is you navigate to a folder on a network share. The folder will be shown as having child folders, even if it doesn't. If you try to expand this kind of node, nothing happens except for the fact the plus sign is removed from the view. For us, this means that whenever we need to add a node that we know should not have children, we should derive from BaseTreeNode , not DemandLoadTreeNode .

DemandLoadTreeNode needs to keep track of whether or not it contains a dummy node, and also needs to listen for the BeforeExpand event from the TreeView (its OnBeforeExpand method will be called whenever this event is received). If the node needs expanding, an overload-able method called LoadChildNodes will be called:

public class DemandLoadTreeNode :
            BaseTreeNode
{
// members...
private bool _loadRequired =
            true;

public DemandLoadTreeNode(Form1 main) :
            base(main)
{
// add the dummy
            node...
Nodes.Add(".");
}

public override void
            OnBeforeExpand()
{
// do we need to demand
            load?
if(_loadRequired == true)
{
// clear the dummy
            node...
Nodes.Clear();

// call the
            overload...
LoadChildNodes();

// flag...
_loadRequired
            = false;
}
}

// LoadChildNodes - do the demand
            load...
protected virtual void
            LoadChildNodes()
{
}
}

AssemblyNode , then, needs to extend DemandLoadTreeNode . As well as the required parameter containing a reference to Form1 , this class also requires a System.Reflection.Assembly instance (so make sure you import System.Reflection ), which is used both to set the text in the first instance, and also held in a private member.

public class AssemblyNode :
            DemandLoadTreeNode
{
// members...
private Assembly
            _assembly;

public AssemblyNode(Form1 main, Assembly assembly)
            : base(main)
{
// set...
_assembly = assembly;

//
            set the text...
Text = _assembly.FullName;
}

public
            Assembly Assembly
{
get
{
return
            _assembly;
}
}
}

Notice how we haven't implemented LoadChildNodes . We'll do that in a short while.

When the application is loaded, we want to load the assembly containing the compiled project code. We can do this by adding a handler for the Load event of Form1 . Double-click on the background of the form in the Designer and add this code:

private void Form1_Load(object sender,
            System.EventArgs e)
{
// add us to the
            tree...
LoadAssembly(this.GetType().Assembly);
}

public
            void LoadAssembly(Assembly assembly)
{
// create a new
            node...
AssemblyNode node = new AssemblyNode(this,
            assembly);

// add
            it...
treeviewTypes.Nodes.Add(node);
}

Before we run the project, let's add a handler for the BeforeExpand event on the tree and wire it up so that it calls the OnBeforeExpand event of the currently selected BaseTreeNode . First off, add this event handler creation call to the constructor:

public Form1()
{
//
// Required for
            Windows Form Designer
            support
//
InitializeComponent();

//
// TODO: Add
            any constructor code after InitializeComponent
//
treeviewTypes.BeforeExpand += new
            _
TreeViewCancelEventHandler(treeviewTypes_BeforeExpand);
}

Then, add this method:

protected void treeviewTypes_BeforeExpand(object sender, _TreeViewCancelEventArgs e)
{
// what node do we have?
if(e.Node is BaseTreeNode)
((BaseTreeNode)e.Node).OnBeforeExpand();
}

If you run the project, you'll notice that AssemblyNode has a plus icon:

However, if you expand the node, you'll find that no children are displayed and that the plus button disappears:

What's happening is that DemandLoadTreeNode is intercepting the OnBeforeUpdate call, is clearing out the child nodes and then offering AssemblyNode the chance to add its own nodes by overloading LoadChildNodes . However, because we haven't overridden this method, nothing is happening. Let's look at this now.

Loading Types from the Assembly

When we display the types inside the assembly, we want to group them by namespace. This means that first of all we need a new class called NamespaceNode . This node will contain types, each one represented by a TypeNode .

For simplicity, when we add NamespaceNode objects, we'll add all the children and avoid the demand load approach. That's because in order to add the namespaces, we have to walk all of the types anyway. The only thing we're saving by not adding TypeNode instances is memory, and we'll assume that once we've scanned an assembly, we'll want to see the types and so this is OK.

NamespaceNode looks like this. Notice that it extends BaseTreeNode , not DemandLoadTreeNode :

public class NamespaceNode : BaseTreeNode
{
public NamespaceNode(Form1 main, string namespaceName) : base(main)
{
// set the text...
Text = namespaceName;
}
}

TypeNode looks like this. Notice how this extends DemandLoadTreeNode and that it stores the related System.Type instance in a private member.

public class TypeNode :
            DemandLoadTreeNode
{
// members...
private Type
            _type;

public TypeNode(Form1 main, Type type) :
            base(main)
{
// store...
_type = type;

//
            text...
Text = type.Name;
}

public Type
            Type
{
get
{
return _type;
}
}
}

Once we have the assembly available in a System.Reflection.Assembly instance, we can get an array of all the types by calling the GetTypes method. This returns an array of System.Type objects. For each type, the namespace name is held in the Namespace property as a string. If we establish a hash table before, we can keep a list of which namespaces have been added and use this hash table as a lookup table so that we know which NamespaceNode instance the new TypeNode objects should be added to. Add this method to AssemblyNode :

protected override void
            LoadChildNodes()
{
// create a hashtable for the
            namespaces...
Hashtable namespaces = new Hashtable();

//
            walk the types and create a list of namespaces...
foreach(Type
            type in Assembly.GetTypes())
{
// do we have the
            namespace?
NamespaceNode nsNode =
            _
(NamespaceNode)namespaces[type.Namespace];
if(nsNode ==
            null)
{
// create a new namespace node...
nsNode = new
            NamespaceNode(Main,
            type.Namespace);
namespaces.Add(type.Namespace,
            nsNode);
Nodes.Add(nsNode);
}

// add this type to the
            node...
nsNode.Nodes.Add(new TypeNode(Main,
            type));
}
}

Run the app now and you'll be able to drill into the namespace and see the types:

Loading Members

If you expand any of the types, you'll find that because we haven't overridden LoadChildNodes , nothing is displayed within the nodes. We'll remedy this now.

There's plenty of useful information that we can display about a type, but in the first instance we'll just add the methods. The System.Type object that we have against each TypeNode object can give us a list of methods through the GetMethods method. This returns an array of MethodInfo objects that provides us with pretty much everything that we need to know about the method. This class is designed both to let us learn about the method's parameters, protection-level, and so on, but also provides a way to make a late bound invocation into the method through the Invoke method.

As you might expect, we'll create a new node called MethodNode to hold the method information. The LoadChildNodes method on TypeNode therefore looks like this:

protected override void
            LoadChildNodes()
{
// iterate the
            methods...
foreach(MethodInfo method in
            Type.GetMethods())
Nodes.Add(new MethodNode(Main,
            method));
}

The new MethodNode class needs to import both the System.Text and System.Reflection namespaces. When we display the method, we want to show all the parameters, so we use the GetParameters method of MethodInfo to return an array of ParameterInfo objects. We also use a System.Text.StringBuilder object to build the string, as this is the most efficient method for string concatenation available in the Framework:

public class MethodNode:
            BaseTreeNode
{
// members...
private MethodInfo
            _method;

public MethodNode(Form1 main, MethodInfo method) :
            base(main)
{
// set...
_method = method;

// method
            name and params...
StringBuilder builder = new
            StringBuilder(method.Name);
builder.Append("(");
bool
            firstParam = true;
foreach(ParameterInfo param in
            method.GetParameters())
{
// first?
if(firstParam ==
            true)
firstParam = false;
else
builder.Append(",
            ");
builder.Append(param.ParameterType.ToString());
builder.Append("
            ");
builder.Append(param.Name);
}
builder.Append(")");

//
            set...
Text = builder.ToString();
}
}

Run the application now and when you drill down into a type you'll see the methods against that type:

You'll notice that you'll get one MethodInfo per overload of a method. In addition, properties are shown as methods, because behind-the-scenes properties are accessed through method calls. If we want (although we're not going to here to save space), we could use the GetProperties method of System.Type to return the properties and remove the matching get_ / set_ methods from the list of methods.

Showing Base Types

It's useful to be able to see the base types that the main types are inherited from. We can do this by adding another set of nodes. We'll add a container node called BaseTypeContainerNode and another node called BaseTypeNode . BaseTypeNode will display the actual name of the type, whereas BaseTypeContainerNode will just display the words "Base Types".

To add the nodes, we'll use recursion. The constructor for BaseTypeNode will take a Type object as a parameter and use the BaseType method to return (not surprisingly), the base type. The constructor will also create a new child node containing the sub-base type and so on:

public class BaseTypeContainerNode :
            BaseTreeNode
{
public class BaseTypeNode :
            BaseTreeNode
{
public BaseTypeNode(Form1 main, Type type) :
            base(main)
{
// add this type...
Text =
            type.FullName;

// now, iterate up...
if(type.BaseType !=
            null)
Nodes.Add(new BaseTypeNode(main,
            type.BaseType));
}
}

public BaseTypeContainerNode(Form1
            main, Type type) : base(main)
{
// text...
Text = "Base
            Types";

// start the iteration...
Nodes.Add(new
            BaseTypeNode(Main, type.BaseType));
}
}

In the LoadChildNodes method for TypeNode , add a call to check to see if there is a base type and, if so, create a new BaseTypeContainerNode :

protected override void
            LoadChildNodes()
{
// add an "extends"
            node...
if(Type.BaseType != null)
Nodes.Add(new
            BaseTypeContainerNode(Main, Type));

// iterate the
            methods...
foreach(MethodInfo method in
            Type.GetMethods())
Nodes.Add(new MethodNode(Main,
            method));
}

Run the project and now you'll be able to navigate the base types:

Updating "textNodeInfo"

So far we've ignored updating the text in textNodeInfo . What we'll do is add a property to BaseTreeNode called NodeInfo . This will return a string that describes the node. In the default implementation of OnClick , we'll get NodeInfo and display it on Form1 's textNodeInfo control.

Here's the override-able NodeInfo property that needs to be implemented on BaseTreeNode :

// NodeInfo property...
public virtual
            string NodeInfo
{
get
{
return "";
}
}

In Form1 , we need to wire up the AfterSelect handler. Add this code to the constructor:

public Form1()
{
//
// Required for
            Windows Form Designer
            support
//
InitializeComponent();

//
// TODO: Add
            any constructor code after InitializeComponent
//
treeviewTypes.BeforeExpand += new
            _
TreeViewCancelEventHandler(treeviewTypes_BeforeExpand);
treeviewTypes.AfterSelect
            += new
            _
TreeViewEventHandler(treeviewTypes_AfterSelect);
}

Then, add this handler method:

protected void treeviewTypes_AfterSelect(object sender, _TreeViewEventArgs e)
{
// what node do we have?
if(e.Node is BaseTreeNode)
((BaseTreeNode)e.Node).OnClick();
}

Now we can overload the NodeInfo property on whatever nodes we like. We'll just do TypeNode . First off, we want to display the access modifiers, type (as in class, enumeration, or interface) and name of the type. There are a bundle of methods on Type that give us information about the type, for example IsInterface , IsPublic , and so on. Add this code to TypeNode :

public override string
            NodeInfo
{
get
{
// get the access
            type...
StringBuilder builder = new
            StringBuilder();
if(Type.IsPublic ==
            true)
builder.Append("public ");
if(Type.IsNotPublic ==
            true)
builder.Append("notpublic ");
if(Type.IsAbstract ==
            true)
builder.Append("abstract ");
if(Type.IsSealed ==
            true)
builder.Append("sealed ");
if(Type.IsClass ==
            true)
builder.Append("class ");
if(Type.IsInterface ==
            true)
builder.Append("interface ");
if(Type.IsEnum ==
            true)
builder.Append("enum ");

// add the
            name...
builder.Append(Type.FullName);
builder.Append("\r\n");

//
            add attribute
            list...
AddAttributes(Type.GetCustomAttributes(true),
            builder);

// return...
return
            builder.ToString();
}
}

The AddAttributes method needs to be implemented on Form1 . What this will do is show us a list of the custom attributes set against the Type . We can reuse this method on other nodes, as classes like MethodInfo and PropertyInfo also support the GetCustomAttributes method. This returns an array of System.Object objects. Here's the method that you need to add to BaseTreeNode :

// AddAttributes...
protected void
            AddAttributes(object[] attributes, _StringBuilder
            builder)
{
// do we have any?
if(attributes == null ||
            attributes.Length == 0)
return;

// add a
            header...
builder.Append("\r\nAttributes:\r\n");

//
            loop...
foreach(object attribute in attributes)
{
// render
            it...
builder.Append(attribute.GetType().ToString());
builder.Append("\r\n");
}
}

Run the project now and you'll be able to see the attributes against the classes:

Summary

In this article we've taken a look at how we can use the System.Reflection namespace to build part of the functionality seen in a tool like Lutz Roeder's Reflector. We've also learned how we can use the TreeView control efficiently to create a tree that only loads information into its child node in a just-in-time or on-demand basis.

Case Study Information
Author | Matthew Reynolds
Technical Editor | Adam Ryland
Project Manager | Helen Cuthill

If you have any questions or comments about this case study, or suggestions for further pieces, please contact the technical editor. If you wish to talk about the case study with other subscribers, please visit our forum here.


MDI with C#

MDI with C#
http://www.csharptoday.com/content.asp?id=1823
Article Rating: 3.5
Published on February 18, 2002
Abstract
Multiple Document Interface (MDI) is the application interface made popular by applications like Microsoft Word and Microsoft Excel. This style has become the de facto standard for Windows based applications and is characterized by a window (called the MDI Container) assuming a dominant position, and other windows (called “MDI Child forms”) opening within the MDI Container boundaries. This is the first of a series of three articles in which Hersh Bhasin will expound MDI & Inheritance in C#. In this article, he will discuss various aspects of using menus, toolbars, and context menus in applications.
Article

Multiple Document Interface (MDI) is the application interface made popular by applications like Microsoft Word and Microsoft Excel. This style has become the de facto standard for Windows based applications and is characterized by a window (called the MDI Container) assuming a dominant position, and other windows (called "MDI Child forms") opening within the MDI Container boundaries.

This is the first of a series of three articles in which I will expound MDI & Inheritance in C#. In this article I will discuss various aspects of using menus, toolbars, and context menus in your application. The second article will concern itself with applying inheritance techniques to create an elegant framework for MDI Child forms. In the third article, I will discuss the theory behind inheritance and how it applies to classes, methods, fields and properties.

Each form in an MDI application can have a menu bar and a tool bar associated with it. The menu bar on the MDI Container form can serve as a "hanger" for the various MDI Child forms that make up the application. The MDI Container menu can in effect serve as the navigation tool for the application. The MDI Child forms can merge the MDI Container menu with its own menu and in this way code can be reused and extended.

Popular use and custom have established certain design guidelines for menu bars. On the MDI Container form, it is usual to have " File " as the first menu item and " Help " as the last. The second last menu item is usually " Window ". The " Window " menu item provides sub menu items that allow you to cascade or tile all open forms. It also provides a "windows-list". The windows-list is a list of all the open forms in the application and the form that has focus has a check mark against its name.

A MDI Child form can have its own menu bar and it can also merge the MDI Container menu bar with its own menu bar. I will describe the process of associating menu bars with forms later, but here I would have you know that attaching a menu bar to a form is as simple as dragging and dropping a MainMenu control from the Visual Studio ToolBox onto the form. If you have a menu bar both on the MDI Container form and on the MDI Child form, then on the MDI Child form you will have a single menu that is a merge of both the menus. In effect you have inherited the MDI Container menu on the MDI Child menu.

A MDI Child form concerns itself with a single aspect of the application. For example, a database centric form would need to provide functionality for record addition, deletion, and modification. It might also provide functionality to sort or filter records. This functionality is provided by methods residing in the body of the MDI Child form. The methods are made accessible by the menu items on the menu bar. It is common to have a toolbar on the MDI Child form. The toolbar is a bar that contains images and icons. Each of these images is associated with a menu item and clicking on them has the same effect as making a menu selection. MDI Child forms can also have context menus. These are menus that pop up when you right click with the mouse.

I will now discuss building various pieces of an MDI application using Visual Studio and C#. I will discuss this topic by building a fictitious accounting application called "Sirius for Windows" (I have build this application using the RC version of Visual Studio.NET). This application will have the following menu structure:

  1. Sirius for Windows MDI Container Menu Structure

Descriptive name \ Actual Name

Top level items

File \ mFile

Financial Accounts \ mFinancial

Invoices \ mInvoices

Inventory \ mInventory

Configure \ mConfigure

Windows \ mWindows

Help \ mHelp

Windows sub menus

Tile Vertical \ mTileVert

Tile Horizontal \ mTileHor

Cascade \ mCascade

File sub menus

Exit \ mExit

Financial Accounts sub menus

Masters \ mMasters

Bank/Cash \ mBank

Sale \ mSale

Purchase \ mPurchase

Journal \ mJournal

(dash)

Trial Balance \ mTB

Balance MDI Child \ mBS

Profit and Loss a/c \ mPL

Graphs \ mGraphs

Figure 1: Sirius for Windows menu structure

2. Create a windows application

Create a new windows application by revving up Visual Studio and selecting File/New/Project/Visual C# Projects in the left pane and Windows Application in the right.

Figure 1b: Creating a new Windows Application

3. Create the MDI Container form

Delete the Form1.cs file that is included by default. I am deleting this file because I want to call this form MDIframe.cs. I could have simply renamed this file, however Visual Studio.NET will not change the references to this form in its body. This seems to be a problem in Visual Studio.NET. Try changing the name by right clicking on the form name in the Solutions Explorer and select Rename. Now view the code and search for Form1.cs. You will see that references to Form1.cs still exist. Of course you can do a find and replace to change these references to the new form name.

Right click on the project name in the Solutions Explorer and select Properties. Note that I have set the Default Namespace property to be Sirius. You can specify any namespace here. All your subsequent objects will be created in the namespace you specify here. A namespace is a logical partitioning of an application. Objects residing in the Sirius namespace, and can only be referred to if you provide a fully qualified path. If you change the Default Namespace property here, make sure to change the namespace on pre-existing forms in your application as Visual Studio will not change the namespace on forms that have already been created. It will only do so on forms that are created subsequent to the namespace change.

Add a new form by right clicking on the project name in the Solutions Explorer and selecting Add /Windows Form. Call this form MDIframe.cs. Now code the Main() method on this form. The Main() method is the entry point of the application, and there is only one Main() function in each application. This function is provided in the Form1.cs, which is created by default. Any subsequent forms added to the application will not have this method. Since we deleted Form1.cs, we must code this method ourselves:

   [STAThread]
static void
            Main()
{
Application.Run(new MdiFrame());
}

Now right-click on the form and bring up the properties for the form. Set the IsMdiContainer property to True. If you do not see this property, right-click on the colored blocks on the top-left corner of the form and select Properties.

Right click on the project name in the Solutions Explorer and select Properties. Select the Start Up Object property as MDIframe.

Figure 2: The Start Up Object and the Default Namespace properties of the project.

4. Create the MDI Container menu bar and menu items

  • Drag and drop a MainMenu from the toolbar onto the form (or click on the MainMenu control in the ToolBar).
  • An object called MainMenu1 will appear at the bottom of the form and a menu bar at the top. The menu bar will have an input box with the caption "Type Here". Replace the word "Type here" with the menu text. Click on a blank area of the menu bar to add additional items. Tab to enter Top-Level menu items. Use the down arrow to enter sub menus. Figure 3 shows the form at this stage.

Figure 3: Adding a menu bar.

  • To add an accelerator, prefix the menu name with &
  • To add a separator line between menu item, replace the default caption "Type Here" with a dash ("-")
  • The menu names will be created with sequentially numbered names, for example, menuItem1, menuItem2, etc. To give it a descriptive name, right-click a menu-item and select Edit Name. Now you can give it descriptive names like mnuFile, mnuHelp, etc. You can also change the names by right clicking on the menu item, selecting Properties and modifying the Name property.
  • To copy menu items, right-click and copy the menu/menu items, Navigate to the destination menu spot and right-click/paste
  • Create the MDI Container menu structure as outlined in Step 1

5. Create a MDI Child form

  • Create a MDI Child form by following the steps outlined in step 3
  • Call this form MDISheetTest.cs.

6. Create the MDI Child menu bar and menu items

  • Create the MDI Child menu structure as outlined below. The process of using the Menu control from the ToolBox and creating it is the same for both the MDI Container and the MDI Child form.

Descriptive name \ Name

Activity \ menuItem1

Sub menus under Activity

Add \ mAdd

Save \ mSave

Delete \ mDelete

Close \ mClose

Set the Merge Order

At this point if you build and run this application (by pressing F5) and open the MDI Child form, you will note that the Activity menu on the MDI Child form appears last and the Windows and Help menus do not appear as the last and second last items on the menu. To fix this, set the MergeOrder property (on the MDIFrame.cs form) of the Configure (mConfigure) menu item to 1, the Windows menu item (mWindows) to 2, and the Help menu item (mHelp) to 3 (you can access this property by right-clicking on the appropriate menu item and selecting Properties and then selecting the property MergeOrder). Now since the Mergeorder of the Activity menu is zero and the other three menu items have MergeOrders greater than the Activity menu item, they will appear after it. Figure 4 shows the MDI Child form with the menu bar.

Figure 4: The menu bar on the MDI Child form.

7. MDI Container form menu scripts

In this step, I will show you how to code the click event of the menu items. I will code the click event for three menu items on the MDI Container menu bar. The first menu item will open the MDI Child form within the boundaries of the MDI Container window. The second menu item allows the user to gracefully exit the application, and the third menu item has sub menu items that perform window management functions like allowing the users to layer or cascade all open MDI Child forms.

To add code to the click event of a menu item, double click the menu item in the form design mode, which will create a blank skeleton of the method call. You can add your code within this skeleton.

Figure 4B: Method skeleton created by Visual Studio.

a. Opening MDI Child forms within the MDI Container

The MDI Child form (MDISheetTest) should be opened each time the MDI Container menu item called mMasters is clicked. Double click on this menu item in the form design mode and put the following code within the code skeleton that was created by VS.NET. Figure 5 shows how multiple open MDI Child forms appear in an MDI framework.

private void mMasters_Click(object sender,
            System.EventArgs e)
{
MDISheetTest NewMDIChild = new
            MDISheetTest();
//Set the parent of the new MDI child
            form.
NewMDIChild.MdiParent = this;
//Display the new MDI
            child form.
NewMDIChild.Show();}

Figure 5: Multiple open MDI Children in a MDI framework.

b. Exit the application (MDI Parent)

The exit menu item on the MDI Container menu bar should gracefully terminate the application after closing any open MDI Child forms. Double click on the mExit menu item and add the following code to the click event:

private void mExit_Click(object sender,
            System.EventArgs e)
{
Form[] childForm = this.MdiChildren
            ;
//close all child forms.
for(int i=0; i <
            childForm.Length ;
            i++)
childForm[i].Close();
Application.Exit();

}

c. Windows List

Normally in MDI applications, a list of open MDI Child windows appears under the "Windows" menu option (this is the convention). The MDI Child window that has focus has a check mark against its name in the list.

I assume that you have created a MDI Container menu item called Windows with sub menu items called mCascade, mTile and mTileHor as per our menu design specified in step 1. Set the MDILIST (Properties/Misc/MDILIST) property (right click on the menu item and select Properties) of menu item Windows to True. That's all you have to do to activate the windows-list. Now open a few instances of the Child form (by clicking mMasters). Click and look under the Windows menu item. You will see the list as shown in Figure 6.

Figure 6: The windows list.

The Windows menu item has sub menu items to layer and cascade open MDI Child forms. Enter the following code in the click events of sub menu items mCascade, mTile and mTileHor.

  • Cascade

    private void mCascade_Click(object sender,
              System.EventArgs e)
{
this.LayoutMdi(MdiLayout.Cascade);
}

  • Layer Vertical

    private void mTile_Click(object sender,
              System.EventArgs e)
{
this.LayoutMdi(MdiLayout.TileVertical);

}

  • Layer Horizontal

    private void mTileHor_Click(object sender, System.EventArgs e)
    {
    this.LayoutMdi(MdiLayout.TileHorizontal);
    }

8. Writing MDI Child Menu Methods

The MDI Child form is intended to handle database data. Thus it needs methods to add, delete and save data to the database. It also needs a method to close the current form. As I am just showing you how to hook up a menu bar to a form and not building an actual database application, I will just display a message box each time the add, save or delete menu items are clicked. Here is the code that should be added to the click events of various menu items.

Coding the Click events of menu items

private void mClose_Click(object sender, System.EventArgs e)
{this.Close();}

private void mAdd_Click(object sender, System.EventArgs e)
{add();}

private void msave_Click(object sender, System.EventArgs e)
{save();}

private void mDelete_Click(object sender, System.EventArgs e)
{delete();}

Now code the add, save and delete methods as follows:

Methods

private void add()
{MessageBox.Show("Add clicked");}

private void save()
{MessageBox.Show("Save clicked");}

private void delete()
{MessageBox.Show("Delete clicked");}

I could have added the code directly in the click event of the various menu items. However you will note that I am forwarding the events to the methods add(), save() and delete(). The advantage of using this technique is that both the menu item and toolbar button can refer to a single method. Another important advantage is that if I inherit from this form, the derived form can override these methods and provide its own implementation of these methods. In my next article, I will demonstrate how derived forms can use the methods created on the base form.

If you intend to have derived forms that inherit from this base form, each of these methods would have a visibility of public (or protected) and would be marked with the keyword virtual. Thus you would code the base add() function as follows:

public virtual void add()
{MessageBox.Show("Add clicked");}

The virtual keyword tells the compiler that the add() method of the appropriate instance should be called and that is why the add() method of the derived class is called, instead of the add() method of the base class.

The derived forms would then create methods with the same name but add the keyword override to provide their own implementation of these methods. For example, a form that inherits from the above base form would code the add() method as follows:

public override void add()
{MessageBox.Show("Derived Add clicked");}

Thus this technique allows me to create menu items that can fire methods on derived forms. Advanced C# users would point out that I could use delegates to achieve this functionality. However this design strategy allows me to have the same functionality that I would get with using a delegate, without the added complexity.

9. Create a toolbar on the MDI Child form

I will now show you how to attach a ToolBar to the MDI Child form:

  • Drag and drop a ToolBar onto the form (Visual Studio gives the default name ToolBar1)
  • Experiment with the toolbar by changing various properties as follows:

i)try ToolBar/right click / Property/Appearance = Flat

ii) Try Toolbar/ right click /Property/BoderStyle = None or FixedSingle or Fixed3D

Drag and drop a ImageList on the form (Visual Studio gives default name = ImageList1)

  • Now Link the ToolBar with an ImageList by setting the ToolBar/ right click /Properties/ImageList = ImageList1 (This appears as a drop down selection)

Figure 6b: Linking the ToolBar and the ImageList.

  • Now add some images by selecting ImageList1/ right click /Properties/Images.C lick on the three dots("..."). This brings up the Image Collection Editor. Click on Add. Browse and select a .bmp (or .ico) image. Note that a number (index) is assigned to this bmp. Repeat this process and select a few more bmp's. There are a number of icons supplied and are located in the folder ...program files\ Microsoft Visual Studio.NET\Common7\icons. The Image Collection Editor is shown in the following figure:

Figure 6c: The Image Collection Editor

  • Add ToolBar Buttons by Selecting ToolBar/ right click /Properties/Buttons and clicking on the three dots ("..."). This brings up the ToolBarsButton Collection Editor. Add Buttons and set the following properties:

i) Add a descriptive Name.

ii) Make the Text property blank

iii) Add a tool tip (the text entered here is displayed when mouse pointer is hovered over the button)

iv) If you need to add separators, select style/separator

v) Set the imageindex property equal to the index of the ImageList bmp that should appear with this Button. The images will appear in the drop down list for you to select.

  • Now code the ButtonClick event of the ToolBar (double click on the ToolBar to create a code skeleton) so that a method is called when a button on the toolbar is clicked. These methods were coded in step 8. The ButtonClic k event of the ToolBar is as follows:

private void toolBar1_ButtonClick(object
            sender, System.Windows.Forms.ToolBarButtonClickEventArgs
            e)
{
if (e.Button ==
            this.toolBarButton1)

{
add();
}
else if ( e.Button
            == this.toolBarButton2 )

{
save();
}
else if (
            e.Button == this.toolBarButton3)

{
this.Close();
}
}

10. Create a context menu on the MDI Child form

Context menu is the pop-up menu that appears when you right click on a form. To create this, do the following:

  • Drag and drop a ContextMenu control from the toolbox onto a blank portion of the MDI Child form (MDISheetTest.cs).
  • An instance of the context menu called contextMenu1 is added to the bottom of the form. Highlight it with your mouse and you will see an object captioned Context Menu at the top of the form. When you single-click it with your mouse, you see the words Type Here. Create menu items as described in step 4. For the purposes of demonstration create two menu items captioned Add and Delete. Right-click on each and select Properties. Give them the names ContextAdd and ContextDelete respectively. Now double-click on each to code their click events as follows:

    private void ContextDelete_Click(object sender, System.EventArgs e)
    {delete();}

    private void ContextAdd_Click(object sender, System.EventArgs e)
    {add();}

  • Now associate this ContextMenu with the form. This is done by setting the Forms ContextMenu property to this context menu (this will appear as a drop down selection in the ContextMenu property of the form). Figure 7 shows the context menu on the Masters form.

Figure 7: The context menu on the Masters form.

Before I conclude this article, I want to provide some tips on compiling the application manually. Working with Visual Studio.NET is great as it automates quite a number of things. However it also creates quite a number of files and folders. The application that we created in this article resides in only two files (four actually as there are also two resources file), but Visual Studio created quite a number of folders and files. This makes the application look more complex than it actually is. I personally use a text editor (TextPad) in conjunction with Visual Studio.NET.

I have provided a folder called SelfCompile in which I put the four files required for this application. These are MDIframe.cs, MDISheetTest.cs, Sirius.MdiFrame.resources and Sirius.MDISheetTest.resources. The resource files are automatically generated by Visual Studio and reside in the sub folder..obj\Debug. These files contain the resources (for example bitmaps) that you use in the application.

I list out all these files in a text file called files.txt, prefixing the resource files with the /res: switch. This file is read by the bat file make.bat that compiles the application to a window exe as follows:

csc /t:winexe /debug+ /r:System.dll /r:System.Data.dll /r:system.windows.forms.dll @files.txt

Summary

I have tried to keep this article quite fast-paced. I have attempted to summarize and consolidate information pertaining to creating C# MDI applications so that a seasoned programmer who is new to C# can have this information at one place. This article concentrated on the GUI and the navigational aspects of a MDI application. In the next article in this series I will attempt to show you how the C# inheritance capabilities can be used with the MDI Child forms to build an elegant windows MDI Child framework.

Article Information
Author | Hersh Bhasin
Technical Editor | Adam Ryland
Project Manager | Helen Cuthill
Reviewers | Shefali Kulkarni, Matthew MacDonald

If you have any questions or comments about this article, please contact the technical editor.


A Grid Layout Manager for your Windows Forms

A Grid Layout Manager for your Windows Forms
http://www.csharptoday.com/content.asp?id=1817
Article Rating: 3
Published on February 8, 2002
Abstract
While designing forms in Visual Studio, you move controls around the screen to achieve a desired look. This involves fixing the absolute position of each control. However, if the form is resized, the relative proportion of the control is lost. This may result in the application having an unpleasant appearance. In this article, Dr. PG Sarang describes the construction of a layout manager called GridLayout for the .NET platform, which helps solve this problem.
Article

While designing the user-interface in the Visual Studio IDE, you either drag the controls from the toolbox or double click on the control, and place them on the Form. A control placed on the Form may be resized and re-located to achieve the desired looks on the screen. In the process, we fix the absolute position of each control on the form. If the Form is re-sized or is opened on a screen with a different resolution, the relative proportion of the controls is lost. This may result in the screen having an unpleasant appearance. For example, the screen shot below shows a typical application screen developed in C#.

If you now resize the above form, the controls on the form remain at the absolute coordinates with respect to the top left corner. The expanded screen is shown below:

Now, we will have a look at Java Layout Managers. The Layout Managers in java allows you to specify the format in which the controls will be displayed on the container.

Java Layout Managers

Java solves the above problem by introducing Layout Managers. If you are a Java programmer, you may be familiar with the layout managers in Java that are used for placing components on a container. Java defines several layout managers for achieving the desired placement of components on a container.

The following layout managers are provided in Java:

  • FlowLayout - It lays out the controls in a row one after another, just the way a word processor arranges the words in a row. When one row completes, the items are put into the next row, and so on.
  • BorderLayout - It divides the screen into five zones namely, North, South, East, West, and Center, and the controls are added in the specified zone.
  • CardLayout - Here the added controls are put on top of one another just the way a stack of cards is created. Only the last added component is visible at the top of the stack.
  • GridLayout - It divides the screen into equal sized cells and places controls in the grid cells. Each component occupies one, and only one, grid cell.
  • GridBagLayout - It is similar to GridLayout, but it allows you to set the size and location of each control. This is the most complicated and most flexible layout manager.

It is beyond the scope of this article to discuss these layout managers. Suffice to say that by default, every Java Container has a layout manager associated with it. This layout manager takes the responsibility of maintaining the relative proportions of the various components added to the container. You may turn off the layout manager and locate the controls at absolute positions as in the case of Visual Studio IDE.

This article describes the construction of one such layout manager called GridLayout for the .NET platform. Let me first describe the functionality of the GridLayout manager in Java.

Java GridLayout Manager

When you use a GridLayout manager, the entire container screen is split into a grid-like structure. You decide on the number of rows and columns in the grid at the time of construction. The total number of cells in a grid should equal the number of components you wish to add to the container. Each component occupies exactly one cell. Each component is stretched to occupy the entire cell area.

The number of components added to the container may not always exactly match the number of grid cells, as the grid is rectangular and you may wish to add an odd-number of components. If the number of components added does not match the number of cells, the layout manager changes the grid size so as to reduce the amount of white space on the container.

The following screen shows a Java Applet on which six buttons are placed:

When you resize the applet parent frame, each button is resized to maintain the relative proportions. The screen shot of the resized applet is shown below:

GridLayout Manager on .NET Platform

To provide the functionality similar to a Java GridLayout manager for forms created on .NET platform, I created a class called GridLayout in C#.

The VS.NET generated API documentation is given below as a reference. The class definition and the entire source are discussed in the next section.

The class defines several private data members that are accessed through the corresponding get / set methods:

Members | Description
parent | Scope : private Container reference
controlArray | Scope : private Collection object to store references to controls added
rows | Scope : private Number of rows
cols | Scope : private Number of columns
hgap | Scope : private Horizontal gap between two controls
vgap | Scope : private Vertical gap between two controls

The class defines the following constructors:

Members | Description
GridLayout | Constructor Parameters : (System.Windows.Forms.Control control) Creates a grid layout with a default of one column per component, in a single row.
GridLayout | Constructor Parameters : (System.Windows.Forms.Control control, int hgap, int vgap) Creates a grid layout with the specified number of rows and columns.
GridLayout | Constructor Parameters : (System.Windows.Forms.Control parent, int rows, int cols, int hgap, int vgap) Creates a grid layout with the specified number of rows, columns, hgap and vgap

The class defines the following interface methods, both the private and public methods are listed below:

Members | Description
Add | Scope : public Returns : void Parameters : (System.Windows.Forms.Control control) Adds the specified component to the layout and updates the display
Remove | Scope : public Returns : void Parameters : (System.Windows.Forms.Control control) Removes the specified component from the layout and updates the display
GetRows | Scope : public Returns : int Parameters : - Gets the number of rows in this layout.
GetColumns | Scope : public Returns : int Parameters : - Gets the number of columns in this layout.
GetVgap | Scope : public Returns : int Parameters : - Gets the vertical gap between components.
GetHgap | Scope : public Returns : int Parameters : - Gets the horizontal gap between components.
SetRows | Scope : public Returns : void Parameters : (int rows) Sets the number of rows in this layout to the specified value and updates the display
SetColumns | Scope : public Returns : void Parameters : (int columns) Sets the number of columns in this layout to the specified value and updates the display
SetHgap | Scope : public Returns : void Parameters : (int hgap) Sets the horizontal gap between components to the specified value. and updates the display
SetVgap | Scope : public Returns : void Parameters : (int vgap) Sets the vertical gap between components to the specified value. and updates the display
SetRows | Scope : public Returns : void Parameters : (int rows) Sets the number of rows in this layout to the specified value and updates the display
SetColumns | Scope : public Returns : void Parameters : (int cols) Sets the number of columns in this layout to the specified value and updates the display
Resize | Scope : private Resize event handler of the container
Update | Scope : public Returns : void Parameters : - Displays the changes made to the container after adding/removing control
Show | Scope : private Returns : void Parameters : - Add controls to the container and display the container
AddControls | scope : private Returns : void Parameters : int height, int widht, int NumberOfColumns Set size and location of each control and add it to the container

GridLayout class

The GridLayout class is a layout manager that is responsible for laying the components on a container. The class receives the container reference in its constructor. The grid size may be specified at the time of construction. Similarly, the horizontal and vertical spacing between the components may be specified at the time of construction. The class provides several overloaded constructors for this purpose.

The components are added to the container by calling the Add method of the layout manager. The layout manager maintains a list of added components in a Collection object. The Remove method of the layout manager removes a specified component from the container. The layout manager computes the size of each component before adding it to the container. Whenever you resize the container, the layout manager re-computes the sizes of all the components contained within.

Let us now look at the class implementation.

Data Members

The class declares the following private data members:

private System.Windows.Forms.Control parent;
private ArrayList controlArray =
ArrayList.Synchronized(new ArrayList());
private int rows;
private int cols;
private int hgap = 0;
private int vgap = 0;

The parent data member stores the reference to the container on which the controls are to be placed. The container in this case may be a Form or a Panel. Both are derived from the Control class. The controlArray data member is of type ArrayList and stores the references to the controls added to the container. The access to this Collection object is synchronized with the help of the Synchronized method.

private ArrayList controlArray = ArrayList.Synchronized(new ArrayList());

The rows member stores the number of rows, and the cols member stores the number of columns in the current layout. The hgap and vgap members store the horizontal and vertical gaps between the components respectively.

Class Constructors

The class defines three overloaded constructors. Each of the constructors receives a parameter of type Control. The caller passes a reference to the container in this parameter. Each constructor also sets the event handler for the Resize event of the parent object (container object).

The first constructor that takes a single parameter of type Control is shown below:

public
            GridLayout(System.Windows.Forms.Control parent)
{
this.parent
            = parent;
this.rows = 1;
this.cols = 1;
this.parent.Resize
            += new System.EventHandler(Resize);
}

The constructor copies the reference to the parent control in a class variable. It sets the rows and cols to 1. You may change the number of rows and columns later by calling the SetRows and SetColumns methods respectively. The layout manager may re-compute the number of rows and columns. This depends on:

  • The number of controls to be added
  • Initial values of Rows and Columns

The second constructor additionally receives the row and column values as two parameters, and copies them in the private members of the class.

public GridLayout( System.Windows.Forms.Control parent,
int rows,
int cols)
{
this.parent = parent;
this.rows = rows;
this.cols = cols;
this.parent.Resize += new System.EventHandler(Resize);
}

The third constructor additionally receives horizontal gap and vertical gap values as parameters, and copies them in the private members of the class.

public GridLayout( System.Windows.Forms.Control parent,
int rows,
int cols,
int hgap,
int vgap)
{
this.parent = parent;
this.rows = rows;
this.cols = cols;
this.hgap = hgap;
this.vgap = vgap;
this.parent.Resize += new System.EventHandler(Resize);
}

Accessor/Mutator methods

The class defines several accessor / mutator ( get / set) methods that operate on the private data members of the class. The GetRows method returns the number of rows:

   public int
            GetRows()
{
return rows;
}

The SetRows method accepts the new value for rows as the method argument, and sets the private data member to the new value. The method also calls the private Show method of the class to update the display:

   public void SetRows(int
            rows)
{
this.rows = rows;
this.Show();
}

Likewise, the class defines the get / set methods for cols, vgap and hgap data members. Each set method, after setting the new value for the data member, updates the display by calling the Show method.

Event Handler

The class defines only one event handler. This event handler is attached to the Resize event of the container. The handler simply calls the Show method of the class. The Show method re-computes the component sizes and updates the display. The Show method is discussed later.

private void Resize (object sender,
            System.EventArgs e)
{
this.Show();
}

Update Method

This is a public method that the client calls to cause an explicit update of the container display. The client would generally call this method after adding a control to the container or after removing an existing control from the container. The method calls the private Show method defined in the class.

   public void
            Update()
{
this.Show();
}

Show Method

This is the most important method of our class and defines most of the class functionality. This method is implicitly called by the layout object after resetting the number of rows, number of columns, hgap and vgap. This is also implicitly called within the resize event handler. The method first determines the client rectangle size of the container on which the controls are to be placed. Each control has a property called ClientRectangle which gives you the size of rectangle on which the controls are to be placed.

System.Drawing.Rectangle ClientRectangle = this.parent.ClientRectangle;

Next, it checks for the added controls in its controlArray list. If the list does not contain any items, we do not need to add anything to the container and the program simply returns to the caller.

// If no controls are added, simply return
if(controlArray.Count == 0)
return;

The program now clears all the previous controls on the container by calling its Clear method:

// Remove all the controls from the container
parent.Controls.Clear();

The program next checks the current value of the rows variable. If this is greater than zero, it indicates that the user has specified the number of desired rows. If this value is zero, we need to calculate the number of rows that depends on the number of components added to the container and the number of desired columns.

If the rows value is greater than zero, the number of columns is determined by dividing the total number of components to be added by the number of rows.

int NumberOfColumns = (int) Math.Ceiling (controlArray.Count/(double)rows);

Here, Math.Ceiling() will return the lowest number which is greater that its argument number.

Once the number of rows and columns is fixed, the cell dimensions are computed by dividing the client rectangle into an equal number of cells.

// Calculate height and width of each control
height = (int)Math.Round(ClientRectangle.Height /(double)rows);
width = (int)Math.Round(ClientRectangle.Width
/(double)NumberOfColumns);

These values must be adjusted to account for the spacing between the rows and columns specified by the vgap and hgap member variables.

// Adjust the height/width for vgap/hgap
height -= (int)Math.Round((vgap*(rows-1))/(double)rows);
width -= (int)Math.Round(
(hgap*(NumberOfColumns-1))/(double)NumberOfColumns);

The program now calls the AddControls method to add all the controls to the container.

AddControls(height, width, NumberOfColumns);

The AddControls method is discussed later.

The logic for placing the components in the case of rows being zero is identical to the above case. We first compute the number of rows from the control count and the desired number of columns.

// if(rows==0) calculate the number of rows from
// control count/cols
int NumberOfRows =
(int)Math.Ceiling(controlArray.Count/(double)cols);

The cell dimensions are then determined and the adjustments for hgap, vgap values are made:

// Set height and widht of each control
height = (int)Math.Round(
ClientRectangle.Height/(double)NumberOfRows);
width = (int)Math.Round(ClientRectangle.Width/(double)cols);
// Adjust the height/width for vgap/hgap
height -= (int)Math.Round(
(vgap*(NumberOfRows-1))/(double)NumberOfRows);
width -= (int)Math.Round((hgap*(cols-1))/(double)cols);

As in the earlier case, the AddControls method is called to add the controls to the container.

AddControls method

The AddControls method is called through the Show method. It iterates through the list of controls to be added, sets the location and dimension for each control, and adds it to the container.

The program first creates an Enumerator for iterating through the list of components to be added to the container.

// Get the list of all the controls in the controlArray
IEnumerator list = controlArray.GetEnumerator();

The starting coordinates for the first control are then set:

// Get the XPos and YPos for the first Control
int YPos = ClientRectangle.Location.Y;
int XPos = ClientRectangle.Location.X;

The program then iterates through the list of controls, sets the location and size for each control and adds it to the container:

      // Add
            controls to the container
while(list.MoveNext())
{
// Get
            the current control from the list
control =
            (System.Windows.Forms.Control)list.Current;
// Set the location
            and size of the control
control.Location = new
            System.Drawing.Point(XPos, YPos);
control.Size = new
            System.Drawing.Size(width, height);
// Add the control to the
            container
parent.Controls.Add(control);
// adjust XPos, YPos
            before adding new control
.
}

Within the loop, if we reach the end of the row, the Xpos and YPos variables are adjusted before a new control is added.

         //
            If the last column of current row has reached then
// update the
            Y position to the new row value and set
// X position to initial
            value.
if(ColumnNumber == NumberOfColumns)
{
YPos +=
            control.Height + vgap;
XPos =
            ClientRectangle.Location.X;
ColumnNumber = 1;
}

If the end of row is not yet reached, we adjust only the XPos value.

         else
{
//
            Otherwise update the X position
XPos += control.Width +
            hgap;
ColumnNumber++;
}

This completes our discussion of the GridLayout class. We will now develop a test application for testing our GridLayout manager.

Test Application

Create a C# Windows Application project in VS.NET. On the default form, we will first add two panels using a grid layout. The panels will be placed in two equal rows on the form. On each panel we will add a few buttons using a grid layout manager. The screen shot of the program output is shown below:

In the class constructor, we create two panel objects:

System.Windows.Forms.Panel p1 = new System.Windows.Forms.Panel();
System.Windows.Forms.Panel p2 = new System.Windows.Forms.Panel();

A GridLayout manager is then created:

GridLayout g = new GridLayout(this,2,0);

Note that we send this as the first parameter in the above constructor indicating that this layout manager will place the controls on the current Form. The grid size is set to 2 rows and a variable number of columns. The number of columns is determined by the number of components present to the form.

The two panels are then added to the layout manager by calling its Add method:

g.Add(p1);
g.Add(p2);

Note that after adding the controls, we need to update the display by calling the Update method of the layout manager:

// Update the Form
g.Update();

We now create a few buttons for adding to the panel:

System.Windows.Forms.Button b1 = new System.Windows.Forms.Button();
System.Windows.Forms.Button b2 = new System.Windows.Forms.Button();
System.Windows.Forms.Button b3 = new System.Windows.Forms.Button();
System.Windows.Forms.Button b4 = new System.Windows.Forms.Button();
System.Windows.Forms.Button b5 = new System.Windows.Forms.Button();

b1.Text = "Button1";
b2.Text = "Button2";
b3.Text = "Button3";
b4.Text = "Button4";
b5.Text = "Button5";

The buttons are added to the first panel by calling the Add method of the layout manager.

         //
            Add controls to the
            panel1
g1.Add(b1);
g1.Add(b2);
g1.Add(b3);
g1.Add(b4);
g1.Add(b5);

A button that is previously added may be removed using the Remove method:

// Remove control from panel1
g1.Remove(b1);

The panel is updated with the help of the Update method:

// Update panel1
g1.Update();

The vertical and horizontal gap between the controls may be set by calling the setHgap and setVgap methods:

         //
            set vertical gap between two controls
g1. setHgap(g1.GetHgap() +
            15);
// set horizontal gap between two
            controls
g1.setVgap(g1.GetVgap() + 15);

Likewise, few buttons are added to the second panel to create the screen shown above.

Use of Layout Managers in Real Life

In addition to the Grid Layout manager, I have created the other layout managers described at the beginning of this article. I use these layout managers in developing screens for .NET applications. Two sample screens from one such application are shown below. I shall describe the construction of other layout managers in future articles.

Conclusion

Java provides several layout managers for placing components on a container. The layout manager assumes the responsibility of maintaining the relative proportions of the controls whenever the container size or the screen resolution changes. This gives the same look to the user at various resolutions and form sizes. The .NET class library does not provide any such layout managers. This article discusses the construction of a Grid Layout manager that places the controls on the container in a rectangular grid. The developer decides the grid size while constructing the layout manager. Each component that is added to the layout manager occupies exactly one grid cell. Thus, the default component size is not honored. The grid manager is intelligent enough to decide on the amount of white space left on the container after all the controls are added to it. The manager automatically reduces the amount of white space by re-calculating the number of rows and columns. The layout manager developed here provides functionality quite similar to Java GridLayout manager. Those who are familiar to Java layout manager should be able to use this new layout manager for the .NET platform easily, as the interface developed is almost identical to the interface exposed by the Java GridLayout manager. This layout manager may be compiled and stored in a DLL and may later be used by any of the .NET languages to place the controls on a Form or a Panel.

Article Information
Author | Dr P.G. Sarang
Technical Editor | Adam Ryland
Project Manager | Helen Cuthill
Reviewers | Andrew Krowczyk, Susan Connery

If you have any questions or comments about this article, please contact the technical editor.


Notification Framework with C#

Notification Framework with C#
http://www.csharptoday.com/content.asp?id=1815
Article Rating: 3.5
Published on February 6, 2002
Abstract
In this article, Catalin Tomescu walks you through the development of a notification framework, along with a sample application that shows how to use the framework. Along the way, we look at how to add features such as being able to broadcast a message to any number of clients, to dispatch messages to clients over the Internet or on a local network, the capability for a client to create custom notification threads on the server side, and the capability to update messages while they are in the queue.
Article

Overview

The idea for this project came to me from one my assignments, to build a simple dispatcher-notification framework for a call center. This article will walk you through the development of the notification framework along with a sample application that shows how to use the framework.

In the process of software development there is often a need to send notifications between applications on the same machine or on different machines, in the same network or on separate networks. The complexities faced in each situation increase as we move from local area networks (LANs) to Internet. There are more aspects to consider when sending messages over the Internet, such as firewalls and proxies. In this situation you must consult with the network administrator, who knows which ports are open on the firewall, so that you can use them in your application. In the case of LANs the problems are simpler because all computers are part of the same network and there are no firewalls.

Take the following scenario for example: a customer calls for assistance with his/her computer. The call is intercepted by the IVR (Interactive Voice Response) system and, according with the user selection, a message is sent to one of the customer call queues. A representative will process the call by picking it from the queue. In such a business the response time is very important, and customer satisfaction is the key to success. Therefore, the customer must wait in the queue as little as possible or otherwise a compensatory action is triggered. In our case, a discount is applied for the customers that wait in the queue more than 10 min.

To accommodate the scenario described above, we need to build an application with the following features:

  • Broadcast a message to any number of clients.
  • Dispatch messages to clients over the Internet or on a local network.
  • Capability for a client to create custom notification threads on the server side.
  • Capability to update messages while they are in the queue.

Before going forward and analyzing possible solutions to the problem we must enumerate other constraints such as:

  • The software should operate in a few disconnected networks.
  • The application will be installed on a Windows operating system.
  • As with many systems, the cost of maintenance and operation should be as low as possible.

Let's see now the possible solutions to the problem stated above:

  • Each client application polls the queue at predefined intervals of time in order to determine if a new message was posted. The most frequent solution is to place the events in a database and client applications execute SQL SELECT statements to retrieve existing records.
  • Using a messaging server like MSMQ. The .NET Framework provides a namespace called System.Messaging that is used to interact with a MSMQ server using an object hierarchy similar to the COM counterpart.
  • Build a custom framework to accommodate special needs.

The first solution is not acceptable because it will cause a delay between the moment the message arrives in the queue and the time clients are notified. If the customer representative would spend time pressing a Refresh button, or the application would consume client and server resources for repeatedly querying the database, it would be a disaster. Even if the solution would involve the use of a background thread, the resources consumed on both the server and client side are considerate and unnecessary. What I mean is that the client and server would require time to create and send responses to requests.

For most scenarios the second solution is a very good candidate, but there are few things to keep in mind:

  • MSMQ requires a Windows 2000/NT Server installation to enable creation of public queues. The MSMQ server allows, depending on the OS installation, creation of public and/or private user defined queues. Public queues are accessible from remote computers, whereas private queues are visible only on the machine where they were created.
  • The lack of message broadcasting. This means that when a message arrives in the queue, the message is dispatched to one of the clients listening on the queue. The client that receives the message can't be predetermined. MSMQ requires additional servers like Microsoft DTC to be installed on the machine. We mentioned earlier that there are two types of queues in terms of visibility: private and public. In terms of transaction support any queue, private or public can be transactional or non transactional. MS DTC is required because of the transactional
  • Queues support in MSMQ. Transactional messaging refers to the coupling of several related messages into a single transaction. Sending messages as part of a transaction ensures that the messages are delivered in order, delivered only once, and successfully retrieved from their destination queue.

The third solution requires building a custom framework that would address all needs. This solution might seem the most costly, but for special cases this is the way to go. Because of the special requirements enumerate above, the MSMQ solution won't fit because it would require installing several Windows Servers, one for each network.

Building the framework

The notification framework is developed using the .NET Framework Release version and C# language. The software can be installed on a computer with .NET framework and does not require any other messaging software.

Architecture

The following UML diagram presents the architecture of my notification framework.

Fig 1 - Notification Framework Architecture

As you can see, the core of the framework, the server and the queues, resides on the server computer. A client will acquire a server object via a TCP .NET Remoting Channel. I chose TCP channels because they are more efficient in terms of performance compared with HTTP channels, which use SOAP to serialize/de-serialize an object. TCP channels are more efficient because of the binary persistence used in the transport protocol, which in this case is TCP. The SOAP channels use HTTP protocol, and the overhead of the SOAP layer makes this type of channel not as efficient as TCP channels. All objects exposed by the framework inherit from the MarshalByRefObject class, which takes care of all transport specific issues.

The Object Factory class, which must implement the ObjectFactory interface, contains the implementation for Message and Queue objects. I choose to leave this outside the server component because someone else might want to write his/her own implementation of message and queue objects, using the IMessage, IQueue and NotificationServerObjectFactory interfaces. This would be useful in cases where a team of developers choose to build different parts of the projects using different languages, or reuse components already built. Because of the language independence added in .NET you can write these objects in any language you prefer: Visual Basic .NET, C #, or anything else which is CLI compliant. The Server component will use the factory object that it is passed during server initialization.

The project is organized at a source code level as follows:

  • NFCommon.dll contains definitions common to all components in the project
  • NFCSharpFactory.dll contains the C # implementation of the factory library.
  • NFServer.exe is the notification server executable.

Subscribers

The method by which the client will receive notifications from the server is by using a .NET delegate. A delegate defines a reference type that can be used to encapsulate a method with a specific signature. A delegate instance encapsulates a static or an instance method. Delegates are roughly similar to function pointers in C++; however, delegates are type-safe and secure.

Each queue of the notification server has a method that allows clients to subscribe for a specific type of notifications:

  • New message in queue
  • Messages deleted from the queue
  • Message updated in the queue

This approach is similar with the publisher-subscriber design pattern. A server application called publisher is the source of events, and a client application interested in receiving these events is the subscriber.

Should a client be interested in receiving any of the above-mentioned types of notifications, it will call the bindSubscriber method to subscribe. When a client is not interested in receiving notification it should call the unbindSubscriber method.

The source code of those two methods is listed below:

public void bindSubscriber(QueueNotification subscriber,
SimpleServerEnums.NotificationTypeEnum subsType) {

m_notifiers[(int)subsType] += subscriber;

if (subsType == SimpleServerEnums.NotificationTypeEnum.NT_New)
if (m_messages.Count>0)
for (int i=0; i< m_messages.Count; i++)
subscriber((Message)m_messages[i]);
}

public void unbindSubscriber(QueueNotification subscriber,
SimpleServerEnums.NotificationTypeEnum subsType) {

m_notifiers[(int)subsType] -= subscriber;
}

As you can see, those methods use as a parameter a QueueNotification type, which is defined as:

public delegate void QueueNotification (IMessage arg);

The only argument is of type IMessage , which is the interface that queue messages implement. The following section will describe the IMessage interface.

In addition to binding the client with the requested delegate, the bindSubscriber function sends notification to the newly attached client for all messages in the queue, only if the subscription requested is of type NotificationTypeEnum.NT_New . This way the client has the opportunity to discover all messages in the queue.

Here is the definition of the enumeration, which contains all types of notifications a client can bind to.

    public sealed class
            SimpleServerEnums {
public enum NotificationTypeEnum : int
            {
NT_Null = -1,
NT_New = 0,
NT_Remove = 1,
NT_Update =
            2
}
}

Messages and Persistence

All messages handled by the framework implement the IMessage interface. This interface is defined as follows:

public interface IMessage {
bool FieldExist (string key);

string getFieldValue (string key);
void setFieldValue (string key, string value);

string get_Body();
void set_Body(string value);

Guid get_Id();

DateTime get_timeEnteredQueue();
void set_timeEnteredQueue(DateTime value);

bool Store(StreamWriter streamW);
bool Load(StreamReader streamR);
}

The messages that the framework handles have few standard fields, but the clients can define as many fields as needed. The standard fields are: ID, BODY and TIME ENTERED QUEUE.

The message interface also defines two methods, Store and Load , for persisting and loading a message from a storage stream. The implementer object of the IMessage interface can choose any type of persistence which is appropriate in the application. Our implementation of IMessage uses the file system for message persistence.

Let's take a look now at the Message class:

public class Message : MarshalByRefObject, IMessage {
Hashtable m_fields;

The default Message constructor initializes the m_fields member and adds the identifier field. This field is of type GUID, which is a unique identifier. The generation is based on a special algorithm, which involves current system time and complex random functions. The GUID is designed to ensure uniqueness between any other GUID generated values anywhere in the world. These properties make it a good choice for message identifier field.

            public
            Message () {
m_fields = new
            Hashtable(5);
m_fields.Add(MessageFieldsEnum.MSG_FIELD_GUID,
            Guid.NewGuid().ToString());
Console.WriteLine("Message()");
}

A second constructor is provided for situations when a simple message is to be created. Besides assigning the identifier, this constructor also sets the body field with the supplied value.

                public
            Message (string body) {
m_fields = new
            Hashtable(5);

m_fields.Add(MessageFieldsEnum.MSG_FIELD_BODY,
            body);
m_fields.Add(MessageFieldsEnum.MSG_FIELD_GUID,
            Guid.NewGuid().ToString());

Console.WriteLine("Message(body)");
}

The field specific functions are:

  • FieldExist check if a field is defined for this message.
  • GetFieldValue returns the value of a field
  • SetFieldValue set the value of a field

A client can check if a particular field is defined for a message by calling the FieldExist function. Message fields are stored in a hash table, which allows element retrieval by key and also ensures that a field is unique in the list of fields managed by the hash table.

        public
            bool FieldExist (string key) {
return
            m_fields.ContainsKey(key);
}

public string getFieldValue
            (string key) {
return (string)m_fields[key];
}

public
            void setFieldValue (string key, string value) {
m_fields[key] =
            value;
}

The next three functions retrieve the values of standard attributes for a message.

        public
            string Body {
get { return
            (string)m_fields[MessageFieldsEnum.MSG_FIELD_BODY]; }
set
            {m_fields[MessageFieldsEnum.MSG_FIELD_BODY] = value;
            }
}

public Guid Id {
get { return new
            Guid((string)m_fields[MessageFieldsEnum.MSG_FIELD_GUID]);
            }
}

public DateTime timeEnteredQueue {
get { return
            DateTime.Parse((string)m_fields[MessageFieldsEnum.MSG_FIELD_TIMEENTEREDQUEUE]);
            }
set { m_fields[MessageFieldsEnum.MSG_FIELD_TIMEENTEREDQUEUE] =
            value.ToString(); }
}

To prevent losing messages stored in queues when the server stops we implemented a persistence mechanism. One of the settings that the framework supports is Storage Location. This location will be used by each queue to create a folder named after queue's GUID. This folder is the storage location for messages managed by the queue.

As you'll notice, each message object implements two functions, Load and Store . These two functions are used when a message is saved to or loaded from a file.

The Store function will create a XML document containing all the fields defined in the message. For each field a separate XML element is created under the root element. Each message will have the following structure:

<message id="05b017f6-8050-4fa0-964f-3e1c801f6580">
<BODY>Bulk 3</BODY>
<GUID>05b017f6-8050-4fa0-964f-3e1c801f6580</GUID>
<TIME_ENTERED_QUEUE>1/8/2002 1:43:23 AM</TIME_ENTERED_QUEUE>
</message>

Let's take a look at the message with a few other fields:

<message id="05b017f6-8050-4fa0-964f-3e1c801f6580">
<BODY>Bulk 3</BODY>
<GUID>05b017f6-8050-4fa0-964f-3e1c801f6580</GUID>
<TIME_ENTERED_QUEUE>1/8/2002 1:43:23 AM</TIME_ENTERED_QUEUE>
<MANAGER_DISCOUNT>$20</ MANAGER_DISCOUNT>
<SERVICER_ID>101</SERVICER_ID>
</message>

To create the XML document we used a namespace provided with the .NET Framework, System.Xml .

        public
            bool Store (StreamWriter streamW) {
bool _ret =
            true;
XmlElement _elem;
XmlText _nodeTxt;

try
            {
XmlDocument _doc = new
            XmlDocument();

_doc.LoadXml("<message id=\"" +
m_fields[MessageFieldsEnum.MSG_FIELD_GUID].ToString() +
            "\"></message>");

ArrayList _keys = new
            ArrayList(m_fields.Keys);
ArrayList _values = new
            ArrayList(m_fields.Values);

for (int i=0; i< _keys.Count;
            i++) {
_elem =
            _doc.CreateElement(_keys[i].ToString());
_nodeTxt =
            _doc.CreateTextNode(_values[i].ToString());
_doc.DocumentElement.AppendChild(_elem);
_doc.DocumentElement.LastChild.AppendChild(_nodeTxt);
}

_doc.Save(streamW);

}
            catch (Exception e) {

_ret =
            false;
Console.WriteLine(e);
}

return
            _ret;
}

When a message is loaded from the storage location using the Load function, the framework recreates all the fields by parsing all child elements of the root element. More details on how the messages are loaded are provided in the Queues section.

        public
            bool Load (StreamReader streamR) {
bool _ret =
            true;
XmlElement _elem;
XmlDocument _doc = new
            XmlDocument();
string attrValue,
            name;

_doc.Load(streamR);
_elem =
            _doc.DocumentElement;
//attrValue =
            _elem.GetAttribute("id");
//m_fields[MessageFieldsEnum.MSG_FIELD_GUID]
            = new Guid(attrValue);

try {
_elem =
            (XmlElement)_elem.FirstChild;
while (_elem != null)
            {
attrValue = _elem.FirstChild.Value;
name =
            _elem.Name;

m_fields[name] = attrValue;
_elem =
            (XmlElement)_elem.NextSibling;
}

} catch (Exception e)
            {
Console.WriteLine(e);
_ret = false;
}

return
            _ret;
}
}

Queues

Queue is one of the most important objects in the framework because it handles all message operations. The interface that defines the queue is IQueue and is defined as:

public interface IQueue {
Guid CreateCustomThread(QueueMessageFilter filter,
int interval, QueueNotification notifyThis);

void PauseCustomThread (Guid id);
void ResumeCustomThread (Guid id);

IMessage CreateMessage(string body);
IMessage CreateMessage();

void bindSubscriber(QueueNotification subscriber,
SimpleServerEnums.NotificationTypeEnum subsType);

void unbindSubscriber(QueueNotification subscriber,
SimpleServerEnums.NotificationTypeEnum subsType);

bool Post (string msg);
bool Post (IMessage msg);

bool Remove (int index);
bool Remove (Guid id);

int Count();

Guid Id ();

IMessage Peek (int index, bool removeFromQueue);
IMessage Peek (Guid id, bool removeFromQueue);

bool Update (int index, IMessage message);
bool Update (Guid id, IMessage message);

ArrayList get_Messages();

bool LoadMessages();
}

I already showed how to bind and unbind subscribers to a queue. Let's now examine remaining functions in the IQueue interface.

Message related functions

Post methods are used to post a message to the queue. First, Post methods takes a string as a parameter, which is useful in the cases when a standard message is enough. That means that if you want to post a message with a standard structure, without any additional fields this method becomes very useful. Secondly, Post methods takes an IMessage as a parameter and it is most useful in cases when a message with a more complex structure needs to be posted to a queue.

The source code for the Post methods is listed below:

There are two Post methods provided. The first one is used when a simple message, containing only the body field is to be posted in a queue. The function creates a Message object and calls the second Post method.

public bool Post(string body) {
return Post(new Message(body));
}

The second Post method is more complex, as it contains the business rule for adding the message to a list of messages and notifying clients of this new message arrival in the queue.

First the message is added to the list of messages, and than a file is created on the disk and the Store method is called with the file Stream as the parameter.

        public
            bool Post (IMessage msg) {
bool _ret = false;

try
            {
msg.set_timeEnteredQueue(System.DateTime.Now);
m_messages.Add(msg);

StreamWriter
            _streamW = File.CreateText(m_storage + "\\" +
            msg.get_Id());
msg.Store(_streamW);
_streamW.Close();

_ret
            = true;

if
            (m_notifiers[(int)SimpleServerEnums.NotificationTypeEnum.NT_New] !=
            null) {
try
            {
m_notifiers[(int)SimpleServerEnums.NotificationTypeEnum.NT_New](msg);

}
            catch (Exception e) {
System.Console.WriteLine(e);
}
}
}
            catch (Exception e)
            {
System.Console.WriteLine(e);
}

return
            _ret;
}

Peek methods are used to get a message from the queue with the option to remove or leave the message in the queue. There are situations when a client wants to only read the information in the message without removing the message in the queue.

Here is the source code for these two methods:

        public
            IMessage Peek (int index, bool removeFromQueue) {
IMessage
            _tmpMsg = null;

try {
_tmpMsg =
            (Message)m_messages[index];
if (_tmpMsg != null &&
            removeFromQueue == true)
Remove(index);

} catch (Exception
            e) {
System.Console.WriteLine(e);
}

return
            _tmpMsg;
}

public IMessage Peek (Guid id, bool
            removeFromQueue) {
int _index = -1;

_index =
            FindIndexOf(id);
if (_index >= 0)
return Peek(_index,
            removeFromQueue);
else
return null;
}

We provided two methods each taking a different first parameter:

  • An integer representing messages' index.
  • Message identifier.

Removing a message from the queue without reading the message is possible by calling one of the Remove functions. We included two methods, each taking either an index or an identifier as a parameter.

The first Remove method takes an index as parameter and contains the code for removing the message from the hash table collection, as well as the code to delete the associated file from the disk. If those two operations are successfully completed, client applications are notified of message removal by calling the delegate associated with the clients interested in message removal notifications.

        public
            bool Remove (int index) {
bool _ret = false;
IMessage
            msg;

try {
msg =
            (Message)m_messages[index];

File.Delete(m_storage + "\\" +
            msg.get_Id());
m_messages.RemoveAt(index);

m_notifiers[(int)SimpleServerEnums.NotificationTypeEnum.NT_Remove](msg);

_ret
            = true;

} catch (Exception e)
            {
System.Console.WriteLine(e);
}

return
            _ret;
}

The second Remove function take a GUID as parameter. This allows client applications to remove messages based on message identifier instead of index. The function first looks up the message in the hash table based on the passed parameter to find out the message index and than call the first Remove method.

        public
            bool Remove (Guid id) {
int _index = -1;

_index =
            FindIndexOf(id);
if (_index >= 0)
return
            Remove(_index);
else {
Console.WriteLine("Message not
            found!");
return false;
}
}

Next let's take a look at the two Update functions:

        public
            bool Update (int index, IMessage msg) {
IMessage _tmp =
            Peek(index, false);
bool _ret = false;

try {
if (_tmp
            != null && (_tmp.get_Id()).Equals(msg.get_Id()))
            {
m_messages[index] = msg;

StreamWriter _streamW =
            File.CreateText(m_storage + "\\" +
            msg.get_Id());
msg.Store(_streamW);
_streamW.Close();

m_notifiers[(int)SimpleServerEnums.NotificationTypeEnum.NT_Update](msg);

_ret
            = true;
}
} catch (Exception e)
            {
System.Console.WriteLine(e);
}

return
            _ret;
}

public bool Update (Guid id, IMessage message)
            {
int _index = -1;

_index = FindIndexOf(id);
if
            (_index >= 0)
return Update(_index,
            message);
else
return false;
}

The Update functions modify the content of the specified message. The message to be updated is specified by index or by identifier. I added an extra check to be sure that a malicious client won't be able to corrupt an existing message. Before updating the message, the function checks if the messages have the same identifier. Although this could be considered very strict security, it is nothing like that. A client can create a message object that implements IMessage interface and pass it to the framework. This is not considered a security hole because the idea of the framework is to make is as open as possible and to allow as much custom development as possible.

Finally the last functions in the message related category is CreateMessage :

        public
            IMessage CreateMessage(string body) {
return (new
            Message(body));
}

public IMessage CreateMessage()
            {
return (new Message());
}

Clients that need to create messages use those functions. The function will delegate the call to the factory's Message object constructor.

Custom operations related

Another feature of my framework allows a client to create a thread that will perform a custom operation on messages in the queue. As you'll see later in the sample application, this is very interesting.

A client calls the CreateCustomThread function to create a custom operation. The parameters of this function are:

  • Message filter is a client defined object that implements a QueueMessageFilter interface which has the purpose of filtering messages that meet a certain criteria.
  • Timer interval is the frequency with which the server will call this custom operation.
  • For client notification the framework uses a delegate of the same type as for other queue notifications: new messages, message updated, message removed.

        public
            Guid CreateCustomThread (QueueMessageFilter filter,
int
            interval, QueueNotification notifyThis) {

if
            (m_clientThreadsObj == null) {
m_clientThreadsObj = new
            Hashtable(10);
}

ClientOperationThread _tmp = new
            ClientOperationThread();
Guid _id = _tmp.Create(this, interval,
            filter,
            notifyThis);

_tmp.Start();
m_clientThreadsObj.Add(_id,
            _tmp);

return _id;
}

The queue contains a list with the custom operations created by clients. To be uniquely identified, each custom operation is associated with an identifier at creation time. ClientOperationThread is the class that internally implements the custom operation. We'll take a look at this class shortly.

It is possible for a client to pause and resume a custom operation by calling the PauseCustomThread and ResumeCustomThread functions respectively.

        public
            void PauseCustomThread (Guid id)
            {
((ClientOperationThread)m_clientThreadsObj[id]).Pause();
}

public
            void ResumeCustomThread (Guid id)
            {
((ClientOperationThread)m_clientThreadsObj[id]).Resume();
}

Let's take a look at the ClientOperationThread class. I use a timer member variable to control the frequency with which the custom operation is executed. The timer is created inside the Create function. The .NET Framework provides a very simple way of dealing with timers. Each time the timer interval expires, the timer object will call a delegate function of type ElapsedEventHandler .

I took advantage of this feature, and inside the internalElapsedHandler function I call the client delegate to notify the client of this event.

    class
            ClientOperationThread {
private int m_timerInterval;
private
            System.Timers.Timer m_timer = null;
private Queue m_queue =
            null;
private QueueMessageFilter m_filter;
private
            QueueNotification m_notify;
private Guid
            m_guid;

~ClientOperationThread() {
if (m_timer != null)
            {
m_timer.Enabled = false;
m_timer = null;
}
}

The destructor contains the code to release the timer resource used. Using the destructor is a better choice than the Finalize method to release critical resources like window, files, operating system objects or network connections because the destructor is called when the program exits or the object is destroyed. The Finalize method is implicitly called by the destructor.

public Guid Create (Queue queue, int timerInterval,
QueueMessageFilter filterObject, QueueNotification notifyThis) {

m_guid = Guid.NewGuid();
m_queue = queue;
m_timerInterval = timerInterval;
m_filter = filterObject;
m_notify += notifyThis;

return m_guid;
}

The client notification is possible using the same delegate function used for subscribers, QueueNotification .

        public
            Guid id() {
return m_guid;
}

private void
            internalElapsedHandler (object sender, ElapsedEventArgs args)
            {
ArrayList msgs = m_queue.Messages;
IMessage msg =
            null;
int i =
            0;

Console.WriteLine("internalElapsedHandler()");

while
            (i < m_queue.Messages.Count) {
msg =
            (IMessage)m_queue.Messages[i];

if (m_filter.Filter(msg))
            {
try {
m_notify(msg);

} catch (Exception e)
            {
Console.WriteLine(e);
}
}
i++;
}
}

On the timer's elapse event, the framework will probe each message with the filter provided by the client and if it meets the criteria, the notification delegate is called.

The class implements a few timer manipulation functions like: Start, Pause and Resume. The purpose of those functions is obvious so I won't get into details.

        public
            void Start() {
if (m_timer == null) {
m_timer = new
            System.Timers.Timer (m_timerInterval);
m_timer.Elapsed += new
            ElapsedEventHandler (internalElapsedHandler);
m_timer.Enabled =
            true;
m_timer.AutoReset =
            true;

Console.WriteLine("ClientOperation thread
            started");
}
else
Console.WriteLine("Already
            started.");
}

public void Pause() {
if (m_timer !=
            null)
m_timer.Enabled = false;
}

public void Resume()
            {
if (m_timer != null)
m_timer.Enabled =
            true;
}
}

Server

Next, let's see how the server object is implemented. The Server object derives from MarshalByRefObject because it will be published through .NET Remoting on a TCP channel. An instance of the Server object will be published as a singleton. When an object is published as a singleton means than all clients will get the same instance of the object. This is important in the context of our design because all clients need to communicate with the same object. If each client connected to a different instance of the server there would be many instances of the same queue and they would not intercommunicate.

public class Server : MarshalByRefObject {
private Hashtable m_queues = null;
private NotificationServerObjectFactory m_factory = null;

private string m_storage, m_infoFile;

The constructor initializes some variables like storage location and object factory. These values should be collected from a configuration file, but for simplicity I hard coded default values.

The Server maintains a hash table with all the queues that were created by clients. Also, the server maintains a file with information on the queues. When the server starts it reads queue information from the file and creates the hash table along with the queues.

        public
            Server () {
m_queues = new Hashtable();

// take the
            storage location from the config file
// TO BE IMPLEMENTED
            LATER
m_storage = Directory.GetCurrentDirectory() +
            "\\";
m_infoFile = "queues.info";

// take the factory
            class from the settings file
// TO BE IMPLEMENTED
            LATER
m_factory = new
            NFCSharpFactory();

LoadQueuesInfo();
}

The operations that are available on the Server object are create queue, delete queue and get queue; all these operations are based on a parameter representing queue's name.

LoadQueuesInfo is the function that loads queues information from the file. As you already saw, at the time a Queue object is created it will load messages stored on disk. Here is the source code for LoadQueuesInfo :

        private
            bool LoadQueuesInfo() {
string _line;
string[]
            _queueInfo;
bool _ret = true;

try {
StreamReader
            _streamR = File.OpenText(m_storage + m_infoFile);
_line =
            _streamR.ReadLine();

while (_line != null) {
_queueInfo =
            _line.Split(new char[] {'|'});

m_queues.Add(_queueInfo[0],
            m_factory.CreateQueueObject(m_factory,
_queueInfo[0], m_storage,
            _queueInfo[1]));

_line =
            _streamR.ReadLine();
}

_streamR.Close();

} catch
            (Exception e) {
Console.WriteLine(e);
_ret =
            false;
}

return _ret;
}

CreateQueue method is called from client applications when a new queue needs to be created. The function takes one parameter, queue name, and creates a new Queue object by delegating the request to the factory object. The newly created Queue object it is than added to the list of queues managed by the server. A folder corresponding to the new queue is also created to disk for message storage.

        public
            bool createQueue (string queueName) {
if (m_queues.Contains
            (queueName) == false) {

m_queues.Add(queueName,
m_factory.CreateQueueObject(m_factory, queueName, m_storage,
            null));

Console.Write("Update queues.info...");

try
            {
IQueue _queue = (IQueue)m_queues[queueName];

if (_queue
            == null) Console.WriteLine("queue is null");

Guid _id =
            _queue.Id();

string _infoFile = m_storage +
            m_infoFile;

Console.WriteLine(_infoFile);

StreamWriter
            _streamW =
            File.AppendText(_infoFile);

_streamW.Write(queueName + "|" +
            _id.ToString() + "\r\n");
_streamW.Close();

} catch
            (Exception e)
            {
Console.WriteLine(e);

m_queues.Remove(queueName);
return
            false;
}

Console.WriteLine ("Added Queue : {0}",
            queueName);
}

return true;
}

The getQueue function retrieves the queue object from the list of queues managed by the server. The parameter supplied represents the name of the queue. If the queue does not exist, null is returned.

        public
            IQueue getQueue (string queueName) {
if
            (m_queues.Contains(queueName))
return
            (IQueue)m_queues[queueName];
else
return
            null;
}

public Hashtable getAllQueues () {
return
            m_queues;
}
}

So far, we have analyzed the implementation of this simple notification framework. In the following sections I'll show you a sample application, which use the framework.

Building a sample

My sample contains few applications; the interaction between these applications is presented in the following diagram.

The EndUserClient application is connected to the IVR application, and based on end user's answers posts a message to a queue called Input . The sample application does not contain the IVR code because of a special hardware required. In order to run the IVR system you need a Dialogic telephony board and the additional drivers and software provided by the manufacturer. The Dialogic board would be connected to the phone line and the IVR software interacts with the board to manage incoming and outgoing calls. To replace this special assembly we provided the EndUserCleint application which will simulate the flow of messages from the IVR system.

Managers monitor the activity in the Input queue using the MngrClient application. The main purpose of this application is to catch messages that stay in the queue more than a specified amount of time.

The CSRClient application allows customer representatives to pick up messages from the queue and register the complaint. In the real world application, when a CSR picks up a message from the queue special software will establish the phone connection between the end user who's been put on hold and the CSR. My simplified sample will print a message that the message has been processed and removed from the queue.

StatsClient is an application that gathers and displays statistics about current session of the framework.

I will not insert the code for each sample application here. as I try to keep the article at a reasonable length. Instead I'll describe the scenario you can follow in testing the application.

The source code contains complete source codes of the framework along with sample applications. I also included the storage of a queue. The messages in this queue are used in the sample application.

To build the sample applications unzip the source code, open a command window and type:

build all

The result of this operation is the creation of the following files: CSRClient.exe , EndUserClient.exe , NFServer.exe , NFCommon.dll , NFCSharpFactory.dll , StatsClient.exe and Manager.exe .

To start the test scenario, open a command window and change the current folder to the location where the framework was compiled.

In the first window start the server with the following command:

NFServer

This will start the notification framework server.

If you take a look at the source code you notice a lot of trace messages. You would probably expect that at this moment to see the messages that show you that the server has found a queue and is loading all the messages available in the storage. But, because our server is exposed as a Remoting singleton with server instantiation, the object is not created yet because no client needs it yet.

In the second window start the stats application with following command:

StatsClient

This command will start the statistic gathering client application. To complete the client startup, type C . You'll immediately notice that in the server window appear trace messages showing that the queue has been created and messages loaded from storage.

If you move to the StatsClient window and type S the result should be

Enter operation:S
Messages in queue=10
Messages processed=0
Menu
C - Connect to server
S - Display statistics
Q - Stop the server
Enter operation:

This shows us that the StatsClient subscriber has been notified that 10 messages are in the queue, and 0 messages had been processed by the CSRClient application. The number of messages in the queue should be equal to the number of files in the storage. If you choose to delete from the original messages shipped with the article this number will vary.

Open a third command window and type EndUserClient. This will start the EndUserClient application and display the options menu. Type C to connect to the server.

Open a new command window and type ManagerClient . This will start the manager application, which will monitor the queue for aged messages. Type C to connect to the server and wait for 2 seconds. The resulting screen shows that the manager app has been notified with all messages that have been in queue more that 5 seconds. Since we started the server more that 5 seconds ago, all messages will be reported as aged.

If you switch back to the server window you'll notice that the custom thread continues to scan for aged messages. Until new messages are inserted in the queue nothing will be reported as aged.

To pause the custom thread you can go back to the Manager app and type P for pause, and R for resume.

From the EndUser window you can post new messages to the queue, and then notice that after 5 seconds the manager will be notified. This will happen in case you don't process the message.

Open a new command window and type CSRClient . This will start the CSR application, which can process messages in the queue. After typing C to connect to the server you will notice that the server will notify all messages available in queue.

To process a message type P and then choose a message from the queue, 5 for example. After you process this message, you will see that a notification message has been sent to interested parties that the message was deleted from the queue.

Move to the stats window and type S, you can notice that the number of messages processed was increased .

This is very simple application that shows how the framework is used for developing messaging solutions.

Conclusion

This article presented a framework, which is an alternative to existing messaging solutions. Without the intention of replacing more complex existing solutions, this framework will help you to solve problems where existing apps don't work, or where a more cost effective solution is needed. A number of technical topics present in the .NET Framework were brought into discussion here, like: remoting and channels, file management, system timers and collections. As always, we encourage you to study those topics in details and the MSDN Library documentation is a good start.

Of course there are improvements that can be done to this solution, such as:

  • Change the persistence mechanism to use a database like SQL Server or any other RDBMS.
  • Create a GUI for the server to easily manage the queues on the server side. This would be a handy tool for a server administrator to check the status of the server, manage queues and monitor the activity of connected users.
  • Add some sort of authentication mechanism, which can include classes of users, permissions for executing different operations on queues. Maybe even extend the server with types of queues that would be associated to classes of users.

Sending message notification between applications is an important part of software development, which could improve the response of an application or make a solution more robust and scalable to large number of users. Companies like Microsoft strive to develop new notification framework like the up-coming .NET Notification Server currently used by MSN Messenger. Java Message Services is another messaging specification, which is implemented by products like iPlanet Message Server or Oracle.

Article Information
Author | Catalin Tomescu
Technical Editor | Adam Ryland
Author Agent | Charlotte Smith
Project Manager | Helen Cuthill
Reviewers | Andrew Krowczyk, Richard Conway

If you have any questions or comments about this article, please contact the technical editor.


Monitoring the File System for Tampering using the FileSystemWatcher Class

Monitoring the File System for Tampering using the FileSystemWatcher Class
http://www.csharptoday.com/content.asp?id=1805
Article Rating: 3
Published on January 23, 2002
Abstract
If anyone is experimenting with an important directory on our system without our permission, it would be good to know. An application that monitors the directory structure and logs any changes would be helpful. This type of application would have been difficult to write before, so hats off to Microsoft and their FileSystemWatcher class. In this article, Amhed Mokhtar introduces the FileSystemWatcher class and provides a sample application to help us monitor any specified directory.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

If anyone is experimenting with an important directory on our system without our permission, it would be good to know - an application that monitors the directory structure and logs any changes would be helpful. This type of application would have been difficult to write before, so hats off to Microsoft and their FileSystemWatcher class. In this article, we talk about the FileSystemWatcher class, and create a sample application to help us monitor any specified directory.

Note that the FileSystemWatcher watches for changes within a directory, not to changes to the directory's attributes themselves. We can also keep watch on remote or networked machines, as well as our local machine.

Requirements

The FileSystemWatcher only works on Windows NT 4.0 and Windows 2000. Note that we cannot watch a remote Windows NT 4.0 computer from a Windows NT 4.0 computer, but we can watch a remote Win2000 computer. We can watch either from a Win2000 computer.

For the code in this article we need:

  • Windows NT 4.0 or Windows 2000
  • .NET Framework Beta 2
  • A text editor, such as Notepad, Text pad or Visual Studio .NET Beta2

Configuring FileSystemWatcher

To configure an instance of the FileSystemWatcher component:

  • Create an instance of the FileSystemWatcher component.
  • Configure the necessary properties and methods for the component.
  • Create Handlers for FileSystem Events.

Creating FileSystemWatcher Component Instances

To create an instance of the FileSystemWatcher component, use one of the following:

//Intializes a new instance of the
            FileSystemWatcher Class
FileSystemWatcher myWatcher = new
            FileSystemWatcher();

//Intializes a new instance with a Path
            property
FileSystemWatcher myWatcher = new
            FileSystemWatcher("c:\\");

//Intializes a new instance along
            with a Path and Filter properties
FileSystemWatcher myWatcher =
            new FileSystemWatcher("c:\\","*.txt");

The component will not watch the specified directory until the Path is set, and EnableRaisingEvents is true.

If you are using Visual Studio .NET, the FileSystemWatcher is readily available in the Toolbox under the Components Tab, and you can directly drag and drop the FileSystemWatcher class; EnableRaisingEvents is true by default.

Properties

There are several properties to set for a FileSystemWatcher component instances to determine how they behave. These properties determine what directories and subdirectories the component instance will monitor and the exact occurrences within those directories that will raise events.

  • Path - Sets or gets the path of the directory to watch.
  • EnableRaisingEvents - Enable or disable the component.
  • Filter - Gets or sets the filter string, used to determine what files are monitored in a directory. Default value is (*.*)
  • IncludeSubdirectories - Enables or disables including subdirectories to be watched.
  • InternalBufferSize - Gets or sets the size of internal buffer. Default is 8K.
  • NotifyFilter - Gets or sets the type of changes to watch for.

Specifying Directories to Watch

To specify the directories to be watched, the Path and IncludeSubdirectories properties are used.

The Path property indicates the fully qualified path of the root directory you want to watch. This can be in standard directory notation (c:\directory) or in UNC format (file://server//directory?WROXEMPTOKEN=620568Zhiox25QOrrsnTPfm53p).

FileSystemWatcher myWatcher = new
            FileSystemWatcher();
MyWatcher.Path = "c:\\";

The IncludeSubdirectories property indicates whether subdirectories within the root directory should be monitored. If the property is set to true, the component watches for the same changes in the subdirectories as it does in the main directory the component is watching.

Specifying the Changes to Watch

Set the Filter property to an empty string ("") to watch for changes in all files. To watch a specific file, set the Filter property to the file name - say " samp.txt ". We can also watch for changes in a certain type of file. For example, to watch for changes in document files, set the Filter property to " *.doc ".

The FileSystemWatcher class does not ignore hidden files. Setting the Filter does not decrease what goes into the buffer.

By setting the NotifyFilter property to one of the NotifyFilters values, we can track several types of changes such as changes in Attributes, the LastWrite date and time, the Size of files or directories, or when security access to a file or directory rights change. These values are all part of the NotifyFilters enumeration. We can set multiple changes to watch for by using "|" operator.

NotifyFilters enumeration

Name | Description
Attributes | The attributes of the file or folder.
CreationTime | The time the file or folder was created.
DirectoryName | The name of the directory.
FileName | The name of the file.
LastAccess | The date the file or folder was last opened.
LastWrite | The date the file or folder last had anything written to it.
Security | The security settings of the file or folder.
Size | The size of the file or folder.

Example:

FileSystemWatcher myWatcher = new
            FileSystemWatcher();
myWatcher.Path = "c:\\";
myWatcher.Filter
            = "*.txt";
myWatcher.EnableRaisingEvents =
            true;
myWatcher.IncludeSubdirectories =
            false;
myWatcher.NotifyFilter = NotifyFilters.LastWrite |
            NotifyFilters.Size;

Our FileSystemWatcher component has the potential to receive an enormous number of events, particularly if we have set it to watch a very high traffic directory. This can cause problems, as the FileSystemWatcher can only receive event notifications at a manageable rate. The system notifies the component of file changes, it stores those changes in a buffer the component creates, and passes it to the Win32 Application Programming Interfaces (APIs). If there are many changes in a short time, the buffer can overflow. This causes the component to lose track of changes in the directory, and it will only provide " blanket " notification and the component will raise an exception. The default size of the buffer is 4KB, but we can increase this using InternalBufferSize. Note that increasing the size of the buffer is expensive, as it comes from non-paged memory that cannot be swapped out to disk, so keep the buffer as small as possible. A 4KB buffer can track changes on approximately 80 files in a directory. Each event takes up 16 bytes in the buffer, plus enough bytes to store the name of the file, in Unicode (2 bytes per character), that the event occurred on. We can use this information to approximate the buffer size you will need.

To avoid a buffer overflow, use the NotifyFilter and IncludeSubdirectories properties to filter out unwanted change notifications.

Methods

In addition to using the FileSystemWatcher component to immediately monitor a specific directory, we can use the WaitForChanged method to wait until a specific event occurs, and then continue with the execution of the thread. For example, if we are working with a Web based news application, we might create an admin portion of the site where users upload their news stories. We could use the WaitForChanged method to watch that directory until the last access date changes, and then begin processing the news directory for new articles.

We specify the type of change to watch for by setting the value of a WatcherChangeType enumeration. The possible values are as follows:

Member Name | Description
All | The creation, deletion, change, or renaming of a file or folder.
Changed | The change of a file or folder. The types of changes include: changes to size, attributes, security settings, last write, and last access time.
Created | The creation of a file or folder.
Deleted | The deletion of a file or folder.
Renamed | The renaming of a file or folder.

WaitForChanged is a synchronous method that returns an object of type WaitForChangedResult. This class contains specific information on the type of change that occurred in the directory. We can access information such as Name, OldName, and TimedOut on this object to find out more about the change. TimeOut  is the time (in milliseconds) to wait before timing out.

If we use it in a Windows Form application, the application would stop responding if the method was used on the UI thread instead of the worker thread.

The following code tracks for all the changes:

myWatcher.WaitForChanged(WatcherChangeType.All);

Alternatively:

//wait for one second before timing out
myWatcher.WaitForChanged(WatcherChangeType.All,1000);

Creating a Handler for FileSystem Events

The FileSystemWatcher raises four events depending on the types of changes that occur in the directory it is watching. These events are:

  • Created - raised whenever a directory or file is created.
  • Deleted - raised whenever a directory or file is deleted.
  • Renamed - raised whenever the name of a directory or file is changed.
  • Changed - raised whenever changes are made to the size, system attributes, last write time, last access time or NTFS security permissions of a directory or file.

To access these events, we have to define handlers that automatically call methods in our code when a change occurs. To create a handler, use the following code:

//Handler for Changed
            Event
myWatcher.Changed += new
            FileSystemHandler(myWatcher_Changed);

//Handler for Created
            Event
myWatcher.Created += new
            FileSystemHandler(myWatcher_Created);

//Handler for Deleted
            Event
myWatcher.Deleted += new
            FileSystemHandler(myWatcher_Deleted);

//Handler for Renamed
            Event
myWatcher.Renamed += new
            RenamedEventHandler(myWatcher_Renamed);

Although some common occurrences, such as copying or moving a file, do not correspond directly to an event, these occurrences do cause events to be raised. When we copy a file, the system raises a Created event in the directory to which the file was copied, but does not raise any events in the original directory. When we move a file, the server raises two events: a Deleted event in the source directory, followed by a Created event in the target directory.

For example, we create two instances of FileSystemWatcher. FileSystemWatcher1 is set to watch " C:\My Documents ", and FileSystemWatcher2 is set to watch " C:\Your Documents ". Now, if we copy a file from " My Documents " into " Your Documents ", a Created event will be raised by FileSystemWatcher2, but no event is raised for FileSystemWatcher1. Unlike copying, moving a file or directory would raise two events. From the previous example, if we moved a file from " My Documents " to " Your Documents ", a Created event would be raised by FileSystemWatcher2, and a Deleted event would be raised by FileSystemWatcher1.

Application Overview

To show how FileSystemWatcher tracks the specified directory, I have provided a simple example application, which shows the status of our directory contents in the textbox whenever a change is made. This application has six class files, all inherited from System.Windows.Forms.

  • Properties.cs - to set the Directory Path that is going to be watched.
  • MonitorFS.cs - Main class file for watching changes to a specified directory. Also allows us to create and drop files and directories.
  • CreateDir.cs - to create a new directory depending on security permissions.
  • CreateFile.cs - to create a new file depending on security permissions.
  • DeleteDir.cs - to delete a particular directory depending on security permissions.
  • DeleteFile.cs - to delete a particular file depending on security permissions.

User Interface

Setting a Path for FileSystemWatcher Component

One very important task is to set the directory path to be watched. This can be done through the startup form. The following screenshots shows how to do this:

We can set the path in two ways - either in standard direction notation ("d:\"), or in Universal Naming Convention (UNC) format ("\\192.168.1.6\d\" or "\\mokhtar\d\").

In the above form, set the valid path, then clicked the Set Path button. To check whether the given path is valid or not, we use the Exists method of the Directory class. As the Directory class is a static one, we do not need to instantiated it.

Private string dirPath = txtPath.Text;
--
--
if (Directory.Exists(dirPath))
- - -

If a directory exists, then the directory path will be set to the myPath property, which will be useful later while instantiating the FileSystemWatcher:

private string mPath;
- -
- -
- -
//property for path
public string myPath
{
get
{
return mPath;
}
set
{
mPath = value;
}
}
- - -
- - -
protected void btnSetPath_Click(object Sender, EventArgs e)
{
---
this.myPath = dirPath;

Monitoring the File System

To monitor the File System, we first have to instantiate the FileSystemWatcher class, then we have to set necessary properties. To instantiate and set the properties of the class, we use the following code:

public class MonitorFS:Form
{
private FileSystemWatcher myWatcher;
---
---
public MonitorFS(string myPath)
{
//Initializing directory path from properties.cs file
DirPath = myPath;
--
---
myWatcher = new FileSystemWatcher(DirPath);
myWatcher.EnableRaisingEvents = true;
myWatcher.IncludeSubdirectories = true;
--
---
--
}
}

Next, we have to create event handlers of type FileSystemEventHandler for Created, Changed and Deleted events, and RenamedEventHandler for Renamed events. These event handlers will call the appropriate procedure when an entry is written to the log.

Event Handlers of Type FileSystemEventHandler

myWatcher.Created += new FileSystemEventHandler(myWatcher_Created);
myWatcher.Changed += new FileSystemEventHandler(myWatcher_Changed);
myWatcher.Deleted += new FileSystemEventHandler(myWatcher_Deleted);

Event Handler of Type RenamedEventHandler

myWatcher.Renamed += new RenamedEventHandler(myWatcher_Renamed);

Procedure for Created Event

protected void myWatcher_Created(object sender, FileSystemEventArgs e)
{
txtInfo.Text += "ChangeType :: " + e.ChangeType.ToString() + "\nFullPath ::"
+ e.FullPath.ToString() + "\n\n";
}

Procedure for Changed Event

protected void myWatcher_Changed(object sender, FileSystemEventArgs e)
{
txtInfo.Text += "ChangeType :: " + e.ChangeType.ToString() + "\nFullPath ::"
+ e.FullPath.ToString() + "\n\n";
}

Procedure for Deleted Event

protected void myWatcher_Deleted(object sender, FileSystemEventArgs e)
{
txtInfo.Text += "ChangeType :: " + e.ChangeType.ToString() + "\nFullPath ::" +
e.FullPath.ToString() + "\n\n";
}

All the above events have common arguments, i.e., the FileSystemEventArgs that provides the following properties:

Property | Description
ChangeType | Gets the type of directory event that occurred.
FullPath | Gets the fully-qualified path of the affected file or directory
Name | Gets the name of the affected file or directory.

Procedure for Renamed Event

protected void myWatcher_Renamed(object sender, RenamedEventArgs e)
{
txtInfo.Text += "ChangeType :: " + e.ChangeType.ToString() + "\nFullPath ::"
+ e.FullPath.ToString() + "\nOld FileName :: "+ e.OldName.ToString() + "\n\n";
}

The RenamedEventArgs provides two more properties in addition to FileSystemEventArgs:

Property | Description
OldFullPath | Gets the previous fully-qualified path of the affected file or directory.
OldName | Gets the old name of the affected file or directory.

We can monitor our File System in the following ways:

  • Outside the application through creating, dropping and renaming files and directories.
  • From the application through creating and dropping files & directories.

Making Changes Outside the Application

Watching Changes on Local Directories

The following screenshots show how the FileSystemWatcher watches changes when a file/directory is created under the root directory of the f:\ and d:\ drives of local system 192.168.1.3. Two events are raised (Created and Renamed) when we create a directory from the outside of the application.

In the next screenshot, we can see the changes when a word document is created. The events called when a new document is created are Created, Changed and Renamed.

Watching Changes on Network Directories

We can also watch changes on network directories. The following screenshot shows how we monitor a network drive. When we create a directory on a network directory, three events are raised: Created (twice), Deleted and Renamed.

Making Changes within the Application

You can watch the changes from the application straight away by creating and dropping files and directories.

The following screenshots show how we create directory:

The above form uses the CreateDir class file to create a directory from the application:

public class CreateDir:Form
{
--
--
public CreateDir()
{
----
---
//Event Handler for Calling btnAddDir procedure
btnAddDir.Click += new EventHandler(btnAddDir_Click);
----
---
}

//Event for Creating Directory
protected void btnAddDir_Click(object Sender, EventArgs e)
{
//Getting path from mypath property
string dir = myPath + "\\" + txtDir.Text;
if(txtDir.Text != "" )
{
//Checks for directory
if (!Directory.Exists(dir))
{
try
{
Directory.CreateDirectory(dir);
this.Hide();
}
//Exception raises if security violated
catch(Exception CrDir)
{
MessageBox.Show(CrDir.Message,"Error Creating Directory",MessageBoxButtons.OK,
MessageBoxIcon.Exclamation);
this.Hide();
}
}
else
{
MessageBox.Show(dir + " Directory already Exists","Directory Creation Error",
MessageBoxButtons.OK,MessageBoxIcon.Stop);
txtDir.Focus();
SendKeys.Send("{HOME}+{END}");
}
}
else
{
MessageBox.Show("Please Enter Directory Name","Directory Creation Error",
MessageBoxButtons.OK,MessageBoxIcon.Information);
txtDir.Focus();
}

}

After creating the directory, the created event will be raised. We can see the effect in the following screenshots:

The other options available are Create New File, Delete File and Delete Directory. This allows us to watch the changes straight away from the application itself.

If we don't have appropriate permissions then exceptions will be raised. There are standard permissions for accessing files and directories - if we breach them, then errors will be raised through exception handling.

Conclusion

As we have seen, the FileSystemWatcher class is great at watching any kind of changes on a specified directory. Using this class, we can write numerous applications, such as mailing our administrator whenever our directory is tampered with.

Article Information
Author | Ahmed Mokhtar
Technical Editor | Adam Ryland
Author Agent | Charlotte Smith
Project Manager | Helen Cuthill
Reviewers | Dave Schultz, Vikram

If you have any questions or comments about this article, please contact the technical editor.


Customized Context Menus for your C# Forms

Customized Context Menus for your C# Forms
http://www.csharptoday.com/content.asp?id=1804
Article Rating: 4.5
Published on January 22, 2002
Abstract
Context menus, also known as popup menus, have become a standard feature on Windows Forms. You use popup menus for displaying a menu to the user whenever the user clicks the right mouse button on a control or on a form. The .NET Framework class library defines a class called ContextMenu to facilitate the creation of such popup menus. Depending on the application’s current context, you may display a different list of menu items to the user using the ContextMenu class. In this article, Dr. P.G. Sarang explores this class and shows how we can create a customized context menu.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

The context menus, also known as popup menus have become a standard feature on Windows Forms. You use popup menus for displaying a menu to the user whenever the user clicks the right mouse button on a control or on a form. A typical context menu, as seen in Microsoft Word, is shown in the screen shot below:

The .NET Framework class library defines a class called ContextMenu to facilitate the creation of such popup menus. Depending on the application's current context, you may display a different list of menu items to the user using the ContextMenu class. The object of a ContextMenu class can be shared by different controls on the form, thereby providing a single event handler method for the various controls. This helps in producing more robust code and eases code maintenance. All these techniques may be combined to provide the customization for any Windows Form whereby the application developer or the user can decide which menu items to display for the various controls of the form. The configuration of menu items may be stored to a disk file and retrieved on Form load. Once the configuration is loaded, it may be used for creating the menus dynamically.

In this article, you will learn many techniques of using Context Menus in your Windows applications. You will learn the following:

  • Creating Context Menus using ContextMenu class.
  • Creating Menus that vary depending on the application's current context.
  • Sharing a menu handler between different controls.
  • Customizing a menu system - Such customization may be done by the application user or the Form developer. The customization information is stored in an XML file.

Creating a popup menu

A popup menu for your windows form may be created programmatically or through the design tools.

Creating menus using design tools

Creating popup menus at design time is easy, and the process is quite similar to designing menus in the earlier versions of Visual Studio. To create menus at the design time, you add the ContextMenu component to your form in the design view. This is shown in the following screen shot:

You will add the menu items to it with the help of the visual menu editor, shown in the screen shot below:

Once the menu is constructed, it is attached to the form or a desired control by setting the controls' ContextMenu property.

Creating menus programmatically

I shall now discuss the creation of a context menu programmatically. This gives you more flexibility in menu design in terms of allowing you to construct the menu system or change the current menu system at the runtime depending on the application context.

Creating The Project

In Visual Studio .Net, create a Windows Application project in C#. The wizard creates a default form for your application and opens it in design view. We will add a popup menu to this form. The menu is displayed whenever the user right-clicks on the form.

The entire project source for this application is available in the downloadable MenuTest application.

Adding BackColor menu to Windows Form

Whenever the user right clicks on the application Form, it will display a menu item called BackColor. When the user clicks on this menu item, the background color of the form will change to some random color.

To create a popup menu programmatically, you first need to instantiate the ContextMenu class:

System.Windows.Forms.ContextMenu mnuContextMenu = new ContextMenu();

Next, we create an instance of the MenuItem class and add it to the context menu. The following code creates a menu item with the caption BackColor and adds it to the mnuContextMenu :

   MenuItem mnuColor = new
            MenuItem("BackColor");
mnuContextMenu.MenuItems.Add(mnuColor);

The ContextMenu class maintains a list of menu items in the MenuItems collection. You use the Add method of the Collections to add the desired menu item. If you add multiple menu items, they will be displayed to the user in the order in which they are added to the list.

We need to set the event handler to be called whenever this menu item is clicked. This is done using the following code segment:

mnuColor.Click += new System.EventHandler(ChangeBackColor);

The ChangeBackColor is the event handling method (delegate) that is called whenever the user clicks the BackColor menu item. The ChangeBackColor method definition is shown below:

private void ChangeBackColor(object sender, System.EventArgs e)
{
this.BackgroundImage = null;
int r,g,b;
r = this.BackColor.R;
g = this.BackColor.G;
b = this.BackColor.B;

      r =
            ((r+1)*91) %255;
g = ((g+1)*91) %255;
b = ((b+1)*91)
            %255;

this.BackColor = Color.FromArgb(r,g,b);
}

The above method has the signature specified for the event handler. The method sets the background image of the component to null before setting the BackColor property. This is required, as setting the BackColor property does not clear the existing background of the form. The color itself is generated programmatically, and will vary with each invocation.

Finally, we need to assign the created context menu to some control. We will assign this to the current instance of the Form object:

this.ContextMenu = mnuContextMenu;

Now, if you right click on the form the popup menu will appear showing the single menu item BackColor. If you click on this menu item, the Form background color will change. If you do this one more time, the background color will change to some other color. Note that a different color value is generated by the program logic on every click of the menu item.

The application code is available in the downloadable MenuTestForm.cs file.

Adding BackgroundImage menu

There is another, shorter way of adding a menu item and attaching an event handler to it at the same time. To illustrate this code, we will now add the BackImage menu item to the earlier code. The click on this menu item will change the background image of the form to a new value.

We first construct an array for storing the names of images files to be used as background images:

   images = new
            String[3];
images[0] = Environment.CurrentDirectory +
            "\\FlyAway.jpg";
images[1]= Environment.CurrentDirectory +
            "\\GoldPetals.jpg";
images[2]= Environment.CurrentDirectory +
            "\\Paradise.jpg";

The BackImage menu item is then created and added to the context menu by calling the overloaded Add method of the MenuItemsCollection class:

mnuContextMenu.MenuItems.Add("BackImage",
new System.EventHandler(ChangeBackImage));

Note that the above line also adds the event handler for the BackImage menu item. The code for the event handler is given below:

private void ChangeBackImage(object sender, System.EventArgs e)
{
imageIndex = (imageIndex+1) %3;
this.BackgroundImage =
System.Drawing.Image.FromFile(images[imageIndex]);
}

The background image of the form is changed by setting is BackgroundImage property. The image itself is loaded from an external file. The filename is taken from images array.

Likewise, we will add one more menu item to our context menu for changing the cursor displayed on the form.

Adding Cursor menu item

First, we declare the cursor array and set the elements to the three standard cursors defined in .NET libraries:

myCursor = new System.Windows.Forms.Cursor[3];
myCursor[0] = System.Windows.Forms.Cursors.Hand;
myCursor[1] = System.Windows.Forms.Cursors.Cross;
myCursor[2] = System.Windows.Forms.Cursors.Default;

The new menu item called Cursor is added to the context menu and the event handler is set to the method called ChangeCursor.

mnuContextMenu.MenuItems.Add("Cursor",
new System.EventHandler(ChangeCursor));

The ChangeCursor method is shown below. Setting the Form's Cursor property changes the cursor shape.

   private void
            ChangeCursor(object sender, System.EventArgs e)
{
cursorIndex
            = (cursorIndex+1) %3;
this.Cursor = myCursor[cursorIndex];
}

The application code is available in the downloadable MenuTestForm.cs file.

Output

When you run the program and right click on the Form, a popup menu will appear as shown in the screen shot below. Selecting the BackColor menu item changes the background color to a random value. Selecting the BackImage menu item sets the background image of the form to a different image on every click. Similarly, clicking on the Cursor menu item changes the Form Cursor to a different shape on every click.

I shall now discuss the technique of sharing a single event handler between the various controls.

Sharing event handlers

It is possible to share a single menu handler between several components. A shared menu handler gives you the advantage of ease of code maintenance and avoiding code replication in several places. For example, you may wish to add a BackColor menu item in the popup menu of several controls. The menu will set the background color of the control. The functionality and the implementation of the menu handler essentially remain the same for all the controls. Thus, we share the same menu handler in such cases. First, we create a context menu as before:

private ContextMenu mnuContextMenu = new ContextMenu();
private MenuItem menuItem = new MenuItem("BackColor");

Next, we add the menu item to the context menu and set the event handler for it:

mnuContextMenu.MenuItems.Add(menuItem);
menuItem.Click += new System.EventHandler(OnBackColor);

The event handler itself is shown below:

private void OnBackColor(object sender, System.EventArgs e)
{
ColorDialog colorDialog = new ColorDialog();
// Keeps the user from selecting a custom color.
colorDialog.AllowFullOpen = true;
// Allows the user to get help. (The default is false.)
colorDialog.ShowHelp = true ;
// Sets the initial color select to the current text color,
// so that if the user cancels out,
// the original color is restored.
colorDialog.Color = mnuContextMenu.SourceControl.BackColor;
colorDialog.ShowDialog();
mnuContextMenu.SourceControl.BackColor = colorDialog.Color; }

The event handler pops up a color dialog and allows the user to select the background color.

If you wish to take different action in the event handler depending on the source of event, you can detect what the source of event is by using SourceControl property of the context menu. For this, we will add few controls on the form as shown in the form design view. I have used the default variable names for all the controls. You may refer the downloadable source file for the control names.

The following shared event handler will set different background colors for different controls.

private void OnBackColor(object sender, System.EventArgs e)
{
if(mnuContextMenu.SourceControl == this)
this.BackColor = Color.Blue;
if(mnuContextMenu.SourceControl == label1)
label1.BackColor = Color.Aqua;
if(mnuContextMenu.SourceControl == checkBox1)
checkBox1.BackColor = Color.AliceBlue;
if(mnuContextMenu.SourceControl == checkBox2)
checkBox2.BackColor = Color.Azure;
if(mnuContextMenu.SourceControl == button1)
button1.BackColor = Color.Magenta;
if(mnuContextMenu.SourceControl == textBox1)
textBox1.BackColor = Color.Orange;
}

The application code for the above example is available in the downloadable file BackColorForm.cs.

The program output is shown below:

I shall now discuss the technique of creating menus that change depending on the current application context.

Creating Context-Sensitive menus

Sometimes the menus that you display to the user are insignificant, depending on the application's current context. For example, in an edit control, the Cut menu item becomes insignificant if the user has not selected any text in the edit box. Depending on the context, you may decide to either disable the ineffective menu items or simply not display them to the user.

In the following example, I will explain the technique of creating context sensitive menus that change depending on the application context. We will create a list box and add a context menu to it. The menu will display a single menu item, called Add that allows the user to add some text to the list box. If the user has added more than one item to the list, we will present a Sort option in the context menu. We will provide a check mark in front of the sort menu option, whenever the user clicks on this option. We will toggle the check mark when the user clicks the sort menu item one more time. Finally, we will add one more menu item called Delete to the context menu. The Delete menu option is added subject to the condition that the user has selected an item in the list.

We create a context menu in the InitializeContextMenu method. The method itself is called in the application's class constructor:

   private void
            InitializeContextMenu()
{
mnuContextMenu = new
            System.Windows.Forms.ContextMenu();
mnuContextMenu.Popup +=
new
            System.EventHandler(this.mnuContextMenu_Popup);
}

We add the popup menu handler to the context menu. We will use this event handler for constructing the menu system dynamically.

The menu is attached to the list box control by setting its ContextMenu property:

listBox1.ContextMenu = mnuContextMenu;

A menu item called Sort is created, and the event handler for it is set in the following code:

mnuSort = new MenuItem ("Sort", new System.EventHandler(OnListSort));

The mnuSort menu item is created only once, so that we can maintain its checked/unchecked state on every invocation of context menu.

In the popup event handler of the context menu, we construct the menus that depend on the user's current context. We first clear the earlier menu items from the list:

mnuContextMenu.MenuItems.Clear();

The statement below adds the Add menu item to the context menu and sets the handler for it:

mnuContextMenu.MenuItems.Add("Add",
new System.EventHandler(OnListAdd));

The Sort menu item is added to the context menu only if the number of items in the list box is more than one:

if (this.listBox1.Items.Count > 1)
mnuContextMenu.MenuItems.Add(mnuSort);

This ensures that the popup menu will not display the Sort option to the user unless, and until, more than one entry exists in the list box. Note that the mnuSort is pre-created so that its checked/unchecked state is retained between menu invocations.

The Delete menu item is added to the context menu only if the user has currently selected an item in the list box.

if (this.listBox1.SelectedItem != null)
mnuContextMenu.MenuItems.Add("Delete",
new System.EventHandler(OnListDelete));

We will now write the respective menu handlers. The menu handler for the Delete menu option simply removes the selected item from the list box. Note that the Delete menu option will not be displayed to the user unless the user has selected some item from the list. Thus, no additional checking is required in the menu handler for the selected item.

private void OnListDelete(object sender, System.EventArgs e)
{
this.listBox1.Items.Remove(this.listBox1.SelectedItem);
}

The Sort menu option toggles the check mark on the menu item and also toggles the Sorted property of the list box:

private void OnListSort(object sender, System.EventArgs e)
{
((MenuItem) sender).Checked = !((MenuItem) sender).Checked;
this.listBox1.Sorted = !(this.listBox1.Sorted);
}

When the Sort is enabled, the entire list box gets sorted. When the Sort is disabled, the items, which will be added, now will be added to the bottom of the list, and the entire list is un-sorted.

The Add menu option requires the user to enter the item string. For this, you will need to create another Form. The Form2 (listing in DialogBox.cs) contains an edit control and OK and CANCEL buttons. It reads the string from the user and copies it in its class variable. Our Add event handler reads this string by calling getString method on Form2 :

   private void
            OnListAdd(object sender, System.EventArgs e)
{
Form2 f = new
            Form2();
f.ShowDialog();
string str = f.getString();
if
            (!(str == null))
this.listBox1.Items.Add(str);
}

When you run this application, the menu items displayed to you will depend on the program's current context. Initially, when there are no items in the list, only the Add menu option is displayed. If the list box contains more than one item, the Sort menu option will appear in the menu list. Clicking the Sort option twice will toggle its state to the original value. If you select an item in the list and popup the context menu, the Delete option will be displayed to you. This shows you how to present different menus to the user depending on the application's current context.

The entire project code is available in the ContextSensitiveMenus folder.

Now, we will combine the various techniques discussed so far to allow the user to customize the entire menu system for the form.

Allowing user customization of menu items

So far we have seen techniques for adding context sensitive menus to windows forms, sharing the event handler between several components, and creating context-sensitive menus. Now, I will show you how to customize the menu items for various controls of the form. Here, the customization means deciding what menu items to be displayed for each control. Such decisions may be made by the application developer or the application user. The application developer at the development time can decide which menu items to be displayed for each control on the form. Alternatively, the application user may decide on the menu items s/he would like to see for each form control. Such information will be stored in an XML-based configuration file. Either in the Form Load method or the Form Constructor, we can read this configuration from the XML file and set the menu system for the Form accordingly.

When the user loads the form, s/he will see only those menu items for each control that were previously selected and stored in the configuration file.

The entire project code is available in the downloadable folder ContextMenuAppl. It is strongly suggested that the reader refers to the code while reading the following section.

Defining ControlConfiguration class

First, we will create a class for storing the names of menu items for each control. The class definition is shown below. The class declares two private variables, the control variable stores the control name, and selectedMethods store the names of all the menu items to be displayed for this control. The class also creates the two properties for manipulating these variables:

using System;

namespace ContextMenuApplication
{
public class ControlConfiguration
{
private string control;
private System.Collections.ArrayList selectedMethods;

public string ControlName
{
set
{
control = value;
}

get
{
return control;
}
}

public System.Collections.ArrayList SelectedMethods
{
set
{
selectedMethods = value;
}
get
{
return selectedMethods;
}
}
}
}

Customization Form

Now, we will discuss the CustomizeForm class that is used for customizing the menu system of the desired form. The screen shot of the form is shown below:

The Controls list box displays the list of the variable names of all the controls on the form to be customized. The Methods list box shows all the available menu handlers for the selected control from the Controls list box. The Selected Methods list box contains the list of selected menu handlers for the specified control. The Add(>>) and Remove(<<) buttons facilitate the menu handler selections. The Save button saves the configuration to the XML file.

Now, let us look at the code for CustomizeForm.

The class constructor receives three parameters:

public CustomizeForm(Form frm, ArrayList menuList,
ArrayList ControlsCollection)
{
InitializeComponent();

this.frm = frm;
this.menuList = menuList;
this.ControlsCollection = ControlsCollection;
InitializeDialog();
}

The first parameter obtains the reference to the form that is to be customized. The second parameter receives the list of menu items for which the form has defined the menu handlers. The third parameter gives the current configuration of the form as an array of ControlConfiguration class objects. The form that allows the customization of its menu items for the various controls calls this dialog box by passing the appropriate parameters to it. We copy the received parameters in the class variables for later use.

In the InitalizeDialog method, we fill the Controls list box by obtaining all the controls from the received form reference:

   private void
            InitializeDialog()
{
IEnumerator list =
            frm.Controls.GetEnumerator();
while
            (list.MoveNext())
{
lstControls.Items.Add
            (((Control)list.Current).Name);
}
if (lstControls.Items.Count
            > 0)
lstControls.SelectedIndex = 0;
}

We set the current selection to the first item in the list box. This triggers the SelectedIndexChanged method shown below:

private void lstControls_SelectedIndexChanged(object sender,
System.EventArgs e)
{
InitMenuList();
InitSelectionLists();
}

This method initializes the Methods list box with all the available menu items. The InitSelectionLists then determines the previously selected menus for the current control and moves the menu items from the Methods list to the Selected Methods list. This completes the dialog initialization process.

The user can now use the Add (>>) and Remove (<<) buttons to move the items between the Methods list and the Selected Methods list. Each time we move an item, we adjust the Configuration Array (ControlsCollection). The Configuration Array is an array of objects of type ControlConfiguration. From the configuration array, we remove the existing entry for the current control, create a new ControlConfiguration object, and add it to the list:

   // Search for existing
            control in the Configuration array
// and delete if
            exists
RemoveExistingElement(lstControls.SelectedItem.ToString());

//
            Create Control Configuration object
ControlConfiguration
            ctrlConfig = new ControlConfiguration();
// Add control
            name
ctrlConfig.ControlName =
            lstControls.SelectedItem.ToString();
// Add selected menu
            names
ArrayList selMenus = new ArrayList();
IEnumerator
            selectedMenus = lstSelectedMenus.Items.GetEnumerator();
while
            (selectedMenus.MoveNext())
{
selMenus.Add(selectedMenus.Current.ToString());
}
ctrlConfig.SelectedMethods
            = selMenus;
// Add created object to the Configuration
            array
ControlsCollection.Add(ctrlConfig);

Saving Configuration to XML

When the user clicks the Save button, we call the WriteXML method to write the current configuration to the XML file. We create an XML file with the same name as the name of the Form:

string xmlFileName = Environment.CurrentDirectory.ToString()
+ "\\" + frm.Name + "ControlConfig.xml";

The program creates an enumerator to iterate through the list of objects in the ControlsCollection array:

IEnumerator list = ControlsCollection.GetEnumerator();

For each array element, the Control Name and its selected methods are written to XML:

   ControlConfiguration
            obj;
while (list.MoveNext())
{
obj =
            (ControlConfiguration)list.Current;
.
xmlTxt.WriteStartElement("control");
xmlTxt.WriteAttributeString("name",obj.ControlName);

IEnumerator
            ilist = obj.SelectedMethods.GetEnumerator();
while
            (ilist.MoveNext())
{
xmlTxt.WriteElementString("method",
ilist.Current.ToString());
}
.
}

At the end we write the end element and close the XML file:

xmlTxt.WriteEndElement();
xmlTxt.Close();

A sample XML file is shown below:

<?xml version="1.0" standalone="yes"?><controls>
<control name="label1">
<method>Font</method>
<method>BackgroundImage</method>
<method>BackColor</method>
</control>

<control name="listBox2">
<method>Remove</method>
<method>Sort</method>
<method>ForeColor</method>
</control>

<control name="btnCustomize">
<method>Font</method>
<method>BackgroundImage</method>
</control>

<control name="label2">
<method>ForeColor</method>
<method>Font</method>
</control>
</controls>

MainForm - the form to be customized

We now discuss the code for the MainForm for which the menu customization is desired.

We create a class variable for ContextMenu :

private System.Windows.Forms.ContextMenu mnuContextMenu =
new ContextMenu();

In the Form constructor, we add the popup menu handler for the context menu:

mnuContextMenu.Popup +=
new System.EventHandler(this.mnuContextMenu_Popup);

We now set the context menu for all the controls of the Form by using an enumerator as shown below:

IEnumerator list = this.Controls.GetEnumerator();
while (list.MoveNext())
{
((Control)list.Current).ContextMenu = mnuContextMenu;
}

We also construct a private ArrayList and add the menu item names to it for which the current Form has implemented the menu handlers:

   private
            System.Collections.ArrayList menuList = new ArrayList();
.
//
            Add the available methods for the controls in the
            list
menuList.Add("BackColor");
menuList.Add("BackgroundImage");
menuList.Add("ForeColor");
menuList.Add("Sort");
menuList.Add("Add");
menuList.Add("Remove");
menuList.Add("Font");

Finally, the previously stored configuration, if any, is read using the ReadXMLConfiguration method:

// Read previously stored XML configuration
ReadXMLConfiguration();

Reading XML Configuration

The ReadXMLConfiuration method opens the XML file from the control folder for reading:

string xmlFileName = Environment.CurrentDirectory.ToString() +
"\\" + this.Name + "ControlConfig.xml";
.
reader = new XmlTextReader(xmlFileName);

The program parses the XML file and constructs the array of ControlConfiguration objects in a while loop shown below:

   string strControl =
            reader.GetAttribute("name");
while(list.MoveNext())
{
if(((Control)list.Current).Name.Equals(strControl))
{
obj
            = new ControlConfiguration();
obj.ControlName =
            strControl;
ArrayList ilist = new
            ArrayList();
while(reader.Read())
{
if
            (reader.IsStartElement("method"))
{
ilist.Add(reader.ReadString());
}
else
{
break;
}
}
obj.SelectedMethods=ilist;
ControlsCollection.Add(obj);
}
}

Setting Menus

Once the ControlsCollection is created, we are ready for the user action. Whenever the user right clicks on any of the controls on the Form, the popup event handler will be called. In this event handler, we will now construct the context menu for the clicked control. The menu items to be displayed for the control will be decided from the list maintained in the ControlsCollection array. Thus, we first create an enumerator for ControlsCollection and locate the current control in the array:

IEnumerator list = ControlsCollection.GetEnumerator();
ControlConfiguration obj;
// Traverse through the list of controls in the ControlsCollection
// searching for the SourceControl property of the ContextMenu.
while (list.MoveNext())
{
obj = (ControlConfiguration) list.Current;
if (mnuContextMenu.SourceControl.Name.Equals(obj.ControlName))
{
.
}
}

Once the control is located, we obtain the list of selected methods from the ControlsCollection array:

IEnumerator ilist = obj.SelectedMethods.GetEnumerator();

For each method, a menu item is created and added to the context menu. We also add the appropriate event handler to the menu item depending on its name:

   while
            (ilist.MoveNext())
{
MenuItem item = new
            MenuItem(ilist.Current.ToString());
mnuContextMenu.MenuItems.Add(item);
if
            (ilist.Current.ToString().Equals("BackColor"))
item.Click += new
            System.EventHandler(OnBackColor);
.
}

Thus, the user sees only the selected menu items in the list and whenever s/he clicks on a displayed menu item, the corresponding event handler will be called.

Finally, what is required is to provide the event handlers for different menu items. The event handler code is mostly similar to the ones discussed earlier and is self-explanatory and is thus not presented here.

The application code is available in the folder ContexMenuAppl.

Conclusion

The ContextMenu class provided in the .NET library allows you to create context menus in your Windows applications. The menus may be created at design time using the menu designer, or may be created programmatically. Creating menus through program code gives flexibility in designing menus, which could change as per the application context. The article has described creating context menus programmatically.

A single menu event handler may be shared between different components. The article has demonstrated how to attach a single menu handler to different components by showing an example application.

A different set of menu items may be displayed to the user depending on the application's current context. The popup event is used for this purpose. In the popup event handler, the menu system is dynamically built. The article has demonstrated how to build a context sensitive menu system with the program example.

Finally, the various techniques were combined to create a customized menu system for the Windows Form. The application developer or the user can configure the menu items to be displayed for the various controls on the form. Such configuration is stored to an XML file. On the Form Load, the configuration is read and the menu system is dynamically built. The application developer can use the technique for customizing the menu system for the various forms in the project. This would require careful design of the event handlers and naming of the various controls on the form. The configuration for each form is stored in an independent XML-based configuration file. The facility of customization may be easily extended to the end-user by adding the customize button on the respective forms.

Article Information
Author | Dr. P.G. Sarang
Technical Editor | Adam Ryland
Author Agent | Charlotte Smith
Project Manager | Helen Cuthill
Reviewers | Saurabh Nandu, David Schultz

If you have any questions or comments about this article, please contact the technical editor.


A Multi-threaded search engine in C#

A Multi-threaded search engine in C#
http://www.csharptoday.com/content.asp?id=1802
Article Rating: 5
Published on January 18, 2002
Abstract
A lot of applications require some sort of search. Sometimes, the information we are looking for is in many different locations. Most of the time, even when the resources are independently and physically separated from each other, we still have to perform the searches one by one in the order in which they are defined in the code. In this article, Tin Lam looks at the simple but sophisticated multithreading model provided by the System.Threading namespace in the .NET Framework, with which we can simply create a thread for each search, start them all at the same time, then have them run in parallel. When the searches are done, we can process their results as each of them return.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

Many applications require some sort of search. The information being searched for is usually located in many different locations. Most applications we build might just be searching from one single location, like a database server, but there are times when we need to search information from multiple locations, such as a database server or a file server. However, most of the time, even when the resources are independently and physically separated from each other, we still have to perform the searches one by one in the order in which they are defined in the code. In Windows, it's simply because the languages we are developing the applications in either just lack the capability or are too hard for us to make all the searches start at the same time, and go in parallel. With the simple but sophisticated multithreading model provided by the System.Threading namespace in the .NET Framework, it is now a relatively easy task. We can simply create a thread for each search, start them all at the same time, then have them run in parallel. When the searches are done, we can process their results as each of them return. Multi-threaded programming is a huge topic, full coverage of it is beyond the scope of this article. This article assumes some basic understanding of the concept of it (you can read more about it from the links at the end of the article). We will keep our focus in this article on showing how to apply it in a very practical way.

The search engine

In this article, we are making use of a little search engine that searches information from three separate locations: a database server, a directory server, and the Internet. It's a web based search engine, and the results are displayed on a web page. We will begin by looking at a standard search engine, without threads. Then look at it with threads. Finally, we will be looking at it with both threads and a timeout. The main purpose of this article is only to demonstrate how to feature threads in searching. We will just be using some static values for the search result, and use the Thread.Sleep() method to simulate the time which would normally spent by the real searches. We will have five seconds for searching the Internet, three seconds for the database server and one second for the directory server.

This sample search engine has been developed and tested in Microsoft .NET SDK beta 2.0. All source code can be found in the support code download, as well as some batch files to compile them. It consists of 7 files:

  • bin\link.cs, a class definition for a link
  • bin\result.cs, a class definition for a result from a search
  • bin\linearSearch.cs, a class definition for the linear search engine
  • bin\parallelSearch.cs, a class definition for the parallel search engine
  • search1.aspx, asp code to call the search method in the linearSearch class
  • search2.aspx, asp code to call the search method in the parallelSearch class
  • search3.aspx, asp code to call the search method in the parallelSearch class with a timeout.

The search engine that searches in order

Here, we have a C# version of the search that's similar to what we usually see in applications:

The link.cs

using System;

public class link
            {

private string _name = "";
private string _url =
            "";

public string name {
get { return _name;
            }
}

public string url {
get { return _url;
            }
}

public link(string pName, string pUrl) {
_name =
            pName;
_url = pUrl;
}
}

It's nothing interesting - just a class that holds a link information, which is just the url and the name.

The result.cs

using System;
using
            System.Collections; // for ArrayList

public class result
            {

private string _label = "";
private string _message =
            "";
private ArrayList _links = new ArrayList();
private bool
            _isCompleted = false;
private DateTime _started;
private
            DateTime _done;

public string label {
get { return _label;
            }
}

public string message {
get { return _message;
            }
set { _message = value; }
}

public ArrayList links
            {
get { return _links; }
}

public bool isCompleted
            {
get { return _isCompleted; }
set {
_isCompleted =
            value;
_done = DateTime.Now;
}
}

public TimeSpan
            timeSpent {
get { return _done.Subtract( _started );
            }
}

public result(string pLabel) {
_started =
            DateTime.Now;
_label = pLabel;
}
}

A little more code is going on here, but it's still pretty staight forward. links holds an ArrayList of link objects. The timeSpent is a TimeSpan object. It starts when the result object is created, and ends when isComplete is set.

The linearSearch.cs

using System;
using System.Threading; //
            for Threading
using System.Collections; // for
            ArrayList

public class linearSearch {

private
            ArrayList _results = null;
private string _keyword;
private
            DateTime _started;
private DateTime _end;

private void
            addResult(result thisResult) {
_results.Add( thisResult
            );
}

public TimeSpan totalTime {
get { return
            _end.Subtract( _started ); }
}

public linearSearch()
            {}

public ArrayList search(string keyword) {

if (
            keyword == null || keyword == "" )
return null;

_started =
            DateTime.Now;
_keyword = keyword;
_results = new
            ArrayList();

searchWebSites();
searchDatabases();
searchDirectoryServers();

_end
            = DateTime.Now;
return _results;
}

private void
            searchDatabases() {

result r = new result("Result of \"" +
            _keyword + "\" from the database servers");
try {
// In real
            world scenarios, you will want to connect to some databases,
//
            and perform some real searches. But here we just make use of some
            static values,
// and use the Thread.Sleep() to simulate some
            delays to make it looks like it's
// actually doing some time
            consumming work.

// found 3 records from the company's book
            inventory databases
// simulating a 3 seconds database
            searches
Thread.Sleep(1000);
r.links.Add( new link("Book - The
            Tin Princess", "/") );
Thread.Sleep(1000);
r.links.Add( new
            link("Book - The Tin Woodman of Oz", "/")
            );
Thread.Sleep(1000);
r.links.Add( new link("Book - The Tin
            Man", "/") );
r.message = "found: " +
            r.links.Count;
r.isCompleted = true;

} catch(Exception e)
            {

r.isCompleted = false;
r.message = "failed";

}
            finally {
// release database connection resources
// then add
            the results
// both partial/full result is
            wanted
addResult(r);
}
}

private void
            searchDirectoryServers() {

result r = new result("Result of
            \"" + _keyword + "\" from the directory servers");
try {
//
            found 2 records from the company's employee directory
//
            simulating a one second search.
Thread.Sleep(1000);
//
            "Christine Yakukilu" is in our search result simply
// because
            there's a "tin" in "Christine"
r.links.Add( new link("Employee -
            Christine Yakukilu", "/") );
r.links.Add( new link("Employee -
            Tin Lam", "/") );
r.message = "found: " +
            r.links.Count;
r.isCompleted = true;

} catch(Exception e)
            {

r.isCompleted = false;
r.message = "failed";

}
            finally {
// release directory server connection resources
//
            then add the results
addResult(r);
}
}

private void
            searchWebSites() {

result r = new result("Result of \"" +
            _keyword + "\" from the Web Sites");
try {
// found 4 web
            sites from the web
// simulating a five seconds site
            searches.
Thread.Sleep(2000);
r.links.Add( new link("Web Site
            - Periodic Table of the Elements: Tin", "/")
            );
Thread.Sleep(1000);
r.links.Add( new link("Web Site -
            Welcome to Tin Lam's Home Page", "/")
            );
Thread.Sleep(1000);
r.links.Add( new link("Web Site - eBay
            Has Great Buys on Tin Toys", "/")
            );
Thread.Sleep(1000);
r.links.Add( new link("Web Site - Tin
            Cup: Music From The Motion Picture", "/") );
r.message = "found:
            " + r.links.Count;
r.isCompleted = true;

} catch(Exception
            e) {

r.isCompleted = false;
r.message = "failed";

}
            finally {
// free up connection resources
// then add the
            results
addResult(r);
}
}
}

Below diagram illustrate how this linearSearch works:

_results is an ArrayList of result objects added from the search methods, as represented by result1, result2 and result3 in the diagram respectively. search1, search2 and search3 represent searchWebSites(), searchDatabases() and searchDirectoryServers() respectively. These three methods do four things:

  • Create the result objects, passing the name of the searches to its constructors.
  • With Try, we are adding some static values as link objects to the result and using Thread.Sleep() for some delay. In the real world, you will want to initialize and open proper connections to the resources, such a database connections, etc, then process and add the real results from the server to the result object's link property.
  • With Catch, we handle any Exceptions thrown from the Try block. Here, we are only setting the isCompleted variable to false, and showing the message saying it has failed. In the real world, you might want to do a little more. Perhaps logging the error messages somewhere, sending an email to report it to someone who's responsible for it, or find some ways to recover from the errors and continue the searches, such as trying the search on some backup servers, etc. You might also want to expand the catch block to include more exception specific catches - maybe some database related exceptions in the searchDatabases(), and some WebRequest and network related exceptions in searchWebSites().
  • With Finally, we are releasing and closing all the opened connections to the resources. We then add the result to the _results property.

Since we are also interested in partial search results, we are adding the result object r to _results in the finally block. For instance, if we were to loop through a DataReader and add each row as a link to the links collection, and suddenly an Exception is thrown on the 11th row, then we could still have the first 10 links. But, if we only wanted a complete search result, then we would have called the addResult() at the end of the try block.

The search1.aspx

<%@ Page Language = "C#" %>

<script language="C#" runat="server">

linearSearch s;
ArrayList results;
result r;
link l = null;

protected void Page_Load(object sender, System.EventArgs e) {
s = new linearSearch();
results = s.search("tin");
}

</script>

<html>
<head><title>Linear Search</title></head>
<body>

<% for ( int i = 0; i < results.Count; i ++ ) {
r = (result)results[i]; %>
<b><%= r.label %></b> - <i><%= r.message %></i>
<% for ( int j = 0; j < r.links.Count; j ++ ) {
l = (link)r.links[j]; %>
<li><a href="<%= l.url %>"><%= l.name %></a>
<% } %>
<br>time: <%= r.timeSpent.Seconds %> second(s).
<p>
<% } %>

<hr>
total time: <%= s.totalTime.Seconds %> second(s).

</body>
</html>

ArrayList results = s.search("tin"); - We are creating the linearSearch object right away and calling its search() method, passing in the string "tin". In all the .ASPX files, we are passing in a static string "tin" to the search methods. In the real world, you might want to setup a form in a web page, or in a Windows Form, letting the user type in the keyword to search for. When the search methods are done, we will have an ArrayList of result objects. Then we just simply loop through the ArrayList and display all their links. When you run this page, assuming you have everything setup correctly, you will see something like the following:

As you can see from the diagram and the screen shot, or just by looking at the code, the total time required for this search is 9 seconds. 5 for the web sites, 3 for the database, and 1 for the directory. Consider this though: since the database might be in another server, and the directory is in another server, then what's the point of having the directory search to wait for the database search to finish? Furthermore, the search on the directory server will just take 1 second, so why does it has to wait for 8 seconds until both the two other searches are done? Even thought it's not doing anything in these 8 long seconds? Plus, if there's any problems in the other two methods, like when the database search method is trying to connect to an overloaded database server, then the directory search method will even have to wait for the database search method to wait for the database server!

The search engine that searches in parallel

The search will be a lot more efficient if we can just start all the searches at the same time, and then process the results in a first-come-first-serve manner as they return. Threads can make this happen. Manipulating threads in Windows isn't easy: VB6 and earlier versions lacked threading capability without some 3rd party components and some unstable workarounds; C++ just doesn't usually fit into the learning timeframe and the development schedule; and Java/VJ++ might not be an ideal choice either. The threading model provided in the System.Threading namespace makes threading so much easier, but it is still very powerful. We can create a thread for each search, and start them all at the same time instead of letting them all wait for the preceding searches to return. We can also process the result as each search returns, so whichever search finishes first will get its result processed first, without caring if other searches have finished (or even succeed) or not.

The diagram below illustrates how this parallelSearch works:

The parallelSearch.cs

using System;
using System.Threading;
            // for Threading
using System.Collections; // for
            ArrayList

public class parallelSearch {

private
            ArrayList _results = null;
private string _keyword;
private
            DateTime _started;
private DateTime _end;

private void
            addResult(result thisResult) {
lock( _results )
            {
_results.Add( thisResult );
}
}

public TimeSpan
            totalTime {
get { return _end.Subtract( _started );
            }
}

public parallelSearch() {}

// for backward
            compartibility
public ArrayList search(string keyword)
            {
return search(keyword, 0);
}

public ArrayList
            search(string keyword, int maxMilliSeconds) {

if ( keyword ==
            null || keyword == "" )
return null;

_started =
            DateTime.Now;
_keyword = keyword;
_results = new
            ArrayList();

// creating the threads that wrap the search
            methods
Thread ps1 = new Thread( new ThreadStart(searchWebSites)
            );
Thread ps2 = new Thread( new ThreadStart(searchDatabases)
            );
Thread ps3 = new Thread( new
            ThreadStart(searchDirectoryServers) );

// calling their
            Start() to start executing the
            methods
ps1.Start();
ps2.Start();
ps3.Start();

// if
            the maxMilliSeconds is greater than 0,
// then we want to do the
            Join() on the threads with timeout.
if ( maxMilliSeconds > 0 )
            {

TimeSpan lDelta;
DateTime lStarted =
            DateTime.Now;

// waits for the thread to be finished with
            timeout, or we abort it
if ( !ps3.Join(maxMilliSeconds)
            )
ps3.Abort();

// work out the remaining time for the 2nd
            thread
lDelta = DateTime.Now.Subtract( lStarted
            );
maxMilliSeconds -= lDelta.Seconds * 1000 +
            lDelta.Milliseconds;
lStarted = DateTime.Now;
maxMilliSeconds
            = maxMilliSeconds > 0 ? maxMilliSeconds : 0;

// waits for
            the 2nd thread with the remaining time, or we abort it
if (
            !ps2.Join(maxMilliSeconds) )
ps2.Abort();

// work out the
            remaining time for the 3rd thread
lDelta = DateTime.Now.Subtract(
            lStarted );
maxMilliSeconds -= lDelta.Seconds * 1000 +
            lDelta.Milliseconds;
maxMilliSeconds = maxMilliSeconds > 0 ?
            maxMilliSeconds : 0;

// waits for the 3rd thread with the
            remain time, or we abort it
if ( !ps1.Join(maxMilliSeconds)
            )
ps1.Abort();
}

// wait for them to clean up themselve
            if we aborted them,
// or simply wait infinitely for them to
            finish if maxMilliSeconds <=
            0.
ps1.Join();
ps2.Join();
ps3.Join();

_end =
            DateTime.Now;
return _results;
}

private void
            searchDatabases() {

result r = new result("Result of \"" +
            _keyword + "\" from the database servers");
try {
// In real
            world scenarios, you will want to connect to some databases,
//
            and perform some real searches. But here we just make use of some
            static values,
// and use the Thread.Sleep() to simulate some
            delays to make it looks like it's
// actually doing some time
            consumming work.

// found 3 records from the company's book
            inventory databases
// simulating a 3 seconds database
            searches
Thread.Sleep(1000);
r.links.Add( new link("Book - The
            Tin Princess", "/") );
Thread.Sleep(1000);
r.links.Add( new
            link("Book - The Tin Woodman of Oz", "/")
            );
Thread.Sleep(1000);
r.links.Add( new link("Book - The Tin
            Man", "/") );
r.message = "found: " + r.links.Count + " in
            total";
r.isCompleted = true;

} catch(ThreadAbortException
            tae) {

r.isCompleted = false;
r.message = "found: " +
            r.links.Count + " before aborted";

} catch(Exception e)
            {

r.isCompleted = false;
r.message = "found: " +
            r.links.Count + " before failed";

} finally {
// release
            database connection resources
// then add the
            results
addResult(r);
}
}

private void
            searchDirectoryServers() {

result r = new result("Result of
            \"" + _keyword + "\" from the directory servers");
try {
//
            found 2 records from the company's employee directory
//
            simulating a one second
            search.
Thread.Sleep(1000);
r.links.Add( new link("Employee -
            Christine Yakukilu", "/") );
r.links.Add( new link("Employee -
            Tin Lam", "/") );
r.message = "found: " + r.links.Count + " in
            total";
r.isCompleted = true;

} catch(ThreadAbortException
            tae) {

r.isCompleted = false;
r.message = "found: " +
            r.links.Count + " before aborted";

} catch(Exception e)
            {

r.isCompleted = false;
r.message = "found: " +
            r.links.Count + " before failed";

} finally {
// release
            directory server connection resources
// then add the
            results
addResult(r);
}
}

private void
            searchWebSites() {

result r = new result("Result of \"" +
            _keyword + "\" from the Web Sites");
try {
// found 4 web
            sites from the web
// simulating a five seconds site
            searches.
Thread.Sleep(1000);
r.links.Add( new link("Web Site
            - Periodic Table of the Elements: Tin", "/")
            );
Thread.Sleep(1000);
r.links.Add( new link("Web Site -
            Welcome to Tin Lam's Home Page", "/")
            );
Thread.Sleep(1000);
r.links.Add( new link("Web Site - eBay
            Has Great Buys on Tin Toys", "/")
            );
Thread.Sleep(2000);
r.links.Add( new link("Web Site - Tin
            Cup: Music From The Motion Picture", "/") );
r.message = "found:
            " + r.links.Count + " in total";
r.isCompleted = true;

}
            catch(ThreadAbortException tae) {

r.isCompleted =
            false;
r.message = "found: " + r.links.Count + " before
            aborted";

} catch(Exception e) {

r.isCompleted =
            false;
r.message = "found: " + r.links.Count + " before
            failed";

} finally {
// free up connection resources
//
            then add the results
addResult(r);
}
}
}

Everything here is the same as in linearSearch.cs, except for the following:

1. Synchonized access to _result:

lock( _results ) {
_results.Add( thisResult );
}

Since all the search methods are running concurrently, there is a good chance that more than one method will finish their search and attempt to add their result objects to _results. Just like all the Collection classes in the System.Collections namespace, ArrayList is not thread-safe by default. If we just leave it the way it is, as in linearSearch, then when two or more threads are calling their Add() at the same time, we might encounter some problems. The values in _results might be corrupted for example. However, by putting a lock on _results, the .NET runtime will enforce the synchronized access to it for us. That is, the first thread enters the method, and it acquires a lock on _results. When the 2nd thread enters the method, and tries to acquire a lock on _results, the runtime will block it from doing so, and force it to wait until the 1st thread releases the lock. Alternatively, you can make an ArrayList thread-safe with this method in the ArrayList class: public static Arraylist Synchronized(ArrayList list); passing in _results as its argument, and have it returns a thread-safe ArrayList. This method basically put a wrap around an ArrayList object, and make sure all its methods and properties are accessed by no more than one thread at any given time.

2. Creating and starting a thread for each search:

Thread ps1 = new Thread( new
            ThreadStart(searchWebSites) );
Thread ps2 = new Thread( new
            ThreadStart(searchDatabases) );
Thread ps3 = new Thread( new
            ThreadStart(searchDirectoryServers)
            );

ps1.Start();
ps2.Start();
ps3.Start();

Just as an executable has a Main() as its entry point, a thread has an entry point too. We create the threads by passing their "entry points" to their constructors. These "entry points" are actually some ThreadStart objects. ThreadStart objects are really just some delegate objects, which we can create by passing the methods that we want to be invoked to their constructors. The methods passed to the ThreadStart must not take any parameters, and can not return anything. This is the reason we declared all three search methods as void and not taking any parameters. We then call their Start() to start the execution of the methods. When we call the Start() methods, the runtime will switch the threads to the "Running" state, so the OS can schedule them for execution. Then, when the threads begin executing and invoke the given methods, the methods will be returned. Then the runtime will move on to the next line of code.

3. Aborting the threads:

if ( !ps3.Join(maxMilliSeconds) )
ps3.Abort();

If the maxMilliSeconds value is greater than 0, then that means a timeout is specified, and we would like to abort the threads if they exceed the timeout. We are timing the timeout by first calling the Join() method on one of the threads. The Join() is overloaded with three signatures, and in this case we are using this one: public bool Join(int millisecondsTimeout). It returns true if the thread has finished executing within the millisecondsTimeout period, or false if the thread is still not done after this period. Here, if the thread has finished before the maxMilliSeconds, then we leave it alone, if not, then we call its Abort() to abort it:

lDelta = DateTime.Now.Subtract( lStarted
            );
maxMilliSeconds -= lDelta.Seconds * 1000 +
            lDelta.Milliseconds;
lStarted = DateTime.Now;
maxMilliSeconds
            = maxMilliSeconds > 0 ? maxMilliSeconds : 0;

We are using two local variables lDelta and lStarted to keep track of the time, so that we can calculate how much time we spent waiting for the first thread to finish. The time we can wait for the 2nd thread is the original maxMilliSeconds minus the time we waited for the first thread. Note that the maxMilliSeconds might turn into a negative value. For example, if we spend all 3500 milliseconds waiting for the first thread, the time span of lDelta will be the value of the call to Join(), plus the time that the CPU spends on the other statements. So, if we just take the maxMilliSeconds and minus the milliseconds we get from lDelta, we will get a negative value. So, we set it to 0 if that is the case. Since the threads are all running at the same time, if one of the threads has exceed the maxMilliSeconds period, then that means all the other threads are either already returned, or they have also been running for maxMilliSeconds. So when we call Join(0) on other threads, we expect them to return true right away, or we abort them too. We call Join() first on the 3rd thread, then the 2nd, then the 1st to demonstrate that the calculations do work.

4. Catching the ThreadAbortException:

} catch(ThreadAbortException tae)
            {

r.isCompleted = false;
r.message = "found: " +
            r.links.Count + " before aborted";

Now, there has been some inconsistency in the documentation of ThreadAbortException. On the .NET beta 2 documentation, it states that: " The ThreadAbortException is a special exception that is not catchable. " However, in the .NET RC0 documentation it states that: " When this method is invoked on a thread, the system throws a ThreadAbortException in the thread to abort it. ThreadAbortException is a special exception that can be caught by application code, but is re-thrown at the end of the catch block unless ResetAbort is called. " " When this method . " refers to Abort(), so it seems like the RC0 documentation is more exact and more accurate here. The example in this article has clearly shown how we catch this exception. So, when this exception is thrown, the program leaves the try block and enters the catch block to set a property to indicate that the search is not completed, and the message indicates that it's aborted. Lastly, the finally block is executed, and the addResult() is called. In the linearSearch class, we can keep adding links to the links collection until they're all added, or when an exception is thrown. In this parallelSearch class, even if there's no error from the code itself, but it's timed out, the ThreadAbortException will be thrown. However, we will still have the links that have been added to the links collection before the timeout.

5. Waiting for the threads to return:

ps1.Join();
ps2.Join();
ps3.Join();

Here we are using another signature of Thread.Join(): public void Join(). When this method is called on a thread, the calling method (in this case, it's our search()) will indefinitely wait for the thread to finish its execution, because we want to make sure all three threads are totally stopped (either done or aborted) before we return the results. So we call their Join() again. Even if we abort the threads, we still want to wait for them to finish their clean up in the catch and finally block. This way, we can be sure that before we return the method to the caller, not only is the clean up done, but the result objects are all added to _results as well.

The search2.aspx

<%@ Page Language = "C#" %>

<script language="C#" runat="server">

parallelSearch s;
ArrayList results;
result r;
link l = null;

protected void Page_Load(object sender, System.EventArgs e) {
s = new parallelSearch();
results = s.search("tin");
}

</script>

<html>
<head><title>Parallel Search</title></head>
<body>

<% for ( int i = 0; i < results.Count; i ++ ) {
r = (result)results[i]; %>
<b><%= r.label %></b> - <i><%= r.message %></i>
<% for ( int j = 0; j < r.links.Count; j ++ ) {
l = (link)r.links[j]; %>
<li><a href="<%= l.url %>"><%= l.name %></a>
<% } %>
<br>time: <%= r.timeSpent.Seconds %> second(s).
<p>
<% } %>

<hr>
total time: <%= s.totalTime.Seconds %> second(s).

</body>
</html>

Everything is the same as in search1.aspx except that the object s is created with a different class, s = new parallelSearch(). When you run this page, assuming you have everything setup correctly, you will see something like the following:

Now it makes much more sense, the total time for the search should be the around the same time as the longest search, which is 5 seconds by the searchWebSites(). Notice that we still have the exact same results!

The search engine that searches in parallel with timeout

We are able to abort threads, which means that we can abort the searches if they exceed the timeout. Note that we can abort the searches in linearSearch too with some additional timing calculations and conditions checking. But in either case, the searches won't always stop completely at the timeout. It also depends on the processing in the catch and finally block too. But in most cases, the threads will stop almost instantly, and the delay won't be noticeable.

Below diagram illustrate how this parallelSearch with timeout works:

The search3.aspx

<%@ Page Language = "C#" %>

<script language="C#" runat="server">

parallelSearch s;
ArrayList results;
result r;
link l = null;

protected void Page_Load(object sender, System.EventArgs e) {
s = new parallelSearch();
results = s.search("tin", 3500);
}

</script>

<html>
<head><title>Parallel Search with timeout</title></head>
<body>

<% for ( int i = 0; i < results.Count; i ++ ) {
r = (result)results[i]; %>
<b><%= r.label %></b> - <i><%= r.message %></i>
<% for ( int j = 0; j < r.links.Count; j ++ ) {
l = (link)r.links[j]; %>
<li><a href="<%= l.url %>"><%= l.name %></a>
<% } %>
<br>time: <%= r.timeSpent.Seconds %> second(s) and <%= r.timeSpent.Milliseconds %> millisecond(s).
<p>
<% } %>

<hr>
total time: <%= s.totalTime.Seconds %> second(s) and <%= s.totalTime.Milliseconds %> millisecond(s).

</body>
</html>

The only difference is the extra parameter 3500 for the timeout value in results = s.search("tin", 3500) ; here we are doing the search with the timeout set as 3.5 seconds. We are using the same parallelSearch class here. When you run it, assuming you have everything setup correctly, you will see something like the following:

Summary

As you can see, the threading model lets us build more efficient and responsive software. It also lets us build a faster and smarter search engine. However, the benefits of using threads in our example comes from the fact that the resources are all located on different machines. If the resources are located on the same machine, then the advantage of this method may be minimal. On the other hand, if you have too many locations to search, then creating a thread for each might also degrade the performance. For instance, the searchWebSites() might be searching thousands of web sites. If you just create a thread for each search, then switching between the threads might be all the CPU will be doing for a few minutes. In this case, you might want to utilize a ThreadPool. C#Today has a really good article on ThreadPool at http://www.csharptoday.com/content.asp?id=1545&WROXEMPTOKEN=620568Zhiox25QOrrsnTPfm53p


Localization using .NET and C#

Localization using .NET and C#
http://www.csharptoday.com/content.asp?id=1723
Article Rating: 4.5
Published on December 21, 2001
Abstract
Developing applications that are targeted towards a wide range of people brings with it several problems: different cultures, languages, currency formats, date formats, number formats, calendars etc. Some questions arise - is it possible to develop a single desktop application or website that can be understood by different groups of people who speak different languages? Is it possible to develop an application in which the user interface varies from language to language with some specific things like currency remaining common for all users (Example: we want all the currency transactions in our application or site using US dollars, but the contents need to be presented in the user’s local language). The answer to these questions is, yes, it is possible, and has been since the introduction of Win32 – but .NET’s built–in classes and utilities make our work simpler and more efficient. In this article, Saravana Kumar looks at Localization using .NET classes.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

Developing applications that are targeted towards a wide range of people brings with it several problems: different cultures, languages, currency formats, date formats, number formats, calendars, etc. Some questions arise:

  • Is it possible to develop a single desktop application or website that can be understood by different groups of people who speak different languages?
  • Is it possible to develop an application in which the user interface varies from language to language with some specific things like currency remaining common for all users (Example: we want all the currency transactions in our application or site using US dollars, but the contents need to be presented in the user's local language).

The answer to these questions is, yes, it is possible, and has been since the introduction of Win32 - but .NET's built-in classes and utilities make our work simpler and more efficient.

The Microsoft Windows International Division defines two areas:

  • Globalization: Coding areas include currency, date/time, number formatting, etc. (For more information refer to the article written by Matthew Reynolds "Globalization using C#" at http://www.csharptoday.com/content.asp?id=1702)
  • Localization: Coding areas include not hard coding strings, no text in graphics, not hard coding error messages, etc.

In this article we are going to look at Localization using .NET classes. Lets us begin with a simple Hello World example. Look at the code below in listing 1. It just prints a string Hello World on the console.

Listing: 1

using System;

public class
            HelloWorld{
public static void Main()
            {
Console.WriteLine("Hello World");
}
}

In the above code snippet the string Hello World is hard coded. The disadvantage of this is if somebody who speaks a different language executes this program, it will still display the same text in English. We need a more intelligent way to present the text, so that the string is not hard coded.

Instead of hard coding the text as shown above we can keep all our text messages in a separate file (called a resource file).

Follow the steps below, which use localization classes to do the same task of the Hello World program discussed above.

Step 1:

Create a text file called HelloWorldStrings.txt, include the following line and save it.

HelloWorld=Hello World

Step 2:

On the command prompt enter the following command (make sure your path variable is set to run command line tools and compilers provided by .NET)

Resgen HelloWorldStrings.txt

Running this command will create a binary resource file (.resources) called HelloWorldStrings.resources

(Resgen is explained in more detail later under the heading "Resgen and AL utility")

Step 3:

Create a C# file called HelloWorldLocalized.cs and include the following code:

Listing: 2

using System;
using
            System.Resources;
using System.Reflection;
using
            System.Globalization;
using System.Threading;

public class
            HelloworldLocalized{
//create a resource manager instance so
            that
//we can get all available resource by specifying
//the
            key
static ResourceManager rm = new
            ResourceManager("HelloWorldStrings",Assembly.GetExecutingAssembly());
//Create
            a new culture info object
CultureInfo ci = new
            CultureInfo(CultureInfo.CurrentUICulture.Name);

public static
            void Main(string[] args) {
HelloworldLocalized objMain= new
            HelloworldLocalized();

if (args[0].Length !=
            0){
objMain.ci = new CultureInfo(args[0]);
//set current
            thread culture to cultureinfo instance we
            created
Thread.CurrentThread.CurrentUICulture=objMain.ci;
}
Console.WriteLine(rm.GetString("HelloWorld"));
}
}

Resourcemanager class

The most important class we used in our HelloWorldLocalized sample is the ResourceManager class. The ResourceManager class is used to get resources for the current culture from resource assemblies. ResourceManagers are expensive to use, so we may want to declare only one, and declare it at the class level. Further, a static (Shared) version of the ResourceManager can be used by many instances of our class, saving even further on memory resources. The below code snippet shows the usage of the ResourceManager class inside our application:

///.
static ResourceManager rm = new ResourceManager("HelloWorldStrings",Assembly.GetExecutingAssembly());
///.

In the above code snippet the first argument of the constructor takes the root name of the resource file and the second argument is the main assembly. We need to identify a particular Assembly to base the ResourceManager on.

The ResourceManager class has functions to retrieve the resources like GetString, GetObject, etc. In our example we used the GetString method to retrieve our HelloWorld string.

Step 4:

Compile our C# source file HelloWorldLocalized.cs using the command line compiler (csc.exe) with the /res option (to embed the resource file) as shown below:

csc /res: HelloWorldStrings.resources HelloWorldLocalized.cs

After executing the above command it will create an executable file called HelloWorldLocalized.exe. Let's view the assembly manifest of HelloWorldLocalized.exe using the Intermediate language disassembler tool (ildasm.exe) as shown below:

ildasm HelloWorldLocalized.exe

It will look something like this:

The lines .mresource public HelloWorldStrings.resources{} tell CLR about the embedded or linked resource to the main assembly.

Functionality wise, the code in listing 1 and listing 2 are identical. Both will print Hello World on the console screen, but the code in listing 2 is more intelligent in that it can print the Hello World message in different languages (if resources are provided) based upon the command line argument passed to it. There is no need to alter or recompile the original source code. If we look at the code in listing 2 we can see the text Hello World is not hard coded, but is taken from a resource file using the ResourceManager class GetString method.

We would expect the output (including the French and German options included in the download) to appear as below:

Figure 1

But, at this point of time our HelloWorldLocalized.exe will work as shown in figure 2. It will print the same message for all languages because we have only created resources for the English language. Before going deeper into the subject by creating further resources, we will see some of the basic topics required for localizability.

Figure 2

Resources and assemblies

When developing an application for localizability we must use 3 types of assemblies. They are:

  • Main Assembly,
  • Feedback Assembly (or fallback assembly)
  • Satellite Assembly

Main Assembly:

When we create our application, it is part of an assembly, which is referred to as the 'main' assembly, since it is the central part of our application. Any code aspects of our application are part of this main assembly. If we change any code in this main assembly, we need to recompile the entire assembly, which can mean a lot of work redistributing our application. In our example above HelloWorldLocalized.exe is the main assembly.

Feedback Assembly:

In our above example we mentioned that for each language (en, fr, de, etc.) it produced the same result Hello World in English. This is because of the feedback assembly. Each application will have only one Feedback assembly, which will be picked up by default if there are no corresponding language specific resources present. Since we didn't create an individual resource file for French and German, when we try to run our application with French or German language it points back to the English language, which is because of our default or Feedback assembly. It's very important to have a feedback assembly inside our application so that at least something is displayed instead of throwing an exception. In our example HelloWorldLocalized.resources is the feedback assembly, this is nothing but the compiled version of our text resource file HelloWorldStrings.txt.

Use: Including a default resource file (Feedback assembly file) simply ensures that at least some reasonable value will be found, without causing an exception to be thrown.

Satellite Assemblies:

Satellite assemblies are used in applications to support language dependent strings. Satellite assemblies hold only resources and no code. They act as an external set of resources to our application. They are also compiled and they have their own assemblies. If we ever need to change information in a satellite assembly, we can do so without having to change the information in our main assembly. Satellite assemblies are compiled as dll's and are referenced by our main assembly. In our example, since we didn't create separate resource files for different languages, we have a single English resource that will act both as satellite as well feedback assembly. So, in our example the satellite assembly is HelloWorldLocalized.resources. The content of both feedback assembly and satellite assembly are same, the main difference between them is in their usage. During runtime, resources will be picked from the satellite assembly; if there is no satellite assembly for a particular language, then the resource will be picked up from the feedback assembly (default for application). If there is no feedback assembly then an exception will be thrown.

Let us now create satellite assemblies for French and German so that our application works as expected in figure 1. We will first work on French. The steps are exactly the same for German except of course for the text in the resource file. Follow the steps below:

Step 1: Create a subfolder called fr

Step 2: Create a text file called HelloWorldStrings.fr.txt and include the following line:

HelloWorld=Bonjour Monde

Step 3: Run the following command at the command prompt from inside the fr directory:

Resgen HelloWorldStrings.fr.txt

This will create a resource file called HelloWorldStrings.fr.resources

Step 4: Run the following command at the command prompt from inside the fr directory:

al /t:lib /culture:fr /embed:HelloWorldStrings.fr.resources /out:HelloWorldLocalized.resources.dll

Our application will now work for French. Notice we did not alter nor recompile our source code. The above command will create a compiled dll resource file, which will be referenced by our main assembly, the ResourceManager does this for us: we do not need any specific code to tell our program how to reference the resource assemblies (all we need to do is to take care of file naming conventions and place them in correct directory - this is explained under "Directory structure and naming convention" later). Any language support can be given to our application by just following the above steps. If we follow the same steps for German with appropriate text message, our application will work for the German language.

Culture and Region

Up to this point we mentioned en, fr, de, etc as different languages, but there are more specific technical terms for this - culture and region. The culture is a set of preferences based on a user's language and cultural conventions. The culture name defines a 2-letter language (en, fr, de, etc) and a 2-letter region code that follow the RFC 1766 hierarchy. You can see all the codes at http://www.dsv.su.se/~jpalme/ietf/language-codes.html?WROXEMPTOKEN=620568Zhiox25QOrrsnTPfm53p.

The region is different from the culture. The region does not represent the user's preferences, but defines the location. Example:

  • fr French (Culture - French)
  • fr-FR French - French (Culture - French , Region - France)
  • fr-CA French - Canadian (Culture - French , Region - Canada)

In our example, a culture of fr-FR and fr-CA will return the same result - Bonjour Monde - but there can be subtle differences between cultures that mean that to really do localization well, we need natives of that culture to perform our translations within the framework provided.

Directory Structure and Naming Convention

The directory structure is very important when dealing with localization in .NET. Our application must follow the same hierarchical directory structure as of a language hierarchy defined by RFC 1766 standard. If we misspell the name of the culture or regions, our application will throw an exception or it will not find the correct culture. The runtime will look for resources using predefined set of rules. So the directory structure should look like this:

In our example, the structure looks like the following:

The resources specific to French must be present inside the fr folder and the resource specific to German must be present inside the de folder. If we have a subculture like French-French, Canadian-French etc it will go as a subfolder inside the French folder fr. Example:

The resource files must be named like this:

  • <myresource>.resources (Example: HelloWorldStrings.resources)
  • <myresource>.xx.resources (Example: HelloWorldStrings.fr.resources)
  • <myresource>.xx-XX.resources (Example: HelloWorldStrings.fr-FR.resources)

The satellite assembly names must be defined in the format:

  • <myproject>.resources.dll (Example: HelloWorldLocalized.resources.dll)

Resgen and AL utility

Resgen is the "Resource File Generator" tool. It converts .txt files and .resx files into .NET common language runtime binary .resources files that can be embedded in a runtime binary executable or compiled into satellite assemblies. So far we used .txt files (HelloWorldStrings.txt) as our resource files, but we can also use a .resx file as a resource file. .resx is an XML formatted resource file. The advantage a .resx file gives us over a .txt file is that we can store object structures inside a .resx file, so long as the object is serializable. The kind of objects we might want to store includes fonts, or actual picture images such as bitmaps, etc.

Using the Resgen utility we can convert files from any one format to any other format. For example we can convert:

  • ".resources" files into ".txt" files or ".resx" files.
  • ".txt" files into ".resx" files or ".resources" files
  • ".resx" files into ".txt" files or ".resources" files

If we do convert this way, we must give Resgen the name of an output file; it will not work if we specify the input file only. In the later part of the article when we develop the same application using visual studio we will be using the .resx format.

Assembly Linker (al.exe) is a stripped down version of the main compiler. The main use of the assembly linker tool is to create assembly manifests, or create a single assembly from multiple module files. Apart from this it's also used to add resources to satellite assemblies as we did in the HelloWorldLocalized example.

Creating resource files programmatically

Until now we kept all our resource files in a text file (HelloWorldStrings.txt) and we converted that into binary form using the Resgen.exe utility provided by the framework. This manual approach will not be feasible at all times. The framework provides us with some base classes by which we can create our resources programmatically. Let us create the same HelloWorldStrings.resources programmatically:

Step 1:

Create a C# file called createResource.cs and include the following code:

using System;
using
            System.Resources;

public class resourceDemo{
public static
            void Main(){
//Create a resource writer class instance by
            specifying
//the .resource file name in the
            constructor
ResourceWriter objRw = new
            ResourceWriter("HelloWorldStringProg.resources");
//add resources
            as name value pair
objRw.AddResource("HelloWorld","Hello
            World");
objRw.Close();

}
}

Step 2:

Compile the createResource.cs file using the following command:

csc createResource.cs

This will result in an executable file called createResource.exe.

Step 3:

Executing the executable file createResource.exe will create a resource file called

HelloWorldStringProg.resources. This binary file is similar to the file HelloWorldStrings.resources, the one we created using the R esgen utility. We can make sure by converting the HelloWorldStringProg.resource s into a text file using Resgen. To do this we need to run the following command:

resgen HelloWorldStringProg.resources HelloWorldStringProg.txt

This will result in a text file (HelloWorldStringProg.txt), which looks similar to our original text resource file HelloWorldStrings.txt.

The above steps show us how to create a resource file programmatically. We used the ResourceWriter class to create a new resource file. We can read a resource file using the ResourceReader class. The below code listing prints all the entries inside the resource file on the console:

using System;
using
            System.Resources;
using System.Collections;

public class
            resourceDemo{
public static void Main(){
//Create a resource
            reader class instance by specifying
//the .resource file name in
            the constructor

ResourceReader objResReader = new
            ResourceReader("HelloWorldStringProg.resources");
//available
            resource can be accessed as a collection
foreach (DictionaryEntry
            objDictEntry in objResReader)
Console.WriteLine(objDictEntry.Key
            + ": " +
            objDictEntry.Value);

objResReader.Close();
}
}

Using Visual Studio .NET

Until now we've used only command-line utilities to create our samples. Even though there are many more steps involved, it's useful to know what's happening behind the scenes. We can create the same application using Visual Studio .NET very easily. Visual Studio .NET hides most of the complexities from the developers, such as its in-built tool to edit resource files. It converts the readable format of the resource file into a binary one. It also allows the insertion of objects such as pictures, fonts, etc. inside the resource file (Note: the object must be serializable). Let's follow the steps to do the same HelloWorldLocalized example in Visual Studio .NET:

Step 1:

Create a new Visual C# Console Application project and call it HelloWorldLocalized as shown in the below figure:

Step 2:

Delete all the default code in Class1.cs and paste the code from listing 2 (in the beginning of the article).

Step 3:

Right-click on the project HelloWorldLocalized in the solution explorer and click Add and then Add new item.

Step 4:

From the available templates select Assembly resource File and name it HelloWorldStrings.resx

This will create a HelloWorldStrings.resx file, which we can view in the solution explorer.

Step 5:

Double-click on the HelloWorldStrings.resx file. By default it will take you to the data view. Enter Hello World for the value field and HelloWorld for the name field as shown in the below figure.

Step 6:

Right-click on the project HelloWorldLocalized in the solution explorer and click Properties from the menu. Set up the command line option to en as shown in the below figure.

Step 7:

The only change we need to make in the source (Class1.cs) is to put the name of the project (HelloWorldLocalized) while instantiating the ResourceManager class. The snippet below shows this:

///.
static ResourceManager rm = new ResourceManager("HelloWorldLocalized.HelloWorldStrings",Assembly.GetExecutingAssembly());
///.

If we compile and run the application it will display Hello World on the console.

So, now we have the application working for the en culture. In order to make it work for another culture, for example fr , repeat the following steps:

  • Step 3,
  • Step 4: Give an appropriate new name, HelloWorldStrings.fr.resx
  • Step 5: Change the value to the corresponding French text. In our case it's Bonjour Monde
  • Step 6: Change the Command Line Argument to fr.

Run the application. Now, it will display Bonjour Monde on the console window. If we want our application to work for German then we can follow the same set of steps.

Winres utility

If we are designing a windows form based application, which needs to be localized and the text need to be altered later, then the .NET framework provides us with a utility called winres.exe (Windows forms resource editor). Winres.exe is a stripped down version of windows form designer. We can directly open the .resource files or .resx files in this editor and we can update corresponding text and, if required, size of the controls. Below is a sample application .resx file opened with winres.exe (Note: it's not a site, it's a Windows forms application resource file that looks like a website):

Summary

In this article we've taken a quick look at how to make Localization work in C#. Most of the core concepts related to Localization are explained in this article. The same approach can be followed in ASP.NET applications as well, which makes developing multilingual websites more easy and scaleable.


Application of the Cache Subsystem for Web Applications

Application of the Cache Subsystem for Web Applications
http://www.csharptoday.com/content.asp?id=1725
Article Rating: 3
Published on December 20, 2001
Abstract
In this article, Eric Rhea explores two approaches to utilizing the caching system. First, he talks about how to make use of the cache subsystem from the web-form code. Then, he takes a look at exploring how to make use of the cache subsystem in the code behind. While doing this, he seeks three answers: (1) how to improve performance, (2) how to make use of similar techniques in both code areas, and (3) how to cache an object which is indeterminate in length and whose values you do not know in advance, and one which you cannot cache using the web-form directive.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

Caching technologies enable you to greatly improve application performance by storing images of items that are frequently accessed. .NET provides the tools for you to improve your application performance. This article serves as an introduction into the code techniques used to apply .NET caching to nearly any web application to assist in solving the problem of application performance.

System Requirements

  • Visual Studio.NET
  • C#
  • Some Web-Form Experience

Defining the Problem

While you may be impressed with the versatility of your code, I can assure you that the user is not. As a matter of fact, for every second your users look at "Loading, please wait.", that is one more second that the competition may be able to jump up and shake your bones. When we start looking at improving the performance of your web application, we start looking at Microsoft's Web Cache object as part of the System.Web.Caching namespace. More to the point, and what we will be discussing in this article is that, in general, the information is divided into two sections:

  • Caching the front end code
  • Caching in the code behind

Microsoft has some excellent documentation on how to cache the served HTML compliant pages, available as part of their MSDN collection at http://msdn.microsoft.com/library/default.asp?url=/library/en-us/cpguidnf/html/cpconaspoutputcache.asp&WROXEMPTOKEN=620568Zhiox25QOrrsnTPfm53p. . This will be our starting point as we move to more complex caching examples within this section. We will be making use of the web-forms directives to enable caching in our application.

The simplest form of enabling the caching model is to include this code segment at the top of your web-form. This will cache your served document for a length of 60 seconds. The duration will always be in units of seconds.

<%@ OutputCache Duration="60">

Graduating slowly to the next level, we'll see that it is possible to cache pages based on various keys. That is, it is possible to cache pages based on your preparation of that page. You truly see this if you were to apply the more "correct" version of the above, which is

<%@ OutputCache Duration="60" VaryByParam="None" %>

Your environment, especially if it is one that is dominated by multiple architectures (SOAP/RPC), may require that you cache the page in a given location. You may be wondering what locations are available. Let me tell you that the directive OutputCache has a parameter called Location, and that parameter will satisfy the great majority of cases. Let's peek at the following table:

Member Name | Description
Any | The Output Cache can be located on the browser client where the request originated, on a downstream server, or on the server where the request was processed.
Client | The Output Cache is located on the browser client where the request originated.
Downstream | The Output Cache is located on a server downstream from the Web server that processed the request.
None | Output caching is not enabled for the requested page.
Server | The Output Cache is located on the Web server where the request was processed.

If you are not particular to where the cache is stored, you would want to make use of "Any". This has the effect of caching the pages on all network areas that could support the cache (i.e. your client, a proxy, the server). If you want to specify only one of those devices, then you may specify which you are interested in.

So, if you wanted to mandate that the cache be stored solely on the client, then set the location to "Client". If you set the location to "none", it will not cache the page. Note, however, that doing so will not disable any controls that you have enabled to cache. This is generally a good idea if you want greater control of what g oes in and out of your cache. I have found that for dynamic applications that have only one or two static controls, making use of "none" is the way to go. Doing so grants you greater control over what is and is not cached. With the others, you have a certain liability in the sense that all the contents of the served page are going to a certain networked area.

If you are not interested in that degree of control over the application, and still want to cache the pages served, evaluate your setup. Do you have a proxy sitting between you and your clients? If so then make use of "downstream". If you do not have any middle-tiers between your clients and your web server, and are keen for caching the pages on the web server to reduce load and improve performance of the server itself, then cache on your web server using "server".

<%@ OutputCache Duration="#ofseconds" Location="Any"%>

VaryByParam asks you whether you want to vary the cache by some passed parameter, such as language. Assume you have a parameter called language, to cache each page based on the language environment in which it was produced, you would put the following line at the top of your web-form document.

<%@ OutputCache Duration="60" VaryByParam="language" %>

What if instead of generating a page with language as I have described above, that you need to vary the language based on various passed headers? Call out the VaryByHeader object. Using Microsoft's example, the object handles whether or not it needs to render the page based on the passed language environments. If it already rendered the page, it simply sends the cached version.

<%@ OutputCache Duration="60" VaryByHeader="language"%>

If you tinker with custom headers, you will want to make use of VaryByCustom. This will allow you to instruct the cache store-to-store based on given values of custom attributes. This is handy if you are generating much in the way of custom scripts for various browser identifiers, or if you want to cache different frame type pages.

<%@ OutputCache Duration="60" VaryByCustom ="frames;browserid"%>

To make use of the VaryByCustom, you'll need to stick in some code in your global.asax file. As an example, take a look at this:

/// <summary>
/// Overriden method
            for custom GetVaryByCustom operands
/// </summary>
///
            <param name="context">Review the System.Web.HttpContext
            object</param>
/// <param
            name="arg"></param>
///
            <returns></returns>
public override string
            GetVaryByCustomString(HttpContext myContext, String arg)
{
//you are caching based on the passed
            argument
switch(arg)
{
//if the requesting browser supports
            frames, cache that kind of page
case "frames":
return
            "Frames=" + context.Request.Browser.Frames;

///Cache by
            browser identifier sent
case "browserid":
//review the
            Request.Browser for list of available keys
return "browserid=" +
            Request.Browser["Name"].ToString();
}

//fell out of
            switch
return null;
}

The folks at ScootASP.Net (see links section) have two strong suggestions when making use of this parameter:

"Note: if using cookies, you'll probably want to wrap the return statement in an 'if cookie = null' type statement. In general, there will be very few cases where you'll actually need to use this code. Programmatically adding / removing cached items can be even more powerful. Note: you can have as many 'cases' in the select / switch statement to cater for as many VaryByCustom requirements that you have in your web application. The over-ridden function returns a string that will be different if the depending item (e.g. app var / cookie / session var) has changed, if the returned string is the same, then the cached version is delivered to the client."

Lastly, we have VaryByControl. This allows you to specify the storage of a particular control identifier on your page. Let's assume that in your web application you have a dropdownlist, which serves as a means to select a particular item from the inventory collection. If this information is largely static, then it makes some sense to cache it on the client.

<%@ OutputCache Duration="600" Location="Client" VaryByControl ="ddlClientInventoryList"%>

Whatever the name of your web-control, as specified by the ID field in the properties dialog, is what you will want to use as the assigned parameter value for VaryByControl.

That's all there is to it! To finish our introduction to the OutputCache directive, I present the complete directive map.

<%@ OutputCache Duration="#ofseconds" Location="Any | Client |
Downstream | Server | None" VaryByControl="controlname" VaryByCustom="browser |
customstring" VaryByHeader="headers" VaryByParam="parametername" %>

Just because you can, does not mean you should! 99% of the time, it is not proper to cache every possible variant of a page on the web-server, client, or a proxy. The rule of thumb is that when you cache, you need to cache only those items that require greater server resource to put together than if they were not cached. Or in other words, if the system requires more resource to put together a component than it does to pull it from the cache, then you will want to pull it from the cache. If you cache everything you have, you flatten the gains you would expect to have received by using the technology.

For example, it is possible to cache a page based on any possible alteration to the page. We can do this by using the * identifier. Any changes in the querystring will result in a newly cached page. Again, for the majority of cases you'll be sacrificing the performance gains that you would see by choosing this method.

<%@ OutputCache Duration="60" VaryByParam="*" %>

So when setting out to make use of the cache directive object, it pays to do some homework, especially if you are operating in a constrained environment.

So far, we've touched base on how to cache using the ASP.NET specific directives. What if you want to handle these things from the C# code that drives your application? We will look into this and explore how to make use of the code from the web application code behind.

The .NET platform provides a wonderful utility to access the caching model from your code-behind. To do this, we'll first look at the caching object as it mirrors the directive for OutputCache, and then we'll step back and see how we can take a more controlled approach to manipulating our cache.

By now you should have a firm grasp on what the various parameters do and mean. Now we will implement that code in the code behind of the application. Because a detailed analysis of these members would go beyond the scope of this article, they are provided here only to give you reference and direction.

Directive Member | C#
Duration | Response.Cache.SetExpires
Location | Response.Cache.SetCacheability
VaryByHeaders | Response.Cache.VaryByHeaders
VaryByParameters | Response.Cache.VaryByParameters
VaryByCustomer | Response.Cache.SetVaryByCustom

With the above table, you can accomplish nearly everything you could achieve by using the OutputCache directive.

Let's move on now to the more interesting Page.Cache. Before we begin, it is entirely possible to invent your own cache. Chances are that you are not interested in reinventing the wheel. They are plenty of good wheels already around. .Net's wheel is already uniformly round, why not make use of it?

The code that I've provided along with this article serves as a basic explorer of the cache system. We'll take a peak at how the basics of the Page.Cache system work, and then move up to pulling out objects and such from it.

There are two points of entry that you will use most commonly with the Page.Cache object. They are insert and remove.

How do you add data to the cache?

The insert function is overridden to provide various doorways to the Cache system. During the refactoring process, I found that I was calling the same action again and again. At the most frequently used level, I found that this call was:

Cache.Insert(the object's id,the object,null,DateTime.MaxValue,TimeSpan.FromMinutes(some int));

Parameter | Value
A_Key | A string to serve as a key (think hashtable)
A_Value | A string to serve as the value (think hashtable)
Null | Does this key rely on any dependencies? Null for no.
DateTime.MaxValue | The time at which the inserted object is removed from the cache.
TimeSpan. | The amount of time from when the object was last accessed when it is removed from the cache.

With the above in mind, we can make a quick method that takes a single object.

public void add_to_cache(object store_me)
{
Cache.Insert(store_me.ID,store_me,null,DateTime.MaxValue,
TimeSpan.FromMinutes(5));
}

With the above, we have simplified access to the cache system. Note that we can pass any object we want into the cache system, and expect to have it expire when the number of minutes (5 in our case) passed from the time of last access is up.

How do you remove an item from the Cache?

To remove an item from the Cache, you merely need to pass in the key with which you originally registered the original object.

Cache.Remove(myObj.ID); //myObj.ID is a string value

Once you have pushed data into the Cache, how do you get the value back out?

The simplest way to do this is to request a key from the cache, and then to return what it is you are expecting. You do this by issuing Cache.Get(string key). This locates a given object within the cache by a key and returns it.

object myOtherObj=this.Cache.Get(myObj.ID);

When you retrieve an item from the cache, in general, it expects that the caller will know what the callee will return. This expects the calling member to be aware of what kind of object you are returning. So remember to cast it appropriately once you return it.

How can I see what is in the cache?

The following logic will populate a referenced TextBox with the data of what is in the cache. In the code you will see some overloads for filtering options. Those overloads will allow you to visualize what you insert into the cache and ignore the system level stuff.

public void whats_in_the_cache(ref
            System.Web.UI.WebControls.TextBox
            cache_list)
{
cache_list.Text = "";
int
            i=0;
foreach(DictionaryEntry rec in
            Cache)
{
i++;
cache_list.Text += "\n" +i.ToString()+
":
            "+ rec.Key.ToString() +", "+
            rec.Value.ToString();
}
}

How can you enumerate members of a specific object retrieved from within the cache collection and assign them to a new object?

Let us presume that we want to take a dropdownlist and assign it to another drop down list, without knowing in advance what the values are going to be, or how many there will be.

First, we want to ensure the drop down list we want to populate is empty.

this.DropDownList1.ClearSelection();
this.DropDownList1.Items.Clear();

Next, we want to add to the cache the object that is going to be transferred to the recently emptied list.

this.add_to_cache(this.ddlFilledList);

Using the power of the foreach keyword, we loop over each ListItem in the dropdownlist, after retrieving it from the cache as an object, and then casting it over to become a dropdownlist. While we enumerate the list, we add each new ListItem object to the DropDownList1 above. By doing this, we effectively transfer the contents of one dropdownlist to another.

Note that once we have retrieved the object, we immediately cast it to the appropriate type.

foreach(ListItem newItem in
            ((System.Web.UI.WebControls.DropDownList)this.Cache.Get("ddlFilledList")).Items
            )
{
this.DropDownList1.Items.Add(newItem);
}

So once we have the item in the cache, we need to instruct the application how to retrieve what we have stashed in the cache so we can use it. The above addresses this need.

The effect of doing this is giving greater control over what we cache, and where we do it, by giving the application the ability to cache on demand. By checking to see if a larger object is already in the cache, we avoid the performance hit for recreating it from scratch. This is especially noticeable for large inventory lists.

The Sample Application

The sample application has a simple set of options available to move through the cache. The button "List All" lists the entire contents of the Page.Cache object. The button "Filtered List" only lists those items you have placed into the Page.Cache object. You may add or remove a key, value pair by clicking on the corresponding "Add" or "Remove" buttons. The "Push Cache To" button takes the dropdownlist object on the left and assigns it to the dropdownlist on the right.

Application Setup

  1. Extract these four files in an accessible location
  2. Open Visual Studio.NET
  3. Create a new Web Application project using c#
  4. Click File, select Add existing item(s)
  5. Navigate to where you have unzipped these files
  6. Select CacheExplorer.aspx, click open
  7. Right click on the file in the solution explorer and set it as the startup file
  8. Strike the F5 key

Limitations and Further Work

There are some limitations to what I've talked about so far. One is that I have only scratched the surface of the cache object and the caching subsystem. If you are working with complicated caching techniques, this article may appear to be basic to you. We did breeze over the Response.Cache object and the power it provides. However, it is expected that once the reader understands part one of this article, then the object should be fairly intuitive. Further work from here could be to look at the tactics and strategies of caching, alternative techniques to improving application performance, and to dig further into Response.Cache and Page.Cache.

Conclusion

By making use of the caching methods we have talked about in areas where you have large pieces of data that are frequently put together, you help to avoid the performance hit that occurs when you piece it together. Your needs drive what you need to cache, but I do have some suggestions to help guide this. First, cache anything that is pieced together from multiple data sources (such as making many ADO connections to create a combo list). Nothing drives up network latency than connecting to a SQL server, an exchange box, active directory, etc. over and over again. Cache custom controls and anything that requires significant computational power from your server, and also cache settings or other frequently accessed information as a way to keep track of user preferences on the fly. This will enable you to scale-up your application. By making use of these methods, you should see a significant reduction in the amount of processing and network latency that would normally be required of your application. This all translates to better performance for the end user.


Building an Online Shopping Cart Using C# Part 2

Building an Online Shopping Cart Using C# Part 2
http://www.csharptoday.com/content.asp?id=1720
Article Rating: 3.5
Published on December 17, 2001
Abstract
In this article, Juan Martinez continues his work in building an online shopping cart application. In part one, he discussed the basic framework of the application, now the time has come to increase the features of the shopping cart and have a complete system for online use. These enhancements will demonstrate how easy it is to add new functionality to the framework, by adding a customer rating control for the available items. He will also build an administration console, which will enable us to manage the online store through the web, and will discuss the security issues on the web, and how to implement SSL in you server for encryption. Finally he will briefly discuss the various payment services available on the web.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

In this article, we shall continue our work in building the online shopping cart application that we began in part one (http://www.csharptoday.com/content.asp?id=1526&WROXEMPTOKEN=620568Zhiox25QOrrsnTPfm53p). We have already discussed the basis framework of the application; we built a category list, item showcase, shopping cart, and checkout system.

Now the time has come to increase the features of the shopping cart and have a complete system for online use. We will build upon our development from part one and enrich the website with new features in the same way we did before, first defining our basic requirements, setting up the necessary modifications, and finally converting it to code.

These enhancements will demonstrate how easy it is to add new functionality to the framework described in the previous article, by adding a customer rating control for the available items. We will also build an administration console, which will enable us to manage the online store through the web. We will also discuss the security issues on the web, and how to implement SSL in your server for encryption. Finally we will briefly discuss the various payment services available on the web.

The application will be designed to work with an SQL Server database for storage. Application logic will be done within the Web Form and presented to the user through the web browser. Core logic will reside in a separate C# component using the code behind technique. It will also be .NET framework Beta 2 compliant.

It is assumed that you have basic knowledge of the C# language, web development knowledge, and database design basics.

Remembering the shopping cart architecture

Before we start to put together our new components, we need to remember the architecture of our existing shopping cart. We will review the shopping cart model and state where our modifications will be. We will also look back at our existing database, which we will then modify to support the new features.

The shopping cart model

The shopping cart model is a simplified flow of information that happens in all online shopping cart implementations, and is the basis for our online cart.

We then have four basic modules in our application:

  • Item Catalog - Here we display the options to our clients in an organized way.
  • Item Details - Here we show the client as much info as we can to show off our product.
  • Shopping Cart - Here we manage the user's selected items.
  • Checkout System - Here we save the items selected and the client's information to close the transaction.

We will build additional features inside the "item details" module by coding new user controls and simply include them in the Web Form.

The administration console will control the information in the database. It will not have any influence on the shopping cart model, as it will act as a back end.

The database model

Now we will take a look at our database schema.

The tables are grouped as follows:

  • Green - The catalog part of our application.
  • Blue - Item details.
  • Orange - Shopping cart basket.
  • Yellow - The checkout system.

The new features of our application will have to interact with this existing database schema. The details will be discussed afterwards.

The job ahead

Now that we have reviewed the work done previously, we need to define our new requirements, and then do the appropriate changes to be able to support them.

Defining our new requirements

As in every software development cycle, we need to define our requirements first, so that we can design software capable of giving satisfaction to our customers.

Our online shopping cart application should add the following features:

  • Include a customer rating component.
  • An administration console to modify the existing information for the catalog section / subsection, items and have a list of orders.
  • The administration console should allow registered users only through a login screen.
  • The console should also provide a way to manage these users.
  • The site should implement SSL encryption in the back end for security.
  • The shopping cart should also accept online transactions through the web.

The customer rating feature

This will be a new feature to integrate into the existing website. The goal of this part of the article is to show how easy it is to develop new functionality into existing websites using the ASP. Net framework.

This new feature requires us to develop a new user control to implement the functionality. This will be uc_item_rating.ascx, used to display the existing comments and add new ones.

Database modifications

The database needs to be modified to accommodate the new feature. We just need to add a new table and a relationship to the item table.

New controls

Our Web Application is made up of user controls. We do this to implement functionality in a modular fashion. It is now time to extend our implementation. We will add a new user control to hold the new functionality. This control will present the list of user reviews, and provide a form to add new comments. It will be added with the following code:

Registered in the page:

<%@ Register TagPrefix="SC" TagName="ItemRating" Src="uc_item_rate.ascx" %>

And included in the page body:

<SC:ItemRating runat="server" />

Code behind classes

The code to achieve the desired behavior resides in the UcItemRating class. The following code populates the rating list and calculates the average rating for the selected item:

int myItemId = 0;
if
            (Request.QueryString["itemId"] != null)
{
myItemId =
            Int32.Parse(Request.QueryString["itemId"]);
}

//Bind the
            rating list
string SQLQuery = "SELECT itemRatingAuthor,
            itemRatingComment, itemRatingRate FROM itemRating
WHERE itemId =
            " + myItemId.ToString() + " ORDER BY itemRatingId ASC";
String
            connString =
            ConfigurationSettings.AppSettings["connString"];

SqlConnection
            myConnection = new SqlConnection(connString);
SqlDataAdapter
            myCommand = new SqlDataAdapter(SQLQuery,
            myConnection);

DataSet ds = new
            DataSet();
myCommand.Fill(ds,
            "itemRating");

MyRatingList.DataSource = new
            DataView(ds.Tables[0]);
MyRatingList.DataBind();

//Calculate
            the average rating for the item
string strSelect = "SELECT
            itemRatingRate AS rate FROM itemRating";
myConnection = new
            SqlConnection(connString);
SqlCommand myCommand2 = new
            SqlCommand(strSelect,
            myConnection);

myConnection.Open();
SqlDataReader dr =
            myCommand2.ExecuteReader();
int total = 0;
double count =
            0;

while (dr.Read())
{
total +=
            dr.GetInt32(0);
count++;
}
double rate = 0;
rate =
            total / count;
myConnection.Close();

lblRating.Text =
            "Average Rating: " + rate.ToString();

The following code adds a new user rating to the list.

string author = "";
string comment =
            "";
int rate = 0;

author = txtAuthor.Text;
comment =
            txtComment.Text;
if (Radio1.Checked)
{
rate =
            3;
}
else if (Radio2.Checked)
{
rate = 2;
}
else
            if (Radio3.Checked)
{
rate = 1;
}
int myItemId =
            0;
if (Request.QueryString["itemId"] != null)
{
myItemId =
            Int32.Parse(Request.QueryString["itemId"]);
}

String
            connString =
            ConfigurationSettings.AppSettings["connString"];

//Retreive
            new Id from main order table
string strSelect = "SELECT
            COUNT(itemRatingId) AS myId FROM itemRating";
SqlConnection
            myConnection = new SqlConnection(connString);
SqlCommand
            myCommand = new SqlCommand(strSelect,
            myConnection);

myConnection.Open();
SqlDataReader dr =
            myCommand.ExecuteReader();
int newRateId = 0;
if (dr.Read())
{
newRateId = dr.GetInt32(0) +
            1;
}
myConnection.Close();

//Add new order to the main
            order table
string strInsert = "INSERT INTO
            itemRating(itemRatingId, itemId,
itemRatingAuthor,
            itemRatingComment, itemRatingRate) "
+ "VALUES (" +
            newRateId.ToString() + ", "
+ "" + myItemId.ToString() +
            ","
+ "'" + author + "',"
+ "'" + comment + "',"
+ "" +
            rate.ToString() + ")";

myConnection = new
            SqlConnection(connString);
myCommand = new SqlCommand(strInsert,
            myConnection);

myConnection.Open();
int queryResult =
            myCommand.ExecuteNonQuery();
myConnection.Close();

txtAuthor.Text
            = "";
txtComment.Text = "";
Radio1.Checked =
            false;
Radio2.Checked = true;
Radio3.Checked =
            false;
BindRatingList();

Final screens

Finally our item screen should look like this:

The administration console

The administration console consists of the pages needed to keep the site up to date. Administration pages should be developed for each table in the database that is to be administered through the web. We will present the development of such a page for the item table.

The user controls

We will develop several controls for this page. These controls contain specific functionality and are the building blocks of our Web Form. The controls are the following:

  • uc_admin_menu.ascx - Displays a list of categories.
  • uc_admin_item.ascx - Displays the list of items for the selected category and provides a form to add new items to that category. This will be used when we need to display the list of items and add new items.
  • uc_admin_item_edit.ascx - Provides a form to modify the information for the selected item. This will be used when we want to see the item's details and modify them.

The code behind classes

We have a couple of user controls that handle the information for the items in the store. The first displays the list of subsections and the list of items in the selected subsection. It allows us to add a new item to the store. The second user control displays a form with the selected item information that allows us to edit the information of the item. It also enables us to delete the item.

The item list

This code populates our list of items. First we define our DataList in the user control. This piece of code lies in the Web Form, and represents how the data will be displayed. It defines the look and feel of the list. Another part of our code is in charge of adding content to it.

<asp:DataList id="MyProductList" runat="server"
BorderColor="black"
BorderWidth="1"
GridLines="Both"
CellPadding="3"
Font-Name="Verdana"
Font-Size="8pt"
Width="600px"
HeaderStyle-BackColor="#aaaadd"
SelectedItemStyle-BackColor="Gainsboro"
RepeatDirection = "Horizontal"
RepeatColumns = "1"
DataKeyField="itemId"
>
<ItemTemplate>
<table>
<tr>
<td width=600 colspan=2><a href='admin_item_mod.aspx?subSectionId=<%# DataBinder.Eval(Container.DataItem,
"catalogSubSectionId") %>&itemId=<%# DataBinder.Eval(Container.DataItem, "itemId")
%>'><span class="itemText">Edit / Delete</span></a></td>
</tr>
<tr>
<td width=300><span class="itemTitle">Item Id:</span></td>
<td width=300><span class="itemText"><%# DataBinder.Eval(Container.DataItem,
"itemId") %></span></td>
</tr>
<tr>
<td width=300><span class="itemTitle">Item name:</span></td>
<td width=300><span class="itemText"><%# DataBinder.Eval(Container.DataItem,
"itemName") %></span></td>
</tr>
<tr>
<td width=300><span class="itemTitle">Item short descripcion:</span></td>
<td width=300><span class="itemText"><%# DataBinder.Eval(Container.DataItem,
"itemShortDescription") %></span></td>
</tr>
<tr>
<td width=300><span class="itemTitle">Item long descripcion:</span></td>
<td width=300><span class="itemText"><%# DataBinder.Eval(Container.DataItem,
"itemLongDescription") %></span></td>
</tr>
<tr>
<td width=300><span class="itemTitle">Item small image:</span></td>
<td width=300><span class="itemText"><%# DataBinder.Eval(Container.DataItem,
"itemSmallImage") %></span></td>
</tr>
<tr>
<td width=300><span class="itemTitle">Item large image:</span></td>
<td width=300><span class="itemText"><%# DataBinder.Eval(Container.DataItem,
"itemLargeImage") %></span></td>
</tr>
<tr>
<td width=300><span class="itemTitle">Item price:</span></td>
<td width=300><span class="itemText"><%# DataBinder.Eval(Container.DataItem,
"itemPrice") %></span></td>
</tr>
<tr>
<td width=300><span class="itemTitle">Item Stock:</span></td>
<td width=300><span class="itemText"><%# DataBinder.Eval(Container.DataItem,
"itemStock") %></span></td>
</tr>
<tr>
<td width=300><span class="itemTitle">Promote in Category Home:</span></td>
<td width=300><span class="itemText"><%# DataBinder.Eval(Container.DataItem,
"itemPromoteCategory") %></span></td>
</tr>
<tr>
<td width=300><span class="itemTitle">Promote in Site Home:</span></td>
<td width=300><span class="itemText"><%# DataBinder.Eval(Container.DataItem,
"itemPromoteSite") %></span></td>
</tr>
</table>
</ItemTemplate>
</asp:DataList>

Then we read from the database and bind the results to our DataList in the user control. This function executes the statement against the database and binds the results to the list:

protected void BindProductList(string
            subSectionId)
{
string SQLQuery = "SELECT * FROM item WHERE
            catalogSubSectionId=" + subSectionId
+ " ORDER BY
            catalogSubSectionId, itemName";
String connString =
            ConfigurationSettings.AppSettings["connString"];

SqlConnection
            myConnection = new SqlConnection(connString);
SqlDataAdapter
            myCommand = new SqlDataAdapter(SQLQuery,
            myConnection);

DataSet ds = new
            DataSet();
myCommand.Fill(ds,
            "item");

MyProductList.DataSource = new
            DataView(ds.Tables[0]);
MyProductList.DataBind();
}

The add button

This code calls the function to add the new item to the store in the currently selected subsection.

<asp:button text="Add Item" OnClick="AddItem_Click" runat=server/>

This code reads the information from the form and saves it to the database. This functionality is fairly simple. We check for the values in the Web Form and use it to build an Insert statement; we then execute it to save the new record.

protected void AddItem_Click(object Source,
            EventArgs e)
{
if (Request.QueryString["subSectionId"] !=
            null)
{
string itemName = txtItemName.Text;
string
            shortDesc = txtShort.Text;
string longDesc =
            txtLong.Text;
string smallImg = txtSmall.Text;
string largeImg
            = txtLarge.Text;
string price = txtPrice.Text;
string stock =
            txtStock.Text;
string myPromoteCategory = "";
string
            myPromoteSite = "";
if (promoteCategory.Checked)
{
myPromoteCategory = "1";
}
else
{
myPromoteCategory = "0";
}
if (promoteSite.Checked)
{
myPromoteSite = "1";
}
else
{
myPromoteSite =
            "0";
}

//Retreive new Id from main item table
String
            connString =
            ConfigurationSettings.AppSettings["connString"];
SqlConnection
            myConnection = new SqlConnection(connString);
string strSelect =
            "SELECT MAX(itemId) AS myId FROM item";
SqlCommand myCommand =
            new SqlCommand(strSelect,
            myConnection);

myConnection.Open();
SqlDataReader dr =
            myCommand.ExecuteReader();
int newId = 0;
if (dr.Read())
{
newId = dr.GetInt32(0) +
            1;
}
myConnection.Close();

//Add new item
string
            strInsert = "INSERT INTO item(itemId, catalogSubSectionId, itemName,
            itemShortDescription,
itemLongDescription, itemSmallImage,
            itemLargeImage, itemPrice, itemStock, itemPromoteCategory,
itemPromoteSite) "
+ "VALUES(" + newId + ", " +
            Request.QueryString["subSectionId"] + ", '"
+ itemName + "', '"
            + shortDesc + "', '" + longDesc + "', '" + smallImg
+ "', '" +
            largeImg + "', " + price + ", " + stock + ", "
+
            myPromoteCategory + ", " + myPromoteSite + ")";

myConnection
            = new SqlConnection(connString);
myCommand = new
            SqlCommand(strInsert,
            myConnection);

myConnection.Open();
int queryResult =
            myCommand.ExecuteNonQuery();
myConnection.Close();

txtItemName.Text
            = "";
txtShort.Text = "";
txtLong.Text = "";
txtSmall.Text
            = "";
txtLarge.Text = "";
txtPrice.Text = "";
txtStock.Text
            = "";
promoteCategory.Checked = false;
promoteSite.Checked =
            false;

BindProductList(Request.QueryString["subSectionId"]);
}
}

The Edit button

This button calls the edit function and saves the new data for the item. This is similar to the add button, in that we get the information from the Web Form and create the appropriate statement.

This displays the button in the Web Form:

<asp:button text="Edit Item" OnClick="EditItem_Click" runat=server/>

This stores the information to the database:

protected void EditItem_Click(object Source,
            EventArgs e)
{
if (Request.QueryString["subSectionId"] !=
            null && Request.QueryString["itemId"] != null)
{
string itemName = txtItemName.Text;
string shortDesc =
            txtShort.Text;
string longDesc = txtLong.Text;
string smallImg
            = txtSmall.Text;
string largeImg = txtLarge.Text;
string price
            = txtPrice.Text;
string stock = txtStock.Text;
string
            myPromoteCategory = "";
string myPromoteSite = "";
if
            (promoteCategory.Checked)
{
myPromoteCategory =
            "1";
}
else
{
myPromoteCategory = "0";
}
if
            (promoteSite.Checked)
{
myPromoteSite = "1";
}
else
{
myPromoteSite = "0";
}

String connString =
            ConfigurationSettings.AppSettings["connString"];
SqlConnection
            myConnection = new SqlConnection(connString);

string
            strUpdate = "UPDATE item "
+ "SET catalogSubSectionId=" +
            Request.QueryString["subSectionId"]
+ ", itemName='" + itemName
            + "', itemShortDescription='" + shortDesc
+ "',
            itemLongDescription='" + longDesc + "', itemSmallImage='"
+
            smallImg + "', itemLargeImage='" + largeImg + "', itemPrice=" +
            price
+ ", itemStock=" + stock + ", itemPromoteCategory=" +
            myPromoteCategory
+ ", itemPromoteSite=" + myPromoteSite +
            ""
+ "WHERE itemId=" +
            Request.QueryString["itemId"];
SqlCommand myCommand = new
            SqlCommand(strUpdate,
            myConnection);

myConnection.Open();
int queryResult =
            myCommand.ExecuteNonQuery();
myConnection.Close();

Response.Redirect("admin_item.aspx?subSectionId="
            +
            Request.QueryString["subSectionId"]);
Response.End();
}
}

The Delete Button

Finally the delete button deletes the item from the database. We look for the selected itemId and use it to construct our Delete statement.

This displays the button:

<asp:button text="Delete Item" OnClick="DeleteItem_Click" runat=server/>

And this code calls the Delete SQL statement:

protected void DeleteItem_Click(object Source, EventArgs e)
{
if (Request.QueryString["subSectionId"] != null &&
Request.QueryString["itemId"] != null)
{
String connString = ConfigurationSettings.AppSettings["connString"];
SqlConnection myConnection = new SqlConnection(connString);

string strDelete = "DELETE FROM item WHERE itemId=" +
Request.QueryString["itemId"];
SqlCommand myCommand = new SqlCommand(strDelete, myConnection);

myConnection.Open();
int queryResult = myCommand.ExecuteNonQuery();
myConnection.Close();

Response.Redirect ("admin_item.aspx?subSectionId=" +
Request.QueryString["subSectionId"]);
Response.End();
}
}

The final screens

This is how our administration page looks. We achieve this final screen by including our user controls into the Web Form. This process is like bolting new equipment onto the site. This modular design enables us to reuse pieces of code and makes the building process much more modular.

This screen (below) shows the Web Form making use of the uc_admin_item.ascx and the uc_admin_item.ascx user controls to display the list of menus and the list of items in the category, as well as the add new form.

The item list and Add feature:

This Web Form utilizes the uc_admin_item.ascx and the uc_admin_item_edit.ascx user control to display the menu and the edit form. Notice that this page represents different functionality, but we have reused the menu control.

The Edit / Delete page:

Setting up security for your server

A very important aspect that must be solved before you expect to ask for users' private information is securing your server's connection to the Internet.

As information travels from the user's computer to your server through the Internet wires, it is exposed to malicious users that could "sniff" into the data and retrieve some vital information like credit card numbers.

We need then to secure the information as it travels through the Internet. To achieve this, we use data encryption. The way this works is that the information is encrypted with an encryption key installed on your server. This ensures that data travels from the client to your server encrypted, so that if someone sniffs into your information, it will be useless. Information travels through https which is the http protocol with encryption, so our URL will look like this: https://yourserver/yourwebform.aspx?WROXEMPTOKEN=620568Zhiox25QOrrsnTPfm53p. It is important to note that this is needed only while important information is transmitted between the client and the host.

For more information on encryption, please see Richard Conway's article on Cryptography - part 1 (http://www.csharptoday.com/content.asp?id=1614&WROXEMPTOKEN=620568Zhiox25QOrrsnTPfm53p) and part 2 (http://www.csharptoday.com/content.asp?id=1540&WROXEMPTOKEN=620568Zhiox25QOrrsnTPfm53p).

This diagram shows the effects of having an encrypted channel to transmit data. The sniffer is unable to retrieve useful information from the wires between the server and your clients.

Now that we know how encryption works, we need to incorporate this encryption for our site. We get this encryption key from a third party vendor, for example Verisign (http://www.verisign.com/?WROXEMPTOKEN=620568Zhiox25QOrrsnTPfm53p). Installation of the key is very straightforward, and Verisign provides complete walkthroughs at http://www.verisign.com/here?WROXEMPTOKEN=620568Zhiox25QOrrsnTPfm53p. Once the key is up and running in your server, you will only need to redirect your Web Forms to use https where vital information is transmitted.

The basic steps to secure your server are:

  • Create an encryption key for your server.
  • Request the activation of your encryption key from Verisign.
  • Install the key into your server and configure IIS to make use of it.
  • Route you important information through https by making your post/get forms go through https://yourserver.com/test.aspx?WROXEMPTOKEN=620568Zhiox25QOrrsnTPfm53p, instead of just http://yourserver.com/test.aspx?WROXEMPTOKEN=620568Zhiox25QOrrsnTPfm53p.

Details on the first three steps are available in the Verisign site at the URL provided above. This includes a detailed step by step guide to installing a SSL key to your server.

Receiving payment from customers

So we have a nice online store, we have a secure connection to protect our customers, but we are still missing the part that lead us to build an online store in the first place! We need to receive payment for the products we are selling.

There are many ways to do this. In the early days, we would just receive the raw data for the order and process it manually. This involved shipping and handling the product, and performing a charge to the credit card, usually with a regular credit card merchant, by forcing a transaction with the number provided by the client.

With today's technology, we have more possibilities. We will discuss a couple of approaches that are available from third parties on the Web. These services can be grouped as follows:

  • Services that receive information from you site for processing
  • Services that provide special APIs to handle the transactions within your system.

Services that receive information from your site for processing

These kinds of services provide you with access to a predefined infrastructure within their servers. This works in a way in which you send a form with specific fields to their servers, and they take control of the whole transaction for you.

Paypal (http://www.paypal.com/?WROXEMPTOKEN=620568Zhiox25QOrrsnTPfm53p) is a third party vendor that offers this service. It does so by hosting an account with them, and posting information to the appropriate request handlers of theirs. There you will be presented with a standard interface (for which you can alter the style) for handling the pay transaction. If the transaction went out successfully, there will be a new item in your orders list for you to ship.

This is a very simple solution for the developer, since you are free of implementing the checkout system for your application. This is done entirely on Paypal's servers, which manage the shipping calculation, credit card charge, security and order administration. A tour of the service can be found at the following URL (http://www.paypal.com/here?WROXEMPTOKEN=620568Zhiox25QOrrsnTPfm53p) where details of the buying process are presented.

Details and price for this kind of service vary from vendor to vendor but the concept is to free you from the payment trouble, and concentrate in building a better site to attract more customers.

Here are the pros and cons for this approach:

Pros:

  • Easy to implement
  • Simple, easy to follow process

Cons:

  • Standard user interface
  • None or limited customization on the payment process
  • Usually share servers with others which leads to a limited number of transactions and performance limitations

Services that provide special APIs to handle the transactions within your system.

These kinds of services take a different approach to providing you with the tools to build your payment solution. They do exactly that; provide you with the tools you need to develop your own solution.

Cybercash (http://www.cybercash.com/?WROXEMPTOKEN=620568Zhiox25QOrrsnTPfm53p) is an example of one of these services providers. It gives you a set of APIs and documentation, which you can use to develop your checkout system. This puts the responsibility on the development team, but also gives them the possibility to create the solution that best fits the needs of the client. For complete documentation on how to develop a solution with Cybercash, go to this URL (https://www.verisign.com/cgi-bin/go.cgi?a=b183338890612000&WROXEMPTOKEN=620568Zhiox25QOrrsnTPfm53p).

Here are the pros and cons for this approach:

Pros:

  • Complex API gives you the power to develop your own solution
  • Gives you direct interaction with the system which gives you much more details on the transaction.
  • Can have more control over the installed platform and can scale to larger sites

Cons:

  • You need to develop the whole checkout system which can be expensive and time consuming
  • Need for server capable of supporting the API and security.

Wrapping up

We have looked at the two basic approaches that are offered on the web for payment solutions. One sets you up with a predefined solution, which you use out of the box. The other provides you with an API for you to create your own solution.

Paypal is basically a packaged product that works out of the box and provides satisfaction to most of our needs in a fast non-expensive way (although there are some small fees involved). On the other hand, services like Cybercash, which provide complex APIs for development, gives us the freedom of developing a more customized solution.

The Cybercash type of services provides the complete set of tools needed to create our application. This is best suited for projects where complex order management is required and need to be developed specifically for the occasion.

There are many more providers of this type of service out there. We can see that they can range from a simple solution to more robust state of the art APIs, so the decision as to which service to choose lies within the developer's specific needs. You need to balance the pros and cons of the vendor and evaluate which is better.

Installing the new version

Once we have finished the second part of our development, we need to deploy the changes. We have two choices here, an upgrade or a new install.

The upgrade path will be needed if you intend to implement the changes on top of the code from part 1 of this article. This will have to be a totally new installation of the site. Files are included for both paths in the downloadable zip file.

Upgrade

The steps needed to do the upgrade are the following:

  • Run the upgrade.sql script to build the new item review table, and add an administrator user with the username "admin" and the password "admin".
  • Copy the uc_item_rating.ascx user control to the root folder of your application to be with the rest of the controls and Web Forms.
  • Copy the administration folder into your root folder so the files exist in http://yourserver/yourapp/admin.

New install

The steps needed to do a new install are as follows:

  • Create a new database in SQL Server
  • Once we have created the database, we need to run the database creation script newinstall.sql, which will create the database schema and populate it with some sample data.
  • Create a new site in your IIS administration program
  • Copy the web.config file to the wwwroot folde .
  • Copy the Web Forms and User controls to the wwwroot folder.
  • Copy the codebehind.dll into your wwwroot/bin directory (no registration required!).

How our online store rates

We should now have our online store up and running with enough features to be production worthy. We have added functionality to the items module to display item reviews, and also a working content management system to manage the information in our online store. We have also discussed some third party services to handle real-time online transactions and Internet security.

Nevertheless, there is still much room for improvement.

One of the coolest features nowadays are customer management systems. These incorporate ideas presented before like related items, user accounts, mailing lists and adds upon them. It is up to you to build the appropriate custom features to fit your needs.

As for technology, we should also implement some best practices, like building upon components to achieve code reuse, and also the use of stored procedures. All this left out for simplicity.

Conclusions

This concludes a two-part article that showed how to create an online shopping cart from scratch. We have developed a catalog / section site to display our products. This was further enhanced with an item rating user control in part two, and an administration console was also developed to manage the information of the online store. We then discussed some security and payment issues and analyzed the resources needed to solve them.

Our application is now capable of handling real world use and will provide a valuable framework for further development.


Creating an ISAPI FIlter HTTP Packet Analyzer

Creating an ISAPI FIlter HTTP Packet Analyzer
http://www.csharptoday.com/content.asp?id=1719
Article Rating: 4.5
Published on December 14, 2001
Abstract
Part of the job as a software engineer is to design, code and debug HTTP/SOAP application systems. Normally, one would use Windows 2000’s Network Monitoring tool to analyze network packets to and from a machine, but this is a hassle to do every time, because by default this tool captures all network packets (including Ethernet, ATM etc) when all we may want is HTTP and the layer above that. To do this in the Network Monitor, we have to filter to only to view HTTP packets, and even then these raw packets are in hexadecimal, which is hard to read or diagnose. To solve this problem, Kenny Lim shows us how to build a tool to monitor HTTP packets.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

Part of my job as a software engineer is designing, coding and debugging my HTTP/SOAP application systems. Normally, I will use Windows 2000's Network Monitoring tool to analyze network packets to and from my machine. This is a hassle to do every time, because by default this tool will capture all network packets (including Ethernet, ATM etc) and all I want is HTTP and the layer above that. To do this in the Network Monitor, I have to filter to only to view HTTP packets, and another problem is that the raw packet (which is in hexadecimal) is hard to read or diagnose.

To solve my problem, I have designed a small analyzing system to analyze and debug my development process. There are two parts to this program:

  • An Internet Server API (ISAPI) filter written in unmanaged Visual C++ (I call this RawHTTP.dll).
  • A smart client program written in .NET Framework using C# programming language (I call this LogAnalyser.exe).

With this tool, I will have a SYSTEM TRAY icon on my machine, which will blink when a new HTTP packet is captured. Then I click on this icon, a window will be spawned and will list HTTP or SOAP packets in summary showing HTTP headers, date-time, sessions, source IP address and so on (Figure 1). If I want to look at a packet in more detail, I drill down into it by selecting a row (Figure 2).

Figure 1

Figure 2

System Requirements

This program was compiled with Microsoft Visual Studio.NET Beta 2 and run in Windows XP Professional. The C++ ISAPI (RawHTTP) is run on IIS 5.0.

The client program (LogAnalyzer) requires the .NET Framework Beta 2, and is written in C#.

Defining the Problem (Server Side)

This tool listens to all HTTP packets coming into and serving out from IIS. To do this, we have two options:

  • Use an ISAPI Filter (Native Code)
  • Use an ASP.NET HttpModule

After some fact finding, I found out that HttpModule or HttpHandler will not be able to listen to all the extensions and requests that are invalid. It is possible in IIS 6.0 by mapping all extensions to an HttpModule, but this will require all extension to be mapped to ASP.NET. This will break existing ASP or other CGI applications. Knowing this, I decided to write an ISAPI filter for this task.

ISAPI Filter (RawHTTP)

An ISAPI filter is nothing more than a conventional Win32 DLL. A DLL that work as ISAPI filter must export 2 functions - which are as follows:

GetFilterVersion

This function is the entry-point function called by IIS on this ISAPI filter. This function must be preset for the filter to work properly. Normally, this is a very good stage to add an initialization routine.

HttpFilterProc

IIS will call this function whenever the event we are interested happens. IIS will pass certain pointers to data structure to this function.

ISAPI filters are the lowest level in writing web applications, and certainly ISAPI can deliver the highest performance if carefully designed. See figure 3 for a simple illustration of where the ISAPI filter fits within IIS. When a packet comes into IIS, it will go through the ISAPI filters first, from the top of the list to the bottom. Any filter can change or discard the packet so that it won't be seen by any ISAPI filter or extension listed after it.

ISAPI extensions (also known as ISAPI applications in the IIS Manager), will not, unlike the filters, intercept every packet. They will be called only if the client browser explicitly calls it (i.e. http://localhost/isapi/myextension.dll?param=something), or if its mapped extension is being called. For example, any call to .ASP will call the asp.dll to process.

Figure 3

There are a few events that can be registered so that IIS will callback to your routine; IIS events range from controlling IIS logging, performing custom authentication, to reading raw HTTP packets. IIS events also happen in a sequence known as event notification order. For example, SF_NOTIFY_READ_RAW_DATA will happen before SF_NOTIFY_PREPROC_HEADERS. For a complete list of the event notification order for ISAPI filters, check out http://msdn.microsoft.com/library/default.asp?url=/library/en-us/iisref/html/psdk/asp/isgu744y.asp&WROXEMPTOKEN=620568Zhiox25QOrrsnTPfm53p.

For our filter code, we will only need 2 notifications from IIS:

  • SF_NOTIFY_READ_RAW_DATA
  • SF_NOTIFY_SEND_RAW_DATA.

SF_NOTIFY_READ_RAW_DATA is the first event notification that will happen when IIS receives the raw HTTP packet from a client browser or Internet application. SF_NOTIFY_SEND_RAW_DATA is the last event notification from IIS that happens when IIS sends HTTP packets generated by server code.

Overall Flowchart for the ISAPI (RawHTTP)

Start coding the ISAPI Filter

Start Visual Studio .NET and choose Visual C++ Projects | Win32 Project, and enter the name of your project (Figure 4). We will then see Figure 5: choose the Application Settings tab and choose DLL. Click Finish and Visual Studio.NET will generate a blank template.

Figure 4

Figure 5

Next add a DEF file into the project from Solution Explorer and name it RawHttp.DEF. The content of the file should be the following:

LIBRARY RawHttp
EXPORTS
GetFilterVersion
HttpFilterProc

This file defines the function that will be exported from this DLL. These are the two entry points required for it to be an ISAPI filter. In the dialog box above, Visual Studio actually has a wizard to generate the EXPORT function automatically. I'm not using this because it won't work in my case. The wizard will generate the __declspec(dllexport) keyword in front of our function declaration. However, ISAPI functions are declared as __stdcall. The compiler won't let us have both. To work around this, we use a DEF file.

Since we have promised to expose these two functions, let's code it. To begin, add a header (.h) file to the project, with the following content:

extern "C"
{
BOOL WINAPI GetFilterVersion(HTTP_FILTER_VERSION *pVersion);
DWORD WINAPI HttpFilterProc(HTTP_FILTER_CONTEXT *pfc, DWORD dwNotificationType, LPVOID pvNotification);
}

The above code is the function prototypes definition. Now, open RawHTTP.cpp (which has been generated by the VC++ Wizard) to add the following implementation code:

GetFilterVersion

BOOL WINAPI
            GetFilterVersion(HTTP_FILTER_VERSION *pVersion)
{
// Register
            Notifications
pVersion->dwFlags = SF_NOTIFY_ORDER_DEFAULT
            |
SF_NOTIFY_READ_RAW_DATA |
SF_NOTIFY_SEND_RAW_DATA;

//
            Setting Version and Description
pVersion->dwFilterVersion =
            HTTP_FILTER_REVISION;
strcpy(pVersion->lpszFilterDesc,
            "RawHTTP");

// Constructing Logger
pLogger = new
            Logger();

return TRUE;
}

This function will register two of the required IIS notifications, set the ISAPI version and description, and finally instantiate a global instance of Logger (Logger is a generic C++ class). The SF_NOTIFY_ORDER_DEFAULT tells IIS to load this ISAPI filter with the default order, which is Low priority in IIS 5.0. If you want to change the priority so that your filter can be called earlier, you can change this flag to SF_NOTIFY_ORDER_HIGH. When the correct notification is received by IIS, IIS will point to the HttpFilterProc function, which is shown below:

HttpFilterProc

DWORD WINAPI HttpFilterProc(HTTP_FILTER_CONTEXT *pFilterContext, DWORD dwNotificationType, LPVOID pvNotification)
{
DWORD dwReturn = SF_STATUS_REQ_NEXT_NOTIFICATION;

switch(dwNotificationType)
{
case SF_NOTIFY_READ_RAW_DATA:
dwReturn = pLogger->ProcessMessage(pFilterContext,
(HTTP_FILTER_RAW_DATA *)pvNotification, Logger::READ);
break;

case SF_NOTIFY_SEND_RAW_DATA:
dwReturn = pLogger->ProcessMessage(pFilterContext,
(HTTP_FILTER_RAW_DATA *)pvNotification, Logger::WRITE);
break;

default: break;
}
return dwReturn;
}

In this function, we have a switch statement to check if the packet is SF_NOTIFY_SEND_RAW_DATA or SF_NOTIFY_READ_RAW_DATA. It will then pass to the Logger class to perform more work.

Logger Class

Our Logger class will be used by this ISAPI to perform all the work. To add a generic class to the project, in Visual Studio .NET, right click on the project and add a class. Choose Generic C++ Class and Open (Figure 6).

Figure 6

The next dialog wizard will appear (Figure 7), enter Logger into the class name field and click Finish. This will generate the skeleton code of Logger.cpp and Logger.h.

Figure 7

Logger.h

The first section of this header file defines the constants for the class. Hard-coding constant values in the source code is a good example. In a real-world application, it is better to have all the constants loaded up from a configuration file.

#define OFFSET_IDENTIFIER 5
#define OFFSET_START 6
#define BLOCK_SIZE 100
#define GUID_SIZE 36
#define LOCALHOST "127.0.0.1"
#define COMMAND_IDENTIFIER "HGI8H"
#define COMMAND_START_IDENTIFIER '*'
#define COMMAND_STOP_IDENTIFIER 'B'
#define COMMAND_START_PATH "C:\\HTTPRAW\\START.KEY"
#define COMMAND_STOP_PATH "C:\\HTTPRAW\\STOP.KEY"
#define LOG_PATH "C:\\HTTPRAW\\LOG.TXT"

Next we have two structures that will hold valuable information for the class. The ContextStore structure is used to store a session number that relates to a session in IIS. Since IIS can use different threads to service the same request and response, this SessionNo will be the key relating all the conversations for a request-response from the browser/application to IIS and back.

InfoStore is the structure that stores information to be logged. Information such as source IP address, datetime, request direction, and HTTP headers are stored in this structure. In our InfoStore structure, we have a constructor and destructor for the structure. In the constructor, we make sure we have initialized our variable. In the destructor, we will free the memory allocated.

typedef struct ContextStore { DWORD
            SessionNo; } SContextStore;

typedef struct InfoStore
{
const static int c_IPAddressLen = 20;
const static int
            c_DetailTimeLen = 20;
const static int c_MillisecondLen =
            5;
const static int c_SessionNoLen = 20;
const static int
            c_DirectionLen = 10;

InfoStore(){
memset(m_IPAddress,
            NULL, c_IPAddressLen);
memset(m_DetailTime, NULL,
            c_DetailTimeLen);
memset(m_Millisecond, NULL,
            c_MillisecondLen);
memset(m_SessionNo, NULL,
            c_SessionNoLen);
memset(m_Direction, NULL,
            c_DirectionLen);
m_Buffer =
            NULL;
}

~InfoStore()
{
if(m_Buffer)
delete []
            m_Buffer;
}
char m_IPAddress[c_IPAddressLen];
char
            m_DetailTime[c_DetailTimeLen];
char
            m_Millisecond[c_MillisecondLen];
char
            m_SessionNo[c_SessionNoLen];
char
            m_Direction[c_DirectionLen];
char * m_Buffer;
long
            m_BufferLen;
} SInfoStore;

The following is the Logger class definition. The Critical Section (m_Section) is used to control synchronization of multiple threads, to stop them from corrupting the log file. Synchronization is needed because ISAPIs are implicitly multi-thread. FilterAction is a C++ enumeration to be passed to a function and differentiate between READ and WRITE. The member field m_IsOperating stores the state if the logging is turned off or on. The member functions will be explained in more detail.

class Logger
{
public:
Logger();
~Logger();

enum FilterAction { READ, WRITE };
DWORD ProcessMessage(HTTP_FILTER_CONTEXT *pFilterContext,
HTTP_FILTER_RAW_DATA *pFilterRawData, FilterAction f);

private:
BOOL CheckHttpCommand(SInfoStore *pInfo);
char * ReadContent(char *pFileName);
SContextStore * ReadContextStore(HTTP_FILTER_CONTEXT
*pFilterContext, FilterAction f);
BOOL ReadInformation(SInfoStore *pInfo, HTTP_FILTER_CONTEXT
*pFilterContext, FilterAction f);
BOOL WriteLog(SInfoStore *pInfo);
BOOL WriteDetailLog(SInfoStore *pInfo);
DWORD ReadRequestPosition(SInfoStore *pInfo);

CRITICAL_SECTION m_Section;
BOOL m_IsOperating;
};

Logger.cpp

Constructor & Destructor

When the Logger is instantiated, the constructor will be called. In the constructor, we initialize the Critical Section to be used as a synchronization object for all the servicing threads. When the object is done, the destructor will be called. When this happens, it will delete the same Critical Section.

#include
            "stdafx.h"

Logger::Logger()
{
// if you don't want your
            ISAPI to be run by default, uncomment the
// following
            statement.
// m_IsOperating =
            false;
::InitializeCriticalSection(&m_Section);
}

Logger::~Logger()
{
::DeleteCriticalSection(&m_Section);
}

Logger::ProcessMessage

ProcessMessage is the member function that is called from HttpFilterProc. This method will be called regardless of READ or SEND notification. To differentiate the caller context, we use a FilterAction enumeration. Along with this enumeration, this method accepts the standard structure from IIS.

DWORD Logger::ProcessMessage(HTTP_FILTER_CONTEXT *pFilterContext, HTTP_FILTER_RAW_DATA *pFilterRawData, FilterAction f)

{
DWORD dwTemp;
DWORD dwReturn = SF_STATUS_REQ_NEXT_NOTIFICATION;
// Code Continues below .
}

Part of the logic of ProcessMessage is to instantiate a SInfoStore structure (look at logger.h), this will store information about the packet.

Next we attempt to create a block of memory to store the incoming data. We won't define a fixed size array to store this, because we want to make sure we have no buffer overrun problem in this project. Buffer overrun is the number one cause of problems like downtime or hacking. When someone knows that there is a flaw in the buffer management (for example, in using a fixed size array), they can transmit more bytes to the server and ultimately take over the IIS process (inetinfo.exe) by injecting malicious code.

Check if the memory allocation is successful; if not, abort the session. If the allocation is ok, we can copy the packet buffer and source IP address into our structure and then check if it is a command from our smart client, LogAnalyzer (for more information about this routine, look into the CheckHttpCommand member function).

If the packet is not a command, the code will check whether the current state (m_IsOperating) is on or off. If it's on, we can proceed to read the remaining information like datetime and session into our structure (SInfoStore). With all the information populated into the structure, we can then write the information to the log file. After writing to the log file, we can leave the critical section so that other threads can get into the log file and write as well. Finally, we de-allocate the structure (SInfoStore), which will then call its destructor.

SInfoStore *pInfo = new
            SInfoStore();
if(pInfo)
{
pInfo->m_Buffer = new
            char[pFilterRawData->cbInBuffer+1];
if(pInfo->m_Buffer)
{
memcpy(pInfo->m_Buffer,
            pFilterRawData->pvInData,
pFilterRawData->cbInBuffer);
pInfo->m_Buffer[pFilterRawData->cbInData]
            = NULL;
pInfo->m_BufferLen =
            pFilterRawData->cbInData;

::EnterCriticalSection(&m_Section);

dwTemp
            =
            pInfo->c_IPAddressLen;
pFilterContext->GetServerVariable(pFilterContext,
            "REMOTE_ADDR",
pInfo->m_IPAddress,
            &dwTemp);
if(!this->CheckHttpCommand(pInfo)){
if(this->m_IsOperating){
this->ReadInformation(pInfo,
            pFilterContext,
            f);
this->WriteLog(pInfo);
}
}
else
dwReturn =
            SF_STATUS_REQ_FINISHED;

::LeaveCriticalSection(&m_Section);

}
}
delete
            pInfo;
return dwReturn;
}

Logger::CheckHttpCommand

One of the features of our tool is the ability to control the filter from a smart client (LogAnalyzer). For example, we can turn off the packet logging when we're not testing a web application. To achieve this, the ProcessMessage member function will call CheckHttpCommand for every packet that comes in. The algorithm is simple. We first check if the request is coming from localhost. If it's not, return immediately. If it is from localhost, the function will then check if an identifier exists in the HTTP header (COMMAND_IDENTIFIER). If this string exists in the packet, it will go further down to check if it's a START (COMMAND_START_IDENTIFIER) command or STOP command (COMMAND_STOP_IDENTIFIER). Finally it will check if the key is matching correctly. The mechanism used for the key is that the client will generate a GUID at the time of the request (START or STOP) and then write the GUID into a file and send it as part of the request command. This method will compare the GUID from the file and the GUID sent up with the request. If they match, the command is authenticated.

BOOL Logger::CheckHttpCommand(SInfoStore
            *pInfo)
{
BOOL bReturn =
            FALSE;
if(strcmp(pInfo->m_IPAddress,
            LOCALHOST)==0)
{
{
char char *pBlock = new
            char[BLOCK_SIZE+1];
if(pBlock){
if(pBlock)
{
strncpy(pBlock,
            pInfo->m_Buffer, BLOCK_SIZE);
pBlock[BLOCK_SIZE] =
            NULL;
char *p = ::strstr(pBlock,
            COMMAND_IDENTIFIER);
if(p){
char *pKey;
char *pPos =
            p+6;
if(*(p+5) ==
            COMMAND_START_IDENTIFIER){
if(p)
{
char *pKey;
char
            *pPos = p+OFFSET_START;
if(*(p+OFFSET_IDENTIFIER) ==
            COMMAND_START_IDENTIFIER)
{
pKey =
            this->ReadContent(COMMAND_START_PATH);
if(pKey){
if(pKey)
{
if(::strncmp(pKey,
            pPos, GUID_SIZE) == 0){
this->m_IsOperating = TRUE;
bReturn
            = TRUE;
}
delete[] pKey;
}
}
else if(*(p+5) ==
            COMMAND_STOP_IDENTIFIER){
{
this->m_IsOperating =
            TRUE;
bReturn = TRUE;
}
delete[] pKey;
}
}
else
            if(*(p+OFFSET_IDENTIFIER) == COMMAND_STOP_IDENTIFIER)
{
pKey
            pKey pKey = this->ReadContent(COMMAND_STOP_PATH);
if(pKey){
            if(pKey)
{
if(::strncmp(pKey, if(::strncmp(pKey, pPos,
            GUID_SIZE) == 0){
0)
this->m_IsOperating =
            FALSE;
bReturn = TRUE;
}

delete[]
            pKey;
}
}
}
delete[]
            pBlock;
}
}
{
this->m_IsOperating = FALSE;
bReturn
            = TRUE;
}
delete[] pKey;
}
}
}
delete[]
            pBlock;
}
}
return return bReturn;
}

Logger::ReadContent

ReadContent is just a helper function that will return the content of a file. This function is use to fetch the GUID from the file.

char * Logger::ReadContent(char
            *pFileName)
{
FILE *fptr = fopen(pFileName,
            "r");
if(fptr){
char *pKey = new
            char[GUID_SIZE+1];
if(pKey)
{
fgets(pKey, GUID_SIZE+1,
            fptr);
pKey[GUID_SIZE] = NULL;
fclose(fptr);
return
            pKey;
}
}
return NULL;
}

Logger::ReadContextStore

ReadContextStore is invoked by ReadInformation. The purpose of this method is to generate our session number when it is a request (READ). To do so, we have to use AllocMem from ISAPI to allocate memory for our data. When we use AllocMem, IIS will handle the memory management for the allocated memory. It will then stuff into pFilterContext. This way, we can set the context (session number) throughout the same request-response. If it's a reply from the IIS (SEND), we can just grab the information from pFilterContext.

SContextStore *
            Logger::ReadContextStore(HTTP_FILTER_CONTEXT
*pFilterContext,
            FilterAction f)
{
SContextStore *pContext = NULL;
if(f ==
            FilterAction::READ){
if(f == FilterAction::READ)
{
pContext
            = (SContextStore *)pFilterContext->AllocMem(pFilterContext,
(pFilterContext, sizeof(SContextStore),
            0);
pContext->SessionNo =
            ::GetTickCount();
pContext->CountNo =
            1;
pFilterContext->pFilterContext =
            pContext;
}
else
{
pContext = (SContextStore
            *)pFilterContext->pFilterContext;
pContext->CountNo =
            pContext->CountNo+1;
pFilterContext->pFilterContext =
            pContext;
}
return pContext;
}

Logger::ReadInformation

ReadInformation will be called by ProcessMessage to populate the SInfoStore structure:

BOOL Logger::ReadInformation(SInfoStore
            *pInfo, HTTP_FILTER_CONTEXT
*pFilterContext, FilterAction
            f)
{
SYSTEMTIME
            t;
::GetSystemTime(&t);
::sprintf(pInfo->m_DetailTime,
            "%02d:%02d:%02d.%d",
t.wHour, t.wMinute, t.wSecond,
            t.wMilliseconds);
::sprintf(pInfo->m_Millisecond, "%d%c",
            t.wMilliseconds, NULL);
::sprintf(pInfo->m_SessionNo, "%d",
            SYSTEMTIME t;
SContextStore *pContext =
            this->ReadContextStore(pFilterContext,
            f)->SessionNo);
f);
::GetSystemTime(&t);
::sprintf(pInfo->m_DetailTime,
            "%02d:%02d:%02d.%d%d", t.wHour, t.wMinute,
t.wSecond,
            t.wMilliseconds,
            pContext->CountNo);
::sprintf(pInfo->m_Millisecond,
            "%d%d%c", t.wMilliseconds, pContext->CountNo,
            NULL);
::sprintf(pInfo->m_SessionNo, "%d",
            pContext->SessionNo);
if(f if(f ==
            FilterAction::READ)
strcpy(pInfo->m_Direction,
            "Incoming");
else
strcpy(pInfo->m_Direction,
            "Outgoing");
return TRUE;
}
strcpy(pInfo->m_Direction,
            "Incoming");
else
strcpy(pInfo->m_Direction,
            "Outgoing");
return TRUE;
}

Logger::WriteLog

WriteLog is used to append to the log file. The log file is created in shared mode, and we set the file pointer to the end before we begin writing to it. In the body, we use the SInfoStore structure and write XML element strings into the log file. However, this function will just log summary information. The generated XML file will be loaded up by the client program as a data grid. The raw detail packet information is stored in the detail file by WriteDetailFile (below):

BOOL Logger::WriteLog(SInfoStore
            *pInfo)
{
DWORD dwTemp;
HANDLE hFile =
            ::CreateFile(LOG_PATH, GENERIC_WRITE,
FILE_SHARE_READ, NULL,
            OPEN_ALWAYS, FILE_ATTRIBUTE_NORMAL, NULL);
if(hFile !=
            INVALID_HANDLE_VALUE){
::SetFilePointer(hFile, 0, 0,
            FILE_END)

::WriteFile(hFile, "<call>", 6, &dwTemp,
            NULL);

::WriteFile(hFile, "<session>", 9, &dwTemp,
            NULL);
::WriteFile(hFile, pInfo->m_SessionNo,
strlen(pInfo->m_SessionNo), &dwTemp,
            NULL);
::WriteFile(hFile, "</session>", 10, &dwTemp,
            NULL);

::WriteFile(hFile, "<time>", 6, &dwTemp,
            NULL);
::WriteFile(hFile, pInfo->m_DetailTime,
strlen(pInfo->m_DetailTime), &dwTemp,
            NULL);
::WriteFile(hFile, "</time>", 7, &dwTemp,
            NULL);

::WriteFile(hFile, "<ip>", 4, &dwTemp,
            NULL);
::WriteFile(hFile, pInfo->m_IPAddress,
strlen(pInfo->m_IPAddress), &dwTemp,
            NULL);
::WriteFile(hFile, "</ip>", 5, &dwTemp,
            NULL);

::WriteFile(hFile, "<dir>", 5, &dwTemp,
            NULL);
::WriteFile(hFile, pInfo->m_Direction,
strlen(pInfo->m_Direction), &dwTemp,
            NULL);
::WriteFile(hFile, "</dir>", 6, &dwTemp,
            NULL);

DWORD dwPos =
            this->ReadRequestPosition(pInfo);
if(dwPos ==
            0)
{
::WriteFile(hFile, "<request>", 9, &dwTemp,
            NULL);
::WriteFile(hFile, "</request>", 10, &dwTemp,
            NULL);
this->WriteDetailLog(pInfo);
}
else
{
::WriteFile(hFile,
            "<request><![CDATA[", 18, &dwTemp,
            NULL);
::WriteFile(hFile, pInfo->m_Buffer, dwPos, &dwTemp,
            NULL);
::WriteFile(hFile, "]]></request>", 13,
            &dwTemp,
            NULL);
this->WriteDetailLog(pInfo);
}

::WriteFile(hFile,
            "</call>", 7, &dwTemp,
            NULL);

CloseHandle(hFile);
return TRUE;
}
return
            FALSE;
}

BOOL Logger::WriteDetailLog(SInfoStore
            *pInfo)
{
BOOL bReturn = FALSE;
DWORD dwTemp;
char
            *pDetailFile = new char[MAX_PATH];
if(pDetailFile)
            {
memst(pDetailFile, 0, MAX_PATH);
sprintf(pDetailFile,
            "C:\\HTTPRAW\\%s%s%s.txt",
            if(pDetailFile)
{
memset(pDetailFile, 0,
            MAX_PATH);
sprintf(pDetailFile, "C:\\HTTPRAW\\%s%s%s.txt",
            pInfo->m_SessionNo,
pInfo->m_Millisecond,
            pInfo->m_Direction);
HANDLE hDetail = CreateFile(pDetailFile,
            GENERIC_WRITE, FILE_SHARE_READ,
NULL, OPEN_ALWAYS,
            FILE_ATTRIBUTE_NORMAL, NULL);
if(hDetail){
WriteFile(hDetail,
            pInfo->m_Buffer, (DWORD)pInfo->m_BufferLen,
            (DWORD)strlen(pInfo->m_Buffer), &dwTemp,
            NULL);
&dwTemp, NULL);
CloseHandle(hDetail);
bReturn =
            TRUE;
}
delete [] pDetailFile;
}
return
            bReturn;
}
if(hDetail)
{
WriteFile(hDetail,
            pInfo->m_Buffer, (DWORD)pInfo->m_BufferLen, &dwTemp,
            NULL);
CloseHandle(hDetail);
bReturn = TRUE;
}
delete []
            pDetailFile;
}
return bReturn;
}

StdAfx.h

Finally, be sure to add the required header file to the project before compiling. Httpfilt.h is required for ISAPI related functions. String.h is required for string manipulation function such as sprintf.

#pragma once

// Exclude rarely-used stuff from Windows headers
#define WIN32_LEAN_AND_MEAN

// Windows Header Files:
#include <windows.h>
#include <httpfilt.h>
#include <string.h>
#include <stdio.h>
#include "Logger.h"

Compiling & Deploying the DLL

Now all the source code is in place, we can build our DLL. By default, Visual Studio will generate the DLL in DEBUG mode, which is fine, as we may wish to debug our ISAPI. For more information about debugging ISAPI, check out http://support.microsoft.com/default.aspx?scid=kb;EN-US;Q183480&WROXEMPTOKEN=620568Zhiox25QOrrsnTPfm53p.

After building the project, we'll have RawHttp.dll in a debug directory. The next step is to install this DLL in our ISAPI. To do this, open up IIS Manager, right click on the server name, and choose Properties (Figure 8).

In the IIS Master Properties dialog box, choose Edit and then the ISAPI filter tab (Figure 9). Click the Add button and specify the ISAPI name and path to our DLL (Figure 10).

Figure 8

Figure 9

Figure 10

Once we have installed the ISAPI filter, open a command window and run iisreset. This will restart the IIS service and load up the RawHTTP.dll. Once we have done this, create a folder c:\HttpRaw. This is the directory used by this tool.

Now we can run a browser or some other Web Stress utility like WAST to hit our web site. One way that I use to check for memory leaks in my DLL is to use the Performance Monitor, select inetinfo.exe, and choose the Private Bytes counter. By having a stress utility run over the course of a day or two, I can monitor for a memory leak happening.

Windows Form Client (HTTPAnalyzer)

The ISAPI Filter will log summary and detail packet information into files. In this section, we will explore how we are going to utilize those log files. The LogAnalyzer program is a Visual Studio .NET Windows Forms solution consisting of 2 forms. The main form is called FORM1 (Figure 1) and the dialog form is called FORM2 (Figure 2).

FORM1

FORM1 is the main startup window for the client program. This window shows summary HTTP packet information in the log files. To start explaining this window, we should start with the Load event.

Form1_Load Event

The Load event is a good place to initialize resources. In this event, we load icon files into Icon objects. This icon will be used to display the icon on the system tray area:

      private
            void Form1_Load(object sender, System.EventArgs e)
{
//
            loading up icon files to be used in the system tray.
oIconNormal
            = new Icon(Application.StartupPath + "\\icon0.ico");
oIconChange1 = new Icon(Application.StartupPath +
            "\\icon1.ico");
oIconChange2 = new Icon(Application.StartupPath +
            "\\icon2.ico");
oIconDisable = new Icon(Application.StartupPath +
            "\\icon3.ico");

In this section, we instantiate a FileSystemWatcher object. The FileSystemWatcher is used to notify the client program when certain filesystem events like file size changes happen. In our case, we tell it to monitor for log.txt file (the log file that our RawHttp ISAPI writes to) and tell it to notify our callback function (OnChanged) when an event happens.

         //
            registering file system watcher.
oFSWatch = new
            FileSystemWatcher();
oFSWatch.Path =
            sCfgPath;
oFSWatch.NotifyFilter =
            NotifyFilters.LastWrite;
oFSWatch.Filter =
            sCfgFileName;
oFSWatch.Changed += new
            FileSystemEventHandler(OnChanged);
oFSWatch.EnableRaisingEvents =
            true;

In this section, we set up the system tray area. We instantiate the NotifyIcon object, setting up the default icon, text, and add an event handler (OnClick). We also create a context (or right click) menu and add a bunch of menu items. Each menu item has an event handler associated with it:

         //
            setting up NotifyIcon and context menu.
tray = new
            NotifyIcon();
ContextMenu ctx = new ContextMenu();
oMenuStatus
            = new MenuItem("Disable Service", new
            EventHandler(OnStatus));
ctx.MenuItems.Add(oMenuStatus);
ctx.MenuItems.Add(new
            MenuItem("Refresh", new
            EventHandler(OnRefresh)));
ctx.MenuItems.Add(new
            MenuItem("Clear/Reset", new
            EventHandler(OnClear)));
ctx.MenuItems.Add(new
            MenuItem("-"));
ctx.MenuItems.Add(new MenuItem("Exit", new
            EventHandler(OnExit)));
tray.ContextMenu = ctx;
tray.Icon =
            oIconNormal;
tray.Text = "HTTP Analyser, no
            packet";
tray.Click += new EventHandler(OnClick);
tray.Visible
            = true;
oBtnStatus.Text = oMenuStatus.Text;

In this section, we send an HTTP GET request to our HttpRaw ISAPI filter. This is a command packet to tell HttpRaw to start logging now the client analyzer has started. To send the HTTP request, we use the WebRequest class. PrepareStartKey is a method that will both create a start GUID and return it. The catch block is empty because the ISAPI won't return status code 200 for the GET request. Therefore, GetResponse will always throw an exception.

         try
{
WebRequest
            req = WebRequest.Create(sStartURL +
            PrepareStartKey());
req.GetResponse();
}
// the catch block
            is empty because ISAPI will never return any response for
            command.
catch(Exception ex){}

In this section, we are setting some properties for the DataGrid object. At the end of this event, we call Reload.

         //
            setting some properties for
            DataGrid.
oDataGrid.PreferredColumnWidth =
            150;
oDataGrid.ReadOnly = true;
oDataGrid.ParentRowsVisible =
            false;
this.Reload();
}

Reload()

The purpose of Reload is to populate the DataGrid in FORM1. To do this, we use data binding. This method is called at the end of the form1_load event, the Refresh menu item and the Refresh button:

private void Reload()
{
try
{

The first step to check in this method is whether log.txt exists or not. If this file does not exist, there's no point proceeding because there's no log file to use.

if(File.Exists("c:\\httpraw\\log.txt"))
{

When we know log.txt exists, we create a memory stream (256KB) as a buffer to hold the final XML used to populate the DataSet object. All the code does is insert a <root> element before and after those XML elements from log.txt.

               DataSet
            oDoc = new DataSet();
MemoryStream ms = new
            MemoryStream(MEMORYSIZE); // 256KB memory
StreamWriter s = new
            StreamWriter(ms);

StreamReader oStreamXml = new
            StreamReader("c:\\httpraw\\log.txt");
s.Write("<root>");
s.Write(oStreamXml.ReadToEnd());
oStreamXml.Close();
oStreamXml
            = null;
s.Write("</root>");
s.Flush();

To ensure that the memory stream can be read into the DataSet, we have to set the position to 0. This will set the internal pointer of the memory stream to the start position. Next, we reading the DataSet object and bind the DataSet with the DataGrid object. This will display the summary HTTP packet in the DataGrid.

               ms.Position
            = 0;
oStreamXml = new
            StreamReader(ms);
oDoc.ReadXml(oStreamXml);
oStreamXml.Close();
oStreamXml
            = null;
oDataGrid.DataSource =
            oDoc;
oDataGrid.NavigateTo(0,"call");
}
}
catch(Exception
            e)
{
MessageBox.Show("System Error or ISAPI not
            running.");
this.Dispose();
}
}

OnChanged

The OnChanged method will be called when the log.txt is changed. This method updates the text in the window to tell the user that a new packet has arrived. In addition, we animate an icon to get attention from the user.

   private void
            OnChanged(object source, FileSystemEventArgs e)
{
tray.Icon =
            oIconChange1;
tray.Text = "Last HTTP capture : " +
            DateTime.Now.ToLongTimeString();
oStatusBar.Text = "New Data
            Available, click Refresh. " +
            tray.Text;
Thread.Sleep(ANIMATIONDELAY); // 25 ms
tray.Icon =
            oIconChange2;
Thread.Sleep(ANIMATIONDELAY); // 25 ms
tray.Icon
            = oIconNormal;
}

oBtnClear_Click

This event handler will be fired when the user click on the CLEAR button. This event handler will enumerate through all the .txt files and delete them from the c:\httpraw folder. With this functionality, the working folder can be cleaned up when the user decides to do so. After calling Delete() on each FileInfo object, we'll refresh the DataGrid:

   private void
            oBtnClear_Click(object sender, System.EventArgs
            e)
{
try
{
DirectoryInfo di = new
            DirectoryInfo(@"c:\httpraw\");
FileInfo[] fi =
            di.GetFiles("*.txt");
int nLength = fi.Length;
for(int
            i=0;i<nLength;i++)
fi[i].Delete();
oDataGrid.DataSource =
            null;
oDataGrid.Refresh();
}
catch(Exception
            ex)
{
MessageBox.Show("Error : Try again later, some files are
            being used by other process.");
}
}

FORM2

FORM2 is a dialog box that will be created when the user click on a row of summary packet to drill down. FORM2 is just a content viewer for the detail packet information. There are a few tabs in FORM2, each tab showing different encoding for the same data. Here are a few important methods in FORM2.

GetPacketContent

The purpose of this method is to read the content from a detailed log file (written by HttpRaw) and display it in the default tab (ASCII). This method will start to read the file using BinaryReader. Next the binary byte will be converted to a character and appended to StringBuilder. When all the bytes from the BinaryReader are converted, it will show the string in a text box (oTextAscii). Please note that because the TextBox control cannot display the character NULL, this method removes any NULL character in the string. This is a limitation of this method.

     private void
            GetPacketContent(string sFileName)
{
try
{
// opening
            the detailed log file.
FileStream fs = new
            FileStream("c:\\httpraw\\" + sFileName, FileMode.Open,
            FileAccess.Read);
if(fs!=null)
{
BinaryReader bs = new
            BinaryReader(fs);
// only handle first 256kb of
            content.
StringBuilder sBuilder = new
            StringBuilder(READSIZE);
m_InternalBytes =
            bs.ReadBytes(READSIZE);
int nLen = m_InternalBytes.Length;
//
            for loop to ignore character NULL and append to
            StringBuilder.
for(int
            i=0;i<nLen;i++)
if(m_InternalBytes[i] !=
            0)
sBuilder.Append(Convert.ToChar(m_InternalBytes[i]));
//
            Display the String from StringBuilder to the
            TextBox.
oTextAscii.Text =
            sBuilder.ToString();
bs.Close();
fs.Close();
}
else
oTextAscii.Text
            = "No detail packet for this record.";
}
catch(Exception
            e)
{
oTextAscii.Text = "No detail packet for this
            record.";
}
}

SelecedtIndexChanged

This event handler will be called when the user changes from one tab to another. This event handler will convert from ASCII text into various types of format to display in a text box.

   private void
            tabControl1_SelectedIndexChanged(object sender, System.EventArgs
            e)
{
// Hexadecimal Tab
if(tabControl1.SelectedIndex ==
            1)
{
string sTemp;
char [] c =
            oTextAscii.Text.ToCharArray();
StringBuilder sBuilder = new
            StringBuilder();
for(int i=0;i<c.Length;i++)
{
//
            Convert string to hexadecimal (base 16)
sTemp =
            Convert.ToString(Convert.ToInt32(c[i]), 16);
if(sTemp.Length ==
            1)
sBuilder.Append("0");
sBuilder.Append(sTemp);
sBuilder.Append("
            ");
}
oTextHex.Text = sBuilder.ToString();
}
// Decimal
            Tab
else if(tabControl1.SelectedIndex == 2)
{
string
            sTemp;
char [] c =
            oTextAscii.Text.ToCharArray();
StringBuilder sBuilder = new
            StringBuilder();
for(int i=0;i<c.Length;i++)
{
//
            Convert string to decimal (base 10)
sTemp =
            Convert.ToString(Convert.ToInt32(c[i]), 10);
if(sTemp.Length ==
            1)
sBuilder.Append("00");
else if(sTemp.Length ==
            2)
sBuilder.Append("0");
sBuilder.Append(sTemp);
sBuilder.Append("
            ");
}
oTextDecimal.Text = sBuilder.ToString();
}
//
            Base64 Tab
else if(tabControl1.SelectedIndex == 3)
{
//
            Convert string to Base64 string
oTextBase64.Text =
            Convert.ToBase64String(m_InternalBytes);
}
}

Limitations and Future Work

The design of this filter is to be used as a personal HTTP/SOAP packet analyzer. This work is not intended to be installed on any production environment, because it logs a lot of details about all incoming and outgoing packets.

Secondly, we use the file system to store XML documents about the HTTP packets. To improve performance, we could use a named pipe to stream data from the ISAPI filter to the client program without I/O to the disk. Writing to memory is much faster than to disk.

Conclusion

The motivation of this tool is to make the debugging and analysis of HTTP or SOAP packets much easier than it is today. By having this tool, I can spot a problem much earlier at the packet level. In addition, this tool allows me to know which culprit machine is sending those "Code Red" packets to my computer ;-)

If you'd like to discuss a broader range of C#Today topics and issues, or if you have any C# related questions you need answers to, please visit our online resource centres at : http://p2p.wrox.com/list.asp?list=csharptoday_discuss&WROXEMPTOKEN=620568Zhiox25QOrrsnTPfm53p and http://p2p.wrox.com/list.asp?list=c_sharp&WROXEMPTOKEN=620568Zhiox25QOrrsnTPfm53p


‘Trickle’ download and deployment of smart–client Windows Forms applications over the web

‘Trickle’ download and deployment of smart–client Windows Forms applications over the web
http://www.csharptoday.com/content.asp?id=1717
Article Rating: 4.5
Published on December 12, 2001
Abstract
The automatic 'trickle' download and deployment feature in .NET allows for painless and hassle-free deployment of smart-client Windows Forms applications and components in the server that can be downloaded and displayed on demand in the client machine. You can dream up innovative ways to serve up dynamically configurable applications that can morph themselves both in structure and behavior on the fly based on user preferences and credentials, all dynamically loaded from the server on demand. In this article, Aravind Corera shows you how to create and deploy windows forms applications and user controls to leverage the features provided by this deployment model.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

The days when rich client apps ruled...

I remember those days when I enjoyed building MFC GUI applications where one could add all the coolest tweaks from sliding toolbars, docking windows, custom navigation bars, or any other bells and whistles that one could possibly dream of. But alas, there was always a problem with the deployment model. Everytime we would add a new feature to keep up with the ever-changing demands of the user, we would ship the application off to them gloating with satisfaction and pride at how our users would feel when they saw that that nice new UI feature we'd added to the application. Little did we realize the beginning of a nerve racking, weekend-wrecking fortnight that was dedicated to solving the problems that users would face with the new application. In almost 95% of the cases, it turned out to be an issue with how the user installed the product, and most often had to do with all the problems brought about by what is called DLL Hell, which the .NET deployment model solves elegantly. Furthermore, such client applications require each of the user's desktops to be updated with the new application, which can be quite a pain - especially if you are deploying applications in a large enterprise.

Thin clients and the elegant deployment model that they bring to the table

To keep users from fretting or fuming from the nightmares of incremental updates, and from updating installations on each of the individual desktops, most companies gradually made a shift to a web based applications, where users use thin clients such as browsers to consume applications that would do most of the processing on the server and present the results to the client. Anytime there is a new feature addition to the application, all we'd need to do is update the applications in the server, and the user gets to use the new version of the application with no change whatsoever on the client. But here's the downside of this model - though technologies like ASP and dynamic web page generation technologies like DHTML can make your web pages look interactive, and user friendly, they really can't match the power and flexibility of rich client GUI applications that you can build with the MFC or Visual Basic, not to mention the hoards of spaghetti scripting code you have to keep up with to cope with a decent UI. Though ASP.NET solves the spaghetti-code issue elegantly (take look at code-behind assemblies for starters) and is a great technology for building web applications, Windows Forms are better suited for developing GUI intensive applications. The bottom line appears to be that we have to sacrifice the power and flexibility of rich client apps in order to provide our application's users with a trouble-free and seamless deployment model.

Let us see what we can do about that.

Back to the future - The .NET 'trickle' download deployment model

With the advent of .NET, we can deploy GUI applications such as Windows Forms applications over the web and update client applications for our users in a seamless and non-interventional manner. We no longer have to sacrifice the power and flexibility of those rich, powerful, and user-friendly GUI applications just to provide an easier deployment model to reach out to our customers.

The .NET automatic deployment model allows us to place assemblies on our web server that can be automatically downloaded on an as-needed basis whenever the client application requires the assembly. Assemblies trickle down to the client machine only when needed and hence the term trickle deployment. This downloaded assembly then executes locally on the client machine and is sand-boxed under the aegis of Code Access Security (CAS). An assembly is downloaded from the server only when needed (lazy-loaded) by the client. Updates can be applied to the assemblies on the server, and if the client application finds that the assemblies it contains are not in sync with the newer versions in the server, it downloads those newer versions from the server into the local download cache. This means that we no longer have to walk to each user's desktop individually updating client applications. There is no user intervention too, and the runtime automatically checks to ensure that the assembly version in user's local cache is the same as the one in the virtual directory in the web server.

System Requirements

  • .NET framework SDK Beta 2.
  • Visual Studio.NET Beta 2
  • Windows 2000 (all flavors).
  • IE 5.x or higher in the client.

Seeing 'trickle' deployment at work...

Let's put together a simple application that demonstrates how this works. Let's assume that we have an application that manages department-wide functions in an organization XYZ Inc. What we'd ideally do is to distribute a minimal shell application (a bootstrap application, if you will) to the users in the organization. This shell forms the basis for launching the rest of the application's UI depending on the user's department. The rest of the application is deployed in a remote web server and is loaded on demand depending on the user's department. This way, any updates to the application can be applied to the web server and the runtime automatically downloads and uses the newer version of the application on the server. If there are no updates, it continues to use the local version in the user's download cache.

Let's start by putting together a minimal shell or bootstrap application that will launch the rest of the application on demand. The shell application in our example is a windows forms application that gets the user id to check which department the user belongs to. After authenticating the user, it makes an attempt to launch a Windows Form that pertains to the user's department

private void login_Click(object sender,
            System.EventArgs e)
{

// Extract the user name, domain
            name, and password
// ...

// Perform authentication to
            ensure the right credentials were supplied
if(checkPassword(strDomainName,
            strUserName,strPassword))
{
// Check if the user is from
            sales. (In our example we'll use the user's
// domain name in
            the login id to determine the department that they
// belong
            too. These values are hardcoded for the sake of brevity.
//
            Ideally, you would read these values from a configuration file or
// maybe even obtain this information dynamically from a
            webservice)

if(strDomainName == "sales.xyz.com")
{
//
            Load the Sales form
LazyLoadXYZSales();
}
else
            if(strDomainName == "recruit.xyz.com")
{
// Load the HR
            form
LazyLoadXYZHumanResources();
}
else
{
MessageBox.Show("Incorrect
            domain name.");
}
}/* end if checkPassword
            */
else
{
MessageBox.Show("Incorrect UserID or password.
            Please try again");
return;
}
}

As seen in the code snippet above, the program determines whether the user is from Sales or HR, and accordingly calls a method to display the corresponding form. We'll see what the LazyLoadXYZSales and LazyLoadXYZHumanResources methods do as we go along. Let's create a new assembly that will host the Sales form so that we can deploy this assembly in a remote web server from which it will be downloaded on demand.

  • Create a new C# Class library project using Visual Studio.NET called Sales
  • Add a new Windows Form to the Sales project as seen below:

The Sales form would, in a real world application, allow the user in the sales department to view, manage and administer the company's sales data. In our example, we'll just throw in a few UI elements into the form just to demonstrate the fact that a Windows form can indeed be loaded off of a web server and executed in the client, seamlessly, without any of the pesky security warning dialogs or user intervention. In the section, Code access security policies enforced by the CLR on downloaded assemblies, we'll see how the .NET runtime uses Code Access Security policies to prevent assemblies downloaded from URLs that are not local file system URLs from performing any operations that might compromise the security of the system.

namespace SalesFormNS
{
public class SalesForm : System.Windows.Forms.Form
{
// Rest of the usual windows forms code goes here ...
}
}

Here's how the completed Windows form application looks - nothing much to talk about.

Repeat the above process to create a form pertaining to the Human Resources department, just as we did for Sales. Once we have the Sales and HR forms nicely squirreled away in separate assemblies named Sales.dll and HumanResources.dll, it's time to deploy them in a web server, so that they can be downloaded and executed by clients.

  • Using Internet Services Manager, create a new virtual directory called XYZInc
  • Make sure that the Execute permissions for the XYZInc virtual root are set to Scripts Only
  • (You can do this through the Virtual Directory property page associated with the virtual root's
  • properties)
  • Place Sales.dll and HumanResources.dll in the physical directory corresponding to XYZInc virtual root

With the above 3 steps done, we're all set for automatic 'trickle' deployment of these assemblies from the web server to the client. Let's see how a client can load the forms contained within these assemblies on demand. In our example, we'll load the Sales form from the Sales.dll assembly if the user is from the Sales department; or the HR form from the HumanResources.dll assembly, if the user is from HR.

Before we go any further, take a look at the download cache in the client's machine. You'll find the download cache in the Assembly/Download folder in your windows installation directory. Windows Explorer displays a custom view through the Fusion shell extension as shown below. Notice that there are currently no assemblies in the client's download cache:

It's now time to take look at the heart of the matter - the code that actually downloads and executes the windows forms in the client's machine. The LazyLoadXYZSales and LazyLoadXYZHumanResources methods in the shell application do just that. Take a look at the code for the LazyLoadXYZSales method:

/* Load the form that manages the company's sales */
void LazyLoadXYZSales()
{
// Load the assembly that contains the form for the Sales application
// (Replace the URL with the location where you've deployed your assemblies)
Assembly salesAssem = Assembly.LoadFrom("http://bingo/XYZInc/Sales.dll");

// Zero in on the type we need to load
Type typeSalesForm = salesAssem.GetType("SalesFormNS.SalesForm", true);

// Create an instance of the sales form
Object objForm = Activator.CreateInstance(typeSalesForm);

// Display the form and get ready to party with it
Form salesForm = (Form)objForm;
salesForm.TopMost = true;
salesForm.Show();
}

As seen above, the Assembly.LoadFrom method can be used to specify a remote URL location where the assembly is deployed. We need to obtain type information on the specific object's type that we are creating an instance of, for which we can use the Assembly 's GetType method. To get type information on the SalesForm windows form, we need to specify its fully qualified type name, SalesFormNS.SalesForm in the call to GetType. Once we get the type information, all we need to do is call the CreateInstance static method of the Activator class to instantiate the object. We can then cast the generic System.Object returned from Activator.CreateInstance to a Windows.Form object and then start calling its methods.

Take a look at the download cache again and what do we see? Notice that the Sales.dll assembly has been downloaded from the URL location specified in the Assembly.LoadFrom method and placed in the local download cache. All subsequent instantiation requests will use the assembly in the local download cache, unless the runtime notices that the version at the remote URL location is different from the one that is cached locally, in which case the newer version of the assembly is pulled down to the download cache. Let's check out this behavior by modifying the SalesForm in the server and redeploying it. We'll just add a PictureBox that displays a welcome banner on the SalesForm and then rebuild the assembly. Then, replace the Sales.dll in the XYZInc virtual directory in the server with the new version of the assembly that you just rebuilt.

Fire up your client application and log on as a user in the Sales department and what do you see - the new version of the form that was deployed in the server.

Peeking back again at the local download cache will show you that two versions of the Sales.dll assembly now exist in the local download cache. The runtime automatically detected the version of the assembly in the local download cache was out of sync with the one at the remote URL location and downloaded the newer version of the assembly for execution. That's .NET automatic deployment and side-by-side execution at work for you¡.

You will notice that the HumanResources.dll is never downloaded to the client application until there's an HR user who logs on using the application. This obviates the installation bloat of all modules on the client's machine, since assemblies are selectively downloaded and executed on demand. The LazyLoadXYZHumanResources method does just what the LazyLoadXYZSales method does, except that it downloads and executes the HRForm in the HumanResources.dll assembly.

Take note that if the Sales.dll and HumanResources.dll assemblies refer to other assemblies deployed in the web server, they can do so as if they were referring to locally deployed assemblies. The .NET runtime is smart enough to use the originating codebase of the downloaded assemblies to probe for other assemblies referenced by them and download them as well, when needed.

You could apply appropriate segmenting strategies to partition your application in such a way that for the most part, assemblies that provide the functionality for the most commonly used features in the application are local to the user's machine. The rest of the assemblies that provide functionality for rarely used features in the application can be downloaded on demand from the web server, if ever those features are used. Such infrequently used features should ideally be identified during the design phase of your application so that you can partition the individual assemblies that constitute your application appropriately for deployment.

Code Access Security (CAS) policies enforced by the CLR on downloaded assemblies

Security is a key issue that we must think about when using automatic 'trickle' deployment. This is because the application uses code that originates from sources other than your local system drive. Today, even the Intranet is no longer a safe haven. Even if the security of a single system on the local Intranet is compromised, there is a good probability that other systems in the Intranet could be sitting ducks for a security attack. So it's a good idea to enforce security restrictions on code originating from other machines within your own corporate LAN. That is precisely what the CLR does. Depending on where the assembly originated and the permission sets associated with the assembly, the CLR enforces the appropriate Code Access Security (CAS) security policies on these assemblies. To further your understanding on Code Access Security, look up Kaushal Sanghavi's Code Access Security article at http://www.csharptoday.com/content.asp?id=1717&WROXEMPTOKEN=620568Zhiox25QOrrsnTPfm53p.

The assembly will have a permission set assigned to it, based on the resultant common permission sets granted to the assembly at the following levels:

  • Machine policy level (MPL)
  • User policy level (UPL)
  • Application Domain policy level (ADPL)

Resultant Permission Set = MPL ¡É UPL ¡É ADPL

where ¡É is an intersection operation.

These policy levels actually contain nodes called code groups, which contain associated permission sets. You might want to learn more about permission sets, code groups, and policy levels by reading Keith Brown's MSDN article, Security in .NET: Enforce Code Access Rights with the Common Language Runtime at http://msdn.microsoft.com/msdnmag/issues/01/02/CAS/CAS.asp?WROXEMPTOKEN=620568Zhiox25QOrrsnTPfm53p. To view the list of code groups at these policy levels, we can use the CASPOL utility with the -listgroups option.

caspol -listgroups

This is how the output looks on my machine:

An assembly can originate from one of the following zones:

  • MyComputer zone
  • Intranet zone
  • Internet zone
  • Trusted sites zone
  • Untrusted sites (Restricted) zone

By default, locally installed assemblies that originate from the MyComputer zone enjoy complete freedom. Assemblies originating from the local Intranet zone are subject to certain restrictions such as manipulating the local machine's registry, accessing files on your local machine, and so on. The same applies to assemblies originating from Internet zone too, except that it's a little more restrictive. By default, assemblies from the Internet zone can connect back only to the server from which they originated. Assemblies originating from Trusted sites zones, by default, contain trust permissions that are identical those originating from the Internet zone. Assemblies originating from Untrusted zones are by default, not allowed to execute at all, period. For a complete discussion on each of these zones and the default security policies associated with assemblies originating from these zones, refer to

http://msdn.microsoft.com/library/en-us/cpguidnf/html/cpcondefaultsecuritypolicy.asp?WROXEMPTOKEN=620568Zhiox25QOrrsnTPfm53p

Whenever a downloaded assembly attempts to perform an operation that is defined to be restrictive by its permission set, the CLR immediately pitches in to prevent that operation and this results in a security exception getting thrown. This prevents lurking security attacks from rogue assemblies downloaded from the internet.

Let's check this out by using the following piece of code in the Sales form in the Sales.dll assembly.

try
{
// Attempt to read a file on the user's drive
FileStream fs = new FileStream(@"c:\MyData.dat",FileMode.Open,
FileAccess.Read);
}
catch (Exception ex)
{
MessageBox.Show(ex.Message,"Oops !. An error occured");
}

If we attempt to run this code from the Sales.dll assembly downloaded from the XYZInc virtual directory, this is what we get.

It turns out that the Sales.dll does not have permissions to perform file IO operations in the local machine. Let's quickly go in and check the default permission set for the Sales.dll assembly. We can do that with the CASPOL utility using the -resolveperm option.

caspol -resolveperm http://localhost/XYZInc/Sales.dll

Here's how the output looks (as usual, only a partial output is shown below since this option spits out reams of verbose XML):

Resolving permissions for level = Enterprise
Resolving permissions for level = Machine
Resolving permissions for level = User

Grant =
<PermissionSet class="System.Security.PermissionSet"
version="1">
<IPermission class="System.Security.Permissions.EnvironmentPermission, mscorl
ib, Version=1.0.2411.0, Culture=neutral, PublicKeyToken=b77a5c561934e089"
version="1"
Read="USERNAME;TEMP;TMP"/>
<IPermission class="System.Security.Permissions.FileDialogPermission, mscorli
b, Version=1.0.2411.0, Culture=neutral, PublicKeyToken=b77a5c561934e089"
version="1"
Unrestricted="true"/>

<!-- Lots of other stuff omitted for brevity -->

<IPermission class="System.Security.Permissions.ZoneIdentityPermission, msc
lib, Version=1.0.2411.0, Culture=neutral, PublicKeyToken=b77a5c561934e089"
version="1"
Zone="Intranet"/>
</PermissionSet>

A quick scan through the permission set will tell you that Sales.dll didn't have the permission to do a file IO operation on a local file since the assembly originates from the Intranet zone.

In the real world, our application installation's deployment package will automatically enforce the right policy levels and permission sets for the requisite assemblies. For now, we'll go ahead and manually increase the permission sets assigned to the Sales.dll assembly - in other words we'll trust Sales.dll with the good faith that it's just an innocent Sales administration application. To do that we'll use the .NET Admin tool, accessible through the MMC snap-in mscorcfg.msc. You'll find this snap-in in the Microsoft.NET\Framework\V1.0.2914 folder (for Beta 2) in your windows directory.

Choose the Increase Assembly Trust option, which brings up the wizard screen shown below:

We'll go ahead and enforce this policy at a machine-wide level so that we won't be bothered enforcing this level of trust for Sales.dll for every user who logs-on in this machine. So choose the Make changes to this computer option, as shown above.

The next screen prompts us for the URL location of the assembly, for which we need to provide the URL path to the virtual directory where Sales.dll has been deployed.

The next screen shown above allows us to choose if we additionally want to trust other assemblies with the same publisher certificate before it lets us into the screen that allows us to alter the trust levels.

Finally, we come to the screen that lets us make the appropriate changes to the permission set for the Sales.dll assembly. The Medium Trust in the Intranet zone does not allow assemblies to access files in the local file system, as indicated by the wizard. It's time now to place all our trust on Sales.dll from the specified URL location and pull up the slider to the Full Trust option.

After one last confirmation screen that displays the summary of changes made along the journey through the wizard screens, we hit the Finish button to confirm the changes.

Now try to execute the File IO code in Sales.dll that caused the security exception earlier, and it works without the slightest whine. The Full Trust option takes care of providing the requisite permission set to the Sales.dll assembly for accessing the files on the local machine. It's not always necessary to provide Full Trust to a downloaded assembly. We can pick and choose the appropriate permissions to compose our custom permission set, and associate that permission set with trusted assemblies.

Using 'trickle' deployment to display Windows Forms controls in our browser

In this section, we'll see how a Windows Form user control that's deployed at a remote URL location can be downloaded and displayed on a browser using the 'trickle' deployment feature in .NET. Let's put together a simple Windows Forms user control called ChristmasClock that displays the number of days until Christmas. So go ahead and create a new Windows Control Library C# project using Visual Studio .NET.

namespace ChristmasClockNS
{
public class ChistmasClockCtrl : System.Windows.Forms.UserControl
{
private System.Windows.Forms.PictureBox christmastreePicBox;

// Create a new Timer object
private System.Timers.Timer m_timer = new System.Timers.Timer();

// Initialize the default ChristmasClockCaption property
private String m_strCaption = "Christmas Clock countdown:";

// Rest of the designer generated code goes here ...
}
}

As shown above, we'll create a Timer object that can be used to trigger the refresh of the control's display area every 1 second. We'll expose a public property called ChristmasClockCaption that users can use to set the caption displayed on the clock.

public string ChristmasClockCaption
{
get
{
return m_strCaption;
}
set
{
m_strCaption = value;
}
}

Our first order of business is to get a timer started in the Load event of the control that will trigger the control to repaint itself every 1 second.

private void ChistmasClockCtrl_Load(object
            sender, System.EventArgs e)
{
// Associate a timer event
            handler for the Elapsed event
m_timer.Elapsed += new
            ElapsedEventHandler(this.UpdateChristmasClock);

// Update the
            clock every 1 second
m_timer.Interval = 1000;

// Start the
            timer
m_timer.Start();
}

private void
            UpdateChristmasClock(object sender, ElapsedEventArgs eArgs)
{
// Ask the control to repaint itself
this.Refresh();
}

We'll subscribe to the timer's Elapsed event by adding our event handler called UpdateChristmasClock, which is referenced by an ElapsedEventHandler delegate instance. The UpdateChristmasClock event handler does nothing more than asking the control to repaint itself, by calling its Refresh method. Take a look at the code for the control's Paint event handler:

private void ChistmasClockCtrl_Paint(object sender, System.Windows.Forms.PaintEventArgs e)
{
Graphics graphics = e.Graphics;

// Get the current date and time
DateTime currTime = DateTime.Now;

// Create a DateTime object representing Christmas Day
DateTime christmasTime = new DateTime(2001,12,25,0,0,0,0);

// Adjust the year just in case we got past Christmas 2001
while(currTime > christmasTime)
christmasTime = christmasTime.AddYears(1);

// How many more days for christmas presents ?
TimeSpan daysUntilChristmas = christmasTime - currTime;

String strDaysUntilChristmas = String.Format(
"{0}\r\n {1} days {2} hours {3} minutes {4} seconds",
m_strCaption, daysUntilChristmas.Days, daysUntilChristmas.Hours,
daysUntilChristmas.Minutes, daysUntilChristmas.Seconds);

// Fill the control's area with a white brush
graphics.FillRectangle(new SolidBrush(Color.White),0,0,Width,Height);

// Display the number of days until christmas
graphics.DrawString(strDaysUntilChristmas, new Font("Arial",12),
new SolidBrush(Color.Crimson),
this.christmastreePicBox.Right + 7, this.christmastreePicBox.Top + 1,
new StringFormat(StringFormatFlags.FitBlackBox));
}

When the control paints itself, it calculates the time span between the current time and Christmas day, and then formats the string accordingly to display it on the control. The controls area is first painted with a white background, and the string is displayed on it at a location offset by the width of the Christmas-tree bitmap in the picture box control.

That's pretty much what we need to do to get the ChristmasClock control up and running. Compile the control and deploy the control's ChristmasClock.dll assembly in the XYZInc virtual root directory that we created earlier.

Let's go ahead and create a HTML page that will consume this control. To test the sample code in this section, we'll need IE 5.x or higher on the client. Here's a simple HTML page that uses the <object> tag to specify the remote URL location of the user control - yes it's the same <object> tag used to create and host traditional ActiveX controls.

<HTML>
<HEAD></HEAD>
<BODY>

<OBJECT ID=ChristmasClockCtrl CLASSID=
http://bingo/XYZInc/ChristmasClock.dll#ChristmasClockNS.ChistmasClockCtrl WIDTH=500 HEIGHT=100>

<PARAM NAME="ChristmasClockCaption" VALUE="Rudolph the red nosed reindeer will be back in:">

</OBJECT>
</BODY>
</HTML>

The CLASSID attribute can be used to point to the remote URL location of the Windows Forms control from where it can trickle down to the browser. We need to specify the qualified type name of the control along with the URL. The type name usually follows the URL after a # delimiter. The PARAM tag can be used to specify the values of various properties for the control, which we can use to set the value of the ChristmasClockCaption property.

When the user navigates to the HTML page, the runtime will trigger the download of the ChristmasClock.dll assembly containing the ChristmasClock Windows forms control from the remote URL location into the local download cache, from where the control is loaded and displayed on the HTML page.

In the attached sample code, you'll find the following:

  • The LogonShellForm folder contains a minimal bootstrap Windows forms application that allows Sales and HR users to logon, which triggers the download of the rest of the application.
  • The TrickleDeployComponents folder contains the code for the Sales.dll and HumanResources.dll assemblies which house the forms used by the Sales and HR departments, respectively.
  • The ChristmasClock folder contains the code for the ChristmasClock Windows Forms control.

Parting thoughts...

As seen in this article, the automatic 'trickle' download and deployment feature in .NET allows for painless and hassle-free deployment of smart-client Windows Forms applications and components in the server that can be downloaded and displayed on demand in the client machine. You can dream up innovative ways to serve up dynamically configurable applications that can morph themselves both in structure and behavior on the fly, based on user preferences and credentials, all dynamically loaded from the server on demand. The age of deploy once, update everywhere rich client apps is here - thanks to .NET.


Demonstrating Attributes and Reflection with a Bug Tracker and Database Manager

Demonstrating Attributes and Reflection with a Bug Tracker and Database Manager
http://www.csharptoday.com/content.asp?id=1715
Article Rating: 4
Published on December 10, 2001
Abstract
With the inclusion of Reflection and Attributes within the .NET framework, Microsoft has taken software development to a new level. In this article, Todd Clemetson demonstrates how the combination of these two technologies can result in some generic and powerful software. After an introduction to .NET Attributes, he looks at two examples of attributes in action that, with some enhancement, can be utilized in your .NET programming project today: 1) a generic database manager for managing objects and 2) a custom Bug Tracking attribute along with a sample viewer application.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

With the inclusion of Reflection and Attributes within the .NET framework, Microsoft has taken software development to a new level. In this article, we will look at how the combination of these two technologies can result in some generic and powerful software. After an introduction to .NET Attributes, we look at two examples of attributes in action that, with some enhancement, can be utilized in your .NET programming project today: a generic database manager for managing objects, and a custom Bug Tracking attribute along with a sample viewer application.

Before we can explore the construction of both the generic database and bug tracking attributes, we need an understanding of what attributes are and how they are used. For supplemental information to this overview, please see the MSDN help packaged in .NET. To work with the code in this article, you will need the following tools:

  • .NET Framework Beta 2
  • SQL Server 7.0 or 2000
  • Visual Studio.NET Beta 2 (or a text editor)

What are Attributes?

.NET Attributes provide the means for a developer to add meta-data that describes, or annotates, specific elements of code such as classes, methods, properties, etc. At compile time, the resulting metadata is placed into the Portable Executable (PE)file, along with the Microsoft Intermediate Language (MSIL). Once metadata is in the PE, other .NET programs may access it using the .NET Reflection API.

Syntactically, attributes are applied using the [AttributeName([Parameters])] notation, where AttributeNameis the name of the attribute, and Parametersis a listing of parameters required by the Attribute's constructor. The following is an example of the .NET Serializable attributeapplied to a class. The Serializableattribute indicates that a class can be serialized.

[Serializable]
Public class
            MyClass
{
public MyClass()
{
// Nothing to see
            here.
}
. . .
}

.NET provides a number of attributes with .NET Beta 2. If you have tried your hand at writing Web Serviceswith .NET, you undoubtedly had to apply the [WebMethod] attribute to any methods exposed to your Web Service users.

Keeping in line with .NET's extensibility, developers may write custom attributes by sub-classing the System.Attributeclass.

Custom Attributes

Creating custom attributes is a fairly simple process. The steps are summarized below:

  1. Create the custom attribute by deriving a class from System.Attribute
  2. If required, define the properties associated with an attribute. Some attributes such as the Serializableattribute shown above do not have properties, and merely indicate a fact about some code.
  3. Define a Constructorfor the attributes that contains the necessary arguments (if any) to construct the attribute.
  4. Optionally, specify the scope of the custom attribute by specifying the AttributeUsage attribute(Yes, attributes can have their own attributes!)

The following is an example of a simple attribute that developers may use to indicate the stability of their code:

[AttributeUsage(AttributeTargets.Method,
AllowMultiple = false, Inherited = true)]
public class CodeUsabilityMeter : Attribute
{
// scale of 1- 10
private int _usabilityRating;
// the name of the developer
private string _developer;
/// <summary>
/// indicates how usable a section of code is
/// on a scale of 1 - 10.
/// </summary>
public int UsabilityRating
{
get
{
return _usabilityRating;
}
set
{
_usabilityRating = value;
}
}
/// <summary>
/// the name of the developer
/// </summary>
public string Developer
{
get
{
return _developer;
}

set
{
_developer = value;
}
}
/// <summary>
/// overloaded. constructs a CodeUsabilityMeter
/// Attribute
/// </summary>
/// <param name="iUsability">on a scale
/// of 1 - 10 how usable the code is
/// </param>
/// <param name="sDev">
/// the name of the developer
/// </param>
public CodeUsabilityMeter(int iUsability, string sDev)
{
this.UsabilityRating = iUsability;
this.Developer = sDev;
}
}

Note the AttributeUsageattribute specified at the beginning of the attribute definition. The AttributeUsageattribute defines how an attribute can be used within an assembly. The AttributeUsageattribute contains the following properties that control a custom attribute's usage:

  • ValidOn(type: AttributeTargets): Defines the places within the code where an attribute may be placed. Possible values are: Assemblies, Classes, Constructors, Delegates, Enumerations, Events, Fields, Interfaces, Methods, Modules, Parameters, Properties, Return Values and Structures. The default value for this parameter is All.
  • AllowMultiple(type: bool): If true, the attribute may be applied more than once to a specific code element. The default is false.
  • Inherited(type: bool): If true, the attribute is inherited by derived classes. The default is false.

Getting back to the CodeUsabilityMeterattribute: we derived from the System.Attributeclass, defined the properties specific to the attribute, and provided a constructor to initialize the attribute's properties.

Now that we have defined our custom attribute, lets put it to use. Here is an example of the CodeUsabilityMeterapplied to methods within a class:

public class MyClass
{
public
            MyClass()
{

}
// indicate that this code is as
//
            good as it gets
[CodeUsabilityMeter(10, "John Smith")]
public
            int SolidCode()
{
return 2 + 2;

}
// this attribute
            (with a ranking of 1) indicates that the probability of
//this
            code will
// defintely crash
[CodeUsabilityMeter(1, "Dave Q.
            Badprogrammer")]
public int ShakeyCode()
{
int i =
            0;
return 2/i;
}
}

Using .NET reflection and our newly created custom attribute, we could write code that evaluates all .NET assemblies within a project, and produces a report listing areas of code that aren't quite ready.

Now that we have a good foundation on .NET attributes, we will dig into some code that defines some custom attributes, and uses .NET reflection to interrogate attributes and do something with it. For a complete code listing of the examples used in this article, please follow the Download Support Material link at the end of this article.

Generic Database Manager

One common requirement of software development projects is the storage, retrieval, update, and deletion of business objects within a database. Quite often, business objects map directly to a table in an underlying database. Typically, a class maps to a table and the individual properties of the class map to the table's columns. Once developers create the required business objects, they must then write SQL to support add, retrieve, updateand deleteoperations.

In this first section, we will examine how .NET attributes can be used to achieve persistence of objects to a database without writing SQL statements for each business object.

In this section, we will:

  • Create three custom attributes used to describe how a class maps to a table in a database.
  • Develop a support class called SqlGeneratorthat uses .NET Reflection to inspect an object and create the appropriate SQL statements.
  • Develop an SqlObjectManagerclass that stores, retrieves, updates, and deletes rows from a SQL Server database given an object.
  • Examine code that exercises the object manager and its support classes.

The class diagram below shows the custom attributes we will design in the next section. Note that each custom attribute derives from System.Attribute:

Table Attribute

Our first attribute specifies the table that a class maps to. The DbTable attribute contains properties indicating the table name and owner. Here is the attribute's complete definition:

[AttributeUsage(AttributeTargets.Class,
AllowMultiple = false, Inherited = true)]
public class DBTable : System.Attribute
{
protected string _tableName = ""; // the name of the table
protected string _tableOwner = ""; // the owner (i.e. dbo)
/// <summary>
/// overloaded. Instantiates a copy of the DbTable Attribute
/// </summary>
/// <param name="colName">The name of the column</param>
/// <param name="colType">The type of the column</param>
public DBTable(string tableName, string tableOwner)
{
// set the instance variables
this.TableName = tableName;
this.TableOwner = tableOwner;
}
/// <summary>
/// The name of the column that this attribute describes
/// </summary>
public string TableName
{
get
{
return _tableName;
}
set
{
_tableName = value;
}
}
/// <summary>
/// The owner of this table in SQL server (i.e. dbo)
/// </summary>
public string TableOwner
{
get
{
return _tableOwner;
}
set
{
_tableOwner = value;
}
}
}

Note the AttributeUsage attribute prior to the class definition. This attribute indicates that the DbTableattribute 1) can only be applied to classes, 2) cannot be used more than once on a particular class, and 3) will be inherited by derived classes.

Column Attribute

The DBColumnattribute specifies which column a particular property represents in a database. This attribute contains the name and the data type of the column, as well as a flag specifying the "nullability" of the column (i.e. whether the column must contain a value). Here is the full definition of the DBColumnattribute:

[AttributeUsage(AttributeTargets.Property,
AllowMultiple = false, Inherited = true)]
public class DBColumn : System.Attribute
{
protected string _columnName = ""; // the name of the column
protected DbType _columnType = DbType.AnsiString; // type of column
protected bool _nullable; // specifies whether the column is nullable
/// <summary>
/// overloaded. Instantiates a copy of the DBColumn Attribute
/// </summary>
/// <param name="colName">The name of the column</param>
/// <param name="colType">The type of the column</param>
public DBColumn (string colName, DbType colType, bool bNullable)
{
// set the instance variables
this.ColumnName = colName;
this.ColumnType = colType;
this.Nullable = bNullable;
}
/// <summary>
/// Specifies whether the column is nullable
/// </summary>
public bool Nullable
{
get
{
return _nullable;
}
set
{
_nullable = value;
}
}
/// <summary>
/// The name of the column that this attribute describes
/// </summary>
public string ColumnName
{
get
{
return _columnName;
}
set
{
_columnName = value;
}
}
/// <summary>
/// The type of the column that this attribute describes
/// </summary>
public DbType ColumnType
{
get
{
return _columnType;
}
set
{
_columnType = value;
}
}
}

Notice that the AttributeUsageattribute indicates that DBColumnmay only apply to a property within a class.

Primary Key Attribute

The last attribute we will define is one that identifies the properties that map to the table's underlying primary key. Later in this article, we will write code that uses the primary key to formulate an appropriate whereclause. Here is the definition of the DBPrimaryKeyField attribute:

[AttributeUsage(AttributeTargets.Property,
AllowMultiple = false, Inherited = true)]
public class DBPrimaryKeyField: System.Attribute
{
protected string _columnName = ""; // the name of the column
protected DbType _columnType = DbType.AnsiString; // the type

/// <summary>
/// overloaded. Instantiates a DBUniqueRow attribute given
/// the column name and type
/// </summary>
/// <param name="colName">
/// the column name that is part of the primary key
/// </param>
/// <param name="colType">
/// the database type that is part of the primary key
/// </param>
public DBPrimaryKeyField(string colName, DbType colType)
{
// set the instance variables
this.ColumnName = colName;
this.ColumnType = colType;
}
/// <summary>
/// the name of the column that is part of the primary key
/// </summary>
public string ColumnName
{
get
{
return _columnName;
}
set
{
_columnName = value;
}
}
/// <summary>
/// the database type of the column that is part of the
/// primary key
/// </summary>
public DbType ColumnType
{
get
{
return _columnType;
}
set
{
_columnType = value;
}
}
}

Note that this attribute may only be applied to properties within the class, and that only one attribute can be applied to a particular property. The only properties of the DBPrimaryKeyFieldattribute are the ColumnNameand ColumnType.

Attributes Applied

Now that we have defined our custom attributes, let's look at a sample class that uses these attributes to specify how a class maps to a table within a database. The Studentclass below shows the above custom attributes in action.

[DBTable("Student", "dbo")]
class Student
{
private string _id; // id of the student
private string _firstName; // first name
private string _lastName; // last name
private string _address; // address
private int _numCourses; // number of courses

/// <summary>
/// The ID of the student
/// </summary>
///
// the student id is the primary key and identifies
// a unique row
[DBPrimaryKeyField("StudentID", DbType.String)]
[DBColumn("student_id", DbType.Guid, false)]
public string ID
{
get
{
return _id;
}
set
{
_id = value;
}
}
/// <summary>
/// the first name of the student
/// </summary>
[DBColumn("first_name", DbType.String, false)]
public string FirstName
{
get
{
return _firstName;
}
set
{
_firstName = value;
}
}
/// <summary>
/// the last name of the student
/// </summary>
[DBColumn("last_name", DbType.String, false)]
public string LastName
{
get
{
return _lastName;
}
set
{
_lastName = value;
}
}
/// <summary>
/// the address of the student
/// </summary>
[DBColumn("address", DbType.String, true)]
public string Address
{
get
{
return _address;
}
set
{
_address = value;
}
}
/// <summary>
/// the number of courses the student is enrolled in
/// </summary>
[DBColumn("num_courses", DbType.Int32, true)]
public int NumberOfCourses
{
get
{
return _numCourses;
}
set
{
_numCourses = value;
}
}

}

In the code above, we provide meta-data that indicates: 1) the table name that the Student classshould persist rows to (Studentin this example), 2) the attributes that indicate a unique row (there is only one key field called StudentID), and 3) the column names that the properties should map to (student_id, first_name, last_name, and address, respectively).

Let's look at some code that uses this metadata (i.e. the custom attributes) in a meaningful way. In the next sections we will write C# code using .NET reflection to generate SQL statements for a particular object. The class generating these SQL statements will be called SqlGenerator. We will also design an SqlObjectManagerclass that exposes a simple interface for managing objects in a SQL server database. The class diagram for these two classes is shown below:

Generating SQL using .NET Reflection

This is where you will start to see the beauty of Attributes and Reflection in action. We are going to write code that inspects objects and generates all of the required SQL statements for managing persistence. Additionally, we will write code to generate SQL used to define a table within SQL Server.

Let's start with a simple method that uses .NET Reflectionto determine the table name that a class maps to. This method is used quite a bit by other methods within the SqlGeneratorclass.

public string GetQualifiedTableName(object ob)
{
Type t = ob.GetType();
// iterate through each attribute in the list
foreach(Attribute att in t.GetCustomAttributes(false))
{
// grab the custom attribute
DBTable dta = att as DBTable;
// make sure it is valid
if(null != dta)
{
// if it is, return the owner.table_name. Note that
// we stop processing here because there can only be one
// table entry/type.
return dta.TableOwner +
"." +
dta.TableName;
}
}
// if we got here, there was not a table attribute specified
// for the type of object passed in. We need to throw an exception
throw new Exception("The DBTable attribute must be specified " +
"for the SQL generator to act on the object.");
}

The GetQualifiedTableNamemethod above iterates through the list of custom attributes applied to the object passed in, and attempts to cast the attribute to our custom DbTableattribute. If the attribute is a DbTableattribute, the method returns the fully qualified name.

Now let's take a look at the GenerateSQLInsertStatementmethod exposed by the SqlGeneratorclass. As the signature implies, this method generates the appropriate insert SQL statement to add a row to a table for an object.

public string
            GenerateSQLInsertStatement(object ob)
{
// initialize SQL
            statement
string strSQLStatement = "INSERT INTO ";
// grab the
            type
Type t = ob.GetType();
// grab the table name that we are
            inserting into
string strTableName =
            GetQualifiedTableName(ob);
// the statement me want to generate
            here is
// INSER INTO TABLE_NAME(COL1,COL2)
            VALUES('1',3958)
strSQLStatement +=
            strTableName;
strSQLStatement += " (";
strSQLStatement +=
            GetColumnList(ob) + ") VALUES(";
strSQLStatement +=
            GetValueList(ob) + ")";
// return the SQL statement.
return
            strSQLStatement;
}

This method initializes the INSERTstatement, extracts the fully qualified table name (using our GetQualifiedTableNamemethod), and appends the column and value lists to the SQL statement. Note that this method invokes two methods called GetColumnListand GetValueList. We will cover these methods next.

private string GetValueList(Object
            ob)
{
Type t = ob.GetType();
// will contain the list of
            values
string strValueList = "";
// flag to indicate whether
            or not this is the
bool bFirstColumn = true;
// first item in
            the value list.

// iterate through each
            property
foreach(PropertyInfo p in t.GetProperties())
{
//
            grab the custom attributes
foreach(Attribute att in
            p.GetCustomAttributes(false))
{
// grab the custom
            attribute
DBColumn dca = att as DBColumn;
// make sure it is
            valid
if(null != dca)
{
// if this is not the first column
            we
// have hit, then we need to preceed
// the column with a
            comma.
if(!bFirstColumn)
{
strValueList += ",
            ";
}
else
{
bFirstColumn = false;
}
// this sample
            code assumes properties
// are not collecections of
            objects.
// this dummy array of objects is
// required to
            satifsy the GetValue
// method.
object []obRetVal = new
            Object[0];
// append the value to the list
strValueList += "'"
            +
p.GetValue(ob, obRetVal).ToString() +
"'";
}
}
}
// return the list of columns.
return
            strValueList;
}

GetValueListstarts by iterating through all the properties within the object that was passed in. For each property, it iterates through each custom attribute. When the method encounters a DbColumn attribute, the value of the underlying property is appended to a comma-separated list. The resulting comma-separated list is returned to the caller.

I won't bore you with the full implementation of GetColumnList. It is identical to GetValueList, except it formulates a comma-separated column list by appending the DBColumn.ColumnNameproperty instead of interrogating the value of the property of the class as follows:

// if this is not the first column we have
            hit,
// then we need to preceed the column with a
            comma.
if(!bFirstColumn)
{
strColumnList += ",
            ";
}
else
{
bFirstColumn = false;
}
// apend the
            column to the list
strColumnList += dca.ColumnName;

The code in the download link at the bottom of this page contains methods for generating Delete, Update, Insert, and SelectSQL statements. If you wish to explore these methods, open the ObjectManagerproject and examine the GenerateSQLDeleteStatement, GenerateSQLUpdateStatementand GenerateSQLSelectStatementmethods.

Managing Objects in the Database

Now that we have a foundation for creating SQL statements for objects, we may now design an object manager that provides an interface for storing and retrieving items within a SQL Server Database. We will examine two methods from our SqlObjectManagerclass that use the custom SqlGeneratorclass to insert and retrieve objects from the database.

Before we jump into some detailed code, take a look at a partial view of the SqlObjectManager's class definition below:

(Note that the SqlObjectManageris geared to operate with the SqlClientdata provider).

/// <summary>
/// Manages DB objects that support the custom DB database
/// attributes
/// </summary>
public class SqlObjectManager
{
/// <summary>
/// the Connection to the database
/// </summary>
private SqlConnection _conn;
/// <summary>
/// Generates SQL statements for objects
/// </summary>
private SqlGenerator _sg = new SqlGenerator();
. . .
. . .
}

The SqlObjectManagerclass definition contains an SqlConnectionmember variable. This variable is accessed via the class's public Connectionproperty. The user of this class must specify the location of the underlying database by setting this connection property. Note that the class maintains an instance of SqlGenerator. This instance is used to formulate the actual SQL statements.

First we will add an object to the underlying database:

public void AddToRepository(object
            ob)
{
SqlCommand _comm = new SqlCommand();
// set the
            command text to the Insert SQL statement
_comm.CommandText =
            _sg.GenerateSQLInsertStatement(ob);
// set the connection
            object
_comm.Connection = _conn;
// lets insert the
            row
_comm.ExecuteNonQuery();
}

The above code block first creates a SqlCommandobject, and then sets the CommandTextproperty to the SQL statement generated by the SqlGenerator.GenerateSQLInsertStatementmethod we created earlier. The last line of code executes the SQL statement.

The RetrieveObjectmethod shown below populates an object's properties with data from a row within a table:

public void RetrieveObject(object ob)
{
// we can't construct a where clause unless the primary
// key is specified
if(this.DoesObjectHaveAPrimaryKeyAttribute(ob))
{
// given this has a primary key, the primary key must
// have all of its values set.
if(this.AreAllPrimaryKeyAttributesSet(ob))
{
SqlCommand _comm = new SqlCommand();
DataSet ds = new DataSet();
// put together the command
_comm.CommandText = _sg.GenerateSQLSelectStatement(ob);
_comm.Connection = _conn;
// create the adapter
SqlDataAdapter da = new SqlDataAdapter(_comm);
// fill the DataSet
int iNumRows = da.Fill(ds);
if(iNumRows > 0)
{
// set the class members with
// the data retrieved
this.SetClassMembers(ob,
ds.Tables[0].Rows[iNumRows - 1]);
}
else
{
throw new Exception("There was no row in " +
"the repository that matched the " +
"specified criteria.");
}

}
else
{
throw new Exception("All primary key properties " +
"must be set before retrieving a unique row.");
}
}
else
{
throw new Exception("Object must have a primary key " +
"to populate it with a unique row.");
}
}

This method must first verify that the object has at least one DBPrimaryKeyField attribute by invoking the DoesObjectHaveAPrimaryKeyAttributemethod. Next, the RetrieveObjectmethod must ensure that all of the properties marked with the DBPrimaryKeyFieldattribute actually contain a value. The AreAllPrimaryKeyAttributesSetmethod performs this logic.

Next, RetrieveObjectsets up the SqlCommandobject by invoking the SqlGenerator.GenerateSQLSelectStatement, followed by issuing the SqlDataAdapter.Fillmethod to populate a DataSet. Finally the properties of the class are set with the values retrieved from the database using the custom SetClassMembersmethod.

The last method we will cover within the SqlObjectManagerclass creates the underlying table that a class represents. This method is very useful for quick table creation:

public void CreateTableForObject(object ob)
{
SqlCommand _comm = new SqlCommand();
// put together the command
_comm.CommandText = _sg.GenerateCreateTableSQL(ob);
_comm.Connection = _conn;
// execute the create table sql
_comm.ExecuteNonQuery();
}

This above method creates an SqlCommandobject, sets the CommandTextand Connectionproperties, and finally executes the SQL that creates the table.

Putting it all together

Now that the SqlObjectManagerclass has been defined, we can write code to manage objects in the database. The following test code sets up the object manager, creates a Studenttable, and manages student records within the table:

static void Main(string[]
            args)
{
try
{
// used to manage our
            students
SqlObjectManager om = new SqlObjectManager();
//
            create a connection to the database
SqlConnection conn =
new
            SqlConnection("server=localhost;uid=sa;" +
"pwd=;database=mydb");
// open up the
            connection
conn.Open();
// set the connection property of the
            object manager
om.Connection = conn;
// create a new
            student
Student s = new Student();
// create a table
            definition for the student

Console.WriteLine("Creating
            student table");
om.CreateTableForObject(s);

// add 5
            students to the database
ExcerciseAddingStudents(om, 5);
//
            retrieve the 5 studens and write
// to the
            console
ExcerciseRetrievingStudents(om, 5);
// update the
            address field of the
// student object and have the
// object
            manager update the
// database
ExcerciseUpdatingStudents(om,
            5);
// retrieve the 5 studens and write
// to the
            console
ExcerciseRetrievingStudents(om, 5);
// clean up the
            database by deleting
// the 5 students we
            created
ExcerciseDeletingStudents(om, 5);
}
catch(Exception
            ex)
{
Console.WriteLine("------------------------------------");
Console.WriteLine("An
            Exception was caught:
            ");
Console.WriteLine("------------------------------------");
Console.WriteLine(ex.Message);

}

}

The code block above creates an instance of the SqlObjectManager, sets the Connectionproperty, creates the student table, adds five student records, retrieves the records, updates them, and finally deletes them. Here are the methods that the above code block invokes:

static void
            WriteStudentHeader()
{

Console.WriteLine("ID\tName\t\t\tAddress\t\t"
            +
"Number of Courses");
Console.WriteLine(new string('-',
            80));
}
static void WriteStudentDetails(Student
            s)
{
Console.WriteLine("{0}\t{1}\t{2}\t{3}",
s.ID,
            s.FirstName.Trim() + " " +
s.LastName.Trim(),
s.Address.Trim(),
s.NumberOfCourses);
}
static void
            ExcerciseDeletingStudents(SqlObjectManager om, int
            iNumStudents)
{
if(iNumStudents >
            0)
{
Console.WriteLine("Deleting {0} student
            records",
iNumStudents);
// iterate through each of the
            students and
// removes them
for(int i = 0; i <
            iNumStudents; i++)
{
// construct the student
Student
            storedStudent = new Student();
storedStudent.ID = i;
// delete
            the
            student
om.DeleteObject(storedStudent);
}
}
else
{
Console.WriteLine("The
            number of students requested " +
"was
            invalid.");
}
}
static void
            ExcerciseUpdatingStudents(SqlObjectManager om, int
            iNumStudents)
{

if(iNumStudents >
            0)
{
Console.WriteLine("Updating {0} student
            records",
iNumStudents);

// iterate through each of the
            students
for(int i = 0; i < iNumStudents;
            i++)
{

Student storedStudent = new
            Student();
storedStudent.ID = i;
// retrieve the
            student
om.RetrieveObject(storedStudent);
// update the
            student
storedStudent.Address = "Updated " + i;
// update the
            repository
om.UpdateRepository(storedStudent);
}
}
else
{
Console.WriteLine("The
            number of students requested " +
"was
            invalid.");
}
}
static void
            ExcerciseRetrievingStudents(SqlObjectManager om, int
            iNumStudents)
{
if(iNumStudents >
            0)
{
Console.WriteLine("");
Console.WriteLine("Listing {0}
            student records",
iNumStudents);
WriteStudentHeader();
//
            iterate through the list of students
for(int i = 0; i <
            iNumStudents; i++)
{
Student storedStudent = new
            Student();
storedStudent.ID = i;
// retrieve the
            object
om.RetrieveObject(storedStudent);
// write out the
            details
WriteStudentDetails(storedStudent);

}
}
else
{
Console.WriteLine("The
            number of students requested " +
"was
            invalid.");
}

}
static void
            ExcerciseAddingStudents(SqlObjectManager om, int
            iNumStudents)
{
if(iNumStudents >
            0)
{
Console.WriteLine("Creating {0} student records",
iNumStudents);
// lets create 50 studen records
for(int i
            = 0; i < 5; i++)
{
// create a new student
Student
            newStudent = new Student();
// set the student
            properties
newStudent.ID = i;
newStudent.Address = i + " Smith
            Drive";
newStudent.FirstName = "FirstName " +
            i;
newStudent.LastName = "LastName " +
            i;
newStudent.NumberOfCourses = i;
// add the student to the
            repository
om.AddToRepository(newStudent);
}
}
else
{
Console.WriteLine("The
            number of students requested " +
"was
            invalid.");
}
}

When executed, the test code above produces the following output:

Bug Tracking Attribute

Consider the following scenario. You recently released a patch to your latest Wizbang product, and one of your developers "fixed" a bug but introduced four more showstoppers that got past your QA department. Unfortunately the code modifications made to "fix" the bug span multiple assemblies and you don't know where to start. What if you could run a utility program that examined the source code and provided a listing of all the affected code related to this bug? Given your programmers have some discipline in using attributes to annotate source code, determining code changes could be accomplished at the click of a button.

The Issue Attribute

Let's define a custom .NET attribute that allows developers to identify areas of code affected by a bug fix. Our custom attribute will specify the issue number, name of the programmer, and a description of the change. Developers should have the ability to place these attributes on classes, methods, properties, enumerations, etc. Here is the attribute's definition:

[AttributeUsage(AttributeTargets.All,
AllowMultiple = true, Inherited = true)]
Public class BugFixDetails : System.Attribute
{
private string _issueID;
private string _programmerName;
private string _description;

public string IssueID
{
get
{
return _issueID;
}
set
{
_issueID = value;
}
}
public string ProgrammerName
{
get
{
return _programmerName;
}
set
{
_programmerName = value;
}
}
public string Description
{
get
{
return _description;
}
set
{
_description = value;
}
}

public BugFixDetails(string ID, string programmer, string desc)
{
this.IssueID = ID;
this.ProgrammerName = programmer;
this.Description = desc;
}

}

Note that the AttributeUsageattribute allows this attribute to be applied to any of the areas supported by .NET attributes. Here is some sample code containing the BugFixDetailsattribute.

[BugFixDetails("1", "John Q. Programmer",
"Changed name of class from Test to Test1")]
public class Test1
{
[BugFixDetails("2", "Suzie B. BugFree",
"Changed _var1 from type object to string")]
private string _var1;
private string _var2;

[BugFixDetails("2", "Suzie B. BugFree",
"Initialized variables in constructor")]
public Test1()
{
_var1 = "init value 1";
_var2 = "init value 2";
}
[BugFixDetails("3", "John Q. Programmer",
"Altered code to account for framework exception being thrown")]
[BugFixDetails("4", "John Q. Programmer",
"Added in code to rethrow exception being thrown from framework")]
public void Method1(int a, int b)
{
. . .
}
. . .
}

The Issue Viewer

Now let's look at a sample application that loads a .NET assembly and searches for all BugFixDetailsattributes matching a particular Issue ID. The screen shot below shows the results of a query for Issue ID #1. Note that the source code download contains a sample class library annotated with BugFixDetailsattributes.

The sample application above opens up the assembly, examines it for BugFixDetailsattributes and lists the following:

  • Scope- The element that the code change was applied to (i.e. class, method, enumeration, property, etc).
  • Name- The name of the element (method name, class name, etc).
  • Programmer- The name of the programmer.
  • Description- A description of the bug

When the user clicks the Find Details for Issuebutton, the following code is executed:

private void btnFindIssueDetails_Click(object sender,
System.EventArgs e)
{
try
{
// make sure they supplied an issue id
if(tbIssueID.Text.Trim() == "")
{
MessageBox.Show("You must specify an issue ID.");
return;
}
if(tbAssemblyFileName.Text.Trim() == "")
{
MessageBox.Show("You must specify the path and " +
"file name of the assembly to inspect.");
return;
}
// clear all the items in the list
listView1.Items.Clear();

// load the assembly so that we can
// interrogate it
Assembly assem =
System.Reflection.Assembly.
LoadFrom(tbAssemblyFileName.Text);
// iterate through each type in the assembly
// we loaded above
foreach(Type t in assem.GetTypes())
{
// add class level issue
// descriptions for this issue id
this.AddIssueDetailsForClass(t,
tbIssueID.Text);
// add method level issue
// descriptions for this issue ID
this.AddIssueDetailsForMethods(t,
tbIssueID.Text);
// add property level issue
// descriptions for this issue ID
this.AddIssueDetailsForProperties(t,
tbIssueID.Text);
// add constructor level issue
// descriptions for this issue ID
this.AddIssueDetailsForConstructors(t,
tbIssueID.Text);
}
}
// catch any exceptions
catch(Exception ex)
{
MessageBox.Show(ex.Message);
}
}

Once the code verifies that the Issue ID and assembly information is entered correctly, it loads the specified assembly and iterates through all of the types. Next, the code adds details for bug fixes within classes, methods, properties, and constructors. Let's examine the method that locates custom BugFixDetailsattributes at the methodlevel.

private void AddIssueDetailsForMethods(Type
            t,
string issueID)
{
// iterate through all the methods of
            this object
foreach(MethodInfo mi in t.GetMethods())
{
//
            grab the custom attributes for each method
foreach(Attribute att
            in
mi.GetCustomAttributes(false))
{
// cast to our custom
            attribute
BugFixDetails dets =
att as
            BugFixDetails;

if(dets != null)
{
// do these issue IDs
            match?
if(dets.IssueID == issueID)
{
// add this issue to
            the
            list
this.AddIssueItemToList("Method",mi.Name,
dets.ProgrammerName,dets.Description);
}
}

}

}
}

The AddIssueDetailsForMethodsfirst iterates through all of the methods within the type, then through all the custom attributes for each method. After verifying the custom attribute is indeed a BugFixDetailsattribute, the method looks at the IssueID of the attribute and, if the Issue IDs match, adds the item to a ListView.

The code that adds the item to the ListView is listed below:

private void AddIssueItemToList(string strScope,
string strName,
string strProgrammer,
string strDescription)
{
// create the listview item for this issue
ListViewItem lvi = new ListViewItem(strScope);
// add all the sub items
lvi.SubItems.Add(strName);
lvi.SubItems.Add(strProgrammer);
lvi.SubItems.Add(strDescription);
// lets add this to our list.
listView1.Items.Add(lvi);
}

The above code block simply creates a new ListViewItem, populates it with the bug details, and adds the item to the ListView control.

Conclusion

Attributes are a very simple, yet powerful part of the .NET framework. In this article, we discussed what attributes are, and the steps for creating them. Once we had a foundation in creating attributes, we designed custom attributes allowing developers to describe the usability of their code, created a framework for managing objects within a database, and provided a method allowing developers to mark sections of code that changed as a result of a bug fix.


Building a special purpose search engine

Building a special purpose search engine
http://www.csharptoday.com/content.asp?id=1708
Article Rating: 4
Published on November 29, 2001
Abstract
In this article, ST Tan & Aldy Karna use the web spider they developed in their previous article as part of a special purpose search engine. They show us how to use the web spider to extract relevant data and place it into a relational database, then how to query this database to get the information we want. This special purpose search engine allows us to focus on getting specific information from the web, with the example allowing us to search for jobs on some well-known sites.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

Although we don't normally think of the web as a database, it is often useful to use this analogy (although obviously databases are far more structured than the web!). People have tried to exploit this analogy by building SQL-like languages for web queries, for example WebSQL (http://www.w3.org/TandS/QL/QL98/pp/wql.html#websql?WROXEMPTOKEN=620568Zhiox25QOrrsnTPfm53p) and WebOQL (http://www.w3.org/TandS/QL/QL98/pp/wql.html#arocena?WROXEMPTOKEN=620568Zhiox25QOrrsnTPfm53p). Although there are similarities in the syntax, the actual algortihms are quite different. The difference is mainly because the web is like a large, unstructured and distributed kind of database. We can query a database, but we have to search the web.

In this article, we will follow a different approach, which consists of a two step process. First we extract information from the web and store it in a relational database, using the same ideas that we looked at in the earlier article, "A C# Web Spider written as a Web Service" (http://www.csharptoday.com/content.asp?id=1558&WROXEMPTOKEN=620568Zhiox25QOrrsnTPfm53p). We then use familiar database queries on this database to get the information we want.

Why not just use one of numerous existing search engines, one may ask? The answer is that we want to focus on searching for specific information on the web, such as collecting job opportunities from a few well-known career sites. Most of these sites require one to fill and submit a form to get information, and each of the sites may have different layouts for the form and result pages. A general-purpose search engine would not help us here, so in this article we will build a special-purpose search engine, written in C#, tailored to a few sites.

The techniques use the same navigation and web information extraction as in our earlier article, using simulation of GET and POST requests, and regular expressions. It is anticipated that the search engine would go out on the web periodically to collect new information. The results are stored in an Access database, which may be made available for end-user querying. Alternatively, the results could be sent to users via email or WAP/SMS. As the web grows bigger and bigger, we need to have programs to automatically search and present the information we need.

Similar techniques could be used for searching for specific news items, comparing flight or hotel prices, etc. An similar application that would be very interesting would be for a company's public relations department to collect articles and other posts on the web which mention the company or its product. In the last case, the engine would have to be extended to search newsgroups and chat rooms as well.

How it works

This program will search three sites for jobs in the field of information technology. The sites are:

  • http://www.jobsdb.com/id/?WROXEMPTOKEN=620568Zhiox25QOrrsnTPfm53p
  • http://www.jobstreet.com/?WROXEMPTOKEN=620568Zhiox25QOrrsnTPfm53p
  • http://www.binuscareer.com/jobseeker/?WROXEMPTOKEN=620568Zhiox25QOrrsnTPfm53p

All three sites use forms to search: the first and the third sites listed use the GET method, and the second uses the POST method. In addition, the program allows you to search by keywords, such as "programmer", "C#", "XML", etc. After the GET or POST is submitted, we get a summary page containing a listing of job opportunities available satisfying the keyword criteria. Each item in the list has a hyperlink to a page containing more detailed information. The program collects information from the summary and detail pages to get information on the starting and expiry dates (if any), the company, location, qualification, and years of experience required. Not all of these pieces of information will always be available.

These pieces of information correspond to the fields of an Access database, where the information will be stored.

When a search is repeated, the program will always verify whether the records it finds already exist in the database. This is useful when a connection times out and a search is repeated. If a site is to be visited periodically, say on a daily basis, only new records will be collected.

The above all sounds quite simple, but the difficulty lies in the lack of uniformity in the sites. As the location of the information we are looking for can vary wildly from site to site, we need to do some customized programming.

Database Diagram of DBJobs.mdb

Example of a summary page resulting from search

Overall program structure

UML Class Diagram (simplified)

In this program, the main entry point is ITJobSearch.aspx, which contains a form where we can select one or more of the job sites to be searched, as well as the keyword. You may recall that one of the main differences between an asp and an aspx page is the fact that an asp page is not a class, whereas an aspx page is a dynamically created class derived from the standard Page class of System.Web.UI. As such, it can naturally communicate with other objects of the classes in the same Namespace.

The methods in the aspx page are:

  • Page_Load(), which is the event handler when the page is loaded
  • DoSearch(), which is the event handler when the submit button is pressed

The three classes JobsDB, JobStreet, and BinusCareer correspond to the three sites to be searched, and they all inherit from an abstract class JobWebSite. This class has the following methods (some of them will be explained in more detail below):

  • InitSearch() is the method which constructs the query string in the case of GET, and the query body in case of POST
  • SearchData() is the main method which calls GetSummaryPage(), GetDetailPage(), and ProcessDetailPage()
  • GetSummaryPage() fetches the summary page using the class WebPage
  • GetDetailPage() fetches the detail page using the class WebPage
  • ProcessDetailPage() saves to database
  • FilterData() eliminates links to jobs which existed in the database

The class WebPage is similar to the identically named class in our previous article (http://www.csharptoday.com/content.asp?id=1558&WROXEMPTOKEN=620568Zhiox25QOrrsnTPfm53p), but with the two new methods, LoadPostPage() and LoadGetPage(). These are methods which return web pages in response to POST and GET methods of queries. The list of methods is:

  • LoadPage() fetches a web page by
              calling either LoadGetPage() or
              LoadPostPage()
  • LoadGetPage() fetches a page using the GET method
  • LoadPostPage() fetches a page using the POST method
  • CollectLinks() collects hyperlinks into an ArrayList to be followed later
  • CollectLink() is specific for JobStreet, where each single job needs to be processed immediately
  • CollectFilterLinks() is specific for JobsDB, to filter links containing some keywords
  • FilterLinks() excludes links such as images

The class JobWebSite will also use methods from the class Results to connect and write to the database. The methods of the Results class are:

  • FindJob() verifies whether the job already existed in the database
  • FindLastJob() finds the job with the largest JobID (specific for JobsDb)
  • SaveJob() writes to persistent storage
  • RemoveSpace() is a utility to remove spaces in strings
  • RemoveTag() is a utility to remove Html tags before writing to database

We are now ready to look at some of the code in more detail.

The aspx web form

For this application we actually have a choice of writing a console (DOS) application, windows form, web form, or a web service. All these types of applications can be written in the C# language, and in fact, they would not differ much internally from each other, the major difference would be the external interface, which could be a DOS command prompt, a Windows application, an aspx page, or a SOAP defined interface. We have decided to use a web form in this article.

ITJobSearch.aspx

We start by looking at the web form ITJobSearch.aspx. This is part of the downloadable code, please see there for a complete listing. Here, we will only take a look at the main parts.

The second line from the top is important, it declares an import statement of the Namespace. It looks like this:

<%@ Import Namespace="JobSearch" %>

The aspx page contains a form and a "Submit" button, which when clicked will call a JavaScript function DoCheck() for client side validation, and upon completion, will call the server-side event handler DoSearch().

The function DoCheck() looks like this:

function DoCheck()
{
if
            (document.frmSearch.txtKeywords.value.length==0)
{
alert("You
            must insert a keyword");
return false;
}
if
            (document.frmSearch.chkBinus.checked==false
            &&
document.frmSearch.chkJobsDb.checked==false
            &&
document.frmSearch.chkJobStreet.checked==false)
{
alert("You
            must select a site");
return false;
}
return
            true;
}

The function DoSearch() looks like this:

void DoSearch(Object sender, EventArgs
            E)
{
String Keywords=txtKeywords.Text;
if
            (chkBinus.Checked==true)
{
BinusCareer ObjBinus=new
            BinusCareer();
try
{
ObjBinus.InitSearch("GET","keywords",Keywords,"ksearch","1");
ObjBinus.SearchData();
if
            (ObjBinus.SearchJobData>1)
LblBinusCareer.Text="There are " +
ObjBinus.SearchJobData.ToString() + " new jobs in
www.BinusCareer.com";
else
LblBinusCareer.Text="There is "
            +
ObjBinus.SearchJobData.ToString() + " new job in
www.BinusCareer.com";
}
catch(Exception
            e)
{
LblBinusCareer.Text="Search Binus Career error => "
+ e.Message;
}
// code for other objects (JobsDb and
            JobStreet) are similar, and
omitted here
}
}

The Page_Load() event handler is only used only to set the time-out parameter. The main routine, DoSearch(), creates object(s) corresponding to the search requests, and then calls the methods InitSearch() and SearchData() of the class JobWebSite.

Some of the methods of the aspx webform are:

  • Page_Load() is called on the event Page_Load
  • DoSearch() is a delegate wrapping of SearchData()
  • SearchData() is the main program which will create and call objects of other classes

Simulating GET and POST methods

All the three job opportunity sites we use in this article require the user to fill in a form to search for jobs. Although they don't require userid and passwords, one needs to specify some search criteria.

GET and POST methods differ in the fact that with GET we put the list of attribute-value pairs (separated by the character "&") within the request URI as in the follows:

http://www.binuscareer.com/jobseeker/search-resp.php?opt_findustryId=1&opt_fpositionId=1&opt_compName=1&
opt_compSize=1&opt_minExperienceYr=1&opt_jobType=1&opt_locType=1&opt_empStatus=1&opt_salary=1&opt_eduGroup=1&
opt_eduDegree=1&opt_flocCountryId=1&opt_locCity=1&opt_skill=1&keywords=programmer&findustryId=11&fpositionId=13&
compName=&compSize=2&minExperienceYr=2&jobType=1&locType=1&empStatus=1&salaryMo=12&feduGroupId=5&degree=5&
flocCountryId=id&locCity=jakarta

Whereas with POST, the list of attribute-value pairs are in the contents of the request.

Incidentally, two of the sites in this example use GET, and one uses the POST method. For the receiving end of the request, i.e. the server, we are lucky, because ASP, ASP.NET, JSP, etc. will parse and put the attribute-value pairs in the WebRequest (or similar) collection object, regardless of whether GET or POST is used. On the sending side however, since we are simulating a form submittal, we have to write two different ways depending on the method.

protected void LoadGetPage() {
WebRequest
            Req = WebRequest.Create(Addr);
Req.Timeout=180000;
WebResponse
            Resp = Req.GetResponse();
StreamReader Sr = new
            StreamReader(Resp.GetResponseStream(),
            Encoding.ASCII);
StringBuilder PageBuffer = new
            StringBuilder();
String Line;
while ((Line = Sr.ReadLine()) !=
            null)
{
PageBuffer.Append(Line);
}
Sr.Close();
Page=PageBuffer.ToString().ToLower();
}

LoadGetPage() is easiest, as it is identical to the LoadPage() we used in the last article. Since the request URI has been prepared somewhere else (in InitString()), all we have to do is pass it in the normal way.

protected void LoadPostPage() {
String
            StrData=Addr.Substring(Addr.IndexOf("?")+1,
Addr.Length-Addr.IndexOf("?")-1);
WebRequest
            Req
            =
WebRequest.Create(Addr.Substring(0,Addr.IndexOf("?")));
Req.Method
            = Methods;
Req.Timeout=180000;

Req.ContentType =
            "application/x-www-form-urlencoded";
StringBuilder UrlEncoded =
            new StringBuilder();
Char[] Reserved = {'?','=',
            '&'};
byte[] SomeBytes = null;
int J=0;

if (StrData
            != null)
{
int
            i=0;
j+1;
while(i<StrData.Length)
{
J=StrData.IndexOfAny(Reserved,
            i);
if
            (J==-1)
{
UrlEncoded.Append(HttpUtility.UrlEncode
(StrData.Substring(i,
            StrData.Length-i)));
break;
}
UrlEncoded.Append(HttpUtility.UrlEncode(StrData.Substring(i,
            J-i)));
UrlEncoded.Append(StrData.Substring(J,1));
i =
            J+1;
}

SomeBytes =
            Encoding.UTF8.GetBytes(UrlEncoded.ToString());
Req.ContentLength
            = SomeBytes.Length;
Stream NewStream =
            Req.GetRequestStream();
NewStream.Write(SomeBytes, 0,
            SomeBytes.Length);
NewStream.Close();
}
else
{
Req.ContentLength
            = 0;
}

WebResponse Resp = Req.GetResponse();
Stream
            ReceiveStream = Resp.GetResponseStream();
Encoding Encode =
            System.Text.Encoding.GetEncoding("utf-8");
StreamReader Sr = new
            StreamReader( ReceiveStream, Encode );
StringBuilder PageBuffer =
            new StringBuilder();
String Line;
while ((Line =
            Sr.ReadLine()) !=
            null)
{
PageBuffer.Append(Line);
}
Sr.Close();
Page=PageBuffer.ToString().ToLower();
}

The LoadPostPage() method is used to make a post request. The WebResponse (lower) part in this program is identical to the part in LoadGetPage(), but the WebRequest object needs extending.

The standard method to make a POST request is described in the .Net Framework Quickstart "How Do I make a POST request?". We only need to make minor changes to the code. At the moment, iIt parses the addr string which contains the attribute-value pairs into a StringBuilder UrlEncoded, which is then converted to a string SomeBytes, which in turn is written to a Stream newStream of the WebRequest object req. In between those steps, we must use the method UrlEncode of the HttpUtility class of System.Web to ensure that punctuation marks, spaces, and other non-ASCII characters are sent and received correctly. In fact special characters are encoded as "%hh", where "hh" is the hexadecimal code of the character.

Where the actual hard work is

After the target page is fetched, we collect links on the page using the method CollectlLinks.

public void CollectLinks(String vPage)
            {
("href\\s*=\\s*(?:'(?<1>[^']*)'|\"(?<1>[^\"]*)\"|(?<1>[^>\\s]+))",
            Regex R;
Match M;
String Link;
R = new
            Regex("href\\s*=\\s*(?:'(?<1>[^']*)'|\"(?<1>[^\"]*)\"|(?<1>[^>\\s]+))",
RegexOptions.IgnoreCase|RegexOptions.Compiled);
for (M =
            R.Match(vPage); M.Success; M = M.NextMatch())
{
Link =
            M.Groups[1].ToString().ToLower();
Link=FilterLinks(Link);
if
            (Link.Length!=0)
BagOfLinks.Add(Link);
}
}

This collects the hyperlinks on the page. Next, we filter out unwanted links, including those pointing to uninteresting objects such as images, and links to jobs already in the database. The remaining links are then followed to get the detail pages where additional information about the jobs is described. Because the layouts of information used on the three sites are all different from each other, we write specific routines in the classes JobsDb, JobStreet, and BinusCareer. We will not go into all the codes here, as they can be found and easily followed in the support code. Instead, we will look at just one of the methods used for JobsDB, in order to get the flavour:

protected override void ProcessDetailPage(String vpage, String vLink) {
String JobId="", Company="", Position="", Qualification="";
String Skill="", Requirement="", Location="";
DateTime StartDate=DateTime.Now, EndDate=DateTime.Now.AddDays(14);
byte YearExperience=255;

// jobid
JobId=Regex.Replace(vLink,".+?R=(?<JobId>.+)","${JobId}");

// look for Position && StartDate && EndDate information
objrex=new Regex("<tr>\\s*<[^<>]+>\\s*<[^<>]+>\\s*<[^<>]+>\\s*<[^<>]+>\\s*<
[^<>]+>\\s*<[^<>]+>\\s*(?<1>[^<>]+)\\s*<[^<>]+>\\s*<[^<>]+>\\s*<[^<>]+>
\\s*<[^<>]+>\\s*<[^<>]+>\\s*<[^<>]+>\\s*<[^<>]+>\\s*<[^<>]+>
\\s*Post\\s*Date:\\s*(?<2>[^<>]+)\\s*<[^<>]+>\\s*</td>\\s*</tr>",
RegexOptions.IgnoreCase|RegexOptions.Compiled);
objm=objrex.Match(vPage);
if (objm.Success)
{
Position=Results.RemoveSpace(objm.Groups[1].ToString());
StartDate=DateTime.Parse(objm.Groups[2].ToString());
EndDate=StartDate.AddDays(14);
}

// Company
objrex=new Regex("<font\\s*face='Arial'\\s*Size='2'\\s*Color='#000000'>\\s*<b>\\s*<font\\s*color=#808080>\\s*(?<
1>[^<>]+)\\s*</font>\\s*</b>\\s*</font>",RegexOptions.IgnoreCase|RegexOptions.Compiled);
objm=objrex.Match(vPage); Company=Results.RemoveSpace(objm.Groups[1].ToString());// The code to find Company, Qualification,
Years of Experience, Skill and Requirements, and Location information on the page are similar to the above and have been omitted here,
all of them relies on regular expressions

// Qualification
objrex=new Regex("<b>\\s*Qualifications\\s*<[^<>]+>\\s*(?<1>[^<>]+)\\s*<[^<>]+>\\s*</b>",RegexOptions.IgnoreCase|RegexOptions.Compiled);
objm=objrex.Match(vPage); Qualification=Results.RemoveSpace(objm.Groups[1].ToString());

// YearExperience
objrex=new Regex("<b>\\s*Yr\\s*Exp:&nbsp;<[^<>]+>\\s*(?<1>[\\d]+)\\s*<[^<>]+>\\s*<
/b>",RegexOptions.IgnoreCase|RegexOptions.Compiled);
objm=objrex.Match(vPage);YearExperience=Convert.ToByte(objm.Groups[1].ToString());

// Skill & Requirement
objrex=new Regex("<li>[^<>]+</li>",RegexOptions.IgnoreCase|RegexOptions.Compiled);
objmc=objrex.Matches(vPage);Skill=Results.RemoveTag(objmc[i].ToString());
else
requirement+="Requirement+="// Location
objrex=new Regex("<td[^<>]+>\\s*<[^<>]+>\\s*<[^<>]+>\\s*Job\\s*Location&nbsp;\\s*<[^<>]+>\\s*<
[^<>]+>\\s*</td>\\s*<td[^<>]+>\\s*<[^<>]+>\\s*<[^<>]+>\\s*<[^<>]+>\\s*(?<
1>[^<>]+)\\s*<[^<>]+>\\s*<[^<>]+>\\s*<[^<>]+>\\s*</td>",
RegexOptions.IgnoreCase|RegexOptions.Compiled);
objm=objrex.Match(vPage);Location=Results.RemoveSpace(objm.Groups[1].ToString());

if (Results.SaveJob(JobId,WebSiteCode,StartDate,EndDate,Company,Position,Qualification,YearExperience,Skill,Requirement,
Location,DateTime.Now)!=true)
throw new JobException("Job " + JobId + " is failed to save");
else
SearchJobData++;
}

As you can see, the programming is quite tedious, as it relies on regular expressions to get the information about jobid, position, start and ending dates, company, years of experience, skill requirements, and location.

Writing to persistence storage

The above method, ProcessDetailPage(), will also write to a database. We use an Access database for simplicity.

DBJobs.mdb structure

The database DbJobs.mdb has two tables, tblWebSites to store information about web sites to be visited, and tblJobs to store search results (only one method of the class is shown, others have been omitted).

public class Results {
public const String ConStr="provider=Microsoft.Jet.OLEDB.4.0; Data Source=C:\\SPSearchEngine\\DbJobs.mdb";
private static OleDbConnection MyConn;
private static OleDbCommand MyOleDbCommand;
private static OleDbDataReader MyOleReader;

public static bool SaveJob(String vjobId,byte vWebSiteCode,DateTime vStartDate, DateTime vEndDate, String vCompany,
String vPosition, String vQualification, byte vYearExperience, String vSkill, String vRequirement, String vLocation,
DateTime vDate)
{
int Rows=0;
MyConn = new OleDbConnection(ConStr);
MyOleDbCommand = new OleDbCommand("insert into tblJobs( JobId, WebSiteCode, StartDate, EndDate, Company,
[Position], Qualification, YearExperience, Skill, Requirement, Location, SearchDate) values('" + vjobId
+ "',"+ vWebSiteCode +",'" + vStartDate + "','" + vEndDate + "','" +
vCompany + "','" + vPosition + "','" + vQualification + "'," + vYearExperience
+ ",'" + vSkill + "','" + vRequirement + "','" + vLocation + "','" +
vDate.ToString() + "')", MyConn);
MyOleDbCommand.Connection.Open();
Rows=MyOleDbCommand.ExecuteNonQuery();
MyConn.Close();

if (Rows>=1)

return true;
else
return false;
}
}

Contents of the supporting material and installation

The supporting material contains all the code for this article. It is assumed that the application will be located in "c:\spsearchengine", set as a virtual directory called "spsearchengine".

A compiled JobSearch.dll file has also been included, which must be placed in the "bin" subdirectory of the root of the virtual directory, i.e. " c:\spsearchengine\bin". If you want to compile yourself, you can use the batch file buildDll.bat to do it, it contains the command:

csc /t:library /out:c:\spsearchengine\bin\JobSearch.dll BinusCareer.cs Results.cs WebPage.cs JobWebSite.cs JobsDB.cs JobStreet.cs

This compiles and produces a .dll file (the subdirectory must be created first). The above command should be changed if you have the "\bin" subdirectory in a different place. The resulting .dll does not need to be registered.

Finally the Access database dbJobs.mdb is also included. It is also assumed to be located in the directory "c:\spsearchengine".

Further work

There are a number of things that still need to be done in an actual production environment, including:

  • Make the user interfaces easier to use and more attractive
  • Add at least half a dozen sites to be searched
  • Add maintenance to the database to delete expired records
  • Add a schedule for the program to automatically fetch information

Add a user interface to the database, either by using a pull (user queries database directly) or push (send info by email or WAP/SMS) model, or both. This would enhance usefulness of the service to people by saving time and bandwidth.

Concluding remarks

In this article, we have seen how to create a special-purpose search engine, written in C#, which is tailored toward three job related web sites. We have looked at how the information is captured, how it is stored in an Access database, and how it is displayed to the user. We have also looked at other more subtle features, such as excluding unwanted objects, like images or links we have already stored.

We hope that the above description gives the reader some idea how the program works. Some of the code is very specific to the sites to be visited, but it is unavoidable given the specialized nature of the search. It should be noted that the sites are constantly changing, and so the application may need to be modified constantly to reflect these changes.


Writing a Messenger Application using C# - Part 2

Writing a Messenger Application using C# - Part 2
http://www.csharptoday.com/content.asp?id=1705
Article Rating: 4
Published on November 23, 2001
Abstract
In his first article, Saravana Kumar developed an assembly MessengerObject.dll, containing of all the functional units of a instant messenging application, and the messenger server, which exposed the public methods of the assembly to the remote clients using .NET remoting. In this article, Saravana develops a Windows Form messenger client application. We also see how to debug the application's SOAP messages.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

In our first article, we developed our assembly MessengerObject.dll, which contained of all the functional units of a messenger application, and the messenger server, which exposed the public methods of the assembly to the remote clients using .NET remoting. In this article, we will develop a messenger client application. This application will allow the end users to communicate with each other instantly via the server. For the user interface, we will create a windows form front end using the capabilities of .NET.

Let us first break down our tasks before proceeding with our client application:

  • Create an instance of the remote object "MessengerObject"
  • Authenticate the user.
  • Build our client with an initial set up (loading all online, offline contacts, and status messages).
  • Send and receive messages with friends.
  • Changing status ("Away", "Be right back", etc).
  • Log out of the server.

Now let us go through each of the above tasks in detail:

Creating an instance of the remote object "MessengerObject"

In Visual Studio .NET, create a new C# Windows Application and call it Client. Before developing our client application, we need to reference the following assemblies inside our application:

  • System.Runtime.Remoting
  • System.Windows.Forms,
  • and MessengerObject.dll (that we created on the first article)

To add the above references in Visual Studio .NET, go to solution explorer, right click on References and Add References. A dialog box will pop up. Select the first two System assemblies from the .NET tab and then click Browse to locate our MessengerObject.dll (present along with the code download).

We need to create an instance of the remote object MessengerObject. Remote objects can easily be created from a client by calling new. The framework contains enough intelligence to realize we are dealing with a remote object and will ensure an instance of the object gets created in the relevant remote application.

The Activator class of the System namespace contains two methods: CreateInstance and GetObject, that can also be used to create an instance of remote objects. The CreateInstance method can be used in place of new to create an object instance, while the GetObject method is normally used to connect to an object at a specified URL.

Since our application is going to communicate via HTTP, and we know our activation URL for our messenger server (http://localhost:1010/Messenger?WROXEMPTOKEN=620568Zhiox25QOrrsnTPfm53p -highlighted in our first article), we will use the GetObject method of the Activator class in our application to create an instance of the remote object.

Below is a code snippet that will create a remote object called objMessengerObject.

using System.Windows.Forms;
using
            System.Runtime.Remoting.Channels.Http;
using
            System.Runtime.Remoting.Channels;
///.
///.
int
            listeningChannel = 0;
HttpChannel channel = new
            HttpChannel(listeningChannel);
ChannelServices.RegisterChannel(channel);
objMessengerObject
            =
            (MessengerObject)Activator.GetObject(typeof(com.Remoting.MessengerObject),Configuration.sRemotingURL);
///.

Even though the object is remotely activated here, we can use this object later on as a local object. The .NET remoting framework will take care of all the hard stuff behind the scenes such as creating a transparent proxy, communication with the original remote proxy on the server, etc. For more information on remoting see http://www.csharptoday.com/content.asp?id=1581&WROXEMPTOKEN=620568Zhiox25QOrrsnTPfm53p.

The second parameter Configuration.sRemotingURL of the GetObject method in the above code snippet is our activation URL, which is stored as a public static member inside our Configuration class. This is shown in the code snippet below:

///.
public class Configuration{
public static String sRemotingURL = "http://localhost:1010/Messenger";
///.

localhost is the name of the machine where our messenger server is running.

Authenticating the User

Users need to log in to the server in order to start communicating with friends. It's important to be sure that we are permitting users who are in our database to log into the server. As soon as we start our client application, the login form pops up. The validateUser method in the server is then called using our remote object objMessengerObject by passing the username and password as arguments.

Figure 1: User Authentication Screen

Below is a code snippet showing the call to the validateUser method:

///..
bool result;
result = objMessengerObject.validateUser(txtUserName.Text,txtPassword.Text);
///..

A boolean value of either true or false is returned from the server after querying the database with the user credentials supplied to the validateUser method. Once the user is successfully authenticated, a client object is created for the user by calling the constructor of the clsMainClient class with appropriate arguments, as shown below:

///.
objClient = new
            clsMainClient(txtUserName,objMessengerObject)
objClient.show();
///.

Since our client application is a windows form application, the clsMainClient class is inherited from the System.Windows.Forms.Form class.

Inside the clsMainClient() constructor, the addUser method of the remote object is called with the user ID and a label box control reference (the details of which are explained later), which allow the server to uniquely identify each online user. The method call is below:

///.
objMessengerObject.addUser (sUserAccount, ref this.lblComPoint);
///.

Building our Client

After successfully authenticating the user and adding the reference to the server, we need to build our main client with initial values (mentioned in the below bulleted list) so that it's ready for communication.

The client application needs to be presented with:

  • Users Online contacts
  • Users Offline contacts, and
  • Menus with status messages like "Away", "Be right back", "On phone", etc.

We perform the above mentioned tasks in the client load event of the clsMainClient class.

Inside the clsMainClient load event we call the method sendOnlineAndOfflineUserList using objMessengerObject by passing the user id as the parameter. This method returns an XML string with all online contacts, offline contacts, and status messages for the user after querying the database. This method call is below:

///.
string sOnlineOfflineXML;
sOnlineOfflineXML = objMessengerObject.sendOnlineAndOfflineUserList (sUserAccount.Trim());
///.

The return XML will look this sample:

<UserList>
<Online><U>julianyap</U></Online>
<Offline>
<U>mario</U>
<U>mike</U>
<U>nigel</U>
<U>helen</U>
</Offline>
<StatusList>
<S id="0">OffLine</S>
<S id="1">Available</S>
<S id="2">Busy</S>
<S id="3">Idle</S>
<S id="4">Be Right Back</S>
<S id="5">On the Phone</S>
<S id="6">Out to Lunch</S>
</StatusList>
</UserList>

Once we have the XML string, which contains online contacts, offline contacts, and status messages from the server, then we can use some of the XML classes in the .NET framework like XmlDocument and XmlNodeList to manipulate this XML string and set up the client. Let us have a look at the clsMainClient_Load event:

private void clsMainClient_Load(object
            sender, System.EventArgs e){
///..
//Fill lBoxOnline list box
            with online contacts
int
            i=0;
objDomeNodeList=objDomDocument.SelectNodes("UserList/Online/U");
foreach(System.Xml.XmlNode
            n in
            objDomeNodeList){
this.lBoxOnline.Items.Add(n.InnerText);

//To
            maintain user status (online or offline) so that we
//can check
            before sending the
            message
aUserIDS.Add(n.InnerText);

//populate context
            menu
this.cMenuSendIMsg.MenuItems.Add(n.InnerText,new
            System.EventHandler(this.cMenuSendIMsg_Click));

//create
            clsFriendsChatWindow for all online users,
//but don't display
            it
frmIW[i] = new
            clsFriendsChatWindow(n.InnerText);
frmIW[i].Name=n.InnerText;
frmIW[i].Text=n.InnerText;
aClientUserIDS.Add(n.InnerText.Trim());
aClientWindowIDS.Add(frmIW[i]);
frmIW[i].Show();
frmIW[i].lblStatus.Text=
            n.InnerText + " status is
            Available";
frmIW[i].picStatusDisplay.Image=image[0];
frmIW[i].Visible=false;
i=i+1;
}
///..
}

The clsMainClient_Load event does the following tasks for us:

  • 1. Picks up users' online friends from the <Online> tag of the XML string and stores them in the array list object aUserIDS. So we can easily check a friend's online status before sending any message to him / her.
  • 2. Adds the online contacts in the <Online> tag to the online contacts list box control (lblBoxOnline), so the user can pick up the friends from the list.
  • 3. Populates the context menu with online contacts, so that the user can easily send messages to a friend by selecting the friend's name from the context menu system tray icon.
  • 4. We need a separate messaging window for each friend, so we create an instance of clsFriendsChatWindow (inherited from the System.Windows.Forms.Form class) for every online friend. We do this by calling the constructor with the friend's id as an argument, adding the reference to the array list object aClientWindowIDS, and making the form invisible by setting up the form's visible property to false. The aClientUserIDS array list is used to support the fetching of the corresponding index of the friend's chat window from the array list aClientWindowIDS with the help of the friend's user id.

The same approach is used for offline friends as well, except we do not add the offline user to the array list aUserIDS and also not to the context menu.

Note: The final step in creating instances of clsFriendsChatWindow and making it invisible for every online and offline contact might not appear to be a feasible solution, as we can create an instance and show them when the user selects a particular friend to chat with. But, problems are encountered in the beta 2 release. When we try to show the clsFriendsChatWindow using the form's show() method inside the lblComPoint_TextChanged event, it hangs the whole application. It works when the event is triggered within the client application locally. But, when we try to capture this event from the server it hangs the whole application. So we take an indirect solution to the problem.

After loading online and offline contacts, we pick up the status messages from the XML string and populate the main menu and context menu. The code snippet, which does this task, is shown below:

///.
//Fill status messages in tray icon context menu and main menu bar
objDomeNodeList=objDomDocument.SelectNodes("UserList/StatusList/S");
foreach(System.Xml.XmlNode n in objDomeNodeList){
this.cMenuMyStatus.MenuItems.Add(n.InnerText ,new System.EventHandler(this.cMenuMyStatus_Click));
this.menuMyStatus.MenuItems.Add(n.InnerText ,new System.EventHandler(this.cMenuMyStatus_Click));
aStatusString.Add(n.InnerText);
}
///.

Figure 2: Context menu with system tray icon

After executing the code in the clsMainClient_Load event the client application will look like this:

Figure 3: Main Client Window

Now our client is set up and the user is ready to communicate with any of their friends listed in the online list box control (lblBoxOnline).

How is communication between server and client(s) is achieved?

So how are we going to communicate between the users? The idea behind this project is to catch an event in the client side that's triggered from the server. Let's see this in more detail.

Inside the constructor clsMainClient() we called the method addUser with two parameters sUserAccount and ref this.lblComPoint. The call to the addUser method stores the reference of the label box control (lblComPoint) passed to it in the server. The server will keep changing the text of this label box control reference in the server with appropriate XML strings for different actions such as when we send or receive messages between users, when we change the status message from the client application, etc. This will change the text of the label box control on the corresponding client application, which in turn triggers the lblComPoint_TextChanged event in the client. The text (XML string) assigned to the Text property of lblComPoint will be loaded into an XmlDocument object and manipulated accordingly within the lblComPoint_TextChanged event.

Send and Receive messages with friends

When a user click on any of their online friends IDs' on the lBoxOnline list box, the corresponding clsFriendsChatWindow object (instant messaging window) associated with that friend will be picked up from the array list aClientWindowIDS (manipulated in the clsMainClient_Load event) and displayed. In order to pick up the corresponding clsFriendsChatWindow object from the array list aClientWindowIDS we need the correct index. This index is obtained from another array list aClientUserIDS, where online friends IDs' are stored using the IndexOf() method of an array list.

The code for lBoxOnline_SelectedIndexChanged is shown below:

private void lBoxOnline_SelectedIndexChanged(object sender,System.Eventargs e){
clsFriendsChatWindow frmIWin
string s;
ListBox objLBox = new ListBox();
objLBox = (ListBox)sender;
s=objLBox.Text;

//pick up the corresponding friend window from
//aClientWindowIDS array list
frmIWin = (clsFriendsChatWindow)aClientWindowIDS[aClientUserIDS.IndexOf(s.Trim())];
frmIWin.RTBMessageDisplay.Text="";
frmIWin.Visible=true;
}

The above code will open an instant messaging window for a particular friend selected from the lBoxOnline list box:

Figure 4: Instant Messaging Window

Now, when the user enters some text in the text box and clicks the Send button, the following tasks are done one by one to deliver the message on the friend's instant messaging window:

1. The static method sendMessage (excerpt shown below) in the clsMainClient class is called, with the friend's ID and the text that needs to be sent to the friend as arguments.

private void btnSend_Click(object sender,
            System.EventArgs e){
if(RTBUserInputMessage.Text!=""){
string
            s,sUserName;
sUserName=this.Name;
s=clsMainClient.sendMessage(sUserName.Trim(),this.RTBUserInputMessage.Text.Trim());
this.RTBUserInputMessage.Text="";
this.btnSend.Enabled=false;
}
}

2. Inside the static method sendMessage, the sUserName (friend ID) is checked to see whether the friend is online by querying aUserIDS array list (because sometimes the friend on the other side might have gone offline when the instance window of them is kept open on the user side). If the friend is online, an XML string (which acts as the message) is formed using the friend's ID from the instant messaging window as the <T> element. Text that needs to be sent to the friend is placed in the <B> element and the user ID in the <F> element. The structure of the XML will look like this:

<M>
<F>julianyap</F>
<T>saravana</T>
<B>julianyap Says: \n Hello there</B>
</M>

The structure is not self-explanatory. One may prefer the actual XML to look more like this:

<Message>
<From>julianyap</From>
<To>saravana</To>
<Body>julianyap Says: \n Hello there</Body>
</Message>

But since the message is transferred regularly, putting extra characters will unnecessarily increase the size of the XML string.

3. After formatting the XML string (message), the getMessage method (excerpt shown below) of our remote object is called by passing this XML string (message) as the argument:

///.
s=objMessengerObject.getMessage(sMessageXML);
///.

This is the main method, which does the job of transferring messages from one client application to another.

4. On the server side, this XML string (message) will be assigned to the corresponding friend's label box control reference Text property. The correct label box reference is selected in the server with the help of the <T> element present in the XML message, which contains the friend's ID.

5. This Text property change, of the label box control reference in the server, will cause the label box Text property on the friend's client application to change, which in turn will trigger the lblComPoint_TextChanged event on the friend's client application.

Now we will look at how the lblComPoint_TextChanged (excerpt shown below) event is handled in the client application. Since the label box text is now an XML string, we can load that string into an XmlDocument object.

private void lblComPoint_TextChanged(object
            sender, System.EventArgs e){
///..
string
            sXML=this.lblComPoint.Text;
oDomDocument.LoadXml(sXML.Trim());
oDomNode
            =
            oDomDocument.SelectSingleNode("M");
///..
///..
sUserTo=oDomNode.SelectSingleNode("T").InnerText;
sUserFrom=oDomNode.SelectSingleNode("F").InnerText;
sBodyText=oDomNode.SelectSingleNode("B").InnerText;
///.
///.
frmIWin=
            (clsFriendsChatWindow)aClientWindowIDS[aClientUserIDS.IndexOf(sUserTo.Trim())];
frmIWin.RTBMessageDisplay.AppendText(sBodyText
            +"\n");
frmIWin.Visible=true;
///..
}

6. The corresponding clsFriendsChatWindow object associated with the friend in the "<F>" element will be picked up from the array list aClientWindowIDS using the index obtained from another array list aClientUserIDS.

Once the clsFriendsChatWindow object is identified (frmIWin in the above code snippet) then the body text from the user present in the <B> of the XML message can be appended to the rich text box RTBMessageDisplay, so the message appears on the friend's instance window.

Changing Status

If the user changes their status (example: "Away ", "Be right back ", etc) in the client application, then the status change needs to be sent to all of their friends who are also online. In the client application the user can change their status in two places, from the context menu, or from the main menu (Figure 2). Both of these selections trigger the same event cMenuMyStatus_Click. Inside the cMenuMyStatus_Click event the sendStatusChangeToAllFriends method (excerpt shown below) is called using our remote object objMessengerObject. On the server, this method will form an XML string as shown below and will be sent to all of the user's online friends' client applications.

<M><S m='1' t='Away'>julianyap</S></M>

///.
objMessengerObject.sendStatusChangeToAllFriends(sUserAccount.Trim(),aStatusString.IndexOf(sUserStatus));
///.

In a similar way to what we have seen while sending and receiving messages, this XML message will be handled by the lblComPoint_TextChanged event.

Logout of the server

When the user logs out of the server, either by closing the application or by clicking on the sign out button in the menu, we need to notify all of the user's online friends about the user's new status (user is going offline) and the resources occupied by the user on the server need to be killed. The removeUser method (excerpt below) of our remote object does this job for us. Although there are number of options for the user to log out of the server, (i.e. the user can log out either by closing the client application, or by clicking Sign out in the context menu or main menu), we'll call the same method in all those places.

///.
objMessengerObject.removeUser(sUserAccount);
///.

Additional information

Dynamic icons:

We need to change icons dynamically inside our application in many places, as different options are changed or chosen (example: system tray, client status bar). Even though it's not necessary, it will give a rich appearance to the client. To do this, we will store the icons in an Icon array so that it will be more efficient than retrieving from a file. The code to achieve this is below:

///.
//icons stored in an icon
            array
icon[0]=new Icon(Configuration.sGraphicsDir +
            "onlinewhite.ico");
icon[1]=new Icon(Configuration.sGraphicsDir +
            "offlinewhite.ico");
icon[2]=new Icon(Configuration.sGraphicsDir
            +
            "awaywhite.ico");
///.

///.
///usage
this.nIcon.Icon=icon[0];
this.statusBarPanel1.Icon=icon[0];
///.

Dynamic Menus:

We created dynamic menus in our application (example: The main menus as well context menus, which are created based upon the values from the database instead of hard coding). Actually there isn't much difference in code between these two menu types. To create a main menu in our application, double-click on the Main Menu icon from the toolbox. Similarly, double-click on the Context Menu icon from the toolbox to create the context menu. We will have a look at code snippets, which dynamically add menu items:

///.
///adds status messages into main and context menus
this.cMenuMyStatus.MenuItems.Add(n.InnerText ,new System.EventHandler(this.cMenuMyStatus_Click));
this.menuMyStatus.MenuItems.Add(n.InnerText ,new System.EventHandler(this.cMenuMyStatus_Click));
///.

System Tray Icon:

In this messenger client application we added a system tray icon to the system tray. Visual Studio .NET eases the job of creating a tray icon. From the toolbox, double click on the NotifyIcon tool and Context Menu tool. On the property page of NotifyIcon, set the contextmenu property to the name of the context menu we added from the toolbox, and select an icon for the Icon property. The system tray icon with context menu is now ready for our application. You can load icons dynamically at runtime as shown below in the code snippet:

///.
icon[0]=new
            Icon(Configuration.sGraphicsDir +
            "onlinewhite.ico");
///.
///.
this.nIcon.Icon=icon[0];
///.

What's happening behind the scenes

We can use the remoting debugging tool developed by Mathew Reynolds in http://www.csharptoday.com/content.asp?id=1525&WROXEMPTOKEN=620568Zhiox25QOrrsnTPfm53p to see what's happening behind the scenes. The screen shot below is taken from his utility when our client tries to authenticate our user.

Figure 5: Remoting debugging tool with SOAP messages

To make our client application work with the debugging tool do the following:

  • 1. Remove the reference "MessengerObject " and reference it back again.
  • 2. Change the sRemotingURL static member inside the configuration class to http://localhost:1009/Messenger?WROXEMPTOKEN=620568Zhiox25QOrrsnTPfm53p instead of http://localhost:1010/Messenger?WROXEMPTOKEN=620568Zhiox25QOrrsnTPfm53p.
  • 3. Run our server "MessengerServer.exe "
  • 4. Run the Debugging tool ("WroxTrace.exe ")
  • 5. Run our client application ("Client.exe ").

All the above-mentioned files are available inside our download for this article. Now you can see the SOAP messages flowing between the client and the server as shown in figure 5. You can use this tool for debugging, to check what's happening in between communication, so that you can improve the efficiency of the system.

Conclusion

In this article we have seen how to develop a messenger client application exploring some of the .NET concepts like remoting, and how to develop a sophisticated windows forms application. If you are more interested in knowing more about instant messaging you can visit http://www.imunified.com/?WROXEMPTOKEN=620568Zhiox25QOrrsnTPfm53p it's a forum formed for all instant messaging companies.

If you'd like to discuss a broader range of C#Today topics and issues, or if you have any C# related questions you need answers to, please visit our online resource centres at : http://p2p.wrox.com/list.asp?list=csharptoday_discuss&WROXEMPTOKEN=620568Zhiox25QOrrsnTPfm53p and http://p2p.wrox.com/list.asp?list=c_sharp&WROXEMPTOKEN=620568Zhiox25QOrrsnTPfm53p


Writing a Messenger Application Using C# - Part 1

Writing a Messenger Application Using C# - Part 1
http://www.csharptoday.com/content.asp?id=1703
Article Rating: 4.5
Published on November 22, 2001
Abstract
A messenger application provides users with a simple way to communicate instantly with each other. This article covers an approach to writing a sample realtime chat messenger application, much like the existing ones like MSN, Yahoo, AOL etc using C#. In this first part of this article, Saravana Kumar covers the data model, the core functionality assembly, the server, and how it is configured using .NET remoting to expose it to remote clients. The second part of the article will cover the client applications.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

A messenger application provides users with a simple way to communicate instantly with each other. This article covers an approach to writing a sample realtime chat messenger application, much like the existing ones like MSN, Yahoo, AOL, etc, using C#. In the first part of this article, we will cover the data model, the core functionality assembly, the server, and how it is configured using .NET remoting to expose it to remote clients. The second part of the article will cover the client applications.

We'll call our application 'Wrox Messenger Service.'

System Requirements

  • Visual Studio .NET Beta 2
  • Windows 2000
  • SQL Server (Version 7.0 or greater)

Architectural overview of Wrox Messenger Service:

Figure 1: WMS Architecture

WMS uses a client-server architecture. All communications between clients are routed through the server. The main advantage of this architecture is that complexity is moved from client to the server. This makes it relatively easy to update the functionality of the system and to make the client as thin as possible. The main disadvantage is that communication between clients is not secure, and the server is introduced as another possible point of failure.

WMS clients communicate in XML through HTTP. The XML used for communication is a SOAP message created through the use of a SOAP formatter (explained later).

Figure 1 shows the basic message flow architecture between clients. Now let's cover the core parts of the application:

  • Database model
  • "MessengerObject" Class
  • Messenger Server, and
  • The Client application

Database Model

In order to make it as simple as possible, let us keep the data model straightforward. We use four SQL Server tables, namely:

  • tblUser (Contains user information like name, id, password, etc)
  • tblFriendsList (Contains user contact list - Friends)
  • tblStatusMessages (to store status messages like Away, Offline, Online, etc)
  • tblLoginStatus (Core table - acts as a lookup table for all communication. Updated whenever the user log into the server, log out of the server, and if user changes his/her status)

Figure 2: Data model diagram

Creating the "MessengerObject" Class (MessengerObject.dll assembly)

The MessengerObject class is the heart of our application, and we will spend most of the article explaining it. All the basic functionality required for the clients to communicate is implemented in this class.

We'll first list all the method names that we will implement on our "MessengerObject" class:

  • private string getConnectionString ()
  • private bool validateUser (string sUserID, string sPassword)
  • private bool changeStatusInDB (string sUserID, int iStatus, int iStatusCommand, int iObjRefNo)
  • private bool updateObjRefNoinDB (int iRefNo)
  • private string sendStatusList ()
  • private string userStatusMessage (string sUserID)
  • private void sendMessage (string message)
  • public string getMessage (string sMessage)
  • public void addUser (string sUserID, ref Label objLabel)
  • public void removeUser (string sUserID)
  • public string sendOnlineAndOfflineUserList (string sUserID)
  • public void sendStatusChangeToAllFriends (string sUserID, int iStatus)

The functional architecture of MessengerObject is shown below:

Figure 3: Functional overview of the MessengerObject class

Let us begin coding.

Create a new Visual C# - Class Library project called MessengerObject.

Rename the default class file to MessengerObject.cs and the class name to MessengerObject.

Include the following namespaces inside MessengerObject.cs file:

using System;
using System.Xml;
using System.Data;
using System.Data.OleDb;

Include the following private members inside the class, which are used for database handling:

//To handle database
private DataTable
            objDT;
private DataSet objDS;
private OleDbDataAdapter objDA
            ;
private OleDbConnection objCN;

//To maintain Database
            Connection string
private string sConnectionString;

Using XML classes

We will maintain a configuration XML file, containing configuration properties like the server name, database name, and connection string. This will make future development easier.

Below is an example:

<?xml version="1.0" encoding="utf-8" ?>
<Messenger>
<Database>
<ServerName>localhost</ServerName>
<DatabaseName>Northwind</DatabaseName>
<ConnectionString>Provider=SQLOLEDB;Data source=localhost;
Integrated Security=SSPI;Database=Northwind</ConnectionString>
</Database>
</Messenger>

In order to retrieve the connection string stored in the XML configuration (Config.xml) file, we need to use the XmlDocument and XmlNode classes in the System.Xml namespace. First load the XML file into the document object model using the load method, and use XPath syntax to retrieve the connection string. Store this in global variable sConnectionString. We can use this variable content wherever we need a connection string to connect to the database.

Add a method getConnectionString and include the code below:

Listing 1 getConnectionString method

private string
            getConnectionString(){
XmlDocument objDomDocument;
XmlNode
            objDomNode;
objDomDocument = new XmlDocument();
objDomDocument.Load("Config.xml");
objDomNode =
            objDomDocument.SelectSingleNode("Messenger");
sConnectionString=objDomNode.SelectSingleNode("Database/ConnectionString").InnerText;
return sConnectionString;
}

This stage of a messenger application is to allow the user to sign in to the server. The username and password provided by the user are checked against the database entry in the table tblUser, and a boolean value of either true or false is returned to the client (since it's a sample tutorial application, security aspects are not given importance, but when it comes to a real time environment, the password must be sent to the server using some form of encryption. You can read more about this at http://www.csharptoday.com/content.asp?id=1540&WROXEMPTOKEN=620568Zhiox25QOrrsnTPfm53p).

Add a method called validateUser and include the code below:

Listing 2 the validateUser method

public bool validateUser(string sUserID,
            bstring sPassword){
string sUserAuth;

//Query which will
            return some data if the supplied username and //password exists in
            the database
sUserAuth="select cUserID,cPassword from tbluser
            where cUserID='" +
sUserID.Trim() + "' and cPassword ='" +
            sPassword.Trim() + "'";

objDA = new
            OleDbDataAdapter(sUserAuth,GetConnectionString());
objDA.Fill(objDS,"user");
objDT =
            objDS.Tables["user"];

if (objDT.Rows.Count > 0)
return
            true;
else
return false;
}

Sending online-offline user list and status list to the client

Once the login process is completed, the next step is to send successfully logged in users an online/offline friends list, and their availability status messages from the database. The method sendOnlineAndOfflineUserList is used to perform this task. The friends and their availability form an XML string, which is sent back to the client application.

Add a method called sendOnlineAndOfflineUserList and add the code listed in listing 3.

Listing 3 the sendOnlineAndOfflineUserList method

public string
            sendOnlineAndOfflineUserList(string sUserID){
string
            sOnline,sOffline;
string sReturnXML="<UserList>";
string
            sOnlineXML="<Online>";
string
            sOfflineXML="<Offline>";
string sStatusList;
string
            sTemp=null;

//create xml for status messages - sendStatusList
            method queries the
//database and return a xml string with all
            available status messages
//in the database
sStatusList =
            sendStatusList();

//query to pick up online friends of the
            user
sOnline="select id,cUserID,cFirstName from tbluser where id
            in (select iFriendid from " &
"tblFriendsList f where
            f.id=(select id from tbluser where cUserID='" + sUserID.Trim() +"')
            " &
"and iFriendid in (select id from tblLoginStatus where
            bStatus=1 and iStatusID > 0))";

objDA = new
            OleDbDataAdapter(sOnline,GetConnectionString());
objDA.Fill(objDS,"Online");
objDT
            = objDS.Tables["Online"];

//Create one element for each
            online friend
foreach(DataRow objDR in objDT.Rows){
sTemp +=
            "<U>" + objDR["cUserID"].ToString().Trim() +
            "</U>";
}

sOnlineXML = sOnlineXML + sTemp +
            "</Online>";
sTemp=null;

//query to pick up offline
            friends of the user
sOffline="select id,cUserID,cFirstName from
            tbluser where id in (select iFriendid from tblFriendsList f "
            &
"where f.id=(select id from tbluser where cUserID='" +
            sUserID.Trim() +"') and iFriendid in " &
"(select id from
            tblLoginStatus where bStatus=0 or iStatusID =0))";

objDA =
            new
            OleDbDataAdapter(sOffline,GetConnectionString());
objDA.Fill(objDS,"Offline");
objDT
            = objDS.Tables["Offline"];

//Create one element for each
            offline friend
foreach(DataRow objDR in objDT.Rows){
sTemp +=
            "<U>" + objDR["cUserID"].ToString().Trim() +
            "</U>";
}

sOfflineXML = sOfflineXML + sTemp +
            "</Offline>";

//Combine all bits into one single xml
            string
sReturnXML=sReturnXML + sOnlineXML + sOfflineXML +
            sStatusList +"</UserList>";

return
            sReturnXML;
}

The resultant XML looks like this:

Adding the user to the server

After sending the contact list and status message list back to the client, we need to do some work on the server. The tasks are:

  • Keep track of the user in the server, by adding a reference to uniquely identify the user
  • Send the user's online status to all his/her friends
  • Update the database with the user's Online Status and unique reference index
  • Optionally, you can send a status message to the server console or event log

Add these two static members inside the MessengerObject class, which is going to keep track of the client references in the server.

static ArrayList aUserIDS = new ArrayList();
static ArrayList aUserReference = new ArrayList();

Add a method called addUser and include the following code in Listing 4.

Listing 4 the addUser method

public void addUser(string sUserID, ref
            Label objLabel){
bool userInsertStatus;
try{
aUserIDS.Add(sUserID.Trim());
aUserReference.Add(objLabel);

if(aUserIDS.Count >
            1){sendStatusChangeToAllFriends(sUserID,1); }

userInsertStatus=changeStatusInDB(sUserID,1,1,aUserIDS.IndexOf(sUserID));
sendMessage( sUserID + " -- has Joined") ;
}
catch
            (Exception objExp){Console.WriteLine(objExp.Message);}
}

Uses of the addUser method

I. Keeping track of the user in the server, by adding a reference

This is the first, as well as the most important, step in the application. So, how we are going to keep track of the client reference? We use the simple approach of referencing a label box control in the client application as a client reference, whose text will be changed constantly with XML strings. After client validation, the addUser method in the server will be called from the client by passing a label box control reference as one of the arguments. This reference will be added to an arraylist aUserReference by the addUser method.

The concept is simple; we are trying to throw an event on the client side from the server. In this case it is the label box text change event. When we change the text of the label box reference with suitable XML strings in the server, it automatically throws the label box text change event on the client, which will be handled by the client. The core parts of this application are these objects, which use the ArrayList class:

  • aUserIDS
  • aUserReference

These are collection objects, which can store any type of object inside its collection. In our example we use them to store a collection of strings (user ids) and label box control references.

II. Sending the user's online status to all their friends

Once we add the client reference in the server, we need to send the user's online status to all their friends who are also online. For example:

  • User 'bgates' logs into the server
  • 'bgates' has two friends 'julian' and 'mike'
  • 'julian' is online and 'mike' is offline when 'bgates' logged in
  • So, 'bgates' online status needs to be sent to 'julian'.

This is done within the method sendStatusChangeToAllFriends.

If 'bgates' is the first user to login to the server then there is no need to send a message to anyone, because its obvious nobody is online. So we don't need to call sendStatusChangeToAllFriends. We'll discuss this method a little later, after explaining more uses of the addUser method.

III. Updating the database with a user's online status and unique reference index assigned to the user in the server

After the user has logged into the server, we store the user information in the database. By default, the tblLoginStatus table will have one entry for each user in the tblUser table. The default values are 0 (meaning offline) for the bStatus field, 100 for the iStatusID field, and -1 for the iObjectRefNo field.

Figure 4: Default values

Figure 5: After the first user has logged into the server

Add a method called changeStatusInDB and include the following code:

Listing 5 the changeStatusInDB method

private bool changeStatusInDB(string sUserID,int iStatus,int
iStatusCommand,int iObjRefNo)
{
//iStatusCommand ## 0-delete;1-insert;2-update ##
//calls the stored procedure InsertStatusChange with required
//parameters. Used to insert user status changes in appropriate
//database tables

try
{
objCN = new OleDbConnection(GetConnectionString());
OleDbCommand objCMD = new OleDbCommand("InsertStatusChange", objCN);
objCMD.CommandType = CommandType.StoredProcedure;

OleDbParameter objPARAM1 = objCMD.Parameters.Add("@sUserID", OleDbType.VarChar, 15);
objPARAM1.Value = sUserID;

OleDbParameter objPARAM2 = objCMD.Parameters.Add("@status", OleDbType.Integer);
objPARAM2.Value = iStatus;

OleDbParameter objPARAM3 = objCMD.Parameters.Add("@statusCommand", OleDbType.Integer);
objPARAM3.Value = iStatusCommand;

OleDbParameter objPARAM4 = objCMD.Parameters.Add("@iObjectRefNo", OleDbType.Integer);
objPARAM4.Value = iObjRefNo;

objCN.Open();
OleDbDataReader objREADER = objCMD.ExecuteReader();
objREADER.Close();
objCN.Close();
return true;
}
catch(Exception objExp){
Console.WriteLine(objExp.Message);
return false;
}
}

IV. Sending a message to the Console

Finally (optional), you can send a message to the server console saying the user has logged in.

Add a method called sendMessage and include the following code:

Listing 6 the sendMessage method

private void sendMessage(string message){

Console.WriteLine ("Added Message : {0}", message);

}

Sending status changes to all of a user's friends

When a user logs in or out of the server, we need to inform their friends about the change. The method SendStatusChangeToAllFriends does the job for us. As well as sending online/offline notification, this function also notifies the user's friends of any change in the user's status in the client application. The message is passed to all of the user's friends in the form of XML.

For example, if 'bgates' changes his status to away in the client application, then this method will be called which in turn will send the XML message shown below to all of his friends who are online:

<M><S m='1' t='Away'>bgates</S></M>

Add a method called SendStatusChangeToAllFriends and insert the following code.

Listing 7 the SendStatusChangeToAllFriends method

public void
            sendStatusChangeToAllFriends(string sUserID,int iStatus){
//iStatus -- 1:online; 0:offline
//sMessage -- like away,out
            to lunch...etc

string sObjectRefNos;
string sMessage;

//update the datebase table with new status message the user
            selected
//by calling the stored procedure
            "updateUserStatus"
objCN = new
            OleDbConnection(GetConnectionString());
OleDbCommand objCMD =
            new OleDbCommand("updateUserStatus", objCN);
objCMD.CommandType =
            CommandType.StoredProcedure;

OleDbParameter objPARAM1 =
            objCMD.Parameters.Add("@sUserID",
OleDbType.VarChar, 5);

objPARAM1.Value = sUserID;

OleDbParameter objPARAM2
            = objCMD.Parameters.Add("@iStatusID",
OleDbType.Integer);
objPARAM2.Value = iStatus;

objCN.Open();
OleDbDataReader objREADER =
            objCMD.ExecuteReader();
objREADER.Close();
objCN.Close();

Label oUserTo = new Label();

//Query to get users
            online friends list
sObjectRefNos="select iobjectRefNo,iStatusID
            from tblloginstatus where bstatus <> 0 " &
"and id in
            (select ifriendid from blfriendslist where id= " &
"(select
            id from tbluser where cUserID ='" + sUserID+ "'))";

//get
            users current status message.
sMessage =
            userStatusMessage(sUserID);

objDA = new
            OleDbDataAdapter(sObjectRefNos,GetConnectionString());
            objDA.Fill(objDS,"ObjectRefNos");
objDT =
            objDS.Tables["ObjectRefNos"];

//retrieve user friends who
            are online from the database and change
//their reference label
            control text with the xml string formed here.

foreach(DataRow
            objDR in objDT.Rows){
oUserTo=(Label)aUserReference[(int)objDR["iobjectRefNo"]];
oUserTo.Text = "<M><S m='" + iStatus + "' t='"+
            sMessage +"'>" + sUserID + "</S></M>";
}
}

How does the function work?

We first query the database for the unique reference IDs of all of the user's friends stored in the table tblLoginStatus. Then we picks up the corresponding reference stored in the arraylist aUserReference. Then, we parse the references into an XML string.

This string will throw the text change events in the appropriate clients, which will be handled by the client application.

Add a method called userStatusMessage, which will be used inside the method SendStatusChangeToAllFriends to get the user's current status, and insert the below.

Listing 8 the userStatusMessage method

private string userStatusMessage(string
            sUserID){

string sUserStatusMessage;

//Query which
            will return users current status base upon the user
            id
sUserStatusMessage="select sMessage from tblStatusMessages
            where iStatusID=" &
"(select istatusid from tblloginstatus
            where id = " &
"(select id from tbluser where cuserid='" +
            sUserID + "'))";

objDA = new
OleDbDataAdapter(sUserStatusMessage,GetConnectionString());
objDA.Fill(objDS,"UserStatusMessage");

objDT
            = objDS.Tables["UserStatusMessage"];
DataRow objDR;
objDR =
            objDT.Rows[0];
return objDR["sMessage"].ToString();
}

Removing the user from the server when the user logs out

If the user logs out of the server, then all of the resources occupied by the user need to be removed. The method removeUser is used for this purpose. The steps taken inside this method are:

  • Send status change (offline) to all of the user friends
  • Remove unique reference from the server
  • Update database tblLoginStatus, which stores status id and object reference index number, and
  • Optionally send a message to the server console

The above steps are more or less the same as what we did with the addUser method.

Add a method called removeUser and insert the following code.

Listing 9 the removeUser method

public void removeUser(string
            sUserID){
int iUserPosition;
bool
            userDeleteStatus;
try{
sendStatusChangeToAllFriends(sUserID,0);

iUserPosition=aUserIDS.IndexOf(sUserID);
aUserReference.RemoveAt(iUserPosition);
aUserIDS.RemoveAt(iUserPosition);

userDeleteStatus=changeStatusInDB(sUserID,0,100,iUserPosition);
//In
            collections such as arraylist, the elements that follow the removed
//element move up to occupy the vacated spot, these
            changes
//need to be updated in the Database since we are
            maintaining the object
// ref number in the
            DB
updateObjRefNoinDB(iUserPosition);
sendMessage(sUserID + "
            -- Disconnected ") ;
}
catch(Exception
            ed){
Console.WriteLine(ed) ;
}
}

Add a method called updateObjRefNoinDB and insert the following code.

Listing 10 the updateObjectRefNoinDB method

private bool updateObjRefNoinDB(int
            iRefNo){
objCN = new
            OleDbConnection(GetConnectionString());
OleDbCommand objCMD = new
            OleDbCommand("updateObjRefNos", objCN);
objCMD.CommandType =
            CommandType.StoredProcedure;

OleDbParameter objPARAM1 =
            objCMD.Parameters.Add("@iRefNo",
            OleDbType.Integer);

objPARAM1.Value =
            iRefNo;

objCN.Open();
OleDbDataReader objREADER =
            objCMD.ExecuteReader();
objREADER.Close();
objCN.Close();
return
            true;
}

Note: The User registration part is not implemented here. The SQL script in the download contains scripts to populate the tables with initial values, such as user information, default status, etc.

Next we'll look at how to send messages between users. Once we finish this part the whole framework is complete. It's very simple to write the function that transfers message from one user to another. Add a method called getMessage and insert the following code.

Listing 11 the getMessage method

public string getMessage(string
            sMessage){
Label oUserTo = new Label();
XmlDocument
            oDomDocument = new XmlDocument();
XmlNode oDomNode;

string
            sUserTo;
string sUserFrom;
string sBodyText;
string
            sXML=sMessage;

oDomDocument.LoadXml(sXML.Trim());
oDomNode
            =
            oDomDocument.SelectSingleNode("M");

sUserTo=oDomNode.SelectSingleNode("T").InnerText;
sUserFrom=oDomNode.SelectSingleNode("F").InnerText;
sBodyText=oDomNode.SelectSingleNode("B").InnerText;

//Pick
            up the corresponding label box reference from the arraylist
//mentioned in the to field of the
            sMessage

oUserTo=(Label)aUserReference[aUserIDS.IndexOf(sUserTo.Trim())];

//Replace
            the text of the label box control with suitable xml
//string
            formed here

oUserTo.Text="<M><F>" +
            sUserTo.Trim() + "</F><T>" + sUserFrom.Trim()
+
            "</T><B>" + sUserFrom.Trim() + "
says:\n " +
            sBodyText.Trim() + "</B></M>";

return
            sMessage;
}

This function works in a very similar way to SendStatusChangeToAllFriends. The only difference between these methods is the XML structure. In SendStatusChangeToAllFriends the XML looks like this:

<M><S m='1' t='Away'>bgates</S></M>

But, in the getMessage method, the XML has a "from" element <F/>, a "to" element <T/>, and a "body" element <B/>. For example:

<M>
<F>bgates</F>
<T>saravana</T>
<B>bgates Says: \n Hello there</B>
</M>

This method is called from the client application with the appropriate XML string as the argument, which contains "from", "to", and "body" tags. The content of the body tag is sent to the client mentioned in the "to" field, by picking up the corresponding client reference from the arraylist aUserReference.

A Test Function to prove our concept

Let us create a dummy function as proof of concept (to trigger a client side event from the server).

Add a method called testFunction and insert the following code.

Listing 12 the testFunction method

public void
            testFunction(System.Windows.Forms.Label l){
l.Text="Well
            done...You got your framework to work successfully
            ";
sendMessage("Client call is processed")
}

Compile the MessengerObject class by pressing Ctrl + Alt +B.

This creates an assembly file called MessengerObject.dll.

That's it. The MessengerObject class is complete. Next we will work on the server part to expose this MessengerObject to remote clients.

Creating the Server

Create a new Visual C# - Console Application project and name it MessengerServer.

Rename the default class MessengerServerSettings.cs and delete all existing default code inside the file.

As we discussed earlier about the addUser method, the database table tblLoginStatus needs to be populated with default values for every registered user. The MessengerServerSettings class is used for this purpose.

In the file MessengerServerSettings.cs include the following namespaces:

using System.Data.OleDb;
using
            System.Data;
using System.Xml;
using System;

Create a class called MessengerServerSettings and include the following private member variables:

private DataTable objDT;
private DataSet objDS;
private OleDbDataAdapter objDA ;
private OleDbConnection objCN;
private string sConnectionString;

private XmlDocument objDomDocument;
private XmlNode objDomNode;

Add a method called getConnectionString and include the following code. This method just retrieves and returns the connections string that is stored in the config.xml file.

public string
            getConnectionString()
{
objDT = new DataTable();
objDS =
            new DataSet();
objDomDocument = new
            XmlDocument();
objDomDocument.Load("Config.xml");
objDomNode =
            objDomDocument.SelectSingleNode("Messenger");
sConnectionString=objDomNode.SelectSingleNode("Database/ConnectionString").InnerText;
return
            sConnectionString;
}

Add another method called initializeServer and include the following code. This method just calls a stored procedure in the database, which updates the table tblLoginStatus with default values.

public bool
            initializeServer()
{
try
{
//calls the stored procedure
            ServerStartupProcedures to populate default //values into the
            database tables
objCN = new
            OleDbConnection(GetConnectionString());
OleDbCommand objCMD = new
OleDbCommand("ServerStartupProcedures",
            objCN);

objCMD.CommandType =
            CommandType.StoredProcedure;

objCN.Open();
OleDbDataReader
            objREADER =
            objCMD.ExecuteReader();
objREADER.Close();
objCN.Close();
return
            true;
}
catch(Exception
            objExp){
Console.WriteLine(objExp.Message);
return
            false;
}
}

We don't need anything more inside this class. Let us now start with our MessengerServer class, which will use remoting to expose the MessengerObject assembly to remote clients.

With DCOM, we are used to calling methods on objects running on a server. The programming models are always the same if objects are used on the server or on the client. Without DCOM, we have to deal with ports and sockets, pay attention to the target platforms because of possibly different data representations, and build a custom protocol, where messages are sent to the socket so that we finally call some methods. DCOM handles all these issues for us. The replacement for DCOM is .NET remoting. In contrast to DCOM, .NET remoting can also be used in Internet solutions and is what makes it more powerful and flexible. DCOM is not flexible and efficient enough for use in Internet solutions. It's possible to adapt and extend every part of the architecture with .NET remoting, so it fits for nearly all remoting scenarios (for more information on remoting, see the links at the bottom of the page).

Put simply, a messenger application is just about moving information bits from point A to point B. So, how are we going to establish a connection between point A and point B? That's where remoting comes into the picture.

Figure 6: Message Flow Architecture

Since both the client and the server reside on different machines/processes, there needs to be a mechanism to establish a relationship between the two. Microsoft .NET remoting provides a rich and extensible framework for objects living in different machines/processes to communicate with each other seamlessly. .NET remoting offers a very powerful yet simple programming model and runtime support for making these interactions transparent.

The client interacts with the server with the help of the HTTP protocol. The message sent from the client to the server is a SOAP message transmitted over HTTP. So, we need to convert all the messages that need to be sent to the server in the form of a SOAP message. The SOAP formatter in the .NET remoting framework does the job for us.

All outbound messages are passed through the SOAP formatter (as shown in figure 6), where the message is changed into XML and serialized, and the required SOAP headers are added to the stream.

Next comes the channel. Channels can be thought of as transport mechanisms to pass messages to and from remote objects. All the method calls are passed to the server through the channel. A channel is a type that takes a stream of data, creates a package according to a particular network protocol, and sends the package to another computer / process. NET remoting supports two types of channel:

  • HttpChannel
  • TcpChannel

Since all our communication takes place via HTTP protocol, we use the HTTP Channel in our application.

The HTTP channel transports messages to and from remote objects using the HTTP protocol. The main advantages of an HTTP channel are:

  • Provides communication between sender and receiver using the HTTP protocol as a transport.
  • Provides encoding of payloads in SOAP, an XML encoding standard.
  • Is compliant with SOAP 1.1.

Let us start coding MessengerServer.

Add the following references to the application:

  • MessengerObject.dll
  • System.Runtime.Remoting

      Add a new file, MessengerServer.cs and include the following namespaces:

      using System;
      using System.Runtime.Remoting;
      using System.Runtime.Remoting.Channels.Http;
      using System.Runtime.Remoting.Channels;

      Have a look at the server code below. It's only few lines of code. Most of the complex parts like establishing a port to listen for the client request, processing the client request, converting the message into appropriate network messages, etc, are already done by the .NET remoting framework.

      namespace com.Remoting
{
public class
            MessengerServer {
public static void Main (){

//Initialize
            database settings
MessengerServerSettings objS = new
            MessengerServerSettings();
objS.InitializeServer();

//Register
            channel for remoting
int listeningChannel = 1010;
HttpChannel
            channel = new
            HttpChannel(listeningChannel);
ChannelServices.RegisterChannel
            (channel);
RemotingConfiguration.RegisterWellKnownServiceType(Type.GetType
("com.Remoting.MessengerObject,MessengerObject"),
"Messenger",
            WellKnownObjectMode.SingleCall);

Console.WriteLine
            ("Messenger Server is up and running on port {0}", isteningChannel);
Console.WriteLine ("Press enter to stop the
            server...");
Console.ReadLine ();
}
}
}

      In the first two lines, we set up the database with initial values.

      Then when the server is started, we need to create and register a HTTP channel that will listen for clients connecting on a specific port (in our case we use port 1010). We also need to register the remote object with the remoting framework by calling RegisterWellKnownServiceType. The parameters for this call are as follows:

      • The full type name of the object being registered, in this case com.Remoting.MessengerObject.
      • The assembly name
      • The name of the endpoint where the object will be published. Clients need to know this name in order to connect to the object. Any string will work; for this example, use "Messenger".
      • The object mode, which can be SingleCall or Singleton.

      We use SingleCall object mode while registering our service. This services one and only one request coming in at a time. Single call objects are useful in scenarios where the objects are required to do a finite amount of work, and are usually not required to store state information. Single Call objects can be configured in a load-balanced fashion.

      Although the server process knows everything about each unique Object type, the client knows only that it wants a reference to an object in another application domain, perhaps on another computer. A uniform resource locator (URL) will identify the server. The URLs that represent unique types to the outside world are activation URLs, which ensure that your remote call is made to the proper type.

      In our case, the URL of our server is http://localhost:1010/Messenger.

      Compile the application by pressing Ctrl + F5 which will compile and start an .exe file.

      Proof of concept (Sample client Application)

      In order to check our remoting set up and our concept of throwing client side events from the server, let us build a sample client program to illustrate. It's a simple windows form application with just a label box control and a button as shown below:

      Figure 7: Proof of concept Client application.

      The figures below show the status of the client window, alert message and the server console after clicking the button.

      As soon as the button is clicked, the text of the label box will be changed by the function testfunction in the server. Because of that, a text change event will be fired on the client, which pops up a message box saying text change event triggered. If this happens then everything is fine.

      Figure 9: Alert caused because of label box control text change event

      Figure 10: Message displayed on server console after processing client request

      Look at the code snippet below. Let us call the method testfunction with the label box reference as the argument:

      //Inside button click
            event

com.Remoting.MessengerObject objTest;
HttpChannel
            channel;
this.button1.Enabled=false;

int listeningChannel
            = 0;
channel = new
            HttpChannel(listeningChannel);

ChannelServices.RegisterChannel(channel);
objTest
            = (com.Remoting.MessengerObject) Activator.GetObject
(typeof(com.Remoting.MessengerObject),Configuration.sRemotingURL);

objTest.testfunction(this.label1);

//
            Inside Label box text change event

MessageBox.Show("text
            change event triggered ");

      What we've done, and what's next!

      In this article we've discussed our implementation assembly MessengerObject and the messenger server. Now, our server is ready to accept requests from various remote clients. In the next article we will develop a sophisticated client application to interact with the server. We will also discuss how to debug a remoting application like this, using the tool developed by Matthew Reynolds in his article at http://www.csharptoday.com/content.asp?id=1525&WROXEMPTOKEN=620568Zhiox25QOrrsnTPfm53p).

      • Brought to you by ownSky!


Visual Studio .NET Custom Wizards

Visual Studio .NET Custom Wizards
http://www.csharptoday.com/content.asp?id=1689
Article Rating: 4.5
Published on October 31, 2001
Abstract
Visual Studio .NET supports a number of built-in project types and wizards that generate skeleton projects and solutions. These project templates are an excellent way to automate the basic plumbing required for very common project types, such as Windows Applications, Class Libraries and Web Services. Given the wide diversity of developer needs, Visual Studio .NET exposes an extensible architecture, where developers can write their own custom project types and wizards. This is an excellent feature that allows developers to write canned skeleton code that can be generated automatically. One of the strongest motivations for writing custom wizards is to write skeleton code once, and have it automatically customized, generated, and added to further projects. In this article, Kaushal Sanghavi explores Visual Studio .NET’s support for building custom wizards, and discusses how developers can create their own wizards and project templates. Further, he shows us how to build a sample wizard that generates a class that maintains a thread pool holding a specified number of threads. Each of these threads will call a user specified method via the use of delegates. All the thread management will be handled by the generated code.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

Visual Studio .NET supports a number of built in project types and wizards that generate skeleton projects and solutions depending on the type of application the developer intend to build. These project templates are an excellent way to automate the basic plumbing required for very common project types, such as Windows Applications, Class Libraries and Web Services. However, Visual Studio .NET supports only those basic project types that are extremely common across the developer community. Given the wide diversity and disparity of developer needs, Visual Studio .NET (and for that matter, previous versions of Visual Studio as well) expose an extensible architecture, where developers can write their own custom project types and wizards. This is an excellent feature that allows developers to write canned skeleton code that can be generated automatically, without having to resort to the infamous "copy-paste" anti-pattern. One of the strongest motivations for writing custom wizards is to write skeleton code once, and have it automatically customized, generated, and added to further projects. In this article, we will explore Visual Studio .NET's support for building custom wizards, and discuss how developers can create their own wizards and project templates. Further, we will build a sample wizard that generates a class that maintains a thread pool holding a specified number of threads. Each of these threads will call a user specified method via the use of delegates. All the thread management will be handled by the generated code.

Note: this article is based on Visual Studio .NET Beta 2 and Windows 2000.

In this article, we use the words Wizards and Templates interchangeably.

Existing Projects Types and Wizards

The following screen shot shows some of the project templates and wizards that are supported by Visual Studio .NET by default.

When a project type and template is chosen, Visual Studio .NET creates a solution and project based on the user's selection, and generates some files that contain the skeleton code. Some project templates, like the Class Library template, generate very simple code that contains one class and a stub for its constructor, while templates such as Visual Studio .NET Add-ins or ATL Server generate quite a few files, each containing substantial code. This generated code is usually stored in a file with placeholders for user input values, such as the class and project name. When a specific project type is chosen, Visual Studio .NET invokes a component that reads these canned (also known as templated) files, and substitutes the placeholders with the user input values. Next, this component creates the required projects and solutions using the Visual Studio .NET Extensibility model. Visual Studio .NET looks for specific files in a fixed location that contain information about the available project templates, and the components that will be responsible for creating new projects based on that template. We will cover this architecture in the next section.

Visual Studio .NET Project Template Architecture

Visual Studio .NET uses a component based model to support the project templates. Each project template has a corresponding component that handles requests for the given template. A single component can handle requests for multiple templates as well. Visual Studio .NET uses two types of files that are used to identify the project templates, and the components that handle requests for these project templates. The first file (identified by a .vsz extension) contains information about the component, and any custom parameters that the component handling the project template might need. The second file (identified by a .vsdir extension) contains information about the description of the project type, and display properties in the New Project dialog box. Let us now explain these two files in detail.

  • VSZ File: This file contains information for a single project template. The name of the file corresponds to an entry in the .vsdir files (explained below), and the contents of the file contains the ProgId or the ClassId of the component that will be invoked when this specific project template is chosen. Further, this file can hold custom parameters that will be passed to the component, when invoked. These parameters are typically used when a single component handles multiple template types. The Visual Studio .NET IDE is responsible for parsing this file and passing the parameters to the handler component.

A sample .vsz file that handles C# Console Projects is shown below. The first line is used for versioning, the Wizard tag is used to point to the ProgId of the component that will handle requests for C# console projects, and the multiple Param tags contain information that will be passed to the VsWizard.VsWizardEngine component. Note that this component handles multiple templates, so the parameters specify the specific template (console project) and the selected language (C#).

VSWIZARD 7.0
Wizard=VsWizard.VsWizardEngine
Param="WIZARD_NAME = CSharpConsoleWiz"
Param="WIZARD_UI = FALSE"
Param="PROJECT_TYPE = CSPROJ"

All components that act as handlers for project templates need to support the IDTWizard interface from the EnvDTE namespace. This wizard contains a single method, Execute, which is called when the template is invoked. Visual Studio .NET passes the base DTE object (used to access the entire Visual Studio .NET object model, please refer to the links section for articles that cover this), a parameter list containing information such as the name and path of the project as specified by the user, and another parameter list that contains the custom parameters from the .vsz files as shown above. This information is sufficient for the component to process the template request, and since the method has access to the root DTE object, the component can create solution and projects by invoking methods on this object.

  • VSDIR File: This file contains information about the individual .vsz files explained above. This information relates mainly to the display of the template in the New Project dialog box displayed by Visual Studio .NET. This display information includes a description for the template that is displayed when the template is selected, information about the icon that will be displayed in the New Project dialog box, and flags that control certain elements on this dialog box. A .vsdir file typically contains a single line entry for multiple .vsz files. The different fields are separated by a "|" character, and the different lines are separated by a new line character. Multiple .vsdir files are supported by Visual Studio .NET.

A sample .vsdir file containing two entries is shown below.

CSharpEXE.vsz|{FAE04EC1-301F-11d3-BF4B-00C04F79EFBC}|#2318|10|#2319|
{FAE04EC1-301F-11d3-BF4B-00C04F79EFBC}|4554| |WindowsApplication

CSharpWebApp.vsz|{FAE04EC1-301F-11d3-BF4B-00C04F79EFBC}|#2326|40|#2327|
{FAE04EC1-301F-11d3-BF4B-00C04F79EFBC}|4551|1|WebApplication1|web

Some of the main fields in this file are described below. For a complete description of these fields and their relative orders, please refer to the links section at the end of this article.

  • RelPathName: This field contains the path for the .vsz file.
  • SortPriority: This field defines the order in which the different templates will be displayed in the New Project dialog box.
  • Description: This field contains a descriptive string that displays information about the specific template.
  • DLLPath: This field points to the DLL that contains an icon that will be displayed for this template in the New Project dialog box.

When the New Project dialog box is invoked, Visual Studio .NET reads all the .vsdir and the .vsz files to display the available project templates. For the C# project type, this is read from the Microsoft Visual Studio.NET\VC#\CSharpProjects folder as shown below. This folder contains all the files that represent the C# templates that are available to the New Project dialog box. Later in the article, when we create our custom wizard, we will have to copy the .vsz and .vsdir files into this folder.

Custom Thread Pooling Template

Now that we have covered the basics of Visual Studio .NET project types and templates, let us look at how we can write our own custom project template that will generate a thread pool that can be used by the developer without any modifications at all. Before we start discussing the custom template, let us focus on writing a thread pool class that will be a part of this template.

A thread pool usually consists of two classes: one class that represents a thread, and another class that represents the object that manages a pool of these thread objects. The thread class is fairly simple: it is simply an encapsulation of the built in System.Threading.Thread class, but the thread pool class is slightly more involved. This thread pool class has to create all the threads in the pool, assign them work, and keep track of idle threads so that further work can be assigned to them. Further, the threads in a thread pool are created once, and if a thread is idle, the thread goes into an efficient wait mode (like a sleep call). When there is work to be processed, the thread pool is responsible for bringing an idle thread back into active mode and assigning it work.

Let us start by discussing the thread class first.

public class ThreadPoolThread
{
public
            ThreadPoolThread
(ThreadPool iThreadPool, ThreadPool.DoWork
            iWorkerMethod)
{
mWorkerMethod +=
            iWorkerMethod;

mThreadPool = iThreadPool;
mThread = new
            Thread (new ThreadStart (ThreadFunc));
mEvent = new
            AutoResetEvent (false);
mThread.Start ();
}

public void
            ProcessRequest ()
{
mEvent.Set ();
}

public void
            Stop ()
{
if (mThread != null)
mThread.Abort
            ();
}

private void ThreadFunc ()
{
WaitHandle[]
            lWaitHandle = new WaitHandle[1];
lWaitHandle[0] =
            mEvent;

while (true){
AutoResetEvent.WaitAll
            (lWaitHandle);
mWorkerMethod ();
mThreadPool.ReturnToPool
            (this);
}
}

private ThreadPool.DoWork
            mWorkerMethod;
private Thread mThread;
private AutoResetEvent
            mEvent;
private ThreadPool mThreadPool;
}

This class has four methods, a constructor that sets up the objects required by the class, a method to wake up the thread, a method to kill the thread, and the worker method where the thread will spend its entire life. The ThreadPoolThread object consists of one Thread object and one AutoResetEvent object, which is used to signal an idle thread. The constructor takes in a reference to the thread pool that this thread belongs to, and it also takes in a delegate which is the user defined method that this thread will invoke as part of its processing. The user of this class will pass this delegate, and it typically will be a method that will accept a request (from a socket or from a web service, etc) and process it. Once this delegate method returns, the thread will go into an efficient wait, blocking the event object. This is an efficient wait state that consumes minimal resources. The thread can be brought out of this wait state by the thread pool, by calling its ProcessRequest method. This method simply calls the Set method of the event object, which causes the event object to go into a signaled state. This causes the AutoResetEvent.WaitAll method to come out of a blocking state, and invoke the delegate method. This continues in an infinite loop, until the thread is terminated by the thread pool calling the Stop method.

Let us now look at the thread pool class. Note that this is a custom implementation of the thread pool, which is different from the System.Threading.ThreadPool class. We will start off by discussing the Start and the Stop methods.

public delegate void DoWork
            ();

public ThreadPool (int iNumThreads)
{
mNumThreads =
            iNumThreads;
}

public void Start (DoWork
            iWorkerMethod)
{
mFreeList = new ThreadPoolThread
            [mNumThreads];
mThreads = new ThreadPoolThread
            [mNumThreads];
mFreeThreads =
            mNumThreads;

ThreadPoolThread lThread;

for (int i = 0;
            i < mNumThreads; i++){
lThread = new ThreadPoolThread (this,
            iWorkerMethod);
mFreeList[i] = mThreads[i] =
            lThread;
}
}

public void Stop ()
{
for (int i =
            0; i < mNumThreads; i++)
mThreads[i].Stop
            ();
}

private int mNumThreads;
private
            ThreadPoolThread[] mFreeList;
private ThreadPoolThread[]
            mThreads;
private int mFreeThreads;

The first thing that the ThreadPool class defines is a delegate that is used by the thread class to call the method that will perform the actual processing, as explained above. The constructor of the thread pool takes in a single parameter which specifies the number of threads that will be created in this pool. The thread pool is started by calling the Start method, which creates a pool of ThreadPoolThread objects, and stores a pointer reference to these objects in the mThreads and mFreeList arrays (we use arrays for simplicity in this sample: a more efficient alternative would be the use of lists). The mThreads array represents all the threads in this pool, and the mFreeList array represents all the threads that are currently free to service requests. All threads start off belonging to the free list. The Stop method simply calls the Stop method on the ThreadPoolThread class that aborts the thread.

Let us now look at the ProcessRequest method that is called by the client when it needs a thread to process a request. Calling this method causes the delegate that was passed to the Start method to be called.

public bool ProcessRequest
            ()
{
ThreadPoolThread lThread = null;
if (mFreeThreads ==
            0)
return false;

lock(this)
{
mFreeThreads--;
for
            (int i = 0; i < mNumThreads; i++)
if (mFreeList[i] !=
            null){
lThread = mFreeList[i];
mFreeList[i] =
            null;
break;
}
}

lThread.ProcessRequest
            ();
return true;
}

The ProcessRequest method needs to find a free thread to process the request, and it starts off by checking the value of the mFreeThreads counter, which indicates the number of available free threads. If no threads are available, the methods returns false. If free threads are available, the thread pool iterates through the mFreeList array looking for the first free thread. Once a thread is found, it marks the thread as busy by removing it from the free list. This method then calls the ProcessRequest method on the free thread, that in turns invokes the user delegate. Since this method can be called from multiple threads, we use the C# lock keyword to protect the mFreeList and mFreeThreads objects for concurrent access.

The ReturnToPool method that is used to return a busy thread to the pool of free threads is shown below. This method is similar to the ProcessRequest method, wherein the method attempts to find a free position in the mFreeList array to add the thread to. Again, we use the lock keyword to guard against concurrent access to the mFreeList and mFreeThreads objects.

public void ReturnToPool (ThreadPoolThread
            iThread)
{
lock (this)
{
mFreeThreads++;
for (int i =
            0; i < mNumThreads; i++)
if (mFreeList[i] ==
            null){
mFreeList[i] =
            iThread;
break;
}
}

return;
}

Now that we have seen the thread pool classes, let us move to creating the object that will handle our custom project template. This is a simple C# class library project, with a single class that implements the IDTWizard interface.

public class CustomWizard : IDTWizard
{
public void Execute(object application, int hwndOwner,
ref object[] contextParams, ref object[] customParams,
ref EnvDTE.wizardResult retval)

{
string lProjName = (string)contextParams[1];
string lSolnName = (string)contextParams[2];

if (!Directory.Exists (lSolnName))
Directory.CreateDirectory (lSolnName);

_DTE lApp = (_DTE)application;
_Solution lSoln = lApp.Solution;

lSoln.Create (lSolnName, lProjName);
lSoln.AddFromTemplate(mProjPath,lSolnName,lProjName,false);
}

// NOTE: change this to the path where you store the downloaded files
private const string mProjPath =
@"C:\C#Today\CustomWizards\ThreadPoolTemplate.csproj";
}

The code block above is all we need for our custom wizard component. The first thing we do in this method is to retrieve the names of the project and the solution that is input by the user in the New Project dialog box. Once that is done, we check to see if the user input path exists, and if not, we use the System.IO.Directory class to create the specified folders. We then use the Solution object from the Visual Studio .NET object model to create a new solution and project based on our ThreadPool classes. We use the AddFromTemplate method of the Solution class to copy over the project files from the thread pool project.

Readers should change the mProjPath variable to point to the location where they have stored the ThreadPoolTemplate.csproj and ThreadPool.cs files.

The .vsdir file for this project template is shown below. We used the Guidgen tool to generate a unique GUID.

Thread Pooled Application.vsz|0|0|0|Generates a
thread pool class.|{FAE04EC1-301F-11d3-BF4B-00C04F79EFBC}|4553|0|0

The .vsz file is as shown below.

VSWIZARD 7.0
Wizard=CustomWizardNS.CustomWizard

Attached Sample Code

The attached sample code contains the CustomWizard and ThreadPoolTemplate projects, and the support vsdir and vsz files.

Please carry out the following steps to set up this sample:

  1. Copy the ThreadPool.cs and ThreadPoolTemplate.csproj files to C:\C#Today\CustomWizards. If you need to copy these to a different path, you will have to change the mProjPath member variable in the CustomWizard class from the CustomWizard project.
  2. Copy the Thread Pooled Application.vsdir and Thread Pooled Application.vsz files to C:\Program Files\Microsoft Visual Studio.NET\VC#\CSharpProjects. Note that this path will be different if you have installed Visual Studio .NET in a different location. Substitute your install path for C:\Program Files\Microsoft Visual Studio.NET.
  3. Compile the CustomWizard project.
  4. Copy CustomWizard.dll to C:\Program Files\Microsoft Visual Studio.NET\Common7\IDE. This is again assuming the default install path for Visual Studio .NET.
  5. Run the following command on the file copied in the previous step. This registers the managed component as a COM component.

regasm CustomWizard.dll

The following screen shot shows our custom project template in the New Project dialog box.

Conclusion

Custom project templates and wizards are a great way for developers to write boilerplate code once, and have it automatically generated for subsequent use. Visual Studio .NET makes it fairly easy to write these custom wizards, and the code for hooking up a custom template is little more than a few lines of code, as shown in this article's sample. Any canned code that is copied over and over again for new projects should be a prime candidate for a custom project template.

Happy .NETing.


Modularizing Object Collections With The Collections API

Modularizing Object Collections With The Collections API
http://www.csharptoday.com/content.asp?id=1684
Article Rating: 4
Published on October 29, 2001
Abstract
The Collections API in the .NET Framework merits close attention by all .NET developers because of the flexibility it offers for writing applications in .NET. We can use it to manage groups or collections of similar objects, standardizing the way in which groups of objects are handled by our programs. In this article, Thiru Thangarathinam takes us through a tour of all the important collection classes by showing us examples, and then goes on to show us how these collection classes can be very useful in the day-to-day work of a developer.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

A collection is a predefined object that stores groups of related objects, making it easier to work with the object group. It provides a convenient way to refer to a related group of items as a single object. The items, or members in a collection, are only related by the fact that they exist in the collection, and they don't have to share the same data type. By using collection classes in our applications, we can create very clean modular code that is flexible and maintainable.

All the collection related classes are contained in the following namespaces:

  • System.Collections - Contains a wide assortment of classes, interfaces and structures that are used to create and manage collections. To get a brief idea of the hierarchy of classes in the System.Collections namespace, please visit http://msdn.microsoft.com/library/default.asp?url=/library/en-us/cpref/html/frlrfsystemcollectionshierarchy.asp&WROXEMPTOKEN=620568Zhiox25QOrrsnTPfm53p.
  • System.Collections.Specialized - Contains classes that allow us to create specialized and strongly typed collections. For example, the StringCollection class is one of the classes that fall in this category, which is meant only for dealing with strings. To get a brief overview of the hierarchy of the strongly typed collection classes, please take a look at http://msdn.microsoft.com/library/default.asp?url=/library/en-us/cpref/html/frlrfSystemCollectionsSpecialized.asp&WROXEMPTOKEN=620568Zhiox25QOrrsnTPfm53p

In this article, we will explore some of the important classes present in the aforementioned namespaces by looking at examples of how to use them. Along the way, we will also identify which collection class to use in which situation, by discussing the pros and cons of each. We will go through this article in stages:

  • An overview of the Collection interfaces
  • The IEnumerable and IEnumerator interfaces
  • The ICollection interface (the foundation of the entire Collection framework), and how the IDictionary interface is used to manage objects.
  • How IList is used to index an object's items
  • The different Collection classes
  • A detailed look at ArrayList, Hashtable, SortedList, Stack, Queue and Comparator
  • The classes in the Systems.Collection.Specialized namespace

Before we start looking at the collection classes, let us briefly look at some of the important collection interfaces that are present in the System.Collections namespace. While looking at the collection interfaces, we will also examine the different properties and methods exposed by these interfaces.

Collection Interfaces

In this section we will provide an overview of the collection interfaces, as familiarity with these interfaces is necessary to get a clear understanding of the collection framework. The collection namespace consists of the following important interfaces.

  • ICollection - This is the core collections interface, as all the collection classes in the .NET Framework are derived from this interface.
  • IEnumerable - This interface exposes a method named GetEnumerator that returns the enumerator for a collection object.
  • IEnumerator - Any collection class that wants its clients to have the capability to loop through the items in the collection using the for..each construct, needs to implement this interface.
  • IList - This represents a collection of elements that can be sorted and accessed through indexes.
  • IDictionary - This is implemented by classes that are required to hold a collection of similar objects, with each individual element in the collection being represented by a key-and-value pair. For example, the Hashtable class, which identifies its elements by means of a key-and-value pair, implements this interface.
  • IDictionaryEnumerator - This is implemented by classes such as hashtable to enumerate the contents.

The following UML class diagram explains the relationship between different interfaces in the System.Collections namespace.

The above diagram shows that the IDictionary and IList interfaces inherit from ICollection, which in turn inherits from the IEnumerable interface. IDictionaryEnumerator derives from the IEnumerator interface, and is used to enumerate the contents of a dictionary that consist of key-and-value-pair elements. Now that we understand the relationship between the different interfaces, let us discuss the properties and methods exposed by these interfaces.

IEnumerable and IEnumerator

Often, we want to cycle through the elements in a collection and display each one. To accomplish this, we can use the methods of the IEnumerator interface. To get a reference to the enumerator interface from a collection class, we need to call the GetEnumerator() method of the IEnumerable interface.

It is important to understand the role of enumerators in the collections framework since they allow us to loop through the elements in a unified and consistent manner, regardless of the collection we are trying to access. In other words, enumerators provide a forward-only read-only cursor for a group of similar objects contained in a collection. The IEnumerator interface consists of the following three methods:

  • MoveNext: Used to move to the next item in the collection. It returns true until it reaches the end of the collection, after which it returns false.
  • Current: Used to retrieve the value of the current item that the enumerator points to.
  • Reset: Sets the enumerator back to its initial state, which is before the first element in the collection.

Since the initial position of the enumerator is before the first element in the collection, we need to call MoveNext before we call Current, else an exception will be thrown.

From the above explanation, it is clear that looping through the collection requires the use of the above three methods. However, we can accomplish the same thing by the use of a for..each construct, which internally invokes the methods of the enumerator for us. One of the main advantages of the for..each construct is its ease of use, and flexibility (by preventing us from getting the dreaded out of bounds array error message).

If we want to use the for..each semantics to loop through the collection, then we need to implement the interface IEnumerable. The for..each construct can be valuable in the following conditions:

  • When we want to loop through a collection without modifying the contents of it, as for..each is read-only. Instead, if we want to have the ability to modify the content, then we need to choose regular looping constructs.
  • If we do not need indexes while looping through a collection, then we can use for..each, which is much simpler to use and optimized to provide better performance.

ICollection and IDictionary

The ICollection interface is the foundation upon which the entire collection framework is built. The ICollection interface exposes methods and properties that are required to describe the size, enumerator and synchronization characteristics for all the collection classes:

  • Count - Retrieves the number of elements contained in the collection.
  • IsSynchronized - Returns true or false, indicating whether access to the collection is synchronized.
  • SyncRoot - Allows us to synchronize access to the collection by providing us with a synchronized wrapper object around the underlying collection.

The IDictionary interface is used to manage groups of unordered objects as keyvalue pairs. The classes that implement IDictionary include SortedList and Hashtable. The IsFixedSize property of the IDictionary interface returns true or false depending on whether the class that implements the IDictionary interfaces is of a fixed size. As the name suggests, the ReadOnly property allows us to determine whether the collection is read only. The Item property is used to set or receive the value of the element in the collection, based on the supplied key. The Keys and Values properties are used to get references to all the keys or all the values in the collection respectively.

IList

The IList interface is responsible for the collection object's ability to have its items indexed individually. The IList interface defines the following important properties:

  • IsFixedSize - Determines if a list is of a fixed size.
  • Item - Allows us to set or receive an item in the list using the index.
  • IsReadOnly - Allows us to determine whether the list is read-only.

The methods defined by the IList interface are as follows:

  • Add - Allows us to add an item to the end of the list
  • Clear - Removes all the items in the list
  • Contains - Allows us to determine if an item is contained within a list
  • IndexOf - Used to determine the index of an item within a list
  • Insert - Inserts an item at a specified offset in the list
  • Remove - Removes an item by reference from the list
  • RemoveAt - Removes an item by an index from the list

Since the IList interface derives from the ICollection interface, it inherits all of its members, but also inherits all the members of IEnumerable (since ICollection derives from IEnumerable).

Apart from the aforementioned interfaces, the System.Collections namespace also contains IComparer and IHashCodeProvider interfaces. The IComparer interface exposes a method called Compare that allows us to compare two objects. The IHashCodeProvider interface is mainly used in conjunction with the Hashtable class and provides us with custom hash code for an object.

Collection classes

Now that we have introduced the collection interfaces, let us take a look at some of the important collection classes, and examine the usefulness of these classes by considering examples. For this demonstration we will create an ASP.NET web application called CollectionExamples, as shown in the following diagram.

After the project is created, we rename the default webform1.aspx to ArrayListExample.aspx, as we are going to start our implementation by looking at ArrayList class.

ArrayList

As we all know, standard arrays are of a fixed length. After arrays are created, they cannot grow or shrink, meaning that we must know in advance how many elements an array will hold. However, there might be times when we may not know until runtime precisely how large an array we need. To handle this kind of situation, the collections framework defines the ArrayList class, which can dynamically increase or decrease the number of elements it can hold. ArrayList can be simply defined as an array whose size can be dynamically increased or decreased at runtime. Since ArrayList implements an IList interface, we can access the individual elements in the array using indexes.

Arraylists are created with an initial size. When this initial size is exceeded, they are automatically enlarged. When elements are removed, it can be shrunk to a smaller size using the TrimToSize method.

The capacity property of an ArrayList returns the number of elements that the ArrayList is capable of storing, and Count returns the number of elements that are actually in the ArrayList. The value of the capacity is always greater than or equal to Count. While adding elements, if the value of Count exceeds capacity, then the capacity of the list is automatically doubled by resizing the ArrayList. The default value for capacity is 16.

Although the capacity of an array can increase dynamically as we add objects, we can also increase the capacity of an ArrayList object manually by setting the Capacity property to an appropriate value. You might want to do this if you know beforehand that you will be storing more items in the collection than it can currently hold. By increasing the capacity at the start, you can prevent several costly reallocations later, thereby increasing the performance of the application.

The following list summarizes the important methods of the ArrayList class.

  • Add - Allows us to add an element to the end of the ArrayList.
  • AddRange - Allows us to add a group of elements that implement the ICollection interface to the end of the ArrayList.
  • Remove - Allows us to remove an object from the ArrayList.
  • RemoveRange - This method is similar to the AddRange method, except that it removes the group of elements instead of adding them to the collection.
  • Insert and InsertRange - These methods are similar in functionality to Add and AddRange, except that they provide for addition of elements at the specified index.
  • Clear - This allows us to remove all the elements from an ArrayList.
  • ToArray - Permits us to copy the contents of an ArrayList into an array. This is very useful in situations when we want to integrate the collections based code with the legacy code.
  • IndexOf and LastIndexOf - These methods are used to search for an element in an ArrayList in a linear fashion. The IndexOf method returns the index of the specified element from the beginning of the ArrayList, and the LastIndexOf method returns the last occurrence of the specified element in the ArrayList.
  • BinarySearch - Unlike the IndexOf and LastIndexOf methods, this method allows us to search for elements in the ArrayList using the efficient binary search mechanism.
  • Reverse - As the name suggests, this allows us to reverse the contents of an ArrayList.

Now let us implement all of the above methods by considering an example. The web form we create for the demonstration of ArrayList looks like the following.

In the Page_Load event, we check the page is requested for the first time, or it is posting back (submitted back) to itself by verifying the IsPostBack property of the Page class. If the page is requested for the first time, then we create an ArrayList object and add a set of elements to it using the Add method. Once we add elements to it, we then assign the ArrayList object to a session level variable. Finally, we invoke the helper function ShowAllElements to display the contents in a listbox. We will see the implementation of ShowAllElements in a moment.

private void Page_Load(object sender,
            System.EventArgs e)
{
// Put user code to initialize the page
            here
if (!Page.IsPostBack)
{
ArrayList langList = new
            ArrayList();
langList.Add("Visual
            C#.NET");
langList.Add("VB.NET");
langList.Add("JScript.NET");
langList.Add("ASP.NET");
Session["ArrayList"]
            = langList;
ShowAllElements();
}
}

The helper function ShowAllElements is used to loop through the elements contained in the session variable, and display them in the listbox.

private void
            ShowAllElements()
{
//Clear all the items in the Languages
            Listbox
lstLanguages.Items.Clear();
ArrayList langList =
            (ArrayList)Session["ArrayList"];
foreach(Object o in
            langList)
{
lstLanguages.Items.Add(o.ToString());
}
}

When the ShowAll button is clicked, we simply invoke the ShowAllElements function to display the items in the collection.

private void btnShowAll_Click(object sender, System.EventArgs e)
{
ShowAllElements();

}

As we have already discussed, the Reverse method is used to reverse the contents of the ArrayList. Once we reverse the contents, we then call the ShowAllElements method to refresh the contents of the listbox.

private void btnReverse_Click(object
            sender, System.EventArgs e)
{
//Reverse the contents of the
            ArrayList
((ArrayList)Session["ArrayList"]).Reverse();
ShowAllElements();
}

In the following method, we check the presence of the item (identified by the value entered in the text box) by invoking the IndexOf method. The IndexOf method returns either the index of the element, or -1, depending on whether the item is found or not. In the first line we cast the session variable to type ArrayList, and then assign it to the local variable langArrayList.

private void btnIndexOf_Click(object
            sender, System.EventArgs e)
{
ArrayList langArrayList =
            (ArrayList)Session["ArrayList"];
int index =
            langArrayList.IndexOf(txtValue.Text);
lblMessage.Text = "The item
            is found in " + index.ToString() + " position";
}

This method is similar to the above method, except that the LastIndexOf method searches for elements from the end of the collection.

private void btnLastIndexOf_Click(object sender, System.EventArgs e)
{
ArrayList langArrayList = (ArrayList)Session["ArrayList"];
int index = langArrayList.LastIndexOf(txtValue.Text);
lblMessage.Text = "The item is found in " + index.ToString() + " position";
}

Here we sort the contents of the ArrayList by invoking the Sort method.

private void btnSort_Click(object sender,
            System.EventArgs e)
{
//Sort the elements contained in the
            ArrayList
((ArrayList)Session["ArrayList"]).Sort();
ShowAllElements();
}

In the following lines of code, we determine the Capacity of the ArrayList, and then show the result in the label named lblMessage.

private void btnCapacity_Click(object
            sender, System.EventArgs e)
{
ArrayList langArrayList =
            (ArrayList)Session["ArrayList"];
int capacity =
            langArrayList.Capacity;
lblMessage.Text = "The capacity of the
            ArrayList is " + capacity.ToString();
}

Here we get the Count of the ArrayList and display it in the label identified by the name lblMessage.

private void btnCount_Click(object
            sender, System.EventArgs e)
{
ArrayList langArrayList =
            (ArrayList)Session["ArrayList"];
int count =
            langArrayList.Count;
lblMessage.Text = "The count of the
            ArrayList is " + count.ToString();
}

In this method, we search for an item in the ArrayList using the BinarySearch method, and display the position of the item in the label.

private void btnBinarySearch_Click(object sender, System.EventArgs e)
{
ArrayList langArrayList = (ArrayList)Session["ArrayList"];
int index = langArrayList.BinarySearch(txtValue.Text);
lblMessage.Text = "The item is found in " + index.ToString() + " position";
}

ArrayList can be very useful in following scenarios:

  • When designing a collection that needs to have the ability to dynamically grow if required.
  • When you want to have the ability to access the individual elements in the collection using indexes.

Hashtable

Hashtable can be defined as a collection that consists of elements, which are made up of key-and-value combinations. The elements in a Hashtable are arranged based on a hash code (a number which is used to uniquely identify the key of an element), which is generated based on the key of the element. When a new entry is added to the Hashtable, the key of the element is converted into a hash code, and then the element is placed into a storage mechanism called Buckets. These buckets are mainly used to help arrange the elements of the hashtable in a way that provides for easy look-up of the individual elements. It is this internal arrangement of the elements that accounts for the high performance of the Hashtable class when it comes to searching for elements in a Hashtable.

As we already said, the Hashtable class implements a hashtable data structure. A hashtable indexes and stores objects in a dictionary-like structure using hash codes as the object's keys. These hash codes are unique values that determine the identity of the objects.

Hashtables are very useful in situations where we want to design collections that need to be easily searchable. Since the contained elements in the hashtable are arranged based on the hash code of their keys, they provide very efficient searching capabilities. Since the internal arrangement of the elements in a Hashtable is based on their key, this faster searching is available only when we search elements based on the key.

Before we look at an example of hashtables in a collection, let us consider the important properties and methods of the hashtable class.

Important properties of Hashtable include:

  • Keys - Gets all the keys of all the elements contained in the hashtable in the form of an object that is derived from ICollection interface.
  • Values - This property is similar to the Keys property, except for the fact that this property returns a collection of values instead of keys.

Important methods of Hashtable include:

  • Add - Allows us to add an element with the specified key and value into the hashtable.
  • Remove - Removes the element with the specified key from the hashtable.
  • ContainsKey and ContainsValue - These methods return true or false depending on whether an element with the specific key or value is contained in the hashtable.

It is important to note that the hashtable does not guarantee the order of the elements in the collection, and the elements in the hashtable are not sorted.

To understand the implementation of Hashtable, let us consider the following example.

We will take a brief look at the code that is required for the above application.

Like the previous example, we start by declaring a variable of type Hashtable and then add elements to it by invoking the Add method. Once the elements are added, we then assign the Hashtable object to a Session level variable. Finally, we invoke the ShowAllElements method to display the elements in a listbox.

private void Page_Load(object sender,
            System.EventArgs e)
{
// Put user code to initialize the page
            here
if (!Page.IsPostBack)
{
Hashtable employees = new
            Hashtable();
employees.Add("1","Nancy");
employees.Add("2","Tom");
employees.Add("3","Mary");
employees.Add("4","Stacy");
Session["Hashtable"]
            = employees;
ShowAllElements();
}
}

In this method, we put the reference to the hashtable that is present in the session variable into a local variable called employees. Then we loop through the elements in the Hashtable by using a for..each construct. While looping through the collection, we use the DictionaryEntry object to hold the individual elements. Once we get a reference to the DictionaryEntry object, we then simply invoke its Key and Value properties to retrieve its contents.

private void
            ShowAllElements()
{
//Clear all the items in the Languages
            Listbox
lstEmployees.Items.Clear();
Hashtable employees =
            (Hashtable)Session["Hashtable"];
foreach(DictionaryEntry entry in
            employees)
{
ListItem item = new ListItem();
item.Value =
            entry.Key.ToString();
item.Text= entry.Value.ToString();
lstEmployees.Items.Add(item);
}
}

Here we just invoke the ShowAllElements to refresh the contents of the listbox.

private void btnShowAll_Click(object sender, System.EventArgs e)
{
ShowAllElements();
}

The ContainsKey method is used to identify whether an element with the specified key is present in the Hashtable. If it is present, it returns true, else it returns false. In the following lines of code, we use the ContainsKey method to check for the presence of an item with the specified key, and then display the result in the label control named lblMessage.

private void btnContainsKey_Click(object sender, System.EventArgs e)
{
Hashtable employees = (Hashtable)Session["Hashtable"];
bool result = employees.ContainsKey(txtValue.Text);
lblMessage.Text = "The key " + txtValue.Text + " is contained in the Hashtable : "
+ result.ToString();
}

The following method is similar to the above method, except that we check for the existence of an element with the specified value.

private void btnContainsValue_Click(object sender, System.EventArgs e)
{
Hashtable employees = (Hashtable)Session["Hashtable"];
bool result = employees.ContainsValue(txtValue.Text);
lblMessage.Text = "The value " + txtValue.Text + " is contained in the Hashtable : "
+ result.ToString();
}

In this method, we determine the count value of the Hashtable, and show it in a label named lblMessage.

private void btnCount_Click(object
            sender, System.EventArgs e)
{
int count =
            ((Hashtable)Session["Hashtable"]).Count;
lblMessage.Text = "The
            count of the Hashtable is " + count.ToString();
}

We use the following method to add an item to the Hashtable. Before adding the element, we call the ContainsKey method to verify whether an element with the specified key already exists. If it does exist, we add it to the Hashtable that is maintained by the session level variable.

private void btnAdd_Click(object sender,
            System.EventArgs e)
{
Hashtable employees =
            (Hashtable)Session["Hashtable"];
//check whether the element
            already exists or not
if (employees.ContainsKey(txtValue.Text)==
            true)
{
lblMessage.Text = "The element already exists
            ";
return;
}
//Add the same value as both key and
            value
employees.Add(txtValue.Text,txtValue.Text);
ShowAllElements();
}

As the name of the event handler suggests, this method is used to remove an item from the Hashtable if it is present. If the element is not present, the Remove method returns without throwing any exception.

private void btnRemove_Click(object sender, System.EventArgs e)
{
//Remove the item with the specified key
((Hashtable)Session["Hashtable"]).Remove(txtValue.Text);
lblMessage.Text = "Element is removed";
ShowAllElements();
}

Hashtable can be very useful in following scenarios.

  • If you want a collection that needs to exhibit high performance while searching for elements.
  • When you expect the collection to ensure unique values (by means of keys). For example, a shopping cart implementation can be efficiently done using an object of type Hashtable.
  • When the sorting order of the elements contained in the collection is not of great importance.

SortedList

If you want to create a collection that not only provides the functionalities of a hashtable, but also needs to have the ability to order the elements in the collection, then SortedList will be the ideal choice. We can define SortedList as a collection that consists of a group of associated keys and values that are sorted by the keys. To provide for the sorted arrangement of the individual elements, the SortedList class internally uses two arrays to store the individual elements, with one array for the keys and another array for the values.

Since SortedList stores all the entries in a sorted manner, it tends to perform slower when compared to a Hashtable. However, the SortedList offers more flexibility by allowing access to the values either through the associated keys or through the indexes.

Since SortedList implements an IList interface, it supports methods that are similar to ArrayList, which are useful in determining and reallocating the size of the list. Like the ArrayList, the Capacity property indicates the number of elements the SortedList can hold. We can also modify the size of the SortedList either by calling TrimToSize or setting the Capacity property to an appropriate value.

There are two ways by which we can remove a specified element from the SortedList.

  • To remove an element based on the key, the Remove method can be used
  • To remove an element based on the index, RemoveAt method can be used

The example web form that we are going to use for this demonstration looks like the following.

In the Page_Load event, we create an instance of a SortedList object and add items to it by using the Add method. We pass the key and value parameters to the Add method as arguments.

private void Page_Load(object sender,
            System.EventArgs e)
{
// Put user code to initialize the page
            here
if (!Page.IsPostBack)
{
SortedList employees = new
            SortedList();
employees.Add("1","Nancy");
employees.Add("2","Tom");
employees.Add("3","Mary");
employees.Add("4","Stacy");
Session["SortedList"]
            = employees;
ShowAllElements();
}
}

We will skip the implementation for the rest of the methods, as they are exactly the same as the methods we have already seen for the Hashtable example. You can download this code as part of the support material of this article.

SortedList can be useful in the following situations.

  • If you want the elements in the collection to be sorted by default.
  • If you want the ability to access the elements in the collection either using indexes or using an enumerator.
  • If you want a collection that needs to ensure uniqueness among its keys.

Stack

A Stack can be considered as a collection that stores elements using Last-In-First-Out (LIFO). As the name suggests, LIFO means that the element that is stored at the end of the collection is the first one to be retrieved. The stack class can be very useful in situations where we want to carry out recursive operations wherein we need to store and retrieve values and process them frequently.

The Stack class is a fundamental collection type that is used within the heart of any modern operating system. For example, it has implications for calls to subroutines where the system stack is used to store the Program Counter when executing a subroutine. On exiting from the subroutine, the Program Counter is popped from the stack and execution continues. Because of the inherent LIFO nature of the Stack class, nested subroutines in a program are automatically catered for.

An everyday example of a stack is a pile of papers on a desk in which the item on the top is the one that is most easily accessed. The easiest way to add a new item to the stack is to place it above all the current items in the stack. In this manner, an item removed from a stack is the item that has been most recently inserted into the stack, for example, the top piece of paper on the pile.

Since the Stack class implements the ICollection and IEnumerator interfaces, we can easily enumerate the items in a Stack class using the for..each construct.

The important methods supported by the Stack class are:

  • Push - Allows us to insert an object at the top of the stack
  • Pop: This method is used to remove and return the object at the top of the stack
  • Peek: This method is similar to Pop except that it does not remove the object from the stack, and so is handy in circumstances where we just want to read an element from the stack without removing it.
  • Clear: Used to remove all the elements from the stack, and count is set to zero

The following picture demonstrates a simple web form that uses the Stack class.

Let us see the code that is required for the above web form. In the Page_Load event, we create an object of type Stack and push elements into it by executing the Push methods.

private void Page_Load(object sender,
            System.EventArgs e)
{
// Put user code to initialize the page
            here
if (!Page.IsPostBack)
{
Stack databaseList = new
            Stack();
databaseList.Push("SQL
            Server");
databaseList.Push("Access");
databaseList.Push("Oracle");
databaseList.Push("Sybase");
Session["Stack"]
            = databaseList;
ShowAllElements();
}
}

Here we enumerate the contents of the Stack object and display its contents in the listbox.

private void
            ShowAllElements()
{
lstDatabaseList.Items.Clear();
Stack
            databaseList = (Stack)Session["Stack"];
foreach(object o in
            databaseList)
{
lstDatabaseList.Items.Add(o.ToString());
}
}

This method is used to pop (remove) elements from the Stack. Before we remove an element, we verify the Count property to ensure that the Stack is not empty. If the Stack is not empty, we pop the element and display the contents of the popped element in the label.

private void btnRemove_Click(object
            sender, System.EventArgs e)
{
string val=null;
Stack
            databaseList = (Stack)Session["Stack"];
//Pop the elements only
            if the count is greater than 0
if (databaseList.Count !=0)
{
val = databaseList.Pop().ToString();
}
lblMessage.Text = "The read item : " +
            val;
ShowAllElements();
}

The following lines of code are similar to the above method, except that here we use the Peek method, which simply reads the element without removing it from the Stack.

private void btnPeek_Click(object
            sender, System.EventArgs e)
{
string val=null;
Stack
            databaseList = (Stack)Session["Stack"];
if (databaseList.Count
            !=0)
{
val =
            databaseList.Peek().ToString();
}
lblMessage.Text = "The read
            item : " + val;
ShowAllElements();
}

Queue

The Queue class is similar to the Stack class except that it works on the principle of First-In-First-Out (FIFO), meaning that the element which is inserted at the front, is the first one to be retrieved. Queues are useful in places where we might want to do sequential processing of the messages in the same order they were stored. For example, let us say that we are monitoring a directory or a file for some predefined changes and we want to process the changes that occur in that directory or file in the same order as they occur. In that case, we can use the Queue collection type to store the details of all the changes, and from there we can process them one by one.

When the number of elements added to the queue reaches the current capacity, the capacity is automatically increased to give room for new elements.

The following list shows the important methods supported by the Queue class:

  • Enqueue - Allows us to add an element at the end of the queue.
  • Dequeue - Used to remove and return the element from the beginning of the queue.
  • Peek - Similar to Dequeue except that this method returns the element from the beginning of the queue without removing the element.

Let us understand the code that is required for the above form.

Here we create an instance of an object of type Queue and add elements to it using the Enqueue method.

private void Page_Load(object sender,
            System.EventArgs e)
{
// Put user code to initialize the page
            here
if (!Page.IsPostBack)
{
Queue databaseList = new
            Queue();
databaseList.Enqueue("SQL
            Server");
databaseList.Enqueue("Access");
databaseList.Enqueue("Oracle");
databaseList.Enqueue("Sybase");
Session["Queue"]
            = databaseList;
ShowAllElements();
}
}

As we already discussed, removing an element requires the use of Dequeue method.

private void btnRemove_Click(object
            sender, System.EventArgs e)
{
string val=null;
Queue
            databaseList = (Queue)Session["Queue"];
//Dequeue the elements
            only if the count is greater than 0
if (databaseList.Count !=0)
{
val = databaseList.Dequeue().ToString();
}
lblMessage.Text = "The read item : " +
            val;
ShowAllElements();
}

Now that we have understood the usage and functionalities of a Queue, we can safely conclude that Queues can be valuable in the following situations.

  • When you need a collection that has the capability to hold the entries in the same order they were entered.
  • When you want to easily add and remove elements from your collection. This is made possible because of the Enqueue and Dequeue methods.
  • When you want to have the ability to read an item from the collection without removing it. The Peek method serves this purpose.

Comparer

Objects of classes that implement IComparable can be ordered. We can use this class to compare objects and check whether they are equal. It can also be used to perform case sensitive comparisons of strings. The Comparer class provides default implementation for the Compare method of the IComparer interface. This method is used to perform a case sensitive comparison of two objects of the same type. This method returns less than zero if the first object is less than the second object, zero if both of them are equal, and greater than zero if the first object is greater than the second.

The .NET framework classes provide excellent support for ordering the contents of a collection or an array through the use of built-in functions. However, there are times where we might want to customize the sorting behavior according to the requirements of our application. To accomplish this we can override the CompareTo method of the IComparable interface to produce a sorting behavior that is in line with the requirements of our application.

Let us consider the following example in which there is an Employee object that holds the details of an employee, such as name and salary. Let us say, for example, that we are loading the details of all the employees into an employee object array, and then we want to show the list of employees in descending order of salary. This requirement necessitates a custom implementation of the IComparer interface, and in this example we will see how to accomplish this.

The Employee class is defined as follows:

public class Employee :
            IComparable
{
string name;
int salary;

public
            Employee(string name, int salary)
{
this.name=
            name;
this.salary = salary;
}

public string Name
{
get
{
return name;
}
}

public int
            Salary
{
get
{
return salary;
}
}

int
            IComparable.CompareTo(object obj)
{
Employee emp =
            (Employee)obj;
if (this.salary > emp.salary)
return
            -1;
if (this.salary < emp.salary)
return
            1;
else
return 0;
}

}

As we can see from the above code, the Employee class has a constructor that takes the name and salary of the employee as its arguments. It also contains the public properties Name and Salary that are used to expose the values present in the private member variables name and salary respectively. Now, let us consider the CompareTo method that allows us to change the default sorting behavior. As we already discussed, the return value of the CompareTo method is the one that determines the sort order. Since we want our sorting to be based on the descending order of the employee's salary, we compare the salary of the two employee objects and return -1 in place of 1, and 1 in place of -1.

Now that we have seen the implementation of the Employee class, let us look at the client code and test the effectiveness of this sorting order.

private void Page_Load(object sender, System.EventArgs e)
{
// Put user code to initialize the page here
if (!Page.IsPostBack)
{

First, we declare an array of employee objects using the following line of code.

Employee[] arrEmp = new Employee[5];

arrEmp[0] = new Employee("Nancy",5000);
arrEmp[1] = new Employee("Andrew",8000);
arrEmp[2] = new Employee("Janet",8500);
arrEmp[3] = new Employee("Margaret",6000);
arrEmp[4] = new Employee("Steven",7500);

Secondly, we assign the above-created employee object array to the session variable using the following line of code.

Session["EmpArray"] = arrEmp;

Finally, we show all the elements in the listbox by invoking the ShowAllElements function.

ShowAllElements();
}
}

In this function, we loop through all the elements in the employee object array and show its contents in the listbox.

private void
            ShowAllElements()
{
lstEmployees.Items.Clear();
Employee[]
            arrEmp = (Employee[])Session["EmpArray"];
foreach(Employee emp in
            arrEmp)
{
lstEmployees.Items.Add(emp.Name + " - "
+
            emp.Salary.ToString());
}
}

In the code shown below, when the static Sort method of the Array class is invoked, the custom implementation that we provided for the CompareTo method in the Employee class gets called internally, and as a result of this we have the entire Employee objects sorted by descending order of their salary.

private void btnSortBySalary_Click(object sender, System.EventArgs e)

{

Array.Sort((Employee[])Session["EmpArray"]);

ShowAllElements();

}

Systems.Collections.Specialized namespace

Now that we have understood the functionalities and usage of the classes present in the System.Collections namespace, let us look at the strongly typed collection classes present in the System.Collections.Specialized namespace. For more information on classes contained in the System.Collections namespace, please take a look at http://msdn.microsoft.com/library/default.asp?url=/library/en-us/cpref/html/frlrfSystemCollections.asp

NameValueCollection

NameValueCollection can be considered as similar to Hashtable except for the following differences. First, it can store key-and-value pairs that can be only of type String. Secondly, it allows us to associate multiple values with a single key. Because of its ability to store multiple values under one key, we can use this collection for storing query strings and form data in web pages. We will see an example of how NameValueCollection can be used to store form data shortly.

Before we move on to the example, let us look at some of the important methods of NameValueCollection class:

  • GetKey - Allows us to retrieve the key of a specified element based on the index.
  • GetValues - Returns the values associated with the specified key in the form of a string array.
  • HasKeys - Returns or true or false depending on presence of the key entries in a NameValueCollection.

In this section we will take a brief look at the code required for enumerating the contents of the form.

private void btnEnumerate_Click(object sender, System.EventArgs e)

We start by declaring an object of type NameValueCollection that can hold all of the form data.

NameValueCollection employees = new
            NameValueCollection();
string[] strEmployees;

Then we assign the Form object to the NameValueCollection object using the following statement.

employees = Request.Form;

To get all the elements the user has selected in the listbox named lstEmployees, we invoke the GetValues method, passing to it the name of the control whose values we want to get.

strEmployees =
            employees.GetValues("lstEmployees");
lblMessage.Text = "Selected
            items are : ";

Once we get the list of selected items in the form of an array, we loop through the array and show its contents in the label called lblMessage.

foreach(string str in
            strEmployees)
{
lblMessage.Text += str + " ";
}
}

The NameValueCollection contains so many methods and properties that we will not be able to provide a comprehensive coverage of all of them. I recommend that you look at http://msdn.microsoft.com/library/default.asp?url=/library/en-us/cpref/html/frlrfSystemCollectionsSpecializedNameValueCollectionClassTopic.asp&WROXEMPTOKEN=620568Zhiox25QOrrsnTPfm53p for further details.

StringCollection

As the name suggests, the StringCollection class is used to contain elements that are of type String. The StringCollection class also allows duplication of its elements. Since the StringCollection class derives from the ICollection and IEnumerable interfaces, the elements in the StringCollection class can be accessed by using either indexes or by implementing a for..each construct.

The following statements can be used to create a StringCollection object and then add elements to it.

StringCollection employees = new
            StringCollection();
employees.Add("Nancy");
employees.Add("Nancy
            with the same key
            value");
employees.Add("Tom");
employees.Add("Tom1");

Putting it all together

Let us summarize what we have learnt about the collection classes by considering the following table, which draws a comparison between the different collection classes we have looked at so far.

Class | Speed | Data Type | Allows Duplicate Values
ArrayList | N/A | Object | Yes
Hashtable | High | Object | No
SortedList | Slow | Object | No
Stack | N/A | Object | Yes
Queue | N/A | Object | Yes
NameValueCollection | N/A | Object | No
StdingCollection | N/A | Stding | Yes

Conclusion

In this article, we have examined the following.

  • Some of the important classes contained in the System.Collections namespace, and their functionality.
  • How to create strongly typed collections using the classes present in the System.Collections.Specialized namespace.
  • How to override the default sorting behavior by incorporating a custom sorting behavior through the use of the IComparable interface.


ASP.NET Server Control Creation

ASP.NET Server Control Creation
http://www.csharptoday.com/content.asp?id=1734
Article Rating: 4
Published on October 22, 2001
Abstract
ASP.NET custom server controls provide a powerful architecture for isolating the atomic functionality of applications, using highly reusable compiled components (dll). This makes enforcing application-wide rules and applying composite view design patterns easy. In this article, Daniel Cazzulino will look at numerous facets of these controls, such as extending their functionality.
Article

As you may already know, ASP.NET custom server controls aren't the same as user controls, which are more like a replacement for include statements (for more on this, read "Composite Control vs. User Control" http://msdn.microsoft.com/library/default.asp?url=/library/en-us/cpguidnf/html/cpconcompositecontrolvsusercontrol.asp&WROXEMPTOKEN=620568Zhiox25QOrrsnTPfm53p), nor are they a cool new way of doing small fancy looking reusable code snippets or cosmetics for your site. In fact, they provide a powerful architecture for isolating the atomic functionality of your application with highly reusable compiled components (DLLs). This should make enforcing application-wide rules and applying composite view design patterns (i.e. portal sites) a breeze. Unfortunately, except for some very simple examples and scarce information, there isn't much help on advanced topics involving server control creation. In this article, we will see how to:

  • Extend ASP.NET server controls to add new functionality
  • Receive and process custom postback events
  • Add a type editor to a property
  • Create a composite control
  • Create a custom type editor
  • Create a custom control builder to customize rendering
  • Raise custom events
  • Add a designer to a control, and provide enhanced design-time support

This article assumes you are familiar with C#, delegates, attributes and common server control behavior. Visual Studio .NET (Beta 2) is not required, but will make things easier!

The RollOverImage control

How many times did you use JavaScript routines, together with some <A> and <IMG> tags, to create "active buttons" functionality? We'll start building a custom server control that provides this behavior, and also shows some advanced options. We will create a new Class Library project, and then add a new class that inherits from the Image web control. We won't extend ImageButton, so that we can see how to add custom events to our controls.

[ToolboxData("<{0}:RollOverImage runat=server></{0}:RollOverImage>")]
public class RollOverImage : System.Web.UI.WebControls.Image, IPostBackEventHandler
{
public event System.EventHandler Click;
private string _rolloverurl = "";
private string _imageurl = "";

The attribute shown is used to customize the behavior or appearance of our control at design time (see http://msdn.microsoft.com/library/default.asp?url=/library/en-us/cpguidnf/html/cpcondesign-timeattributesforcomponents.asp&WROXEMPTOKEN=620568Zhiox25QOrrsnTPfm53p for a complete list). The ToolboxData attribute is used to display the name of the control in the toolbox. We also define a public event of the type System.EventHandler, as we just need a generic event with no custom handler or event arguments. Now, let's write the RollOverUrl property as follows:

[
Bindable(true), Category("Appearance"), DefaultValue(""),
Description("The image displayed where the user rolls over the image with the mouse."),
Editor(typeof(ImageUrlEditor), typeof(UITypeEditor))
]
public string RollOverUrl
{
get { return _rolloverurl; }
set { _rolloverurl = value; }
}

Besides the usual attributes and property assessors, the interesting thing here is the Editor attribute, which tells the property browser which editor to use when changing this property value. This attribute receives two parameters in the constructor: the type that will perform the edition, and its base type. According to the documentation, this will always be UITypeEditor, but I still can't figure out why is it necessary, as it could be obtained through reflection by the attribute's constructor! If we hadn't put the Editor attribute in the property declaration, we would have had a common text property in the property browser, and would have to manually specify the image path, which is an error prone process. The following screenshot shows what we get with the attribute, after clicking on the ellipsis button next to our property:

The ImageUrl property declaration is almost the same. The IPostBackEventHandler interface is needed for our control to receive postback events fired by the user. This interface has only one member, defined as void RaisePostBackEvent(string eventArgument), which we implement to fire our custom event.

//Let inheritors override our default
            implementation of the event
protected virtual void
            OnClick()
{
if (Click != null) { Click(this,
            System.EventArgs.Empty); }
}
public void
            RaisePostBackEvent(string
            eventArgument)
{
OnClick();
}

Now we need a way to cause a postback event on the client side, because as MSDN help says "This control only displays an image. If you need to capture mouse clicks on the image, use the ImageButton control" (from "Image Class" definition, http://msdn.microsoft.com/library/en-us/cpref/html/frlrfSystemWebUIWebControlsImageClassTopic.asp?WROXEMPTOKEN=620568Zhiox25QOrrsnTPfm53p).

We are going to use the AddAttributesToRender(HtmlTextWriter writer) method to append our custom data just before we render the output (the old Response.Write):

protected override void AddAttributesToRender(HtmlTextWriter writer)
{
writer.AddAttribute("onMouseOver", "javascript:this.src='" + _rolloverurl + "'");
writer.AddAttribute("onMouseOut", "javascript:this.src='" + _imageurl + "'");
writer.AddAttribute("onClick", "javascript:" + Page.GetPostBackEventReference(this);
base.AddAttributesToRender(writer);
}

We add our attributes, and then give the base class a chance to add its own. We can easily identify the usual events we are used to handling in client-side JavaScript. The first two provide the client script to swap the src property of the image, to cause the rollover effect. The last one provides the JavaScript function needed to cause the postback, which is nothing more than this (the source view of the page sent to the browser):

<img onMouseOver="javascript:this.src='images/NewHi.gif'"
onMouseOut="javascript:this.src='images/NewLo.gif'"
onClick="javascript:__doPostBack('Image1','')" id="Image1"
src="/customcontrols/images/NewLo.gif" border="0"/>
<!-- This is the postback javascript function -->
<input type="hidden" name="__EVENTTARGET" value="" />
<input type="hidden" name="__EVENTARGUMENT" value="" />
<script language="javascript">
<!--
function __doPostBack(eventTarget, eventArgument) {
var theform = document.HomePage;
theform.__EVENTTARGET.value = eventTarget;
theform.__EVENTARGUMENT.value = eventArgument;
theform.submit();
}
// -->
</script>

This way, we can add postback events to whichever control we like. The GetPostBackEventReference method has an overload that even allows us to pass an additional argument to the server postback event being called, the RaisePostBackEvent method.

Here is what is happening: the client-side script submits the form through the __doPostBack JavaScript function, then the page framework receives this submission, including the __EVENTTARGET and __EVENTARGUMENT hidden fields, which it uses to locate the control that fired the event. It looks for a control with the same ID that implemented IPostBackEventHandler, and calls its RaisePostBackEvent. If we don't implement this interface, we will cause a form submission, but no event processing will occur. Inside this method we raise our own event, Click().

I've tested this control on IE 4+, and NS6+, and it behaves and looks the same on both platforms. However, the JavaScript functions, both roll over, and postback won't work on any NS browsers prior to version 6.

Here we see the result, before and after rollover, on both browsers:

The ToolBar control

Now let's move forward and build a container of RollOverImages. We are going to inherit this control from System.Web.UI.WebControls.WebControl so that we get Width, Height, BorderStyle, etc, automatically. We also need to implement an InamingContainer interface to ensure that there's not going to be a naming conflict with other controls on the page, potentially even another ToolBar. This interface is empty, and is just a marker. Its purpose it to add the container name, followed by an underscore, to all the controls contained in it. For example, a RollOverImage whose ID is "Open" and is inside a ToolBar with ID "tbrMain" will have its ID set at runtime to "tbrMain_Open". This is why System.Web.UI.Control defines two properties: Control.ID and Control.ClientID. Here's the ToolBar declaration:

[
DefaultProperty("Text"),
ToolboxData("<{0}:ToolBar runat=server></{0}:ToolBar>"),
ParseChildren(false),
ControlBuilder(typeof(CustomControls.ToolBarControlBuilder))
]
public class ToolBar : System.Web.UI.WebControls.WebControl, INamingContainer
{
}

The important attributes here are the ParseChildren and ControlBuilder. ParseChildren must be set to false to indicate that child controls don't have to be parsed as properties of the ToolBar control itself. In other words, at control hierarchy recreation time, don't try to locate a property for every inner tag found. This is important, since our child RollOverImage tags are going to be contained inside the outer ToolBar tags.

The other attribute, ControlBuilder, needs some more explanation.

You might be confused about what exactly is meant by a ControlBuilder, a ControlDesigner and a UITypeEditor, so here's what I found from reading the scarce documentation:

  • ControlBuilder: we need this class to tell the parser what to do with the tags and content it finds inside our control's tags. Its main responsibility is to return a type when certain tags are encountered, so the page parser can build the hierarchy (more on this below).
  • ControlDesigner: by inheriting this class, we can create a custom HTML code generator for our control. A complex user interface can be added to edit a control, and based on user input, automatically generate HTML code at design-time. Its main functionality is to provide us with shortcuts in building complex tag trees, the same way the DataGrid designer does.
  • UITypeEditor: we inherit this class when we want to provide our control users with more friendly editors for a property of a certain type. This is the popup form, or custom drop-down menu, that our control user will see when editing the value in the property browser. An example is the ImageUrl type editor shown before.

As the toolbar control is going to be composed by RollOverImage controls, it would be desirable to capture their Click events and raise a unique event for the whole toolbar. We will take advantage of delegates and create a custom event handler and event arguments class.

The steps are:

  • Create the class that inherits from System.EventArgs, which will be the custom arguments provided by our delegate. This is just like any other class, so we can define properties and constructors.

public class ToolBarClickEventArgs : System.EventArgs

  • Define the delegate type

public delegate void ToolBarClickEventHandler(object sender,
ToolBarClickEventArgs e);

  • Declare the event inside our class

public event ToolBarClickEventHandler Click;

By convention, event argument classes should finish with EventArgs, event handler classes with EventHandler, and event raising method (which should be in only one place) OnEventName. In this case, it will be called OnImageClick.

protected virtual void OnImageClick(object
            sender, System.EventArgs e)
{
ToolButtonEnum btn =
            ToolButtonEnum.Custom;
RollOverImage img = (RollOverImage)
            sender;
if (Click != null)
{
try
{
btn =
            (ToolButtonEnum)Enum.Parse(btn.GetType(),
            img.ID);
}
catch
{
// If we can't parse the enum with
            the ID, it means it
// is a custom RollOverImage that has been
            added manually
// in the page
}
Click(this, new
            ToolBarClickEventArgs(btn));
}
}

As we have given the inner controls the same ID as the Enum item, we parse it using the Enum class, and get an Enum type with the value. Just in case we fail, we have set a default value to return. Now, to get this method called when inner images are clicked, we just need to add ourselves to the call chain of the image Click event.

img.Click += new System.EventHandler(OnImageClick);

This call must be placed inside the code that creates the child controls' hierarchy inside our toolbar.

Although we could have used the OnInit, OnPreRender or Render methods to set up our controls, or directly rendered the output, we're going to use CreateChildControls, as it gets called before postback or rendering, and provides easier interaction with the other method we will use, AddParsedSubObject.

Here is the sequence we will implement:

  1. The page framework begins to parse the page.
  2. It encounters a <ToolBar> tag, knows it uses a control builder, and instantiates both.
  3. It then encounters a <RollOverImage> tag, and passes it to the control builder to see what to do.
  4. The control builder tells it to create a RollOverImage control for this tag.
  5. The page framework creates the control and fills its properties.
  6. It calls AddParsedSubObject on the ToolBar, and passes the created child control.
  7. If we receive a RollOverImage control, we add it to the ToolBar private variable holding the collection.
  8. After parsing the whole tag, CreateChildControls is called, and the Controls property is filled with the images found in the private variable.
  9. If it is a postback, after filling all the controls, the page framework calls the RaisePostBackEvent on the RollOverImage control that it corresponds to as it implements the IPostBackEventHadler interface.
  10. If it is a postback, the Click event raised by the control will call the toolbar OnImageClick method added to the handler, and in turn, the toolbar will raise its own Click event.
  11. Finally, Render is called to produce the actual output, where we verify that the child controls have been properly created, and call the base object's Render method.
  12. All objects are discarded.

It is extremely important to know and foresee at which point in the life of the page we want to process things, because this may affect the design and code we need to write.

First, let's look at the control builder that will be called by the parser:

public class ToolBarControlBuilder :
            ControlBuilder
{
/// <summary>
///Return a Type only
            for "rolloverimage" inner tags, so anything
///else inserted
            thought the HTML editor is ignored.
///
            </summary>
public override Type GetChildControlType(String
            tagName,
IDictionary attributes)
{
if
            (String.Compare(tagName, "rolloverimage", true) == 0)
return
            typeof(RollOverImage);
return null;
}

///
            <summary>
///Tell the parser we need an opening and closing
            tag.
/// </summary>
public override bool HasBody()
{
return true;
}
}

That's it for the builder. It is quite simple, as all we need from it is to give the parser the object type to create for a specified tagName, and we only care about one: rolloverimage. We use String.Compare in order to ignore case issues. Now the parser creates the object and fills its properties just like in the case of a single RollOverImage control, and then calls AddSubParsedObject:

protected override void
            AddParsedSubObject(Object obj)
{
if (obj is RollOverImage)
{
_selectedimages.Add((RollOverImage)obj);
this.ChildControlsCreated
            = false;
}
}

We set the flag indicating that the control children need to be recreated, as we altered our private variable.

The following method creates the object's hierarchy:

protected override void
            CreateChildControls()
{
//Set default values
int count = 0;
int maxbuttons = 0;
base.Height = 0;
//Clear any existing
            controls in the collection.
this.Controls.Clear();
foreach
            (RollOverImage img in _selectedimages)
{
count +=
            1;
try
{
this.Controls.Add(img);
base.Width = new
            Unit(base.Width.Value + _IMAGE_WIDTH);
// Add our handler for the
            event
img.Click += new
            System.EventHandler(OnImageClick);
}
catch
{
//Add a
            meaningful error message to the control
Label lbl = new
            Label();
lbl.ForeColor =
            System.Drawing.Color.Red;
lbl.Font.Bold = true;
lbl.Text =
            "Some values in the list of displayed buttons are
            invalid.";
this.Controls.Add(lbl);
this.Controls.Add(new
            LiteralControl("<BR>"));
}

// Take into account the
            buttons to calculate the width.
if (count > maxbuttons)
            maxbuttons = count;
if (count ==
            _buttonsperline)
{
this.Controls.Add(new
            LiteralControl("<BR>"));
count =
            0;
}
}
}

And the rendering method:

protected override void
            Render(HtmlTextWriter output)
{
//Check the child controls and
            render the output though our base
            class
EnsureChildControls();
base.Render(output);
}

With this in place, our toolbar declaration looks like this, and gets rendered perfectly:

<ct:ToolBar id="tbrMain"
            runat="server">
<rolloverimage id="Comments"
            imageurl="images/CommentsLo.gif"
rolloverurl="images/CommentsHi.gif"
            />
<rolloverimage id="Copy" imageurl="images/CopyLo.gif"
rolloverurl="images/CopyHi.gif" />
<rolloverimage
            id="Cut" imageurl="images/CutLo.gif"
rolloverurl="images/CutHi.gif" />
<rolloverimage
            id="Delete" imageurl="images/DeleteLo.gif"
rolloverurl="images/DeleteHi.gif" />
<rolloverimage
            id="Open" imageurl="images/OpenLo.gif"
rolloverurl="images/OpenHi.gif"
            />
</ct:ToolBar>

This is how it looks now:

We have done some work in the RollOverImageCollection class, implementing a custom Enumerator, and an overloaded indexer which will allow us to refer to an element either with the control's ID or by its index, with tbrMain.DisplayedButtons["Open"] or tbrMain.DisplayedButtons[0]. The collection also implements an ICollection interface and IEnumerable.

So, we can treat the events fired by the toolbar in two ways:

private void Open(object sender,
            System.EventArgs e)
{
this.lblResult.Text = "Open
            Clicked!!";
}

private void Save(object sender,
            System.EventArgs e)
{
this.lblResult.Text = "Save
            Clicked!!";
}

private void ToolBarClick(object sender,
            ToolBarClickEventArgs e)
{
switch (e.Button)
{
case
            ToolButtonEnum.Open:
this.lblResult.Text = "Open
            Clicked!!";
break;
case
            ToolButtonEnum.Save:
this.lblResult.Text = "Save
            Clicked!!";
break;
default:
this.lblResult.Text = "Can't
            tell which one Clicked!!";
break;
}

this.lblResult.Text
            = sender.ToString() + " clicked on ";
this.lblResult.Text +=
            System.DateTime.Now.ToLongTimeString();
this.lblResult.Text += "
            - arguments: " + e.Button.ToString();
}

private void
            Page_Init(object sender, EventArgs
            e)
{
InitializeComponent();
this.tbrMain.Click += new
            ToolBarClickEventHandler(ToolBarClick);

// Capture events
            individually on a per button
            basis
this.tbrMain.DisplayedButtons["Open"].Click += new
            EventHandler(Open);
this.tbrMain.DisplayedButtons["Save"].Click
            += new EventHandler(Save);
}

The TypeEditor

Now let's look at the TypeEditor for our RollOverImageCollection class. If we view the ToolBar's DisplayedButtons property declaration in the downloadable code, we find:

[
Category("Appeareance"),
Editor(typeof(RollOverImageCollectionEditor), typeof(UITypeEditor)),
DesignerSerializationVisibility(DesignerSerializationVisibility.Hidden)
]
public RollOverImageCollection DisplayedButtons
{
get { return _selectedimages; }
set
{
_selectedimages = value;
this.ChildControlsCreated = false;
EnsureChildControls();
}
}

We could have used the overloaded version of the EditorAttribute that receives a string in the constructor. The advantages would have been that the assembly containing these design-time components wouldn't be needed at compile-time, and when deploying the application we wouldn't need to distribute design-time components that wouldn't be used, effectively reducing the distributables' size. However, for type-safety, we use the other method.

The DesignerSerializationVisibilityAttribute tells the page designer not to persist this property in the toolbar tag as an attribute. If we don't disable this behavior, what we get is an attribute that looks like this:

<ct:ToolBar id="tbrMain" runat="server" DisplayedButtons="(Collection)">

And the control won't get rendered again, as it can't recreate a RollOverImageCollection object from the string "(Collection)".

In its most basic form, to get a type editor up and running, we will need to override the GetEditStyle and EditValue methods inherited from UITypeEditor.

public override UITypeEditorEditStyle GetEditStyle(ITypeDescriptorContext context)
{
return UITypeEditorEditStyle.Modal;
}

public override object EditValue(ITypeDescriptorContext context, IServiceProvider provider, object value)
{
RollOverImageCollection col = (RollOverImageCollection) value;
RollOverImageCollectionForm form = new
RollOverImageCollectionForm(col.Clone());
if (form.ShowDialog() == System.Windows.Forms.DialogResult.OK)
{ return form.SelectedImages; }
else
{ return col; }
}

First we tell the host (the property browser or whatever) that our editor is going to be modal. We receive the object to be edited in the EditValue method. From here, it's good old Windows Forms programming! We create a backup clone of the received object, just in case the user cancels the action. We create a new form, show it, and return the result. The Windows Form looks like this:

I won't be showing the code at this point, as this isn't a Win Forms tutorial. You know how to create it, add controls, dock them, add events, create a constructor, and return results from the selection. Some advice: use the DialogResult property of buttons, and/or the form AcceptButton and CancelButton properties, as they will save you some coding, and is just what I missed about Delphi! (Thanks Anders Hejlsberg.)

When the form returns, the property is changed and that's all. However, this isn't at all helpful if we can't persist this selection in the designed page. It is strange, but when the changed property doesn't get persisted, we continue to view the new selected images in the page designer, but as soon as we close the page and re-open it, or compile the project, we loose them altogether. So what we need is a Designer.

The Designer

First, let's add an attribute to the ToolBar's declaration, which will look like this:

[
DefaultProperty("Text"),
ToolboxData("<{0}:ToolBar runat=server></{0}:ToolBar>"),
ParseChildren(false),
ControlBuilder(typeof(CustomControls.ToolBarControlBuilder)),
Designer(typeof(CustomControls.Design.ToolBarDesigner))
]
public class ToolBar : System.Web.UI.WebControls.WebControl, INamingContainer
{
}

The same thing we said about passing a string to the attribute constructor applies to the DesignerAttribute declaration. At a minimum, we will want to override GetPersistInnerHtml, GetErrorDesignTimeHtml and Initialize.

public class ToolBarDesigner : ControlDesigner
{
private ToolBar _toolbar;

public override void Initialize(IComponent component)
{
base.Initialize(component);
_toolbar = (ToolBar) component;
}

protected override string GetErrorDesignTimeHtml(Exception e)
{
return CreatePlaceHolderDesignTimeHtml("There was an error rendering the
control.<br>Check to make sure all properties are valid.");
}

public override string GetPersistInnerHtml()
{
System.IO.StringWriter text = new System.IO.StringWriter();
HtmlTextWriter writer = new HtmlTextWriter(text);

foreach (RollOverImage img in _toolbar.DisplayedButtons)
{
writer.WriteBeginTag("rolloverimage");
writer.WriteAttribute("id", img.ID);
writer.WriteAttribute("imageurl", img.ImageUrl);
writer.WriteAttribute("rolloverurl", img.RollOverUrl);
writer.Write(" />");
}

return text.ToString();
}

In GetPersistInnerHtml, we used an HtmlTextWriter, based on a StringWriter, to generate the output. The string returned by this method is placed in the code inside our Toolbar's tags. From there, it is just an iteration through the component property, combined with an output generator. We can add another enclosing tag, <displayedbuttons>, but we must ignore it in the ControlBuilder. If we return the type RollOverImageCollection, the parser will keep saying it can't find a RollOverImage property on the collection object. Now, as soon as we accept changes in the type editor, the selected images will be written to the HTML source. The tags will look just as if we had added them manually, as we did previously.

The designer also allows us to add "verbs", which show up on the popup menu of the control, and at the bottom of the property browser.

We are going to add a verb and provide the same editing capabilities we put on the type editor.

public override DesignerVerbCollection Verbs
{
get
{
DesignerVerb[] verbs = new DesignerVerb[]
{ new DesignerVerb("Edit Images...", new System.EventHandler(EditImages)) };
return new DesignerVerbCollection(verbs);
}
}

public void EditImages(object sender, System.EventArgs e)
{
RollOverImageCollection col = _toolbar.DisplayedButtons;
RollOverImageCollectionForm form = new
RollOverImageCollectionForm(col.Clone());
if (form.ShowDialog() == System.Windows.Forms.DialogResult.OK)
{
_toolbar.DisplayedButtons = form.SelectedImages;
UpdateDesignTimeHtml();
}
}

As you can see, there are some differences in EditImages compared to EditValue in the type editor. First, we don't need to return any value, because we have access from the designer to the whole object being designed. Second, we have to call UpdateDesignTimeHtml to get the output written in the page.

The downloadable code has been extensively commented to help understanding.

Conclusion

As you can see, web application creators have been given a powerful tool for building reusable components. As I stated at the beginning, this isn't just cosmetics, it's a fundamental shift from the way ASP developers used to think. This is because the days of copy & paste and #include are gone for good, but you need to understand concepts never used before in web development. However, once you get to master its secrets, your web apps will get on a par with the best n-tier projects around.

We have learnt to extend existing controls, add custom tags to the control design-time rendering, improve programmer productivity by providing type editors and designers, and take advantage of attributes.

You can use these techniques to provide your developers with custom controls wrapping business logic, such as user authentication, standard DML manipulation, site navigation, etc., avoiding code duplication, enforcing your development standards and improving productivity. You could even give these controls to business partners, as an easy to use front-end to your services (maybe web services).

If you'd like to discuss a broader range of C#Today topics and issues, or if you have any C# related questions you need answers to, please visit our online resource centres at : http://p2p.wrox.com/list.asp?list=csharptoday_discuss&WROXEMPTOKEN=620568Zhiox25QOrrsnTPfm53p and http://p2p.wrox.com/list.asp?list=c_sharp&WROXEMPTOKEN=620568Zhiox25QOrrsnTPfm53p


Using the TreeView Control

Using the TreeView Control
http://www.csharptoday.com/content.asp?id=1683
Article Rating: 3.5
Published on October 19, 2001
Abstract
One of the most common ways of representing data, and one used all the time by Microsoft, is in the form of a tree. In this article, Matthew Reynolds takes a look at how we can build an application that uses the classic "Tree on left, data on right" TreeView approach (a familiar example of this being Windows Explorer). We will see how to create an extensible architecture that provides a decent separation between data and presentation, and also see how to create user controls to support the editing of each of the application items.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

One common UI metaphor that Microsoft has been using for years now is representing data in a tree form. I myself have built many applications over the years that use trees to allow a user to navigate an application, and now that I work exclusively in .NET I find I'm constantly creating instances of the TreeView control.

In this article we'll take a look at how we can build an application that uses the classic "Tree on left, data on right" approach that's so familiar.

The project in this article was built with .NET Beta 2 Build 9254.

Building the Project

Let's kick off straight away by building a project. Create a new Visual C# - Windows Application project and call it TreeView.

Usually, whenever we see a tree, we are given a slider to the right of it that lets us stretch out the width of the tree so that we can see all of the nodes. The automatic layout feature of .NET lets us do this very easily by using Panel and Splitter controls.

First of all, on the Designer for Form1 add a new Panel control. Set the Dock property to Left.

Now, add a splitter control. This will automatically dock to the left, next to the first panel.

Finally, add another Panel control. Set its Name property to panelViewPane and set its Dock property to Fill.

You can't see it very clearly in this screenshot, but basically it fills the entire right portion of the form.

Now, add a TreeView control to the first panel. Set its Name property to treeviewObjects and its Anchor property to Top, Bottom, Left, Right.

Now we have an application that follows the classic tree metaphor that we're all familiar with. Let's look now at how to populate this tree and make the application do something exciting.

Adding Nodes

In this article, we'll see one way that the TreeView control can be used in an application. This is very much "my way" of doing it and I'll be presenting a couple of improvements to the way Microsoft expects us to use the TreeView. However, be aware, this isn't necessarily the "right way" - feel free to explore and find the best solution for you.

Most applications that use a tree have the concept of a master (or "root") object that contains other types of objects. Typically, types of objects are grouped together. In Visual Studio's Solution Explorer, the root object is the solution itself. The solution contains projects, which in turn contain documents and other items. Some of these other items (for example References) act as containers for specific kinds of objects, in this case assemblies that are referenced in the project.

As well as there being two distinct kinds of nodes - container nodes and document nodes - nodes are almost always linked to an internal object representation that gives them some action. For example, if I select any of the nodes in the tree, the Properties window updates to present more information about the node. Likewise, all of the nodes respond to a right-click from the mouse and some nodes respond to double-clicking by opening up a document.

However, Solution Explorer doesn't follow the classic "tree on left, data on right" model. Typically in this model, selecting a node changes the data displayed on the right. To see this in action, open Windows Explorer. Notice how selecting a folder from the tree always causes the items contained within that object to be displayed on the right. As we're trying to follow this classic model, selecting nodes in our tree will update the data on the right. We will eventually implement the right-click, or "context menu" and we'll ignore double-clicking.

To add nodes to the tree, the TreeView expects to be given populated System.Windows.Forms.TreeNode objects. Each TreeNode object, as well as the TreeView control itself supports a property called Nodes that returns a System.Windows.Forms.TreeNodeCollection object. Adding the new TreeNode objects to this collection causes the nodes to be added to the tree, and therefore displayed.

In fact, there's a little more to it than this. Ideally, we need to derive new classes from TreeNode specific to the kinds of data that we want to represent. So, if we want to represent a project, we might create a class called ProjectNode. If we want to represent a document, we might create a class called DocumentNode. These classes themselves control their own appearance in terms of the icons that are used and the text that's displayed, but also control the list of nodes that appear under them. So, a class called DocumentsNode (notice the plural) would be in charge of digging through the project looking for documents, creating new DocumentNode objects and adding them to its own Nodes collection. (We'll do this in a moment, so hang on if it's getting fuzzy).

What's important is that the ProjectNode class is totally distinct and separate from whatever class is actually holding the project, i.e. there must be a clear and distinct separation of data and presentation in this model. The Project class (if we were to call it that) contains all of the data about the project, but hands off to ProjectNode to present it to the tree.

Without getting too deep into this subject, the reason why we do this is related to serialization. If we want to save the project, we only want to save the project data, not the details of the presentation as this can be dynamically recreated at load time. The split we've discussed doesn't confuse the serialization process by expecting the TreeNode-derived objects to serialize themselves.

At Last. Adding Nodes

One thing I alluded to earlier was the fact that TreeNode as it stands has some shortcomings. One particular point is that the node does not support an event, or other notification mechanism, as to when the node itself is selected in the tree. We'll create a new class called TreeNodeBase that defines an override-able OnSelect method.

Firstly though, we need classes to hold a project. Create a new class called Project. When the code editor appears, add a namespace import declaration to System.Collections:

using System;
using System.Collections;

Then, add a member called Documents to the Project class:

public class Project
{
// members...
public ArrayList Documents = new ArrayList();

Now, create a new class called Document. This will hold two blocks of text, so add these members:

   public class
            Document
{
// members...
public static int NextId =
            1;
public int Id;
private string _title = "";
private
            string _text = "";

Next, replace the default constructor with this one:

      public
            Document()
{
// set the id...
Id =
            NextId;
NextId++;
}

Then, add this new version of ToString:

      public
            override string ToString()
{
if(Title != null)
return
            Title;
else
return "Untitled" +
Id.ToString();
}

Finally, add these properties:

      public
            string Title
{
get
{
return
            _title;
}
set
{
_title = value;
}
}

public
            string Text
{
get
{
return
            _text;
}
set
{
_text = value;
}
}

To make life easier for us initially, open the Project class again and replace the default constructor with this one:

      public
            Project()
{
// create some documents...
for(int n = 0; n
            < 5; n++)
Documents.Add(new Document());
}

Each node in the tree, as well as having its own storage requirements, must be able to reference both the project and the top-level node in the tree. This, together with the OnSelect method will make up our initial TreeNodeBase implementation. Create a new class called TreeNodeBase, and first of all add these namespace import declarations:

using System;
using System.Windows.Forms;

Now, set the class to inherit from TreeNode. This will enable us to add instances of classes derived from TreeNodeBase to the tree.

public class TreeNodeBase : TreeNode
{

Each node in the tree ultimately needs access to the Project instance in addition to having its own storage requirements. To TreeNodeBase, add this member and constructor:

   public class TreeNodeBase
            : TreeNode
{
// members...
public Project
            Project;

// constructor...
public TreeNodeBase(Project
            project)
{
this.Project = project;
}
}

As we discussed before, add this method:

// OnSelect...
public virtual void OnSelect(Form1 appView)
{
}

OK, now we have a base object, let's create ProjectNode. Add a new class called ProjectNode and set it to derive from TreeNodeBase.

public class ProjectNode : TreeNodeBase
{

The constructor for this class needs to take a Project object, but it also needs to pass that object back to the TreeNodeBase constructor. We do that by using the base keyword. In addition, the constructor also needs to update its text. Add this new constructor:

public ProjectNode(Project project) : base(project)
{
// text...
this.Text = "MyProject";
}

The last step is to get Form1 to create a Project object, and add a new ProjectNode to the tree. Open the code editor for Form1 and add this member:

public class Form1 : System.Windows.Forms.Form
{
// members...
public Project Project;

Next, find the constructor for Form1 and add this code to create a new Project object, and add a new ProjectNode instance to the tree:

      public
            Form1()
{
//
// Required for Windows Form Designer
            support
//
InitializeComponent();

// create a
            project...
this.Project = new Project();

// add
            it...
treeviewObjects.Nodes.Add(new
            ProjectNode(this.Project));
}

Run the project now, and you should see the project added to the tree.

OK, so that's done next to nothing, but you should be able to see the principle. Let's look now at how we can add the documents to the tree.

Adding Documents

In this section, we're going to add a new node to the tree called Documents. This node will contain the documents that are part of the project (remember, we create five Document objects whenever we create a Project object). We'll need two new classes: DocumentsNode and DocumentNode.

Create a new class called DocumentsNode. Set it to inherit from TreeNodeBase.

public class DocumentsNode : TreeNodeBase
{

Again, we need an adapted constructor. This time though, as well as setting the text for the node, we also need to iterate the Document objects referenced in the project and create new DocumentNode objects for each. Each new DocumentNode object is added to the Nodes collection, and therefore displayed on the tree.

      public
            DocumentsNode(Project project) : base(project)
{
// set the
            text...
this.Text = "Documents";

// add the
            documents...
foreach(Document document in
            project.Documents)
{
// create a new document
            node...
DocumentNode newNode = new DocumentNode(project,
            document);

// add
            it...
this.Nodes.Add(newNode);
}

//
            expand...
this.Expand();
}

Of course, this won't work without DocumentNode being implemented, so create a new class called DocumentNode and as usual set it to derive from TreeNodeBase.

public class DocumentNode : TreeNodeBase
{

DocumentNode objects also need to know what document they refer to. Add this member:

public class DocumentNode : TreeNodeBase
{
// members...
public Document Document;

Now, add this constructor. We'll use the ToString method on the document to get the text.

      public
            DocumentNode(Project project, Document document) :
            base(project)
{
// set the document...
this.Document =
            document;

// set the text...
this.Text =
            this.Document.ToString();
}

As each node should be responsible for managing the nodes that it contains, it is up to ProjectNode to create and manage the presentation of DocumentsNode. Add this code to the constructor on ProjectNode:

public ProjectNode(Project project) : base(project)
{
// text...
this.Text = "MyProject";

// create documents...
this.Nodes.Add(new DocumentsNode(project));

// expand...
this.Expand();
}

Run the project and you should see this:

Selecting Nodes

At this point we can prove that we can update the TreeView control and display the documents that make up the project. What we need to do now is get the project to a state where selecting document nodes displays fields on the right hand side of the form that allow us to edit the document.

To do this, we're going to create a new User Control called DocumentView. Whenever the node is selected, we'll create a new instance of the view and pass it back to Form1 to display.

Create a new User Control now called DocumentView. Add two text boxes:

  • On the first text box, set the Name property to textTitle. Set the Anchor property to Top, Left, Right.
  • On the second text box, set the Name property to textText. Set the Anchor property to Top, Bottom, Left, Right. Also, set the MultiLine property to True and the ScrollBars property to Vertical.

Open the code editor for DocumentView and add this private member:

public class DocumentView : System.Windows.Forms.UserControl
{
// members...
private Document _document;

Then, add this property. Whenever Document is set, we'll update the data bindings on the two controls so that the Title and Text properties are displayed.

// Document property...
public Document Document
{
get
{
return _document;
}
set
{
// set...
_document = value;

// update...
textTitle.DataBindings.Add("Text", _document, "Title");
textText.DataBindings.Add("Text", _document, "Text");
}
}

Now all that remains is to wire up the selection process. Whenever the selection on the TreeView is changed, an AfterSelect event will be thrown. Open the Designer for Form1 and double-click on the TreeView. Add this code:

private void treeviewObjects_AfterSelect(object sender,
System.Windows.Forms.TreeViewEventArgs e)
{
// get the node...
TreeNodeBase node = (TreeNodeBase)e.Node;
node.OnSelect(this);
}

This event handler will case the TreeNode we return into a TreeNodeBase. We then call OnSelect on this node.

When the DocumentNode receives OnSelect, it's at this point that we'll create our new DocumentView object and populate it with the Document object that DocumentNode is linked to. To display the control, we need to give it to Form1 and let it worry about how to display it.

The Panel control can contain controls. The Controls property on this control allows us to add new controls to be displayed on the panel. We're going to abstract panelViewPane.Controls into a property called View on Form1. When requested, this will return the first entry in the collection. When set, this will clear the collection and add the control we were given to the collection. We also set the control to fill the panel by setting its Dock property to Fill. Add this code:

      // View
            property...
public Control
            View
{
get
{
if(panelViewPane.Controls.Count ==
            0)
return null;

return
            panelViewPane.Controls[0];
}
set
{
//
            reset...
panelViewPane.Controls.Clear();

//
            set...
if(value != null)
{
// dock...
value.Dock =
            DockStyle.Fill;

//
            add...
panelViewPane.Controls.Add(value);
}
}
}

Now, open DocumentNode. Add this code:

      //
            OnSelect...
public override void OnSelect(Form1 app)
{
//
            create a control...
DocumentView view = new
            DocumentView();
view.Document = this.Document;

// set
            it...
app.View = view;
}

Run the project and you'll see how a new instance of DocumentView is created and displayed.

Adding Images

Usually when working with tree controls, we expect to see icons representing each of the items. Let's briefly see how to do this.

First of all, we need some images, and it's best to use image files. Visual Studio .NET includes an icon editor that lets us design our own. To create images, use Solution Explorer to create a new icon file. In this example, we've grabbed Solution Explorer 's icons, and you can find these in the download package that accompanies the article.

The TreeView control needs to get its images from an ImageList control. Open the Designer for Form1 and add a new ImageList control. Set its Name property to imagelistTree.

Now, find and select the Images property and click the ellipsis ("... ") button. Add the images to the control.

Close the collection editor. Select the TreeView control and set its ImageList property to imagelistTree.

As each node is responsible for its own presentation, we need to change the constructor of each of the three TreeNodeBase-derived classes to set the icons.

public ProjectNode(Project project) : base(project)
{
// text...
this.Text = "MyProject";
this.ImageIndex = 0;
this.SelectedImageIndex = this.ImageIndex;

and:

public DocumentsNode(Project project) : base(project)
{
// set the text...
this.Text = "Documents";
this.ImageIndex = 1;
this.SelectedImageIndex = this.ImageIndex;

and:

      public
            DocumentNode(Project project, Document document) :
            base(project)
{
// set the document...
this.Document =
            document;

// set the text...
this.Text =
            this.Document.ToString();
this.ImageIndex =
            2;
this.SelectedImageIndex = this.ImageIndex;
}

Run the project and you should see the images:

Adding a Default View

At the moment, if you select the MyProject or Documents nodes, the display is not properly updated - either you see nothing, or you see whatever document editor was previously displayed. Ideally, we need to a default view that shows something when a node does not specifically provide an editor.

One cool thing we can do is use a ListView control to display a list of the nodes contained within the node on the right hand side whenever MyProject or Documents is selected. This is in keeping with how the Management Console works - when no specific display is available, it shows a list of contained objects.

To do this, create a new User Control called DefaultView. Add a new ListView control to the form. Set its Name property to listviewNodes and its Anchor property to Top, Bottom, Left, Right. In addition, set the View property to Details. Using the Columns property, add a new column called Name.

Open the code editor for DefaultView and add this private member:

   public class DefaultView :
            System.Windows.Forms.UserControl
{
// members...
private
            TreeNode _node;
public Form1 AppView;

private
            System.Windows.Forms.ListView listviewNodes;
private
            System.Windows.Forms.ColumnHeader columnHeader1;
private
            System.ComponentModel.Container components = null;

public
            DefaultView()
{
InitializeComponent();
}

protected
            override void Dispose( bool disposing )
{
if( disposing
            )
{
if(components !=
            null)
{
components.Dispose();
}
}
base.Dispose(
            disposing );
}

There's no specific need to use TreeNodeBase rather than TreeNode, so we'll use the simpler version of the control to give us the maximum potential for reusing the control.

Next, add this property to give the ListView control an image list. We'll use this to give the list the same image list used by the TreeView.

// ImageList property...
public ImageList SmallImageList
{
get
{
return listviewNodes.SmallImageList;
}
set
{
listviewNodes.SmallImageList = value;
}
}

When the Node property on the view is set, we need to loop through all of the contained nodes and create new ListViewItem objects for each and add them to the list. We also need to assign an image to each item that matches the image used on the node, and seeing as both the ListView and the TreeView will end up using the same image list, we can use the same indexes.

      // Node
            property...
public TreeNode Node
{
get
{
return
            _node;
}
set
{
// set...
_node = value;

//
            reset...
listviewNodes.Items.Clear();

// each
            node...
foreach(TreeNode node in _node.Nodes)
{
// create a
            new item...
ListViewItem newItem = new
            ListViewItem(node.Text);
newItem.ImageIndex =
            node.ImageIndex;

// add
            it...
listviewNodes.Items.Add(newItem);
}
}
}

All that remains now is to tweak the tree's AfterSelect handler. Before the node's OnSelect method is called, we need to clear the view by setting the View property to null. If after calling OnSelect, View is still null, we take our cue and create a new DefaultView control, populate it and show it.

      private
            void treeviewObjects_AfterSelect(object sender,
System.Windows.Forms.TreeViewEventArgs e)
{
// get the
            node...
TreeNodeBase node = (TreeNodeBase)e.Node;

// clear
            the view...
View = null;

//
            select...
node.OnSelect(this);

// do we have
            one?
if(View == null)
{
// default...
DefaultView
            newView = new DefaultView();
newView.SmallImageList =
            treeviewObjects.ImageList;
newView.Node = node;

// set
            it...
View = newView;
}
}

Run the project and you'll notice that whenever the MyProject or Documents node is selected, the child nodes are displayed in the list.

Summary

In this article, we looked at using the TreeView control to create the classic "tree on left, data on right" metaphor so common in Windows, not least of all in Windows Explorer itself. We saw how to create an extensible architecture that provides a decent separation between data and presentation and saw how to create user controls to support the editing of each of the application items.


Creating a flexible sorting procedure using IComparators and the QuickSort Algorithm

Creating a flexible sorting procedure using IComparators and the QuickSort Algorithm
http://www.csharptoday.com/content.asp?id=1590
Article Rating: 2.5
Published on October 12, 2001
Abstract
There are many cases where a sorting functionality is needed for an application. For simple applications, this sorting is done on arrays containing simple value objects such as integers or strings. However, most applications have custom classes and data structures specifically designed and implemented for that application. As such, it is difficult to create a sorting procedure that can be reused in both the simple and the more complex application. The solution that is the use of a standard sorting algorithm along with a comparator class, and John Schenken takes a look at this in his article. The sorting algorithm is written just like normal, except that instead of doing direct comparisons, the comparator object is used to return the result of a comparison.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

There are many cases where a sorting functionality is needed for an application that is being developed. For simple applications, this sorting is done on arrays containing simple, single value objects such as integers or strings. However, most applications have custom classes and data structures specifically designed and implemented for that application. As such, it is difficult to create a sorting procedure that can be reused in both a simple and a more complex application. In order to create a reusable section of code, we need to make that code extremely flexible, while still keeping the code simple and manageable.

The solution that provides simplicity, manageability, and flexibility, is the use of a standard sorting algorithm along with a comparator class. The sorting algorithm is written just like normal, except that instead of doing direct comparisons, the comparator object is used to return the result of a comparison. It may be hard to visualize exactly how this works, so we will look at a quick example.

Typically a sort method would compare an item in an array (an integer, for example) to another item (integer) and determine if one is greater than the other. An example of this would be:

(a[lo] < Mid)

Conversely, when using a comparator object, that same comparison would be done using this code:

(objCompare.Compare(a[lo],oMid) < 0)

The typical convention for comparisons is to return a value less than 0 if the first item is less than the second item, a value of 0 if the two items are equal, and a value of greater than 0 if the first item is greater than the second item.

At this point, many will notice that this sounds very much like the IComparable interface that is provided in Visual Studio .NET. It is important to address this question, as it goes to the core of this article. Put simply, implementing the CompareTo method (as required by the IComparable interface) requires the developer to design and implement only one comparison criteria. An object cannot easily be created to compare on different criteria. By contrast, when the comparison is not tied to the object's implementation, then it is much easier to create multiple comparison criteria. Implementing an IComparable interface works very well for the applications where objects and data structures contain only single values, or where they are always sorted on the same property, but it is in the cases where an application would benefit from quickly being able to sort on any of N properties on an object that the comparator objects show their advantage.

The comparator object is responsible for performing the comparison of the objects, or properties of the objects, and returning the correct value. This object will be passed two instances of an object, perform the appropriate comparison, and then return the result of that comparison. The implementation of the comparator object determines how the comparison is done. The comparison can be implemented to compare the objects themselves, single properties of each object, or multiple properties of each object. The limitations are solely in the mind of the developer implementing the comparator. In order to create a new comparison criteria, a developer need only implement a new comparator object. In order to enforce the interface of a comparator object, it is necessary to define an IComparator interface. Any comparator object will need to implement this interface in order to be used from the sorting algorithm. The full details of this interface will be described later in the article. For what we are designing in this article, the comparator object is the more difficult concept to grasp. The sorting algorithm is very simple and requires few changes in order to support comparators. Any current sorting algorithm implementation can be converted to use comparators, but for this article we will be using the Quick Sort algorithm. The Quick Sort was chosen simply for its speed of sorting. For more details on various sorting algorithms and their implementations, point your browser to http://max.cs.kzoo.edu/~abrady/java/sorting/?WROXEMPTOKEN=620568Zhiox25QOrrsnTPfm53p. Although the sorts are implemented in java, the site is useful for some background reading of sorting. Additionally, since C# and Java are fairly similar, the java code on that site was used for creating the C# version of the Quick Sort algorithm.

Implementing the Comparator classes

Seeing as the comparator classes are the key feature in this article, it is important that we focus on them first. As we noted, the purpose of a comparator is to take two objects, compare them in the desired manner, and return the result of the comparison as an integer value. Since it is important that all implementations of comparator classes provide the same methods, we will first define an interface. As the interface will be used for comparator classes, we will call it the IComparator interface. The only method that is required for a comparator is Compare, which takes two objects as its parameters. Thus the IComparator interface would be declared in an IComparator.cs file as:

using System;

namespace
            ComparatorExample
{
public interface IComparator
{
int
            Compare(object objFirst, object objSecond);
}
}

This interface states that a comparator class will provide a Compare method, this method will take two objects as parameters, and then return an integer value. By writing comparators to use this interface, we have made the comparators pluggable. Any class that implements this interface can now be used in this application.

On the subject of implementation, that is the next step we need to take. The IComparator interface only defines which methods must be provided, but not the implementation of those methods. This is where the power and flexibility of comparators becomes apparent. It is up to the developer to determine how a comparison is performed. To start with a simple example, we will look at creating an IntegerComparator class that implements the IComparator interface. Here is the code for the IntegerComparator.cs class:

using System;

namespace
            ComparatorExample
{
/// <summary>
/// Comparator
            class that performs comparisions on integer objects.
///
            </summary>
public class IntegerComparator :
            IComparator
{
public IntegerComparator()
{
// Nothing to
            do here
}

public int Compare(object objOne, object
            objTwo)
{
int iOne = (int)objOne;
int iTwo =
            (int)objTwo;

return
            iOne.CompareTo(iTwo);
}
}
}

As you can see, this is a simple comparator. All that is necessary is to implement the Compare method. In this case, the Compare method first converts the two objects into integer objects. Since C# provides a CompareTo method for the integer (as well as many other types) objects, it is not necessary to write our own.

In reality, applications require the ability to sort on specific properties of an object, not necessarily on the object itself. It is in these applications that the comparators really shine. Lets take the example of an order tracking system. We have a class, orderInfo, which exposes properties for: Order Number, Customer Name, and Order Date. By using comparator objects, we can quickly and easily sort an array of orderInfo classes on any of these three properties. In order to do so, we must first implement comparators for each field. The attached code includes the full implementation of OrderDateComparator.cs, OrderNumberComparator.cs, and CustomerNameComparator.cs. Here we will only focus on the implementation details of the Compare method within each comparator.

These three comparators were chosen specifically to illustrate sorting on different data types while still using the same sorting algorithm. The OrderDateComparator does a comparison of DateTime types, the OrderNumberComparator does comparisons of integers, and the CustomerNameComparator does a comparison of strings. All of the comparators have the same basic approach:

  1. Cast the objects into orderInfo objects
  2. Retrieve the desired property from the orderInfo object
  3. Use the CompareTo method to perform the comparison.

Since the approach for the three classes is so similar, we will only look at the OrderDateComparator implementation of the Compare method:

public int Compare(object objOne, object objTwo)
{
// Convert the passé in objects into orderInfo type objects
orderInfo orderOne = (orderInfo)objOne;
orderInfo orderTwo = (orderInfo)objTwo;

// Perform the comparison using the OrderDate property
return (orderOne.getOrderDate().CompareTo(orderTwo.getOrderDate()));
}

The only difference in the other two comparators is that they use either the getOrderNumber() or the getCustomerName() method instead of the getOrderDate() method.

Implementing the Quick Sort Algorithm in C#

Now that we have created a number of comparator objects, all implementing the IComparator interface, we, we will look into the aspect of actually creating the Quick Sort algorithm that will make use of these objects. As mentioned earlier, the C# implementation here is evolved from the java code QSortAlgorithm.java, authored by James Gosling (his personal site is located at http://java.sun.com/people/jag/?WROXEMPTOKEN=620568Zhiox25QOrrsnTPfm53p). Many sites host pages displaying this code, and it can be found at http://max.cs.kzoo.edu/~abrady/java/sorting/QSort.html?WROXEMPTOKEN=620568Zhiox25QOrrsnTPfm53p as well.

The key points of the code are its use of objects instead of explicit types (int, String, etc.). Instead of the comparator object requiring a specific data type, it is capable of accepting any data type. Then, within the comparator, that object is cast into the correct data type. Also the Quick Sort method has an additional parameter that is used to pass in the comparator object (any object which implements the IComparator interface). Within the code, the only strongly types variables are the integers used to navigate through the array. The Quick Sort method has no idea of the types of objects it is sorting, and that is what leads to its flexibility. Since the method is not specifically written to support/sort a specific data type, it is able to sort any data type (as long as there is a corresponding comparator for that data type).

The full code of the quickSort.cs can be downloaded from this article, but the important code from this file is:

public void QuickSort(object[] a, int lo0,
            int hi0, IComparator objCompare)
{
int lo = lo0;
int hi =
            hi0;
object oMid;

if ( hi0 > lo0)
{
/*
            Arbitrarily establishing partition element as the
* midpoint of*
            the array.
*/
oMid = a[ ( lo0 + hi0 ) / 2 ];

// loop
            through the array until indices cross
while( lo <= hi
            )
{
/* find the first element that is
*greater than or
            equal to
* the partition element starting from the
* left
            Index.
*/
while((lo < hi0) &&
(objCompare.Compare(a[lo],oMid)<
            0))
{
++lo;
}

/* find an element that is smaller
            than
* or equal to the partition element
* starting from the
            right Index.
*/
while(( hi > lo0) &&
            (objCompare.Compare(a[hi],oMid) > 0
            ))
{
--hi;
}

// if the indexes have not crossed,
            swap
if( lo <= hi )
{
swap(a, lo,
            hi);
++lo;
--hi;
}
}

/* If the right index has
            not reached the left
* side of array must now sort the left
            partition.
*/
if( lo0 < hi )
QuickSort( a, lo0, hi,
            objCompare );

/* If the left index has not reached the right
* side of array must now sort the right partition.
*/
if(
            lo < hi0 )
QuickSort( a, lo, hi0, objCompare
            );

}
}

There are really only four lines that have changed. Two of those changes are the change to use a comparator on the objects instead of performing a direct comparison. In the 'standard' implementation of Quick Sort, the method would contain two lines similar to these:

while( ( lo < hi0 ) && ( a[lo] < mid ) )
.
while( ( hi > lo0 ) && ( a[hi] > mid ) )

For the Quick Sort implemented with comparator objects, those two lines would instead look like this:

   while((lo < hi0)
            &&
(objCompare.Compare(a[lo],oMid)<
            0))
.
while(( hi > lo0) &&
            (bjCompare.Compare(a[hi],oMid) > 0 ))

The other important difference in this version of Quick Sort is that the lines which recursively call the Quick Sort method must also pass along the objCompare comparator instance. This is necessary so that all of the recursive calls use the correct comparator object. There are other approaches though. It would be possible to set a private m_Comparator member variable on the QSortAlgorithm class. Then, before calling the Quick Sort method on the class, the calling code would first have to set that to the correct comparator. There are advantages and disadvantages to each approach. Having a method that first requires the calling of another method is not as simple as providing a single method that has parameters for specifying all of the required information.

As can be seen, the code for a comparator based Quick Sort class is very similar to a typical Quick Sort implementation. The normal implementation would need to know they type of objects being compared, or at least which property of the object was to be compared. When dealing with an object that exposes many properties, it would be a giant chore to write a separate Quick Sort method for each property. Instead, by using a comparator object, it is only necessary to create one Quick Sort method. It is still necessary to create a comparator class for each property in order to be able to sort based on that property.

Showtime: Sorting an array using QuickSort and comparators

Seeing how we have covered the design and implementation of both comparators and the Quick Sort algorithm, the next logical step is to detail how the two work together to make a flexible sorting procedure. For illustrating this we will again use the example of sorting an array of orderInfo class instances. The attached ComparatorExample application contains the code that we will cover in this section.

The first thing we need is an array of orderInfo instances. In many applications this array will be retrieved from a database or other source. For this application, we will just be creating an array and populating it ourselves. The application contains this code to fill the array:

m_Orders[0] = new
            orderInfo(159985,"Smithers",new
            DateTime(1999,11,05));
m_Orders[1] = new
            orderInfo(154485,"Johnson",new DateTime(1998,01,25));
m_Orders[2]
            = new orderInfo(165808,"Wilson",new
            DateTime(2000,7,18));
m_Orders[3] = new
            orderInfo(240658,"Edwin",new DateTime(2001,8,12));
m_Orders[4] =
            new orderInfo(9958,"Barbury",new DateTime(1997,12,22));

When the application is executed, you will see this dialog appear:

When first launched, the orders are not sorted in any specific order (other than the order in which they were added to the listbox). When the user selects a sort criteria (Order Date, Order Number, or Customer Name) and then click the Sort button, the list will be re-sorted based on that criteria. The code that effects that action is:

private void btnSort_Click(object sender,
            System.EventArgs e)
{
IComparator objCompare;

// Create
            the correct comparator instance
if
            (rbOrderDate.Checked)
objCompare = new
            OrderDateComparator();
else if
            (rbOrderNumber.Checked)
objCompare = new
            OrderNumberComparator();
else
objCompare = new
            CustomerNameComparator();

QSortAlgorithm QSort = new
            QSortAlgorithm();

// Then call the QuickSort method and pass
            in
// the comparator.
QSort.QuickSort(m_Orders,m_Orders.GetLowerBound(0),
m_Orders.GetUpperBound(0),objCompare);

fillListBox();
}

The code will first determine which of the criteria (radio buttons) were selected by the user. In the case that no criteria was specified, the code will default to using the CustomerName criteria. It is important to note here that the objCompare is declared as an IComparator object and not as a specific comparator. The QuickSort method requires an IComparator type object, and will give a compile time error if a OrderNumberComparator, CustomerNameComparator, or OrderDateComparator is provided. However, since each of those comparators implements the required methods, the IComparator object can be set to an instance of any of those types.

Once the comparator is created, it is then used in the Quick Sort method. This method will directly affect the m_Orders array, and will not have a return value. Once the method is completed, the m_Orders array will be correctly sorted by the selected criteria. The fillListBox() method performs the task of clearing the listbox and then re-populating it, in order, from the items in the array. At that point, the dialog will then reflect the newly sorted array of items. If we had clicked on Order Number and then Sort, the application would show this:

Instead of the orders being randomly ordered, they are now in ascending order by their Order Numbers.

Summary

As has been illustrated by this application, the use of comparator objects will greatly simplify sorting an array of objects based on any criteria. The QSortAlgorithm class, with its Quick Sort method, can then be re-used in any C# application. The majority of the sorting work is completed and a developer need only implement a new comparator class and immediately use that class to perform a sort with different criteria. It is also very powerful in that the developer could implement a comparator to actually compare multiple properties of the object. Using the orderInfo example, it is possible that an application would need to be able to sort on both Order Number and Order Date at the same time. A new comparator class could easily perform this action, and this point exactly illustrates how powerful and useful comparators can be with any sorting algorithm, though in this case we used Quick Sort.

If you'd like to discuss a broader range of C#Today topics and issues, or if you have any C# related questions you need answers to, please visit our online resource centres at: http://p2p.wrox.com/list.asp?list=csharptoday_discuss&WROXEMPTOKEN=620568Zhiox25QOrrsnTPfm53p and http://p2p.wrox.com/list.asp?list=c_sharp&WROXEMPTOKEN=620568Zhiox25QOrrsnTPfm53p


Programmatically Managing Windows Services & Device Drivers with C#

Programmatically Managing Windows Services & Device Drivers with C#
http://www.csharptoday.com/content.asp?id=1587
Article Rating: 3.5
Published on October 11, 2001
Abstract
An impressive feature of the .NET framework is the support and management features that .NET offers for Windows Services. The .NET framework provides an extensive set of class libraries for managing Windows Services, which go beyond the Services MMC snap–in in Windows 2000 and allow us to manage low level device drivers programmatically, or start and stop multiple services at the same time. In this article, Jim Joy walks us through the anatomy of a windows service and build a “Service Device Manager”, a Win32 application giving us the ability to manage the services and low–level device drivers that are running on your Windows machine.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

An impressive feature of the .NET framework is the support and management features that .NET offers for Windows Services. The .NET framework provides an extensively exhaustive set of class libraries for managing Windows Services, that goes beyond the Services MMC snap-in in Windows 2000, and allows us to manage low level device drivers programmatically, or start and stop multiple services at the same time. In this article, we'll walk through the anatomy of a windows service and build a "Service Device Manager", a Win32 application giving us the ability to manage the services and low-level device drivers that are running on your Windows machine.

The "Service & Device Manager" Application

The "Service & Device" application is a Win32 application that has been developed in C# using the beta 2.0 version of the .NET framework. This application uses the System.Windows.Forms namespace for its user interface and the System.ServiceProcess namespace for its core functionality.

Before we dive into the details of developing our "Service & Device Manager", lets take a look at what a windows service is. If you are already familiar with windows services, you will want to skip this and go straight to the Building the "Service & Device Manager" Application section.

Windows Services & the .NET Framework

Windows services provide us with the ability to create long-running executable applications that manage and run in their own Windows sessions. They are highly configurable entities - they can be automatically started when the Windows machine starts up, can be paused and resumed, and do not necessarily have any user interface. These characteristics make services the best suited for use on a server or when you need uninterrupted, long-running functionality that does not interfere with other users who may be working on the same machine. Services can also be run based on the credentials of the logged in user - the security context of the logged in user can be different from that of another user.

The .NET framework makes it extremely easy to create Windows services - we simply create an application that is installed as a service. A classic example is of a disk monitoring service. You may want to monitor all write activities that take place on the Master Boot Record (MBR) of your hard drive - you could write a windows service application that continuously monitors the MBR of your hard drive for any write activities, and then take preventive actions accordingly. Classic examples of existing Windows Services include the MSFTPSVC service, which provides FTP publishing capabilities, or the SMTPSVC service, which provides e-mail capabilities using the Simple Mail Transfer Protocol (SMTP).

Windows Service applications run in their own security context and can be started along with the initialization of Windows, which means that before the user logs into the system, the services installed in the system are started.

The .NET framework comes with a command line utility for installing services. Once we create and build an application, it can be installed as a service by running the InstallUtil.exe utility at the command prompt and passing the path of your executable file. Please remember that a service application is different from an ordinary Win32 application - we cannot start the service by simply running the executable file after we build it, it has to be installed as a service. Similarly, if we want to debug your service, we must install our service and then attach a debugger to it.

Points to Note

While writing service applications, we need to take care of a few requirements.

  • Every service application should contain a Main method, which will serve as the entry point of the service.
  • The Main method for our service application must, in turn, issue a Run command for the service that our application is intended to implement. The Run method is the one that actually loads the service into the Services Control Manager.
  • Since service applications run in a different windows session than other applications, any user interface components that are generated by a service application will not be visible to the logged in user and this can cause your service to pause indefinitely. Care should be taken to avoid such circumstances. A typical example of such a mistake is including "Message Boxes" in a service application to notify users about errors. If we do that, the message box will be triggered by the service application and since the user will never be able to see it, the program execution will be suspended indefinitely. Ideally, we should log the errors raised by a service in the Windows Event Log rather than raise them as message boxes or any other form of user interface.

Windows Service Lifespan

There are several stages in the lifespan of a service - I'm sure that most of you will be aware of them.

  • The first one is the installation stage where the service is installed in the system where it will be run. This is the process that loads the service into the Services Control Manager of the running computer.
  • Once the service is installed, the next step is to start the service. This is done manually by using the Services Control Manager. With the .NET framework, we are able to manage tasks like these (Start, Stop, etc) programmatically - the ServiceController class defined in the System.ServiceProcess namespace gives us this ability. The ServiceController class can be used to bind to an existing windows service, and we can run the service programmatically by calling the Start method of the bound ServiceController object. Once we call the Start method, program execution gets passed to the OnStart method of our service application.
  • Once successfully started, the service is said to be in its running stage. From the running stage, the service can move into different stages like Paused, Stopped, etc. The service can also have short intermediate stages like
    • Start Pending, which means that a start operation has been issued on the service and it has not been completed,
    • Continue Pending which means a continue command was issued on a paused service and is not completed,
    • Pause Pending, which means that a pause command was issued to a running service and is not completed, or
    • Stop Pending which means that a stop command was issued to a service and it was not completed. The status of a service can be obtained by querying its Status property (ServiceController.Status).

Any of these above mentioned actions could be performed on a service either manually, using the Services Controller Manager, or programmatically using associated methods like OnStop, OnPause, OnContinue, etc.

In this article, we will walk through the construction of a "Service and Device Driver Manager" application. This application will allow users to manage Windows Services and low-level device drivers that are already installed on a Windows PC. We will not cover the details of creating Windows Services in this article, although you can read more at Rob Schieber's .NET Services article.

Building the "Service and Device Manager" Application

The "Service & Device Driver Manager" uses the Windows Forms library to implement its user interface, and the ServiceProcess library to implement its functionality. The application consists of two forms, which are:

  • ServiceManager.cs - This form gives the user a list of the different services that are installed on their Windows 2000 system. This form also provides the user with service management capabilities like starting, stopping and pausing services. It also has a toggle feature, which allows the user to toggle between a list of installed services and a list of installed device drivers.
  • ServiceProperties.cs - This form lists the different properties associated with a service / device driver, such as the status, type, display name, etc. This form also gives the user a list of dependencies - the services / drivers that depend on the selected entity, as well as the services / drivers the selected entity depends upon.

Now, let's take a deep look into the user interface and functionalities implemented by these forms.

ServiceManager.cs

Take a look at the following screen shot, that shows the main screen of our application, which is implemented by the ServiceManager.cs file.

Main form of the Service and Device Manager application, which displays the list of services / device drivers installed in the system.

The ServiceManager.cs source designs and implements the main screen of our application - the one that displays the list of Windows Services / device drivers available in the underlying system, as well as well as providing the user with options to perform tasks on these services like Start, Stop, Pause, etc. This file defines one Form named ServiceManager, which is inherited from the System.Windows.Forms.Form class. The form also provides the user with buttons for starting, stopping and pausing services, as well as to resume paused services. It also houses a list view that lists the services / device drivers installed in the system depending upon the option we choose from the toggle button provided.

using System.Windows.Forms
{
class ServiceManager : Form
{
public ServiceManager()
{
.
}
}
}

The Main() method calls the static Run() method of the Application class, which is used to start and stop applications. The Run() method is called to start an application message loop, and it also creates an instance of the ServiceManager class and makes it visible on the screen.

Public static void Main()
{
Application.Run(new ServiceManager());
}

The ServiceManager class also houses a ListView object derived from the System.Windows.Forms.ListView class, and seven Button objects derived from the System.Windows.Forms.Button class. Let's look at the way in which the Buttons and Column Headers are created and used in this form. The simplest way to create and use a control on a form is to simply instantiate the class and set its different properties to suite your choice. But, if there are a number of controls which should have similar properties (for example, it is a general practice to have the same size and fonts for all buttons in a form), doing it this way can lead to a lot of unnecessary code. The best way out is to use inheritance - derive your own class, set the common properties, and then instantiate the class - that's exactly what is done here. For example:

Class UtilButton:Button

{
public UtilButton(String buttonText ,
            Size buttonSize , Point buttonPoint)
{
this.Text =
            buttonText;
this.Size = buttonSize;
this.Location =
            buttonPoint;
}
}

Here, the UtilButton class is derived from the System.Windows.Forms.Button class and has a constructor, which sets the Text, Size and Location properties of the button. Now creating the button is as simple as instantiating UtilButton.

private UtilButton
            StatusButton;
StatusButton = new UtilButton(""Device Drivers" ,
            new Size(88,32) , new Point(12,325));
StatusButton.Click += new
            EventHandler(OnChangeDisplay);
this.Controls.Add(StatusButton);

The above code creates the Status button and adds it to the Controls collection of the ServiceManager form so that it becomes a part of the form. A similar approach is followed to create the Column Headers for the List View too - the ServiceHeader class derived from the ColumnHeader class has a constructor that sets the Text, Width and TextAlign properties of the Column Header.

The Event Handlers for the StartButton are defined as follows.

//Binds the OnStartService method to the Click event of the button.
StartButton.Click += new EventHandler(OnStartService);

The OnStartService method, which is the event handler, is defined as follows

protected void OnStartService(Object mySender , EventArgs myArgs)
{
.
}

Clicking on the StartButton button fires up the OnStartService method, which performs the necessary steps for starting the selected service.

Basics apart, let's dive into the details of service management using the .NET framework. The "Service and Device Manager" application initially lists the services that are installed in the underlying Windows environment. It also provides capabilities to Start, Stop and Pause services, as well as resume paused services. All this and more is accomplished by the ServiceController class defined in the System.ServiceProcess namespace.

The ServiceController class provides the programmer with a means of connecting to existing windows services and low-level device drivers, allowing us to control their behavior and extract information about them. Using the ServiceController class, we can connect to an existing service or device driver by setting the ServiceName and MachineName properties of the ServiceController class. The machine name is set to the local machine by default, so we need not specify this unless we want to connect a service that runs on a different computer. We can then use the class to start, stop, and otherwise manipulate the service. The ServiceController class is generally used in an administrative scenario - for example, we may want to create an application that is capable of sending custom commands (like Start, Stop, etc) to a Windows Service. Since the Services snap-in MMC does not support custom commands, the easy way for a developer to accomplish these tasks is to instantiate the ServiceController class, bind it to the desired service, and programmatically issue the custom commands. However, care should be taken when we issue custom commands to a service programmatically - if we try to stop a service that cannot be stopped, an exception occurs (in the Services snap-in, the Stop button will be disabled for a service that cannot be stopped).

The GetServices method of the ServiceController class returns a list of services (except device driver services) installed on a specific computer (the default is the local machine, so you can safely ignore the Machine Name). The GetServices() method returns an array of ServiceController objects, each of which represents an individual service running on the machine.

ServiceController[]
            ServiceArray;
ServiceArray =
            ServiceController.GetServices();
foreach(ServiceController
            IndService in ServiceArray)
{
ListViewItem newService = new
            ListViewItem(IndService.ServiceName);
newService.SubItems.Add(IndService.DisplayName);
newService.SubItems.Add(IndService.Status.ToString());
if(IndService.MachineName
            ==
            ".")
{
newService.SubItems.Add("Local");
}
else
{
newService.SubItems.Add(IndService.MachineName);
}
IndService.Close();
lvwServices.Items.Add(newService);
}

The foreach loop is used to iterate through the array of ServiceController objects returned by the GetServices() method, and each individual service, represented by the IndService object, is then added into the List View.

Let's take a look at the different properties that the ServiceController object exposes.

  • The ServiceName property gives us the name of the executable for the service application while the
  • DisplayName property gives a description of the service.

For example, if we take into consideration the FTP Service, then the ServiceName property will return MSFTPSVC, the name of the service executable, and the DisplayName property will return FTP Publishing Service, a description of the service.

The status of the service is obtained by reading the Status property. This returns the current state (status) of the service, and its value will be one among the different values of the System.ServiceProcess.ServiceControllerStatus enumeration. These values are:

  • ContinuePending - this status value indicates that a "Continue" action has been initiated on a paused service, and that the action is not yet completed.
  • Paused - this status value indicates that the service is now in the "Paused" state.
  • PausePending - this status value indicates that a "Pause" action has been initiated on a service, and that the action is not yet completed.
  • Running - this status value indicates that the service is now in the "Running" state.
  • Stopped - this status value indicates the service is in the "Stopped" state.
  • StopPending - this status value indicates that a "Stop" action has been initiated on a service, and that the action is not yet completed.

The "Device Drivers" button provided in the ServiceManager form allows the user to toggle the display between services and device drivers. This is accomplished by calling the GetDevices() method of the ServiceController class - this call is controlled by a Boolean variable named _SrvsDrvs - if it's true, then the Services are listed, if false, the device drivers are listed.

The Start, Stop, Pause and Continue operations work more or less in a similar fashion - first, a ServiceController object is created and bound to the service on which the action is to be performed, and then the method corresponding to the desired action is called. For example, to start a selected service which is stopped:

protected void OnStartService(Object
            mySender , EventArgs myArgs)
{
ListViewItem myItem =
            lvwServices.FocusedItem;
ServiceController myIndService = new
            ServiceController(myItem.Text);
try
{
if(myIndService.Status
            != ServiceControllerStatus.Stopped)
{
//service is already in
            a state from which it cannot be started (running, paused
            etc)
}
else
{
myIndService.Start();
ServiceController
            myTempSrvc;
ServiceControllerStatus TmpStatus;
do
{
myTempSrvc = new
            ServiceController(myIndService.ServiceName);
TmpStatus =
            myTempSrvc.Status;
myTempSrvc.Close(); }
while(TmpStatus !=
            ServiceControllerStatus.Running);
}
}
catch(Exception
            myExcp)
{
//Errors
            occurred
}

finally
{
myIndService.Close();
LoadServicesOrDrivers();
}
}

An object of the ServiceController class named myIndService is created and is bound to the selected service, whose name is obtained from the myItem ListViewItem object. Once the myIndService object is bound to the desired service, the current status of the service is queried and the Start method is called which will start the service. When we call the Start() method, the status of the service is first changed to StartPending, and when the Start() method returns, it is changed to Running. This service can be stopped / paused only when the status of the service has become Running.

Now let us take a look at the need for a temporary ServiceController object named myTempSrvc in the above code sample. As mentioned above, once a start command is issued against a service, the status of the service is first changed to StartPending. Before the service can be actually started, it may require that the services that it depends on (e.g. the SMTP Service, which requires the IIS Admin service to be running for it to start) should be started first. This can cause a delay in the starting of our service. However, the Start() method returns as soon as the status of the service is changed to StartPending, so if we reload the List view immediately after calling the Start() method, we may see the status of the service as StartPending, and not Running. The best way to avoid this problem is to poll the status of the service until it has become Running, and then call the LoadServicesOrDrivers() method, which will reload the List view. For this purpose, we use the temporary ServiceController object. The myTempSrvc object is bound to the service continuously (in a loop) and its status is checked until the status returns Running. Once the status becomes Running, execution jumps out of the loop and the List view is reloaded.

The try, catch, and finally blocks are used to trap any exceptions that might occur during the processing. The catch block catches any exception that is raised in the try block and the finally block is used to execute any cleanup action. Note that the finally block is executed in any case - whether or not an exception is raised. So, the finally block becomes a good target for placing cleanup code. In our example, the myIndService object is closed, thereby freeing all resources used by the object in the finally block.

All the other actions on the installed services are performed in a similar way - stopping a service is accomplished by calling the Stop() method, pausing is done with the Pause() method, and resuming a paused service is done by calling the Continue() method.

Now let's look at the ServiceProperties form, which lists the different properties associated with a service / device driver and dependencies.

ServiceProperties.cs

ServiceProperties.cs defines the ServiceProperties form that displays the different properties associated with a selected service / device driver. Take a look at the screenshot below:

The Service Properties form that shows the properties of a selected service / device driver as well as it's dependencies.

This form houses three list view objects - one for listing out the properties of the service /device driver, one for displaying the services that depend on the selected service, and one for displaying the list of services that the selected service depends on. Since these three list view objects share a common set of properties (like width, height, etc), they are instantiated from the SrvListView class, which is in turn inherited from the System.Windows.Forms.ListView class. Please see the code snippet given below:

// SrvListView inherits from the System.Windows.Forms.ListView class
class SrvListView : ListView
{
//Constructor for initializing selected properties
public SrvListView(Point myPoint, Size mySize, int myTabIndex)
{
this.Location = myPoint;
this.Size = mySize;
..
this.View = View.Details;
this.AllowColumnReorder = true;
..
this.TabIndex = myTabIndex;
}
}
.
private SrvListView lvwProperties //decalring the list view objects;
private SrvListView lvwDependents;
private SrvListView lvwMasters;
.
// Instantiating the SrvListView objects
lvwProperties = new SrvListView(new Point(20,20) , new Size(490,120) , 0);

The first list view object, lvwProperties, displays the various properties associated with a selected service which describe the different characteristics of the service - whether the service can be paused or stopped, the current status of the service, the type of the service, etc.

The ServiceProperties class has one constructor that takes in the name of the service that is passed from the ServiceManager form. The ServiceProperties class uses a ServiceController object named myService to bind to this service, and then uses the different properties that are exposed by the ServiceController class. These properties include:

ServiceName - gives the name o the executable service.

  • DisplayName - gives the display name (description) of the service.
  • MachineName - gives the name of the machine where the service is running.
  • Status - gives the current status of the service. This status will be one among the different values for the ServiceControllerStatus enumerator.
  • ServiceType - gives the type of the service, which will be one of the different values for the ServiceType enumerator.
  • CanPauseAndContinue - indicates whether the service can be paused and resumed.
  • CanStop - indicates whether the service can be stopped.
  • CanShutDown - indicates whether the service needs to be notified if the system is shutting down.

Please note the usage of these properties of a Windows Service in the code snippet given below:

class ServiceProperties
{
private
            ServiceController myService;
public ServiceProperties(String
            ServiceName)
{
//Binding to a service
myService = new
            ServiceController(ServiceName);
.

//Using the CanStop
            property
if(myService.CanStop == true)
{
//This means that
            the service can be stopped
}
else
{
//This means that
            the service cannot be stopped

//Using the
            CanPauseAndContinue property
if(myService.CanPauseAndContinue ==
            true)
{
//This means that the service can be paused and
            resumed
}
else
{
//This means that the service cannot be
            paused and resumed
}

//Using the CanShutDown
            property
if(myService.CanShutDown == true)
{
//This means
            that the service should be notified in the event of a system shut
            down
}
else
{
//This means that the service need not be
            notified in the event of a system shut
            down
}
}
}
}

Note that properties like CanStop, CanPauseAndContinue, etc, will reflect the current status of the service - for example, for a stopped service, the CanStop properties will return false which means that the service cannot be stopped at the given point, but when the service is running, the same property may return true depending on the type of service.

The ServiceType property returns the type of the service, and its value belongs to the ServiceType enumeration defined in the System.ServiceProcess namespace. Windows services and device drivers can be of the following types:

  • Adapter - represents a service for a hardware device that requires its own driver.
  • FileSystemDriver - A File System Driver, which is also a Kernel Driver like the NTFS driver, FAT driver, etc.
  • InteractiveProcess - a service that can interact with the Windows desktop.
  • KernelDriver - a kernel device driver such as a hard disk driver.
  • RecognizerDriver - A file system driver used during the Windows startup process to determine the different file systems present in the system (NTFS, FAT, etc).
  • Win32OwnProcess - a service that runs as an independent Win32 process without sharing processes with other processes. A good example is the MSDTC (Distributed Transaction Coordinator) service.
  • Win32ShareProcess - a service that can share processes with other Win32 services. A good example is the IIS Admin service.

Among these service types, Win32OwnProcess, Win32ShareProcess and InteractiveProcess represent types of services, while the other enumerations represent device drivers. Also, if you are creating services using the .NET framework, you'll be able to create only services of the types Win32OwnProcess and Win32ShareProcess.

The ServiceProperties form also displays the dependencies of the selected service. These dependencies include the services that depend on the selected service to run, and the services that the selected service depends on to run.

The DependentServices property of the ServiceController class returns an array of Services (as ServiceController objects) that are dependant on the selected service. As with the GetServices() method, we use a foreach loop for iterating through this array and extract each individual service.

private ServiceController[] depServiceArray;
depServiceArray = myService.DependentServices;
foreach(ServiceController depService in depServiceArray)
{
//Add dependent service to the list view
depService.Close();
}

Similarly, the ServicesDependedOn property of the ServiceController class returns an array of services that the selected service depends upon for running:

depServiceArray = myService.ServicesDependedOn;
foreach(ServiceController depService in depServiceArray)
{
//Add dependent service to the list view
depService.Close();
}

The ServiceProperties form displays the similar details for a selected device driver also. The ServiceController class treats a Windows Service as well as a device driver in pretty much the same way - both are treated as services of different types (as explained in the ServiceType enumeration)

Running the Application

The "Service and Device Manager" was developed using the beta 2.0 version of the .NET Framework. Unzip the 2 files ServiceManager.cs and ServiceProperties.cs into a directory, and compile the application from the command line:

C:\your-directory>csc/out:ServiceManager.exe ServiceManager.cs ServiceProperties.cs

Enhancements to the Application

There are a number of areas in which we can enhance the application:

  • We can build and incorporate the capability of issuing custom commands to a selected service.
  • An option of starting, stopping, pausing and resuming a group of services can also be added to the application (by allowing the user to select multiple services from the main form, and then giving the Start, Stop, Pause commands).
  • We could write to the Event log and record the activities that were performed on a service (e.g. - how many times a given service was stopped, the user, the date/time, etc).
  • As far as modifications to the user interface, a sort option for all the list views could be provided.

Conclusion

In this article we've covered the various capabilities offered by the .NET framework for administering existing Windows services and device drivers. The System.ServiceProcess namespace provides the developer with a set of rich classes that makes the tasks of writing and installing new services, as well as managing existing services, easier than ever before.

If you'd like to discuss a broader range of C#Today topics and issues, or if you have any C# related questions you need answers to, please visit our online resource centres at : http://p2p.wrox.com/list.asp?list=csharptoday_discuss&WROXEMPTOKEN=620568Zhiox25QOrrsnTPfm53p and http://p2p.wrox.com/list.asp?list=c_sharp&WROXEMPTOKEN=620568Zhiox25QOrrsnTPfm53p


Visual Studio .NET Add-Ins Part 2

Visual Studio .NET Add-Ins Part 2
http://www.csharptoday.com/content.asp?id=1577
Article Rating: 4.5
Published on September 19, 2001
Abstract
In this first part of his article on Visual Studio .NET Add-Ins, Kaushal Sanghavi looked at the support that Visual Studio .NET provides for writing add-ins and macros that can extend and customize the development environment. In this second part, he further demonstrates the use of VS .NET add-Ins by building two custom add-ins: one that adds an AutoSave feature to the current project by saving the open files at a specific interval, and another one that synchronizes the VS .NET task list with that of Microsoft Outlook.
Article

Introduction

In the first part of our article on Visual Studio .NET Add-Ins, we discussed the support that Visual Studio .NET provides for writing add-ins and macros that can extend and customize the development environment. We looked at the Add-in Wizard that Visual Studio .NET provides, and we covered the Visual Studio .NET automation model, as well as the anatomy of a very simple add-in. The add-in we covered in Part 1 of this article was a very simple one that used the System.Web.Mail namespace to send the currently open document via email. In this article, we will further demonstrate the use of Visual Studio .NET Add-Ins by building two custom add-ins; one that adds an AutoSave feature to the current project by saving the open files at a specific interval, and another one that synchronizes the Visual Studio .NET task list with that of Microsoft Outlook.

This article is based on the Beta 2 release of Visual Studio .NET, and the attached sample code requires Visual Studio .NET Beta 2, Windows 2000 and Microsoft Outlook to run.

Visual Studio .NET Add-Ins Recap

Before we start discussing the custom add-ins, let us provide a brief recap of Visual Studio .NET Add-ins. For further details, please refer to Part 1 of this article.

Visual Studio .NET add-ins provide a way to extend the development environment. These add-ins are standard .NET components that integrate seamlessly with the Visual Studio .NET development environment, and have access to all the objects and features that the development environment offers. These objects are known as the Visual Studio .NET Automation Model , and provide access to all the built-in objects such as projects, open files, solutions, tasks, build configurations, the debugger, etc. An add-in can access the entire feature set of the development environment by accessing these objects and invoking their methods and properties. This leads to a very powerful framework for completely customizing and extending the environment to meet the needs of individual developers.

Writing a Visual Studio .NET add-in is as simple as invoking the Add-in Wizard that Visual Studio .NET provides. The Wizard currently supports C#, VB .NET or C++ as the development language. The Wizard generates all the method stubs and the skeleton code that is necessary for interfacing with the development environment. All you need to do is write the application logic within these stubs to develop the add-in. These add-ins are managed components that implement the IDTExtensibility2 and the IDTCommandTarget interfaces. The IDTExtensibility2 interface provides methods that fire when the add-in is loaded, unloaded, installed and uninstalled, and the IDTCommandTarget interface provides methods that are executed when the add-in is invoked.

Visual Studio .NET add-ins can be loaded or unloaded using the Add-In Manager from the Tools menu of the IDE.

AutoSave Visual Studio .NET Add-In

A lot of Microsoft products such as Word and Excel provide an Auto Save feature that saves the open document automatically, after a specified interval. This is a great feature for users who are used to typing furiously without a break for long periods of time. The last thing you want is for your system to crash, causing you to lose all your work. Visual Studio however lacks such an Auto Save feature, and in this section we will build a custom Visual Studio .NET add-in that offers this feature for Visual Studio .NET developers. This add-in will take in a user defined time interval and store it in the registry, so that this interval is persisted across different sessions of the Visual Studio .NET add-in. The add-in will spawn a thread that wakes up after the specified time interval, and saves all the open documents and the projects in the solution. Further, the add-in supports turning itself off, useful if the user wants to disable the auto save feature.

We use the Visual Studio .NET Add-in template from within the Extensibility Projects under Visual Studio .NET, which in turn launches the Add-in Wizard, to generate the skeleton code for our add-in. We will not cover the details of creating an add-in, or the steps in the Add-in Wizard in this article; please refer to Part 1 of this article for details on the process of creating an add-in.

Before we take a look at the code behind this add-in, let us describe the design of this add-in. In order for an add-in to automatically save the projects and files periodically after a specific interval, it needs to persist for this period, and it needs a method that will be called after this period has expired. We have chosen to store the time period value in the registry, as a DWORD value, under the add-in key itself. If this value is not found, the add-in creates one with a default time interval of 300 seconds. Further, to ensure that the add-in can perform the save periodically, we will use a thread that will sleep for the specified period, wake up, do its job of saving the projects and files, then go back to sleep. This will continue until the user either unloads the add-in, or turns off the auto save feature. It is worth noting that we had a choice of using a timer for this purpose, but we chose to use a thread instead. This is because it leads to a much cleaner design, and allows for easy coding and management of the thread.

Now that we have explained, in brief, the basic design behind this add-in, let us take a look at some code. We will start by taking a look at the Exec method, which is called when the user invokes the add-in. Note that since we don't have any custom code that runs when the add-in is loaded, we have not added any code to the OnConnection method.

The code for the Exec method is shown below.

public void Exec(string commandName,
EnvDTE.Visual StudioCommandExecOption executeOption,
ref object varIn, ref object varOut, ref bool handled)
{
handled = false;
if(executeOption == EnvDTE.Visual StudioCommandExecOption.Visual StudioCommandExecOptionDoDefault)
{
if(commandName == "AutoSaveAddin.Connect.AutoSaveAddin")
{
RegistryKey lLocalMachine = null;
RegistryKey lKey = null;
try{
lLocalMachine = Registry.LocalMachine;
lKey = lLocalMachine.OpenSubKey
(@"SOFTWARE\Microsoft\VisualStudio\7.0\AddIns
AutoSaveAddin.Connect", true);

mTimeInterval =
(int)lKey.GetValue("AutoSavePeriod");
}
catch (System.NullReferenceException){
if (lKey != null)
lKey.SetValue ("AutoSavePeriod", mTimeInterval);
}

mDlgBox = new AutoSaveAddinDialog (mTimeInterval, applicationObject);
mDlgBox.Show ();

handled = true;
return;
}
}
}

The first thing we need to do is to obtain the value of the auto save period from the registry. We use the RegistryKey object from the Microsoft.Win32 namespace to access the registry. We look for the AutoSavePeriod value under the SOFTWARE\Microsoft\VisualStudio\7.0\AddIns\AutoSaveAddin.Connect key. This is the key under which the other add-in properties are stored. Since the AutoSavePeriod is a custom property, this property will not be present in the registry the very first time the add-in is run. Thus we check to see if the AutoSavePeriod value exists under the add-in key, and if it is not found, we create the property with a default auto save period of 5 minutes. Once the registry manipulation is complete, we can pass the value to the form. The form will accept user input as to whether to turn on the auto save feature, and also what the required auto save period should be. This form is shown in the following screen shot.

As shown in the above code segment, the Exec method of the add-in creates an instance of this form, then passes it the default time interval read from the registry. Since this AutoSave form will require access to the open documents and projects in the IDE, we also pass the DTE object to this form. This DTE object is the root object of the Visual Studio .NET Automation Model, and all the objects in the development environment can be accessed directly or indirectly from the DTE object. Further, this DTE object is cached throughout the lifetime of the add-in, and thus can be safely passed to the AutoSave form, and can be accessed by it. Furthermore, since the auto save interval can be changed many times during the life of the add-in, we cannot cache the value read from the registry, and therefore have to access the registry for every invocation of the add-in.

Let us now take a look at the code behind the AutoSave form shown in the above screen shot. We will start by taking a look at the constructor for this form.

public AutoSaveAddinDialog (int iTimeInterval, _DTE iDTE)
{
InitializeComponent();

mDTE = iDTE;
mTimeInterval = iTimeInterval;
txtInterval.Text = mTimeInterval.ToString ();

if (mThread == null)
mThread = new System.Threading.Thread (new ThreadStart (this.ThreadProc));
}

The constructor is fairly simple. We store the time interval and the DTE object passed by the Exec method in private data members, and we display the passed time interval on our form. Further, since we will be using a thread to periodically wake up and save the projects and documents, we create a thread and assign it to the mThread member variable. A key point to note here is that the same instance of the add-in can be invoked multiple times, and every time the add-in is invoked, the Exec method is called, which in turn instantiates the AutoSave form and displays it. However, irrespective of the number of times the add-in is invoked, we need a single thread to perform the auto save. Thus, if each instance of the form created its own thread, then we would have multiple threads running, each trying to save the documents, leading to total chaos.

In order to ensure that at any given point of time, a single thread at most is active, we make the mThread data member a static data member. This ensures that a single instance of this data member will be created, irrespective of the number of instances of the form that are created, and that all instances can access the thread object.

private static System.Threading.Thread mThread = null;

Since the mThread data member is static, we check to see if the value of this variable is null in the constructor. If it is, we create a new instance of a thread object. Let us now take a look at the code that executes when the user clicks on the OK button of the AutoSave form.

private void btnOK_Click(object sender, System.EventArgs e)
{
if (chkAutoSave.Checked == true){
mTimeInterval = Int32.Parse (txtInterval.Text);

RegistryKey lLocalMachine = Registry.LocalMachine;
RegistryKey lKey = lLocalMachine.OpenSubKey
("SOFTWARE\\Microsoft\\VisualStudio\\7.0\\AddIns\\AutoSaveAddin.Connect", true);

if (lKey != null)
lKey.SetValue ("AutoSavePeriod", mTimeInterval);

if (mThread.ThreadState == ThreadState.Unstarted)
mThread.Start ();
}

if (chkAutoSave.Checked == false)
StopThread ();

this.Close ();
}

The first thing we need to do if the user has enabled the auto save, is to read the auto save interval from the form and save it back to the registry, so that future invocations of the add-in can access this new value. We use the RegistryKey objects from the Microsoft.Win32 namespace to access the registry and persist the value. Once this is persisted, we check the state of the auto save thread. If the thread has not started (this will be true the very first time the user invokes the add-in), we start the thread and close the form. On the other hand, if the user has disabled the auto save feature, we need to kill the thread, as we do not need the thread to wake up periodically to save the documents. The StopThread method is shown below.

public void StopThread
            ()
{
mThread.Abort ();
mThread = null;
}

Let us finally look at the ThreadProc method, which is the function that the auto save thread executes that performs the saving of the documents and projects.

private void ThreadProc ()
{
while
            (true){
// convert the interval into
            milliseconds
System.Threading.Thread.Sleep (mTimeInterval *
            1000);

mDTE.Documents.SaveAll ();

if (mDTE.Solution ==
            null)
continue;

Projects lProjects =
            mDTE.Solution.Projects;
foreach (Project lProject in
            lProjects)
lProject.Save
            (lProject.FullName);
}
}

As you can see from the above code segment, the ThreadProc method consists of a single infinite loop. The reason for the infinite loop is that the auto save thread will continue to run forever until the add-in is unloaded, or until the auto save is disabled. In both these cases we simply kill the thread by calling the StopThread method. In the ThreadProc method, we use the mDTE object which is passed to the form by the Exec method, in order to call the SaveAll method on the Documents collection. This saves all the open documents in the editor. Finally, we use the Projects collection to iterate through all the projects in the solution and save each one.

In order to ensure that the auto save thread is killed when the add-in is unloaded (via the Add-In Manager), we call the StopThread method from the OnDisconnection method of the add-in object as shown below.

public void OnDisconnection(ext_DisconnectMode disconnectMode, ref object[] custom)
{
if (mDlgBox != null)
mDlgBox.StopThread ();
}

As you can see from this discussion, adding an auto save feature is fairly simple, and the actual save method is about five lines of code!

Outlook Sync Visual Studio .NET Add-In

This add-in will synchronize the contents of the Task List from Visual Studio .NET with the contents of Microsoft Outlook's Tasks. This is not a true synchronization, since we merely update the contents of Outlook, and do not delete any tasks from Outlook even if they are deleted from Visual Studio .NET. This is to ensure that no tasks from Outlook are deleted accidentally. While synchronizing, we create a separate category under Outlook called Visual Studio.NET Tasks - OutlookSyncAddin , and add all the tasks under this category. If any of the tasks are updated, they get updated in Outlook as well, and if the task is marked as complete, the corresponding Outlook task is marked complete too.

This add-in was developed using the Outlook objects ( MSOUTL.OLB ), and though this has been tested using Outlook XP, this should work with all recent versions of Outlook, including Outlook 2000.

Similar to the auto save add-in, this add-in also creates a form that is used to trigger the sync process, and is invoked by the Exec method of the add-in. The method passes the root DTE object, which is required to access Visual Studio .NET's task list. The Exec method is shown below.

public void Exec(string commandName, EnvDTE.Visual StudioCommandExecOption
executeOption, ref object varIn, ref object varOut, ref bool handled)
{
handled = false;
if(executeOption == EnvDTE.Visual StudioCommandExecOption.Visual StudioCommandExecOptionDoDefault)
{
if(commandName == "OutlookTasksAddin.Connect.OutlookTasksAddin")
{
dlgOutlookTasks lDlg = new dlgOutlookTasks(applicationObject);
lDlg.Show ();

handled = true;
return;
}
}
}

Again, similar to the auto save add-in, we use static member objects ( mOutlookApp, mNamespace ) to access the Outlook objects. This is done so that we don't create duplicate objects for every invocation of the add-in. The constructor is shown below.

public dlgOutlookTasks(_DTE
            iDTE)
{
InitializeComponent();

mDTE = iDTE;

if
            (mOutlookApp == null)
mOutlookApp = new Outlook.Application
            ();
if (mNamespace == null)
mNamespace =
            mOutlookApp.GetNamespace ("MAPI");
}

Let us now take a look at the methods for retrieving the tasks lists from Outlook and from Visual Studio .NET.

private Outlook._Items GetOutlookTasks ()
{
_Items lAllTasks;

lAllTasks = mNamespace.GetDefaultFolder
(OlDefaultFolders.olFolderTasks).Items;

string lFilter = "[Categories] = '" + mCustomCategory +"'";
_Items lTasks = lAllTasks.Restrict (lFilter);

return lTasks;
}

private TaskItems GetVisual StudioTasks ()
{
TaskList lList = (TaskList)mDTE.Windows.Item
(Constants.Visual StudioWindowKindTaskList).Object;
TaskItems lTasks = lList.TaskItems;

return lTasks;
}

We use the Outlook Namespace object to get the Tasks folder, and since we are only interested in the contents of our custom category, we filter the results by our category. Similarly, we use the DTE methods to obtain the Visual Studio .NET TaskList object, and we retrieve the TaskItems collection from the TaskList object.

Once both these collections are obtained, we use a sync algorithm to update the contents of Outlook's tasks as shown below.

private void btnSync_Click(object sender, System.EventArgs e)
{
Outlook._Items lOutlookTasks = GetOutlookTasks ();
TaskItems lVisual StudioTasks = GetVisual StudioTasks ();
_Items lExistingTasks;
_TaskItem lExistingTask;

if (chkLog.Checked)
lstLog.Items.Clear ();

string lFilter;
string lSubject;
foreach (EnvDTE.TaskItem lVisual StudioTask in lVisual StudioTasks)
{
try
{
lSubject = lVisual StudioTask.Description + ":" +
lVisual StudioTask.FileName + ":" + lVisual StudioTask.Line;
}
catch (System.NotImplementedException)
{
lSubject = lVisual StudioTask.Description;
}

lFilter = "[Subject] = '" + lSubject.Trim() + "'";
lExistingTasks = lOutlookTasks.Restrict (lFilter);

if (lExistingTasks != null && lExistingTasks.Count > 0)
{
lExistingTask = (_TaskItem)lExistingTasks.GetFirst ();
lExistingTask.Complete = lVisual StudioTask.Checked;
lExistingTask.Save ();

if (chkLog.Checked)
lstLog.Items.Add ("Updated: " + lSubject);

continue;
}

_TaskItem lNewTask = (_TaskItem)
mOutlookApp.CreateItem (OlItemType.olTaskItem);
lNewTask.Subject = lSubject.Trim ();
lNewTask.Complete = lVisual StudioTask.Checked;
lNewTask.Categories = mCustomCategory;
lNewTask.Save ();

if (chkLog.Checked)
lstLog.Items.Add ("Added: " + lSubject);
}
}

The sync algorithm is a very simple one. We simply use the Description , FileName and Line properties of Visual Studio .NET's tasks as a key which forms the subject line of Outlook's task. We search for a task with this subject under our custom category, and if this is found, we update the task with the completion status. On the other hand, if this task is not found in Outlook, we use the Outlook objects to create a new task and save it to the Tasks folder. Remember that, as we mentioned earlier in this section, we do not delete any tasks from Outlook.

One caveat to note here is that user defined tasks in Visual Studio .NET do not implement the FileName and Line properties, and in this case, we simply use the Description as our key and subject.

The above screen shot shows the simple GUI that displays the status of each sync item.

For this add-in to work, you need to have Outlook installed on your machine. If you have multiple profiles set up, or the profile is password protected, you will be prompted to choose a profile and\or enter your credentials to log into Outlook when this add-in is invoked. If you have already logged into Outlook and Outlook is running on your machine, this add-in will not prompt you for any credentials.

Conclusion

Visual Studio .NET makes the process of developing add-ins extremely simple and straight forward. While we have only covered a few objects from the Visual Studio .NET Automation Model, interested developers should spend some time playing around with other objects that are accessible from within an add-in, to develop add-ins that meet their individual requirements.


Creating a custom DataGridColumnStyle in C#

Creating a custom DataGridColumnStyle in C#
http://www.csharptoday.com/content.asp?id=1656
Article Rating: 4
Published on September 18, 2001
Abstract
The .NET framework provides many unique controls like the DataGrid, which is a data bound list control that displays the items from a data source in a table. The DataGrid is the most powerful and flexible control in the .NET toolbox. In this article, Chad DePue takes a look at one aspect of the DataGrid’s flexibility, by creating a custom column. The DataGrid provides two types of columns for free, the DataGridTextBoxColumn and the DataGridBoolColumn, but there are many types of data that won’t fit into either of these two column types. In the example given, we want to search a directory for icon (.ICO) files and display each icon as a row in the DataGrid. Neither of the existing column styles will do this for us, so we create our own.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

Microsoft has provided a powerful library for creating client Windows applications in the .NET framework. Many of the controls that ship with Windows Forms are wrappers on the common Win32 controls that have shipped with every version of Windows since Win95. Textboxes, buttons, sliders, and listboxes, to name just a few, are familiar territory for C++ and Visual Basic developers. The .NET framework also provides controls that exist only in the .NET world. These controls, like NotifyIcon, ErrorProvider, and DataGrid, take advantage of features in .NET Windows Forms that are unavailable to traditional application developers. They allow the developer to override classes and implement interfaces that change the way the control interacts with the user.

The DataGrid is a data bound list control that displays the items from a data source in a table. The control allows you to select, sort, edit, add and delete items. Each row in the DataGrid represents a record in the data source. It also understands relationships in the tables of some data sources, and can display records hierarchically. For example, the control could show a list of customers, and a sublist of the orders for those customers. The DataGrid is the most powerful and flexible control in the .NET toolbox.

In this article, we'll take a look at one aspect of the DataGrid's flexibility by creating our own custom column. The DataGrid provides two types of columns for free, the DataGridTextBoxColumn and the DataGridBoolColumn. These two columns can handle any string or boolean data, and with the powerful formatting options of the DataGridTextBoxColumn, there are few string formatting options that are unavailable. There are many other types of data, however, that won't fit into either of these two column types . In this example, we want to search a directory for Icon (.ICO) files and display each icon as a row in the DataGrid. Neither of the existing column styles will do this for us.

To create a new column style, a class must extend the abstract class DataGridColumnStyle. This abstract class is the base class for DataGridBoolColumn and DataGridTextBoxColumn. Each subclass must implement methods to handle painting, editing, and formatting in the DataGrid. Because our column displays icons, in-place editing makes little sense, and the class will not allow editing in the DataGrid.

To create the DataGridIconColumn class we will need to:

  • Create a class that extends from DataGridColumnStyle
  • Implement the abstract methods Paint, Edit, Commit, Abort, GetMinimumHeight, GetPreferredSize, and GetMinimumSize

To show the DataGridIconColumn in action we will:

  • Create a sample form to host the DataGrid
  • Create a IconDataSet object to hold the data
  • Bind the DataGrid to the IconDataSet
  • Load a directory of .ICO files into the DataGrid

Extending from DataGridColumnStyle

First, create a new project, and add a DataGridIconColumn class:

  1. Start Visual Studio
  2. Create a new C# Windows Application Project.
  3. Name your project " IconChooser "
  4. Select View | Solution Explorer
  5. In the Solution Explorer, right-click on Form1.cs and choose " Delete "
  6. Right-click on the project and choose Add | Add New Item.
  7. Choose " Class " and name the class DataGridIconColumn.cs

The newly created class will already have "using System". Add the namespaces for System.Drawing and System.Windows.Forms:

using System;
using System.IO;
using System.Drawing;
using System.Windows.Forms;

Change the DataGridIconColumn class declaration to extend DataGridColumnStyle:

public class DataGridIconColumn : DataGridColumnStyle
{
public DataGridIconColumn()
{
}

If you attempt to compile the class at this point, you will be presented with various errors. These are due to the fact that DataGridIconColumn isn't overriding the abstract methods in DataGridColumnStyle. The methods that we must later override are (these correspond to the compile errors):

protected override void Paint(Graphics g, Rectangle bounds,
CurrencyManager source, int rowNum, bool alignToRight);
protected override void Paint(Graphics g, Rectangle bounds,
CurrencyManager source, int rowNum);
protected override void Paint(Graphics g, Rectangle bounds,
CurrencyManager source, int rowNum, Brush backBrush,
Brush foreBrush, bool alignToRight);
protected override void Edit(CurrencyManager source, int rowNum,
Rectangle bounds, bool readOnly);
protected override void Edit(CurrencyManager source, int rowNum,
Rectangle bounds, bool readOnly, string instantText);
protected override void Edit(CurrencyManager source, int rowNum,
Rectangle bounds, bool readOnly, string instantText, bool cellIsVisible);
protected override void Abort(int rowNum);
protected override int GetMinimumHeight();
protected override int GetPreferredHeight(Graphics g, object value);
protected override Size GetPreferredSize(Graphics g, object value);
protected override bool Commit(CurrencyManager dataSource, int rowNum);

Later, we will also add two Formatting methods, so that the column can be hosted in the designer:

public string Format();
public string FormatInfo();

Now we need to implement these methods. We'll start w ith the Sizing methods, as they are the easiest to implement. Icons either have a 32x32 pixel image, a 16x16 pixel image, or both. Since the GDI+ painting methods will automatically scale the image if necessary, we'll set the minimum width and height of our column and its rows to 32 pixels:

protected override int GetMinimumHeight()
{
return 32; // 32 pixel minimum makes icons look better
}

protected override int GetPreferredHeight(
Graphics g,
object value
)
{
return 32;
}

protected override Size GetPreferredSize(
Graphics g,
object value
)
{
return new Size(32,32);
}

Overriding the editing methods is easy, because we don't allow editing of the icons:

protected override bool Commit(
CurrencyManager dataSource,
int rowNum
)
{
return true; // since we're read-only, return true
}

protected override void Edit(
CurrencyManager source,
int rowNum,
Rectangle bounds,
bool readOnly
)
{
// do nothing - read only
}

protected override void Edit(
CurrencyManager source,
int rowNum,
Rectangle bounds,
bool readOnly,
string instantText
)
{
// do nothing - read only
}

protected override void Edit(
CurrencyManager source,
int rowNum,
Rectangle bounds,
bool readOnly,
string instantText,
bool cellIsVisible
)
{
// do nothing - read only
}

protected override void Abort(int rowNum)
{
// do nothing - read only
}

Now we override the painting methods, where the icon is rendered in the cell. There are three Paint() methods, and we must implement all of them. DataGridColumnStyle provides a useful helper function called GetColumnValueAtRow(). This method returns an object that represents the value in the cell at this row and column. We cast this object to a string, and then use the string to load the icon from the disk. The GDI+ graphics object then draws the icon within the rectangle that represents this cell:

protected override void Paint(
Graphics g,
Rectangle bounds,
CurrencyManager source,
int rowNum,
bool alignToRight
)
{
try
{
// get icon filename
string sFileName =
this.GetColumnValueAtRow(source,rowNum).ToString();
// load icon
Icon i = new Icon(sFileName);

// draw icon in provided rectangle
g.DrawIcon(i,bounds);
}
catch (Exception e)
{
}

}

The second Paint() method simply calls the first one, with alignToRight set to false:

protected override void Paint(
Graphics g,
Rectangle bounds,
CurrencyManager source,
int rowNum
)
{
this.Paint(g,bounds,source,rowNum,false);
}

The third method supplies a backBrush and foreBrush. We ignore the foreBrush as we will always paint the icon with its own colors. The backBrush is painted onto the graphics surface before the icon, then the icon is painted over the surface. This background color is generally either the white background of the grid, or the blue background of a selected row.

protected override void Paint(
Graphics g,
Rectangle bounds,
CurrencyManager source,
int rowNum,
Brush backBrush,
Brush foreBrush,
bool alignToRight
)
{

try
{
// paint background
g.FillRegion(backBrush,new Region(bounds));

// load icon
string sFileName =
this.GetColumnValueAtRow(source,rowNum).ToString();
Icon i = new Icon(sFileName);

// we want to paint a maximum of a 32x32 area
// even if the size of the cell's rectangle is
// larger than 32 pixels.
int w = Math.Min(bounds.Width,32);
int h = Math.Min(bounds.Height,32);

// because we're using a custom sized rectangle
// we need to create it
Rectangle r = new Rectangle(bounds.X,bounds.Y,w,h);

// now draw the icon in that rectangle
g.DrawIcon(i,r );
}
catch (Exception e)
{
}
}

Now add the two Formatting methods needed to function in the designer:

public string Format
{
get
{
return "";
}

set
{
}
}
public string FormatInfo
{
get
{
return "";
}

set
{
}
}

Creating the sample form

We'll create a sample form to host a DataGrid, and provide a method of entering a directory to search for .ICO files.

  1. Right click on the IconChooser project and choose Add | Add New Item.
  2. Click on "Windows Form" on the list to the right of the Add New Item dialog
  3. Name the form " IconChooser.cs ", and click " Open "

Adding controls to the Form

Now we will add a TextBox to type a path to our icon directory, a button to load the icons from the directory, and a DataGrid to display the icons. Once we have this sample form working, we'll create our new grid column and hook it up with the DataGrid.

You can use the Toolbox to drag the TextBox, button, and DataGrid onto the form.

  1. Select View | Toolbox
  2. Click on the " Windows Forms " tab in the Toolbox
  3. Double click on the TextBox, button, and DataGrid.

Now you should have three controls stacked on top of each other at the top of the form. We'll quickly position the controls on the form, and then use the Anchor tag to give them the correct behavior when the form is resized.

  1. Drag the DataGrid so that the bottom of the grid is just above the bottom of the Form, and the left side of the grid is next to the left side of the form.
  2. Expand the DataGrid so that it has the same amount of space between its edges and the Form's edges, and it takes up about half of the Form's height.
  3. Right click on the DataGrid and choose Properties.
  4. Choose the Anchor property, and anchor the DataGrid to the bottom, left, and right sides of the Form.
  5. Resize the TextBox across the form, and anchor it to the left, top, and right sides of the Form.
  6. Move the button to the right of the TextBox and above the DataGrid, and anchor it to the top and right sides of the Form.
  7. Set the Text property of TextBox1 to "\Program Files\Microsoft Visual Studio.NET\Common7\Graphics\icons\Computer". This directory will contain icons installed with Visual Studio.NET.
  8. Set the Text property of Button1 to "Load Icons".

Enlarge the DataGrid and rearrange the controls so that the form looks like this:

Creating an IconDataSet

Because the DataGrid is a "Data Bound" control, it cannot display arbitrary data as a TextBox or a ListBox can. A DataGrid must be bound to a DataSet or a DataTable, or a class that implements IList or IListSource. To show the icon files on our sample form, we'll create an IconDataSet, bind the DataGrid to it, and then populate the IconDataSet with one row for each Icon file.

To create the IconDataSet :

  1. Right-click on the IconChooser project in the Solution Explorer and choose Add New Item
  2. Click on " Data Set " on the right side of the Add New Item dialog
  3. Name the DataSet " IconDataSet.xsd ", and click " Open "

The XML Schema designer will open with a blank surface on which we can drag objects. This designer is used to build objects that hold rows of data, and describe the relationships between those objects. Our schema is simply a table of Icon descriptions and filenames.

  1. Choose View | Toolbox to show the Toolbox with available XML Schema objects
  2. Drag an element object from the Toolbox to the designer surface
  3. Type "Icon" in place of the word "element"

The element should look like this:

  1. Type "Description" in the cell directly underneath "Icon", and leave "string" in the second column
  2. Type "Filename" in the first column of the next row, and leave "string" in the second column

The resulting Icon element will look like this:

Save the IconDataSet and close the window. Now, we need an instance of an IconDataSet on the form. The procedure for adding any "typed" instance of a DataSet on a Windows Form is not intuitive:

  1. View IconChooser.cs in the designer
  2. Open the Toolbox and click the Data tab
  3. Drag a DataSet from the toolbox to the designer surface

You will be prompted with the following:

Click OK, and iconDataSet1 will be added to the bottom of the Form window.

Code behind the Form

The code to populate the DataGrid iterates through the .ico files in the directory and adds an IconRow for each file. Double-click on the button to add a click event handler, and then enter this code:

      private
            void button1_Click(object sender, System.EventArgs
            e)
{
DirectoryInfo d = new
            DirectoryInfo(textBox1.Text);
FileInfo[] fi =
            d.GetFiles("*.ico");

iconDataSet1.Clear();

foreach(FileInfo
            f in
            fi)
{
iconDataSet1.Icon.AddIconRow(f.Name,f.FullName);
}
iconDataSet1.AcceptChanges();
}

Because we created a new Form, we need to add a static Main() method to instantiate our form:

public static void Main()
{
Application.Run(new IconChooser());
}

DataGrid Mappings

The last step in our example is adding the DataGrid's column mappings. Columns are automatically mapped in the DataGrid by default when it is bound to a DataSet. We cannot use this automatic column mapping because the DataGrid will not know to use our custom column. Firstly, bind the DataGrid to our iconDataSet1 object by setting the DataSource property of the grid to be " iconDataSet1.Icon ". Then, click on the TableStyles property in the property grid. The DataGridTableStyle dialog will appear. Click " Add ", and the dialog should look like this:

  • Set the MappingName property to "Icon"
  • Click the GridColumnStyles button to open the DataGridColumnStyles editor
  • Click " Add " twice to add two DataGridTextBoxColumn objects
  • Set the header text and mapping name of the first column to "Description"
  • Set the name of the second column to be " dataGridIconColumn1 "
  • Set the second column's header text and mapping name to " Filename "
  • Click OK, and then OK again, to return to the designer.

Testing the New Column

At this point you should be able to build and run the application. The first column will display the description and the second will display the filename. If you receive compilation errors, you will need to change the following lines in the declaration section of the form:

private IconChooser.IconDataSet iconDataSet1;

to:

private IconDataSet iconDataSet1;

And in the InitialiseComponent() method of the form:

this.iconDataSet1 = new IconChooser.IconDataSet();

to:

this.iconDataSet1 = new IconDataSet();

You will note that our new column type did not show up in the dialog. Unfortunately, there is no way in .NET Beta2 to register our column type. We must perform a quick search and replace on the code in our form to use our new column.

In the declaration section of the form, find this line:

private System.Windows.Forms.DataGridTextBoxColumn dataGridIconColumn1;

And replace it with this:

private DataGridIconColumn dataGridIconColumn1;

In the InitializeComponent() method of our form, find this line:

this.dataGridIconColumn1 = new System.Windows.Forms.DataGridTextBoxColumn();

And replace it with this:

this.dataGridIconColumn1 = new DataGridIconColumn();

Once this code change has been made, you can safely use the designer, and even the DataGridColumnStyles editor, without any problem.

We're ready to test our new column. Build the project, start the application, and click "Load Icons". You should see a form like this:

Summary

Now you know how to create custom DataGrid columns, and manipulate them in the Visual Studio .NET designer. With this knowledge, and the flexibility of the DataGrid, the C# developer has unlimited control of the type and format of data displayed on a Windows Form.


Building a Moderated Discussion Forum in C#

Building a Moderated Discussion Forum in C#
http://www.csharptoday.com/content.asp?id=1655
Article Rating: 4.5
Published on September 14, 2001
Abstract
In this article Juan Martinez tackles one of the most common features on a website: A discussion forum. Forums are a primary tool for building online communities and driving traffic to the website. We will step through the development of a moderated discussion forum from scratch. Analysis, design, implementation and deployment will be covered to furnish a working moderated forum in C#. This article is available as a $20 e-doc from http://www.amazon.com/exec/obidos/ASIN/B00005T5SR/ref=nosim/wrox-20.
Article

In this article we will tackle one of the most common features on a website - a discussion forum, one of the primary tools for building online communities and driving traffic to the website. We will step through the development of a moderated discussion forum from scratch. Analysis, design, implementation and deployment will be covered to furnish a working moderated forum in C#.

Our moderated forum will have a list of categories which will have forums. The forums will have the choice to be manageable or open to the public. Forum administration will be handled in the same web forms by using a login system which will provide personalization according to the user.

The application will work on an IIS server and SQLServer 2000 will be used as the database backend. ASP.NET pages will be used for presentation; C# user controls and code behind will implement the core logic of the application. This release will work with the .NET framework beta 2.

The following knowledge is assumed:

  • Regular knowledge of the C# language.
  • Regular knowledge of Web Form development and the code behind technique.
  • Database design knowledge.
  • Ability to properly configure an IIS server.
  • Ability to manage the SQLServer and implement the database script.

Doing the analysis

Prior to doing the programming labor we will do some analysis and design work. The first step will be to discuss the flow of information in the discussion forum. We will also establish our requirement list, and based on this the database schema will come up.

Forum flow

In this particular application we intend to develop a discussion forum which will have the choice to be moderated of not. The difference relies in the ability of users to freely post messages. In the regular forum users can post messages without restraint while in the moderated version messages should wait for approval from the administrator.

Regular forum

We see here a diagram that describes the steps to post a message in our regular forum.

  • 1 - The message thread is presented.
  • 2 - The user posts a new message or replies to an existing message.
  • 3 - User logs into the system.
  • 4 - The message is saved to the database.
  • 5 - The message is authorized by default.
  • 6 - Forum thread is updated with the new entry and changes are reflected.

Moderated forum

In our moderated version of the forum a user posts and an authorized administrator should manually accept the message.

  • 1 - The message thread is presented.
  • 2 - The user posts a new message or replies to an existing message.
  • 3 - User logs into the system.
  • 4 - The message is saved to the database.
  • 5 - The message is checked by the site administrator. Here the administrator will see all messages, accepted and not accepted and will check the messages for display.
  • 6 - If the message is rejected by the administrator it is deleted from the system
  • 7 - If the message is accepted then it is marked as authorized
  • 8 - Forum thread is updated with the new entry and changes are reflected.

Our moderated forum will have the option to be moderated or not, so we will be implementing both these two models. Technically it is just a matter of setting an authorized flag on or off, but we will deal with that later on.

Requirements

We now have a clear vision of the models we will be implementing, so it is now time to list our requirements. This will be the basis of our further development.

These are the requirements that should be accomplished:

  • Have a list of categories.
  • Have forums grouped in categories.
  • Forums should have the option of being moderated or not.
  • Users should register in the system to be able to do postings to the forums. Reading messages is allowed without registration.
  • Site management should be done within the site by checking for administrator privileges upon login and displaying the proper administrative tools.

Database model

Now that we know our requirements we will do some database design. We need a total of four entities to handle this application's data. These are:

  • User - This table will hold the name, email, password and permissions for the user. The email field will be used as the username for login purposes. We will distinguish our normal users from administrators with the IsAdministrator field which will have a 0 for normal users and a 1 for administrators. Further security levels, like 2 for less privileged moderators, can be included and will only need the appropriate validation in the WebForms.
  • Category - The category table will hold a list of categories with their own descriptions.
  • Forum - The forum table will store a list of forums with its descriptions. It saves a reference to the parent category and the choice to be moderated or not. The IsModerated field will have a 0 for regular forums and a 1 for moderated forums.
  • Message - The message table will handle most of the work by storing the messages in the forums. It saves a reference to the forum it belongs, to the user that posted it and a reference to the parent message. If it is 0, then it is a root message (new posting).

Conceptual model

The conceptual model shows us the relationships between tables and is the main model for our database.

Physical model

From the conceptual model we generate the physical model that shows all foreign keys and is the final design for the database. From here we will generate our database creation script.

SQL Script

The following script will be used to create the database in the SQLServer database. This should be executed in the Query Analyzer. A content script is provided with the code download to insert some dummy data in the database.

This creates the category table. We use an identity field as a key. This takes the responsibility of creating unique numbers for the key from us.

CREATE TABLE [dbo].[category] (
[categoryId] [int] IDENTITY (1, 1) NOT NULL ,
[categoryName] [varchar] (255) COLLATE SQL_Latin1_General_CP1_CI_AS NOT NULL ,
[categoryShortDescription] [text] COLLATE SQL_Latin1_General_CP1_CI_AS NOT NULL ,
[categoryLongDescription] [text] COLLATE SQL_Latin1_General_CP1_CI_AS NOT NULL
) ON [PRIMARY] TEXTIMAGE_ON [PRIMARY]
GO

This creates the index for the table. An index is needed to help the performance of the database. Indexes are placed on key fields and other frequently searched fields in tables. In our case we will index only the key fields.

ALTER TABLE [dbo].[category] WITH NOCHECK ADD
CONSTRAINT [PK_CATEGORY] PRIMARY KEY CLUSTERED
(
[categoryId]
) ON [PRIMARY]
GO

CREATE INDEX [category has several forums_FK] ON [dbo].[forum]([categoryId]) ON [PRIMARY]
GO

Next this piece of code creates the references between the category table and the forum table. Each forum must belong to a category and this constraint ensures this. We also specify that upon category deletion, all forums will be deleted in cascade.

ALTER TABLE [dbo].[forum] ADD
CONSTRAINT [FK_FORUM_CATEGORY _CATEGORY] FOREIGN KEY
(
[categoryId]
) REFERENCES [dbo].[category] (
[categoryId]
) ON DELETE CASCADE ON UPDATE CASCADE
GO

The rest of the tables follow the same approach as the one described here. For the complete code listing please look at the code download.

Stored Procedures

We shall use stored procedures to manipulate our data in the database. We created a stored procedure for the insertion, edition and deletion of each table.

The AddCategory stored procedure.

CREATE PROCEDURE [AddCategory]
(
@categoryName_2 [varchar](255),
@categoryShortDescription_3 [text],
@categoryLongDescription_4 [text]
)

AS
INSERT INTO category
(
[categoryName],
[categoryShortDescription],
[categoryLongDescription]
)
VALUES
(
@categoryName_2,
@categoryShortDescription_3,
@categoryLongDescription_4
)
GO

The UpdateCategory stored procedure.

CREATE PROCEDURE [UpdateCategory]
(
@categoryId_1 [int],
@categoryName_2 [varchar](255),
@categoryShortDescription_3 [text],
@categoryLongDescription_4 [text]
)
AS
UPDATE [category]
SET [categoryName] = @categoryName_2,
[categoryShortDescription] = @categoryShortDescription_3,
[categoryLongDescription] = @categoryLongDescription_4
WHERE
(
[categoryId] = @categoryId_1
)
GO

The DeleteCategory stored procedure.

CREATE PROCEDURE [DeleteCategory]
(
@categoryId_1 [int]
)

AS
DELETE [category]
WHERE
(
[categoryId] = @categoryId_1
)
GO

As with the table structure, the rest of the procedures follow the same logic and will not be presented.

Web application architecture

We will start our design process by defining the architecture to be used in our application. We have four main screens in our application which are the registration screen, the category list, the forum list and the message thread screen. We will dig into each one of them now.

Registration Form

This will be registration.aspx. This screen is a simple Web Form to allow new users to register and be able to post into our forums.

User controls:

  • uc_title.ascx - Will display the main title of the page and the name of the user if logged.
  • uc_login.ascx - will provide a way to login to the system. If the user is already logged then it will a logout button.
  • uc_login_add.ascx - will display a web form to save new users details.

Code behind classes:

  • uc_title.ascx.cs - looks for session data to display in the forum header.
  • uc_login.ascx.cs - will validate the user to provide the login / logout functionality.
  • uc_login_add.ascx.cs - will handle the insertion of the new user into the database.

Screen structure:

Category list

This will be category.aspx, which will display a list of categories and provide a way to add new categories to administrators.

User controls:

  • uc_category.ascx - will display a list of categories using a data list control. It will provide an edit template if the user is an administrator.
  • uc_category_add.ascx - will provide the functionality to add a new category using a server side Web Form.

Code behind classes:

  • uc_category.ascx.cs - will do the work of binding the data list with data.
  • uc_category_add.ascx.cs - will handle the addition of new categories.

Screen structure:

Forum list

This will be forum.aspx. The forum list is similar to the category list. We display a list of available forums and provide a way to add new ones to the administrators.

User controls:

  • uc_forum.ascx - will display a list of forums using a data list control. It will provide an edit template if the user is an administrator.
  • uc_forum_add.ascx - will provide the functionality to add a new forum using a server side Web Form.

Code behind classes:

  • uc_forum.ascx.cs - will bind the forum data list to display the forums.
  • uc_forum_add.ascx.cs - will handle the addition of new forums.

Screen structure:

Message screen

The message screen consists of our threaded list of messages. Additional forms are used to post and reply to messages. The most interesting part of this is the threaded message list and since we do not have a recursive data list we will resort to some tricks to achieve that.

User controls:

  • uc_message.ascx - will display a threaded list of messages from the selected forum. It will also provide the view pane for the messages contents and a reply form.
  • uc_message_add.ascx - will provide the functionality to post a new message using a server side Web Form.

Code behind classes

  • uc_forum.ascx.cs - will bind the forum data list to display the messages in a threaded fashion. Here we will construct our DataTable recursively first, and when we are finished we will bind it to our DataList control.
  • uc_forum_add.ascx.cs - will handle the addition of new message posts.

Screen structure:

Making it happen

We have our plan set up and we will now start to do some programming. We will iterate through each of the screens described above and show the code necessary to make this application work.

Registration Form

We begin our development in the simplest of forms, the registration form. This will also include the development of two components that we will reuse throughout the site.

The container web form registration.aspx

This code encapsulates the user controls and will provide our basic site wireframe which we will reuse.

<%@ Register TagPrefix="uc1" TagName="uc_title" Src="uc_title.ascx" %>
<%@ Page language="c#" Codebehind="registration.aspx.cs" AutoEventWireup="false"
Inherits="AspTodayForum.registration" debug="True"%>
<%@ Register TagPrefix="uc1" TagName="uc_login" Src="uc_login.ascx" %>
<%@ Register TagPrefix="uc1" TagName="uc_login_add" Src="uc_login_add.ascx" %>
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" >
<HTML>
<HEAD>
<meta content="Microsoft Visual Studio 7.0" name="GENERATOR">
<meta content="C#" name="CODE_LANGUAGE">
<meta content="JavaScript (ECMAScript)" name="vs_defaultClientScript">
<meta content="http://schemas.microsoft.com/intellisense/ie5" name="vs_targetSchema">
</HEAD>
<body MS_POSITIONING="GridLayout">
<form id="registration" method="post" runat="server">
<table cellSpacing="0" cellPadding="2" width="800" border="1">
<tr>
<!-- Start Title Control -->
<td width="800" colSpan="2" valign="top">
<uc1:uc_title id="Uc_title1" runat="server">
</uc1:uc_title>
</td>
<!-- End Title Control -->
</tr>
<tr>
<!-- Start Login Control -->
<td width="200" valign="top">
<uc1:uc_login id="Uc_login2" runat="server">
</uc1:uc_login>
</td>
<!-- End Login Control -->
<!-- Start Content -->
<td width="600" valign="top">
<uc1:uc_login_add id="Uc_login1" runat="server">
</uc1:uc_login_add>
</td>
<!-- End Content -->
</tr>
</table>
</form>
</body>
</HTML>

The registration.aspx.cs code behind class

Here we just save the page name in a session for redirection.

private void Page_Load(object sender, System.EventArgs e)
{
// Put user code to initialize the page here
Session["currentPage"] = "registration.aspx";
}

The uc_title.ascx user control

This is the code of the user control that displays the title of the page:

<%@ Control Language="c#" AutoEventWireup="false" Codebehind="uc_title.ascx.cs"
Inherits="AspTodayForum.uc_title"%>
<table width="800" border="0" cellpadding="0" cellspacing="0">
<tr>
<td>
<asp:Label id="lblTitle" runat="server">
Moderated C# Forum</asp:Label>
</td>
<td align="right">
<asp:Label id="lblUser" runat="server">
</asp:Label>
</td>
</tr>
</table>

The uc_title.ascx.cs code behind class

This is the code behind the uc_title.ascx user control. It checks for session data and displays it in the forum header.

if (Session["userName"] == null && (string)Session["userName"] == "")
{
lblUser.Text = "";
}
else
{
lblUser.Text = (string)Session["userName"];
}

The uc_login.ascx user control

This control will display a form for login the user to the system. If logged it will display a logoff button.

<%@ Control Language="c#" AutoEventWireup="false" Codebehind="uc_login.ascx.cs"
Inherits="AspTodayForum.uc_login" debug="True"%>
<table width="200" border="0" cellpadding="0" cellspacing="0">
<tr>
<td width="200">
<asp:Label id="lblHeader" runat="server">
Login</asp:Label>
</td>
</tr>
<tr>
<td width="200">
<IMG height="10" src="images/spacer.gif" width="200">
</td>
</tr>
<tr>
<td width="200">
<asp:Label id="lblEmail" runat="server">
Email:</asp:Label>
</td>
</tr>
<tr>
<td width="200">
<asp:TextBox id="txtEmail" runat="server">
</asp:TextBox>
</td>
</tr>
<tr>
<td width="200">
<asp:Label id="lblPassword" runat="server">
Password:</asp:Label>
</td>
</tr>
<tr>
<td width="200">
<asp:TextBox id="txtPassword" runat="server" TextMode="Password">
</asp:TextBox>
</td>
</tr>
<tr>
<td width="200">
<IMG height="10" src="images/spacer.gif" width="200">
</td>
</tr>
<tr>
<td width="200">
<asp:Button id="btnLogin" runat="server" Text="Login">
</asp:Button>
</td>
</tr>
</table>
<table width="200" border="0" cellpadding="0" cellspacing="0">
<tr>
<td width="200">
<asp:Button id="btnLogout" runat="server" Text="Logout" Visible="False">
</asp:Button>
</td>
</tr>
</table>

The uc_login.ascx.cs code behind class

Here is the code to validate the user information, logging in and out.

This is the page initialization code. It checks for session info to determine if we should present the login form or the logout button.

private void Page_Load(object sender,
            System.EventArgs e)
{
// Put user code to initialize the page
            here
if (Session["userEmail"] == null)
{
Session["userId"]
            = "";
Session["userName"] = "";
Session["userEmail"] =
            "";
Session["userIsAdministrator"] =
            "0";
SwitchForm(1);
}
else
{
if
            ((string)Session["userEmail"] == "")
{
SwitchForm(1);
lblHeader.Text = "Login";
}
else
{
SwitchForm(0);
lblHeader.Text = "User " +
            (string)Session["userName"] + " logged
            in.";
}
}
}

The following function switches the forms from login form to logout button. We accomplish this by modifying the visibility of the web controls:

private void SwitchForm (int
            whichForm)
{
//Switch user form
if (whichForm == 0)
{
lblEmail.Visible = false;
txtEmail.Visible =
            false;
lblPassword.Visible = false;
txtPassword.Visible =
            false;
btnLogin.Visible = false;
btnLogout.Visible =
            true;
}
else
{
lblEmail.Visible =
            true;
txtEmail.Visible = true;
lblPassword.Visible =
            true;
txtPassword.Visible = true;
btnLogin.Visible =
            true;
btnLogout.Visible = false;
}
}

This is the login button code. We open a data reader with a query that looks for information with the provided email and password. If we find a user we log him in, else we display an error message.

private void btnLogin_Click(object sender,
            System.EventArgs e)
{
//Retreive user info
string
            connString =
            ConfigurationSettings.AppSettings["connString"];

//Retreive
            new Id from main order table
string strSelect = "SELECT userId,
            userName, userIsAdministrator FROM forumUser WHERE userEmail='" +
txtEmail.Text + "' AND userPassword='" + txtPassword.Text +
            "'";
SqlConnection myConnection = new
            SqlConnection(connString);
SqlCommand myCommand = new
            SqlCommand(strSelect,
            myConnection);

myConnection.Open();
SqlDataReader dr =
            myCommand.ExecuteReader();
if (dr.Read())
{
//Save in
            session variables
Session["userId"] =
            dr.GetInt32(0).ToString();
Session["userName"] =
            dr.GetString(1);
Session["userEmail"] =
            txtEmail.Text;
Session["userIsAdministrator"] =
            dr.GetBoolean(2).ToString();

//Switch user
            form
SwitchForm(0);
lblHeader.Text = "User " +
            (string)Session["userName"] + " logged
            in.";
Response.Redirect((string)Session["currentPage"]);
}
else
{
lblHeader.Text = "The information was
            incorrect.";
}
myConnection.Close();
}

Finally we see the logout button, which simply empties the session variables and switches to the login form.

private void btnLogout_Click(object sender,
            System.EventArgs e)
{
//Reset variables
Session["userId"] =
            "";
Session["userName"] = "";
Session["userEmail"] =
            "";
Session["userIsAdministrator"] = "0";

//Switch user
            form
SwitchForm(1);
lblHeader.Text =
            "Login.";
Response.Redirect((string)Session["currentPage"]);
Response.End();
}

The uc_login_add.ascx user control

This control displays a form for a new user to register on the system.

<%@ Control Language="c#" AutoEventWireup="false" Codebehind="uc_login_add.ascx.cs"
Inherits="AspTodayForum.uc_login_add" debug="True"%>
<table cellSpacing="0" cellPadding="0" width="600" border="0">
<tr>
<td width="600" colSpan="2">
<asp:label id="lblInstructions" runat="server">
Please provide your information to register with us.</asp:label>
</td>
</tr>
<tr>
<td width="600" colSpan="2">
<IMG height="10" src="images/spacer.gif" width="600">
</td>
</tr>
<tr>
<td width="200">
<asp:label id="lblName" runat="server">
Complete name:</asp:label>
</td>
<td width="400">
<asp:textbox id="txtName" runat="server" MaxLength="255">
</asp:textbox>
</td>
</tr>
<tr>
<td align="left" width="200" colSpan="1" rowSpan="1">
<asp:label id="lblEmail" runat="server">
Email:</asp:label>
</td>
<td width="400">
<asp:textbox id="txtEmail" runat="server" MaxLength="255">
</asp:textbox>
</td>
</tr>
<tr>
<td width="200">
<asp:label id="lblPassword" runat="server">
Password:</asp:label>
</td>
<td width="400">
<asp:textbox id="txtPassword" runat="server" TextMode="Password" MaxLength="255">
</asp:textbox>
</td>
</tr>
<tr>
<td width="600" colSpan="2">
<IMG height="10" src="images/spacer.gif" width="600">
</td>
</tr>
<tr>
<td width="600" colSpan="2">
<asp:button id="btnRegister" runat="server" Text="Register">
</asp:button>
</td>
</tr>
<tr>
<td width="600" colSpan="2">
<IMG height="10" src="images/spacer.gif" width="600">
</td>
</tr>
<tr>
<td width="600" colSpan="2">
<asp:Label id="lblResult" runat="server" Width="600px">
</asp:Label>
</td>
</tr>
</table>

The uc_login_add.ascx.cs code behind class

This class has the code necessary to insert the new user to the database. We first check to see if the email is not already registered in the database. If it is not registered we call a stored procedure to save the user information to the database.

private void btnRegister_Click(object
            sender, System.EventArgs e)
{
string connString =
            ConfigurationSettings.AppSettings["connString"];
SqlConnection
            myConnection = new SqlConnection(connString);

string
            strSelect = "SELECT userId FROM forumUser WHERE userEmail='" +
            txtEmail.Text + "'";
SqlCommand myCommand = new
            SqlCommand(strSelect,
            myConnection);

myConnection.Open();
SqlDataReader dr =
            myCommand.ExecuteReader();
bool isRegistered = false;
if
            (dr.Read())
{
isRegistered =
            true;
}
myConnection.Close();

if (isRegistered)
{
lblResult.Text = "The email address is already registered
            in the database.";
}
else
{
myConnection = new
            SqlConnection(connString);
myCommand = new
            SqlCommand("AddForumUser", myConnection);
myCommand.CommandType =
            CommandType.StoredProcedure;

myCommand.Parameters.Add(new
            SqlParameter("@userName_1",
            SqlDbType.NVarChar));
myCommand.Parameters.Add(new
            SqlParameter("@userEmail_2",
            SqlDbType.NVarChar));
myCommand.Parameters.Add(new
            SqlParameter("@userPassword_3",
            SqlDbType.NVarChar));
myCommand.Parameters.Add(new
            SqlParameter("@userIsAdministrator_4",
            SqlDbType.Bit));

myCommand.Parameters["@userName_1"].Value =
            txtName.Text;
myCommand.Parameters["@userEmail_2"].Value =
            txtEmail.Text;
myCommand.Parameters["@userPassword_3"].Value =
            txtPassword.Text;
myCommand.Parameters["@userIsAdministrator_4"].Value
            = 0;

try
{
myConnection.Open();
int queryResult =
            myCommand.ExecuteNonQuery();
myConnection.Close();
}
catch
{
//Add
            error handling
}
finally
{
lblResult.Text = "Success!
            You have been succesfully registered in the system.
Now please
            login with your email and password to be able to post
            messages.";
}
}

The final screen

This is how our registration form will finally look.

Category list

The category list is composed of the title control, the login control and a new uc_category and uc_category_add controls. These two new controls provide the functionality to display and manage categories.

The uc_category.ascx user control

<%@ Page language="c#" Codebehind="category.aspx.cs" AutoEventWireup="false"
Inherits="AspTodayForum.category" %>
<%@ Register TagPrefix="uc1" TagName="uc_title" Src="uc_title.ascx" %>
<%@ Register TagPrefix="uc1" TagName="uc_login" Src="uc_login.ascx" %>
<%@ Register TagPrefix="uc1" TagName="uc_category_add" Src="uc_category_add.ascx" %>
<%@ Register TagPrefix="uc1" TagName="uc_category" Src="uc_category.ascx" %>
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" >
<HTML>
<HEAD>
<meta name="GENERATOR" Content="Microsoft Visual Studio 7.0">
<meta name="CODE_LANGUAGE" Content="C#">
<meta name="vs_defaultClientScript" content="JavaScript (ECMAScript)">
<meta name="vs_targetSchema" content="http://schemas.microsoft.com/intellisense/ie5">
</HEAD>
<body MS_POSITIONING="GridLayout">
<form id="category" method="post" runat="server">
<table cellSpacing="0" cellPadding="2" width="800" border="1">
<tr>
<!-- Start Title Control -->
<td width="800" colSpan="2" valign="top">
<uc1:uc_title id="Uc_title1" runat="server">
</uc1:uc_title>
</td>
<!-- End Title Control -->
</tr>
<tr>
<!-- Start Login Control -->
<td width="200" valign="top">
<uc1:uc_login id="Uc_login1" runat="server">
</uc1:uc_login>
</td>
<!-- End Login Control -->
<!-- Start Content -->
<td width="600" valign="top">
<table width="600" border="0" cellpadding="0" cellspacing="0">
<tr>
<td width="600">
<uc1:uc_category id="Uc_category1" runat="server">
</uc1:uc_category>
</td>
</tr>
<tr>
<td width="600">
<img src="images/spacer.gif" width="600" height="10">
</td>
</tr>
<tr>
<td width="600">
<uc1:uc_category_add id="Uc_category_add1" runat="server">
</uc1:uc_category_add>
</td>
</tr>
</table>
</td>
<!-- End Content -->
</tr>
</table>
</form>
</body>
</HTML>

This is how we should see our user control:

The uc_category.ascx.cs code behind class

The uc_category control is in charge of displaying a list of categories and provides the necessary functionality to edit and delete a category.

We bind our data list control with the following code:

protected void Bind_List()
{
String
            connString =
            ConfigurationSettings.AppSettings["connString"];

SqlConnection
            myConnection = new SqlConnection(connString);
SqlDataAdapter
            myCommand = new SqlDataAdapter("SELECT categoryId, categoryName,
            categoryShortDescription,
categoryLongDescription FROM category
            ORDER BY categoryName", myConnection);

DataSet ds = new
            DataSet();
myCommand.Fill(ds,
            "category");

dlCategory.DataSource = new
            DataView(ds.Tables[0]);
dlCategory.DataBind();
}

We do some validation in the form display. We check if the user has administrator rights. If this is true then we present the administration buttons, else we just display the information for the user.

<% if ((string)Session["userIsAdministrator"]=="True") { %>

<% } %>

The rest of the code manages the different events, which are the edit, update, delete and cancel events of the datalist control. These events make possible the editing of the information in the same control and page.

This function selects the record in the datalist and activates the modification template.

private void dlCategory_EditCommand(object source, System.Web.UI.WebControls.DataListCommandEventArgs e)
{
dlCategory.EditItemIndex = (int)e.Item.ItemIndex;
Bind_List();
}

This function calls the delete stored procedure to delete the record from the database.

private void dlCategory_DeleteCommand(object source, System.Web.UI.WebControls.DataListCommandEventArgs e)
{
String connString = ConfigurationSettings.AppSettings["connString"];
int categoryId = (int)dlCategory.DataKeys[e.Item.ItemIndex];

SqlConnection myConnection = new SqlConnection(connString);
SqlCommand myCommand = new SqlCommand("DeleteCategory", myConnection);
myCommand.CommandType = CommandType.StoredProcedure;

myCommand.Parameters.Add(new SqlParameter("@categoryId_1", SqlDbType.Int));
myCommand.Parameters["@categoryId_1"].Value = categoryId.ToString();

try
{
myConnection.Open();
int queryResult = myCommand.ExecuteNonQuery();
} myConnection.Close();
}
catch
{
//Add error handling
}
finally
{
Bind_List();
}
}

This next function deactivates the modification form for the record in the datalist.

private void dlCategory_CancelCommand(object source, System.Web.UI.WebControls.DataListCommandEventArgs e)
{
dlCategory.EditItemIndex = -1;
Bind_List();
}

This function calls the stored procedure to modify the information for the selected record.

private void dlCategory_UpdateCommand(object source, System.Web.UI.WebControls.DataListCommandEventArgs e)
{
String connString = ConfigurationSettings.AppSettings["connString"];
int categoryId = (int)dlCategory.DataKeys[e.Item.ItemIndex];

SqlConnection myConnection = new SqlConnection(connString);
SqlCommand myCommand = new SqlCommand("UpdateCategory", myConnection);
myCommand.CommandType = CommandType.StoredProcedure;

myCommand.Parameters.Add(new SqlParameter("@categoryId_1", SqlDbType.Int));
myCommand.Parameters.Add(new SqlParameter("@categoryName_2", SqlDbType.NVarChar));
myCommand.Parameters.Add(new SqlParameter("@categoryShortDescription_3", SqlDbType.NText));
myCommand.Parameters.Add(new SqlParameter("@categoryLongDescription_4", SqlDbType.NText));

myCommand.Parameters["@categoryId_1"].Value = categoryId.ToString();
myCommand.Parameters["@categoryName_2"].Value = ((TextBox)e.Item.FindControl("txtCategoryName")).Text;
myCommand.Parameters["@categoryShortDescription_3"].Value =
((TextBox)e.Item.FindControl("txtCategoryShortDescription")).Text;
myCommand.Parameters["@categoryLongDescription_4"].Value =
((TextBox)e.Item.FindControl("txtCategoryLongDescription")).Text;

try
{
myConnection.Open();
int queryResult = myCommand.ExecuteNonQuery();
myConnection.Close();
}
catch
{
//Add error handling
}
finally
{
dlCategory.EditItemIndex = -1;
Bind_List();
}
}

The uc_category_add.ascx user control

This user control has to save the new category info from the user. This is very simple as we only insert the data from the web form. We do the administrator check as well.

The uc_category_add.ascx.cs code behind class

private void btnSaveNew_Click(object sender, System.EventArgs e)
{
//Add new category to the main order table
string connString = ConfigurationSettings.AppSettings["connString"];
SqlConnection myConnection = new SqlConnection(connString);

SqlCommand myCommand = new SqlCommand("AddCategory", myConnection);
myCommand.CommandType = CommandType.StoredProcedure;

myCommand.Parameters.Add(new SqlParameter("@categoryName_2", SqlDbType.NVarChar));
myCommand.Parameters.Add(new SqlParameter("@categoryShortDescription_3", SqlDbType.NText));
myCommand.Parameters.Add(new SqlParameter("@categoryLongDescription_4", SqlDbType.NText));

myCommand.Parameters["@categoryName_2"].Value = txtName.Text;
myCommand.Parameters["@categoryShortDescription_3"].Value = txtShort.Text;
myCommand.Parameters["@categoryLongDescription_4"].Value = txtLong.Text;

try
{
myConnection.Open();
int queryResult = myCommand.ExecuteNonQuery();
myConnection.Close();
}
catch
{
//Add error handling
}
finally
{
Response.Redirect((string)Session["currentPage"]);
Response.End();
}
}

Forum list

The forum list has a very similar functionality to the category list. We will again reuse the title and login controls and we will introduce two new ones: the uc_forum.ascx and uc_forum_add.ascx.

We implement the following new components:

  • Uc_forum.ascx - The user control that displays the list of forums.
  • Uc_forum.ascx.cs - The code behind the user control that displays the list of forums.
  • Uc_forum_add.ascx - The user control that adds a new forum to the list.
  • Uc_forum_add.ascx.cs - The code behind that saves the information of the new forum.

This code is basically the same as the category screen so we will just present the final screens of the site and move on.

Message screen

The message screen will be the main interface for our application. This is where the users view, post and reply to messages. We will again reuse the title and login controls and we will introduce two new ones: the uc_message.ascx and uc_message_add.ascx.

The basic code is the same as above for posting new messages to forums and replying to existing posts, so we implement the following new components:

  • Uc_forum.ascx - The user control that displays the list of messages.
  • Uc_forum.ascx.cs - The code behind the user control that displays the list of messages. Here we build the recursive list and we will be described shortly.
  • Uc_forum_add.ascx - The user control that posts new messages to the forum.
  • Uc_forum_add.ascx.cs - The code behind that saves the information of the new posted message.

The main challenge here is that we need to display a threaded list of messages. We should do this easily with a recursive function that writes the contents to the page with Response.Write(). This is not proper Web Form programming though. Therefore we will use the strategy of building the DataTable recursively first and then binding the constructed DataTable to the DataList.

We do this by providing a general Bind_List() function as usual, but we will construct the DataTable here and then call the recursive function to populate our DataTable. The DataTable is passed as a parameter to the recursive function. Since C# treats every variable by reference, we need not to do anything special here, we will just manipulate the DataTable in each recursive call of the function to add rows.

protected void Bind_List()
{
DataTable
            dt = new DataTable();

dt.Columns.Add(new
            DataColumn("messageId", typeof(Int32)));
dt.Columns.Add(new
            DataColumn("userId", typeof(Int32)));
dt.Columns.Add(new
            DataColumn("messageTitle", typeof(string)));
dt.Columns.Add(new
            DataColumn("messageDate", typeof(DateTime)));
dt.Columns.Add(new
            DataColumn("deep", typeof(Int32)));

Bind_List_Recursive(dt,
            0, 0);

dlMessage.DataSource = new
            DataView(dt);
dlMessage.DataBind();
}

The recursive function is in charge of reading data from the database and adding rows to the Original DataTable.

protected void Bind_List_Recursive(DataTable
            dt, int parentMessageId, int deep)
{
DataRow dr;
String
            connString =
            ConfigurationSettings.AppSettings["connString"];

SqlConnection
            myConnection = new SqlConnection(connString);
SqlDataAdapter
            myCommand = null;
if
            ((string)Session["userIsAdministrator"]=="True")
{
myCommand
            = new SqlDataAdapter("SELECT messageId, userId, messageTitle,
            messageDate
FROM message WHERE forumId=" +
            (string)Session["forumId"] + " AND parentMessageId=" +
parentMessageId.ToString() + " ORDER BY messageDate,
            messageTitle", myConnection);
}
else
{
myCommand = new
            SqlDataAdapter("SELECT messageId, userId, messageTitle, messageDate
FROM message WHERE forumId=" + (string)Session["forumId"] +
" AND messageIsAuthorized=1 AND parentMessageId=" +
            parentMessageId.ToString() +
" ORDER BY messageDate,
            messageTitle", myConnection);
}

DataSet ds = new
            DataSet();
myCommand.Fill(ds, "message");

DataTable dTable
            = ds.Tables[0];
foreach(DataRow dtRow in dTable.Rows)
{
dr
            = dt.NewRow();
dr[0] =
            Int32.Parse(dtRow["messageId"].ToString());
dr[1] =
            Int32.Parse(dtRow["userId"].ToString());
dr[2] =
            dtRow["messageTitle"].ToString();
dr[3] =
            DateTime.Parse(dtRow["messageDate"].ToString());
dr[4] = deep *
            25;
dt.Rows.Add(dr);

Bind_List_Recursive(dt,
            Int32.Parse(dtRow["messageId"].ToString()), (deep +
            1));
}

myConnection.Close();
}

After populating the datalist we have a list of messages in the proper threaded order, we must now provide depth to the messages. We use the variable deep to provide the threaded look to the messages. We bind this property to the width of a spacer image that lies before the message title. This way we effectively have our threaded list of messages.

Finally here is how our threaded message list looks:

Deployment of the site

These steps should be taken to deploy the site.

  • Create a new database in SQLServer called moderatedForum.
  • Once we have created the database we need to run the database creation script provided.
  • Run the dummy data script to insert sample content to the database.
  • Create a new virtual directory in your IIS administration program.
  • Copy the files provided to the virtual directory you just created.

Improvements to our forum

The moderated forum built here provides the skeleton of a very powerful web application. It is now up to you to roll out the final product and polish up the details. Let's list the possible improvements you might want to do to make it a better app.

Feature improvements

  • Email notifications upon message reply
  • Email notification service for new forums and miscellaneous announcements
  • Give the forums a time span and be able to close them so that you just read the contents.

Technical improvements

  • Build based on components for performance and possible code reuse

Conclusion

The system presented shows the basis of a working moderated forum. It involves the main aspects of a moderated forum like the category list to group forums, the list of forums which can be moderated or not, and the threaded message list. Basic administration functionality has been included within the site. Hopefully this article has given you the knowledge to build your own application with ease and to add new functionality to meet your needs.


Visual Studio .NET Add-Ins

Visual Studio .NET Add-Ins
http://www.csharptoday.com/content.asp?id=1645
Article Rating: 4.5
Published on September 7, 2001
Abstract
Extending any development environment can greatly enhance developer productivity by customizing and automating tasks and processes. Visual Studio .NET greatly enhances the add-in model by providing easy access to the entire automation model, and by providing the add-in wizard to generate the plumbing code necessary to hook the add-ins into the development environment.
Article

Introduction

Developers often feel the need to customize and extend any development environment that they work in. Given the extremely diverse needs, requirements, and often quirks of the developer community, it is next to impossible to build a product or an environment that can cater to each individual's needs directly out of the box. In order to meet this goal, most developer tools (and for that matter, most consumer products) allow for the product to be customized and extended by the end user. Visual Studio .NET is no exception, and allows excellent support for developers to customize and extend the development environment. Visual Studio .NET supports this in the form of Macros and Add-Ins . In this article, we will focus on Visual Studio .NET's support for developing Add-Ins and explain the process of developing an add-in component to extend the environment.

This article is based on the Beta 2 release of VS .NET, and the attached sample code requires VS .NET Beta 2 and Windows 2000 to run.

Macros and Add-ins

Visual Studio .NET supports macros and add-ins, which form the basis for extending the development environment. Macros are primarily used to automate repetitive actions via a series of keystrokes and mouse clicks. Add-ins on the other hand are used to build compiled objects that can perform heavy duty processing. These add-ins are very similar to standard components that can not only access the automation objects that are exposed by the Visual Studio .NET IDE, but can also access any COM or .NET component on the local or remote machine. Add-ins are .NET components that can be written in any .NET compliant language. These add-ins can be used to perform any action that can extend and augment the VS .NET IDE, and since add-ins have access to the entire object model of the development environment, it is possible to write extremely powerful and functional components. Some examples of add-ins might be a tool to email files directly from within VS .NET, a component that performs an automated (unattended) build of one or more VS .NET projects, a component that can sync up VS .NET's task list with that of Microsoft Outlook, and so on. VS .NET provides an add-in manager that manages all the installed add-ins on a given machine. This add-in manager can be found under the Tools menu in VS .NET.

The above screen shot displays the Add-in Manager and one of the add-ins that ship with VS .NET.

Visual Studio .NET Automation Model

In order for add-ins to extend the IDE, the add-ins need to act as though they were part of the environment itself. This is necessary for two reasons: primarily, so that they can extend and customize every feature of the environment; and secondly, so that add-ins can provide seamless integration with the IDE. In fact, once an add-in is installed and activated, it behaves as though the add-in were a native component written by the original IDE developers. VS .NET allows for this by exposing all the objects that the IDE consists of, via a well defined object model called the VS .NET Automation Model . The Automation Model is an object based API that allows developers to access the same features and functionality that the IDE offers. Essentially, the IDE exposes the guts of its features via the automation model. Any component can access objects from this model and invoke methods and properties to gain access to the IDE's features. For example, the automation model exposes the Project object that corresponds to a VS .NET project. This object supports the Save and SaveAs methods that can be utilized by an add-in to auto-save the project based on a configurable time interval. VS .NET provides various objects that deal with solutions, projects, windows, editors, debuggers and events to name a few. The complete automation model chart can be found at http://msdn.microsoft.com/vstudio/nextgen/images/img_AutomationObjectModelL.gif?WROXEMPTOKEN=771912Z0RySoapf4LDrfueUzN2 \t _new&WROXEMPTOKEN=620568Zhiox25QOrrsnTPfm53p.

In addition to the automation objects that are exposed by the IDE, the automation model itself can be augmented by user defined automation objects. These user defined objects are custom components that are written by the developers and can be categorized broadly into the following categories.

  • Macros: Objects that are used to record and play back repetitive actions, and also to customize the environment.
  • Add-ins : Objects that are used to customize and extend the development environment.
  • Wizards : Objects that guide a user through a series of actions to accomplish a task. User defined wizards can appear under the New Project or New Item menu objects.
  • Toolbars : Objects that present a user interface via a series of command buttons. Add-ins can expose their own toolbars.
  • Option Pages : Custom pages that appear under the Tools\Options menu. They are used to gather custom configuration information, such as the auto-save time interval.

Creating Custom Add-Ins

Now that we have covered some of the basics of writing an add-in, let us now discuss the process of creating a custom add-in. We will build an add-in that allows you to email the currently open file as an attachment, using SMTP. We will use the Add-In Wizard from Visual Studio .NET to build this add-in.

We start by selecting the VS .NET Add-in project from within the Extensibility Projects. We enter the name and the location of the add-in and this launches the add-in wizard.

Since we will be using C# as the development language, we select C# in step 1.

Since our add-in will be hosted by the VS .NET IDE, we select the second checkbox. Use the first checkbox if the macros editor will host the add-in as well.

In step 3, we provide the name and description of the add-in. This name and description will be used by the add-in manager to display our add-in.

The above screenshot displays the various properties that can be set for the add-in. The remaining steps of the wizard collect some further information, and at the end of the all the steps the wizard will generate the template code for the add-in. This template has the skeleton code with the methods that will be executed when certain events are fired. These events correspond to the add-in startup and shutdown, loading and unloading, and the actual execution of the add-in.

Let us now discuss the generated code.

As you can see from the above screen shot, by default the wizard generates two projects: one project for the actual add-in, and another project that serves as a deployment project for the add-in. When the deployment project is built, it generates a MSI package that can be used to install the add-in on a different machine. A complete discussion of VS .NET Deployment Projects is beyond the scope of this article, we will focus only on the add-in project in the article. Further, in the attached sample code, we have deleted the EmailAddInSetup project, and will focus exclusively on the EmailAddIn project.

As you can see in the above screen shot, the wizard generates a single .cs file that contains all the code necessary to handle the add-in execution. (The AssemblyInfo.cs file simply contains assembly level attributes. Attributes are annotations to the code that can be used to set properties at a class, assembly or method level). The Connect.cs file is the file that contains a single object that represents the add-in.

namespace EmailAddIn
{
using System;
using Microsoft.Office.Core;
using EnvDTE;
using System.Runtime.InteropServices;
public class Connect : Object, IDTExtensibility2, IDTCommandTarget
{
public Connect()
{
}

/// <summary>
/// Implements the OnConnection method of
/// the IDTExtensibility2 interface.
/// Receives notification that the Add-in is being loaded.
/// </summary>
public void OnConnection(object application,
ext_ConnectMode connectMode,
object addInInst, ref object[] custom)
{
applicationObject = (_DTE)application;
addInInstance = (AddIn)addInInst;
if(connectMode == ext_ConnectMode.ext_cm_UISetup)
{
object []contextGUIDS = new object[] { };
Commands commands = applicationObject.Commands;
_CommandBars commandBars =
applicationObject.CommandBars;

try
{
Command command = commands.AddNamedCommand(addInInstance, "EmailAddIn",
"EmailAddIn", "Executes the command for EmailAddIn", true,
59, ref contextGUIDS,int)vsCommandStatus.vsCommandStatusSupported+
(int)vsCommandStatus.vsCommandStatusEnabled);
CommandBar commandBar =
(CommandBar)commandBars["Tools"];
CommandBarControl commandBarControl =
command.AddControl(commandBar, 1);
}
catch(System.Exception /*e*/)
{
}
}

}

/// <summary>
/// Implements the OnDisconnection method of
/// the IDTExtensibility2 interface.
/// Receives notification that the Add-in is
/// being unloaded.
/// </summary>
public void OnDisconnection(ext_DisconnectMode disconnectMode,
ref object[] custom)
{
}

/// <summary>
/// Implements the OnAddInsUpdate method of
/// the IDTExtensibility2 interface.
/// Receives notification that the collection of
/// Add-ins has changed.
/// </summary>
public void OnAddInsUpdate(ref object[] custom)
{
}

/// <summary>
/// Implements the OnStartupComplete method of
/// the IDTExtensibility2 interface.
/// Receives notification that the host application
/// has completed loading.
/// </summary>
public void OnStartupComplete(ref object[] custom)
{
}

/// <summary>
/// Implements the OnBeginShutdown method of the
/// IDTExtensibility2 interface.
/// Receives notification that the host application
/// is being unloaded.
/// </summary>
public void OnBeginShutdown(ref object[] custom)
{
}

/// <summary>
/// Implements the QueryStatus method of the
/// IDTCommandTarget interface.
/// This is called when the command's availability
/// is updated
/// </summary>

public void QueryStatus(string commandName,
EnvDTE.vsCommandStatusTextWanted neededText, ref
EnvDTE.vsCommandStatus status, ref object commandText)
{
if(neededText ==
EnvDTE.vsCommandStatusTextWanted.vsCommandStatusTextWantedNone)
{
if(commandName == "EmailAddIn.Connect.EmailAddIn")
{
status = (vsCommandStatus)
vsCommandStatus.vsCommandStatusSupported |
vsCommandStatus.vsCommandStatusEnabled;
}
}
}

/// <summary>
/// Implements the Exec method of the
/// IDTCommandTarget interface.
/// This is called when the command is invoked.
/// </summary>
public void Exec(string commandName, EnvDTE.vsCommandExecOption
executeOption, ref object varIn, ref object varOut, ref bool handled)
{
handled = false;
if(executeOption ==
EnvDTE.vsCommandExecOption.vsCommandExecOptionDoDefault)
{
if(commandName == "EmailAddIn.Connect.EmailAddIn")
{

handled = true;
return;
}
}
}
private _DTE applicationObject;
private AddIn addInInstance;

}
}

The above code segment shows the skeleton template code that is generated by the wizard. We have deleted some comments for the sake of clarity. As you can see, the generated code contains a single class named Connect . This class implements the IDTExtensibility2 interface which all add-ins implement. This interface defines the necessary methods that an object has to implement in order for the IDE to be able to load and invoke the add-in. These methods act as events within the environment and are raised by the IDE. Further, the add-in also implements the IDTCommandTarget interface which defines methods for handling named commands. A named command is a specific action that an add-in supports. In our sample add-in, we will support a single command, EmailAddIn.Connect.EmailAddIn , which will read the active file in the IDE and email it. Let us briefly discuss the methods that these interfaces define.

The following is a list of the members of the IDTExtensibility2 interface. These methods are hooks to events that the hosting application fires. This is useful for an add-in that integrates deeply within the hosting environment and needs knowledge about the state of the host

  • OnAddInsUpdate : This method is called by the application hosting the add-in when the list of currently installed add-ins changes. This happens when a new add-in is installed, or an existing add-in is unloaded. This method is fired for all the active add-ins in the host application, and can be used by an add-in to gain information about other add-ins that are active. For example, if one add-in is dependent on another, unloading of the first add-in could lead the dependent one to unload itself.
  • OnBeginShutdown : This method is called by the application hosting the add-in, when the application is being shut down. This method is only called for active add-ins, and can be used to perform clean up or to save the state of the add-in. Note that this method could be called more than once, for example: if the user aborts the shut down of the hosting application after this method has been invoked. The next time the application is attempting a shut down, the same method will be called for all active add-ins. Thus, if you have any code that is non-reentrant or is supposed to run only once, you need to guard it with some kind of a flag or mutex.
  • OnConnection : This method is called by the application hosting the add-in when the add-in is being loaded. If the add-in is marked to load on startup, this method will be called during application startup, and if the add-in is loaded by the user after the host application has started, this method will be called when the user loads the add-in. The hosting application passes in a parameter to this method that specifies how the add-in was loaded. There is a special value for this parameter that signifies the first time the add-in loads after being installed. You can use this for displaying the EULA or a welcome message, or perform any processing that is necessary during the very first loading of the add-in.
  • OnDisconnection : This method is called by the application hosting the add-in, when the add-in is being unloaded. This could be called during application shutdown, or could be called if the user explicitly unloads the add-in from the Add-in manager. During application shutdown, the OnBeginShutdown method is called as well. However, the OnBeginShutdown method is called before the OnDisconnection method. The OnDisconnection method is generally used to perform cleanup functions or to save the state of the add-in.
  • OnStartupComplete : This method is called by the application hosting the add-in when the hosting application has finished its startup procedures. Note that this method is called only when the add-in is marked to load on application startup. If the add-in is explicitly loaded by the user via the toolbar, or the Add-in manager, this method is not called.

Note that the IDTExtensibility2 does not declare any methods for the actual processing of the add-in. These methods that contain the actual add-in logic are declared by the IDTCommandTarget interface. This interface declares methods for handling named commands. This interface defines two methods that should be implemented by the add-in.

  • QueryStatus : The QueryStatus method is used to communicate the status of the add-in to the hosting application.
  • Exec : The Exec method is called by the hosting application, when the add-in is invoked. Since a single add-in can support multiple named commands, the hosting application passes the name of the currently invoked command to the Exec method. The Exec method checks the name of the command being passed in, and if the command is recognized and supported by the add-in, it performs the necessary processing.

Now that we have explained the process of using the Add-in wizard to create an add-in, and discussed the anatomy of an add-in, let us discuss the code behind the Email Add-in.

Email Add-In

The email add-in is a very simple add-in that demonstrates how we can use add-ins to extend the development environment. The entire add-in consists of less than 25 lines of actual code (there is a lot of boiler plate code that is generated by the wizard). Let us start with examining the Exec method of the add-in, which is the method invoked when the user invokes the add-in.

public void Exec(string commandName,
EnvDTE.vsCommandExecOption executeOption,
ref object varIn,
            ref object varOut, ref bool handled)
{
handled =
            false;
if(executeOption
            ==
envDTE.vsCommandExecOption.vsCommandExecOptionDoDefault)
{
if(commandName
            == "EmailAddIn.Connect.EmailAddIn"){
Document lDoc;
lDoc =
            applicationObject.ActiveDocument;

MailForm lForm = new
            MailForm (lDoc.FullName);
lForm.Show ();

handled =
            true;
return;
}
}
}

As you can see from the code segment above, the Exec method is fairly simple. We check if the named command matches that of our add-ins, and if so, we retrieve the full name and path of the active document.

public void OnConnection(object application,
ext_ConnectMode connectMode,
object addInInst, ref object[] custom)
{
applicationObject = (_DTE)application;
// other code omitted for clarity
}
private _DTE applicationObject;

In order to retrieve the document name, we need access to the development environment's automation model (object model). The top-level element of this automation model is passed by the hosting application to the add-in as a parameter to the OnConnection method. As you can see from the above code segment, we cache the top-level element in a private data member called applicationObject . This object is valid throughout the life of the add-in, and can be used in other methods of the add-in. Thus, in the Exec method of the add-in, we use this applicationObject and invoke the ActiveDocument property to gain access to the currently open document in the IDE. Once we have the location of this document, we can invoke our MailForm to send the document as an attachment.

The MailForm form is as shown above. The name and path of the attachment is passed to the MailForm constructor, and the user can enter the other fields to send the document as an email attachment. The code for the MailForm is shown below.

private void btnSend_Click(object sender,
            System.EventArgs e)
{
try{
MailMessage lMail = new
            MailMessage();

lMail.To = txtTo.Text;
lMail.Cc =
            txtCc.Text;
lMail.From = txtFrom.Text;
lMail.Subject =
            txtSubject.Text;
lMail.Body = "Attachment using
            EmaillAddin";
lMail.Attachments.Add(new
            MailAttachment(txtAttachment.Text));

SmtpMail.Send(lMail);
}
catch
            (Exception lException){
MessageBox.Show
            (lException.Message);
}

this.Close ();
}

We use the MailMessage and SmtpMail classes from the System.Web.Mail namespace to access the SMTP email services. Using these classes is fairly simple as you can see above. The first thing you need to do is add a reference to the System.Web dll which contains the above namespace. Then, we create a new instance of the MailMessage class and set the different properties on this class corresponding to the standard email fields. Once this object is created, we use the static Send method of the SmtpMail class to send the SMTP mail. If there is an error in the send, an exception is thrown, and we simply display the exception message in a message box.

In order to install this add-in, you need to register the add-in such that VS .NET recognizes the component as an add-in and loads it. This is done by adding certain entries in the registry as shown below. For our sample add-in, the attached zip file contains a file named Install_EmailAddIn.reg . Merging the contents of this file with that of your local machine's registry will create the required registry entries. Note that the Add-In Wizard automatically performs the registration, but if the Add-In wizard is not used then this type of registration is required.

The above screen shot displays the add-in and some of its properties. For example, the CommandLineSafe value indicates that the Add-in was designed to avoid displaying a user interface when invoked by the VS .NET command line.

Once the add-in is installed, you can use the Add-in manager to load it, as shown in the following screen shot.

Conclusion

Extending any development environment can greatly enhance developer productivity by customizing and automating tasks and processes. Visual Studio .NET greatly enhances the add-in model by providing easy access to the entire automation model, and by providing the add-in wizard to generate the plumbing code necessary to hook the add-ins into the development environment. In this article, we explained the fundamentals behind VS .NET Add-ins and wrote a very simple add-in to demonstrate the concepts. In Part 2 of this article, we will develop an add-in that adds an AutoSave feature to the IDE, where all the open documents will be saved periodically, and another add-in that syncs up the contents of the VS .NET task list with that of Outlook's.


Windows Forms User Interface Design with C#

Windows Forms User Interface Design with C#
http://www.csharptoday.com/content.asp?id=1570
Article Rating: 3.5
Published on September 6, 2001
Abstract
Microsoft’s .NET Framework has provided developers with a great new way to easily develop powerful applications. In this article, Nikhil Dabas shows how to build several GUIs with Windows Forms.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

Microsoft's .NET Framework has provided developers with a great new way to easily develop powerful applications. Windows Forms are but a small part of the framework, but vast nonetheless. This article explains how to build user interfaces using this new programming model. Windows Forms natively supports several of the 'traditional' GUI types, such as dialog, SDI (Single Document Interface) or MDI (Multiple Document Interface). Other types of GUI, like an Explorer-style interface or a wizard, are easily implemented with Windows Forms. It is also possible to give your application the 'cool' look with skins. This article explains how to build all these types of GUIs with Windows Forms.

The Windows Forms Application Model

In any Windows Forms application, you will be deriving your application's GUI class from the Form class (System.Windows.Forms namespace). This class is a truly powerful and flexible class, because it can be bent and twisted to implement all of the GUI types mentioned above. Essentially, if you've not read the basics already, here is how you make any kind of GUI:

  1. Derive your application's GUI class from System.Windows.Forms.Form .
  2. Add all user interface elements to the form in its constructor.
  3. Add a Main member function, with the STAThreadAttribute attribute on it, to allow the application to be run as an executable.

That's it! Just compile and run your application. These steps are, however, just an example, and there are several approaches to this task.

Let's see how to implement each of the GUI types mentioned above.

Dialog or SDI Applications

Read the heading carefully: Dialog or SDI. They are essentially the same, with almost no difference whatsoever. When you derive from Form, you get a dialog-based application. If you have Visual Studio.NET, you just put a few controls on the form, and you're done. I'll be exploring the SDK, or Notepad way of doing things, however. The world's simplest dialog, that does absolutely nothing, is not a work of art, but it shows you how easy it is to use Windows Forms:

namespace HelloWorldTest
{
using
            System;
using System.Windows.Forms;
using
            System.Drawing;

class HelloDialog :
            Form
{
[STAThread]
public static void Main(string[]
            args)
{
Application.Run(new HelloDialog());
}

public
            HelloDialog()
{
Button but = new Button();
but.Text =
            "Click Me";
but.Location = new Point(74, 75);
but.Size = new
            Size(60, 40);
but.Click += new
            EventHandler(Button_Click);

this.Controls.Add(but);
}

protected
            void Button_Click(object sender, EventArgs
            e)
{
MessageBox.Show("You
            clicked!");
}
}
}

As is obvious, the HelloDialog class has been derived from Form, and a single button is added to the form in its constructor. To compile this, just type: csc Dialog.cs at a command prompt.

Building an SDI application couldn't be easier, as Windows Forms provides two great properties on all controls, Anchor and Dock , which allow the framework itself to dynamically compute positions and sizes of controls on a form. A brief overview.

Anchoring

When you anchor a control to its containing control, you can specify which of its edges are always at a fixed distance from the specified edge of the parent control. For example, if you specify that a control is anchored to the right edge of the form ( AnchorStyles.Right ), and the form is resized, the control shall move as the form is resized. That is, the distance between the control's right edge and the parent form's right edge will always remain the same.

Docking

When a control is docked, the control's specified edge or edges will be docked, or attached, to the parent form. For example, if you specify that a control be docked to the bottom of the form ( DockStyle.Bottom ), the control's bottom edge will be sized and aligned to fit its parent form.

The humble Notepad application is a perfect example of this. Even though it is not a .NET application, it does what any Windows Forms application would with the Dock property. The textbox displayed by notepad is docked to all edges of the windows, so it sizes to use all available client area. The same effect can be achieved in Windows Forms by setting the Dock property of a control to DockStyle.Fill .

So, back to the SDI application I was talking about. All you have to do is add whatever control you want as the document, and set its Dock property to DockStyle.Fill . The control will fill all the available client area on the form, thus giving it the appearance of an SDI application. For an example, see SDI.cs .

MDI Applications

Depending on your application, you might want to use an MDI instead of an SDI, especially if your application is going to open more than a few documents at once and you don't want to clutter the user's taskbar. As stated earlier, you have to derive a class from Form. The MDI parent form must have its IsMdiContainer property set to true. You should then set the MdiParent property of all child forms to an instance of the parent form. You can show these child forms like any other form, by setting its Visible property to true or calling the Show method. You can design these forms like any other forms, there are no special techniques involved. However, you should only add controls like toolbars and status bars to the MDI parent form.

If you look at the Window menu of any MDI application, you will see a list of the currently open documents below the commands for arranging the windows. Adding such a list is easy, all you need to do is set the MdiList property of the Window menu to true . This adds a list of all open MDI child windows to the end of this menu. An example has been provided in MDI.cs .

Explorer-style Applications

An explorer-style application should have two different controls that display two different views, and some way to resize the border between them. Implementing this is easy in Windows Forms. The two controls can be any controls that you would normally use on a Form. The resizing is handled by a Splitter control. You have to dock all three controls so that they align properly. Set the Dock property of both the left-side control and the splitter to DockStyle.Left , and then set the Dock property of the right-side control to DockStyle.Right . For an example, see Explorer.cs .

Wizards

A wizard is an essential part of any commercial application today. Windows Forms, however, have no built-in support for wizards. To make a native implementation of a Wizard '97 style wizard (that is, without using PInvoke or COM Interop), I have created two classes that allow you to create a wizard as easily as another Windows Forms application.

The WizardForm class

The WizardForm class represents the wizards itself. This class is therefore derived from Form. This class manages the wizard buttons and other controls, as well as all the pages in the wizard. The major element here is the Pages property, as this allows you to add the actual pages to the wizard. As can be seen, this property is of type PageCollection , which itself is derived from Form.ControlCollection . This has been done so that you can use the Pages property just like the Controls property in a Form, with "add" and "remove" member functions. The ShowWizard method does exactly what it sounds like, it shows the wizard! The Enable XXX methods enable or disable the appropriate button on the wizard. The WizardButtonClick event is raised whenever the user clicks a button on the wizard. This event can be cancelled, so that the wizard will not move to the next page.

The CurrentPage property gets or sets the currently displayed page in the wizard.

The class itself does nothing earth shattering. All pages are stored in the Pages collection, and clicking the "next" or "back" buttons swaps the current page with the appropriate page. You can also set the CurrentPage property to another page in the Pages collection, and the specified page is swapped in or out of the Pages collection. Internally, all swapping is done through the CurrentPage property's set member. Because of the way the Form.ControlCollection class is implemented, the new page must be taken out of the Pages collection before it can be added to the wizard as another control. The index of the page in the Pages collection is stored, so that when it is put back in the Pages collection it is stored at the correct index. This allows you to correctly access a page in the Pages collection by index.

As per Wizard 97 specifications, a panel must be displayed with a title and subtitle for the current page on top of the page. The title and subtitle properties are picked up from the page and set on the wizard form whenever the current page is changed. This panel, however, is not to be displayed on the "welcome" and "completion" pages. A check is done for this and the panel is displayed only if the new page is an interior page.

The WizardPage class

All pages in the wizard are derived from the WizardPage class. This class itself is derived from ContainerControl . The System.Windows.Forms.Form class is also derived from ContainerControl , so we get most of the functionality required in a wizard page ready-made for us. This class is much simpler than the WizardForm class. The PageType property gets or sets the type of the wizard page: Interior, Welcome, or Completion. These types are contained in the WizardPageType enumeration. The read-only property Wizard allows you to access the WizardForm this page is associated with. The Title and Subtitle properties set the title and subtitle for the page respectively. The Activated and Deactivate events are raised whenever the page is made the current one, or is swapped out. The Deactivate event can be cancelled, so the page remains the current page if you cancel this event.

Using the classes

Using the classes couldn't be simpler! You have to design your wizard pages like any other forms, but deriving them from WizardPage . You can also derive a class from WizardForm , though that is not necessary. Once you have class instances of all your pages and the WizardForm , you have to add the pages to the wizard's Pages collection. Here is some sample code from the SampleWiz.cs example:

    // The class
            constructor
public SampleWizard()
{
page1 = new
            WizPage1();
page2 = new WizPage2();
page3 = new
            WizPage3();
page4 = new
            WizPage4();

this.Pages.Add(page1);
this.Pages.Add(page2);
this.Pages.Add(page3);
this.Pages.Add(page4);
}

In this example, a class is derived from WizardForm because the wizard is the main user interface of the application. You would normally not need to do this because a wizard is a helper, not the main GUI. To show the wizard, you just have to call the ShowWizard method. This method returns a variable of type WizardResult , which tells you how the wizard was dismissed. It can be either WizardResult.Cancelled or WizardResult.Finished : the meanings are obvious.

Skins

Skins can give an application a high degree of "coolness". Every application has its own unique GUI, and they are so varied that a generalized class for handling skinning cannot be easily built. That is, if you want to support skins in your application, you will need to implement the entire UI yourself, providing little extensibility or flexibility.well, not exactly. Obviously, you can derive classes for GUI elements from what the .NET Framework provides. Implementing skins is not an atomic task. A skin implementation has many aspects, depending on your application. Most applications will require non-rectangular forms as well as custom GUI elements.

Non-Rectangular Controls

Windows Forms has made it easy to make non-rectangular windows. Every class derived from System.Windows.Forms.Control has a Region property, which allows you to define any shape for the control. This property is of type System.Drawing.Region , which can be constructed with a System.Drawing.Drawing2D.GraphicsPath as an argument. It is easy enough to create a GraphicsPath with whatever shapes you want. Here is an example, from Shaped.cs :

GraphicsPath ellipsePath = new GraphicsPath();
ellipsePath.AddEllipse(2, 22, 300, 100);
this.Region = new Region(ellipsePath);

This code simply creates an elliptical form. You can do the same for any other control as well, like a button. There are scenarios, however, where creating a path is not easy. You might, for example, have a very irregularly shaped bitmap. Of course, you could spend a long time tracing the path, or have some software do it. Nevertheless, there is another way to do this: set the TransparencyKey property of the form to your bitmap's background color. This way, even though the form is still rectangular, it looks like it is shaped to the bitmap. However, this method is not the accepted way of doing things, and is inefficient for this purpose. You should do this only as a last resort, when creating a path is impossible.

Custom Controls

There are several ways to make the .NET Framework controls look skinned. You can, of course, set their clipping regions and make them non-rectangular. Other than that, you can set a background image for the form itself and display a picture on controls instead of text. Sometimes, however, you will almost certainly have to handle the Paint event and handle the drawing yourself.

Custom Menus

A menu item is not another type of control, so you will have to owner-draw it if you want to skin it. You have to set the OwnerDraw property of a menu item to true and then handle the DrawItem and MeasureItem events to actually draw it.

Conclusion

Windows Forms offer almost everything a developer might want to create a graphical user interface for an application. As this article clearly demonstrated, creating a familiar-looking and consistent GUI is simple and easy with Windows Forms. This article also demonstrated the capabilities of the System.Windows.Forms.Form class, a truly powerful and versatile class. Any kind of GUI design is possible with this class. Windows Forms have many more features, which are not covered in this article. This framework has support for dozens of controls, like status bars, menu bars, and almost any other control you would normally use.


Creating a .NET assembly in C#

Creating a .NET assembly in C#
http://www.csharptoday.com/content.asp?id=1563
Article Rating: 4
Published on September 3, 2001
Abstract
In this article, Dinar Dalvi looks at assemblies in detail. He guides us as we learn about writing assemblies, signing them, adding them to the global cache, and how to write applications that use these assemblies. Along the way, he goes into detail about every aspect of assemblies.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

Microsoft .NET framework has introduced new features aimed to simplify application deployment and solve DLL hell. Developers are familiar with the versioning and deployment issues with DLL's. Many of you will have spent time with regedit.exe trying to find why the DLL suddenly stopped working after someone installed some application on the machine. .NET uses assemblies to solve versioning and deployment problems. The main problem with COM/COM+ components was with maintaining dependencies and versioning problems. The versioning problem exists in the DLLS, as it is not clear which version of the DLL an application needs. The DLLS does not track and enforce the dependencies. The COM DLL's seemed to solve this problem by implementing a separation of the interface and the implementation, but then the changes in the implementation broke the existing applications.

Installing a new application caused some old applications to break, as the DLLs were overwritten: sometimes by a newer version of the DLL, sometimes by the older version of the DLL. The common effect: to crash a running application. To tackle this problem, Windows 2000 came up with the concept of side-by-side execution. Windows 2000 protects the system files so they are not overwritten. However, if you look at the side-by-side model for a COM component the question arises of how to uninstall one of the versions. Will the registry entries go away? And what about the threading configuration? One DLL can be an apartment thread, while the latest one can be compiled as a free threaded component. Well, as always, the last version installed is the default one. It overrides the previous versions. This happens because the configuration for a COM component is not stored in the component DLL. Dependencies are not tracked under the traditional DLL architecture. .NET's answer to these problems is Assemblies.

What is an assembly?

Assemblies are the building blocks of .NET Framework applications. They form the fundamental unit of deployment, version control, reuse, and security permissions. Assemblies are self-describing through metadata called a manifest. An assembly is a collection of types and resources that are built to work together and form a logical unit of functionality. An assembly provides the common language runtime with the information it needs to be aware of type implementations. An assembly contains code that the runtime executes. Each assembly can have only one entry point. An assembly can be a single file or a number of files.

Let us spend some time to see what an assembly is not. You cannot deploy an application as an assembly. Assembly is a place for the class deployment. A typical assembly is shown below:

An assembly manifest

Assembly Manifest consists of the assembly name, version number, culture, and strong name information making up the assembly's identity. To view the contents of an assembly manifest, programmatically use the System.Reflection class.

Information | Description
Assembly name | Assembly identity or text holds the assembly's name.
Version number | A major and minor version number, and a revision and build number. These numbers are used by the runtime to enforce the version policy.
Culture | Information on the language the assembly supports. This information must be used to designate an assembly as a satellite assembly containing culture or language-specific information. Note: An assembly containing culture information is automatically considered to be a satellite assembly.
Strong name information | If the assembly is signed with a strong name the manifest contains the public key from the creator.
List of all files in the assembly | A hash and the filename for each file contained in the assembly. This happens if an assembly consists of more than one file. Note: All files that make up the assembly must be in the same directory as the file containing the assembly manifest.
Type reference information | Information used at runtime to map a type reference to the file that contains its declaration and implementation. This is used for types that are exported from the assembly.
Information on referenced assemblies | Names and hashes of the other assemblies that this assembly in turn needs to reference

Assembly manifests can be viewed by using the utility ILDASM.EXE.There are many ways to create assemblies. One way is to use development tools, like Visual Studio.NET. You can use tools provided in the .NET Framework SDK to create assemblies with modules created in other development environments. CLR (common language runtime) APIs, such as Reflection.Emit, can be used to create dynamic assemblies.

I have created a simple assembly with just HelloFoo in it. Here is the Code:

using System;

namespace
            Foo
{
public class HelloFoo
{
public
            HelloFoo()
{

}
}
}

I used Notepad to write this code. Then we compile this from the command line using the following syntax.

csc /t:library Foo.cs

Let us look at the manifest for the above assembly.

In the manifest, the assembly mscorlib is referenced. Mscorlib is one of the key components for the CLR. Mscorlib contains all the System libraries.

Private assembly

Many developers will be happy developing the private assemblies. There is no need for special management, registration and versioning. When you compile a C# program the assembly produced, by default, will be a private assembly which, by name, suggests that it is going to be used by one application. This assembly is not visible to any other application in the file system. If there is an application that uses many private assemblies then all those assemblies will have to be distributed along with the application. Creating a private assembly is easy since there is no worry of any naming problems. The only thing to remember is that the name must be unique in your application.

Please note that the private assembly must be placed in the same folder (or a subfolder) on the Windows file system as the calling application. One other thing with private assemblies is that there is a scope for versioning problem. If a component in your application uses version 2 for the assembly, but your application uses version 3, then the question arises of which version is used. The answer to this is the one you have referenced in your program. This kind of problem must be solved during the development stage.

Shared assembly

A shared assembly is one that is used by multiple applications. Shared assemblies have some additional requirements to solve the sharing problem. Why do we need a shared assembly? Say you have written a generic library, or perhaps some WinForms / WebForms controls which you want to sell. If the assembly needs to be shared then the naming requirements are stricter. The name must be unique across the entire system.

Shared assemblies are typically deployed in the global assembly cache. The global assembly cache is a machine-wide store for shared assembly, which is found in WINNT\assembly folder.

Create cabinet files

Cabinet files (or CAB files) can be created with the Visual Studio.NET deployment as well, or alternatively by using the Win32 SDK compression tool. You can compress one file or a complete directory for deployment or download. As the CAB file is in a compressed format it is faster to download the file.

Deploying an assembly into the global assembly cache

Use the installer called the global assembly cache tool ( Gacutil.exe ) provided by the .NET Framework SDK to install the assembly in the global assembly cache.

Notes:

  • In deployment scenarios, Windows Installer 2.0 must be used to install assemblies into the global assembly cache.
  • Use Windows Explorer or the global assembly cache tool only in development scenarios because they do not provide assembly reference counting and other features provided by the windows installer.
  • Administrator privileges are needed to install assemblies into the global assembly cache.

Assemblies deployed in the global assembly cache must have a strong name. When an assembly is added to the global assembly cache, integrity checks are performed on all files that make up the assembly. The cache performs these integrity checks to ensure that an assembly has not been tampered with (for example, when a file has changed but the manifest does not reflect the change).

Strong Names

Sn.exe , or Strong Name Utility, generates a public/private key pair. For our discussion I won't be explaining the public key private key concept, but I will emphasize one thing: to share the assembly the assembly must be signed by the public/private key. The compiler writes the public key in the manifest and creates a hash for all files belonging to the assembly and signs the hash with the private key.

Note: While distributing the assembly we never give away the private key, but we always distribute the Public Key. We will generate a temporary key for testing purpose, but when distributing the assembly we can replace the key with the real company key, probably by getting a key from a signing company.

A strong name consists of the following:

  • Assembly identity
  • Assembly text name
  • Assembly version number
  • Culture information (if provided)
  • A public key
  • A digital signature.

Assemblies with the same strong name are expected to be identical. Strong names guarantee name uniqueness by relying on unique key pairs. No one can generate the same assembly name that you can, because an assembly generated with one private key has a different name than an assembly generated with another private key. Strong names provide a strong integrity check. However, it is worth noting that strong names do not imply the level of trust as provided by digital signatures. Earlier we created a DLL called foo.dll . Now let us look at the manifest for the same DLL after signing it with a strong key.

If you look closely you will see that the public key is stored in the manifest.

Installing an assembly into the global assembly cache.

The following command installs an assembly in the GAC:

gacutil -i <assembly name> (where <assembly name> is the actual assembly name.)

Say we have an assembly called myAssembly.dll , the following command installs the myAssembly.dll into the global assembly cache.

gacutil -i myAssembly.dll

To view the list of assemblies in the global cache, type the following command:

Gacutil -l

To remove the assembly from the global cache, use this command:

Gacutil -u <file name>

All Visual Studio compilers are capable of producing assemblies. If we have one or more modules (metadata without a manifest), Al.exe (which is provided by the framework) can be used to create an assembly with the manifest in the separate file.

In traditional windows programming we wrote reusable code in the form of a DLL or a COM component. When the C# code is compiled, the compiler does not emit assembly language, but Intermediate Language. In fact the compiler does more, it emits IL packaged into an assembly. The same thing happens when we compile the code into the library. So the assembly can be a DLL, .EXE file, or combination of both.

In this article we will:

  • Build a simple C# assembly.
  • Generate a secure key for signing the assembly.
  • Install the assembly in the GAC.
  • Write client applications to use the assembly.

Requirements

For our application I have used Visual .NET Beta2 installed on Windows 2000 server.

Those of you who are still using .NET Beta1 will need to switch to the Beta 2 environment as the namespaces have been modified. For example, NamesSpace System.WinForms is now System.Windows.Forms .

Building C# assembly.

From the start menu open the Visual Studio.NET IDE environment. Then, from the file menu, select the New Project option. Then select C# Projects and choose Class Library from the templates and name the project as MyAssembly .

This will open a blank project with only a namespace and the constructor in it.

We then add the following code to the project.

using System;

namespace
            MyAssembly
{
/// <summary>
/// Summary description
            for MyAssembly.
/// </summary>
public class
            Class1
{
public Class1()
{
public string RetStr(string
            str )
{
return
            (str.ToUpper());
}

}
}
}

We are not doing anything fancy here, we are simply going to convert the string to upper case and return it to the calling program. You can use Winforms or a program to get the data from the database.

Let us now generate a cryptographic key using sn.exe .

Usage Syntax: SN [-q(uiet)] <option> [<parameters>]

Sn.exe , which is shipped with the .NET Framework, has many options. To see all the options and the description for the options type the following command on the command prompt.

Usage Syntax: sn -h

Examples

The following command creates a new random key pair and stores it in mykey.snk .

sn -k mykey.snk

The following command stores the key in mykey.snk in the container MyContainer in the strong name CSP.

sn -i mykey.snk MyContainer

The following command extracts the public key from mykey.snk and stores it in myPublicKey.snk .

sn -p keyPair.snk myPublicKey.snk

The following command verifies the assembly myDll.dll .

sn -v myDll.dll

The following command deletes MyContainer from the default CSP.

sn -d MyContainer

In this scenario, the public key is set at compile time and signing is performed later when the private key is known.

We can name the key file whatever we want, but for our example we will call it myAssembly.key .

With this we have generated the key in the project folder. Now we will assign the key to our assembly.

Open the file called AssemblyInfo.cs , which has the code to control the assembly info. The Visual Studio.NET wizard generates this code:

using System.Reflection;
using System.Runtime.CompilerServices;

//
// General Information about an assembly is controlled through the following
// set of attributes. Change these attribute values to modify the information
// associated with an assembly.
//
[assembly: AssemblyTitle("")]
[assembly: AssemblyDescription("")]
[assembly: AssemblyConfiguration("")]
[assembly: AssemblyCompany("")]
[assembly: AssemblyProduct("")]
[assembly: AssemblyCopyright("")]
[assembly: AssemblyTrademark("")]
[assembly: AssemblyCulture("")]

//
// Version information for an assembly consists of the following four values:
//
// Major Version
// Minor Version
// Build Number
// Revision
//
// You can specify all the values or you can default the Revision and Build Numbers
// by using the '*' as shown below:

[assembly: AssemblyVersion("1.0.*")]

//
// In order to sign your assembly you must specify a key to use. Refer to the
// Microsoft .NET Framework documentation for more information on assembly signing.
//
// Use the attributes below to control which key is used for signing.
//
// Notes:
// (*) If no key is specified, the assembly is not signed.
// (*) KeyName refers to a key that has been installed in the Crypto Service
// Provider (CSP) on your machine. KeyFile refers to a file which contains
// a key.
// (*) If the KeyFile and the KeyName values are both specified, the
// following processing occurs:
// (1) If the KeyName can be found in the CSP, that key is used.
// (2) If the KeyName does not exist and the KeyFile does exist, the key
// in the KeyFile is installed into the CSP and used.
// (*) In order to create a KeyFile, you can use the sn.exe (Strong Name) utility.
// When specifying the KeyFile, the location of the KeyFile should be
// relative to the project output directory which is
// %Project Directory%\obj\<configuration>. For example, if your KeyFile is
// located in the project directory, you would specify the AssemblyKeyFile
// attribute as [assembly: AssemblyKeyFile("..\\..\\mykey.snk")]
// (*) Delay Signing is an advanced option - see the Microsoft .NET Framework
// documentation for more information on this.
//
[assembly: AssemblyDelaySign(false)]
[assembly: AssemblyKeyFile("../../MyAssembly.key")]
[assembly: AssemblyKeyName("")]

If you are using an IDE to sign an assembly with a strong name, you need to understand where the IDE looks for the key file. The current directory can change frequently when developing using an IDE, and the output directory can often be a dynamically created directory. When using an IDE such as Visual Studio.NET, you should put the key file in the project directory.

Now we compile the assembly. After compilation we need to add the assembly to the GAC.

Usage : gacutil [options] [assembly]

Gacutil has a few options to choose from. To view all the options type

gacutil /?

We add the assembly into the GAC by typing:

Gacutil -I MyAssembly.dll

The picture below shows that our assembly was added into the GAC.

Assembly Versioning

Before we proceed let us quickly discuss assembly versioning. Each assembly has a four-part version number (1.0.0.0) as part of its identity. Developers and the administrators must understand how the version numbers work, as it is the key to how the CLR enforces version dependencies. The version number is represented as a four-part number with the following format:

<major version>.<minor version>.<build number>.<revision>

Any changes made to the major and minor versions are considered to be incompatible changes.

Example: Say we remove some types or functions altogether from the assembly. The class loader uses this information so that the incompatible versions of that assembly are never loaded by default.

On the other hand the bug fixes etc are considered as compatible versions.

The CLR determines which version of the dependent assembly to load. By default, the assembly that is loaded to resolve the reference must have the same major and minor numbers as were recorded in the reference. If the numbers don't match then the assembly is considered as incompatible, and is therefore not loaded by default. The CLR will pick the assembly with the highest build or revision number.

For example, suppose our assembly referenced another assembly called myRefAssembly . If our assembly MyAssembly was built with the version 1.1.0.0 for myRefAssembly, but version 1.1.0.1 was found at runtime for myRefAssembly , then the version 1.0.0.1 will be loaded and not version 1.0.0.0. The version number for the referenced assembly is stored in the manifest for the referencing application.

As we have compiled the assembly, signed the assembly and installed the assembly into the GAC, let us start building the client applications to access our assembly.

Assembly Clients

VBClient

Open the .NET environment and choose a VB Windows Applicaton. Add 2 text boxes and one command button to the form and the screen should look like the picture below.

Now we will add the assembly to our project. Click on the Project option and choose Add Reference . Browse to the folder where you compiled the assembly and add the assembly to the project reference. Now that the assembly is added to our application we add the following code to the button click event.

Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button1.Click
Dim oMyAssembly As New MyAssembly.Class1()
TextBox2.Text = oMyAssembly.RetStr(TextBox1.Text)
End Sub

Now we compile and run our application.

Csharp Client

Let us write the client code in C#. Open the .NET environment and choose the C# Windows Application and add the following code:

using MyAssembly;

namespace
            CCharpClient
{
/// <summary>
/// Summary description
            for Form1.
/// </summary>
public class Form1 :
            System.Windows.Forms.Form
{
private
            System.Windows.Forms.TextBox textBox1;
private
            System.Windows.Forms.TextBox textBox2;
private
            System.Windows.Forms.Button button1;
/// <summary>
///
            Required designer variable.
/// </summary>

private
            void button1_Click(object sender, System.EventArgs e)
{
Class1
            a = new MyAssembly.Class1();
textBox2.Text = a.RetStr
            (textBox1.Text);
}
}
}

After we run the compiled application the screen looks like this:

A few things the developer must know about the assemblies.

  • An assembly can be considered as a logical DLL.
  • An assembly contains the code, which will be executed by the runtime; there can be only one entry point for the assembly.
  • The assembly contains the permissions that are requested and granted for the application to run.
  • The manifest has the rules, which are used to resolve the types and resource needed. The manifest also contains if there are may other dependent assemblies.
  • An application when executed calls only the needed assemblies.
  • Multiple assemblies with same name can be executed side-by-side.

Application Domains

What is an application domain? Well it is a virtual boundary to isolate an application. In other words, all objects created within the application scope are created within the application domain. Now the question arises: can there be one or more application domains? Yes, and one or more applications can run in the single application domain. A process runs only one application, and one process can have multiple application domains.

Let us look at the picture below:

An assembly is loaded into an application domain before getting executed. As we see in the picture, we have a process that has 2 domains.

In this instance, assembly A for the domain assembly A is loaded, while the Instance B needs Assembly A and Assembly B to be loaded.

Conclusion:

Today we learnt how to:

  • Write an assembly
  • Sign the assembly using Sn.exe
  • Add the assembly to the Global Cache using gacutil.exe .
  • Write an application to use the assembly.

We also learnt that:

  • Assemblies are self-describing. The metadata, which is included in the assembly, describes the assembly.
  • Version dependencies are stored in the assembly.
  • Assemblies can be loaded side-by-side.
  • Assemblies use application domain for security and isolation.
  • Installation for the assembly is very easy. Xcopy , or a simple drag and drop, is enough to transfer the assembly to a different machine.

We also learnt that: assemblies are the basic part of the .NET framework, and are designed to simplify application deployment and to solve versioning problems that can occur with component-based applications. Assemblies are self-describing components. Assemblies have no dependencies on registry entries; that they enable zero-impact application installation. Assemblies simplify uninstalling and replicating applications. Assemblies help to avoid "DLL Hell" by solving versioning problems. We saw that private assemblies must be distributed with the application. We also saw that we need to add the assembly into the GAC if we want other applications to be able to share it. To distribute the assembly in the production environment use Windows installer 2.0 rather than using Xcopy for deployment.


Building an Online Shopping Cart using C# and ASP.NET Part 1

Building an Online Shopping Cart using C# and ASP.NET Part 1
http://www.csharptoday.com/content.asp?id=1552
Article Rating: 4.5
Published on August 21, 2001
Abstract
In this article Juan Martínez discusses and implements the building blocks of an online shopping cart - covering in this part item catalogs, item details, a cart and a checkout system. This acts as the foundation for further articles, where Juan covers other functionality like administration pages and setting up credit card payments.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

As web developers we are required to face a wide variety of application needs. Each web site developed is unique and furnished according to each client's specific needs. The fact is that websites are indeed fabricated for each client with different specifications but every site shares some common characteristics. Those parts of the site that share functionality features can be treated as separate applications to be reused.

In this case we will address the development cycle for one of the common blocks in today's websites - an Online Shopping Cart.

We will analyze the development of a shopping cart as a group of components described clearly before we implement them, thus allowing us to use this knowledge in areas other than ASP.NET. After a description of each component the implementation will be explained using C#.

The application will be designed to work with a SQLServer database for storage. Application logic will be done within the Web Form and presented to the user through the web browser. Core logic will reside in a separate C# component using the code behind technique. It will also be .NET framework Beta 2 compliant.

It is assumed that you have regular knowledge of the C# language, web development knowledge and database design basics.

Setting up the basis

First we will take some time to understand how a simple shopping cart works. From this knowledge we will draw some conclusions and state our requirements. From these requirements the database design will emerge.

After these steps we will have a clear path of development and be ready to implement our online shopping cart in the C# language.

Digging in the Online Shopping Cart Model

We will first take a look at a simplified diagram of an Online Shopping Cart. These are the functionality blocks to be discussed.

We then have four basic modules:

  • Item Catalog - Here we display the options to our clients in an organized way.
  • Item Details - Here we show the client as much info as we can to show off our product.
  • Shopping Cart - Here we manage the user's selected items.
  • Checkout System - Here we save the items selected and the client's information to close the transaction.

These are the basic blocks to be implemented in our online store. It includes the indispensable functionality that will be described in detail later. These blocks could be enriched further with more features, which will be covered in later articles.

Defining requirements

As in every software development cycle, we need to define our requirements first so we can design software capable of giving satisfaction to our customers.

Our online shopping cart application should do the following:

  • Have a list of categories and subcategories.
  • Items should be arranged in its corresponding subcategory.
  • Items could be selected for category and home promotion.
  • Each product should have an id, name, short and long descriptions, small and large images, stock and price.
  • Users should be able to add products to the basket and remove them.
  • The user should be given an order number and will be able to track it through an order tracking system.

Generating our database model

From our requirements we define the database schema. The tables are shown as a conceptual model, with all tables used in this version of the shopping cart.

The tables are grouped as follows:

  • Green - The catalog part of our application.
  • Sky blue - Item details.
  • Orange - Shopping cart basket.
  • Yellow - The checkout system.

Catalog Section

The green part corresponds to the catalog section. It is composed of two tables. This is a very simple arrangement in which the subsection table inherits the section Id. This way we can display the items in a section/subsection approach. As this approach will work in most cases, you could improve this to a completely flexible design using recursive sections, which will inherit a parent Id.

Item Details

The item details part of our model is a trimmed down version of an item details design. We have only one table in which we save the vital information of the item such as description, price and images. We have a couple of Boolean values that are used to specify if the item should be displayed as a home or section item.

Shopping cart basket

The shopping basket is a simple table that is used temporarily to store the items selected by the user. We just save the session Id, the item id and the quantity. We will implement this as an in-memory data table and hold our selected items in a session variable. This table is shown to show you're the information that needs to be retained during our session.

Checkout system

This is the most complicated part or our system. Upon checkout we create an order Id in the order table. Then the items stored in the shopping basket are transferred and saved in our order Items table. The user information is stored in the order details table. These two tables inherit the order Id. We have a fourth table which also inherits the order Id. This is the order Progress table and it is used for order tracking. As progress is done, the online shop administrator should add a record to this table indicating the progress done to date. This is then checked by the buyer.

Let the coding begin

As our design basis is done, we are all set to start our coding. The architecture will be based on simple Web Forms calling custom User Controls. Core logic will be done in separate components using the code behind technique.

Application framework

We start by setting our basic framework. We make use of our web.config file to save important application information such as our connection string. This is how our configuration file will look:

<configuration>
<appSettings>
<add key="connString" value="server=(local)\NetSDK;database=ShoppingCart;Trusted_Connection=yes" />
</appSettings>
</configuration>

To retrieve this information we use the following code snippet.

String connString = ConfigurationSettings.AppSettings["connString"];

As you can see, this is a very easy and convenient way to store application wise data.

Category List Block

All being set, we have our first task which is to display a list of products which will be filtered by section and subsection if selected.

The first task is to set up the workspace of our Web Forms. We set up our language and add the appropriate namespaces for our code to work.. We set C# as our language and import the System namespace for general purposes. We then add the System.Data and System.Data.OleDb since we will be using SQLServer as our database engine. We should set the debug flag to false once we deliver this application to the real world.

<%@ Page Language="C#" Debug="true" %>
<%@ Import Namespace="System" %>
<%@ Import Namespace="System.Data" %>
<%@ Import Namespace="System.Data.SqlClient " %>

We now set our references to our custom User Controls.

<%@ Register TagPrefix="SC" TagName="Site" Src="uc_header.ascx" %>
<%@ Register TagPrefix="SC" TagName="Section" Src="uc_catalog_section.ascx" %>
<%@ Register TagPrefix="SC" TagName="SubSection" Src="uc_catalog_subsection.ascx" %>
<%@ Register TagPrefix="SC" TagName="ProductList" Src="uc_catalog_product_list.ascx" %>

We reference our controls by embedding them into the aspx file.

<SC:Site runat="server" />

This will be our complete catalog WebForm, the catalog.aspx file.

<%@ Page Language="C#" Debug="true" %>
<%@ Register TagPrefix="SC" TagName="Site" Src="uc_header.ascx" %>
<%@ Register TagPrefix="SC" TagName="Section" Src="uc_catalog_section.ascx" %>
<%@ Register TagPrefix="SC" TagName="SubSection" Src="uc_catalog_subsection.ascx" %>
<%@ Register TagPrefix="SC" TagName="ProductList" Src="uc_catalog_product_list.ascx" %>
<html>
<head>
<title>Shopping Cart in C# - Catalog</title>
<link rel="stylesheet" href="style.css">
</head>
<body>
<form runat=server>
<SC:Site runat="server" />
<table width="800" cellspacing="0" cellpadding="3" border="0" style="border-color:Black;border-width:1px;border-style:solid;">
<tr>
<td width="800" align="right" bgcolor="#CCCCCC"><span class="itemTitle">Item Catalog</span></td>
</tr>
</table>
<table width="800" cellspacing="0" cellpadding="0" border="0">
<tr>
<td width="200" height="100%" rowspan="2" valign="top"><SC:Section runat="server" /></td>
<td width="600" height="10" valign="top"><SC:SubSection runat="server" /></td>
</tr>
<tr>
<td width="600" height="100%"><SC:ProductList runat="server" /></td>
</tr>
</table>
</form>
</body>
</html>

We have used four custom user controls in our first Web Form. We shall now go towards implementing these custom controls, most of which, will be reused through out the site.

Lets start by implementing the simplest of our user controls, uc_header.ascx. This file contains the "logo" of the site and a couple of links. We encapsulate this into a control since it will be repeated through the whole site. This is the code:

<table width="800" cellspacing="0" cellpadding="3" border="0" style="border-color:Black;border-width:1px;border-style:solid;">
<tr>
<td width="400"><a href="catalog.aspx"><span class="mainTitle">Shopping Cart Site</span></a></td>
<td width="400" align="right"><a href="cart.aspx">View Cart</a><span class="mainText">
| </span><a href="tracking.aspx">Track your order</a></td>
</tr>
</table>

As you can see this is just a static table, reminiscent of old style include files. Next we code a more complex user control, uc_catalog_section.ascx which displays the list of sections available on the site. This is done by binding a data retrieval codebehind class to the user control.

This is our User Control Code:

<%@ Control Language="C#" Debug="true" Inherits="CodeBehind.UcSection" %>
<%@ Import Namespace="System.Data" %>
<%@ Import Namespace="System.Data.SqlClient" %>

<html>
<head>
<title></title>
<script language="C#" runat="server"></script>
</head>
<body>

<font face="Verdana" size="-1">
<asp:DataList id="MySectionList" runat="server"
BorderColor="black"
BorderWidth="1"
GridLines="Both"
CellPadding="3"
Font-Name="Verdana"
Font-Size="8pt"
Width="200px"
HeaderStyle-BackColor="#aaaadd"
SelectedItemStyle-BackColor="Gainsboro"
>
<HeaderTemplate>
Sections
</HeaderTemplate>
<ItemTemplate>
<a href="catalog.aspx?sectionId=<%# DataBinder.Eval(Container.DataItem,"catalogSectionId") %>
&sectionIndex=<%#Container.ItemIndex%>">
<%# DataBinder.Eval(Container.DataItem, "catalogSectionName") %></a>
</ItemTemplate>
<SelectedItemTemplate>
<%# DataBinder.Eval(Container.DataItem, "catalogSectionName") %>
</SelectedItemTemplate>
</asp:DataList>
</font>

</body>
</html>

We place some more html into the control and we drop a DataList object to handle the rendering of the information. The <%# DataBinder.Eval(Container.DataItem, "catalogSectionName") %> code is used to select specific information from the bound data and put it in place.

In our codebehind class we first have to declare our DataList as protected. We then do some database work. Once we have the information we need ready from our database, we bind this data to our Web Control, the DataList.

Our User Control inherits the code behind class and that way they can work together. It is important to declare your shared variables (such as Web Form Controls) as protected so that the can be reached within the code behind class.

public class UcSection : UserControl
            {
protected DataList MySectionList;
protected Label
            MyLabel;

protected void Page_Load(object sender, EventArgs e)
            {
if (!IsPostBack) {
Bind_MySectionList();
}

if
            (Request.QueryString["sectionIndex"] != null)
            {
MySectionList.SelectedIndex =
            Int32.Parse(Request.QueryString["sectionIndex"]);
}
}

protected
            void Bind_MySectionList() {
String connString =
            ConfigurationSettings.AppSettings["connString"];
SqlConnection
            myConnection = new SqlConnection(connString);
SqlDataAdapter
            myCommand = new SqlDataAdapter("SELECT catalogSectionId,
            catalogSectionName
FROM catalogSection ORDER BY
            catalogSectionName", myConnection);
DataSet ds = new
            DataSet();
myCommand.Fill(ds,
            "catalogSection");

MySectionList.DataSource = new
            DataView(ds.Tables[0]);
MySectionList.DataBind();
}
}

We then have to implement our subsection user control. This control will do a similar task to the section control. We will just add a filter to the Query applied to the database. We will select subsections corresponding to the selected section. The rest is pretty much the same.

"SELECT catalogSubSectionId, catalogSubSectionName FROM catalogSubSection WHERE catalogSectionId= " +
sectionId + " ORDER BY catalogSubSectionName"

We then need to display our product list. We have three types of products:

  • Site Products: These are shown if no section is selected
  • Section Products: These are shown if a section is selected but no subsection is selected.
  • Normal Products: These are products that belong to the selected subsection.

We need to filter the three possibilities and create the proper query. This is done in the UcProductList class with this code. After we have our query we bind it to a DataGrid Web Control to display the items in a table arrangement.

string SQLQuery = "SELECT itemId, itemName, itemShortDescription, itemSmallImage, itemPrice, itemStock
FROM item WHERE itemPromoteSite=1 ORDER BY itemName";

int mysectionId = 0;
if (Request.QueryString["sectionId"] != null) {
mysectionId = Int32.Parse(Request.QueryString["sectionId"]);
}
if (mysectionId != 0) {
//If we have a section selected we filter products for this section
SQLQuery = "SELECT item.itemId, item.itemName, item.itemShortDescription, item.itemSmallImage, item.itemPrice, item.itemStock
FROM (catalogSection INNER JOIN catalogSubSection ON catalogSection.catalogSectionId = catalogSubSection.catalogSectionId)
INNER JOIN item ON catalogSubSection.catalogSubSectionId = item.catalogSubSectionId
WHERE (((catalogSection.catalogSectionId)=" + mysectionId + ") AND ((item.itemPromoteCategory)=1))";
}

int mysubsectionId = 0;
if (Request.QueryString["subsectionId"] != null) {
mysubsectionId = Int32.Parse(Request.QueryString["subsectionId"]);
}
if (mysubsectionId != 0) {
//If we have a subsection selected we filter products for this subsection
SQLQuery = "SELECT itemId, itemName, itemShortDescription, itemSmallImage, itemPrice, itemStock FROM item
WHERE catalogSubSectionId=" + mysubsectionId + " ORDER BY itemName";
}

This concludes our catalog presentation layer which consists of four main user controls.

  • The first control is responsible for displaying a list of sections.
  • The second control is responsible for the subsection list.
  • The third control is responsible for displaying a grid of items. This list of items is retrieved from one of three sql queries depending on the situation.
  • A final control renders the top of the page.

All the controls work together to form catalog.aspx. The user controls make use of codebehind classes for database access and bind the DataLists in our page.

Our catalog.aspx file should yield something like this:

Item Details

Our next step in building our shopping cart is to show off the item details. We first will use some of the code done for the catalog presentation before. We will take the basic html framework and three user controls. We will reuse the header, section and subsection control.

The difference here will be to replace the item list control with a new item details control. We will use a data list Web Control to display the items characteristics as well as its corresponding class in our code behind repository.

The code to do this simple task is as follows for the user control:

<%@ Control Language="C#" Debug="true" Inherits="CodeBehind.UcItemDetails" %>
<%@ Import Namespace="System.Data" %>
<%@ Import Namespace="System.Data.SqlClient" %>

<script language="C#" runat="server"></script>
<font face="Verdana" size="-1">
<asp:DataList id="MyItemDetails" runat="server"
BorderColor="black"
BorderWidth="1"
GridLines="Both"
CellPadding="3"
Font-Name="Verdana"
Font-Size="8pt"
Width="600px"
HeaderStyle-BackColor="#aaaadd"
SelectedItemStyle-BackColor="Gainsboro"
RepeatDirection = "Horizontal"
RepeatColumns = "1"
>
<ItemTemplate>
<table>
<tr>
<td valign="top">
<img src="images/<%# DataBinder.Eval(Container.DataItem, "itemLargeImage") %>" width="200" height="200">
</td>
<td valign="top">
<span class="itemTitle"><%# DataBinder.Eval(Container.DataItem, "itemName") %></span>
<br>
<span class="itemText"><%# DataBinder.Eval(Container.DataItem, "itemLongDescription") %></span>
<br><br>
<span class="itemText">Price: <%# (DataBinder.Eval(Container.DataItem, "itemPrice", "{0:c}")) %></span>
<br>
<span class="itemText">Stock: <%# DataBinder.Eval(Container.DataItem, "itemStock") %></span>
<br><br>
<a href="catalog.aspx?sectionId=<%=sectionId%>&sectionIndex=<%=sectionIndex%>&subsectionId=<%=subsectionId%>
&subsectionIndex=<%=subsectionIndex%>">Keep On Shopping!</a>
<span class="itemText"> | </span>
<a href="cart.aspx?cartAction=1&itemId=<%# DataBinder.Eval(Container.DataItem, "itemId") %>">Add To Cart</a>
</td>
</tr>
</table>
</ItemTemplate>
</asp:DataList>
</font>

And our code behind class:

protected void Bind_ItemDetails() {
int
            myItemId = 0;
if (Request.QueryString["itemId"] != null)
            {
myItemId =
            Int32.Parse(Request.QueryString["itemId"]);
}
string SQLQuery
            = "SELECT itemId, itemName, itemLongDescription, itemLargeImage,
            itemPrice, itemStock
FROM item WHERE itemId = " +
            myItemId;
String connString =
            ConfigurationSettings.AppSettings["connString"];
SqlConnection
            myConnection = new SqlConnection(connString);
SqlDataAdapter
            myCommand = new SqlDataAdapter(SQLQuery, myConnection);
DataSet
            ds = new DataSet();
myCommand.Fill(ds,
            "item");

MyItemDetails.DataSource = new
            DataView(ds.Tables[0]);
MyItemDetails.DataBind();
}

Further enhancements can be done to this simple module. This form shows the stored details of the item and can be enriched with user reviews and related item lists.

Our resulting screen should look as follows:

Shopping cart basket

The shopping cart basket implements the majority of the functionality. It should provide ways to add, edit and delete items from it. The basket is merely a temporal storage area for the user to group the items of interest. We then need some session management to be able to recognize which items belong to each client in its own session.

ASP.NET, as in previous asp applications, provides the necessary objects to work with sessions. In this case we will take advantage of the Session object. We will store a DataTable object as a Session variable therefore all shopping cart basket operations are done in memory. This is a very fast approach but will need to be provided with enough memory to handle concurrent clients.

The shopping basket module comprises two fundamental parts. The first involves managing the items that the user wants to have in the basket. The second is in charge of displaying the items currently in the basket. We will see the details now.

We instruct our Web Form to add an item to the cart with querystring variables. We just add an item to the DataTable. To implement the update and delete methods we bind the DataTable to the DataList Web Control and perform the operations.

void Page_Load(Object Sender, EventArgs E)
            {
if (Request.QueryString["cartAction"] != null)
            {
this.cartAction =
            Int32.Parse(Request.QueryString["cartAction"]);
}

if
            (Session["ShoppingCart"] == null) {
Cart = new
            DataTable();
Cart.Columns.Add(new DataColumn("ItemId",
            typeof(string)));
Cart.Columns.Add(new DataColumn("Item",
            typeof(string)));
Cart.Columns.Add(new DataColumn("Qty",
            typeof(string)));
Cart.Columns.Add(new DataColumn("Price",
            typeof(string)));
Session["ShoppingCart"] = Cart;
}
else
            {
Cart = (DataTable)Session["ShoppingCart"];
}

CartView
            = new DataView(Cart);
CartView.Sort = "Item";

if
            (!IsPostBack) {
//Add new entry to the shopping cart
int
            myCartAction = 0;
if (Request.QueryString["cartAction"] != null)
            {
myCartAction =
            Int32.Parse(Request.QueryString["cartAction"]);
}
if
            (myCartAction==1) {
//Take the items details from the
            database
string myItemId = "0";
if
            (Request.QueryString["itemId"] != null) {
myItemId =
            Request.QueryString["itemId"];
}
string SQLQuery = "SELECT
            itemName, itemPrice FROM item WHERE itemId = " + myItemId;
String
            connString =
            ConfigurationSettings.AppSettings["connString"];

SqlConnection
            myConnection = new SqlConnection(connString);
SqlCommand
            myCommand = new SqlCommand(SQLQuery,
            myConnection);
myConnection.Open();
SqlDataReader dr2 =
            myCommand.ExecuteReader();

if (dr2.Read()) {
DataRow dr =
            Cart.NewRow();
dr[0] = myItemId;
dr[1] =
            dr2.GetString(0);
dr[2] = "1";
dr[3] =
            dr2.GetSqlMoney(1).ToString();
Cart.Rows.Add(dr);
}
}

BindCartList();
BindTotalList();
}
}

The functions to update and delete data are as follows:

protected void DataCartList_DeleteCommand1(Object Sender, DataListCommandEventArgs e) {
string item = ((Label)e.Item.FindControl("Label4")).Text;
CartView.RowFilter = "Item='"+item+"'";
if (CartView.Count > 0) //item exists in cart
CartView.Delete(0);
CartView.RowFilter = "";

CartList.EditItemIndex = -1;
BindCartList();
BindTotalList();
}

protected void DataCartList_EditCommand1(Object Sender, DataListCommandEventArgs e) {
CartList.EditItemIndex = (int)e.Item.ItemIndex;
BindCartList();
BindTotalList();
}

protected void DataCartList_CancelCommand1(Object Sender, DataListCommandEventArgs e) {
CartList.EditItemIndex = -1;
BindCartList();
BindTotalList();
}

protected void DataCartList_UpdateCommand1(Object Sender, DataListCommandEventArgs e) {
string itemId = ((Label)e.Item.FindControl("Label1")).Text;
string item = ((Label)e.Item.FindControl("Label2")).Text;
string qty = ((TextBox)e.Item.FindControl("Text1")).Text;
string price = ((Label)e.Item.FindControl("Label3")).Text;

// with a database, we'd use an update command. Since we're using an in-memory
// DataTable, we'll delete the old row and replace it with a new one
//remove old entry
CartView.RowFilter = "Item='"+item+"'";
if (CartView.Count > 0) //item exists in cart
CartView.Delete(0);
CartView.RowFilter = "";

//add new entry
DataRow dr = Cart.NewRow();
dr[0] = itemId;
dr[1] = item;
dr[2] = qty;
dr[3] = price;
Cart.Rows.Add(dr);

CartList.EditItemIndex = -1;
BindCartList();
BindTotalList();
}

Finally we need to calculate the subtotal, tax and total cost of the order. We use the BindTotalList() function to generate content and bind it to another Web Control in our page.

protected void BindTotalList() {
double
            mySubTotal = 0;
double myTaxRate = 0.15;
double myTax =
            0;
double myTotal = 0;

DataTable CartTable =
            (DataTable)Session["ShoppingCart"];
foreach(DataRow myRow in
            CartTable.Rows) {
Double tempPrice =
            Double.Parse(myRow[3].ToString());
Int32 tempQty =
            Int32.Parse(myRow[2].ToString());
mySubTotal += tempPrice *
            tempQty;
}

myTax = mySubTotal * myTaxRate;
myTotal =
            mySubTotal + myTax;

CartTotal = new
            DataTable();
CartTotal.Columns.Add(new DataColumn("SubTotal",
            typeof(string)));
CartTotal.Columns.Add(new DataColumn("Tax",
            typeof(string)));
CartTotal.Columns.Add(new DataColumn("Total",
            typeof(string)));

DataRow drTotal =
            CartTotal.NewRow();
drTotal[0] =
            mySubTotal.ToString();
drTotal[1] =
            myTax.ToString();
drTotal[2] =
            myTotal.ToString();
CartTotal.Rows.Add(drTotal);

CartTotalView
            = new DataView(CartTotal);
CartTotalList.DataSource =
            CartTotalView;
CartTotalList.DataBind();
}

The shopping cart screen looks like this:

Checkout System

The checkout system has the responsibility of saving the client info and selected items for processing. We use three tables to save the client info. These are the following:

  • orderData - Stores the date of the order and generates the order Id.
  • orderItem - List of items bought, it also saves the price at the time of purchase.
  • orderDetail - Saves shipping and billing information for the order.

We divide our process into three steps:

  • Shipping and Billing Information - Here we get the shipping information. With this info we can calculate the shipping costs. This should be done accordingly to each store's necessities. We validate credit card information also. This should be done using a specific provider like paypal.
  • Confirmation - Here we present a summary of the information received and show the list of items to be bought, as well as the grand total of the order. The user is asked to submit the information if it is correct.
  • Summary - We finish the transaction and show the user the order number.

We post the info from the first to the second step. We then use Web Form Controls to ask for confirmation. If the info is correct, then we register the order for our client.

We do need to implement some form validation before posting the information and implement some credit card validation through an external provider or your own software.

Tracking your order

The final part of our online shopping cart application is the tracking system. We will implement a basic tracking system which will ask the user for their email and order number. The system will then show a list of milestones registered in the tracking database.

We use a single table to save the tracking information. It consists of the order id and the information of the goals achieved. Upon valid email and order number we display the list of events below.

This is the code required:

protected void ButtonView_Click(object
            Source, EventArgs e) {
//Look for order id with provided email
            and order number
String connString =
            ConfigurationSettings.AppSettings["connString"];
string strSelect
            = "SELECT orderId FROM orderDetail WHERE orderId=" + orderIdBox.Text
            + "
AND orderDetailShippingEmail='" + emailBox.Text +
            "'";
SqlConnection myConnection = new
            SqlConnection(connString);
SqlCommand myCommand = new
            SqlCommand(strSelect,
            myConnection);

myConnection.Open();
SqlDataReader dr =
            myCommand.ExecuteReader();
int newOrderId = 0;
if (dr.Read())
            {
//emailBox.Text =
            "Encontramos!!!";
Bind_TrackingList();
}
myConnection.Close();
}

protected
            void Bind_TrackingList() {
string SQLQuery = "SELECT
            orderProgressText, orderProgressDescription, orderProgressDate FROM
            orderProgress
WHERE orderId = " + orderIdBox.Text;
String
            connString =
            ConfigurationSettings.AppSettings["connString"];

SqlConnection
            myConnection = new SqlConnection(connString);
SqlDataAdapter
            myCommand = new SqlDataAdapter(SQLQuery,
            myConnection);

DataSet ds = new
            DataSet();
myCommand.Fill(ds,
            "orderProgress");

MyTrackingList.DataSource = new
            DataView(ds.Tables[0]);
MyTrackingList.DataBind();
}

Deployment of the site

Deployment of this particular application is very simple.

  • Create a new database in SQLServer
  • Once we have created the database we need to run the database creation script provided.
  • Create a new site in your IIS administration program
  • Copy the web.config file to the wwwroot folder.
  • Copy the Web Forms and User controls to the wwwroot folder.
  • Copy the codebehind.dll into your wwwroot/bin directory (no registration required!).

Improvements to our online shopping cart application

The shopping cart we just made takes a basic approach to the application providing the central framework of an online shopping cart. Further enhancements can be made and will be the responsibility of the reader. These enhancements should be made to enrich the shopping cart application give a better service to the online community. Some ideas are presented here.

Feature improvements

  • Recursive sections for n levels of profundity
  • Item reviews
  • Item related items
  • User accounts and management
  • Mailing list to customers
  • Specific shipping costs calculations

Technical improvements

  • Build based on components for performance and possible code reuse
  • Migrate to SQL Server database using stored procedures for performance
  • Implement SSL in your server (a must but out of scope!)
  • Implement real time credit card processing and charging.

Conclusions

The system presented shows the basis of a working online shopping cart. It involves the main aspects of an online store which are the catalog of items, the item detail module, the shopping cart basket and the checkout module. Every shopping cart online today must implement these basic parts to be functional. Hopefully this article has given you the knowledge to build your own with ease and to add new functionality to meet your needs as ideas for improvement have been given.

In future articles, we will cover some of the important requirements of a real - world shopping cart that have been left out of this article for the sake of brevity - a management console to add, modify and delete categories, items and orders; item reviews; SSL for credit card payments; and a discussion of working with a credit card processing provider.


Creating a Windows Service in C# - a Network Status Monitor

Creating a Windows Service in C# - a Network Status Monitor
http://www.csharptoday.com/content.asp?id=1545
Article Rating: 5
Published on August 6, 2001
Abstract
Windows services have always been rather difficult to build but with the dawn of C# and .NET comes the ability to effortlessly create Window services. In this article Brian D Patterson shows us how to build a Windows service that checks the availablity of a network. To implement this, we see how to access the Windows API through standard Windows DLLs, build a Ping class that uses raw sockets to ping remote hosts at regular intervals, and finally see how to send a Email should the host not respond to the ping.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

Windows services have always been rather difficult to build but with the dawn of C# and .NET comes the ability to effortlessly create Window services. In this article we see how to build a Windows service that checks the availability of a network. To implement this, we see how to access the Windows API through standard Windows DLLs, build a Ping class that uses raw sockets to ping remote hosts at regular intervals, and finally see how to send a Email should the host not respond to the ping.

Service Design Considerations

A Windows service has several characteristics you should keep in mind when begin your design.

  • A service should be able to operate with no user interface and no user intervention. Since your application will be capable of running when a user is not logged into the system, it must only display windows and other forms when the user specifically asks the service to. If the service attempts to open a window when nobody is logged in, an exception will be thrown and your service will die.
  • A service should trap all errors and log them to the Event Log or some other acceptable location. By logging all error messages to the Event Log or a common log file, you will not run the risk of popping open a window when nobody is logged into the system thus killing your service.

User Interface

Windows services typically don't contain a user interface but rather an administration program that connects to the service via named pipes, TCP/IP or any number of other methods. At bare minimum most services would contain an INI file or some other form of configuration. The sample application we are about to build contains neither and our setup options are coded as constants in the source files. Most of you are quite familiar with the methods of reading and writing to the registry and INI files so I'm not going to bore you with the code to do so.

Logging Interface

There are several acceptable methods for logging important events within a Windows service. More often than not, developers choose to record this important information in the Event Log. If this program handles critical operations, you may choose to send email and/or send a network broadcast message. You may also choose to log this information to a flat file that can then later be processed to generate statistics on up-time, number of critical errors, etc.

Sample Application

The sample application we are about to develop is a Windows service that checks the availability of a network host. To accomplish this, our service must ping the host at specified intervals and if the host does not respond we have the capability to send a email and/or send a network popup message.

We will cover sending email using an SMTP server, using DllImport to access the Win32 API for sending our network message, C# timers, and raw sockets to ping a remote host. The ping portion of this program is a substantial amount of code so we will only cover the important pieces. The source code has been heavily commented so learning its behavior shouldn't be a problem.

Permissions

Our sample application uses raw sockets to send ICMP (Internet Control Message Protocol) messages to network hosts. If you are using Windows NT or Windows 2000 you must have administrative privileges on your box to do this.

Skeleton Service

Creating a skeleton service is a fairly simple task. To begin creating our sample application, create a new C# project and select Windows Service as shown below. Give this new project a name and then click OK.

With that done, you will need to set the name of your service by selecting Service1.cs in the Solution Explorer and then changing the name shown in your properties window, as seen below.

You may also notice the hyperlink in the bottom of the properties window that says Add Installer. When you click this, Visual Studio will add a supporting class to your project that handles registering your service in the registry as well as setting the login properties. You may click this link now and when the wizard has completed its task, double click on ProjectInstaller.cs in the Solution Explorer. Your code window will now contain serviceProcessInstaller1 and serviceInstaller1. Click the serviceInstaller1 so you can modify its properties. In the Properties window you can modify DisplayName to anything you wish. This will be the name that shows up in the Services control panel. Now click the serviceProcessInstaller1 so we can change the service account. The Account property tells Windows if this service will log in to the system as a user or as a local service when it first starts up. Change this property to LocalSystem.

You have completed a skeleton service application which will now compile and run. Be aware that if you compile this application and attempt to run it from within Visual Studio you will get an error stating that services must be installed using installutil.exe. I will explain these steps a bit later in the article. If you would like to test your service right now, you can skip down to the Starting the Service section later in this article.

This service requires several namespaces that are not included by default. They are:

using System.Windows.Forms;
using System.Runtime.InteropServices; // Needed for DllImport so we can access the Win32 API
// Include Web.Mail for access to SMTP email features.
using System.Web.Mail;

In addition to namespaces, our project utilizes several constants that contain information like email addresses and polling intervals. Add this code to the top of your source file, immediately after your class declaration:

// This is the IP Address or name of the host to check
// NOTE: Change this to the machine that you wish to poll
const string CheckRemoteHost = "192.168.1.102";

// Name of the computer where a network message should be sent.
// NOTE: Change this to a machine name that you wish the network
// messages to be sent to.
const string PopupTo = "DevPC";

// Send an alert email to this person.
// NOTE: Change this to the email address of the person to send alert
// email to.
const string EmailTo = "briandpatterson@home.com";

// When the email is sent - the reply address will be the following.
// NOTE: Change this to the email address of the person that
// sent the notification.
const string EmailFrom = "briandpatterson@home.com";

// Specify your SMTP server here. If you are running IIS
// or a similar SMTP server locally, simply put 'localhost' here.
const string EmailServer = "mail.peoria1.il.home.com";

// Check our remote host this often.
const int CheckInterval = 5; // In Minutes

We will now need to add a DllImport statement. DllImport allows us to access the Win32 API so we can later send a network message to a specified host.

[DllImport("Netapi32",EntryPoint="NetMessageBufferSend",SetLastError = true,
CharSet = CharSet.Unicode, ExactSpelling = true, CallingConvention =
CallingConvention.StdCall)]
public static extern int spu(string servername, string msgname, string
fromname, string buf, ref int buflen);

Most of this is pretty self explanatory, but if you would like more information on DllImport feel free to look at the MSDN help files located at http://msdn.microsoft.com/library/default.asp?url=/library/en-us/csspec/html/vclrfcsharpspec_B_8.asp&WROXEMPTOKEN=620568Zhiox25QOrrsnTPfm53p. The entry point into this DLL is going to be NetMessageBufferSend. All characters strings are going to be in Unicode, as specified by CharSet and we are going to use a standard calling convention. Immediately after this DllImport we declare our function and all it's parameters. Some of you may be familiar with the sysimport from .NET Beta 1. You can no longer use this in Beta 2 and must instead use the DllImport.

After adding these lines of code, we are ready to begin adding some functionality.

OnStart Method

The OnStart method will run when you first start your service. It is here where we need to create a timer that will fire at a specified interval. In this example, I have created a constant where you can specify the number of minutes that the timer should fire. This constant value is then multiplied by 60000 milliseconds. Here is the code for our OnStart:

protected override void OnStart(string[]
            args)
{
System.Timers.Timer aTimer = new
            System.Timers.Timer();
aTimer.Tick += new
            EventHandler(OnTimer);
aTimer.Interval = CheckInterval * 60000
            ;
aTimer.Enabled = true;
}

Once we create the new timer, we need to create an event handler for it that will be called when the timer fires. We then set the interval of our timer (in milliseconds) and then enable the timer.

The OnTimer event we created will run when the timer has reached its specified interval. Here is the code for our OnTimer event:

public static void OnTimer(Object source,
            EventArgs e)
{
netPing np = new netPing();
string
            buf;
string servername = null;
string msgname =
            PopupTo;
string fromname= SystemInformation.ComputerName;
if
            (netPing.Ping(CheckRemoteHost) < 1)
{
buf = "Network device
            is not responding..";
int buflen =
            buf.Length;
spu(servername,msgname,fromname,buf,ref
            buflen);
SendEmail();
}
}

You will notice that within the first line of code we are creating a new netPing object. This is our class that actually handles the socket communications required to ping a remote host over the network. We will get to this later in out article.

As you may recall, we added a DllImport statement to our project earlier. The point of this was to access the necessary API to send a network message. Our OnTimer method simply pings the remote host and checks the return code. If this return code is less than one, our ping failed for one reason or another. All possible return values are shown in the table below.

Return Code | Description
1 | Ping reply received
0 | Request timed out
-1 | Host DNS not found
-2 | Failed to build packet
-3 | Failed to build packet with Checksum
-4 | Exception while waiting for return data

If it was indeed a failure we assign a string to buf that will be sent as a network message to a specified computer using our spu function. We then can call SendEmail which sends an email out regarding the status of the remote host.

SMTP Interface

The SMTP interface is simply a public method that sends a predefined message to an email address of choice.

There are a couple different SMTP (Simple Mail Transfer Protocol) interfaces but we are using the Web version because it allows us to change the SMTP server and not require IIS to run on the local machine. Below is the code for that interface.

public static void
            SendEmail()
{
MailMessage mm = new MailMessage();
mm.To =
            EmailTo;
mm.From = EmailFrom;
mm.Subject = "Network Device
            Status";
mm.BodyFormat = MailFormat.Text;
mm.Body = "Network
            device is not responding!!";
SmtpMail.SmtpServer =
            EmailServer;
SmtpMail.Send(mm);
}

As you may have noticed, this method uses several variables that we have defined as constants at the top of our source file. Please modify these constants with your own email address and preferences.

Ping Class

As I mentioned before, our ping class is rather lengthy so I won't describe it all in detail but I have documented the source code very heavily so it should be easy to figure out.

I have seen several versions of a ping program for C# on the internet. However, all of the versions I have come across use blocking sockets. This is a serious problem when building a ping program because when ping attempts to wait for a response, if the host is down your program will stall waiting for a response and not have the ability to timeout.

To overcome this you could use the asynchronous BeginReceiveFrom call or you can use the method we implement here. The method I implement is defining the socket as non-blocking and then catching any network exceptions while trying for 3 seconds to receive a response. You may choose to change to timeout period from 3 seconds to as much or as little as you wish. Below is a snippet from the class which contains our timeout period and the network methods I have just discussed.

socket.Blocking =
            false;
try
{
nBytes=socket.ReceiveFrom(ReceiveBuffer,256,0,ref
            EndPointFrom);
}
catch (SocketException e)
{
// Ignore
            the 'Socket Not Ready' error and
// return a failure for all
            other exceptions.
if (e.ErrorCode != 10035) // Socket not
            ready
return -1;
}
if(nBytes>0)
return
            1;
timeout=System.Environment.TickCount -
            dwStart;
if(timeout>3000)
return 0; // Our request to the
            system has timed out

I use a try-catch because you will usually receive a Socket Not Ready exception when trying to create a non-blocking socket that listens for data. We catch this exception and ignore it and exit from the method if any other error code exists. As you can see, we use a TickCount to count the number of milliseconds since the ICMP message was sent. Ping requests should return within a few seconds. If they do not it is safe to assume that the packet did not arrive at the destination or that we won't be receiving a response.

The method used in this example isn't a method that you could use in any ordinary network application because we are waiting for any number of return bytes and then continuing. This would be unacceptable for any other application but will suffice for us because whether we receive only one byte or 60, we know that the network host is up and responding.

Bringing it all together

Now that we have built our sample application, it is now time to register it as a system service. To do this you must go to a console window and navigate to the place where the executable is located. Once there, type:

Installutil <name of your application.exe>

This will run the .NET installer and register your application as a system service. If you do not see any output messages when you run this command there could be several things wrong. The most typical of these being; if you previously had Visual Studio.NET Beta 1 installed and uninstalled it, it may have left your path pointing to the wrong .NET libraries and utilities. Change this path to the appropriate location, reboot and try again.

If all goes well, the installer will display a message saying that the installation was successful. You can now close your console window. If you ever wish to uninstall your service you need to use the Installutil.exe application in this fashion:

Installutil \u <name of your application.exe>

Configuring the Service

We can configure our service several ways. You can either use the Server Explorer in Visual Studio and navigate to the services as show here:

You can also right click on My Computer and select Manage. This will take you into Computer Management where you can start and stop the service as well as change its properties. The property page of your service will look similar to this:

Once you are in the property page, you can change the Startup Type of your service. This will determine if your service has to start up manually or automatically.

Login Options

It's generally ok to leave your logon settings to Local System Account for your service. However, if your service accesses network resources that require certain privileges, you may need to create a new account for it to use. In any event, this account must have the Log on as System Service privilege.

Starting the service

To start your service you can use the Computer Management window or from a console window. From a console window you would simply type:

Net start <name of your service>

If your service starts up normally, Windows will tell you. If your service does not start up you can check you Application Event log for any error messages. If at any time you need to stop your service just type:

Net stop <name of your service>

Testing 1..2..3

Once you have started the service up, wait 5 minutes and ensure no network messages or emails arrive in your inbox saying the network host is down. If this network host is a PC on your LAN, try shutting it down now. The next time the computer polls the machine you should receive an email and a broadcast network message. If you are having these broadcast messages sent to a non-NT based machine, you will need to run winpopup.exe or you will not receive any network messages. Windows NT and Window 2000 users need not run anything because your messenger service should be running in the background already (as a service).

Conclusion

This example service is very robust but a bit more extensibility could be added. The C# services also contains the ability to have their name, login parameters and display name stored within a configuration file. This would allow the end user to name the service anything they wished as well as allow them to store settings for the polling period and email addresses. With a little hard work and a bit of typing, anything is possible!


.NET Metadata and Custom Attributes in C#

.NET Metadata and Custom Attributes in C#
http://www.csharptoday.com/content.asp?id=1602
Article Rating: 4.5
Published on July 18, 2001
Abstract
Attributes allow you to declaratively inject metadata information into types such as assemblies, classes, interfaces, and methods etc. At runtime, they allow you to elicit the values of these declarative properties through Reflection. In this article, Aravind Corera shows you how to define custom attributes, use them in your applications, and glean out the metadata properties from these attributes at runtime using reflection. And finally, to see how all this fits in, the author will show you how to build a trivial Project Status tracking system based on custom attributes.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

Attributes allow you to declaratively inject metadata information into types such as assemblies, classes, interfaces, and methods etc. At runtime, they allow you to elicit the values of these declarative properties through Reflection. If you've been using the .NET framework for a while, you might have used attributes predefined in the .NET framework from the all-popular WebMethod attribute to the platform invoke DllImport attribute. You can define your own custom attributes too, allowing you to specify user defined metadata information for your types. Essentially, tools and applications can walk the assembly using reflection, read the attributes tagged to your types, and then use the metadata values emitted by the attributes to perform specific actions dictated by the tool or application. For example, you could define a custom attribute called MyCoolComment that allows users to comment their classes, methods and possibly any of the other types using the MyCoolComment attribute. When users build their assembly, the MyCoolComment attribute properties for the various types are stored as a part of the assembly metadata. Now, they can run an introspection utility, which can use reflection on the compiled assembly to read type information and extract the metadata values emitted by the MyCoolComment attribute; and possibly even generate documentation files based on the comments specified by developers as parameters to the MyCoolComment attribute. Custom attributes combined with Reflection give you all the power you need to build applications that leverage the rich-runtime metadata properties that can be emitted into .NET types. We'll take a look at how you can define custom attributes, use them in your applications, and glean out the metadata properties from these attributes at runtime using reflection. And finally, to see how all this fits in, we'll build a trivial Project Status tracking system based on custom attributes.

Declarative programming with Attributes in the pre - .NET era

The concept of using Attributes to declaratively tag your types with specific properties has been used right from the IDL days in Classic COM where you could tag your interfaces, coclasses and other types with specific properties to define how MIDL generates a type-library for you based on the type metadata that you specified. Take a look at the following fragment of code that defines an interface in an IDL file:

[
object,
uuid(BE246480-1FDD-4A17-B07D-6A90D49F2174),
dual,
helpstring("IHotelReservation Interface"),
pointer_default(unique)
]
interface IHotelReservation : IDispatch
{
[id(1), helpstring("method DoesHotelHaveInternetConnectivity")]
HRESULT DoesHotelHaveInternetConnectivity ( [out,retval] VARIANT_BOOL* pbConnectivity);

[propget, id(2), helpstring("property HotelLocation")]
HRESULT HotelLocation([out, retval] BSTR *pVal);

...

};

In the above IDL code snippet, your interfaces, its methods and properties are tagged with attributes such as object, uuid, dual, id, helpstring and so on to their properties and characteristics. Utilities could then use the value of these attributes to gather information about these types and take application specific actions accordingly. In the IDL code snippet example that you just saw, tools such as MIDL could read the attributes for the interface and when it finds, for example, that a dual attribute is tagged to the interface, it understands that the interface wants to get the best of both early-binding and late-binding and generates the type-library accordingly to convey that information. Tagging types with attributes in IDL files to associate properties with them is an example of how attributes were used in the pre-.NET days. Although attributes were available back then, there was no definite programming model available to languages to define these attributes on types and gather information on the values of these properties they represented at runtime. Attributes are first class citizens with well-defined programming constructs in the .NET framework for defining them, using them in your apps, and extracting metadata from them at runtime using Reflection.

Attributes and declarative programming in the .NET world

Attributes play a key role in the .NET framework and make it so easy to qualify your types with certain properties. For example, you could tag one of your methods with the WebMethod attribute to indicate that the method is exposed as a publicly available method in a web service:

public class WeatherService : System.Web.Services.WebService
{
...

[WebMethod(Description="Returns the Temperature in Fahrenheit",
EnableSession=true)]
public float GetTemparatureAtCity(String CityName)
{
// Calculate Temparature
return fTemperature;
}

}

In the above example, you see that the WebMethod attribute is applied to the GetTemperatureAtCity method. Further, the attribute uses additional parameters such as Description and EnableSession. These parameters are called Named parameters: essentially key-value pairs that provide a way for you to specify additional properties for the attribute. We will take a look at this in detail in the next section. Tools and utilities can now use reflection to walk the WeatherService class and determine the methods intended to be exposed as publicly available methods in the web service by finding the ones that are tagged with the WebMethod attribute. It really is that easy to associate metadata information with your types using attributes. There are many other similar attributes to declaratively associate metadata information with your types using the .NET framework, and, in addition, it gives you the flexibility to define your own user-defined attributes (custom attributes) that you can use to associate certain user-defined properties with a type. For example, if your application supports plugins to dynamically provide extended functionality, you could define an attribute called MyCompanyPlugin and ask all plugins to tag their assemblies with this attribute. Your application could then use Reflection to introspect these assemblies and ascertain if it's a plugin associated with your application. In the Classic COM world, you could use interfaces and Component Categories to achieve such plugin functionality. Using attributes to achieve the same thing is a lot easier. Attributes combined with Reflection in the .NET world make it a breeze to emit metadata declaratively into your types.

Defining Custom Attributes using C#

Let's create a custom attribute called SourceCodeTracker that is used by developers to specify work-in-progress details about the current class or method they are implementing. It tags a method or class with the developer's name, email address, when the code was last modified, the reason the code was modified and whether or not the implementation is complete. We will then use reflection on a class that is injected with the SourceCodeTracker attribute to get details on the status of completion of the various tasks. But before we do that, let's see how attributes are defined in the .NET world.

Attributes are essentially classes that extend the System.Attribute class. Attribute classes defining an attribute follow the naming convention AttributeNameAttribute. So if you were defining a custom attribute called SourceCodeTracker, then the class defining the attribute would look like this:

public class SourceCodeTrackerAttribute : System.Attribute
{
...
}

Attributes allow you to specify properties either through positional parameters or through named parameters. Positional parameters are mandatory and need to be specified every time you declare the attribute. Named parameters on the other hand, are optional and it's left to you whether or not you want to include them with the attribute. Internally, the Attribute class defining the attribute represents Positional parameters using parameters passed to the constructor and represents named parameters using set methods associated with public properties. If you omit some of the named parameters, then the class defining the attribute usually uses default values associated with the underlying properties. Take a look at the following example of how a class is injected with the SourceCodeTracker attribute:

[SourceCodeTracker("John Doe","johndoe@mycompany.com",
DateModified = "03/05/2001 13:30",
ReasonModified = "Initial Coding"
Status = IMPLEMENTATION_STATUS.InProgress)
]
public class AirCombatSimulatorController
{
...
}

The SourceCodeTracker attribute has two Positional parameters representing the name of the author (JohnDoe) and the author's email address (johndoe@mycompany.com). The rest of the parameters DateModified, ReasonModified and Status are named parameters. So the attribute's class definition needs to provide a constructor to accept the two positional parameters. Overloaded constructors could also be provided to allow the user to ignore one of the parameters or specify additional parameters as positional parameters. The Class also needs to provide a set method representing the corresponding public property for each of the 3 named parameters. Here's how the class representing the SourceCodeTracker attribute would look.

Listing 1: SourceCodeTracker.cs

namespace MyCompanyAttributes
{
using System;

// Status of task completion enumerated data type
// Used by the 'Status' Named Parameter
public enum IMPLEMENTATION_STATUS
{
NotStarted = 0,
OnHold,
InProgress,
FunctionallyComplete
};
public class SourceCodeTrackerAttribute : System.Attribute
{
// Private data fields
private String m_strAuthor = "Default Author";
private String m_strEmailAddress = "default-author@MyCompany.com";
private String m_strModifiedDate = System.DateTime.Today.ToString();
private String m_strReasonModified = "Not specified";
private IMPLEMENTATION_STATUS m_enumStatus =
IMPLEMENTATION_STATUS.NotStarted;

// Attribute constructor that takes the Author's Name and Email
// Address as positional parameters.
public SourceCodeTrackerAttribute(String strAuthor,
String strEmailAddress)
{
m_strAuthor = strAuthor;
m_strEmailAddress = strEmailAddress;
}

// Returns the name of the Author
// Accessor (Read only - Cannot be used as a Named parameter)
// (Can only be used as a positional parameter through the
// constructor)
public String Author
{
get {return m_strAuthor; }
}

// Returns the Author's Email Address
// Accessor (Read only - Cannot be used as a Named parameter)
// (Can only be used as a positional parameter through the
// constructor)
public String EmailAddress
{
get { return m_strEmailAddress; }
}

// Named Parameter indicating the date that this
// Class/Method was last modified
public String DateModified
{
get { return m_strModifiedDate;}
set { m_strModifiedDate = value;}
}

// Named parameter that indicates Implementation Status
// of this Class/Method
public IMPLEMENTATION_STATUS Status
{
get { return m_enumStatus; }
set { m_enumStatus = value; }
}

}/* end class SourceCodeTrackerAttribute */

}/* end namespace MyCompanyAttributes */

You can create a VS.NET Visual C# Class Library Project for the SourceCodeTrackerAttribute class and compile it to generate the SourceCodeTracker.dll. Alternatively, you could use the following command from the DOS command-line:

csc /out:SourceCodeTracker.dll /target:library SourceCodeTracker.cs

As seen in the code snippet above, the positional parameters that are set through the constructor expose only a get method for the corresponding properties whereas the other three named parameters expose both a get and set method for the corresponding property. This ensures that the positional parameters can be modified only through the constructor and not through the PropertyName=Value syntax as used by named parameters. Please take note though, that nothing prevents you from having a set method as well for the property associated with a Positional parameter. Having such a provision allows you to use that specific property both as a positional parameter and as a named parameter.

Attributes, as you've seen could be applied to whole plethora of types like classes, interfaces, structs, enums, constructors, methods, fields, delegates etc. They could be applied to an entire .NET Assembly or Module. Attributes applied to whole assemblies or modules are called global attributes. By default, any attribute you define can be injected to any of the available types. But most often, you will find that the attributes that you define should be used only against specific types. For example, it would be absurd if the WebMethod attribute were applied to a type like a class or struct. It makes sense to use the WebMethod attribute only on a method. So you need some way of telling the attribute that it can be applied only on certain target types. This information needs to be a part of the actual attribute class definition. Here's where the AttributeUsage attribute helps.

[AttributeUsage(Validon,
AllowMultiple=allowmultiple,
Inherited=inherited)
]

The AttributeUsage attribute allows you to specify the usage scenarios for an attribute. The Validon property of this attribute helps you set a range of target types to which this attribute could be applied. Although the SourceCodeTracker attribute would in theory accommodate being applied to any of the available types, for the purpose of this demonstration, we will ensure that our SourceCodeTracker attribute can be applied only to classes and methods and not to any of the other types. To do that we'll have to use the AttributeUsage attribute on the SourceCodeTrackerAttribute class definition like this:

[AttributeUsage(AttributeTargets.Class | AttributeTargets.Method)]
public class SourceCodeTrackerAttribute : System.Attribute
{

}

The Validon property of the AttributeUsage attribute is used as a positional parameter and it takes values defined in the AttributeTargets enumeration. You will need to do a bitwise OR operation to indicate that the attributes can be applied to either a Class or a Method. So if you attempt to use the SourceCodeTracker attribute on any other type other than a class or method, such as an interface or a delegate as shown in the following code snippet, you would get a compile-time error saying that the attribute cannot be used on that type. This ensures some deal of type safety by ensuring that you do not associate the attribute with any arbitrary type:

[SourceCodeTracker("John Doe","johndoe@mycompany.com",
DateModified = "03/05/2001 13:30",
ReasonModified = "Defined the Motion Sensor Interface",
Status = IMPLEMENTATION_STATUS.InProgress)
]
interface IAircraftMotionSensor
{
...........
}

Here's the kind of error message that would be returned from the compiler:

Attribute 'SourceCodeTracker' is not valid on this declaration type. It is valid on 'Class, Method' declarations only.

There are two other named parameters in the AttributeUsage attribute namely, AllowMultiple and Inherited. The inherited named parameter allows derived classes to inherit the attributes that are used in the base class. The AllowMultiple Named parameter allows you to use the attribute multiple times in the same type instance. For example, if the SourceCodeTrackerAttribute class was defined as follows:

[AttributeUsage(AttributeTargets.Class | AttributeTargets.Method,
AllowMultiple=true,
Inherited=true)
]
public class SourceCodeTrackerAttribute : System.Attribute
{

}

Then, you could use the SourceCodeTracker attribute multiple times as shown in the example below:

[SourceCodeTracker("John Doe","johndoe@mycompany.com",
DateModified = "03/05/2001 13:30",
ReasonModified = "Initial Coding"
Status = IMPLEMENTATION_STATUS.InProgress),

SourceCodeTracker("Laura Adams","lauraadams@mycompany.com",
DateModified = "03/25/2001 09:30",
ReasonModified = "To Fix Bug B345 in Bug Report"
Status = IMPLEMENTATION_STATUS. InProgress)
]
public class AirCombatSimulatorController
{

}

Also, any class that inherits from the AirCombatSimulatorController class inherits all the AirCombatSimulatorController class' attributes as well, since the Inherited parameter is set to true.

Using Custom Attributes to emit metadata into your types

We'll now use the SourceCodeTracker custom attribute that we created in the previous section to add some metadata information to a C# class and its methods. We'll assume that a group of developers are developing an Aircraft Combat Simulator .NET Component. At any point of time, more than one developer could be working on the same class (Under version control system). Each one is assigned a specific functionality to code. Each developer has been asked to tag the method he/she is currently working on with the SourceCodeTracker attribute, so that the project manager can monitor the progress of the tasks by using an automated tool that introspects the assembly and looks for these attributes.

So here's what the developers, John Doe, Harry Smith, and Laura Adams coding the Aircraft Combat Simulator Engine are doing with the attributes in their code:

Listing 2: AircombatSimulator.cs

using System;
using MyCompanyAttributes;
[SourceCodeTracker("John Doe","johndoe@mycompany.com",
DateModified = "03/05/2001 13:30",
ReasonModified = "Initial Coding of the Core Controller Engine",
Status = IMPLEMENTATION_STATUS.InProgress),

SourceCodeTracker("Harry Smith"," harry@mycompany.com ",
DateModified = "03/18/2001 13:35",
ReasonModified = "To fix somes issues in the tracking library"
Status = IMPLEMENTATION_STATUS. InProgress)

]
public class AirCombatSimulatorController
{

public AirCombatSimulatorController()
{
// Some stuff goes in here
}

[SourceCodeTracker("John Doe","johndoe@mycompany.com",
DateModified = "03/05/2001 13:40",
ReasonModified = "Fixed some Engine Initailization routines",
Status = IMPLEMENTATION_STATUS.InProgress)
]
public void InitializeSimulatorEngine(long lLibraryHandle)
{
// Some stuff goes in here
}

[SourceCodeTracker("Harry Smith","harry@mycompany.com",
DateModified = "03/07/2001 11:10",
ReasonModified = "Added a placeholder for the Combat Range Adjustor",
Status = IMPLEMENTATION_STATUS.NotStarted)
]
public void SetCombatRange(long lKilometres)
{
// Some stuff goes in here
}

...........

[SourceCodeTracker("Harry Smith","harrysmith@mycompany.com",
DateModified = "03/18/2001 13:35",
ReasonModified = "Changed call sequence to the Radar-Jamming System
Library",
Status = IMPLEMENTATION_STATUS.InProgress)
]
public bool IsPassiveTrackingEnabled()
{
// Some stuff goes in here
bool bPassiveTracking = true;
return bPassiveTracking;
}

}/* end class AirCombatSimulatorController */

In the code snippet above, you see that each developer has added the SourceCodeTracker attribute to the current method that he/she is coding. At any point of time, when an intermediate build is made on the assembly, the project manager could run an application that uses Reflection on the assembly to extract all the details specified in the SourceCodeTracker attribute for the various methods and classes to ascertain the status of the tasks assigned to the developer.

You can create a VS.NET Visual C# Class Library Project for the AirCombatSimulatorController class and compile it to generate the AirCombatSimulator.dll. Alternatively, you could use the following command from the DOS command-line:

csc /out:AirCombatSimulator.dll /target:library /r:SourceCodeTracker.dll AircombatSimulator.cs

Using Reflection to elicit information from Custom Attributes

Well now we have put the metadata in place, how do we access it? To do so, you will have to turn to .NET Reflection. Using reflection, you will need to walk the assembly and get back type-information for all the types that your attribute is targeted at. You would then have to extract the attribute's values from the type-information.

Here are the steps to get the SourceCodeTracker attribute's metadata values for all the methods of the AircraftCombatSimulator class in the AircaftSimulator Assembly.

1. Load the Assembly and get the Type details for the class:

Assembly assem = Assembly.Load("AircaftSimulator");
Type typeClass = assem.GetType("AircraftCombatSimulator",true);

The code shown above loads the AircraftSimulator assembly and then gets the System.Type instance associated with the AircraftCombatSimulator class in the assembly. The second parameter, which is true indicates that an exception needs to be thrown if the class is not found in the assembly.

2. Get metadata information for the methods in the class:

Once you have the Type information for the class, you can use reflection and call Type.GetMethods, Type.GetMembers, Type.FindMembers or a similar method to get metadata information on the members of the class. Let's quickly put together a helper class called SourceCodeAttributeIntrospector that does the class introspection for us using reflection. The static method ExtractAttributeMetadata of the SourceCodeAttributeIntrospector class takes in the Type of the class (that you got back from Step 1) and returns an ArrayList containing objects with the desired metadata. Since you know that the target types of your SourceCodeTracker attribute could only be classes and methods, you could use the Type.GetMethods method to get the methods in the class. This returns an array of MethodInfo objects. Each MethodInfo object contains details of the method that it represents.

public class SourceCodeAttributeIntrospector
{
public static ArrayList ExtractAttributeMetadata(Type typeClass)
{
ArrayList trackingArray = new ArrayList();

// Use Relection to get the list of methods in the
// Class Specified
MethodInfo[] methodInfoArray = typeClass.GetMethods();

...........

return trackingArray;

}/* end ExtractAttributeMetadata */

}/* end class SourceCodeAttributeIntrospector */

3. Looking up the attribute's metadata values from the type information:

To ascertain if the method that the MethodInfo object represents, was originally tagged with the SourceCodeTracker attribute, you need to iterate through the MethodInfo array (that you got back in Step 2) using a construct such as the C# foreach statement and check the custom attributes associated with the method. Notice that the System.Reflection.MethodInfo class provides a GetCustomAttributes method that it inherits and overrides from the System.Reflection.MemberInfo class. This allows you to get back an array of attributes associated with the method. The reason that an array of attributes is returned is because you set the AllowMultiple named parameter to true in the SourceCodeTrackerAttribute class definition's AttributeUsage attribute. This allows more than one SourceCodeTracker attribute to be assigned to a single method or class. You can then iterate through the attribute array using the C# foreach statement and examine each attribute object in the array. Since the GetCustomAttributes method requests only for SourceCodeTracker attributes, you can assume that the attributes returned in the array are indeed SourceCodeTracker attributes and can safely cast a generic attributes in the array to a SourceCodeTracker attribute. You are now ready to party on the metadata values associated with this SourceCodeTracker attribute. To access the values, use the get methods for the various public properties that the SourceCodeTrackerAttribute class exposes:

public class SourceCodeAttributeIntrospector
{
public static ArrayList ExtractAttributeMetadata(Type typeClass)
{
ArrayList trackingArray = new ArrayList();

...........

// Walk the list of methods and get the custom attributes
// associated with each method.
foreach(MethodInfo methodInfo in methodInfoArray)
{
// Check if this method uses the SourceCodeTracker attribute
Object[] attribArray =
methodInfo.GetCustomAttributes(typeof(SourceCodeTrackerAttribute));

foreach(Attribute attrib in attribArray)
{
// Cast from generic Attribute to a SourceCodeTrackerAttribute
SourceCodeTrackerAttribute srcAttrib =
(SourceCodeTrackerAttribute)attrib;

// Populate an user-defined object (CodeTrackingData instance)
// with all the metadata values that have been
// emitted into the type by the attribute
CodeTrackingData trackData = new CodeTrackingData();
trackData.MethodName = methodInfo.ToString();
trackData.Author = srcAttrib.Author;
trackData.AuthorEmailAddress = srcAttrib.EmailAddress;
trackData.LastModified = srcAttrib.DateModified;
trackData.ReasonModified = srcAttrib.ReasonModified;
trackData.Status = srcAttrib.Status;

// Add the metadata details to an ArrayList
int nIndex = trackingArray.Add(trackData);
trackData.Index = nIndex;
trackingArray[nIndex] = trackData;

}/* end outer foreach Attribute */

}/* end inner foreach MethodInfo */

return trackingArray;

}/* end ExtractAttributeMetadata */

}/* end class SourceCodeAttributeIntrospector */

The CodeTrackingData class is just a user-defined class into which you can conveniently stash away the metadata values of the SourceCodeTracker attribute. You'll see in the next section how to use the ArrayList containing these CodeTrackingData objects as a data source to populate an ASP.NET DataGrid Server Control and map each of the CodeTrackingData class' properties to the individual columns in the DataGrid.

public sealed class CodeTrackingData
{
public CodeTrackingData() {}

private int m_nIndex = -1;
private String m_strMethodName = "";
private String m_strAuthorName = "";
private String m_strAuthorEmailAddress = "";
private String m_strLastModifiedDate = "";
private String m_strReasonModified = "";
private IMPLEMENTATION_STATUS m_enumStatus =
IMPLEMENTATION_STATUS.NotStarted;

public int Index
{
get { return m_nIndex; }
set { m_nIndex = value; }
}

public String MethodName
{
get { return m_strMethodName; }
set { m_strMethodName = value; }
}

public String Author
{
get { return m_strAuthorName; }
set { m_strAuthorName = value; }
}

...........

public IMPLEMENTATION_STATUS Status
{
get { return m_enumStatus; }
set { m_enumStatus = value; }
}

}/* end class CodeTrackingData */

That's all there is to defining and using custom attributes in your .NET applications.

Putting it all together - Creating an ASP.NET page to present the Attribute Metadata

Let's quickly put together an ASP.NET application that uses the SourceCodeAttributeIntrospector class to present information on any assembly that contains classes and methods tagged with the SourceCodeTracker attribute.

1. Examining the code in the ASP.NET page:

Let's take a look at the code for this ASP.NET page. For the sake of brevity, we'll take a look at only the most important part of the ASP.NET code, because the rest of the code in the ASP.NET page is mostly the formatting elements for the look and feel which you could look up in the default.aspx file in the source-code bundle that's provided for download with this article.

Listing 3: default.aspx

<%@ Page language="c#" Codebehind="SourceTrackerClientNS"
AutoEventWireup="false"
Inherits="SourceTrackerClientNS.SourceTrackerClient"%>

...........

<form method=post runat="server">

<asp:textbox id=AssemblyNameText runat="server"></asp:textbox>

<asp:button id=ButtonGetClass onClick=ButtonGetClass_Click
runat="server" Text="Get Classes"></asp:button>

<asp:dropdownlist id=DropDownListClasses runat="server"
OnSelectedIndexChanged="DropDownListClasses_SelectedIndexChanged"
autopostback="True"></asp:dropdownlist>

<asp:button id=ButtonTrackSource onClick=ButtonTrackSource_Click
runat="server" Text="Track Source Code"></asp:button>

...........

<asp:datagrid id=TaskGrid runat="server" DataKeyField="Index"
OnSelectedIndexChanged="OnDisplayTaskDetails"
AutoGenerateColumns="false">
...........

<property name="Columns">

<asp:ButtonColumn Text="Details" ButtonType="PushButton"
CommandName="Select"></asp:ButtonColumn>

<asp:BoundColumn HeaderText="Method Name"
DataField="MethodName"></asp:BoundColumn>

<asp:BoundColumn HeaderText="Author"
DataField="Author"></asp:BoundColumn>

</property>

</asp:datagrid>

...........

<table>
<tr>
<td><asp:label id=MethodNameLabel runat="server"></asp:label></td>
</tr>
...........

<tr>
<td><asp:label id=LastModifiedLabel runat="server"></asp:label></td>
</tr>
...........
<tr>
<td><asp:label id=ModifiedReasonLabel runat="server"></asp:label></td>
</tr>
...........
<tr>
<td><asp:hyperlink id=HyperLinkMail runat="server"></asp:hyperlink></td>
</tr>
...........
<tr>
<td><asp:image id=StatusImage runat="server"></asp:image></td>
</tr>
..........
<tr>
<td><asp:label id= StatusLabel runat="server"></asp:label></td>
</tr>
</table>

</form>

As seen in the code snippet above, the ASP.NET client page uses a C# code-behind assembly class called SourceTrackerClientNS.SourceTrackerClient to handle most of the Page level events that occur in the page. We will examine the code-behind assembly in more detail in the next section. The page consists of an ASP.NET TextBox Server control ( AssemblyNameText) through which you can enter the name of the assembly that you need to introspect for attributes. When the Get Classes button is clicked, the assembly is introspected and the classes in the assembly are added to the ASP.NET Server DropDownList control (DropDownListClasses). When a class from the DropDownList is selected, and the Track Source Code button is clicked, the SourceCodeAttributeIntrospector class' ExtractAttributeMetadata static method is used to get all the metadata values of the SourceCodeTrackerAttribute for the methods present in the selected class. The MethodName and the Author metadata values are then displayed in the ASP.NET Server DataGrid control (TaskGrid). The DataGrid control also displays a Details button, which triggers a SelectedIndexChanged event when clicked. Based on the currently selected item in the DataGrid, the corresponding ASP.NET Server Label, HyperLink and Image controls that are laid out in a HTML table below the data grid are updated with more information on the selected item such as the Last Modified Date, Reason for Modification, link to the Author's email address, and the Status of the task.

2. Examining the code in the C# code-behind assembly used by the ASP.NET page:

The page level event handlers for the ASP.NET page that we saw above are housed in a C# code-behind assembly. These handlers perform most of the underlying work including introspecting the methods in the class and populating the ASP.NET server side controls with data. Let's dissect the SourceTrackerClient class in the C# code-behind assembly and see how it interacts with the ASP.NET page. We'll break down the code in the code-behind assembly into parts based on the logical flow of the user's interaction with the ASP.NET page and examine the code in each part in detail. Take a look at the code below that defines the SourceTrackerClient class:

namespace SourceTrackerClientNS
{
using System;
using System.Web.UI;
using System.Web.UI.WebControls;
using System.Collections;
using System.Reflection;
using MyCompanyAttributes;

public class SourceTrackerClient : Page
{
protected Button ButtonGetClass;
protected TextBox AssemblyNameText;
protected Label ErrorLabel;
protected DropDownList DropDownListClasses;
protected Button ButtonTrackSource;
protected DataGrid TaskGrid;
}/* end class SourceTrackerClient */

}/* end namespace SourceTrackerClientNS */

The SourceTrackerClient class inherits from the System.Web.UI.Page class that represents the ASP.NET intrinsic page object. Notice that all the ASP.NET Server controls used in the page are included as protected members in the SourceTrackerClient class. This allows you to manipulate the controls from within your class' methods and event handlers.

To start with, let's take a look at how the user would go about interacting with our ASP.NET page and we'll see how the appropriate event handlers and methods in the code-behind assembly respond to user actions. The user typically starts by typing into the ASP.NET TextBox control (AssemblyNameText), the name of the assembly that houses the classes for which source code needs to be tracked. Then, the user clicks the Get Classes button. This button's Click event handler in the code-behind class calls PopulateClassesFromAssembly helper method which loads the assembly specified and gets the list of types in it. It then iterates through every type and adds types that are classes to the DropDownList (DropDownListClasses) by setting the ArrayList containing the class names as the DataSource of the DropDownList.

// Called by the 'Get Class' button's click event
public void ButtonGetClass_Click(object Source, EventArgs e)
{
// Load the Drop-down list box with the fully qualified
// typename of the classes in this assembly
PopulateClassesFromAssembly();
}

// Fills the drop down list box with the fully qualified
// type name of the various classes in the Assembly.
// (The Assembly name is the one specified in the TextBox)
private void PopulateClassesFromAssembly()
{
try
{
// Load the assembly
Assembly assem = Assembly.Load(AssemblyNameText.Text);

// Get the types in the assembly
Type[] typeArray = assem.GetTypes();

ArrayList arrayClasses = new ArrayList();
foreach(Type type in typeArray)
{
// If the type is a Class, add it's fully
// qualified type name to the list
if(type.IsClass)
{
arrayClasses.Add(type.FullName);
}

}/* end foreach */

// Populate the DropDownList with the classes from
// the assembly
DropDownListClasses.DataSource = arrayClasses;
DropDownListClasses.DataBind();

}
catch(Exception ex)
{
ErrorLabel.Text = ex.Message;
}
}

Once the list of classes has been displayed in the DropDownList, the next logical step the user would take is to select a class in the DropDownList and click the Track Source Code button. The handler function for the Click event of the Track Source Code button uses the SourceCodeAttributeIntrospector class that we saw earlier to load all the metadata values associated with the SourceCodeTrackerAttribute. It does this for all the methods in the class and stashes away the metadata values in an ArrayList containing CodeTrackingData objects. These details are then rendered to an ASP.NET DataGrid Server Control (TaskGrid) by setting the DataSource of the DataGrid to the ArrayList containing CodeTrackingData objects. Specifically, the second and third columns in the DataGrid are mapped to the MethodName and the Author properties of the CodeTrackingData class. This ArrayList is then stored away as Session state so that when the Details button (displayed in the first column) in the DataGrid is clicked, the ArrayList stored in the session variable can be used for retrieving more details on the selected item.

// Called by 'Track Source' button's click event
public void ButtonTrackSource_Click(object Source, EventArgs e)
{
// Introspect the class using Reflection and populate
// the DataGrid
LoadTrackingData();
}

// Called by DropDownList box's SelectedIndexChanged event
public void DropDownListClasses_SelectedIndexChanged(object sender,
System.EventArgs e)
{
if(DropDownListClasses.SelectedItem != null)
{
// Introspect the class using Reflection and populate
// the DataGrid
LoadTrackingData();
}
}

// Associates a data source containing the
// Source tracker attribute metadata with the
// DataGrid control.
private void LoadTrackingData()
{
try
{
TaskGrid.DataSource = TrackingDataSource;
TaskGrid.SelectedIndex = -1;
TaskGrid.DataBind();
}
catch(Exception ex)
{
ErrorLabel.Text = ex.Message;
}
}

// Property that returns the ICollection interface
// of the data source that will be bound to the
// Task DataGrid
public ICollection TrackingDataSource
{
get
{
// Load the Assembly
Assembly assem = Assembly.Load(AssemblyNameText.Text);

// Get the type information for the class chosen in the
// drop-down list
Type type= assem.GetType(DropDownListClasses.SelectedItem.Text,true);

// Get back an array containing objects that describe
// the attribute metadata
ArrayList trackingArray =
SourceCodeAttributeIntrospector.ExtractAttributeMetadata(type);

// Stash it away as session state
Session["TRACKER_ARRAY"] = (ArrayList)trackingArray;

// Return the Data Source for the DataGrid
// (Returns the ICollection interface of the ArrayList
// containg the CodeTrackingData objects)
return (ICollection)trackingArray;

}
}

Now, since the DataGrid control is populated with the available methods in the class and their corresponding authors, the next thing a user might want to do is to browse through the items available in the DataGrid to get more information on the listed item such as the Last Modified Date, Reason for Modification, Status of the task and so on. The user does this by selecting the Details button in the first column of the DataGrid. When the Details button is clicked, the SelectedIndexChanged event is raised, which is routed away to the OnDisplayTaskDetails handler in the code-behind assembly.

The OnDisplayDetails handler gets the index of the currently selected item in the DataGrid and retrieves the corresponding KeyField. You will notice that the <asp:datagrid> element in the ASP.NET page has a DataKeyField attribute that maps the DataGrid's row index to the actual primary key data value in the DataSource. In our example, we use the CodeTrackingData class' Index property as the primary key in the DataKeyField. You can use this index to retrieve the corresponding CodeTrackingData object from the ArrayList representing the DataGrid 's data source that you stored away as Session state earlier. Once you get the CodeTrackingData object corresponding to the currently selected item in the DataGrid, you can display the rest of the metadata values such as the Date of Modification, Reason Modified, Author's email address and the Status of the task in the ASP.NET Web controls laid out in a HTML table below the DataGrid control.

// Called when the a new item is selected in the DataGrid
// (Invoked when the 'Details' button is clicked)
protected void OnDisplayTaskDetails(object sender, EventArgs e)
{
// Get the selected item index
int nSelectedTask = TaskGrid.SelectedIndex;
if(nSelectedTask != -1)
{
// Get the primary field value of the data source
// at the selected item position
int nIndex = (int)TaskGrid.DataKeys[nSelectedTask];

// Retrieve the object that contains the metadata for
// the selected index. You need to retrieve the object from
// the ArrayList that we stashed away as session state
ArrayList trackingArray = (ArrayList)Session["TRACKER_ARRAY"];

CodeTrackingData trackingData =
(CodeTrackingData)trackingArray[nIndex];

// Update the details of the tasks in the <TABLE>
// located below the Data Grid.
UpdateFields(trackingData);
}
}

// Updates the Labels below the DataGrid with the details
// based on the DataGrid's currently selected index
private void UpdateFields(CodeTrackingData trackingData)
{
MethodNameLabel.Text = trackingData.MethodName;
LastModifiedLabel.Text = trackingData.LastModified;
ModifiedReasonLabel.Text = trackingData.ReasonModified;
HyperLinkMail.Text = trackingData.Author;
HyperLinkMail.NavigateUrl = "mailto:" +
trackingData.AuthorEmailAddress;

if(trackingData.Status ==
IMPLEMENTATION_STATUS.FunctionallyComplete)
{
StatusLabel.Text = "Functionally Complete";
StatusImage.ImageUrl = "FullyComplete.gif";
}
...........
}

You can create a VS.NET Visual C# Class Library Project for the code-behind C# assembly SourceTrackerClient.dll. This assembly contains:

  • SourceTrackerClient class definition that inherits from System.Web.UI.Page representing the ASP.NET page that uses this code-behind assembly,
  • SourceCodeAttributeIntrospector class definition and
  • CodeTrackingData class definition.

You would need to reference the SourceCodeTracker.dll that contains the definition of the SourceCodeTracker attribute in your project before compiling it.

Alternatively, you could use the following command from the DOS command-line:

csc /out:SourceTrackerClient.dll /target:library
/r:System.dll,System.Web.dll,SourceCodeTracker.dll SourceCodeTrackerClient.cs

To test this page, you need to create a virtual directory and add the default.aspx file to the physical directory pointed to by the virtual root. You'll also need to copy the following into the bin sub-directory of application's root:

  • SourceCodeTracker.dll (which defines the SourceCodeTracker attribute),
  • The compiled C# code-behind assembly SourceTrackerClient.dll used by the ASP.NET page,
  • Any assemblies that you want to query for attributes (such as the AirCombatSimulator.dll class='codeintext'>

    Alternatively, you can place assemblies such as the AirCombatSimulator.dll in the Global Assembly Cache (GAC) so that the Assembly resolver will not choke when trying to locate the assemblies for loading.

    Conclusion

    In this article, we took a look at how attributes were used in the pre-.NET era in type definition languages like IDL to help tools like MIDL acquire information on the properties and characteristics of types. We saw how attributes are first class citizens with well-defined programming constructs in the .NET framework. We looked at how predefined attributes are being used extensively in the .NET framework to provide information to introspection tools, utilities and applications. We then took a dive into custom attributes and learnt how to define a custom attribute class in C# and saw how these custom attributes could be tagged to your types to associate specific properties with them. Then we used reflection to query these types at runtime for the emitted metadata values. We put all the concepts into practice by defining a trivial custom attribute called the SourceCodeTracker attribute that helps keep track of the status of specific tasks assigned to developers.

    Finally, we built an ASP.NET application to build a presentation layer over these pieces to see how they fit together. Hopefully, this article helped you understand the role attributes play in the .NET framework, and taught you how to define and use custom attributes in your applications. You might want to put in some effort to extend this trivial SourceCodeTracker attribute into a more well defined attribute that could be used in your projects to enable automated status tracking. The usage scenarios for attributes are limitless and you can dream up hundreds of innovative ways to put your .NET types on steroids using custom attributes.


Windows Explorer in C#

Windows Explorer in C#
http://www.csharptoday.com/content.asp?id=1614
Article Rating: 3
Published on July 9, 2001
Abstract
Windows Explorer is an application that explores your machine and displays files and folders. This application simulates the Windows Explorer that comes along with Windows OS. As we all know, Windows explorer manages the folders and files in your machine. In this article, K.Niranjan Kumar shows us how to create a Windows Explorer application using C# and the Visual Studio.NET Beta 1 IDE.
Article

Introduction

Windows Explorer is a dialog based application developed using C# with the Visual Studio.NET Beta 1 IDE. This application explores your machine and displays a list of your files and folders. It simulates the Windows Explorer that comes with the Windows OS. A screen shot of this application can be seen in Figure 1. As we all know, Windows explorer is used to manage the folders and files in your machine. The application is divided into two sections; on the left side there is a tree view control, which displays drives and folders, and the right side contains a list view control, which displays the files in the selected folder. On selecting a drive or folder in the tree view control, the application refreshes the list view control, showing all the files and sub-folders it contains.

Figure 1.The Main Application window.

Developing the Application

Requirements

In order to build such an application, we need to have a form that contains a tree control on its left side, and a list control on its right side. The tree control should display your drives and folders when the application is started. The listview control will display files when a particular folder or drive is selected.

Designing Windows Explorer

First and foremost of any GUI based application will be the designing of the windows and controls. We will design our window with the Form Designer using the following steps:

  • Open Visual Studio.NET IDE.
  • Create a new Project, select C# Windows Application. This automatically creates a new window.
  • Change the properties of the window to make it Dialog based.
  • Set the BorderStyle to FixedDialog.
  • Set MaximizeBox and MinimizeBox properties to False.
  • Set StartPosition to CenterScreen.
  • Set Title Property to "Windows Explorer ". The Properties window for our Form is shown in Figure 2.

Figure 2: Form Properties Window

The user interface for our application will contain a TreeView control to display drives and folders on left side of the main window, then a listview control on right side of the window to display files.

Creating child controls

To create controls for our application, open the Toolbox window then go to the Form designer. Select the form, click View menu and select Toolbox option or press Ctrl+Alt+X key to open Toolbox window (shown in Figure 3). In the Toolbox window, select the Win Forms tab, where you can see windows controls. Here, drag the TreeView control and place it on the left side of our window. Similarly, we need a ListView control for listing files. Drag a ListView control and place it on the right side of our window. The other controls we need to create are a Main Menu, ToolBar, ToolTip, StatusBar, ImageList for TreeView, ImageList for Toolbar. ImageList and ToolTip are invisible controls, so they are placed in a separate pane at the bottom of the Form Designer.

Figure 3: Toolbox Window

The TreeView control will be associated with an Imagelist. We need to create separate icons for folders, drives, floppy drives and the root node. Set the Imagelist property of TreeView control to select the Imagelist. Each Image will have a unique index in the ImageList control.

Files will be displayed on the right side of the window. We will also display the properties of the files listed, such as size, date of creation and date of last modification along with the file name. Therefore, we need to change the View property of ListView control to Report. Set the property FullRowSelect to true.

Next, we need to create a Main menu and place it on the top of our main window form. The following items will be added to menu File -> Close. View -> Refresh and Help -> About. Associate this menu to our main window. To do this, go to the Form property window, select the property Menu and select your menu from the combo box. Now we have attached a menu to the main window form.

After creating the main menu, place a Toolbar control below it. Then create a new Imagelist for the toolbar. Add Images to the Imagelist by clicking on the Collections button in the property window of the Imagelist. Set the Imagelist property of the toolbar to the newly created one. At the bottom of the window, we will have a status bar, which will display the number of folders and files. Create two new panes in the status bar by selecting the Status panes property then set the Show panes property to true.

Create a new ToolTip control and name it m_tooltip. As soon as you create a tooltip control, you can see a property Tooltip on m_tooltip property for all controls. Give it an appropriate message in this place for all controls. The message that you give here will be displayed as tool tips when you move your mouse over a control. The properties of the tooltip such as duration of time for the tool tip to be displayed etc., can be changed in ToolTip property window.

Adding drives and folders in TreeView control

Now we have finished designing our application, we have to write code to populate our controls to see it in action. The first thing we will do is to create a root node with the caption "My computer". We do this by calling the Add() method of the TreeNode class. The following code shows us how to add a node to a treeview control. m_tnRootNode is a member variable of our window class. It is the first node to be added to the tree control; therefore it is called the root node. Here we are creating the TreeNode object and adding it to the tree control. The first parameter "My computer" is the text associated with the node, while the second parameter is ImageIndex and finally, the third parameter is SelectedImageIndex for this node. ImageIndex property uniquely identifies an image in ImageList that is in turn associated with TreeView control. It specifies which image is to be associated with TreeNode object when the node is not selected. On the other hand SelectedImageIndex property associates an image with a selected TreeNode.

If you want a different image to be shown when a TreeNode is selected, you need specify an index by changing the SelectedImageIndex property. In this example, we will not show a different image for the root node; therefore we will give the same value, 3, for the second and third parameters. This is shown below:

m_tnRootNode = new TreeNode("My
            Computer",3,3);
m_tvFolderView.Nodes.Add(m_tnRootNode);

After creating the root node, the control is passed to PopulateTreeView(), where we determine available drives. In order to determine available drives, we will call the function GetLogicalDrives() that returns a string array. This function is available in Environment static class under System namespace.

string[] l_arrAvailableDrives = Environment.GetLogicalDrives();

We will create a node for each drive under the root ("My computer") and then call the recursive function PopulateFolders() up to 'n' levels to display the folders inside each drive. Therefore, PopulateFolders() will be called recursively for every folder inside a drive and returns when it finishes adding all folders to the tree node. This way, it adds all folders and sub-folders under a drive to the TreeView control. The following code shows how to create a node under the root node.

/*Create the Drive Root node */
TreeNode l_tnDriveNode = new TreeNode(l_strCurrDrive, l_iImageIndex, l_iSelectedIndex);

The first parameter (l_strCurrDrive) is the text of the node, the second parameter (l_iImageIndex) is the image index and third is the selected image index(l_iSelectedIndex). The following code shows how to add our currently created node (drive node) to root node.

/*Add the Node
            */
l_tcDriveCollection.Add(l_tnDriveNode);

In the treeview control we always represent data in the form of either parent or children. In order to create a child node in treeview, we need to get the Nodes collection for the parent node. This is achieved using the Nodes property of TreeNode class, which returns a TreeNodeCollection object, adding a node to this object will add a child item to our parent node in the tree control. Therefore, we are getting the TreeNodeCollection object from current node like follows:

/* Create a list for this root node */
TreeNodeCollection l_tcFileCollection = l_tnDriveNode.Nodes;
PopulateFolders(l_strCurrDrive,l_tcFileCollection);

When we get through all the elements in the array, we call the PopulateFolders() function to display folders inside the current drive. We will not populate the floppy drive, so we will neglect it and populate the rest of drives. PopulateFolders() takes two parameters; the first one is the current drive or folder path, the second being the collection to the parent folder.

Inside the PopulateFolders() function, we need to add all the folders under the current drive. This is done using the Directory class in the System.IO namespace. The Exists property of the Directory class will return true if the directory exists, otherwise it returns false. We will check whether this directory exists or not by using this property.

/*Create a dir object */
Directory
            l_curDir = new Directory(l_strCurrDir);

/*Get sub directories
            */
if ( l_curDir.Exists == false ) {
//give a error message
            and return
}

The GetDirectories() function returns a Directory array that contains the list of sub directories under the current directory. Using a foreach loop construct we get through all the directories and add it to our parent directory. In such a case we are checking whether the name of the folder is "RECYCLED" or "RECYCLER". If the Recycle bin is configured for multiple drives other than the ' C:\ ' drive, the name of Recycle bin folder will be "RECYCLER ". These directories represent machine's recycle bin, so we are associating a different icon to be displayed for this node.

Directory[] l_childDirs = l_curDir.GetDirectories();

The two lines of code in the PopulateFolders() adds node to the current parent node.

TreeNode l_curNode = new
            TreeNode(l_strDirName,l_iImageIndex,l_iSelectedImage);
l_tcCollection.Add(l_curNode);

If the currently added node is a directory, then we need to add the sub directories of the current directory as child nodes. Calling the function recursively by passing the directory path and Nodes collection does this:

PopulateFolders(l_strFullDirPath,l_curNode.Nodes);

When the function returns back out of all calls, we will have our TreeView populated with all directories in the current drive. Calling the same function in a loop for all drives will result the TreeView to be populated with all drives and all folders.

Displaying files in ListView control

When a folder or drive is selected in the Tree control, we have to display the files in the ListView control. For this we need to trap the AfterSelect event of TreeView control. This is where we will list all files in our ListView control. To add this event, go to design view of the window, then select the TreeView control and press F4 to display the property window. In the property window select the Event tab, then select the AfterSelect event from the left side of the grid control and give a name (OnSelectFolderView is the name I have given here) to the function. That's it, you have added an AfterSelect event to the TreeView control. After adding this you will have a function looking something like this:

protected void OnSelectFolderView (object sender, System.WinForms.TreeViewEventArgs e)

The first parameter, sender, represents the object that has initiated this event. In our case it is the TreeView control. We are only bothered about TreeViewEventArgs at this moment. This object contains a property node that represents the currently selected node. Text property of a node gives us the text associated with the current node. If the user selected the floppy drive, we call PopulateFolders() passing the TreeNode object, to populate the folders of 'A:\' recursively. If the user has selected some other folder, then we call the function PopulateListView() passing the current tree node.

Before inserting any item to the ListView control, we need to add columns to it. Remember that our ListView control is made as a Report view. In a report view, we have to add columns before the items are added. The first parameter is the index number, next the text, then the width in pixels and then the alignment of the text. See the ListView control in Figure 1.

m_lvFileView.Clear();
m_lvFileView.InsertColumn(0,"Name",225,HorizontalAlignment.Left);
m_lvFileView.InsertColumn(1,"Size",70,HorizontalAlignment.Right);
m_lvFileView.InsertColumn(2,"Created",100,HorizontalAlignment.Left);
m_lvFileView.InsertColumn(3,"Modified",100,HorizontalAlignment.Left);

The Clear() function is called before adding columns. This will clear any columns that have been added to our ListView control previously.

Inside PopulateListView() we get the full path of the currently selected folder using the property FullPath of TreeNode object. This property also returns our root node text My Computer. Therefore this text is removed in the next line of code.

string l_strCurPath = l_tnCurrNode.FullPath;
l_strCurPath = l_strCurPath.Remove(0,12);

A Directory object is then created and checked whether it exists, then the GetFiles() method is called which will return an array of File object. Using a foreach loop, each element of returned array is traversed and added to ListView control.

File[] l_curDirFiles =
            l_curDir.GetFiles();
foreach (object obj in l_curDirFiles)
            {
//Add to list view
}

Items in the ListView control are added using InsertItem() method of ListView class. InsertItem takes item index and ListItem object. ListItem object represents the list view item. We create a new ListItem object by passing a name, image index (0 in our case) and sub items to it. Sub items are the one that will be displayed in other columns (e.g. size, created date, modified date).

ListItem l_liItem = new
            ListItem(l_strFileName,0,l_strSubItems);
l_liItem.ImageIndex =
            0;
l_liItem.StateImageIndex = 0;
//Add item to ListView
            control
m_lvFileView.InsertItem(l_iCount,l_liItem);

Activating your menu items

Now we have finished the main part of our application. We still have to activate our menu items and associate the menu with the ToolBar control. First we need to add handlers for menu items. To do this, select your menu, press F4 for property window of menu (see to Figure 4a.) Now select the event tab in the property window (see Figure 4b.) Select the click event on the left side of grid and give a name to the event handler.

Figure 4(a): Menu Property Window

Figure 4(b): Menu Event Window

Name the File->Close() click
            event as OnClickClose(). Then add the
            following code, where Close() is the
            method of Form class, which closes the window. In our case it closes
            the application.

this.Close();

Next for View->Refresh menu item, add a handler for the click event and call the PopulateTreeView() method which will refresh all the folders and files in the system. Help->About will invoke our version information dialog. First we need to create the window and then call the ShowDialog() method of the Form class to show the window as a modal dialog.

AboutDlg l_dlgAbout = new
            AboutDlg();
l_dlgAbout.ShowDialog();

Attaching your menu to ToolBar control

The main purpose of having a toolbar in any application is to access the menu items easily with an icon based interface. Now we will see how to associate a menu item to a toolbars control's button. For this we have to add the ButtonClick event of Toolbar, choose OnToolBarClick() as the function name to this event. Then we will have a function with two parameters. The second one is what we are interested in. It takes an object of type ToolBarButtonClickEventArgs. This object will represent the button on which the event has occurred. This object has a property called button. This represents the button itself. ImageIndex property of button object will give you the Image index of the ImageList associated with the Toolbar control. This tells us that we have clicked the button. Therefore, the image at index zero represents the first button and so on.

OnClickClose() is the function
            associated with File->Close menu
            item and RefreshFolders() is
            associated with the View->Refresh
            menu item. Finally, OnClickAbout() is
            associated with Help->About menu
            option. All this is shown in the following code:

protected void OnToolBarClick (object
            sender, System.WinForms.ToolBarButtonClickEventArgs e){
if (
            e.button.ImageIndex == 0 ) {
/* First button clicked, close the
            window */
OnClickClose(sender,e);
}
else if (
            e.button.ImageIndex == 1 ) {
/* Second button clicked, Refresh
            folders,files*/
RefreshFolders(sender,e);
}
else if (
            e.button.ImageIndex == 2 ) {
/* Show about dialog
            */
OnClickAbout(sender,e);
}
}

Coding the LinkLabel control

Another interesting control in .NET framework is the LinkLabel control. This control can be used as a link to a web page or to e-mail. After placing this control on your form, add a LinkClicked event to it. Add the code, which is shown below, to the event handler to take you to Microsoft's home page:

m_linkLableEmail.LinkVisited = true;
System.Diagnostics.Process.Start("http://www.microsoft.com");

Setting LinkVisted property to true will change the color of the LinkLabel control like we normally get when we visit a web site. To open your mail client, you have use mailto protocol, for example:

m_linkLableEmail.LinkVisited = true;
System.Diagnostics.Process.Start("mailto://someone@somewhere.com");

Summary

Now we have finished the Windows Explorer application. There are many enhancements that can be made to this application such as creating folders, opening files when we double click list the control item and drag and drop feature etc. These will be left as an assignment to the readers of this article.


Most Recently Used File List Helper In C#

Most Recently Used File List Helper In C#
http://www.csharptoday.com/content.asp?id=1597
Article Rating: 3.5
Published on June 27, 2001
Abstract
In this article, Rob Birdwell presents an object-oriented approach to appending an MRU (Most Recently Used) file list to the menu of any WinForm application in C#. This implementation will take advantage of some of C#’s powerful features: delegates, events, properties and serialization, and presumes at least a cursory knowledge of .NET and C#. After reading this article, you will be able to incorporate this MRU file list implementation into your own applications and even extend it to suit your own needs if you so desire. Regardless of your current programming experience, language preference or bias, there’s bound to be something of interest for you here – so read on!
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

In this article, I will present an object-oriented approach to appending an MRU ( Most Recently Used ) file list to the menu of any WinForm application in C#. This implementation will take advantage of some of C#'s powerful features: delegates , events , properties and serialization . This article presumes at least a cursory knowledge of .NET and C#, however, C++, VB.NET and Java programmers will find much of interest here. We will cover an example of writing and building a .NET Library (.DLL) that can be used by a WinForm, and we will work with several different namespaces available in the CLR ( Common Language Runtime ) of the .NET Framework . After you've read this article, you will be able to incorporate this MRU file list implementation into your own applications and even extend it to suit your own needs if you so desire. Regardless of your current programming experience, language preference or bias, there's bound to be something of interest for you here - so read on!

My First Attempt

A few years ago, when I was writing Windows apps in C++, the product features team requested that I add an "MRU list" to our application's main menu, an application that is typically used to append a list of recently used files to one of the menu items (usually "File"). I quickly came to the conclusion that this would indeed be a useful feature in our application. I gave the team an estimate of "a few hours, or so" and set out on integrating this seemingly trivial feature.

Unfortunately, my MRU file list implementation was identified by one of the testing leads ("Eagle Eye Dan") as the culprit responsible for introducing a series of new bugs into our application. Most notably:

  • The MRU file list wasn't displaying the menu items in a consistent order - this was mostly just irritating, but still needed fixing.
  • Files that no longer existed on the system remained on the MRU list, even though our application wasn't setup for this kind of scenario - which caused some very cryptic warning messages to be displayed. This was serious since it made our application look less than professional.
  • I'd introduced a memory leak by not freeing up one of my array allocations!

These bugs were fixed, then a couple weeks later I was asked to implement the same MRU file list feature in another application! Unfortunately, I never anticipated the "reusability" aspect of my original MRU implementation - it was truly spaghetti code, liberally woven into the fabric of the existing application code. Although I was able to integrate my previous implementation of the MRU file list into the new application, it certainly wasn't very pretty.

C# and .NET - It's Getting Better All The Time!

But let's look beyond my past coding foibles! It's a different world now. I'm a little older and wiser - plus I have C# and the CLR to leverage from. When new technologies emerge, like C# and .NET, it's only natural to look back on previously built components and evaluate if they could be implemented any better. Since there was clearly room for an improved MRU implementation, I set out on designing an MRUHelper namespace in C# that would redeem my checkered past with previous MRU efforts. The class design is fairly straightforward:

Overview of MRUHelper Classes

  • MRUList - provides the core logic/functionality and manages the ArrayList of items - not necessarily file names. This class also has methods to add and remove items from the list, delegate feedback, post events, and some very simple serialization methods.
  • MRUFileList - deriving its core functionality from MRUList , this class extends the MRU concept to "files" by optionally preventing files from being added to the list if they don't exist on the current file system. Both MRUList and MRUFileList are GUI independent - in other words, they don't know anything about MenuItem or WinForm objects.
  • MRUMenuFileList - this class extends the base functionality of MRUList and MRUFileList by rendering the list to a WinForm menu. It also handles the details of posting messages back to the WinForm to notify it when menu items have been added, removed and/or selected.
  • MRUEventInfo - derived from the CLR's EventArgs class, an instance of this class is sent whenever an MRU event is posted.
  • MRUMenuItem - extends the CLR's MeuItem class to pass the actual file name of the selected menu item.

MRUHelper Core Functionality

I'm not aware of any hard and fast rules as to what constitutes a "standard" MRU implementation. But generally, MRU implementations abide by most, if not all, of the following guidelines:

  • The most recently used item should always appear first on the list.
  • There should be a maximum limit of items that are managed - typically four, but sometimes more or less.
  • Items that aren't used eventually "fall off" the list and are replaced by more recently used items.
  • If the items happen to be files (the most common case) they should not be allowed on the list if they no longer exist on the file system.

The MRUHelper namespace handles all of the above guidelines and provides the following extended functionality:

  • Manages the details of appending and updating MRUMenuItem objects on the corresponding menu.
  • Implements basic serialization methods to save/load the MRU list data. This functionality is "virtual" which means that it can (and probably should) be overridden.
  • Provides event notifications whenever MRU items are added, removed or selected from the list.
  • Ability to alias or abbreviate long file names on the menu. This feature can also be overridden to conform to any graphical size limitations of the menu.

In the sections to follow, we'll take a closer look at various parts of the MRUHelper code to see how everything fits together. The complete source code for the MRUHelper namespace is available for download - see the related links section below. You will, of course, need the latest version of Microsoft's .NET Framework to compile and run the related source files.

Feedback - Let's Delegate It!

One of the first decisions I made when laying out my MRUHelper implementation was that I wasn't going to hardcode Console.Write statements throughout the code. (For the non-.NET audience, the Console.Write .NET method can be used to output formatted text to the console or screen - similar to the printf statement in C/C++). I wanted to provide a way for the application interacting with the MRUHelper classes to be responsible for displaying any MRUHelper feedback "their way" - whether that meant displaying popup message boxes, outputting feedback to a status bar, outputting text to the console, or ignoring feedback completely, it didn't matter. This was clearly a job for the C# delegate functionality.

Delegate Diversion

Delegates in C# are very similar to function pointers in C++, however C# delegates are much easier to use and type-safe. Here's an easy way to conceptualize delegates. First, imagine you have a job in mind, something you'd rather give (or "delegate") to someone else. We could express this scenario using a delegate, in a diversionary hunk of sample code, like this:

// Delegate sample code...
public class
            MyJobs
{
// declare a function as a delegate
// we won't
            do this work, of course!
public delegate void
            TakeOutTheTrash();

// now declare an object of the same type
// to serve as the delegate placeholder
public
            TakeOutTheTrash trashHandler;

// Now pretend we have some
            dirty work to do.
public void DirtyWork()
{
// ah, we have
            to do some dirty work
// let's ask someone else to do this:
if
            ( trashHandler != null )
trashHandler ();
}
}

//
            Meanwhile, say we have a class
// that specializes in
            trash...
public class TrashToTreasures
{
//
            constructor
public TrashToTreasures()
{
MyJobs j = new
            MyJobs();
j.trashHandler =
new
            MyJobs.TakeOutTheTrash(Trash2Money);
j.DirtyWork();
}

//
            This method gets the call!
public void
            Trash2Money()
{
Console.WriteLine ("Trash equals
            $$$$!");
}
}

Delegates in MRUHelper

The following excerpt shows how a "Feedback" delegate was setup in the MRUList class implementation:

...

public enum MRUFeedbackType
{
ERROR,
WARNING,
INFORMATIONAL,
};

public class MRUList
{
...

public delegate void MRUFeedback ( string sMessageText,
MRUFeedbackType feedbackType );
protected MRUFeedback m_FeedbackDelegate;
...
public MRUFeedback Feedback {set { m_FeedbackDelegate = value; }}
...

In the above listing, the MRUFeedback delegate is declared with two arguments: a string to pass the message text, and an MRUFeedbackType enum to distinguish what type of message is being sent. A protected member, m_FeedbackDelegate , of type MRUFeedback , is declared to serve as the placeholder for the method. Additionally, a Feedback property is declared for a more intuitive public access.

Since the MRU feedback functionality is delegated, the MRUList class doesn't actually do the work of displaying the feedback; it just calls the delegate method (so long as it's not null) and lets that handler deal with the display issues. Here's how the MRUTestForm (the WinForm application that implements the MRUHelper ) registers its own feedback handler to display any MRUHelper feedback to a ListBox control:

namespace MRUTestForm
{
...
using MRUHelper;

public class MainForm : System.Windows.Forms.Form
{
...

private MRUMenuFileList m_MRUMenuFileList;

public MainForm()
{
...
InitMRUHelper();
}

private void InitMRUHelper()
{
...
m_MRUMenuFileList.Feedback =
new MRUList.MRUFeedback ( OnMRUFeedback );
...
}

public void OnMRUFeedback ( string sMessageText,
MRUFeedbackType feedbackType )
{

switch ( feedbackType )
{
case MRUFeedbackType.ERROR:
feedbackListbox.Items.Insert( 0,
"ERROR Message: " + sMessageText );
break;

case MRUFeedbackType.WARNING:
feedbackListbox.Items.Insert( 0,
"WARNING Message: " + sMessageText );
break;

case MRUFeedbackType.INFORMATIONAL:
feedbackListbox.Items.Insert( 0,
"INFORMATIONAL Message: " + sMessageText );
break;

default:
break;
}

...

The feedback delegate handler has complete control over how MRUHelper feedback will be displayed. Of course, it's totally optional to write a feedback delegate handler, but it's easy to implement and serves to demonstrate the informational messages the MRUHelper classes generate.

Event Processing

In order for the MRUHelper classes to interact with other modules or applications (such as a WinForm ), I used another C# language construct: Events . Events are very similar to delegates and are typically used to signal a state change to any process that might be interested. This is particularly useful for the MRUHelper since events will signal when the MRU list changes:

The MRUList base class implements and posts two events: OnAddItem and OnRemoveItem . The derived class MRUMenuFileList implements and posts an OnSelectItem event. Since the MRUList class doesn't really have a concept of "selection", the OnSelectItem event is better suited, I believe, for the MRUMenuFileList implementation.

My goal with MRUHelper was to design it so that any WinForm application could simply tell the MRUHelper what menu to work with (typically "File") and let the MRUHelper classes handle the details of displaying the MRU items on the menu, formatting the text, and handing the menu selection state. There was, however, a minor dilemma to resolve: If the WinForm doesn't actually create or manage the items on its own menu, how does it know when a menu item is selected? Again, events come to the rescue!

The following code excerpt shows how the MRUMenuFileList class implementation declares and fires off an OnSelectItem event to notify the WinForm:

...

public class MRUMenuFileList : MRUFileList
{
private MenuItem m_parentMenuItem;
private ArrayList m_MRUMenuItems;
public event MRUEvent OnSelectItem;

public MRUMenuFileList ( MenuItem parent,
bool bFilesMustExist, int iMaxItems )
: base ( bFilesMustExist, iMaxItems )
{
m_parentMenuItem = parent;
m_MRUMenuItems = new ArrayList();

// Register for notification events....
OnAddItem +=
new MRUList.MRUEvent(OnMRUItemAdded);
OnRemoveItem +=
new MRUList.MRUEvent(OnMRUItemRemoved);
}

protected void OnMRUMenuItemClicked (object sender,
EventArgs info)
{
if ( OnSelectItem != null )
{
// Sender is always a MRUMenuItem...
MRUMenuItem mi = (MRUMenuItem)sender;

// Fire off an event to anyone interested -
// the main form should be interested in knowing
// their own menu item was selected - but since we
// created it dynamically we need to tell them!
MRUEventInfo eventInfo
= new MRUEventInfo ( mi.MRUItemValue,
this,
MRUEventType.SELECTEDITEM );
OnSelectItem ( this, eventInfo );
}
}
...

public override void Refresh ()
{
...
// re-create the MRU menu items
for ( int i=0; i<m_MRUList.Count; i++)
{
...
MRUMenuItem mi =
new MRUMenuItem (sMenuText,
sItemValue,
new System.EventHandler(OnMRUMenuItemClicked) );
m_parentMenuItem.MenuItems.Add (
m_parentMenuItem.MenuItems.Count, mi );
m_MRUMenuItems.Add (mi);
}
}

Notice that the MRUMenuFileList class doesn't instantiate the OnSelectItem event - it's up to the WinForm to create and register its own event handlers for all available MRUHelper events. The OnMRUMenuItemClicked event method handler is assigned when the virtual method Refresh is called. Although the MRUTestForm implementation is really just a shell application, it demonstrates how to register for MRUHelper events and setup the respective event handlers:

...

public class MainForm : System.Windows.Forms.Form
{

...

public MainForm()
{
...
InitMRUHelper();
}

private void InitMRUHelper()
{

...

m_MRUMenuFileList.OnAddItem
+= new MRUList.MRUEvent ( OnMRUItemAdded );
m_MRUMenuFileList.OnRemoveItem
+= new MRUList.MRUEvent ( OnMRUItemRemoved );
m_MRUMenuFileList.OnSelectItem
+= new MRUList.MRUEvent ( OnMRUItemSelected );

...

}

public void OnMRUItemAdded ( object sender, MRUEventInfo info )
{
UpdateCurrentFeedbackList();
}

public void OnMRUItemRemoved ( object sender, MRUEventInfo info )
{
UpdateCurrentFeedbackList();
}

public void OnMRUItemSelected ( object sender, MRUEventInfo info )
{
DialogResult dr =
MessageBox.Show ( "You selected : " +
info.itemName +
" - should this be the MRU File?",
"Make Top MRU?",
MessageBoxButtons.YesNo );

if ( dr == DialogResult.Yes )
m_MRUMenuFileList.AddMRUItem( info.itemName );
}

private void UpdateCurrentFeedbackList()
{
// Update feedback list...
currentMRUListbox.Items.Clear();
for ( int i=0; i<m_MRUMenuFileList.Count; i++)
currentMRUListbox.Items.Add(m_MRUMenuFileList[i]);

if ( currentMRUListbox.Items.Count > 0 )
currentMRUListbox.SelectedIndex = 0;
}

...

Serialization

In most applications, the MRU settings would be merely one part of a collection of settings and options for the entire application. Settings are typically loaded when the application starts and saved, if necessary, either when the application closes down or immediately after a particular setting has changed. The details of how and where these settings are stored are, in my opinion, best left to the judgment of the application developer.

The MRUList base class supports two virtual methods to allow the MRU items to persist and reload when called upon. This is, however, a very simple serialization implementation. Keep in mind that Save/Load are virtual methods and can, and probably should, be overridden. Since there are a variety of ways an application might need to save its various settings, I intentionally leave it up to you, the developer, to decide the best method to Save/Load your own settings.

That said, the default Save / Load methods offered by the MRUList base class will work just fine for saving and loading MRU items to a self-contained file, and are particularly suitable for demonstration purposes. The following code demonstrates the MRUList class's simplified Save/Load methods:

public virtual void Load ( string sourceFile
            )
{
m_MRUList.Clear();
...
File f=new
            File(sourceFile);
Stream
            s=f.Open(FileMode.Open);
BinaryFormatter b=new
            BinaryFormatter();

ArrayList
            tempList=(ArrayList)b.Deserialize(s);
tempList.Reverse();

foreach
            ( object o in tempList)
AddMRUItem ( (string)o
            );

s.Close();
...
}

public virtual void Save (
            string sourceFile )
{

FileStream s=File.Open(sourceFile,
            FileMode.Create);
BinaryFormatter b=new
            BinaryFormatter();
b.Serialize(s,m_MRUList);
s.Close();

}

Application Startup

When the sample MRUTestForm application first loads, it performs some basic initialization and loads the MRU items accordingly:

private void InitMRUHelper()
{
...

// Load existing MRU settings, if any...
if ( File.Exists (
Application.StartupPath + "\\MRU.dat"))
{
m_MRUMenuFileList.Load (
Application.StartupPath + "\\MRU.dat" );
UpdateCurrentFeedbackList();
}
...
}

Refreshing the menu when items are added or removed takes place automatically. The MRUList.Load method calls the MRUList class's virtual Refresh method, which is overridden by the MRUMenuFileList class, to update the menu items, if any. This makes for one less detail the application needs to deal with.

Application Shutdown

Conversely, when the sample MRUTestForm shuts down, the MRUList's Save method is invoked as follows:

...

protected void MainForm_Closing (object sender, System.ComponentModel.CancelEventArgs e)
{
DialogResult dr =
MessageBox.Show (
"Would you like to save the MRU data to MRU.dat?",
"Save MRU data to MRU.dat?",
MessageBoxButtons.YesNoCancel );

if ( dr == DialogResult.Cancel )
{
e.Cancel = true;
return; // don't close down after all!
}

if ( dr == DialogResult.Yes )
m_MRUMenuFileList.Save(
Application.StartupPath + "\\MRU.dat" );
}

...

Obviously, a "real" application would probably not choose to prompt the user to save their MRU settings with a popup MessageBox.Show just before closing down, but for demonstration purposes, this should be seen as entirely appropriate.

Using MRUHelper

If you've followed along this far, you've probably noticed that the MRUHelper implementation is doing quite a bit of work for us. Finally, we can reap the benefits of this code base and use the MRUHelper in any WinForm application with very little effort. The following is a screen shot of the MRUTestForm :

The MRUTestForm is a standard .NET WinForm with a menu , listbox , splitter , and another listbox to display feedback. If you click on the " File " menu, you will see the following available sub-menu items:

  • Select a File - provides an open file dialog. No files are actually opened in the MRUTestForm but the MRUHelper tracks them.
  • Remove All MRU Menu Items - will prompt the user to remove the displayed MRU items from the sub-menu.
  • MRU Files Must Exist - if checked, only files that exist are allowed on the MRU menu - useful to test the behavior when files are added then removed from the file system later.
  • Exit - terminates the MRUTestForm and prompts to save the MRU files to a local file, MRU.dat .
  • MRU Menu Items - these are the menu items that are automatically appended to the bottom of the menu. By default, if the file length is longer than 30 characters, the text on the menu is abbreviated using a very simple algorithm. Of course, if a more complex abbreviation algorithm was desired (for example, to abbreviate the text based on the pixel width of the sub-menu), the GenerateFormattedMenuText method of the MRUMenuFileList class, which is declared virtual, could be overridden.

Source Code

The following files make up the source code used to build and test the MRUHelper library and the MRUTestForm application:

Code File | Purpose
MRUHelper.cs | Primary enum and class definitions
MRUMenuFileList.cs | Derived class for working with GUI
MainForm.cs | Sample WinForm implementation
makeall.bat | Compiles and builds the targets

Summary

I set out to make the MRUHelper classes re-usable and robust, and also to explore some of the possibilities of .NET and C#. Because the MRUHelper is compiled as a .NET Library (.DLL), it can be used in any .NET application or module, including an application written in VB.NET, the latest .NET incarnation of the Visual Basic language . For example, a VB.NET programmer could use and even extend the functionality of the MRUHelper classes in his or her own application, even though the MRUHelper was written in C#. This is possible because of the .NET Common Language Runtime, which is shared by all .NET languages, thus allowing true cross-language interoperability.

Hopefully, this MRUHelper offering will be useful to a wide audience in demonstrating not only how to implement an MRU file list, but also, and perhaps even more importantly, some of the basic C# constructs like classes , delegates , events , using CLR objects and how to create re-usable libraries that can, in turn, be quickly put to use in a WinForm. Thanks to C#, development projects like this are lots of fun - C# is, I believe, a great language to work with! I can't say that C# or .NET will completely save me from errors in logic or judgment, but I think even "Eagle Eye Dan" would agree - this implementation of an MRU file list is pretty cool!


Web Site Error Logging and Reporting System

C# Language

C# Language (Sort by Name)

.NET Code Generation with Visual Studio .NET Enterprise Architect
.NET Diagnostics
.NET Memory Management and Garbage Collection - Part 1
.NET Memory Management and Garbage Collection - Part 2
.NET Serialization
Advanced Features in C#
Advanced Techniques for InterThread Communication
An In-Depth Look at the DateTimeFormatInfo Class
An In-Depth Look at the NumberFormatInfo Class
An Open Invocation

Attributes in .NET
Beginner's Hangman
Building Windows Forms on-the-fly with XML and C#
C# to UML Reverse Engineering
Communication Between Multiple Threads
Data Binding in C#
Developing a Generic Thread Pool in C#
Efficient String Manipulations with the StringBuilder Class
Emitting Dynamic Assemblies on the fly using Reflection.Emit
Error Handling in C#

Exploring the System.Array Class
Exposing Type Information
Functionality and design of a thread pool class
Globalization in C#
Graphical Representation of Data Grids Using C# and GDI+
IEnumerable, IEnumerator, and Dictionaries
Introducing XGen XSQL for .NET
Java stream Vs C# Streams
Mapping Windows Forms to A Relational Database - Part 1
Mapping Windows Forms to A Relational Database - Part 2

Mapping XML to C# objects using reflection
Migrating from C++ to C#
Monitoring Processes and Threads in .NET
Multiple Thread Techniques for a C# Windows Service
Multi-threading In C#
Networking Support Provided by the .NET Framework
Object-Oriented Programming in C# - Introduction
Object-Relational Mapping with Reflection and Design Patterns
Owner Drawn Menus in Visual Studio .NET Using C#
Polymorphism in C# - Creating a Drawing Program

Programmatically Wiring Up Events in C#
Programming Web Services with C# Using SQL
Remoting in C# - Part 1
The Theory of Inheritance in C#
Thread Pools
Threading in the .NET Framework
Understanding .NET Remoting
Understanding Delegates in C#
Understanding Memory management in the .NET Framework
Using inheritance to create a MDI Child form framework in C#

Using Late Bound COM Objects
Using Legacy COM(+) in .NET - Part 1
WordPad in C#
Working with Web Data in C#
Writing a Multithreaded Port Scanner in C#
Writing Unsafe Code in C#


Mapping XML to C# objects using reflection

Mapping XML to C# objects using reflection
http://www.csharptoday.com/content.asp?id=1921
Article Rating: 4
Published on October 7, 2002
Abstract
Nowadays XML is the de facto standard for exchanging data between different systems and platforms, so providing a mechanism that allows us to import XML is very useful. In this case study, Giuseppe Naccarato will be using the XmlReader object and reflection to build a powerful general-purpose component for mapping XML to C# objects.
Case Study

Nowadays XML is the de facto standard for exchanging data between different systems and platforms, so providing a mechanism that allows us to import XML is very useful. In this case study, we will be using the XmlReader object and reflection to build a powerful general-purpose component for mapping XML to C# objects.

The XmlReader object is the standard way in which .NET applications read XML documents. Reflection enables a program to inspect and manipulate data. It can be used to find or create types at runtime, invoke methods dynamically, or to discover information about classes and their members. To end the piece, we will be looking at how the component can be used in a real-world situation. This will be an application showing a list of items and their stocks. It can import XML documents, which contain information about customer or purchase orders, in order to add new articles and/or update the stock level information. Developing this example, you can figure out how the proposed component works and why it is more powerful and flexible than .NET XML serialization.

Introduction

The idea is to develop a C# class which imports XML documents following a specified XML-schema. This class, by using the information coming from the XML, will be able to create class-instances using reflection. In addition, the data-members of the class will be filled with data contained in the XML. For example:

<class name="Person">
<field name="firstname" value="Robert" type="string"/>
<field name="surname" value="Hill" type="string"/>
</class>

The C# class, which imports this XML, will create an instance of the Person class where:

person.firstname = "Robert"
person.surname = "Hill"

Obviously that was a very trivial example. In more complex circumstances, the XML document could contain nested classes and array of objects. The developed C# component has to be able to cope properly with these situations as well.

At first glance, importing XML by using reflection might not seem to be the best strategy because the .NET platform already provides serialization for this purpose. Serialization is a very cool C#/.NET feature, and unlike Java, C# provides XML serialization, which serializes and deserializes objects to and from XML. In spite of this, there are some situations where serialization is less than adequate. An example of this is when we have to import information from an external system that is producing data in an XML structure that is different to the one used by .NET serialization.

To solve this problem we have two choices: transformthe source XML into an .NET serialization structure by using XSLT, or to create a component that reads the XML and builds C# objects from it. The first method can be an effective and powerful strategy, but you would have to choose the second way if you are also interested in the data-types. In fact, by using the .NET serialization structure, you can't specify the data-types. This is a significant limitation, as sometimes the data-type is an important issue that we cannot leave out.

This case study will cover many situations in which you have necessarily to import XML whose structure is already designed and can't be changed (perhaps because other systems have been using it for years and cannot be altered). In those situations, it is clear that serialization can't help us at all. In addition, unlike serialization, the component we will be developing can use the standard C# classes without adding any lines of code to make them "serializable".

System Requirements

This system requires Visual Studio .NET 1.0 or later running on Windows 2000 or Windows XP. The case study assumes that the reader has a good understanding of C#, XML and XML schema. An understanding of XmlReader objects and reflection would be beneficial, but not essential. You can find links to other case studies and articles on these topics in the related links section of this case study.

Defining The Problem

The focus of this case study is explaining a way to carry out the data exchange between two systems where the importer is implemented in C#, and the exporter provides data in XML following a specified XML schema. Importing data from external systems is a situation that occurs frequently in the real-world, and there are many different strategies for addressing these problems. Before XML, the data exchange was made by a common data-structure that both systems could understand. Data was written in a file (i.e. CSV files) or streamed in a socket following a specific proprietary protocol. The system importing the data was able to decode that protocol in a way that meant it could update the entities present in its database. To apply this strategy it was necessary to define the common data-structure, develop encoding and decoding algorithms according to the data-structure, and implement the code to import the data to the target system.

Today, thanks to XML, the strategies used to address this problem have become simpler. Now, it is no longer necessary to define the protocol for the data-structure, as that protocol is XML, nor do we need to define the encoding/decoding algorithms. In fact, most modern languages provide modules to read and write XML, for example SAX and DOM, both of which have been developed just for this reason. With .NET you can use the System.Xml namespace, which provides classes like XmlReader and XmlWriter, for reading and writing XML code.

The system reading the XML has to import and create the data according to its own design. Very often, in object-oriented systems, importing data means creating objects. Hence, the developer has to implement code for translating the XML elements in objects. This can be a tough job, because typically each class will have a different 'structure' (i.e. methods and attributes). For example, the Person class is sure to be radically different from the Invoice class. The first one would have fields such as name, lastname, and dateOfBirth. The second one on the other hand would have fields like number, date, and amount. That means that for each class you would have to implement different code to create the objects. We want to avoid that situation.

So, what do we want? We would like to create a C# general-purpose component that can read XML and use it to create C# objects, which belong to any defined class. The component must meet the following criteria:

  • it can create new instances of any predefined class
  • it can import any XML written according to a predefined XML schema
  • no changes must be made to the target class source
  • data-types must be supported.

This is what we will be developing in this case study. Going back to the serialization issue, you should be able to see that serialization does not meet all of the criteria above. Using serialization you can "create new instances of any predefined class", so the first point is met, but you can't "import any XML written according to a predefined XML schema". Serialization uses its own schema that could be incompatible with the one used by the exporting system. It is true that you can allow serialization to read different XML structures by using the C# attributes, but this would mean modifying the source code of the target class, thus meaning the third point would not be met. This is a very important point, as you might not have the option of accessing the source code of the classes (for example, the classes may be defined inside a dll developed by a third party). In addition, serialization does not support data-types at all.

Case Study Structure

The case study is structured as follows:

Component Design and Code

  • XML Parser and XmlReader objects.
  • Reflection and metadata
  • Mapping simple objects
  • Mapping complex objects
  • Mapping array of objects
  • Internal Reference

The sample application

  • UI Design and coding
  • Database Design
  • Importing XML
  • Application Setup

Conclusion

  • Review
  • Limitations and further work
  • Related links

Component Design and Coding

Now that the problem has been defined, let's develop our solution. To do this, we have to know about some important facilities that .NET provides. First, we have to understand the way in which .NET and C# deal with XML, and secondly we need to look at how we can dynamically create C# objects by using reflection.

The idea is to develop a C# class that reads the XML following a predefined XML schema. While it is reading the XML elements, it must be able to create the right C# class and fill its fields with the data coming from the XML. The name of the class, the fields, and the values are written in the XML, thus the component, which uses reflection, can dynamically create the instances and return them.

XML Parser and XmlReader objects

XML plays a fundamental role in the .NET framework. The framework not only provides modules allowing you to use XML in your application, but it uses XML itself for a number of reasons, such as configuration files and source code documentation. In addition, .NET-based technologies like ADO.NET and Web Services work and deal with XML. To allow programmers to use XML, the .NET framework provides the System.Xml namespace, which consists of a set of classes whose main purpose is to read, manipulate, and write XML code. The classes I'm going to use in this case study are System.Xml.XmlReader, and its derived classes such as XmlTextReader and XmlValidatingReader.

The XmlReader class is an abstract class that provides fast, non-cached XML data. XmlTextReader is a concrete implementation of XmlReader that supplies fast text-stream access to XML data. Using XmlTextReader is the standard .NET way of reading the contents of an XML document. The XmlTextReader follows neither the SAX nor the DOM philosophy. However, it is closer to SAX than DOM because like SAX, it parsers the XML and enables applications to process elements in the same moment that they have been read. Unlike DOM, the XmlTextReader object does not create an in-memory object model.

The way XmlTextReader reads the XML document is very similar to the way the contents of the tables in a database are read. The XmlTextReader seeks the XML elements and supplies them to the application together with their attributes. Usually, the content of an XML document can be seen as a tree. The XmlTextReader class implements a visiting algorithm for the XML tree based on the so-called node-first approach. This means that for each XML sub-tree found, the root is visited first followed recursively by all of its children, from first to last. Readings are made in a forward-only and read-only way; therefore, writing is not allowed and the visit cannot return back to nodes already visited.

Using the XmlTextReader class you can easily develop an XML Parser that checks if a document is well-formed, that is whether it respects the XML syntax or not. If you need to check the semantic meaning of an XML document as well, you have to use the XmlValidatingReader class. This is an XmlReader extension that allows us to specify the kind of validation the parser needs. Some of possible choices are: none, DTD, or XML Schema (XSD).

To implement an XML validating parser in C#, first we have to create the XmlTextReader object and the XmlValidatingReader object:

// XmlReader inizializzation
// the
            string filename is the XML file name
XmlTextReader txtReader =
            new XmlTextReader(filename);
XmlValidatingReader xtReader = new
            XmlValidatingReader(txtReader);

As you can see, you need both XmlTextReader and XmlValidatingReader instances, because XmlTextReader does not directly derive from the XmlValidatingReader class. To instantiate an XmlValidatingReader object you have to pass the XmlTextReader as a parameter to the constructor.

Now, you can set the validation type using the ValidationType property:

// Set the ValidationType to XML-Schema (XSD)
xtReader.ValidationType = ValidationType.Schema;

ValidationType is an enumeration. The possible values are: Auto, DTD, None, Schema, and XDR. In this case study we will always use the XML schema validation.

The next step is to define a validation event handler, which is basically a method that will be called by the framework when an error in the XML document occurs. To set a validation event handle we use:

// Set the validation handler
xtReader.ValidationEventHandler +=
new ValidationEventHandler (this.ValidationEventHandle);

The invoked function is this.ValidationEventHandle ; we will see its implementation in a little while.

At this point, we can call the Read method of XmlValidatingReader, which starts to read the document. The method returns each time an XML entity is found. By "entity", I am referring to elements, attributes, or tags. Thus, the xtReader object will contain all the information about the XML entity. The method returns false when there are no more entities, which will happen when the reading operation has ended. In this first example, I invoke the Read method until it returns false. The xtReader object information will not be used:

while (xtReader.Read())
            {
}
xtReader.Close();

The validation event handler implementation is as follows:

public void ValidationEventHandle (object sender, ValidationEventArgs args){
throw new System.Xml.XmlException(args.Message,null);
}

When an error occurs, the validation event handler will throw an exception whose message is derived from the args parameter.

In this case study I'm going to use the following properties and methods of the XmlReader class:

Type | Name | Description
Property | Name | Gets the qualified name of the current node.
Property | Value | Gets the text value of the current node.
Property | NodeType | Gets the type of the current node.
Method | MoveToAttribute | Moves the XmlReader object to the specified attribute.
Property | AttributeCount | Gets the number of attributes for the current element

The XMLImporter

If you assemble the code fragments above in a C# class you can get a simple XML validating parser. We want more though. We have to implement a class that will be used as the basic class for this entire case study. It has to be able to parse an XML document and delegate to a user-defined handler to process the XML entities it finds. What I'm going to develop is an XML parser that follows the design shown in the figure below:

This class diagram shows the XMLImporter class, which is our target, and its relationship with the other classes and interfaces. Basically, the XMLImporter is an XML-schema-based validating parser that reads (via the import method) the XML and calls a user-defined method each time an element requires processing. This method, called processElement, belongs to the interface Handler. As you may see from the diagram, the XmlImporter holds a Header object. The Header interface code is as follows:

public interface Handler {

// processElement is called by the XmlImporter
// each time an XML element is being processed

void processElement(XmlValidatingReader xtReader);

// processObject shall be called by the user
// to process an XML imported root object

void processObject(object obj);
}

The processElement method, as already mentioned, is called from the XMLImporter each time an XML element is found. The xtReader parameter represents the XmlValidatingReader, and stores all the information about the XML element. Consequently, you can implement this method in a way that means you can use the XML element for whatever purpose you wish. The processObject method will be explained in detail later. To briefly explain, it will be invoked after a C# object is created starting from the XML element. By implementing this method you can use the object for the purposes that we set out for the final application earlier in this case study. To use the XmlImporter, we must implement the Handler interface. However, a dummy implementation of Handler is provided; it is the DefaultHandler :

public class DefaultHandler : Handler
            {
public void processElement(XmlValidatingReader
            xtReader){
}
public void processObject(object
            obj){
}
}

The XmlImporter instances using the DefaultHandler are merely XML validating parsers.

In the class diagram you can see that XmlImporter is aggregated to a ValidationEventHandler. Even if this is represented as a Class in the diagram, the implementation is a method, exactly the same as the ValidationEventHandler method we saw in the previous section.

Now, we have the recipe to implement the XmlImporter class. Below is the code:

using System;
using System.Xml;
using
            System.Xml.Schema;

namespace Wrox.Xml.Mapping {

public
            class XmlImporter {

// The user-defined Handler
private
            Handler handler;

// Constructor (using a default
            handler)
public XmlImporter() {
this.handler = new
            DefaultHandler();
}

// Constructor (using a passed
            handler)
public XmlImporter(Handler handler) {
this.handler =
            handler;
}

// Read the XML files
public void
            import(string strFilename ) {

// XmlReader
            inizializzation
XmlTextReader txtReader = new
            XmlTextReader(strFilename);
XmlValidatingReader xtReader = new
            XmlValidatingReader(txtReader);

// Set the ValidationType to
            XML-Schema (XSD)
xtReader.ValidationType =
            ValidationType.Schema;

// Set the validation
            handler
xtReader.ValidationEventHandler +=
new
            ValidationEventHandler (this.ValidationEventHandle);

try
            {
// Parsing
while (xtReader.Read()) {
// Call the
            processElement of the user-defined
            handler
handler.processElement(xtReader);
}
}
catch
            (Exception e){
throw new Exception(e.Message);
}
finally
            {
xtReader.Close();
}
}

// Validation Event
            Handle
public void ValidationEventHandle (object sender,
ValidationEventArgs args) {
throw new
            System.Xml.XmlException(args.Message,null);
}
}
}

The class has two constructors: the first one creates an XmlImporter instance using DefaultHandler, the second one creates an XmlImporter instance using the Handler object passed as parameter. The import method takes the XML file name as input, reads it using the XMLValidatingReader, and for each XML element calls the processElement method of the Header object. Below is a straightforward implementation of Handler, which writes the name of the XML elements and their attributes to the console:

public class MereHandler : Handler
            {
public void processElement(XmlValidatingReader xtReader){
if
            (xtReader.NodeType==XmlNodeType.Element){
Console.WriteLine("Element:
            " + xtReader.Name);
for (int
            i=0;i<xtReader.AttributeCount;i++)
            {
xtReader.MoveToAttribute(i);
Console.WriteLine(" attribute
            {0} = {1}",
xtReader.Name,xtReader.Value);
}
}
public
            void processObject(object obj){
}
}

Notice that it first checks whether the xtReader object contains information about elements, which is indicated when NodeType = XmlNodeType.Element. XmlNodeType is an enumeration, which indicates the type of the XML entity: Element, Attribute, Comment, Document, and so on. Then, the name of the element and all its attributes are written to the console.

The following code fragment instantiates an XmlImporter using a MereHandler object, and calls the import method for the file called example.xml :

xmlImporter = new XmlImporter(new
            MereHandler());
xmlImporter.import("example.xml");

The application will show on the console the elements and the attributes contained in the file.

It should be very clear at this point that by implementing the Header interface, and in particular the processElement method, in the right way, you can transform the XML elements into C# objects. In fact, by using the xtReader object you have all the information you need: the name of the class, the fields, and their values. But how can we create dynamically objects starting from their names and fill them with the right data? The answer to this question is reflection.

Reflection and Metadata

Reflection is a generic term that covers some powerful features available within the .NET Framework. Java programmers have used reflection since Java was introduced. Now, Visual Basic, C#, ASP, and C++ (managed) programmers can also take advantage of reflection. As you will see in this section, by using reflection you can find out information about the types in your program or in other assemblies, and also to read other metadata from assembly manifests. Most of the classes are in the System.Reflection namespace. It's not the purpose of this case study to completely cover all subjects about reflection, but I will introduce the concepts that are useful to us in reaching our target.

Reflection will be used in this case study to provide a set of functionalities allowing us to:

  • create a C# object from a string representing the name of the class
  • discover a field in a C# object from a string representing the name of the field
  • set a value for that field.

The System.Type class helps us to achieve the first objective. A Type instance contains information about a specified C# base type as well as a user defined C# class. A way to get a Type instance is to use the static method GetType. This code fragment shows how you can create a Type object representing the class Person :

Type typeClass = Type.GetType("Person");

The typeClass variable is an instance of Type. By using typeClass we can get metadata containing different kinds of information about the class Person, such as its fields, properties, methods, constructors, etc. By using the Activator class you can even create an instance of Person :

Type typeClass = Type.GetType("Person");
if (typeClass==null)
Console.WriteLine("Class not found");
else
Person p = (Person) Activator.CreateInstance(typeClass);

The code above creates an instance of the Person class by using the GetType method, and then it checks if the returned reference is null. If null is returned then it means that the Person class does not exist, otherwise an instance of Person is created by invoking the static method CreateInstance of the System.Activator class. This system class contains methods to create types of objects locally or remotely, or obtain references to existing remote objects. In this case study, I'm going to use this class just to create the object defined in the XML document. Therefore, we have found a solution for the first objective: we can create C# objects starting with the name of the class.

Through the Type class, we can also get information about the fields contained in the class. Take a look at this code:

Type typeClass = Type.GetType("Person");
FieldInfo fieldInfo = typeClass.GetField("lastname");

The System.Reflection.FieldInfo class has the same meaning for fields as the System.Type class has for classes. An instance of FieldInfo contains information about a field of a class. As shown in the code, you can get a FieldInfo object by using the GetField method of Type. Thus, objective two is also solved: we can discovery a field of a C# class starting with its name.

To complete our objectives, we have to find a way to set the value of a field. This is a very trivial operation: the SetValue method of FieldInfo allows us to set a value for a field belonging to a specified object. The method takes two parameters: the parent object of the field, and the value. The following code shows how to set the value "Hill" to the lastname field of Person :

Type typeClass =
            Type.GetType("Person");
Person p = (Person)
            Activator.CreateInstance(typeClass);
FieldInfo fieldInfo =
            typeClass.GetField("lastname");
fieldInfo.setValue(p,"Hill");

We will be using the concepts covered in this section to map the XML elements to a C# object. In fact, by reading the XML you can get the name of the classes, and the name and the value of the fields as strings. Using this information - together with reflection - you can create the objects and fill the fields with the right values.

Mapping a simple object

We will start by implementing a component that only maps simple objects. A simple object is an object that only contains fields belonging to base types. To map the elements, the component has to know the XML structure; that is how the classes and the fields are represented inside the XML document. An example of the XML file is:

<class
            name="Temperature">
<field name="city" value="London"
            type="string"/>
<field name="dateTime" value="09/08/2002
            12.00" type="time"/>
<field name="temperature" value="23"
            type="short"/>
<field name="scale" value="C"
            type="char"/>
<field name="humidity" value="0.56"
            type="double"/>
</class>

There are two elements: <class> and <field>. The <class> element has the name attribute representing the name of the class. The <field> element has three attributes: name is the name of the field; value is the value of the field; and type that is one of the predefined data-type.

The XML schema describing the XML structure is as follows:

<?xml version="1.0" ?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">

<!-- Type -->
<xs:simpleType name="Type">
<xs:restriction base="xs:string">
<xs:enumeration value="string" />
<xs:enumeration value="integer" />
<xs:enumeration value="short" />
<xs:enumeration value="double" />
<xs:enumeration value="char" />
<xs:enumeration value="boolean" />
<xs:enumeration value="date" />
<xs:enumeration value="time" />
</xs:restriction>
</xs:simpleType>

<!-- Field -->
<xs:complexType name="Field">
<xs:attribute name="name" type="xs:string"/>
<xs:attribute name="value" type="xs:string" />
<xs:attribute name="type" type="Type" />
</xs:complexType>

<!-- Class -->
<xs:complexType name="Class">
<xs:sequence>
<xs:element name="field" type="Field"
minOccurs="0" maxOccurs="unbounded" />
</xs:sequence>
<xs:attribute name="name" type="xs:string" />
</xs:complexType>

<!-- classes -->
<xs:element name="classes">
<xs:complexType>
<xs:sequence>
<xs:element name="class" type="Class" maxOccurs="unbounded" />
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:schema>

As you may notice, this case study considers just eight data-types: string, integer, short, double, char, boolean, date, and time. An XML document, which is of a valid schema, consists of a sequence of <class> elements. A <class> element contains a sequence of <field> elements. The first version of the component we're going to develop only works with XML document that are valid for this schema.

A simple approach we can use to develop the mapping component is to implement the processElement of the Handler interface in a way that it reads the XML, and creates the objects and sets the field values by using reflection. Certainly, that is a perfectly valid solution, but we would not do this directly in the processElement method. Instead, we will divide the code reading the XML from the code creating the objects. For that reason, I thought to introduce the MappingFramework interface. The Handler implementation will have a MappingFramework instance and will delegate the issue of creating objects to it. Thus, we have distinct modules: the Handler, which reads and recognizes the XML elements, and the MappingFramework, which creates the objects and sets the field. The MappingFramework interface is as follows:

public interface MappingFramework {
void
            startClass(string className, string id);
object
            endClass();
void startField(string name, string val, string
            type);
void endField();
void startClassField(string name,
            string id);
void endClassField();
void startArray(string
            name);
void endArray();
void startItem(string
            className);
void endItem();
}

The interface contains five pairs of methods called startXXX and endXXX. For example, startClass will be called by the Handler when a <class> element is found, endClass will be invoked when a </class> element is found. For mapping simple objects, we will only implement the couple of methods related to the <class> and field <elements>. We will discuss the other methods in the next section. An empty implementation of MappingFramework is provided: this is the abstract class EmptyMappingFramework where each method does nothing.

Before starting the implementation of the mapping component for simple objects, I have to tell you something about the data-types. I already said that the component supports data-types, those are the ones indicated into the XML schema. However, those are generic data-types, thus we have to convert them into C# data-types when the field value is set. For this purpose, we will introduce the TypeConverter interface:

public interface TypeConverter {
string convert(string xmlType);
}

The convert method converts the generic-schema-defined data-type into the equivalent C# data-type. This is the implementation (TypeConverterImpl) used in this case study:

public class TypeConverterImpl :
            TypeConverter {
public string convert(string xmlType){
if
            (xmlType.CompareTo("string")==0)
return "System.String";
if
            (xmlType.CompareTo("short")==0)
return "System.Int16";
if
            (xmlType.CompareTo("integer")==0)
return "System.Int32";
if
            (xmlType.CompareTo("double")==0)
return "System.Double";
if
            (xmlType.CompareTo("char")==0)
return "System.Char";
if
            (xmlType.CompareTo("boolean")==0)
return "System.Boolean";
if
            (xmlType.CompareTo("date")==0)
return "System.DateTime";
if
            (xmlType.CompareTo("time")==0)
return
            "System.DateTime";
return null;
}
}

You can even implement your own TypeConverter to modify the data-types that this case study supports, or to add new ones. Each MappingFramework should have a TypeConverter instance.

Now we are ready to implement MappingFramework for the simple objects. These are the first lines of code:

public class MappingFrameworkImpl1 : EmptyMappingFramework {

// The current object
private object currentObj = null;

// The TypeConverter
private TypeConverter converter = new TypeConverterImpl();

The MappingFrameworkImpl1 class extends the EmptyMappingFramework abstract class. It has two private members: currentObj( the current object processed) and converter (a TypeComverterImpl instance).

The implementation of the startClass method is as follows:

public override void startClass(string className, string id) {

// Get the Type
Type typeClass = Type.GetType(className);
if (typeClass==null)
throw new System.Xml.XmlException("Class not found: " + className,null);

// Create the object
currentObj = Activator.CreateInstance(typeClass);
}

The method creates the object instance of the class named className by using reflection features such as Type.GetType and Activator.CreateInstance. The object is stored in the currentObj data member, thus this becomes the current object processed. The id parameter is not used for the simple object.

The implementation of the startField method is:

public override void startField(string name, string val, string type) {

// It Works on the current object
object obj = currentObj;

// Get the Type and FieldInfo
Type typeClass = obj.GetType();
FieldInfo fieldInfo = typeClass.GetField(name);
if (fieldInfo == null)
throw new System.Xml.XmlException
("Field not found: " + obj.GetType() + "." + name,null);

// Get the C# Type using the TypeConverter
string cSharpType = converter.convert(type);
if (fieldInfo.FieldType.ToString().CompareTo(cSharpType)!=0)
throw new System.Xml.XmlException(
"Different type found: " + obj.GetType() + "." + name +
"\n\rXML type: " + type +
"\n\rC# type: " + fieldInfo.FieldType,null);

try {
if (cSharpType.CompareTo("System.String")==0) {
fieldInfo.SetValue(obj,val);
}
if (cSharpType.CompareTo("System.Int32")==0) {
int v = int.Parse(val);
fieldInfo.SetValue(obj,v);
}
if (cSharpType.CompareTo("System.Int16")==0) {
short v = short.Parse(val);
fieldInfo.SetValue(obj,v);
}
if (cSharpType.CompareTo("System.Char")==0) {
char v = val[0];
fieldInfo.SetValue(obj,v);
}
if (cSharpType.CompareTo("System.Double")==0) {
double v = double.Parse(val);
fieldInfo.SetValue(obj,v);
}
if (cSharpType.CompareTo("System.Booelan")==0) {
bool v = bool.Parse(val);
fieldInfo.SetValue(obj,v);
}
if (cSharpType.CompareTo("System.DateTime")==0) {
DateTime v = DateTime.Parse(val);
fieldInfo.SetValue(obj,v);
}
}
catch (Exception) {
throw new System.Xml.XmlException(
"Error in field " + obj.GetType() +
"." + fieldInfo.Name +
"\n\rValue \"" + val +
"\" cannot be contaied in the type " + fieldInfo.FieldType
,null);
}
}

The Handler invokes the startField method when a <field> element is read from the XML. The method has three string parameters:

name | The name of the field (ex: "lastname")
val | The value of the field (ex: "Hill")
type | The generic-schema-defined type of the field (ex: "string")

This method does three things:

  • It creates the FieldInfo instance related to the field represented by name
  • It converts the generic-schema-defined type into a C# data-type by using the TypeConverter
  • It creates the right value (according to the C# data-type) and sets the value for that field

This method is the part of the proposed software that checks the data-types coming from the XML. As you can see, the method throws an exception both when the XML data-type is not the same as the one defined for the C# field, and when the value is not compatible with the data-type of the field.

The last MappingFramework method we implemt is closeClass. Here is the code:

public override object endClass(){
// Retur the current object
return currentObj;
}

This method is called by the Handler when a </class> element is read. This means that the information about the class is finished, thus the method returns the current object that, at this point, should have all the fields filled with the right values.

Now we can start to implement the Handler. We will name the class SimpleObjectHandler. The first lines of the code are:

public abstract class SimpleObjectHandler :
            Handler {

// Mapping Framework
protected MappingFramework
            framework;

// Constructor (using user's Mapping
            Framework)
public SimpleObjectHandler(MappingFramework framework)
            {
this.framework = framework;
}

// This must be
            implemented by derivated classes
public abstract void
            processObject(object obj);

The class is abstract because it does not implement the processObject method, which will be implemented by the concrete class extending SimpleObjectHandler. We will look at this topic later. The only data member of the class is a MappingFramework instance that, as you can see, will be initialized by the constructor.

The most interesting part of the SimpleObjectHandler class is the method processElement. It reads the XML elements and calls the right MappingFramework methods. This is the implementation:

public void
            processElement(XmlValidatingReader xtReader) {

// XML
            Elements
if (xtReader.NodeType==XmlNodeType.Element){

//
            Element is <class>
if (xtReader.Name.CompareTo("class")==0)
            {
xtReader.MoveToAttribute("name");
string className =
            xtReader.Value;
// Call the mapping
            framework
framework.startClass(className,null);
}

//
            Element is <field>
if (xtReader.Name.CompareTo("field")==0)
            {
// Find the name, value and type
            attributes
xtReader.MoveToAttribute("name");
string name =
            xtReader.Value;
xtReader.MoveToAttribute("value");
string val
            = xtReader.Value;
xtReader.MoveToAttribute("type");
string
            type = xtReader.Value;
// Call the mapping
            framework
framework.startField(name,val,type);
}
}

// XML End-Elements
if (xtReader.NodeType ==
            XmlNodeType.EndElement) {
// Element is </class>
if
            (xtReader.Name.CompareTo("class")==0) {
// Call the mapping
            framework
object obj = framework.endClass();
// Call
            overridden
            processObject
processObject(obj);
}
}
}

When the parser reads the <class> element, the method invokes startClass from the mapping framework; the className string is the value of the attribute name. If the read element is <field>, then the startField method is called - the name, val, and type parameters are taken from the respective attributes. If the read element is </class>, then the endClass method is called and then the processObject method is invoked. The class overriding that method could use the imported object in the same way as a normal C# object.

Now that the Handler and the MappingFramework have been implemented, we can carry out a simple program that reads the XML and shows the created objects with its fields. This is the code:

using System;
using System.Xml;
using
            System.Reflection;
using System.Collections;
using
            Wrox.Xml.Mapping;

// Example of using
            SimpleObjectHandler
public class ExampleHandler1:
            SimpleObjectHandler {
public ExampleHandler1() :
base(new
            MappingFrameworkImpl1()) {
}
public override void
            processObject(object obj){
// Show the created
            object
Console.WriteLine(obj); // ToString must be
            overriden
}
}

public class MainProgram {

public
            static void Main (String[] a) {

if
            (a.Length<1){
Console.WriteLine("No
            Params");
return;
}
string filename = a[0];

XmlImporter xmlImporter = new XmlImporter(new
            ExampleHandler1());

try
            {
xmlImporter.import(filename);
}
catch(System.Xml.XmlException
            e) {
Console.WriteLine(e.Message);
}
}
}

The class ExampleHandler1 extends SimpleObjectHandler and implements the abstract processObject method in order to show the imported object (the ToString method must be overridden for the object). The Main program is a console application taking in the XML file name as input. It creates an XMLImporter object, passing to the constructor a new ExampleHandler1 object. Then, it invokes the import method. Let's suppose that there is the following class in the system:

public class Temperature {

public string city;
public DateTime dateTime;
public short temperature;
public char scale;
public double dblHumidity;

public override string ToString() {
return "{" + city + ","
+ dateTime.ToString() + ","
+ temperature + ","
+ scale + ","
+ dblHumidity + "}";
}
}

Let's also suppose that the example-1.xml contains temperature elements like in the XML code we saw at the beginning of this section. The program, when given example-1.xml as input data, will show the imported Temperature objects, as in the following screen shot:

What we have done can work with simple objects. Now we will look at how, by implementing the MappingFramework and the Handler interfaces in a different way, you can get a more powerful component that can map complex objects, array of objects, and internal referenced objects.

Mapping complex object

A complex object is an object containing native type fields and nested objects. In this section, we will reimplement both Handler and MappingFramework in order that the component allows importing complex objects as well. The first thing to do is to define the way we describe the nested object in the XML document. We can modify the structure used for the simple objects, adding a new tag, <classfield>. This means the field is not a base-type, but another class. An example of XML code might be the following:

<class name="Country">
<field name="name" value="Italy" type="string"/>
<classfield name="continent">
<class name="Continent">
<field name="name" value="Europe" type="string"/>
<field name="population" value="320000000" type="integer"/>
</class>
</classfield>
</class>

The Country class has two fields: name and continent. The first one is a string, which is a base-type. The field continent is a <classfield>, hence it represents the Continent class that is nested in Country. To get the schema working with XML, you have to make some changes to the schema you have seen for the simple objects. In particular, you have to add the <classfield> element definition:

<!-- ClassField -->
<xs:complexType name="ClassField">
<xs:sequence>
<xs:element name="class" type="Class" maxOccurs="unbounded" />
</xs:sequence>
<xs:attribute name="name" type="xs:string" />
</xs:complexType>

In addition, you have to change the Class definition so that it can support a sequence of both <field> and <classfield> elements:

<!-- Class -->
<xs:complexType name="Class">
<xs:sequence>
<xs:element name="field" type="Field"
minOccurs="0" maxOccurs="unbounded" />
<xs:element name="classfield" type="ClassField"
minOccurs="0" maxOccurs="unbounded" />
</xs:sequence>
<xs:attribute name="name" type="xs:string" />
</xs:complexType>

You can find the complete XML schema in the support material for this case study - the name of the file is mapping-2.xsd.

To implement the MappingFramework interface for the complex objects, we can start from the one we implemented for the simple objects and try to understand what has to be changed. The main difference is represented by the current object concept (currentObj data member). For supporting nested objects, you cannot use a single member variable to store the object currently processed, because the current object changes each time a nested objects occurs in the document. After a nested object has been processed, the parent object returns to be the current object. Consequently, we need to store the objects in a stack; the current object will always be the element at the top of the stack. Although the .NET Framework provides the Stack class, we can't use it in this context. We need a special version of Stack, which contains these methods:

public void push(object obj) | Pushes an object into the stack
public object pop() | Takes out the object at the top of the stack
public object top(int n) | Returns the object that is -n objects from the top. top(0) // Returns the top top(-1) // Return the object before the top
public bool empty() | Returns true if the stack has no object

The System.Collections.Stack included in .NET does not provide a method similar to top(int n) and, as you will see, we need such method in this situation. The MappingFrameworkImpl2 that extends EmptyMappingFramework starts like this:

public class MappingFrameworkImpl2 : EmptyMappingFramework {

// Stack for the objects
private Stack stackObject = new Stack();

// TypeConverter
private TypeConverter converter = new TypeConverterImpl();

The private member stackObject is the stack where the current objects will be stored.

Let's implement the MappingFramework interface methods. The startClass method now works on the object at the top of the stack that can either be a root or a nested object:

public override void startClass(string
            className, string id) {

// Get the Type
Type typeClass =
            Type.GetType(className);
if (typeClass==null)
throw new
            System.Xml.XmlException("Class not found: " +
            className,null);

// Create a new object
object obj =
            Activator.CreateInstance(typeClass);
stackObject.push(obj);

}

After the object is created, the method puts it in onto the stack. From this moment on, the object is the current object.

The startField method is basically the same as the startField method we implemented for the simple objects. The only difference is that we are now working on the object at the top of the stack instead of working on the currentObject variable:

public override void startField(string name, string val, string type) {

// It works on the stack top-object
object obj = stackObject.top(0);

// ... same implementation of the one for
// the simple objects ...

}

The endClass method is as follows:

public override object endClass(){

object top = stackObject.pop();
if (stackObject.empty())
return top;

return null;
}

The method takes the element from the top of the stack but only returns it when the stack is empty, that is, when all the nested objects have been processed. This means that the endClass method only returns a non-null value if the last object we process is a root object.

For this implementation, we also need to implement the startClassField method of MappingFramework. Each time the parser reads a <classfield> element, the Handle calls startClassField. This method attaches the nested object to the right field of its parent. The method has two parameters: name and id. The name string represents the name of the field; id is not used here. When this method is called, there is the object at the top of the stack that must be assigned to the field specified by name (child variable). The object previously at the top is the object holding the field specified by name (parent variable). What the method does is to set the value of the parent's name field to the object child. Here is the startClassField implementation:

public override void startClassField(string name, string id) {

// The child is the current object...
object child = stackObject.top(0);
// ... and the parent the previuos one
object parent = stackObject.top(-1);

// Get the Type and the FieldInfo
Type typeClass = parent.GetType();
FieldInfo fieldInfo = typeClass.GetField(name);
if (fieldInfo == null)
throw new System.Xml.XmlException
("Field not found: " + parent.GetType() + "." + name,null);

// Attach the child object to the parent
try {
fieldInfo.SetValue(parent,child);
}
catch(Exception) {
throw new System.Xml.XmlException
("Error setting field: " + parent.GetType() + "." + name,null);
}
}

Implementing MappingFramework is not enough; you also have to make some little changes to the Handle implementation. In the support material provided with this case study, you can find the class ComplexObjectHandler implementing Header for supporting complex objects. The implementation is very close to the SimpleObjectHandler implementation. The only difference is that in the processElement, there is a new if statement to check the <classfield> element. In this case, the startClassField method is called. Here is a code fragment:

public void
            processElement(XmlValidatingReader xtReader) {

if
            (xtReader.NodeType==XmlNodeType.Element){

// Element is
            <class>
if (xtReader.Name.CompareTo("class")==0) {
//
            ...

// Element is <field>
if
            (xtReader.Name.CompareTo("field")==0) {
// ...

// Element
            is <classfield>
if
            (xtReader.Name.CompareTo("classfield")==0) {

// Find name
            attribute
while(xtReader.NodeType!=XmlNodeType.Element)
xtReader.Read();
xtReader.MoveToAttribute("name");
string fieldName =
            xtReader.Value;

// Find name attribute (of the nested
            class)
while(xtReader.NodeType!=XmlNodeType.Element)
xtReader.Read();
xtReader.MoveToAttribute("name");
string
            className = xtReader.Value;

// Call the mapping framework
            (creating the nested class)
framework.startClass(className,null);
// Call the mapping framework (attaching the nested
            class)
framework.startClassField(fieldName,null);
}
//
            ...
}

When the parser reads a <classfield> element, the method reads the name attribute and puts it in the fieldName string. According to the XML schema, the next element must be <class>, and so the method reads the name attribute of <class> representing the name of the class, and puts it in the className string. Finally, the startClass and the startClassField methods are invoked.

MappingFrameworkImpl2 and ComplexObjectHandler can be used to implement a concrete Handler that reads the XML and shows the created objects, its fields, and in the proper case, the nested objects:

public class ExampleHandler2: ComplexObjectHandler {
public ExampleHandler2() :
base(new MappingFrameworkImpl2()) {
}
public override void processObject(object obj){
// Show the created object
Console.WriteLine(obj); // ToString must be overriden
}
}

Following what we looked at in the previous section, you could execute a Main program using ExampleHandler2.

Mapping array of object

In this section, we add a new feature to the component to import objects containing array of objects. An example of XML using arrays is as follows:

<class name="Country">
<field name="name" value="Italy" type="string"/>
<array name="majorCities">
<item name="City">
<field name="name" value="Rome" type="string"/>
<field name="county" value="Lazioe" type="string"/>
</item>
<item name="City">
<field name="name" value="Napoli" type="string"/>
<field name="county" value="Campania" type="string"/>
</item>
</array>
</class>

As you can see, there are two new elements: <array> and <item>. The <array> element indicates the field is an array; it has the attribute name, which is the name of the array field. An <array> is a sequence of <item> elements, which have exactly the same structure of the <class> element. The XSD code fragment to define the <array> element is as follows:

<!-- Array -->
<xs:complexType name="Array">
<xs:sequence>
<xs:element name="item" type="Class" maxOccurs="unbounded" />
</xs:sequence>
<xs:attribute name="name" type="xs:string" />
</xs:complexType>

Moreover, the <class> element has to allow <array> elements:

<!-- Class -->
<xs:complexType name="Class">
<xs:sequence>
<xs:element name="field" type="Field"
minOccurs="0" maxOccurs="unbounded" />
<xs:element name="classfield" type="ClassField"
minOccurs="0" maxOccurs="unbounded" />
<xs:element name="array" type="Array"
minOccurs="0" maxOccurs="unbounded" />
</xs:sequence>
<xs:attribute name="name" type="xs:string" />
</xs:complexType>

The complete XML schema is present in the support material of this case study; the name of the file is mapping-3.xsd.

To support arrays you have to implement both a new MappingFramework and a new Handler. The MappingFramework implementation will use two stacks, one for the current objects and the other one for the current arrays. The philosophy behind the stacks is the same as before. The beginning of the class MappingFrameworkImpl3, which implements MappingFrameworks, is as follows:

public class MappingFrameworkImpl3 : EmptyMappingFramework {

// Stack for the objects
private Stack stackObject = new Stack();

// Stack for the arrays
private Stack arrayStack = new Stack();

The implementation of startClass, startField, startFieldClass and endClass are the same as the ones we implemented previously. However, we now need to add the implementation of new methods such as startArray, startItem and endArray.

The Handler calls the startArray method when the parser reads an <array> element. This is the implementation:

public override void startArray(string name)
            {

// Get the top-object from the stack
object obj =
            stackObject.top(0);

// Get the Type and the FieldInfo
Type
            typeClass = obj.GetType();
FieldInfo fieldInfo =
            typeClass.GetField(name);
if (fieldInfo == null)
throw new
            System.Xml.XmlException
("Field not found: " + obj.GetType() +
            "." + name,null);

try {
// Create a new
            ArrayList
ArrayList array = new ArrayList();
// Attach that
            array to the current object
fieldInfo.SetValue(obj,array);
//
            Push the array into the
            array-stack
stackArray.push(array);
}
catch(Exception)
            {
throw new System.Xml.XmlException
("Error setting field: " +
            obj.GetType() + "." + name,null);
}

The name parameter represents the name of the field containing a reference to the array. This field is always an ArrayList ; in fact, I decided to use this structure to map the arrays. The method creates a new empty ArrayList and assigns it to the name field of the current object, which is at the top of the stackObject. In addition, the ArrayList is pushed onto the stackArray.

The Handler calls the method startItem when the parser reads an <item> element. The method creates the object, pushes it on the stackObject, and finally adds the created object to the ArrayList located at the top of the stackArray :

public override void startItem(string
            className) {

// Get the Type
Type typeClass =
            Type.GetType(className);
if (typeClass==null)
throw new
            System.Xml.XmlException("Class not found: " +
            className,null);

// Create a new object
object obj =
            Activator.CreateInstance(typeClass);
stackObject.push(obj);

//
            Add the item to the top-stack-array element
ArrayList array =
            (ArrayList) stackArray.top(0);
array.Add(obj);
}

The endArray method is called by the Handler when the parser finds an </array> element. The implementation is:

public override void endArray()
            {
stackArray.pop();
}

It takes the last ArrayList processed from the stackArray.

ArrayHandler is the implementation of the Handler which works with the simple objects, complex objects, and arrays. An version of the class is shown below, the code shows only the differences with ComplexObjectHandler :

public abstract class ArrayHandler : Handler
            {

// ...

public void
            processElement(XmlValidatingReader xtReader) {

if
            (xtReader.NodeType==XmlNodeType.Element){

// Element is
            <class>
if (xtReader.Name.CompareTo("class")==0) {
//
            ...

// Element is <field>
if
            (xtReader.Name.CompareTo("field")==0) {
// ...

// Element
            is <classfield>
if
            (xtReader.Name.CompareTo("classfield")==0) {
//...

//
            Element is <array>
if (xtReader.Name.CompareTo("array")==0)
            {

// Find name
            attribute
while(xtReader.NodeType!=XmlNodeType.Element)
xtReader.Read();
xtReader.MoveToAttribute("name");
string
            arrayName = xtReader.Value;

// Call the mapping
            framework
framework.startArray(arrayName);
}

// Element
            is <item>
if (xtReader.Name.CompareTo("item")==0) {
//
            Find name attribute
xtReader.MoveToAttribute("name");
// Call
            the mapping framework
framework.startItem(xtReader.Value);
}

}

// XML End-Elements
if (xtReader.NodeType
            == XmlNodeType.EndElement) {

// Element is </classes>
            or </item>
if (xtReader.Name.CompareTo("class")==0
||
            xtReader.Name.CompareTo("item")==0) {
// Call the mapping
            framework
object obj = framework.endClass();
// Call
            overridden processObject
// only for not nested classes
            (obj!=null)
if (obj!=null) processObject(obj);
}

//
            Element is </array>
if
            (xtReader.Name.CompareTo("array")==0) {
// Call the mapping
            framework
framework.endArray();
}
}
}
}

When an <array> element is found, the processElement method gets the name of the field holding the array (nameArray), and calls the startArray method. When an <item> element is found, the method gets the name of the class and invokes the startItem method. The processElement methods treats the </item> element like a </class> element. Eventually, when the parser reads the </array> element, the endArray method is called.

MappingFrameworkImpl3 and ArrayHandler can be used to implement a concrete Handler that reads the XML and shows the created objects, its fields, and if applicable, the nested objects and the arrays:

public class ExampleHandler3: ArrayHandler {
public ExampleHandler3() :
base(new MappingFrameworkImpl3()) {
}
public override void processObject(object obj){
// Show the created object
Console.WriteLine(obj); // ToString must be overriden
}
}

Following on from what we looked at in the "simple object" section, you could execute a Main program using ExampleHandler3.

Internal References

In this section, we're going to look at how to enhance the component so that it can deal with internal references among <class> elements. This will be the final version of the component mapping XML to C# objects. Let's consider the following XML:

<class name="Person",
            id="1000">
<field name="surname" value="Hill"
            type="string"/>
<field name="name" value="Robert"
            type="string"/>
</class>

<class
            name="Company">
<field name="name" value="Hill and Son"
            type="string"/>
<classfield name="chairman"
            id="1000"/>
</class>

When the component reads the Person object, it creates the "Robert Hill" Person object whose id is 1000. Then the component reads the <class name="Company"> element, and creates the "Hill & Son" Company object, whose field chairman is a reference to the Person with id=1000 which was previously created (i.e. "Robert Hill"). The internal references are useful for two main reasons:

  • They allow us to avoid defining the same class lots of times within the same XML document; you can define the class once and then reference it by using the id number;
  • They permit us to carry out one-to-n relationship between class elements.

In the previous example, I wrote the Person class just once. In addition, if "Robert Hill" is the chairman of another company as well, then we can simply add the new company with a reference to the "Robert Hill" Person, like this:

<class name="Company">
<field name="name" value="Hill Ltd" type="string"/>
<classfield name="chairman" id="1000"/>
</class>

Hence, two companies - "Hill and son" and "Hill Ltd" - have the same chairman, "Robert Hill". The component will create two Company objects whose chairman field is a reference to the same Person object, so only one Person object is created! Unfortunately we can't have this feature with the developed component, we have to add the new "internal reference" functionality.

The first step is to adapt the XML schema for supporting this new feature. This is a very simple operation, all we have to do is add the attribute id to the related XSD complex-type of both the <class> and <classfield> elements:

<!-- Add this line to Class and ClassField complex type -->
<xs:attribute name="id" type="xs:string"/>

The id attribute is not mandatory: you specify it only when you want to use the internal references. The complete XSD file is part of the support material for this case study, the name of the file is mapping.xsd.

As usual, first we implement the MappingFramework and then the Handler. The idea is to put the objects having an id into a hash table. Thus, when a <classfield> has the id, it is used to retrieve the right object from the hash table. The MappingFrameworkImpl, the definitive version, is a class implementing MappingFramework. It starts like this:

public class MappingFrameworkImpl : EmptyMappingFramework {

// Stack for the objects
private Stack stackObject = new Stack();

// Stack for the arrays
private Stack stackArray = new Stack();

// Cache for the ID-Objects
private Hashtable idTable = new Hashtable();

As you may notice, the class has the idTable data member, which is the hash table where we put the object with the id.

The startClass must be implemented again. This final version will use the id parameter:

public override void startClass(string
            className, string id) {

// Get the Type
Type typeClass =
            Type.GetType(className);
if (typeClass==null)
throw new
            System.Xml.XmlException("Class not found: " +
            className,null);

// Create a new object
object obj =
            Activator.CreateInstance(typeClass);
stackObject.push(obj);

//
            Store the object whether it has ID
if (id!=null)
try
            {
idTable.Add(id,obj);
}
catch(Exception e){
throw new
            System.Xml.XmlException(e.Message,null);
}
}

You may see that after the creation, the object, if the id is non-null, is put into the hash table.

The startClassField must also be implemented again:

public override void startClassField(string
            name, string id) {

object child = null;
object parent =
            null;

if (id==null){
// XML Element classfield is followed
            by a class definition
// In this case the child is the current
            object...
child = stackObject.top(0);
// ... and the parent
            the previuos one
parent = stackObject.top(-1);
}
else
            {
// XML Element classfield has a class ID
// In this case the
            child in the hashtable...
child = idTable[id];
// ... and the
            parente is the current object
parent = stackObject.top(0);
if
            (child==null)
throw new System.Xml.XmlException("Id not found: "
            + id,null);
}

// Get the Type and the FieldInfo
Type
            typeClass = parent.GetType();
FieldInfo fieldInfo =
            typeClass.GetField(name);
if (fieldInfo == null)
throw new
            System.Xml.XmlException
("Field not found: " + parent.GetType() +
            "." + name,null);

// Attach the child object to the
            parent
try
            {
fieldInfo.SetValue(parent,child);
}
catch(Exception)
            {
throw new System.Xml.XmlException
("Error setting field: " +
            parent.GetType() + "." + name,null);
}
}

If id=null, the method has the same behavior as the startClassField method you have seen in the previous section. When id is not null, the parent object is at the top of the stack and the child object is in the hash table. That's because the <classfield> element, which has id, is not followed by any class element, thus no child object has been created and put into the stack. Instead, the child should be in the hash table, because it has been created by some previous class element having id.

Other methods of MappingFramework are unchanged from the implementation you have seen, in the previous section, for the arrays.

CompleteHandler is the definitive implementation of the Handler, which works with simple objects, complex objects, arrays, and internal references. An amended version of the class is here shown, the code represents the differences with the ArrayHandler implementation:

public abstract class CompleteHandler :
            Handler {

// ...

public void
            processElement(XmlValidatingReader xtReader) {

if
            (xtReader.NodeType==XmlNodeType.Element){

// Element is
            <class>
if (xtReader.Name.CompareTo("class")==0)
            {

// Find name
            attribute
xtReader.MoveToAttribute("name");
string ClassName =
            xtReader.Value;

// Find id
            attribute
xtReader.MoveToAttribute("id");
string id =
            null;
if (xtReader.Name.CompareTo("id")==0)
id =
            xtReader.Value;

// Call the mapping
            framework
framework.startClass(ClassName,id);

}

//
            Element is <field>
if (xtReader.Name.CompareTo("field")==0)
            {
// ...

// Element is <classfield>
if
            (xtReader.Name.CompareTo("classfield")==0) {

// Finds name
            attribute
while(xtReader.NodeType!=XmlNodeType.Element)
xtReader.Read();
xtReader.MoveToAttribute("name");
string fieldName =
            xtReader.Value;

// Finds id
            attribute
xtReader.MoveToAttribute("id");
string idField =
            null;
if (xtReader.Name.CompareTo("id")==0)
idField =
            xtReader.Value;

if (idField==null) {
// id not
            present
// Processe the nested <class>
            element
while(xtReader.NodeType!=XmlNodeType.Element)
xtReader.Read();

//
            Find name attribute (of the nested
            class)
xtReader.MoveToAttribute("name");
string className =
            xtReader.Value;

// Find id attribute (of the nested
            class)
string idClass =
            null;
xtReader.MoveToAttribute("id");
if
            (xtReader.Name.CompareTo("id")==0)
idClass =
            xtReader.Value;

// Call the mapping framework (creating the
            nested class)
framework.startClass(className,idClass);
//
            Call the mapping framework (attaching the nested
            class)
framework.startClassField(fieldName,null);
}
else
            {
// id present
// Call the mapping framework
// lookup
            referred class where id =
            idField
framework.startClassField(fieldName,idField);
}

}

// Element is <array>
if
            (xtReader.Name.CompareTo("array")==0) {
// ...

// Element
            is <item>
if (xtReader.Name.CompareTo("item")==0)
            {
//...

}

// XML End-Elements
if
            (xtReader.NodeType == XmlNodeType.EndElement) {
//...
}
}

As you may notice, even this is an abstract class. This is because the processObject method has not been implemented. Therefore, to use the CompleteHandler, you have to extend it, providing the implementation of processObject. This method depends on the problem you are facing, and by implementing it you can specify how the application has to process the created objects. I deliberately made CompleteHandler abstract so that you must necessarily implement the processObject method - an instance of CompleteHandler without a processObject complete implementation is nonsense.

In the code above, the changed parts are the <class> and <classfield> elements. When the parser reads the <class> element, the method tries to find the id. If it is present, then the id is passed to the startClass method. The situation is a little bit complex when the parser finds a <classfield> element. According to the XML schema, this element can either have the id attribute or be followed by a <class> element. In the first case, the startClassField method is called, passing the field name and the id (see else block). When the id of the <classfield> is not present, then a <class> element follows. Thus, the method gets the name of the class and, if applicable, the id. Finally, it calls the startClass and startClassField methods.

MappingFrameworkImpl and CompleteHandler can be used to implement a concrete Handler that reads the XML and shows the created objects and its fields:

public class ExampleHandler4: ArrayHandler
            {
public ExampleHandler4() :
base(new MappingFrameworkImpl())
            {
}
public override void processObject(object obj){
// Show
            the created object
Console.WriteLine(obj); // ToString must be
            overriden
}
}

Following what was explained in the "simple object" section, you could execute a Main program using ExampleHandler4.

The Sample Application

To show how the developed component can be integrated in a real-word context, I'm going to implement a sample application. This can be seen as a module performing a particular task of a logistics system. The application consists of a form where the warehouse-stock is showed article-by-article. External systems, via XML documents, inform the application that new customer orders or purchase orders have arrived. The application reads the XML, creates the customer orders and/or the purchase orders, updates the stock of the involved articles, and records the warehouse movement. Through the form, the user can monitor the article stocks and the recorded movements.

The final application will look like the one shown below:

Database Design and Structure

The class diagram representing the database of the application is shown the following diagram:

The Order abstract class can have two generalizations: CustomerOrder and PurchaseOrder. The first one is an order coming from a customer requiring some goods. The second one is an order made by the warehouse to a supplier for purchasing some kind of goods. Both types of orders have OrderLine aggregated, which tells us, among other things, what the article is and how many pieces are required. In fact, as you can see, the OrderLine class has an association to the class Article, which is the article that can be purchased or sold. Each article contains a list of Movement objects. The system creates a movement each time the article-stock changes due to a customer order or a purchase order.

The application database consists of in-memory C# objects. We could also permanently store the objects in a database such as SQL Server or Oracle, but that is not the purpose of this case study. The main goal of this piece is to explain how to import data from XML and create objects using it, not how to store objects into a relational database.

The code for the abstract class Order is as follows:

public abstract class Order {
public int number;
public DateTime date;
public ArrayList orderLines;
}

An order consists of a number, a date, and an array of OrderLine objects. There are two generalizations of this abstract class: CustomerOrder and PurchaseOrder, which are implemented this way:

public class CustomerOrder : Order
            {
public string customer;
}

public class PurchaseOrder
            : Order {
public string supplier;
}

As you can see, the only difference is that the first class has a reference to a customer and the second one to a supplier. Obviously, in more complex systems there would be many other differences.

The OrderLine, CustomerOrderLine, and PurchaseOrderLine implementations are as follows:

public abstract class OrderLine {
public
            int number;
public int quantity;
public double
            price;
public Article article;
public abstract int
            getRelativeQuantity();
public abstract string
            getType();
}

public class CustomerOrderLine :
            OrderLine{
public override int getRelativeQuantity(){
return
            -quantity;
}
public override string getType(){
return
            "Customer Order Line ";
}
}

public class
            PurchaseOrderLine : OrderLine{
public override int
            getRelativeQuantity(){
return quantity;
}
public override
            string getType(){
return "Purchase Order
            Line";
}
}

The OrderLine class holds four data members representing the number, the article, the quantity of the article (in pieces), and the price. In addition, the class has two abstract methods that will be implemented by its generalizations.

In fact, the CustomerOrderLine class implements getType in order to return the type of the line as a string, that is "Customer Order Line". The getRelativeQuantity returns -quantity, because for a customer order the goods are leaving the warehouse. Likewise, for the PurchaseOrderLine the getType method returns "Purchase Order Line", and the getRelativeQuantity returns quantity, because in this case the goods are coming in the warehouse.

The Article class is:

public class Article {
public string
            code;
public string description;
public int stock;
public
            ArrayList movements = new ArrayList();
}

An article has a code and a description. The stock member is the quantity currently present in the warehouse. The movements ArrayList contains a list of Movement objects:

public class Movement {
public int
            number;
public OrderLine cause;
}

A warehouse movement consists of a number and a cause, which is the OrderLine that causes the movement.

The class that stores, finds, and manages those business objects is called DataController ; it contains these methods:

// Add an article
public void addArticle(Article article)

// Find an article
public Article lookupArticle(string code)

// Retur all the articles
public ArrayList getAllArticles()

// Find an order
public Order lookupOrder(int number)

// Add an order
public void addOrder(Order order)

// Import objects from an XML file
// It uses the XmlImporter and the CompleteHandler
public void import(string fileName)

You can find the complete implementation of DataController in the support material supplied with this case study.

UI Design and Coding

To obtain a form that looks like the one we saw before, you can follow these steps:

1. Add a new Window Form to the project into the file MainForm.cs.

2. Sets the following properties of the form:

Property | Value
Name | MainFrom
Text | Stock Situation

3. Add the Labels Articles and Movements to the MainForm.

4. Add a ListView named listArticle and put in under the Articles Label.

5. Add a ListView named listMovement and put it under the Movements Label.

6. Add the Label XML File to the MainForm.

7. Add a Text Box next the XML File label and set the property name to "txtXmlFileName".

8. Add a new Button next to the txtXmlFileName Text Box with these properties:

Property | Value
Name | btnImport
Text | Import

The listArticle ListView will show all the articles present in the warehouse and their stocks. When the user clicks on a row, the listMovement will show all the movements related to the selected article. To make the handling of the lists to the MainForm independent, I implemented two classes: ArticleListView and MovementListView. Both implementations are very similar. Here is the code for ArticleListView :

public class ArticleListView {

// The
            ListView
private ListView listView;

// The
            DataController
private DataController controller;

//
            Constructor
public ArticleListView(ListView listView,
            DataController controller){

this.listView =
            listView;
this.controller = controller;

// Set the
            listView proporties
listView.Clear();
listView.View =
            View.Details;
listView.FullRowSelect =
            true;
listView.GridLines = true;

// Column
            Headers
listView.Columns.Add("Article code", -2,
            HorizontalAlignment.Left);
listView.Columns.Add("Description",
            -2, HorizontalAlignment.Left);
listView.Columns.Add("Stock", -2,
            HorizontalAlignment.Left);

// Reload the List
reload();

// Set empty images for the items
ImageList
            imageListSmall = new ImageList();
ImageList imageListLarge = new
            ImageList();
listView.LargeImageList =
            imageListLarge;
listView.SmallImageList =
            imageListSmall;

}

// Add an article to the
            List
public void addArticle(Article article) {
ListViewItem
            item = new
            ListViewItem(article.code,0);
item.SubItems.Add(article.description);
item.SubItems.Add(""+article.stock);
listView.Items.AddRange(new
            ListViewItem[]{item});
}

// Reload the List
public
            void reload() {
listView.Items.Clear();
ArrayList articles =
            controller.getAllArticles();
for (int i=0; i<articles.Count;
            i++)
addArticle((Article)articles[i]);
}
}

The class has the listView data member that is the List View relative to the articles, and controller that is an instance of the DataController class. The constructor accepts as input both listView and controller. It also sets the properties for listView, the column headers, and populates the list. The class has two public methods. The first one is addArticle, which adds an article to the list. The second one is reload, which reloads listView with the articles provided by the controller instance. The MainForm will only access the article List View (listArticle) by an instance of the ArticleListView class (named articleListView), and the movement List View (listMovement) by an instance of MovementListView (named movementListView). You can find the complete implementation of MovementListView with the support material provided with this case study.

The MainForm class has the following data members:

// DataController
private DataController controller = new DataController();
// ListView for the articles
private ArticleListView articleListView;
// ListView for the movements
private MovementListView movementListView;

The MainForm_Load method will create the ArticleListView :

private void MainForm_Load(object sender,
            System.EventArgs e) {
articleListView = new
            ArticleListView(listArticle, controller);
}

When the user selects an article, the listArticle_SelectedIndexChanged method will be invoked. It creates a new MovementListView populated with the movements of the selected article:

private void listArticle_SelectedIndexChanged(object sender, System.EventArgs e) {
// When the user clicks on a ArticleListView item
// a movementListView is created with the movements
// belonging to the selected article
if (listArticle.SelectedItems.Count != 1) return;
string code = listArticle.SelectedItems[0].SubItems[0].Text;
Article article = controller.lookupArticle(code);
movementListView =
new MovementListView(listMovement, controller, article);
}

If you create some articles and run the application, you should obtain something like this:

Importing XML

By means of the txtXmlFileName Text Box, you can specify the XML file to import. Pressing the Import button causes the importing process to start. Below is the button click event for the Import button:

private void btnImport_Click(object sender,
            System.EventArgs e) {
// Import XML and reload the
            ArticleListView
try
            {
controller.import(txtXmlFileName.Text);
articleListView.reload();
}
catch(Exception
            ex){
MessageBox.Show(ex.Message);
}
}

As you may notice, the method calls the import method of the DataController class, and then reloads the articleListView by calling the reload method. If no exception occurs, you should see the imported articles and their movements inside the lists.

The import method of the DataController class is implemented like this:

public void import(string fileName) {
XmlImporter xmlImporter = new XmlImporter(new
            ImportBO(this));
xmlImporter.import(fileName);
}

The method creates a new XmlImporter passing a new ImportBO handler and then calls the import method. The ImportBo handler is a concrete implementation of the CompleteHandler :

class ImportBO : CompleteHandler {

// Use a DataController to deal with the data
private DataController controller;

// Constructor
// MappingFrameworkImpl is used
public ImportBO(DataController controller) :
base(new MappingFrameworkImpl()){
this.controller = controller;
}

// ...to be continued

It has a DataController data member (controller). The constructor takes in the DataController class as input and calls the base constructor, passing a new MappingFrameworkImpl instance.

The processObject method is finally implemented in the following way:

public override void processObject(object
            obj){

// Import an Article
if (obj is Article)
            {
importArticle((Article)obj);
}

// Import an
            Order
if (obj is Order) {
importOrder((Order)
            obj);
}

}

In this application, the imported objects can only be articles or orders. The method delegates the operations to the importArticle and importOrder private methods.

The importArticle method adds a new Article object to the database handled by the DataController. The method is quite smart, as it in fact only adds the article if it is not already present in the database. This is the code:

private void importArticle(Article article) {
Article stockedArticle = controller.lookupArticle(article.code);
if (stockedArticle==null)
controller.addArticle(article);
}

The importOrder method is a little bit more complex. Actually, importing an order means:

  • Checking if the order is already booked;
  • Adding the order to the database;
  • Ceating a new movement for each order line;
  • If needed, adding new articles involved in the order line;
  • Updating the stock.

The code that does all these things is:

private void importOrder(Order order)
            {

Order bookedOrder =
            controller.lookupOrder(order.number);
// Continue only if the
            order does not exist
if (bookedOrder==null) {
// Add the
            order
controller.addOrder(order);
// Processing lines to
            create movements
for (int i=0; i<order.orderLines.Count; i++)
            {
// Get the line
OrderLine line = (OrderLine)
            order.orderLines[i];
// Import the
            article
importArticle(line.article);
// Set line.article to
            the imported article or to
// an already existent article with
            the same code
line.article =
            controller.lookupArticle(line.article.code);
// Add a
            movement
Movement movement = new Movement();
movement.number =
            line.article.movements.Count + 1;
movement.cause =
            line;
line.article.movements.Add(movement);
// Update the
            stock
line.article.stock +=
            movement.cause.getRelativeQuantity();
}
}
}

The application is now complete. The following XML contains three articles and a purchase order:

<!-- firstload.xml -->
<?xml
            version="1.0" encoding="utf-8" ?>

<classes
            xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:noNamespaceSchemaLocation="..\mapping.xsd">

<class
            name="Article" id="01005">
<field name="code" value="01005"
            type="string"/>
<field name="description" value="Sony
            Mobile Phone" type="string"/>
</class>

<class
            name="Article" id="01006">
<field name="code" value="01006"
            type="string"/>
<field name="description" value="Philips TV
            Color" type="string"/>
</class>

<class
            name="Article" id="01007">
<field name="code" value="01007"
            type="string"/>
<field name="description" value="Aiwa
            Video-Tape Recorder"
            type="string"/>
</class>

<class
            name="PurchaseOrder">
<field name="number" value="300"
            type="integer"/>
<field name="supplier" value="SGH Ltd"
            type="string"/>
<field name="date" value="13/08/2002"
            type="date"/>
<array name="orderLines">
<item
            name="PurchaseOrderLine">
<field name="number" value="1"
            type="integer"/>
<field name="quantity" value="5000"
            type="integer"/>
<field name="price" value="59.90"
            type="double"/>
<classfield name="article"
            id="01005"/>
</item>
<item
            name="PurchaseOrderLine">
<field name="number" value="2"
            type="integer"/>
<field name="quantity" value="3500"
            type="integer"/>
<field name="price" value="139.90"
            type="double"/>
<classfield name="article"
            id="01006"/>
</item>
<item
            name="PurchaseOrderLine">
<field name="number" value="3"
            type="integer"/>
<field name="quantity" value="1500"
            type="integer"/>
<field name="price" value="199.90"
            type="double"/>
<classfield name="article"
            id="01007"/>
</item>
</array>
</class>

</classes>

If you import this file, the form should look like the one in the figure below, where you can see the movements related to the article 01005:

Now, you can try to import another file having both customer orders and purchase orders. For example, use the following:

<!-- order.xml -->
<?xml
            version="1.0" encoding="utf-8" ?>

<classes
            xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:noNamespaceSchemaLocation="..\mapping.xsd">

<class
            name="Article" id="01005">
<field name="code" value="01005"
            type="string"/>
<field name="description" value="Sony
            Mobile Phone" type="string"/>
</class>

<class
            name="Article" id="01006">
<field name="code" value="01006"
            type="string"/>
<field name="description" value="Sony TV
            Color" type="string"/>
</class>

<class
            name="Article" id="01007">
<field name="code" value="01007"
            type="string"/>
<field name="description" value="Aiwa
            Video-Tape Recorder"
            type="string"/>
</class>

<class
            name="CustomerOrder">
<field name="number" value="400"
            type="integer"/>
<field name="customer" value="N-B Ltd"
            type="string"/>
<field name="date" value="13/08/2002"
            type="date"/>
<array name="orderLines">
<item
            name="CustomerOrderLine">
<field name="number" value="1"
            type="integer"/>
<field name="quantity" value="50"
            type="integer"/>
<field name="price" value="99.90"
            type="double"/>
<classfield name="article"
            id="01005"/>
</item>
<item
            name="CustomerOrderLine">
<field name="number" value="2"
            type="integer"/>
<field name="quantity" value="100"
            type="integer"/>
<field name="price" value="399.90"
            type="double"/>
<classfield name="article"
            id="01006"/>
</item>
</array>
</class>

<class
            name="CustomerOrder">
<field name="number" value="401"
            type="integer"/>
<field name="customer" value="Hill Ltd"
            type="string"/>
<field name="date" value="14/08/2002"
            type="date"/>
<array name="orderLines">
<item
            name="CustomerOrderLine">
<field name="number" value="1"
            type="integer"/>
<field name="quantity" value="150"
            type="integer"/>
<field name="price" value="99.90"
            type="double"/>
<classfield name="article"
            id="01005"/>
</item>
<item
            name="CustomerOrderLine">
<field name="number" value="2"
            type="integer"/>
<field name="quantity" value="300"
            type="integer"/>
<field name="price" value="299.90"
            type="double"/>
<classfield name="article"
            id="01007"/>
</item>
</array>
</class>

<class
            name="PurchaseOrder">
<field name="number" value="301"
            type="integer"/>
<field name="supplier" value="SGH Ltd"
            type="string"/>
<field name="date" value="16/08/2002"
            type="date"/>
<array name="orderLines">
<item
            name="PurchaseOrderLine">
<field name="number" value="1"
            type="integer"/>
<field name="quantity" value="1000"
            type="integer"/>
<field name="price" value="59.90"
            type="double"/>
<classfield name="article"
            id="01005"/>
</item>
<item
            name="PurchaseOrderLine">
<field name="number" value="2"
            type="integer"/>
<field name="quantity" value="500"
            type="integer"/>
<field name="price" value="239.90"
            type="double"/>
<classfield name="article"
            id="01007"/>
</item>
</array>
</class>

</classes>

It contains two customer orders and a purchase order. Now, the stock situation for article 01005 is changed because 50 pieces have been sold by the customer order 400, and 150 pieces by the customer order 401. Also, the purchase order 301 purchases 1000 new pieces. Hence, the stock is now 5800, as illustrated in this figure:

Application Setup

If you want to try this out for yourself, download the support material for this case study from the C#Today web site and follow the instructions in the readme.txt file included.

Case Study Review

This case study showed you a way to build a component to map XML in C# objects. The end component has the following properties: new instances of whatever predefined class can be imported; only elements written according to a predefined XML schema can be imported; no change to the target-class source-code has to be made in order to make it "importable"; and data-types are supported.

To develop the component we used XmlReader objects, XML schemas, and reflection. We reached the final release of the component in four steps: mapping simple objects, complex objects, arrays, and internal references.

Finally, a real-world example was developed. This showed how to integrate a component into applications. The sample application used the mapping component to import articles and orders from an external system to a logistics .NET-based system.

Any Limitations or Further Work

Obviously in this case study I could not completely cover such a vast subject, but by using simple improvements, the user can create a powerful tool to include in his own systems. Various improvements can be made. For example, the component does not map properties, enumerations, and base-type arrays. However, following these case study-guidelines it should be easy to add those features to the component. Moreover, another improvement might be to add a feature to create XML starting from the C# objects. In this way, two C#-based systems can both export and import XML data in the same format.

Acknowledgments

I wish to thank Marcello Valeri and Valentina Muraglia for helping me to plan out and review this case study. Special thanks are also sent to the C#Today team for their work.

Bibliography

S. Robinson at all, "Professional C# 2nd Edition", Wrox Press 2002.


Migrating from C++ to C#

Migrating from C++ to C#
http://www.csharptoday.com/content.asp?id=1909
Article Rating: 4.5
Published on September 30, 2002
Abstract
A C++ programmer cannot jump right into programming in C# just by knowing the syntax. The reason is that these two languages differ largely in design, even though they are syntactically similar. This change in design makes the language transition from C++ to C# a harder one. There are many cases where existing code in C++ needs to be converted to C#. This case study by SG Ganesh and GR Prakash is designed to help make the shift as smooth as possible. It also provides a clear picture of what to expect and what not to expect in such a transition. The approach taken and issues involved are discussed in detail, and the problems that arise when making the transition are highlighted, and feasible solutions shown. By reading this case study, you will get a good understanding of all the aspects of migrating code from C++ to C#.
Case Study

A C++ programmer cannot jump right into programming in C# just by knowing the syntax. The reason is that these two languages differ largely by design, though they are syntactically similar. This change in design and approach makes the language transition from C++ to C# a harder one.

There are many cases where existing code in C++ needs to be converted to C#. This case study is designed to help in making the shift as smooth as possible. It also provides a clear picture of what to expect and what not to expect in such a transition. The approach taken and issues involved are discussed in detail, and the problems that arise when making the transition are highlighted, and feasible solutions shown.

By reading this case study, you will get a good understanding of all the aspects of migrating code from C++ to C#.

Introduction

Since its beginning in the 1980s, C++ has come a long way. It has a large established user base, tested software, its own tools (compilers, etc), and lots of experienced programmers. It has also developed its own idioms and techniques for programmers to write effective software. C++ programmers are comfortable in getting things done with the facilities that are provided with it in an efficient manner.

.NET is a powerful new platform with a great deal of promise. C# is designed from the ground up to harness the power of this new framework. It provides a whole host of features and is strongly based on C++. C# is an object oriented language and is the first component-oriented language in the C family. It also makes writing Windows and Web applications faster and easier. C# is gaining wide acceptance and it is clear that it is here to stay for a long time.

C# is not a replacement for C++, and it is more than likely that both will be used widely for the foreseeable future. However, there are many practical cases where there is a necessity to migrate from C++ to C#. For instance, your company's policy may be to change all existing code to .NET, or perhaps you wish to take advantage of some of the facilities made available in .NET.

The question is, how do we make the transition as smooth as possible while getting the best results? Adopting a new language doesn't just mean converting the existing code from C++ to C#. By just knowing the syntax, a C++ programmer cannot straightaway start programming in C#. These two languages differ largely by their design and approach towards problem solving, which makes the language transition harder.

System Requirements

It is preferable that the reader has access to the C# compiler available in Microsoft Visual C#.NET. This case study is for those programmers coming from a C++ background, who are new to C# or have just started programming in it. The programmers with a good understanding of C# are in a better position to understand the approaches taken in the conversion process.

Case Study Structure

The case study consists of three main sections:

  • The approach

In this section, we briefly cover the basic theory that is necessary for understanding the issues in conversion. It is possible that you may not be clear about few of the C# features mentioned here - they are covered in the following section.

  • Comparing C++ and C# features:

In this section we look at the different features of the two languages which are necessary to make the conversion possible.

  • Steps in converting existing code:

The steps that are required for converting the existing code from C++ to C# are covered in this section. An example of converting class hierarchies from C++ to C# is also covered.

The Approach

What is the best approach for getting equipped for a smooth transition from C++ to C#? Understanding! Migrating from one language to another involves a considerable effort. This is not because of a change in syntax, rather because of changes in methodology - design approach, underlying technology, and the approach towards problem solving. Understanding that there is such a fundamental shift, and having the knowledge of where the major differences lie, will help a lot.

The underlying translation models for C++ and C# are quite different. C++ follows a static linkage model, meaning that the source code is compiled by the compiler to result in object code. The object files are linked to result in an executable file. The operating system loads and controls the execution of the program. The language features are designed with this approach in mind. For example, there is no support for reflection. Moreover, the code is only source code portable and not much runtime support is available.

C# follows an entirely different translation model - it combines compilation and interpretation. The source code is converted to an intermediate format known as MSIL (Microsoft Intermediate Language). A virtual machine, referred to as the CLR (Common Language Runtime), takes over to execute the instructions. The execution is thus in the hands of CLR and the code executed is referred to as managed code. This change in translation model is reflected in the language features as well.

A very important difference is in the area of memory management: the programmer no longer has the complete control of the lifetime of the objects in the heap. The garbage collector takes care of deleting objects whose lifetime is over. So there is no need for the keyword "delete". However, there are destructors in C#. If the "delete" keyword is not available in C#, then what is the use of destructors? In reality, the destructor syntax in C# is very misleading, especially for programmers from a C++ background. They are actually finalizers that are called before an object is garbage collected.

Another issue to understand is the change in design criteria. C++ is designed for experienced programmers in mind and 'trusts the programmer' (in the C tradition). So no extensive runtime checking is done, there are implicit casts and promotions in function calls. These features have proven to be very useful, but also very bug-prone. Therefore, only experienced programmers should use them. However, C# is designed so that even novice users can learn it fairly easily, and is also designed with robust software in mind. It performs extensive runtime checking with very few implicit conversions and tries to make the life of the programmer easier.

How can this understanding of language design change help in transition from C++ to C#? Let us use an example. A single argument constructor also serves the purpose of a conversion operator in C++. When a conversion is required, that constructor will be called implicitly because, it 'trusts the programmer': it is assumed that the C++ programmer is aware of it. Such implicit calls may lead to subtle bugs, like:

class Stack{
public:
Stack (int
            initivalCapacity);
// constructor that takes int as an
            argument
// other members
};

// now consider the
            code
Stack s;
s = 25;

// implicit conversion, a new
            Stack object is created with int as argument
// s = Stack(25);
// beware! the programmer may have programmed without being
            aware that the
// constructor with int argument is called for
            the conversion operation
// from int to Stack

To avoid such problems, you cannot use single argument constructors as conversion operators in C#. You have to support explicit conversion operators for that. Also, you can appreciate the use of implicit and explicit keywords in C# better. With this knowledge you are better equipped now. When you write equivalent C# code, you will also need to examine if a conversion operator needs to be implemented and decide if it should be declared as explicit or implicit if the original C++ code had any single argument constructors.

The problem solving approach also differs considerably in these two languages. Consider writing a simple calculator program. You require a postfix expression evaluator, and for that you may prefer to have your own reusable version of Stack. The interface for Stack is well-known and the logic is pretty straight forward. Still, your approach towards solving such problems may be entirely different depending on the language you use.

In C++, you would write a template class for the stack. If you want to evaluate an integral expression then you will instantiate an integer version from that Stack template class. It has its own benefits like static type checking. You can use this same implementation for any type of expression, for example floating point expression, without any changes. It is also extensible.

In C#, all objects come from the common base class 'object', and so you can write a Stack class which stores 'objects'. Since all the objects inherit from this class, you can store virtually any object in that Stack. When you retrieve the elements, you have to employ dynamic type checking to make sure that the types don't mix-up.

As you can see, even for the same well-defined problem, the problem solving approach differs considerably and you make a different set of decisions and an entirely different implementation depending on the language you are using!

Another important factor in the transition from C++ to C# is that it is a transition from an unmanaged environment to a managed environment. In C++ there is only trivial support from the runtime available, whereas C# has the sophisticated .NET runtime environment. C++ programmers need to make special efforts to understand the advantages with the managed environment. For example, reflection is a powerful feature which can be used to generate and execute assemblies dynamically. Runtime checks ensure that the security privileges are available for providing access to resources. You have array bounds checking, versioning support and most important of all - components that are created from any language can interact freely. However, it should be noted that the managed environment also comes with restrictions: you can no longer allocate objects anywhere you wish - you can only allocate to the heap. Also, you cannot do generic programming with templates as you could in C++, as .NET doesn't support it yet. The concept and benefits of a managed environment are new to C++ programmers, and hence exposure to the facilities with the underlying framework is essential to get the most out of C#.

In essence, having a broad picture of these two languages and understanding the differences in the underlying technology and approaches to design and problem solving are essential for migrating from C++ to C#.

Comparing C++ and C# Features

The first requirement that is needed to move from C++ to C# is a shift in your mindset. C++ is a language which trusts the programmer. This provides the programmer with the ability to do whatever he wants. This power does have drawbacks though - it can be misused and can end up causing major headaches. C# on the other hand, doesn't trust the programmer as much. It takes many of the responsibilities from the programmer and enables him to concentrate on the bigger picture. It removes a few features that were error prone, and introduces new ones that simplify programming.

Let us now compare the features available in the two languages.

Data Types

The types in C++ can be subdivided into three categories: primitive types, aggregate types, and pointer types. The primitive types are: bool, char, int, float, double, wchar_t. The aggregate types are those that are composed of other types. These include arrays, structures, unions and enums. Both pointers and references are called as pointer types. In C#, things are a little different, as it only has value and reference types. The value type is capable of storing data by itself, whereas the reference type cannot. It stores a reference, which points to the actual data. The value types can be thought of as equivalent to the primitive types in C++. They are derived from the class System.ValueType. These types can be stored in the stack frame of a method. The reference types cannot be stored in the stack frame, only in the heap.

However, a difference between C++ and C# data types are their size. While the size of most of the types is implementation-dependent in C++, we have fixed sizes in C#. We need to be cautious while converting between the available types. For example, in C++ there is long double, which is 10 bytes. There is no long double type in C#, and double occupies 8 bytes. There is a new type, decimal, available in C# that occupies 16 bytes. Since the decimal type occupies 6 more bytes than the long double (in C++) you may think that you should be able to store a value in decimal in 16 bytes what long double stores in 10 bytes. However, decimal isn't used to give a wider range, rather it's used for getting a more precise value, as in the case of currency values. If your intention of using long double is for higher precision, you don't have any problems, however if it was for a wider range you may have trouble.

Unsigned types are supported in C#, but they are better avoided because using them makes the code non CLS-compliant (Common Language Specification - compliant).

References

We have seen many C# programmers considering C++ references equivalent to C# references. This is wrong! Actually C# references are closer to C++ pointers. Remember that the references in C++ serve as a name alias. They are sure to point to an object, and sure to point to the same object throughout the scope of the reference. However, it's different in C#. Just like pointers, they can be defined without initializers, but they can point to different objects at different times, and they can even point to nothing - the null (actually they throw NullPointerException and not NullReferenceException when you attempt to refer a null reference!). So:

//C++
MyClass &ref = null; // error, references cannot be null
MyClass &ref = obj; // needs an initializer
ref = anotherObj // Error: can't change the reference

//C#
MyClass ref; // OK initializer not needed
ref = anObj;
ref = anotherObj; // OK change the reference
ref = null; // allowed

You can think of C# references as 'restricted and safe C++ pointers':

// C++
string * s;
s = new
            string;

// C#
string s;
s = new string();

Declarations and Definitions

With the discussion of C# references, you may have noticed one drastic difference in the semantics of the following statement:

string str;

This same statement will be interpreted in different ways by these languages. A C++ compiler sees this statement as a definition of a variable called str. It allocates a new stack object (or data area if declared globally) and calls the default constructor on the allocated object (a string object in this case). A C# compiler sees the same statement as a declaration for the reference variable str. It allocates space for the reference alone. It neither allocates space for an object nor calls the constructor. This should be done explicitly be the programmer:

str = new string();

This statement now allocates memory for the object in the heap and then calls the default constructor.

C# combines declarations and definitions together, whereas C++ clearly distinguishes between the two. For this reason, there are no function prototypes and forward declarations. The C# compiler carefully checks for definite assignment - you cannot use a variable without initializing it. Such facilities help avoid bugs, and greatly simplify the life of the programmer.

Structs

Except for the default access specifier, C++ never differentiates between structs and classes. Both of them are functionally the same. A struct can contain methods and can be inherited by a class, but C# takes a different path. Here the structs are just placeholders of other data types and no behavior can be specified. This means that the structs can no longer contain any methods. No classes can inherit from them. The advantage is that, as they are value types, they can be stored in the stack frame. They do not require any indirection and so are more efficient than classes.

When you want to group some related data where no methods have to be associated with them, structs are the best solution. When we want to model a real world entity with both data and methods, classes have to be used. For example, 'Point' in a graph is a simple aggregate type, and for that a struct can be used. Implementing a 'Vehicle' type may require encapsulating lots of data and methods operating on it, and for that, classes are better suited. Actually there are no hard-and-fast rules for deciding between structs and classes. A good rule-of-thumb is to use structs for the simplest aggregate types and classes for any non-trivial types.

One notable advantage when you use structs is that they are allocated on the stack itself and there is no memory overhead. Lots of memory will be saved when hundreds of objects are created, for example a big array of struct type. When you use a class type, the objects will be allocated on the heap and hence a lot of memory overhead is involved (In the current version of .NET, 10 more bytes are occupied for each heap object compared to an equivalent stack object!). So, using structs for small types can lead to saving significant amounts of memory.

MyStruct [] sArr = new MyStruct[10];

Whereas for the class type:

MyClass [] oArr = new
            MyClass[10];
for(int i = 0; i < 10; i++)
oArr[i] = new
            MyClass();

Arrays

Arrays are the simplest data structures that are widely used in programming. In C++, arrays are treated as a contiguous memory location. The low level nature of arrays create problems with object oriented programming. A base class pointer cannot be used to iterate through the array of derived class objects:

class Base{
public:
// Base class data
            members
virtual void boo();
};

class Derived: public
            Base{
public:
// Derived class data members
virtual void
            boo();
};

void foo(){
Derived dArr[10];
Base * bPtr
            = dArr;
for(int i =0; i<10; i++)
bPtr[i]->boo();
//
            Will not work properly
}

This is because, the size of the base class object may not be equal to the size of the derived class object. The compiler cannot identify the proper object at the time of compilation. C# is a dynamic language and has fully-fledged support from the run time. Arrays are no longer contiguous location in the memory. This makes such operations legal and safe.

C# does not treat arrays as mere continuous memory locations. It adds object-oriented characteristic by giving a class System.Array, from which all the arrays inherit. This class abstracts the operations on an array and can be casted into any of the arrays. Remember that arrays of all types are also derived from this class. As arrays are instances of a class, they are always reference types and this holds good for the arrays of value types. This helps in bound checking for every access in an array, but a problem is that has it to be allocated on the heap only.

Both the languages support rectangular and jagged arrays. For rectangular arrays, a chunk of plain memory locations are allocated and indexing is done on it. In C++, jagged arrays can be implemented by having a pointer array and allocating memory dynamically for each array. The same idea is followed in C#, but instead of pointers, references are used. This makes optimal use of space, since the sub-arrays may be of varying length. The compromise is that additional indirections are needed to refer to access sub-arrays. This access overhead is not there in rectangular array since all the sub-arrays are of same size.

// C++ language example for 'rectangular
            arrays'
float rectArr[5][20];

// C# rectangular arrays,
            note the difference in syntax
float [,] rect = new float
            [5,20];

// C++ language example for 'jagged arrays'
float
            **ptr;
ptr = new float *[5];
for (i=0; i< 5; i++)
ptr[i]
            = new float [20];

// C# example for 'jagged arrays'
float
            [] [] ptr;
ptr = new float[5][];
for(int i=0; i<5;
            i++)
ptr[i] = new float[20];

When more than one method of representation is supported, at some point the user will require to switch from one representation to another. Here, to convert from one array type to another, techniques called boxing and un-boxing are used (discussed later). It also should be noted that C# supports 'Indexer' members that allow array-like access to data structures.

Enums

Enumerations are of the type int in C and in C++; its type depends on the number of enumeration constants declared. C#, as an improvement over the old enumeration, allows you to specify the type of the enumeration:

enum holidays : ubyte{
Sunday = 0,
Saturday = 1
}

C# enums differ from C/C++ enums in that the enumerated constants need to be qualified by the name of the enumeration when they are used.

enum workingDay { mon,tue,wed,thur,fri
            };
workingDay today;
today = workingDay.mon;
//note that
            mon is qualified by workingDay

This name.member syntax helps the enumeration constants to remain in a separate namespace, thus preventing them from polluting the global namespace. Furthermore, it prevents name clashes between two different enums:

// C#: no name clashes with other enum members
enum Days { mon, tue, wed, thur, fri, sat, sun };
enum CosmicObjs { earth, mars, jupiter, sun, moon};
enum Companies {sun, microsoft, dell, digital, compaq};
myDay = Days.sun;
computer = Companies.sun;
cosmicObject = CosmicObjs.sun;

Variable Length Argument Lists

Experience has shown that programmers prefer C style printf format, because it is convenient for exact format specification and is easy to use. C# provides 'params' for the support of variable length argument lists. So you can write your functions using this facility as in:

int MyPrintf(string format, params object [] args);

For printing, C follows the format string with variable length argument strings; C++ uses << with cout; Java has overloaded the + operator. In C#, to print the arguments, the numbering should be as follows:

Console.WriteLine("{1} {2} {3}", i, obj, "someString);

Writing 'Unsafe' Code

C++ is good for writing low-level code, which is useful for programming systems with features like pointer arithmetic. C# understands the importance of that, and allows 'unsafe casts', pointers, and pointer arithmetic to be performed in code segments that are explicitly labeled as unsafe. Note that the keyword 'unsafe' may be misleading - it just specifies that is isn't managed code and that it may perform low-level operations. Also, it is not as easy nor as powerful as in C++.

Argument Passing

When we pass a variable to a method, we are not sure whether it will get modified or not. To ensure that the variable should not be modified, the programmer should use the const qualifier for that argument in that method. The absence of such const qualifiers indicate that the variable could be used for multiple return values in C#. It introduces two new keywords to achieve these multiple return values. If the method has multiple return values, it should explicitly use the ref or out keyword.

Furthermore, C# supports two new types of arguments: ref and out. When we pass an argument to a method, the caller should be aware that the parameter may be modified. The ref keyword indicates this. As wekk as during the method definition, the ref keyword is also used in the method invocation:

//C++
void foo(MyClass & arg1,
            MyClass & arg2){
// other code;
arg1 = newValue1;
arg2
            = newValue2;
}

foo(obj1, obj2);
// Note: the caller may
            not expect obj1 and obj2 will change

//C#
int foo(ref
            MyClass arg1, ref MyClass arg2){
arg1 = newValue1;
arg2 =
            newValue2;
}

foo(ref obj1, ref obj2);
// Now the
            programmer is aware that obj1 & obj2 may be changed

In a few cases, we may want to initialize the arguments only in the method. The use of the ref keyword will be flagged as an error by the compiler as a definite assignment has to be done before the first use. One elementary way to avoid the error is to initialize the variable with the default value and then to pass it to the method. C# introduces a new keyword for this situation. Instead of ref, we can use out, which doesn't force the caller to initialize the variable. However, it is mandatory for the method to assign some value to it.

//C#
void foo(ref MyClass arg1, out MyClass arg2){
// other code;
arg1 = someValue; // optional
arg2 = someValue; // need to assign some value
}

MyClass obj1, obj2;
obj1 = aValue; // need to initialize
foo(ref obj1, out obj2); // note obj2 is not initialized

Class Abstraction

Just like C++, the basic unit of abstraction is a class. The access specifiers public, protected and private have the same meaning in both the languages. In addition, C# provides internal and protected internal access specifiers. The internal members are available to the whole assembly and the protected internal to the assembly and the derived classes. Why do you ever need these access specifiers? There are few cases where you need to access members of other classes in the same assembly but shouldn't be exposed to the external classes. Since friend access is not there in C#, this can be a useful feature particularly when you are designing libraries.

Inheritance

C# doesn't support multiple class inheritance. It only supports single inheritance, but you can still inherit from multiple interfaces. Pure abstract classes in C++ can be treated as interfaces in C#. There are many restrictions in using interfaces for inheritance. You can only have public abstract methods, and no fields are allowed (not even const fields). However, one interface can inherit from another interface.

C# only supports public inheritance. Not having private or protected inheritance doesn't affect the functionality as such. There are a few inconveniences with this approach, for example, once you implement ICloneable, all the classes that inherit from that class becomes automatically cloneable, as only public inheritance is available.

The Object Base Class

C# doesn't support templates as .NET doesn't support it yet. However, a weaker form of generic programming is supported in C# through the System.Object base class. This is the apex class for all the objects. This includes the value types like structs and ints and reference types like arrays and strings. This property is exploited in the Collections provided in the framework that works in terms of Objects.

The standard libraries of both C++ and C# provide support for the container classes. Consider this example of using the vector class:

MyClass obj;
string str = "string
            object";
const int size = 5;

vector<MyClass>
            vect(size);

vect[0] = obj;
vect[1] = str;
// Compiler
            Error: vect can store only MyClass and not others

// insert
            more elements
// iterator provides a pointer-like syntax for
//traversing the
            container

cout<<vect[0]<<vect[1]<<endl;
vector<int>::iterator
            iter = vect.begin();
while(iter != vect.end()){
cout <<
            *iter;
// calls overloaded << operator of MyClass
iter++;
}

Thus, you can have elements of only one type, and the traversing and accessing is done through iterators. With C#, .NET provides an equivalent container class for vector - the ArrayList container:

// Creates and initializes a new
            ArrayList
MyClass obj;
string str = "string
            object";
ArrayList arrLst = new ArrayList();
arrLst.Add(obj);
arrLst.Add(str);

// can simply use foreach statement for
            traversing the colletion
foreach(MyClass elem in
            arrLst){
Console.WriteLine( " {0} ", elem);
}
// throws
            'InvalidCastException' as the second element is a string

Operator Overloading

In C++ almost all the operators can be overloaded - there are only a few operators like the conditional operator, . operator, .* and .-> operators that cannot be overloaded. C# provides support for operator overloading but to a limited extent. The syntax for overloading the operators is:

// C++
<return type>
            ClassName::operator <the operator> (arguments)
// usage
            example
class MyClass{
public:
MyClass operator + (MyClass
            &rhs);
};

// C#
<return type> public static
            operator <the operator>(arguments)
// usage example
class MyClass{
public static MyClass operator + (MyClass lhs,
            MyClass rhs){}
}

The main difference is that while you can have member or global (mostly friend) functions in C++, you have static methods for overloading in C#.

Although the syntax looks similar there are a few constraints imposed by C# for operator overloading. The most important are:

  • The methods should be declared as public and static.
  • Many of the operators are required to be overloaded in pairs. For example, if you define == you should overload the != operator also.
  • If you define the + operator, the compiler defines the += operator for you to make things easier.

//C++
class CPPClass{

protected: // can be public or protected or private

bool operator ==(CPPClass &rhs);
// the another argument is passed implicitly by 'this' pointer
// note no != operator defined

bool operator ++(); // type of the return value is not forced
bool operator +(CPPClass &rhs);
bool operator +=(CPPClass &rhs);
// += is not implicitly defined

static int operator-(CPPClass &lhs, CPPClass &rhs);
// both static and non static methods are allowed
};

//C#
class CSharpClass{

//note that all the operators are public and static

public static bool operator ==( CSharpClass lhs, CSharpClass rhs){}
public static bool operator !=( CSharpClass lhs, CSharpClass rhs){}
// relational operators should be overloaded in pair

public static CSharpClass operator ++(CSharpClass arg);
// return type and argument types are forced for few operator

public static bool operator +(CSharpClass rhs);
// += is implicitly defined by the compiler when
// binary + is defined
}

Exceptions

Exception handling in C# is similar to C++. The exception specification of a method lists all the possible exceptions that the method might throw. In C++, when the method doesn't lists any exceptions, beware that it is then allowed to throw any exception, and there is no constraint for a method to catch the exceptions thrown. Further, exceptions are not only thrown in the form of classes, but also in the form of primitive types.

The C# exception handling mechanism is much simpler and more elegant. Firstly, a method cannot throw the exceptions that are not listed in the exception specification of the method. Catching of the exceptions is mandatory and only objects of Exception (or derived from) are thrown.

//C++
void foo(){
throw 10;
throw
            MyException();
throw "This is an Error";
}

void boo()
            throw (int, Exception){
throw "Something is wrong";
//Error:
            can only throw int / Exception
}

void doo() throw
            (){
// guaranteed that no exceptions will be
            thrown
}

//C#

void foo(){
//will not throw any
            exception
}

void boo() throws IOException{
throw new
            IOException(); //OK
throw new MyException();
// Error:
            allowed to throw only IOException
}

Namespaces

Namespaces are supported in C++ for better organizing the code and are valuable in large-scale programming. In C#, the syntax for declaring and organizing classes in a namespace is similar to that of C++. There is no concept of header files (C# design is such that there is no need for header files, for example, it combines declarations and definitions) and you have to use the using directive to open up the members in the namespace for access in the code. You can also have aliases:

using alias_name = namespace_or_type;

Just like in C++, you can have nested namespaces. The syntax is a bit different:

namespace outer.inner{
// some members
}

Note that you have to use one namespace within another for a similar goal in C++:

namespace outer{
namespace inner{
//
            some members
}
}

There is an importance difference between the namespaces in C++ and C#. In C++, namespaces are logical entities and no physical enforcement of namespaces exists. However in C#, in addition to logical separation, a physical separation of namespace members and enforcement of hierarchy is there in the form of assemblies and sub-assemblies. This enables the namespace rules to be enforced at the physical level.

Properties

It is common for a C++ programmer to give the get and set methods for data members. Not only does this help in abstracting the details, but it also gives a few advantages such as that the user cannot assign illegal values to the field, such as 500 to a field called age, or the programmer can give a read-only version of the member, such as size of a container, etc.

class MyClass{
private:
int
            someInt;
int length;
public:
inline int
            getLength(){
return length;
}
inline int
            getSomeInt(){
return someInt;
}
inline void setSomeInt(int
            arg){
if(arg >= minValue && arg <=
            maxValue)
someInt = arg;
else
error("illegal
            value");
}
};

//usage:
MyClass
            anObj;
anObj.setSomeInt(100);
int len =
            anObj.getLength();

As most of these methods are inlined, the performance isn't affected. However there are two problems with the usage of such functions. The first is that the syntax of accessing them is a bit unwieldy. The next is that the approach itself violates the object oriented programming guidelines. An object is supposed to expose a behavior and not the implementation. By these methods, obviously the object exposes its private fields to the user. C# provides a whole new way to handle this situation through properties.

Properties are very much like the get-set methods, but syntactically different. Consider this example written with properties in C#:

class MyClass{
private int
            someInt;
private int length;
public int
            Length{
get{
return length;
}
}
public int
            SomeInt(){
get{
return someInt;
}
set{
if(value >=
            minValue && value <= maxValue)
someInt =
            value;
else
error("illegal
            value");
}
}
}
//usage:
MyClass
            anObj;
anObj.SomeInt = 100;
// set the value of the field
            through mutator property
int len = anObj.Length;
// get the
            value of the field through accessor property

Note that a variable value is used in the set method. It is the implicit parameter passed to the method by the compiler. Its type is the same as that of the property. As we can see, the syntax is more intuitive to use.

Indexers

We tend to have many container classes that are used to hold a set of objects. Stacks, Queues, Maps and Hashtables are just a few such important containers. There are many other objects that can also be viewed as containers. For example, a menu can be thought of a container of the menu items. In most cases we will need to access the objects in the containers through an indexer. In C++ this can be done by overriding the array subscript operator []. We can override it not only with integers, but with any object we want, which sometimes makes the subscripting more meaningful:

class
            EmployeeContainer{
private:
Employee
            emp[100];
public:
Employee& operator[](int
            empNo){
//return the employee with the
            empNo
}
Employee& operator[](string name){
// return
            the employee with the name
}
};

void
            foo(){
EmployeeContainer empCont;
// add the employees to the
            container
Employee emp1 = empCont[5];
empCont["Pranni"].age =
            24;
}

C# introduces the indexers to fit this problem of indexing a container. The equivalent Employee class can be written in C# as:

class EmployeeContainer{
private Employee
            emp[100];
public Employee this[int empNo]{
// implement it
            like a property
get{
return emp[empNo];
}
set{
emp[empNo] = Employee;
}
}

public Employee
            operator[string name] {
// implement it like a
            property
get{
// getting Employee index mapped by string
            info
}
set{
// code for setting Employee detail at index
            position
}
}
}

void foo(){
EmployeeContainer
            empCont = new EmployeeContainer();
// add the employees to the
            container
Employee emp1 = empCont[5];
empCont["Pranni"].age =
            24;
}

Attributes

Attributes are a significant addition to C#. When you are creating your own types or components, there is a necessity to associate related details of the components and their elements. In COM you used type libraries to achieve such functionality. Traditionally, comments and macros are used in C++ programming for storing the metadata about the class and/or its members. C#'s attributes are far more powerful and you can give meta-information for many language elements: fields, methods, events, etc. You can retrieve and examine such meta-information at runtime using reflection (discussed later). There are two types of attributes: intrinsic (predefined) and custom attributes.

C# supports a preprocessing facility but there is no separate tool - it is handled by the compiler itself. The preprocessor support has restricted use though, for example you cannot have macros. One of the uses of the preprocessor is conditional methods and that is achieved through Conditional attributes. It is an intrinsic attribute used for including the method depending on the condition. In C++ you use preprocessor facilities directly.

//C# code

#define DEBUG
// such
            definitions should occur only in the beginning

class
            MyClass{
[Conditional("DEBUG")]
public static void
            debugFunction(string
            message){
cout<<message<<endl;
}
// other
            members
}

C#'s conditional methods are very powerful when used with the Debug and Trace classes available with the System.Diagnostics namespace. There are many such useful attributes; one is Serializable, which is discussed later.

You can define your own custom attributes. You have to derive your class from the AttributeUsage class. Here is one simple example for maintaining the code comments from the author of the code:

using
            System;
[AttributeUsage(AttributeTargets.All,
            AllowMultiple=true)]
// tells that this attribute can be used on
            any program element
// and there can be multiple entries for
            each use of attribute

public class CommentAttribute :
            Attribute{
public CommentAttribute(string
            comment){
this.commentText = comment;
}
private string
            commentText;
public string CommentText{
get{
return
            commentText;
}
}
}

[Comment("Written by Ganni and
            Pranni")]
class GuineaPig {
// ...
}

class Test
            {
public static void Main(){
Attribute[] attributes =
            Attribute.GetCustomAttributes(typeof(GuineaPig));
//This static
            method GetCustomAttributes
//is used to retrieve the attribute
            info
foreach(CommentAttribute attribute in attributes)
Console.WriteLine(attribute.CommentText);
}
}

You can use the custom attributes with the same special syntax as in intrinsic attributes and there is no need to call the constructor explicitly - you can initialize the attribute directly. The static method GetCustomAttributes of the Attribute class is used for retrieving the attributes by passing the type.

Callback Functions

Function pointers are a useful facility in C/C++. The following example shows a real world example of using function pointers. Say you want to write a menu program. The aim is to write a program that will call a corresponding function that is selected in the menu at runtime. Therefore, we have to declare a function pointer whose signature matches the functions that are written for the menu:

void (*menuSelector)( );
// get the
            input from the user - selection of the menu item

switch(select){
case NEW : menuSelector = & New( );
            break;
case OPEN : menuSelector = & Open( ); break;
//
            assign the address of the corresponding function to
            menuSelector
}

menuSelector( );
// now call the
            selected functionality

The calling of functions using these function pointers, whose value is determined at runtime, is known as 'call back'. C# provides support for callback functions and it is called 'delegates' (you can also consider it as an improved version of the 'function objects' in C++).

Delegates closely resemble function pointers, and C# promises that delegates are type-safe, secure, and object-oriented. A delegate is capable of a holding a reference to another function so that function can be called later. Even multiple functions can be installed like that. Callbacks are valuable for event handling. C# also supports events that are useful in the case of event driven programming like Windows Forms:

public delegate void Selector();
//
            Selector is the type that can be used to instantiate
//
            delegates that take no arguments and return nothing

public
            Selector menuSelector;

public void
            New(){
Console.WriteLine("You selected 'New'
            option");
}

public void Open(){
Console.WriteLine("You
            selected 'Open' option");
}

string select;
// get the
            value of select from calling the menu...

Test t = new
            Test();
switch(select){
case "New" : t.menuSelector = new
            Selector(t.New);
break;
case "Open" : t.menuSelector = new
            Selector(t.Open);
break;
// ...
// register the selected
            method to menuSelector
}
t.menuSelector( );
// call the
            delegate and it will inturn call the registered method

Reflection and RTTI

When doing object oriented programming, we treat an object as if it were an more general type. So, for example, we can view a Dog as a mammal, an animal, or even simply a living thing. So when we have a more generalized version, sometimes we would like to know what the exact type is and act accordingly. Say if we have a living thing, we would perform some operations on a mammal, that we wouldn't on an amphibian. We would perform even more specific operations if it were a Dog. In such cases, RTTI (Run Time Type Identification) comes into the picture. C++ provides the typeid operator and a set of classes that enable the querying of the type of an object at runtime. This operator will return the exact type of the object only if there is at least one virtual function in it:

class Base{
// no virtual methods
void
            Base1();
void Base2();
};
class Derived1 : public
            Base{
virtual void vMethod();
};
class Derived2: public
            Derived1{
};

void foo(){
Derived2 d2Obj;
Base*
            bPtr;
bPtr =
            &d2Obj;
cout<<typeof(*bPtr)<<endl;
Derived1
            *dPtr;
dPtr =
            &d2Obj;
cout<<typeof(*dPtr)<<endl;
}
//output:
//
            class Base
// class Derived

Reflection is a feature available only in dynamic (interpreted) languages. Reflection is a powerful facility as we can dynamically load classes, create objects, change their properties, and invoke methods on it. Although fully exploiting the power of reflection will not be explored in this case study (see http://www.csharptoday.com/content.asp?id=1852&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgqfor a whole piece on the topic), here is a sample that loads an assembly and invokes its methods dynamically:

using System;
using
            System.Reflection;

class ReflectionTest{
// this method
            will be called dynamically
public void
            InvokeDynamic(){
Console.WriteLine("Hello, dynamic
            world!");
}
public static void Main(){
Type t =
            Type.GetType("ReflectionTest");
// get the type by passing the
            name of this class
MethodInfo m =
            t.GetMethod("InvokeDynamic");
object o =
            Activator.CreateInstance(t);
// Activator is a class defined in
            System namespace
// you can use it to create objects (remote or
            local)
m.Invoke(o, null);
// the second argument is the list
            of arguments passed
// to Invoke - null in this case
}
}
// output:
// Hello, dynamic world!

Memory Management

Moving from C++ to C# takes away a lot of the programmer's freedom. C++ allows you to determine whether to create an object in the stack or on the heap, whereas C# doesn't. The change from unmanaged to managed environment has drawbacks to. C# is a dynamic language and all the allocation is done on the heap. Only value types are allocated on the stack. So, you have to allocate the memory for all the objects on the heap manually, even for those objects you used to allocate statically in C++. In C#, in addition to using 'new' for dynamic allocation for heap objects, you can use it for stack objects (structs) to call the constructors.

The difference in where the objects are allocated is significant. For example, when casting is done from a value type to a reference type, memory needs to be allocated on the heap and initialized. This process is referred to as 'boxing'. For example:

int i = 10;
object o = i;

Note that you don't need an explicit cast here, as it is an 'upcast'. When the conversion is done from reference type to value type, it is referred to as 'unboxing'. However you need explicit casting to do that as it is a 'downcast':

int i = 10;
object iRef = i;
int j = iRef + 100; // doesn't compile, needs explicit cast
int k = (int)iRef +100; // now OK

Such conversions are not possible in C++ as there is no common base class. Boxing and unboxing are costly operations and need to be avoided whenever possible as it involves creation and destruction of objects.

Garbage Collection

The burden of managing the memory is greatly reduced in C#, as the garbage collector automatically reclaims the unused/unreferenced objects. With garbage collection, most of the problems with managing the memory like dangling pointers and memory leaks are gone. Garbage collection is only for memory objects, but there are other resources like network connections that need to be released when the object is recollected. This is done in the finalize method. C# still supports C++'s destructor syntax, but C# destructors are 'syntactic sugar' for finalizers.

~MyClass(){
// release resources like
            database connections }

is equivalent to:

protected override void
            Finalizer(){
try{
// release resources like database
            connections }
finally{
base.Finalize();
}
}

which is little tedious to type, and hence the destructor syntax is convenient. The meaning of destructors is not the same in these languages even though the syntax is the same. There is no assurance that the object will be garbage collected or finalizers will be called immediately when there are no more references to that object. If there are important resources like file handles or database connections that are released in C++ destructor code, you shouldn't go for Finalize in C#. Rather, you have to implement the IDisposable interface, override the Dispose method, and write the code for releasing such connections or handles.

using
            System.Runtime.InteropServices;

class MyClass :
            IDisposable{

MyClass(){
// get resources
}

public void Deallocate(){
// code for releasing
            resources here
}
public void Dispose(){
Deallocate();
GC.SuppressFinalize(this);
// since Dispose is called, the
            Finalize method should
// not be called... so tell GC to
            suppress call to
// Finalizer method
}

~MyClass(){
Deallocate();
}

public static
            void Main(String []args){
MyClass obj = new MyClass();
// use
            obj;
obj.Dispose();
}
}

To be more precise: it is not possible to determine exactly when the garbage collector will be called, and so C# doesn't have deterministic finalization. To overcome this, you have to implement the IDisposable interface and provide the implementation for the Dispose method. After you use the object, you can release it by calling the Dispose method explicitly. Who is responsible for calling this method for objects that are from various sources? The time honored C++ principle of disposing heap objects applies to this also: 'whoever allocated the memory has to recollect it'.

Steps in Converting Existing Code

There are cases where systems that are written in C++ need to be ported to C#. The .NET environment can use C++ code directly in two cases:

  • When the classes are written in Managed Extensions to C++
  • If they are COM components

If the application is written as COM components, then the component can be used directly in .NET. In the case of COM components, you can use the Type Library Importer (tlbimp.exe) utility. It reads the COM type library information and converts it to an equivalent .NET assembly as a proxy class that contains the necessary metadata. However, it should be noted that the code is still unmanaged.

'Managed extensions to C++' (MEC) is a set of extensions to the C++ language provided by Microsoft that can be compiled to code targeting .NET environment. Most of the existing C++ code is not for component programming; so the code cannot be used directly in C#. MEC is new to the programming world and hence there is no possibility that legacy code is written in that.

C# provides support for low-level programming and has facilities to make use of legacy code. For example, the methods that are available in the DLLs can be accessed by declaring such methods with the DllImport attribute. You have to declare such methods as extern - it has a similar use as in C++ for accessing methods from other languages. It can be applied only to methods implemented externally. Say, you want to use your favorite MessageBox in traditional Windows programming:

[DllImport("User32.dll")]
public static extern int MessageBox
(int h, string m, string c, int type);
// now you can use it in your C# code

This feature is of great use if yours is a code library or framework and not a full-fledged application. You just need to declare the methods in your C# code and can make use of them by storing them in DLLs.

When you want to convert existing C++ code to run under the .NET platform, the following decisions need to be made. If the code is simple enough that it can be rewritten without much effort, then you can go for C#. Practically, C++ code may involve low-level programming like accessing hardware features. Such functionality can be done in C# itself to some extent due to its support of C like structures and allowing restricted use of native pointers. At the level where full-control over resources is required, you can do explicit memory management as well. Such code should be done in 'unsafe' blocks. If it is complex enough that it cannot be handled with the facilities that are available in 'unsafe' then direct conversion could be made from C++ to Managed Extensions to C++. Code written like that is accessible from C# code. All this means that the tested, legacy C++ code need not be discarded and you can still use it under .NET environment, albeit as unmanaged code.

Thinking of one-to-one correspondence of functionality leads to poor design and fragile code. Translating C++ code on a line-by-line basis is not feasible as the two languages differ considerably in their functionality and support. Let us illustrate this with an example. In C#, all the functions have to be abstracted inside classes, as no global functions or data is supported. C# doesn't support global variables/functions because it strictly enforces class as the basic abstraction mechanism. So, when you are moving to C#, it is better to stick to the C# mindset - don't think in terms of C++. To illustrate how these ideas materialize, consider the following example of converting the class hierarchies.

Converting the class hierarchies

Designing class hierarchies differs drastically in C++ and C#. This is because multiple class inheritance is not supported in C#, only public inheritance is. Consider the following hierarchy available in C++:

class Base1{
// pure abstract base
            class
}

class Base2{
// abstract base
            class
}

class Base3{
// concrete
            class
}

class Derived: public Base1, protected Base2,
            private Base 3{
}

Base1 can be represented as an interface as a C++ pure abstract base, which is equivalent to an interface in C#. The Base2 can be an abstract class in C#. The problem arises here because multiple class inheritance is involved, as there can be only one base class in C#. If possible, try to convert Base2 into an interface. That implementation is available for a few of the methods. In the other cases, those implementations can be provided in the concrete class, thus making Base2 as an interface feasible. The problem arises when there are data members. In that case, having it as an interface is not feasible - moving data members is not advisable.

In general, this can be solved by having Base3 inheriting from Base2. Since Base2 is an abstract class, it can better serve as base, rather than Base3 serving as a base class for Base2. The C++ code has private, protected, and public inheritance. How can they be handled in C#? Note that C# supports only public inheritance. So, you are forced to use public inheritance for all the three types of inheritance supported in C++, public, private and protected. Using public inheritance doesn't affect the functionality. The real difference lies in abstraction. In C# solution, all the members are exposed and the hierarchy looks like this:

interface IBase1{
}
// the naming
            convention in C# suggests interfaces to use I prefix before
            name

abstract class Base2{
}

class Base3 : Base2 {
}

class Derived: IBase1, Base3{
}

Having the exact C++ hierarchy in C# is not possible. However, this can be achieved to some extent by understanding the inheritance model supported in these languages.

Case Study Review

Migrating from C++ to C# is not easy as it may seem. C# is strongly based on C++, but the two languages differ in their design. The syntactic similarities between the two languages can be misleading, as there are many semantic and pragmatic differences. There are many places where the C++ programmer will truly get lost when he starts programming in C#.

A C++ programmer needs to have a good understanding of the migration process and should be clear in his/her approach to get best results from such a transition. The two languages differ in many fundamental ways: design approach, memory management, problem solving approach, and the underlying translation technology are just a few differences. To get the best results, it is essential that the programmer has an overall view of such issues.

The second section of the case study is not just looking at the differences in features. Rather, it's a discussion of how the transition can be done from C++ to C# by analyzing its features. Naturally, a clear picture emerges of what to expect and what not to expect in such a transition.

When there is a necessity to convert the existing code from C++ to C#, a set of decisions needs to be made. If the code is available as COM components, it can be used directly instead of manually converting the code. If the code is a library/framework available as DLLs, then no conversion needs to be done and it can be used directly in C#. Managed extensions to C++ can be used for minimal changes in the code and the application becomes available in the .NET environment. A decision needs to be made if it is necessary to rewrite the whole code in C#. In that case, line-by-line conversion of code is not feasible and such transition will need significant effort on the programmers part. It will also necessitate a change in design approach and new strategies.


Programming Web Services with C# Using SQL

Programming Web Services with C# Using SQL
http://www.csharptoday.com/content.asp?id=1850
Article Rating: 3.5
Published on July 15, 2002
Abstract
The purpose of this case study by Jeannine and Glenn Gailey is to give C# developers an overview of XML Web Services and related technologies, an explanation of why they would want to build an XML Web Service that interacts with SQL Server, and to walk through a demonstration of how to build just such a web service.
Case Study

The purpose of this case study is to give C# developers a brief overview of XML Web Services and related technologies, an explanation of why they would want to build an XML Web Service that interacts with SQL Server, and to walk you through a demonstration of how to build just such a web service. For the purposes of this case study, the service will be fairly simple: a web service, built with Visual Studio .NET using C# that allows you to request an XML document that represents a user's resume data stored in a relational SQL Server database.

System Requirements

To get the most out of this case study, you should already be familiar with general C# syntax as well as with .NET programming practices and the Microsoft .NET Framework. You should also have at least some general knowledge of XML, Structured Query Language (SQL), and relational databases.

In order to recreate or run the code included with this sample, you will need to have the following installed on your machine:

  • Microsoft SQL Server 7.0 or later
  • Microsoft Visual Studio .NET (tested with build 9466)
  • Microsoft Data Access Components (MDAC) version 2.7 or higher (installed by Visual Studio .NET)
  • Microsoft .NET Framework (installed by Visual Studio .NET)

Introduction

The first section of this case study will give you an overview of XML Web Services and the underlying technologies, including brief overviews and samples, as well as .NET Web Services architecture. The second section will give an overview of the design of the resume server Web service, including why and how you would use SQL Server 2000 for an XML Web Service. We will also discuss the benefits of SQL Server 2000 over SQL Server 7 (or earlier) when developing XML Web services. The third section will walk you through how we built the resume server service, including showing C# code examples, designing the database, and deploying and testing the service using Visual Studio .NET.

It is safe to say that the Internet has forever changed the way in which many individuals do business and conduct their affairs. Why call your stockbroker to find out how your portfolio is doing when you can find this information online? No need to trudge down to the ticket office or stay on hold forever trying to get tickets for your favorite sports team; now you can look up the home games schedule and purchase tickets for the seats of your choosing with a few clicks of your mouse. Many people now pay their bills and transfer funds at a Web site rather than handing a deposit slip to a cashier. No longer a novelty, accessing information on demand via the Internet has become a regular event. Now, with personalized Web portal pages that are configured for users' preferences, users are becoming accustomed to a more individualized Internet experience, while at the same time investing more and more in non-PC devices that have access to the Internet.

What Are Web Services?

Web services are essentially a new and efficient way for companies to provide data that is stored in their secure and proprietary databases to their customers (whether other XML Web services or end-user client applications). For example, Google now provides an XML Web service that allows third party web sites to directly search the data in its database.

The vision of Microsoft, IBM, Sun and many other companies is to use XML Web services as a way to provide a discoverable, standards-based architecture to loosely coupled applications in a decentralized environment. Web services are essentially self-describing applications that can be published, located, and invoked across the Internet. These services can provide functionalities that range from simple requests to complicated business processes. Access to these functionalities is provided by XML-based messages defined by the Simple Object Access Protocol (SOAP). Since a Web service is discoverable and accessed via standard protocols (like XML and HTTP), it can be used by other applications or Web services that "speak" to these protocols without requiring that these clients possess detailed knowledge about the service's underlying implementation.

For example, when a client accesses a Web service method, it only knows about what is needed to invoke the method. The client has no idea whether the request is serviced by an underlying COM-based service running on a Windows NT Server or by a JAVA-based service running on a Sun Solaris server. Furthermore, since SOAP is a text-based, XML-based protocol, the method call can be made through the firewall using HTTP on port 80.

Since XML Web services use SOAP to provide access to that service's functionalities via HTTP, a client application that consumes an XML Web service can be running on any device or on any platform (Linux, UNIX, Solaris, and others) with internet connectivity. This means that other non-PC devices, like cell phones, PDAs, Tablet PCs, and even web-enabled appliances, can consume SOAP messages and transform the encapsulated information as needed for a particular device. For example, consider that a certain XML Web service provides stock information (stock tickers were, in fact, some of the earliest consumer XML Web services). The SOAP message containing the requested stock price information would be the same whether it was requested by an application running on a PC or on a cell phone.

Architecture

In the past, the interaction between applications on different platforms has been a tightly-coupled process. While a server-side component might be designed to work well with a particular client, reusing this component to service a different client application usually involved learning the programming interface of, and writing additional custom code to handle, each new application and platform. This interaction model is cumbersome and labor intensive at design-time as both parties must agree on things like data types, message formats, and bindings.

Windows DNA Architecture

Many of you will be familiar with the traditional Windows DNA architecture as depicted in the following diagrams. The Windows DNA architecture is an n-tier solution based on the tightly coupled Component Object Model (COM) and Distributed Component Object Model (DCOM) technologies. In this distributed architecture, when a DCOM client needed to call a method on a remote DCOM server, the COM service on the client created a proxy object for "marshaling" the procedure call. This marshaling process involved serializing data from the client into a bit-stream of binary data that was de-serialized at the other end. The following diagram shows the traditional n-tier application architecture:

XML Web Services Architecture

Putting a new twist on Windows DNA and other traditional client-server architectures, XML Web services employ a message-based call structure. This enables the exchange of data between heterogeneous systems that are loosely coupled. Rather than having to marshal a call into a stream of platform-specific binary data, a SOAP message is serialized as a standard XML stream. XML Web services can be implemented on any platform and adhere to public standards being developed by organizations such as the World Wide Web Consortium (W3C). XML Web services allow server applications to provide their functionalities via the Internet (even through firewalls) to other applications, regardless of the platform or device on which the client application is. Below is an example of this architecture:

Web Services Implementation

Implementing an XML Web service requires the following high-level tasks, all of which are greatly simplified by developing your service in one of the number of emerging integrated development environment (IDE) that support XML Web services:

  • Code your application. You can either write a new managed code Web service application or port an existing COM-based application.
  • Setup a SOAP listener. In order to respond to SOAP requests, you need to be able to properly receive and handle them. The listeners parse the incoming SOAP messages and call the appropriate objects. Listeners must be active at both endpoints of the SOAP communication. Microsoft provides both an ASP and an ISAPI listener. The ISAPI listener gives you better performance, but the ASP listener allows you to easily run custom processing on incoming SOAP messages.
  • (Optional) Develop discovery and description mechanisms for your service. We will discuss discovery and description later in this chapter.

The Role of Open Standards

XML Web services can be built in a variety of ways because they are by nature flexible, and can be implemented on whatever platform the architects are most comfortable working with. In this age of distributed computing, people expect applications to "talk" to each other, but seamless communication usually involves writing lots of extra code. Instead of wasting time writing this customized code to handle communications with various remote interfaces, you can spend more time writing the features that your users want.

The difference between developing an XML Web services-enabled application and developing a traditional application that uses remote procedure calls on COM or DCOM is the manner in which an object's interfaces can be accessed. Prior to XML Web services, communication between applications required writing custom code to handle incoming data streams of various formats on varying platforms. Instead, an XML Web service exposes its object's interfaces through open Internet protocols. That way, instead of an implementation that requires that remote procedure calls be made via some platform-specific binary protocol, developers can access remote objects using a ubiquitous Internet messaging standard. Although there can be a performance impact to processing XML instead of binary formats, for many, the interoperability potential outweighs these costs. A Web service can be seen as made up of the following four layers:

  • Service Transport Layer - uses standard Internet transport protocols like TCP/IP and HTTP and provides message transport between applications.
  • XML Messaging Layer - defines a message using an XML message-based protocol, like SOAP, to encapsulate application information.
  • Service Description Layer - describes the XML-based interface using a protocol like WSDL to simplify consumption of a web service.
  • Service Discovery Layer - lists XML Web services in a centralized, common registry, using a protocol like UDDI, so that applications can easily find the appropriate service that they need.

The following sections introduce the technologies that are the building blocks for XML Web services. All of these technologies are either approved or proposed as standard Internet technologies.

Internet Communication Protocols

Traditionally, a number of protocols could be used to connect to and transfer data via the Internet, including:

  • Transmission Control Protocol/Internet Protocol (TCP/IP) - the ubiquitous, stream-oriented protocol that is the base protocol for all internet messaging
  • User Datagram Protocol (UDP) - for simple, packet-oriented communication
  • Hypertext Transfer Protocol (HTTP) - for transmitting text-based markup via TCP/IP. Hypertext is transported through firewalls via port 80.
  • Secure Socket Layer (SSL) - an authentication and encryption protocol for communication over TCP/IP. The HTTPS protocol is essentially HTTP that uses SSL and connects to a port other than 80.
  • File Transfer Protocol (FTP) - for transferring binary files

Like most Internet applications, XML Web services use HTTP as its primary message transport. This allows messages to be transmitted through the firewall on port 80, avoiding the hassle of having to open new ports to enable the traditional binary communication used by DCOM or the Common Object Request Broker Architecture (CORBA).

In addition to these traditional Internet communication protocols, XML Web services have spawned two new XML-based message protocols, namely Simple Object Access Protocol (SOAP) and Direct Internet Message Encapsulation (DIME). We'll cover SOAP and DIME shortly. First, let's look some more at XML.

XML

The beauty of XML is that it is a self-describing, data-centric format. This means that all the information that an application needs in order to use the data in an XML document is supplied by the XML mark-up. As a result, an XML data stream can be easily interpreted and rendered appropriately, regardless of the specific client device accessing the information. For example, when an incoming XML document contains data for an e-mail message, that XML can be parsed, rendered, and formatted either for display as an HTML page or for a handheld device like a PDA or cellular phone. Also, because XML is a simple text based format, it can be easily transmitted using HTTP.

SOAP

The new (but already entrenched) Internet protocol known as SOAP is an XML standards-based protocol that enables XML Web services by prescribing a standard delivery mechanism for XML-based message payloads.

SOAP is officially defined by the W3C (http://www.w3.org/TR/SOAP?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq) as a protocol that "provides a simple and lightweight mechanism for exchanging structured and strongly-typed data between peers in a decentralized, distributed environment using XML. SOAP does not itself define any application semantics such as a programming model or implementation specific semantics; rather it defines a simple mechanism for expressing application semantics by providing a modular packaging model and mechanisms for encoding application defined data. This allows SOAP to be used for a large variety of purposes ranging from messaging systems to remote procedure call (RPC) invocations."

A SOAP message is made up of the following:

  • An envelope that defines a framework for describing what is in a message and how to process it
  • A set of rules for how to handle different data types
  • A convention for representing remote procedure calls and responses.

As an open and extensible protocol, using SOAP as the remote access mechanism has a number of benefits over the traditional binary distributed protocols like DCOM and CORBA, including that all applications that access the XML Web service using SOAP can use the same wire protocols. This enables applications to communicate, without the need for custom binaries, run-time libraries, or other platform-specific information that has plagued cross-platform data transfer in the past. SOAP messages are also self-describing, where the header of the message carries all of the information required to process the message at the other end. Lastly, since SOAP messages are plain-text XML documents, they can be transmitted through firewalls via HTTP and even SMTP.

The following is an empty SOAP message that has a root envelope element and its two child elements, header and body:

<Envelope>
<Header>
The SOAP header is an optional element that contains data about what the message contains and how to process it.
</Header>
<Body>
The SOAP body is a required element that contains the XML data needed to access the interfaces of a specific object
</Body>
</Envelope>

The schema for a SOAP message specifies a root Envelope element with child elements named Header (which is optional) and Body (which is required). Because the SOAP message shown in the example follows the specification schema for SOAP, it makes sense to define the namespace "s" by reference to the URI of the W3C Schema for a SOAP envelope at http://schemas.xmlsoap.org/soap/envelope/. However, because namespaces are used only to promote the uniqueness of objects in a document, the namespace does not necessarily have to reference a schema location.

Discovery and Description

Two key concepts of XML Web services are discovery and description. In order for applications to consume XML Web services, they need to be able to locate them and know how to interact with them. A number of XML standard technologies have been proposed that will make automatic discovery and consumption of XML Web services a reality. These include the following:

  • Universal Description, Discovery, and Integration (UDDI)
  • Discovery of Web Services (DISCO)
  • Web Service Description Language (WSDL)

UDDI and DISCO are technologies that essentially "publish" XML Web services so that they can be "discovered" and used by applications. WSDL is an XML format that describes the exposed functionality of an XML Web service.

DISCO is an XML Web service discovery method that is supported today by Microsoft Visual Studio .NET that allows developers to locate and download WSDL documents from a known URL. DISCO files are simple XML documents that contain the location of WSDL description documents for a given XML Web service. Visual Studio .NET is able to parse a DISCO file to determine the location of the specified WSDL files. In the future, DISCO will likely be replaced by emerging Web service standards, like WS-Inspection.

WSDL

WSDL, an XML Web services description standard supported by the W3C, describes how to access an XML Web service. The role that WSDL plays in describing XML Web services is in many ways comparable to what the Interface Definition Language (IDL) does to describe DCOM and CORBA services. A WSDL document describes the following parts of the service:

  • Data type definitions -as defined by the service's schema documents (XSD).
  • Messages formats-defines the required components of the SOAP messages supported for communicating with the service, particularly header elements.
  • Port Types -defines the various input and output operations supported by the service.
  • Bindings -defines transport protocols and detailed message information relating to the operations available on a port.
  • Ports -defines access to the service as a combination of a binding and a network address.
  • Service -defines the name of the XML Web service.

The following sample shows the structure of an empty WSDL document:

<?xml version="1.0" encoding="utf-8"?>
<definitions namespace_definitions>
<types>.</types>
<message name=message_name>.</message>
<portType name=portType_name >
<operation name=operation_name>.</operation>
...
</portType>
<binding name=binding_name >
<operation name=operation_name>
<input name=input_name>.</input>
<output name=output_name>.</output>
...
</operation>
...
</binding>
<service name=service_name >
<port name=port_name>.</port>
...
</service>
</definitions>

A sample from the WSDL file for the resume service sample can be found in Appendix A.

Securing Web Services

As great as all of this discoverability and interoperability sounds, don't forget that XML Web services transmit data, your client's data, through the wild unknown of the Internet. While SOAP messages excel at providing a standardized interface for remotely accessing Web services, they do present an additional vulnerability. Unlike traditional protocols like DCOM and CORBA that communicate via binary protocols, SOAP messages are transmitted as a stream of plain-text XML over HTTP. Because of this, it is even more important to design security into your XML Web service.

When planning for Web service security, it is important to remember two things: 1) web services are exposing your company's valuable data, and 2) they are accessed via the Internet, which is an inherently insecure place. Based on discussions in the developer community, it is clear that security concerns are one of the major roadblocks to the wide-spread adoption of Web services, due in part to Microsoft's track record on security and the complexities of implementing secure multi-tier systems. Even though most security experts feel that Microsoft's .NET platform provides sufficient security features, Web services still need to be designed from the ground up to use those features and to implement security best practices, not just whatever is easiest to bring the service online.

Best Practices

Although Microsoft has proposed new Web services standards that provide much needed security features for XML Web services, aside from using XML encryption SOAP does not define any security for messages. Since this is the case, security for XML Web services must be provided either by the service, by the message transport, or the Web server itself. Since in many ways XML Web services have much in common with today's Web-based applications (i.e. HTTP transport and hosted by Web servers), many Internet security principles will apply to securing services. Securing an XML Web service can be achieved by using some combination of the following techniques:

  • Apply good Web security techniques, like validating all user input. Fortunately, SOAP messages can be strongly typed, which makes it easier to catch malicious data.
  • Operate the services based on the principle of least privilege.
  • Implement custom message-based security. For Web services that do not allow anonymous access, you must require pass authentication and encryption information, like client certificates or Kerberos keys, in the SOAP header that could be required to access your service
  • Use transport-level security techniques like SSL, which work as long as the message does not require any additional routing at which point the security context of an SSL message is lost.
  • Use encryption. SSL can encrypt an entire SOAP message across the wire, and ASP.NET supports encryption for SOAP at the method (root node) level, which will be defined by the forthcoming SOAP encryption standards.

The security infrastructures for an XML Web service will differ depending on the sensitivity of the data that is provided by the service, and as usual, there is a tradeoff between security, code complexity, and performance. For example, any transaction using SSL will be slower than over HTTP.

Authentication and Authorization

Although XML Web services are based on open standards and designed to be discoverable, this does not mean that they should be accessible by unauthorized clients. Why give data away for free because you are unable to control who is accessing your Web service methods? Also, protecting your Web service with adequate authentication and authorization schemes just adds an additional layer of complexity for potential hackers to get through. In fact, a quick way to see rapid decreases in the performance of your servers is to provide a useful and unrestricted XML Web service, like a stock ticker or sports reporting service. It will not be long until some enterprising developer has created a client application that uses your service. When such an application is poorly written, i.e. frequent requests and improper use of caching, or when such an application becomes widely distributed you will find that your initial resource estimates for your service were way too low.

Another possible way to implement authentication for your XML Web service is to use a third-party solution like .NET Passport. The .NET Passport service does the work of user authentication for you. Users can access your service by logging in to .NET Passport and getting an encrypted cookie with their Passport Unique Identifier (PUID). This cookie can then be decrypted and used by your service to access user-specific data. While today .NET Passport only solves the end-user authentication problem, future releases of this service will be based on the Kerberos v5 encryption protocols and Web service security standards that will extend .NET Passport into the realm of inter-service authentication and trust broker. For more information, see http://www.passport.com/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq.

SQL Injection Attacks

Without going into great detail, it is possible under certain conditions to take advantage of SQL syntax to inject potentially damaging statements into database queries that are being executed by an internet application or XML Web service. For example, if a service takes a username variable to create the query SELECT * FROM myTable WHERE username='Joe', maliciously passing the variable as Joe '; DROP TABLE myTable -- can do a lot of damage. These vulnerabilities can be avoided by assuming that you will have malicious clients and practicing good design principles for your database-based services. Techniques for preventing SQL injection attacks include:

  • Never allowing clients to supply custom SQL statements, this is just asking for trouble.
  • Validating incoming data and escape out any dangerous SQL characters like single quotes and double dashes.
  • Avoid building queries by concatenating strings. Stored procedures should be used instead.

For an excellent discussion of SQL injection, see http://www.nextgenss.com/papers/advanced_sql_injection.pdf?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq.

Denial of Service Attacks

One of the biggest vulnerabilities of Web services, even one with good authentication and authorization schemes, is the denial of service (DoS) attack. A DoS attack can be launched by having a few computers send a barrage of multiple-valid SOAP messages that require database lookups. Even if the attacker cannot be authenticated, the SOAP request must still be handled and a response must be generated This type of attack can be mitigated somewhat by enabling session state for your service and using state to prevent clients from making concurrent requests. Although this may prevent unauthorized requests against any of your more cumbersome business logic, a large enough attack may still shut down the service with login request alone. Enabling session state also can have negative performance implications as well.

One way to head-off these malicious requests is to apply custom logic, like ISAPI filters, to intercept malicious packets at the Web server. Another emerging solution to this type of attack is using XML filters to validate and inspect the contents of SOAP requests at the firewall.

Web Service Security Standards

In the future, we can look forward to emerging standards from the Web Services Interoperability Organization (WS-I), including Microsoft, IBM, and others, that will define security for XML Web services. The foundational security specification will be WS-Security, which defines an end-to-end message security protocol that supports a wide variety of security models. Following this specification, which is currently in the beta-release stage, there will be other protocols, which will further clarify how federation works, including WS-Trust, WS-SecureConversation, WS-Authorization and WS-Federation. As we mentioned earlier, standards are being developed jointly by in the WS-I that will support rich authentication and authorization protocols for Web services. For more information on emerging Web services standards, see http://www.ws-i.org/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq.

We will discuss the security considerations of the resume service in the following sections. For an excellent treatise on creating secure applications and services, you will want to read Writing Secure Code by Michael Howard and David LeBlanc.

Overview of the Resume Service

In the following section, we will discuss the general design of our resume service. We will also look at why Visual Studio .NET, C# and SQL Server 2000 make a good combination when designing and implementing XML Web services using Microsoft .NET.

Resume Service Scenario

The general scenario for this simple Web service is one where a company wants to provide a central storage service for the resumes of job seekers. Prospective employers and recruiters could subscribe to this service and then job seekers would have a central repository where they could submit their resume once and have it be accessible by the various service subscribers. Employers and recruiters could then search the database for specific employee attributes (like previous employers, education, skills, etc) via Web service search methods published in the WSDL file and accessible via SOAP.

Resumes of interest to an employer can then be requested from the service based on a unique ID, and the resume data will be returned in a structured XML document, which can be transformed via XSLT into a Web page, print document, report, or whatever best meets the needs of the employer.

We will only be covering a simplified portion of this complex scenario in our sample code. We will be using a simplified database schema and only showing the Query method, which allows clients to search parts of the database for text strings, and the GetResume and GetResumeXML methods, which return a resume as an XML document based on the supplied UserID.

Web services and databases

XML Web services are all about transmitting data in a usable, standard format that relies on SOAP and XML. In today's scalable enterprise application, data is stored in relational databases like Oracle 9i, IBM DB2, Microsoft SQL Server, and others. These applications are designed for the fast, efficient storage and manipulation of large volumes of data. Any commercial web site that you visit is ultimately accessing data stored in a relational database, and since the purpose of XML Web services is to provide data, web service data will also reside in such a database. Although it is possible to conceive of and develop other Web services that can in essence capture data from a data-stream, recordset, or even another web service (like applications that do custom searches of Amazon), even this data initially came from a database.

Although there are many proponents of storing and accessing data using XML documents, relational databases still have a number of advantages over structured document-style storage. Because relational databases allow you to "normalize" or store data in many simplified tables, these "pieces" of data can be readily built, using a SQL statement, into many different kinds of datasets, including XML documents. Also, storing data relationally improves its reusability and scalability, because instead of applications holding locks on an entire large storage document, locks are held on individual tables for a shorter period of time, which allows other applications to access the remaining tables. This is one of the main reasons why relational databases are generally considered to provide faster data access than structured storage mechanisms.

Another major advantage of using a relational database is that most of the enterprise database products use stored procedures. Stored procedures are chunks of SQL code that are maintained by the database and can be executed and cached. Since stored procedures can contain SQL statements to query the database of modify data, they provide an excellent way to restrict access to the database. Like component methods, arguments (known as parameters) can be passed to stored procedures, thus eliminating the risk of allowing client or business tier components to craft SQL statements on the fly. We discuss some of the additional security benefits of databases later in the security section.

Why Use C# and SQL Server for Web services

Even though Microsoft has designed the Common Language Runtime (CLR) - the foundation of the .NET Platform - and Visual Studio .NET to be used with all .NET enabled languages (including new twists on old favorites like PERL.NET and COBOL.NET), C# is being touted by Microsoft as the language of choice for creating Web services in .NET. By being less "user friendly" than Visual Basic .NET (its major .NET rival) C# is believed by many to produce cleaner intermediate language (IL) code that allows for better scalability of services and server-side components. In fact, Microsoft developed both Visual Studio and the .NET Framework almost exclusively using C#.

SQL Server 2000 is a good choice for supplying the database back-end for this type of scenario for the following reasons:

  • It runs on Windows and integrates well with Microsoft's .NET CLR.
  • It offers is a scalable backend database solution that allows you to execute stored procedures.
  • It can return data from a query as an XML document using FOR XML directive.
  • SQLXML 3.0, a middleware component that extends the XML capabilities of SQL Server, enables you to create XML Web services and apply XML templates to relational data.

To keep things more straightforward, we chose not to use SQLXML for the resume service sample, but if we wanted more control over the format of the XML returned by the service, SQLXML would be the way to go.

Note: if you choose to use an earlier version of SQL Server when developing your XML Web service, you will be passing-up on many of the key Web service features, including XML support and the FOR XML clause.

Security for the Resume Service Sample

Although security for Web-based services and applications is a critically important topic, this application is not intended to teach Web services security, as this service runs under anonymous authentication and we do not have an authentication scheme for the service. We have however designed security into the infrastructure by:

  • Accessing SQL Server via a login granted to the ASPNET account. This login is then assigned to the restricted db_datareader role in the ResumeServer database.
  • Granting Execute permissions for the ASPNET login in SQL Server only to the stored procedures used by the application.
  • Escaping out characters in query strings that might be used by a hacker to inject SQL code into the stored procedures.

Architecture of the Resume Web service:

The resume service sample has been designed based on the traditional Microsoft .NET n-tier, scalable architecture. Resume data is stored in relational tables in Microsoft SQL Server 2000. Web service components, coded in C#, search and retrieve data from the database by calling stored procedures in SQL Server via the ADO.NET managed provider for SQL Server. The Query procedure provides search functionality for the service, and the GetResume procedure that is featured in this application, returns resume data as an ADO .NET dataset. This dataset is serialized to a SOAP message by an ASP.NET process running on the IIS server. Since Microsoft's .NET Web services are strongly typed, a .NET aware Web service client is able to receive and parse the message into an instance of the dataset on the client machine. To support non-.NET aware clients, our application has an additional method GetResumeXML that returns the raw XML data as a string instead of as a dataset object. In order to use this raw XML data, the client will have to parse the document and extract the desired data. These methods will be described in detail in the following sections.

In order to call these methods, a client will have to send a properly formatted SOAP request message, which is passed by our Internet Information Service (IIS) ISAPI filter to the Web service components. Visual Studio .NET generates the DISCO and WSDL documents needed for clients to discover and understand the service.

The following diagram represents the architecture for our resume service:

In the .NET world, the process for accessing the resume service would look like this:

  1. The Web services consumer discovers a Web services provider (UDDI or some other mechanism)
  2. Consumer reads a description file for the Web service that tells it what methods, data types, ports, and bindings are available on the service.
  3. Consumer sends a request for a Web service method to the service via SOAP
  4. ISAPI running on the provider's Web server (ASP.NET) receives the SOAP request, parses it, and invokes the desired method on the middle-tier (ideally this is coded in C#)
  5. The Web service method calls down to the database using the ADO.NET data provider which requests that SQL Server run the specified stored procedure.
  6. SQL Server runs the procedure that returns the requested data via ADO.NET.
  7. The middle-tier application takes the data, processes it (if necessary), and passes it to ASP.NET where it is serialized into a SOAP message and sent back to the requesting consumer.

Sample SOAP Messages

As we mentioned earlier, SOAP messages are used to send requests to and receive responses from an XML Web service. The following XML documents show examples of these SOAP messages. In the case of our sample, we are not using any of the properties of the SOAP header, so these messages only include <body> elements.

Query Request Message

<?xml version="1.0" encoding="utf-8"?>
<soap:Envelope xmlns:xsi=http://www.w3.org/2001/XMLSchema-instance?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq
xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
<soap:Body>
<Query xmlns="http://www.webbish6.com">
<queryString>Cincinnati</queryString>
<queryType>2</queryType>
</Query>
</soap:Body>
</soap:Envelope>

Query Response Message

<?xml version="1.0" encoding="utf-8"?>
<soap:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
<soap:Body>
<QueryResponse xmlns="http://www.webbish6.com">
<QueryResult>
<int>1</int>
<int>2</int>
</QueryResult>
</QueryResponse>
</soap:Body>
</soap:Envelope>

Coding the Sample Application

The following section describes the process for developing the resume service, starting with defining the database schema.

Database Schema

In order to simplify the database design, the database for this sample (named ResumeServer) has been designed to allow each user to have only one resume stored in the service. Each user has related employer and education information. It contains the following user tables:

  • User - this table includes columns for UserID (the primary key that is used by the stored procedures to select a single user resume), first name, last name, and email address. All other tables relate back to this table, so that the dataset will only contain data related to a given user. This table might also be expanded to add columns for other user-based resume information like awards, skills, interests, and the like.
  • Position - this table stores information about positions that a user has had, and includes columns for position start and end date, title, description, and name and phone number of references. Each user can have multiple entries in the Position table, but a position must relate to only one employer. Position has a foreign key relationship to User on UserID and to Employer on EmployerID.
  • Employer - this table stores information about employers, including name, type of business, address and if the employer is a client of the service. The role of this table could be greatly expanded in a real world scenario to store additional information for companies that are clients of the service. Since a single employer will have many related positions and users, Position relates to Employer by a foreign key on EmployerID.
  • Education - this table stores information about a user's education, including type of degree, major, date completed and any comments. Education relates to User by a foreign key on UserID.
  • Major - this is a lookup table that maps the integer value in Education.MajorID to a nvarchar(50) text description with the name of the major.
  • Degree - this is a lookup table that maps the integer value in Education.DegreeType to a nvarchar(50) text description with the name of the type of degree.
  • Institution - this table stores information about where the user obtained the related education. It includes columns for name, city, state and type of institution.

The following diagram shows the table relationships for the ResumeServer database:

Although the database schema for this sample application is fairly simple, even more richness could be added to support enhanced functionalities. For example, if you wanted users to be able to store more than one resume in the service, then you could add a Resume table, relating to User, that would function as the User table in this sample. In this case, you would query based on a resume ID value rather than a user ID.

Although authentication has not been included in this sample, any authentication method (whether a custom scheme or something out-of-the-box like Microsoft Passport .NET) will require changes to the User table to store authentication information ranging from username and password data to PUIDs in the case of .NET Passport.

Stored Procedures

Since data in this database has been fairly well normalized, the aggregated data that relates to a given user is retrieved via a series of SELECT SQL statements. The complete resume is returned by executing the following three stored procedures:

  • spr_GetUserInfo
  • spr_GetEmployerInfo
  • spr_GetEducationInfo

Each of these procedures returns a related section of the resume data. As we will see, all three of the result sets from these procedures are added to the dataset before it is returned by the service. In addition, the service can support any number of string-based query procedures via the Query Web method, with one for each custom query procedure written for the service. This sample uses the spr_QueryEmployer and spr_QueryInstitution stored procedures to query on specific columns of the Employer or Institution table respectively for text that matches the supplied string.

Now, let's take a closer look at these SQL Server stored procedures.

spr_GetUserInfo Procedure

The spr_etUserInfo procedure returns information from the User table. The following SQL code creates this stored procedure:

CREATE PROC spr_GetUserInfo
@UserID int
AS
IF EXISTS(SELECT * FROM [User] WHERE UserID=@UserID)
BEGIN
SELECT [User].FirstName, [User].LastName
FROM [User]
WHERE [User].UserID=@UserID
END
ELSE
SELECT 'Invalid User ID' AS [Error] WHERE 1=1
GO

If no data exists for the passed @UserID value, then a null dataset will be returned.

spr_GetEmployerInfo Procedure

The spr_GetEmployerInfo procedure returns information on former positions and employers for a given user. The following SQL code creates this stored procedure:

CREATE PROC spr_GetEmployerInfo
@UserID int
AS
IF EXISTS(SELECT * FROM [User] WHERE UserID=@UserID)
BEGIN
SELECT Employer.EmployerName, Employer.City, Employer.State,
Position.Title, Position.[Description], Position.StartDate,
Position.EndDate FROM [User]
INNER JOIN Position ON [User].UserID=Position.UserID
INNER JOIN Employer ON Employer.EmployerID=Position.EmployerID
WHERE [User].UserID=@UserID
ORDER BY Position.EndDate
END
GO

As in spr_GetUserInfo, we don't return schema information if an empty row is returned.

spr_GetEducationInfo Procedure

The spr_GetEducationInfo procedure returns information on education degrees received and institutions attended by a given user. The following SQL code creates this stored procedure:

CREATE PROC spr_GetEducationInfo
@UserID int
AS
IF EXISTS(SELECT * FROM [User] WHERE UserID=@UserID)
BEGIN
SELECT Institution.InstitutionName, Institution.City, Institution.State,
Degree.[Description] AS 'Degree', Major.[Description] AS 'Major',
Education.DateCompleted FROM [User]
INNER JOIN Education ON [User].UserID=Education.UserID
INNER JOIN Major ON Education.MajorID=Major.MajorID
INNER JOIN Degree ON Education.DegreeType=Degree.DegreeType
INNER JOIN Institution ON
Institution.InstitutionID=Education.InstitutionID
WHERE [User].UserID=@UserID
GROUP BY Institution.InstitutionName, Education.DateCompleted
ORDER BY Institution.InstitutionName, Education.DateCompleted DESC
END
GO

Again, when an empty row is retuned, this procedure simply returns a null dataset.

spr_QueryEmployer

The spr_QueryEmployer stored procedure is used to search the database for users where employer name matches the search text string. This procedure is called by the service's Query method by passing a string to the @searchtext parameter. The following code implements this procedure on the server:

CREATE PROC QueryEmployer
@searchtext nvarchar(50)
AS
SELECT DISTINCT [User].UserID FROM [User]
INNER JOIN Position ON [User].UserID=Position.UserID
INNER JOIN Employer ON Employer.EmployerID=Position.EmployerID
WHERE Employer.EmployerName LIKE '%' + @searchtext + '%'
ORDER BY [User].UserID
GO

By using the LIKE keyword and wildcards, this procedure will find any records containing the search text anywhere in EmployerName.

spr_QueryInstitution

Another query procedure included in this sample is spr_QueryInstitution, which returns user records where the educational institution name matches the search string. The following code implements spr_QueryInstitution :

CREATE PROC QueryInstitution
@searchtext nvarchar(50)
AS
SELECT DISTINCT [User].UserID FROM [User]
INNER JOIN Education ON [User].UserID=Education.UserID
INNER JOIN Institution ON Institution.InstitutionID=Education.InstitutionID
WHERE Institution.InstitutionName Like '%' + @searchtext + '%'
ORDER BY [User].UserID
GO

Again, using the LIKE keyword and wildcards allows us to find any records containing the search text anywhere in InstitutionName.

Additional Stored Procedures

Although not part of this sample, a complete resume service would require sets of stored procedures, analogous to the previous procedures, that would be used to insert and update data in the service. Additional stored procedures could also be created to provide an improved ability to query the service.

Web Service Components in C#

The business-tier component of the resume service is implemented by the C# ResumeService class. The public methods of this class are exposed via ASP.NET as XML Web service methods. These methods are called by clients via SOAP message requests in XML that are converted by an ASP.NET process on the Web server and passed to the appropriate method on the ResumeService object. These methods then, in turn, access the data in SQL Server via the SqlDataAdapter class to execute the appropriate stored procedure. Data is returned to an ADO.NET DataSet object by invoking the Fill method on the data adapter. Our sample application provides the following methods on the ResumeService class:

  • Query
  • GetData (private)
  • GetResume
  • GetResumeXML

In order to expose these public methods as Web service methods, in Visual Studio the appropriate Web service attribute is applied to both the ResumeService class and each exposed method. For example, the following code shows the WebService attribute applied to the ResumeService class and the WebMethod attribute applied to the GetResume method:

[WebService(Namespace="http://www.webbish6.com")]
public class ResumeService : System.Web.Services.WebService
{
...
[WebMethod(Description = "Returns a strongly-typed .NET DataSet
containing resume data", EnableSession = false)]
public DataSet GetResume(int userID)
{
...
}
}

These attributes also allow you to specify other properties of the service, like namespace and method descriptions. Although not included in this sample, other WebMethod attributes allow you specify more advanced properties of the service, like providing unique names for overloaded methods, enabling session state, and participating in transactions. When your C# Web service is compiled to Intermediate Language (IL) code assemblies, these attributes are converted to metadata and stored with the managed code assemblies. At run-time, ASP.NET interprets this metadata using functionality provided by the .NET Frameworks through the System.Reflection namespace when determining how to expose Web service methods and generate the WSDL for the service.

Query Method

The Query method is a public method that provides the ability to search the database using a specified query stored procedure for the specified string. This method returns an integer array of UserID values that match the selected query text. Query takes the following two arguments:

queryString as string - passes the text that will be used in the WHERE clause of the given query stored procedure. To prevent possible SQL string injections, single-quotes and double dashes are escaped by the following code:

queryString = queryString.Replace("'","''");
queryString = queryString.Replace("--","");

queryType as int - enumerates the type of query that is being executed. In this sample, the two queries are QueryEmployer and QueryInstitution, which map to values 1 and 2 respectively. In order to expand the search functionality of the service, you can easily just add additional query stored procedures in the database, then expand the following switch statement with a new case for each procedure name as appropriate:

switch(queryType)
{
case 1:
queryName = "spr_QueryEmployer";
break;
case 2:
queryName= "spr_QueryInstitution";
break;
default:
throw(new ArgumentOutOfRangeException("queryType",
"The value for queryType is invalid"));
}

After escaping the text in queryString and setting the correct value for queryName, this method uses the SqlDataAdapter object to execute the stored procedure with the specified query text in the following code:

// Prepare the data adapter to execute the stored procedure
SqlDataAdapter myDataAdapter = new SqlDataAdapter(queryName, myConn);
myDataAdapter.SelectCommand.CommandType = CommandType.StoredProcedure;
myDataAdapter.SelectCommand.Parameters.Add("@searchtext", SqlDbType.NVarChar, 50).Value=queryString;
try
{
// Execute the command on the data adapter and return the results
// to the open DataSet
myDataAdapter.Fill(myDataSet);
}
catch (SqlException e)
{
Debug.Write("Unable to execute command and fill dataset. \n" +
e.ToString() + "\n");
}
finally
{
myConn.Close();
}

The data is returned to a DataSet object by calling the Fill method on the SqlDataAdapter. Once the data has been returned to the DataSet object, the following code writes the UserID values to an integer array and returns the array:

// Select the first (and only) returned table (User)
DataTable myDataTable = myDataSet.Tables[0];

// Select the first (and only) returned column (UserID)
DataColumn myDataCol = myDataTable.Columns[0];

// Define an integer array to hold the returned values
int[] userID = new int[myDataTable.Rows.Count];

// For each row, load the UserID value to the integer array.
for(int i=0; i < myDataTable.Rows.Count; i++)
{
DataRow myDataRow = myDataTable.Rows[i];
userID[i] = (Int32)myDataRow[myDataCol];
}

return userID;

After the array of UserID integers has been returned from one of the stored procedures exposed by the Query method, these integer values are passed one-at-a-time either to the GetResume method or the GetResumeXML method in order to retrieve the resume data that corresponds to each UserID.

GetData Method

The GetData method is a private method that is called by both GetResume and GetResumeXML. This method takes a userID value, makes the connection to SQL Server, executes the stored procedures, and returns the resume data as an ADO.NET DataSet object. The following code implements this method:

private DataSet GetData(int userID)
{
// DataSet automatically opens and closes our connection (myConn)
DataSet myDataSet = new DataSet("Resume");

// Create an array for our three procedures
string[] myProcs = {"GetUserInfo", "GetEmployerInfo", "GetEducationInfo"};

// Loop through each procedure
for(int i=0;i<myProcs.Length;i++)
{
// Create a data adapter supplying the procedure name,
// parameter and connection
SqlDataAdapter myDataAdapter =
new SqlDataAdapter(myProcs[i], myConn);
myDataAdapter.SelectCommand.CommandType =
CommandType.StoredProcedure;
myDataAdapter.SelectCommand.Parameters.Add("@UserID",
SqlDbType.Int, 4).Value=userID;

try
{
// Fill this DataAdapter into the open DataSet
myDataAdapter.Fill(myDataSet);
}
catch (SqlException e)
{
Debug.Write("Unable to execute command and fill dataset. \n" +
e.ToString() + "\n");
}
finally
{
myConn.Close();
}
}
return myDataSet;
}

GetResume Method

The GetResume method is the primary method used to retrieve a user's resume data from the service. This method should be called by .NET-aware clients as it returns a strongly-typed ADO .NET DataSet object that is sent to the client via SOAP as XML and is reconstructed by a .NET-aware client. Non-.NET clients should use the GetResumeXML method because it returns a structured XML document rather than just the XML representation of the DataSet object.

This method takes an integer representing a valid UserID value in the database and returns an ADO.NET DataSet object. The following code is used to implement this method:

[WebMethod(Description = "Returns a
            strongly-typed .NET DataSet
containing resume data",
            EnableSession = false)]
public DataSet GetResume(int
            userID)
{
DataSet myDataSet = new
            DataSet();
try
{
myDataSet = GetData(userID);
}
catch
            (Exception e)
{
Debug.Write("GetData call failed with the
following exception: " + e.ToString() +
            "\n");
}

return myDataSet;
}

The dataset is retrieved by calling the private GetData function and passing the userID value. The returned DataSet object is then sent to the client as a SOAP response message. We will see an example SOAP response method later on.

GetResumeXML

The GetResumeXML method is called by clients who are not able to handle a .NET DataSet object. Instead, this method returns an XML fragment representing the requested user's resume as a string within a SOAP response. The following code implements the GetResumeXML method:

[WebMethod( Description = "Returns resume data as a string for .NET
unaware clients", EnableSession = false)]
public string GetResumeXML(int userID)
{
string myResumeXml = "";

try
{
// Pass the userID to the GetData method that returns a dataset
DataSet myDataSet = GetData(userID);

// Write the contents of the dataset to an XML string
myResumeXml = myDataSet.GetXml();
}

catch(Exception e)
{
Debug.Write("Unable to get data. Method returned with exception " +
e.ToString() + "\n");
}

return myResumeXml;
}

This method then retrieves a DataSet object by passing the userID value to the private GetData method. The dataset is then written to an XML string by calling the ext'>GetXML ='codeintext'> method on the DataSet object.

Exceptions

In this sample, exception errors are handled by using the appropriate try, catch, finally C# syntax. Method calls or other operations that have the potential to generate exception errors are executed inside a try block. Each try block is then followed by a catch block, which is used to write the exception to the debugger and possibly run some other custom code, and optionally a finally block, where we close any open connections to the database. When an exception is thrown by an XML web service method, the SOAP response message returned by the service will contain a SOAP <Fault> element. This <Fault> element contains information relating the exception. .NET clients can de-serialize this SOAP Fault as a SoapException.

Testing

Visual Studio .NET makes testing of .NET Web services almost a trivial matter. You do not have to write a separate XML Web service consuming client application because Visual Studio .NET generates one for you automatically when you start the service from within the IDE. When a .NET XML Web service is started for debugging in Visual Studio .NET, the IDE first generates an ASP.NET start page for the service that contains links to the service description WSDL file, as well as each method exposed by the service. For our sample, this start page looks like the following:

In this screen, the descriptions provided for each method come from the Description property of the WebMethod attribute. Clicking on a method name loads another ASP.NET page that contains form fields that correspond to the arguments of the method. A sample of the SOAP request and response messages, as well as the HTTP GET and POST calls for this method, are also displayed. The following screen-shot shows what this page looks like for the Query method:

Entering valid values in the fields and clicking the Invoke button on this page submits a test request to the service running in the debugger via an HTTP POST. If the request was successful, a new browser window is launched which displays an XML document fragment that contains the response data. The following screen shows the XML fragment representing the integer array data returned by HTTP POST to the Query method:

When a handled exception occurs, this screen displays information about the exception. If the exception is unhandled, this screen may return an HTTP error code of 500, indicating an internal server error.

In order to test the sample code available with this case study supply, values in the following table will return data from the service:

Method | Input (type) | Valid Input
Query (employer queryType = 1) | QueryString (string) | "Microsoft"
Query (institution queryType = 2) | QueryString (string) | "Cincinnati"
GetResume | UserID (int) | 1 or 2
GetResumeXML | UserID (int) | 1 or 2

Deployment

Once our XML Web service code has been completed, tested and debugged, there is the final deployment hurdle to clear. For the most part, an XML Web service built based on .NET is deployed like any other ASP.NET internet application, but with a few additions. A project can be deployed either by copying files to the proper location in the V-Root of the production Web server, or by creating a new deployment project in Visual Studio .NET. If you are planning to deploy your service in a large enterprise environment, you might want to look into using Microsoft's Application Center Server to simplify the daunting task of deploying and managing Web applications on large groups of Web servers.

When deploying your service, you will want to make sure that the windows accounts that are being used to access SQL Server are still valid. For example, if you developed your service using a local machine account, you may need to change it to a domain-wide account. In many cases, when connecting to SQL Server on a separate machine is will also be necessary to use ASP.NET impersonation to provide the correct login to SQL Server.

The final step of deployment involves publishing the dynamic discovery document (DISCO). This document, generated by Visual Studio .NET, tells Web service clients how to get to your service. The DISCO document is published by default in the root directory of the service, so if you do not want everyone to be able to discover your service, you will want to rename, move or delete the .vsdisco file. Based on the information in this file, clients can then request from your service the WSDL file that they need to understand how to access the services.

Scalability

For a production Web service application, like any other internet-based enterprise application, scalability is a major concern. Although fine-tuning the performance of an XML Web service is outside of the scope of this case study, the following techniques should be investigated as potential ways to improve scalability:

  • Use best practices for accessing the database, including closing (or allowing the provider to close) any unused connections that you explicitly open. When using ADO.NET, connections opened by the provider are automatically garbage collected back to the connection pool.
  • Avoid having ASP.NET manage session state. Instead, track this information in the application and include state information in SOAP headers.
  • Use clusters servers and run the Web server and database on separate machines.
  • Use SQL Server Replication to publish the database to multiple production database servers from a single source database.

Conclusion

Microsoft's .NET platform is an ideal environment for developing, hosting, and consuming XML Web services, and C# is the language of choice for developing Web service applications. C# components can efficiently retrieve data stored relationally in SQL Server via the ADO.NET provider that executes stored procedures on the database. These component methods can then be easily exposed as Web service methods. You don't even have to write one line of XML code to deploy and publish discovery documentation documents for your service because Visual Studio generates these documents and ASP.NET does the hard work of serializing your C# methods into SOAP messages.

Appendix: A

WSDL File Representation of the Resume Service

The following is a portion of the WSDL document generated by Visual Studio .NET for our sample service. For brevity, we have removed all of the XML description that is not related to the Query method. The complete WSDL file can be found in the source files for this project as ResumeService.wsdl.

<?xml version="1.0" encoding="utf-8" ?>
<definitions xmlns:http="http://schemas.xmlsoap.org/wsdl/http/"
xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/"
xmlns:s="http://www.w3.org/2001/XMLSchema"
xmlns:s0="http://www.webbish6.com"
xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/"
xmlns:tm="http://microsoft.com/wsdl/mime/textMatching/"
xmlns:mime="http://schemas.xmlsoap.org/wsdl/mime/"
targetNamespace="http://www.webbish6.com"
xmlns="http://schemas.xmlsoap.org/wsdl/">
<types>
<s:schema elementFormDefault="qualified"
targetNamespace="http://www.webbish6.com">
<s:import namespace="http://www.w3.org/2001/XMLSchema" />
<s:element name="Query">
<s:complexType>
<s:sequence>
<s:element minOccurs="0" maxOccurs="1"
name="queryString" type="s:string" />
<s:element minOccurs="1" maxOccurs="1"
name="queryType" type="s:int" />
</s:sequence>
</s:complexType>
</s:element>
<s:element name="QueryResponse">
<s:complexType>
<s:sequence>
<s:element minOccurs="0" maxOccurs="1"
name="QueryResult" type="s0:ArrayOfInt" />
</s:sequence>
</s:complexType>
</s:element>
<s:complexType name="ArrayOfInt">
<s:sequence>
<s:element minOccurs="0" maxOccurs="unbounded"
name="int" type="s:int" />
</s:sequence>
</s:complexType>
...
</s:schema>
</types>
<message name="QuerySoapIn">
<part name="parameters" element="s0:Query" />
</message>
<message name="QuerySoapOut">
<part name="parameters" element="s0:QueryResponse" />
</message>
...
<message name="QueryHttpGetIn">
<part name="queryString" type="s:string" />
<part name="queryType" type="s:string" />
</message>
...
<message name="QueryHttpPostIn">
<part name="queryString" type="s:string" />
<part name="queryType" type="s:string" />
</message>
<message name="QueryHttpPostOut">
<part name="Body" element="s0:ArrayOfInt" />
</message>
...
<portType name="ResumeServiceSoap">
<operation name="Query">
<documentation>
Returns an array of User IDs that match a query
</documentation>
<input message="s0:QuerySoapIn" />
<output message="s0:QuerySoapOut" />
</operation>
...
</portType>
<portType name="ResumeServiceHttpGet">
<operation name="Query">
<documentation>
Returns an array of User IDs that match a query
</documentation>
<input message="s0:QueryHttpGetIn" />
<output message="s0:QueryHttpGetOut" />
</operation>
...
</portType>
<portType name="ResumeServiceHttpPost">
<operation name="Query">
<documentation>
Returns an array of User IDs that match a query
</documentation>
<input message="s0:QueryHttpPostIn" />
<output message="s0:QueryHttpPostOut" />
</operation>
...
</portType>
<binding name="ResumeServiceSoap" type="s0:ResumeServiceSoap">
<soap:binding transport="http://schemas.xmlsoap.org/soap/http"
style="document" />
<operation name="Query">
<soap:operation soapAction="http://www.webbish6.com/Query"
style="document" />
<input>
<soap:body use="literal" />
</input>
<output>
<soap:body use="literal" />
</output>
</operation>
...
</binding>
<binding name="ResumeServiceHttpGet" type="s0:ResumeServiceHttpGet">
<http:binding verb="GET" />
<operation name="Query">
<http:operation location="/Query" />
<input>
<http:urlEncoded />
</input>
<output>
<mime:mimeXml part="Body" />
</output>
</operation>
...
<binding name="ResumeServiceHttpPost" type="s0:ResumeServiceHttpPost">
<http:binding verb="POST" />
<operation name="Query">
<http:operation location="/Query" />
<input>
<mime:content type="application/x-www-form-urlencoded" />
</input>
<output>
<mime:mimeXml part="Body" />
</output>
</operation>
...
</binding>
<service name="ResumeService">
<port name="ResumeServiceSoap" binding="s0:ResumeServiceSoap">
<soap:address location=
"http://localhost/ResumeService/ResumeService.asmx" />
</port>
<port name="ResumeServiceHttpGet" binding="s0:ResumeServiceHttpGet">
<http:address location=
"http://localhost/ResumeService/ResumeService.asmx" />
</port>
<port name="ResumeServiceHttpPost" binding="s0:ResumeServiceHttpPost">
<http:address location=
"http://localhost/ResumeService/ResumeService.asmx" />
</port>
</service>
</definitions>

Case Study Information
Author | Jeannine & Glenn Gailey
Editor | Adam Ryland
Reviewers | Phil Sidari, Richard Conway, Matthew MacDonald

If you have any questions or comments about this case study, or suggestions for further pieces, please contact the technical editor. If you wish to talk about the case study with other subscribers, please visit our forum here.

Fast Track Books from Wrox Press
If you want a C# book that is compact, fast-paced but still provides the quality of information that you have come to expect from a Wrox book - you want Fast Track C#, newly available from Wrox Press.

This book will help you to quickly learn about the following:
  • Understand what C# is, and how it fits into the .NET Framework
  • A concise introduction to all the main features of the C# language, with Java and C++ comparisons noted
  • Using the .NET base classes
  • Writing and deploying Windows applications with C# and Visual Studio .NET
  • Creating and configuring .NET assemblies
  • Accessing databases with ADO.NET, and using XML in C#
  • Integrating your existing COM components with the .NET Framework
  • COM+ Services in the .NET Framework
  • Building web applications with ASP.NET pages
  • Creating web services in C#

C#Today - ?2002 Wrox Press Limited


.NET Code Generation with Visual Studio .NET Enterprise Architect

.NET Code Generation with Visual Studio .NET Enterprise Architect
http://www.csharptoday.com/content.asp?id=1849
Article Rating: 4.5
Published on July 8, 2002
Abstract
In this case study, Tony Loton looks at the Visual Studio .NET / Visio for Enterprise Architects integration features that allow you to generate code for any .NET language from a UML model in Visio. For .NET designer-developers running with the Enterprise version of Visual Studio, this out-of-the-box solution could be a more rational choice than shelling out on an additional CASE tool.
Case Study

In this case study we'll look at the Visual Studio .NET / Visio for Enterprise Architects integration features that allow you to generate code for any .NET language from a UML model in Visio. For .NET designer-developers running with the Enterprise version of Visual Studio, this out-of-the-box solution could be a more rational choice than shelling out on an additional CASE tool.

Introduction

Visual Modeling techniques, in particular the Unified Modeling Language (UML), have a good track record for improving the quality of software systems, and UML has become an industry standard notation for analysis and design.

To find out more about the UML, check out the Object Management Group website at http://www.omg.org or the Rational website at http://www.rational.com.

In recent years the UML tools market has been dominated by Rational and a few other big players, with vendors of inexpensive - or free - tools cashing in on the reluctance of many developers to part with their hard earned money.

Now there's a new kid on the block, in the form of the Visio for Enterprise Architects that comes bundled with Visual Studio .NET Enterprise version. Visio has been the poor relation in terms of visual modeling with UML, because historically it has been more of a diagramming tool than a fully-fledged CASE tool, but when it comes to IDE integration - and in particular .NET code generation - it has a lot to offer.

It is true that the Rational XDE product provides a fully integrated solution for UML modeling and Visual Studio .NET development, but at a price. If you've already spent your last dollar on the Visual Studio .NET Enterprise version, the bundled solution could well be the more rational option.

The problem is that it's not too well documented, nor publicized, so you're probably not aware of what can be achieved. I'll use this case study to put that right, but to start with I should say something about why you would want to bother with code generation in the first place.

Why Generate Code?

You've worked hard designing your application using UML and now you're ready to turn that design into code. You did design the application before writing any code, didn't you?

As you've specified all of the packages, classes, attributes and operation signatures in the model - maybe thousands of them - you certainly don't want to retype all of that information again in the form of C# code. No, what you want is to get a head start on the coding by leveraging the information contained within the model. That's where code generation plays its part, and for any end-to-end project it's an invaluable feature.

The code generator will take the strain of setting up the namespace structure, creating the class definitions, and establishing the associations between related classes via member variables, and that's not to mention taking all of the documentation entries from the model and inserting them as comments in the code.so that you know what it's meant to do.

However, it won't write complete programs for you. Most of the code generators that I've encountered, including this one, take information only from the static UML model. that is, the class diagrams. No information is taken from the dynamic model (sequence and state diagrams) that describe the runtime behavior of the application.

System Requirements

For this case study you'll need the Visual Studio .NET Enterprise version that comes bundled with Visio for Enterprise Architects. You might have purchased that intentionally, or you might be unaware that it comes with your MSDN Universal Subscription.

Defining The Problem

In a nutshell the problem is that of taking a set of classes designed in UML.

and turning them into template source code like this (extract only shown).

// Static Model

namespace BankingDomain.Account
{

public abstract class AccountType
{

private decimal balance;

public virtual decimal Balance
{
get
{
return 0;
}
}
public virtual decimal AvailableBalance
{
get
{
return 0;
}
}
}// END CLASS DEFINITION AccountType

} // BankingDomain.Account

// Static Model

namespace BankingDomain.Account
{

public class CurrentAccount : AccountType
{

private decimal unclearedFunds;

public new decimal AvailableBalance
{
get
{
return 0;
}
}
}// END CLASS DEFINITION CurrentAccount

} // BankingDomain.Account

We'll look at how to solve that problem using the tools I've mentioned, and we'll also look at the deeper problem of understanding the UML-to-C# code mapping. If you're not using C#, but you came to the C# Today site anyway, rest assured that much of what I'll say is applicable to the other .NET languages too. For the moment that means C#, Visual Basic .NET, and C++, but I imagine J# will be supported before too long.

Case Study Structure

In the Defining the Solution section of this case study we'll look at how to launch the code generator from within Visio for Enterprise Architects and how to direct the output to a Visual Studio .NET solution or to a set of source files. We'll also look at some of the mappings from UML notation to C# language constructs.

In the Sample Model Code Generation section I'll offer a Visio UML model from which you can generate some code. Don't expect too much here, but it will serve to demonstrate some additional code generation features.

At the end, I'll summarize what we've covered in the Case Study Review section.

Defining The Solution

From the outset I've defined the solution as comprising the Visio for Enterprise Architects / Visual Studio .NET combination. So rather than defining the solution in this section I'll take some time to describe it.

Introducing Visio for Enterprise Architects

You might have missed the fact that your MSDN universal subscription (if you have one) contains a version of Visio - called Visio for Enterprise Architects - that is integrated with the Visual Studio .NET, and which supports UML modeling for .NET applications. Even if you bought the Visual Studio .NET (Enterprise Version) specifically, not as part of an MSDN subscription, you might still have missed the significance of the Enterprise Architect version of Visio that is included.

In recent years, designers and developers have been able to choose between the two most popular tools, Rational Rose and Together Control Center (at http://www.togethersoft.com/products/controlcenter/index.jsp?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq), a few older ones like Select Enterprise, or any one from a number of free tools that are not widely used. Visio had been the poor relation in terms of visual modeling with UML because historically it has been more of a diagramming tool than a fully-fledged CASE tool; lacking in comprehension of the model itself and with little or no support for semantic checking, reporting, code generation, reverse engineering and information exchange. That's still true to a certain extent, but in addition to the basic diagramming it now supports:

  • UML modeling from Use Cases, through Dynamic and Static Modeling, to Deployment.
  • Semantic Checking of the model.
  • Code Generation and Reverse Engineering via Visual Studio .NET integration.
  • Reporting, and Model Interchange via XML Metadata Interchange (XMI, described at http://www.omg.org/technology/documents/formal/xmi.htm?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq).

It's now much more credible as a modeling tool, especially for .NET developers wishing to take advantage of the Visual Studio integration features.

I'll not offer a tutorial on drawing UML diagrams in Visio because you'll be able to progress using the ready-made UML model that I've provided. If this case study turns you on to the potential, you can then take the time to learn how to build your own Visio UML models. The Drawing Types / Software / UML model diagrams section of the Microsoft Visio Help is a good starting point.

I'll concentrate on the code generation aspect and therefore tell you just what you need to know to make that work.

How to Generate Code

Generating .NET language code from a UML model is straightforward. With your model loaded into Visio just choose the UML, Code, Generate menu option as shown here.

The Generate dialog is displayed, and this allows you to specify the target language, the location for the generated files, and the classes / packages for which you want code to be generated.

If you leave the Add Classes to Visual Studio Project box unchecked, the result will be a directory hierarchy that reflects the UML package hierarchy and which contains the generated code in a series of .cs C# source files. Note that in this case, the directory hierarchy is always contained within a top level directory called TopPackage.

If you check the Add Classes to Visual Studio Project box you can then specify a name for the target Solution and Project, thereby overriding the default TopPackage. For the sake of variety I'll be specifying the target language as Visual Basic this time.

Once the code has been generated, the Visual Studio .NET application is not launched automatically as you may have expected; so, you need to find the Solution file and double-click to open it. You will be rewarded with something like this:

I told you it was equally applicable to other .NET languages, and that's all there is to it. The code generation process looks quite straightforward, and it is, but to get the most out of it you'll need to understand how the UML constructs map onto the target .NET language.

UML to C# Code Mapping

You've seen above that it's just as easy to generate Visual Basic code - or for that matter C++ code - as it is to generate C# code. As a C# Today reader, I'll assume that it's the UML to C# mapping that you're really interested in, so that's what we'll look at here.

Defining the complete mapping is beyond my scope for this case study, but we'll look at a few of the more interesting mappings. First I'll show you how to discover the mappings for yourself using the Code Preview facility.

Code Preview

You can preview the code that will be generated for any class in the Visio Model Explorer. As an example, suppose we have a very simple model containing a single package (MyPackage) with a single class (MyClass).

We can double-click that class in the Model Explorer or on a diagram (or right-click the class and choose Properties), then select the Code Generation Options category like this:

Then we can press the Preview Code button to see the code mapping for MyClass and MyPackage. Note that by selecting from the Target language pick list you can view the code as C#, Visual Basic or C++. For C#, the preview shows:

That's the quickest way to get a feel for what the code will look like, and for a complete picture you can examine that actual source files that are generated.

C# Mapping for Packages and Classes

It's not possible to generate code from a model containing only packages, so we'll look at the package mapping indirectly by examining MyClass. As shown in the code preview, the generated C# code for the simple case will be as follows. Note that the package has been mapped to a namespace:

// Preview of code that will be generated in the implementation file:

namespace MyPackage
{

public class MyClass
{

}// END CLASS DEFINITION MyClass

} // MyPackage

Packages may contain other packages, ad infinitum, in a model like this one:

In which case the generated code will be:

// Preview of code that will be generated in the implementation file:

namespace MyPackage.MyPackage2
{

public class MyClass2
{

}// END CLASS DEFINITION MyClass2

} // MyPackage.MyPackage2

Notice how the package hierarchy has been mapped to a compound namespace having a dot-separated path MyPackage.MyPackage2, rather than by embedding one namespace definition inside the other like this:

namespace MyPackage
{
namespace
            MyPackage2
{

public class
            MyClass2
{

}
}

}

Upon generating code from a model containing packages (by choosing the UML, Code, Generate menu option) you will see that the package hierarchy is reflected in the file system directory hierarchy. Notice how the default TopPackage directory has a subdirectory MyPackage that contains a source file for MyClass and a subdirectory MyPackage2:

Each class definition is written out individually to its own source file. Two classes in the model results in two source files with .cs extensions (assuming C# language), placed into a directory structure reflecting the namespace hierarchy.

Abstract Class

A typical class-level feature is the abstract modifier. For any class defined as abstract (via the UML Class Properties Dialog) the keyword abstract will be inserted automatically into C# code like this:

// Preview of code that will be generated in the implementation file:

namespace MyPackage
{

public abstract class MyClass
{

}// END CLASS DEFINITION MyClass

} // MyPackage

Utility Class

Suppose you want a class to represent a shared (global) data store, in other words a class with only static (class scope) members. You can do this by applying the <<utility>> stereotype to an existing class or by dragging the pre-defined Utility shape onto a diagram.

The Visio UML online Help Documentation states that a utility class:

"is mapped in the same way as a regular class, with one exception: by default, all methods in a utility class are class scope operations. This means that for utility classes, classifier is the default setting for the OwnerScope option in the UML Operation Properties dialog box, and in the generated code, the keyword static is placed in front of the type expression."

To summarise, that means that in the UML Class Properties dialog you would expect all operations to be marked as classifier scope, and you would expect the code for all operations to include the static modifier.

Now suppose I use the UML Class Properties dialog to manually change one operation to instance scope, as shown here:

The following listing shows that in code generation the two operations would both be prepended with the static modifier, despite the instance scope of the second operation:

// Static Model

namespace MyPackage
{

public class MyUtilityClass
{

public static bool myFirstUtilityOperation()
{

}

public static bool mySecondUtilityOperation()
{

}

}// END CLASS DEFINITION MyUtilityClass

} // MyPackage

So it seems that the <<utility>> stereotype always takes precedence over the contents of the UML Class Properties dialog. For the record, if you remove the <<utility>> stereotype from the class and regenerate the code, the static modifier is not present on the second operation.

In this section we've looked at the Operations category of the UML Class Properties dialog, and earlier we looked at the Code Generation category of this dialog; in both cases this was to show some interesting features of code generation. Additional categories allow you to fully define a class by specifying its attributes, stereotype, etc., but a full discussion of every category is beyond the scope of this case study.

Now classes do not standalone, so in the next two sections we'll look at how classes may be related through generalization or association.

C# Mapping for Generalization

Depending on which way you look at it, generalization (factoring common features into a super class) may alternatively be referred to as specialization (adding additional features to a sub class). Either way, what we're really talking about is one class inheriting from another as shown in this diagram.

For C#, that inheritance is mapped using the colon ":" notation as you can see below. You can also see that in this case, because the sub class (specialization) is in a different package from the super class (generalization), a using statement has been generated to import the foreign namespace.

// Preview of code that will be generated in the implementation file:

namespace MyPackage.MyPackage2
{

using MyPackage;
public class MyClass2 : MyClass
{

}// END CLASS DEFINITION MyClass2

} // MyPackage.MyPackage2

Inheritance (i.e. implementation) of multiple interfaces is allowed as far as C# code generation is concerned, but if you attempt to generate code for a class that inherits from multiple superclasses you will be rewarded with the message:

C# does not support multiple inheritance: ClassName

However the code will have been generated as though the multiple inheritance is allowed, as follows. Don't expect it to compile though!

// Static Model

public class SubClass : Superclass1, Superclass2
{

}// END CLASS DEFINITION SubClass

C# Mapping for Associations

Consider an association for which there is an explicit association class, like the one shown next. An EmploymentContract (association class) exists by virtue of a link between a Person and a Company in an employee - employer relationship.

The Visio UML online Help Documentation says of an association class "...association role ends are generated as private attributes in the association class, providing navigability." Which means that the classes at each end of the association have no direct knowledge of each other. The code generated for the association class (shown below) includes member variables to link instances of the two associated classes.

// Static Model

namespace MyPackage
{

public class EmploymentContract
{

private MyPackage.Person employee = null;

private MyPackage.Company employer = null;

}// END CLASS DEFINITION EmploymentContract

} // MyPackage

The generated code will look like that even if the association ends are not marked explicitly for navigation (note the lack of navigability arrows at either end of the association in the diagram).

Where no association class exists, the two associated classes are linked together directly in code via member variables as follows, but a member variable linking the associated class is only generated if the association is marked as navigable - using the UML Association Properties dialog - in the relevant direction.

// Static Model

namespace MyPackage.MyPackage2
{

public class Person
{

private MyPackage.MyPackage2.Company employer;

}// END CLASS DEFINITION Person

} // MyPackage.MyPackage2

// Static Model

namespace MyPackage.MyPackage2
{

public class Company
{

private MyPackage.MyPackage2.Person employee;

}// END CLASS DEFINITION Company

} // MyPackage.MyPackage2

Where an association has a multiplicity that is not one-to-one, for example an employer is associated with many employees, a member variable of an appropriate container type is generated as shown in the following C# code:

// Static Model

namespace MyPackage.MyPackage2
{

public class Company
{

private System.Collections.ArrayList employee;

}// END CLASS DEFINITION Company

} // MyPackage.MyPackage2

That ArrayList is fine for an unqualified one-to-many relationship, but where an association is qualified (has some property for looking up subsets of linked elements or individual linked elements) an appropriate key-value container is generated instead. Note how comments have been generated as an indication to the developer of the key- and value- types that should be inserted into the Hashtable.

// Static Model

namespace MyPackage.MyPackage2
{

public class Company
{

// Key(s) : MyPackage::Association1::employee::empNumber(string)
// Value : MyPackage::MyPackage2::Person

private System.Collections.Hashtable employee;

}// END CLASS DEFINITION Company

} // MyPackage.MyPackage2

That's as far as I'll go with the UML to C# mapping as my intention was really just to whet your appetite enough for you to look deeper into the mapping yourself. In my sample model I've used a variety of UML constructs for which you can determine the mapping by examining the generated code.

Sample Model Code Generation

The model that I've provided is in a Visio file called CodeGenModel.vsd. It's not a complete model of a system design, nor an example of good modeling practice. It's included solely so that you have something against which you can try the code generator, and so that you can examine some C# code (or Visual Basic, if you prefer) that contains mappings for various UML constructs.

Loading the Sample Model and Looking at the Diagrams

You can load the sample model into Visio for Enterprise Architects simply by double-clicking the CodeGenModel.vsd file, or by using the Visio File, Open menu option. If you just want to look at the model wihout generating code, you can also open it in Visio Professional 2002.

Initially you should be presented with the "Account Classes" Static Structure diagram, simply because that diagram was on display when I last saved the model. In the Visio Model Explorer you'll see that the full path to that diagram is Top Package / BankingDomain / Account / Account Classes. So if for some reason that diagram is not on display you can double-click it now. I'm drawing your attention to that one because it's what you see in the first two pictures of this case study, and it contains the Account class that we'll be looking into in the next section.It's just one of a number of class diagrams that are in the sample model. The top-level diagram Top Package / Static Structure-Main shows how the model is broken down into a number of packages:

Within the BankingDomain package there is an Account package containing an Account Classes diagram (as you have seen), as well as a Customer package containing a Customer Classes diagram as follows:

I've highlighted the structure of the whole model not as an invitation for you to comment on the design (it's only a pretend model) but to emphasize the fact that the hierarchic model structure provides various degree of granularity when generating code.

As you saw there, the Generate dialog lets you specify entire packages and / or individual classes.

Generating Code and Examining the Results

To generate the code for this model you can follow the steps given in the earlier How to Generate Code section. In a nutshell, choose the UML, Code, Generate menu option and step through the dialogs.

If you check the Add Classes to Visual Studio Project box you'll end up with a solution that you can load into Visual Studio .NET and which looks something like this:

At this point I encourage you to examine those source files and try to relate them back to the original model. I'd like you to pay special attention to the Account.cs source file that contains the code Account class. In particular, take a look at the AvailableBalance property as shown here:

public decimal AvailableBalance
{
get
{
return AccountData.AvailableBalance;
}
}

This code generator really is clever. Not only can it generate namespace and class definitions, implement generalizations as inheritance, and implement associations via member variables, but it can also guess at what you want each method to do and therefore write the method code for you!

Actually, no. As far as I know, none of the popular CASE tools can do that, because they take account of only the static model (i.e. class diagrams) when generating code. To write method code would require an understanding of the dynamic model as expressed via collaboration diagrams and state diagrams.

I wrote the method implementation code myself, inside the Visio model. To see how, you need to go back into Visio, double-click the Account class, and choose the Operations category. You will see that there is a single operation - actually a property - called AvailableBalance.

Now press the Methods. button and you'll see the code that I wrote for that method.

I suppose you could write the entire application by filling in the UML Method Properties dialogs in Visio, but I don't recommend it as you should really be writing the code in the Visual Studio .NET IDE.

Now you're probably wondering how the AvailableBalance operation actually became a property in the generate code. To see how, you need to click the Properties. box of the UML Class Properties dialog and then choose the Code Generation Options category as shown next:

As you can see, the operation "Kind" is Property and I've opted to generate only a Get method for it.

You might be interested to know that you can use the pull-down list to mark operations as other kinds for code generation, so as to map them as constructors, destructors, events, indexers, operators, procedures or properties.

At this point I encourage you to examine those source files and try to relate them back to the original model. Then alter the model as little, or as much as you like, to see what effect your changes have on the generated code. That's what I've been doing, which is why the model is not as tidy as it might be!

Mapping Classes to Components

You will notice that the sample model contains a UML Component Diagram at the location Top Package / Component-Main in the model explorer. It looks like this:

The reason for drawing a component diagram is to show how the software will actually be broken down into separate libraries (DLLs) and executables (EXEs) for distribution and deployment. Each component will become the deployment vehicle for a number of classes from the model, so - for example - the AccountComponent might contain the Account, AccountType, CurrentAccount and SavingsAccount classes. Indeed, I specified that mapping in the model itself via the UML Component Properties dialog.

What has this to do with code generation? Well thinking back to my Rational Rose days, I'm pretty sure that you could generate code on a per-component basis as well as (or instead of) from the original class diagrams.

I like that idea because I see an affinity between component stereotypes (executable, library, etc.) and Visual Studio .NET project types (web application, console application, class library, etc.). So, you could generate a separate project - of the appropriate type - for each component or set of components in the model.

Alas, no. As far as I can tell the code generation must be performed according to the package / class granularity supported by the Generate dialog (shown earlier), which is fine if your package structure corresponds with the intended component structure, but it doesn't have to. It's perfectly possible to map any number of classes, from any combination of packages, onto any component. Of course, you could move the generated classes between projects after code generation but you'd lose the direct traceability back to your model.

Case Study Review

I began this case study by introducing Visio for Enterprise Architects, which comes bundled with the Enterprise version of Visual Studio .NET, as a tool for modeling .NET applications with UML, and in particular for turning a UML model into .NET language code. There are other tools, but if you have an MSDN Universal Subscription you'll have got this tool as part of the package.

I showed how to generate code as a Visual Studio .NET solution or as a set of standalone source files using the UML, Code, Generate menu option in Visio. If you're serious about code generation you'll have to become familiar with the UML-to-C# mapping so I documented a few of the more common mappings - namely packages, classes, generalization, and associations - as examples.

I offered a sample model from which you can generate some code, and I used that model to show off some of the additional code generation options provided by Visio.

Finally I highlighted the role of component modeling, the class-to-component mapping, and the fact that this (unfortunately) has no effect on the code generation process.

It seems to me that the Visual Studio .NET / Visio for Enterprise Architects is not too well documented, nor publicized, so it may well have escaped your notice until now. I've put that right, I hope, and if you're keen on UML modeling for .NET then it's certainly a toolset to consider; but not the only one.

A note about other UML tools

As the bundled solution, I presume this toolset to be what Microsoft would want you to be using. A good enough reason for many people to use it, but - I suspect - a good enough reason for many people not to! To show that I'm not totally biased, and not paid by Microsoft, I'll point out that there are other tools capable of generating .NET code from a UML model.

Rational XDE Professional (http://www.rational.com/products/xde/dotneted/index.jsp?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq) has been designed as an eXtended Development Environment for developers. At the time of writing, I've not found time to check out the features, nor to compare the code generation with that of Visio for Enterprise Architect solution described here. From what I've read, XDE is fully integrated with the Visual Studio .NET and provides bi-directional synchronization between the (C#) code and the UML model. What that should mean is that there is no code generation phase as such, but a constant adoption of the code to reflect changes in the model. and vice versa. As I remember it, that kind of model-code synchronization was one of the main factors in the popularity of the Together UML modeling tool amongst Java developers. The only problem with XDE, it seems, is that this product alone could cost more than an MSDN Universal subscription!

Visual UML (http://www.visualuml.com/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq) is a relatively inexpensive and well-respected UML tool. Again I can't speak from personal experience, but I understand that it can run as an add-in to Visual Studio .NET and it provides code generation (plus reverse engineering) for the C# language.

A note about Reverse Engineering

Code Generation is just one aspect of the integration between Visio for Enterprise Architects and Visual Studio .NET, the flip side of that coin being Reverse Engineering. Suppose you didn't do all your UML modeling before writing the code. There's still a chance to impress the boss by transforming that code back into a model in Visio.

Just choose the Project, Visio UML, Reverse Engineer option in Visual Studio .NET, like this:

I should warn you, however, that the reverse engineering feature is not an exact counterpart of the code generation feature. You'll get all of the classes in the Visio Model Explorer, populated with their operations and attributes, but none of the diagrams will be drawn.

Whereas an association will map to a member variable in code, that member variable will not map back to an association when reverse engineered. That's true of Visio for Enterprise Architects, it's probably not true for Rational XDE, and it need not be true if you create your own reverse engineering tool as described in my previous case study "C# to UML Reverse Engineering" at http://www.csharptoday.com/content.asp?id=1742&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq.

In a nutshell then, round-trip engineering is not really viable using this toolset, but it's not claimed to be and in my experience the round-tripping provided by other tools does not always live up to the hype anyway. Treat it as strictly one-way: "Design" - "Generate Code" - "Implement", with an optional first step of "Reverse Engineer Existing Code".

Limitations and Further Work

Code generation is intended as a one-time-only activity in the transition from design to implementation. As stated in the Visio UML online Help Documentation:

"Note After you have generated code for a specific UML system and then modified that code in a source editor, you can't return directly to the UML model in Visio to further refine or edit the same code. Each time you generate code from a UML diagram, the resulting skeleton code becomes a starting point for a new development project."

The one-way nature of the code generation raises the potential danger of overwriting code that you have modified since it was generated originally.simply by generating again! Well they've thought of that so, each time you generate code, any existing source files are first renamed.

The following directory listing shows an original set of (Visual Basic) source files timed at 16:23 that have been automatically suffixed with ~1 to distinguish them from a new set of source files generated at 17:14.

17:14 <DIR> .
17:14 <DIR> ..
17:14 684 Account.vb
17:14 376 AccountType.vb
16:23 376 AccountType~1.vb
16:23 684 Account~1.vb
17:14 360 CurrentAccount.vb
16:23 360 CurrentAccount~1.vb
17:14 358 SavingsAccount.vb
16:23 358 SavingsAccount~1.vb

That's really useful, trust me!

The UML to C# mappings that I've shown are those of the Visual Studio .NET / Visio for Enterprise Architects code generator. To my knowledge the UML to .NET language mappings have not yet been standardized to ensure that all UML tools generate C# code in the same way. So expect different results if you use one of the other tools that I've mentioned above.

If you're generating code to run on the .NET platform, chances are that you'd like to relate your application classes in the model to the underlying .NET framework classes with which they interact. It's a pity then that Microsoft didn't think to provide a base model pre-populated with the .NET classes, particularly as you get such a thing with Rational XDE. Well I spotted that problem early on, so -- without blowing my own trumpet too loudly -- I can point you in the direction of the URL http://www.lotontech.com/visualmodeling/DotNetModel.html?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq.

Case Study Information
Author | Tony Loton
Editor | Adam Ryland
Reviewers | Matthew MacDonald, Phil Sidari, David Schultz

If you have any questions or comments about this case study, or suggestions for further pieces, please contact the technical editor. If you wish to talk about the case study with other subscribers, please visit our forum here.

Fast Track Books from Wrox Press
If you want a C# book that is compact, fast-paced but still provides the quality of information that you have come to expect from a Wrox book - you want Fast Track C#, newly available from Wrox Press.

This book will help you to quickly learn about the following:
  • Understand what C# is, and how it fits into the .NET Framework
  • A concise introduction to all the main features of the C# language, with Java and C++ comparisons noted
  • Using the .NET base classes
  • Writing and deploying Windows applications with C# and Visual Studio .NET
  • Creating and configuring .NET assemblies
  • Accessing databases with ADO.NET, and using XML in C#
  • Integrating your existing COM components with the .NET Framework
  • COM+ Services in the .NET Framework
  • Building web applications with ASP.NET pages
  • Creating web services in C#


Graphical Representation of Data Grids Using C# and GDI+

Graphical Representation of Data Grids Using C# and GDI+
http://www.csharptoday.com/content.asp?id=1848
Article Rating: 5
Published on July 1, 2002
Abstract
The .NET Framework implementation of the Graphic Device Interface (GDI+) provides powerful and flexible capability for the generation of a wide variety of graphical representations of data. In this case study, an application is created that displays data in multiple graphical formats, including pie charts, multi-line plots, 3D slice plots, 3D surface plots, and color mapping images. The study presents in detail the coding necessary to construct each form of graph. The application includes data input formats that allow the reader to utilize the program as a generic data viewer for any data that can be exported into a file that has the designated format. This enables the readers to immediately view their own data using the developed application.
Case Study

The .NET Framework implementation of the Graphic Device Interface (GDI+) provides powerful and flexible capability for the generation of a wide variety of graphical representations of data. The data represented could be a snapshot of business indices for a particular moment (such as this quarter's sales displayed by region for example), historical business indices (such as quarterly sales or daily stock prices over a given time period), scientific data (false-color satellite images, etc), or an imaginary landscape created for a gaming environment (hills, buildings, etc). All of these types of data can be viewed as a data grid, or table, which can be represented graphically using several different formats (line graphs, pie charts, bar charts, 3D bar charts, 3D surfaces, etc).

GDI+ improves upon the earlier GDI implementation in several ways. First, there are new capabilities, such as gradient brushes, alpha blending, cardinal splines (curve-fitting algorithms that join a set of points using curves instead of straight lines), matrix transformations, and support for many new image formats. In addition, the programming model has changed in GDI+ compared with GDI. The most profound change is the replacement of the handle-based device context model in GDI with the much more object-oriented approach represented by the GDI+ Graphics object.

Several articles are available that provide good introductory reading on using the GDI+ graphics classes with C# and Windows Forms. See the links at the bottom of this case study.

This case study will focus on areas not already covered in detail in the previous articles. For example, in this case study we will consider the problem of perspective and how it can be introduced to generate three-dimensional graphs suitable for presentations. While the geometrical equations and techniques are not the focus of the study, adequate attention is given to them to illustrate the fundamental problems that must be solved if we are to render data into three-dimensional images.

System Requirements

The software is developed using Visual Studio .NET Version 1.0 Release (1.0.3705).

Knowledge of the C# language and Windows Forms programming techniques is required.

Introductory knowledge of graphics programming using Graphical Device Interface (GDI+) is useful.

To understand every line of the code, knowledge of three-dimensional geometry is required; however, the focus of the case study is on programming using GDI+, not mathematical equations.

Defining the Problem

Data is everywhere, in business reports, corporate databases, satellite telemetry, etc. Who needs to understand data? Business executives, investors, scientists - almost anyone who has a job has to understand data of some type.

Unfortunately, most data is not organized in a manner where it can easily be comprehended and understood. Of what use is the data if it cannot be understood? Increasingly, the problem we face today is that we have huge reservoirs of data, yet we lack methods for rendering the data into a form where it can be assessed and conclusions drawn.

Most data sets that require analysis and presentation are complex, and cannot be easily described in words or even in a table. An example would be grids of data where there are multiple sets of values that are similar, but which represent different instances or conditions to which the data apply. For example, corporate financial performance indices for a single company over a specified time period represent a single instance of a data set. However, it might be more meaningful to compare these same performance indices over the same time period for several different companies. In this case, we need a means to display what could be called a "data grid", that is, a grouping of several instances of similarly structured data. A data grid doesn't have to originate with corporate data. Scientific data gathered from several repeated experiments can also be seen as representing a "data grid".

Data grids can typically be displayed in a table. However, tables are not always ideal, because in a table of numbers, the relation between the various instances is not typically apparent "at first glance". Significant mental work is required to compare values across rows and columns, look for trends and differences, etc. Working with a table is time consuming and tedious, and tables are also only functional for data sets that have a relatively small number of values. You cannot realistically expect an audience to draw conclusions of any type from a table with more than a half-dozen columns and perhaps a dozen rows. A very large table will quickly overwhelm the audience to whom we are trying to convey information.

Enter plots and graphs! One of the simplest ways of representing multiple instances of similar two-dimensional data (data grids), is the multi-line plot. While line plots are useful, they do not always present well. Grids of data can be displayed in other types of plot as well, for example 3D slice plots, 3D surfaces, and color mapped images.

Case Study Structure

The case study presents five different types of graph:

  • Pie Chart: used to demonstrate basic GDI+ concepts and illustrate the problem of visual perspective.
  • Multi-Line Plot: a basic plot of multiple data sets each sharing a common range of X and Y values.
  • 3D Slice Plot: a three-dimensional rendering of multiple data sets each sharing a common range of X and Y values, suitable for presentations.
  • 3D Surface Plot: a three-dimensional rendering of a grid of data points where each square of adjacent points is plotted as though it was a solid plate (such that all points on the opposite side of the plate are hidden).
  • Color Mapping Image Plot: a representation of a data grid as an image, where the data values are scaled using a color bar.

Defining the Solution

Windows Forms, C#, and GDI+ can be applied as the foundation for generating powerful and flexible graphics that represent complex data grids. The fundamentals of GDI+ are contained in the Graphics class: the Point structure, SolidBrush, Font, Color, Pen, FillPie(), FillPolygon(), DrawPolygon(), DrawString(), and other structures, classes, properties, and methods. These will be discussed as we go through the case study.

Database Design

The application does not use a database, but the data represented in the plots could certainly come from a database. The focus of the application is data grids. By a "data grid" we mean a set of data set instances, where the data values in each data set have characteristics in common with the data values in the other data sets. An example would be a SQL table, where each row could be termed a data set consisting of values of a specified type. A set of rows from a single table would be a data grid. For the application, the data grid must consist of numeric values, but the principle is the same: you have M data sets, each consisting of N data values which are of corresponding types and meaning across the data sets (just as data values in the same SQL column but in different rows/records can be said to "correspond" to one another).

UI & Overall Software Design

The focus of the case study application is the generation of the graphics. For this reason, the application has a very simple user interface, with very little complicated coding. Controls for selection of plot type, data input file, tilt angle, and rotation angle are located on the left of the form, along with a button that launches the selected graphing function.

Declaration and initialization of several variables for holding data and colors is performed outside of the individual graphing functions to reduce the "clutter" in those functions, and enable greater focus to be put on the actual graphical rendering of the input data - which is complex in several of the cases due to the requirements for accurate three-dimensional perspective based on user selections of tilt and rotation angle.

The data input functions are also separated from the graphics functions, for a similar reason. These consist of code for reading basic text files, and it is assumed that the reader already knows these basics of C# programming. The data is stored in simple text files to provide an easy-to-use format to which data from a wide variety of sources could be readily converted. The input data could originate from a SQL table, a spreadsheet, an XML file, a mainframe flat file, or a binary data stream output by a scientific instrument. The text file formats are intended to be self-documenting, as they map directly into the variables used to create the graphs. It would be very easy for the reader to replace the text data input functions with others that directly read a specific type of input data.

The core of the program is in the five methods that generate the plot types identified above, namely pieChart(), multiLinePlot(), slice3DPlot(), surface3DPlot(), and colorMappingPlot().

Pie Chart Component

A pie chart is a simple type of plot that is widely used in business and even in scientific presentations of data because it is so easily understood. The biggest slice stands out, and slices that are small relative to other slices are immediately revealed as being less significant. Yet, without a powerful graphics library, generation of a pie chart using software is very difficult. Consider a simple graphics toolkit that provides some basic functions such as drawing a filled rectangle, but has no functionality whatsoever involving arcs, radial lines, circles, ellipses, etc. Drawing a pie slice with such a toolkit will be exceedingly difficult, and would also require substantial knowledge of the geometrical equations involved.

Fortunately, GDI+ provides a very powerful and easy to use Graphics method named FillPie that is designed explicitly for this purpose. The FillPie method creates a filled in pie slice using a specified Brush, geometrical parameters defining an ellipse (note that a circle is a type of ellipse), and values defining the angular region within the ellipse that is to be filled using the specified Brush.

The FillPie method has three overloads. One overload allows the use of a RectangleF structure for defining the ellipse. In the other two overloads, the ellipse is specified using the top left x and y coordinates and the width and height of the rectangle that encloses the ellipse. The only difference between the last two overloads is that in one case the numeric parameters are of type int, while in the other case the numeric parameters of are of type float. For details, refer to the "FillPie method" discussion in the Visual Studio .NET documentation.

The existence of multiple overloads for Graphics methods where the only difference between some overloads is the numeric parameter type (int or float) is a very common feature of GDI+. For development of graphics components using a strict pixel-based mapping, use of the int numeric parameters will be useful. However, if the graphics component is to be designed such that the displayed graphics will be resizeable (for example, when the user resizes the window form on which the graphic appears), then the use of the float parameter lists might be preferred, since the displayed graph could be more accurately scaled to any size with greater precision.

Pie Chart Code

The pieChart() method draws the pie chart based on the input set of pieValues[]. First, the input values must be "normalized". By "normalization" we mean that a scaling factor must be computed that can be applied to the data such that the scaled values will total to 360, since the full pie will contain 360 degrees. The scaleFactor is found by summing the raw input values and dividing 360.0 by that total:

double rawValueTotal = 0.0F;
double
            scaleFactor = 0.0F;

for (int i=0; i<pieSlices;
            i++)
{
rawValueTotal += pieValues[i];
}

scaleFactor
            = 360.0F / rawValueTotal;

Drawing the pie chart requires only a few lines of code once the location of the graphics on the form have been specified. CreateGraphics is called to create the Graphics object for the form, the graphics area is cleared to the same background color used by the form (to erase any previous graphics before the new graphics are drawn), and FillPie is called in a loop that sweeps around the pie circle, consecutively drawing the pie slices using the pre-defined colors[] color set.

You will note that the FillPie() method defines the position of angle 0.0 as the 3:00 position on the clock. A positive angle is a rotation in the clockwise direction from this starting position. Also, the angles passed to FillPie() must be in units of degrees. This is important to remember, because when making graphics that require the use of geometrical formulas and equations, it is often necessary to convert angles from degrees to radians. If an angle is sent to a method using the wrong units, the resultant plot will either make you laugh or frown (depending on your mood at the time), but it will certainly not be what you intended.

float x = 200.0F;
float y =
            50.0F;
float width = 300.0F;
float height = 300.0F *
            tiltFactor;
float angleSize = 0.0F;

Graphics g =
            CreateGraphics();

g.Clear(Color.LightGray);

for (int
            i=0; i<pieSlices; i++)
{
angleSize =
            (float)pieSliceAngle[i]; //rawValues[i] *
            scaleFactor;
pieBrush.Color =
            colors[i];
g.FillPie(pieBrush,x,y,width,height,angleStart,angleSize);
angleStart
            += angleSize;
}

Putting Pie Slices into Perspective

GDI+ makes drawing a basic pie chart using Windows Forms and C# a very easy task. However, what if we want to get fancy and make the pie chart look as though we are viewing it on a slant? The first inclination is to think "No problem, I'll just redefine my rectangle that bounds the pie ellipse from a square shape to a rectangular shape and apply the same FillPie() loop." That was my first idea, anyway.

Wrong! To see this, look at the base (not tilted) pie chart that results from the example file Pie5.txt :

Here is what results if we apply the original idea of simply changing the rectangle that bounds the pie ellipse from a square to a rectangle that has the same width but a reduced height:

If you look closely at the "squished" pie chart, and imagine that you are actually viewing a circular pie where the top is tilted away from you and the bottom is tilted toward you, you will notice that the individual pie slices on the not-tilted and tilted pies appear to differ in size. Continuing to look closely, you can see that the angles for each corresponding pie slice are identical in the two graphs. But, identical pie slice angles in a tilted and a not-tilted pie suggest different pie slice sizes to us, visually. This is the effect of perspective.

For correct perspective, the angles of the individual pie slices must change when we want to represent a pie viewed on a tilt. You can prove this by doing a simple experiment: print the not-tilted pie above, hold it in front of you, and slowly tilt the top away from you and the bottom toward you, and observe the pie slices.

So how must the graph be drawn if we want to represent a tilted pie and have the slice sizes appear correctly proportioned? Geometrical equations must be applied to define the "projection" of the tilted pie onto the geometric plane that is the form on which the graph is plotted.

Essentially, the "projection" defines the grid location of each point on the surface that has been tilted on a plane that is perpendicular to the line of sight from the observer. Since we are tilting the top of the pie slice "away" from us (and the bottom toward us), while keeping the horizontal line through the center of the pie slice fixed, the X values of all the points are the same on the projection as they are on the tilted pie. However, the Y values change by the factor called tiltFactor in the program. Specifically, each new Y value on the projection is the original Y value mutiplied by the cosine of the tilt angle:

tiltFactor = cos(tiltAngle)

So, if we are not tilting the pie at all, tiltFactor = cos(0 degrees) = 1.0, giving us the original untilted coordinate values. If the pie is tilted by 90 degrees, we get tiltFactor = cos(90 degrees) = 0.0, meaning that all the Y values are zero. This, in fact, is exactly what we would expect: tilting the pie by 90 degrees would give us an edge-on view, which means we'd see none of the pie at all. Mathematically, all the Y values project to a value of zero on the "projection", resulting in us seeing nothing. For this reason, in the program we limit the tilt for the pie graph to a range from 0 to 85 degrees.

This all sounds very complicated, and it is not within the scope of this case study to provide a full discussion of mathematical equations. If you are eager to explore the mathematical aspects of this type of programming, many good references are available, on the Internet and in books.

Suffice to say, new angles for the pie slices must be calculated based on the tilt angle and the position of the slice within the pie. The equations utilize the Math library sine and cosine functions, and are implemented in the code section immediately preceding the section that draws the pie chart. The strange looking "DTOR" variable is the factor that converts D egrees TO R adians (3.14159 / 180.0).

double scaleTotal = 0.0;
double lastX =
            0.0F;
double lastY = 0.0F;
double lastAngle = 0.0F;
double
            newAngle = 0.0F;
for (int i=0; i<pieSlices;
            i++)
{
scaleTotal = pieValues[i] * scaleFactor +
            scaleTotal;
pieSliceX[i] = (float)Math.Cos(scaleTotal *
            DTOR);
pieSliceY[i] = (float)Math.Sin(scaleTotal * DTOR) *
            tiltFactor;
newAngle = (float)(180.0 / 3.14159 *
            Math.Atan2(pieSliceY[i],
pieSliceX[i]));
if (newAngle <
            0.0F)
{
newAngle += 360.0F;
}
pieSliceAngle[i] =
            newAngle - lastAngle;
//MessageBox.Show("scaleTotal = " +
            scaleTotal + " x,y = " +
pieSliceX[i] + "," + pieSliceY[i] + "
            newAngle = " +
newAngle);
lastX = pieSliceX[i];
lastY =
            pieSliceY[i];
lastAngle = newAngle;
}

The result of this recalculation of pie slice angles as "projected" onto the plane of the form is a pie chart where the pie slices all appear properly sized no matter what angle is selected for its tilt. Based on the tilt angle, we can calculate the Y value on the projected plane for the border points on the outer edge of each pie slice. Since the X values are the same on the projection as on the untilted pie, we then can calculate the newAngle from the center of the pie to the projection of the tilted pie slice using the Atan2 arctangent function.

With the angles properly recalculated, the pie chart at 75 degrees tilt looks like this:

Pie Chart Input Data File Structure

The input data for the pie chart is stored in text files with a simple format. If you generate your own text files in the following format, place the file into the directory where the program file DataGridGDI.exe is located, add the file name to the comboDataFile Items collection, and rebuild the application, you will be able to use the application to generate pie charts using your data set. Note that in the current build of the software, the pie chart is limited to at most 10 pie slices.

The pie chart input data file consists of a file type identification line consisting of the word "Pie", a line listing the number of slices in the pie, and lines that list the floating point value to be represented in each pie slice. The file structure details are as follows:

Line | Data Type | Description
1 | String | file type identifier: "Pie"
2 | Int | nSlice: number of pie slices (up to 10)
3 to nSlice+2 | float | Pie slice values

Here is a printout of one of the example pie chart files included in the case study (Pie5.txt):

Pie
5
34.2
23.4
23.1
12.1
4.5

Multi-Line Plot Component

A pie chart is fine for representing the relative size of a small set of data values. However, most data sets do not consist of a single dimension where all we need to see is the relative size of the values. One of the most common ways of representing grids of data is the multi-line plot. In this type of plot, a line represents a data set, and all the data set lines are plotted on a single graph using a common X and Y axis. The lines will typically be plotted using different colors or different line types, to distinguish the data sets. Graphs of this type will usually include a legend, which identifies the data set that corresponds to each line (although, in this case study, we are not including legends).

To view an example multi-line plot generated by the case study application, run DataGridGDI.exe, select plot type "Multi-Line" and data file "Slice5.txt", and click the "Plot" button. The form should look like this:

Multi-Line Plot Code

The multiLinePlot() method generates the multi-line plots in the DataGridGDI.exe application. The code is relatively simple because with multi-line plots there is no option to tilt or rotate the graph, so no considerations regarding perspective are required. The parallelogramGrid() method is called to draw the axis grid against which the lines are plotted. You might wonder why we need a "parallelogram" grid in this instance. In fact, we don't. However, a rectangle is a type of parallelogram, and the parallelogramGrid() function is required for the 3D Slice plot described in the next section. So, to minimize code, we call this method rather than write code specifically for creating a rectangular grid. parallelogramGrid() is described in greater detail in the 3D Slice plot discussion.

To label the axes grid appropriately, the lineOfNumericLabels() method was written. This function draws a set of numeric labels along a line specified by two Point structures using a specified Pen. This code is worth examining briefly for its use of the Graphics.DrawString() method, which is the primary method for labeling plots and graphs with alphanumeric information.

private void lineOfNumericLabels(Graphics g,
            Pen myPen, Point p1, Point p2,
float value0, float value1, int
            nLabels)
{
int yOff = 0;
int xOff = 0;
float fval =
            0.0F;

SolidBrush labelBrush = new
            SolidBrush(myPen.Color);
Font labelFont = new
            Font("Arial",10);

Point pA = p1;

float rng = value1 -
            value0;
int xTot = p2.X - p1.X;
int yTot = p2.Y -
            p1.Y;
g.DrawString(value0.ToString(),labelFont,labelBrush,
            (float)(p1.X),
(float)(p1.Y));
for (int i=1; i<nLabels;
            i++)
{
xOff = i*xTot/(nLabels-1);
yOff =
            i*yTot/(nLabels-1);
pA.X = p1.X + xOff;
pA.Y = p1.Y +
            yOff;
fval = value0 + rng *
            (float)(i)/(float)(nLabels-1);
g.DrawString(fval.ToString(),labelFont,labelBrush,
(float)(pA.X),
            (float)(pA.Y));
}
}

DrawString() sounds simple enough, but the method actually offers six different overloads. In all cases, a Font and Brush must be supplied, along with the string to be displayed. The overloads differ in how the location of the string is defined (either as a PointF, a RectangleF, or a pair of float x and y values) and whether a StringFormat object is defined. The StringFormat class allows definition of text layout information such as alignment, line spacing, clips, etc. Although it is not applied in the current version of the case study application, it certainly would be very useful in any larger scale graphics application.

Once the axes grid has been drawn and the axes labeled, the data set lines can be drawn. This is done by converting the floating point X and Y values for each point into the corresponding pixel locations on the form and using DrawLine() to draw a line segment connecting each X, Y location. As is the case with the majority of GDI+ Graphics methods, DrawLine() has multiple overloads, permitting the end points of the line to be specified as Point structures, PointF structures, int x,y sets, or float x,y sets.

The code for drawing the data set lines is as follows:

int xLast = 0;
int yLast = 0;
for (int
            iLine=0; iLine<nSlice;
            iLine++)
{
myPen.Color=colors[iLine];

for (int i=0;
            i<nPts; i++)
{
int x = xLeft +
            (int)((xPts[i]-xMin)/(xMax-xMin) *
(float)(xRight-xLeft));
int
            y = yBot - (int)(((float)yPts.GetValue(iLine,i)
            -
yMin)/(yMax-yMin) * (float)(yBot-yTop));
if
            (i>0)
{
g.DrawLine(myPen,xLast,yLast,x,y);
}
xLast =
            x;
yLast = y;
}
}

While not used in the case study application, GDI+ also includes the convenient and very efficient DrawLines() method. This method can be used to draw a line linking a set of points contained in a Point[] or PointF[] array. Using the DrawLines() method thus eliminates the need for a line segment plotting loop, and we would expect the time required to execute the drawing of the line to be smaller than the time required when the line segments are individually drawn. However, DrawLines() can only be used when your line data is in the form of Point or PointF structures.

Multi-Line Plot Input Data File Structure

The input data for the multi-line plots is stored in simple text files, to make it easy for the user to apply the application to new data sets. Again, if you generate your own text files in the following format, place the file into the directory where the program file DataGridGDI.exe is located, add the file name to the comboDataFile Items collection, and rebuild the application, you will be able to use the application to generate multi-line plots using your data set. Note that in the current build of the software, the multi-line plot is limited to at most 10 data sets each having a maximum of 25 points.

The multi-line input data file contains file type identification (the word "Slice"), the number of data sets, the number of data points in each data set, the minimum and maximum X and Y axis values, and the data values. The file structure details are as follows:

Line | Data Type | Description
1 | string | file type identifier: "Slice"
2 | int | nSlice: number of data sets / slices (up to 10)
3 | int | nPts: number of data values in each data set / slice
4 | float | xMin: minimum value for the X axis
5 | float | xMax: maximum value for the X axis
6 | float | yMin: minimum value for the Y axis
7 | float | yMax: maximum value for the Y axis
8 to nPts+7 | float | data values for data set # 1
nPts+8 to 2*nPts+7 | float | data values for data set #2
… through nSlice*nPts +7 | float | data values for all remaining data sets

Here is a portion of the file Slice5.txt, which is used as the multi-line and 3D surface plot example case by the DataGridGDI.exe application:

Slice
5
16
10.0
40.0
0.0
4.0
1.0
1.5
2.5
3.2
…

3D Slice Plot Component

A multi-line plot is certainly a functional method for displaying multiple instances of similar data sets. However, when it comes to making presentations (sales or otherwise), a line plot does not stand out. For a presentation, you need something more exciting, snazzy, fancy - whatever you'd like to call it - than a line plot. The data you are presenting may be the same, but the presentation itself can speak volumes.

A 3D Slice plot is one of the possibilities for presenting the same type of data that is readily presented in a multi-line plot in a more impressive way. To see an example of this, using the same data displayed in the multi-line plot above, select plot type "3D Slice", data file "Slice5.txt", tilt 10, Rotation 15, and click the "Plot" button. The form window should look like this:

To see different 3D renderings of the data, you can select different values for tilt and rotation, and re-click "Plot".

From the start, it's clear that the software to build this type of graph must be much more complicated than that which was required to build the multi-line plot. How many lines of code will it take to calculate where one slice ends and the next one begins? How many lines to determine where faces are entirely hidden from view? And not only that, we're going to let the user adjust the tilt and rotation, and this thing is going to be drawn correctly each time?

It sounds like a programming nightmare. Or, at minimum, something that will require an enormous amount of code, full of complex geometrical equations and a thousand if and switch case conditions to cover all the hidden and adjacent surface possibilities.

Indeed, the output 3D Slice plots suggest great complexity, and the code certainly is much more difficult than the multi-line code. Yet, if we break down the task into smaller pieces, apply a few geometrical tricks to simplify the equations (physicists are always doing that, so why not us too?), and "cheat" a little, using the fact that the computer is quicker than the eye, we can actually make a powerful, flexible 3D Slice plot function in fewer than 250 lines of C# code! In the next section, we'll see how we do it.

3D Slice Plot Design

The 3D Slice plot code is implemented in the method slice3Dplot() and the methods it calls. First let's break down the graph into its basic components. The 3D Slice graph consists of essentially a three-dimensional multi-walled grid, with appropriate axis labels, and a set of three-dimensional data slices constructed as "walls" with height varying in correspondence to the data values in the slice.

The three-dimensional grid is made up of three parallelogram-shaped grids (one of which is a rectangle), all joined together on a shared edge. Here's a view of a three-dimensional grid by itself.

The three-dimensional data slices have a "roof" that is of a certain width and consists of "plates" joining consecutive data points, a front wall which extends from the roof to the floor grid and which has no joints, and a thin wall at the right end of the slice that is visible only when the slice is rotated slightly. Here's a picture of a slice all by itself:

These are the essential components that compose the 3D Slice plot. To make the plot, all we need to do is make graphics functions that draw these objects in accordance with the selected tilt and rotation, join them together at the appropriate places, and hide all surfaces and lines that would be hidden if the graph components were solid objects.

3D Perspective: Simplified

To simplify the geometrical equations that must be applied to produce the three-dimensional effect, we fix the back grid rigidly in place. This is not an entirely realistic representation of what occurs if you are holding a three-dimensional object in your hand and you start tilting and rotating it. However, we are not trying to represent an actual object in the 3D Slice plot-we're simply trying to display slices of data in a manner that impresses and conveys the relationship between the data slices. So, we don't need to be concerned about the strict laws of object perspective in this case.

"Nailing" the back axis grid to the form, as it were, permits us to simplify the relation between the user's selections for tilt and rotation and how the axis grid and slices are drawn. A tilt value can be converted into an offset that defines how far below the top of the back wall we will draw the upper front corner of the left grid wall. Similarly, the rotation value is converted into an offset that defines how far to the left of the back wall we will draw the front edge of the left wall. In other words, the tilt angle is converted into a Y offset, and the rotation angle is converted into an X offset.

The fine details of this logic need not concern all readers. The point is that fixing the back wall axis grid in place and applying linear offsets for all other positions on the left wall and floor grids simplifies the mathematics tremendously, which makes the code much simpler and significantly less lengthy.

In addition, we simplify matters further by limiting the tilt and rotation ranges to values that ensure that our point of observation is always slightly above and slightly to the right of the graph. This means the top edge of the grid on the left wall will always be drawn down and to the left of the back wall.

How to 'Cheat' While Hiding 'Hidden' Surfaces

One of the biggest problems faced by 3D graphics programming pioneers was the issue of hidden surfaces. How could you calculate when a line or surface should not be drawn, because it would not be seen by a viewer because another surface is located between it and the viewer? The mathematical analyses required to resolve this problem were complex and very time consuming, since the computers at that time were very slow. I know this from personal experience, having developed an application that represented three-dimensional scientific models (in spherical coordinates) in the form or graphics that could be rotated or tilted in any direction. It was a great tool for the scientists, but the code was complex, and though it made good pictures, it was still possible to trick it into drawing lines that should not have been drawn, or leaving some lines much shorter than they should have been.

If we had to apply hidden line/surface techniques to make our 3D Slice plot, the code would indeed be lengthy and complex. However, the advent of graphics toolkits that could draw filled-in shapes changed everything. Instead of calculating if a line or surface was hidden, all you had to do was draw your surfaces starting with the one farthest away from the viewer, and draw each and every surface in order from farthest away to closest to the viewer. What happens is that all surfaces that should be hidden are overdrawn by the precise surfaces that hide them, resulting in a perfect hidden surface rendition without using a single hidden surface equation!

So, this is surely the technique we should implement here. We will draw the slices from farthest away to closest to the viewer. That is, we will start by drawing the slice that touches the back wall. We will draw that entire slice, not giving any thought to what part of it might be hidden in the final graph. Then we will draw the other slices in exactly the same way, simply drawing the entire slice where it belongs, with no consideration for which parts of it, if any, will be visible or covered over by another slice.

When this method first became possible, computers and graphics cards were slow enough that you could watch the back surfaces being drawn then covered over by the surfaces in front of them. But with today's computers and high speed graphics, the user of the program will never know whether you performed complex hidden surface calculations or simply painted over as many layers as necessary to arrive at the proper 'hidden' perspective.

3D Slice Plot Code

The slice3DPlot() function starts with initialization of various data structures - Point structures, SolidBrush objects, a Pen, and the form's Graphic. A SolidBrush is an object that is used to fill a shape, such as a polygon, with a solid color. In slice3DPlot(), two SolidBrush objects are defined, because each slice is plotted using a different color for the roof of the slice and the walls of the slice. Drawing the slice this way makes the graph easier to view, and makes its three-dimensionality more apparent.

private void slice3DPlot()
{
double
            DTOR = 3.14159 / 180.0;

Point p1Base = new
            Point(0,0);
Point p2Base = new Point(0,0);
Point p3Base = new
            Point(0,0);
Point p4Base = new Point(0,0);

Point p1 = new
            Point(0,0);
Point p2 = new Point(0,0);
Point p3 = new
            Point(0,0);
Point p4 = new Point(0,0);

Graphics g =
            CreateGraphics();
g.Clear(Color.LightGray);

Pen myPen =
            new Pen(Color.Black);
SolidBrush topBrush = new
            SolidBrush(Color.Black);
SolidBrush sideBrush = new
            SolidBrush(Color.Black);

int xLeft = 250;
int xRight =
            550;
int xTot = xRight - xLeft;
int yTop = 50;
int yBot =
            250;
int yTot = yBot - yTop;

Following the initialization section, the variables sliceXoff and sliceYoff are calculated. These values are the pixel offsets that give the graph its three-dimensional appearance based on the values for tilt and rotation selected by the user. Developers who will be using the .NET Math libraries should note the DTOR factor that is applied to the user's angle selections to convert the units from D egrees TO R adians.

When we look at the 3D Slice plot, we notice that, like the multi-line plot, the 3D Slice plot uses an X/Y axis grid to represent the magnitude of the data along the X and Y axes. The axis grid at the back of the 3D Slice plot is in fact identical to the one used in the multi-line plot. However, the 3D Slice plot requires additional axis grids. Along the left wall, the grid delineates the individual data slices and marks the Y scale again. There is also a floor grid, which delineates the data slices and the X axis scale. The floor grid is not visible in the example case, because the data slices are stacked on top of it. As with the multi-line plot, a good place to start in drawing the 3D Slice plot is to draw the axis grid. In the case of the 3D Slice plot, this is performed by the axis3dGrid() method. axis3dGrid() calls both parallelogramGrid() and lineOfNumericLabels() to draw a three-dimensional grid based on the tilt and rotation settings specified by the user.

The parallelogramGrid() method draws a grid in the shape of a parallelogram that is defined using three points. The points are considered to be three of the four vertices of the parallelogram. The fourth vertex is easily located by applying geometry. In addition to the points, parallelogramGrid() receives the Graphics, a Pen, and integer parameters nX and nY, which define the number of "intervals" into which the X and Y axes are to be divided. For example, if nX is 2, then the grid will have a grid line halfway down the X axis, dividing the X axis into 2 intervals.

The code for parallelogramGrid() is not complex, but it does illustrate the type of manipulation of Point structures and the use of DrawLine() that is required to have a generalized function for generating axis grids that appear tilted and rotated.

private void parallelogramGrid(Graphics g,
            Pen myPen, Point p1, Point p2,
Point p3, int nX, int
            nY)
{
int yOff = 0;
int xOff = 0;

SolidBrush
            labelBrush = new SolidBrush(myPen.Color);

//draw lines
            parallel to p1-p2
Point pA = p1;
Point pB = p2;
int xTot =
            p3.X - p1.X;
int yTot = p3.Y -
            p1.Y;
g.DrawLine(myPen,pA,pB);
for (int i=1; i<nY;
            i++)
{
xOff = i*xTot/(nY-1);
yOff = i*yTot/(nY-1);
pA.X
            = p1.X + xOff;
pA.Y = p1.Y + yOff;
pB.X = p2.X + xOff;
pB.Y
            = p2.Y + yOff;
g.DrawLine(myPen,pA,pB);
}

//draw lines
            parallel to p1-p3
pA = p1;
pB = p3;
xTot = p2.X -
            p1.X;
yTot = p2.Y - p1.Y;
g.DrawLine(myPen,pA,pB);
for (int
            i=1; i<nX; i++)
{
xOff = i*xTot/(nX-1);
yOff =
            i*yTot/(nX-1);
pA.X = p1.X + xOff;
pA.Y = p1.Y + yOff;
pB.X
            = p3.X + xOff;
pB.Y = p3.Y +
            yOff;
g.DrawLine(myPen,pA,pB);
}

}

Once the 3D axis has been drawn, slice3DPlot() draws the individual slices. First, the topBrush.Color and sideBrush.Color properties are set in a switch() statement. In all cases, a dark color is used for the top color, and a lighter version of the same color is used for the side walls.

for (int iSlice=0; iSlice<nSlice;
            iSlice++)
{
switch(iSlice)
{
case 0:
topBrush.Color
            = Color.Red;
sideBrush.Color = Color.LightPink;
break;
case
            1:
topBrush.Color=Color.Blue;
sideBrush.Color =
            Color.LightBlue;
break;
case 2:
topBrush.Color=Color.Green;
sideBrush.Color =
            Color.LightGreen;
break;
case 3:
topBrush.Color=Color.Yellow;
sideBrush.Color =
            Color.LightGoldenrodYellow;
break;
case 4:
topBrush.Color=Color.SlateGray;
sideBrush.Color =
            Color.LightSlateGray;
break;
}

Next, the slice is drawn. This requires some fairly complex coding. What we are really doing is breaking down the drawing of a single slice into a drawing of multiple columns, each with a side wall and a roof plate bounded by two consecutive data points. We need to scale the actual data points to find the height of the column above the floor, but the Y value on the Graphic of the floor beneath each slice is different for each slice, and the Y value on the Graphic for the floor at the front face of a slice is different from the Y value on the Graphic for the floor at the back face of the same slice.

If your goal is to learn how to draw complex three-dimensional solid figures using C# and GDI+, you might want to look carefully at this code and try to follow what's happening. If your interest is more in learning about the basic GDI+ classes, and you simply would like to use the application, tracing this section of code line-by-line is probably not needed.

   //back left pixel position
            on plot floor
p1Base.X = xLeft + sliceXoff * iSlice /
            nSlice;
p1Base.Y = yBot + sliceYoff * iSlice /
            nSlice;

//front left pixel position on plot floor
p2Base.X
            = xLeft + sliceXoff * (iSlice + 1) / nSlice;
p2Base.Y = yBot +
            sliceYoff * (iSlice +1) / nSlice;

//back left pixel position
            at top of column
p1.X = p1Base.X;
p1.Y = p1Base.Y -
            (int)(((float)yPts.GetValue(iSlice,0) - yMin) /
(yMax-yMin) *
            (float)(yTot));

//front left pixel position at top of
            column
p2.X = p2Base.X;
p2.Y = p2Base.Y -
            (int)(((float)yPts.GetValue(iSlice,0) - yMin) /
(yMax-yMin) *
            (float)(yTot));

//loop for all points in the slice
for
            (int i=1; i<nPts; i++)
{
//front right pixel position at
            bottom of column
p3Base.X = xLeft + sliceXoff * (iSlice + 1) /
            nSlice +
i*xTot/(nPts-1);
p3Base.Y = yBot + sliceYoff *
            (iSlice + 1) / nSlice;

//front right pixel position at top of
            column
p3.X = p3Base.X;
p3.Y = p3Base.Y -
            (int)(((float)yPts.GetValue(iSlice,i) -
yMin) / (yMax-yMin) *
            (float)yTot);

//back right pixel position at top of
            column
p4.X = p1Base.X + i*xTot/(nPts-1);
p4.Y = p1Base.Y -
            (int)(((float)yPts.GetValue(iSlice,i) -
yMin) / (yMax-yMin) *
            (float)yTot);

//draw roof
Point[] polygonPoints = {p1, p2,
            p3,
            p4};
g.FillPolygon(topBrush,polygonPoints);
g.DrawPolygon(myPen,
            polygonPoints);

//draw front wall
Point[] polygonPoints2 =
            {p2, p2Base, p3Base,
            p3};
g.FillPolygon(sideBrush,polygonPoints2);

//redraw
            line where roof intersects front wall
g.DrawLine(myPen, p2,
            p3);

//for first point, redraw line at intersection with left
            axis
if (i == 1)
{
g.DrawLine(myPen, p2,
            p2Base);
}

//save computed points on right side as new
            points on left side
p1 = p4;
p2 = p3;
p2Base =
            p3Base;
}

//fill in the right wall for this
            slice
p4Base.X = p4.X;
p4Base.Y = p1Base.Y;
Point[]
            polygonRightWall = {p3, p3Base, p4Base,
            p4};
g.FillPolygon(sideBrush,polygonRightWall);
g.DrawPolygon(myPen,
            polygonRightWall);

Essentially, everything is plotted with reference to the fixed back wall grid. The xLeft and xRight integers are the pixel positions on the Graphic that define the back wall X axis limits. Likewise, the yTop and yBot integers define the pixel positions that limit the back wall Y axis. Points p1Base and p2Base are computed as the back and front pixel positions for the current data point on the floor axis, which is the base of the column we are drawing. Points p1 and p2 are the points directly above p1Base and p2Base at the pixel height corresponding to the numeric Y value for the slice for that X axis position. These four points define the left face of the column we are about to draw, but since the graph is tilted such that the left face is always hidden, we do not draw this surface. Rather, we use these points to draw the roof plate and front face of the column formed by the first data point and its immediate successor. This happens in the loop.

Within the loop, we first compute p3Base, the pixel position at the front wall of the slice on the floor axis for the next point in the slice. In other words, point p3Base is the equivalent of point p2Base, except that p3Base is that position for the next value in the slice data set. Point p4Base is computed as the pixel position at the floor along the back wall of the slice for the next point in the slice. Points p3 and p4 are the points directly above p3Base and p4Base at the pixel height corresponding to the numeric Y value for the slice for that X axis position. These four points define the right face of the column we are about to draw. Meanwhile, points p2, p2Base, p3Base, and p3 define the front face of the column; and points p1, p2, p3, p4 are the pixel positions that define the roof plate for this section of the slice.

Now we can start drawing! We define Point[] array polygonPoints[] as {p1, p2, p3, p4}, that is, the roof of the column between data values i-1 and i. We invoke the FillPolygon() method using the overload construct that calls for a Point[] array (there are three other overloads). We also send FillPolygon() the topBrush to define the fill color. After the roof has been drawn, we call DrawPolygon() to draw the same polygon using a black pen. We do this to provide an outline for the roof plate for this individual column. Outlining the roof plate enhances the perspective view and makes the information represented by the slice easier to comprehend visually.

Next we draw the front wall. To do this, we define Point[] array polygonPoints2[] as {p2, p2Base, p3Base, p4}, then call FillPolygon(), sending the lighter colored sideBrush. After the front wall has been filled, we must redraw the roof line connecting points p2 and p3, which is erased by the FillPolygon() call.

The loop proceeds until columns representing all the data points in the slice have been painted (roof and front wall). The last required action is to paint the right edge of the slice, which is defined by the final set of points p3, p3Base, p4Base, and p4. Again, we call FillPolygon() with the lighter-colored sideBrush to color the wall, and then DrawPolygon() outlines that edge of the slice to make it more easily distinguishable from the adjacent slices.

And there it is: a powerful, flexible tool for drawing 3D Slice plots, coded in about 250 lines of C# (including comments and lines left blank to enhance code readability)!

3D Slice Plot Input Data File Structure

The input data file structure used for 3D Slice plots is the same format that is used for multi-line plots. Refer to the file structure detailed above if you would like to generate data files that can be read into DataGridGDI.exe and used to generate 3D Slice plots.

3D Surface Plot Component

As part of the 3D Slice plot coding, we developed code to draw a kind of surface. The upper faces of each slice in that plot are in fact a surface consisting of a series of "plates" where the width of the surface is the width of a single plate, and the length of the surface is the length of the slice along the X axis. Given this introduction to methods for drawing surfaces composed of plates that are displayed with perspective, we can now create a more general 3D Surface plot function.

In this case, we assume the input data represents a type of object (for example, a geographic topography). If the input data represents an object, then we need to be careful to represent the object accurately from all the points of view we will provide using the tilt and rotation angle controls. That is, for a 3D Surface plot, we want to apply geometric equations related to perspective in a much more rigorously scientific manner. Just as in the case of the pie chart, when the object is tilted, we want it's features to appear on the screen exactly as they would appear if we were holding the object before our eyes and slowly tilted it. The same applies for rotation. What is displayed by the graphics should match what we would see if we were rotating a physical object of the same shape as we held it before our eyes.

To see an example 3D Surface plot, select plot type "3D Surface", data file "Surface11.txt", tilt 20, rotation 30, and click the "Plot" button. The form window should look like this:

Here we are plotting an 11-by-11 grid of data points. The actual data points are located at the intersections of the lines drawn across the shape. The lines connect adjacent data points.

To see different 3D renderings of the data, you can select different values for tilt and rotation, and re-click "Plot". A useful way to get a good feel for the shape being presented is to select a single value for tilt, then slowly vary the rotation, or select a value for rotation and slowly vary the tilt.

3D Surface Plot Code

Although we need much greater accuracy of perspective for the 3D Surface plot, by applying some limitations we can once again reduce the complexity of both the geometric equations and the code necessary to create the three-dimensional rendering. For simplicity, the surface3DPlot() method generates the three-dimensional surface graphics using the following assumptions and limitations:

  • the input data is in the form of a square grid of values.
  • the height of the surface in the Z direction is not correlated with unit lengths in the X and Y directions.
  • all data sets will be scaled to the same maximum height in the Z direction.
  • the surface can only be tilted between 0 and 90 degrees, where the tilt involves the surface area closest to the viewer tilting down while the surface area farthest from the viewer tilts up.
  • the surface can only be rotated between 0 and 90 degrees, and rotation is in a clockwise direction.
  • rotation is applied first, then tilt is applied.
  • the perspective rendering will not include a "vanishing point" (that is, lines that are parallel on the surface will be drawn parallel in the graphics).

It may seem like this amount of assumption and limitation will result in a surface plot that hardly looks realistic. But this is not the case. If you study the 3D graphics in many computer and video games, you will find that assumptions and limitations of this type are regularly employed. The reason is simple: the graphics still look very realistic, but the rendering requires a much smaller set of code, resulting in the very fast completion of graphical displays that is required in games and other software that simulates three-dimensional environments.

The first actions undertaken by surface3DPlot() are conversion of the tilt and rotation angles into radians and normalization of the input data values (which will be plotted as the Z heights above the X/Y plane). The normalization computes the scaling based on the absolute value of the input data, so that a surface that includes large dips can be rendered with the same ease and accuracy as a surface that consists of all positive values.

private void surface3DPlot()
{
double
            DTOR = 3.14159 / 180.0;

//tilt with front edge going down,
            far edge rising
double tiltRadians = DTOR *
            (double)numericUpDownTilt.Value;

//rotation in clockwise
            direction
double rotationRadians = - DTOR *
            (double)numericUpDownRotation.Value;

//normalize data values
            to maximum of 100
double min=z[0][0];
double
            max=z[0][0];
for (int i=0; i<nY; i++)
{
for (int j=0;
            j<nX; j++)
{
if (z[i][j] < min)
{
min =
            z[i][j];
}
else if (z[i][j] > max)
{
max =
            z[i][j];
}
}
}

double maxAbs =
            Math.Max(Math.Abs(min),Math.Abs(max));
double scaleFactor =
            100.0/maxAbs;

for (int i=0; i<nY; i++)
{
for (int
            j=0; j<nX; j++)
{
z[i][j] *=
            scaleFactor;
}
}

The next steps are initialization of Point structures that will be used to draw the surface plates, and creation of the Graphic, a Pen for outlining each plate, and a SolidBrush for coloring the plate polygons.

Point p1 = new Point(0,0);
Point p2 = new
            Point(0,0);
Point p3 = new Point(0,0);
Point p4 = new
            Point(0,0);

Graphics g =
            CreateGraphics();
g.Clear(Color.LightGray);

Pen myPen =
            new Pen(Color.Black);
SolidBrush myBrush = new
            SolidBrush(Color.Aqua);

A few lines later, the mathematical calculations to apply the selected tilt and rotation are performed. The position of each point in the data grid after it has been tilted and rotated is stored in the Point[] array rotatedPoints[]. Once again, it is not critical to understand the actual equations. In effect, the equations apply trigonometric functions to compute the geometric "projection" of the three-dimensional shape onto the plane of the Windows form. First the position of each data point is computed after the shape is rotated about its center point on the X/Y plane by the number of degrees selected by the user. Then, the position of each data point is calculated with the selected tilt angle applied to the shape, assuming that the tilt axis passes through the center of the shape and is at a right angle to the point of view after the rotation has occurred. It sounds complicated, and it is. Fortunately, the code itself is not that bad:

Point[][] rotatedPoints = new
            Point[nX][];
for (int i=0; i<nX; i++)
{
rotatedPoints[i]
            = new Point[nY];
}

int xLeft = 200;
int xRight =
            450;
int xTot = xRight - xLeft;
int xCenter = (xLeft +
            xRight)/2;

int yTop = 50;
int yBot = 300;
int yTot =
            yBot - yTop;
int yCenter = (yTop + yBot)/2;

//calculate
            rotated point positions, and apply tilt to Y values
double
            sinTilt = Math.Sin(tiltRadians);
double cosTilt =
            Math.Cos(tiltRadians);

for (int iY=0; iY<nY;
            iY++)
{
double baseY = (double)(iY*yTot)/(double)(nY-1) -
            (double)yTot/2.0;
for (int jX=0; jX<nX; jX++)
{
//rotate
            all data points about the X/Y axis center point
double baseX =
            (double)(jX*xTot)/(double)(nX-1) -
(double)xTot/2.0;
double
            radius = Math.Sqrt(baseX*baseX + baseY*baseY);
double angle =
            -Math.Atan2(baseY,baseX);
double newAngle = angle +
            rotationRadians;
if (newAngle > 3.14159)
{
newAngle -=
            3.14159*2.0;
}
rotatedPoints[jX][iY].X = (int)(radius *
            Math.Cos(newAngle)) +
xCenter;

//apply tilt to the Y
            values
rotatedPoints[jX][iY].Y = -((int)(radius *
            Math.Sin(newAngle) *
sinTilt + z[jX][iY]*cosTilt)) +
            yCenter;
}
}

Finally, we plot the surface, starting with a row of plates we know cannot be in front of any other row of plates. We know this because we purposely limited the rotation angle range and the way tilt is applied to force this to be the case. Since we know an order in which the surface plates can be plotted such that the plate we are currently drawing will never be behind a plate that has already been drawn, drawing the surface can be performed in a very simple double loop. The first loop, for (int iY=0; iY<nY; iY++) begins with the first Y value (the upper back edge of the grid) and loops through all the Y values (moving toward the front edge of the grid). The inner loop, for (int iX=0; iX<nX; iX++) loops across the X values for each Y value.

Once again, we identify the four pixel locations that define the plate to be drawn (equivalent to the "roof" plates in the 3D Slice plot). These points are loaded into the Point[] array polygonPoints[], which is sent to the FillPolygon() method along with myBrush. After the polygon has been filled, its outline is traced for perspective clarity and definition through a call to DrawPolygon().

After all the plates have been drawn, we are done.

3D Surface Plot Input Data File Structure

The input data for DataGridGDI 3D Surface plots is stored in text files with a simple format. The format is shared with the Color Mapping image plot type discussed in the next section. Again, if you generate your own text files using the format and add the file name to the comboDataFile Items collection, you will be able to use the application to generate 3D Surface plots using your data. Note that the 3D Surface is currently limited in size to a maximum 50-by-50 data grid.

The 3D Surface plot input data file consists of a file type identification line consisting of the word "Image", a line listing the number of values along the X or Y axis, and lines that list each floating point value. The file structure details are as follows:

Line | Data Type | Description
1 | string | file type identifier: "Image"
2 | int | gridSize: number of values in the X and Y directions (up to 50)
3 to gridSize*gridSize+2 | float | Z values

Here is a printout of a portion of the Surface11.txt input data file, which is the example case displayed above:

Image
11
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
…

Color Mapping Image Plot Component

Sometimes the number of data values that need to be presented far exceeds not only what can reasonably be displayed in a table, but even what can reasonably be displayed in the types of plot presented in the previous sections. Consider the case where a scientific instrument is gathering vast quantities of data. The data may be stored as a large array of floating point values, with dimensions of hundreds of elements along the X and Y axes. In this case, attempting to draw even a 3D Surface plot might be fruitless, because the number of pixels per plate could be too small for the individual plates to be distinguishable on a standard-sized video monitor.

A solution that works for this case is to represent the data as an image, with each value represented in a single pixel using a color scale. One name for this type of data representation is a "Color Mapping Plot". Sometimes this type of "image" is also called a "false color" plot, because it can be used to present data from sensors that measure emissions that are invisible to human beings (such as infrared radiation) in a form that looks like a visual image. Since the colors on the plot are from the "visible spectrum", while the data is from an invisible portion of the spectrum (for example, the infrared), the plots are said to display "false color".

Regardless of what we call them, these plots can be very useful, and it is not that difficult to generate them using C# and the GDI+ classes. To see examples of Color Mapping plots, select the plot type "Color Mapping", data file "Image256.txt" or "Surface11.txt", and click the "Plot" button. For the Color Mapping plots, the tilt and rotation settings are not used. Here is what you should see as the result when the Image256.txt file is displayed:

This type of plot is very useful in cases where a very large amount of data must be represented. In this case, we are presenting a 256-by-256 point data grid. With this many points, many other types of graph cannot be used-unless you have a very large high-resolution monitor! This type of plot is useful for many scientific imaging systems, which can consist of a grid of tiny sensors, each of which records an intensity value (perhaps infrared light, ultraviolet light) or some other physical parameter.

For an interesting comparison, try generating a Color Mapping plot for the Surface11.txt file that was used in demonstrating the 3D Surface plot. The tiny 11-by-11 pixel image you see next to the top of the color bar can be seen to represent the same mountains and dips you see in the 3D Surface plot.

Color Mapping Image Plot Code

The colorMappingPlot() method consists of sections that initialize graphics objects, normalize the input data values, create a 128-color scale, and draw the color mapped image. Most of this type of coding has been presented in the discussion of the previous plot types. However, the creation of a color scale is something new.

To create the 128-color scale that will represent the values in the data grid, we first define the Color[] array scaleColors[]. The goal is to select 128 colors which when viewed in order to produce a kind of rainbow spectrum, that can be seen as suggesting high, medium, or low values. A commonly used method is to have red represent high values (because we associate high levels of heat with red-fire is red) and blue represent low values (we sometimes see bluish tints in ice). This color coding is sometimes used on water faucets, with red representing the hot water and blue representing the cold water.

To generate a spectrum of colors with red at one end of the spectrum and blue at the other, we can use the Color.fromArgB() method. The fromArgB() method creates a Color structure based on integer values corresponding to ARGB color components. ARGB stands for "alpha, red, green, blue". The "alpha" value is a measure of the intensity of the selected color components. To generate our color spectrum, we start out by setting the red RGB component to its maximum value (255) and the green and blue components to their minimum (0). This produces a pure red color. For the next 42 colors, we slowly increase the green RGB component to its maximum value, yielding the yellow color that is seen about 1/3 of the way down from the top of the color scale. For the next 43 colors, we reduce the red RGB component toward zero. This produces the pure green color seen about 2/3 of the way down from the top of the color scale. Finally, for the last third of the scale, we keep the red RGB component at 0, the green component at 255, and we increase the blue component to its maximum, producing the light bluish color at the bottom of the color scale.

Color[] scaleColors = new Color[128];
for
            (int i=0; i<128; i++)
{
if
            (i<43)
{
scaleColors[i]= Color.FromArgb(255, i*6,
            0);
}
else if (i<86)
{
scaleColors[i] =
            Color.FromArgb(255-(i-43)*6, 255,
            0);
}
else
{
scaleColors[i] = Color.FromArgb(0, 255,
            (i-86)*6);
}
}

g.FillRectangle(curBrush,150,50,40,256);
for
            (int i=0; i<128; i++)
{
curBrush.Color =
            scaleColors[i];
g.FillRectangle(curBrush,150,50+i*2,40,2);
}

Pen
            myPen = new Pen(Color.Black,1);
SolidBrush labelBrush = new
            SolidBrush(myPen.Color);
Font labelFont = new
            Font("Arial",10);

g.DrawString(max.ToString("E3"),labelFont,labelBrush,
            132.0F,
            35.0F);
g.DrawString(min.ToString("E3"),labelFont,labelBrush,
            132.0F, 307.0F);

The DrawString() method is called to label the minimum and maximum values for the color scale, and the image is drawn by converting each data value into the scale range of 0 to 127 and plotting the pixel using the color stored in that scaleColors[] index position.

Color Mapping Plot Input Data File Structure

The input data file structure used for Color Mapping plot input data is the same format that is used for 3D Surface plots. The only difference is that for Color Mapping plots the data value dimension maximum is a 300-by-300 grid. Refer to the file structure detailed above if you would like to generate data files that can be read into DataGridGDI.exe and used to generate Color Mapping image plots.

Using the DataGridGDI.exe Application for Other Data Sets

DataGridGDI.exe comes with several example input data files that illustrate the various plot types. However, the intent was also to provide a convenient means for the reader to use the application for plotting other data sets as needed.

As the application exists in the case study, the names of any files that can be used as inputs for plotting must be listed in the comboDataFile Items collection. If you can format your grid of data into one of the formats listed above, place the file into the directory where DataGridGDI.exe is located, add the file name to the comboDataFile Items list, rebuild the application, and select the desired plot type, select your file, and click "Plot". If the file is properly formatted and the number of elements is within the specified boundaries, you should see a graphical representation of your data and be able to apply the tilt and rotation settings exactly as you can do with the examples that come with the case study.

Limitations and Further Work

Clearly, the possibilities for using C# and GDI+ to represent grids of data have just barely been touched upon by the DataGridGDI application and this case study. Some standard capabilities, such as triggering an OnPaint() method to automatically redraw the plot as needed, were left out of the current program for the sake of simplicity. Clearly, in a "production" version, we would surely want to provide much more flexible and generic capabilities for reading input data, including datasets and XML.

Development of a full toolkit for representation of data grids could surely occupy an entire book! For each plot type that was presented, a great many extensions are possible, and many would be highly desirable.

Conclusion

We have utilized many of the core GDI+ graphic structures, classes, and methods to present data using pie charts, multi-line plots, 3D slice plots, 3D surface plots, and color mapping image plots. The developed application provides a means for the reader to apply the developed graphing functions to arbitrary data sets, as long as the data sets can be converted into simple text files of the type specified in the case study.

Case Study Information
Author | Kevin Farnham
Editor | Adam Ryland
Reviewers | Sean Schade, Phil Sidari, Matthew MacDonald

If you have any questions or comments about this case study, or suggestions for further pieces, please contact the technical editor. If you wish to talk about the case study with other subscribers, please visit our forum here.


An Open Invocation

An Open Invocation
http://www.csharptoday.com/content.asp?id=1846
Article Rating: 5
Published on June 17, 2002
Abstract
The introduction of the .NET framework and the arrival of C# have made great benefits available to developers. However, there will be many occasions where developers need to bridge the gap between unmanaged code and the .NET environment. In this case study, Roger Rowland examines exactly what is necessary to employ P/Invoke (Platform Invocation Services) to allow use of the OpenGL 3D graphics library from C#. The end result is an appreciation of the possible pitfalls of interoperation, an introduction to 3D graphics programming, and the development of an OpenGL Web Service delivering tailor made 3D images on demand.
Case Study

The introduction of the .NET framework and the arrival of C# have undoubtedly made great benefits available to developers. Many things, which were once difficult or problematic, are now much easier and this allows developers to concentrate more on functionality, which is a good thing.

However, .NET in no way replaces decades of software development. There will be many occasions where developers need to bridge the gap between unmanaged code and the .NET environment if systems are to move forward in an incremental manner – not many can afford to redevelop all of their systems from scratch and, even if they could, they’d be unlikely to do so without some compelling reasons. There is a fine line to be walked here. Too much reliance on legacy code and platform dependence and you risk losing out on the flexibility that may be available in the future as .NET becomes more established on other platforms and operating systems. Too little value placed on legacy code and you risk redevelopment costs for largely intangible benefits. Not an easy decision, and one that cannot be reached on economic grounds alone. Microsoft of course realised that this would be an issue and so has provided methods whereby this gap may be bridged in a technical sense, although with few guidelines about where this is appropriate and how it should be managed. These methods are of course COM Interoperability and Platform Invocation Services (P/Invoke).

One downside of using Platform Invocation Services – which effectively ties an application down to a particular environment – is that any future moves to make the .NET framework platform-independent might compromise the portability of these applications. The irony of this is that in some areas where cross-platform portability is already established, there is no other way to access these features from within .NET. One of these areas is OpenGL.

And Your Point Is ….. ?

By using OpenGL as a practical example, this case study will demonstrate the technical issues involved in using P/Invoke and touch on areas where design decisions can help to minimise the negative aspects referred to above. In the process, we will not only learn a little about OpenGL, but will develop two applications: the first a "simple" OpenGL Windows Forms application; the second, an enhancement of this to an OpenGL Web Service able to process requests for rendering 3D text and to supply these as dynamic bitmaps or jpegs directly to a browser. The overall result will be an appreciation of the obstacles that you as a developer will inevitably face when migrating existing systems to .NET, together with a useful application combining the powers of .NET and OpenGL which couldn’t feasibly be done without either.

System Requirements

This system requires Visual Studio .NET V1.0 or later running on Windows 2000 or Windows XP. The case study assumes that the reader has a good understanding of .NET in general, and C# in particular. A background in Windows GDI would be beneficial, as would any exposure to OpenGL although neither is essential.

Case Study Structure

The case study begins with an overview and history of OpenGL, followed by a comparison with DirectX and a description of the Windows OpenGL implementation.

After briefly discussing the impact of .NET, the first of the two sample applications is developed in some detail. The development of the Web Service is described by building on from the first application, after which some concluding remarks put the whole exercise in context.

  • What is OpenGL?
  • A Very Brief History
  • How Does it Work?
  • OpenGL vs. DirectX
  • Windows OpenGL Implementation
  • Keeping Things in Context
  • OpenGL and .NET
  • Sample Applications
  • OpenGL in a Windows Form
  • Grabbing a Device Context
  • Pixel Formats
  • Building a Wrapper
  • Size Doesn’t Matter?
  • Creating a Rendering Context
  • Lights, Cameras, …
  • Material Properties
  • Display Lists
  • Generating 3D Characters
  • How OpenGL Maps to a Window
  • Painting Pictures
  • And, finally …
  • An OpenGL Web Service
  • Rendering to a Bitmap
  • Outline Design
  • Implementing the Web Method
  • BMP to JPG
  • The Consumer
  • Limitations
  • Conclusion

What is OpenGL?

In almost all literature introducing OpenGL, you will be told that OpenGL is "a software interface to graphics hardware". More specifically, and unlike GDI and GDI+, it is primarily concerned with 3D graphics. It is very fast, even in its pure software implementations, and most modern video cards support some level of OpenGL hardware acceleration. It provides a way for application developers to produce realistic 3D images in real time with a quality close to that produced by ray tracers. Anyone who has played Quake or Doom will have witnessed OpenGL in action and have some appreciation of the possibilities. In its early days, OpenGL was used to produce some of the special effects in the film Terminator Two: Judgment Day.

A Very Brief History

OpenGL is an open standard (the "GL" stands for Graphics Library) and evolved from the earlier IRISGL developed by Silicon Graphics Inc. (SGI) for use on their own graphics accelerated hardware. OpenGL resulted from SGI’s desire to improve the portability of IRISGL, and the specification of OpenGL is now controlled by the OpenGL Architecture Review Board (ARB). The ARB’s founding members include SGI, DEC, IBM, Intel and Microsoft.

The OpenGL ARB was formed in the early 1990’s. Version 1.0 of the OpenGL specification was released in 1992, and Microsoft first included a software implementation of OpenGL with Windows NT 3.5, closely followed by Windows 95. Now there is an OpenGL implementation included with all versions of Windows from Windows 98 upward.

How Does it Work?

OpenGL itself is a procedural rather than a descriptive language. So, instead of the developer describing the end result he wants to produce, he describes in detail the steps required to produce it. This might include describing 3D vertices, colours, surface properties, lighting conditions, camera position, and so on. OpenGL will then use this information to render a 3D scene onto a given two-dimensional display. Given the rendering speeds possible with accelerated graphics cards, OpenGL is ideal for producing 3D animations. It is also possible to use OpenGL functions to "project" a point on the screen (e.g. from a mouse cursor) back into the 3D model to identify what the user is pointing at. This opens the door for an interactive 3D graphical interface that has many exciting possibilities.

The OpenGL API consists of more than 120 commands and functions which allow the developer to specify exactly the objects required in his scene. As well as functions describing graphics primitives (points, lines, polygons), there are some powerful functions which handle texture mapping and 3D surface rendering although many of these require a deeper understanding of computer graphics. The functions all begin with a " gl " prefix and were designed for portability to be primarily called via C or C++. For example, the following code snippet shows the function calls required to describe a red rectangle with corners at (0.25, 0.25, 0.0) and (0.75, 0.75, 0.0).

glColor3f(1.0f, 0.0f, 0.0f);
glBegin(GL_POLYGON);
glVertex3f(0.25f, 0.25f, 0.0f);
glVertex3f(0.75f, 0.25f, 0.0f);
glVertex3f(0.75f, 0.75f, 0.0f);
glVertex3f(0.25f, 0.75f, 0.0f);
glEnd();

Don’t worry too much about these functions for now. There are numerous examples and tutorials on the official OpenGL web site at http://www.opengl.org/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq that demonstrate most of the available features (and there are plenty!). For the purposes of this case study, we will be using only a small subset of the OpenGL API and every function will be explained as we use it.

OpenGL vs. DirectX

We couldn’t go too much further without briefly mentioning DirectX. On reading the above, you might see that there’s a big overlap between OpenGL and Microsoft’s DirectX API’s. While it’s true that you may also use elements of DirectX (i.e. Direct3D) to perform some of the same functions as OpenGL, DirectX is really a much wider set of API’s, specifically designed to allow games developers or other multimedia application developers to have low level access to hardware. This includes not only graphics, but also sound effects, music, input devices (joysticks, etc.), and network facilities for multi-player applications.

If you frequent any of the graphics newsgroups, you’ll find plenty of people willing to give their opinion of why OpenGL is better than DirectX or vice versa. Personally, I think this is not really a useful debate and so won’t be entering into it here. Suffice it to say that, as far as Microsoft is concerned, both API’s are supported by the operating system and, as far as video card vendors are concerned, both technologies are often catered for in terms of hardware acceleration. I have found OpenGL much easier to learn than DirectX, and this is helped by the large base of freely available sample code readily accessible over the internet.

Also, at the time of writing, as with OpenGL, there is no inbuilt support for using DirectX from within .NET. Hmmm...more later.

Windows OpenGL Implementation

In the Windows environment, the OpenGL API is implemented in the library opengl32.dll. This contains all of the " gl " functions. There is a second library, glu32.dll, which provides a number of utility functions that screen the developer from some of the intricacies of 3D graphics programming by wrapping low-level OpenGL calls into more friendly functions. These are provided with every OpenGL implementation and the functions all have the prefix " glu ". Together, these DLL’s encompass the whole of Microsoft’s implementation of OpenGL that automatically manages interaction with any hardware accelerated video card present. If no hardware acceleration is available, this software handles all the functionality.

So, the OpenGL API is an open standard. This means that a developer on any platform will use the same " gl " functions and parameters to produce the same (or similar) result. However, at some point it becomes necessary for an OpenGL application to interact with the operating system in order to display an image. OpenGL itself does not provide any functions for window management, user interaction, or file I/O. These are performed by a small set of functions that are developed specifically for the particular host environment and serve as the bridge between OpenGL and the host system. In Windows, these functions are known as the "wiggle" functions (due to the prefix " wgl ") and are also contained within opengl32.dll, although the prototypes are defined in the wingdi.h header file. There are a couple more genuine gdi32.dll functions which have specific OpenGL features, but these will be explained when we use them.

Keeping Things in Context

All drawing in OpenGL revolves around an OpenGL rendering context. This is different to a device context (DC) although it is created from one in order to ensure that a compatible image is produced for eventual display.

Before any OpenGL calls can be made, the calling thread must have a current rendering context. This is achieved by the two functions wglCreateContext() and wglMakeCurrent(). The first takes a handle to a device context as a parameter and returns a handle to a new rendering context. The second attaches a given rendering context to the current thread and associates this with a given device context (not necessarily the same one it was created from, although they must be compatible).

It is possible to create any number of rendering contexts, although only one of these may be made current in any given thread. All OpenGL calls will be implicitly directed to the calling thread’s current rendering context. Also, a rendering context may only be current to one thread at any one time.

If you’ve followed this, you’ll see that an application may have multiple rendering contexts, all producing images for different windows. As the OpenGL library is thread-safe, it is also possible to have different threads using different rendering contexts simultaneously. From this basis, things can get very complicated very quickly.

OpenGL and .NET

Let’s just pause for a while and take a quick recap on what we have found so far. Here are two highlights and one observation:

  • There is no inbuilt 3D graphics support in the Framework Class Library (FCL)
  • OpenGL provides an established portable standard for 3D graphics modeling
  • The need for high-impact, multimedia interfaces make 3D graphics desirable

Taking the last point first, this is pretty important. Everything about today’s applications and web sites screams out "presentation". Nice graphics is a good thing. Why is there no support in the latest development environment for one of the fastest growing technologies? Will Microsoft expect developers to use third-party tools to produce static images or animations? Will there really be no need for any interactive 3D interfaces in tomorrow’s applications? Will we always need to use legacy code to achieve these effects? I think not. Whilst developments may prove me wrong, I suspect that a future .NET release will include some 3D graphics capability in the FCL, possibly a wrapper to DirectX or a re-engineered version of same. Don’t quote me on that yet though.

These are all thoughts that went through my mind when considering this particular issue. You will surely have similar thoughts about whatever "legacy" features you need to implement. Does .NET provide an alternative? If not, is it genuinely useful? If so, might it be there in the future? If so, in what form? Should I write one and sell it? In the above case, obviously I haven’t got a clue. I don’t rub shoulders with Bill Gates and I have nothing more than an educated guess to go on, but the point is that these are the questions which should influence your decision on how you design systems for interoperability, erring on the side of "don’t do it at all unless you have to".

Now take the second point above – OpenGL already provides a standard API. So, if a future .NET does include 3D support, it will inevitably be object oriented. OpenGL is not (there is an add-on called Open Inventor which wraps OpenGL in C++ classes but it’s not a standard). So the next question is, if we’re going to develop an interface to OpenGL from C#, do we re-engineer an OO interface? If so, we’re second guessing what the future may provide and risk getting it wrong - then this will make our applications more difficult to port to the "new" .NET should it subsequently appear. If we just expose the non-OO API to our C# applications, we lose the benefits that might have been gained from an OO environment.

This is not a straightforward decision and there are no right answers. However, the consequences of getting it wrong could be costly in development terms. You need to think carefully about each individual requirement before committing to a development path. It’s not appropriate to adopt a "general" migration policy.

In this case, and to avoid agonising too long over the emerging dilemmas, I have decided that it makes more sense to just expose those features of the OpenGL API that I need than to develop a complete interface or to re-engineer a new one. The ability to take advantage of existing sample code is a big plus for opting to mirror the existing API, and by just exposing the functions I’m using I automatically minimise any future redevelopment. It is unfortunate that this means the OpenGL interface will not be object oriented as this is incompatible with the fundamental design of the C# language. If you’re interested in looking at the way others have tackled the issue, there is an open source development of a C# to OpenGL interface at http://sourceforge.net/projects/csgl/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq which takes a slightly different approach.

So, having made this policy decision – do as little as possible to just make it work in C# and still look like the real API – we need to stop talking and start coding to make sure the above argument is not totally academic. The first step will be to make a simple Windows Forms application that will display an OpenGL 3D image using C#.

Sample Applications

OpenGL in a Windows Form

The user interface for this will be pretty basic - we just need to prove the concept.

The figure above shows the form design – a Panel that will be used to display the image, and a Button to close the form. You can’t really get much more basic than that!

What we’ll do is to create this form, intending to display the OpenGL image in a Panel control (we could use the whole window or indeed any control which exposes a device context) and implement the basic functionality first.

Once we’ve done that, we’ll design the class to wrap the OpenGL functions that we’ll need to generate an image then hook these into the appropriate places in the form.

So, create a new Windows Form application, call it OGLWindow, rename the generated Form1 class to OGLWindow and create the UI elements as follows:

Control | Property | Value
Form1 | (Name) Size Text | OGLWindow 360, 320 OpenGL Window
panel1 | (Name) Anchor BackColor BorderStyle Font Location Size | openglPanel Top, Bottom, Left, Right Window Fixed3D Arial, 12pt 8, 8 336, 200
button1 | (Name) Anchor Location Text | closeButton Bottom 140, 248 Close

Add an OnClick event handler, closeClicked, for the Button control to allow the form to be closed:

private void closeClicked(object sender,
            System.EventArgs e)
{
this.Close();
}

That’s the form done for now. Next we need to find the definitions of the functions we need to import and build a C# wrapper.

The OpenGL include files are called gl.h and glu.h and in a standard installation of Visual Studio .NET will be in the VC7\PlatformSDK\Include\Gl folder. These contain all of the types, defines, and prototypes for the OpenGL API functions we need. For the "wiggle" functions and other GDI stuff, we also need to consult wingdi.h from the VC7\PlatformSDK\Include folder.

Grabbing a Device Context

The first thing we’ll need to do is to create a rendering context based on the device context of the openglPanel control. Both the rendering context and the device context are defined as Win32 handles so we can define member fields for holding these as IntPtr objects. While we’re at it, we’ll also include private fields for holding a reference to the Panel’s Graphics object and to contain the text string we’re going to render:

private Graphics openglGraphics;
private IntPtr hDC; // handle to device context
private IntPtr hRC; // handle to rendering context
private string words3d;

These can all be initialised in the form’s constructor once the UI elements have been created:

public OGLWindow()
{
// Required for
            Windows Form Designer support
InitializeComponent();

hDC =
            new IntPtr(0);
hRC = new IntPtr(0);
words3d = "C#
            Today";
openglGraphics = openglPanel.CreateGraphics();
hDC =
            openglGraphics.GetHdc();
...

Now, if you consult the .NET documentation and refer to the CreateGraphics() and GetHdc() methods, you’ll see that each GetHdc() must strictly be paired with a ReleaseHdc() (like we would have in C or C++). Experiments indicate that ReleaseHdc() must be called from the same Graphics instance used for GetHdc() or it will throw an exception, and yet the documentation clearly states that the Graphics object returned by CreateGraphics() is only valid for the current Windows message. That’s fine for the Graphics object but, as we really need to hold onto this hDC for longer than a single Windows message (and there’s no reason why we shouldn’t), there would appear to be no way to obey all the rules and release everything cleanly. This means we either ignore the documentation and let the garbage collector sort it out or we assume the documentation is in error and do our best to clean up properly.

This is not ideal as it still leaves us in doubt as to whether or not this code will survive the next .NET service release. It’s really only arisen because we’ve started to dip into areas where we shouldn’t really be anymore (i.e. grabbing handles to Windows objects), and possibly trying to shortcut good design principles (e.g. subclassing a Panel to create an OpenGLPanel control), but to develop a simple prototype, we’re left with little choice. Taking the documentation about the CreateGraphics() method as being genuine means that we would need to release this reference when the constructor ends, and therefore would also need to release the hDC at this point. Whilst this still works, the overhead of re-acquiring and releasing the hDC during painting causes noticeable performance degradation. The important thing to do is to document any such instances carefully so that they can be identified as potential hot spots should everything stop working in the future.

Pixel Formats

Having retrieved a device context compatible with the window we’re using for display, the next actions we need to take are:

  • Set the pixel format of the device context
  • Create a rendering context based on the device context
  • Make the rendering context current

Once we’ve done this, we should be ready to start calling OpenGL API functions from the main thread.

We need to set the pixel format of the DC before creating the rendering context as this defines an internal pixel format compatible with that in the DC at the time it’s created. There’s no way to change this afterward. Also, for a window device context (as opposed to a memory DC), it’s only allowable to set the pixel format once.

The functions required to set the DC pixel format are defined in wingdi.h and are called ChoosePixelFormat() and SetPixelFormat(). The first of these decides the most suitable format to use based on the format we request in the passed PIXELFORMATDESCRIPTOR and the capabilities of the physical device. So, if we request a 32 bit colour depth but the graphics card only supports a maximum of 24 bits, we’ll get returned an index to a 24 bit format instead. Whatever index is returned is passed to SetPixelFormat() to set that format for the given device context. The content of the PIXELFORMATDESCRIPTOR is important and I’ll cover that in more depth in a minute.

These gdi calls are logically separate from the OpenGL API so it makes sense to wrap these in a different class. We’ll also define a namespace of Wrox.OpenGL for all of these definitions and reference this in a using clause in the OGLWindow class. So...

Building a Wrapper

Add a new Class item to the OGLWindow project and name it OpenGL.cs. Remove all of the generated code and replace it with the following:

using System;
using System.Runtime.InteropServices;

namespace Wrox.OpenGL
{
public abstract class GDI
{
// GDI function wrapper goes here
}
public abstract class GL
{
// OpenGL function wrapper goes here
}
}

This is where we’ll define the two wrapper classes – GDI and GL. The first will handle anything we need to call from gdi32.dll which we can’t get in some other way via the FCL, and the second will encapsulate any types or functions we need from the OpenGL API. Note that we’re also using System.Runtime.InteropServices to ease access to the Platform Invocation Services classes.

Before we go further, add the following lines to the top of the code in the OGLWindow class:

using System.Runtime.InteropServices;
using Wrox.OpenGL;

This gives us easy access to P/Invoke from the form class and will allow us to reference any wrapped gdi or OpenGL functions by just prefixing them with GDI. or GL. respectively. This way, our application code will look as near to existing OpenGL calls as possible. For instance, the earlier rectangle example could eventually be coded in C# as:

GL.glColor3f(1.0f, 0.0f, 0.0f);
GL.glBegin(GL.GL_POLYGON);
GL.glVertex3f(0.25f, 0.25f, 0.0f);
GL.glVertex3f(0.75f, 0.25f, 0.0f);
GL.glVertex3f(0.75f, 0.75f, 0.0f);
GL.glVertex3f(0.25f, 0.75f, 0.0f);
GL.glEnd();

So, to get back to our pixel formats in the GDI class, by looking at the prototypes for ChoosePixelFormat() and SetPixelFormat() in wingdi.h, we see we will need to define the PIXELFORMATDESCRIPTOR structure and some constants or enumerations for initialising various flags and attributes therein. The structure itself is defined in the header file thus:

typedef struct tagPIXELFORMATDESCRIPTOR { //
            pfd
WORD nSize;
WORD nVersion;
DWORD dwFlags;
BYTE
            iPixelType;
BYTE cColorBits;
BYTE cRedBits;
BYTE
            cRedShift;
BYTE cGreenBits;
BYTE cGreenShift;
BYTE
            cBlueBits;
BYTE cBlueShift;
BYTE cAlphaBits;
BYTE
            cAlphaShift;
BYTE cAccumBits;
BYTE cAccumRedBits;
BYTE
            cAccumGreenBits;
BYTE cAccumBlueBits;
BYTE cAccumAlphaBits;
BYTE cDepthBits;
BYTE cStencilBits;
BYTE cAuxBuffers;
BYTE iLayerType;
BYTE bReserved;
DWORD dwLayerMask;
DWORD dwVisibleMask;
DWORD dwDamageMask;
}
            PIXELFORMATDESCRIPTOR;

The dwFlags item is a set of bit flags that define the properties of the pixel buffer. This is important as one of these flags (PFD_SUPPORT_OPENGL) is used to specifically request a pixel format that supports OpenGL. Other flags of note are PFD_DRAW_TO_WINDOW, which we need for this application, PFD_DRAW_TO_BITMAP which we’ll need later, and PFD_DOUBLEBUFFER which is required if we’re providing any animation in our scenes as it allows OpenGL to do all drawing to the back buffer while a fast gdi function, SwapBuffers(), effects the display to screen. What this means in practice is that the system allocates a second screen buffer (the "back buffer") – usually in memory on the video card – and OpenGL always renders output to this "hidden" buffer. The SwapBuffers() function instructs the video card to switch the display to the content of the back buffer, which it can do extremely quickly. The memory which held the previously displayed image then becomes the back buffer for the next drawing operation. As this happens entirely in the video hardware and involves no copying of memory, this operation is faster even than the traditional BitBlt() used by GDI based animation techniques.

For this dwFlags field, we follow the framework guidelines and create an enumeration instead of defining constants. We’ll also use the System.FlagsAttribute custom attribute type to take advantage of the framework’s ability to format the output for debugging, etc. So, the enumeration PixelFlags is defined within the GDI class as follows:

[Flags]
public enum PixelFlag : uint
{
PFD_DOUBLEBUFFER = 0x00000001,
PFD_STEREO = 0x00000002,
PFD_DRAW_TO_WINDOW = 0x00000004,
PFD_DRAW_TO_BITMAP = 0x00000008,
PFD_SUPPORT_GDI = 0x00000010,
PFD_SUPPORT_OPENGL = 0x00000020,
PFD_GENERIC_FORMAT = 0x00000040,
PFD_NEED_PALETTE = 0x00000080,
PFD_NEED_SYSTEM_PALETTE = 0x00000100,
PFD_SWAP_EXCHANGE = 0x00000200,
PFD_SWAP_COPY = 0x00000400,
PFD_SWAP_LAYER_BUFFERS = 0x00000800,
PFD_GENERIC_ACCELERATED = 0x00001000,
PFD_SUPPORT_DIRECTDRAW = 0x00002000,
PFD_DEPTH_DONTCARE = 0x20000000,
PFD_DOUBLEBUFFER_DONTCARE = 0x40000000,
PFD_STEREO_DONTCARE = 0x80000000
}

The ability to redefine #define’s from Win32 header files as enumerations in C# provides us with an opportunity to enforce some degree of control over the use of our exposed unmanaged code in that we can use the compiler to verify that we at least have a value from the relevant enumeration in an argument to a function call, and discourages the use of hard coded numeric values instead. In a similar vein, we’ll define two more enumerations to be used by the PIXELFORMATDESCRIPTOR, one for iPixelType and one for iLayerType :

public enum PixelType : byte
{
PFD_TYPE_RGBA = 0,
PFD_TYPE_COLORINDEX = 1
}

public enum LayerType : sbyte
{
PFD_MAIN_PLANE = 0,
PFD_OVERLAY_PLANE = 1,
PFD_UNDERLAY_PLANE = -1
}

Defining the PIXELFORMATDESCRIPTOR is then a matter of defining a structure holding the matching data types for those defined in the header file, substituting the above enumerations where needed.

One important addition to any structures which are intended to be passed to Windows API functions is to specify the StructLayoutAttribute as LayoutKind.Sequential. In managed code, the CLR is free to reorder the fields within a structure for greater efficiency and this would obviously have disastrous effects if subsequently passed to an API function expecting a strict layout. In addition, some value types (so-called non-blittable types) have different representations in managed and unmanaged memory and need to be marshalled across the interoperation boundary.

By specifying LayoutKind.Sequential, we make sure that the structure in unmanaged memory is set out in the same order as its declaration in the managed type definition. For a struct, the C# compiler adopts this behaviour by default, but it is much safer to explicitly specify this sequential layout where we know it is required. If you subsequently redefine the struct as a class, the compiler will assume Layoutkind.Auto and this could end up as a hard-to-track-down bug.

We can now complete the definition of the fields in the PIXELFORMATDESCRIPTOR structure in the class GDI as follows:

[StructLayout(LayoutKind.Sequential)]
public
            struct PIXELFORMATDESCRIPTOR
{
public ushort nSize;
public
            ushort nVersion;
public PixelFlag dwFlags;
public PixelType
            iPixelType;
public byte cColorBits;
public byte
            cRedBits;
public byte cRedShift;
public byte
            cGreenBits;
public byte cGreenShift;
public byte
            cBlueBits;
public byte cBlueShift;
public byte
            cAlphaBits;
public byte cAlphaShift;
public byte
            cAccumBits;
public byte cAccumRedBits;
public byte
            cAccumGreenBits;
public byte cAccumBlueBits;
public byte
            cAccumAlphaBits;
public byte cDepthBits;
public byte
            cStencilBits;
public byte cAuxBuffers;
public LayerType
            iLayerType;
public byte bReserved;
public uint
            dwLayerMask;
public uint dwVisibleMask;
public uint
            dwDamageMask;
}

Note that all fields have public accessibility to allow full access to the complete structure and to simplify the definition. In this situation this is exactly what we want – the only areas where we might want to perform any validation have been implemented as enumerations so the compiler handles this for us.

Size Doesn’t Matter?

Now we hit another area requiring some thought. So far, there’s no constructor defined in the above structure, so all non-default fields would need to be set by the calling function. One of these is nSize, which needs to hold the size of the complete structure. In C or C++ we’d habitually use sizeof() to make sure this value is always correct and while there is a sizeof() function in C#, it can only be used in an unsafe code block.

The existence of unsafe code requires that the project is compiled with the /unsafe command line switch, and this disqualifies it from being verifiably safe in the CLR. This in turn means that there may be knock-on security implications. We’ll need a certain level of trust anyway to interact with unmanaged code via P/Invoke, but do we really want to deviate from the type-safe environment of the CLR unless we really have to?

We could hard-code the structure size in a constructor. We know what it should be (40 bytes) or can very quickly find out (I temporarily coded a sizeof() call and displayed the result in a message box!). We know it’s not going to change unless we mess around with the structure definition and we’d only need to do this in the unlikely event that the unmanaged structure is redefined in the operating system.

Neither of these options is wholly satisfactory. Personally, I would resist declaring to the CLR that my assembly is not verifiably safe unless it really needs to be, so I would hard code the value. This really goes against the grain after many years considering this to be very bad coding, and if something else in the code absolutely requires an unsafe block then I’ll change my mind. For now though, we’ll declare a constructor for PIXELFORMATDESCRIPTOR which will allow us to just set those fields of interest and leave the rest more or less hidden from our calling code.

The constructor looks like this:

public PIXELFORMATDESCRIPTOR(PixelFlags
            Flags, byte ColorBits,
byte DepthBits)
{
nSize =
            40;
nVersion = 1;
dwFlags = Flags;
iPixelType =
            PixelTypes.PFD_TYPE_RGBA;
cColorBits = ColorBits;
cRedBits =
            0;
cRedShift = 0;
cGreenBits = 0;
cGreenShift =
            0;
cBlueBits = 0;
cBlueShift = 0;
cAlphaBits =
            0;
cAlphaShift = 0;
cAccumBits = 0;
cAccumRedBits =
            0;
cAccumGreenBits = 0;
cAccumBlueBits = 0;
cAccumAlphaBits
            = 0;
cDepthBits = DepthBits;
cStencilBits = 0;
cAuxBuffers
            = 0;
iLayerType = LayerTypes.PFD_MAIN_PLANE;
bReserved =
            0;
dwLayerMask = 0;
dwVisibleMask = 0;
dwDamageMask =
            0;
}

The only things we really need to bother about (unless we’re intending to cater for 8-bit video cards and mess around with palettes!) are the pixel property flags, the colour depth, and the size of the z-buffer. The last of these could arguably be defaulted to 16 or 32 with no problems, but we’ll leave it parameterised for now. If you’re interested, the remaining fields are all described on MSDN although most of these are ignored by ChoosePixelFormat() and SetPixelFormat().

The final step in defining the GDI wrapper class is to add the definitions of the above functions:

[DllImport("gdi32.dll")]
public static extern int ChoosePixelFormat(IntPtr hdc,
ref PIXELFORMATDESCRIPTOR ppfd);

[DllImport("gdi32.dll", SetLastError = true)]
public static extern int SetPixelFormat(IntPtr hdc, int iPixelFormat,
ref PIXELFORMATDESCRIPTOR ppfd);

Some things to note here are:

  • The functions are defined as static so we don’t need to instantiate a GDI object to use them.
  • They’re also defined as external so there’s no function body.
  • The DllImport attribute is used to specify the DLL that contains the implementation.

Also, note that the DllImport attribute for SetPixelFormat() has a named parameter, SetLastError, which is set to true. This is very important when using P/Invoke to access Windows functions that were designed to report exceptions using SetLastError(). In C or C++, any error status would have been retrieved by calling GetLastError() immediately after the failing function. However, when using P/Invoke, the CLR may well call additional API functions before your code has a chance to get the last error status, and so this may be cleared or overwritten by the time you get it.

By using SetLastError = true in DllImport, the CLR caches the result from GetLastError() and allows this to be retrieved from the calling application via the GetLastWin32Error() method provided by the Marshal class in the System.Runtime.InteropServices namespace. This is why we added the using clause earlier.

Before we finish with the GDI wrapper class, we’ll add two more functions we’ll need later. One of these, SwapBuffers(), has already had a mention in passing and the other is SelectObject() that we need later to select a font into our device context. Define these functions as follows and then we can forget about the GDI class:

[DllImport("gdi32.dll")]
public static extern IntPtr SelectObject(IntPtr hdc, IntPtr hgdiobj);
[DllImport("gdi32.dll")]
public static extern int SwapBuffers(IntPtr hdc);

Creating a Rendering Context

Now we have the GDI wrapper class in place, we can continue with the long forgotten OGLWindow constructor and add the code required to create an OpenGL rendering context using the device context we grabbed from the Panel control.

We may in future want to use the pixel format setting for a number of windows, so it’s a good idea to put this functionality into a separate method. Create a private method in the OGLWindow class as follows:

private void setDCPixelFormat(ref IntPtr dc, GDI.PixelFlags flags,
byte colorBits, byte depthBits)
{
// Select the pixel format for the given device context
GDI.PIXELFORMATDESCRIPTOR pfd =
new GDI.PIXELFORMATDESCRIPTOR(flags, colorBits, depthBits);

// choose a pixel format that best matches that required
int nPixelFormat = GDI.ChoosePixelFormat(dc, ref pfd);

// set the pixel format for the device context
if (0 == GDI.SetPixelFormat(dc, nPixelFormat, ref pfd))
{
int errno = Marshal.GetLastWin32Error();
throw new Exception("Error in SetPixelFormat() = " + errno);
}
}

From the previous explanations, none of this should contain any surprises. Now finish the OGLWindow constructor by adding the following code after the point where we’d grabbed the hDC :

...
// select the pixel format
GDI.PixelFlags flags =
GDI.PixelFlags.PFD_DRAW_TO_WINDOW |
GDI.PixelFlags.PFD_SUPPORT_OPENGL |
GDI.PixelFlags.PFD_DOUBLEBUFFER;
setDCPixelFormat(ref hDC, flags, 24, 16);

// create the rendering context and make it current
hRC = GL.wglCreateContext(hDC);
GL.wglMakeCurrent(hDC, hRC);

// set lighting parameters and create display lists
setupOpenGL(ref hDC);
}

That gets us an OpenGL rendering context compatible with the Panel control we’re using for display and allows us finally to start using OpenGL in earnest. We’ll cover the " wgl " functions used above a little later when we finalise the GL class. Just leave these lines in the constructor for now.

Lights, Cameras, …

There’s one more private method to be defined, setupOpenGL(), which we called from the OGLWindow constructor above. This is where we set up the OpenGL scene in terms of general lighting, material properties, and create display lists (explained later) for any characters we’ll be displaying in 3D. To explain all of this in detail before showing you any code would be confusing. What I’ll try to do is to go through it piecemeal and explain the relevant background in the process.

The first task will be to set up the lighting. To do this, we’ll need access to the OpenGL functions glEnable() and glLightfv() which we’ll do by adding external static functions to the GL class we created earlier in the file OpenGL.cs. We’ve been through this process in detail before so I won’t labour the details except where there’s anything new or important.

The header file for OpenGL, gl.h, includes a large number of #define ’s for various flags and constants. The use of these within OpenGL is often not straightforward. Some definitions may be used as mutually exclusive options in certain functions and at first sight might be suitable for implementing as enumerations. However, in some cases, sub-sets of these logical groupings are used in different functions where an enumerated type would make no sense. There’s absolutely nothing we can do about this – so to avoid confusing the issue any more than necessary, I’ll define all of these as constants and, in addition, I’ll only define the ones we need for this application.

So, the GL class will start off like this:

public abstract class GL
{
// opengl constants
public const uint GL_LIGHTING = 0x0B50;
public const uint GL_LIGHT0 = 0x4000;
public const uint GL_AMBIENT = 0x1200;
public const uint GL_DIFFUSE = 0x1201;
public const uint GL_SPECULAR = 0x1202;
public const uint GL_POSITION = 0x1203;

// opengl functions
[DllImport("opengl32.dll")]
public static extern void glEnable(uint cap);
[DllImport("opengl32.dll")]
public static extern void glLightfv(uint light, uint pname,
float[] someParams);

Then the setupOpenGL() method in OGLWindow.cs will use these definitions as follows:

private void setupOpenGL(ref IntPtr dc)
{
// set up the opengl environment
float[] whiteLight = { 0.4f, 0.4f, 0.4f, 1.0f };
float[] diffuseLight = { 0.8f, 0.8f, 0.8f, 1.0f };
float[] specular = { 0.9f, 0.9f, 0.9f, 1.0f};
float[] lightPos = { -100.0f, 200.0f, 50.0f, 1.0f };

GL.glEnable(GL.GL_LIGHTING);
GL.glLightfv(GL.GL_LIGHT0, GL.GL_AMBIENT, whiteLight);
GL.glLightfv(GL.GL_LIGHT0, GL.GL_DIFFUSE, diffuseLight);
GL.glLightfv(GL.GL_LIGHT0, GL.GL_SPECULAR, specular);
GL.glLightfv(GL.GL_LIGHT0, GL.GL_POSITION, lightPos);
GL.glEnable(GL.GL_LIGHT0);

In essence, this is just saying – enable lighting, light zero looks like this, switch light zero on. The float arrays define either RGBA values (red, green, blue, alpha) for colours of light components or (x,y,z,w) for position. A brief explanation of these terms might help:

  • Ambient light is light that has been scattered so much that it appears to come from all directions. It’s like a background light level. When ambient light strikes a surface, it gets scattered equally in all directions.
  • Diffuse light comes from a particular direction. If it hits a surface straight on, it’s brighter than if it glances across it. Once it has hit a surface, like ambient light, it scatters in all directions.
  • Specular light, like diffuse light, comes from a particular direction. When it hits a surface though, it bounces off in a preferred direction. It’s what makes things look shiny.

The RGB values are the relative levels of red, green and blue in the relevant light component and lie in the range 0.0 to 1.0. The alpha value (A) is used for more advanced OpenGL effects, like blending, and we can safely leave this set to 1.0 and forget about it. Other values are useful for rendering semi-transparent objects (glass, plastics, etc.) or for producing fog effects.

As far as position goes, the (x,y,z) bit is pretty obvious. The w, though, is less so. If w is zero, this means that the light source is a directional one and (x,y,z) defines the direction. If w is non-zero, it means that the light source is positional and (x,y,z) defines the position. So in the above case, it’s a positional light situated to the left of the scene, high above and way behind. The RGB values of each component are equal in all cases so the light will be vaguely white.

The lighting parameters available in OpenGL include many more options to allow the creation of almost any conceivable lighting conditions. For this scene we’ve just used a single light, which will illuminate our text in a reasonably realistic manner.

Material Properties

The light in a scene is only one element that influences how the illuminated object will appear. It is also possible (though not mandatory) to define material properties for the surfaces of any object we define that will describe how they react when light strikes them. Similar to the light components, a material has ambient, diffuse and specular reflection properties which basically define what colour and how shiny it is. There is a fourth component for materials, emission, which allows objects themselves to emit light (glow) but we won’t use that here.

A useful shortcut to defining each of these material properties is to allow some of them to automatically track the object’s colour. This avoids having to specify different degrees of different colours for different objects. We can just tell OpenGL that an object will adjust, for example, it’s ambient and diffuse material properties according to its colour. We’ll do that here and the additional definitions we’ll need in the GL class to do so are:

...
public const uint GL_COLOR_MATERIAL = 0x0B57;
public const uint GL_FRONT = 0x0404;
public const uint GL_AMBIENT_AND_DIFFUSE = 0x1602;
public const uint GL_SHININESS = 0x1601;

[DllImport("opengl32.dll")]
public static extern void glColorMaterial(uint face, uint mode);

[DllImport("opengl32.dll")]
public static extern void glMaterialfv(uint face, uint pname,
float[] someParams);

[DllImport("opengl32.dll")]
public static extern void glMateriali(uint face, uint pname,
int param);

The way we call these functions from setupOpenGL() is as follows:

GL.glEnable(GL.GL_COLOR_MATERIAL);
GL.glColorMaterial(GL.GL_FRONT, GL.GL_AMBIENT_AND_DIFFUSE);
GL.glMaterialfv(GL.GL_FRONT, GL.GL_SPECULAR, specular);
GL.glMateriali(GL.GL_FRONT, GL.GL_SHININESS, 128);

This first enables the colour-material tracking and then dictates that the front surface of any object will automatically adjust its ambient and diffuse reflection properties according to its colour. We could then just use the glColor3f() function to switch colours and the surface properties will change too.

Note that we’re using the same colour for specular light (bright white) as we did for the lighting parameters so we’ll get white highlights. The GL_SHININESS attribute works with specular reflections to control the size and brightness of the highlight. It takes a value from 0.0 to 128.0, with a higher value giving a brighter highlight. Adjusting this can make the material appear dull, wet, metallic, etc.

Good fun to experiment with these effects if you desire – they can be changed as required at any time and will only affect the appearance of any objects defined afterward. Indeed, it is possible to define a single object with, for example, different colours and/or material properties for every vertex. OpenGL can be instructed to smoothly interpolate these properties across the surface of an object and this can be used to achieve some quite spectacular effects with little effort.

More definitions in the GL class:

public const uint GL_DEPTH_TEST = 0x0B71;

[DllImport("opengl32.dll")]
public static extern void glClearColor(float red, float green,
float blue, float alpha);
[DllImport("opengl32.dll")]
public static extern void glColor3f(float red, float green,
float blue);

These allow us to set the background colour, the foreground colour, and to enable depth testing. Depth testing is important for hidden surface removal. If depth testing is enabled (this is what the depth bits parameter in the pixel format relates to), OpenGL keeps track of the z-order of objects in the scene and doesn’t bother to render surfaces of objects that are obscured by objects in front of them.

Add the following code to setupOpenGL() to define a black background, a red foreground and to enable depth testing:

GL.glColor3f(1.0f, 0.0f, 0.0f);
GL.glClearColor(0.0f, 0.0f, 0.0f, 1.0f );
GL.glEnable(GL.GL_DEPTH_TEST);

We’ve now set up our lighting, our material properties for the objects we’re going to model, and their colours. The next thing to do is describe some 3D objects.

Display Lists

As mentioned earlier, objects are described to OpenGL in terms of vertices (corners if you like) and faces. There are numerous ways to describe objects and some useful shortcuts for common shapes.

There’s a lot of internal calculation required to produce a 2D representation of a 3D object, and when the same object is displayed more than once in a scene (or in different positions or orientations in successive frames), a lot of this calculation is needlessly duplicated.

To overcome this, OpenGL provides display lists. These are pre-compiled lists of OpenGL commands that can be stored internally and referred to later to speed up the process of object creation.

Where 3D text is concerned, there is another useful shortcut provided by one of the "wiggle" functions, wglUseFontOutlines(). This takes advantage of the availability of TrueType fonts on a system to automatically generate one or more display lists containing the necessary OpenGL commands required to describe each character. As a TrueType font gives a mathematical description of the shape of each character, OpenGL can work out the relative positions of vertices, decide which vertices form connected faces and extrude a flat font into three dimensions. The resulting display lists can then be simply inserted into a scene by the calling application after it has defined their colour, size, orientation, position, etc.

To save us the trouble of defining a 3D object from scratch, we’ll use this technique to generate our display. The " wgl " functions are defined in wingdi.h but implemented in opengl32.dll, so we’ll implement them in our GL wrapper class. We need a couple of structures to support wglUseFontOutlines() and a couple of constants, so define these as follows:

   [StructLayout(LayoutKind.Sequential)]
public
            struct POINTFLOAT
{
public float x;
public float
            y;
}

[StructLayout(LayoutKind.Sequential)]
public
            struct GLYPHMETRICSFLOAT
{
public float
            gmfBlackBoxX;
public float gmfBlackBoxY;
public POINTFLOAT
            gmfptGlyphOrigin;
public float gmfCellIncX;
public float
            gmfCellIncY;
}
public const uint GL_UNSIGNED_SHORT =
            0x1403;
public const uint WGL_FONT_POLYGONS = 0x1;

There’s nothing different here to the way we treated the PIXELFORMATDESCRIPTOR earlier. The function definitions are:

[DllImport("opengl32.dll", SetLastError = true)]
public static extern IntPtr wglCreateContext(IntPtr hdc);

[DllImport("opengl32.dll")]
public static extern int wglMakeCurrent(IntPtr hdc, IntPtr hglrc);

[DllImport("opengl32.dll")]
public static extern int wglDeleteContext(IntPtr hglrc);

[DllImport("opengl32.dll", CharSet=CharSet.Unicode)]
public static extern int wglUseFontOutlines(IntPtr hdc, uint first,
uint count, uint listbase, float deviation, float extrusion,
uint format, ref GLYPHMETRICSFLOAT lpgmf);

Note the use of SetLastError = true for wglCreateContext(), but also CharSet = CharSet.Unicode in the DllImport attribute for wglUseFontOutlines(). This named parameter allows us to link to a different entry point in opengl32.dll at run time.

The CharSet field controls name mangling and also dictates how string arguments are marshalled over the interoperation boundary. The default is CharSet.Ansi, but this won’t work for us, as we’ll be specifically using Unicode characters to generate our 3D text later. By specifying CharSet.Unicode, we’ll automatically link to the Unicode version, wglUseFontOutlinesW(), if it exists.

There are other ways of achieving this and it’s not really essential to use the above method unless the function includes string arguments, but it is worth mentioning the use of another option, CharSet.Auto, which has the effect of appending "A" to the function name in Ansi environments and "W" in Unicode environments as this applies to many Win32 API function names.

Generating 3D Characters

Having defined the above functions, we can add a few lines of code to setupOpenGL() that will result in a display list being created for each of the first 128 Unicode characters:

IntPtr hfont = openglPanel.Font.ToHfont();
IntPtr holdfont = GDI.SelectObject(dc, hfont);
GL.GLYPHMETRICSFLOAT[] lpgmf = new GL.GLYPHMETRICSFLOAT[128];
GL.wglUseFontOutlines(dc, 0, 128, 1000, 0.0f, 1.0f,
GL.WGL_FONT_POLYGONS, ref lpgmf[0]);
GDI.SelectObject(dc, holdfont);

This will generate 128 display lists that are indexed by number. If you look at the prototype for the wglUseFontOutlines() function, you’ll see that the display lists will be numbered from 1000 upwards, and the generated characters will be extruded 1.0 unit in the z-direction.

In the process of generating the display lists, the function also populates the GLYPHMETRICSFLOAT array with some useful information about the generated shapes. We won’t use these here but it is often used to scale and position text accurately in a 3D scene.

We could select any TrueType font into the device context to get whatever characters we need. For this simplified example I’ve chosen to use the same font we set earlier for the Panel control – Arial 12pt – but this could be changed as necessary or desired.

How OpenGL Maps to a Window

We’re almost ready to do some drawing to a window and see the results, but before we can do that, we need to tell OpenGL how to map its 3D scene onto our 2D window. This is important when the user resizes the form because we need to keep OpenGL in step with the target display.

I’m not going to get into too much detail here, as this area can be tricky to get to grips with. The first bit though is no problem, and the main thing to say is that OpenGL will draw a scene to any rectangular area on a given display context. It doesn’t need to use the whole DC, in fact you can render different scenes to different areas of the same window. The function used to define the viewport (as it’s known) is understandably called glViewport() and we can define this in the GL class as follows:

[DllImport("opengl32.dll")]
public static extern void glViewport(int x, int y,
int width, int height);

This is pretty self-explanatory and, in this example, we will always map to the whole Panel. We’ll put all of the code that manages this task in a private method. That way we can call it from setupOpenGL() to set the initial mapping, and also call it when the form is resized to make sure we refresh the display. Finish off the setupOpenGL() definition as follows, and then create a new private method in the OGLWindow class to handle resizing:

// set initial viewport
changeSize(openglPanel.Width, openglPanel.Height);
}

protected void changeSize(int width, int height)
{
// resize opengl view if form size changes
float range = 125;

// use the whole window
GL.glViewport(0, 0, width, height);

// reset the coordinate system before modifying
GL.glMatrixMode(GL.GL_PROJECTION);
GL.glLoadIdentity();

// establish clipping volume
if (width <= height)
GL.glOrtho(-range, range, -range*height/width,
range*height/width, -range*2.0f, range*2.0f);
else
GL.glOrtho(-range*width/height, range*width/height,
-range, range, -range*2.0f, range*2.0f);

// set up transformation to draw the string
GL.glTranslatef(-110.0f, 0.0f, -5.0f);
GL.glScalef(60.0f, 60.0f, 60.0f);

GL.glMatrixMode(GL.GL_MODELVIEW);
GL.glLoadIdentity();
}

The first part of this method calls glViewport() to set the display area. The remainder of this code sets up the viewing volume (effectively a rectangular box) in which our objects will sit and shifts the coordinate system about to make sure our objects are in view. It would take a long time to explain the background to this code in detail and I’m not sure that it’s appropriate here. For now, take it on trust that all we’re really doing at a conceptual level is shifting the camera to point it where the action is expected and adjusting the focus and zoom. The figure below shows how the OpenGL co-ordinate system is mapped to the Panel control by the above glViewport() and glOrtho() functions.

There’s a few more definitions needed in the GL wrapper class to support these additional functions and these should be included as follows:

public const uint GL_MODELVIEW = 0x1700;
public const uint GL_PROJECTION = 0x1701;

[DllImport("opengl32.dll")]
public static extern void glLoadIdentity();

[DllImport("opengl32.dll")]
public static extern void glMatrixMode(uint mode);

[DllImport("opengl32.dll")]
public static extern void glOrtho(double left, double right,
double bottom, double top, double zNear, double zFar);

[DllImport("opengl32.dll")]
public static extern void glPopMatrix();

[DllImport("opengl32.dll")]
public static extern void glPushMatrix();

[DllImport("opengl32.dll")]
public static extern void glScalef(float x, float y, float z);

[DllImport("opengl32.dll")]
public static extern void glTranslatef(float x, float y, float z);

These are all simple translations of the prototypes in gl.h.

We can now add an override to catch the OnResize() event and make sure we resize the OpenGL view by calling the changeSize() method:

protected override void OnResize(EventArgs e)
{
// resize opengl view if form size changes
base.OnResize(e);

// change opengl view
changeSize(openglPanel.Width, openglPanel.Height);
}

So that takes care of mapping our scene to the window and automatically adjusting things if the user resizes the form. Now we can build the code that does the easy bit – putting the picture on the screen.

Painting Pictures

The first thing to do is to override the form’s OnPaint() method. In here, we’ll just call the base class handler, and then call a private method to draw the OpenGL scene to the back buffer, then call the GDI function SwapBuffers() to display it on screen:

protected override void
            OnPaint(PaintEventArgs e)
{
// handle repaint of whole
            form
base.OnPaint(e);
renderScene(words3d);
GDI.SwapBuffers(hDC);
}

That’s easy enough. We can now implement the renderScene() method that we use to build the whole scene. By putting all of this code in one place, we can develop a nice harness for any OpenGL objects we might want to experiment with and we have a framework for testing them out.

Before we do this, there are a few final definitions in the GL wrapper class:

public const uint GL_COLOR_BUFFER_BIT = 0x00004000;
public const uint GL_DEPTH_BUFFER_BIT = 0x00000100;

[DllImport("opengl32", CharSet=CharSet.Unicode)]
public static extern void glCallLists(int n, uint type, char[] lists);
[DllImport("opengl32.dll")]
public static extern void glClear(uint mask);
[DllImport("opengl32.dll")]
public static extern void glFlush();
[DllImport("opengl32.dll")]
public static extern void glListBase(uint aBase);
[DllImport("opengl32.dll")]
public static extern void glRotatef(float angle, float x, float y,
float z);

This completes the list of functions and types in the GDI and GL wrapper classes that we need for the whole application. All of this code is in OpenGL.cs and we can add to it, re-use it in other projects, or build it as a class library for others to use. We’ll be enhancing it further in the next sample application but for now it’s complete.

The method which does the drawing, renderScene(), is defined as:

private void renderScene(string
            words)
{
// render opengl scene
char[] lists =
            words.ToCharArray();
GL.glClear(GL.GL_COLOR_BUFFER_BIT |
            GL.GL_DEPTH_BUFFER_BIT);

// save the matrix state and do the
            rotations
GL.glMatrixMode(GL.GL_MODELVIEW);

// draw the
            string
GL.glPushMatrix();

GL.glRotatef(6.0f, 1.0f, 0.0f,
            0.0f);
GL.glRotatef(355.0f, 0.0f, 1.0f,
            0.0f);

GL.glListBase(1000);
GL.glCallLists(lists.GetLength(0),
            GL.GL_UNSIGNED_SHORT, lists);

GL.glPopMatrix();

GL.glFlush();
}

This is much simpler than it looks. The first statement produces an array of Unicode characters from the string we want to render. These will be used to access the set of display lists we created in setupOpenGL(). The next statement just clears the OpenGL buffers before drawing.

When we created the display lists, we chose to number them from 1000 upwards, so the two lines in the code:

   GL.glListBase(1000);
GL.glCallLists(lists.GetLength(0),
            GL.GL_UNSIGNED_SHORT, lists);

set the base display list index at 1000 then "call" one saved display list for each character in our string, based on its Unicode value. As long as we haven’t used any characters outside the range 0 – 128, we can use this method to display a 3D string of any length or combination of characters. Of course, we could have generated display lists for all 65536 Unicode characters but this would consume an alarming amount of memory.

One thing to note about the display lists we generated with wglUseFontOutlines() is that each list of OpenGL commands ends with a translation (i.e. a shift of xyz position) to ensure that a subsequent character is drawn in the correct place relative to the preceding character. This is all derived automatically from the TrueType font definition and allows proportionally spaced fonts to be rendered correctly with no additional input from the developer.

The remaining code in the above method just serves to rotate the 3D object slightly to offset it from the face-on position. That way we’ll be able to see the 3D structure more clearly and also see the effect of our lighting and material properties.

The last OpenGL command in this code, glFlush(), deserves a mention. OpenGL is designed around a client/server model so, technically, it is possible to have the OpenGL server processes operating on a separate machine in a network. Depending on the exact nature of the particular OpenGL implementation in use, the drawing commands may have been buffered in a network buffer or in an accelerated graphics card buffer. The glFlush() command is used to ensure that any pending commands are flushed from any intermediate buffers. This doesn’t necessarily mean that the commands will have finished executing; it just means that they will not sit in a buffer waiting to begin. The documentation for glFlush() helpfully explains that "the glFlush function forces execution of OpenGL functions in finite time"!

And, finally …

Nearly there now – just need to make sure that the OpenGL rendering context, along with our grabbed device context, are disposed of when the application terminates.

Override the OnClosing() method and add the following code:

protected override void
            OnClosing(CancelEventArgs e)
{
// tidy opengl when form
            closes
base.OnClosing(e);

// deselect the current
            rendering context and delete it
GL.wglMakeCurrent(IntPtr.Zero,
            IntPtr.Zero);
GL.wglDeleteContext(hRC);

// release the
            device context
openglGraphics.ReleaseHdc(hDC);
}

None of this should by now require any explanation, so save everything, rebuild the project and give it a run:

The figure above shows the end result. We haven’t added any animation so it’s just a static image. You could add a Timer to the form and play around with the glRotate() parameters to make the text move. All you need to do is to invalidate the form to force a repaint and OpenGL does the rest.

Also, the image should repaint itself when necessary and should respond appropriately when the form is resized. If you drag the bottom right corner of the window, you will notice that OpenGL renders a new image based on the revised viewport size and so the quality and definition increases as the image gets bigger.

So we’ve proved the concept - we can make OpenGL work from within a C# application and we’ve developed a number of useful routines that can be employed with little or no change in other applications.

An OpenGL Web Service

An awful lot has already been written about Web Services in the .NET environment. They are very easy to develop and Visual Studio .NET makes it a straightforward task to consume Web Services from ASP.NET or Windows Forms applications. So what would be the point of an OpenGL Web Service?

There are a couple of things that come to mind –

Not everyone who needs 3D graphics is an expert in the OpenGL API. There might be scope for a service which would allow a novice to describe online exactly what sort of image he needs to create and have that delivered to him at the click of a button. "Put a blue sphere here, add a textured pyramid below, put this text to the right and give it a yellow background" – Click – "not quite right, make the sphere bigger and reduce the whole image to 320x240" – Click – "Great." – right-click and save the generated image. Maybe there could be a database of image templates to shortcut the process, pick a template, shift things around, change colours, see the result, save it.

Of course there’s software that will do this sort of thing that you can load on the desktop, but you’ll need the memory required to load it, a suitable OpenGL implementation, the right fonts, a compatible graphics card, operating system, etc. And you’ll probably have to pay for it. You’ll need upgrades, maintenance, support, training, whatever. By putting the onus on the server to control the OpenGL environment, we could potentially offer top quality 3D images interactively to anyone who had a browser – no matter what platform, operating system, processor, memory or graphics card. Maybe we could even automate the production of animated GIF’s – customised company logos and web page graphics elements, all handled by a Web Service 24x7 with no human intervention.

I think there’s enough possibilities to justify another proof of concept, but for this we’ll need a few changes to the first application.

Rendering to a Bitmap

The most obvious change is that we won’t be rendering images to a window – we’ll need to render them off-screen to a bitmap via a memory device context. The downside of using a memory DC is that we lose any benefit from hardware accelerated graphics – all of the rendering is handled by software so it’s measurably slower. This can be offset by adding processor and memory resource, but it’s a point that is very often overlooked and needs to be borne in mind if the service is to scale well.

To get started, create a new Web Service, name it OGLWebServ, and add the existing OpenGL.cs item to the project. Add the following statements to the top of OGLWebServ.asmx.cs :

using System.Drawing.Imaging;
using System.IO;
using System.Runtime.InteropServices;
using Wrox.OpenGL;

This will give us easy access to the classes we’ll need from the FCL and from our GDI and GL wrappers we created earlier.

From the OGLWindow class we created in the first application, copy and paste the private methods changeSize() and renderScene() – we will use these unchanged in the Web Service.

Also from OGLWindow, copy the setDCPixelFormat() method. This will be used after adding one small change. As shown in the following code, add a call to the OpenGL command glFlush() just prior to SetPixelFormat(). The reason for this is that, when using a memory device context, it is allowable to call SetPixelFormat() more than once and, depending on the OpenGL implementation and the graphics card used, this can cause some strange bugs if buffers are not flushed correctly beforehand. In most cases this is not necessary, but as this causes negligible overhead and can save a lot of fruitless debugging, it’s something I do as a matter of course when using OpenGL to render to an off-screen bitmap.

The setDCPixelFormat() function now looks like this:

private void setDCPixelFormat(ref IntPtr dc, GDI.PixelFlags flags,
byte colorBits, byte depthBits)
{
// Select the pixel format for the given device context
GDI.PIXELFORMATDESCRIPTOR pfd =
new GDI.PIXELFORMATDESCRIPTOR(flags, colorBits, depthBits);

// choose a pixel format that best matches that required
int nPixelFormat = GDI.ChoosePixelFormat(dc, ref pfd);
GL.glFlush();
if (0 == GDI.SetPixelFormat(dc, nPixelFormat, ref pfd))
{
int errno = Marshal.GetLastWin32Error();
throw new Exception("Error in SetPixelFormat() = " + errno);
}
}

Also some small changes to setupOpenGL() – this time just to parameterise the foreground and background colours, the font name, and the width and height of the image:

private void setupOpenGL(ref IntPtr dc, int width, int height, int textR,
int textG, int textB, int backR, int backG, int backB,
string fontName)
{
// set up the opengl environment
float[] whiteLight = { 0.4f, 0.4f, 0.4f, 1.0f };
float[] diffuseLight = { 0.8f, 0.8f, 0.8f, 1.0f };
float[] specular = { 0.9f, 0.9f, 0.9f, 1.0f};
float[] lightPos = { -100.0f, 200.0f, 50.0f, 1.0f };

GL.glEnable(GL.GL_LIGHTING);
GL.glLightfv(GL.GL_LIGHT0, GL.GL_AMBIENT, whiteLight);
GL.glLightfv(GL.GL_LIGHT0, GL.GL_DIFFUSE, diffuseLight);
GL.glLightfv(GL.GL_LIGHT0, GL.GL_SPECULAR, specular);
GL.glLightfv(GL.GL_LIGHT0, GL.GL_POSITION, lightPos);
GL.glEnable(GL.GL_LIGHT0);
GL.glEnable(GL.GL_COLOR_MATERIAL);
GL.glColorMaterial(GL.GL_FRONT, GL.GL_AMBIENT_AND_DIFFUSE);
GL.glMaterialfv(GL.GL_FRONT, GL.GL_SPECULAR, specular);
GL.glMateriali(GL.GL_FRONT, GL.GL_SHININESS, 128);

// set text and background colours
GL.glColor3f(textR/255.0f, textG/255.0f, textB/255.0f);
GL.glClearColor(backR/255.0f, backG/255.0f, backB/255.0f, 1.0f );

GL.glEnable(GL.GL_DEPTH_TEST);

// use the requested font to create display lists
Font textFont = new Font(fontName, 12);
IntPtr hfont = textFont.ToHfont();
IntPtr holdfont = GDI.SelectObject(dc, hfont);

GL.GLYPHMETRICSFLOAT[] lpgmf = new GL.GLYPHMETRICSFLOAT[128];
GL.wglUseFontOutlines(dc, 0, 128, 1000, 0.0f, 1.0f,
GL.WGL_FONT_POLYGONS, ref lpgmf[0]);

GDI.SelectObject(dc, holdfont);

// set initial viewport
changeSize(width, height);
}

Note that the RGB colours will be assumed to be in the conventional 0 – 255 range rather than the 0.0 – 1.0 range used by OpenGL. These are converted in the relevant calls above.

That’s probably 75% of the service written, using just small variations on the pre-existing sample. It would be possible to treat this more rigorously –for example, parameterise lighting values and material properties – but we’ll limit the changes to those shown as this is just a proof of concept.

The next method required is a bit of a disappointment for me. With the introduction of GDI+ and the Image and Bitmap classes in the FCL, I would have hoped that there would be an easier way to create a DIB section and select it into a memory device context. Maybe:

Bitmap offscreen = new Bitmap(width, height);
hDC = Graphics.FromImage(offscreen).GetHdc();

which works fine if using GDI+ drawing methods. However, it seems that the internal workings of these classes is just not suitable for OpenGL. Any attempt to SetPixelFormat() on the hDC results in an error (Invalid Function). This means we have to revert to API calls to create a memory device context and a DIB section. The additional definitions required in the GDI wrapper class are:

   [StructLayout(LayoutKind.Sequential)]
public
            struct RGBQUAD
{
public byte rgbRed;
public byte
            rgbGreen;
public byte rgbBlue;
public byte
            rgbReserved;
}

[StructLayout(LayoutKind.Sequential)]
public
            struct BITMAPINFOHEADER
{
public uint biSize;
public int
            biWidth;
public int biHeight;
public ushort biPlanes;
public ushort biBitCount;
public uint biCompression;
public uint biSizeImage;
public int biXPelsPerMeter;
public int biYPelsPerMeter;
public uint biClrUsed;
public uint biClrImportant;
}

[StructLayout(LayoutKind.Sequential)]
public struct
            BITMAPINFO
{
public BITMAPINFOHEADER bmiHeader;
public
            RGBQUAD bmiColors;
}

[DllImport("gdi32.dll")]
public
            static extern IntPtr CreateCompatibleDC(IntPtr
            hdc);

[DllImport("gdi32.dll")]
public static extern IntPtr
            CreateDIBSection(IntPtr hdc,
ref BITMAPINFO pbmi, uint iUsage,
            ref IntPtr ppvBits,
uint hSection, uint
            dwOffset);

[DllImport("gdi32.dll")]
public static extern
            int DeleteDC(IntPtr hdc);

[DllImport("gdi32.dll")]
public
            static extern int DeleteObject(IntPtr hgdiobj);

None of this needs any special attention. As before, I have steered clear of the need for unsafe code blocks by the use of struct, ref and IntPtr, but we have the same problem as we did with the PIXELFORMATDESCRIPTOR in the BITMAPINFOHEADER, we need the size of the structure.

Using the same reasoning as before, half-closing my eyes and gritting my teeth while doing so, I have hard coded the structure size when initialising the header. I’m still not one hundred percent happy with this dilemma – I understand that it’s not really logical to have a sizeof function for a managed type, where the CLR can rearrange fields at will, but where we specifically request LayoutKind.Sequential, I don’t see why an unsafe mode is required to retrieve the size of a structure.

Anyway, to continue with the Web Service, the method required to create a DIB section and return a handle to it is defined as:

private IntPtr createDIB(ref IntPtr dc, int
            width, int height)
{
// create dibsection in memory
            dc
GDI.BITMAPINFO bmInfo = new
            GDI.BITMAPINFO();
GDI.BITMAPINFOHEADER bmHeader = new
            GDI.BITMAPINFOHEADER();
IntPtr ppvBits = new IntPtr(0);

//
            setup dib header and info
bmHeader.biSize =
            40;
bmHeader.biWidth = width;
bmHeader.biHeight =
            height;
bmHeader.biPlanes = 1;
bmHeader.biBitCount =
            24;
bmHeader.biCompression = 0;
bmHeader.biClrUsed =
            0;
bmHeader.biClrImportant = 0;
bmHeader.biSizeImage =
            0;
bmInfo.bmiHeader = bmHeader; return GDI.CreateDIBSection(dc,
            ref bmInfo, 0, ref ppvBits, 0, 0);
}

This just wraps the definition of the DIB header and the API call to create it into a method we can use to create a DIB section of any size for any device context.

We’ve now copied a few chunks of code to allow us to get the basic OpenGL interaction working in the Web Service. Before we write the web method that uses these functions, we’ll take a quick look at the overall design of this sample to understand how it will operate in practice.

Outline Design

The primary (and, at present, only) purpose of the Web Service OGLWebServ is to receive requests describing text strings, colours, font names, etc. The service will respond to these requests by using OpenGL to generate a 3D image based on the details supplied by the consumer, and return this as a byte array. The consumer will therefore sit between the Web Service and the end user. In this sample application, the end user will be using a web browser to interact with the consumer, which will be an ASP.NET application named OGLWebCons.

The Web Service will expose just one Web Method – GetPiccy(), which will expect parameters describing the 3D text to be rendered. It will generate the bitmap, compress to jpeg, and return the content as raw bytes in a byte array.

The ASP.NET consumer will consist of two pages. The first will contain web controls to collect the required parameters along with an Image control to display the result. The second page will itself contain no HTML, but will act as the consumer, execute the GetPiccy() method on the Web Service, and deliver the returned byte array directly to the Response object using a binary write.

The first page will use the second page as the ImageUrl for the Image control in order to allow dynamic generation of the image. Parameters obtained from the web browser will be passed to the second page in the QueryString.

Implementing the Web Method

The web method GetPiccy() will receive the following information from which it will generate an image:

  • Font name
  • Text colour (as RGB values in the range 0 – 255)
  • Background colour (also as RGB values in the range 0 – 255)
  • Text string to be rendered (up to 10 characters)
  • Image quality (in the range 0 – 100, with 100 being highest quality)
  • Image width in pixels
  • Image height in pixels

All of the processing will be handled during one pass. The method will create a memory device context, create a DIB section, create an OpenGL rendering context, set lighting parameters, generate the image to the memory DC, convert to Bitmap, compress to jpeg and return a byte array. The definition and initialisation within GetPiccy() is as follows:

[WebMethod]
public byte[] GetPiccy(string
            fontName, int textR, int textG, int textB,
int backR, int backG,
            int backB, int quality,
int width, int height, string
            text)
{
IntPtr hDC = new IntPtr(0);
IntPtr hRC = new
            IntPtr(0);
IntPtr hOldDIB = new IntPtr(0);
IntPtr hDIB = new
            IntPtr(0);

// create memory dc and select a dib
hDC =
            GDI.CreateCompatibleDC(IntPtr.Zero);
hDIB = createDIB(ref hDC,
            width, height);
hOldDIB = GDI.SelectObject(hDC,
hDIB);

After declaring local variables to hold the necessary Windows handles, the GDI wrapper class is used to create a memory device context and to create and select a DIB section into it, sized according to the requested width and height. We need to retain these handles to clean up at the completion of the method as these are created by unmanaged code and so not subject to garbage collection.

Just as in the Windows Form example, we have to set the pixel format before creating an OpenGL rendering context. Note that this time we are requesting a format that allows rendering to bitmap and is not double-buffered:

// select the pixel format
GDI.PixelFlags flags =
GDI.PixelFlags.PFD_DRAW_TO_BITMAP |
GDI.PixelFlags.PFD_SUPPORT_OPENGL;
setDCPixelFormat(ref hDC, flags, 24, 16);

// create the rendering context and make it current
hRC = GL.wglCreateContext(hDC);
GL.wglMakeCurrent(hDC, hRC);

Having created the rendering context, we can use the same functions as before – setupOpenGL() and renderScene() - to set up the lighting and material properties, generate the display lists in the required font, and then render the scene.

setupOpenGL(ref hDC, width, height, textR, textG, textB, backR,
backG, backB, fontName);
renderScene(text);

That’s the image created – we can now go straight ahead and delete the rendering context as we don’t need it anymore:

// deselect the current rendering context and delete it
GL.wglMakeCurrent(hDC, IntPtr.Zero);
GL.wglDeleteContext(hRC);

The image we need is still in the DIB section selected into the memory device context. It would be nice if we could now convert this into a Bitmap object to make the remaining tasks easier. The good news is that we can do just that:

// convert result to bitmap object
Bitmap wrapBmp = Bitmap.FromHbitmap(hDIB);

Next we need to convert this to jpeg format, compressed using the quality setting (0 – 100) requested in the parameter to the GetPiccy() method.

BMP to JPG

More good news! GDI+ has built-in support for six image formats – BMP, JPEG, GIF, Exif, PNG and TIFF. We just need to supply the correct parameters for encoding using the required quality. This is done through the ImageCodecInfo class, to which we supply an EncoderParameter. The codec is particular to one of the above image formats and the encoder parameter is a generic class that is used to pass any parameters to any of the codecs.

Just one more helper method is required, to find and return the particular codec we need. These are not addressable in any simple manner, so we have no choice but to enumerate them all and look for the mime type of the format we want. The mime types are documented on MSDN and are guaranteed to be supported by the .NET framework, so we don’t take too much risk in assuming we’ll find it. The helper method is defined as follows:

private ImageCodecInfo GetCodecInfo(string mimeType)
{
// return codec info for given mime type
ImageCodecInfo[] allCodecs = ImageCodecInfo.GetImageEncoders();
int numCodecs = allCodecs.GetLength(0);

// search available codecs for matching mime type
for (int i = 0; i < numCodecs; i++)
{
if (allCodecs[i].MimeType == mimeType)
return allCodecs[i];
}

// if not found, return null
return null;
}

So in the GetPiccy() method, we can get a reference to the jpeg codec like this:

ImageCodecInfo jpegICI = GetCodecInfo("image/jpeg");

Then we need to jump a few hoops to set up an EncoderParameter to specify the image quality:

EncoderParameters encoderParams = new EncoderParameters();
long[] lquality = new long[1];
lquality[0] = (long)quality;
EncoderParameter encoderParam =
new EncoderParameter(Encoder.Quality, lquality);
encoderParams.Param[0] = encoderParam;

Once we have the codec and the quality parameter, we can compress the Bitmap to jpeg format and retrieve the compressed data as a byte array in three lines of code. Now if that’s not a significant productivity benefit brought about by the power of .NET then maybe you should talk with someone who has tried to do the same in C or C++. Here’s the code:

MemoryStream memstr = new MemoryStream();
wrapBmp.Save(memstr, jpegICI, encoderParams);
byte[] picbytes = memstr.ToArray();

That’s it – no mess and no fuss. Stream out the Bitmap to memory through an image encoder then grab the raw output in an array of bytes ready to return as the result of the method. First though we have to tidy up the unmanaged side of things, so the method completes as follows:

   // tidy up gdi
            objects
GDI.SelectObject(hDC,
            hOldDIB);
GDI.DeleteObject(hDIB);
GDI.DeleteDC(hDC);

//
            return byte array
return picbytes;
}

The OGLWebServ Web Service is now complete. Build and run it to make sure everything is fine, supply some valid parameters and you should get back an XML format message with a Base64Binary return value.

To see if this message actually contains the image we requested, we’ll need to develop the consumer application.

The Consumer

As mentioned above, the consumer is an ASP.NET application named OGLWebCons. The user interface for this is shown below:

Text boxes are used to collect most of the parameters required for calling the web method. The exception is the drop-down list of fonts that will be populated from the lists of installed fonts on the server.

In this case, the ASP.NET application resides on the same physical server as the Web Service, so we can populate this list directly during page load. If this was not the case, the Web Service would need to be enhanced to include a second web method supplying the list of installed fonts.

private void Page_Load(object sender, System.EventArgs e)
{
// load font combo on first load
if (!IsPostBack)
{
// get installed fonts
InstalledFontCollection allFonts =
2new InstalledFontCollection();
FontFamily[] fonts = allFonts.Families;

// populate combo
fontNames.Items.Clear();
foreach (FontFamily font in fonts)
{
fontNames.Items.Add(font.Name);
}

// preselect first font
fontNames.SelectedIndex = 0;
}
}

The Image control imgPic will be used to display the result of the web method call. Make sure that this is set for EnableViewState = False in the UI designer to prevent the application persisting the content via the viewstate. The source for the image will be set once the refreshButton is clicked:

private void refreshClick(object sender,
            System.EventArgs e)
{
// build parameters into query
            string
StringBuilder sb = new StringBuilder();
foreach (string
            fieldKey in Request.Form.Keys)
{
if (fieldKey !=
            "__VIEWSTATE")
{
if (sb.Length ==
            0)
sb.Append("getpiccy.aspx?");
else
sb.Append("&");

//
            add key name
sb.Append(fieldKey);
sb.Append("=");

//
            add key
            value
sb.Append(Server.UrlEncode(Request.Form[fieldKey]));
}
}
imgPic.Src
            = sb.ToString();
}

All this is doing is building the URI to the second web form in the application – getpiccy.aspx. All of the form fields are appended to the URI with the exception of the view state. The resulting URI will look something like:

getpiccy.aspx?refreshButton=Refresh&height=480&width=640&text=C%23+Today&
quality=90&backBlue=0&backGreen=0&backRed=0&textGreen=0&textBlue=0&
textRed=255&fontNames=Arial

The target page – getpiccy.aspx – is defined with no HTML. Add this to the project as a new web form, open getpiccy.aspx in the UI designer and click the HTML tab. Then remove all of the generated HTML to just leave the following line:

<%@ Page language="c#"
            Codebehind="getpiccy.aspx.cs" AutoEventWireup="false"
            Inherits="OGLWebCons.getpiccy" %>

Now add a web reference to the Web Service OGLWebServ to generate the proxy class required to access GetPiccy(). For consistency, rename the reference to OGLWebServ as shown below:

Open getpiccy.aspx.cs in the editor and add the following statement to the top of the code:

Using OGLWebCons.OGLWebServ;

The remainder of the code for this page is very simple. We need to extract the parameters from the supplied QueryString, open a connection to the Web Service, execute the GetPiccy() web method provided by the Web Service, then return the result directly to the Response object. This then provides the content of the Image control in the first page.

All of this is done during the page load event as follows:

private void Page_Load(object sender, System.EventArgs e)
{
// get image parameters from query string
string fontName = Request.QueryString["fontNames"];
int textR = Convert.ToInt32(Request.QueryString["textRed"]);
int textG = Convert.ToInt32(Request.QueryString["textGreen"]);
int textB = Convert.ToInt32(Request.QueryString["textBlue"]);
int backR = Convert.ToInt32(Request.QueryString["backRed"]);
int backG = Convert.ToInt32(Request.QueryString["backGreen"]);
int backB = Convert.ToInt32(Request.QueryString["backBlue"]);
int quality = Convert.ToInt32(Request.QueryString["quality"]);
int width = Convert.ToInt32(Request.QueryString["width"]);
int height = Convert.ToInt32(Request.QueryString["height"]);
string text = Request.QueryString["text"];

// refresh image from server
OGLWebServ.OGLWebServ webserv = new OGLWebServ.OGLWebServ();
byte[] picbytes = webserv.GetPiccy(fontName, textR, textG,
textB, backR, backG, backB, quality, width, height, text);

// output binary as response
Response.BinaryWrite(picbytes);
}

Limitations

That completes all of the coding, although I have intentionally omitted some of the exception trapping and error handling that would be required for a commercial application. In this sample we’re really just exercising our ability to use OpenGL – we’re not updating databases or hogging any essential system resources.

The finished result is shown below. Enter parameters as required and click the button:

Conclusion

This case study has covered quite a bit of ground, including an introduction to 3D graphics programming using the OpenGL implementation freely available with all recent versions of Windows and many other platforms besides.

If this is your first exposure to these techniques then you may well find that it’s introduced a spark of excitement as you mull over the possibilities. When I first encountered OpenGL in 1993, I was certainly enthused and spent many hours reading tutorials, manuals, sample code - anything I could get my hands on. I used OpenGL from C, from Borland Delphi, from VC++ and even from Visual Basic. I developed an OpenGL ActiveX control using ATL that allowed me to plug the pretty graphics elements into many more environments. The advances in graphics card technology and general speed improvements have automatically made my previous developments bigger, better, faster and prettier without me lifting a finger. This is a nice feeling.

Now I find I can access OpenGL from C# and so have another powerful weapon to add to the arsenal. The Web Service we developed above was relatively quick and painless and yet the possibilities for extending this concept are endless. There are obviously very significant benefits to be gained by combining the ease of development in the .NET environment with other established technologies, both within platforms and across platforms – I’m sure OpenGL is only one of many examples.

However, we’ve also encountered a number of pitfalls and compromises that must be reached when interoperating with so-called legacy code. I believe that there is not sufficient emphasis being placed on the importance of a) allowing interoperation, and b) making the right choices when doing so. Surely this is one of the keys to releasing the power available from historically platform-dependent code by exposing that power in a Web Service? Should it be so difficult? Should we be steered away from this path?

It’s evident from my experimentation with OpenGL via C# that it is slower. It has to be really. The extra layer introduced by Platform Invocation Services to allow interaction between managed and unmanaged code and memory does have a performance hit. This is most noticeable when using OpenGL to produce animation, although I have to say it’s much less of an overhead than I’d assumed it would be. With the current pace of hardware development it won’t be long before this overhead is rendered academic.

This case study may have raised as many questions as it’s answered but overall it has been a very useful exercise in combining the old with the new, the established with the emerging, and if it’s kindled a few sparks (not flames!), then I’ll be happy...

...maybe.

Case Study Information
Author | Roger Rowland
Editor | Adam Ryland
Reviewers | Andrew Krowczyk, Saurabh Nandu, John Hodgeson

If you have any questions or comments about this case study, or suggestions for further pieces, please contact the technical editor. If you wish to talk about the case study with other subscribers, please visit our forum here.


Object-Relational Mapping with Reflection and Design Patterns

Object-Relational Mapping with Reflection and Design Patterns
http://www.csharptoday.com/content.asp?id=1770
Article Rating: 3.5
Published on May 6, 2002
Abstract
The object-oriented structure of C# is very different from the relational model of database engines like SQL Server or Oracle, making it somewhat challenging to transfer complex object structures to and from a series of database tables. Unfortunately, this is a very common problem. In this case study, Jeremy Miller is going to address this by building a reusable, extensible persistence framework to bi-directionally map objects to database storage. This approach is made possible by the new reflection capabilities in the .NET runtime framework. In the process of building the framework, we will explore the inherent issues of object/relational mapping. Because an understanding of design patterns is crucial to building a framework, we will examine the usage of several design patterns and the motivation for using them.
Case Study

The object-oriented structure of C# is very different from the relational model of database engines like SQL Server or Oracle, making it somewhat challenging to transfer complex object structures to and from a series of database tables. Unfortunately, this is a very common problem. We are going to address this by building a reusable, extensible persistence framework to bi-directionally map objects to database storage. This approach is made possible by the new reflection capabilities in the .NET runtime framework. In the process of building the framework, we will explore the inherent issues of object/relational mapping. Because an understanding of design patterns is crucial to building a framework, we will examine the usage of several design patterns and the motivation for using them.

System Requirements

You will need a SQL Server database and the .NET runtime to run the samples. This case study assumes readers will have a working knowledge of the C# language and exposure to object oriented concepts. UML is used for modeling the C# objects. A basic knowledge of relational databases is also assumed. TSQL or PL/SQL programming would also be useful.

Object-Relational Mapping

The typical structure of a .NET enterprise application will include an object-oriented middle tier authored in C# (or another language like VB.NET), and a data access layer consisting of a relational database engine (RDBMS) like SQL Server or Oracle and data manipulation with SQL or stored procedure languages like Transact SQL or PL/SQL.

When starting a new development project, the classic object oriented approach is to begin modeling the class structure by capturing the key concepts of the problem domain. Next we would begin to decipher the relationships between the classes and describe the interactions between them. The actual class structure is driven by the necessary behavior inherent in these interactions. Navigation between classes is achieved through in-memory references from one class to another. This reference is typically from a parent object to a child object. Object identity in the middle tier is again through in-memory references (and stack or heap location to the .NET runtime).

At the same time, data modelers are designing the database. Whereas the object model is driven by behavior and data, the database model is driven strictly by the storage requirements of the data. Uniqueness and identity is crucial to database design. A primary key, consisting of one or a combination of columns in the table, identifies individual rows in a database table. Relationships between database tables are expressed by duplicating the primary key columns of the parent table in the child table as a foreign key. The navigation between related tables is from child to parent, the opposite of the typical object relationship.

This mismatch might be easier in some cases to solve by using an object-oriented database (OODBMS), or possibly by using the new serialization capabilities of the .NET runtime to simply save a serialized image of the object hierarchy to the database. However, many, if not most, projects will also have reporting or online decision support requirements. In this case a relational database is advantageous as most reporting software is built to display relational data.

Persistence Framework

Fortunately, the issues we face in making an object-oriented application work with a relational database are finite. There is a great deal of commonality in mapping the persistence of each class type in the middle tier to database tables. This leads to the desire to build a Persistence Framework to centralize the common functionality in a reusable class structure.

A framework is an extendable set of objects to provide services. Typically a framework provides the implementation for the core, invariant functions of a service, and requires a developer to extend abstract methods or classes to plug in the variant functionality. A framework is an example of white box reuse, where an understanding of the internals of a module is somewhat necessary to effectively use the functionality. Building and using a framework is also a good way of standardizing and minimizing the effort in providing instrumentation, security, and error handling in your application.

Any persistence framework must provide all the services necessary to translate an object structure into the database storage. This includes all mapping, identity, and relationship functionality. The framework must also provide all transactional control when making multiple changes.

Domain Model Pattern

The basic architecture of the middle tier business logic components assumed by this article is based on the Domain Model pattern. This is the classical OO approach of building an application around the "nouns" of the problem domain. These domain classes will encapsulate both the data of the system and the business logic behavior of the system. All other services like messaging, user interface, data persistence, and instrumentation are handled in other service layers. It is important to untangle the business classes from system classes because those pesky users will constantly change their processes.

This can save a lot of time while building a large application. As an example, a given system may involve a hundred different transaction types, but only 8-10 different domain classes. Instead of writing stored procedures or SQL to make 100 different actions to the database, we can write only what is necessary to load and persist the 8-10 domain classes.

Any given transaction would involve a message or UI command being received by some sort of controller class. The controller class would load the appropriate domain objects with the persistence framework, manipulate the domain objects, and persist the changes to the database with the persistence framework.

Usage of Stored Procedures

The framework presented in this case study is dependent upon the usage of stored procedures in the database and other proprietary features like SQL Server's auto number functionality. In my opinion, the usage of stored procedures is highly advisable for the following reasons:

  1. Stored procedures are precompiled in the database and execute more efficiently than dynamic SQL.
  2. Building dynamic SQL in the middle tier is tedious and potentially inefficient due to the string manipulation. It can also be difficult to debug.
  3. Network traffic between application servers and database servers is reduced.
  4. They provide a layer of abstraction between the code and the database. A stored procedure is presenting a simplistic interface to the code. Hard coded SQL in the middle tier code is a maintenance problem. By putting all of the SQL in stored procedures, a DBA can optimize the SQL without impacting the source code. All the SQL in the project is readily available, no stepping through code just to capture the SQL statements being run against the database.
  5. The mapping strategy we are using in the framework is easier to accomplish by mapping object attributes to stored procedure parameters rather than directly to tables.

Part of the intent of building a persistence framework in the first place is the differences in structure between the database and the class structure. Mapping classes to databases is relatively clean when one class maps to one and only one table. However, most non-trivial applications will have more complicated cases such as:

  1. Multiple classes mapping to a single table.
  2. One class stored in multiple tables.
  3. Inheritance relationships. There is a multitude of ways to structure tables to store different subclasses in a hierarchy, each with different consequences for the object-relational mapping. See the link to Scott Ambler's website at the bottom of the case study for much more information on database structures.

Rather than deal with all of these possibilities in a dynamic SQL generation scheme, we can sidestep the issue altogether by mapping to a stored procedure instead. The actual structure of the database is abstracted behind the stored procedure signature.

The proprietary nature of stored procedures is a drawback. If the application were to be ported to another database engine, at a minimum, all the stored procedures would have to be rewritten. In many cases, especially for building in-house applications like I do, switching database engines is an unlikely occurrence. Regardless, in order to make our framework portable, we will be using creational patterns to abstract the configuration of the stored procedures used in the mappings.

Some of the links at the end of the case study present a very different viewpoint on this issue. If portability were more important for your application than pure performance, a strategy that relies on systematically creating dynamic SQL rather than stored procedures would be more appropriate.

Database Mapper Pattern

There are two general strategies to map persistent classes to a database. The first strategy is to have the domain classes control their own persistence. This direct mapping design calls for the domain classes to control the data layer classes. The second strategy is to make other classes responsible for all of the persistence functionality. With this indirect mapping approach, the control between layers is inverted. The data layer classes call into the business domain objects. Below is a comparison between the two strategies.

Indirect Mapping | Direct Mapping
The domain objects are strongly coupled to the database structure. Each domain object is responsible for persisting its properties. | The domain objects are loosely coupled to the database structure. The domain objects have no knowledge of the database.
The domain objects contain both their business logic and unrelated persistence code. This is a violation of the principle of High Cohesion. | The domain objects only contain business logic. The mapping classes only contain persistence code. The system is highly cohesive.

The approach we will be using in our framework for indirect mapping will be the Database Mapper pattern (also called Database Broker). This pattern calls for each persistent domain class to have its own mapper class that handles all loading and persisting for the domain class. Of course, writing a database mapper class for every possible domain class will quickly become tedious. With a language like C# that supports reflection, we can use a metadata (literally data about data) driven approach to build a generic mapper class. The metadata-based mapper can dynamically generate the mapping from an object to a database command.

The actual mappings between classes and the database can be stored separately from the domain classes and configured at runtime on demand, based on the object type and action requested (add, delete, or update). This mapping metadata can be changed without touching any source code. Having a metadata approach like this means that your database programmers can build their stored procedures and SQL commands simultaneously with the C# developers building their classes. Both changes in the C# application and the database structure can be accommodated easily.

It is important to note that using reflection will incur a performance penalty. How much this will affect an application is not yet clear. My thought on this subject is to go down the path of using reflection.

If it is determined that reflection is too slow, use the mapping schema that is already built to auto-generate individual Database Mapper classes for each domain class.

Object Identifier Pattern

In order to synchronize identity and relationships between the middle tier objects and the database rows, we will use the Object Identifier (OID) pattern. This pattern calls for each object to be assigned a unique value. These OID values must correlate to primary key values in the database. Each new object created in the system must be assigned a unique OID. This OID value can be alphanumeric, but should probably be a long integer. At a minimum, the OID must be unique over a class type and probably any subclasses.

This strategy has an immediate effect on the database structure. To fully take advantage of the OID strategy, the database should use surrogate keys for identification rather than one or more fields with business meaning. Surrogate keys are typically an autonumber field in SQL Server or generated by a Sequence in Oracle. These keys are created by the database itself and have no meaning other than as an identifier.

Using surrogate keys in the database has a couple of advantages. First, the database joins are simplified because only a single column is necessary to completely define the relationships between tables. The database is also optimized because the key fields are always a simpler data type as opposed to searching on multiple string fields. The database and the middle tier structure can be shielded from changes in business processes that alter the meaning of columns in the parent table.

As a real world example, my company has a project in progress to merge multiple instances of a legacy system into one instance to save maintenance. In order to complete the merge, the key structure of the main table had to change to reflect both the order number and now the region name. This change in key structure forced a great deal of shotgun surgery to both the database structure and forced a small change to almost every single class in a very large Visual Basic application. If the legacy system had used surrogate keys instead, the impact of merging the instances would have been much smaller.

Another reason to use an OID strategy is to standardize the mechanical work of building relationships between objects. Because all the OID values are of a common type, it is now possible to begin centralizing activities like object persistence and retrieval and transaction auditing in a framework design.

For this case study we will be using the autonumber capabilities in SQL Server to manage OID assignment and management. We could use GUID's assigned in the middle tier, but this would be less efficient as the database keys would become much more complex and the storage requirements could go up dramatically. Using some other sort of middle tier mechanism for assigning the OID would also be undesirable because the application could be running in a cluster without any shared state. A crucial function of the persistence framework is to not only assign the OID values, but also to dynamically associate the new OID values as foreign keys when persisting an object hierarchy for the first time.

Batch Fetching and Aggregation

Merely loading a single object is not enough. Almost every domain object will have references to other domain objects. In most cases you will probably need more than just the top layer of an object hierarchy. The object relationships can take the form of a one-to-one, one-to-many, or the dreaded many-to-many structure.

The framework needs to be able to load an entire object hierarchy and set the proper object references. Conversely, when persisting the object hierarchy, the framework should be able to associate foreign key values to child objects without using recursive references or storing the foreign key values in the child objects themselves. However, every child object is not necessary in every interaction. In order to save the heap thrash inherent in instantiating objects and unnecessary round trips to the database we will be building a Virtual Proxy (also called Lazy Fetch /Materialization) collection Supertype to fetch the child objects on demand. If the child collection is never used within a particular operation, the child objects will never be fetched. It is important to note that the code to implement the batch fetching is encapsulated into the collection subclasses. Again, we are trying to minimize the coupling between the domain objects and the persistence framework.

Transactional Management

The framework must be able to manage transaction boundaries. This involves both controlling the commit and rollback actions against a database session and tracking the transactional state of each domain object (i.e. new, deleted, modified). In addition, an application may have the requirement to keep a transactional audit trail. The framework should have the capability to log a transaction and the action details of the transaction.

In designing the framework, we must be very careful that the database commands are called in a certain order. If a parent object with several child objects is being inserted into a database, it is important that the parent is persisted first and its OID assigned so that when the child objects are inserted, the parent OID is available to set the foreign key relationship in the database.

Case Study Structure

  • Mapping a Single Object
    • Strategy Pattern
    • Reflection and IObjectMember
    • ICoercionStrategy
    • Mapping with IObjectMember and ICoercionStrategy
  • Fetching a Batch of Objects
    • BatchFetcher Class Diagram
    • BatchFetcher Sequence Diagram
    • BatchFetcher Code
    • Iterator Pattern
  • Mapping Schema
    • Mapping Schema Table Structure
    • Translating the Mapping Schema into ObjectMap and BatchFetcher Instances
    • Builders Class Diagram
    • CoercionStrategyFactory and ObjectMemberFactory
    • Factory and Flyweight Patterns
    • Builder Pattern
    • CommandBuilder
  • Building the Framework Classes
  • PersistenceManager
    • Façade Pattern
  • Relating Objects
    • Visitor Pattern
    • Layer Supertype Pattern
  • VirtualProxyCollection
    • Virtual Proxy Pattern
    • Template Pattern
    • Building VirtualProxyCollection and an OrderCollection Subclass
    • VirtualProxyPattern.Build() Steps
    • Building a Subclass of VirtualProxyCollection
  • PersistenceTransaction
  • Porting the Framework to Different Databases
    • Abstract Factory Pattern
  • Making the Framework Industrial Strength
  • Conclusion

Mapping a Single Object

Transferring data from one object instance to one database stored procedure requires a structural mapping of an object to an ADO.NET command, and a mapping of each object member to a parameter of the command. This leads immediately to the following class structure using the Database Mapper pattern.

The AttributeMap class performs a transfer of values from one object member to a database command parameter. The ObjectMap class aggregates the AttributeMap class and encapsulates the entire transfer. An ObjectMap instance accomplishes the goal of creating a metadata based database mapper by varying the IDbCommand member and configuring its AttributeMap collection at runtime.

As mentioned previously, using a stored procedure simplifies our mapping strategy class by abstracting the database structure from the data access code. All ObjectMap needs to know is the stored procedure name, and the pairings of command parameters to object members. ObjectMap is blissfully unaware of details like storing an object in multiple tables.

Because a stored procedure can have both input and output parameters, we will make no distinction between loading and persisting a class in the ObjectMap.Transfer() method. The execution of the mapping is thus in three parts as shown in the sequence diagram below:

Of course, in making the transfer from command parameter to object instance we immediately discover that the data type of the parameters will not necessarily match the .NET types of our objects. The AttributeMap class will need to explicitly coerce the value being passed from the database into the expected type of the object attribute. For example, a database engine may store a true/false value as 0 or 1. In the mapping, this value must be a true Boolean type. The coercion process must also gracefully handle null values in the database with some sort of default value.

In our case, we are using reflection to dynamically access the property or field values of an object class. Using reflection in C#, we will use the System.Reflection.FieldInfo class to get and set field values, and the System.Reflection.PropertyInfo class to do the same with properties. Obviously, there must be some means for the AttributeMap class to select between using FieldInfo and PropertyInfo.

We could solve the variance in attribute mapping by using lots of conditional statements in the AttributeMap class, but this clutters the code and could easily cause maintenance difficulties. Using a brute force approach, we could build a separate AttributeMap class for every single permutation of field/property and object type, but even with inheritance this could become cumbersome. Fortunately, there is an elegant OO solution called the Strategy Pattern.

Strategy Pattern

The strategy pattern calls for building a family of interchangeable algorithms encapsulated into separate classes with a common interface. The strategy pattern can be used in place of conditional logic to provide branching behavior. In the case where the varying portion of a process is relatively small and isolated, the strategy pattern can be a simpler and more extensible alternative to using class inheritance.

There are three parts to a strategy pattern:

  1. The individual strategy classes, each with a different behavior
  2. A common interface supported by every strategy class
  3. A context class that references a strategy object to do work.

Using the strategy pattern does have some drawbacks. There is extra overhead with the increased number of objects and extra overhead with the increased calls between context and strategy objects.

The AttributeMap class is a perfect opportunity for using the strategy pattern. As discussed above, the two varying parts of AttributeMap are accessing the different reflection mechanisms and type coercion. To solve these variances, we will define two different strategy interfaces: IObjectMember and ICoercionStrategy.

Reflection and IObjectMember

Reflection refers to a programming language's ability to read its own metadata. The reflection API in the .NET runtime is realized through a hierarchy of classes centered on the System.Type class. The System.Type class in .NET completely describes a class by capturing metadata about the class from its assembly. The Type class can be used to discover everything there is to know about a class-its methods, constructors, properties, etc. The Type class can also be used to dynamically read and set properties of an unknown object at runtime. Every class in .NET exposes a Type instance through a method called GetType() that is inherited from the System.Object class.

In order to dynamically get or set the value of an object field, we get a reference to the proper System.Reflection.FieldInfo object by calling the Type.GetField(string FieldName) method. An example of this is below in the constructor function on the FieldMember class. Likewise, the System.Reflection.PropertyInfo class is used to get and set property values.

The IObjectMember interface will be used as the strategy interface to encapsulate the usage of the reflection classes. We need to have at least two different concrete classes, FieldMember to use the FieldInfo class for field attributes, and PropertyMember to use the PropertyInfo class for property attributes. IObjectMember and FieldMember are shown below.

public interface IObjectMember
{
Object GetValue(Object Target);
void SetValue(Object Target, Object Value);
Type MemberType();
}

public class FieldMember : IObjectMember
{
private FieldInfo _FieldInfo;

public FieldMember(string FieldName, Type ObjectType)
{
_FieldInfo = ObjectType.GetField(FieldName);
}

public Object IObjectMember.GetValue(Object Target)
{
return _FieldInfo.GetValue(Target);
}

public void IObjectMember.SetValue(Object Target, Object Value)
{
_FieldInfo.SetValue(Target, Value);
}

public Type IObjectMember.MemberType()
{
return _FieldInfo.FieldType;
}
}

ICoercionStrategy

ICoercionStrategy classes will handle the type casting between ADO.NET types and the base .NET types. They must also be able to set a default value in the case of the database value being null. As the class diagram below shows, it is possible to build as many particular implementations as required.

The basic interface defines the methods available to transfer data values between an IObjectMember object and a command parameter.

public interface ICoercionStrategy
{
void Load(Object Target, IObjectMember ObjectMember, Object Value);
void SetParameter
(
Object Target,
IObjectMember ObjectMember,
IDataParameter Parameter
);
}

The DoubleCoercionStrategy class is a specific implementation of the ICoercionStrategy that is used for class members of type double. Note the casting in the Load() method and the usage of 0 as a default value in the case of a null from the database.

public class DoubleCoercionStrategy : CoercionStrategy
{
public override void ICoercionStrategy.Load
(
Object Target,
IObjectMember ObjectMember,
Object Value
)
{
if (Value == null)
ObjectMember.SetValue(Target, 0);
else
ObjectMember.SetValue(Target, (double)Value);
}
}

Mapping with IObjectMember and ICoercionStrategy

The AttributeMap class merely calls into its IObjectMember and ICoercionStrategy members to do type casting and reference without any knowledge of the internal algorithms. At this point AttributeMap is nothing but a coordinator between its child objects. The framework can be extended to use additional data types and reflection mechanisms without altering the AttributeMap class. As we will see later in the study these strategy classes will be reused for completely different mapping mechanisms.

ObjectMap Code

Now that we have mechanisms for coercion and dynamic access to object properties via reflection, we can finally build ObjectMap. The ObjectMap class is the base implementation of the IObjectMap interface for mapping a single object. The ObjectMap uses a collection of AttributeMap objects to represent the detailed transfer between stored procedure parameters to object members. ObjectMap can accommodate any combination of input and output parameters in a single pass. This is important because of our usage of SQL Server autonumber fields as the source of object identifiers. On an insert operation, the object identifier value is created by the database and immediately assigned to the object. By making ObjectMap bi-directional, we eliminate the need for separate classes for inserting, updating, and deleting actions.

When an AttributeMap is added to ObjectMap, the AttributeMap is queried to determine whether it is an input and/or output parameter. Based on the determined direction of each AttributeMap, ObjectMap will store a reference to each AttributeMap in one or both of a pair of ArrayList objects, one for input parameters, and another for output parameters. When ObjectMap executes, it will iterate through the input AttributeMap instances and call AttributeMap.Persist(). ObjectMap will then call Execute() on its IDbCommand member and iterate through the output AttributeMap instances, calling AttributeMap.Load() on each instance.

public interface IObjectMap
{
long ForeignKeyValue{get; set;}
void Transfer(Object Entity);
void Transfer(Object Entity, long ForeignKeyValue);
void Transfer(IDbConnection Connection, Object Entity);
void Transfer(IDbConnection Connection, Object Entity, long ForeignKeyValue);
}

public class ObjectMap : IObjectMap
{
private IDbCommand _Command;
private IDataParameter _ForeignKeyParameter;
private ArrayList _InputMaps;
private ArrayList _OutputMaps;

public ObjectMap(IDbCommand Command)
{
_InputMaps = new ArrayList(5);
_OutputMaps = new ArrayList(5);
_Command = Command;
}

public ObjectMap(IDbCommand Command, string ForeignKeyParameterName) : this(Command)
{
if (ForeignKeyParameterName != null && ForeignKeyParameterName.Length > 0)
_ForeignKeyParameter = (IDataParameter)Command.Parameters[ForeignKeyParameterName];
}

public override string ToString()
{
string Desc = "ObjectMap\n" + DataConfiguration.GetCommandDescription(_Command);
Desc += "\nInput Maps";
foreach (AttributeMap _Att in _InputMaps)
Desc += "\n " + _Att.ToString();

Desc += "\nOutput Maps";
foreach (AttributeMap _Att in _OutputMaps)
Desc += "\n " + _Att.ToString();

return Desc;
}

public long ForeignKeyValue
{
get
{
if (_ForeignKeyParameter != null)
return (long)_ForeignKeyParameter.Value;
else
return 0;
}

set
{
if (_ForeignKeyParameter != null)
_ForeignKeyParameter.Value = value;
}
}

public void Transfer(Object Entity)
{
foreach (AttributeMap _AttributeMap in _InputMaps)
_AttributeMap.Persist(Entity);

try
{
_Command.ExecuteNonQuery();
}
catch (Exception e)
{
Console.WriteLine(this.ToString());
Console.WriteLine(e.ToString());
}

foreach (AttributeMap _AttributeMap in _OutputMaps)
_AttributeMap.Load(Entity);
}

public void Transfer(Object Entity, long ForeignKeyValue)
{
this.ForeignKeyValue = ForeignKeyValue;
Transfer(Entity);
}

public void Transfer(IDbConnection Connection, Object Entity)
{
_Command.Connection = Connection;
Transfer(Entity);
}

public void Transfer(IDbConnection Connection, Object Entity, long ForeignKeyValue)
{
_Command.Connection = Connection;
this.ForeignKeyValue = ForeignKeyValue;
Transfer(Entity);
_Command.Connection = null;
}

public void addAttribute(AttributeMap Attribute)
{
if (Attribute.isInput)
_InputMaps.Add(Attribute);

if (Attribute.isOutput)
_OutputMaps.Add(Attribute);
}
}

}

Fetching a Batch of Objects Use Case

Now that we can load or persist a single object, what about efficiently retrieving a batch of objects with one database call? Specifically, how do we retrieve all the child objects of a particular parent object? The most efficient way to retrieve tabular data from the database is to use a System.Data.IDataReader returned from a stored procedure to read the database results serially.

The functional decomposition of batch fetching is very similar to that of mapping a single object, but with a few crucial differences. Instead of capturing the input parameters from the very object being mapped, we will be capturing the input parameters by mapping values from a parent object to the database command parameters. The BatchFetchInput class below represents this mapping. The other mapping is from the IDataReader to the newly created objects. The BatchFetchDetail class implements this behavior by retrieving a value from the IDataReader by index and applying the value to an object member. Both classes utilize the IObjectMember and ICoercionStrategy classes that we built earlier. Finally the BatchFetcher class will aggregate both the BatchFetchInput and BatchFetchDetail classes and manage the data command and reader.

BatchFetcher Class Diagram

BatchFetcher Sequence Diagram

Note that BatchFetcher makes a call to BatchFetchInput.Transfer() for each BatchFetchInput, and likewise a call to BatchFetchDetail.Load() for each BatchFetchDetail. The client continues to make calls to BatchFetcher.Read() and BatchFetcher.Load() until the end of the IDataReader is reached.

BatchFetcher Code

public class BatchFetcher
{
private IDbCommand _Command;
private IDataReader _DataReader;
private BatchFetchDetail[] _Details;
private BatchFetchInput[] _Inputs;
private string _strName;

public BatchFetcher
(
string Name,
IDbCommand Command,
BatchFetchDetail[] Details,
BatchFetchInput[] Inputs
)
{
_Command = Command;
_strName = Name;
_Details = Details;
_Inputs = Inputs;
}

public string Name
{
get { return _strName; }
}

public void Fetch(Object Parent)
{
// Seed the Command
foreach (BatchFetchInput _Input in _Inputs)
_Input.Transfer(Parent);

IDbConnection conn = DataConfiguration.GetConnection();
conn.Open();
_Command.Connection = conn;

_DataReader = _Command.ExecuteReader(CommandBehavior.CloseConnection);

}

public bool Read()
{
return _DataReader.Read();
}

public void Load(Object Obj)
{
foreach (BatchFetchDetail Detail in _Details)
Detail.Load(Obj, _DataReader);
}
}

Iterator Pattern

The BatchFetcher class itself does not create the new objects. The client class calling in to the BatchFetcher is responsible for passing in a new object that is loaded with values from the IDataReader results. The client must continue making calls to the BatchFetcher until the end of the IDataReader is reached.

The BatchFetcher class is an example of an Iterator Pattern. The goal of an iterator is to provide sequential access to a list of elements without exposing the grungy details of the actual storage. The iterator should encapsulate all of the traversal mechanisms and state. The IEnumerator interface and its implementations are examples of the iterator pattern that is built into the .NET runtime.

In this case, the client is making repeated calls to BatchFetcher.Read() until the value returned is false, denoting the end of the list. When the client calls into BatchFetcher.Load(), the IDataReader is already positioned at the next record. By using an iterator pattern, the client is not exposed to BatchFetcher's internal structure.

Mapping Schema

A major part of the vision for our persistence framework is to be able to store the mapping configuration of all of the domain classes external to the code. All the data necessary to configure both the ObjectMap and BatchFetcher instances must be stored in an easily accessible way. This not only includes mapping object parameters to database stored procedure parameters, but also how to create and configure the database command objects on the fly.

For a couple of reasons, I suggest using metadata tables in the normal database schema rather than something like XML files or the registry (do not laugh, I have seen a commercial product that uses the registry as a transactional database schema!). One reason is the ease of maintenance. Part of the metadata needed to generate the mapping classes is already stored in the database's data dictionary. In SQL Server the sysobjects and syscolumns tables store the configuration of stored procedures. In Oracle we could use the ALL_ARGUMENTS synonym to retrieve all of the stored procedure metadata. By using the databases own metadata, we are always using the current configuration of a database command. In a later section I will discuss the necessary steps to make our framework portable across database engines by isolating dependencies on the RDBMS data dictionary tables and type system by using the Abstract Factory pattern.

Another major advantage of the metadata mapping would be to expose the persistence mechanism and all the SQL to DBA's for much easier maintenance and trouble-shooting. Nothing is as irritating as trying to determine what SQL is breaking your system when the SQL is dynamically built in the code, not to mention that our code will be aesthetically pleasing by removing all of the ugly string concatenation required to build SQL strings on the fly. With our mapping framework and adequate application logging we will always know the exact SQL being called by examining the mapping configuration and the application audit trails.

Ideally, the persistence framework would be administrated through a wizard user interface that could automatically use both the database metadata and the reflected members of the domain entities to simplify the configuration. The wizard would be able to use both sets of metadata to validate the mappings based on type compatibility and completeness. My original thoughts for building an object/relational persistence framework was for this wizard to enable a database programmer to very quickly connect the middle tier to the database in parallel with the application programmers building the middle tier. If you undertake the effort to build an industrial strength persistence layer, I would strongly advise investing the time to build a good administrative tool. And when you are done, please send me the code!

Mapping Schema Table Structure

The table structure consists of eight tables, shown below:

Table | Description
OR_ASSEMBLY | Lookup table of the possible Assembly names of all of the persistent classes. This would be useful for building administrative wizards and for pre-building ObjectMap configurations.
OR_CLASS | Lookup table listing all of the persistent classes.
OR_ACTION_TYPE | Lookup table that stores the types of OR_MAP records, i.e. "Insert", "Delete", "Retrieve", or "Update"
OR_MAP | Top-level storage of all the possible ObjectMap configurations. Matches a class and action type to a particular stored procedure.
OR_MAP_DETAIL | Contains the parameter to member details of ObjectMap configurations. Note that it does not store the member type (Field/Property) or the parameter configuration (size, type, direction). All or this information is determined at runtime or by utilizing the database data dictionary.
OR_BATCH_FETCH | Top-level storage of all the possible BatchFetcher configurations. Matches a class to a particular stored procedure and a parent class that will be provide the input values (usually just foreign key values)
OR_BATCH_FETCH_INPUT | Contains the parameter to member details of the BatchFetchInput configurations.
OR_BATCH_FETCH_DETAIL | Contains the IDataReader index to member details of BatchFetcher configurations

Translating the Mapping Schema into ObjectMap and BatchFetcher Instances

The most difficult aspect of using the mapping classes is building the correct configuration of the child objects out of a jumble of detail and strategy classes. All of the configuration data is now available in the mapping schema; we just need a systematic way to use this data to build BatchFetcher and ObjectMap classes.

The question becomes "Where should this functionality live?" The code behind the BatchFetcher and ObjectMap classes is fairly simple and we want it to stay that way. Coupled with a desire to make the framework facilitate changing database engines, we clearly do not want to place the configuration code into the mapping classes that is tied to a specific database engine. The object-oriented solution to this issue is to use the Builder Pattern to encapsulate the configuration code into new classes called ObjectMapBuilder and BatchFetcherBuilder that are responsible for nothing but creating and configuring the mapping classes by retrieving data from the database mapping tables. We also need to create a shared mechanism to build IDbCommand objects from the database dictionary called CommandBuilder.

Builders Class Diagram

CoercionStrategyFactory and ObjectMemberFactory

Before we build BatchFetchBuilder and ObjectMapBuilder, it would also be advantageous to have shared mechanisms for picking the correct ICoercionStrategy and IObjectMember classes for a given member type. The classic OO solution is to use the Factory Pattern for the class instantiation.

Also, the ICoercionStrategy classes are all stateless. The application will be more efficient if we could reuse ICoercionStrategy objects rather than incur "heap thrash" by instantiating more objects. We can use the Flyweight Pattern to efficiently share ICoercionStrategy objects.

Factory and Flyweight Patterns

The Factory pattern is used when the class of objects to be created is not known until runtime. The Factory pattern solution is to encapsulate the instantiation of interchangeable classes in a method or separate class.

The Flyweight pattern is a means to efficiently share instances of small objects. A flyweight object must be an object that can be simultaneously used in multiple contexts, preferable a stateless object. The client is responsible for passing in necessary contextual information as input parameters to method calls on the flyweight.

The ObjectMemberFactory uses the factory pattern to return either a PropertyMember or FieldMember as appropriate for the particular object member.

public static IObjectMember GetObjectMember
(
Type ObjectType,
string MemberName
)
{
IObjectMember _ObjectMember;
MemberInfo _Member = ObjectType.GetMember(MemberName)[0];

if (_Member.MemberType.Equals(MemberTypes.Field))
_ObjectMember = new FieldMember((FieldInfo)_Member);
else
_ObjectMember = new PropertyMember((PropertyInfo)_Member);

return _ObjectMember;
}

The CoercionStrategyFactory class uses both patterns. The public GetStrategy() method is a factory method. The method first tries to find a previous instance of the proper ICoercionStrategy to reuse as a flyweight. Failing this check, it instantiates a new instance and stores a reference for later. The client receives a new ICoercionStrategy object without knowing the concrete class.

public class CoercionStrategyFactory
{
private Hashtable _Hash;

public CoercionStrategyFactory()
{
_Hash = new Hashtable(5);
}

public ICoercionStrategy GetStrategy(string TypeName)
{
if (_Hash.ContainsKey(TypeName))
return (ICoercionStrategy)_Hash[TypeName];
else
{
ICoercionStrategy _Strategy = BuildStrategy(TypeName);
_Hash.Add(TypeName, _Strategy);
return _Strategy;
}
}

private ICoercionStrategy BuildStrategy(string TypeName)
{
switch (TypeName)
{
case "System.Int32":
return new IntegerCoercionStrategy();
case "System.Int64":
return new LongCoercionStrategy();

case "System.DateTime":
return new DateCoercionStrategy();

case "System.Double":
return new DoubleCoercionStrategy();

case "System.String":
return new StringCoercionStrategy();

default:
return new CoercionStrategy();
}
}
}

The flywheel and factory patterns are often combined with families of small strategy classes like ICoercionStrategy.

Builder Pattern

The Builder pattern separates the construction of a complex object into separate classes. The result is that the construction algorithm is independent of the complex object or any of its subclasses. In our framework, the ObjectMap class is only effective when it has the correct configuration of child members, but ObjectMap has no knowledge of how to configure itself. By putting all of this construction logic into ObjectMapBuilder instead, we can keep ObjectMap simple. ObjectMapBuilder can also build different ObjectMap configurations.

In our case, using the Builder pattern eases the effort of switching database engines. The implementation of the CommandBuilder class in this case study is dependent on SQL Server. In all cases, we use the generic "IDb*" interfaces in the mapping classes instead of the SQL Server specific "Sql*" interfaces. If we need to modify the framework to work with Oracle or DB2, we can rebuild a new version of CommandBuilder and the ObjectMap/BatchFetcher hierarchies would be unaffected!

CommandBuilder

The CommandBuilder class is a builder class that configures IDbCommand objects. In its constructor function, CommandBuilder creates an ADO.NET DataSet with two related tables, one with a list of all procedures used in a mapping configuration and a second table containing all the parameter configuration information for each procedure.

The original intention behind CommandBuilder, and all of the other Builder classes in the framework, was to cache all of the configuration data in memory on the first call to the class. Subsequent calls to a builder class would use the cached DataSet for speedy access. In the prototype application that included this framework, I was placing the DataSet in a static member of the builder class and using COM+ object pooling to pin the builder classes in memory. In the case of a very large application, the sheer number of stored procedures and objects may require a change in caching strategy.

The DataSet is built by using the following two views that retrieve all of the relevant information.

CREATE VIEW OR_PROCEDURE_VW
AS
SELECT DISTINCT PROCEDURE_NAME FROM OR_MAP
UNION
SELECT DISTINCT PROCEDURE_NAME FROM OR_BATCH_FETCH

CREATE VIEW OR_PARAMETER_VW
AS
SELECT
sysobjects.name AS PROCEDURE_NAME,
syscolumns.length,
syscolumns.isoutparam AS IS_OUTPUT,
syscolumns.type AS DATA_TYPE,
syscolumns.name AS PARAMETER_NAME
FROM
sysobjects INNER JOIN syscolumns ON sysobjects.id = syscolumns.id
WHERE
sysobjects.name IN (SELECT PROCEDURE_NAME FROM OR_PROCEDURE_VW)
ORDER BY
sysobjects.name,
syscolumns.colorder

When a client requests a command by calling the GetCommand() method, CommandBuilder does the following steps to build the new command:

  1. Find the proper row in the procedure row
  2. Create a new SqlCommand object
  3. Set the command text
  4. Set the command type to a stored procedure.
  5. Iterate through the parameter rows. For each row, call BuildParameter() to create an SqlParameter object based on the row and append the new parameter to the command.

public IDbCommand GetCommand(string ProcedureName)
{
SqlCommand _Command = new SqlCommand(ProcedureName);
_Command.CommandType = CommandType.StoredProcedure;
DataRow _CommandRow = tableProcedures.Rows.Find(ProcedureName);

foreach (DataRow ParameterRow in _CommandRow.GetChildRows(FK))
{
BuildParameter(ParameterRow, _Command);
}

return _Command;
}

private void BuildParameter(DataRow ParameterRow, SqlCommand Command)
{
string _ParameterName = ParameterRow[colParameter].ToString();
int _ParameterType = (byte)ParameterRow[colType];

SqlParameter _Parameter;

switch (_ParameterType)
{
case 39:
int size = (System.Int16)ParameterRow[colLength];
_Parameter = new SqlParameter
(
_ParameterName, System.Data.SqlDbType.VarChar, size
);

break;

case 108:
_Parameter = new SqlParameter
(
_ParameterName,
DbType.Int32
);
break;

default:
throw(new Exception("Unknown Parameter Type (" + _ParameterType + ")"));
}

// Determine the Direction
int _Direction = (int)ParameterRow[colIsOutput];
if (_Direction == 1)
_Parameter.Direction = ParameterDirection.Output;

Command.Parameters.Add(_Parameter);
}

ObjectMapBuilder and BatchFetchBuilder work very similarly. Both builder classes use CommandBuilder, CoercionStrategyFactory, and ObjectMemberFactory to get class members.

Compare the sample code in OneOfOneTest.cs that builds an ObjectMap instance manually versus the exact same functionality in OneOfOneTest2.cs. Using the builder classes dramatically simplifies the usage of the ObjectMap class, but it is still too much code. What we really want is a domain object to load itself in a constructor function, but not be tightly coupled to the data layer. Why would we want to have to go through so many repetitive steps of getting and executing ObjectMap objects and managing transaction boundaries and deal with so many little classes, especially when it will be entire hierarchies of objects at once?

Again reaching into the bag of design patterns, we will complete the framework by using:

  1. The Façade pattern to simplify the usage of the framework
  2. The Layer Supertype pattern to easily hook our domain classes into the framework
  3. The Visitor pattern to enable an entire hierarchy of objects to be persisted in a single transaction

Building the Framework Classes

So far we have a lot of building blocks, but no framework yet. We still need services to relate objects, manage transactions, track the status of the persistent entity objects, and provide an implementation of the Object Identifier pattern to uniquely identify our objects. In this section we will be building the following class structure to manage and expose the functionality of the mapping and builder classes we created above.

PersistenceManager

The first step in the design of framework is to define a simple, unified interface to all the services of the framework called the PersistenceManager. This class will be the top most level class in our framework. It provides a means to load or persist any class with one method call. As we will see later, the PersistenceManager will also persist an entire hierarchy of entity objects with one call. The PersistenceManager class is a lot like most IT managers, in that it does no real work, but instead is responsible for delegating work to the proper subsystem classes like ObjectMap and BatchFetcher.

Façade Pattern

The PersistentManager class is an example of a Façade Pattern. The intent of using a façade is to provide a high level interface to a complicated group of subsystem classes. The goal is to hide the details of the subsystem members from a client. In this case, a client only has to use PersistentManager to persist objects. The client is not exposed to the myriad array of builder, strategy, and mapping objects that are doing the actual work. Façade also allows you to tightly control the entry points into a layer of your application. By only exposing PersistentManager to clients, we are loosely coupling the persistence framework from the rest of the application, leading to easier maintenance later.

Relating Objects

Object instances are rarely useful by themselves. Much of the power of object-oriented programming is in forming relationships between entity objects. The framework needs to able to build these relationships on the fly and also relate parent identifiers to child objects when persisting an object hierarchy. When persisting a hierarchy, it will also be paramount to order the operations so that a parent object will always be persisted before a child object to maintain relational integrity in the database. Also, not every object in a hierarchy will be modified in any given transaction. We obviously need a way to track the transactional state of each object to avoid unnecessary calls to the database.

It would simplify the construction of the framework if each entity class could expose a common way to identify itself (the Object Identifier or OID pattern) and a mechanism for navigating from parent to child. The navigation will be accomplished through a variation of the Visitor Pattern. The commonality between entity classes can be accomplished by using the Layer Supertype Pattern .

Visitor Pattern

The Visitor pattern is an odd beast. Visitor enables you to define operations across an entire object hierarchy without affecting the hierarchy. The actual Visitor class would be passed into each object in the hierarchy. The visited object would accept the Visitor and immediately call an operation on the Visitor and pass in itself as the only argument. In the variation we will be using, the visited class will be responsible for passing on the Visitor to all of its children classes.

For our framework we are only using the Visitor pattern to order the persistence of a hierarchy. In a nontrivial order management system, you could easily need to add additional Visitor types to perform operations like "Validate" or "Display" that would require some sort of operation across the entire hierarchy and reuse the Visitor/Visited Object infrastructure.

For an example, think of a large multigenerational family at a family reunion at a park. The family has hired a PhotographerVisitor to take individual portraits of each family member. To ensure that no family member is left out, they come up with a system by which the PhotographerVisitor is introduced to a GrandParent first. The GrandParent asks the PhotographerVisitor to take their picture, and then introduce the PhotographerVisitor to each of their Parent children. The Parents follow the same pattern of posing for the portrait and then introducing the PhotographerVisitor to their collection of Child members. This works well. The next week a PhoneSolicitorVisitor contacts a GrandParent and offers them an amazing discount plan on calls to their family, but the family must also join the plan. The GrandParent promptly signs up for the plan and uses the system from the family reunion to send the PhoneSolicitorVisitor to the Parent children and so forth.

To support the Visitor pattern we will define two interfaces, IVisitor and IVisitable. The IVisitor interface defines two methods, addReference() and endReference(). As we will see later, the dual call into IVisitor facilitates tracking child to parent references to get foreign key values without using the dreaded recursive reference.

The interplay between these interfaces looks like the following:

Layer Supertype Pattern

We have already determined that the entity classes need to have a common way of exposing their identity and transactional state. The entity classes will also need to support the Visitor mechanisms. For the purpose of loading state from a constructor function, the entity classes will need access to PersistenceManager. We do not want to place repeated code in the entity classes themselves that support all of the persistence calls because this defeats the purpose of separating the code in the first place.

We can finesse this issue by using the Layer Supertype pattern to create an abstract class called DomainEntity that encapsulates all of the common functionality. The Layer Supertype pattern is just the usage of an abstract class to collect the common features of a layer. Each class in the layer must inherit from the supertype class.

The code behind DomainEntity class is shown below. The class also implements some of the structure to support passing an IVisitor to its children members. To fully support the Visitor, a subclass just has to override the GetVisitableChildren() method to present a list of its members. DomainEntity also enables an object to be loaded from the database if the OID value is known. Subclasses will inherit this functionality automatically (you still need to create the data structures and define the mapping for each particular action and object).

Each object will track its own transactional status in the State property. An entity object will need to set its State to MappingActionEnum.Update in order to be persisted later. The framework will handle other state changes automatically. There are other alternatives, as this can be burdensome and even easy to foul up. I did this primarily to enable dynamically passing object hierarchies between multiple transaction controllers. If this case does not exist it may be easier to set the State property external to the entity objects.

public abstract class DomainEntity :
            IVisitable
{
private MappingActionEnum _State;
private long
            _PK;

void IVisitable.acceptVisitor(IVisitor
            Visitor)
{
Visitor.addReference(this);

if (_State !=
            MappingActionEnum.Delete)
{
IVisitable[] children =
            GetVisitableChildren();
foreach (IVisitable child in
            children)
child.acceptVisitor(Visitor);

Visitor.endReference();
}
}

protected
            void Load()
{
if (this.State !=
            MappingActionEnum.Insert)
{
PersistenceManager _Manager = new
            PersistenceManager();
_Manager.Load(this);
}
}

public
            DomainEntity(long PrimaryKey)
{
this.PrimaryKey =
            PrimaryKey;
Load();
}

public long
            PrimaryKey
{
get {return _PK;}
set {_PK =
            value;}
}

protected virtual IVisitable[]
            GetVisitableChildren()
{
IVisitable[] children = new
            IVisitable[0];
return children;
}

public
            MappingActionEnum State
{
get
{
if (this.PrimaryKey
            <= 0)
return MappingActionEnum.Insert;
else
return
            _State;
}

set
{
_State =
            value;
}
}

protected void markUpdated()
{
_State
            = MappingActionEnum.Update;
}

}

VirtualProxyCollection

Let us say we have two DomainEntity subclasses, a parent and a child. When we load a parent object we want to be able to start using child objects without any additional calls. For example, consider the following class structure of our Order Management System:

When we fetch an existing Customer object by calling the constructor function we want to be able to retrieve children Order objects from Customer immediately.

Customer oldCustomer = new Customer(1);
Order oldOrder = oldCustomer.Order(1234);

We will be building the abstract VirtualProxyCollection class as a Template Pattern class for collections in the persistence framework. The VirtualProxyCollection will provide the basis for using a BatchFetcher class to build a hashtable of child objects as the parent object is initialized.

But what if we do not need to access the child Order objects in a given transaction? In this case it would be wasteful to make a database round trip and build objects that will never be used. Instead, what if we could quietly fetch the children objects when they are first requested, without the parent object being aware? There is an OO design pattern for just this need, the Virtual Proxy Pattern.

Virtual Proxy Pattern

The Virtual Proxy class is a placeholder that builds expensive objects on demand. This is also known as Lazy Fetch or Lazy Materialization. Regardless, the purpose is to have a way to retrieve and configure objects only when they are absolutely needed. Using a collection class is a relatively simple way to do a Virtual Proxy. The parent object has a reference to the proxy collection object. The proxy collection object will only load the collection when a child member(s) is requested, otherwise the proxy collection object does nothing. The parent object is blissfully unaware of the mechanics of the proxy collection.

Template Pattern

The Template Pattern is the foundation of most object-oriented frameworks. The Template class is usually an abstract class that defines the skeleton of an algorithm and the implementation of the unvarying parts of the algorithm. Particular algorithms are built by subclassing the Template superclass and overriding the abstract methods to fill in the missing pieces.

Building VirtualProxyCollection and an OrderCollection Subclass

We can define a class structure for VirtualProxyCollection (shown below) with a subclass for the OrderCollection class that retrieves child Order objects for a Customer parent object.

The algorithm for retrieving a batch is shown in the sequence diagram below.

VirtualProxyCollection.Build() Steps

  1. Call GetBatchName(). This protected member must be overridden by a subclass. The value returned is just the name of a BatchFetcher configuration.
  2. Use an instance of PersistenceManager to get an instance of BatchFetcher with the configuration specified by GetBatchName().
  3. Call BatchFetcher.Fetch(Parent), passing in the reference to the parent DomainEntity object of the VirtualProxyCollection. The BatchFetcher object will use the parent for the input parameters.
  4. Until BatchFetcher.Read() is false,
  5. Call GetNewEntity() to return a new instance of the class that is aggregated by VirtualProxyCollection. This is an abstract member that must be overridden by a subclass.
  6. Call BatchFetcher.Load() to load the new object instance.
  7. Call AddChild() to add the new object to the internal Hashcode structure. By default this adds the object using the DomainEntity.PrimaryKey property as the key in the Hashcode. This method can be overridden by a subclass to use a different property for the key.

Building a Subclass of VirtualProxyCollection

The following code is the OrderCollection subclass of VirtualProxyCollection. Note that a grand total of two methods comprising a total of two lines of code are all that is absolutely necessary to build a subclass. After overriding GetBatchName() and GetNewEntity(), just add any public methods like AddOrder(), DeleteOrder(), and GetOrder().

public class OrderCollection : VirtualProxyCollection
{
public OrderCollection(DomainEntity Parent) : base(Parent)
{

}

protected override string GetBatchName()
{
return "GetOrders";
}

protected override DomainEntity GetNewEntity()
{
return new Order(0);
}

public Order GetOrder(long OrderID)
{
return (Order)this.GetObject(OrderID);
}

public void DeleteOrder(long OrderID)
{
this.Delete(OrderID);
}

public void AddOrder(Order order)
{
this.AddChild(order);
}
}

PersistenceTransaction

The final step in the framework is to build an implementation of IVisitor that manages the transactional boundaries of persisting an object hierarchy. The PersistenceTransaction class in the structure below will provide this functionality.

The sequence diagram for using PersistentTransaction to save a two-deep object hierarchy is below.

The addReference() method starts the actual work. If the object needs to be inserted, deleted, or updated based on DomainEntity.State, a call will be made to the PersistenceManager parent to retrieve the proper ObjectMap configuration. The foreign key value is determined by calling the private GetForeignKey() method. PersistentTransaction uses a System.Collections.Stack object to keep references to previous DomainEntity objects in a current transaction. If the stack has a nonzero count the GetForeignKey() method returns the PrimaryKey value of the previous DomainEntity object in the hierarchy. When an object is successfully transferred, its state is set to "Pending" and the object is added to the visited stack. Calling endReference() lets PersistenceTransaction know to pop() the last referenced DomainEntity off of the visited stack.

When all of an object hierarchy has been persisted, the FinishTransaction() method will be called to commit the database transaction. If the commit is successful, PersistenceTransaction will iterate through all of the classes that were affected by the database transaction and set their State to "Unchanged."

The actual code of PersistentTransaction is below.

public class PersistenceTransaction : IVisitor
{
private Stack _VisitedStack;
private ArrayList _PendingList;
PersistenceManager _Manager;
IDbTransaction _Transaction;
IDbConnection _Connection;

public PersistenceTransaction(PersistenceManager Manager)
{
_Manager = Manager;
}

public void StartTransaction()
{
// 10 is an arbitrary size for the initialization
_VisitedStack = new Stack(10);
_PendingList = new ArrayList(10);

_Connection = DataConfiguration.GetConnection();
_Connection.Open();

_Transaction = _Connection.BeginTransaction(IsolationLevel.RepeatableRead);
}

void IVisitor.addReference(DomainEntity VisitedObject)
{
MappingActionEnum _State = VisitedObject.State;
if
(
_State == MappingActionEnum.Delete
||
_State == MappingActionEnum.Update
||
_State == MappingActionEnum.Insert
)
{
ObjectMap _Map = _Manager.GetObjectMap(VisitedObject, VisitedObject.State);
_Map.Transfer
(_Connection, VisitedObject, GetForeignKey());

VisitedObject.State = MappingActionEnum.Pending;

_VisitedStack.Push(VisitedObject);
}
}

private long GetForeignKey()
{
if (_VisitedStack.Count == 0)
return 0;
else
return ((DomainEntity)_VisitedStack.Peek()).PrimaryKey;
}

void IVisitor.endReference()
{
_VisitedStack.Pop();
}

public void SaveEntity(DomainEntity Entity)
{
((IVisitable)Entity).acceptVisitor(this);
}

public void FinishTransaction()
{
_Transaction.Commit();
_Connection.Close();

foreach (DomainEntity _Entity in _PendingList)
_Entity.State = MappingActionEnum.Unchanged;

_PendingList = null;
_VisitedStack = null;
}
}

Example Usage of the Framework

Again we will insert and retrieve our "Rollo Tomassi" customer, but this time with the PersistenceManager. Compare this operation with the earlier samples in OneOfOneTest.cs and OneOfOneTest2.cs. Not only is the code simpler, but also there is no additional code necessary to persist or load children objects of the Customer objects!

using System;
using OMS;
using
            DataMapping;

namespace ObjectRelationalSample
{
public
            class OneOfOneTest3
{
public static void
            Main()
{
PersistenceManager _PersistenceManager = new
            PersistenceManager();

Customer customer = new
            Customer("Rollo", "Tomassi", "(555)
            555-5555");
Console.WriteLine(customer.ToString());

//
            Insert the new
            Customer
_PersistenceManager.Persist(customer);
Console.WriteLine("The
            new CustomerID is " + customer.PrimaryKey);

long CustomerID =
            customer.PrimaryKey;

// Retrieve the newly inserted
            Customer
Customer newCustomer = new
            Customer(customer.PrimaryKey);

Console.WriteLine(newCustomer.ToString());

Console.WriteLine("Press
            any key to exit.");
int i = Console.Read();
}
}
}

Porting the Framework to Different Databases

The example framework we built in this case study was built around using a SQL Server database engine. Now, what happens when we need to port the framework to using an Oracle database?

The changes might not be that extensive. All the mapping classes strictly use the generic interfaces in the System.Data.SqlClient package. We need to create a new database schema and metadata storage built around Oracle's data dictionary. The builder classes ObjectMapBuilder, CommandBuilder, and BatchFetchBuilder are bound at least in part to SQL Server. These classes would need to be altered to work with the System.Data.OleDb package. In addition, the ICoercionStrategy classes could be subtly different.

Abstract Factory Pattern

If we want the framework to work with multiple databases without alteration, we need to add another layer of abstraction. Specifically we need to use the Abstract Factory pattern to dynamically select the proper builder classes for the database being used. Say we build two or more versions of builder objects - OracleCommandBuilder and SQLServerCommandBuilder. The Abstract Factory pattern is a Factory class or method that would create the proper builder classes based on the current database engine being used. When PersistenceManager creates an ICommandBuilder object, it will now use an Abstract Factory method to retrieve a concrete OracleCommandBuilder or an SQLServerCommandBuilder depending upon the configured database. A similar process would be used to retrieve specific versions of the other builder classes and CoercionStrategyFactory.

Making the Framework Industrial Strength

If we are really going to use this framework in an enterprise application, we need to make some structural additions. Error handling and general instrumentation is conspicuously missing. One of the great advantages of investing in framework construction is centralizing the grunge work of error handling and instrumentation into the framework itself.

The framework is based on the idea of metadata driven mappings. In order to make the framework efficient and scaleable, the ObjectMap and BatchFetcher configurations must be immediately available. This may be a perfect opportunity to use COM+ object pooling to pin pre-configured objects in memory, or to at least keep the underlying metadata schema in memory.

There are several persistence scenarios that are not addressed by our simple framework. Among these scenarios is a many to many relationship with association classes. This particular issue could be addressed by subclassing the VirtualProxyCollection class to set object references in association classes upon loading. In some cases it might be easier to have a custom collection class that delegates to multiple BatchFetcher objects and sets the references of the association classes.

Many-to-many conditions lead to a very clear warning consistent in all the reading I have done on this subject - build the simplest framework you possibly can. Do not try to build a one-size-fits-all solution. It may be easier to tweak the code between projects. Think very hard before you decide to build a framework. The upfront investment only pays off for large products. A somewhat likely scenario is that there will soon be commercially available products that could be purchased off the shelf as generic object/relational mapping frameworks.

In the meantime, we know how to build our own framework.

Using the Sample Code

To use the sample framework, follow these steps:

  1. Create a new SQL Server database
  2. Run the SQL script Article.SQL
  3. In the file Mapping.CS, find the class DataConfiguration. Change the static property ConnectionString() to return a connection string to your database. In a real application this property would need to read the connection string from some sort of metadata file or an encrypted registry setting.
  4. Compile the project DataMapping.vbproj.
  5. Build your domain classes, making each class inherit from the DataMapping.DomainEntity abstract class.
  6. Build a stored procedure for every class to perform the following actions:
  7. Update
  8. Insert
  9. Delete
  10. Retrieve
  11. Create a mapping definition in the database that defines the pairing of:
  12. Class and action type to stored procedure name in the OR_MAP table
  13. Stored procedure parameter to object member pairings in the OR_MAP_DETAIL table

Case Study Review

Building an object/relational persistence framework is a great way to simplify the construction of an enterprise application. The capabilities of the C# language and the .NET runtime makes this approach possible. Using reflection makes the framework extremely extensible, as any class can be persisted. Placing the mapping configuration in an external schema and removing the SQL from the application code makes the application easier to maintain and support.

Using object-oriented design patterns is essential to building flexible systems. Design patterns are also a common vocabulary for object-oriented programmers. By calling out in our documentation or naming convention the usage of a pattern, we make very clear the code's purpose and describe the interaction between classes. By being aware of design patterns and the motivation for using them, we have tools in our toolbox to deal with reoccurring problems in software design. There is a danger of overusing design patterns upfront to over-engineer a system with unnecessary abstractions. Just do not use a pattern for the sake of using a pattern.

Case Study Information
Author | Jeremy Miller
Technical Editor | Adam Ryland
Project Manager | Helen Cuthill
Reviewers | Andrew Krowczyk, Saurabh Nandu, Venka RK, Adam Ryland

If you have any questions or comments about this case study, or suggestions for further pieces, please contact the technical editor. If you wish to talk about the case study with other subscribers, please visit our forum here.


C# to UML Reverse Engineering

C# to UML Reverse Engineering
http://www.csharptoday.com/content.asp?id=1837
Article Rating: 4.5
Published on April 29, 2002
Abstract
In this case study, Tony Loton looks at reverse engineering a C# program into a UML model, using the .NET System.Reflection classes to introspect a compiled assembly and a Rational Rose script to build an object model from the types and members that we discover. If you've been wondering about what reflection is and why you would ever want to use it, you'll see a really interesting application of that feature here. And if you're interested in reverse engineering .NET programs, you'll go away with a simple C# program / Rose script combination that will help you to do just that.
Case Study

In this case study we'll look at reverse engineering a C# program into a UML model, using the .NET System.Reflection classes to introspect a compiled assembly and a Rational Rose script to build an object model from the types and members that we discover. If you've been wondering about what reflection is and why you would ever want to use it, you'll see a really interesting application of that feature here. And if you're interested in reverse engineering .NET programs, you'll go away with a simple C# program / Rose script combination that will help you to do just that.

Introduction

You may be well used to reverse engineering C++, Java, VB or other code with a (Unified Modeling Language) CASE tool like Rational Rose, Together ControlCenter, or Select Enterprise. Have you ever thought about how the reverse engineering features of those tools work? Does your tool version not yet support the C# language, and you'd like to add that support?

While a fully-featured production-quality reverse engineering tool is a tad outside our scope, this case study will nonetheless firmly establish the principles of constructing a Unified Modeling Language Class Diagram like this:

from the information contained in a compiled .NET assembly.

From a technical perspective we'll concentrate mainly on the classes of the .NET System.Reflection namespace, but towards the end we'll look at an alternative based on the System.Text.RegularExpression classes. Oh, and you might just learn a little about the Rational Rose scripting language too.

System Requirements

To implement the C# aspect of this case study - a worthy result in its own right - you'll need the .NET Framework SDK from http://msdn.microsoft.com/netframework/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq, and I'll assume you're working from the command line. Of course, you could use Visual Studio .Net instead.

To complete the study you will need Rational Rose. I used Rational Rose '98 to develop the script, and in my experience that means you'll be able to run it in that version or any later version. If you're not currently using this tool and you'd like to complete the case study, you should be able to download a trial version at http://www.rational.com/tryit/rose/index.jsp?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq.

If you're using a different CASE tool, such as Together ControlCenter or Select Enterprise, the script code should at least provide you with some pointers for implementing a similar add-on to your tool.

Introducing UML

If you're completely new to the Unified Modeling Language (UML), and the notation as shown in the previous figure, you can find out more about it at http://www.omg.org/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq (the Object Management Group's website) or http://www.rational.com/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq (Rational Software's website), and I can recommend the book "UML Distilled" by Martin Fowler (Addison-Wesley). Modeling an application with UML is important because:

"Developing a model for an industrial-strength software system prior to its construction or renovation is as essential as having a blueprint for large building. Good models are essential for communication among project teams and to assure architectural soundness. As the complexity of systems increase, so does the importance of good modeling techniques." (taken from the Rational website)

Of course, the implication is that we produce the model before writing the code; or at least in conjunction with writing the code, in an iterative cycle. It doesn't quite work like that in the real world, and on many of the projects I've encountered there has been a large body of code that has been written before formal modeling techniques have been adopted.

How do you get that already-written code into the model? By reverse-engineering it.

Case Study Structure

Here is your in-a-nutshell roadmap showing how the case study will unfold:

  • The Defining the Problem section tells you about the problem we're looking to solve, in the form of a simple C# program to be reverse engineered.
  • The Defining the Solution section will tell you about the Reflection and Rose Script technologies that will be employed to solve the problem, and how they will interoperate.
  • The Component Design and Coding section will step through the code of the solutions, line by line.
  • The Alternative Approach using Source Code and Regular Expression section does exactly what it says.
  • The Case Study Review section summarizes what has been achieved in the case study and how the solution may be improved.

Defining The Problem

I'll define the problem in terms of a C# program that I've written for a mini banking application. It contains examples of class definitions, fields (or attributes), properties, methods and inheritance, all of which we'd like to take account of in building a UML class diagram as shown in the introduction. Here's the complete program - comprising three classes - with some explanation of how we'll map to UML.

using System;

namespace Banking
{

The class Account should become a UML Class:

public class Account
{
private static int _nextAccountNumber;

The fields should become UML Attributes of the Account class:

        private
            int _accountNumber;
private double _balance;

public
            Account()
{
_accountNumber=_nextAccountNumber;
_nextAccountNumber++;
}

These properties should become UML Attributes of the Account class:

public int AccountNumber
{
get {
            return _accountNumber; }
}

public double Balance
{
get { return _balance; }
set { _balance=value;
            }
}

}

The SavingsAccount class should become a UML Class that specializes (inherits from) the Account class:

public class SavingsAccount : Account
{

The applyInterest() method should become a UML Operation on the SavingsAccount class:

public double applyInterest(double rate)
{
Balance=Balance*rate;
return Balance;
}
}

The CurrentAccount class should become a UML Class that specializes (inherits from) the Account class:

public class CurrentAccount : Account
{

The linked SavingsAccount should be modeled as a UML Association:

private SavingsAccount _surplusAccount;

The directdebit() method should become a UML Operation on the CurrentAccount class:

public double directdebit(double
            amount)
{
Balance=Balance-amount;
return
            Balance;
}
}

}

Now try to relate that code back to the diagram and you'll fully appreciate the problem that we're looking to solve.

For completeness, before we move on I'll just mention that the Account class has one more method, Main(), that I wrote to prove to myself - and you - that the code does actually do something. So you can compile the source file Banking.cs using the command:

csc Banking.cs

Don't worry about warning messages of the form Banking.cs(58,32): warning CS0169: The private field 'Banking.CurrentAccount._surplusAccount' is never used, and when you run it you will see as output:

Current Account 0 has balance 100
Current Account 1 has balance 50
Savings Account 2 has balance 1100

So there we have a functional, if simplistic, application that has a fairly interesting internal structure which we can now set about reverse engineering.

Defining The Solution

We'll concentrate on the System.Reflection classes as the basis of our solution, thus allowing us to introspect a compiled .NET assembly. This means that we need not worry if the original source code is no longer available, and also means that our one-size-fits all solution should work just as well for VB.NET and C++ with Managed Extensions assemblies as it does with C# assemblies. all thanks to the Common Language Runtime system.

As an alternative approach, we could work from source code by employing some kind of parsing mechanism. This would be more difficult to implement and would be language-specific, but we would have access to some additional information - for example, comments - that would not be available using the reflection approach. For the record, I would use regular expressions to help with parsing the source code and I'll say more about that later.

Whatever technique we use for introspecting the code, we'll run our reverse engineering program (in this case to introspect the assembly Banking.dll) like this:

ReverseEngineer Banking

To produce output of the form:

T Banking.Account
S System.Object
F _accountNumber:System.Int32
F _balance:System.Double
M get_AccountNumber:System.Int32
M get_Balance:System.Double
M set_Balance:System.Void
P AccountNumber:System.Int32
P Balance:System.Double

T Banking.CurrentAccount
S Banking.Account
F _surplusAccount:Banking.SavingsAccount
M directdebit:System.Double

T Banking.SavingsAccount
S Banking.Account
M applyInterest:System.Double

Each block of text describes a class type (prefixed T) contained in the assembly, along with its super class (prefixed S), any fields (prefixed F), any methods (prefixed M), and any properties (prefixed P). Properties and fields are described in the form name:Type and methods in the form name:ReturnType.

Note that in this case study we'll not be expanding the parameters of each method, nor will we be dealing with implemented interfaces. Both are perfectly possible to do, and require no significant additional knowledge, so I'll leave those as exercises.

The intermediate file will be provided as input to our Rose script thus completing a two-stage process:

The introspection is de-coupled from the model building for maximum flexibility. We could switch from the reflection-based to regex-based approach without affecting the Rose script, and conversely we could implement additional model building scripts for other CASE tools.

The two main technologies we'll be using are .NET Reflection and Rational Rose Scripting, so before we look at the implementation code I'll just say a few words about each one.

Reflection

The .NET Framework Glossary (ms-help://MS.NETFrameworkSDK/sdkstart/html/cpglor.htm) defines reflection as:

"The process of obtaining information about assemblies and the types defined within them, and creating, invoking, and accessing type instances at run time."

Reflection is not unique to the .NET framework, and is a feature of some other languages and platforms, so I found an alternative definition within the Java SDK documentation that reads:

"Reflection enables code to discover information about the fields, methods and constructors of loaded classes.The API accommodates applications that need access to. the members declared by a given class."

That's a better definition for this case study because we'll be doing exactly what it says; discovering information about fields and methods - i.e. the members - of loaded classes. What's important from the first definition, however, is that we'll be obtaining the information from assemblies.

Reflection support is provided in the .NET framework by the classes of the System.Reflection namespace, and we'll be making use of these classes:

  • System.Reflection.Assembly: representing the source assembly that we're reverse engineering, and providing access to the Types contained within.
  • System.Type: strictly not a reflection class, but a Common Language Runtime class representing a program class, interface, or fundamental (e.g. Array) type in an assembly. It provides access to the type Members.
  • System.Reflection.MemberInfo: representing a member of a given Type, the member being a method, property or field. A MemberInfo instance may be cast to one of the specific instances that follow.
  • System.Reflection.MethodInfo: providing information about a method member, its return type and parameters.
  • System.Reflection.FieldInfo: providing information about a field member and its type.
  • System.Reflection.PropertyInfo: providing information about a property member and its type.

Rational Rose Scripting

Rational Rose has been one of the most widely used Object Modeling CASE tools in recent years, and as far back as I can remember it has had a scripting language that provides programmatic access to the Rose application and the model(s) housed within it.

The scripting language is called Summit BasicScript. If you're familiar with early versions of Visual Basic (many visitors to this site, I suppose) you'll be quite comfortable with it. Access to the UML model (actually the meta-model) is provided by the Rose Extensibility Interface (REI) classes that represent the model elements - classes, methods and so on - as well as the UML diagrams.

We'll make use of those classes via the scripting language, but the same set of classes are available for manipulation from your favorite Microsoft language via the OLE/COM objects of Rose Automation. If that's your preference it shouldn't be too difficult to convert the script to a VB program, or for that matter even a C# program.

You can find more information about the scripting language and the extensibility classes in the Rose Entensibility Interface / Summit BasicScript Reference and Rose Extensibility Interface / Rose Extensibility Reference Rational Rose help topics.

Component Design and Coding

Our solution comprises two components: a C# program called ReverseEngineer.cs and a Rational Rose script called BuildModel.ebs.

ReverseEngineer.cs (C# Introspection Program)

using System;

We'll be using the System.Reflection classes, so let's import them:

using System.Reflection;

Our ReverseEngineer class will be in the namespace reverse:

namespace reverse
{
public class ReverseEngineer
{

We'll run it via the Main(.) method:

public static void Main(string[] args)
{
if (args.Length>0)
{

Here we read the user-supplied assembly name, specified without the .dll extension, and load the assembly from the application directory or the global assembly cache using the LoadWithPartialName() method:

string assemblyName=args[0];
Assembly sourceAssembly
Assembly.LoadWithPartialName (assemblyName);

if (sourceAssembly==null) return;

Before we move on, I'll just mention that there's another way to load an assembly, using the LoadFrom() method. The difference is that LoadWithPartialName() loads the named assembly from the application directory or the global assembly cache, whereas the LoadFrom() method takes a complete file path including the .dll extension.

Now we get the types - i.e. classes - contained within the assembly, and we print each one out along with its super class (. BaseType).

Type[] types=sourceAssembly.GetTypes();

foreach (Type thisType in types)
{
System.Console.WriteLine("\nT "+thisType);
System.Console.WriteLine("S "+thisType.BaseType);

and call a separate method to list the members of this type / class.

listMembers(thisType);
}
}
}
}
}

If you've not yet worked with reflection you might be surprised at how easy that was. Now the listMembers(.) method is implemented as follows. It takes in one of the Type s that we discovered in the assembly.

public static void listMembers(Type thisType)
{

Next we find out what members - that is fields, properties and methods - are defined on this type. We're combining various BindingFlag s bit masks to specify which members we're interested in, and for this case study it's limited to the public and non-public instance members. Keep in mind the fact that you could ask for static members too, by including an additional BindingFlag.

MemberInfo[] members=thisType.GetMembers(BindingFlags.Public
| BindingFlags.NonPublic
| BindingFlags.Instance);

Now step through the members.

foreach (MemberInfo thisMember in members)
{

The next line is quite important, and interesting. It ensures that we list this member only if it was actually defined on this class, i.e. not rolled down from the super class. We'll output the inherited members when we encounter the super class itself, and a UML generalization line will show the super class / subclass relationship.

if (thisMember.DeclaringType==thisMember.ReflectedType)
{

Alternatively, we could have fetched only declared members in the first place by including BindingFlags.DeclaredOnly flag.

In the following switch statement we output the member name and type, whether a method, field or property. Notice how in each case we cast the Member instance to a concrete MethodInfo, PropertyInfo or FieldInfo instance.

switch (""+thisMember.MemberType)
{
case "Method":
{
MethodInfo methodInfo=(MethodInfo) thisMember;
System.Console.WriteLine("M "
+thisMember.Name+":"+methodInfo.ReturnType);
break;
}
case "Field":
{
FieldInfo fieldInfo=(FieldInfo) thisMember;
System.Console.WriteLine("F "
+thisMember.Name+":"+fieldInfo.FieldType);
break;
}
case "Property":
{
PropertyInfo propertyInfo
=(PropertyInfo) thisMember;
System.Console.WriteLine("P "+thisMember.Name
+":"+propertyInfo.PropertyType);
break;
}
}
}
}
}

That's all there is to it as far as reflecting the contents of an assembly goes. To remind you, we'll see as output something like this:

T Banking.CurrentAccount
S Banking.Account
F _surplusAccount:Banking.SavingsAccount
M directdebit:System.Double

T Banking.SavingsAccount
S Banking.Account
M applyInterest:System.Double

I chose that format for representing the structure of the source assembly because it's relatively simple to interpret in the Rose script. At this point you might like to think about alternative representations; possibly as XML, and more specifically the XML Metadata Interchange (XMI) format designed specifically for exchanging object models.

BuildModel.ebs (Rose Model Building Script)

For completeness, I'll step through the code of the Rose script as I did for the C# code. For those of you who are using Rational Rose, which will be many of you, this will provide you with a handy "take away" component and all the information you need to adapt it for your needs. Even if you're not using Rose, the principles should be interesting nonetheless.

The script has a Main routine that asks for a file name (not the name of an assembly DLL, but the name of a file containing the output from the C# ReverseEngineer program) to reverse-engineer. When a filename has been selected, the BuildModel subroutine is called:

Sub Main
FileName$ = OpenFileName$ ("Reverse Engineer", "All files:*.*")
If FileName$ <> "" Then
BuildModel FileName$
End Sub

Not too taxing so far, I hope, particularly if you're familiar with early versions of Visual Basic. Now it gets more interesting inside the BuildModel subroutine.

Sub BuildModel (FileName As String)

Here we open the console window to show printed messages:

Viewport.Open

Now we get a reference to the current object model:

Dim MyModel As Model
Set MyModel = RoseApp.CurrentModel

Classes are contained within packages, which in Rose are called categories. We create a category (or package) called ReverseEngineeredModel to hold our reverse-engineered classes, if it does not already exist:

catName$="ReverseEngineeredModel"
If MyModel.RootCategory.Categories.FindFirst (catName$) = 0 Then
Set NewCategory = MyModel.RootCategory.AddCategory (catName$)
End If

Whether we created the category or it already existed, either way we get hold of it from the model:

i%= MyModel.RootCategory.Categories.FindFirst (catName$)
Dim targetCategory As Category
Set targetCategory=MyModel.RootCategory.Categories.GetAt(i%)

As we process the entries in the input file we'll always know which is the current class (or Type in reflection terminology):

Dim currentClass As Class

Now open the input file, read each line, and print it:

Open FileName$ For Input Access Read As #1

While Not EOF (1)
Line Input #1, inString$
Print inString$

Only process non-blank lines:

If Len(inString$) >0 Then

Get the leftmost character of the line as the inputType and the remainder of the line as the inputText:

inputType$=Left$(inString$,1)
inputText$=Right$(inString$,Len(inString$)-2)

Are we dealing with a new Type/Class (T), or the current class's Super class (S)? Or is it a Method (M), Field (F) or Property (P)?

Select Case inputType$

It's a new type so we'll add it as a class in the model, or get hold of it if it already exists:

Case "T"
className$=inputText$
i%= targetCategory.GetAllClasses().FindFirst (className$)
If i%=0 Then
Set currentClass=targetCategory.AddClass (className$)
Else
Set currentClass=targetCategory.Classes.GetAt(i%)
End If

It's a super class, so add it to (or fetch it from) the model and form an inherit relationship with the current class:

Case "S"
superClassName$=inputText$
i%= targetCategory.GetAllClasses().FindFirst (superClassName$)
If i%=0 Then
Set superClass=targetCategory.AddClass (superClassName$)
Else
Set superClass=targetCategory.Classes.GetAt(i%)
End If

Set theInheritRelation=currentClass.AddInheritRel("",superClassName$)

It's a method, so add it as an operation on the current class:

Case "M"
methodName$=Item$(inputText$,1,1,":")
methodType$=Item$(inputText$,2,2,":")
Set theOperation=currentClass.AddOperation(methodName$,methodType$)

It's a field or property, which we add to the current class as a simple attribute if it comes from the System namespace. If the field or property refers to an object type not in the System namespace, we create (or find) a class for the field / property type and add an association relationship with the current class:

Case "F","P"
fieldName$=Item$(inputText$,1,1,":")
fieldType$=Item$(inputText$,2,2,":")

If Left$(fieldType$,7) = "System." Then
Set theAttribute=currentClass.AddAttribute(fieldName$,fieldType$,"")
Else
associateClassName$=fieldType$
i%= targetCategory.GetAllClasses().FindFirst (associateClassName$)
If i%=0 Then
Set superClass=targetCategory.AddClass (associateClassName$)
Else
Set superClass=targetCategory.Classes.GetAt(i%)
End If

Set theAssociation=currentClass.AddAssociation(fieldName$,associateClassName$)
End If

End Select

End If

Wend

End Sub

In that code I made some assumptions about how the assembly contents should be mapped to UML notation. Here is the mapping that I chose, which you might like to re-think yourself.

C# to UML Mapping

  • TYPE (T): Each type is mapped to a UML class using the type's full name including the namespace, for example a class called Banking.Account. All classes are placed in the ReverseEngineeredModel category / package.
  • SUPERCLASS (S): A super class is mapped to a class in the same way as a type (above). A generalization / specialization (or "inherits") relationship is established with the current class.
  • METHOD (M): A method is mapped to a UML operation on the current class, with the method's return type as the operation return type. In this case study we're ignoring the method parameters.
  • FIELD (F): Each field is mapped to an attribute of the current class (for a type in the System namespace) or as an association to a separate class of the field type.
  • PROPERTY (P): In this implementation a property is treated in exactly the same way as a field.

My mapping of fields is interesting because I've chosen to assume a simple attribute if the field is a (built-in) System type, but a separate associated class if the field is not of a recognized .NET type. Perfectly logical to me, and probably what you'd want, but I note that in contrast, the reverse engineering feature of Visio for Enterprise Architects (bundled with Visual Studio.NET Enterprise) always maps fields onto simple attributes. I'll say a bit more about Visio later.

Sample Walkthrough

Let's walk through a sample reverse engineering task using the two components we've developed.

Step 1: Create the source assembly

In the "Defining the Problem" section I offered a simple C# Banking program as our reverse engineering source. We'll compile that program as an executable (in case you want to run it) and as an assembly.

csc Banking.cs
csc /target:library Banking.cs

Step 2: Reverse Engineer

If you've not compiled the ReverseEngineer.cs file, you can do so now by typing:

Csc ReverseEngineer.cs

Now we can run it with the Banking.dll assembly as input with the command:

ReverseEngineer Banking > reverse.txt

We've redirected the output to a file so that we can provide it as input to the next step, and that file will now contain the text:

T Banking.Account
S System.Object
F _accountNumber:System.Int32
F _balance:System.Double
M get_AccountNumber:System.Int32
M get_Balance:System.Double
M set_Balance:System.Void
P AccountNumber:System.Int32
P Balance:System.Double

T Banking.CurrentAccount
S Banking.Account
F _surplusAccount:Banking.SavingsAccount
M directdebit:System.Double

T Banking.SavingsAccount
S Banking.Account
M applyInterest:System.Double

Step 3: Building the Model

We launch Rational Rose and open the script BuildModel.ebs using the Tools / Open Script menu option. Or you could simply double-click the BuildModel.ebs file to launch Rose with the script already displayed.

With the script displayed in the editing window, we run it using the green Start button:

When the file dialog appears, we choose the reverse.txt file to which we redirected the output of the ReverseEngineer program:

The Viewport window displays the result of the script's Print statements, thus showing the lines of the input file that have been processed:

The other result is an object model, based on the reverse engineered code, which you can see in the Rose model browsing window. It comprises a ReverseEngineeredModel category (or package) containing the types (classes) and members (methods, fields and properties) from the assembly.

To really see the value in what we've done, you can open the Main class diagram and drag the classes onto it, giving:

Notice in particular how the super class / subclass inheritances are shown, along with the _surplusAccount association between the Banking.CurrentAccount and the Banking.SavingsAccount classes.

Alternative Approach using Source Code and Regular Expressions

I hinted earlier that there was an alternative approach to the first stage of reverse engineering. This second approach would work by parsing the original source code rather than by reflecting the contents of a compiled assembly. For your information, the reverse engineering feature of the Visual Studio .Net Enterprise Architect works from source code rather than from compiled assemblies.

If you've read some of my other articles you'll have guessed that I'm a big fan of regular expressions, and whenever I hear the word parsing I think of them. I won't provide a tutorial here as you can find one in my "Introducing .NET Regular Expressions with C#" article at http://www.csharptoday.com/content.asp?id=1786&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq.

What I will do is demonstrate that in principle we could reverse engineer from source code, and that utilizing the classes of the .NET System.Text.RegularExpressions namespace would be one way of going about it. Anything you learn here will be strictly by example.

RegexReverseEngineer.cs (C# Source Code Parsing)

using System;

We'll import System.IO (so that we can read the source file) and System.Text.RegularExpressions (so that we can work with regular expressions):

using System.IO;
using System.Text.RegularExpressions;

namespace reverse
{
public class RegexReverseEngineer
{

Our Main() method simply reads the source file supplied as an argument and passes the contents into the parseSource() method:

        public
            static void Main(string[] args)
{
String
            inputString="";
String inputLine;

StreamReader sr =
            File.OpenText(args[0]);

while
            ((inputLine=sr.ReadLine())!=null)
inputString=inputString+inputLine;

sr.Close();

parseSource(inputString);
}

The following parseSource() method is where all the work takes place:

public static void parseSource(String text)
{
string currentNameSpace="";

Our regular expression will expect to see the start of the file or a semicolon or a closed curly brace or an open curly brace, followed by zero or more spaces:

string regex=@"(^|;|\}|\{)\s*";

Then it will optionally expect to see the word public or private, which we'll refer to as <visibility>:

regex=regex+@"((?<visibility>(public|private))\s+)?";

Followed optionally by the word static, which we'll call <scope>:

regex=regex+@"(?<scope>(static)\s+)?";

Next we'll optionally expect to see the word class or namespace, which we'll refer to as <element>:

regex=regex+@"((?<element>(namespace|class))\s+)?";

After which there must be one or more word characters that will be the data <type>, such as SavingsAccount, int or double:

regex=regex+@"(?<type>\w+)";

And optionally one or more of word character or underscore plus maybe an open brace (for a method) as the <name>. Note that if the <element> is "class" there will be a type (like "SavingsAccount") but no name, whereas a method will have a type (like "double") and a name (like "applyInterest"):

regex=regex+@"(\s+(?<name>(\w|_)+\(?))?";

Now set up the Regex (regular expression) instance and match it against the source text:

Regex reverseRegex=new Regex(regex);
Match reverseMatch = reverseRegex.Match(text);

Step through the matching text segments from the source:

for ( ; reverseMatch.Success;
reverseMatch = reverseMatch.NextMatch())
{

Extract the contents of the visibility, element, scope, type and name named groups from the match:

string visibility=""+reverseMatch.Groups["visibility"];
string element=""+reverseMatch.Groups["element"];
string scope=""+reverseMatch.Groups["scope"];
string type=""+reverseMatch.Groups["type"];
string name=""+reverseMatch.Groups["name"];

if (type.Length>0)
{

We must at least have a type entry. More specifically have we found a namespace?

                    if
            (element.Equals("namespace"))
{
//
            NAMESPACE
currentNameSpace=type+".";
}

Or have we have found a class that we can output as a Type (T)? If so, we'll write it out with the current namespace prefixed:

else if (element.Equals("class"))
{
// CLASS
System.Console.WriteLine("\nT "+currentNameSpace
+type);
}

Or have we found a method that we can output as M?

                    else
            if (type.Length>0 && name.EndsWith("("))
{
//
            METHOD
name=name.Replace("(","");
System.Console.WriteLine("M
            "+name+":"+type);
}

Or have we found a variable that we can output as a Property (P) or Field (F)? Actually we won't distinguish the two, therefore we simply assume a property.

                    else
            if (type.Length>0 && name.Length>0)
{
//
            PROPERTY

The following IF statement is a fudge because the regular expression is not perfect:

                        if
            (!type.Equals("return") &&
            !type.Equals("using"))
{

If the type contains no dot ( "Account" rather than "Banking.Account") we'll prefix the current namespace, but we won't do that if it's a simple type like int or double:

                            if
            (type.IndexOf(".")<0 && !type.Equals("int")
&&
            !type.Equals("double"))
type=currentNameSpace+type;

System.Console.WriteLine("P
            "+name+":"
+type);
}
}
}
}
}
}
}

Results of the regular expressions approach

We can compile and run that program by typing:

> csc RegexReverseEngineer.cs
> RegexReverseEngineer Banking.cs > reverse.txt

Notice how we're including the .cs extension when specifying the source file. Here is the output, in which you'll notice that the data types extracted from source code are different from those determined by the reflection method. Look back now to see what I mean.

T Banking.Account
P _nextAccountNumber:int
P _accountNumber:int
P _balance:double
P AccountNumber:int
P Balance:double
M Main:void

T Banking.CurrentAccount
P _surplusAccount:Banking.SavingsAccount
M directdebit:double

T Banking.SavingsAccount
M applyInterest:double

Using this approach, we've not extracted the super class information, but only because I couldn't quite figure out how to build that into the regular expression. It's perfectly possible though!

The output listed above is in exactly the right format for our Rose script, so we'll run that script again, but not until we've made a small change to one line as follows:

If Left$(fieldType$,7) = "System." Or fieldType="int" Or fieldType="double"

Here is a picture of the resulting model. Similar enough to the reflection-based model to show the validity of the alternative approach, but different enough to give you something to think about.

That's it. We started with some C# code and ended up with a UML model presented as a class diagram, and all we needed were two simple components: a C# program (using reflection or regular expressions) and a Rational Rose script. Let's review what we've achieved and look into where we could take it next.

Case Study Review

We set out with the aim of building a UML model from a C# program, a task which I called "reverse engineering", and I presented a solution in the form of a two-stage process:

For the first stage I suggested .NET Reflection as the preferred approach for determining the internal structure of a C# program, its type and its methods. Why? Because it's quite easy to work with and potentially useful for .NET languages other than C#. For the record, I have tested that claim by running the ReverseEngineer program against some VB.NET and C++ with Managed Extensions assemblies as follows:

>ReverseEngineer CompVB

T CompVB.StringComponent
S System.Object
F StringSet:System.String[]
M GetString:System.String
M get_Count:System.Int64
P Count:System.Int64

>ReverseEngineer CompVC

T CompVC.StringComponent
S System.Object
F StringsSet:System.String[]
M GetString:System.String
M get_Count:System.Int32
P Count:System.Int32

T $ArrayType$0x76a8870b
S System.ValueType

Notice the differences in some of the member types, and in case you're wondering, I can tell you that those assemblies came from the "Platform Introduction" tutorial sample that ships with the .NET Framework SDK.

For the second stage of the process I offered an implementation in the form of a Rational Rose script, which you could make use of immediately..if you have that CASE tool. If you're using another modeling tool, that script may nonetheless be useful as an example for your own tool-specific script or add-in. In particular, you might like to think about the Assembly-to-UML mapping I've used and perhaps improve on it.

Having established a working two-stage process for reverse engineering, we then looked at an alternative implementation of the first stage. That time using regular expressions to parse the original source code.

Comparison of Reflection vs. Regular Expressions approaches

As you have no doubt guessed, I prefer the reflection approach because in principle we can use it to reverse engineer any assembly DLL created in any .NET language, and we're not restricted by the absence of original source code.

Where the regular expressions approach wins out, I think, is in building a model that you can relate more directly to the original code. The data types are the language data types (e.g. int) rather than the .NET Common Language Runtime types (like System.Int32). With a better regular expression you could even pick out source code comments that are not available from the assembly, thus making for an even more meaningful model.

Use in the Real World

If you're thinking this is just a toy I can tell you that back at the office we've been working hard with these techniques to construct a UML model of the .NET Framework classes themselves, for Rational Rose and for Visio for Enterprise Architects. You've seen a few Rose models, so here's a screenshot of some System.IO namespace classes rendered in Visio.

More generally, we've created a solution for reverse engineering any .NET assembly to a UML model in Rational Rose or Visio for Enterprise Architects, using the techniques described here. You can find out more about the framework models and the reverse engineering solution at http://www.lotontech.com/visualmodeling?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq.

Limitations and Further Work

In good case study fashion, we've taken a real-world problem (most projects I've worked on have included some form of reverse engineering) and we've established a practical solution that has demonstrated the power and utility of the .NET System.Reflection or System.Text.RegularExpressions classes.

It doesn't end there as I can think of some ideas for where to take this next. Here are my thoughts.

Intermediate File Format

I kept the intermediate file format quite simple, mainly because my Rose scripting skills are not up to parsing a more complex structure such as XML. But I do think that XML would be a sensible choice for the intermediate results, particularly in the XMI variant that was devised specifically as a format for exchanging information between modeling tools.

In theory we could then do away with the Rose script completely, relying instead on the XMI import feature of our modeling tool to build a model from an XMI representation of the assembly contents.

Enhanced UML Mapping

In this case study we've established the principles of building a reverse-engineering solution, but there's more information in a source assembly that we could use in building the UML model.

For example, we've taken account of the super class of each type in the assembly, but not the interfaces that it implements. And we've taken account of the method names and return types, but not the parameters of each method. Furthermore, I've made an arbitrary decision to map System -type properties and fields onto UML attributes and non- System -type properties and fields as separate (associated) classes.

The mapping of methods has so far taken account of the method name only. From the MethodInfo instance we could get hold of the method parameters and pass those through to the model-building script too.

As for packaging, I've kept it simple by placing all reverse engineered classes in a "ReverseEngineeredModel" category with the full names (including namespace prefix) of those classes indicating their true packages. It would be better, I think, for the UML package structure to reflect the namespace structure, and that's perfectly possible because the Rose Category object may itself contain additional (sub) categories to form a proper package hierarchy.

As an exercise I'll leave you to ponder the UML mapping and possibly complete it. No additional techniques will be required, just a lot more thought plus some more reflection and Rose scripting practice.

Interfacing to other UML CASE Tools

Do you wonder why I've chosen Rational Rose as the platform for showing how we could construct a UML model, particularly as the Visual Studio .NET has a bundled UML tools in the shape of Visio for Enterprise Architects?

I think it's fair to say that Rose is still the most widely used visual modeling tool, so it should appeal to many of you, and it has an easy-to-use scripting language that provides direct access to the UML meta model. That capability is shared by tools such a Together ControlCenter and Select Enterprise, both of which provide access to the meta model via some form of scripting or a set of COM interfaces. If you're using one of those tools you should be able to figure out how to write an equivalent script to that provided for Rose.

My early investigations showed that there was no such easy route into the meta model of Visio for Enterprise Architects. It's true that you can access the contents of a Visio solution via VBA or COM, so as to manipulate the diagrams, but gaining access to the UML-specific information seems to have eluded me so far. Not a problem if we have the source code because the Visual Studio .NET integration supports reverse engineering of any .NET language program at no extra cost, but what of already-compiled assemblies?

I've concluded that the best way to reverse engineer a .NET assembly into Visio is to use the information gathered via reflection to generate a source code equivalent of the class- and method- signatures, then let Visual Studio .NET do the hard work of turning that source code into a UML model. Trust me, it can be done.

Other Uses for Reflection

I've used this case study as a vehicle for demonstrating a practical use for the reflection features of .NET. Besides reverse engineering a UML model, these reflection techniques have some other practical uses like auto-generation of JavaDoc-like API documentation. Probably better to do that from source code though, so as to capture the comments.

As more food for thought, on reflection-based applications you might like to take a look at an article that I wrote for JavaWorld, entitled "Make and EJB from Any Java Class with Java Reflection" (http://www.javaworld.com/javaworld/jw-12-2000/jw-1215-anyclass.html?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq). It presents some ideas for generating additional code - in this case EJB home and remote interfaces - by using reflection to examine an existing class.

Runtime Reverse Engineering

As implemented in many modeling tools, the term reverse engineering can be taken in a limited sense to mean static reverse engineering, and that's what we've dealt with in this case study. Start with a non-executing binary or source code file and end up with a static UML class diagram.

UML is not limited to static modeling, and the UML notation supports dynamic modeling of the execution behavior of applications. I'm thinking here of sequence diagrams and collaboration diagrams, which you normally draw up-front to describe how you expect the application to behave. Wouldn't it be good if you could reverse engineer those diagrams too, to show how the application really does behave when you run it?

I solved that problem some time ago for Java, by utilizing the Java Platform Debugger Architecture (JPDA) or - for web applications - the Servlet Filters and Event Listeners features. And now I've thought up a solution for the .NET framework that makes use of the System.Diagnostic.Trace and System.Diagnostic.StackTrace classes. I might just write it up as a case study soon.

Case Study Information
Author | Tony Loton
Technical Editor | Adam Ryland
Project Manager | Helen Cuthill
Reviewers | Andrew Krowczyk, Saurabh Nandu, Daniel Cazzulino, Adam Ryland

If you have any questions or comments about this case study, or suggestions for further pieces, please contact the technical editor. If you wish to talk about the case study with other subscribers, please visit our forum here.


An In-Depth Look at the NumberFormatInfo Class

An In-Depth Look at the NumberFormatInfo Class
http://www.csharptoday.com/content.asp?id=1754
Article Rating: not yet rated
Published on March 4, 2002
Abstract
NumberFormatInfo (NFI) is the class in System.Globalization that provides culture-sensitive formatting for currency, numbers, and percentages. Next to confusing short dates (is 11-08-90 the 8th of November or the 11th of August?), numeric values displayed without regard to user expectations can cause annoyance and misinterpretation. What does 2,103 really mean? Is it two thousand one hundred and three, or two and one hundred and three thousandths? In financial or medical software the answer can be crucial. In this article, Bill Hall explores the methods and properties of NFI to discover how it can help us to create displays that meet user requirements for a given world region.
Article

NumberFormatInfo (NFI) is the class in System.Globalization that provides culture-sensitive formatting for currency, numbers, and percentages. Next to confusing short dates (is 11-08-90 the 8th of November or the 11th of August?), numeric values displayed without regard to user expectations can cause annoyance and misinterpretation. What does 2,103 really mean? Is it two thousand one hundred and three, or two and one hundred and three thousandths? In financial or medical software the answer can be crucial. In this article, we will explore the methods and properties of NFI to discover how it can help us to create displays that meet user requirements for a given world region. Assisting us is a form written in C# that shows most of the properties and methods of the class. The program is interactive; you can change specific properties, including separators, grouping, signs, and precision, and view the results in any .NET-supported culture.

The implementation of NFI has its roots in two Win32 functions, GetNumberFormat and GetCurrencyFormat. Although useful and versatile, these two API's are not particularly convenient to apply, especially when you want to change a parameter from the standard locale offering. For example, users like to see an integer with many digits with group separators, as its value is less likely to be interpreted incorrectly. The number 210398398750 is hard to read, but 210,398,398,750 (or if you prefer 201.398.398.750) is much easier to sort out visually. In Win32, a straightforward application of GetNumberFormat groups the whole part correctly according to the locale setting, but always tacks on a fractional component as well. To eliminate the fractional part, the programmer has to fill out six fields in a NUMBERFMT structure by repeated calls to GetLocaleInfo using LCTYPES that correspond to the NUMBERFMT fields, and then set the fractional digits to zero. Using NFI, a simple property setting, NumberDecimalDigits = 0, does the trick.

An instance of NFI is a property of a specific culture instance of System.Globalization.CultureInfo. A specific culture is determined by both language and region. A parent culture, which references language but not region, contains only a null value for its NFI. The reason is that the features of NFI are dependent on a particular world region as well as language. For example, here are two rather different number formats from two countries bordering on each other and sharing a common language:

French (Switzerland) | 123'456'789'876'543'210.99
French (France) | 123 456 789 876 543 210,99

Before reading this article you may find it helpful to review our previous one about .NET's CultureInfo class in Internationalization With The .NET CultureInfo Class & C#. You may also want to look at An In-Depth Look at the DateTimeFormatInfo Class, as the two classes have similar uses.

Simple Number Formatting

It is easy to control the format of a number. Just use the variable's ToString method with one of the eight formatting letters given in Table 1. Not all data types will work with all formatting strings; for example, you cannot use the hex formatter with a decimal type. If in doubt, use a try-catch block.

The typical code is val.ToString("<formatstring>"); where val is a numeric data type such as double, integer, decimal, float, etc. Unfortunately, not all of the possible format strings are completely documented in the .NET 1.0 Help. For example, the percentage formatters "p" and "P" are missing. Also, there is no explanation of why lower and upper case forms are provided, although apparently the reason is to case certain letters in the output (see, for example, the scientific and hexadecimal results) in Table 1.

Type | Format string | Output (lower case) | Output (upper case)
Currency | C, c | $123,456,789.99 | $123,456,789.99
Integer | D, d | 123456789 | 123456789
Scientific | E, e | 1.234568e+008 | 1.234568E+008
Fixed point | F, f | 123456789.99 | 123456789.99
General | G, g | 123456789.987654 | 123456789.987654
Number | N, n | 123,456,789.99 | 123,456,789.99
Round Trip | R, r | 123456789.98765433 | 123456789.98765433
Percent | P, p | 98.77 % | 98.77 %
Hexadecimal | X, x | 75bcd15 | 75BCD15

Table 1: Formatting Characters for Numbers

Of these, only Currency, Number, and Percentage are controlled by NFI. For example, in the table above, the thread culture just happened to be en-US when the results were recorded. However, we can create a new culture, say Swiss (French), and then augment the formatting by referencing the CultureInfo's implicit NFI as follows:

CultureInfo ci = CultureInfo.CreateSpecificCulture(de-CH);
String result = Val.ToSTring("<character>", ci);

In this case, three of the outputs above now appear as shown in Table 2; the rest remain the same:

Currency | SFr. 123'456'789.99 | SFr. 123'456'789.99
Number | 123'456'789.99 | 123'456'789.99
Percent | 98.77% | 98.77%

Table 2: Swiss-French Number Formats

So, most of the time, you don't have to explicitly reference an NFI instance if the current thread's CultureInfo matches the user's requirements since CultureInfo itself contains an instance of NFI. However, if you want to obtain information unique to NFI or modify the output, then you will have to obtain one explicitly. This is very easy to do with the line of code NumberFormatInfo nfi = ci.NumberFormat; where ci is a specific (not parent) CultureInfo object. Now let's take a look at the sample application that accompanies this article to see what is provided.

NumberFormatInfo Details

Figure 1 shows the demo program with the culture set to Russian (Russia). The form itself is divided into three sections, reflecting the partition of properties and methods for Currency, Number, and Percentage. Within each section the properties themselves are shown. Also displayed are the resulting formats for number, currency, and percentage. At the bottom are some additional pieces of information about how to correctly render "not a number" and negative and positive infinity in the selected culture.

You should be aware that although the formats tend to be somewhat similar for currency, number, and percent, each should be regarded as a separate formatting task. Often, programmers tend to think of currency formatting as a special kind of number, but that is not the case. You cannot just drop in a currency symbol in front of a number expression and expect to have a culturally correct result. Rounding rules can be very different between currencies and numbers. Numbers can have fractional parts but the currency may have none (Japanese Yen, Indonesian Rupees). Placement of the currency symbol in relationship to the number is another issue -front or back - spaced or not - from the actual value. The problem becomes even more acute when working with negative values. Not all cultures use a minus sign, but if present, positioning becomes even more complicated when considering relative order of symbol, sign, and value. If you run through the listed cultures you can find sixteen ways of arranging debit currency and five ways for showing negative numbers. Even countries using the Euro have their own individual styles: sometimes the Euro symbol is in front (Italy, Austria) and sometimes behind the value (Portugal, Greece, Luxembourg, Germany, Spain, Finland, France).

Another interesting problem is how to separate whole part grouping of numbers. You may have noticed the row of nine NumericUpDown controls in the currency and number subareas (we could have also done the same for percentage as well). Many Indian cultures actually separate whole parts not simply by three's but twos and threes such as this example from Hindi -1,23,45,67,89,87,65,43,210.99. In NFI, you can actually create up to 9 possible group sizes and in any order. In the demo program you can experiment with the various possibilities. This feature is probably overkill, both in the class implementation in .NET as well as in the demo program, but it was probably just as easy to implement the general case as it was to meet the specific requirements of the Indian locales. It does, however, make writing the demo program harder!

Figure 1: NumberFormatInfo Demo

By the way, you may notice that for some cultures, the text may not look quite right, especially for the fields at the bottom of the form. In this case, you might want to add additional language support. You can do this using the Regional Settings dialog and your Windows installation CD. Just click on the additional language settings you need and then insert the CD when requested.

NumberFormatInfo Properties

The main properties that you can change are described next. Note they are different properties for each number, currency, and percent and are not shared. For example, you will find separate properties for the number of decimal digits used in formatting currency, number, and percentage, namely CurrencyDecimalDigits, NumberDecimalDigits, and PercentDecimalDigits, respectively. For simplicity in the description below, we have grouped them together by category and used a descriptive name for each. Not all are present for each number type.

Decimal Separator | The string separating whole and fractional parts
Group Separator | The string separating groups of whole part digits
Decimal Digits | An integer representing the number of fractional digits (0-99)
Currency Symbol | The string representing the currency symbol
Positive Sign | A string that denotes that a number is positive
Negative Sign | A string that denotes that a number is negative
Percent Symbol | A string that denotes a percentage value
Per Mille Symbol | A string that denotes a per mille (thousandth) value
Positive Pattern | A digit representing positioning of symbol, value, and spacing. Ranges are (0-3) for currencies and (0-2) for percentages. For example, the possible orderings are for credit currency are $n, n$, $ n, and n $, where $ represents the monetary string.
Negative Pattern | A digit representing possible positioning of symbol, sign, value, and spacing. Ranges are (0-15) for currencies, (0-4) for numbers, and (0-2) for percentages. For example, negative percentages could have the forms -98.77 %, -98.77%, or -%98.77.
Group Sizes | An array of 9 values, each ranging from 1 to 9, that describes how many elements to group for each whole part segment. The last element can be zero to indicate repetition of the next to last value.

Changing the Default Formats

NumberFormatInfo gives you a set of useful defaults formats. By using the properties of a suitable NFI you can modify the output. There may be many reasons for doing this. Let's give an example. Many European countries now use the Euro, but historical data might have to be rendered using a pre-Euro format. Here is an example from Portugal using the Euro.

Adjusting the decimal separator to ($) and the currency symbol to Escudos provides the older format:

We have already mentioned the example of providing a readable format to an integer just by adjusting the number of decimal digits. In this example, we have adjusted the number of decimal digits in a number format to none.

If we want to go wild with group sizes, try this one. Just for fun, we also changed the negative pattern so that the negation sign is at the back and spaced away from the value. We could have also changed the separators and signs. These examples should give you a good idea of what NFI can be made to do for you.

By the way, after you have completely messed us the settings in the demo you can reset them to the culture's defaults by using the Reset button!

Code Samples

There is absolutely nothing difficult about working with NFI. As for the demo form itself, the only complicated part is managing the Group Sizes display, and that is not particularly interesting, just tedious. The details of generating a specific CultureInfo list have already been discussed in An In-Depth Look at the DateTimeFormatInfo Class.

Here are some details about the test values used in the program and the code for rendering the currency, number, and monetary displays. Notice that we alternated use of upper and lower case formatting characters, but found no differences in results for the currently supported cultures. We also used the NFI explicitly in the ToString methods, i. E. curci.NumberFormat, but the code works the same if just curci is used. Here, curci is just a writeable copy of the currently selected specific culture.

private CultureInfo curci;
private
            decimal d = 123456789876543210.987654321M;
private decimal n =
            -123456789876543210.987654321M;
private double p =
            1234.987654321;
private double pn =
            -1234.987654321;

private void ShowCurrency()
            {
txtCurrency.Text = d.ToString("c",
            curci.NumberFormat);
txtNegativeCurrency.Text = n.ToString("C",
            curci.NumberFormat);
}

private void ShowNumber()
            {
txtNumber.Text = d.ToString("n",
            curci.NumberFormat);
txtNegativeNumber.Text = n.ToString("N",
            curci.NumberFormat);
}

private void ShowPercent()
            {
txtPercent.Text = p.ToString("p",
            curci.NumberFormat);
txtNegativePercent.Text = pn.ToString("P",
            curci.NumberFormat);
}

The rest of the code manages the events that occur when the user alters the selected culture or modifies the fields; most is as easy as setting or reading a property.

Conclusion

Respect your clients and customers preferences by making good use of the formatting (NumberFormatInfo, DateTimeFormatInfo) classes of CultureInfo. These classes will give you culturally correct results for your thread of execution almost without intervention or modification on your part. Use these classes right from the beginning of project development, and you'll be able to get to market without the wasteful and expensive step of an internationalization rewrite.

If you would like to read some background on the issues of internationalizing a product take a look at the two articles located here: http://www.mlmassociates.cc/thoughts_on_internationalization.htm?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq. For more information you should consult http://www.microsoft.com/globaldev?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq.

Article Information
Author | Bill Hall
Technical Editor | Adam Ryland
Project Manager | Helen Cuthill
Reviewers | Andrew Krowczyk, Saurabh Nandu

If you have any questions or comments about this article, please contact the technical editor.

C#Today - ?2002 Wrox Press Limited


An In-Depth Look at the DateTimeFormatInfo Class

An In-Depth Look at the DateTimeFormatInfo Class
http://www.csharptoday.com/content.asp?id=1827
Article Rating: 4
Published on March 1, 2002
Abstract
The DateTimeFormatInfo (DTFI) class is one of the most important members of the .NET System.Globalization namespace. Although acting mainly as the IFormattable interface for the DateTime structure, DTFI is also informational. You can, for example, find out the names of the days of the week and months of the year in native form, an amazing number of alternate date and time patterns that are acceptable in a particular culture, and which week is the first one of a new year. In addition, DTFI provides the main access to the more than half-dozen calendars supported by .NET. In this article, Bill Hall will show you how easy it is to use the facilities of DTFI, and will gain experience in writing code that functions correctly independently of language and world region.
Article

The DateTimeFormatInfo (DTFI) class is one of the most important members of the .NET System.Globalization namespace. Although acting mainly as the IFormattable interface for the DateTime structure, DTFI is also informational. You can, for example, find out the names of the days of the week and months of the year in native form, default time and date separators, ante- and post-meridiem strings for time expressions, an amazing number of alternate date and time patterns that are acceptable in a particular culture, and which week is the first one of a new year. In addition, DTFI provides the main access to the more than half-dozen calendars supported by .NET. After you read this, you will see how easy it is to use the facilities of DTFI, and will gain experience in writing code that functions correctly independently of language and world region.

System.Globalization.CultureInfo affects nearly every class in System.Globalization, and DTFI is no exception. In fact, an instance of CultureInfo normally contains a DTFI object (except in a special case, explained below). So, you may find it helpful to review our previous article about cultures in .NET in Internationalization With The .NET CultureInfo Class & C#. For an introductory article to DTFI, see Managing Culturally Correct Date and Time Formats in C#, and you may also want to glance through it for some basic information on DTFI usage in both client and web programming. Most of the material here is based on a single C# form that we wrote to illustrate and exercise the many methods and properties of DTFI. You will find it helpful to download the program and run it yourself as you read through the article.

Obtaining a DateTimeFormatInfo Instance

Most of the time, DTFI does its job without you being aware of it. For example, if we create an instance of DateTime and then display its contents with the ToLongDateString or ToShortDateString methods then we will obtain a reasonably well-formatted result that meets user expectations for the CultureInfo of the current thread. But, you have to work directly with a DTFI for some problems. For example, suppose you want to provide the user with a list of the days of the week in his or her language and highlight the start day. Using DTFI we can carry out this task very easily. Notice also that you don't have to research this information yourself.

// use the current thread culture
CultureInfo ci = CultureInfo.CurrentCulture;

// get a DateTimeFormatInfo from CultureInfo
DateTimeFormatInfo dtf = ci.DateTimeFormat;

// clear the listbox
lstDayNames.Items.Clear();

// get weekday list, full and abbreviated
string [] daynames = dtf.DayNames;
string [] abbrevdaynames = dtf.AbbreviatedDayNames;

// probably seven days but don't assume anything!
int len = daynames.GetLength(0);
// format the result using a string formatter (makes translation easier!)
for (int i = 0; i < len; i++)
lstDayNames.Items.Add(String.Format("{0} ({1})", daynames[i], abbrevdaynames[i]));

// choose the first day of the week in this locale and select it
int fd = lstDayNames.SelectedIndex = (int)dtf.FirstDayOfWeek;

You can write very similar code in ASP.NET, however, you will want to reflect the user's preferred culture not that of the server's thread culture. One way would be to use the HTTP_ACCEPT_LANGUAGE parameter. However, you want to be careful to create a specific culture. For example, suppose the user's browser language preference is Russian. In both IE and Netscape, the language tag transmitted to the server is "ru". If you instantiate a CultureInfo using new CultureInfo("ru"), you'll be out of luck getting to a DTFI since the CultureInfo object is a parent culture, and parent cultures have only a null value for their DTFI.

By contrast, a specific culture must be instantiated by a parameter that contains both language and regional content. Examples are en-UK for English (United Kingdom), ar-JO (Arabic, Jordan), fr-FR for French (France), etc. Why is a specific culture necessary? It is because a DTFI contains information that can be both region and language dependent. For example, the usual UK short date form of day/month/year differs from its US month/day/year counterpart, despite a (more or less!) common language. Another example can be found between the two German speaking regions Austria (de-AT) and Germany (de-DE), where the name of the first month of the year differs in spelling: 'Januar' in Germany, but 'Jänner' in Austria. In this example, both regions even share a common frontier, but an Austrian customer will be annoyed if your software mixes them up- and vice versa.

The implication is that if you create a parent culture - one that references language only - then the instance's DateTimeFormatInfo property is null. A similar comment holds for another formatter class, NumberFormatInfo. So, if you expect to be able to use culture-specific dates and times (or numbers, currencies, and percentages), use CultureInfo.CreateSpecificCulture to obtain a specific CultureInfo instance. Even if you instantiate it with a language only (i. e. "de", "en", "ar"), you will get a specific culture with a default region. Thus the solution to the problem above is solved by using the call CultureInfo.CreateSpecificCulture("ru"). Once you have a specific culture, you can use CultureInfo.DateTimeFormat to obtain a usable DTFI.

By the way, web programmers tend to disregard the browser language tag and depend more on a previously set cookie for such information. Even if set, it may not indicate a user's preferences. However, many web sites, including http://www.microsoft.com/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq, seem to use it as an initial preference. For testing purposes, I often modify my browser language setting and later forget what I have done. More than once, I've found myself looking at Microsoft's Chinese or Indian web site wondering how I got there.

Basic Features of DateTimeFormatInfo

An instance of DTFI contains lots of useful information. Figure 1 (below) will give you an idea of its scope. In this case, the selected culture is English (United States), mainly to provide easy readability for most programmers. However if you have the demo program at hand, try viewing some of the other 100+ cultures available. You may not be able to read everything, but it will help you gain awareness and appreciation of how many different ways there are for handling date, time, and calendar related issues.

At the top of the form is a list of specific cultures obtained basically by calling the static method:

CultureInfo [] ci = CultureInfo.GetCultures(CultureTypes.SpecificCultures);

Internally, the enumeration is obtained indirectly. List and dropdown controls allow you to insert a reference to a class as an item in the list. If you do this, then the display string is determined by the ToString method of the object. Since the return from ToString is not always very user friendly, I derived a class from CultureInfo whose ToString method returns the CultureInfo.DisplayName property. The derived class is the one actually inserted into the culture dropdown list. Using this technique, the friendly name matches the current .NET runtime language. When I have to use the selected culture, I have all its methods and properties immediately available. Even better, the list is sorted correctly according to the current thread culture and I don't have to maintain any kind of internal data structure; the control is storing the data and doing all the work for me.

Figure 1: DateTimeFormatInfo Demo Program

Here is the code for the derived class.

using System;
using
            System.Globalization;

namespace WinDateTimeFormat {
public
            class MyCLInfo : CultureInfo {
public MyCLInfo(string name) :
            base(name) {}
public MyCLInfo(string name, bool useoverride)
: base(name, useoverride) {
}
public override string
            ToString() {
return base.DisplayName
}
}
}

The second controlling list in the demo program is the list of calendars of the culture. The .NET Framework supports eight calendars, although a detailed discussion of the Calendar class is beyond the scope of this article. However, the particular calendar in use greatly affects the content of a DTFI. In the demo program, you can view any combination of culture and available calendars. You may see more than one Gregorian calendar for a given culture. Even the en-US culture has two; one is US English and the other is Localized. Just below the calendar list you can see the Gregorian Calendar type name.

Here is a complete list of all supported calendars along with some long date displays. All displays represent the same instant of time. You can clearly see the impact a change of calendar might have on a string representation. By the way, a Calendar class provides a way for managing time increments and decrements. The action of adding a day or a month can produce quite different results from one calendar to another, and can be very difficult to do correctly, especially for lunar calendars. Are any of these alternate calendars useful in the real world? You bet, if you are doing business with a customer whose preferred calendar is one of them!

Gregorian Calendar | Sunday, November 11, 2001
Hebrew Calendar | 25 ???? 5762
Hijri Calendar | 25 ?????, 1422
Japanese Calendar | ?? 13?11?11?
Julian Calendar | Monday, October 29, 2001
Korean Calendar | ?? 4334? 11? 11? ???
Taiwan Calendar | 90?11?11?
Thai Buddhist Calendar | 11 ????????? 2544

By the way, you may need to add additional language support for you Windows 2000 or XP system to see some of these forms. To do this, get your installation CD and go to the Regional Settings dialog. Check the language groups you want, and insert the CD when requested.

Let's look again at Figure 1 to see a number of useful items. We have already discussed the native names of the weekdays and the first day of the week. But, you can also read a list of month and abbreviated month names, strings for the equivalent of 'AM' and 'PM' in other locales, default field separators for date and time expressions, calendar era names, and a value that may be new to you: the first week of the year. In fact, System.Globalization actually has an enumerator for this: the CalendarWeekRule:

FirstDay | The first week of the year starts on the first day of the year and ends before the following designated first day of the week. The value is 0.
FirstFourDayWeek | The first week of the year is the first week with four or more days before the designated first day of the week. The value is 2.
FirstFullWeek | The first week of the year begins on the first occurrence of the designated first day of the week on or after the first day of the year. The value is 1

Accounting and other rules for a given world region can be affected by such rules, and some years may have more or less than 52 weeks, particularly when the FirstFourDayWeek rule is in effect. Examples of locations using this rule include Denmark, The Netherlands, Estonia, Faro Islands, Finland, Switzerland, Germany, Iceland, Italy, Latvia, Norway, Poland, and Sweden. The only FirstFullWeek country seems to be Pakistan. Most countries use the FirstDay option.

Date and Time Options

The main feature of DTFI is the plethora of formats for date and time in an amazing number of combinations. For the U. S. alone you can find 129 formats. Why so many? The main reason is to be able to provide your customer with a complete list of culturally acceptable formats. You don't have to do the research - it has been done for you. In addition, there are a few specialized forms such as RFC 1123 and ISO 8601. All together, there are eleven categories with one aggregating all the rest. A glance at Figure 1 shows the available patterns on the left as dropdown lists, and the current selection on the right as formatted by DTFI according to the current culture and calendar. All values are based on DateTime.Now, which changes each time you select a new culture or calendar.

How do you enumerate each pattern category? The basic DTFI method is GetAllDateTimePatterns using a character parameter, or none at all if all formats are required. Early .NET documentation was very confusing on which characters to use; it took some experimentation to find out what really works. Fortunately, the help has improved with the 1.0 release. Below is a code fragment from the demo application that enumerates all the forms for the full date and time patterns (this group is selected by the characters 'f' and 'F') and inserts the results into a combo box. The routine also selects the default pattern if DTFI has a corresponding property for that category. Also note that the text direction is set to RightToLeft for bi-directional writing systems such as Arabic, Farsi, Divehi, Hebrew, Urdu, and Syriac. Actually, this step is not entirely necessary since .NET will manage right-to-left behavior correctly (including reorienting the controls) if the appropriate thread culture has been set. You can see this in the demo program by setting your user locale in the Regional Settings dialog to one the Arabic locales. A similar technique works when serving a web page from .NET; in the code-behind, set the thread locale accordingly in Page_Load().

// full date time
// clear the combo
            box
cbFullDateTimePattern.Items.Clear();

// enumerate the
            full date time patterns and load them
foreach(string pattern in
            dtf.GetAllDateTimePatterns('F'))
cbFullDateTimePattern.Items.Add(pattern);

//
            enumerate the full date, short time patterns and load
            them
foreach(string pattern in
            dtf.GetAllDateTimePatterns('f'))
cbFullDateTimePattern.Items.Add(pattern);

//
            there is a default FullDateTimePattern method.
// find it in the
            list and select it
cbFullDateTimePattern.SelectedIndex =
cbFullDateTimePattern.FindStringExact(dtf.FullDateTimePattern);

//
            set the text direction
txtFullDateTime.RightToLeft =
            textdir;

There are differences, by the way, if the formatting character is lower case or upper case. Not all are explained very well. In the code fragment above, 'f' enumerates all full dates with short times (no seconds) and 'F' provides full dates with long times. A similar comment holds for 'g' and 'G' when applied to general dates and times. Others are explicitly separated, such as 'd' for short date, and 'D' for long date. Others, such as year/month ('y', 'Y') and month/day ('m', 'M') seem to generate the same patters for both upper and lower case, and I have not been able to infer from examples whether there are significant differences.

DateTimeFormatInfo Patterns

As you have probably guessed, you can pass a date-time pattern obtained from DTFI directly to the appropriate overloaded ToString method of DateTime. You can even create your own patterns, although if you are trying to get a culturally correct result, this may not be advisable. Here is how a selected full date time pattern is used to create the corresponding output in the demo program. The code here is in response to a user changing the combo box selection containing the full date time format strings. A similar refresh takes place if the user changes the calendar.

// full date time
// dt is a DateTime instance created from DateTime.Now
// cbFullDateTimePattern is the combo box showing full date time patterns
// txtFullDateTime is a text box holding the formatted result
// curci is the currently selected CultureInfo instance

protected void
cbFullDateTimePattern_TextChanged(object sender, System.EventArgs e) {
try {
txtFullDateTime.Text = dt.ToString(cbFullDateTimePattern.Text,
curci.DateTimeFormat);
} catch {
}
}

A complete discussion of pattern types is documented in the .NET help. Look in the Index for DateTimeFormatInfo - about DateTimeFormatInfo class. However, just by looking at the demo program you will certainly figure most of them out. For example the full date pattern

dddd, MMMM dd, yyyy h:mm:ss tt

is interpreted as follows: use the full name of the weekday (dddd), the full name of the month (MMMM), a two digit number with leading zero for the day (dd), a four-digit year (yyyy), an hour with no leading zero (h), a two-digit minute with leading zero (mm), a two-digit seconds display with leading zero (ss), and a meridiem string such as AM or PM (tt).

Did you look carefully at Figure 1? In the long date pattern there is some English text, 'Today it is '. In fact, in the demo program all the combo boxes containing date patterns are writeable so that you can insert your own text. Be sure to enclose your text in single quotes, however. You can add text as needed when rendering displays and pass the resulting combination of text and date/time pattern to the ToString method. However, it is strongly recommended that you use a formatter to make the result localizable to other languages. Fragments or concatenated strings can be nearly impossible to translate because of differing word order, syntax, and grammar constraints. For example, you should format this string like this before passing it on to the DateTime.ToString method:

// cbFullDateTimePattern is a combo box
            holding a list of date time patterns
string
            strMyDate;
strMyDate = String.Format("Today it is {0}",
            cbFullDateTimePattern.Text);

Default Formats in DateTimeFormatInfo

Having all those alternate formats can be useful, but usually you are happy just to have one. Here is a table of date and time default properties, patterns, and examples sorted by the left column. I've chosen Czech to illustrate. The property names are self-documenting (if you read English!)

As a concluding comment, although the month name for February in Czech is únor, it is not used quite right here in most of the displays. The form shown is in the nominative case (brush off your grammar book - it is the dictionary form) but it should appear as "února" (the genitive or 'of' case) in most of the examples. Case endings in languages often take the place of prepositions. In this case you translate "11. února" as the "11th of February". The same problem exists in .NET with related languages such as Russian, Ukrainian, Polish, Slovak, and others. Declensional languages can add a new layer of complication in both internationalization and localization of programs and web sites and it is always best if such details can be handled by the runtime system. Interestingly, Microsoft does it right in Win32 in GetDateFormat, although it took them several years to get around to the task. I'm told that this oversight in .NET will be fixed 'soon'.

Conclusion

.NET provides a powerful and versatile way to manage date and time formats and to provide essential information about months of the year, days of the week, first day of the week and first week of the year, alternate calendars and eras, and a large selection of alternate date and time patterns that meet local cultural expectations. Of course, the ideas for the class have roots in Win32, and if you would like to obtain a useful utility that runs there, you can download it from http://www.mlmassociates.cc/dl-win32.htm?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq. Look for the nLocale application. You can also find quite a lot of information on the web site about internationalization on Win32 and .NET systems as well as links to some other sites dealing with the subject. You can also find a long list of references to articles and books there at http://www.mlmassociates.cc/biblio.htm?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq. Finally, don't overlook the Microsoft site http://www.microsoft.com/globaldev?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq, which provides an often-overlooked resource for international development.

Article Information
Author | Bill Hall
Technical Editor | Adam Ryland
Project Manager | Helen Cuthill
Reviewers | Andrew Krowczyk, Saurabh Nandu

If you have any questions or comments about this article, please contact the technical editor.


The Theory of Inheritance in C#

The Theory of Inheritance in C#
http://www.csharptoday.com/content.asp?id=1752
Article Rating: 4
Published on February 28, 2002
Abstract
Object Oriented programming in C# is quite a vast topic. This article by Hersh Bhasin, the third in the series, focuses specifically on the inheritance capability of C#. It starts with examining the key protagonists in this area, namely classes, fields, properties, and methods. There are a large number of keywords and concepts in the C# vocabulary. These concepts and keywords can apply (or not apply) to the data members, and it is easy to lose track of the association of keywords and concepts with members and classes. In this article, Hersh summarizes and clarifies these concepts and keywords and describe how they are associated with classes and members.
Article

Object Oriented programming in C# is quite a vast topic. This article focuses specifically on the inheritance capability of C#. It starts with examining the key protagonists in this area, namely classes, fields, properties, and methods. There are a large number of keywords and concepts in the C# vocabulary. These concepts and keywords can apply (or not apply) to the data members, and it is easy to lose track of the association of keywords and concepts with members and classes. In this article I will summarize and clarify these concepts and keywords and describe how they are associated with classes and members.

At the very start of this article, I have provided a table that summarizes how the keywords and concepts relate to classes, methods, properties, and fields. In the body of the article I will expand on these concepts and keywords. The two models of inheritance, namely the Classical (is-a) and the Containment-Delegation (has-a), serve as a backdrop to my discussion on issues like member visibility, invocation (instance and static members), class constructors, class inheritance, and class types (abstract, sealed, and nested). Various keywords like static, virtual, abstract, readonly, override and hidden are also discussed and explained. Finally, I take a look at the interface approach to programming and discuss the issue of multiple inheritance. C# does not support multiple inheritance, however similar results can be obtained by using a service-based architecture which I describe in the text.

Classes and Entities

A class is the fundamental data type in in C#. It is comprised of members. These members are entities like Fields, Properties or Methods. Here is a brief description of each of these entities:

  • A class is a custom UDT (user defined type) that is composed of data (fields or properties), and methods that act on this data.
  • Methods are the various functions that make up the class.
  • Properties are a set of assessor and mutator functions that are used to "set" or "get" Field values. The accessor functions (the Get functions) get called when values are requested from a property. Similarly the mutator function (the Set function) get called when values are assigned to a property. Here is an example.

private String ls_connStr;
public String
            ConnStr
{
get
{
return
            ls_connStr;
}
set
{
ls_connStr =
            value;
}
}

  • Fields are public data. These are variables declared with a public scope. For example, we can declare the variable ls_connStr with a public scope as follows:

public String ls_connStr;

The ls_connStr variable can now be directly accessed by other classes (without requiring the assessor and mutator functions). This is not a good interface design, as the assessor and mutator functions allow you to perform internal logic necessary before assigning or returning the property value. In the samples download, I have a sub folder called PropertyExample which demonstrates the use of a property assessor to set local values. In this example, I pass two parameters to the property called TheProperty. These parameters are passed as a single string, separated by a comma. I then make use of the Split method to extract the two parameters and store them in local variables.

    using
            System;
public class PropertyExample
{
private string
            ParameterA;
private string ParameterB;

private string
            m_TheProperty;

public string TheProperty
{
get{return
            m_TheProperty;}

set{
m_TheProperty =
            value;

if(m_TheProperty =="")
return;

char[] sep =
            {','};
Array arr = m_TheProperty.Split(sep);
ParameterA =
            arr.GetValue(0).ToString();
ParameterB =
            arr.GetValue(1).ToString();
}
}

public void
            Display()
{
Console.WriteLine("Parameter A: " +
            ParameterA);
Console.WriteLine("Parameter B: " +
            ParameterB);
}
}

public class
            test
{

public static void
            Main()
{
PropertyExample my = new
            PropertyExample();
my.TheProperty =
            ("Hersh,Bhasin");
my.Display();
}
}

Methods, properties and types are collectively called "members", as they are all members of a given class.

The various concepts and keywords applicable to members and classes are shown in Table 1. A cross against a member or a class means that the crossed concept or keyword can be applied to that member or class.

Table 1: The Relationship Table

Visibility or Scope

If you refer to Table 1, you will note that a method, property, or field (i.e. class members) can have a visibility of public, private, protected, internal, or protected internal. The default scope for a member is private.

Classes can only have two visibility levels (public and internal). The default scope for a class is internal. The visibility levels are discussed below:

public : This keyword applies to both members and classes. For a class, it means that any object within the same assembly as well as other assemblies can create an instance of the class. For a member, it means that such a (public) member is accessible from an object instance or any subclass.

private : This is the default visibility level for members. Such members are only accessible within the class that has defined them.

protected : A protected member is only accessible to the base and all its derived classes.

internal : This keyword applies both to members and classes. Such members and classes are publicly accessible to all types within an assembly but are private to types outside of the assembly.

protected internal : This provides protected or internal access.

Invocation (Instance and Static members)

Class members are associated with an instance of a class by default. A class instance is created by invoking (instantiating) a class using the new keyword. An instance of a class gets its own copy of the class data.

There may be times when we want to store data that is unchanged across class instances. This might be the case when we need to provide the functionality of global variables available in other procedural languages. In such cases, it would be memory inefficient to let each class have its own copy of the non-changing variable. You can associate a class member with the class as a whole (instead of with an instance of a class) by using the keyword static.

The following example (Static.cs) demonstrates making reference to static fields, properties and methods. Note that the static member can be referred to directly by putting the name of the class before the dot (".") operator.

Static.cs

using System;
public class
            globals

{
/**** Static Field *****/
public static
            string CompanyName;

/****Static Property****/
private
            static string ls_ConnStr;
public static string
            ConnStr

{
get { return ls_ConnStr; }
set { ls_ConnStr =
            value;}
}

/****Static Method****/
public static
            void TheMethod()
{
Console.WriteLine("Result from TheMethod :"
            );
Console.WriteLine("Company Name:" +
            CompanyName);
Console.WriteLine("Connection String:" +
            ConnStr);

}
}

public class test
{
public
            static void Main()
{
globals.CompanyName = "Sirius
            Consulting";
Console.WriteLine("Company Name:" +
            globals.CompanyName);

globals.ConnStr = "Some Connection
            String";
Console.WriteLine("Connection String:" +
            globals.ConnStr);

globals.TheMethod();
}

}

Static properties must operate on static fields. Hence in the above example, ls_ConnStr must also be marked static.

The following example (Instance.cs) demonstrates making reference to Instance fields, properties, and methods. Note that to refer to an instance member, you first have to create an instance of a class using the new keyword.

Instance.cs

using System;
public class
            globals

{
/**** Instance Field *****/
public string
            CompanyName;

/****Instance Property****/
private
            string ls_ConnStr;
public string ConnStr

{
get { return
            ls_ConnStr; }
set { ls_ConnStr =
            value;}
}

/****Instance Method****/
public void
            TheMethod()
{
Console.WriteLine("Result from TheMethod :" );
Console.WriteLine("Company Name:" +
            CompanyName);
Console.WriteLine("Connection String:" +
            ConnStr);

}

}

public class test
{
public
            static void Main()
{

globals MyGlobals = new
            globals();

MyGlobals.CompanyName = "Sirius
            Consulting";
Console.WriteLine("Company Name:" +
            MyGlobals.CompanyName);

MyGlobals.ConnStr = "Some
            Connection String";
Console.WriteLine("Connection String:" +
            MyGlobals.ConnStr);

MyGlobals.TheMethod();

}
}

Thus a member declared as static belongs to the whole class, whereas an instance member belongs to a object which inherits from that class. A static member can be called without instantiating an object of that class, while instance members needs an object of that class to be instantiated.

Inheritance

There are two types of inheritance models, and C# supports them both. These models are:

  • The is-a relationship model. This is also called the Classical inheritance model
  • The has-a relationship model. This is also called the Containment/Delegation model.

The is-a relationship model

The is-a relationship model is an ancestor-descendant type of relationship. In such a model, the ancestor class is used to define general characteristics that are common to all descendents. The descendent class inherits all the public and protected members of the ancestor class and is free to use, extend, or override the functionality provided by the ancestor class.

To establish this type of relationship, you ask yourself whether the descendant class has an is-a relationship with the ancestor. This is the type of relationship I established when I developed the transaction sheet framework in my previous article (http://www.csharptoday.com/content.asp?id=1722&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq). I created an ancestor transaction class and established that a bank transaction is a is-a transaction, and so could inherit from the transactions class. Similarly journal, sales, and purchases are all transactions and can also inherit from the ancestor transaction class.

In the following example, I develop a FamilyMember class. The FamilyMember class is a class that describes a nuclear family unit comprised of a wife, a son, a daughter and a stepdaughter, and is used by the head of the family to work out the monthly allowance to be given to various family members. The code for the class follows:

FamilyMember.cs

using System;
public class FamilyMember

{ private string name;
private float basicAllowance;
private static string surname;
private string pastime;

/****Constructors*****/
//static constructor
static FamilyMember()
{
Surname = "Smith";
}

//Custom constructor #1
public FamilyMember(string TheName,string ThePastime)
{
this.Pastime = ThePastime;
this.Name = TheName;
}

//Custom constructor #2: If only name supplied, forward to
//constructor #1
public FamilyMember(string TheName )
:this(TheName, "No Pastime"){}

/*
Default constructor. This constructor is provided by default. However it
is removed when a custom constructor is defined. You must explicitly
redefine it if you want to be able to create an instance of the class with
zero arguments like so: FamilyMember John = new FamilyMember();
*/

public FamilyMember(){}

/*****Properties*******/
//a static property
public static string surname
{
get{return surname;}
set{surname = value;}
}

public string Pastime
{
get{return pastime;}
set{pastime = value;}
}

public string Name
{
get{return name;}
set{name = value;}
}

public float BasicAllowance
{
get{return basicAllowance;}
set{basicAllowance = value;}
}
/****Methods*****/

public virtual void MonthlyAllowance(float TheBasicAllowance)
{
BasicAllowance =TheBasicAllowance;
}

public void GetDetails()
{
Console.WriteLine(" Name : " + Name + " " + Surname) ;
Console.WriteLine(" Pastime : " + Pastime) ;
Console.WriteLine(" Allowance : " + BasicAllowance.ToString());
}

}

This class has one static property called Surname which holds the family name. It makes sense to store the family name in a static variable as it will be unchanged for all family members. This class also has a static constructor which sets the Surname property to its initial value. There can only be one static constructor, which cannot take any arguments, and cannot take a visibility modifier. It must however have the static modifier. The sole purpose of this static constructer is to assign initial values to static data.

The FamilyMember class has three properties. These properties are called Name, Pastime and BasicAllowance, and the use of each is evident from their captions. Each property has a set of assessor and mutator functions that are used to "set" or "get" Field values.

This class has two custom constructors and one default constructor. A constructor is a method that has the same name as the class and is used to set the initial values of class member variables. A constructor cannot have a return value. Each C# class is provided with a default constructor which takes no arguments. You are free to provide additional constructors that take arguments. However, the default constructor is "silently" removed when a custom constructor is defined. You must explicitly redefine it if you want to be able to create an instance of the class with zero arguments.

If the calling class supplies two arguments, constructor #1 gets called which uses the passed arguments to set the Name and Pastime field values. If only the Name parameter is passed to this class, constructor #2 is called, which supplies a default value to the pastime field, captioned "No Pastime ", and then forwards the Name and Pastime values to constructor #1. Note the use of the keyword this to denote self-reference.

The FamilyMember class has two methods. The first is called MonthlyAllowance and this calculates the monthly allowance that a family member receives from the head of the family. The second function is called GetDetails and this method simply outputs the class details to the console.

I have a class called RunIt, which serves as a testing platform for the FamilyMember class. It has a single function called Main and this serves as the entry point for this application. Here is the code of this class.

Class: RunIt

/*************The Entry point
            *********************/

public class RunIt
{
public
            static void Main()
{

/********FamilyMember class
            **************/

//Static property
Console.WriteLine("The
            Family name is: " + FamilyMember.Surname);

//Call
            constructor #1
FamilyMember TheWife = new
            FamilyMember("Susan","Bridge");
TheWife.MonthlyAllowance(120);
TheWife.GetDetails();

//Call
            constructor #2
FamilyMember TheStepDaughter= new
            FamilyMember("Cinderella");
TheStepDaughter.GetDetails();
}
}

This class displays the family name as follows:

/
/Static property
Console.WriteLine("The Family name is: " + FamilyMember.Surname);

Note that since the Surname property is a static property, there was no need to create a new instance of the FamilyMember class.

The wife is called Susan and plays Bridge. An instance of the FamilyMember class is created by using the new keyword, and this information is provided to Constructor #1 which sets the initial state of the class instance. The wife is given a monthly allowance of 120 dollars, and this information is conveyed to the class via the method MonthlyAllowance.

The stepdaughter is called Cinderella and I am sad to say, the other family members do not treat her very well. She is not allowed to have any pastimes or hobbies, and is not given any monthly allowance. Consequently C onstructor #2 is used to set her initial state. This constructor accepts only the name argument as a parameter. It then calls Constructor # 1 and passes to it the name argument and "No Pastime" as the pastime argument.

A son is a is-a family member, and can thus inherit from the FamilyMember class. Consequently, I have created a Son class as follows:

The Son class

/*******************Son**************************/
public class Son :FamilyMember
{
private int merits;

public int Merits
{
get{return merits;}
set{merits = value;}
}
/****Constructor*****/
public Son(string TheName,string ThePastime,int TheMerits )
:base(TheName, ThePastime)
{
this.merits = TheMerits;
}
public override void MonthlyAllowance(float TheBasicAllowance)
{
BasicAllowance =TheBasicAllowance + 10*Merits;
}
}

The Son class provides its own implementation of the MonthlyAllowance() method. The son's monthly allowance is calculated on the number of merits he accumulates over the month. He receives a merit each time he does something good, like receiving a good grade at school. His monthly allowance is calculated at a base amount of ten dollars, plus ten dollars for each merit that he accumulates. An instance of the son class is created in the RunIt class as follows:

/**********Son class***************/
Son
            Tom = new Son("Tom","Basketball",
            5);
Tom.MonthlyAllowance(10);
Tom.GetDetails();

In C#, inheritance is accomplished by using the colon operator (:). A "inheritable" member should have a scope, other than of private. It is good practice to mark such "inheritable" members as protected.

The constructor of the Son class accepts three parameters. These are the Name, Pastime and Merits properties. Of these three, the Name and Pastime properties are defined in the base class, and the Merit property in the Son class. Consequently the son constructor deals with the Merit property "locally" and calls the base (FamilyMember) constructor by making use of the base keyword as follows:

public Son(string TheName,string ThePastime,int TheMerits )
:base(TheName, ThePastime)
{
this.merits = TheMerits;
}

Two keywords are often used in association with methods in a is-a inheritance model. These are the virtual and the override keywords. When the base class wishes to define a method that may be overridden in the derived class, it must mark it as virtual. In our current example, both the base and the derived classes have a method called MonthlyAllowance(). When we call the MonthlyAllowance() method in the derived (the Son class) class, the virtual keyword tells the compiler that the MonthlyAllowance () method of the appropriate instance should be called, and that is why the MonthlyAllowance () method of the derived class is called, instead of the MonthlyAllowance() method of the base class.

The override keyword is used by the method in the derived class which wants to provide its own implementation of the method defined in the base class. The base class code will be ignored (or overridden) and the derived class code for that method will be effective. You can still call the base method by using the base keyword. Thus to call the MonthlyAllowance() method of the base class in the MonthlyAllowance() method of the derived class, you would write

base.MonthlyAllowance();

You can use a technique called method hiding to prevent users from triggering a base class method. This is done by using the new keyword in the method definition. For example to hide the MonthlyAllowance() method in the base class, in the derived class you will define this method as follows:

public new void MonthlyAllowance(float
            TheBasicAllowance)
{
BasicAllowance =TheBasicAllowance +
            10*Merits;
}

This guarantees that if the derived class is instantiated and the MonthlyAllowance() method called, the most derived implementation will be called. In other words instead of overriding the base method, you have hidden it and broken the relationship existing between the base and the derived method. Normally you would want to override and not hide a base method.

This process of overriding base methods is an aspect of Polymorphism and is called function overriding. C# also supports function overloading, and you can thus have methods with the same names but differing signatures. In the samples download, I have included a subfolder called overloading. In there you will find an example of function overloading. I have overloaded a function called TheMethod so that it accepts zero and one parameters as follows:

public void TheMethod()

{ Console.WriteLine("Method called with no parameters" );}

public void TheMethod(string s)

{ Console.WriteLine("Method called with parameter " + s );}

I can successfully call these methods as follows:

son myTest = new
            son();
myTest.TheMethod();
myTest.TheMethod("Hersh
            Bhasin");

However I cannot overload a method based just on its return type. In other words, I cannot overload it as follows:

/***Will not work****/
public int
            TheMethod()

{
Console.WriteLine("Method called with no
            parameters" );
return 1;
}

The compiler will complain that a method with same number of parameters (i.e. zero parameters) already exists.

The daughter class is similar to the son class. The daughter needs to look pretty to pursue her pastime (Boys) so her monthly allowance is calculated at a standard rate plus a make-up allowance. Here is the code for the daughter class:

The Daughter class

/*******************Daughter*************************/
public class Daughter :FamilyMember
{
private int makeUpAllowance;

public int MakeUpAllowance
{
get{return makeUpAllowance;}
set{makeUpAllowance = value;}
}
/****Constructor*****/
public Daughter(string TheName,string ThePastime,int TheMakeUpAllowance)
:base(TheName, ThePastime)
{
this.makeUpAllowance = TheMakeUpAllowance ;
}
public override void MonthlyAllowance(float TheBasicAllowance)
{
BasicAllowance =TheBasicAllowance + MakeUpAllowance;
}

}

The class RunIt creates a new instance of the daughter class as follows:

/**********Daughter
            class***************/
Daughter Mary = new Daughter("Mary","Boys",
            60);
Mary.MonthlyAllowance(10);
Mary.GetDetails();

Abstract

The abstract keyword is used with classes and methods. This is discussed in the following paragraphs.

An abstract class means that the class exists to define the methods, fields, or properties that will be used by most derived classes. The designer of an abstract class does not intend that such a class will actually be instantiated (i.e. created using the new keyword). Consequently an abstract class can never be instantiated and can only be inherited from.

When you mark a method as abstract, you provide no default implementation of the method at all. You leave it to the derived classes to provide its own implementation of the method. Here is an example of an abstract function:

public abstract class test{

public abstract void someMethod();

}

Note that no implementation is provided for the someMethod method. Any derived class of test would override the someMethod method and implement it. If the derived class did not implement the someMethod method, it will also be considered an abstract class and could not be directly created (but can be inherited from). If even a single method in a class is marked abstract, the class itself should be marked abstract else the compiler will raise an error.

In my previous article (Using inheritance to create a MDI Child form framework in C#), I had shown you how the ActivityAncestor class (and its add() method) could be made abstract.

Sealed

When you mark a class as sealed, it cannot be sub classed any further. In other words it cannot be inherited from. As an example, note that the System.String class has been sealed and you cannot inherit from this class. In the FamilyMember example, the Son and Daughter classes should be marked as sealed as it does not make sense to inherit from them (for example; a son's offspring is not a type of son or daughter- he/she is a grandson or a granddaughter. They are of course family members, and thus can directly inherit from the Familymember class).

Casting and Reflection

Since the Son and Daughter classes derive from the FamilyMember class, I can cast the Son and Daughter classes to the FamilyMember class. This allows me to put all the derived classes in an array and deal with them together. I have provided an example in the reflection subfolder in the code download. I can now rewrite the main() function of the RunIt class as follows:

public static void Main()
{

FamilyMember[] TheArray ={new FamilyMember("Susan","Bridge"),
new FamilyMember("Cinderella"),new Son("Tom","Basketball", 5),
new Daughter("Mary","Boys", 60)};

for(int i=0; i <TheArray.Length; i++)
{
Type t = TheArray[i].GetType();
string myName = t.FullName;
string papaName = t.BaseType.ToString();

Console.WriteLine("The class " + myName + " derives from " + papaName);

}

}

I can now check and confirm the is-a relationship. I do this by using the System.Type class within the System.Reflection namespace. The System.Object defines a method named GetType() that returns an instance of the Type class. I use the FullName property of the Type class to extract the name of the class and the BaseType property to get the name of the base object. The following is the output I get from running this program:

The class FamilyMember derives from System.Object
The class FamilyMember derives from System.Object
The class Son derives from FamilyMember
The class Daughter derives from FamilyMember
Press any key to continue .

The has-a or Contaiment/Delegation model

The second inheritance model is the has-a or Containment/Delegation model. In this model, one class acts as a "container" for another class. The container class creates a private instance (using the new keyword) of the contained class and thus its members are not accessible to the outside world. The container class provides methods that access the methods of the contained class. These methods of the container class are called delegates, hence the name Containment/Delegation.

The question you ask is "is this model a has-a". For example I have a stereo in my room. There is no is-a relation between the stereo and the room. The stereo cannot be said to be is-a type of room. However we can say that the room has-a stereo. Now if I have a stereo class and a room class, the room class will be the containing class and the stereo the contained class.

In the constructor event of the room class, I will create an instance of the stereo class as follows:

private stereo myStereo = new stereo();

Now suppose the stereo class has a method called Volume that turns the stereo volume up or down based on a passed integer as follows:

public void Volume (int
            theVolume)
{
StereoVolume = theVolume;

}

In the room class you will have to provide a delegate which acts as a wrapper to the Volume method of the stereo class. This is required because the stereo instance created in the room class is marked private. The outside world will call the delegate in the room class to change the stereo volume. This delegate could be as follows:

public void RoomVolume (int
            theVolume)
{
myStereo.Volume(theVolume);

}

Here is the code for the class:

Containment.cs

using System;
public class room :
            Object
{
private stereo myStereo = new stereo();
public
            void RoomVolume (int
            theVolume)
{
myStereo.Volume(theVolume);
}

}

public
            class stereo
{
private int StereoVolume;
public void Volume
            (int theVolume)
{
StereoVolume =
            theVolume;
Console.WriteLine("Turned up the Volume to: " +
            StereoVolume);
}

}

public class
            RunIt
{
public static void Main()
{

room MyRoom =
            new room();
MyRoom.RoomVolume(20);

}
}

Nested Classes

The contained class can also be nested within the containing class. In this case, the contained class is seen as a "helper" to the containing class. We can nest the stereo class within the room class as follows:

Nested.cs

public class room : Object

{
//outer class can create an instance of nested class
private stereo myStereo = new stereo();

private class stereo
{
private int StereoVolume;

public void Volume (int theVolume)
{
StereoVolume = theVolume;
Console.WriteLine("Turned up the Volume to: " + StereoVolume);
}

}

public void RoomVolume (int theVolume)
{
myStereo.Volume(theVolume);
}

}

Service-based architecture, Multiple Inheritance and Interfaces

A drawback of using the classical is-a model of inheritance is that the base class might become top heavy. You might load too much (non-generic) functionality in the base class. C# does not support multiple inheritance, else you might split functionality in multiple classes. However the way around this is to use a service-based architecture. I had explained this with the example of using a database class in my last article and I mention it here for completeness. Basically you use subservient classes that contain very specific functionality and which are very loosely coupled with other classes. Thus a database service class could have CRUD (create, read, update, delete) functionality. Any class wanting to use the functionality existing in the database class would simply create an instance of that class and start calling its methods. Thus multiple class libraries can be accessed. For example, if your database class is called dbClass and has a method called RunSQL (which accepts a SQL statement as parameter), any class that wants to use this method would do so as follows;

dbClass myDb = new dbClass();
mydb.RunSql("Delete * from masters where code = 2")

Had you used the classical (is-a) approach, you would have created this method in the base class. To take the example of the Transaction entry class described in my last article, this would have been scripted in the ActivityAncestor class . However doing so limits the usefulness of this function. Any class that needs to use this function must inherit from the ActivityAncestor class. If we, however keep this function in a separate class (the dbClass) , this function becomes accessible to any class that needs it.

It is contended by some authors that an aspect of interface based programming called multiple implementation is the way around to supporting multiple inheritance in C#. I do not agree with this. With multiple implementation you can have a class supporting multiple behaviors but it still has to implement methods defined in the interface that it supports. On the other hand, multiple inheritance has more to do with inheriting functionality and code-reuse. When a derived class inherits from a base class, it gets all of its protected (or public) members. It only has to re-code the inherited methods if it wants to provide its own implementation of the method, and typically it will do so by overriding the base method. In my opinion, the service-based architecture is the way we can simulate multiple inheritance in C#.

Here is a run down on interfaces. Briefly, an interface is a named set of abstract members. It is pure protocol, and interfaces never provide default implementation of methods or data types. This is left to any class that implements the interface. C# also supports what is called multiple implementation, which is to say that a class can implement more than one interface, in this way supporting numerous behaviors. When a class chooses to implement an interface, it does so by providing the interface names as a comma delimited list in the class definition (after the name of its base class). For example, if we created an interface called IdbClass that has the signature of a single method called RunSql, we would create this interface as follows:

interface IdbClass
{
void RunSql(string SQL) ;
}

The Bank and the Sales classes are derived from ActivityAncestor and implement this interface as follows:

public class ActivityAncestor
{

}

public class Bank:ActivityAncestor,IdbClass
{
public void RunSql(string SQL)
{
Console.WriteLine (SQL + " Where type = Bank" );
}
}

public class Sales:ActivityAncestor,IdbClass
{
public void RunSql(string SQL)
{
Console.WriteLine (SQL + " Where type = Sales");
}
}

Please note that both the derived classes had to implement the RunSql method. I can now use the is keyword to check if a class implements this interface and if it does, I can cast it to the IdbClass interface and call the RunSql method implemented by the class as follows:

public class RunIt
{
public static
            void Main()
{

ActivityAncestor[] s ={new
            ActivityAncestor(),new Bank(),new Sales()};

for(int i = 0; i
            < s.Length; i++)
{
if(s[i] is
            IdbClass)

{
((IdbClass)s[i]).RunSql("Delete * from
            Masters");
}
}

}
}

The code for this example can be found in the sub folder called interface in the samples download.

Now here is where I disagree with the contention that multiple implementation with interfaces is the way around multiple inheritance. What an interface does, is that it provides a guarantee that any class implementing a given class will implement a given set of functionality. Thus classes that implement a given interface become type compatible and exhibit polymorphic behavior. It will not provide you the code reuse techniques expected from using multiple inheritance (you will note that both the Bank and the Sales class have to provide implementations of the RunSql method, which is duplication of work). In my opinion, the service-based architecture is the true way around to multiple inheritance. I can write the RunSql method only once (in the class dbClass) and both the Bank and Sales class can use this (by creating a new instance of dbClass and calling the RunSql method).

Summary

In this article I have attempted to summarize and elucidate the relationship existing between methods, properties, fields, classes and the various concepts and keywords. I have looked at using inheritance with C# using two relationship models. These models are the ancestor-descendant and the containment-delegation models.

Article Information
Author | Hersh Bhasin
Technical Editor | Adam Ryland
Project Manager | Helen Cuthill
Reviewers | Stephen Trudel, Richard Conway

If you have any questions or comments about this article, please contact the technical editor.


Beginner's Hangman

Beginner's Hangman
http://www.csharptoday.com/content.asp?id=1781
Article Rating: 5
Published on February 23, 2002
Abstract
Hangman is the well-known game where the user guesses the letters of a mystery word. This game is played worldwide, and is one of the few children’s games based on capital punishment. To learn the C# language, Adam Ryland decided to try and write a few simple programs – Hangman is the first of those. During this article, we look at some of the basic features of any language – defining variables, calling methods and designing the user interface.
Article

In this article, Adam Ryland creates a simple Hangman program in C#, demonstrating some of the basic programming techniques that can be used in the language.

Introduction

Hangman is the well-known game where the user guesses the letters of a mystery word. This game is played worldwide, and is one of the few children's games based on capital punishment. To learn the C# language, I decided to try and write a few simple programs - Hangman is the first of those. During this article I will be looking at some of the basic features of any language - defining variables, calling methods and designing the user interface.

System Requirements

The user will need the final release version of Visual Studio .NET. A basic understanding of programming techniques is assumed, but even a complete novice should be able to pick up the techniques used in this article very quickly.

Designing The Interface

To begin with, I will run through the design of the user interface. For this simple program, we will only be using one Window form.

The first step is to open up Visual Studio. Go to File, then New, then Project. From the resulting Window, choose Visual C# Projects from the Project Types section, then Windows Application from the Templates part. In a stunningly original twist, we will change the name of the project to Hangman.

Now you will be presented with a blank form:

This form is where we will design the user interface. With the form highlighted, open up the Properties Window: you can do this be either going into the View menu and clicking Properties Window, pressing the F4 key, or clicking the Properties Window icon in the icon bar at the top of the screen. Once you have the Properties Window up, change the following properties:

  • Name to "Hangman"
  • Size to "312,640"
  • StartPosition to "CenterScreen"
  • Text to "C#Today Hangman"

Once these four properties have been changed, the Properties Window should look like this:

And the form should now look like this:

It's now time to start putting some things on the form. We'll start by adding two labels, one at each end of the form, and two buttons. The labels are purely aesthetic by the way.

Load up the toolbar if it isn't already showing, either via the View menu, the shortcut (Ctrl-Alt-X), or by the icon in the top bar. You'll find Label is the second icon down in the toolbar. Double click it to make a label appear on the form. Drag the label to the top of the form. Using the Properties Window (click on the label to make sure that the Properties Window is looking at the label's properties and not the forms): change the label's name to "lblTitle", the text to "C#Today Hangman", and the colors to whatever you feel like (I used a blue background with white writing). Repeat this process for a second label, but this time position it at the bottom of the screen. Change its name to "lblBy", its text to http://www.csharptoday.com/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq (or your name), and its colors to whatever the top label is.

We now want to add two buttons, positioning them just above the label at the bottom of the screen. Find the Button icon in the toolbar and double click it to bring a button onto the form. Do it twice, so that you have two buttons. The first button's name should be "cmdNew", and the text should be "New Game". The second button should have the name "cmdExit" and the text "Exit". Once you've done that, the form should look something like this:

Next we want to add the 26 letters of the alphabet, which the user will eventually click on to make guesses. These will be labels, resized to make them into squares, and placed in the lower half of the form. Their names will be "lblA", "lblB", "lblC", etc, and the text of each one will obviously simply be the letter itself. A quick tip is to do "lblA" first, changing the colors and size appropriately. If you then copy and paste this label, the new label will retain the colors and size of "lblA". This means you only have to change the name and text of the new label, saving you filling in the other information 25 times! By the time you have finished, the form will look something like this:

Next we want to add a place to show the word that is to be guessed. For simplicity, I am setting the word to always be seven letters long. These seven letters will be represented by seven labels, with the names "lblLetter1", "lblLetter2", etc. For aesthetics, I am also putting two thin labels (containing no text) above and below these seven letters. These will simply be called "lblBreak1" and "lblBreak2". I positioned them on the form like this:

Next we will add a message bar to the game. This will be invisible to the user while the game is in progress, but will appear in place of the 26 letters with the message "Winner" or "Loser" at the appropriate times. Add a label in the middle of the 26 letters, and call it "lblMessage". I made it a lime green color so that it would stand out. It doesn't matter if it overlaps or obscures some of the 26 letters, as they will not be on screen at the same time at any point. You should now have a form that looks something like this:

The final thing to add to the form is the actual graphics of the gallows. Because this is meant to be a programming exercise, I only created three different graphics for this, but you can of course create more - you can find mine in the code download for this article. For each stage, I added a PictureBox (from the toolbar) and named it "picStage1", "picStage2", etc. By using the image property from the Properties Window, I then set the picture box to contain the correct picture. These were then added above the word-to-be-guessed, on top of each other (which won't matter, as only one picture will be visible at any time). The completed form will now look something like this:

The interface is now complete, but of course, the program doesn't do anything yet. Let's deal with that in the next section.

Coding The Program

To start coding, right-click on "Hangman.cs" in the Solution Explorer window (if this is not visible, click on the appropriate icon in the top bar).

You will be presented with a menu - click on "View Code". You will see that a lot of the code has already been generated. Let's start by adding the basics and work up to a full solution.

The Variables

The first things we need to add are three variable declarations. Put the following three lines in the public Hangman class:

public bool[] guess = new bool[26];
public char[] myword = new char[7];
public int wrong = 0;

Guess is a Boolean array, containing 26 elements. These represent each letter of the alphabet in order, and whether they have been guessed or not. If guess[0] is set to true for example, that means that the user has guessed the letter "A" during this game. It is declared as public so that it is available to everyone.

Myword is an array of characters, AKA a string. It contains 7 elements, each one referring to one letter of the word to be guessed. This is used to store the word.

Wrong is a simple integer that counts the amount of incorrect guesses the user has made. It is initialized to zero to begin with.

Setting Everything Up

When the program first starts, we need to set all the variables up, make sure everything that is supposed to be visible is visible, create the new word, etc. To do this, we will create a new method called set_up_screen(). This method has the following tasks that it should achieve:

  • Set the hangman graphic to the correct state (invisible)
  • Set the "message" label to the correct state (invisible)
  • Initialize the "guess" array
  • Initialize the "myword" array
  • Set the seven "lblLetter" labels to display "?" in their text boxes
  • Make all 26 letters of the alphabet visible
  • Set the "wrong" integer value to 0

To make things simpler, initializing the "myword" array (of more succinctly, choosing the word to be guessed) will be in a separate method, random_word(). Add this to the code:

public void random_word()
{
int
            randno;
Random rand = new Random();
randno =
            rand.Next(16);
if (randno==1)
{
myword[0]='A';
            myword[1]='N'; myword[2]='G'; myword[3]='E';
            myword[4]='L';
myword[5]='I'; myword[6]='C';
}
if
            (randno==2)
{
myword[0]='J'; myword[1]='E'; myword[2]='R';
            myword[3]='I'; myword[4]='C';
myword[5]='H';
            myword[6]='O';
}
if (randno==3)
{
myword[0]='V';
            myword[1]='A'; myword[2]='M'; myword[3]='P';
            myword[4]='I';
myword[5]='R'; myword[6]='E';
}
if
            (randno==4)
{
myword[0]='E'; myword[1]='D'; myword[2]='I';
            myword[3]='T'; myword[4]='O';
myword[5]='R';
            myword[6]='S';
}
if (randno==5)
{
myword[0]='F';
            myword[1]='E'; myword[2]='A'; myword[3]='R';
            myword[4]='I';
myword[5]='N'; myword[6]='G';
}
if
            (randno==6)
{
myword[0]='T'; myword[1]='O'; myword[2]='P';
            myword[3]='I'; myword[4]='C';
myword[5]='A';
            myword[6]='L';
}
if (randno==7)
{
myword[0]='E';
            myword[1]='N'; myword[2]='T'; myword[3]='R';
            myword[4]='O';
myword[5]='P'; myword[6]='Y';
}
if
            (randno==8)
{
myword[0]='R'; myword[1]='E'; myword[2]='D';
            myword[3]='W'; myword[4]='O';
myword[5]='O';
            myword[6]='D';
}
if (randno==9)
{
myword[0]='K';
            myword[1]='N'; myword[2]='E'; myword[3]='E';
            myword[4]='C';
myword[5]='A'; myword[6]='P';
}
if
            (randno==10)
{
myword[0]='P'; myword[1]='O'; myword[2]='L';
            myword[3]='L'; myword[4]='U';
myword[5]='T';
            myword[6]='E';
}
if (randno==11)
{
myword[0]='R';
            myword[1]='E'; myword[2]='G'; myword[3]='U';
            myword[4]='L';
myword[5]='A'; myword[6]='R';
}
if
            (randno==12)
{
myword[0]='H'; myword[1]='A'; myword[2]='L';
            myword[3]='I'; myword[4]='B';
myword[5]='U';
            myword[6]='T';
}
if (randno==13)
{
myword[0]='S';
            myword[1]='K'; myword[2]='I'; myword[3]='P';
            myword[4]='P';
myword[5]='E'; myword[6]='R';
}
if
            (randno==14)
{
myword[0]='T'; myword[1]='E'; myword[2]='R';
            myword[3]='R'; myword[4]='A';
myword[5]='C';
            myword[6]='E';
}
if (randno==15)
{
myword[0]='A';
            myword[1]='R'; myword[2]='S'; myword[3]='E';
            myword[4]='N';
myword[5]='I'; myword[6]='C';
}
if
            (randno==16)
{
myword[0]='G'; myword[1]='E'; myword[2]='S';
            myword[3]='T'; myword[4]='U';
myword[5]='R';
            myword[6]='E';
}
}

As you can see, this is a simple method - it generates a random number between 1 and 16 (inclusive), and fills in the myword array with a seven letter word. This is slightly limited as the user will have seen all the words after only a few turns, but again, this is only a programming exercise.

Now that random_word() has been defined, we can now complete set_up_screen(). Add the following code:

public void
            set_up_screen()
{
picStage1.Visible=false;
picStage2.Visible=false;
picStage3.Visible=false;
lblmessage.Visible=false;
for
            (int
            counter=0;counter<26;counter++)
{
guess[counter]=false;
}
random_word();
lblLetter1.Text="?";
lblLetter2.Text="?";
lblLetter3.Text="?";
lblLetter4.Text="?";
lblLetter5.Text="?";
lblLetter6.Text="?";
lblLetter7.Text="?";
lblA.Visible=true;
lblB.Visible=true;
lblC.Visible=true;
lblD.Visible=true;
lblE.Visible=true;
lblF.Visible=true;
lblG.Visible=true;
lblH.Visible=true;
lblI.Visible=true;
lblJ.Visible=true;
lblK.Visible=true;
lblL.Visible=true;
lblM.Visible=true;
lblN.Visible=true;
lblO.Visible=true;
lblP.Visible=true;
lblQ.Visible=true;
lblR.Visible=true;
lblS.Visible=true;
lblT.Visible=true;
lblU.Visible=true;
lblV.Visible=true;
lblW.Visible=true;
lblX.Visible=true;
lblY.Visible=true;
lblZ.Visible=true;
wrong
            = 0;
}

Again, this is a simple method. Mostly it is just setting values to their initial states. You will notice that we use a loop to initialize the Guess array. The loop is controlled by three conditions enclosed in brackets after the word "For".

for (int counter=0;counter<26;counter++)

The first condition sets the starting point of the loop - in this case, an integer counter is set to zero. The second condition sets the boundary - here we are making sure that counter will not go above 26. The final condition controls how the loop advances - counter gets increased by 1 every loop in this example.

Find the Hangman() method (which is called when the program is first started), and add a call to set_up_screen :

public Hangman()
{
//
// Required
            for Windows Form Designer
            support
//
InitializeComponent();
Set_up_screen();

//
//
            TODO: Add any constructor code after InitializeComponent
            call
//
}

Now that we have set_up_screen defined, we can also fill in the code necessary for the two buttons we added at the design stage - "New Game" and "Exit". If you go back to the Design View of the form, double-clicking on each button will take you back into the Code View and generate the correct headings for the methods associated with the buttons. Fill them in with the following code:

private void cmdExit_Click(object sender, System.EventArgs e)
{
Application.Exit();
}

private void cmdnew_Click(object sender, System.EventArgs e)
{
set_up_screen();
}

Very simply, "Application.Exit();" causes the program to close, and hence is all we need in terms of code from the Exit button. We have already seen what set_up_screen does, and that is all we need to call when the New Game button is pressed.

Responding To A Guess

We now need to write the methods that will be used when the user makes a guess by clicking on one of the letters of the alphabet.

The method we are going to create next is called Guessed(), and will be called every time the user guesses a letter. This method has the following tasks:

  • If the guess is correct, reveal the appropriate letters in the word-to-be-guessed.
  • If the guess is incorrect, display the appropriate graphic
  • If the user has won the game, alter the screen accordingly
  • If the user has lost the game, alter the screen accordingly

For us to continue, we first need to create four other methods that Guessed() will use to complete these tasks.

The first is number_to_alpha. This method takes in an integer number, and returns the corresponding letter (which is why it is defined as public char). We'll see later why this will be necessary. Add this code to the project:

public char number_to_alpha(int
            which_number)
{
if (which_number == 0) return 'A';
if
            (which_number == 1) return 'B';
if (which_number == 2) return
            'C';
if (which_number == 3) return 'D';
if (which_number == 4)
            return 'E';
if (which_number == 5) return 'F';
if
            (which_number == 6) return 'G';
if (which_number == 7) return
            'H';
if (which_number == 8) return 'I';
if (which_number == 9)
            return 'J';
if (which_number == 10) return 'K';
if
            (which_number == 11) return 'L';
if (which_number == 12) return
            'M';
if (which_number == 13) return 'N';
if (which_number ==
            14) return 'O';
if (which_number == 15) return 'P';
if
            (which_number == 16) return 'Q';
if (which_number == 17) return
            'R';
if (which_number == 18) return 'S';
if (which_number ==
            19) return 'T';
if (which_number == 20) return 'U';
if
            (which_number == 21) return 'V';
if (which_number == 22) return
            'W';
if (which_number == 23) return 'X';
if (which_number ==
            24) return 'Y';
if (which_number == 25) return 'Z';
return
            'A';
}

Next up we need a method to display the correct graphic, which we'll call draw(). As we only have three graphics to display, this is a simple piece of code, controlled by the value of Wrong - how many incorrect guesses have been made:

public void draw()
{
if
            (wrong==1)
{
picStage1.Visible=true;
picStage2.Visible=false;
picStage3.Visible=false;
}
if
            (wrong==2)
{
picStage1.Visible=false;
picStage2.Visible=true;
picStage3.Visible=false;
}
if
            (wrong==3)
{
picStage1.Visible=false;
picStage2.Visible=false;
picStage3.Visible=true;
}
}

We need a method that will tell us whether the user has won the game. To do this, we simply look at the labels displaying the word-to-be-guessed. Any letter than has not been successfully guessed will show a "?" in the label, so all we need the method to do is look at each label in turn - if none of them display a "?", then the user has correctly guessed all the letters. The code is as follows (notice that this method returns a value, a Boolean, which is why it is specified as public bool):

public bool winner()
{
bool
            correct;
correct = true;
if (lblLetter1.Text=="?") correct =
            false;
if (lblLetter2.Text=="?") correct = false;
if
            (lblLetter3.Text=="?") correct = false;
if (lblLetter4.Text=="?")
            correct = false;
if (lblLetter5.Text=="?") correct = false;
if
            (lblLetter6.Text=="?") correct = false;
if (lblLetter7.Text=="?")
            correct = false;
return correct;
}

The final method we need is one to set the screen accordingly when the user has either won or lost the game. We'll call it game_over(). Very simply it removes all the letters of the alphabet (prohibiting the user from making any further guesses) and displays the word (the word will already be visible if the user has won the game, but we will also show the user what the word was if they lost - to rub in the pain of defeat!).

public void
            game_over()
{
lblLetter1.Text=myword[0].ToString();
lblLetter2.Text=myword[1].ToString();
lblLetter3.Text=myword[2].ToString();
lblLetter4.Text=myword[3].ToString();
lblLetter5.Text=myword[4].ToString();
lblLetter6.Text=myword[5].ToString();
lblLetter7.Text=myword[6].ToString();
lblA.Visible=false;
lblB.Visible=false;
lblC.Visible=false;
lblD.Visible=false;
lblE.Visible=false;
lblF.Visible=false;
lblG.Visible=false;
lblH.Visible=false;
lblI.Visible=false;
lblJ.Visible=false;
lblK.Visible=false;
lblL.Visible=false;
lblM.Visible=false;
lblN.Visible=false;
lblO.Visible=false;
lblP.Visible=false;
lblQ.Visible=false;
lblR.Visible=false;
lblS.Visible=false;
lblT.Visible=false;
lblU.Visible=false;
lblV.Visible=false;
lblW.Visible=false;
lblX.Visible=false;
lblY.Visible=false;
lblZ.Visible=false;
}

Now that we have made these four methods, we can build up Guessed(). The first part of the code for this method is as follows:

for (int
            counter=0;counter<26;counter++)
{
if (guess[counter]==true)
{
for (int counter2=0;counter2<7;counter2++)
{
if
            (myword[counter2]==number_to_alpha(counter))
{
if
            (counter2==0) lblLetter1.Text=myword[0].ToString() ;
if
            (counter2==1) lblLetter2.Text=myword[1].ToString() ;
if
            (counter2==2) lblLetter3.Text=myword[2].ToString() ;
if
            (counter2==3) lblLetter4.Text=myword[3].ToString() ;
if
            (counter2==4) lblLetter5.Text=myword[4].ToString() ;
if
            (counter2==5) lblLetter6.Text=myword[5].ToString() ;
if
            (counter2==6) lblLetter7.Text=myword[6].ToString()
            ;
}
}
}
}

This may look a little daunting at first, but if you break it down, it should be easy to see what it does. Firstly we start a loop from 0 to 25, which is done so that we can look at each element of the Guess array in turn. If an element of Guess is true (which you'll remember means the user has made a guess of this letter during the game), then we go into a second loop. This loop runs from 0 to 6, which allows us to examine each element of the myword array (the word to be guessed) in turn. We use the number_to_alpha method from earlier here - it takes in the "outer" loop (the one running from 0 to 25, referring to each letter in the alphabet) and passes back the character that we are currently looking at. We compare this returned value to each letter in the word-to-be-guessed in turn. If a letter in the word-to-be-guessed has been guessed by the user, then we display that letter on the screen to the user. This is a slightly inefficient method to use, as we look at all 26 letters on every guess, not just the letter that has been guessed, but in such a small program this inefficiency has no real consequences. It is quite simple to alter the program to only deal with each letter as it is guessed, but this is left to the reader as an exercise.

Once we have done the process, we then call the Draw() method that we defined earlier to display the correct graphic.

Finally, we examine the two "end game" possibilities - whether the user has won or lost. We have already defined a method to check whether the user has won. To find whether the user has lost, we simply examine the Wrong integer value - if it equals 3, then the user has lost. This value would obviously change depending on how many different stages the hanging graphic has been split into. If either state is true, then we call the game_over() method we made earlier, change the text of the message label, and make it visible to the user. The code for these two would be as follows:

if
            (winner()==true)
{
game_over();
lblmessage.Text="Winner";
lblmessage.Visible=true;
}

if
            (wrong==3)
{
game_over();
lblmessage.Text="Loser";
lblmessage.Visible=true;
}

All that is left to do now is to actually call the Guessed() method. This is done when the user clicks a letter \ makes a guess. First of all we need to create a method that checks whether a user's guess was correct or not:

public bool correct_guess(char
            what)
{
int result;
result =
            what.CompareTo(myword[0]);
if (result==0) return true;
result
            = what.CompareTo(myword[1]);
if (result==0) return
            true;
result = what.CompareTo(myword[2]);
if (result==0)
            return true;
result = what.CompareTo(myword[3]);
if
            (result==0) return true;
result =
            what.CompareTo(myword[4]);
if (result==0) return true;
result
            = what.CompareTo(myword[5]);
if (result==0) return
            true;
result = what.CompareTo(myword[6]);
if (result==0)
            return true;
return false;
}

As you can see, it takes in a parameter called "what" that contains the letter than was guessed. We use the CompareTo method that all char values have to see if it is the same as any of the letters in the word-to-be-guessed (CompareTo returns 0 if the characters match). We return a boolean value indicating whether it was a successful guess or not.

Clicking on a letter of the alphabet's label in Design View will create the appropriate heading for that method. We then fill in the code - below is the code for "A":

private void lblA_Click(object sender,
            System.EventArgs e)
{
if (correct_guess('A')==false)
            wrong++;
guess[0]=true;
guessed();
lblA.Visible=false;
}

As you can see, we check whether it was a correct guess or not using the correct_guess method that we created moments ago. If the guess was incorrect, we add 1 to the Wrong integer value. We also set the appropriate element of the Guess array to true to indicate a guess using that letter, then call Guessed() which we created earlier. Finally, we make that letter invisible to stop people making the same guess twice. This needs to be done for each letter in the alphabet - the same code is used for each, except that correct_guess is called with a different letter each time, and the element of Guess that is being changed needs to be altered.

With that, the program is complete!

Further Work

As I mentioned, this is simply a programming exercise, and so the finished application is far from perfect. The reader may wish to try to implement some of the improvements listed below as exercises:

  • Make the word-to-be-guessed of variable length - this would involve altering the function that returns the word to be guessed along with the number of letters it contains, perhaps altering the amount of guesses the user gets, altering the user interface so that it can handle words of different lengths, etc.
  • Change it so that there are the correct amount of stages in the hangman graphic - this would involve drawing new graphics for each stage, and altering the "draw" function.
  • Change the code so that each guess is dealt with individually, not all 26 letters every time. This would mean changing the way guesses are handled, and not use a loop.

Conclusion

In this article we've seen how to implement a simple Hangman game, and seen the basics of how to use C#, and some of the thought-process that goes into designing a program.

Article Information
Author | Adam Ryland
Editor | Adam Ryland
Reviewers | Vickie Pring


Using inheritance to create a MDI Child form framework in C#

Using inheritance to create a MDI Child form framework in C#
http://www.csharptoday.com/content.asp?id=1748
Article Rating: 3
Published on February 19, 2002
Abstract
C# ‘s support for inheritance can be utilized to create very elegant frameworks for MDI Child forms .A framework comprises a hierarchy of objects, inherited from a common parent object. Such a framework can be distributed as a foundation class library for other developers to use, fostering the spirit of Rapid Application Development (RAD). In this article, Hersh Bhasin will show you how to use inheritance to build a MDI Child framework.
Article

C# 's support for inheritance can be utilized to create very elegant frameworks for MDI Child forms .A framework comprises a hierarchy of objects, inherited from a common parent object. Such a framework can be distributed as a foundation class library for other developers to use, fostering the spirit of Rapid Application Development (RAD).

Inheritance with window forms involves abstracting the most commonly used functionality to an ancestor form. For example, a database centric application like an accounting application would need to present many faces of a transaction form to the user. An accounting transaction can be a journal entry, a cash transaction, a sale transaction, or a purchase transaction. All these transactions have a common denominator. They all need methods to insert, update, or delete records from the database. They also need to provide users the ability to navigate the database records by providing methods to scroll to the first, next, previous, and last record in a data set. All these methods and functions need to be easily accessible from a menu bar and optionally from the toolbar. The approach followed by development languages like Visual Basic 6, which did not support inheritance, was to attach a menu bar/toolbar with each form. This involved quite a bit of repetitive work for the developers, as each form would have similar methods, menus, and toolbars. Inheritance allows us to abstract commonly used functionality to an ancestor form. Any form that needs that functionality can simply inherit from the ancestor form. In my earlier example of the accounting application, we would create an ancestor transaction maintenance form. This form would have a menu bar, a tool bar, and methods that get called from the menu bar/toolbar. The various transaction types would inherit from this ancestor and would not have to recreate these objects. This arrangement leads to easier code maintenance. Commonly used code is maintained at one place and if you need to change it, you only change it at one place and the change is replicated to all the descendant forms.

In this article I will continue building the Sirius for Windows (SFW) MDI accounting application introduced in my previous article (http://www.csharptoday.com/content.asp?id=1797&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq). In that article, I showed you how to build the basic MDI framework for the SFW application. This included a detailed menu structure for the MDI Container form and for the MDI Child forms. I also showed you how to build toolbars and context menus for the application.

In this article, I will assume that you have already created the MDI framework as outlined in my previous article. This article will attempt to show you how to use inheritance to build a MDI Child framework.

I have used the RC version of Visual Studio.Net to build this application.

1. SFW's application design

An accounting application needs to show many facets of a transaction entry form. A transaction can be a journal entry, a sale transaction, a purchase transaction, or a cash transaction. Each of these transaction forms needs methods to interact with the database in order to insert, modify, or delete records. They might also need to provide some common features like scrolling, filter, or sort capabilities.

Figure 1 shows the application design of SFW. This application comprises of a MDI Container form. The MDI Container menu bar serves as the navigation tool for the entire application. All the MDI Child forms are accessible from the MDI Container form menu bar. The MDI Container form also handles the window form management functionality (i.e. the ability to tile or cascade open MDI Child forms) and the Help functionality. As every MDI Child form will merge the MDI Container menu bar with its own menu bar, the individual MDI Child forms will not have to write code for help and window form management.

SFW has a common base (ancestor) form for all the transaction entry forms. The menu bar on this form gets the application navigation and windows management functionality of the MDI Container window by merging the MDI Container menu with its own. It adds menu items that will be used by all the transaction forms. These menu items are to add, save or delete database records. Skeleton methods (i.e. method signatures) called add(), delete() and save() are also created in the body of the form. These methods are called from the clicked event of the menu items. These methods will be overridden by the descendant forms who will provide their own implementation of these methods. They serve as placeholders for the actual calls and serve to link the menu items to method signatures.

You create a derived (descendant) form that inherits from the base form for each of the transaction types. Thus you will have a Bank transaction form, a Sales transaction form, etc, all of which are descendants of the base form. The derived forms will inherit all the menu items of the MDI Container form, as well as the menu items and method skeletons of the base form.

Figure 1 shows how the various menus and methods flow into each other. Since the derived forms inherit the methods of the base form, I have shown the method names prefixed by an ":" in Figure 1. This denotes inheritance. These methods will now be scripted by each of the descendents according to the logic and rules pertaining to that form.

Figure 1: The inheritance hierarchy

In the demonstration that follows, two projects are created within the same solution. Project #1 is called "Framework" and this holds the MDI Container form and various derived transaction forms. Project #2 is called AncestorLib and it holds the base MDI Child class that all transaction MDI Child classes inherit from.

2. Create Project #1 and the MDI Container form/menu

In this step, Project #1 is created. This project will hold the MDI Container form and various derived transaction forms.

  • ·Create a new windows application by revving up Visual Studio and selecting File/New/Project/Visual C# Projects in the left pane and Windows Application in the right. I have called this project "Framework".
  • Delete the Form1.cs file that is created by default.
  • In my previous article (MDI and C#), you created the MDI Container form called MDIframe.cs. Add this form to the current project by right-clicking on the project name in the Solution Explorer and selecting Add/Add Existing item and browsing to locate the form MDIframe.cs in the project created in the previous article.
  • Open MDISheetTest.cs in design view. Right click on the menu caption Activity and select Properties. This menu item might be called menuItem1. Call it mActivity.
  • Also add the MDI Child form (MDISheetTest.cs) created in my previous article to the current project in the same way
  • Change the project namespace as appropriate. The namespace I am using is Sirius and MDIframe.cs makes a reference to this namespace. To change the namespace, right click on the project name in the Solution Explorer and select Properties. Change the Default Namespace property to Sirius or the namespace of your choice. Figure 2 shows how the MDI Container form with the menu bar should look like at this stage.

Figure 2: The MDIFrame

3. Create Project#2 and an Ancestor form/menu

In this step you create the base MDI Child form. The base form needs to be in a project separate from its descendants. It also needs to be compiled into a DLL. Please follow the following steps to create this form:

  • Create a new project within current solution by selecting File/New/Project and then Visual C# Projects in the left pane and Class Library in the right. Be sure to check the button that says Add to Solution. This is shown in Figure 3. I have called this project AncestorLib. Note that I selected this as a Class Library instead of as a Windows form. This will produce a DLL instead of an exe. I want a DLL also because I will not be implementing the ancestor, but just inheriting from it.

Figure 3: Adding a new project to the solution.

  • ·After performing this step you will have two projects in the current Solution. The project created in this step holds the ancestor (base) form, and the project created in step 2 will hold all the derived forms and the MDI-Container form.
  • It's better to have all objects in the same namespace. When the new project is created it may be in a different namespace than the descendent. Modify the namespace attribute by right-clicking project and selecting Properties/General/Default namespace. I have created all my classes in the Sirius namespace.
  • Delete the Class1.cs form that is created by default.
  • Add the MDI Child form MDISheetTest.cs that was created in my last article (MDI and C#) to this project by right-clicking on the project name in the Solution Explorer and selecting Add/Add Existing item and browsing to locate the form MDISheetTest.cs in the project created in the previous article. We will have this form act as the ancestor or base class for all transaction types.
  • Rename MDISheetTest.cs by right clicking on it in the Solution Explorer and choosing Rename. Call it ActivityAncestor.cs
  • Now view the code of ActivityAncestor.cs (right click in Solution Explorer and select View Code). You will note that even though you renamed the file, the code makes reference to its old name MDISheetTest.cs. Do a find and replace (from the Edit menu option) so that ActivityAncestor replaces all occurrences of MDISheetTest.

4. Create a descendant form

In this step, I will create a Bank transaction form. In a real application this form will be used to record bank transactions. For the purposes of this demonstration however, I will simply display message boxes from various methods.

A bank transaction is a is-a type of transaction and hence can derive from the ActivityAncestor class I created in step 3. The is-a model of inheritance is explained in my next article (The Theory of Inheritance in C#). The derived form will inherit the menu, toolbar, and methods of the Ancestor form. As you will remember, the methods coded in the Ancestor forms were add, save, delete. The derived form will provide its own implementation of the add, save and delete methods by overriding these.

  • In project #1 (Framework) inherit from the ActivityAncestor form created in step 3 by right-clicking on the project name and selecting Add/Add Inherited Form. Call the new form Bank.cs.
  • From the selection window that appears (called Inheritance Picker), select the ActivityAncestor form as shown in Figure 4.

Figure 4: The inheritance Picker window

  • For some reason, Bank.cs is not created in the specified Default Namespace of Sirius, which was set in step 3. To fix this, open this file in code view and change the namespace to Sirius or your default namespace.
  • Notice how the Bank class inherits from the ActivityAncestor class.

    public class Bank : Sirius.ActivityAncestor
    {
    }

  • Add code in the MDI Container menu item mBank to open the Bank form. Open the MDIframe.cs form in design view. Locate the menu item captioned Bank/Cash under the Financial Accounts menu. Double click on it to get a blank event handling code skeleton. Add the following code:

    private void mBank_Click(object sender, System.EventArgs e)
    {
    //Create a new instance of Child
    Bank NewMDIChild = new Bank();
    //Set the parent of the new MDI child form.
    NewMDIChild.MdiParent = this;
    //Display the new MDI child form.
    NewMDIChild.Show();
    }

  • Build and run the solution at this time just to make sure everything is going fine. You can do this by pressing F5. If you get build errors, check and fix the namespace of Bank.cs to your default namespace as explained above. Click on the Bank/Cash menu item to get the Bank form and ensure that you are able to call the add, save, close, and delete methods (by clicking on the appropriate menu items) on the Bank form. These methods have been coded in the ActivityAncestor form.

5. Extend objects on the derived form

You will note that in Bank.cs, you cannot add menu items onto the menu bar. Also, when you click on the toolbar, nothing happens (i.e. Visual Studio.NET does not create a blank method skeleton). In other words Visual Studio.NET is not allowing us to inherit from the base form objects. This is because the Modifiers property of the various objects and methods on the ActivityAncestor form are marked as Private. You need to change this to Protected or Public in order to extend or override this.

(If you are using Beta2 version, you see Family instead of Protected in Visual Studio.NET). Change the Modifiers property (right click the object and select Properties/Modifiers) to Protected for the mainMenu1, imageList1 and toolbar1 objects, and the top level menu item mActivity, and then build the solution. Now you can extend the descendant menu. In the code view of ActivityAncestor.cs, note that the correct modifier Protected has been used to assign visibility to these objects.

To check that you can indeed add on to the menu in the derived form, add a couple of menu items. I have added the following:

Menu caption \ Menu name

Main Menu

Bank reports \ mBankReports

Sub-menus

Receipts Register \ mReceiptsRegister

Payments Register \ mPaymentsRegister

6. Override methods in the derived form

The Bank form will provide its own implementation of the add, delete and save methods that have been coded in the AcitivityAncestor form. In order to do so, the Bank form will create methods with same names but will mark them with the override keyword.

ActivityAncestor, which was created by renaming the MDIShetTest.cs form (created in my previous article) will have a visibility of private for these three methods. This means that any form that inherits from ActivityAncestor will not be able to access these functions. Hence their scope (or visibility) have to be changed to protected. In ActivityAncestor, these method names should also be marked with the virtual keyword. As both the base and derived forms have methods with same names, the virtual keyword tells the compiler that the relevant method associated with a particular instance of the class should be called. Hence the appropriate method on the Bank form is called instead of the method with the same name on the ActivityAncestor form.

The three methods on the ActivityAncestor form should look like this:

protected virtual void add()
{MessageBox.Show("Add clicked");}

protected virtual void save()
{MessageBox.Show("Save clicked");}

protected virtual void delete()
{MessageBox.Show("Delete clicked");}

The Bank forms implementation of the three methods should look like this:

protected override void add()
{MessageBox.Show("Derived Add clicked");}

protected override void delete()
{MessageBox.Show("Derived delete clicked");}

protected override void save()
{MessageBox.Show("Derived save clicked");}

Now when you build and run the application, you will observe that the add, delete and save methods on the Bank form are called when you select the appropriate menu option from the menu.

You can find the source code for this article in the subfolder called "FrameWork". It includes the complete Visual Studio generated application files. I have also provided a subfolder called "SelfCompile" under the Framework folder which has all the required files for this application and which allows you to compile the application manually (and use a text editor instead of the Visual Studio IDE).

The Design Philosophy

The design of this application is based on the classical theory of inheritance (also known as the is-a model of inheritance). I intend to discuss this model at a greater length in the next article in this series (The Theory of Inheritance in C#). What this model says is that we look for an is-a relationship between classes. Thus Bank, Sales, Purchases, Journal Entries and so on are all types of Transactions and thus can inherit from a common ancestor. The implementation of functionality in this model flows from the abstract to concrete. Thus an ancestor class will define data (methods, fields and properties) in an abstract way, often just providing method signatures and leaving it to the descendants to provide a concrete implementation of such functionality.

If used inappropriately, this model can become top-heavy. Too much non-generic functionality can be added to the ancestor. Another drawback is that the ancestor and descendant objects are very tightly coupled with each other, creating very rigid objects. For this reason another architecture, called the Service-based architecture, is used in conjunction with the classical inheritance model. A service class contains non-visual methods and data and each service is responsible for a single, distinct function. When a base object requires this service, it is instantiated and all work is delegated to the service object. These service objects are very loosely coupled to one another. As an example of the two approaches, consider the interaction with the database. Each of the transaction forms would have to interact with the database to add, insert or delete records from the database. We could write a couple of methods to interact with the database and either place them in the base class ActivityAncestor or write a separate class (called say dbClass) that contains this functionality. Then we could declare this class with protected scope in the base class (so that derived classes can also use its functionality) and instantiate it there (using the new keyword) as follows:

protected dbClass mydb = new dbClass();

We can now call all its functions by using the mydb reference. For example, if this class contained a function called RunSql that accepted a SQL statement, we would call it as follows:

mydb.RunSql("Delete from transactions where doc_NO = 1")

In this way we have avoided loading the base class with the database interaction functionality. Another advantage flowing out of this approach is that the dbClass is not coupled with the ActivityAncestor class and can be used with classes that are not derived from the ActivityAncestor class.

The strategy I have adopted in this application is that I have used method overriding to call appropriate methods on the derived forms. I know that each of my transaction forms will have to interact with the database to add, save and delete data. I wanted to have this functionality available from the menu bar and toolbar on each MDI Child form. Instead of having a menu bar and a tool bar on each child sheet, I had it on the ancestor. I passed on the click event of the menu bar and toolbar onto methods coded on the ancestor form. Each of the descendant forms had methods with the same names, which override same-named methods of the base form method and provided its own implementation. In this way, I was able to avoid duplicating code.

The disadvantage of this approach is that I cannot remove a menu item or a toolbar icon from the menu bar or the tool bar. This problem however is very easily overcome. I can hide or disable (i.e. gray out) unwanted items on the menu bar or tool bar. Here is how I can hide or disable a toolbar or a menu item on the base form from a derived form (this code is commented out in the bank.cs form):

/*Examples of extending the framework*/
base.toolBarButton1.Enabled = false;
base.toolBarButton1.Visible = false;
base.mAdd.Visible = false;
base.mAdd.Enabled = false;

A very interesting consequence of using this approach is that you can put all the (instances of) descendents of ActivityAncestor in an array and deal with them together. I have provided an example in the ColorChange subfolder in the code download. In an accounting application, each transaction type has a different color (thus in this example all Bank transaction instances are Red and all Sales transaction instances Bisque) . I want to give the users of my application the ability to choose the color they want for various types of transactions. I have added a method to ActivityAncestor (the base form) to change the form BackColor as follows:

public virtual void ChangeFormColor(Color
            c)
{
this.BackColor=c;
}

I have also added a new derived form called Sale.cs, which is basically a copy of Bank.cs. I added this form so that there were more transaction types for demonstration. I have also created a test form called EntryPoint.cs which has the main() function and is the entry point of this demonstration. You do not need to go through the Visual Studio IDE as I have provided a bat file to compile the program. The EntryPoint class has a method called testButton_click() which gets fired when you click on the test button that resides on this form. This is its code:

protected void testButton_Click (object
            sender,System.EventArgs e)
{

ActivityAncestor[] TheArray
            ={new ActivityAncestor(),new Bank(),
new Sales()};

for(int
            i=0; i <TheArray.Length; i++)
{
Type t =
            TheArray[i].GetType();
string s =
            t.FullName;
switch(s)
{
case
            "Sirius.ActivityAncestor":
TheArray[i].ChangeFormColor(Color.Orange);
break;
case
            "Sirius.Bank":
TheArray[i].ChangeFormColor(Color.Red);
break;
case
            "Sirius.Sales":
TheArray[i].ChangeFormColor(Color.Bisque
            );
break;
}
MessageBox.Show(s);
TheArray[i].Show();
}

}

Since the Bank and Sales classes are derived from the ActivityAncestor class, they can be cast to the ActivityAncestor class and put in an array. It is now a simple matter to iterate the array and call the ChangeFormColor on the base form by passing it a color object. The thing to be noted here is how I obtain the class name. The System.Object defines a method named GetType() that returns an instance of the Type class. I use the FullName property of the Type class to extract the name of the class. Figure 5 shows the resulting output.

Figure 5: Handling classes as an array

From the above example you will note a strange phenomenon. I am actually instantiating the ActivityAncestor class using the new keyword. The real purpose of this class is to provide default state data and behavior for sub classes. I do not intend for this class to be actually instantiated and used. In other words, the base AncestorActivity class, as it is, is a concrete class. I should have made it into an abstract class. I have provided a sub folder called "Abstract" in the code download, in which I have made the ActivityAncestor class abstrac t. I did that by adding the abstract keyword to the class definition as follows:

abstract public class ActivityAncestor : System.Windows.Forms.Form
{
}

If I now try to instantiate this class, I will get an error as shown in Figure 6.

Figure 6: You cannot instantiate an abstract class

I have also marked the add() method as abstract. An abstract method does not provide a default implementation. The derived classes are given the responsibility of providing details behind abstract methods, thus enforcing polymorphism. If a derived class does not provide an implementation of an abstract method, it too is considered an abstract class and cannot be initiated. This is how the add() method now looks in the ActivityAncestor class:

abstract public void add();

The testButton_Click method of the EntryPoint class calls the add() method of each derived class as follows:

protected void testButton_Click(object
            sender,System.EventArgs e)
{

ActivityAncestor[] TheArray
            ={new Bank(),new Sales()};

for(int i=0; i
            <TheArray.Length;
            i++)
{

TheArray[i].Show();
TheArray[i].add();
}

}

The resulting output is shown in Figure 7:

Figure 7: Implementing abstract class and methods

Summary

The inheritance capabilities of C# can be used to create elegant MDI Child frameworks. One such approach was shown in this article. The most generic or commonly used functionality is abstracted away to an ancestor form. The derived forms can then extend or override this functionality as required. In my next article, I will discuss the theory of inheritance in detail and will consider how it applies to fields, methods, properties and classes.

Article Information
Author | Hersh Bhasin
Technical Editor | Adam Ryland
Project Manager | Helen Cuthill
Reviewers | Matthew MacDonald, Saurabh Nandu

If you have any questions or comments about this article, please contact the technical editor.


Introducing XGen XSQL for .NET

Introducing XGen XSQL for .NET
http://www.csharptoday.com/content.asp?id=1811
Article Rating: 4.5
Published on January 31, 2002
Abstract
In today’s article, Paul Looijmans introduces you to XGen, an open-source XML-based XML Generation framework for Microsoft’s .NET platform written entirely in C#. The first extension of this framework is XSQL, a set of tags for generating XML from relational databases. The article contrasts XGen XSQL with SQL Server’s FOR XML solution for generating XML and discusses some of the specifics of both XGen and XSQL.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

Ever since XML became popular a few years ago, its relationship with relational database technology has been the topic of a lot of discussion. XML evangelists tout its flexibility and simplicity, while proponents of the relational database solution point to the dangers of storing information in native XML: information can get scattered throughout the filesystem and there is no proven method to link different XML documents together easily. This leads to a lot of information redundancy. The database proponents further point to the fact that XPath can't match SQL in terms of data selection and (especially) data manipulation capabilities.

The World Wide Web Consortium (W3C) (http://www.w3c.org/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq) is working hard to improve XML, and a lot of the arguments the relational database proponents used in the past are no longer valid. Standards such as XPointer which allows XML documents to be linked together, and the upcoming XQuery standard which defines a more powerful query language for XML, as well as the rise of so-called XML databases, have made XML a more viable solution for storing data.

On the other hand even some of the more hardened XML proponents have had to admit that using a relational database is the best solution for a wide range of problems, so the emergence of XML won't mean that relational databases will disappear anytime soon. Of course the different relational database management systems (RDBMS) like Oracle 9i (http://www.oracle.com/ip/deploy/database/oracle9i/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq), Microsoft SQL Server 2000 (http://www.microsoft.com/sql/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq) and IBM DB2 (http://www-3.ibm.com/software/data/db2/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq) haven't stood still during the rise of XML, and one of the focal points of the development of these databases has been their integration with the XML standard. It's fair to say Oracle is currently ahead of SQL Server 2000 in its XML integration, mainly because Oracle's release schedule is much shorter than Microsoft's: the new release of SQL Server, codenamed Yukon, is not scheduled for release anytime soon, but since the release of SQL Server 2000 Oracle has already released two significant upgrades to their RDBMS software. Microsoft is however making periodic upgrades of the SQL Server 2000 XML capabilities available through the so-called SQLXML Web Releases (http://www.microsoft.com/sql/techinfo/xml/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq).

The SQLXML Web Releases have so far never used the .NET platform for XML-enabling SQL Server 2000 even though the platform has been in public beta for more than a year. This is where XGen, the subject of today's article, comes in: XGen is an open-source framework for generating XML from any data source. It is based on the .NET platform and completely written in C#. Today, I will introduce the XGen framework to you and discuss XSQL, which uses XGen to generate XML from any relational database that supports OLEDB. In a follow-up article, I will discuss the technical implementation of XGen and XSQL more in-depth.

In order to better appreciate the flexibility of XGen XSQL I will start off by briefly discussing the functionality that SQL Server 2000 combined with the Web Releases offers for generating XML from relational databases so I can contrast Microsoft's solution with the XGen XSQL solution.

The SQL Server 2000 solution: FOR XML & Templates

The biggest advantage of exposing information in relational databases as XML is that the information can be used by any program on any platform, from a huge database on a clustered server to a spreadsheet on a Palm computer. XML's simplicity and flexibility combined with the backing of all industry giants has made it the de-facto standard for exchanging information. Another advantage is the ever-growing array of XML-related standards like DOM for programmatically accessing and changing XML, XPath for selecting specific fragments in an XML document, XSLT for transforming XML, XML Schema (XSD) to define XML structure, and many more. When SQL Server 2000 was released, it contained a new way to generate XML from the database. By using the FOR XML clause, the database, in conjunction with the SQL Server OLEDB provider, was able to generate XML. As an example let's say we have the following query:

SELECT * FROM Authors;

To generate XML instead of the usual ADO recordset or ADO.NET DataReader from this query we would have to modify it like this:

SELECT * FROM Authors FOR XML [raw|auto|explicit];

As you can see you have to specify a mode with the FOR XML clause. There are three possibilities:

  • Raw: The most basic way to format your results. All rows in the recordset are transformed to ROW xml elements with attributes matching the column names.
  • Auto: This mode is slightly more advanced than the raw mode. The biggest difference is that the auto mode puts a little more hierarchy into your results when the query is a JOIN query, although you'll have to stick to very strict implicit rules that make it hard to easily use this feature.
  • Explicit: This mode gives you complete freedom over the format of the generated XML. This freedom comes at a high price though: you will have to build a so-called universal table in your query that adheres to strict rules regarding column names and structure. When your XML document is more complex than 2 or 3 levels of nesting writing these FOR XML EXPLICIT queries becomes very cumbersome and the result is very hard to debug and maintain.

The generated XML can be accessed through ADO and ADO.NET, but since XML is often document-centric it is also possible (since SQLXML WebRelease 1) to store the FOR XML query in an XML file that can contain a small set of special tags that can generate XML. Upon request an IIS ISAPI application replaces those special tags with the results of a query. An example of a template is:

<author>
<sql:query xmlns:sql="urn:schemas-microsoft-com:xml-sql">
SELECT TOP 2 * FROM Authors FOR XML RAW
</sql:query>
</authors>

When requested through IIS, the file is processed on the server and would look like this on the client:

<authors>
<row firstname="Stephen" lastname="King"/>
<row firstname="William" lastname="Shakespeare"/>
</authors >

As you can see it's possible to generate XML from SQL Server 2000. So what's the problem, you might ask. The biggest problem is the complex T-SQL code you have to write when you want total control over the way your XML is presented. Like I said before, interoperability with different programs and platforms is a big plus, but this interoperability implies that your XML is formatted in a standard way, often based on XML Schemas or DTDs. The FOR XML EXPLICIT method forces you to build long queries with sometimes complex logic that is very hard to maintain. Another problem is the fact that the FOR XML solution cannot be extended in any way, you're stuck with the RAW, AUTO and EXPLICIT modes. When you have some kind of XML generation task that you want to reuse, you'll have to use the good old copy/paste method, a.k.a. a maintenance nightmare. The XGen framework, which is discussed in the following section, tries to solve these problems.

The XGen solution: XSQL

As mentioned in the introduction, XGen is an open-source framework for generating XML from any data source. XSQL uses the XGen interfaces and base classes to build so-called taghandlers for tags that generate XML from any relational database that supports OLEDB.

Both XGen and XSQL are based on the .NET framework, written in C#, and open-source so you can see how all the different taghandlers are implemented. If there is no taghandler available for your specific needs it's easy to extend the existing classes to add any type of XML generation functionality you might need. This extensibility is definitely one of the strong points of the XGen framework and something that wouldn't be possible in the 'old' ASP/VB-COM world.

Before I continue with the description of some of the technical aspects of XGen and XSQL, it's important to clarify the relationship between the XGen XSQL and Oracle's XSQL. It's fair to say that Oracle XSQL was the inspiration for building the XGen framework and XGen XSQL. After building a very data-intensive application in record time using Oracle XSQL, I decided the .NET platform needed a similar tool for quickly building XML driven applications. Because Oracle XSQL is a closed-source application and is based on a different platform (Java, Servlets, & JDBC), the code base for both applications has very little in common. Also, Oracle XSQL is very much targeted towards the Oracle 8i/9i databases and its PL/SQL language, offering several Oracle-specific tags. Although Oracle XSQL is essentially database-agnostic Oracle doesn't go to great lengths to emphasize this, choosing instead to focus squarely on using XSQL with an Oracle database, which is understandable from their point of view.

XGen XSQL on the other hand offers some tags with advanced relational-XML logic, and since it is based on the XGen framework it offers better extensibility. Furthermore, XGen and XGen XSQL take full advantage of the features the .NET platform offers, including state of the art XML processing and data access capabilities as well as the revolutionary ASP.NET platform. In short, Oracle XSQL and XGen XSQL share their name and a common purpose, namely generating XML from relational databases, but not much else. If you want more information about Oracle XSQL, you can download the Oracle XDK for Java (http://otn.oracle.com/tech/xml/xdk_java/content.html?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq), read this extensive introduction into XSQL (http://download-east.oracle.com/otndoc/oracle9i/901_doc/appdev.901/a88894/adx10xsq.htm?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq), or view the release notes (http://otn.oracle.com/docs/tech/xml/xdk_java/doc_library/Production9i/java/xsql/readme.html?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq) for the latest version, which also includes some examples. You have to become an OTN member (http://otn.oracle.com/membership/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq) before you can access the links.

The XGen XML Generation Framework

Since XSQL is built on top of Xgen, it's important to have a basic understanding of what XGen does. If you want more information about XGen's structure, you will find the documentation at http://www.codeworks.nl/xml/xgen/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq. XGen can be found in the Codeworks.XGen namespace and is made up of three parts:

  • The XGen processor, to process XGen files into XML. Basically all the processor does is walk through the XGen XML file and call a few methods from the handler class associated with the tag or processing instruction in the XGen configuration.
  • Interfaces that a handler must implement so that the XGen processor can call it (either an ITagHandler or an IpiHandler, which is a handler for a processing instruction).
  • XML, like DecodeLexicalParams, which allows you to place a parameter in your XML that is resolved at runtime from an ExecutionContext dependant collection of parameters. For example, when an XSQL page is called through IIS (using an ASP.NET HttpHandler, the ASP.NET alternative to ISAPI applications) the parameters collection contains values from the Session, Cookie, Form and QueryString collections respectively. The XGen syntax for parameters is as follows: SELECT * FROM Authors WHERE id={@id}; The id parameter is inserted at runtime, so when you post a form with an id field containing the value 20, the query executed at runtime would be: SELECT * FROM Authors WHERE id=20; It would be very cumbersome if every handler would have to implement its own parameter parser, that's why it's easier to extend TagHandler, which includes a method to decode parameters, than to implement ITagHandler, which, being an interface, can only contain method and property signatures.

As you can see, the basic XGen architecture is very simple. The XGen processor can be called from any .NET class by simply calling one of the XGenProcessor's overloaded ProcessToXXX methods. The Codeworks.XGen.Web namespace contains the classes to use XGen with ASP.NET: an ASP.NET HttpHandlerFactory, an HttpHandler (not to be confused with an XGen tag- or processing instruction handler) and an XGen specific implementation of the well known System.Web.UI.Page class that adds a Content property through which you can access the processed XGen page. By using the classes in the Codeworks.XGen.Web namespace you can use ASP.NET pages as a stylesheet for your XGen pages. I'll be showing you how to do this in the sample application at the end of the article.

Most of the complexity of generating XML is left to the handlers. In the next section we'll take a look at these handlers by examining an example from XGen XSQL.

XGen XSQL: Generating XML from relational database

XGen XSQL is nothing more than a collection of classes that extend TagHandler and generate XML from SQL statements. The simplest example is the query taghandler whose tag at first glance looks a lot like the SQL Server 2000 template example. In its most simple form an XML file with XSQL tags looks like this:

<results>
<xsql:query xmlns:xsql="www.codeworks.nl/XGen/XSQL/1.0" connection="pubs">
SELECT * FROM Authors;
</xsql:query>
</results>

As you can see, the XML is nearly identical to the SQL Server 2000 template example. There are really only two noteworthy differences:

  • A connection attribute was added. An XSQL Query tag requires either a connection or a connection string attribute identifying which database it should use to execute the query. The connection attribute specified here has the value "pubs", which is linked to a connection string in the XGen configuration file (web.config in a web setting).
  • There is no FOR XML clause in the SELECT statement. Since XSQL is database-agnostic it doesn't use database specific XML generating features like SQL Server's FOR XML. Of course if want to make use of SQL Server's FOR XML feature, you can always write a taghandler that does this yourself.

After processing the XML file, the XGenProcessor returns the following XML:

<results>
<rowset>
<row id="1">
<firstname>Stephen</firstname>
<lastname>King</lastname>
</row>
<row id="2">
<firstname>William</firstname>
<lastname>Shakespeare</lastname>
</row>
</rowset>
</results>

The result is formatted differently than the SQL Server template example: columns are mapped to individual nodes instead of to attributes, the row tag has an id attribute, and a rowset element was added. If this was all XSQL did, it would offer exactly the same functionality as SQL Server 2000 does with only slightly different results.where is the extra power?

First of all, the XSQL Query tag offers an extensive set of attributes that you can use to modify the XML output. To name but a few:

  • The columns-as-attributes attribute allows you to define a comma-separated list of columns that should be added to the row element as attributes instead of as extra nodes, which is the default.
  • The page-size attribute, together with the page attribute, allows you to page through the database results easily.
  • The rowset and row attributes allow you to define the names of the rowset and row elements in the output.
  • The tag-case attribute allows you to set the case for all elements and attributes in the output.

These attributes allow you to format your output XML in a lot of ways and thus allow greater flexibility.

Secondly, if the XSQL Query tag doesn't offer the kind of XML formatting functionality you are looking for, it's easy to build your own TagHandler extension that does. Most of the time all you do is extend the XGen TagHandler base class and override one or both of the methods that that base class offers. Those methods are Initialize and HandleTag, which are both pretty self- explanatory. In the follow-up article I'll walk you through the steps needed to build your own taghandler by looking at one of the standard taghandlers in XSQL.

XSQL Examples

We've seen the simplest XSQL tag in action already, but there are two more advanced tags that really show the power of the XGen framework and XSQL.

A simple example of the hierarchical-query-tag:

<xsql:hierarchical-query xmlns:xsql="www.codeworks.nl/XGen/XSQL/1.0"
connection="pubs">
<xsql:parentquery>
SELECT * FROM Authors;
<xsql:childquery parent-join-column="au_id"
child-join-column="au_id">
SELECT * FROM Titles
</xsql:childquery>
</xsql:parentquery>
</xsql:hierarchical-query>

This tag offers the ability to easily build any level of nesting into your XML output. The hierarchical-query tag offers many of the same formatting attributes as the query tag, including the ability to define which columns to show as attributes. These attributes can be set individually on each level of nesting, providing optimal flexibility.

The second, more advanced tag is my personal favorite because it shows you how a relational database, seen by many XML proponents as too rigidly structured compared to XML, can store information in a hierarchical manner with virtually no limitations, just like XML!

The so-called infinite-hierarchy query has a simple syntax:

<xsql:infinite-hierarchy
            xmlns:xsql="www.codeworks.nl/XGen/XSQL/1.0"
connection="pubs"
parent-column="section_id"
child-column="parent_section"
columns-as-attributes="*"
child-rowset-element="">
SELECT
            * FROM Sections;
</xsql:infinite-hierarchy>

Producing the following XML:

<?xml version="1.0" encoding="utf8" ?>
<rowset>
<row id="1" Name="XML For Dummies">
<row id="2" Name="Introduction">
<row id="3" Name="What is XML?" />
<row id="3" Name="Why Use XML?" />
</row>
<row id="3" Name="XML Syntax">
<row id="4" Name="Element" />
<row id="4" Name="Attribute" />
<row id="4" Name="Comments" />
<row id="4" Name="Processing instruction">
<row id="5" Name="Common processing instructions">
<row id="6" Name="xmlstylesheet" />
</row>
</row>
<row id="6" Name="Entities" />
</row>
</row>
</rowset>

So how does it work? In its most basic form, the query has to contain two columns that contain the parent and the child identifiers. The columns that contain these values must be specified in the XSQL tag. The parent column can be any unique value, most of the time it will be an auto-increment integer column. The child column contains the unique value of its parent row, or NULL if it is a root node. The Access table that this example uses adheres to these simple rules. It contains all sections of a book. Some books will only have chapters, others will have a more complex structure, with paragraphs, etc. The table is included in the sample pubs.mdb database and looks like this:

Using ADO.NET, it's possible to easily build an infinite hierarchy: a hierarchy with arbitrary depths. Some examples of real-world applications of an infinite hierarchy are a filesystem, a threaded message board, or a website structure. If you stick to the simple rules that a parent column must be unique and that the child column contains the unique value of its parent row, then you can easily create XML using the infinite-hierarchy-query taghandler. Of course, there are other ways to create an infinite hierarchy in a relational database, if you want to define other rules for building an infinite hierarchy you can build your own TagHandler extension.

Presentation Options

After XGen has generated your XML, you may want to format it in a certain way for presentation. XGen offers a simple xml-stylesheet processing instruction handler that can transform the generated XML with an XSLT stylesheet. By writing custom processing instruction handlers you can format the results generated by XGen in any way you want. The Codeworks.XGen.Web namespace contains a special processing instruction handler that offers the aforementioned XSLT transformation capabilities, plus the ability to use ASP.NET pages as XML stylesheets. When a visitor requests an XGen page containing the following stylesheet declaration:

<?xml-stylesheet type="asp.net" href="Webform1.aspx" ?>

The XGen page is generated and the ASP.NET page is called to display the results. By extending the CodeBehind page from Codeworks.XGen.Web.UI.AspDotNet.Page instead of System.Web.UI.Page you gain access to a Content property that contains the XGen processing result in the form of an XmlDocument. Also included in the Codeworks.XGen.Web namespace is a set of XSLT-like webcontrols that allow you to easily access the Content property from an ASP.NET page and a TagHandler extension that generates XML containing the request parameters from the Querystring, Form, Cookie and Session collections.

The following code-fragment from an ASP.NET CodeBehind page shows how you can access the contents of the underlying XGen page's result through the Content property:

public class AspStylesheetDemo : Codeworks.XGen.Web.UI.AspDotNet.Page
{
protected System.Web.UI.WebControls.TextBox textBox;

public aspstylesheet()
{
Page.Init += new System.EventHandler(Page_Init);
}

private void Page_Load(object sender, System.EventArgs e)
{
this.textbox.Text = Content.SelectSingleNode("//rowset/row[@id = '1']/name/text()").Value;
}

This powerful feature allows you to completely separate data from presentation. You could use XSLT to transform the data on a page for use in pure XML applications like Web Services and use the ASP.NET stylesheets for presentations in HTML, allowing you to use the full power of the ASP.NET framework, including WebControls for reusable UI components and CodeBehind for a good separation between logic and presentation.

The following very simple example from an ASP.NET page shows you how you can use the webcontrols in the Codeworks.XGen.Web.UI.AspDotNet.Xslt namespace to quickly build data driven websites using XGen and ASP.NET:

<%@ Register TagPrefix="xslt" Namespace="Codeworks.XGen.Web.UI.AspDotNet.Xslt"
Assembly="Codeworks.XGen.Web" %>
<html>
<body>
<table>
<xslt:foreach select="//rowset/row" runat="server">
<tr>
<xslt:foreach select="/" runat="server">
<td>
< xslt:valueof select="."/>
</td>
</xslt:foreach>
</tr>
</xslt:foreach>
</table>
</body>
</html>

As you can see, the combination of ASP.NET and XSLT-like Webcontrols gives you a powerful tool for building very dynamic web pages that can access data from any data source. Webcontrols for conditional logic are also available (xsl:if, xsl:choose, xsl:when and xsl:otherwise). Of course, it's also possible to use the 'traditional' ASP.NET programming model with an ASP.NET page and a CodeBehind class. You could call the XGenProcessor from the CodeBehind class and process an XGen page there, the resulting XmlDocument contains the same XML as the Content property would when you an ASP.NET stylesheet. Again the keywords are flexibility, extensibility, and ease of use.

An XGen XSQL sample application

The sample application contains two sample files each for all three current XSQL taghandlers : a simple version with the bare minimum attributes and a complex version that uses some of the available attributes to format the results. The sample application is based on a modified version of the 'pubs' Access database, which is also included. It also contains examples of how to use the xml-stylesheet processing instruction with an XSLT stylesheet and an ASP.NET stylesheet. For installation instructions see the included readme.txt file. The compiled XGen and XSQL dlls are also included in the download, if you want access to the source files go to http://www.codeworks.nl/xml/xgen?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq.

The sample was written for and compiled on the final release of the .NET framework (http://msdn.microsoft.com/downloads/default.asp?url=/downloads/sample.asp?url=/msdn-files/027/000/976/msdncompositedoc.xml&frame=true&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq) but since you have access to all the sources, it shouldn't be too difficult to recompile the different projects to make the framework compile on the Beta 2 and RC 1 releases of the .NET framework.

Conclusion

I have tried to give you a pretty thorough introduction to XGen and XSQL. I hope the introduction has made you curious enough to try it out. If you think this article was a little light on in-depth code examples, I can assure you this will be generously compensated in the follow-up article, where I will discuss the actual code in XGen and XSQL. Some of the topics I'll discuss in that article are:

  • Writing custom ASP.NET handlers
  • Writing custom configuration section handlers
  • How the Xslt Webcontrols work
  • How to write good documentation using C#'s built in XML documentation tags and how to convert the generated XML into MSDN-library like documentation using the open-source NDOC program.
  • How to write a custom taghandler. If you have any ideas on what kind of taghandler is missing from XSQL at the moment let me know, maybe I'll build it for the upcoming article.
  • The reason for providing separate OLEDB and SQL Server versions of all standard taghandlers and how you can easily build your own database specific versions of the base taghandlers by following a specific inheritance pattern.

As said before, the entire project is open-source. Codeworks believes in the power of the XGen framework, and is dedicated to improving and maintaining it for the future. If you are interested in helping us or if you have any questions or comments, please don't hesitate to contact us (paullooijmans@codeworks.nl), we look forward to hearing from you. You can find all the latest information, documentation and software on XGen and XSQL at http://www.codeworks.nl/xml/xgen?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq.

Article Information
Author | Paul Looijmans
Technical Editor | Adam Ryland
Author Agent | Charlotte Smith
Project Manager | Helen Cuthill
Reviewers | Andrew Krowczyk, Iikka Meriläinen

If you have any questions or comments about this article, please contact the technical editor.


Emitting Dynamic Assemblies on the fly using Reflection.Emit

Emitting Dynamic Assemblies on the fly using Reflection.Emit
http://www.csharptoday.com/content.asp?id=1810
Article Rating: 4
Published on January 30, 2002
Abstract
Reflection.Emit is a powerful API on the .NET Platform that allows you to dynamically emit transient and persistable assemblies at runtime. Unlike the CodeDOM API’s that deal with generating language specific source code dynamically, Reflection.Emit works on a much lower-level, language neutral – MSIL. In this article Saurabh Nandu takes you through the nitty-gritty of working with Reflection.Emit API.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

As you already know, the .NET Platform is packed with extremely useful APIs. In previous C# Today / ASPToday articles (see the links at the end of the article) you have already seen how the metadata packed within .NET Assemblies can be queried and invoked through the Reflection API. You might have also read about the CodeDOM API that helps you to dynamically generate language specific source code, which can later be compiled into an assembly. In this article, we will be working with the Reflection.Emit API that deals with dynamically generating assemblies, but unlike CodeDOM API, deals on a much lower, MSIL (Microsoft Intermediate Language) level.

There are many benefits of having the capability to dynamically generate assemblies; a few of them are listed below:

1) Persist install time information in an assembly.

Many applications need to persist their preferences depending upon the operating system, hardware available, etc. Installer scripts can use the Reflection.Emit API to generate assemblies containing such information on the fly while installing the application.

2) Persist user specific (personalization) information into an assembly.

Personalized information is the key point in client desktop applications today; you can use the Reflection.Emit API to generate assemblies containing personalization information either at install time or later. Some of you might think that embedding personalization information in a XML configuration file is a better idea, but in some cases, for security reasons, persisting the information assemblies might prove to be a better solution.

3) Security of Intellectual Property

It is a practice in many current applications to distribute encrypted application libraries to the consumers, and the data within the libraries is read, decrypted and used from memory, only at runtime. Reflection.Emit allows you to generate transient assemblies that live only in the system memory.

4) Harnessing features not present in the coding language, but available on the .NET Platform

This is a hidden benefit that you get while working with the Reflection.Emit. Since you are working with MSIL and not your particular coding language, you can tap the features that are not supported by your language. For example, VB.NET does not support working with unmanaged code, but using Reflection.Emit in VB.NET you can generate an assembly that interoperates with unmanaged code.

5) Tool and Compiler Developers

Tools like obfuscators, Code Generators, etc, need to generate dynamic assemblies, and they can utilize the Reflection.Emit API to emit their assemblies.

Metadata and CLR

The .NET Platform consists of a very wide architecture in order to support multiple languages. MSIL, or Microsoft Intermediate Language, is the assembly language equivalent on the .NET Platform. All the managed languages compile into the same MSIL code, this is the magic that makes multi-language inheritance, multi-language interoperability, and multi-language seamless debugging possible. Under the hood, the CLR (Common Language Runtime, or CLI - Common Language Infrastructure, as per ECMA submissions) is a very sophisticated and wide-scoped to support multi-language constructs. The CLS (Common Language Specification), a small subset of the CLR, defines a set of rules that all compiler vendors should implement, could implement, or entirely leave out.

All the managed languages, including C# and VB.NET, have been constructed upon CLS, hence essentially the compiler developers of these languages have picked the functionalities from the CLR that they wanted and left a few functionalities that were not very relevant to their specific languages. For example, even though CLR supports interoperating with unmanaged code, VB.NET does not allow you to do this.

The motive behind the above discussion is the give you a feel of the depth of the CLR, since while working with Reflection.Emit you have the ability to tap into the full power of the CLR.

ILDASM your best friend

As I have mentioned before, Reflection.Emit deals with MSIL directly, and I know that most of us (including me!) get fizzed-out if someone asks to code in MSIL directly!! Here is where the trusted tool ILDASM (Intermediate Language Disassembler) comes in very handy. The best way I discovered to start working with MSIL is to first create sample test applications in C#, view them in ILDASM, and then use Reflection.Emit to mimic the C# compiler behavior. I also recommend you use the ILDASM /all /adv (switch) so that all the ILDASM options are available to you. The full specification for the metadata and CLR is not available in the .NET Reference Documentation, but it's available in the separate document file ' Partition II Metadata.doc ' generally placed in the

%drive%:\ Program Files\Microsoft.NET\FrameworkSDK\Tool Developers Guide directory.

Try comparing the IL codes from ILDASM with its function from the documentation file, and you will understand how you program actually works! It's tedious, but definitely fun!!

Some interesting observations

The C# compiler hides a lot of details of the assembly generated from the programmer to make the programmer's task easy. But while working with Reflection.Emit, you need to understand the services the C# compiler provides you, so that you can correctly write your assemblies. Following are some of the interesting jobs the C# compiler performs for you:

1) Initialize class level member fields in the constructor. Even though you don't provide a constructor for your class, the C# compiler automatically provides you with a default constructor for every class, and all the class level member fields are correctly initialized to their default values.

2) In C#, we generally define properties with get and set accessors. However, the C# compiler actually defines the property and two methods, get_PropertyName and set_PropertyName, which contain the code that you write within the get and set accessors.

3) Similar to properties, Indexers in C# also use the get and set accessors. Hence internally, (MSIL) indexers have the get_Indexer and set_Indexer methods that represent the get and set accessors respectively.

4) The delegate keyword used in C# to define a delegate is actually expressed in MSIL as a new class that extends the System.MulticastDelegate class.

5) An event defined is C# is expressed in MSIL with the following elements: a private class field with the data-type of the event handler delegate, an event definition, and two methods namely add_EventName and remove_EventName. These methods are used add and remove the event handling delegates to the event respectively.

6) The compiler-emitted code always uses fully qualified class names, i.e. the namespace name followed by the class name. So, the use of the using keyword is just useful for the C# compiler to map the class names with the relevant assemblies, at MSIL level there is no use of the using keyword.

7) Enumerations are emitted as sealed classes that extend from the System.Enum class.

8) Structures are emitted as sealed classes that extend from the System.ValueType class.

Follow these!

The Reflection.Emit API is built upon the Reflection API. In fact, most of the Reflection.Emit classes extend from their counterparts in the Reflection API. So, it would be a good idea to brush-up your Reflection concepts before you dive into Reflection.Emit ! One word of caution: Reflection.Emit does not check the validity and syntax of the IL code you are generating, and it's only at runtime that an error will pop-up if your code is wrong as one would have expected. So be prepared to battle with long hours of debugging in case you have a very large assembly to emit!!

You need to follow a particular order of steps to successfully emit an assembly. These are shown below:

1) Define the Assembly.

2) Define the modules within the assembly.

3) Define the Types within each module.

4) Define the Type members like fields, events, properties, methods.

4.1) If you are defining a method property then you need to define its internal members.

5) Create the Type and Save the assembly.

Personalization Example

I have created a little personalization and security example that demonstrates the use of Reflection.Emit. It's a general practice to distribute demo versions of the applications, which have some limited functionalities. If the user registers the application by paying a fee, the application is activated and the functionalities are restored.

In this example, I have a Windows Forms application, SecuredForm. This application relies on the library SecuredInformation.dll to work in demo or release mode. The default library that is distributed with this application restricts the application to DEMO mode. There is one more Console Application (GenerateSecured) that should remain with the original programmers. As soon as someone registers the application, a new library is generated using Reflection.Emit. Just copy this new ' SecuredInformation.dll ' (located within the SecuredForm directory) and now the application will run in release mode!!

Actually the power of this API would be seen when you would implement the GenerateSecured class as an ASP.NET Web Form and dynamically generate registered assemblies for registered users. I have refrained from taking that approach due to brevity of code!

I do understand that this alone is not a very secure solution, but remember the aim of this example is just to show you the usage of Reflection.Emit API! The full source code is available in the download material in Visual Studio.NET Beta2 format.

SecuredForm - The Windows Forms Application

As I described earlier, this is a Windows Forms application, and it picks up the values from the library SecuredInformation.dll to work either in debug or release mode. Below I have displayed the code of the Setu pForm method that checks the SecuredInformation.dll library and runs the application in the relevant mode:

private void SetupForm()
{
this.Text =
            SecuredInformation.ApplicationName;
// Get the Applications
            Name
companyLabel.Text = SecuredInformation.CompanyName;
//Get
            the Registered Companies
            Name
if(SecuredInformation.IsDemo)
{
advanceButton.Enabled
            = false;
//Enable advance functionalities only if mode is
            Release
}
else
{
advanceButton.Enabled
            =true;
}
}

SecuredInformation.dll - The Library

The SecuredInformation.dll library implements a few static properties that are used by the SecuredForm application. The code for the SecuredInformation library is shown below:

using System;
namespace
            SecuredApplication
{

public class
            SecuredInformation
{
private static string companyName ="DEMO
            Company";
private static string applicationName ="DEMO
            Application";
private static string applicationVersion
            ="1.0.0.0";
private static bool isDemo = true;

public
            static string CompanyName
{
get{ return companyName;
            }
}

public static string ApplicationName
{
get{
            return applicationName;}
}

public static string
            ApplicationVersion
{
get{ return
            applicationVersion;}
}

public static bool
            IsDemo
{
get{ return isDemo;}
}
}
}

Figure 1 below displays the SecuredForm application when the default SecuredInformation.dll library is present. You can see that the Advanced Functionality button is disabled since the application is working in demo mode.

Figure 1: Secured Application in Demo mode

GenerateSecured - The Console Application

The GenerateSecured is the console application that will be used by the manufacturer to generate a new version of the SecuredInformation.dll library with the registration details of the company. It takes the user input from the console screen and generates the necessary assembly based on that information, using the Reflection.Emit API. Figure 2 shows the GenerateSecured application.

Figure 2: GenerateSecured Console Application

Once the SecuredInformation.dll is generated by the GenerateSecured application, take this library and place it within the SecuredForm applications directory. Now run the SecuredForm application and you will be able to view the full functionality of the application, as shown in figure 3.

Figure 3: Secured Application in Release mode

The goal of the GenerateSecured application is to use Reflection.Emit to generate the SecuredInformation.dll library, which is similar to the compiled version of the code shown above (see the SecuredInformation section) with four static fields and four static properties. Also remember that these assemblies are strong named, so you need to use the same strong name key (generated by the sn.exe tool) while generating the demo and release version of the library. Since I did not want to limit my discussion to this example, I describe the general steps you should take in order to work with any application using Reflection.Emit below. The code snippets shown in these sections are combined to form the GenerateSecured application.

Let's Get Started!!

Define the Assembly

An Assembly is the packaging unit on the .NET Platform. All applications on the .NET Platform compile to assemblies, that are self-describing i.e. it contains metadata describing itself contained within the assembly.

AssemblyName class (System.Reflection namespace)

All .NET Assemblies contain an additional assembly manifest within its metadata. The .NET runtime cannot load an assembly that does not contain the assembly manifest. The assembly manifest contains the name, version, version compatibility, hash algorithm, and strong name, as well as the culture info of the assembly. The AssemblyName class is used to generate this assembly manifest. In case you want to produce a strong named assembly, use the StrongNameKeyPair class (System.Reflection namespace) along with the AssemblyName class.

Listing 1: Definition of the strong named assembly

//Open a stream to the key file to be used for strong naming
System.IO.FileStream fin =
new System.IO.FileStream("secured.snk", FileMode.Open, FileAccess.Read, FileShare.Read);
//Get a StrongNameKeyPair object
System.Reflection.StrongNameKeyPair kp =
new System.Reflection.StrongNameKeyPair(fin);
fin.Close();
//Define the AssemblyName
System.Reflection.AssemblyName asmName = new System.Reflection.AssemblyName();
asmName.Name ="SecuredInformation";
asmName.Version = new Version(1,0,0,0);
asmName.KeyPair=kp;

Persistable and Transient Assemblies

Before proceeding any further, you have to first make the critical decision on what kind of an assembly you want to generate. There are basically two types of assemblies you can generate:

1) Transient Assemblies - These are created in memory, and they can reference other persisted assemblies. Since they are created are at runtime persisted assemblies cannot reference them. Once you shutdown the application that created the assembly, it's removed from the memory.

2) Persistable Assemblies - These are created and saved to the disk for future reuse.

The AssemblyBuilderAccess enumeration (System.Reflection.Emit namespace) defines three options that can be used while defining your dynamic assembly:

a) Run - Use this option if you want to define transient assemblies.

b) Save - Use this option to define persistable assemblies. You cannot invoke any members of this assembly unless you save it first.

c) RunAndSave - Use this option to define persistable assemblies that you can invoke before /after saving the assembly to disk.

AssemblyBuilder class (System.Reflection.Emit namespace) and AppDomain class (System namespace)

To define a dynamic assembly we use the AppDomain object. We have the choice whether to define the new dynamic assembly either within the same application domain in which the current application resides, or inside a new application domain. Various overloads of the method DefineDynamicAssembly from the AppDomain class are used to define a new dynamic assembly, this method returns a AssemblyBuilder object. In case you are building a Console or Windows application, remember to call the SetEntryPoint method of the AssemblyBuilder class and pass it the MethodBuilder object for the Main method that will start the application.

Listing 2: Define a Dynamic Assembly

AssemblyBuilder asmBuilder = System.AppDomain.CurrentDomain.DefineDynamicAssembly(asmName,
System.Reflection.Emit.AssemblyBuilderAccess.Save);

The code snippet above defines a new dynamic persistable assembly within the same application domain.

Define modules within the assembly

Each assembly can consist of multiple modules and resource files. Each module can contain multiple Types.

ModuleBuilder Class (System.Reflection.Emit namespace)

The DefineDynamicModule method from the AssemblyBuilder class returns a ModuleBuilder object that's used to create dynamic modules. The string parameter passed to the DefineDynamicModule is generally the filename that will be used to save the assembly.

Listing 3: Defining a dynamic module

ModuleBuilder asmModule =
asmBuilder.DefineDynamicModule("SecuredInformation.dll");

Above listing defines a new dynamic module with the name " SecuredInformation.dll ".

Add Resources

If your assembly depends on resource files, you can use either use the AddResource method from the AssemblyBuilder class to add a resource file that has already been created, or use the DefineResource method of the AssemblyBuilder class that returns a ResourceWriter object (System.Resources namespace). This object can later be used to customize the resource files.

Listing 4: Add an existing resource file

asmBuilder.AddResource("SecuredApplication.SecuredForm","SecuredApplication.SecuredForm.resource");

The above listing adds an existing manages resource file ' SecuredAppslication.SecuredForm.resource ' into the assembly.

Define Types within each module

Each module can contain a number of classes, structures, enumerations, and interfaces. Remember that even delegates, structures, and enumerations are actually implemented as Types.

TypeBuilder Class (System.Reflection.Emit namespace)

The DefineType method from the ModuleBuilder class is used to define a new TypeBuilder object. There are various overloads of this method so be sure to pick-out the right overload to suit your class requirements. You can use the TypeAttributes enumeration (System.Reflection namespace) to define various class attributes like Sealed, Interface, Abstract, Public, Private, etc.

Listing 5: Define a class

TypeBuilder asmType =
asmModule.DefineType("SecuredApplication.SecuredInformation",
TypeAttributes.Class |
TypeAttributes.Public | TypeAttributes.BeforeFieldInit);

The above listing defines a class " SecuredApplication.SecuredInformation ". Remember that MSIL only deals with fully qualified class names, and hence it will automatically mark " SecuredApplication " as the namespace.

Listing 6: Define an Interface

TypeBuilder asmInterface =
asmModule.DefineType("SecuredApplication.IDescription",
TypeAttributes.Public |
TypeAttributes.Interface |
TypeAtributes.Abstract |
TypeAttributes.AutoClass |
TypeAttributes.AnsiClass );

Define Type members like fields, properties, events and methods

Type members are the most important part of a Type.

FieldBuilder class (System.Reflection.Emit namespace)

The FieldBuilder class is used to create class level members. The DefineField method from the TypeBuilder class is used to create new FieldBuilder objects. The FieldAttributes enumeration (System.Reflection namespace) is used to define field attributes like Public, Private, Static, etc.

Listing 7: Defining Fields

FieldBuilder appName =
asmType.DefineField("applicationName",
typeof(string),
FieldAttributes.Private|
FieldAttributes.Static);

FieldBuilder appVer =
asmType.DefineField("applicationVersion",
typeof(string),
FieldAttributes.Private|
FieldAttributes.Static);

FieldBuilder compName =
asmType.DefineField("companyName",
typeof(string),
FieldAttributes.Private|
FieldAttributes.Static);

FieldBuilder isDemo =
asmType.DefineField("isDemo",
typeof(bool),
FieldAttributes.Public|
FieldAttributes.Static);

The above listing defines some static fields.

ConstructorBuilder Class (System.Reflection.Emit namespace)

Constructors are a part of every class. The C# compiler defines a default class constructor for every class, but while using Reflection.Emit you have to remember to add a constructor to every class. In order to define a default class constructor that takes no parameter, and just calls the parent classes constructor, use the DefineDefaultConstructor method of the TypeBuilder class. If you want to provide parameters to your constructor or perform some custom initializations then you have to use the DefineConstructor method of the TypeBuilder class that returns a ConstructorBuilder object.

Once you have a ConstructorBuilder object, and if your constructor has parameters, you have to use the DefineParameter method of the ConstructorBuilder class and define the constructor parameters. Lastly, the most important step in all constructors and method is to get an ILGenerator object (System.Reflection.Emit namespace).

ILGenerator Class (System.Reflection.Emit namespace)

The ILGenerator class is used to produce IL codes for your constructor and method implementation. ILGenerator also has methods to define try-catch-finally blocks. Dealing with the ILGenerator is one of the most tedious tasks of working with Reflection.Emit !!

This is where you get to play with raw MSIL OpCodes.

Listing 8: Defining a static constructor

//Define the static constructor
ConstructorBuilder statConst =
asmType.DefineConstructor(MethodAttributes.Private |
MethodAttributes.Static |
MethodAttributes.SpecialName |
MethodAttributes.RTSpecialName |
MethodAttributes.HideBySig,
CallingConventions.Standard,
null);
//Get the ILGenerator
ILGenerator constGen = statConst.GetILGenerator();

constGen.Emit(OpCodes.Ldstr,"Secured Application - From MasterCSharp");
constGen.Emit(OpCodes.Stsfld,appName);
constGen.Emit(OpCodes.Ldstr,"MasterCSharp");
constGen.Emit(OpCodes.Stsfld,compName);
constGen.Emit(OpCodes.Ldstr,"1.0.0.0");
constGen.Emit(OpCodes.Stsfld,appVer);
constGen.Emit(OpCodes.Ldc_I4_0);
constGen.Emit(OpCodes.Stsfld,isDemo);
constGen.Emit(OpCodes.Ret);

In the above listing you can see that I have used the Emit method of the ILGenerator class to emit the IL OpCodes to initialize the class level fields in the constructor. I first load each literal like the strings, ints, etc, and then call the respective fields (FieldBuilder instances) to store the values. You should refer to the Metadata documentation for full information on the OpCodes and their functions.

MethodBuilder class (System.Reflection.Emit namespace)

The major functional parts of any class are methods. Hence each type might have many methods implemented. You use the DefineMethod method of the TypeBuilder class to define a new method. A MethodBuilder object is created by this method. The MethodAttributes enumeration (System.Reflection namespace) is used to provide the various method attributes like Public, Private, Static, Virtual, Final, etc.

Listing 9: Defining methods

//Define the method
MethodBuilder pAppNameGet =
asmType.DefineMethod("get_ApplicationName",
MethodAttributes.Static |
MethodAttributes.Public | MethodAttributes.SpecialName |
MethodAttributes.HideBySig,
typeof(string),
null);
//Create the method body
pAppNameGet.CreateMethodBody(null,0);

//Get the ILGenerator
ILGenerator pAppNameGen = pAppNameGet.GetILGenerator();
pAppNameGen.DeclareLocal(typeof(string));
pAppNameGen.Emit(OpCodes.Ldsfld,appName);
pAppNameGen.Emit(OpCodes.Stloc_0);

//Create a label
Label nextCode = pAppNameGen.DefineLabel();
pAppNameGen.Emit(OpCodes.Br_S,nextCode);
//Mark the Label
pAppNameGen.MarkLabel(nextCode);

pAppNameGen.Emit(OpCodes.Ldloc_0);
pAppNameGen.Emit(OpCodes.Ret);

In the above listing you see that after defining a new method, I have called the CreateMethodBody method of the MethodBuilder class. Always remember to first call the method and then start using the ILGenerator. If you do not call this method, a compiler error will be thrown stating that no method body was defined!

You use the DeclareLocal method of the MethodBuilder class to declare internal method fields.

One more interesting object is in the System.Reflection.Emit.Label class. While dealing with IL, you sometimes need to branch to other IL lines, in such cases use the Label object and call the appropriate branching OpCode. Then in the IL Line number where you want the control to jump to, call the MarkLabel method of the ILGenerator class.

PropertyBuilder Class (System.Reflection.Emit namespace)

The DefineProperty method of the TypeBuilder object is used to define a new property. I mentioned earlier that as properties are internally implemented as get_PropertyName and set_PropertyName methods, you will have to first define the appropriate methods for the property, and then call the SetGetMethod and SetSetMethod methods of the PropertyBuilder and pass the appropriate MethodBuilder objects.

Listing 10: Defining a read-only property

//Define the property
PropertyBuilder pAppName =
asmType.DefineProperty("ApplicationName",
PropertyAttributes.None,
typeof(string),
null);
//Set the Get Accessor method
pAppName.SetGetMethod(pAppNameGet);

The above listing defines the property and then sets the get accessor for the property. The MethodBuilder that acts as the get accessor has been defined in Listing 9.

EventBuilder class (System.Reflection.Emit namespace)

Just like properties, events too depend on two methods: add_EventName and remove_EventName. So, after getting an EventBuilder object from the DefineEvent method of the TypeBuilder class, use the SetAddOnMethod and SetRemoveOnMethod methods to set the necessary methods that will be called to add and remove your event. You can also set the method that will raise this event by calling SetRaiseMethod. All the above three set methods take MethodBuilder objects as parameters.

Create the Type and Save the Assembly

This is the final step. Call the CreateType method on all the types you have defined so that proper types are generated. If you are dealing with transient assemblies, then the Type object returned by the CreateType method can be used via Reflection to invoke the necessary members. In case of a persistent assembly, after calling the above method, call the Save method on the AssemblyBuilder object to save the assembly to disk.

This completes the process of Reflection.Emit.

Listing 11: Saving the Assembly

asmType.CreateType();
asmBuilder.Save("SecuredInformation.dll");

Conclusion

Reflection.Emit is a very powerful API, and it allows you to harness the full power of the .NET Framework. But this power comes at a price of difficult coding and dealing directly with MSIL code. However, if used in the correct places it can prove to be very useful. Lastly, one note of caution: many of you might have got excited by the features of Reflection.Emit and might be thinking that you might take someone else's assembly (probably the .NET Framework class libraries) and save modified versions of it using Reflection.Emit ! Even though it sounds as an interesting proposition, there is no way you can achieve this since there is no functionality in the Reflection classes that help you get the IL Op Codes of the class implementation. So don't waste your precious time on it . although I found this fact the hard way round!

Article Information
Author | Saurabh Nandu
Technical Editor | Adam Ryland
Author Agent | Charlotte Smith
Project Manager | Helen Cuthill
Reviewers | Andrew Krowczyk, Kaushal Sanghavi

If you have any questions or comments about this article, please contact the technical editor.


Working with Web Data in C#

Working with Web Data in C#
http://www.csharptoday.com/content.asp?id=1808
Article Rating: 5
Published on January 28, 2002
Abstract
Think of the World Wide Web as a giant database management system in which the web sites are the databases and the web pages are the database tables. Now think about how useful it would be to gather data from those tables in a form that could be processed, queried, and combined with other, otherwise unrelated, data in new combinations…using C# and the .NET framework. In this article, Tony Loton looks at how to: drive HTTP GET and POST requests to web sites using the WebRequest and Stream classes, capture the responses via the WebResponse class, and pick out specific nuggets of information from a web page using regular expressions and the Regex class. The end result is a small but useful engine and a demonstration application, that goes a long way towards allowing data to be selected from web pages as though selecting from relational database tables, using SQL syntax.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

Think of the World Wide Web as a giant database management system in which the web sites are the databases and the web pages are the database tables. Now think about how useful it would be to gather data from those tables in a form that could be processed, queried, and combined with other -- otherwise unrelated -- data in new combinations.using C# and the .NET framework.

In this article you will learn how to:

  • Drive HTTP GET and POST requests to web sites using the WebRequest and Stream classes.
  • Capture the response - containing HTML or XML data - via the WebResponse class.
  • Pick out specific nuggets of information from a web page using regular expressions and the Regex class.

The end result will be a small but useful engine and a demonstration application, that goes a long way towards allowing data to be selected from web pages as though selecting from relational database tables, using SQL syntax.

Introduction

The seeds of this idea were sown in my original article "Accessing the World's Biggest Database" published on the JavaWorld web site in March 2001 at the URL

http://www.javaworld.com/javaworld/jw-03-2001/jw-0316-webdb.html?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq. That idea manifested itself in the WebDataKit commercial implementation, an early version of which was written up as a case study in "Professional Java Data" (Wrox Press). If this article whets your appetite for finding out how the idea originated, or how to take it further, those will be useful sources of information.

Having tested out the principles with Java, I decided to turn my attention to the Microsoft platform, first using - believe it or not - plain old (actually plain new) Microsoft Excel and then using C# and .NET. In this article we'll concentrate on the C# / .NET approach, but I'll use Excel as a handy way to introduce the problem we're looking to solve.

System Requirements

To write this article I used the Visual Studio.NET beta 2 release, and for the (optional) Microsoft Excel trials I used Excel 2002 that came bundled with Office XP.

I'll assume a basic knowledge of the Visual Studio.NET environment and the C# language, to a level at which you can compile and run an application if I provide the source (.cs) files.

Defining Problem

As a regular author and reviewer for Wrox, and a regular reader of Wrox books, I'm always interested to find out new titles will be appearing soon, which I can discover by visiting this URL (http://www.wrox.com/Books/Books.asp?section=1_2&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq):

Currently I'm interested to know what .NET books will be appearing in the near future, but I'm too lazy to visit the site and trawl through the list manually. Wouldn't it be great if I had a program, a software agent, which would visit the web page and tell me if any .NET books were listed?

The shortest route to automation of that task is to use the support for Web Queries provided by Microsoft Excel, so we'll look briefly at that first.

Excel Web Queries

If you have a suitable version of Excel (see System Requirements) you can try out a very simple web query by typing in and running this basic macro:

Sub WroxFutureTitles()

Set thisSheet = Workbooks(1).Worksheets(1)
Set theseResults = thisSheet.QueryTables _
.Add(Connection:="URL;http://www.wrox.com/Books/Books.asp?section=1_2", _
Destination:=thisSheet.Cells(1, 1))
With theseResults
.WebSelectionType = xlEntirePage
.WebFormatting = xlNone
.Refresh
End With

End Sub

The end result is a worksheet containing the HTML text elements comprising the page, each element being uniquely addressable via a cell reference. In the following screenshot you can see that the new book titles appear in cells D8:D23. I'm sure that for some of you it would be quite trivial to enhance the original macro to traverse column D looking for cells containing the text ".NET"; thus achieving my original objective.

At this point, start to think about some of the other potential applications. What about a macro that visits a financial web site to pick out the latest share prices for the stocks in your portfolio, with an enhancement to highlight the biggest risers? If you have a potential application in mind, and you're comfortable using VBA macros, feel free to duck out now to experiment. After which, I hope you'll come back!

If you're staying with me, or you've just come back (nice to see you again), we'll now look at how to solve the same problem using C# and .NET. In a nutshell, our C# solution must:

  • Allow text content to be captured from web sites.
  • Make each HTML (or XML) element uniquely addressable.

And for good measure

  • Allow sets of elements to be picked out if they match certain criteria (like containing the text ".NET").

Defining Solution

One approach would be to build on the functionality provided by Excel, perhaps by driving the Excel application itself - via its COM interfaces - from a C# program, or by treating an Excel spreadsheet with an embedded web query as an ODBC data source to be read via the .NET OleDBDataAdapter.

I failed on both counts. In the former case, this was probably because my COM skills are so rusty after spending several years with my head in Java. In the latter case, it was as a result of this dialog message:

Yes I did try to reuse it again later with OleDbDataAdapter, to no avail, and later I noticed the following statement in the .NET Data Providers section of the .Net Framework Developer's Guide :

The OLE DB .NET Data Provider does not work with the OLE DB Provider for ODBC (MSDASQL).

In any event, it turns out that piggy-backing Excel is perhaps not the best approach. It limits the user base for my killer applications (yes, I have plans) to those users with Excel, and it restricts the deployment location to the client tier only.

One thing I'd really like to do is to capture data from various third-party web sites into my own web tier, then combine the data and re-present it in the form of a portal based on ASP pages and / or Web Forms. So, in this article, we'll aim for a self-contained C# / .NET solution that will potentially run on any tier without unnecessary dependencies, as does my original Java implementation.

Component Design

Our requirements will be handled by a single class, called WebDataEngine, which will live within the webdata namespace. Here's how the WebDataEngine.cs file begins, with a set of using statements, the class declaration, and a parameterless constructor :

using System;

using System.Collections;
using System.Net;
using System.IO;
using System.Text;
using System.Text.RegularExpressions;

namespace webdata
{
public class WebDataEngine
{

public WebDataEngine() {}

The class will have just three (static) methods to perform these three functions:

  • getTextFromURL(.) - Submit a HTTP request to a URL and capture the response.
  • getElements(.) - Return a list of uniquely adressable HTML tagged elements.
  • getFilteredElements(.) - Filter the elements using regular expressions.

Typically these three methods will be invoked in sequence, each taking input from the previous one.

Submitting a HTTP request and capturing the response

Most requests for web pages are issued using the HTTP GET method. The GET method allows us to specify the URL of the page and - in the case of a dynamic (JSP, ASP or CGI) page - a set of query parameters appended to the URL.

http://www.somesite.com/somewhere/somepage.html?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq (for a static page)

or

http://www.somesite.com/somesearch?searchword=someword&numresults=10&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq (for a dynamic page)

Some sites require dynamic pages to be requested via the HTTP POST method, which hides the query parameters in the URL and passes them separately as a stream of data. You can determine which method is required, and what parameter values must be passed, by analysing the HTML form tags.

The following HTML form definition (simplified) was extracted from the URL http://www.wrox.com/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq and it tells me that we may submit a HTTP POST request to the URL http://www.wrox.com/Search/Search.asp?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq, passing the string textfield=mysearchterm as the POSTed stream of data.

<form method=" post " action=" /Search/Search.asp " id=form1
            name=form1>
.
<input type="text" name=" textfield " size="15"
            maxlength="50">
<input type="image"
            src="/Includes/images/search.gif" name="submit" id=submit border="0"
            width="120" height="12">
.
</form>

Incidentally, if the method had been "get", we would in that case be able to submit a HTTP GET request via the URL http://www.wrox.com/Search/Search.asp?textfield=mysearchterm&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq.

Our engine will support both submission methods for getting text content from a URL. So, the parameters to the getTextFromURL(.) method will be the URL (which may have query parameters appended), the postText (which is optional and only relevant to POST submissions), and a string indicating which method should be used - "GET" or "POST":

public static String getTextFromURL (String url, String postText
, String method)
{

The first step in this method creates a WebRequest and sets its HTTP submission method:

WebRequest request = WebRequest.Create (url);
request.Method = method;

If the method is "POST" we obtain a Stream to which we can write the postText:

   if (method.Equals("POST")
            && (request is HttpWebRequest))
{
request =
            (HttpWebRequest) request;
request.ContentType =
            "application/x-www-form-urlencoded";
request.ContentLength =
            postText.Length;

Stream outputStream =
            request.GetRequestStream ();
StreamWriter writer=new
            StreamWriter(outputStream);

writer.Write(postText);

writer.Close();
outputStream.Close
            ();
}

Did you notice that we tested the type of the request, which was created by the call to WebRequest.Create(url), before we posted the postText ? All URLs beginning http:// or https:// will result in a HttpWebRequest instance (subclass of WebRequest) that allows data to be posted. However, if we supply a URL beginning file:// relating to a file held on disk, the resulting WebRequest will not be of type HttpWebRequest, and will not support the posting of data.

Whether we post data or not, we must capture the response. For simplicity we're reading the entire response - i.e. the full HTML page - into a text string and returning this to the caller:

WebResponse response = request.GetResponse();
Stream responseStream = response.GetResponseStream();
StreamReader reader = new StreamReader
(responseStream,System.Text.Encoding.Default);

String text=reader.ReadToEnd ();

reader.Close();

return text;
}

Now that we have a way of getting hold of the content of a web page, we need to make each of the elements within the page individually addressable, so that we can pick out just the ones we want.

Making HTML elements uniquely addressable

The getElements(.) method will take in a string containing the entire content of a web page. It will give back a list of elements (smaller text strings), each having a unique key or address of the element within the page. Notice that this method simply delegates to an additional class called TagParser :

public static ArrayList getElements(String taggedText)
{
String[] htmlTags=new String[] {"<html>"
,"<table>"
,"<tr>"
,"<th>"
,"<td>"
};

TagParser tagParser=new TagParser(htmlTags);

return tagParser.getElements(taggedText);
}

We won't be distracted by the code of the TagParser class, because that class simply encapsulates an algorithm that won't teach anything new about C# or .NET. It's not even a well-written algorithm, rather a quick-and-dirty approach to picking out text from within tags so as to move this article along. We'll treat it as a black box that is easily replaceable and for which we know the inputs and outputs.

Inputs and Outputs of the TagParser

Here is a very simple extract of a HTML page:

<html>
<body>
<table>
<tr><td>Wrox</td><td>Press</td></tr>
<tr><td>Tony</td><td>Loton</td></tr>
</table>
</body>
</html>

We would request this via the getTextFromURL(.) method, and supply to the TagParser as a single string:

<html><body><table><tr><td>Wrox</td><td>Press</td></tr><tr><td>Tony</td><td>Loton</td></tr></table></body></html>

As output, we'll get an ArrayList of TaggedElement s, key-content pairs of the form:

.html#0.table#0.tr#0.td#0 = Wrox
.html#0.table#0.tr#0.td#1 = Press
.html#0.table#0.tr#1.td#0 = Tony
.html#0.table#0.tr#1.td#1 = Loton

The text string "Tony" is (reading backwards) contained within the 0th table data tag (.td#0), of the 1st table row (.tr#1), of the 0th table (.table#0) of the HTML page. Those keys are like cell references in Excel, shown earlier, but now not limited to two dimensions.

Notice that the tags used to formulate the element keys are limited to those specified in the htmlTags array within the getElements(.) method listed above. HTML supports many tags - like <b> (bold) - that have no structural significance, and would make the keys unnecessarily long, so we ignore them. In fact, in that example we even ignored the <body> tag by not including it in the htmlTags array.

Finally, you'll be interested to know that the resulting ArrayList contains key-value pairs encapsulated in instances of a TaggedElement class that has the following definition. Notice the use of read-only C# key and content properties here:

public class TaggedElement
{
private
            String elementKey;
private String elementContent;

public
            TaggedElement(String key, String
            content)
{
this.elementKey=key;
this.elementContent=content;
}

public
            string key
{
get { return elementKey; }
}

public
            string content
{
get { return elementContent;
            }
}
}

So that describes the inputs and outputs of the TagParser, which are exactly the same as the inputs and ouputs of the getElements(.) method that wraps it.

Filtering with regular expressions

Each element is individually addressable via its key, and now we just need a mechanism for picking out the single elements, or groups, that we're interested in. We'll use the .NET Regular Expressions feature for this.

A full coverage of regular expressions is beyond the scope of this article, but you will learn a little by example when we look at the sample application later.

The final method of our WebDataEngine class, shown next, takes in a complete list of TaggedElements - called originalElements - obtained from the previous getElements(.) method. It also takes in a regular expression - called filter - as a string that will be used to filter the original elements.

From the filter string we will create at Regex instance, which will be matched against each element's key or content (according to the setting of the contentFilter flag) in turn. The matching elements will be appended to a new ArrayList named filteredElements.

public static ArrayList
            getFilteredElements(ArrayList originalElements, String filter, bool
            contentFilter)
{
ArrayList filteredElements=new
            ArrayList();

Regex filterRegex = new Regex(filter);

foreach (TaggedElement element in originalElements)
{
Match theMatch=null;

if (contentFilter) theMatch =
            filterRegex.Match(element.content);
else theMatch =
            filterRegex.Match(element.key);

if (theMatch.Success)
            filteredElements.Add(element);
}

return
            filteredElements;
}

UI Design

To test-drive the WebDataEngine we'll use a form that allows you to enter:

  • The URL of the source web page
  • Some additional text to post (optional)
  • A (regular expression) filter for element keys
  • A (regular expression) filter for element contents

With a little slight of hand I've set out the fields of this WebDataForm to capture the required values in such a way, and with appropriate labels, to make the jump to something approaching SQL. Remember that one of my ambitions was to use SQL syntax to select data from web pages?

This maps onto the parameters of the three WebDataEngine methods as follows:

SELECT keyFilter AS field
FROM url WITHPOST postText
WHERE field MATCHES contentFilter

UI Code

The code for the WebDataForm is provided for download, but is not essential to the plot, as it simply calls the methods of the WebDataEngine - in turn - in response to the user pressing the Run button, then adds the resuting elements into the list box. Here is the essential code:

String method="GET";
if
            (postBox.Text.Length>0) method="POST";

String
            taggedText=webdata.WebDataEngine.getTextFromURL(urlBox.Text
,postBox.Text,method);

ArrayList
            elements=webdata.WebDataEngine.getElements(taggedText);

elements=webdata.WebDataEngine.getFilteredElements(elements
,keyFilterBox.Text,false);

elements=webdata.WebDataEngine.getFilteredElements(elements
,contentFilterBox.Text,true);

The Sample Application

As a demonstration of how to use this sample front-end to the WebDataEngine, let's go back to the original problem that we solved using Excel. That problem was to pick out the latest .NET books from the Wrox web site:

Using default regular expression filters of \w* (match anything), we get:

Next you will see what happens if we add an appropriate key filter in the SELECT clause:

That keyFilter entry in the SELECT field should have been self-explanatory up to the final few characters, which is where the magic happens. Here are those final characters:

.tr#\d+.td#0$

The text \d+ is regular expression syntax for matching one or more decimal digits, and the text $ is regular expression syntax for saying the the pattern must match right up to the end of the line.

So, now we have a list containing just the book titles from that page, but remember that we want to know specifically about the .NET books. So, we add a (regular expression) content filter in the WHERE.MATCHES field, like this:

Problem solved, but keep in mind the fact that the WebDataEngine is the essential component and this WebDataForm is just an example front-end. Now that we have the ability to capture web data into our C# programs, we need not be limited by simple displaying the results in a list box. Why not load the data into a database for later processing, or re-present the filtered results - along with data from other pages - as a summary web page on your local intranet?

Application Setup

Everything you need is provided in the CSharpWebData project. Just load this project into Visual Studio .NET, build, and run the WebDataForm.

Review and Further Work

Our WebDataEngine has reduced the problem of selecting the latest .NET books from the Wrox site down to the problem of devising an SQL statement like this:

SELECT .html#0.table#1.tr#0.td#1.table#1.tr#1.td#0.table#0.tr#\d+.td#0$ AS field
FROM http://www.wrox.com/Books/Books.asp?section=1_2
WHERE field MATCHES \w*.NET\w*

In solving that problem we've created a handy WebDataEngine that may be used in a great many applications, based on the idea of extracting useful data automatically from the World Wide Web. Think about portals, web agents, share price trackers, and the like.

The ideas I've discussed are not brand new with this article, and my coverage of the C# / .NET implementation reflects the work that I've done previously with Java. The Java work went much further and it provides some interesting ideas for me (or you) to tackle next.

Futher Work

Here are some of the ideas I have for building on the work done here.

HTML and XML Parsing

For convenience, I put together a quick-and-dirty parser, the TagParser class, to strip out the text between HTML tags, and to form unique keys by numbering the tags and each level of the HTML hierarchy. Did I mention that it works with XML too? Take a look at the following screenshot in which I examine the Two Gentlemen of Verona example XML file to pick out all the Acts / Scenes in which the character "SILVIA" speaks:

Looks impressive, but this parser has no comprehension of what constitutes valid HTML or XML content, and it operates on any text file that simply contains <starttag> and </endtag> combinations. I hid the code, and wrapped the functionality up in the getElements(.) method of the WebDataEngine to make it easily replaceable.

Replacing that parsing code seems a a logical next step, and - for the record - in the Java implementation I used the HTML Parser provided in the java.swing package and the SAX XML Parser provided by the Java API for XML Parsing.

I've not yet looked too deeply into the .NET support for XML parsing, but I think the XMLReader class would be a good starting point. I'd expect it to cope with XHTML, but for traditional HTML you might just need to go back to a home-grown parser like the one I've used. I dare say you could build one using regular expressions!

Improved SQL Support

I introduced the idea of using SQL as the syntax for extracting web data by arranging the fields of the WebDataForm as a SQL-like template.

Taking this to its logical conclusion - as I did with Java - we would end up with a full SQL engine that accepts queries as free form text. It would allow data to be selected from multiple pages (think of them as tables) and would join together the results to form new row sets.

To whet your apetite, take a look at the following SQL statement that selects the titles and authors (again, from the URL http://www.wrox.com/Books/Books.asp?section=11_2&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq) of those books that appear in the top ten list at the URL http://www.wrox.com/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq. Note the slight difference in syntax used by the Java implementation:

SELECT allbooks.table[1].tr[0].td[1].table[1].tr[1].td[0].table[0].tr[*]
.td[0].a[0].text[0] AS allbooks.title
, allbooks.table[1].tr[0].td[1].table[1].tr[1].td[0].table[0].tr[*].td[1]
.text[0] AS author
, topten.table[1].tr[0].td[2].table[0].tr[8].td[0].table[0].tr[*].td[1]
.a[0].text[0] AS topten.title

FROM http://www.wrox.com/Books/Books.asp?section=11_2 AS allbooks
, http://www.wrox.com/ AS topten

WHERE allbooks.title=topten.title

To prove that it really is possible to do that, here's a screenshot showing the query running in the Java implementation:

You might be interested to know that "Professional Java Data" (Wrox Press) contains a Case Study chapter describing how something like that can be built using Java. The ideas should be readily convertible into C#.

Conclusion

We started by looking at a problem that could be solved by extracting data from live web pages. That problem was "how to find out which Wrox future titles relate to the .NET platform", and we discussed briefly how that problem could be solved using nothing more sophisticated than Microsoft Excel and a little VBA. A perfectly valid approach if you prefer.

For those of you more interested in C# and the .NET framework (most of you I guess), we stepped through a solution based on the implementation of a useful WebDataEngine class that has just three static methods. In this class we made use of:

  • the WebRequest class for issuing HTTP requests to web sites
  • the WebResponse class for capturing the HTTP response
  • the Regex class for performing pattern matching on character strings using regular expressions

Because these are .NET classes, not special to C#, there's no reason why you couldn't apply exactly the same techniques using Visual Basic or C++.

A sample front-end was demonstrated in the form of the WebDataForm, which we treated as a template for running basic SQL queries against web sources.

Finally I told you about some of the enhancements that I plan for this C# code, to bring it in line with the Java implementation. Maybe you'll beat me to it!

Article Information
Author | Tony Loton
Technical Editor | Adam Ryland
Author Agent | Charlotte Smith
Project Manager | Helen Cuthill
Reviewers | Iikka Meriläinen

If you have any questions or comments about this article, please contact the technical editor.


Functionality and design of a thread pool class

Functionality and design of a thread pool class
http://www.csharptoday.com/content.asp?id=1806
Article Rating: 3.5
Published on January 24, 2002
Abstract
Multithreading is a very complex topic. Programmers have to deal with difficulties like resource sharing and thread synchronization, thread lifetime management, and starvation. Fortunately, the new Microsoft .NET technology provides a complete OO solution to handle these problems. Using the System.Threading namespace, you have classes for thread synchronization, callback scheduling with timers, thread management, and thread pooling. This is the first part of a two part article by Vincenzo Tenisci that looks at thread pooling and thread management. In this first part, he looks at thread pool design and implementation, gives an overview of the .NET Threadpool class, and designs and implements a custom thread pool.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

We all know that multithreading is a very complex topic. Some of the main difficulties that a programmer must handle are:

  • Resource sharing and Thread synchronization.
  • Thread lifetime management.
  • Shared resource initialization problems (also know as starvation).

Fortunately the new Microsoft .NET technology with the CLR provides a complete OO solution to handle these problems. Using the System.Threading namespace, you have classes for thread synchronization (like classic Event and Mutex, or more complex objects like Monitor and ReaderWriterLock), callback scheduling with timers, thread management, and thread pooling.

This is the first part of a two part article that looks at thread pooling and thread management. In this first part we will focus on:

  • Showing various aspects of thread pool design and implementation, with an overview to the .NET ThreadPool class in the System.Threading namespace.
  • Designing and implementing a custom thread pool to take under control the thread life management.
  • Exposing unmanaged thread pool classes to the CLR (like the new C++ ATL Server ThreadPool class) so that you can implement custom thread pool classes based on existing standard code.
  • Designing and implementing a custom thread pool class that is conformed to the .NET BeginInvoke/EndInvoke idiom for asynchronous method calls via delegate types.

This article assumes that you have experience in multithread programming and concepts. You can read Matthew Reynold's C#Today article on Thread Pooling at http://www.csharptoday.com/content.asp?id=1627&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq

The source code is compiled with MS Visual Studio .NET Beta 2.

What is a Thread Pool?

Thread pooling is a well-known technique applied in server applications to limit the number of threads used to handle client requests. The main advantages of thread pooling are:

  • Threads inside the pool are generally pre-created at the server start-up, avoiding thread creation and destruction at client request that can cause reduced server performance under heavy loads.
  • Avoiding thread context switching and limits the memory usage.

Generally, a thread pool object is implemented using high performance queues where client requests are enqueued and a dispatcher that takes the requests from the queue and dispatches them to the first available thread of the pool. A thread in a pool is called a worker thread, and the request/response that is enqueued/produced by the pool is generally called the worker item. Thread pool objects can be fixed-size or dynamic-size. In the first case, the size of the pool doesn't change during its lifetime. A fixed number of threads are created at the time of pool initialisation. This can be useful, for example, if you have to control a limited number of hardware resources like serial ports. In this case you perform an "a priori" synchronization, setting the pool size to the maximum number of physical resources that you can handle, so that the thread pool can assign the first free resource to the client request. The dynamic-size type is more sophisticated. It is used when you don't know how many client requests you have to handle (TCP/IP or WEB servers for example). Starting with a minimum number of threads (generally equal to N. processors * 2), the pool can increase or decrease its size in response to the server load, according to a sophisticate algorithm.

The diagram below shows a generic thread pool design schema.

The System.ThreadPool class.

In the .NET System.Threading library there is a ThreadPool class. It is a dynamic-size thread pool. It provides a pool of worker threads where you can enqueue work items that:

  • Can be related to a wait operation. You can create threads in a sleeping state, waiting for an event to occur.
  • Can be related to a timer-queue. You can "enqueue" timers that periodically call a callback function (delegate) that is executed inside a thread in the thread pool.
  • Can be directly handled by a thread in the thread pool.
  • Can be used with all Microsoft 32-bit operating systems.
  • Can create and destroy threads automatically, depending by the number of requests.

Let's see a simple C# example:

class TestPoolApp
{
static void
            Main()
{
string str = "Hello from
            ThreadPool";
ThreadPool.QueueUserWorkItem(new
WaitCallback(TestCallback),str);
Thread.Sleep(2000);
}

static
            void TestCallback(object
            state)
{
Console.WriteLine(state);
}

}

As we can see, the ThreadPool class is very simple. There is a static method (QueueUserWorkItem) that accepts a delegate of type WaitCallback. This delegate type will be invoked inside a worker thread managed by the system. The first time that QueueUserWorkItem is invoked, a thread pool is created by the system. This means that:

  • There is only one thread pool object per process.
  • You cannot control exactly how many threads the pool can handle.
  • The system manages thread lifetime inside the pool. Once QueueUserWorkItem is called, the work item cannot be cancelled.

These features can be useful in certain applications, but frankly, when I use thread pool objects, I like to have more control over thread management inside the pool. Due to this fact, I decided to write a custom thread pool object with these requirements:

  • It is a fixed-size thread pool.
  • You can create more than one thread pool object (with it's own number of threads) inside your application.
  • It exposes the CLR to the new ATL Server ThreadPool class, so that you also have standard source code for a ThreadPool class.
  • You can take this class as a starting point to write more sophisticate custom thread pool classes

Before we go further, we have to talk about how a thread pool is internally implemented: enter the IoCompletionPort API.

Thread pooling with IoCompletionPort.

Writing a thread pool for Windows NT/2000 or XP is a very simple task using the IoCompletionPort API (http://msdn.microsoft.com/library/default.asp?url=/library/en-us/fileio/filesio_4z1v.asp&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq). This is a low-level API exposed by the operating system. This means that we are outside the .NET managed world so, if you want to use them, you have to write C++ code. Internally they are implemented using the request queue and dispatcher objects illustrated in figure 1.

Here are the general steps that you can follow to write thread pools using this API:

  • First, you have to call CreateIoCompletionPort() to get a handle on the port and to specify the maximum number of threads related to the port queue.

Here is an example:

hPort = CreateIoCompletionPort(INVALID_HANDLE_VALUE, NULL, 0, nNumThreads);

  • Second, you have to create a number of threads equal to nNumThreads. Each thread will call the GetQueuedCompletionStatus() API, waiting for a request from the IoCompletionPort queue. You can also create a manual reset event synchronization object to synchronize thread creation and shutdown.

Here is an example of a thread function for a thread pool:

   #define THREAD_EXIT
            ((OVERLAPPED*) ((__int64) -1))

DWORD
            ThreadProc()
{
SetEvent(m_hThreadEvent);

while(GetQueuedCompletionStatus(hPort,
            &dwBytesTransfered,
&dwRequestData, &pOverlapped,
            INFINITE))
{
if (pOverlapped == THREAD_EXIT) // Shut
            down
{
break;
}

//TODO: get data from
            dwRequestData

//TODO: write your own thread logic handling
            here!

}

SetEvent(m_hThreadEvent);
}

  • Third, you have to call PostQueuedCompletionStatus() when you want to post a request to the completion port queue. Here is an example:

PostQueuedCompletionStatus(hPort, 0, (DWORD) requestData, NULL)

Where requestData is a struct containing information related to the client request.

Note the THREAD_EXIT definition. It is used to enqueue the thread pool shutdown request. Here is an example:

void ShutdownThreadPool()
{
for (int i
            = 0; i < nNumThreads; i++)
{
ResetEvent(m_hThreadEvent);

PostQueuedCompletionStatus(hPort,
            0, 0, THREAD_EXIT );
DWORD dwRet =
            WaitForSingleObject(m_hThreadEvent, dwMaxWait);

//TODO Handle
            timeout for WaitForSingleObject
}
}

For each thread inside the pool we call the PostQueuedCompletionStatus() API, passing the THREAD_EXIT value to the pOverlapped parameter so that each worker thread inside the pool can exit from the while loop (see the ThreadProc code above). As you can see, each thread in the pool calls SetEvent(m_hThreadEvent) at the beginning and at the end. This means that each time a thread exits, the m_hThreadEvent event is signalled. So if you want to perform a shutdown operation on the thread pool, first you must call ResetEvent(), then call PostQueuedCompletionStatus(), passing THREAD_EXIT in the pOverlapped argument, and calling WaitForSingleObject(), waiting for the m_hThreadEvent to be signalled. This operation is performed for each thread inside the pool. So, in this scheme, if you specify dwMaxWait milliseconds for each thread shutdown operation, you will wait for a maximum of dwMaxWait*nNumThreads milliseconds to complete the thread pool shutdown operation.

The ATL Server Thread Pool class.

The ATL server is the new C++ class library provided by Visual Studio .NET. It contains a lot of useful classes to write high performance server applications. You have classes for performance counters, ISAPI filters, tracing and logging and, yes, thread pooling.

The declaration of the thread pool class is in atlutil.h :

template <
class Worker ,
class ThreadTraits = DefaultThreadTraits
>
class CThreadPool :
public IThreadPoolConfig

The CThreadPool class uses the IoCompletionPort API to perform thread pooling in the same manner as explained before. It has all the features that a complete thread pool class must have: dynamic resizing of thread pool, binding to external handles that represent an object that can handle asynchronous notifications using IoCompletionPort. For example, a WinSock2 socket handle, an MSMQ queue handle, or a file handle. The principal methods are:

HRESULT Initialize(
void * pvWorkerParam = NULL,
int nNumThreads = 0,
DWORD dwStackSize = 0,
HANDLE hCompletion = INVALID_HANDLE_VALUE )

With this method, you can define the number of threads, pass some initialisation data, and bind the thread pool to an external IoCompletionPort handle.

HRESULT SetSize(int nNumThreads)

This method is used to dynamically increase or decrease the size of the pool.

BOOL QueueRequest(Worker::RequestType request)

This method is used to enqueue request data to the completion port queue.

void Shutdown (DWORD dwMaxWait=0)

This method handles the thread pool shutdown.

As you can see, CthreadPool is a template class. To provide the code to process work items in a thread pool, you must write the Worker class conforming to the worker archetype (in generic programming, the word archetype is often used to indicate that a class must implement the interface). Here is an example of a Worker archetype:

class PoolWorker
{
public:
typedef DWORD_PTR RequestType;

virtual BOOL Initialize(void *pvParam);

virtual void Terminate(void* /*pvParam*/);

void Execute(RequestType dw, void *pvParam, OVERLAPPED* pOverlapped);

virtual BOOL GetWorkerData(DWORD /*dwParam*/, void ** /*ppvData*/);
};

If you go back to the code of ThreadProc, in the IoCompletionPort API section, you can imagine how the Worker archetype is utilized by the thread pool:

The Initialize() method is called before entering the while loop, and it receives the data (pvParam) passed in from the pvWorkerParam in the Initialize method of the CThreadPool class. This is useful if you need some initialisation data that can be used by the worker class.

The Execute() method is called inside the while loop (when the worker thread receives a request via the GetQueuedCompletionStatus(..) API). This is the most important method, because it is here where you have to implement the client request handling.

The Terminate (..) method is called before exiting the ThreadProc code. It is used to handle resource shutdown and cleanup.

As you can see, the new ATL Server ThreadPool class hides from you all the complexities associated with thread pool design and implementations (to use it you have only to implement the worker archetype). You have total control of thread creation and destruction and, most important, you have the source code.

Thread Pooling: Exposing unmanaged code to the CLR.

Now we enter into the most interesting part of our discussion. We know how a thread pool is implemented in the CLR and with the ATL server library. The ATL server ThreadPool class is a pure C++ class. In the .NET world, we say that it is an unmanaged object. So why don't we try to expose the unmanaged ATL class to the CLR so that we can use it directly in other languages like C# or VB? We will now look at an explanation, which can also be found in the source code.

I have created a managed C++ class library project called SimplePool. This namespace exposes the ThreadPool class and a delegate of type:

public __delegate void ThreadPoolProc(String __gc * pObj);

This delegate works like the WaitCallback in the System.ThreadPool class (with the exception that it accepts a String type parameter). Before digging into the code let's see a C# example:

The file TestWorker.cs contains the definition of a class whose only purpose is to implement a static function that represents the single cast delegate method used to initialize the ThreadPool object:

using System;
using
            System.Threading;

namespace Tester
{
///
            <summary>
/// Test class for thread pool.
///
            </summary>
public class TestWorker
{
public
            TestWorker()
{}

//Test delegate method for SimplePool
            object
public static void ExecuteForSimplePool(String str)
{
lock(typeof(TestWorker))
{
Console.WriteLine("Hello
            from {0}. Thread
            ID:
{1}",str,Thread.CurrentThread.GetHashCode());
//Get
            ThreadPool call context test
            data
Console.WriteLine(ThreadPoolCallContext.GetData("ctxData"));
}

Thread.Sleep(0);
}
}
}

The file app.cs contains the definition of the classes that represent the starting point of a Win32 console application. They show you how to use the new ThreadPool objects. Here is an extract of the file, related to the SimplePool test.

using System;
using
            SimplePool;

namespace Tester
{
///
            <summary>
/// Application Tester class for simple thread
            pool
/// </summary>
public class
            AppSimplePoolTester
{
static void Main(string[]
            args)
{

SimplePool.ThreadPool pool= new
            SimplePool.ThreadPool ();

try
{
//Initialize thread
            pool object with five threads
pool.Init(5,new
            SimplePool.ThreadPoolProc(TestWorker.
ExecuteForSimplePool));

//Enqueue
            twenty client requests to thread pool
for(int i = 0; i < 20;
            i++)
{
String request = "Request N. " +
            i;
pool.PostRequest(request);
}
}
catch(Exception e)
{
Console.WriteLine(e.Message);
}
finally
{
//Thread pool
            shutdown
pool.Shutdown(5000);
}
}
}
}

Here is a description of the main parts of the sample code:

  • using SimplePool: imports the SimplePool library. This is the C++ class library which is implemented the managed thread pool object.
  • pool.Init(): creates a thread pool of five threads and the ThreadPoolProc delegate from the TestWorker.ExecuteForSimplePool static method.
  • pool.PostRequest(): enqueues twenty client requests, passing string data to a thread in the thread pool.
  • pool.Shutdown(5000) : thread pool shutdown with a timeout of five seconds.

The figure below shows the result:

You will notice that twenty client requests generates only five distinct thread ID's. This is because we have created a thread pool of five threads. In this case we have a fixed-size thread pool.

Now it's time to explain how the mixing of managed and unmanaged code is performed. I have created C++ library project cal led ThreadLib. First of all we have to notice that in the stdafx.h file we have this line of code:

#using <mscorlib.dll>

This is necessary to expose to C++ language the .NET class library.

Inside the threadlib.h header file are defined the managed ThreadPool class and the unmanaged Worker archetype class for the ATL Server thread pool. The main problem is to understand how to pass managed types (the String param and the delegate address) to the unmanaged part, and how to retrieve managed types from unmanaged data types.

To use the ATL ThreadPool class in our project we have to add these lines:

#include <atlbase.h>
#include <atlutil.h>

Here is the definition of the delegate type:

public __delegate void ThreadPoolProc(System::String __gc * pStr);

Now let's see the declaration of the managed ThreadPool object:

public __gc class ThreadPool
            {
public:
ThreadPool();
~ThreadPool();

void Init(int
            nThreads,ThreadPoolProc* pProc);
void PostRequest(System::String
            __gc * pStr);

void Shutdown(int
            timeout);
private:
CThreadPool<PoolWorker> __nogc *
            m_pool;
};

The ThreadPool class is declared with the __gc keyword to indicate that this object is garbage collected. Inside this class you must have a private member that points to the unmanaged ATL ThreadPool object, so it is declared with the __nogc keyword. This is the technique used to declare unmanaged types inside a managed class. Due to the presence of an unmanaged pointer type, you must write the following class constructor and destructor to handle the object life correctly:

ThreadPool::ThreadPool(void)
{
m_pool
            = new
            CThreadPool<PoolWorker>;
}

ThreadPool::~ThreadPool(void)
{
if(m_pool)
delete
            m_pool;

m_pool = 0;
}

Now lets examine the code of the ThreadPool Init() method:

void ThreadPool::Init(int
            nThreads,ThreadPoolProc* pProc)
{
ThreadPoolProc __pin * pp =
            pProc;

m_pool->Initialize(reinterpret_cast<void*>(pp),nThreads);
}

This method is used to set the pool size (nThreads param) and the delegate type. As you can see, to pass the managed delegate pointer to the unmanaged ATL ThreadPool, you have to convert the delegate pointer to a pinning pointer (using __pin *) preventing the managed objects from being moved by the CLR during garbage collection. After this, you can convert the pinning pointer to the void pointer required by the ATL ThreadPool's Initialize method.

Here is the PostRequest() method:

void ThreadPool::PostRequest(System::String
            __gc * pStr)
{
IntPtr intptrDescr =
System::Runtime::InteropServices::Marshal::StringToHGlobalAuto(pStr);

m_pool->QueueRequest(
(PoolWorker::RequestType)intptrDescr.ToPointer()
);
}

This method is used to enqueue client requests to the thread pool. The String type handling from managed to unmanaged code is performed using the static method StringToHGlobalAuto() of the System::Runtime::InteropService::Marshal class. This method copies the managed string into the heap and returns an IntPtr() object. Calling the ToPointer() member function we get a void* pointer to the unmanaged memory buffer. After this we can enqueue the request to the thread pool with the ATL Server class.

Now let's examine the unmanaged Worker Execute () method:

inline void PoolWorker::Execute(RequestType
            dw, void *pvParam, OVERLAPPED* pOverlapped) throw()
{
try
            {
void __pin * pp(m_tpProc);
ThreadPoolProc* pProc =
            reinterpret_cast<ThreadPoolProc*>(pp);

//Call the
            single cast
            delegate.
pProc->Invoke(reinterpret_cast<wchar_t*>(pData));
}
            catch(...) {
//TODO: Error handlig
}

System::IntPtr
            intptrDescr(pData);
System::Runtime::InteropServices::Marshal::FreeHGlobal(intptrDescr);
}

This method performs the unmarshalled data operations (from unmanaged to managed types). First you have to convert the unmanaged void* pointer to the delegate function into a void* pinned pointer. After this you can perform the casting to the ThreadPoolProc* managed delegate type. Second, call the single cast delegate so the function is executed inside the first available thread of the pool. After invoking the single cast delegate, you must destroy the unmanaged string buffer using the FreeHGlobal function. But wait a moment! In the Invoke method, a cast to the unmanaged wchar_t* type is performed. The delegate type accepts a managed String* type. Is this an error? The answer is no, because the managed C++ String class provides these two overloaded constructors:

String(char __nogc*)

and

String(wchar_t __nogc*)

So you can initialize a String object with an array of unmanaged characters.

A Custom ThreadPool for .NET asynchronous call.

So far, so good, now we have a custom thread pool class that exposes the unmanaged ThreadPool ATL Server class, but there are some limitations:

  • First, our class can handle only string data types.
  • Second, the delegate type is accepted in the Init method so our thread pool can handle requests for only one delegate instance.

Before to go further, we have to do a brief introduction of a new feature exposed by the .NET runtime. Enter asynchronous method calls.

With delegate types, we can perform asynchronous method invocation:

  • The BeginInvoke method enqueues requests to a thread of the CLR ThreadPool class. It returns an IAsyncResult object that represents call state. It can also accept an AsyncCallback object that is invoked at the end of the method execution.
  • Alternatively , the EndInvoke method can be used instead of the AsyncCallback object. It blocks the execution flow until the end of delegate method.
  • You can also use the IsCompleted method of the IAsyncResult object returned by the BeginInvoke method to poll the thread execution status.

To handle the BeginInvoke/EndInvoke idiom with a custom thread pool, we have to write a more sophisticated class with these requirements:

  1. The method PostRequest () must accept the delegate type and an Object type variable like CLR ThreadPool.QueueUserWorkItem () method.
  2. IAsyncResult/AsyncCallback handling.

The source code is in the C++ ThreadLib project. It contains the new ThreadPool class. Here is a description of the various classes exposed by the ThreadLib namespace:

  • delegate void ThreadPoolProc(Object __gc * pObj): delegate type for ThreadPool. Now it accepts a generic object type.
  • ThreadPool : the new thread pool class. It has three methods
  • void Init(int nThreads);
  • void PostRequest(ThreadPoolProc* pProc,Object __gc * pObj);
  • void Shutdown(int timeout);
  • TPAsyncResult : implementation of IAsyncResult interface.
  • CallCtxData and AsyncCallCtxData : context classes that represent the information related to a call request to ThreadPool object. CallCtxData is used internally when PostRequest method is called. AsyncCallCtxData must be used when we implement the BeginInvoke/EndInvoke idiom (see below).
  • ThreadPoolSlots : encapsulates a synchronized HashTable collection. It is used internally by the ThreadPool object, to store context classes associated with a particular thread. Each instance of ThreadPool class has its own ThreadPoolSlots object. The PostRequest method, when invoked, creates a new slot associated with a String GUID. This GUID is transmitted to a thread of the pool that uses it to retrieve the correct context data.
  • ThreadPoolCallContext : similar to ThreadPoolSlots, is used to store and retrieve call context data from one thread to another.

Here is a logical sequence diagram for thread pool operations when a client posts a request:

Now that we have this new ThreadPool class, we can write the following base class (implemented in the AsyncLib.dll C# class library) that implements the .NET asynchronous method call style

namespace AsyncLib
{
//Interface for
            synchronous method call
public interface IThreadPoolDelegate
{
void HandleRequest(object request);
}

//Interface
            for asynchronous method call
public interface
            IAsyncThreadPoolDelegate
{
IAsyncResult BeginRequest(object
            request,
AsyncCallback cb, object state);
void
            EndRequest(IAsyncResult callResult);
}

//Base class for
            ThreadPool delegate methods
public abstract class
            ThreadPoolDelegate :
IThreadPoolDelegate ,
            IAsyncThreadPoolDelegate, IDisposable
{
private bool disposed
            = false;

public ThreadPoolDelegate(int poolSize)
{
proc
            = new ThreadPoolProc(this.HandleRequest);
mPool = new
            ThreadPool();

mPool.Init(poolSize);
}

~ThreadPoolDelegate()
{
//60 seconds for
            timeout
mPool.Shutdown(60000);
}

//to be implemented by
            derived class
public abstract void HandleRequest(object
            request);

// ThreadPoolProc delegate declaration
private
            ThreadPoolProc proc;

// BeginRequest method: delegate for
            asynchronous calls
public virtual IAsyncResult
            BeginRequest(
object request, AsyncCallback cb, object
            state)
{
//Initialize AsyncCallCtxData
AsyncCallCtxData
            data = new AsyncCallCtxData();
data.m_del = proc;
data.param =
            request;
data.wclbk = cb;
data.iar.pState =
            state;

//Post async request to ThreadPool
            object
mPool.PostRequest(proc,data);

return
            data.iar;
}

//End Request Method
public virtual void
            EndRequest(IAsyncResult call)
{
//Wait the end of thread
            execution
call.AsyncWaitHandle.WaitOne();

//Free the
            associated data slot
TPAsyncResult data =
            (TPAsyncResult)call;
data.FreeSlot();
}

//IDisposable
            implementation
public void
            Dispose()
{
if(!this.disposed)
{
//remove the object
            from the finalization queue
GC.SuppressFinalize(this);

//60 seconds for
            timeout
mPool.Shutdown(60000);
this.disposed =
            true;
}
}

//Custom thread pool
protected ThreadPool
            mPool;
}

}

As we can see, we have defined a base class for a delegate type that is able to post asynchronous requests to our custom thread pool. The only thing that we have to do is to derive our classes from it and to implement the abstract HandleRequest method. Here is an example:

//Tester delegate for async calls
class AsyncTester: AsyncLib.ThreadPoolDelegate
{
public AsyncTester(): base(3) {}

public override void HandleRequest(object request)
{
lock(typeof(TestWorker))
{
Console.WriteLine("Hello from HandleRequest {0}. Thread
ID: {1}",request,Thread.CurrentThread.GetHashCode());
}

Thread.Sleep(0);
}
}

In this case we set the pool size to 3. Here is an example of utilization of the new delegate type:

//This statement will call Dispose method
            automatically
using(AsyncTester tt = new AsyncTester())
{
try
{
AsyncLib.IAsyncThreadPoolDelegate ih =
(AsyncLib.IAsyncThreadPoolDelegate)tt;

IAsyncResult ir1 =
            ih.BeginRequest("test method1",null, null);
IAsyncResult ir2 =
            ih.BeginRequest("test method2",null, null);
IAsyncResult ir3 =
            ih.BeginRequest("test method3",null, null);
IAsyncResult ir4 =
            ih.BeginRequest("test method4",null, null);
IAsyncResult ir5 =
            ih.BeginRequest("test method5",null, null);
IAsyncResult ir6 =
            ih.BeginRequest("test method6",null,
            null);

ih.EndRequest(ir1);
ih.EndRequest(ir2);
ih.EndRequest(ir3);
ih.EndRequest(ir4);
ih.EndRequest(ir5);
ih.EndRequest(ir6);

}
catch(Exception
            e)
{
Console.WriteLine(e.Message);
}

}

If we try to execute this code, we will see that the thread pool object, using only three threads, handles the six asynchronous requests. You will find this example in the source code.

Summary

With a robust thread pool class, you are able to write very powerful server applications. In this article we have seen what .NET can do for us, and how we can use C++ and the ATL server library to have more control over thread management inside a thread pool. We have also seen how we can write and use a custom thread pool to conform to the new .NET asynchronous method calls model. You can use the ThreadPool class as a starting point to develop all the features exposed by the ATL ThreadPool class and to write multithreaded .NET applications and services. In the next article, we will see how to use this custom thread pool to encapsulate thread handling using declarative programming with the new .NET interception framework.

Article Information
Author | Vincenzo Tenisci
Technical Editor | Adam Ryland
Author Agent | Charlotte Smith
Project Manager | Helen Cuthill
Reviewers | Andrew Krowczyk, Kaushal Sanghavi

If you have any questions or comments about this article, please contact the technical editor.


Remoting in C# - Part 1

Remoting in C# - Part 1
http://www.csharptoday.com/content.asp?id=1798
Article Rating: 4.5
Published on January 15, 2002
Abstract
In large modern applications, business logic and database access is often distributed into multiple tiers of components on separate machines, to take advantage of distributed processing, load balancing, and scalability performance gains. This requires some form of distributed communication technology. Previous technologies that accomplished this task have been Remote Procedures Calls, CORBA, and Distributed COM. The .NET Framework contains a very robust model for distributing objects across process and machine boundaries, called Remoting, and in this article, Zach Greenvoss explores the fundamental concepts used to create distributed objects, host them, and utilize them in a client application. In the next articles we will cover more advanced remoting topics like remoting events, custom channel sinks, and advanced lease management for client–activated objects.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

In modern applications, business logic and database access is often distributed into multiple tiers of components. In small applications these tiers can exist on a single machine, however as applications scale up it becomes increasingly necessary to partition each tier on separate machines to take advantage of distributed processing, load balancing, and scalability performance gains. This requires some form of distributed communication technology. Previous technologies that accomplished this task have been Remote Procedures Calls, CORBA, and Distributed COM. The .NET Framework contains a very robust model for distributing objects across process and machine boundaries, called Remoting, and we will explore the fundamental concepts used to create distributed objects, host them, and utilize them in a client application.

Remoting and How it Differs to Web Services

.NET remoting is a huge topic, and we are only going to cover the basics of how to create and call a distributed object in this article. In a future article we will discuss more advanced topics such as advanced lease management, remoting events, and custom channel sinks. Remoting is used whenever code in different Application Domains must communicate with each other. This can be two separate applications running on the same machine, or two objects communicating between physical tiers in a distributed system. As you will see, the entire remoting system can be configured through XML files, thus enabling easy maintenance and quick changes to the deployment model.

A lot of discussion has centered around Web Services and how they facilitate communication over standard Internet protocols like HTTP and SOAP. Remoting is not a replacement for Web Services: they each have their place in a well designed .NET solution; understanding when to use each will be important in designing .NET applications. Web Services utilize HTTP and SOAP to expose functionality available to any client connected to the Internet. They use a single-call methodology that inherently retains no state. In contrast, remoting is a very rich protocol that can expose objects over HTTP or TCP/IP using either SOAP or Binary encoding for the messages sent between client and server. Remoting is also very extensible, and can be customized to suit your specific application's needs.

Remoting is a great technology to use within the firewall and between tiers in a .NET application (note remoting works in the .NET world only). When interacting with external systems, leveraging the standards that Web Services provide is crucial. However, when using internal components where speed and performance are paramount, remoting with Binary encoding over TCP/IP is probably the best choice. We also have the option of using remoting over HTTP with SOAP encoding, thereby creating an interoperable Web Services-like solution. Again, remember that Remoting can expose components in a rich object-oriented fashion, where Web Services are strictly a stateless single-call technology. There are many options available, and understanding each .NET technology is crucial to choosing the best solution for your project.

Remoting Architecture

The diagram below illustrates the remoting architecture. .NET applications are partitioned into Application Domains, which roughly translate to process boundaries. Remoting is used to communicate across the Application Domain boundaries - whether they are local to the machine or across the network. The server Application Domain hosts the remote object and listens for requests from clients. The client utilizes a proxy object that looks and acts like the actual object, but all calls on this object are forwarded to the server.

Channels are used to connect Application Domains, and most of the work required for remoting is performed within the channel. The .NET Framework ships with two Channel objects: the HTTPChannel and the TCPChannel. Each communicates over the network using a different protocol, depending on the channel type. When a client makes a call on a remote object, the proxy object intercepts the method call and forwards it to the channel. A formatter is used to serialize the message into an appropriate encoding. This could be SOAP or Binary encoding, or it could be a custom designed formatting scheme. An optional number of channel sink objects can also be attached to a channel. Channel sinks are objects that process remoting messages as they pass through a channel. These are a point of developer customization, and logging or filtering channel sinks can be created. The channel then sends the encoded message physically over the network. Depending on the type of channel, this could utilize the HTTP or TCP/IP protocol. Once the message has arrived at the server it passes through a set of server channel sink objects and finally through another formatter to deserialize the message. Then the remoting system interprets the message and calls the correct object method. The return values from this are then sent back along the same channel in the reverse direction.

High-Level Remoting Architecture

Remoting enables you to build an object model that can be distributed across tiers inside or outside your organization. Because of the flexibility it offers, you can have objects exposed internally via TCP/IP channels, and use external objects across the Internet through HTTP channels. Remoting allows you to create a loosely coupled architecture that retains the business logic on the server while still providing client applications with rich functionality and object-oriented development paradigms.

System Requirements

  • .NET Framework Beta 2
  • Visual Studio.NET Beta 2
  • Windows 2000 or Windows XP

Creating a Remotable Object

Building an object that can be called from remote clients is relatively simple. By default, the .NET Framework serializes objects by value. This means that when an object is invoked, it is copied to the local Application Domain and used within the same process. Therefore, instead of running on the server or back-end process, the object is now executing within the client's Application Domain. To enable remoting and to create a distributed object this behavior must be overridden, and to do this an object must inherit from System.MarshalByRefObject. This places the remoting plumbing in place that enables the object to be invoked by a remote proxy and communicate over a remoting channel.

Let's actually create an object that we can call remotely. Our object will contain a method that returns the current Application Domain's name. This will enable us to easily tell if the application is being hosted in a remote server or if it is being used locally. Create a new C# Class Library project in Visual Studio .NET and call it RemoteObjectLib (all projects can be found in the code download for this article).

Add the following code to the class. This is the complete listing of the code required for our remote object.

using System;

namespace
            RemoteObjectLib
{
public class RemoteObject :
            MarshalByRefObject
{
public string GetData()
{
return
            AppDomain.CurrentDomain.FriendlyName;
}
}
}

Note the object inherits from MarshalByRefObject. This is all that is necessary to mark a class as remotable in the .NET Framework. The only method in our object simply returns the name of the current Application Domain it is running within. When we build our client application, we will see that the Application Domain this remote object is running in is different to the client's Application Domain.

Hosting a Remote Object

The remote object must execute inside some form of hosting application. This application runs on the remote server and processes requests from clients for hosted objects. As we will see, the remoting subsystem takes care of almost all these details for the developer.

A developer has many options when building remoting solutions. In addition to the three different types of hosting applications we will discuss next, the type of object activation and remoting protocols used can be customized. This flexibility makes .NET remoting a very powerful technology, but it also makes it very confusing to learn initially. We will walk through the hosting options available, as well as the activation type and protocol options. We will cover each of these in depth, and hopefully clarify the options available with remoting.

1. Custom Application

There are three options to choose from when hosting a remote object. One of these is to build a custom .NET application that will run on the remote machine acting as an object server. This application can be a very simple console based application that must be started manually or it could be a more sophisticated Windows Service that will start automatically and host the remote objects. The .NET Framework enables the creation of Window Service applications very easily, and this task is not as daunting as it might have been in the pre-.NET days. In an enterprise solution this would be the preferred choice. However, we are going to build a simple console server for brevity. I leave it as an exercise for the reader to incorporate the lessons learned in this article into a Windows Service remoting host.

Create a new C# Console client called RemoteServer.

We have to reference the following namespace in the Class1.cs file:

using System.Runtime.Remoting;

This is the namespace for the Remoting API, and it contains a large number of objects and additional namespaces. We need to reference the namespace to use the RemotingConfiguration object later.

Add the following code to the Main method so the complete code looks like this:

using System;
using
            System.Runtime.Remoting;

namespace RemoteServer
{
class
            RemoteServer
{
[STAThread]
static void Main(string[]
            args)
{
Console.WriteLine("Registering
            objects.");

try
{
RemotingConfiguration.Configure("Config.xml");
}
catch(Exception
            ex)
{
Console.WriteLine("Error registering
            Remoting.");
Console.WriteLine(ex.ToString());
Console.ReadLine();
return;
}

Console.WriteLine("Objects
            registered, server
            started...");
Console.ReadLine();

}
}
}

As you can see there really is not much code required for this application. The only line that really does anything interesting is the call to RemotingConfiguration.Configure(). This uses a configuration file to configure the settings for the remoting system. We are going to cover remoting configuration files in detail later in this article, for now just know they provide a way to easily configure the remoting system and register the hosting application to listen for client requests on exposed objects. Of course, we cannot run the application successfully until we have created the configuration file, but since we don't have a client yet anyway this isn't too much of a problem.

All this application does is register the remoting objects in the configuration file, and then wait for requests from clients. Note that there is no code for processing client requests - this is all handled by the .NET Framework and the remoting system. This application simply waits for a user to press enter by using the Console.ReadLine() method.

2. Internet Information Services

The other option available to host remoting objects is IIS. This Web server can manage HTTP requests to remote objects, saving us the problem of creating a hosting application ourselves. In addition, all of the security options and benefits bundled with IIS are available to use for our hosted objects. One important caveat is that objects hosted within IIS can communicate only via the HTTPChannel, the TCP/IP protocol cannot be used. We will cover hosting objects in IIS in a future article, including applying authorization and authentication security to remoting.

3. Component Services

Component Services can also be used to host a remote object. By hosting an object in Component Services we gain access to all of COM+'s features, including transaction management, JIT activation and queued components. These are all available for objects in the .NET Framework, but to use them involves a slightly different process than a simple remote object. To enable access to the COM+ features, an object must derive from ServicedComponent. Once this requirement has been met, various attributes can be applied to the methods and properties to facilitate the Component Services features. As this article is primarily concerned with .NET remoting, we will leave Serviced Components to another article.

Activation Type

The next design decision that must be made is how the server objects will be activated. Two options are available: server-activated objects, or client-activated objects.

  • Server activated objects are controlled by the server application hosting the remote objects. This means the server manages the lifetime of the object and determines when it can be garbage collected.
  • Client activated objects are more like traditional COM components, where the client itself is responsible for the lifetime of the object. .NET does not use reference counting for this, instead a sophisticated lease system exists to control when client-activated objects are no longer in use and can be safely garbage collected.

To complicate matters further, server-activated objects can be one of two types. An object can be a Singleton, which means every call from every client is received and processed by a single object. The other option is SingleCall, which means every call from a client is received and processed by a separate and unique instance of the object. Note that these two options are only available for server-activated objects. The following diagram illustrates how method calls from clients are processed by the two different server-activated activation options.

If you are designing and implementing server-activated objects, care must be taken to avoid keeping state between object method calls. A SingleCall object will not retain any state in the object, as a new object is created every time a request is received. Perhaps more dangerous are Singleton objects, since they will retain state information, but because multiple clients access the same object it is not guaranteed that any data saved to the object is correct. If classic object-oriented state must be maintained in the objects, client-activated objects must be used. Server-activated objects should be used when scalability is paramount, as they utilize fewer resources than client-activated objects.

As mentioned previously, client-activated objects work very much like classic COM components. A client obtains a proxy to an object, and the server maintains that unique object for the client until the client tells the server it no longer requires it. Instead of reference counting, .NET utilizes a time based lease management system that is more efficient and less prone to error. A lease is basically a time period that the server will keep this object alive for this particular client. Every method call on the object will reset the lease time, and before destroying the object the server will poll the client, asking if the object can be garbage collected. This way the client can ensure that the objects it needs remain active on the server. However, if the client is disconnected the server will eventually destroy the objects on its own, thus eliminating a common memory-leak problem with server-utilized resources in previous technologies.

The prime difference between client-activated objects and server-activated objects is that the client-activated objects can safely retain state between method calls. This more closely resembles traditional object-oriented programming, but may not always be the most efficient use of resources. Server-activated objects offer a bigger benefit in terms of performance and scalability, while the client-activated objects provide the ease of use and stateful model many developers are familiar with. As always, the tradeoffs must be weighed, as no single solution will work for every project.

In terms of the code, nothing needs to change in the remote object itself to expose it as either server-activated or client-activated. The only difference is in the hosting application when the object is registered with the remoting subsystem. We will examine how to do this later when we examine remoting configuration files.

Channels and Formatters

Channels are a very important component in the remoting architecture, and the type of channel used is an important design decision. Recall that the channel is the medium of communication between Application Domains and across machine boundaries. All requests and data pass through a channel between the client and the server hosting application. Because of this, the channel objects are based on network protocols that determine how the data will travel over a network between the client and server. The .NET Framework contains two types of channels: one based on the HTTP protocol and one based on raw TCP/IP socket communication.

A design decision must be made to determine if performance is more significant than interoperability. The HTTP channel can send and receive data over any HTTP compliant network, including the Internet and across most firewalls. The TCP/IP channel uses a proprietary protocol that may not be available on all networks, and the ports required may not be open on all firewalls. However, the TCP/IP channel is significantly faster than HTTP, and for internal-only components it is probably an excellent choice.

Channels contain a series of channel sinks, which process the messages passing through the channel. These channel sinks are arranged serially, and when one processes a message, it passes the message to the next in line. The first and last channel sinks also perform special processes. The first is called a formatter, and it encodes the message into a specific format before sending it along the channel. The last channel sink is a transporter, which sends the message along the network to the remote hosting application where the server channel receives it and begins to process it through its sequence of channel sinks. Developers implement channel sinks as a way to plug functionality into the remoting architecture. Message logging, filtering, and encryption are all possible with custom channel sinks.

The channel type determines the transporter object, since an HTTP channel will utilize a HTTP protocol object to send message, while a TCP/IP channel will utilize a TCP/IP protocol object. However, the formatter object is completely independent of the channel. Two formatters are shipped with the .NET Framework: a SOAP formatter and a Binary formatter. Note that this means that a SOAP formatter does not have to be used with an HTTP channel, and a Binary formatter does not have to be used with the TCP/IP channel, although this is the default configuration.

The formatter object serializes messages into a specific encoding. The SOAP formatter creates valid 1.1 SOAP method invocation requests. The Binary formatter creates a proprietary RPC-like invocation packet. This packet is significantly smaller than the SOAP message, and can travel faster over the wire, while the SOAP message can be sent to any system capable of interpreting SOAP 1.1 messages. Again, a design decision must be made between interoperability and performance.

In the next section we will examine configuration files in detail, and we will examine how to expose a remote object via the HTTP or TCP/IP channels with either SOAP or Binary serialization. Also note that the entire Remoting infrastructure, like much of the .NET Framework, is customizable. New channel sinks, formatters, and even channels can be created by developers. This allows us to build a custom distributed object infrastructure, while relying on the .NET remoting system for as little or as much of the gritty details as we desire. It really is a very powerful and flexible system.

XML Configuration Files

While .NET remoting can be programmatically configured, this is not a solution that works well for most situations. As projects grow and requirements change, applications need to be flexible and maintainable. By hardcoding remoting configuration information into the code when a change is made, the code must be recompiled every time - not acceptable in an enterprise solution. A far better solution is to use a remoting configuration file. We have touched on this a few times before now, and we are finally going to cover it in detail.

A remoting configuration file is needed in two places: the server and the client. On the server it registers the remotable object with the remoting system, and configures the correct channel to listen for client requests. On the client this file performs a similar task, except that it registers the object as a proxy, therefore the code it creates is a configured object with a proxy reference to the server. The format of the two configuration files is slightly different, but obviously very similar. These files must be deployed with the application, but can be included as part of the general application's configuration file.

The configuration file contains all the various design decisions we have discussed so far. It is where the activation type, channel, and formatter are specified for the remote object. We register this configuration file with the remoting system by calling RemotingConfiguration.Configure() with the name of the configuration file, as we saw in our console hosting application. An application hosted in IIS does not call this method. Instead the remoting configuration data merely needs to be included as part of the Web.config file.

Now let's get down to actually creating the configuration file. Create a new text file and save it as Config.xml. Note that this file can be called anything, just pass in the correct file name to the call to RemotingConfiguration.Configure(). Add the following six lines to the file:

<configuration>
<system.runtime.remoting>
<application>
</application>
</system.runtime.remoting>
</configuration>

These tags will always exists in the remoting configuration file regardless of other options. This simply specifies the beginning of the remoting section. Note that there are a host of other options available in the configuration schema; please see the .NET Framework documentation for the complete details.

Inside the application tag we are going to place the remotable objects that this application exposes. We create the server configuration file first - we will see in the next section how to create the client configuration file. Add a <service> tag within the <application> tag. This will contain all objects this application exposes. We now have two choices, depending on the activation type of the object.

Server-Activated

The following is the complete <service> section for a server-activated object. The <wellknown> tag is used to denote a server-activated object type. Three mandatory attributes must be included inside this tag. The type attribute specifies the object type to be remoted. It is in the format of Namespace.ObjectName, AssemblyName. Don't forget to include the assembly name after the comma! The objectUri attribute specifies the object's Uniform Resource Identifier. Clients use this when they make requests for this object. The standard practice is to have this be the object name plus .rem or .soap depending on the formatting type (.rem for Binary, .soap for SOAP). The mode attribute can either be the value Singleton or SingleCall. This determines what type of server activation is used with this object. Recall that SingleCall objects are newly created for every method call from a client, while a Singleton object is shared across all client requests and only one object ever exists.

<service>
<wellknown type="RemoteObjectLib.RemoteObject, RemoteObjectLib"
objectUri="RemoteObject.rem"
mode="Singleton"/>
</service>

Client-Activated

The following is the same section for a client-activated object. Note that the tag used in this case is <activated> and the only attribute used is type. This attribute must include the name of the object exposed as well as the assembly name this object is located in.

<service>
<activated type="RemoteObjectLib.RemoteObject, RemoteObjectLib"/>
</service>

Channels

We must now configure the type of channel and formatter used by the hosting application. This can be very simple if you want to use one of the standard configurations, or it can be a bit more complex and involved if you want to develop your own remoting components. We will be using the <channels> tag, which will be added inside the <application> tag. This defines channels to be used with this remoting application. You can specify channels in other places as templates, however this is beyond the scope of this article.

There can be several <channel> tags within the <channels> tag. Each specifies a channel this server application will use to handle remoting requests. Note this means your objects will be available to clients using any listed channel type. This way an internet application can connect to the object over an HTTP Channel, while intranet applications can connect over a TCP/IP channel.

Unless you are designing a custom channel object, most of the configuration files you create will reference one of the existing channel objects, either the TCP/IP Channel or the HTTP Channel. To reference these channels, use the ref attribute in the <channel> tag, using "tcp " or "http " respectively. In addition, a port attribute must be declared to specify on which port to listen for remoting requests. The following illustrates how to configure remoting to listen for request on both TCP/IP and HTTP channels. This section would be included inside the <application> tag.

<channels>
<channel ref="tcp"
            port="6000"/>
<channel ref="http"
            port="8080"/>
</channels>

By default, the TCP/IP channel utilizes the Binary formatter, and the HTTP channel utilizes the SOAP formatter. If this is the behavior you desire, nothing extra needs to be set in the configuration file to achieve this. However, if you would like Binary serialization over HTTP, or SOAP serialization over TCP/IP, a few extra settings must be set.

The <serverProviders> tag can be included within the <channel> tag. This tag is used to define the channel sink providers, which are simply objects that provide access to a channel sink. Recall that a channel can contain any number of optional channel sinks, which can enforce various business rules or logging functionality. In addition to these, which must be created by a developer as none are shipped with the .NET Framework, a single formatter must be specified. This is a special channel sink that appears first in the channel sink chain and formats the message into a specific format. Again, as everything is customizable in the remoting system, it is even possible to build your own formatter sinks to customize the format of .NET messages.

The formatter sink is declared with the <formatter> tag as illustrated below. The ref attribute references a previously defined (or intrinsic) formatter sink. To reference the SOAP formatter, use the "soap " id, and to reference the Binary formatter, use the "binary " id. Again, if you are building and deploying a custom formatter sink then this tag will include type and assembly information for this custom object, but since we are simply referencing the intrinsic formatters all we need is the ref attribute.

<channels>
<channel ref="http" port="8080">
<serverProviders>
<formatter ref="binary"/>
</serverProviders>
</channel>
</channels>

This example section declares an HTTP channel, but uses the binary formatter sink. Again remember that this is the server configuration file; there will be some changes for the client file. The following example illustrates how to use a TCP/IP channel with a SOAP formatter:

<channels>
<channel ref="tcp" port="6000">
<serverProviders>
<formatter ref="soap"/>
</serverProviders>
</channel>
</channels>

Note that any number of <provider> tags can also exist under the <serverProvider> tag. It is with these tags that we define the channel sink chain, and the order they are declared under the <serverProvider> is the order messages pass through them. Again, this is a point of customization to add functionality into the .NET remoting system as messages pass through.

We are now ready to put it all together and build our complete configuration file. As there are a number of different options we can choose, we will only show a single configuration file. However, in the code download for this article can be found a series of configuration files that illustrate all the options covered so far. These should provide an excellent starting point for building your own remoting applications.

<configuration>
<system.runtime.remoting>
<application>
<channels>
<channel ref="tcp" port="6000">
<serverProviders>
<formatter ref="soap"/>
</serverProviders>
</channel>
</channels>
<service>
<wellknown type="RemoteObjectLib.RemoteObject, RemoteObjectLib"
objectUri="RemoteObject.soap"
mode="Singleton"/>
</service>
</application>
</system.runtime.remoting>
</configuration>

Save this file as Config.xml into the same directory as the RemoteServer application. Note that it must exist in the RemoteServer/bin/debug directory, because this is where the application actually runs from. The RemoteObjectLib.dll file also needs to be copied into this same directory. If this is not done the hosting application will not be able to load the correct object, unless the assembly has been placed into the Global Assembly Cache. Try running the RemoteServer application to ensure no exceptions are raised. If everything works correctly, the following screens should appear:

Client Settings

In the next section we will build a simple client to consume the remote object, but first let's look at how to build the client configuration file. This file is used by the client application to register the objects via a proxy, the location of the remote objects, and what channels and formatters to use. This file is very similar to the server version, but there are a few key differences.

In a client configuration file the <service> tag is replaced with a <client> tag, representing the objects that this application will consume remotely. Note that both a <service> and a <client> tag can exist within a configuration file if an application both consumes remote objects as well as exposes objects itself. Inside of the <client> tag can exist <wellknown> tags for server-activated objects, and <activated> tags for client-activated objects in a similar fashion to the server configuration file. The type of activation in the client configuration file must match what has been specified on the server for an object.

If you are using server-activated objects, two attributes must be included within the <wellknown> tag. These are url and type. The url attribute is the complete Uniform Resource Locator address of this object. This must contain the protocol type, server machine name, remote port, and object URI in the format: ProtocolType://ServerName:PortNumber/ObjectURI. The protocol type will be either tcp or http, depending on the type of channel specified. Both the server name and port number must be specified. The object URI is declared in the server configuration file as the objectUri. The type attribute defines the remote object using the same format as the server configuration file: Namespace.ObjectName, AssemblyName. The .NET Framework uses this information to intercept any requests for this object type and forwards them to the local proxy, which in turn pushes messages to the correct remoting channel. The following is an example of a <client> section of a configuration file for our previously exposed object:

<client>
<wellknown type="RemoteObjectLib.RemoteObject, RemoteObjectLib"
url="tcp://localhost:6000/RemoteObject.soap"/>
</client>

If we are consuming a client-activated remote object, the configuration file will be slightly different. Instead of the <wellknown> tag the <activated> tag is used to declare remote objects. Within this tag only a single attribute is available: the type attribute. This again must declare the object type and assembly name. In addition, within the <client> tag, the additional attribute url must be added to reference the remote server. This looks much like the url we discussed previously, however without any object-specific information. An example of a complete <client> section for our remote object is shown below, if it was exposed as a client-activated object.

<client url="tcp://localhost:6000">
<activated type="RemoteObjectLib.RemoteObject, RemoteObjectLib"/>
</client>

Client Channels

Clients must also register what channels they will use to communicate with the server. This is done in exactly the same manner as the server configuration settings with one exception. If custom channel sinks are used they must be declared with the <clientProviders> tag instead of the <serverProviders> tag. The options available are the same, but there must be a distinction between server channel sinks and client channel sinks.

One option that may be used on the client is to specify zero for a port number. This tells the client to use any available port on the client machine to communicate with the server. Servers will most likely never use this setting because clients must have a well-known port to connect to. However clients that only need to make requests on servers do not need to have a hardcoded port number. Often they will simply declare the type of channel they will use and specify zero for the port number to utilize the next available port.

That's it - we can now build our complete client configuration file to communicate with our remote object. The complete file is shown below. Again, due to the many options available this is only one possible configuration. This particular file references the server-activated object that the previous server configuration file exposes. In the download for this article there are a number of other templates to use for client configuration files in various situations. Save this file as ConfigClient.xml; we will use it in the next section.

<configuration>
<system.runtime.remoting>
<application>
<channels>
<channel ref="tcp" port="0">
<clientProviders>
<formatter ref="soap"/>
</clientProviders>
</channel>
</channels>
<client>
<wellknown type="RemoteObjectLib.RemoteObject, RemoteObjectLib"
url="tcp://localhost:6000/RemoteObject.soap"/>
</client>
</application>
</system.runtime.remoting>
</configuration>

Consuming a Remote Object

Now that we have a client configuration file, we can now build a client and see this remoting business in action. One of the requirements of a good distributed object system is that it must be simple for clients to use remote objects. Fortunately, when a remoting configuration file is in place, objects can be created and used in exactly the same manner as standard local objects.

Create a new C# console project in Visual Studio.NET titled RemoteClient.

Add the following code, and don't forget the namespace declarations at the top!

using System;
using System.Runtime.Remoting;
using RemoteObjectLib;

namespace RemoteClient
{
class RemoteClient
{
[STAThread]
static void Main(string[] args)
{
Console.WriteLine("Registering Remoting.");

try
{
RemotingConfiguration.Configure("ConfigClient.xml");
}
catch(Exception ex)
{
Console.WriteLine("Error registering Remoting.");
Console.WriteLine(ex.ToString());
}

Console.WriteLine("Current AppDomain: {0}", AppDomain.CurrentDomain.FriendlyName);

//Create remote object
RemoteObject aRemoteObject = new RemoteObject();
Console.WriteLine("Remote Object's AppDomain: {0}",aRemoteObject.GetData());

//Wait for user to press enter before quitting
Console.ReadLine();
}
}
}

To get this to work we need to add the RemoteObjectLib assembly to this project. Do this by selecting Project->Add Reference. Select Browse and navigate to the RemoteObjectLib.dll file in the RemoteObject/bin/Debug directory. The assembly will be added to the client application, but only as a type library. Remote objects themselves will not run within the client's application domain. This is demonstrated by the output of this application, as the remote object will echo out the Application Domain it is running within. Nonetheless, this illustrates an important issue with .NET remoting: the client application must have the remote assembly. In a future article, I will discuss how to design remoting objects using interfaces so an "empty" assembly can be distributed to the clients and all the functionality can be maintained on the server.

Back to the code itself, the first thing happening is that it registers the client configuration file. Copy the ClientConfig.xml file we created earlier into the RemoteClient/bin/Debug directory with the executable. The application will echo out its current Application Domain name. The application then creates a RemoteObject, which is created remotely on the server, and calls the GetData() method. This returns the Application Domain of the remote object, which will be different to the local client application.

Note that we created the remote object with the new keyword, and other than the call to RemotingConfiguration.Configure(), everything looks like a standard application.

We are now ready to see this system in action. First start up the hosting application. As we have developed the configuration files to reference the local machine, I am assuming this means simply starting the hosting application. If you wish to test this more realistically, simply replace 'localhost' with the name of the remote server in the client configuration file and run the hosting application on the sever. Don't forget the server configuration file and the RemoteObjectLib assembly must be in the same directory as the hosting application.

Once the server hosting application, RemoteServer.exe is started, and it has registered the remoting objects and is awaiting client requests, run RemoteClient.exe. The screen should look like the following:

This application displays the current Application Domain, and then calls the remote object to retrieve the Application Domain of the remote object. The local application is not hosting the object; it is being executed on the server.

Conclusion

.NET remoting is a large topic, and we have only covered the basics, but it is still very extensive. The techniques discussed can be used to build sophisticated distributed object architectures in .NET. Using remoting configuration files makes this an enterprise solution that can adapt quickly to change.

In this article we covered the following topics:

  • What .NET remoting is and how it works
  • What the MarshalByRefObject class is and how it relates to remoting
  • How to create a remoteable object
  • How to host a remote object in a server application
  • How to create remoting configuration files for easy management and maintenance
  • How to consume a remoting object as a client.

In the next article I will cover advanced remoting topics like remoting events, custom channel sinks, and advanced lease management for client-activated objects. Until then, have fun with .NET!


Mapping Windows Forms to A Relational Database Part 2

Mapping Windows Forms to A Relational Database Part 2
http://www.csharptoday.com/content.asp?id=1700
Article Rating: 4.5
Published on January 8, 2002
Abstract
The purpose of this article series is to create an application for saving time and effort in mapping relational databases. In this installment, Levent Camlibel focuses on generating Windows Forms and C# objects, using the custom objects developed in the first part of the article.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

The purpose of this article series is to create an application for saving time and effort in mapping relational databases. In this final part we will focus on generating Windows Forms and C# objects, using the custom objects developed in the first part of the article.

System Requirements

This software requires Visual Studio .NET Beta 2 or later to compile, and Microsoft SQL Server or MSDE for database access.

Project Structure

There are three Visual Studio .NET projects and three namespaces in the project group. These are:

  • Mapping.CustomObjects: This namespace is for custom objects and will be explained in detail later.
  • Mapping.WinForms: This namespace is for the project that generates Windows Forms.
  • Mapping.ClassObjects: This namespace is for the project that generates C# database related objects

Before getting into a discussion about generating Windows Forms and C# objects, let us recap what we covered yesterday on custom objects.

Custom Objects

  • The Database object can be considered as DataSet object,
  • The Table object can be considered as DataTable and
  • The Column object can be considered as DataColumn in .NET framework.
  • Databases, Tables and Columns objects can be considered as Collections of above objects.

For the discussion of .NET DataSet versus custom objects, please refer to the first part of the article.

Article Structure

  • Generating Windows Forms
  • Generating C# Objects

Windows Forms are the reflection of a table in a relational database. Let's see a Windows Form generated by this application:

The title of the Windows Form is taken from the name of a database, such as "USERS". Each Label control in the Windows Form reflects one field in the database table. We will cover how to parse database fields into the right Windows Forms controls, such as TextBox, DateTimePicker, or CheckBox. In addition, you will see three Button Windows controls for inserting, updating, and reporting data. These buttons will initiate the right C# objects to insert, update, and report data. Now we have an idea about what we want to accomplish, let's start with the subject of generating Windows Forms.

Generating Windows Forms

A Windows Form control, either a Textbox, Label, DateTimePicker, or CheckBox, will map to each field in a relational database. We map each database field to the right Windows Forms control based on the data type of the field. For example, if the data type is System.DateTime, we drag and drop a DateTimePicker on the Windows Form.

Let's look at the code and start with the main part of generating Windows Forms (WinForms.cs):

namespace Mapping.WinForms
{
public class WinForms
{
protected String connectionString;
protected StreamWriter streamWrite;
protected FileStream streamFile;
protected int PointX;
protected int PointY;
protected String className;
protected Databases dataBaseCollection;
public static string ONE_TAB ="\t";
public static string TWO_TAB ="\t\t";
public static string THREE_TAB ="\t\t\t";
public static string FOUR_TAB ="\t\t\t\t";
public static string FIVE_TAB ="\t\t\t\t\t";
public static string SIX_TAB ="\t\t\t\t\t\t";
public static string NINE_TAB ="\t\t\t\t\t\t\t\t\t";

public WinForms(Databases DataBaseCollection, String ConnectionString)
{
this.dataBaseCollection=DataBaseCollection;
this.connectionString=ConnectionString;
}

The WinForms class is the main class used to generate Windows Forms. The above code gives some idea of how it is initiated, and what the instance variables are.

  • connectionString: holds the connection string for database access
  • streamWrite: an instance of StreamWriter object in order to write code to files
  • streamFile: an instance of FileStream object in order to define the file access
  • PointX: holds horizontal distance (points)
  • PointY: holds vertical distance
  • className: holds Table name in a relational database, and is used to name windows forms and class files.
  • dataBaseCollection: an instance of Databases custom object. This object holds all information that is parsed from a relational database. We parsed it in the first part of the article.
  • ONE_TAB and others: used to indent code for easy reading.

When the user initiates this object, the user has to provide two arguments, DataBaseCollection, and ConnectionString. DatabaseCollection must have all the information of the relational database, otherwise the software will not be able to generate Windows Forms and C# objects.

After initiating the WinForms object, the user can use the following function to generate Windows Forms.

public void CreateWinForms(String FileLocation, String Namespace, String DataNameSpace)

This function requires three arguments:

  • FileLocation: a folder name in which the user can save Windows Forms. For example, C:\projects,
  • Namespace: When the application generates Windows Forms, it will add this value as the namespace of the Windows Forms.
  • DataNameSpace: This will be the namespace declaration of C# objects and is also used for referencing C# objects in your Windows Forms.

The following code is taken from the CreateWinForms function in WinForms.cs. When we generate Windows Forms, we use the following objects:

  • WindowsFormDesignerGeneratedCodes Object: This object generates the pre-created code in the Windows Forms, similar to the code we see in the "+ Windows Form Designer generated code" section of a Windows Form. We will look at this object in detail later.
  • CreateInstanceVariables Object: This object is responsible for generating instance variables for the Windows Forms.
  • WinFormFunctions Object: This object is responsible for generating Insert, Update, and Report functions that are linked with C# database layer objects.

WinForms.cs continues:

WindowsFormDesignerGeneratedCodes
            windesGene= new
            WindowsFormDesignerGeneratedCodes();
CreateInstanceVariables
            instVar=new CreateInstanceVariables();
WinFormFunctions
            winFunctions =new
            WinFormFunctions(this.connectionString);

for(int i=0; i<
            this.dataBaseCollection[0].Tables.Count;
            i++)
{
className=this.dataBaseCollection[0].Tables[i].TableName;

///check
            whether file location has \ at the end of filelocation
            string
///if it is not there, append the string with "\"
            character

FileLocation=FileLocation.Trim();
if(FileLocation.Substring(FileLocation.Length-1,1)!="\\")
FileLocation+="\\";

///if
            the filelocation is not exist
///create one, here
if(!Directory.Exists(FileLocation))
{
Directory.CreateDirectory(FileLocation);
}
try
{
streamFile
            = new FileStream(FileLocation+"frm"+className+".cs",
            FileMode.Create, FileAccess.Write);

streamWrite = new
            StreamWriter(streamFile);

streamWrite.WriteLine("using
            System;");
streamWrite.WriteLine("using
            System.Data;");
streamWrite.WriteLine("using
            System.Drawing;");
streamWrite.WriteLine("using
            System.Collections;");
streamWrite.WriteLine("using
            System.ComponentModel;");
streamWrite.WriteLine("using
            System.Windows.Forms;");
streamWrite.WriteLine("using
            "+DataNameSpace+";");
streamWrite.WriteLine();
streamWrite.WriteLine("namespace
            "+Namespace);
streamWrite.WriteLine("{");
streamWrite.WriteLine(ONE_TAB+"public
            class frm"+className.Replace(" ", "_")+":
            System.Windows.Forms.Form");
streamWrite.WriteLine(ONE_TAB+"{");

Let's explain the code above.

for(int i=0; i< this.dataBaseCollection[0].Tables.Count; i++)

This loop iterates all tables in the databaseCollection custom object. We need this because we want to generate one Windows Form for each database table.

FileLocation=FileLocation.Trim();
if(FileLocation.Substring(FileLocation.Length-1,1)!="\\")
FileLocation+="\\";

if(!Directory.Exists(FileLocation))
{
Directory.CreateDirectory(FileLocation);
}

The above code checks the validity of the file location. If the folder does not exist, we create one. The next step is to instantiate the FileStream and StreamWriter objects by using:

streamFile = new
            FileStream(FileLocation+"frm"+className.Replace(" ", "_")+".cs",
            FileMode.Create, FileAccess.Write);

streamWrite = new
            StreamWriter(streamFile);

We use the "frm" suffix and name of a database table to name the Windows Forms. The following code ensures that there is no space in table name. For example, the table name might be Order Details:

className.Replace(" ", "_")+".cs"

The result will be a file called frmOrder_Details.cs

The rest of the code writes necessary namespace declarations in a Windows Form. The example output of generated code might be:

using System;
using System.Data;
using System.Drawing;
using System.Collections;
using System.ComponentModel;
using System.Windows.Forms;
using Camlibel.Data; // database layer namespace(this namespace is entered when the user configures the application).

namespace Camlibel.WinForms //This is also entered by the user
{
public class frmOrder_Details: System.Windows.Forms.Form
{

After this point, it is time to generate instance variables using the CreateInstanceVariables object. The following code will call the function WriteInstanceVariables of this object.

instVar.WriteInstanceVariables(ref streamWrite,this.dataBaseCollection[0].Tables[i].Columns);

As you see, we pass StreamWriter object as reference, and Columns custom object of a particular database table. We pass StreamWriter object as reference because we want to write the code into the same file defined in the WinForms object.

public void WriteInstanceVariables(ref StreamWriter streamWrite, Columns ColumnCollection)
{
for(int i=0;i<ColumnCollection.Count;i++)
{
streamWrite.Write(WinForms.TWO_TAB+"private ");
streamWrite.Write("System.Windows.Forms.Label ");
streamWrite.WriteLine("lbl"+ColumnCollection[i].ColumnName.Replace(" ","_")+";");

if(ColumnCollection[i].Type=="System.DateTime")
{
streamWrite.Write(WinForms.TWO_TAB+"private ");
streamWrite.Write("System.Windows.Forms.DateTimePicker ");
streamWrite.WriteLine("dt"+ColumnCollection[i].ColumnName.Replace(" ","_")+";");
}
else if(ColumnCollection[i].Type=="System.Boolean")
{
streamWrite.Write(WinForms.TWO_TAB+"private ");
streamWrite.Write("System.Windows.Forms.CheckBox ");
streamWrite.WriteLine("ch"+ColumnCollection[i].ColumnName.Replace(" ","_")+";");
}
else
{
streamWrite.Write(WinForms.TWO_TAB+"private ");
streamWrite.Write("System.Windows.Forms.TextBox ");
streamWrite.WriteLine("txt"+ColumnCollection[i].ColumnName.Replace(" ","_")+";");
}
streamWrite.WriteLine();
}
streamWrite.WriteLine(WinForms.TWO_TAB+"private System.Windows.Forms.Button btnInsert;");
streamWrite.WriteLine(WinForms.TWO_TAB+"private System.Windows.Forms.Button btnUpdate;");
streamWrite.WriteLine(WinForms.TWO_TAB+"private System.Windows.Forms.Button btnReport;");
}

As you see from the code above, we generate one of three different Windows Form controls depending upon the data type of the database field.

1- If the data type is DateTime, we generate code to declare a DateTimePicker Windows Form control. Moreover, we used a "dt" suffix in order to name the control. WinForms.TWO_TAB adds two tabs space to the line.

streamWrite.Write(WinForms.TWO_TAB+"private ");
streamWrite.Write("System.Windows.Forms.DateTimePicker ");
streamWrite.WriteLine("dt"+ColumnCollection[i].ColumnName.Replace(" ","_")+";");

For example, the above code generates the output below:

private System.Windows.Forms.DateTimePicker dtOrderDate;

2- If the data type is Boolean, we generate a code to declare a CheckBox Windows Form control. The naming convention is "ch".

3- If the data type is not DateTime or Boolean, we generate a Textbox Windows Form control. The naming convention is "txt". For example:

private System.Windows.Forms.TextBox txtFirstName;
private System.Windows.Forms.CheckBox chStatus;

Moreover, every database field will have a Label Windows Form control next to them. The naming convention is "lbl". However, this application does not handle Byte[] types such as variants, or pictures. After parsing columns to instance variables, we add three Button Windows Form controls for inserting, updating and reporting purposes.

After this point, we need to write code to generate the following:

public
            Form1()
{
InitializeComponent();
}
protected override
            void Dispose( bool disposing )
{
if( disposing )
{
if
            (components != null)
{
components.Dispose();
}
}
base.Dispose( disposing
            );
}

If you look at the WinForms.cs, you will see the code to do this:

streamWrite.WriteLine(TWO_TAB+"private
            System.ComponentModel.Container components =
            null;");
streamWrite.WriteLine(TWO_TAB+"public
            frm"+className.Replace("
            ","_")+"()");
streamWrite.WriteLine(TWO_TAB+"{");
streamWrite.WriteLine(THREE_TAB+"InitializeComponent();");
streamWrite.WriteLine(TWO_TAB+"}");
streamWrite.WriteLine(TWO_TAB+"protected
            override void Dispose( bool disposing
            )");
streamWrite.WriteLine(TWO_TAB+"{");
streamWrite.WriteLine(THREE_TAB+"if(
            disposing
            )");
streamWrite.WriteLine(THREE_TAB+"{");
streamWrite.WriteLine(FOUR_TAB+"if(components
            !=
            null)");
streamWrite.WriteLine(FOUR_TAB+"{");
streamWrite.WriteLine(FIVE_TAB+"components.Dispose();");
streamWrite.WriteLine(FOUR_TAB+"}");
streamWrite.WriteLine(THREE_TAB+"}");
streamWrite.WriteLine(THREE_TAB+"base.Dispose(disposing);");
streamWrite.WriteLine(TWO_TAB+"}");

Now we are ready to generate code for initializing components. To do this, we use the following function that requires the table name (classname), StreamWriter instance by reference, and Columns custom object that consists of all necessary information.

windesGene.WriteDesignerGeneratedCodes(this.className,ref streamWrite,this.dataBaseCollection[0].Tables[i].Columns);

Let's see what happens in this function (in WindowsFormDesignerGeneratedCodes.cs):

public void WriteDesignerGeneratedCodes(String ClassName, ref StreamWriter streamWrite, Columns ColumnCollection)
{
this.className=ClassName;
StringBuilder strRow= new StringBuilder();
streamWrite.WriteLine(WinForms.TWO_TAB+"#region Windows Form Designer Generated Code");
streamWrite.WriteLine(WinForms.TWO_TAB+"private void InitializeComponent()");
streamWrite.WriteLine(WinForms.TWO_TAB+"{");

The above code generates the following output:

#region Windows Form Designer generated code

private void InitializeComponent()
{

As you see, we declare strRow as StringBuilder. I have used this variable to hold the class names (table names) in order to generate the following code output:

this.Controls.AddRange(new System.Windows.Forms.Control[] {
this.label3,
this.txtDataNameSpace,
this.btnCode,
this.txtnameSpace,
this.NameSpace,
this.txtPassword,
this.txtUserName,
this.label6,
this.label5,
this.btnWinForms,
this.txtLocation,
this.label4,
this.txtDataBase,
this.label2,
this.treeTables,
this.txtServer,
this.label1,
this.btnGetTables});

Let's continue. It is now time to generate code for initializing components on the form. To do this, we use a for loop to iterates the columns custom object:

for(int i=0;i<ColumnCollection.Count;
            i++)
{
streamWrite.Write(WinForms.THREE_TAB+"this.");
streamWrite.Write("lbl"+ColumnCollection[i].ColumnName.Replace("
            ","_")+"=");
streamWrite.WriteLine("new
            System.Windows.Forms.Label();");

if(ColumnCollection[i].Type=="System.DateTime")
{
streamWrite.Write(WinForms.THREE_TAB+"this.");
streamWrite.Write("dt"+ColumnCollection[i].ColumnName.Replace("
            ","_")+"=");
streamWrite.WriteLine("new
            System.Windows.Forms.DateTimePicker();");
}
else
            if(ColumnCollection[i].Type=="System.Boolean")
{
streamWrite.Write(WinForms.THREE_TAB+"this.");
streamWrite.Write("ch"+ColumnCollection[i].ColumnName.Replace("
            ","_")+"=");
streamWrite.WriteLine("new
            System.Windows.Forms.CheckBox();");
}
else
{
streamWrite.Write(WinForms.THREE_TAB+"this.");
streamWrite.Write("txt"+ColumnCollection[i].ColumnName.Replace("
            ","_")+"=");
streamWrite.WriteLine("new
            System.Windows.Forms.TextBox();");
}
}
streamWrite.WriteLine(WinForms.THREE_TAB+"this.btnInsert
            = new
            System.Windows.Forms.Button();");
streamWrite.WriteLine(WinForms.THREE_TAB+"this.btnUpdate
            = new
            System.Windows.Forms.Button();");
streamWrite.WriteLine(WinForms.THREE_TAB+"this.btnReport
            = new System.Windows.Forms.Button();");

The code above generates, for example, the following component-initializing code:

this.txtDataBase = new
            System.Windows.Forms.TextBox();
this.txtLocation = new
            System.Windows.Forms.TextBox();
this.txtServer = new
            System.Windows.Forms.TextBox();
this.btnWinForms = new
            System.Windows.Forms.Button();

After initializing the code, we have to put these components on the form. The properties of a component might change based on the type of the component. The following code designs Windows Forms and puts the necessary components on it.

for(int i=0;i<ColumnCollection.Count;
            i++)
{

if(Count %
            2==0)
{
PointY+=32;
PointX=8;
}
strRow.Append(WinForms.NINE_TAB+"lbl"+ColumnCollection[i].ColumnName.Replace("
            ","_")+",\r\n");

streamWrite.Write(WinForms.THREE_TAB+"this.");
streamWrite.Write("lbl"+ColumnCollection[i].ColumnName.Replace("
            ","_")+".");
streamWrite.WriteLine("Location=new
            System.Drawing.Point("+PointX+","+PointY+");");
streamWrite.Write(WinForms.THREE_TAB+"this.");
streamWrite.Write("lbl"+ColumnCollection[i].ColumnName.Replace("
            ","_")+".Text=");
streamWrite.WriteLine("\""+ColumnCollection[i].ColumnName.Replace("
            ","_")+"\";");
streamWrite.Write(WinForms.THREE_TAB+"this.");
streamWrite.Write("lbl"+ColumnCollection[i].ColumnName.Replace("
            ","_")+".Name=\"");
streamWrite.WriteLine("lbl"+ColumnCollection[i].ColumnName.Replace("
            ","_")+"\";");
streamWrite.WriteLine();
PointX+=100;
if(ColumnCollection[i].Type=="System.DateTime")
{
strRow.Append(WinForms.NINE_TAB+"dt"+ColumnCollection[i].ColumnName.Replace("
            ","_")+",\r\n");
streamWrite.Write(WinForms.THREE_TAB+"this.");
streamWrite.Write("dt"+ColumnCollection[i].ColumnName.Replace("
            ","_")+".");
streamWrite.WriteLine("Location=new
            System.Drawing.Point("+PointX+","+PointY+");");
streamWrite.Write(WinForms.THREE_TAB+"this.");
streamWrite.Write("dt"+ColumnCollection[i].ColumnName.Replace("
            ","_")+".Name=\"");
streamWrite.WriteLine("dt"+ColumnCollection[i].ColumnName.Replace("
            ","_")+"\";");
streamWrite.Write(WinForms.THREE_TAB+"this.");
streamWrite.Write("dt"+ColumnCollection[i].ColumnName.Replace("
            ","_")+".Size=");
streamWrite.WriteLine("new
            System.Drawing.Size(100,
            20);");
streamWrite.Write(WinForms.THREE_TAB+"this.");
streamWrite.Write("dt"+ColumnCollection[i].ColumnName.Replace("
            ","_")+".Format=");
streamWrite.WriteLine("System.Windows.Forms.DateTimePickerFormat.Short;");
}
else
            if(ColumnCollection[i].Type=="System.Boolean")
{
strRow.Append(WinForms.NINE_TAB+"ch"+ColumnCollection[i].ColumnName.Replace("
            ","_")+",\r\n");
streamWrite.Write(WinForms.THREE_TAB+"this.");
streamWrite.Write("ch"+ColumnCollection[i].ColumnName.Replace("
            ","_")+".");
streamWrite.WriteLine("Location=new
            System.Drawing.Point("+PointX+","+PointY+");");
streamWrite.Write(WinForms.THREE_TAB+"this.");
streamWrite.Write("ch"+ColumnCollection[i].ColumnName.Replace("
            ","_")+".Name=\"");
streamWrite.WriteLine("ch"+ColumnCollection[i].ColumnName.Replace("
            ","_")+"\";");
streamWrite.Write(WinForms.THREE_TAB+"this.");
streamWrite.Write("ch"+ColumnCollection[i].ColumnName.Replace("
            ","_")+".FlatStyle=");
streamWrite.WriteLine("System.Windows.Forms.FlatStyle.Popup;");
}
else
{
strRow.Append(WinForms.NINE_TAB+"txt"+ColumnCollection[i].ColumnName.Replace("
            ","_")+",\r\n");
streamWrite.Write(WinForms.THREE_TAB+"this.");
streamWrite.Write("txt"+ColumnCollection[i].ColumnName.Replace("
            ","_")+".");
streamWrite.WriteLine("BorderStyle =
            System.Windows.Forms.BorderStyle.FixedSingle;");
streamWrite.Write(WinForms.THREE_TAB+"this.");
streamWrite.Write("txt"+ColumnCollection[i].ColumnName.Replace("
            ","_")+".");
streamWrite.WriteLine("Location=new
            System.Drawing.Point("+PointX+","+PointY+");");
streamWrite.Write(WinForms.THREE_TAB+"this.");
streamWrite.Write("txt"+ColumnCollection[i].ColumnName.Replace("
            ","_")+".Name=\"");
streamWrite.WriteLine("txt"+ColumnCollection[i].ColumnName.Replace("
            ","_")+"\";");

if(ColumnCollection[i].Type=="System.String")
{
streamWrite.Write(WinForms.THREE_TAB+"this.");
streamWrite.Write("txt"+ColumnCollection[i].ColumnName.Replace("
            ","_")+".MaxLength=");
streamWrite.WriteLine(ColumnCollection[i].Length+";");
streamWrite.WriteLine();
}
else
{
streamWrite.WriteLine();
}
}
PointX+=150;
Count++;
}

DateTimePicker, Label, TextBox and CheckBox might have different properties. Because of that, we use If statements to separate components. The code generates, for example, the following output.

this.txtnameSpace.BorderStyle =System.Windows.Forms.BorderStyle.FixedSingle;
this.txtnameSpace.Location = new System.Drawing.Point(376, 136);
this.txtnameSpace.Name = "txtnameSpace";
this.txtnameSpace.TabIndex = 20;
this.txtnameSpace.Text = "Camlibel.WinForms";

So now, we have completed the basic Windows Form design. But we want to add functionality to insert, update, and report to the database. Because of that, we create another C# object for this task in WinFormFunctions.cs. The following code is taken from WinForms.cs:

winFunctions.WriteInsertFunction(ref streamWrite, this.dataBaseCollection[0].Tables[i].Columns,this.className);

winFunctions.WriteReportFunction(ref streamWrite, this.dataBaseCollection[0].Tables[i].Columns,this.className);

winFunctions.WriteUpdateFunction(ref streamWrite, this.dataBaseCollection[0].Tables[i].Columns,this.className);

These functions will generate insert, update, and report functions. Let's see the code behind it:

public void WriteInsertFunction(ref StreamWriter streamWrite, Columns ColumnCollection,String className)
{
String forwardValue=ParseTypesForInsert(ColumnCollection);
streamWrite.WriteLine(WinForms.TWO_TAB+"private void btnInsert_Click(object sender, System.EventArgs e)");
streamWrite.WriteLine(WinForms.TWO_TAB + "{");
streamWrite.WriteLine();
streamWrite.WriteLine(WinForms.THREE_TAB+"try");
streamWrite.WriteLine(WinForms.THREE_TAB+"{");
streamWrite.WriteLine(WinForms.FOUR_TAB +"cls"+className.Replace(" ", "_")+
" _"+className.Replace(" ", "_")+"= new "+ "cls"+className.Replace(" ", "_")+
"(@\""+this.connectionString+"\");");
streamWrite.WriteLine(WinForms.FOUR_TAB +" _"+className.Replace(" ", "_")+
".Insert("+forwardValue+");");

streamWrite.WriteLine(WinForms.THREE_TAB+"}");
streamWrite.WriteLine(WinForms.THREE_TAB+"catch(Exception ex)");
streamWrite.WriteLine(WinForms.THREE_TAB+"{");
streamWrite.WriteLine(WinForms.FOUR_TAB+"throw ex;");
streamWrite.WriteLine(WinForms.THREE_TAB+"}");
streamWrite.WriteLine(WinForms.TWO_TAB+"}");
}

The output might look like the following:

private void btnInsert_Click(object sender,
            System.EventArgs e)
{
try
{
clsFirmPermission
            _FirmPermission= new
            clsFirmPermission(@"server=(local)\vste;uid=sa;pwd=;database=supplynet;");
_FirmPermission.Insert(Int32.Parse(txtFirm_ID.Text),
            Int32.Parse(txtPermission_ID.Text),
Int32.Parse(txtSNET_ID.Text),
            txtPermission_Value.Text);
}
catch(Exception ex)
{
throw
            ex;
}
}

As can be seen, during this process, the code will instantiate the database layer object for inserting data. After that, it will call Insert function with parsed values. So, let us see how to handle the parsing. The following code will parse the value based on data type:

private String ParseTypesForInsert(Columns
            ColumnCollection)
{
int Col=0;
String
            forwardValue="";
for(int
            i=0;i<ColumnCollection.Count;i++)
{
Col++;
if(Col == 4)
{
forwardValue+="\r\n"
            +WinForms.FIVE_TAB;
Col=0;
}
if(ColumnCollection[i].AutoIncrement==false)
{
switch
            (ColumnCollection[i].Type)
{
case
            "System.String":
forwardValue+="txt"+ColumnCollection[i].ColumnName.Replace("
            ","_")+".Text, ";
break;
case
            "System.Int16":
forwardValue+="Int16.Parse(txt"+ColumnCollection[i].ColumnName.Replace("
            ","_")+".Text), ";
break;
case
            "System.Int32":
forwardValue+="Int32.Parse(txt"+ColumnCollection[i].ColumnName.Replace("
            ","_")+".Text), ";
break;
case
            "System.Int64":
forwardValue+="Int64.Parse(txt"+ColumnCollection[i].ColumnName.Replace("
            ","_")+".Text), ";
break;
case
            "System.DateTime":
forwardValue+="DateTime.Parse(dt"+ColumnCollection[i].ColumnName.Replace("
            ","_")+".Text), ";
break;
case
            "System.Decimal":
forwardValue+="Decimal.Parse(txt"+ColumnCollection[i].ColumnName.Replace("
            ","_")+".Text), ";
break;
case
            "System.Double":
forwardValue+="Double.Parse(txt"+ColumnCollection[i].ColumnName.Replace("
            ","_")+".Text), ";
break;
case
            "System.Byte[]":
forwardValue+="Byte.Parse(txt"+ColumnCollection[i].ColumnName.Replace("
            ","_")+".Text), ";
break;
case
            "System.Single":
forwardValue+="Single.Parse(txt"+ColumnCollection[i].ColumnName.Replace("
            ","_")+".Text), ";
break;
case
            "System.Boolean":
forwardValue+="ch"+ColumnCollection[i].ColumnName.Replace("
            ","_")+".Checked, ";
break;

default:
throw new
            Exception("Type is
            unknown");
}
}
}
forwardValue=forwardValue.Remove(forwardValue.Length-1,1);

if(forwardValue.TrimEnd().LastIndexOf(",")==forwardValue.TrimEnd().Length-1)
{
forwardValue=forwardValue.TrimEnd().Remove(forwardValue.TrimEnd().Length-1,1);
}
return
            forwardValue;
}

We have now completed the Windows Forms. Now we will look at the C# objects, which are the database access layer objects.

Generating C# Objects

Why do we need C# objects? C# objects will allow Windows Forms to access and manipulate Microsoft SQL database. In other words, the C# objects' task is to provide a channel between Windows Forms and Microsoft SQL Server. This part of the project consists of:

  • Class Mapping
  • Class Functions
  • Class Instance Variables
  • Class Properties

We will see how we can parse a relational database into a C# class file. It follows the same logic as parsing to Windows Forms. The ClassMapping object has a CreateCSharp function which generates a C# class file. Here is a code snip:

public void CreateCSharp(String
            FileLocation, String Namespace)
{
ClassInstanceVariables
            instVar= new ClassInstanceVariables();
ClassProperties clsPro=
            new ClassProperties();
ClassFunctions clsFunc= new
            ClassFunctions();

for(int i=0; i<
            this.dataBaseCollection[0].Tables.Count;
            i++)
{
this.className=this.dataBaseCollection[0].Tables[i].TableName;

///check
            whether file location has \ at the end of filelocation
            string
///if it is not there, append the string with "\"
            character

FileLocation=FileLocation.Trim();
if(FileLocation.Substring(FileLocation.Length-1,1)!="\\")
FileLocation+="\\";

fs
            = new FileStream(FileLocation+"cls"+this.className.Replace(" ",
            "_")+".cs", FileMode.Create, FileAccess.Write);

streamWrite =
            new StreamWriter(fs);

StringBuilder insertFunction= new
            StringBuilder("public void Insert(");
StringBuilder
            updateFunction= new StringBuilder("public void
            Update(");
StringBuilder insertStatement= new
            StringBuilder("INSERT INTO "+"["+this.className+"]
            (");
StringBuilder updateStatement= new StringBuilder("UPDATE
            "+"["+this.className+"] SET ");
StringBuilder
            insertStatementValue= new StringBuilder();
ArrayList PrimaryKeys
            = new ArrayList();
ArrayList AutoIncrement = new
            ArrayList();

streamWrite.WriteLine("using
            System;");
streamWrite.WriteLine("using
            System.Text;");
streamWrite.WriteLine("using
            System.Data;");
streamWrite.WriteLine("using
            System.Data.SqlClient;");
streamWrite.WriteLine();

streamWrite.WriteLine("namespace
            "+Namespace);
streamWrite.WriteLine("{");
streamWrite.WriteLine(ONE_TAB+"public
            class cls"+this.className.Replace(" ",
            "_"));
streamWrite.WriteLine(ONE_TAB+"{");

//
            CreateInstanceVariables();
instVar.WriteInstanceVariables(ref
            streamWrite,this.dataBaseCollection[0].Tables[i].Columns);

streamWrite.WriteLine(TWO_TAB+"public
            cls"+this.className.Replace(" ", "_")+"(String
            ConnectionString)");
streamWrite.WriteLine(TWO_TAB+"{");
streamWrite.WriteLine(THREE_TAB+"this.connString=ConnectionString;");
streamWrite.WriteLine(TWO_TAB+"}");
streamWrite.WriteLine();

clsPro.WriteProperties(refstreamWrite,this.dataBaseCollection[0].Tables[i].Columns,
ref insertFunction,ref insertStatement, ref
            insertStatementValue,
ref updateFunction,ref updateStatement, ref
            PrimaryKeys,
ref
            AutoIncrement);

if(clsPro.PrimaryKeyCount>0)
{
updateFunction.Remove(updateFunction.Length-2,2);
updateStatement.Remove(updateStatement.Length-2,2);
updateFunction.Append(")");
clsFunc.WriteUpdateFunction(ref
            streamWrite,updateFunction,updateStatement,PrimaryKeys
            );
}
insertFunction.Remove(insertFunction.Length-2,2);

insertStatement.Remove(insertStatement.Length-2,2);

insertStatementValue.Remove(insertStatementValue.Length-(insertStatementValue.Length-
insertStatementValue.ToString().LastIndexOf(",")),
insertStatementValue.Length-insertStatementValue.ToString().LastIndexOf(","));
insertFunction.Append(")");

clsFunc.WriteInsertFunction(ref
            streamWrite,insertFunction,
            insertStatement,insertStatementValue);
clsFunc.WriteReportFunction(ref
            streamWrite);
streamWrite.WriteLine(ONE_TAB+"}");
streamWrite.WriteLine("}");

//
            Close the writer and underlying file.
streamWrite.Close();
}

Let me explain the following variables:

  • UpdateFunction
  • UpdateStatement
  • InsertStatementValue
  • InsertFunction
  • insertStatement

UpdateFunction Variable

This variable holds information for generating the following output:

public void Update(System.Int32 Contract_ID, System.Int32 Supplier_Product_ID, System.Decimal Contract_Price)

UpdateStatement Variable

This variable holds information for generating the following code: for example,

Customer is the table name here:

UPDATE [Contract_Product] SET
            [Contract_ID]="+Contract_ID+",
            [Supplier_Product_ID]="+Supplier_Product_ID+",
[Contract_Price]="+Contract_Price+"";
" WHERE
            [Contract_ID]="+Contract_ID+" AND "+"
            [Supplier_Product_ID]="+Supplier_Product_ID;

We put class names into "[ ]" because some table names might contain spaces.

InsertFunction Variable

This variable holds information for generating the following code: For example:

public void Insert(System.Int32 Contract_ID, System.Int32 Supplier_Product_ID, System.Decimal Contract_Price)

insertStatement Variable

This variable holds information for generating the following code: For example:

SQLStatement="INSERT INTO [Contract_Product] (Contract_ID, Supplier_Product_ID, Contract_Price)";

InsertStatementValue Variable

This variable holds information for generating code similar to the following:

strInsert.Append(SQLStatement+" VALUES
            (");
strInsert.Append(Contract_ID+",");
strInsert.Append(Supplier_Product_ID+",");
strInsert.Append(Contract_Price+")");

When we update the database, we do so based on primary keys. There is no way to update the database based on some criteria unless there is one more primary key. Therefore the following code makes sure that the C# object will have update function if there is one or more primary key in the table:

if(clsPro.PrimaryKeyCount>0)
{
}

Otherwise, the C# object will not have any update function.

In this function, we call the following code to generate instance variables in C# object:

instVar.WriteInstanceVariables(ref streamWrite,this.dataBaseCollection[0].Tables[i].Columns);

public void WriteInstanceVariables(ref StreamWriter streamWrite, Columns ColumnCollection)
{
for(int i=0;i<ColumnCollection.Count;i++)
{
string st=ColumnCollection[i].Type;
streamWrite.Write(ClassMapping.TWO_TAB+"private ");
streamWrite.Write(st);
streamWrite.WriteLine(" _"+ColumnCollection[i].ColumnName.Replace(" ","_")+";");
}
streamWrite.WriteLine(ClassMapping.TWO_TAB+"private String connString;");
streamWrite.WriteLine();
}

The above code will generate output similar to the following:

private System.Int32 _Contract_ID;
private System.Int32 _Supplier_Product_ID;
private System.Decimal _Contract_Price;

As you see, instance variables start with underscore character "_" plus field name.

After generating instance variables, it is time to generate object properties. The following code will generate the necessary output:

clsPro.WriteProperties(ref streamWrite,this.dataBaseCollection[0].Tables[i].Columns,
ref insertFunction,ref insertStatement, ref insertStatementValue,
ref updateFunction,ref updateStatement, ref PrimaryKeys, ref AutoIncrement);

public void WriteProperties(ref StreamWriter streamWrite, Columns ColumnCollection,
ref StringBuilder insertFunction,ref StringBuilder insertStatement,
ref StringBuilder insertStatementValue, ref StringBuilder updateFunction,
ref StringBuilder updateStatement, ref ArrayList PrimaryKeys, ref ArrayList AutoIncrement )
{
int CountKey=0;
for(int i=0;i<ColumnCollection.Count;i++)
{
string st=ColumnCollection[i].Type;
streamWrite.Write(ClassMapping.TWO_TAB+"public ");
streamWrite.Write(st);
streamWrite.WriteLine(" "+ColumnCollection[i].ColumnName);
streamWrite.WriteLine(ClassMapping.TWO_TAB+"{");
streamWrite.WriteLine(ClassMapping.THREE_TAB+"get {return this._"+
ColumnCollection[i].ColumnName.Replace(" ","_")+ ";}");
streamWrite.WriteLine(ClassMapping.THREE_TAB+"set { _"+
ColumnCollection[i].ColumnName.Replace(" ", "_")+"=value; }");

streamWrite.WriteLine(ClassMapping.TWO_TAB+"}");

if (ColumnCollection[i].PrimaryKey==true)
{
CountKey++;
PrimaryKeys.Add(ColumnCollection[i].ColumnName);
}

updateFunction.Append(st +" "+ColumnCollection[i].ColumnName.Replace(" ", "_")+", ");
updateStatement.Append("["+ColumnCollection[i].ColumnName+"]="+
AppendUpdateValues(st,ColumnCollection[i].ColumnName.Replace(" ", "_")));

if(0==CountKey)
{
this.CountKeys=0;
}
else
{
this.CountKeys=CountKey;
}
if(ColumnCollection[i].AutoIncrement==true)
{
AutoIncrement.Add(ColumnCollection[i].ColumnName);
}
else
{
insertFunction.Append(st +" "+ColumnCollection[i].ColumnName.Replace(" ", "_")+", ");
insertStatement.Append(ColumnCollection[i].ColumnName.Replace(" ", "_")+", ");
insertStatementValue.Append(AppendInsertValues(st,
ColumnCollection[i].ColumnName.Replace(" ", "_")));
}
}
}

The above code will generate output similar to the following:

public System.Int32 Contract_ID
{
get
            {return this._Contract_ID;}
set { _Contract_ID=value;
            }
}
public System.Int32 Supplier_Product_ID
{
get
            {return this._Supplier_Product_ID;}
set {
            _Supplier_Product_ID=value; }
}
public System.Decimal
            Contract_Price
{
get {return this._Contract_Price;}
set {
            _Contract_Price=value; }
}

I believe the following code requires explanation:

if(ColumnCollection[i].AutoIncrement==true)
{
AutoIncrement.Add(ColumnCollection[i].ColumnName);
}
else
{
insertFunction.Append(st
            +" "+ColumnCollection[i].ColumnName.Replace(" ", "_")+",
            ");
insertStatement.Append(ColumnCollection[i].ColumnName.Replace("
            ", "_")+",
            ");
insertStatementValue.Append(AppendInsertValues(st,
ColumnCollection[i].ColumnName.Replace("
            ", "_")));

The above code ensures that if the field is autoincremented, the insert function will not have this field. If the field is not autoincremented, the insert function will have this field. We need this so that we do not attempt to insert data to an autoincremented field (doing so would return a database error).

It is time to generate the data access functions. The following code can generate the insert function:

public void WriteInsertFunction(ref
            StreamWriter streamWrite, StringBuilder strInsertFunction,
StringBuilder insertStatement,StringBuilder
            insertStatementValues)
{
streamWrite.WriteLine(ClassMapping.TWO_TAB+strInsertFunction.ToString());
streamWrite.WriteLine(ClassMapping.TWO_TAB+"{");

streamWrite.WriteLine(ClassMapping.THREE_TAB+"SqlConnection
            conn= new
            SqlConnection(this.connString);");
streamWrite.WriteLine(ClassMapping.THREE_TAB+"SqlCommand
            comm= new
            SqlCommand();");
streamWrite.WriteLine();
streamWrite.WriteLine(ClassMapping.THREE_TAB+"StringBuilder
            strInsert=new
            StringBuilder();");
streamWrite.WriteLine(ClassMapping.THREE_TAB+"String
            SQLStatement;");
streamWrite.WriteLine();
streamWrite.WriteLine(ClassMapping.THREE_TAB+"SQLStatement=\""+
            insertStatement.ToString()+")\";");
streamWrite.WriteLine(ClassMapping.THREE_TAB+"strInsert.Append(SQLStatement"+"+\"
            VALUES
            (\");");
streamWrite.WriteLine(ClassMapping.THREE_TAB+insertStatementValues.ToString()+")\");");

streamWrite.WriteLine();
streamWrite.WriteLine(ClassMapping.THREE_TAB+"try");
streamWrite.WriteLine(ClassMapping.THREE_TAB+"{");
streamWrite.WriteLine(ClassMapping.FOUR_TAB+"comm.CommandText=strInsert.ToString();");
streamWrite.WriteLine(ClassMapping.FOUR_TAB+"comm.ExecuteNonQuery();");
streamWrite.WriteLine(ClassMapping.THREE_TAB+"}");
streamWrite.WriteLine(ClassMapping.THREE_TAB+"catch(Exception
            ex)");
streamWrite.WriteLine(ClassMapping.THREE_TAB+"{");
streamWrite.WriteLine(ClassMapping.FOUR_TAB+"throw
            ex;");
streamWrite.WriteLine(ClassMapping.THREE_TAB+"}");
streamWrite.WriteLine(ClassMapping.TWO_TAB+"}");
}

The above code will generate output similar to the following:

public void Insert(System.Int32 Contract_ID,
            System.Int32 Supplier_Product_ID, System.Decimal
            Contract_Price)
{
SqlConnection conn= new
            SqlConnection(this.connString);
SqlCommand comm= new
            SqlCommand();

StringBuilder strInsert=new
            StringBuilder();
String SQLStatement;

SQLStatement="INSERT
            INTO [Contract_Product] (Contract_ID, Supplier_Product_ID,
            Contract_Price)";

strInsert.Append(SQLStatement+" VALUES
            (");

strInsert.Append(Contract_ID+",");
strInsert.Append(Supplier_Product_ID+",");
strInsert.Append(Contract_Price+")");

try
{
comm.CommandText=strInsert.ToString();
comm.ExecuteNonQuery();
}
catch(Exception
            ex)
{
throw ex;
}
}

So far, we have developed Windows Forms and C# objects with functions, properties, instance variables and so on. Now let's look at the sample application that ties everything together.

The Sample Application

I have developed a small application in Visual Studio .NET RC environment. In this solution, there are three different projects:

  • Project for custom objects
  • Project for Generating Windows Forms
  • Project for Generating C# Objects.

The solution file is in the DataWinForms folder. After starting the application, you must configure the following for your own system:

  • Server Name: the name of SQL Server or IP number
  • Database Name: the name of Database in the SQL server
  • UserName:
  • Password:
  • File Location: the folder such as c:\project, c:\temp
  • Namespace: the namespace for Windows Forms
  • Data Layer Namespace: the namespace for C# objects

After configuring the necessary information, click on "Get Tables". This will provide data access, parsing the database into our custom objects. When this is done, it will let you know about it. You can see at the left side of the screen that the database tables are already parsed into custom objects.

When you click, for example, on Order Table, you will see three fields: Order_ID, Contract_ID, and Order_Date. When you click on these fields, you will see the properties of these fields such as data type, length, PrimaryKey and so on.

After parsing database into custom objects, click on "Generate Windows Forms", this will generate Windows Forms, and clicking on "Generate C# Class Files" will generate C# objects for you. I generated Windows Forms and C# objects in to C:\temp5\ folder. The following figure show how it might look:

Now, after generating Windows Forms and C# objects, we can add these files to our Visual Studio .NET projects.

How can we do that? Let's assume that we have a new C# Windows Application project in Visual Studio .NET. To add these generated files into your project, go to Project | Add Existing Item. It might look like the following:

Select all the files, and click on Open. You will see all files in the project Solution Explorer window. Now what? We have to create menus for the Windows Forms. We do not need to do so for the C# objects because they are already connected to the Windows Forms. That's all. Have fun!

Conclusion

We have developed software which generates Windows Forms and C# objects, by using .NET objects, custom objects, and Microsoft SQL Server 2000. This application can be extended for other databases such as Oracle, MySQL, DB2 and so on.

If you'd like to discuss a broader range of C#Today topics and issues, or if you have any C# related questions you need answers to, please visit our online resource centres at : http://p2p.wrox.com/list.asp?list=csharptoday_discuss&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq and http://p2p.wrox.com/list.asp?list=c_sharp&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq


Mapping Windows Forms to A Relational Database Part 1

Mapping Windows Forms to A Relational Database Part 1
http://www.csharptoday.com/content.asp?id=1690
Article Rating: 3
Published on January 7, 2002
Abstract
In this article, Levent Camlibel detailes the custom mapping tool for Windows Forms and C# objects he has developed, to reduce time and effort in mapping a SQL Server database. The application names variables, parses values to right data types, and generates basic functions in seconds - an effort that might otherwise takes days to code manually. In this, the first part of the article, we only deal with constructing custom objects, getting database schema information, and applying it to the custom objects. In the second part tomorrow we deal with generating the Windows Forms and C# objects.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

Companies have been trying to reduce their software development time and cost for a long time. I have been thinking of how costs can be reduced, and have developed a custom mapping tool for Windows Forms and C# objects to reduce time and effort in mapping a SQL Server database. Conveniently naming variables, parsing values to right data types, and generating basic functions might take days, or even weeks to do if you consider tens of tables, with hundreds of fields in a database. The application we develop in this article intends to reduce this effort to seconds.

  • This first part of the article will only deal with constructing custom objects, getting database schema information, and applying it to the custom objects.
  • The second part of the article will deal with generating the Windows Forms and C# objects.

System Requirements

This software requires Visual Studio .NET Beta 2 to compile, and Microsoft SQL Server or MSDE for database access.

Introduction

After connecting to the database, we will retrieve the schema of the database by using the FillSchema function. After retrieving the database schema, we parse it into custom objects that will be explained later. We will then have all the necessary information for generating Windows Forms and C# objects using these custom objects, and software developers can focus on business rules and other subjects more important than naming variables, parsing them to right data types, and writing basic database related functions.

Component Design

Introduction to Custom Objects

I have developed custom objects to hold information about databases, tables, and fields (columns). The systematic schema of custom objects is as follows:

The Database object can be considered as a DataSet object, the Table object can be considered as a DataTable and the Column object can be considered as a DataColumn in the .NET framework. Databases, Tables and Columns objects can be considered as Collections of the above objects. Let's look at the custom objects in detail.

Databases Custom Object

The Databases object is a collection of Database objects. The Databases object can be considered as multiple DataSets in one object. You can do the same thing with the ArrayList object:

ArrayList arrDatabases= new
            Arraylist();
DataSet myFirstDataSet= new
            DataSet();
arrDatabases.Add(myFirstDataSet);

So you can add a number of DataSet objects into an ArrayList. However, I have used the Databases object for this purpose. For convenience, we have extensively used the Indexer feature of C# in these custom objects. This object supports two properties:

  • Count Property: this holds the number of Database objects in the Databases object.
  • Index Property: this holds the index number of a Database object when the user adds a new Database object to the Databases object. This property does not persist an index number for each Database object. Instead it accesses to the ArrDatabase variable that holds the index number.

Let's see what indexer looks like:

public Database this[String DataBaseName]

The single line of code above provides access to the ArrayList by using the DataBaseName string value. To find the correct object, we have to iterate through the ArrayList from first item to last item.

public Database this[int Index]

The above line also provides access to the indexer through the Index value. This way of accessing data provides a performance boost. The second way of accessing data is more powerful and fast. Because of this, I have created an Index property to access the Database object. This raises the point of whether to use the Index or ArrayList iteration to find the right span class='codeintext'>Database object. Using Index has a constant order of complexity, whereas iterating through ArrayList has "n" order of complexity in the worst case. A constant order of complexity is always better.

using System;
using
            System.Collections;

namespace
            Mapping.CustomObjects
{

public class Databases
{
private ArrayList arrDatabase;
private Int32
            count;
private Int32 index;

public
            Databases()
{
arrDatabase=new
            ArrayList();
count=0;
}

public Database this[String
            DataBaseName]
{
get
{
foreach(Database val in
            arrDatabase)
{
if(val.DataBaseName.ToLower()==DataBaseName.ToLower())
return
            val;
}

return
            null;
}
set
{
Add(DataBaseName,value);
}
}

public
            Database this[int Index]
{
get
{
if(Index <0 ||
            Index>arrDatabase.Count)
throw new ArgumentOutOfRangeException
            ("Index is out of boundary");
return
            ((Database)this.arrDatabase[Index]);
}
}

public void
            Add(String DataBaseName, Database dataBase)
{
dataBase.DataBaseName=DataBaseName;
arrDatabase.Add(dataBase);
count++;
this.index=count-1;
}
public
            void Add(String DataBaseName)
{
Database data=new
            Database();
Add(DataBaseName,data);

}

public Int32
            Count
{
get{return this.count;}
}
public Int32
            Index
{
get{return
            this.index;}
}

}
}

There are many ways to add Database objects to the Databases object.

  • One is to use the Indexer feature
  • Another is to use the Add function of this object.

Let's see some examples:

Databases DatabaseCollection= new Databases();
Database database =DatabaseCollection["Database1"]

or

DatabaseCollection.Add("Database1");

or

Database database= new
            Database();
DatabaseCollection.Add("Database1, database);

The code above shows the ways of adding Database objects to the Databases custom object. First, instantiate the Databases object and then use the Add function to add a Database object. You can either provide just the name of the database object, or provide the name and instance of the Database object.

Whenever you just use the name of a Database object, the Add function in Databases object instantiates a new Database object and calls the function of:

public void Add(String DataBaseName, Database dataBase)

This function adds the Database object to arrDatabase (ArrayList), and increments Count by one, the number of Database objects in the Databases object. Moreover, it assigns the Index number of the latest Database object in this collection.

int
            DatabaseIndex=databases.Index;
databases =new
            Databases();
databases.Add(this.txtDataBase.Text);

int
            DatabaseIndex=databases.Index;

Database Custom Object

The Database custom object is comparable to a DataSet object in the .NET framework. This object has only two properties: DataBaseName and the Tables object. We will look at the Tables object in detail later. A user can add a Table by using the AddTable function that receives a TableName argument. Database objects can be considered as containers for Tables object.

using System;
using
            System.Collections;

namespace
            Mapping.CustomObjects
{
/// <summary>
/// Summary
            description for Database.
/// </summary>
public class
            Database
{
private String dataBaseName;
private Tables
            tables;
public Database()
{
tables= new
            Tables();
}
public String DataBaseName
{
get{return
            this.dataBaseName;}
set{this.dataBaseName=value;}
}
public
            Tables Tables
{
get{return this.tables;}
}
public void
            AddTable(String
            TableName)
{
this.tables.Add(TableName);
}
}
}

Tables Custom Object

The Tables object can be considered as a container for the Table object. This object has two properties, Count (the number of Table objects), and Index for the last Table object added. The Tables object also supports Indexers. The usage of Indexers in Databases is also the same for this object, returning and / or setting Table custom objects. You can access Table objects by using either Index or TableName. If you use TableName, you should iterate ArrayList to find the given value. However, if you use the Index property, you can directly access this Table object in the ArrayList without iterating all the values in it.

using System;
using
            System.Collections;

namespace
            Mapping.CustomObjects
{
/// <summary>
/// Summary
            description for Tables.
/// </summary>
public class
            Tables
{
private ArrayList arrTables;
private int
            count;
private Int32 index;

public Tables()
{
arrTables= new ArrayList();

}
public Table this[int
            Index]
{
get
{
if(Index <0 ||
            Index>arrTables.Count)
throw new ArgumentOutOfRangeException
            ("Index is out of boundary");
return
            ((Table)arrTables[Index]);
}
set
{
Add(value);
}
}
public
            Table this[String TableName]
{
get
{
foreach(Table val
            in
            arrTables)
{
if(val.TableName.ToLower()==TableName.ToLower())
return
            val;
}
return null;
}
set
{
Add(TableName,value);
}
}
public void Add(Table
            table)
{
if(table.TableName.Trim().Length==0)
throw new
            Exception("Please First Enter Table
            Name");

Add(table.TableName, table);
}
public void
            Add(String TableName)
{
Table tab=new
            Table();
Add(TableName,tab);
}
public void Add(String
            TableName, Table
            table)
{
table.TableName=TableName;
arrTables.Add(table);
count++;
this.index=count-1;
}
public
            Int32 Count
{
get{return this.count;}
}
public Int32
            Index
{
get{return this.index;}
}

}
}

Table Custom Object

The Table custom object is like a relational database table. This custom object holds TableID and TableName properties. The TableID property is used to hold a TableID value received from SQL Server. Therefore, it is a unique value. This object also holds a Columns object that is like a collection of Column objects.

using System;
using
            System.Collections;

namespace
            Mapping.CustomObjects
{
/// <summary>
/// Summary
            description for Table.
/// </summary>
public class
            Table
{
private String tableName;
private int
            tableID;
private Columns column;
public
            Table()
{
column= new Columns();
}
public String
            TableName
{
get{return
            this.tableName;}
set{this.tableName=value;}
}
public Int32
            TableID
{
get{return
            this.tableID;}
set{this.tableID=value;}
}
public Columns
            Columns
{
get{return this.column;}
}
public void
            AddColumn(String
            ColumnName)
{
this.Columns.Add(ColumnName);
}
}
}

Columns Custom Object

This object also uses the Indexer, which returns a Column object. This object has properties of Index and Count. This object also uses the Add function with which user can add Column objects by name.

using System;
using
            System.Collections;

namespace
            Mapping.CustomObjects
{
/// <summary>
/// Summary
            description for Columns.
/// </summary>
public class
            Columns
{
private ArrayList arrColumn;
private Int32
            count;
private Int32 index;

public
            Columns()
{
arrColumn= new ArrayList();
}
public Column
            this[String ColumnName]
{
get
{
foreach(Column val in
            arrColumn)
{
if(val.ColumnName.ToLower()==ColumnName.ToLower())
return
            val;
}
return null;
}
set
{
Add(ColumnName,
            value);
}
}
public Column this[int
            Index]
{
get
{
if(Index <0 ||
            Index>arrColumn.Count)
throw new ArgumentOutOfRangeException
            ("Index is out of boundary");
return
            ((Column)arrColumn[Index]);
}
set
{
Add(value);
}

}
public
            void Add(Column
            column)
{
if(column.ColumnName.Trim().Length==0)
throw new
            Exception("Please First Enter Column Name
            ");
Add(column.ColumnName,column);
}
public void Add(String
            ColumnName, Column
            column)
{
column.ColumnName=ColumnName;
arrColumn.Add(column);
count++;
this.index=count-1;
}
public
            void Add(String ColumnName)
{
Column col= new
            Column();
Add(ColumnName, col);
}

public Int32
            Count
{
get {return this.count;}
}
public Int32
            Index
{
get{return this.index;}
}
}
}

Column Custom Object

The Column object can be compared to a field in a relational database, and is the most important object among these custom objects. This object is the main source for parsing database tables into custom objects. The Column object has the following properties:

  • ColumnName: returns / sets the field name in a Database table.
  • IsNullAble: returns / sets whether the column is nullable or not
  • Type: returns / sets data type of column (field)
  • Length: returns / sets the length of field
  • IsPrimaryKey: returns / sets whether the column is primary key or not
  • AutoIncrement: returns / sets whether the column is auto increment or not.

using System;
using
            System.Collections;

namespace
            Mapping.CustomObjects
{
/// <summary>
/// Summary
            description for Column.
/// </summary>
public class
            Column
{
private String columnName;
private bool
            isNullAble;
private String dataType;
private Int32
            length;
private bool isPrimaryKey;
private bool
            autoIncrement;
public Column()
{

}
public String
            ColumnName
{
get{return
            this.columnName;}
set{this.columnName=value;}
}
public
            Int32 Length
{
get{return
            this.length;}
set{this.length=value;}
}
public String
            Type
{
get{return
            this.dataType;}
set{this.dataType=value;}
}
public bool
            IsNullAble
{
get{return
            this.isNullAble;}
set{this.isNullAble=value;}
}
public bool
            PrimaryKey
{
get{return
            this.isPrimaryKey;}
set{this.isPrimaryKey=value;}
}
public
            bool AutoIncrement
{
get{return
            this.autoIncrement;}
set{this.autoIncrement=value;}
}
}
}

The design of custom objects is like the design of a database with a few properties. It is possible to extend these custom objects. I felt I do not need any more properties or functions, but you can extend them as you wish. For example you may want to add a new property to the Column custom object like the IncrementSeed property.

DataSet Object vs Custom Objects

You may wonder why we have developed custom objects to map a relational database, as opposed to the DataSet object. While it is true the application can be ported to use DataSet without much effort, I felt it valid to learn about how to handle data.

There is also a small performance penalty over using only the DataSet with this method. The GetTables function returns a SqlDataReader object that holds table names of the relational database. When we iterate through this SqlDataReader, the GetFields function is called to return fields with necessary information. This is the step that generates a performance penalty. The GetFields function returns a DataTable object that is iterated through, and inserts the necessary information into, custom objects. Instead of using custom objects, you can populate (merge) myDataSet with a new table each time SqlDataReader reads data from database.

while(reader.Read())
{
newNode = new
            TreeNode(reader["Name"].ToString());
newNode.Tag=reader["ID"].ToString();
root.Nodes.Add(newNode);

DataTable
            columnTable=
            data.GetFields("["+reader["Name"].ToString()+"]");
myDataSet.Merge(columnTable);
}

The above code will handle the mapping of any database.

Parsing A Relational Database to Custom Objects

To parse a relational database, we first need to access and get the table information from the database. The following code returns a SqlDataReader that has the table names and TableIDs in the database.

public SqlDataReader
            GetTables()
{
SqlConnection conn= new
            SqlConnection(this.connectionString);
conn.Open();
string
            Query = " select name, Id from where type='U' and name <>
            'dtproperties' order by name ";

SqlCommand comm= new
            SqlCommand(Query,conn);

SqlDataReader
            reader=comm.ExecuteReader(CommandBehavior.CloseConnection);
return
            reader;
}

There is an alternative way to access tables; here is the code (which is not in the project code):

public SqlDataReader
            GetTables()
{
SqlConnection conn= new
            SqlConnection(this.connectionString);
SqlDataReader
            reader;
try
{
conn.Open();
SqlCommand comm= new
            SqlCommand("sp_tables"
            ,conn);
comm.CommandType=CommandType.StoredProcedure;

SqlParameter
            paramtable_type=new
            SqlParameter("@table_type",SqlDbType.NVarChar);
paramtable_type.Value="'Table'";
comm.Parameters.Add(paramtable_type);

reader=comm.ExecuteReader(CommandBehavior.CloseConnection);
}
catch(SqlException
            ex)
{
throw ex;
}

return reader;
}

We can use either approach to get table names from Microsoft SQL Server. The code above returns a SqlDataReader object that we have to iterate through, assigning database, tables, and fields to custom objects. Let's do it.

The following code (in frmMapping.cs) receives table names from SqlDataReader. While we iterate this reader, we add a new node to the TreeView on the form to show the table names. After that, we can now add this table to our databases custom object by using AddTable function.

SqlDataReader
            reader=data.GetTables();

while(reader.Read())
{
newNode
            = new
            TreeNode(reader["Name"].ToString());
databases[DatabaseIndex].AddTable(reader["Name"].ToString());

int
            TableIndex=databases[DatabaseIndex].Tables.Index;

newNode.Tag=reader["ID"].ToString();
databases[DatabaseIndex].Tables[TableIndex].TableID=Int32.Parse(reader["ID"].ToString());
root.Nodes.Add(newNode);

After that, we can get the Index property of the database object, which refers to the last table added. Moreover, we can assign a value to the TableID property of our Table custom object by using reader["ID"].ToString().

Now that we have the necessary table information, we can get field information from the database for each table. The following code returns a DataTable with schema information:

DataTable columnTable= data.GetFields("["+reader["Name"].ToString()+"]");

The following code returns a DataTable with table mapping information:

public DataTable GetFields(String
            TableName)
{
SqlConnection conn= new
            SqlConnection(this.connectionString);
DataSet myDataSet=new
            DataSet();
conn.Open();

SqlDataAdapter comm= new
            SqlDataAdapter("SELECT * from
            "+TableName,conn);

comm.FillSchema(myDataSet,
            SchemaType.Mapped,TableName);

return
            myDataSet.Tables[0];
}

Microsoft's documentation states that a FillSchema operation adds a DataTable to the destination DataSet. It then adds columns to the DataColumnCollection of the DataTable, and configures the following DataColumn properties, if they exist, at the data source:

  • AllowDBNull
  • AutoIncrement
  • MaxLength
  • ReadOnly
  • Unique

FillSchema also configures the PrimaryKey property.

After receiving a DataTable, we can extract the PrimaryKeys of the table by using the following code:

DataColumn[] columnKeys=columnTable.PrimaryKey;

The following code iterates the database fields. As you may remember, we can use the Index property of our custom objects to directly access them without iterating the ArrayList.

foreach(DataColumn col in columnTable.Columns)
{
TreeNode colNode;
TreeNode FieldNode= new TreeNode(col.ColumnName);

We can now add a field to the custom object by using:

databases[DatabaseIndex].Tables[TableIndex].AddColumn(col.ColumnName)

After adding columns to the Tables custom object, we can get the Index value for that field. We can access all information by using Index without the iteration, giving us a performance gain (compensating for the performance penalty discussed earlier!)

During this process, we need to figure out whether this column is in the primary keys or not. If it is, the isInKeys Boolean variable will have a value of true. If true, we can assign true to the PrimaryKey property of the Column custom object. Otherwise, it will be assigned false:

int
            ColumnIndex=databases[DatabaseIndex].Tables[TableIndex].Columns.Index;

databases[DatabaseIndex].Tables[TableIndex].Columns[ColumnIndex].Type=col.DataType.ToString();
databases[DatabaseIndex].Tables[TableIndex].Columns[ColumnIndex].IsNullAble=col.AllowDBNull;
databases[DatabaseIndex].Tales[TableIndex].Columns[ColumnIndex].Length=col.MaxLength;
databases[DatabaseIndex].Tables[TableIndex].Columns[ColumnIndex].AutoIncrement=col.AutoIncrement;

bool
            isInKeys=false;
foreach(DataColumn datCol in
            columnKeys)
{
if(datCol.ColumnName==col.ColumnName)
{
isInKeys=true;
break;
}
}

if(isInKeys==true)
{
databases[txtDataBase.Text.Trim()].Tables[TableIndex].Columns[ColumnIndex].PrimaryKey=true;

}
else
{
databases[txtDataBase.Text.Trim()].Tables[TableIndex].Columns[ColumnIndex].PrimaryKey=false;
}

The Sample Application

In this project group, there are three different projects:

  • DataWinFormsMappingCls (Project for custom objects)
  • DataWinForms (Project for Generating Windows Forms)
  • DataClassObjectsCls (Project for Generating C# Objects.)

So far, we have only developed DataWinFormsMappingCls; we will cover the rest of the solution tomorrow.

Any Limitations or Further Work

The software does not support variant type values. In other words, you might get a compile error if you have Byte[] type variables in your software-generated SQL statements. Other than that, it will parse and generate necessary code for you. Another limitation is that Databases, Tables, and Columns custom objects do not support the IEnumerator interface. Because of that, we are not able to use foreach() to iterate custom objects. However, this should not be difficult to implement.

Conclusion

We have used .NET and custom objects to map a Microsoft SQL server database. This part of the article has only dealt with constructing custom objects and pumping database schema information into them. I have tried to discuss the custom objects in detail because of some arguments about why they exist. So far we have worked on how to parse relational databases to custom objects. During this process, we have extracted the necessary information from a relational database, and assigned these values to custom objects. The next step is to work on code that generates Windows Forms and C# objects by using these custom objects.

Appendix A: .NET Objects Used

I have extensively used a few .NET objects, StreamWriter, FileStream and Directory, in this particular article. I will talk about these briefly in this Appendix.

Directory

The Directory class (in the System.IO Namespace) is used to perform various operations on directories such as Create, Move, Delete, and Enumerate. The Directory class provides static methods, as follows, in order to deal with directories on disk. In other words, all methods of the Directory class are static and can therefore be called without having an instance of a directory.

Directory Class (some static methods)

  • CreateDirectory: Creates directory as specified by path.
  • Delete: Deletes a directory and its contents.
  • Exists: Determines whether the given path refers to an existing directory on disk.
  • GetDirectories : Gets the names of subdirectories in the specified directory.
  • GetFiles: Returns the names of files in the specified directory.
  • GetLogicalDrives: Retrieves the names of the logical drives on this computer.
  • GetParent: Retrieves the parent directory of the specified path, including both absolute and relative paths.

FileStream

The FileStream class, constructed for reading and writing files on a file system, extends the Stream class and provides access to the standard input, output and error streams for files. It is in the System.IO namespace. Microsoft's documentation indicates that FileStream buffers input and output for better performance. The FileStream class can open a file in one of two modes, either synchronously or asynchronously; the default is to open files synchronously, but provides a constructor to open files asynchronously. The constructor is:

public FileStream(string path,
FileMode mode,
FileAccess access,
FileShare share)

FileMode, FileAccess and FileShare are enumerations that might be used in the FileStream construction.

FileMode Enumeration

This specifies how the operating system should open a file. The following modes control the file operations:

  • Append: opens the file if it exists and seeks to the end of the file, or creates a new file.
  • Create: Operating system creates a new file. If the file already exists, it will be overwritten.
  • CreateNew: Operating system creates a new file.
  • Open: specifies that the operating system should open an existing file
  • OpenOrCreate: Operating system opens a file if it exists, if not, a new file will be created.

FileAccess Enumeration

This defines constants for read, write, or read/write access to a file. You should choose Read or Write when reading or writing from a file, respectively. If you want to read and write simultaneously, you should use ReadWrite mode.

  • Read: Read access to the file.
  • ReadWrite: Read and write access to the file.
  • Write: Write access to the file.

FileShare Enumeration

This contains constants for controlling the kind of access other FileStreams can have to the same file.

  • None: Declines sharing of the current file.
  • Read: Allows subsequent opening of the file for reading.
  • ReadWrite: Allows subsequent opening of the file for reading or writing.
  • Write: Allows subsequent opening of the file for writing.

StreamWriter

This implements a TextWriter for writing characters to a stream in a particular encoding. In other words, the StreamWriter class is useful whenever you need to write character-based data. StreamWriter defaults to using an instance of UTF8Encoding unless specified otherwise. This can be changed by supplying a System.Text.Encoding object reference. The StreamWriter class is in the System.IO namespace.

Use of Objects above

When we generate a Windows Form for each table in the database, we first check the FileLocation variable that is provided by the user. The FileLocation variable holds the folder name for the generated Windows Forms. We first trim the value in the variable to make sure that there is no white space before and after the value of the FileLocation variable. The second thing to do depends upon whether the FileLocation variable has the character of "\" at the end of its value. If not, we add a "\" character to the end of the FileLocation variable's value. The third thing to do is to check if the folder already exists. If it doesn't, we create one. Before we create this folder, we might want to ask the user whether they want to create the folder or not by using a MessageBox object.

FileLocation=FileLocation.Trim();
if(FileLocation.Substring(FileLocation.Length-1,1)!="\\")
FileLocation+="\\";

if(!Directory.Exists(FileLocation))
{
Directory.CreateDirectory(FileLocation);
}

After generating a folder for Windows Forms, it is time to create one file on disk for each Windows Form in this project. The first part of the FileStream object is the path. We set this to be the value for FileLocation, plus "frm", plus className (a variable which holds the table name in a relational database), and finally the ".cs" extension. There might be some cases where the table name in a database has some whitespace amongst its words. If so, we replace the whitespace with an underscore character. The result might be frmCustomer.cs or frmOrder_Details.cs that refers to the Windows Form for the Customer table or Order Details table respectively in a database. Therefore, each database table will have its own Windows Form. I have used FileMode.Create with FileAccess.Write permission mode to create a new file overwriting the previous file if it exists.

streamFile = new FileStream(FileLocation+"frm"+className .Replace(" ", "_")+".cs",
FileMode.Create, FileAccess.Write);

After instantiating the streamFile object, we can declare streamWrite as an instance of the StreamWriter object, using streamFile as an argument. In other words, it initializes a new instance of the StreamWriter class for the specified stream, using the default encoding and buffer size. The following code show how to use StreamWriter to write to a file on disk. First, it writes a statement of "using System;" and then "using System.Data;" to the FileStream.

streamWrite = new StreamWriter(streamFile);

streamWrite.WriteLine("using System;");
streamWrite.WriteLine("using System.Data;");

C#Today - © 2002 WroxPress Limited


Owner Drawn Menus in Visual Studio .NET Using C#

Owner Drawn Menus in Visual Studio .NET Using C#
http://www.csharptoday.com/content.asp?id=1747
Article Rating: 4
Published on December 28, 2001
Abstract
If you’ve ever attempted to create your own menus in a Windows Application environment, you know how hard it is to “Owner Draw?anything. The idea sounds so straightforward though, doesn’t it? HAll we want to do is subclass the menu and provide our own code that determines the look and feel/functionality! Well, if you knew C++ you could do it with a little pain, black magic and some salt. VB too, if you wanted to mess around with same ugly API’s such as a C++ developer would deal with. One would think that Microsoft would have come up with an easy way to do such a thing. We could pay some money for a third party OCX that would allow you some control over the menus, but that’s no fun if we don’t have the source code. Luckily for us, this all changes with .NET. In this article, Andrew Krowczyk shows us how easy it is to provide a slick enhanced menu system.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

If you've ever attempted to create your own menus in a Windows Application environment, you know how hard it is to "Owner Draw" anything. The idea sounds so straightforward though, doesn't it? Hey I just want to subclass the menu and provide my own code that determines the look and feel/functionality! Well, if you knew C++, you could do it with a little pain, black magic and some salt. VB? Yeah, if you wanted to mess around with ugly API's (such as a C++ developer would deal with). One would think that Microsoft would have come up with an easy way to do such a thing. I suppose that you could pay some money for a third party OCX that would allow you some control over the menus, but that's no fun if you don't have the source code. Luckily for us, this all changes with .NET. In this article we'll see how easy it is to provide a slick enhanced menu system.

In particular, I really like the look of the Menus that are used in the Visual Studio .NET IDE. I've also seen this same type of menu in the latest version of XML Spy and a few other applications. Why Microsoft didn't provide a nice way for us to flip a switch in our Windows Forms code to make our menus look the same I don't know why. I do have to say that if you develop an app and run it on Windows XP, the menus do look a bit different than normal, but what about apps when they are run on Windows 2000?

For sake of clarification, let's take a look at exactly what I'm talking about. Here we see a Visual Studio .NET menu:

By the end of the article, we'll see how to turn our basic Windows Form menu from this default style:

Into this custom style:

One thing you might notice about the second image above is that it doesn't look exactly like the Visual Studio .NET menu as pictured earlier in the article. Unfortunately Owner Drawn Menus are "tacked" on to the existing menu that is provided by the Windows Form class. Because of this, we can't exactly simulate the borderless look and shadow effect of the Visual Studio .NET IDE menus. More than likely, the Microsoft .NET team is using their own basic Menu APIs and such to make the menus look the way they do. Unfortunately they don't expose that functionality to us, so we can only come so close to simulating the IDE's menus. All in all, I think the results are pretty close.

System Requirements

This article assumes that you have some familiarity of C# and the Visual Studio .NET development environment and have produced one or two simple Windows Form applications using these technologies. This code should work equally well on Beta 2 and RC 1.

What are Owner Drawn Menus Anyway?

Well, disregarding the fact that the name gives it away, an Owner Drawn menu is a menu in which the application or "Owner" of the menu instance provides its own GUI drawing/rendering code, in contrast to letting Windows draw the menu. In essence the Owner is simply subclassing the menu and overriding the code used to create the graphical interface part of the menu.

Why would we want to do such a thing? Well, first of all, the menus that are provided in any programming language for Windows don't give you much of anything when it comes to menu look/appearances. So, if you have something special in mind for the look and feel of your menus then Owner Drawing is the only way to go.

What about Windows XP? Windows XP "Windows" drawn menus look different to Windows 2000 menus, but they are still just standard menus, and we really don't have much control over them.

For example, here is what a simple menu looks like on a Windows XP system with the Blue XP Theme, and the same menu on a Windows 2000 machine:

So what is the point of discussing XP vs. Win2K? The point is that the Menu drawing code is very much an Operating System procedure. In XP, the OS draws menus differently to Win2K by default (because of the theme like nature of XP). Our owner drawn menus will work in either Operating System, as we will be taking control of the drawing mechanism.

So now that we've decided that we'd like to take a stab at creating Owner Drawn menus, let's see what we need to do.

How does .NET Help Us?

By now you are all probably thinking that gee, .NET must provide a way to Owner Draw a menu. Well, yes it does. This is a factor of a couple things. First off, .NET gives us some great namespaces with methods that we can use to draw things by ourselves. For instance, we have the System.Drawing and System.Drawing.Text namespaces that give us control of many functions that allow us to actually draw our menus.

We are also allowed to use great Object Oriented Programming (OOP) techniques such as Polymorphism (to override method definitions) and Inheritance. Although these are more a byproduct of the programming languages used, rather than the .NET Framework.

Now that we know that there are a few things in .NET that can help us, let's see how that translates into the code used in our menu namespace DotNetMenu.

DotNetMenuItem Class

There are a few different bits of code in the DotNetMenu namespace that are of interest to us, but since we are focusing on the Owner Drawn implementation part, we'll skip over a few of the other methods included in the class, as they don't really have a huge importance as to the overall understanding of the class. Ok, so let's begin!

When we take a look at the DotNetMenu namespace, we notice that there are 2 classes defined within the code - DotNetMenuItem, and ThemeColors. As we might expect, the DotNetMenuItem is the class that inherits and overrides the MenuItem class provided by the .NET framework. The ThemeColors class is just something that is used to get the correct colors for the menu structure when different themes are used in Windows. This is a somewhat hacked implementation as there isn't a method that allows us to figure out what colors to really use for the Owner Drawn menu attributes based on a particular theme, so in this case we just give a best guess instead of hardcoding 1 color combination for the whole lot of menus. If anyone can think of a better way to get this color scheme information, please do email me with your idea! That being said, we aren't going to focus much on the ThemeColors class, we'll just assume that it's a class that gives us the correct RGB color values for such items as Menu Highlighting and such.

For the sake of brevity, in some cases, we'll leave out some of the cluttering code in the listing displayed within the article. Of course all of the code is included in the sample download application, and organized for clarity!

The DotNetMenuItem class in its basic form, inherits from the MenuItem class which is a member of the System.Windows.Forms namespace:

public class DotNetMenuItem :
            MenuItem
{

// constructors
//
            ---------------------------------------------------
public
            DotNetMenuItem(string name, EventHandler handler, Shortcut shortcut)
            : this(name, handler)
{
this.Shortcut =
            shortcut;
}

public DotNetMenuItem(string name,
            EventHandler handler) : base(name, handler)
{
//set the owner
            draw flag equal to true
//since we are going to owner draw these
            menus
OwnerDraw = true;
}

//default
            constructor
public DotNetMenuItem() : base(){ OwnerDraw =
            true;}

When we are talking about owner drawn menus there are a few events that are very important to notice. These are the OnMeasureItem and DrawItem event handlers.

If we jog our memories related to Windows programming and message based programming you will remember that Windows programs rely on a system of messages being passed back and forth. If you want to handle those messages, you need to write a Windows Procedure that will listen to the messages that Windows is sending, and then you can react and do something when a particular message comes across. Let's say for example, a button click.

In the realm of Owner Drawn items, there are two very important Windows messages that we need to know about. These are WM_MEASUREITEM (http://msdn.microsoft.com/library/default.asp?url=/library/en-us/winui/combobox_85d9.asp&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq) and WM_DRAWITEM (http://msdn.microsoft.com/library/default.asp?url=/library/en-us/wcesdkr4/htm/_wcesdk_win32_wm_drawitem.asp&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq). Let's take a look at these methods.

OnMeasureItem

As you may remember, the WM_MEASUREITEM message is sent to the owner of an owner drawn menu before it is displayed. Basically this message allows Windows to figure out the size of the control/menu before it sends out the WM_DRAWITEM message. As you will see, the code in this method does a few things to determine the final height and with of the MeasureItemEventArgs argument coming in.

Follow along with the comments in the code snippet below to see what is going on in the code.

//override the OnMeasureItem
            method
protected override void OnMeasureItem(MeasureItemEventArgs
            e)
{
base.OnMeasureItem(e);

// measure shortcut text if
            it's there
// this takes into consideration all of the things
            that
// use menu shortcuts that need to be displayed
// in
            the actual menu text
if (Shortcut != Shortcut.None)
{
string text = "";
int key = (int)Shortcut;
int ch =
            key & 0xFF;
if (((int)Keys.Control & key) > 0)
text
            += "Ctrl+";
if (((int)Keys.Shift & key) > 0)
text +=
            "Shift+";
if (((int)Keys.Alt & key) > 0)
text +=
            "Alt+";

if (ch >= (int)Shortcut.F1 && ch <=
            (int)Shortcut.F12)
text += "F" + (ch - (int)Shortcut.F1 +
            1);
else
{
if ( Shortcut == Shortcut.Del)
{
text +=
            "Del";
}
else
{
text +=
            (char)ch;
}
}
shortcuttext = text;
}

//take into
            consideration if we want to actually just show a
//menu seperator
            bar
if (Text == "-")
{
e.ItemHeight = 8;
e.ItemWidth =
            4;
return;
}

//if this is a top level menu, we don't
            show a shortcut
bool topLevel = Parent ==
            Parent.GetMainMenu();
string tempShortcutText =
            shortcuttext;
if ( topLevel )
{
tempShortcutText =
            "";
}
//measure the width of the text
int textwidth =
            (int)(e.Graphics.MeasureString(Text + tempShortcutText,
            SystemInformation.MenuFont).Width);

//measure the height of
            the text (plus some padding)
int extraHeight = 4;
e.ItemHeight
            = SystemInformation.MenuHeight + extraHeight;
//also take
            consideration of top level menu differences
if ( topLevel
            )
e.ItemWidth = textwidth - 5;
else
e.ItemWidth =
            Math.Max(160, textwidth + 50);

// save menu item
            height
itemHeight = e.ItemHeight;
}

Wow, that was interesting huh? Well, now we have the height and width for our MenuItem. The next message that would be sent by Windows would be the WM_DRAWITEM message. Of course we need to handle this message too.

OnDrawItem

The OnDrawItem event handler simply makes some calls to a few other methods that we need to draw the menu in our own way. Please notice that this method makes calls to: DrawBackground, DrawIcon, DrawCheckmark, DrawSeparator, and DrawMenuText. We are going to skip a discussion of the DrawCheckmark and DrawSeparator to try to not bore you with too much code.

//again, override the OnDrawItem of the
            MenuItem class
protected override void
            OnDrawItem(DrawItemEventArgs e)
{
if ( doColorUpdate)
{
DoUpdateMenuColors();
}

base.OnDrawItem(e);

Graphics
            g = e.Graphics;
Rectangle bounds = e.Bounds;
//determine if
            the item is selected
bool selected = (e.State &
            DrawItemState.Selected) > 0;
bool toplevel = (Parent ==
            Parent.GetMainMenu());
bool hasicon = Icon != null;
bool
            enabled = Enabled;

//call the drawbackground method with
            flags
DrawBackground(g, bounds, e.State, toplevel, hasicon,
            enabled);
//if there is an icon, draw it, if there is a checkmark
            draw it too
if (hasicon)
DrawIcon(g, Icon, bounds, selected,
            Enabled, Checked);
else
if (Checked)
DrawCheckmark(g,
            bounds, selected);
//of course we need to handle the separator
            bar
if (Text == "-")
{
DrawSeparator(g, bounds);
}
else
{
//this isn't a separator, so draw the actual
            text!
DrawMenuText(g, bounds, Text, shortcuttext, Enabled,
            toplevel, e.State);
}
}

As you can see, the OnDrawItem as implemented here simply makes calls to our other methods that do the actual drawing in the style ala Visual Studio .NET IDE. Now let's look at the code for some of those methods.

DrawBackground

As you might guess, this takes care of drawing the background of the menu. It's here that we also take care of the case in which the user is selecting the menu.

public void DrawBackground(Graphics g, Rectangle bounds, DrawItemState
state, bool toplevel, bool hasicon, bool enabled)
{
//set the selected variable depending on the state of the item
bool selected = (state & DrawItemState.Selected) > 0;

//if the item is selected, OR the item is in the MouseOver
//state, we want to show the menu item in a highlighted
//look and feel
if (selected || ((state & DrawItemState.HotLight) > 0))
{
if (toplevel && selected)
{ // draw toplevel, selected
bounds.Inflate(-1, 0);
g.FillRectangle(new SolidBrush(stripeColor), bounds);
ControlPaint.DrawBorder3D(g, bounds.Left, bounds.Top, bounds.Width, bounds.Height, Border3DStyle.Flat,
Border3DSide.Top | Border3DSide.Left | Border3DSide.Right);
}
else
{ // draw menuitem, selected OR toplevel, hotlighted
if ( enabled )
{
g.FillRectangle(new SolidBrush(selectionColor), bounds);
g.DrawRectangle(new Pen(borderColor), bounds.X, bounds.Y, bounds.Width - 1, bounds.Height - 1);
}
else
{
// if disabled draw menuitem unselected
g.FillRectangle(new SolidBrush(stripeColor), bounds);
//you may wonder what this whole STRIPE_WIDTH
//thing is. If you remember, the VISUAL STUDIO .NET style
//menu consists of a gray vertical bar and
//a white background for the rest of the menu
//this takes into account that STRIPE of gray
bounds.X += STRIPE_WIDTH;
bounds.Width -= STRIPE_WIDTH;
g.FillRectangle(new SolidBrush(bgColor), bounds);
}

}
}
else
{
if (!toplevel)
{ // draw menuitem, unselected
g.FillRectangle(new SolidBrush(stripeColor), bounds);
bounds.X += STRIPE_WIDTH;
bounds.Width -= STRIPE_WIDTH;
g.FillRectangle(new SolidBrush(bgColor), bounds);
}
else
{
// draw toplevel, unselected menuitem
g.FillRectangle(SystemBrushes.Control, bounds);
}
}
}

DrawIcon

public void DrawIcon(Graphics g, Image icon,
            Rectangle bounds, bool selected, bool enabled, bool
            ischecked)
{
//set the transparency color for the
            icon
Color transparentColor = Color.FromArgb(0, 128,
            128);
Bitmap tempIcon =
            (Bitmap)icon;
tempIcon.MakeTransparent(transparentColor);

int
            iconTop = bounds.Top + (itemHeight - BITMAP_SIZE)/2;
int iconLeft
            = bounds.Left + ( STRIPE_WIDTH - BITMAP_SIZE)/2;
//check if
            enabled, if not we'll draw it as disabled
if (enabled)
{
if (selected)
{
//here we are being a little bit
            sneaky. To simulate
//the slight drop shadow effect when a icon
            is
//moused over, we'll first draw the item as a
            disabled
//image slightly off from where the icon position
            really
//is, and then we'll draw the icon over it.
//this will
            give the effect desired!
ControlPaint.DrawImageDisabled(g, icon,
            iconLeft + 1, iconTop, Color.Black);

g.DrawImage(icon,
            iconLeft, iconTop-1);
}
else
{
//otherwise, we just
            draw the image icon
g.DrawImage(icon, iconLeft + 1,
            iconTop);
}
}
else
{
ControlPaint.DrawImageDisabled(g, icon, iconLeft + 1,
            iconTop, SystemColors.HighlightText);
}
}

Last but not least, we'll talk about the drawing of the menu text for our Owner Drawn menus.

DrawMenuText

public void DrawMenuText(Graphics g,
            Rectangle bounds, string text, string shortcut, bool enabled,
bool toplevel, DrawItemState state )
{
StringFormat
            stringformat = new StringFormat();
stringformat.HotkeyPrefix =
            ((state & DrawItemState.NoAccelerator) > 0) ?
            HotkeyPrefix.Hide : HotkeyPrefix.Show;

//in the case where
            the high contrast mode is on, we need to display
//white text
            instead of black
bool highContrast = false;
bool
            whiteHighContrast = false;
if ( SystemColors.Control.ToArgb() ==
Color.FromArgb(255,0,0,0).ToArgb() ) highContrast = true;
if
            ( SystemColors.Control.ToArgb() ==
Color.FromArgb(255,255,255,255).ToArgb() ) whiteHighContrast =
true;

//if menu is a top level, take out the &
//
            so that the menu text is centered
if ( toplevel )
{
int
            index = text.IndexOf("&");
if ( index != -1 )
{
text =
            text.Remove(index,1);
}
}

//determine the text width
int textwidth = (int)(g.MeasureString(text,
            SystemInformation.MenuFont).Width);
int x = toplevel ?
            bounds.Left + (bounds.Width - textwidth) / 2: bounds.Left + iconSize
            + 10;

int topGap = 4;
if ( toplevel ) topGap = 2;
int y
            = bounds.Top + topGap;

//create a new brush
Brush brush =
            null;

//figure out what color to use for the brush that's
            going to be
//used to draw the menu text
if
            (!enabled)
brush = new SolidBrush(Color.FromArgb(120,
            SystemColors.MenuText));
else if ( highContrast )
brush = new
            SolidBrush(Color.FromArgb(255, SystemColors.MenuText));
else
brush = new SolidBrush(Color.Black);

if (
            whiteHighContrast && ( (state & DrawItemState.HotLight)
            > 0 || ( (state & DrawItemState.Selected)
> 0
            && !toplevel )) )

brush = new
            SolidBrush(Color.FromArgb(255, Color.White));

//using the
            graphics.drawstring method, we pass in the text
//the font we ant
            to use, the brush (for color), the x and
//y coordinates and the
            string format
//this will draw the text for us
g.DrawString(text, SystemInformation.MenuFont, brush, x, y,
            stringformat);

// don't draw the shortcut for top level
            menus
if ( !toplevel )
{
// draw shortcut right
            aligned
stringformat.FormatFlags |=
            StringFormatFlags.DirectionRightToLeft;
g.DrawString(shortcut,
            SystemInformation.MenuFont, brush, bounds.Width - 10 , bounds.Top +
            topGap, stringformat);
}
}

And that's about it. We've now completed our task of creating an Owner Drawn menu. Don't you feel a sense of accomplishment and excitement? Well, maybe that's a little on the flaky side, but I do think Owner Drawn menus are very nice. Now let's take a brief look at the other part of the article, the XML configuration file. Towards the end of the article, we talk about the sample application and such. But if you run the sample code that's included with the article, you will see that the code that we just talked about above will render a menu that looks like this:

XML Config File

The menu implementation provided here uses a XML configuration file to load the menu settings for display. Although this isn't a necessary to use the DotNetMenuItem class, we'll briefly go over the architecture for sake of completeness of the sample application that is bundled with this article.

Let's take a look at a piece of the Menu.xml file that is included in the download:

<?xml version="1.0" encoding="UTF-8"
            ?>
<MainMenu>
<Menu Name="File" Shortcut=""
            Icon="" Checked="False" Enabled="True">
<Menu
            Name="&amp;New" Shortcut="CtrlN" Icon="" Checked="False"
            Enabled="True" />
<Menu Name="&amp;Open"
            Shortcut="CtrlO" Icon="" Checked="False" Enabled="True"
            />
<Menu Name="&amp;Close" Shortcut="CtrlC" Icon=""
            Checked="False" Enabled="True" />
<Menu Name="-"
            Shortcut="" Icon="" Checked="False" Enabled="True" />
<Menu
            Name="A&amp;dd Project" Shortcut="" Icon=""
Checked="False"
            Enabled="True">
<Menu Name="&amp;New Project..."
            Shortcut="" Icon=""
. . .

As you can see, the XML file is fairly straightforward and easily modified. The basic structure is more simply reduced to:

<!--formatted for readability ->
<MainMenu>
<Menu></Menu>
<Menu></Menu>
<Menu>
<Menu></Menu>
</Menu>
</MainMenu>

Where we can nest menus to provide submenus quite easily.

Processing the XML Config File

Displaying the contents of the XML file in the Windows Form application is pretty straightforward. In our main starting point of the Main.cs file, we set the .Menu item of the form to the result of a call to our CreateMainMenu method.

public static void Main(string[]
            args)
{
Form1 mainForm = new Form1();
mainForm.Menu = new
            MainMenu(CreateMainMenu());

Let's now take a quick look at the CreateMainMenu method. First we need to get the path of the XML document. From there we'll create a new XMLDocument object and load the XML file.

public static MenuItem[]
            CreateMainMenu()
{
string menuFile = Application.StartupPath +
            "\\" + "menu.xml";
XmlDocument doc = new
            XmlDocument();
doc.Load(menuFile);

Next we'll grab all of the nodes and start looping through them and calling the ProcessMenuItem on each element. This will take care of setting up the Main menu on the form.

XmlNodeList
            topMenus = doc.DocumentElement.ChildNodes;
MenuItem[] menuItems =
            new MenuItem[topMenus.Count];
int i = 0;
foreach ( XmlElement
            element in topMenus )
{
menuItems[i++] =
            ProcessMenuItem(element);
}

return
            menuItems;
}

ProcessMenuItem Method

>The ProcessMenuItem method takes the XML from the XML document and sets up the Menu as needed. As we noted above, this method is called for each XML Element in the configuration file. The code below is commented to make the reading of the function easier.

public static MenuItem
            ProcessMenuItem(XmlElement xmlelement)
{
//get the name for
            the menu
string menuName =
            xmlelement.Attributes["Name"].InnerXml;

if (
            menuName.IndexOf("&") != -1 )
menuName =
            menuName.Replace("&amp;", "&");

//here we grab the
            other attributes from the XML document
string menuShortcutString
            = xmlelement.Attributes["Shortcut"].InnerXml;
string menuIcon =
            xmlelement.Attributes["Icon"].InnerXml;
string menuChecked =
            xmlelement.Attributes["Checked"].InnerXml;
string menuEnabled =
            xmlelement.Attributes["Enabled"].InnerXml;

Shortcut
            menuShortcut = Shortcut.None;
//determine if there is a shortcut
            presented for this element
if ( menuShortcutString != "" )
{
menuShortcut =
(Shortcut)((Shortcut.F1.GetType()).InvokeMember(menuShortcutString,
            BindingFlags.GetField, null,
Shortcut.F1, new
            object[0]));
}

//create the parent menu as an Owner Drawn
            menu
DotNetMenuItem parentMenu = new DotNetMenuItem(menuName, new
            EventHandler(SelectMenuItem), menuShortcut);

//set up the
            menu icon, checked and enabled portions of the menu
if ( menuIcon
            != "" )
parentMenu.Icon = GetIcon(menuIcon);
if ( menuChecked
            == "True" )
parentMenu.Checked = true;
if ( menuEnabled ==
            "False" )
parentMenu.Enabled = false;

//now take a
            look to see if there are indeed child submenus for
            this
//particular menu item. If there are, then we'll recursively
            call
//the ProcessMenuItem method to generate the submenus
            accordingly!
XmlNodeList nodes = xmlelement.ChildNodes;
if (
            nodes.Count > 0 )
{
MenuItem[] menuItems = new
            MenuItem[nodes.Count];
int i = 0 ;
foreach ( XmlElement node
            in nodes )
{
menuItems[i] = ProcessMenuItem(node);
if (
            menuItems[i] !=
            null)
parentMenu.MenuItems.Add(menuItems[i]);
i++;
}
}
return
            parentMenu;
}

Although it isn't necessary to generate the menus this way, I've found it quite useful to be able to easily define the menu structures in an XML configuration file because of the ease of use and simplicity of the configuration file approach. Because of this, you could easily have more than one menu for your application, and you could switch between them in your application more easily than having a predetermined Menu structure defined via the IDE. It's up to you whether or not you use this approach, but as I tend to like this method.

Looking at the Sample Application

Now that we've covered the basics of the Owner Drawn menu and the XML configuration file, we'll take a look at the sample application in the download. As you can see, I tried to loosely simulate the File menu of the Visual Studio .NET IDE.

One thing of particular interest is the code used to simulate the raised shadow effect of the icons in the menu as the mouse hovers over the menu item (we talked about this briefly in a comment within the code discussion above). As you may or may not have noticed, there is only one bitmap for each of the icons, not 2 that get swapped as you might think. (Like a image rollover in a web type application). Instead we use a tricky .NET method called DrawImageDisabled. It is a member of the System.Windows.Forms.ControlPaint namespace. We call this method, and then call the DrawImage method for the icon. This results in 2 images, with the offset used in the DrawImageDisabled call simulating a dropped shadow type effect.

ControlPaint.DrawImageDisabled(g, icon, iconLeft + 1, iconTop, Color.Black);

As you can see, when the mouse is moved over the menu item, the icon seems "raised" off of the menu with a slightly gray drop shadow effect.

One more point to note. The menu items in the sample application are wired to message boxes that display the menu item chosen when clicked. As you might expect, this is accomplished just like any other menu handling in Visual Studio .NET.

The SelectMenuItem method takes care of this handling as described above.

public static void SelectMenuItem(object sender, EventArgs e)
{
if typeof(DotNetMenuItem).IsInstanceOfType(sender))
{
DotNetMenuItem item = (DotNetMenuItem)sender;
string menuText = item.Text;

MessageBox.Show(menuText);
}
}

Loose Ends

Application Setup

To set up the sample application on your machine, simply unzip the project files into a directory and double click on the .SLN file. The project will open up in Visual Studio .NET. From there, you can compile or run in debug mode as you would with any other C# Windows Form application.

Side note on using the Menu Creator in Visual Studio .NET with NO Config File

There is a bit of added complexity when we use the XML configuration file approach to setting up the Menu structures. The problem comes when hooking in the event handler code for the menu events. An alternative approach that alleviates all of those problems is just to use the Visual Studio .NET IDE to develop your menus as normal, then replace all the occurrences of the System.Windows.Forms.MenuItem declaration in the code with the DotNetMenuItem declaration. This resolves any problems and allows us to use the Visual Studio .NET generated menu handling code with our owner drawn menus.

To use the IDE to create the menu:

  • Go about creating your menu with the IDE tool as usual
  • Once finished designing the menu, open up your code and do a find/replace. Replace all of the occurrences of System.Windows.Forms.MenuItem with YourNamespace.DotNetMenuItem

This should allow you to use this Owner Drawn menu class simply without having to bother with the XML configuration file.

Take a look at the download file folder called "source without xml menu" to view this code.

Conclusion

In this article, we presented a method of simulating the look and feel of the Visual Studio .NET IDE menus for use in our own applications. We learned about Owner Drawn menus in C# and a XML configuration file approach that simplifies the creation and storage of menu structures. Armed with this knowledge, we can add Visual Studio .NET like menus to our applications that will run under Windows 2000 and Windows XP. I haven't yet tried this out on Windows 98 or Windows ME/NT so you are on your own if you are writing apps for those target platforms. Hopefully you'll all see how .NET allows you to somewhat easily (dare I say) Owner Draw your menus. Just another note - you could also leverage this code and move on to owner drawing other controls in Windows. The basic premise is the same, and it shouldn't be that difficult to accomplish once you've gotten the hang of it.

If you'd like to discuss a broader range of C#Today topics and issues, or if you have any C# related questions you need answers to, please visit our online resource centres at : http://p2p.wrox.com/list.asp?list=csharptoday_discuss&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq and http://p2p.wrox.com/list.asp?list=c_sharp&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq
C#Today - © 2002 WroxPress Limited


Using Late Bound COM Objects

Using Late Bound COM Objects
http://www.csharptoday.com/content.asp?id=1722
Article Rating: 4
Published on December 18, 2001
Abstract
In this article, Beth Breidenbach describes how to late bind to COM objects in .NET. Interoperability with COM objects has been well–integrated into the .NET runtime. But judging by the postings in the .NET newsgroups there appears to be some lingering confusion about how to bind to COM objects at runtime. This article will discuss the different methods available for invoking COM objects at runtime. After a brief review of late binding (and why it should sometimes be avoided), we will walk through three different invocation scenarios and explore the appropriate C# syntax to use. We end with a brief look at the relative performance of each method.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

This article will describe how to late bind to COM objects in .NET. Interoperability with COM objects has been well-integrated into the .NET runtime. Indeed, early binding is quite straightforward and has already been covered quite well in Kaushal Sanghavi's articles on COM(+) interoperability (see reference list at the end of this article). Judging by the postings in the .NET newsgroups, however, there appears to be some lingering confusion about how to bind to COM objects at runtime. This article will discuss the different methods available for invoking COM objects at runtime. After a brief review of late binding (and why it should sometimes be avoided), we will walk through three different invocation scenarios and explore the appropriate C# syntax to use. We will end with a brief look at the relative performance of each method.

Prerequisites

Our discussion assumes you understand the basic concepts of COM interoperability in .NET. If you are just getting up to speed on the topic, see the list of references at the end of this article. In particular, you should understand what a Runtime Callable Wrapper is, and how one is generated.

Introductory Concepts

Late Binding Defined

Binding associates a method with a pointer to the method's memory location. In early binding, the object's client is bound to the vtable locations of the object's methods at the time of compilation. Late binding, in contrast, identifies a method's location at runtime via human-readable names. To enable late binding, the target object must implement the IDispatch interface. The IDispatch::GetIDsOfNames and IDispach::Invoke methods allow object interrogation and method invocation at runtime. The OleView application (bundled with Visual Studio 6) can be used to determine whether the IDispatch interface is supported by your target component. Notice the IDispatch inherited interface for this component:

Assuming this object has a ProgID of "TestObject.TestClass" and is located on a server named "OurServer," the typical VB6 code for late binding would look like this:

Dim oMyObject As Object
Set oMyObject = CreateObject("TestObject.TestClass", "OurServer")
MsgBox oMyObject.Echo1("echo this back to me")

In the example above, the variable is declared as a generic object. The CreateObject call uses the ProgID to locate the dll on the specified server. Note that the server parameter is not required if the target dll is located on the local box.

Why Use Late Binding

As you probably know, Microsoft recommends early binding whenever possible. It provides the best performance because your application binds directly to the address of the required functions. As Microsoft's Knowledge Base article #Q245115 (http://support.microsoft.com/default.aspx?scid=kb;EN-US;q245115&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq) states when talking about early binding, "in terms of overall execution speed, it is at least twice as fast as late binding". So why worry about how to late bind at all?

There are scenarios for which late binding is justified. You may not know the exact binary interface until runtime. The object may exist in multiple versions and have improperly adapted interfaces between the versions. You may be developing web applications on a shared host and be unable to get a RCW created for the COM object you want to invoke. Or, the object may have been compiled in VB with improper version compatibility settings, resulting in constantly changing GUIDs even when the version hasn't changed. In these scenarios, a defensive programmer needs to remove his or her code a bit from the object's binary signature. The tradeoff is one of flexibility versus performance and should be weighed carefully.

.NET to COM Interop Review

The .NET runtime supports the creation of a managed wrapper for COM objects to handle translations between .NET and COM. This Runtime Callable Wrapper (RCW) acts as a bridge between .NET and COM. If you add a reference to a COM component in your Visual Studio .NET project, a RCW will automatically be generated for you (note that Visual Studio .NET asks you first if a wrapper should be generated, rather than just automatically creating it for you). Alternatively, you can use the tlbimp utility that installs with the .NET Framework Software Development Kit to create the wrapper manually. Manual generation of the RCW allows for better control of the process. Command-line switches allow specification of namespace, output file, strong name information, and references, among others. More details are found in the .NET Framework SDK documentation.

With the ability to create an RCW, early bound COM interoperability is quite straightforward. Having now laid some groundwork, let's look at how to accomplish COM late binding in .NET.

Late Binding in .NET: The Basic Invocation

Conceptual Overview

At a basic level, late binding in .NET is similar to the pre-.NET era. A generic variable is declared, a call to create the target component is made, and its methods are invoked. The concepts are the same, it is the implementation specifics that differ.

Late binding in C# requires a familiarity with the System.Reflection namespace. In particular, we are interested in the Activator, Assembly, and Type classes, as well as the InvokeMember method.

  • Type is the root of all reflection operations. All objects support the GetType method, which returns a Type object representing the instance's type. Once we have a COM object's type definition, we are able to create an instance of that object and invoke its methods.
  • Activator is a static class used to create an object instance once we have its Type or assembly name. Activator returns either an object or ObjectHandle. In the case of the ObjectHandle we immediately unwrap it to retrieve the underlying object.
  • Assembly can be used if you have an RCW for the object and know its name. Assembly objects can be instantiated by file name, which allows you to bypass using the Activator object. I will show example code of to do this, though I personally prefer to use Activator (I hate to remember more than one syntax for an action if I don't have to, and in this case I have yet to find a technical reason to prefer Assembly over Activator.)
  • InvokeMember is a method of the Type object. We use it to forward our method calls to the instantiated object.

A Simple COM Invocation

For the purposes of this article, we will use a COM object named "TestObject " created in Visual Basic 6. To emulate bad COM practices, the Project | Properties | Component setting has been set to No Compatibility. This will cause the GUID to change every time the component is compiled - wreaking havoc with early-bound applications. The figure below shows that TestObject contains one class, "TestClass," with the two methods "Echo1 " and "Echo2". The component is available in the download.

We will call the two methods of this (rather simplistic) COM object from a C# Windows Forms application and display the returned text in textbox controls.

Here is the full code for the button_click routine (note that error handling should be added). The routine is found in the file Form1.cs in the download's LateCOMTester project.

private void btnBasic_Click(object sender, System.EventArgs e)
{
// overloaded method, can pass the server name as second
// parameter if necessary.
Type myType = Type.GetTypeFromProgID("TestObject.TestClass");

// heavily overloaded method call. Can also be used with
// assembly file names if you have them.
object myObject = Activator.CreateInstance(myType);

// method parameters are passed in an object array.
// Must set these up before invocation
object [] oParms = new Object[]{"Echo This"};

// Invoke the method and cast the result to the correct
// object type.
string myString = (string) myType.InvokeMember("Echo1",
BindingFlags.InvokeMethod,null,myObject,oParms);

txtEcho1.Text = myString;

// Reset for the second method call, to pass in 2 parameters
oParms = null;
oParms = new Object[] {"Echo Two", "with this"};
myString = (string) myType.InvokeMember("Echo2",BindingFlags.InvokeMethod, null,
myObject, oParms);

txtEcho2.Text = myString;

}

Loading the Type

The GetTypeFromProgID method allows you to load the Type object for a given registered COM component. This method of type location is best used when there is no pre-generated RCW or if you don't know the RCW's file name. For example, a RCW generated by Beta2 of the Visual Studio IDE will have the version number appended to the file name, while the Release Candidate bits for Visual Studio do not append version information in the file's name. A later example will show how to get the Type object when the RCW's name is known.

// overloaded method, can pass the server name as second
// parameter if necessary.
Type myType = Type.GetTypeFromProgID("TestObject.TestClass");

Creating the Object

CreateInstance is a heavily overloaded method of the static class Activator. In this usage we pass it Type and receive an object in return. The overloads for this method that accept a Type object always return back an object. Those overloads that accept the name of an assembly return an ObjectHandle object, which must then be unwound to retrieve the base object. A later example will show how to unwind the ObjectHandle.

// heavily overloaded method call. Can also be used with
// assembly file names if you have them.
object myObject = Activator.CreateInstance(myType);

Invoking the Method

Invoking the COM object's methods is a bit less intuitive. For starters, the parameters must be passed as an array of objects. In this case the method Echo1 takes a single string parameter, while Echo2 requires two string parameters. The code example shows setting up the parameter array for each method call.

As the code illustrates, the method invocation is performed via the Type object. In its simplest form InvokeMember accepts five parameters:

  • name: the name of the method to call.
  • invokeAttr: A bitmask comprised of one or more BindingFlags that specify how the method search is conducted. The usual flag is BindingFlags.InvokeMethod. The flag is technically not required to be specified. However, as BindingFlags.Public | BindingFlags.Instance is the default (and is not what we desire here) you will want to make a point of providing the BindingFlags.InvokeMethod value for this flag.
  • binder: A binder object is used to define specific properties for binding. A binder parameter is not required for this scenario.
  • target: The object on which to invoke the specified method
  • args(): The object array of parameters required for the method.

   // method parameters are
            passed in an object array.
// Must set these up before
            invocation
object [] oParms = new Object[]{"Echo
            This"};

// Invoke the method and cast the result to the
            correct
// object type.
string myString = (string)
            myType.InvokeMember("Echo1",
BindingFlags.InvokeMethod,null,myObject,oParms);
txtEcho1.Text
            = myString;

// Reset for the second method call, to pass in 2
            parameters
oParms = null;
oParms = new Object[] {"Echo Two",
            "with this"};
myString = (string)
            myType.InvokeMember("Echo2",BindingFlags.InvokeMethod,
            null,
myObject, oParms);
txtEcho2.Text = myString;

The general concepts introduced thus far will get you where you need to go, 90% of the time. This section deals with some of the less likely, yet potentially more painful scenarios that can occur in the real world.

Late Binding in .NET: With a Runtime Callable Wrapper Available

Often, if you have a RCW you would be much better off simply doing an early bind to the object. If you're doing a late bind in this scenario, you are generally attempting to protect yourself from a third party who has made COM programming errors - either allowing the GUID to change when the version hasn't, or incorrectly handling versioning of the interfaces.

In such a scenario you have two options. The most desirable option is to regenerate an RCW and recompile your project to it. If, however, you have reason to believe that the object (and its RCW) may be switched out post-install, you may need to bind to the RCW's name and its underlying object at runtime. There are two ways of coding this requirement, with either the Activator or the Assembly object. Let's start with the Activator:

Late Binding to an RCW via the Activator

As you read the code you'll notice we used the method CreateComInstanceFrom, and followed it with an Unwrap call. Activator has three versions of "Creates": CreateInstance, CreateInstanceFrom, and CreateComInstanceFrom. Each of these methods has several overloads. In each case the overloads not taking a Type parameter will return ObjectHandle instead of an object. To retrieve the underlying object we add an Unwrap call to the create method:

// load the object via the assembly and type name
// add the UnWrap() call to get the underlying base object
// NOTICE that the class name is now "TestClassClass" instead of
// "TestClass" - this is the default from the VS tlbimp output.
object myObject =Activator.CreateComInstanceFrom("Interop.TestObject.dll",
"TestObject.TestClassClass").Unwrap();

// this would also have worked, though the quality of exceptions thrown
// is not as informative
//object myObject =Activator.CreateInstanceFrom("Interop.TestObject.dll",
// "TestObject.TestClassClass").Unwrap();

// from the unwrapped object, get the type definition
Type myType = myObject.GetType();

// set up the parameters and invoke the methods
// from this point forward, the code is the same as the previous example

Contrary to what one might expect, the second parameter for the "Create" in this scenario must be "TestObject.TestClassClass " instead of "TestObject.TestClass." This naming convention is the output when the assembly is generated via Visual Studio .NET. To be sure of the class name you can (temporarily) reference the RCW into your project and view it in the Object Browser:

Late Binding to an RCW via the Assembly

The alternative coding for the same scenario uses the Assembly object:

Here we show using the Assembly object's LoadFrom method to load the assembly. Once the assembly is loaded, we call its CreateInstance method to return the needed object. Unlike the Activator versions, Assembly's CreateInstance overloads all require the assembly file parameter and none return an ObjectHandle.

// load the assembly
Assembly myAsm;
myAsm = Assembly.LoadFrom("Interop.TestObject.dll");

// from assembly, get the target object
object myObject = myAsm.CreateInstance("TestObject.TestClassClass");

// from assembly, get the type definition
Type myType = myAsm.GetType("TestObject.TestClassClass");
// this would have worked too
//Type myType = myObject.GetType();

// set up the parameters and invoke the methods
// from this point forward, the code is the same as the previous example

Late Binding in .NET: When You Need a Non-Default Interface

Veteran COM programmers are familiar with the ability of a COM object to implement multiple interfaces. When late-binding to a COM server, you will sometimes need to call a method from a non-default interface. In this case, we are only going to late bind to the COM object. The interface is referenced into the project via normal methods, including generation of an RCW. This example uses a COM object "Process1Impl.Processor1Impl " that implements the "Process.Processor " interface. The project code is available in this article's download material.

If you've read the earlier examples, much of this code will be familiar. We acquire the object's default type definition from its ProgID and use Activator to create an instance of that type. A variable is declared of the interface's type, and we cast the object to the interface. Interface method calls are then made as normal - no InvokeMethod syntax is required.

// get the implementing COM object
Type oType = Type.GetTypeFromProgID("Process1Impl.Processor1Impl");
object oTemp = Activator.CreateInstance(oType);

// cast it to the generic interface
Process.Processor myProc = (Process.Processor) oTemp;

// call the interface's method
string myOut = myProc.Process("ThisParam","ThisAction");
txtEcho1.Text=myOut;

Late Binding in .NET: Monitoring Performance of the Various Invocation Methods

We have seen a variety of ways to bind to a COM object at runtime. As I pointed out in the beginning of this article, late binding itself carries some rather severe performance penalties. To avoid impacting performance further, you will want to evaluate the relative speed of these invocation methods for your own COM object. To do this, I suggest maintaining a simple test harness to invoke the target COM object each of the three ways on your version of Visual Studio .NET. This will help you make choices for your current code. The tests should then be re-run when the final .NET bits are released to determine if your invocation strategy needs to change. To help you get started, our .NET project provides a simplistic performance testing harness. These final routines of the project call each of the three invocation examples 5,000 times and report the number of seconds required for each set of 5,000 invocations (as a caveat, this is a simplistic example. The COM methods are only passing small strings, and everything is instantiated locally).

In my testing, the simple COM instantiation, without an RCW, was sometimes marginally faster. If the application timed this "simple" routine before timing the two RCW-related methods, the simple example was generally about 1 second faster. On the other hand, if the simple example was timed last, the performance of all three methods was equivalent. Try these timings on your own late-bound objects as your mileage may vary.

Note that the .NET Framework SDK license prohibits formal benchmarking of the code at this time (a reasonable requirement considering we're working with beta code), so these are not authoritative recommendations in this area. Your mileage will vary from mine, and will undoubtedly also vary between Beta 2, Release Candidate 1, and the final released code.

Conclusion

While we should try to avoid late binding whenever possible, there are times when it's the correct solution. Through the Reflection namespace, we have the tools necessary to late bind. We used the Type, Activator, and Assembly classes to create COM objects several different ways. In addition, we took a moment to discuss some syntax options available for creating the objects and acquiring the type definitions. Take a few minutes to explore System.Reflection and see how it can benefit your projects.

C#Today - © 2002 WroxPress Limited


Exposing Type Information

Exposing Type Information
http://www.csharptoday.com/content.asp?id=1709
Article Rating: 3.5
Published on November 30, 2001
Abstract
In his previous article, Morgan Skinner looked at the TypeDescriptor class, and how it could be used to retrieve various pieces of information from a type (or object). In certain situations however, you might want to augment or bypass the information returned from the TypeDescriptor, and in order to do this you need to implement the ICustomTypeDescriptor interface on your type. In this article, Morgan examines the ICustomTypeDescriptor interface, and show some examples of where it might be used, such as to to construct a list of properties on the fly when reading records from a database and displaying them on screen. Using the ICustomTypeDescriptor interface, it is possible to give each record its own list of properties, which might be useful if you want to permit or deny access to those properties based on some row level security settings.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

In my previous article for C#Today (http://www.csharptoday.com/content.asp?id=1646&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq), I showed several examples of how to utilise the new Property Grid control when displaying data for a class. We created a simple class with a few properties, and throughout the example described a number of attributes that can be placed on those properties to make them appear in a more palatable fashion in the user interface. Some examples were adding the Description and Category attributes onto properties, which are then utilised by the Property Grid when displaying an object.

When enumerating properties (and events, fields, etc) on a type, we utilised the TypeDescriptor class, defined within the System.ComponentModel namespace. The TypeDescriptor class can be used to retrieve from a type (or an object), the properties, attributes, events and various other pieces of information.

In certain situations however, you might want to augment or bypass the information returned from the TypeDescriptor, and in order to do this you need to implement the ICustomTypeDescriptor interface on your type. This interface is called within TypeDescriptor to determine the properties, events, etc, that a type supports, and can be used to generate this information dynamically. One example of where you might want to construct the list of properties on the fly is when reading records from a database and displaying them on screen. Using the ICustomTypeDescriptor interface, it is possible to give each record its own list of properties, which might be useful if you want to permit or deny access to those properties based on some row level security settings.

In this article, we will examine the ICustomTypeDescriptor interface, and show some examples of where it might be used. To accompany the article there is a .ZIP file included which contains all the source code presented here.

The ICustomTypeDescriptor interface

This interface, defined within the System.ComponentModel namespace, is used by the TypeDescriptor class to return information about the type. Its definition is shown below:

interface ICustomTypeDescriptor
{
AttributeCollection GetAttributes();
string GetClassName();
string GetComponentName();
TypeConverter GetConverter();
EventDescriptor GetDefaultEvent();
PropertyDescriptor GetDefaultProperty();
object GetEditor(Type editorBaseType);
EventDescriptorCollection GetEvents();
EventDescriptorCollection GetEvents(Attribute[] attributes);
PropertyDescriptorCollection GetProperties();
PropertyDescriptorCollection GetProperties(Attribute[] attributes);
object GetPropertyOwner(PropertyDescriptor pd);
}

This interface may seem rather daunting at first sight, as with twelve methods, an implementation would not appear to be a trivial undertaking. Before we delve into these methods to provide some custom implementations, I'll explain exactly how the interface is used.

Say you have an object called my_obj and you want to obtain the list of properties for that object, in the same way that the Property Grid control does. We could utilise the static TypeDescriptor.GetProperties() method, which would return us a collection of all public properties on that object.

Object my_obj = <something or other> ;

PropertyDescriptorCollection coll = TypeDescriptor.GetProperties (my_obj);

Within the GetProperties method, the default implementation checks the object for ICustomTypeDescriptor, and if found, utilises that interface to supply the required information. If not, the default implementation of TypeDescriptor uses reflection to iterate through all the properties of the type, reads the attributes (such as Description, etc) and creates a collection of PropertyDescriptors for each property. If you really want to know how it's done, then run ildasm and have a look at the IL for this method.

So, providing your own type descriptor information via ICustomTypeDescriptor sounds difficult, doesn't it? Seeing that TypeDescriptor immediately looks for ICustomTypeDescriptor and then calls it, how are you going to implement the calls that you want to override, but utilise the stock implementation of (say) GetProperties in your object.

Thankfully, help is at hand. For each TypeDescriptor method such as GetProperties, there's another that takes a Boolean, which is used to alter the way the method works. If true, this alters the method so as to ignore the ICustomTypeDescriptor interface, and pass the call on into the TypeDescriptor class itself. I've presented the pseudo code for the GetProperties (object, bool) method below, to show how this works internally:

public static PropertyDescriptorCollection GetProperties
( object o , bool ignoreCustomTypeDescriptor )
{
if ( null == o )
throw new ArgumentNullException ( "component" ) ;
if ( ignoreCustomTypeDescriptor )
return <some stuff from TypeDescriptor > ;
else
return (o as ICustomTypeDescriptor).GetProperties ( o ) ;
}

Armed with this knowledge, a simple implementation of ICustomTypeDescriptor can simply delegate all calls into the TypeDescriptor class. Here's a snippet of a class I've created which implements this interface by delegating all calls through to the TypeDescriptor class. The full code is available in the DelegatingDescriptor subdirectory of the .ZIP file.

public AttributeCollection
            GetAttributes()
{
// Return a collection of attributes for the
            type.
// This calls up to the TypeDescriptor class, which
//
            exposes a stock implementation. The 'true' parameter
// means
            "don't use the ICustomTypeDescriptor implementation",
// which is
            necessary to avoid infinite recursion
return
            TypeDescriptor.GetAttributes ( this , true ) ;
}

public
            string GetClassName()
{
// Again delegate to the
            TypeDescriptor implementation
return TypeDescriptor.GetClassName
            ( this , true ) ;
}

public string
            GetComponentName()
{
// Again delegate to the TypeDescriptor
            implementation
return TypeDescriptor.GetComponentName ( this ,
            true ) ;
}

// Other methods removed for
clarity

As you can see, providing a default implementation of the methods you don't want to override is as easy as calling the TypeDescriptor class. The example displays a simple property sheet as follows:

Unsurprisingly, this is exactly the same as would be displayed if no custom type descriptor interface has been defined.

Now that I've shown a trivial implementation of this interface, let's do something more useful.

Creating your own Properties

When the GetProperties method is executed on an object, you should get a collection of PropertyDescriptor objects in return. These are used to provide descriptive information about a given property, hence the name. Some examples of information available are the name, description, category, and the type of a given property.

In the previous article we used the Description and Category attributes on a property to affect the way that the user interface was displayed. This information is used, together with the property itself, to generate an instance of PropertyDescriptor for each property accessible from the object.

So, if there is already a way of fully describing a property, why might you want to create your own implementation of ICustomTypeDescriptor ? One scenario where this information might be useful is when you have a class that has dynamic information, such as a class that wraps a database table. If the table can be modified by the user (by adding in columns), then one hard-coded class won't do.

Another example would be where you have a class whose properties change based on some other information, such as the user who is accessing the class. An example might be an employee class, which exposes the 'salary' property to certain privileged users. You can of course implement this in any number of other ways, but for simplicity and ease of use, a custom implementation of ICustomTypeDescriptor is a good alternative.

Creating your own PropertyDescriptors is relatively straightforward. Derive a class from PropertyDescriptor, implement a few methods, and add this to the collection returned from the GetProperties call.

In my next example, we'll expose some fields from a class as false properties, by implementing ICustomTypeDescriptor. The code for this example is available in the FieldsAsProperties subdirectory of the downloaded .ZIP file.

The steps in this example are as follows:

  • Create a subclass of PropertyDescriptor
  • Return a PropertyDescriptorCollection in response to the GetProperties calls.

The first step is necessary as we need to override some of the basic methods of PropertyDescriptor, plus that class is abstract, and the current crop of subclasses in .NET don't do what we need.

Our 'properties' are going to be fields of a class for now, so we want to design a PropertyDescriptor subclass which gets and sets the value of a given field when used. The best way to do this is to use reflection to return the FieldInfo instance of the given field, and this class can be used to get and set the field value, regardless of its visibility on the defining class. The FieldInfo class is defined within the System.Reflection namespace. Here's an example:

public class
            MyClassWithFields
{
public MyClassWithFields (
            )
{
}

private int m_age = 34 ;
private string m_name
            = "Morgan" ;
private string m_address = "Not a chance"
            ;
}

To obtain the field information for any of these fields, all we need is an instance of this class, and we can do the rest with the Type class.

MyClassWithFields thingy = new MyClassWithFields ( ) ;

// Define the flags which are used when looking up the field - look for
// instance fields, which are public, protected or private, and ignore
// casing of the field.
BindingFlags flags = BindingFlags.Instance | BindingFlags.IgnoreCase|
BindingFlags.Public | BindingFlags.NonPublic ;

FieldInfo age = thingy.GetType().GetField ( "m_age" , flags ) ;

// Set the age to 99
age.SetValue ( thingy , 99 ) ;

This code shows how to obtain the FieldInfo information for ' m_age ' from a class, and also how to set the value of this field. We'll use the same in our subclass of PropertyDescriptor.

A PropertyDescriptor subclass : CruftedProperty

Here's the class I came up with, to wrap a field in a PropertyDescriptor. The code is available in the FieldsAsProperties subdirectory of the associated .ZIP file.

public class CruftedProperty : PropertyDescriptor
{
// Construct a property from the name of a field.
public CruftedProperty ( object owner , string fieldName ,
string caption , Attribute[] atts )
: base ( fieldName , atts )
{
m_componentType = owner.GetType ( ) ;
m_displayName = caption ;
m_fieldInfo = owner.GetType().GetField ( fieldName ,
BindingFlags.Instance|BindingFlags.IgnoreCase|
BindingFlags.Public|BindingFlags.NonPublic ) ;
}

// Return the type of the parent object
public override System.Type ComponentType
{
get { return m_componentType ; }
}

// Return a flag indicating whether the property is read/write
// or read only. Default here is always read/write
public override bool IsReadOnly
{
get { return false ; }
}

// Return the System.Type of the property itself
public override System.Type PropertyType
{
get { return m_fieldInfo.FieldType ; }
}

// Return a flag indicating whether the value of this property on
// the passed object is resettable - i.e. has changed from the
// default. For this trivial implementation always return true
public override bool CanResetValue ( object component )
{
return true ;
}

// Return the value of this property on the passed object.
// This delegates to the field info object cached in the constructor
public override object GetValue ( object component )
{
return m_fieldInfo.GetValue ( component ) ;
}

// Set the value of this property on the passed object.
// Also delegates to the field info object cached in the constructor.
public override void SetValue ( object component , object value )
{
m_fieldInfo.SetValue ( component , value ) ;
}

// Reset the value to it's original - here just set it to null
public override void ResetValue ( object component )
{
m_fieldInfo.SetValue ( component , null ) ;
}

// Return true if the value of the property should be serialized
public override bool ShouldSerializeValue ( object component )
{
return false ;
}

// Return the name displayed for this property
public string DisplayName
{
get { return m_displayName ; }
}

// Local variables used to cache information
private FieldInfo m_fieldInfo ;
private System.Type m_componentType ;
private string m_displayName ;
private string m_description ;
}

It's a bit lengthy, so I'll describe the members in brief. The constructor for CruftedProperty takes an instance of an object (which is the object on which the field is defined), the name of the field as a string, the textual caption to be shown on screen, and an array of attributes. This information is cached internally, and we can retrieve the FieldInfo object for the passed fieldname from the passed information.

The rest of the methods are fairly simple - GetValue and SetValue delegate to the FieldInfo implementations, DisplayName returns the caption passed to the constructor, and the other methods return mainly defaults. A complete implementation should do a little more, I know!

So, that's the first part of the example, now to create the PropertyDescriptorCollection.

Returning a PropertyDescriptorCollection

To return your own property descriptor collection, you must provide an implementation of both GetProperties methods of ICustomTypeDescriptor.

Attribute[] atts = new Attribute[] { } ;

CruftedProperty age = new CruftedProperty ( this , "m_age" ,
"Age" , atts ) ;
CruftedProperty name = new CruftedProperty ( this , "m_name" ,
"Name" , atts) ;
CruftedProperty address = new CruftedProperty ( this , "m_address" ,
"Address" , atts ) ;

return PropertyDescriptorCollection (
new PropertyDescriptor[] { age, name, address } ) ;

This example code shows how you can use the CruftedProperty class to create a property descriptor for each of the three fields presented earlier. So, putting this all together, we have a class which implements ICustomTypeDescriptor, a class which acts like a property descriptor, allowing us to get/set the value of a field just as if it were a property. Now all we need is to see what happens when the .exe file is run:

You would be forgiven for thinking that this is the same image as presented earlier - apart from the lack of descriptive text at the bottom of the property grid, the two images are all but identical.

Taking a look back at the code where we create the property descriptors, you may have noticed that we used an empty attribute collection for each field. If we were to have supplied some attribute instances (such as Description or Category ), these would be used to alter the display. The snippet of code below shows how to define the description for a given property:

Attribute[] atts = new Attribute[]
{ new DescriptionAttribute ( "Blah Blah Blah" ) } ;
CruftedProperty age = new CruftedProperty ( this , "m_age" ,
"Age" , atts ) ;

This would result in the following display for the Age property:

You can naturally supply several attributes to the property descriptor class constructor. Alternatively, you can override the virtual Attributes property on the parent class of PropertyDescriptor ( MemberDescriptor ), and return a collection of attributes generated as requested. As for description, you can always override the Description property to return the text instead of using an attribute.

Fully Dynamic Properties

The last example in this article will show one way to create properties on the fly. An instance where this might be useful is where you are using a database table that can be modified by the user to tailor the system to their own requirements. What you might want to do is have a class which retrieves customer records, and if any new fields are added to this table, these will become available in the user interface.

In this example, we'll create a new base class which implements ICustomTypeDescriptor, then override this to create a dynamic person class which adds the appropriate properties for a person. Just to make it more realistic, we'll store these dynamic properties within a DataTable. The properties exposed from the class will therefore be linked to columns in the data table. The picture below shows how this all fits together:

I've created a base class, 'DynamicProperties', which implements ICustomTypeDescriptor, and contains a DataTable which consists of a single row, where we will store the values of the dynamic properties. The ' DynamicPerson ' class inherits from DynamicProperties, and implements the function shown below to add in its set of properties:

public override void AddColumns ( )
{
this.AddProperty ( "Age" , typeof ( int ) ,
"The age of the person" ) ;
this.AddProperty ( "Name" , typeof ( string ) ,
"The name of the person" ) ;
this.AddProperty ( "Address" , typeof ( string ) ,
"The address of the person" ) ;
}

The base class AddProperty method adds a new column of the appropriate type into the internal DataTable. This data table is then used as the basis for the custom property descriptor collection returned from the GetProperties calls. The code within GetProperties loops through each DataColumn in the internal table, and adds an object that derives from PropertyDescriptor into a cached collection.

foreach ( DataColumn dc in m_table.Columns )
m_propertyCache.Add ( new ColumnProperty ( dc ) ) ;

The constructor for ColumnProperty accepts a DataColumn instance, which is cached internally for use by the other methods:

private class ColumnProperty : PropertyDescriptor
{
public ColumnProperty ( DataColumn dc )
: base ( dc.ColumnName , new Attribute[] { } )
{
m_column = dc ;
}

// Cached data column
DataColumn m_column ;
.
}

This ColumnProperty class is one I've designed, which will provide a custom implementation of PropertyDescriptor when passed a data column. This implementation delegates many of the calls to the data column itself, such as Description, IsReadOnly and PropertyType. As a data column already has a Caption property, we can use this for the descriptive text shown on screen. The column also has a ReadOnly flag, which can be used to provide the implementation of IsReadOnly. A column also has a column type, available as the DataType member, which we'll use in the implementation of the PropertyType member of PropertyDescriptor.

The other thing to do in my ColumnProperty class is to provide implementations of GetValue, SetValue and ResetValue. These methods are passed an object on which to get/set/reset the value, and these will be passed an instance of the DynamicProperties class (in this example, a DynamicPerson ). We can check that the passed object is indeed an instance of the appropriate class, and if so pass the Get/Set/Reset value calls onto the base DynamicProperties class. Here's the implementation of the SetValue member:

public override void SetValue ( object component , object value )
{
DynamicProperties dyn = component as DynamicProperties ;

if ( null != dyn )
dyn[m_column.ColumnName] = value ;
else
throw new ArgumentException ( "Component is not derived from DynamicProperties" ) ;
}

All this does is to check to ensure that the passed 'component' instance is derived from the appropriate class, and if so, it sets the value of the named property on the base class. We use the cached column to find the name of the 'property' that should be set, and pass this into the dynamic properties class:

public object this[string name]
{
get
{
// This is a little contrived, as normally your class would link
// to a row in the database, rather than containing the whole table
// with a single row
return m_table.Rows[0][name] ;
}
set
{
m_table.Rows[0][name] = value ;
}
}

The above method of DynamicProperties simply sets the value of the appropriate column in the appropriate row to the value passed in. The GUI displayed is pretty much the same as the earlier examples, so I won't show it again.

Showing Rows the easy way

The previous example went to some lengths to create a property descriptor collection for a set of rows in a data table, which could be used when you want to have complete control over how a row is displayed, and which columns are actually put on screen.

If all you want to do is display a row in a PropertyGrid (quick and dirty style), you'll be pleased to know that there's a class which implements ICustomTypeDescriptor over a data table, so you can basically throw a row from a table into the PropertyGrid. Here's the code:

DataTable dt = {Get a data table}
            ;
DataView dv = new DataView ( dt ) ;

// Assign the first
            row from the data view to a property
            grid.
propertyGrid.SelectedObject = dv[0] ;

This is so simple that I've not provided an example project. Instead, there is some code(commented out) within the DynamicProperties example, in the Form1.cs file. This will create a simple data table and display a row from that table in a property grid.

Readers may also be interested in looking at the DataRowView class (http://msdn.microsoft.com/library/default.asp?url=/library/en-us/cpref/html/frlrfSystemDataDataRowViewClassTopic.asp&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq), which also deals with displaying data.

Conclusion

This article has presented a few methods of exposing custom properties from a class to the property grid control. The ability to create your own custom type descriptor implementation is very useful if the standard, reflection based mechanism doesn't quite do what you want it to.

The implementations of ICustomTypeDescriptor were usable, however incomplete. You might want to add an implementation of GetEvents, which returns a (potentially) custom list of events. You could also return a custom TypeConverter from the GetTypeConverter method, and possibly a custom GUI editor from the GetEditor method.

If you'd like to discuss a broader range of C#Today topics and issues, or if you have any C# related questions you need answers to, please visit our online resource centres at : http://p2p.wrox.com/list.asp?list=csharptoday_discuss&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq and http://p2p.wrox.com/list.asp?list=c_sharp&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq
C#Today - © 2002 WroxPress Limited


Networking Support Provided by the .NET Framework

Networking Support Provided by the .NET Framework
http://www.csharptoday.com/content.asp?id=1695
Article Rating: 3.5
Published on November 7, 2001
Abstract
The .NET Framework provides a wide range of excellent features that developers can take advantage in building networking applications. The System.Net classes are similar to Microsoft's WinInet API (used for internet protocol programming), in that they allow applications to get and send data using Internet protocols. The .NET classes are quite different in design and function however, the main difference being that the .NET classes are designed with an emphasis on standing up to a high-stress server environment. In this article, Laxminarayana Konda explains how to get data from a remote URL by using different classes, and how to do download files and supply credentials to a remote URL. We wrap up by creating TCP Listeners and TCP clients.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

Using a layered approach, the System.Net classes enable applications to access networks with varying levels of control, depending upon the needs of the application. They cover nearly every scenario on the Internet today - from fine-grained control over sockets, to a generic request/response model. The System.Net classes expose a robust implementation of the HTTP 1.1 protocol, with authentication, pre-authentication, encryption, proxy support, pipelining, and connection management.

The System.Net classes are designed for writing scalable, high-performance middle-tier applications. A common scenario for Web servers today involves making multiple requests to back-end or external servers as a result of a single client browser request. This scenario requires a robust middle-tier networking stack that can stand up to a high load. It appears that the System.Net classes were designed specifically with this kind of requirement in mind, such as connection management, pipelining, keep-alive, and asynchronous send and receive ensuring strong support for the middle tier.

Let's take a closer look at these classes. From a logical perspective, they contain three layers as shown below: the request/response layer, an application protocols layer, and a transport layer. The WebRequest and WebResponse classes represent the Request/Response layer. The TCP classes make up the Application Protocols layer, and Sockets represents the Transport layer.

In this article, we will see how to get HTML data using a combination of WebRequest and WebResponse classes, and then we will implement the same functionality using the WebClient class alone. Secondly, we will see how to get HTML files from a remote URL, how to supply default credentials to the URL, and wrap it up by creating a TCPListener (a console application) and TCPClient (a windows application).

Please note that this article does not intend to provide exhaustive coverage of the individual features of .NET Networking, instead we focus on understanding which basic features to use in what networking situations.

Let us divide our application logically into three parts:

  • Getting metadata (HTML), download files from the remote URL and scenarios
  • Supplying credentials to the remote URL
  • Implementing TCPListener and TCPClient

Prerequisites

The prerequisites for this article are:

  • Visual Studio.Net Beta 2
  • SQL-Server 2000 and
  • Windows 2000 Professional.

Getting metadata (HTML), and downloading files from a remote URL

Open Visual Studio .NET and create a C# windows application (the complete source code is in the download).

Add two textboxes, one to take URL input, and another to display HTML data, then add a button to implement the code in the click event as shown below:

We start by importing all the namespaces that we require:

using System;
using System.Drawing;
using System.Collections;
using System.ComponentModel;
using System.Windows.Forms;
using System.Data;
using System.Net;
using System.Net.Sockets;
using System.Threading;
using System.IO;
using System.Text;
using System.Diagnostics;

  • System.Net is an implementation of network services for programmers to develop applications that use internet resources without worrying about specific in depth details of the protocol.
  • System.Net.Sockets is the interface for controlling access to the network.
  • System.Threading is used for multi-threaded programming.

Now let's implement code for the WebRequest button, which reads the URL and displays HTML in the display textbox using a combination of the WebRequest and WebReponse classes.

  • WebRequest is an abstract class, the object for this class is created using the Create method of the WebRequest class:

WebRequest WReq = WebRequest.Create("http://www.CSharpToday.com");

  • The WebResponse class is used to handle responses in a Request/Response model. This class can be used to access any resource on the network that is addressable with a Uniform Resource Identifier (URI). Client applications should never create WebResponse objects directly; they should be created by calling the GetResponse() method on a WebRequest class. For example:

WebResponse objResp = objReq.GetResponse()

  • If there are any errors in accessing the network, the WebRequest and WebResponse throws the exception called WebException.

Now let's go through the code of the WebRequest button.

WebRequest objReq = WebRequest.Create(txtURL.Text);

Here we create the WebRequest object of the URL typed in the URL textbox. The following code talks to the remote URL using WebRequest and gets the Response from the URL in the WebResponse object.

txtDisplay.Text = "";
try
{
WebResponse objResp = objReq.GetResponse();
Stream objStream = objResp.GetResponseStream();
byte[] readData = new byte[512];
int bytesRead = objStream.Read(readData,0,512);
while (bytesRead >0 )
{
Encoding objEn = System.Text.Encoding.GetEncoding("utf-8");
txtDisplay.Text = txtDisplay.Text + objEn.GetString(readData,0,bytesRead);
bytesRead = objStream.Read(readData,0,512);
}
}
catch(Exception ex)
{
txtDisplay.Text = "Error Message :" + ex.Message;
}

The above code is very simple and straightforward. We create the WebRequest object of the URL that we want to talk to, and then we call the GetResponse() method of the WebRequest class. When this method is called, the response from the URL is given in the WebResponse object. WebResponse supports a method called GetResponseStream(). This method returns a data stream, which is captured in the Stream object. The Read() method of the Stream reads into the buffer, and using the Encoding class, is encoded to the text and displayed in the Display textbox as shown below:

Now let us add another button - call it WebClient. This button has the same functionality as the WebRequest button; let us see how WebClient returns the same functionality in a much simpler way. Before walking through the code, let us discuss the Webclient class and its supported methods. Here we use the two methods, DownloadData() and DownloadFile().

The WebClient class is used to send and get data from the Uniform Resource Identifier (URI) or URL. This class can be instantiated directly as shown below:

WebClient objWeb = new WebClient();

  • Some of the important features of this class are sending data in the form of bytearrays, collections and files. The supporting methods for this functionality are UploadData() , UploadValues() and UploadFile().

Another set of methods like DownloadData(), DownloadFile() gets the data from the Uniform Resource Identifier (URI) or URL

Now let's see how the WebClient class made our life easy in implementing the code. In just three steps:

WebClient objWeb = new
            WebClient();
byte[] pagedata;
string pageHTML;
UTF8Encoding
            objUTF8 = new UTF8Encoding();
txtDisplay.Text =
            "";
try
{
pagedata =
            objWeb.DownloadData(txtURL.Text);
pageHTML =
            objUTF8.GetString(pagedata);
txtDisplay.Text =
            pageHTML;
}
catch(Exception ex)
{
txtDisplay.Text =
            "Error Message :" + ex.Message;
}

In the above code we have created a WebClient object. The DownLoadData() method of the WebClient object returns the data in the form of a byte array. It takes the URL as the input parameter. The byte array is then encoded using the UTF8Encoding class (which translates between Unicode and UTF-8), the GetString() method converts the byte array to a string, and the string is displayed in the txtDisplay text control. The output of the above code is shown below:

Let us add one more button and call it GetHTML. This uses the WebClient class again. It downloads the HTML file and displays the message in the display textbox control.

WebClient objWeb = new
            WebClient();
DirectoryInfo oDir = new
            DirectoryInfo(Application.StartupPath +
            "/Test.htm");

try
{
txtDisplay.Text =
            "";
Path.GetFullPath(Application.StartupPath);

//DownLoad
            the file
objWeb.DownloadFile(txtURL.Text,Path.GetFullPath(oDir.ToString()));

//Display
            the Path of the file
txtDisplay.Text = "File Saved at " +
            Path.GetFullPath(oDir.ToString());
}
catch(Exception
            ex)
{
txtDisplay.Text = "Error Message :" +
            ex.Message;
}

In the above code we call the DownloadFile() of the WebClient class, which uses two parameters. One is the location from where you want download the file, and other is the location on your hard disk where you want to save the file.

Let's try to download the home page from my favorite bookstore, Barnes & Noble. After clicking the GetHTML button we get the following message in the txtDisplay control:

now open Test.htm from the local harddisk.

Scenarios:

So far, we have seen how to send and receive data to and from a URI/URL using the WebRequest / WebResponse and WebClient classes. Let us talk about the best scenarios in which to use these classes.

The WebRequest / WebReponse classes can be used for Request/Response like protocol-agnostic communication.

The WebClient class can be used for uploading the values, uploading data file from one place to other and downloading files from a Uniform Resource Identifier (URI) or URL.

Supplying credentials to the remote URL

Now let us add one more button to illustrate how we supply default credentials to the remote URL. To obtain this functionality we use the System.Net.CrendentialCache class. This class stores credentials; applications that access Internet resources can use this class and supply credentials when required. To check for the authentication we have to set a few settings in IIS. Go to the properties of any of the virtual directories for which you want to check the authentication, click Directory Security | Edit and uncheck the Anonymous access checkbox. Then select the Integrated Windows authentication checkbox. This means that we have to supply windows credentials to access any resource under that virtual directory.

Now lets try to get HTML data from a remote URL without sending credentials and see how it responds to our request. Here is the code to get the HTML data without sending credentials:

try
{
HttpWebRequest
            oWebRequest;
HttpWebResponse oWebResponse;
StreamReader
            oStmReader;
String sURL;
String sLine;
//Clear the Display
            Text
txtDisplay.Text = "";
sURL = txtURL.Text;

//
            instantiating the HttpWebRequest

oWebRequest =
            (HttpWebRequest)WebRequest.Create(sURL);
//Get the Response from
            the URL
oWebResponse =
            (HttpWebResponse)oWebRequest.GetResponse();

oStmReader = new
            StreamReader(oWebResponse.GetResponseStream(),
            Encoding.ASCII);
while (oStmReader.Peek() != -1)
{
sLine =
            oStmReader.ReadLine();
txtDisplay.Text = txtDisplay.Text + "\r" +
            sLine;
}
oStmReader.Close();
}
catch(Exception
            ex)
{
MessageBox.Show(ex.Message);
}

When we execute the above code we get the following message:

The code for supplying credentials is shown below. The DefaultCrendentials property of the CrendentialCache class is used and supplied to the Credentials properties of the WebRequest class before making a request to the remote URL.

try
{
HttpWebRequest
            oWebRequest;
HttpWebResponse oWebResponse;
StreamReader
            oStmReader;
String sURL;
String sLine;
//Clear the Display
            Text
txtDisplay.Text = "";
sURL = txtURL.Text;

//
            instantiating the HttpWebRequest
oWebRequest =
            (HttpWebRequest)WebRequest.Create(sURL);

// Supply the
            default credentails to the URL
oWebRequest.Credentials =
            CredentialCache.DefaultCredentials;

//Get the Response from
            the URL
oWebResponse =
            (HttpWebResponse)oWebRequest.GetResponse();
//Display the
            HTML

oStmReader = new
            StreamReader(oWebResponse.GetResponseStream(),
            Encoding.ASCII);
while (oStmReader.Peek() != -1)
{
sLine =
            oStmReader.ReadLine();
txtDisplay.Text = txtDisplay.Text + "\r" +
            sLine;
}
oStmReader.Close();
}
catch(Exception
            ex)
{
MessageBox.Show(ex.Message);
}

In the above code everything is the same except for the statement below:

oWebRequest.Credentials = CredentialCache.DefaultCredentials;

The DefaultCredentials Property of the CredentialCache class is supplied to the WebRequest object. DefaultCredentials represents the system credentials for the current security context in which the application is running, and applies only to NTLM (Windows NT LAN Manager).

After supplying the credentials we can get HTML data as shown below:

Implementing TCPListener and TCPClient

The last topic of this article concerns the creation of TCPListener and TCPClient. The functionality of the TCPListener is to sense if any clients are connected, and, if any, receives the TCP packet from the client and send the data to the client. In our example it sends the XML data to the TCP clients which are pulled from the database. That data can be parsed clientside, but we are just displaying XML data in the txtDisplay text control. The listener starts a different thread using System.Threading namespace. We create the TCPListener as a Console application as shown below.

Let's discuss the Thread, Socket and TCPListener classes before going into the details of the code in the application.

  • The Thread class creates threads. To set the thread priority for execution, the ThreadPriority enumeration is used. The Start() method is used to start the method. To instantiate the thread class we use ThreadStart.
  • The Socket class is used as the transport service for communication in the network. Once the Socket object is created it is connected to an endpoint through the Connect() method. The data is sent using the Send() and SendTo() methods and receives the data using Receive() and ReceiveFrom() methods. Both TCPListener and TCPClient use this class for data transfer.
  • The TCPListener creates a host process to listen the connected TCP clients. The TCPListener is built on the Socket class to give TCP services. The listener uses the Start() method to start the listening service and the AcceptSocket() method to accept the incoming socket connection request. Using the Socket class it sends and receives the data.
  • To create a TCPClient we use the Socket class and its methods for communication to remote host.

The constructor of the class retrieves data from the database in an XML string, and stores it in a member variable:

private TcpListener m_TcpListener;
private int m_Port = 49152;
private static int m_ClientConn=0; //Clients count
private string m_strMessage ;

The m_TcpListener variable is used as a TCPListener class object, m_Port is the port number through which the Listener will listen, m_ClientConn is used to keep track of the number of clients connected to the listener, and m_strMessage is used for storing XML data from the database. The code below pulls the records from the database and stores them in the member variable in string format:

      String
            strSelect = "Select * from authors";
//Connection
            String
String strConn =
            "server=localhost;database=pubs;uid=sa;pwd=;";
SqlConnection
            objConn = new SqlConnection(strConn);
// Open the database
            connection
objConn.Open();

// SqlDataAdapter to excute the
            query
SqlDataAdapter objDataAdp = new
            SqlDataAdapter(strSelect,objConn);

// Fill the records in the
            Dataset object
DataSet objData = new
            DataSet();
objDataAdp.Fill(objData, "authors");

// Get the
            XML data from the dataset object
m_strMessage =
            objData.GetXml();

SqlDataAdapater and DataSet return the records from the database and store them in the member variable m_strMessage. Whenever a client makes a connection, the listener senses the connection and sends the data which is stored in the member variable. The Thread and the TCPListener start code is:

   m_TcpListener = new
            TcpListener(m_Port);
Thread ListenThread = new Thread (new
            ThreadStart(Listener));
ListenThread.Name =
            "TCPListenerServer";
ListenThread.Priority =
            ThreadPriority.Highest;
ListenThread.Start();

In the above code we instantiated the TCPListener by providing the port number - remember we should use the same port for the client when we are connecting to the listener. Using the Start method we can start the listener. When we instantiate the Thread we call the ThreadStart delegate by passing the void function Listener() to it. In this function the listener listens to the client sockets and receives and sends the data. Let's see the code:

Public void Listener()
{
Console.WriteLine ("Working Thread: {0}", Thread.CurrentThread.Name);
While (true)
{
Socket objSocket = m_TcpListener.AcceptSocket ();
If (objSocket.Connected)
{
m_ClientConn = m_ClientConn + 1;
Console.WriteLine ("Number of clients connected are {0}",m_ClientConn);
Byte[] byteReceived =new Byte[100];
// received request(data) from client
int nReceived = objSocket.Receive(byteReceived, byteReceived.Length,0);
Console.WriteLine ("Length of the data received is: {0} ", nReceived);
string strReceived = System.Text.Encoding.ASCII.GetString(byteReceived);
Console.WriteLine ("Received data is: "+strReceived);
// Sending Data to the client
Byte[] byteXML =System.Text.Encoding.ASCII.GetBytes(m_strMessage.ToCharArray());
objSocket.Send(byteXML,byteXML. Length, 0);
}
else
{
Console.WriteLine("Socket is not connected");
}
}
}

The AcceptSocket() method of the TCPListener class gives the socket instance to handle the incoming request. Once we have the socket object, we can receive the incoming message using the Receive() method, the message is received in terms of bytes, and then the bytes are encoded to string. To send the message to the client we have to encode the string message to bytes and then use the Send() method. We can send or receive the data only in terms of bytes. After executing the above code we will see the command window as shown below:

If any client is connected and sends the data to the same port as the listener, then the listener displays the number clients connected, the length of the data is displayed, and the string is displayed as shown below.

To integrate this with a windows form, we copy the TCPListener.exe to the Networking\bin\Debug folder, create one more button on the form and name it TCPListener. In the button click event, use the Process.Start() method to run the process, as shown below:

Process.Start(Application.StartupPath + "/TCPListener.exe");

Now lets look at the code that talks to the TCPListener. There are a few classes besides the Socket class - IPAddress, IPEndPoint and Dns.

  • Dns is a sealed class, this class doesn't need to instantiate the object and methods of this class can be called directly. This class can be used to retrieve information about a specific host from the Internet Domain Name System (DNS). Its Resolve() method queries a DNS server for the IP address associated it.
  • The IPAddress class contains the address of a computer on an IP (Internet Protocol) network.
  • The IPEndPoint class contains the host and port information needed by an application to connect to a service on a host. By combining the host's IP address and port number of a service, the IPEndPoint class forms a connection point to a service.

The code shown below makes a connection to the listener, sends and receives data, and displays it in the display textbox control. Lets create one more button called TCPClient and put all the code shown below in the button click event:

int nCommPort = 49152;
string HostName =
            "localhost";
Socket TcpSocket;
IPAddress HostIP =
            Dns.Resolve(HostName).AddressList[0];
IPEndPoint HostEP = new
            IPEndPoint(HostIP,nCommPort);
TcpSocket = new
            Socket(AddressFamily.InterNetwork,SocketType.Stream,ProtocolType.Tcp);

//
            Get the client name
string strHome = Dns.GetHostName();
string
            strMessage = "Request from " + strHome;
byte[] byteMessage =
            System.Text.Encoding.ASCII.GetBytes(strMessage.ToCharArray());

//
            Connect to
            Server
try
{
TcpSocket.Connect(HostEP);
if(TcpSocket.Connected)
{
TcpSocket.Send(byteMessage,byteMessage.Length,0);

//
            receive from server

StringBuilder objStrBuilder = new
            StringBuilder();
byte[] byteReceived = new byte[50];
Int32
            nReceived =
            TcpSocket.Receive(byteReceived,byteReceived.Length,0);
objStrBuilder.Append(System.Text.Encoding.ASCII.GetString(byteReceived,0,nReceived));

//
            read the rest of the
            data

while(!(nReceived<byteReceived.Length))
{
nReceived
            =
            TcpSocket.Receive(byteReceived,byteReceived.Length,0);
objStrBuilder.Append(System.Text.Encoding.ASCII.GetString(byteReceived,0,nReceived));

}

string
            strReceived = objStrBuilder.ToString();
txtDisplay.Text =
            "Received Message from Server: \r" ;
txtDisplay.Text =
            txtDisplay.Text +
            strReceived;

}
else
{
txtDisplay.Text = "Unable to
            connect to the server";
}
}
catch(Exception
            ex)
{
txtDisplay.Text = ex.Message;
}

The Dns.Resolve(HostName).AddressList[0] gives the IP address of the Listener - in our case the Listener and Client is on the same machine. The IPEndPoint is created by supplying the host IP address and Port number. In creating the socket we use the AddressFamily enumeration to specify the addressing scheme that a socket instance uses to resolve an IP address. We use the Socket.Type enumeration to specify the type (either stream/Raw/SeqPacket) and finally the Protocol type enumeration specifies the protocols that the socket object supports. By supplying the IPEndPoint object to the Connect() method of the Socket object we can connect to the listener. The Send() method of the socket is used to send the data to the listener, and the Receive() method is used to receive the data from the listener in Bytes and encoded to string. The received data from the listener is shown below:

Conclusion

In this article we have seen the following:

  • How to get HTML data using various classes provided by .NET framework.
  • How to download files from a remote URL.
  • How to Supply Default Credentials to a Remote URL.
  • How to Create a TCPListener and a TCPClient, and communicate between them.

I'm sure anyone who has written networking programs in the past can attest that the .NET way seems much cleaner and easier.

If you'd like to discuss a broader range of C#Today topics and issues, or if you have any C# related questions you need answers to, please visit our online resource centres at : http://p2p.wrox.com/list.asp?list=csharptoday_discuss&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq and http://p2p.wrox.com/list.asp?list=c_sharp&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq
C#Today - © 2002 WroxPress Limited


Developing a Generic Thread Pool in C#

Developing a Generic Thread Pool in C#
http://www.csharptoday.com/content.asp?id=1739
Article Rating: 4
Published on October 30, 2001
Abstract
One of the C# language’s most powerful capabilities is its built-in support for threads. Threads are a powerful abstraction for allowing non-blocking parallel operations. Often there is a need to perform operations asynchronously. In languages like C/C++, you either have to write your own library from scratch, or use a third party solution. If continuous creation of these threads in any program is not controlled, the machine will run out of memory pretty soon. In this article, Tejaswi Redkar looks at the concept of thread pooling, and why it is important to use thread pools in server systems. Along the way, we will also develop a generic thread pool library that will be available for use on a regular basis.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

One of the C# language's most powerful capabilities is its built-in support for threads. Threads are a powerful abstraction for allowing non-blocking parallel operations. Often there is a need to perform operations asynchronously. In languages like C/C++, you either have to write your own library from scratch, or use a third party solution. If continuous creation of these threads in any program is not controlled, the machine will run out of memory pretty soon. In this article, we will go over the concept of thread pooling and why it is important to use thread pools in server systems. Along the way, we will also develop a generic thread pool library that will be available for use on a regular basis.

This article assumes that the reader has the following pre-requisites:

  • Knowledge of at least one object-oriented language (e.g. C#, C++, etc)
  • Exposure to .NET Technologies, and one .NET-based language like VB.NET or C#

System requirements:

  • Windows 2000 Professional or Server plus Service Pack2
  • .NET Framework Beta 2
  • Visual Studio .NET is not a must, but speeds up the development process

Multithreading vs. Multitasking

Multitasking

Multitasking is the ability of an operating system to execute multiple processes simultaneously. For example, in figure 1 below, the operating system runs SQL Server, MSMQ and COM+ runtime at the same time. There three different processes do not necessarily know about each other, although there are several ways in which the processes can communicate with each other.

Figure 1: Multitasking Overview

Multithreading

Multithreading is the ability of a process to execute different pieces of code simultaneously.

For example, figure 2 shows a process with multiple threads running within a single instance of the Common Language Runtime (CLR). Thus, the relationship between a thread and a process is similar to the relationship between the operating system and a process. Every thread has its own local storage space called Thread Local Storage, which it does not share with any other thread. However, there are some exceptions, like static variables, which can be shared across different threads in the same process.

Figure 2: Multithreading overview

Use of Multithreading in Enterprise Applications

Multithreading opens up design possibilities that are impractical in sequential programs. Threads liberate you from using blocking method invocations wherever they are not needed.

Using threads, you can additionally trigger new independent activities than run concurrently, with or without waiting for their completion. Threads are widely used in enterprise applications where some component might be reading data from a database table and another thread might be sending a message to a message queue. Even SQL Server and MSMQ use threads extensively. For example, you can create multiple message queue objects in your program, and allocate each one a separate thread for sending message(s) to the queue. This makes programming with MSMQ more scalable and also improves the performance.

Developing a Simple Threading application in C#

In this section, we will go over the development process of creating a simple multithreaded application in C#.

using System;
//Reference the System.Threading namespace.
//The namespace contains all the classes required for thread programming in the .NET framework.
using System.Threading;
public class MyFirstThread {
public static void Main(){
//Create an array of threads of size 5
Thread[] myThreads = new Thread[5];

for(int i = 0 ; i < myThreads.Length ; i++){
//Create the Thread and its delegate for method Count
myThreads[i] = new Thread(new ThreadStart(Count));
//Start the Thread
myThreads[i].Start();
}//end of for loop
}//end of Main

public static void Count() {
for (int count = 1 ; count <= 10 ; count++)
Console.Write(count + " ");
Console.Write(" | ");

}//end of Count
}//end of class

Compiling and Running MyFirstThread.cs

To compile and run the MyFirstThreat.cs, use the following two statements at the prompt:

Command Prompt>csc /t:exe MyFirstThread.cs

Command Prompt> MyFirstThread

Figure 3: Compiling and Running MyFirstThread.cs

Code Explanation

  • Import the System and System.Threading namespace
  • Create class MyFirstthread
  • Create the Main() method to run it as an executable. The Main method is the entry point of any executable program
  • Create an array of Threads of size 5
  • Instantiate each Thread in the array

Note: The instantiation of Thread requires creating a ThreadStart delegate that takes the method name as the parameter. This method will be called when you start the thread. Delegates are similar to function pointers in C/C++. We have to pass a function name that specifies the starting point of our thread to this delegate.

  • Start each Thread in the array

Note: Thread won't run until you invoke the Start() method on it.

In the .NET platform, Threading classes are included in the System.Threading namespace. In C#, a thread is a sealed class. This means that you cannot subclass it, and it must be constructed with a ThreadStart delegate that refers to a method that has both its parameters and return values as void. This method will be called when you invoke the Start() method on the Thread object.

The Importance of Thread Pooling

In a typical database application, the database connections are created on the fly by the database-client applications. While designing applications, the application developer seldom knows about the number of connections that might be created simultaneously at run-time. As the use of the client application increases, the number of database connections increase, and at some point the database may become incapable of handling more than a certain number of connections due to licensing or resource limitations. A good solution to this problem is to create a database connection pool that will store pre-created database connections, and specify the maximum and minimum number of connections that can be stored in the pool. This approach will also improve performance, as the application won't have the burden of creating the database connection objects, it will just lease an object from the pool, and then return it to the pool for recycling once it is done. To comparing database connections to threads, an application might hit resource limitations even while creating new threads for every client request.

It has long been known that one of the ways of improving the performance of a software application is to reduce the overall number of objects created and destroyed. C# programs are no exception to this. Also, the performance degrades when you create a new object for every request, especially with regards to the run-time environment garbage collection performance - many short-lived objects may result in overactive garbage collection. Each object consumes some amount of memory. The larger the set of objects, the larger the amount of memory used, which may lead to excessive paging and thus hamper performance.

In an effort to solve the same old problem of resource limitation and performance, I have created a generic thread pool library that can be used in any .NET application, wherever thread pooling is needed.

Developing a generic Thread Pool library

Creating an interface

Figure 4 shows the UML class diagram for our ThreadPool interface.

Figure 4: UML class diagram for the Thread Pool interface

The thread pool design starts with an interface IThreadPool that needs to be implemented either by an implementation class or the actual thread pool class. Interfaces abstract the functionality from implementation and provide plugability. For example, any class that wishes to provide the thread pool functionality can implement the IThreadPool interface. This approach provides a choice to the client application between different thread pool implementations at run-time.

A good example of interface abstraction is the relationship between a Television (TV) and a Video Cassette Recorder (VCR ) or a DVD player. You can switch to your VCR without modifying the TV, or vice versa, as long as they both understand the interface. Thus, an interface provides abstraction, plugability, and acts as an agreement or contract between the client application and the server implementation.

Creating the implementation class

Figure 5: GenThreadPoolImpl class

Figure 5 shows the structure of the GenThreadPoolImpl class. The GenThreadPoolImpl class implements the IThreadPool interface, and so it provides the implementation of the addJob and getStats operations of the IThreadPool interface.

Description of GenThreadPoolImpl class

Properties

  • MaxIdleTime: Represent the maximum idle time in milliseconds that a thread can remain in the thread pool.
  • MaxThreads: Represents the maximum number of threads that the thread pool can hold.
  • MinThreads: Represents the minimum number of threads that should always be present in the thread pool.

MaxIdleTime does not apply to the minimum number of threads in the pool, because there should always be a minimum number of threads present in the thread pool, irrespective of the MaxIdleTime.

  • Debug: Has a Boolean value. When true, it prints the state of the thread pool to the console after every invocation.

Attributes

Note: ArrayList are variable length Arrays. By default, ArrayLists are not synchronized. To synchronize an ArrayList, you need to add a synchronized wrapper to it. Adding the synchronized wrapper makes the ArrayList thread safe, and as a result, multiple threads will be able to access the objects in it without any race condition(s).

  • ArrayList _noSyncPendingJobs: Represents the total number of pending jobs in the thread pool.
  • ArrayList _pendingJobs: Represents a synchronized wrapper over the _noSyncPendingJobs Arraylist.
  • ArrayList _noSyncAvailableThreads: Represents the total number of available threads at any time in the thread pool
  • ArrayList _availableThreads: Represents a synchronized wrapper over the _noSyncAvailableThreads Arraylist.

Adding a synchronized wrapper makes our ArrayList thread-safe. Inherently, ArrayLists are not thread safe, and can give unexpected results if used in multi-threaded applications.

Constructors

  • public GenThreadPoolImpl(int maxThreads, int minThreads, int maxIdleTime, bool debug_): Initializes the GenThreadPoolImpl with maxThreads, minThreads, minIdleTime, and debug mode.
  • public GenThreadPoolImpl(): Initializes the GenThreadPoolImpl to default values in the GenThreadPoolImpl constructor

In both the constructors we should add synchronized wrappers to the _noSyncAvailableThreads and _noSyncPendingJobs ArrayLists. For example:

_noSyncAvailableThreads = new ArrayList
            ();
this._availableThreads = ArrayList.Synchronized
            (this._noSyncAvailableThreads);

Operations/Methods

  • void addJob(Thread job): The addJob() method takes a job as a parameter, which is an object instance of a Thread class. The thread class must have been initialized, but not started, before passing it to the addJob() operation. This method is non-blocking - it does not return anything.

The addJob() operation carries out the following functionality:

  • Locks the current object while processing the job, so that only one object can be added to the pool at a time. This makes the pool thread safe and easier to manage and debug.

Lock(this)

  • Adds the job to the _pendingJobs ArrayList.

_pendingJobs.Add (job);

  • Get the index of the first idle thread in the pool

Int index = findFirstIdleThread();

  • If there are no free threads, checks if the maximum size of the pool has been reached, and then if needed, creates a new thread and starts it.

//check the Thread availability
if((_maxThreads == -1) ||( _availableThreads.Count < _maxThreads))

//create a new Thread and start it
Thread t = new Thread(new ThreadStart (new GenPool(this, this) run ));

  • If there are free threads, it reuses the existing threads and sends a notification to all the waiting threads with the Monitor.Pulse static method.

//Get the available thread
lock(((ThreadElement)_availableThreads[index]).getMyThread ())
{
//Sends a notification to all the waiting threads
Monitor.Pulse ((ThreadElement)_availableThreads[index]).getMyThread ());

  • int findFirstIdleThread(): This method checks the _availableThreads ArrayList for the first available or idle thread, and returns the index of that thread in the Arraylist.
  • int findIdleThreadCount(): This method goes through the _availableThreads ArrayList and returns the total number of idle threads in the thread pool.
  • int findThread(): This method finds and returns the index of the current thread in _availableThreads ArrayList.
  • void removeThread(): This method removes this (current) thread from the _availableThreads Arraylist.
  • getStats(): Return the current statistics of the thread pool. It returns a Stats structure that has the following representation (Figure 6):

Figure 6: Stats structure

As we can see from the diagram, the Stats structure can be broken down into the following pieces:

  • maxThreads is the maximum number of threads that the thread pool will hold
  • minThreads is the minimum number of threads that should always be active in the threads pool
  • maxIdleTime is the maximum time for which a thread can remain idle in the pool before it can be garbage collected
  • pendingJobs are the jobs that are pending in the pool or waiting for a thread to be assigned to them
  • jobsInProgress is the number of jobs that are in progress currently

GenPool Class

The following figure represents the GenPool class:

Figure 7: GenPool class

Each thread in the pool is an instance of the GenPool class. The GenPool class has a run() method that is supplied to the ThreadStart delegate while initializing the Thread. The implementation of the run method (or, in other words, the thread) starts off by checking the pending jobs queue, and keeps processing jobs until none are available. It then waits for the idle time during construction, if one was specified, or until it is notified of a new job, whichever happens first. After waking up, the thread checks if it needs to destroy itself, which happens if it has been idling too long (as indicated by the idle time configuration parameter) and there are more than the minimum threads required in the pool. This loop of three actions continues for as long as the thread is in existence.

How to use the Thread Pool

In the project mentioned above, we have developed a GenThreadPool library (DLL) that can be reused in any application of the developer's choice. In this section, we will develop a sample test application to demonstrate the use of GenThreadPool library.

Step 1: Create a Console Application (see figure 8 below)

Figure 8: Console Application

Step 2: Name the console application TestGenThreadPool

Step 3: Rename Class1 to TestPool.cs

Step 4: Add a reference to the GenThreadPool.dll from Project\Add Reference menu

Step 5: Reference the GenThreadPool namespace that we just added in Step 4

using GenThreadPool;

Step 6: In the main method, create an instance of the GenThreadPoolImpl class:

GenThreadPool.IThreadPool tp = new GenThreadPoolImpl (1000, 10, 300, true);

Where:

  • First parameter, set to 1000, is the maximum number of threads in the thread pool.
  • Second parameter, set to 10, is the minimum number of threads in the thread pool.
  • Third parameter, set to 300, is the maximum idle time that a thread can stay in the thread pool before it is destroyed and garbage collected.
  • Fourth parameter, set to true, indicates that the thread pool is run in debug mode. In debug more, the thread pool prints out messages that can be useful to trace the working of the thread pool.

Step 7: Create a method in TestPool to pass to the ThreadStart delegate. This method will be executed when the thread pool runs this thread.

Step 8: Create a thread object and its delegate:

Thread t1 = new Thread (new ThreadStart (new TestPool () run ));

Please note the passing of the TestPool object along with its run method. Also note that the thread won't run till you call Start method on it.

Step 9: Add the thread to the thread pool

Tp.addJob (t1);

You can add as many threads as you want to the thread pool, and the thread pool will execute them. Also remember that the creation of threads depends on the system memory available and the CPU capacity. Sometimes you might get an Out of Memory exception if the number of threads increases beyond the capacity of the memory.

Developing a MSMQ Message Sender using the Thread Pool

In a large-scale MSMQ environment, it is often necessary to send multiple messages to the MSMQ from a single client. For example, in an email scheduler system, the email scheduler queries the database for new emails every 30 minutes, and then sends all the retrieved email to MSMQ for further processing. In this case, the scheduler has to create a new thread for sending each message to the MSMQ. If there are thousands of new emails every five minutes, then the scheduler will have to create thousands of new threads, one for each email message. Creation of a new thread for every message requires new memory to be allocated, and also stresses the garbage collector to scavenge these newly created resources. Potentially, this makes the system very slow to respond, because a lot of the time is required for garbage collection. To solve this problem, we can use our GenThreadPool to send messages to the MSMQ. GenThreadPool will pool the threads and reuse them whenever required, thus avoiding the creation of new threads all the time.

Brief Description of the Application (MSMQConsole)

Using the MSMQConsole, you can do the following:

  • Create Queues
  • Send Messages to the MSMQ
  • Receive Messages from MSMQ

MSMQConsole is a handy tool for testing MSMQ connectivity.

In this article, we are concerned only with the sending of messages to the MSMQ.

STEP 1: Creating the Queue

Figure 9: Create Queue

STEP 2: Send Messages

In Figure 9, MSMQConsole sends messages to MSMQ. Note that the default number of messages is 500, i.e. MSMQConsole will send 500 similar messages to the MSMQ, and each message will be sent using a thread. MSMQConsole dumps the thread pool statistics on the list box after every 10 messages sent. This will help us figure out how the thread pool operates. We can also dump the ThreadPool statistics by clicking the ThreadPool Stats button.

Figure 10: Send Messages

STEP 3: Receiving Messages from MSMQ

Figure 10 shows how to receive messages from MSMQ synchronously using the MSMQConsole. We can also receive the messages from MSMQ asynchronously, but this is out of scope of this article. GenThreadPool will be very useful if we were to receive thousands of messages simultaneously from the MSMQ and then forward them for further processing. This is left as an assignment for the reader.

Figure 11: Receive Messages

Improvements needed to the GenThreadPool

GenThreadPool can be further enhanced to support the following features:

Thread Priorities

Currently all new threads that are created have the same priority as the thread on which the addJob() call is being made when that thread is created. The GenThreadPool can be tested for threads with different priorities. To do this we need to create a thread:

Thread t = new Thread (new ThreadStart (this run ));

Then set ThreadPriority :

t.priority = ThreadPriority.Highest;

and then add it to the thread pool:

tp.addJob (t);

Daemon Threads

Currently threads in the thread pool are not daemon threads. As a result, the pool will force the application to stay alive even if there is a single thread active. Threads that remain active in the background, even after the application stops, are call daemon threads. The GenThreadPool can also be tested with daemon threads by adding threads with their IsBackground property set to true.

Conclusion

We have seen how to create a generic thread pool library that can be reused in any multithreaded application. However, it is extremely important to remember that using thread pools will not automatically improve the performance of a poorly performing C# program. The performance of a program depends on many factors, only one of which is thread management. For a small number of jobs, the overhead of using a thread pool exceeds the overhead of creating and destroying a new thread for each job. So, one should not get carried away with the using a thread pool. The design should be pragmatic, based upon a lot of factors, such as the number of threads created by the application in a specific period of time.

If you'd like to discuss a broader range of C#Today topics and issues, or if you have any C# related questions you need answers to, please visit our online resource centres at : http://p2p.wrox.com/list.asp?list=csharptoday_discuss&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq and http://p2p.wrox.com/list.asp?list=c_sharp&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq
C#Today - © 2002 WroxPress Limited


Monitoring Processes and Threads in .NET

Monitoring Processes and Threads in .NET
http://www.csharptoday.com/content.asp?id=1687
Article Rating: 4
Published on October 24, 2001
Abstract
In this article, Melanie Talmadge shows us how to write a process and thread monitoring system in .NET using C#. We cover the System.Diagnostics namespace with its Process and ProcessThread classes for working with process and threads, and write our own Task Manager application, similar to that which comes with Windows NT and Windows 2000. This program displays all the running processes, their related resources and related threads.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

A process is an instance of a running application on a system. A process itself is nothing besides memory address space. Besides the space, a process also owns resources including files, dynamic and virtual memory allocations, and threads. Each process has a unique ID. For a process to accomplish anything, it must own thread(s). Each process has at least one main thread, and can have any number of secondary threads, also known as worker threads.

A thread is a path of execution that executes some part of code in a predefined manner. A thread executes this code in a processes address space. Each thread has a unique ID, and allocates its own resources such as CPU registers and stack.

In this article, we will discuss how to return all the available processes on a machine, and all the threads corresponding to a process. The sample application we develop shows the total number of running processes, their IDs, and corresponding information such as physical and virtual memory, starting time, etc. This program also lets us monitor available threads, information that is not available in Windows Task Manager.

System.Diagnostics Namespace

In the .NET Framework, the System.Diagnostics namespace defines classes that allow us to debug and trace applications, read and write event logs, and monitor processes and system performance using performance counters. The major classes include Debug, Counter, EventLog, Process, ProcessThread, StackTrace, and Trace.

A discussion of all these classes is out of the scope of this article - we will cover only the Process class and its related classes.

The Process Class

In .NET, the Process class enables us to start and stop processes on a local or a remote machine. The Process class also enables us to monitor system resources, such as all running processes, their ID, threads corresponding to process IDs, and threads related to each process. It also lets us monitor resources allocated by processes and threads, such as physical memory, paged memory, virtual memory, peak paged memory, and peaked virtual memory size.

Creating an instance and get all running Processes

There are two ways to create an instance of a Process class. Either we call the Process constructor followed by the Start method to start a process, or we can directly call the GetProcesses method, which returns all the running processes on a machine. The Process class constructor doesn't take any parameters. See the following definition of the Process constructor.

Using System.Diagnostics;

public
            Process();

Process prcss = new
            Process();
prcss.Start();

OR:

Process[] procs = Process.GetProcesses();

Get all the running processes

The GetProcesses method of the Process class returns all the running processes on a machine. The following code returns all the processes and writes to the system console.

Process[] procs = Process.GetProcesses();
foreach(Process proc in procs)
{
Console.Write("Process Name:"+proc.ProcessName +",");
}

Get Process Names and Process IDs

We use the Process Name and Id properties to get process name and process IDs of the processes:

Process[] procs = Process.GetProcesses();
foreach(Process proc in procs)
{
Console.Write("Process Name:"+proc.ProcessName +",");
Console.Write("Process Id:"+ proc.Id.ToString()+",");
}

Get Process Resources Information

By using the Process class we can even discover resource data such as physical and virtual memory for a process. The WorkingSet property returns the physical memory and VirtualMemorySize returns the virtual memory, in bytes, for a process.

Process[] procs =
            Process.GetProcesses();
foreach(Process proc in
            procs)
{
Console.Write("Virtual Memory Size:"+
            proc.VirtualMemorySize.ToString()+",");
Console.Write("Physical
            Memory:"+
            proc.WorkingSet.ToString()+",");
proc.WorkingSet.ToString()+",");
}
}

Get Process Time

We discover the startup time of a process, by using the StartTime property.

Process[] procs = Process.GetProcesses();
foreach(Process proc in procs)
{
string stratTime = proc.StartTime.ToString();
string threadPriority = proc.BasePriority.ToString();
}

Get Thread Priority

We can even gather other information, such as the priority of a thread, with the BasePriority property:

Process[] procs = Process.GetProcesses();
foreach(Process proc in procs)
{
string threadPriority = procList[i].BasePriority.ToString();
}

Return all the threads corresponding to a process

We use the Threads property to return all the running threads corresponding to a process:

Public ProcessThread [] Threads();

Return Maximum and Minimum physical memory

We use MaxWorkingSet and MinWorkingSet members to get the maximum and minimum values of a working set.

Getting Processor Time

the UserProcessorTime and TotalProcessorTime properties return the time that the process has spent running since it started in this application, and usage time of the CPU respectively.

Start a Process

We can use the Start method of the Process class to start a method. The Start method takes one parameter, the file name of the .exe we want to start:

Process.Start(FileName);

We can even use the GetProcessByName and GetProcessById methods to get a process by using names and IDs. Both of the methods return a Process type.

Killing a Process

The Kill method is used to abort a process. We must ensure that procs is the process we want to abort. As discussed earlier, procs is an array of Process types.

Process[] procs = Process.GetProcesses();

..... ...

procs[i].Kill();

Or:

Process.Kill();

Freeing Resources

The Close method of the Process class frees all resources associated with a process.

procs[i].Close();

Be careful when using the Kill method to terminate a process. The Kill method causes abnormal termination of an application and should be used carefully. There are chances of losing data if we call the Kill method. However using the Close method is safe. The Close method makes sure that the process has finished its message loop, finished its job and then requests termination. There may be the possibility that Close will ask user to close an application.

Sample Program:

This simple program displays all the running processes and their corresponding information.

Listing ProcInfo.cs Sample Code

using System;
using System.Diagnostics;

namespace ProcessInfo
{
/// <summary>
/// Summary description for Class1.
/// </summary>
class Class1
{
static void Main(string[] args)
{
Process[] procs = Process.GetProcesses();
foreach(Process proc in procs)
{
// Process Name
Console.WriteLine("Process Name:"+proc.ProcessName +",");
// Process Start Time
Console.WriteLine("Start Time:"+ proc.StartTime+",");
// Process ID
Console.WriteLine("Process Id:"+ proc.Id.ToString()+",");
// Process Virtual Memory
Console.WriteLine("Virtual Memory Size:"+ proc.VirtualMemorySize.ToString()+",");
// Process Physical Memory
Console.WriteLine("Physical Memory:"+ proc.WorkingSet.ToString()+",");

// Other resources
Console.WriteLine("Paged Memory:"+ proc.PagedMemorySize.ToString()+",");
Console.WriteLine("Peakworkingset Memory:"+ proc.PeakWorkingSet.ToString()+",");
Console.WriteLine("Total Processor Time:"+ proc.TotalProcessorTime.ToString()+",");
Console.WriteLine("Peak virtual Memory:"+ proc.PeakVirtualMemorySize.ToString()+",");
Console.WriteLine("User Processor Time:"+ proc.UserProcessorTime.ToString()+",");
Console.WriteLine("Prev. Processor Time:"+ proc.PrivilegedProcessorTime.ToString()+",");
Console.WriteLine("============================================");
}

}
}
}

The ProcessThread Class

The ProcessThread class represents a Win32 thread. This class can be used to gather information about a thread such as thread ID, starting time of a thread, base and current priorities, address of the function called when the thread was started, and more.

Get threads

We use the Threads member of the ProcessThreads class to return an array containing all the threads for that process:

public ProcessThread [] Threads();
ProcessThread [] threads = proc.Threads;

Returning a thread ID

Every thread has a unique ID. We can call the Id property of the thread to get the unique ID of a thread.

private ProcessThread [] m_Threads;
string str = m_Threads[i].Id.ToString();

Return the priority of a Thread

A process has two types of priorities - base and current. The base priority is computed by combining the process priority class with the priority level of the associated thread. The base priority of the thread changes depending on the time and resource allocation by the Operating System. The current priority might be different to the base priority depending on how the OS schedules the thread.

The BasePriority and CurrentPriority properties return the base and the current priority of a thread.

Strstr = m_Threads[i].BasePriority.ToString();
Strstr = m_Threads[i].CurrentPriority.ToString();

How Much Time Has The Thread Spent Running?

You can also find out how much time a thread has spent using the CPU or running in the current process by using UserProcessorTime, TotalProcessorTime, and PrivilegedProcessorTime properties.

str = m_Threads[i].UserProcessorTime.ToString();
str = m_Threads[i].TotalProcessorTime.ToString();

Thread State

We can use the ThreadState property to find out the current state of the thread - whether it is Initialized, Running, Waiting, etc. This property returns the ThreadState enum type:

Strstr = m_Threads[i].ThreadState.ToString();

The ThreadState enum members are described in the following table:

Member | Description
Initialized | Thread has been initialized, but has not started yet.
Ready | Thread is in ready state.
Running | Thread is running.
Standby | Thread is in standby state.
Terminated | Thread has exited.
Transition | Thread is transitioning between states.
Unknown | Thread state is unknown.
Wait | Thread is waiting.

The Sample Project - The ProcessMonitor

Now it's time to implement the above concepts in an application. This application lets us start or abort a process, and displays information such as the name of the processes running on the machine, their IDs, physical and virtual memory usage, and their priorities. It looks something like this:

The Code

We use a ListView control in our application to display process information. The InitializeListView method adds column headers, their names and sizes to the ListView control. We'll see this method in a moment.

A row in a ListView control is represented as a ListViewItem. To add a row to the ListView, we need to create an object of a ListViewItem class, call the ListView.Items.Add method, and then pass ListViewItem as an argument.

// Add Item to the list view
ListViewItem
            item = new ListViewItem(str,
            0);
listView1.Items.Add(item);

InitializeListView() Method

This method adds column headers, names, and sizes, and sets other column properties. The Add method of ListViewColumnCollection adds a column header to the control. This method takes the name of the column, the size, and the alignment as arguments.

// Initializes ListView
private void InitializeListView()
{
// Add ListView Column Headers
listView1.Columns.Add("Process", 70, HorizontalAlignment.Left);
listView1.Columns.Add("Proc ID", 70, HorizontalAlignment.Left);
listView1.Columns.Add("Priority", 70, HorizontalAlignment.Left);
listView1.Columns.Add("Physical Mem(KB)", 90, HorizontalAlignment.Left);
listView1.Columns.Add("Virtual Mem(KB)", 90, HorizontalAlignment.Left);
listView1.Columns.Add("Start Time", 150, HorizontalAlignment.Left);
}

FillThreadView() Method

The FillThreadView method is the method that reads all the processes, and fills the list box with process information.

// Initializes ListView
private void
            FillThreadView()
{
string [] str = new String[6];
// Get
            all the running processes on the system
procs =
            Process.GetProcesses();
nProcessCount =
            procs.Length;
textBox1.Text =
            nProcessCount.ToString();

foreach(Process proc in
            procs)
{
// Physical Mem in KB
long physicalMem =
            proc.WorkingSet/1024;
long virtualMem =
            proc.VirtualMemorySize/1024;
str[0] =
            proc.ProcessName.ToString();
str[1] =
            proc.Id.ToString();
str[2] =
            proc.BasePriority.ToString();
str[3] =
            physicalMem.ToString();
str[4] = virtualMem.ToString();
str[5]
            = proc.StartTime.ToString();

// Add Item to the list
            view
ListViewItem item = new ListViewItem(str,
            0);
listView1.Items.Add(item);
}
}

Start Button

The Start Process button let us browse our machine directory, and pick an .exe program we want to launch. Here we use Process.Start(filename) to start a process.

// Start Process
private void
            button1_Click(object sender, System.EventArgs
            e)
{
OpenFileDialog fdlg = new OpenFileDialog();
fdlg.Title = "Open a Program" ;
fdlg.InitialDirectory =
            @"c:\" ;
fdlg.Filter = "Exe files (*.exe)|*.*|All files
            (*.*)|*.*" ;
fdlg.FilterIndex = 2 ;
fdlg.RestoreDirectory =
            true ;
if(fdlg.ShowDialog() == DialogResult.OK)
{
Process.Start(fdlg.FileName);
}
}

Kill Process

The Kill Process button aborts the selected process in the list. ListView's SelectedItems property returns the selected row. We call the Process.Kill method to kill a process.

// Kill Process
private void
            button2_Click(object sender, System.EventArgs
            e)
{
try
{
ListViewItem li =
            listView1.SelectedItems[0];
string str =
            listView1.SelectedItems[0].SubItems[1].Text;
nCurProcId =
            Convert.ToInt32(str);

for ( int i=0; i<nProcessCount;
            i++)
{
if ( procs[i].Id == nCurProcId
            )
{
procs[i].Kill();
return;
}
}
listView1.Items.Remove(li);
}
catch (Exception exp)
{
Console.WriteLine
            (exp.Message);
return ;
}
}

Refresh Button

The Refresh button refreshes the processes information in the list view, which calls the RefreshList method. The RefreshList method reloads the process information.

// Refresh
private void
            button3_Click(object sender, System.EventArgs
            e)
{
RefreshList();
}

protected void RefreshList
            ()
{
listView1.Clear();
InitializeListView();
FillThreadView();
}

Thread Info Button

The Thread Info button launches a new form, which displays information about threads for the selected process in the listview.

// Thread Info
private void
            button4_Click(object sender, System.EventArgs
            e)
{
ListViewItem li = listView1.SelectedItems[0];
string
            str = listView1.SelectedItems[0].SubItems[1].Text;
int nCurProcId
            = Convert.ToInt32(str);

Form2 frm = new
            Form2();
frm.DisplayForm(nCurProcId);
}

The other boxes on the form:

  • Total Processes: Displays the total number of processes running on the machine.
  • Auto Refresh: If we check the Auto Refresh check box, our application refreshes the process list and the data every second.

The Thread Information Form

Form2 is a Thread Monitor. This displays the total threads running corresponding to a given process, and returns their statistics such as ID, Current Priorities, thread state, and start time.

The ProcessThread Class

Use ProcessThread to get information about a thread that is currently running on the system. Basically, a process can have one or more threads and the code, data, and other resources of a program in memory. The major ProcessThread class properties are described in the following table, which we're going to use in our program later.

The ProcessThread Class Properties

BasePriority, CurrentPriority | Base and current priorities of a thread.
Id | Thread ID.
PriorityLevel | Priority level of the thread.
PrivilegedProcessorTime | Amount of time that a thread has spent running.
StartAddress | Address of the function that started this thread.
StartTime | Thread starting time.
ThreadState | Current state of a thread.
TotalProcessorTime | Total amount of time thread has spent using the processor.
UserProcessorTime | Amount of time a thread has spent running code inside the application.

The DisplayForm Method

The DisplayForm method shows the form and calls the FillThreadList method. We'll discuss this method in a moment.

// Fills list with all available threads and
            other info for a process
public void DisplayForm ( int nProcID
            )
{
procId = nProcID;
this.Show();
m_Threads =
            GetThreads(nProcID);
FillThreadList(nProcID );
}

The InitializeThreadList Method

The InitializeThreadList method initializes list view column headers. Its pretty similar to what we've seen above.

private void InitializeThreadList()
{
// Add ListView Column Headers
threadListView.Columns.Add("ID", 50, HorizontalAlignment.Left);
threadListView.Columns.Add("Priority", 70, HorizontalAlignment.Left);
threadListView.Columns.Add("Base Priority", 70, HorizontalAlignment.Left);
threadListView.Columns.Add("Tot Proc Time", 90, HorizontalAlignment.Left);
threadListView.Columns.Add("State", 90, HorizontalAlignment.Left);
threadListView.Columns.Add("Start Time", 150, HorizontalAlignment.Left);
threadListView.Columns.Add("User Processor Time", 150, HorizontalAlignment.Left);
}

The FillThreadList Method

The main functionality is written in the FillThreadList method. The Process.Threads property of a process returns all threads corresponding to a process in the form of a ProcessThreadCollection. The ProcessThreadCollection is a collection of threads.

protected void FillThreadList ( int nProcID
            )
{
string [] str = new String[7];
m_Threads =
            GetThreads(nProcID);

ProcessThreadCollection
            threads;
Process proc;
proc =
            Process.GetProcessById(nProcID);

// Read the Process.Threads
            property.
threads = proc.Threads;
// Read desired
            ProcessThread property.

for ( int i=0; i<threads.Count;
            i++)
{
str[0] = threads[i].Id.ToString();
str[1] =
            threads[i].CurrentPriority.ToString();
str[2] =
            threads[i].BasePriority.ToString();
str[3] =
            threads[i].TotalProcessorTime.ToString();
str[4] =
            threads[i].ThreadState.ToString();
str[5] =
            threads[i].StartTime.ToString();
str[6] =
            threads[i].UserProcessorTime.ToString();

// Add Item to the
            list view
ListViewItem item = new ListViewItem(str,
            0);
threadListView.Items.Add(item);
}
textBox1.Text =
            m_Threads.Count.ToString();
}

Summary

In Microsoft's .NET Framework Library, the System.Diagnostics namespace contains classes to work with system processes and threads. This namespace also provides classes that allow us to debug and trace our code. In this article, we covered the Process, Threads, and ProcessThread classes to create a thread and process monitoring application.

C#Today - © 2002 WroxPress Limited


IEnumerable, IEnumerator, and Dictionaries

IEnumerable, IEnumerator, and Dictionaries
http://www.csharptoday.com/content.asp?id=1677
Article Rating: 2.5
Published on October 23, 2001
Abstract
One of the many new and useful features in C# is the ability to use foreach loops to iterate through items in an array or collection. This relies on the implementation of two interfaces from the System.Collections namespace: IEnumerable and IEnumerator. In this article, Karli Watson takes a look at exactly what goes on under the hood in foreach loops. By knowing how things work we can take advantage of the system to create novel classes, and will be better equipped to customize some of the more advanced collection classes that the .NET framework has to offer, allowing us to streamline our code by bypassing much of the overhead inherent in more verbose collection classes. To illustrate this, Karli derives a class from System.Collections.DictionaryBase and modifies the way this class works with foreach loops.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

One of the many new and useful features in C# is the ability to use foreach loops to iterate through items in an array or collection. This relies on the implementation of two interfaces from the System.Collections namespace: IEnumerable and IEnumerator.

In this article we'll take a look at exactly what goes on under the hood in foreach loops. By knowing how things work here we can take advantage of the system to create novel classes, and will be better equipped to customize some of the more advanced collection classes that the .NET framework has to offer. As an example of this we'll derive a class from System.Collections.DictionaryBase and modify the way this class works with foreach loops.

foreach Loops

To recap, C# foreach loops have the following syntax:

foreach (<itemType> <item> in
            <collection>)
{
<item>.DoStuff();
}

Here, <collection> is an object that contains <itemType> items. The loop will iterate through these items, placing each one in <item> in turn, and leaving us to manipulate them. The only restriction here is that we only get read access to <item>.

For example, we can iterate through an array of strings in the following way:

string[] stringArray = {"String 1", "String 2", "String 3"};
foreach (string testString in stringArray)
{
Console.WriteLine("String found: {0}", testString);
}

This code will simply output the three strings in stringArray to the screen.

How foreach Loops Work

What actually happens in a foreach loop is as follows:

  • <collection>.GetEnumerator() is called, which returns an IEnumerator reference. This method is available through implementation of the IEnumerable interface, although this is optional.
  • The MoveNext() method of the returned IEnumerator interface is called.
  • If MoveNext() returns true then the Current property of the IEnumerator interface is used to get a reference to an object, which is placed into <item>.
  • The above two steps repeat until MoveNext() returns false, at which point the loop terminates.

It is worth implementing the IEnumerable interface rather than just having a public GetEnumerator() method on your class. Although this won't affect foreach functionality it may break other code that relies on this interface. In addition, implementing interfaces can be done explicitly, which prevents the interface members being used through the default interface of your class (as we'll see shortly).

This behavior is fairly basic, but not at all obvious. Let's look at an example that illustrates the steps involved, with a class that mimics the 'three string' array we looked at in the last section. This class will be called ForeachClass, and will be accessed using the following simple code:

using System;

namespace
            ForeachEnumeration
{
class EntryPointClass
{
static void
            Main()
{
Console.WriteLine("Creating new ForeachClass
            instance.");
ForeachClass testObj = new
            ForeachClass();
Console.WriteLine("Iterating through instance
            using foreach:");
foreach (string testString in
            testObj)
{
Console.WriteLine("String found: {0}",
            testString);
}
Console.WriteLine("Iteration
            completed.");
}
}
}

This code creates an instance of the ForeachClass class and iterates through it with a foreach loop, writing out progress information as it goes along. The same three strings used in the previous example are hardcoded into ForeachClass, so there is no need to initialize them here.

The code for ForeachClass is as follows:

using System;
using
            System.Collections;

namespace
            ForeachEnumeration
{
public class ForeachClass : IEnumerator,
            IEnumerable
{
private int itemsRead;

public
            ForeachClass()
{
itemsRead = 0;
}

IEnumerator
            IEnumerable.GetEnumerator()
{
Console.WriteLine("IEnumerable.GetEnumerator()
            called.");
Console.WriteLine("IEnumerable.GetEnumerator()
            returning \"this\".");
return this as
            IEnumerator;
}

bool
            IEnumerator.MoveNext()
{
Console.WriteLine("IEnumerable.MoveNext()
            called, {0} item(s) read.",
itemsRead);
itemsRead++;
if
            (itemsRead <=
            3)
{
Console.WriteLine("IEnumerable.MoveNext() returning
            \"true\".");
return
            true;
}
else
{
Console.WriteLine("IEnumerable.MoveNext()
            returning \"false\".");
return false;
}
}

void
            IEnumerator.Reset()
{
Console.WriteLine("IEnumerable.Reset()
            called.");
}

object
            IEnumerator.Current
{
get
{
Console.WriteLine("IEnumerable.Current
            get accessor called.");
string returnString = "String " +
            itemsRead.ToString();
Console.WriteLine("IEnumerable.Current get
            accessor returning"
+ " \"{0}\".", returnString);
return
            returnString;
}
}
}
}

This class implements IEnumerable and IEnumerator explicitly, as can be seen by the member declarations of the members:

IEnumerator IEnumerable.GetEnumerator()
{
...
}

bool IEnumerator.MoveNext()
{
...
}

void IEnumerator.Reset()
{
...
}

object IEnumerator.Current
{
...
}

These explicit declarations have no accessibility modifier (interface members are implicitly public) and refer to the members using names qualified with that of the interface. This means that the following code is invalid:

ForeachClass testObj = new ForeachClass();
System.Collections.IEnumerator tesObjEnumerator = testObj.GetEnumerator();

Instead we'd need to use the following to call this method manually:

ForeachClass testObj = new ForeachClass();
System.Collections.IEnumerable testObjEnum = testObj;
System.Collections.IEnumerator testObjEnumerator = testObjEnum.GetEnumerator();

Explicit interface declarations have the advantage that their functionality is encapsulated more logically, as it is only available through interface references.

The GetEnumerator() method of the IEnumerable interface simply returns this, as the class implements IEnumerator.

The IEnumerator methods have implementations that 'fake' a collection of strings. A private member, itemsRead, keeps track of how many times MoveNext() is called, which returns false after three reads. This member is also used to build the string returned by the get accessor of the Current property.

The output of this code is as follows:

This is exactly as expected, following the sequence of events for the foreach loop described earlier.

Dictionaries

Dictionary collections are collections that allow access to the items they contain via key values rather than indices. This basically involves the following:

  • Storing items along with associated key values.
  • Using key values as parameters for Add() and Remove() methods for items as required
  • (Optionally) using an indexer with a non-numeric type.

As an example, consider a dictionary class called People that stores Person objects. Rather than using code such as:

People myPeople = new People();
// myPeople filled with people, code for this not shown.
Person myPerson = myPeople[2];

We can instead use:

People myPeople = new People();
// myPeople filled with people, code for this not shown.
Person myPerson = myPeople["Bartholomew"];

Here a string value is being used as a key to access the Person object required. The string used here is simply the name of the person, which is fine as an example, although you should bear in mind that names aren't unique, and keys should be!

The .NET framework provides us with base classes to make the implementation of a strongly typed dictionary class simple. For example, we could implement a Person class as follows:

using System;

namespace
            PeopleLib
{
public class Person
{
private string
            name;
private int age;

public string
            Name
{
get
{
return name;
}
set
{
name =
            value;
}
}

public int Age
{
get
{
return
            age;
}
set
{
age = value;
}
}

public
            Person() : this("Default Name", 0)
{
}

public
            Person(string newName, int newAge)
{
name = newName;
age =
            newAge;
}
}
}

And contain instances of this class in a People class defines like this:

using System;
using
            System.Collections;

namespace PeopleLib
{
public class
            People : DictionaryBase
{
public void Add(Person
            newPerson)
{
//The name entered should be
            unique
Dictionary.Add(newPerson.Name,
            newPerson);
}

public void Remove(string
            name)
{
Dictionary.Remove(name);
}

public
            People()
{
}

public Person this[string
            name]
{
get
{
return Dictionary[name] as
            Person;
}
set
{
Dictionary[name] =
            value;
}
}
}
}

Here the Add() method takes a Person object and, using the Name property of this object as a key, stores the object in a dictionary. We also use a Remove() method to remove named People objects, and a string-based indexer to access Person objects via their name.

All of this code makes use of the protected Dictionary property of the base DictionaryBase class, which is an IDictionary interface reference. This interface gives us access to a weakly typed dictionary collection object, and we can perform the required casts in our derived class to make the exposed collection strongly typed.

The following client code makes use of this collection class:

using System;
using PeopleLib;
using
            System.Collections;

namespace PeopleLibClient
{
class
            Client
{
static void Main()
{
People myPeople = new
            People();
myPeople.Add(new Person("Karli",
            25));
Console.WriteLine("Karli's age: {0}",
            myPeople["Karli"].Age);
}
}
}

However, the following code will fail:

using System;
using PeopleLib;
using
            System.Collections;

namespace PeopleLibClient
{
class
            Client
{
static void Main()
{
People myPeople = new
            People();
myPeople.Add(new Person("Karli",
            25));
myPeople.Add(new Person("Donna", 32));
myPeople.Add(new
            Person("Rob", 26));
myPeople.Add(new Person("Mike",
            26));
myPeople.Add(new Person("Helmut", 38));
foreach (Person
            person in myPeople)
{
Console.WriteLine("Name: {0} Age: {1}",
            person.Name, person.Age);
}
}
}
}

The reason for this is that the foreach loop is using an enumerator defined on the base DictionaryBase class that returns DictionaryEntry objects rather than Person objects. In order to account for this we need to modify our foreach loop as follows:

foreach (DictionaryEntry item in myPeople)
{
Person person = item.Value as Person;
Console.WriteLine("Name: {0} Age: {1}", person.Name, person.Age);
}

This is fine if we want to be able to access individual key values, but not much use here, resulting in code that isn't that intuitive.

However, since we know exactly what happens when foreach loops execute, we can modify this code without too much of a problem.

Custom Enumeration in Dictionaries

The DictionaryBase class provides an implementation for GetEnumerator() that we don't want to use. Instead, we want to provide our own enumeration code. So, we need to provide a new implementation of this method that hides the original one, which is possible using the new keyword:

      new
            public IEnumerator GetEnumerator()
{
//
            Implementation
}

The question now is what we do when this function is called. We can achieve the functionality we desire (that foreach loops get Person objects, not DictionaryEntry objects) in the following way:

  • When our GetEnumerator() implementation is called we call the base class GetEnumerator() and store a reference to the IDictionaryEnumerator interface returned (this inherits from IEnumerator, which is why it works normally with foreach loops).
  • Instead of returning a reference to this enumerator, our new GetEnumerator() method return an IEnumerator reference designed by us (for convenience we can implement this directly on the People class).
  • The members of this new enumerator make use of the base class IDictionaryEnumerator members, but we customize them to do what we want them to do, rather than what happens by default with the base class implementation.

The code for all this is as follows:

using System;
using
            System.Collections;

namespace PeopleLib
{
public class
            People : DictionaryBase, IEnumerator
{
private
            IDictionaryEnumerator baseEnumerator;

// unchanged members
            not shown for brevity

new public IEnumerator
            GetEnumerator()
{
baseEnumerator =
            base.GetEnumerator();
return this;
}

object
            IEnumerator.Current
{
get
{
DictionaryEntry currentEntry
            =
(DictionaryEntry)baseEnumerator.Current;
return
            currentEntry.Value;
}
}

bool
            IEnumerator.MoveNext()
{
return
            baseEnumerator.MoveNext();
}

void
            IEnumerator.Reset()
{
baseEnumerator.Reset();
}
}
}

The new implementations of MoveNext() and Reset() simply pass calls to the base class methods. The Current property is more interesting. In the get accessor we intercept the DictionaryEntry object obtained from the base class (using explicit casting, as this object is a struct, so the as operator doesn't work here), and return the Value property of this object. This property is, as we saw earlier, one of the Person objects in the collection.

With this code we can perform iteration through Person objects in a People object using the code we saw earlier, with no problems:

         foreach
            (Person person in myPeople)
{
Console.WriteLine("Name: {0}
            Age: {1}", person.Name, person.Age);
}

Giving us a result, when we run the code, of:

Summary

In this article we've taken an in-depth look at how enumeration works in C#, and specifically what goes on behind the scenes when we use foreach loops. The experience gained through this allowed us to build a class that used the enumeration interfaces IEnumerable and IEnumerator to output data in a non-standard way. Knowledge of this subject can be very useful when streamlining code for optimum efficiency, as we can bypass much of the overhead inherent in more verbose collection classes.

We also looked at a way to obtain items directly from a dictionary collection in foreach loops, rather than having to get them through other class instances. This kind of customization is typical in OOP programming, where we have the ability to derive new classes from existing ones. It is often the case that we want to modify the behavior base classes, whether by expanding or restricting existing functionality, and this is one way to do it if enumeration is an issue.

Hopefully this article will inspire you to delve deeper into the System.Collections namespace, and create more powerful collection classes for storing your data.

C#Today - © 2002 WroxPress Limited


Globalization in C#

Globalization in C#
http://www.csharptoday.com/content.asp?id=1728
Article Rating: 4.5
Published on October 16, 2001
Abstract
As application designers, we should strive to make sure our applications are as usable as possible, irrespective of where in the world the user resides. Win32 developers have long had access to functions that allow them to properly represent dates, currency values, and so on. It’s important that whenever we have to show the user dates, times, monetary values, or numbers, we use the globalization features in .NET. These features use the globalization settings in Windows to properly format these values into strings for display. In this article, Matthew Reynolds looks at the basics of properly formatting dates, times and numbers for display. In the second part of the discussion, he looks at how we can build a Windows Form that can alter the text on the controls depending on the user’s settings.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

Since the very early versions of Windows, Microsoft has been sensitive to the fact that applications are used all over the world in a variety of different languages and cultures. As application designers, we should strive to make sure our applications are as usable as possible, irrespective of where in the world the user resides.

Win32 developers have long had access to functions that allow them to properly represent dates, currency values, and so on. For example, someone in the UK will regard the date 7/6/2001 as 7th June 2001, whereas someone in the US will regard that same date as July 6 2001. It's important that whenever we have to show the user dates, times, monetary values, or numbers, we use the globalization features in .NET. These features use the globalization settings in Windows to properly format these values into strings for display.

In this article, we're going to kick off by looking at the basics of properly formatting dates, times and numbers for display. In the second part of the discussion, we're going to look at how we can build a Windows Form that can alter the text on the controls depending on the user's settings.

Dates, Times, etc.

The base currency of globalization is the "culture", a culture being a set of rules that defines how information should be represented to the user. In .NET, cultures are accessed through System.Globlalization.CultureInfo. Each culture understood by .NET is represented by a string, and this string is typically made up of two parts. The first part is the "base language", such as English, German, etc. and the second part is a dialect. For example, "en-GBUK" is UK English, "de-CH" is Swiss German, and so on.

When your .NET application starts, the appropriate culture to use is automatically determined by the Framework. Let's build an application now to illustrate how this can work.

Open Visual Studio.NET and create a new Visual C# - Windows Application project. Call it FormattingDemo.

When the Designer for Form1 appears, add a new ListBox control and a ListView control, like this:

  • On theListBox control, set theName property to listCultures, theIntegralHeight property to False, and theAnchor property to Top, Bottom, Left.
  • On theListView control, set theName property to listviewStrings, theView property toDetails, and theAnchor property to Top, Bottom,Left, Right.

On theListView control we also need some columns. Find and select the Columns property, and click the small ellipsis ("...") button to the right of the property. This will open up the ColumnHeader Collection Editor. Add two new columns and set theWidth of each to 150. On the first column, set theText property to "Name " and on the second, set the Text property to "Value".

The first thing we'll demonstrate is how we can find the current culture, and display the culture name in the list box.

To work with cultures, we need two namespaces:System.Threading and System.Globalization. Add references to both of these to the top of the code listing:

using System;
using System.Drawing;
using System.Collections;
using System.ComponentModel;
using System.Windows.Forms;
using System.Data;
using System.Threading;
using System.Globalization;

Each thread keeps track of its own culture, which means that if we want to find the culture we have to ask the thread. (If you don't know what a thread is, don't worry about it! There are loads of articles on http://www.csharptoday.com/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq about threading). We can do this by asking the Thread class for the static CurrentThread property to get a handle to the current thread, and then by asking the Thread object we get back its CurrentCulture property.

Find the constructor for Form1, and add a call to add the value of Thread.CurrentThread.CurrentCulture to the list box.

      public
            Form1()
{
//
// Required for Windows Form Designer
            support
//
InitializeComponent();

// add the
            culture...
listCultures.Items.Add(Thread.CurrentThread.CurrentCulture);
}

Run the project and you'll see your culture appear in the list:

Formatting Values

So, you can now see how we can find the current culture of the application. You don't actually need to do this in order to format strings.

If the project is running, close it. Open the Designer for Form1 and double-click on the list box to create a newSelectedIndexChanged event handler. Add this call to UpdateView, which we will build in a moment.

private void listCultures_SelectedIndexChanged(object sender,
System.EventArgs e)
{
UpdateView();
}

Then, add these two methods:

public void UpdateView()
{
// reset the list...
listviewStrings.Items.Clear();

// get the date...
DateTime theDate = DateTime.Now;

// add some strings...
AddString("Long date", theDate.ToLongDateString());
AddString("Short date", theDate.ToShortDateString());
AddString("Long time", theDate.ToLongTimeString());
AddString("Short time", theDate.ToShortTimeString());

// numbers...
float theValue = 123456.789F;
AddString("Number", theValue.ToString("n"));
AddString("Currency", theValue.ToString("c"));
AddString("Negative number",
(0 - theValue).ToString("n"));
AddString("Negative currency",
(0 - theValue).ToString("c"));
}

private void AddString(string name, string value)
{
// create a new item...
ListViewItem item = new ListViewItem(name);
item.SubItems.Add(value);

// add it...
listviewStrings.Items.Add(item);
}

We'll look at how this works in a moment. First, run the project and select the culture from the list.

By and large, to format strings you can rely on variations of ToString available on the types themselves. For example, we can convert a DateTime value to long string with this call:

AddString("Short date", theDate.ToShortDateString());

Nothing complicated there!

Working with numbers is a little trickier. .NET has comprehensive number formatting functionality built in, and usually this is accessed through the ToString method on the value, although the static String.Format method can also be used to more or less to achieve the same effect.

In this example, we've used two format strings that tap into the culture settings to properly format the string. "n" tells ToString that it's supposed to be working with the built-in number format. "c" tell it to use the built-in currency format:

AddString("Number", theValue.ToString("n"));
AddString("Currency", theValue.ToString("c"));

Changing Cultures

In day-to-day work, it's unlikely you'll ever force a culture change of your application. It's typically best to rely on .NET giving you a culture to use, or ignore the whole issue altogether and stick to the ToString variations I've described. However, to demonstrate the power of locales, let's take a look at how we can switch between locales.

If the project is running, close it. Find the constructor for Form1 and add a few more cultures to the list:

   public
            Form1()
{
//
// Required for Windows Form Designer
            support
//
InitializeComponent();

// add the
            culture...
listCultures.Items.Add(Thread.CurrentThread.CurrentCulture);

//
            add new cultutres...
listCultures.Items.Add(new
            CultureInfo("en-US"));
listCultures.Items.Add(new
            CultureInfo("de-CH"));
listCultures.Items.Add(new
            CultureInfo("es-ES"));
listCultures.Items.Add(new
            CultureInfo("nb-NO"));
listCultures.Items.Add(new
            CultureInfo("hi-IN"));
listCultures.Items.Add(new
            CultureInfo("ja-JP"));
listCultures.Items.Add(new
            CultureInfo("sk-SK"));
listCultures.Items.Add(new
            CultureInfo("sv-FI"));
}

Don't worry if you don't recognize these cultures - we'll be displaying their names. However, if you want to find a complete list, look upSystem.Globalization.CultureInfo in the MSDN documentation.

Find thelistCultures_SelectedIndexChanged method and add a call to set theThread.CurrentThread.CurrentCulture property to whatever one was selected from the list.

private void listCultures_SelectedIndexChanged(object sender,
System.EventArgs e)
{
// set the culture...
Thread.CurrentThread.CurrentCulture =
(CultureInfo)listCultures.Items[listCultures.SelectedIndex];

// update the view...
UpdateView();
}

Next, find the UpdateView methods. Add a call to get back the current culture, and add a few variations of the name to the list. (I've omitted some code here for brevity).

public void UpdateView()
{
// reset the list...
listviewStrings.Items.Clear();

// get the name...
CultureInfo culture = Thread.CurrentThread.CurrentCulture;
AddString("Name", culture.Name);
AddString("Native name", culture.NativeName);
AddString("English name", culture.EnglishName);

Run the project and you should be able to select different cultures.

Notice how it's not necessary to alter the code that actually formats the string. Because we've changed the culture of the thread, all of the ToString variations automatically adapt their behavior so that the correct information is displayed.

Finding Local Names

Before we leave our discussion of formatting, it's worth looking at how we can get hold of the day and month names from the culture. This is a fairly common requirement, and getting the Framework to return translations is far easier than doing it yourself!

CultureInfo supports a property called DateTimeFormat. This returns a System.Globalization.DateTimeFormatInfo object that supports MonthNames and DayNames properties. These two properties both return an array of strings containing the names.

To try it, find the UpdateView method and add this code to the bottom. (I've omitted some code for brevity).

         //
            numbers...
float theValue = 123456.789F;
AddString("Number",
            theValue.ToString("n"));
AddString("Currency",
            theValue.ToString("c"));
AddString("Negative number", (0 -
            theValue).ToString("n"));
AddString("Negative currency", (0 -
            theValue).ToString("c"));

// months...
int index =
            1;
foreach(string month in
            culture.DateTimeFormat.MonthNames)
{
AddString("Month " +
            index.ToString(), month);
index++;
if(index ==
            13)
break;
}

// days...
index = 1;
foreach(string
            day in culture.DateTimeFormat.DayNames)
{
AddString("Day " +
            index.ToString(), day);
index++;
if(index ==
            8)
break;
}
}

Run the project and you'll see the localized names:

Language Strings

The trickier globalization problem to solve is related to the layout of forms. If you want to create a multilingual application, it's necessary to replace all of the text literals on the forms used on buttons, lists, etc. with a new strings targeted to whatever language you're dealing with. For example, if you have a button marked "Create Customer" on your form, someone who's more comfortable with French than English might prefer to see "Créez Le Client".

Prior to .NET, this problem was typically solved using a separate resource file containing all of the strings that the application needs. When the form was loaded, the strings were loaded from the file and the various properties updated. One issue with this was that the controls themselves could not change size or location. When translating from English to other languages, it's often the case that the translated text is bigger than the original English, which means that text would either appear clipped on the translated versions, or the buttons on the English version had to be oversized in order to accommodate the strings.

.NET takes a different approach. The code that powers the form, i.e. the event handlers and your own code, is separated from the code that described the way the form is laid out. This separation means that you can have multiple separate layout portions for each form, once for each language that you require. These separate portions are stored in .resx files.

You've already been using .resx files all the time that you've been building Windows Forms. Usually, Visual Studio keeps them hidden. On the Solution Explorer window's toolbar you'll see a button marked Show All Files. If you click this, you'll be able to see the .resx file for Form1.

Using Solution Explorer, create a new Windows Form called LocalizedForm. You'll notice that by default this form has one .resx file.

This file contains the base implementation of the form. This will be used if no language specific version of the form exists when the new form object is created, in order that the form can be displayed.

In the Properties window, change the Localizable property to True.

On the Designer, add a button. Set the Text property to "Hello, world!"

Now, change the Language property of the form to German. The form will appear the same, but you'll notice that a new .resx file has appeared.

Change the Text property of the button to "Hallo welt!"

Open the Designer for Form1 and add a new button. Set its Name property to buttonShow, its Text property to "Show me!" and the Anchor property to Bottom, Left. Double-click on the button to create a newClick event handler. Add this code:

      private
            void buttonShow_Click(object sender, System.EventArgs
            e)
{
LocalizedForm form = new
            LocalizedForm();
form.Show();
}

When the form object is created, the current culture is examined. In fact, there's a wrinkle here in that each thread keeps track of a "culture" and a "UI culture". So far, we've been changing the Thread.CurrentThread.CurrentCulture property, which relates to formatting. If we want the automatic form localization functionality to work, we need to change Thread.CurrentThread.CurrentUICulture.

Find the listCultures_SelectedIndexChanged method and add a call to change CurrentUICulture.

private void listCultures_SelectedIndexChanged(object sender,
System.EventArgs e)
{
// set the culture...
Thread.CurrentThread.CurrentCulture =
(CultureInfo)listCultures.Items[listCultures.SelectedIndex];
Thread.CurrentThread.CurrentUICulture =
Thread.CurrentThread.CurrentCulture;

// update the view...
UpdateView();
}

Run the project. Click the button and you should see the English version of the form.

Now, from the list select de-CH. Click the button and you'll see the German version:

Summary

In this article we learned the principles behind localization, and saw how the Framework is able to present dates and numbers in a way that matches the users preference without our intervention. We also saw how CultureInfo can be used to tell the application the localization settings to use. Finally, we looked at how to build forms that adjust their text and layout according to the users locale.

If you'd like to discuss a broader range of C#Today topics and issues, or if you have any C# related questions you need answers to, please visit our online resource centres at : http://p2p.wrox.com/list.asp?list=csharptoday_discuss&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq and http://p2p.wrox.com/list.asp?list=c_sharp&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq
C#Today - © 2002 WroxPress Limited


Data Binding in C#

Data Binding in C#
http://www.csharptoday.com/content.asp?id=1727
Article Rating: 3.5
Published on October 15, 2001
Abstract
ADO.NET, the database connectivity library included with the .NET Framework, provides powerful ways of working with data from all kinds of data sources. Underpinning most of the ADO.NET functionality is XML.In this exercise, Matthew Reynolds takes a look at working with XML and Data Binding. Specifically, he shows how we can create an application that lets us store a small database of Web sites in an XML file on disk.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

One of the most common requirements of all .NET applications is the ability to work with, and manipulate, data. In the past, working with data has tended to be something that requires a considerable amount of housekeeping, mostly in the area of retrieving data from whatever data source you're using, setting properties on controls, and changing the data in the original source when the user makes changes to the controls.

Working With XML Data

ADO.NET, the database connectivity library included with the .NET Framework, provides powerful ways of working with data from all kinds of data sources. Underpinning most of the ADO.NET functionality is XML, the de facto method for data exchange on the Internet. Data drawn from SQL Server as a DataSet, for example, can be saved as an XML file, or sent as XML data from a Web Service to a client. Alternatively, data sent by a business partner in XML can be transformed using XSLT (XML Style sheet Language Transformations) into a different format, imported into a DataSet, and then injected into an Oracle database. In short, any data that you can retrieve using ADO.NET can be turned into XML, and XML can be changed into a form where it can be put back into a data source.

In this exercise, we're going to take a look at working with XML and Data Binding. I'm going to assume that you're familiar with XML, but if you're not, there are plenty of articles on http://www.asptoday.com/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq and http://www.csharptoday.com/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq. Specifically, we're going to create an application that lets us store a small database of Web sites in an XML file on disk. Each Web site that we represent will be able to hold three pieces of information: a name, a URL, and a description. Here's an XML file that contains four Web sites:

<?xml version="1.0" encoding="utf-8"?>
<WebSites xmlns="http://tempuri.org/WebSites.xsd">
<WebSite>
<Name>.NET 247</Name>
<URL>http://www.dotnet247.com/</URL>
<Description>.NET Programmers Reference Guide</Description>
</WebSite>
<WebSite>
<Name>C#Today</Name>
<URL>http://www.csharptoday.com/</URL>
<Description>Great C# content!</Description>
</WebSite>
<WebSite>
<Name>Angry Coder</Name>
<URL>http://www.angrycoder.com/</URL>
<Description>Vent your frustrations at Angry Coder</Description>
</WebSite>
<WebSite>
<Name>Developer.*</Name>
<URL>http://www.developerdotstar.com/</URL>
<Description>Articles on the art of programming</Description>
</WebSite>
</WebSites>

There are a number of ways of loading this data into a .NET application, but the technique we're going to see here involves using Visual Studio.NET to create a "typed DataSet ".

A typed DataSet is typically implemented in a set of new classes derived from DataSet, DataTable and DataRow. They allow direct access to the underlying data through properties like Name, Email, CustomerId, etc. rather than requiring that the developer access the data through the DataRows indexer by providing a column name or ordinal number.

Rather than labor the discussion on "what exactly a typed DataSet is", we'll ask Visual Studio to create one so we can take a closer look at some code. Create a new Visual C# - Windows Application project and call it MyWebSites.

Using Solution Explorer, right-click on the MyWebSites project and select Add - Add New Item. Expand the Local Project Items object and select Data. Select XML File, give it the name of WebSites, and click OK.

When the new file appears, put some data in the file that exactly matches the format described above. If you try to copy and paste directly from this page, Visual Studio will try and be clever, and mangle the data. Instead, paste the data into Notepad, and then copy and paste from Notepad into Visual Studio.

As I mentioned before, .NET and ADO.NET are very keen on XML as a way of representing data. If you want, you can see the XML data represented in a grid format, much like a DataGrid, by clicking the small XML Data button at the bottom of the editor window.

To work with XML data, ADO.NET needs a schema. This defines the structure of the data contained within the XML data that you're trying to load. Visual Studio can create a schema for you, and this is what we'll do now. Flip back to the XML view and from the menu, select XML - Create Schema. If you look over at Solution Explorer, you'll now see the .xsd file containing the schema:

Double-click on the schema file to open it and you'll see a diagrammatic representation of the schema.

To create our typed DataSet, select Schema - Generate DataSet from the menu. Visual Studio will do this, but you may not be able to instantly see the new file. On Solution Explorer, click the Show All Files button on the toolbar and you'll find the WebSites.cs file within WebSites.xsd.

If you open that file, the first thing you'll see is that our new WebSites class is derived from System.Data.DataSet :

[Serializable()]
[System.ComponentModel.DesignerCategoryAttribute("code")]
public class WebSites : System.Data.DataSet {

If you scroll down a little inside the class, you'll be able to find another class called WebSiteRow, this class being derived from System.Data.DataRow :

public class WebSiteRow : DataRow {

It's in here that the magic really starts. Look for a property called Name (using the drop-down boxes at the top of the screen is an easy way to do this!) It will look like this:

            public
            string Name {
get {
try {
return
((string)(this[this.tableWebSite.NameColumn]));
}
catch
            (InvalidCastException e) {
throw new
            StrongTypingException(
"Cannot get value because it is DBNull.",
            e);
}
}
set {
this[this.tableWebSite.NameColumn] =
            value;
}
}

What this property does is abstract the access to the Name column, or rather the Name node, inside our XML file, so that we can use a property like this:

Console.WriteLine(myWebSiteRow.Name);

.rather than having to do this:

Console.WriteLine(myWebSiteRow["Name"]);

Apart from being faster (you'll notice that the property uses an integer ordinal to get the column value, which is faster than looking up by name), it's also convenient and far more natural to use.

Binding Data

Now that we have the data, how do we use it?

The first thing we need to do is load the data. We'll do this on the Load event of the form, and we'll keep a copy of the DataSet around as a member. Open the code editor for Form1 and add the _webSitesDataSet member, and a member called _filename.

public class Form1 : System.Windows.Forms.Form
{
// members...
private WebSites _webSiteDataSet;
private string _filename;

We're going to assume that the XML file is stored in the same folder as the application assembly itself. At the moment, the WebSites.xml file you created is not in this folder, as Visual Studio will create it in the project source code directory rather than the bin\Debug folder. Copy both this file, and the WebSites.xsd file into the bin\Debug folder.

To test the load process, we'll load the data directly into a DataGrid. Once we're convinced that works, we'll look at how to bind to controls. Using the Designer, add a new DataGrid control. Don't worry about changing the name or any of the other properties, as we'll be deleting this in a little while.

Open the code editor for Form1 again, and add this code to Form1's constructor:

      public
            Form1()
{
//
// Required for Windows Form Designer
            support
//
InitializeComponent();

// load the data
            set...
_filename = Environment.CurrentDirectory +
            "\\WebSites.xml";
_webSiteDataSet = new
            WebSites();
_webSiteDataSet.ReadXml(_filename);

// test
            the binding...
dataGrid1.DataSource =
            _webSiteDataSet;
}

Run the project and you'll see this:

Now that we're confident that the data can be loaded OK, let's look at how we can bind data. Close the project and open the Designer. Delete the DataGrid control and add three Label and three TextBox controls.

  • Top text box - change the Name property to textName. Change the Anchor property to Top, Left, Right. Clear the Text property.
  • Middle text box - change the Name property to textUrl. Change the Anchor property to Top, Left, Right. Clear the Text property.
  • Bottom text box - change the Name property to textDescription. Change the Anchor property to Top, Bottom, Left, Right. Also, change the MultiLine property to True and the ScrollBars property to Vertical. Clear the Text property.

Data Binding needs three things in order to work:

  • A property on a control. This property will be set to the value of the data when the data binding begins. In addition, when the property is changed, the underlying data value will be changed back again.
  • A data source. This can be any list in .NET, which may be a DataSet, a DataTable, an array, a collection, and so on. We're going to use a DataTable.
  • A "navigation" expression. This is one of the hardest things to get your head around with data binding. Basically, you provide the control with a route to the data you're looking for. We'll see this in a moment.

We briefly mentioned before that we're going to use a DataTable to do the binding. At the moment we're storing a DataSet in the private _webSiteDataSet, so we need to add a property to the form that returns the first table in the set. Add this code to Form1 :

// WebSites property...
public DataTable WebSites
{
get
{
if(_webSiteDataSet != null)
return _webSiteDataSet.Tables[0];
else
return null;
}
}

Next, we need to bind the Text property of each of the three TextBoxes to the Name, Url and Description properties of the current row in the DataTable returned by the WebSites property. Don't worry about "the current row" for now. This will initially be the first row in the DataTable, but we'll see how to change this later.

Find the constructor for Form1 and replace the line that set the DataBinding property on the now-deleted DataGrid with this code instead:

      public
            Form1()
{
//
// Required for Windows Form Designer
            support
//
InitializeComponent();

// load the data
            set...
_filename = Environment.CurrentDirectory +
            "\\WebSites.xml";
_webSiteDataSet = new
            WebSites();
_webSiteDataSet.ReadXml(_filename);

// bind
            the text boxes...
textName.DataBindings.Add("Text", WebSites,
            "Name");
textUrl.DataBindings.Add("Text", WebSites,
            "Url");
textDescription.DataBindings.Add("Text", WebSites,
            "Description");
}

What we're doing in each case is adding a new data-binding directive to the DataBindings collection on each control. The first parameter specifies the property to bind to (specifically textName.Text, textUrl.Text and TextDescription.Text), the data source (in our case a DataTable) and the navigation string.

The navigation strings are names of properties based on whatever's being bound to. The data binding code is smart enough to know that if we're binding to a DataTable, we actually only want to bind to the current row. At the moment, this is the first row, but because we're using a strongly-typed DataSet, we don't have a System.Data.DataTable object, but in fact have a WebSites.WebSiteDataTable object that is derived from System.Data.DataTable. Moreover, it knows that each row is actually a WebSite.WebSiteRow object not a System.Data.DataRow.

When it comes to evaluate the meaning of Name, Url or Description, what it's actually doing is using Reflection to find a property with the same name. When it finds one, the value of this property is retrieved and passed to whatever property we've bound to the control itself.

We're not limited to one level of properties. If we were binding to an object that had a property called Widget and that object had a property called Manufacturer which returned a WidgetManufacturer object, and that object had a property called Name and returned a string, we could provide a navigation string of " Widget.Manufacturer.Name " and the binding would still work as intended.

However, let's try running the project now. You'll see this:

Paging Records

We've alluded to the fact that somehow the data binding code is capable of keeping track of the current record. This is done through a property on the form itself called BindingContext. This property provides access to a central repository of " System.Windows.Forms.BindingManagerBase " objects.

The first time we bound to the DataTable, the form's BindingContext was informed by the control of the fact that something was binding to the DataTable. This action created what is in effect a record pointer in the CurrencyManager, this record counter being set initially to 0, or the first record. If we change this record pointer, automatically all of the controls that bind to the DataTable will be informed of the fact the pointer has changed, and will update their own properties accordingly.

To prove this, add a button to the form. Change its Name property to buttonNext and its Text property to TextNext. Also, change its Anchor property to Bottom, Right.

Now, double-click on the button and add this code:

private void buttonNext_Click(object sender, System.EventArgs e)
{
WebSiteBindingManager.Position += 1;
}

We'll need to implement WebSiteBindingManager, but all this property does is ask BindingContext for whatever BindingManagerBase -derived object is coordinating the controls' binding to the data.

// WebSiteCurrencyManager property
public BindingManagerBase WebSiteBindingManager
{
get
{
return this.BindingContext[WebSites];
}
}

Now try running the project. Clicking the Next button will display the next Web site in the table.

In Windows applications that deal with data represented in this kind of form, it's usual to have what looks like a set of VCR buttons that allow navigation. For some reason, Microsoft chose not to include a common control for this, so rather than building one, we'll use a TrackBar control. (It's more "proper" to use a set of buttons to do this. However, using a TrackBar is a quick-and-dirty solution as we only have to build one control and one event handler to provide a mechanism that lets us move through all of the available records).

Open the Designer for Form1. Delete the button if you like (we won't be using it any more), and add a new TrackBar control. Set its Anchor property to Bottom, Left, Right and the Name property to trackWebSite.

The first thing with the TrackBar is to set its Maximum property when the form loads. Find the constructor for Form1 and add this call to set the Maximum property:

      public
            Form1()
{
//
// Required for Windows Form Designer
            support
//
InitializeComponent();

// load the data
            set...
_filename = Environment.CurrentDirectory +
            "\\WebSites.xml";
_webSiteDataSet = new
            WebSites();
_webSiteDataSet.ReadXml(_filename);

// bind
            the text boxes...
textName.DataBindings.Add("Text", WebSites,
            "Name");
textUrl.DataBindings.Add("Text", WebSites,
            "Url");
textDescription.DataBindings.Add("Text", WebSites,
            "Description");

// set the track bar
trackWebSite.Maximum
            = WebSites.Rows.Count;
}

Flip over to the Designer and double-click on the TrackBar to create a new Scroll event handler. Add this code:

private void trackWebSite_Scroll(object sender, System.EventArgs e)
{
// set the position...
WebSiteBindingManager.Position = trackWebSite.Value - 1;
}

If you run the project, you'll find that you can scroll through all of the available Web sites.

Saving Changes

Data Binding is a two way process, so if you change the values in the text box, the underlying data in the DataSet will also be changed. The upshot of this is that if we save the DataSet back to the XML file using the WriteXml method, the changes will be committed to the file. All that we have to do in order to save changes to disk is hook into the form's Closed event and call WriteXml.

Add this method to Form1 :

      //
            OnClosed...
protected override void OnClosed(EventArgs
            e)
{
// save?
if(_webSiteDataSet !=
            null)
_webSiteDataSet.WriteXml(_filename);
}

Now whenever you close the application, changes to the data will be saved to the file. You should note, though that you need to tab off of the field you're editing in order to save the changes to the DataSet before closing the application. So, if you change a field and immediately click the close button, the changes will not be saved. However, change a field, click on another field, and then click C lose, and everything will work OK.

Summary

In this article we took a close look at data binding in Windows Forms, and also saw how we could work with data stored in XML files through the DataSet functionality in ADO.NET.

If you'd like to discuss a broader range of C#Today topics and issues, or if you have any C# related questions you need answers to, please visit our online resource centres at : http://p2p.wrox.com/list.asp?list=csharptoday_discuss&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq and http://p2p.wrox.com/list.asp?list=c_sharp&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq
C#Today - © 2002 WroxPress Limited


Building Windows Forms on-the-fly with XML and C#

Building Windows Forms on-the-fly with XML and C#
http://www.csharptoday.com/content.asp?id=1580
Article Rating: 4
Published on September 27, 2001
Abstract
In this article, Ashiwn Kamanna takes the concept of an XML driven Form builder a step further than the basics as he discusses how to eliminate the requirement for Form development, not only in an ASP based web application, but also in any potential client of an application. He discusses how to build a Form dynamically in a C# based windows application, and also discusses some object oriented patterns as he walks us through an example.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

If we cannot completely stop the building of the entire Graphic User Interface, we can certainly reduce a lot of the burden on the Form developers. In his ASPtoday article article, Owen Cutajar demonstrated an XML Driven Form Builder. This article takes that concept a step further to discuss how to eliminate the requirement for Form development not only in an ASP based web application, but also in any potential client of an application. In a distributed application, various clients are connected to an application server or a server that exposes web services. These clients can be web browsers, windows applications or handheld devices. Separate programs are written to display the Forms in the different clients. Instead, these programs can be replaced by generic programs to build the Forms on-the-fly from an XML file downloaded from a web server. This XML file will define the platform-neutral Form interface.

With a wider acceptance and implementation of SOAP and Web services, this will be a common scenario in many applications. Also, as technologies with an Open Access model like the Microsoft 'HailStorm' Services emerge, where the user has a common information model, the elements like <HailStorm:myAddress> or <HailStorm:myProfile> will evolve as a natural step. The client applications can build the interfaces as per the standards of the specific application. Once a framework is in place, the Form designers do not have to do any recoding - they can simply update the XML file manually, or use a tool built to update the XML. This change will take effect on all the clients.

Currently XForm is one such standard from the W3C for web based XHTML Forms. You can see some examples of this at http://www.mozquito.org/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq.

In this article we will discuss how to build a Form dynamically in a C# based windows application, and discuss some object oriented patterns as we walk through the example.

The Steps involved in the process

Before we jump into our C# specific example, let us see the generic steps that any application trying to build Forms from an XML file should follow.

  • Get the XML file from the database and publish it for the clients.
  • The client programs download the XML file from the web server.
  • The program parses the XML file and builds the corresponding Form. A web client can use a generic XSLT; the windows based clients can use the respective DOM or SAX implementations for the language.
  • On Form submission, the programs insert the Form data into the database.

Requirements

  • .NET SDK Beta 2 This example will not work with the Beta 1 SDK.
  • Microsoft SQL Server 7.0 or above. There is a single table used in the application. You can create it in any other database Server and change the connection string and the SQL queries in the code accordingly.

The Database tables

There are two tables in our example. The first one, tblForm, maintains the Form definition. This data can be fed in through another utility program, like a generic Form designer. An XML generator program that reads the data from this table will generate our XML Form definition document.

TblForm

The Form data will be stored into this table after the user submits the dynamically generated Form.

tblFormData

In the code download you will find an SQL script file script.sql, which contains the script to create these tables. You can run this script in an existing test database or create a new database on the SQL Server. You will have to provide the appropriate connection string to connect to this database in the MyFormData.cs file that we shall discuss shortly.

The User Interface XML Document

In this article I assume that this XML file is available to us so that we can concentrate on the Form building, rather than the generation of XML from the database. The following figures show the XML document structure.

If you are using the Visual Studio .NET IDE, you will no longer have to run IE 5.0 or an XML Editor to confirm that your XML is well formed. The Data View for XML displays error messages similar to the ones that appear in the IE 5.0 Browser. The Data View provides an editable interface to work with the document. This simplifies your task of manually perusing through the document to check the specific elements.

The Data View of the interface.xml file editor in Visual Studio .NET's built-in editor.

The following figure shows the XML view. This view provides a code editor interface for XML files. An XForm text field looks like this <xform:textbox />. In our example, the field type is kept in the <type> element. The <target> element that is used for a potential web client was used in Owen Cutajar's article.

The XML View of the interface.xml file editor in Visual Studio .NET's Built-in Editor.

As discussed above, we will publish this XML file on the web server. So, you will have to copy this file from the code download to the web root folder (e.g. C:\Inetpub\wwwroot\). We will access this file in our C# Windows application through the URL " http://localhost/interface.xml ". If you want to keep this file in any other location, you may have to create a virtual directory and change the URL used in the file Form1.cs file which we shall talk about later in this article.

The Class Diagram

Following is an UML class diagram that denotes the class relationships and their individual responsibilities.

Creating the project

The following figure shows the creation of the C# windows application using the VS .NET IDE:

This creates a project with a default file Form1.cs. Rename this file to XMLForm.cs. We shall use this code to create our abstract base class, XMLForm. This is the class that provides most of the functionality of building the Windows Form from an XML file. We shall later create the class Form1 that inherits from the XMLForm class, and provides customization to the Form built by the XMLForm class. The reason behind using the IDE generated file is that this provides us with a basic template of a Windows Form to work within. The following figure shows the default code generated by the IDE.

After renaming this file to XMLForm.cs, we shall start modifying this code to suit our requirements. We will add the classes XMLForm, MyTextBox, MyCheckBox and MyControlfactory, and the interface IMyControl that are shown in the class diagram above. We shall later create a separate file, MyFormData.cs, for the data access class MyFormData. After adding the classes and making the modifications to the generated code (as in the last figure), the XMLForm.cs file looks like this:

using System;
using System.Drawing;
using System.Collections;
using System.Windows.Forms;
using System.Xml;

namespace WindowsFormBuilder
{ //The XMLForm base class
public abstract class XMLForm : System.Windows.Forms.Form
{
protected System.Windows.Forms.Button btnSubmit ;
public XMLForm( int userID , int versionID , string url )
{
InitializeComponent();
}
public override void Dispose()
{
base.Dispose();
}
private void InitializeComponent()
{
//**TO ADD Form Building code here**
}
private void submitButton_Click(object sender, System.EventArgs e){}
}//XMLForm class ends
//***Other classes used by the XMLForm class ***
public class MyControlFactory
{ //******add code for the MyControlFactory class ***
}
public interface IMyControl
{
void InitializeControl(int fieldID ,int userID , string caption ,
string name ,int x_axis ,int y_axis ,int tabIndex);
void Save();
}
public class MyTextBox : System.Windows.Forms.Panel , IMyControl
{ //******add code for the MyTextBox class ***
}
public class MyCheckBox : System.Windows.Forms.Panel , IMyControl
{ //******add code for the MyCheckBox class ***
}
}//WindowsFormBuilder

Now that we have the basic class definitions ready, we shall start adding code to them as I explain each of these classes one by one. We shall first look at the other classes in the file, and finally look at our main class, XMLForm, that essentially builds the Form and uses these classes.

The IMyControl Interface

public interface IMyControl
{
void InitializeControl(int fieldID ,int userID , string caption ,
string name ,int x_axis ,int y_axis ,int tabIndex);
void Save();
}

The IMyControl interface exposes the two methods InitializeControl() and Save(). These methods will be called on by the concrete implementations of this interface to initialize the controls and to persist the contained data to the database respectively.

The MyControlFactory class

public class MyControlFactory
{
public static IMyControl CreateMyControl(string type )
{
switch ( type )
{
case "Text":
return new MyTextBox();
case "Checkbox":
return new MyCheckBox();
default:
throw new System.Exception("Control type "+type+" not suppoted.");

}
}
}

This class exposes the factory method CreateMyControl() for the creation of the specific controls. It creates and returns the appropriate control object to the client, or throws an Exception if the type is not supported.

The Custom Controls

We shall now see how to create the custom controls that provide the implementation for the IMyControl interface. The main aim behind using these controls is that the Form Builder is able to work with all the controls added to the Form, as they all belong to a single type, ImyControl, and have a similar behavior that is required by the Form Builder. This saves a lot of type checking and code in the Form Builder that uses these controls.

MyTextBox Control

This class inherits from the System.Windows.Forms.Panel class and implements the IMyControl interface. This is a composition of the System.Windows.Forms.TextBox and the System.Windows.Forms.Label Controls. The custom property Text is exposed for external objects to be able to read or edit the Text property of the TextBox control.

public class MyTextBox :
            System.Windows.Forms.Panel , IMyControl
{
TextBox textBox =
            new TextBox();
Label label = new Label();
private int fieldID
            ;
private int userID;

public override string
            Text
{
get
{
return textBox.Text;
}
set
{
textBox.Text= value;
}
}
public
            MyTextBox()
{
}

public void InitializeControl( int
            fieldID , int userID , string caption , string name ,
int x_axis
            ,int y_axis , int tabIndex )
{
this.fieldID = fieldID ;
this.userID = userID ;

// Initialize the Panel
            control.
this.Location = new System.Drawing.Point(x_axis,y_axis
            );
this.Size = new Size(192 , 40 );

// Initialize the
            Label and TextBox controls.
label.Location = new Point(8
            ,8);
label.Text = caption;
label.Name =
            "lbl_"+name;
label.Size = new Size(50 + caption.Length , 24
            );
textBox.Location = new Point(88 ,8);
textBox.Name =
            "txt_"+name ;
textBox.TabIndex = tabIndex ;
textBox.Size = new
            Size(80, 20);

// Add the Label and TextBox controls to the
            Panel.
this.Controls.Add(label);
this.Controls.Add(textBox);

this.SuspendLayout();
}

public
            void Save()
{
WindowsFormBuilder.MyFormData.CreateField(
            fieldID , userID, this.Text );
}

}

MyCheckBox Control

This class inherits from the System.Windows.Forms.Panel class and implements the IMyControl interface. This is a composition of the System.Windows.Forms.CheckBox and the System.Windows.Forms.Label Controls. The custom property Checked is exposed for external objects to be able to read or edit the Checked property of the CheckBox control. The Database table tblFormData does not have a Boolean field to store the data, so to accommodate the generic values the save() method converts the boolean value to a string.

public class MyCheckBox :
            System.Windows.Forms.Panel , IMyControl
{
CheckBox checkBox =
            new CheckBox();
Label label = new Label();
private int fieldID
            ;
private int userID ;
public bool
            Checked
{
get
{
return checkBox.Checked;
}
set
{
checkBox.Checked= value ;
}
}

public MyCheckBox(){}

public void
            InitializeControl(int fieldID , int userID , string caption , string
            name ,
int x_axis ,int y_axis , int tabIndex )
{
int
            chkXCord = 0;
this.fieldID = fieldID ;
this.userID = userID ;
// Initialize the Panel control.
this.Location = new
            System.Drawing.Point(x_axis,y_axis );
this.Size = new Size(192 ,
            40 );

// Initialize the Label and TextBox
            controls.
label.Location = new Point(8 ,8);
label.Text =
            caption ;
label.Name = "lbl_"+name;
chkXCord = 100 +
            caption.Length ;
label.Size = new Size(chkXCord , 24
            );
checkBox.Location = new System.Drawing.Point(chkXCord + 10 ,
            8);
checkBox.Name = "chk_"+name ;
checkBox.TabIndex = tabIndex
            ;
checkBox.Size = new System.Drawing.Size(16, 24);

// Add
            the Label and TextBox controls to the
            Panel.
this.Controls.Add(label);
this.Controls.Add(checkBox);

this.SuspendLayout();

}

public
            void Save()
{
string text = null ;
if( this.Checked
            )
{
text = "YES";
}
else
{
text =
            "NO";
}
WindowsFormBuilder.MyFormData.CreateField( fieldID
            ,userID, text );
}

}

The save method in the above two classes uses the CreateField() method of the MyFormData class. This method saves the Form data provided as arguments into the database tables. We shall shortly see the implementation of the class MyFormData. The above two custom controls inherit from the System.Windows.Forms.Panel class. Please note that to create these Custom Controls you can alternatively have them inherit from the System.Windows.Forms.Control class or the System.Windows.Forms.UserControl class with no other change in the above code.

The XMLForm base class

This is an abstract base class that inherits from the Form class of the System.Windows.Forms namespace. This class provides the functionality of downloading the XML interface definition file from the web server, interpreting it, and building the Form accordingly. The constructor gets called through the constructor of a derived class when it is instantiated. This constructor in turn calls the InitializeComponent() method, which does the job of the Form building. The InitializeComponent() method walks through the XML document using the System.Xml.XmlTextReader class. Before we jump into the code, let us take a brief look at why we chose this class when we had the other alternative of using the W3C DOM implementation, the System.Xml.XmlDocument class. Note that there currently is no implementation of the SAX2 standard in the .NET Framework SDK.

Choice of the System.Xml.XmlTextReader Class for walking through the XML document.

The XmlTextReader class offers a forward-only parsing. The .Net platform introduces an alternative method for parsing an XML document that exposes a pull model, rather than the push model found in SAX. Rather than loading the entire document into memory, as DOM does, the XmlTextReader class pulls only one node at a time. The forward-only stream creation means that only one node is in memory at any given time. In our example, we do not need to insert, update, or delete nodes within the XML document, so using this model will be more efficient that that of DOM. For more information on the System.Xml namespace read XML in .NET.

The following is the code that goes into the XMLForm class. It is recommended that you simultaneously glance through the interface.xml document structure as you read through the Form building code, which uses this XML document.

public abstract class XMLForm : System.Windows.Forms.Form
{
protected System.Windows.Forms.Button btnSubmit ;
private string url ; //this will hold the HTTP address for the interface.xml file
/*the y_cord variable will be used to maintain a cursor along ther Y-AXIS for
adding the next control along the on the Form*/
private int y_cord = 0;
/*The versionID variable is used to identify the Form definition for the
appropriate version if there are different version of the Form definition in the
XML File*/
private int versionID ;
private int userID ;//to identify the user who submits the Form Data
private const int Y_DIFF = 40; //The distance between two controls on the Y-AXIS

public XMLForm( int userID , int versionID , string url )
{
this.versionID = versionID ;
this.userID = userID ;
this.url = url ;
InitializeComponent();
}

public override void Dispose()
{
base.Dispose();
}

private void InitializeComponent()
{
int fieldCount =0; /*used to keep track of the number of fields added and also the
tab order in the form*/
bool bValidVersion = false ;
/*the controls are not added unless the valid version is found in the XML
document/*
int x_axis= 50 ; //the distance of all the controls from the left margin in the
Form
IMyControl ctrl = null; /* control of the interface type, reused to store the
Custom Controls */
string nodeName = null;
Hashtable myField = new Hashtable();/* The data structure used for state management
as the XmlTextReader does not retain any state */

this.SuspendLayout();
try
{
XmlTextReader xmlReader = new XmlTextReader(url);
While (xmlReader.Read())
{
if (xmlReader.NodeType == XmlNodeType.Element)
{
nodeName = xmlReader.Name;
//keep track of the number of fields added
if( nodeName == 'field' )
{
fieldCount++;
}
if (nodeName == 'frame' && xmlReader.AttributeCount > 0)
{
/*check whether we are processing the frame node that belongs to the
version chosen by the user*/
While (xmlReader.MoveToNextAttribute())
{
if ((xmlReader.LocalName)== 'version' )
{
if( IsValidVersion( Int32.Parse(xmlReader.Value) ) )
{
bValidVersion = true;
}
}
}
}
}
else if(xmlReader.NodeType == XmlNodeType.Text)
{
Switch (nodeName)
{
Case 'type':
/*create the custom control by passing the type as argument to the
Factory method*/
ctrl = MyControlFactory.CreateMyControl(xmlReader.Value);
break;
Case 'background':
//convert HTML color code to System.Drawing.Color
this.BackColor = ColorTranslator.FromHtml( xmlReader.Value );
break;
Case 'title':
//the title in XML becomes the Form title
this.Text = xmlReader.Value ;
break;
Case 'id':
goto case 'caption';
Case 'name':
goto case 'caption';
Case 'caption':
myField[ nodeName ] = xmlReader.Value;
break;
}
}
else if (xmlReader.NodeType == XmlNodeType.EndElement)
{
if( xmlReader.Name == 'field' && bValidVersion )
{
y_cord = y_cord + Y_DIFF;
ctrl.InitializeControl( Int32.Parse( myField[ 'id' ].ToString() )
, userID , myField[ 'caption' ].ToString() ,
myField[ 'name' ].ToString() , x_axis, y_cord ,fieldCount-1 );
this.Controls.Add((System.Windows.Forms.Control) ctrl );
myField.Clear();
}
}
}
}
catch(Exception e )
{
MessageBox.Show('An Exception occured:'+ e.Message );
}

As mentioned earlier, the XmlTextReader maintains only one node in memory as it parses the document with a forward only cursor. Just like in the SAX model, developers have to create their own data structures for state management. In documents with complex hierarchies, these data structures can be pretty complex. In our XML document we have to retain the memory of the one field as we get to the end tag of the field element. This is realized by the use of a hash table object myField. As the end tag of a field element is reached, the control is intialized and is added to the controls collection of the Form using the add method of the Form.Controls collection. The InitializeControl() method is called on every control created by the factory method. The advantage of defining the IMyControl Interface is more apparent now, as we initialize every control in the same manner irrespective of its type. The details of initialization are encapsulated into the specific controls. Note that the variable bValidVersion is a flag indicating whether we are currently parsing the frame element with the right version. In the above code we are checking for this flag before adding the custom control to the Form. The XML file in our test application contains only a single frame element. With a greater number of frame elements we would have to check for this flag at all other places, which I have avoided here to keep it simple.

this.AutoScaleBaseSize = new
            System.Drawing.Size(5, 13);
this.ClientSize = new
            System.Drawing.Size(280 , y_cord + 100 );

this.btnSubmit =
            new System.Windows.Forms.Button();
this.btnSubmit.Location = new
            System.Drawing.Point(88, y_cord+Y_DIFF );
this.btnSubmit.Name =
            "button1";
this.btnSubmit.Size = new System.Drawing.Size(184,
            24);
this.btnSubmit.TabIndex = fieldCount
            ;
this.btnSubmit.Text = "Submit";
this.btnSubmit.Click += new
            System.EventHandler(this.submitButton_Click);

By this step the Event Handler submitButton_Click has subscribed itself with the button " btnSubmit". On a click event on the button, this Event handler will be notified so that it does the job of submission of the Form data to the database.

this.Controls.Add( this.btnSubmit
            );

this.ResumeLayout(false);
}

This is a simple utility method which checks if the version is a valid one. This is provided as a separate method so as to accommodate future changes in the logic for validation. You may always return trueif you want to have a single version.

private bool IsValidVersion(int versionID
            )
{
return (this.versionID == versionID );
}

Visual Studio .NET provides a Windows Form designer test container to test your Form. This Designer adds the generated code to the InitaializeComponent() method. Although the drag-and-drop Form building process simplifies development, for special kinds of applications like this one we have to write our own Form building code. Owing to the dynamic nature of the Form, the Form designer will not be able to load the Form into the design view, so the only way to test your code will be to run the application.

A glance at the XMLForm.cs file in the design mode will show something like this:

Looks awkward? Wait! We are not done yet. This looks awkward because this is all the designer could interpret out of the file. The actual Form will be generated when we pass the URL for the XML file, which we will get at after a few more steps. A word of caution: never try to click or drag-and-drop some control over this Form at this stage. If you do that, the designer will regenerate the code and you will loose all the code that you manually added so far.

Handling the Form Data Submission

On clicking the submit button, the observer is notified and the method submitButton_Click is invoked. The IEnumerator interface supports simple iteration over a collection. An Iterator provides a way to access the elements of an aggregate object (the concrete implementation of the IMyControl interface in this case) sequentially without exposing its underlying representation.

private void submitButton_Click(object
            sender, System.EventArgs e)
{
IEnumerator ctrlEnum =
            this.Controls.GetEnumerator();
try
{
while
            (ctrlEnum.MoveNext())
{
if (!
            ctrlEnum.Current.GetType().ToString().Equals("System.Windows.Forms.Button")
            )
{
IMyControl ctrl = ( IMyControl )
            ctrlEnum.Current;
ctrl.Save();
}
}
MessageBox.Show("Successfully
            saved the Form Data");
}
catch(Exception exception
            )
{
MessageBox.Show("An exception occurred. Could not save the
            Form Data.");
}

}

We can see the polymorphic behavior in the above code. The save() method is called on every element in the controls collection. The concrete implementation of the IMyControl Interface, like the MyTextBox and the MyCheckBox, have the best knowledge of their internal class representation and have been provided with the behavior of how to save themselves to the database. For instance the MyCheckBox class checks for a Boolean value of checked status and converts it to a string as "YES" or "NO".

Saving the Form Data into the Database: MyFormData Class

We shall look at creating the class MyFormData that handles the database access. We have maintained it in a separate .cs file so as to separate the data access layer from the presentation. This class exposes a method CreateField() which is called from the save method of the custom controls. This method saves the Form data to the database table tblFormData. From the Solution Explorer, add a new Item as shown below:

The following is the code for the MyFormData class:

using System;
using
            System.Data;
using System.Data.OleDb;

namespace
            WindowsFormBuilder
{

public class MyFormData
{
private static System.Data.OleDb.OleDbConnection myConnection =
            null ;
private static String connString =
            "Provider=SQLOLEDB;Server=FormServer;uid=sa;pwd=;database=Form";

//static
            constructor creates an instance of the OleDbConnection
            class
static MyFormData()
{
myConnection = new
            System.Data.OleDb.OleDbConnection
            (connString);
myConnection.Open();
}

The static constructor makes the connection object available when the MyFormData class is first referenced.

//creates a field in the database table
            tblFormData
public static void CreateField( int fieldID ,int
            userID , string strValue )
{
myConnection =
            GetConnection();
String sql = "insert into tblFormData (iFieldID
            , iUserID , vcValue , sdtCreatedDate ) values " +
"(" + fieldID +
            "," + userID + ",'" + strValue + "','" + DateTime.Now.ToString() +
            "' )" ;

System.Data.OleDb.OleDbCommand cmd = new
            System.Data.OleDb.OleDbCommand();
cmd.Connection=myConnection;
cmd.CommandText=sql
            ;
if( cmd.ExecuteNonQuery() < 1 )
{
throw new
            System.Exception("The Field could not be saved into the Data
            Store.");
}
}

//GetConnection ensures a single
            instance of the OleDbConnection class
public static
            System.Data.OleDb.OleDbConnection GetConnection()
{
if(
            myConnection == null )
{
myConnection = new
            System.Data.OleDb.OleDbConnection
            (connString);
myConnection.Open();
}
else if (
            myConnection.State == System.Data.ConnectionState.Closed
            )
{
myConnection.Open(); }
return myConnection
            ;
}

}

A Connection pool is used in multi-user applications, like a web application, where connection is a valuable resource, and releasing and establishing connections for every request would greatly reduce the system performance. When you use the OLEDB .NET Data Provider, you do not need to enable connection pooling because the provider manages this automatically. In Windows based single user applications, developers develop their modules separately, and when the modules are integrated it comes to our notice that more than one simultaneous connection object is created. Using a singleton pattern for the creation of the connection allows us to instantiate and maintain a single connection object. The singleton design pattern ensures that a class only has one instance, and provides a global point of access to it. The connection should be obtained through a single method called GetConnection() all through the application. A call to the close method releases the connection to the connection pool, or closes the connection if the connection pooling is disabled. A call to the static method OleDbConnection.ReleaseObjectPool() indicates that the connection pooling can be cleared when the last underlying OLE DB Provider is released.

Extending the framework and Customizing the Form in the derived Form1 class

Now that we have created the base class XMLForm, and the data access class MyFormData, let us do the final step to get our application running. We shall add a Windows Form as shown below:

With the new file Form1.cs added, the Solution Explorer looks as below:

Make this class inherit from the WindowsFormBuilder.XMLForm instead of the System.Windows.Forms.Form that it was inheriting from in the generated code. With that step, all the XML Form building functionality from the base class XMLForm becomes available in this derived class. The Main() method starts the application by creating an instance of the Form1 class. If you have not placed the interface.xml file in the web root folder, you will have to provide the appropriate URL in the argument to the constructor.

using System;
using
            System.Windows.Forms;
using System.Drawing;
using
            WindowsFormBuilder;

public class Form1 :
            WindowsFormBuilder.XMLForm
{
public Form1(string formTitle
            ,int userID , int versionID , string url ):base( userID , versionID
            , url )
{
//customizations
this.Text = formTitle
            ;
this.BackColor = Color.Beige;
this.Font = new
            System.Drawing.Font("Microsoft Sans Serif", 10
            );
this.btnSubmit.Size = new System.Drawing.Size(100,
            24);
}

[STAThread]
static void Main()
{
Application.Run(new Form1("C# XML Form Builder",1 , 1 ,
            "http://localhost/interface.xml" ));

}
}

We saw how we can provide customizations to the base class, like changing the background color, the font and the size for example. The button btnSubmit has a protected access modifier allowing the derived classes to customize it. The classes can override the event handler, and provide a different OnClick Event handler to provide a different way of saving the Form data. This may be used to save to another XML file instead of saving to the database for example. Now that all the pieces are in place, we can build the project and run it to see the results.

The resulting C# Form

The following figure shows how the generated Form looks after applying the customizations in the derived class.

You can enter the relevant data into the Form and save it to the database by clicking the submit button. That completes our building and using of the XML Form.

If you wish to try generating some extra stuff on-the-fly, and you are new to the System.Windows.Forms assembly, or you have been using only the drag-and-drop feature of the Form designer, here is a small short-cut to make your job simpler. You can learn from the IDE code generator. First, use the drag-and-drop feature to build a static Form, keeping in mind the Form that you want to be generated dynamically, and then study the code that is generated. Then try to get that code into some conditional statements. This kind of framework is more familiar to a Visual Basic Windows application developer, where he is made to think in terms of the event-handlers rather then the nuances of the Form building. But here (both in C# as well as VB .NET, as they share the .NET Framework SDK) as long as you wish to do so, you are free to modify the code to suit your own Form building logic.

Changes from the .NET SDK Beta1 to .NET SDK Beta2 used in the code

You will notice the following changes from Beta1 to Beta2 that I have used in the code. To find more about other changes go to Change Documents.

.NET SDK Beta1 | .NET SDK Beta2
System.WinForms Namespace | System.Windows.Forms Namespace
System.ADO Namespace | System.OleDb Namespace
ADOConnection class | OleDbConnection class
ADOCommand class | OleDbCommand class

Visual Basic .NET Form inheritance from a C# ClassLibrary

We implemented the XML Form builder in C#, now what if you require the same functionality in a Visual Basic .NET Form? Let us see how we make use of the above code to achieve the same in Visual Basic .NET, without having to re-invent the wheel. All you need to do is to convert the "WindowsFormBuilder" namespace into a .NET Class Library and you are ready with a Base class for the both the .NET languages. The first step is to create a new Class Library. This creates a project with the default file Class1.cs. Delete this file and add the two files XMLForm.cs and MyFormData.cs to the project, and then build the project. This get you ready with the WindowsFormBuilder.dll. The next step is to create a Visual Basic .NET Windows Application. Add a reference to the component file WindowsFormBuilder.dll that you just created by using the browser button.

Delete all code in the auto-generated file Form1.vb and cut and paste the code below into this file.

Public Class Form1
Inherits WindowsFormBuilder.XMLForm

Public Sub New()
MyBase.New(1, 1, "http://localhost/interface.xml")
End Sub
End Class

Build and run the application and you will see the Form generated without any added customizations. Being a VB middleware COM developer, although I've not worked much on Visual Basic Windows Applications, my experience with Java helped me create a Windows Form on-the-fly from XML within a minute. You can find the Class Library "WindowsFormBuilder.dll"; in the code download, and try out the Visual Basic .NET Form inheritance. However, the data access class will not work unless you have the same connection string as in the code download. So, to get the fully working version, it is recommended that you follow the above steps to create your own Class Library. A good introduction to .NET Windows Form Inheritance can be found at http://www.asptoday.com/content/articles/20010424.asp?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq

Inheritance or Aggregation?

Whether to inherit or to aggregate has always been a point of discussion amongst Object Oriented Designers. So, was our approach of specialization from the base class XMLForm right? As long as we aim towards using an XML based dynamic Form generation, we are on the right track. In the future we could be adding different classes to the WindowsFormBuilder namespace, providing different implementations like "database" driven Form building, building Forms from a serialized Form Object, or a "static" Form. So, if we want to have the flexibility at run time to change the implementation, we will find that aggregation will be a better solution.

Next steps and Conclusion

  • Develop a web or Windows based tool to define the Form and store it in the database.
  • If we were to develop a Web Service for serving the interface definition file, there would be no need to download the entire XML document from the web server if the XML is cached on the client and we could invoke a SOAP method with the last cached version or Time-Stamp. In response, the client would receive an indication to use the cached file or a single 'frame' element corresponding to the latest version of the Form that could be returned by the Web Service.
  • The Windows client is assumed to be on the same network as the database server. This might not always be case; they can be across the Internet. So building a Web service would be an ideal step.
  • Add an isValid() method to the IMyControl interface, and provide implementation for the field validations.
  • Building Forms on the fly will have performance limitations over Forms containing design time controls. The trade-offs between flexibility and performance have to be considered by the designers.

In this article we saw how to build a .Net Windows Form dynamically from an XML file. I shall discuss Building Dynamic Interfaces with XML and ASP.NET server controls in a future article.

C#Today - © 2002 WroxPress Limited


Multiple Thread Techniques for a C# Windows Service

Multiple Thread Techniques for a C# Windows Service
http://www.csharptoday.com/content.asp?id=1579
Article Rating: 4
Published on September 26, 2001
Abstract
Visual Studio .NET allows us to get a basic Windows service up and running quickly. The next step is to give our service the high-performance features and configurability required for real world applications. In this article, Scott Allen demonstrates how to use multithreaded techniques in .NET to achieve higher throughput and scalability in a sample Windows service. In addition, we cover configuration of the service via an XML file. In order to demonstrate the techniques discussed in the article we develop a "stock alerts" Windows service, which retrieves stock quotes from a web service and emails interested parties when a stock price moves past a specific threshold. The topics covered inside the article are also applicable to other .NET application types.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

Visual Studio .NET allows us to get a basic Windows service up and running quickly. The next step is to give our service the high-performance features and configurability required for real world applications. This article demonstrates how to use multithreaded techniques in .NET to achieve higher throughput and scalability in a sample Windows service. In addition, the article covers configuration of the service via an XML file. The topics covered inside the article are also applicable to other .NET application types. For more information on the basics of a Windows service, including requirements and installation, consult Brian Patterson's C# article Creating a Windows Service in C# - a Network Status Monitor.

In order to demonstrate the techniques discussed in the article we develop a Windows service to provide "stock alerts". The stock alert service retrieves stock quotes from a web service and emails interested parties when a stock price moves past a specific threshold.

Configuration

Services have several options available for storage and retrieval of application specific configuration information. For example, the stock alert service needs to keep a list of user names and email addresses. Each of these users will have a list of stock symbols with a high and low price threshold for the service to watch. In a file production application, you probably want to store configuration information in the database, but for this article we will use an XML configuration file. The technique below works well for either scenario. The basic code is all ready for you to try out in the download: simply open StockAlert.sln in Visual Studio .NET and follow the instructions here and in the readme.txt file in the download to customize and create the service for yourself.

Turn XML into an Object

One technique for reading and writing XML from C# is to use a typed DataSet. A typed DataSet turns XML elements into DataTables with the XML attributes as columns. Typed DataSets also allow you to access the underlying data with strongly typed properties and the help of the IDE's Intellisense. We create a typed DataSet by starting with a representative XML file. The stock alert service uses the following XML structure to store the users, symbols, and thresholds.

<?xml version="1.0" encoding="utf-8" ?>
<Alerts>
<User Name="Jane Smith" email="jane.smith@nowhere.net">
<Watch Symbol="IBM" Hi="130" Lo="92" />
<Watch Symbol="MAT" Hi="140" Lo="82" />
<Watch Symbol="TOY" Hi="150" Lo="70" />
</User>
<User Name="Alex Lifeson" email=" alex.lifeson@nowhere.net ">
<Watch Symbol="AYE" Hi="140" Lo="82" />
<Watch Symbol="TOY" Hi="120" Lo="80" />
</User>
</Alerts>

With the XML file open in the Visual Studio IDE, right click on the alerts.xml pane and click on "Create Schema ". This creates an XSD schema file as shown below:

Make sure to double-check all of the data types, as the generated schema often defaults all the attributes typed to string. The schema generator understands the parent-child relationship between a user and their watch list, and generates additional ID columns for a DataRelation to work correctly. You can also generate typed DataSets from OleDB data providers.

With the schema in place, you can click on "Generate DataSet " from the "Schema " menu. Visual Studio saves classes for the typed DataSet in a source code file underneath the XSD file. Select "Show All Files " in the solution explorer to see the Alerts.cs file:

The generated classes make it extremely easy to maneuver through the XML. The users and their stocks become tables with collections of rows. As an example, the following code snippet displays all of the users watching the stock symbol TOY.

Alerts alerts = new
            Alerts();
alerts.ReadXml(XML_FILE_PATH);

Alerts.WatchRow
            [] watches;
watches = (Alerts.WatchRow
            [])alerts.Watch.Select("SYMBOL = 'TOY'");

for(int i = 0; i
            < watches.Length; i++)
{
Alerts.UserRow user =
            watches[i].UserRow;
Console.WriteLine("User {0} High: {1} Low:
            {2}",
user.Name, watches[i].Hi, watches[i].Lo);
}

Multithreading

Our next step is to concentrate on retrieving quotes and sending notifications. The quickest implementation is to simply loop through the dataset periodically and compare the quotes to the user thresholds. The quick implementation could take the following code and place it inside a loop.

StockQuote.StockQuoteWebService quoteService
            =
new StockQuote.StockQuoteWebService();

for(int i = 0; i
            < alerts.User.Count; i++)
{
Alerts.WatchRow[] watches =
            alerts.User[i].GetWatchRows();
for(int j = 0; j <
            watches.Length; j++)
{
double quote;
string stock =
            watches[j].Symbol;

quote =
            quoteService.GetQuote(stock);

if(quote < watches[j].Lo ||
            quote > watches[j].Hi)
{
// send notification and delete
            watch
            record
}
}
}

Thread.Sleep(WAITTIME);

The code shown above can complete the basic task, but there are a few more requirements for our service:

  • The service needs to take advantage of additional CPUs on multiple processor machines and run web service requests concurrently.
  • The service should retrieve quotes at regular intervals (every three minutes).

These requirements give us a scalable and dependable service for the users. To satisfy the requirements we dig into the System.Threading namespace to add additional capabilities to our service.

Service Start

The OnStart method is where the service begins to work.

protected override void OnStart(string[]
            args)
{
alerts = LoadAlerts();
eventFinished = new
            AutoResetEvent(false);
timer = new Timer(new
            TimerCallback(OnTick), null, WAIT_TIME, 0);
}

The service launches by loading the user preferences into the typed DataSet (described earlier) and creating a new AutoResetEvent (described later). The service then initializes a timer to begin the process of checking stock quotes every three minutes. These preferences do not change during the lifetime of the application. If you have a user interface provided to change and update the stock alerts configuration, the service would need to reload the DataSet each time the timer fires.

Timer Queues

It is important to realize the timer is not the Timer class from System.Windows.Forms. As mentioned in the documentation, the forms based timer works best in applications with Windows and UI message pumps, and thus is not a good match for our service. Our timer is a lightweight resource built on the new timer queue APIs in Windows 2000 and resides in the System.Threading namespace.

The Beta 2 documentation for System.Threading.Timer is still sparse, but you can pick up many of the required details from the Win32 API reference for timer queues (http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dllproc/hh/winbase/synchro_0z03.asp&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq). The first parameter is a TimerCallback delegate with the method to invoke when the timer expires. The second parameter is of type object and allows the caller to pass state information to the event handler.

The third and fourth parameters to the Timer constructor are dueTime and period. The dueTime specifies how many milliseconds to count before the timer invokes the callback function. The period parameter (also specified in milliseconds) creates a reoccurring timer.

After the timer counts off WAIT_TIME (dueTime) milliseconds, the timer fires the OnTick event and stops. If instead we pass WAIT_TIME as the period parameter, the timer would fire OnTick and automatically reset itself to countdown again.

It is tempting to create a periodic timer and blindly have the timer continue to invoke the OnTick method. However, a "one shot" timer is the safer alternative in this situation. Consider a case where we have a slow web service and a large number of quotes to retrieve. If we do not finish retrieving all of our quotes before the next OnTick invocation, the requests begin to overlap and eventually we might bog down our service with threads and web service requests. As we will see next, the service tries to retrieve quotes as close to the specified interval as possible without allowing a logjam to occur.

OnTick

Once the timer fires OnTick the heavy work begins.

protected void OnTick(object
            state)
{
long nextTime = 0;
try
{
DateTime start =
            DateTime.Now;
Hashtable symbols =
            GetUniqueSymbols();
nRequests =
            symbols.Count;

if(nRequests >
            0)
{
QueueGetQuotes(symbols);
eventFinished.WaitOne();
if(alerts.HasChanges())
{
alerts.AcceptChanges();
alerts.WriteXml(CONFIG_FILE);
}

TimeSpan
            elapsed = DateTime.Now - start;
nextTime = WAIT_TIME -
            (long)elapsed.TotalMilliseconds;
if(nextTime <
            0)
{
nextTime = 0;
}

}
}
catch(Exception
            e)
{
EventLog.WriteEntry(e.Message,
            EventLogEntryType.Error);
}
finally
{
timer = new
            Timer(new TimerCallback(OnTick), null, nextTime,
            0);
}
}

The first step is to mark the start time. We need to calculate the elapsed time for processing and start a new timer before the method exits to keep the OnTick three minutes apart. The new timer is created inside a finally block to ensure a new timer is created, even when an exception occurs.

There is a Change method on the System.Threading.Timer class, and I have successfully used the Change method to reload a one shot timer in .NET. According to the Win32 documentation, a one shot timer is not updateable via ChangeTimerQueueTimer, so perhaps the .NET framework is performing some magic! Until the product provides better documentation, the safe course is to start a new timer.

Next, the service retrieves all of the unique stock symbols from the alerts DataSet. We only want to make one request of the web service for each stock symbol, even if the symbol is present in more than one user's watch list. Although the DataTable can interpret a wide variety of expressions in the Select method, there does not appear to be an equivalent for SQL's SELECT DISTINCT. Instead, we build the unique collection of symbols using a for loop and a Hashtable.

protected Hashtable
            GetUniqueSymbols()
{
Hashtable symbols = new
            Hashtable();
for(int i = 0; i < alerts.Watch.Count;
            i++)
{
if(!symbols.ContainsKey(alerts.Watch[i].Symbol))
{
symbols.Add(alerts.Watch[i].Symbol,
            null);
}
}
return symbols;
}

Thread Pools

The service calls QueueGetQuotes asynchronously to fetch the quotes.

protected void QueueGetQuotes(Hashtable
            symbols)
{
foreach(DictionaryEntry d in
            symbols)
{
ThreadPool.QueueUserWorkItem(new
            WaitCallback(GetQuote), d.Key);
}
}

There are several techniques available for asynchronous processing in .NET. For example, the preferred asynchronous programming model in .NET is to use the BeginInvoke member of our StockQuoteWebService class (see http://msdn.microsoft.com/code/default.asp?URL=/code/sample.asp?url=/MSDN-FILES/026/002/420/msdncompositedoc.xml&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq for a code sample with this technique). Alternatively, we could create an array of System.Thread objects to start and stop worker threads. Both of these techniques are feasible, but the StockQuoteService demonstrates a third technique using the ThreadPool class.

The System.ThreadPool class keeps the burden of worker thread management in the framework and operating system. The thread pool also allows us to work asynchronously and take advantage of multiple CPUs on the host machine. Even on single CPU machines, the thread pool improves throughput by intelligently managing the threads. For a more detailed look at the underlying architecture of the thread pool, consult Jeffrey Richter's article on MSDN (http://www.microsoft.com/msj/defaultframe.asp?page=/msj/0499/pooling/pooling.htm&nav=/msj/0499/newnav.htm&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq).

The QueueUserWorkItem method accepts a WaitCallback delegate to the method for the worker thread to execute. We use the second parameter (state, a generic object) to pass the stock symbol to the method. Before we look at the GetQuote method, we can finish the remainder of the processing in OnTick.

Synchronization Events

We created an AutoResetEvent when the service started processing. The AutoResetEvent and ManualResetEvent classes have nothing in common with the events and event handlers commonly associated with user interface controls. The above classes build upon the Win32 CreateEvent API, and we use these objects to synchronize threads. A worker thread can notify a sleeping thread of an important occurrence by signaling an event.

A thread invoking the WaitOne method of an event object is going to block (sleep) when the event is in the non-signaled state. We created our event in the non-signaled state during the OnStart method of the service (the constructor parameter). When our last worker thread finishes processing, we change the event to the signaled state using the Set method. If the thread inside OnTick is sleeping when the event signals, then it wakes up and resumes execution. The rest of the OnTick method saves the alerts in XML (if there are changes detected) and computes the dueTime for the new timer.

AutoResetEvent objects automatically reset themselves to the non-signaled state after releasing a single waiting thread. In contrast, the ManualResetEvent allows any number of threads to resume processing, and requires an explicit Reset call to return to the non-signaled state.

It is possible for the thread in OnTick to reach the waitOne method after the worker threads have already completed processing. This is part of the fun in multithreaded programming: trying to spot potential deadlocks, race conditions, and missed signals! Fortunately, the AutoResetEvent remains signaled if the program calls Set and no threads are waiting.

GetQuote

The GetQuote method performs most of the work we saw in the first code iteration.

protected void GetQuote(object
            state)
{
try
{
// retrieve the stock symbol
string
            symbol = (string)state;
// retrieve a quote for the
            symbol
double quote =
            CallWebService(symbol);

if(!double.IsNaN(quote))
{
//
            select all the rows with this stock symbol
Alerts.WatchRow []
            watches;
string query = String.Format("SYMBOL = '{0}'",
            symbol);
watches = (Alerts.WatchRow
            [])alerts.Watch.Select(query);

for(int i = 0; i <
            watches.Length; i++)
{
// for each row, check the threshold
            values.
if(quote < watches[i].Lo || quote >
            watches[i].Hi)
{
Alerts.UserRow user =
            watches[i].UserRow;
SendNotification(symbol, quote, user.Name,
            user.email);
// remove the row to stop further
            notifications.
watches[i].Delete();
}
}
}
}
catch(Exception
            e)
{
EventLog.WriteEntry(e.Message,
            EventLogEntryType.Error);
}
finally
{
if(Interlocked.Decrement(ref
            nRequests) ==
            0)
{
eventFinished.Set();
}
}
}

The method begins by retrieving the stock symbol passed in the state parameter and passing the symbol to the web service for a quote (we will examine the CallWebService method in the next section). Once we have a valid quote we select all the watch entries for the symbol and check the threshold values. If the quote exceeds a threshold, we send an email notification and delete the watch row so the program does not send further notifications. The SendNotification method is listed below:

protected void SendNotification(string
            stock, double quote, string name,
string
            email)
{
MailMessage message = new
            MailMessage();
message.From = FROM_ADDRESS;
message.Subject =
            SUBJECT_LINE;
message.To = String.Format("{0} <{1}>", name,
            email);
message.Body = String.Format("Hello {0},\n The price of
            {1} is now {2}!",
name, stock,
            quote);
SmtpMail.Send(message);
}

Interlocking

Before the GetQuote method exits, we need to decrement and then check the number of outstanding requests left. If the thread has just finished the last of the requests, the thread sets the eventFinished event to signal the thread waiting in OnTick. Once again we use a finally block to ensure this piece of code is always executed. It is important to use the Interlocked class to decrement nRequests, since multiple threads are sharing this variable. The Decrement function subtracts one and checks the result in an atomic operation. Without an atomic operation, it is possible for one thread to decrement the value and then a second thread interrupt and decrement the value again, before the first thread has a chance to check the resulting value. Multiple threads are also modifying the DataSet, but each thread is responsible for a single symbol and therefore each thread modifies a different set of rows.

The Web Service

The code to execute the web service call is shown below:

protected double CallWebService(string symbol)
{
StockQuote.StockQuoteWebService quoteService =
new StockQuote.StockQuoteWebService();

double quote = double.NaN;

try
{
quote = quoteService.GetQuote(symbol);
}
catch(Exception e)
{
string message = String.Format("GetQuote failed for symbol: {0}. {1}",
symbol, e.Message);
EventLog.WriteEntry(message);
}
return quote;
}

Our web service should look like this:

To debug the service, a simple test web service is included in the accompanying download. The first request for a given symbol always returns a quote of 100.0. Each subsequent request for the same symbol follows a trend and continues to increment or decrement the symbol's quote by five points on each subsequent call. When we run the code as per instructions in the readme.txt file we get an alert in our email should the stock fall below the value specified in alerts.xml. Note that we will need SMTP Server running in order to send the email.

[WebMethod]
public double GetQuote(string
            symbol)
{
if(Application[symbol] ==
            null)
{
Application[symbol] =
            100.0;
}
else
{
if(symbol.GetHashCode() % 2 !=
            0)
{
Application[symbol] = (double)Application[symbol] -
            5.0;
}
else
{
Application[symbol] =
            (double)Application[symbol] + 5.0;
}
}
return
            (double)Application[symbol];
}

Downloads

The accompanying download for this article includes the full source code for the Windows service and web service projects. The code also uses a semaphore class to throttle the number of concurrently executing web service requests. To run the service successfully you will need to build both projects, and then install the service with the installutil.exe provided by the framework. Make sure you change the alerts.xml file, and modify the constants at the bottom of StockAlertService.cs to reflect your configuration. Finally, double check the URL for the web service in the StockQuoteWebService.cs file.

Conclusion

Multithreaded programs are difficult to design, implement, and troubleshoot. The .NET framework does a superb job wrapping and extending the underlying Win32 threading and synchronization primitives into an object-oriented framework. However, the developer is still responsible for careful coding to avoid pitfalls. I hope that this article and accompanying code demonstrate the potential of the typed DataSet, and the multithreading capabilities of the .NET framework. These classes help reduce the amount of code and increase the performance of your .NET applications.

C#Today - © 2002 WroxPress Limited


Thread Pools

Thread Pools
http://www.csharptoday.com/content.asp?id=1653
Article Rating: 4.5
Published on September 12, 2001
Abstract
Using threads makes your multitasking applications run more smoothly. However, it takes finite resources to create and destory threads, resources we can save if we use thread pooling. In this article, Matthew Reynolds looks at how to use the thread pool in .NET. We see how to build a demo application that shows the typical model used in service applications ?i.e. a client connects, a client requests some work be done and then the client disconnects.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

In April, I wrote an article for ASPToday on the use of System.Threading to provide multithreading to your applications. From the feedback I received, I know this is a hot topic to all of you out there, so in this article I'll be looking at an alternative way to use threads: the Thread Pool.

For those of you who don't subscribe to ASPToday and don't have access to previously published articles, I'll quickly take you through a few essential threading concepts.

A thread is simply a way of making different parts of your program run simultaneously. You've already seen threading in action in Microsoft Office: Word spell checks a document as you type, Excel recalculates data as you change cells and Outlook can check for new e-mails as you're reading ones you've already downloaded.

Threads are particularly useful when developing applications that have a complex user interface but need to do something. For example, if you have an application that searches for all of the .doc files on your computer, you'd build the user interface in the normal way and run the program. The user interface portion, i.e. the form and controls, runs in the "primary application thread". Every program when running has to have at least one thread. When you click a button or indicate in some other way that the program needs to look for all of the .doc files, you "spin up" another thread. This other thread does the work, while the primary thread keeps running allowing you to continue to use the user interface.

This approach makes your program run far more "smoothly". Imagine you wanted to click a button on the form but that program was also scanning all of the files in a given folder. In an application with only a single thread you'd have to wait for the file scanning to be completed before the button would respond to your click. This gives the impression that the application "stalls", creating a far less compelling user experience. With two threads, because each thread is running independently, the user interface responds as quickly as if the application wasn't scanning files because neither operation is affecting each other.

The main difficulty when developing multithreaded applications is it's far easier to write programs that hang or crash for seemingly random reasons, and far harder to debug the application to discover the cause of the crash. To ensure multithreaded applications run free of problems, you need to get involved with something called "synchronization". Unfortunately for us this topic is way beyond the scope of this discussion, but was covered previously in the ASPToday Threading article.

Threads in Services

The two most common places to find threads are when building desktop applications and when building service applications. (For the most part, ASP and ASP.NET developers don't need to worry about threads, hence the reason why this follow-up article is on C#Today, not ASPToday!)

In this article I'm going to introduce the concept of threads as far as they are used by service applications. (Check out Rob Schieber's .NET Services article for more information on service applications.)

Service applications typically work by providing an end-point that client applications can connect to. A Web server is a classic example - you run a program on the server that listens for connections from a Web browser. When a browser connects, that browser requests a resource and waits for the server to respond - either returning the resource or some reason why it cannot be returned. When the browser has received the response, it disconnects and the server waits for another incoming connection.

This request/response can be looked on as a single package, and in this article we'll be calling this a "work item". The browser has provided the server with a "job of work". The server has done the job and then sent the results back.

Imagine you have five browsers all simultaneously connecting and providing a job to the server. Each of those five jobs will run in five separate threads simultaneously. This provides maximum efficiency to the browser - at least in theory! With only a single thread, you'd have a queuing situation going on. Browser B would have to wait for Browser A's response to be returned before Browser B can start work. Browser C would have to wait for Browser B and so on until all five had been received. Although Browser A would "feel" a good response time, Browser E could potentially wait a very long time. Again, we come back to "smoothing" out the way the application works.

The problem with threads is that it takes time to spin up a thread. Although the time taken is minimal, if we didn't have to spin up a thread every time we have a connection come into the server, we'd obviously save time. In theory, if we have five threads that have all finished their work, we can reuse those same threads for subsequent connections.

This is precisely the theory behind thread pooling. For each program (known as a "process"), Windows manages a thread pool. If we request a thread from the pool and it's empty, Windows will spin up a new thread and pass it to us. If later on we request another thread and there's one available in the pool, Windows will give us the existing thread back for us to reuse. Windows also carefully monitors the utilization of the pool and, if need be, will spin up new threads to help increase the application response times and spin down old threads if they have been idle for a long time.

Using the Thread Pool

In this article we're going to take a look at how we can use the thread pools in our application. As this has been a pretty dry discussion so far, the application we're going to build is going to use graphics to illustrate the use of threads.

We're going to build a user control that shows circles bouncing around inside a small box, like this:

Each one of these circles represents a client. In order to move itself around the screen, the client has to ask the server to adjust its movement - i.e. it provides a work item to the server that says "here I am, this is my direction, this is my speed, now move me".

Whenever a circle is moved, the circle causes an event to be fired. We can use this event to determine which threads are running on the system at any one time. (The screenshot above shows that we know about four threads, and we can also see which one of the circles the thread has moved and the time at which the adjustment was made.)

Building the Client

Open Visual Studio.NET and create a new Visual C# - Windows Application project. Call the project Thread Pool Demo. We'll concentrate on building a user control that displays the bouncing circles first, so in Solution Explorer create a new User Control called CircleView.

When the control has been created, change the BackColor property to black using the Custom tab.

Add a new Timer control to the control. Change its Name property to timerUpdateView. Change its Enabled property to True.

Using Solution Explorer once more, create a new class called Circle. Add a namespace import declaration to System.Threading and System.Drawing:

using System;
using System.Drawing;
using System.Threading;

Each circle will have an ID, a location, a speed, a color, a size and a direction. The direction will be one of the following: down and right, down and left, up and right and up and left. Depending on which edge of the control the circle strikes, we'll be responsible for adjusting the direction. (So, if the circle hits the right-hand edge of the control and is heading "up and right", it will need to move "up and left" and so on.)

To store the direction we'll use an enumeration. Add this enumeration to Circle:

public class Circle
{
// enum...
public enum CircleDirection
{
DownRight = 0,
DownLeft = 1,
UpRight = 2,
UpLeft = 3
}

Then, add these members:

      //
            members...
public int Id;
public Point Location;
public
            CircleDirection Direction;
public Size Size;
public Color
            Color;
public int Speed;
public CircleView View;

The size, color, location and speed of the circle will be chosen at random when the circle is created. Add this code to the constructor:

      public
            Circle(CircleView view)
{
// store the view...
this.View =
            view;

// set up...
Random random = new Random();
int
            width = random.Next(20, 50);
Size = new Size(width,
            width);
Color = Color.FromArgb(random.Next(255),
            random.Next(255), random.Next(255));
Speed = random.Next(5,
            25);
Location = new Point(random.Next(view.Width),
            random.Next(view.Height));
}

When we need to paint the circle, we'll ask it to do it for us. Add this method:

      //
            Paint.
public void Paint(Graphics graphics)
{
// how big
            are we.
RectangleF rectangle = new RectangleF(Location.X -
            (Size.Width / 2),
Location.Y - (Size.Height / 2),
Size.Width, Size.Height);

// draw the circle.
Brush
            brush = new SolidBrush(Color);
graphics.FillEllipse(brush,
            rectangle);

// draw the text.
String buf =
            Id.ToString();
brush = new
            SolidBrush(Color.White);
StringFormat format = new
            StringFormat();
format.Alignment =
            StringAlignment.Center;
format.LineAlignment =
            StringAlignment.Center;
graphics.DrawString(buf, View.Font,
            brush, rectangle, format);
}

To hold the circles, we'll use an array list on the CircleView class. We'll need to use some classes from System.Threading, so add a namespace import declaration to the top of the code listing:

using System;
using System.Collections;
using System.ComponentModel;
using System.Drawing;
using System.Data;
using System.Windows.Forms;
using System.Threading;

Open the code editor for CircleView and add these members:

public class CircleView : System.Windows.Forms.UserControl
{
// members...
private ArrayList _circles = new ArrayList();
private ReaderWriterLock _lockCircles = new ReaderWriterLock();

To begin with, we only want to work with a single circle just to demonstrate the principle. Later we'll use the AdjustNumCircles method to increase and reduce the number of running circles. Add this method to CircleView:

// AdjustNumCircles - change the number of running circles...
public void AdjustNumCircles(int numRequired)
{
// lock...
_lockCircles.AcquireWriterLock(-1);

// create a circle...
Circle circle = new Circle(this);

// add it to the list...
_circles.Add(circle);

// release the lock...
_lockCircles.ReleaseWriterLock();
}

Don't worry about the AcquireWriterLock and ReleaseWriterLock calls for the moment. I'll explain those in a little while in the section on synchronization.

To paint the circles, every time the timerUpdateView timer fires we'll call Invalidate to redraw the circles. We'll then iterate through our array and ask each circle to repaint itself. Open the Designer for CircleView and double-click on the Timer to create a new Tick event handler. Add this code:

private void timerUpdateView_Tick(object sender, System.EventArgs e)
{
Invalidate();
}

To paint the control we'll need to build CircleView_Paint and wire in the event handler. First of all, add this method:

      private
            void CircleView_Paint(object sender, PaintEventArgs e)
{
//
            use anti-aliasing...
e.Graphics.SmoothingMode =
            System.Drawing.Drawing2D.SmoothingMode.AntiAlias;

//
            lock...
_lockCircles.AcquireReaderLock(-1);

// go through
            the circles...
foreach(Circle circle in _circles)
{
//
            paint...
circle.Paint(e.Graphics);
}

//
            unlock...
_lockCircles.ReleaseReaderLock();
}

To make the control look neat, we're going to use anti-aliasing. This smoothes down the edges of the circle so that the display looks more pleasing. You can learn more about anti-aliasing and other GDI+ features in this Cool GDI+ Features article.

To wire up this method, open out the Component Designer Generated code region and add the last line shown here to InitializeComponent:

      private
            void InitializeComponent()
{
this.components = new
            System.ComponentModel.Container();
this.timerUpdateView = new
            System.Windows.Forms.Timer(this.components);
//
//
            timerUpdateView
//
this.timerUpdateView.Enabled =
            true;
this.timerUpdateView.Tick +=
new
            System.EventHandler(this.timerUpdateView_Tick);
//
//
            CircleView
//
this.BackColor =
            System.Drawing.Color.Black;
this.Name =
            "CircleView";
this.Size = new System.Drawing.Size(192,
            136);
this.Paint += new
            PaintEventHandler(this.CircleView_Paint);
}

Build the project and open the Designer for Form1. Scroll to the bottom of the Toolbox to find the CircleView control.

Draw a new CircleView control onto the form:

Change the Name property of the control to circleView.

Double-click on the background of the form to create a new Load event handler. Add this code:

      private
            void Form1_Load(object sender, System.EventArgs e)
{
// update
            the caption...
this.Text += " - Primary Thread #" +
            System.Threading.Thread.CurrentThread.GetHashCode();

//
            adjust the number of
            circles...
AdjustNumCircles(1);
}

(Don't worry about the code to update the caption - we'll cover this later.)

Then, add this method:

      //
            AdjustNumCircles - adjust the circle view...
public void
            AdjustNumCircles(int numRequired)
{
// update the
            view...
circleView.AdjustNumCircles(numRequired);
}

Run the project. A circle should appear in the control:

Moving the Circle

Of course, this is only half the battle. We need to move the circle around the view using threads, or rather move the circle around the view using threads provided to us through the thread pool.

What we need to do is create a unit of work that the client has requested. This is synonymous with a browser connecting to a Web server. The circle makes a request; the request is answered with a response. In our case, the response will be the new location and direction (if any) of the circle. The new position and direction will be reported back by calling a public method on Circle. (The extra bit of work that we need to do involves getting the circle to tell us what thread was used to perform the move. This will let us build up an idea of how the thread pool is working under the hood, which is the point of this exercise.)

Create a new class called CircleMovementService. CircleView will ask this class to move each of the circles that it knows about every time the timerUpdateView timer fires. CircleMovementService will take the circle and create a new WorkItem object for that circle. WorkItem will then queue itself with the thread pool and wait until it is asked to run. When it's asked, it will look at the position and direction of the circle and pass the new position and direction back to the circle. Some time after this, CircleView will update the display and you'll be able to see the new position of the circle.

Create a new class called CircleMovementService. This class has a single static method whose job it is to create a new WorkItem object. If you were writing a Web server, this class would be the one responsible for listening for incoming network connections. It would then create a new request object (our "WorkItem " object) and ask that item to do some work.

Add this method:

      //
            RequestMovement - called by the circle when it needs to
            move...
public static void RequestMovement(Circle
            circle)
{
// create a work item...
WorkItem item = new
            WorkItem(circle);

//
            queue...
item.Enqueue();
}

Next, create a new class called WorkItem. This class needs to know about System.Drawing and System.Threading, so add these namespace import directives:

using System;
using System.Drawing;
using System.Threading;

Next, add this public member and new constructor to the class:

public class WorkItem
{
// members...
public Circle Circle;

public WorkItem(Circle circle)
{
// set...
this.Circle = circle;
}

To get the thread pool to give us a thread, we have to queue a work item. This is done using a static method on System.Threading.ThreadPool, as shown here:

      //
            Enqueue - add the item to the queue...
public void
            Enqueue()
{
// add it...
ThreadPool.QueueUserWorkItem(new
            WaitCallback(DoWork));
}

At some time in the future, when the thread has been created or allocated, DoWork will be called from the new thread. We can then do our work. This code is a little complex, but if you look carefully, all we're trying to do is examine the position and direction of the circle and provide a new one. We then pass these new values through to the Circle through the OnCircleMoved method, which we'll build in a moment:

// DoWork - called by the thread to do work...
private void DoWork(object state)
{
// store the details...
Point newLocation = new Point(Circle.Location.X, Circle.Location.Y);
Circle.CircleDirection newDirection = Circle.Direction;

// how big is the view?
Rectangle viewBounds = new Rectangle(0,0, Circle.View.Width,Circle.View.Height);
viewBounds.Inflate(0 - (Circle.Size.Width / 2), 0 - (Circle.Size.Width / 2));

// which way?
switch(newDirection)
{
case Thread_Pool_Demo.Circle.CircleDirection.DownRight:
newLocation.X += Circle.Speed;
newLocation.Y += Circle.Speed;
if(newLocation.Y >= viewBounds.Bottom)
newDirection = Thread_Pool_Demo.Circle.CircleDirection.UpRight;
if(newLocation.X >= viewBounds.Right)
newDirection = Thread_Pool_Demo.Circle.CircleDirection.DownLeft;
break;

case Thread_Pool_Demo.Circle.CircleDirection.UpRight:
newLocation.X += Circle.Speed;
newLocation.Y -= Circle.Speed;
if(newLocation.Y <= viewBounds.Top)
newDirection = Thread_Pool_Demo.Circle.CircleDirection.DownRight;
if(newLocation.X >= viewBounds.Right)
newDirection = Thread_Pool_Demo.Circle.CircleDirection.UpLeft;
break;

case Thread_Pool_Demo.Circle.CircleDirection.DownLeft:
newLocation.X -= Circle.Speed;
newLocation.Y += Circle.Speed;
if(newLocation.Y >= viewBounds.Bottom)
newDirection = Thread_Pool_Demo.Circle.CircleDirection.UpLeft;
if(newLocation.X <= viewBounds.Left)
newDirection = Thread_Pool_Demo.Circle.CircleDirection.DownRight;
break;

case Thread_Pool_Demo.Circle.CircleDirection.UpLeft:
newLocation.X -= Circle.Speed;
newLocation.Y -= Circle.Speed;
if(newLocation.Y <= viewBounds.Top)
newDirection = Thread_Pool_Demo.Circle.CircleDirection.DownLeft;
if(newLocation.X <= viewBounds.Left)
newDirection = Thread_Pool_Demo.Circle.CircleDirection.UpRight;
break;
}

// call the method...
Circle.OnCircleMoved(newLocation, newDirection);
}

Now, add this method to Circle:

// OnCircleMoved - called when the circle has been moved...
public void OnCircleMoved(Point newLocation, CircleDirection newDirection)
{
// set the location and direction...
Location = newLocation;
Direction = newDirection;
}

To queue the work item we need to revisit CircleView. Open the code editor for CircleView and make these changes to timerUpdateView_Tick:

private void timerUpdateView_Tick(object sender, System.EventArgs e)
{
Invalidate();

// suggest that the circle moves itself...
_lockCircles.AcquireReaderLock(-1);
foreach(Circle circle in _circles)
CircleMovementService.RequestMovement(circle);
_lockCircles.ReleaseReaderLock();
}

Run the project - the circle should now move around the view.

Reporting Threads

The purpose of this discussion is to demonstrate how the thread pool uses the threads. To show this, we need to add a ListView control to Form1.

Open the Designer for Form1. Select the CircleView control and set its Anchor property to Top, Left, Right.

Then, add a new ListView control to the form. Change its Name property to listviewThreads and change its Anchor property to Top, Bottom, Left, Right. Also, change its View property to Details.

Select the Columns property. Click the ellipsis button to open the editor and add three new columns. Change their Text properties to Thread ID, Circle ID, Last Used. Set the Width property to 100 for each.

You should have something like this:

Now, create a new class called ThreadListViewItem. Add a namespace import directive for System.Threading in the usual way.

Set the class to inherit from System.Windows.Forms.ListViewItem, like this:

public class ThreadListViewItem : System.Windows.Forms.ListViewItem
{

Next, add this public member and constructor:

      //
            members...
public Thread Thread;

public
            ThreadListViewItem(Thread thread)
{
// store the
            circle...
this.Thread = thread;

// add blank
            data...
this.Text = "Thread #" +
            thread.GetHashCode();
this.SubItems.Add("");
this.SubItems.Add("");
}

Now, add the Update method. We'll call this whenever we detect that the circle has been moved:

      //
            Update - update the thread...
public void Update(Circle
            circle)
{
// set the text...
SubItems[1].Text =
            circle.ToString();
DateTime updateTime =
            DateTime.Now;
SubItems[2].Text = updateTime.ToLongTimeString() +
            "." + updateTime.Millisecond.ToString();
}

When the circle is updated, ultimately we want to fire an event called CircleMoved that the form will be able to pick up.

Open the code editor for CircleView and add this delegate before the class definition:

namespace Thread_Pool_Demo
{
// delegate...
public delegate void CircleMovedHandler(object sender, CircleMovedEventArgs e);

public class CircleView : System.Windows.Forms.UserControl

Then, add the CircleMoved member:

public class CircleView : System.Windows.Forms.UserControl
{
// members...
private ArrayList _circles = new ArrayList();
private ReaderWriterLock _lockCircles = new ReaderWriterLock();
public CircleMovedHandler CircleMoved;

Next, add this method:

      //
            OnCircleMoved - move the circle...
public void
            OnCircleMoved(Circle circle)
{
// fire the
            event...
object[] args = new Object[2];
args[0] =
            this;
args[1] = new
            CircleMovedEventArgs(circle);
Invoke(CircleMoved,
            args);
}

Open the code editor for Circle and call OnCircleMoved after updating the location and direction of the circle:

// OnCircleMoved - called when the circle has been moved...
public void OnCircleMoved(Point newLocation, CircleDirection newDirection)
{
// set the location and direction...
Location = newLocation;
Direction = newDirection;

// tell the view...
View.OnCircleMoved(this);
}

While you're there, add this alternative version of ToString:

public override String ToString()
{
return "Circle #" + Id.ToString();
}

Open the code editor for Form1. Tie in the CircleMoved handler in the constructor for the form:

      public
            Form1()
{
//
// Required for Windows Form Designer
            support
//
InitializeComponent();

// add a
            handler...
circleView.CircleMoved += new
            CircleMovedHandler(circleView_CircleMoved);
}

Next, add this public member:

public class Form1 : System.Windows.Forms.Form
{
// members...
private System.Threading.ReaderWriterLock _lockThreadList = new System.Threading.ReaderWriterLock();

Then, add this method:

      public
            void circleView_CircleMoved(object sender, CircleMovedEventArgs
            e)
{
//
            lock...
_lockThreadList.AcquireWriterLock(-1);

// update
            the list...
Boolean foundThread =
            false;
foreach(ThreadListViewItem item in
            listviewThreads.Items)
{
//
            match?
if(item.Thread.GetHashCode() ==
            e.Thread.GetHashCode())
{
//
            update...
item.Update(e.Circle);
foundThread =
            true;
}
}

// do we need to add one?
if(foundThread
            == false)
{
// create and add...
ThreadListViewItem item =
new
            ThreadListViewItem(e.Thread);
listviewThreads.Items.Add(item);
item.Update(e.Circle);
}

//
            unlock...
_lockThreadList.ReleaseWriterLock ();
}

Run the project and the thread should be shown in the list:

We haven't paid much attention to keeping the circle exactly in the bounds of the control, so you may find that it "squashes" when it hits a wall!

When we want to know what thread we're running in, we can call the static CurrentThread property on System.Threading.Thread. This returns a System.Threading.Thread object. We can call GetHashCode on this object to return a unique ID of the thread. You can see that the caption bar is reporting a thread with ID 35. This is the ID of the primary application thread, or rather the thread that's drawing all of the controls on the form. In the list, we can see that thread 50 is drawing circle 0, so we know that the code to update the circle's position is indeed running in a separate thread.

The point of the thread pool is that we don't have to specifically spin up or spin down threads. We use QueueUserWorkItem and it will ask Windows to create a thread for us. This thread then calls DoWork, which moves the circle and, eventually, tells the form which thread moved which circle so that we can report the work on the screen.

Many Circles

To demonstrate some more functionality we need to make our view display many circles at once. This will show us how the threads are used when load on the system increases. (This is synonymous with many users connecting to a server at once.)

If the project is running, close it. Add a TrackBar control and a Label control to Form1. Change the Name property of the TrackBar control to trackbarNumClients and the Name property of the Label control to labelNumClients.

Also on the TrackBar control, change the Maximum property to 64 and the Value property to 1.

Double-click on the TrackBar control. Add this code to the new Scroll event handler:

private void trackbarNumClients_Scroll(object sender, System.EventArgs e)
{
AdjustNumCircles(trackbarNumClients.Value);
}

Find the AdjustNumCircles method on Form1 and add this code to update the label:

      //
            AdjustNumCircles - adjust the circle view...
public void
            AdjustNumCircles(int numRequired)
{
// update the
            view...
circleView.AdjustNumCircles(numRequired);
labelNumClients.Text
            = circleView.NumCircles;
}

Now, open the code editor for CircleView. Find the AdjustNumCircles method and make these changes. What we're trying to do is add new circles to the collection when the required number is increased, and remove circles when the required number is decreased. We also need to go through the entire list and update the IDs of the circles after we've made the adjustment.

      //
            AdjustNumCircles - change the number of running circles...
public
            void AdjustNumCircles(int numRequired)
{
if(_circles.Count ==
            numRequired)
return;

//
            lock...
_lockCircles.AcquireWriterLock(-1);

// create
            more?
if(numRequired > _circles.Count)
{
for(int n =
            _circles.Count; n < numRequired; n++)
{
// create a
            circle...
Circle circle = new Circle(this);

// add it to
            the list...
_circles.Add(circle);
}
}
else
{
//
            how many...
int numToRemove = _circles.Count -
            numRequired;
for(int n = 0; n < numToRemove; n++)
{
//
            remove the first one...
_circles.RemoveAt(0);
}
}

//
            update the ids...
int id = 0;
foreach(Circle circle in
            _circles)
{
circle.Id = id;
id++;
}

// release
            the lock...
_lockCircles.ReleaseWriterLock();
}

Now run the project and you should be able to create more clients. Move the slider up to eight and you should find that the work is be shared between two or three threads. (The number of threads used will vary depending on your computer's configuration.)

After a while, you'll find that the thread pool increases to eight threads. This is because Windows has detected that the load on the pool has reached a point where it's worth creating eight threads.

If you reduce the number of clients to 0 and then back to 1, you'll notice that the all of threads available in the pool are used.

Experiment with moving the slide around to see how the thread pool reacts. You may find on occasion that you have more threads than you think you should have. This is a "dead thread" that we still know about but has actually been spun down by Windows.

A Word About Synchronization

In this article we've pretty much ignored talking about synchronization, although I've had to take steps to ensure that the program won't crash when you run it!

Broadly speaking, synchronization problems occur when two threads try to access the same object simultaneously. Imagine that we have one thread removing items to a list and another thread reading items from the same list. If the second thread has an item that's about to be deleted from the list, it's likely some problem would ensue. What we need to do is control the access to the list. In this case, we make sure that when we're changing the list no one else can read from it. We also protect against anyone else making changes until we're finished. This is known as a "writer" lock.

Likewise, if we have two threads reading from the list, we want to prevent another thread from writing to it until both threads have finished. This is known as "reader" lock.

Throughout this article we've been using System.Threading.ReaderWriterLock objects to control access to our lists, as this is most likely to be the place where the most problems occur. When we want to lock it for reading, we call AcquireReaderLock. When we've finished reading, we call ReleaseReaderLock. We then use similar methods for writer locks. In this instance, we use a reader lock to prevent anyone else adding new circles or removing existing ones while we're painting. If a thread is changing the list when we come to paint, we'll wait until that thread has signaled that it has finished by calling ReleaseWriterLock.

      private
            void CircleView_Paint(object sender, PaintEventArgs e)
{
//
            use anti-aliasing...
e.Graphics.SmoothingMode =
            System.Drawing.Drawing2D.SmoothingMode.AntiAlias;

//
            lock...
_lockCircles.AcquireReaderLock(-1);

// go through
            the circles...
foreach(Circle circle in _circles)
{
//
            paint...
circle.Paint(e.Graphics);
}

//
            unlock...
_lockCircles.ReleaseReaderLock();
}

Summary

In this article we took a look at how to use the thread pool in .NET. We built a demo application that shows the typical model used in service applications - i.e. a client connects, a client requests some work be done and then the client disconnects.

C#Today - © 2002 WroxPress Limited


Writing a Multithreaded Port Scanner in C#

Writing a Multithreaded Port Scanner in C#
http://www.csharptoday.com/content.asp?id=1571
Article Rating: 4
Published on September 10, 2001
Abstract
The .NET Framework provides very comprehensive support for both multithreaded and sockets programming. In this article, Peter McMahon looks at these two aspects in detail as he creates a port scanner.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

The .NET Framework provides very comprehensive support for both multithreaded and sockets programming, so in an effort to teach myself some of these two aspects of the .NET Framework, I set about to create a port scanner. In this article, I will give you an introduction to multithreaded programming in C#, as well as very basic sockets programming, as I guide you through the process of creating a port scanner.

What is a port scanner?

In the 1980's, the hacker's (to be technically correct, I should say "cracker's") favorite tool for scanning a computer network for vulnerable entry points was a tool known as a "war dialer". This piece of software dialed a range of phone numbers, checking to see which ones were answered by a modem. The hacker would then use this information to try and break into the network. However, with the advent of the Internet and TCP/IP, the primary means of breaking into a network has evolved into finding out which TCP/IP ports are open on a network's server, thus generally indicating what type of software is running on it (such as web servers/FTP servers etc). Known vulnerabilities for this software are tested on the machine, according to which ports are open. Depending on the vulnerability, the attacker can be allowed to read any file on the server, create a buffer overflow (effectively shutting down that service on the machine), be allowed full administrative access, or any variation thereof. However, before any vulnerability can be exploited, it is essential that the attacker know which ports are open on the server. A port scanner is a piece of software that, given the address of a server, will return which ports are open on the server and which are closed. System administrators today often use port scanners to check their own servers, so that they can see their servers from an attacker's point of view, and lock them down accordingly, namely by closing unneeded ports and continually applying patches to the software running on the open ports.

Note: This article is not intended as a guide to server security, or to be a comprehensive guide to port scanning technologies, but rather an introduction to multithreaded programming in .NET using the example of a port scanner.

How does a port scanner work?

There are numerous ways in which a port scanner finds out which ports are open on a server. There is a link to Fyodor's page on this subject at the end of the article, which is a fairly comprehensive guide on the subject by the author of the renowned NMAP scanner. This article will cover the simplest method of port scanning, which is commonly known as the connect() method, due to the syntax for opening a TCP/IP connection in C++. Cracking is sometimes compared to breaking into a house, and in line with this metaphor, port scanning is checking which doors (and windows) are open. The connect() method is the most direct method of approach, and for the purposes of the house-breaking metaphor, is analogous to going up to each door of the house, turning the handle and pushing to see if the door will open. Basically it involves attempting to initiate a full TCP socket connection with all the ports that the scanner has been told to check. If a "couldn't connect" exception is not raised, then it knows that the port it is checking is open, but if an exception is raised, then it knows that the port is not open. Again I stress that this isn't the only, or necessarily the best way, to perform a port scan, but it is certainly the simplest, and for our purpose of demonstrating multithreaded programming in .NET, certainly quite adequate.

Foundations for the port scanner

To begin with, we'll create a skeleton for our port scanner that implements the bare minimum to scan a range of ports on a specified machine. The program will run as a console application, although it would be a trivial affair to port it to a WinForms application because of the continuity of the .NET programming model.

Firstly, our application must accept three arguments from the command line: machine to scan (either IP address, or domain name); port to start scans from; and port to end scans on. If anything other than three arguments are given, a usage message must be displayed informing the user of the expected arguments. Secondly, the port number arguments must be converted to integers. Finally, a loop must then be made, going from the starting port to the end port, with each iteration attempting a connection to the port and writing the result to the console. The code would look something like this:

using System;
using
            System.Net.Sockets;

public class MainApp {
public static
            void Main(string[] args) {
Console.WriteLine("portscanner
            v0.1");
Console.WriteLine();
if (args.Length == 3) {
try
            {
int start = Convert.ToInt32(args[1]);
int end =
            Convert.ToInt32(args[2]);
for (int i=start; i<=end; i++)
            {
TcpClient objTCP = new TcpClient();
try
            {
objTCP.Connect(args[0], i);
Console.WriteLine("Port " +
            i.ToString() + " is
            open!");
objTCP.Close();
}
catch(SocketException e)
            {
Console.WriteLine("Port " + i.ToString() + " is not
            open!");
}
}
}
catch (FormatException e)
            {
Console.WriteLine("start/end scan range must be entered
            as
integers");
}
} else {
Console.WriteLine("Usage:
            portscanner host startport
            endport");
Console.WriteLine();
Console.WriteLine("host -
            IP/hostname of machine to scan");
Console.WriteLine("startport -
            port number to start scan from
[0-65536]");
Console.WriteLine("endport - port number to end
            scan on
[0-65536]");
}
}
};

There is a static entry point "Main" for the application that includes an array of strings where the arguments from the command line can be passed. The length of the arguments array is checked to ensure that three arguments are passed, otherwise a usage message is displayed. If the program proceeds past this check, the program attempts to convert the two port arguments to integers. If non-integer values are entered in the command line, a FormatException exception will be raised, and caught by the structured error handling in place, which will then inform the user that they must enter the port scan range as integer values. This is all standard C# code for a skeleton console application that accepts arguments. The following lines are where the port scanning actually takes place. There is a "for" loop iterating from the indicated starting port to the end port, inclusive. For each iteration of the loop, a TcpClient object is created, and then there is an attempt to open a TCP connection to the host address argument on the port that the loop's counter variable contains. If the connect method fails (because a connection could not be established, and therefore it can be assumed that the port is not open), a SocketException exception is raised. If this happens, then the error handling code of the try..catch block informs the user that the port could not be opened. However, if a connection was established, then the port must have been open. Since we're not actually going to send or receive any data to or from the port, as we only want to know if it opens, we close it immediately, and thereafter inform the user that we were able to successfully open that port.

To test out the code for yourself, save the above code as portscanner.cs and compile it using the C# command-line compiler, csc.exe . No switches are required. Run the newly formed executable using the following line:

C:\>portscanner 127.0.0.1 20 25

Depending on what machine you're running it on, the results will look similar to the following. If you have a default IIS installation, then FTP (port 21) and SMTP (port 25) should both be running.

C:\>portscanner 127.0.0.1 20 25
portscanner v0.1

Port 20 is not open!
Port 21 is open!
Port 22 is not open!
Port 23 is not open!
Port 24 is not open!
Port 25 is open!

C:\>

As you can see, the program checks the ports in a linear fashion. It checks port 20, outputs the result, then moves onto checking port 21, outputs the result, and so forth. The problem with this approach is that it's incredibly slow. The reason for this is that it has to wait for the current check to complete before it can attempt the next connect. Even when testing the port scanner on your local machine, it takes about three quarters of a second per port to check ports that don't open. When scanning a remote machine, the lag can be much greater, depending on connection speed. Imagine scanning a remote machine for ports 1 to 200 when each check has a delay of 2 seconds? That's more the 6 minutes! Considering that it would be advisable to check from ports 1 to 1024 (the port range generally used by system services), the scan would take a ludicrously long amount of time to complete.

Enter Multithreading

The kind of problem created by the linear port scanning approach is one that multithreading is particularly adept at solving. The area in which multithreaded programming makes a lot of sense is in scenarios where there are many operations that have to take place, not in any particular order, that involve a lot of waiting and aren't extremely resource-hungry (i.e. they don't use up 100% of the CPU when they're executing). A good example of this is the thread that Microsoft Word uses to run in the background while you're using Word to check for spelling errors. There'll be another thread that checks for grammatical errors, and so forth. Most of the time these threads are waiting for you to type something (in comparison to the time it takes them to actually perform their respective checks on the words that you've typed, they actually do spent most of their time dormant, even though you may be typing at 100 words per minute).

In the case of our port scanner, most of the time in the port-checking operation is spent waiting for the port to reply. Windows allows multiple ports to be opened at the same time (otherwise you wouldn't be able to browse the web and check your email at the same time), so we'll try to get close to a scenario where connections to all the ports in the specified range are attempted, then wait until all the ports have responded (either with a SocketException , or with an actual connection). As seasoned programmers with experience in threading will tell you, there's a vast difference between theory (read: what you'd think should logically happen) and practice. This is because of the nuances of the Windows operating system and how it manages threads. Nonetheless, as is definitely the trend with the .NET Framework, Microsoft has made it fairly easy to do multithreaded programming, once a fine and complex art that only guru C++ developers could practice, although it is still important to understand that threading is by no means a basic topic, and that if anyone ever writes a book "Multithreaded programming made easy", the title definitely isn't telling the whole truth.

The main problem with our port scanner is that we don't want to create only one or two threads. In fact, we're probably going to want to create more than 100 threads. This is because every attempt at opening a port is going to be a new thread, so if the user wants to check from port 1 to port 1024, 1024 threads will be created. In normal circumstances, creating so many System.Threading.Thread objects would cause the CPU utilization to rocket to 100%, and the memory usage would be outrageous, due to so many threads being created and running simultaneously. In addition, due to nuances in the Windows thread management system, some threads may not execute at all, and other such non-compliant behavior may be observed. However, Microsoft has provided a perfect solution for our scenario in the System.Threading.ThreadPool class. This class provides functionality specifically for the scenario where threads will be created that spend most of their time waiting. The class manages all the aspects of creating threads for you, and uses the machine running the program's specifications as a basis for how many threads it will allow to run simultaneously, so as to avoid starving the CPU and RAM by creating too many threads at once. The ThreadPool class provides a static, or shared, method " QueueUserWorkItem " with the following definition:

public static extern bool QueueUserWorkItem(
WaitCallback callback,
object state
);

The WaitCallback object that is passed is actually a delegate. Delegates, if you haven't come across them before, can be thought of as .NET's implementation of function pointers, and basically allow methods to be represented as "variables". They're not actually variables, although in some ways they do act like them, such as the fact that they can be passed into other methods as parameters, as is done with the QueueUserWorkItem method. In this case, the WaitCallback delegate points to the method that must be called when the thread created by the QueueUserWorkItem is created. The object that is passed is an optional parameter that allows you to pass an object (which can be any class, since everything is derived from System.Object in .NET) to the method specified in the WaitCallback delegate. This is actually very useful, as this value can be used to alter the functionality of the method that the delegate is pointing to, which it will do in our case. The calling of the QueueUserWorkItem method effectively results in the creation of a new thread when a space in the thread pool becomes available.

When using threads, the method that is called to be executed by the thread is usually contained in a class other than the one calling the thread, as it can be useful for keeping count of threads in existence, amongst other reasons. Therefore, we will separate our actually scanning from the main application. Here is a method in its own class that will attempt a connection to the specified to port and write to the console if it managed to successfully open the port.

public class PortScan {
public string
            Addr;

public void Scan(object portArg) {
int port =
            (int)portArg;
TcpClient objTCP = new TcpClient();
try
            {
objTCP.Connect(Addr, port);
Console.WriteLine("Port " +
            port.ToString() + " is
            open!");
objTCP.Close();
}
catch(SocketException e)
            {
}
}
};

The only area that requires comment is the fact that the port to scan, which is passed as a parameter, is passed as an object rather than an integer. The integer 'port' variable is set as the passed object, typecast as an integer. Notice too, that the TcpClient object that we have created tries to open a connection to Addr , a string, although this has not yet been set. We don't need to pass it as a parameter to the Scan() method since the address we are scanning remains constant, and it will be set after the PortScan class is created by the Main() method.

The following is a revision of the MainApp class, that now adds Worker Items to the machine's process's thread pool that do the actual port scanning.

public class MainApp {
public static void
            Main(string[] args) {
Console.WriteLine("portscanner
            v0.1");
Console.WriteLine();
if (args.Length == 3) {
try
            {
int start = Convert.ToInt32(args[1]);
int end =
            Convert.ToInt32(args[2]);
if ((start >= 0 && start
            <= 65536) && (end >= start))
            {
Console.WriteLine("Starting scan... (this may take a
            few
minutes)");
PortScan objPortScan = new
            PortScan();
objPortScan.Addr = args[0];
for (int i=start;
            i<=end; i++) {
try
            {
ThreadPool.QueueUserWorkItem(new
WaitCallback(objPortScan.Scan),
            i);
}
catch (Exception e)
            {
}
}
Console.WriteLine("Scan complete.");
} else
            {
Console.WriteLine("start/end scan range is out of range
[0-65536]");
}
}
catch (FormatException e)
            {
Console.WriteLine("start/end scan range must be entered
            as
integers");
}
} else {
Console.WriteLine("Usage:
            portscanner host startport
            endport");
Console.WriteLine();
Console.WriteLine("host -
            IP/hostname of machine to scan");
Console.WriteLine("startport -
            port number to start scan from
[0-65536]");
Console.WriteLine("endport - port number to end
            scan on
[0-65536]");
}
}
};

The lines where dramatic change has occurred are these:

Console.WriteLine("Starting scan... (this
            may take a few minutes)");
PortScan objPortScan = new
            PortScan();
objPortScan.Addr = args[0];
for (int i=start;
            i<=end; i++) {
try {
ThreadPool.QueueUserWorkItem(new
            WaitCallback(objPortScan.Scan), i);
}
catch (Exception e)
            {
}
}
Console.WriteLine("Scan complete.");

Firstly, an instance of the PortScan class is created. The Addr member is then set to reflect the address to scan that was passed as a command-line argument. The loop is identical to before, looping from the start port to the end port, inclusively. However, instead of performing the actual scanning, it queues a new thread to the thread pool, which will execute it when a thread becomes available, thus avoiding resource starvation. The WaitCallback delegate is calling the PortScan class's Scan method, and the counter variable 'i' is going to be passed into it as a parameter.

Another change that has been made to the class is the inclusion of a check to ensure that a valid range of ports has been entered.

This code is fine, except for one minor detail, which is that it doesn't wait until all the threads have finished executing. In other words, if you compiled and ran it in its present form, no ports would be checked, as the program would have exited before any of the threads had a chance to execute, seemingly immediately after outputting "Starting scan.".

Waiting until all the threads have completed is a task with two main aspects to it. Firstly, we need to keep track of how many ports we've finished scanning, and secondly, we need to make the program wait until the number of ports we've finished scanning is the same as the total number it was supposed to scan. Microsoft again comes to the rescue here, with the ManualResetEvent class that provides just the functionality we need for the latter. The ManualResetEvent class has two methods pertinent to our problem, Set() and WaitOne() . The Set() method will be called by our program when all the ports have been checked, and the WaitOne() method will force the program to wait for a specified time period for the Set() method to be called.

With the waiting part of the problem solved, we just need to keep count of the number of ports we've scanned. This may sound trivial, but again, it's not quite as simple as it seems because we're dealing with multiple threads, which means unpredictable results because a method may be called by two threads at exactly the same time. Again, Microsoft provides a perfect solution to the problem in the Interlocked class's static ' Increment ' method. Its definition is as follows:

public static int Increment(
ref int location
);

Basically it accepts an integer variable by reference and increments it by 1, but it does it in such a way that prevents multithreaded applications from scenarios where the variable may be incremented twice if incremented normally (i.e. int++;), without using the Interlocked class.

With both these problems solved, here is the final source code listing for the program:

using System;
using
            System.Net.Sockets;
using System.Threading;

public class
            PortScan {
public int count = 0;
public int maxCount =
            0;
public ManualResetEvent eventDone = new
            ManualResetEvent(false);
public string Addr;

public void
            Scan(object portArg) {
int port =
            (int)portArg;
Interlocked.Increment(ref count);
TcpClient
            objTCP = new TcpClient();
try {
objTCP.Connect(Addr,
            port);
Console.WriteLine("Port " + port.ToString() + " is
            open!");
objTCP.Close();
}
catch(SocketException e) {
if
            (e.ErrorCode == 11001) {
Console.WriteLine("The address to scan
            could not be
            found.
Exiting...");
eventDone.Set();
}
}
finally
            {
if (count == maxCount)
            {
eventDone.Set();
}
}
}
};

public class
            MainApp {
public static void Main(string[] args)
            {
Console.WriteLine("portscanner
            v0.1");
Console.WriteLine();
if (args.Length == 3) {
try
            {
int start = Convert.ToInt32(args[1]);
int end =
            Convert.ToInt32(args[2]);
if ((start >= 0 && start
            <= 65536) &&
(end >= 0 && end <=
            65536)) {
Console.WriteLine("Starting scan... (this may take a
            few
minutes)");
PortScan objPortScan = new
            PortScan();
objPortScan.maxCount = end - start +
            1;
objPortScan.Addr = args[0];
for (int i=start; i<=end;
            i++) {
try
            {
ThreadPool.QueueUserWorkItem(new
WaitCallback(objPortScan.Scan),
            i);
}
catch (Exception e)
            {
}
}
objPortScan.eventDone.WaitOne(Timeout.Infinite,true);
Console.WriteLine("Scan
            complete.");
} else {
Console.WriteLine("start/end scan range
            is out of range
[0-65536]");
}
}
catch (FormatException
            e) {
Console.WriteLine("start/end scan range must be entered
            as
integers");
}
} else {
Console.WriteLine("Usage:
            portscanner host startport
            endport");
Console.WriteLine();
Console.WriteLine("host -
            IP/hostname of machine to scan");
Console.WriteLine("startport -
            port number to start scan
            from
[0-65536]");
Console.WriteLine("endport - port number to
            end scan on
[0-65536]");
}
}
};

Note that when the WaitOne method is called, Timeout.Infinite is passed as the TimeSpan , so that the program will wait indefinitely for the threads to complete execution. Also note that if the TcpClient.Connect() method raises an exception that indicates that the specified address could not be found (error code 11001), the scan is aborted, since there's not point in continuing the scan if the destination host can not be reached.

If you now compile the program, you will notice that the scan time is significantly less than when a linear scanning method was used.

Conclusion

This article has demonstrated some of the more powerful features of Microsoft's threading support in the .NET framework through an interesting and pertinent example, in which multithreading provides a very noticeable performance advantage. However, there is still much more to the System.Threading namespace than has been covered here, but hopefully this article has proved to be a decent introduction to some of the threading capabilities of the .NET framework, and will inspire you to create more efficient and powerful applications using the .NET framework and its threading capabilities.

Footnote: It would not be fair for me to close without mentioning the help that Willy Denoyette has given to me in implementing the ThreadPool and ManualResetEvent classes in my prototype of the application - Thanks Willy!

C#Today - © 2002 WroxPress Limited


WordPad in C#

WordPad in C#
http://www.csharptoday.com/content.asp?id=1567
Article Rating: 3.5
Published on August 31, 2001
Abstract
In this article, Niranjan Kumar develops a Word Pad application in C# using the RichTextBox control. This control allows us to manipulate rich text format documents and normal text documents as well. Here we will see the various features of the RichTextBox control, and how the .NET framework makes it easy to work with this control. Clipboard manipulation, Printing documents, implementing print preview and usage of various common dialogs is also illustrated in detail.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

Word Pad is a Windows application that ships along with Windows OS. It is used in editing text documents and rich text documents. .NET provides a special class called RichTextBox with which we can do RTF manipulation.

In this article we will learn about the RichTextBox control, implementing print preview and usage of common dialogs in C#. We will develop a Word Pad application in C# using the RichTextBox control. This control allows us to manipulate rich text format documents and normal text documents as well. Here we will see the various features of the RichTextBox control, and how the .NET framework makes it easy to work with this control. Clipboard manipulation, Printing documents, implementing print preview and usage of various common dialogs is also illustrated in detail.

Rich Text Format (RTF)

Microsoft designed rich text format for interchanging documents between Microsoft Word and other word processing applications. Unlike Word documents, RTF documents are plain text with predefined tags and formats as defined by its specification. Although developed by Microsoft, RTF is supported by many applications like WordPerfect, FrameMaker, Interleaf and on different OS like Unix, Apple, Macintosh etc. More information about RTF and its specification can be found at http://msdn.microsoft.com/library/?url=/library/en-us/dnrtfspec/html/rtfspec.asp?frame=true&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq.

The RichTextBox control is a special kind of text control that comes with the .NET framework. Unlike the TextBox control, it allows you to work with RTF documents, change font for a particular portion of text, allows formatting of text, change the color of the text, allows paragraph formatting, provides clipboard functionality and more. It allows you to save the data as an RTF file or a text document. You can place bulletins, display web style links, print documents and lot more.

Designing our Word Pad

Lets jump into the code and learn more about the RichTextBox control. Our only system requirement is Visual Studio.NET Beta 2. Start the VS.NET IDE and click File->New->Project. From the Project types select Visual C# Project and from templates option select Windows Application, as seen below.

Give the project the name WordPad and click OK. The project will be created with default Form named Form1 (Form1.cs). Now we need to design our Word Pad application (Wordpad.exe) as shown below.

Now we need to design our Form.

Designing our Main window

On Form1's Properties window, set Name to MainWindow, WindowState to Maximized and Text to WordPad.

Creating child controls

The controls that we will have in our form are RichTextBox, ToolBar and ImageList. ImageList will have images that will be associated with the ToolBar control. Actions will be invoked via the ToolBar control.

Go to the ToolBox window (View->ToolBox menu option) to display all available controls in .NET and select Windows Form tab from ToolBox. Drag and drop the RichTextBox control from the toolbox into our form.

Set the following properties to the control - Name to m_richTextBox and AcceptsTab to True. This enables us to enter tabs inside the control.

Add an ImageList control from the toolbox window.

Set the Name property to m_imageListToolBar. Add images to ImageList by clicking the Images ellipsis (.) button from the properties window. Add the images (found in the code download) in the sequence as shown in wordpad.exe above.

Note that you need to create Buttons with a Style of PushButtons and ImageIndex to a valid number. To separate two buttons, you need to create an empty toolbar button with Style set to Separator.

Next we need the ToolBar control to activate operations like Opening file, saving file, printing document and clipboard operations etc. Drag and drop a ToolBar control from the toolbox to the form.

Set Name to m_ToolBar and ImageList to m_imageListToolBar. We then need to add buttons for our various actions. Select the Buttons property and click the ellipsis (.) button from the right side of the property window. This will launch ToolBarButton Collection Editor window we see above.

In this editor, click the Add option to add a button in the toolbar control. You can associate a button with an image using the ImageIndex property. The ToolTipText property defines the tool tip for the toolbar button. The Style property specifies the style of a Toolbar button. The default value is PushButton. The Separator value will display a gap between two toolbar buttons. (Refer to the WordPad.exe screenshot above for separator between the Save as and Print options). Add the options New, Open, Save, Save As, Print, Print Preview, Undo, Redo, Cut, Copy, Paste, Select All, Delete, Font, Color, About and Exit to the toolbar control.

In order to open files and save to hard disk, we need file open/save system dialogs. .NET provides controls for these manipulations. Drag and drop the OpenFileDialog control from the toolbox window.

In the Properties window, set Name to m_openFileDialog, Filter to "Text documents|*.txt|RTFdocument|*.rtf", and Title to "Open Document".

The Filter property defines the types of files and their extensions that will be listed in the "Files of type" drop down option in the Open file dialog. The string "Text documents" will be displayed in the combo box. On selecting this option *.txt will be applied to file list box. In our application we will be dealing with normal text files (.txt) and Rich text files (.rtf).

To save data back to hard disk, we need to know the name of file and path. For this we need the SaveFileDialog box. Drag and drop the control and set the following properties - Name to m_saveFileDialog, FileName to Untitled, Filter to "Text Documents|*.txt|TextFormat|*.rtf" and Title to "Save Document"

Unlike normal text fields, RichTextBox allows us change the font and style settings of selected portion of your document. Here we will also allow the user to change font and style settings to all the text or to selected text at runtime. In order to select font and style settings, we include another control called FontDialog. Drag and drop a FontDialog object from toolbox and use m_fontDialog for its Name property.

To select system color palette, we need to include the control ColorDialog from the toolbox. Create a control and give it a Name of m_colorDialog.

We will also incorporate print features in our program. So create an object of type PrintDocument from the toolbox.

Getting into the Code

Now we have completed the design of our Word Pad window, let's move to the code where we will see how to make our application perform its actions.

The first and foremost thing that we need to do is set the size of the RichTextBox control to occupy the entire area of our window except for the ToolBar area. Whenever the window is resized, the size of m_richTextBox needs to be altered and adjusted in such a way that it fits exactly inside the window. When the window is resized, the Resize event of the window will be called. We will add a handler to this event and adjust the size of the m_richTextBox object accordingly. Go to the Event handler window and add the OnResizeMainWindow to our form.

The next step is to write code that resizes the child control m_richTextBox whenever the window is resized. In the following code fragment the this keyword represents the window itself, whereas Width and Height are properties and it are assigned to the corresponding counterparts of the RichTextBox object. The value of 15 and 60 are used in order to make the size of RichTextBox control smaller than the parent window's size. Width is reduced by 15 pixels so that the control width is not bigger than window width. Height is reduced by 60 pixels because the RichTextBox should be displayed below the toolbar.

m_richTextBox.Width = this.Width -
            15;
m_richTextBox.Height = this.Height - 60;

Now we should activate the menu items. When a toolbar button is clicked the ButtonClick event of the toolbar control will be called. So add an event handler OnButtonClickToolBar() to the ButtonClick event of toolbar control.

After adding the handler, go to the definition of OnButtonClickToolBar(). The first parameter is of type object, which represents the object that invoked this event. In this case it is the toolbar itself. The second parameter is the ToolBarButtonClickEventArgs object and contains the information about the event on which the click has occurred. The Button property of this object will return a Button object. The ImageIndex property of the Button object will give the image index of that button. Remember that our ToolBar control is associated with an ImageList object. So by using the property Button.ImageIndex we can know the option on which the click has occurred. See the following code fragment of the OnButtonClickToolBar() function.

int l_iImageIndex = e.Button.ImageIndex;
switch (l_iImageIndex){
case 0 :
/* New option is clicked */
break;
case 1 :
/* Open option is clicked */

The l_iImageIndex variable will contain the index of the image associated with the toolbar button. This image index is given during the design of the toolbar in the ToolBarButton Collection Editor window. If toolbar is not associated with Images then the Text property can be set to unique values (while designing the Tool bar) and it can used to check to determine if the button is clicked.

File handling

When the New button is clicked, we need to initialize all member variables and show an empty RichTextBox control to allow the user to enter text. This is quite simple and straightforward. The complexity comes when the Open option is selected. In this case, we need to show the file open system dialog box. After selecting a file and clicking the open button, the file has to be read and shown to the user in the RichTextBox control. The FileOpen() function does this and it is called in the second switch-case statement in the OnButtonClickToolBar(). See the following code.

private void FileOpen(){
string
            l_strFileName = "";
string l_strTemp = "";

/* If the
            document is not saved, prompt the user to save it */
if (
            m_bSaveFlag == false ){
if ( MessageBox.Show("Do you wish to save
            the current document ?","Word Pad",
MessageBoxButtons.YesNo,
            MessageBoxIcon.Information) ==
            DialogResult.Yes){
FileSave();
}
}

if
            (m_openFileDialog.ShowDialog() == DialogResult.OK )
            {
l_strFileName = m_openFileDialog.FileName;
m_strDocumentName
            = l_strFileName;
}
else {
return;
}

/* Read the
            file */
FileStream l_fStream = null;
string l_strFileContents
            = "";
try {
l_strTemp = "";
int l_iRetval = 1;
l_fStream
            = new FileStream (l_strFileName,
            FileMode.Open,FileAccess.Read);
/* Read 1k at a time */
Byte[]
            l_Byte = new Byte[1024];
l_Byte.Initialize();
for ( ;
            l_iRetval != 0 ; ) {
l_Byte.Initialize();
l_strTemp =
            "";

/* Read from file */
l_iRetval =
            l_fStream.Read(l_Byte,0,1024);

/* Convert Byte array into
            string */
l_strTemp =
            Encoding.ASCII.GetString(l_Byte,0,1024);
l_strFileContents +=
            l_strTemp;
}
}
catch( FileNotFoundException e1) {
string
            l_strMessage = "File not found or permission
            denied";
MessageBox.Show(l_strMessage,"Word Pad",
            MessageBoxButtons.OK,MessageBoxIcon.Error);
return;
}
catch
            (Exception e2) {
string l_strError = "Error :" +
            e2;
MessageBox.Show(l_strError,"Word Pad",
            MessageBoxButtons.OK,MessageBoxIcon.Error);
return
            ;
}
finally {
if ( l_fStream != null )
            {
l_fStream.Close();
}
}

l_strTemp =
            l_strFileName.ToLower();
if ( l_strTemp.EndsWith(".rtf") )
            {
m_richTextBox.Rtf = l_strFileContents;
}
else
            {
m_richTextBox.Text = l_strFileContents;
}
this.Text =
            "Word Pad " +
            l_strFileName;
m_richTextBox.Select(0,0);
m_bSaveFlag =
            false;
}

The boolean variable m_bSaveFlag denotes whether the file is saved or not. If the opened file is not saved, m_bSaveFlag will contain false. In this case, the user is prompted to save the file. When the user chooses to save the file, the FileSave() method is called. We will shortly visit the FileSave() method.

The m_openFileDialog object's ShowDialog() method is called to display the system file open dialog window. After selecting a .rtf or .txt file, and clicking the open button from the dialog, the value of DialogResult.OK will be returned. In this case the file names are retrieved using the FileName property. The FileStream class is used to read the input file. Its constructor takes the name of the file, its mode of opening and its access type and creates the object. When the file is not found a FileNotFoundException is raised. Inside the for loop the file is read as a Byte array and converted to a string using the Encoding class. The Encoding.ASCII.GetString() method takes the Byte array object and its size as arguments and returns a string object. To use this class we need to include the System.Text namespace. The file data is concatenated into a string object and finally, depending upon the extension of the file, the contents are assigned to the m_richTextBox object. The rtf property of RichTextBox represents the string in Rich text format, while the Text property represents normal text. RichTextBox handles both Rich text strings and normal strings. The finally block closes the opened file.

Our next step is to save the edited document or a new document. See the FileSave() method which will be called when the user clicks Save. The code fragment below opens the File Save system dialog box and returns DialogResult.OK when the user clicks Save.

if ( m_saveFileDialog.ShowDialog() ==
            DialogResult.OK ) {
l_strFileName =
            m_saveFileDialog.FileName;
}
else
            {
return;
}
m_strDocumentName = l_strFileName;

Here we get the filename and call SaveDocument() to saves the contents of the m_richTextBox object to the file. See the following code fragment taken from the SaveDocument() function. Depending upon the extension of file (.rtf or .txt) the string is extracted accordingly using a Rtf or Text property and written to file.

/* Write to file */
FileStream l_fStream
            = null;
try {
l_fStream = new FileStream (l_strFileName,
            FileMode.Create, FileAccess.Write);
Byte[] l_Byte;
if (l_iType
            == 1 ) {
l_Byte = new
            Byte[m_richTextBox.Text.Length];
l_Byte.Initialize();
l_Byte =
            Encoding.ASCII.GetBytes
            (m_richTextBox.Text.ToString());
}
else {
l_Byte = new
            Byte[m_richTextBox.Rtf.Length];
l_Byte.Initialize();
l_Byte =
            Encoding.ASCII.GetBytes
            (m_richTextBox.Rtf.ToString());
}
l_fStream.Write(l_Byte,0,l_Byte.Length);
}
catch
            (Exception e2) {
/* Error occurred */
string l_strError =
            "Error :" + e2;
MessageBox.Show(l_strError,"Word
            Pad",MessageBoxButtons.OK, MessageBoxIcon.Error);
return
            ;
}
finally {
/* Close the file */
if ( l_fStream !=
            null ) {
l_fStream.Close();
}
}

The file is opened in create mode and the contents of the m_richTextBox is copied to a Byte array using the Encoding.ASCII.GetBytes() method. Remember that the Encoding class is available in the System.Text namespace. Then using the Write() method of the FileStream class, the contents of the Byte array are written to file. The file is closed in the finally block.

Printing and Print Preview

Now we have completed the file-handling portion of our application. The next important area of our application is activating the Print and Print Preview options. .NET provides a class called PrintDocument which handles printing activities. First we need to create an instance of this class and call the Print() method. This will fire the PrintPage event in which we need to write code that actually writes our data to the printer. See the Print Preview window of our application below:

We will first invoke the Print system dialog using the PrintDialog class and will check for the return value of the ShowDialog() method. If the return value is DialogResult.OK, we will call the Print() method of the m_printDocument object. Remember that m_printDocument is of a PrintDocument class that is responsible for printing documents. Before calling Print(), the contents of the rich text control is stored in a StringReader object which is later used in the PrintPage event. See the code fragment below taken from FilePrint() method.

PrintDialog l_pdDialog = new
            PrintDialog();
l_pdDialog.Document = m_printDocument;
if (
            l_pdDialog.ShowDialog() == DialogResult.OK){
this.Cursor =
            Cursors.WaitCursor;
m_iStratPos = 0;
/* Read the contents of
            the RichTextBox into StringReader */
string l_strText =
            m_richTextBox.Text;
m_strReaderObj = new
            StringReader(l_strText);
/* Call Print method
            */
m_printDocument.Print();
this.Cursor =
            Cursors.Arrow;
}

Now we need to add a handler for the PrintPage event in the m_printDocument object. This function will be called for every page by the .NET framework. PrintPage will be fired automatically till the HasMorePages property of PrintPageEventArgs is set to false. See the last if condition in the following code where the value of the string l_strLine is checked. If it is null it means that the string is empty, and the e.HasMorePages is set to false. PrintPageEventArgs is the second argument to the event handler OnPrintPageDocument(). The following code prints the document to a printer that is selected from the PrintDialog object.

private void OnPrintPageDocument(object
            sender, System.Drawing.Printing.PrintPageEventArgs e) {
float
            l_fLinesPerPage = 0;
float l_fYPosition = 0;
int l_iRowCount =
            0;
float l_fFontHeight = 0;
string l_strLine =
            null;

float l_fLeftMargin = e.MarginBounds.Left;
float
            l_fTopMargin = e.MarginBounds.Top;

Font l_Font =
            m_richTextBox.Font;
l_fFontHeight =
            l_Font.GetHeight(e.Graphics);
/* Create the brush object
            */
SolidBrush l_BrushObj = new SolidBrush(Color.Black);

/*
            Calculate the Lines per page */
l_fLinesPerPage =
            e.MarginBounds.Height / l_fFontHeight;

/* loop through the
            string using StringReader object and print each line */
for ( ;
            l_iRowCount < l_fLinesPerPage && (l_strLine =
            m_strReaderObj.ReadLine()) != null ; m_iStartPos++ )
            {

l_fLeftMargin = e.MarginBounds.Left;

/* Calculate
            the position to print the characters */
l_fYPosition =
            l_fTopMargin + ( l_iRowCount * l_fFontHeight);

int l_iLen =
            l_strLine.Length;
int j = 0;
for ( ; j < l_iLen ;
            m_iStartPos++,j++){
/* Select one character and find its font
            settings */
m_richTextBox.Select(m_iStartPos,1);
try {
Font
            l_Fontprint = m_richTextBox.SelectionFont;
/* Get the maximum
            height */
if ( l_fFontHeight < (l_Fontprint.GetHeight() / 2)
            ){
l_fFontHeight = l_Fontprint.GetHeight() / 2;
}

/*
            Extract the character */
string l_strTemp =
            l_strLine[j].ToString();
/* Print the character
            */
e.Graphics.DrawString(l_strTemp, l_Fontprint, l_BrushObj,
            l_fLeftMargin, l_fYPosition, new StringFormat());

/*
            Increment X position to print next character */
l_fLeftMargin +=
            l_Fontprint.SizeInPoints;
}
catch ( Exception e2){
/* If
            error in printing text */
string l_strError = "Error while
            printing :" + e2.Message;
MessageBox.Show(l_strError,"Word Pad",
            MessageBoxButtons.OK,MessageBoxIcon.Error);
e.HasMorePages =
            false;
l_BrushObj.Dispose();
return;
}
}
l_iRowCount++;
}

/*
            If there are lines to be processed, set HasMorePages to true
            */
if( l_strLine != null){
e.HasMorePages = true;
}
else
            {
e.HasMorePages =
            false;
}

l_BrushObj.Dispose();
}

The second argument PrintPageEventArgs contains information about the printer and its settings. The first step is to create a brush with which we can draw text or paint to the device. SolidBrush is the class used with which we create a brush with black color as specified in its constructor Color.Back. Depending upon font settings the height (l_fYPosition) of every line and the number of lines it can print in one page (l_fLinesPerPage) are calculated. The DrawString() method will actually write the text data into selected device, in this case a printer. It takes the string value, Font object, the created brush object, the position to write and a new StringFomat object and produces the output.

First we read a line from m_strReaderObj (see outer for loop) and store it in l_strLine. Later from this string each character is extracted and its font settings are determined (see inner for loop). Font settings are determined in two steps. First we select every character using the Select() method and then retrieve the character's font using the SelectionFont property. Later this character is printed using DrawString() with the same font object. After printing this character, the l_fLeftMargin (X position) is incremented so that the next character will not be printed in the same X position.

When the file is read completely or when the number of lines printed in one page reaches the maximum value (l_fLinesPerPage), the for(;;) is terminated. The HasMorePages property of PrintPageEventArgs is set to true or false depending upon the length of text. Finally Dispose() of SolidBrush is called to destroy our brush object. When the property HasMorePages is set to true, the PrintPage event will be called again till HasMorePages is set to false.

Implementing the Print preview option is pretty simple. We have to create a PrintPreviewDialog object, associate this object with our PrintDocument member m_printDocument and call its ShowDialog() method. Calling ShowDialog() will fire the PrintDocument object's PrintPage event. This time the PrintPageEventArgs object event will represent the Print preview dialog area and not the printer. See the following code fragment from the function FilePrintPreview(). Also note that the StringReader member object is filled with contents of the m_richTextBox object.

string l_strText =
            m_richTextBox.Text;
m_iStartPos = 0;

/* Read the text into
            StringReader */
m_strReaderObj = new
            StringReader(l_strText);
PrintPreviewDialog l_ppDialog = new
            PrintPreviewDialog();
l_ppDialog.Document =
            m_printDocument;
l_ppDialog.FormBorderStyle =
            FormBorderStyle.Fixed3D;

/* Show the dialog
            */
l_ppDialog.ShowDialog(this);

Now we have completed implementing the print and print preview options for our Word Pad application.

Clipboard manipulation

We are still left with yet another interesting part of windows programming - clipboard operations. The RichTextBox control contains member functions which allow us to copy text, cut the text into clipboard, and paste from the same. See the following table for clipboard functions.

Function | Purpose
Undo() | Undo previous change
Redo() | Redo previous change
Cut() | Cuts data and put into clipboard
Copy() | Copy data and put into clipboard
Paste() | Paste data from clipboard into RichTextBox control
SelectAll() | Select all text in the control

All functions are straight forward, except for Paste(), in which the type of data in the clipboard has to be determined and then pasted in the control. See the following code that pastes text data from clipboard into RichTextBox control.

/* Get Clipboard data */
IdataObject
            l_objClipboard = Clipboard.GetDataObject();
/* Determine the type
            */
if ( l_objClipboard.GetDataPresent(DataFormats.Text ) == true
            ) {
/* Copy Text data from clipboard
            */
m_richTextBox.Paste(DataFormats.GetFormat(DataFormats.Text));
}

GetDataObject() is a static method of the Clipboard class which returns an IDataObject object. The GetDataFormat() method of IDataObject will take a DataFormats argument and returns true if the data present is in the specified format. We determine the type of data using enumerated constant DataFormats.Text for text data. The other valid values are DataFormats.Bitmap, DataFormats.Dib and more. Implementing other formats of clipboard operations are left as assignment to the reader. Font and Color of RichTextBox can be changed using SelectionFont and ForeColor properties.

Displaying & Activating Web Links

The .NET Framework provides the capability of displaying web links in the RichTextBox control. In order to display web links, set the property DetectUrls to true. After setting this, when you type a web link or a mail link, RichTextControl will show it with a blue underline as shown below.

When clicking on such a link, nothing will happen because we need to trap the LinkClicked event and write code to launch the desired application. So add a handler to this event and write the following code to the handler.

Process.Start(e.LinkText);

The second argument to the event handler is LinkClickedEventArgs object. The LinkText property of this object will return you the URL text. The Process class is available in the System.Diagnostics namespace, so include this namespace at the top of the file. Start() is a static method of the Process class. In the above figure, clicking on the http web link will open your default browser with this URL. Clicking on the mailto link will open your default mail client application.

Conclusion

We have completed development of the Word Pad application in C#. In this article we discussed the Rich Text format, and the .NET framework RichTextBox control. We then developed the Word Pad application and learned about RichTextBox and it's various methods and clipboard support. We learnt in detail about Printing documents and implementing Print preview in C# apart from ToolBar and ImageList controls.

C#Today - © 2002 WroxPress Limited


Advanced Features in C#

Advanced Features in C#
http://www.csharptoday.com/content.asp?id=1625
Article Rating: 4
Published on August 10, 2001
Abstract
When Microsoft introduced its .NET strategy, it realized the need for a language that was a good combination of VB and C++ ?both simple to develop and powerful. C# is a new language that aims to meet these requirements. C# is a fairly simple language whose design has been highly influenced by C and C++, but unlike C and C++, C# is a much stricter language that protects developers from shooting themselves in the foot, and offers much better support for object oriented–ness than its predecessors. However, C#’s simplicity is not to be confused with the powerful features that this language offers. In this article, Kaushal Sanghavi explores some advanced features of the C# language, like auto documentation, boxing, and indexers.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

Visual Basic, C and C++ have been the primary development languages on the Windows platform for many years now. Java has come to the fore in the recent past, but it has been a language clouded by debates ranging from legal issues to performance. Visual Basic is a great language; simple to learn and develop, and it has matured considerably over the past few years. However, there has always been an issue with the performance of VB and many system features have not been easily accessible to this language. C and C++ have always been first class citizens in the Windows world and offer great performance. However, these languages have been fairly difficult to learn and the learning curve has been steep. When Microsoft introduced its .NET strategy, it realized the need for a language that was a good combination of VB and C++ - both simple to develop and powerful. C# is a new language that aims to meet these requirements. C# is a fairly simple language whose design has been highly influenced by C and C++, but unlike C and C++, C# is a much stricter language that protects developers from shooting themselves in the foot, and offers much better support for object oriented-ness than its predecessors. However, C#'s simplicity is not to be confused with the powerful features that this language offers. In this article, we will explore some advanced features of the C# language.

Note that this article is based on Beta 2 of Microsoft Visual Studio .NET and assumes that readers have a basic knowledge of C#, Visual Basic and C++.

Classes and Interfaces

A basic requirement of any object oriented language is its support for classes and interfaces. Visual Basic had a crude concept of classes, with no support for polymorphism or parameterized constructors. C++ fully supported classes, but had crude support for interfaces. For C++ developers, an interface was essentially a class with all pure virtual methods. (A pure virtual method is a virtual method that has no implementation, and derived classes have to override this method). Further, any class with at least one pure virtual method automatically became an abstract base class that could not be instantiated. C# makes all this support much cleaner and much more explicit with keywords that communicate the exact intention of the developer.

  • Classes: A class is very similar to its counterpart in the C++ language. Unlike C++, C# does not support multiple inheritance. A very simple class is as shown below.

public class MyClass
{
public MyClass() {}

public MyClass(int iValue) {}

private MyClass(string iValue) {}

~MyClass () {}

public void foo () {}
}

C# allows overloaded parameterized constructors (Overloaded methods are methods that have the same name but differ in their arguments). Overloaded constructors are useful when the class supports multiple forms of initialization. For example, a character string can be created by copying another character string, or as an empty string or as a representation of an integer. Further, C# supports finalizers in the form of destructors. Note, that C#'s destructors are very different from destructors in C++ or the Class_Terminate method in VB. C#'s destructors are non deterministic and there is no knowing when the .NET garbage collector will call the object's destructor. For more information on .NET garbage collection, please refer to my article at http://www.csharptoday.com/content.asp?id=1578&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq.

  • Interfaces: Interface based development is a very powerful way of building loosely coupled and extensible systems. With interface based development, objects interact with each other using interfaces rather than using names of the concrete classes. Since the objects don't really know the other class, classes can be plugged out at will, and as long as the new classes support the same interfaces, the calling client will still work. This also helps in building polymorphic systems, since instances of different objects could be passed transparently to the same client object based on different conditions, as long as they support the same interfaces. Traditional languages have not really supported interfaces natively, but in C#, interfaces are a first class citizen. C# supports the interface keyword that can be applied to an interface. The following is a very simple example of an interface in C#.

// declare an interface called
            MyInterface
public interface MyInterface
{
void foo
            ();
void foo (int iValue);
int bar ();

// declare a
            string property called MyProp
string
            MyProp{
get;
set;
}

}

// MyClass inherits
            from MyInterface and has to implement all
// the methods and
            properties from above
public class MyClass :
            MyInterface
{
public void foo () {}
public void foo (int
            iValue){}
public int bar () {return 1;}

public string
            MyProp{
get{
return mStr;
}

set{
mStr =
            value;
}
}

private string mStr;
}

The code block above displays an interface with three methods and one property. The class that implements this interface needs to implement all the methods and properties that have been declared by the interface. A client can now access the MyClass object via the MyInterface interface as shown below:

MyInterface lInterface = new MyClass
            ();
SomeMethod (lInterface);

public void SomeMethod
            (MyInterface iInterface)
{
iInterface.foo ();
}

As you can see from the code above, the class MyClass is being referenced entirely on the basis of the interface MyInterface that this class implements. Now if there were another class that implemented the same interface MyInterface, an instance of this new class could be passed to the SomeMethod method transparently. Of course, this is assuming that the interface documents its semantics clearly for the classes to implement.

Interfaces have to adhere to the following guidelines. Note that while these are some key points, this is not a complete list.

  • All the methods and properties in an interface are public, and you cannot use the public (or private or protected) keyword in an interface's method declaration.
  • Classes implementing an interface have to mark all of the implemented methods as public.
  • Interfaces cannot declare constructors or destructors.
  • Interfaces cannot provide an implementation of any member methods.
  • Interfaces cannot declare data members.
  • Interfaces cannot be instantiated; classes implementing the interface can be instantiated.
  • Interfaces can derive from other interfaces; interfaces cannot derive from classes.
  • Non-inheritable classes: C# supports classes inheriting from other public classes and interfaces. While this is a key feature of any object oriented language, under certain conditions, developers might want to stop any classes from inheriting a particular class. C# supports this natively via the sealed keyword. Once a class is marked as sealed, no class can derive from this class.

public sealed class MyClass
{.}

  • Abstract classes (must inherit): A counterpart to a sealed class is a class that has to be inherited. C# supports this via the abstract keyword. A class marked as abstract cannot be instantiated, and has to be inherited by another class. Further, an abstract class can contain an abstract method as well. An abstract method is a method that does not have an implementation, and the class inheriting from the abstract class has to provide the implementation of all abstract methods in the abstract class. Note that only abstract classes can contain abstract methods. The difference between an abstract class and an interface is that an abstract class can provide definitions for member methods and properties, while an interface is merely a declaration and cannot provide any definition.

   public abstract class
            MyClass
{
public MyClass (int iValue) {}

~MyClass
            (){}

public abstract void foo ();
}

Boxing and Unboxing

The .NET framework introduces a new way of allocating and deallocating objects. Objects in the .NET world are created on the managed heap which is managed by the .NET Garbage Collector. Based on this, types in .NET are separated in two categories; types that are allocated on the managed heap and are managed by the Garbage Collector, and types that are allocated on the stack and are active only for the duration of the method call (if these are passed to another method, their values are copied over). The former are called Reference Types and the latter are called Value Types. Under the .NET framework, the System.Object class is the base class of all types; all types can be converted to an Object type and instances of an Object can be cast to its corresponding type. All classes are considered reference types, and all other types such as native types (int, long, double, etc) and structs are considered value types. Under the .NET framework, the System.Object class is the base class of all types; all types can be converted to an Object type and instances of an Object can be cast to its corresponding type. Since value types and reference types are created and destroyed differently, this can create a problem when value types are cast as reference types and vice versa. Consider the following sample of code where we convert an integer native data type to an object.

int lIntVal = 12;
object lObj = lIntVal;

The code shown above is perfectly legal code and will compile in C#. However as we discussed above, the lIntVal is a value type and is created on the stack and will be destroyed after the current method exits, while the lObj is a reference type that is created on the managed heap. To handle this conversion between value types and reference types, the .NET framework introduces the notion of Boxing. Boxing is the method of converting a value type into a reference. The system allocates a new instance of the reference type on the managed heap, copies the value of the value type into the newly allocated reference type, and returns a reference or pointer. The key point to note here is that this creates a copy of the value on the new reference type, and not a direct a reference to the value type. Thus if you were to change the value of lIntVal in the above example, it would not affect the value of the lObj object. Boxing is handled implicitly by C# when it encounters a conversion between value types and reference types. This will be true for most of the .NET languages, but some still require boxing to be specified explicitly on the code by the developer; such is the case with C++ with Managed Extensions.

As its name suggests, Unboxing is the exact reverse of boxing. When a reference type is converted to a value type, C# needs to copy the value of the reference type into the value type. Again, the .NET framework handles this implicitly when it encounters this conversion. However, during unboxing, C# checks to see whether the conversion can actually take place - and if the types are not compatible, it throws an InvalidCastException.

object lObj = 12;
int lIntVal = (int) lObj; // valid unboxing

// invalid conversion attempt. InvalidCastException thrown.
string lStrVal = (string) lObj;

As you can see from the above example, attempting to unbox to a string the object containing an integer throws an exception, while the cast to an integer succeeds. Wrapping all unbox operations in a try/catch block is a good programming practice.

Although C# handles boxing and unboxing implicitly, developers should keep in mind that boxing incurs a performance hit, and this should be avoided as much as possible.

C# Indexers

Most traditional programming languages support the notion of arrays, which is a group of objects of the same type. These arrays allowed you to access the individual array elements based on the position of the element. This position was also known as the array index, and would be a numeric value into the array. Using these numeric indexes, it was very difficult to gain control over the access of the array elements; the language itself would resolve the reference to the appropriate element. C# makes customization of these array elements easy by introducing the notion of Indexers. Indexers are similar to C# properties, except that you can pass parameters to the accessors that can be used in resolving the reference to the array element. Using indexers, it is possible to index a class instance just like an array.

lIndexedObject ["somekey"] = lValue;
lIndexedObject [12] = lValue;

The code above shows how indexers can be used with an object instance. Note that we use both string and integer values to index into our object. In fact, any data type can be used as an index value, and you can declare an indexer with more than one parameter. The following example shows a simple indexer declaration:

   public class
            MyIndexedClass
{
public string this [int iIndex, string
            iStr]{
get{
return mStr;
}

set{
mStr =
            value;
}
}

public string this [int
            iIndex]{
get{
return mStr;
}

set{
mStr =
            value;
}
}

string mStr;
}

public class
            MyClientClass
{
public void foo ()
{
MyIndexedClass
            lObj = new MyIndexedClass ();

string lStr = lObj [10,
            "keyVal"];
lObj [12, "keyVal2"] = "test";
}
}

As you can see from the above code sample, declaring an indexer is similar to declaring a property. An indexer supports the Get and Set methods similar to C# properties. This allows you to declare read only, write only and read-write indexers. Further, as the above example shows, you can overload indexers as long as they differ in the number or type of parameters or both. Accessing the indexers is similar to accessing array elements as shown in the client class method. In this sample example, we ignore the indexer parameters, but in a real world application, these parameters would be used as keys into some internal data structure that would map to an element. This data structure could be a private array or a linked list or a hashtable, or any structure that can hold key value pairs.

Indexers are a very powerful language construct that lets developers customize access to a collection of objects. Traditional languages have either never provided this support (Visual Basic), or have made it cumbersome (operator overloading in C++). C# makes this very simple by making it a first class language construct.

Collections, foreach and IEnumerable

Developers often need to obtain a collection of objects of a similar type, store them together, and iterate through them one at a time. If these objects are stored in an array, traversing the array based on its index is easy, however if these objects are stored in a collection the traversal is different. Similar to the Visual Basic For Each statement, C# also supports a foreach statement that allows for consistent traversal through arrays and collections. Using the foreach statement with arrays is simple and requires no extra effort from the developer.

int[] lArray = new int[] {1,2,3,4,5,6};

foreach (int lInt in lArray)
Console.WriteLine (lInt);

When using arrays, all you need to do is declare a variable of the same type as that contained in the array, and then use the foreach statement as shown above. Since all arrays in .NET derive from the System.Array class, the foreach statement can use the Length property of the array object to determine how many iterations are necessary to traverse through the entire array.

Using the foreach statement with custom collections, however, requires some additional effort. Using the foreach statement with a custom collection requires the custom collection to implement the IEnumerable interface from the System.Collections namespace. This interface declares a single method called GetEnumerator that returns an instance of a class that supports the IEnumerator interface. The foreach statement uses the methods from this interface to iterate through the collection. The IEnumerator interface declares the following methods that should be implemented by the custom collection class.

  • Current: This property returns the current element from the collection.
  • MoveNext: This method moves the enumerator forward to the next element in the collection.
  • Reset: This method resets the enumerator to the beginning of the collection.

The foreach statement uses the GetEnumerator method to obtain an instance of this collection's custom enumerator. Once the enumerator is obtained, the foreach statement resets the enumerator, and then uses the MoveNext and Current methods to traverse through the collection. The custom collection's implementation of the enumerator can determine the details of implementing these methods that can traverse through the collection.

   public class MyCollection
            : IEnumerable, IEnumerator
{
public MyCollection(int[]
            iArray)
{
mArray = new int [iArray.Length];

int lIndex
            = 0;
foreach (int lVal in iArray)
mArray [lIndex++] =
            lVal;
mIndex = -1;
}

public IEnumerator GetEnumerator
            ()
{
return this;
}

public object
            Current
{
get{
return mArray
            [mIndex];
}
}

public bool MoveNext
            ()
{
mIndex++;
return mIndex <
            mArray.Length;
}

public void Reset ()
{
mIndex =
            -1;
}

private int mIndex;
private int[]
            mArray;
}

The code block above shows a custom collection object that internally stores integer values in a simple array. The MyCollection class implements both the IEnumerable and IEnumerator interfaces. As you can see, the GetEnumerator method simply returns a reference to itself, since the same object is implementing both the collection and the enumerator. The MoveNext and the Current methods simply update the mIndex member variable to point to the current element in the array. The foreach statement calls the MoveNext method before it calls the Current property and as a result, both the constructor and the Reset method set the index to -1. The MoveNext method increments it by one, and the Current method simply returns the array element indexed by the mIndex data member.

While this seems to be a very simple implementation of the custom collection and enumerator, it suffers from one huge drawback. The above class exposes all the enumerator methods publicly and there is nothing stopping a client class from calling these methods incorrectly. A different implementation would hide the enumerator methods inside a nested class. This would ensure that the enumerator can be created only by the collection class; this ensures that the collection can set up the enumerator correctly and perform any necessary validations. The following code shows this implementation.

   public class MyCollection2
            : IEnumerable
{
public MyCollection2(int[]
            iArray)
{
mArray = new int [iArray.Length];

int lIndex
            = 0;
foreach (int lVal in iArray)
mArray [lIndex++] =
            lVal;
}

public IEnumerator GetEnumerator ()
{
return
            new MyCollEnum(mArray);
}

private int[] mArray;

private class MyCollEnum : IEnumerator
{
public
            MyCollEnum(int[] iArray)
{
mArray = new int
            [iArray.Length];

int lIndex = 0;
foreach (int lVal in
            iArray)
mArray [lIndex++] = lVal;
mIndex =
            -1;
}

public object Current
{
get{
return mArray
            [mIndex];
}
}

public bool MoveNext
            ()
{
mIndex++;
return mIndex <
            mArray.Length;
}

public void Reset ()
{
mIndex =
            -1;
}

private int mIndex;
private int[] mArray;
}
}

As you can see from the above code block, the collection class contains a private nested class that implements the enumerator. The key point here is that the nested class is private. This ensures that this class cannot be directly created by other objects; only the collection class can create an instance of this class. This hides the enumerator from the outside world, and the only way to access it is via the GetEnumerator method exposed by the collection class. A client can access this collection as shown below.

      int[]
            lArray = new int[] {1,2,3,4,5};
MyCollection lCol = new
            MyCollection (lArray);

foreach (int lVal in lCol)
// do
            stuff with lVal

Documentation

Documentation is one area that is despised by most developers. Most developers are required to write design documents, help with architecture and test plan documents, and document code by the use of comments. Very often, there is duplication of effort between these different documents. Often, we have to write a document that describes all the classes and methods in a given source file. Not only does this create duplication, maintaining this document is difficult. Every time there is a change to the class's methods, you need to remember to update the document. Incorrect or out of date documentation is often worse than no documentation at all!

In order to solve this problem, the C# language integrates documentation within the language itself. The language natively supports keywords that are used by the compiler to generate documentation. These keywords can be embedded within the source code via special comment tags, and during compilation, the compiler recognizes these keywords and generates an external XML file that represents the code comments. Since this document file is created from the code itself, as long as the code comments are updated correctly, the XML documentation will never get out of synch with the code. Further, since the documentation is generated by the compiler, and is embedded within comments, there is no run time penalty either to performance or to the size of the binary assembly.

C# introduces a new form of comments, which are applied to a single line marked as ///. This is different from regular comments that are marked as // for single line comments or /* and */ for block comments. These comments can be declared on type declarations (classes, data members, etc), methods and properties. The following is a simple example of documenting our collection class from the above section.

/// <summary>
/// This is a sample
            test class demonstrating some advanced C# features.
///
            </summary>
/// <remarks>
/// This class declares a
            collection of integers.
/// The collection class implements the
            IEnumerable and
/// IEnumerator interfaces to support the C#
            foreach construct.
/// </remarks>

public class
            MyCollection : IEnumerable, IEnumerator
{

///
            <summary>
/// Constructor
/// </summary>
///
            <param name="iArray">
/// Array containing elements to be
            copied into this object
/// </param>

public
            MyCollection(int[] iArray)
{
mArray = new int
            [iArray.Length];

int lIndex = 0;
foreach (int lVal in
            iArray)
mArray [lIndex++] = lVal;
mIndex = -1;
}

///
            <summary>
/// Called by foreach to get an
            enumerator.
/// </summary>
/// <returns>
/// An
            instance of a class implementing IEnumerator.
///
            </returns>

public IEnumerator GetEnumerator
            ()
{
return this;
}

/// <summary>
///
            Returns the current element from this collection.
///
            </summary>
/// <returns>
/// Current element. Cast
            this to an integer.
/// </returns>

public object
            Current
{
get{
return mArray [mIndex];
}
}

///
            <summary>
/// Moves the enumerator forward by one
            position.
/// </summary>
/// <returns>
///
            Boolean value representing whether the current element is
///
            valid or not.
/// </returns>

public bool MoveNext
            ()
{
mIndex++;
return mIndex <
            mArray.Length;
}

/// <summary>
/// Resets the
            enumerator to the start of the collection.
///
            </summary>

public void Reset ()
{
mIndex =
            -1;
}

private int mIndex;
private int[]
            mArray;
}

When the above code is compiled, it generates the following XML file.

As you can see from the above screenshot, the generated XML file contains the comments as specified in the source code. As you can see, each member name has a prefix that describes the member. T: is used to indicate a type (class), M: is used to indicate a method, P: is used to indicate a property and F: is used to indicate a field (data member).

The following is a brief description of some of the tags that we have used in the above code.

  • summary: Specified at the method or class level and describes the object in brief.
  • remarks: Specified at the method or class level and contains a description of the object.
  • param: Specified at the method level and describes a parameter for the method.
  • returns: Specified at the method level and describes the return value from the method.

There are many more tags that can be used; a complete description can be found by following the links in the Links section below.

In Visual Studio .NET Beta 2, the documentation option is turned off by default. This can be turned on from the project properties as shown in the screen shot below.

Attached Sample Code

The attached sample code is a simple form that creates two collection classes as we described earlier in the article. The collections contain an integer array from 1 through the number entered in the first textbox. On clicking the Invoke button, we create the appropriate collection class and use foreach to enumerate through the collection and display the contents in the Elements text box. Further, as we described above, the MyCollection class is annotated with C# documentation. The generated XML file is shown in the first screenshot in the above section.

Conclusion

C# is an extremely powerful language. However, as with all new languages, developers will spend some time getting up to speed on its many features and how to derive the most from this language. Microsoft is betting heavy on the .NET framework, and C#, arguably, is going to be the language of choice for most .NET development. Developers should explore this language and attempt to keep ahead of the curve in the next few months before this language hits mainstream development in the enterprise.

Happy .NETing.

C#Today - © 2002 WroxPress Limited


Understanding Memory management in the .NET Framework

Understanding Memory management in the .NET Framework
http://www.csharptoday.com/content.asp?id=1549
Article Rating: 4
Published on August 9, 2001
Abstract
Thanks to Microsoft’s .NET, we don’t need to worry about memory management in our applications. But we still need to take a close look at what is happening with memory management in .NET runtime environment, and this is why: though memory management in .NET is completely hidden from developers, there are some dos and don’ts that we should know about Garbage Collection so that we can keep our applications performing at their best. In this article, Chandu Thota looks at the Garbage Collection process by defining a hypothetical “Business Card Holder Problem? and then through some code samples developed in C#.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

Thanks to Microsoft's .NET, we don't need to worry about memory management in our applications at all! But we still need to take a close look at what is happening with memory management in .NET runtime environment though. Here is why: though memory management in .NET is completely hidden from developers, there are some Dos and Don'ts that we should know about Garbage Collection so that we can keep our applications up to their best performance. In this article, we will look at the Garbage Collection process by defining a hypothetical "Business Card Holder Problem", and then through some code samples developed in C#.

Garbage Collection analogy - a Business Card Holder Problem

Garbage Collection (GC) is responsible for all memory management activities that are being performed in the .NET framework runtime, from allocating new objects in the memory to cleaning up unused objects. To understand how GC works in .NET, we need to understand the following internal data structures controlled by GC:

  • Managed Heap
  • Finalization Queue
  • Freachable (pronounced "f-reachable") Queue

Instead of getting into technical definitions and details of the data structures mentioned above, let us understand memory management through analogy - by defining a hypothetical "Business Card Holder Problem" to manage business cards in a business card holder:

  • Business card holder can only hold 100 business cards.
  • Have to keep each and every new business card only in the above business card holder.
  • Should destroy unused business cards.
  • Should check for any important notes on the back of the each un-used business card (there could be an important phone number!!) before destroying it, and ask your assistant to note it down if necessary. (Imagine you have an office assistant to do this job!)

With the above rules in place, you'll do just fine with the first 100 cards, as all you do is to insert each new business card into the holder. On the arrival of 101st card, all of a sudden you have a problem: you have to make some room to accommodate this new card in the holder.

Since the holder is already full (according to rule #2 mentioned above, you should use the same card holder) now you sit back and go through all the 100 business cards to see if there are any un-used cards, so that you can make room for new ones.

In that process say you have identified some cards that are no longer in use. Abiding to the rule #3 and rule #4 mentioned above, you have to check the back of each business card and see if there is any important information before destroying them.

Let us say that you found several cards with important phone numbers and you wanted your assistant to copy the information in to a phone book, so you keep them in the "OUT" basket of your desk where your assistant has access.

At this time you have destroyed the un-used cards with no important information and the other cards with information are on your assistant's desk while he/she copies the information into a phone book. And eventually those cards will be destroyed too.

After all this, you are done with your business cards management and now you have space for few new cards.

Now, replace:

  • Business Cards with Objects
  • Business Card Holder with Managed Heap
  • Out basket on your desk with Finalization Queue
  • Your 's Assistant's Desk with Freachable Queue
  • Yourself with GC
  • Your assistant with Special Runtime Thread dedicated to execute class destructor (internally Finalize) methods.

(I'm using Destructor and Finalize loosely at the same level since the compiler will inject Finalize() method into your class Destructor.)

That's it! Now you know how GC works in .NET runtime.

Though the details of the GC process are more complicated than above, that is the essence of how GC works in .NET. However there are some interesting details in this example and let us see what really they mean in the world of GC.

  • All your assistant does is to copy information from card to phone book: Which means:

The .NET Runtime has a special dedicated Thread to execute Destructor/ Finalize method implementations of the objects marked by GC for collection. GC doesn't really have any control over the special thread (though you have control over your assistant in real life) on how and when to execute Destructor/Finalize methods.

Since the Finalize process runs on a different Thread, your Finalize implementation should not assume any access to data/variables on the main application's Thread' stack.

Apart from the above details there are few other important concepts in the GC process: Generations, Resurrection and WeakReferences. Though we will take a look at these three concepts more elaborately in next section with examples, let us introduce them briefly now:

Generations

A Generation is a token (with an integer value) assigned to an object on the managed heap to represent its age. By rule of thumb all objects that are newly created on managed heap will be created with Generation ZERO (0). And say for example an object survived a Garbage Collection cycle (could be due to some strong references from the Application) then the generation of that object will be automatically incremented by 1, until the highest generation of 2. The higher the Generation, the more privileged the stay (i.e. longer) on the managed heap.

The whole idea of implementing generations is to make GC process more efficient. The rule that rules generation process is "the newer the object on the heap the lesser the expected life will be". At this time GC in .NET Framework supports a maximum of 2 generations.

Resurrection

Resurrection can be explained as the process of getting a dead (decided by GC as it un-referenced from application) object back to life by referencing it during the Finalization process after being moved to the Freachable Queue (Finalized but Reachable", hence the name for the Queue).

From the above Business Card Holder analogy resurrection can be explained like this: Say you have changed your mind and you have decided not to destroy a particular business card which has already been moved to your assistant's desk, we say that card is being resurrected!

Weak References

If there is a reference to an object on the managed heap from an application then we say that object has a "strong reference" and the GC will never collect it. However GC supports references called "weak references". Weak Referencing is a mechanism where you set a reference (using the WeakReference object type) to an object you don't need currently, but might need in future. GC will consider this kind of reference as a weak reference and it will collect the object only case of a low memory situation.

With this background let us move forward to experiment with the GC in real time with some code samples in C#.

Exploring GC: C#'s way

Before we get into code samples, we should keep in mind that forcing garbage collection from an application to collect unused memory is not a recommended practice. However there are some cases where you can take the liberty of forcing GC to collect from your application. A good example could be a case where we know that an object is consuming a huge chunk of resources and is no longer being referenced, so it is better to force a garbage collection instead of waiting for the GC to do it.

The GC class is a member of the System namespace in the .NET framework. You can access the GC class documentation on line at System.GC.

Let us look at some code samples now.

myObj

Let us define a simple object sub classed from "Object" (I called it myObj). We will be using this object to illustrate GC behavior in all the examples. (I have eliminated implementation code for more clarity. Please refer to supporting material for full implementation.)

private String name;

The above property holds the name of the object's name.

private Boolean ResurrectionFlag;

The above property is used to set the Resurrection property.

public myObj(String s)

The above constructor accepts name while instantiating the object.

public myObj(String s, Boolean f)

The above Constructor is being called to instantiate Resurrectable objects.

public void DisplayGeneration()

The above function can be called to detect the generation of the object.

public void Dispose()

Implementing the Dispose method is a better way to do your own resource cleanup process without depending on GC to take care of cleanup through the Finalize method. This is for one simple reason: GC's decision to dispose of objects is completely based on the demand for memory on the managed heap, but there could be some objects that need to be cleaned up even if we have no demand for memory on the managed heap.

For example, think of a database connection, you might want to close your database connection after your use it instead of leaving it on the heap till GC collects them in next cycle. (And remember you never know when that next cycle will occur!)

~myObj()

Implementation of the class Destructor.

public void WriteIt(String status)

The above method will be called several times in our examples to log the status of the object into a text file.

We could also use a simple Console.WriteLine() to see the output on console screen instead of a text file.

MyStaticMethods

Now let us prepare some static methods for the main application to have more structured calls to GC methods and logging function. I have defined a class called myStaticMethods with the following methods:

   static public void
            ForceCollection() {
WriteIt(" Forcing a garbage
            collection");
GC.Collect();
}

The above function makes a call to GC.Collect() which would be a forced collection.

static public void ForceCollection(int generation) {
WriteIt(" Forcing a GC.Collect for generation " + generation);
GC.Collect(generation);
}

The above function is an overloaded version of ForceCollection, but the difference is, this would force garbage collection on a specific generation of objects residing on the managed heap.

static public void LetFinalizersRun() {
WriteIt(" Waiting for Finalizers to complete");
GC.WaitForPendingFinalizers();
WriteIt(" Finalizers are complete");
}

The function shown above will request the GC to wait till the "Finalization Thread" is done with the Finalization process.

Now, let us prepare a "Main" function to call the above static methods and also the methods that we are going to discuss in the next section.

public static int Main(String[] args)
            {
WriteIt("Main Thread started( Hashcode # " +
Thread.CurrentThread.GetHashCode() + "
            )");
try
{
//Insert code to call functions in the next
            section
//Example : Basics();
}
catch (Exception
            e)
{
Console.WriteLine(e.ToString());
}
WriteIt("End of
            Main Thread");
return 0;
}

Now we are ready to try some sample runs using above static methods and myObj.

(Again, please refer to support material and read readme.txt if you need more instructions on how to run the above example, in case if you are beginner to C#)

GC's Basic Functionality

Let us call the following function from the above Main() program:

private static void Basics()
{
myObj
            obj1 = new myObj("Basics1");
obj1 = null;
ForceCollection();
LetFinalizersRun();
myObj obj2 = new
            myObj("Basics2");
ForceCollection();
LetFinalizersRun();
Console.WriteLine(obj2.ToString());
}

When you compile the main program and run it, a log file will be created with name gclog.txt in the same folder where the application is located and the following text will be displayed on the console.

And the log file contents will look like below:

We can conclude the following from the above output:

  • "Basics1 " is being collected upon forced collection and being finalized on the thread with hashcode # 8 (which is different from the main thread!).
  • "Basics2 " is alive as long as the application is alive because (you will see string "myObj " displayed on your console) there is a Strong Reference from the last statement in the Basics function.
  • Try the above function with a commented last line. Surprisingly "Basics2 " will also be collected along with "Basics1 " in the GC process though it was not explicitly set to "null". It is because the JIT compiler decided to set it to null since there are no references to that objects in the application at all!

Resurrection Example

To understand the concept of resurrection, let us consider the function below:

private static void Resurrection ()
{
myObj objImrtl = new myObj("Resurrection",
            true);
objImrtl =
            null;
ForceCollection();
LetFinalizersRun();
objRef.WriteIt("Still
            alive after Finalize called");
}

The constructor sets the resurrection flag to "true", which adds a reference to the "Resurrection " object from the Finalize method. Now when this method is being executed by Finalization thread, it comes back to life thanks to the strong reference.

When you run the Main program with a call to the function above the log file will be created like below:

GC.KeepAlive Example

The GC class has a method to make an object ineligible for garbage collection, from the starting of the application to the point where you make a call to this function. So the importance of this function lies in its location.

Let us call the function below from a Main program and see the results.

private static void
            KeepAlive()
{
myObj objKAL = new
            myObj("KeepAlive");
ForceCollection();
LetFinalizersRun();
GC.KeepAlive(objKAL);
ForceCollection();
LetFinalizersRun();
}

The results will look like this in the logfile:

As you can see the object "KeepAlive " survived the first garbage collection process because of the Gc.KeepAlive reference to it after the first collection. However in the second forced garbage collection it has been collected because this collection happened AFTER the KeepAlive call.

Implementing Dispose

As we discussed earlier, implementing the dispose method is a good practice when you know that you have to cleanup your objects.

In the first instance of myObj ("Disposed") from the following example we are explicitly calling the Dispose method in which we could add our cleanup code like closing connections etc.

In the second instance of myObj ("Not Disposed"), we will let the GC do the Finalization process.

   private static void
            DisposeMe()
{
myObj objDspsMe = new
            myObj("Disposed");
objDspsMe.Dispose();
objDspsMe =
            null;
ForceCollection();
LetFinalizersRun();
objDspsMe =
            new myObj("Not Disposed");
objDspsMe =
            null;
ForceCollection();
LetFinalizersRun();
}

When we run the Main program with the above function call the log will look like below:

As you can see from above the object is being collected both times. But the only difference is Dispose is more efficient than the GC based Finalize.

Weak Reference

In this function let us have a weak reference set to object "WeakRef ", with the "trackResurrection" option set to true.

private static void WeakRef()
{
myObj
            objWk = new myObj("WeakRef");
WeakReference wr = new
            WeakReference(objWk, true);
objWk = null;
ForceCollection();
LetFinalizersRun();
objWk = (myObj)
            wr.Target;
WriteIt(" Strong reference to object obtained: " +
            (objWk != null));
if (objWk != null) {
objWk.WriteIt(" Thanks
            for letting me live!");
}
objWk = null;
wr = null;
}

When we call the above function from the Main program we see the log like below:

As you can see, we could obtain the strong reference of the object "WeakRef" through a weak reference object, though it was being marked for GC collection process.

But if you try this example with the "trackResurrection" flag set to "false" for the weakReference object you will not be able to obtain a strong reference.

More on Finalization Queues

There is a method we can use in the GC class to manually register and suppress Finalization of a particular object. Let us see how they work.

Consider the following example. We called the GC.ReRegisterForFinalize method for the object being created. By doing this, all we are doing is adding a pointer (an entry) to the "FinalizeQs " object in the Finalize Queue.

private static void
            FinalizeQs()
{
myObj objFinalizeQs = new myObj("FinalizeQs");
GC.ReRegisterForFinalize(objFinalizeQs);
//GC.SuppressFinalize(objFinalizeQs);
objFinalizeQs
            = null;
ForceCollection();
LetFinalizersRun();
}

When we call the above method from the Main program, we see the log below.

The interesting thing to notice in the above output is that Finalize is being executed twice on Thread 8. The first time is because we registered for Finalization manually and the second time is due to the GC's default registration for finalization!

If you try the same code by uncomment the call to the SuppressFinalize method, Finalize will be executed only once. What the SuppressFinalize method does is to remove the pointer from the Finalize Queue!

Generations

In this final method let us take a look at the generations concept of GC. Let us force garbage collection several times while not setting reference of the object to null and see what happens to its generation!

private static void
            Generation()
{
WriteIt("GC supports up to generations: " +
            GC.MaxGeneration);
myObj objGen = new
            myObj("Generation");

objGen.DisplayGeneration();

ForceCollection();
LetFinalizersRun();

objGen.DisplayGeneration();

ForceCollection();
objGen.DisplayGeneration();

ForceCollection();
objGen.DisplayGeneration();

objGen = null;

ForceCollection(0);
LetFinalizersRun();

ForceCollection(1);
LetFinalizersRun();

ForceCollection(2);
LetFinalizersRun();

}

When we call the above method from the Main program, what we see is this:

As you can see the object was created on the managed heap with generation 0. When it survived the first forced garbage collection, the generation was incremented to 1 by GC. And it was incremented to 2 in next forced garbage collection. Since the maximum generation supported by runtime at this is 2, so the generation stays 2 even if you force garbage collection again.

GC also supports collection for specific generation of objects. And the object in the above example is being collected in a call for generation 2 objects.

Summary

Memory management in .NET is completely hidden from developers, yet it is very efficient. In the above discussion we confined ourselves to a single threaded case. As you can guess the GC process will be much more complicated in a multi-threaded situation.

Finally, Garbage Collection and memory management used to be an unknown territory to a VB/ASP Programmer right from the beginning. This is past with the arrival of .NET! Now with the new .NET languages one can easily implement the above methods and can have more control over memory management to design more efficient applications.

C#Today - © 2002 WroxPress Limited


.NET Serialization

.NET Serialization
http://www.csharptoday.com/content.asp?id=1532
Article Rating: 4.5
Published on July 27, 2001
Abstract
Serialization is the technology whereby an object is flattened into a linear stream that can be easily passed across process boundaries and machines. The .NET framework relies heavily on SOAP and Remoting, and thus needs a disciplined approach to object serialization. Keeping this in mind, the .NET framework exposes a consistent approach to serializing and deserializing objects, and supports classes and interfaces that are responsible for this. In this article, Kaushal Sanghavi explores the .NET support for Serialization and discusses how to build a class that supports custom serialization.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

Serialization is the technology wherein an object is flattened into a linear stream that can be easily passed across process boundaries and machines. Ever since the early days of RPC, serialization has presented numerous challenges to designers and developers of distributed applications. In a distributed system, or even in a message passing system, how does one transport a data structure across process boundaries? This has posed significant issues, which have merely been compounded with the advent of DCOM, SOAP and other distributed protocols. Serialization has often been tricky and problematic and has had to be reinvented by developers over and over again. This has partly been to a lack of a common standard that can be enforced, and partly due to a lack of simple API support from the platform. The .NET framework relies heavily on SOAP and Remoting, and thus needs a disciplined approach to object serialization. Keeping this in mind, the .NET framework exposes a consistent approach to serializing and deserializing objects, and supports classes and interfaces that are responsible for this. In this article, we will explore the .NET support for Serialization and we will discuss how to build a class that supports custom serialization.

To run this code you will need:

  • VS.NET Beta 2 & associated requirements (MDAC 2.7, IE6, .NET Framework, Windows 2000)

Serialization

As we mentioned above, Serialization is the technique in which an object is converted to a flat stream of data that can be passed between process and machine boundaries. This flat stream needs to be in a format that is recognized by both ends of the communication channel, so that the object can be serialized and reconstructed seamlessly. Ideally, we need a standard that is recognized and supported by multiple systems. This is gaining more and more importance since systems are far more distributed and disjoint that ever before. This coupled with the fact that complex systems are being built composing of loosely coupled heterogeneous objects, requires a common approach to serialization. While existing technology does not fully support this notion, XML is gaining popularity and is well on its way to becoming the de facto standard of interoperability on the public internet.

Serialization is used extensively in the .NET framework. SOAP uses serialization, Remoting uses serialization, Web Services use serialization, MSMQ uses serialization, .NET sockets use serialization; the list is endless.

As you can see from the above diagram, the concept of serialization is very simple. You take an in memory object and store its contents linearly. Thus you can have one stream that represents all the internal data of an object. This gets slightly more involved when an object contains nested objects, but even then the concept remains the same. Take every object and break it down into its native data types, and store it in a contiguous manner. This stream can now be persisted to disk or sent across any communication channel, and the receiving object can easily convert the stream back into an object. Of course, the serialized stream needs to store additional information that describes its contents. This is in the form of some metadata that describes that S1 is a string that is of a particular length, I1 is an integer represented by 4 bytes, S2 is again a string of a particular length and so on. The nested objects O1 and O2 would be represented by their own member data types. Once all this type information is made available to the deserializer, constructing an object from this stream is trivial. The challenge, however, lies in the metadata that describes the serialized string. If the serializer and deserializer do not agree on the metadata format, interpreting the serialized data is impossible. There is where XML fits in nicely. Since XML is commonly supported, most objects would be able to read the serialized content in XML format and reconstruct the object. In our simplistic sample above, the serialized content would like the XML below.

<MyObject>
<S1>
value of S1
</S1>
<I1>
value of I1
</I1>
<S2>
value of S2
</S2>
<O1>
contents of O1 and their values here
</O1>
<O2>
contents of O2 and their values here
</O2>
</MyObject>

Given this XML, a deserializer can create an object of type MyObject, and set the values of its data members to the values that are specified in the XML payload. As you can see, the serializer and the deserializer are totally isolated from each other and do not require any knowledge of each other. As long as the format is consistent and standard, the actors don't matter.

XML Serialization

The .NET framework supports converting an object to an XML stream and vice versa natively. We will not elaborate on this topic in detail since this topic has already been explained in Matt Reynold's excellent article, which can be found at http://www.asptoday.com/content/articles/20010403.asp?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq. Instead, in this section we will explain some of the key concepts of XML Serialization.

The System.XML.Serialization namespace contains the XMLSerializer class that contains properties and methods that support this conversion. This class uses reflection to call all the public properties on your object and it converts this data into XML. The object that is being serialized does not have to do anything special in order to support serialization; the object is completely oblivious of this. The following is a simple example of serialization using the XMLSerializer.

   public class
            MyObject
{
public MyObject ()
{
mStr =
            "default";
}

public MyObject (string iStr)
{
mStr =
            iStr;
}

public string Data
{
get{
return
            mStr;
}
set{
mStr = value;
}
}

private string
            mStr;
}

private void Serialize_Click(object sender,
            System.EventArgs e)
{
MyObject lObject = new MyObject
            ("abcd");
StreamWriter lWriter = null;
XmlSerializer
            lSerializer;

try{
lWriter = File.CreateText
            (@"D:\obj.xml");
lSerializer = new XmlSerializer
            (typeof(MyObject));
lSerializer.Serialize (lWriter,
            lObject);
lWriter.Close();
}
catch (Exception
            lException){
string lError = lException.Message;
Debug.Write
            (lError);
}
}

The above code creates a StreamWriter instance and an XmlSerializer instance. The stream writer represents the text file that the object will be persisted to and the serializer is the object that performs the actual serialization. The StreamWriter needs to know the name and location of the text file and the XmlSerializer needs to know the type of the object being serialized. Once the serializer is run on the MyObject class, the serialized data looks like the XML below:

<?xml version="1.0" encoding="utf-8"?>
<MyObject xmlns:xsi=http://www.w3.org/2001/XMLSchema-instance
xmlns:xsd=http://www.w3.org/2001/XMLSchema>
<Data>
abcd
</Data>
</MyObject>

There are a few important conditions that the object has to adhere to in order for it to be serialized:

  • Object needs to support a Public Default Constructor: The XMLSerializer class can only work with objects that expose a public default constructor. A default constructor is a constructor that does not take any parameters. Since the serializer and deserializer need to be able to create an empty instance of an object before it can start getting or setting properties, a public default constructor is mandatory. In this absence of this, the deserializer cannot determine what parameters to pass to a non-default constructor, and thus it will not be able to construct this object. This is a key concept to grasp; the deserializer creates an instance of the object by invoking the default constructor. Once this is created, the deserializer starts invoking the public Set properties on this object to set the values that the original object had at the time of serialization.
  • Only Public Properties with both Get\Set methods will be serialized: Since the XMLSerializer does not have access to the private members of your class, the only way it can get to its contents is via the publicly exposed properties. Since the serializer and the deserializer have to be synched up in order for serialization to work, the XMLSerializer will only invoke public properties that have both the Get and the Set methods. The serializer will invoke the Get property to obtain the value of the property and the deserializer will invoke the Set property to recreate the object from the serialized stream. Properties that are private, or properties that expose only one of the Get and Set methods will be ignored.

       public class
              MyObject
{
public MyObject ()
{
mStr =
              "default";
mStr2 = "default value 2";
}

public
              MyObject (string iStr)
{
mStr = iStr;
}

public
              string Data
{
get{
return mStr;
}
set{
mStr =
              value;
}
}

private string
              Data2
{
get{
return mStr2;
}
set{
mStr2 =
              value;
}
}

private string mStr;
private string
              mStr2;
}

As you can see from the above code, the Data2 property is private. Serializing this object will create an XML document that ignores Data2.

<?xml version="1.0" encoding="utf-8"?>
<MyObject xmlns:xsi=http://www.w3.org/2001/XMLSchema-instance
xmlns:xsd=http://www.w3.org/2001/XMLSchema>
<Data>
abcd
</Data>
</MyObject>

  • Only Public data members will be serialized: Again, since the serializer cannot access the private data members of a class (we are assuming that the class has no knowledge of the serializer and thus has not allowed friend access to it), it can only deal with the public data members. All the public data members will be serialized and assigned values to when the object is being deserialized.

   public class
            MyObject
{
public MyObject ()
{
mStr =
            "default";
mStr2 = "default value 2";
}

public MyObject
            (string iStr, string iStr2)
{
mStr = iStr;
mStr2 =
            iStr2;
}

private string mStr;
public string
            mStr2;
}

As you can see from the above code, mStr data member is private and does not have any accessor properties. Thus in the serialized XML, this data member will be ignored. Further, note that the node name is the name of the public data member.

<?xml version="1.0" encoding="utf-8"?>
<MyObject xmlns:xsi=http://www.w3.org/2001/XMLSchema-instance
xmlns:xsd=http://www.w3.org/2001/XMLSchema>
<mStr2>
efgh
</mStr2>
</MyObject>

  • Public members with public properties will be serialized twice: If a class has a public data member that is also exposed as a public property, this member will be serialized twice. Since the serializer cannot relate data members with the corresponding properties, it simply serializes all the public data members and invokes all the public properties. Note that this is not incorrect, it is simply inefficient. This is especially important when dealing with large objects. (Since the .NET framework is still in Beta, we do not have any concrete performance numbers yet.)

   public class
            MyObject
{
public MyObject ()
{
mStr =
            "default";
mStr2 = "default value 2";
}

public MyObject
            (string iStr, string iStr2)
{
mStr = iStr;
mStr2 =
            iStr2;
}

public string Data2
{
get{
return
            mStr2;
}
set{
mStr2 = value;
}
}

private
            string mStr;
public string mStr2;
}

In this example, the mStr2 member data is public and is exposed via the Data2 property. In this case, this will appear as a duplicate in the serialized XML.

<?xml version="1.0" encoding="utf-8"?>
<MyObject xmlns:xsi=http://www.w3.org/2001/XMLSchema-instance
xmlns:xsd=http://www.w3.org/2001/XMLSchema>
<mStr2>
efgh
</mStr2>
<Data2>
efgh
</Data2>
</MyObject>

SOAP and Binary Serialization

As we discussed in the above section, XML serialization using the XMLSerializer is extremely simple and totally transparent to the class. However, there are certain limitations on using this class for serialization as we discussed. Using the XMLSerializer, only public members and properties will be serialized. Private data members and properties are ignored. However, there might be certain cases where this is too restrictive. A class might want all its data members to be serialized irrespective of whether they are exposed via public properties or not. However, since the XMLSerializer does not require that the object know about the serialization before hand, this is the only possible solution. The key point to note is that since the object does not know about the XMLSerializer, if the XMLSerializer were able to serialize all the private data members of the object, it could possibly be a security issue. If an object can get to another object's private data members without the target object's knowledge, a malicious object could wreak havoc on these objects.

In order to avoid this, but at the same time be able to take advantage of serializing the entire object, the .NET framework provides Serialization Formatter objects. These objects are similar to the XMLSerializer in that they can flatten an entire object graph into a stream. However, the difference is that these formatters serialize all the data members of a given class; but for these formatters to be able to do so, the object needs to grant the formatter's permission to access its data members. Further, the object can specify which of its members can be serialized and which cannot. This is communicated to the formatters via the use of attributes. If an object specifies the required attribute on the class, it signifies that a formatter can gain access to its member data and serialize it to any channel it desires.

   [Serializable]
public
            class MyObject
{
public MyObject (string iStr, string
            iStr2)
{
mStr = iStr;
mStr2 = iStr2;
}

private
            string mStr;
private string mStr2;
}

Any class that can be serialized by a formatter needs to specify the Serializable attribute. This attribute is declared in the System namespace and is specified at the class level. In order for a formatter to serialize an object, not only does the object need to specify the Serializable attribute, but all objects that are nested within this object need to specify this attribute. If even a single object in the graph is not marked as Serializable, serialization for the entire graph fails.

   [Serializable]
public
            class MyObject
{
public MyObject (string iStr, string
            iStr2)
{
mStr = iStr;
mStr2 = iStr2;
}

private
            string mStr;
private string mStr2;
private
            NonSerializableObject mObject;
}

class
            NonSerializableObject
{
}

Serialization for the above class definition will fail since a contained object is not marked as Serializable.

using System.Runtime.Serialization.Formatters.Soap;
using System.Runtime.Serialization.Formatters.Binary;

private void Serialization_Click(object
            sender, System.EventArgs e)
{
MyObject lObject = new MyObject
            ("abcd", "efgh");

SoapFormatter lSFormatter = new
            SoapFormatter ();
BinaryFormatter lBFormatter = new
            BinaryFormatter ();

try{
FileStream lStream = new
            FileStream
(@"D:\obj.xml",FileMode.Create);
lSFormatter.Serialize
            (lStream, lObject);
lStream.Close ();

FileStream lStream2
            = new
            FileStream
(@"D:\obj2.bin",FileMode.Create);
lBFormatter.Serialize
            (lStream2, lObject);
lStream2.Close ();

}
catch
            (Exception lException){
string lError =
            lException.Message;
Debug.Write (lError);
}
}

The above code block shows you can use a SOAPFormatter and a BinaryFormatter to serialize our test object. Again, the only support that the test object needs to provide is the marking of the Serializable attribute. The following is the serialized XML that is generated by the SOAPFormatter.

<SOAP-ENV:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns:SOAP-ENC="http://schemas.xmlsoap.org/soap/encoding/"
xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/"
SOAP-ENV:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"
xmlns:a1="http://schemas.microsoft.com/clr/nsassem/TestSerialization/TestSerialization">

<SOAP-ENV:Body>
<a1:MyObject id="ref-1">
<mStr id="ref-3">
abcd
</mStr>
<mStr2 id="ref-4">
efgh
</mStr2>
</a1:MyObject>
</SOAP-ENV:Body>
</SOAP-ENV:Envelope>

While the Serializable attribute allows a formatter to serialize an object, it is possible that the object requires some of its data members to be serialized, but is sensitive about some other members that it would not want to be serialized. The .NET framework supports another attribute, NonSerialized, which when specified on a class or data member requires the formatter to ignore the value of the data member. Marking this on a member ensures that the value of this member does not appear on the serialized data. Note that this can be set only on member data, not on properties.

[Serializable]
public class MyObject
{
public MyObject (string iStr, string iStr2)
{
mStr = iStr;
mStr2 = iStr2;
}

private string mStr;

[NonSerialized]
private string mStr2;
}

<SOAP-ENV:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns:SOAP-ENC="http://schemas.xmlsoap.org/soap/encoding/"
xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/"
SOAP-ENV:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"
xmlns:a1="http://schemas.microsoft.com/clr/nsassem/TestSerialization/TestSerialization">

<SOAP-ENV:Body>
<a1:MyObject id="ref-1">
<mStr id="ref-3">
abcd
</mStr>
</a1:MyObject>
</SOAP-ENV:Body>
</SOAP-ENV:Envelope>

As you can see from the above example, the data member mStr2 is marked as NonSerialized, and is thus absent from the serialized XML.

Custom Serialization

All the discussion above has centered around an object being serialized by external entities. The object itself has been a passive observer and apart from setting a few attributes, does not do much to participate in its own serialization. While this suffices for most objects, there might be a few cases where the object needs to serialize itself. Consider the case of an object that is going to be passed around machine boundaries that contains sensitive information like personal or financial data. The designer of this object might not want the object to be serialized and stored as plain text (XML is essentially plain human readable text). Even the binary formatter does not help in this situation, since deserializing the binary data is trivial. In this case, the object might want to encrypt its state during serialization such that if a malicious user sniffed the serialized data over the network, it wouldn't hurt. The same object can now decrypt the data during deserialization provided the deserializer can prove to the object that it is an authorized operation.

The .NET framework provides the ISerializable interface that can be implemented by classes that need to support custom serialization. When this interface is implemented by a class, all the serializers and formatters call methods on this interface that can be implemented to customize the serialization. Any class that supports the ISerializable interface needs to support the following methods.

  • Constructor (SerializationInfo, StreamingContext): This constructor is called by the formatter during deserialization. The SerializationInfo class contains the methods to retrieve the name value pairs that are stored during serialization. Since this constructor should be called only during deserialization, a good practice is to declare this constructor as private. This ensures that clients do not inadvertently call this constructor with the incorrect parameters.
  • GetObjectData (SerializationInfo, StreamingContext): This is the counterpart of the above method, and is called by the formatter during serialization. This is the method wherein the object populates the SerializationInfo object with the name value pairs that it might want to serialize. Note that objects should serialize all relevant data in this method; the same data that is added to the SerializationInfo object will be passed to your object's constructor during deserialization.

These are the only two methods that an object needs to support for custom serialization. Now that we have covered what an object needs to support in order to customize its serialization, let us take a look at an object that implements ISerializable. We will implement a very simple security scheme that should be treated only as a demonstration of a concept. Note that it is trivial to bypass this scheme. When serializing our object, we will query the user name of the currently logged on user, and store that in the serialized stream. During deserialization, we check this identity against the currently logged on user, and only if the two match, will we allow the object to be deserialized. This is a very simple scheme to ensure that the serializing and deserializing users are the same. If not, we raise an exception during deserialization.

[Serializable]
public class MyCustomObject : ISerializable
{
public MyObject (string iStr, string iStr2)
{
mStr = iStr;
}

private MyObject (SerializationInfo iInfo, StreamingContext iContext)
{
WindowsIdentity lIdentity = WindowsIdentity.GetCurrent ();
string lCurrUserName = lIdentity.Name;
string lUserName = iInfo.GetString ("UserName");

if (lUserName != lCurrUserName)
throw new SerializationException ("Access Denied");

mStr = iInfo.GetString ("mStr");
return;
}

void ISerializable.GetObjectData (
SerializationInfo iInfo,
StreamingContext iContext)
{
WindowsIdentity lIdentity = WindowsIdentity.GetCurrent ();
string lUserName = lIdentity.Name;

iInfo.AddValue ("UserName", lUserName);
iInfo.AddValue ("mStr", mStr);
return;
}

private string mStr;
}

As you can see from the above code block, we use the WindowsIdentity class from the System.Security.Principal namespace to obtain the currently logged on user's username. This is added to the SerializationInfo object with the UserName key and is retrieved in the private constructor. The client that is used to call this object is shown below.

private void Serialization_Click(object
            sender, System.EventArgs e)
{
MyCustomObject lObject = new
            MyObject ("abcd", "efgh");
SoapFormatter lFormatter = new
            SoapFormatter ();

try{
FileStream lStream = new
            FileStream
(@"D:\obj.xml",FileMode.Create);
lFormatter.Serialize
            (lStream, lObject);
lStream.Close ();

lStream = new
            FileStream (@"D:\obj.xml",FileMode.Open);
MyCustomObject lObject2
            = (MyCustomObject)
lFormatter.Deserialize
            (lStream);
lStream.Close ();
}
catch (Exception
            lException){
string lError = lException.Message;
Debug.Write
            (lError);
}
}

We simply use the SOAPFormatter to serialize and deserialize the object. The serialized XML is as shown below. Note that in addition to the data member value, we also store the user name of the logged on user.

<SOAP-ENV:Envelope
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns:SOAP-ENC="http://schemas.xmlsoap.org/soap/encoding/"
xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/"
SOAP-ENV:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"
xmlns:a1="http://schemas.microsoft.com/clr/nsassem/TestSerialization/TestSerialization">

<SOAP-ENV:Body>
<a1:MyObject id="ref-1">
<UserName id="ref-3">
KAUSHAL_HOME2\kaushal
</UserName>
<mStr id="ref-4">
abcd
</mStr>
</a1:MyObject>
</SOAP-ENV:Body>
</SOAP-ENV:Envelope>

The following screenshot is from the attached code. The first part of the sample code deals with using the XMLSerializer, SOAPFormatter and BinaryFormatter to serialize objects in a standard way. This deals with the MyObject class from the MyObject.cs file. The second part of the sample uses custom serialization as we described above. The MyCustomObject class in the MyCustomObject.cs file implements the ISerializable interface and uses the currently logged on user name to authenticate the deserializer. Note that there is very little error handling in the code; most of the code is wrapped around a single try-catch block. Further, please make sure to use the same Serializer for both serializing and deserializing of the objects; if different serializers are used, the deserializer will raise an exception.

Conclusion

The .NET framework makes serialization of objects simple to design and develop. More importantly, the .NET framework attempts at creating a unified standard for serializing and deserializing objects that is very important for building distributed heterogeneous systems. Further, the use of XML and SOAP ensures that the protocols used are well supported and are industry standard.

Since the .NET framework supports XML and SOAP serialization, it is now possible to serialize and deserialize objects on different platforms. An object can be serialized as XML and passed across the wire to a different system and if the schemas are well known, the object can be reconstructed from the XML on any system.

Happy .NETing.

C#Today - © 2002 WroxPress Limited


Exploring the System.Array Class

Exploring the System.Array Class
http://www.csharptoday.com/content.asp?id=1538
Article Rating: 4
Published on July 24, 2001
Abstract
Arrays play a major role in manipulating data in the programming world and every language supports arrays to satisfy the programmers. Well, .NET also supports arrays with all .NET-enabled languages and exposes an Array class in the System namespace for array manipulation. The Array class has lots of features like searching (Binary Search), Sorting, cloning, etc. In this article Srinivasa Sivakumar uses C# as the language to describe the features of the System.Array class.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

Arrays play a major role in manipulating the data in the programming world and every language supports arrays to satisfy the programmers. Well, .NET also supports arrays with all .NET-enabled languages and exposes an array class in the System namespace for array manipulation. The array class has lots of features like searching (Binary Search), Sorting, cloning, etc. In this article I'm going to use C# as the language to describe the features of the System.Array class.

What is An Array?

Arrays are data structures that contain ordered set of values from the same data type (VB6 arrays are different, because VB6 variant data type variables can hold virtually any data). Mostly arrays are accessible via their indexes.

Why do we need the Array Class?

Before .NET, each language had it's own way of handling arrays. Not every language had the option to search the array, enumerate the array, etc. For example, if we needed to sort an array then we'd to write code for that. This code changes based on the language that we use. With .NET we've the universal Class Library (.NET Framework), the universal runtime (Common Language Runtime [CLR]) and common data types (All the .NET-enabled languages inherit their data types from the System namespace. For example, the String data type is inherited from System.String class.), which makes it possible build a single class and use it across all the .NET enabled languages such as VB.NET, C#, etc.

What does the System.Array class being to our plates?

In .NET, all the arrays are inherited from System.Array (base class) and the System.Array class is inherited from the System.Object class. This way, the System.Array object can inherit all the members of System.object like ( GetType , ToString , etc.). For more information about the System.Object class check out Dr. GUI's article #2 at http://msdn.microsoft.com/library/welcome/dsmsdn/drguinet01312001.htm?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq.

There are lots of useful methods and properties exposed by the System.Array class that we can use in our code day in and day out. Here is the list of main features of the System.Array class:

  • Sorting an array.
  • Binary Searching
  • Copying the whole or part of an array to a new array.
  • Clearing the whole or part of an array.
  • Reversing an array.
  • etc...

Let's see these features one by one in this article.

Static (Shared) Methods

The System.Array class exposes static ( Shared in VB.NET) methods. The static methods are non-instantiated methods and to access them we don't have to create an object of the type System.Array . For example, the Sort method is a static method. We'd call this method as:

Array.Sort(arrayname);

On the other hand, if we want to access the instantiated methods, we need to create an array object variable and access the methods. The CopyTo method of the System.Array class is an instantiated method. To access this method we've to create an object of the type array.

Int[] intArray = { 1, 2, 6, 5, 7, 3, 4};
intArray.CopyTo(...);

Sorting

The System.Array class provides a Sort method that we can use to sort the arrays. The Sort method is available in many flavors. Let's use a simple Sort method to begin with.

protected void Page_Load(object sender,
            EventArgs e)
{
int[] intArray =
            {11,3,15,66,22,554};

grdunSorted.DataSource =
            intArray;
grdunSorted.DataBind();

//Sort the
            Array
Array.Sort(intArray);

grdSorted.DataSource =
            intArray;
grdSorted.DataBind();
}

In the above code, we've declared an integer array. Then we bound the array to a Data Grid control to show the unsorted array. Then we've used the Sort method of the array class to sort the array (The Sort method takes the array as an parameter). Then we bound the array to a Data Grid control to show the sorted array. Here is the screen shot of the above code:

Sorting Two Arrays

We can also sort two arrays where one array containing the keys and the other containing the values. Let's see an example of this:

protected
            void Page_Load(object sender, EventArgs e)
{
int[] intArray =
            {11,3,15,66,22,554};
String[] strItems = { "Eleven", "Three",
            "Fifteen",
"Sixty Six", "Twentry Two", "Five fifty four"
            };

grdunSorted.DataSource =
            intArray;
grdunSorted.DataBind();
grdunSorted1.DataSource =
            strItems;
grdunSorted1.DataBind();

//Sort the
            Array
Array.Sort(intArray,strItems);

grdSorted.DataSource
            = intArray;
grdSorted.DataBind();
grdSorted1.DataSource =
            strItems;
grdSorted1.DataBind();
}

In the above code we've created two arrays, one holding the keys and the other holding the items equivalent to the keys. We've sorted the arrays and shown them in four different data grids. Here is the screen shot of the above code.

Sorting arrays based on Comparison roles

In the above two examples we've sorted arrays. When we're sorting the arrays we didn't give any specific comparison methods to follow while sorting. There are four similar implementations of the Sort method that are very similar to the ones we've already seen. Here is the syntax.

public static void Sort(Array, IComparer);
public static void Sort(Array, Array, IComparer);
public static void Sort(Array, int, int, IComparer);
public static void Sort(Array, Array, int, int, IComparer);

The last parameter IComparer is an interface (In .NET Framework all the interfaces start with I ), which returns values based on which one of the comparing operation. There are few implementations of IComparer interface is available like, Comparer (Case-Sensitive implementation of the IComparer interface), CaseInsensitiveComparer (Case-Insensitive implementation of the IComparer interface), etc. The Comparer is the default IComparer interface.

<script language="C#" runat="server">
public class StringComp : IComparer
{
public int Compare (object a, object b)
{
string s1 = a as string;
string s2 = b as string;
return s1.CompareTo(s2);
}
}

protected void Page_Load(object sender, EventArgs e)
{
String[] strItems = { "Eleven", "Three", "Fifteen", "Sixty Six",
"Twentry Two", "Five fifty four" };

grdunSorted.DataSource = strItems;
grdunSorted.DataBind();

//Sort the Array
Array.Sort(strItems,new StringComp());

grdSorted.DataSource = strItems;
grdSorted.DataBind();
}
</script>

As you can see, we've implemented a new class, which inherits the " IComparer " interface. Then we're overloading the Compare method and implementing the comparison role. Here is the screen shot of the above code.

Sorting a section of an array

We can also sort a particular section of an array with the Sort method. The syntax is:

public static void Sort(Array array, int index, int length);

We've to pass the array, index where the sorting should start (Do remember .NET Arrays starting index is always zero) and the length how long the sorting should happen. Here is an example and the screen shot of the code.

protected void Page_Load(object sender, EventArgs e)
{
int[] intArray = {11,3,15,66,22,554};

grdunSorted.DataSource = intArray;
grdunSorted.DataBind();

//Sort 2'd, 3'd and 4th elements of the Array
Array.Sort(intArray,2,4);

grdSorted.DataSource = intArray;
grdSorted.DataBind();
}

We can also implement the IComparer interface for the sort methods.

Sorting a section of two arrays

We can also sort a particular section of two arrays with the Sort method. The syntax is:

public static void Sort(Array keys, Array items, int index, int length);

Here is an example and the screen shot of the code.

<script language="C#" runat="server">
protected void Page_Load(object sender, EventArgs e)
{
int[] intArray = {11,3,15,66,22,554};
String[] strItems = { "Eleven", "Three", "Fifteen", "Sixty Six",
"Twentry Two", "Five fifty four" };

grdunSorted.DataSource = intArray;
grdunSorted.DataBind();
grdunSorted1.DataSource = strItems;
grdunSorted1.DataBind();

//Sort 2'd, 3'd and 4th elements of the Array
Array.Sort(intArray,strItems,2,4);

grdSorted.DataSource = intArray;
grdSorted.DataBind();
grdSorted1.DataSource = strItems;
grdSorted1.DataBind();
}
</script>

Search

The IndexOf method of the Array class can help us to search for a particular element in an array. Let's see an example of this.

<script language="C#"
            runat="server">
protected void Page_Load(object sender,
            EventArgs e)
{
int[] intArray =
            {11,3,15,3,22,3,554,2000,3,3,902};
int Index = 0, StartIndex =
            0;
String Output="";

//Search the array
while(Index !=
            -1)
{
Index = Array.IndexOf (intArray, 3, StartIndex);
if
            (Index != -1)
{
Output = Output + Index + ",
            ";
}
StartIndex = Index + 1;
}

//Bind the
            array
grd.DataSource =
            intArray;
grd.DataBind();

//Display the seach
            indexs
lbl.Text = "<B>The value 3 found in the following
            indexes: </B>" + Output;
}
</script>

The IndexOf method returns "-1", if the element not found. In the above code, we're using the while loop to search for the number 3 in the given array. Here is the screen shot for the above code:

Binary Search

The System.Array class also supports a BinarySearch method. The BinarySearch method is available in four flavors:

public static int BinarySearch(Array, object);
public static int BinarySearch(Array, object, IComparer);
public static int BinarySearch(Array, int, int, object);
public static int BinarySearch(Array, int, int, object, IComparer);

The simplest form of BinarySearch is passing an array object and the value to be searched (as an object). Let see an example for this.

<script language="C#"
            runat="server">
protected void Page_Load(object sender,
            EventArgs e)
{
int[] intArray =
            {11,3,15,3,22,3,554,2000,0,3,902};
Object
            SearchObj;

SearchObj = 3;

//Bind the
            array
grd.DataSource = intArray;
grd.DataBind();

int
            Index =Array.BinarySearch(intArray, SearchObj);
if (Index < 0
            )
{
//Display the search result
lbl.Text = "<B>The
            object 3 not found in the input
            array</B>";
}
else
{
//Display the search
            result
lbl.Text = "<B>The object 3 found in the following
            index: </B>" + Index;
}
}
</script>

In the above code, we're passing the array and the search value ( 3 ) to the BinarySearch method. Here is the result:

When I looked at the search result it was incorrect. It should have found the element 3 at the index 1. Instead it found the element 3 at the index 5. So, I sent an email to Microsoft .NET team about this problem and I got reply from Kit George. Here is the reply.

Srinivasa, BinarySearch makes no guarantee on what location it will find the searched argument. It simply guarantees that IF it exists in the array, it will return the index of the first element it finds, not necessarily the index of the first instance that occurs. Try this for a proof of concept: Make an array of ONLY 3's. BinarySearch probably won't return the first index. Make an array of only one 3, no other elements. In this case it does return 0, but that's just because the search works. Make an array with no 3's, and you will get -1. Regards, Kit

This means we can only use the BinarySearch method to find out if a given value is present in the array is not. For searching we can use the previous example with the IndexOf method. If you are interested you can play with the other overloads of the BinarySearch methods.

Clearing an array

The Clear method of the Array class can help us to clear a partials or full array. Let's see an example of this.

protected void Page_Load(object sender,
            EventArgs e)
{
int[] intArray =
            {1,2,3,4,5,6};

grd1.DataSource =
            intArray;
grd1.DataBind();

//Clear 2'd and 3'd elements of
            the Array
Array.Clear(intArray,2,2);

grd2.DataSource =
            intArray;
grd2.DataBind();
}

In the above example, we're clearing the second and third elements in the array. Here is how the screen shot looks:

Note: The Clear method only clears the values in the array elements. It won't delete those array elements.

Copying an array

The Copy method of the Array class can help us to copy a partial or full array. Let's see an example of this.

protected void Page_Load(object sender,
            EventArgs e)
{
int[] intArray = {11,3,15,66,22,554};
int[]
            intNewArray = {1, 1, 1, 1, 1, 1};

grd1.DataSource =
            intArray;
grd1.DataBind();
grd2.DataSource =
            intNewArray;
grd2.DataBind();

//Copy the
            array
Array.Copy(intArray,intNewArray,intArray.Length);

grd3.DataSource
            = intNewArray;
grd3.DataBind();
}

In the above code, we've copied all the elements from the one array to another. Here is the screen shot of the above code.

There is a restriction that we've to consider before using the Copy method. That is both source and destination arrays should be one-dimensional. There are few more remarks about how the copy will be performed when arrays are different. Please check the .NET Framework Reference for more information.

We can also copy a section of one array to another. Here is an example:

protected void Page_Load(object sender,
            EventArgs e)
{
int[] intArray = {11,3,15,66,22,554};
int[]
            intNewArray = {1, 1, 1, 1, 1, 1};

grd1.DataSource =
            intArray;
grd1.DataBind();
grd2.DataSource =
            intNewArray;
grd2.DataBind();

//Copy the
            array
Array.Copy(intArray,2,
            intNewArray,2,3);

grd3.DataSource =
            intNewArray;
grd3.DataBind();
}

In the above example, we've copied three elements from the source array to the destination array starting at the index 2. Here is screen shot of the above example.

The Array class also has an instance method called " CopyTo " which can be used from the array instance. The main difference between the Array.Copy and CopyTo method is that the Array.Copy method takes length as its parameter, and the CopyTo method takes the starting array element as its parameter. Here is an example:

protected void Page_Load(object sender, EventArgs e)
{
int[] intArray = {11,3,15,66,22,554};
int[] intNewArray = {1, 1, 1, 1, 1, 1,1,1};

grd1.DataSource = intArray;
grd1.DataBind();
grd2.DataSource = intNewArray;
grd2.DataBind();

//Copy the array using the CopyTo instance method
intArray.CopyTo(intNewArray,1);

grd3.DataSource = intNewArray;
grd3.DataBind();
}
</script>

In the above example, we've copied all the elements of the intArray to the intNewArray starting from the first index. Here is the screen shot:

The CopyTo method is more concentrated on the target of the destination array and the array will be copied at the index specified. The next major difference is that with the CopyTo method all the elements in the source array will be copied, and the Array.Copy method gives us the flexibility to specify the range.

Reverse an array

The Reverse method allows us to reverse an entire array or part of it. Here is an example.

protected void Page_Load(object sender, EventArgs e)
{
int[] intArray = {11,3,15,66,22,554};

grd1.DataSource = intArray;
grd1.DataBind();

//Reverse the array
Array.Reverse(intArray, 2, 4);

grd2.DataSource = intArray;
grd2.DataBind();
}
</script>

In the above code we've reversed 4 elements starting from index 2 in the array. Here is a screen shot of the above code:

If we want to reverse the entire array then, we can simply call the Reverse method and pass the array as a parameter to it.

'Reverse the entire array
Array.Reverse(intArray);

Cloning

The Array class also supports a Clone method, which can be used to clone the array. When we're cloning a shallow copy of the array is created. Here is a simple example:

protected void Page_Load(object sender, EventArgs e)
{
int[] intArray = {11,3,15,66,22,554};

//Clone the array
int[] intArray = (int[])Array.Clone();
}
</script>

Enumerating Array

We can enumerate an array with the GetEnumerator method. The GetEnumerator method will return an object of the IEnumerator interface defined in System.Collections namespace. The IEnumerator interface exposes MoveNext method and Current property. The MoveNext method will return true or false based on the next element of the array. The Current property will return the current value of the array element. Here is an example for this:

protected void Page_Load(object sender,
            EventArgs e)
{
int[] intArray = {11,3,15,66,22,554};
String
            Output="";

//Bind the array
grd.DataSource =
            intArray;
grd.DataBind();

// enumerate the array with
            GetLowerBound and GetUpperBound
IEnumerator intEnum =
            intArray.GetEnumerator();

while (intEnum.MoveNext())
Output = Output + intEnum.Current + ", ";

//Display the
            seach indexs
lbl.Text = "<B>Enumerated Values: </B>"
            + Output;
}
</script>

The above code will enumerate the array and list the values. Here is the screen shot:

We can also use the GetLowerBound and GetUpperBound methods of the array object to enumerate the array. Here is an example:

<script language="C#"
            runat="server">
protected void Page_Load(object sender,
            EventArgs e)
{
int[] intArray = {11,3,15,66,22,554};
String
            Output="";

//Bind the array
grd.DataSource =
            intArray;
grd.DataBind();

// enumerate the array with
            GetLowerBound and GetUpperBound
for (int
            i=intArray.GetLowerBound(0);i<=
            intArray.GetUpperBound(0);i++)
Output = Output + intArray[i] + ",
            ";

//Display the seach indexs
lbl.Text =
            "<B>Enumerated Values: </B>" + Output;
}
</script>

How many dimensions?

The Rank method of the Array class can used to find the number of dimensions the array has.

<script language="C#"
            runat="server">
protected void Page_Load(object sender,
            EventArgs e)
{
int[] intArray =
            {11,3,15,66,22,554};

//Bind the array
grd.DataSource =
            intArray;
grd.DataBind();

//Display the # of
            DIM
lbl.Text = "<B># of DIM: </B>" +
            intArray.Rank;
}
</script>

Here is the screen shot:

Exception

When manipulating arrays, it always a good idea to check for the exception " IndexOutOfRangeException " with in a try...catch block. This is the most common exception you'll experience in your code.

try
{
...
...
}
//Catch the Index Out Of Range Exception
catch(IndexOutOfRangeException Idx)
{
...
...
}
//Catch all other exceptions
catch(Exception Ex)
{
...
...
}

Other useful Methods and Properties

The Array class also exposes a few other Methods and properties. The following table shows their usage:

Property/Method | Usage
GetLength | Returns the number of elements in the given dimension.
SetValue | We can use this method to set value to an array element. For example if you want to set value to a 0th element of an three-dimensional array, Ary.SetValue(1000, 0,0,0) This equivalent to the VB.NET syntax, Ary(0,0,0) = 1000 There are three SetValue syntaxes available to handle single, double and three-dimensional arrays.
GetValue | With GetValue we can read data from an array element by specifying the array index.
Length | This property returns the total number of elements in all dimensions.

Summary

In this article we've seen the features of the System.Array class and how this class is language neutral. This class will be very useful in our day-to-day programming activities when we're dealing with Arrays.

Until Next Time, Happy Programming!

C#Today - © 2002 WroxPress Limited


Writing Unsafe Code in C#

Writing Unsafe Code in C#
http://www.csharptoday.com/content.asp?id=1608
Article Rating: 4.5
Published on July 20, 2001
Abstract
The Microsoft .NET framework introduces the notion of managed code, which is code that runs in the Common Language Runtime (CLR). The CLR tries to ensure that the code that executes is “safe? as in, no memory corruption issues, no security violations etc. While this is a big boon for most programmers who have spent countless nights chasing down a buffer overrun, it also has its disadvantages. Managed code runs in the CLR and is slower than native code. Under certain situations, developers need the ability to use pointers to write performance critical code. C# is the only language amongst all the .NET languages so far (in addition to Managed C++) to support this ability. In this article, Kaushal Sanghavi looks into C#’s support for writing unsafe code.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

The Microsoft .NET framework introduces the notion of managed code, which is code that runs in the Common Language Runtime (CLR). The CLR tries to ensure that the code that executes is "safe", i.e., no memory corruption issues and no security violations etc. While this is a big boon for most programmers who have spent countless nights chasing down a buffer overrun, it also has its disadvantages. Managed code runs in the CLR and is slower than native code. Under certain situations, developers need the ability to use pointers to write performance critical code. C# is the only language amongst all the .NET languages so far (in addition to Managed C++) to support this ability. In this article, we will look into C#'s support for writing unsafe code.

Managed, Unmanaged and Unsafe Code

Before we start discussing unsafe and unmanaged code, let us explain briefly what is meant by all these different types of code. This is a key notion that developers need to grasp, but is often found confusing.

Managed Code

This is code written by any .NET language (VB .NET, C#, JS .NET, MEC etc) that runs under the CLR. The .NET compilers generate intermediate code called Microsoft Intermediate Language (MSIL). This MSIL code is converted to native code by the CLR at runtime, and derives all the benefits of the CLR such as memory management, type checking, and exception handling etc.

Unmanaged Code

This is legacy COM or Win32 code that was written using a non-.NET compiler. This code is compiled directly into native code and cannot enjoy any benefits that the CLR offers. In fact, this code is completely unaware of the existence of the CLR.

Unmanaged code can, however, be accessed from managed code. This is made possible by the support for COM Interop by the CLR as well as the Platform Invocation Services, better known as PInvoke. Unmanaged code can only be written using a pre-.NET compiler. No .NET compiler can emit or compile unmanaged code.

Unsafe Code

This is the middle ground between managed and unmanaged code. Unsafe code can be written in a .NET language like C# and will run in the context of the CLR. However, unsafe code can directly access memory and manage memory by allocating it and pinning it to a fixed location. Managed code does not support accessing memory directly. Note, that unsafe code can be written only in a .NET language.

In spite of the code being unsafe, the CLR will ensure that all checks except memory management are in place. Thus, if you try to coerce a bad cast using unsafe code, the .NET compiler will catch it and raise a compiler error. Unsafe code is very different from unmanaged code; unsafe code is indeed managed code with a few exceptions. C# allows you to write unsafe code, but unmanaged code cannot be written in C#, or for that matter, by any .NET language.

Fixed and Unsafe Code

The CLR manages memory for you automatically. Memory is allocated from the heap and when all references to the memory are freed up, the garbage collector can reclaim the memory. Further, when the garbage collector runs, it can move chunks of memory around in order to compress all used and free memory. This is used to avoid heap fragmentation. Within safe managed code, this is not a problem because memory is never accessed directly and thus objects can be moved around in memory.

The runtime can resolve all access to this object to its new location in the heap. However, this causes an issue with pointers. Pointers contain the memory address of an object and they always expect the object to be present at that location. Suddenly, if the garbage collector moves the object from that address to another, accessing the object via the pointer can lead to a program crash because the memory contents might be unknown. Worse, it could corrupt memory such that a particular sequence of execution could cause a crash or unexpected results. This is worse, since debugging this problem could be extremely difficulty and time consuming. What we need is a way to let the garbage collector know that we are using pointers and that it should not move the contents of the memory block that is being accessed via pointers.

C# allows exactly this ability to write unsafe pointer code. The ability of C# to hold the contents of memory fixed is called pinning. When an object is pinned, it is guaranteed that the object will remain in memory while there are outstanding pointers. The C# language supports four keywords that developers can use to write unsafe code; unsafe, fixed, sizeof and stackalloc.

Unsafe

This keyword signifies an unsafe context in which pointers will be allowed. Once a block of code is marked as unsafe, the C# compiler will allow you to write code that uses pointers, dereferences pointers and uses the & operator to obtain the address of the object that you want to point to. The C# compiler will force the usage of the unsafe keyword before you can use pointers. The unsafe keyword can be applied to any block of code; this could be an entire method or property or a single line of code or a block of code. However, this keyword cannot be applied to an entire class.

Fixed

This keyword pins an object down to a fixed memory location while the fixed code block is being executed. Thus if the garbage collector were to be invoked, the pinned objects would not be relocated and the pointer values would remain the same. Managed types cannot be pinned down even with the fixed keyword. However, if the managed type has value types, you can declare pointers to these value types. Doing so requires the runtime to pin the entire managed object to a fixed location in memory. A single fixed statement can only pin objects of a certain data type, however if you need to pin objects of different data types, C# allows you to nest multiple fixed blocks, as shown below:

public unsafe void strcpy (char[] iSrc, char[] oDest)
{
int lSrcLen = iSrc.Length ;
int lDestLen = oDest.Length ;
int lCopyCount = lSrcLen > lDestLen ? lDestLen : lSrcLen;

fixed (char* pSrc = iSrc, pDest = oDest)
{
char* lSrc = pSrc;
char* lDest = pDest;

for (int i = 0; i < lCopyCount; i++)
*lDest++ = *lSrc++;
}
}

The above code block shows a very simple example of how you would use the unsafe and fixed keywords to write a C-like string copy function. The unsafe declaration for the method strcpy denotes that this method will be using pointer operations, and the fixed block for the iSrc and oDest char arrays pins these two arrays in memory while we do a copy. We use the lDest and lSrc pointers to read the value of the current character and then use the ++ operator to move on to the next character until we reach the end of the string.

Note that we can only obtain a pointer reference to the char array inside a fixed block. If we try this outside, the fixed block, the compiler will raise an error; CS0029: Cannot implicitly convert type 'char[]' to 'char*'.

public unsafe void strcpy (char[] iSrc, char[] oDest)
{
char* pSrc = iSrc; // NOT allowed
}

Further, value types can be referenced by pointers without having to pin these types. However, if a type is passed by reference to a method, then that object will have to be pinned before its address can be assigned to a pointer type. Trying to access an object passed by reference outside a fixed block raises a compiler error: CS0212: You can only take the address of unfixed expression inside of a fixed statement initializer.

public unsafe void foo (int iParam, ref int oRefParam)
{
int lLocalVar = 0;
int *p3 = &lLocalVar; // OK

int *p = &iParam; // OK
int *p2 = &oRefParam; // NOT allowed
}

A single fixed block can only pin objects of a certain type. If you need to pin objects of different types, you will have to use nested fixed blocks.

public unsafe void foo (ref int oRefParam, char[] iCharArray)
{
// NOT allowed
fixed(int *p2 = &oRefParam, char* pChar = iCharArray)
{}

fixed(int *p2 = &oRefParam)
fixed (char* pChar = iCharArray) // OK
{}
}

C# allows you to pin only unmanaged types. Managed types cannot be pinned. Attempting this will raise a compiler error : CS0208: Cannot take the address or size of a variable of a managed type.

public class ManagedType
{
public int mData;
}

public class UnsafeCode
{
public unsafe void foo ()
{
ManagedType lObj = new ManagedType ();

// NOT allowed
fixed (ManagedType* pManagedObj = &lObj)
{}
}
}

However, if in the above example, you used a struct instead of a class for defining the ManagedType, C# will allow you to obtain a pointer to an object of this type. In fact, structs are considered value types and you do not need the fixed statement to take its address. The same holds true for enumerations.

public struct StructType
{
public int mData;
}

enum EnumType
{
a, b, c
}

public class UnsafeCode
{
public unsafe void foo ()
{
StructType lObj = new StructType();
EnumType lEnumObj = new EnumType ();

StructType* pStructObj = &lObj; // OK
EnumType* pEnumObj = &lEnumObj; // OK
}
}

sizeof

This keyword is used to obtain the size of a type. This can be used only for value types and can only be used in an unsafe context. Using this operator, we can rewrite our string copy function from above as follows.

public unsafe void strcpy (char[] iSrc,
            char[] oDest)
{
int lSrcLen = iSrc.Length ;
int lDestLen =
            oDest.Length ;
int lCopyCount = lSrcLen > lDestLen ? lDestLen
            : lSrcLen;
fixed (char* pSrc = iSrc, pDest = oDest)
{
char*
            lSrc = pSrc;
char* lDest = pDest;
for (int i = 0; i <
            lCopyCount; i++){
*lDest = *lSrc;
lDest += sizeof
            (char);
lSrc += sizeof (char);
}
}
}

stackalloc

This keyword is used to allocate a block of memory on the stack. This memory can only be allocated for unmanaged pointer types, and since it is allocated on the stack, the garbage collector will not free or relocate this memory block. As a result, this memory can be accessed outside of a fixed block. However, since we are dealing with direct memory references, this keyword can only be used in an unsafe context. Once this unsafe context ends, the memory is reclaimed. Since the memory is allocated on the stack, and the stack is destroyed when the function returns, stackalloc can only be used with local variables. The compiler will raise an error if you use stackalloc to allocate memory for a member value type.

public unsafe int* Get1ThruN (int
            iCount)
{
int* pArray = stackalloc int[iCount];
int* lArr =
            pArray;

for (int i = 0; i < iCount; i++)
lArr[i] =
            i;

return lArr;
}

The above example demonstrates the use of the stackalloc keyword. We have a sample method that takes in an integer as input, and returns an array of integers from 1 to the number that was passed in. Note that since we are dealing with value types, we do not need to pin down any objects. In this case, pArray is allocated on the stack using stackalloc.

However, if you run this sample, you will realize that when the array that this method returns, does not contain the numbers 1 to N. In fact, if you step through the returned array in the debugger, you will realize that the returned array contains only 0s. How is this possible?

As we discussed above, stackalloc uses the stack to allocate memory. When the function returns, this memory is destroyed and the stack is popped empty. Thus, all our references to objects that were allocated using stackalloc are now invalid. This is a key point to note; never return an object that is allocated using stackalloc. This object is valid only for the life of the function in which it is allocated; its value is not preserved across function calls. Thus, while the above sample demonstrates how you might use the stackalloc, its usage is flawed. The right way to implement this function is shown below.

public unsafe int* Get1ThruN (int iCount)
{
int* lArr;

int[] pArray = new int [iCount];
fixed (int* lTempArr = pArray)
lArr = lTempArr;

for (int i = 0; i < iCount; i++)
lArr[i] = i;

return lArr;
}

In the code above, the memory is now allocated using the new operator that uses the heap. The heap is accessible across function calls, and now you will see that the caller method does indeed get an array of integers from 1 to N.

Using unsafe code between C# and VB.NET

C# is the only .NET language that currently supports unsafe code. Other .NET languages only generate safe managed code. However, it is possible that certain components are developed in C# and certain components are developed in ASP.NET or VB.NET and that these components have to interact. By default, other .NET languages can access unsafe C# code as long as all the elements of the unsafe code are encapsulated within the C# code methods. Thus, if you have a C# method that takes in standard managed types as arguments and returns a managed type, but internally uses pointer manipulation, then this method can be accessed by all .NET languages. However, if the C# method has a single pointer argument, or returns a pointer value, this method can only be accessed from C# clients. Since other managed languages do not support pointers, their compilers will not be able to resolve any reference to a pointer value and as a result, any unsafe method that uses pointers as arguments or return types is ignored by the compiler. (All of my tests have been with VB.NET; this behavior may differ slightly for compilers of other managed languages).

public class UnsafeCode
{
public unsafe int PointerArgumentMethod (int* i)
{
return 1;
}

public unsafe int* PointerReturnedMethod ()
{
return null;
}
}

If you have a class as defined in the code above, none of the methods can be accessed from VB.NET. The following screenshot shows the assembly produced by the above code.

Figure 1 - Unsafe Code Class

Now let us try to write a VB .NET client that can access this class:

Protected Sub Button1_Click(ByVal sender As Object, _
ByVal e As System.EventArgs)

Dim lUnsafeObject As New UnsafeCode()

lUnsafeObject.PointerReturnedMethod()
lUnsafeObject.PointerArgumentMethod(Nothing)

Return
End Sub

Although the VB.NET client can access the unsafe object's public methods, the VB.NET compiler cannot generate code to invoke these methods. VB .NET seems to choke on a method that takes pointer arguments, i.e. the PointerArgumentMethod, or any methods that return a pointer, i.e. PointerReturnedMethod. If you try to invoke any of the methods with pointer types, the VB .NET compiler will throw a compiler error as shown below.

The return type or parameter types of PointerReturnedMethod contain a type
that is not supported.
The return type or parameter types of PointerArgumentMethod contain a type
that is not supported.

As a rule of thumb, if you want clients written in other languages to access your object's public methods, do not use pointers as arguments or return types. You can use unsafe code within the method, but try to restrict the function interface to managed types.

Attached Sample Code

Now that we have discussed the concepts behind writing unsafe code in C# and using it from non-C# clients, let us briefly explain the attached sample code which demonstrates these concepts. The sample code consists of the following three VS .NET projects.

  • C#UnsafeCode: This project contains the UnsafeCode class that contains unsafe code and has methods with pointer types.
  • CSharpManagedClient: This project contains a C# managed client executable that calls methods on the UnsafeCode object.
  • VBManagedClient: This project contains a VB .NET managed client executable that calls methods on the UnsafeCode object. The difference between this and the C# client is that the VB .NET client can only call methods that don't contain pointer types.

The above screenshot displays the C# managed client. The first function that it tests is the string copy function that uses pointers. The Copy button calls the unsafe method strcpy on the UnsafeObject. The next two buttons, Buggy Gen and Working Gen demonstrate the usage of the stackalloc keyword to allocate memory. As we discussed earlier in the article, memory allocated by stackalloc is valid only for the life of the method in which it was allocated. To prove this point, we have two versions of a method that generates numbers from 1 to a specified number serially. The Buggy Gen button calls a method that returns memory allocated using stackalloc. This is obviously incorrect, and has weird results as you can see from the above screenshot. The Working Gen on the other hand, uses the new keyword to generate memory and thus works fine as shown below.

The VB .NET managed client is fairly simple. Since the only method it can invoke is the unsafe strcpy method, we simply demonstrate this method by copying a source string to a destination string. Any attempt to call the other methods will result in compiler errors as we discussed in the earlier section.

Conclusion

The .NET framework provides an excellent opportunity for developers for rapid development by providing support for common features in the runtime itself. Memory management is one such feature that has plagued developers over the years now. However, when used carefully and with discretion, pointers and direct memory access can provide a significant boost to application performance. C# provides the ability to do just this, and this might very well be the distinguishing factor for developers when having to choose between development in C# or VB.NET or, for that matter, any other .NET language.

Happy .NETing.

Links

http://msdn.microsoft.com/vstudio/nextgen/technology/csharpintro.asp?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq

http://msdn.microsoft.com/theshow/Episode008/default.asp?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq

http://msdn.microsoft.com/library/default.asp?URL=/library/dotnet/csspec/vclrfcsharpspec_Start.htm&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq

http://msdn.microsoft.com/library/default.asp?URL=/library/dotnet/csspec/vclrfcsharpspec_a_8.htm&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq

http://msdn.microsoft.com/library/default.asp?URL=/library/dotnet/csspec/vclrfcsharpspec_a.htm&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq

http://msdn.microsoft.com/library/default.asp?URL=/library/dotnet/cpref/frlrfSystemMissingMethodExceptionClassTopic.htm&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq

Introduction

The Microsoft .NET framework introduces the notion of managed code, which is code that runs in the Common Language Runtime (CLR). The CLR tries to ensure that the code that executes is "safe", i.e., no memory corruption issues and no security violations etc. While this is a big boon for most programmers who have spent countless nights chasing down a buffer overrun, it also has its disadvantages. Managed code runs in the CLR and is slower than native code. Under certain situations, developers need the ability to use pointers to write performance critical code. C# is the only language amongst all the .NET languages so far (in addition to Managed C++) to support this ability. In this article, we will look into C#'s support for writing unsafe code.

Managed, Unmanaged and Unsafe Code

Before we start discussing unsafe and unmanaged code, let us explain briefly what is meant by all these different types of code. This is a key notion that developers need to grasp, but is often found confusing.

Managed Code

This is code written by any .NET language (VB .NET, C#, JS .NET, MEC etc) that runs under the CLR. The .NET compilers generate intermediate code called Microsoft Intermediate Language (MSIL). This MSIL code is converted to native code by the CLR at runtime, and derives all the benefits of the CLR such as memory management, type checking, and exception handling etc.

Unmanaged Code

This is legacy COM or Win32 code that was written using a non-.NET compiler. This code is compiled directly into native code and cannot enjoy any benefits that the CLR offers. In fact, this code is completely unaware of the existence of the CLR.

Unmanaged code can, however, be accessed from managed code. This is made possible by the support for COM Interop by the CLR as well as the Platform Invocation Services, better known as PInvoke. Unmanaged code can only be written using a pre-.NET compiler. No .NET compiler can emit or compile unmanaged code.

Unsafe Code

This is the middle ground between managed and unmanaged code. Unsafe code can be written in a .NET language like C# and will run in the context of the CLR. However, unsafe code can directly access memory and manage memory by allocating it and pinning it to a fixed location. Managed code does not support accessing memory directly. Note, that unsafe code can be written only in a .NET language.

In spite of the code being unsafe, the CLR will ensure that all checks except memory management are in place. Thus, if you try to coerce a bad cast using unsafe code, the .NET compiler will catch it and raise a compiler error. Unsafe code is very different from unmanaged code; unsafe code is indeed managed code with a few exceptions. C# allows you to write unsafe code, but unmanaged code cannot be written in C#, or for that matter, by any .NET language.

Fixed and Unsafe Code

The CLR manages memory for you automatically. Memory is allocated from the heap and when all references to the memory are freed up, the garbage collector can reclaim the memory. Further, when the garbage collector runs, it can move chunks of memory around in order to compress all used and free memory. This is used to avoid heap fragmentation. Within safe managed code, this is not a problem because memory is never accessed directly and thus objects can be moved around in memory.

The runtime can resolve all access to this object to its new location in the heap. However, this causes an issue with pointers. Pointers contain the memory address of an object and they always expect the object to be present at that location. Suddenly, if the garbage collector moves the object from that address to another, accessing the object via the pointer can lead to a program crash because the memory contents might be unknown. Worse, it could corrupt memory such that a particular sequence of execution could cause a crash or unexpected results. This is worse, since debugging this problem could be extremely difficulty and time consuming. What we need is a way to let the garbage collector know that we are using pointers and that it should not move the contents of the memory block that is being accessed via pointers.

C# allows exactly this ability to write unsafe pointer code. The ability of C# to hold the contents of memory fixed is called pinning. When an object is pinned, it is guaranteed that the object will remain in memory while there are outstanding pointers. The C# language supports four keywords that developers can use to write unsafe code; unsafe, fixed, sizeof and stackalloc.

Unsafe

This keyword signifies an unsafe context in which pointers will be allowed. Once a block of code is marked as unsafe, the C# compiler will allow you to write code that uses pointers, dereferences pointers and uses the & operator to obtain the address of the object that you want to point to. The C# compiler will force the usage of the unsafe keyword before you can use pointers. The unsafe keyword can be applied to any block of code; this could be an entire method or property or a single line of code or a block of code. However, this keyword cannot be applied to an entire class.

Fixed

This keyword pins an object down to a fixed memory location while the fixed code block is being executed. Thus if the garbage collector were to be invoked, the pinned objects would not be relocated and the pointer values would remain the same. Managed types cannot be pinned down even with the fixed keyword. However, if the managed type has value types, you can declare pointers to these value types. Doing so requires the runtime to pin the entire managed object to a fixed location in memory. A single fixed statement can only pin objects of a certain data type, however if you need to pin objects of different data types, C# allows you to nest multiple fixed blocks, as shown below:

public unsafe void strcpy (char[] iSrc, char[] oDest)
{
int lSrcLen = iSrc.Length ;
int lDestLen = oDest.Length ;
int lCopyCount = lSrcLen > lDestLen ? lDestLen : lSrcLen;

fixed (char* pSrc = iSrc, pDest = oDest)
{
char* lSrc = pSrc;
char* lDest = pDest;

for (int i = 0; i < lCopyCount; i++)
*lDest++ = *lSrc++;
}
}

The above code block shows a very simple example of how you would use the unsafe and fixed keywords to write a C-like string copy function. The unsafe declaration for the method strcpy denotes that this method will be using pointer operations, and the fixed block for the iSrc and oDest char arrays pins these two arrays in memory while we do a copy. We use the lDest and lSrc pointers to read the value of the current character and then use the ++ operator to move on to the next character until we reach the end of the string.

Note that we can only obtain a pointer reference to the char array inside a fixed block. If we try this outside, the fixed block, the compiler will raise an error; CS0029: Cannot implicitly convert type 'char[]' to 'char*'.

public unsafe void strcpy (char[] iSrc, char[] oDest)
{
char* pSrc = iSrc; // NOT allowed
}

Further, value types can be referenced by pointers without having to pin these types. However, if a type is passed by reference to a method, then that object will have to be pinned before its address can be assigned to a pointer type. Trying to access an object passed by reference outside a fixed block raises a compiler error: CS0212: You can only take the address of unfixed expression inside of a fixed statement initializer.

public unsafe void foo (int iParam, ref int oRefParam)
{
int lLocalVar = 0;
int *p3 = &lLocalVar; // OK

int *p = &iParam; // OK
int *p2 = &oRefParam; // NOT allowed
}

A single fixed block can only pin objects of a certain type. If you need to pin objects of different types, you will have to use nested fixed blocks.

public unsafe void foo (ref int oRefParam, char[] iCharArray)
{
// NOT allowed
fixed(int *p2 = &oRefParam, char* pChar = iCharArray)
{}

fixed(int *p2 = &oRefParam)
fixed (char* pChar = iCharArray) // OK
{}
}

C# allows you to pin only unmanaged types. Managed types cannot be pinned. Attempting this will raise a compiler error : CS0208: Cannot take the address or size of a variable of a managed type.

public class ManagedType
{
public int mData;
}

public class UnsafeCode
{
public unsafe void foo ()
{
ManagedType lObj = new ManagedType ();

// NOT allowed
fixed (ManagedType* pManagedObj = &lObj)
{}
}
}

However, if in the above example, you used a struct instead of a class for defining the ManagedType, C# will allow you to obtain a pointer to an object of this type. In fact, structs are considered value types and you do not need the fixed statement to take its address. The same holds true for enumerations.

public struct StructType
{
public int mData;
}

enum EnumType
{
a, b, c
}

public class UnsafeCode
{
public unsafe void foo ()
{
StructType lObj = new StructType();
EnumType lEnumObj = new EnumType ();

StructType* pStructObj = &lObj; // OK
EnumType* pEnumObj = &lEnumObj; // OK
}
}

sizeof

This keyword is used to obtain the size of a type. This can be used only for value types and can only be used in an unsafe context. Using this operator, we can rewrite our string copy function from above as follows.

public unsafe void strcpy (char[] iSrc,
            char[] oDest)
{
int lSrcLen = iSrc.Length ;
int lDestLen =
            oDest.Length ;
int lCopyCount = lSrcLen > lDestLen ? lDestLen
            : lSrcLen;
fixed (char* pSrc = iSrc, pDest = oDest)
{
char*
            lSrc = pSrc;
char* lDest = pDest;
for (int i = 0; i <
            lCopyCount; i++){
*lDest = *lSrc;
lDest += sizeof
            (char);
lSrc += sizeof (char);
}
}
}

stackalloc

This keyword is used to allocate a block of memory on the stack. This memory can only be allocated for unmanaged pointer types, and since it is allocated on the stack, the garbage collector will not free or relocate this memory block. As a result, this memory can be accessed outside of a fixed block. However, since we are dealing with direct memory references, this keyword can only be used in an unsafe context. Once this unsafe context ends, the memory is reclaimed. Since the memory is allocated on the stack, and the stack is destroyed when the function returns, stackalloc can only be used with local variables. The compiler will raise an error if you use stackalloc to allocate memory for a member value type.

public unsafe int* Get1ThruN (int
            iCount)
{
int* pArray = stackalloc int[iCount];
int* lArr =
            pArray;

for (int i = 0; i < iCount; i++)
lArr[i] =
            i;

return lArr;
}

The above example demonstrates the use of the stackalloc keyword. We have a sample method that takes in an integer as input, and returns an array of integers from 1 to the number that was passed in. Note that since we are dealing with value types, we do not need to pin down any objects. In this case, pArray is allocated on the stack using stackalloc.

However, if you run this sample, you will realize that when the array that this method returns, does not contain the numbers 1 to N. In fact, if you step through the returned array in the debugger, you will realize that the returned array contains only 0s. How is this possible?

As we discussed above, stackalloc uses the stack to allocate memory. When the function returns, this memory is destroyed and the stack is popped empty. Thus, all our references to objects that were allocated using stackalloc are now invalid. This is a key point to note; never return an object that is allocated using stackalloc. This object is valid only for the life of the function in which it is allocated; its value is not preserved across function calls. Thus, while the above sample demonstrates how you might use the stackalloc, its usage is flawed. The right way to implement this function is shown below.

public unsafe int* Get1ThruN (int iCount)
{
int* lArr;

int[] pArray = new int [iCount];
fixed (int* lTempArr = pArray)
lArr = lTempArr;

for (int i = 0; i < iCount; i++)
lArr[i] = i;

return lArr;
}

In the code above, the memory is now allocated using the new operator that uses the heap. The heap is accessible across function calls, and now you will see that the caller method does indeed get an array of integers from 1 to N.

Using unsafe code between C# and VB.NET

C# is the only .NET language that currently supports unsafe code. Other .NET languages only generate safe managed code. However, it is possible that certain components are developed in C# and certain components are developed in ASP.NET or VB.NET and that these components have to interact. By default, other .NET languages can access unsafe C# code as long as all the elements of the unsafe code are encapsulated within the C# code methods. Thus, if you have a C# method that takes in standard managed types as arguments and returns a managed type, but internally uses pointer manipulation, then this method can be accessed by all .NET languages. However, if the C# method has a single pointer argument, or returns a pointer value, this method can only be accessed from C# clients. Since other managed languages do not support pointers, their compilers will not be able to resolve any reference to a pointer value and as a result, any unsafe method that uses pointers as arguments or return types is ignored by the compiler. (All of my tests have been with VB.NET; this behavior may differ slightly for compilers of other managed languages).

public class UnsafeCode
{
public unsafe int PointerArgumentMethod (int* i)
{
return 1;
}

public unsafe int* PointerReturnedMethod ()
{
return null;
}
}

If you have a class as defined in the code above, none of the methods can be accessed from VB.NET. The following screenshot shows the assembly produced by the above code.

Figure 1 - Unsafe Code Class

Now let us try to write a VB .NET client that can access this class:

Protected Sub Button1_Click(ByVal sender As Object, _
ByVal e As System.EventArgs)

Dim lUnsafeObject As New UnsafeCode()

lUnsafeObject.PointerReturnedMethod()
lUnsafeObject.PointerArgumentMethod(Nothing)

Return
End Sub

Although the VB.NET client can access the unsafe object's public methods, the VB.NET compiler cannot generate code to invoke these methods. VB .NET seems to choke on a method that takes pointer arguments, i.e. the PointerArgumentMethod, or any methods that return a pointer, i.e. PointerReturnedMethod. If you try to invoke any of the methods with pointer types, the VB .NET compiler will throw a compiler error as shown below.

The return type or parameter types of PointerReturnedMethod contain a type
that is not supported.
The return type or parameter types of PointerArgumentMethod contain a type
that is not supported.

As a rule of thumb, if you want clients written in other languages to access your object's public methods, do not use pointers as arguments or return types. You can use unsafe code within the method, but try to restrict the function interface to managed types.

Attached Sample Code

Now that we have discussed the concepts behind writing unsafe code in C# and using it from non-C# clients, let us briefly explain the attached sample code which demonstrates these concepts. The sample code consists of the following three VS .NET projects.

  • C#UnsafeCode: This project contains the UnsafeCode class that contains unsafe code and has methods with pointer types.
  • CSharpManagedClient: This project contains a C# managed client executable that calls methods on the UnsafeCode object.
  • VBManagedClient: This project contains a VB .NET managed client executable that calls methods on the UnsafeCode object. The difference between this and the C# client is that the VB .NET client can only call methods that don't contain pointer types.

The above screenshot displays the C# managed client. The first function that it tests is the string copy function that uses pointers. The Copy button calls the unsafe method strcpy on the UnsafeObject. The next two buttons, Buggy Gen and Working Gen demonstrate the usage of the stackalloc keyword to allocate memory. As we discussed earlier in the article, memory allocated by stackalloc is valid only for the life of the method in which it was allocated. To prove this point, we have two versions of a method that generates numbers from 1 to a specified number serially. The Buggy Gen button calls a method that returns memory allocated using stackalloc. This is obviously incorrect, and has weird results as you can see from the above screenshot. The Working Gen on the other hand, uses the new keyword to generate memory and thus works fine as shown below.

The VB .NET managed client is fairly simple. Since the only method it can invoke is the unsafe strcpy method, we simply demonstrate this method by copying a source string to a destination string. Any attempt to call the other methods will result in compiler errors as we discussed in the earlier section.

Conclusion

The .NET framework provides an excellent opportunity for developers for rapid development by providing support for common features in the runtime itself. Memory management is one such feature that has plagued developers over the years now. However, when used carefully and with discretion, pointers and direct memory access can provide a significant boost to application performance. C# provides the ability to do just this, and this might very well be the distinguishing factor for developers when having to choose between development in C# or VB.NET or, for that matter, any other .NET language.

Happy .NETing.

Links

http://msdn.microsoft.com/vstudio/nextgen/technology/csharpintro.asp?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq

http://msdn.microsoft.com/theshow/Episode008/default.asp?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq

http://msdn.microsoft.com/library/default.asp?URL=/library/dotnet/csspec/vclrfcsharpspec_Start.htm&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq

http://msdn.microsoft.com/library/default.asp?URL=/library/dotnet/csspec/vclrfcsharpspec_a_8.htm&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq

http://msdn.microsoft.com/library/default.asp?URL=/library/dotnet/csspec/vclrfcsharpspec_a.htm&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq

http://msdn.microsoft.com/library/default.asp?URL=/library/dotnet/cpref/frlrfSystemMissingMethodExceptionClassTopic.htm&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq

C#Today - © 2002 WroxPress Limited


Polymorphism in C# - Creating a Drawing Program

Polymorphism in C# - Creating a Drawing Program
http://www.csharptoday.com/content.asp?id=1606
Article Rating: 3.5
Published on July 19, 2001
Abstract
Polymorphism is a very broad term and has several meanings. However, in the object–oriented programming world, when people talk about polymorphism, they generally mean the concept of one interface with many implementations. To be more specific, the term is used to describe reference variables that may at run–time refer to objects of different classes. In this article Budi Kurniawan shows us what all the buzzwords are, why polymorphism is useful, and how to write a drawing application that illustrates the use of polymorphism.
Article

Polymorphism is a very broad term and has several meanings. However, in the object-oriented programming world, when people talk about polymorphism, they generally mean the concept of one interface with many implementations. To be more specific, the term is used to describe reference variables that may at run-time refer to objects of different classes. In this article you will learn what all the buzzwords are, why polymorphism is useful, and how to write a drawing application that illustrates the use of polymorphism.

Polymorphism is one of the three most important principles in object-oriented programming, the other two being encapsulation and inheritance. For those new to object-oriented programming, polymorphism is unfortunately not an easy concept. Therefore, to truly understand polymorphism, you need to know the different types of polymorphism and which type is normally referred to when this word is mentioned in object-oriented programming discussions or in C# language books.

There are two broad types of polymorphism: universal polymorphism and ad-hoc polymorphism.

  • Universal polymorphism is considered purer than ad-hoc polymorphism and is also called "true polymorphism". It allows a value to have not only many different types, but also an infinite number of them.
  • Ad-hoc polymorphism is less pure than universal polymorphism and can be regarded as "apparent polymorphism". It allows a value to have only a finite set of different types, not an infinite number.

Universal polymorphism is subdivided into parametric polymorphism and inclusion polymorphism; and ad-hoc polymorphism can be subdivided into overloading and coercion. The four types of polymorphism are explained below:

  • parametric polymorphism: this is the purest form of polymorphism, and the one most people mean in an object-oriented context. The same object or function can be used uniformly as a parameter in different type contexts without changes. A function can be thought as parametric polymorphic if it allows the type of its parameters to be determined by an implicit or explicit type parameter, and then performs the same kind of work irrespective of the parameter type.
  • inclusion polymorphism is what we know as subtyping and inheritance. This type of polymorphism allows an object to belong to a number of classes, which are sometimes overlapping. In other words, a class can be included in another class to create an inheritance hierarchy.
  • overloading is what is known as method overloading in object-oriented programming languages. It allows multiple functions to have the same name, yet still perform different functions.
  • coercion allows a parameter to a function to be converted to the type expected by that function to avoid a type error.

For those of you who like to dig deeper, you can start with the following excellent article at http://www.cs.ucl.ac.uk/staff/L.Braine/polymorphism.html

Polymorphism in C#

In object-oriented programming in general, when people talk about polymorphism, it is parametric polymorphism that they mean. In the sense of "one interface, many implementations", polymorphism has various applications. In this article, you will find two. The first application is a simple code that will introduce polymorphism. The second is a Windows drawing application.

Simple Example of Polymorphism

Suppose you are writing a word processing program that has a Print function. In the real world, there are many types of printers from many different manufacturers. How do you write the Print function so that it can print to many different printers? For this case, let's pretend that the current sophisticated operating systems do not yet exist. Without polymorphism, you will have to write the implementation code for each printer. As a new printer arrives and needs to be supported, you have to re-compile your word processing program again to include function that can print to the new printer. What an endless pain!

With polymorphism, however, you can just write an interface called IPrinter with a few methods. Each printer can then implement its own class that implements IPrinter. When a new printer comes, all you need to do is provide a class for the new printer that implements the same interface IPrinter. In fact, this interface acts as a contract between your word processor and any Print class implementation. To prove this concept, consider the code in Listing 1.

Listing 1: A simple polymorphism example in C#

using System;
using
            System.Text;

interface IPrinter {
void Print(String
            s);
}

class CannonPrinter: IPrinter {
public void
            Print(String s) {
Console.WriteLine("Printed by Cannon:" +
            s);
}
}

class EpsomPrinter: IPrinter {
public void
            Print(String s) {
Console.WriteLine("Printed by Epsom:" +
            s);
}
}

public class WordProcessor {
public static
            void Main(String[] args) {
if (args.Length==0)
            {
Console.WriteLine("Usage: WordProcessor <printer
            name>");
return;
}

String message = "Using
            polymorphism in C#";
IPrinter p = null;

if
            (args[0].Equals("Cannon")) {
p = new
            CannonPrinter();
}
else if (args[0].Equals("Epsom")) {
p =
            new EpsomPrinter();
}

if (p != null)
p.Print(message);
}
}

The IPrinter interface in Listing 1 has only one method: print. This method accepts the message to be printed by the printer. The IPrinter interface is the contract between the WordProcessor class (the application) and the printer manufacturers. Any printer manufacturer who wants its printer to be useable from WordProcessor must implement this interface. In the example, two printers, CannonPrinter and EpsomPrinter, implement IPrinter. Cannon writes the CannonPrinter class that implements IPrinter, and Epsom writes the EpsomPrinter class implementing IPrinter. Both classes provide implementations of the Print method. For the sake of simplicity, the Print method simply outputs the message to the console, preceded by the printer name.

To use a printer from WordProcessor, you pass the printer name as an argument to the Main method. If the argument is "Cannon", p will be instantiated with a CannonPrinter object. If the argument is "Epsom", p will be instantiated with an EpsomPrinter object. The variable p is actually a reference of type IPrinter. See how p can be assigned other types of objects? It could be a CannonPrinter or it could be an EpsomPrinter. In essence, it could be any object that implements IPrinter. You have just seen polymorphism in action!

This phenomenon is not illogical at all. Both CannonPrinter and EpsomPrinter implements IPrinter. You can think of implementing an interface kind of extending a class. So, here we have an "is a" relationship between CannonPrinter and EpsomPrinter and IPrinter. In other words, CannonPrinter "is an" IPrinter. EpsomPrinter is also an IPrinter. Because of this, it is logical to "put" either CannonPrinter or EpsomPrinter into p, because both CannonPrinter and EpsomPrinter are IPrinter. This is an example of one interface with many implementations.

The Shape Draw Application

The following example is a simple C# Windows application whose only purpose in life is to demonstrate polymorphism. The application enables you to draw multiple shapes by entering the two coordinates and selecting the type of shape you want. The EXE file of this program is included in the download. To run the source code of this application, you need Visual Studio.NET beta, that you can download or order from http://msdn.microsoft.com/vstudio/nextgen/beta.asp?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq. There are three types of shapes: Line, Rectangle and Ellipse. Every shape is encapsulated in a class and each shape implements the IShape interface.

The code for the interface and the shape classes is given in Listing 2.

Listing 2: The IShape Interface and the three classes that implement it: Line, Rectangle and Ellipse

interface IShape
{
void draw(Graphics
            g);
}

class Line: IShape
{
private int x1, x2, y1,
            y2;
public Line(int x1, int y1, int x2, int y2)
{
this.x1 =
            x1;
this.y1 = y1;
this.x2 = x2;
this.y2 =
            y2;
}

public void draw(System.Drawing.Graphics g)
{
g.DrawLine(new Pen(Color.Brown), x1, y1, x2, y2);
}
}

class Rectangle: IShape
{
private int x1,
            x2, y1, y2;
public Rectangle(int x1, int y1, int x2, int
            y2)
{
this.x1 = x1;
this.y1 = y1;
this.x2 =
            x2;
this.y2 = y2;
}

public void
            draw(System.Drawing.Graphics g)
{
g.DrawRectangle(new
            Pen(Color.Green), x1, y1, (x2 - x1), (y2 -
            y1));
}
}

class Ellipse: IShape
{
private int x1,
            x2, y1, y2;
public Ellipse(int x1, int y1, int x2, int
            y2)
{
this.x1 = x1;
this.y1 = y1;
this.x2 =
            x2;
this.y2 = y2;
}

public void
            draw(System.Drawing.Graphics g)
{
g.DrawEllipse(new
            Pen(Color.Red), x1, y1, (x2 - x1), (y2 - y1));
}
}

The IShape interface has one method: draw. Implementation for this method will be provided by any class that implements IShape. The draw method accepts a System.Drawing.Graphics object on which the shape will be drawn. To draw a shape, two coordinates are required: (x1, y1) and (x2, y2). These coordinates must be available internally in the implementing class and the shape drawn will be determined by these coordinates.

The Line class has one constructor. The constructor accepts four integer values: x1, y1, x2, y2. As you can guess, these values are used as the coordinates for the end points of the line. Implementation of the draw method is straightforward using the DrawLine method of the Graphics object. The DrawLine method overload used in the Line class accepts five arguments: a Pen object, and the four integers x1, y1, x2 and y2. I know you must be wondering how you can get the Graphics and Pen objects for the draw method. However, please bear with me. Now I will focus on the interface and the classes. The Pen object will be explained further in the section to come: "Creating the Graphics and the Pen objects".

The Rectangle and Ellipse classes are also very simple. Both have a constructor that accepts four arguments: x1, y1, x2, y2. These four integers are used as the coordinates. For Rectangle, these two coordinates are used as top left coordinate and right bottom coordinate. For Ellipse, these are the same coordinates for the smallest rectangle that you need to draw to cover the entire ellipse.

The DrawRectangle method of the Graphics class is used to draw a rectangle. Like DrawLine, this method overload accepts five arguments: the Pen object, the x coordinate of the top left corner, the y coordinate of the top left corner, the width, and the height.

In Ellipse, to draw we use the DrawEllipse method of the Graphics object. How it works is very similar to the DrawRectangle method.

Creating the Graphics and the Pen objects

Of course a drawing program needs a canvas to draw on. The System.Drawing namespace provides the Graphics class with a large number of drawing methods readily available once we get a Graphics object. However, to instantiate the Graphics object we need a container. This is where the RichControl class steps in. This class, included in the System.WinForms namespace, is the parent class for all controls. And RichControl is appropriate for our program because of two factors. First, RichControl, like any other control, can be added easily to a Form. Secondly, the RichControl class has a method named CreateGraphics that returns its Graphics object. Using this method, you can then easily get a Graphics object that you can use as your canvas, as follows:

Graphics g = rc.CreateGraphics();

where rc is the RichControl object that is added to the form. Having the Graphics object g, you can then use any of its methods to draw many different shapes, including lines, rectangle and ellipses.

However, one thing is still missing. You need a Pen to draw! Pen is part of the System.Drawing namespace and specifies the width and style of any shape you draw. For example, its DashStyle property specifies several varieties of dashed lines. For this program, however, we will not delve too deeply into this class. We just need to create a Pen object quickly for the DrawLine, DrawRectangle and DrawEllipse methods. To create a Pen, you need to use one of its constructors, one of which accepts one parameter: a Color structure. Fortunately, a Color structure (also part of the System.Drawing namespace) is very easy to use because it has a number of static properties. For instance, to get a black color, you just need to specify the following statement: Color.Black. In the same token, to get a red you specify Color.Red.

To get a red Pen, you just need to use the new keyword with one of the Pen class's constructors:

new Pen(Color.Red)

Drawing the Shapes

The drawing program as seen above has one RichControl control, three buttons to select a shape, four labels and four text boxes. These text boxes are important because you type the coordinates here. When you click one of the buttons, a method named DrawShape will be called, as follows:

protected void button1_Click (object sender, System.EventArgs e)
{
DrawShape("Line", (object) txt_x1.Value, (object) txt_y1.Value,
(object) txt_x2.Value, (object) txt_y2.Value);
}
protected void button2_Click (object sender, System.EventArgs e)
{
DrawShape("Rectangle", (object) txt_x1.Value, (object) txt_y1.Value,
(object) txt_x2.Value, (object) txt_y2.Value);
}
protected void button3_Click (object sender, System.EventArgs e)
{
DrawShape("Ellipse", (object) txt_x1.Value, (object) txt_y1.Value,
(object) txt_x2.Value, (object) txt_y2.Value);
}

The DrawShape method is given as follows:

void DrawShape(String str, object sx1,
            object sy1, object sx2, object sy2)
{
try
{
int x1 =
            Int32.FromString((string) sx1);
int y1 =
            Int32.FromString((string) sy1);
int x2 =
            Int32.FromString((string) sx2);
int y2 =
            Int32.FromString((string) sy2);
Graphics g =
            rc.CreateGraphics();
IShape shape = null;
if
            (str.Equals("Line"))
{
shape = new Line(x1, y1, x2, y2);
}
else if (str.Equals("Rectangle"))
{
shape = new
            Rectangle(x1, y1, x2, y2);
}
else if
            (str.Equals("Ellipse"))
{
shape = new Ellipse(x1, y1, x2,
            y2);
}
shape.draw(g);
}
catch (InvalidCastException
            e)
{
Console.WriteLine(e.ToString());
return;
}
catch
            (ArgumentNullException
            e)
{
Console.WriteLine(e.ToString());
return;
}
}

The DrawShape method first converts the value of the text boxes to integers.

int x1 = Int32.FromString((string) sx1);
int y1 = Int32.FromString((string) sy1);
int x2 = Int32.FromString((string) sx2);
int y2 = Int32.FromString((string) sy2);

Conversion errors will be caught by InvalidCastException. This happens if you try to put non-numeric value in one of the text boxes. Also, if you click the button before entering values to the four text boxes, an ArgumentNullException will be thrown.

The most interesting part of the DrawShape method is given below:

  IShape shape = null;
if
            (str.Equals("Line"))
{
shape = new Line(x1, y1, x2, y2);
}
else if (str.Equals("Rectangle"))
{
shape = new
            Rectangle(x1, y1, x2, y2);
}
else if
            (str.Equals("Ellipse"))
{
shape = new Ellipse(x1, y1, x2,
            y2);
}
shape.draw(g);

This part is where polymorphism happens. shape is a reference variable for IShape, however, depending on the value of the first argument to DrawShape (either "Line", "Rectangle" or "Ellipse"), shape will reference a Line object or a Rectangle object or an Ellipse object. Another instance of polymorphism in action.

What is more amazing is when the draw method is invoked - it knows which shape to be drawn!

Summary

Polymorphism is one of the three main principles of object-oriented programming. This article discussed polymorphism in C# with two examples. The first is a very simple example to get you familiar with polymorphism. The second example is a drawing application that demonstrates polymorphism in action and shows how polymorphism can be useful not only in theory but also in practice.

C#Today - © 2002 WroxPress Limited


Programmatically Wiring Up Events in C#

Programmatically Wiring Up Events in C#
http://www.csharptoday.com/content.asp?id=1601
Article Rating: 3.5
Published on July 17, 2001
Abstract
One of the most powerful features of .NET is the ability to create dynamic forms for Windows Applications. Although possible before, it’s now very easy to achieve in .NET. This enables us to obtain objects at runtime derived from System.Windows.Forms.Control and use them in exactly the same manner as if we’d painted them on with the forms designer at design time. Dynamic controls can be used to adjust your application’s user interface depending on certain environmental information - for example a database administration tool, where a button can be dynamically added to the tool for each table detected within a database. The slightly tricky aspect of dynamic control creation is wiring up event handlers to the controls. In this article, Matthew Reynolds shows us how to dynamically create controls in C# and wire up methods to the control’s events. This article is available as an $8 e-doc at http://www.amazon.com/exec/obidos/ASIN/B00005T5SP/ref=nosim/wrox-20.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

One of the most powerful features of .NET is the ability to create dynamic forms for Windows Applications. Although this was possible before, it's now very easy to achieve in .NET. These enable us to obtain objects at runtime derived from System.Windows.Forms.Control and use them in exactly the same manner as if we'd painted them on with the forms designer at design time.

The slightly tricky aspect of dynamic control creation is wiring up event handlers to the controls. In this article, we'll be looking at how to dynamically create controls in C# and wire up methods to the control's events.

Creating the Project

Create a new Visual C# - Windows Application project now and call it DynamicButtons.

We need to create a basic form layout that looks like this:

You'll need to call the button cmdCreateButtons and the text box should be called txtLog. Make sure that txtLog has its Multiline property set to True and that ScrollBars is set to Vertical.

When cmdCreateButtons is clicked, we want to add six buttons to the form positioned in the blank space to the right of txtLog. When we configure the event handlers for these new buttons, we need to know which button originated the call. To do this, we need each of the buttons to have an additional integer property called ID that we'll number 1 through 6 as we create them.

One feature of .NET that VB developers need to get used to is the idea of inheriting existing classes from the Framework and extending them. This technique is extremely powerful. In our case, what we want to do is create a new class called DynamicButton and inherit it from System.Windows.Forms.Button. This means that our new class will have all the functionality of a normal Button control, but it will also have the other stuff that we need, specifically a new property called DynamicId. As our class is derived from Button, it can do anything that a button control should be able to do, i.e. it responds to user clicks, it can be placed on a form, and so on.

Moreover, a really important aspect of inheritance is that anyone who knows how to use a Button can also use our DynamicButton. In this case, however, they won't necessarily know how to use our extended functionality, but for most part they can use it as a normal button. They can also derive their own classes from DynamicButton and add their own functionality.

Create a new class called DynamicButton and add this code:

namespace DynamicButton
{
using
            System;

public class DynamicButton :
            System.Windows.Forms.Button
{
public int
            DynamicId;

public
            DynamicButton()
{
}
}
}

Literally, that's it! Now we have a new class that behaves exactly like a button, but now has an integer property called DynamicId.

In fact, we do want to add a little more extra functionality to our button. Specifically, we want to supply the ID whenever the button is created, and we also want to set the default text of the button to reflect this ID:

        public
            DynamicButton(int newId)
{
// set the ID...
DynamicId =
            newId;

// set the display...
Text = "Dynamic Button " +
            DynamicId;
}

Now we can move on to creating the buttons.

Creating the Buttons

To create the buttons, we need to add handler code behind the cmdCreateButtons button. This is relatively simple code, but looks a little complicated because we have to manage the layout of the new buttons ourselves, i.e. we have to keep track of the coordinates of the new button as we're adding them.

The first part of the handler sets up the loop:

protected void cmdCreateButtons_Click
            (object sender, System.EventArgs e)
{
// work out where the
            new buttons should be positioned...
int spacing = 8;
int y =
            txtLog.Top;
int width = this.Width - txtLog.Right - (3 *
            spacing);
int height = 25;

// loop and create the new
            buttons...
int n = 0;
for(n = 0; n < 6; n++)

Here, we've said that the height of the button will always be 25 and that y will start at the same point as the top of txtLog. We've also specified that spacing is 8, and this is value that we'll use to space out the buttons.

To create the button, we do this:

// create the new button...
DynamicButton newButton = new DynamicButton(n);

Notice how we pass the value of n through to the constructor as the DynamicId of the button. Next we need to position the button on the form:

newButton.Left = txtLog.Right + spacing;
newButton.Top = y;
newButton.Width = width;
newButton.Height = height;

The complicated bit there is deciding on the width of the button. We want it to be equally spaced between the right edge of txtLog and the right edge of the form itself.

Finally, we add the button to the control array and move y down to the position of the next button:

      // add
            the button to the form...
this.Controls.Add(newButton);

//
            next...
y += (height + spacing);
}
}

If we run the project now, this is what we'll get:

Of course, although the buttons have been created, they won't actually do anything. For this, we need to wire up the event handlers.

Wiring up Event Handlers

In .NET, there's a subtle difference between the ways that the term delegate is used. A "delegate" can be defined using the Delegate keyword and this describes the signature of the method to be used with the delegate. The signature of a method is determined through a combination of the parameters and the return type. So, a method that takes a single Object parameter and returns a String will have a different signature to a method that takes a Hashtable parameter and returns a String.

Where this gets confusing is that the method that's called is also called a delegate. For the remainder of this article, a "delegate" will refer to the definition of a delegate as constructed with the Delegate keyword. A "delegated method" will refer to the method that will actually be called as a result of a delegate invocation.

To wire up an event handler, we have to create a delegated method with this signature:

void DelegatedMethod(Object sender, System.EventArgs e)

What's really important here is that the name of the method that we create is totally arbitrary, i.e. the method can have any name that we wish it to have. So long as we create a method that has no return value, but accepts a System.Object and then a System.EventArgs object, we'll be OK. As you might have guessed, the order of parameters affects the signature of the method. If we reversed the order of the parameters, our method would not have the required signature for our delegated method.

All of the event handlers in the Windows.Forms control set fit this model, meaning that we can create some pretty powerful event handlers. We could, if we wanted, have every event from every control on a form routed through to the same delegated method. In fact, we can call a delegated method on any object we want from any other object. So, a button control on a form object called MyDialog could call a delegated method on a form object called MyDialogOwner.

The sender parameter provides a reference to the control that fired the event. In order to get at our DynamicId property, we have to cast this to a DynamicButton type. The e parameter specifies some event specific arguments. We're not going to worry about this here, but for some events this contains some extra information about what actually called the event.

Here's our click handler:

// DynamicButton_Click...
protected void
            DynamicButton_Click(object sender, System.EventArgs e)
{
//
            cast the sender to a button...
DynamicButton button =
            (DynamicButton)sender;

// msgbox...
AddToLog("Clicked " +
            button.DynamicId);
}

There's nothing at all unusual about this method. Now we have to wire up this method to the Click event of our dynamic buttons. Just before we call Controls.Add, we can add a handler:

      // wire
            up the handler...
newButton.Click += new
            System.EventHandler(this.DynamicButton_Click);

// add the
            button to the form...
this.Controls.Add(newButton);

Now, whenever the button wants to fire the Click event, the Framework goes through the list of delegated methods wired to that event firing each one in turn.

This is an important point - in .NET events can have more than one handler. This means that we could create another delegated method, wire it up using a new System.EventHandler instance and both methods would get called when the event was fired. Of course, the methods are not called simultaneously as this would, in effect, create a multithreaded application, and this is not something we want to approach casually.

We haven't defined what our AddToLog function looks like, so let's so that now:

// AddToLog - updates the log
            view...
private void AddToLog(String buf)
{
// update the
            log control...
txtLog.Text = (buf + "\r\n" +
            txtLog.Text);
}

Now let's try running the application and clicking on the buttons:

Other Handlers

Handlers that respond to other events on DynamicButton can be added in a similar manner. Firstly, we create methods for each one:

// DynamicButton_Enter...
protected void
            DynamicButton_Enter(object sender, System.EventArgs e)
{
//
            cast the sender to a button...
DynamicButton button =
            (DynamicButton)sender;

// msgbox...
AddToLog("Enter " +
            button.DynamicId);
}

//
            DynamicButton_Leave...
protected void DynamicButton_Leave(object
            sender, System.EventArgs e)
{
// cast the sender to a
            button...
DynamicButton button = (DynamicButton)sender;

//
            msgbox...
AddToLog("Left " + button.DynamicId);
}

Then we can add handlers for each one:

// wire up the handler...
newButton.Click += _
new System.EventHandler(this.DynamicButton_Click);
newButton.MouseEnter += _
new System.EventHandler(this.DynamicButton_Enter);
newButton.MouseLeave += _
new System.EventHandler(this.DynamicButton_Leave);

Now if you run the application again, messages will be added to the log as you hover the mouse over the buttons.

Another Example

Now that we've seen an example of how to create dynamic controls, let's take a look at how this technique can be used in your applications.

Dynamic controls can be used to adjust your application's user interface depending on certain environmental information. A classic example of this is adding new options to a toolbar when new "plug ins" or modules are added to the applications directory. For example, installing Adobe Acrobat on my computer automatically adds a button for creating an Acrobat document to my Word toolbar.

Another example could be on a database administration tool. When the tool is connected to the database, a button could be added to the UI representing each table contained within the database. Or, we could create a set of text box controls containing the names of all the XML files contained in a certain directory, and so on.

In this example, we're going to create an application that is able to load an assembly from disk, scan that assembly for types that inherit from System.Windows.Forms.Control and display a button on the form for each type that it finds. Clicking on the button will create an instance of the control and display it on the form.

Creating the Project

Create a new Visual C# - Windows Application project and call it ControlLoader. For these first steps, we won't be painting any controls onto the new form, but you might want to change the Text property to something like " Control Container ".

Add these members to the form:

   public class Form1 :
            System.Windows.Forms.Form
{
// members...
private ArrayList
            _buttons = new ArrayList();
private int _nextY =
            ButtonSpacing;
private Control _containedControl;

//
            constants...
const int ButtonHeight = 25;
const int
            ButtonSpacing = 5;
const int ButtonWidth = 200;

We're going to maintain a list of the buttons that we add in an ArrayList called _buttons. As each button is added to the form, we're going to need to position it at the correct y-coordinate, hence _nextY. We're only going to contain a single control at a time, and this will be stored in _containedControl. Finally, we're going to need some metrics that describe the layout of the buttons and these are described in the three constants at the end.

The buttons themselves will be constructed from a new class derived from System.Windows.Forms.Button. This new class will be called TypeButton and will have an extra property called ControlType that will contain a System.Type object. This Type object will represent the control in the assembly that we load. Create a new class called TypeButton and add a reference to the System.Windows.Forms namespace:

using System;
using System.Windows.Forms;

Next, add this code:

   public class TypeButton :
            System.Windows.Forms.Button
{
public Type
            _controlType;

public TypeButton()
{
}

//
            ControlType - get or set the property...
public Type
            ControlType
{
get
{
return
            _controlType;
}
set
{
_controlType = value;
this.Text
            = _controlType.FullName;
}
}
}

As you can see, when we set the ControlType property, we change the text of the button to be that of the full name of the type.

The only method we need to create on TypeButton is one that creates an instance of the control. Add this method that uses System.Activator to create an instance of the class and cast it to a System.Windows.Forms.Control:

      //
            CreateInstance - create an instance of the type...
public Control
            CreateInstance()
{
// return an instance of the new
            control...
return
            (Control)Activator.CreateInstance(ControlType);
}

Adding the Buttons

For the first test run, we'll add a TypeButton that represents an instance of a System.Windows.Forms.DataGrid control. This will let us test the logic of what we've done without having to go through the extra hassle of loading an assembly and scanning its types.

Let's look at the Form1.AddType method. This method will take a System.Type object and add a new TypeButton to the form. Firstly, we need to create an instance of a TypeButton and set its ControlType property to the new type:

      //
            AddType - add a type button to the layout...
public void
            AddType(Type controlType)
{
// firstly, create a new type
            button...
TypeButton button = new
            TypeButton();
button.ControlType = controlType;

Secondly, we need to add the TypeButton to our ArrayList that contains our list of buttons:

// secondly, add this button to our array...
_buttons.Add(button);

Now we can position the button on the form and add the button to the form's control list:

         //
            now, position the button...
button.Left =
            ButtonSpacing;
button.Width = ButtonWidth;
button.Top =
            _nextY;
button.Height = ButtonHeight;

// adjust the next y
            value...
_nextY += (ButtonHeight + ButtonSpacing);

//
            display the button...
this.Controls.Add(button);

Finally, we need to wire up the button's click event so that we can contain an instance of the control it represents:

         //
            next, wire up an event handler...
button.Click += new
            EventHandler(this.ButtonClick);
}

Of course, we haven't built ButtonClick yet, so let's do that:

// ButtonClick - called whenever a button is clicked...
protected void ButtonClick(object sender, System.EventArgs e)
{
// convert the sender to a type button...
TypeButton button = (TypeButton)sender;

Our first job is to cast sender to a TypeButton. That lets us use CreateInstance to create the control. Before we can do that, though, if we're already containing a control, we need to remove it from the form's control list:

// if we already have a contained control, remove it...
if(_containedControl != null)
Controls.Remove(_containedControl);

Now we can create the control:

// create an instance of the new control...
_containedControl = button.CreateInstance();

Finally, we can position the control on the form:

         //
            position the control on the form...
_containedControl.Left =
            ButtonWidth + (3 * ButtonSpacing);
_containedControl.Top =
            ButtonSpacing;
_containedControl.Width = this.Width -
            _containedControl.Left - (4 *
            ButtonSpacing);
_containedControl.Height = this.Height - (8 *
            ButtonSpacing);
this.Controls.Add(_containedControl);
}

Testing What We Have

Before we can see how far we've got, we need to add a button to the form. We'll use the System.Windows.Forms.DataGrid control. Add this code to Form1:

      private
            void Form1_Load(object sender, System.EventArgs e)
{
// on
            load, add a button type...
DataGrid grid = new
            DataGrid();
AddType(grid.GetType());
}

Now try running the project. If you click the System.Windows.Forms.DataGrid button, you'll see this:

Loading an Assembly

To round of this example, we'll show how we can load up an assembly at run time, scan through the assembly looking for types that derive from System.Windows.Forms.Control and add buttons for each one we find.

To load the assembly, we need a URL. This URL can either point to an assembly on the local computer, the network or somewhere on the Internet. In this example, we're going to use the code base for the current assembly. For this exercise, this will have the effect that it will load up all the types from the assembly that we're currently building. However, the same technique will work irrespective of where the assembly actually comes from.

The first thing to do is get this URL. Add this code to Form1_Load:

      private
            void Form1_Load(object sender, System.EventArgs e)
{
// on
            load, add a button type...
DataGrid grid = new
            DataGrid();
AddType(grid.GetType());

// find the name of
            our assembly...
String filename =
            this.GetType().Module.Assembly.CodeBase;

This line is a roundabout technique for getting hold of the assembly that implements the class of the object that the code is called from. In this case, it gets hold of the URL of the assembly containing Form1.

We need to use try.catch when we load the assembly, as there are a lot of things that can go wrong with this routine. We use a shared method on Assembly to do this:

// try and load the assembly...
try
{
// use load from...
Assembly controlAssembly = Assembly.LoadFrom(filename);

Once we have the assembly we can iterate through the types:

            //
            now get a list of types...
foreach(Type testType in
            controlAssembly.GetTypes())
{

The trick now is to create an instance of the type that we're testing, but cast it to Control. As all this is wrapped in another try.catch if we get to the next line we know that we have a fully qualified control.

// try and create an instance of the control
// and cast it to a control...
try
{
Control testControl = (Control)Activator.CreateInstance(testType);

Here's a wrinkle. System.Windows.Forms.Form is derived from Control as it uses Control 's containerization properties to display the controls drawn onto the form. If we check the TopLevelControl property, this will always be set if the class is derived from form.

                  //
            we need to make sure it's not a "top level"
            control...
if(testControl.TopLevelControl == null)
{
// if
            we got here, it is a
            control...
AddType(testType);
}
}

Finally, we can round off the two exception handlers:

               catch
{
//
            if we got here, we don't care about the
            object!
}
}
}
catch(Exception
            ee)
{
MessageBox.Show("The assembly could not be loaded. " +
            ee.Message);
}
}

Before we test this, we should drop another couple of controls into the project. Create a first class called DemoTextBox and add this inheritance clause:

public class DemoTextBox : System.Windows.Forms.TextBox

Now, create another class, this time called DemoMonthCalendar and add this clause:

public class DemoMonthCalendar : System.Windows.Forms.MonthCalendar

Build and run the project. You'll get something like this:

Summary

In this article, we learned how to create controls on the fly and add them to a form. We created a new class derived from System.Windows.Forms.Button that enabled us to add extra functionality and properties to the button. We also saw how to wire up methods to the new controls that get called whenever the events are fired. Finally, we examined how we could scan through an assembly looking for classes and use the techniques we've learned to build a simple tool that would load and display those classes at the user's command.

C#Today - © 2002 WroxPress Limited


Advanced Techniques for InterThread Communication

Advanced Techniques for InterThread Communication
http://www.csharptoday.com/content.asp?id=1599
Article Rating: 4
Published on July 16, 2001
Abstract
In Kaushal Sanghavi's previous article on interthread communication, he discussed some of the common and traditional ways of communication in a multithreaded system, which suffice for most situations. However, under certain conditions, we might want to use special techniques to allow threads within a process or across processes to communicate. In this article, Kaushal discusses some advanced techniques for inter-thread communication. Some of these techniques are not currently supported in the .NET framework natively, and we will use C#’s ability to call unmanaged code to achieve this.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

In my previous article (URL for Part 1 here), I discussed some of the common and traditional ways of communication in a multithreaded system. Most of these techniques are widely used and suffice for most situations. However, under certain conditions, we might want to use special techniques to allow threads within a process or across processes to communicate. In this article, we discuss some advanced techniques for inter-thread communication. Some of these techniques are not currently supported in the .NET framework natively, and we will use C#'s ability to call unmanaged code to achieve this.

The Sample Code

For this article I have developed an .exe file which will demonstrate the three methods discussed below, namely shared memory, named pipes, and filesystem watcher. The code to create the executable is included in the download at the end of the article.

Shared Memory

Under Windows, every process runs in its own address space. An address space is a contiguous block of memory where the code and data of the process reside. Since every process has its own address space, it is not possible for one process to gain access to another process's address space. This is important to ensure that no other process can gain access to a process's data or code, which is important from a security standpoint. While it is important to isolate and protect one process from another, it is equally important to allow cooperating processes to share a region of memory that can be accessed by both processes. If this were possible, then cooperating processes needing to share data can efficiently access the data directly in memory without having to marshal data across process boundaries. The .NET framework does not allow direct access to memory, and thus shared memory is not natively supported by .NET. However, the Win32 API supports methods that allow creation and manipulation of memory blocks that can be shared across multiple processes. C# can utilize the .NET support for unmanaged code to access these APIs. However, shared memory can only be used by processes running on the same machine and cannot cross machine boundaries.

The advantage of using shared memory is that it allows processes to use a memory block to share data without having to marshal it or pass it through a proxy object, which would provide a big boost to the performance of the application. Direct access is available to all processes. Further, it is possible to restrict access to the memory block to only processes that possess the valid credentials to access the data. This is necessary to ensure that only valid processes can access data, and that the data is not compromised. This is achieved via using security descriptors that are used by the Win32 API calls.

The following example demonstrates the use of shared memory via memory mapped files.

public unsafe bool Create (uint iMemSize,
            string iMemName)
{
mHandle = CreateFileMapping ((uint)
            0xffffffff, 0,
PAGE_READWRITE, 0, iMemSize, iMemName);

if
            (mHandle == 0 || mHandle == INVALID_HANDLE_VALUE)
return
            false;

uint lpView = MapViewOfFile (mHandle, FILE_MAP_READ |
            FILE_MAP_WRITE,
0, 0, 0);

if (lpView == 0)
            {
CloseHandle (mHandle);
return false;
}

mpCurrPos =
            mpBufferStart = (byte*)lpView;
mCurrOffset = 0;
mMemSize =
            iMemSize;

mMutex = new Mutex (false, iMemName +
            "Mutex");
return true;
}

This method uses the Win32 API calls to create a file mapping object. This method can be used to load any file into memory and access it using the normal file access functions. This provides extremely efficient access to files, since the entire file is loaded into memory. Note that, depending on the size of the file and available memory, the file could be loaded into physical or virtual memory. However, in our case, we want to treat a block of memory as a file, but we do not want to load an existing file into memory. Windows provides for this, by allowing the use of the system page file for this purpose. Using the system page file allows us to use a block of memory using file access functions. The special handle value 0xffffffff specifies the use of the page file. Further, since this block of memory needs to be shared across processes, we need the separate processes to be able to identify and access the same block. We do this by specifying a name for this memory block, using which, any process with the right privileges can access this block. Note that this name has to be unique across the machine, not only across memory blocks but across all system objects. Thus, if someone has created a mutex or any other system object with the same name, this call will fail since the name is not unique. This is the reason we append the "Mutex" key to the unique name when we create the Mutex object.

Once the file is mapped and the memory blocks are initialized, we need to obtain a handle to this memory that can be passed to the file access functions. Further, we need to map sections of this file into the calling process's address space. The MapViewOfFile call copies a section of the memory mapped file into the process's address space, and now this data is available to all threads within this process. Note that although the view is in a specific address space, any changes made to the view are reflected in the main memory block and replicated to all the open views of the file. Thus, changes made by a process are visible to all processes with access to this view.

Further, since the memory block will be shared across multiple processes, we need to protect the memory to ensure that there is no data corruption due to concurrent access. We use a mutex to provide this protection. Since we use a named mutex, and mutexes can be shared across process boundaries, only a single process will have access to the mutex.

The .NET framework does not natively support this functionality, and as a result, we have to use the Win32 APIs to use shared memory. We use the Interop features available in C# to call the Win32 APIs. A detailed discussion of Interop is beyond the scope of this article, but the following code demonstrates how you can access the Win32 APIs from within your C# code.

[DllImport("Kernel32", CharSet=CharSet.Auto)]
private static extern uint CreateFileMapping (uint hFile,
uint lpAttributes, uint flProtect,
uint dwMaximumSizeHigh, uint dwMaximumSizeLow,
String lpName);

[DllImport("Kernel32", CharSet=CharSet.Auto)]
private static extern uint MapViewOfFile (uint hFileMappingObject,
uint dwDesiredAccess, uint dwFileOffsetHigh,
uint dwFileOffsetLow, uint dwNumberOfBytesToMap);

public unsafe uint Read (ref byte[] oBuffer, uint iNumBytesToRead)
{
uint lBytesAvailFromCurrentOffset = mMemSize - mCurrOffset;

uint lBytesToRead = (uint)Math.Min(iNumBytesToRead,
lBytesAvailFromCurrentOffset);

fixed (void* lpBuffer = oBuffer)
CopyMemory (lpBuffer, mpCurrPos, lBytesToRead);

return lBytesToRead;
}

public unsafe void Write(byte[] iBuffer, uint iNumBytesToWrite)
{
uint lBytesAvailFromCurrentOffset = mMemSize - mCurrOffset;

uint lBytesToWrite = (uint)Math.Min(iNumBytesToWrite,
lBytesAvailFromCurrentOffset);

fixed (void* lpBuffer = iBuffer)
CopyMemory (mpCurrPos, lpBuffer, lBytesToWrite);
}

Running the above code from the executable returns the result:

Once the memory mapped files are created and the views mapped, we can use the traditional file access functions to read and write data. However, since we are using the traditional methods, we have to deal with pointers to be able to call the Win32 functions. C# provides the unsafe and fixed keywords that allow managed code to use pointers. Note that pointers can be used only for types that are supported in both the managed and unmanaged worlds. Dereferencing a managed type is not currently supported.

Mailslots and Named Pipes

Shared memory is useful when you want to treat a block of memory as a file. It allows you to seek to the right location in the memory block and do a read or a write. Further, with shared memory you have to manage access to the memory to ensure that there are no race conditions between the reader and the writer. Shared memory provides a generic data sharing and communication mechanism, but if all you need is to pass messages between a reader and a writer, mailslots and named pipes are also an available option. Mailslots and Pipes are simple communication mechanisms that allow messages to be passed between threads and processes. Unlike shared memory, these work over a network and the cooperating processes need not be on the same machine. Shared memory uses a unique name to create a memory block and all readers and writers on that machine will have to access this single block of memory. Mailslots and pipes internally use shared memory to allow the threads to communicate.

A pipe is similar to a queue structure with two ends. Messages go in from one end, and are retrieved from the other end. A one way pipe forces the messages to flow in a single direction, while a duplex pipe allows threads to read and write from its end of the pipe. There are two kinds of pipes, Anonymous pipes and Named pipes. Anonymous pipes work only within a machine boundary and since the pipes are not identified by names, the reader and writer process need to have access to the pipe handles before they can use the pipes. In this case, one of the threads would be responsible for creating the pipe and passing the handle to the other thread. Now that both threads have access to the pipe, they can start reading from or writing to the pipe. As with shared memory, pipes have security descriptors that allow and restrict access to the pipe. This ensures that only authorized processes can access the pipe.

Mailslots and pipes are nearly identical except that mailslots can be used to send broadcast messages on the network, allowing the same message to have multiple recipients. Pipes on the other hand allow for only a single reader-writer pair to communicate. Multiple readers and writers can use the same name for creating the named pipe, which creates multiple conduits of communication. However, for every conduit, there is a single reader and a single writer. Since mailslots and pipes are similar, we will focus on pipes in this article; however, the principles apply to both.

public class NamedPipeWriter
{
public uint Create ()
{
mHandle = Win32APICalls.CreateNamedPipe (mName,
Win32APICalls.PIPE_ACCESS_OUTBOUND,
Win32APICalls.PIPE_TYPE_MESSAGE |
Win32APICalls.PIPE_READMODE_MESSAGE |
Win32APICalls.PIPE_WAIT,
Win32APICalls.PIPE_UNLIMITED_INSTANCES,
100, 100, 0, 0);

uint lRetVal = Win32APICalls.GetLastError ();
return lRetVal;
}

public bool WaitForReader ()
{
return Win32APICalls.ConnectNamedPipe (mHandle, 0);
}

public unsafe uint Write (byte[] iBuffer, out uint oBytesWritten)
{
bool lRetVal;
uint lBytesWritten = 0;
uint lError;

fixed (byte* lpBuffer = iBuffer)
lRetVal = Win32APICalls.WriteFile (mHandle, lpBuffer,
(uint)iBuffer.Length, &lBytesWritten, 0);

oBytesWritten = lBytesWritten;
lError = Win32APICalls.GetLastError ();
return lError;
}

}

public class NamedPipeReader
{
public uint Create (bool iWaitForWriter)
{
if (iWaitForWriter)
Win32APICalls.WaitNamedPipe (mName,
Win32APICalls.NMPWAIT_WAIT_FOREVER);

mHandle = Win32APICalls.CreateFile (mName,
Win32APICalls.GENERIC_READ,
0, 0, Win32APICalls.OPEN_EXISTING, 0, 0);

uint lRetVal = Win32APICalls.GetLastError ();
return lRetVal;
}

public unsafe uint Read (byte[] iBuffer, uint iBufferLength,
out uint oBytesRead)
{
bool lRetVal;
uint lBytesRead = 0;

fixed (byte* lpBuffer = iBuffer)
lRetVal = Win32APICalls.ReadFile (mHandle, lpBuffer,
iBufferLength, &lBytesRead, 0);

oBytesRead = lBytesRead;
uint lError = Win32APICalls.GetLastError ();
return lError;
}
}

// constants and declarations from winnt.h and winbase.h
public class Win32APICalls
{
public const uint PIPE_ACCESS_OUTBOUND = 2;
public const uint PIPE_TYPE_MESSAGE = 4;
public const uint PIPE_READMODE_MESSAGE = 2;
public const uint PIPE_WAIT = 0;

[DllImport("Kernel32")]
public static extern uint GetLastError();

[DllImport("Kernel32", CharSet=CharSet.Ansi)]
public static extern uint CreateNamedPipe (String lpName,
uint dwOpenMode, uint dwPipeMode,
uint nMaxInstances, uint nOutBufferSize,
uint nInBufferSize, uint nDefaultTimeOut,
uint lpSecurityAttributes);

[DllImport("Kernel32", CharSet=CharSet.Auto)]
public static extern uint CreateFile (String lpFileName,
uint dwDesiredAccess, uint dwShareMode,
uint lpSecurityAttributes,
uint dwCreationDisposition,
uint dwFlagsAndAttributes,
uint hTemplateFile);

[DllImport("Kernel32", CharSet=CharSet.Auto)]
public static unsafe extern bool ReadFile (uint hFile,
byte* lpBuffer, uint nNumberOfBytesToRead,
uint* lpNumberOfBytesRead,
uint lpOverlapped);

[DllImport("Kernel32", CharSet=CharSet.Auto)]
public static unsafe extern bool WriteFile (uint hFile,
byte* lpBuffer, uint nNumberOfBytesToWrite,
uint* lpNumberOfBytesWritten,
uint lpOverlapped);
}

The above two classes encapsulate the functionality of a reader and a writer. Note that the code above shows only a few important methods, please refer to the attached code for the complete class declarations. When we run the code from the executable, we obtain the following result:

The writer uses the CreateNamedPipe Win32 API call to create a named pipe that will be created with the specified access modes. In the sample above, we will use a one way pipe, where we have explicit readers and writers. If you wanted to use a duplex pipe, all you need to do is change the parameters to this method; the rest of the code is identical for simplex and full duplex pipes. Once the pipe is created, the writer can wait for readers to connect to this pipe before it starts writing data into the pipe. Since the pipe is already created for the reader, the reader cannot use the same CreateNamedPipe call to create the pipe. If it does so, a different instance of the pipe will be created. Instead, the reader uses the CreateFile call with the OPEN_EXISTING flag which specifies that the reader wants to open an existing pipe. Once this call succeeds, both the reader and writer can use the traditional file access functions to read and write data into the pipe.

File System Watcher

Shared memory and named pipes require the reader and writer to both use the same techniques for communication. Shared memory and named pipes are suited for systems where both the reader and the writer run on homogeneous systems. Thus, you cannot have a reader running on a Windows machine and a writer on an IBM 390 system. This is due to the fact that shared memory and named pipes use proprietary protocols for communication that are not interoperable with other systems. Further, even if you have entirely homogeneous systems, it might not be possible to redesign both the producer and the consumer to utilize these techniques. A legacy application may not have the capabilities of exposing data using these communication methods.

.NET introduces a new method for threads and processes to communicate in situations similar to the ones discussed above. The .NET framework has the ability to monitor the file system for changes, and using delegates, it can notify a thread of any change in the file system. Thus a consumer thread could be waiting for a new file to be created in a particular folder that will contain the data that it needs. Now, the producer of this data could be running on any system or utilizing legacy systems to produce data; all it needs to do is create a file with this data and drop it into the specified folder. File formats have been established for a long time now, and it is extremely easy to create a file format that can be accessed from heterogeneous systems. For example, a standard ASCII text file can be read by almost all versions of operating systems available. Thus, a legacy application can create an ASCII file with processing instructions and pass it on to a .NET application, which is monitoring the file system. This provides an extremely powerful mechanism for different parts of a system to communicate with minimal effort. In fact, Microsoft BizTalk Server uses a similar technique for its messaging capabilities.

The System.IO namespace provides a class FileSystemWatcher that supports this feature. This class allows you to monitor changes to a file or directory and raises events corresponding to each change. Currently, the FileSystemWatcher triggers events for the following actions:

  • Created: This is triggered when a file or directory is created.
  • Deleted: This is triggered when a file or directory is deleted.
  • Renamed: This is triggered when a file or directory is renamed.
  • Changed: This is triggered when a file or directory's attributes change.

Since every directory maintains information about the files under that directory, an event that is triggered for a file change will usually trigger an event for its parent directory as well. If you are only interested in monitoring a specific set of files or directories, you can use the Filter and IncludeSubdirectories properties to narrow the scope of events that will be raised. This is important because on a system that has a lot of changes, there will be numerous events being raised, which is a performance consideration. Further, the events are buffered by the system in an internal buffer which has a default size of 8K. You should increase this buffer size if a lot of events are being triggered to avoid a buffer overrun.

public class FileWatcher
{
public
            FileWatcher(string iDirectory)
{
mWatcher = new
            FileSystemWatcher (iDirectory);
mWatcher.IncludeSubdirectories =
            false;
mWatcher.EnableRaisingEvents =
            false;

mWatcher.Created +=
new
            FileSystemEventHandler(FileCreatedHandler);
}

public void
            Start ()
{
mWatcher.EnableRaisingEvents =
            true;
}

public void Stop
            ()
{
mWatcher.EnableRaisingEvents = false;
}

public
            static void FileCreatedHandler(object source,
FileSystemEventArgs
            e)
{
string lPath = e.FullPath;
FileStream lStream =
            null;

try{
lStream = new FileStream (lPath,
FileMode.Open, FileAccess.Read);
}
catch (Exception
            lException){
System.WinForms.MessageBox.Show
(lException.Message, "Error");
return;
}

int
            lFileLen = (int)lStream.Length;
byte[] lBytes = new byte
            [lFileLen+1];
lStream.Read (lBytes, 0, lFileLen);

string
            lDataRead;
lDataRead = System.Text.Encoding.ASCII.GetString
            (lBytes);
System.WinForms.MessageBox.Show (lDataRead,
            lPath);

lStream.Close ();
}

private
            FileSystemWatcher mWatcher;
}
}

The above class monitors a directory for a file to be created, and once a new file is created, it tries to open the file and display its contents in a message box. In a real world scenario, this thread would read the contents of the file and invoke another thread to perform the processing related to the new file. In this example, we only monitor the directory specified for creation of new files. We do not monitor any other changes, and we do not monitor any changes to subdirectories. Once the FileCreatedHandler delegate is specified and the FileSystemWatcher started, every time a new file is created in the specified directory, the FileCreatedHandler method will be invoked by the .NET framework.

We can see this in action in the sample code. We start the filesystemwatcher:

And if we create a new file at the location specified in Path to Monitor, we get the following feedback:

Conclusion

As we have discussed in the above sections, in addition to the traditional means of communication, there are quite a few other advanced techniques available to developers. In addition to the techniques discussed in this article, Windows and the .NET framework provide other means such as RPC, MSMQ, Sockets, and DDE. In any multithreaded system, threads need to communicate and share data in order to cooperate. Designers of such systems should carefully evaluate all the options that are available and chose the one that is suitable to their systems. At times, a hybrid approach is also acceptable where a subset of the system uses certain techniques and another subset uses a different approach. Inter-thread and interprocess communication is a crucial piece of any design, and should be treated as such.

Happy .NETing.

C#Today - © 2002 WroxPress Limited


Communication Between Multiple Threads

Communication Between Multiple Threads
http://www.csharptoday.com/content.asp?id=1613
Article Rating: 4.5
Published on July 13, 2001
Abstract
Multithreaded programming has been dreaded by many developers over the years. Whenever more than a single thread is alive, developers have to take into account memory corruption, race conditions, deadlocks and a whole other slew of issues. Further, with multiple threads, you have to devise a scheme of sharing data across threads and a scheme for threads to communicate and work with each other. In this article, Kaushal Sanghavi explores some of the techniques that can be used in a multithreaded application to safely share data and communicate between multiple threads.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

Multithreaded programming has been dreaded by many developers over the years. Whenever more than a single thread is alive, developers have to take into account memory corruption, race conditions, deadlocks and a whole other slew of issues. Further, with multiple threads, you have to devise a scheme of sharing data across threads and a scheme for threads to communicate and work with each other. In this article, we will explore some of the techniques that can be used in a multithreaded application to safely share data and communicate between multiple threads.

Threads

A thread is a unit of execution. A thread is the unit that is granted CPU cycles by the operating system scheduler. Every thread has its own stack, but shares the process's memory. This, usually, is the root cause of all problems, because we now have multiple entities accessing the same resource. If this resource is not protected, its state could become corrupt. For example, if you have one thread reading the resource and another thread writing the resource, depending on the order in which these threads execute, it is possible that the resource is partially read and then is overwritten by the writer thread. Thus, the reader ends up with some state that existed prior to the writer changing the state, and some state written by the writer. Obviously, this is inconsistent and should be avoided by locking the resource that is about to be manipulated.

Threads are usually used when multiple actions need to be processed, and these actions may or may not be related. While there are a few cases where multiple threads can be entirely independent of each other, under the most common multi threaded scenarios, threads interact with each other. This interaction could be in terms of either sharing data or passing messages or both. Sharing data implies that multiple threads operate on the same data. Some threads could be readers while some threads could be writers. In this case, the data has to be protected against multiple writers. Data also has to be protected against readers and writers; the only safe scenario is where there are multiple readers and no writers. Further, threads need not always compete for the same data in order to cooperate. Consider the case of a pipelined scenario where a certain set of actions occur in a fixed pattern, and each step depends on the results of the prior step. In this case, a thread could represent each step in the pipeline, and once a thread finishes its step, it needs to signal the next thread to take over. In this case, the threads work cooperatively, but do not compete for the same data objects. Instead, threads would wait for the prior thread to present it with the data to be processed.

Inter Thread Communication

As we discussed in the previous section, threads need to communicate with each other. This communication could be via data sharing or message passing, or both. There are various techniques that can be used to achieve this; some techniques are specific to threads within a process while some can be used irrespective of whether the threads are in the same process or in separate processes. The model for this communication usually involves some objects that represent the data being shared, some objects that will be used to protect this data, and some objects to pass messages. The typical pattern involves a thread passing a message to another thread suggesting that the latter can process the data, and if there are multiple threads receiving the same message, they are synchronized by the object protecting the data. In the next few sections, we will discuss some of the most common objects that can be used for this communication.

Shared Data Objects

This is the most common form of data sharing within a process. There are multiple threads in a process, each of which has access to all the data in that process. This data usually comprises of all the public data objects that are available to the threads in the process. Threads can directly access this data by using the object names, and do not need to use any special accessor methods to get to this data. There are a few different ways in which this data can be protected, which guarantees that only a single thread will have access to this data at any given time. The System.Threading namespace provides the classes that can be used for synchronization.

The pattern for accessing data objects within a process is to wait for a synchronization object to allow you access to the data. Once you have access, you manipulate the data and once you are done, communicate this to the synchronization object, so that the next thread waiting in line can access this object. The shared data should be placed in an object and all access to the data should be via accessor methods that use the synchronization object. The .NET framework provides Mutexes, Monitors, Events, and ReadWrite Locks as synchronization primitives.

System.Threading.Mutex

This class provides a primitive for mutually exclusive access to data. This can be used by threads within a process as well as threads across processes. If the mutex is to be shared across processes, the mutex should be created with a unique name that is known to all the participating processes. Thus, if Process1 and Process2 want to share a mutex, they need to create the mutex with the same name, which is unique across all processes.

    public class
            SharedDataObject
{
public SharedDataObject ()
{
mMutex =
            new Mutex ();
}

public int
            Data
{
get{
mMutex.WaitOne();
int lData =
            mData;
mMutex.ReleaseMutex();
return
            lData;
}
set{
mMutex.WaitOne();
mData =
            value;
mMutex.ReleaseMutex();
}
}
private int
            mData;
private Mutex mMutex;
}

The code above shows how to use a mutex to protect data. In this case, if two threads were to access the same instance of SharedDataObject, its data would be protected by the member mutex. This primitive should be used when you need exclusive access to some data. Further, this is the only synchronization primitive that can be used across process boundaries.

System.Threading.Monitor

Monitors are similar to mutexes in that they allow exclusive access to a block of code and\or data. However, unlike a mutex, it is not possible to instantiate a monitor object and use it for synchronization. Monitors work on any object that is derived from System.Object and use the object's SyncBlock to provide synchronized access. All objects that derive from System.Object have an associated SyncBlock index and can be used as synchronization objects. A monitor sets up an object's SyncBlock via the Enter and Exit methods, and once this is established, you can call Monitor methods on this object. These methods are usually in the form of a Wait or Pulse method; the Wait method blocks for the object to be signaled, and the Pulse method wakes up a waiting thread. Unlike the Mutex object, Monitors also provide a way to wake up multiple threads via the PulseAll method.

   public void Consumer(
            )
{
Monitor.Enter (this);

if
            (!mDataAvailable)
Monitor.Wait (this);

ConsumeData
            ();
Monitor.Pulse (this);
Monitor.Exit (this);
}

public void Producer (int n)
{
Monitor.Enter
            (this);

if (mDataAvailable)
Monitor.Wait
            (this);

ProduceData ();
Monitor.Pulse
            (this);
Monitor.Exit (this);
}

The code above shows a typical Producer Consumer scenario. The Producer produces data and waits for a Consumer to consume the data before producing more data. The producer and the consumer use a monitor as a critical section to ensure that only a thread can be active inside the code block at a time. They also use the Wait and Pulse methods to communicate with the other thread. This is the difference between a mutex and a monitor, where a mutex is usually used to simply provide mutually exclusive access to data, while a monitor is used both as a critical section and as a signal. C# provides a native keyword, Lock, which encapsulates the Enter and Exit methods. Thus the above code could be written as follows with the same semantics:

   public void Consumer(
            )
{
lock (this) {

if (!mDataAvailable)
Monitor.Wait
            (this);

ConsumeData ();
Monitor.Pulse (this);
}
}

public void Producer ()
{
lock (this) {

if
            (mDataAvailable)
Monitor.Wait (this);

ProduceData
            ();
Monitor.Pulse (this);
}
}

System.Threading.AutoResetEvent and System.Threading.ManualResetEvent

Mutexes and monitors provide for synchronized access to code and data. This is useful when you only want a single thread to execute a piece of code or to access shared data. In addition to this, there are scenarios where threads don't need to guard against each other's execution, but need to pass messages to each other. Consider the case of the pipelined execution, a thread finishing its step needs to communicate this to the next thread. No data needs to be protected, the thread simply wants to let a waiting thread know that its time to wake up and perform a specific action. This can be achieved via the use of Events. Events are system objects that can be in one of two states, signaled or non-signaled. A thread can wait for an Event to be signaled, and can set or reset an event. Thus, in the case of the pipeline threads, the next thread could be waiting for an event to be signaled, and once the prior thread has completed its processing, it can set the event. This causes the waiting thread to wake up and continue execution.

There are two types of Events, AutoReset and ManualReset events. AutoReset events are used to signal a single thread. Once a sleeping thread resumes execution, the event state gets reset automatically and other threads that are blocked on this event remain blocked. ManualReset events, on the other hand, are used to wake up multiple threads. A ManualReset event remains in the signaled state until it is explicitly reset.

   //using auto reset
            events
public AutoResetEvent mStep1Event = new AutoResetEvent
            (false);
public AutoResetEvent mStep2Event = new AutoResetEvent
            (false);

public void PipelineStep1( )
{
PerformStep1
            ();
mStep1Event.Set ();
}

public void PipelineStep2
            ()
{
mStep1Event.WaitOne ();
PerformStep2
            ();
mStep2Event.Set ();
}

   // using manual reset
            events
public ManualResetEvent mStep1Event = new ManualResetEvent
            (false);
public ManualResetEvent mStep2Event = new
            ManualResetEvent (false);

public void PipelineStep1(
            )
{
PerformStep1 ();
mStep1Event.Set ();
}

public void PipelineStep2 ()
{
mStep1Event.WaitOne
            ();
mStep1Event.Reset ();
PerformStep2 ();
mStep2Event.Set
            ();
}

The code above shows two steps in a pipeline with a separate thread executing each step. The first thread completes its execution and signals an event that the second thread is waiting on. The second thread blocks on this event, and once this is signaled it performs processing for step 2. Once this is complete, it signals an event that implies that step 2 is complete and so on. As you can see, there is no data being protected in this case, threads simply communicate using these events.

System.Threading.ReaderWriterLock

As we discussed above, a mutex can be used to protect a reader from a writer. However, in the mutex sample, the reader uses the same mutex to guard against other readers as well. This is highly inefficient, since multiple readers can be allowed concurrent access without corrupting the state of the data. We need a lock that will allow multiple readers in but only allow a single writer, and ensure that readers and writers do not have concurrent access. The ReaderWriterLock class provides this functionality. This class supports the AcquireReaderLock, AcquireWriterLock, UpgradeToWriterLock and DowngradeFromWriterLock methods. As the name suggests, a reader can acquire a ReaderLock that will allow multiple readers concurrent access to the data, but block writers. Similarly, a WriterLock will guarantee that only a single thread will have access to the data, no thread can obtain a ReaderLock or another WriterLock till this thread releases the acquired lock. Further, it is also possible for a reader to change its mind and decide that it now needs to update the data. However, if the reader has only acquired the ReaderLock, it should not be updating the data; it should instead release the ReaderLock and obtain a WriterLock. It can either do this by releasing one lock and acquiring another, or by calling the UpgradeToWriterLock method, which internally performs the same action.

      public
            class SharedDataObject
{
public SharedDataObject
            ()
{
mReaderWriterLock = new ReaderWriterLock
            ();
}

public int
            Data
{
get{
mReaderWriterLock.AcquireReaderLock
            (-1);
int lData = mData;
mReaderWriterLock.ReleaseReaderLock
            ();
return
            lData;
}
set{
mReaderWriterLock.AcquireWriterLock
            (-1);
mData = value;
mReaderWriterLock.ReleaseWriterLock
            ();
}
}
private int mData;
private ReaderWriterLock
            mReaderWriterLock;
}

The code above replaces the mutexes from the original sample by the ReaderWriterLock. Now multiple readers can be active at the same time, and threads will be blocked only if there is a conflict between readers and writers. Like all other primitives, a thread can acquire the same lock multiple times without blocking, but the lock should be released as many times as it is acquired.

   mReaderWriterLock.AcquireReaderLock
            (-1);
mReaderWriterLock.AcquireReaderLock
            (-1);
mReaderWriterLock.AcquireReaderLock (-1);

int lData
            =
            mData;

mReaderWriterLock.ReleaseReaderLock();
mReaderWriterLock.ReleaseReaderLock();
mReaderWriterLock.ReleaseReaderLock();

return
            lData;

The code above will not cause the thread to block. Note that each Acquire has a matching Release; without this the lock will never be released and the other threads will be permanently blocked. Further, when a reader needs to convert itself to a writer, it is recommended that it call the UpgradeToWriterLock method. If a reader tries to acquire a write lock without releasing the read lock or vice-versa, the thread will be deadlocked.

mReaderWriterLock.AcquireReaderLock (-1);
int lData = mData;
mReaderWriterLock.AcquireWriterLock (-1);
mData++;
mReaderWriterLock.ReleaseWriterLock ();
mReaderWriterLock.ReleaseReaderLock ();

The code above will deadlock. The AcquireWriterLock() will be blocked even though the same thread has the read lock. The following code should be used to convert a reader to a writer. Similarly, you could use the DowngradeFromWriterLock to convert a write lock to a read lock. However a Downgrade can only be called after an Upgrade is called, if you have acquired a WriterLock and you want to convert that to a ReaderLock, you will have to release the WriterLock and acquire a new ReaderLock.

mReaderWriterLock.AcquireReaderLock (-1);
int lData = mData;
mReaderWriterLock. UpgradeToWriterLock (-1);
mData++;
mReaderWriterLock.ReleaseWriterLock ();

Waiting on multiple Objects

All the examples above showed threads communicating with each other using a single object. However there are cases where threads need to communicate using multiple messages, or multiple threads need to communicate. In the Producer-Consumer paradigm, it is possible that a consumer needs data from multiple producers before it can use the produced data. In the pipeline paradigm, it is possible that a step can be executed only when the previous step is complete and when another external condition is satisfied. Thus, threads will need to block on multiple mutexes or events.

Consider the case where a class has two properties each of which is protected by its own mutex:

    public class
            SharedDataObject
{
public SharedDataObject
            ()
{
mData1Mutex = new Mutex ();
mData2Mutex = new Mutex
            ();
}

public int
            Data1
{
get{
mData1Mutex.WaitOne();
int lData =
            mData1;
mData1Mutex.ReleaseMutex();
return
            lData;
}
set{
mData1Mutex.WaitOne();
mData1 =
            value;
mData1Mutex.ReleaseMutex();
}
}

public int
            Data2
{
get{
mData2Mutex.WaitOne();
int lData =
            mData2;
mData2Mutex.ReleaseMutex();
return
            lData;
}
set{
mData2Mutex.WaitOne();
mData2 =
            value;
mData2Mutex.ReleaseMutex();
}
}

private int
            mData1;
private Mutex mData1Mutex;

private int
            mData2;
private Mutex mData2Mutex;
}

Further, consider a method that needs to access both these properties, and in order to do so it needs to grab both the mutexes. One way to do this is as shown below, where the method grabs both the mutexes and then updates the data:

   public void UpdateAllData
            ()
{
mData1Mutex.WaitOne ();
mData2Mutex.WaitOne
            ();

ProcessData (mData1,
            mData2);

mData2Mutex.ReleaseMutex();
mData1Mutex.ReleaseMutex();
}

However the code above has a danger of deadlocking. Consider the case where a thread successfully executes the first line and grabs the first mutex. Now, the operating system preempts this thread and suspends it, and executes another thread. If this thread were to call another method that executed the following code, this will cause a deadlock.

      mData2Mutex.WaitOne
            ();
mData1Mutex.WaitOne ();

ProcessData2 (mData1,
            mData2);

mData1Mutex.ReleaseMutex();
mData2Mutex.ReleaseMutex();

Now we have one thread holding onto the first mutex and waiting for the second one, and another thread holding onto the second mutex and waiting for the first one. The first thread will never release the first mutex and the second thread will never release the second mutex, and the threads will be deadlocked.

While this seems to be a trivial example, and one could argue that no one would write code like this, in a system with a few hundred thousand lines of code, a few hundred objects and tens of thread pools, a similar situation could be very easily reached. The inherent problem in this approach is that a thread holds onto a lock while it asks for another lock. This is dangerous, as we have seen above. What we need is a way in which a thread could atomically test the state of each object that it needs to wait upon and grab all the locks in a single atomic operation. If this were possible, then no other thread could preempt the current thread and grab a lock that the current thread requires.

The .NET framework provides a method to do exactly this via the System.Threading.WaitHandle class. This class supports methods that work on any synchronization object, and allow a way to wait for multiple objects.

  • WaitAll: This method is used when a thread needs to block on multiple objects and needs to lock all the objects. This method takes an array of objects that can be waited on. Note that the Event and Mutex classes inherit from the WaitHandle class and can be used wherever a WaitHandle object is required.
  • WaitAny: This method is similar to the WaitAll method, with the only difference being that this wait will be satisfied when at least one object is signaled.

Using these methods, the above code should be rewritten as follows.

   Mutex[] lMutexArray = new
            Mutex [2];
lMutexArray [0] = mData1Mutex;
lMutexArray [1] =
            mData2Mutex;
WaitHandle.WaitAll (lMutexArray);

ProcessData
            (mData1,
            mData2);

mData1Mutex.ReleaseMutex();
mData2Mutex.ReleaseMutex();

This guarantees that the thread will either acquire both mutexes or none. While this prevents deadlocks, please note that deadlocks could arise from multiple scenarios and there is no single solution to avoiding them.

Sample Application

When you unzip the included download files, you will find a sample application and precompiled .exe to try out.

To compile the project yourself, you will need Visual Studio.NET.

Conclusion

Multithreading is a double edged sword. If used judiciously and carefully, it could give a significant boost to application performance and responsiveness. However, if not designed right, it might seriously hamper your application. Further, with multithreading, you need to beware of deadlocking and race conditions and ensure that your test plans cover all scenarios, especially concurrent stress scenarios.

C#Today - © 2002 WroxPress Limited


.NET Diagnostics

.NET Diagnostics
http://www.csharptoday.com/content.asp?id=1537
Article Rating: 4.5
Published on July 12, 2001
Abstract
Developers often find it necessary to have some support tools to aid in their effort to build applications. There is a whole family of products that are extremely popular with most developers, like debuggers, profilers and tracing utilities. Further, it is often necessary to understand the state of the environment where your application is active. This could include information about the processes running, the available resources like physical memory or storage space, and running threads to name a few. The .NET framework introduces a clean and simple way to obtain diagnostic information and support for providing custom diagnostic tools. In this article, Kaushai Sanghavi explores the features that the .NET framework provides for collection diagnostic information, and demonstrates how this support can be extended by writing a custom application that collects tracing information from the running processes.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

Developers often find it necessary to have some support tools to aid in their effort to build applications. There is a whole family of products that are extremely popular with most developers, like debuggers, profilers and tracing utilities. Further, it is often necessary to understand the state of the environment where your application is active. This could include information about the processes running, the available resources like physical memory or storage space, and running threads to name a few. All this debug and diagnostic information can be vital in understanding why an application might be misbehaving under a given situation, while working perfectly in another. Windows has always provided a way to access the low level information about the system and the environment, but this support, at best, has been kludgy and complex. The .NET framework introduces a clean and simple way to obtain diagnostic information and support for providing custom diagnostic tools. In this article, we will explore the features that the .NET framework provides for collection diagnostic information, and we will demonstrate how writing a custom application that collects tracing information from the running processes can extend this support.

System.Diagnostics

The System.Diagnostics namespace provides classes that encapsulate all the functionality that is needed by an application to collect diagnostic information. This information can be broadly divided into the following units of functionality.

Debugging Attributes

The .NET framework provides support for attribute-based programming. Thus a class could declare attributes that declaratively specify the behavior of the class. This was first introduced by MTS and COM+ where an object could declaratively request support for transactions and synchronization. Similar to these attributes, the .NET framework allows a class to specify attributes for debugging and diagnostics. For example, the following attribute specifies that the method should only be included if the symbol is defined.

[Conditional("SomeCustomCondition")]
public void ConditionalMethod ()
{

// do work
return;
}

The most common use of this attribute would be to include methods in your application that are useful only for a particular configuration. For example, you might want to include a Dump method that works only in the Debug configuration.

[Conditional("DEBUG")]
public void
            Dump()
{

// dump the state of all objects to a
            file
return;
}

Event Log Interaction

The system Event Log is a central repository for collecting any information about different processes. The Event Log consists of three predefined logs, one for Application level messages, one for System messages and one for Security messages. Applications should log any errors or warnings to the Application Log for administrators to view. Further, there are many tools that hook into the Event log and send out alert messages when an error is logged in the Event log. The System.Diagnostics namespace provides the EventLog class that allows for interaction with the Windows Event Log. This class supports both reading and writing entries into the log. Further, you can create your own custom logs and write to it as shown in the following example.

public void LogEvent (string
            iMessage)
{
try {
EventLog.CreateEventSource("SourceName",
            "LogName");
}
// since we want to create the EventSource only
            if it doesn't
// already exist, we don't have any code in the
            catch block.
catch (Exception e)
{}

EventLog lLog = new
            EventLog ();
lLog.Source = "SourceName";
lLog.WriteEntry
            (iMessage);
return;
}

Performance Monitor Interaction

The Performance Monitor is a tool that gathers real time data from applications and provides it to analyze the performance of the system. The performance monitor collects data in the form of counters from all applications that are registered with the performance monitor and plots these counters in a graphical form. This is extremely useful to see the state of the system in real time and also observe the relation between different processes and counters. A widely used example would be to observe the effect of running an application on system memory and CPU utilization. This can be invaluable in detecting memory leaks and poorly behaving applications. Developers who have built custom counters for the performance monitor under Windows NT or Windows 2000 can attest to the fact that this was not really the simplest task to achieve. However, the .NET framework makes this process as simple as instantiating a class and calling methods on it.

public void DoWork ()
{
PerformanceCounter lCounter =
new PerformanceCounter ("CategoryName", "CounterName");
lCounter.ReadOnly = false;
lCounter.Increment ();
}

Process Control

Often it is necessary to obtain information about a process in order to analyze its execution. This information could be as simple as obtaining the process id, or as detailed as obtaining its working set and peak paged memory count. The Process , ProcessModule and ProcessThread classes in System.Diagnotics provide information about any running process on the network. This information can then be used to troubleshoot a misbehaving process. In the next section, we will write a simple application that lists all the running processes in the system and dumps out some useful information about each process.

Trace Control

Applications usually need a tool that records certain checkpoints in the code. In conjunction with the debugger, this provides information to the developer about the application's execution. This is especially useful for multithreaded applications that are very difficult to debug. When running within a debugger, a single thread is active at any given time and thus race conditions are extremely difficult to reproduce from inside a debugger. Further, in order to debug an application, the developer has to be physically present where the application is running. This is very difficult to achieve on a production box. Tracing is the mechanism where useful information is dumped from a running application that can be used to replay the application execution. This dump could be in a simple file that contains the state of the running objects, the thread information etc. The System.Diagnostics namespace provides the Trace and the TraceListener classes that provide this support. We will explore this in greater detail in a following section.

Process Enumeration

A process is a collection of one or more threads and an address space. Put very simply, a process is a running application. When a process starts, it launches a single main thread, which in turn could spawn additional threads. Further, any memory that is allocated by any thread within a process belongs in the same address space, and when a process dies, it takes down all threads and all resources allocated by the threads with it. When a process starts, it can load additional modules. These are usually DLLs that provide the base functionality supported by the operating system. In addition, it is possible that a process will load additional custom DLLs. In fact, in a component based architecture, a process is merely the object that houses other modules that perform the guts of the application's functionality.

The Sysem.Diagnostics namespace has classes that allow you to interact with processes on any given machine. The Process class within this namespace encapsulates most of this functionality. In addition to the Process class, the ProcessModule class and the ProcessThread class encapsulate the modules that are loaded by a process and the threads spawned by the process respectively. These three classes support all the features needed to start, stop or query a given process. The following sample shows a very simple application that loads all the running processes and queries each for some information. In this sample, we only capture a subset of the information. Further, for any given application, you can query all the modules that have been loaded by this process.

The above screen shot shows the active processes on the local machine. Once you select a process, it will query the process for different properties mainly related to the memory consumed. As we mention before, for this sample we only gather a subset of the information; developers should feel free to tweak this as they see fit for their application's needs. Further, you can query the process for information about the modules that have been loaded by this instance of the process. The following screenshot shows the list of modules that are loaded by Notepad when it starts up:

Let us now look at the code for this sample:

private void btnProcesses_Click(object sender, System.EventArgs e)
{
Process[] lProcesses = Process.GetProcesses ();
cbProcesses.Items.Clear ();

for (int i = 0; i < lProcesses.Length; i++)
cbProcesses.Items.Add
(lProcesses[i].ProcessName + ":" + lProcesses[i].Id);
return;
}

Retrieving the list of active processes on a given machine is trivial. We invoke the GetProcesses static method on the Process class that returns an array of Process instances. Each instance of this collection represents an active process on the local machine. In this sample, we work off the local machine; there is an overloaded version of the same method that takes in a string parameter for the machine name. Note that in order to use the overloaded machine, you need to have access to enumerate the processes on the remote machine. Once we retrieve this list, we obtain the name and the process identifier of each process and add it to the combo box on our form.

private void cbProcesses_SelectedIndexChanged
(object sender, System.EventArgs e)
{
try{
Process lProcess = Process.GetProcessById (GetProcId());

txtPriority.Text = lProcess.BasePriority.ToString ();
txtTitle.Text = lProcess.MainWindowTitle;
txtPagedMem.Text = lProcess.PagedMemorySize.ToString ();

txtNonPagedMem.Text =
lProcess.NonpagedSystemMemorySize.ToString ();

txtMinSet.Text = lProcess.MinWorkingSet.ToString ();
txtMaxSet.Text = lProcess.MaxWorkingSet.ToString ();
txtResponding.Text = lProcess.Responding.ToString ();
txtProcTime.Text = lProcess.TotalProcessorTime.ToString();
txtThreads.Text = lProcess.Threads.Count.ToString ();
txtSet.Text = lProcess.WorkingSet.ToString ();
txtPrivateMem.Text = lProcess.PrivateMemorySize.ToString ();
txtStartTime.Text = lProcess.StartTime.ToString ();
}
catch (Exception lException){
MessageBox.Show (lException.Message,"Exception!!");
}

return;
}

Querying the process for its properties is as simple as invoking the properties and methods on the Process class. As you can see from the above code block, once we obtain the process that we are interested in, all we do is call the properties to obtain the data we are interested in. The Process class supports many other properties in addition to the ones shown above.

private void btnModules_Click(object sender,
            System.EventArgs e)
{
Process lProcess =
            Process.GetProcessById (GetProcId());
ProcessModuleCollection
            lModules = lProcess.Modules;
ProcessModule lModule;
string
            lModuleNames = "";

for (int i = 0; i < lModules.Count;
            i++){
lModule = lModules[i];
lModuleNames +=
            lModule.ModuleName + ":" + lModule.FileName;
lModuleNames +=
            "\n";
}

MessageBox.Show (lModuleNames, "Loaded
            Modules");
}

The modules loaded by a given process can be obtained by invoking the Modules property on the Process object. This returns a collection of all the modules loaded by this process. Once this is obtained, you can use the methods and properties on the ProcessModule class to obtain detailed information about each individual module.

private void btnKill_Click(object sender, System.EventArgs e)
{
Process lProcess = Process.GetProcessById (GetProcId());
DialogResult lRes = MessageBox.Show
(mWarning, "WARNING!!!!", MessageBoxButtons.YesNo);

if (lRes == DialogResult.Yes)
try{
lProcess.Kill ();
MessageBox.Show ("Killed!!!","Kill");
}
catch (Exception lException){
MessageBox.Show (lException.Message,"Unable to Kill");
}
}

In addition to obtaining information about existing processes on a machine, the Process class also supports creating and terminating processes. The above code block shows how you can use the Kill method in the Process class to terminate a process. Note that this is a dangerous way to end a process since you could lose unsaved data and it might cause the entire system to be unstable. Again, to kill a process you need to have the required privileges. If you don't have the privileges, or you try to kill a process that cannot be killed, the Kill method will raise an exception.

Custom Trace Application

As we mentioned earlier in this article, tracing is an invaluable tool to gather data about a running application. Traditionally, developers would build their own hand crafted versions of a trace class to implement this functionality. The .NET framework introduces a simple unified way to collect and store trace data. The System.Diagnostics namespace contains the Trace class that encapsulates this functionality. The Trace class supports methods ( Write , WriteLine etc) for outputting debug information. Your application can call these methods to log any information that might be useful in troubleshooting the application. Different applications may want this data to be logged in a different manner in different repositories. Some applications might want it reflected to the console, while some may want to redirect it to the Event Log and some may write it to a text file on the local disk. The System.Diagnostics namespace defines an abstract class TraceListener which defines where the data should be redirected to. Currently, the .NET framework supplies three different implementations of this interface.

  • DefaultTraceListener: This calls OutputDebugString to redirect the output to the command window.
  • EventLogTraceListener: This redirects the output to the Windows Event Log.
  • TextWriterTraceListener: This redirects the output to a text file on the local disk.

The DefaultTraceListener is active by default; you can add the other listeners by calling the Trace.Listeners.Add method, passing the listener of your choice.

private void btnKill_Click(object sender, System.EventArgs e)
{
Process lProcess = Process.GetProcessById (GetProcId());
DialogResult lRes = MessageBox.Show
(mWarning, "WARNING!!!!", MessageBoxButtons.YesNo);

if (lRes == DialogResult.Yes)
try{
lProcess.Kill ();
MessageBox.Show ("Killed!!!","Kill");
}
catch (Exception lException){

EventLog lEventLog = new EventLog
("Application", ".", "Diagnostics Sample");

EventLogTraceListener lELListener =
new EventLogTraceListener(lEventLog);

Trace.Listeners.Add (lELListener);
Trace.Write ("Exception while Killing: " +
lException.Message);

MessageBox.Show (lException.Message,"Unable to Kill");
}
}

The above sample shows how we can modify our Kill method to trace information to the Event Log. In order to see this in action, you should try to kill the Idle process (ID = 0). This will raise an Access Denied exception and the error will be logged to the event log.

Any class that derives from the TraceListener abstract class can act as a target for the Trace calls. At the least, this derived class needs to override the Write and the WriteLine methods. Once your class does this, you can now add it to the registered listeners, and the .NET runtime will route the trace calls to your class.

public class CustomTraceListener :
            TraceListener
{
public CustomTraceListener (string
            iFileName)
{
mFileName = iFileName;
mFile = new
            StreamWriter (mFileName, true);
mFile.AutoFlush =
            true;
}

~CustomTraceListener ()
{
Dispose
            (true);
}

protected override void Dispose (bool
            disposing)
{
if (disposing)
if (mFile !=
            null){
mFile.Close ();
mFile = null;
}
base.Dispose(
            disposing );

GC.SuppressFinalize (this);
}

override
            public void Write (string iStr)
{
WriteLine
            (iStr);
return;
}

override public void WriteLine
            (string iStr)
{
DateTime lDate = DateTime.Now;
Process
            lProc = Process.GetCurrentProcess();
int lProcId =
            lProc.Id;

string lPreamble = lProcId.ToString
            ();
lPreamble += ":";
lPreamble += lDate.ToString
            ();
lPreamble += ":";
string lLogData = lPreamble +
            iStr;

mFile.WriteLine (lLogData.ToCharArray
            ());
return;
}

private StreamWriter mFile =
            null;
private string mFileName;
}

The above code block shows a custom listener class that can be used to log trace data. This class derives the TraceListener class and implements the Write and the WriteLine methods. It uses a StreamWriter to create a text file and it logs the data to the text file. In addition to the data, it also logs the current timestamp and the process id of the logging process. As you can see from the above sample, creating a sample listener is pretty straightforward and you can customize the trace listeners to support the needs of your application easily.

The following is a list of the main files that are a part of the attached code sample.

  • DiagnosticsForm.cs: This contains the Form for the sample code, and the code to obtain the list of processes and modules.
  • CustomTraceListener.cs: This contains the code for implementing the custom Trace Listener. This object gets invoked if you check the Use Custom Tracer on the main form.

Conclusion

Diagnostics is an important but often overlooked feature of application development. Since most applications start small and simple, developers tend to ignore designing and planning for diagnostic support from the very beginning. By the time, they realize the need for this, it is either too late to come up with a clean design or the impact to the schedule is too severe to implement this support. However, the time and effort spent in building a clean robust infrastructure is often worth it, and ends up saving precious troubleshooting time in a production environment. Further, the .NET framework makes it extremely simple to add this support, and developers should try to take advantage of this support.

Happy .NETing

Links

http://msdn.microsoft.com/library/enus/cpguidnf/html/cpconintroductiontomonitoringmanagingwindowsprocesses.asp?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq

http://msdn.microsoft.com/library/en-us/cpguidnf/html/cporiintroductiontomonitoringframeworkfeatures.asp?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq

http://msdn.microsoft.com/library/en-us/dncsspec/html/vclrfcsharpspec_17_4_2.asp?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq

http://msdn.microsoft.com/library/en-us/cpref/html/frlrfSystemDiagnostics.asp?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq

http://msdn.microsoft.com/library/en-us/cpref/html/frlrfSystemDiagnosticsSymbolStore.asp?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq

C#Today - © 2002 WroxPress Limited


Error Handling in C#

Error Handling in C#
http://www.csharptoday.com/content.asp?id=1595
Article Rating: 2.5
Published on July 11, 2001
Abstract
In any programming language, error handling is an important feature. Properly checking and dealing with errors is one of the keys to writing robust and reliable applications. In C#, errors are handled by isolating the potentially erroneous code. When an error does happen, it is already expected and can be handled safely. In this article, Laylian Chiang shows us how to deal with errors wisely in C#.
Article

In any programming language, error handling is an important feature. Properly checking and dealing with errors is one of the keys to writing robust and reliable applications. In C#, errors are handled by isolating the potentially erroneous code. When an error does happen, it is already expected and can be handled safely. In this article, we will see how to deal with errors wisely in C#.

In many programming languages, errors are handled by detecting the return value of a function. For example, you probably have seen code that looks like this.

bool retval = MyFunction();
if (retval ==
            0) {
// success, MyFunction returns an expected result
// do
            something here
}
else {
// o o. An error has just
            happened
// better warn the user
}

What's wrong with this error handling mechanism? That works fine, but every return value has to be checked for an error. Even if you do not want to know the return value, you should not call the function just like this.

MyFunction();

You need to always check the return value. As a result, as your code becomes larger and more complex, you can imagine how nightmarish your life, as a programmer would be.

In other languages, errors are managed by writing statements to tell the program where to go if an unexpected error happens. In Visual Basic, this is achieved by using various On Error Goto or On Error Resume statements. If you are a Visual Basic programmer, you know how this error handling strategy can make your code very messy.

Fortunately, C# uses a different approach to error handling: by localizing the area of danger and throwing an exception. Exception handling allows the programmer to almost effortlessly detect errors without writing special code to test return values, and without the worry that carefully checking for errors will turn his/her code to spaghetti. We'll see how this is done in the next section.

Try and Catch

In C#, whenever you know that some part of your code might raise an error, you isolate that part in a block, guarded by the keyword try.

try {
// this is guarded region, i.e. the part of code that might raise an error
.
.
.
}

When an error does occur, an exception is thrown without first trying to execute the lines of code next to the line that causes the error. The program flow is then transferred to another block called the catch block. The complete picture is then depicted by a try - catch block, such as this.

try {
// this is guarded region,
// i.e. the part of code that might raise an error
.
.
.
}
catch ( [xxx]Exception e ) {
// this block is executed when
// an error occurs in the try block
.
.
.
}

In the catch block above, [xxx] in [xxx]Exception denotes the type of exception object thrown. The catch block must be present whenever you use a try block, otherwise the compiler will raise an error.

The code in Listing 1 shows code that performs division by zero. The code is not handled.

Listing 1: Unhandled code that performs division by zero

class Test {

public static void
            Main() {
int a = 9;
int c = a - 9;
int b = a / c; //
            beware, c = 0!!!
}
}

When the compiled code is run, the code will throw an exception with the following error message: Exception occurred: DivideByZeroException: Attempted to divide by zero. at Test.Main(). To add insult to injury, the program will crash.

The code in Listing 2, on the other hand, guards the code that may potentially perform a divide by zero operation.

Listing 2: Guarding the potentially erroneous code

using System;
class Test {
public
            static void Main() {
try {
int a = 9;
int c = a - 9;
int
            b = a / c; // beware, c = 0!!!
}
catch (DivideByZeroException)
            {
Console.WriteLine("You tried to perform a division by
            zero");
}
}
}

When the code in Listing 2 is compiled and run, an exception is thrown from the try block. This time, however, the exception is caught. The user will see the message You tried to perform a division by zero on the console. The program does not crash. It exits gracefully.

Note that you don't need to specify a variable for your exception object as the argument to your catch statement if you don't plan to use it, like the code in Listing 2. If you do, however, there are useful methods and properties that you have access to. This will be explained later.

Sometimes, the code in the try block may throw more than one type of exception. If this is the case, you can handle each individual exception separately using multiple catch blocks for that try -guarded region. When the try block throws an exception, the exception will be matched with the catch block that defines the same exception type. The code then becomes

try {
// this is guarded region, i.e. the part of code that might raise an error
.
.
.
}
catch ( [xxx-1]Exception e ) {
// this block is executed when exception of type xxx-1 is thrown
.
.
.
}
catch ( [xxx-2]Exception e ) {
// this block is executed when exception of type xxx-2 is thrown
.
.
.
}
.
.
.
catch ( [xxx-n]Exception e ) {
// this block is executed when exception of type xxx-n is thrown
.
.
.
}

Therefore, if the try block throws an exception of type [xxx-1], execution will continue to the code in the catch block that catches [xxx-1]Exception. If [xxx-2] is thrown, the program flow will be transferred to the first line of code in the second catch block. After the code in the corresponding catch block is executed, the program will continue to the line past the last catch block.

When using multiple catch blocks, however, you need to be aware of the hierarchy of all the exception classes used. The following section therefore discusses the structure of exception classes. Understanding this is critical to successful exception handling.

The Exception Class and Its Derivatives

All C# exception classes derive from the class called Exception, which is part of the System namespace in the .NET Framework. For example, the DivideByZeroException that we use in Listing 2 is derived from ArithmeticException, which is derived from SystemException, which itself is a child class of the Exception class. See Figure 1.

Figure 1: The hierarchical structure of DivideByZeroException

Object
Exception
SystemException
ArithmeticException
DivideByZeroException

The Exception class has some useful methods such as ToString and GetBaseException. Since Exception is the root of all other exception classes, all exception classes inherit these methods too. The ToString method, for example, is used to return the message from a particular exception. A derived exception class normally overrides this method to give the user a specific error message. For example, changing the catch block in Listing 2 into,

catch (DivideByZeroException e)
            {
Console.WriteLine( e.ToString() );
}

,will return the appropriate error message to the user, i.e. an Attempted to divide by zero message.

The GetBaseException method returns the original exception that caused that exception and other exceptions linked to it. For details, see the System.Exception class in the .NET Framework reference.

So far, I have not told you why understanding the hierarchy of exception classes is crucial. So please read on to find out.

Since Exception is the root of all other exception classes, it is legal and valid to use the Exception class for any type of exception. Listing 2, for example, is re-written in Listing 3 to demonstrate this.

Listing 3: Using Exception to catch all exceptions.

using System;
class Test {
public
            static void Main() {
try {
int a = 9;
int c = a - 9;
int
            b = a / c; // beware, c = 0!!!
}
catch ( Exception e )
            {
Console.WriteLine( e.ToString() );
}
}
}

If you don't see any impact other than the change from DivideByZeroException to an Exception in the catch clause, compile and run the code. What do you see as the error message? The same divide by zero error message! How does the Exception object know that this is a DivideByZeroException ? Thanks to polymorphism, you can up cast a derived object to its super object or super object's super object (e.g. from DivideByZeroException to Exception) yet still run the correct method (i.e. the ToString method of the DivideByZeroException object). This proves to be powerful. If you are rushing to finish your code and don't know or don't have time to look up each correct exception class for your catch block, you can use this one for all Exception class. It will catch any type of exception.

However, when you want different code to be run when different exceptions are thrown, you need to have a catch block for each exception. What you need to be careful about is when an exception that you try to catch is derived from another exception that is also present in another catch clause. For example, you might want to catch a DivideByZeroException at the same time your code in the try block might throw an ArithmeticException object. In this case, the catch clause that catches DivideByZeroException must appear before the one that catches ArithmeticException, like in Listing 4.

Listing 4: The derived exception class must appear before the parent

using System;
class Test {

public
            static void Main() {
try {
int a = 9;
int c = a - 9;
int
            b = a / c; // beware, c = 0!!!

// some other code here that
            might
// throw ArithmeticException

}
catch (
            DivideByZeroException e ) {
Console.WriteLine( e.ToString()
            );
}
catch ( ArithmeticException e ) {
Console.WriteLine(
            e.ToString() );
}
}
}

If both catch blocks in Listing 4 are reversed, no exception will ever be caught by the catch clause that catches DivideByZeroException, because any DivideByZeroException will be caught by the catch clause that catches ArithmeticException. The compiler detects this and will refuse to compile if a catch clause that catches a derived exception appears after a catch clause that catches another exception that is in the higher position in the hierarchy.

Also note that because the Exception class can match any other exception class, you can write a catch clause that catches an Exception at the end of a try - catch block to guarantee all exceptions are caught. This last catch clause serves as the last bastion in your error handling strategy.

Finally

When writing a try - catch block to handle exceptions, sometimes there is some piece of code that needs to be performed regardless whether or not an exception has been thrown. For example, you may want to guarantee an open file is closed when an exception occurs. Or, you might want to do some other cleanup.

If you are in this kind of situation, you can use the optional finally clause that accompanies the try - catch block. Whatever is in the finally block will be executed whether or not an error occurs in the try block. The syntax is like this.

try {
// code that may be erroneous
.
.
.
}
catch ( [xxx-1]Exception e ) {
// this block is executed when exception of type xxx-1 is thrown
.
.
.
}
catch ( [xxx-2]Exception e ) {
// this block is executed when exception of type xxx-2 is thrown
.
.
.
}
.
.
.
catch ( [xxx-n]Exception e ) {
// this block is executed when exception of type xxx-n is thrown
.
.
.
}
finally {
// do the cleanup or other things
// you want to be performed
// after the try - catch block
.
.
.

}

Passing the exception to the Caller

When the potentially erroneous code is in a function that you call from your code, you have several options to deal with the exception. You can either handle it locally in the called function, or you can let the caller deal with it.

When you decide to handle it locally, the caller does not to worry about handling the error. This is shown in Listing 5.

Listing 5: Handling the exception locally in the called function.

using System;
class Test {

public
            double divide(int a, int b) {
try {
return a/b;
}
catch
            ( DivideByZeroException e ) {
Console.WriteLine( e.ToString()
            );
}

}

public static void Main() {
Test t = new
            Test();
double result = t.divide(9, 0);
Console.WriteLine(
            result );
}
}

In the code in Listing 5, the divide method might be passed a zero for the argument b. However, since a divide by zero attempt will be caught in the method, the calling code in the Main method does not need a try - catch block.

The other option is to let the calling code handle the exception. This is illustrated in Listing 6, where the divide method does not have a try - catch block. This block, however, is found in the calling code.

Listing 6: Let the caller handle the exception

using System;
class Test {

public
            double divide(int a, int b) {
return a/b;
}

public
            static void Main() {
Test t = new Test();
try {
double
            result = t.divide(9, 0);
Console.WriteLine( result
            );
}
catch (Exception e) {
Console.WriteLine( "Error:" +
            e.ToString() );
}
}
}

You can also re-throw the same exception that you catch in the called function. This is given in Listing 7.

Listing 7: Throwing the same exception

using System;
class Test {

public
            double divide(int a, int b) {
try {
return a/b;
}
catch
            (DivideByZeroException e) {
Console.WriteLine( "An exception has
            occurred in the divide method" );
throw
            e;
}
}

public static void Main() {
Test t = new
            Test();
try {
double result = t.divide(9,
            0);
Console.WriteLine( result );
}
catch
            (DivideByZeroException e) {
Console.WriteLine( e.ToString()
            );
}
}
}

After the DivideByZeroException object is caught in the divide method, the exception is thrown again. This time, the caller must also handle it separately.

Alternatively, you can catch an exception, then throw a new exception to the caller, embedding your own error message to the exception object. This is given in Listing 8.

Listing 8: Throwing a new exception

using System;
class Test {

public
            double divide(int a, int b) {
try {
return a/b;
}
catch
            (DivideByZeroException) {
throw (new DivideByZeroException(
"the argument b is zero"));
}
}

public static void
            Main() {
Test t = new Test();
try {
double result =
            t.divide(9, 0);
Console.WriteLine( result );
}
catch
            (DivideByZeroException e) {
Console.WriteLine( e.ToString()
            );
}
}
}

Conclusion

Error handling is the key to writing robust and reliable applications. In C# error handling is achieved by isolating the potentially erroneous code and throwing an exception. In the code, this is done using the try -catch - finally block. C# shares the exception classes of the .NET Framework. In this framework, the Exception class is the root of all other exception classes.

When the potentially erroneous code is in another function that you call, you have choices as to handling the exception. You can handle it locally in the called function, or you can pass the exception object to the caller. When you choose the latter, you can either pass the same exception object or create a new one with more detailed information. For more information about the C# error handling mechanism, see the language reference at http://msdn.microsoft.com/vstudio/nextgen/technology/clangref.doc. Or, for more details on the Exception class and its derivatives, download the .NET reference from http://msdn.microsoft.com.

C#Today - © 2002 WroxPress Limited


Attributes in .NET

Attributes in .NET
http://www.csharptoday.com/content.asp?id=1593
Article Rating: 4
Published on July 10, 2001
Abstract
Developers have traditionally been adopting a functional approach to programming. Everything is a function or method, and functionality is explicitly asked for. While this works for building a monolith style application, component based programming doesn’t exactly fit into this methodology. Loosely coupled systems are now being advocated over closely coupled systems. Furthermore, declarative and attribute based programming is gaining popularity where attributes can be declared on a class that annotate the behavior of the code and communicate this to the runtime and other objects. The .NET framework brings mainstream attribute based programming to the Windows environment by allowing the use of predefined and custom attributes on classes, and most code elements. In this article, we will look into the support that the .NET framework provides for attribute based programming and we will demonstrate the use of custom attributes by building a sample application that can trace all calls to your component by merely declaring a single attribute.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

Developers have traditionally been adopting a functional approach to programming. Everything is a function or method, and functionality is explicitly asked for. While this works for building a monolith style application, component based programming doesn't exactly fit into this methodology. Loosely coupled systems are now being advocated over closely coupled systems. Furthermore, declarative and attribute based programming is gaining popularity where attributes can be declared on a class that annotate the behavior of the code and communicate this to the runtime and other objects. The .NET framework brings mainstream attribute based programming to the Windows environment by allowing the use of predefined and custom attributes on classes, and most code elements. In this article, we will look into the support that the .NET framework provides for attribute based programming and we will demonstrate the use of custom attributes by building a sample application that can trace all calls to your component by merely declaring a single attribute.

Attributes

MTS and COM+ introduced the concept of declarative programming. Using the MTS explorer or the COM+ admin tool, you could specify an object to be transactional or to require synchronization. Further, you could specify security declaratively, by specifying the roles that this component could be used by. The MTS and COM+ runtime would then ensure that the object did get the level of support that it had specified. This was a great feature, since you did not have to program these features into your component. All you needed to do was specify the properties (or attributes) on the object, and the compiler, at runtime, would ensure that the object was serviced appropriately. MTS and COM+ worked on the notion of interception; all calls to the object would be intercepted. Then during runtime they would obtain information about the caller and look up the object's properties to ensure valid semantics. While, this was a great feature, the only drawback to this was that the object did not inherently store its properties. Thus, if the same object (or component) had to be moved to a different machine, an administrator would have to ensure that the attributes were set up correctly on the new machine. While programming against the COM+ catalog could do this, it was prone to errors. This was due to the fact that the object and its metadata were stored as two separate entities.

The .NET framework solves this problem by ensuring that the object and its metadata are stored as one single unit. In the .NET world, an assembly contains various objects as well as the relevant metadata for these objects. The object and its metadata cannot be separated, and since the unit of deployment is an assembly, you are guaranteed that once an object is generated, it will always have access to the metadata that was specified at the time of compilation.

An attribute is specified on a class or a method within square brackets. You specify the attribute within square brackets and specify any parameters that need to be passed to the attribute within parenthesis. If an attribute requires no parameters, you can use empty parenthesis, or simply specify the attribute name in square brackets.

[Conditional("DEBUG")]
public void DumpTrace ()
{
// dump the state of the object to a file.
// you only need this in a debug release

return;
}

A very simple example of using an attribute on a managed object is the use of methods and classes that are valid only in a specific configuration of the assembly. This is most commonly used to allow debug methods that are valid in the Debug version but are omitted from the Release version. The code block above shows a skeleton method that is only valid if the DEBUG symbol has been defined. This method could dump the state of all of the object's member variables to a file. This file can then be viewed to try to replay the state of the object at various checkpoints. If we use the ILDasm tool that ships with the .NET framework, we can view the assembly code that is generated by the C# compiler. The following code block shows the generated MSIL (Microsoft Intermediate Language) for the above method.

.method public hidebysig instance void DumpTrace() cil managed
{
.custom instance void
[mscorlib]System.Diagnostics.ConditionalAttribute::.ctor(string) =
( 01 00 05 44 45 42 55 47 00 00 ) // ...DEBUG..

// Code size 3 (0x3)
.maxstack 8
IL_0000: br.s IL_0002
IL_0002: ret
} // end of method SomeObject::DumpTrace

As you can see from the above code block, the assembly contains the metadata along with the code for the method. This metadata contains the attribute as well as the value of the attribute. Thus the .NET runtime or any tool capable of reading assemblies can obtain the fact that this method supports an attribute of type System.Diagnostics.ConditionalAttribute and the value of this attribute is a string literal "DEBUG".

Note that although the attribute class is named ConditionalAttribute, we refer to it as Conditional in our code. The C# and VB .NET compilers are intelligent enough to append the literal Attribute at the end of an attribute to find the type definition of the attribute. The compiler first looks for the named attribute and if it cannot be found, it tries again after appending the Attribute literal. Thus the following two declarations are identical.

// style 1
[Conditional("DEBUG")]
public void DumpTrace ()
{
// dump the state of the object to a file.
// you only need this in a debug release

return;
}

// style
            2
[ConditionalAttribute("DEBUG")]
public void DumpTrace
            ()
{
// dump the state of the object to a file.
// you only
            need this in a debug release

return;
}

.NET Framework Attribute Classes

In the above section, we explained how attributes worked and why we needed attributes. Now let us look into the details of how attributes are specified and how you can define custom attributes. We will also look at the different kinds of attributes that are supported by the .NET framework.

The System namespace defines the Attribute class, which is the base class for all attributes. All custom attributes need to derive from this class and can be applied to the following code elements.

Assembly, Class, Constructor, Delegate, Enum, Event, Field, Interface, Method,
Module, Parameter, Property, ReturnValue, Struct

This is an important feature of attributes, since an attribute can be applied to only a few methods within a class or a few classes within an assembly. This allows fine-grained control over the declaration of attributes. The System.AttributeTargets is an enumeration that defines these targets. Since attributes can be applied to any number of elements, an attribute class needs to specify what target element it can be applied to. Further, an attribute needs to specify a few other properties such as whether classes that inherit a class with attributes, inherit the base class's attributes as well and whether a class can have multiple copies of the same attribute. In order to specify these properties, the System namespace provides the AttributeUsageAttribute class. This class is an attribute itself, and can be applied to other attributes. The following are some of the important properties supported by this class.

  • AllowMultiple: This specifies whether more than one instance of this attribute can be applied to the same target element. The default is false.
  • Inherited: This specifies whether classes that derive from this class inherit the attribute. This also applies to methods that override the base class's methods. The default is false.
  • ValidOn: This specifies the targets that this attribute can be applied to. This can be any target from the AttributeTargets enumeration. The default is that the attribute can be applied to all target elements.

In addition to these, the following is a list of some of the important methods and properties that the System.Attribute class supports.

  • GetCustomAttribute, GetCustomAttributes: These methods are used to retrieve the custom attributes that are defined on any given type. This is used by objects that need to query a type for its attributes.
  • IsDefined: This method is used by objects to determine the existence of specific attributes on any given type.
  • IsDefaultAttribute: This property specifies whether the attribute value that is specified on a type is the default value. This is useful for determining types that have overridden the default values of certain custom attributes.

Writing Custom Attributes

Now that we have covered some of the basic principles of attributes under the .NET framework and how they are useful, let us look at writing a simple custom attribute. As we discussed in the above section, a custom attribute class needs to inherit from the System.Attribute class, and it needs to specify its usage by specifying the appropriate properties for the AttributeUsageAttribute class. Let us write a simple attribute class that is applicable at the Class level and specifies whether any methods within the class use unmanaged or unsafe code. This can be a valuable piece of audit information that can be used for detecting classes that are entirely based on managed code.

[AttributeUsage(AttributeTargets.Class | AttributeTargets.Struct,
AllowMultiple = false, Inherited = true)]
public class UnmanagedCodeAttribute : System.Attribute
{
public UnmanagedCodeAttribute (bool iUsesUnmanaged, bool iUsesUnsafe)
{
mUsesUnmanaged = iUsesUnmanaged;
mUsesUnsafe = iUsesUnsafe;
}

public bool UsesUnmanaged
{
get{
return mUsesUnmanaged;
}
}

public bool UsesUnsafe
{
get{
return mUsesUnsafe;
}
}

private bool mUsesUnmanaged;
private bool mUsesUnsafe;
}

The above code block shows our custom attribute class. This attribute can only be applied to classes and structs, can be specified only once and can be passed on to derived classes. The attribute class itself is very simple. It derives from the System.Attribute class and the constructor takes in two parameters, which specify whether the class on which this attribute is applied supports unsafe and unmanaged code. Furthermore, this class supports two read only properties that allow for these values to be retrieved.

The following code shows how this attribute can be applied to a class:

[UnmanagedCode (false, false)]
public class ManagedClass
{
public void foo ()
{
// some managed code here
}
}

[UnmanagedCode (true, false)]
public class UnmanagedClass
{
public void foo ()
{
// invoke some unmanaged code here
}
}

As you can see from the above code, we have two classes that this attribute is applied to. The attribute is applied at the class level, and the first class is purely managed code while the second class contains unmanaged code but no unsafe code.

The following code shows how the attribute values can be retrieved:

public static void Main()
{
bool
            lUsesUnsafe;
bool lUsesUnmanaged;
Object[] lCustomAttrs =
typeof(ManagedClass).GetCustomAttributes(false);

for (int
            i = 0; i < lCustomAttrs.Length; i++)
if (lCustomAttrs[i] is
            UnmanagedCodeAttribute){
lUsesUnsafe = ((UnmanagedCodeAttribute)
lCustomAttrs[i]).UsesUnmanaged;

lUsesUnmanaged =
            ((UnmanagedCodeAttribute)
lCustomAttrs[i]).UsesUnsafe;
}
else{
lUsesUnsafe =
            false;
lUsesUnmanaged = false;
}
}

In order to retrieve the custom attribute values for a given type, we use the GetCustomAttributes method from the System.Type class to obtain the collection of custom attributes that have been specified on this class. Also, since there could be multiple custom attributes on any given class, we look for an attribute of type UnmanagedCodeAttribute, and if this attribute exists, we use the UsesUnmanaged and UsesUnsafe properties of the UnmanagedCodeAttribute class to determine whether the class uses unmanaged and unsafe code.

Attributes can also specify optional named parameters that can be used as shown below:

[AttributeUsage(AttributeTargets.Class |
            AttributeTargets.Struct,
AllowMultiple = false, Inherited =
            true)]
public class UnmanagedCodeAttribute :
            System.Attribute
{
public UnmanagedCodeAttribute (bool
            iUsesUnmanaged, bool iUsesUnsafe)
{
mUsesUnmanaged =
            iUsesUnmanaged;
mUsesUnsafe = iUsesUnsafe;
mUsesWin32 =
            false;
}

public bool UsesUnmanaged
{
get{
return
            mUsesUnmanaged;
}
}

public bool
            UsesUnsafe
{
get{
return mUsesUnsafe;
}
}

//
            Optional argument
public bool UsesWin32API
{
get{
return
            mUsesWin32;
}
set{
mUsesWin32 =
            value;
}
}

private bool mUsesWin32;
private bool
            mUsesUnmanaged;
private bool
            mUsesUnsafe;
}

[UnmanagedCode (false, false,
            UsesWin32API=true)]
public class Win32Class
{
// ... code
            here
}

As you can see from the above example, the UsesWin32API is a property that is optional. If a class wants to communicate this information, it can do so by using the property as a named argument to the attribute constructor.

A key point to note here is that attributes are specified on a type. Attributes are not specified on instances of types. Thus, all instances of a given type will always have the same values of the attributes; you cannot have one instance of a class with one value of the attribute and another instance of the class with another value. Attributes are code annotations that are applied at compile time and can only be retrieved at run time. Once the code is compiled, the values are read only and the metadata cannot be modified.

Sample Custom Attribute Class

Now that we have covered details of custom attributes under .NET, let us write a real application that uses custom attributes. Chris Sells and Keith Brown from DevelopMentor (http://www.develop.com/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq) have built an excellent tool that traces all method calls and return values from a native COM component (http://www.sellsbrothers.com/tools/index.htm%20/l%20tracehook?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq). In this article, we will build a similar tool to work with .NET applications (please interpret the term "similar" with a huge grain of salt. Chris's tool is much much better and is production caliber; our tool is merely a sample to demonstrate a concept). Any class that wants to take advantage of this functionality can do so by specifying a custom attribute. Also, since we will be using interception events to hook into the object's method calls, any object that needs to use this class must be to derived from the ContextBoundObject class.

Note that this sample uses .NET interception events and contexts to capture and trace method calls. A complete description of interception events and remoting is beyond the scope of this article and since our primary intention in this article is to demonstrate the use of custom attributes, we will not discuss the remoting concepts.

[AttributeUsage(AttributeTargets.Class | AttributeTargets.Struct,
AllowMultiple=false, Inherited=true)]
public class dotNETTraceHookAttribute : ContextAttribute
{
public dotNETTraceHookAttribute(string iFileName)
: base("dotNETTraceHook")
{
mFileName = iFileName;
}

public override bool IsContextOK(Context iContext,
IConstructionCallMessage iConsCall)
{
return false;
}

public override void GetPropertiesForNewContext
(IConstructionCallMessage iConsCall)
{
iConsCall.ContextProperties.Add
(new dotNETTraceObject(mFileName));
}

string mFileName;
}

The above code block shows the definition of our custom attribute class. The trace object will be logging data to a text file and thus the attribute constructor takes in a single parameter which is the name of the file you want the data to be logged to. Further, since we will be using the call context of the object, we inherit from the ContextAttribute class. This class, in turn, inherits from the System.Attribute class. Since we want this attribute to apply to a class or a struct, we use the AttributeTargets enumeration to specify this. Further, we want a single use attribute that can be inherited by derived classes. The AttributeUsageAttribute properties specify these values. Since we derive from ContextAttribute, we need to override the IsContextOK and GetPropertiesForNewContext calls. These are the calls that let us stuff our custom tracing object into the call context; this object will now be called when a client executes methods on the context bound object. By passing false from the IsContextOK method, we force the creation of a new context, which will have our custom trace object. The GetPropertiesForNewContext creates a new instance of our trace object and stuffs it into the context. As you can see from the code above, the custom attribute class itself is trivial; now let us take a quick look at the trace object itself.

public class dotNETTraceObject
:
            IContextProperty, IContributeObjectSink, IMessageSink
{

//
            Remoting methods omitted from here.
// Please refer to the
            attached code for a complete code sample

private void
            TraceMessage(IMessage iMessage)
{
if (!(iMessage is
            IMethodMessage))
return;

IMethodMessage lMethod =
            (IMethodMessage)iMessage;

StreamWriter lFile = new
            StreamWriter (mFileName, true);
lFile.Write
            (Process.GetCurrentProcess().Id + ":"
+ DateTime.Now.ToString ()
            + ": ");
lFile.Write(lMethod.TypeName + "::" + lMethod.MethodName
+ "(");

for (int i=0 ; i<lMethod.ArgCount ; i++
            ){
if (i > 0)
lFile.Write(",
            ");
lFile.Write(lMethod.GetArg(i));
}
lFile.WriteLine(")");
lFile.Close
            ();
}

private string mFileName;
}

The above code shows the definition of the trace object. We only show the actual method that captures the trace data and logs it to a file. Readers who are interested in all the .NET Remoting plumbing methods should refer to the attached code for a complete listing. The TraceMessage method is called by a method that intercepts all method calls for this object. The IMessage parameter contains all the information that we need to perform our tracing logic. In this sample, we are only interested in tracing method calls from a client to the object that supports our custom attribute. As a result, if the message parameter does not represent a method call, we ignore the message. However, if it is a method call, we use the IMethodMessage interface methods to determine the name of the class, the name of the method and the values of the parameters that are being passed to this method, and we output this data to the specified file.

[dotNETTraceHook("D:\\Tracehook.log")]
class AttributeClient : ContextBoundObject
{
public void DoWork (string iStr, int i)
{}

public void DoWork2 (string iStr)
{}

static void Main(string[] args)
{
AttributeClient lClient1 = new AttributeClient ();
lClient1.DoWork ("client1", 1);

AttributeClient lClient2 = new AttributeClient ();
lClient2.DoWork2 ("client2");
}
}

The above code block shows a sample class that uses our custom attribute. As you can see, the class itself needs to do nothing special to take advantage of our tracing object. All it needs to do specify our custom attribute dotNETTraceHook and pass it the path of the output file. If the developer of this class decides that she does not want to use our tracing object, she simply removes the reference to our custom attribute. No other code has to change. This demonstrates the beauty of attribute-based programming. The following screenshot shows the log file that is generated from the above sample run.

Conclusion

Attribute based programming provides a clean and elegant way to build a system of loosely coupled objects. Attributes are merely annotations to the code that are stored in the metadata and since the metadata is a part of the managed assembly, developers do not have to worry about the code and the metadata getting out of synch. The .NET framework uses attributes extensively and attribute based programming is here to stay.

C#Today - © 2002 WroxPress Limited


Threading in the .NET Framework

Threading in the .NET Framework
http://www.csharptoday.com/content.asp?id=1609
Article Rating: 3.5
Published on July 5, 2001
Abstract
Being able to execute multiple blocks of code simultaneously is extremely useful. In Visual Basic 6.0 and before, whenever your application was executing code, the application would freeze and not accept input from the user. Now, with any .NET language (including C# and VB.NET) all you need to do is launch another thread, and your application can keep accepting input. Similarly, ASP developers were taught never to explicitly store an object reference in Session or Application scope. Because Visual Basic did not allow free–threaded objects, any VB COM object would have slowed the system down had different page requests accessed the same object. With ASP.NET you can now create objects that you can legally store in Session or Application scope. In this article, Marin Smith explains how to implement complex multi–threaded .NET applications.
Article

Threading enables developers to execute multiple blocks of code simultaneously. Traditionally, developers had to program in C++ to use multiple threads, but now, any developer who uses .NET can easily develop multithreaded applications.

Being able to execute multiple blocks of code simultaneously is extremely useful. In Visual Basic 6.0 and before, whenever your application was executing code, the application would freeze and not accept input from the user. Now, with any .NET language (including C# and VB.NET) all you need to do is launch another thread, and your application can keep accepting input. Similarly, ASP developers were taught never to explicitly store an object reference in Session or Application scope. Because Visual Basic did not allow free-threaded objects, any VB COM object would have slowed the system down had different page requests accessed the same object. With ASP.NET you can now create objects that you can legally store in Session or Application scope.

This article will help you fully understand how to implement complex multi-threaded .NET applications.

Writing Thread-safe Classes

Multi-threading applications tend to be more error prone, so as a result you must be more careful when implementing objects for use in multi-threaded applications. The most common method of protecting against race cases (when simultaneously executing blocks of code attempt to modify the same data) is by using the lock(Object){} syntax. The lock syntax allows you to lock on an object, and only one thread can obtain a lock on a specific object at a time, thus protecting from race cases, but instead allowing the possibility of deadlock. Most often code for an object will need only one lock. In this case, then you simply pass the lock keyword the this reference: lock(this){}.

Fine-Grained Object Locking

Acquiring a lock on the this reference is not always optimal. Imagine an object that had several instance variables that could each be updated individually. If hundreds of threads access the same object, each thread would all be waiting for the same object lock, even though they might be accessing or updating different properties of the object causing unnecessary slowdown. In this case you would not want to lock on the this reference, but on the object you were updating. To do this you would simply change the referenced object in the lock statement. This allows much more fine-grained locking than simply locking all of the object's other methods.

Using Threads

Of course, making objects thread-safe is not very useful if the application that uses them only executes with one thread. To use multiple threads, the application must create more of them. A thread starts by launching a function. The thread is actually passed a delegate to the function it is to begin executing. A full discussion of delegates is an article unto itself (and here is an excellent one: http://msdn.microsoft.com/msdnmag/issues/01/04/net/net0104.asp?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq), but all you need to really know to use threads is that you pass the constructor the name of a function that returns void and takes no arguments, like so:

void MyProc() {
// Do stuff
            here.
}

static void Main()
// Create the
            thread.
System.Threading.Thread t = new
            System.Threading.Thread(
new
            System.Threading.ThreadStart(MyProc));

// Start the
            thread.
t.Start();

// Do more work here.
}

A thread is created in the Unstarted state. Once Thread.Start() is called, the thread starts executing and enters the Running state. Once a thread is running several events can occur:

  • The Thread.Suspend() method is called. When this method is called on the thread, the thread enters the Suspended state. It can only exit this state when the Thread.Resume() is called. If Thread.Suspend() is called once the thread is already in the Suspended state, nothing happens. Only one call to Thread.Resume() is needed to resume the thread regardless of how many times Thread.Suspend() was called.
  • Any of the Thread.Wait(), Thread.Join(), or Thread.Sleep() methods are called (for more information, see below). When any of these methods are called the thread enters the WaitSleepJoin state.
  • The Thread.Stop() method is called. The thread then enters the StopRequest state and waits for a safe point to stop. A safe point to stop is one a point at which the system could run a garbage collection. Once the thread reaches the safe point, it enters the Stopped state.
  • The Thread.Abort() method is called. The thread then waits for a safe point and throws a ThreadAbortException. This exception cannot be caught, but all finally blocks will be executed. The reason for this odd behavior is that even if the Exception could be caught, nothing could be done to recover from it because the system has requested that the thread itself be aborted. Thus, it only makes sense to allow the finally blocks to execute because handling the exception could only clean up code just like a finally statement would. Once the thread executes the finally blocks, the thread is aborted and enters the Aborted state.

Once a thread starts, it cannot reenter the Unstarted state. Similarly, once a thread has reached the Stop or Aborted states, it cannot be started again, or otherwise leave those states.

It is important to note that these states are not mutually exclusive. Thread.Abort(), Thread.Stop(), or Thread.Suspend() can be called while a thread is suspended or in the WaitSleepJoin state. In this case once the thread resumes executing, the requested action is immediately carried out. The following demonstrates threads moving through all of the different possible states.

public void Go() {
// The main thread is in the running state
System.Threading.Thread t1 = new System.Threading.Thread(
new System.Threading.ThreadStart(MyProc));
// Thread t1 is in the unstarted State.

t1.Start();
// Thread t1 is running

t1.Suspend()
// Thread t1 now has a Suspend Requested. Once it reaches
// a safe place to stop it will be in the suspended state.

t1.Resume()
// resume must be called to return a thread from suspended to running

Thread.Sleep()
// tells the main thread to enter the WaitSleepJoin state.

t1.Abort()
// t1 is unconditionally aborted.

}

The Thread Class

Most interaction with the System.Threading namespace will come via the Thread class. The Thread class encapsulates most of the logic needed to control the way threads behave. The most commonly used static methods, usually referred to as Thread Relative Statics, are methods and properties that refer to the thread currently executing. Listed below are the more commonly used Thread Relative Static methods:

  • void Sleep(Int32) and Sleep(Timespan):
    This overloaded method tells the current execution path to sleep for a specified amount of time. If for some reason, the thread gets woken up, a ThreadInterruptedException is thrown. Because this method can only be called by the current thread and not on a reference to a thread that may also be executing, the thread sleeps immediately and does not need to wait for a safe point for garbage collection like for the Thread.Suspend() method (see below).
  • Object GetData(LocalDataStoreSlot)
    This retrieves data from a specified slot in Thread Local Storage. To use this method Slots need to have been already initialized (see below).
  • void SetData(LocalDataStoreSlot, Object)
    Stores data in Thread Local Storage to be retrieved using GetData().
  • LocalDataStoreSlot AllocateDataSlot() and LocalDataStoreSlot AllocatNamedDataSlot(String)
    These allocate a data slot for use with the above two methods.

The Thread class also provides the static property Thread.CurrentThread. This returns a reference to the object for the currently running thread. The current thread can then access any of the instance methods or properties on itself. The most commonly used instance methods are:

  • Thread.Abort(), as described above causes a thread to abort. An uncatchable ThreadAbortException is silently thrown and the finally blocks will be executed.
  • Thread.Interrupt() interrupts a thread that is in the WaitSleepJoin State. If a thread is not in the WaitSleepJoin state, it will be interrupted when it next attempts to enter the WaitSleepJoin state.
  • Thread.Join() causes the calling thread to enter the WaitSleepJoin state. The calling thread only starts again once the thread referenced finishes executing and enters the Stopped state.
  • Thread.Suspend() suspends a thread. The thread is suspended once it has reached a safe point for garbage collection.
  • Thread.Resume() resumes a thread that is in the suspended state. If a thread is in the suspended state it will be resumed regardless of how many times Thread.Suspend() was called.
  • Thread.Start() tells a thread to start executing.

Thread Interaction

By now you may have a picture of how threads may communicate. First, there is the Thread.Abort() method. While this is not really communicating, it is a form of thread interaction. If a thread is not performing some critical function and you need to permanently stop it from executing, this method will simply stop it. Next, there is the Thread.Suspend(). If the system needs to conserve resources, or stop a thread for any other reason without doing something as drastic as aborting it, this method will temporarily stop the thread from executing. A thread can also pause itself. It does so by calling the Thread Relative Static method Thread.Sleep(). This simply allows a thread to let itself stop for a specified amount of time. Of course, these are all very primitive forms of thread interaction. Often your program needs to know when a specific thread has finished executing, or when any number of threads have finished executing. For this (as mentioned briefly above) .NET provides the Thread.Join() method.

Once the Thread.Join() method is executed on a thread reference, the calling thread enters the WaitSleepJoin state. The Thread.Join() method also allows a timeout parameter so that the calling thread continues executing once the thread has slept for a specific amount of time if the referenced thread does not die quickly enough.

WaitHandles

While these methods usually suffice for most programs, .NET also provides other synchronization primitives for even more fine-grained control. All of these primitives derive from the abstract base: System.Threading.WaitHandle. A WaitHandle allows a thread to call the overloaded WaitHandle.WaitOne() method and enter the WaitSleepJoin state. The WaitHandle then signals the thread to resume executing. The operational semantics of each WaitHandle are different, and allow the thread to resume under different circumstances (they are covered in more depth below). The WaitHandle class also provides the static WaitHandle.WaitOne(WaitHandle[]) and WaitHandle.WaitOne(WaitHandle[]) methods that allow a thread to wait for any or all of a group of WaitHandles to signal it to resume executing. .NET provides three basic WaitHandle implementations: System.Threading.AutoResetEvent, System.Threading.ManualResetEvent, and System.Threading.Mutex.

ManualResetEvent and AutoResetEvent WaitHandles

ManualResetEvent and AutoResetEvent instances can be in either of two states: signaled or nonsignaled. Threads wait by calling any of the WaitHandle methods. If signaled, they allow a waiting thread to pass, and if nonsignaled, waiting threads block. The Set() method switches the state of the instance to signaled. The difference between the two classes is that once signaled, a ManualResetEvent object remains signaled until you call Reset() while an AutoResetEvent object allows only one thread to resume, and then returns to the nonsignaled state.

Mutexes

Mutexes are another type of WaitHandle. A thread can "own" a Mutex. A thread requests ownership of a Mutex by calling any of the WaitHandle wait methods and then subsequently releases ownership of it by calling Mutex.ReleaseMutex(). If a thread requests ownership of a thread more than once (by calling any of the wait methods more than once) then it must call Mutex.ReleaseMutex() the same number of times. If one thread already owns a Mutex, then another thread that requests ownership must wait until the thread that already owns the Mutex calls Mutex.ReleaseMutex(). To show how Mutexes work, suppose you have an object that allows a specific number of reads, but to write, no other thread can be reading or writing, then you could use Mutexes thusly:

public class MyProperty {
private Mutex[]
            mutexes = {new Mutex(), new Mutex(), new Mutex()}
private Object
            mStoredProperty;

public void Update(Object newVal)
            {
{Mutex.WaitAll(mutexes);
mStoredProperty =
            newVal;
for(Int32 i = 0; i < mutexes.Length; i++)
            {
mutexes[i].ReleaseMutex();
}
}

public Object Get()
            {
Mutex heldMutex = mutexes[Mutex.WaitAny(mutexes)];
Object
            returnVal = mStoredProperty;
heldMutex.ReleaseMutex()
return
            returnVal;
}
}

Here, whenever a thread calls Get(), it obtains any one of the Mutexes and because there are three Mutexes, three threads can simultaneously access the data. However, only one thread can use the object at all when the Update() method is called.

Mutexes and WaitHandles are methods of making threads wait for actions from other threads. If no threads wait on them, then they do nothing by themselves.

Thread Pools

There is a certain amount of overhead associated with starting and stopping a thread. Because in many applications many threads would be idle most of the time or perform small operations and die quickly it is desirable to conserve resources by sharing already created threads. To do this, .NET provides a ThreadPool class that is easy to use. The static method ThreadPool.QueueUserWorkItem() allows you to queue a delegate. Similarly, the static ThreadPool.RegisterWaitForSingleObject() method allows you to register a delegate to be called at a certain interval, much like a timer. There is only one thread pool per application, and thus all of the methods are static. At any given time, up to 64 work items can be queued. You probably will not use thread pools too often, but if you do, nothing else will probably allow equal performance.

Threading Details

Up to this point, we've taken for granted that objects have an associated lock, but haven't looked much into how this is implemented. This is a topic that could span several books, so a brief introduction will be given here.

Each application has a SyncBlock table. Before a lock is ever requested, each object contains an empty reference to an object in this SyncBlock table. When a lock is first requested on an object, it fills this reference and creates a SyncBlock in the table. This SyncBlock allows the object to be locked and unlocked.

.NET provides a Monitor class that encapsulates most of the object locking and deals with these SyncBlocks. C# replaces the lock code blocks with Monitor.Enter(Object) and Monitor.Exit(Object) calls. So the code:

lock(myObject) {
//Do Some Work.
}

gets translated to:

Monitor.Enter(myObject)
//Do Some Work
Monitor.Exit(myObject)

A thread can also attempt to obtain an object lock, by calling the Monitor.TryEnter(Object) and optionally specifying a timeout. If this call fails, then the method returns false, but the thread is never blocked in this case and keeps executing if the lock is not free.

Common Pitfalls

Programming multithreaded applications is much more complex that programming single threaded ones. Besides the obvious deadlock, there are several common mistakes that crop up when programming for multiple threads.

Dealing with Value Type Operations

The operation z = z + 1 is not thread safe. If two threads are executing simultaneously, both threads could read the same value for z. Having both read the same value for z, they would then both assign the same incremented z. However, two threads approached this code so that is not what was intended, and the count is off. For instances like these you either need to lock the integers or use the static methods of the Interlocked class.

Locking on Value Types

Consider the following code:

Int32 n = 1;

public void Go()
            {
lock(n) {
// Do work!
}
}

This will execute and no thread will ever block. The problem here is that when the thread attempts to lock, it boxes the integer, so it copies the integer to the stack, and each time this is called will result in a different copy of the integer being created on the stack. So every time the code executes it will obtain a lock on a different object that will result in no locking at all.

Problems With Using the Monitor or Mutex Classes

Consider the following code:

Object n;

public void Go()
            {
Monitor.Enter(n);
// Do Work!
return
            5;
Monitor.Exit(n);
}

This function will never release the lock because the return call causes the function to exit before calling the Monitor.Exit(n) statement. For this reason, if you use the Monitor class you should be very careful. The same applies to Mutexes. If you forget to call ReleaseMutex(), then the Thread will forever hang onto it, and all other threads will block.

Sample Code

Included with this article is some sample code to demonstrate the use of Mutexes. Enjoy trying it out!

Conclusion

This article has covered just the basics of writing multithreaded classes and applications in .NET. Several books could be written just on this topic. However, since .NET is so new, there isn't very much documentation. Microsoft's .NET site (http://msdn.microsoft.com/net/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq) covers more material including the full interface specification, and the best way to learn more is just to try writing as much code as possible. The more of the System.Threading namespace that you use, the more you will understand how to use threading to its fullest potential.

C#Today - © 2002 WroxPress Limited


Understanding .NET Remoting

Understanding .NET Remoting
http://www.csharptoday.com/content.asp?id=1607
Article Rating: 4
Published on July 4, 2001
Abstract
The .NET remoting infrastructure is the core facilitator for communication between multiple app domains in the .NET framework. Exposing your .NET objects as web services is extremely simple using the remoting framework and yet it gives you all the power and flexibility to fine tune the performance of your Web services to meet specific customized business processing needs. In this article, Aravind Corera examines the individual building blocks that constitute the .NET remoting framework such as the hosting environments, transport channels, serialization formatters, lifetime management and activation policies and shows you how to put these pieces together to build scalable, reliable and high-performance web services.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

The remoting infrastructure provided by the .NET runtime is a very powerful model that allows you to leverage the services of objects residing in different App Domains and host machines. The remoting model in .NET is extremely simple yet brings with it a whole myriad of rich features that allows you to fine tune the scalability and performance of your web service. It allows for communication between multiple App Domains either in the:

  • same process,
  • different processes,
  • same machine,
  • or different machines.

It's also the key enabler of Web Services in the .NET framework. We've seen how web services provide loosely coupled information processing islands in the Internet cloud that serve up consumers using standards like SOAP. What is important to understand is that the .NET remoting framework allows us to easily expose a .NET object to clients as a Web Service. (We will see exactly where SOAP fits into the .NET remoting stack when we talk about the message serialization formatters).

It's also worth noting that two of the three transport channels provided out-of-the box by the remoting framework carry SOAP payloads by default. It also facilitates CLR object remoting that marshals entire object references across the transport channel wire that can be unmarshalled and used at the other end by the client. What we will be examining in this article are the individual pieces that constitute the .NET remoting framework. We will take a look at the hosting options, the transport channels, the serialization formatters and the activation & lifetime management options that constitute the remoting infrastructure. Let's take a look at each of these pieces so that you can use them to your advantage to assemble web services that suit your business processing needs coupled with the best possible scalability and performance.

Getting Started

To work with the examples and the sample applications used in this article, you will need the .NET Framework SDK Beta 1 bits. You should be able to compile and run all of the samples from the command-line, so it's not mandatory that you have Visual Studio.NET installed. Most of the samples we will be looking at will use ASP.NET client pages to consume the remoted objects. So you'll need to have IIS installed to run the samples. You can use the MakeAll.bat batch file in sample source code bundle provided with this article to build all the source files.

Let's actually get down to business and see how to actually expose a .NET object using remoting, so that it can accept service requests from remote clients. Now let's consider the options that are available to you to host the object remotely. You could host the object from:

  • Internet Information Server (IIS).
  • A managed .NET executable
  • COM+ Services. (Hosting your object from COM+ Services allows you to leverage all the rich features that COM+ provides such a Object pooling, Transaction Services, Just in Time Activation etc).

First let's take a look at the .NET object that we are going to remote. We will then host this object in Internet Information Server (IIS), so that it is available to remote clients. The example that we will take up is a fictitious Pizza Diner that publishes it's daily menu through a service deployed at a well known URI that customers can use to look up what's hot on the menu at their favorite pizza restaurant. Here's the code for the .NET object that we intend to remote:

Listing 1: Diner.cs

namespace MyDiner
{
using
            System;
using System.Collections;

///
            <summary>
/// PizzaDetails - Provides pizza details
///
            Each element in the ArrayList is an object of type
            PizzaDetails
/// </summary>
[Serializable]
public
            class PizzaDetails {

public String m_strName;
public
            String m_strDescription;
public float m_fCost;

public
            String Name {
get {return m_strName;}
set {m_strName =
            value;}
}

public String Description {
get {return
            m_strDescription;}
set {m_strDescription =
            value;}
}

public float Cost {
get {return
            m_fCost;}
set {m_fCost = value;}
}

}//class
            PizzaDetails

/// <summary>
/// The Pizza Diner
            Remote Object
/// </summary>
public class Diner :
            MarshalByRefObject {

public Diner()
            {

System.Console.WriteLine("Constructor called");
listPizzaMenu = new
            ArrayList();
populatePizzaMenu();

}//Constructor

public
            ArrayList getPizzaMenu() {

System.Console.WriteLine("Received
            remote request from client");
return
            listPizzaMenu;

}//getPizzaMenu

private void
            populatePizzaMenu() {

PizzaDetails pizza1Obj = new
            PizzaDetails();
pizza1Obj.m_strName = "All Green - Weight
            watcher's delight";
pizza1Obj.m_strDescription = &
            _
"Green peppers, pineapple, olives and " & _
"lettuce
            with a low calorie cheese topping";
pizza1Obj.m_fCost =
            8.50f;
listPizzaMenu.Add(pizza1Obj);
...

}//populatePizzaMenu

private
            ArrayList listPizzaMenu =
            null;

}//class

}//namespace

You can compile the above code from the command line using:

csc /out:MyDiner.dll /target:library Diner.cs

This produces the MyDiner.dll assembly for the remote object.

Now, the first thing that strikes you when you take a look at the code snippet above, is that the Diner class inherits from the MarshalByRefObject (MBR). All objects that you want to expose to clients at a well-known URN through .NET remoting need to derive from MarshalByRefObject . This will ensure that the client gets an appropriate Object Reference ( ObjRef ) back that it can use thereafter to construct a proxy for the remote object from the information that it ferreted out from the ObjRef . Under the hood, actually, there are 2 proxies created on the client, a TransparentProxy and a RealProxy . The RealProxy is the one that routes the call to the remote machine that hosts the object. Though you really don't have to be bothered about how the client interacts with the created proxies, it's nice to know what actually happens under the covers.

Hosting the Remote Object on IIS

As was mentioned earlier, your remote object can be hosted in a number of ways. Let's look at how you could host the object in Internet Information Server 5.0 (IIS) . You need to do a couple of things first to get the object hosted in IIS. The first step involves creating a new Virtual Directory. So let's go ahead and create a virtual directory called MyDiner . Create a bin subdirectory under the physical folder location pointed to by your web application's virtual root, and copy the MyDiner.dll assembly that hosts the Diner remote object into the bin subfolder. There's just one more thing to do before we are set. This step involves creating a remote configuration file called remoting.cfg  that you need to place in the virtual root.

The format of the information that you need to provide in remoting.cfg is as follows:

Name#[Name of the Application(Needs to be same as the virtual root)]
WellKnownObject#[FullyQualifiedTypeName]#[AssemblyName]#[ObjectURIEndpoint]#[ObjectMode]

So going by the above format, the remoting.cfg file for our Diner object should look something like this:

Name#MyDiner
WellKnownObject#MyDiner.Diner#MyDiner#MyDiner/Diner.soap#Singleton

Make sure that the name you provide in the Name# section is the same name as the virtual root of your web application. Once you have done this, point your browser to the URI Endpoint that you provided in the remoting.cfg . If everything went off well, you should see the Services Description Language(SDL) contract for the Diner object. (Please note that in Beta 2, the contract would be in Web Services Description Language (WSDL) standard. For more pointers on WSDL, please refer to the Related Links section below).

Accessing the remote object from an ASP.NET page

Now let's try to access the remote Diner Object. Let's use an ASP.NET page hosted on another machine to access this remote object.

Method 1: (Explicitly Registering the Channel & activating the object without a configuration file)

To activate the remote Diner object, we need to do a couple of things first. First, we need to register a channel. The channel will then serve as the conduit to carry the activation and service requests to the remote object. You can register the channel by using the ChannelServices.RegisterChannel . The most commonly used channels (that ship out-of-the-box) are the HTTPChannel and the TCPChannel . You will find them in the System.Runtime.Remoting.Channels.HTTPChannel & System.Runtime.Remoting.Channels.TCPChannel namespace respectively. Here's an example of how you would create and register an HTTPChannel.

   HTTPChannel chan = new
            HTTPChannel();
ChannelServices.RegisterChannel(chan);
..
ChannelServices.UnregisterChannel(chan);

Once you register the channel, you are ready to activate the remote object at the well known URI and make method calls on it. To do this, you need to call Activator.GetObject . This method takes the Type of object to invoke and it's remote URI endpoint as its parameters.

// Create an instance of the remote
            object
Diner dinerObj = & _
(Diner)Activator.GetObject(typeof(Diner),"http://bingo:80/MyDiner/Diner.soap");

//
            Get Today's Menu from the remote Pizza Service
if(dinerObj !=
            null) m_listPizza = dinerObj.getPizzaMenu();

Once you get a reference to the Diner object from the Activator.GetObject method call, you can invoke methods on it as if it were a local object. The actual call is made on the proxy that's created by the remoting runtime during the Activator.GetObject call and this proxy ( RealProxy ) is responsible for routing the call to the remote object. You would need to reference the classes & types in the server assembly from your client application. In our case, we need to reference the Diner class in our client code. So, you can either XCOPY the server assembly directly to the bin subfolder of your client application's virtual root. This is the easiest way to reference the server assembly. Please note, that though you are copying the server assembly to the client's bin folder, it will be used only to reference the types exposed by the remote object. As an analogy, this is very similar to how a typelibrary is registered on a client's machine in COM to early bind to the interfaces and types in the server at compile time, though the actual call could be remoted using D COM . Alternatively, you could also use SOAPSUDS.EXE (that ships with the framework) to point to the URI endpoint and make it generate a metadata proxy containing the server types. SOAPSUDS uses the Service Description Language (SDL) contract file (will be WSDL in Beta 2) and constructs a metadata proxy that the client can use. This proxy inherits from System.Runtime.Remoting.Services.RemotingClientProxy , which you can confirm by opening up the proxy using the IL Disassembler (ILDASM). The following command shows you how to generate a metadata proxy using SOAPSUDS .

soapsuds -url:http://MyRemoteServer/MyDiner/Diner.soap -oa:MyDiner.dll

Take a look at the codebehind C# source that's used by the ASP.NET client to activate and invoke methods on the remote Diner Object.

Listing 2: C# source for code-behind assembly - PizzaRemotingClientHTTP.cs

namespace PizzaRemotingClientHTTP
{
using System;
using System.Web.UI;
using System.Web.UI.WebControls;
using System.Collections;
using System.Runtime.Remoting;
using System.Runtime.Remoting.Channels.HTTP;
using MyDiner;

public class RemoteClient : Page
{
public RemoteClient() {

connectToPizzaRemotingService();

}//constructor

private void connectToPizzaRemotingService() {

HTTPChannel chan = new HTTPChannel();
try {
// Register the HTTP Channel
ChannelServices.RegisterChannel(chan);

// Create an instance of the remote Diner object
Diner dinerObj = (Diner)Activator.GetObject(typeof(Diner), & _
"http://bingo:80/MyDiner/Diner.soap");

// Get Todays Menu from the remote Pizza Service
if(dinerObj != null)
m_listPizza = dinerObj.getPizzaMenu();

//Unregister the HTTP channel
ChannelServices.UnregisterChannel(chan);

}//try
catch(Exception e) {

//Unregister the channel
ChannelServices.UnregisterChannel(chan);

}//catch

}//connectToPizzaRemotingService

// Data Source that will be bound to the repeater
public ICollection PizzaMenu {
//Property Accessor
get {
return m_listPizza;
}
}//PizzaMenu

protected override void OnLoad(EventArgs e) {

base.OnLoad(e);
// Bind the data to controls in this chain if this
// not a postback
if(!IsPostBack) {

DataBind();
}//IsPostback

}//onLoad

public System.Web.UI.WebControls.Repeater pizzaRepeater;
ArrayList m_listPizza = null;

}//class

}// namespace

You can compile the code-behind assembly by using the following command:

csc /out:PizzaRemotingClientHTTP.dll
/target:library
/r:System.Runtime.Remoting.dll,System.Web.dll,MyDiner.dll PizzaRemotingClientHTTP.cs

Make sure that you copy the codebehind assembly ( PizzaRemotingClientHTTP.dll ) in the bin subfolder of the consumer's virtual root. The ASP.NET consumer page uses the connectToPizzaRemotingService method in the RemoteClient object of the codebehind assembly to invoke the object exposed at the URN endpoint http://bingo/MyDiner/Diner.soap?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq and calls its getPizzaMenu method. It then uses an ASP.NET Datalist control to render the ArrayList returned from the getPizzaMenu method.

This is the listing for the ASP.NET page that consumes the Diner Web Service using the code-behind assembly that we just saw in the previous section.

Listing 3: ASP.NET Client page

<%@ Page language="C#" Codebehind="PizzaRemotingClientHTTP" & _
inherits="PizzaRemotingClientHTTP.RemoteClient" %>

.........

<asp:DataList runat="server" id="PizzaList"
RepeatColumns="2" RepeatDirection="Vertical" RepeatMode="Table" DataSource='<%# PizzaMenu %>'
Width="100%">
<property name="AlternatingItemStyle">
<asp:TableItemStyle BackColor="#f5deb3"/>
</property>
<template name="ItemTemplate">
<asp:Panel runat=server font-size="12pt" font-bold="true">
<%# DataBinder.Eval(Container.DataItem, "Name") %>
</asp:Panel>
<asp:Label runat=server font-Size="8pt"
Text='<%# DataBinder.Eval(Container.DataItem, "Description") %>'>
</asp:Label>
<asp:Label runat=server font-Size="10pt" Text="$" ForeColor="Blue">
</asp:Label>
<asp:Label runat=server font-Size="10pt" ForeColor="Blue"
Text='<%# DataBinder.Eval(Container.DataItem, "Cost") %>' >
</asp:Label>
</template>
</asp:DataList>

Method 2: (Using a configuration file on the client to register the channel and specify the object's URN)

Creating and registering the channel explicitly and then using Acivator.GetObject is just one way of invoking the remote object. You can also specify the channel and the URI endpoint of the remote object through a configuration file , and just instantiate the Diner object using new . The client application will use the information in the loaded configuration file to register the channel and activate the object at the URI endpoint specified.

The general format of the client configuration file is as follows:

Name#Name of the Application
Assembly#[AssemblyName]#[RemoteApplicationName]#[FullTypeName]=[ObjectURI]
RemoteApplication#[RemoteApplicationName]#[RemoteApplicationURI]
Channel#[ChannelAssemblyName]#[ChannelFullTypeName]#[Name=Value;Name=Value]

So based on the above format, here's how your configuration file ( MyDinerClient.cfg ) would look like:

Name#MyDinerClientConfig
Assembly#MyDiner#MyDiner#MyDiner.Diner=HTTP://bingo:80/MyDiner/Diner.soap
RemoteApplication#MyDiner#HTTP://bingo:80/MyDiner
Channel#System.Runtime.Remoting#System.Runtime.Remoting.Channels.TCP.TCPChannel
Channel#System.Runtime.Remoting#System.Runtime.Remoting.Channels.HTTP.HTTPChannel

To load this configuration file, we have to use the RemotingServices.ConfigureRemoting method and pass it the name of the client configuration file to use. The information in the configuration file is used to register the channel and use the specified remote URI endpoint to activate the object. You must load the configuration file only once, typically when your application starts up. Shown below is the global.asax file for an ASP.NET consumer application that loads the configuration file.

Listing 4: global.asax used by the ASP.NET consumer application

<%@ Import Namespace="System.Runtime.Remoting" %>
<script runat="server" language="vb">
Sub Application_OnStart()
RemotingServices.ConfigureRemoting(HttpContext.Current.
Server.MapPath("MyDinerClient.cfg"))
End Sub
</script>

So now, since we have all the details specified in the configuration file, we do not have to explicitly register the channel or provide details about the object's URN endpoint in our code. You would simply have to create the object using new and then call its methods as shown below:

// Create an instance of the remote object
Diner dinerObj = new Diner();

// Get Today's Menu from the remote Pizza Service
if(dinerObj != null)
m_listPizza = dinerObj.getPizzaMenu();

If there's an inkling of doubt whether your ASP.NET page is accessing the locally deployed assembly instead of the remote one, simply shut down the remote server or rename the assembly on the remote server and you will see that the client will choke.

Hosting the remote object in a managed .NET application executable

In the last section, we hosted the remote .NET object in IIS and then used an ASP.NET page to invoke the object. Now, let's see how we can host the same object in a managed .NET application executable. Take a look at this snippet of code that shows you how your managed application host should register a channel and expose the remote object at a well known remote URN endpoint.

Listing 5: Managed Host Application - DinerManagedHost.cs

namespace MyDinerHost
{
using System;
using System.Runtime.Remoting;
using System.Runtime.Remoting.Channels.TCP;

class DinerHost
{
public static void Main(String[] args) {

// Create and register the TCPChannel
// Let's listen at port 8080
TCPChannel chan = new TCPChannel(8080);
ChannelServices.RegisterChannel(chan);

// Register the object at a well known URI End point
RemotingServices.RegisterWellKnownType ("MyDiner",
"MyDiner.Diner",
"MyDiner/Diner.soap",
WellKnownObjectMode.Singleton);

// Ready to listen for requests.
// Up and running until the user presses a key
Console.WriteLine("John Doe's Pizza Menu Query Host is up and running");
Console.WriteLine("Press any key to shut me down .....");
Console.ReadLine();
Console.WriteLine("Pizza Service going down. Please Wait .....");

// Unregister the TCP Channel
ChannelServices.UnregisterChannel(chan);

}//Main

}//class

}//namespace

You will also need to place the MyDiner.dll assembly in the same directory as the host application or register it with the global assembly cache so that the host application can locate the assembly that contains the Diner object.

You can compile the source from command line using:

csc /out:MyDinerHost.exe /target:exe /r:System.Runtime.Remoting.dll,MyDiner.dll DinerManagedHost.cs

This time we will try using a TCP channel for our remote invocation. You will notice how the Main entry point in the application registers a TCP Channel, so that it can listen for incoming requests on a specified port. In the earlier IIS hosting example, IIS took care of listening for HTTP requests. Now, since we are hosting the object in our very own hosting application, we need to register the channel and then listen for requests on that channel. In the code snippet that we just saw, the ChannelServices.RegisterChannel method call registers a TCP channel and listens on port 8080 for requests. Then, we expose the remote object at a well-known URN endpoint using RemotingServices.RegisterWellKnownType . The last parameter in RegisterWellKnownType defines the actual activation policy. In this example, we use the Singleton activation policy that uses the same object instance to service every client request. We will take a more detailed look at the various activation policies in the Lifetime management and Activation policies section. So if you use the earlier ASP.NET client to access the service hosted in your .NET executable, you would just need to register a TCP channel in the client instead of a HTTP Channel.

The code snippet below shows how to register a TCP Channel and then activate and invoke a method on the remote object.

// Create and register a tcp
            channel
TCPChannel chan = new
            TCPChannel(8081);
ChannelServices.RegisterChannel(chan);

//
            Create an instance of the remote object
Diner dinerObj =
            (Diner)Activator.GetObject(typeof(Diner),
"tcp://localhost:8080/MyDiner/Diner.soap");

//
            Get Todays Menu from the remote Pizza Service
if(dinerObj !=
            null) m_listPizza = dinerObj.getPizzaMenu();

// Unregister
            the channel
ChannelServices.UnregisterChannel(chan);

Transport Channels

Channels provide an abstraction for the actual conduits that will carry the message across the wire. The HTTP Channel, TCP Channel, and SMTP Channels are provided out of the box by the remoting framework. The HTTP Channel is your best bet if you are planning to deploy your server in an Internet environment and the consumers of your services are outside your firewall. Messages sent through the HTTP channel are by default, SOAP payloads. The TCP channel offers better performance than the HTTP channel. It uses a binary payload by default, but it can also carry SOAP payloads if required. If all the consumers of your service are within your own internal network/LAN and inside the firewall, then the TCP channel can be used since it offers better performance using raw sockets with a binary payload. But if you are looking at deploying the remote object across the Internet, that needs to be accessible by clients or consumers outside your firewall, the HTTP channel is the best option.

The SMTP channel also uses a SOAP payload by default. Eventually, it's all matter of deciding on the tradeoffs between performance and interoperability. The good thing about the remoting architecture is that it does not mandate that you have to use a specific formatter with a particular transport channel. So you could use the Binary formatter with the HTTP channel or use the SOAP formatter with the TCP channel. The pluggable channel & serialization formatter architecture allows you to mix'n'match channels with appropriate payloads to suit your needs & requirements.

To write a custom channel, your channel classes would need to implement the IChannelSender and the IChannelReceiver interface. More specifically, each of these interfaces inherits from the IChannel interface. Your IChannelSender interface implementation should take care of client side chores. You will need to create a message sink that dispatches the message to the server object's URN endpoint after serializing it. At the server end, your IChannelReceiver implementation should listen for requests, deserialize the message received, should make the method call on the object, serialize the response and send it back to the client. The message sink created by the ChannelSender object in the client should then read back the response message and deserialize it. The .NET framework SDK ships with a custom channel sample that uses Named Pipes to communicate between the client and the server. So you might want to take a look at that sample to get a better understanding on how you could put together your own custom channel. You'll find this sample under the Remoting samples folder.

Here is a brief summary of what we discussed:

HTTP Channel | TCP Channel
Uses the SOAP serialization formatter by default. | Uses the binary serialization formatter by default.
Suitable for Internet deployment and works well with firewalls. | More suitable for deployment within an internal network/LAN
Performance is fairly good. | Very good performance since it uses raw sockets and a binary payload.
High interoperability. Channel can be used to talk to a large number of services (deployed even in disparate platforms) that can speak SOAP. | Not as interoperable as the HTTP channel.

Lifetime management & Activation policies

A .NET remote object's host can decide how it wants to manage the object's lifetime and activation policy when it registers the object at the URN endpoint using RemotingServices.RegisterWellKnownType . The lifetime and activation options that are available include SingleCall , Singleton , a n d Client Activated Objects (CAO ).The last parameter in RemotingServices.RegisterWellKnownType takes a WellKnownObjectMode enum value that specifies the mode to use.

  • SingleCall objects don't maintain any state between calls and every call to the object would be serviced by a new instance of the object. So essentially, they are stateless and are perfect candidates for load balancing and would scale-out well if deployed in a server farm.
  • Singleton objects follow the classic Singleton pattern where the same instance of the object is used to service requests from consumers. This pattern is useful when you need to maintain shared data or resources between all clients. For example, say you wanted to maintain a hit counter that tracked the number of hits on a server resource.
  • Client Activated Objects (CAO) use the activation & lifetime pattern that we see in object models like COM. When the client calls new or Activator.CreateInstance to create a CAO object, a remote request is sent in the form of an activation message that is intercepted by the Remoting Activator and the remote object is invoked. An object reference( ObjRef ) is passed back to the client. Every call that the client makes using the ObjRef that was handed over, is serviced by the same object on the server, thus allowing you to maintain state between method calls. This is quite different from the SingleCall and Singleton models because in both those models, the remote object is activated only when the first method call is made on the proxy and not during the actual instance creation.

The procedure for exposing an object as a Client Activated Object (CAO) is slightly different from that of exposing an object as a Singleton or SingleCall object. In the CAO model, we have to indicate the object we are exposing using the Export attribute instead of the WellKnownObject attribute in the remote configuration file that the host will be loading. We already took a look at the WellKnownObject attribute in the remoting.cfg file when we hosted the object in IIS. This attribute needs to be used if you are exposing your object with a Singleton or SingleCall lifetime policy. But if you want to expose the object as a Client Activated Object (CAO) , then you must use the Export attribute and specify the assembly name and the type name of the object that you are exposing as a Client Activated Object.

Here's the configuration file that we would use if we wanted to expose the Diner object as a CAO . In this example, the loaded configuration file contains enough information to tell the host to register a HTTP Channel - so we do not have to register it explicitly from our code.

Listing 6: CAO Example - Configuration file loaded by the remote object host

Name#MyDiner
Export#MyDiner#Diner
Channel#System.Runtime.Remoting#System.Runtime.Remoting.Channels.HTTP.HTTPChannel#ports=8080

If you were to consume this object from a client application and were using the configuration file approach, your client could simply use new to instantiate the object after loading the configuration file (like the one shown in listing 7) by calling RemotingServices.ConfigureRemoting .

Listing 7: CAO Example - Configuration file loaded by the CAO consumer

Name#MyDinerClient
Assembly#MyDiner#MyDiner#Diner
RemoteApplication#MyDiner#HTTP://localhost:8080/MyDiner
Channel#System.Runtime.Remoting#System.Runtime.Remoting.Channels.HTTP.HTTPChannel

The example above assumes that the host application (http://localhost:8080/MyDiner?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq) for the remote object is listening in port 8080 after registering a HTTP Channel and has loaded the configuration file shown above in Listing 6.Alternatively, you could also use Activator.CreateInstance in your client to activate the object by specifying the URN of the application hosting the object.

Here's a summary of the various remoting lifetime management/activation policies:

SingleCall | Singleton | Client Activated Object
Every call to the object creates a new object instance to service the call. | Same object instance is used to service all client requests. | Very similar to the COM programming model. When the client unmarshals an object reference & makes a method call on that reference, it will get serviced by the same object instance in the server, thus allowing you to maintain state between method calls.
Will scale-out well when deployed in a server farm. | Will not scale-out in server farm. | Will not scale-out as well as the SingleCall model.
Stateless model | Stateful model | Stateful model

Serialization Formatters

Before an object can be sent across the transport channel, it needs to serialized and packed into a data format that can be transmitted across the wire. At the other end of the wire, the serialized data is read and deserialized back to the actual object or type that it represents. The Serialization is done by message-serialization formatters that convert the field or object's state into a format that is suitable for storage or transmission. The .NET framework provides you with two message-serialization formatters out-of-the-box. They include a binary serialization formatter which converts an object's state into a binary stream and a SOAP serialization formatter which converts it into an XML string representation that's human readable. The binary serialization formatter is slightly faster. But the SOAP serialization formatter is the way to go when talking to other web services in the internet deployed in other platforms that typically accept and respond only to SOAP conformant requests from consumers. Typically, you need not be concerned about how the remoting runtime packs the method call and parameters into a SOAP envelope during serialization. The HTTP channel's default SOAP message formatter automatically does this for you. You could use an application like tcpTrace (see the Related links section for more information on how you could obtain and use tcpTrace ) to snoop in on the SOAP packets being sent across the channel which could help you to track problems during debugging.

Conclusion

As I write this article, there is so much of excitement in the internet community about Microsoft's unveiling of the Hailstorm initiative with a vision for anytime, anywhere, user-centric based Web Services.

Everyday a huge number of business organizations are jumping onto the Web Services bandwagon to deliver software as a service by exposing their business processes as Web Services. They also collaborate and aggregate with other Web Services. This allows them to a complete a business process by leveraging existing infrastructure provided by other tested & proven web services that are specialized in their own business domains. The power of XML and SOAP together with initiatives like WSDL (Web Services Description Language) make it easy for applications to connect to web services at communication endpoints that are deployed in other platforms, thus enabling disparate applications on different platforms to talk to each other and leverage each other's services seamlessly. Furthermore, standards like the UDDI (Universal Description, Discovery and Integration) allow businesses to discover web services that provide the kind of services and offerings they are looking for. The remoting framework that's provided with the .NET platform makes it easy to create and deploy web services. It provides the infrastructure you need such as various hosting options, message formatters, transport channels, lifetime management and activation policies. The .NET remoting building blocks that we saw in this article will help you create robust and scalable web services that meet the needs of your business process. Hopefully, this article will get you started in your journey in discovering the richness and robustness of this powerful architecture.

C#Today - © 2002 WroxPress Limited


.NET Garbage Collection Part 2

.NET Garbage Collection Part 2
http://www.csharptoday.com/content.asp?id=1605
Article Rating: 4.5
Published on July 3, 2001
Abstract
In part 1 of this article, Kaushal Sanghavi discussed the basics of the C runtime heap and the drawbacks of this heap. Further, he discussed how the managed heap works in the .NET runtime and covered the basics of garbage collection. Garbage collection is a CPU bound process that can cause a significant impact on performance. However, the .NET runtime implements quite a few optimizations to improve the speed and efficiency of the garbage collector. In this article, we will explore some of these optimizations and discuss some of the advanced topics related to garbage collection.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

In part 1 of this article (http://www.csharptoday.com/content.asp?id=1578&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq), we discussed the basics of the C runtime heap and the drawbacks of this heap. Further, we discussed how the managed heap works in the .NET runtime and covered the basics of garbage collection. Garbage collection is a CPU bound process that can cause a significant impact on performance. However, the .NET runtime implements quite a few optimizations to improve the speed and efficiency of the garbage collector. In this article, we will explore some of these optimizations and discuss some of the advanced topics related to garbage collection.

Generations

When the managed heap runs out to space to allocate new objects, the Garbage Collector (GC) starts a Collection. A collection is the process wherein the GC detects all free objects, reclaims their memory and compacts the heap by moving allocated memory into one contiguous section of the heap. However, detecting and freeing up all objects and compacting the entire heap can be a very expensive operation. Under the most common situations, reclaiming and compacting only a portion of the heap would suffice, and this would not be as expensive as compacting the entire heap. The .NET garbage collector uses the concept of Generations to achieve this.

Put very simply, a generation is a reflection of how old an object is. When a new object is allocated, it is said to belong to Generation 0. When the GC performs a collection, objects that are in use are not collected. These in use objects are said to have survived the collection and they move from Generation 0 to Generation 1. Similarly, the next time there is a collection, some of these Generation 1 objects will still be in use and they are now promoted to Generation 2. Once an object is in Generation 2 it stays there until it is considered garbage. Currently, the .NET GC supports generations 0, 1 and 2. Every time an object survives a collection, it is moved to the next generation. When a collection takes place, the GC only looks at generation 0 objects and if reclaiming and compacting generation 0 objects frees up enough memory, all other objects belonging to different generations are ignored. The rationale behind this is that most objects are created, used and destroyed. If an object has been alive for a long time, it usually implies that the object serves a long lasting function in the application and will probably be in use for a long time. Given this assumption, it makes sense to only look at newer objects because the probability of reclaiming newer objects is higher. This is a very simple optimization but is applicable to most situations and can derive significant gains. However, if the GC cannot find enough generation 0 objects to clean up, it will look at generation 1 and generation 2 objects till it can find enough memory to reclaim. In fact the GC can follow any algorithm to determine the generation it wants to reclaim first.

Using generations, the GC can start working with a subset of the heap and hope that the memory reclaimed from this subset is sufficient to process the pending requests. The use of generations should be entirely transparent to the user and to your application. However, if an object needs to know what generation it belongs to, it can use the GetGeneration method exposed by the System.GC class. Further, the GC class supports the MaxGeneration property which returns the maximum number of generations that are currently supported. The Beta 2 release of the .NET framework supports up to 3 generations (0, 1, and 2). Although the GC class does support these methods to determine the GC details, developers are strongly recommended to not make any assumptions about how the GC treats collections and generations, since these algorithms are subject to change in future releases.

Finalization

Many objects hold on to resources for the entire lifetime of the object. A common example would be an object that deals with a database or a file. Such an object would create a connection to the database or open a file in its constructor and then hold on to the connection or the file handle till the object was destroyed. C++ provided a cleanup method for objects called the destructor and VB provided the Class_Terminate method which would be called when an object is destroyed. The object can then perform cleanup in this method, which would typically involve freeing up memory and releasing any resources (e.g. Database connections, file handles etc) that were acquired by the object during its lifetime. The .NET runtime provides a similar method called Finalize that is invoked by the GC when the object is being reclaimed. This method is optional and the GC determines whether the object being reclaimed supports this method or not. If the method is found, the GC will invoke this method to allow the object to clean up after itself.

Note that the Finalize method is not really equivalent to a destructor in C++ or the Class_Terminate in VB. These methods are called immediately the object is destroyed. However, in .NET an object is not necessarily destroyed as soon as the last reference to it is released. It is quite possible for an object to have no outstanding references, but still be alive for a considerable period of time before a collection occurs. Note that in .NET, objects are truly destroyed only when the GC starts a collection. Thus, the Finalize method might not be the ideal method to put cleanup code in. Certain resources like database connections are expensive and should be immediately released when they are no longer needed. Since, there is no knowing when the Finalize method will be invoked by the GC, it is not recommended to perform this kind of clean up in the Finalize method. Instead, the object should provide another method called Dispose which is similar to a destructor. When your application is about to release its reference on an object, it should call the Dispose method on the object which performs all clean up of the object. Since this method will be called just before the object is about to be released, this is the perfect location for cleaning up expensive resources. In fact, this is as close as you can get to a destructor under the .NET framework.

A Finalize method adds considerable overhead to the GC. Now not only does the GC have to detect deleted objects and compact their memory, if these objects have a Finalize method, the GC has to invoke the Finalize method for each of these objects. Keep in mind that the GC will not be able to complete the collection till the Finalize method returns. Thus, developers should provide a Finalize method only if needed; you should try to use the Dispose method instead.

Note that the GC cannot afford to wait for the Finalize method to return before it can continue and as a result when the GC encounters an object with the Finalize method, it starts another thread to call this method. While this thread will invoke the Finalize method for all objects, the GC continues with the process of reclaiming and compacting memory. However, it cannot reclaim objects before their Finalize method is invoked, and as a result it will ignore these objects till the next collection. The thread invoking the Finalize methods will execute between collections at a lower priority than other threads. Objects with a Finalize method thus need two collections to be completely destroyed. However, if your object supports a Dispose method and this method is invoked by another object in the application, the GC need not call the Finalize method during the collection process. Again, the assumption is that since the object has already been cleaned up, there is no need to run the clean up process again. Objects can communicate this information to the GC by using the SuppressFinalize method. This lets the GC know that the object has cleaned up and can be immediately reclaimed without needing to invoke the Finalize method.

   public void Dispose
            ()
{
//clean up here

// after the cleanup, suppress the
            Finalize call
GC.SuppressFinalize (this);
}

Note that the C# compiler supports a destructor like syntax for writing a Finalize method. Thus, the following two lines of code are equivalent, but the Beta 2 version of the C# compiler will force you to use the destructor syntax.

// the explicit Finalize method declaration
override protected void Finalize ()

// the C++ like destructor syntax
~MyClass ()

Resurrection

The .NET GC uses Application Roots to determine if an object is accessible from another object. If it is accessible, then there exists a reference to this object and it should not be reclaimed. However, when there are no references to an object, the object cannot be accessed by the application and the GC can reclaim this object. Now if this object supports the Finalize method as we discussed in the above section, the GC will have to call this Finalize method. It is possible that in this method, an object might place a reference to itself in another object. Once this is done, a reference to the current object does exist and the object is now in-use. Since the object is in-use, the GC cannot collect this object. This feature of an object to stop the GC from collecting itself during the collection process is called Resurrection. Once an object is resurrected, all objects that this object maintains a reference to will be resurrected as well. However, this is not recommended and can result in unexpected situations. The order in which the GC finalizes objects and reclaims objects is not fixed. Thus, it is possible for an inner object to be reclaimed before the outer object can be reclaimed. Thus, if an object is being resurrected, it might be possible that an object that it refers to is no longer in existence, and all references to that object might result in exceptions.

public class GlobalObjectContainer
{
public static Resurrection mResurrectionObj;
}

public class Resurrection
{
public Resurrection ()
{
mInnerObject = new InnerObject ();
}

// destructor syntax used for the Finalize method definition
~Resurrection()
{
// resurrect this object
GlobalObjectContainer.mResurrectionObj = this;
}

public void DoStuff ()
{
// after resurrection, there is no guarantee
// that this object will be alive. the inner object
// could have been finalized before the outer object

mInnerObject.foo ();
}

private InnerObject mInnerObject;
}

public class InnerObject
{
public void foo ()
{
// do stuff
return;
}
}

In the sample code above, if the Resurrection object decides to resurrect itself, it might be in trouble. This is because the mInnerObject member object might have already been finalized. If this is so, then calling the foo method on this object could have unpredictable results. Due to this, developers are not recommended to use resurrection, and if they do, you have to be very careful about the state of the objects that will be accessed from your resurrected object.

Strong and Weak References

When an object is accessible from an application root, the GC will not reclaim the object. Such a reference is called a Strong Reference, because it guarantees that the object will not be reclaimed. Similarly, when there are no existing references on an object, the object becomes a candidate for collection. This works fairly well for small objects that can be easily created or destroyed. However, consider an object that occupies a lot of memory, but can be easily recreated. A perfect example is an object that loads some lookup tables from a database. These lookup tables can grow to be quite large, and as a result this object could occupy a large amount of memory. However, if this object were destroyed, reconstructing this object is fairly simple. All you need to do is to establish a connection with the database and load the lookup table into the object. If the system has the memory available to always keep the tables loaded, the application would benefit from a significant performance boost. However, if memory is getting scarce, this object is fairly expendable.

The .NET GC supports exactly these kinds of objects that can occupy a lot of memory but also can be released if the memory demands get high. Such objects are called Weak References, and the GC is free to collect these objects if it so desires. Your application can create an object and mark it as a weak reference, thereby allowing the GC to reclaim the object if necessary. Whenever your application tries to access this object, it should ensure that the object does indeed exist, and if it does exist, your application can invoke methods on the object. However, before invoking a method on the object, you should convert the weak reference into a strong reference by assigning the object to a local variable. This is necessary to ensure that the object is not reclaimed in the middle of processing a request.

public void DoStuff ()
{
LargeObject
            lLargeObj = new LargeObject ();
WeakReference lWeakRef = new
            WeakReference (lLargeObj);

// delete the only strong
            reference to the object
// this is a candidate for collection
            now
lLargeObj = null;

// attempt to create a strong
            reference from the weak reference
lLargeObj = (LargeObject)
            lWeakRef.Target;

// if the object has not been reclaimed yet,
            use the object
// else create a new object and use the new
            one
if (lLargeObj != null)
lLargeObj.foo
            ();
else{
lLargeObj = new LargeObject ();
lLargeObj.foo
            ();
}
}

Sample Code

There is an example of this in the sample code. Download and run the executable to see:

Unmanaged Code and Unsafe Code

All our discussion on the GC has dealt with managed code and the assumption that the .NET runtime has the luxury to relocate objects, and the ability to deterministically detect an object's type. These assumptions hold true as long as we are dealing solely with safe managed code. However, .NET allows you to write unsafe managed code in C# or Managed C++ that uses pointers and pins down objects to a fixed location in memory. Further, your managed code can interact with unmanaged code using COM Interop or PInvoke. When dealing with unsafe or unmanaged code, the GC cannot detect the object types and it cannot relocate objects in memory.

public unsafe void strcpy (char[] iSrc,
            char[] oDest)
{
int lSrcLen = iSrc.Length ;
int lDestLen =
            oDest.Length ;
int lCopyCount = lSrcLen > lDestLen ? lDestLen
            : lSrcLen;

fixed (char* pSrc = iSrc, pDest =
            oDest)
{
// if the GC is invoked while this block is
            executing,
// the GC will be suspended till the
// fixed
            block has finished execution

char* lSrc = pSrc;
char*
            lDest = pDest;

for (int i = 0; i < lCopyCount;
            i++)
*lDest++ = *lSrc++;
}
}

Consider the above piece of unsafe code in C#. This is a simple string copy function that is written using pointers. If the GC was invoked while the fixed block was executing, and the GC relocated the iSrc and oDest objects, the pointers to these objects could now be pointing to a random area in memory. This is obviously unacceptable. To avoid this problem, the GC suspends all threads that are in a "safe state" before it starts a collection. A safe state is a state where the .NET runtime has complete control over the thread's execution. A thread executing in unsafe or unmanaged code is not considered to be in a safe state. The GC will wait for all threads to reach a safe state before it starts a collection. It is thus possible to write unsafe code that can affect the performance of the entire system. Consider a block of unsafe code as below:

public unsafe void Delay ()
{
fixed (int* lPtr = null)
{
Thread.Sleep (5000);
}
}

This could delay the GC execution by 5 seconds if a GC starts a collection just as a thread enters the fixed block. Developers should keep this in mind as they write unsafe code or make calls into unmanaged code.

The exact algorithm that the .NET runtime uses to determine safe state differs slightly from the above discussion and this is another area where the details are subject to change in subsequent releases.

Conclusion

Garbage collection is an expensive operation, but the .NET GC implements numerous optimizations to improve this process. While the .NET runtime can perform these optimizations internally, it does need cooperation from the developer. As discussed in this article, you should be very careful when you write unsafe code, or finalizable objects or resurrectable objects. These objects can have a severe impact on the performance of the GC and subsequently on the entire system.

Happy .NETing.

C#Today - © 2002 WroxPress Limited


.NET Memory Management and Garbage Collection Part 1

.NET Memory Management and Garbage Collection Part 1
http://www.csharptoday.com/content.asp?id=1604
Article Rating: 4.5
Published on July 2, 2001
Abstract
Memory allocation and management has been one of the most troubling areas of programming for many developers for a long time now. This is especially true for languages like C or C++ where the entire responsibility of managing resources rests with the developer. But the common thread that runs through the principles and ideology of the .NET framework has been to make life easy for developers - all peripheral and plumbing issues should be handled by the runtime, and Memory management is no exception to this rule. We will explore the details of the .NET Garbage Collection scheme in a two part article. In this article, Kaushal Sanghavi covers the basic details of memory management under .NET, and in the next article, we will discuss some of the more advanced features of the .NET Garbage Collector.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

Memory allocation and management has been one of the most troubling areas of programming for many developers for a long time now. This is especially true for languages like C or C++ where the entire responsibility of managing resources rests with the developer. The developer has to ensure that all resources are allocated before use, used properly and released after they had outlived their purpose. As trivial as this may sound, I have yet to meet a developer who got this right the first time around. Developers tend to use resources before they are allocated or after they are freed up, forget to allocate resources or forget to free up resources, or all of the above! Further, this category of bugs is worse than any other category, because the results are totally unpredictable and cannot be consistently reproduced. I have been in many situations, where my code worked perfectly all the way through development and testing, but would throw the infamous GPF (General Protection Fault) only at a client site or during an important demo.

The common thread that runs through the principles and ideology of the .NET framework has been to make life easy for the developers. Developers should concentrate on writing the logic that their application is based upon; all peripheral and plumbing issues should be handled by the runtime. Memory management is no exception to this rule. The .NET framework completely absolves the developer from having to worry about memory allocation and deallocation. The .NET runtime contains a special object called the Garbage Collector that does this for you. The garbage collector ensures that only valid objects will be accessible from your program, and once an object is no longer accessible from any object, the garbage collector can automatically detect this and free up its resources. We will explore the details of the .NET Garbage Collection scheme in a two part article. In this article, we will cover the basic details of memory management under .NET, and in the next article, we will discuss some of the more advanced features of the .NET Garbage Collector.

Traditional Memory Allocation

Traditionally Windows programs use either the Win32 heap or the C runtime heap for dynamically allocated memory. The malloc, free, new and delete operators use the C runtime heap and the GlobalAlloc function uses the Win32 heap. Since most of the concepts are the same for both heaps, we will discuss the C runtime heap in this article. The biggest problem with using the C runtime heap has been its efficiency and the fact that it fragments memory. In this section, we will discuss the C runtime heap, as this will help you understand the benefits of the .NET memory allocation scheme and the garbage collector.

The C runtime heap is a linked list of free memory blocks. The size of the blocks can vary; let us assume that the initial size of each block is 1024 bytes. A heap with 4 free contiguous blocks is shown in the above diagram. Now every time, there is a request for memory (this could be via the new or malloc operation in C, C++ or VB, or a CreateObject call in VB, or any other call that requires allocating memory), the heap manager scans through the entire heap, skipping over allocated blocks till it finds a block that can fulfill the request. For example, let us suppose that the heap manager is asked to allocate 1000 bytes of memory. In this case, this is trivial since the very first block has the required memory.

The heap now looks like the one above. Now imagine another request for 2000 bytes. Again, the heap manager has to start looking from the beginning of the heap for a block of size 2000 bytes.

The heap looks like the one above. As you can see, this is a slow process since the heap manager starts looking at the head of the list, and it has to skip allocated blocks till it finds a free block of the required size. Further, as you can see from the above diagrams, there are small chunks of memory that are free but they get skipped so that memory can be allocated on a block boundary. Thus, 24 bytes in the first block are free but the heap manager did not use these 24 bytes when it needed a 2000 byte block. Further, allocation of the 2000 byte block causes 48 bytes to remain free in the third block. This causes heap fragmentation, because there are small blocks of unallocated memory that are not being utilized. Further, let us assume that there is another request for a 1024 byte block.

If the entire heap consists of only 4 blocks each of 1024 bytes, and the next request requires 72 bytes of memory, the request will fail with an "Out of memory" error. This is the result of fragmentation, 72 bytes are available, they are just not available together and there is nothing the C runtime heap can do about it. While this seems to be a trivial example, there is a very real issue in a system that allocates and deallocates memory rapidly. This can cause the heap to be very badly fragmented, and requests for memory could be denied in spite of there being more than the desired free memory.

.NET Memory Allocation

As we discussed in the above section, the C runtime heap has performance and fragmentation issues. This is partly due to the fact that the C runtime heap allows pointers to memory that can be changed arbitrarily. For example, the C and C++ languages allow pointers of type void*, which can point to any possible object of any possible type. Thus the heap cannot deterministically guess the type of object this pointer points to just by looking at the pointer. Further, since pointers are allowed, the location of the object in memory has to be fixed, and the heap cannot arbitrarily relocate objects.

The .NET framework tries to solve these issues by using a managed memory allocation scheme. Since managed code does not allow pointers, and since the .NET runtime only allows strongly typed objects, the .NET heap does not face the same issues as the C runtime heap. A key point to note here is that since .NET does not allow direct memory access via pointers, the .NET runtime is free to relocate objects in memory as long as all the variables that point to this object are correspondingly updated. This allows the .NET runtime the luxury to make two key assumptions:

  • Heap size is infinite: The .NET heap assumes that the heap is infinite, and as a result it only maintains a pointer that points to memory directly after the last allocated memory block. Compare this to the C runtime heap, which maintains a pointer to the head of the heap. Thus, every time there is a request for a new memory block, the .NET heap does not scan through the entire list of allocated blocks to find a free block. Even though there might be a free block in the middle of the heap, the .NET heap will always allocate memory at the end of the heap. This is a huge performance gain, since all that the heap does is allocate memory and move the pointer to point beyond the newly allocated object. Memory allocation thus requires a single pointer access and no heap scans.
  • Objects can be arbitrarily relocated: Since no pointers are allowed in managed code, the .NET runtime can relocate objects and update the variables that point to the object. This is not as difficult as it sounds, because the JIT compiler (the Just In Time compiler that converts MSIL code to native code) already has a table to all object references. Relocating objects allows the .NET runtime to move objects around in the heap to avoid heap fragmentation. Further, it is this ability that allows it to avoid looking for free spaces in the middle of the heap, since it can make the assumption that eventually the heap will be compacted and the free memory in the middle of the heap will be reclaimed.

Let us assume a similar heap as in the above section. The difference now is that the heap is not a linked list of available memory blocks, but a single contiguous chunk of memory with a pointer pointing to the next free location. When the heap starts, the entire heap is free, so the pointer points to the beginning of the heap. Now let us see what the heap looks like after 2 allocations, one of 1000 bytes and another one of 2000 bytes.

As you can see, the memory is allocated contiguously and the free pointer keeps moving down the heap. Now let us assume that your program releases the 1000 byte object and its memory is free. The heap now looks as follows.

As you can see from the above diagram, even though the 1000 bytes are released, the free pointer does not move to the head of the list. This is one of the basic principles of the .NET memory allocation scheme; the free pointer will always point beyond the last allocated object. Further, if there is another request for a 1000 byte object, the object will be allocated from the end of the heap, the first free 1000 bytes are ignored.

We now end up with a heap that can only allocate an object that requires less than 96 bytes. Since the heap managed will only look at the end of the heap to allocate memory, it completely misses the first free 1000 bytes and it will not reuse that block. Thus if there is a request for a 500 byte object, the heap manager will realize that there is no more memory left at the end of the heap and it starts a special process called Garbage Collection.

.NET Garbage Collection

Put very simply, garbage collection is a scheme of finding objects that are no longer in use and reclaiming their memory. This is a key function of any garbage collector; since the burden of freeing memory no longer rests on the developer's shoulders, someone needs to do this. Further, once this freed up memory is reclaimed, the Garbage Collector (GC) needs to compact the memory so that we don't have any free holes of memory in the heap. The heap is always divided into two contiguous blocks; one contiguous block of free memory and one contiguous block of allocated memory. The GC is responsible for maintaining these blocks in the heap. Thus, in the above example when the heap manager cannot find free space to allocate memory, it invokes the GC to perform a Collection. A Collection is the process of reclaiming free objects and compacting the heap.

A Collection starts with the GC identifying all free objects that need to be reclaimed. The GC starts this process by assuming that all objects in the heap are garbage. It now tries to build a list of objects that are in use by an application and any object that is not in this list is deemed to be free and is released. Identifying the objects that are in use by an application is not simple. Before we discuss how the GC determines in-use objects, let us think about what an in-use object means. Any object that is in use needs to have a reference to it; this is the only way you can access the object from your application. Thus if you have a local variable in a method that points to an object, the object cannot be freed by the GC, since you could use the local variable to invoke methods and properties on that object, and the object better be alive to service these requests. The GC uses a similar algorithm to determine in-use objects. All global variables, class members, local variables in the current executing method, parameters to the current executing method are called Application Roots. An application root is a pointer to an active in-use object. The JIT compiler maintains this list of application roots internally and can pass it on to the GC during a collection. The GC starts with each member in the application roots and finds all objects that can be accessed using this root. This is an iterative algorithm since a root could point to an object, which in turn points to another object, which again points to a third object. The GC traverses this object graph and marks all the in-use objects for a given root. Once this process is complete for all the roots in the application, the GC has a list of all active objects. If an object does not have another object pointing to it, or a variable pointing to it, this object will never be accessed from your application and is a candidate to be reclaimed. Using this algorithm, the GC can find all freed objects and reclaims the memory. Further, since the GC has access to the JIT compiler, it can access all the roots and update their values to point to the newly relocated objects. This is entirely transparent to the application, and the variable need not even know that a collection just took place.

After a collection, our sample heap will look as shown below.

During a collection, the GC relocates Object B and C to the beginning of the heap, so that all the free space now lies contiguously at the end of the heap. The free pointer now points to 1096 bytes free and thus the request for the 500 byte object can now be satisfied.

However, if there is another request for a 1000 byte block, the request will be denied with an OutOfMemoryException. This is because there is no free space available and even a collection will not be able to find free memory that can be allocated.

System.GC

The GC is invoked by the .NET runtime when the heap is full and the current memory allocation request cannot succeed. Apart from a slight hit to performance (since .NET is still in a Beta mode, we do not have concrete numbers representing the performance hit), the GC process should be entirely transparent to the user and the application. Further, note that there is a single GC and heap for all managed applications, and thus a collection affects all applications. As you can deduce from the above sections, there is a considerable amount of work that needs to be done to identify free objects, to relocate objects and to update the roots to point to the relocated objects. As a result, the GC should rarely be invoked, and applications should rely on the runtime to manage the GC process. However, there might be situations where your application possesses enough knowledge to justify explicitly invoking a collection. For example, if you are writing a networked application service, and suddenly your application detects that the network is down; it could use this opportunity to tear down all its internal structures and force a collection. There are many other situations where your application releases a lot of memory and wants to be a good managed citizen by explicitly invoking the GC. The .NET framework provides the GC class in the System namespace that allows you to explicitly interact with the GC. The following are some of the relevant methods in the class.

  • GetTotalMemory: This method returns the total amount of allocated memory. Note that this might not be exactly equal to the real allocated memory, since some objects might be freed up in the next collection. Till the GC determines an object to be free, its memory is included in this count.
  • Collect: This method should be called to force a collection. Note that this should only be called when it is really necessary to invoke the GC. Calling this repeatedly could bring the entire system to its knees. Further, when the GC is invoked, all other applications are suspended, so this should be called only when necessary.
  • KeepAlive: As we discussed in the above section, the GC uses application roots to determine in-use objects. This works well with managed objects. However, if your application is passing a managed object to an unmanaged object, via COM Interop or PInvoke, it is possible that the object is in use, but there is no managed root pointing to this object. This method can be used to communicate this information to the GC. This ensures that the object is treated as in-use and kept alive even in the absence of a managed root.

public class GCHelper
{
public void Allocate (int iSize, int iNum)
{
byte[] lObject;

for (int i = 0; i < iNum; i++)
lObject = new byte[iSize];

return;
}

public long TotalMemory
{
get
{
return GC.GetTotalMemory(false);
}
}

public string Collect ()
{
DateTime lBeforeTime = DateTime.Now;
GC.Collect ();
DateTime lAfterTime = DateTime.Now;

TimeSpan lDiff = lAfterTime - lBeforeTime;
return lDiff.TotalMilliseconds.ToString ();
}
}

The above sample shows a very simple sample of how you could invoke the GC methods directly. The following screenshot shows how you could allocate any number of objects of a given size and use the GC methods to determine the total amount of allocated memory. Further, you could force a collection and see how long the collection takes. In all our sample runs where we allocated anywhere between 1000 to 10,000 objects of size varying from 1000 to 10,000 bytes, the time for an induced collection was less than 15 ms. However, this could be due to the fact that the system has already performed a collection to satisfy the allocation request.

Conclusion

The C runtime heap has its share of issues that the .NET Garbage Collector tries to avoid. However, a collection is a CPU intensive operation that can freeze other applications and cause performance degradation. In this article, we have discussed the very basic concepts of how memory is managed in the .NET world, and how the GC functions. In the next article, we will discuss the numerous optimizations that the GC implements to ensure that a collection is as quick and efficient as possible. Further, we will discuss some of the advanced features of the GC such as Finalization, Resurrection, Generations and Weak References.

Happy .NETing.

C#Today - © 2002 WroxPress Limited


Efficient String Manipulations with the StringBuilder Class

Efficient String Manipulations with the StringBuilder Class
http://www.csharptoday.com/content.asp?id=1592
Article Rating: 3
Published on June 22, 2001
Abstract
String manipulation is one of the most important operations in any programming language. In C#, Java or even in Visual Basic, there is the String class that represents a piece of text. However, in those languages string operations using the String class are slow and inefficient. In the .NET platform there is a much better way of performing some of the most important string operations with the StringBuilder class, a class not so popular as the String class but will give you an edge once you master it. In this article, Budi Kurniawan shows you how to use it in C#.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

String manipulation is one of the most important operations in any programming language. In C#, Java or even in Visual Basic, there is the String class that represents a piece of text. However, in those languages string operations using the String class are slow and inefficient. In the .NET platform there is a much better way of performing some of the most important string operations - by using the StringBuilder class. This is a class not so popular as the String class but will give you an edge once you have mastered it. This article shows you how to use it in C#. The C# compiler is currently at its public beta version and is included in the .NET Framework SDK, downloadable from http://msdn.microsoft.com.

How many times have you programmed string manipulation code similar to the Count class in Listing 1?

Listing 1: String manipulation using the String class

using System;

public class Count
            {
public static void Main(String[] args) {
String s = null;
for (int i=1; i<=5; i++) {
// concatenate the String
            s
s += i + ",";
}
Console.WriteLine("Counting to five : " +
            s);
}
}

Everything looks normal, doesn't it? But, have you ever noticed how long this operation takes to complete? And, have you noticed that the following string manipulation in the Count class is very inefficient and should be avoided?

    String s = null;
for (int i=1; i<=5; i++) {
// concatenate the String
            s
s += i + ",";
}

In a short and simple Count class like the above, the time lapse is probably too short to measure. However, in a time-critical program such as an Internet application where even one millisecond matters, the code for the string manipulation like above is a big No-No. Why?

Once a String object is created by instantiating the String class, the resulting String object is immutable - which means that once the String object is created, the characters contained in the string cannot be modified. When you do change the string, say by concatenating smaller strings like the one in the Count class or by using one of the methods of the String class, the operation you perform will return a new String object, rather than the same instance the operation is performed on. Creating a new object is a time-consuming and resource extensive operation. Fortunately, you can avoid the overhead and inefficiency of the immutable String object by using a StringBuilder object.

The StringBuilder Class

The .NET platform provides the sealed (meaning not inheritable) StringBuilder class in the System.Text namespace, which is to be found in the assembly mscorlib.dll . The StringBuilder class is a companion class for String, and StringBuilder objects are not immutable. Operations on a StringBuilder object will result in a modified version of the same instance of object. No time is wasted creating a new object. The StringBuilder class provides useful methods such as Append() and Replace() . A problem with C# developers is that they get too used to using String . It would be beneficial to learn what StringBuilder can do and make it a habit to use StringBuilder to perform string manipulations.

For instance, rather than concatenating strings, the above Count class can be re-written as the code in Listing 2, utilizing the Append() method of the StringBuilder class.

Listing 2: String manipulation using the StringBuilder class

using System;
using
            System.Text;
public class Count {

public static void
            Main(String[] args) {
StringBuilder sb = new
            StringBuilder();
for (int i=1; i<=5; i++) {
// concatenate
            the String
            s
sb.Append(i).Append(",");
}
Console.WriteLine("Counting
            to five : " + sb);
}
}

Believe it or not, when the compiled Count.exe is run, this new code runs at least 30% faster than the previous one. (You can do your benchmark by first performing the string operations, and then a similar manipulation that uses StringBuilder .) By any good programmers standards, 30% faster is a big saving!

Therefore, to concatenate strings, instead of saying:

s = "a piece of string" + anotherStringVariable;

It would be better to use the StringBuilder class.

The Benchmark Testing

Any claim that StringBuilder is faster than String will be baseless without a benchmark testing. For this article, I carried out two tests. The first one compares the time taken for string manipulations using String and StringBuilder objects . The second test compares a StringBuilder object that is instantiated with no argument constructor and a StringBuilder object that is instantiated with an initial capacity of 100 (for more details about the constructors, see the following section "The StringBuilder Class's Constructors"). Both results are displayed in Table 1 below.

Length (characters) | String | StringBuilder() | StringBuilder(100)
10 (j<5) | 400,612 | 200,306 | 200,306
20 (j<10) | 801,224 | 400,612 | 400,612
35 (j<15) | 1,201,836 | 600,918 | 500,765
50 (j<20) | 1,602,601 | 701,071 | 600,918
95 <j<35) | 3,505,049 | 1,402,142 | 1,101,683

Table 1: Benchmark testing result

Graph 1, below, also demonstrates the performance gains you can achieve when using StringBuilder :

Graph 1: Benchmark testing result

One thing crucial in benchmark testing is the tool to measure the time. It must be precise enough and must by no means affect the testing process. In VB we have the Timer function that we can use to measure the time. In C#, there is no such a function, however we're in luck because the System namespace of the .NET Framework provides the DateTime structure. One of its properties is the

static Now that retrieves a DateTime representing the current date and time. Another property is Ticks that retrieves the 100-nanosecond tick count for a DateTime instance. Combining these two, we can get a Timer with 100-nanosecond precision.

DateTime.Now.Ticks

By taking measurement before the string manipulation and after, and calculating the difference, we get

the time for that string manipulation. The other issue when doing benchmark testing is find a computer slow enough to enable us to measure the time taken for an operation. Having only a Pentium 550MHz PC with 256MB of RAM, I have to create a for loop (5000 times) to get something measurable (otherwise it will simply return with zero).

    for (int i=1;
            i<=5000; i++) {
// the code for string manipulation
            here
}

The second loop is for string manipulation to build a string with a length of x characters, where x is

10, 20, 35, 50 and 95. Changing the j <y part of the second for loop does this. Therefore, using

a string of 10 characters long, we would program:

      for (int
            j=0; j<5; j++) {
// the code to concatenate the string
}

The first test is displayed in the second and third column of Table 1. It shows that it takes 400,612 normalized time units to build a string of 10 characters using String but only takes 200,306 if we use the Append method of StringBuilder . The longer the string built, the greater the difference is. For a string of 95 characters, for example, the difference is about 2.5 times.

The second test is displayed in the third and fourth column. The third column is the time taken when using StringBuilder instantiated with a no-argument constructor. The fourth column is StringBuilder instantiated with 100 characters as it initial capacity. This testing result shows that there is no difference between the two StringBuilder for building a string of 10 characters. This is consistent with the theory that the initial capacity is 16, so no CPU cycles are spent to allocate more room, as the StringBuilder grows bigger. The 20-character string also shows no difference, even though we expect the no-argument constructor to take slightly more time. However, the time to allocate 4 additional characters is too small to be noticeable. As the StringBuilder appends more characters than its initial capacity, we can see that the one with an initial capacity of 100 takes less time.

The testing code can be found in Listing 3.

Listing 3: The code for the benchmark testing

using System;
using
            System.Text;

public class StringOperations {
public static
            void Main() {
String s = null;
long t1 =
            DateTime.Now.Ticks;
for (int i=0; i<5000; i++) {
s = null;
for (int j=0; j<5; j++) {
// concatenate the String s
s
            += j + ",";
}
}
long t2 =
            DateTime.Now.Ticks;
Console.WriteLine("s = " +
            s);
Console.WriteLine("String: " + (t2 -
            t1));

StringBuilder sb = new StringBuilder();
t1 =
            DateTime.Now.Ticks;
for (int i=1; i<=5000; i++) {
sb = new
            StringBuilder();
for (int j=0; j<5; j++)
            {
sb.Append(j).Append(",");
}
}
t2 =
            DateTime.Now.Ticks;
Console.WriteLine("sb = " +
            sb);
Console.WriteLine("StringBuilder: " + (t2 - t1));

t1
            = DateTime.Now.Ticks;
for (int i=1; i<=5000; i++) {
sb =
            new StringBuilder(100);
for (int j=0; j<5; j++)
            {
sb.Append(j).Append(",");
}
}
t2 =
            DateTime.Now.Ticks;
Console.WriteLine("sb = " +
            sb);
Console.WriteLine("StringBuilder (100) : " + (t2 -
            t1));
}
}

The StringBuilder Class's Constructors

Before you can use StringBuilder , you must create a StringBuilder object using one of its class's constructors. There are six constructors that you can choose from to create your StringBuilder object.

StringBuilder()

This no argument constructor is the one used in the previous code. With this constructor, you allocate an empty StringBuilder with a default capacity of 16 characters. This constructor is most efficient if you don't think you will exceed the 16 characters limit. If the limit is exceeded anyway, some extra steps must be performed so that the StringBuilder can hold more characters. These extra steps include allocating a new character array that is large enough for all characters, copying all the characters from the original StringBuilder and the extra characters, and discarding the old StringBuilder object. These extra steps are expensive and take a lot of CPU cycles to complete. Making sure that you allocate enough capacity for your StringBuilder object (by using some other constructors) is one of the keys to efficient string manipulation. As you have seen, using this constructor is very simple:

StringBuffer sb = new StringBuffer();

StringBuilder( string value )

This constructor lets you initialize the new instance of the StringBuilder class using the string object passed as the argument value.

StringBuffer sb = new StringBuffer( "This is the initial value." );

StringBuilder( int capacity )

If your StringBuilder object will hold more than 16 characters, it is good practice to allocate enough capacity using this constructor. However, choosing a number much larger than your need will result in more memory being allocated unnecessarily for the new StringBuilder object.

Using this constructor, you just need to pass a positive integer for the capacity argument.

StringBuffer sb = new StringBuffer( 32 );

If you exceed the initial capacity, no exception will be thrown, however the operation will incur overhead for allocating a new character array as mentioned above.

StringBuilder( int capacity, int maxCapacity )

This constructor initializes a new, empty instance of the StringBuilder class that starts with a specified capacity and can grow to a specified maximum. An example is as follows.

StringBuffer sb = new StringBuffer( 32, 64 );

If you exceed the maximum capacity by appending more characters than the StringBuilder object can hold, the following exception will be thrown.

System.ArgumentOutOfRangeException: requiredLength

StringBuilder( string value, int capacity )

This constructor allows you to initialize a new instance of the StringBuilder class from the specified string with the specified capacity. The size of capacity can grow as more space is required.

StringBuilder( string value, int startIndex, int length, int capacity )

With this constructor you can initialize a new instance of the StringBuilder class from the specified substring, with the specified capacity. The arguments are explained below.

value

The string that contains the substring used to initialize the new instance. If value is a null reference, the new StringBuilder will contain the empty string.

startIndex

The position at which the substring begins.

length

The number of characters in the substring.

capacity

The suggested starting size of the StringBuilder . This size can increase as more characters are added. The maximum number of characters the current string may contain is set by capacity.

For example, the following line of code will create a StringBuilder object with the initial value of "Hello" and a capacity of 32 characters.

StringBuilder sb = new StringBuilder( "Hello hello world", 6, 5, 32 );

The StringBuilder Class's Methods

The StringBuilder class has the following methods.

Append

This method allows you to append character(s) to the end of the current StringBuilder object. This method has a total of 18 overloads, ranging from the Append method that accepts a single Unicode character as its argument to the Append method that accepts a string of characters. If necessary, you can use multiple Append methods in a single operation, like described in the following lines of code.

StringBuffer sb = new StringBuffer( "this
            ");
String s1 = "is";
int n = 15;
// use multiple Append
            methods here
sb.Append( s1 ).Append( " ").Append( n
);

AppendFormat

This method, with six overloads, replaces zero or more format specifications with the appropriately formatted value of an object. The format specifications are embedded within a string and are of the form {N:FormatString} , where N is a zero-based number indicating the argument that is to be formatted and FormatString is a format string. For example, if the string assigned to format is "You have selected {0:####} units of our products." and the string to be formatted is an integer with the value of 10, then the return value will be "You have selected 10 units of our products."

EnsureCapacity

You can use this method to ensure that the StringBuilder object has enough room for a specific number of characters. The following example only appends the string " is watching. " if the EnsureCapacity method returns a value that is greater than 30.

  StringBuilder sb = new
            StringBuilder( 16, 32 );
sb.Append("The whole world");
int a =
            sb.EnsureCapacity( 30 );
if (a > 30)
sb.Append(" is
            watching. ");

Insert

This method inserts a specified object into the StringBuilder object at a specified position. The Insert method has 17 overloads, allowing you to insert any value type into a StringBuilder object. For example, you can insert a character into a StringBuilder object at a specified position.

using System;
using
            System.Text;
public class StringBuilderInsert {

public
            static void Main(String[] args) {
StringBuilder sb = new
            StringBuilder( 16, 32 );
sb.Append("The world is watching
            ");
sb.Insert(4, "whole " );
Console.WriteLine( sb
            );
}
}

In the StringBuilderInsert class above, a StringBuffer sb is created with a capacity of 16 and a maximum capacity of 32, and then the string "The world is watching " is appended to it. Afterwards, an insert operation is performed using the Insert method. The insert point is at the fourth position and the string "whole " is inserted.

Remove

Complementing the Insert methods, the Remove method allows you to remove a specified string of characters from a StringBuilder object. The signature of this method is as follows.

public StringBuilder Remove( int startIndex, int length )

where startIndex is the starting position where the removal is to be done and length is the number of characters that needs to be removed from the StringBuilder object.

The following example removes the string "whole " from the string "The whole world is watching ".

using System;
using
            System.Text;
public class Remove {
public static void
            Main(String[] args) {
StringBuilder sb = new StringBuilder( 16,
            32 );
sb.Append("The whole world is watching ");
sb.Remove(4,
            6);
Console.WriteLine( sb );
}
}

When the compiled code is run, the following text will appear on the console.

The world is watching

Replace

The Replace method is used to replace all instances of a specified character replaced with a new character or a string with another string. There are four overloads of this method that you can choose that suits your need. The Replace method that can replace all instances of a specified character with a new character has the following signature.

public StringBuilder Replace( char oldChar, char newChar )

where oldChar is the character to be replaced and newChar is the character to replace with. For example, the following Replace class replaces all '|' characters with a blank space.

using System;
using
            System.Text;
public class Replace1 {

public static void
            Main(String[] args) {
StringBuilder sb = new StringBuilder( 16,
            32
            );
sb.Append("The|whole|world|is|watching|");
sb.Replace('|',
            ' ');
Console.WriteLine( sb );
}
}

If you run the compiled file, you will see the following output at the console.

The whole world is watching

Another overload of the Replace method has the following signature.

public StringBuilder Replace( string oldValue, string newValue )

where oldValue is the old string that needs to be replaced and newValue is the replacement string.

The Replace2 class below demonstrates the use of this method.

using System;
using
            System.Text;
public class Replace2 {

public static void
            Main(String[] args) {
StringBuilder sb = new StringBuilder( 16,
            32 );
sb.Append("The whole world is watching
            ");
sb.Replace("whole", "new");
Console.WriteLine( sb
            );
}
}

When the compiled code is run, you should see the following output at the console.

The new world is watching

Notice how "whole" has been replaced by "new"

The two other overloads let you specify the range the character/string replacement should take place. Any character or string that lies outside the range won't be modified.

The StringBuilder Class's Properties

In addition to the methods, Table 2 lists the properties of the StringBuilder class.

Property | Description
Capacity | Retrieves or sets the number of characters the StringBuilder can hold.
[] | The StringBuilder indexer is used to get or set a character at a specific position.
Length | Retrieves or sets the length of the StringBuilder object.
MaxCapacity | Retrieves the maximum capacity of the StringBuilder object.
Table 2: The properties of the StringBuilder class

Summary

In this article we have seen why the String object is not suitable for all types of string manipulations. We have also seen how the StringBuilder class can be used without the overhead and inefficiency of the String class. Therefore it is fair to conclude, that you should always use the StringBuilder class wherever possible.

C#Today - © 2002 WroxPress Limited


Object-Oriented Programming in C# - Introduction

Object-Oriented Programming in C# - Introduction
http://www.csharptoday.com/content.asp?id=1591
Article Rating: 2.5
Published on June 21, 2001
Abstract
C# is a fully object-oriented programming (OOP) language from Microsoft. It is robust, and it is also one of the four official languages on the .NET Framework. In this article, Laylian Chiang covers the basic OOP features of C#, and assumes some basic understanding of OOP in general.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

C# is a fully object-oriented programming (OOP) language from Microsoft. It is robust, and it is also one of the four official languages on the .NET Framework. This article covers the basic OOP features of C#, and assumes some basic understanding of OOP in general.

Why OOP?

Have you ever wondered why modern programming languages tend to be object-oriented?

One aspect that makes OOP languages superior to procedural programming languages is their main benefit: code reuse. If you need a program to calculate the home loan interest and someone else has already written such code, it is cheaper and faster to buy the piece of code from that person than to write your own or hire a programmer to write it. You can then include it in your own application. OOP is by design modular, in which each class encapsulates, among other things, data, functions and certain behaviors so users of your class do not need to know what is going on inside the class when they are using it.

A C# Class

Talking about an OOP language, it is natural to start by having a look at what a class looks like. A C# class starts with the class keyword followed by the class name. Enclosing the class contents is a pair of curly brackets. Therefore, the very basic form of a class takes this form.

class ClassName {
}

Classes are the heart of an OOP language.

Class Members

For a class to be meaningful, it has to have one or more member. Class members can include fields and functions. The class members are explained below.

Fields

A field is a member that represents a variable. There are three types of variables, value types, reference types and pointer types. However, pointer types are only used in the unsafe mode when you require superfast code or need to interface with existing software, such as COM objects or native C code in DLLs.

Value types include predefined data types, enum types, and struct types. The table below lists each of the predefined data types, and provides examples of each.

Type | Description | Examples
sbyte | 8-bit signed integral type | sbyte val = 12;
short | 16-bit signed integral type | short val = 12;
int | 32-bit signed integral type | int val = 12;
long | 64-bit signed integral type | long val1 = 12; long val2 = 34L;
byte | 8-bit unsigned integral type | byte val1 = 12; byte val2 = 34U;
ushort | 16-bit unsigned integral type | ushort val1 = 12; ushort val2 = 34U;
uint | 32-bit unsigned integral type | uint val1 = 12; uint val2 = 34U;
ulong | 64-bit unsigned integral type | ulong val1 = 12; ulong val2 = 34U; ulong val3 = 56L; ulong val4 = 78UL;
float | Single-precision floating point type | float value = 1.23F;
double | Double-precision floating point type | double val1 = 1.23 double val2 = 4.56D;
bool | Boolean type; a bool value is either true or false | bool value = true;
char | Character type; a char value is a Unicode character | char value = 'h';
decimal | Precise decimal type with 28 significant digits | decimal value = 1.23M;

Reference types include class types, interface types, delegate types, and array types. For instance, a class that has a number of field members is given below.

class Class1 {
int x;
int y;
}

Methods

A method is a function that performs a specific task and can return a value. A typical method can also have a list of arguments. A method signature is as follows.

[AccessModifier] ReturnValue MethodName( ArgumentList )

For example, you can have a method called DoubleNumber that accept an int argument and returns a long that is the double of the passed argument.

public long DoubleNumber( int number )
            {
return ( 2 * number );
}

If your method does not return anything, we use the keyword void as the return value part of the method. The following is a void method.

public void Display( int number )
            {
System.Console.WriteLine( number );
}

Instance Members and Static Members

All field and methods explained so far are instance members. It means that to use those members, you first need to create an instance of the object. Only after the object is instantiated, can you access an instance member. Both field and function members are accessed by calling the object reference and the field/function member separated by a dot ( . ).

The code in Listing 1 shows an example of two classes, CalledClass and CallingClass . See how CalledClass is instantiated from CallingClass and how its data and function members are accessed.

Listing 1: Code that demonstrates how to instantiate a class and use its members.

using System;
class CalledClass
            {
public int i = 13;
public long DoubleNumber( int number )
            {
return ( number * 2 );
}
}

class CallingClass
            {
public static void Main(String[] args) {
// instantiate
            CalledClass using the new keyword
CalledClass cc = new
            CalledClass();

// accessing CalledClass's field member
            i;
System.Console.WriteLine( "i = " + cc.i
            );
System.Console.WriteLine( "Double of 12 is " +
            cc.DoubleNumber( 12 ));
}
}

When you run the compiled code, you will see the following output:

i = 13
Double of 12 is 24

If you try to access an instance member before instantiating an object, an error message will be raised at compile time:

An object reference is required for the nonstatic field, method, or property 'CalledClass.member'.

Static Members

While instance members can only be used after you create an instance of an object, static members refer to members belonging to the class. You can use static members without first instantiating an object. In other words, the member can be used without the existence of an object. Also note that since you don't need an object reference, you use the class name in front of the member name.

Listing 2 shows the code where CalledClass now has two static members: y and TripleNumber .

Listing 2: Static members

using System;
class CalledClass
            {

public int i = 13;
public long DoubleNumber( int number
            ) {
return ( number * 2 );
}

// static
            members
public static int y = 15;
public static long
            TripleNumber( int number ) {
return ( number * 3
            );
}
}

class CallingClass {

public static void
            Main(String[] args) {

// accessing CalledClass's static
            members y and TripleNumber
Console.WriteLine( "y = " +
            CalledClass.y );
Console.WriteLine( "Triple of 12 is " +
            CalledClass.TripleNumber( 12 ));
}
}

If you run the compiled code, this is the result:

y = 15
Triple of 12 is 36

See how we don't need an instance of CalledClass to use its static members?

You may have observed that the Main method, the entry point of a class, is also a static method. Why? Because Main is the first method to be called even before an instance of object (in this case, CallingClass ) itself is created. Without being a static method, Main can never be called.

You will probably ask this question. Can you call a static member from an object instance? No, just in case you try the following code.

// instantiate CalledClass using the new keyword
CalledClass cc = new CalledClass();
// try accessing the static method TripleNumber
Console.WriteLine( "Triple of 12 is " + cc.TripleNumber( 12 ));

A common mistake by many OOP language beginners is trying to call an instance member from a static member. For example, in the following code, nonstatic DoubleNumber is called from the static Main method:

using System;
class Hello {
public
            static void Main(String[] args) {
Console.WriteLine( "Double of
            12 is " + DoubleNumber( 12 ));
}
public long DoubleNumber( int
            number ) {
return ( number * 2 );
}
}

This will raise a compile error:

error CS0120: An objet reference is required for the nonstatic field, method, or property 'Hello.DoubleNumber(int)'.

Some people will then try to make DoubleNumber static too. This works; however, making everything static results in messy code. A better solution is to instantiate the object and call the member with the object reference. This is done in the code in Listing 3.

Listing 3: Instantiating a class to use its instance members

using System;

class Hello {
public
            static void Main(String[] args) {
// instantiate this
            class
Hello h = new Hello();
Console.WriteLine( "Double of 12
            is " + h.DoubleNumber( 12 ));
}
public long DoubleNumber( int
            number ) {
return ( number * 2 );
}
}

When you have a number of classes, you can organize those classes using namespaces . Classes of similar functionality can be placed in the same namespace.

Accessibility

As mentioned several times in the above paragraphs, a class member can have some level of accessibility. Accessibility is important because you can control the class member visibility from outside the class. If you don't want a certain method to be called from outside the class, for example, you can make it private by adding the private access modifier in front of the method declaration.

private int MyMethod()

By default, without an access modifier, a class member is private. However, explicitly stating it by using the keyword private makes your code more readable.

On the other hand, if you want your class member to be accessible by anyone, make it public.

Between those two extremes (private and public), a class member can have internal , or protected or protected internal .

  • The internal modifier is used when you want the access to be limited to the same assembly as the class,
  • while protected is selected when you want access to be limited to the containing class and classes derived from the containing class.
  • The last modifier, protected internal means a member can be accessed from either a class that could access it through the internal access path or a class that could access it through a protected access path.

Inheritance

Inheritance is one of the main principles in OOP. It allows a class to be extended when you can use some members of that class but want more. In OOP jargon, the class that is extended is called the parent class and the class that extends it is called the child class or the derived class. For example, if you have a class called NumberCruncher with two methods, add and subtract , and you want a more powerful class that does add and subtract as well as multiply, you can extend NumberCruncher . Of course you can build a completely new class that has the three methods, however by extending NumberCruncher you save writing the two existing methods: Code reuse.

To extend a class, you need to mention the parent class name after the child class. The parent and child classes are separated by a colon:

class ChildClass: ParentClass {
}

The code in Listing 4 shows how a new class named PowerfulNumberCruncher extends the NumberCruncher class.

Listing 4: Demonstrating inheritance

using System;

class NumberCruncher
            {
public int add(int a, int b) {
return ( a+b
            );
}

public int subtract(int a, int b) {
return ( a-b
            );
}
}

class PowerfulNumberCruncher: NumberCruncher
            {
public long multiply(int a, int b) {
return ( a*b
            );
}
}

class User {
public static void Main(String[]
            args) {
// instantiate this class
PowerfulNumberCruncher pnc =
            new PowerfulNumberCruncher();
Console.WriteLine( "4 + 5 = " +
            pnc.add(4,5) );
Console.WriteLine( "4 - 5 = " + pnc.subtract(4,5)
            );
Console.WriteLine( "4 * 5 = " + pnc.multiply(4,5)
            );
}
}

If you run the compiled User class, you will see the following output on your console.

4 + 5 = 9
4 - 5 = -1
4 * 5 = 20

See how you can use the multiply method as well as add and subtract from PowerfulNumberCruncher ? You inherit all the public and protected methods from the parent class.

What if you need a method from the parent class but it does not work quite as you expect? Can you modify its behavior? Yes, you can, and this in OOP is called method overriding. For instance, the PowerfulNumberCruncher in Listing 4 can be modified so it overrides the subtract method with its own. Now, instead of a-b , the subtract method returns b-a . For a method to be overridable, the method must be virtual. This is shown in Listing 5.

Listing 5: Method overriding

using System;

class NumberCruncher
            {
public int add(int a, int b) {
return ( a+b
            );
}

virtual public int subtract(int a, int b) {
return
            ( a-b );
}
}

class PowerfulNumberCruncher:
            NumberCruncher {

override public int subtract(int a, int b)
            {
return ( b-a );
}
public long multiply(int a, int b)
            {
return ( a*b );
}
}

class User {
public static
            void Main(String[] args) {
// instantiate this
            class
PowerfulNumberCruncher pnc = new
            PowerfulNumberCruncher();
Console.WriteLine( "4 + 5 = " +
            pnc.add(4,5) );
Console.WriteLine( "5 - 4 = " + pnc.subtract(4,5)
            );
Console.WriteLine( "4 * 5 = " + pnc.multiply(4,5)
            );
}
}

Now, when you run the program, here is what you will see.

4 + 5 = 9
5 - 4 = 1
4 * 5 = 20

Conclusion

We have seen how C# presents classes and the basics of OOP. The language itself is much richer than what we've seen here - OOP is a very broad topic. For example, discussions on constructors, polymorphism and other types of class members, such as properties and indexers, have been left out for later. To learn more about this language, you can download the C# language reference from http://msdn.microsoft.com/vstudio/nextgen/technology/clangref.doc. A number of online sources such as http://www.csharphelp.com/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq and http://www.csharp.com/links.html?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq are also helpful.

C#Today - © 2002 WroxPress Limited


Java stream Vs C# Streams

Java stream Vs C# Streams
http://www.csharptoday.com/content.asp?id=1539
Article Rating: 3
Published on June 19, 2001
Abstract
Most applications deal with reading data in from a source, doing something with it, and outputting it in some new form. The source can be many things, user input, local files, another application, a hardware device, etc. Data Input and output streams are the foundation of working with data in the .NET APIs and understanding them is an important part of programming for this new environment. This article, by Mark Tutt, is geared towards Java programmers considering migrating to .NET, however C# developers unfamiliar with the .NET stream classes should also find the information useful. Mark attempts to review the basic Java stream classes and illustrate the analogous .NET classes, demonstrating how one would write the equivalent code using C#.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

Most applications deal with reading data in from a source, doing something with it, and outputting it in a new form. The source can be many things, user input, local files, another application, a hardware device, etc. Data input and output streams are the foundation of working with data in the .NET APIs and understanding them is an important part of programming for this new environment.

This article is geared towards Java programmers considering migrating to C# and .NET, however C# developers unfamiliar with the .NET stream classes should also find the information useful. I will attempt to review the basic Java stream classes and illustrate the analogous .NET classes, demonstrating how one would write the equivalent code using C#.

Stream Basics

When one first examines the comparison between the Java stream classes and those found in the .NET Beta 1 SDK, there is a momentary panic, since there appear to be far fewer classes in .NET.

Figure 1 Java Stream Classes

Figure 2 .NET Stream Classes

However, appearances are deceiving. The .NET stream classes are based on a single abstract class System.IO.Stream. .NET classes inheriting from this class can optionally read or write data. The JDK stream classes are derived from one of four base classes, java.io.InputStream, java.io.OutputStream, java.io.Reader or java.io.Writer. The reason for 4 base classes is because before JDK 1.1, the input and output stream classes only supported 8-bit byte streams, and had no facilities for working with 16-bit Unicode character streams. The .NET development team had the advantage of learning from this, and thankfully avoided this somewhat confusing situation.

Since most of the JDK classes follow the separate Reader/Writer model, you will see them implemented as separate classes for reading and writing data. An example of this, starting at the top of the list, would be the java.ioBufferedInputStream and java.io.BufferedOutputStream, or their character equivalents, java.io.BufferedReader and java.io.BufferedWriter. The corresponding .NET class is simply System.IO.BufferedStream, which offers both reading and writing methods, and makes it much cleaner for the developer.

The constructors for the classes for both API's are virtually identical. The first Java constructor syntax creates a buffered stream with a default buffer size of 512 bytes, whereas the .NET default buffer size is 4096 bytes.

[Java]

BufferedInputStream( InputStream in );
BufferedInputStream( InputStream in, int size );

BufferedOutputStream( OutputStream in );
BufferedOutputStream( OutputStream in, int size );

BufferedReader( Reader in );
BufferedReader( Reader in,int size );

BufferedWriter( Writer in );
BufferedWriter( Writer in,int size );

[C#]

public BufferedStream( Stream stream );
public BufferedStream( Stream stream, int bufferSize );

Buffered stream classes read and write blocks of data from their source stream into a buffer, reducing the number of calls to low level functions to access the underlying data source. An example of this would be reading and writing records from a disk file. Unless the records were fairly large, reading the records individually might require an excessive number of disk access operations, which could degrade performance, while the same argument could be made for writing records to the file. Utilizing the buffered stream classes allows the developer to determine an optimal 'chunk' of data to be buffered, or allow the system to use a default setting.

File Streams

Since I'm using reading and writing files as our first example, we should discuss the classes that relate to this before putting them together for an example.

The file stream classes in both APIs are somewhat less similar. For this class, the .NET API offers a plethora of options, which allow the developer to assert a fine grained control, and tune the application for the intended use.

[Java - Binary Files]

FileInputStream( String name );
FileInputStream( File file );
FileInputStream( FileDescriptor fdObj );

FileOutputStream( String name );
FileOutputStream( String name, boolean append );
FileOutputStream( File file );
FileOutputStream( FileDescriptor fdObj );

[Java - Character Files]

FileReader( String name );
FileReader( File file );
FileReader( FileDescriptor fdObj );

FileWriter( String name );
FileWriter( String name, boolean append );
FileWriter( File file );
FileWriter( FileDescriptor fdObj );

[Java - Files which require random access, i.e. a mixed Read/Write usage]

RandomAccessFile( String name, String mode );
RandomAccessFile( File file, String mode );

[C#]

FileStream( int handle, FileAccess access );
FileStream( string name, FileMode mode );
FileStream( int handle, FileAccess access, bool ownsHandle );
FileStream( string name, FileMode mode, FileAccess access );
FileStream( string name, FileMode mode, FileAccess access, FileShare share);
FileStream( int handle, FileAccess access, bool ownsHandle, bool useAsync );
FileStream( string name, FileMode mode, FileAccess access, FileShare share, int bufferSize );
FileStream( int handle, FileAccess access, bool ownsHandle, bool useAsync, int bufferSize );
FileStream( string name, FileMode mode, FileAccess access, FileShare share, int bufferSize, bool useAsync );

Starting with the arguments from the top, FileAccess is an enumeration of the file access options available: Read, ReadWrite and Write. FileMode is an enumeration of the file opening options: Create, CreateNew, Append, etc. File share is yet another enumeration, this one of the sharing options for sharing files between processes. The useAsync parameter signals the system to use overlapped (asynchronous) IO if available, and the buffersize parameter sets the streams internal buffers.

For full details of these options, check the .NET SDK documentation. For our purposes, we are going to use the constructor form: FileStream(string name, FileMode mode, FileAccess access);

Now, it is possible to read and write data using just the classes we have covered thus far, however, it's not very convenient. These low level classes provide basic methods for reading and writing single bytes or arrays of bytes (characters for the Java classes based on java.io.Reader and java.io.Writer), but that's not how we normally work with application data these days. We tend to think of our basic data types, i.e. ints, floats and strings, rather than the individual bytes they are made up of.

Fortunately, the developers of both APIs understood this, and provide classes that are in line with the needs of client programmers. Still, the knowledge of the underlying classes is important if you need to implement your own stream classes.

Binary Streams

For the example we are going to use, we are going to read and write binary data, so these classes allow you to read primitive types from their underlying streams:

[Java]

DataInputStream( InputStream in );
DataOutputStream( OutputStream out);

[C#]

BinaryReader( Stream input );
BinaryWriter( Stream output );
BinaryReader( Stream input, Encoding encoding );
BinaryWriter( Stream output, Encoding encoding );

The Encoding parameter allows the developer to set the character encoding for the stream. There are several options here, found in the System.Text namespace. Java streams do not include these options as they are written in a platform independent manner. You may need to investigate the encoding options if you are writing applications that will need to be localized, however going into this subject is outside the scope of this article. Richard Grimes describes encoding briefly in his article "Streams and .NET" at http://www.codeguru.com/cs_data/streaming.html?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq, however I have not found a great deal of other information available online.

Putting what we've covered this far together to assemble streams for reading and writing files takes the following forms:

[Java - read]

FileInputStream in =
            new FileInputStream("test.data");
BufferedInputStream bin = new
            BufferedInputStream(in, 4096);
DataInputStream din = new
            DataInputStream(bin);

[Java - Write]

FileOutputStream
            out = new FileOutputStream("test.data");
BufferedOutputStream
            bout = new BufferedOutputStream(out, 4096);
DataOutputStream dout
            = new DataOutputStream(bout);

[C#]

FileStream fStream
            = new FileStream("test.data",
            System.IO.FileMode.OpenOrCreate,FileAccess.ReadWrite);
BufferedStream
            bStream = new BufferedStream(fStream);

BinaryReader rStream =
            new BinaryReader(bStream);

BinaryWriter wStream = new
            BinaryWriter(bStream);

If you wanted to arbitrarily read and write the file, in Java you would have to take a somewhat different approach. To fill this need, you would use the java.io.RandomAccessFile class:

RandomAccessFile rFile = new RandomAccessFile("test.data", "rw");

However, while it can work with the stream classes, this class provides methods for reading and writing primitive data types, as well as the positioning information, i.e. seek(long pos), skipBytes(int n), and getFilePointer() (returns the current offset in the file.) Since this class provides all of this functionality, a stream class is not commonly used with it. If you do, you end up giving up some of the functionality this way, for instance the DataInputStream and DataOutputStream classes do not offer the full range of movement options, only the skip(long bytes) method, which requires the developer to do a little math. By comparison, the BinaryReader and BinaryWriter classes offer the seek(int offset, SeekOrigin) method, which I find to be a little more intuitive. (SeekOrigin is another enumeration in the System.IO namespace, containing three members Begin, Current and End.) Please note that for the BinaryReader class, you have to access the underlying stream with BinaryReader.BaseStream() to access the base streams seek method. (I wonder if anyone can tell me why a seek method didn't make it in the BinaryReader class itself?)

Ok, now we've covered what we needed to know for our first example. We're going to create several instances of a Customer class, populate each one with some data, and write them out to a file. Then we'll read the file, and display to the screen what we have read.

[Java]

//First the customer
            class.

public class Customer
{
public int
            idNumber;
public String Name;

public Customer(int number,
            String FullName)
{
idNumber = number;
Name =
            FullName;
}
}

//And the relevant functions from our
            demonstration program.

static void SaveCustomers( Vector
            CustList ) {

try {
// Create our output
            stream
FileOutputStream out = new FileOutputStream("test.data");
BufferedOutputStream bout = new BufferedOutputStream(out,
            4096);
DataOutputStream dout = new
            DataOutputStream(bout);

//Cycle through the Vector and write
            out the customer objects
for (int i = 0; i < CustList.size();
            i++) {
Customer tCustomer =
            (Customer)CustList.elementAt(i);
// Write the idNumber as an
            integer
dout.writeInt(tCustomer.idNumber);
// Write the name
            as a UTF encoded
            string
dout.writeUTF(tCustomer.Name);
}

dout.close();
}
            catch (IOException e) {}
}

static void LoadCustomers(
            Vector CustList) {

try {
// Create our input
            stream
FileInputStream in = new FileInputStream("test.data");
BufferedInputStream bin = new BufferedInputStream(in,
            4096);
DataInputStream din = new DataInputStream(bin);
//
            Simple use of a boolean to tell when we should quit
            reading
boolean more = true;

while (more == true) {
try
            {
// Read the ID as an integer
int id = din.readInt();
//
            Read the name as a UTF encoded string
String name =
            din.readUTF();
// Create a new customer from the read data
//
            And add it to the passed Vector
CustList.addElement(new Customer(
            id, name ));
} catch( EOFException e) {
more =
            false;
}}
}

din.close();
} catch (IOException e)
            {}
}

[C#]

namespace Binary
{
using
            System;

public class Customer
{
public int
            idNumber;
public string Name;

public Customer(int number,
            string FullName)
{
idNumber = number;
Name =
            FullName;
}
}
}

And the relevant functions from our demonstration program.

   public static void
            SaveCustomers( ArrayList CustList ) {
// Create our output
            stream
FileStream fStream = new
            FileStream("test.data",
System.IO.FileMode.OpenOrCreate,FileAccess.Write);

BufferedStream
            bStream = new BufferedStream(fStream);
BinaryWriter wStream = new
            BinaryWriter(bStream);

// Cycle through the ArrayList and
            write out the customer objects
for (int i = 0; i <
            CustList.Count; i++) {
Customer tCustomer =
            (Customer)CustList[i];
// BinaryWriter's Write method is
            overloaded, so you
// don't have to specify the data
            type
wStream.Write(tCustomer.idNumber);
wStream.Write(tCustomer.Name);
}

wStream.Close();
}

public
            static void LoadCustomers( ArrayList CustList) {

// Create
            our input stream
FileStream fStream = new
            FileStream("test.data",
System.IO.FileMode.Open,FileAccess.Read);

BufferedStream
            bStream = new BufferedStream(fStream);
BinaryReader rStream = new
            BinaryReader(bStream);
bool more = true;

while (more ==
            true) {
try {
// Read the ID as a 32 bit integer
int id =
            rStream.ReadInt32();
// Read the name as a string
string name
            = rStream.ReadString();
// Create a new Customer object and add
            it to the
// passed ArrayList
CustList.Add( new Customer( id,
            name ));
}
catch( EndOfStreamException e) {}
more =
            false;
}
}

rStream.Close();
}

Object Serialization

Before I get the email flood pointing out that there is an easier way to do this using the serialization features of the APIs, I'll demonstrate that next.

In order to persist an object in Java using the API functionality, the object class must implement the java.io.Serializable interface. To do the same thing with an object in C#, the object class must implement the System.Runtime.Serialization.ISerializable interface. Additionally, the .NET serialization mechanism uses a formatter object, for our example we're going to use the BinaryFormatter found in the System.Runtime.Serialization.Formatters.Binary namespace.

Both APIs provide means of controlling serialization of class variables. In Java, a class implementing the Serializable interface declares variables that should not be serialized as transient variables, where the .NET API uses the metadata tag [NonSerialized] to determine that a variable should not be saved.

We need to use different Java stream classes to support object serialization.

ObjectInputStream(InputStream in)
ObjectOutputStream(OutputStream out)

You can serialize .NET objects to its various stream types using different formatters. While we're going to use the BinaryFormatter and BinaryReader/Writer classes here, for example if our stream was based on a HTTP socket, we could use the SoapFormatter from the System.Runtime.Serialization.Formatters.Soap namespace. You could do this with this example simply by replacing the BinaryFormatter with a SoapFormatter.

So here's our example code, rewritten to use object serialization:

[Java]

public class Customer
            implements Serializable
{
public int idNumber;
public
            String Name;

public Customer(int number, String
            FullName)
{
idNumber = number;
Name =
            FullName;
}
}

static void SaveCustomers(Vector
            CustList) {

try {
FileOutputStream out = new
            FileOutputStream("test.data");
BufferedOutputStream bout = new
            BufferedOutputStream(out, 4096);
ObjectOutputStream dout = new
            ObjectOutputStream(bout);

for (int i = 0; i <
            CustList.size(); i++) {
Customer tCustomer =
            (Customer)CustList.elementAt(i);
dout.writeObject(tCustomer);
}
dout.close();
}
            catch (IOException e) {}
}

static void LoadCustomers(
            Vector CustList) {

try {
FileInputStream in = new
            FileInputStream("test.data");
BufferedInputStream bin = new
            BufferedInputStream(in, 4096);
ObjectInputStream din = new
            ObjectInputStream(bin);
boolean more = true;

while (more
            == true) {
try {
Customer tCustomer =
            (Customer)din.readObject();
CustList.addElement(tCustomer);
}
            catch( ClassNotFoundException e) {
more =
            false;
}}
}

din.close();
} catch (IOException e)
            {}
}

[C#]

namespace Binary
{
using
            System;

[Serializable]
public class
            Customer
{
public int idNumber;
public string
            Name;

public Customer(int number, string
            FullName)
{
idNumber = number;
Name =
            FullName;
}
}
} public static void SaveCustomers( ArrayList
            CustList ) {
FileStream fStream = new FileStream("test.data",
System.IO.FileMode.OpenOrCreate,FileAccess.Write);
BufferedStream
            bStream = new BufferedStream(fStream);
BinaryFormatter bFormatter
            = new BinaryFormatter();

for (int i = 0; i <
            CustList.Count; i++) {
Customer tCustomer =
            (Customer)CustList[i];
bFormatter.Serialize(bStream,
            tCustomer);
}
bStream.Close();
}

public static void
            LoadCustomers( ArrayList CustList) {
FileStream fStream = new
            FileStream("test.data",
System.IO.FileMode.Open,FileAccess.Read);

BufferedStream
            bStream = new BufferedStream(fStream);
BinaryFormatter bFormatter
            = new BinaryFormatter();
bool more = true;
while (more ==
            true) {
try
            {
CustList.Add((Customer)bFormatter.Deserialize(bStream));
}
catch(
            SerializationException e) {
more =
            false;
}
}
bStream.Close();
}

Text Streams

Reading and writing text streams is fairly similar. We've already covered some of the required classes in Java, however here are the new ones we're going to use:

PrintWriter(OutputStream out);
PrintWriter(OutputStream out, boolean autoFlush);
PrintWriter(Writer out);
PrintWriter(Writer out, boolean autoFlush);

The autoFlush parameter determines if the stream buffer should be flushed when one of the println methods is called. You could of course use the FileWriter class directly, or any of the other character stream classes, but PrintWriter offers a friendlier interface for our example. After all, software development is all about making choices, isn't it?

LineNumberReader(Reader in, int sz);
LineNumberReader(Reader in);

The sz parameter determines the size of the buffer used. The LineNumberReader class extends BufferedReader, so it is not necessary to use a buffered stream class with it.

[C#]

StreamWriter( string path );
StreamWriter( Stream stream );
StreamWriter( string path,
            bool append );
StreamWriter( Stream stream, Encoding encoding );
StreamWriter( Stream stream, Encoding encoding, int bufferSize
            );
StreamWriter( string path, bool append, Encoding encoding );
StreamWriter( string path, bool append, Encoding encoding, int
            bufferSize );

StreamReader( string path );
StreamReader(
            Stream stream );
StreamReader( string path, Encoding encoding );
StreamReader( Stream stream, Encoding encoding );

The .NET API gives you a shortcut option here. You can construct a StreamReader or StreamWriter for character files directly from the path and filename, without having to first construct a stream object, as you do in Java when using the PrintWriter class we chose for this example. The other parameters should be familiar from the previous examples.

The following trivial examples demonstrate reading and writing lines of test:

[Java]

FileOutputStream out = new
            FileOutputStream("test.txt");
BufferedOutputStream bout = new
            BufferedOutputStream(out, 4096);
PrintWriter pWriter = new
            PrintWriter(bout);
pWriter.println("some
            string");

FileInputStream out = new
            FileInputStream("test.txt");
LineNumberReader lReader = new
            LineNumberReader(bout, 4096);
String s =
            lReader.readLine();

[C#]

StreamWriter sWriter = new
            StreamWriter("text.txt");
SWriter.WriteLine("some
            string");

StreamReader sReader = new
            StreamReader("text.txt");
String s =
            sReader.ReadLine();

So far we have covered reading and writing both binary and text data using the stream classes. I'll cover one more area before wrapping up.

Memory Streams

Many of today's applications are frequently at least network aware, and as such need to read and write data from TCP/IP sockets. Fortunately, since you can use the stream classes that we've already covered with your socket code, it's not a lot different, at least for the basics. The following functions serialize and de-serialize objects into a byte array using memory streams. (Socket classes in both API's deal with writing byte arrays.)

[Java]

static byte[] serialize(Object
            s)
{
ByteArrayOutputStream bo = new
            ByteArrayOutputStream();

try {
ObjectOutputStream os = new
            ObjectOutputStream(bo);

os.writeObject(s);

return
            bo.toByteArray();
} catch (Exception e)
            {
System.out.println(e.getMessage());
}
}

Object
            deserialize(byte[] data) {
try {
ByteArrayInputStream bi = new
            ByteArrayInputStream(data);
ObjectInputStream is = new
            ObjectInputStream(bi);

return is.readObject();
} catch
            (Exception e) {
System.out.println(e.getMessage());
}
}

[C#]

public static byte[] serialize(object
            s)
{
try {
MemoryStream ms = new
            MemoryStream();
BinaryFormatter b = new
            BinaryFormatter();
b.Serialize(ms,s);

return
            ms.ToArray();
} catch (Exception e) {
throw
            Trace.error(Trace.SERIALIZATION_FAILURE,
            e.Message);
}
}

public object deserialize(byte[] data)
{
try {
MemoryStream ms = new
            MemoryStream(data);
BinaryFormatter b = new
            BinaryFormatter();

return b.Deserialize(ms);
} catch
            (Exception e) {
throw Trace.error(Trace.SERIALIZATION_FAILURE,
            e.Message);
}
}

As you can see, the Java ByteArrayInputStream and ByteArrayOutputStream are directly compatible with the .NET MemoryStream class in this example.

The .NET API offers another class for working with socket data that is perhaps a little easier to work with, however, to my knowledge there is no corresponding class in the Java API. This is the NetworkStream class. Here is a brief example of its use:

[Writing data]
// Attach to socket 5008
            on the local machine
TCPClient myClient = new
            TCPClient("localhost",5008);
// Create a BinaryWriter for the
            NetworkStream class
// returned from the TCPClient.GetStream()
            method
BinaryWriter bWriter = new
            BinaryWriter(myClient.GetStream());

[Reading data]
//
            Listen on socket 5008
TCPListener myListener = new
            TCPListener(5008);
// TCPListener.Accept() returns a Socket
            object
Socket mySocket = myListener.Accept();
// Create a
            network stream for the connected socket
NetworkStream nStream =
            new NetworkStream(socket);
// Attach a BinaryReader object to the
            stream
BinaryReader bReader = new
            BinaryReader(nStream);

Using the NetworkStream classes allows the developer to skip the implicit conversion of byte arrays used by the socket.

Additional Subjects

This article did not address all of the stream classes in both API's, but should have given you enough information to build on if you run into usage of one of the classes I did not cover. There is one possibility that you may encounter that I will mention though. There are several stream classes in Java that currently do not have a correlation in the .NET API. However, it is fairly straightforward to implement similar classes in C# with a little research.

For instance, the java.util.zip.* classes implement support (including stream classes) for reading and writing files in the common Zip file format. Implementing something like this sounds like a monumental task, unless you know that these classes in Java are little more than a wrapper around a modified version of the zlib library, and in fact only expose a portion of it's functionality. zlib is a freely available data-compression library for use on virtually any computer hardware and operating system that is also legally unencumbered; that is, not covered by any patents. Since you can make DLL calls from C# code, writing a wrapper and letting the DLL do all the real work makes it a bit easier, after all Sun thought so. A Win32 DLL version of zlib is available from:

http://www.winimage.com/zLibDll/.http://www.winimage.com/zLibDll?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq

Both binary versions and source code are available.

If there is enough interest, perhaps I can do another article on extending the .NET Stream classes to demonstrate this.

If you're not interested in rolling your own classes, XCEED Software is currently offering a beta version of their XCEED Zip Compression Library v5.0, which offers this functionality for .NET developers. You can find more information at http://www.xceedsoft.com/dotnet?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq, the currently available version requires .NET Beta 2.

Conclusion

A zip file containing sample applications written in Java and C# accompanies the article. While the Java code was written using Visual Cafe and includes the project files for that environment, it can be compiled and executed using just the JDK. The following screen capture shows the output from both the C# and Java programs created.

Hopefully this article has given you enough information to see that porting code using the Java API's stream classes to C# and .NET is not difficult, and that there are a number of similarities between the two APIs. There are a lot of Java programmers out there who, with an introduction to the .NET APIs could quickly become proficient C# programmers. Likewise, there is a large body of Java code available from various books and online sources that address a wide variety of programming needs, and understanding how to translate that into C# just might come in handy!

C#Today - © 2002 WroxPress Limited


Understanding Delegates in C#

Understanding Delegates in C#
http://www.csharptoday.com/content.asp?id=1603
Article Rating: 3
Published on June 18, 2001
Abstract
Languages like C++, Pascal, Modula support the concept of function pointers. These function pointers allow you to plug-in a function call with the matching signature in your program code at the runtime. It means that the exact function you are going to call through your program code need not be known at the compile time and you may call any function at the runtime. C# defines a new construct called delegate that behaves identically to function pointers without the drawbacks of pointers. In this article, P. G. Sarang will discuss the importance of delegates, how to create them and how to use them in your C# program code.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

Languages like C++, Pascal, Modula support the concept of function pointers. These function pointers allow you to plug-in a function call with the matching signature in your program code at the runtime. It means that the exact function you are going to call through your program code need not be known at the compile time and you may call any function at the runtime. A typical situation for this will be a sort routine where depending on the type of data, one can decide on the sorting algorithm to be used at runtime. Another application will be for a state machine in compiler construction. This is achieved by creating a function pointer in your program code. Such a function pointer is initialized to point to the desired function at the runtime and is de-referenced to invoke the desired function. The disadvantage is that function pointers carry all the drawbacks associated with pointers. A pointer variable at any point of time may hold an invalid address and cause memory corruption resulting even in a system crash.

C# defines a new construct called a delegate that behaves identically to function pointers without the drawbacks of pointers. In this article, I will discuss the importance of delegates, how to create them and how to use them in your C# program code. Delegates though they behave as pointers, they are fully object oriented. The System namespace in .NET libraries defines a class called Delegate. The major difference between a function pointer and a delegate would be that the function pointer does not contain any object reference while a delegate contains both an object reference and the method. Thus, a delegate may be used to invoke a function of any object.

Thus, delegates enable scenarios that C++ and some other languages have addressed with function pointers. Unlike function pointers, delegates are object-oriented, type-safe, and secure. Delegates are reference types that derive from a common base class: System.Delegate. A delegate instance encapsulates a method-a callable entity. For instance methods, a callable entity consists of an instance and a method on the instance. For static methods, a callable entity consists of a class and a static method on the class. There are three steps in defining and using delegates:

  • Declaration
  • Instantiation
  • Invocation

Declaring delegates

I will start off by showing you how to declare a delegate. To declare a delegate use the following syntax

delegate void MyDelegate();

This declares a delegate called MyDelegate. The declared delegate will be used to reference a function that returns void and does not take any arguments. If you wish to create a delegate that refers to a function that takes a parameter of string type and returns a void, you will use the following syntax.

delegate void MyDelgate(string str);

To declare a delegate that represents a function, which returns a long and takes two parameters of int type, you will use the following syntax.

delegate long MyDelegate (int i, int j);

Having seen how to declare the delegates, we will now look into instantiating delegates.

Instantiating Delegates

Although delegates behave in most ways like other classes, C# provides special syntax for instantiating a delegate instance. The newly created delegate instance then refers to either:

  • The static method referenced in the delegate-creation-expression, or
  • The target object (which cannot be null) and instance method referenced in the delegate-creation-expression, or
  • Another delegate

Once instantiated, delegate instances always refer to the same target object and method.

As delegate is a class defined in System namespace, you will instantiate a delegate just the same way you instantiate any other class, i.e. by using new keyword. The class constructor receives a parameter that specifies the method name to be used during delegate invocation. For example, you may declare a delegate as follows.

delegate long Operation (int i, int j);

We will use this delegate to invoke various operations on two integers. (See Class1.cs for full program listing). The operations such as Add, Multiply, Power, etc. are defined with the help of several class and instance methods. The actual operation to be invoked will be decided at the run time.

The following statement creates a delegate instance that references a method called Add. Note that Add must be declared elsewhere in the code and must have the signature specified in the declaration of Operation delegate.

Operation Op = new Operation(Add);

A reference to the created delegate object is then assigned to a variable of type Operation. Delegate objects are immutable, i.e. once they are created, and it is not possible to change the reference to the method they are currently referring to. However, you may use the same variable reference to point to another delegate object referring to another method. For example, the following statement reassigns variable Op to refer to a new delegate object that is used for invoking Multiply method. Note that both Add and Multiply methods must have same signatures.

Op = new Operation (Multiply);

Now, having seen how to declare and instantiate a delegate, I will show you how to invoke it.

Invoking Delegates

To invoke a delegate object, you use the following syntax.

d ();

This will invoke the method specified in the delegate constructor. Obviously, the above statement also indicates that the invoked method does not take any parameters and returns void.

To invoke Op delegate defined in previous paragraph, you will use following syntax,

long l = Op (10, 20);

Depending on whatever Op refers to, the appropriate class method; Add or Multiply will be called. Listing 1 provides the full program source that shows how to declare, instantiate and invoke a delegate.

Where is the Object Reference?

As I mentioned earlier, the delegate object contains both the object reference and the method name. In the above examples, you do not see any object reference. When you create a delegate object, you have to specify the object reference along with the method name. In the earlier example, the methods are static, thus there was no need to specify an object reference. Note that the static methods are always associated with its class.

To illustrate how to include an object reference while creating a delegate, we will now define an instance method called Power in our Class1 definition (Refer Listing 1 of Class1.cs). To invoke Power method through a delegate object, you will use the following code snippet.

Class1 obj = new Class1();
Operation Op = new Operation (obj.Power);
l = (long)PowerOp (5, 7);

Note that the Power method must have the same signature as specified in the delegate declaration.

Since you pass an object reference while instantiating delegate object, you will be able to invoke a method on any object at run time. The method need not belong to the current object and may belong to any other class. The only restriction is that the method signature must match the delegate declaration.

Composing Delegates

The delegates can be composed using "+ " operator. A composed delegate consists of two ore more delegates of the same type. When such a composed delegate is invoked, all delegates it consists of would be invoked in the order in which they are defined in the composition. There could be situations where you may like to invoke a series of operations on your data. In such situations, you may like to decide at runtime which operations to be invoked and the sequence in which the operations should be invoked. This is where the delegate composition will come to our help.

Only the delegates having void return type can be composed. The delegate to be composed may take one or more parameters. This can better be explained through a program code (Refer to Listing 2).

We declare a delegate called Operation as follows

delegate void Operation (int[] Numbers, int operand);

Operation delegate refers to a method that takes two parameters. The first parameter is an integer array and the second parameter is an integer operand that is used for operating on each element of the array. We will define two methods called Add and Multiply with the signature matching the above delegate signature. Add method adds the value specified by the second argument to each element of the array and Multiply method multiples each element of the array by the value specified by the second argument. We will then compose a new delegate that calls both Add and Multiply methods in a specified sequence.

We create two delegate instances with the following program statements,

Operation Op1 = new
            Operation(Add);
Operation Op2 = new
            Operation(Multiply);

Op1 delegate invokes Add method and Op2 delegate invokes Multiply method. The Add and Multiply methods have following signatures that match the delegate signature.

private static void Add (int[] Numbers, int operand)
private static void Multiply (int[] Numbers, int operand)

The two delegates, Op1 and Op2 can now be composed in a single delegate by using "+ " operand.

Operation Combined = Op1 + Op2;

The composed delegate is invoked same way as you invoke any other delegate. The following statement invokes our composed delegate by taking two arguments required by the functions that delegate refers to.

Combined (NumArray, 5);

The above statement first invokes the Add method adding 5 to each element of the array and then it invokes Multiply method multiplying each element of the array by 5. The program prints the array at the various stages of the operation so that you can verify this easily.

If you change the delegate composition as follows, the Multiply operation will be invoked before the Add operation.

Combined = Op2 + Op1;

When you run the application, you will get the following output on your console. The program first invokes the two delegates, Op1 and Op2, followed by their composition, Op1 + Op2, followed by another composition Op2 + Op1.

Original Array:
10, 20, 30, 40, 50,

Adding 5 to each element
15, 25, 35, 45, 55,

Multiplying each element by 5
75, 125, 175, 225, 275,

Demonstrating delegate composition:
Invoking Add followed by Multiply operation
Adding 5 to each element
Multiplying each element by 5
400, 650, 900, 1150, 1400,

Invoking Multiply followed by Add operation
Multiplying each element by 5
Adding 5 to each element
2005, 3255, 4505, 5755, 7005,

Press any key to continue

Removing elements from a Composed delegate

You may use the "- " operator to remove an element from a composed delegate. For example, in the composed delegate shown above, Op1 may be removed using the statement:

Combined = Combined - Op1;

Similarly, you may remove Op2 by using the statement:

Combined = Combined - Op2;

When you invoke the resultant delegate, only the operations currently available in the resultant delegate will be invoked in the order specified in the original composition.

Delegate Class

Having seen how to use delegates in your program code, we will now look behind the scenes into the implementation details. The System namespace defines a Delegate class with the following signature.

public abstract class Delegate : ICloneable, ISerializable

As mentioned earlier, the instance of Delegate stores the method's entry point and the object reference to which the method belongs. Function pointers in C, C++ do not contain the object reference. The System namespace defines a subclass of Delegate class called MulticastDelegate. An instance of Delegate class stores a reference to a single method, while an instance of MulticastDelegate can store references to multiple methods that are invoked in the specified order. The multiple methods are stored in an invocation list maintained by the Delegate object. To add to the invocation list, the class defines a Combine method and to remove from the invocation list, a Remove method is provided. The "+ " and "- " operators used in delegate composition in previous examples are the overloaded operators that call these methods.

In addition to this, the Delegate class also overloads the equality ("== ") and inequality ("!= ") operators for comparison of two delegate objects. The two delegates with the same methods, the same target objects and the same invocation lists are considered equal. The delegate type, i.e. singlecast or multicast is ignored in the evaluation of equality or inequality conditions. If the delegates to be compared refer to the static methods on the same class, the comparison returns true. If the methods refer to instance methods and are the same method on the same object, the comparison returns true. In all other cases, the methods and targets are not considered equal and the comparison returns false. In case of multicast delegates, the invocation lists for the two delegates must be identical. The invocation lists are considered identical if they have the same order and each corresponding element from the two delegate objects refer to the same method and target.

In addition to this, Delegate class defines a Public instance property called Method that returns the static method of the class represented by the delegate. The following code snippet prints the signature of the method represented by the Op delegate of Listing 1.

MethodInfo info = Op.Method;
Console.WriteLine(info.ToString());

The Delegate class also defines another property called Target. The Target property returns the object on which the delegate operates. The following code snippet obtains the target object and prints the class it represents on the user console.

object obj = Op.Target;
Console.WriteLine(obj.ToString());

The Delegate class defines several other methods. I shall discuss only two of them here. The reader is referred to the C# documentation for the details on the remaining class methods. The GetInvocationList method of Delegate class returns the list of delegates in an array. The following code segment obtains the invocation list of our Combined Delegate of Listing 2 and prints the names of all the methods contained in the composed delegate.

Delegate[] d =
            Combined.GetInvocationList();
foreach (Delegate del in
            d)
{
Console.WriteLine(del.Method.ToString());
}

The GetType method returns the Type of object on which delegate operates. The returned Type object can be introspected by using the Reflection API (Refer to System.Reflection package in C# documentation.)

Uses of Delegates

To handle events, you need an event handler. Delegates provide an easy way of defining these event handlers. I will discuss event handling in a separate article. Delegates are also very useful in defining callbacks. A reference to a callback method is passed as a delegate at runtime. Thus, the user can decide which method will be invoked as a callback at the execution time. The typical application of this is the sort routine. The user can decide on the sorting algorithm at the runtime depending on the data available and plug in the desired callback method by passing an appropriate delegate object to the sort routine.

C# uses delegates along with the event keyword to provide a very clean solution to event handling. An interesting and useful property of a delegate is that it does not know or care about the type of the object that it references. Any object will do; all that matters is that the method's signature matches the delegate's. This makes delegates perfectly suited for "anonymous" invocation. "

Listing 1: Class1.cs

namespace ConsoleApplication1
{
using
            System;

public class Class1
{
delegate void
            MyDelegate();
delegate long Operation (int i, int
            j);

public Class1()
{
}

public static int
            Main(string[] args)
{
// create a delegate of type
            MyDelegate
MyDelegate d = new MyDelegate(Method1);
// invoke
            delegate, this invokes Method1
d();
// create new delegate
            instance that refers
// to Method2
d = new
            MyDelegate(Method2);
// following invocation calls
            Method2
d();

// create a delegate of type Operation
//
            that refers to Add method
Operation Op = new
            Operation(Add);
// Call Add method through the delegate
long l
            = Op (10, 20);
Console.WriteLine("Sum of 10 and 20 equals " +
            l);
// create a delegate of type Operation
// that refers to
            Multiply method
Op = new Operation (Multiply);
// Call
            Multiply method through the delegate
l = Op (1639,
            1525);
Console.WriteLine("1639 multiplied by 1525 equals " +
            l);

// Create an object instance for calling instance
//
            methods through delegates
Class1 obj = new Class1();
// create
            a delegate of type Operation
// that refers to Power
            method.
Op = new Operation (obj.Power);
// Call Power method
            through the delegate
l = (long)Op (5, 7);
Console.WriteLine("5
            raised to 7 equals " + l);
return 0;
}

public
            static void Method1()
{
Console.WriteLine ("Method1
            invoked");
}

public static void
            Method2()
{
Console.WriteLine ("Method2
            invoked");
}

private static long Add (int i, int
            j)
{
return (i+j);
}

private static long Multiply
            (int i, int j)
{
return (i*j);
}

private long Power
            (int i, int j)
{
return ((long)Math.Pow(i,
            j));
}
}
}

The following screen shot shows the output for class1.exe:

Listing 2: DelegateEx2.cs

namespace ConsoleApplication2
{
using
            System;

public class DelegateEx2
{
// delegate
            declaration
delegate void Operation (int[] Numbers, int
            operand);

// class constructor
public
            DelegateEx2()
{
}

public static int Main(string[]
            args)
{
// Instantiate Operation delegate to refer to Add
            method
Operation Op1 = new Operation(Add);
// Instantiate
            Operation delegate to refer to Multiply method
Operation Op2 =
            new Operation(Multiply);

// Declare an integer array
int[]
            NumArray =
{
10,20,30,40,50
};

// Print the
            contents of original array
Console.WriteLine("Original
            Array:");
foreach (int i in NumArray)
{
Console.Write(i +
            ", ");
}
Console.WriteLine("\n");

// Call first
            delegate to Add 5 to each element of array
Op1 (NumArray,
            5);
// Print modified array contents
foreach (int i in
            NumArray)
{
Console.Write(i + ",
            ");
}
Console.WriteLine("\n");

// Call second delegate
            to multiply each element of array by 5
Op2 (NumArray, 5);
//
            Print modified array contents
foreach (int i in
            NumArray)
{
Console.Write(i + ",
            ");
}
Console.WriteLine("\n");

// Combine both
            delegates and invoke both Add and Multiply
// Operations in the
            specified sequence
Console.WriteLine("Demonstrating delegate
            composition:");
Console.WriteLine
("Invoking Add followed by
            Multiply operation");

// Compose delegates
Operation
            Combined = Op1 + Op2;
// Invoke composed delegate
Combined
            (NumArray, 5);
// Print modified array contents
foreach (int i
            in NumArray)
{
Console.Write(i + ",
            ");
}
Console.WriteLine("\n");

// Change delegate
            composition and invoke
Console.WriteLine
("Invoking Multiply
            followed by Add operation");
Combined = Op2 + Op1;
Combined
            (NumArray, 5);
// Print modified array contents
foreach (int i
            in NumArray)
{
Console.Write(i + ",
            ");
}
Console.WriteLine("\n");

return 0;
}

//
            Adds the value specified by second argument to each element
//
            of the array specified in the first argument
private static void
            Add (int[] Numbers, int operand)
{
Console.WriteLine ("Adding
            {0} to each element", operand);
for (int i=0;
            i<Numbers.Length; i++)
{
Numbers[i] +=
            operand;
}
}

// Multiplies each element of the array
            specified in the first
// argument by the value specified by
            second argument
private static void Multiply (int[] Numbers, int
            operand)
{
Console.WriteLine ("Multiplying each element by
            {0}", operand);
for (int i=0; i<Numbers.Length;
            i++)
{
Numbers[i] *= operand;
}
}
}
}

The following screen shot shows the output for DelegateEx2.exe:

Conclusion:

In this article, we have studied the use of delegates. Delegates are equivalent to function pointers in C and C++. However, since these are object-oriented, they are type-safe and do not carry the disadvantage associated with pointers. The article demonstrated how to declare a delegate, how to initialize a delegate and how to invoke a delegate. The composition of delegates was discussed. The composition allows you to invoke multiple delegates in the specified order in a single method call. Finally, the System.Delegate class and its important methods were discussed and some practical uses of delegates were highlighted.

C#Today - © 2002 WroxPress Limited


Using Legacy COM(+) in .NET ?Part 1

Using Legacy COM(+) in .NET ?Part 1
http://www.csharptoday.com/content.asp?id=1540
Article Rating: 3.5
Published on June 15, 2001
Abstract
The .NET framework introduces new technologies aimed at enhancing developer productivity and improving development cycles. .NET introduces new paradigms for software development that are very different from the existing paradigms. While this makes new development easier, companies cannot afford to ignore their investment in existing applications. Further, the upgrade cycles are different for different components, and very few companies can afford the luxury of rewriting all existing applications using the .NET framework. Keeping this in mind, Microsoft has designed .NET with the ability to interact with existing COM and COM+ components. We will explore the details of this interoperability in a two part article. In this part, Kaushal Sanghavi covers the basics of how COM and .NET components interact.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

The .NET framework introduces new technologies aimed at enhancing developer productivity and improving development cycles. .NET introduces new paradigms for software development that are very different from the existing paradigms. While this makes new development easier, companies cannot afford to ignore their investment in existing applications. Further, the upgrade cycles are different for different components, and very few companies can afford the luxury of rewriting all existing applications using the .NET framework. Keeping this in mind, Microsoft has designed .NET with the ability to interact with existing COM and COM+ components. We will explore the details of this interoperability in a two part article. In this article we will cover the basics of how COM and .NET components interact and in part two of this article, we will cover the advanced features of interoperability.

The interoperability features in COM and MTS under Windows NT are similar to the ones for COM+ under Windows 2000. In the rest of this article, we will use COM to refer to both COM and COM+.

COM and .NET Components

COM and .NET are similar but, at the same time, very different technologies. Before we discuss how these two can interact, let us cover some of the fundamentals of COM and .NET components.

Let us start by describing a COM component. Put very loosely, a COM component is any object that conforms to the binary standards that COM enforces, and that supports the IUnknown interface. The IUnknown interface allows clients to invoke the COM object and also controls the object lifetime. The underlying principle behind COM specifies that interfaces are immutable and once published, they never change. Each interface and object is uniquely identified by a GUID - Globally Unique Identifier, which has a 128 bit value, and is stored in the registry. The COM runtime maps COM object calls to a GUID, which points it to the location of the COM object. This information is stored in the registry under the HKEY_CLASSES_ROOT\CLSID key. All registered classes are stored here, and usually have an InprocServer32 or LocalServer key that point to the location of the object. A client uses this information to locate a COM object and to utilize its services. Further, COM components generate type libraries that can be used by calling clients to obtain information about the COM object and its exposed interfaces and types.

.NET managed components behave very differently from COM components. Managed components do not interact with the registry to store and locate information. Instead, the metadata is self contained in the assembly and the runtime looks for the assembly in the local or global cache. Further, they do not utilize GUIDs to identify objects, and they do not support the IUnknown interface. Object lifetime is managed by the .NET runtime via garbage collection, and unlike COM components, clients do not explicitly call methods to create and destroy references. A managed component is identified by its namespace and object name and it does not need to be registered with any runtimes. The .NET runtime will look into the global assembly cache or the application cache to locate the assembly; this information is not obtained from the registry. Further, every assembly has embedded metadata that completely describe the assembly and its types. Unlike COM, managed components do not generate type libraries.

Invoking COM Components from .NET

As you can see from the above section, the semantics of object creation and maintenance are very different under COM and .NET. For a managed component to invoke a COM component, at the least, it would need to know the class IDs and interface IDs of the component, and it would need to call the AddRef and Release methods on the COM component to maintain its lifetime. However, managed components are not aware of the IUnknown interface, and it is not practical for the managed components to explicitly support this interface solely to support legacy COM components. What we need is seamless support from the .NET runtime that allows managed components to transparently invoke COM components in the exact same manner as it would invoke a managed component. The .NET runtime supports this by creating a managed wrapper around the COM object that takes care of the incompatibilities between .NET and COM. This is the Runtime Callable Wrapper (RCW) that takes a COM component and makes it accessible to managed components. The RCW acts as a COM component on one side and as a managed component on the other, acting as a bridge between COM and .NET. Note that, since there is now an extra layer that the calls have to go through, these calls will not be as efficient as calls that are made directly to the COM components.

Let us now focus on creating the metadata for an existing COM component. The simplest way to create this is to use the Type Library Importer (TlbImp). This utility ships with the .NET framework and reads a COM type library and generates and equivalent .NET assembly. Once this assembly is created, you can reference this assembly in your managed code and invoke the COM component via calls to this assembly.

Consider a simple VB 6.0 component with a single method as follows. The method belongs to the SimpleCalculator class in a project named VB6Calc:

Public Function Add(ByVal iOperand1 As Integer, ByVal iOperand2 As Integer)_
As Integer
Add = iOperand1 + iOperand2
End Function

You can run the TlbImp on this COM DLL as follows to generate an equivalent assembly.

tlbimp VB6Calc.dll /out:ManagedVB6Calc.dll

The namespace that the managed class belongs to is the same as the type library, which is VB6Calc, and the class and method names remain the same. Further, since every managed object is derived from System.Object, the methods supported by System.Object are supported by this object as well. Using this assembly is identical to using a native managed component.

SimpleCalculator lCalc = new
            SimpleCalculator();
int lResult = lCalc.Add (2,3);
string
            lString = lCalc.ToString ();
System.Type lType = lCalc.GetType
            ();

In addition to using the TlbImp, you can use VS .NET to import a COM type. You can do this by adding a Reference to the COM DLL, which creates a managed assembly and adds it to the application's local cache. Further, you could also use the TypeLibConverter class from the System.Runtime.InteropServices namespace to convert an in memory type library to metadata. Once this is converted, you can use reflection to dynamically bind to the COM component and invoke methods. Reflection is the support provided by the runtime to dynamically query components and invoke methods on them.

Runtime Callable Wrappers

As we discussed in the above section, a runtime callable wrapper is the bridge between a COM component and a managed component. In this section, we will describe in brief the tasks carried out the RCW.

  • Map underlying interfaces: The RCW maps calls to the underlying COM interfaces. Every time a new interface is requested, the RCW performs a QueryInterface on the underlying COM component and caches the returned interface. On subsequent calls, the cached interface is made available to the managed component. There is a single RCW for each COM component that is being accessed.
  • Maintain Object Lifetime: COM requires using AddRef and Release to maintain object lifetime. However, since the managed components do not explicitly call AddRef and Release, the RCW has to maintain a reference on the COM object to ensure that it is not released while a managed object is still connected to it. The RCW maintains a reference on each interface pointer it wraps, and calls Release on these interfaces when it itself is being freed by the garbage collector.
  • Marshalling: This is the most important functionality of the RCW. The RCW marshals the calls from the managed component to the COM component. This involves converting types into those recognized by the managed and unmanaged worlds, handling errors in the COM component and handling return values and output parameters. Further, this ensures that all threading and apartment issues are resolved. The RCW creates a COM apartment when calling the unmanaged legacy COM component. This apartment depends on the threading model of the legacy COM component and can be a STA or a MTA (or free threaded). If the threading models of the managed component and the legacy COM component are compatible, then COM and the runtime will allow direct calls to the legacy COM object. This offers the best performance, but if the apartment models are not compatible, all calls are marshaled through a proxy created by the COM runtime. This incurs a performance hit, but cannot be avoided if the thread semantics are to be preserved. When the managed code makes its first call into the COM object, COM requires the RCW to call CoInitializeEx to initialize the COM apartment. If the managed object wants to explicitly specify the type of apartment that is created, it can use the ApartmentState property that is exposed by the managed thread as follows.

    Thread.CurrentThread.ApartmentState = ApartmentState.STA; //STA
    Thread.CurrentThread.ApartmentState = ApartmentState.MTA; // MTA
    // runtime decides this
    Thread.CurrentThread.ApartmentState = ApartmentState.Unknown;

  • Consume Interfaces: A COM component supports multiple interfaces such as IUnknown, IDispatch, IErrorInfo that provide the plumbing necessary for COM to work. These have no parallels and are not required in the managed world. The RCW consumes these interfaces and does not expose them to the managed object.

Invoking .NET Components from COM

Now that we know how to access COM components from .NET components, let us focus on accessing .NET components from COM components. Similar to the RCW, we need a bridge between the unmanaged and managed worlds that will translate and interpret the semantics on each side for object creation and method invocation.

When a client tries to create a COM component, the first step the runtime needs to take is to locate the executable that hosts this component. This is done by looking at the ClassId or ProgId values in the registry and mapping them to a LocalServer or InprocServer32 key. This key contains the location of the executable that is loaded into memory by the COM runtime. Once this is loaded, the client invokes a series of QueryInterface calls to obtain the interface pointers on which it can invoke the method calls. Thus for every managed component that is exposed as a COM component there needs to be a minimum of a registry entry pointing to the managed component and there should also be support for IUnknown which will allow clients to obtain interface pointers and manage the object lifetime. Similar to the RCW, the .NET framework creates a wrapper for managed components called COM Callable Wrapper (CCW). This is the bridge that allows legacy COM components to call managed components. This is completely transparent to both the calling object and the managed component. The client believes that it is calling a COM component and the managed object believes that it is being called by another managed object.

The .NET framework provides a utility similar to the TlbImp utility, called TlbExp. This is the Type Library Exporter and works in the reverse direction as TlbImp. TlbExp takes a managed assembly and creates a type library from this assembly's metadata. This type library can now be used from legacy COM applications to create an early bound reference to the managed type. Consider the managed class definition as follows:

namespace ManagedCalc
{
using
            System;

public class SimpleCalculator
{
public int Add
            (int iOperand1, int iOperand2)
{
return iOperand1 +
            iOperand2;
}
}
}

Running TlbExp on this will generate a type lib that can be accessed from an unmanaged client.

tlbexp ManagedCalc.dll /out:UnmanagedCalc.tlb

We can open this type library in OleView to view its contents:

As you can see from the above screenshot, the generated type library looks identical to a type library that would have been generated from a native COM component. Further, you can see that it supports the Add method from our class and the methods inherited from System.Object, as well as supporting the IDispatch interface, which is derived from IUnknown. You can now add a reference to this type library from a VB client, and take advantage of early binding to access this object.

Creating the type library allows clients to access this managed component at design time. However since no entries are created in the registry for this component, the COM runtime will be unable to locate the managed component and the CCW. The .NET framework provides another tool that creates the correct registry entries that allow the COM runtime to locate the assembly. This tool is called RegAsm and should be run on your managed assembly as follows:

regasm ManagedCalc.dll /tlb:UnmanagedCalc.tlb

This tool registers the assembly and the type library. Note that TlbExp only creates the type library; it does not register it, while RegAsm will create and register the type library. We can verify this by examining the registry.

As we can see from the above screenshot, a CLSID key is generated by RegAsm for our managed type. However, the Default string value does not point to the location of our assembly, it is pointing to C:\WINNT\System32\MSCorEE.dll. Since our assembly cannot be directly accessed from a COM component, the InprocServer32 points to MSCorEE.dll which is responsible for creating the CCW that will delegate calls to the managed object.

COM Callable Wrappers

The tasks carried out by the COM Callable Wrappers are very similar to those carried out by the Runtime Callable Wrappers, albeit in the reverse direction. Let us describe in brief some of the key responsibilities of the CCW.

  • Map Managed Object Classes: The CCW maps calls from COM components to the underlying managed classes. One of the basic principles of COM requires that all interfaces on the same object have the same IUnknown pointer. The CCW has to support this by ensuring that for every managed object invoked by the client, a single CCW exists.
  • Maintain Object Lifetime: COM clients call AddRef and Release to control object lifetime. The CCW maps these calls to a holding a reference on the managed object. This ensures that the managed object will not be garbage collected while the COM client is still active.
  • Marshalling: Similar to the RCW, this marshals calls and parameters from the unmanaged world into the managed world.
  • Impersonate Interfaces: COM components are required to support IUnknown at least. Dispinterfaces support IDispatch as well. Since managed objects are not required to support these interfaces, the CCW has to implement these itself and impersonate the managed object when methods on these interfaces are invoked.

Limitations

As we have seen in the above sections, it is possible for managed and unmanaged objects to interact. However, there are certain limitations to this approach, based primarily on marshalling issues. Not all types have an equivalent in both the managed and the unmanaged spaces, which prevents objects using these types from interacting. Further, there are certain features that are not supported in the unmanaged world, like parameterized constructors, which again prevent interoperability. The following are some of the main limitations to interoperability between COM and .NET components.

  • RCWs work only with methods that use Safe Arrays. Variable length arrays are not supported.
  • RCWs map all successful HRESULT as a single success return. Thus, components using multiple success HRESULTs to pass status information will not work correctly.
  • TlbImp does not import module level functions.
  • TlbImp does not import methods with type unsafe arguments, such as void*.
  • CCWs can only expose classes with default constructors. Parameterized constructors are not exposed.
  • Static methods in a class are not exposed.
  • Const fields in a class are not exposed.

The Download Code

I have included a sample download to demonstrate calling a COM object from managed code and also to demonstrate calling a .NET component from unmanaged code. This takes the form of the simple Calculator example mentioned earlier:

Conclusion

Interoperability between COM and .NET was an important design goal for the .NET runtime. The RCWs and CCWs provide the plumbing necessary to allow managed and unmanaged components to interact and work transparently. This article dealt with the most simplest example of interoperability. In the next part of this article, we will explore some of the advanced features such as error handling, COM+ services, shared properties and event handling.

Happy .NETing.

C#Today - © 2002 WroxPress Limited


Web Site Error Logging and Reporting System

Web Services

Web Services (Sort by Name)

.NET Services
Accessing Web Services from Windows Client
Asynchronous invocation of web services
Building a List Server
Building a Simple ASP.NET Editor for Web Configuration
Building Message Board Web Services with C# and .NET - Part I, The Web Services
C# Browser History Monitoring Application - Part 2
COM+ and C#
Creating a .NET Assembly Listener for Remoting
Creating an Order Tracking Application using .NET Framework

Developing A Rich Client Forum Reader Using Windows Form and Web Service
Integrating an HTML returning C# Web Service with our Websites
Interoperating Java, VB6, and .NET Web Services
Introduction to UDDI
Mail Tracking and Analysis System
Mobile Enabling Existing Applications with the Mobile Internet Toolkit
Monitoring IIS Systems
Multiple Thread Techniques for a C# Windows Service
Programming the GXA
Programming Web Services with C# Using SQL

Remote querying your data warehouse using web services and XML
Transactions In Web Services
Using Web Services Asynchronously
Web Services Performance and Scalability Design Considerations


Programming the GXA

Programming the GXA
http://www.csharptoday.com/content.asp?id=2014
Article Rating: 5
Published on December 2, 2002
Abstract
In this case study, Richard Conway looks at the GXA (Global XML Web Services Architecture). The GXA provides principles, specifications, and guidelines for advancing the protocols of today's XML Web services standards to address more complex and sophisticated tasks. The result of GXA could be that agreements and implementations can be streamlined with standards such that the issue of interoperability will be one of the past; no longer will we have to worry about different TP (transaction processing) standards such as XA or the MSDTC, or different credential management standards when calling a java application from a .NET application or vice versa - this will all be implicit within the GXA, and each respective web service environment will support it.
Case Study

The surge in the use of the Internet over the last few years has given rise to an increased number of agreements on standards covering everything from browser scripting languages to XML APIs. The speed at which these standards are being ratified has accelerated due to the ever-dependent nature of business transactions and economic integration on the web. Through the 1990's we were tirelessly subjected to many delayed standards involving the most widely used RPC standard, CORBA, followed by Microsoft's own DCOM standard. The challenge of integrating distinct systems - each with their own trademark reliant on different RPC standards, written in different languages, and existing on different platforms - has driven the adoption of standards which compliment the hyped XML web services in order to allow these said systems to enjoy a complete set of implementation independent features akin to those of DCOM and CORBA. As XML web services have begun to be adopted by major corporations, the challenge is to now provide the web service infrastructure to allow web services to be used independently of the underlying implementation and provide advanced facilities such as transaction, messaging, naming, and routing which are inherent in many other proprietary distributed object protocols (whether a web service application is written in .NET, Java/EJB, PHP, etc.).

Web Services reflect a concerted effort by software vendors to end the disintegration that has ensued from proprietary protocols. They now have innate support in .NET, J2EE, and other programming languages/environments. The current web service standards provide the most basic functionality offered through RPC-based protocols such as DCOM. The actual invocation request to the web service is done through SOAP (Simple Object Access Protocol) that contains method names and parameter name\values. SOAP has a corresponding response that contains the return types and values. Web services' SOAP requests are normally contained within the body of an HTTP request and as an addendum, existing HTTP headers will normally contain a specific SOAPAction header which will enable the routing of the message to the appropriate web service.

In order to provide appropriate language-binding and (de)serialization, proxy classes in C#, java, etc, can be written to enable the total abstraction of the web service. Tools such as wsdl.exe supplied with the .NET Framework can use the web service standard WSDL (Web Services Description Language) to provide a C# or VB.NET proxy class representation of the web service. WSDL simply describes the web service; namely all the methods, parameters, and return values; tools such as wsdl.exe can then be used to allow us to appear as if we are calling a method on an object (with respect to DCOM, WSDL is reminiscent of a type library). The last currently ratified web service standard is UDDI (Universal Description and Discovery Interface). This is another XML protocol (like the above two) that can be used to query or write entries into an online business directory that is used to provide listings of corporate web service offerings.

So, in these three protocols we can find a web service, describe the service interface, and invoke methods on the service. It doesn't take long to realize that we've just described facets of RPC-based protocols such as DCOM and CORBA. However, they are not quite the same! Both DCOM and CORBA offer a wide range of services that can be used to provide security, transactions, reliable messaging, and more. In order to be able to extend the value of web services, we must ensure that the above services are currently offered independently in the web services model. This means that transactions, routing, security, and encoding should be supported as part of the web services model. In this way, there is no fixed dependency between web services and other application layer protocols such as HTTP, FTP, and SSL. The drive for these new standards is coming from Microsoft, IBM, and Sun, as well as many other firms who are members of standards body working committees. The result of this cooperation is the GXA (Global XML Web Services Architecture), and if successful, agreements and implementations can be streamlined with standards such that the issue of interoperability will be one of the past; no longer will we have to worry about different TP (transaction processing) standards such as XA or the MSDTC or different credential management standards when calling a java application from a .NET application or vice versa - this will all be implicit within the GXA, and each respective web service environment will support it.

Where we are now

This case study follows the release of the WSDK (Web Services Development Kit) that is the first foray into the world of the GXA. Earlier this year, Microsoft released a DIME sample that introduced the GXA to many developers worldwide. DIME is a way of encoding web service request/response packet content. By encoding, I refer to the transport layer's independent need to use web services, since many transport layers (e.g. HTTP) will re-encode the binary data of a SOAP attachment (such as image data).

Since this it not a very efficient way of transferring data, DIME (a transport independent protocol) has been proposed to allow free encoding schemes. We shall delve into DIME later in this case study.

The GXA follows an amalgamation of technologies and pseudo-standards which represent the historical stepping-stones to some of the standards we find are about to be ratified. WS-Security is the newly proposed standard that is present in the WSDK and encompasses the following three protocols.

  • XML-Encryption
  • XML-Signature
  • XKMS (XML Key Management)

These three protocols are currently supported within the System.Security.Cryptography namespace in the core Framework class library. They individually and respectively allow the encryption of SOAP messages, embedding a digital signature into a SOAP message, and specifying a public key distribution protocol. This way, keys and cryptographic keys and data can be serialized and deserialized to XML.

It has to be stressed that as XML web services formed the push for .NET (even ardent Microsoft cynics have been quietly pleased with respect to the level of XML Web Services support in .NET and VS.NET) so will the evolution of the GXA form the backbone of the next version of .NET. In this way, we have to understand that the GXA is in evolution at the moment. The historical context of each of the protocols covered in this case study will lend credence to this idea! It may seem weird to the uninitiated of the Microsoft public beta program how disjointed this WSDK is, but rest assured that by the final ratification of all the GXA protocols, it will integrate seamlessly with the VS.NET (the PDC 2001 suggested that many of the updates in the next version of VS.NET will be centered around the inclusion of the GXA).

Introduction to WSDK Programming

Before focusing on DIME and WS-Attachments specifically, it is necessary to explain the key features of the WSDK and how to integrate it into our future web services. As you can imagine, this is the first installment of this SDK, so there will be minor modifications and additions on subsequent releases. However, this release has designed a fairly cohesive programming model which will one day be part of VS.NET and specifically the SOAP and web services classes, such that some of the configuration features we will use to configure our projects whilst programming the WSDK will almost certainly no longer be necessary in the future.

Readers who are familiar with the .NET My Services SDK will appreciate the XML markup representative for WS-Security and WS-Routing that seem to have changed little since the release of the .NET My Services SDK beta. In fact, this is an encouraging sign since as Microsoft put .NET My Services on hold we can assume that the enterprise web services model is still a big part of their strategy with the release of the WSDK.

For those readers who have little or no knowledge of .NET My Services, in summary: it is a suite of Enterprise Web Services which Microsoft (at one stage) were to host in a data center which could be used to store all types of information. Different services such as .NET Calendar and .NET Contacts had XML schema that would allow the use of Personal Information Management through a SOAP interface. Judging by recent comments of Steve Balmer, the company is now considering repackaging .NET My Services to be hosted by corporations rather than Microsoft and extending current offerings such as .NET Passport so that it exists within a federated model, allowing .NET My Services information sharing beyond corporate boundaries. The current .NET My Services offering includes .NET Passport, .NET Presence, and .NET Notifications. More can be found out about the current state of .NET My Services at http://msdn.microsoft.com%20/myservices?WROXEMPTOKEN=706301ZgHcQjRStEwcXgGTvV8s.

The WSDK can be downloaded from the Microsoft site at http://msdn.microsoft.com/downloads/default.asp?URL=/downloads/sample.asp?url=/MSDN-FILES/027/001/997/msdncompositedoc.xml&WROXEMPTOKEN=706301ZgHcQjRStEwcXgGTvV8s

Once installed, projects can be written using the GXA protocols described in this case study by simply adding a project reference to the Microsoft.WSDK.dll assembly.

In order to begin WSDK programming we must first consider a web services client that has to construct the SOAP envelope. The WSDK exposes a new class called SoapContext. SoapContext can be considered in the same light as HttpContext, which allows access to ASP.NET objects such as Request, Response and Cache. SoapContext will allow us access to all of the web service protocols that we will use in this case study and abstract the creation of such things as security credentials, routing rules, and attachments. By this, I mean that we won't have to create the XML document representative of these GXA protocols ourselves and begin by appending nodes to the DOM containing our SOAP envelope!

Building a WSDK Client

Let's briefly recap somewhat on how to create a web service proxy class in C#. Once we have written a web service, we can then generate the web service WSDL that describes the web service including web method calls, parameters and return values. We would do this by simply using the Add Web Reference menu item whilst right clicking on the project in the solution explorer pane. This will produce a window which allows the entry of a URI post-fixed by the ?WSDL query string. When the reference is added, a C# file called reference.cs will be created which contains the C# proxy code abstracting calls to the web service and based on the WSDL contract. In order to view this file we must ensure that the Show Files menu option in the Project menu is checked.

Perusing this file will reveal that the web service proxy inherits from System.Web.Services.Protocols.SoapHttpClientProtocol.

This class contains all the information about the packaged SOAP message, but obviously doesn't support the supplementary web service protocols that the WSDK contains. In order for us to use the GXA protocols, our client acting proxy class must inherit from WSDKClientProtocol instead of SoapHttpClientProtocol . In order to do this, we must remove the text above and replace it with WSDKClientProtocol. We will also need to add a using directive at the top of the proxy file:

using Microsoft.WSDK;

We will now, when creating an instance of the proxy class used by a client application to invoke web methods, have access to the SoapContext, allowing us to construct a SOAP Envelope including the currently supported GXA protocols.

Configuring the WSDK environment

The web service must also be able to parse the message and interpret any use of the GXA protocols. Currently the web service .asmx files use an entry to the machine.config file (this file contains all the default settings for our web applications so that they don't have to be replicated in every web.config file - this file resides in the C:\[win_dir]\Microsoft.NET\Framework\[version]\Config directory).

The entry is an HttpHandler that processes the SOAP Envelope sent by the client:

<add verb="*" path="*.asmx" type="System.Web.Services.Protocols.WebServiceHandlerFactory,
System.Web.Services, Version=1.0.3300.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" validate="false" />

Rather than change this handler, a more generic HttpModule is used to filter the GXA messages to the web service. In this way, it will handle the request distinctly from the HttpHandler, enabling the use of GXA over different mediums and not related to .asmx pages. This can be added to the web.config file in the web service project that will allow web service traffic containing GXA protocols to be parsed.

<httpModules>
<add name="WSDK" type="Microsoft.WSDK.HttpModule, Microsoft.WSDK, Version=1.0.0.0, Culture=neutral,
PublicKeyToken=31bf3856ad364e35" />
</httpModules>

Now that we have an understanding of the basics of configuring the WSDK for use, let's introduce the DIME protocol.

Introduction to DIME

Direct Internet Message Encapsulation (commonly known as DIME) emerged from Internet encoding standards that would force binary data to be encoded into Base64. Base64 encoding is achieved through the use of bitwise segmentation, i.e. every 64 bits a representative character is used from the Base64 alphabet, which includes all numbers and characters as well as a series of symbols - all of these are printable characters. Base64 will have to contain a total number of bytes that is a multiple of three, as 3 bytes contain 24 bits, and this will be converted into 4 sets of 6 bits and padded with 2 high order bits. In the event that the string isn't a multiple of 3, it will be padded with an '=' sign to maintain a correct length.

As you can imagine, transferring large binary objects in the body of a web service message can be very cumbersome (it will also increase the size of the message, increasing network traffic by approximately 33% - in effect the 2 high order bits in a Base64 character will always be redundant zeros). Multiply this by several hundred requests per minute and the encoding begins to be important. DIME is the solution to this problem (well one of the proposed solutions!). It is effectively a transport protocol for attachment data so that large binary attachments can have different encoding independent of the underlying transport. DIME is optimal for transferring image or movie data such as JPEG, GIF or MP3 (since no encoding rules are applied, meaning that the application layer has to do less conversions) and can better be used to deliver streaming content through a web service.

DIME is a record based protocol. This means that the attachment information is compartmentalized into a number of records. A DIME message can therefore be considered as one or more DIME records. Each record will contain information about itself, such as its length and other optional parameters. The DIME Message is given a unique ID that can be propagated to the DIME records. The ID can then be used to correlate message chunks with messages. It is also possible to have the message contain a single record, rather than segmenting the message. Allowing this type of segmentation behavior is analogous to how HTTP can be used to create to chunked responses by enabling a fixed byte length for the response size.

DIME can be considered as an addendum to XML in that there may be occasions where it will be better for use to transfer binary data with the SOAP message. As well as the use of inefficient encoding by HTTP, we can also argue a case that many systems will have to undergo further work and inefficiencies to be able to have SOAP talk to a web service, so allowing binary data in the form of DIME to be transferred with the SOAP data will allow such systems to send information to a web service without several conversions to and from XML.

Fig 1 - The relation between DIME and SOAP routing rules - the DIME packet will contain a series of payloads in chunks, the first of which is the web service routing header information

The table below can be used for reference purposes to acknowledge the headers contained in a DIME record:

Header Name | Description | Value Range
MB | Message Begin - This marks the beginning of the DIME Message - If the flag is set then it refers to the first DIME record in the message. | 0 Clear 1 Set
ME | Message End - This marks the end of the DIME Message - If the flag is set then it refers to the last DIME record in the message. | 0 Clear 1 Set
CF | Chunked Format - This refers to a message chunk. This header is set only if there are no chunks of binary data and message attachments (SOAP attachments are discussed later in the chapter). The first chunk and subsequent middle chunks will have the CF flag set. The last chunk should have the flag clear. | 0 Clear 1 Set
DATA_LENGTH | This represents the length of the DATA payload - the specification makes clear that the payload must be padded to be a multiple of four Octets - this padding length, however, is not included in the DATA_LENGTH | 32 Bit unsigned Integer value
DATA | This is the actual payload | Bit stream of DATA payload
ID | This is the ID of the message - it is an URI that is used to identify the message. There is no requirement to suggest that each message chunk within a chunked payload has to contain an ID value - only the first one. The ID must be padded so that its length is divisible by 4 octets as with the DATA. | Bit stream of ID value
ID_LENGTH | This is the length of the ID header not including the padding value. | 13 Bit unsigned Integer value
TYPE | The TYPE header should be the absolute value of the suggested TNF header. It can be anything, including a MIME identifier or an URI. It can also indicate other things in this header, such as character encoding. | Bit stream of TYPE value
TYPE_LENGTH | The length of the type value not including the padding length. | 13 Bit unsigned Integer value
TNF | This is the Type Name Format. It indicates the value in the TYPE field. The value 0 should be used in any middle chunked payload. The value 1 should be used for a well-known media types such as image/jpeg for image data. An absolute URI can be used to reference a WS-Routing message http://schemas.xmlsoap.org/rp/?WROXEMPTOKEN=706301ZgHcQjRStEwcXgGTvV8s. Other URIs can be used for other protocols. An unknown TNF value signifies that it is effectively an octet stream, and the application receiving the message may implicitly know what it is in this case without the need for definition | 0 Unchanged 1 Media type 2 Absolute URI 3 Unknown 4 None 5-7 Reserved

As you can see from the table, the DIME protocol is very thorough. The best way to get acquainted with DIME is through the Microsoft DIME sample which preceded the WSDK by some months and can be found on the MSDN website. This sample contains all the code for the first version of the DIME parser. Even though it has been updated within the WSDK, it will still give a good insight into the working of DIME.

The WSDK exposes several DIME classes, among them:

  • DimeReader
  • DimeWriter
  • DimeRecord
  • DimeAttachment

For the purposes of this example, we can concentrate solely on the 4th class, as this release of the WSDK doesn't support DIME chunking (I will display some code later in this example to demonstrate how to view individual DIME records however - it won't function correctly until the next release though).

The DimeReader and DimeWriter classes read and write to and from any stream. This can be a NetworkStream, FileStream, CryptoStream, etc. The DimeWriter class should be configured to output a series of DimeRecords to the stream, and the DimeReader to read records sequentially from the stream. The DimeRecord class can be returned from the DimeReader, or created and written to by a DimeWriter. In this way, the DIME records containing all the headers specified in the above table can be used to write records to the web service call. The bad news is that the current implementation of the WSDK only supports a single DIME chunk, so even though we have access to the reader and writer classes, there are no individual records returned to us. This will change in line with the DIME protocol in a future release of the WSDK.

Building a DIME web service

We'll consider the DIME web service first, which is simply a standard VS.NET web service project. At this point, the web.config file should have been configured correctly (i.e. the HttpModule should have been added). To enable DIME however, another entry must be created:

<webServices>
<soapExtensionTypes>
<add type="Microsoft.WSDK.Dime.DimeExtension, Microsoft.WSDK, Version=1.0.0.0, Culture=neutral,
PublicKeyToken=31bf3856ad364e35"
priority="1" group="0"/>
</soapExtensionTypes>
</webServices>

This is used to alert a section handler within the WSDK.dll assembly that DIME is being sent to the web service. The above must be added to the <system.web> web service block.

In addition to this, we should begin to use the tracing capabilities of the WSDK to allow us to diagnose any exceptions processed by the WSDK filter. This simply writes the SOAP request and response to two named files that can be used to diagnose and check the content of the SOAP messages between client and server.

<microsoft.wsdk>
<diagnostics>
<trace enabled="true"
input="request.txt"
output="response.txt"/>
<faultDetail enabled="true" />
</diagnostics>
</microsoft.wsdk>

The trace tags are fairly self-explanatory, simply writing the SOAP message to the files request.txt and response.txt. The <faultDetail enabled="true" /> enables error messages to be captured, detailing the exact fault that occurred within the request.

The web service (called DimeAttachmentService) will process images sent from a windows forms client and save them to a temporary directory on the hard drive. For the purposes illustrated, all code is written in loopback using the localhost alias, but as long as the rules described so far are adhered to, then this sample can be used from any two visible points on a network. The namespace of the web service should be changed to reflect the uniqueness of the service rather than simply using the default namespace http://tempuri.org/[web_service]. This is done through the use of attributes above the class definition:

[WebService(Namespace="http://richardc.wrox.com")]
public class DIMEAttachmentService : WebService

Our web service only exposes one method, called AcceptAttachments (taking no parameters and with a return type of void - however, there are no restrictions on how you must write web services to use the WSDK). The byte array bDimeArray is declared in the example below to buffer content from the DIME image data - it can be used to read from the underlying DIME stream and then write to a file (or another stream).

try
{
string extension = String.Empty;
byte[] bDimeArray = new Byte[2048];
//create the graphic file
switch(HttpSoapContext.RequestContext.Attachments[0].Type)
{
case "image/bitmap":
extension = "bmp";
break;
case "image/jpeg":
extension = "jpg";
break;
case "image/gif":
extension = "gif";
break;
default:
extension = "bin";
break;
}

The code above uses the static property RequestContext of the HttpSoapContext provided and populated through the use of the HttpModule and referenced by including the using statement.

using Microsoft.WSDK;

This RequestContext contains a collection of properties that can be used to inspect the contents of the GXA message. This forms the basis for our use of the WSDK. As can be inferred from the example, there is an equivalent ResponseContext that gives the SoapContext for the response message to the client. In fact, our use of this has actually taken precedence over the web method call itself that is simply used to process the GXA message. The Attachments property returns a DimeAttachmentsCollection object that can be enumerated via a for-each loop or other index based loop to return each DimeAttachment object. Above, I purposely index element zero since only one attachment is sent through by the client. The Type property will return the MIME type of the file serialized to DIME. This is essential to reconstruct the message correctly, as no file naming data is stored with the DIME message, so the extension cannot be determined without:

  • Using the DimeAttachment.Type property populated by the client
  • Adding an input argument to the web method naming the file
  • Inspecting the file data to retrieve data about the file type (overkill, and in some cases impossible!)

The following will create an output file for the image and return a stream that can be written directly to the output file. Obviously in a real-world application we may want to create random image names, carefully avoiding name collision, and store these values in a database table or something so that an application can dynamically access images sent to a web service over DIME.

fileImage = File.Create(@"C:\temp."+extension);
//use the attachment stream
Stream dimeBaseStream = HttpSoapContext.RequestContext.Attachments[0].Stream;

The last thing that needs to be done is writing the actual DIME data to the file.

while(0!=(dime =
            dimeBaseStream.Read(bDimeArray, 0,
            2048)))
{
fileImage.Write(bDimeArray, 0,
            dime);
fileImage.Flush();
}

That was quite easy. The WSDK has abstracted our use of DIME through the GXA protocol WS-Attachments. This can be used to avoid any knowledge of DIME altogether and focus solely on the DIME attachment data sent - so even though in future version of the WSDK (when chunking will be supported) access will be given to the DimeRecord itself - there may be no need to use it (unless optimising the size of DIME packets for a particular reason). The following code could be used to determine information about the DimeRecord (s) in an accompanying stream (the current implementation doesn't support this though, so an exception will be thrown if this code is used! I simply included this so that a correlation could be made between the table illustrating DIME headers earlier and the property values of the DimeRecord class).

//read each dime record individually using the DimeReader ;
DimeReader dr = new DimeReader(dimeBaseStream);
while(null!=(imageRecord = dr.ReadRecord()))
{
swDimeRecord.WriteLine("Record Number: {0)", count);
swDimeRecord.WriteLine("Chunked: {0}", imageRecord.Chunked);
swDimeRecord.WriteLine("Chunk Size: {0}", imageRecord.ChunkSize);
swDimeRecord.WriteLine("Length: {0}", imageRecord.ContentLength);
swDimeRecord.WriteLine("Id: {0}", imageRecord.Id.AbsolutePath);
swDimeRecord.WriteLine("Is First Record: {0}", imageRecord.MessageBegin);
swDimeRecord.WriteLine("Is Last Record: {0}", imageRecord.MessageEnd);
swDimeRecord.WriteLine("Message Type: {0}", imageRecord.Type);
swDimeRecord.WriteLine("Format Type: {0}", imageRecord.TypeFormat.ToString());
swDimeRecord.WriteLine("Record Content: {0)", (new StreamReader(imageRecord.BodyStream)).ReadToEnd());
count++;
}

Building a DIME Client

In this example, I want to illustrate the use of the DimeAttachment class that can be used by the client (and the web service) to add any number of attachments to the SOAP message using DIME. The project DimeClient is a form that will allow an image to be sent to a web service using DIME.

In order to use DIME, we must have two using statements at the top of our C# file.

using Microsoft.WSDK;
using Microsoft.WSDK.DIME;

We must create a web reference to the DimeAttachmentService and change the proxy class so that it inherits from Microsoft.WSDK.WSDKClientProtocol (which will expose the SoapContext) as described earlier in this case study.

This client declares four form widgets and a stream to hold the image data from a file. When the textbox is clicked, an OpenFileDialog control is launched allowing the user to select from Bitmap, Gif, and Jpeg files. When one of these is selected, the image is loaded into the PictureBox control. When the button is pressed, the web service method is invoked after adding the image data to the request stream as a DimeAttachment.

private System.Windows.Forms.PictureBox pictureBox1;
private System.Windows.Forms.OpenFileDialog GraphicsOpener;
private System.Windows.Forms.TextBox txtImage;
private System.Windows.Forms.Button btImage;
private System.IO.Stream stream; private string ext = String.Empty;

The PictureBox is populated by setting the Image property to an Image object derived from the static FromStream() method of the Image class, and the stream is also stored in a member called stream. The file extension is then extracted from the FileName property of the OpenFileDialog control. This will be used to determine the type of image sent to the server.

private void txtImage_Click(object sender,
            System.EventArgs
            e)
{
GraphicsOpener.ShowDialog();
try
{
extension =
            GraphicsOpener.FileName.Substring(GraphicsOpener.FileName.LastIndexOf(".")
            + 1,
GraphicsOpener.FileName.Length -
            (GraphicsOpener.FileName.LastIndexOf(".") +
            1));
}
catch(Exception ex)
{
throw
            ex;
}
System.IO.Stream fStream =
            GraphicsOpener.OpenFile();
pictureBox1.Image =
            Image.FromStream(fStream);
stream = fStream;
}

When the button is clicked, the web service method is invoked. An instance of the web service proxy is created, and we create a DIME attachment specifying the MIME type, a value from the TypeFormat enum , and the stream itself, which will be turned into a DimeRecord.

The RequestSoapContext property of the proxy class returns a SoapContext object, allowing us to use the Attachments property to return a DimeAttachmentsCollection class and add an attachment to the collection. When the AcceptAttachments() web method is called, the web service method is invoked and the DIME and WS-Attachments data is sent to the web service.

DimeClient.localhost.DIMEAttachmentService localService = new DimeClient.localhost.DIMEAttachmentService();
switch(extension)
{
case "bmp":
extension = "bitmap";
break;
case "jpg":
extension = "jpeg";
break;
case "gif":
extension = "gif";
break;
}
DimeAttachment da = new DimeAttachment("image/"+extension, Microsoft.WSDK.Dime.TypeFormatEnum.MediaType, stream);
localService.RequestSoapContext.Attachments.Add(da);

// In this section we trap the actual web service call and return a string message which can be used to write to a log
file or database
try
{
localService.AcceptAttachments();
}
catch(System.Web.Services.Protocols.SoapHeaderException she)
{
string tmp = she.Message;
}
catch(System.Net.WebException we)
{
string tmp = we.Message;
}

Fig 2 - The Dime Image Client

WS-Attachments

The WS-Attachments specification is simply a suggestion as to how DIME messages can be used to encapsulate the underlying SOAP envelope in conjunction with the attachment data. This proposal follows on from an earlier W3C candidate proposal called SwA (Soap with Attachments) that was submitted in order to find a solution for binary data sent in XML. Where SwA fails however, is to define a definition for all of the message parts, so it isn't evident where one "attachment" begins and the other ends without reading the entire message. WS-Attachments define that the MIME type of the entire stream should be set to application/dime. This value is set by default when attachments are used with the SOAP message. The WS-Attachments specification suggests that the first DIME record is the SOAP Envelope itself, which has a MIME type set to application/xml+soap. Each subsequent record will be encoded with the MIME type of the relevant file attachment. WS-Attachments can be considered as the definition of the message rules to allow DIME to be incorporated into the web services model whereas DIME itself is simply the message format of the binary data.

DIME support was added to the SOAP toolkit version 3.

Using WS-Routing

WS-Routing allows a transport independent definition describing routing rules for the SOAP message. If we inspect the configuration files for the SOAP request and response in our DIME sample, we will see several SOAP headers that are used to control the routing rules independently of the underlying transport.

The WS-Routing specification contains definitions to enable a SOAP message to be routed to a particular destination and using defined proxy services either on route to the web service or on route from the web service. The core WS-Routing tags are:

  • path
  • action
  • to
  • id
  • from

<soap:Header>
<wsrp:path soap:actor="http://schemas.xmlsoap.org/soap/actor/next" soap:mustUnderstand="1"
xmlns:wsrp="http://schemas.xmlsoap.org/rp">
<wsrp:action>http://richardc.wrox.com/AcceptAttachments</wsrp:action>
<wsrp:to>http://localhost/DimeAttachmentService/service1.asmx</wsrp:to>
<wsrp:id>uuid:c554413b-26a8-4828-9a44-ab90a0e56929</wsrp:id>
</wsrp:path>
</soap:Header>

Each WS-Routing header block must contain a structure like the one shown above for the DimeAttachmentService.

The action tag represents the namespace URI to the service. The actual WS-Routing specification uses the soap:// protocol that has yet to have an established IANA Berkley standard port ascribed to it.

The to tag denotes the physical URI of the service.

The id tag denotes the unique id of the SOAP message.

The WS-Routing specification also makes reference to the use of DIME with WS-Routing. The first DIME record (which we described previously while reviewing WS-Attachments) MUST contain a WS-Routing block that governs where the DIME Message is sent to. From this point on, we will categorize an entity such as the WSDK server resource (HttpHandler) that parses the WS-Routing request as a SOAP router. This is an ever-popular form of address.

WS-Routing actually provides far more versatility than I've demonstrated with the above XML. It is quite possible that fragments of the message need to be processed by different web service endpoints (these may belong to different parts of a workflow process for example) between the sender and the recipient of the SOAP message. WS-Routing additionally supports this, enabling the message to be sent via another web service(s) on the way to the recipient or to another web service(s) on the response trip back to the sender. Using the words "web service" in the last sentence is somewhat of a misnomer, as we are really referring to a SOAP router that can have a set of rules based on a referral cache (which we'll come to later). The advantage of this is that we can develop a series of filters and change the SOAP message as it passes through each URI specified in the route, adding or subtracting message blocks or adding additional security between route points.

The XML fragment below includes the <fwd> and <rev> tag with a <via> collection of tags nested in each (there is in fact only one tag in each, but there could be many). On the way to the DimeAttachmentService destination the request and corresponding response will be routed through the WS-RoutingSample web service. The uses for this could be tenfold, as we could filter some of the traffic or modify the request at any one of the via collection points. We could additionally add to the request at any of these points.

<wsrp:path soap:actor="http://schemas.xmlsoap.org/soap/actor/next"
soap:mustUnderstand="1" xmlns:wsrp="http://schemas.xmlsoap.org/rp">
<wsrp:action>http://richardc.wrox.com/AcceptAttachments</wsrp:action>
<wsrp:to>http://localhost/DimeAttachmentService/service1.asmx</wsrp:to>
<wsrp:fwd>
<wsrp:via>http://localhost/WS-RoutingService/Service1.asmx</wsrp:via>
</wsrp:fwd>
<wsrp:fwd>
<wsrp:via>http://localhost/WS-RoutingService/Service1.asmx </wsrp:via>
</wsrp:fwd>
<wsrp:from>mailto:techierebel@yahoo.co.uk</wsrp:from>
<wsrp:id>uuid:eaf98d36-e7a3-4c7f-8bdd-946330734c46</wsrp:id>
</wsrp:path>

I won't replicate the documentation here, but suffice it say that the WSDK is nothing more than a collection of filters that serialize and de-serialize the SOAP request. This is effectively all the change in inheritance hierarchy does, and all the use of the HttpModule on the server side does too. To configure the SOAP router adds more configuration code to the application by having to register an HttpHandler. The chain of events is therefore:

  • The HttpModule populates the SoapContext when the request is sent to the server by parsing the request for GXA protocol markup.
  • The HttpHandler traps the request if it is to the specified requested page
  • Control will then pass to the web service which will answer the request

In order to use a SOAP router, we must add the following entry to the web.config file in our WS-RoutingSample web service application directory (we should also add the WSDK HttpModule, as in the previous example, and to every forthcoming project with which we intend to use the WSDK). Running the sample web service application WS-RoutingSample will result in a browser error initially. This is simply because the requests that the router supports can only be given in SOAP, and the browser sends an HTTP GET to the web service. However, we can continue past this point and operate the web service in debug mode.

<httpHandlers>
<add verb="*" path="Service1.asmx" type="Microsoft.WSDK.Routing.RoutingHandler,
Microsoft.WSDK, Version=1.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" />
</httpHandlers>

The important thing to remember about WS-Routing is that it specifies a model for a SOAP router, not for a SOAP request. This means that we can use WS-Routing to specify that the SOAP message should be sent via a series of locations, but none of these locations will service web service requests (unless they correlate to the namespace in the SOAP action header). Any number of <add> tags can be included in this file to trap requests to specific .asmx pages (as many as there are web services).

The current use of the WS-Routing protocol in this early release of the SDK is simply a conglomeration of WS-Referral so that routing rules can be created for services that have been closed down or have a fault. This is unfortunate, but in subsequent releases of the SDK it will prove an invaluable addition to the GXA supported by Microsoft.

In the example with this case study I have included a dummy project called DIMESecondService that acts as a SOAP router. It doesn't contain a project per se, as it is simply a web application (virtual directory) with a web.config file and another file that contains a cache of all the web service redirections. This file contains a WS-Referral document that contains a set of routing rules for the DIMEAttachmentService web service.

The first task is to change the constructor of the web service proxy class in the DIME client to point to a different URL. Beware of case sensitivity here - if it doesn't resolve the URL name in the DIMESecondService cache then an 'endpoint not found' exception will be generated. Unfortunately, in this respect, the SOAP router appears to be case-sensitive (this doesn't include the server name - it simply refers to the application name).

public DIMEAttachmentService() {
this.Url = "http://localhost/DIMESecondService/Service1.asmx";
}

In order to add a referral cache to the DIMESecondService application and add the ability to monitor the incoming and outgoing soap messages, we should add the following to the web.config file in the application directory. The name attribute of cache contains the name of a file in the application directory that contains the routing rules for the web service. We can actually have multiple cache elements, each pointing to a different file which can exist inside or outside of the application directory. It is important that the ASP.NET user has write access to this file.

<microsoft.wsdk>
<referral>
<cache name="reroute.config" />
</referral>
<diagnostics>
<trace enabled="true"
input="inputTrace.txt"
output="outputTrace.txt"/>
</diagnostics>
</microsoft.wsdk>

When the request reaches the SOAP router it will automatically forward it to the DIMEAttachmentService based on the rules in the WS-Referral document. The output below shows that the AttachmentService sends it to itself, but uses a via tag, which is actually the SOAP endpoint.

<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<soap:Header>
<wsu:Timestamp xmlns:wsu="http://schemas.xmlsoap.org/ws/2002/07/utility">
<wsu:Created wsu:Id="Id-6e4b530c-8a00-4261-9c95-13b3dc426236">2002-10-20T18:16:53Z</wsu:Created>
<wsu:Expires wsu:Id="Id-09f81254-4984-4700-872c-348f57bb2641">2002-10-20T18:21:53Z</wsu:Expires>
<wsu:Received Actor="http://localhost/DIMESecondService/Service1.asmx"
Delay="198">2002-10-20T18:16:54Z</wsu:Received>
</wsu:Timestamp>
<wsrp:path soap:actor="http://schemas.xmlsoap.org/soap/actor/next" soap:mustUnderstand="1"
xmlns:wsrp="http://schemas.xmlsoap.org/rp">
<wsrp:action>http://richardc.wrox.com/AcceptAttachments</wsrp:action>
<wsrp:to>http://localhost/DIMESecondService/Service1.asmx</wsrp:to>
<wsrp:fwd>
<wsrp:via>http://localhost/DIMEAttachmentService/Service1.asmx</wsrp:via>
</wsrp:fwd>
<wsrp:from>mailto:techierebel@yahoo.co.uk</wsrp:from>
<wsrp:id>uuid:b813a1d3-ee5c-405f-b9df-7767462f83bd</wsrp:id>
</wsrp:path>
</soap:Header>
<soap:Body>
<AcceptAttachments xmlns="http://richardc.wrox.com" />
</soap:Body>
</soap:Envelope>

The referral document is a simple logical structure as depicted below. The <for> . <exact> tag structure uses an exact endpoint node to watch for. This is defined as the exact tag value. If the URL value doesn't conform exactly to this then it is discarded and the router returns an 'endpoint not found' exception (a failure at any point in the chain of web service redirects will return a SOAP fault to the client thus stopping the SOAP message in its tracks).

A <for> ... <prefix> tag structure can be used instead, which can trap a page value by simply using the current virtual directory prefix. This isn't always wanted though. The <go> tag represents the action that occurs if the condition is found to be true. The <if/> tag (which has to be present) is used to provide a further series of conditions if the first <for> condition has been matched. This can currently only involve invalidating the message if the TTL (described in the Timestamp section below) has expired.

Fig 3 - Message referral with a redirect

<?xml version="1.0" ?>
<r:referrals
xmlns:r="http://schemas.xmlsoap.org/ws/2001/10/referral">
<r:ref>
<r:for>
<r:exact>http://localhost/DIMESecondService/Service1.asmx</r:exact>
</r:for>
<r:if/>
<r:go>
<r:via>http://localhost/DIMEAttachmentService/Service1.asmx</r:via>
</r:go>
<r:refId>uuid:00000000-0011-1111-9000-00009999aaaa</r:refId>
</r:ref>
</r:referrals>

There is an object model for both WS-Routing and WS-Referral that can be used to define routing rules in the original message request headers. The following code allows access to the <path> tag that we've discussed in this section. We can effectively add a From value in the form of a URI. This can be a mail address, as below. We can also add Fwd or Rev via element so that our message can be routed to different endpoints and SOAP routers.

localService.RequestSoapContext.Path.From = new Uri("mailto:techierebel@yahoo.co.uk");
localService.RequestSoapContext.Path.Fwd.Add(new Via(new System.Uri("http://localhost/DIMESecondService/Service1.asmx")));
localService.RequestSoapContext.Path.Rev.Add(new Via(new System.Uri("http://localhost/WS-RoutingService/service1.asmx")));

The diagram below shows a very advanced use of SOAP routing that would demonstrate the dynamic and practical nature of the protocol. In order for an online holiday dealer to send a holiday confirmation message, it can send the message via a lookup service that could locate the user (possibly using an identifier - like a Passport Identifier - which is a unique ID for each .NET Passport user). This message could be forwarded to the Mobile SMS Gateway (it was envisaged that .NET Notifications be congruent with this architecture) and then onto the intended recipient (who was originally intended to receive the message using the WS_Routing to field).

Fig 4 - Message flow through WS-Routing

Understanding Timestamps

Every SOAP message carries with it a timestamp that enables us to set an expiry on the message. This can be used to time messages out so that responses outside of this value will be invalid. In order to set the expiry on the SOAP message, the Ttl property of the TimeStamp class is used with a number measured in milliseconds.

service.ResponseSoapContext.Timestamp.Ttl = 8000;

In addition to this value, the current time is also enforced using the same message. Delays to the web service can cause expiry of the message before it is returned, but synchronization issues between client and server are not a problem as it is the client that processes the timestamp when the response has been received, and will automatically generate a timeout as a result. By default, the expiry is set to 5 minutes after the generation of the message.

A ReceivedCollection is available which records all of the Timestamp values of all of the services visited by this request on route to the endpoint. The example XML that the collection encapsulates can be seen earlier in this case study in the WS-Routing section.

Programming WS-Security

It's always worth reiterating that the movement towards using the GXA is a movement towards the independent use of web services unbound by transport or application level protocol. So far in this case study we've looked at how we can use SOAP over DIME and also how we can route SOAP messages and generate routing rules for those SOAP messages. When combined with this last protocol, WS-Security, we have a formidable start to the first Microsoft GXA implementation.

WS-Security has already had a barrage of practical examples across the Internet and within the Microsoft samples collection. It encompasses all of the security elements that would normally entail the use of other protocols such as windows authentication protocols or SSL (the WSDK implementation of this is open, meaning that LogonUser APIs or ADSI can be used to verify credentials). Using WS-Security we can:

  • Send credentials to a web service
  • Send an X509 certificate to a web service
  • Sign a SOAP message
  • Encrypt a SOAP Message

I won't re-cover old ground on all these facets of cryptography and security and how they are used with .NET (this can be a very big complex subject - the forthcoming C# Data Security Handbook (http://www.wrox.com/books/1861008015.htm?WROXEMPTOKEN=706301ZgHcQjRStEwcXgGTvV8s) by Wrox Press provides a good introduction into the subject). Some readers will be familiar with my two articles on CSharpToday which delve into the use of cryptography with .NET (Part 1 is at http://www.csharptoday.com/content.asp?id=1640&WROXEMPTOKEN=706301ZgHcQjRStEwcXgGTvV8s, part 2 is at http://www.csharptoday.com/content.asp?id=1566&WROXEMPTOKEN=706301ZgHcQjRStEwcXgGTvV8s), however, in order to understand WS-Security, it is fairly mandatory that we are clear about what can be done with WSDK.

Authentication

In order for the user to authenticate to the web service it needs to be able to transfer credentials across the wire in a secure manner. In this first example, we'll look at the most simple system of plain text username and password credentials which a client application will propagate over the wire in order to access a web service. As with the previous examples, we'll start by looking at how we would build a web service. The WSDK has a modular approach to credentials allowing the creation of Password providers that can be used to provide a custom means of authenticating a user. We'll be looking at the creation and implementation of our proprietary Password provider in the following example.

The AuthenticationService web service will use a Password provider to scan a database, checking to see if the user has sent the correct credentials. If the user is successful, then the web method called Randomize will return an integer value which can be used as key for several other services (we can assume this value is changed every ten minutes to protect the information services).

The web method is a simple random number generator. For the purposes of the real world, if this number was akin to a SecureID number then our server would have to be in sync with another server to ensure that the numbers aren't out of step. The purpose here, however, is to demonstrate the use of WS-Security.

[WebMethod]
public int Randomizer()
{
int[] possibleNumbers = { 10, 12, 14, 16, 18, 19, 29 };
Random rnd = new Random();
return possibleNumbers[rnd.Next(7)];
}

The Password provider that was mentioned earlier is created by implementing the IPasswordProvider class in the Security namespace of the WSDK. This derived class must implement a single GetPassword method with the signature below. This method is invoked before the web method using the username from the WS-Security headers to do some sort of lookup to discover the users password. Whatever is returned from this method, be it a correct or incorrect password - we don't have to do the password check process ourselves - is handled by the WSDK HttpModule.

public class AuthService : IPasswordProvider
{
public string GetPassword(string username)
{
SqlConnection cn = new SqlConnection("Server=localhost; Database=Security; Integrated Security=SSPI");
cn.Open();

SqlCommand cmd = new SqlCommand(String.Format("SELECT Password FROM tblPlainTextUsers
WHERE Username='{0}'", username), cn);
return (string)cmd.ExecuteScalar();
}
}

In order to implement this, we have to add another section to the web.config file so that the section reader in the HttpModule will be able to use our password provider class.

<microsoft.wsdk>
<security>
<passwordProvider type="AuthenticationService.AuthService, AuthenticationService" />
</security>
</microsoft.wsdk>

Fig 5 - The Login Client Application

The client application will allow passwords to be sent in plain text or a hashed value to the web service. Hashes are algorithms that produce a fixed length byte array output for every string. The hash input value leads to a consistent output value irrespective of the number of time the operation is used on the string. Hashes are commonly used for checking the integrity of a document or for digital signatures (which we will be demonstrating later in the case study). It is very common to use one-way hashing and store password values in the database so that login requests are rehashed and compared to a database hash - in this way, the plaintext password isn't kept in physical storage. We will discuss the encryption option further on in this example.

When the button is pressed, the event handler is invoked and the code below executed. The PasswordOption enumeration can take three values:

  • SendPlainText
  • SendNone
  • SendHashed

It may seem confusing as to why there is a radio button enabling encryption, but no corresponding enum value (as above). Encryption via WS-Security is used solely to encrypt the SOAP body rather than the envelope. This is important, as the headers should be parsed without relying on WS-Security. To encrypt the headers in transit, an application layer protocol such as SSL can be used. In the case of passwords, the hash value is far more efficient than an encrypted password value.

Obviously the first two are very insecure, as no hashing and no password are applied respectively. The last option applies a SHA1 hash on the password, resulting in the password text being reduced to a 20 byte hash and encoded into Base64 to ensure non-printable characters are escaped.

PasswordOption option;
rc1000.Service1
            service = new
            rc1000.Service1();
if(rbSendPlain.Checked)
{
option =
            PasswordOption.SendPlainText;
}
if(rbSendHashed.Checked)
{
option
            =
            PasswordOption.SendHashed;
}
if(rbSendEncrypted.Checked)
{
option
            =
            PasswordOption.SendPlainText;
EncryptMessage();
}
UsernameToken
            user = new UsernameToken(txtUsername.Text, txtPassword.Text,
            option);
service.RequestSoapContext.Security.Tokens.Add(user);
service.Randomizer();

In the event that the password or username is spelled incorrectly, the web method is not actually invoked. A response is sent from the web service informing the user that the security token could not be validated (authenticated or authorized). This message refers to the fact that the password provider will always return a security token if the user has been authenticated by supplying the correct credentials. The request will return a UsernameToken class instance that can be verified by the web method. This UsernameToken will reflect the UsernameToken object sent by the client (as is shown in the code above). The class constructor takes a username, password, and one of the three aforementioned options. The SoapContext is obtained using the technique shown in the examples up to now, and the Security object is returned. In order to use the UsernameToken at the client we can add it to the Tokens collection. The UsernameToken is simply one of the possible token types we can use with the message.

Fig 6 - Resultant exception from a credentials failure

If we specifically choose the send in signed hash form using the client application, the following WS-Security markup will be generated by the WSDKClientProtocol proxy class. The XML is time-stamped using UTC (notice the inconsistent prefix with the timestamp - this belong to a different namespace - which allows us a modicum of control through a WSDK class which we'll refer to later in the case study) and another tag called Nonce is displayed. The Nonce value is a random value that, like the password digest, is encoded using Base64 and sent to the server. The Nonce contains nothing special, but in a signed message can be used to create a challenge response situation. The challenge-response order might look like this.

  • Client signs the SOAP message with a private key
  • The Web service then uses the public key to un-sign the message and verify the authenticity of the sender and extract the nonce value (in this way, the web service knows that the client sent the message)
  • Nonce is reused in further exchanges, and knowledge of nonce is tested by both client and server to verify the authenticity of each party

<wsse:Security soap:mustUnderstand="1" xmlns:wsse="http://schemas.xmlsoap.org/ws/2002/07/secext">
<wsse:UsernameToken xmlns:wsu="http://schemas.xmlsoap.org/ws/2002/07/utility"
wsu:Id="SecurityToken-d7a01f46-6cd0-4f35-9374-da0a385eb1cd">
<wsse:Username>Richard</wsse:Username>
<wsse:Password Type="wsse:PasswordDigest">54igiotNEb8w3IvBdxiZwTJ9Z5U=</wsse:Password>
<wsse:Nonce>XvyJJyIxUTfixbStYLyENA==</wsse:Nonce>
<wsu:Created>2002-10-17T22:13:59Z</wsu:Created>
</wsse:UsernameToken>
</wsse:Security>

If the password value was typed wrongly, then a SOAP fault is generated which can be caught by the client. In the previous screenshot, the exception was of type SoapHeaderException as the request contained an invalid UsernameToken that couldn't be authenticated by the WSDK Password Provider (due to a credential failure). Such an exception would be raised by the web service and encoded within the body of the SOAP response. A password can never be reverse engineered from the hash value - it's possible, however, that brute force attacks can be used to discover the password value as the same password (case-sensitive) will derive the same hash value every time (using the SHA1 algorithm). In this case, it's usually good form to write intelligent applications that recognize multiple attempts to login to the same account unsuccessfully.

<soap:Fault>
<faultcode xmlns:code="http://schemas.xmlsoap.org/ws/2002/07/secext">code:FailedAuthentication</faultcode>
<faultstring>The security token could not be authenticated or authorized</faultstring>
<faultactor>http://rc1000/AuthenticationService/service1.asmx</faultactor>
</soap:Fault>

Encryption

One of the overriding uses for the UsernameToken class is propagate an encryption key.

To give a quick recap for users new to cryptography: there are two common types of encryption algorithm used to generate keys and encrypt data in use today throughout the Internet (and within Windows authentication protocols). The first uses a symmetric key that both parties must have to be able to encrypt and decrypt a message. The second uses a pair of mathematically related keys called a public-private key pair that can be used in tandem to encrypt and decrypt a message. By convention, the public key is sent to the world and messages are encrypted by all parties that cannot be decrypted by anything other than the private key. It is therefore essential that this key is kept secret. As the use of asymmetric encryption is bi-directional, the private key can be used to perform a similar encryption routine to the public key. When this occurs, this is treated as a signature as the public key can then reverse or decrypt the message, meaning that the sender could only have been the owner of the private key - allowing the authenticity of the sender to be proved. In a subsequent section we will consider the impact of these technologies and the real world problems and solutions associated with them.

In the example below, we create an asymmetric key pair and send the public key to the web server with the UsernameToken. In order to do this, we have to create the key pair using the RSA algorithm (widely used in SSL/TLS and other key exchange technologies). When the key has been generated, it must be de-serialized to XML so that it can be sent within the SOAP message. The following namespaces must be used to support this operation:

using System.Security.Cryptography;
using System.Security.Cryptography.Xml;

To being with, we'll use symmetric encryption via the TripleDES algorithm. This will use a 168 bit key by default. This is currently the only supported algorithm we can use with the existing WSDK, but in reality it should be perfectly adequate for all our encryption purposes. When the encryption radio button is selected, we default to sending the password value by hash and creating an instance of the TripleDESCryptoServiceProvider class. Rather than calling the GenerateKey method though, we will use a named key that can be any series of 16 bytes (we would usually, however, make it somewhat more dynamic and read the value from a secure configuration medium - this may be by using a file with authorization entitlement only for a particular windows user). The EncryptionKey class encapsulates the WSDK key and the XML KeyInfo structure, which is a derivation of the XKMS protocol ratified by the W3C in order to create an XML framework for key management. The KeyInfo property on the EncryptionKey class can return the KeyInfo XML. The EncryptionKey class cannot be passed directly to the Security object, it can only be used as a Constructor argument for the EncryptedData object. This will create a block of encrypted data in the SOAP body as per the XML-Encryption W3C standard.

if(rbSendEncrypted.Checked)
{
option =
            PasswordOption.SendHashed;
des = new
            TripleDESCryptoServiceProvider();
des.Key = new byte[]{ 2, 3, 5,
            7, 11, 13, 17, 19, 23, 29, 31, 37, 41, 43, 47,
            53};
}
EncryptionKey ekDesKey = new
            SymmetricEncryptionKey(des);
service.RequestSoapContext.Security.Elements.Add(new
            EncryptedData(ekDesKey));

The resultant XML will appear like the XML below, the contents having been encrypted. From this we can discern the use of the Triple DES algorithm using a Cipher-Block-Chaining mode. The CipherValue tag indicates a Base64 encoded string that contains the encrypted SOAP body text, or in this case, the call to the AuthenticationService web method.

<xenc:EncryptedData Id="EncryptedContent-6939904d-96ae-4c68-bfe1-fbd2e4292f3f"
Type="http://www.w3.org/2001/04/xmlenc#Content" xmlns:xenc="http://www.w3.org/2001/04/xmlenc#">
<xenc:EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#tripledes-cbc" />
<KeyInfo xmlns="http://www.w3.org/2000/09/xmldsig#" />
<xenc:CipherData>
<xenc:CipherValue>eYoGjMyISHNJgcDQ/hf3TJMh1q9ylo1yYpxbpaVyvsZvzpuWKyFq84SDTfbwedW1S0Syp914zpo=</xenc:CipherValue>
</xenc:CipherData>
</xenc:EncryptedData>

In addition to this, there is also an XML encryption reference in the SOAP headers to instruct the WSDK web service filter to search for the XML encrypted body text. The EncryptedContent Id will refer to the ID in the encrypted body above.

<xenc:ReferenceList xmlns:xenc="http://www.w3.org/2001/04/xmlenc#">
<xenc:DataReference URI="#EncryptedContent-6939904d-96ae-4c68-bfe1-fbd2e4292f3f" />
</xenc:ReferenceList>

The web server needs to be able to decrypt these messages, so we can use an interface like the IPasswordProvider interface that will facilitate the web WSDK obtaining the correct decryption key. We simply write a class that implements the IDecryptionKeyProvider interface and will return the Decryption Key to the HttpModule. Enabling a custom deduction of a decryption key as we have seen for passwords means that we can derive the key from a variety of sources and determine which key to use based on a specific request. For example, we could add a custom SOAP header which can be used here to determine the specific key we need, or we can parse the Username token directly to do the same. Our class must implement the GetDecryptionKey method that will return an object of type DecryptionKey. This is a very simple non-deterministic use of an IDecryptionKeyProvider, but we can use the KeyInfo collection parameter (which reflects the KeyInfo XML structure) to return information about the key itself so that we can determine which key to get. The client can add a series of clauses to the KeyInfo property of the EncryptionKey object that will enable a unique identification of the key through a lookup operation by the web service.

public class DecryptionClass :
            IDecryptionKeyProvider
{
public DecryptionKey
            GetDecryptionKey(string alg, KeyInfo
            key)
{
TripleDESCryptoServiceProvider des = new
            TripleDESCryptoServiceProvider();
des.Key = new byte[]{ 2, 3, 5,
            7, 11, 13, 17, 19, 23, 29, 31, 37, 41, 43, 47, 53};
return
            ((DecryptionKey)(new SymmetricDecryptionKey(des)));
}
}

In order for this method to be invoked and the decryption key to be retrieved by the web service, we have to add an entry into the web.config file.

<security>
<decryptionKeyProvider type="AuthenticationService.DecryptionClass, AuthenticationService" />
</security>

As yet, the WSDK doesn't support encryption through the use of a key pair directly, but this might not be the best means of using asymmetric key encryption in any case. It does support the use of X509Certificates and adds a rich programming model as an addendum for the .NET cryptographic classes that have limited support for X509 certificates.

An X509 certificate is a collection of properties including a public key, expiration date, organizational name (domain name), and related business descriptions and functions, which is issued and signed by a trusted third party. In this way, when the client receives the certificate, they can check the signature of the certificate by using the public key of the authority that distributed the certificate to verify that the certificate is authentic. Don't worry so much about digital signatures yet, as we will focus on them in the next section, the important point here is the premise that X509 certificates can be used as a means of verifying identity and can also be used to encapsulate the corresponding public key. The consequence here is that we can download a certificate from a server as it is freely distributable, and use it to encrypt content "on the wire" as well as prove the identity of the server. Therefore we can encrypt all content to the server which only it can read since it has the corresponding private key.

Most of us are used to downloading and using X509 certificates every day whilst browsing the Internet over https. SSL/TLS uses certificate distribution from both client (optional) and server to enforce a secure session.

Certificate Generation

Quite unremarkably, .NET contains some tools that are incredibly useful for testing and development purposes for generating both CA certificates and either personal or web server certificates. When this is done, other tools can be used to add the CA certificate to the root authority store list and also to check that the certificate we have created is a valid one.

To begin using the tools that come packaged with VS.NET, we can choose the submenu item VS.NET tools from the VS.NET menu and select the "Command Prompt" option. This will load all the environment variables necessary to be able to access the programs we'll be using from the command prompt (for users that have only installed the .NET Framework, we can navigate to the directory "drive_letter:\Program Files\Microsoft Visual Studio .NET\FrameworkSDK\Bin" or add it to the PATH environment variable using SET).

The process to create a server certificate is two-fold. Firstly we have to create an issuer certificate and install it into the Trusted Root certificate store which will allow us to allot trust on the computer (i.e. if this was on an intranet we could set ourselves up as a CA and ensure that we all certificates on the intranet have to be issued by us) for any certificates issued by the CA we have created. The second stage is to create a certificate which is signed by our issuing CA (using the private key) and can thus be validated by checking the signature - by using the public key contained in the issuer's certificate which is installed into the Root Authority certificate store.

To begin we use the tool makecert :

>makecert -a sha1 -sv richard.pvk -n "CN=Richard" -d Richard -r richard.cer

This will elicit a response, either acknowledging that it was successful or giving us a detailed error description along with a helpful list of all the command line options. The above options and their effects are listed below.

-a - This allows us to specify the type of authentication algorithm we wish to use on the certificate. When the certificate is created, a digest or MAC is created using the certificate as input data and this value is signed using the issuer's private key. The digest value, when retrieved using the public key from the issuer's certificate (which will exist in the certificate store), will have to match the calculated certificate digest value.

-sv - This option creates a private key file called " richard.pvk " in the default directory. This will be used to sign all issued certificates.

-n - This represents the Common Name which is a hierarchical notation used by LDAP.

-d - This represents the friendly name, which in this case I've specified as the same as the common name, but which is usually distinct.

-r - This specifies that we want the certificate to be self-signed. As there is a hierarchical specification to this trust, there can be many authorities above this or below this. Using this option will establish that there is nothing above this that needs to sign - in effect it is a CA.

The last option is the name of the certificate file that has a unique .cer extension.

There are some things in the list that are not described, but occur by default. A serial number id allotted to the certificate (this can be specified, but to ensure uniqueness, it's best to let makecert determine this value automatically). The start date and expiry date of the certificate can be specified, but by not specifying them they default to the certificates creation date and the beginning of 2040 respectively.

Editor's Note : The next version of the WSDK, called WSE (Web Services Enhancements) will change to support empty key usage bits, allowing both encryption and digital signing. So, certificates generated using makecert.exe and will work correctly with the examples in the case study.

Now that we have created the CA issuing certificate richard.cer with the associated private key file, we have to install it in the Root Authority certificate store. To do this we use the certmgr tool. Simply issue the following command on the command line to start the utility.

>certmgr

A window form will be shown with all the certificates located in various certificate stores.

Fig 7 - Certmgr application showing certificate store.

To install the CA certificate I simply moved to the "Trusted Root Certification Authorities" tab and clicked the Import. button. This starts the certificate import wizard.

The certificate installation wizard can be started from the certmgr utility above or by actually double clicking on the .cer file and pressing the Install Certificate . button. This will start the same wizard as described above. The certificate, when not installed to the root (and as it is self-signed), will be represented as non-trusted and invalidated as below:

Fig 8 - The server certificate

Once this is installed, we can generate the certificate that we will use throughout the example code, and similarly a certificate we can use for client validation of credentials in the SSL client. Many of you may have noticed that we seem to have done this in a very round-about way since we could have just self-signed the server and client certificates and not developed an Issuing Authority. This is just down to emulating how we would do things if we used a certificate authority such as Verisign or Thawte. I guess I'm just a purist in this respect.

>makecert -$ commercial -d "Example website" -n "CN=www.example.org" -iv richard.pvk -ic richard.cer example.org.cer

This differs slightly from the generating options of the CA certificate. The -iv and -ic options specify the private key and certificate file respectively. The -$ option can take commercial or personal values which specify what the certificate is for.

The above example use of makecert shows how to create a certificate hierarchy, but we can't actually do anything with them as the certificates we've generated have no key-pair information and no Key Usage information. If we use the -sv switch to generate a private key for the example.org certificate and then install this certificate to the personal folder viewable through the certmgr application (or you can create an MMC file to view all the different key stores on the machine with the Certificates plugin).

The last thing that needs to be done is to download the pvkimprt.exe console application from the MSDN website and run it. This will generate a .pfx file that can be used to import the example.org certificate into the certificate store to be used for the next example. This certificate will be able to encrypt and sign and have been signed by the Richard certificate (trusted root we created earlier).

We have just created a certificate that can be used to sign SOAP messages. Unfortunately, the tools available can sometimes be insufficient to generate a certificate that we can use for all encryption and signing purposes and we have to use combinations of both tools and SSPI (Windows security API). The alternative, however, is to generate a certificate chain from the Microsoft test site at http://sectestca1.rte.microsoft.com/?WROXEMPTOKEN=706301ZgHcQjRStEwcXgGTvV8s which enables the creation of certificates we can use in our examples. Initially we have to download and install the certificate SECTESTCA1 to the root through CertMgr as demonstrated above. When this is done, we must generate a certificate request via IIS that is done by gaining a context menu and choosing properties. Under the directory security tab there is a Server Certificate button, which when pressed, will generate a request. This request can either be answered by Microsoft Certificate Services, or for our purposes (if we don't have CS installed), we can cut and paste the resultant file text and generate and download the certificate or certificate chain that includes the SECTESTCA1 certificate. All instructions for use are provided on the website. Using and programming Certificate Services is beyond the scope of this case study but CS comes free with Windows 2000 server platforms and provides the best implementation for easy certificate and enforcement of corporate PKI (n.b As of the final edit of this case study, the SECTESTCA1 web site had been taken down with no plans to renew the free certificates which have now expired).

Another useful tool for generating test certificates is OpenSSL. OpenSSL is an open source library and a command line application that can be used to generate certificates and convert certificates between encoded formats. The source is freely available and it needs both a C compiler (VC++6 will do) and the perl runtime to function correctly. More information on this can be found at http://www.openssl.org/?WROXEMPTOKEN=706301ZgHcQjRStEwcXgGTvV8s.

Of course we can always use a real Certificate Authority such as Verisign or Thawte and use a specific certificate for testing purposes. Using the techniques above, certificates can be both installed and packaged with their respective private keys.

Using X509 Certificates for encryption

Once the certificates have been installed, they can be used for cryptographic operations. The X509CertificateStore class can be used to inspect all certificate stores of both the current user and the local machine. A series of constants exist which can be used to obtain certificate stores.

The CurrentUserStore or LocalMachineStore methods will use these string constants as arguments for getting a handle to the appropriate store. In order to enumerate the certificate collection we simply call the OpenRead() method. The store contains a Certificates collection that we can use to enumerate the certificates in the store. In order to pick a single certificate we simply use a check on the name of the certificate (alternatively we could check something more unique such as the certificate hash code!). The last line of code will use the certificate to encrypt the content. As the EncryptedData class constructor takes an argument of type ISecurityElement, we have to create an X509SecurityToken. In order for the certificate to be valid all of the listed properties must be supported.

X509SecurityToken x509token = null;
X509CertificateStore x509store = X509CertificateStore.LocalMachineStore(X509CertificateStore.MyStore);
x509store.OpenRead();
foreach(X509Certificate cert in x509store.Certificates)
{
if(cert.GetName().ToLower().IndexOf("cn=rc1000")>-1)
{
x509token = new X509SecurityToken(cert);
Debug.WriteLine(String.Format("Has private key: {0}", cert.PrivateKeyAvailable));
Debug.WriteLine(String.Format("Can Encrypt: {0}", cert.SupportsDataEncryption));
Debug.WriteLine(String.Format("Can sign: {0}", cert.SupportsDigitalSignature));
Debug.WriteLine(String.Format("Can encipher: {0}", cert.SupportsKeyEncipherment));
Debug.WriteLine(String.Format("Is valid: {0}", cert.IsCurrent));
break;
}
else
{
Debug.WriteLine(cert.GetName());
}
}
service.RequestSoapContext.Security.Elements.Add(new EncryptedData(x509token));

This is all that is necessary to add to the client so that it can encrypt the contents with an X509 certificate. The following XML is generated as a result:

<xenc:EncryptedKey Type="http://www.w3.org/2001/04/xmlenc#EncryptedKey" xmlns:xenc="http://www.w3.org/2001/04/xmlenc#">
<xenc:EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#rsa-1_5" />
<KeyInfo xmlns="http://www.w3.org/2000/09/xmldsig#">
<wsse:SecurityTokenReference>
<wsse:KeyIdentifier ValueType="wsse:X509v3">EO74+nJi7Gp46E4jDRRktN3nsHs=</wsse:KeyIdentifier>
</wsse:SecurityTokenReference>
</KeyInfo>
<xenc:CipherData>
<xenc:CipherValue>ro6FjMvBaGQwR8dCHzMbhhKG7UXr6YdAdMWVH6GruBgFdtIVBygIE+yWQhAhSMH0WZzSw2HNaW32
+gIeD1rofndBFkdz3hXtgl2vxHhCzNRNh9zWHsDjpovQrrIiRS+
Hs6wzOIXl1J9XejCv7sjyXvJn2f/UHdu6OPRaSuKJw58=</xenc:CipherValue>
</xenc:CipherData>
<xenc:ReferenceList>
<xenc:DataReference URI="#EncryptedContent-260c7335-6131-4d49-9b8b-ac2fdb13e00a" />
</xenc:ReferenceList>
</xenc:EncryptedKey>

In order for the web service to decrypt the content, the following must be added to the <Microsoft.wsdk> section (if this is missing a SOAP fault will be returned to the client). No additional code is necessary. We should also ensure that the ASP.NET worker process has ample permission to be able to access the local machine certificate store (for this it needs to have administrative permissions which can be gained by changing the ProcessModel tag to run under SYSTEM as opposed to MACHINE in the machine.config file).

<security>
<x509 storeLocation="LocalMachine" verifyTrust="false" allowTestRoot="false" />
</security>

Signing SOAP Messages with an X509 Certificate

Signing SOAP Messages is quite simple too. What do we mean by a digital signature? Well, a certificate is an encapsulation of a public-private key pair giving meaning to the uses of the keys. As we are the owner of the X509 Certificate and have the private key we can use the private key as opposed to the public key to "sign" the contents of the message giving us a resultant signature. Signature algorithms, however, are not used to sign the message contents explicitly, but are used to sign a hash value (which is much smaller than the message) proving the integrity (through the hash) and the identity of the sender through the private key signature. In the code below, a signature value is used by first obtaining the certificate from a call to FindCertificateByHash (and using the byte array copied from the certificate thumbprint) and then adding the signature to the message as we would add any cryptographic security token.

byte[] byHashVal = new byte[]{0x42, 0x63, 0x0e, 0xa0 , 0x15 , 0xc4 , 0x29 , 0xcf , 0xf5 , 0x0e , 0xf7 ,
0xd7 , 0x6c , 0x9c , 0x87 , 0x2c , 0x1a , 0x2f , 0xe7 , 0x7d};
X509CertificateStore x509store = X509CertificateStore.CurrentUserStore(X509CertificateStore.MyStore);
x509store.OpenRead();

X509Certificate x509cert =
(Microsoft.WSDK.Security.Cryptography.X509Certificates.X509Certificate)x509store.FindCertificateByHash(byHashVal)[0];
X509SecurityToken x509token = new X509SecurityToken(x509cert);
service.RequestSoapContext.Security.Elements.Add(new Microsoft.WSDK.Security.Signature(x509token));

This code results in a large block of XML, involving a number of hash values that are canonicalized so that the server can reconstruct the digest. As in the previous example, the signature verification is implicit (as long as the web.config entry exists as in the previous example). If the signature cannot be verified then an exception is thrown.

<Signature xmlns="http://www.w3.org/2000/09/xmldsig#">
<SignedInfo>
<CanonicalizationMethod Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#" />
<SignatureMethod Algorithm="http://www.w3.org/2000/09/xmldsig#rsa-sha1" />
<Reference URI="#Id-7eaf4ce6-31fc-4d22-9fa8-a4346727d5de">
<Transforms>
<Transform Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#" />
</Transforms>
<DigestMethod Algorithm="http://www.w3.org/2000/09/xmldsig#sha1" />
<DigestValue>G0iDmAGM2KYP8SGjnjp2u3E6lQ4=</DigestValue>
</Reference>
</SignedInfo>
<SignatureValue>h7ElBcpYXC5WlWP+G3No15R8qlLLo6DobgLkzhSqUxiJO/+55u5XNX9Mv989x8KztjxlWSHb6+
4jHxngOl1HVo12EdLCcyxyiZQ/9TEZB+8zk2IC8JRve2/sFhBnRgcLKMDkaU0kb5WKa4lpkBAJNRzqny4t+
eXwTJxFwLrnzcI=</SignatureValue>
<KeyInfo>
<wsse:SecurityTokenReference>
<wsse:Reference URI="#SecurityToken-96115805-3ae3-4738-bca1-512dffb5c60e" />
</wsse:SecurityTokenReference>
</KeyInfo>
</Signature>

Programming a Filter Based Architecture

The nature of the filter-based architecture used in the WSDK should come as no surprise to many experienced web developers. IIS is built on the use of filters enabling various requested content to be parsed and modified - in fact, ISAPI filters have given us web developers the key to all our needs, enabling the development of applications which can be used to filter all types of content and maintain complete control over web requests and responses. The only drawback here is that they are incredibly difficult to build, debug, and maintain, and bad code, including memory leaks and miscalculated logic, can easily break an application.

Whenever we request a .NET file type, be it .aspx or .asmx, we use a special ISAPI filter to handle the request (aspnet_isapi.dll). This will manage the use of the ASP.NET Runtime and act as a container for the CLR. In fact, ASP.NET extends this model to allow pluggable filters of its own called HttpModules that can be used to map any requests to a class and modify the request, or HttpHandlers that can be used to extend the model and map a specific request to a file type. For example, we could quite easily develop a new file type and map this file type within IIS to an HttpHandler. IIS, through the aspnet_isapi.dll will delegate the request to the appropriate HttpHandler that is specified in one of the numerous configuration files (for local use it will normally be web.config, for global use it will normally be machine.config). It is in fact this very approach that Microsoft has used to facilitate calls to web services. A web service uses a known file type (.asmx) so whenever a web service is requested an HttpHandler processes the request and acts as a stub layer between the SOAP request and the web service.

The core web service architecture is built through the .NET Framework using an extensibility model. In the earlier section we touched on the DimeExtension class. In the original DIME sample, this class was an attribute that could be applied to a web service endpoint. This could then be used to parse a SOAP/DIME message.

This application architecture is not new, and for those of us that are somewhat adept at using web services by now it should be apparent that this is a replication of a SoapExtension class. The SoapExtension class is used to parse a SOAP message and apply specific operations to the messages at different stages of the parsing. The core method that can be overridden in the SoapExtension is ProcessMessage (the same name given to the overridable method in an HttpHandler). The argument SoapMessage is an enumeration value that returns the point in the SOAP parsing process. Possible enumeration values are.

  • BeforeSerialize
  • AfterSerialize
  • BeforeDeserialize
  • AfterDeserialize

I won't harp on about the amazing applications we can write and the low level of control that we have using SoapExtension in conjunction with attribution, since this topic has been covered widely especially in web related articles on this site. My references to SoapExtension are really just an indication of the filters and message parsing which is available at every state of the SOAP message (i.e. before and after being parsed by the ASP.NET runtime). It should come as no surprise then that this behavior is replicated in the WSDK. Using the WSDK we can define filters that can be used to pre-parse either the request or response. There are three immediately obvious and necessary uses for this:

  • Adding or removing SOAP header information from the message can be done through this mechanism so that we can remove or change security before it arrives at the WSDK parser, or change routing information.
  • Developing an additional protocol unsupported by the WSDK. Indeed we can link our implementation of another GXA protocol such as WS-Transaction using an input output filter (in order to do this correctly though we would have to make modifications on the client by adding SOAP headers - which the WSDK wouldn't understand).
  • Using the technique of input-output filters we can also write the request-response out to any physical storage - a database, file, etc. The benefit of this is that we can selectively choose which portion of the message we wish to record.

In the following example we'll discuss how to filter content using a WSDK filter, which will write the Timestamp information to a file and leave everything else intact. In order to create an input filter file logger we must create a class that derives from the SoapInputFilter abstract base class. Two overridable methods can be used to help filter content. The first, CanProcessHeader, can be used to determine whether the filter can read a particular XML node. This is very opportune for production purposes where we need to avoid the overhead of parsing the whole message in this filter. The ProcessMessage method takes a SoapEnvelope parameter which can be used to access the entire SOAP packet, both header and body. The example below simply logs the timestamp Created and Expires tag values to a log file. This method will filter the SOAP request.

public class SecretInputFilter : SoapInputFilter
{
public override void ProcessMessage(SoapEnvelope inEnvelope)
{
FileStream fStream = File.Open("C:\\timestamp.txt", FileMode.Append, FileAccess.Write, FileShare.ReadWrite);
NameTable xTable = new NameTable();
xTable.Add("wsu");
XmlNamespaceManager xnms = new XmlNamespaceManager(xTable);
XmlNode TimeStampNode = inEnvelope.SelectSingleNode("//Timestamp", xnms);
string created = TimeStampNode.SelectSingleNode("Created", xnms).InnerText;
string expires = TimeStampNode.SelectSingleNode("Expires", xnms).InnerText;
StreamWriter swStream = new StreamWriter(fStream);
swStream.WriteLine("Message created: {0}, expires: {1}", created, expires);
swStream.Flush();
swStream.Close();
}

protected override bool CanProcessHeader(XmlElement ele, SoapContext context)
{
return true;
}
}

The corresponding response will be filtered by the SecretOutputFilter class that derives from the SoapOutputFilter abstract class. This class will only support the ProcessMessage method. In order to use the filters, however, we must add a small section to the web.config file. We must ensure that the values used in the web.config file are correct otherwise the service will simply return a Service Unavailable SOAP fault message to the client.

<microsoft.wsdk>
<filters>
<output>
_ <add name="SecretOutputFilter"
_ type="Wrox.CsharpToday.RichardC.SecretOutputFilter,DIMEAttachmentService.dll" />
</output>
<input>
_ <add name="SecretInputFilter"
_ type="Wrox.CsharpToday.RichardC.SecretInputFilter,DIMEAttachmentService.dll" />
</input>
</filters>
</microsoft.wsdk>

Future of the GXA

This case study has been about a first step towards an architecture that could take distributed computing into the realm of the Internet - much like the proposed idea of the applet-CORBA "object web" model proposed with the implementation of java applet computing. However, this is much more feasible and firewall friendly and, unlike the development of CORBA, would have to be absolute and not proprietary. .NET has been touted as the best development resource for developing Enterprise web services and even the Microsoft skeptics have had to redefine their views in this respect. The WSDK is the first step to the reality of the web services model. At the PDC 2001, where there was a phenomenal amount of hype around .NET My Services, it was possible to see where the Microsoft vision was headed. The fact that .NET My Services was put on hold was probably 50% to do with the fact that the infrastructure doesn't exist on the web for this type of enterprise framework.

The WSDK is the first of its kind in the world of GXA programming, as it consolidates many of the GXA protocols under one banner, SDK. Many .NET My Services aficionados should be comfortable with the workings of the WSDK and see it as a bridge to the eventual release of the .NET My Services packaged product.

In the overall solution we will soon see these protocols in the realm of daily development. As we move towards an interconnected world, and the barriers towards businesses come down in an attempt to create secure internet communities which span multiple corporate boundaries products (such as Federated Microsoft Passport), they will begin to be used in conjunction with Kerberos AP tickets, X509 certificates and many of the GXA protocols to create a new frontier for business based on information sharing.

Indeed the specification for WS-Security will include Kerberos AP tickets to allow a more federated model. (Kerberos is beyond the scope of this piece, but to summarize, it is a distributed authentication protocol whereby a server will nominate a ticket - called a TGT - that can be used to access services within a distributed network. It's actually much more complicated than this, so to fill in the blanks you can use some of the links in the section below).

It is highly probable that in future releases of the WSDK we will see more of the same. Among the contenders are WS-Transaction, which as the name says will be a GXA protocol which supports distributed transactions (in exactly the same way as the XA transaction protocol is used across an intranet), WS-Coordination will be the glue that holds or 'coordinates' the transactions themselves across a distributed system. WS-Inspection is a protocol based on named files that are expected within directories and web roots that will allow web services within a site to be discovered dynamically. As you can see, there are many protocols which are on the whiteboard and waiting for a standard body to ratify, so much so that the outcome of the standards agreement could one day be the internet equivalent of CORBA (hopefully IIOP - Internet Inter Orb Protocol - rather than proprietary ORBs!).

Microsoft have developed a long-term strategy in the .NET Framework which not only includes programming languages and environments but also servers. For example, the next generation of SQL Server and BizTalk will be fully CLR compliant allowing the writing of stored procedures and components in a CLR-compliant language such as C#! The net result of this, and adoption and integration of the web services model into both these products as well as shared web services such as .NET Notification services, signifies that the GXA will also be a large part of the server market offering.

The danger from this point is that toolkits such as the WSDK take a backward step along with equivalent toolkits from the Open Source world and IBM or Sun. Most of us remember the browser wars from the earliest version of Netscape and IE and most of us remember the programming nightmare it caused by having to write two versions for every client. Similarly, the only way that the GXA will work correctly is if all parties writing software rigidly stick to standards defined by standards bodies so that we should expect exactly the same results from code calls if web services are written in C# as if they are written in Java.


Developing A Rich Client Forum Reader Using Windows Form and Web Service

Developing A Rich Client Forum Reader Using Windows Form and Web Service
http://www.csharptoday.com/content.asp?id=1902
Article Rating: 4
Published on September 23, 2002
Abstract
In this case study, Steve Shi demonstrates how to develop a set of XML Web Services to expose the contents and functionalities of a traditional discussion forum website, as well as a rich-client application that would consume these web services. By interacting with the supporting Web Services, the rich-client application provides such basic functionalities as browsing forum contents, posting new topics, and replying to existing topics - hence the name rich-client forum reader. Furthermore, the most notable feature of the rich-client forum reader is the ability to cache all forum contents retrieved from the supporting Web Services in an internal local database, thereby providing an offline forum browsing capability. The forum reader is implemented as a full-fledged Windows Forms application under the .NET platform, with its underlying component architecture strictly based on classic object oriented analysis and design principles. It sports an intuitive GUI that loosely resembles that of Microsoft Outlook, with the underling data access components fulfilling their tasks of retrieving and replying to online forum messages by interacting with the supporting web services on the forum server.
Case Study

In this case study, Steve Shi demonstrates how to develop a set of XML Web Services to expose the contents and functionalities of a traditional discussion forum website, as well as a rich-client application that would consume these web services. By interacting with the supporting Web Services, the rich-client application provides such basic functionalities as browsing forum contents, posting new topics, and replying to existing topics - hence the name rich-client forum reader. Furthermore, the most notable feature of the rich-client forum reader is the ability to cache all forum contents retrieved from the supporting Web Services in an internal local database, thereby providing an offline forum browsing capability. The forum reader is implemented as a full-fledged Windows Forms application under the .NET platform, with its underlying component architecture strictly based on classic object oriented analysis and design principles. It sports an intuitive GUI that loosely resembles that of Microsoft Outlook, with the underling data access components fulfilling their tasks of retrieving and replying to online forum messages by interacting with the supporting web services on the forum server.

Introduction

A Brief Look at the Forum Reader Application

What is a forum reader application? What do we use it for? We'll briefly answer these questions now, and a more in-depth discussion will be presented in the sections that follow.

The so-called rich-client forum reader that we've developed is a client-side windows application that provides the same functionalities as a discussion forum website, together with some enhancements. Practically, it is developed as an extension to a traditional forum website rather than a replacement of it. Typically, companies running forum websites would be interested in developing such software and distributing it among their forum users. As a forum user, other than using a web browser to visit the forum website, you can alternatively use the forum reader to access and interact with the same online forum contents. For example, you can use the forum reader to browse online forum contents, post new topics, and reply to existing topics. Besides these functionalities, the forum reader can automatically save all the online forum contents that you have viewed in its internal local database, which makes for easy offline reference in the future. This is a unique feature of the forum reader, as by contrast, it is unavailable in the browser-based client.

However, in order to support the forum reader to perform its functions, a set of Web Services must be developed and installed on the forum web server in the first place. The purpose of these Web Services is to provide programmatic access to the backend database of the forum website. Or, to put it in another way, these Web Services serve to expose the online forum contents and functionalities that are intended to be accessed by the rich-client forum reader. Therefore, by interacting with these supporting Web Services, the forum reader could perform such basic tasks as retrieving forum topics from, or posting new topics to, the online forum database. As I mentioned earlier, the retrieved forum contents are not only displayed in the user interface but also saved in an internal local database of the forum reader. We will explain this feature in detail a little later.

What do we mean by "rich-client"?

As you have seen, when we talk about the forum reader application, we often refer to it as "rich-client" forum reader. Why do we qualify the forum reader application with the term "rich-client"? Literally, the term "rich-client" has a twofold meaning, which is "both rich and client-side". However, this is only a vague interpretation. Precisely, in .NET terminology, the term "rich-client" is typically used to refer to windows application that performs the role as the client-side entity in a distributed client/server architecture. This term is specifically coined to distinguish windows-based client-side application from browser-based client (i.e., the web browser). The word "rich" in this term both implies and stresses the point that windows applications can take advantage of a rich set of resources and features that are provided by the underlying operating system and the .NET platform. However, by contrast, these resources and features are not accessible from a browser-based client.

The Perspective, Scope, and Merits of this Case Study

With the advent of the .NET platform and the shipment of Visual Studio.NET, the development of traditional Windows applications that have built-in support for consuming Web Services has been dramatically simplified. With this case study, you will have the chance to walk through the design and development of a complete application implemented with Windows Forms and Web Service technology. Moreover, in contrast to other kinds of articles that drill down into one specific .NET programming technique or a particular piece of .NET technology, this case study aims to depict the big picture of the component architecture of a sample .NET application and examine the key design considerations as well as the implementation details related with each component. By and large, the focus of this case study is more on the architectural model than on specific coding details and techniques.

What's the merit of this case study for the readers?

In a sense, the forum reader application that we'll examine in this case study is a full-fledged one in terms of both design consideration and implementation. It has incorporated a bit of almost everything as far as a moderately complex Windows Forms application is concerned. For example, at the design stage, all four major components that make up the entire object model of a typical windows application have been addressed. These include the user interface component, the data access component, the business logic component and the external system interaction component. On the other hand, at the implementation stage, it employs quite a handful of .NET-centric technologies and techniques, such as Windows Forms GUI widgets, object oriented programming in C#, multithreaded components, ASP.NET Web Services, data access with ADO.NET and stored procedures, etc.

So by reading through this case study, you will have a bird's eye view of the various architectural components of an entire Windows Forms application as well as a few relevant close-ups of some coding practices.

System Requirements

You should have a set of necessary software in place in order to experiment with the source code in the download package accompanying this article. These include:

  • Microsoft .NET platform 1.0 (with service pack 1 or above)
  • Visual Studio.NET (Visual C#)
  • SQL Server 2000
  • IIS 5 or above

As regards the level of expertise assumed in this case study, you should have:

  • A solid handle on the C# programming language
  • Familiarity with database design and implementation using SQL Server and stored procedures
  • Familiarity with ADO.NET
  • A working knowledge of windows programming using .NET Windows Forms
  • Basic knowledge of Web Service design with ASP.NET
  • A basic understand of Unified Modeling Language (UML) notation

Case Study Structure

In this case study, we'll walk through the design and development of the forum reader application and examine the inner workings of its major components. Here in the list below, we briefly summarize each section that follows so that you could easily get your bearings during the walkthrough.

Defining The Problem

This section talks about the specific business problem that the forum reader application intends to solve.

Defining The Solution

  • Solution Overview
  • The Forum Reader Application In Context
  • Defining the Functionalities of the Forum Reader
  • The Local Caching Mechanism of the Forum Reader

Database Design and Structure

All the tables, fields, constraints and stored procedures are covered in this section:

  • Online Forum Database
  • Local Forum Database

Web Service Design

This section discusses the development of the supporting Web Services:

  • The Data Access Layer
  • The Façade Layer

Componentization of the Forum Reader

This section describes the component architecture of the forum reader application:

  • Physical Component Packages
  • Functional Sequence

Architectural Overview of the Overall Class Model

This section takes a detailed look at the entire class model of the forum reader application.

User Interface Design

  • Laying out the GUI widgets
  • Other UI-related classes
  • Examining GUI behaviors
  • Marshalling Cross-Thread Function Call

Remote Data Access Component

This section briefly talked about the Web Service proxies used in the application.

Local Data Access Component

This section explains the component that accesses local forum database.

The Business Logic Component

This section describes the details of each of the business logic related classes.

  • CategoryManager Class
  • TopicsManager Class
  • RepliesManager Class
  • AccountManager Class

A Symphony of Objects Interaction

This section demonstrates how the various objects collaborate with each other to deliver a particular functionality as a whole.

Application Setup

This section gives step-by-step installation instructions for the download source files.

Defining The Problem

Our problem is concerned with online discussion forum websites, especially those with considerable amount of forum contents. Let's set the scene:

The chances are, as a programmer, you often go to various technology-centric forum websites to seek answers to your technical questions or to look for solutions to the problems that plague you. Whenever you find a particular topic of your interest, an intuitive approach you take to reserve it for future reference is simply saving the web page to your local hard disk. As the volume of topic and reply swells up on the forum website, your local directory tends to become bloated if you're a frequent forum visitor. Moreover, your locally stored topic pages easily become out of sync with the original online topics if there're new replies posted, and your further attempts to re-sync your locally stored forum web pages with their online counterparts will result in more files being saved to your local disk. The upshot of it all is you end up having a whole slew of unordered and discrete files in your local directory. It wouldn't be a pleasant experience if you had to sift through these files for useful information in the future. In this circumstance, you start to ask whether it is possible to have some sort of software utility that could facilitate the process of retrieving and viewing online forum contents as well as saving them locally for future reference.

So this story depicts and also defines our problem. In the following sections, we'll introduce the solution that we've devised.

Defining The Solution

Solution Overview

Figure 1: The solution includes two major functional components

Broadly speaking, the solution that we devised includes two major functional components: a rich-client forum reader application and the supporting forum Web Services. So we naturally took a two-step approach to implement this solution.

First, we designed a set of Web Services that expose the forum contents by directly interacting with the online forum database. At one end, these Web Services invoke the same set of stored procedures that are also used by the forum website to store and retrieve forums contents. At the other end, these Web Services expose the forum contents as well as the functions to manipulate them through public interfaces (i.e. Web Methods) that could potentially be consumed by any Web Service enabled client application. In effect, these Web Services serve to bypass both the presentation layer and the business logic layer of the traditional forum website, and expose its data access layer through a public programming interface that wraps a direct call to the database. Specifically, these Web Services perform three primary tasks:

  • To return the requested forum contents such as the categories, forums, topics or replies to the client.
  • To accept the information posted from the client, such as new topics, replies, or account registration requests, and then add this data to the relevant forum tables.
  • To authenticate the user through a logon process and maintain the user's session state in the server using native ASP.NET session management facilities.

Secondly, we built a traditional windows application called the rich-client forum reader. By interacting with the Web Services implemented at the forum server, the forum reader application encapsulates a number of essential forum interaction functionalities that correspond to those of its forum website counterpart, together with additional add-on enhancements. Specifically, the basic functionalities of the forum reader include navigating different forums, browsing through the topics of a given forum, viewing replies, posting new topics, replying to existing topics, registering membership, login/logout, etc. The main task of this development was to extract the business logic layer and presentation layer of the forum website and rebuilt them into a self-contained rich-client forum reader application. The forum reader application is in effect a client-side replica of the forum website but in a different incarnation. In addition, the forum reader has implemented a local caching mechanism to save retrieved forum contents in an internal local database, which takes advantage of the local processing power and local data storage capabilities that are unavailable with the web browser. This particular feature represents the most notable enhancement to the forum reader that makes it more functional than the forum website. A more detailed description of this feature will be given a little later.

The Forum Reader Application In Context

When this solution is brought to life in a real world production environment, the forum website owner will want to distribute his own custom-built forum reader application among the forum users. It goes without saying that if there are a thousand different users who have actually installed and used the forum reader application in their own computers, then there will potentially be a thousand distinct running instances of the forum reader application that all access the same online forum database. Below is a diagram illustrating the functional role of the forum reader application in relation to other components of the overall solution, including the forum Web Services, the original discussion forum website, and the database server. With this diagram, you should have a good picture of the forum reader application in the context of the real world scenario. Additionally, the diagram clearly illustrates the fact that whether a user views the forum contents using a web browser or using the forum reader, he ultimately accesses the same online forum database.

Figure 2: The forum reader application in production environment

Defining the Functionalities of the Forum Reader

Practically, we could extract as many functionalities as possible from the forum website and inject them into the design of our forum reader application. However, we've only covered a subset of them in the sample application that is essential for forum interactions. Specifically, the forum website functionalities that we choose to reconstruct in the forum reader application are listed below:

  • Allowing the users to retrieve a list of available categories and forums from the remote forum Web Services, and viewing the result in a tree view hierarchy for forum navigation.
  • Allowing the users to retrieve a selected page of topics in a particular forum from the forum Web Service, and then viewing the result in a list view.
  • Allowing the users to retrieve the replies to a particular topic, and then view the result in a rich textbox .
  • Allowing the users to register a new account, login, or logout from the forum with his account.
  • Allowing the users to post new topics or send replies to an existing topic against the forum Web Service. However, users must be logged on to the forum before they can do so.
  • Saving all the forum contents retrieved from the forum Web Services in a local database and allowing the users to browse through these saved forum contents.

The functionalities listed above serve as an outline of the project specification for the development of the forum reader application. As you have seen, all the functionalities list above, except for the last one, merely echo the online forum website counterpart. However, considering the last one in the above list, we have implemented a local caching mechanism in the forum reader application that is a unique feature compared with the forum website. We won't go into details about this local caching mechanism now, because a more detailed description will be given in the next section.

Below is a USE CASE diagram illustrating all the functionalities mentioned above, which we have implemented in the forum reader application.

Figure 3: Forum Reader Application Functionalities

The Local Caching Mechanism of the Forum Reader

As we've mentioned in the preceding sections, the capability to save online forum contents locally is the most notable feature of the forum reader. Now, let us take an in-depth look at the so-called local caching mechanism.

What is the local caching mechanism?

You can use the forum reader to view online forum contents. Here, by "forum contents", we mean the forum topics and replies as well as the list of available categories and forums. Obviously, in order to display the specific forum contents that you request to view, the forum reader must first retrieve the requested forum contents from the remote forum Web Services. After the requested forum contents have been retrieved, they are not only displayed in the user interface, but also saved in an internal local database of the forum reader. As you browse through the forum contents, a series of requests to retrieve the relevant forum contents are sent to the forum Web Services one by one. Consequently, the forum contents corresponding to each request are then retrieved, and displayed in the user interface. More importantly, during this process, the retrieved forum contents corresponding to each request are also saved in an internal local database of the forum reader, one piece at a time. More precisely, the saving of each piece of retrieved forum contents happens immediately before they are displayed in the user interface. This is what the local caching mechanism of the forum reader all about.

With each installation of the forum reader, a local database is created and maintained in the local directory where the forum reader is installed. This local database is only for the internal use of the forum reader and it is implemented using a MS Access database. The local caching mechanism is implemented in such a way that whenever the forum reader is used by a given user to view online forums, it automatically save whatever forum contents that the user has viewed during each session into this internal local forum database. In other words, the forum reader automatically maintains a local forum repository by accumulating all the forum contents that the user has viewed in a private local database.

Note that the term "local forum database" is used frequently throughout this case study and it all refers to the client-side forum database maintained by the forum reader application. Additionally, one point you should be clear about is that each particular installation of the forum reader maintains one and only one local forum database. This implies that if the forum reader has been installed on one thousand different client computers, then there will be precisely one and only one local forum database existing in each of these one thousand client computers.

What exactly has been saved in the local database?

Let us use an example to answer this question. Suppose you are the user of a particular forum website that has implemented the aforementioned Web Services and provides a custom-built forum reader for public download. You download the forum reader from this website and install it on your computer. Then, instead of using your web browser to visit the forum website, you start to view the online forum contents using your newly installed forum reader. The fact is, from then on, whatever online forum contents (i.e. topics, replies, etc.) you have ever viewed through this forum reader will all be saved in the internal local database of the forum reader. On the other hand, the online forum contents that you haven't bothered to view through your forum reader will not make their way into your local forum database. In effect, the so-called local forum database maintained by the forum reader is a partial copy of the online forum database and specifically, it only contains the specific forum contents that you have previously viewed through this forum reader. One point to note is that the local forum database does NOT contain a complete duplicate of the entire online forum contents unless in an extreme case you have manually traversed and viewed all the contents of the online forums using the forum reader. The local forum database serves the purpose of storing only those forum contents (i.e., topics, replies, etc.) that you have actually viewed through the forum reader. Put in another way, the functional role of the local forum database is to aggregate all the online forum contents that you have actually viewed using the forum reader application.

What exactly has NOT been saved in the local database?

This is an odd question. However, knowing a definite answer to this question helps you to fully understand the local caching mechanism. Practically, it's very likely that they are some, if not many online forum contents that you haven't retrieved and viewed through the forum reader. The online forum contents that you haven't retrieved and viewed will NOT be saved in the local database of your forum reader. Put in another way, you won't find in the local database any online forum contents that you haven't previously retrieved and viewed.

What will happen if some newly retrieved forum contents already exist in the local database?

Suppose, at an earlier time, you retrieved the replies to a certain topic from the forum Web Service and as a result, these replies were saved locally. Now, if you retrieve the replies to the same topic again in order to check if there are new replies posted, will the currently retrieved replies overwrite the previously saved ones? Naturally, if the currently retrieved forum contents are more up-to-date, then they will overwrite existing ones. Otherwise, the forum reader will not bother to save the currently retrieved forum contents because they are identical to the existing data. This is the logic used in the local caching mechanism to determine whether newly retrieved forum contents should overwrite existing ones in the local database.

How can you control the local caching mechanism?

The fact is, you don't have much control over the local caching mechanism. The local caching happens whenever you retrieve and view some online forum contents. The saving of the retrieved forum contents is an automatic process, which happens behind the scenes. The only way you can order the forum reader to save some specific online forum contents locally is simply to view them. This means the local caching mechanism that we implement in the forum reader is a kind of passive caching. You view some online forum contents and they will be saved locally; if don't view anything, nothing will be saved. The local caching mechanism implemented in the forum reader doesn't have the capability to grab an entire online forum (including all of its topics and replies) in one go. If you want to achieve this effect, you have to manually traverse the forum by viewing every topic and reply therein.

On a related note, after the online forum contents have been saved locally, they will persist in the local database from that point on. For the sample application, we haven't implemented the feature to delete forum contents stored in the local database. However, it shouldn't be a difficult task to add this feature if you so desire.

The Benefit of the Local Caching Mechanism

After all, what good is it maintaining such a local forum database or implementing such a local caching mechanism? As you can imagine, the purpose of it is to support offline reading. Given the fact that a local forum database contains all the forum contents a given user has ever viewed, a new functionality based on this feature could be implemented in the forum reader application that allows the users to browse through the locally stored forum contents without connecting to the internet. If you simply want to revisit a previously viewed forum topic or reply, you just need to browse through the locally stored forum contents, thereby avoiding the hassle of getting on the Internet and browsing through the online forums. In short, this feature makes for quick and easy offline reference without the need to wait for the slow Internet response. On the other hand, as an additional bonus, a certain amount of internet traffic is reduced on the forum server because of this offline browsing feature. By and large, the forum reader, in conjunction with the supporting web services, make an excellent compliment to traditional discussion forum websites.

Database Design and Structure

In essence, our forum reader application reflects a variation of the traditional client/server solution architecture with a modern day Web Services technology. By virtue of this nature, our database design issues are concerned with two functionally interrelated but physically discrete databases, namely an online forum database and a local forum database.

Online Forum Database

The so-called online forum database is simply a backend database server that stores forum data and contents. This database server is originally installed to support a front-end discussion forum website. It is against this existing forum database server that we've built a new interface using ASP.NET Web Services to both expose and interact with its contents. In other words, we simply built a new data access layer to an existing forum database server using Web Services, rather than adding a new dedicated database server to solely service our forum reader application. Because an existing forum website's database schema should have already been properly designed with all necessary tables and stored procedures in place, so the Web Services we built simply access these existing tables and stored procedures. Usually, when implementing such a solution in a real world scenario, there is no actual work concerned with designing the online forum database. With the online forum database here, our only concern is only to examine its schema and get familiar with its stored procedures.

In our sample application, we choose Microsoft SQL Server as the database server for the online forum. The reason for this choice is simply because the .NET framework provides a set of more optimized classes to support SQL Server. Specifically, these classes are grouped under the namespace System.Data.SqlClient, and they directly communicate with Microsoft SQL Server using a low-level proprietary protocol, which bypasses both OLE DB and ODBC. Therefore, by using these classes, we get dramatic performance benefits.

Online Forum Database Schema

Because the main purpose of this case study is to demonstrate the design and development of the entire forum reader application, as a trade-off I purposely choose the online forum database schema to be reasonably simple and straightforward for the sake of clear illustration. As for the sample online forum database, it only incorporates just enough tables and fields to support a bare minimum of essential discussion forum functionalities. When you have a clear picture of the entire application architecture, you should have no problem to adapt it to work with any sophisticated forum database structures.

The following diagram shows the full online forum database schema. As regards the column naming convention in the database schema definition, we use Pascal Casing for all primary key and foreign key columns. For all non-key columns, Camel Casing is used. Even though Pascal Casing and Camel Casing are typically recommended in the .NET class naming conventions, here I also use them for the naming of the database table columns. For those of you who happen to be unfamiliar with these two capitalization styles for identifiers, here is a brief definition:

  • In Pascal Casing, the first letter in the identifier and the first letter of each subsequent concatenated word are capitalized.
  • In Camel Casing, the first letter of an identifier is lowercase and the first letter of each subsequent concatenated word is capitalized.

Figure 4: Online Forum Database Schema

One-to-Many Relationship

From the above diagram, you can easily identify a cascading one-to-many relationship existing between the forum tables. Specifically, a particular forum category could have many forums grouped together where each one in turn could possibly contain many discussion topics where each topic tends to receive many replies. This coherent model of one-to-many relationship is enforced by first installing a primary key in each table and then establishing foreign key constraints between the related tables to reference the primary keys.

Additionally, if you have experience of developing a forum website, you may also notice another design consideration from the above diagram. That is, the topic and reply entities are separated into two different tables instead of being combined into a single one.

While the above diagram is already self-explanatory as regards the meaning of each table and field, a brief summary of their exact definitions are presented here for your reference:

Categories Table

The Categories table stores the information describing the categories by which the forums are grouped together. An auto-numbering identity field named CategoryID is defined as the primary key of this table, which is also used for foreign key reference from the Forums table. The categoryName field stores the name of categories. There is a unique constraint placed on the categoryName field to ensure that no duplication of category name among different categories is ever allowed.

Forums Table

The Forums table stores the information about the subject focus and a brief description of the forums in the forumName field and the description field respectively. The identity field ForumID serves as the primary key, whereas the CategoryID field refers to the category to which a specific forum belongs. The CategoryID field is also used to create a foreign key constraint, which references the same name primary key field of the Category table. In addition, there is a unique constraint applied to the forumName field to ensure that all forum names must be unique.

Topics Table

The Topics table stores the topics that are the initial messages posted against a particular forum. Conceptually, every topic in this table is intended to represent the subject for a new thread of discussion, which will aggregate the relevant replies in the Replies table.

The TopicID field is the primary key identifying each topic. The ForumID field is a foreign key that refers to the forum to which a specific topic is posted.

The subject and message fields represent the caption and inner contents of the topic, while datePosted represents the date when a topic is posted.

The Originator field represents the user who posted a specific topic. This is a foreign key field referencing the relevant user account data stored in the Users table.

The viewCount and replyCount fields are used to record statistical information about a particular topic, which reflects the level of popularity of each thread of discussion. The lastPost field indicates the time at which the latest reply pertaining to a topic is posted. Specifically, whenever a given topic is viewed either through the forum website or from our forum reader application, its viewCount should be incremented. Whenever a new reply to a given topic is posted to the Replies table, the replyCount of this topic should be augmented together with the lastPost value updated in accordance with the date of this reply.

The Revision field is used to identify whether a topic has been changed since the last time we looked. To give an example, when a new topic is posted to the Topics table, its Revision field initially takes the value of zero. From that point on, whenever any new replies targeting this topic are added to the Replies table, the Revision value of this very topic should be incremented by 1. By employing this small technique, it will enable the forum reader application to keep track of whether a particular topic has had any new replies since it was cached previously.

Replies Table

The Replies table stores the replies to existing topics in the Topics table. The ReplyID field is the primary key identifying each reply, and the TopicID field is a foreign key referencing the target topic. The PostedBy field stores the user account ID of the sender of a particular reply. It is also a foreign key referencing the relevant user account in the Users table. The message and datePosted fields have identical meaning to that of the corresponding fields in the Topics table. However, the Revision field is simply a reserved field for future use and not currently used in this sample application. The forum reader doesn't have the functionality to modify replies. However, if you want to implement this functionality, you can use the Revision field of the Replies table to keep track of the revision number of each reply.

Users Table

The Users table stores the detail of user accounts, which are created by the users and used to authenticate the users' session when they are posting topics or replies. Note that in this particular application we only authenticate the users upon topic posting and replying actions, whereas we don't authenticate in the case of viewing or replying to topics. The UserID field is the primary key identifying each user. The userName and userPassword represent the users' login name and password. The userRealName and userEmail fields are simply some additional information about the user.

Stored Procedures

All data in the online forum database are accessed through a set of stored procedures. Here's a brief summary of them:

  • getCategories : returns all category records in the Categories table.
  • getForums : returns all forum records in the Forums table.
  • getOnePageOfTopics : returns a specified page of topics.
  • getReplies : takes a TopicID value as an input parameter and returns all the replies to this topic.
  • getTopic : takes a TopicID value as input parameter and returns the entire topic record.
  • addTopic : adds a new topic to the Topics table.
  • addReply : adds a reply record to the Replies table.
  • createAccount : creates a user account record in the Users table.
  • login : authenticates the user against the user account data stored in the Users table.

Local Forum Database

The local forum database refers to the one that is inherently an integral part of the rich-client forum reader application. It's an MS Access 2000 database residing in the same directory in the client computer as the forum reader application is installed. The forum reader uses ad hoc SQL to access this local database via ADO.NET, because MS Access database doesn't support standard stored procedures. Functionally, the local forum database serves as a local cache used by the forum reader to store the forum contents grabbed from the remote online forum database via the forum Web Services. The cached forum data in the local database can persist as long as you don't delete them directly through MS Access.

One point to note is that the local database is only meant to serve the internal use of the forum reader. Deleting records from the local database directly through MS Access or by any other means would of course cause certain locally cached forum contents to go away. However, this will not harm the functionalities of the forum reader as long as you don't modify the schema or the structure of the local database.

On a related note, even though here we have chosen MS Access to implement the local database of the forum reader application, but this shouldn't be the only choice. Theoretically, you can choose whatever SQL-92 compliant databases that are supported by ADO.NET. Possible choices include Microsoft Visual FoxPro, dBase, Paradox, etc. If you want to modify the forum reader application to work with these databases, you have to change the relevant codes of the data access component to accommodate the new database you choose.

Local Forum Database Schema

Figure 5: The complete schema of the local forum database

Because the purpose of the local database is to save the retrieved forum data locally, for the most part the local database schema mirrors the online database schema only with a small number of variations. Here we only concentrate on these variations.

No primary keys in the four tables are auto numbered because their values are already predetermined by the server-side online forum database.

The Users table that originally exists in the online forum database is excluded from the local forum database. You absolutely don't want every installation of the forum reader application to have a local copy of the entire Users table. Consequently, in the local database, the Originator and the PostedBy fields of the Topics and Replies table have been redefined to store the actual user real name rather than the UserID as a foreign key. The reason is obvious - when the users browse through the locally cached forum topics and replies, we certainly want to display the real names of the people who posted the topics and replies. However, as we mentioned, the Users table doesn't exist in the local database, therefore we must store the real names along with each topic and reply.

Three new fields have been added to the Category, Topics and Replies table in the local database. They are the isObsolete field of both the Category and Topics tables, and the isViewed field of the Topics Table. Sometimes the administrator of the forum website may decide to remove a whole forum or even an entire category from the online forum database for whatever reason. However, this forum or category that is to be deleted from the online forum database may have already been cached in the local database of the forum reader application. More often than not, the user would prefer to maintain this particular local forum or category for later reference if they contain content is of interest to them. In this circumstance, we set a flag on the isObsolete field of that forum or category to indicate that its counterpart on the server-side forum database has been deleted.

Besides these variations, all other aspects of the local forum database schema are virtually identical to the online forum database.

Web Service Design

Figure 6: Web Service Architecture

The Web Service component exposes the forum contents and functionalities to the rich-client forum reader. This component has a three-layer architecture. The data layer is simply the forum database hosted on SQL Server. The data access layer consists of a data access object together with a set of stored procedures inside the forum database. The business façade layer has two separate Web Service modules, namely the forum service and the account service. These two Web Services use SOAP as the transport protocol to interact with all the forum reader clients and serve their own particular business purposes.

All the source code files of the Web Service component are contained in a single Visual Studio.NET project named ForumService.

The Data Access Layer

You can find the source codes for the data access class from the file named dataaccess.cs in the download package. The data access class (dataaccess.cs) performs two different tasks. One is to create a database connection object. The other is to build a thin layer that wraps around each store procedure and abstracts all accesses to the database.

If you look at the source codes, you will find that there is a method named GetSqlConnection() that reads the connection string from the configuration file and creates a connection object. Each of the other methods in this class simply wraps a specific stored procedure and creates a SqlCommand object with the SqlParameter properly configured.

The main purpose of the data access class is to isolate the objects belonging to the business logic layer from the details of the database access information. We don't want the Business Logic Layer to include any information on what was necessary to access the database. To keep the layers of the architecture distinct and the design clean, we would need to wrap access to the stored procedures.

The Business Logic Layer

The logic layer represents the public interface that the forum reader application interacts with. It consists of two Web Services, namely the forum service and the account service.

Forum Service

The codes for the forum service are contained in the file called Services.asmx in the download package. You can find it in the ForumService project.

The purpose of the forum service module (Services.asmx) is to actually expose the forum contents and communicate them to the rich-client forum reader. Let's first take a look at one of its Web Methods:

[WebMethod (Description="Fetch a given page
            of topics in the specified forum", EnableSession=true)]
public
            DataSet GetOnePageOfTopics(short forumID,short pageNumber, short
            pageSize,ref int totalTopics,ref int returnCode)
{
//
            Validating input parameters
if ((forumID < 1) || (pageNumber
            < 1) || (pageSize <1))
{
//return adhoc error code if
            parameters are invalid
//check the file ServiceErrors.txt for
            error description
returnCode = 1007;
return
            null;
}
//Parameters are valid.
else
{
DataSet ds =
            new DataSet();
SqlConnection conn =
            dataAccess.GetSqlConnection();
try
{
//Retrieve the topics
            from DB based on the parameters
SqlCommand cmd =
            dataAccess.GetOnePageOfTopics(conn);
cmd.Parameters["@ForumID"].Value
            = forumID;
cmd.Parameters["@PageNumber"].Value =
            pageNumber;
cmd.Parameters["@PageSize"].Value =
            pageSize;
conn.Open();
SqlDataAdapter sda = new
            SqlDataAdapter(cmd);
sda.Fill(ds,"Topics");
conn.Close();

//Here,
            returnCode will always be 0 indicating no error.
returnCode =
            Convert.ToInt32(cmd.Parameters["@RETURN_VALUE"].Value);

//return
            total number of topics in this forum
totalTopics =
            Convert.ToInt32(cmd.Parameters["@TotalTopics"].Value);
}
catch(Exception)
{
//throw
            SoapException for all unexpected errors.
throw new
            SoapException("It was not possible to retrieve the topics.",
SoapException.ServerFaultCode);
}
finally
{
//make
            sure DB connections are closed
if (conn.State ==
            ConnectionState.Open)
{
conn.Close();
}
}
//return
            the topics data to the calling client.
return
            ds;
}
}

This Web Method returns a DataSet object containing a page of topics within a particular forum that are specified by the input parameter forumID. Because topics are returned to the client in a page-by-page manner, the parameter pageNumber is used to specify which page of topics to fetch and the pageSize parameter specifies the maximum topics allowed in a page. This Web Method also returns the total number of the topics that currently exist in that particular forum, and this value is returned through the ref parameter totalTopics.

While this is all quite straightforward, there are a few points to note:

  • We have enabled session support for the Web Method with the attribute declaration [WebMethod(EnableSession=true)]. This is also the case with all other public Web Methods of the two Web Services. This is necessary because we decided to enlist ASP.NET session management to maintain state for the authenticated user. We'll examine the authentication scheme as well as the issues concerned with enabling session support for Web Services in the next section.
  • Input parameters are always validated at the beginning of the Web Method.
  • All identifiable error conditions are signified by returning ad hoc error codes to the calling client through a " ref " parameter. The meanings of these error codes are documented in the text file named ServiceErrors.txt. At the client side, when the forum reader receives such an error code after calling the forum Web Services, it identifies the error condition by checking the error code and then displays meaningful messages accordingly..
  • As for all the web methods that return a particular kind of forum data to the client, the returned data are always in the form of a .NET DataSet.

The above points represent some general principles that we conformed to when coding each Web Method of the two Web Services. Here we list all the other methods in the forum service module together with a description of their respective purposes:

  • GetCategoriesForums : returns all the records in the Category and Forums table.
  • GetOneTopicWithItsReplies : returns the details of a specific topic in the Topics table and all of its relevant replies in the Replies table.
  • AddTopic : adds a new topic to the Topics Table.
  • AddReply : adds a reply record to the Replies table.
  • IsUserAuthenticated : returns the UserID value that was previously saved in the session state when the user was authenticated.

You may have noticed that we haven't provided any Web Methods to remove or alter the forum contents. If you need these functionalities when you develop your own version of the forum reader application, you can add them to the forum service.

Account Service

The account service module deals with user account creation and handles the login/logout process. It has three Web Methods:

  • The Login method handles the user login process and authenticates the user session.
  • The CreateNewLogin method takes the user's account registration detail and creates a user account for the user. Note that after the account is created, the user is automatically logged in.
  • The Logout method let the server explicitly abandon the user's session state data, which effectively returns the user back to the unauthenticated state.

A Simple Authentication Scheme

Now we will go into more detail about how to implement a simple authentication scheme for the Web Method calls.

Whenever a particular user posts a new topic or sends a reply to the forum service, the forum service must first obtain the identity of the user who is sending this request. Then, the identity of the user (UserID) in conjunction with the new topic or reply that the user has posted are saved to the Topics or Replies table as a whole. Any unauthenticated posting request from the forum reader should be denied. This is the reason why we have to implement some sort of authentication scheme.

Because ASP.NET provides a very flexible, easy to use, and scalable session state management mechanism, we simply based our user authentication scheme on the intrinsic ASP.NET session state management facility.

The default ASP.NET session scheme relies on HTTP cookies to track user session state. A cookie is typically used to track session state for browser-based clients. In that case, an HTTP cookie is sent along with the HTTP response to the client and placed in the client browser whenever a user makes the first HTTP request during a particular session. From that point on, ASP.NET will check for the presence of this cookie in every HTTP request sent by the browser so as to identify the browser and maintain the state for it at the server. However, cookies can also be used with ASP.NET Web Services. In fact, ASP.NET will also place session cookies in every Web Service client whenever the client makes the first Web Method invocation during a session. This means that for every active Web Service client, its specific state information is also automatically maintained by ASP.NET at the server. What this means to our authentication scheme for Web Method invocations is that we could store the user identity data (such as UserID value) in the session state whenever the user has successfully completed the logon process. The presence of this user identity data in its session state could indicate that the current session of that particular user has been authenticated. Otherwise, if this identity token (UserID) doesn't exist in the user's session state, it simply means the user hasn't logged in. By checking this identity token (UserID) against the Users table, our Web Service could identify which user is making the current invocation. Now, let's take a look at the code of the Login method that handles the user login process:

[WebMethod (Description="Authenticate the user session", EnableSession=true)]
public void Login(string name, string password, ref int returnCode)
{
//Validate input parameters.
if ((name == null) || (password == null) || (name.Length > 100) ||
(name.Length ==0) || (password.Length > 100) )
{
//return adhoc error code if parameters are invalid
//check the file ServiceErrors.txt for error description
returnCode = 1018;
return;
}
SqlConnection conn = dataAccess.GetSqlConnection();
try
{
//Authenticate the user
SqlCommand cmd = dataAccess.Login(conn);
cmd.Parameters["@Name"].Value = name;
cmd.Parameters["@Password"].Value = password;
conn.Open();
cmd.ExecuteNonQuery();
conn.Close();

//Pick up the return code from the stored procedure "login",
//Possible values are 0, 1013, 1014. 0 indicating successful
//authentication. Other values indicate errors.
//check the file ServiceErrors.txt for error description.
returnCode = Convert.ToInt32(cmd.Parameters["@RETURN_VALUE"].Value);

//Authenticated. (account name and password both correct).
if (returnCode == 0)
{
//Store the UserID in the session, as a token to
//indicate the user has been authenticated.
this.Session["UserID"] = cmd.Parameters["@UserID"].Value;
}
}
catch(Exception)
{
//throw SoapException for all unexpected errors.
throw new SoapException("It was not possible to login at the moment.",
SoapException.ServerFaultCode);
}
finally
{
if (conn.State == ConnectionState.Open)
{
conn.Close();
}
}
}

This code shows that the user supplied name and password are checked against the Users table by executing a SqlCommand that wraps around the relevant stored procedure. If the name and password are correct then a value of zero together with the UserID value will be returned from the stored procedure. Upon discovering that a value of zero has been returned from the stored procedure, the UserID value is then saved to the session state. If the AddTopic or AddReply Web Method of the forum service needs to obtain a user's identity, they can simply check whether session.["UserID"] exists in the current user's session state.

By means of the simple code described above, our Web Service authentication scheme is implemented. The upside is that the code is very neat and clean thanks to the ASP.NET session state management mechanics. Moreover, this authentication scheme is also secure enough because apart from a cookie containing the ASP.NET SessionID, no actual user identity data is ever sent across the wire to the client. In addition, the cookie is only valid for the current session and will be discarded by the server when it times out. We could boldly claim that our Web Service authentication scheme is as secure as ASP.NET session management. However, because this scheme is heavily dependent upon cookie-based session management, one pitfall that we must be aware of is that we must implement the cookie container in the rich-client forum reader application and associate the cookie container with every Web Service call. If the client-side cookie container is not implemented, our entire authentication scheme simply won't work at all. In that event, to make matters worse, a new session is created on every client request, which occupies a bit of server resources until it's timed out. After all, the good news is that with .NET, implementing a cookie container is a snap. We'll examine how to implement a cookie container in the forum reader a little later. .

As a side note, if you're interested in using a cookieless mechanism for supporting sessions with Web Services, you can find an article on this topic in MSDN website, which is entitled "Using ASP.NET Session State in a Web Service". The link to this article is provided in the links section of this case study.

Enter the Forum Reader Application Design

Now that we have looked at the supporting Web Services in some detail, we can begin to dive into the centerpiece of this case study: the design of the rich-client forum reader.

Componentization of the Forum Reader

Figure 7: Component Architecture of the Forum Reader

The forum reader application is implemented as a traditional windows application using the .NET Windows Forms technology. From an object-oriented point of view, the application is composed of four distinct logical components, namely the user interface component, the business logic component, the (local) data access component and the external system integration component.

The user interface component only deals with the user interaction aspect of the application. On the one hand, it receives user requests (such as retrieving topics or sending a reply) from the user but never attempts to process these requests by itself - instead it delegates these requests to the business logic component to actually fulfill them. On the other hand, the user interface component receives the forum data that has been processed by the business logic component and then display them in the various GUI widgets.

The business logic component plays the role of an intermediary between the user interface and the underlying local data access component or the Web Service proxies. It receives various requests from the user interface and then enlists the Web Service proxies and/or the local data access component to fulfill the requests.

The local data access component abstracts access to the local database. It encapsulates data access logic of both saving data to and retrieving data from the local database.

The forum service proxy and account service proxy together comprise the external system integration component for the forum reader application. Here we label the Web Service proxies as the external system integration component in that from the vantage point inside the rich-client forum reader, the remote forum Web Service could be viewed as an external system indeed. Apart from this object-oriented methodology concept, the good news is that there's no coding required on our part to create this component because Visual Studio.NET could automatically generate the plumbing codes for any Web Service proxy.

Now with the above description, we have a general overview of the high level components of the forum reader application. The separation of the entire application logic into these four specialized components is based on classic object-oriented analysis and design methodology. This typical pattern of componentization allows us to isolate problems and group objects according to problem domain or technology area. In a nutshell, doing so enables a higher degree of object reuse and a higher level of system/application maintainability over the course of its life.

Physical Component Packages

If you look at the source code in the download package accompanying this case study, you could find that except for the external system integration component (the Web Service proxies), each of the other three components is contained in its own Visual Studio.NET project. Specifically, the user interface component is contained in the ForumClient project. The business logic component is contained in the BusinessRule project. The local data access component is contained in the LocalDataAccess project. The two Web Service proxies reside in the BusinessRule project. The ForumClient project is dependent on the BusinessRule project in order to be successfully compiled. Likewise, the BusinessRule project is dependent on the LocalDataAccess project in order to be successfully compiled.

Functional Sequence

Now let's take a brief look at an example of the functional sequence of actions that takes place among the different components. In a typical situation, the business logic component would use the forum Web Service proxies to retrieve a page of topics from the remote forum service at the request of the user interface component. After it completes the retrieval of the specified topics, it uses the local data access component to store the retrieved topics in the local database. Finally, the business logic component notifies the user interface component of the retrieved topics by firing an event. The user interface component is pre-configured to subscribe to such events sent from the business logic component. Thereby the user interface component extracts the topics data from the event argument and display them in the user interface. Below is the UML sequence diagram illustrating this process:

Figure 8: A sample of functional sequence between these components

All the data retrieval behaviors in the forum reader application follow the same pattern as shown in the above diagram. In addition, they are all implemented in the same vein. We'll cover this process in depth in the section about the business logic component.

Architectural Overview of the Overall Class Model

Here we illustrate the big picture of the overall class model of the forum reader application:

Figure 9: Overall class model of the forum reader application

Our class model design directly reflects the component architecture that is described above. From the class model diagram, we can easily distinguish to which component a particular class belongs. For example, the MainForm class, the PostNewTopicForm class, the ReplyForm class, the LoginForm class, and the CreateAccountForm class altogether comprise the entire user interface component, while the the CategoryManager class, the TopicsManager class, the RepliesManager class, and the AccountManager class form the entire business logic component. The LocalDataAccess class alone constitutes the local data access component.

Given our high level component design outlined in the preceding section, deriving such an overall class model should be intuitive. However, there're some design considerations that we should be aware of:

  • The business logic objects are factored out into four distinct classes, namely the CategoryManager class, the TopicsManager class, the RepliesManager class and the AccountManager class. We come up with such a factorization simply because we want each business logic class to deal with only one specific kind of forum data. In other words, we want all the actions relating to a particular kind of forum data be encapsulated in a single class. The name of these classes implies the specific data that these classes process. For instance, the CategoryManager class deals with retrieving the categories and forums information from the remote forum service as well as saving them to the local database.
  • The MainForm is the main user interface that is populated with the various GUI controls for the user to interact with. The MainForm also performs the role of central management in that it creates all the business logic objects and maintains references to them throughout the lifetime of the application. It delegates tasks to the specific business logic objects by calling their methods directly through these references. However, in the opposite direction, the business logic components have no direct reference to the MainForm or any other GUI objects. The business logic components have to communicate data or information to the MainForm only through firing events. Another point worth noting is that all the business logic components are singletons. There is only one instance for each business logic object existing through the lifetime of the MainForm or the application. All the business logic objects are created by the MainForm upon application start-up, thereafter they all have their lifetimes in parallel with that of the MainForm.
  • There are a few other helper classes in the application that are not included in the above class model diagram for the sake of clear illustration.

User Interface Design

Figure 10: The main user interface of the forum reader application

Laying out the GUI widgets

The forum reader application has a main user interface that loosely resembles the one used in Microsoft Outlook. At first sight, we could see from the above screenshot that the main form features an OutlookBar control containing a TreeView control on the left side and the right side containing a ListView control above a RichTextBox control. The OutlookBar control is a third party offering, not from Microsoft. Here I must give credit to its developer Mr. Carlos H. Perez for providing such a fantastic control. Now let's examine how these various controls are laid out on the main form in this fashion. The technique for it is to partition the screen real estate into several panes using Panel controls and then place Splitter controls on the border between the adjacent panes that needs resizing. The Splitter controls allow independent resizing of the other bordering controls.

A long and narrow Panel named topPanel is docked horizontally against the top edge of the main form. Immediately under it, a Panel named outlookBarPanel is docked on the left side. Another Panel called rightPanel fills the entire right side. The topPanel contains a Label control to display the information about the particular forum being viewed. The outlookBarPanel is filled with the outlookBar control, and this outlookBar control in turn hosts two TreeView controls that are used to display the categories and forums information for forum navigation. The TreeView control named onlineForumTreeView is used to display the categories and forums information directly retrieved from the forum Web Service, while the TreeView control named localForumTreeView displays the same kind of data retrieved from the local database. A Splitter control is fixed between the outlookBarPanel and the rightPanel. The rightPanel contains a child panel called listViewPanel that is docked on the upper part within the rightPanel. The bottom area of the rightPanel is filled with a RichTextBox control named repliesRichTextBox. The repliesRichTextBox is used to display all the replies to a particular topic. A Splitter control is slotted between the listViewPanel and the repliesRichTextBox. Finally, a ToolBar control (topicNavToolBar) together with a ListView control (topicListView) on top of it is positioned inside the listViewPanel. The ListView control is used to display the topic list and the ToolBar below it is used to navigate topics in a page-by-page fashion. This is all it takes to craft our Microsoft Outlook-like user interface.

Other UI-related classes

Besides the main user interface, there are several other standalone forms that are displayed on an ad hoc basis:

  • The LoginForm is used to input the account name and password for the login process.
  • The CreateAccount form is used to input user account registration information.
  • The NewTopicForm and ReplyForm are used to input new topics and reply to existing topics respectively.

Another important aspect of the user interface is the context menu. Our forum reader application relies heavily upon contextual menus for smooth user interaction. For example, the onlineForumTreeView, the localForumTreeView and the topicListView all have their own specific contextual menus for performing the relevant tasks.

Examining GUI behaviors

Now let's give the forum reader a test run to get a taste of its GUI behaviors. When the application is launched, by default the left side TreeView control displays the available categories and forums cached in the local database. Right clicking anywhere in the TreeView control brings up a context menu containing the menu item Refresh Category. Click this menu item and after a few seconds, the available categories and forums are displayed in the TreeView. Right Click on the TreeView control again, this time choose Latest Topics and in a moment a list of topics are displayed in the ListView control. Right click any topic in the ListView control and choose GetReplies in the context menu, then the relevant replies are displayed in the RichTextBox control. These are some typical behaviors of the main user interface. I leave the rest of them for you to explore.

Marshalling Cross-Thread Function Call

A principle of Windows Programming dictates that all function calls to a GUI control must occur in its creation thread. This implies we should not operate a window control from anywhere other than its creation thread. The reason why this principle should be complied with is to do with the inner workings of native W32 windows, especially the single-threaded apartment memory model. However, this is a very complex issue and explaining the rationale behind it with just a few words is almost impossible here. Typically, it requires at least a few dedicated book chapters to fully explain the related concepts. So here, we just take it for granted that this principle represents a doctrine or dogma of windows programming. For further information about this issue, you can follow the relevant links at the bottom of this case study. You can also visit the MSDN website to get more information about threading and windows programming.

Usually, multithreaded programming is associated with user interface-based applications that need to perform a time-consuming operation without affecting the end user. In our case, the forum reader calls the remote Web Services across the internet to retrieve forum contents. This is, by nature, a time-consuming operation. So, in the implementation of the business logic classes, we create new worker threads to perform the Web Service calls. . When these business logic objects finish retrieving data from the remote Web Services, they would fire events to send data or message to the Mainform for display. However, when the MainForm picks up the data or message from the event argument in order to display them in a particular control, it has no knowledge that the event is actually fired from a worker thread that is different from the main GUI thread. In this case, due to the above-mentioned windows programming principle, the event handler must marshal the function call to any GUI control from the worker thread to the main UI thread.

To give an example to demonstrate this trick, suppose we have a helper method that displays text in a StatusBarPanel like this:

//This method is not thread-safe when making cross-thread call.
Private void ShowStatusBarInfo(StatusBarPanel statusBarPanel,string info)
{
statusBarPanel.text = info;
}

However, if this method is to be called from another method running on a different thread, then according to the above-mentioned windows programming principle, it cannot be called directly without the necessary marshalling. We must transform the original method to incorporate the necessary marshalling codes. The following revised code does this work with the help of a delegate:

//
delegate void ShowStatusBarInfoDelegate(StatusBarPanel statusBarPanel,string info);

private void ShowStatusBarInfo(StatusBarPanel statusBarPanel,string info)
{
//The caller is from another thread that is different from
// the main UI thread. ('this' refers to the mainForm,which is
// of course residing on the main UI thread.
if (this.InvokeRequired == true)
{
//Create the delegate that represents this method itself.
ShowStatusBarInfoDelegate showStatusBarInfo =
new ShowStatusBarInfoDelegate(ShowStatusBarInfo);
//Calling this method itself again through the delegate and
//marshal the call to the main UI thread.
this.Invoke(showStatusBarInfo,new object[] {statusBarPanel,info});
}
//The caller is from the main UI thread.
else
{
statusBarPanel.Text = info;
}
}

Each UI class in Windows Forms (that is, every class that ultimately derives from System.Windows.Forms.Control) has a property that you can safely use from any thread to detect whether a call to the UI class is from a different thread. The name of the property is InvokeRequired, which returns true if the calling thread needs to pass control over to the creating thread before calling a method on that object. Also, every UI class has a method called Invoke that could be used to marshal the call to its own method from any other thread to its own creation thread. Put it in another way, if an object existing on a worker thread calls the Invoke method of a UI class, then the Invoke method will be executed on the main UI thread rather than on the worker thread. This is exactly the effect of marshalling a function call from a worker thread to the main UI thread. Because the invoke method requires a delegate in its parameter list, that's why we define a delegate called ShowStatusBarInfoDelegate. This delegate has exactly the same signature as the method to be called.

If you look at the source code of the MainForm class, you could see that we actually marshaled all the event handlers to the main UI thread.

Remote Data Access Component

In our forum reader application, the remote data access component consists of two Web Service proxies as shown in the figure below:

Figure 11: Web Service Proxies

Visual Studio.NET greatly simplifies the process of creating Web Service proxy classes. We just need to add the web references that point to the URLs of the Web Services to be called and Visual Studio.NET will handle the creation of the Web Service proxy classes automatically. As shown in the above diagram, we've added two web references to the BusinessRule project, namely the ForumServiceProxy and the accountServiceProxy. The ForumServiceProxy contains the class that encapsulates method calls to the forum service, whereas the accountServiceProxy encapsulates method calls to the account service. These Web Service proxies are used by the business logic objects in order to communicate with the remote Web Services.

Local Data Access Component

Figure 12: LocalDataAccess project

The Local Data Access component has only one class called LocalDataAccess, which is contained in the Visual Studio.NET project named LocalDataAccess. The LocalDataAccess class serves the purpose of accessing the local forum database. It encapsulates all the logic concerned with saving data to the local forum database as well as extracting data from it, thereby insulating the business logic classes from the details of accessing local database. For example, after the business logic objects retrieve certain forum contents from the forum Web Service, they immediately call the LocalDataAccess object to save the retrieved forum contents to the local database.

The LocalDataAccess class uses ad hoc SQL to interact with the local database via ADO.NET. Three methods in this class handle the retrieval of different kinds of forum data from the local database. They are GetCategoriesForums, GetTopics and GetReplies, which retrieve the data of categories, topics, and replies respectively. All the other methods in this class deal with saving different kinds of forum data to the local database, such as categories, topics and replies. One point worthy of note is that these methods all employ a bit of decision logic in the saving process. This decision logic is illustrated in the diagram below:

Figure 13: The decision logic employed when saving data to the local database

The idea is to check whether the data to be saved already exists in the local table. If it doesn't, then we insert it into the table. Otherwise, we only need to update the existing data if its revision value is out of sync with the data to be saved.

The Business Logic Component

In the forum reader application, the business logic component refers to the one that sits in between the data access component (either remote or local) and the user interface component. It consists of a set of classes that take orders from the GUI and delegate them to the data access components, or at other times, they transfer the results retrieved from the data access components to the GUI. In this section, we'll examine the inner workings of these business logic classes.

All the classes belonging to the business logic component are contained in the Visual Studio.NET project named "BusinessRule". You can find them in the download package.

CategoryManager Class

Figure 14: CategoryManager Class

The source code for the CategoryManager class is contained in the file named CategoryManager.cs. The responsibility of the CategoryManger class includes fetching categories and forums either from the forum Web Service (GetOnlineCategoriesForums method) or from the local database (GetLocalCategoriesForums method). The public field IsFetching is a flag used to indicate that the CategoryManager object is currently in the process of fetching data from the remote service.

Multithreaded Programming

Calling remote Web Services to fetch data can be a long-running operation. If we directly use the main UI thread to perform this task, then the user interface will temporarily freeze up while waiting for the Web Service call to complete. This is because the main UI thread has been held up and couldn't respond to user interaction until the Web Service call returns. We can get around this bad user experience by using multithreaded programming techniques. Specifically, we can call the remote Web Service in a multithreaded way in order to free up the main UI thread and keep it alive for continuous user interaction.

When making multithreaded calls to Web Services, there are two choices open to us. One is to create a new worker thread and call the Web Service synchronously on that thread. The other is to call the Web Service asynchronously. In fact, both methods are suitable for our purpose. Here, we choose the former one, which is to call the Web Service synchronously on a new thread, and it turns out this method works for our forum reader without a hitch.

Now let's look at the code for the GetOnlineCategoriesForums method and the startFetchData method of the CatetegoryManger class, which implements the multithreaded Web Service call. You can find this code in the file named CategoryManager.cs in the download package:

public void
            GetOnlineCategoriesForums()
{
//Create a new thread and
            specify startFetchData method as the code
//to be executed on the
            thread.
fetchDataThread = new Thread(new
            ThreadStart(startFetchData));

//Name the thread for debugging
            purpose.
fetchDataThread.Name="Fetch Categories";

//Check
            if the user has requested to cancel the retrieving of
//categories and forums.
if (this.AbortRequested ==
            true)
{
//Dispose of the thread.
fetchDataThread =
            null;
//Set AbortRequested to indicate no
            cancellation.
this.AbortRequested = false;

//Check if
            there are subscribers hooked up with the event.
if
            (FetchCategoriesCompleted != null)
{
//Raise the event
            indicating user's cancellation.
FetchCategoriesCompleted(this,new
            FetchEventArgs(FetchStates.Aborted,null,null));
}
//Set the
            flag to indicate the CategoryManager object is free
//to service
            new fetching request.
this.IsFetching = false;
}
//The user
            didn't request to cancel the fetching.
else
{
//Triggar the
            thread to
            execute.
fetchDataThread.Start();
}
}

private
            void startFetchData()
{
try
{
//Create the Web Service
            proxy object
ForumServiceProxy.ForumService remoteService = new
            ForumServiceProxy.ForumService();

//Attach the cookie
            container to maintain session.
remoteService.CookieContainer =
            this.CookieContainer;

//Check if user has requested to cancel
            the fetching.
if (this.AbortRequested ==
            true)
{
this.AbortRequested = false;
if
            (FetchCategoriesCompleted !=
            null)
{
FetchCategoriesCompleted(this,new
            FetchEventArgs(FetchStates.Aborted,null,null));
}
this.IsFetching
            = false;
return;
}

//Retrieve the Categories and forums
            from the Web Service.
DataSet categories =
            remoteService.GetCategoriesForums();

//Check if user has
            requested to cancel the fetching.
if (this.AbortRequested ==
            true)
{
this.AbortRequested = false;
if
            (FetchCategoriesCompleted !=
            null)
{
FetchCategoriesCompleted(this,new
            FetchEventArgs(FetchStates.Aborted,null,null));
}
this.IsFetching
            = false;
return;
}
//Save the retrieved data in the local
            database.
LocalDataAccess localDB = new
            LocalDataAccess();
localDB.SaveCategoriesForums(categories);

this.AbortRequested
            = false;
if (FetchCategoriesCompleted != null)
{
//Fire the
            event to send the retrieved data to the event
            handler
FetchCategoriesCompleted(this,new
            FetchEventArgs(FetchStates.Success,categories,null));
}
this.IsFetching
            = false;
}

catch(Exception ex)
{
this.AbortRequested
            = false;
if (FetchCategoriesCompleted != null)
            FetchCategoriesCompleted(this,
new
            FetchEventArgs(FetchStates.Exception,null,ex));
this.IsFetching =
            false;
}
}

The code might appear a bit confusing because we've included additional logic to cancel the fetching operation at the user's request. First, let's see how multithreading is put to work in this code.

The UI component calls the GetOnlineCategoriesForums method of the CategoryManager class whenever the user requests to retrieve the categories and the forums. Inside this method, what the code does is to create a new thread to perform this task. The new thread executes the startFetchData method to perform the Web Service call. Upon completion of the Web Service call, the startFetchData method fires an event encapsulating the retrieved data in the event argument. By creating new thread to do the work, the call from UI component quickly returns from the GetOnlineCategoriesForums method, without having to wait for the actual Web Service call to be completed. The MainForm is configured to subscribe to the event that is fired from the startFetchData. When the event is fired, the MainForm picks up the retrieved categories and forms from the event argument. Here we could see that the event is fired from the new thread that we created in the GetOnlineCategoriesForums method. This is the reason why cross-thread marshalling is necessary when the MainForm responds to this event.

After we get the idea of how multithreading works in the code, let's see how to implement the functionality that allows the user to cancel a pending request.

A user request in our application typically requires two steps to complete. The first step is to retrieve the data from the remote Web Service. The second step is to save the retrieved data to the local database. Because these two steps are all executed on the same thread, we would naturally think that by terminating this running thread we could effectively cancel the user request. Terminating a thread only requires the one line of code that is to call the abort method on that thread. However doing it this way has potential pitfalls. Specifically, we have no control over the exact point of execution at which the .NET runtime terminates our thread. In fact, if we call the abort method on a thread, the thread could be terminated at any point during the execution. What this means is that aborting a thread has a fair chance of leaving the running context of our application in an unstable state, for example if the internals of the .NET runtime had not performed its usual housekeeping work for our application at the time of the thread being aborted. In this event, you might encounter all sorts of strange exceptions relating to the internals of .NET runtime, such as errors with ADO.NET or errors on SOAP messaging to name but a few. Because these errors occur in the internals of .NET, we have no way to get it right from the code in our own application. The upshot of it all is that our application simply couldn't function properly after the thread has been aborted unless we restart the program again.

So attempting to cancel a user request by means of aborting its running thread is simply the wrong approach. We simply couldn't rely on .NET runtime to abort our thread safely. As a rule of thumb, after a thread is started, we have to let it run its course till it reaches its normal exit point and returns.

Due to the lack of a safe way to terminate a thread, I have to make do with a checkpoint approach. I add a flag called AbortRequested in the CategoryManager class. When the user requests to cancel the retrieval of the categories and forums, I set this flag with the value of true. Inside the GetOnlineCategoriesForums and startFetchData method, I insert the following code snippet to probe the value of AbortRequested at certain points:

if (this.AbortRequested == true)
{
fetchDataThread = null;
this.AbortRequested = false;
if (FetchCategoriesCompleted != null)
{
FetchCategoriesCompleted(this,new FetchEventArgs(FetchStates.Aborted,null,null));
}
this.IsFetching = false;
}

Whenever the AbortRequested is detected to have the value of true, we immediately fire an event indicating this situation and then exit the thread. So this code snippet serves as a checkpoint, and we scatter it in several different points in the code in order to capture the user's cancellation request as early as possible. For example, if after the data has been retrieved from the remote service the code detects that the user had actually requested to cancel this operation, then the code would immediately return without bothering to save the retrieved data to the local database.

After all, the idea of this approach is to exit the thread as quickly as possible instead of aborting a thread. While this is not an elegant and ideal approach, it at least fits the bill in our application.

TopicsManager Class

Figure 15: TopicsManager Class

The source code for the TopicManager class is contained in the file named TopicsManager.cs. The TopicsManger class has the following major responsibilities:

  • The GetOnlineTopics method retrieves forum topics from the forum Web Service.
  • The GetLocalTopics method retrieves forum topics from the local database.
  • The PostNewTopic method sends a new topic to the forum Web Service in order to be inserted in the Topics table of the online forum database.
  • The TopicManager object fires an event called FetchTopicsComplete when it finishes retrieving the topic data from the forum service. The topic data retrieved are encapsulated in the event argument, which is of type FetchEventArgs.

RepliesManager Class

Figure 16: RepliesManager Class

The source code for the RepliesManager class is contained in the file named RepliesManager.cs.

The RepliesManager class has the following major responsibilities:

  • The GetOnlineReplies method retrieves the replies to a particular topic from the forum Web Service.
  • The GetLocalReplies method retrieves the replies to a particular topic from the local database.
  • The ReplyToTopic method sends a reply message to the forum Web Service in order to be inserted in the Replies table of the online forum database.
  • The RepliesManager object fires an event called FetchRepliesComplete when it finishes retrieving the reply data. The reply data retrieved from the forum service is encapsulated in the event argument, which is of type FetchRepliesEventArgs. As a side note, the FetchRepliesEventArgs class derives from the FetchEventArgs class.

Again, the RepliesManager class makes use of the same multithreading technique as the CategoryManger class.

AccountManager Class

Figure 17: AccountManager Class

The source code for the AccountManager class is contained in the file named AccountManager.cs.

The AccountManger class has the following major responsibilities: The Login method sends the user- supplied login name and password to the Account Web Service in order to get its session authenticated by the service. If the login process is completed successfully, we set the flag IsLoggedIn to true. Other objects in the application could ascertain whether the current session has been authenticated or not by checking the IsLoggedIn flag of this class.

A subtle point worthy of note is that we need to implement the cookie container (System.Net.CookieContainer) and use it with every Web Service call, because we use cookie based session state management for our Web Service authentication scheme. Therefore, we have added a CookieContainer reference to each of the four business logic objects, and in the MainForm class there is a method named InitializeCookieContainers that is used to actually create a new CookieContainer object and assign its reference to each of the four business logic objects. Now that the business logic objects have the CookieContainer, they attach it to every Web Service call they make so that the user session is maintained across the various Web Service calls in the application.

The Logout method does the opposite thing in that it asks the Account Web Service to destroy the user's current session. Any Web Service call after logout will be processed by the server in the context of a new and unauthenticated session until the user logins again. There is also a subtle point associated with the logout mechanism that we should be aware of. The user session may be timed out by the server, however in that event the forum reader is not informed of this situation immediately, so the IsLoggedIn flag still has the value of true - which is not correct. This is not a problem, because next time when the user post a new topic or send a reply, he will be alerted and at that point the IsLoggedIn flag will be set to false.

Lastly, the CreateAccount method sends the user registration request to the Account Web Service in order to create an account for the user. Immediately after the account is created, the user's current session is automatically authenticated by the Web Service.

Object Interaction

At this point, we've covered all four logical components of the forum reader application. In this section, we'll bring these various components together and put them in action. We'll demonstrate how these various components collaborate with each other to deliver a particular functionality as a whole.

From an object oriented viewpoint, a specific functionality or task of a given application is fulfilled by the underlying sequences of object interactions. Here we take a close-up at one of the most noticeable and most significant sequence of object interactions inside the forum reader application, one that fulfils the task of grabbing a page of forum topics from the forum Web Service and displaying them in the ListView control. This one is a roundtrip in that it starts with the user hitting the Latest Topics menu item to trigger the sequence and ends with the requested forum topics eventually being displayed to the user.

Below is the UML sequence diagram that illustrates this process:

Figure 18: Sequence Diagram for retrieving the latest topics from the web service

First, let's set the scene by retrieving the categories and forms from the forum Web Service. When the forum reader is initially launched, by default, the available categories and forums are retrieved from the local database and displayed in the left-side TreeView. Now if you want to check whether there are new categories or forums added to the online forum, you can right click on the TreeView under the caption Online Forum to bring up its context menu.

Figure 19: Click Refresh Categories in the context menu of the TreeView control

Next, click the Refresh Categories menu item in the context menu. After a short period, the available categories and forums in the online forum database are displayed in the TreeView. Naturally, the forums are displayed as leaf nodes in the TreeView.

Depending on the situation, each leaf node could be in any of three distinctive states: unselected, selected or highlighted. A leaf node is in selected state when you left click on it. When you right click on a leaf node, it will turn into highlighted state. We differentiate between these two states because the users tend to expect that the actions triggered by the context menu items are normally orientated at the highlighted leaf node rather than the selected one. Because the TreeView control already has a property called SelectedNode, we just need to add the highlightedNode field to the MainForm class to track the currently highlighted node. Besides these node state differentiations, the class type of the TreeNode in the TreeView has also been tweaked to support our functional purpose. Specifically, we derived our own type of category TreeNode called CategoryTreeNode in order to attach the CategoryID and CategoryName data to the TreeNode. Similarly, we derive our own ForumTreeNode type to include the ForumID and ForumName data.

Additionally, two public fields in the TopicsManager object, namely CurrentTopicsPage and TopicsPageToFetch, play an important role in the topics retrieving procedure. They are all of type TopicsPageInfo, which encapsulates information about a particular page of topics. The TopicsPageToFetch is used to provide a temporary record of the particular page of topics that is to be fetched. The CurrentTopicsPage is used to store the information about the page of topics that is currently displayed in the ListView.

Now that we've set the scene, let's dive in a bit further.

Right click a leaf node representing a forum in the TreeView and then click Latest Topics in the context menu that follows:

Figure 20: Click Latest Topics in the Context menu of the TreeView

At this moment, the onlineTopicsMenuItem_Click event handler is invoked and it in turn calls the getOnlineLatestTopics method. Inside the getOnlineLatestTopics method we first make sure that the topicsManager object is not already engaged in an ongoing fetching operation. After that, we disable all the buttons on the forum navigation toolbar except the stop button. By doing this, we ensure that the user cannot send more than one request at a time to retrieve topics from the Web Service. Next, we extract the ForumID, ForumName , CategoryID and CategoryName values from the highlighted forum node and its parent category node. With this data, we create the TopicsPageInfo object to denote the particular topic page that we intend to fetch. Here we want to fetch the Latest Topics that is page number one, so we assigned the value 1 to the pageNumber parameter in the constructor of TopicsPageInfo. Next, we delegate the request to the topicsManager object by calling its GetOnlineTopics method and thus the control is transferred.

Inside the GetOnlineTopics method, we first check whether the user has requested to cancel the request during this short moment. This is a bit fussy but we still put this checkpoint code snippet there just to play safe. Next, we spin up a new thread called Fetch Topics and execute the startFetchTopics method on that thread.

Within the startFetchTopics method, we create the forum Web Service proxy object and call its GetOnePageOfTopics method with the parameter values obtained from the TopicsPageToFetch object that we created in the previous step. The forum Web Service proxy then routes the call to the remote forum Web Service and fetches the topic data from there. The response from the forum Web Service includes the topic data that we requested together with the total number of topics that currently exist in that particular forum. As a partial result, we save the value of this total number in the TotalTopics property of the TopicsPageToFetch object. Next, we create the LocalDataAccess object and call its SaveTopics method to save the retrieved topic data to the local database. After that, we fire the event FetchTopicsCompleted with the topic data embedded in its event argument. At this point, the control of this thread leaves the TopicsManager object and passes back to the MainForm again. (Also, at this point the Web Service proxy object and the LocalDataAccess object that we created in the startFetchTopics method will go out of scope and will be garbage collected by the .NET runtime.)

The MainForm responds to the event FetchTopicsCompleted with the event handler called OnFetchOnlineTopicsCompleted. Inside this event handler, we first check if this is an event indicating the user's cancellation of a previous request. If it is, we display a note indicating this situation in the status bar. Otherwise, we invoke a helper method called FillListView to display the fetched topics in the ListView control. Finally, we restore the buttons on the forum navigation toolbar to their appropriate state in accordance with the current page being fetched.

After reading through the information above, you should have a clear view of how the functionality of retrieving topics actually works under the hood. You should also be in a good position to explore the inner workings of other functionalities in the forum reader application on your own.

Application Setup

All the source code associated with this case study is contained in a single download file named forumreader_steveshi.zip (please find in the downloads section of this case study). It is assumed that you will typically install both the forum reader application and the supporting Web Service modules on the same box for ease of experiment. The following installation procedures are described with this assumption in mind. Of course, you may just as well install them separately on two computers to simulate a real world client/server environment.

Step by Step Installation Instructions

There are three major steps to complete the installation of the entire forum reader application:

  • Install the SQL Server Database
  • Install the Web Service Modules
  • Install the Rich-Client Forum Reader

Please note before proceeding:

  • To create the database you will need administrator privileges for the local SQL Server.
  • To create the Web site you will need administrator privileges on local machine.

Unzip the Download Package

Unzip the files in the download package to a directory on your local hard disk. We'll refer to the subdirectory named sourcecode as the installation directory in the following steps.

Install the SQL Server Database

Run the forums.sql file in the databasescript subdirectory with SQL Query Analyzer. This will create the WroxForums database including all the tables, stored procedures, primary keys, foreign key constraints, unique constraints, and some sample forum data.

Install the Forum Web Services

  • Create a virtual directory named forumservice under your default website and make this virtual directory point to the forumservice subdirectory under the installation directory. Make sure this virtual directory is created as an IIS application.
  • Open the file Webconfig.web in the sub-directory forumservice with NotePad. Locate the <appSettings> section and specify the DataConnectionString key with your own SQL Server database connection information.
  • Give it a test run by pointing your web browser to http://localhost/forumservice/service.asmx and http://localhost/forumservice/accountservice.asmx. You should see the default Web Service test page that ASP.NET provides. This indicates you have successfully completed the installation of the Web Service module.

Install the Rich-Client Forum Reader

The source code of the rich-client forum reader is right there in the installation directory, just click the solution file to view it in Visual Studio.NET.

Case Study Review

In this case study, we've walked through the major design and development aspects of a rich-client forum reader application together with its supporting Web Services. Here we summarize what we have covered during the walkthrough.

We began by identifying a potential problem with traditional discussion forum websites, especially those with extremely large volumes of forum contents, and followed up with an overview of the solution that was devised. The solution entails the development of two Web Service modules and a client-side windows application called the rich-client forum reader. The Web Service modules expose online forum contents and functionalities for the rich-client forum reader to consume. The rich-client forum reader allows the end user to view online forum contents and interact with the forum functionalities by calling forum Web Services. It also stores the retrieved forum contents in a local database for offline browsing. So broadly speaking, through this case study, you learned how to expose server database contents through Web Services and how to develop rich-client windows application to consume these services.

Next, we defined all the specific functionalities that we decided to implement in the rich-client forum reader application. These functionalities include viewing online forum categories, topics, and replies as well as login/logout the forum, posting new topics and replying to existing topics. In addition, the application must enable the user to view the locally stored forum contents.

When we started to explore the actual design and development, the first thing we saw was the database schema of both the online forum database and the client side local forum database. We explained the purpose and meaning of each table and field as well as the foreign key constraints, unique name constraints and the stored procedures. So, you were presented with a precise view of the data that the application deals with.

Next, we looked at the design of the supporting Web Services. The Web Services were designed to conform to a typical three-layer architecture, with the forum database being the data layer and the Web Service module itself containing a data access layer and a business façade layer. As this is a widely adopted architecture for Web Service design, so you can apply it to your own Web Service development projects. We also took a detailed look at the authentication scheme used in the Web Services. We leveraged intrinsic ASP.NET session state management to authenticate Web Method calls.

Next, we drilled down into the details of the design and development of the forum reader application. Our design is both component oriented and object oriented. We illustrated how the application logic is separated into four specialized components, each dealing with one particular aspect of the application. Specifically, these components are the local data access component, the business logic component, the user interface component, and the external system integration component. We then took a detailed look at the overall class model of the application. As you have seen, the class model design directly reflects the model of the component architecture. By examining this classic method of componentization and the overall class model, you could strength your knowledge about object oriented design and analysis methodology.

Next, we explained how the various GUI widgets in the main user interface are laid out. As you have seen, we used several panel controls and splitter controls to model our user interface after the style of Microsoft Outlook. We then looked at a specific UI-related coding technique that is used to marshal cross-thread function call.

We then took a look at the local data access class and illustrated the decision logic that is used when saving forum data to the local database. We use MS Access database to implement the local cache of forum data and use ADO.NET to access it.

Next, we peeked inside the four classes that make up the business logic component. We looked at how and why the entire business logic is factored out into these four separated classes, namely the CategoryManager class, the TopicsManager class, the RepliesManager class, and the AccountManager class. We then described the functional purposes of these classes. We also examined how to take advantage of multithreaded programming techniques to make Web Service calls. We explained the pitfalls with aborting a running thread and introduced a feasible workaround. As .NET simplifies multithreaded programming, so you could leverage it in your own development project to provide the end user with a more responsive application.

Finally, we wrapped up this case study by examining the detail procedure that involves a set of various objects interacting with each other in order to fulfill a particular piece of functionality.

Overall, with this case study you have a good chance to look at quite a lot of topics in the realm of Windows Forms and Web Service programming under .NET. You also have the chance to get your hands dirty with the entire design and development phase of a small yet complete application.

Further Work

Last but not least, this sample forum reader application represents a generic solution paradigm, in that the server exposes data and functionality using XML web services, whereas the rich-client application retrieves the exposed data and then consumes, stores, presents and does whatever it sees fit on the data in collaboration with the Web Services. Conceptually, any system that has its data being retrieved and locally consumed by multiple clients fits into this pattern.

Practically, this typical solution pattern could be applied to many similar problem domains and business scenarios of this nature. Take yourself as being a member of CSharpToday.com for example. I believe it is more than likely that when you find an appealing case study on CSharpToday.com, you would first hit "save-as" to save it to your local hard disk and then read it at your leisure. But, what happens when your CSharpToday collection of case studies and articles reaches into the thousands and your local directory become equally bloated accordingly? I guess at that moment you would thank them if the CSharpToday team gives you a kind of rich-client case study reader to help both aggregate your discrete, unordered files and sort them out into the original categories and shapes. This is just one case where a rich-client reader utility makes perfect sense. There's more examples along the same line, especially concerned with various online contents providers. What about a MSDN client reader hooked up with the MSDN website, a campus digital library reader.? Hopefully this case study lets you see the potential niche area of such applications.

C#Today - © 2002 WroxPress Limited


Monitoring IIS Systems

Monitoring IIS Systems
http://www.csharptoday.com/content.asp?id=1864
Article Rating: 4
Published on September 4, 2002
Abstract
In this case study, Paul Wilton looks at monitoring the IIS services. We create two programs. The first will be a Windows Service that does the actual monitoring of the IIS services. As a Windows Service can't have a user interface, a second program will be created which allows the IIS Monitoring service to be managed and important parameters set. While creating these programs we'll also be looking at how we can manage IIS programmatically using .Net. We'll see how we can find out what sites have been defined on IIS, what the properties of these sites are, and whether they are running or not. We'll see how we can restart an individual site or, as a last resort, we'll be seeing how a full re-start of the IIS services can be performed.
Case Study

I run a Windows 2000 web and e-mail server, and I was finding that when I least expected it things stopped working. Important e-mails I was expecting never arrived and only later did I spot that the SMTP service of IIS was not functioning. It was for this reason I created a .Net based IIS services monitor. At regular intervals this will check to see if important IIS services such as HTTP, FTP and SMTP services are still up and running. If they are not then I wanted them re-started and a note made of the failure.

In this case study we'll be creating two programs. The first will be the Windows Service that does the actual monitoring of the IIS services. As a Windows Service can't have a user interface, a second program will be created which allows the IIS Monitoring service to be managed and important parameters set. Of course this is not the only IIS monitoring program available, many commercial programs are available, for example Microsoft Operations Manager. The advantage with the one in this case study is its free, simple, and easily modifiable by someone with C# knowledge.

While creating these programs we'll also be looking at how we can manage IIS programmatically using .Net. We'll see how we can find out what sites have been defined on IIS, what the properties of these sites are, and whether they are running or not. We'll see how we can restart an individual site or, as a last resort, we'll be seeing how a full re-start of the IIS services can be performed, useful for those occasions when IIS is having such a bad day that nothing but a total restart will get it running again. If we wanted we could take things further and force a restart of the entire computer, but that's not something we will do in this example.

Requirements

We'll be assuming in this case study that you already have a good knowledge of C# and also of the Visual Studio.Net development environment. Obviously for the program to work you'll need a system with IIS installed, and Windows XP.

Defining The Problem

IIS Services can sometimes fail without warning, when you least expect it. Web and e-mail services are vital to companies and even a few hours downtime can result in significant financial losses or at the very least inconvenience. However it's not always possible, especially in smaller companies, to be constantly checking to see if the IIS Services are performing correctly. What is required is a program that checks at regular intervals to see if the IIS Services are functioning. By functioning, we mean not just running but able to be connected to. There are some occasions when a service is "running" but attempts to connect to it fail. The program needs to replicate the normal connection method, for example the request of a web page or start of an SMTP session.

It's not good security practice to leave a server unattended and logged onto. We therefore need our program to run in the background and to check IIS Services regardless of whether someone is logged in to the server or not.

It would be nice to minimize disruption to other IIS Services. So for example, if it were just one web site that has failed, it'd be preferable to just attempt to restart that site, not the whole web service, or even the whole IIS Service. However, if an attempt to start a single site fails, then a full re-start of IIS is the best solution as it cures most IIS issues.

Regular failure of services indicates a problem somewhere, one that needs to be investigated by the system administrator. We must therefore make a note of each failure. It might also be nice to send the system admin an SMS message or e-mail to alert him of problems, although for reasons of space we don't include this functionality in the program (though it would be very easy to add).

What we don't want our program to do is to restart sites that we have deliberately stopped. For example, for security reasons we may have stopped the administration web site - we don't want to over ride this. We therefore need to allow the system administrator to specify which sites should be monitored.

So to summarize what we want our program to do:

  • Check that IIS Services are running and able to accept the normal requests
  • To always run in the background regardless of whether a user is logged into the server
  • If an individual site has failed then make a note and start the failed site
  • If when the program next checks a site that had been started and the site is still not responding then we need to do a full IIS re-start. This unloads the whole IIS process and is very effective at getting services running again. A note should be made of the failure and total re-start.
  • To not check all sites but only those specified by the system administrator.

Defining The Solution

Our solution will consist of two parts. First we will create the IIS Service Monitor management console. This will be a standard Windows program. It will allow us to set various parameters relating to the monitor service including:

  • Monitoring Interval
  • Which HTTP, FTP and SMTP sites to monitor

For this case study, the saved settings will be stored in the Windows registry, though an obvious alternative would be a config file. The console will also allow us to stop and start the service.

Then we will create the IIS Services Monitoring service itself. When the service is started, it reads the settings defined by the console and then in intervals defined in the console it checks various IIS services by making a connection to them. Any settings stored in the registry won't be picked up until the service is re-started.

Building The IIS Service Monitor Console

First we need to open VS.Net and create a new Windows Application project:

Now within VS.Net let's layout the controls on the default form:

As the screen shot above shows, the form is fairly simple. It contains three checked list boxes, one each for the HTTP, FTP, and SMTP services. When the program is run, these list boxes will be populated with the details of all the sites running on the local server, as shown below:

The check box within the list box allows the user to select which sites they want to be monitored. In the screen shot above, I have decided not to monitor the admin website or the default ftp website.

The form also contains a text box in which we can enter the monitoring interval (i.e. how often to check the services) and three buttons: one to start the service, one to stop it, and one that saves any changes made to the service settings.

Shown in the table below are the essential property settings for the form and its controls:

Control Type | Properties
Form | Name : frmManagementConsole Text : IIS Service Monitor Management Console
Text Box | Name : txtMonitoringInterval Text : 240
Button | Name : cmdStartMonitoring Text : Start Monitoring
Button | Name : cmdStopMonitoring Text : Stop Monitoring
CheckedListBox | Name : lstServicesToMonitorHTTP
CheckedListBox | Name : lstServicesToMonitorFTP
CheckedListBox | Name : lstServicesToMonitorSMTP
Button | Name : cmdSaveSettings Text : Save Settings

Ok we have the form laid out, let's start adding some code.

References

Our project makes use of a number of references that are not included in a default Windows Application project. From the screenshot below, you can see that the project needs two references added, one to System.DirectoryServices and one to System.ServiceProcess. We do this by right clicking References, clicking the Add button, and then selecting the two references that need to be added.

The DirectoryServices reference allows us to access Windows 2000's Active Directory, and from that, the properties of IIS. The ServiceProcess reference will allow us to start and stop Windows services, such as the IISServiceMonitor service that we'll be creating later in the case study.

Our project will be using various namespaces not specified by default, so to save us typing long names let's add some using keywords at the top of the project just below the existing default ones:

using System.DirectoryServices;
using System.ServiceProcess;
using System.Text.RegularExpressions;
using Microsoft.Win32;

The Form Load

When the form is loaded, we want to populate the list boxes with the available data for each of the three IIS we monitor: web, ftp and smtp. We also want to retrieve from the registry details of which sites were selected to be monitored, and also the monitoring interval set by the user. We therefore need to add code to the Form's Load event that does this. The code is quite extensive so let's look at it bit by bit.

Starting from the top, we first retrieve the details of the monitoring interval and then use this to set the value in the txtMonitoringInterval text box:

private void frmManagementConsole_Load(object sender, System.EventArgs e)
{

RegistryKey regKey =
Registry.LocalMachine.OpenSubKey("System\\ServicesMonitor", true);

int monitoringInterval = (int)regKey.GetValue("MonitoringInterval", 60000);
monitoringInterval = monitoringInterval / 1000;
this.txtMonitoringInterval.Text = monitoringInterval.ToString();

All the settings we'll be saving will be saved, as above, under the LocalMachine\System\ServicesMonitor registry key - a key that will be accessible to the monitoring service even when no user is logged on. The monitoring interval is stored in milliseconds as this is the value required by the timer that'll be used in the IISServicesMonitor service. If no registry key exists, i.e. if this is the first time the program has been run, then 60,000 is used as the default value, i.e. 60 seconds.

When we display the interval to the user via the text box, it's in seconds, not milliseconds, and so we need to divide the value retrieved from the registry by 1000.

The next task is to obtain the previously selected sites, if any, to be monitored. Again this information is retrieved from the registry and comes in the form of a delimited string that contains the name of each site to be monitored. We'll be using the information shortly to add check marks in the list boxes for http, ftp and smtp monitoring:

String HTTPValuesToCheck = regKey.GetValue("HTTPSitesToCheck").ToString();
String FTPValuesToCheck = regKey.GetValue("FTPSitesToCheck").ToString();
String SMTPValuesToCheck = regKey.GetValue("SMTPSitesToCheck").ToString();
Hashtable sHTTPSitesToCheck = getSitesToBeChecked(HTTPValuesToCheck, Server);
Hashtable FTPSitesToCheck = getSitesToBeChecked(FTPValuesToCheck, Server);
Hashtable SMTPSitesToCheck = getSitesToBeChecked(SMTPValuesToCheck, Server);

The function getSitesToBeChecked(), which we'll create shortly, takes the delimited string obtained from the registry and splits it to obtain the individual site names and details to be checked. It returns the results in a Hashtable.

Now let's obtain a list of all the sites available for the http, ftp and smtp services and add an entry to the listboxes on our form:

m_HTTPSites = GetIISSites(Server, "W3SVC");
foreach (Hashtable Site in m_HTTPSites.Values)
{
lstServicesToMonitorHTTP.Items.Add(Site["SiteName"] + " - " +
Site["ServerComment"]);
}

// Get the details of all the FTP sites and display in a listbox
m_FTPSites = GetIISSites(Server, "MSFTPSVC");

foreach (Hashtable Site in m_FTPSites.Values)
{
lstServicesToMonitorFTP.Items.Add(Site["SiteName"] + " - " +
Site["ServerComment"]);
}

// Get the details of all the SMTP sites and display in a listbox
m_SMTPSites = GetIISSites(Server, "SMTPSVC");
foreach (Hashtable Site in m_SMTPSites.Values)
{
lstServicesToMonitorSMTP.Items.Add(Site["SiteName"] + " - " +
Site["ServerComment"]);
}

We set the global variables m_HTTPSites, m_FTPSites and m_SMTPSites to the Hashtable returned by function GetIISSites() (which we will create later). This function accesses the active directory and obtains a list of all the sites for a particular service, creates a Hashtable object for each site, and then adds it to the Hashtable object in the global variables. The site Hashtable contains the details of the site: for example, its site name, its friendly name, port number, ip address, and so on. Then a foreach loop loops through the global variables, and for each Hashtable object containing site details, it adds an entry to the form's list boxes.

We have already found out which sites were selected to be monitored last time the console was used, and we've also populated the list boxes with details of every site. Now we need to go through each item in the list boxes and see if they were sites previously marked to be monitored. If they are, then we tick the check box for that item:

// Retrieve existing selected sites and
            update listboxes
Regex getSiteNameRegex = new Regex("\\d+",
            RegexOptions.Compiled);
Int32 iListItemCounter;
String
            sSiteName;
for (iListItemCounter = 0;
iListItemCounter <
            this.lstServicesToMonitorHTTP.Items.Count;
            iListItemCounter++)
{
sSiteName =
this.lstServicesToMonitorHTTP.Items[iListItemCounter].ToString();

if
            (
            sHTTPSitesToCheck.Contains(getSiteNameRegex.Match(sSiteName).Value))
{
this.lstServicesToMonitorHTTP.SetItemChecked(iListItemCounter,
            true);
}
}

for ( iListItemCounter = 0;
iListItemCounter < this.lstServicesToMonitorFTP.Items.Count;
            iListItemCounter++)
{
sSiteName =
this.lstServicesToMonitorFTP.Items[iListItemCounter].ToString();
if
            (FTPSitesToCheck.Contains(getSiteNameRegex.Match(sSiteName).Value))
{

this.lstServicesToMonitorFTP.SetItemChecked(iListItemCounter,
            true);
}
}

for (iListItemCounter = 0;
iListItemCounter <
this.lstServicesToMonitorSMTP.Items.Count;
            iListItemCounter++)
{
sSiteName =
this.lstServicesToMonitorSMTP.Items[iListItemCounter].ToString();

if
            (SMTPSitesToCheck.Contains(getSiteNameRegex.Match(sSiteName).Value))
{
this.lstServicesToMonitorSMTP.SetItemChecked(iListItemCounter,
            true);
}
}

By default, the start service button is enabled and the stop disabled. This is fine if the service is not running, but if it is we want to make sure that the service start/stop buttons are set so that if the service is already running it can be stopped but not started again:

ServiceController serviceMonitor = new
ServiceController("IISServiceMonitor");
if
            (serviceMonitor.Status ==
            ServiceControllerStatus.Running)
{
cmdStartMonitoring.Enabled
            = false;
cmdStopMonitoring.Enabled = true;
}

We have enlisted the help of the ServiceController class and created a new object, passing the name of our service to the constructor. When we create the IIS Service Monitor it's important that we keep the same name we have used here.

We then use the Running property that returns true or false to see if the service is currently running. If it is, we disable the start button and enable the stop button.

In the load event we used three global variables that haven't yet been declared, so lets rectify that and add the following to the top of the class definition:

const String Server = "localhost";
const String TestPage = "test.htm";
Hashtable m_HTTPSites;
Hashtable m_FTPSites;
Hashtable m_SMTPSites;

You'll also notice that two constants have been declared: Server and TestPage. These will be used in the functions we're about to create.

Lets start by creating the function getSitesToBeChecked() that we used in the Load function to get a list of the sites that need to be monitored. Remember this information was obtained from the registry as a delimited string, and is passed to the function along with the ip address or name of the default server:

private Hashtable getSitesToBeChecked(String sSitesString,
String sDefualtServer )
{

sSiteString will contain the sites to be checked. A tab delimits each site. Within the site information is 5 more bits of information: site name, server comment, server, port, and header.

A colon delimits each of these bits of information. Site name is the name of the site, but it's not the friendly name we know our sites by, for example "Default", "Administration", etc. It's actually the name by which IIS knows our sites and is a number. Server comment is the nice friendly name we use for each site. Server is the name or ip address of the server. Port is the port via which the site is accessed. For a website, this is normally port 80, for ftp its port 20 or 21, and smtp is normally port 25. However we can set up sites to operate on different ports.

At the top of the function we declare four variables:

Regex getDataValuesRegex = new
Regex("(?<sitename>[^:]*):(?<Servercomment>[^:]*)") +
":(?<Server>[^:]*):(?<port>[^:]*):(?<header>[^:]*)",
RegexOptions.Compiled);
Match siteMatch;
Hashtable SitesToCheck = new Hashtable();
Hashtable SiteProperties;

The first is a Regex, this is our regular expression that will split the five bits of individual information contained about a single site. We have defined named regular expression groups for each bit of information, this will make it easier to access via code the information. So for example:

(?<sitename>[^:]*)

Specifies a group called sitename and says that the group contains any number of characters from zero upwards that are not a colon. As soon as a colon is found no more characters are matched for the group. This is then followed by:

:(?<Servercomment>[^:]*)

This specifies that following the sitename group must be a colon. Then we define another group of characters called Servercomment. Again, this is described as containing zero or more characters so long as they are not a semi-colon. The regular expression continues in the same way, of a colon followed by a group, and so on. It makes extracting the individual bits of site information very easy. Also note the regular expression option Compiled is set: this will cause the regular expression to be compiled into something more machine friendly and speeds up matching.

The next variable defined is siteMatch. When we come to split the site information string, this will be used to contain each match for a site. We'll then use it to extract the specific site data.

The variable SitesToCheck will contain a hashtable object. Each item in the hashtable will be another hashtable object, stored in the variable SiteProperties, containing information about a specific site that must be monitored.

Now all we need to do is split the sSitesString containing each site delimited by a tab, and then loop through the array returned. In each iteration we create a new hashtable object and store the site name, server comment, server, port, and header information for the site:

String[] sSites = sSitesString.Split('\t');

foreach( String sSite in sSites )
{
siteMatch = getDataValuesRegex.Match(sSite);
SiteProperties = new Hashtable();
SiteProperties.Add("SiteName", siteMatch.Groups["sitename"].Value);
SiteProperties.Add("ServerComment",
siteMatch.Groups["Servercomment"].Value);
SiteProperties.Add("Port", siteMatch.Groups["port"].Value);
SiteProperties.Add("Header", siteMatch.Groups["header"].Value);
if (siteMatch.Groups["Server"].Value == "")
{
SiteProperties.Add("Server", sDefualtServer);
}
else
{
SiteProperties.Add("Server", siteMatch.Groups["Server"].Value);
}
SitesToCheck.Add(SiteProperties["SiteName"], SiteProperties);
}

Note how we use the siteMatch, which is a Match object, to obtain the characters matched by each of the groups in our regular expression. We didn't need to name our groups, and could just have easily used numbers to refer to them, but using names makes the code that little bit more readable.

Finally we return the hashtable object containing all the details of sites currently specified to be monitored:

return SitesToCheck;

Our next function is GetIISSites(). This function returns all the sites for a particular service. Examples of services include web, ftp (file transfer), smtp (simple mail transfer protocol), pop (post office protocol) and so on. The table below lists them:

Service Name | Description
w3svc | Web Services
msftp | File Transfer Protocol service
smtp | Simple Mail Transfer Protocol (e-mail send/receive)
pop | Post office Protocol (for picking up e-mails off a server)

We don't monitor the pop service, so the function's parameter sService will contain either w3svc, msftp, or smtp. The sServer parameter in this case study will always be "localhost", but it could be an ip address or the name of the server:

private Hashtable GetIISSites(String
            sServer, String sService)
{
return IISSites;
}

In this function we hook up to the IIS metabase directory on the local machine, and then use this to extract the information we need. We'll be using the System.DirectoryServices namespace that makes accessing directories very easy. First we need to get a path to the directory tree containing the information about the service we are interested in, and get a reference to a DirectoryEntry object that will represent that node in the directory hierarchy:

String IISPath = "IIS://" + Server + "/" +
            sService;
System.DirectoryServices.DirectoryEntry
            IIS;
Hashtable siteProperties;
String sSiteName;
IIS = new
            DirectoryEntry(IISPath);
Hashtable IISSites = new
            Hashtable();
Regex isANumber = new Regex("\\d+");

IIS contains the node for a particular IIS service, and each site is represented by a child DirectoryEntry object of that IIS service node. What we now need to do is walk through each child (i.e. site) and get its details and add them to a hashtable:

foreach
            (System.DirectoryServices.DirectoryEntry Site in
            IIS.Children)
{
if ( isANumber.IsMatch(Site.Name)
            )
{
sSiteName = Site.Name;
siteProperties = new
            Hashtable();
siteProperties.Add("SiteName",
            sSiteName);
siteProperties.Add("ServerComment",
Site.Properties["ServerComment"].Value);
siteProperties.Add("ServerBindings",
Site.Properties["ServerBindings"].Value);
IISSites.Add(sSiteName,
            siteProperties);
}
}

Not all the child objects are necessarily sites, some are simply there for the operating system. How do we tell which are which? Well all the sites with a number for a name are our web/ftp/smtp virtual sites. That's why we use a regular expression to check to see if the Site.Name property contains a numerical value. Remember it's Site.ServerComment that contains the friendly name we are used to seeing in the IIS management console. If we have found one of our virtual sites, we access its properties and create a new hashtable object into which we store the properties such as Site Name, Server Comment, and Server Bindings. Note the ServerBindings contains things like port number, as each site can potentially have a different port number to distinguish it from other sites, and also a header, again to distinguish it from other sites on the same server. The hashtable object created is then added to the hashtable object referenced in IISSites.

Once we have looped through each site we then return the hashtable object references IISSites:

return IISSites;
}

That completes all the code that executes when the form is loaded so let's turn to the code for each of the three buttons on the form, starting with the cmdStartMonitoring button's click event:

private void cmdStartMonitoring_Click(object sender, System.EventArgs e)
{
ServiceController serviceMonitor = new
ServiceController("IISServiceMonitor");
if (serviceMonitor.Status != ServiceControllerStatus.Running)
{
serviceMonitor.Start();
}

cmdStartMonitoring.Enabled = false;
cmdStopMonitoring.Enabled = true;
}

If this code looks familiar, it's because it's similar to code we had in the form's load event which checked to see if our IISServiceMonitor service was running, and enable/disabled the start/stop buttons accordingly. It uses the ServiceController class to check if the IISServiceMonitor service is running, and if it's not, then start it running and then enable/disable the start/stop buttons accordingly.

The click event for the cmdStopMonitoring button is, as you'd expect, simply the reverse of the code above:

private void cmdStopMonitoring_Click(object sender, System.EventArgs e)
{
ServiceController serviceMonitor = new
ServiceController("IISServiceMonitor");
if (serviceMonitor.Status != ServiceControllerStatus.Stopped)
{
serviceMonitor.Stop();
}

cmdStartMonitoring.Enabled = true;
cmdStopMonitoring.Enabled = false;
}

The final bit of code we need to create for the console is that for the cmdSaveSettings button's click event. Essentially it's the reverse of what we did in the load event. Instead of loading the saved settings from the registry, we'll be saving the settings (i.e. monitoring interval and which sites to monitor) to the registry.

Before we change the settings we need to first stop the service, if it is actually running:

private void cmdSaveSettings_Click(object sender, System.EventArgs e)
{
ServiceController serviceMonitor = new
ServiceController("IISServiceMonitor");
Boolean doRestartAfterChanges = false;
if (serviceMonitor.Status != ServiceControllerStatus.Stopped)
{
serviceMonitor.Stop();
doRestartAfterChanges = true;
}

As we'll see later, it's when the IISServiceMonitor service is started that the initial reading of the registry and settings takes place, so to make any of our changes update a currently running IISServiceMonitor service we need to re-start it.

Now we need to create a RegistryKey object and set it to point to where we store the settings for the IISServicesMontior, i.e. in the LocalMachine\\System\\ServicesMonitor part of the registry:

RegistryKey regKey;
regKey =
            Registry.LocalMachine.CreateSubKey("System\\ServicesMonitor");
regKey.SetValue("MonitoringInterval",
Int32.Parse(txtMonitoringInterval.Text)
            * 1000);

Regex getSiteNameRegex = new Regex("\\d+",
            RegexOptions.Compiled);
Int32 itemIndex;
String
            sSiteName;
Hashtable sSiteParameters;

We need to save the details of which http, ftp and smtp sites need to be monitored. Let's start with the http. We'll do this by going through each item in the appropriate check list box, seeing if it has been checked by the user. If it has been checked, then we get the site's details from the hashtable object stored in global variable m_HTTPSites that we populated with a list of all the available http sites when the form was loaded. To get a reference to the correct hashtable object within the m_HTTPSites hashtable we need the site name, which we added to the front of the description shown in the list box. Having got the hashtable object for that site, we then add the site properties to the string we'll be saving in the registry. A tab character delimits each site and each bit of information within the site details is delimited by a colon:

// Create the string that stores HTTP sites to check
String sHTTPSitesToCheck = "";
for (itemIndex = 0; itemIndex < lstServicesToMonitorHTTP.CheckedItems.Count; itemIndex++)
{
sSiteName =
getSiteNameRegex.Match(lstServicesToMonitorHTTP.CheckedItems[itemIndex].ToString()).Value;
sSiteParameters = (Hashtable)m_HTTPSites[sSiteName];
sHTTPSitesToCheck = sHTTPSitesToCheck + sSiteName + ":" +
sSiteParameters["ServerComment"] + ":" +
sSiteParameters["ServerBindings"] + "\t";
}

if (sHTTPSitesToCheck != "")
{
sHTTPSitesToCheck = sHTTPSitesToCheck.Remove(
sHTTPSitesToCheck.Length - 1, 1);
}

The code to create the string that'll be stored in the registry is identical for the ftp and smtp services, it's just the global variable and list box that changes:

// Create the string that stores FTP sites to check
String sFTPSitesToCheck = "";

for (itemIndex = 0; itemIndex < lstServicesToMonitorFTP.CheckedItems.Count; itemIndex++)
{
sSiteName =
getSiteNameRegex.Match(lstServicesToMonitorFTP.CheckedItems[itemIndex].ToString()).Value;
sSiteParameters = (Hashtable)m_FTPSites[sSiteName];
sFTPSitesToCheck = sFTPSitesToCheck + sSiteName + ":" +
sSiteParameters["ServerComment"] + ":" +
sSiteParameters["ServerBindings"] + "\t";
}

if (sFTPSitesToCheck != "")
{
sFTPSitesToCheck = sFTPSitesToCheck.Remove(sFTPSitesToCheck.Length - 1, 1);
}

// Create the string that stores SMTP sites to check
String sSMTPSitesToCheck = "";
for (itemIndex = 0; itemIndex < lstServicesToMonitorSMTP.CheckedItems.Count; itemIndex++)
{
sSiteName =
getSiteNameRegex.Match(lstServicesToMonitorSMTP.CheckedItems[itemIndex].ToString()).Value;
sSiteParameters = (Hashtable)m_SMTPSites[sSiteName];
sSMTPSitesToCheck = sSMTPSitesToCheck + sSiteName + ":" +
sSiteParameters["ServerComment"] + ":" +
sSiteParameters["ServerBindings"] + "\t";
}

if (sSMTPSitesToCheck != "")
{
sSMTPSitesToCheck = sSMTPSitesToCheck.Remove(
sSMTPSitesToCheck.Length - 1, 1);
}

Having got out the information we need to store, let's update the registry:

regKey.SetValue("HTTPSitesToCheck", sHTTPSitesToCheck);
regKey.SetValue("FTPSitesToCheck", sFTPSitesToCheck);
regKey.SetValue("SMTPSitesToCheck", sSMTPSitesToCheck);

Finally we need to restart the IISServicesMonitor service, assuming if was running when we started the save. We first refresh, just in case somehow it's been restarted already. Then, if doRestartAfterChanges is true (i.e. the service was running before the save button was clicked and it's not running now) then we start it. When it restarts, the new settings will be read in by the service:

serviceMonitor.Refresh();

if
            (doRestartAfterChanges && serviceMonitor.Status !=
            ServiceControllerStatus.Running)
{
serviceMonitor.Start();
}
MessageBox.Show("Settings
            Updated", "Success");
}
}

That completes the coding for the IIS Service Monitor Console. Our next task is to create the service itself.

The IISServiceMonitor

First we need to open VS.Net and create a new C# Windows Service project with the name IISServiceMonitor, as shown below:

Most of the code will be going inside a new class called clsIISServiceAccess. Let's create that class now by right clicking the project name in the solution explorer and selecting "Add new item", and then selecting a blank class as shown below:

Give the class the name clsIISServiceAccess.cs.

At the top of the class let's add the private variables that will be used throughout the class:

public class clsIISServiceAccess
{
#region private varables
String m_sServer; // contains the name or ip address of the local server
String m_sHTTPTestPage; // holds the location of the test page used to
// check that http services are working
Int32 m_iMonitoringInterval; // the interval, in milliseconds, that the 3
// services are monitored
Hashtable m_FTPSitesToCheck; // contains a hashtable object for each ftp
// site to be checked
Hashtable m_SMTPSitesToCheck; // contains a hashtable object for each smtp
// site to be checked
Hashtable m_HTTPSitesToCheck; // contains a hashtable object for each ftp
// site to be checked
System.Timers.Timer m_HTTPTimer; // a timer object - used to call code to
// check that the http service is still running ok
System.Timers.Timer m_FTPTimer; // a timer object - used to call code to
// check that the ftp service is still
System.Timers.Timer m_SMTPTimer; // a timer object - used to call code to
// check that the smtp servic still running
#endregion

Each of the private class variables above is set/get using public properties, as shown in the code below:

#region Public Properties
public String Server
{
get
{
return m_sServer;
}

set
{
m_sServer = value;
}
}

public String HTTPTestPage
{
get
{
return m_sHTTPTestPage;
}

set
{
m_sHTTPTestPage = value;
}
}

public Int32 MonitoringInterval
{
get
{
return m_iMonitoringInterval;
}

set
{
m_iMonitoringInterval = value;
}
}

public Hashtable HTTPSitesToCheck
{
get
{
return m_HTTPSitesToCheck;
}

set
{
m_HTTPSitesToCheck = value;
}
}

public Hashtable FTPSitesToCheck
{
get
{
return m_FTPSitesToCheck;
}

set
{
m_FTPSitesToCheck = value;
}
}

public Hashtable SMTPSitesToCheck
{
get
{
return m_SMTPSitesToCheck;
}

set
{
m_SMTPSitesToCheck = value;
}
}

#endregion

Let's turn to the public methods for this class, of which there are six. However they break down into two types - methods that start the monitoring of a particular IIS service, and methods that stop monitoring. We will start by looking at the one that starts the monitoring of HTTP sites:

public void
            StartMonitoringHTTP()
{
try
{
if (m_HTTPSitesToCheck !=
            null)
{
m_HTTPTimer = new
            System.Timers.Timer();
m_HTTPTimer.Elapsed+=new
ElapsedEventHandler(checkHTTPRunning);
m_HTTPTimer.Interval =
            m_iMonitoringInterval;
m_HTTPTimer.Enabled =
            true;
}
}
catch (Exception
            exp)
{
EventLog.WriteEntry("StartMonitoringHTTP()",
exp.Message, EventLogEntryType.Error);
}
}

As there is no user interface, it's vitally important any exceptions are caught using a try.catch statement and the exception details written to the NT event log.

The if statement checks to see there are actually any sites to check. If there are, then we create a new timer object and store its reference in class variable m_HTTPTimer. We then add our checkHTTPRunning() method, which we create later, to be called by the timer.

Looking at the reverse, i.e. stopping monitoring:

public void
            StopMonitoringHTTP()
{
try
{
if (m_HTTPTimer !=
            null)
{
m_HTTPTimer.Enabled = false;
m_HTTPTimer =
            null;
}
}
catch (Exception
            exp)
{
EventLog.WriteEntry("StopMonitoringHTTP()",
            exp.Message,
EventLogEntryType.Error);
}
}

If the reference in m_HTTPTimer is not null, then we disable the timer to stop it, and then allow it to be garbage collected by setting m_HTTPTimer to null, which removes any references to the timer object.

The start and stop methods for ftp and smtp are identical to the http ones, it is just the global variable and method called that changes:

public void
            StartMonitoringFTP()
{
try
{
if (m_FTPSitesToCheck !=
            null)
{
m_FTPTimer = new
            System.Timers.Timer();
m_FTPTimer.Elapsed+=new
ElapsedEventHandler(checkFTPRunning);
m_FTPTimer.Interval =
            m_iMonitoringInterval;
m_FTPTimer.Enabled =
            true;
}
}
catch (Exception
            exp)
{
EventLog.WriteEntry("StartMonitoringFTP()",
            exp.Message,
EventLogEntryType.Error);
}
}

public
            void StopMonitoringFTP()
{
try
{
if (m_FTPTimer !=
            null)
{
m_FTPTimer.Enabled = false;
m_FTPTimer =
            null;
}
}
catch (Exception
            exp)
{
EventLog.WriteEntry("StopMonitoringFTP()",
exp.Message,
EventLogEntryType.Error);
}
}

public
            void StartMonitoringSMTP()
{
try
{
if
            (m_SMTPSitesToCheck != null)
{
m_SMTPTimer = new
            System.Timers.Timer();
m_SMTPTimer.Elapsed+=new
ElapsedEventHandler(checkSMTPRunning);
m_SMTPTimer.Interval =
            m_iMonitoringInterval;
m_SMTPTimer.Enabled =
            true;
}
}
catch (Exception
            exp)
{
EventLog.WriteEntry("StartMonitoringSMTP()",
exp.Message, EventLogEntryType.Error);
}
}

public
            void StopMonitoringSMTP()
{
try
{
if (m_SMTPTimer !=
            null)
{
m_SMTPTimer.Enabled = false;
m_SMTPTimer =
            null;
}
}
catch (Exception
            exp)
{
EventLog.WriteEntry("StopMonitoringSMTP()",
exp.Message, EventLogEntryType.Error);
}
}

That's all the public methods for the class completed, so now let's turn to the private methods. Let's start by looking at the checkHTTPRunning() method.

To test if each http site is working we'll be retrieving a test page called "testpage.htm", which you'll need to create and add to the root directory of each site to be monitored. Our method takes two arguments, we use neither, they are there because this method is actually called by the timer.

As with the other methods, the main code is within a try.catch clause, just in case things go wrong. After declaring two variables we then have the start of the foreach clause that'll loop through each site in the m_HTTPSitesToCheck class variable:

private void checkHTTPRunning(Object source, ElapsedEventArgs e)
{
try
{
String urlToTest;
Boolean bServiceRunning;
foreach (Hashtable HTTPSite in m_HTTPSitesToCheck.Values )
{

Now within the foreach loop we begin building up the url to the test page. First we need to check if it's port 443, the port normally used for secure communications. If it is we need to start with https rather than http:

if (HTTPSite["Port"].ToString() ==
            "443")
{
urlToTest = "https://" +
            HTTPSite["Server"].ToString();
}
else
{
urlToTest =
            "http://" + HTTPSite["Server"].ToString();
}

Although web services normally use port 80 or 8080, it is possible that a site will use a different port, as a way to distinguish it from other sites on the same server. We therefore need to check if a port number has been defined, and if so add it to our url:

if (HTTPSite["Port"].ToString() !=
            "")
{
urlToTest = urlToTest + ":" +
            HTTPSite["Port"].ToString();
}

Finally we check to see if the url needs a header. Again, some sites distinguish themselves using a unique header that we'll need to add to the url. At the end of the url we add the name of the test page:

if (HTTPSite["Header"].ToString() !=
            "")
{
urlToTest = urlToTest + ":" +
            HTTPSite["Header"].ToString();
}
urlToTest = urlToTest + "/" +
            m_sHTTPTestPage;

Now that we have our url, let's use it to attempt to open a stream and read the test page:

try
{
WebClient myClient = new
            WebClient();
System.IO.Stream response =
            myClient.OpenRead(urlToTest);
response.Close();
response =
            null;
}
catch (WebException
            httpException)
{
bServiceRunning = false;
}

We're not actually interested in getting the page's data, we simply want to attempt to read it to see if we can. If we can't, then it suggests a problem with the web service, either that it's not running or perhaps we've not put the test page in the right place.

Finally we come to the end of the foreach and call the method CheckRestartSuccess(). This method, which we create next, will check to see if the service was running, or if a previous attempt to restart the service was successful or not. It'll make a note of when a site was down, of attempted restarts, and of their success:

CheckRestartSuccess(bServiceRunning, "W3",
            HTTPSite);
}
}
catch (Exception
            exp)
{
EventLog.WriteEntry("ServerMonitor.checkHTTPRunning()",
exp.Message, EventLogEntryType.Error);
}

}

Let's look at the CheckRestartSuccess() method used in the method we just discussed, as well as by the other service checking methods. It takes three arguments: whether the site is running, whether it's a w3, ftp or smtp service, and finally a hashtable object containing the site details.

We start at the top of the method by seeing if the service is not running. If it's not running then clearly we are going to have to attempt to restart the site, or if an attempt was made to restart the site and clearly it has failed then we need to re-start the whole of IIS:

private void CheckRestartSuccess(Boolean bServiceRunning, String sService, Hashtable Site)
{
if (!bServiceRunning)
{
if ((Site.ContainsKey("FullRestartAttempted")))
{
NotifyOfServiceStatus(sService + " - " + Site["ServerComment"],
sService + " site " +
Site["ServerComment"] + " was down. " +
"ServerMonitor's attempt to restart " +
"IIS has failed.", EventLogEntryType.Error);
Site.Remove("FullRestartAttempted");
}
else if ((Site.ContainsKey("SiteRestartAttempted")))
{
Site.Remove("SiteRestartAttempted");
NotifyOfServiceStatus(sService + " - " + Site["ServerComment"],
"Attempt to start " + sService +
" site " + Site["ServerComment"] +
" was unsuccessful. " +
"Attempting to restart all of IIS.",
EventLogEntryType.Warning);
RestartServices();
Site.Add("FullRestartAttempted", false);
}
else
{
NotifyOfServiceStatus(sService = " - " + Site["ServerComment"],
sService + " site " + Site["ServerComment"] +
" was down. " +
"ServerMonitor will attempt to re-start the site",
EventLogEntryType.Warning);
RestartService(Site["Server"].ToString(),
sService + "SVC", Site["SiteName"].ToString());
Site.Add("SiteRestartAttempted", false);
}

In the second if statement we check to see if the hashtable object referenced in the variable Site contains the key "FullRestartAttempted". When a restart is attempted, a key is added to that site's hashtable object so that we can tell if the restart is successful next time the site is checked. If the site is not running now and the key "FullRestartAttempted" exists, it means a full restart of IIS was done and despite this the service is still not back up and running. In this case, we would have no option but to stop checking that site and make a note of this in the event log.

If "FullRestartAttempted" was not found, then we check to see if a restart of that site was attempted. When a site is initially found to be down, we first attempt to restart just the site, not the whole of IIS. So if the site is still down and the key "SiteRestartAttempted" has been found, it means the site restart failed and we need to attempt a full re-start of IIS.

Finally in the else statement we deal with the case where this is an initial failure and no restart of any type has been attempted. We therefore just attempt to restart the site and add a key to the site's hashtable to make a note of this.

In the remainder of the method we deal with the situation where the site is now running, but we want to check to see if last time it was checked it was down and had to be restarted. The first else if deals with the case where a full restart has been attempted and this has been successful. We make a note of this success using the NotifyOfServiceStatus() method we create shortly, and then remove the "FullRestartAttempted" key from the site's hashtable object. The second else if deals with the case where a restart of just the site was attempted and this has been successful - again we make a note of this and remove the key:

else if ((Site.ContainsKey("FullRestartAttempted")))
{
NotifyOfServiceStatus(sService + " - " + Site["ServerComment"],
"full restart of IIS was successful",
EventLogEntryType.Information);
Site.Remove("FullRestartAttempted");
}
else if ((Site.ContainsKey("SiteRestartAttempted")))
{
NotifyOfServiceStatus(sService + " - " + Site["ServerComment"],
sService + " site " + Site["ServerComment"] +
" has been restarted successfully.",
EventLogEntryType.Information);
Site.Remove("SiteRestartAttempted");
}
}

In the CheckRestartSuccess() method we used a number of methods we have yet to create, including the NotifyOfServiceStatus(), RestartService() and RestartServices() methods. Let's start with NotifyOfServiceStatus():

private void NotifyOfServiceStatus(String sService,
String sMessage,
EventLogEntryType warningLevel )
{
EventLog.WriteEntry(sService, sMessage, warningLevel);
}

You can see that this is a very simple method that simply writes the message to the event log. In fact, we could dispense with the method altogether, but it does have a purpose. Let's say we want to do more than add an entry to the event log, perhaps we want to send an e-mail or text message to the system administrator warning them of the problems. In this case this method is the ideal place to add the code to do that.

Next, let's create the RestartService() method which restarts just one site of a particular service:

private void RestartService(String sServer , String sServiceName ,
String sSiteName)
{
String IISPath = "IIS://" + sServer +
"/" + sServiceName + "/" + sSiteName;
System.DirectoryServices.DirectoryEntry IIS;
IIS = new DirectoryEntry(IISPath);
Int32 iSiteServiceStatus;
iSiteServiceStatus =
Int32.Parse(IIS.Properties["ServerState"].Value.ToString());
if (iSiteServiceStatus == 5 || iSiteServiceStatus == 6)
{
IIS.Invoke("Continue");
}
else if (iSiteServiceStatus == 3 || iSiteServiceStatus == 4)
{
IIS.Invoke("Start");
}
}

We're using similar code to access the site as we used in the console when we accessed a site and obtained it's properties. The difference here is that we'll be invoking methods rather than just accessing properties. At the top of the method we create the path to the DirectoryEntry object in the IIS metabase that we're interested in, i.e. the site that needs starting. Then we get the site's current status. It could be the site has been paused rather than actually stopped, in which case we need to invoke the continue() method. If it's stopped then we need to invoke the start() method. A service status of 5 or 6 means the site is paused or being paused. A status of 3 or 4 means the site is stopped or being stopped.

We've seen how to restart a site, now let's look at how we restart the whole of IIS. Our method RestartServices() does the full restart of all of IIS's services. The method first stops the monitoring timers, as it can sometimes take a while for IIS to restart. We don't want to start monitoring again until the restart is complete. Then we call FullIISRestart() where the actual restart is done. Finally all the timers are started going again:

private void RestartServices()
{
//
            stop Server checks;
Hashtable TimersEnabled = new
            Hashtable();
TimersEnabled.Add( "HTTPTimer",
            m_HTTPTimer.Enabled);
m_HTTPTimer.Enabled =
            false;
TimersEnabled.Add("FTPTimer",
            m_FTPTimer.Enabled);
m_FTPTimer.Enabled =
            false;
TimersEnabled.Add("SMTPTimer",
            m_SMTPTimer.Enabled);
m_SMTPTimer.Enabled =
            false;

FullIISRestart();

// start Server
            checks;
m_HTTPTimer.Enabled =
            (bool)TimersEnabled["HTTPTimer"];
m_FTPTimer.Enabled =
            (bool)TimersEnabled["FTPTimer"];
m_SMTPTimer.Enabled =
            (bool)TimersEnabled["SMTPTimer"];
}

Turning to the FullIISRestart() method, this restarts IIS by stopping the IISAdmin service, then starting it again. Stopping it will automatically cause all the services within IIS to be stopped, as they are all dependant on this service being running. It's a very effective way of clearing problems with IIS, in fact if a full restart doesn't solve the problem then it's most likely that a system admin is needed. We use the ServiceController class as our means of controlling the IISAdmin service. Then if the service is not stopped, then we stop it.

We give IIS time to shut down by using Thread.Sleep() - this puts our code to "sleep" but still gives other running programs and services the chance to run. On waking, we need to start the services again. Unfortunately we can't just start IISAdmin service and expect the other services to restart, we need to restart each service individually. However, IISAdmin must be running before any other IIS services can be started. We don't know which services are running on IIS, as this varies from server to server depending on installation, so instead we loop through each service dependant on IISAdmin and start them. The DependentServices property gives us a collection of those services, and we can easily loop through and start them:

private void FullIISRestart()
{
//
            stop IIS;
ServiceController iisAdminController = new
ServiceController("IISADMIN");
if (iisAdminController.Status
            !=
            ServiceControllerStatus.Stopped)
{
iisAdminController.Stop();
}

//
            give IIS time to shut down;
Thread.Sleep(30000);

// start
            iis;
if (iisAdminController.Status !=
            ServiceControllerStatus.Running)
{
iisAdminController.Start();
}

//
            Give IISAdmin a chance to re-start;
Thread.Sleep(10000);

foreach ( ServiceController dependantService in
iisAdminController.DependentServices)
{
if
            (dependantService.Status !=
            ServiceControllerStatus.Running)
{
dependantService.Start();
}
}
//
            give the other services a chance to restart before monitoring
            them;
Thread.Sleep(60000);
}

We've now got all the code to check that the web service is running, we need to check the ftp and smtp services. We'll be checking the services are running by creating a new session, either ftp or smtp. If we can connect to the server, start a session, and get a response from a session we started then we know that the service is running ok.

Let's start with the code that monitors the ftp service. This method gets called by the timer we started to check the ftp service. The code loops through each site, represented by a hashtable object in the m_FTPSitesToCheck hashtable. It calls the GetServerStatusCode() method and it's this method that attempts to start a session, in this case ftp session, with the server and returns the response code which will indicate if the session was successful or not. A code of 220 means a session started correctly, anything else suggests a failure. If there was a failure, then we need to restart the site, and possibly IIS, and so we call the CheckRestartSuccess() method that we created earlier to do just that:

private void checkFTPRunning(Object source, ElapsedEventArgs e)
{
String sResponseCode;
String sResponseMessage;
Boolean bServiceRunning;
try
{
foreach (Hashtable Site in m_FTPSitesToCheck.Values)
{
sResponseMessage =
GetServerStatusCode(Site["Server"].ToString(),
Int32.Parse(Site["Port"].ToString()), 10);
sResponseCode = sResponseMessage.Substring(0, 3);

if (sResponseCode != "220")
{
bServiceRunning = false;
}
else
{
bServiceRunning = true;
}

CheckRestartSuccess(bServiceRunning, "MSFTP", Site);
}
}
catch (Exception ftpException)
{

EventLog.WriteEntry("ServerMonitor.checkFTPRunning()",
ftpException.Message,
EventLogEntryType.Error);
}
}

The code that checks if the smtp service is running is almost identical, except for which global variables are used, and that the name of the service is smtp rather than msftp:

private void checkSMTPRunning(Object source, ElapsedEventArgs e)
{
String sResponseCode;
String sResponseMessage;
Boolean bServiceRunning;
try
{
foreach (Hashtable Site in m_SMTPSitesToCheck.Values)
{
sResponseMessage =
GetServerStatusCode(Site["Server"].ToString(),
Int32.Parse(Site["Port"].ToString()), 10);
sResponseCode = sResponseMessage.Substring(0,3);
if (sResponseCode != "220")
{
bServiceRunning = false;
}
else
{
bServiceRunning = true;
}
CheckRestartSuccess(bServiceRunning, "SMTP", Site);
}
}
catch (Exception smtpException)
{
EventLog.WriteEntry("ServerMonitor.checkSMTPRunning()",
smtpException.Message,
EventLogEntryType.Error);
}
}

The final method we need to add to the class is GetServerStatusCode(). We saw this used above in the ftp and smtp checking methods to see if a particular site is running. We'll be using the System.Net.Sockets namespace to make the network connections necessary for ftp and smtp communications. Starting with the function header:

private String GetServerStatusCode(String sServer, Int32 iPort, Int32 iTimeOut)
{
try
{

The function takes three parameters, sServer, which is the ip or name of the server to connect to, iPort, which is the port number (usually 25 for smtp and 20 or 21 for ftp), and iTimeOut, which is how long the code should wait for a response from the server.

The protocol in both ftp and smtp is tcp ip, and so we declare the variable tcpClient and set it to reference a new TcpClient object. This class provides the tcp connectivity, reading the information requires the NetworkSystem class:

System.Net.Sockets.TcpClient tcpClient = new
System.Net.Sockets.TcpClient();
System.Net.Sockets.NetworkStream
            networkStream;
Double Start;
Double Tmr;
byte[]
            bytes;
byte[] sendBytes = new byte[6];
String
            sResponseCode;

Next we connect to the server and open a data stream ready to read the response back from the server to the incoming request:

tcpClient.Connect(sServer,
            iPort);
networkStream = tcpClient.GetStream();
tcpClient = new
            System.Net.Sockets.TcpClient();

Now we wait for the server's response. We keep looping the while data is not available via the network stream. If we exceed the time limit then we return from the function with the value "TIMEOUT":

Start =
            DateTime.Now.TimeOfDay.TotalSeconds;

bytes = new
            byte[tcpClient.ReceiveBufferSize];

while
            (!networkStream.DataAvailable)
{
Tmr =
            DateTime.Now.TimeOfDay.TotalSeconds - Start;
if (Tmr >
            iTimeOut)
{
sResponseCode = "TIMEOUT!";
return
            sResponseCode;
}
}

If we have got this far then it means some response has been returned by the server and as it's the start of a new session, that response will be a code and message telling us if the server has successfully accepted our new session request. What we need to do now is get the response, assuming data is available:

if (networkStream.DataAvailable)
{
//
            Returns the data received from the host to the
            console.;
networkStream.Read(bytes, 0,
            tcpClient.ReceiveBufferSize);
sResponseCode =
            Encoding.ASCII.GetString(bytes);

The response comes as a byte array which we need to convert to a string using the Encoding.ASCII.GetString() method.

We don't want to leave the session hanging, so let's terminate it cleanly rather than letting it time out, as this helps free server resources:

sendBytes[0] = (byte)'Q';
sendBytes[1] =
            (byte)'U';
sendBytes[2] = (byte)'I';
sendBytes[3] =
            (byte)'T';
sendBytes[4] = 13;
sendBytes[5] =
            10;
networkStream.Write(sendBytes, 0, 6);
}

Closing a session simply involves sending "QUIT" followed by a line feed and carriage return.

Finally, if no data was available to read, we set sResponseCode to "TIMEOUT". At the end of the method we close the tcp session:

else
{
sResponseCode =
            "TIMEOUT!";
}

if (tcpClient !=
            null)
{
tcpClient.Close();
}

tcpClient =
            null;
return sResponseCode;
}
catch (Exception
            ex)
{
return ex.Message;
}
}

That completes the code for the class. We now need to add code to the OnStart and OnStop events of our Service. These events fire when the Service is first started and when it's ended. In OnStart we'll read in the settings from the registry that were placed there by the console:

protected override void OnStart(string[] args)
{
try
{
RegistryKey regKey;
regKey = Registry.LocalMachine.OpenSubKey("System\\ServicesMonitor", true);
String sServer = regKey.GetValue("Server", "localhost").ToString().Trim();
String sTestPage = regKey.GetValue("TestPage",
"test.htm").ToString().Trim();
String HTTPValuesToCheck = regKey.GetValue("HTTPSitesToCheck",
"").ToString().Trim();
String FTPValuesToCheck = regKey.GetValue("FTPSitesToCheck",
"").ToString().Trim();
String SMTPValuesToCheck = regKey.GetValue("SMTPSitesToCheck",
"").ToString().Trim();
Int32 iMonitoringInterval = (Int32)regKey.GetValue("MonitoringInterval",
300000);

Then we create a new object based on the clsIISServiceAccess we created above, and set the properties to the values just obtained from the registry:

m_IISServiceAccess = new clsIISServiceAccess();
m_IISServiceAccess.MonitoringInterval = iMonitoringInterval;
m_IISServiceAccess.Server = sServer;
m_IISServiceAccess.HTTPTestPage = sTestPage;

In the remaining code we check to see if any sites for each of the three services, web, ftp and smtp, need checking. We use getSites() to split the string obtained from the registry and return a hashtable object containing all the sites needing monitoring:

if (HTTPValuesToCheck != "")

{
m_IISServiceAccess.HTTPSitesToCheck = getSites(HTTPValuesToCheck, sServer);
m_IISServiceAccess.StartMonitoringHTTP();
}

if (FTPValuesToCheck != "")

{
m_IISServiceAccess.FTPSitesToCheck = getSites(FTPValuesToCheck, sServer);
m_IISServiceAccess.StartMonitoringFTP();
}

if (SMTPValuesToCheck != "")

{
m_IISServiceAccess.SMTPSitesToCheck = getSites(SMTPValuesToCheck, sServer);
m_IISServiceAccess.StartMonitoringSMTP();
}
}
catch (Exception exp)
{
EventLog.WriteEntry("OnStart()", exp.Message, EventLogEntryType.Error);
}

}

The method getSites() is identical in every way to getIISSites() in the console program and so is simply repeated here:

private Hashtable getSites(String sSitesString, String sDefualtServer )
{
Regex getDataValuesRegex = new Regex("(?<sitename>
[^:]*):(?<Servercomment>[^:]*):(?<Server>[^:]*):(?<port>
[^:]*):(?<header>[^:]*)",
RegexOptions.Compiled);
Match siteMatch;
Hashtable SitesToCheck = new Hashtable();
Hashtable SiteProperties;
String[] sSites = sSitesString.Split('\t');

foreach( String sSite in sSites )
{
siteMatch = getDataValuesRegex.Match(sSite);
SiteProperties = new Hashtable();
SiteProperties.Add("SiteName",
siteMatch.Groups["sitename"].Value);
SiteProperties.Add("ServerComment",
siteMatch.Groups["Servercomment"].Value);
SiteProperties.Add("Port", siteMatch.Groups["port"].Value);
SiteProperties.Add("Header",
siteMatch.Groups["header"].Value);
if (siteMatch.Groups["Server"].Value == "")
{
SiteProperties.Add("Server", sDefualtServer);
}
else
{
SiteProperties.Add("Server",
siteMatch.Groups["Server"].Value);
}
SitesToCheck.Add(SiteProperties["SiteName"],
SiteProperties);
}

return SitesToCheck;
}
}

Next comes the OnStop event, which fires when the service is stopped. We simply call the IISServiceAccess class's StopMonitoring methods for each of the services and then set the m_IISServiceAccess to null to free its reference:

protected override void OnStop()
{
//
            TODO: Add code here to perform any tear-down necessary
to stop
            your
            service.
try
{
m_IISServiceAccess.StopMonitoringFTP();
m_IISServiceAccess.StopMonitoringHTTP();
m_IISServiceAccess.StopMonitoringSMTP();
m_IISServiceAccess
            = null;
}
catch (Exception
            exp)
{
EventLog.WriteEntry("OnStart()", exp.Message,
EventLogEntryType.Error);
}
}

We have been using a global variable, m_ IISServiceAccess, which we need to define at the top of the class:

private clsIISServiceAccess m_IISServiceAccess;

That completes all of the code for this case study. However before we run the service we need to set the application up.

Application Setup

The management console needs no set up - simply run it. However there are a few steps to undertake before we can use the monitoring service.

First in the service project within VS.Net we need to add an installer to our project. From the design view click the Add Installer link, which is found in the properties box. See below:

We need to make sure that the service name is IISServiceMonitor, and also make sure it's run under the localsystem account.

We now need to let Windows know about our service. First we need to compile a release version of our monitor. Then we need to use the command line program InstallUtil.exe to register the service by running this program and passing it the name and path of our IISServiceMonitor.exe, as shown below:

Our service is now ready to use, and we can change its settings using the console we created.

Conclusion

This case study has demonstrated a lot of useful techniques for manipulating IIS, techniques that can also be extended to general Active Directory manipulation too. We've seen how we can access the properties and methods of IIS.

I've had the monitor service installed on a live server for four months with no problems, so what we have here is a reliable basic IIS monitor. However it is basic, and there are plenty of additions that could be made:

  • Full restart of the machine if a restart of IIS failed
  • Defining a specific webpage for each site
  • Sending an e-mail to the administrator should a site fail
  • Sending an SMS message to an administrator
  • Sending an administrative alert
  • Keeping an easily accessible log of problems and providing access to it from the console
  • Monitoring of the POP (Post Office Protocol Service)
  • Defining a sites level of importance. For example, we may only want to restart IIS if an important site has gone down.

The code has been designed to be easily extendable, so adding the above can be done without seriously breaking the code. If we want very sophisticated monitor then we could purchase a commercial product like Microsoft Operations Manager.

C#Today - © 2002 WroxPress Limited


Programming Web Services with C# Using SQL

Programming Web Services with C# Using SQL
http://www.csharptoday.com/content.asp?id=1850
Article Rating: 3.5
Published on July 15, 2002
Abstract
The purpose of this case study by Jeannine and Glenn Gailey is to give C# developers an overview of XML Web Services and related technologies, an explanation of why they would want to build an XML Web Service that interacts with SQL Server, and to walk through a demonstration of how to build just such a web service.
Case Study

The purpose of this case study is to give C# developers a brief overview of XML Web Services and related technologies, an explanation of why they would want to build an XML Web Service that interacts with SQL Server, and to walk you through a demonstration of how to build just such a web service. For the purposes of this case study, the service will be fairly simple: a web service, built with Visual Studio .NET using C# that allows you to request an XML document that represents a user's resume data stored in a relational SQL Server database.

System Requirements

To get the most out of this case study, you should already be familiar with general C# syntax as well as with .NET programming practices and the Microsoft .NET Framework. You should also have at least some general knowledge of XML, Structured Query Language (SQL), and relational databases.

In order to recreate or run the code included with this sample, you will need to have the following installed on your machine:

  • Microsoft SQL Server 7.0 or later
  • Microsoft Visual Studio .NET (tested with build 9466)
  • Microsoft Data Access Components (MDAC) version 2.7 or higher (installed by Visual Studio .NET)
  • Microsoft .NET Framework (installed by Visual Studio .NET)

Introduction

The first section of this case study will give you an overview of XML Web Services and the underlying technologies, including brief overviews and samples, as well as .NET Web Services architecture. The second section will give an overview of the design of the resume server Web service, including why and how you would use SQL Server 2000 for an XML Web Service. We will also discuss the benefits of SQL Server 2000 over SQL Server 7 (or earlier) when developing XML Web services. The third section will walk you through how we built the resume server service, including showing C# code examples, designing the database, and deploying and testing the service using Visual Studio .NET.

It is safe to say that the Internet has forever changed the way in which many individuals do business and conduct their affairs. Why call your stockbroker to find out how your portfolio is doing when you can find this information online? No need to trudge down to the ticket office or stay on hold forever trying to get tickets for your favorite sports team; now you can look up the home games schedule and purchase tickets for the seats of your choosing with a few clicks of your mouse. Many people now pay their bills and transfer funds at a Web site rather than handing a deposit slip to a cashier. No longer a novelty, accessing information on demand via the Internet has become a regular event. Now, with personalized Web portal pages that are configured for users' preferences, users are becoming accustomed to a more individualized Internet experience, while at the same time investing more and more in non-PC devices that have access to the Internet.

What Are Web Services?

Web services are essentially a new and efficient way for companies to provide data that is stored in their secure and proprietary databases to their customers (whether other XML Web services or end-user client applications). For example, Google now provides an XML Web service that allows third party web sites to directly search the data in its database.

The vision of Microsoft, IBM, Sun and many other companies is to use XML Web services as a way to provide a discoverable, standards-based architecture to loosely coupled applications in a decentralized environment. Web services are essentially self-describing applications that can be published, located, and invoked across the Internet. These services can provide functionalities that range from simple requests to complicated business processes. Access to these functionalities is provided by XML-based messages defined by the Simple Object Access Protocol (SOAP). Since a Web service is discoverable and accessed via standard protocols (like XML and HTTP), it can be used by other applications or Web services that "speak" to these protocols without requiring that these clients possess detailed knowledge about the service's underlying implementation.

For example, when a client accesses a Web service method, it only knows about what is needed to invoke the method. The client has no idea whether the request is serviced by an underlying COM-based service running on a Windows NT Server or by a JAVA-based service running on a Sun Solaris server. Furthermore, since SOAP is a text-based, XML-based protocol, the method call can be made through the firewall using HTTP on port 80.

Since XML Web services use SOAP to provide access to that service's functionalities via HTTP, a client application that consumes an XML Web service can be running on any device or on any platform (Linux, UNIX, Solaris, and others) with internet connectivity. This means that other non-PC devices, like cell phones, PDAs, Tablet PCs, and even web-enabled appliances, can consume SOAP messages and transform the encapsulated information as needed for a particular device. For example, consider that a certain XML Web service provides stock information (stock tickers were, in fact, some of the earliest consumer XML Web services). The SOAP message containing the requested stock price information would be the same whether it was requested by an application running on a PC or on a cell phone.

Architecture

In the past, the interaction between applications on different platforms has been a tightly-coupled process. While a server-side component might be designed to work well with a particular client, reusing this component to service a different client application usually involved learning the programming interface of, and writing additional custom code to handle, each new application and platform. This interaction model is cumbersome and labor intensive at design-time as both parties must agree on things like data types, message formats, and bindings.

Windows DNA Architecture

Many of you will be familiar with the traditional Windows DNA architecture as depicted in the following diagrams. The Windows DNA architecture is an n-tier solution based on the tightly coupled Component Object Model (COM) and Distributed Component Object Model (DCOM) technologies. In this distributed architecture, when a DCOM client needed to call a method on a remote DCOM server, the COM service on the client created a proxy object for "marshaling" the procedure call. This marshaling process involved serializing data from the client into a bit-stream of binary data that was de-serialized at the other end. The following diagram shows the traditional n-tier application architecture:

XML Web Services Architecture

Putting a new twist on Windows DNA and other traditional client-server architectures, XML Web services employ a message-based call structure. This enables the exchange of data between heterogeneous systems that are loosely coupled. Rather than having to marshal a call into a stream of platform-specific binary data, a SOAP message is serialized as a standard XML stream. XML Web services can be implemented on any platform and adhere to public standards being developed by organizations such as the World Wide Web Consortium (W3C). XML Web services allow server applications to provide their functionalities via the Internet (even through firewalls) to other applications, regardless of the platform or device on which the client application is. Below is an example of this architecture:

Web Services Implementation

Implementing an XML Web service requires the following high-level tasks, all of which are greatly simplified by developing your service in one of the number of emerging integrated development environment (IDE) that support XML Web services:

  • Code your application. You can either write a new managed code Web service application or port an existing COM-based application.
  • Setup a SOAP listener. In order to respond to SOAP requests, you need to be able to properly receive and handle them. The listeners parse the incoming SOAP messages and call the appropriate objects. Listeners must be active at both endpoints of the SOAP communication. Microsoft provides both an ASP and an ISAPI listener. The ISAPI listener gives you better performance, but the ASP listener allows you to easily run custom processing on incoming SOAP messages.
  • (Optional) Develop discovery and description mechanisms for your service. We will discuss discovery and description later in this chapter.

The Role of Open Standards

XML Web services can be built in a variety of ways because they are by nature flexible, and can be implemented on whatever platform the architects are most comfortable working with. In this age of distributed computing, people expect applications to "talk" to each other, but seamless communication usually involves writing lots of extra code. Instead of wasting time writing this customized code to handle communications with various remote interfaces, you can spend more time writing the features that your users want.

The difference between developing an XML Web services-enabled application and developing a traditional application that uses remote procedure calls on COM or DCOM is the manner in which an object's interfaces can be accessed. Prior to XML Web services, communication between applications required writing custom code to handle incoming data streams of various formats on varying platforms. Instead, an XML Web service exposes its object's interfaces through open Internet protocols. That way, instead of an implementation that requires that remote procedure calls be made via some platform-specific binary protocol, developers can access remote objects using a ubiquitous Internet messaging standard. Although there can be a performance impact to processing XML instead of binary formats, for many, the interoperability potential outweighs these costs. A Web service can be seen as made up of the following four layers:

  • Service Transport Layer - uses standard Internet transport protocols like TCP/IP and HTTP and provides message transport between applications.
  • XML Messaging Layer - defines a message using an XML message-based protocol, like SOAP, to encapsulate application information.
  • Service Description Layer - describes the XML-based interface using a protocol like WSDL to simplify consumption of a web service.
  • Service Discovery Layer - lists XML Web services in a centralized, common registry, using a protocol like UDDI, so that applications can easily find the appropriate service that they need.

The following sections introduce the technologies that are the building blocks for XML Web services. All of these technologies are either approved or proposed as standard Internet technologies.

Internet Communication Protocols

Traditionally, a number of protocols could be used to connect to and transfer data via the Internet, including:

  • Transmission Control Protocol/Internet Protocol (TCP/IP) - the ubiquitous, stream-oriented protocol that is the base protocol for all internet messaging
  • User Datagram Protocol (UDP) - for simple, packet-oriented communication
  • Hypertext Transfer Protocol (HTTP) - for transmitting text-based markup via TCP/IP. Hypertext is transported through firewalls via port 80.
  • Secure Socket Layer (SSL) - an authentication and encryption protocol for communication over TCP/IP. The HTTPS protocol is essentially HTTP that uses SSL and connects to a port other than 80.
  • File Transfer Protocol (FTP) - for transferring binary files

Like most Internet applications, XML Web services use HTTP as its primary message transport. This allows messages to be transmitted through the firewall on port 80, avoiding the hassle of having to open new ports to enable the traditional binary communication used by DCOM or the Common Object Request Broker Architecture (CORBA).

In addition to these traditional Internet communication protocols, XML Web services have spawned two new XML-based message protocols, namely Simple Object Access Protocol (SOAP) and Direct Internet Message Encapsulation (DIME). We'll cover SOAP and DIME shortly. First, let's look some more at XML.

XML

The beauty of XML is that it is a self-describing, data-centric format. This means that all the information that an application needs in order to use the data in an XML document is supplied by the XML mark-up. As a result, an XML data stream can be easily interpreted and rendered appropriately, regardless of the specific client device accessing the information. For example, when an incoming XML document contains data for an e-mail message, that XML can be parsed, rendered, and formatted either for display as an HTML page or for a handheld device like a PDA or cellular phone. Also, because XML is a simple text based format, it can be easily transmitted using HTTP.

SOAP

The new (but already entrenched) Internet protocol known as SOAP is an XML standards-based protocol that enables XML Web services by prescribing a standard delivery mechanism for XML-based message payloads.

SOAP is officially defined by the W3C (http://www.w3.org/TR/SOAP?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq) as a protocol that "provides a simple and lightweight mechanism for exchanging structured and strongly-typed data between peers in a decentralized, distributed environment using XML. SOAP does not itself define any application semantics such as a programming model or implementation specific semantics; rather it defines a simple mechanism for expressing application semantics by providing a modular packaging model and mechanisms for encoding application defined data. This allows SOAP to be used for a large variety of purposes ranging from messaging systems to remote procedure call (RPC) invocations."

A SOAP message is made up of the following:

  • An envelope that defines a framework for describing what is in a message and how to process it
  • A set of rules for how to handle different data types
  • A convention for representing remote procedure calls and responses.

As an open and extensible protocol, using SOAP as the remote access mechanism has a number of benefits over the traditional binary distributed protocols like DCOM and CORBA, including that all applications that access the XML Web service using SOAP can use the same wire protocols. This enables applications to communicate, without the need for custom binaries, run-time libraries, or other platform-specific information that has plagued cross-platform data transfer in the past. SOAP messages are also self-describing, where the header of the message carries all of the information required to process the message at the other end. Lastly, since SOAP messages are plain-text XML documents, they can be transmitted through firewalls via HTTP and even SMTP.

The following is an empty SOAP message that has a root envelope element and its two child elements, header and body:

<Envelope>
<Header>
The SOAP header is an optional element that contains data about what the message contains and how to process it.
</Header>
<Body>
The SOAP body is a required element that contains the XML data needed to access the interfaces of a specific object
</Body>
</Envelope>

The schema for a SOAP message specifies a root Envelope element with child elements named Header (which is optional) and Body (which is required). Because the SOAP message shown in the example follows the specification schema for SOAP, it makes sense to define the namespace "s" by reference to the URI of the W3C Schema for a SOAP envelope at http://schemas.xmlsoap.org/soap/envelope/. However, because namespaces are used only to promote the uniqueness of objects in a document, the namespace does not necessarily have to reference a schema location.

Discovery and Description

Two key concepts of XML Web services are discovery and description. In order for applications to consume XML Web services, they need to be able to locate them and know how to interact with them. A number of XML standard technologies have been proposed that will make automatic discovery and consumption of XML Web services a reality. These include the following:

  • Universal Description, Discovery, and Integration (UDDI)
  • Discovery of Web Services (DISCO)
  • Web Service Description Language (WSDL)

UDDI and DISCO are technologies that essentially "publish" XML Web services so that they can be "discovered" and used by applications. WSDL is an XML format that describes the exposed functionality of an XML Web service.

DISCO is an XML Web service discovery method that is supported today by Microsoft Visual Studio .NET that allows developers to locate and download WSDL documents from a known URL. DISCO files are simple XML documents that contain the location of WSDL description documents for a given XML Web service. Visual Studio .NET is able to parse a DISCO file to determine the location of the specified WSDL files. In the future, DISCO will likely be replaced by emerging Web service standards, like WS-Inspection.

WSDL

WSDL, an XML Web services description standard supported by the W3C, describes how to access an XML Web service. The role that WSDL plays in describing XML Web services is in many ways comparable to what the Interface Definition Language (IDL) does to describe DCOM and CORBA services. A WSDL document describes the following parts of the service:

  • Data type definitions -as defined by the service's schema documents (XSD).
  • Messages formats-defines the required components of the SOAP messages supported for communicating with the service, particularly header elements.
  • Port Types -defines the various input and output operations supported by the service.
  • Bindings -defines transport protocols and detailed message information relating to the operations available on a port.
  • Ports -defines access to the service as a combination of a binding and a network address.
  • Service -defines the name of the XML Web service.

The following sample shows the structure of an empty WSDL document:

<?xml version="1.0" encoding="utf-8"?>
<definitions namespace_definitions>
<types>.</types>
<message name=message_name>.</message>
<portType name=portType_name >
<operation name=operation_name>.</operation>
...
</portType>
<binding name=binding_name >
<operation name=operation_name>
<input name=input_name>.</input>
<output name=output_name>.</output>
...
</operation>
...
</binding>
<service name=service_name >
<port name=port_name>.</port>
...
</service>
</definitions>

A sample from the WSDL file for the resume service sample can be found in Appendix A.

Securing Web Services

As great as all of this discoverability and interoperability sounds, don't forget that XML Web services transmit data, your client's data, through the wild unknown of the Internet. While SOAP messages excel at providing a standardized interface for remotely accessing Web services, they do present an additional vulnerability. Unlike traditional protocols like DCOM and CORBA that communicate via binary protocols, SOAP messages are transmitted as a stream of plain-text XML over HTTP. Because of this, it is even more important to design security into your XML Web service.

When planning for Web service security, it is important to remember two things: 1) web services are exposing your company's valuable data, and 2) they are accessed via the Internet, which is an inherently insecure place. Based on discussions in the developer community, it is clear that security concerns are one of the major roadblocks to the wide-spread adoption of Web services, due in part to Microsoft's track record on security and the complexities of implementing secure multi-tier systems. Even though most security experts feel that Microsoft's .NET platform provides sufficient security features, Web services still need to be designed from the ground up to use those features and to implement security best practices, not just whatever is easiest to bring the service online.

Best Practices

Although Microsoft has proposed new Web services standards that provide much needed security features for XML Web services, aside from using XML encryption SOAP does not define any security for messages. Since this is the case, security for XML Web services must be provided either by the service, by the message transport, or the Web server itself. Since in many ways XML Web services have much in common with today's Web-based applications (i.e. HTTP transport and hosted by Web servers), many Internet security principles will apply to securing services. Securing an XML Web service can be achieved by using some combination of the following techniques:

  • Apply good Web security techniques, like validating all user input. Fortunately, SOAP messages can be strongly typed, which makes it easier to catch malicious data.
  • Operate the services based on the principle of least privilege.
  • Implement custom message-based security. For Web services that do not allow anonymous access, you must require pass authentication and encryption information, like client certificates or Kerberos keys, in the SOAP header that could be required to access your service
  • Use transport-level security techniques like SSL, which work as long as the message does not require any additional routing at which point the security context of an SSL message is lost.
  • Use encryption. SSL can encrypt an entire SOAP message across the wire, and ASP.NET supports encryption for SOAP at the method (root node) level, which will be defined by the forthcoming SOAP encryption standards.

The security infrastructures for an XML Web service will differ depending on the sensitivity of the data that is provided by the service, and as usual, there is a tradeoff between security, code complexity, and performance. For example, any transaction using SSL will be slower than over HTTP.

Authentication and Authorization

Although XML Web services are based on open standards and designed to be discoverable, this does not mean that they should be accessible by unauthorized clients. Why give data away for free because you are unable to control who is accessing your Web service methods? Also, protecting your Web service with adequate authentication and authorization schemes just adds an additional layer of complexity for potential hackers to get through. In fact, a quick way to see rapid decreases in the performance of your servers is to provide a useful and unrestricted XML Web service, like a stock ticker or sports reporting service. It will not be long until some enterprising developer has created a client application that uses your service. When such an application is poorly written, i.e. frequent requests and improper use of caching, or when such an application becomes widely distributed you will find that your initial resource estimates for your service were way too low.

Another possible way to implement authentication for your XML Web service is to use a third-party solution like .NET Passport. The .NET Passport service does the work of user authentication for you. Users can access your service by logging in to .NET Passport and getting an encrypted cookie with their Passport Unique Identifier (PUID). This cookie can then be decrypted and used by your service to access user-specific data. While today .NET Passport only solves the end-user authentication problem, future releases of this service will be based on the Kerberos v5 encryption protocols and Web service security standards that will extend .NET Passport into the realm of inter-service authentication and trust broker. For more information, see http://www.passport.com/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq.

SQL Injection Attacks

Without going into great detail, it is possible under certain conditions to take advantage of SQL syntax to inject potentially damaging statements into database queries that are being executed by an internet application or XML Web service. For example, if a service takes a username variable to create the query SELECT * FROM myTable WHERE username='Joe', maliciously passing the variable as Joe '; DROP TABLE myTable -- can do a lot of damage. These vulnerabilities can be avoided by assuming that you will have malicious clients and practicing good design principles for your database-based services. Techniques for preventing SQL injection attacks include:

  • Never allowing clients to supply custom SQL statements, this is just asking for trouble.
  • Validating incoming data and escape out any dangerous SQL characters like single quotes and double dashes.
  • Avoid building queries by concatenating strings. Stored procedures should be used instead.

For an excellent discussion of SQL injection, see http://www.nextgenss.com/papers/advanced_sql_injection.pdf?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq.

Denial of Service Attacks

One of the biggest vulnerabilities of Web services, even one with good authentication and authorization schemes, is the denial of service (DoS) attack. A DoS attack can be launched by having a few computers send a barrage of multiple-valid SOAP messages that require database lookups. Even if the attacker cannot be authenticated, the SOAP request must still be handled and a response must be generated This type of attack can be mitigated somewhat by enabling session state for your service and using state to prevent clients from making concurrent requests. Although this may prevent unauthorized requests against any of your more cumbersome business logic, a large enough attack may still shut down the service with login request alone. Enabling session state also can have negative performance implications as well.

One way to head-off these malicious requests is to apply custom logic, like ISAPI filters, to intercept malicious packets at the Web server. Another emerging solution to this type of attack is using XML filters to validate and inspect the contents of SOAP requests at the firewall.

Web Service Security Standards

In the future, we can look forward to emerging standards from the Web Services Interoperability Organization (WS-I), including Microsoft, IBM, and others, that will define security for XML Web services. The foundational security specification will be WS-Security, which defines an end-to-end message security protocol that supports a wide variety of security models. Following this specification, which is currently in the beta-release stage, there will be other protocols, which will further clarify how federation works, including WS-Trust, WS-SecureConversation, WS-Authorization and WS-Federation. As we mentioned earlier, standards are being developed jointly by in the WS-I that will support rich authentication and authorization protocols for Web services. For more information on emerging Web services standards, see http://www.ws-i.org/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq.

We will discuss the security considerations of the resume service in the following sections. For an excellent treatise on creating secure applications and services, you will want to read Writing Secure Code by Michael Howard and David LeBlanc.

Overview of the Resume Service

In the following section, we will discuss the general design of our resume service. We will also look at why Visual Studio .NET, C# and SQL Server 2000 make a good combination when designing and implementing XML Web services using Microsoft .NET.

Resume Service Scenario

The general scenario for this simple Web service is one where a company wants to provide a central storage service for the resumes of job seekers. Prospective employers and recruiters could subscribe to this service and then job seekers would have a central repository where they could submit their resume once and have it be accessible by the various service subscribers. Employers and recruiters could then search the database for specific employee attributes (like previous employers, education, skills, etc) via Web service search methods published in the WSDL file and accessible via SOAP.

Resumes of interest to an employer can then be requested from the service based on a unique ID, and the resume data will be returned in a structured XML document, which can be transformed via XSLT into a Web page, print document, report, or whatever best meets the needs of the employer.

We will only be covering a simplified portion of this complex scenario in our sample code. We will be using a simplified database schema and only showing the Query method, which allows clients to search parts of the database for text strings, and the GetResume and GetResumeXML methods, which return a resume as an XML document based on the supplied UserID.

Web services and databases

XML Web services are all about transmitting data in a usable, standard format that relies on SOAP and XML. In today's scalable enterprise application, data is stored in relational databases like Oracle 9i, IBM DB2, Microsoft SQL Server, and others. These applications are designed for the fast, efficient storage and manipulation of large volumes of data. Any commercial web site that you visit is ultimately accessing data stored in a relational database, and since the purpose of XML Web services is to provide data, web service data will also reside in such a database. Although it is possible to conceive of and develop other Web services that can in essence capture data from a data-stream, recordset, or even another web service (like applications that do custom searches of Amazon), even this data initially came from a database.

Although there are many proponents of storing and accessing data using XML documents, relational databases still have a number of advantages over structured document-style storage. Because relational databases allow you to "normalize" or store data in many simplified tables, these "pieces" of data can be readily built, using a SQL statement, into many different kinds of datasets, including XML documents. Also, storing data relationally improves its reusability and scalability, because instead of applications holding locks on an entire large storage document, locks are held on individual tables for a shorter period of time, which allows other applications to access the remaining tables. This is one of the main reasons why relational databases are generally considered to provide faster data access than structured storage mechanisms.

Another major advantage of using a relational database is that most of the enterprise database products use stored procedures. Stored procedures are chunks of SQL code that are maintained by the database and can be executed and cached. Since stored procedures can contain SQL statements to query the database of modify data, they provide an excellent way to restrict access to the database. Like component methods, arguments (known as parameters) can be passed to stored procedures, thus eliminating the risk of allowing client or business tier components to craft SQL statements on the fly. We discuss some of the additional security benefits of databases later in the security section.

Why Use C# and SQL Server for Web services

Even though Microsoft has designed the Common Language Runtime (CLR) - the foundation of the .NET Platform - and Visual Studio .NET to be used with all .NET enabled languages (including new twists on old favorites like PERL.NET and COBOL.NET), C# is being touted by Microsoft as the language of choice for creating Web services in .NET. By being less "user friendly" than Visual Basic .NET (its major .NET rival) C# is believed by many to produce cleaner intermediate language (IL) code that allows for better scalability of services and server-side components. In fact, Microsoft developed both Visual Studio and the .NET Framework almost exclusively using C#.

SQL Server 2000 is a good choice for supplying the database back-end for this type of scenario for the following reasons:

  • It runs on Windows and integrates well with Microsoft's .NET CLR.
  • It offers is a scalable backend database solution that allows you to execute stored procedures.
  • It can return data from a query as an XML document using FOR XML directive.
  • SQLXML 3.0, a middleware component that extends the XML capabilities of SQL Server, enables you to create XML Web services and apply XML templates to relational data.

To keep things more straightforward, we chose not to use SQLXML for the resume service sample, but if we wanted more control over the format of the XML returned by the service, SQLXML would be the way to go.

Note: if you choose to use an earlier version of SQL Server when developing your XML Web service, you will be passing-up on many of the key Web service features, including XML support and the FOR XML clause.

Security for the Resume Service Sample

Although security for Web-based services and applications is a critically important topic, this application is not intended to teach Web services security, as this service runs under anonymous authentication and we do not have an authentication scheme for the service. We have however designed security into the infrastructure by:

  • Accessing SQL Server via a login granted to the ASPNET account. This login is then assigned to the restricted db_datareader role in the ResumeServer database.
  • Granting Execute permissions for the ASPNET login in SQL Server only to the stored procedures used by the application.
  • Escaping out characters in query strings that might be used by a hacker to inject SQL code into the stored procedures.

Architecture of the Resume Web service:

The resume service sample has been designed based on the traditional Microsoft .NET n-tier, scalable architecture. Resume data is stored in relational tables in Microsoft SQL Server 2000. Web service components, coded in C#, search and retrieve data from the database by calling stored procedures in SQL Server via the ADO.NET managed provider for SQL Server. The Query procedure provides search functionality for the service, and the GetResume procedure that is featured in this application, returns resume data as an ADO .NET dataset. This dataset is serialized to a SOAP message by an ASP.NET process running on the IIS server. Since Microsoft's .NET Web services are strongly typed, a .NET aware Web service client is able to receive and parse the message into an instance of the dataset on the client machine. To support non-.NET aware clients, our application has an additional method GetResumeXML that returns the raw XML data as a string instead of as a dataset object. In order to use this raw XML data, the client will have to parse the document and extract the desired data. These methods will be described in detail in the following sections.

In order to call these methods, a client will have to send a properly formatted SOAP request message, which is passed by our Internet Information Service (IIS) ISAPI filter to the Web service components. Visual Studio .NET generates the DISCO and WSDL documents needed for clients to discover and understand the service.

The following diagram represents the architecture for our resume service:

In the .NET world, the process for accessing the resume service would look like this:

  1. The Web services consumer discovers a Web services provider (UDDI or some other mechanism)
  2. Consumer reads a description file for the Web service that tells it what methods, data types, ports, and bindings are available on the service.
  3. Consumer sends a request for a Web service method to the service via SOAP
  4. ISAPI running on the provider's Web server (ASP.NET) receives the SOAP request, parses it, and invokes the desired method on the middle-tier (ideally this is coded in C#)
  5. The Web service method calls down to the database using the ADO.NET data provider which requests that SQL Server run the specified stored procedure.
  6. SQL Server runs the procedure that returns the requested data via ADO.NET.
  7. The middle-tier application takes the data, processes it (if necessary), and passes it to ASP.NET where it is serialized into a SOAP message and sent back to the requesting consumer.

Sample SOAP Messages

As we mentioned earlier, SOAP messages are used to send requests to and receive responses from an XML Web service. The following XML documents show examples of these SOAP messages. In the case of our sample, we are not using any of the properties of the SOAP header, so these messages only include <body> elements.

Query Request Message

<?xml version="1.0" encoding="utf-8"?>
<soap:Envelope xmlns:xsi=http://www.w3.org/2001/XMLSchema-instance?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq
xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
<soap:Body>
<Query xmlns="http://www.webbish6.com">
<queryString>Cincinnati</queryString>
<queryType>2</queryType>
</Query>
</soap:Body>
</soap:Envelope>

Query Response Message

<?xml version="1.0" encoding="utf-8"?>
<soap:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
<soap:Body>
<QueryResponse xmlns="http://www.webbish6.com">
<QueryResult>
<int>1</int>
<int>2</int>
</QueryResult>
</QueryResponse>
</soap:Body>
</soap:Envelope>

Coding the Sample Application

The following section describes the process for developing the resume service, starting with defining the database schema.

Database Schema

In order to simplify the database design, the database for this sample (named ResumeServer) has been designed to allow each user to have only one resume stored in the service. Each user has related employer and education information. It contains the following user tables:

  • User - this table includes columns for UserID (the primary key that is used by the stored procedures to select a single user resume), first name, last name, and email address. All other tables relate back to this table, so that the dataset will only contain data related to a given user. This table might also be expanded to add columns for other user-based resume information like awards, skills, interests, and the like.
  • Position - this table stores information about positions that a user has had, and includes columns for position start and end date, title, description, and name and phone number of references. Each user can have multiple entries in the Position table, but a position must relate to only one employer. Position has a foreign key relationship to User on UserID and to Employer on EmployerID.
  • Employer - this table stores information about employers, including name, type of business, address and if the employer is a client of the service. The role of this table could be greatly expanded in a real world scenario to store additional information for companies that are clients of the service. Since a single employer will have many related positions and users, Position relates to Employer by a foreign key on EmployerID.
  • Education - this table stores information about a user's education, including type of degree, major, date completed and any comments. Education relates to User by a foreign key on UserID.
  • Major - this is a lookup table that maps the integer value in Education.MajorID to a nvarchar(50) text description with the name of the major.
  • Degree - this is a lookup table that maps the integer value in Education.DegreeType to a nvarchar(50) text description with the name of the type of degree.
  • Institution - this table stores information about where the user obtained the related education. It includes columns for name, city, state and type of institution.

The following diagram shows the table relationships for the ResumeServer database:

Although the database schema for this sample application is fairly simple, even more richness could be added to support enhanced functionalities. For example, if you wanted users to be able to store more than one resume in the service, then you could add a Resume table, relating to User, that would function as the User table in this sample. In this case, you would query based on a resume ID value rather than a user ID.

Although authentication has not been included in this sample, any authentication method (whether a custom scheme or something out-of-the-box like Microsoft Passport .NET) will require changes to the User table to store authentication information ranging from username and password data to PUIDs in the case of .NET Passport.

Stored Procedures

Since data in this database has been fairly well normalized, the aggregated data that relates to a given user is retrieved via a series of SELECT SQL statements. The complete resume is returned by executing the following three stored procedures:

  • spr_GetUserInfo
  • spr_GetEmployerInfo
  • spr_GetEducationInfo

Each of these procedures returns a related section of the resume data. As we will see, all three of the result sets from these procedures are added to the dataset before it is returned by the service. In addition, the service can support any number of string-based query procedures via the Query Web method, with one for each custom query procedure written for the service. This sample uses the spr_QueryEmployer and spr_QueryInstitution stored procedures to query on specific columns of the Employer or Institution table respectively for text that matches the supplied string.

Now, let's take a closer look at these SQL Server stored procedures.

spr_GetUserInfo Procedure

The spr_etUserInfo procedure returns information from the User table. The following SQL code creates this stored procedure:

CREATE PROC spr_GetUserInfo
@UserID int
AS
IF EXISTS(SELECT * FROM [User] WHERE UserID=@UserID)
BEGIN
SELECT [User].FirstName, [User].LastName
FROM [User]
WHERE [User].UserID=@UserID
END
ELSE
SELECT 'Invalid User ID' AS [Error] WHERE 1=1
GO

If no data exists for the passed @UserID value, then a null dataset will be returned.

spr_GetEmployerInfo Procedure

The spr_GetEmployerInfo procedure returns information on former positions and employers for a given user. The following SQL code creates this stored procedure:

CREATE PROC spr_GetEmployerInfo
@UserID int
AS
IF EXISTS(SELECT * FROM [User] WHERE UserID=@UserID)
BEGIN
SELECT Employer.EmployerName, Employer.City, Employer.State,
Position.Title, Position.[Description], Position.StartDate,
Position.EndDate FROM [User]
INNER JOIN Position ON [User].UserID=Position.UserID
INNER JOIN Employer ON Employer.EmployerID=Position.EmployerID
WHERE [User].UserID=@UserID
ORDER BY Position.EndDate
END
GO

As in spr_GetUserInfo, we don't return schema information if an empty row is returned.

spr_GetEducationInfo Procedure

The spr_GetEducationInfo procedure returns information on education degrees received and institutions attended by a given user. The following SQL code creates this stored procedure:

CREATE PROC spr_GetEducationInfo
@UserID int
AS
IF EXISTS(SELECT * FROM [User] WHERE UserID=@UserID)
BEGIN
SELECT Institution.InstitutionName, Institution.City, Institution.State,
Degree.[Description] AS 'Degree', Major.[Description] AS 'Major',
Education.DateCompleted FROM [User]
INNER JOIN Education ON [User].UserID=Education.UserID
INNER JOIN Major ON Education.MajorID=Major.MajorID
INNER JOIN Degree ON Education.DegreeType=Degree.DegreeType
INNER JOIN Institution ON
Institution.InstitutionID=Education.InstitutionID
WHERE [User].UserID=@UserID
GROUP BY Institution.InstitutionName, Education.DateCompleted
ORDER BY Institution.InstitutionName, Education.DateCompleted DESC
END
GO

Again, when an empty row is retuned, this procedure simply returns a null dataset.

spr_QueryEmployer

The spr_QueryEmployer stored procedure is used to search the database for users where employer name matches the search text string. This procedure is called by the service's Query method by passing a string to the @searchtext parameter. The following code implements this procedure on the server:

CREATE PROC QueryEmployer
@searchtext nvarchar(50)
AS
SELECT DISTINCT [User].UserID FROM [User]
INNER JOIN Position ON [User].UserID=Position.UserID
INNER JOIN Employer ON Employer.EmployerID=Position.EmployerID
WHERE Employer.EmployerName LIKE '%' + @searchtext + '%'
ORDER BY [User].UserID
GO

By using the LIKE keyword and wildcards, this procedure will find any records containing the search text anywhere in EmployerName.

spr_QueryInstitution

Another query procedure included in this sample is spr_QueryInstitution, which returns user records where the educational institution name matches the search string. The following code implements spr_QueryInstitution :

CREATE PROC QueryInstitution
@searchtext nvarchar(50)
AS
SELECT DISTINCT [User].UserID FROM [User]
INNER JOIN Education ON [User].UserID=Education.UserID
INNER JOIN Institution ON Institution.InstitutionID=Education.InstitutionID
WHERE Institution.InstitutionName Like '%' + @searchtext + '%'
ORDER BY [User].UserID
GO

Again, using the LIKE keyword and wildcards allows us to find any records containing the search text anywhere in InstitutionName.

Additional Stored Procedures

Although not part of this sample, a complete resume service would require sets of stored procedures, analogous to the previous procedures, that would be used to insert and update data in the service. Additional stored procedures could also be created to provide an improved ability to query the service.

Web Service Components in C#

The business-tier component of the resume service is implemented by the C# ResumeService class. The public methods of this class are exposed via ASP.NET as XML Web service methods. These methods are called by clients via SOAP message requests in XML that are converted by an ASP.NET process on the Web server and passed to the appropriate method on the ResumeService object. These methods then, in turn, access the data in SQL Server via the SqlDataAdapter class to execute the appropriate stored procedure. Data is returned to an ADO.NET DataSet object by invoking the Fill method on the data adapter. Our sample application provides the following methods on the ResumeService class:

  • Query
  • GetData (private)
  • GetResume
  • GetResumeXML

In order to expose these public methods as Web service methods, in Visual Studio the appropriate Web service attribute is applied to both the ResumeService class and each exposed method. For example, the following code shows the WebService attribute applied to the ResumeService class and the WebMethod attribute applied to the GetResume method:

[WebService(Namespace="http://www.webbish6.com")]
public class ResumeService : System.Web.Services.WebService
{
...
[WebMethod(Description = "Returns a strongly-typed .NET DataSet
containing resume data", EnableSession = false)]
public DataSet GetResume(int userID)
{
...
}
}

These attributes also allow you to specify other properties of the service, like namespace and method descriptions. Although not included in this sample, other WebMethod attributes allow you specify more advanced properties of the service, like providing unique names for overloaded methods, enabling session state, and participating in transactions. When your C# Web service is compiled to Intermediate Language (IL) code assemblies, these attributes are converted to metadata and stored with the managed code assemblies. At run-time, ASP.NET interprets this metadata using functionality provided by the .NET Frameworks through the System.Reflection namespace when determining how to expose Web service methods and generate the WSDL for the service.

Query Method

The Query method is a public method that provides the ability to search the database using a specified query stored procedure for the specified string. This method returns an integer array of UserID values that match the selected query text. Query takes the following two arguments:

queryString as string - passes the text that will be used in the WHERE clause of the given query stored procedure. To prevent possible SQL string injections, single-quotes and double dashes are escaped by the following code:

queryString = queryString.Replace("'","''");
queryString = queryString.Replace("--","");

queryType as int - enumerates the type of query that is being executed. In this sample, the two queries are QueryEmployer and QueryInstitution, which map to values 1 and 2 respectively. In order to expand the search functionality of the service, you can easily just add additional query stored procedures in the database, then expand the following switch statement with a new case for each procedure name as appropriate:

switch(queryType)
{
case 1:
queryName = "spr_QueryEmployer";
break;
case 2:
queryName= "spr_QueryInstitution";
break;
default:
throw(new ArgumentOutOfRangeException("queryType",
"The value for queryType is invalid"));
}

After escaping the text in queryString and setting the correct value for queryName, this method uses the SqlDataAdapter object to execute the stored procedure with the specified query text in the following code:

// Prepare the data adapter to execute the stored procedure
SqlDataAdapter myDataAdapter = new SqlDataAdapter(queryName, myConn);
myDataAdapter.SelectCommand.CommandType = CommandType.StoredProcedure;
myDataAdapter.SelectCommand.Parameters.Add("@searchtext", SqlDbType.NVarChar, 50).Value=queryString;
try
{
// Execute the command on the data adapter and return the results
// to the open DataSet
myDataAdapter.Fill(myDataSet);
}
catch (SqlException e)
{
Debug.Write("Unable to execute command and fill dataset. \n" +
e.ToString() + "\n");
}
finally
{
myConn.Close();
}

The data is returned to a DataSet object by calling the Fill method on the SqlDataAdapter. Once the data has been returned to the DataSet object, the following code writes the UserID values to an integer array and returns the array:

// Select the first (and only) returned table (User)
DataTable myDataTable = myDataSet.Tables[0];

// Select the first (and only) returned column (UserID)
DataColumn myDataCol = myDataTable.Columns[0];

// Define an integer array to hold the returned values
int[] userID = new int[myDataTable.Rows.Count];

// For each row, load the UserID value to the integer array.
for(int i=0; i < myDataTable.Rows.Count; i++)
{
DataRow myDataRow = myDataTable.Rows[i];
userID[i] = (Int32)myDataRow[myDataCol];
}

return userID;

After the array of UserID integers has been returned from one of the stored procedures exposed by the Query method, these integer values are passed one-at-a-time either to the GetResume method or the GetResumeXML method in order to retrieve the resume data that corresponds to each UserID.

GetData Method

The GetData method is a private method that is called by both GetResume and GetResumeXML. This method takes a userID value, makes the connection to SQL Server, executes the stored procedures, and returns the resume data as an ADO.NET DataSet object. The following code implements this method:

private DataSet GetData(int userID)
{
// DataSet automatically opens and closes our connection (myConn)
DataSet myDataSet = new DataSet("Resume");

// Create an array for our three procedures
string[] myProcs = {"GetUserInfo", "GetEmployerInfo", "GetEducationInfo"};

// Loop through each procedure
for(int i=0;i<myProcs.Length;i++)
{
// Create a data adapter supplying the procedure name,
// parameter and connection
SqlDataAdapter myDataAdapter =
new SqlDataAdapter(myProcs[i], myConn);
myDataAdapter.SelectCommand.CommandType =
CommandType.StoredProcedure;
myDataAdapter.SelectCommand.Parameters.Add("@UserID",
SqlDbType.Int, 4).Value=userID;

try
{
// Fill this DataAdapter into the open DataSet
myDataAdapter.Fill(myDataSet);
}
catch (SqlException e)
{
Debug.Write("Unable to execute command and fill dataset. \n" +
e.ToString() + "\n");
}
finally
{
myConn.Close();
}
}
return myDataSet;
}

GetResume Method

The GetResume method is the primary method used to retrieve a user's resume data from the service. This method should be called by .NET-aware clients as it returns a strongly-typed ADO .NET DataSet object that is sent to the client via SOAP as XML and is reconstructed by a .NET-aware client. Non-.NET clients should use the GetResumeXML method because it returns a structured XML document rather than just the XML representation of the DataSet object.

This method takes an integer representing a valid UserID value in the database and returns an ADO.NET DataSet object. The following code is used to implement this method:

[WebMethod(Description = "Returns a
            strongly-typed .NET DataSet
containing resume data",
            EnableSession = false)]
public DataSet GetResume(int
            userID)
{
DataSet myDataSet = new
            DataSet();
try
{
myDataSet = GetData(userID);
}
catch
            (Exception e)
{
Debug.Write("GetData call failed with the
following exception: " + e.ToString() +
            "\n");
}

return myDataSet;
}

The dataset is retrieved by calling the private GetData function and passing the userID value. The returned DataSet object is then sent to the client as a SOAP response message. We will see an example SOAP response method later on.

GetResumeXML

The GetResumeXML method is called by clients who are not able to handle a .NET DataSet object. Instead, this method returns an XML fragment representing the requested user's resume as a string within a SOAP response. The following code implements the GetResumeXML method:

[WebMethod( Description = "Returns resume data as a string for .NET
unaware clients", EnableSession = false)]
public string GetResumeXML(int userID)
{
string myResumeXml = "";

try
{
// Pass the userID to the GetData method that returns a dataset
DataSet myDataSet = GetData(userID);

// Write the contents of the dataset to an XML string
myResumeXml = myDataSet.GetXml();
}

catch(Exception e)
{
Debug.Write("Unable to get data. Method returned with exception " +
e.ToString() + "\n");
}

return myResumeXml;
}

This method then retrieves a DataSet object by passing the userID value to the private GetData method. The dataset is then written to an XML string by calling the ext'>GetXML ='codeintext'> method on the DataSet object.

Exceptions

In this sample, exception errors are handled by using the appropriate try, catch, finally C# syntax. Method calls or other operations that have the potential to generate exception errors are executed inside a try block. Each try block is then followed by a catch block, which is used to write the exception to the debugger and possibly run some other custom code, and optionally a finally block, where we close any open connections to the database. When an exception is thrown by an XML web service method, the SOAP response message returned by the service will contain a SOAP <Fault> element. This <Fault> element contains information relating the exception. .NET clients can de-serialize this SOAP Fault as a SoapException.

Testing

Visual Studio .NET makes testing of .NET Web services almost a trivial matter. You do not have to write a separate XML Web service consuming client application because Visual Studio .NET generates one for you automatically when you start the service from within the IDE. When a .NET XML Web service is started for debugging in Visual Studio .NET, the IDE first generates an ASP.NET start page for the service that contains links to the service description WSDL file, as well as each method exposed by the service. For our sample, this start page looks like the following:

In this screen, the descriptions provided for each method come from the Description property of the WebMethod attribute. Clicking on a method name loads another ASP.NET page that contains form fields that correspond to the arguments of the method. A sample of the SOAP request and response messages, as well as the HTTP GET and POST calls for this method, are also displayed. The following screen-shot shows what this page looks like for the Query method:

Entering valid values in the fields and clicking the Invoke button on this page submits a test request to the service running in the debugger via an HTTP POST. If the request was successful, a new browser window is launched which displays an XML document fragment that contains the response data. The following screen shows the XML fragment representing the integer array data returned by HTTP POST to the Query method:

When a handled exception occurs, this screen displays information about the exception. If the exception is unhandled, this screen may return an HTTP error code of 500, indicating an internal server error.

In order to test the sample code available with this case study supply, values in the following table will return data from the service:

Method | Input (type) | Valid Input
Query (employer queryType = 1) | QueryString (string) | "Microsoft"
Query (institution queryType = 2) | QueryString (string) | "Cincinnati"
GetResume | UserID (int) | 1 or 2
GetResumeXML | UserID (int) | 1 or 2

Deployment

Once our XML Web service code has been completed, tested and debugged, there is the final deployment hurdle to clear. For the most part, an XML Web service built based on .NET is deployed like any other ASP.NET internet application, but with a few additions. A project can be deployed either by copying files to the proper location in the V-Root of the production Web server, or by creating a new deployment project in Visual Studio .NET. If you are planning to deploy your service in a large enterprise environment, you might want to look into using Microsoft's Application Center Server to simplify the daunting task of deploying and managing Web applications on large groups of Web servers.

When deploying your service, you will want to make sure that the windows accounts that are being used to access SQL Server are still valid. For example, if you developed your service using a local machine account, you may need to change it to a domain-wide account. In many cases, when connecting to SQL Server on a separate machine is will also be necessary to use ASP.NET impersonation to provide the correct login to SQL Server.

The final step of deployment involves publishing the dynamic discovery document (DISCO). This document, generated by Visual Studio .NET, tells Web service clients how to get to your service. The DISCO document is published by default in the root directory of the service, so if you do not want everyone to be able to discover your service, you will want to rename, move or delete the .vsdisco file. Based on the information in this file, clients can then request from your service the WSDL file that they need to understand how to access the services.

Scalability

For a production Web service application, like any other internet-based enterprise application, scalability is a major concern. Although fine-tuning the performance of an XML Web service is outside of the scope of this case study, the following techniques should be investigated as potential ways to improve scalability:

  • Use best practices for accessing the database, including closing (or allowing the provider to close) any unused connections that you explicitly open. When using ADO.NET, connections opened by the provider are automatically garbage collected back to the connection pool.
  • Avoid having ASP.NET manage session state. Instead, track this information in the application and include state information in SOAP headers.
  • Use clusters servers and run the Web server and database on separate machines.
  • Use SQL Server Replication to publish the database to multiple production database servers from a single source database.

Conclusion

Microsoft's .NET platform is an ideal environment for developing, hosting, and consuming XML Web services, and C# is the language of choice for developing Web service applications. C# components can efficiently retrieve data stored relationally in SQL Server via the ADO.NET provider that executes stored procedures on the database. These component methods can then be easily exposed as Web service methods. You don't even have to write one line of XML code to deploy and publish discovery documentation documents for your service because Visual Studio generates these documents and ASP.NET does the hard work of serializing your C# methods into SOAP messages.

Appendix: A

WSDL File Representation of the Resume Service

The following is a portion of the WSDL document generated by Visual Studio .NET for our sample service. For brevity, we have removed all of the XML description that is not related to the Query method. The complete WSDL file can be found in the source files for this project as ResumeService.wsdl.

<?xml version="1.0" encoding="utf-8" ?>
<definitions xmlns:http="http://schemas.xmlsoap.org/wsdl/http/"
xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/"
xmlns:s="http://www.w3.org/2001/XMLSchema"
xmlns:s0="http://www.webbish6.com"
xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/"
xmlns:tm="http://microsoft.com/wsdl/mime/textMatching/"
xmlns:mime="http://schemas.xmlsoap.org/wsdl/mime/"
targetNamespace="http://www.webbish6.com"
xmlns="http://schemas.xmlsoap.org/wsdl/">
<types>
<s:schema elementFormDefault="qualified"
targetNamespace="http://www.webbish6.com">
<s:import namespace="http://www.w3.org/2001/XMLSchema" />
<s:element name="Query">
<s:complexType>
<s:sequence>
<s:element minOccurs="0" maxOccurs="1"
name="queryString" type="s:string" />
<s:element minOccurs="1" maxOccurs="1"
name="queryType" type="s:int" />
</s:sequence>
</s:complexType>
</s:element>
<s:element name="QueryResponse">
<s:complexType>
<s:sequence>
<s:element minOccurs="0" maxOccurs="1"
name="QueryResult" type="s0:ArrayOfInt" />
</s:sequence>
</s:complexType>
</s:element>
<s:complexType name="ArrayOfInt">
<s:sequence>
<s:element minOccurs="0" maxOccurs="unbounded"
name="int" type="s:int" />
</s:sequence>
</s:complexType>
...
</s:schema>
</types>
<message name="QuerySoapIn">
<part name="parameters" element="s0:Query" />
</message>
<message name="QuerySoapOut">
<part name="parameters" element="s0:QueryResponse" />
</message>
...
<message name="QueryHttpGetIn">
<part name="queryString" type="s:string" />
<part name="queryType" type="s:string" />
</message>
...
<message name="QueryHttpPostIn">
<part name="queryString" type="s:string" />
<part name="queryType" type="s:string" />
</message>
<message name="QueryHttpPostOut">
<part name="Body" element="s0:ArrayOfInt" />
</message>
...
<portType name="ResumeServiceSoap">
<operation name="Query">
<documentation>
Returns an array of User IDs that match a query
</documentation>
<input message="s0:QuerySoapIn" />
<output message="s0:QuerySoapOut" />
</operation>
...
</portType>
<portType name="ResumeServiceHttpGet">
<operation name="Query">
<documentation>
Returns an array of User IDs that match a query
</documentation>
<input message="s0:QueryHttpGetIn" />
<output message="s0:QueryHttpGetOut" />
</operation>
...
</portType>
<portType name="ResumeServiceHttpPost">
<operation name="Query">
<documentation>
Returns an array of User IDs that match a query
</documentation>
<input message="s0:QueryHttpPostIn" />
<output message="s0:QueryHttpPostOut" />
</operation>
...
</portType>
<binding name="ResumeServiceSoap" type="s0:ResumeServiceSoap">
<soap:binding transport="http://schemas.xmlsoap.org/soap/http"
style="document" />
<operation name="Query">
<soap:operation soapAction="http://www.webbish6.com/Query"
style="document" />
<input>
<soap:body use="literal" />
</input>
<output>
<soap:body use="literal" />
</output>
</operation>
...
</binding>
<binding name="ResumeServiceHttpGet" type="s0:ResumeServiceHttpGet">
<http:binding verb="GET" />
<operation name="Query">
<http:operation location="/Query" />
<input>
<http:urlEncoded />
</input>
<output>
<mime:mimeXml part="Body" />
</output>
</operation>
...
<binding name="ResumeServiceHttpPost" type="s0:ResumeServiceHttpPost">
<http:binding verb="POST" />
<operation name="Query">
<http:operation location="/Query" />
<input>
<mime:content type="application/x-www-form-urlencoded" />
</input>
<output>
<mime:mimeXml part="Body" />
</output>
</operation>
...
</binding>
<service name="ResumeService">
<port name="ResumeServiceSoap" binding="s0:ResumeServiceSoap">
<soap:address location=
"http://localhost/ResumeService/ResumeService.asmx" />
</port>
<port name="ResumeServiceHttpGet" binding="s0:ResumeServiceHttpGet">
<http:address location=
"http://localhost/ResumeService/ResumeService.asmx" />
</port>
<port name="ResumeServiceHttpPost" binding="s0:ResumeServiceHttpPost">
<http:address location=
"http://localhost/ResumeService/ResumeService.asmx" />
</port>
</service>
</definitions>

Case Study Information
Author | Jeannine & Glenn Gailey
Editor | Adam Ryland
Reviewers | Phil Sidari, Richard Conway, Matthew MacDonald

If you have any questions or comments about this case study, or suggestions for further pieces, please contact the technical editor. If you wish to talk about the case study with other subscribers, please visit our forum here.

Fast Track Books from Wrox Press
If you want a C# book that is compact, fast-paced but still provides the quality of information that you have come to expect from a Wrox book - you want Fast Track C#, newly available from Wrox Press.

This book will help you to quickly learn about the following:
  • Understand what C# is, and how it fits into the .NET Framework
  • A concise introduction to all the main features of the C# language, with Java and C++ comparisons noted
  • Using the .NET base classes
  • Writing and deploying Windows applications with C# and Visual Studio .NET
  • Creating and configuring .NET assemblies
  • Accessing databases with ADO.NET, and using XML in C#
  • Integrating your existing COM components with the .NET Framework
  • COM+ Services in the .NET Framework
  • Building web applications with ASP.NET pages
  • Creating web services in C#

C#Today - © 2002 WroxPress Limited


Mail Tracking and Analysis System

Mail Tracking and Analysis System
http://www.csharptoday.com/content.asp?id=1845
Article Rating: 4.5
Published on June 10, 2002
Abstract
One of the easiest, fastest and most reliable ways of communication is through e-mail. Many people carry out business with the help of e-mail, as e-mail is one of the primary means of communication in eBusiness. In this case study, Vinod Kumar looks at a Mail Client software solution that uses XML Webservices to store mails which have been sent or received in a centralized database for future analysis. A mail analyzing solution is also discussed.
Case Study

The easiest, fastest and most reliable method of communication is via e-mail. Many people carry out business with the help of e-mail. This kind of business communication can be classified into two main categories:

  • Inter-office communication, which is the communication within an organization between various departments. In this situation, the organization uses its own mail server for communication, or can use an external one.
  • Communication with external clients, where there is little control over how the user sends\deals with mails.

In the latter scenario, the tracking of the mails sent and received is tedious and difficult for to the following reasons:

  • Each client may use a different mail server for the purpose of sending and receiving mails; it would be difficult or highly impossible to keep track of the mails.
  • Suppose if each of them have their own mail account information, it would be difficult to get all the information and perform analysis.

To solve this kind of problems, we develop mail client software, called WebMail, which stores the mailing information sent and received in a centralized database. This stored information is used for mail analysis, which is discussed in the later part of this case study.

System Requirement

Before continuing, please take note of the following system requirements for the purposes of this discussion, as well as for working with the sample code accompanying the case study:

  • Windows 2000 or XP with IIS
  • . Net framework
  • Visual Studio .NET RTM
  • SQL Server 7.0 or 2000

We can use any external mail server that supports SMTP and POP for sending and receiving the mails using this tool. This case study requires some familiarity with the Microsoft .Net Framework, C#, and XML. Familiarity with SMTP and POP protocols is not necessary, but would be helpful. For a good background on these topics, please refer to:

POP3 standards

RFC 1939

ftp://ftp.isi.edu/in-notes/rfc1939.txt?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq

RFC1730

ftp://ftp.isi.edu/in-notes/rfc1730.txt?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq

A POP3 Client and C#

http://www.csharptoday.com/content.asp?id=1681&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq

TcpClientClass

http://msdn.microsoft.com/library/en-us/cpref/html/frlrfSystemNetSocketsTcpClientClassTopic.asp?frame=true&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq

NetworkStreamClass

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/cpref/html/frlrfSystemNetSocketsNetworkStreamClassTopic.asp&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq

EncodingClass

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/cpref/html/frlrfsystemtextencodingclasstopic.asp&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq

The WebMail System

Now let’s see how the WebMail works. This software consists of two different applications, namely a windows (WebMailClient) application and a Web Service (WebMailWS). The Windows application is used to send and receive the mails using any mail server that supports SMTP and POP protocol. The WebMailClient application also helps to update the mail sent and received information to a remote database using the WebMailWS Web service after storing the information in the client place in an XML format. Since a remote database can be situated anywhere in the world, we use WebServer for storing the Mail Information in a heterogeneous network. Once the information is stored, it can be used for mail analysis. Since we are using XML to store a copy of the mail information in the client location before storing it in the remote location, a client can get access to the mail information without a connection to the remote database.

Each time a mail is sent using this software, a unique identity number is generated and is appended to the subject (the WebMailClient tool hides the Unique ID from the user) along with the reference MailID (if any) and department code. This unique number is used for further references to the mail. The ClientWebMail application receives and stores only the mails that have this unique ID.

Database Design and Structure

As we see from the architecture of the system, two data stores are used to store and provide information for our application. An XML data source is used to store the information locally in the client, whereas an SQL Server database is used to store the information in the remote location.

Mail Tracking and Analysis System’s Database design

The database structure consists of five tables. Here is a list of tables that are added to the database.

The AccessInfo table sets the Access privilege for a user (through EMailID) to access others email information. This stores the EmailID, which is mapped to the AccessID.

The Department table stores the Department code, along with the name.

The AddressBook table stores the EmailID and password, along with the department code and AccessID.

The Send Mail table stores sent Mail information like Mail ID, DeptCode, Reference ID, Sender Email ID, sent date, subject, and message.

The ReceivedMailInfo table stores the mail receivers information along with the mail’s received date and Mail Read status.

The diagram below shows the table relationship:

Business Processes

The scope of this case study is to show how to manage the mails sent and received between a organization, which includes storing the mail information in a centralized database. The case study is split into three parts for better understanding.

In the first part, we build the Mail component, which helps us send and receive mails. The SendMail class sends mails using SMTP standard, and the POP3Client Class helps to receive mails, which supports the POP protocol.

In the second part, we discuss the web service (WebMailWS) that is used to perform login authentication and update the remote database with the sent and received mail’s information.

In the third part we build a client WinForm application (WebMailClient), which act as an interface to send and receive mails and also helps in analyzing the mails.

Component Design

The Mail component solution has four components. One is used to send mail, and the other three are used to receive mail.

The SendMail Class

Let’s examine the SendMail class that is used for sending mails. This class contains a set of private variables that are used to hold mail information. Some of these variables are exposed in the form of properties, since it allows us to exercise a greater degree of control over the values that are assigned to the variables. The other variables like m_Date and m_MessageID are used to set the sent date of the mail and the unique mail Message ID. m_SMTP is set to the TCPClient class, which is present in the System.Net.Sockets namespace. The TcpClient is used to establish a connection with the remote host. Application-level protocols such as FTP and HTTP are built on this class, which in turn is built upon the Socket class to provide TCP services at a higher level of abstraction.

The diagram below shows all the members in SendMail class:

public class
            SendMail
{
protected string m_MessageID=null;
protected
            string m_SMTPServer=null;
protected string
            m_From=null;
protected string m_FromName=null;
protected
            string m_To=null;
protected string m_Subject=null;
protected
            string m_Body=null;
protected string m_Date=
            DateTime.Today.ToString ();
protected TcpClient m_SMTP = null;
public SendMail()
{
}
//property to get and set From
            EmailID
public string From
{
get
{
return
            m_From;
}
set
{
m_From = value;
}
}
//property
            to get and set From Name
public string
            FromName
{
get
{
return
            m_FromName;
}
set
{
m_FromName =
            value;

}
}
//property to get and set To
            EmailID's
public string To
{
get
{
return
            m_To;
}
set
{
m_To = value;

}
}
//property
            to get and set Subject of the Mail
public string
            Subject
{
get
{
return
            m_Subject;
}
set
{
m_Subject =
            value;

}
}
//property to get and set Body of the
            Mail
public string Body
{
get
{
return
            m_Body;
}
set
{
m_Body =
            value;

}
}
//property to get and set SMTP
            ServerName
public string SMTPServer
{
get
{
return
            m_SMTPServer;
}
set
{
m_SMTPServer =
            value;

}
}

The Send method is used to send mails to the remote mail server. This method establishes a connection to the remote server using the TCPClient class, which returns a NetworkStream object that is used to read and write data to the remote host. This method uses two other methods, namely IsValidReply and SendMessage, to read and write data to the remote server. The IsValidReply method is used to read and validate the data as commands sent from the remote server. SendMessage sends the message as a command to remote server:

public string Send()
{
try
{
m_SMTP= new TcpClient(m_SMTPServer,25);
NetworkStream strm =
            m_SMTP.GetStream();
string reply = '';
// Server must replay
            220 - Server OK
reply = IsValidReply(strm);
if(reply !=
            'OK')
{
return reply;
}
// Send Hello to
            server
SendMessage('EHLO ' + m_SMTPServer +
            '\r\n',strm);
reply = IsValidReply(strm);
if(reply !=
            'OK')
{
return reply;
}
// Send From Mail ID to the
            server
SendMessage('MAIL FROM: <' + m_From +
            '>\r\n',strm);
reply = IsValidReply(strm);
if(reply !=
            'OK')
{
return reply;
}
// sending TO mailID to
            server
//Regular expression is used to split the
//to emailid
            which are seperated by comma delimiter
Regex RgxMail=new
            Regex(',');
string[] strMailSplit =
            RgxMail.Split(m_To);
foreach(string t in strMailSplit)
{
SendMessage('RCPT TO: <' + t + '>\r\n',strm);
reply =
            IsValidReply(strm);
if(reply != 'OK')
{
return
            reply;
}
}
SendMessage('DATA \r\n',strm);
reply =
            IsValidReply(strm);
if(reply != 'OK')
{
return reply;
}

The code below is used to send the message body data to the remote server, and then disconnects. This message body is built using a function buildMessageBody.

// Sending Message Body
byte[] bMime
            =
            Encoding.Default.GetBytes(buildMessageBody());
strm.Write(bMime,0,bMime.Length);
//
            Notify End of Data
SendMessage('\r\n.\r\n',strm);
reply =
            IsValidReply(strm);
if(reply != 'OK')
{
return reply;
}
// disconnecting from the
            Server
SendMessage('QUIT\r\n',strm);
return
            'OK';
}
catch(Exception x)
{
return 'ERR ' +
            x.Message;
}
}

Note: '\r\n?should be appended to each command sent. This is because SMTP specification indicates that a CRLF pair terminates all commands. SendMessage has two arguments, Data (of type string) and Stream (of type NetworkStream). Stream is a reference type variable, and Data is a value type variable. The Data argument receives SMTP commands and returns values such as 250, 220, and 354 and so on (signifying if a particular command sent to the server is carried out correctly or has failed). Before writing to the Network stream, the string has to be converted to byte format. To achieve this conversion, we use the Encoding.ASCII property of the Encoding class (System.Text Namespace) and the GetBytes function, which returns bytes out of characters.

private void SendMessage(string data,NetworkStream stream)
{
byte[] byteData = Encoding.ASCII.GetBytes(data.ToCharArray());
stream.Write(byteData,0, byteData.Length);
}

The IsValidReply method is used to read and validate the data as commands sent from the remote server. This method uses stream reader to read the data. This data, which is in byte format, is converted to a string type using the Encoding.ASCII property of the Encoding class and the GetString function. A waiting mechanism is deployed, which waits for the remote server to reply within a specific time frame before throwing an error code as a return value.

private string
            IsValidReply(NetworkStream strm)
{
string serverReply =
            'ERR';
int counter = 0;
while(true)
{
//checking for
            dada avaliability
if(strm.DataAvailable)
{
//setting byte
            size
byte[] buffer = new byte[1024];
int bytes =
            strm.Read(buffer,0,1024);
serverReply =
            Encoding.ASCII.GetString(buffer,0,bytes);
serverReply.Trim();
serverReply
            =
            serverReply.Substring(0,serverReply.Length-2);
break;
}
//waiting
            for the server replay
Thread.Sleep(300);
counter++;
if(counter > 50)
{
break;
}
}
// 250 -
            Command was Sucessful, 354 - Start inputing Data, 220 - Server is
            ready
if(serverReply.Substring(0,3) == '250' ||
            serverReply.Substring(0,3) == '354' || serverReply.Substring(0,3) ==
            '220')
{
return 'OK';
}
else
{
return
            serverReply;
}
}

The buildMessageBody() method is used to build the mail information before sending it using the SendMessage method. This method uses the BuildBody and SplitTo functions to build the mail message:

private string buildMessageBody()
{
m_MessageID = Guid.NewGuid().ToString();
string
            MailBuilder = '';
MailBuilder += 'From: \'' + m_FromName + '\'
            <' + m_From + '>\n';
MailBuilder += 'To: ' + SplitTo() +
            '\n';
MailBuilder += 'Subject: ' + m_Subject +
            '\n';
MailBuilder += 'Date: ' + m_Date + '\n';
MailBuilder +=
            'MIME-Version: 1.0\n';
MailBuilder += 'Content-Type:
            multipart/mixed;\n';
MailBuilder +=
            '\tboundary=\'----=_NextPart_' + m_MessageID +
            '\'\n';
MailBuilder += BuildBody();
return
            MailBuilder;
}

The BuildBody function is used to build the body of the message, and the SplitTo function is used to split and format the receiver’s e-mail in POP standard, for sending data using the Regular Expression RegEx class (in the System.Text.RegularExpression namespace). A comma delimiter separates the EMailID.

private string BuildBody()
{
string retVal=””;
retVal += '------=_NextPart_' + m_MessageID
            + '\n';
retVal += '\tcharset=\'iso-8859-1\'\n';
retVal +=
            'Content-Transfer-Encoding: 8bit\n\n';
retVal += 'STARTBODY\n\n';
retVal += m_Body + '\n';
retVal += 'ENDBODY\n\n';
retVal
            += '\n';
return retVal;
}
private string SplitTo()
{
string retVal = '';
Regex Regex=new Regex(',');
string[]
            strSplitParts = Regex.Split(m_To);
foreach(string t in
            strSplitParts)
{
retVal += '<' + t + '>';
if(i <
            strSplitParts.Length)
{
retVal +=
            ',\n\t';
}
else
{
retVal += '\n';
}
}
return
            retVal;
}

Now let’s look in more detail at the three classes that are used for receiving mails from the mail server.

POP3Client Class

The POP3Client class is used to send and receive POP3 commands to/from the POP3 email server. Using the POP3Client class, the user can connect to the POP3 server, read messages, and so on.

The diagram below show all the members in the POP3Client class:

This class has a set of private variables that are used to hold POP server information. Let’s look at some of these variables in the form of properties. A variable to hold the POP3MessagesInfo class instance is also declared.

public class Pop3Client
{
private
            TcpClient m_pPop3_Client = null;
private string m_Pop3Server =
            '';
private string m_UserName = '';
private string m_Password
            = '';
private bool m_LoggedIn = false;
private
            Pop3MessagesInfo m_MessagesInfo = null;
public string Pop3Server
{
get{ return m_Pop3Server; }
}
public string
            UserName
{
get{ return m_UserName; }
}
public string
            Password
{
get{ return m_Password; }
}

The LogIn method is used to validate the user with his username and password after establishing the connection to the remote server using the TCPClient class by the POP3Client method. The LogIn Method uses two functions, IsValidReply and SendMessage, to read and write the data to the remote server.

public bool LogIn(string userName,string
            password)
{
////POP Command USER - Sending USER Name for Mail
            Server
///Authentication
byte[] reply = SendMessage ('USER '
            + userName + '\r\n');
// Check if userName is
            ok.
if(IsValidReply(reply)){
////POP Command PASS - Sending
            Password for Mail Server
///Authentication
reply =
            SendMessage ('PASS ' + password + '\r\n');
// Check if password
            is ok.
if(IsValidReply (reply)){
m_LoggedIn =
            true;
m_UserName = userName;
m_Password = password;
return
            true;
}
}
return false;
}

SendMessage has two arguments, Data (of type string) and Terminator. This method is used to send the data, and waits until it gets the reply from the server for the data sent:

private byte[] SendMessage (string
            data,string terminator)
{
byte[] retVal =
            null;
try
{
NetworkStream strm =
            m_pPop3_Client.GetStream();
if(data.Length > 0){
byte[]
            byte_data =
            Encoding.ASCII.GetBytes(data.ToCharArray());
strm.Write(byte_data,0,byte_data.Length);
}
MemoryStream memStrm = new MemoryStream();
// Create
            loop
int counter = 0; // For command time out.
while(true){
if(strm.DataAvailable){
byte[] receivedData =
            new byte[m_pPop3_Client.ReceiveBufferSize];
int size =
            strm.Read(receivedData,0,m_pPop3_Client.ReceiveBufferSize);
memStrm.Seek(0,
            SeekOrigin.End);
memStrm.Write(receivedData,0,size);
// Create
            buffer for Compare eg. '\r\n.\r\n'.
byte[] bufComp = new
            byte[10];
// Read last 10 symbols from result
            stream
memStrm.Position = memStrm.Length -
            7;
memStrm.Read(bufComp,0,10);
string strCompare =
            Encoding.ASCII.GetString(bufComp);
// Check if message ends with
            terminator
if(strCompare.IndexOf(terminator) > -1){
break;
}
counter = 0;
}
else
{
System.Threading.Thread.Sleep(100);
}
if(counter
            > 100){
// ('time
            out');
break;
}
counter++;
}
retVal =
            memStrm.ToArray();
memStrm.Close();
}
catch(Exception
            x)
{
//('erro:' + x.Message);
}
return
            retVal;
}

The IsValidReply method is similar to the IsValidReply method in the SendMail class. This method uses stream reader to read the data, and this data, which is in the byte format, is converted to a string type using the Encoding.ASCII class and the GetString function. After converting the data to string, the data is validated.

private bool IsValidReply(byte[] reply)
{
if(reply != null){
string str = Encoding.ASCII.GetString
            (reply);
str =
            str.TrimStart();
if(str.StartsWith('+OK')){
return
            true;
}
}
return false;
}

The getMessagesList method is a public method. It is used to get the list of mails from the remote mail server. This method invokes the POP3MessageInfo class, which in turn gets the list of mails as an arraylist.

public Pop3MessagesInfo getMessagesList
            ()
{
// If not loggedIn, throw
            exception.
if(!m_LoggedIn){
throw new Exception('You must
            login before using this system !');
}
//POP Command LIST -
            Will list the mail info
byte[] reply = SendMessage
            ('LIST\r\n','.\r\n');
m_MessagesInfo = new
            Pop3MessagesInfo(reply);
return m_MessagesInfo;
}

Pop3MessagesInfo Class

The POP3MessageInfo class is used to list the number of mails for a user in the mail server, along with the size of each mail in an ArrayList. This class gets all the messages data in byte format and converts the byte data into a string and then splits the messages using the Split function:

private ArrayList m_MsgList = null;
public Pop3MessagesInfo(byte[] replay)
{
m_MsgList = new
            ArrayList();
string str =
            System.Text.Encoding.ASCII.GetString(replay);
string[] lines =
            str.Split(new Char[]{'\n'});
string[] msgs =
            null;
if(lines.Length > 3){
msgs = new
            string[lines.Length-3];
for(int
            i=1;i<lines.Length-2;i++){
msgs[i-1] = lines[i];
}
foreach(string s in msgs){
//
            System.Windows.Forms.MessageBox.Show(s);
string[] msgInf =
            s.Split(new char[]{' '});
if(msgInf.Length > 1){
string
            msgNr = msgInf[0];
int size =
            Convert.ToInt32(msgInf[1]);
m_MsgList.Add(new
            Pop3Message(msgNr,size));
}
else{
string msgNr =
            msgInf[0];
m_MsgList.Add(new
            Pop3Message(msgNr,-1));
}
}
}
}

The GetMessage() method is used to retrieve the mail message from the server after receiving the mail number as an argument. This method sends the POP command 'RETR' along with the mail number, using the SendDataWaitReplay function to get the message.

public byte[] getMessage(string
            messageNo)
{
// If not loggedIn, throw
            exception.
if(!m_LoggedIn){
throw new Exception('LogIn before
            using this function !');
}
//POP Command RETR - Will Retrive
            the the mails
byte[] replay = SendDataWaitRelpay('RETR ' +
            messageNo + '\r\n','.\r\n');
// Check that message is
            complete
if(EndsWith(replay,'.\r\n')){
return
            replay;
}
else{
return null;
}
}

The LogOff Method sends the POP command “QUIT?to logout from the Mail Server, but only after checking the logged in status.

public void LogOff()
{
// If not
            loggedIn, throw exception.
if(!m_LoggedIn){
throw new
            Exception('You must logged in to use this function
            ');
}
////POP Command QUIT - Will Disconnect from the
            Server.
SendDataWaitRelpay('QUIT
            \r\n');
m_pPop3_Client.Close();
}

WebMailWS - WebService

Now we are in the second part of our case study. Here we build a WebService that is used for storing the mail sent and received information. This webservice is also used to provide other information like MailDetails, Address Book, and Department List.

The WebMail Web service exposes seven methods to Internet clients:

  • IsUserValid
  • SendMailInfo
  • ReceiveUpdateInfo
  • getMailDetails
  • getAddressBook
  • getDeptDetails
  • getUserNames

Let's see each of these methods in detail.

We declare a class WebMailService that extends the WebService class from the System.Web.Services namespace. This namespace consists of the classes that enable us to build and use the Web Service.

public class WebMailService : System.Web.Services.WebService
{
private string m_UserName=null;
private string m_Password =null;
private string connStr=ConfigurationSettings.AppSettings['connStr'];

The WebService class is the base class of the WebmailService class. There can be any number of public or private classes. However, if a class is to expose some or all of its methods as part of the web service, the class and the method should be declared public.

The Web Service Attribute [WebService()] is used to specify the name of the web service and is also used in describing and providing help pages for the WebService.

IsUserValid WebMethod

IsUserValid is one of the public methods in the class WebMailService. This method validates a remote client by taking the UserName and Password as arguments. This validation allows the remote client to access various other services in the WebMailService class. Here we use the SqlDataReader object to access the database, since SqlDataReader provides a means of reading a forward-only stream of rows from a database in a much faster way. In this web service method, SqlDataReader is built using the SqlCommand object by executing the SQL statement using the ExecuteReader method. The user authentication fails if the reader returns a null value.

public bool isUserValid (string
            UserName,string Password)
{
try
{
string
            strSQL='';
m_UserName
            =UserName;
m_Password=Password;
SqlConnection sqlConn=new
            SqlConnection(connStr);
strSQL+=' Select emailid,password from
            Addressbook ';
strSQL+= ' where emailid='' + m_UserName + '' and
            ';
strSQL += ' password='' + m_Password + ''';
SqlCommand
            sqlComm = new
            SqlCommand(strSQL,sqlConn);
sqlComm.Connection.Open();
SqlDataReader
            dbReader =sqlComm.ExecuteReader(CommandBehavior.CloseConnection
            );
if (dbReader.Read())
{
//valid user
return
            true;
}
else
{
// authentication failed
return
            false;
}
}
catch
{
return false;
}
}

The SendMailInfo web method is used to store the mail send information of the remote client in the database. This web method takes MailID, Receivers Email ID, Mail Body, Mail Subject, etc as arguments and updates the database once the user has been validated. This method uses SQLConnection, SQLCommand and SQLDataReader objects for database update.

public string SendMailInfo(string
            UserName,string Password,string MailId,
string RefId,string
            ReceiverEMails,string DeptCode, string subject,string
            Message)
{
try
{
string SQL=null;
StringBuilder
            strSQL= new StringBuilder ();
m_UserName =UserName;
m_Password
            =Password;
if(IsUserValid(m_UserName,m_Password))
{
strSQL.Append (
            'Insert into SendMail values ('' + MailId + ''');
strSQL.Append
            (' ,'' + RefId + '',' + DeptCode + ','' + UserName +
            ''');
strSQL.Append (' ,'' + DateTime.Today + '','' + subject +
            '','' + Message + '')');
SqlConnection sqlConn=new
            SqlConnection(connStr);
SqlCommand sqlComm = new
            SqlCommand();
sqlComm.CommandText=strSQL.ToString();
sqlComm.Connection
            = sqlConn;
sqlComm.Connection.Open
            ();
sqlComm.ExecuteNonQuery();
sqlComm.Connection.Close();
Regex
            RgxEmail=new Regex(',');
string[] RecEmails =
            RgxEmail.Split(ReceiverEMails);
SqlCommand sqlCommUpdate= new
            SqlCommand();
foreach(string RecEmail in
            RecEmails)
{
SQL='';
SQL='insert into ReceiveMail values
            ('' + MailId + '','' + RecEmail + '','N','' + DateTime.Today +
            '','N')';
sqlCommUpdate.CommandText=SQL;
sqlCommUpdate.Connection
            =
            sqlConn;
sqlCommUpdate.Connection.Open();
sqlCommUpdate.ExecuteNonQuery();
sqlCommUpdate.Connection.Close();
}
return
            'Mail Send Successfully' ;
}
else
{
return 'User Not
            Valid';
}
}
catch(Exception e)
{
return
            e.ToString();
}
}

ReceivedDBUpdate WebMethod

The ReceivedDBUpdate WebMethod is used to set the flag “Y?in the Received column of the ReceiveMail table. This WebMethod gets the UserName, Password (as type string) and Received Mail information of a remote client as a dataset. The UserName and Password are used for the remote client authentication, and the dataset, which holds the information about the mail received by the remote user, is used to update the Mails Received status for all the MailID's in the Dataset. This method is called when a remote client receives a new mail or a set of new mails. These mails are the ones sent by any of the other clients using the WebMailClient software.

public string ReceivedDBUpdate(string
            UserName,string Password,DataSet MailUpdate)
{
try
{
string strSQL= null;
m_UserName =UserName;
m_Password
            =Password;
SqlConnection sqlConn=new SqlConnection(connStr);
if(isUserValid(m_UserName,m_Password))
{
SqlCommand
            sqlCommUpdate= new SqlCommand();
DataSet dsMailUpdate= new
            DataSet();
dsMailUpdate = MailUpdate;
//gets no of rows in the
            dataset
for (int i=0; i < dsMailUpdate.Tables[0].Rows.Count;
            i++)
{
strSQL='';
strSQL +='Update ReceiveMail set
            Received='Y',MailRead='Y',';
strSQL += ' RecDate='' +
            DateTime.Today + ''';
strSQL += ' where REMail='' + UserName + ''
            and ';
//get the mail id from the daatset
strSQL += '
            MailID='' +
            dsMailUpdate.Tables[0].Rows[i].ItemArray.GetValue(0).ToString() +
            ''';
sqlCommUpdate.CommandText=strSQL.ToString();
sqlCommUpdate.Connection
            =
            sqlConn;
sqlCommUpdate.Connection.Open();
sqlCommUpdate.ExecuteNonQuery();
sqlCommUpdate.Connection.Close();
}
}
return
            'Updated';
}
catch(Exception e)
{
return
            e.ToString();
}
}

MailDetails WebMethod

The MailDetails WebMethod takes the UserName and Password as arguments and returns mail information as a dataset. This method calls the function MailInfo for returning the dataset. This method not only gets the mail information of logged-in users, but also mail information for the other users that the logged-in user has permission to view.

public DataSet MailDetails(string
            UserName,string Password)
{
DataSet ds=new
            DataSet('SendMail');
m_UserName =UserName;
m_Password
            =Password;
if(IsUserValid
            (m_UserName,m_Password))
{
return MailInfo();
}
else
{
return ds;
}
}

SendMailInfo Method

The SendMailInfo function is used to get all the information from the SendMail table along with its corresponding information from the ReceiveMail table. The code starts out by filling a dataset with the SendMail table. The SQL query populates only the user’s mail information, which can be accessed by the logged-in user. The dataset is again filled with the ReceiveMail table. The relationship between these two tables in the dataset is created using the DataRelation object. This relationship enables us to see what links information from one table to another. The essential elements of a DataRelation are the name of the relationship, the two tables being related, and the related columns in each table. Here we relate the MailID columns of the two tables and keep the relationship name as Receiver Details.

public DataSet SendMailInfo()
{
DataSet ds;
StringBuilder strSQL = new
            StringBuilder();
SqlConnection myConnection = new
            SqlConnection(connStr);
// Create a
            SqlDataAdapter.
SqlDataAdapter myAdapter = new
            SqlDataAdapter();
myAdapter.TableMappings.Add('Table',
            'SendMail');
myConnection.Open();
strSQL.Append ('select *
            from sendmail where sender in');
strSQL.Append (' (select emailid
            from accessinfo where accessid=');
strSQL.Append (' (select
            accessid from addressbook where ');
strSQL.Append (' emailid=''+
            m_UserName + ''))');
SqlCommand myCommand = new
            SqlCommand(strSQL.ToString(),myConnection);
myCommand.CommandType
            = CommandType.Text;
myAdapter.SelectCommand = myCommand;
ds
            = new DataSet('Mail');
myAdapter.Fill(ds);
// Create a second
            Adapter and Command.
SqlDataAdapter adpReceive = new
            SqlDataAdapter();
adpReceive.TableMappings.Add('Table',
            'ReceiveMail');
strSQL.Remove(0,strSQL.Length);
strSQL.Append
            ('select * from ReceiveMail where REmail in');
strSQL.Append ('
            (select emailid from accessinfo where accessid=');
strSQL.Append
            (' (select accessid from addressbook where ');
strSQL.Append ('
            emailid=''+ m_UserName + ''))');
SqlCommand cmdReceive = new
            SqlCommand(strSQL.ToString(),myConnection);
adpReceive.SelectCommand
            = cmdReceive;
adpReceive.Fill(ds);
myConnection.Close();
// Create DataRelation.
System.Data.DataRelation
            dr;
System.Data.DataColumn dc1;
System.Data.DataColumn dc2;
dc1 = ds.Tables['SendMail'].Columns['MailID'];
dc2 =
            ds.Tables['ReceiveMail'].Columns['MailID'];
//setting the
            relation
dr = new System.Data.DataRelation('Receiver Details',
            dc1, dc2);
// Add the relation to the
            DataSet
ds.Relations.Add(dr);
return ds;
}

As well as these four methods, we also have another three web methods, which are all basically used to get general information like DepartmentList, AddressBook and UserName.

WebMethod Name | Purpose
getAddressBook | This web method is used to get all the email ids from the address book.
getDeptDetails | This web method is used to get the available department name along with the department code.
getUserName | This web method is used to get the EmailID along with the department code and department name.

Below is the screen shot of the web page rendered by the .NET Framework:

WebMailClient

Now we move to the last part of this case study. Here we build a user interface, which helps to send and receive e-mails. This interface is also used to keep track of all the mails sent using this software.

WebMailClient has a single WinForm. This does four different activities. For each of these activities we have a separate panel control. Before going further, let’s take a look at the four different activities:

  • Login to the System
  • Sending Mails
  • Receiving Mails
  • Mail Analysis

The ClientWebMail class contains a set of private variables that are used to hold information throughout the application.

public class ClientWebMail :
            System.Windows.Forms.Form
{
private string m_UserName =
            '';
private string m_Password = '';
private string m_DeptCode
            = '';
private bool m_LoggedIn = false;
private string
            XMLFilePath= Application.StartupPath + @'\';
// to store
            Addressbook info
DataSet dsAddress=new DataSet();
// to store
            departmentlist info
DataSet dsDept= new DataSet();
// to store
            username with DeptCode
DataSet dsUserName= new DataSet();
//
            to store MailInfo for analysis
DataSet dsMailInfo= new
            DataSet();
// creating a instant of the
            webservice
WebMailService WebMailWS= new WebMailService
            ();

Login to the System

The remote web server has to authenticate the user before he can access the system.

A panelControl panelDefault is used as a container to get the UserName and Password for authentication.

cmdLogin_Click Event

The cmdLogin_Click event is triggered when the user clicks the cmdLogin button after entering the UserName and Password. This event calls the IsUserValid WebMethod for authenticating the user. Once the user is authenticated, the event sets the variables m_UserName and m_Password to the username and password respectively, and then checks for the existences of the AddressBook XML file of that user. The AddressBook XML file is the one that stores the EmailID of the users using this Software. If the XML file does not exist, then a new one is created and will store the AddressBook data using the WebMethod getAddressBook. The data in the XML file is used to populate the EmailID’s in the listbox lstAddress. After populating the lstAddress, the cmdLogin_Click event populates the dataGrid with the mail information from the XML file that stores the downloaded mail information. If the XML file does not exist, then it will create a new one using the WebMethod MailDetails, which returns all the mail information from the remote database. We can even synchronize the XML file with updated information from the remote database. The cmdLogin_Click event also populates the cboDept with department codes using another WebMethod, getDeptDetails. The getUserName WebMethod is called to get the Department Code. Since most of the information is gathered from the remote WebService, we need not connect to the remote WebService regularly.

private void cmdLogin_Click(object
            sender, System.EventArgs e)
{
// checking for the textbox -
            empty
if (txtLoginID.Text==''
            ||txtLoginPassword.Text=='')
{
MessageBox.Show ('Enter
            UserName and
            Password');
}
else
{
statusBar.Panels[1].Text
            ='Connecting to WebService for User Validation';
//calling the
            remote webmethod
if
            (WebMailWS.IsUserValid(txtLoginID.Text.Trim(),txtLoginPassword.Text.Trim()))
{
statusBar.Panels[1].Text = 'Login
            Succeded';
statusBar.Panels[0].Text = txtLoginID.Text.Trim() + '
            is Logged In.';
// assigning the value
m_LoggedIn
            =true;
m_UserName=txtLoginID.Text.Trim();
m_Password=txtLoginPassword.Text.Trim();
string
            fileXML= XMLFilePath + txtLoginID.Text.Trim() + '_AddressBook.xml';
statusBar.Panels[1].Text = 'Checking for Address Book XML
            file..';
// checking for addressbook XML exist
FileInfo fi =
            new FileInfo (fileXML);
if
            (!fi.Exists)
{
statusBar.Panels[1].Text = 'File Does Not
            exist';
statusBar.Panels[1].Text = 'Getting the AddressBook
            information from WebService';
// If file does not exist
            Addressbook info
// got using the Webservice
DataSet DsAddXML=
            WebMailWS.getAddressBook ();
XmlDataDocument XD=new
            XmlDataDocument(DsAddXML);
statusBar.Panels[1].Text = 'Creating
            the XML File..';
// saving the addressbook XML
            file
XD.Save(fileXML);
}
statusBar.Panels[1].Text =
            'Popualting the Address Book';
statusBar.Panels[1].Text
            ='Reading the Address Book XML File';
dsAddress.ReadXml(fileXML);
statusBar.Panels[1].Text ='Populating the Address Book In
            ListBox';
lstAddressBook.DataSource
            =dsAddress;
lstAddressBook.DisplayMember='EMails.EMailID';
lstAddressBook.ValueMember
            ='EMails.EMailID';
fileXML= XMLFilePath + txtLoginID.Text.Trim()
            + '_MailInfo.xml';
FileInfo fiWS = new FileInfo (fileXML);
if
            (!fiWS.Exists)
{
statusBar.Panels[1].Text = 'Connecting to the
            WebService..';
WebMailWS.Credentials
            =System.Net.CredentialCache.DefaultCredentials;
statusBar.Panels[1].Text = 'It may take some time to download
            the
            MailInfo..';
dsMailInfo.Merge(WebMailWS.MailDetails(m_UserName,m_Password));
statusBar.Panels[1].Text = 'Getting the Mail
            Information..';
dsMailInfo.WriteXml(fileXML,XmlWriteMode.WriteSchema
            );
statusBar.Panels[1].Text = 'Populating the Information in the
            Grid';
//populating the mail info in
            DataGrid
dataGrid1.SetDataBinding(dsMailInfo,'SendMail');
}
else
{
statusBar.Panels[1].Text
            = 'Getting the Mail Information from Local
            Database';
dsMailInfo.ReadXml(fileXML,XmlReadMode.ReadSchema
            );
dataGrid1.SetDataBinding(dsMailInfo,'SendMail');
}
if
            (cboDept.Items.Count==0)
{
dsDept=WebMailWS.getDeptDetails();
dsUserName=WebMailWS.getUserNames
            ();
DataViewManager dvmDept = new
            DataViewManager();
dvmDept.DataSet =
            dsDept;
dvmDept.DataViewSettings[0].Sort =
            'Deptcode';
dvmDept.DataViewSettings[0].RowFilter='Distinct
            Deptcode';
//populating the DeptCode in combo
            using
//GetDeptDetails WebMethod
cboDept.DataSource
            =dvmDept;
cboDept.DisplayMember='Department.DeptName';
cboDept.ValueMember
            ='Department.DeptCode';
DataViewManager dvmDeptCode = new
            DataViewManager();
dvmDeptCode.DataSet =dsUserName;
dvmDept.DataViewSettings[0].RowFilter= 'EmailID='' + m_UserName
            + ''';
// getting the DeptCode of the loggedin
            user.
m_DeptCode =
            dvmDept.DataViewSettings[0].Table.Rows[0].ItemArray.GetValue(0).ToString
            ();
}
statusBar.Panels[1].Text ='';
panelDefault.Visible
            =false;
panelAnalysis.Visible =true;
Form.ActiveForm.Text
            ='WebMailClient -
            Analysis';
}
else
{
statusBar.Panels[1].Text = 'User
            Validation Fails Enter Correct UserName and
            Password';
}
}
}

Sending Mails

The next activity that is performed using the ClientWebMail class is sending mails. The PanelControl panelWrite is used as a container for other controls to get the mail information.

cmdSendMail_Click Event

The cmdSendMail_Click event is triggered when the user clicks the cmdSendMail button after entering the mail information. An instance of the SendMail class is created for sending the mail. After sending the mail, the mail information is stored in the remote database using the WebMethod SendMailInfo. Finally, the mail information is also stored in the XML file using the XmlDataDocument.

private void cmdSendMail_Click(object
            sender, System.EventArgs e)
{
try
{
XmlDataDocument
            XD=new XmlDataDocument();
string XMLBuilder = '';
if
            (txtUserName.Text==''
            ||txtHost.Text=='')
{MessageBox.Show('Information as to be
            Filled');}
else
{
//sending mail to remote
            server
statusBar.Panels[1].Text = 'Preparing to Send
            Mail..';
SendMail SendMail=new
            SendMail();
SendMail.SMTPServer=txtHost.Text.Trim();
SendMail.FromName
            =txtUserName.Text.Trim();
SendMail.From=txtUserName.Text.Trim() +
            '@' + txtHost.Text.Trim() ;
SendMail.To
            =lstAddressBook.Text.Trim()
            ;
SendMail.Subject=txtMailID.Text.Trim() + '#' + m_DeptCode + '#'
            + txtRefID.Text.Trim() + '#'
+
            txtSubject.Text.Trim();
SendMail.Body=txtBody.Text.Trim();
statusBar.Panels[1].Text
            = 'Sending Mail..';
SendMail.Send();
statusBar.Panels[1].Text
            = 'Mail has been Sent';
statusBar.Panels[1].Text = 'Preparing to
            Update the Mail info in WebService';
//updating the mail
            information n remote
            db
WM.SendMailInfo(m_UserName,m_Password,txtMailID.Text.Trim(),txtRefID.Text.Trim(),
lstAddressBook.Text.Trim(),m_DeptCode,txtSubject.Text.Trim
            (),txtBody.Text.Trim());
statusBar.Panels[1].Text = 'Updated the
            WebServie..';
string fileXML= XMLFilePath +
            txtLoginID.Text.Trim() + '_SendMail.xml';
statusBar.Panels[1].Text = 'Checking for Address Book XML
            file..';
// checking for addressbook XML exist
FileInfo fi =
            new FileInfo (fileXML);
if (!fi.Exists)
{
XD.LoadXml('');
XD.Save(fileXML);
}
statusBar.Panels[1].Text =
            'Writing the information in Local XML file.';
//creating the XML
            Structure
XMLBuilder += '';
XMLBuilder += ''
            + txtMailID.Text.Trim() + '';
XMLBuilder += '' +
            txtRefID.Text.Trim() + '';
XMLBuilder += '' +
            m_DeptCode + '';
XMLBuilder += '';
Regex
            Regex=new Regex(',');
string[] parts =
            Regex.Split(lstAddressBook.Text.Trim());
foreach(string t in
            parts)
{
XMLBuilder += '' + t +
            '';
}
XMLBuilder += '';
XMLBuilder
            += '' + txtSubject.Text.Trim() +
            '';
XMLBuilder += '' + txtBody.Text.Trim() +
            '';
XMLBuilder += '';
XmlNode currNode;
//Writing to the XML file
XmlDataDocument XDWriteMail = new
            XmlDataDocument ();
XDWriteMail.Load(fileXML);
XmlDocumentFragment docFrag =
            XDWriteMail.CreateDocumentFragment();
docFrag.InnerXml=
            XMLBuilder;
currNode =
            XDWriteMail.DocumentElement;
currNode.InsertBefore(docFrag,
            currNode.FirstChild );
//save the output to a
            file
statusBar.Panels[1].Text = 'Saving the XML
            File..';
XDWriteMail.Save(fileXML);
txtMailID.Text ='MID' +
            Guid.NewGuid().ToString();
statusBar.Panels[1].Text = 'New mail
            Id is generated you can send new mail
            now..';
}
}
catch(Exception
            ex)
{
statusBar.Panels[1].Text
            =ex.ToString();
}
}

cmdUpdateAddress_Click Event

The panel control panelWrite contains a command button cmdUpdateAddress. On clicking this, the cmdUpdateAddress_Click event gets triggered. This event helps to update the EMailID listed in the listbox lstAddress (lstAddress gets populated once the user is authenticated), with the latest one available in the remote database using the getAddressBook web method. This event also gives an option to update the AddressBook XML file with the latest Email Address from the remote database.

private void
            cmdUpdateAddress_Click(object sender, System.EventArgs e)
{
//calling the WebMethod
dsAddress=
            WM.GetAddressBook();
string message = 'Do you want to update the
            Client AdressBook?';
string caption = 'Update Client AddressBook
            ';
MessageBoxButtons buttons =
            MessageBoxButtons.YesNo;
DialogResult result;
result =
            MessageBox.Show(this, message, caption,
            buttons,
MessageBoxIcon.Question,
            MessageBoxDefaultButton.Button1,
MessageBoxOptions.RightAlign);
if(result
            == DialogResult.Yes)
{
//saving the latest adddressbook in XML
            file
string fileXML= XMLFilePath + txtLoginID.Text.Trim() +
            '_AddressBook.xml';
XmlDataDocument XD=new
            XmlDataDocument(dsAddress);
XD.Save(fileXML);
}
//populating the lstAddressBook with latest
            AddressBook
lstAddressBook.BindingContext['dsAddress'].SuspendBinding();
lstAddressBook.DataSource
            =dsAddress;
lstAddressBook.DisplayMember='EMails.EMailID';
lstAddressBook.ValueMember
            ='EMails.EMailID';
lstAddressBook.Refresh();
}

Receive Mail

The next activity that can be done using the ClientWebMail class is to check for new mail. The panel control panelChecMail is used as a container for showing the mail received.

menuCheckMail_Click Event

The menuCheckMail_Click event helps to get the e-mail from the remote mail server, update the received information in the remote database, and also updates the local XML file with the received mail information.

The code below creates an instant of the POP3Client class and builds a DataTable dtMailBox to store the received mail information.

private void menuCheckMail_Click(object
            sender, System.EventArgs e)
{
if
            (m_LoggedIn)
{
Pop3Client clnt = new
            Pop3Client(m_UserName,110);
string MailID=null;
string
            RefID=null;
string Date=null;
string From=null;
string
            Dept=null;
string Subject=null;
string Body=null;
string
            fileXML= XMLFilePath + txtLoginID.Text.Trim() + '_MailReceived.xml';
Regex Regex =new Regex('#');
string[]
            subjectArray;
DataSet dsOffLineMailinfo =new
            DataSet('MailInfo');
DataSet dsMailinfo =new
            DataSet('MailInfo');
DataTable dtMailBox=new
            DataTable('MailBox');
DataRow
            newRow;
dtMailBox.Columns.Add('MailID');
dtMailBox.Columns.Add('RefID');
dtMailBox.Columns.Add('DeptCode');
dtMailBox.Columns.Add('Sender');
dtMailBox.Columns.Add('MailSize');
dtMailBox.Columns.Add('Date');
dtMailBox.Columns.Add('Subject');
dtMailBox.Columns.Add('Body');

Next we get all the mail messages from the mail server using the getMessageList and getMessage methods of the POP3Client class. The received messages are split into MailID, RefID, Subject, Body, etc, by the getString function and are stored in the DataTable dtMailBox.

if(clnt.LogIn(m_UserName ,m_Password))
{
// Get messages list from server
Pop3MessagesInfo
            msgsInfo = clnt.GetMessagesList();
// Loop all messages and show
            them
foreach(Pop3Message m in
            msgsInfo.MessageInfo)
{
byte[] mime =
            clnt.GetMessage(m.MessageNr);
string
            str=System.Text.Encoding.ASCII.GetString(mime);
From =getString
            (str,'From:','To:');
Subject=getString
            (str,'Subject:','SendDate:');
subjectArray =
            Regex.Split(Subject);
if (subjectArray.GetUpperBound(0)>=
            3)
{
MailID= (string)subjectArray.GetValue(0);
Dept=
            (string)subjectArray.GetValue(1);
RefID=
            (string)subjectArray.GetValue(2);
Subject =
            (string)subjectArray.GetValue(3);
}
else
{
continue;
}
Date=getString
            (str,'SendDate:','MIME-Version:');
Body=getString
            (str,'STARTBODY','ENDBODY');
if
            (isMailReceived(MailID.Trim()))
{
newRow=
            dtMailBox.NewRow();
newRow['MailID']=MailID.Trim();
newRow['RefID']=RefID.Trim();
newRow['DeptCode']=Dept.Trim();
newRow['Sender']=From.Trim();
newRow['MailSize']
            =
            m.Size.ToString();
newRow['Date']=Date.Trim();
newRow['Subject']=Subject.Trim();
newRow['Body']=Body.Trim();
dtMailBox.Rows.Add(newRow);
}
}
clnt.LogOff();
int
            intMessageCount=dtMailBox.Rows.Count ;

Next the newly-received mail stored in the datatable dtMailBox are used to populate the listview lstReceiveMail control. To find the new mails we compare the MailId that is gotten from the received mails with the MailID's already stored in the XML file using the function IsMailReceived. The remote database is also updated with the new mail information using the WebMethod ReceivedDBUpdate.

for (int i=0; i <
            dtMailBox.Rows.Count; i++)
{
lstReceiveMail.Items.Add(dtMailBox.Rows[i].ItemArray.GetValue(0).ToString(),0);
lstReceiveMail.Items[i].BackColor=System.Drawing.Color.LightGray
            ;
lstReceiveMail.Items[i].SubItems.Add(dtMailBox.Rows[i].ItemArray.GetValue(1).ToString());
lstReceiveMail.Items[i].SubItems.Add(dtMailBox.Rows[i].ItemArray.GetValue(2).ToString());
lstReceiveMail.Items[i].SubItems.Add(dtMailBox.Rows[i].ItemArray.GetValue(3).ToString());
lstReceiveMail.Items[i].SubItems.Add(dtMailBox.Rows[i].ItemArray.GetValue(4).ToString());
lstReceiveMail.Items[i].SubItems.Add(dtMailBox.Rows[i].ItemArray.GetValue(5).ToString());
lstReceiveMail.Items[i].SubItems.Add(dtMailBox.Rows[i].ItemArray.GetValue(6).ToString());
}
DataSet
            dsWS = new
            DataSet();
dsWS.Tables.Add(dtMailBox);
WebMailWS.ReceivedDBUpdate(m_UserName
            ,m_Password,dsWS)
public bool IsMailReceived(string MailID )
{
string fileXML= XMLFilePath + txtLoginID.Text.Trim() +
            '_MailReceived.xml';
FileInfo fi=new FileInfo(fileXML);
if
            (fi.Exists)
{
XmlDocument doc = new
            XmlDocument();
doc.Load(fileXML);
XmlNodeList elemList =
            doc.GetElementsByTagName('MailID');
for (int i=0; i <
            elemList.Count; i++)
{
if
            (MailID.Equals(elemList[i].InnerXml.Trim()))
{
return
            false;
}
}
return true;
}
else
{
return
            true;
}
}

Now we again populate the Listview with the mail information stored previously in the XML file and then update the new mail information in the XML file.

FileInfo fi=new FileInfo(fileXML);
if(fi.Exists)
{
dsOffLineMailinfo.ReadXml(fileXML);
}
if (dsOffLineMailinfo.Tables.Count > 0 )
{
for (int
            j=0; j < dsOffLineMailinfo.Tables[0].Rows.Count ;
            j++)
{
lstReceiveMail.Items.Add(dsOffLineMailinfo.Tables[0].Rows[j].ItemArray.GetValue(0).ToString(),0);
lstReceiveMail.Items[j].SubItems.Add(dsOffLineMailinfo.Tables[0].Rows[j].ItemArray.GetValue(1).ToString());
lstReceiveMail.Items[j].SubItems.Add(dsOffLineMailinfo.Tables[0].Rows[j].ItemArray.GetValue(2).ToString());
lstReceiveMail.Items[j].SubItems.Add(dsOffLineMailinfo.Tables[0].Rows[j].ItemArray.GetValue(3).ToString());
lstReceiveMail.Items[j].SubItems.Add(dsOffLineMailinfo.Tables[0].Rows[j].ItemArray.GetValue(4).ToString());
lstReceiveMail.Items[j].SubItems.Add(dsOffLineMailinfo.Tables[0].Rows[j].ItemArray.GetValue(5).ToString());
lstReceiveMail.Items[j].SubItems.Add(dsOffLineMailinfo.Tables[0].Rows[j].ItemArray.GetValue(6).ToString());
}
}
dsOffLineMailinfo.Merge(dtMailBox);
dsOffLineMailinfo.WriteXml(fileXML);

Mail Analysis

The final activity that is done using the ClientWebMail class is analyzing the mail received. The panel control panelAnalysis is used as a container for showing the mail information that is populated in a dataGrid by the time the user has logged in to the system. The mail information taken from the remote database using the web method GetMailDetails is stored in a dataset dsMailInfo. A search criterion is provided to refine the mail information with respect to department code, Username, and by date using the DataView searchDV object.

private void cmdSearch_Click(object
            sender, System.EventArgs e)
{
DataView searchDV = new
            DataView();
if
            (cboDept.Items.Count!=0)
{
searchDV=dsMailInfo.Tables['SendMail'].DefaultView
            ;
searchDV.RowFilter ='DeptCode = '' + cboDept.SelectedValue +
            ''';
if(!cboUName.Text.Equals (''))
{
searchDV.RowFilter
            ='Sender = '' + cboUName.SelectedValue +
            ''';
}
searchDV.RowFilter ='SendDate >= '' + dtpFrom.Value
            + '' and SendDate <='' + dtpTo.Value + ''';
dataGrid1.SetDataBinding( searchDV ,'');
}
}

Scope for Improving the WebMail

We have now developed a WebMail mail client. Even though most features have been provided, we can still improve both the design and functionality of the WebMail. We can:

  • Provide an option to delete the mails both in the Mail server as well in the remote database.
  • Provide an option to send an attachment with the mail could be provided.
  • Since the mail information can be stored in the local disk, more analyzing options could be provided.
  • We could provide an option to get part of the mail information from the remote server, thus avoided bulk download of mail information from remote server.
  • We could improve the interface of the WebMailClient by adding HTML support.
  • We could increase the security of the XML file stored in the local disk by encrypting the file.

Summary

In this case study we have tried to develop a real time application, which helps to track and analyze mail. We also have developed a send and receive mail component that can be easily integrated with the other application.

As we have seen, this case study has been divided into three parts for better understanding; the first part explains the process involved to build the send and receive mail component, the second part explains the process involved for login authentication and for updating the mail information in a remote database using a WebService, while the third part explains the interface used to send and receive mails and to do mail analysis.

Case Study Information
Author | Vinod Kumar
Editor | Adam Ryland
Reviewers | Andrew Krowczyk, Saurabh Nandu, Shefali Kulkarni

If you have any questions or comments about this case study, or suggestions for further pieces, please contact the technical editor. If you wish to talk about the case study with other subscribers, please visit our forum here.

C#Today - © 2002 WroxPress Limited


Building a List Server

Building a List Server
http://www.csharptoday.com/content.asp?id=1763
Article Rating: not yet rated
Published on April 8, 2002
Abstract
In this case study, Matthew Reynolds takes a look at how we can build a simple list server. This list server is designed to handle “jobs?of several thousand e-mails, and handle returned and bounced e-mails. We will see how to build a single Windows Service application that we can put somewhere on our network and allow several computers on the network to queue messages for distribution. For message distribution, we’ll use Microsoft SMTP Service. However, we’ll actually design the system so that we can use a different service for message distribution. For example, we’ll create an “engine?that rather than connecting to the Microsoft SMTP Service through the System.Web.Mail namespace, uses Exchange or perhaps some third-party component.
Case Study

In this case study, I'm going to be taking a look at how we can build a simple list server. This list server is designed to handle "jobs" of several thousand e-mails, and handle returned and bounced e-mails.

What we want to be able to do is build a single Windows Service application that we can put somewhere on our network and allow several computers on the network (or perhaps from outside the firewall on another network or the Internet) to queue messages for distribution. We're not going to concern ourselves with maintaining a database containing the potential recipients of the e-mails, or tools to query this database and ultimately send e-mails to them - we'll simply create an endpoint that we can send the messages to and have our application send them.

For message distribution, we'll use Microsoft SMTP Service that comes as standard with various flavors of Windows 2000. However, we'll actually design the system so that we can use a different service for message distribution. For example, we'll create an "engine" that rather than connecting to the Microsoft SMTP Service through the System.Web.Mail namespace, uses Exchange or perhaps some third-party component like the ones that have been in common use by ASP developers for a while now, like ASPEmail and JMail.

For message return, we have a number of options. Typically, should a message be rejected, a message will be sent to the address specified as the sender in the originating message, for example, bounce@wrox.com. We can pick this message up using a POP3 component, and we could either write one of these ourselves, or reuse one. In this particular example, we're going to be using a third-party component. (The one we're going to use offers a free trial, although you'll find a mix of free and paid components on the market. We'll discuss this in more detail when we get to the point in the exercise where we need to handle incoming e-mails).

In terms of designing the actual service itself, as we've already said, what we need is a known endpoint that we can send messages to where they will be queued and ultimately sent on their way to the final destination. Those of you thinking in a typical .NET fashion may at this point think "Web service". What I'm keen to do is to skirt around this a little and make our approach capable of supporting a Web service, but actually design it from a perspective of using Remoting rather than Web services.

Remoting vs. Web Services

While Web services is a cool technology that should (in theory) provide ultimate platform interoperability - meaning in this case that UNIX clients could queue messages to our Windows list server - that interoperability does come at a price. Having to package every call in SOAP, which involved numerous type conversions that are particularly wasteful when turning integer values into text values and back again, and having to work over HTTP, while quick is not as efficient as it could be. If we're confident that both our client and server are both going to be implemented on Windows, or specifically .NET on Windows, we can do this more efficiently.

This is where Remoting comes in. Remoting and Web services share a common code base in the Framework, and you can think of Remoting as being an extensible, proprietary object distribution technology that does a similar thing to Web services. You expose an object on the network and clients can connect over that network to call methods on that object.

Remoting, however, is not tied to the "SOAP over HTTP" paradigm that Web services are. We can, if we want, send a stream of binary data over a TCP socket. This is substantially more efficient than Web services because the fact we're sending binary data means we don't have to convert the values in order to get them into an XML document, and the fact that we're using TCP means we don't get the overhead of HTTP. What we do lose with "Binary over TCP" is interoperability. Sending binary data in this way assumes that the client machine represents data internally in the same manner that the server machine does. This may not be true, and is in fact why Web services works so well.

Imagine you have a number represented internally by Computer A. This computer will take a certain amount of memory (say two, four, eight bytes, whatever really) to represent the number and write a string of binary digits into this block. If you take this "pattern" from Computer A and move it to Computer B that is the same platform (e.g. two Windows computers), the number represented remains constant. However, if Computer C is a different platform, the pattern may not represent the same number. To get around this problem, SOAP calls for the numbers to be converted to text, for which common standards exist. Or rather, the common standard doesn't matter, as each platform will choose its own internal data representation standard for its own ends.

What's important here is that although we're saying that we're not going to use Web services, we actually can. Web services are a superb technology for gluing an alternative access layer on top of an existing layer. What we can do is create a separate Web service that sits on top of the Remoting architecture that enables other platforms to use our system.

Here's a diagram that represents where we're trying to get to:

The actual message transfer and delivery outside of the private network through the firewall isn't of any concern to us. We'll rely on other products to actually do this for us - i.e. Microsoft SMTP Service to distribute the messages to the remote SMTP server; the remote SMTP server will deal with distributing the messages to the recipients, or return it to the bounce mailbox; the bounce mailbox will be managed by another mail server application, and so on.

The Object Model

What we're looking to do is build a single class library that we can include in our client applications that has the capability of connecting to the remote service (although we say "remote", the service could be on the same computer). Each client application will be able to create a new "job" (which is a collection of outbound messages - such as one customer service e-mail, or 10,000 newsletter messages), and once the job has been created that application will be able to queue messages to that job. At some point in the future, once all of the messages have been queued, either the client application or another application will be able to say, "OK, process the job", at which point all of the queued messages will be sent on their way.

Here's a list of some of the classes that we'll need:

  • Service -one instance of this class will exist inside of the Windows Service process (when we're debugging, we'll be using a Console Application, not a Windows Service). This class is responsible for setting up Remoting and managing jobs.
  • ServiceConnector - this is the Remoting endpoint that the client applications will connect to. It enables them to create new jobs.
  • Job - this class will be responsible for managing a job list. Among others, it contains a method called Enqueue that is used to queue the message.
  • Message - this class will contain the message that will be sent.
  • Recipient - this class represents a name and e-mail address pair.
  • Configuration - this class contains the configuration of the service.
  • SelfSerializer - a helper class that enables a class to serialize itself to and de-serialize itself from an XML document. We'll use this to persist a couple of the other classes (specifically Message and Configuration) to disk.

Queue Structure

What we haven't looked at so far is how we will actually store the messages. There are a few approaches that we can use for this. The method that I'm going to suggest involves creating a file and folder structure on disk. In your case, it might be appropriate to use a database like SQL Server as the message store, or even Microsoft Exchange Server.

When a new job is created, a new folder will be created to contain messages for that job. As messages are queued, an XML document representing the message will be saved to the job folder.

Building the Service

To kick off, we'll build the ListServer project. This project will contain all of the classes that we need to support the service, and also to connect to the service from a client application.

The first thing we need to do is get the file structure on disk configured, and also to get the ServiceConnector object made available over Remoting. This information will be stored in a configuration file, which not surprisingly we'll manage through a class called Configuration. This class will contain fields to hold the basic Remoting information and some other configuration information.

We're going to hold this information in an XML document. What's more, we're going to give the Configuration class the capability of saving and loading this document whenever it's appropriate to do so. A pattern that I recommend for doing this is to create a new class capable of performing serialization and de-serialization and extending that class with specific "document" classes. In this application, the base class is called SelfSerializer, and we'll derive Configuration from this. The pattern also calls for a static Load method (which you can consider to be similar to a constructor, but one that takes a filename. Using a separate method called Load is more intuitive to developers consuming your class than having a curious looking constructor that appears to take a filename) and an instance Save method.

This class also needs to be decorated with the System.SerializableAttribute attribute:

[Serializable()] public class
            SelfSerializer
{
public SelfSerializer()
{
}

SelfSerializer will use System.Xml.Serialization.XmlSerializer to do its magic. We'll also need the System.IO namespace to provide file I/O functionality, so add these namespaces to SelfSerializer:

using System;
using System.IO;
using System.Xml.Serialization;

The Save method is pretty straightforward. We've created two separate versions of the method : one that takes a filename and one that takes a System.IO.Stream object. In the filename version, we need to create a System.IO.FileStream and pass the new stream to the alternate version of the method:

// Save - save the file to disk...
public
            void Save(string filename)
{
// file...
FileStream stream =
            null;
try
{
// open...
stream = new FileStream(filename,
            FileMode.Create);
Save(stream);
}
finally
{
if(stream
            != null)
stream.Close();
}
}

XmlSerializer really is very easy to use! All we need is a System.Type object representing whatever it is we're trying to serialize and then call the Serialize method:

public void Save(Stream stream)
{
// serialize it...
XmlSerializer serializer = new XmlSerializer(this.GetType());
serializer.Serialize(stream, this);
}

The Load method is a little more complicated. When we create XmlSerializer we need to provide a Type object that tells it how to interpret the file. However, from inside a static method, we don't have access to the GetType method as we do in Save. What this means is that we need to create protected methods that take the Type that we want to use and create a static method inside of the derived class. Here are the Load methods:

// Load - we don't know the class, so we
            need that...
protected static object Load(Type type, string
            filename)
{
// file...
FileStream stream =
            null;
try
{
// open the stream...
stream = new
            FileStream(filename, FileMode.Open);
return Load(type,
            stream);
}
finally
{
if(stream !=
            null)
stream.Close();
}
}

protected static object
            Load(Type type, Stream stream)
{
// deserialize
            it...
XmlSerializer serializer = new
            XmlSerializer(type);
return
            serializer.Deserialize(stream);
}

Create the new Configuration class and derive it from SelfSerializer. Add fields to support the list owner and the Remoting details (ports and "application name"). Don't worry too much about the port numbers that I've chosen here. They're arbitrary, but be advised that ports are "free for all" on a particular computer, so some other service may already be running on that port, in which case you'll need to select another.

public class Configuration : SelfSerializer
{
// fields...
public string ListOwnerAddress = "bounce@wrox.com";
public int TcpPort = 8000;
public int HttpPort = 8001;
public string ApplicationName = "WroxListServer";

When we call the Load method, we'll give it the name of a config.xml file. For our purposes, we're going to assume that this file always exists in the c:\Wrox ListServer folder. If we attempt to load this file and the folder doesn't exist, we'll need to create it. Likewise, if the file doesn't exist, we'll need to create a default version of it - which is why we have default values stored against each of the members in the Configuration file.

Here's the implementation of Load that needs to be added to Configuration:

// Load - de-serialize the configuration
            from disk...
public static Configuration Load(string
            filename)
{
Configuration config = null;

// do we have
            this file?
FileInfo info = new
            FileInfo(filename);
if(info.Directory.Exists ==
            false)
info.Directory.Create();
if(info.Exists ==
            false)
{
// create a new configuration file...
config = new
            Configuration();

// save the
            file...
config.Save(filename);
}
else
config =
            (Configuration)Load(typeof(Configuration),
filename);

//
            return it...
return config;
}

Inside of the actual service process, we're going to have one instance of Service running. Primarily, the responsibility of this object is to manage the queues, but it also performs some ancillary functions such as spinning up the threads to support bounced e-mail collections. The constructor for Service needs to accept a Configuration instance. In addition to this, we also need to be able to get at the current, active instance of Service from anywhere else inside the process, which we'll do through a static property called CurrentService.

public class Service
{
// fields...
private static Service _currentService;
private Configuration _configuration;

public Service(Configuration configuration)
{
// store...
_currentService = this;
_configuration = configuration;
}

public static Service CurrentService
{
get
{
return _currentService;
}
}

public Configuration Configuration
{
get
{
return _configuration;
}
}
}

Exposing "ServiceConnector" Over Remoting

When we want to start the service running, so that we can create jobs and ultimately queue and distribute messages, we'll load the configuration file from disk, create a new instance of Service from this, and call the StartService method. At this stage, all we want this to do is establish the Remoting channels and make the ServiceConnector class available.

Before we do that, let's go into a little more depth with Remoting. For Remoting, we need two things: a channel and a formatter. A channel defines how the Remoting message will be transferred (this is a different "message" from the e-mail messages that we've been talking about so far. Think of it as a package containing the details of the remote method and the parameters). We've already mentioned "HTTP" and "TCP" in our earlier discussions. These are two examples of channels. A formatter defines how the message will be structured. We've already spoken about these too - we have "SOAP" and "Binary" provided as standard with the Framework.

The other consideration that we have is that of object creation and lifetime. When we're using objects with Remoting, there are three different kinds of object creation: singleton, single call and client activated. Client-activated objects are used in situations where you need to specifically create an instance of an object on a remote machine, which if you bear with me is not what we actually need to do.

The type of Remoting object that we want is "singleton". This object is designed for highly scalable use. What happens is that the first inbound request for a singleton object causes an instance of the object to be created. This single object instance is shared across all requests from that point until some point in the future when the object is closed down. If 10,000 clients all make a request to methods on that object, only one instance of the object is ever created. This is great from a scalability perspective. It's important that this kind of object does not hold state, as this can cause all sorts of scalability and thread synchronization problems.

This is why we have a separate ServiceConnector object. It's this object that is actually registered with Remoting and it's responsible for finding and deferring the call onto a method on the single instance of Service that actually controls the service. If you can imagine, we don't want to control the lifetime of this singleton object - we'd rather Remoting does this for us. When an instance is created to support an inbound call, we don't care what instance is actually used to do this. What we do care about is finding the all-important Service instance.

Single-call is the final type of Remoting object. This type of object exists for a "single call" - i.e. it is created to support an inbound call and it is then disposed of after that call. Not as good from a scalability perspective as we either have zero or some unknown number of objects taking up memory, rather than a singleton approach where we either have zero or one.

For both singleton and single-call objects, we must inherit from System.MarshalByRefObject. This is a special class that tells Remoting that the object should never move outside of the application domain that it was created in. Imagine we have a method on a remote object and as the return value from the object we pass an instance of a class called Customer. By default, Remoting will attempt to serialize that object and pass it back down the wire to the caller, whereupon it's reconstituted into an object and used as normal. However, if this object extends MarshalByRefObject, that object will not be moved. Instead, the object will remain executing in the remote process space and what will be returned will be a reference to that object. Subsequent calls will be passed down the wire back to the newly created remote object. This is an important point, which we'll cover in more detail later.

When it comes to registering the object with Remoting, we need a URI (Uniform Resource Identifier). This URI will be requested by the remote client as an endpoint to connect to. A good approach to use is to combine an arbitrary application name with the name of the type followed by the identifier ".remote", for example:

ApplicationName/NamespaceName.TypeName.remote

So, if we have an application name of MyApplication, and a type name of MyNamespace.MyObject, we'd get this:

MyApplication/MyNamespace.MyObject.remote

To access that object, we need to know the protocol, the hostname of the server, and the port. So, if we have an HTTP channel on port 8001 on CORRADO (the name of my file server here in the office), the full URL would be:

http://corrado:8001/MyApplication/MyNamespace.MyObject.remote

However, when we pass the URI to System.Runtime.Remoting.RemotingConfiguration when we want to actually register the object, we don't need to specify the port, protocol or hostname as this is inferred from the settings on the machine, or particular in the channel configuration. Remember, when we come to register the channels we'll provide the ports and the protocols that we need.

Before adapting Service so that it can configure the Remoting settings, create the new ServiceConnector class. This class has to extend System.MarshalByRefObject.

public class ServiceConnector :
            MarshalByRefObject
{
public
            ServiceConnector()
{
}
}

In Service, we need a fairly broad set of namespaces to be available. Add these to the Service class:

using System;
using System.IO;
using System.Threading;
using System.Collections;
using System.Runtime.Remoting;
using System.Runtime.Remoting.Channels;
using System.Runtime.Remoting.Channels.Tcp;
using System.Runtime.Remoting.Channels.Http;

In addition to this, you'll also need to add a reference to the System.Runtime.Remoting.dll assembly otherwise you won't have access to all of the namespaces.

Inside the StartService method, the first thing we need to do is to register the channels. The ports from these will be read from the configuration file, and are made available through the TcpPort and HttpPort fields of the Configuration object available through the Configuration property. This is just an issue of creating new TcpChannel and HttpChannel services and passing them to ChannelServices for registration.

// StartService - run the
            service...
public void StartService()
{
// create
            channels...
Console.WriteLine("TCP channel listening on port
            {0}",
Configuration.TcpPort);
IChannel channel = new
            TcpChannel(Configuration.TcpPort);
ChannelServices.RegisterChannel(channel);
Console.WriteLine("HTTP
            channel listening on port {0}",
Configuration.HttpPort);
channel = new
            HttpChannel(Configuration.HttpPort);
ChannelServices.RegisterChannel(channel);

Next, we need to register ServiceConnector with the Remote service:

// register a Remoting end-point...
Type type = typeof(ServiceConnector);
string uri = string.Format("{0}/{1}.remote",
Configuration.ApplicationName, type.ToString());
Console.WriteLine("Listening on {0}", uri);
RemotingConfiguration.RegisterWellKnownServiceType(type, uri,
WellKnownObjectMode.Singleton);

At this point, remote objects can attempt to connect to the remote ServiceConnector object.

Creating the Folder Structure

What we've missed so far is that we don't have any code to make sure that the folder structure is properly configured to manage the lists. Although we've created c:\Wrox ListServer on demand, inside this folder we need three folders: Jobs, Bounce and Unsubscribe.

We'll change the Configuration object so that it handles all of this for us. Firstly, add these three constants to Configuration:

public class Configuration : SelfSerializer
{
// fields...
public string ListOwnerAddress = "bounce@wrox.com";
public int TcpPort = 8000;
public int HttpPort = 8001;
public string ApplicationName = "WroxListServer";

// const...
public const string JobsFolderBaseName = "Jobs";
public const string BounceFolderBaseName = "Bounce";
public const string UnsubscribeFolderBaseName = "Unsubscribe";

In addition, add the private _baseFolder field as well:

public class Configuration : SelfSerializer
{
// fields...
private DirectoryInfo _baseFolder;
public string ListOwnerAddress = "bounce@wrox.com";

Inside the Load method, we need to set the BaseFolder property:

// Load - de-serialize the configuration
            from disk...
public static Configuration Load(string
            filename)
{
Configuration config = null;

// do we have
            this file?
FileInfo info = new
            FileInfo(filename);
if(info.Directory.Exists ==
            false)
info.Directory.Create();
if(info.Exists ==
            false)
{
// create a new configuration file...
config = new
            Configuration();

// save the
            file...
config.Save(filename);
}
else
config =
            (Configuration)Load(typeof(Configuration),
filename);

//
            configure the base folder...
config.BaseFolder =
            info.Directory;

// return it...
return
            config;
}

The BaseFolder property needs to make sure that all of the ancillary folders exist. This is done through a call to CheckFoldersExist. In addition, this property needs to be decorated with System.Xml.Serialization.XmlIgnore because we don't want to save its details when we serialize the document.

[System.Xml.Serialization.XmlIgnore()]
public
            DirectoryInfo BaseFolder
{
get
{
return
            _baseFolder;
}
set
{
_baseFolder =
            value;
CheckFoldersExist();
}
}

Before we build CheckFoldersExist, we need three properties that return System.IO.DirectoryInfo objects for each of the folders. Each of these properties needs to be decorated with XmlIgnore:

[System.Xml.Serialization.XmlIgnore()]
public DirectoryInfo JobsFolder
{
get
{
return new DirectoryInfo(BaseFolder.FullName + "\\" +
JobsFolderBaseName);
}
}

[System.Xml.Serialization.XmlIgnore()]
public DirectoryInfo BounceFolder
{
get
{
return new DirectoryInfo(BaseFolder.FullName + "\\" +
BounceFolderBaseName);
}
}

[System.Xml.Serialization.XmlIgnore()]
public DirectoryInfo UnsubscribeFolder
{
get
{
return new DirectoryInfo(BaseFolder.FullName + "\\" +
UnsubscribeFolderBaseName);
}
}

All that CheckFoldersExist has to do is go through each of the folders making sure that they are in place. We've already checked that BaseFolder exists, but for completeness we'll do this again in case this method is used from a derived class. The name implies that all folders are checked:

protected void
            CheckFoldersExist()
{
if(BaseFolder.Exists ==
            false)
BaseFolder.Create();
if(JobsFolder.Exists ==
            false)
JobsFolder.Create();
if(BounceFolder.Exists ==
            false)
BounceFolder.Create();
if(UnsubscribeFolder.Exists ==
            false)
UnsubscribeFolder.Create();
}

We're still at a point where we can't run the project, but we'll solve this now. Create a new Console Application called ListServerHost inside the existing solution. You can do this by right-clicking on the ListServer solution in Solution Explorer and selecting Add - New Project.

You'll also need to add a reference to the ListServer project. In addition, using Solution Explorer right-click on ListServerHost and select Set as Start-up Project.

Inside the Class1 class, add a namespace import directive to ListServer:

using System;
using ListServer;

namespace ListServerHost

Once you've done that, you can load up Configuration and also start the service running:

[STAThread]
static void Main(string[]
            args)
{
// load the configuration...
Configuration config =
            Configuration.Load(
"c:\\Wrox ListServer\\config.xml");

//
            start up the service...
Service service = new
            Service(config);
service.StartService();

//
            wait...
Console.ReadLine();
}

When the service is running, you'll see this:

At this point, there isn't any functionality in the service that we can use. However, if you look at the c:\Wrox ListServer server folder, at least we can see that we have created the folders and the config.xml file automatically.

Here's what the contents of the config.xml file looks like:

<?xml version="1.0" ?>
<Configuration xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<ListOwnerAddress>bounce@wrox.com</ListOwnerAddress>
<TcpPort>8000</TcpPort>
<HttpPort>8001</HttpPort>
<ApplicationName>WroxListServer</ApplicationName>
</Configuration>

Let's move on and get to a point where we can create jobs.

Creating Jobs

Although we've got a lot of magical plumbing going on, ideally we want to get to a point where the application that actually wants to use the service doesn't have to do very much work at all to consume the service. There's no reason why the application developer needs to consider the fact that the service is actually running remotely. This level of abstraction is a common approach in server-based development. Consider a SQL Server connection string - you don't care if the server is local or remote. It's up to the application to do the abstraction. What's cool here is that all this Remoting magic makes life easy for the server-side developer too, and we don't have to do much to make life typically easy for the client-side developer.

Before when we introduced Remoting, I mentioned briefly about this object called System.MarshalByRefObject that had the effect of pinning an object into a given application domain and prevented it from being serialized and passed down the wire to the caller. We can use this to great effect when working with jobs.

When an application wants to send messages, it needs a "job". Typically it will create this job as part of whatever process involves walking the list of recipients for the message. In order to create a job, the application needs to know:

  • The hostname of the server,
  • The protocol and port that should be used,
  • The application name.

These three items are combined to form the URL of the remote ServiceConnector object. Typically when using Remoting, we'll use the GetObject method on System.Activator to get hold of a proxy capable at connecting to the remote object. However, in our case, what we're going to do is add a static method to ServiceConnector that knows how to form and how to connect to the remote object. First, add this enumeration to ServiceConnector:

public class ServiceConnector : MarshalByRefObject
{
// enum...
public enum ServiceProtocol { Tcp, Http };

Next, add this method to ServiceConnector:

// Connect - connect to the remote service...
public static ServiceConnector Connect(ServiceProtocol protocol,
int port, string hostname, string applicationName)
{
// url...
Type type = typeof(ServiceConnector);
string url = string.Format("{0}://{1}:{2}/{3}/{4}.remote",
protocol.ToString(), hostname, port.ToString(),
applicationName, type.ToString());
Console.WriteLine("Connecting to {0}", url);

// connect...
return (ServiceConnector)Activator.GetObject(type, url);
}

When Connect is called, it looks like the caller will be returned a ServiceConnector object. Instead, the caller will actually be given a "transparent proxy" that supports the same methods and properties as the remote object. The remote object remains pinned into the application domain of the remote service. As in this case the remote object is a singleton, only one instance of this object will ever be created. Subsequent calls to Connect will return a new proxy instance that connects to the same remote instance of the object.

Creating the job is a relatively involved process so let's go through it step-by-step.

The "Job" Class

The first thing we need is a class that represents the job. Imagine we want to send out a newsletter. The first step is to create the job, then populate the job with outbound messages, and then process the job so that the outbound messages are sent. Among other things, this class will need to hold the name of the job and the name of the folder on disk into which new messages will be queued. Folders will not be recycled, so they'll be used for a single job. It's up to the administrator to clear out old jobs (the reason why we do this is because a copy of the sent mail is kept around on disk after it has been sent. From personal experience, I've found it useful to have a copies of these documents should someone need to audit the newsletter batch). A job also needs to keep track of its own state - and this is used to record whether the job is open and is receiving messages, whether it is sending messages, or whether it has closed. Eventually, this class will be able to spin up its own thread in order to send messages on to their final destination, but we're not going to worry about this for now.

The Job class needs to extend MarshalByRefObject - for reasons that I'll explain in a little while. For now, add this code to Job:

using System;
using System.IO;
using
            System.Threading;
using System.Runtime.Serialization;
using
            System.Runtime.Serialization.Formatters.Binary;

namespace
            ListServer
{
public class Job : MarshalByRefObject
{
//
            fields...
private DirectoryInfo _folder;

public
            Job(DirectoryInfo folder)
{
// store...
_folder =
            folder;
}

public DirectoryInfo
            Folder
{
get
{
return _folder;
}
}

public
            string Name
{
get
{
return
            Folder.Name;
}
}
}
}

The "JobCollection" Class

The Service class will need to hold a collection of Job objects. We can do this pretty easily by inheriting from System.Collections.CollectionBase and adding our own implementations for Add and Remove, and by adding our own indexer.

public class JobCollection : System.Collections.CollectionBase
{
public JobCollection()
{
}

public void Add(Job job)
{
List.Add(job);
}

public void Remove(Job job)
{
List.Remove(job);
}

public Job this[int index]
{
get
{
return (Job)List[index];
}
set
{
List[index] = value;
}
}
}

In addition to these methods, it's useful to add methods to find a Job object given the name ("Find") and to quickly discover if the collection already contains a Job with the given name ("Contains").

public Job Find(string jobName)
{
//
            loop...
foreach(Job job in
            InnerList)
{
if(string.Compare(job.Name, jobName, true) ==
            0)
return job;
}
return null;
}

public bool
            Contains(string jobName)
{
if(Find(jobName) != null)
return
            true;
else
return false;
}

Creating new Jobs

To create a job, the application ultimately needs to get hold of an instance of ServiceConnector and call the CreateJob method. We've already seen how the caller can use the static Connect method on this class to get hold of a reference to the remote object. Once the caller has done that, he or she can use the CreateJob method. This method needs to defer to a new method with the same name on the instance of Service that is actually managing the application. This instance is retrieved using the static CurrentService property, like this method, which should be added to ServiceConnector:

public Job CreateJob(string jobName)
{
return Service.CurrentService.CreateJob(jobName);
}

To hold a list of jobs, Service needs a field that holds an instance of a JobCollection object. However, because this service may be being used by multiple applications simultaneously (imagine multiple clients on the network all calling into it), it's imperative that we properly manage thread concurrency issues.

To the Service class, add the _jobs and _jobsLock fields:

public class Service
{
// fields...
private static Service _currentService;
private Configuration _configuration;
private JobCollection _jobs = null;
private ReaderWriterLock _jobsLock = new ReaderWriterLock();

The CreateJob method (this time on Service, rather than on ServiceConnector) accepts a name. Each job exists in its own folder within the c:\Wrox ListServer\Jobs folder. As no two sibling folders can have the same name, no two jobs can have the same name either. For this reason, when we're given the name of a job, we have to find the next available name for it. For example, if MyMailingList is already taken, we'll try MyMailingList_2. If that's taken, we'll try MyMailingList_3, and so on.

To properly handle the thread synchronization, we'll only allow a single thread to do this at a time. We'll obtain a writer lock before we start the process, and we'll release that lock when we've finished, or when an exception has been thrown.

// CreateJob - create a new job...
public
            Job CreateJob(string jobName)
{
Console.WriteLine("Creating
            job {0}", jobName);

// lock...
try
{
//
            lock...
_jobsLock.AcquireWriterLock(-1);

// can't have two
            jobs with the same name, so loop...
int n = 1;
DirectoryInfo
            newFolder;
while(true)
{
// try a name...
string
            useName;
if(n == 1)
useName = jobName;
else
useName =
            string.Format("{0}_{1}", jobName,
n.ToString());

// try
            it...
newFolder = new
DirectoryInfo(string.Format("{0}\\{1}",
Configuration.JobsFolder.FullName,
useName));
if(newFolder.Exists == false)
break;

//
            next...
n++;
}

// create the new
            job...
Console.WriteLine("Created job {0}",
newFolder.FullName);
Job job = new Job(newFolder);

//
            add it...
_jobs.Add(job);

// return...
return
            job;
}
finally
{
//
            unlock...
_jobsLock.ReleaseWriterLock();
}
}

The trick now is when we actually return the new Job object to the client. Because Job extends MarshalByRefObject, it never moves out of the application domain in which it has been created - or to put it more simply, it never moves from the server to the client. What this means is that any calls we make to this object in effect become remote calls. Therefore, if we have a method on Job called Enqueue and as a parameter this takes the details of a message that we want to send, what in effect happens is that the message is automatically transferred over the network without us having to actually do anything special to make this happen.

Loading Existing Jobs

Before we can try this, there's just one thing that we have to do. After we've configured the ServiceConnector object for Remoting inside of StartService, we need to look at the c:\Wrox ListServer\Jobs folder to see if there are any existing jobs. If there are, we need to load this into the collection too. Eventually, we want to get to a point where if the service stops for some reason when it's processing a job, we want to be able to restart that job when the service is restarted.

Add this code to StartService in Service:

// StartService - run the
            service...
public void StartService()
{
// create
            channels...
Console.WriteLine("TCP channel listening on port
            {0}",
Configuration.TcpPort);
IChannel channel = new
            TcpChannel(Configuration.TcpPort);
ChannelServices.RegisterChannel(channel);
Console.WriteLine("HTTP
            channel listening on port {0}",
Configuration.HttpPort);
channel = new
            HttpChannel(Configuration.HttpPort);
ChannelServices.RegisterChannel(channel);

//
            register a Remoting end-point...
Type type =
            typeof(ServiceConnector);
string uri =
            string.Format("{0}/{1}.remote",
Configuration.ApplicationName,
            type.ToString());
Console.WriteLine("Listening on {0}",
            uri);
RemotingConfiguration.RegisterWellKnownServiceType(type,
            uri,
WellKnownObjectMode.Singleton);

// load the existing
            jobs...
try
{
//
            lock...
_jobsLock.AcquireWriterLock(-1);

// create a new
            job list...
_jobs = new JobCollection();

//
            walk...
foreach(DirectoryInfo folder in
Configuration.JobsFolder.GetDirectories())
{
// create the
            new job...
Job job = new
            Job(folder);
_jobs.Add(job);
}
}
finally
{
//
            unlock...
_jobsLock.ReleaseWriterLock();
}
}

Adding another "CreateJob"

The final method that we need to add is to the Job class itself. This static method will make it a little easier to create the jobs. Rather than having to connect to a remote ServiceController instance and then call CreateJob, we'll make it so that we can do that in a single call.

Add this method to Job:

public static Job CreateJob(ServiceConnector.ServiceProtocol
protocol, int port, string hostname, string
applicationName, string jobName)
{
ServiceConnector connector =
ServiceConnector.Connect(protocol, port, hostname,
applicationName);
return connector.CreateJob(jobName);
}

Testing

To test what we've done, ideally we want a separate client application to connect to the server application. Visual Studio provides a great technique for debugging both client and server projects simultaneously, which I'll now introduce.

To the existing solution, add a new Console Application project. Call it ListServerClient. Add a reference to the ListServer project.

Right-click on the ListServer Solution in Solution Explorer and select Properties. Check on Multiple Startup Projects and set ListServerHost and ListServerClient to Start.

Now if we select Debug - Start from the menu, both the client and the server need will be started, and both will share the same ListServer project as the source of the list server application's objects.

Before we do that, we need to add some code to Class1 in ListServerClient. As the server will take a short while to start, we'll prompt the user to hit the Return key before the client attempts to connect to the server.

[STAThread]
static void Main(string[] args)
{
// wait...
Console.WriteLine("Press Return to test the server...");
Console.ReadLine();
Console.WriteLine();

// create a new job...
Job job = Job.CreateJob(ServiceConnector.ServiceProtocol.Tcp,
8000, "localhost", "WroxListServer", "MyMailingList");
Console.WriteLine("Created job {0}", job.Name);

// wait...
Console.ReadLine();
}

Run the project now and wait until the server has finished starting - i.e. wait until the "Listening on" message appears. Flip over to the client application and hit Return. The new job will be created. Here's the debug output from the server:

...and here's the debug output from the client:

You'll notice that on the client side, in the debugging information we make a call into the Name property of the newly returned Job object. Because Job is inherited from MarshalByRefObject, the object never moves. Therefore in this case what's actually happening is that a remote call is being made.

While this is great from a plumbing perspective (all we have to do to expose remote functionality to a client is add new methods to Job), it's worth keeping an eye on this from a performance perspective. Every call that's made has to go over the wire, which may be a staggeringly bad idea depending on what it is that you're trying to do as you could potentially flood your network with traffic. For example, imagine having an indexer on this object and attempting to retrieve every one of, say, 1,000 items accessible through the indexer. In this example, 1,000 individual network requests would be serviced, where it might actually be more useful to transfer all 1,000 items in one block and work through it once it's in place (transferring data to work on in a block fashion is often more efficient than downloading it piecemeal).

At this point, our new job has been created and a new folder has been created to service it:

If you attempt to run the client application again, a new job will be created, but as we mentioned before it will have a unique name. For example:

Queuing Messages

So we can create new jobs. How do we actually send messages?

All we need to do is build some classes that can represent a message. Once we've done that, we'll be able to instantiate instances and pass them over to the remote server by calling a method on the Job object.

The first class we require is Recipient. This will be able to hold the e-mail address of the recipient of an e-mail message, and can also optionally hold the name of the recipient. The class looks like this:

[Serializable()] public class
            Recipient
{
// fields...
private string _email;
private
            string _name;

public Recipient()
{
}

public
            Recipient(string email)
{
_email = email;
}

public
            Recipient(string email, string name)
{
_email =
            email;
_name = name;
}

public string
            Email
{
get
{
return _email;
}
set
{
_email
            = value;
}
}

public string
            Name
{
get
{
return _name;
}
set
{
_name =
            value;
}
}
}

Notice the fact that the class is marked with the Serializable attribute, because we'll want to send it over the wire.

We'll override ToString so that it returns a string representation of the recipient. If a name has been specified, this string needs to take the form:

name <emailaddress>

whereas, if no name is specified, it just contains the e-mail address. Here's the code:

public override string
            ToString()
{
if(Name != string.Empty)
return
            string.Format("{0} <{1}>", Name, Email);
else
return
            Email;
}

On occasion we may want to hold a collection of Recipient objects. For example, we may want a single message to go to a group of recipients. Although you may say "That's the point of a list server!", remember that in most cases we don't want to have a single message containing our entire newsletter database. Each message ideally has to be individually crafted and sent. Add a new RecipientCollection class. Notice how we inherit from CollectionBase and again mark it as Serializable. We'll also create alternate versions of Add to make it easier for the caller to add recipients:

[Serializable()] public class
            RecipientCollection :
System.Collections.CollectionBase
{
public
            RecipientCollection()
{
}

public void Add(Recipient
            rec)
{
List.Add(rec);
}

public void Add(string
            email)
{
Add(new Recipient(email));
}

public void
            Add(string email, string name)
{
Add(new Recipient(email,
            name));
}

public void Remove(Recipient
            rec)
{
List.Remove(rec);
}

public Recipient this[int
            index]
{
get
{
return
            (Recipient)List[index];
}
set
{
List[index] =
            value;
}
}
}

On those occasions where we do need to pass a list of recipients over to Microsoft SMTP Service, the set format for this is to separate each e-mail address with a semicolon. We'll override ToString again such that it goes through each recipient in the collection and adds each one to a string. We'll use the overridden ToString method on Recipient to get the address in the correct format:

public override string ToString()
{
//
            loop...
System.Text.StringBuilder builder =
new
            System.Text.StringBuilder();
for(int n = 0; n < Count;
            n++)
{
if(n !=
            0)
builder.Append(";");
builder.Append(this[n].ToString());
}

//
            return...
return builder.ToString();
}

Now we can move on to the Message class itself. This class simply requires a bundle of fields to support the various recipient lists, the subject of the message, and the body of the message. We'll also store another field called SendProblemDescription, which we'll talk about when we come to actually send the message. This class extends SelfSerializer:

[Serializable()] public class Message : SelfSerializer
{
// fields...
private Guid _id = Guid.Empty;
public Recipient From = new Recipient();
public RecipientCollection To = new RecipientCollection();
public RecipientCollection Cc = new RecipientCollection();
public RecipientCollection Bcc = new RecipientCollection();
public string Subject;
public string Body;
public string SendProblemDescription;

Notice as well how one of those fields is a System.Guid. For those of you who are unfamiliar with these, a GUID ("Globally Unique Identifier") is a great way of creating an ID for something that's guaranteed to be unique, even across different machines. It's actually a vast number (128-bits compared to the 32 bits of your average integer variable), that's made up by combining several unique system characteristics (including a unique value in your network card) with random numbers. No two GUIDs can ever be the same (while not technically true, this is practically true). Using a GUID against a message is a great way of uniquely identifying the messages no matter where they are created or where they eventually end up.

We'll on-demand create one of these using the Id property. We'll use it when we come to serialize the message:

public Guid Id
{
get
{
if(_id ==
            Guid.Empty)
_id = Guid.NewGuid();
return
            _id;
}
set
{
_id = value;
}
}

Finally, we need a Load method that defers to the Load method in the base SelfSerializer class:

public static Message Load(string filename)
{
return (Message)Load(typeof(Message), filename);
}

Now we can turn our attention to queuing the message. Once we've constructed the message and configured its properties, we'll pass it over to Job through the Enqueue method. Because the Job instance is actually running remotely, the effect of the call will be to transfer the Message object across the network to the server. However, before we do that we need to look at the "job state".

Job State

We mentioned before that a job can be in a certain "state". Specifically, a job can either be "open" (i.e. accepting messages), "running" (i.e. sending queued messages out to the recipients), "closed" (i.e. the send process has completed), or it can be in an "error" state. This state will be held in a file called state.dat in the specific job folder.

To start with, create a new enumeration in Job that can hold the state of the job:

public class Job :
            MarshalByRefObject
{
// fields...
private DirectoryInfo
            _folder;
private JobState _state;
private ReaderWriterLock
            _stateLock = new ReaderWriterLock();
private Thread
            _thread;
private bool _threadCancel;

// enum...
public
            enum JobState { Open, Running, Closed, Error };

We want the job to be able to report its state, but we don't want the consumer to be able to specifically set the state. Instead, we want methods on the Job object to do this for us as appropriate. Here's the state method:

public JobState
            State
{
get
{
return _state;
}
}

To determine the location of the state.dat file, we just need the full name of the folder, and the name of the state file:

public string StateFilename
{
get
{
return Folder.FullName + "\\state.dat";
}
}

When it's time to change the state of the job, we'll use BinaryFormatter to just write the value of the internal _state member out to the file:

protected void SetState(JobState
            newState)
{
FileStream stream = null;
try
{
//
            lock...
_stateLock.AcquireWriterLock(-1);

// set
            it...
_state = newState;

// save it to disk...
stream =
            new FileStream(StateFilename, FileMode.Create);
IFormatter
            formatter = new BinaryFormatter();
formatter.Serialize(stream,
            _state);
}
finally
{
//
            unlock...
_stateLock.ReleaseWriterLock();

//
            close...
if(stream !=
            null)
stream.Close();
}
}

"Enqueue"

Now that we have a Message class, and we have set up the job so that we know what state it's in, we can implement the Enqueue method.

All this method has to do is check the state of the job to make sure that it is "open" and, if it is, serializes the message to the job's folder. To get the name of the file, it uses the Id property (which returns a GUID) and tacks .xml onto the end.

Here's the Enqueue method that should be added to Job:

public void Enqueue(Message message)
{
Console.WriteLine("Queue message {0}", message.Id.ToString());

// check the state...
switch(State)
{
case JobState.Open:

// save it...
string filename = string.Format("{0}\\{1}.xml",
Folder.FullName, message.Id.ToString());
message.Save(filename);
break;

default:
throw new InvalidOperationException();
}
}

Notice how we use the Save method of the base SelfSerializer class to write the file to disk.

With all of that in place, we should be in a position where we can try sending a message. Open the Class1 class inside of the ListServerClient project and add code to create a new message object and queue it:

[STAThread]
static void Main(string[]
            args)
{
// wait...
Console.WriteLine("Press Return to test
            the
            server...");
Console.ReadLine();
Console.WriteLine();

//
            create a new job...
Job job =
            Job.CreateJob(ServiceConnector.ServiceProtocol.Tcp,
8000,
            "localhost", "WroxListServer",
            "MyMailingList");
Console.WriteLine("Created job {0}",
            job.Name);

// create the message...
Message message = new
            Message();
message.From.Email =
            "server@wrox.com";
message.To.Add("auser@wrox.com");
message.Subject
            = "Test message";
message.Body = "This is a test
            message!";

// queue...
Console.WriteLine("Queueing message
            {0}",
message.Id.ToString());
job.Enqueue(message);

//
            wait...
Console.ReadLine();
}

Run the project and you'll see something like this on the client:

If you look in the appropriate job folder, you'll see the message:

And if you open the XML file, you'll see the document representing the message:

<?xml version="1.0" ?>
<Message xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<From>
<Email>server@wrox.com</Email>
</From>
<To>
<Recipient>
<Email>auser@wrox.com</Email>
</Recipient>
</To>
<Cc />
<Bcc />
<Subject>Test message</Subject>
<Body>This is a test message!</Body>
<Id>ced330b3-4c7d-467e-a016-8af1e556161f</Id>
</Message>

If you run the client again, another message will be queued and so on, ad infinitum.

Sending Messages

Now that we can queue messages, we can turn out attention to how you actually send messages. Rather than tie our application in so that it has to use Microsoft SMTP Service, we'll create an extensible architecture so that we can plug in new "mail engine" classes.

One central premise of object orientation is the idea of "polymorphism". We should already be very familiar with how this works - it's the idea that by extending classes, we can add new functionality or alter existing behavior, whilst at the same time use those classes with methods and properties that understand the base classes.

In .NET, we can achieve polymorphism in two ways. One way is that we can create a base class that we expect people to extend from in order to add their own functionality. For example, imagine a base class called MailEngine that implements the method Send. MicrosoftSmtpMailEngine could extend this class and override Send such that it uses the Microsoft SMTP Service. Likewise, we could create AnotherMailEngine that also extends MailEngine, but provides a specific implementation of Send.

The other way to achieve polymorphism is to use interfaces. In this method, you sketch out a collection of methods and properties related to a common function. In our case, our IMailEngine interface (interfaces typically start with a capital "I") would implement a single method: Send. Any class that implements this interface effectively enters into a contract that guarantees that each method or property defined on the interface has an appropriate implementation on the implementing class.

The limit with the first inheritance approach is that in .NET, a class can only extend a single class, which means we can run into some limitations. If we already have a class capable of communicating with the mail server that already extends an existing class, we have to jump through some hoops to get the polymorphism working. Interfaces on the other hand can be retrofitted to most classes, so we'll use that approach.

In the ListServer project, create a new class called IMailEngine. By default, VS .NET will create this class with the class directive, whereas we actually want to use the interface directive.

public interface IMailEngine
{
void Send(Message message);
}

Now we can create a new class that implements IMailEngine. Create a new class called MicrosoftSmtpMailEngine, make it implement IMailEngine, and add a stub implementation for Send.

public class MicrosoftSmtpMailEngine :
            IMailEngine
{
public
            MicrosoftSmtpMailEngine()
{
}

public void Send(Message
            message)
{
// stub implementation...
}
}

To send messages, we're going to use the System.Web.Mail namespace. This contains a number of classes, the most relevant here being MailMessage and SmtpMail. MailMessage is a class for holding an e-mail message, and is conceptually similar to our Message class. We've written our own because we want superior handling of recipients (MailMessage does not support an easy way of providing a name alongside an e-mail address), and we also want to be able to control serialization. SmtpMail provides access to the Microsoft SMTP Service itself.

To access System.Web.Mail, you'll need to add the System.Web.dll assembly to your project.

Our Send method therefore has to construct a new MailMessage object and pass it to SmtpMail so that it will be queued once again within the Microsoft SMTP Service before it's sent on its way to its final destination.

Here is the implementation of Send:

public void Send(Message message)
{
//
            build the message...
MailMessage mail = new
            MailMessage();

// basic stuff...
mail.From =
            message.From.ToString();
mail.Subject =
            message.Subject;
mail.Body = message.Body;

// to, cc,
            bcc...
mail.To = message.To.ToString();
mail.Cc =
            message.Cc.ToString();
mail.Bcc =
            message.Bcc.ToString();

// send
            it...
SmtpMail.Send(mail);
}

Notice here how we're using the ToString method of RecipientCollection to return a properly formatted string back. Remember, we need to separate each recipient with a semi-colon and use the format name <emailaddress> if a name has been specified for the recipient.

We'll use the same mail engine irrespective of the job that we're processing. Eventually we'll want to provide a way of specifying an alternative mail engine rather than the built in MicrosoftSmtpMailEngine, and we'll do this through Configuration. Add this method to Configuration:

public IMailEngine CreateMailEngine()
{
return new MicrosoftSmtpMailEngine();
}

Sending Mail

To process a job, all we have to do is walk through each message in the job folder one by one, de-serialize it from XML, and pass it to the IMailEngine -implementing object for processing. This is a classic example of a process that's best executed in a thread. We may want to process multiple jobs simultaneously, and so we can run each one in its own thread - or at the very least we want to ensure that any threads attempting to queue messages into new jobs will not run into blocking problems.

Job, then, needs to support a thread. We also need a method that we can use for a controlled shutdown of the thread should the service be stopping when we're trying to process the thread. To Job, add the new _thread and  _cancelThread members:

public class Job : MarshalByRefObject
{
// fields...
private DirectoryInfo _folder;
private JobState _state;
private ReaderWriterLock _stateLock = new ReaderWriterLock();
private Thread _thread;
private bool _threadCancel;

When we want to run the job, we'll call the Run method. This method is responsible for checking the state of the job using the State property and spinning up the thread.

This part is easy. All we need to do is create a new System.Threading.Thread object and call start. The only valid state in which Run can be called is "open".

public void Run()
{
// check our
            state...
switch(State)
{
case JobState.Open:

//
            create and start the new thread...
_threadCancel =
            false;
_thread = new Thread(new
ThreadStart(ThreadEntryPoint));
_thread.Start();
break;

default:
throw
            new InvalidOperationException();
}
}

Inside the thread, things have the potential to get a little more complicated. What we want to do is have two folders inside the job folder. One ("Sent") will be used to store a copy of the message once it has been sent. The other ("Problem") will be used when the mail engine reports an exception in sending the message.

Create constants in Job to store the basic names of these two folders:

public class Job : MarshalByRefObject
{
// fields...
private DirectoryInfo _folder;
private JobState _state;
private ReaderWriterLock _stateLock = new ReaderWriterLock();
private Thread _thread;
private bool _threadCancel;

// enum...
public enum JobState { Open, Running, Closed, Error };

// const...
public const string StateBaseFilename = "state.dat";
public const string SentBaseFolderName = "Sent";
public const string ProblemBaseFolderName = "Problem";

Then, create new properties for each folder:

public DirectoryInfo SentFolder
{
get
{
return new DirectoryInfo(Folder.FullName + "\\" +
SentBaseFolderName);
}
}

public DirectoryInfo ProblemFolder
{
get
{
return new DirectoryInfo(Folder.FullName + "\\" +
ProblemBaseFolderName);
}
}

We need three methods to help us control the lifetime of the thread. If the service needs to stop, we need to stop sending messages and resume the queue when the service restarts. We need a Cancel method to allow the Service controller close to tell us to stop processing:

public void Cancel()
{
_threadCancel = true;
}

We also need an IsCancelled method that we can use to determine our own state:

public bool IsCancelled()
{
return _threadCancel;
}

Finally, we need a Join method that returns only when the thread itself has been finished. This method should not be called from within the thread itself otherwise we'll enter a deadlock. We'll give the thread 30 seconds to close naturally, and if this expires, we'll abort it as something would have gone wrong if we don't naturally drop out of the thread loop:

public void Join()
{
// do we have a
            thread?
if(_thread !=
            null)
{
Cancel();
if(_thread.Join(new TimeSpan(0, 0, 30))
            == false)
_thread.Abort();
_thread =
            null;
}
}

Now the thread code itself. The first thing we need to do is change our state to "running":

// ThreadEntryPoint...
private void ThreadEntryPoint()
{
Console.WriteLine("Running job {0}", Name);

// update the state...
SetState(JobState.Running);

After we've done that, we can ask Configuration to give us the mail engine that we're supposed to use:

// get the engine...
IMailEngine engine =
Service.CurrentService.Configuration.CreateMailEngine();

The next step is to make sure that the Sent and Problem folders exist. If they do not, we'll need to create them:

// check that the folders exist...
if(SentFolder.Exists == false)
SentFolder.Create();
if(ProblemFolder.Exists == false)
ProblemFolder.Create();

Now we can look at how to process the messages. We use the Folder property to return a DirectoryInfo object back, and we call the GetFiles method to get a list of all of the .xml files in the folder:

// get a list of the files...
FileInfo[] messageFiles = Folder.GetFiles("*.xml");

We loop through the messages one by one, calling IsCancelled each time. If we've been cancelled, we'll drop out of the loop:

// walk each message...
int messageIndex
            = 0;
while(IsCancelled() == false)
{
Message message =
            null;
try
{

Per iteration of the loop, we load the next message and give it to the mail engine for sending:

// process it...
message =
Message.Load(messageFiles[messageIndex].FullName);

// send it...
engine.Send(message);

If we get to this point, no exceptions have been thrown, and we can assume that the message was properly processed. We put the message into the Sent folder:

// move it...
string newFilename =
string.Format("{0}\\{1}.xml",
SentFolder.FullName,
message.Id.ToString());
messageFiles[messageIndex].MoveTo(newFilename);
}

If anything does go wrong, we want to save the message into the Problem folder. As well as this, we'll take the description of the exception and store it in the SendProblemDescription property of the message:

catch(Exception ex)
{
// ignore any
            problems that happen here...
try
{
// do we have a
            message...
if(message != null)
{
// save the
            details...
message.SendProblemDescription =
ex.GetType().ToString() + ":" +
ex.Message;

message.Save(messageFiles[messageIndex].FullName);

//
            move it...
string newFilename =
string.Format("{0}\\{1}.xml",
ProblemFolder.FullName,
message.Id.ToString());
messageFiles[messageIndex].MoveTo(newFilename);
}
}
catch
{
}
}

Finally after all that, we can move onto the next message:

// next...
messageIndex++;
if(messageIndex == messageFiles.Length)
break;
}

After we've walked all of the messages, we can set our state to "closed":

// finished...
SetState(JobState.Closed);
Console.WriteLine("Job {0} finished", Name);
}

To test this, call Run from within the client application after queuing a message:

// queue...
Console.WriteLine("Queueing
            message {0}",
message.Id.ToString());
job.Enqueue(message);

//
            run...
job.Run();

Run the project now and the server will report that the job has been executed:

In addition, you'll notice that the message file has been saved to the Sent folder:

Providing that you've changed the sample code to use a valid e-mail address, rather than the dummy e-mail address in the source code download, eventually the message will come through to your Inbox:

Handling Bounces, Returned Mail and "Unsubscribes"

Sending messages is just one part of the battle. The problems really start when messages come back! If for any reason the remote server cannot process the inbound message, the message will be returned along with an explanation to the "from" address specified in the outbound message.

Messages will be returned for a number of reasons. In some cases, the e-mail address that you've specified will be invalid, or there will be a problem with the recipient's account. For example (and this often happens with Hotmail), the recipient may not have enough free storage available to receive the message. Or the account may not exist or have been closed.

If the server doesn't reply to the message to tell you something's wrong, the actual recipient might. This can happen for, again, a number of reasons. A classic with mailing lists is the, "I'm out of the office at the moment" response. A recipient's e-mail client in response to any inbound message automatically sends these.

Another common response is the "unsubscribe" response. Traditionally, mailing lists allow people to opt out of future mailings by responding with the word UNSUBSCRIBE as the subject. We need to scan the subjects and if they do appear to be "unsubscribe" messages, we need to put them into a separate bucket for processing by another application.

In this implementation, we're not going to give the service the capability of connecting to a database and marking the messages. For example, if the user appears to want to unsubscribe, we don't want to send them messages again - otherwise we run the risk of upsetting them. Likewise, if the server is returning a message to us, we want to mark the account with a "bounce count". Once this bounce count gets high enough we stop sending them e-mail. For example, "three strikes, you're out!"

The problem we have to overcome is that while the Framework comes with support for sending e-mail, it does not come with support for receiving e-mail. To get around this problem, we need to use a third-party component.

The market for third-party COM components is well developed, and the "POP3 Download" sector of this market contains a number of products, some of them free. A great resource for finding new ASP/COM components is Aspin, which you can find at http://www.aspin.com/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq.

The component that I'm going to use is DynuPOP3. I've used this component in the past, and I've personally never experienced problems with it (Disclaimer: Your mileage may vary!) You can find it at http://www.dynu.com/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq.

Using the Component

Although COM has largely been replaced by a number of new technology innovations in .NET, Microsoft recognizes that the industry's investment in COM is huge and therefore, COM is by no means "dead". We can use "interop" to call down into a COM component that we like. What's more, we can also use our .NET components as if they were COM components - although we won't see how to do that in this article.

To use our DynuPOP3 component in our application, use Solution Explorer to add a new reference to the project. Change to the COM tab and find the Dynu 1.0 Type Library. Double-click it to add it to the Selected Components list.

Once you press OK, Visual Studio will create a set of wrapper classes that enable this COM component to be used from within your .NET application!

You can use the component just like a normal .NET object. However, there are a few things to be aware of.

Firstly, Microsoft have done a lot of work in evangelizing the way that components are supposed to be constructed and used in .NET, mainly to get over a lot of the confusion with COM components. .NET objects are supposed to stick to strict naming conventions, including case conventions that make it easier to move from one non-case sensitive language (like VB .NET) to another case-sensitive language (like C#). One classic place that we find this is in acronyms that are treated as normal works, e.g. System.Xml.XmlDocument is correct, whereas System.XML.XMLDocument is not.

Secondly, building rich object models is easier in .NET than it was in COM - although there's nothing new or special about .NET in this regard. It's simply easier to build new classes and also to build rich collection classes in VS .NET than it was in VS6.

Thirdly, there's no exception handling built into COM components, which means that error trapping and checking gets a bit more (frankly) primitive. Keep an eye on the state of the object and check return values.

Fourthly, we're using COM, which means we've potentially returned to the world of DLL Hell. If you are deploying an application that uses COM components through interop, be advised that the application may get broken should the user deploy a different version of the component than the one you've designed it to work with.

What these facts mean is that if you haven't done any COM work for a while and then move into doing some interop work like we're about to do, you will need to slip out of the .NET mindset and into the mindset of whatever the component manufacturer feels is logic.

Case in point - when using a .NET component, if you call a method like RetrieveMessage, you'd expect that method to return a new object containing the message. However, with DynuPOP3, this call actually sets properties on the same object that contains the method. No big deal, but something to be aware of.

Bounce Accounts

To handle inbound e-mails, we need Service to set up another thread that can potentially monitor multiple accounts and take one of three actions:

  • If the subject is "UNSUBSCRIBE", this message needs to be copied into the Unsubscribe folder.
  • If the subject appears to be one of a number of standard subjects used by mail servers, the message is either ignored (because it's a delay, or "warning") message, or copied into the Bounce folder.
  • If the subject is not understood, the message needs to be forwarded to the list owner. This will be the case if the recipient replies to the message.

We'll store a list of the accounts in a new BounceAccounts collection in Configuration. This collection will contain a list of BounceAccount objects, each one containng a username, password and server name that will be used to connect to the POP server. The BounceHandler class will walk each of these accounts, download and process any e-mail, and then wait for a set period of time before starting again.

First off, here's BounceAccount. This class needs to be serializable:

[Serializable()] public class
            BounceAccount
{
// fields...
public string
            Username;
public string Password;
public string
            ServerName;
public int Port = Pop3Port;

//
            const...
public const int Pop3Port = 110;

public
            BounceAccount()
{
}
}

Notice there how we've included a constant containing the default port for POP3, which happens to be 110.

We then need a custom collection class to hold a list of BounceAccount objects. This class also needs to be serializable:

[Serializable()] public class
            BounceAccountCollection :
System.Collections.CollectionBase
{
public void
            Add(BounceAccount acc)
{
List.Add(acc);
}

public
            void Remove(BounceAccount
            acc)
{
List.Remove(acc);
}

public BounceAccount
            this[int index]
{
get
{
return
            (BounceAccount)List[index];
}
set
{
List[index] =
            value;
}
}
}

To add this collection to Configuration, simply add a new public field called BounceAccounts.

[Serializable()] public class Configuration : SelfSerializer
{
// fields...
private DirectoryInfo _baseFolder;
public string ListOwnerAddress = "bounce@wrox.com";
public int TcpPort = 8000;
public int HttpPort = 8001;
public string ApplicationName = "WroxListServer";
public BounceAccountCollection BounceAccounts =
new BounceAccountCollection();

The issue we have now is that we need to get the details of the accounts into the configuration XML file. This is always a bit "chicken and egg", as it's silly to try and guess the potential structure of the XML file, but without having data it's hard to generate the file. What I've done here is in the server code, I find out if BounceCollections is empty and, if it is, I add a new account and save the configuration file:

[STAThread]
static void Main(string[]
            args)
{
// load the configuration...
string configFilename
            = "c:\\Wrox ListServer\\config.xml";
Configuration config =
            Configuration.Load(configFilename);

// do we have any bounce
            accounts configured?
if(config.BounceAccounts.Count ==
            0)
{
// create one...
BounceAccount acc = new
            BounceAccount();
acc.Username = "bounce";
acc.Password =
            "bounce";
acc.ServerName =
            "mail.mymailserver.com";
config.BounceAccounts.Add(acc);
config.Save(configFilename);
}

//
            start up the service...
Service service = new
            Service(config);
service.StartService();

//
            wait...
Console.ReadLine();
}

You'll need one bounce account per mailbox that you configure to handle returned e-mails. In this case, I've used a fake account. If you can imagine, if my messages are set to come from bounce@dotnet247.com, the mailbox I need to use is the one that I would use to pick up messages from this account.

Run the server and you'll find that the configuration file has been updated.

Creating "BounceHandler"

The next step is to build a new class that can handle downloading the mails. Create a new class called BounceHandler and add these namespace import declarations:

using System;
using System.IO;
using System.Web.Mail;
using System.Threading;

Controlling the lifetime of this thread is a little more complex than controlling the lifetime of the job thread. The issue here is that we want to design from the perspective of having quite a long period of time when the thread is actually doing nothing. This is the timeout period between checking mails - much like you already have your e-mail client configured to only check mail periodically, rather than constantly.

If the thread is working, we want to use the "cancel" pattern that we used before. As we're walking the e-mails, at some point we'll be able to check to see if we are cancelled and, if so, drop out of the thread. If, however, we're sleeping we could be sleeping for ten minutes, in which case we want to abort the thread.

Add fields to BounceHandler to support a Thread object, a _cancelThread flag, a _isSleeping flag, and a TimeSpan that will hold the timeout period at 10 minutes by default.

public class BounceHandler
{
// fields...
private Thread _thread;
private bool _cancelThread;
private bool _isSleeping;
public TimeSpan Timeout = new TimeSpan(0,10,0);

Spinning up the thread is easy. If the thread hasn't already been started, we create a new Thread object and start it.

// SpinUp - start the thread...
public
            void SpinUp()
{
// do we already have a thread?
if(_thread
            != null)
throw new InvalidOperationException();

// run
            it...
_cancelThread = false;
_isSleeping = false;
_thread =
            new Thread(new
            ThreadStart(ThreadEntryPoint));
_thread.Start();
}

Cancel, IsCancelled and IsSleeping are all self-explanatory:

public void Cancel()
{
// flag
            it...
_cancelThread = true;
}

public bool
            IsCancelled()
{
return _cancelThread;
}

public bool
            IsSleeping()
{
return _isSleeping;
}

The Sleep method uses the static Sleep method available on Thread, but it changes the flag before and after the Sleep call. It also won't drop into a sleep state when the thread has been cancelled.

protected void Sleep(TimeSpan
            timeout)
{
// if we're cancelled, stop...
if(IsCancelled()
            == true)
return;

// flag, sleep, wait...
_isSleeping =
            true;
Thread.Sleep(timeout);
_isSleeping =
            false;
}

Join works in a similar fashion to the Join method on Job. However, this one checks to see if the thread is sleeping and, if it is, issues the Abort call instantly, rather than waiting 30 seconds for the thread to finish naturally:

public void Join()
{
//
            thread?
if(_thread == null)
return;

//
            cancel...
Cancel();

// are we sleeping? if so, instantly
            abort...
if(IsSleeping() ==
            true)
_thread.Abort();
else
{
// try it
            gently...
if(_thread.Join(new TimeSpan(0,0,30)) ==
            false)
_thread.Abort();
}

//
            join...
_thread.Join();
_thread = null;
}

Now we can turn our attention to the thread itself. Before we do that though, let's take a quick look at the logic.

As we mentioned before, messages will either come through with the subject as "UNSUBSCRIBE", in which case we store them in the Unsubscribe bucket where they can be picked up by whatever application is actually going to handle updating their information nt the database, or the messages are going to come through with a failure message, or a warning message, or we won't understand what to do with them.

This gives us four states that we can express in an enumeration in BounceHandler:

public class BounceHandler
{
// enum...
public enum InboundMessageType { Unknown, BounceWarning,
BounceError, Unsubscribe };

How we tell whether we have a failure or warning message is a little trial and error. Each mail server will do it slightly differently, although what we will do is use the two most common replies, which come from Microsoft Exchange Server, or from Sendmail, a very popular e-mail server.

If Exchange cannot process a message, we'll receive a message with the original subject prefixed with "Undeliverable:", like this:

Undeliverable: The Mailing List Subject

If Sendmail cannot process a message straightaway we'll get a delay message, which we'll ignore:

Delivery Status Notification (Delay)

If after a period of time the message still cannot be processed, we'll get this back:

Delivery Status Notification (Failure)

These are the only cases we're going to handle in this implementation. Anything that's not understood will be forwarded directly to the list owner, which is specified in the ListOwnerAddress property of Configuration.

Here's the bounce handler thread. The first thing we do is to create a mail engine. This will be used to forward unprocessed messages onto the list owner:

// ThreadEntryPoint...
private void ThreadEntryPoint()
{
Console.WriteLine("Bounce handler started");

// create a mail engine...
IMailEngine engine =
Service.CurrentService.Configuration.CreateMailEngine();

Then, we keep processing until the thread is cancelled:

// loop...
while(IsCancelled() == false)
{

Per iteration, we go through every bounce account specified in the configuration file:

// go through each account...
foreach(BounceAccount acc in
Service.CurrentService.Configuration.BounceAccounts)
{

Each time, we use DynuPOP3 to connect to the remote server:

// connect...
DYNULib.POP3Class pop = new DYNULib.POP3Class();
pop.Connect(acc.ServerName, acc.Username, acc.Password,
acc.Port);

To find out how many messages are on the server, we use GetStatistics:

// how many messages?
pop.GetStatistics();

Next, we walk each message, provided that we haven't been cancelled. DynuPOP3 numbers the messages from 1, not 0.

// walk the messages...
for(int
            messageIndex = 1; messageIndex <= pop.NumberOfMessages
&& IsCancelled() == false;
            messageIndex++)
{

RetrieveMessage downloads the message from the server. The second parameter indicates if you want to delete the message from the server or not. 1 specifies that you do. When debugging, you might find it useful to set this to 0.

// get the message and delete it...
pop.RetrieveMessage(messageIndex, 1);

We then create a new Message object based on the details of the message on the server:

// create a new message object...
Message message = new Message();
message.From.Email = pop.From;
message.From.Name = pop.FromName;
message.Subject = pop.Subject;
message.Body = pop.Body;

Now we can examine the subject of the message. First off, we check to see if it is UNSUBSCRIBE:

// examine the subject...
InboundMessageType messageType =
InboundMessageType.Unknown;
if(string.Compare(message.Subject, "unsubscribe", true)
== 0)
messageType = InboundMessageType.Unsubscribe;
else
{

If it isn't, we compare it with our list of known subjects. If it starts with "undeliverable:" we know that it is an error:

// is it one of a known set?
string subject = message.Subject.ToLower();
if(subject.StartsWith("undeliverable:") == true)
messageType = InboundMessageType.BounceError;
else
{

Otherwise, we compare it with a list of known subjects:

switch(subject)
{
case
"delivery status notification (failure)":
messageType =
InboundMessageType.BounceError;
break;

case
"delivery status notification (delay)":
messageType =
InboundMessageType.BounceWarning;
break;
}
}
}

Once we know what the message is, it's time to decide what to do with it:

// now what?
string newFilename =
            string.Empty;
switch(messageType)
{
case
            InboundMessageType.Unsubscribe:

// save it to the unsubscribe
            folder...
newFilename = string.Format("{0}\\{1}.xml",
Service.CurrentService.Configuration.UnsubscribeFolder.FullName,
            message.Id);
break;

case
            InboundMessageType.BounceWarning:
// ignore it if it's just a
            warning...
break;

case
            InboundMessageType.BounceError:

// save it to the bounce
            folder...
newFilename = string.Format("{0}\\{1}.xml",
Service.CurrentService.Configuration.BounceFolder.FullName,
            message.Id);
break;

default:

// forward it to the
            list
            owner...
message.To.Add(Service.CurrentService.Configuration.ListOwnerAddress);
engine.Send(message);
break;
}

//
            save it?
if(newFilename !=
            string.Empty)
message.Save(newFilename);
}

//
            cancel?
if(IsCancelled() == true)
break;
}

After processing all of the accounts, we sleep until the timeout has expired:

// wait for the
            timeout...
Sleep(Timeout);
}
}

If you want to try this, you'll need to send some messages to one of the mailboxes that you can specify in the bounce accounts list. After a period of time, check the Unsubscribe and Bounce folders and you should find the messages sorted appropriately. If you sent a message without an understood subject, you should receive this in your Inbox (providing that ListOwnerAddress is properly configured), appearing as if it's come from the original recipient:

Summary

In this article, we took a detailed look at how to build a list server for .NET. We provided a mechanism whereby we could expose the functionality of the list server over Remoting and access it from any machine on the network whereby messages could be queued and ultimately sent on to their chosen recipient.

Case Study Information
Author | Matthew Reynolds
Technical Editor | Adam Ryland
Project Manager | Helen Cuthill
Reviewers | Andrew Krowczyk, Saurabh Nandu, David Schultz, Adam Ryland

If you have any questions or comments about this case study, or suggestions for further pieces, please contact the technical editor. If you wish to talk about the case study with other subscribers, please visit our forum here.

C#Today - © 2002 WroxPress Limited


Asynchronous invocation of web services

Asynchronous invocation of web services
http://www.csharptoday.com/content.asp?id=1834
Article Rating: 4
Published on April 1, 2002
Abstract
One of the compelling features of the .NET Framework is the feature-set it provides which makes it possible to design and develop effective applications using a variety of mechanisms. One such feature is the asynchronous support that is built into the Framework. Asynchronous programming can be defined as the ability of a method to return immediately, without waiting (or without blocking on the calling thread) for the called method to finish its execution. When the called method has finished its execution, it notifies the caller by invoking the callback function that was specified at the time of invoking the method asynchronously. In this case study, Thiru will discuss some of the common scenarios where asynchronous processing can be used. While discussing them, we will also demonstrate the different asynchronous mechanisms built into the Framework, and then see which one to use in what situations.
Case Study

One of the compelling features of the .NET Framework is the feature-set it provides which makes it possible to design and develop effective applications using a variety of mechanisms. One such feature is the asynchronous support that is built into the Framework. Asynchronous programming can be defined as the ability of a method to return immediately without waiting (or without blocking on the calling thread) for the called method to finish its execution. When the called method has finished its execution, it notifies the caller by invoking the callback function that was specified at the time of invoking the method asynchronously. As you can see, asynchronous calling provides a lot of flexibility when compared to a synchronous approach wherein the calling thread is blocked and has to wait until the called function completes. Asynchronous calling also provides for more parallelism. The need for asynchronous processing becomes very important in the development of distributed applications, which typically depend on different independent and remote entities, such as web services, to perform its operations. However it is very important to realize that asynchronous processing cannot be used in all cases, as sometimes the applications may need the method call to complete before it can go any further. In this case study, we will discuss some of the common scenarios where asynchronous processing can be used. While discussing them, we will also demonstrate the different asynchronous mechanisms built into the Framework, and then see which one to use in what situations.

Introduction

Software applications are expected to deliver a wide variety of solutions to address different kinds of customer and business needs. Web services can be used to implement solutions of this kind and they can fundamentally transform the ability of businesses to interact and exchange information with each other. However, building effective and high performance applications becomes a challenge the moment we start integrating web services into our applications. This is primarily due to the fact that the web services we are trying to invoke can be present anywhere in the Internet. To make matters worse, we do not have any control over the public networks on which our request has to travel, as well as over the machines on which the web services reside. If you take all these aspects into consideration, you will agree that making a synchronous blocking web service call from a web page may not be a wise choice as it is going to have a tremendous impact on the user's browsing experience on the site.

However web services support both synchronous and asynchronous communication between the client application and the web service. Even though both the mechanisms are supported, it is very hard to incorporate asynchronous web services in a web application due to the tightly coupled nature of the data from the web service and the information we are trying to show on the web form. In this case study, we will explore the mechanics of invoking Web Services via an asynchronous mechanism. We'll also look into the .NET framework and the infrastructure it provides for asynchronous processing.

Another important feature of the .NET Framework that provides for asynchronous programming is the FileSystemWatcher component. We will have a detailed discussion on the FileSystemWatcher component in the later stages of this case study.

In this case study, we will cover two important aspects of asynchronous programming. First, we will see how we can create a solution in which we will replace the synchronous, blocking calls to the web service with the asynchronous approach by using the combination of Windows Service Application, a server side Timer component, and asynchronous invocation of a web service. Secondly, we will see how we can use the FileSystemWatcher component to monitor a directory and watch for changes, and also take action if a new file is created.

Introduction to the Case Study

This case study is based on a web site called ShoppingAssistant, which provides an onestop shop for consumers who want to find out information such as the products that are on sale, availability of products in different stores, and price comparisons of the product across different stores. It also provides consumers with the product information and reviews of them, and highlights stores within a specified mileage radius that feature the item. ShoppingAssistant also supplies the consumers with access to the online guide that provides the best local values and product information. The business model of ShoppingAssistant web site revolves around the following two main requirements.

  • The need to seamlessly get information from their partners. Even though the information they display in the site is not particularly time-sensitive, it is still required that the information is updated once in a while (may be once in a day), if not real time updates.
  • There is also the need to generate reporting data (for example, which is the most popular product on the site) for their business partners.

Throughout this case study, we will address the above requirements by considering a step-by-step approach in which we will be continuously refining the solution as we go along.

Currently, ShoppingAssistant gets all the required information from its content provider partners in the form of an XML feed. It accomplishes this by making a remote request to the .asp file or any other server side scripting file, receiving the response in the form of XML, and converting the response back to a form that is usable in the site. Even though this approach serves its purpose, it is very obvious that this requires a lot of plumbing and infrastructure code to accomplish the desired output, and also, due to the tightly coupled nature of the design, even minor changes in the logic that is used to generate the XML document might end up breaking the client side implementation of the application.

Having realized the need to come up with a better and more effective automated solution for getting the XML information feed, ShoppingAssistant started looking around for content providers who can satisfy their need. Luckily they found a group called ContentPublisher who expose the information required by ShoppingAssistant in the form of web services. ShoppingAssistant decided to take advantage of the web services capability exposed by the ContentPublisher, and they have decided to implement the following architecture in their web site.

Architecture of the System

As you can see, the ShoppingAssistant web site incorporates the functionality implemented by the .NET component (ShoppingAssistantLib) and the ContentPublisher web service into its web site.

When the user comes to the site and performs operations such as registration and login verification, the ShoppingAssistant web application invokes the methods of the .NET component ShoppingAssistantLib to carry out those tasks.

The ShoppingAssistant web application connects to the remote web service across the Internet to fetch information whenever the user performs any of the following actions.

  • Browsing the categories page
  • Browsing the product listing page
  • Navigating to the product details page

Business Processes

Clearly the scope of the case study is to show how to implement asynchronous approaches in developing web applications. But before we do that, we will implement the solution in the traditional way that involves synchronous calls. Once we have created this, we will then analyze the disadvantages involved in the approach. Then we will see how to overcome this by adopting the alternative asynchronous approach.

The business processes we are going to discuss are as follows:

  • Login Process - Registered users are given the option to login to our site and retrieve their personal profile. The login system allows the users to identify themselves to the system. The user must provide an existing user id and a valid password to be allowed to enter the system.
  • New user Registration process - New users are given the opportunity to become members of the site by filling out one of our online form and selecting their preferences. In this step, the user is asked to create a unique user id, which is used to identify the user in the system. The user can also fill out details like name and address than can be used by the system administrator to contact the user when necessary. After the user has entered all the details and once the data has been validated, then the user's profile is stored in the database for later retrieval.
  • Catalog browsing process - Users can browse through the list of products available in the site and also their prices
  • Products listing process - In this process, the list of products available in a particular category is shown with a hyperlink that takes the users to more information about the product
  • Products details process - When the user clicks the hyperlink of a particular product, they are taken to the page where detailed explanation about the particular product is shown
  • Login process - Allows the user to log out of the site, which internally nulls the cookie thereby terminating the session
  • Reporting data gathering process - In this step, before showing the products information on the web site, we store the list of products shown in the page in a separate table, which can, later, be used for reporting purposes.

We will split the implementation of this case study into the following parts.

  • In the first part, we will create a web service named ContentPublisherService, whose primary job is to expose information related to various categories and products that are available on sale to the other web sites. The consumers of the web service can then consume this information in any fashion they like on their site.
  • In the second part, we will see create an ASP.NET web client application that consumes the information exposed by the web service. After defining the solution, we will discuss the performance implications that we have to incur due to the synchronous and blocking calls that we make to the web service.
  • Now that we have identified the shortcomings in the solution, we will define an alternative approach to address this issue. To accomplish this, we will create a solution that employs the combination of Windows Service application (that has a timer component on it) and asynchronous invocation of a web service. In the Elapsed event of the Timer control, we will write code to invoke the web service at frequent intervals. We will store all of the information gathered from the web service in a local folder (in the form of XML documents) for the web client application to make use of.
  • In this part, we will see how to effectively incorporate reporting capabilities into the application. For example, we might want to create a report (that is used for marketing promotional and analysis purposes) wherein we want to find out how many items of a particular product type have been viewed in one particular day. Reports of this nature require us to store this information somewhere, while at the same time doing so without compromising on the response time of the site. To accomplish this, we will create a solution that uses a combination of synchronous and asynchronous approaches. Before drawing on the page, we will create an XML string dynamically that consists of information about the items in the page that we are trying to draw, and then store that XML string in an XML file in a pre-defined directory. Once the information is persisted in the directory, the calling ASP.NET application immediately returns and continues with the task of drawing the page, thereby minimizing the amount of processor cycles spent on storing this information. Since this directory is being monitored by the FileSystemWatcher component (that is hosted in a windows service application), this file will be automatically read and its contents stored in the database.

Implementation

Now that we have understood the business processes involved, let us construct the individual building blocks that are required for implementing this solution. We will split our discussion of the remaining part of the case study into the following sections:

  • Database Design
  • Implementation of ContentPublisher web service
  • Implementation of .NET component (ShoppingAssistantLib)
  • Implementation of ShoppingAssistant web application
  • Implementation of asynchronous web services and windows service application to access the information from the web service
  • Implementation of windows service and FileSystemWatcher component to facilitate information gathering in the site without affecting the performance of the site.

We will start our discussion of the implementation by looking at the database design that is required to support our application.

Database Design

As we can see from the architecture of our system, two data stores are used to store and provide information to our application. Since the main aim of the case study is to lay emphasis on .NET web application development using asynchronous mechanisms, the databases we are going to use will have minimum number of tables required to implement this solution.

Firstly, we will consider the design of the ShoppersInfo database.

ShoppersInfo database design

This database is basically used to store and retrieve information about the list of users registered on our site. The entity relationship diagram for the ShoppingAssistant database is as follows:

The Users table has the following structure:

Name | DataType | Length | AllowNull | Description
UserID | Varchar | 20 | No | Represents the user id
Password | Char | 10 | No | Indicates the password
Name | Varchar | 128 | Yes | Name of the user
Address | Varchar | 128 | Yes | Address of the user
City | Varchar | 50 | Yes | City of the user
State | Char | 2 | Yes | State of the customer
Zip | char | 9 | Yes | Zip code of the user's area

The ReportInfo table is defined as follows:

Name | DataType | Length | AllowNull | Description
ProductID | Int | 4 | No | Represents the Product ID
CategoryID | Int | 4 | No | Indicates the CategoryID
Browser | Varchar | 256 | Yes | Name of the Browser
RequestType | Varchar | 256 | Yes | Browser request type
Authenticated | varchar | 50 | yes | Type of authentication

The UserLogIn stored procedure returns 1 if a record with the supplied user name and password is found in the Users table, otherwise it returns -1. It uses output parameters to pass the results back to the caller.

CREATE Procedure UserLogin
(
@UserID varchar(20),
@Password varchar(10),
@RetValue int OUTPUT
)
As
SELECT * FROM Users WHERE UserID = @UserID AND Password = @Password
IF @@Rowcount < 1
SELECT @RetValue = -1
ELSE
SELECT @RetValue = 1

As the name suggests, the InsertUser procedure inserts a record into the users table based on the supplied parameters.

CREATE Procedure InsertUser
(
@UserID char(20),
@Password char(10),
@Name varchar(128),
@Address varchar(128),
@City varchar(50),
@State char(2),
@Zip char(9))
as

Insert into Users(UserID,Password,Name,Address,City,State,Zip)
Values(@UserID,@Password,@Name,@Address,@City,@State,@Zip)

GO

The InsertReportInfo stored procedure simply adds a record to the ReportInfo table, which is used as a reporting information-gathering tool.

Create Proc InsertReportInfo
(
@ProductID int,
@CategoryID int,
@Browser varchar(256),
@RequestType varchar(256),
@Authenticated varchar(50)
)
as
Insert into ReportInfo(ProductID,CategoryID,Browser,RequestType,Authenticated)

Values(@ProductID,@CategoryID,@Browser,@RequestType,@Authenticated)
GO

ContentPublisher database design

It consists of Categories and Products tables. The entity relationship diagram for the ContentPublisher is as follows:

The above diagram shows that the ProductID column acts as the primary key for the Products table, and CategoryID column is used as the primary key for the Categories table. It also shows that the referential integrity is enforced between the Products and Categories tables through the use of CategoryID column.

The Categories table looks like this:

Name | DataType | Length | AllowNull | Description
CategoryID | Int | 4 | No | Indicated the category id
CategoryName | varchar | 50 | No | Represents the category name

The table design for the Products table is:

Name | DataType | Length | AllowNull | Description
ProductID | Int | 4 | No | Represents the product id
CategoryID | Int | 4 | No | Indicates the category id
ModelNo | Varchar | 50 | Yes | Model number of the product
ModelName | Varchar | 50 | Yes | Model name of the product
Image | Varchar | 50 | Yes | Image name of the product
Price | Money | 8 | No | Price of the product
Description | Varchar | 2000 | Yes | Description of the product
OnSale | char | 1 | no | Specifies whether the product is on sale

Let us consider the stored procedures in the ContentPublisher database that we will be using in our application. The stored procedure CategoriesGet returns all the categories present in the Categories table:

CREATE Procedure CategoriesGet
As
SELECT * FROM Categories
GO

The ProductsGet stored procedure is used to return all the available products:

CREATE Procedure ProductsGet

(
@CategoryID int
)
As
SELECT * FROM Products
WHERE CategoryID = @CategoryID
GO

The ProductsGetByCategoryID stored procedure is used to return all the products that belong to a particular category.

CREATE Procedure ProductsGetByCategoryID

(
@CategoryID int
)
As
SELECT * FROM Products
WHERE CategoryID = @CategoryID
GO

As the name suggests, the ProductDetailsGet procedure returns the details of a particular product based on the passed product id.

CREATE Procedure ProductDetailsGet

(
@ProductID int
)
As
SELECT * FROM Products
WHERE ProductID = @ProductID
GO

Implementation of web service

In this section, we will create a web service called ContentPublisherService that contains methods to expose products information to its consumers. Through its methods, the web service provides information such as lists of product categories available, lists of products in a particular category, and the details of a specific product.

Now let us get started by creating the web service project using Visual Studio.NET. We create the web service by going through the following steps:

  • In the New Project dialog box, we select Visual C# Projects from the left hand side and select ASP.NET web service from the right side.
  • We will name this web service project as ContentPublisherService.
  • We will also change the name of the default class from service1 to ProviderService.

After the project is created, we import the System.Data.SqlClient namespace into our project. This is done to have access to a set of high performance classes that expose functionality to access the SQL Server database in the managed environment.

Let us look at the code required for implementing the web service. Firstly, we need to import all the required namespaces:

using System;
using System.Collections;
using System.ComponentModel;
using System.Data;
using System.Data.SqlClient;
using System.Diagnostics;
using System.Web;
using System.Web.Services;
using System.Configuration;

Let us examine the ProductDetails class, which is used to represent a single instance of a product in our site. The ProductDetails class contains protected member variables that are used to hold information about a product. We will expose these member variables in the form of properties, since it allows us to exercise a greater degree of control over the values that can be assigned to the member variables. It also permits us to execute a set of statements when values are assigned to the member variables.

public class ProductDetails
{
protected int m_ProductID;
protected int m_CategoryID;
protected string m_ModelNo;
protected string m_ModelName;
protected string m_Image;
protected string m_Price;
protected string m_Description;

public int ProductID
{
get
{
return m_ProductID;
}
set
{
m_ProductID = value;
}
}

public int CategoryID
{
get
{
return m_CategoryID;
}
set
{
m_CategoryID = value;
}
}

public string ModelNo
{
get
{
return m_ModelNo;
}
set
{
m_ModelNo = value;
}
}

public string ModelName
{
get
{
return m_ModelName;
}
set
{
m_ModelName = value;
}
}

public string Image
{
get
{
return m_Image;
}
set
{
m_Image = value;
}
}

public string Price
{
get
{
return m_Price;
}
set
{
m_Price = value;
}
}

public string Description
{
get
{
return m_Description;
}
set
{
m_Description = value;
}
}
}

Even though the web service consists of many methods, we will only examine the important methods.

Returning a Dataset from the web service

In this section, we will create a method called CategoriesGet that returns a Dataset as a resultant value from the web service. Returning a dataset from a web service is an extremely powerful feature of web services, as datasets can be used to store complex information and relationships in an intelligent XML based schema.

public class ProviderService :
            System.Web.Services.WebService
{
[WebMethod(Description="The
            CategoriesGet method allows a remote client to retrieve all the
categories available in the site.",
            EnableSession=false)]

public DataSet
            CategoriesGet()
{
DataSet oDataSet;
SqlConnection
            oSQLConnection;
SqlDataAdapter oSQLCommand;
oSQLConnection =
            new
            SqlConnection(ConfigurationSettings.AppSettings["connectionString"]);
try
{
oSQLCommand = new SqlDataAdapter("CategoriesGet",
            oSQLConnection);
//Set the CommandType to indicate that we want
            to execute a stored
            procedure
oSQLCommand.SelectCommand.CommandType =
            CommandType.StoredProcedure;
oDataSet = new
            DataSet();
oSQLCommand.Fill(oDataSet, "Categories");
//Return
            the DataSet to the caller of the web service
return
            oDataSet;
}
catch (Exception ex)
{
throw ex;
}
finally
{
//Close the connection if it is still open
if
            (oSQLConnection.State == ConnectionState.Open)
{
oSQLConnection.Close();
}
}
}

Let us walk through the above code. As you can see from the following declaration, the ProviderService class is derived from the class System.Web.Services.WebService. When we inherit a class from the WebService class, our class will have access to the common ASP.NET objects such as Application, Session, User and Context.

public class ProviderService : System.Web.Services.WebService
{

In the code shown below, the attribute WebMethod indicates that the method is to be exposed as web callable method and when it is deployed, the ASP.NET runtime provides all the plumbing required to make this method callable across the internet using protocols like HTTP. We also specify a brief description of the functionality of our web method by using the named parameter Description of the WebMethod attribute. In our case, since we do not want to store session state for our method, we set it to false. Setting EnableSession to false allows us to indicate to the ASP.NET runtime that we do not want the session specific data to be stored, thereby eliminating the performance overheads that may be caused due to the amount of resources required for storing the session state.

[WebMethod(Description="The CategoriesGet method allows a remote client to retrieve
all the categories available in the site.", EnableSession=false)]

As the name of the method suggests, CategoriesGet returns a DataSet as the return value of the method.

public DataSet
            CategoriesGet()
{
DataSet oDataSet;
SqlConnection
            oSQLConnection;
SqlDataAdapter oSQLCommand;

In this line, we create an instance of the SqlConnection object, passing to it the connection string that is required for establishing a connection to the database.

oSQLConnection = new SqlConnection(ConfigurationSettings.AppSettings["connectionString"]);

In the above line, we get the connection string from the <appsettings> section of the web.config file using the AppSettings property of the ConfigurationSettings class. To be able to dynamically retrieve the settings, we need to have the connection string defined as follows in the web.config file.

<appSettings>
<add key="connectionString" value="server=localhost;uid=sa;pwd=;database=ContentProvider" />
</appSettings>

The web.config file is used to store all the configuration information for an ASP.NET application, and it essentially replaces the IIS Metabase that was used to store the configuration settings in a traditional ASP application. Web.config is located in the root directory of the web application. This file is made up of nested hierarchical XML tags that are utilized to store information such as type of authentication, debugging settings, handling of sessions, and much more. Also, as we can see from the above, it can be used as a placeholder to store settings that are specific to the application. This practice of storing configuration settings in the web.config file gives administrators of the web site the ability to change the configuration settings dynamically without having to touch the application code.

Here we enclose all of the executable statements in a try..catch block to make sure that exceptions are handled appropriately:

try
{

Next, we create an instance of an SqlDataAdapter object, passing to its constructor the name of the stored procedure we want to execute and the Sqlconnection object that we created in the previous step.

oSQLCommand = new SqlDataAdapter("CategoriesGet", oSQLConnection);

We then set the SelectCommand property to the appropriate value to indicate that we want to execute a stored procedure:

//Set the CommandType to indicate that we want to execute a stored procedure
oSQLCommand.SelectCommand.CommandType = CommandType.StoredProcedure;

Now we use the Fill method to retrieve the data from the Categories table by executing the previously assigned stored procedure against the data source.

oDataSet = new DataSet();
oSQLCommand.Fill(oDataSet, "Categories");
//Return the DataSet to the caller of the web service
return oDataSet;
}

If any error occurs during the execution of the above statements, control will be automatically transferred to the catch block, where we simply raise the error back to the client:

             catch
            (Exception ex)
{
throw ex;
}

Since the statements in the Finally block are executed regardless of whether the statements in the try block are executed successfully or not, we use this opportunity to inspect the State property of the SqlConnection object. If the connection is still open, we close it by calling the Close method of the connection object. (You can leave this part if you wish, as the Close method will do an implicit check anyway).

      finally
{
//Close
            the connection if it is still open
if (oSQLConnection.State ==
            ConnectionState.Open)
{
oSQLConnection.Close();
}
}
}

Returning a user-defined object from the web service

As the name of the method suggests, this method returns a user-defined object named ProductDetails as the return value of the web service.

The method is defined as follows:

[WebMethod(Description="The ProductDetailsGetObject method allows a
remote client to retrieve the details of a product based on the product id in the form of a ProductDetails
object.", EnableSession=false)]
public ProductDetails ProductDetailsGetObject(int iProductID)
{
DataSet oDataSet;
SqlConnection oSQLConnection;
SqlDataAdapter oSQLCommand;
DataTable dtProdTable;
ProductDetails oProdDetails;
oSQLConnection = new SqlConnection(ConfigurationSettings.AppSettings["connectionString"]);
try
{
oProdDetails = new ProductDetails();
oSQLCommand = new SqlDataAdapter("ProductDetailsGet", oSQLConnection);

oSQLCommand.SelectCommand.CommandType = CommandType.StoredProcedure;
oSQLCommand.SelectCommand.Parameters.Add(new SqlParameter("@ProductID", SqlDbType.Int));
oSQLCommand.SelectCommand.Parameters["@ProductID"].Value = iProductID;

oDataSet = new DataSet();
oSQLCommand.Fill(oDataSet, "Products");
//Assign the table to the DataTable object variable
dtProdTable = oDataSet.Tables["Products"];
//Assign the column values to the ProdDetails object
oProdDetails.ProductID = Convert.ToInt32(dtProdTable.Rows[0]["ProductID"].ToString());
oProdDetails.CategoryID = Convert.ToInt32(dtProdTable.Rows[0]["CategoryID"].ToString());
oProdDetails.ModelNo = dtProdTable.Rows[0]["ModelNo"].ToString();
oProdDetails.ModelName = dtProdTable.Rows[0]["ModelName"].ToString();
oProdDetails.Image = dtProdTable.Rows[0]["Image"].ToString();
oProdDetails.Price = dtProdTable.Rows[0]["Price"].ToString();
oProdDetails.Description = dtProdTable.Rows[0]["Description"].ToString();
//Return the object to the caller
return oProdDetails;
}

catch (Exception ex)
{
throw ex;
}
finally
{
//Close the connection if it is still open
if (oSQLConnection.State == ConnectionState.Open)
{
oSQLConnection.Close();
}
}

}

The above method is similar to the previous method CategoriesGet except for the difference that in this method, we convert the contents of the dataset into a user-defined object. Finally we return the ProductDetails object back to the callers of this method.

Apart from the above methods, the ContentPublisher web service also exposes other methods such as ProductsGetByCategoryID and ProductDetailsGetDataSet. Since these methods are implemented in a similar way, we will not consider them in detail here.

Now that we have had a brief look at the web service methods, let us turn our attention to the implementation of the .NET component named ShoppingAssistantLib, which provides such functionalities as checking the credentials of the user, and registering the users in the ShoppingAssistant web site.

Implementation of .NET component ShoppingAssistantLib

We will start by creating a new Visual C# Class library project named ShoppingAssistantLib as shown below:

Once the project is created, we will change the name of the default class from Class1 to User. And we will also add a new class named UserDetails that looks like the following.

public class UserDetails
{
protected
            string m_UserID;
protected string m_Password;
protected string
            m_Name;
protected string m_Address;
protected string
            m_City;
protected string m_State;
protected string
            m_Zip;

public string UserID
{
get
{
return
            m_UserID;
}

set
{
m_UserID = value;
}

}

public string Password
{
get
{
return
            m_Password;
}

set
{
m_Password =
            value;
}

}

public string
            Name
{

get
{
return
            m_Name;
}
set
{
m_Name = value;
}
}

public
            string Address
{
get
{
return
            m_Address;

}

set
{
m_Address =
            value;
}
}

public string
            City
{

get
{
return
            m_City;
}
set
{
m_City =
            value;
}

}

public string
            State
{
get
{
return
            m_State;
}

set
{
m_State =
            value;
}

}

public string
            Zip
{
get
{
return m_Zip;
}
set
{
m_Zip =
            value;
}

}

As you can see from the above, the UserDetails class provides the data structure for storing the necessary details of a specific user. An instance of this class will represent a user in our users table.

The User class consists of two methods named CheckLogIn and SaveUserDetails. We will discuss both of these methods in detail in a moment. Before that we will consider the code of the user defined exception class named InValidLoginException, which is defined as follows:

public class InValidLoginException :
            Exception
{
public InValidLoginException(string sMessage) :
            base(sMessage)
{

}
}

By employing user-defined exception classes, we can publish to the clients that we will only be raising exceptions of predetermined type. It also allows us to implement custom exception processing logic like logging the errors to the event log and sending an email to the administrator, any time an error occurs. In the constructor of the InValidLoginException class, we invoke the constructor of the base exception class and store the exception message for later retrieval.

The CheckLogin method is defined as follows:

public bool CheckLogIn(string
            sUserName,string sPassWord )
{
SqlConnection
            oSQLConnection;
SqlCommand oSQLCommand;
oSQLConnection = new
            SqlConnection(ConfigurationSettings.AppSettings["connectionString"]);
try
{
oSQLCommand = new SqlCommand("UserLogIn",
            oSQLConnection);
//Mark the Command as a
            SPROC
oSQLCommand.CommandType =
            CommandType.StoredProcedure;

//Add the User name
            parameter
SqlParameter oSQLUserNameParam = new
            SqlParameter("@UserID", SqlDbType.VarChar,
            20);
oSQLUserNameParam.Value =
            sUserName;
oSQLCommand.Parameters.Add(oSQLUserNameParam);

//Add
            the password parameter
SqlParameter oSQLPassWordParam = new
            SqlParameter("@Password", SqlDbType.VarChar,
            10);
oSQLPassWordParam.Value =
            sPassWord;
oSQLCommand.Parameters.Add(oSQLPassWordParam);
//Add
            the return value parameter
SqlParameter oSQLRetValueParam = new
            SqlParameter("@RetValue", SqlDbType.Int,
            4);
oSQLRetValueParam.Direction =
            ParameterDirection.Output;
oSQLCommand.Parameters.Add(oSQLRetValueParam);
//Execute
            the
            command
oSQLConnection.Open();
oSQLCommand.ExecuteNonQuery();
oSQLConnection.Close();

//Obtain
            the return value of the stored procedure into a variable
int
            iRetValue = (int)oSQLRetValueParam.Value ;
if (iRetValue == -1)
{
throw new InValidLoginException("Invalid
            Login");
}
else
{
//Return true to indicate that the
            login is successful
return true;
}
}
catch(Exception
            ex)
{
throw ex;
}
finally
{
if
            (oSQLConnection.State ==
            ConnectionState.Open)
{
oSQLConnection.Close();
}
}
}

In the above method, we authenticate a customer's user name and password against the ShoppersInfo database. We return true if the user name and password match up against a record in the Users table, otherwise we raise an exception of type InValidLoginException.

The SaveUserDetails method allows us to save the details of a user to the database. It takes an object of type UserDetails as the argument and executes a stored procedure named InsertUser.

public bool SaveUserDetails(UserDetails oUserDetails)
{
SqlConnection oSQLConnection;
SqlCommand oSQLCommand;
oSQLConnection = new SqlConnection(ConfigurationSettings.AppSettings["connectionString"]);
try
{
oSQLCommand = new SqlCommand("InsertUser", oSQLConnection);
oSQLCommand.CommandType = CommandType.StoredProcedure;

oSQLCommand.Parameters.Add(new SqlParameter("@UserID", SqlDbType.Char, 20));
oSQLCommand.Parameters["@UserID"].Value = oUserDetails.UserID;

oSQLCommand.Parameters.Add(new SqlParameter("@Password", SqlDbType.Char, 10));
oSQLCommand.Parameters["@Password"].Value = oUserDetails.Password;

oSQLCommand.Parameters.Add(new SqlParameter("@Name", SqlDbType.VarChar, 128));
oSQLCommand.Parameters["@Name"].Value = oUserDetails.Name;

oSQLCommand.Parameters.Add(new SqlParameter("@Address", SqlDbType.VarChar, 128));
oSQLCommand.Parameters["@Address"].Value = oUserDetails.Address;

oSQLCommand.Parameters.Add(new SqlParameter("@City", SqlDbType.VarChar, 50));
oSQLCommand.Parameters["@City"].Value = oUserDetails.City;

oSQLCommand.Parameters.Add(new SqlParameter("@State", SqlDbType.Char, 2));
oSQLCommand.Parameters["@State"].Value = oUserDetails.State;

oSQLCommand.Parameters.Add(new SqlParameter("@Zip", SqlDbType.Char, 5));
oSQLCommand.Parameters["@Zip"].Value = oUserDetails.Zip;

oSQLConnection.Open();
oSQLCommand.ExecuteNonQuery();
oSQLConnection.Close();
return true;
}

catch (Exception ex)
{
throw ex;
}
finally
{
if (oSQLConnection.State == ConnectionState.Open)
oSQLConnection.Close();
}
}

ReportInfo class

Apart from UserDetails and User classes, we also define a class named ReportInfo whose main purpose is to expose methods that allow us to add reporting related information to the database.

public class ReportInfo
{

The implementation of the SaveReportInfo method is quite simple; it accepts the supplied input parameters and executes a stored procedure named InsertReportInfo, passing to it all the parameters:

public bool SaveReportInfo(int intProductID, int intCategoryID,
string strBrowser, string strRequestType,
string strAuthenticated)
{
SqlConnection oSQLConnection;
SqlCommand oSQLCommand;
string strConnection;
strConnection = "server=localhost;uid=sa;pwd=;Initial Catalog=ShoppersInfo";
oSQLConnection = new SqlConnection(strConnection);
try
{
oSQLCommand = new SqlCommand("InsertReportInfo", oSQLConnection);
oSQLCommand.CommandType = CommandType.StoredProcedure;

oSQLCommand.Parameters.Add(new SqlParameter("@ProductID", SqlDbType.Int));
oSQLCommand.Parameters["@ProductID"].Value = intProductID;

oSQLCommand.Parameters.Add(new SqlParameter("@CategoryID", SqlDbType.Int));
oSQLCommand.Parameters["@CategoryID"].Value = intCategoryID;
oSQLCommand.Parameters.Add(new SqlParameter("@Browser", SqlDbType.VarChar, 256));
oSQLCommand.Parameters["@Browser"].Value = strBrowser;

oSQLCommand.Parameters.Add(new SqlParameter("@RequestType", SqlDbType.VarChar, 256));
oSQLCommand.Parameters["@RequestType"].Value = strRequestType;

oSQLCommand.Parameters.Add(new SqlParameter("@Authenticated", SqlDbType.VarChar, 50));
oSQLCommand.Parameters["@Authenticated"].Value = strAuthenticated;

oSQLConnection.Open();
oSQLCommand.ExecuteNonQuery();
oSQLConnection.Close();
return true;
}

catch (Exception ex)
{
throw ex;
}
finally
{
if (oSQLConnection.State == ConnectionState.Open)
oSQLConnection.Close();
}
}
}

Implementation of ShoppingAssistant Web Application

In this part of the case study, we will discuss the implementation of the ShoppingAssistant web application that uses the following building blocks that we have already created:

  • ContentPublisherService - ASP.NET Web service

Provides information such as lists of categories available, products present in each category, and the details of the product. We add a reference to this web service by selecting the Add Web Reference option.

  • ShoppingAssistantLib -C# Class library

Exposes a method that allows the ShoppingAssistant application to create new users in the ShoppingAssistant database. It also provides for validation of a particular user's credentials against the database. We add a reference to this library by using the Add Reference option.

We will start with our discussion of the web application by considering the creation of a user control named Header.ascx that is used to display standard header information in all the pages of the application.

Header User Control

In the ShoppingAssistant web application, to provide consistent look across the pages, we will create a user control named _Header.ascx that is used to provide the standard heading. The code of the user control _Header.ascx looks like the following:

<%@ Control Language="c#" AutoEventWireup="false"
Codebehind="_Header.ascx.cs" Inherits="ASPToday.C_Header" %>
<asp:Table id="tblTop" style="Z-INDEX: 101; LEFT: 12px; POSITION: absolute;
TOP: 11px" runat="server" Width="735px" Height="55px" ForeColor="Blue">
<asp:TableRow>
<asp:TableCell>
<img src="Images/newwroxlogo.gif">
</asp:TableCell>
<asp:TableCell>
<img align="right" src="Images/newwroxhead.gif">
</asp:TableCell>
</asp:TableRow>
<asp:TableRow HorizontalAlign="Center">
<asp:TableCell ColumnSpan="2">
Asynchronous Programming in .NET
</asp:TableCell>
</asp:TableRow>
</asp:Table>

Login Process

In our site, the user must be logged in to perform tasks such as browsing through the site and adding a new product to the database. As we already discussed, the login page authenticates the customer's user name and password against the ShoppersInfo database. Once validated, the user is redirected to the categories listing page. If the user does not have a valid login, he can opt to create one by clicking the hyperlink C lick here if you are a new user on the login page. Clicking the hyperlink takes the user to the registration page where the user provides all the necessary details for completing the registration.

Underneath the Login page, we implement a forms-based login authentication mechanism, which allows the customers to authenticate themselves to the ShoppingAssistant application. This form-based authentication technology depends on cookies to store the authentication information for the current user. Once the user is authenticated, cookies are used to store and maintain session information thereby allowing us to identify the current user.

To enable forms-based authentication in our application, we need to make the following entry in the web.config file.

    <authentication
            mode="Forms">
<forms name="ShoppingAssistantAuth"
            loginUrl="Login.aspx"
            path="/"></forms>
</authentication>

The loginUrl attribute specifies the name of the login page that we want the users to be redirected to any time they access a resource that does not allow anonymous access.

For every web form that we want to secure using the forms-based authentication mechanism, we need to add an entry to the web.config file. For example, to set the restrictions of authenticated user access for a page called ProductListing.aspx, we need to set the following entry directly under the configuration tag of the web.config file.

<location path="ProductListing.aspx">
<system.web>
<authorization>
<deny users="?" />
</authorization>
</system.web>
</location>

When a user attempts to access the ProductListing.aspx page, the ASP.NET forms-based security system will automatically redirect them to the Login.aspx page, and will continue to prevent them from accessing it until they have successfully validated their user name and password credentials to the ShoppingAssistant application. Similarly we protect the other above-mentioned pages also using the entries in the web.config file.

Now we will turn our attention back to the code that is needed for implementing the login functionality in our application.

The code-behind file of the login page (Login.aspx.cs) contains the following statements. Firstly, we import the ShoppingAssistant namespace in to our application.

private void btnLogin_Click(object sender,
            System.EventArgs e)
{
try
{
User oUserLogIn = new
            User();
//CheckLogIn returns true to indicate that the login is
            successful
if (oUserLogIn.CheckLogIn(txtUserName.Text,
            txtPassword.Text) ==
            true)
{
FormsAuthentication.SetAuthCookie(txtUserName.Text,
            true);
Response.Redirect("CategoryListing.aspx");
}
}
catch(InValidLoginException ex)
{
lblMessage.Visible =
            true;
lblMessage.Text = ex.Message;
}
}

When the user enters the user id and password and then clicks Login, we invoke the CheckLogin method to verify if the user is present in the Users table. As we already discussed, the CheckLogin method returns true if the login is successful, else it throws an exception of type InValidLoginException.

If the login is successful, we then invoke the SetAuthCookie method to generate an authentication ticket for the authenticated user name and password, and attach it to the cookies collection of the outgoing response. Once the cookie is generated, it is used to maintain information about the session information for every user that logs in to our site. In the next line, we perform a redirect to the CategoryListing.aspx page and display information about the categories available in the site.

CategoriesListing process

All the categories present in the site are displayed through the CategoryListing.aspx page. In the categories listing page, we use an asp:DataGrid web control to display the categories. We bind this control directly to the DataSet returned by the web service in the Page_Load event. When called from browser, the Categories listing page looks like the following:

In the CategoriesListing.aspx.cs file, we write the following statements of code:

private void Page_Load(object sender, System.EventArgs e)
{
// Put user code to initialize the page here
ProviderService oService = new ProviderService();
DataSet oDataSet;
oDataSet = oService.CategoriesGet();
//Assign the Default view of the Categories table to the Categories datagrid
gridCategories.DataSource = oDataSet.Tables["Categories"].DefaultView;
}

In the above lines of code, we create an instance of the web service object, invoke its CategoriesGet method, and finally bind the resultant dataset to the data grid.

ProductListing process

The ProductListing.aspx page shows the list of products available on the site based on the category the user has selected in the categories listing page. The screenshot of the ProductListing page is as follows:

The Page_Load event handler contains the code to invoke the web service, retrieve data from it, and then display its contents into a data grid control:

private void Page_Load(object sender,
            System.EventArgs e)
{
// Put user code to initialize the page
            here
int iCategoryID;
ProviderService oService = new
            ProviderService();
DataSet oDataSet;
iCategoryID =
            Convert.ToInt32(Request.QueryString["CategoryID"]);
oDataSet =
            oService.ProductsGetByCategoryID(iCategoryID);
gridProducts.DataSource
            =
            oDataSet.Tables["Products"].DefaultView;
gridProducts.DataBind();
}

In the Page_Load event, we retrieve the CategoryID passed in the query string, and supply it as a parameter when invoking the web service method ProductsGetByCategoryID.

Product Details listing process

When the user selects a particular product from the list, the user is taken to the product details page where details about the product are shown. For showing the product details, we will create a reusable user control named ProductDetailControl.ascx, which encapsulates all the code required for retrieving the details of the product from the web service. It exposes a public property called ProductID. By setting the ProductID property to an appropriate value, we can determine the product whose details are to be shown.

The user control uses an asp:Repeater web control to display the information about the specified product.

Let us take a brief look at the code of the code-behind file (ProductDetailControl.ascx.cs) for the user control:

private void Page_Load(object sender, System.EventArgs e)
{
// Put user code to initialize the page here
ProviderService oService = new ProviderService();
DataSet oDataSet;
//Get the ProductID from the Querystring
ProductID = Convert.ToInt32(Context.Request.QueryString["ProductID"]);
oDataSet = oService.ProductDetailsGetDataSet(ProductID);
//Bind the Control to the DataView returned by the method
prodDetailControl.DataSource = oDataSet.Tables["Products"].DefaultView;
prodDetailControl.DataBind();

//Hide the list if no items are in it
if (prodDetailControl.Items.Count == 0)
prodDetailControl.Visible = false;
}

Here again, we directly bind the dataset returned by the web service to the asp: repeater web control.

Testing the ShoppingAssistant application

Now that we have constructed the different parts of the application, let us test it by navigating to the login page of our site. If we enter a valid user id and password and click login, we will be directed to the categories listing page where all the categories in the site are displayed. Clicking the hyperlink Show all Products takes us to the product listing page where we can see all the products that belong to the selected category. If we click the hyperlink Product Details in the product-listing page , we should be able to see the details of the specific product.

However there is one disadvantage in the above approach. Due to the synchronous approach we have adopted, every time a request is made, we go across the Internet to retrieve the details from the web service. Obviously the response time is severely impacted due to the network trip that is made. Since the network loads can be unpredictable, systems can become backlogged or unable to process requests in a timely manner. This may lead to system failure as services time out or become unavailable. When this occurs, consumers of synchronous web services may lock critical resources while waiting for a result that may never come.

To overcome this, we will implement an asynchronous approach wherein we will implement a combination of a Windows service application and a Timer component and call the web service asynchronously. The result of this asynchronous web service execution is then saved into a local XML file, which will then be used by the ShoppingAssistant web application.

Implementation of asynchronous web services and windows service application to access the information from the web service

As we mentioned already, in this section we will see how we can substitute the synchronous approach with the asynchronous approach, thereby improving the response time of the site. The proposed architecture to accomplish the above requirement is as follows:

Here, our requirement is to get the data from the remote web services in an asynchronous manner and then store that information in the form of XML files in the local folder. Building a solution of this kind requires us to employ some of the features of .NET Framework such as Windows Service applications and the server side thread-safe Timer component. We will tie the individual pieces together in the following manner.

  • A Windows service application that has the Timer component invokes the remote web service in an asynchronous fashion. While invoking the web service, it also passes the name of the function, which is used to call back once the result is returned from the web service. Once the web service has finished its execution, it notifies the client application by invoking the callback function. This callback function then writes the returned result into a local XML file for the ShoppingAssistant web application to make use of.
  • Due to the presence of the Timer component, we execute the above-mentioned action for every pre-determined amount of time, and this ensures that we have the latest information from the web service in the local XML file. The frequency with which the web service is invoked is determined by the value set for the I nterval property of the timer component.
  • The ShoppingAssistant web forms application then reads the information to be displayed on the site from the local XML file instead of making a blocking synchronous call to the remote web service.

Before we start looking at the code, let us briefly go over some of the .NET features like Windows service and Timer component.

Windows Service

There are times where you may want to have your code always running on your server. If you have ever done any work in MSMQ, you might have created an application that polls the message queue every predefined amount of time and checks for new messages. In that case, the application that checks the queue for messages should always be running as a Windows NT Service to be able to have the ability to poll the message queue frequently. These windows services do not have any user interface and we can configure windows services in such a way that they can be automatically started when the computer starts, and they can also be paused and restarted at any time.

Prior to Visual Studio.NET, if you wanted to write a windows service application either you had to use the template provided by ATL (Active Template Library that exposes a set of classes used for COM programming in the Windows environment) or if you are a VB programmer, you have to embed custom NT service controls in VB to achieve the same functionality. But with Visual Studio .Net, you can easily create an application that has the capability to run as a Service. Visual Studio.Net is supplied with a new project template called Windows Service that provides all the plumbing required for creating the application that can run as a Service. When you create a Visual Studio .Net project as a Service, you can write code to respond to the actions like when the service is started, paused, resumed, and stopped.

Once you create the service, it has to be installed using InstallUtil.exe (Command line utility) which is used to install the service on the local computer. After you install the service, you can start, stop, pause, and resume it using the Service Control Manager.

Timer Component

One of the exciting features of the .NET framework is the ability to create applications that can raise events at scheduled intervals. The Timer component that is contained in the System.Timers namespace is a server side component that exposes methods and properties through which we can capture events raised by the timer component and take specific actions according to the requirements.

You can create an instance of the Timer component by using any one of the following ways:

  • You can drag an instance of the timer component from the Components tab of the Toolbox to the designer.
  • You can create an instance of the timer component programmatically after importing the namespace System.Timers namespace.

The Timer component exposes a property named Interval, the value of which determines the specified interval at which the Elapsed event is raised. As the name suggests, the Elapsed event is raised whenever the specified time interval expires, and we can write code in this event to implement the custom processing logic that we want to execute.

In this case study, we are going to use the Timer component to raise events at the elapsed intervals that will allow us to make a request to a remote web service in an asynchronous fashion. The frequency at which we make the request is determined by the value of the Interval property. It is also important to note that the server based timer component is different from the windows based timer control that is contained in the System.Windows.Forms namespace. Since the server-based timer components are designed for use with worker threads in a multithreaded environment, they can be more efficient in multithreaded environment unlike their windows based counterparts.

Let us get started by creating the windows service application.

Implementation of Windows Service application

To create a Windows Service application in Visual C#, select File->New Project and then

select Visual C# Projects from the Project Types, and select Windows Service from the List of templates.

Once we create the new project WinInformationPollingService, we open up the default service class and rename it to PollingService.

Through the property pages of the PollingService, we can set properties such as CanStop and CanShutdown to either true or false. These settings determine what methods can be called on your service at runtime. For example, when the CanStop property is set to true, the OnStop method will be automatically called when the service is stopped through the Service Control Manager.

Once we set all the properties to appropriate values, we need to add the installers that are required for installing our windows service application onto the target machine as a service. To add these installers, we click on the hyperlink Add Installer (shown in the above figure) in the properties window of the PollingService class.

Before we start looking at code, let us briefly explore some of the theory behind the creation of Windows Service applications.

The two main classes involved in Windows Service creation are:

  • System.ServiceProcess.ServiceBase

The ServiceBase class exposes the following methods and we can override these methods to determine what happens when we change the state of the service (like starting, stopping, etc) in the Services Control Manager.

We can write code in the following events depending on the requirements.

  • OnStart - Invoked when the service is started.
  • OnPause - Invoked when the service is paused.
  • OnStop - Invoked when the service stops running.
  • OnContinue - To decide the behavior that should happen when the service resumes normal functioning after being paused for a while.
  • OnShutDown - To indicate what should happen just prior to system shutdown, if the service is running at that time.
  • System.ServiceProcess.ServiceProcessInstaller and System.ServiceProcess.ServiceInstaller

These classes are used to install and uninstall the service. As we already mentioned, these classes are automatically added when we click the Add Installer in the properties windows of the service class.

Once we carry out all these operations, we can install or uninstall the Service by using the InstallUtil.exe utility. An alternative approach to this would be to use the Set Up and Deployment project template to create setup windows installer executables, which can later be executed to install the windows service on to the client machine.

Since our windows service has to invoke the methods of the ContentPublisherService to get the information, we add a reference the web service that we created in the previous step. In Visual Studio.NET, we can do this using the Add Web Reference option.

To add the web reference, right click on the project in the Solution explorer and select Add Web Reference. In the Add Web Reference dialog box, enter the path of the ..asmx file of the web service.

When you add a web reference of a web service to your project, Visual Studio.NET automatically generates a proxy class that not only interfaces with the web service, but also provides a local representation of the web service.

Let us look at the code required to implement our windows service:

using System;
using System.Collections;
using System.ComponentModel;
using System.Data;
using System.Diagnostics;
using System.ServiceProcess;
using System.Xml;
using System.Data.SqlClient;

public class PollingService : System.ServiceProcess.ServiceBase
{
private localhost.ProviderService m_oService;
private System.IAsyncResult m_oCategoriesAsyncResult;
private System.IAsyncResult m_oProductsAsyncResult;

protected override void OnStart(string[] args)
{

m_oService = new localhost.ProviderService();
timer1.Enabled = true;

}

private void WriteCategories(IAsyncResult oRes )
{
DataSet oCategoriesDataSet;
oCategoriesDataSet = m_oService.EndCategoriesGet(oRes);
XmlDataDocument objCategoriesDataDoc = new XmlDataDocument(oCategoriesDataSet);
objCategoriesDataDoc.DataSet.WriteXmlSchema("C:\\AsynchProgramming\\Categories.xsd");
objCategoriesDataDoc.DataSet.WriteXml("C:\\AsynchProgramming\\Categories.xml");

}

private void WriteProducts(IAsyncResult oRes )
{
DataSet oProductsDataSet;
oProductsDataSet = m_oService.EndProductsGet(oRes);
XmlDataDocument objProductsDataDoc = new XmlDataDocument(oProductsDataSet);
objProductsDataDoc.DataSet.WriteXmlSchema("C:\\AsynchProgramming\\Products.xsd");
objProductsDataDoc.DataSet.WriteXml("C:\\AsynchProgramming\\Products.xml");

}

private void timer1_Elapsed(object sender, System.Timers.ElapsedEventArgs e)
{
//Invoke the CategoriesGet method in asynchronous fashion with the callback method
m_oCategoriesAsyncResult = m_oService.BeginCategoriesGet(new AsyncCallback(WriteCategories), null);

//Invoke the CategoriesGet method in asynchronous fashion with the callback method
m_oProductsAsyncResult = m_oService.BeginProductsGet(new AsyncCallback(WriteProducts), null);
}

}

We start by importing all the required namespaces. As you can see, we have also imported the System.Xml namespace to have access to the XML related classes.

using System;
using System.Collections;
using System.ComponentModel;
using System.Data;
using System.Diagnostics;
using System.ServiceProcess;
using System.Xml;
using System.Data.SqlClient;

In the following lines of code, we then declare objects of type ProviderService and IasyncResult:

private localhost.ProviderService m_oService;
private System.IAsyncResult m_oCategoriesAsyncResult;
private System.IAsyncResult m_oProductsAsyncResult;

The OnStart method will be executed when the service is started from the Service Control Manager. In this method, we create an instance of our PollingService and enable the timer component so that the Elapsed event is fired at periodic intervals:

protected override void OnStart(string[]
            args)
{

m_oService = new
            localhost.ProviderService();
timer1.Enabled =
            true;

}

As we already said, the windows service application has got a timer component that has its interval property set to 5000 milliseconds. Whenever this interval elapses, the timer component automatically fires the Elapsed event. In this event, we call the BeginCategoriesGet and BeginProductsGet methods of the web service. We also pass an instance of the AsyncCallback delegate object. We pass the name of the callback method that is to be called when the asynchronous operation completes to the constructor of the AsyncCallback class as an argument. Both the BeginCategoriesGet and BeginProductsGet methods return an object of type IAsyncResult that can later be used to check the status of the asynchronous operation.

private void timer1_Elapsed(object sender, System.Timers.ElapsedEventArgs e)
{
//Invoke the CategoriesGet method in asynchronous fashion with the callback method
m_oCategoriesAsyncResult = m_oService.BeginCategoriesGet(new AsyncCallback(WriteCategories), null);

//Invoke the CategoriesGet method in asynchronous fashion with the callback method
m_oProductsAsyncResult = m_oService.BeginProductsGet (new AsyncCallback(WriteProducts), null);
}

Since we passed WriteCategories as the name of the callback function at the time of invoking the BeginCategories method, when the CategoriesGet method (of the web service) has finished its execution it will automatically invoke the WriteCategories method. In the WriteCategories method, we will write the resultant value from the web service into a local XML file. The case with the invocation of WriteProducts method is similar.

The WriteCategories method is called once the web service invocation is returned from the web service. It is also passed an object of type IAsycnResult that encapsulates the results of the asynchronous call that we made.

Now that we have a fully-functional windows service application that asynchronously calls the web service and ensures that the latest categories and products information are available locally, we can modify our web forms to retrieve data from the local .xml files, instead of calling out to the remote web service every time the web form is drawn.

To illustrate the modification that needs to be done, let us consider the CategoryListing page, where we have the following lines of code in the Page_Load event:

private void Page_Load(object sender, System.EventArgs e)
{
// Put user code to initialize the page here
ProviderService oService = new ProviderService();
DataSet oDataSet;
oDataSet = oService.CategoriesGet();
//Assign the Default view of the Categories table to the Categories datagrid
gridCategories.DataSource = oDataSet.Tables["Categories"].DefaultView;
}

Now that we have the data available locally, let us modify the above code to read from the local xml file thereby avoiding the costly network (or Internet call) to the web service. After modification, the code looks like the following:

private void Page_Load(object sender,
            System.EventArgs e)
{

XmlDataDocument objDataDoc = new
            XmlDataDocument();
objDataDoc.DataSet.ReadXmlSchema("C:\\AsynchProgramming\\Categories.xsd");
objDataDoc.Load("C:\\AsynchProgramming\\Categories.xml");
gridCategories.DataSource =
            objDataDoc.DataSet.Tables["Categories"].DefaultView;
gridCategories.DataBind();
}

Similar to the above, we perform the same modification to all the other web forms as well to take advantage of the information that is available locally.

If you go back to the ShoppingAssistant web site and navigate through the pages, you will find that the performance and throughput of the site have improved vastly due to the change in the way we obtain the data for display.

Implementation of FileSystemWatcher component to facilitate information gathering

Now we have a working web site application that shows all the information related to categories and products available in the site. Let us say, for example, one of the partners of the ShoppingAssistant site wants to find out the number of times a particular product has been viewed by the customers. To accomplish this, we need to store the details about the product information (that we are displaying on the page) in a database. If we were to handle this process of storing of data in a synchronous way (like making a database call across the network and storing that information in the database), the user would have to wait until the reporting information is stored in the database because the control will not be returned to the caller application until the changes are saved. This might result in an increase in the response time for the resultant page to be drawn on the user's browser.

Let us now see how we can adopt an asynchronous approach to accomplish the desired result. We will add some more elements to our existing architecture.

When the user navigates to the Products page, we perform a number of independently running operations to ensure that the data relevant for reporting purposes is stored in the database:

  • When the user navigates to the ASP.NET web page (web form that displays the product details), we take the products information before showing it in the web form, convert that information into an xml file and store it in the specified directory.
  • This specified directory is already set up for monitoring for creation of new files by the FileSystemWatcher component, which is placed in a Windows Service Application that always runs as a service.
  • When an .xml file is created in that directory, an event is automatically raised to the Windows Service application that passes the path of the file to our ShoppingAssistantLib, which, in turn, parses the file and stores the details in a database table called ReportInfo.

By performing the above operation in an asynchronous fashion, we allow the control to return to the application as soon as the reporting information is captured in an xml file (as opposed to storing that information in a database through an expensive network call). After that, a separate set of operations will be used to save the details to the database. This kind of handling not only helps in increasing the performance, but also provides the users with the best possible browsing experience.

We will split this part of the implementation into two sections. In the first section, we will modify the Products web form to store the details of the products information into an xml file before we show the products information to the user. In the second section, we will see how to set up the FileSystemWatcher component on our windows service application to monitor the directory for the creation of files.

Modification to the Products web form to support asynchronous report gathering

In this section, we will modify the products web forms and add the capability to store the products related reporting information into a local xml file before showing the product details. Since all of the display related to a particular product is encapsulated in the user control named ProductDetailControl.ascx, we need to modify it. For that reason, we add the following lines of code to the Page_Load event.

We start by declaring all the required variables:

int iProductID;
int iCategoryID;
string strBrowser;
string strRequestType;
string strAuthenticated;

XmlDataDocument objDataDoc = new XmlDataDocument();

Here we get the ProductID from the query string:

iProductID = Convert.ToInt32(Request.QueryString["ProductID"]);

Now we load the XmlDataDocument object with an xml schema and xml data by using the following statements:

objDataDoc.DataSet.ReadXmlSchema("C:\\AsynchProgramming\\Products.xsd");
objDataDoc.Load("C:\\AsynchProgramming\\Products.xml");

We then filter the DataView object by using the RowFilter property:

objDataDoc.DataSet.Tables["Products"].DefaultView.RowFilter = "ProductID=" + iProductID;

Here, we get the details of the request and store them in local variables:

//Store the parameters related to the Request into local variables
strBrowser = Request.UserAgent.ToString();
strRequestType = Request.RequestType.ToString();
strAuthenticated = Request.IsAuthenticated.ToString();

From the filtered DataView object, we get the ProductID and the CategoryID values:

//Get the corresponding CategoryID into local variable
iCategoryID = Convert.ToInt32(objDataDoc.DataSet.Tables["Products"].Rows[0]["CategoryID"].ToString());
//Call the CreateXMLDocument method to save the details related to reporting to the database

Finally we invoke the CreateXMLDocument method, which we will see in a moment, and pass in all the required parameters.

CreateXMLDocument(strBrowser,strRequestType,strAuthenticated,iCategoryID,iProductID);

The CreateXMLDocument method is defined as follows:

public bool CreateXMLDocument(string strBrowser, string strRequestType, string strAuthenticated,
int intcategoryID, int intproductID)
{

XmlTextWriter objWriter = null;
string strDocument;

try
{
strDocument = "ReportInfo.xml";
objWriter = new XmlTextWriter("C:\\AsynchProgramming\\Drop\\" + strDocument, null);

objWriter.Formatting = Formatting.Indented;

We use the WriteStartDocument to write out the XML declaration with the version and the standalone attributes. Since we pass false to the WriteStartDocument method, the version attribute will be set to "1.0" and the standalone will have the value "no".

objWriter.WriteStartDocument(false);

In this line, we write a Comment to indicate that the ShoppingAssistant application has created this file for reporting purposes:

objWriter.WriteComment("This file represents the information collected for reporting purposes");

Here we write out the specified start tag with the name of the element as ReportInfo:

objWriter.WriteStartElement("ReportInfo",
            null);
objWriter.WriteElementString("Browser", strBrowser);
objWriter.WriteElementString("RequestType", strRequestType);
objWriter.WriteElementString("Authenticated", strAuthenticated);
objWriter.WriteElementString("CategoryID",
            intcategoryID.ToString());
objWriter.WriteElementString("ProductID",
            intproductID.ToString());

After writing out all the child elements, we then write the end tag for the ReportInfo element by using the WriteEndElement method:

      objWriter.WriteEndElement();

//Write the XML to file and close the
            objWriter
objWriter.Flush();
objWriter.Close();
return
            true;
}
catch(Exception ex)
{
throw
            ex;
}
finally
{
}
}

A typical xml file created by the above lines of code looks like the following:

Adding monitoring capability to windows service application using FileSystemWatcher

To add monitoring capabilities, we will add the following lines of code to the PollingService.cs that is part of the windows service application (WinInformationPollingService).

We start by adding the following declaration to the PollingService class:

private FileSystemWatcher mobjFileSystemWatcher;

Now that we have added the above declaration, let us add the following statements to the OnStart method:

//Configure the FileSystemWatcher to watch for changes
mobjFileSystemWatcher = new FileSystemWatcher();

Here we set all the properties of the FileSystemWatcher object so that it starts monitoring the directory named Drop for creation of new .xml files.

We set the path to indicate that we want to monitor the directory called Drop for creation of new .xml files:

mobjFileSystemWatcher.Path = @"C:\AsynchProgramming\Drop";
//Set the Filter to watch for .xml files
mobjFileSystemWatcher.Filter = "*.xml";
//Filter for Last Access changes
mobjFileSystemWatcher.NotifyFilter = NotifyFilters.LastAccess;

As we are interested in watching only for files, not directories or subdirectories, we set the IncludeSubdirectories property to false:

mobjFileSystemWatcher.IncludeSubdirectories = false;

In the following line, we hook up the Changed event of the FileSystemWatcher to the procedure named OnChanged :

// Add event handlers.
mobjFileSystemWatcher.Changed += new FileSystemEventHandler(OnChanged);

Finally, we enable the component to begin watching for the creation of new files:

//Enable the component to begin watching for changes
mobjFileSystemWatcher.EnableRaisingEvents = true;

Once we have enabled the FileSystemWatcher to watch for new files, whenever a new file is created, the OnChanged procedure will be called. In the OnChanged procedure, we invoke a method called SaveReportInfo, which reads the contents of the generated xml file and stores that in the database.

// Define the event handlers.
private
            void OnChanged(object source, FileSystemEventArgs e)
{
//Read
            the contents of the file and save it to the
            database
SaveReportInfo(e.FullPath);
}

As the name of the method suggests, SaveReportInfo reads the contents of the xml file and invokes the SaveReportInfo method of the ReportInfo to store the details to the database:

   private void
            SaveReportInfo(string strPath)
{
XmlTextReader
            objReader;
XmlDocument objDocument;
XmlNode objReportInfoNode;

string strBrowser;
string strRequestType;
string
            strAuthenticated;
int intCategoryID;
int
            intProductID;
objReader = new XmlTextReader(strPath);
try
{
objDocument = new
            XmlDocument();
objDocument.Load(objReader);

We use the DocumentElement property of the XmlDocument object to get the reference of the root XmlElement of the Document:

objReportInfoNode = objDocument.DocumentElement;
//Get all the values from the XML document into local variables

Once we have a reference to the root xml element, we can then retrieve values from its child nodes using the statements shown below:

strBrowser =
            objReportInfoNode.ChildNodes.Item(0).InnerText;
strRequestType =
            objReportInfoNode.ChildNodes.Item(1).InnerText;
strAuthenticated
            = objReportInfoNode.ChildNodes.Item(2).InnerText;
intCategoryID =
            Convert.ToInt32(objReportInfoNode.ChildNodes.Item(3).InnerText);
intProductID
            = Convert.ToInt32(objReportInfoNode.ChildNodes.Item(4).InnerText);

ReportInfo objReport = new
            ReportInfo();
objReport.SaveReportInfo(intProductID,intCategoryID,strBrowser,strRequestType,strAuthenticated);
}
catch
            (Exception ex)
{
throw
            ex;
}
finally
{
objReader.Close();
}

}

Now that we have created the Windows Service application, it has to be installed into the Service Control Manager. It can be done using any of the following ways:

  • Using InstallUtil utility
  • Using the Setup and Deployment project

Putting it all together

Now that we have constructed the different parts of the application, let us test the application by going through the following steps:

  • First, we start the PollingService by opening up the Service Control Manager, right clicking on PollingService and selecting Start from the context menu. Once the service is started, it will keep polling the web service and refresh the contents of the local xml files with the latest information from the web service.

  • Now if we navigate to our ShoppingAssistant web application and browse through the Categories and Products web forms, the latest information from the local xml files will be used to display the information. This is made possible due to the windows service application (that is always running), that continuously invokes the web service to get the latest information from the web service and refreshes the local xml files with that information.
  • While navigating to the product details page, you will also find that all the details related to the displayed product are added to the ReportInfo table, which can be later used for reporting purposes. This is made possible due to the combination of the following operations. Firstly, the web form that displays the Product details page creates an xml file and drops it on a specified directory that is being monitored by the FileSystemWatcher (that is sitting on a windows service application). Secondly, since the FileSystemWatcher monitors the directory for the creation of new files, an event is automatically raised whenever a new file is created. The windows service application captures the event, reads the contents of the xml file and then stores them in the ReportInfo table in the database.

Possible Applications

The asynchronous mechanisms that we discussed above can also be used in the following situations:

  • If the application allows some kind of tolerance in terms of the nature of the data they get from external systems, then it becomes a classical case to implement asynchronous web services.
  • FileSystemWatcher is very powerful in that it can be can be very useful in situations such as system explorers or file managers where we need to handle the file system handling in an asynchronous way.

Case Study Review

In this case study, we have looked at the following features:

  • How to effectively implement asynchronous capabilities into a web application
  • Discussed different asynchronous mechanisms such as asynchronous web services and FileSystemWatcher and understood the scenarios in which they can be used in a production application.
  • How to create a .NET Application that has the capability to run as a Windows Service using Visual Studio. NET.
  • How to use the rich .NET XML classes available in System.XML namespace to write to an .xml file and read from an .xml file.

Although the application we created was simple in functionality, it should provide a solid foundation for understanding how to build high performance, scalable, flexible and reliable web applications using the asynchronous features of the .NET framework such as XML based web services, and FileSystemWatcher.

Case Study Information
Author | Thiru Thangarathinam
Technical Editor | Adam Ryland
Project Manager | Helen Cuthill
Reviewers | Andrew Krowczyk, Saurabh Nandu, Sean Schade, Adam Ryland

If you have any questions or comments about this case study, or suggestions for further pieces, please contact the technical editor. If you wish to talk about the case study with other subscribers, please visit our forum here.

C#Today - © 2002 WroxPress Limited


Using Web Services Asynchronously

Using Web Services Asynchronously
http://www.csharptoday.com/content.asp?id=1820
Article Rating: 4
Published on February 13, 2002
Abstract
As web services continue to grow in popularity, our applications will become more dependant on distributed software components. Today our remote procedure calls typically run across well known and centrally managed local networks. In contrast, web services allow us to penetrate most firewalls and reach across the Internet into networks beyond our control. Along with this flexibility comes unpredictable network latency and bandwidth, and a need to apply new techniques in building applications relying on these unknown parameters. For example, you would not want the GUI of a Windows Form application freezing for ten seconds as the code waits on the result of a sluggish web service call. Fortunately, there are several options available in .NET to use a web service asynchronously. In this article, Scott Allen will examine a few of these techniques in building a responsive and well behaved application.
Article

As web services continue to grow in popularity, our applications will become more dependant on distributed software components. Today our remote procedure calls typically run across well known and centrally managed local networks. In contrast, web services allow us to penetrate most firewalls and reach across the Internet into networks beyond our control.

Along with this flexibility comes unpredictable network latency and bandwidth, and a need to apply new techniques in building applications relying on these unknown parameters. For example, you would not want the GUI of a Windows Form application freezing for ten seconds as the code waits on the result of a sluggish web service call. Fortunately, there are several options available in .NET to use a web service asynchronously. This article will examine a few of these techniques in building a responsive and well behaved application.

Introducing the Map Making Application

The sample application presented in this article allows a user to enter a start address and a destination address. The application then makes three calls to a web service to retrieve a map of the starting address (figure 1), a map of the destination address (figure 2), and the directions to move from start to finish (figure 3). The code for the sample web service is included in the download material. The web service uses Microsoft MapPoint to generate the maps and directions, but this idea is for demonstration purposes only. Please read the MapPoint licensing agreement before using MapPoint in a multi-user environment.

Figure 1

Figure 2

Figure 3

The code shown below is the first attempt at building the mapping application. In this sample, program control reaches the GetDirections method after the user has typed a start and end locations in a modal dialog box. The GetDirections method makes three calls to the mapping web service. Each of the first two calls return a JPG image as an array of bytes. It is relatively simple to translate the bytes back into a JPG image by placing the bytes into an instance of the MemoryStream class and passing the stream to the static FromStream method of the Image class.

protected void GetDirections(string start,
            string finish)
{
MapWS.MapService mapService = new
            MapWS.MapService();

byte [] result =
            mapService.GetMap(start);
MemoryStream memStream = new
            MemoryStream(result);
UpdateStartImage(Image.FromStream(memStream));

result
            = mapService.GetMap(finish);
memStream = new
            MemoryStream(result);
UpdateFinishImage(Image.FromStream(memStream));

string
            directions = mapService.GetDirections(start,
            finish);
UpdateDirections(directions);
}

The web service returns the directions as a string of HTML. To display the directions we use the Microsoft Web Browser control. This requires our project to reference both the Microsoft HTML Object library (mshtml.dll) and the Web Browser Control library (shdocvw.dll). With these references in place, the following code inserts the HTML directions into an instance of the web browser control.

protected void UpdateDirections(string newDirections)
{
MSHTML.IHTMLDocument2 htmlDoc =
(MSHTML.IHTMLDocument2)axWebBrowser.Document;
htmlDoc.body.innerHTML = newDirections;
}

To initialize the web browser control we place some code after the InitializeComponent method in the main form's constructor. The following code loads a blank document into the web browser for us to manipulate.

public MapClient()
{
//
// Required for Windows Form Designer support
//
InitializeComponent();

// navigate to a blank page so we have a valid axWebBrowser.Document
object url = "about:blank";

object flags = null, targetFrameName = null,
postData = null, headers = null;

axWebBrowser.Navigate2(ref (object)url,
ref flags,
ref targetFrameName,
ref postData,
ref headers);
}

Improving the Interface

The GetDirections method shown above works, however, the user experience needs room for improvement. Because our user interface thread makes all three calls to the web service, the application appears to freeze for 10 to 30 seconds as the method retrieves the JPG and HTML data. Many users will assume the application has locked up and might attempt to kill the process. In order to provide a more professional looking application, we need to call the web service asynchronously and leave the GUI responsive to the user. The .NET framework makes this easy by providing a proxy class with asynchronous capabilities for any web service. A proxy class sits between your code and the web service and facilitates communication, even adding additional capabilities.

Visual Studio .NET generates the web service proxy class when you select a web service from the Project -> Add Web Reference menu selection. Alternatively, you can run the web services description language tool (WSDL.EXE) from the command line for more flexibility in the code generation. You can read more about the WSDL.EXE tool on MSDN: http://msdn.microsoft.com/library/en-us/cptools/html/cpgrfwebservicesdescriptionlanguagetoolwsdlexe.asp?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq.

The proxy class contains three methods for each method in the web service. As shown in the object browser below, the proxy class contains GetMap, BeginGetMap, and EndGetMap. These three methods access the GetMap functionality of the web service. You might notice the methods closely mirror the functionality of a delegate's Invoke, BeginInvoke, and EndInvoke methods (see P.G. Sarang's article for more information on using delegates in C#: http://www.csharptoday.com/content.asp?id=1577&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq). Invoke is used to make a synchronous call, while the BeginInvoke and EndInvoke start and end an asynchronous call.

Figure 4

Begin and End

For the proxy class generated in our application, the GetMap method completes the web service call synchronously, while the BeginGetMap and EndGetMap methods start and finish an asynchronous call. Before we jump into using these two methods, take a look at an excerpt from the generated proxy class below:

public System.IAsyncResult BeginGetMap( string location,
System.AsyncCallback callback,
object asyncState) {

return this.BeginInvoke("GetMap", new object[] {
location}, callback, asyncState);
}

public System.Byte[] EndGetMap(System.IAsyncResult asyncResult) {

object[] results = this.EndInvoke(asyncResult);
return ((System.Byte[])(results[0]));

}

The BeginGetMap method has several responsibilities. First, this method needs to accept all of the in and ref parameters for the web service method. Since this method will return before the web service call is complete, the method does not return the image bytes for the completed map, but an object implementing the IAsyncResult interface. The IAsyncResult interface encapsulates the status of an asynchronous operation and we will examine the interface in more detail later. Underneath the covers, BeginGetMap places a work request into the runtime's thread pool. The actual call to the web service happens on a worker thread managed by the CLR. We will discuss the last two parameters to the BeginMapMethod in the next section when we build a method to use the web service asynchronously.

The only way to get the results of the asynchronous operation started by BeginGetMap is to call EndGetMap. The EndGetMap method contains the out and ref parameters as well as the return value for the web service call. You can use the following strategies for ending an asynchronous call:

  • Callback delegates
  • IAsyncResult WaitHandles
  • Blocking on end methods
  • Polling
  • Canceling

The following sections detail these different techniques.

Using Callback Delegates

The best match for our mapping application is to use a callback delegate to complete the web service call. A callback delegate is analogous to sending a request for processing off with a self- addressed stamp envelope. We can continue along our way and know that when the processing is finished, the person processing our request can return information to us and let us know when the job is complete. A callback delegate allows the thread processing the request call back to us to signal when the web service method is complete.

The following listing contains the new implementation of the GetDirections method:

protected void GetDirections(string start, string finish)
{
MapWS.MapService mapService = new MapWS.MapService();

mapService.BeginGetMap(start,
new AsyncCallback(StartImageCallback),
mapService);

mapService.BeginGetMap(finish,
new AsyncCallback(FinishImageCallback),
mapService);

mapService.BeginGetDirections(start, finish,
new AsyncCallback(DirectionsCallback),
mapService);
}

The new version of GetDirections allows a more responsive user interface since the BeginGetMap and BeginGetDirection return control to the calling thread before the web service call completes. When the worker thread executing the web service call reaches completion it will invoke the AsyncCallback delegate we pass into the Begin methods. The target of an AsynchCallback delegate is a method returning void and accepting an object implementing IAsyncResult as the only parameter. One programming guideline is to append "callback" to your method name to clearly identify methods used as the target of a callback delegate. The following code listing displays the three callback methods for the mapping application:

protected void StartImageCallback
            (IAsyncResult ar)
{
MapWS.MapService mapService =
            (MapWS.MapService)ar.AsyncState;
byte [] result =
            mapService.EndGetMap(ar);
MemoryStream memStream = new
            MemoryStream(result);
UpdateStartImage(Image.FromStream(memStream));
}

protected
            void FinishImageCallback(IAsyncResult ar)
{
MapWS.MapService
            mapService = (MapWS.MapService)ar.AsyncState;
byte [] result =
            mapService.EndGetMap(ar);
MemoryStream memStream = new
            MemoryStream(result);
UpdateFinishImage(Image.FromStream(memStream));
}

protected
            void DirectionsCallback(IAsyncResult ar)
{
MapWS.MapService
            mapService = (MapWS.MapService)ar.AsyncState;
string directions =
            mapService.EndGetDirections(ar);
UpdateDirections(directions);
}

Let us revisit the GetDirectons method for a moment and discuss the last parameter to the begin methods. This last parameter is an object reference known as asyncState. The framework does not use this object reference, but it does carry the reference along during the asynchronous operation and allows you to retrieve it inside of the callback method. You can use this parameter to pass along any context or required information needed by the callback method to complete the operation. For this application, we simply pass along a reference to the MapService object. We retrieve the MapService reference inside of the callback method using the AsyncState property of the IAsyncResult interface.

With a MapService and IAsyncResult reference in hand, it is easy for the callback methods to finish the asynchronous calls using the EndGetMap and EndGetDirection methods. However, there is some additional code needed to update the display. We can see these methods listed next:

protected delegate void UpdateImageDelegate(Image newImage);
protected delegate void UpdateDirectionsDelegate(string newDirections);

protected void UpdateStartImage(Image newImage)
{
if(this.InvokeRequired)
{
BeginInvoke(new UpdateImageDelegate(UpdateStartImage),
new object [] { newImage });
}
else
{
imgStart.Image = newImage;
}
}

protected void UpdateFinishImage(Image newImage)
{
if(this.InvokeRequired)
{
BeginInvoke(new UpdateImageDelegate(UpdateFinishImage),
new object [] { newImage });
}
else
{
imgFinish.Image = newImage;
}
}

protected void UpdateDirections(string newDirections)
{
if(InvokeRequired)
{
BeginInvoke(new UpdateDirectionsDelegate(UpdateDirections),
new object [] { newDirections });
}
else
{
MSHTML.IHTMLDocument2 htmlDoc =
(MSHTML.IHTMLDocument2)axWebBrowser.Document;
htmlDoc.body.innerHTML = newDirections;
}
}

It is important to remember when building Windows Forms applications how form controls are bound to a specific thread. This is not a limitation unique to the .NET framework, but has always been a part of Windows programming. Since we are now using worker threads to complete the web service calls, we need to ensure the worker threads are not the threads making updates to our user interface. One way to check is to use the InvokeRequired property of a control. If the property returns true, you are not on the thread bound to the control and should not attempt to manipulate the control. To move the execution onto the correct thread you can use the Invoke (synchronous) or BeginInvoke (asynchronous) methods of a Control derived object. For more information on threading issues in Windows Forms you can view the MSDN sample " Multithreaded Windows Forms Control Sample " at the following URL: http://msdn.microsoft.com/library/en-us/cpguidnf/html/cpcondevelopingmultithreadedwindowsformscontrol.asp?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq.

The callback delegates are a good solution for our forms based application. There is no need for us to explicitly create new threads to manage, and the runtime takes care of letting us know when the time is right to retrieve the results. However, other applications may find the IAsyncResult interface useful in implementing a different approach.

Using IAsyncResult

WaitHandles

Imagine we are already working on a background thread and we need to process the three web service calls for mapping. In this case we may still want to start the calls asynchronously, for example, to run them concurrently on a multiple processor machine. We would like to be able to wait until all three calls are complete and gather the results all from the same method. For this scenario, we make use of the AsyncWaitHandle property of IAsyncResult. The following listing demonstrates the method described above:

protected string start;
protected string
            finish;

protected void GetDirections(string start, string
            finish)
{
this.start = start;
this.finish =
            finish;
Thread worker = new Thread(new
            ThreadStart(GetDirectionsWorker));
worker.Start();
}

protected void
            GetDirectionsWorker()
{
MapWS.MapService mapService = new
            MapWS.MapService();
IAsyncResult arStart, arFinish,
            arDirections;

arStart = mapService.BeginGetMap(start, null,
            null);
arFinish = mapService.BeginGetMap(finish, null,
            null);
arDirections = mapService.BeginGetDirections(start,
            finish, null, null);

WaitHandle [] wh = {
arStart.AsyncWaitHandle,
arFinish.AsyncWaitHandle,
arDirections.AsyncWaitHandle
};

WaitHandle.WaitAll(wh);

byte [] result =
            mapService.EndGetMap(arStart);
MemoryStream memStream = new
            MemoryStream(result);
UpdateStartImage(Image.FromStream(memStream));

result
            = mapService.EndGetMap(arFinish);
memStream = new
            MemoryStream(result);
UpdateFinishImage(Image.FromStream(memStream));

string
            directions =
            mapService.EndGetDirections(arDirections);
UpdateDirections(directions);

}

In the above code we pass null values for the callback and asyncState parameters and save the return values of the begin methods. We use the WaitHandle of each IAsyncResult to build an array for the WaitHandle WaitAll method. As each web service call completes, the corresponding wait handle is set. The WaitAll method blocks the current thread until every wait handle in the array is set.

Blocking On End Methods

It is important to note that you do not need to wait for the web service calls to complete before calling the EndGetMap and EndGetDirection methods. For example, if you simply want to start the web service calls and execute some other calculations while the calls are running, you may go directly to the EndGetMap and EndGetDirection methods without waiting for a notification from the runtime. If you reach one of the end methods before the corresponding web service call is complete, the end method will simply block until the results are available. The following listing demonstrates the technique:

protected string start;
protected string
            finish;

protected void GetDirections(string start, string
            finish)
{
this.start = start;
this.finish =
            finish;
Thread worker = new Thread(new
            ThreadStart(GetDirectionsWorker));
worker.Start();
}

protected void
            GetDirectionsWorker()
{
MapWS.MapService mapService = new
            MapWS.MapService();

IAsyncResult arStart, arFinish,
            arDirections;
arStart = mapService.BeginGetMap(start, null,
            null);
arFinish = mapService.BeginGetMap(finish, null,
            null);
arDirections = mapService.BeginGetDirections(start,
            finish, null, null);

//
// Do some other processing while
            the web service calls
// happen asychronously ....
//
//
            If the calls are not complete by the time we start calling
//
            the EndGetMap and EndGetDirections, the calls will block
//
            until we are done.
//

byte [] result =
            mapService.EndGetMap(arStart);
MemoryStream memStream = new
            MemoryStream(result);
UpdateStartImage(Image.FromStream(memStream));

result
            = mapService.EndGetMap(arFinish);
memStream = new
            MemoryStream(result);
UpdateFinishImage(Image.FromStream(memStream));

string
            directions =
            mapService.EndGetDirections(arDirections);
UpdateDirections(directions);
}

Polling

It is also possible to poll for the completion of a web service call via the IsCompleted property of the IAsycResult interface. As demonstrated in the following code, this technique may be useful if you need to send "keep alive" messages to the user or another computer as the calls run to completion.

protected string start;
protected string
            finish;

protected void GetDirections(string start, string
            finish)
{
this.start = start;
this.finish =
            finish;
Thread worker = new Thread(new
            ThreadStart(GetDirectionsWorker));
worker.Start();
}

protected void
            GetDirectionsWorker()
{
MapWS.MapService mapService = new
            MapWS.MapService();

IAsyncResult arStart, arFinish,
            arDirections;
arStart = mapService.BeginGetMap(start, null,
            null);
arFinish = mapService.BeginGetMap(finish, null,
            null);
arDirections = mapService.BeginGetDirections(start,
            finish, null, null);

while( !arStart.IsCompleted &&
!arFinish.IsCompleted &&
!arDirections.IsCompleted)
{
// Perform some calculations
            here, perhaps sending some
// messages for animation to let the
            user know we are
// still
            processing..
Thread.Sleep(100);
}

byte [] result =
            mapService.EndGetMap(arStart);
MemoryStream memStream = new
            MemoryStream(result);
UpdateStartImage(Image.FromStream(memStream));

result
            = mapService.EndGetMap(arFinish);
memStream = new
            MemoryStream(result);
UpdateFinishImage(Image.FromStream(memStream));

string
            directions =
            mapService.EndGetDirections(arDirections);
UpdateDirections(directions);
}

Canceling

Finally, it is possible to cancel an asynchronous call. The object implementing the IAsyncResult interface is actually an instance of a WebClientAsyncResult class from the System.Web.Services.Protocols namespace. Once the Abort method is used, it is no longer possible to retrieve the results. In our example below, the call to EndGetDirections will throw a System.Net.WebException since we have already aborted the operation.

protected string start;
protected string
            finish;

protected void GetDirections(string start, string
            finish)
{
this.start = start;
this.finish =
            finish;
Thread worker = new Thread(new
            ThreadStart(GetDirectionsWorker));
worker.Start();
}

protected void
            GetDirectionsWorker()
{
MapWS.MapService mapService = new
            MapWS.MapService();

IAsyncResult arStart, arFinish,
            arDirections;
arStart = mapService.BeginGetMap(start, null,
            null);
arFinish = mapService.BeginGetMap(finish, null,
            null);
arDirections = mapService.BeginGetDirections(start,
            finish, null, null);

// abort the directions call
            ***
((WebClientAsyncResult)arDirections).Abort();

byte []
            result = mapService.EndGetMap(arStart);
MemoryStream memStream =
            new
            MemoryStream(result);
UpdateStartImage(Image.FromStream(memStream));

result
            = mapService.EndGetMap(arFinish);
memStream = new
            MemoryStream(result);
UpdateFinishImage(Image.FromStream(memStream));

//
            This call to EndGetDirections will result in an Exception
//
            since the call was already cancelled!
string directions =
            mapService.EndGetDirections(arDirections);
UpdateDirections(directions);
}

Exceptions in Asynchronous Programming

If you are worried about missing exceptions with the runtime performing all of the background work, rest assured the framework does not let you down. If an exception occurs during a begin method, the asynchronous call has failed to start and there are no results to gather from the end method. If the worker thread generates an exception during asynchronous processing, the runtime saves the exception and re-throws when you call the end method. It is also important to note that if the thread starting an asynchronous web service call terminates before the call finishes, the runtime throws an exception. If you face this situation you can use the IAsyncResult WaitHandle to block the calling thread until the web service call completes.

Conclusions

By taking advantage of the built in infrastructure of the CLR you can avoid building your own thread pooling and thread management features for asynchronous programming. The .NET framework provides consistent and flexible patterns to accomplish asynchronous programming tasks with web services. You can poll for completion, wait for completion, or allow the runtime to notify you of completion. The pattern you choose to use depends on the needs of your specific application. You can apply these same patterns to other areas of the .NET framework, including socket communications, file I/O, and any method encapsulated by a delegate.

Article Information
Author | Scott Allen
Technical Editor | Adam Ryland
Project Manager | Helen Cuthill
Reviewers | Sean Schade, Stephen Trudel

If you have any questions or comments about this article, please contact the technical editor.

C#Today - © 2002 WroxPress Limited


C# Browser History Monitoring Application (Part 2)

C# Browser History Monitoring Application (Part 2)
http://www.csharptoday.com/content.asp?id=1713
Article Rating: 4
Published on December 7, 2001
Abstract
In this article, Jan Narkiewicz shows us how to develop a Winlogon Notification DLL. At present, SWPWinLogonPkg.dll starts and stops the service on logon and logout respectively. Administrators might feel even more reassured if this DLL installed the monitoring service on logon for only certain users. Similarly, the DLL would uninstall the service when a monitored user logged off the machine. The infrastructure presented in this article can easily be extended to provide any requested checks and balances in order to prevent the browser history monitoring application from being used inappropriately.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

In the first installment of this article (Practical .NET - A Browser History Monitor, Part I) a service was developed (SWPWebWatcherService) that monitored a user's browser history file, and when the browser history was changed, this change was mirrored in an alternate location. The only flavor of change that was not mirrored was deletes. The rationale behind this was so as to determine a user's surfing habits.

In this installment, a Winlogon Notification DLL (SWPWinLogonPkg.dll) will be developed. When a user logs in, the service SWPWebWatcherService will be started, and when a user logs off this service will be stopped. The reason WinLogon DLL is provided is ultimately to place a restriction on the browser history monitoring service. At present, SWPWinLogonPkg.dll starts and stops the service on logon and logout respectively. Administrators might feel even more reassured if this DLL installed the monitoring service on logon for only certain users. Similarly, the DLL would uninstall the service when a monitored user logged off the machine. The infrastructure presented in this article could easily be extended to provide any requested checks and balances in order to prevent the browser history monitoring application from being used inappropriately. Adding install and uninstall functionality to SWPWinLogonPkg.dll might prove to be a good exercise for a reader seeking additional insights into the behavior of WinLogon Notification DLL's and services.

Readers might also find adding install and uninstall features to SWPWinLogonPkg.dll to be a good review of C++. Yes, this DLL was implemented in C++ rather than C#. The reason for this is because WinLogon Notification DLL's expose functions with a specific signature. These functions are called by the operating system when a variety of logon events are raised (logon, logout, workstation lock, workstation unlock, etc.). A managed language such as C# cannot expose functions, hence an unmanaged language had to be selected. Visual Studio .NET supports managed and unmanaged development using C++. Visual Studio .NET was hence used to develop SWPWinLogonPkg.dll as a C++ project, namely an unmanaged Win32 DLL.

Winlogon Notifications

For the Browser History Monitoring application, SWPWebWatcherService, it is desirable to know when a user logged on and logged out of the system. The specific Winlogon Notifications that are generated on logon and logged out are:

  • Logoff - this event is raised when a user logs off the system.
  • Logon - this event is raised when a user logs onto the system. When the Logon event is received, it is important to recognize that the user is not presently logged onto the network. It is impossible to perform network operations when handling the Logon event. If network access is required, use the event generated when the shell starts (StartShell).

These notifications are setup by creating an unmanaged DLL, SWPWinLogonPkg.dll, that exposes a specific set of functions: SWPWLEventLogon (called when the user logon notification is received), and SWPWLEventLogoff (called when the user logoff notification is received). The prototypes for these functions are as follows (as specified in MSDN, http://msdn.microsoft.com/library/default.asp?url=/library/en-us/security/hh/logauth/notify_29yd.asp&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq):

VOID <FunctionName> (WLX_NOTIFICATION_INFO *pInfo);

The Winwlx.h header file is where the WLX_NOTIFICATION_INFO structure, used in the previous function prototypes, is defined. This structure contains logon-related information that can be used by each notification-handling function. One data member of WLX_NOTIFICATION_INFO is UserName. Recall that the SWPWebWatcherService uses the username in order to determine if the browser history monitoring should be initiated or not. It would have been possible in the WinLogon Notification DLL, SWPWinLogonPkg.dll, to validate the username given that it is a data member of the WLX_NOTIFICATION_INFO structure. The validation code was placed in the service in order to expedite the logon and logoff process. The idea is not to slowdown the logon and logoff notification code in SWPWinLogonPkg.dll otherwise logon and logoff will be subsequently slowed down. For example, a network access within the logon or logoff handler of the SWPWinLogonPkg.dll DLL might prove costly from a performance perspective.

The managed C++ DLL was created using Visual Studio .NET (File, New, Project, Visual C++ Project, Win32 Project) and specifying a project name of SWPWinLogonPkg. This sequence of commands ultimately displays the Win32 Application Wizard dialog. Once this dialog is displayed by Visual Studio .NET, the Application Setting should be set to specify DLL.

The first step in developing the DLL is to specify function prototypes for the logon and logoff handler (functions SWPWLEventLogon and SWPWLEventLogoff respectively). The form of the prototype was already discussed. The prototypes are created in the SWPWinLogonPkg.h header file. These prototypes require that the WLX_NOTIFICATION_INFO structure is defined, so SWPWinLogonPkg.h must include the Winwlx.h header file. In the SWPWinLogonPkg.h header file the __declspec(dllexport) keyword is used to cause these notification-handling functions to be exported by the DLL. An __declspec(dllimport) is specified so that a calling process will treat these functions as being imported from a DLL. The C++ construct, extern "C", is used to specify that the header file should compile the functions as C. This avoids having the C++ compiler altering (mangling) the function name. For those not versed in the art of C++ development, name mangling is how C++ compilers handing multiple functions/methods with the same name but different parameters. C does not support overload functions/methods, hence extern "C" disables name mangling.

The SWPWinLogonPkg.h header file was created using Visual Studio .NET's File menu's New menu item. The File option is selected from the sub-menu displayed by New. The Visual C++ project category should be selected from within the New File dialog displayed. From within the file types displayed, Header File (.h) should be selected. The SWPWinLogonPkg.h header file is as follows:

#include "Winwlx.h"

// The following ifdef block is the standard way of creating macros which
// make exporting from a DLL simpler. All files within this DLL are compiled
// with the WLOGONPKG_EXPORTS symbol defined on the command line. This
// symbol should not be defined on any project that uses this DLL. This way
// any other project whose source files include this file see
// WLOGONPKG_API functions as being imported from a DLL, whereas this DLL
// sees symbols defined with this macro as being exported.
#ifdef WLOGONPKG_EXPORTS
#define WLOGONPKG_API __declspec(dllexport)
#else
#define WLOGONPKG_API __declspec(dllimport)
#endif

extern "C"
{
WLOGONPKG_API void WLEventLogon(WLX_NOTIFICATION_INFO *pInfo);
WLOGONPKG_API void WLEventLogoff(WLX_NOTIFICATION_INFO *pInfo);
}

The source file for the SWPWinLogonPkg DLL is named SWPWinLogonPkg.cpp. At the top of this file is the necessary snippet of code that causes the functions to be exported from the DLL, by ensuring the __declspec(dllexport) form of the function is compiled (specifically, that __declspec(dllexport) is specified when the WLOGONPKG_EXPORTS macro is defined):

#define WLOGONPKG_EXPORTS
#include "stdafx.h"

The SWPWLEventLogon and SWPWLEventLogoff functions implemented in this source use standard Win32 Service Control Manager functions in order to start (on logon) and stop (on logout) the SWPWebWatcherService service. The Win32 functions used by the logon notification-handling function, SWPWLEventLogon, include the following (listed in the order in which they appear in the function):

  • OpenSCManager - opens a handle to the service control manager (the manager for all services).
  • OpenService - using the service control manager's handle, opens a handle to a specific service. When the service is opened using OpenService, the SERVICE_START access should be specified since the service is to be started.
  • StartService - using a specific service's handle, starts said service.
  • CloseServiceHandle - closes the handle to a specific service.

The code associated with the SWPWLEventLogon and SWPWLEventLogoff functions uses message boxes to display errors. Clearly a better scheme should be adopted, but for now it suffices to show any issues that arise. The code associated with the SWPWLEventLogon function is as follows:

void SWPWLEventLogon(WLX_NOTIFICATION_INFO *pInfo)
{
SC_HANDLE hSCManager = ::OpenSCManager(NULL,
NULL,
SC_MANAGER_CONNECT);
if (hSCManager != NULL)
{
SC_HANDLE hService = ::OpenService(hSCManager,
"SWPWebWatcher",
SERVICE_START);
if (hService != NULL)
{
if (!::StartService(hService, NULL, NULL))
{
::MessageBox(NULL,
"StartService fail in SWPWLEventLogon",
"SWPWinLogonPkg.dll error",
MB_SERVICE_NOTIFICATION | MB_OK);
}
::CloseServiceHandle(hService);
return ;
}

else
{
::MessageBox(NULL,
"OpenService fail in WLEventLogon",
"SWPWinLogonPkg.dll error",
MB_SERVICE_NOTIFICATION | MB_OK);
}

::CloseServiceHandle(hSCManager);
}

else
{
::MessageBox(NULL,
"OpenSCManager fail in WLEventLogon",
"SWPWinLogonPkg.dll error",
MB_SERVICE_NOTIFICATION | MB_OK);
}
}

The Win32 functions used by the logon notification-handling function, SWPWLEventLogoff, include:

  • OpenSCManager - see the documentation above.
  • OpenService - using the service control manager's handle, opens a handle to a specific service. When the service is opened using OpenService, the SERVICE_STOP access should be specified since the service is to be stopped.
  • ControlService - using a specific service's handle to perform an action against the service (SERVICE_CONTROL_STOP).
  • CloseServiceHandle - see the documentation above.
  • CloseServiceHandle - see the documentation above.

The code associated with the SWPWLEventLogoff function is as follows:

void SWPWLEventLogoff(WLX_NOTIFICATION_INFO *pInfo)

{
SERVICE_STATUS sStatus;
SC_HANDLE hSCManager = ::OpenSCManager(NULL, NULL, SC_MANAGER_CONNECT);

if (hSCManager != NULL)
{
SC_HANDLE hService = ::OpenService(hSCManager,
"SWPWebWatcher",
SERVICE_STOP);
if (hService != NULL)
{
if (!::ControlService(hService,
SERVICE_CONTROL_STOP,
&sStatus))
{
::MessageBox(NULL,
"ControlService fail in WLEventLogoff",
"SWPWinLogonPkg.dll error",
MB_SERVICE_NOTIFICATION | MB_OK);
}

::CloseServiceHandle(hService);
return ;
}

else
{
::MessageBox(NULL,
"OpenService fail in WLEventLogoff",
"SWPWinLogonPkg.dll error",
MB_SERVICE_NOTIFICATION | MB_OK);
}

::CloseServiceHandle(hSCManager);
}
else
{
::MessageBox(NULL,
"OpenSCManager fail in WLEventLogoff",
"SWPWinLogonPkg.dll error",
MB_SERVICE_NOTIFICATION | MB_OK);
}
}

A separate project was developed in order to test SWPWinLogonPkg.dll. The code that ultimately tests the DLL is a console application, SWPWinLogonPkgTest.exe. The SWPWinLogonPkgTest.exe console application is written as an unmanaged C++ application since it is calling an unmanaged DLL. It would have been possible to write a managed application in C# to call the function's SWPWinLogonPkg.dll. Given that SWPWinLogonPkgTest is three lines of code, it was not worth the setup of developing a C# application to call the unmanaged code. The code contained in the SWPWinLogonPkgTest test application is as follows:

int _tmain(int argc, _TCHAR*
            argv[])
{
WLX_NOTIFICATION_INFO
            info;

::ZeroMemory(&info, sizeof
            info);
SWPWLEventLogon(&info);
SWPWLEventLogoff(&info);

return
            0;
}

Winlogon Installation Registry Settings

How does Windows know to associate the SWPWinLogonPkg DLL and its SWPWLEventLogon function with logon notifications and its SWPWLEventLogoff function with logout notifications? It uses the registry. Specifically, a sub-key corresponding to the notification DLL must be created under:

HKEY_LOCAL_MACHINE\Software\Microsoft\Windows NT\CurrentVersion\Winlogon\Notify

So under the previous registry path, an SWPWLEventLogon sub-key is created. The registry values under this key are what describe the DLL, its notification functions, and a plethora of other properties with respect to the notification DLL's behavior. The critical registry values specified are:

  • Asynchronous (type, REG_DWORD) - setting this value to 1 specifies that the DLL can handle WinLogon notifications asynchronously. When this value is set to 1, the notification-handling functions are called in a separate thread, otherwise they are called in the same thread as the calling entity.
  • DllName (type, REG_EXPAND_SZ) - specifies the DLL (name and path) that contains the notification-handling functions.
  • Impersonate (type, REG_DWORD) - specifies whether the security context under which the notification-handling functions are called are the systems security context (Impersonate not equal to 1), or the security context of the user logging on (Impersonate set to 1).
  • Logoff (type, REG_SZ) - specifies the name of the function that is called by the logoff event (e.g. SWPWLEventLogoff).
  • Logon (type, REG_SZ) - specifies the name of the function that is called by the logon event (e.g. SWPWLEventLogon).

A complete list of the registry values that can be specified under the Notify key can be found at:

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/security/hh/logauth/notify_29yd.asp&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq

The SWPWLogonPkg.reg registry file is provided to setup the keys necessary to register the SWPWinLogonPkg DLL. This file was created using the RegEdit.exe utility, and first typing in the desired registry entries by hand. Once the entries were entered in the registry, the root key (.\Notify\SWPLogonPkg) was selected from RegEdit's File menu, and the Export Registry File menu item was selected. The contents of the registry file, SWPLogonPkg.Reg, are as follows:

Windows Registry Editor Version 5.00

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Winlogon\Notify\SWPWLogonPkg]
"Asynchronous"=dword:00000000
"DllName"="D:\ASPToday\ASPTodayWebWatcher\WebWatcher\bin\Debug\SWPWLogonPkg.dll"
"Logon"="WLEventLogon"
"Logoff"="WLEventLogoff"
"Impersonate"=dword:00000000

The DLL path is specified in the SWPWLogonPkg.reg registry file next to the DllName value name. In order to use the registry file, this path has to be changed to reflect where a given administrator has placed the SWPWinLogonPkg DLL.

Other Winlogon Notifications

Logon and logout are not the only flavors of Winlogon notifications generated. The other type of notifications generated include notifications that convey the state of the screen saver:

  • StartScreenSaver - this notification is raised when the screen saver is initiated, but before it is actually displayed. This function is specified in the registry under the StartScreenSaver value name.
  • StopScreenSaver - this notification is raised when the screen saver has stopped. This function is specified in the registry under the StopScreenSaver value name.

Notifications raised include notifications that pertain to whether or not a particular system has been locked by a user, or subsequently unlocked:

  • Lock - this notification is raised when the user "locks" the workstation. A locked workstation still has the user logged in, but cannot be accessed without the user passwords, unless this is overridden by an administrator. This function is specified in the registry under the Lock value name.
  • Unlock - This event occurs when the user unlocks the workstation, or when a system administrator overrides the lock and logs the user off. This function is specified in the registry under the Unlock value name.

Notifications are raised when the system starts or shutdowns, and when the system's shell is loaded:

  • Shutdown - this notification is raised when the system is shutting down. This function is specified in the registry under the Shutdown value name.
  • StartShell - this notification is raised after a logged on user has specified a shell. The "desktop" itself in Windows is a shell (Explorere.exe). When the windows shell is displayed, the StartShell notification is raised. When this notification is raised, not only is the user logged on, but the notification handler can access the network. The feat was impossible to perform when handling the Logon notification. This function is specified in the registry under the StartShell value name.
  • Startup - this notification is raised at system start up. A system can be started up when it is powered on initially or after a restart has been initiated. This was was impossible to perform when handling the Logon notification. This function is specified in the registry under the Startup value name.

Performing SCM in .NET - An Alternate Architecture

Consideration was given to having the Winlogon Notifications DLL call a .NET class library. The unmanaged DLL would receive the notification, and the managed class library would use the class in order to act as a Service Control Manger for the application. The assembly that implements the aforementioned class library would be SWPWebServiceControlManger. The basic layout for this scheme is as follows:

The code originally in SWPWInLoginPkg.dll that started and stopped the service amounted to approximately a dozen lines of code. Although managed code is preferable, it was not worth the effort to deploy managed SCM, because in order to achieve this, the following extra steps would need to be taken:

  • Create a class library, SWPWebServiceControlManger
  • Implement a service control manager (start/stop) in the class library
  • Run the .NET utility, TlbExp.exe, in order to extract a type library for the SWPWebServiceControlManger assembly.
  • Convert the unmanaged DLL, SWPWInLoginPkg, to use COM calls (CoInitialize, CoCreateInstance, etc.) in order to call the managed class exposed by the class library, SWPWebServiceControlManger.

Compared to the previously reviewed rigmarole, it was vastly simpler to implement the handling of Logon and Logout events using purely Win32 in an unmanaged C++ DLL.

Viewing the Mirrored History

When Windows Explorer is used to view a user's history folder, the history folder looks something like the following:

When the mirrored version of the history folder is viewed using Windows Explorer, the view is radically different:

In reality, the directory structure for both the history and the mirrored version of the history are identical. The history file has been setup with a Windows shell extension that provides a viewer with respect to the files and directories contained in the history folder. This accounts for the elegant appearance when the history folder is viewed using Windows Explorer. Could the history folder shell extension be made available to the mirrored history folder? Yes! This would involve calling a certain number of legacy COM interfaces and tweaking the registry accordingly. Those interested in setting this up should see Dino Esposito's "Visual C++ Windows Shell Programming," Wrox Press, 1998. For an overview of this text, see the following: http://www.wrox.com/Books/Book_Details.asp?ISBN=1861001843&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq

Notice that the pertinent technologies are COM and the registry - legacy windows technology. Again, here is the SWPWebWatcherService application, a perfectly fine C#/.NET application, and still it relies on legacy technology. Rather than reviewing shell programming using legacy Windows development techniques, a simpler solution is available for viewing the contents of a mirrored history folder. The solution is as follows:

  • Create a dummy user
  • Copy the contents of the mirrored history folder to the history folder of the dummy user
  • View the dummy user's history file in Windows explorer since it is now easily readable.

Clearly setting up a dummy user is a bit wasteful with respect to system resources, but it is a quick fix. The point that we keep reiterating in this discussion of how the shell behaves is, yes, .NET is here, but development is still intertwined with the legacy world (the registry and the Windows shell).

Please see the document 'Setup for browser history monitor.doc' (in the support material) for help with setting up this project, as it includes some changes to the registry that are needed.

Conclusion

The whole point of the exercise in monitoring a user's browser history was to demonstrate how .NET simplifies development with respect to services (the standard Visual Studio .NET services project). The simplification included the development of the server code (starting, stopping, etc.) using the ServiceController class, and installing the service using the ServiceInstaller and ServiceProcessInstaller classes. Also shown as part of the SWPWebWatcherService service was the use of .NET in order to access the contents of the Windows registry (the Microsoft.Win32 namespace and the RegistryKey class). The SWPWebWatcherService service also used the System.IO 's DirectoryInfo file, and the FileSystemWatcher classes.

Also demonstrated was that Visual Studio .NET is still more than capable of developing unmanaged code using C++. At times such code is required, as was demonstrated when handling Windows Logon Notifications. This DLL was specified using registry settings that ultimately ensured the specified DLL was loaded (SWPWInLoginPkg.dll), and that specific functions were called on logon (SWPWLEventLogoff) and logoff (SWPWLEventLogoff). These functions were used to start/stop the SWPWebWatcherService service.

The lesson to be learned by this unmanaged C++ endeavor is that even in a world gone .NET and C# crazy, there are still times when it is necessary to revert back to writing unmanaged code. Those who doubt this should take a look at Chapter 4 of Wrox's, "Professional .NET Programming", ISBN 1861005563. In this chapter, a .NET Runtime Host is developed in C++ using Visual Studio .NET. A Runtime Host is an unmanaged application used to load managed code so that it runs within .NET. Most .NET applications are executed using the Runtime Hosts that are deployed with the .NET Framework. Developers seeking an additional layer of customization and performance have no choice but to write unmanaged code in order to implement a Runtime Host of their very own.

Unmanaged code interfacing with C# and every .NET language will be a theme for years to come. As has been demonstrated, this interaction between C# and the unmanaged world is not strictly bounded at the DLL or COM interface level.

If you'd like to discuss a broader range of C#Today topics and issues, or if you have any C# related questions you need answers to, please visit our online resource centres at : http://p2p.wrox.com/list.asp?list=csharptoday_discuss&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq and http://p2p.wrox.com/list.asp?list=c_sharp&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq
C#Today - © 2002 WroxPress Limited


Transactions In Web Services

Transactions In Web Services
http://www.csharptoday.com/content.asp?id=1701
Article Rating: 4.5
Published on November 20, 2001
Abstract
The ASP.NET Framework enables transactional processing in classes, Web Forms and Web Services. We can use transactions to ensure an entire set of functionality successfully completes, or all data changes are rolled back if a portion of the functionality fails. In this article, Doug Seven looks at how we can implement transactional processing in a Web Service. The syntax for enabling transactions in a Web Service is different from the syntax for doing the same in either a class or a Web Form.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

The ASP.NET Framework enables transactional processing in classes, Web Forms and Web Services. We can use transactions to ensure an entire set of functionality successfully completes, or all data changes are rolled back if a portion of the functionality fails. In many cases, transactions can help us to ensure the results we expect are achieved. A good example of this is in financial processing, such as when a series of financial transactions must all complete to achieve the result we expect, such as the classic example of a debit and a credit. If one of the two functions fails, then neither should be executed.

In this article, we will look at how we can implement transactional processing in a Web Service. The syntax for enabling transactions in a Web Service is different from the syntax for doing the same in either a class or a Web Form.

For the code in this article you will need:

  • .NET Framework Beta 2
  • Windows 2000 with IIS 5.0 and Component Services
  • Internet Explorer 6.0
  • SQL Server 7.0 or 2000
  • A text editor, such as Notepad, TextPad, or Visual Studio.NET Beta 2

Understanding Transactions

Transactions work on a pass or fail system - commit or abort. A group of functions are handled together as one unit. The transaction manager, Windows Component Services, handles the entire set of functionality, including tracking both the original and new values of any data being changed. Component Services does this in such a way that even if there is a machine restart, the data is persisted, and any changes can be rolled back (set back to their original values) without any interaction with the user. Figure 1 shows how a transaction works.

Figure 1

In the transaction, each method is invoked, and a success or failure vote is made (commit or abort). If any one method votes to abort, then the entire transaction is aborted, and any changes are rolled back to their original values.

In the pre-.NET days, to build a transactional component required not only writing the code, but also creating a transaction package in Component Services. This meant, on any machine handling the transaction, opening the Component Services management console (MMC) and going through the wizard to add a new package to the COM+ Applications node.

Figure 2

The .NET Framework makes building transactional components, including Web Services, very easy to do. In fact, we don't even have to do anything in the COM+ management console - it can all be done automatically. For Web Services, the .NET Framework enables "Services without Components," which enables us to use Enterprise Services (formerly COM+) to manage transactions and pooling without creating a COM+ package. All of the work of managing the state of the transaction, and the data that is being changed, is done behind the scenes.

The Sample Application Object Model

For this sample application, we are going to build a Web Service that exposes a method for transferring money from one bank account to another. In the sample, we will be passing bank account numbers across the Internet, in SOAP messages, so in a real application we would want to use SSL encryption - but that's not necessary for this tutorial.

The application has four tiers: a data access tier (AccountDB); a business layer (Account); a Web Service layer (FinancialUtil); and a user interface tier (FinancialUtilClient). In a Web Service application, the proxy client is typically on another server, and likely an application belonging to a different company. For this sample, we will have all four tiers in one application, but the concepts are the same - we will use the proxy as if it were on another server, in a separate application. Figure 3 shows the application object model.

Figure 3

The ASPToday.Data.AccountDB class is our data access class. This class exposes the methods we need to retrieve and update the data:

  • getAllAccountNums() : returns a DataSet with a single DataTable containing the account numbers and the names of the account owners.
  • getAccountInfo(String _acctNum) : takes in the account number and returns an SqlDataReader with the account data for a single account.
  • getBalance(Int32 _id) : takes in the account ID value and returns the account balance as a Decimal data type.
  • debitOrCreditAccount( Boolean _credit, Int32 _id, Decimal _amount) : takes in a boolean value specifying if this is a credit or a debit, an account ID, and the dollar amount to either debit or credit.

The ASPToday.WebServiceTrans.Account class is the business tier component. An instance of this class represents a single account. There is a default constructor, and an overloaded constructor. The overloaded constructor has a single parameter, the account number. If the overloaded constructor is used, the object populates its properties with the account data.

  • Properties:
    • ID : a 32-bit integer that is the primary key value from the database.
    • AccountNumber : a string that is the account number.
    • Name : a string that is the account owner's name.
    • Balance : a decimal that is the account balance. The property setter will throw an "Insufficient Funds" exception if there is an attempt to set the value below 0.
  • Methods:
    • getAccountInfo() : uses the AccountNumber value to invoke AccountDB.getAccountInfo(), and populates the properties with the data returned.

The ASPToday.WebServiceTrans.FinancialUtil class is our Web Service. This Web Service class interacts with the AccountDB component, and the Account component:

  • GetAllAccountNumbers() : this Web method invokes AccoutDB.getAllAccountNums() and returns the DataSet to the caller.
  • TransferMoney(Decimal _amount, String _fromActNum , String _toActNum) : this is the Web method that will be managed by a transaction. It creates two instances of the Account class, one for the _fromActNum and one for the _toActNum, and invokes AccountDB.debitOrCreditAccount() on each of them. If one of the methods fails, then the transaction will abort, and any changes will be rolled back.

The ASPToday.ProxyClients.FinancialUtilClient is the proxy client. It exposes the same methods as FinancialUtil.

The application interacts with an SQL Server 2000 database. For simplicity, we will be putting all of our T-SQL commands in the AccountDB class, rather than using stored procedures. In a live production application we would want to use stored procedure for improved performance.

In SQL Server, create a new database named ASPToday. In the database, add one table, Accounts.

Column Name | Data Type | Notes
ID | Int | Primary key, Identity column, Seed = 1
Acctnum | Nvarchar | Size = 10
Name | Nvarchar | Size = 50
Balance | Money | Default value = 0

Figure 4

Add a couple of records to the table:

ID | Acctnum | Name | Balance
1 | 1234567890 | Doug Seven | 1000
2 | 0987654321 | Donny Mack | 1000

Building the Application

Now that we understand the design of the application, we can start building it. Let's start with the AccountDB class:

  1. Create a new C# Web application in Visual Studio .NET, named ASPToday.
  2. Create a folder named Data.
  3. Add a new class file named AccountDB.

In the Account DB class, add the following code:

using System;
using System.Data;
using
            System.Data.SqlClient;
using System.Configuration;
using
            System.Text;
using System.EnterpriseServices;

namespace
            ASPToday.Data
{
public class AccountDB
{
//Create a
            class-level SqlConnection object
private SqlConnection con =
new SqlConnection
            (
ConfigurationSettings.AppSettings["ASPTodayConString"]
            );

//*******************************************************//
//
            getAllAccountNums - Returns a DataSet with one
// DataTable. The
            DataTable contains two columns,
// acctnum (Account Number) and
            name (Account Owner
//
            Name.
//*******************************************************//
public
            DataSet getAllAccountNums()
{

SqlDataAdapter sda = new
            SqlDataAdapter (
"SELECT acctnum, name FROM Accounts",
con
            );
DataSet ds = new DataSet();
try
{
sda.Fill ( ds,
            "Accounts" );
return ds;
}
catch
{
return
            null;
}
}

//*******************************************************//
//
            getAccountInfo - Returns a SqlDataReader with a single
// row of
            data - the Account data for the specified
            account
//*******************************************************//
public
            SqlDataReader getAccountInfo ( String _acctNum )
{
SqlCommand
            cmd = new SqlCommand (
"SELECT id, acctnum, name, balance " +
"FROM Accounts WHERE acctnum ='" +
_acctNum + "'",
con
            );
try
{
con.Open();
return cmd.ExecuteReader (
CommandBehavior.CloseConnection );
}
catch
{
return
            null;
}
}

//*******************************************************//
//
            getBalance - Returns a Decimal data type value that
// is the
            Balance for the specified
            account.
//*******************************************************//
public
            Decimal getBalance ( Int32 _id )
{
SqlCommand cmd = new
            SqlCommand (
"SELECT balance FROM Accounts WHERE id = "
            +
_id,
con );
try
{
con.Open();
Decimal
            _newBalance = (Decimal)cmd.ExecuteScalar
            ();
con.Close();
return
            _newBalance;
}
catch
{
return
            -1;
}

}

//*******************************************************//
//
            debitOrCreditAccout - Deducts or adds money to the
// specified
            accout. The bool value is used to specify if
// the action should
            be a credit (true) or a debit
            (false).
//*******************************************************//
public
            void debitOrCreditAccount ( Boolean _credit,
Int32 _id,
Decimal _amount )
{
StringBuilder _sql = new
            StringBuilder();
_sql.Append ( "UPDATE Accounts SET balance =
            balance" );

//If this is a credit, add the amount to the
            balance
if ( _credit )
{
_sql.Append ( " + "
            );
}
//If this is a debit, subtract the amount from the
            balance
else
{
_sql.Append ( " - " );
}
_sql.Append (
            _amount.ToString() );
_sql.Append ( "WHERE id = "
            );
_sql.Append ( _id.ToString() );

SqlCommand cmd = new
            SqlCommand ( _sql.ToString(), con );
try
{
con.Open();
cmd.ExecuteNonQuery
            ();
}
finally
{
if ( con.State == ConnectionState.Open
            )
{
con.Close();
}
}
}
}
}

The AccountDB has a class-level SqlConnection object, which uses a connection string from the web.config configuration file. Open the web.config file in the root directory of the Web application, and add the following in the <configuration> element:

<appSettings>
<add key="ASPTodayConString"
value="server=localhost;database=ASPToday;Trusted_Connection=true;" />
</appSettings>

When the SqlConnection is created, the ConfigurationSettings class will pull the connection string from the configuration file.

All of the methods in the AccountDB class are fairly straightforward. They connect to the database and execute the specified T-SQL command.

Now, build the Account class:

  1. Create a new folder in the ASPToday application named WebServiceTrans.
  2. Add a new class file named Account.
  3. In the Account class, add the following code:

using System;

namespace
            ASPToday.WebServiceTrans
{
//*******************************************//
//
            Account class represents a single Account from the
//
            databse.
//*******************************************//
public
            class Account
{
//Define properties
public Int32
            ID;
public String AccountNumber;
public String
            Name;
//Define a Balance property
private Decimal
            _balance;
public Decimal Balance
{
get
{
return
            _balance;
}
set
{
//If the new Balance is less than
            0
//throw an exception - Insuficient funds
if ( value < 0
            )
{
throw ( new System.Exception("Insufficient
            funds."));
}
//Otherwise, set the
            balance
else
{
_balance =
            value;
}
}
}

//*******************************************//
//
            getAccountInfo - Sets the properties of the Account
// object
            based on the AccountNumber
            property.
//*******************************************//
public
            void getAccountInfo()
{
ASPToday.Data.AccountDB _actDB =
new ASPToday.Data.AccountDB();
//Use the IDataReader object
            to get the DataReader
//This enables use with any .NET Data
            Adapter
System.Data.IDataReader _reader =
_actDB.getAccountInfo ( this.AccountNumber );
//Populate the
            object properties
while ( _reader.Read() )
{
this.ID =
            (Int32) _reader["id"];
this.Name = (String)
            _reader["name"];
this.Balance = (Decimal) _reader["balance"]
            ;
}
_reader.Close();
}

//*******************************************//
//
            Default constructoer - all properties are
            null.
//*******************************************//
public
            Account()
{
}

//*******************************************//
//
            Overloaded constructor - populates the Account object
// based on
            the specified Account
            Number.
//*******************************************//
public
            Account ( String _actNum )
{
this.AccountNumber =
            _actNum;
getAccountInfo();
}
}
}

The Account class exposes two constructors, a default constructor, and an overloaded constructor. The overloaded constructor takes an account number as its only parameter, and invokes getAccountInfo(). The getAccountInfo() method invokes the AccountDB.getAccountInfo() method and populates the properties of the object. In the Balance property, we evaluate the value in the property setter. If the value is below zero then we throw a system exception with "Insufficient Funds" in the Exception.Message property.

Building the Web Service

The Web Service exposes two Web methods, GetAllAccountNumbers() and TransferMoney(). The TransferMoney() method is the only method that will use a transaction.

  1. In the WebServiceTrans folder, create a new Web Service named FinancialUtil.asmx
  2. In the code behind for the Web Service, add the following code:

using System;
using System.Collections;
using System.ComponentModel;
using System.Data;
using System.Diagnostics;
using System.Web;
using System.Web.Services;
using System.EnterpriseServices;

namespace ASPToday.WebServiceTrans
{
public class FinancialUtil : System.Web.Services.WebService
{
//Create a class-level instance of the AccountDB class
ASPToday.Data.AccountDB _actDB =
new ASPToday.Data.AccountDB();

For the Web Service we create a class-level AccountDB object. This will be used in both Web methods.

//*******************************************//
// GetAllAccountNumbers - Returns a DataSet with one
// DataTable. The DataTable contains two columns,
// acctnum (Account Number) and name (Account Owner
// Name.
//*******************************************//
[WebMethod()]
public DataSet GetAllAccountNumbers ()
{
return _actDB.getAllAccountNums();
}

We use the [WebMethod()] attribute to specify that this method should be exposed as a Web method - allowing it to be called from anywhere across the Internet. In the method we simply return the DataSet that is returned from AccoutnDB.getAllAccountBalances().

    //********************************************//
//
            TransferMoney - transfers an amount of money from
// one account
            to another. This WebMethod implements
// Transaction to enable
            data recovery if an
            exception
//occurs
//*******************************************//
[
            WebMethod() ]
public Decimal[] TransferMoney ( Decimal _amount,
String _fromActNum,
String _toActNum )
{
//Create
            debit and credit Account objects
Account _debit = new Account (
            _fromActNum );
Account _credit = new Account ( _toActNum
            );

//Create an array of balances to return to the
            caller
Decimal[] _array = new Decimal[2];

//Perform the
            credit and debit - the database gets updated
            here
_actDB.debitOrCreditAccount ( true, _credit.ID,
            _amount);
_actDB.debitOrCreditAccount ( false, _debit.ID, _amount
            );

try
{
//Update the balances:
//If an
            Account.Balance goes below 0,
//an exception is thrown by the
            Account object
_credit.Balance = _actDB.getBalance ( _credit.ID
            );
_debit.Balance = _actDB.getBalance ( _debit.ID
            );
}
//Catch the exception from the Account object
catch (
            Exception ex )
{
//Throwing an exception will cause the
            transaction to abort
//The database changes get rolled back here
            automatically
throw (new System.Exception ( ex.Message )
            );
);
}

//Return the new balances in the
            array
_array[0] = _debit.Balance;
_array[1] =
            _credit.Balance;

return _array;
}

The transfer money method creates two instances of the Account class, one to represent the debit account, and one to represent the credit account. The method returns an array of decimals. The array will have two objects in it: the first will be the debit account balance, and the second will be the credit account balance.

We invoke the Account.debitOrCredit() method for each Account object in a try / catch block. If either of them throws an exception, we catch it, and re-throw it. This is one of the critical points of how transactions work in Web Services. If the class that is transaction-enabled throws an exception, the transaction will automatically abort. The key here is that it must be the transaction-enabled class that throws the exception. If the exception was thrown in the Account object, and we did not re-throw it, the transaction would not abort, because the exception did not occur in the transaction-enabled object. This means that the credit account would get the money, but the debit account may not have the money deducted. By re-throwing the exception, we will be forcing the transaction to abort.

Of course, we have not added the transaction code yet, so let's test the application first. Click on the Project menu and choose Add Reference. Select System.EnterpriseServices and click OK. Build the application by clicking the Build menu, and choosing Build.

Right click on the FinancialUtil Web Service file in the Solution Explorer, and click Set as Start Page, then run the application.

Since each account has $1,000 in it (in the database), we can test the Web Service by transferring $800 from on account to the other. The first time we do this, the Web Service should execute, and transfer the money. The second time, the "Insufficient Funds" exception should be thrown.

Figure 5

After you have tested the Web Service twice (transferred $1,600), check the database. You will see that one account has -$600, and the other has $2,600. Reset the account balance to $1,000.

Implementing a Transaction

Implementing a transaction in a Web Service requires only two modifications to our existing code. In the [WebMethod()] attribute, we need to specify that this Web method requires a new transaction. This is done using the TransactionOption.RequiresNew enumeration:

[ WebMethod ( false, TransactionOption.RequiresNew) ]

The first argument is a boolean value specifying if the Web method should enable the Session state - we don't need it for this Web method, so it is set to false, which is also the default value, but there is no WebMethod() signature that takes only the TransactionOption attribute. The following table lists describes all of the TransactionOption enumerations.

TransactionOption | Description
TransactionOption.Disabled | Ignore any transaction in the current context.
TransactionOption.NotSupported | Create the component in a context with no governing transaction.
TransactionOption.Supported | Share a transaction if one exists; create a new transaction if necessary.
TransactionOption.Required | Create the component with a new transaction, regardless of the state of the current context.
TransactionOption.RequiresNew | Share a transaction if one exists.

Secondly, we use the [AutoComplete] attribute to specify that the transaction should complete unless an exception is thrown.

[ WebMethod ( false, TransactionOption.RequiresNew) ]
[ AutoComplete ]
public Decimal[] TransferMoney ( Decimal _amount,
String _fromActNum,
String _toActNum )

This is one of the primary differences between implementing a transaction in a class versus a Web Service. In a class, the transaction attributes are applied at the class level; in a Web Service the transaction attributes are applied at the Web method level. In a Web Service, only the Web methods that have the TransactionOption set will use transaction.

Of course, we could leave off the [AutoComplete] option, and handle committing or aborting transaction in code.

try
{
//Update the balances:
//If
            an Account.Balance goes below 0,
//an exception is thrown by the
            Account object
_credit.Balance = _actDB.getBalance ( _credit.ID
            );
_debit.Balance = _actDB.getBalance ( _debit.ID
            );
ContextUtil.SetCommit;
}
//Catch the exception from the
            Account object
catch ( Exception ex
            )
{
ContextUtil.SetAbort;
}

The System.EnterpriseServices.ContextUtil class is used to vote on a transaction. This code is only necessary if you want to manually commit or abort the transaction. Any exception that is raised in the transaction-enabled Web method will cause the transaction to abort, so using [AuotComplete] means we do not have to manually vote on the transaction. We are assuming it should commit, unless an exception is thrown in the Web method.

When the Web method is invoked, it is managed as a transaction. Another difference between a class transaction and a Web method transaction is how they use Component Services. The first time a transaction-enabled class is used, a new package is added to Component Service. With Web method transactions, the transaction is managed, but a package is never added to Component Services. This is due to the new feature of Enterprise Services, Services without Components. The transaction is managed without creating a component because it is a WebMethod() transaction, not a class transaction.

Rebuild the application. When we test the Web Service this time, try transferring more money than we are allowed to, and the Web method should throw an exception.

System.Web.HttpException: Exception of type System.Web.HttpException was thrown. ---> System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.Exception: Insufficient funds.

When the Web method throws the exception, data changes are rolled back. We can verify this in the database.

With this design, we have put the onus of catching the exception on the client. With the exception, we pass the error information to the client, and it becomes their responsibility to handle it appropriately.

Building the Proxy Client

To create a proxy client in Visual Studio .NET, we simply add a Web Reference to the Web Service URL. Behind the scenes this creates a proxy client class.

On the menu click Project and select Add Web Reference.

  • In the address text box, type the URL to the Web Service and press Enter.
  • The Web Service information page will be displayed.

Figure 6

  • Click Add Reference.

If you are not using Visual Studio .NET, you can create a proxy client by executing the following command at the command prompt:

wsdl.exe /out:C:\Inetpub\wwwroot\ASPToday\FinancialUtilProxy.cs /n:ASPToday.localhost

Figure 7

The /out: parameter specifies the output file that should be created. By default, the proxy class is written in C# (use the /l: argument to specify VB for Visual Basic .NET, or JS for Jscript .NET). The /n: parameter specifies the namespace for the proxy client. I chose ASPToday.localhost since that is what Visual Studio .NET will use. Visual Studio .NET will create the proxy in a namespace representing the domain name where the Web Service resides. For example, if the Web Service was at http://www.wrox.com/myservice.asmx?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq, then Visual Studio .NET would create the namespace ASPToday.com.wrox.

With the proxy client created, we can build a Web Form (or any other client application) that will interact with the Web Service.

  1. Create a new folder named ProxyClients
  2. Add a new Web Form named FinancialUtilClient.aspx
  3. Design a Web Form like the one shown here:

Figure 8

In the code behind class for the Web Form, we will invoke the GetAllAccountNumers() Web method to bind the two DropDownList server controls. When the Transfer Money LinkButton is clicked, we will invoke the TransferMoney() Web method.

In the FinancialUtilClient code behind, add the highlighted code:

using System;
using
            System.Collections;
using System.ComponentModel;
using
            System.Data;
using System.Drawing;
using System.Web;
using
            System.Web.SessionState;
using System.Web.UI;
using
            System.Web.UI.WebControls;
using
            System.Web.UI.HtmlControls;
using System.Text;

namespace
            ASPToday.ProxyClients
{
public class FinancialUtilClient :
            System.Web.UI.Page
{
protected System.Web.UI.WebControls.Label
            Results;
protected System.Web.UI.WebControls.DropDownList
            XferFrom;
protected System.Web.UI.WebControls.DropDownList
            XferTo;
protected System.Web.UI.WebControls.TextBox
            Amount;
protected System.Web.UI.WebControls.LinkButton
            XferButton;

//Define a page-level proxy client class
            object
private ASPToday.localhost.FinancialUtil _finUtil =
new ASPToday.localhost.FinancialUtil();

public
            FinancialUtilClient()
{
Page.Init += new
            System.EventHandler(Page_Init);
}

private void
            Page_Load(object sender, System.EventArgs e)
{
if (
            !Page.IsPostBack )
{
//Get the Account Numbers and bind
            them
//to the DropDownList controls
DataSet _ds = new
            DataSet();
_ds =
            _finUtil.GetAllAccountNumbers();
XferFrom.DataSource =
            _ds;
XferTo.DataSource =
            _ds;
Page.DataBind();
}
}

//LinkButton.Click Event
            Handler
private void XferButton_Click(object sender,
            System.EventArgs e)
{
//Capture the user specified
            values
Decimal _amount = Decimal.Parse ( Amount.Text.Trim()
            );
String _from = XferFrom.SelectedItem.Value;
String _to =
            XferTo.SelectedItem.Value;

try
{
//Invoke the Web
            Service
Decimal[] _balances = _finUtil.TransferMoney (
_amount, _from, _to );
//Build the Result
            text
StringBuilder _result = new
            StringBuilder();
_result.Append ( "<b>");
_result.Append
            ( XferFrom.SelectedItem.Text );
_result.Append ( "("
            );
_result.Append ( _from );
_result.Append ( ")"
            );
_result.Append ( " Balance: </b>" );
_result.Append (
            _balances[0].ToString() );
_result.Append ( "<br><b>"
            );
_result.Append ( XferTo.SelectedItem.Text );
_result.Append
            ( "(" );
_result.Append ( _to );
_result.Append ( ")"
            );
_result.Append ( " Balance: </b>" );
_result.Append (
            _balances[1].ToString() );
//Set the Result Label's Text property
            to the string
Results.Text = _result.ToString();
}
catch (
            Exception ex )
{
//Render the exception message in the Result
            Label
Results.Text = ex.Message;
}
}

private void
            Page_Init(object sender, EventArgs
            e)
{
InitializeComponent();
}

#region Web Form
            Designer generated code
/// <summary>
/// Required
            method for Designer support - do not modify
/// the contents of
            this method with the code editor.
/// </summary>
private
            void InitializeComponent()
{
this.XferButton.Click += new
System.EventHandler(this.XferButton_Click);
this.Load += new
            System.EventHandler(this.Page_Load);

}
#endregion

}
}

In the XferButton_Click(), if the transaction completed successfully, we will render the new account balances to the page.

Figure 9

In the event handler, if the transaction aborts, we catch the exception and render the Exception.Message property to the client.

Figure 10

Of course, for a real-world application, we would want to clean this up into a user-friendlier message.

Summary

Implementing transactions in an ASP.NET Web Service does not require a lot of effort. In the example, we added two lines of code to a Web Service and the Web Service was transaction-enabled. Transactions allow us to group a set of functions together and either commit or abort them all as one unit. Unlike previous application development, the .NET Framework handles all of the transaction registering and management for us, so all we need to do is add code to transaction-enable a Web method.

C#Today - © 2002 WroxPress Limited


Accessing Web Services from Windows Client

Accessing Web Services from Windows Client
http://www.csharptoday.com/content.asp?id=1692
Article Rating: 4
Published on November 8, 2001
Abstract
Over the last several years, we have seen the Internet evolve from delivering static and simple HTML pages, into using dynamic, programmable and reusable web components. Microsoft's .NET framework introduces Web Services and SOAP that take the Internet a step further, by allowing the web applications to be more distributable, reusable and resourceful. In this article, Ravi Venkatarajalu look at how to combine the features of Web Services, SOAP and the .NET framework with Microsoft Excel 2000. We will also see how to serialize a Dataset and send it as part of a HTTP Request to a Web Service.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

Over the last several years, we have seen the Internet evolve from delivering static and simple HTML pages, into using dynamic, programmable and reusable web components. Microsoft's .NET framework, Web Services, and SOAP take the Internet a step further, by allowing web applications to be more distributable, reusable and resourceful.

Web Services allow developers to create software components that can be reused and integrated regardless of geographic location, using standard web protocols. Web Services are similar to DCOM, CORBA and Java RMI, but these technologies use their own proprietary software methodologies, whereas Web Services use ubiquitous protocols, such as HTTP, and encode their data as XML. Although lots of technologies support Web Services, Microsoft .NET helps to build it easier and faster than most of the currently existing technologies.

Microsoft .NET web services presently support three protocols: HTTP GET, HTTP POST and SOAP. The GET and POST methods of the HTTP protocol need no introduction as they were widely used in CGI, ASP, and other server environments. SOAP is based on a W3C specification that describes a lightweight and simple eXtensible Markup Language (XML) protocol to enable the exchange of information in a decentralized, distributed environment. Similar to the GET and POST methods (in terms of responses and requests), SOAP is used for exchanging messages between the client and the server. SOAP uses XML to transfer messages, and this gives a clear advantage over the name/value pairs used by the GET and POST methods, in that now more complex information and better structure can be passed back and forth. For example, a Dataset can be serialized using XML and passed to the web service using SOAP, whereas this is not possible with the GET method. Another distinctive advantage of SOAP is that it can be used with other protocols such as SMTP.

In this article, we'll look at how to combine the features of Web Services, SOAP and the .NET framework with Microsoft Excel 2000. We will also see how to serialize a Dataset and send it as part of a HTTP Request to a Web Service.

The aim of this article is to explain, using an example, of accessing Web Services from a more powerful windows client rather than a web browser. Let us use Microsoft Excel 2000 as the Windows Client.

We will need:

  • Microsoft .NET Framework Beta 2
  • Microsoft Excel 2000
  • Microsoft XML Parser 2.6 or above
  • Microsoft SQL Server

The database we will use is the same as the SQL server database used by the site http://www.ibuyspy.com/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq. The script for the database can be downloaded from the site http://www.gotdotnet.com/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq. If you need a Microsoft XML Parser, it can be downloaded from http://msdn.microsoft.com/downloads/default.asp?url=/MSDN-FILES/027/001/677/msdncompositedoc.xml&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq

I assume that the reader has a basic understanding of the .NET framework, a knowledge of creating a basic ASP.NET web service, and knows how to connect to the SQL server database using ADO.NET, and also to use VBA.

The Problem

One of the common problems with today's e-commerce web sites is that their catalog information is so huge you have to scroll through many pages just to find the product you want to buy. Although the search capability helps in finding a product quickly, the results may also return multiple pages. For example just type "Windows" in the search text box at http://www.amazon.com/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq - it will return with more than 10000 items! With 25 items per page, this comes to 400 pages.

The Solution

With Microsoft Office so common nowadays, why don't we download the information into an Excel spreadsheet and choose the item we really want to buy, and then send the selected items to a web service that converts the selected items into an order? Although there may be some practical implications in a Business-to-Consumer environment, this idea works very well for a Business-to-Business environment.

In this article, we will see how to download the catalog information from a website to an Excel spreadsheet using an ASP.NET web service. Also, we will look at how to export the selected items as a Dataset to a Web Service that converts this to an order. After the order is processed, an invoice is delivered by the service. The Excel spreadsheet consists of two sheets, one to show the catalog information, and the other to show the invoice that is created after successfully creating an order.

Web Services Provider

The Web Service provider for this article is the ASP.NET web service that runs on the Microsoft Internet Information Server and uses ADO.NET objects to access the database. The file CatalogServices.asmx contains two web methods GetItemsByCategory and PlaceOrder. The GetItemsByCategory method returns the catalog information, and the PlaceOrder method handles the logic to create an order.

Creating the Catalog Web Service

First, let us create the catalog Web Service. The catalog Web Service is a very simple service that will return the catalog information of the site. The catalog information is the list of products that are exposed by the site to the user. For the purpose of presentation, the products have to be grouped by their category. To achieve this, we create a Dataset with two tables, Categories and Products, and link them as a parent child tables by setting the Nested property of the relation object to true.

public DataSet
            GetItemsByCategory()
{
SqlConnection myConn;
SqlDataAdapter
            myCmd;
DataSet dsCatalog = new
            DataSet("Catalog");
DataRelation
            drCategory_Product;
DataColumn
            dcCategory_CategoryID,dcProduct_CategoryID;

//Open a new SQL
            Connection
myConn = new SqlConnection(strConn);

//Create a
            Category Dataset
myCmd = new
            SqlDataAdapter(strSQL_Categories,myConn);
myCmd.Fill(dsCatalog,"Category");

//Create
            a Product Dataset
myCmd = new
            SqlDataAdapter(strSQL_Products,myConn);
myCmd.Fill(dsCatalog,"Product");

dcCategory_CategoryID
            = d
            sCatalog.Tables["Category"].Columns["CategoryID"];

dcProduct_CategoryID
            = dsCatalog.Tables["Product"].Columns["CategoryID"];

//Create
            a relationship between the Product and
            Category
drCategory_Product = new
DataRelation("Category_Products",dcCategory_CategoryID,
dcProduct_CategoryID);

//Make
            the relationship nested
drCategory_Product.Nested =
            true;

dsCatalog.Relations.Add(drCategory_Product);

myConn.Close();

return
            dsCatalog;
}

For more on how to create data relations go to the following link:

http://msdn.microsoft.com/library/en-us/cpref/html/frlrfSystemDataDataRelationClassTopic.asp?frame=true&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq

Because the Categories and Products table are nested XML, the output is displayed as follows:

<Category diffgr:id="Category2" msdata:rowOrder="1">
<CategoryID>15</CategoryID>
<CategoryName>Deception</CategoryName>
<Product diffgr:id="Product17" msdata:rowOrder="16">
<ProductID>374</ProductID>
<CategoryID>15</CategoryID>
<ModelNumber>DNTGCGHT</ModelNumber>
<ModelName>Counterfeit Creation Wallet</ModelName>
<UnitCost>999.99</UnitCost>
</Product>
<Product diffgr:id="Product19" msdata:rowOrder="18">
<ProductID>376</ProductID>
<CategoryID>15</CategoryID>
<ModelNumber>CITSME9</ModelNumber>
<ModelName>Cloaking Device</ModelName>
<UnitCost>9999.99</UnitCost>
</Product>
</Category>

As you can see, the products are grouped under the category. If the nested property is set to false, the XML outputs the categories list first, followed by the list of products.

The Microsoft Excel client will then call this Web Service. The code to call the Web Service is given below.

HttpReq.Open "GET",
"Http://localhost/services/CatalogServices.asmx/GetItemsByCategory", False
HttpReq.send

HttpDom.Load HttpReq.responseXML

The returned information, which is in XML form, is then parsed and displayed as an Excel spreadsheet by looping through the XML DOM object.

'Get the Category node
Set nodeCategories = HttpDom.documentElement.selectNodes("//Category")
If (Not nodeCategories Is Nothing) Then
'Get number of categories
numCategories = nodeCategories.Length
'Start Printing from Row 5
iStartRow = 5
'Loop throught the categories
For I = 0 To numCategories - 1
'Print the Category information
Call PrintCategory(nodeCategories.Item(I).childNodes(1).Text, iStartRow)
iStartRow = iStartRow + 1
'Get the list of products for each category
numProducts = nodeCategories.Item(I).childNodes.Length
For J = 2 To numProducts - 1
'Go to the node of each product
Set nodeProducts = nodeCategories.Item(I).childNodes(J).childNodes
'Print the product information
Call PrintProduct(nodeProducts, iStartRow)
iStartRow = iStartRow + 1
Next
Next
End If

For more help on how to transverse a XML DOM tree please go to:

http://msdn.microsoft.com/library/en-us/xmlsdk30/htm/xmconxmldomuserguide.asp?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq

Creating the Order Web Service

The Order Web Service places an order for the items selected by the user. Once the products are displayed on the spreadsheet, the user can select the item they want to buy by entering the required quantity in the Quantity column. The selected items are then placed as an order. To do this we can follow two approaches. A simple approach is to call the existing order service of the http://www.ibuyspy.com/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq web site. Another approach (that is a bit complex for this article) is to serialize the selected items into a Dataset and then pass the Dataset to a Web Service method that in turn places the order.

A Simple Way

The http://www.ibuyspy.com/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq site has a Web Service called InstantOrder. This Web Service expects four parameters: UserID, Password, Product ID and Quantity. We can make a SOAP messaging protocol call to this Web Service, and the VBA code to do this is listed below:

Private Sub PlaceOrder(ByVal ProductID As Integer, ByVal Qty As Integer)
Dim HttpReq As New XMLHttp
Dim HttpDom As DOMDocument

' Call the web service to get an XML document
HttpReq.Open "POST", "Http://www.ibuyspy.com/store/InstantOrder.asmx", False

Dim strBody As String
' Build the SOAP Body

strBody = ""
strBody = strBody & "<soap:Envelope xmlns:xsi="
"Http://www.w3.org/2001/XMLSchema-instance"" xmlns:xsd="
"Http://www.w3.org/2001/XMLSchema"" xmlns:soap="
"Http://schemas.xmlsoap.org/soap/envelope/"">"
strBody = strBody & " <soap:Body>"
strBody = strBody & " <OrderItem xmlns="
"Http://tempuri.org/"">"
strBody = strBody & " <userName>guest</userName>"
strBody = strBody & " <password>guest</password>"
strBody = strBody & " <productID>ProductID</productID>
strBody = strBody & " <quantity>Qty</quantity>
strBody = strBody & " </OrderItem>"
strBody = strBody & " </soap:Body>"
strBody = strBody & "</soap:Envelope>"

'Content Type must be set to XML
HttpReq.setRequestHeader "Content-Type", "text/xml"
HttpReq.setRequestHeader "SOAPAction", "Http://tempuri.org/OrderItem"
HttpReq.send strBody

'Check the Request status
If HttpReq.Status = 200 Then
MsgBox "Order Placed Successfully"
Else
ProcessError (HttpReq.responseXML.xml)
End If
Set HttpReq = Nothing

End Sub

Let me explain the code starting with some introduction to SOAP.

A SOAP object model consists of three objects:

  • SOAP Envelope
  • SOAP Header
  • SOAP Body

The SOAP Envelope is the root element that encapsulates other elements and also contains the namespace information and encoding style attribute that are used when the XML is serialized. In this article, we use the default namespaces and URI values.

The SOAP Header is an optional element that is used for carrying encryption information, as well as other additional information, that cannot be passed with the Body element. Since this element is optional, it is not included in the above code.

The SOAP Body element is the important piece of the model, where the real action takes place. This element contains the method and their signatures that are required to execute the Web Service. In fact you can put any valid XML (except XML processing instructions) inside the Body element. As you can see from the above code, the Body element consists of the method and signature information. The method name is Place Order, with four arguments: user id, password, Product ID and Quantity.

The entire SOAP packet is formed in XML and passed with the HTTP Request. Please note that we have to set the Content-Type header to "text/xml ". This indicates that the data with the HTTP Request is in XML form.

Processing the Error

The order is placed if all the parameters given are correct, but what happens if there is an error? For example, what happens if the user id or password that were provided are wrong? In this case, the status of the HTTP request will be returned as 500, which stands for "Internal Server Error". How will we know what caused the internal server error? SOAP takes care of this by capturing any exceptions that are thrown by the method that is executing the Web Service. SOAP has a special Fault element that captures any server-side exceptions. Since the fault response is in the form of XML, it can be parsed, and the actual error message can be shown to the user. For the above example (incorrect password) the SOAP response will be like this:

<?xml version="1.0"?>
<soap:Envelope xmlns:soap="Http://schemas.xmlsoap.org/soap/envelope/">
<soap:Body>
<soap:Fault>
<faultcode>soap:Server</faultcode>
<faultstring>System.Web.Services.Protocols.SoapException: Server was unable to process request.
---&gt; System.Exception: Invalid Login at
Catalog.PlaceOrder(String UserID, String Password, DataSet CartSet)</faultstring>
<detail/>
</soap:Fault>
</soap:Body>
</soap:Envelope>

A Complex Way

The greatest disadvantage of the above approach is that you can only place an order for one item at a time. Suppose if you are ordering say, ten items, you have to call the Web Service ten times, which is not an efficient way of accessing Web Services because of the overhead of the wire calls being made each time. The solution to this problem is to bind all the products that are selected by the user, and pass them as a single SOAP packet to the Web Service that handles the further processing.

With Microsoft .NET the solution is very simple. This is because we can serialize a Dataset as XML, and pass this XML packet with the HTTP Request. The Web Service method can then de-serialize the Dataset and any further processing to place the order is handled by the method. Thus, by one call, we can place the order, and the order confirmation can be returned to the calling Request.

Let us explore the steps that are required in this process.

  • Step 1: Build a Web Service method that can de-serialize the Dataset and place the order.
  • Step 2: Build the Schema information for the Dataset.
  • Step 3: Compiling the SOAP Information.
  • Step 4: Check for the status, and if the status is "OK " create an Invoice.

Step 1 - Building the Web Service

The PlaceOrder Web Method accepts three parameters: user id, password and the Cart Dataset that contains the collection of the products required by the user. Actually this method consists of four parts: customer validation, adding items to the shopping cart, creating the order, and returning the order details. Once the order is successfully created, the details are retrieved from the database and the order details are returned by this method in the form of a Dataset. The returned XML is parsed in the Excel client, and displayed as an invoice to the user.

The GetCustomerID method validates the customer and returns the CustomerID. If the user id or password is incorrect, then it returns a zero, in which case an exception is raised. The foreach statement loops through the Dataset and calls the AddItemToCart that adds the selected items into the shopping cart. The CreateOrder method creates the Order from the shopping cart. All these methods make use of the stored procedures in the "store " database.

After the Order is created successfully, the order details are retrieved from the database and a Dataset with two tables, Order and OrderItems, is created with a parent-child relationship, and are returned to the calling request.

public DataSet PlaceOrder(String
            UserID,String Password,DataSet CartSet)
{
int iCustomerID =
            GetCustomerID(UserID,Password);
if (iCustomerID == 0)
{
//Login Failed. Inform the user
throw new Exception
            ("Invalid Login");
}

//Build the Shopping
            Cart
foreach(DataRow dr in
            CartSet.Tables["Product"].Rows)
{
int ProductID =
            Int32.Parse(dr["ProductID"].ToString());
int Qty =
            Int32.Parse(dr["Quantity"].ToString());
AddItemToCart(iCustomerID,ProductID,Qty);
}

//Create
            the Order
int iOrderID = CreateOrder(iCustomerID);
if(iOrderID
            == 0)
{
throw new Exception("UnExpected Error : Order couldn't
            be completed");
}
else
{
//Prepare for returning a Order
            dataset
//for creating the invoice
SqlConnection
            myConn;
SqlDataAdapter myCmd;
DataSet dsOrder = new
            DataSet("Orders");
DataRelation drOrder_Items;
DataColumn
            dcOrder_OrderID,dcItem_OrderID;

String strSQL_Order = "Select
            OrderID,OrderDate,ShipDate,FullName From Orders,Customers Where
Orders.CustomerID = Customers.CustomerID and OrderID=" +
            iOrderID;

String strSQL_OrderItems = "Select
            OrderID,Products.ProductID,ModelName,ModelNumber,Quantity,Products.UnitCost
From OrderDetails,Products Where OrderDetails.ProductID =
            Products.ProductId And OrderID=" + iOrderID;;

myConn = new
            SqlConnection(strConn);

myCmd = new
            SqlDataAdapter(strSQL_Order,myConn);
myCmd.Fill(dsOrder,"Order");

myCmd
            = new
            SqlDataAdapter(strSQL_OrderItems,myConn);
myCmd.Fill(dsOrder,"OrderItems");

dcOrder_OrderID
            = dsOrder.Tables["Order"].Columns["OrderID"];

dcItem_OrderID
            =
            dsOrder.Tables["OrderItems"].Columns["OrderID"];

drOrder_Items
            =
            new
DataRelation("Order_Items",dcOrder_OrderID,dcItem_OrderID);

drOrder_Items.Nested
            =
            true;

dsOrder.Relations.Add(drOrder_Items);

myConn.Close();

return
            dsOrder;
}
}

Step 2 - Building Schema and Data

The products selected by the user from the Excel spreadsheet have to be assembled into XML format and combined with the SOAP packet. Let us now look at this in detail.

Building the schema

The XML Schema Definition Language (XSD) is used to define the structure of the XML documents. It replaces the DTD (Document Type Definition) that used to be used to define the syntax of the XML documents. The new schema language is more powerful and precise, and allows us to specify the exact data type of the element's contents.

The XSD schema is composed of root level "schema" element that includes the namespace definition. The XSD supports two types of elements: simple type and complex type. The complex type elements can contain other elements, whereas the simple type elements cannot. In the following code, CartSet is a complex type element that contains another complex element, Product. The complex type element Product contains two elements: ProductID and Quantity. When this schema information is de-serialized to a Dataset, the complex element CartSet becomes the Dataset, and the complex element Product becomes the table in the Dataset. The two elements ProductID and Quantity become the columns for the Product table. This schema can be generated using the Schema generator that comes with Visual Studio .NET.

<xsd:schema id=""CartSet"" targetNamespace=""""
xmlns="""" xmlns:xsd=""Http://www.w3.org/2001/XMLSchema""
xmlns:msdata=""urn:schemas-microsoft-com:xml-msdata"">"
<xsd:element name=""CartSet"" msdata:IsDataSet=""true"">"
<xsd:complexType>"
<xsd:choice maxOccurs=""unbounded"">"
<xsd:element name=""Product"">"
<xsd:complexType>"
<xsd:sequence>"
<xsd:element name=""ProductID"" type=""xsd:int""
minOccurs=""0"" />"
<xsd:element name=""Quantity"" type=""xsd:int""
minOccurs=""0"" />"
</xsd:sequence>"
</xsd:complexType>"
</xsd:element>"
</xsd:choice>"
</xsd:complexType>"
</xsd:element>"
</xsd:schema>"

Preparing the data

As the following piece of code shows, the data is collected from the Excel spreadsheet and an XML document is formed.

strData = ""
strData = strData & " <CartSet xmlns="""">"

For I = 5 To ActiveCell.SpecialCells(xlLastCell).Row
strID = Worksheets(1).Cells(I, 1)
strQty = Worksheets(1).Cells(I, 5)
If IsNumeric(strID) Then
ID = CLng(strID)
If IsNumeric(strQty) Then
Qty = CInt(strQty)
If Qty > 0 Then
strData = strData & " <Product diffgr:id="
"Product" + CStr(numItems + 1) + """ msdata:rowOrder="
"" + CStr(numItems) + """>"

strData = strData & " <ProductID>" + CStr(ID) + "</ProductID>"

strData = strData & " <Quantity>" + CStr(Qty) + "</Quantity>"

strData = strData & " </Product>"
numItems = numItems + 1
End If
End If
End If
Next
strData = strData & " </CartSet>"

This information will then be combined with the Schema information and other required SOAP headers and passed with the HTTP Request.

Step 3 - Compiling the SOAP Packet

Dim HttpReq As New XMLHttp
Dim HttpDom As DOMDocument

' Call the web service to get an XML document
HttpReq.Open "POST", "Http://localhost/services/CatalogServices.asmx", False

The following section of the code builds the SOAP packet, and then binds the packet with the schema information, before sending it to the Web Service. The Body element of the SOAP packet consists of three elements: user id, password and Cart set, as expected by the Web Service method PlaceOrder.

Dim strBody As String

strBody = ""
strBody = strBody & "<soap:Envelope xmlns:xsi="
"Http://www.w3.org/2001/XMLSchema-instance"" xmlns:xsd="
"Http://www.w3.org/2001/XMLSchema"" xmlns:soap="
"Http://schemas.xmlsoap.org/soap/envelope/"">"
strBody = strBody & " <soap:Body>"
strBody = strBody & " <PlaceOrder xmlns="
"Http://tempuri.org/"">"
strBody = strBody & " <UserID>guest</UserID>"
strBody = strBody & " <Password>guest</Password>"
strBody = strBody & " <CartSet>"
REM schema information goes here
StrBody = strBody & strSchema
strBody = strBody & " <diffgr:diffgram xmlns:msdata="
"urn:schemas-microsoft-com:xml-msdata"" xmlns:diffgr="
"urn:schemas-microsoft-com:xml-diffgram-v1"">"
strBody = strBody & strData
strBody = strBody & " </diffgr:diffgram>"
strBody = strBody & " </CartSet>"
strBody = strBody & " </PlaceOrder>"
strBody = strBody & " </soap:Body>"
strBody = strBody & "</soap:Envelope>"

HttpReq.setRequestHeader "Content-Type", "text/xml"
HttpReq.setRequestHeader "SOAPAction", "Http://tempuri.org/PlaceOrder"
HttpReq.send strBody

The following section of code checks the status of the request. If the Order is successfully created, it returns a value 200. Then, the returned XML is parsed and an invoice is created in the Excel spreadsheet. If there is any errors, SOAP returns a fault element that is parsed and the message is shown to the user.

If HttpReq.Status = 200 Then
PrintInvoice (HttpReq.responseXML.xml)
Else
Debug.Print HttpReq.responseXML.xml
ProcessError (HttpReq.responseXML.xml)
End If
Set HttpReq = Nothing

Step 4

The PlaceOrder Web Method returns an XML document that contains the Order Information that was successfully created by the method. The following VBA code parses the XML returned by the web service, and creates an invoice on the Excel spreadsheet. The code also calculates the total amount of the order and displays it to the user.

Private Sub PrintInvoice(response As String)
Dim I, iStartRow As Integer
Dim HttpDom As New DOMDocument
Dim nodeOrder, nodeItems As IXMLDOMNodeList
HttpDom.loadXML (response)
Debug.Print response

Set nodeOrder = HttpDom.documentElement.selectNodes("//Order")
Call PrintInvoiceHeader(nodeOrder.Item(0).childNodes)
Set nodeItems = nodeOrder.Item(0).childNodes

iStartRow = 8
For I = 4 To nodeItems.Length - 1
Call PrintInvoiceDetail(nodeItems.Item(I).childNodes, iStartRow)
iStartRow = iStartRow + 1
Next
iStartRow = iStartRow + 1
With Worksheets(2)
.Cells(iStartRow, 5).Value = "Total"
. Cells(iStartRow, 6).FormulaR1C1 =
=SUM(R[" & CStr(8 - iStartRow) & "]C:R[-1]C)"
.Activate
End With
End Sub

Conclusion

In this article we have learned some advanced concepts of accessing a Web Service from a windows client using SOAP and Microsoft Excel. Also, we have learned how to return a Dataset to a Web Service for further processing and reporting any errors if found. There is much more we can do with this application, like crafting a GUI interface in Windows Forms that would make this application more palatable. For example, we could have built a tree view to list the categories, and another list view to list the items under that category, as well as a simple form to add the required items to the shopping cart and finally create an order. Also, keeping this as base, we could build more complex enterprise business-to-business applications. One such example can be found at http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnexcl2k2/html/odc_xlb2b.asp&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq.

C#Today - © 2002 WroxPress Limited


Interoperating Java, VB6, and .NET Web Services

Interoperating Java, VB6, and .NET Web Services
http://www.csharptoday.com/content.asp?id=1738
Article Rating: 3.5
Published on October 25, 2001
Abstract
In this article, Catalin Tomescu illustrates the integration and interoperability of web services. He creates a Tax Calculator web service, which integrates two existing applications: a State Tax Calculator and a Federal Tax Calculator. To further complicate things, the two web services that our tax calculator integrates are built using different technologies ?the state tax calculator application is built using Apache SOAP for Java and the federal tax calculator application is built using Visual Basic 6.0 and the MS SOAP 2.0 Toolkit. Finally, we see how to call our Tax Calculator web service from a variety of clients - a C# console application, a VB 6 console application, and an ASP.NET web application.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

There is little doubt that web services are the future of distributed applications. We've seen a tremendous increase in the number of web services available on the Internet. Stock checkers, weather services, book finders, map and address finders - you name it. We don't need to reinvent the wheel; instead we integrate services offered by online providers and our application is almost ready. This article will illustrate how to create a tax calculator web service by integrating two existing applications: a State Tax Calculator and a Federal Tax Calculator. To further complicate things, the two web services that our tax calculator integrates are built using different technologies - the state tax calculator application is built using Apache SOAP for Java, and the federal tax calculator application is built using Visual Basic 6.0 and the MS SOAP Toolkit (STK).

Not long ago, SOAP emerged as a standard for transferring XML data over HTTP, and many companies rewrote their applications using this standard. Most of the new development is focused on writing services and not standalone applications. As we should expect, not all providers are using the same programming language, the same tools, or the same operating system in their development efforts. This creates problems when writing clients for the various different implementations of SOAP. Although most of the various implementations of the SOAP protocol are more or less compliant with the standard, there are certain important differences - for example, the lack of WSDL (unless we use IBM's WebServices ToolKit) in the Java implementation for web services makes interoperability with frameworks like .NET and MS SOAP Toolkit less straightforward than we might expect. In this article, we'll see how to overcome these problems.

Prerequisites

  • MS SOAP Toolkit Gold 2.0
  • Visual Basic 6.0
  • MS XML 3.0
  • MS .NET Framework Beta 2
  • Apache SOAP for Java 2.2
  • Tomcat 4.0.1
  • JDK 1.3.2
  • Xerces 2.0
  • IIS installed on a Windows NT or 2000 platform (Professional edition is fine for our sample).

Reading this first will aid your understanding of this article:

  • The SOAP specification available at http://www.w3c.org/soap?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq.
  • The WSDL specification available at http://www.w3c.org/WSDL?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq.

What we Learn

The article first shows how to build the two underlying web services, a Java State Tax Calculator, and a VB 6.0 Federal Tax Calculator. Then, our .NET Tax Calculator integrates these two web services to provide a single interface for calculating the amount of income tax an individual owes. (In our scenario the income tax is calculated as a sum of federal and state tax). Finally, we will see how to create a couple of different clients to consume the Tax Calculator web service.

Federal Tax Calculator

The Federal Tax Calculator is a web service that computes the amount of federal tax to be deducted, given the annual gross income and marital status of a person. This, of course, is a simplified business logic for federal tax computation.

The application is developed using MS SOAP Toolkit 2.0. The particular thing about this framework is that it assumes that web services are implemented as COM objects. The STK helps us interface the COM object with SOAP requests and responses by providing two types of SOAP listeners:

  • An ASP listener. This listener is implemented as an ASP page that decodes the SOAP request, invokes the COM method on the specified COM object, and packs the response as a SOAP response object.
  • An ISAPI listener. This listener is implemented in the SOAPISAP.dll and performs the same actions as the ASP listener.

The process of enabling a COM object for SOAP communication consists of generating a WSDL file. When we generate the web service .WSDL file from the COM object, we must choose the type of listener we want to use for the particular service. Note that there is no restriction on using both of them for the same web service, as long as the names of the .WSDL files are distinct. The ISAPI Listener is the default one and the one we will use.

Now let's write the COM object which contains the implementation of our FedTaxCalculator. You can write the COM object with whatever tool you want (VC++, VB, Delphi, etc). Here we use VB 6.0.

Here is the content of the FedTaxCalculator.cls file containing the FedTaxCalculator class:

Option Explicit

Public Function ComputeTax(AnnualGrossIncome As Double, MaritalStatus As String) As Double

Dim taxPercent As Single

If AnnualGrossIncome <= 75000 Then
taxPercent = 0.25
ElseIf AnnualGrossIncome <= 150000 Then
taxPercent = 0.3
End If

If MaritalStatus = "M" Then
taxPercent = taxPercent - 0.03
End If

ComputeTax = AnnualGrossIncome * taxPercent
End Function

Create a new VB project, call it FedTaxCalculator, add a class called FedTaxCalculator, and paste the above code into it. Save the project and build the .dll file.

Register the .dll using regsvr32 from the Run command line:

Regsvr32 <path>\FedTaxCalc.dll

Deployment

Using the WSDL Generator utility (which comes with MS SOAP Toolkit), generate the .WSDL file for the FedTaxCalculator service.

When prompted, use a virtual directory called TaxCalculator. If using the ISAPI listener, configure your virtual directory to use SOAPISAP.dll for files with a .wsdl extension (see STK documentation and setup instructions in the download for more details). After we have generated the WSDL file our service is ready to respond to requests.

State Tax Calculator

The State Tax Calculator is a web service that computes the amount of state tax to be deducted, based upon the State we are in. This web service is built using the Apache SOAP for Java Toolkit. Similarly to the MS SOAP Toolkit, we can reuse our existing Java classes and expose them as web services. The Apache implementation of SOAP differs from Microsoft's approach (for both .NET and the SOAP Toolkit) in that the Apache SOAP doesn't rely on .wsdl files to describe a web service. Instead, the Apache SOAP expects us to utilize all the features that the SOAP envelope offers and pass all the information the Apache SOAP needs to make the method invocation:

  • Parameter names
  • Parameter types
  • Parameter order
  • Method call
  • How to serialize the result and how to de-serialize input parameters

Because of this requirement there are couple of problems when trying to connect from a client built with Microsoft toolkits. We'll look at the solution later in this section, but for now let's focus on the code for our StateTaxCalculator service.

Package samples.TaxCalculator;

import
            java.*;

public class StateTaxCalculator {
/**
* This
            function computes the amount of state tax for 2 fictitious states,
            XX and YY.
* The function implements different rules for each of
            the states.
*/
public double ComputeTax(double
            AnnualGrossIncome, String MaritalStatus,
String State, int
            MonthsInState)
{
double taxPercent = 0, tax = 0;

if
            (State.equals("XX")) {
if (AnnualGrossIncome <
            75000)
taxPercent = 0.02;
else
taxPercent =
            0.035;

if (MaritalStatus.equals("M"))
taxPercent -=
            0.005;

tax = AnnualGrossIncome * taxPercent * MonthsInState /
            12;
}
else if (State.equals("YY")) {
if (AnnualGrossIncome
            < 85000)
taxPercent = 0.03;
else
taxPercent =
            0.045;

if (MaritalStatus.equals("M"))
taxPercent -=
            0.003;

if (MonthsInState > 8)
tax = AnnualGrossIncome *
            taxPercent;
else
tax = AnnualGrossIncome * taxPercent *
            MonthsInState / 12;
}

return tax;
}
}

As we can see, the ComputeTax function has four parameters:

  • annual gross income,
  • marital status of the person for which the tax is computed,
  • the state in which the person resides, and
  • the number of months the person has been a resident of this state.

This is a standard Java class and nothing more. To compile the class we would save the file as StateTaxCalculator.java and compile it with the command line:

javac StateTaxCalculator.java

However, this StateTaxCalculator.class is already included in the download in the Samples.TaxCalculator package so that you don't need to reconfigure your Tomcat server (assuming you have already deployed the SOAP samples).

Deployment

We assume you have already downloaded, installed, and configured the Java Runtime Environment 1.3 SDK, Apache SOAP Toolkit 2.2 binaries, the Apache SOAP Quickstart with Xerces 2.0 XML Parser, and Tomcat 4.0. If you haven't, complete setup instructions are included in the download's full_setup_instructions.txt file.

The Apache implementation of SOAP uses a Servlet listener for SOAP requests. In order to make our class available for a SOAP request we must deploy it as a web service using a deployment descriptor. A deployment descriptor is an XML file that contains information about the web service, such as:

  • where the Java class is located,
  • what methods it supports,
  • the encoding used.

Here is the deployment descriptor for the StateTaxCalculator service:

<isd:service StateTaxCalc xmlns:isd="http://xml.apache.org/xml-soap/deployment"
id="urn:StateTaxCalculator">
<isd:provider type="java"
scope="Application"
methods="ComputeTax">
<isd:java class="samples.TaxCalculator.StateTaxCalculator" static="false"/>
</isd:provider>

<isd:faultListener>
org.apache.soap.server.DOMFaultListener
</isd:faultListener>

</isd:service>

Create a file called DeploymentDescriptor.xml that contains the above code. All deployment descriptors for Java web services have the same structure. Only the ID of the service, the Java class where the service resides, and the methods that a service exposes, differ from one service to another.

The deployment descriptor is very important; in fact it is crucial for our web service to work correctly. As mentioned earlier, the Apache SOAP relies on the client to supply complete information about parameters. Notice that in the above deployment descriptor there is no mention about the parameters type, the order of the parameters, or how to serialize / de-serialize each parameter. A Java client will correctly construct a valid SOAP request package, but a Microsoft client (either STK or .NET) fails to do this in the absence of a .WSDL file.

Note: We could use the IBM WSTK at http://www.alphaworks.ibm.com/tech/webservicestoolkit/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq to create .WSDL files for Java classes, but have shown an alternate implementation for completeness of the article.

To overcome this we must add one more section to the deployment descriptor:

<isd:mappings>
<isd:map encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"
xmlns:x="" qname="x:AnnualGrossIncome"
javaType="java.lang.Double"
xml2JavaClassName=
"org.apache.soap.encoding.soapenc.DoubleDeserializer"/>

<isd:map encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"
xmlns:x="" qname="x:MaritalStatus"
javaType="java.lang.String"
xml2JavaClassName=
"org.apache.soap.encoding.soapenc.StringDeserializer"/>

<isd:map encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"
xmlns:x="" qname="x:State"
javaType="java.lang.String"
xml2JavaClassName=
"org.apache.soap.encoding.soapenc.StringDeserializer"/>

<isd:map encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"
xmlns:x="" qname="x:MonthsInState"
javaType="java.lang.Integer"
xml2JavaClassName=
"org.apache.soap.encoding.soapenc.IntDeserializer"/>
</isd:mappings>

Now the deployment descriptor is complete, and we can notify the Tomcat server about the presence of a new web service. Open a command window, change the current folder to the location where you saved StateTaxCalculator.java, and run the deploy.bat file which executes the following command:

java org.apache.soap.server.ServiceManagerClient
http://localhost:8080/soap/servlet/rpcrouter deploy DeploymentDescriptor.xml

To complete the deployment of our StateTaxCalculator, all we have to do is copy the contents of our donwnload's samples\TaxCalculator folder into a corresponding SOAP_HOME\samples\TaxCalculator folder on the server, where SOAP_HOME is the location where the Apache SOAP for Java is installed.

We should now be able to see our deployed service in the Apache SOAP Admin tool. Surf to http://localhost:8080/soap/:

We can view the details of the web service by clicking on its name:

In future, this information will help us to create our own deployment descriptors for any class, by clicking on the Deploy button and following the template shown above.

Surf to the url of the web service and we should see the following:

Tax Calculator

Tax Calculator is the .NET web service that integrates the State and Federal tax calculators to provide a unique interface for tax computation. We have a few choices when writing a web service with the .NET framework:

  • web service using C# or VB.NET
  • Visual C++ web service

The most complex way to write it is using Visual C++. We'll use the simpler and faster way to build a web service, by creating a web service in C#.

Web services written with .NET consist of an .asmx file. We place the .asmx file into a virtual directory where it can be accessed and that's it! The .asmx file is compiled the first time a user accesses it. Thereafter, the file is recompiled only if the server is restarted or the file is changed. This approach offers some flexibility in the development process because we don't have to stop the IIS server for a deployment of the web service.

Let's see the code for the TaxCalculator web service. The TaxCalculator.asmx file contains three classes. The first class is the main class, TaxCalculator, which exposes the ComputeTax function.

The implementation resembles a class written in C#. The only new element is marking the class and the methods for access using the SOAP protocol, using the Webservice and Webmethod attributes.

  • The WebService attribute applied to a class will enable the SOAP code for handling HTTP requests using the SOAP protocol.
  • To mark a method in a class as web enabled, we must specify the WebMethod attribute for that method.

Here is the content of the TaxCalculator.asmx file:

<%@ WebService Language="C#"
            Class="TaxCalculator" %>

using System;
using
            System.Web.Services;
using System.Diagnostics;
using
            System.Xml.Serialization;
using
            System.Web.Services.Protocols;
[WebService(Namespace="http://www.catalin.com/TaxCalculator/")]
public
            class TaxCalculator: WebService {

[
            WebMethod(Description="compute state and fed
            tax")]
[System.Web.Services.Protocols.SoapRpcMethodAttribute(
"http://www.catalin.com/TaxCalculator/ComputeTax",
RequestNamespace="http://www.catalin.com/TaxCalculator/",
ResponseNamespace="http://www.catalin.com/TaxCalculator/")]

public
            double ComputeTax(double AnnualGrossIncome, string MaritalStatus,
string State, int MonthsInState) {

double taxFed = 0,
            taxState = 0;

FedTaxCalculator objFTCalc = new
            FedTaxCalculator();
StateTaxCalculator objSTCalc = new
            StateTaxCalculator();

taxFed =
            objFTCalc.ComputeTax(AnnualGrossIncome, MaritalStatus);
taxState
            = objSTCalc.ComputeTax(AnnualGrossIncome, MaritalStatus,
State,
            MonthsInState);

return (taxState +
            taxFed);
}
}

The TaxCalculator class must access the State and Federal tax calculator services to get this job done. How do we access these services? In order for a .NET client or web service to access another web service, we need to create a proxy class, which has the same interface as the service. Whenever a call is made through this proxy server, the .NET framework forwards the call to the actual service as a SOAP message, and then retrieves the result from the SOAP response message. A proxy class inherits from the SoapHttpClientProtocol class, which will handle all SOAP communication with the server. Again, notice that attributive programming is the key for setting up each method with information about request and response namespaces and SOAP action.

The .NET framework provides a tool called wsdl.exe to generate the proxy for a web service. This executable can generate the proxy class based on the WSDL file corresponding to a web service.

Use the following command line to generate the proxy for the FedTaxCalculator service:

wsdl.exe http://<path to FedTaxCalculator>/FedTaxCalculator.WSDL

The command should generate a file called FedTaxCalculator.cs containing the following content:

[System.Web.Services.WebServiceBindingAttribute(Name="FedTaxCalculatorSoapBinding",
Namespace="http://tempuri.org/wsdl/")]
public class FedTaxCalculator : System.Web.Services.Protocols.SoapHttpClientProtocol {

[System.Diagnostics.DebuggerStepThroughAttribute()]
public FedTaxCalculator() {
this.Url = "http://localhost/TaxCalculator/FedTaxCalculator.WSDL";
}

[System.Diagnostics.DebuggerStepThroughAttribute()]
[System.Web.Services.Protocols.SoapRpcMethodAttribute
("http://tempuri.org/action/FedTaxCalculator.ComputeTax",
RequestNamespace="http://tempuri.org/message/",
ResponseNamespace="http://tempuri.org/message/")]
[return: System.Xml.Serialization.SoapElementAttribute("Result")]
public System.Double ComputeTax(ref System.Double AnnualGrossIncome,
ref string MaritalStatus) {
object[] results = this.Invoke("ComputeTax", new object[] {
AnnualGrossIncome,
MaritalStatus});
AnnualGrossIncome = ((System.Double)(results[1]));
MaritalStatus = ((string)(results[2]));
return ((System.Double)(results[0]));
}

[System.Diagnostics.DebuggerStepThroughAttribute()]
public System.IAsyncResult BeginComputeTax(
System.Double AnnualGrossIncome, string MaritalStatus,
System.AsyncCallback callback, object asyncState) {
return this.BeginInvoke("ComputeTax", new object[] {
AnnualGrossIncome,
MaritalStatus}, callback, asyncState);
}

[System.Diagnostics.DebuggerStepThroughAttribute()]
public System.Double EndComputeTax(System.IAsyncResult asyncResult,
out System.Double AnnualGrossIncome, out string MaritalStatus) {
object[] results = this.EndInvoke(asyncResult);
AnnualGrossIncome = ((System.Double)(results[1]));
MaritalStatus = ((string)(results[2]));
return ((System.Double)(results[0]));
}
}

Paste the content of the FedTaxCalculator.cs into the end of the TaxCalculator.asmx file.

At this point, the TaxCalculator is able to call the Federal Tax Calculator. Let's move next to the State Tax Calculator. The same idea must be applied here too. The only problem is that for a Java web service there is no .wsdl file from which we can automatically generate a proxy. In this situation the solution is to write the proxy class by hand. Since we already have the proxy class for federal tax, we'll modify it to meet our needs.

The major changes that must be made are:

  • The parameter list must be modified accordingly for the StateTaxCalculator service
  • The namespaces used in the proxy must be identical to the ones used by the web service

Here is the proxy class for the StateTaxCalculator service:

[System.Web.Services.WebServiceBindingAttribute(Name="StateTaxCalculatorSoapBinding",
Namespace="http://tempuri.org/wsdl/")]
class StateTaxCalculator : System.Web.Services.Protocols.SoapHttpClientProtocol {

[System.Diagnostics.DebuggerStepThroughAttribute()]
public StateTaxCalculator() {
this.Url = "http://localhost:8080/soap/servlet/rpcrouter";
}

[System.Diagnostics.DebuggerStepThroughAttribute()]
[System.Web.Services.Protocols.SoapRpcMethodAttribute(
"http://localhost:8080/soap/servlet/rpcrouter/ComputeTax",
RequestNamespace="urn:StateTaxCalculator",
ResponseNamespace="urn:StateTaxCalculator")]
[return: System.Xml.Serialization.SoapElementAttribute("Result")]
public System.Double ComputeTax(double AnnualGrossIncome, string MaritalStatus,
string State, int MonthsInState) {

object[] results = this.Invoke("ComputeTax", new object[] {
AnnualGrossIncome,
MaritalStatus,
State,
MonthsInState});

return ((System.Double)(results[0]));
}

[System.Diagnostics.DebuggerStepThroughAttribute()]
public System.IAsyncResult BeginComputeTax(double AnnualGrossIncome,
string MaritalStatus, string State, int MonthsInState,
System.AsyncCallback callback, object asyncState) {

return this.BeginInvoke("ComputeTax", new object[] {
AnnualGrossIncome,
MaritalStatus,
State,
MonthsInState}, callback, asyncState);
}

[System.Diagnostics.DebuggerStepThroughAttribute()]
public System.Double EndComputeTax(System.IAsyncResult asyncResult,
double AnnualGrossIncome, string MaritalStatus,
string State, int MonthsInState) {

object[] results = this.EndInvoke(asyncResult);
return ((System.Double)(results[0]));
}
}

The changes that have been made from the first proxy class are:

  • Because Apache SOAP for Java uses the id of the service as a namespace for SOAP messages, the namespace was replaced with urn:StateTaxCalculator.
  • The list of parameters in the ComputeTax method was changed to: double AnnualGrossIncome, string MaritalStatus, string State, int MonthsInState.

Append this StateTaxCalculator.cs code to the end of TaxCalculator.asmx file and save it. We no longer need the proxy .cs files and now have a web service that is able to call some other services, gather results from these services, and produce a response.

Deployment

Create a virtual directory in the IIS server named TaxCalc and place the TaxCalculator.asmx file into it. To test the service, open a new instance of IE and type in the address http://localhost/TaxCalc/TaxCalculator.asmx?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq. If no errors are encountered we should be able to see a page containing information about the service:

To see the WSDL file open the IE and type in the address http://localhost/TaxCalculator/TaxCalculator.asmx?WSDL&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq.

Accessing the TaxCalculator service

So far we have developed a web service (TaxCalculator) that integrates the functionality of the other two services (StateTaxCalculator and FedTaxCalculator) in order to compute the total amount of income tax payable for a given person. The TaxCalculator service exposes its functionality for client applications.

In this final section we'll see how to access the TaxCalculator service from client applications written in different languages.

A .NET Console Client

Writing a C# client for TaxCalculator is very simple once we generate the proxy class. To generate the proxy class for the C# web service open a command window and execute:

wsdl.exe http://localhost/TaxCalculator/TaxCalculator.asmx?WSDL

The tool will generate a file called TaxCalculator.cs which we can compile and easily use in another C# program, like this:

using System;

/**
* This is a C# client for computing taxes.
*/
public class TaxCalculatorClient {
public static void Main() {

String State = "XX", MaritalStatus = "M";
double AnnualGrossIncome = 76000,
TotalTax = 0;
int MonthsInState = 9;

/**
* Create proxy object to access the tax service.
*/
TaxCalculator objTaxCalc = new TaxCalculator();

/**
* Call the ComputeTax method on each of the services.
*/
TotalTax = objTaxCalc.ComputeTax(AnnualGrossIncome, MaritalStatus,
State, MonthsInState);

/**
* Display the result.
*/
Console.WriteLine("John Doe");
Console.WriteLine("Annual Gross Income: ${0}", AnnualGrossIncome);
Console.WriteLine("Resident of : {0}", State);
Console.WriteLine("Marital status : {0}", MaritalStatus);
Console.WriteLine("No of months in {0} : {1}", State, MonthsInState);
Console.WriteLine("Total tax : ${0}", TotalTax);
}
}

As you can see, the code is very simple and the proxy object is used as a standard C# object. We simply call the method and the framework handles everything, from creating the SOAP envelope message, to retrieving the result from the SOAP response.

Save the C# client into a file called TaxCalculatorClient.cs, then compile the TaxCalculator.cs as a library with:

csc.exe /t:library TaxCalculator.cs

Then compile the client using:

csc.exe /reference:TaxCalculator.dll TaxCalculatorClient.cs

Open a command window, change the current folder to the location where you saved and compiled the C# proxy and client classes, and execute TaxCalculatorClient.exe.

The result should be:

A VB 6.0 MS SOAP Toolkit client

Writing a client for TaxCalculator with the MS SOAP Toolkit is simple when we have the .WSDL file. The client is written in Visual Basic and uses the high level API available with the MS SOAP Toolkit.

The CallTaxCalculator function uses the high level API to create the SOAP request and to parse the server response.

Private m_soapClient As SoapClient

Private Const END_POINT_URL = "http://localhost/TaxCalculator/TaxCalculator.asmx"

Public Sub CallTaxCalculator()

Dim objServer As SoapClient, result As Double
Dim AnnualGrossIncome As Double, MaritalStatus As String, State As String, MonthsInState As Integer

AnnualGrossIncome = 76000
MaritalStatus = "M"
State = "XX"
MonthsInState = 9

On Error GoTo catch_all_err
Set objServer = New SoapClient
Call objServer.mssoapinit(END_POINT_URL + "?WSDL")

result = objServer.ComputeTax(AnnualGrossIncome, MaritalStatus, State, MonthsInState)

' display the results
Debug.Print "Name: John Doe"
Debug.Print "Annual Gross Income: " + CStr(AnnualGrossIncome)
Debug.Print "Marital status: " + MaritalStatus
Debug.Print "State: " + State
Debug.Print "Months in state: " + CStr(MonthsInState)
Debug.Print "Total Tax: " + CStr(result)

cleanup:
On Error Resume Next
Set objServer = Nothing
Exit Sub

catch_all_err:
Debug.Print Err.Description
End Sub

Private Sub main()
Call CallTaxCalculator()
End Sub

Create a new Visual Basic 6.0 project, add a module, and paste the above code. In the VB project add references to MSXML 3.0 and MS SOAP Type Library, set the start point of the project main function, compile and execute. The result should be the same as with the C# client.

An ASP.NET Web Client

Finally, we will see how to create an ASP.NET client using C#. This is even easier than with the above two methods. Once we have opened Visual Studio .NET and created a new C# ASP.NET Web Application we only have to follow 3 steps:

Create the User Interface by dragging five TextBox controls and one Button control from the Toolbox in Visual Studio .NET:

Add a web reference to the web service at http://localhost3/TaxCalc/TaxCalculator.asmx?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq:

Add the click handler code in Webform1.aspx (remember that TextBoxes can only handle string types, so we need to convert AnnualGrossIncome and TotalTax).

private void CalculateTax_Click(object sender, System.EventArgs e)
{
string StringAnnualGrossIncome = TextBox1.Text;
double DoubleAnnualGrossIncome = Convert.ToDouble(StringAnnualGrossIncome);
string StringMonthsInState = TextBox4.Text;
int IntMonthsInState = Convert.ToInt32(StringMonthsInState);

TaxCalculatorClient.localhost3.TaxCalculator ws =
new TaxCalculatorClient.localhost3.TaxCalculator();
double DoubleTotalTax = ws.ComputeTax(DoubleAnnualGrossIncome,
TextBox2.Text, TextBox3.Text, IntMonthsInState);

string StringTotalTax = Convert.ToString(DoubleTotalTax);
TextBox5.Text = StringTotalTax;

}

Run the project by holding down Ctrl + F5 and we should see the following result:

Conclusion

Writing web services and client applications using different languages and toolkits is a necessity today. There will always be issues and problems because of different approaches used by software providers and open source projects. Hopefully this article will help you solve some of the problems when writing C# web services and client applications. You may have further questions - leave your comments in the discussion forum and we will cover them in a future article.

If you'd like to discuss a broader range of C#Today topics and issues, or if you have any C# related questions you need answers to, please visit our online resource centres at : http://p2p.wrox.com/list.asp?list=csharptoday_discuss&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq and http://p2p.wrox.com/list.asp?list=c_sharp&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq
C#Today - © 2002 WroxPress Limited


Web Services: Performance and Scalability Design Considerations

Web Services: Performance and Scalability Design Considerations
http://www.csharptoday.com/content.asp?id=1673
Article Rating: 4
Published on October 10, 2001
Abstract
Designing high-performance Web Services can be a difficult task. In this article, Chandu Thota takes a close look at various design considerations involved when building scalable and high performance Web Services/applications. He then uses a sample Web Service to show how different caching options can improve the performance!
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

Designing high-performance Web Services can be a difficult task. For example, what if you were asked to design a Web Service to render some "content" to clients? All you have to do is to access some back-end data source (say SQL Server) and render it to the client, right? But what if your Web Service is supposed to render some content like the infamous Starr Report? Did you put in enough thought into your Web Service's performance and scalability to serve an unexpected 20 Million requests in one day? That's where designing Web Services becomes a serious job!

In this article, we'll take a close look at some of the various design considerations involved when building scalable and high performance Web Services/applications. We will then use a sample Web Service to see how different caching options can improve the performance!

All code snippets and samples are in C#. Also, you will need ASP.NET premium edition (http://msdn.microsoft.com/downloads/default.asp?URL=/code/sample.asp?url=/MSDN-FILES/027/001/659/msdncompositedoc.xml&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq) installed on top of .NET Beta2 to see the caching effects.

Performance: Where do we lose and how do we gain?

As we all know, a web service consumer is just a proxy (we'll call it a "proxy" from now on) and every time it receives requests from users, it has to make calls to the web service provider (let us call it a "provider" from now on). With this model, we need to keep the following two important things in mind:

  • We are making an extra round trip to a remote server, the provider, which will add a delay element to the final response time.
  • The provider's infrastructure should be scalable enough to handle all the requests from proxies from all over the web.

Now let us see different levels where we could gain performance:

  • Networking level, for example, in TCP/IP.
  • Network domain/host addressing system, for example, in the Domain Name Service, DNS.
  • Operating system level, for example, "virtual host" support.
  • Distributed operating system software layer, for example, by the distributed file system.
  • Web application level, for example, in the Web server or browser.
  • Web application utilities and servers, such as proxies, and network caches.
  • Processor level, by implementing "Web Gardens" on multi-processor machines.

Since we are the ones who develop web services as web applications on provider servers, we are going to focus on Web application level. We will also take a look at the Web application utilities layer, to see how one can adopt the architecture of Content Delivery Networks (CDN) to web services, to provide "proxy caching".

Performance Considerations: The Provider's Perspective

As we discussed before, a provider is the one who runs the web service. Considering the fact that most web applications are data driven, let us start with Data Access layer.

  • Tune your database

An optimally indexed database is a huge plus while building a high-performance web service application. Proper indexing could dramatically reduce the time needed to run queries.

  • Use stored procedures to access data

In the .NET framework, SQL server-based data access is the recommended choice for building a high-performance data access layer. It will be an additional performance boost to use SQL stored procedures instead of ad-hoc queries.

  • SqlDataReader Class Vs DataSet Class

Using the SqlDataReader class will give an added performance advantage since it implements a Tabular Data Stream (TDS) protocol to read data directly from a database connection. (TDS is an "in-house" protocol developed and used by Microsoft and Sybase for data transfer between the database server and the client. There is no published documentation for TDS, however one can find some preliminary documentation for version 5.0 at http://www.ibiblio.org/freetds/tds.html?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq. MS SQL2000 server uses TDS v8.0).

  • Interoperability considerations

In .NET, interoperability is very easy to implement, but it could become a bottleneck to your application's performance when there are excessive calls to un-managed COM component functions. If this is the case, you should consider either re-designing the COM component to reduce data marshalling, or porting it to managed code!

  • Caching Application Data

One of the greatest features of .NET is caching. Using proper caching mechanisms will buy you a huge performance gain in most cases (as we will see later in this article).

Since we are talking specifically about "Web Services", it would be appropriate to mention the ability to cache web services in .NET. A simple setting of a property in the "Web Method" attribute can cache the Web Service. See below:

[WebMethod(CacheDuration=60)]

The above setting would cache a Web Service for 60 seconds!

  • Sharing application cache among Web Services

Sharing the application cache among different web services on a particular provider server is a good idea to improve performance. For example, if you are providing two web services, and if both of them access a product catalog, it is a good idea to cache the product catalog details into the application cache the first time when you access it (from either of the Web Services).

  • Avoid exceptions

Simply put, avoiding situations where exceptions would be raised increases your application's performance.

  • Disable Sessions

When not in use, disable the sessions by setting " EnableSession " to " false " in the WebMethod attribute. See below:

[WebMethod(EnableSession=false)]

  • Disable Buffer Response

Since we want to render the response as fast as possible, it is always a good idea to disable the option " BufferResponse " by setting to " false " in WebMethod attribute, as shown below:

[WebMethod(BufferResponse=false)]

Performance Considerations: The Proxy's Perspective

Though the above design considerations (except the last two) are also applicable to proxy (the consumer of Web Service) applications, the following items should be kept in mind while designing a high-performance proxy application.

Since the proxy applications are usually in the User Interface layer of a web service scenario, let us start with ASP.NET optimization considerations.

  • Disable Session State

When not in use it is a good idea to disable the session state. This can be done either at a page level using a " @ page " directive, or at application configuration level using the " sessionstate " node. See below:

Page level setting:

<%@ Page EnableSessionState="false" .. %>

Application level setting ( web.config file):

<sessionState mode="off". . />

  • Avoid unnecessary round trips

There are two different types of "round trips" for a proxy: from client to proxy server, and from proxy server to provider server. The client to proxy server round trip can be avoided by deploying client side form data validations and " Page.IsPostback " implementations. The round trip from the proxy server to the provider server can be avoided by implementing "proxy caching" mechanisms for appropriate applications. This is where we can adopt the CDN architecture to provide proxy caching, which improves the performance dramatically. The benefits of proxy caching include:

  • Reduced user-perceived delay.
  • Lighten the load on the provider server (of the web service).
  • Nearly optimizes the network bandwidth usage.

However the proxy caching cannot be used for specific applications that deal with personalized content, real time data, etc. Good examples of applications that could be implemented with proxy caching would be product catalogs and non-personal content related applications. However, the downside of proxy caching is "stale content" in the proxy server cache. So, a well thought design decision must be taken for cache duration for proxy caching.

  • Disable "view state"

Disable "view state" options to maintain the state of the server controls when not required. This could be done on two levels, control level and page level. The property being used to do this is " MaintainState ".

Page level setting:

<%@ Page MaintainState="false" .. %>

Control level setting:

<asp:datagrid MaintainState="false" .. />

With this background, it's time get our hands dirty. Let us take a look at a sample "Daily Horoscope" Web Service, and how its performance varies with different caching options. I'm choosing a "caching" parameter to demonstrate performance variations because it is the cheapest, easiest, and yet most powerful way (in fact "Web Caching" even spawned a new industry: Content Delivery Networks - CDN, which is growing at a fantastic rate!).

"Daily Horoscope" Web Service: Performance Test

In this section we are going to:

  • Build a "Daily Horoscope" web service
  • Run stress tests using WAS (Web Application Stress tool from Microsoft) under three different conditions:
    • No Caching enabled both on Provider and Proxy
    • Caching enabled on Provider (Web Service) and NO enabled caching on proxy
    • Caching enabled both on Proxy and Provider

I intentionally chose MS Access to be the database for this Web Service since it is less scalable compared to MS SQL Server.

Daily Horoscope Web Service

I'm not going into the detail of building Web Services, because it is not the topic of interest in this article. However, we will take a quick look at what it is about.

The " asmx " output looks like the figure below for the "Daily Horoscope" Web Service:

Figure 1

We then invoke the " GetDailyHoroscope " with an integer value equivalent to your star sign (for example, Aries is 1, Leo is 2, and so on.) as shown below:

Figure 2

The web service will query the database (which is MS Access in our case) and return the daily horoscope as a string (see below) for the star sign that is being passed in.

Figure 3

The sample output from our web service for the star sign "Aries" is shown in Figure 3.

For detailed C # code, please take a look at the " service1.asmx " file in the support material downloads. Now let us take a peek at a "consumer" for this Web Service. As we discussed above, we are supposed to invoke the Web Service with an integer between 1 and 12 (corresponding to the 12 star signs). So, we have built the User Interface layer for the web users like this:

Figure 4

When a client selects his/her star sign from the dropdown list and submits the form, we will present the daily horoscope as shown below:

Figure 5

Again, for the C# implementation of consumer, please take a look at " Webform1.aspx " from the support material downloads.

Now we are ready to run the stress tests.

For those who are new to WAS, it is a freely downloadable Web application stress testing tool from Microsoft. It is downloadable, and we can also take a look at the documentation online at http://homer.rte.microsoft.com/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq.

Specifications of WAS test environment

We "simulate" stress tests on a Windows 2000 machine with the following specifications:

  • Single Pentium III processor
  • RAM as low as 256 MB!
  • MS-Access Database (located on the same machine as the web server)
  • One Client

The WAS settings for the tests are shown below:

  • Varied stress levels from 1 to 10 (usually the recommendation is to use 10 to 100 on a full fledged server, however, I have chosen the above stress level after considering the CPU's usage factor).
  • Kept the stress multiplier (Sockets/Thread) constant at 1.
  • Set six different page groups with six different "star sign" selections with equal distribution percentage.
  • Each sample test's duration set to 60 seconds.
  • Performance counters considered: requests served per second, TTLB (Time To Last Byte) and CPU utilization.
  • Performance counters were recorded every 5 seconds during the test.

The test was then run under the following three different conditions by varying the stress from 1 to 10 in each condition:

  • No caching is enabled on both proxy and provider (Web Service)
  • Caching enabled for Web Service and NO caching at proxy. Web Service was cached for 60 seconds as shown below:

[WebMethod( Description="Pass in your sun sign and get your daily horoscope!", CacheDuration=60)]

  • Caching enabled both on Proxy and Provider.

Web Service was cached for 60 seconds as shown in the previous test, and at the proxy we have enabled caching for 60 seconds, as shown below:

<%@ OutputCache Duration="60" VaryByParam="sunsign"%>

Stress Test Data Analysis

Now we have the reports from WAS for the simulated stress tests. Let us take a look at a few interesting figures that caught my attention. If you wish to take a glance at the full report, you can do so by downloading the support material for this article.

Performance: Requests Per Second

When it comes to performance of a web application, one of the major factors to be considered is the "maximum requests" that it can handle. The unit for this factor is Requests Per Second, or RPS for short. Actually RPS is considered to be one of the factors of a server's performance. However, it should be remembered that an efficient (optimum CPU and Resource consuming) web application increases Server performance. At an average constant stress the RPS graph looked like below:

Figure 6

As we can see, the capacity of serving Requests Per Second has gone up dramatically with caching in effect! A 35 % overall gain in RPS (which means performance!) with caching enabled both on Proxy and Provider!

Scalability: Time To Last Byte (TTLB)

TTLB is an important factors to be considered when we think of Scalability. Naturally, the higher the TTLB, the slower the response, which means web site users will have a slow web surfing experience. The following graph is plotted for Load Percentiles Vs TTLB in Milliseconds. A load percentile can be explained as the fraction of the load expressed as a percentage considering the maximum load to be 100%. A detailed discussion on "load percentiles" can be found at Microsoft's WAS Knowledge Base. Again, I have considered the average stress for this graph, and the result is shown below in Fig. 7:

Figure 7

There are some interesting points to be noted from the above graph.

  • The response on the web site is comparably equal for the "No Caching" and "Web Service Cached" scenarios for the first 50 % of the web site users.
  • As more and more users started using the site, the response became unusually slow for the "No Cached" case (the blue line). We can see there is a 13 % increase in TTLB from the 25 % to 75 % load percentile.
  • The TTLB almost maintained a straight line (the black line) in the case of "Web Service Caching", with varying load percentiles. The overall increase in TTLB was only 5%.
  • Something unusual happened in the scenario where both the Proxy and Provider were enabled for caching! The response was outrageously slow for all load percentiles, and it got worse with the increased load percentiles. This is something we didn't expect! If we look at the CPU utilization, it will explain the problem. The CPU utilization was at an average of 99.84%. So the bottleneck here is the processor, since it is serving the RPS's beyond it's capacity (the WAS documentation states that in most cases the processor is the bottleneck). It was serving 1443 requests per minute, where as it was only 1179 requests per minute for "No Caching", and 1309 requests per minute for "Web Service Cached" cases. So, this is a situation that we cannot control within our applications. However, having a multi-processor server might help in this case.

Conclusion

Though the tests we have conducted are not real-time tests with multiple servers and clients, they will help us understand some key concepts about how careful design considerations boosts performance significantly. It is worthwhile to remember that every small design consideration counts for something toward overall efficiency and scalability of our web applications, which provide a better experience for end users. Finally, Web Services not only offer cool technology, but they also pose a great challenge. They demand more efficiency and scalability. After all efficiency and scalability are the "mantra" of our developer community, aren't they?

Please note: The testing in this article was done using a beta product, and so may not neccessarily be accurate.

If you'd like to discuss a broader range of C#Today topics and issues, or if you have any C# related questions you need answers to, please visit our online resource centres at : http://p2p.wrox.com/list.asp?list=csharptoday_discuss&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq and http://p2p.wrox.com/list.asp?list=c_sharp&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq

C#Today - © 2002 WroxPress Limited


Building a Simple ASP.NET Editor for Web Configuration

Building a Simple ASP.NET Editor for Web Configuration
http://www.csharptoday.com/content.asp?id=1660
Article Rating: 4
Published on October 5, 2001
Abstract
In this article, Shannon Place shows us how to build a simple web based editor for the web.config file of ASP.NET. The article covers the basics of the new web.config file structure, as well as showing the configuration file hierarchy of the .NET environment. It also demonstrates how the XmlDocument object and its methods are used to load, modify, and write out XML from a .aspx page. The final code download is a simple web configuration file editor that allows a user to change the configuration settings of a web application through another web application, without needing to have explicit knowledge of the web.config file and its format.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

In this article, a simple web based editor is built for the web.config file of ASP.NET. This article will cover the basics of the new web.config file structure, as well as helping the reader to understand the configuration file hierarchy of the .NET environment. It will also demonstrate how the XmlDocument object and its methods are used to simply load, modify, and write out XML from a .aspx page . The final code download is a simple web configuration file editor that allows a user to change the configuration settings of a web application through another web application, without needing to have explicit knowledge of the web.config file and it's format.

web.config file overview

In ASP.NET we no longer need to use the MMC for IIS to implement many of the configuration settings for a web application. These configuration settings are now stored in an XML based file named web.config. Storing these items in a text based file allows ease of editing, as well as ease of deployment by implementing what's know as XCopy deployment. This simply means that a site can be deployed on a new machine, or a hundred new machines, by simply pushing the code and configuration files onto the machine or machines, at which time the site configuration is taken care of automatically using the setting in the web.config file. Before looking into the structure and content of a simple web.config file, it's important to understand the hierarchy of configuration employed by ASP.NET web applications.

ASP.NET Configuration Hierarchy

ASP.NET utilizes a hierarchy of files for web application configuration. At the root of this hierarchy is the machine.config file which is located in the same directory as the ASP.NET installation. This file contains the master settings for ASP.NET and serves as a repository for all of the ASP.NET default settings. Lower in the hierarchy are the root web.config files. These files contain configuration settings specific to a web application and are located in the root of the web application. Note that web.config files do not need to be located in sub directories of the machine.config directory. Settings in these files override the machine.config settings. Below this, web.config files may be placed in sub-directories of the web application by the web developer or a system support person. These files contain configuration settings that are specific to parts of a web application located in child directories. Sub directories that do not contain a web.config file will inherit the settings from the configuration file in the above them in the hierarchy. This hierarchy can continue down child directories allowing for a very fine grain control of web application settings. ASP.NET monitors these files for changes at all times. When a configuration file is updated, ASP.NET will implement those changes immediately for any new requests without requiring a restart of the application. Currently executing requests will complete using the old configuration values until the request is fulfilled. For a more complete explanation of .NET configuration hierarchy see http://msdn.microsoft.com/library/default.asp?url=/library/en-us/cpguidnf/html/cpconconfigurationinheritance.asp&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq.

web.config structure

As stated above, the .NET configuration files are all text files in XML format. There are a great many aspects of .NET that can be configured with these files other than ASP.NET. While these other aspects are important, they are outside of the scope of this article and will not be covered. For a full description of .NET configuration files see http://msdn.microsoft.com/library/default.asp?url=/library/en-us/cpguidnf/html/cpconaspnetconfiguration.asp&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq.

Below is an example web.config file for a simple ASP.NET application.

<configuration>
<system.web>
<pages buffer="true"
enableSessionState="true"
enableViewState="true"/>
<trace enabled="true"
requestLimit="10"
pageOutput="false"
traceMode="SortByTime"
localOnly="true" />
<customErrors mode="On"
defaultRedirect="ErrorPage.html" />
</system.web>
</configuration>

ASP.NET configuration settings are located between the opening and closing <system.web> tags, all of which needs to be located inside of the main <configuration> tags. The web.config file also needs to be valid XML with tag names being exact as they are case sensitive. The ASP.NET environment will return an error if the web.config file does not properly parse when loaded by the environment. Three configuration settings will be used in this article and are outlined below. For a list of all configuration setting, use the following link http://msdn.microsoft.com/library/default.asp?url=/library/en-us/cpgenref/html/gngrfaspnetconfigurationsections.asp&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq

Configuration Sections

<pages>

This section deals with page wide configuration elements such as buffering and viewstate. Page elements may be enabled and disabled through this section.

<trace>

This section allows configuration of ASP.NET trace settings. Among other items, it allows the developer to enable or disable tracing, and determine whether the trace output is streamed to page directly, or stored in memory for viewing with the ASP.NET trace tool.

<customErrors>

This section allows custom error pages to be defined, as well as controlling whether the customer error system is enabled or disabled.

Code Design Motivation

In order to edit the web.config file through the web, the page performing the configuration work needs to be in a separate web application or Application Domain from the web application for which it is changing the configuration settings. This is because of the way ASP.NET deals with configuration file changes. Recall from the explanation above, that when ASP.NET detects changes in the web.config file, it reconfigures itself according to those changes, but allows current requests to finish using the current configuration settings. Given this, it follows that the file performing the configuration would benefit by not changing along with the new configuration settings, as it becomes hard to detect the new changes. This is not to say that the simple web configuration tool will only work in it's own application domain, but results may appear non deterministic at times, and therefore a bit more difficult to debug.

Figure 1 illustrates the environment of the web configuration tool as it was developed and tested. A web application with the alias " webconfig " points to a directory containing the configuration editing aspx code. A second web application with the alias " testsite " points to a directory representing the production web application to be configured. The WebConfigEdit.aspx code will load, edit, and write out the web.config for the test site using the simple web based interface of the aspx page.

Figure 1

WebConfigEdit.aspx Control Modes

The WebConfigEdit.aspx page works in two modes, normal and postback. In normal mode, the code simply initializes form elements with the current settings in the web.config file and displays them as choices to the user. In postback mode, the code reads in the user selections and applies those selections to the web.config file. The following code illustrates the overall control flow of the WebConfigEdit.aspx code.

void Page_Load(Object sender, EventArgs
            e){

. . .
//code omitted for clarity
. . .

//If
            the page is in postback mode, update the config
            file
if(Page.IsPostBack){
UpdatePageItems(ref
            objXML);
UpdateTraceItems(ref objXML);
UpdateCustomErrors(ref
            objXML);
objXML.Save(CONFIG_FILE);
}else{
//else use
            the config file to initialize the form
            elements
InitializePageItems(objXML);
InitializeTraceItems(objXML);
InitializeCustomErrors(objXML);
}
}

Loading web.config

Loading the web.config file from as aspx page is a very straightforward operation that relies on the fact that the configuration file is a well-formed XML document. The XmlDocument class in the System.Xml namespace has a simple Load(string) method that allows an XML document to be parsed and loaded into memory given a file path as a string. This allows the web.config file to be loaded into an XML DOM that the code can now search and edit easily using the DOM interface. The following C# code in the Page_Load method loads the web.config XML file into an XmlDocument object.

//Path to the configuration file for the
            target web application
const String CONFIG_FILE =
            "C:\\TestSite\\web.config";

//Create the XML Document
            object
XmlDocument objXML;
objXML = new
            XmlDocument();

//Load the XML Document representing the
            web.config file
objXML.Load(CONFIG_FILE);

Of course I agree that a hard-coded path to the configuration file is not ideal. Storing the path in an appSetting section of a web.config file would be more appropriate, but a string constant suits the purpose of this article just fine.

Initializing the ASP.NET form elements

As mentioned above, when the page is first loaded it needs to display the current values of the web.config file using form elements. ASP.NET Server Controls are used for this as they allow consistent server-side coding throughout the application. The HTML code snippet below illustrates the use of the asp:Checkbox control and the ASP.NET Textbox control in the WebConfigEdit.aspx page.

<asp:CheckBox id="chkBuffer" runat="server" text="Buffering" />

<asp:Textbox id="txtRequestLimit" runat="server" size="3" />

These controls can now be referenced from the C# code in the Page_Load method. To initialize a configuration element, the element's XML node first needs to be located within the XML DOM. This is done using the SelectSingleNode(string) method of the XmlDocument class. This method returns an XML node object given an XPath statement as a string, without having to iterate through the DOM tree looking for a desired node. The C# code snippet below shows a method from WebConfigEdit.aspx that will initialize the page configuration items.

void InitializePageItems(XmlDocument
            objXML){
//Get the pages XML Element
XmlElement pages =
            (XmlElement)objXML.SelectSingleNode("//pages");

//Set the
            check boxes according to the current file
            selections
if(pages.GetAttribute("buffer") ==
            "true"){
chkBuffer.Checked =
            true;
}
if(pages.GetAttribute("enableViewState") ==
            "true"){
chkEnableViewState.Checked = true;
}
}

Note that the code is using the less efficient, but more flexible //pages XPath shortcut. This informs the XPath engine to search for any page node at all levels of the XML document instead of supplying the explicit path. When an XPath statement does not define the fully qualified path, the XPath engine needs to check more nodes than it does for the fully qualified path. This is done for code clarity only. For more information about XPath see http://msdn.microsoft.com/library/default.asp?url=/library/en- http://msdn.microsoft.com/library/default.asp?url=/library/en-us/xmlsdk30/htm/xmconxpath.asp&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq. Using the SelectSingleNode method, a node representing the <pages> configuration settings is returned. This is then explicitly cast as an XmlElement object to take advantage of the GetAttribute(string) method of that class. This method allows the lookup of a value for an XML Node attribute by name. The code can now initialize the checkbox or textbox element using the value of the attributes for the given <pages> configuration sections. Figure 2 below shows the rendered page for an example web.config file.

Figure 2

Updating the web.config file

In postback mode, the code needs to read the posted data choices from the form elements and use those to update the appropriate values of the web.config file. To accomplish this, once again the XML data of the web.config file needs to be loaded into XML DOM as before. Once the web.config information is in the DOM, we can utilize form values given by the user to update the DOM elements. Below is a C# code snippet that performs this work for the <pages> configuration section.

void UpdatePageItems(ref XmlDocument
            objXML){
//Get the pages XML Element
XmlElement pages =
            (XmlElement)objXML.SelectSingleNode("//pages");

//Set the
            fields according to the user
            selections
if(chkBuffer.Checked){
pages.SetAttribute("buffer",
            "true");
}
else{
pages.SetAttribute("buffer",
            "false");
}
if(chkEnableViewState.Checked){
pages.SetAttribute("enableViewState",
            "true");
}
else{
pages.SetAttribute("enableViewState",
            "false");
}
}

The first item to understand is the use of the ref keyword in the parameter list of the method as shown below.

void UpdatePageItems(ref XmlDocument objXML)

This indicates that the parameter is to be passed by reference instead of by value, which is the default in C#. The code needs to pass this item by reference to avoid creating deep copies of the XML object each time a method is called to update the XML. Remember, the XML object was created in the Page_Load method using the web.config data. Passing this to a method by value would give the method it's own local copy of the XmlDocument object. Any edits to this object would not be reflected in the original XmlDocument object created in Page_Load. When this object is then saved back to disk, none of the edited information will be included. Passing the object by reference allows the method to modify the original object and have those modifications persist after the method exits.

Once the method has a reference to the XML DOM, it needs to locate the desired node to be edited. Like the initializing code, this also uses the SelectSingleNode method, casting the return as an XmlElement object.

XmlElement pages = (XmlElement)objXML.SelectSingleNode("//pages");

Now, instead of the code checking the XML element to determine the setting of the form element, the code simply uses the value of the form element to edit the configuration file settings. This is done using the SetAttribute(string,string) method of the XmlElement object. This method takes a string indicating the name of the attribute, and another string indication the value for that attribute.

Saving the web.config file

Once the web.config file's XML DOM representation has been modified, it needs to written out again to disk for the changes to be implemented by ASP.NET. This is done using the Save(string) method of the XmlDocument object as the C# code snippet below illustrates.

objXML.Save(CONFIG_FILE);

The Save method simply takes a string parameter representing a file path, and serializes the XML out to that location. In this case, the code uses the same file path as was used to read in the web.config file, thus overwriting the configuration file for the web application.

Caveats

Of course, as implemented, the code has some issues. First, this page would need to be in a secure location as a user can effectively change the way the site is functioning by accessing this aspx page. Second, this code assumes that the configuration item it is looking for already exists in the web.config file. Therefore, this is not a general solution to edit any configuration file, but rather an example of how a web.config file could be modified simply over the Web. Lastly, notice that only a few web.config elements are covered in this treatment. As stated earlier, there are a great many setting that can be configured in the ASP.NET configuration model. This article uses a few for examples, but any of these setting could be configured in the same way by extending the code presented.

C#Today - © 2002 WroxPress Limited


Introduction to UDDI

Introduction to UDDI
http://www.csharptoday.com/content.asp?id=1666
Article Rating: 4.5
Published on October 4, 2001
Abstract
Traditionally, developers have used third party components and libraries to aid their development efforts. Vendors rely on listing their products in well known directories that provide categorized listings. These directories provide descriptions that are then used by clients to discover the required services. The standard used for describing and discovering web services was named as Universal Description, Discovery and Integration, also known as UDDI. In this article, Kaushal Sanghavi will explain the motivation behind UDDI and how developers can use UDDI to describe their web services, and also to discover third party web services.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

Traditionally, developers have used third party components and libraries to aid their development efforts. In this case, the third party vendor is usually a well known player in its own area of expertise, and developers chose vendors based on experience and reference. For example, there are very well known vendors that provide setup utilities and grid controls to name a few. While this works well once a vendor is established, for a new entrant into the market, experiences and references are hard to come by. In this case, vendors rely on listing their products in well known directories that provide categorized listings. These directories provide descriptions that are then used by clients to discover the required services. With the advent of web services and the programmable web, the need was felt to take these static listings one step forward, and make them dynamic repositories that could be queried and programmed in a uniform way. This was the motive behind a group of companies, like Microsoft, IBM, and Ariba, to name a few, to come up with a standard for describing and discovering web services. This standard was named as Universal Description, Discovery and Integration, also known as UDDI.

In this article, we will explain the motivation behind UDDI and how developers can use UDDI to describe their web services, and also to discover third party web services. Since UDDI is still under development and the specifications are still in Beta stages of review, in this article we will focus mainly on the Discovery aspect of UDDI, as far as sample code is concerned. We will cover the description, invocation and integration features in a future article.

Web Services and the Programmable Web

In the last year or two, commerce on the Internet has exploded to the forefront. There are a lot of business-business transactions that are taking place on the Internet. Businesses commonly provide and consume services from other businesses, and application integration over the Internet is here to stay. A lot of this technology has been based on closed proprietary platforms, and has required specific customizations even for routine functions. The advent of XML and SOAP as widely accepted, platform agnostic standards has attempted to solve this problem and provide a generic way to access the services that vendors might wish to offer. Further, with the advent of the Microsoft .NET technologies and web services, we move one step closer to heterogeneous interoperability on the Internet. The Internet and the World Wide Web are no longer massive collections of static content, but are capable of interaction and are entirely programmable. This in fact has been the cornerstone of Microsoft's new strategy and vision for the Internet.

As you can see from the above diagram, systems will now be built using a collection of services offered by heterogeneous vendors, exposed as XML based web services. In order to support this infrastructure, there is a need for a vendor to be able to publish the services they offer, and for a consumer of services to be able to search for the same, and once discovered, be able to seamlessly integrate with these services. Without this support, it will be very difficult for vendors to offer web services for loosely coupled systems. UDDI was developed as a solution to this problem.

UDDI

Universal Description, Discovery, and Integration, as the name suggests, is a protocol that allows publishers of web services to provide a description of their services, and that allows consumers of web services to discover them and integrate them with their own systems. UDDI is a specification for distributed web based information registries of web services. UDDI provides a Business Registry that stores the information that is registered by vendors and businesses about their products and services. This registry is distributed and replicated, and can be accessed by humans as well as programs that are written to the UDDI specifications.

The first component of this specification deals with the description of web services. UDDI allows each vendor or business to expose all the details about a web service that a consumer will need to access the service. This allows consumers to obtain a human readable version of this description, and at the same time, it allows a completely generic search program to obtain the same description. Similarly, UDDI allows searching of the registry to discover the required services. There is no specification or requirement on what a particular business service constitutes, since the needs of different businesses are widely disparate. UDDI simply allows the discovery of services based on specific parameters, it makes no assumptions about what these services might or might not be. Finally, once a service has been discovered, and the service is a consumable web service, UDDI offers a standard way of integrating this service. This is all based upon XML and SOAP.

The Microsoft UDDI SDK

Now that we have covered some of the basic fundamentals of UDDI, let us discuss how developers can actually use UDDI. UDDI offers a platform independent API that can be used to interface with the UDDI registry. This API is based on sending SOAP messages to the registry, with the XML payload containing the data that you need to access the registry with. The registry end point processes this request, and returns the response using the response XML payload in the SOAP message. Since all the interaction with the UDDI registry is based on SOAP and XML, developers are free to choose the language and tool of choice for building this interface component. Currently, Ariba, IBM and Microsoft all provide toolkits and SDKs that can be used to develop UDDI interfacing components. In this article, we will focus on the UDDI SDK that is provided by Microsoft. Microsoft provides a UDDI SDK for Visual Studio 6.0, and one for .NET. We will focus on using the .NET UDDI SDK in this article. You can download the Microsoft UDDI SDK v1.5.2 from http://www.microsoft.com/downloads/release.asp?ReleaseID=30880&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq. There is a patch available to this SDK at http://msdn.microsoft.com/vstudio/sp/vsnetb2sp2/default.asp?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq. Further, Microsoft provides a fully functional test site that can be used for testing your UDDI components at http://test.uddi.microsoft.com/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq Another interesting use of UDDI is demonstrated by the partnership between Microsoft and RealNames that allows UDDI searches directly from the address bar of IE. http://www.microsoft.com/presspass/press/2001/Jun01/06-14RealNamespr.asp?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq

Note that while we use the Microsoft UDDI SDK, you can very well handcraft your own using the SOAP toolkit. While this approach would offer more flexibility, you are better off using the UDDI SDK since all the development and testing is done for you.

The Microsoft UDDI SDK (we will refer to this simply as UDDI SDK in the rest of the article), provides objects that are capable of interacting with the UDDI registry. These objects construct and deconstruct the XML and SOAP messages that are required to interact with the UDDI registry. Further, these objects handle all the authentication and error handling for these requests and responses.

The UDDI API supports two broad categories of operations.

  • Inquiry API: This deals with retrieving information from the registry and performing searches on the registry.
  • Publishing API: This deals with updating information in the registry. This includes adding of new businesses or services under an existing business, or the update of existing data.

These functions can work either on the name of a business, or the interfaces (aka web services) that a business implements and exposes. Under the UDDI terminology, these web service interfaces are called tModels. A tModel is a UDDI data structure that represents a service. A tModel contains a name, description and an overview URL that contains more information about the service. A detailed response for a tModel is shown below.

<tModelDetail generic="1.0" operator=
"Microsoft Corporation" truncated="false" xmlns=
"urn:uddi-org:api">
<tModel authorizedName="0100200Z99" operator=
"www.foo.com/services/uddi" tModelKey=
"uuid:82224D38-2EFB-37A9-ACEA-EA091B59DDC5">
<name>
uddi-org:smtp
</name>
<description xml:lang="en">
some web service
</description>
<overviewDoc>
<description xml:lang="en">This tModel is used to describe a
Web service that performs function zzz.
</description>
<overviewURL>
http://www.uddi.org/specification.html
</overviewURL>
</overviewDoc>
<categoryBag>
<keyedReference keyName="tModelType" keyValue="transport"
tModelKey="uuid:C1ACF15C-8561-3393-8D60-28B675E51AB3"/>
</categoryBag>
</tModel>
</tModelDetail>

Most of the functions that the Inquiry API supports come in two flavors: one that takes in a business name, and one that takes in a tModel name. Given that the amount of data that could be returned by a simple search might be quite large, the Inquiry API offers three patterns that can be used for retrieving data.

  • Browse Pattern: This is the most common pattern used for hierarchical data retrieval. In this pattern, you start with the top level data that is only a summary of data. Starting from this summary data, you drill down and retrieve only the subset of data that you are interested in. This limits the data transferred to only the data that is of use. The UDDI SDK supports methods of the form find_xxx. These calls correspond to the Browse pattern.
  • Drill-down Pattern: Once you have identified the exact datum that you are interested in, you can use the drill-down pattern to retrieve all the details about this item. This typically returns the full registered information about the selected datum. The UDDI SDK supports methods of the form get_xxx. These calls correspond to the Drill-down pattern.
  • Invocation Pattern: In order to invoke a remote service that is registered within the UDDI registry, the application needs to obtain details about the remote service and how to interface with it. These invocation details form the invocation pattern.

Now that we have covered some of the basics behind the UDDI SDK, let's finally get to looking at some (C#) code! Note that in this article, we will use one of the samples that are installed when you install the Microsoft UDDI SDK. We have made some minor modifications to this sample for clarity.

Let us start with a very simple method that queries a registry to find all the businesses that contain a particular search string. In order to use the Microsoft UDDI SDK for .NET, you will need to install the SDK, and add a reference to the Microsoft.Uddi.Sdk.dll in your VS .NET project.

   Inquire.Url =
            "http://test.uddi.microsoft.com/inquire";
FindBusiness lFB = new
            FindBusiness();
lFB.Name = txtBusiness.Text;

BusinessList
            lBL = lFB.Send();

if (lBL.BusinessInfos.Count ==
            0)
tvData.Nodes.Add (new TreeNode ("No Businesses were
            found"));

foreach (BusinessInfo lBI in
            lBL.BusinessInfos)
tvData.Nodes.Add (new
            TreeNode(lBI.Name));

The above code segment shows how we use the Inquire, FindBusiness, BusinessList and BusinessInfo objects to obtain the names of the businesses that match our search criteria. We start off by specifying the URL property on the Inquire object, to point to the UDDI business registry that we will be using. Since we are developing a simple sample, we will use the Test UDDI Site that Microsoft provides. This is the URL to which all requests will be sent. Once this is done, we instantiate a new FindBusiness object and specify our search criteria, and use the BusinessList and BusinessInfo objects to iterate through all the returned business names and add them to a tree control. A tree control is very similar to the left pane of the Windows explorer, which allows for a hierarchical representation of data.

The above code segment generates a SOAP message to the UDDI registry, and the XML payload generated for a search for a business with the name Microsoft, is shown below.

<?xml version="1.0" encoding="UTF-8"?>
<Envelope xmlns="http://schemas.xmlsoap.org/soap/envelope/">
<Body>
<find_business generic="1.0" xmlns="urn:uddi-org:api">
<name>Microsoft</name>
</find_business>
</Body>
</Envelope>

Note that using the UDDI SDK encapsulates all the SOAP and XML details from the developer, and they do not need to know about the details of generating the request payload or interpreting the response payload.

Searching on a tModel is almost identical as you can see from the following code segment.

Inquire.Url = "http://test.uddi.microsoft.com/inquire";
FindTModel lFtM = new FindTModel();
lFtM.Name = txtBusiness.Text;

TModelList lTML = lFtM.Send();
tvData.Nodes.Clear();

if (lTML.TModelInfos.Count == 0)
tvData.Nodes.Add (new TreeNode ("No tModels were found"));

foreach (TModelInfo lTI in lTML.TModelInfos)
tvData.Nodes.Add (new TreeNode(lTI.Name));

Now that we have retrieved all the data that matches our search criteria, let us drill down upon this data and retrieve detailed information about a particular business. As with all hierarchical data, it is very difficult to determine the number of levels of data that will be present; and as a result, we drill down one level at a time. While drilling down for the Business objects, we can determine the descriptions and the binding signatures for each business service that we have found. These bindings provide information about invoking the service, such as the URL of the endpoint and the identifiers that are used for each service. Similarly, a tModel has a description and an overview document that describes the particular interface.

A binding has a unique key, an access point, one or more tModel instances, and an optional set of descriptions. A web service is represented by a binding. This binding has an access point, which is the URL to the web service (which is called the web service endpoint), and tModel instances which correspond to the WSDL interfaces that the web service implements.

A tModel instance has a tModel key, an overview, and an optional set of descriptions and initialization parameters. A web service has a tModel instance for each WSDL interface that it implements, with the tModel keys equal to the keys associated with each WSDL file. Thus, for a consumer to invoke a web service that is discovered using UDDI, the consumer needs to obtain the description of the web service using this WSDL file and then invoke the remote web service as described by the corresponding WSDL. An example binding is described later in this section where we describe how to publish a binding to the UDDI registry.

The code for obtaining the tModel details is shown below.

   GetTModelDetail lTMInfo =
            new GetTModelDetail();
lTMInfo.TModelKeys.Add
            (tvData.SelectedNode.Tag.ToString());
TModelDetail lTMD =
            lTMInfo.Send();

foreach (TModel lTM in
            lTMD.TModels){
TreeNode lNode = new TreeNode
("Descriptions("
            + lTM.Descriptions.Count + ")" );

foreach (Description lDesc
            in lTM.Descriptions)
lNode.Nodes.Add
            (lDesc.Text);

TreeNode lNode1 = new TreeNode ("Overview
            Document");
foreach (Description lDesc in
            lTM.OverviewDoc.Descriptions)
lNode1.Nodes.Add
            (lDesc.Text);

lNode1.Nodes.Add
            (lTM.OverviewDoc.OverviewURL);
tvData.SelectedNode.Nodes.Add
            (lNode);
tvData.SelectedNode.Nodes.Add (lNode1);
}

As you can see from the code above, obtaining the details for a single level of the hierarchical data is pretty straightforward. The GetTModelDetail object retrieves a list of TModelDetail objects for a particular tModel key. Once this list of tModel instances is retrieved, we can obtain the description and overview document information, if any, for each tModel. This is repeated for each level until we reach the last level of data.

Now that we have covered the discovery of services in detail, and the invocation of web services in brief, let us briefly discuss the process of publishing data in the UDDI registry. This process is known as Description since we will be publishing details about the business, the services and interfaces that it provides, into the UDDI registry.

   Publish.Url = "https:
            //test.uddi.microsoft.com/publish";
Publish.User = "testuser"; //
            use Passport account
Publish.Password = "testpwd"; // use
            Passport account

SaveBusiness lSB = new SaveBusiness
            ();
BusinessEntity lBE = new BusinessEntity ();
lBE.Name =
            "C#Today Sample Business Entity";
lBE.Descriptions.Add ("C#Today
            Sample Business Entity Description");

BusinessService lBS =
            new BusinessService ();
lBS.Name = "C#Today Sample Business
            Service";
lBS.Descriptions.Add ("C#Today Sample Business Service
            Description");

BindingTemplate lBT = new BindingTemplate
            ();
AccessPoint lAP = new AccessPoint
(URLTypeEnum.Https,
            "https://www.microsoft.com/test.asp");
lBT.AccessPoint =
            lAP;
lBT.Descriptions.Add ("C#Today Sample Binding Template
            Description");

TModelInstanceInfo lTMII = new
            TModelInstanceInfo ();
// use a GUID generator to generate this
            unique key
lTMII.TModelKey =
            "uuid:80372C89-F7EC-4ff6-B2A5-1422F98B82F8";

lBT.TModelInstanceDetail.TModelInstanceInfos.Add
            (lTMII);
lBS.BindingTemplates.Add
            (lBT);
lBE.BusinessServices.Add
            (lBS);
lSB.BusinessEntities.Add (lBE);

lSB.Send
            ();

The above code segment shows how we can publish our business entity to the UDDI registry. All we need to do is to create a service with the appropriate end point (called as AccessPoint, which we point to a non-existent site to begin with), and specify the descriptions and the tModelKey. Note that you need access to the UDDI registry and permissions to publish the data. This can be obtained by registering for a Microsoft Passport and then using those credentials in the above code block.

Sample Code

In the attached sample code for this article, you will find a sample on the Discovery aspect of UDDI. In this sample, we accept a search string, and use it to search for all matching business entities or tModels. The following screen shot shows a business search for the search string Microsoft. The code behind this form is identical to the code segments that we have discussed in the above section. Since the UDDI specification is still under development and review, currently we only demonstrate the Discovery mechanism; we will cover the publishing and invocation samples in a later article, as the specs and SDKs stabilize.

Conclusion

UDDI will play a very important role in promoting business-business exchanges, and for vendors and consumers to come together to access data in a very consistent way. Like all new technologies, UDDI is still evolving and maturing. Thus, you should take anything you read about this technology with a grain of salt, as the original developers of UDDI are still in the process of refining and tuning the standard.

Happy .NETing.

C#Today - © 2002 WroxPress Limited


Integrating an HTML returning C# Web Service with our Websites

Integrating an HTML returning C# Web Service with our Websites
http://www.csharptoday.com/content.asp?id=1658
Article Rating: 3.5
Published on September 21, 2001
Abstract
In this article, Andrew Krowczyk discusses a relatively simple but useful Web Service that fulfills the following scenario. Let’s say for example, Yahoo decided to write a Web Service that would allow anyone with a website to integrate a “Recent News?box into their own site. The data would be generated by Yahoo or its Content Provider, but would add the capability for anyone to encapsulate this piece of functionality into their own site seamlessly. As Andrew shows us, this is quite easy to do using .NET Web Services.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

In this article, we'll discuss a relatively simple but useful Web Service that fulfills the following scenario. Let's say for example, Yahoo decided to write a Web Service that would allow anyone with a web site to integrate a "Recent News" box into their own site. The data would be generated by Yahoo or its Content Provider, but would add the capability for anyone to encapsulate this piece of functionality into their site seamlessly. As you will see, this is quite easy to do using .NET Web Services.

Web Services in a Nutshell

A Web Service is an API to implement application communication over the internet. If you have an idea how DCOM (Distributed Component Object Model), or Remote Method Invocation works, then you basically know what a Web Service is trying to emulate. Unfortunately, communicating with such objects using the above methods is somewhat hard and limited. Typical implementation problems include network protocol issues across distributed private/public networks, limited development languages and tools that integrate with DCOM/RMI, and lack of standardized communication protocols. Web Services simplify this communication barrier significantly.

A Web Service at the simplest level uses SOAP and HTTP to make middleware/business objects available over HTTP and allow consumer applications the ability to invoke methods on these remote objects.

Prerequisites

In order to create and run the Web Service that we'll develop in this article, you need to have some familiarity or access to the following tools or ideas:

  • Some knowledge about .NET in general.
  • Familiarity of C# language syntax.
  • Access to the .NET Framework Beta 2.
  • Optionally have Visual Studio .NET installed. This will allow you to edit C# files. You can get by with a simple text editor if necessary.

The "Recent News" Web Service Architecture

The basic idea of our Recent News Web Service is to provide the capability for webmasters to integrate a news box with recent headlines into their web pages in the same way as Yahoo, by calling a method on a remote Web Service. As we can see in the architecture below, we will be building both an ASP.NET web page and a C# application that will consume the Recent News Web Service. The Web Service will simply return the appropriate headlines and stories from an XML file that will exist on the server machine hosting the Web Service. In a real live system, we assume that this XML file is automatically populated on a regular basis with up to date news information (for information on how to do this, see: http://www.asptoday.com/content/articles/20010622.asp?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq).

Building the Web Service

For this sample application, we will build a WebService that contains 2 functions. Both of them will return information about Recent News headlines. For simplicity's sake in our demonstration applications, we'll return data as preformatted HTML in one function and delimited text in the other function.

  • GetNews () - will return n article headlines and bodies in the following format. This is done so that the data can be easily parsed by the calling consumer application if need be.
  • GetNewsHTML () - will return n article headlines in preformatted HTML that resembles the Recent News box on Yahoo. We return HTML so that displaying the results of the Web Service is very easy. The output should look like the figure below.
  • In our sample application, we will be coding all of the Web Service functionality in C# code file that is referenced by the Web Service (.asmx file).

If we look at the file wsRecentNews.asmx, we will see the code in the Recent News Web Service that we are building.

using System;
using System.Collections;
using System.ComponentModel;
using System.Data;
using System.Diagnostics;
using System.Web;
using System.Web.Services;
using System.Xml ;
using System.IO ;

The first lines tell us the Web Service which namespaces to include as references for the code. We are including such namespaces as System.IO and System.XML. It is very important to include the System.Web.Services namespace as this is the library that we are deriving our Web Service from.

Next we declare a namespace that we will be using for the Web Service. For simplicity, we are calling this RecentNewsWebService.

namespace RecentNewsWebService
{

//insert our own namespace, replacing "http://tempuri.org"
[WebService(Namespace="http://yahoo.com/webservice/")]
public class RecentNews : WebService
{

Here we define our RecentNews class that derives from the WebService class, a member of System.Web.Services. The attribute above the class definition is very important. By default, all Web Services have a namespace of http://tempuri.org/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq, which is basically short for "temporary URI". Changing this default namespace to something more specific allows for a way to distinguish your Web Service from other Web Services that have been published.

We also need to make sure that all of the methods that we want to expose to the consumer applications of our Web Service are marked as public methods. They also need to be explicitly marked with the [WebMethod] attribute. This designates the method as an available WebMethod call. Without it, the method would be hidden to the outside consumer applications. Here we define 2 webmethods called GetNews and GetNewHTML. These methods take an integer that represents the number of headlines that are to be returned from the XML file. They then in turn give calls to getData, which is a private method that retrieves data from the XML file.

  [WebMethod]
public string
            GetNews(int iNumber)
{
return getData(iNumber,1)
            ;
}

[WebMethod]
public string GetNewsHTML(int
            iNumber)
{
return getData(iNumber,2) ;
}

The getData method does one of two things; it either returns the headline information in preformatted HTML, or just delimited text. As you can see in the code, we declare a FileStream object, which we use to open the recentnews.xml file on the web server. We also create a DataSet object to contain our XML "database". A call is made to the ReadXml method of the DataSet object to read in the XML schema and data information from our XML file.

private string getData(int iNumber,int iReturnMethod)
{
FileStream fin ;
string sOutPut = "";

//It is also to declaring "FileShare.ReadWrite"
//which will allow other users too to read this file
//Also the System Admins will have access to update the File.
fin= new FileStream(Server.MapPath("db/recentnews.xml"),
FileMode.Open, FileAccess.Read,FileShare.ReadWrite ) ;

DataSet ds = new DataSet() ;

//Read the XmlSchema and Data from the stream
ds.ReadXml(new StreamReader(fin)) ;

//close the stream
fin.Close() ;

After the XML data is read into the DataSet, we either build up HTML or delimited text output. I basically grabbed the shell table outline HTML from the Yahoo site and integrated it into this code. The for statement will add the correct number of table rows containing the headlines.

      if(iReturnMethod
            == 2)
{
//Format the output as HTML
sOutPut += "<table
            bgcolor=3366cc border=0 cellspacing=0 width=50>";
sOutPut +=
            "<tr><td>";
sOutPut += "<table bgcolor=ffffff
            cellpadding=2 cellspacing=0
border=0>";
sOutPut +=
            "<tr><td align=center bgcolor=e0d0b0 nowrap
            colspan=2>";
sOutPut += "<table border=0 cellspacing=0
            cellpadding=0 width=180>";
sOutPut += "<tr><td
            align=center><font face=arial size=2><b>In the
News</b></font></td></tr>";
sOutPut
            += "</table>";
sOutPut +=
            "</td></tr>";
sOutPut += "<tr><td
            colspan=2>";
sOutPut += "<table cellpadding=0 cellspacing=0
            border=0>";
sOutPut += "";

for(int i=0;i<iNumber;
            i++)
{
string headline =
            ds.Tables[0].Rows[i]["headline"].ToString() ;
sOutPut +=
            "<tr><td
            valign=top><b>&nbsp;&#183;&nbsp;</b></td>";
sOutPut
            += "<td width=100%><small><a
            href=http://www.yahoo.com>";
sOutPut += headline ;
sOutPut
            +=
            "</a></small></td></tr>";
}

sOutPut
            += "</table>";
sOutPut +=
            "</td></tr>";
sOutPut += "</table>";
sOutPut
            += "</td></tr>";
sOutPut +=
            "</table>";

}

If we aren't calling the GetNewsHTML method, then just build up a long string that is delimited in the correct fashion.

      else
{

//loop
            to the number of articles requested
for(int i=0;i<iNumber;
            i++)
{
string headline =
            ds.Tables[0].Rows[i]["headline"].ToString() ;
string body =
            ds.Tables[0].Rows[i]["body"].ToString() ;
sOutPut += headline +
            "@" + body + "*";
}
}

Now that we are done, we can just simply return the sOutPut string as the return string of the method call. At this point, sOutPut will contain either text or preformatted HTML output.

      //If no
            match found then return a error statement
return
            (sOutPut);
}
}

At this point, our Web Service is pretty much complete. As you have hopefully seen, writing a Web Service isn't that difficult. It is basically just a class with a few parameters and a little bit of code to distinguish it as a Web Service.

Deploy the Service

To deploy the Web Service, we need to set up a virtual directory in IIS (here called RecentNewsWebService). Copy wsRecentNews.asmx into the directory. Copy the RecentNews.xml to a directory called "db ".

To test the Web Service, open your Internet browser. Navigate to http://localhost/recentnewswebservice/wsRecentNews.asmx?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq. This will cause the Web Service runtime to automatically generate a test page for your Web Service. The test page displays information regarding the Web Service - the available web methods as well as information about each method and a way to test the method's functionality. If you take a look at the figure below, you will see a listing for the GetNews and GetNewsHTML methods. You can also see that they both take a parameter called iNumber and have return types of type string.

Consuming the Service

There are two ways to test or consume the Web Service that we just built. The service can simply be tested via the .asmx page as seen above. However it is most likely we want to integrate the call into our own application. In this article we'll examine two options - invoking the Web Service call from both an ASP.NET page and a C# console application. Before we can consume the Web Service, we need to discuss a few things such as learning the service description and building a proxy library.

The Service Description

Before an application can consume a Web Service, the client application must know what methods the Web Service supports and how exactly it can use those services. One neat feature of Web Services in .NET is the automatic generation of an XML formatted document in something called WSDL (Web Service Definition Language). This document specifies the information that the clients need to communicate with the Web Service. A simple way to generate this WSDL document is to call the Web Service file on the server with the following parameter: ?WSDL. So, in our example this would be retrieved by navigating to the following URL http://localhost/recentnewswebservice/wsRecentNews.asmx?WSDL&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq.

This call will return the following information in the web browser.

Build a Proxy

Another important point is how we can use Proxy libraries to call methods from a Web Service. Using a Proxy makes accessing a Web Service from a Windows type application very easy. Because a Proxy library abstracts away all of the underlying SOAP call implementation details, instantiating and making calls on the Web Service is as easy as it would be with a local COM object. The web proxy is a class that looks just like the Web Service, exposing the same function calls. The difference lies in the details. The internal implementations of the Web Service functions in the Proxy class consist of packing/unpacking the parameters of the method calls using SOAP to send to the real Web Service.

To create a Proxy library .dll for our Web Service we must use the .NET tool called WSDL.exe. Given our Web Service, this utility will create a Proxy .dll using the WSDL document. To generate the Proxy class, simply open up a command prompt. Type the following command:

wsdl http://localhost/recentnewswebservice/wsrecentnews.asmx?wsdl /n:RecentProxy /out:RecentNewsWebService.cs

As you might notice, there are three different command line flags that are being passed to the call to WSDL.

  • <URL> gives the location of the web page that the proxy is being created for,
  • /n:<namespace> indicates the name of the namespace to be used, and
  • /out:<name> tells the utility what the output file should be named.

The result of the above call is the generation of a C# file called RecentNewsWebService.cs in the same directory from which the command was run.

The resulting file consists of code that is derived from System.Web.Services.Protocols.SoapHttpClientProtocol. Although it's not important to intimately know what the code in the Proxy class does, it's interesting to take a quick look at the autogenerated code to see what's there:

//--------------------------------------------------------------------------//
            <autogenerated>
// This code was generated by a tool.
//
            Runtime Version: 1.0.2914.16
//
// Changes to this file may
            cause incorrect behavior and will be lost if
// the code is
            regenerated.
//
            </autogenerated>
//--------------------------------------------------------------------------//
// This source code was auto-generated by wsdl,
            Version=1.0.2914.16.
//
namespace RecentProxy {
using
            System.Diagnostics;
using System.Xml.Serialization;
using
            System;
using System.Web.Services.Protocols;
using
            System.Web.Services;

[System.Web.Services.WebServiceBindingAttribute(Name="RecentNewsSoap",
Namespace="http://yahoo.com/webservice/")]
public class
            RecentNews :
System.Web.Services.Protocols.SoapHttpClientProtocol
            {

[System.Diagnostics.DebuggerStepThroughAttribute()]
public
            RecentNews() {
this.Url =
"http://localhost/recentnewswebservice/wsrecentnews.asmx";
}

[System.Diagnostics.DebuggerStepThroughAttribute()]

[System.Web.Services.Protocols.SoapDocumentMethodAttribute(
"http://yahoo.com/webservice/GetNews",
RequestNamespace="http://yahoo.com/webservice/",
ResponseNamespace="http://yahoo.com/webservice/",
Use=System.Web.Services.Description.SoapBindingUse.Literal,
ParameterStyle=System.Web.Services.Protocols.SoapParameterStyle.Wrapped)]

public
            string GetNews(int iNumber) {
object[] results =
            this.Invoke("GetNews", new object[] {
iNumber});
return
            ((string)(results[0]));
}

[System.Diagnostics.DebuggerStepThroughAttribute()]
public
            System.IAsyncResult BeginGetNews(int iNumber,
System.AsyncCallback callback, object asyncState) {
return
            this.BeginInvoke("GetNews", new object[] {
iNumber}, callback,
            asyncState);
}

[System.Diagnostics.DebuggerStepThroughAttribute()]
public
            string EndGetNews(System.IAsyncResult asyncResult) {
object[]
            results = this.EndInvoke(asyncResult);
return
            ((string)(results[0]));
}

[System.Diagnostics.DebuggerStepThroughAttribute()]
[System.Web.Services.Protocols.SoapDocumentMethodAttribute("
http://yahoo.com/webservice/GetNewsHTML",
RequestNamespace="http://yahoo.com/webservice/
ResponseNamespace="http://yahoo.com/webservice/",
Use=System.Web.Services.Description.SoapBindingUse.Literal,
ParameterStyle=System.Web.Services.Protocols.SoapParameterStyle.Wrapped)]
public
            string GetNewsHTML(int iNumber) {
object[] results =
            this.Invoke("GetNewsHTML", new object[] {
iNumber});
return
            ((string)(results[0]));
}

[System.Diagnostics.DebuggerStepThroughAttribute()]
public
            System.IAsyncResult BeginGetNewsHTML(int iNumber,
System.AsyncCallback callback, object asyncState) {
return
            this.BeginInvoke("GetNewsHTML", new object[] {
iNumber},
            callback,
            asyncState);
}

[System.Diagnostics.DebuggerStepThroughAttribute()]
public
            string EndGetNewsHTML(System.IAsyncResult asyncResult) {
object[]
            results = this.EndInvoke(asyncResult);
return
            ((string)(results[0]));
}
}
}

After the Proxy class has been created, we can then use the C# compiler utility from the command line to compile the file into a .dll. To do this, we need to run the following command from a command prompt:

csc /target:library /r:System.dll;System.Web.dll,System.Web.Services.dll;System.Data.dll;System.Xml.dll RecentNewsWebService.cs

There is a .bat file in the download code that will execute both the Proxy class building step and the C# compilation step for you.

The ASP.NET Consumer

Consuming a Web Service from ASP.NET is relatively simple. For our Recent News example, we'd like to be able to call this from an ASP.NET page and pass it the number of headlines to return. More often than not, you'll probably just code in to retrieve a set number of headlines. To demonstrate how one would do this, we want to retrieve the HTML from the Web Service that will display the following Recent News image.

The next screenshot depicts the page show to the user.

After calling the Web Service.

Now that we see what we are building, lets take a look at the code. The following code snippets are from the RecentNewsClient.aspx page found in the downloadable code.

<%@ Import Namespace="System" %>
<%@ Import Namespace="RecentProxy" %>
<%@ Page Language="C#" %>
<html>
<head>

The top of the ASP.NET page imports 2 namespaces, System and RecentProxy. Depending on what we are accessing in our ASP page, you may or may not have to include other namespaces. The important namespace is the RecentProxy namespace. This is the name of the proxy namespace that we created when making the proxy .dll for the ASMX Web Service. If we don't include this namespace, we will not be able to access the functionality of the Web Service.

<script language="C#" runat="server">
private void getNews_Click(object sender, EventArgs e)
{

//Create a Instance of the Proxy Class
RecentNews rn = new RecentNews() ;

//Call the "GetNews" method of the Web Service on the Proxy Object
//The Proxy object inturn communicates to the Web Service
//Remember the Proxy cannot do anything except act as a bridge between
//your Web Service and the client. It cannot replace the Web Service.
int input = Convert.ToInt32(number.Text);
string result = rn.GetNewsHTML(input) ;

//Extract the Data from the Result
ArticleList.InnerHtml="<b>" + result.ToString() + "</b>" ;
}
</script>

The above code snippet is where the Web Service call gets made. We create an instance of the proxy class RecentNews. Once an object is instantiated, we can simply call the method GetNewsHTML, which will return the number of articles that we want in preformatted HTML. We need to cast the input text as an integer using the Convert.ToInt32 method. This needs to be done because the GetNewsHTML method expects an integer with the number of articles that need to be returned from the XML file. The last part of the above snippet simply sets the InnerHTML on the ArticleList <DIV> tag with the results of the Web Service call. The resulting output can be seen in the second screenshot above.

The rest of the code in the .aspx file is listed below.

<title>Recent News Service Client</title>
</head>
<body>
<center>
<form runat="server">
<table border="1" width="60%" cellpadding="1" cellspacing="2">
<tr>
<td colspan="2">
<b>Enter the number of articles to return.</b>
</th>
</tr>
<tr>
<td>
<asp:textbox id="number" runat="server" />
</td>
</tr>
<tr>
<td colspan="2">
<asp:Button onClick="getNews_Click" Text="Fetch !"
runat="server" />
</td>
</tr>
<tr>
<td colspan="2">
<div id="ArticleList" runat="server" />
</td>
</tr>
</table>
</form>
</center>
</body>
</html>

The C# Consumer

We are also including a C# consumer application, to see how easy it is to integrate the calls to the Web Service in a console application. These techniques can easily be applied to a WinForms application if so desired.

The first thing we need to do is create a .cs file. As noted in the ASP.NET consumer, we need to reference the Proxy class so that we can call methods on the Web Service. We'll use the using statement to reference the Proxy library.

using System ;
using RecentProxy ;

We reference the RecentProxy that we declared when compiling the Proxy. After we've included the namespace in our console application, using it is as simple as creating and instantiating the class and calling the GetNews method.

//A class which consumes the Web
            Service
public class RecentNewsTest {

public static void
            Main(string[] argv)
{
Console.WriteLine("Welcome to the Recent
            News Client") ;
Console.WriteLine("Stand by to retrieve 5
            articles...") ;
Console.WriteLine("");

//Return 5 articles
            for demonstration
int iNumber = 5;

//Create a instance of
            the Proxy Class
RecentNews rn = new RecentNews() ;

//Make
            a Call on the Web Service Method "GetNews" and
//pass the number
            of articles into it

string result = rn.GetNews(iNumber)
            ;

//now print out the results
Console.WriteLine(result)
            ;

Console.WriteLine("");
Console.WriteLine("Press Enter to
            Exit") ;
Console.ReadLine() ;
}

}

This console application simply writes out 5 article headlines and bodies to the console. The code creates a new instance of the Proxy Class RecentNews. Calling the GetNews method will return a string with the results in the raw string format (as opposed to HTML format in the GetNewsHTML call).

Building the sample console application that we just created is quite easy. Type the following command (included in BuildC#ClientApp.bat in the download code) at a command prompt:

csc /r:System.dll;System.Web.Services.dll;RecentNewsWebService.dll RecentNewsClient.cs

This command invokes the C# compiler and creates an output file named RecentNewsClient.exe. When we run the executable, a command console pops up and asks for the number of articles we wish returned. As you can see from the output screenshot below, the results are dumped to the console as a long string.

What's Included With This Article

The following files can be found in the downloadable code file. Below are descriptions of the package files. I described earlier in the article how to set up and call the Web Service from IIS.

Included Files

  • wsRecentNews.asmx - This is the Web Service itself.
  • RecentNewsClient.aspx - This is the ASP.NET consumer application for the Web Service.
  • RecentNewsClient.cs - This is the C# client consumer application for the Web Service.
  • RecentNews.xml - This is the XML file that contains the Headline and Story information that is to be returned by the Web Service.
  • BuildProxy.bat - Batch file that will create both the Proxy class and compile the Proxy .dll for the Web Service.
  • BuildC#ClientApp.bat - Batch file that will create the .cs (C#) console consumer application for the Web Service.

Conclusion

In this article, we've seen how easy it is to develop Web Services in .NET. We've also built a useful Web Service that can be used as a basis for further development and shows how integral Web Services may eventually become. As we move more and more towards distributed applications on an Internet wide scale, the abilities that .NET gives us to write services that interoperate smoothly between platforms will be revolutionary in the ability to share data and work together.

C#Today - © 2002 WroxPress Limited


Mobile Enabling Existing Applications with the Mobile Internet Toolkit

Mobile Enabling Existing Applications with the Mobile Internet Toolkit
http://www.csharptoday.com/content.asp?id=1568
Article Rating: 4
Published on September 5, 2001
Abstract
With every new technology wave one goes about building applications with new tools and frameworks. Quite a lot of development time and cost are spent on building applications with current technology. However protecting ones investment is paramount. This article is aimed at protecting your existing development investment ?by taking your existing applications and moving them forward in to the new wave of the wireless world. Sanjay Shetty shows us how to do this with the aid of managed COM and the Microsoft Mobile Internet Toolkit.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

With every new technology wave one goes about building applications with new tools and frameworks. Quite a lot of development time and cost are spent on building applications with current technology. However protecting ones investment is paramount. This article is aimed at protecting your existing development investment - it does so by taking your existing applications and moves them forward in to the new wave of the wireless world.

Most of us developing in the Windows world are familiar with COM and object oriented development; Many of our applications are built using COM. We'll begin by examining one such application - a banking application. We will extend it using two new frameworks; primarily Microsoft's Web Services model based on the .Net framework beta 2 and the Mobile Internet Toolkit beta 2.

Prerequisites

I assume in this article that you've had some experience with the .Net Framework Beta 2 and the Mobile Internet ToolKit Beta 2. I recommend that you set up the sample code present in source.zip at this stage, by following the instructions present in the setup.doc file. This will enable you to follow the article and understand the process more easily.

Once we've done the above, let's start by understanding what a web service is and then look at the existing application in more detail.

What is a Web Service?

As defined by Microsoft a web service is "A programmable application component accessible via standard Web protocols". In other words, it's a component that can be called remotely, over the Internet, from a client application, using standard web technologies such as XML and SOAP.

It's about making your application available in a manner in which existing or other applications, irrespective of the platform, may consume or use parts of our application.

Web services have been touted as the next evolution of the Internet, where applications are accessed over the web. E.g. Altavista's translation service or Microsoft's Passport authentication and wallet service. So in our applications, most of which require authentication services, why not use the Passport service developed by Microsoft? Another way to put it would be if we have an existing application then one can expose it in a standard manner such that other applications which one builds can use it over the web. Web services are all about the programmable web.

Current scenario:

Let's assume we are part of a bank and make information available to our customers via a web site in addition to ATM, which were present at a fixed location. So we've now provided our customers with the ability to view details about their bank accounts with us online. The figure below shows a standard login screen for such a bank.

A customer could perform a number of functions, like checking their account balance or view a statement online. Figure 2 shows the user after logging in, where the bank offers a choice of viewing the account balance or latest statement.

We've developed this application using COM components and ASP at the front end to serve online clients. The task in front of us is to now futureproof our current application. The first goal is to ensure that clients with WAP enabled mobile phones can view their account balances and their current bank statement. Essentially the business logic is the same but the presentation logic needs to be changed.

We have certain restrictions though:

  • A tight budget
  • An ever-changing device environment. Today the mobile phone, tomorrow the pocket pc, or a smart phone.

The solution is to use the web services model and the Microsoft Mobile Internet ToolKit.

Lets look at our current application in more detail. It represents a basic online web page where a bank allows their customers to view their bank balance and their bank statement online. This page is coded in traditional ASP. It makes a call to a COM component netdb.dll. This component has 3 methods

verifyUser - essentially accepts a user name and password and checks against a database for existence of a customers details.

  • getBalance - retrieves the current balance amount for a user using the account id.
  • getStatement - retrieves the current statement for a user from the transaction table again using the account id. This includes details of money withdrawn from an ATM or a check that has been deposited or cleared.

Our task is to take this current application and future proof it. Since we're building on the Microsoft platform and the future out there is the .NET Framework we're going to do everything on that framework.

Managed code

Our first task is to convert our existing COM component to a managed component on the .NET framework. We're not talking about recoding our component, but making it .NET usable. The process of converting a COM component to a managed creates metadata about the COM component so that the .NET framework can use it - that is all - we do not need to recode our component.

In order to do this, we use a utility available with the framework called the tlbimp.exe. tlbimp.exe is found in the Bin directory of the .NET Framework SDK installation directory. This utility generates the required metadata information by reading a COM type library. The output of this utility is a binary file that contains the metadata of our COM component. The following command line command enables creation of this wrapper:

tlbimp netdb.dll NETDBLib.dll

Tlbimp essentially creates a wrapper for our COM component making it usable on the .NET platform. We've called our wrapper NETDBLib.dll. There will now be a new namespace called NETDBLib with all the methods and classes of the COM DLL.

Once we've created a .NET compatible component, we need to expose this component as a web service. In the .NET framework a web service is essentially an .asmx file, which contains code enabling one to consume an application over the web programmatically using standard protocols such as SOAP (usually XML over HTTP). While using the .NET framework one need not bother about the internals of SOAP messaging and writing SOAP code as the .NET framework hides and does that dirty work for us.

Creating a Web Service

Let's now expose this managed component of ours into a web service via an .asmx file. In the Web Service code(.asmx) we define WebMethod() functions which act as an entry point for our Web Service clients. A WebMethod tag before your method names indicates to the framework that these methods are your web service methods, which you wish to make available to other consumers. In these methods you can invoke your COM component (through the .NET wrapper) and call the appropriate methods. The only things to consider are the return values. There is a limited range for return values in a .NET Web Service. For more information about return value types see http://www.aspng.com/quickstart/aspplus/doc/webservicetypes.aspx?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq.

Let's take a look at the web service code. As I mentioned earlier our COM component has three methods. The web service code will consist of methods with the same signatures. Check the code below - it has three methods; verifyUser, getBalance and getStatement, each accepting the same input and providing the same output as our COM component.

<%@ WebService Language="c#"
            Class="Bankcpp" %>

using System;
using
            System.Collections;
using System.Configuration;
using
            System.ComponentModel;
using System.Data;
using
            System.Diagnostics;
using System.Web;
using
            System.Web.Services;
using
            System.Web.Services.Protocols;
using System.IO;
using
            NETDBLib;

public class Bankcpp :
            System.Web.Services.WebService
{
CDataService ds=new
            CDataService();

[WebMethod]

public int
            verifyUser(string User,string Password)
{

return
            (ds.verifyUser(User,Password));
}

[WebMethod]

public
            string getBalance(int AccId)
{
return
            (ds.getBalance(AccId));
}

[WebMethod]

public string
            getStatement(int AccId)
{
return
            (ds.getStatement(AccId));
}
}

An .asmx web service file begins with <%@ WebService Language="c#" Class="Bankcpp" %> essentially these directives indicate to the framework that this is a web service coded in the language C# you could be using VB too, and the class which one is using is Bankcpp.

We reference our assembly at the top of the code with the using NETDBLib; command (this assembly should be present in a "bin " directory created at the same level in which our web service exists).

We then declare our class in the normal way and indicate that it inherits from the system web service class.

public class Bankcpp : System.Web.Services.WebService

We also create an instance of our class in the COM component and called it "ds"

CDataService ds=new CDataService();

Next, we declare our methods with a signature similar to those present in the COM component. Within each method we make a call to our instance of the managed code.

return (ds.verifyUser(User,Password));

For instance here we call the verifyUser method of the component and pass it the username and password as parameters.

Once we've created the asmx file we've essentially exposed our code and made it SOAP callable.

Open the bankcpp.asmx file that you'll find in the source code provided in your browser window. When the .NET framework finds a .asmx file containing the WebMethod directive it generates code associated so that our web service is exposed in the correct manner making it web callable. If you want to see the associated SOAP code defining this web service, you need to enter http://www.csharptoday.com/HYPERLINK%20view%20the%20SDLhttp://localhost/bankcpp/bankcpp.asmx?wsdl&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq to view the WSDL contract which other consumers need to follow when invoking the application you've just exposed. (Here I'm assuming you've hosted the application on your local server (localhost) following the instructions present in the source code file.

It would be a good idea to save this output as a .wsdl file, as we will be using it later. Save it as BankCpp.wsdl in the same directory as bankcpp.asmx.

Now we can use this functionality that we have exposed in our mobile application.

Review

Let's cover what we have achieved so far. In order to expose existing code as a web service we first made a .NET compatible wrapper for our COM component, then created a .asmx file(web service file), via which we exposed the managed code as a web service using the [WebMethod] directives.

Let's now build a front-end mobile application using the Microsoft Mobile Internet ToolKit and consume this web service we've created.

Consuming a web service

The .NET framework provides a utility called wsdl.exe, which takes a WSDL file as input and generates a proxy which can then be used by your client pages.

/language:VBwsdl /namespace:BankService http://localhost/bankcpp/bankcpp.asmx

This generates a source file called BankCpp.cs and specifies a namespace BankService that will contain all the proxy code.

Using the C# command line compiler, csc.exe, we can compile our C# source file:

Csc.exe /t:library BankCpp.cs

What we've done with the command line compiler is specify 2 things:

  • Create a library (dll) rather than an executable (exe) as indicated by the "library" keyword
  • Secondly specify the source file to compile, BankCpp.cs

The result is BankCpp.dll, a complete proxy to our BankCpp.asmx ASP.NET Web Service that we can now use in .NET applications to communicate via SOAP to our Web Service. Let's use this proxy to build Mobile Internet ToolKit based ASP.NET pages that consumes our Web Service.

Let's go Mobile

Mobile web pages are pretty simple to create. What we essentially need to do here in our code is reference the proxy which will make calls to our web service. If you recollect the way to reference a library in C# is via the "using Namespace " statement. In ASP.NET one uses the similar statement Import Namespace="namespace".

<%@ Page
            Inherits="System.Web.UI.MobileControls.MobilePage" Language="C#"
            %>
<%@ Register TagPrefix="Mobile"
            Namespace="System.Web.UI.MobileControls"
            Assembly="System.Web.Mobile"%>
<%@ Import
            Namespace="BankService"%>

The first two lines of any Mobile Internet ToolKit page are always the same, where one inherits from System.Web.UI.MobileControls.MobilePage and specify the TagPrefix of "Mobile " to indicate all the mobile controls on the page. In addition on Line 2 one specifies the Assembly System.Web.Mobile. When the .NET Framework compiles this page any control which has the tag Mobile:ControlName will be evaluated as a Mobile control belonging to the namespace System.Web.UI.MobileControls.

We then import the Bankcpp namespace that contains our web service.

Look at the code below, which is the login.aspx page:

<%@ Page
            Inherits="System.Web.UI.MobileControls.MobilePage" Language="C#"
            %>
<%@ Register TagPrefix="Mobile"
            Namespace="System.Web.UI.MobileControls"
            Assembly="System.Web.Mobile"%>
<%@ Import
            Namespace="BankService"%>

<script runat="server"
            language="c#">

protected void GoCommand_OnClick(Object
            sender, EventArgs e)
{
int AcctID;
try
{
Bankcpp
            DataService = new Bankcpp();
AcctID =
            DataService.verifyUser(AcctIDEdit.Text,
            PasswordEdit.Text);
}
catch(Exception excep)
{
AcctID =
            -1;
}
if (AcctID > 0 )
{

Session["AcctID"] =
            AcctID;

Response.Redirect("operation.aspx");
}
else
{
ErrorMessage.Text
            =AcctID.ToString() + "Not Logged
            on";
}

}
</script>

<Mobile:Form
            runat="server" >

<Mobile:Label
            runat="server">Account
            ID:</Mobile:Label>
<Mobile:TextBox runat="server"
            id="AcctIDEdit" />
<Mobile:Label runat="server">Enter
            Password:</Mobile:Label>
<Mobile:TextBox runat="server"
            id="PasswordEdit" Password="true" />
<Mobile:Command
            runat="server" OnClick="GoCommand_OnClick" Text="Go!"
            />
<Mobile:Label runat="server" id
            ="ErrorMessage"/>

</Mobile:Form>

The code immediately above this paragraph defines a mobile form, which allows the user to enter the Account ID and password. Refer to the figure below. When the user enters an appropriate user ID and password ( e.g. Account id "np001" and password "n") and then chooses the Options menu on the phone and presses the Go button, it runs the code attached to the OnClick of the same, which is GoCommand_OnClick. This function creates an instance of our proxy BankCpp and invokes the VerifyUser method with the parameters AcctIDEdit that contains the user's account ID and password.

If the account id is valid it allows the user to proceed to the next page, operation.aspx, where the user can then decide whether to check the bank balance or view the account statement. Source.zip contains a file called Account id and pass.txt that lists valid account id's and passwords.

Mobile Capabilities

The Mobile Internet ToolKit is built upon an extensible framework that provides support for a variety of devices, and future support for additional devices. These mobile devices may vary greatly in capabilities, such as screen resolution, color, and input devices available. These differences affect the way Mobile Web Forms pages are rendered.

To work with these differences, the Mobile Internet ToolKit includes an extensible Mobile Capabilities component, built upon the Browser Capabilities feature of ASP.NET. This component identifies the client device, collects together capability information from multiple sources, and makes it available to the Mobile Web Forms page and the mobile controls. This capability information is also available to your application, in case you want to alter your application logic for different devices.

You could also check login.aspx in a plain vanilla browser such as IE. The Mobile Internet ToolKit framework at the back end figures out the kind of device being used to browse the page and generates pages accordingly. The beauty of this is that these same pages can be viewed on a Pocket PC too without a single line of code change.

The figure below shows the same page (login.aspx) when viewed by a PocketPC. Without rewriting code one can now service users with different wireless devices.

There are a number of devices, which the Mobile Internet ToolKit fully supports:

  • Mitsubishi T250
  • Nokia 7110
  • Pocket PC with Microsoft Pocket Internet Explorer version 4.5

Microsoft has performed limited testing for the following devices:

  • Ericsson R380
  • Microsoft Internet Explorer 5.5
  • Microsoft Internet Explorer 6.0
  • Mitsubishi D502i
  • NEC N502i
  • Nokia 6210
  • Palm VIIx
  • Palm V
  • Panasonic P502i
  • RIM Blackberry 950
  • RIM Blackberry 957
  • Samsung 850
  • Siemens S-35i
  • Sprint Touchpoint phone

Microsoft has performed limited testing using the following device emulators:

  • Microsoft Mobile ExplorerT v 2.01 emulator
  • Pocket PC with Microsoft Pocket Internet Explorer version 4.0
  • Phone.com UP.SDK 4.1 emulator with generic skin
  • Phone.com UP.SDK 3.2 for WML emulator with Mitsubishi T250 skin
  • DoCoMo 502 emulator

The Mobile Internet ToolKit may support additional devices however they have not been tested.

Last words

We have seen the simplicity of the .NET framework in mobile enabling existing applications. At the end of this article we might ask - why not just consume our earlier original COM component via our mobile pages? It is true we could do that, but then we're not future proofing the application by building a solution on open protocols like SOAP, XML and HTTP, or enabling our application to be available to others in a programmable way, making way for new revenue streams which were earlier not conceivable. For example, we can now make available functionality within our application to others, like providing merchants with the ability to do a credit check against the bank balance of a user, to verify whether the user has sufficient balance to make payment for a particular service. Essentially once one has exposed functionality as a web service it can be easily integrated along with another service programmatically.

Another thing to notice is that we haven't used the Visual Studio.Net IDE at all. I've done this on purpose to make sure we understand what is happening beneath the hood, and one gets a better understanding of the subject matter.

You could view both the asp pages and the aspx pages associated in the article with standard browsers such as Internet Explorer. However the best way to check this is by downloading a mobile emulator such as the one from http://www.yourwap.com/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq and using the Pocket PC emulator provided along with the Microsoft embedded visual tools.

C#Today - © 2002 WroxPress Limited


Building Message Board Web Services with C# and .NET -- Part I, The Web Services

Building Message Board Web Services with C# and .NET -- Part I, The Web Services
http://www.csharptoday.com/content.asp?id=1569
Article Rating: 4.5
Published on September 4, 2001
Abstract
In this article, Chris Pike looks at how web services are constructed using C# and .NET. He does this by examining one of the most popular web applications, the message board. Amongst the topics looked at are the issues concerned with designing web service applications, and how to create the web service in Visual Studio .NET and C#.
Article

Introduction

It's difficult to avoid the popularity and hype that is surrounding web services. Each software vendor has some initiative concerning web services and there is (as always) great speculation about the future of the market for them. Whichever way it turns out, web service architectures provide a very different way of thinking about software development. From client-server to n-tier systems, to distributed computing, web service applications represent the culmination of each of these architectures in combination with the internet. Simple services like including the weather note from http://www.weather.com/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq on your web site were a precursor to today's web services technology. However, we still haven't realized the implications on having a standard way to discover, implement, and integrate web services. Imagine large distributed networks of services that you can use to develop your application so your applications become a conglomeration and integration of third party services, and of course, some of your own custom services.

The purpose of this article is to explain how web services are constructed using C# and .NET. To do this, we're going to examine one of the most popular web applications, the message board. First we're going to look into some issues concerning designing web service applications, and how they're different from client-server and n-tier designs. Then we'll look at creating the web services in Visual Studio .NET and C#. In Part 2 of this series, we'll build clients for these web services using ASP and ASP.NET - and maybe some other languages as well!

For this article I'll be using a SQL Server 2000 database, Visual Studio .NET Beta 2, the .NET Framework Beta 2, and Windows 2000 Professional. Now, let's get started!

Application Design

In order to begin building our message board web services, we're going to nail down some of the requirements. We should have services that provide the following functionality:

  • Add a new message board user
  • Login (verify) and existing message board user
  • Update a message board user record
  • Get a message and its hierarchy
  • Get a single message
  • Add a message to the board
  • Search the message board for some specific text

That's simple enough. Let's get to the database design.

Database Design

For this implementation, we're going to use two tables: MessageUsers and Messages . Notice the database diagram below. There is a one-to-many relationship in the messages table, so each message can be a parent message and have many child messages. This will enable our application to keep track of the replies for each message, and to represent each message and its children as a hierarchy. Here's the database design:

For performance reasons, we're going to wrap our database functionality into stored procedures. These procedures are listed below:

  • mb_AddMessage

CREATE PROCEDURE mb_AddMessage
@UserID int,
@ParentMessageID int = 0,
@MessageTitle varchar(100),
@MessageText ntext
AS

insert into Messages (UserID, ParentMessageID, MessageTitle, MessageText,
DateAdded, LastViewed)
values (@UserID, @ParentMessageID, @MessageTitle, @MessageText, GetDate(),
GetDate())

  • mb_AddUser

CREATE PROCEDURE mb_AddUser
@Login varchar(20),
@Password varchar(15),
@EMailAddress varchar(75)
AS

insert into MessageUsers (Login, Password, EMailAddress, DateAdded)
values (@Login, @Password, @EMailAddress, GetDate())

  • mb_GetMessage

CREATE PROCEDURE mb_GetMessage
@MessageID int
AS

select *
from Messages
where MessageID = @MessageID

  • mb_GetMessageList

CREATE PROCEDURE mb_GetMessageList
@ParentMessageID int
AS

select *
from Messages
where ParentMessageID = @ParentMessageID

Choosing the web services

There are interesting design considerations to note when building web service applications. First of all, we have to tackle the idea of "software as a service". Now, if you're accustomed to building client-server applications with fat clients, this can be a big change. Folks that are multi-tier application, and web developers, sometimes take for granted that everybody understands loosely coupled, distributed, thin-client architectures. This aspect is crucial to our understanding of software as a service. It really stems from an understanding of object-oriented programming. In most scenarios, we look at an object as an entity that has both properties (or attributes) and methods (or services). One of the interesting issues in implementing such an object as an MTS/COM+ component involves the marshalling of object properties across the wire. For performance reasons, the practice for building MTS/COM+ components has become to put all of the necessary parameters into the method declaration. This way only one wire trip is made to execute the method. This is similar to the web services architecture, in that we're not so much concerned with the properties as we are with the services, and every "property" has to be passed with the method call. This is significantly different from the way objects behave in fat client applications. In fat client applications you can normally access any of the properties of the object at any time without a performance penalty. This is because the object and the client are on the same machine, so there's no network round trip taking place just to set a property. You can hold on to those properties as long as you like. However, with web services, our web method has to know everything about what it needs to do each time it is called, and it can release all of those resources once it has generated the XML output.

Each of our web methods in the message board should be fully encapsulated and loosely coupled. Each method can be used independently of the other methods. Likewise, the web methods have to be completely stateless. We want to make these services as flexible and generic as possible since we're not really sure how our clients will consume them. We could assume that they'll use them for a message board, but we could theoretically market the addUser , userLogin , and updateUser methods as a set of user management services. Who knows? Maybe we can interface these services with Microsoft's PassPort to provide extra information to the user. As long as we keep our services flexible and easy to consume, there will be no restrictions as to what kind of application can use them. For now, however, we'll stick with the message board service.

For this application, we're going to create the following services:
  • addUser (string Login, string Password, string EmailAddress)
  • userLogin (string Login, string Password)
  • updateUser (int MessageUserID, string Login, string Password, string EmailAddress)
  • addMessage (int MessageUserID, int ParentMessageID, string Subject, string MessageText)
  • getMessageList (int ParentMessageID)
  • getMessage (int MessageID)

The addUser service will call the stored procedure mb_AddUser to add the user to the database. This service will return the new MessageUserID . The userLogin method will verify that the given user exists, and then return the MessageUserID . The updateUser method will update the user information based on the primary key MessageUserID . This function returns a string that contains a success or failure message. The addMessage method will call the mb_AddMessage stored procedure and add a message based on the given MessageUserID and ParentMessageID . If the ParentMessageID is not specified, then this message is not a reply to another message. This method will also return a string that describes if the message was added successfully. The getMessageList method will return the messages based on the ParentMessageID . The last method, getMessage will return a single message based on the specified MesssageID .

Creating the Project

You'll find that VS.NET does a great job of masking the complexity of web services implementation. Although we'll definitely need to know about all of what's going on "under the hood", VS.NET will get everything started for us. First, create a new project by going to the " File ", " New ", " Project " menu, shown below:

A Brief Overview of the files created by Visual Studio

Once you click " OK ", VS.NET accesses your web server and begins creating folders and files. Here's a listing and overview of the files that are created and added to the " MessageBoard " project folder in your web root:

  • AssemblyInfo.cs

This is the assembly file for our C# program. This file contains C# code that sets information about our program: product and company information, version information, and security information.

  • Global.asax

This file points to the Global.asax.cs file using the statement <%@ Application Codebehind="Global.asax.cs" %> .

  • Global.asax.cs

This is the C# file that contains the code for the web application. Here you have the methods that handle application and session events: Application_Start , Session_Start , Application_BeginRequest , Application_EndRequest , Session_End , Application_End . This is a hidden, dependent file of Global.asax .

  • Global.asax.resx

This is our application resource file. This is typically an XML file that stores resources in the following format:

<data name="test">
<value>first resource string</value>
<value>second<value>second resource string</value>
</data>

Of course, you can also store images and other object resources in the resource file as well. This file is hidden, and dependent on Global.asax.cs .

  • Licenses.licx

This file contains license information about the components used in the web service program.

  • MessageBoard.asmx

This file is similar to the Global.asax file in that it simply directs control to another file ( MessageBoard.asmx.cs ) using the CodeBehind attribute of the WebService tag as follows:

<%@ WebService Language="c#"
            Codebehind="MessageBoard.asmx.cs"
            Class="MessageBoard.MessageBoardService" %>

This file is also called the "web service entry point" and this is the file that a user would access to access the web service.

  • MessageBoard.asmx.cs

This file contains the C# code behind the web services. Here's where we'll be doing all of the implementation work. This file is a hidden, dependent file of MessageBoard.asmx .

  • MessageBoard.asmx.resx

This is another resource file for the MessageBoard web service file. This is a hidden, dependent file of MessageBoard.asmx.cs .

  • MessageBoard.csproj

This file is an XML file that basically stores all of the settings for the C# project. This file stores debug and production configuration information, project and files dependencies, as well as class references.

  • MessageBoard.csproj.webinfo

This file contains the path to the MessageBoard.csproj file.

  • MessageBoard.vsdisco

This file is an XML file that contains links to discovery information for the web services in the MessageBoard.asmx.cs file.

  • Web.config

This file stores web configuration information for ASP.NET.

Implementing the web services

Now that we understand a little more about what VS.NET is doing with all those files, we can begin implementing our web methods. The primary file we'll be working with is MessageBoard.asmx.cs . VS.NET gives us a framework file to start with, even a framework for a web method. Let's examine the framework method:

[WebMethod]
public string
            HelloWorld()
{
return "Hello World";
}

This method encapsulates our ever-wonderful "Hello World" as a web service. The primary line to notice here is the [WebMethod] attribute. This attribute notifies ASP.NET that this is a web method. This attribute is required for methods that will be used as web services. The [WebMethod] attribute has several properties that denote specific actions for the specified web method. These properties are:

  • BufferResponse
  • CacheDuration
  • Description
  • EnableSession
  • MessageName
  • TransactionOption
  • TypeId

The BufferResponse tells the processor that you wish to buffer the response of the web method. Normally, you'll want to set the BufferResponse property to true so that the entire response is constructed before it is sent back to the caller. This is similar to the Response.Buffer method in ASP. You can also set the CacheDuration for the method, which sets how long the cache should be held. Since we're not using session state, we can use the EnableSession property to turn off session tracking for our web method as well. For more information on the WebMethod attribute, see the MSDN documentation on the WebMethod attribute.

Data Access

There are several options for data access in .NET. Since we're going to be using SQL Server for our database, we're going to use the System.Data.SqlClient library. This library is written specifically for SQL Server, and provides the best performance when connecting to SQL Server databases. However, System.Data.SqlClient is only for SQL Server, so if we were planning to connect to any other data sources we'd want to use the System.Data.OleDb library. Just keep that in mind, that with System.Data.SqlClient you're tied to SQL Server. For the most part, the objects provided through ADO.NET are similar to those traditionally provided by ADO. To view an object reference for ADO.NET (which is the System.Data library), see the MSDN documentation for the System.Data library.

Each web method will make it's own connection to the database. So, we're going to define one set of objects (an SqlConnection and an SqlCommand object) across all of our web methods. We're also going to define a connection string for our database. Here are the declarations for the SqlConnection and SqlCommand objects that appear at the beginning of our web service:

private System.Data.SqlClient.SqlConnection sqlConn;
private System.Data.SqlClient.SqlCommand sqlCmd;

private const string CONNECTION_STRING =
"Server=cpike2;Database=Message;UserID=iusr_cpike2;Pwd=iusr_cpike2;
Trusted_Connection=true";

As with any object, at some point we've got to set it to a new instance. In the web service, a method is generated by VS.NET that is called InitializeComponent . The contents of this method are shown below:

private void
            InitializeComponent()
{
// Set SqlClient
            components
this.sqlConn = new
            System.Data.SqlClient.SqlConnection();
this.sqlCmd = new
            System.Data.SqlClient.SqlCommand();

}

So that we don't have to rewrite our database connection code in each web service method, we're going to construct a private method to encapsulate that action. The code for this method is below:

// OpenConnection - opens a connection to the SQL Server database
private string OpenConnection(System.Data.SqlClient.SqlConnection sqlConn, string sConnectionString)
{
sqlConn.ConnectionString = sConnectionString;

try
{
sqlConn.Open();
}
catch (System.Data.SqlClient.SqlException sqlExc)
{
throw(sqlExc);
}
}

Now that we have a data access object setup, and connecting and connection method built, we're ready to begin implementing our web service methods. For this article, we're going to detail the getMessageList method.

Making the Web Methods

As noted above in the design phase, the getMessageList method takes a ParentMessageID as a parameter and provides the list of child messages for that message. Going step-by-step through the getMessageList method, we first have to give the [WebMethod] attribute:

[WebMethod(Description=" This method gets the message list for a thread.",BufferResponse=true)]

For this method, we specify the Description and the BufferResponse properties. Setting the BufferResponse property verifies that the entire method reply will be sent all at once. The default setting for BufferResponse is true . The only time you might want to set it to false is when you are handling large amounts of data, but do some research regarding how SOAP acts when buffering is off (.NET automatically disables SOAP extensions for web service methods that are not buffered). Next we get to the method declaration:

public DataSet getMessageList(int iParentMessageID)

First, note that this method is declared public . For a method to be a web service, there are two specific requirements: the method must be public in scope, and the class must run inside an ASP.NET application. In examining the method declaration, you'll notice the method returns a DataSet object. This is one of the most impressive features of the .NET architecture. The DataSet object can be persisted straight to XML, and even better, the web services implementation in .NET handles that for you. Therefore, our web method will automatically return XML. Microsoft has made XML a very large and crucial part of the .NET Framework. After working with XML in previous versions of ASP, VB, and MTS/COM+, there were times when you'd have to build XML persistence into your objects on your own, especially when transferring objects across the wire as XML. VS.NET shields much of that complexity from the developer by encapsulating XML functionality into the objects themselves.

Next, we're going to set our SqlClient objects and build the SQL statement that calls the stored procedure. Here we're going to use the SqlDataAdapter object to bridge the gap between the System.Data.DataSet object and the System.Data.SqlClient.SqlCommand object. The SqlDataAdapter allows you to fill a DataSet object from an SqlCommand .

{
DataSet dstData = new DataSet();
System.Data.SqlClient.SqlDataAdapter adapter = new
System.Data.SqlClient.SqlDataAdapter();
string sSQL = "exec mb_GetMessageList @ParentMessageID = " +
iParentMessageID;

Now that our data access objects are set, it's time to send the query to the database. Using a try - catch block to catch any database connection and query execution problems, we connect to the database, execute the query, and fill the DataSet .

   try
{
this.OpenConnection(sqlConn,CONNECTION_STRING);

adapter.SelectCommand
            = new
            System.Data.SqlClient.SqlCommand(sSQL,
sqlConn);
adapter.Fill(dstData);

return(dstData);
}
catch(Exception
            e)
{
return(null);
}
finally
{
sqlConn.Close();
sqlCmd.Dispose();
}
}

Make special note of the finally block! The finally block executes regardless of the outcome of the try - catch block. This code will clean up our SqlClient objects before the web method ends.

Testing and Debugging

As a part of VS.NET, Microsoft has packaged an integrated testing and debugging mechanism for web services. Developers can set breakpoints in C# code that will stop executing during a web method. For those faithful web developers out there, integrated debugging is a welcome sight. Also, when you execute the web service project, you get an interface in the browser that allows you to choose and select a web method for testing. This is shown below:

Once you select a method, a form appears that allows you to enter the parameters and invoke the method. This page also displays the formats for requests to the method using SOAP, HTTP-GET, and HTTP-POST. This interface provides an excellent way to test and debug your web services. The form for the getMessageList method is shown below:

Now, let's look at the fruits of our labor. Here are the XML results of the getMessageList web service:

<?xml version="1.0" encoding="utf-8"?>
<DataSet xmlns="http://tempuri.org/">
<xsd:schema id="NewDataSet" targetNamespace="" xmlns=""
xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:msdata="urn:schemas-microsoft-com:xml-msdata">
<xsd:element name="NewDataSet" msdata:IsDataSet="true">
<xsd:complexType>
<xsd:choice maxOccurs="unbounded">
<xsd:element name="Table">
<xsd:complexType>
<xsd:sequence>
<xsd:element name="MessageID" type="xsd:int" minOccurs="0" />
<xsd:element name="ParentMessageID" type="xsd:int" minOccurs="0" />
<xsd:element name="UserID" type="xsd:int" minOccurs="0" />
<xsd:element name="MessageTitle" type="xsd:string" minOccurs="0" />
<xsd:element name="MessageText" type="xsd:string" minOccurs="0" />
<xsd:element name="DateAdded" type="xsd:dateTime" minOccurs="0" />
<xsd:element name="LastViewed" type="xsd:dateTime" minOccurs="0" />
</xsd:sequence>
</xsd:complexType>
</xsd:element>
</xsd:choice>
</xsd:complexType>
</xsd:element>
</xsd:schema>
<diffgr:diffgram xmlns:msdata="urn:schemas-microsoft-com:xml-msdata" xmlns:diffgr="urn:schemas-microsoft-com:xml-diffgram-v1">
<NewDataSet xmlns="">
<Table diffgr:id="Table1" msdata:rowOrder="0">
<MessageID>1</MessageID>
<ParentMessageID>1</ParentMessageID>
<UserID>1</UserID>
<MessageTitle>Test</MessageTitle>
<MessageText>Test</MessageText>
<DateAdded>2001-07-27T22:08:09.7400000-04:00</DateAdded>
</Table>
<Table diffgr:id="Table2" msdata:rowOrder="1">
<MessageID>2</MessageID>
<ParentMessageID>1</ParentMessageID>
<UserID>1</UserID>
<MessageTitle>RE: Test</MessageTitle>
<MessageText>Whatta Test!</MessageText>
<DateAdded>2001-07-27T22:08:57.6600000-04:00</DateAdded>
</Table>
</NewDataSet>
</diffgr:diffgram>
</DataSet>

The first part of this result is the schema that describes the persisted DataSet object. A <Table> tag represents each row, and a tag that contains the column name represents each column, in the form <ColumnName> . In Part 2, we'll put together some web services clients that will transform this information into some presentation-ready HTML!

Conclusion

Web services make an exciting new way to build distributed applications, and the possibilities for implementation are really yet to be realized. In this article we stepped through some of the issues when designing web service applications, and how to put together a simple set of message board web services in VS.NET. We didn't have the opportunity in this article to cover all of the intricacies of web services in C# and .NET, but this should be enough to get you started. Be sure not to miss Part 2 of this series, where we'll build the web service clients for our message board application and put together the final product.

C#Today - © 2002 WroxPress Limited


Creating a .NET Assembly Listener for Remoting

Creating a .NET Assembly Listener for Remoting
http://www.csharptoday.com/content.asp?id=1565
Article Rating: 4.5
Published on August 29, 2001
Abstract
A web service to most is a business object that is hosted on a web server and exposes a set of methods and instructions to use those methods. However some organizations see a “web?server such as IIS as a perceived risk and want to protect their business logic on an “application?server. In this article Tim Heuer examines a way around that problem, by introducing .NET remoting. Remoting introduces the same benefits as DCOM does with interprocess communication across computer boundaries, but enables us to allow our remoted objects to reside outside IIS and yet still service SOAP requests to a simple business component over HTTP.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

A web service to most is a business object that is hosted on a web server and exposes a set of methods and instructions to use those methods. In the Microsoft platform, we have typically seen examples of various web services created in ASP.NET and hosted in Internet Information Server (IIS). Therein lies a problem for some organizations.IIS. Some organizations see a "web" server as a perceived risk and want to protect their business logic on an "application" server. Some do not see IIS as an application server and would therefore not allow business logic to be hosted in that type of environment.

In this article we will examine a way around that problem, by introducing .NET remoting. Remoting introduces the same benefits as DCOM does with interprocess communication across computer boundaries. The remoting concepts in the .NET Framework provide the application developer with a host of flexible options for exposing business logic to other layers of the application. Let's take a look at how we can overcome the obstacle of not allowing our remoted objects reside in IIS and yet still service SOAP requests to a simple business component over HTTP.

It is important to note that the example of organizations not allowing a "web" server to act as an "application" server is just an example, and not the only use for remoting. Remoting should be examined when needed, keeping in mind best practices (see links below for Remoting architecture reference documentation).

.NET Remoting Step by Step

The first step in examining remoting is to obviously have something to remote. Again, in our example this is a simple business object that needs to be exposed to another application layer, but cannot reside as an IIS hosted web service.

Let's first create the business component.

Creating the Assembly

We will create a simple business object that performs two types of functions: a simple string manipulation and a simple math function.

We create our assembly by creating a new C# Class which will house our string manipulation logic. First we include any using statements we may need to make our programming a little easier. For our components the only one we require is the System namespace of the framework. Add a using statement as follows:

using System;

Next as any good assembly should have we want to ensure we include a namespace declaration to distinguish which namespace this object operates under.

namespace com.timheuer.development.remoting

For our string manipulation function we are going to create a simple method that returns a string indicating "Hello " and the parameter passed to it. We will also provide an overloaded method to include a greeting other than "Hello" in the event someone would want to say something different.

Our string manipulation class StringFunctions.cs should look like this:

using System;

namespace com.timheuer.development.remoting
{
public class StringFunctions
{
public string SayHello(string strName)
{
return string.Concat("Hello ",strName);
}

public string SayHello(string strName, string strGreeting)
{
return string.Concat(strGreeting,strName);
}
}
}

Similarly, let's create a math class MathFunctions.cs that implements a simple addition function:

using System;

namespace com.timheuer.development.remoting
{
public class MathFunctions
{
public int AddNumbers(int intNumber1, int intNumber2)
{
int returnNumber = 0;
returnNumber = intNumber1 + intNumber2;

return returnNumber;
}
}
}

Now we have an assembly with essentially three different methods that can be executed. This assembly is now ready to be consumed within a client application and will return the appropriate values upon being called.

Our assembly is not, however, ready to be consumed by a client needing to access it remotely.

Our first step in allowing our component to be accessed remotely is to inherit MarshalByRefObject. This will provide the base class we require in order to access our assembly remotely over the wire.

Our classes now should look like this:

using System;

namespace com.timheuer.development.remoting
{
public class MathFunctions : MarshalByRefObject
{
public int AddNumbers(int intNumber1, int intNumber2)
{
int returnNumber = 0;
returnNumber = intNumber1 + intNumber2;

return returnNumber;
}
}
}

The string manipulation class would have the same MarshallByRefObject inheritance in it as well:

using System;

namespace com.timheuer.development.remoting
{
public class StringFunctions : MarshalByRefObject
{
.
}
}

Our assembly is now ready to be hosted in a remoted environment. The next step is to create that environment to host and expose that business logic. We can compile this assembly now into RemotedObject.dll and have it prepared for use. Since Visual Studio.NET Beta 2 is more of a stable build and is now out more to the masses, I recommend using the tool provided for the platform. Compiling assemblies and developing using great features such as IntelliSense and AutoComplete statements making developing in Visual Studio.NET easier. Of course you could compile from a command line as well. In that case you would want to familiarize yourself with the command line options. In our above example, we'd compile the RemotedObject as follows:

csc /out:RemotedObject.dll /t:library MathFunctions.cs StringFunctions.cs /r:System.dll

Creating the Server

The server to host this newly compiled business logic must now be created. Initially let's examine the namespaces we will need to perform this task. The following framework assemblies and namespaces will be used in creating the server. Some may not make sense initially, but we will list them all here for now.

System
System.IO
System.Runtime.Remoting
System.Runtime.Remoting.Channels
System.Runtime.Remoting.Channels.HTTP
System.Runtime.Remoting.Channels.TCP
System.Windows.Forms
System.ServiceProcess

Before we continue, let's look at some of the terms we will be using to create our server. Channels are used to transport messages to and from remoted objects. Requests are routed through these channels and the responses are routed back the same way. At least one channel must be registered with the remoting service before any remote object can be called. You can register the same channel listening on different ports, but not two channels on the same port.

An HTTP Channel is a transport for messages that uses the SOAP protocol as default. All messages are passed through the SOAP formatter, where the message is serialized into XML and all necessary SOAP headers are appended. Conversely, a TCP Channel uses a binary formatter by default to serialize messages to a binary stream and transport the stream to the target URI. We can, however mix and match these channels and protocols to best meet our needs.

Let's first assume we are building a console application that will host our remoted object. We must first register the channel. This channel is essentially the protocol and location of how a remote layer will access our logic. Within Main[]we must first create and register the channel.

HttpChannel httpchnl = new HttpChannel(8080); //the 8080 denotes what port to run on
ChannelServices.RegisterChannel(httpchnl);

Once the channel is registered and available, we need to let our channel and remoting component know about our business component we want to expose.

RemotingServices.RegisterWellKnownType("RemotedObject",
"RemotedObject.MathFunctions", "MathFunctions.rem", WellKnownObjectMode.SingleCall);

We repeat this instruction for our StringFunctions class as well. Our assembly is now registered via an HTTP channel on port 8080 and ready for use. Notice in the third parameter we created a MathFunctions.rem parameter. This is how this particular service will be accessed when called from the client. In our example in the HTTP channel, this is a portion of the URL. Our remoting service can now be compiled and started. Once started it is available and waiting for requests. However, this does not give us any flexibility on which ports or channels to remote our object on. Likewise, when a new assembly is desired to be remoted using this remoting service, it requires us to modify our console application, recompile and restart the service. For some this may be acceptable, but for most we desire a little more flexibility in configuring our remoting service. Let us see how to do this.

Flexible Configuration

The benefit of remoting in the .NET Framework is it allows for the use of configuration files all over the place. We see this in such places as machine.config and web.config. These files are very helpful in placing configuration specifics in an easily readable format that can be read easily by our application. A more important benefit is that by using configuration files, we allow our application to have a longer compiled lifetime as we can simply change the configuration settings in these files to meet our needs in most cases. I believe these configuration files to be a good thing and an improvement over past mechanisms of configuration settings like utilizing the registry. It can, however, get confusing and it is important to understand the sequence at which configuration files get applied. Typically the machine.config settings get applied first and then it sequences "down" the line. As a general rule of thumb, a web.config file will apply settings at the root application in which it resides. The lowest level of configuration gets applied when the config file is specific for the application. For example timheuer.exe.config would apply settings for timheuer.exe in the same directory.

This is perfect in our example of our remoting service. In most cases we want to ensure flexibility in our service and be able to add and/or remove remoted components with ease. For this let's create our own configuration file called remotingservice.config and place our settings in this. The .NET remoting framework provides a definition for actions we can perform in a configuration file used specifically for RemotingServices (see link below regarding file format). In our example, we want to make sure we can register our channel and our service can be remoted. These two actions are simple in our configuration file. Let us first create the base for our configuration file:

<configuration>
<system.runtime.remoting>
<application name="Remoting Service">

This XML configuration format provides the basis for us to perform our required actions. First let's add the information required to register our channel. To do this we add a channels node and a child node for each channel we wish to register.

<channels>
<channel type="System.Runtime.Remoting.Channels.Http.HttpServerChannel,
System.Runtime.Remoting" port="8080"/>
</channels>

If we wanted to add more channels we would do that here. That's it! Next we must register our services, which we wish to be hosted on these channels and in this service. We do this by creating a service node and place our wellknown child nodes within it.

<service>
<wellknown mode="Singleton" type="com.timheuer.development.remoting.MathFunctions,
RemotedObject" objectUri="MathFunctions.rem"/>
</service>

We also add a wellknown node for our string class as well here. Our configuration file is now complete and ready for use. The objectUri attribute will provide the resource identifier for the application where it will be hosted. The mode we choose is Singleton. Singletons are useful when resources can be shared across messages, meaning all messages are sent to the same object instance.

We can now change our console application to read our configuration file and no longer depend on compiled lines of code registering specific channels and services within the code itself. We change our console application to read our configuration file as follows:

RemotingConfiguration.Configure("remotingservice.config");

The remoting service now reads the configuration file to configure all channels and services within it and perform the appropriate actions on those directives. We can compile our console application and allow it to run, awaiting new requests. Our compilation options do not change for this new application as the configuration namespaces all reside within the overall Remoting namespace.

Making the Server a Service

Some organizations and application developers would like even greater flexibility in a remoting host application. The ideal situation for this would be to not have this application be a console application, but rather a Windows service. The .NET Framework now provides an easier mechanism for creating and deploying Windows services.

Utilizing the Visual Studio.NET IDE, a template is provided which provides all the necessary areas required for a base Windows service.

After creating the template, we only need to add our commands in the OnStart function as well as ensure our service has all the appropriate naming we would like to identify it with. We can also perform actions within the OnPause, OnContinue, and OnStop functions by overriding these methods with our own.

As mentioned earlier, if you name your configuration file the same as your compiled application (with the exception of the .config extension), your application can access it using the CongifurationSettings namespace of .NET. We could have chosen to do that here, but are showing a different method for completeness. Our new Windows service now assumes that a file called remotingservice.config resides in the same path as the service itself (hence the reference to System.Windows.Forms to get the application startup path easily). Additionally in our application, we create even greater flexibility for the service, by enabling it to have a startup parameter on the service to override the default configuration file and look at a configuration file in the path provided. This allows administrators to test the configuration files possibly prior to overwriting old ones.

The end result of the Windows service OnStart function should look like this:

protected override void OnStart(string[] args)
{
try
{
string strCurrentDir = Application.StartupPath;
string strServerConfig = String.Concat(strCurrentDir,"\\remotingservice.config");

//check if any parameters have been passed in
if (args.Length != 0)
{
//optional config file specified
strServerConfig = args[0].ToString();
EventLog.WriteEntry("Remoting Listener",
"Remoting Listener was started using optional parameter: " +
strServerConfig + ".",EventLogEntryType.Information);
}

//load the configuration file
RemotingConfiguration.Configure(strServerConfig);

}
catch (Exception e)
{
EventLog.WriteEntry("Remoting Listener",
"Remoting Listener encountered an error: " +
e.Message,EventLogEntryType.Error);
}

}

The second part to creating a Windows service is to also include the appropriate installer classes to ensure the service will install as a Windows service and register appropriately. Again, the good thing is that Visual Studio.NET provides a template for this as well.

We simply need to include this template in our project and add our specific parameters. For our project I added the following:

private ServiceProcessInstaller
            processInstaller = new ServiceProcessInstaller();
private
            ServiceInstaller serviceInstaller = new
            ServiceInstaller();

public
            RemotingServiceInstaller()
{
// This call is required by the
            Designer.
InitializeComponent();

processInstaller.Account
            = ServiceAccount.LocalSystem;

serviceInstaller.ServiceName =
            "Remoting Listener";
serviceInstaller.StartType =
            ServiceStartMode.Manual;
serviceInstaller.DisplayName = "Remoting
            Listener";

Installers.Add(serviceInstaller);
Installers.Add(processInstaller);

}

Our Windows service is now ready to be compiled and installed. After compiling the service, we install the service by using the installutil application provided by the .NET Framework. After ensuring your configuration file is complete and application compiled, from a command line you would perform the following command:

(yourpathtoyourapplication):>installutil RemotingListener.exe

The output of this command should be similar to:

Your service should now be installed and ready to be started. To verify the service was installed correctly, you can check the Services applet and see it installed:

Once started (in our example) the configuration file will be read and the service is now ready to handle remote call requests.

A few things to remember regarding our service example (which can be changed by you of course to your needs):

  • Our service assumes it is in the same place as the business objects (i.e., RemotedObject.dll must be in same directory as RemotingListener.exe)
  • The remotingservice.config file must be located in the same place as the service executable (unless a startup parameter is used)

Consuming the Remoting Assembly

Throughout the example I've been talking about that the "service is now available to receive requests" but have not explained what that meant. To be consistent, let's still assume we are using an HTTP channel running on port 8080. Some may also read this and wonder why we are using this as HTTP is synonymous with IIS and we mentioned at the beginning of the article that we would not use IIS to host the service. This is still correct. The sample we have created runs on a machine without IIS even installed. HTTP does not assume that IIS is the host for all HTTP requests. Remember, HTTP is a protocol for communication over TCP/IP - you can listen for and respond to HTTP communication requests without a "web" server.

Nevertheless, let's examine way we can consume this remoted object. Our remoted object is running on our server under port 8080 on an HTTP protocol. When the service is running, we should be able to see its definition by accessing one of our registered services. Remember in the configuration file when we set the objectUri parameter to be StringFunctions.rem ? This is where that takes place. In order to see the definition of our business component, we can simple point to:

http://localhost:8080/MathFunctions.rem?WSDL&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq

Note when you see this, you are looking at generated Web Service Descriptor Language (WSDL). Here is a sample of what would be generated:

The WSDL generated provides a definition to what methods are exposed and their required parameters, returns etc. The best thing about it is that remoting took care of this for us! Because we now have a complete definition of our remoted objects and associated methods, we can use them in several ways.

To consume them we have several methods of consuming these objects.

Using WSDL

As we just mentioned, a complete WSDL document is generated and can be distributed to anyone needing to consume the service. This is great for those clients who may not be on a Microsoft platform. For example, you could distribute this new WSDL document to a java developer in your organization and they should be able to consume it given the definition. WSDL provides a complete definition of all necessary attributes, data types, execution methods, namespaces, etc. required for an external application to consume the web service. Given these definitions any application conceivable could consume the web service by creating their own custom proxy based on the location information and definitions provided.

Using Visual Studio.NET

Visual Studio.NET provides an extremely easy mechanism for accessing remoted objects. Remember the earlier statement of how every remoted object is a web service - here is a good example of this. In VS.NET when you need to consume a web service you add a Web Reference. The same is true for our example of a remoted object.

Within the IDE, we can specify we want to add a Web Reference and point to the URL mentioned above:

Note that VS.NET retrieves the WSDL and gives us the option of adding that reference. Once added we can use the remoted object as if it was an assembly we developed locally:

Notice the "Web References " folder in our Solution Explorer now has a reference to that object.

Using SOAPSUDS

Another method of consuming this service is to deploy a proxy object similar to how COM+ would have done for us. It is important to note that when adding a Web Reference in VS.NET that is exactly what happens - a proxy object is created for use.

A utility called SOAPSUDS can be used to create a proxy object for the remoted object. To use SOAPSUDS, we simply execute from a command line the following command:

soapsuds -url:http://localhost:8080/MathFunctions.rem?WSDL -gc

The "gc " switch command tells SOAPSUDS to generate code for the proxy. The result of the above command will generate a file called RemotedObject.cs which looks like this:

namespace com.timheuer.development.remoting
            {
using System;
using
            System.Runtime.Remoting.Messaging;
using
            System.Runtime.Remoting.Metadata;
using
            System.Runtime.Remoting.Metadata.W3cXsd2001;

[SoapType(XmlNamespace="http://schemas.microsoft.com/clr/nsassem/
com.timheuer.development.remoting/RemotedObject",
XmlTypeNamespace="http://schemas.microsoft.com/clr/nsassem/
com.timheuer.development.remoting/RemotedObject")]
public
            class MathFunctions :
            System.Runtime.Remoting.Services.RemotingClientProxy
{
//
            Constructor
public
            MathFunctions()
{
base.ConfigureProxy(this.GetType(),
            "http://12.28.151.17:8080/MathFunctions.rem");
}
// Class
            Methods
[SoapMethod(SoapAction="http://schemas.microsoft.com/clr/nsassem/
com.timheuer.development.remoting.MathFunctions/RemotedObject#AddNumbers")]
public
            Int32 AddNumbers(Int32 intNumber1, Int32 intNumber2)
{
return
            ((MathFunctions) _tp).AddNumbers(intNumber1,
            intNumber2);
}

}
}

Alternatively we could have also provided an "oa " switch command to compile the source file into an assembly for us to use. We could now distribute this proxy to those wanting to consume it for use. .NET also provides other tools for creating proxy code and WSDL documents such as the WSDL.EXE utility. Soapsuds is in many cases the more valuable as it will actually generate proxy code for us as well as the wsdl file.

A New Web Service Proxy Wizard

Microsoft has just released another mechanism of creating these proxy objects. The Web Service Proxy wizard is implemented as an "ATL Object Wizard" within Microsoft Visual C++? It provides a GUI interface to providing the necessary parameters for creating the proxy object. All these methods of creating proxy objects will be covered in another article.

Conclusion

The result of our examination here is that it is possible to create a distributed application in .NET with the help of .NET Remoting and web services. It is also important to note that our example lives on an "application" server with nothing else than the .NET Framework - IIS isn't even present regardless of our use of the HTTP channel.

Additionally, we focused on using HTTP as a channel. We could have easily created a TCP channel to host the application. Running TCP with binary transport will increase performance of the transmission and application, but may not be available in all situations due to crossing platforms and domain boundaries. By default the HTTP channel uses SOAP, but we could have configured it to use binary transport. Likewise with TCP which can use SOAP. As a matter of fact you may want to have multiple channels in your application for different consumers of the object. With the use of the configuration file, the Windows service could be stopped (or paused depending on your particular implementation of the service), the configuration file could add another channel of TCP type running on a different port, and then the service could be re-started.registering the new channel! Our MathFunctions class is now accessible on both the HTTP channel as well as a TCP channel at the same time.

Of course we only examined a simple example of strings and adding numbers, but the theory applies to any object needing remoting for any reason. Again, I mentioned in our example that our reason was for a requirement of a physical separation of the layers, but there may be many instances where remoting objects in this manner is desirable to your application or another web service you may want to expose.

The important thing to remember and implement from this article is to provide your remoting host with the flexibility of using configuration files so as not to have a large interrupt in service. Implementing this concept as a Windows service also provides additional monitoring capabilities that you may have had to create yourself should your implementation been different.

C#Today - © 2002 WroxPress Limited


.NET Services

.NET Services
http://www.csharptoday.com/content.asp?id=1564
Article Rating: 4.5
Published on August 28, 2001
Abstract
In this article, Rob Schieber, looks at .NET services. He writes and implements a Windows Service agent that writes custom entries to the event log using Microsoft's C# NET.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

If you have ever tried to write an NT service prior to the Visual Studio .net beta, I'm sure you will agree that it can be very difficult. Before Visual Studio .Net, C or C++ was the only way that one could write a reliable NT service. This required a painstaking amount of coding as well in-depth knowledge of the Windows NT Service architecture. With Visual Studio.Net, you'll find that Microsoft has greatly simplified this task, making the ordeal of writing a service much clearer and straightforward. In this article I will explain how to create a service application using Microsoft's Visual Studio.Net beta.

Requirements

This case study uses Microsoft's C#.NET running on Microsoft Visual Studio Beta 1.

Services Overview

A service is essentially an executable without a user interface that runs for indefinite, usually long periods of time. You can start a service manually, or set it to start and stop with the operating system. Windows services can be controlled in a number of ways. You can access and control windows services through the Service Control Manager, programmatically, or through the command line.

Reasons To Use a Windows Service

Using a windows service is definitely not appropriate for every circumstance, but under certain conditions they can be extremely useful. Below are some good indicators of whether or not a Windows Service might be useful.

  • If you need to run a long running process without an interactive user (i.e., without a logged on user) such as on a web server.
  • When object initialization is time consuming and the object needs to be in a waiting state for a client.
  • Your object needs a persistent state and does not care if a client is connected to it.

Common Windows Services Usage Patterns

The decision to use a Windows Service is rarely a black and white decision, but some situations are definitely more appropriate than others. A more useful way of making the decision to use a service is to try to fit your objective into a usage pattern, or common scenario where your service might be a good fit. Below are a few of the more common usage patterns for a windows service:

Monitor (or Listener)

The Monitor can be used to monitor the status of system services, resources, or even clients.

Agent

The Agent does background work that needs to get done without a client user. It can also do intensive processing work to prepare resources in advance of an incoming client.

Quartermaster (or Resource Pooler)

The Quartermaster hands out pre-initialized resources to clients.

Switchboard

The Switchboard makes the location of resources irrelevant to the client. The client contacts the switchboard for resources, and the switchboard connects the client to the proper resource.

Business Object

An 'Industrial Strength' Business Object differs from a typical in-process COM DLL in that it runs inside a service, providing a combination of rules, data and calculating power upon request by a client. Business Objects are appropriate for services when you need intensive calculation strength, fault isolation, or when the Business Object has intensive pre-initialization work to do before it can get started.

Above are the five most common design patterns for Windows Services. For this article we will create an Agent type object that will write to the event log periodically.

Creating a New Windows Service

As I said before, creating a windows service in .NET is not too difficult. Let's begin by opening the Visual Studio.NET IDE. Select New , and then Project. from the File menu. The New Project dialog box should appear with a list of templates to choose from. Select the Empty Project option and call this project " ServiceProject " and click OK . The directory I will be using for this article is " C:\ServiceApps ".

Note: Although we do have the option to create a new project using the "Windows Service" template, lets ignore this and create an Empty Project. This will allow me to better explain the basics of creating a windows service.

Writing a Service Programmatically

To write a service programmatically, we will need to satisfy the following requirements:

  • Build a class that derives from the ServiceBase abstract base class.
  • Load and run the service in our main () function.
  • Initialize the class using the initialize () function.
  • Create an installer class to install the service.

To create our service we need to access the System.ServiceProcess namespace, which is not included by default. To add this reference, select Project from the main menu, then Add Reference . Scroll down to the " System.ServiceProcess.dll " component, then click Select , then click OK . We should now be able to use the ServiceProcess members.

To add our C# class, right click ServiceProject in the Solution Explorer window, then select Add , then Add New Item . Select the C# Class icon, then change the name from class1.cs to SimpleService.cs and click Open .

After you add the class, the C# IDE should construct an empty class for us that we can use to write our service.

We can now begin coding our service. Modify the code in the SimpleService class file to look like the code below:

namespace ServiceProject
{
using
            System;
using System.ServiceProcess;
using
            System.Diagnostics;

/// <summary>
/// This class
            will create a simple service that writes to the event log every 10
            seconds.
/// </summary>
public class SimpleService :
            System.ServiceProcess.ServiceBase
{
//TODO: add Initialize()
            method

public SimpleService()
{
//
// TODO: Add
            Class implementation
//
}

//TODO: add main method to
            run service

//TODO: add overides
}
}

As you can see, our class SimpleService is deriving its functionality through the System.ServiceProcess.ServiceBase class. Since the ServiceBase class is abstract, we cannot implement it directly, we must derive from it. We do this in our class declaration with the code SimpleService : System.ServiceProcess.ServiceBase . Which essentially reads, "Class SimpleService derives from ServiceProcess.ServiceBase ". This is a core concept of object oriented programming which you will need to understand in the .NET world.

Ok, now we have a basic idea of what our class needs to look like, let's start replacing each " TODO: " with our actual code. First, lets add the implementation of our Initialize function. Replace the line which reads: "//TODO: add Initialize( ) method " with the following code:

private void InitializeComponent ()
{
this.ServiceName = "SimpleService";
this.CanPauseAndContinue = true;
this.CanShutdown = true;
this.CanStop = true;
}

The InitializeComponent method will set all of our properties that define our service. The first property that we need to set is ServiceName . Among other things, the ServiceName defines the name that will appear in the SCM display. Also, setting this property will define the source property of the EventLog members of the ServiceBase class.

The three other properties that we set in this class ( CanPauseAndContinue , CanShutdown , and CanStop ) define how the service will be able to function after it is installed. CanShutdown determines whether the service should be notified if the operating system is shutting down. CanPauseAndContinue and CanStop add functionality to pause, continue and stop the service after it is running.

Now we need to add our Main () to this class. To do this, replace the " //TODO: Add main.. " code with the code below:

static void
            Main()
{
System.ServiceProcess.ServiceBase[]
            ServicesToRun;
ServicesToRun = new
            System.ServiceProcess.ServiceBase[] { new SimpleService()
            };
System.ServiceProcess.ServiceBase.Run(ServicesToRun);
}

The Main method of a C# Windows Service executable will define the services to run, and then run the services. If you look at the three lines of code above, you will see that the first line defines the object ServicesToRun as an array of the ServiceBase type. The second line of code defines and creates that array. The above case is for a singular, simple service. However, if I had created another service, a statement like the one below would also be perfectly legal:

ServicesToRun = new
            System.ServiceProcess.ServiceBase[] {new SimpleService(), new
            ComplexService()};

The third call simply calls the run method of the ServiceBase class. Optionally, you don't have to use an array at all. The third line of code calls the Run method, which essentially loads the service.

If you look back on the code written so far, you should have an idea about the flow in service applications. Now let's add the implementation code to our public SimpleService member. To do this, replace the commented text " //TODO: Add Class Implementation " with the code below:

InitializeComponent ();
Timer tmr = new
            Timer();
tmr.Interval= 5000;
tmr.Enabled = true;

tmr.Elapsed+=new
            System.Timers.ElapsedEventHandler(OnTimedEvent);

The InitializeComponent () call in the first line will set the properties of our service. For demonstration purposes, I have added a class called OnTimedEvent which will write an event to the event log every five seconds. Add the OnTimedEvent code below after the Main method:

public static void OnTimedEvent(object source, System.Timers.ElapsedEventArgs e)
{
if (!EventLog.SourceExists("SimpleService"))
EventLog.CreateEventSource("SimpleService", "Application");

EventLog.WriteEntry("SimpleService", "5 seconds have passed");
}

Overriding ServiceBase Members

A key feature of .NET services is the ability to override ServiceBase 's protected methods. This is very useful, because it allows us to add special processing, operations, notifications, etc, when an event occurs. For example, I could hypothetically create an operators group and net send each of those operators when a shutdown occurred. For our SimpleService , we will override the OnStop and OnContinue methods. Replace the " //TODO: add overrides " section with the code below:

protected override void
            OnContinue()
{
System.Diagnostics.EventLog el = new EventLog()
            ;
el.Source = "OnContinue";
el.WriteEntry("the SimpleService
            service has resumed resumed!");
}

protected override void
            OnStop()
{
System.Diagnostics.EventLog el = new EventLog()
            ;
el.Source = "OnStop";
el.WriteEntry("the SimpleService
            service has stopped!");
}

protected override void
            OnStart(string[] args)
{
System.Diagnostics.EventLog el = new
            EventLog() ;
el.Source = "OnStart";
el.WriteEntry("the
            SimpleService service has started!");
}

Installing the Service

So far we have built a class, which we will use to implement our simple service. In order to get the class functioning as a service, some logistics need to take place. In other words, we have created a service, now we need to tell Windows that the service exists, and tell it how to load and handle it. To do this we need to build an installer class , the .NET framework already provides us with this class, called ServiceInstaller .

Really, the installer helper class that we will build never actually comes in contact with the class we've already created. Instead our installer helper class will just take care of the grunt work such as creating the appropriate registry entries, so that a utility that we will use later on can create and register the service.

To begin with, lets add another empty class to our project. Do this by right clicking on the ServiceProject class, then selecting Add , then Add New Item. from the ServiceProject context menu. The Add New Item dialog should appear. Select the C# Class option, then name the class SimpleServiceInstaller.cs , and then click OK .

We need to add a reference to the System.Configuration.Install.dll . Do this in the same way we did this in our SimpleService class.

Now that we have our new class file added, let's add the skeleton code that we will need to create our installer. Paste the code below into your new SimpleServiceInstaller class.

namespace ServiceProject
{
using
            System;
using System.Configuration.Install;
using
            System.Collections;
using System.ServiceProcess;
using
            System.ComponentModel;
[RunInstallerAttribute(true)]
public
            class MyProjectInstaller: Installer
{
private ServiceInstaller
            serviceInstaller;
private ServiceProcessInstaller
            processInstaller;

private void InitializeComponent
            ()
{
}

public MyProjectInstaller()
{
// TODO:
            Instantiate installers for process and services.
// TODO:Set the
            security context that this service will run under
// TODO:Set
            initialization attributes for this service
// TODO: Add
            installers to collection.
}
}
}

As you can see, there are a number of TODO 's that need addressing before we get this installer class working, but first let's discuss the class itself.

For the namespace, you must use the same namespace as the service you created, which in this case is ServiceProject . Further down you'll find the [RunInstallerAttribute(true)] statement. Setting the RunInstallerAttribute value to true will invoke an installation utility called InstallUtil.exe when the assembly is installed. We will get to this topic later, but for now, let's examine the rest of this class.

The MyProjectInstaller derived class that we will be using derives its functionality from the System.Configuration.Install.Installer base class. We need this class to access the functionality of the ServiceInstaller and ServiceProcess installer members. These two classes perform similar, but equally essential, operations when installing a class.

ServiceInstaller

  • Performs actions specific to the actual service that is associated with it.
  • Writes values in the HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services sub key in the registry.
  • Creates an EventLog source for the service.
  • Each service installed must have a ServiceInstaller instance associated with it.
  • Each service's ServiceName attribute must correlate with the ServiceName attribute in the Service's class.

ServiceProcessInstaller

  • Does work common to all services in an executable.
  • Sets security context to run service under.
  • Each service application must have a ServiceProcessInstaller instance associated with it.
  • When instantiated, the ComponentInstaller class, will be called.

As you can see from the points above, these to classes do similar, but important work.

Now that we have a little background on these classes, let's implement them in our installer class. To start, we need to instantiate a ServiceInstaller class and a ServiceProcessInstaller class. Replace the line that says " //TODO: Instansiate installers for processes and services " with the following code:

   processInstaller = new
            ServiceProcessInstaller();
serviceInstaller = new
            ServiceInstaller();

The above code will create new instances of the ServiceProcessInstaller and ServiceInstaller classes. These classes will handle our installation grunt work.

One of the attributes that we can define is which account this service will run under. In this case we will set the RunUnderSystemAccount property to true. It should be noted that running a service under the system account is the same as running under an administrative account. To set this attribute, replace the " TODO://Set the security context that the service will run under " with the following code:

processInstaller.Account = System.ServiceProcess.ServiceAccount.LocalSystem;

Alternately, we could set this property to false, in which case the installer would ask for a username and password to run the service on during installation.

Next, we need to set the attributes of our services. Specifically, let's tell the installer how the service can be started. Some possible values are disabled , manual , and automatic . We can set this property using a structure C# has provided called ServiceStart . The other property we will set in our service will be the service name. This is an important property. The ServiceName s must correlate with the names defined in our actual service class. Failing to do this may cause some difficult-to-trace errors. To set the attributes we just talked about, paste the following code where it says " TODO: Set initialization attributes for this service ".

serviceInstaller.StartType = System.ServiceProcess.ServiceStartMode.Manual;
serviceInstaller.ServiceName = "SimpleService";

Setting the start type to manual will force us to start the service either programmatically or through the Service Control Manager.

Finally, we need to add our installers to the installers collection we do this with a call to Installers.add ( ) . To do this, replace the " TODO://Add installers to collection " comment with the code below:

Installers.Add(serviceInstaller);
Installers.Add(processInstaller);

If we had multiple services to install, we could have issued multiple add commands.

At this point we have done everything that we need to do programmatically to create our service. Now we need to actually run build and install the service.

First, make sure your work is saved. Next select the Build option from the Build menu item.

The .NET framework comes with a nice command line utility called InstallUtil . We will use this to install our new service. To do this, go to a command prompt and enter the following command:

InstallUtil C:\ServiceApps\ServiceProject\bin\Debug\ServiceProject.exe

(Assuming C:\ServiceApps\ is where your project is saved!)

After issuing this command, the installer starts doing it's work. Notice that it installs the service and registers an event source.

If your screen looks similar to the one above, then you have successfully installed your service. Now, we need to start it using the Service Control Manager (SCM).

The Service Control Manager

The Service Control Manger , or more often called the SCM (affectionately known as "Scum"), is a graphical utility that we can use to manage our services.

To access our service, open the Administrative Tools shortcut from the Control Panel , and then select the Services shortcut. Opening this tool will bring up the Service Control Manager. Find your service, which should be called "Simple Service", then right click on it and hit Start .

After clicking start, you will receive a message that windows is trying to start the service. After the service is started, examine the event log and look for messages from our service. The events should be written in the application log.

It looks like our service is running, now let's stop our service by right clicking our component and selecting the Stop option.

Congratulations, you have just created and run a Windows Service in C#!

Summary

In this article we wrote and implemented a Windows Service agent that writes custom entries to the event log using Microsoft's C#. NET. To accomplish this, we created our service class and an installer, installed the class using the InstallUtil utility and ran and controlled the service using the SCM. Microsoft's C#.NET took care of most of the dirty work for us, leaving us more time to focus on more important issues such as design. I'm sure anyone who has written services in the past can attest that the .NET way seems much cleaner and easier.

C#Today - © 2002 WroxPress Limited


Remote querying your data warehouse using web services and XML

Remote querying your data warehouse using web services and XML
http://www.csharptoday.com/content.asp?id=1562
Article Rating: 4
Published on August 24, 2001
Abstract
Until now, accessing a distant data warehouse couldn’t be done in a straightforward manner. If data is dispersed over multiples sites, querying one remote data warehouse in particular is still a challenge. Thinking in terms of Web Services, you could present a Web Service on each site that would provide some programmatic query functionalities on the local data warehouse. A remote client could then remotely query the data warehouse by using its related Web Service programmable application logic. In this article, Mathieu Clerte looks at how Web Service development has been simplified under the Microsoft .NET Framework, compared to the previous SOAP Toolkit. He also shows how to build a C# Web Service which will drive an MSSQLServer OLAP data warehouse using the .NET framework.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

Until now, accessing a distant data warehouse couldn't be done in a straightforward manner. In the case of a worldwide company where data is dispersed over multiples sites, querying one remote data warehouse in particular is still a challenge, without even thinking about downloading the entire database and locally reconstructing the data warehouse. Moreover, this would be simply dealing with security problems during data transfer. Thinking in terms of Web Services, this company would present a Web Service on each site that would provide some programmatic query functionalities on the local data warehouse. A remote client could then remotely query the data warehouse by using its related Web Service programmable application logic. In this article, we will see how Web Service development has been simplified under the Microsoft .NET Framework, compared to the previous SOAP Toolkit. We will learn how to build a C# Web Service which will drive an MSSQLServer OLAP data warehouse using the .NET framework.

UNDERSTANDING THE WHOLE THING

A note on Web Services

Web Services are a recent technology. Let's take a look to what it is, and how it can be used. A simple definition of a Web Service is a "programmable application logic accessible using standard Internet protocols." The general architecture of a Web Service approaches a standard object component: an interface made up of methods accessible by the client, and an implementation consisting of the underground code, hidden from the client. Unlike current component technologies, Web Services are not accessed via object-model-specific protocols such as the Distributed Component Object Model (DCOM) or Remote Method Invocation (RMI), but via Web protocols and data formats such as HTTP and XML. Thus, a Web Service can be "consumed" from any platform, in any programming language, in a large compatibility range, and its object-model nature makes it highly reusable. As it uses HTTP as its transfer protocol, the Web Service communicates easily with the client through firewalls and you don't need to open specific ports and deal with application and users permissions, which was one more reason to create security holes in your network. Regarding this security, you will build a solid and secured Web Service publication just by combining it with SSL encryption, thus protecting the data transferred to and from the client.

Why use XML?

Reasons to use the eXtensible Markup Language include the facts that the .NET architecture is based on XML, and furthermore, the data transferred to and from a Web Service is formatted using the XML. Moreover, XML is a very user-friendly, compliant file format, and its hierarchical nature provides a simple way to represent, store and move hierarchical data from a simple database or a more complex data warehouse. In this case study, we will use XML for the following operations:

  • To store the multi-dimensional SQL query command sent to the Web Service
  • To store the data tables returned to the consumer by the Web Service

Why the .NET Platform ?

Of course, you can use other platforms and languages than the ones below for developing a Web Service, but again, Microsoft defines the .NET platform as "A Platform for Web Services". Why? Simply because using the .NET platform, an application is constructed using multiple Web Services that work together to provide data and services for the application. Web Service development using the System.Web.Services class is easy and faster than using other SOAP applications development methods such as the SOAP Toolkit. Moreover, VisualStudio.NET fully integrates Web Service construction by reducing its complexity to a one-class-and-two-clicks operation. Furthermore, the .NET Platform provides an XML driving specific namespace called System.Xml , which implements the old XML Document Object Model in a better way. It provides special support for XML-stored database content manipulations with new objects like the XmlDataDocument object.

About SQLServer OLAP Services

Also called Analysis Services, SQLServer OLAP Services is a middle-tier server for online analytical processing (OLAP) and data mining providing a rapid client access to the data warehouse. Data from the data warehouse is extracted, summarized, organized, and stored in multidimensional structures for rapid response to end user queries. Programmatically speaking, OLAP Services can be managed through an object model called Decision Support Objects (DSO), and queries to OLAP cubes can be sent by using the ADO Multi-Dimensional library ( ADOMD.DLL ). A multi-dimensional query is an advanced SQL query with specific functions, and in general three or more crossed variables. For example:

SELECT { Country } ON ROWS, { Categories } ON COLUMNS, { Years } ON PAGES
FROM [SalesCube] WHERE Measures.[Sales Count]

This query will return you a cross-table giving the total sales for each country in each product category by year.

Concretes applications at work

Some Possible uses

One way to use a Web Service querying your data warehouse is to obtain cross-tables and statistical data through the Internet from a distant site. For example, a regional manager could access, at any time, all the information for the needs of each store of each region, such as stock levels. Furthermore, we could imagine a centralized web application consuming each store's Web Service to aggregate local data and obtain a general statistical view for the entire region. In the past, this operation would imply replication of local data on the central site, and the reconstruction of a huge data warehouse. Using Web Service in this context will save a lot of time as we no longer have to transfer full database packages, but only queries results, and we can explode the old centralized global data warehouse into multiples lighter data stores. The next figure illustrates this whole concept:

Implementing Web Services in an existing network

How does all this fit into your company? As you will see, adding Web Services to your intranet doesn't imply any remodeling of your existent software or website implementation: they are simply additions. Web Services can coexist together on published websites on the same platform without conflicting.

The minimum requirements for .NET Web Services to work are:

  • Windows 2000 (either Pro or Server) or NT 4 with (Wks or Server) with Service Pack 6a.
  • .NET Framework.

THE OLAPSERVICE WEB SERVICE

Making Web Service: A Short Overview

As only privileged people have a beta, and because building a Web Service with VisualStudio.NET is pretty easy, we will focus on development under the .NET Framework. However, if you've got the IDE, it is pretty simple to adapt the code in a new project. This article will teach you how to create a Web Service using VisualStudio.NET.

Let's walk through our Web Service construction: first, we need to prepare the web server to host the Web Service. Open IIS and create a new website, or a new virtual website in an existing one. You need a free virtual directory (under the default website root, for example) pointing to your Web Service physical location, for example the \code folder of the code sample attached to this article. The screenshot below shows IIS under the MMC:

Under the root, add an empty folder named \bin that will contain all the libraries required by the Web Service. The Web Service configuration information has to be written in a configuration file named Config.web ( Web.Config in beta2) like any other ASP.NET application. This file will be XML structured.

Then comes the time to write the main class. Web Services are recognized by IIS under the extension .asmx . By the way, you need to create an empty text file named " OLAPService.asmx " at the root of the service website folder. The main class can be written in any language supported by the .NET Framework, even inside the .asmx itself, or outside, in an external file.

Compile the Web Service, including any external references by using an appropriate compiler. If your code is written in C#, you will use csc.exe, or vbc.exe for VB.NET.

Once the service is published, some tasks are required for the client to make use of it. Generate a proxy of the Web Service to give you programmatic access to its methods. It's a set of one .sdl file called " SDL Contract " and one assembly. Create an instance of the proxy, and use it to call the Web Service methods in the client application code.

Importing external COM libraries

The Common Language Runtime manages all code that runs inside the .NET Framework. Code that executes under the control of the runtime is called managed code. Conversely, code that runs outside the runtime is called unmanaged code. COM components, ActiveX interfaces, and Win32 API functions are all unmanaged. The Type Library Importer ( TlbImp.exe ) utility converts the classes and interfaces contained in a COM type library to a .NET Framework assembly. Visual Studio generates the assembly for you when you add a reference to a type within a given type library.

The OLAPService Web Service we are about to build will communicate with MSSQL Server Analysis Services via a special OLEDB provider called "OLEDB Provider for OLAP". Programmatically speaking, querying an OLAP cube requests a particular library called ActiveX Data Objects Multi-Dimensional (or msadomd.dll ). This library hosts a set of objects and methods useful for both building or sending queries to the server, and for manipulating OLAP databases structures. As a COM component, the msadomd.dll library is unmanaged and needs to be converted with the following command-line statements:

tlbimp %SYSTEMROOT%\System32\dllcache\msadomd.dll /out:path_to_your_bin_directory\bin\ADOMD.dll

If the source msadomd.dll is not found, try the following path: Program files\Common files\System\Ado . The command above will create an " ADOMD.dll " .NET assembly providing a managed interface for the msadomd.dll COM component. In order to instantiate objects from the ADOMD assembly, we need to register it in the Web Service configuration file by adding the following Xml code:

<configuration>
<compilation debugmode="true">
<assemblies>
<add assembly="*"/>
</assemblies>
</compilation>
</configuration>

Now you're ready to begin writing the Web Service.

Writing the OLAPService class

The OLAPService Web Service contains one class, called OLAPquery, defined in the OLAPService namespace. The only method of that class is called ExecuteMDX() and executes a multi-dimensional ( mdx ) query and returns its result as a DataSet object. If the "ON PAGES" statement is used in the multi-dimensional query, the DataSet returned will contain as many DataTable as there are page occurrences. If only the "ON ROWS" and "ON COLUMNS" statements are used, the DataSet will have one table. A big advantage of the DataSet object is that it can carry more than one table. Thus, we save a lot of time at client-side while browsing the result of a complex query, by switching from table to table in the DataSet, instead of re-posting each time the query. At the beginning of the class, we must ensure that the necessary assemblies are referenced:

namespace OLAPService {
using
            System;
using System.Data;
using System.Reflection;
using
            System.Web.Services;
using System.Xml;
using
ADOMD;

Then we declare the class itself and its own method, preceded by the [WebMethod] keyword specifying that it is a Web Service method.

public class OLAPquery: WebService {

[WebMethod]
public DataSet ExecuteMDX(string sSqlMdx, string sDatabase)
{

The inner code of the method is written below. It first connects to the OLAP database specified as the second parameter of the method, and opens the resulting CellSet specified as the first parameter.

   ADOMD.Cellset aCellSet =
            new ADOMD.Cellset();;

string sConn = "Provider=MSOLAP;Data
            Source=localhost;database=" +
sDatabase +
            ";";
aCellSet.Open(sSqlMdx, sConn);

// Navigates and reads
            CellSet
if (aCellSet.State != 1)
return null;

short
            nPages;
string pageName;

The third axis of the CellSet corresponds to what I call the "Page" axis. We count the number of pages, which are the number of occurrences of the variable specified before the "ON PAGE" statement in the mdx query. If no "ON PAGES" statement is used then we consider that there is only one page, and two axes.

if (aCellSet.Axes.Count > 2)
{
//Counts the number of pages
nPages = aCellSet.Axes[2].Positions.Count.ToInt16();
} else
nPages = 1;

The time has come to build and fill the DataSet with the CellSet calculated values. We loop through the number of pages, building and filling one DataTable for each page occurrence. We name the current DataTable to be the same as the Caption property of the related third axis Member.

   DataSet ds = new
            DataSet();
DataTable aTable;

// Adds and Fills a DataTable
            for each Page variable (Axes[2])
for (int iPage = 0; iPage <
            nPages; iPage++)
{
if (nPages >1)
{
ADOMD.Member
            PageHeader =
            aCellSet.Axes[2].Positions[iPage].Members[0];
pageName =
            PageHeader.Caption;
}
else
{
pageName =
            "results";
}

// Build DataTable structure
aTable =
            ds.Tables.Add(pageName);

The first column of each DataTable contains row labels. Thus, we build it with the title of the LevelName property of the rows axis (Axes[1]) :

// Build Rows headers column
DataColumn rowsHeadersColumn = new DataColumn();
rowsHeadersColumn.Caption = aCellSet.Axes[1].Positions[0].Members[0].LevelName;
aTable.Columns.Add(rowsHeadersColumn);

The first axis of the CellSet, or Axes[0], represents the axis for the "ON COLUMNS" term of the multi-dimensional query. We add as many DataColumn in the current DataTable as there are Positions in the columns axis.

      // Build
            other columns
foreach (ADOMD.Position CScolumn in
            aCellSet.Axes[0].Positions)
{
// adds a new DataColumn for
            each column
DataColumn aColumn = new
            DataColumn();
aColumn.Caption =
            CScolumn.Members[0].Caption;
aTable.Columns.Add(aColumn);
}

Now that we have defined the current DataTable's structure, we fill each row, column by column.

The main loop walks through the rows axis, defined as the second axis of the CellSet, or Axes[1].

A new row is added to the DataTable, and in the inner loop we walk through the columns axis to get each cell 's value.

         //
            Fills datatable contents
for (int iRow=0;iRow <
            aCellSet.Axes[1].Positions.Count;iRow++)
{
DataRow newRow =
            aTable.NewRow();
newRow[0] =
            aCellSet.Axes[1].Positions[iRow].Members[0].Caption;
for (int
            iCol=0; iCol <
            aCellSet.Axes[0].Positions.Count;iCol++)
{

Here comes a tough part. In order to get the current Cell value, we need to grab the Item property of the CellSet object. Actually, we could get a Cell from a CellSet with the following statement:

   CellSet myCellSet;
Cell
            myCell;
myCell = myCellSet.Item[colum, row];
// or myCell =
            myCellSet.Item[colum, row, page];

Strangely, this property cannot be accessed directly from the CellSet object. Of course, there is always a solution! The Item property is related to the get_Item() public method of the CellSet object, who's interface is:

get_Item([in] class System.Object[]& marshal( safearray variant) idx) runtime managed ole internalcall

Thus by calling the get_Item() method directly with a parameter of an array of objects specifying the cell 's coordinates, we can retrieve the cell as a Cell object. In our example, if there is more than one page, we add the current page number iPage as a third coordinate of the cell.

      object[]
            args = new object[1];
object[] coords;
if (nPages
            >1)
{
coords = new Object[3];
coords[0] =
            iCol;
coords[1] = iRow;
coords[2] = iPage;
}
else
{
coords = new Object[2];
coords[0] = iCol;
coords[1] =
            iRow;
}
args[0] = coords;

We get back the Item property by first defining it inside a PropertyInfo object using the GetProperty() method of the ADOMD.CellSet object type. Then we retrieve the aCellSet property for our CellSet object with the GetValue() method of the PropertyInfo object. We cast the result to an ADOMD.Cell object, and retrieve its FormattedValue property.

   Type t =
            Type.GetType("ADOMD.Cellset");
PropertyInfo p =
            t.GetProperty("Item");
object obj = p.GetValue(aCellSet,
            BindingFlags.GetProperty,null , args,null);
ADOMD.Cell aCell =
            (ADOMD.Cell) obj;

The method above was the only possible way under Beta 1. However, the get_item method works perfectly in Beta 2, so we can replace the code above with the following lines:

System.Array args = (System.Array) coords;

ADOMD.Cell aCell = aCellSet.get_Item(ref args);

(I wanted to show you the old Beta 1 method because it's a little tricky, and may be useful if you want to use other imported COM's that don't accept direct methods or properties calls.)

Then we can get the cell's value:

newRow[iCol+1] =
            aCell.FormattedValue;

}
aTable.Rows.Add(newRow);
}
}

Finally, we close the CellSet and return the filled DataSet .

      aCellSet.Close();
return
            ds;
}
}
}

Compiling the Web Service

Actually, you could skip this part and use your Web Service as a .asmx un-compiled file. In that case, the server would compile it automatically when a client requests it. However, pre-compiling your class will enhance the Service's functionality and you will get faster responses. I noted that more errors appeared when using this service without pre-compilation. However, I also noted that less running errors appeared when querying OLAP using a pre-compiled service than with a service that was Just-In-Time compiled.

The following command-line statement will compile OLAPService.cs :

csc /out:\bin\OLAPService.dll /t:library /r:system.web.services.dll /r:system.xml.serialization.dll
/r:\bin\ADOMD.dll OLAPService.cs

As you can see, we need to include both the system standard assemblies and custom assemblies used by the service ( ADOMD.dll ). To complete the OLAPService construction we need to create an empty text file called OLAPService.asmx at root level, which will be the Web Service user's front-end. At the beginning of this file we include some directives which identifies the file as being a Web Service:

<%@ WebService Language="C#" Class="OLAPService.OLAPquery" %>

At this time, a new assembly called OLAPService.dll exists in the \bin folder with an OLAPService.asmx file referring to it.

Testing the Web Service

Now that the Web Service is built, let's test it! To test your Web Service, access your web browser and open your OLAPService.asmx by entering its URL using this format:

http://servername/virtualdirectory/OLAPService.asmx

The Web Service's HTML description file is displayed. The following screenshot shows the web page generated for the OLAPService Web Service:

Clicking ExecuteMDX takes you to the following screen:

To invoke the ExecuteMDX() method, enter an multi-dimensional query and a database name in the ' value ' textboxes, then click ' invoke '. The service then returns your query's result in DataSet XML format. The screenshot below shows a resulting query example:

Moreover, if an error occurs due to inaccessible database or a badly written query, the error is returned in plain text.

A Sample Client Application

Accessing OLAPService

Now that you've built your Web Service and checked its functionality, you can think about its exploitation by a client application. To programmatically invoke the service's web methods, you need to build a proxy to your Web Service for the client application. Each Web Service provides an interface for its methods in an XML file called Service Description Language contract, or SDL. This file is published on the following URL:

http://servername/virtualdirectory/myservice.asmx?SDL

To create your proxy, you need to generate a class from your Web Service's SDL contract file. Then, you will compile this class and use it in your client code.

There are two ways to retrieve the SDL contract:

  • Go to your Web Service's SDL contract URL, and save the file under the .sdl extension. Use this file as source for the proxy utility.
  • Refers directly to the SDL contract URI as source for the proxy utility.

Generate the proxy class using the WSDL tool with the following command-line options:

Wsdl /language:CSharp /namespace:OLAPService /out:OLAPQuery.cs
http://localhost/webservice/OLAPService.asmx?WSDL

Then compile the new OLAPService.cs class with:

csc /out:OLAPServiceproxy.dll /t:library /r:system.web.services.dll
/r:system.xml.dll /r:System.Data.dll OLAPquery.

We must add a reference to the System.Data.dll assembly because the ExecuteMDX() web method returns a DataSet object. Once the proxy assembly is created, move it into the \bin directory of your client application, and then modify the Web.Config configuration file to reference all assemblies from the \bin directory, like you did while building the Web Service.

A Simple Web Application

We are now ready to build a small client for our Web Service.

Let us use a very common and often used data warehouse called "FoodMart". For those who began with OLAP, don't worry: this data warehouse is automatically installed with Analysis Services.

We will build a page using the OLAP Service to query that database and retrieve stores sales by state for the USA region. In OLAP terms, this expression is equivalent to the following mdx query:

SELECT { [Store].[USA].Children } ON ROWS, { MEASURES.[Store sales] } ON COLUMNS FROM [Sales]

We will then simply call an instance of the OLAPService Web Service, and called ExecuteMDX() , providing the query above as the first parameter and "FoodMart" as the second. We will then set the first table of the returned DataSet as DataSource for a standard DataGrid object.

Here is the code for that web page:

<%@ Page language="C#"
            debug="true"%>

<%@ Import Namespace="System"
            %>
<%@ Import Namespace="System.Data" %>
<%@
            Import Namespace="OLAPService" %>

<script language="C#"
            runat="server">
void Page_Load() {
OLAPquery oq = new
            OLAPquery();
string mdxquery = "SELECT { { [Stores.MEMBERS] } ON
            ROWS, { MEASURES.
[Stores sales] } ON COLUMNS FROM [Sales] WHERE
            { [Time].[Year].[1999] }";

DataSet ds =
            oq.ExecuteMDX(mdxquery, "FoodMart" );
if (ds != null)
            {
datagrid1.DataSource =
            ds.Tables[0].DefaultView;
datagrid1.DataBind();
}

}
</script>

<html>
<body>
<form
            runat="server">
<asp:datagrid id="datagrid1"
            runat="server"/>
</form>
</body>
</html>

The next figure shows how the page appears in IE:

CONCLUSION

In this article, we build a Web Service whose function was to query a specified OLAP data warehouse and return the result to a far client. We discovered how powerful the .NET Framework environment is for building and hosting Web Services. Moreover, we learned that sharing distant data warehouses will no longer be complicated when applying the Web Services technology to data mining. With Web Services, it is no longer necessary to store centralized data in a huge repository of information, as was the standard model in the past. From now on, we can think of a new schema, with smaller satellite data warehouses based in multiple distant sites, connected over the Internet, which can be queried remotely using the Web Services technology. In a few words, Web Services opens a new era of data sharing for the data miners.

C#Today - © 2002 WroxPress Limited


COM+ and C#

COM+ and C#
http://www.csharptoday.com/content.asp?id=1615
Article Rating: 4.5
Published on July 26, 2001
Abstract
In this world of connected network applications, it is quite reasonable that your web applications are expected to scale to meet the needs of the millions of users who will be accessing your web site. COM+ is a core element in the Windows DNA 2000 methodology to create distributed applications that are scalable, extensible and maintainable. In this article, Thiru Thangarathinam helps us understand that COM+ is still an integral part of Microsoft’s strategy to creating n-tiered web applications even in the .NET world. He also shows us how to create a transactional C# object that makes use of COM+ Services in an integrated and logical manner. Finally, we wrap it up by seeing how a transactional C# object can be exposed as a Web service to allow Internet clients to consume it using ubiquitous protocols like XML and SOAP.
Article

Introduction

Component Services in Windows 2000 has a wealth of Enterprise functionality that provides all the plumbing required for creating distributed, web-enabled applications. Although COM+ Services were originally designed for use with COM components, it would be nice if .NET components could also use them because of the services such as object pooling, database connection pooling, sharing resources, role based security and distributed transaction processing monitoring that COM+ runtime provides.

In .NET framework, there is a separate namespace called System.EnterpriseServices that contains all the classes required for providing COM+ services to applications. For an introduction to COM+, please take a look at the following link from TechNet:

http://www.microsoft.com/technet/treeview/default.asp?url=/TechNet/prodtechnol/host/proddocs/hisdoc/appint/comti_introduction_to_com_and_com_.asp

Before we go ahead and create a sample application that makes use of all these classes, let us take a quick tour of some of the important classes present in the System.EnterpriseServices namespace.

ServicedComponent

To be able to create .NET objects that make use of COM+ services, we need to derive our objects from the System.EnterpriseServices.ServicedComponent class. This class provides methods for our objects to interact seamlessly with a COM+ runtime thereby utilizing the COM+ services. It contains the following methods that are directly related to the lifetime of the object:

  • Activate - Allows us to initialize the object when the object is activated. By object activation, we mean the process in which either a new instance of the object is created or an existing object from the pool is used
  • Deactivate - We can use this method to release all the resources that may have been initialized during the Activate method.
  • CanBePooled - Allows us to notify the COM+ whether we want our object to be pooled for reuse.

It also provides the following method that facilitates passing parameters to the object at the time of object construction:

  • Construct

Connection strings to the database is one of the typical parameters that can be passed to the object through the Construct method. In the later part of this article, we will see examples of how to implement these methods in our application.

ContextUtil

Using ContextUtil is the preferred method to access the object context of COM+. The methods present in the ContextUtil class are the primary means through which applications written in managed code can communicate with COM+ runtime. This class can be considered as a replacement for ObjectContext that was used in Pre .Net world in languages such as VB6, C++, ATL, and so on. To get a better understanding of transactions in COM+, please take a look at the following article from MSDN:

Writing MTS style transactions with VB and SQL Server 7.0.

As we already discussed, two important features provided by COM+ are transaction processing monitoring and Object Pooling. COM+ internally maintains two flags. These are a Done bit and a Consistency bit, which are used to vote for the outcome of a transaction and to control the deactivation of objects.

  • SetComplete - This method sets both done and consistency flags to true which indicates that the current transaction can be committed and the object can be deactivated.
  • SetAbort - This method sets the done bit to true and consistency flag to false, which indicates that the current transaction be rolled back, and the object deactivated.
  • EnableCommit - Sets the done bit to false and consistency flag to true.
  • DisableCommit - Sets both done and consistency flags to false.

We can use the above methods to work on the combination of done and consistency bits so that we can indicate COM+ runtime about the state of the transaction and whether the object can be deactivated.

However, there are times where you may want to manipulate the done and consistency bits individually so that you can have finer level of control over the transaction control and the object deactivation processes.

The ContextUtil class exposes the following two properties through which we can deal with the done bit and consistency bit individually.

  • MyTransactionVote - Consists of a property get and set through which we can either get or set the value of the consistency bit.
  • DeactivateOnReturn - Consists of a property get and set, which are used to either get or set the value of the done bit.

AutoCompleteAttribute

Before we go on to look at the AutoCompleteAttribute, let us see what an attribute is and the role of attributes in .NET framework. Attributes are declarative tags that allow the developer to attach additional information to assemblies, classes, methods or properties. This additional information is made available to the applications at runtime through the process known as reflection. We can define reflection as the process, which allows us to examine the types at runtime.

When a method is decorated with the AutoComplete attribute, it indicates that when the method returns successfully, the SetComplete method will be automatically called, and if the method throws an exception, the SetAbort will be called to abort the transaction.

It is also important to note that when we refer to the AutoCompleteAttribute class in our code, we will be simply referring to it only as "AutoComplete ". When the compiler comes across this line, it automatically appends the keyword "Attribute " to it and searches for the class named AutoCompleteAttribute, which it will find. We will see an example of this in action when we create our transactional C# object in the later part of the article.

ObjectPoolingAttribute

This class is used to configure object pooling for a .NET object. Using this class, we can set properties for the maximum pool size, minimum pool size, creation timeout and whether object pooling is enabled for a .NET object.

Registering a .NET object in COM+ Catalog

When you compile a COM component, it creates a type library, which provides COM+ with all the information that is required while installing the component into COM+. But .NET Components, when compiled, emit an assembly (which contains manifest, MSIL, metadata, etc) that is completely different from the type library. So we need to create a type library from the assembly so that the assembly can be installed into COM+ transparently.

There are three ways of registering a .NET component in COM+ in Windows 2000:

  • Using the Regsvcs command line utility. This command line utility takes the name of the assembly as the input, creates a type library by reading the assembly and finally registers the object with COM+.
  • Using the automatic registration mechanism provided by the .NET framework. At runtime, the CLR (Common Language Runtime) automatically creates a type library based on the assembly and COM+ uses the information present in the type library to install the objects.
  • Using the RegistrationHelper class that is part of the system.EnterpriseServices namespace. The InstallAssembly method of the RegistrationHelper class can be used to programmatically register .NET objects with COM+.

In this article, we will see the first two methods of registration.

Implementation

In the first part of the article, we will create a Transactional C# object that leverages the COM+ services of Windows 2000 and then we will go on to create a client application that makes use of the C# object. We will be using ADO.NET to access the database. We will also see how we can take advantage of the automatic registration mechanism provided by CLR to register the objects into COM+.

In the second part of the article, we will create a simple transactional C# 'Hello World' object and then discuss the steps, which we need to follow if we want to expose this object as a web service.

Implementation of a Transactional C# object (Authors)

In this part, we will create a C# class called Authors that contains a single method called AddAuthor which will insert the details of an author into the authors table of the pubs database. Let us take a look at the code of the AddAuthor method. To help avoid redundant typing, we import all the namespaces that we will be using in the application:

using System;
using System.Data;
using Microsoft.ComServices;
using System.Data.SqlClient;
using System.EnterpriseServices;

The ApplicationName attribute indicates the name of the COM+ application that will automatically be created when a client application tries to execute this assembly for the first time:

[assembly: ApplicationName("AuthorsApplication")]
[assembly: Description("ASPToday Demo Authors Application")]

Here, we specify that we want our Authors class to be contained in the AuthorsComponent namespace:

namespace AuthorsComponent
{

In this line of code, we decorate the authors class with the transaction attribute required, which indicates if the caller of this object runs in a transaction, it will be inherited by this object, otherwise it will create a new Transaction for itself and run within the scope of that transaction:

[Transaction(TransactionOption.Required)]

The ObjectPooling attribute allows us to specify the minimum number of objects and the maximum number of objects that exist in the Pool:

[ObjectPooling(MinPoolSize=2,MaxPoolSize=50)]

The ConstructionEnabled attribute allows us to specify a default construction string that will be passed to the object when the client invokes the methods of the object. This construction string is passed to the object during the creation of the object through the Construct method. It is important to note that all the attributes specified here in the code, are automatically set in the component when the component is registered with COM+:

[ConstructionEnabled(Default="server=localhost;uid=sa;pwd=;initial catalog=pubs")]

In this line, we derive the Authors class from the ServicedComponent, which allows the Authors object to make use of the services provided by COM+. Serviced Components are generally hosted in COM+ services as in-process servers (library applications).

public class Authors :
            ServicedComponent
{
public Authors()
{

}

Here, we declare a module level variable to store the connection string.

string mstrConstructString;

As we have already discussed, the Construct method is called during the creation of the object and this method is passed as a string that is specified either programmatically or through the COM+ explorer. In the Construct method, we capture the connection string that is passed, into a module level variable so that it can be used by the AddAuthor method for establishing connection with the database.

public override void Construct(string strConstructString)
{
mstrConstructString = strConstructString;
}

In this line, we enable the AutoComplete attribute for the AddAuthor method. If the method executes successfully and no exceptions are thrown, then the AutoComplete attribute automatically commits all the transactions for us by calling the SetComplete method of ContextUtil. If an exception occurs during the execution of that method, then the SetAbort of ContextUtil is automatically called to undo the transaction.

[AutoComplete]
public bool AddAuthor(string strAuID,
string strAuLName,
string strAuFName,
string strPhone,
string strAddress,
string strCity,
string strState,
string strZip,
int intContract)
{
string strCommandName;
SqlCommand objCommand;
SqlParameter objParam;

Now, we create an instance of the SqlConnection Object, passing the module level variable ConnectionString as an argument.

SqlConnection objConnection = new SqlConnection(mstrConstructString);

We enclose the following statements in a try catch block to catch any errors that may occur during the execution of those statements:

try
{

We assign the name of the stored procedure to be executed to a local variable. This stored procedure is very simple and it inserts a record into the Authors table of Pubs database. The code for the stored procedure is available for download along with the support material of this article.

strCommandName = "spInsertAuthors";

This line of code instantiates an SqlCommand object and two parameters are passed to its constructor. The first one being the name of the stored procedure to be executed and the second one is the SqlConnection object that was created in the previous step:

objCommand = new SqlCommand(strCommandName, objConnection);

By setting the CommandType property of the objCommand object to CommandType.StoredProcedure, we indicate that we want to execute a stored procedure:

objCommand.CommandType = CommandType.StoredProcedure;

The SqlParameter object represents a parameter to an SqlCommand. Here, we create an instance of the SqlParameter passing in the arguments that describe the parameter being passed to the stored procedure.

objParam = objCommand.Parameters.Add(new SqlParameter ("@sAuid", SqlDbType.VarChar, 20));

We set the Direction property to specify that it is an input only parameter. Other values that can be assigned are output only, bi-directional, or a return value parameter.

objParam.Direction = ParameterDirection.Input;
objParam.Value = strAuID;

We follow the same procedure for all the parameters that are passed to the spInsertAuthors stored procedure:

objParam = objCommand.Parameters.Add(new
            SqlParameter ("@sAulname", SqlDbType.VarChar,
            40));
objParam.Direction =
            ParameterDirection.Input;
objParam.Value =
            strAuLName;

objParam = objCommand.Parameters.Add(new
            SqlParameter ("@sAufname", SqlDbType.VarChar,
            20));
objParam.Direction =
            ParameterDirection.Input;
objParam.Value =
            strAuFName;

objParam = objCommand.Parameters.Add(new
            SqlParameter ("@sPhone", SqlDbType.Char, 12));
objParam.Direction
            = ParameterDirection.Input;
objParam.Value =
            strPhone;

objParam = objCommand.Parameters.Add(new
            SqlParameter ("@sAddress", SqlDbType.VarChar,
            40));
objParam.Direction =
            ParameterDirection.Input;
objParam.Value =
            strAddress;

objParam = objCommand.Parameters.Add(new
            SqlParameter ("@sCity", SqlDbType.VarChar,
            20));
objParam.Direction =
            ParameterDirection.Input;
objParam.Value =
            strCity;

objParam = objCommand.Parameters.Add(new
            SqlParameter ("@sState", SqlDbType.Char, 2));
objParam.Direction
            = ParameterDirection.Input;
objParam.Value =
            strState;

objParam = objCommand.Parameters.Add(new
            SqlParameter ("@sZip", SqlDbType.Char, 5));
objParam.Direction =
            ParameterDirection.Input;
objParam.Value =
            strZip;

objParam = objCommand.Parameters.Add(new SqlParameter
            ("@sContract", SqlDbType.Bit));
objParam.Direction =
            ParameterDirection.Input;
objParam.Value =
            intContract;

In this line, we open the ActiveConnection to the Database:

objCommand.ActiveConnection.Open();

ExecuteNonQuery of the SqlCommand object is used to execute a command that does not return any rows against the ActiveConnection:

objCommand.ExecuteNonQuery();

In this line, we return true to indicate that the method is executed successfully. Because of the AutoComplete that we set in the beginning of the method, the transaction will be automatically committed at this point:

return true;
}

If any errors occur during the execution of the above statements, we raise the exception back to the caller:

catch(Exception ex)
{
throw ex;
}

The statements in finally are always executed regardless of whether the statements in the try block are executed successfully or not. Here we inspect the state property of the SqlConnection to check whether the connection is open. If the Connection is open, it is closed.

finally
{
if ( objConnection.State == ConnectionState.Open )
objConnection.Close();
}

Now, let us compile the AuthorsComponent application and create an assembly. The next step is to create a client application, which invokes the AddAuthor method to store the author details to the database.

Implementation of Client Application

First, we select a Windows C# Application from the new project dialog box and we will name the project as AuthorsComponentClient.

Once the project is created, we add reference to the AuthorsComponent by right clicking on the project in the solution explorer, selecting Add reference and then navigating to the path of the AuthorsComponent assembly DLL.

The form that we are going to use for testing our AuthorsComponent looks like this. When the user enters all the details and submits the form, we invoke the AddAuthor method of the Authors class to save the author details to the database.

Let us look at the code that is required for creating an instance of the Author class and invoking its AddAuthor method:

namespace AuthorsComponentClient
{
using System;
using System.Drawing;
using System.Collections;
using System.ComponentModel;
using System.Windows.Forms;
using System.Data;
using AuthorsComponent;

public class AuthorsClient : System.Windows.Forms.Form
{
protected void cmdSaveAuthorsDetails_Click (
object sender, System.EventArgs e)
{
int intContract;

In this line, we convert the text the user enters into the txtContract textbox into an integer and store it in a variable.

intContract = Int16.Parse(txtContract.Text);

Since we have already imported the AuthorsComponent namespace into our application, we need not qualify the Author class with the namespace:

Authors objAuthors = new Authors();

Now, we can call the AddAuthor method of the Author class to save the author details into the database:

objAuthors.AddAuthor(txtAuthorID.Text,
txtAuthorLastName.Text,
txtAuthorFirstName.Text,
txtPhone.Text,
txtAddress.Text,
txtCity.Text,
txtState.Text,
txtZip.Text,
intContract);

MessageBox.Show("Author Details Successfully saved");
}

When we execute the client application, the CLR checks to see whether the assembly is already registered with the COM+ catalogue. If it is already registered, it creates an instance of the object and hands it over to the client. If it is not then it automatically registers the assembly with the COM+.

This kind of registration is called Lazy registration, as it does not require any effort on the part of the developer to register and configure the object to use the services provided by COM+. In the following snapshot, we can see that the name of the assembly name and the assembly description attributes, which we specified in our code, are automatically added to the COM+ application at the time of automatic registration. It is important to remember that the attributes we set at the assembly level correspond to the Application level entries in the in COM+ explorer.

We also specified the transaction attributes and the construction string at the Author class level.

If you right click on the AuthorsComponent Authors class that is present in the components folder of the AuthorsApplication, you can see a window that is similar to the following.

Exposing a Transactional C# object as a web service

In this part of the article, we will create a simple HelloWorld C# object (designed to run in COM+) and then see how to expose this object as a web service so that internet clients can consume this object using internet protocols like XML and SOAP. For an overview of transactions in COM+, please take a look at the following article from Don Box:

Windows 2000 brings significant redefinements to the COM+ Programming model.

Implementation of HelloWorldComponent

First we select a C# class library project from the new project dialog box and we will call this project as HelloWorldComponent.

namespace HelloWorldComponent
{

Here, we import the namespaces that will be required for our application.

using System;
using System.EnterpriseServices;

We want this object to be running as part of the same transaction as that of the calling client, if the calling client is already running in a transaction.

[Transaction(TransactionOption.Supported)]

We derive our HelloWorld class from ServicedComponent to make use of the COM+ services that are encapsulated in ServicedComponent class.

public class HelloWorld : ServicedComponent
{

In this step, we declare the ReturnHelloWorld() method that returns a string back to the caller.

public string ReturnHelloWorld()
{
try
{

Now, we call the SetComplete method of the static ContextUtil class to commit our transactions and as well as to deactivate the object. In the Authors class, we achieved the similar functionality by decorating the AddAuthor method with the AutoComplete attribute, which took care of automatically calling either SetComplete (when the method returns successfully) or SetAbort (when exceptions occur) for us.

   ContextUtil.SetComplete()
            ;
return "Hello World";
}
catch(Exception
            ex)
{

In this line, we call SetAbort to roll back any database changes that we might have done in any of the previous steps.

ContextUtil.SetAbort();
throw ex;
}

Then we will compile the application to create an assembly. This time around, we will manually register the object with COM+ using the regsvcs command line utility.

In the above command, the switch /c indicates that the COM+ application has to be created for the first time and if the application already exists, an exception is thrown. If the assembly is successfully registered with COM+, you should be able to see a new application called HelloWorldApplication that looks something similar to the following, in COM+ Services explorer.

Implementation of HelloWorldDelegate

Once we successfully register the object with COM+, the next step is to expose this object as a web service. To be able to make a method a web callable (web method) one, there are two important requirements that should be met.

  • The method should be decorated with the keyword WebMethod, which indicates that this method is web-callable.
  • The class that contains the method should be derived from WebService class.

In the above example, we have already derived HelloWorld class from ServicedComponent for leveraging COM+ services. Since the CLR does not allow a class to be derived from more than one class, we cannot derive the HelloWorld class from the WebService class. To overcome this constraint, we will create a separate class called HelloWorldDelegate, which simply delegates method calls to the underlying HelloWorld class. Because of its delegating nature, let us call this class as a delegate and we derive this delegate class from the WebService class.

namespace HelloWorldComponentDelegate
{
using System;
using HelloWorldComponent;
using System.Web.Services;

public class HelloWorldDelegate : WebService
{

The attribute [WebMethod] indicates that the HelloWorld method is to be exposed as a web method and when it is deployed, Visual Studio.NET runtime provides all the plumbing required to make this method discoverable to the clients who wish to use this web service.

[WebMethod]public string HelloWorld()
{

In this line of code, we create an instance of the HelloWorld object:

HelloWorld oHelloWorld = new HelloWorld();

Here, we delegate the call to the ReturnHelloWorld method of our HelloWorld object that is hosted in COM+.

return(oHelloWorld.ReturnHelloWorld());
}

Finally, let us build the application into a DLL.

Implementation of HelloWorldDelegateWebService

Now that we have a class that is derived from the web service class and a method that is decorated with the keyword WebMethod, it is very simple to expose this object as a web service. For an introduction to web services, please take a look at the article Web Services with ASP.NET from MSDN.

Let us create a new C# web service project and call it HelloWorldDelegateWebService. We can then rename the default web service (WebService1) to HelloWorldWebService and also add a reference to the HelloWorldDelegate assembly, which we created in the previous step. We will then open the HelloWorldWebService.asmx file and modify its contents to look like the following:

<%@ WebService Language="c#" Class="HelloWorldComponentDelegate.HelloWorldDelegate" %>

That is all we have to do for exposing a precompiled object as a web service. If you compile your web service application and navigate to the HelloWorldWebService.asmx page, you should see a screen similar to the following. This is a test harness, which ASP.NET creates for testing the web services:

Clicking the Invoke button brings up the following screen, which shows the result of the HelloWorld web method's execution.

Prerequisites

To execute the code that is available for download, you need to have the following software installed on your machine:

  • Windows 2000 Professional/Server with Service Pack 1
  • SQL Server 2000
  • Visual Studio.NET Beta2

Conclusion

In this article, we have discussed how to leverage COM+ services from managed environment using the classes supplied by the .NET framework. We have also seen how easy it is to expose a precompiled .NET object as a web service, which allows clients to consume the exposed functionality through Internet protocols like XML and SOAP.

I hope you find this article useful and thank you for reading.

C#Today - © 2002 WroxPress Limited


Creating an Order Tracking Application using .NET Framework

Creating an Order Tracking Application using .NET Framework
http://www.csharptoday.com/content.asp?id=1598
Article Rating: 4
Published on June 28, 2001
Abstract
With an ever–increasing amount of business being done on the web, it becomes a challenge for organizations to determine how to best take advantage of the enormous new business opportunities that exist on the Web. The Microsoft .NET framework provides an ideal platform for the Internet and exposes a unified and simplified programming model for developers to create rich Next Generation web applications. It also provides all the underlying plumbing that is required for implementing asynchronous but still reliable solutions for the internet thereby improving developers' productivity. In this article, Thiru Thanganathiram demonstrates how some of the new features of .NET framework can be used to create an asynchronous and reliable Order Tracking mechanism for a real time E–commerce website.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

With an ever-increasing amount of business being done on the web, it becomes a challenge for organizations to determine how to best take advantage of the enormous new business opportunities that exist on the Web. The Microsoft .NET framework provides an ideal platform for the Internet and exposes a unified and simplified programming model for developers to create rich Next Generation web applications. It also provides all the underlying plumbing that is required for implementing asynchronous but still reliable solutions for the internet thereby improving developers' productivity. In this article, we will see how some of the new features of .NET framework can be used to create an asynchronous and reliable Order Tracking mechanism for a real time E-commerce website.

Business Scenario

In this article, we will implement an asynchronous Order Tracking mechanism for a fictitious Web site. The simple web site used for the demonstration consists of 3 web pages.

  • The first page shows the list of books available in the site.
  • When the user clicks a book, the user is taken to the page where the user enters all the details required for submitting the order.
  • Once the user submits the details, the confirmation page is shown.

It is assumed that the user, who places the orders through the site, already has an account number assigned to him at the time of the registration.

If we were to handle the order submission in a synchronous way, the user would have to wait till the order information is stored in the database because the control will not be returned to the application till the changes are committed. This might result in increase in the response time for the resultant page to be drawn on the user's browser.

Since we are going to handle the order submission in asynchronous fashion, the control will be returned to the application as soon as the order details are captured in an xml file. After that, a separate set of operations will be used to carry out the tasks such as saving the details to the database and sending email to the user. This kind of handling not only helps in increasing the performance but also provides the users with the best possible browsing experience.

How the Asynchronous Order Workflow works

When the user places the order in the site, a number of independently running operations are used to complete the order request.

  • The user navigates to the ASP.NET web page (Web form) and places the order by entering all the details
  • The submitted request is processed by an ASP.NET web page that converts the Order information into an .xml file and stores it in a specified directory. Then it gives the message to the user that the Order Information has been submitted.
  • This specified directory is already set up for monitoring for creation of new files by the FileSystemWatcher component, which is placed in a Windows Service Application that always runs as a service.
  • When an .xml file is created in that directory, an event is raised to the Windows Service application that passes the path of the file to a C# Transacted object that runs in Component Services in Windows 2000.
  • C# object takes the path of the .xml file, parses the file and stores the details in a table called Orders in the SQL Server database and it also sends an Email to the user with the order number indicating that the order has been submitted.

Before we start looking at the code, let me briefly introduce some of the new .Net features such as Windows Service, FileSystemWatcher and .NET Framework XML API that we will be using in this application.

Windows Services

Prior to Visual Studio.Net, if you wanted to write a windows service application you either had to use the template provided by ATL, or embed custom NT service controls in VB to achieve the same functionality. But with Visual Studio.Net, you can easily create an application that has the capability to run as a Service. Visual Studio.Net is supplied with a new project template called Windows Service that provides all the plumbing required for creating the application that can run as a Service. When you create a Visual Studio.Net project as a Service, you can write code to respond to the actions that should happen when the service is started, paused, resumed and stopped.

Once you create the service, it has to be installed using the command line InstallUtil.exe utility which is used to install the service in the local computer. Once we install the service, we can start, stop, pause and resume it using Service Control Manager.

FileSystemWatcher Component

One of the exciting new features of the .NET framework is the ability to create applications that can interact with file system and raise events at scheduled intervals based on the activities in the file System. The FileSystemWatcher component that is contained in the System.IO namespace exposes methods and properties through which we can watch for changes to directories and files and take specific actions when these changes occur.

The FileSystemWatcher component can be created in two different ways:

  • You can drag an instance of the FileSystemWatcher component from the Components tab of the Toolbox to the designer.
  • You can create an instance of the FileSystemWatcher component programmatically after setting reference to the System.IO assembly.

More information on FileSystemWatcher component can be found in the following MSDN link.
http://msdn.microsoft.com/library/dotnet/cpguide/cpconconfiguringfilesystemwatchercomponentinstances.htm?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq

.NET Framework XML API

The .NET Framework provides a set of classes for developers to create applications based on xml. These xml classes are contained in the System.XML namespace. The methods and properties of the classes present in the System.XML namespace are similar to its predecessor, the MSXML parser, which it is designed to eventually replace.

All the .Net framework xml classes are derived from two abstract classes called XmlReader and XmlWriter:

  • XmlReader provides a fast, forward-only, read-only cursor for reading an xml document and
  • XmlWriter provides a mechanism for producing xml document streams that conformstreams.

The output produced by these classes conforms to the W3C recommendations. In this article, we will be using XmlTextWriter and XmlTextReader for writing into and reading from an xml file.

For more information on Xml classes in .NET Framework, please take a look at the following article from MSDN:
http://msdn.microsoft.com/library/periodic/period01/xml0101.htm?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq.

Implementation of the System

We will split the whole application into three different parts:

  • User Interface Tier (ASP.NET Web Forms) - Created using Visual C#.
  • Windows Service Application - Created using Visual Basic.Net.
  • C# Transacted Class that runs in Component Services in Windows 2000.

For this application we use a table called Orders to store the details of the order.

Implementation of User Interface Tier (ASP.NET Web Forms)

The Web Application consists of 3 different web forms. To create a Web Application in Visual C#, select File->New Project and then select Visual C# Projects from the Project Types and select Web Application from the List of templates.

Once you create the project, right click on the project name in Solution Explorer to add reference to the System.Xml assembly that is required for making use of the .Net Framework Xml classes. We use the xml classes for creating an .xml file on the fly when the user places an order in the site.

The first page BookList.aspx shows the list of books available in the site for purchasing.

The code for BookList.aspx is available for download and it is simple except it uses one of the new server side controls called <asp;hyperlink runat=server> control to navigate to another page when the user clicks on the hyperlink of the name of the book. QueryString parameters are also passed along the way to the GetOrders.aspx.

For example, the <asp:hyperlink> control for the book name Professional ASP.NET looks like this:

<asp:hyperlink navigateurl="GetOrders.aspx?ISBN=1861004885&Name=Professional ASP.NET"
id="lnkProfASP" runat="Server">Professional ASP.NET
</asp:hyperlink>

The navigateurl property identifies the path of the url to be navigated to when the user clicks on the hyperlink.

When the user clicks on a particular book, the user is taken to the web form GetOrders.aspx where he can place an order for that book by entering the account number, (which is assigned to every user who purchases books through this site), Email address and all the necessary details.

The screenshot of the GetOrders.aspx page looks similar to this:

Lets dissect the code of GetOrders.cs which is the code-behind file for the GetOrders.aspx page:

using System;
using System.Collections;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Web;
using System.Web.SessionState;
using System.Web.UI;
using System.Web.UI.WebControls;
using System.Web.UI.HtmlControls;

In this line, we import the System.Xml assembly into our application.

using System.Xml;

In the Page_Load event, we capture the QueryString parameters from the Params collection that are passed from the BookList.aspx and assign them to appropriate controls.

protected void Page_Load(object sender, EventArgs e)
{
//The following line returns true when the browser hits the page for the first time.
if (!IsPostBack)
{

lblBookTitle.Text = Request.Params["Name"];
lblISBN.Text = Request.Params["ISBN"];

}

}

Whenever the user enters all the details and clicks the Place Order button, the following lines of code are executed in the server side.

public void btnPlaceOrder_Click (object sender, System.EventArgs e)
{

XmlTextWriter objWriter = null;
string strDocument;

Here, we enclose all the executable statements in a try..catch block to handle any exceptions that may occur during the execution of those statements.

try
{

This line of code initializes a new instance of the Guid class and obtains the new Guid (Globally Unique Identifier) by invoking the ToString method of that class. Every time a customer places an order, an uniquely generated Guid is used to name the xml file.

strDocument = System.Guid.NewGuid().ToString() + ".xml";

We create the .xml file on the C:\Orders directory with the name of the file being the value contained in the variable strDocument.

objWriter = new XmlTextWriter("c:\\Orders\\" + strDocument, null);

To format the XML Document, we assign the Indented value to the Formatting property of the XmlTextWriter. When this option is set, child elements are indented using Indentation and IndentChar properties. IndentChar property decides which character to use for Indenting and Indentation property determines how many IndentChars to write for each level in the hierarchy.

Here we use the default values for Indentation and IndentChar, which are 2 and space respectively.

objWriter.Formatting = Formatting.Indented;

We use the StartDocument to write out the XML declaration with the version and the standalone attributes. The version attribute will be set to "1.0" and the standalone will have the value "no" as we pass false as an argument to the WriteStartDocument method.

objWriter.WriteStartDocument(false);

In this line, we write a Comment to indicate that the Order Tracking Application has created this file.

objWriter.WriteComment("This file represents order placed by the user");

Here we write out the specified start tag with the name of the element as order.

objWriter.WriteStartElement("order", null);

We write all the following elements as Child Elements of the order element and these elements are all of type string.

objWriter.WriteElementString("firstname", txtFirstName.Text);
objWriter.WriteElementString("lastname", txtLastName.Text);
objWriter.WriteElementString("email", txtEMail.Text);
objWriter.WriteElementString("accountnumber", txtAccountNumber.Text);
objWriter.WriteElementString("isbn",lblISBN.Text);

Then we close the order element by using the WriteEndElement method.

objWriter.WriteEndElement();
//Write the XML to file and close the objWriter
objWriter.Flush();
objWriter.Close();

The xml file that was created dynamically looks like this:

After we write the file, we transfer the control to the ShowResult.aspx that is used to show the final confirmation page.

Server.Transfer("ShowResult.aspx?BookTitle=" + lblBookTitle.Text + "&ISBN=" + lblISBN.Text);

}
catch (Exception ex)
{

We throw any errors that may occur during the execution of the above statements.

throw ex;
}

The statements in finally block are always executed regardless of whether the statements in the try block are executed successfully or not.

      finally
{

if (objWriter !=
            null)
objWriter.Close();
}
}

Once the user enters the details and submits the order, the user is redirected to the final confirmation page that looks something similar to this.

Implementation of Windows Service Application

To create a Windows Service application in Visual Basic.NET, select File->New Project and then select Visual Basic Projects from the Project Types and select Windows Service from the List of templates.

Once you create a new project, go to the properties of the Default Service project called WindowsFileSystemService, open up the default service (Service1) that is created and rename it to FileSystemService using its properties window. You can also set properties like CanStop and CanShutdown through the Properties window. These properties can also be set programmatically and these settings determine what methods can be called on your service. For example, when the CanStop property is set to true, the OnStop method can be called and the code that is associated with that method will be executed.

Let us briefly explore some of the theory behind the Windows Service applications.

The two main classes involved in Windows Service creation are:

  • System.ServiceProcess.ServiceBase We can override methods of this class when creating a service and define the code to determine how the service functions . The ServiceBase class exposes the following methods and we can override these methods are overridden to determine what happens when we change the state of the service (like starting, stopping, etc) in the Services Control Manager.

We can write code in the following events depending on the requirements.

  • OnStart - Invoked when the service is started.
  • OnPause - To indicate what should happen when the service is paused.
  • OnStop - To indicate the behavior of the service when the service stops running.
  • OnContinue - To decide the behavior that should happen when the service resumes normal functioning after being paused for a while.
  • OnShutDown - To indicate what should happen just prior to system shutdown.

The Run method of the ServiceBase class is the main entry point for the service. When we create a service using the Windows Service template in Visual Studio.Net, it automatically inserts a line in the Sub Main method to run the service. Let's quickly look at the code of this method:

Shared Sub Main()

Here, an Array of type ServiceBase is declared:

Dim ServicesToRun() As System.ServiceProcess.ServiceBase

All the Services contained by the application can be added and run together. Since we are creating only one service, we add only the UserService1 to the application.

ServicesToRun = New System.ServiceProcess.ServiceBase() {New UserService1()}

Finally, the Run method of ServiceBase is invoked passing in the array of Services created in the previous step as an argument.

System.ServiceProcess.ServiceBase.Run(ServicesToRun)
End Sub

  • System.ServiceProcess.ServiceProcessInstaller and System.ServiceProcess.ServiceInstaller These classes are used to install and uninstall the service. ServiceProcessInstaller is used when the Windows Service project contains more than one service. To add these classes to your Windows Service right click on the Design Editor of your service and select Add Installer from the Context menu and Visual Studio.Net automatically adds these two classes to the project and allows you to install or uninstall the Service by using InstallUtil.exe utility.

After we set all these options, we have to add a reference (By right clicking on the project name in the Solution Explorer) to the C# Class Library Application which we will be creating in the later part of this article.

We add this line of code to import our C# Class Library namespace into the application:

Imports WinFileSystemServiceComponent

Public Class UserService1
Inherits System.ServiceProcess.ServiceBase

The WithEvents keyword specifies that the variable mobjFileSystemWatcher that will be used to handle the events raised by the FileSystemWatcher component.

Private WithEvents mobjFileSystemWatcher As FileSystemWatcher

All the variables declared with the keyword WithEvents should be declared at the module level. As soon as you declare a variable with the keyword WithEvents, the variable mobjFileSystemWatcher appears in the left-hand drop down of the Code Editor with the right hand side drop down having all the event procedures of FileSystemWatcher class. From the right hand side drop down, you can select either the Created, Deleted, Renamed or Changed events and write code to handle these actions.

Here we create an instance of the Orders class and call the AddOrder method passing in the path the file that is created as an argument.

Public Sub mobjFileSystemWatcher_Created(ByVal sender As Object,
ByVal e As System.IO.FileSystemEventArgs)
Handles mobjFileSystemWatcher.Created

Dim objOrders As New Orders()
objOrders.AddOrder(e.FullPath)
End Sub

Code in the OnStart method is executed when the Service is started from the Service Control Manager.

Here we set all the properties of the FileSystemWatcher object so that it starts monitoring the Orders directory for creation of new .xml files.

Protected Overrides Sub OnStart(ByVal args() As String)

mobjFileSystemWatcher = New FileSystemWatcher()

We set the path to indicate that we want to monitor the Orders directory for new .xml files.

mobjFileSystemWatcher.Path = "C:\Orders\"
mobjFileSystemWatcher.Filter = "*.xml"

WatcherTarget enumeration is used to indicate that we want to watch for a file, directory, or both files and directories. But only one member in the enumeration can be used at a time.

Here, we set the value to File since we want to monitor the directory for the creation of new Files.

mobjFileSystemWatcher.Target = WatcherTarget.File

As we are interested in watching only for files not directories or subdirectories, we set the IncludeSubdirectories property to false.

mobjFileSystemWatcher.IncludeSubdirectories = False

The members of the ChangedFilters enumeration can be combined to watch for more than one type of change at a time. For example, you can watch for any changes in the size of a file or folder, and for any changes in the LastWrite time. This raises an event whenever there is a change in the file or folder size, or a change in the LastWrite time of the file or folder.

mobjFileSystemWatcher.ChangedFilter = ChangedFilters.LastWrite

Finally, we enable the component to begin watching for the creation of new files.

mobjFileSystemWatcher.Enabled = True
End Sub

Now that we have created the Windows Service application, it has to be installed using InstallUtil.exe utility. We need to navigate to the directory path that contains the WindowsFileSystemService.exe file and then execute the following command from the command prompt.

InstallUtil WindowsFileSystemService.exe

To uninstall the existing service, we add the switch /u to the above command.

InstallUtil /u WindowsFileSystemService.exe

Once we install the service, go to Start ->Control Panel -> Administrative Tools -> Services and then select the FileSystemService to start, stop or pause or resume the service.

Implementation of C# Transacted Class that runs in Component Services

To create a Visual C# Class library, select Visual C# from the Project Types in the left hand side and select Class Library from the list of project templates from the right side.

Once we create the project, we set reference to the following assemblies: System.IO, System.Xml, System.Data and System.Web.

The code for the C# Transacted class is available for download along with the support material for this article and it goes like this:

using System;

We import the Microsoft.ComServices namespace to leverage COM+ Services like Object Pooling and COM+ Transactions. The .NET Framework provides a set of new classes that make COM+ Programming a whole lot easier. It also makes XCOPY Deployment possible, i.e.; if you want your component to run in a Transaction, you simply add a Transaction attribute to the top of the class, compile the component and copy the compiled component into the Bin Directory of the Client application.

When the object is activated for the first time from a .NET Application (Managed Code), it will automatically be registered with COM+ for you. However, if you want to create a new instance of the class from UnManaged Code, then the registration in the Component Services has to be done manually.

using Microsoft.ComServices;

The System.Data namespace contains all the ADO.NET classes required for accessing the backend database. For an introduction to ADO.NET, please refer to the article ADO.NET for the ADO Programmer.

using System.Data;
using System.Data.SQL;

The System.Web.Util namespace contains an SmtpMail class that is used for describing properties of an Email message and also for sending the Email.

using System.Web.Util;

The System.Xml is the overall namespace for all the XML Classes that provide universally accepted standards based support for XML Processing.

using System.Xml;

Next we specify the COM+ Application's name as it appears in the COM+ catalog:

[assembly: ApplicationName("OrderTracking")]

We define the namespace for this application as WinFileSystemServiceComponent

namespace WinFileSystemServiceComponent

The Transaction attribute is used to set the type of automatic transaction requested by the component. TransactionEnum can take any one of these values: Ignored,

None, Supported, Required and RequiresNew. For more information on Transaction Attributes in COM+, Please refer to Windows 2000 Brings Significant Refinements to the COM(+) Programming Model by Don Box.

We set the Transaction Attribute to Required which indicates that if the caller of this object runs in a Transaction, it will be inherited by this object, otherwise it will create a new Transaction for itself and run within the scope of that transaction.

[Transaction(TransactionOption.Required)]

Since the Orders class derives from the ServicedComponent class, it is automatically serviced by Microsoft Windows 2000 Component Services (COM+ Services). Serviced Components are generally hosted in COM+ Services as an In-Process Server (i.e.; as a Library Application).

public class Orders : ServicedComponent
{

The AutoComplete attribute indicates that the transaction governing the object will automatically call SetComplet() should the method call return successfully. If the method call throws an Exception, the transaction will be aborted.

[AutoComplete]

The AddOrder method takes the path of the XML File to be processed as an argument.

public String AddOrder(string strPath)

{

XmlTextReader objReader;
XmlDocument objDocument;
XmlNode objOrderNode;
SQLParameter objParam ;
SQLCommand objCommand;
SQLParameter objParam;
MailMessage objMailMessage;

Now, we create an instance of the SQLConnection Object passing the ConnectionString as an argument.

SQLConnection objConnection =
new SQLConnection("server=localhost;uid=sa;pwd=;Initial Catalog=Home");

We enclose the following statements in a try..catch block to catch any errors that may occur during the execution of those statements.

      try
{
objReader
            = new XmlTextReader(strPath);
objDocument = new
            XmlDocument();
objDocument.Load(objReader);

The DocumentElement property of the XmlDocument is used to get reference to the root XmlElement of the Document.

objOrderNode = objDocument.DocumentElement;

Get the values of the First name, Last name, Email, Account Number and ISBN from the Child Nodes of the Order node.

strFirstName = objOrderNode.ChildNodes.Item(0).InnerText;
strLastName = objOrderNode.ChildNodes.Item(1).InnerText;
strEMail = objOrderNode.ChildNodes.Item(2).InnerText;

The FromString method of Int32 returns a 32 bit signed integer that is equivalent to the number specified in the String.

intAccountNumber = Int32.FromString(objOrderNode.ChildNodes.Item(3).InnerText);

strISBN = objOrderNode.ChildNodes.Item(4).InnerText;

We assign the name of the Stored procedure to be executed to a local variable. This stored procedure inserts a record into the Orders table.

strCommandName = "spInsertOrder";

This line instantiates a SQLCommand object passing the Name of the Stored procedure to be executed and the SQLConnection object that was created in the previous step.

SQLCommand objCommand = new SQLCommand(strCommandName, objConnection);

By setting the CommandType property of the objCommand object, we indicate that we want to execute a Stored Procedure.

objCommand.CommandType = CommandType.StoredProcedure;

The SQLParameter object represents a parameter to a SQLCommand. Here, we create an instance of SQLParameter passing in the arguments that describe the parameter being passed to the stored procedure.

objParam = objCommand.Parameters.Add(new SQLParameter("@ISBN", SQLDataType.VarChar, 50));

We set this Direction property to specify that it is an input only parameter. Other values that can be assigned are output only, bidirectional, or a return value parameter.

objParam.Direction = ParameterDirection.Input;
objParam.Value = strISBN;

We follow the same procedure for all the parameters except for the final one. This is used as an Output parameter to return the newly added Order Number from the Stored Procedure.

objParam = objCommand.Parameters.Add(new SQLParameter("@OrderNumber", SQLDataType.Int));

objParam.Direction = ParameterDirection.Output;

Open the ActiveConnection to the Database.

objCommand.ActiveConnection.Open();

ExecuteNonQuery of the SQLCommand object is used to execute a command against an ActiveConnection, and does not return any rows against the ActiveConnection.

objCommand.ExecuteNonQuery();

Once the Stored Procedure is executed, we retrieve the return value of the Stored Procedure from the Parameters collection of the SQLCommand object.

intOrderNumber = Int32.FromString((objCommand.Parameters["@OrderNumber"].Value.ToString()));

Here, we create an instance of the MailMessage object that is part of the System.Web.Util namespace to send Emails.

objMailMessage = new MailMessage();

Now, we set the properties of the MailMessage object.

objMailMessage.To = strEMail;
objMailMessage.From = "Orders@OrderTracking.com";
objMailMessage.Subject = "Order Details";
objMailMessage.BodyFormat = MailFormat.Html;
objMailMessage.Body = "<html><body>Your Order has been Recieved </b> Your Order Number is :"
+ intOrderNumber.ToString() + " </body></html>";

Finally, we send the mail the using the Send method of the SmtpMail using the values set in the MailMessage object.

SmtpMail.Send(objMailMessage);

}

If any errors occur during the execution of the above statements, we raise the exception back to the caller.

catch (Exception ex)
{
throw ex;
}

The statements in finally are always executed regardless of whether the statements in the try block are executed successfully or not. Here we inspect the state property of the SQLConnection to check whether the connection is open. If the Connection is open, it is closed.

finally
{

if ( objConnection.State
            == DBObjectState.Open
            )
objConnection.Close();
}

}

When this C# Transacted class is instantiated by the Windows Service application for the first time, it is automatically registered with Component Services Explorer.

To see how the COM+ auto-registration mechanism worked, open up the Component Services from Start->Settings->Control Panel->Administrative Tools. If we open up the COM+ Applications folder, we see the presence of the OrderTracking application. This was created in response to the Assembly attribute we specified in the previous step. You can also verify the Transaction attribute of the Orders class, which will have the value set to Required due to the TransactionOption enum value we set programmatically.

For more information on creating .NET components for Component Services in Windows 2000, please take a look at the following MSDN article. Microsoft .NET Framework Component Services, Part 1

Putting it all together

Now that we have put together the different parts of the application, let us test the application by navigating to the BookList.aspx file that is present in the OrderTracking directory. When you click on particular book, you will be taken to the GetOrders.aspx page where you enter all the order details. When you submit the order, an .xml file will be created on the fly in the C:\Orders directory and you will get the confirmation message that the order has been submitted. This dynamically generated file will be captured by the Windows Service application which in turn will invoke the method of the C# Transacted class to process the order.

Conclusion

In this article, we have seen the following:

  • How to create a .NET Application that can run as a Windows Service using Visual Studio.Net.
  • How to use the rich .NET XML classes available in System.XML namespace to write to an .xml file and read from an .xml file.
  • How to use the FileSystemWatcher component to monitor a directory or file for changes.

Although the application we have created was simple in functionality, it should provide a solid foundation for understanding how .NET platform can be leveraged to build reliable, flexible and scalable 24x7 internet applications in an asynchronous fashion.

C#Today - © 2002 WroxPress Limited


Web Site Error Logging and Reporting System

XML

XML (Sort by Name)

.NET Serialization
Building .NET Reporting Solutions with XML and GDI+
Building Windows Forms on-the-fly with XML and C#
Cool Tool for XSLT - Part 1
Cool Tool for XSLT - Part 2
Interoperating Java, VB6, and .NET Web Services
Introducing XGen XSQL for .NET
Mapping XML to C# objects using reflection
MSXML DOM and .NET DOM - A practical comparison of working with the DOM in MSXML 4.0 and C#.NET
Network Programming in C#

Owner Drawn Menus in Visual Studio .NET Using C#
Remote querying your data warehouse using web services and XML
Remoting in Cs - Part 1
SOAP Support in .NET
Working with XML as an Object Model
XML Querying Using .NET and SQL Server 2000


Network Programming in C#

Network Programming in C#
http://www.csharptoday.com/content.asp?id=1994
Article Rating: 4.5
Published on November 18, 2002
Abstract
This case study by Irfan Rizvi focuses on network programming in C# using sockets. Sockets provide a medium of communication between two applications, whether those applications reside on the same computer, two different computers connected to each other on local network, or two computers connected to the internet.
Case Study

This case study focuses on network programming in C# using sockets. Sockets provide a medium of communication between two applications, whether those applications reside on the same computer, two different computers connected to each other on local network, or two computers connected to the internet.

A Brief Introduction To Networks

There are two main protocols that are used in network related applications; both have their own advantages and disadvantages. TCP (Transmission Control Protocol) is a highly reliable connection-oriented protocol, but sends acknowledgements for each request and response and therefore is more time consuming. TCP is the most widely used protocol because of its reliability; it is the standard protocol of the Internet.

The other protocol is UDP (User Datagram Protocol), which is a connectionless protocol - this means no acknowledgements are sent after request or responses, and hence it is faster then TCP but less reliable. UDP is used in applications in which time is the critical factor.

If you are new to sockets, you might be interesting to know that almost all servers, browsers, and almost everything in a TCP/IP network environment are using socket technology in the background. The theory behind TCP/IP network communication is that every machine on a network has a unique address (its IP address). You might be wondering that if each computer has but a single address, how can they differentiate which data is for which application when multiple applications are running on a machine? For example, if you have two browser windows open, each requesting information from a different website, then how does the computer know which browser to send what data to? The answer to this is addressing, which is done on two levels.

The first level is NSAP (Network Service Access Point), usually called IP addressing, which identifies the machine. When the data reaches the correct machine, we need to find the correct SAP (Service Access Point). This is the second level, which is level addressing. Service Access Points are also called port addresses, and these uniquely identify each application. Sockets are made up of a unique combination of an IP address and a port address, allowing the correct port address on the correct machine to be found.

The main aim of this case study is to demonstrate network programming in C# using sockets and TCP. .NET provides base classes for TCP server and clients in its System.Net namespace, and there are several classes available to deal with various requirements of network applications.

Introduction

In this case study we will be using a simple client/server application to demonstrate how socket programming works. This application uses TCP for the communication.

In this application, the server lets multiple clients to connect to it. Once the client is connected, the server can view all of its information like name, IP address, user name, the port at which it is connected, and the protocol it is using.

The server can also request the list of processes running on the client, and can view information related to each of these processes such as process name, start time, etc. Similarly, the server can also request the list of services that are available on the client, and information relating to these. These services are the same ones that you can manually access via Control Panel.

The server can perform several operations on the clients such as pausing, resuming or stopping a service. Through the Stop Client operation, the server can also disconnect the client - that is, the client will no longer be connected to the network.

By reading this case study, you will gain valuable knowledge regarding sockets, TCP, and the knowledge required to start programming client server applications.

Networks In More Detail: Theory and Technology

In this section we will take a detailed look at what networks are, how they work, and how the .NET architecture supports network programming.

OSI Reference Model

The OSI (Open Systems Interconnections) model is based on the proposal developed by the International Standards Organization (ISO) as a first step towards an international standardization of the protocols used. This model is called the Open System Interconnections Reference Model because it deals with connecting open systems, that is, systems that are open for communication with other systems.

The OSI Model has seven layers which are described briefly as follows:

Physical Layer: This layer handles the transmission of bits over the network. It provides the physical connection for the transmission of data among the network devices. The physical layer is responsible for making sure that data is read the same way on the destination device as it is sent from the source device. For example, the physical layer specifications include the amount of voltage on a cable, how a signal changes to signify a 1 or a 0 being sent, and in what order a signal is sent.

Data Link Layer: The main purpose of this layer is to provide a reliable method of transmitting data across the physical media. This layer breaks the input data into frames, transmits the frames sequentially, and processes the frames when they reach are received. It adds a header and trailer to the frames it creates - this allows the destination device to see when a frame begins or ends.

Network Layer: This layer is responsible for routing information from one network device to another; the network layer decides what path data will take if the destination device is located on another network. It accepts messages from the source, converts them into packets, and make sure that the packets are directed towards the destination. It is also responsible for deciding on the best route the packet should take through the network

Transport Layer: The transport layer is the fourth layer of the OSI model. It provides a transport service between the session layer and the network layer. This service takes information from the session layer and splits it up into packets if necessary, then passes this information to the network layer and checks to make sure the information arrived at the destination device successfully. The transport layer adds a header that includes error detection (to detect corruption of data), sequencing (as the message is split into several packets, this allows it to be re-assembled correctly), and flow control messages (which are used to make sure the destination is receiving information correctly).

Session Layer: This layer lets users establish a connection (called a "Session") between devices. Once the connection has been established, the session layer can manage the dialog. To establish a session, a user must provide the remote address they want to connect to. These addresses are not like the network addresses; they are intended for the user, and are easier to remember.

Presentation Layer: The presentation layer establishes the format in which data is exchanged. This layer is responsible for character set or numeric translations needed. It is also responsible for data compression to reduce the amount of data transmitted, as well as encryption.

Application Layer: The top layer of the OSI model is the application layer. This layer is the interface between the user's applications and the network. It allows the application the user sees to transfer files, send emails, and to do anything else it needs to on the network.

TCP/IP Protocol Suite

The Transmission Control Protocol/Internet Protocol (TCP/IP) suite, also known as the Internet Protocols, is a collection of industry-standard protocols. The TCP/IP suite is made up of many protocols, not just TCP and IP, and has a broad feature set due to its large number of open-standard protocols. Over the years, the individual components have evolved to handle almost any need a network user may have.

The original designs for TCP/IP (Transmission Control Protocol/ Internet Protocol) were stated long before the OSI model was developed; instead of the OSI's seven layers, TCP/IP has just four layers. The four layers can be loosely matched to the OSI model as follows:

Network Access Layer: This layer corresponds to the physical and data link layers of the OSI model.

Internet Layer: This layer roughly matches up with the network layer of the OSI model. Both of these layers are responsible for moving data to other devices on the network.

Host-to-Host Layer: This one is similar to the transport layer of the OSI model. The job of both of these layers is to communicate between peers on the network. As a result, almost all devices on a TCP/IP network are considered host, whether they are workstations, servers, or network-attached printers.

Process/Application Layer: This fourth layer does the same job as the top three layers of the OSI model, which is to provide network services.

TCP (Transmission Control Protocol)

In client-server applications, the server will provide some service such as processing database queries or sending out current stock prices. The client uses the service provided by the server, in these cases to either display database query results to the user or to make stock purchase recommendations to an investor. The communication that occurs between the client and the server must be reliable. That is, no data can be dropped, and it must arrive at the client side in the same order in which the server sent it.

TCP provides a reliable, point-to-point communication channel that client-server applications on the Internet use to communicate with each other. TCP was specifically designed to provide a reliable end-to-end byte stream over an unreliable internetwork. An internetwork differs from a single network because different parts may have widely different topologies, bandwidths, delays, packet sizes, and other parameters. TCP was designed to dynamically adapt to properties and to be robust in the face of many kinds of failures.

To communicate over TCP, a client program and a server program establish a connection to one another. Each program binds a socket to its end of the connection. To communicate, the client and the server each reads from and writes to the socket bound to the connection. The client and server send an IP packet to each other; this packet contains the TCP header that includes flow control, sequencing, and error checking (as touched upon earlier).

Each client is given the same port number of the server so that all the data being sent reaches the correct virtual connection. A port is similar to a mailbox. When data is delivered to a computer via the network, it must be sent to a process on the computer. A computer may have multiple processes running on it. Each service that expects data from the network will register itself onto a port number. For example, an internet web server uses port 80, while a mail server uses port 25. Inside the header of the network packet is a port number for which the data is intended, similar to a postal address for a letter.

Sequence Numbers: TCP uses a 32-bit sequence number that counts bytes in the data stream. Each TCP packet contains the starting sequence number of the data in that packet, and the sequence number (called the acknowledgment number) of the last byte received from the remote peer. Forward and reverse sequence numbers are completely independent, and each TCP peer must track both its own sequence numbering and the numbering being used by the remote peer.

Window Size and Buffering: Each endpoint of a TCP connection will have a buffer for storing data that is transmitted over the network, waiting until the application is ready to read the data. This lets network transfers take place while applications are busy with other processing, which improves overall performance. To avoid buffer overflow, TCP sets a Window Size field in each packet it transmits. This field contains the amount of data that may be transmitted into the buffer. If this number falls to zero, the remote TCP can send no more data. It must wait until buffer space becomes available and it receives a packet announcing a non-zero window size.

What is an IP Address?

In order for two applications to make a connection, they need to be able to identify each other. Computers on networks have a unique identifier called an IP address which is represented in dot-notation, an example would be 10.20.120.127. No two computers on the same network can have the same IP address. As we discussed earlier, the IP address consists of a network address and a host address. The network address represents the network to which this machine belongs, while the host address is the address of the machine itself. IP addresses can either be dynamic or static. There are some IP addresses that are used for special proposes, such as 127.0.0.1, which is known as the loop back address, etc.

What are Ports?

IP addresses are network addresses which identify a particular machine on a network. However, there are many applications running on a single machine, many of them communicating over the network, so each of these applications need an address to uniquely identify them. The solution to this problem is ports. Ports consist of an integer address which ranges from 1 to 65536, and each application on a machine has a port address. Servers usually have standard port addresses which are used to handle clients. For example, all web servers have a default port address of 80.

What are Sockets?

A socket is a unique combination of an IP address and a port number, identifying a destination that data can be sent to.

Normally, a server runs on a specific computer and has a socket that is bound to a specific port number. The server just waits, listening to the socket for a client to make a connection request. The client knows the hostname or the address of the machine on which the server is running, and the port number to which the server is connected. To make a connection request, the client tries to rendezvous with the server on the server's machine and port. If everything goes well, the server accepts the connection. Upon acceptance, the server gets a new socket, bound to a different port. It needs a new socket (and consequently a different port number) so that it can continue to listen to the original socket for connection requests while tending to the needs of the connected client.

On the client side, if the connection is accepted, a socket is successfully created and the client can use the socket to communicate with the server. Note that the socket on the client side is not bound to the port number used to rendezvous with the server. Rather, the client is assigned a port number local to the machine on which the client is running. The client and server can now communicate by writing to, or reading from, their sockets.

Socket Programming in C#

Network programming in Windows is possible using socket communication. A socket is like a handle to a file. You can use socket programming to have two applications communicate with each other. The applications are typically on different computers, but they can be on the same computer, or even the same process or thread of the same application.

For the two applications to talk to each using sockets, either on the same or different computers, one application is generally a server that keeps listening to the incoming requests and the other application acts as a client and makes the connection to the server application. The server application can either accept or reject the connection. If the server accepts the connection, a dialog can begin between the client and the server. Once the client is done with whatever it needs to do, it can close the connection with the server. Connections are expensive in the sense that servers allow finite connections to occur. This is because there are 65536 available ports on a machine, and another reason is that usually servers create a thread for each client, and as the number of client grows, large number of threads are running simultaneously, which will consume the server's resources.

During the time the client has an active connection, it can send data to the server and/or receive data. The complexity begins here. When either side (client or server) sends data, the other side is supposed to read the data. But how will the other side know when data has arrived? There are two options - either the application needs to poll for the data at regular intervals, or there needs to be some sort of mechanism that would enable applications to get notifications when new data arrives. Well, Windows is an event driven system, so the notification system is the obvious best choice.

The System.Net namespace

The .NET Framework provides a wide range of excellent features that developers can take advantage of while building networking applications. The System.Net classes are similar to Microsoft's WinInet API (used for internet protocol programming) in that they allow applications to get and send data using Internet protocols. The .NET classes are quite different in design and function however, the main difference being that the .NET classes are designed with an emphasis on standing up to a high-stress server environment and present an object based API model, while the WinInet was a functional API.

The System.Net namespace provides a simple programming interface for many of the protocols used on networks today. The WebRequest and WebResponse classes form the basis of what are called pluggable protocols, an implementation of network services that enables you to develop applications that use Internet resources without worrying about the specific details of the individual protocols.

The System.Net namespace contains a number of classes that can be used to communicate with resources on the Internet.

  • Socket programming can be done through System.Net.Sockets.Socket class.
  • Web resources can be downloaded by using the System.Net.WebRequest class.
  • Use System.Net.IPAddress to represent an IP address. You can use System.Net.Dns to resolve the IP address to a name ( System.Net.IPHostEntry ) or vice versa.
  • System.Net.WebProxy objects can be used to pass requests through a proxy server.

Since our case study only focuses on the System.Net.Sockets.Socket class, we will discuss only this class here.

System.Net.Sockets.Socket

The Socket class creates a managed version of an Internet transport service. Once the Socket is created, the Socket is bound to a specific endpoint through the Bind method, and the connection to that endpoint is established through the connect method. Data is sent to the Socket using the Send or SendTo methods, and data is read from the Socket using the Receive or RecieveFrom methods. After you are done with the Socket, the ShutDown method is used to disable the Socket, and the Close method to close the Socket.

The Socket class is used by the Microsoft .NET Framework to provide Internet connections to TcpClient, UdpClient, WebRequest, and the derived classes. We will only be discussing TcpListener and TcpClient here.

TcpListener: The TcpListener class builds upon the Socket class to provide TCP services at a higher level of abstraction. You can use a TcpListener to listen for connections from TCP clients. Application level protocols such as FTP and HTTP are built on the TcpListener class. You can create an instance of the TcpListener class by providing one of the following constructor argument lists: an IPEndPoint containing your local IP address and port number, your IP address and port number, or just your port number (in this case, the default network interface will be used).

TcpClient: The TcpClient class builds upon the Socket class to provide TCP services at a higher level of abstraction. TcpClient provides simple methods for connecting, sending, and receiving data over a network. You can connect with a remote host using one of the following two ways:

  • Create an instance of the TcpClient class, using either no parameters or using a local IPEndPoint, and then calling one of the three available TcpClient Connect methods.
  • Create an instance of the TcpClient class using the host name and port number of the device to which you intend to connect. This constructor will automatically establish a connection.

To send and receive data, a GetStream method is used that obtain a NetworkStream that can send and receive data on the underlying connected Socket. After utilizing the write and read methods available through the NetworkStream, the Close method is used to release all resources associated with the TcpClient.

I/O Stream

Streams in C# usually involve reading and writing bytes to and from the console, the filesystem, or the network. In C#, the stream paradigm is used more generally whenever a program needs to move or operate on a group of bytes.

To read and write to and from the console, C# provides a single class called System.IO.Stream. Instead of having two objects, one for reading and one for writing, a C# Stream object needs to be tested for its capabilities via the CanRead and the CanWrite properties. This class has methods to operate on one byte at a time, along with methods that operate on an array of bytes.

NetworkStream: This implements the standard .NET Framework stream mechanism to send and receive data through network sockets. NetworkStream supports both synchronous and asynchronous access to the network data stream. NetworkStream does not support random access to the network data stream.

Applying Our Knowledge

We have now taken a detailed look at the theory behind networks, and you should now have a good knowledge of why and how they are used. We will spend the rest of this case study using this theoretical knowledge to practically implement our client\server application, showing how all its features work as we go.

A Note About Loopback

Please note before we begin that the application that we will be creating in this case study is network related, and so it is better that client and server should run on separate machines - but this is not absolutely necessary. You can run both server and client on the same machine using the Loopback address. This address is a special IP number (127.0.0.1) that is designated for the software loopback interface of a machine. The loopback interface has no hardware associated with it, and it is not physically connected to a network.

An Overview Of The Final Application

The application we will be creating looks like the following:

From this screen (on the server), the administrator will be able to view all the clients that are connected to the server (in the left hand box). Selecting a client will cause the relevant details (such as IP Address, Username, the communication protocol being used, etc) to be shown in the top right hand collection of boxes. In the bottom right hand box, the administrator can view either the list of processes or services currently being run on that client. By using the buttons in the toolbar at the top, the administrator can manipulate these, for example pausing, resuming or stopping a process.

Now that we have seen what the final application will look like, and seen what it will actually do, let's get down to the hard work of putting together the code for this project. By closely following the following code and explanation, you will gain a good understanding of how sockets work in C#, what namespaces are important, and some of the issues involved, such as dealing with byte streams.

Component Design and Coding

Server Description

The above diagram represents the class architecture of the server. The main class is Server, which is the start point of this application and handles the User Interface; it also starts a thread of serverListener class which will run in the background and handle the incoming client connection requests. Whenever a new client arrives, this thread will make an object of ClientInfo class with the information about that client, and pass this object to Server where it will be added to its collection. The serverListenerArgs class acts as a wrapper class for transporting ClientInfo instances from the serverListener class to the Server class. The ProcessObject class and ServiceObject class are representatives of a Process and a Service.

Now we are going to discuss each class individually:

Class Description: Server

The Server class actually handles the User Interface via events. Most of the code is generated by Visual Studio itself - we will not describe this generated code, as it is out of the scope of this case study.

Attributes

Name | Type | Access Modifier | Description
alClientInformation | ArrayList | Private | Container for all clients
slListener | serverListener | Private | Listener object
GUI object | System.Windows.Form.* | Private | All other form objects

Methods

private void
            InitializeComponent()
{
this.slListener = new
            serverListener(1024);
this.slListener.OnClientSignal += new
            serverListener.ClientSignal(this.ClientConnectionHandler);
}

Although this method is Visual Studio generated, some of it is worth mentioning:

this.slListener = new serverListener(1024);

This statement starts the server's listener thread by making an instance of the serverListener class, while

this.slListener.OnClientSignal += new serverListener.ClientSignal(this.ClientConnectionHandler);

registers the ClientSignal event with the ClientConnectionHandler method.

private void ClientConnectionHandler (object sender , serverListenerArgs args)
{
ClientInfo ciClient = args.ClientInfo;
alClientInformation.Add (ciClient);

lstClients.Items.Add(ciClient.MachineName);
}

This is the event handler of a new client connection. Whenever a new client request arrives, the listener throws a notification about this request, with all the information about that client in an object of serverListenerArgs which is inherited from EventArgs class. One thing worth mentioning here in the architecture of event driven programming is that all the event handlers in .NET receive two and only two arguments; the first one is of object type which describes the sender from whom the notification is coming, and the second is the one through which all the information about that particular event is sent. This second argument must be an object of EventArgs class or any class inherited from EventArgs. We will describe the serverListenerArgs class later.

Getting back on track, this method or event handler will be initiated whenever a client arrives. It receives the information about that client (via the parameters) and adds it to its ClientInformation container. The client's machine name is added to the list box to show which client has been connected.

private void lstClients_SelectedIndexChanged(object sender, System.EventArgs args)
{
ClientInfo ciClient = (ClientInfo) alClientInformation [lstClients.SelectedIndex];

tbMachineName.Text = ciClient.MachineName;
tbUserName.Text = ciClient.UserName;
tbIPAddress.Text = ciClient.StrIPAddress;
tbPort.Text = ciClient.PortNumber.ToString() ;
tbProtocol.Text = ciClient.SConnectedSocket.ProtocolType.ToString();

if(rdbProcesses.Checked)
{
lstServices.Items.Clear();
IEnumerator en = ciClient.ProcessList.GetEnumerator();
while (en.MoveNext())
{
ProcessObject processObject = (ProcessObject) en.Current;
lstServices.Items.Add(processObject.StrProcessName);
}
}
else if (rdbServices.Checked)
{
lstServices.Items.Clear();
IEnumerator en = ciClient.ServiceList.GetEnumerator();

while (en.MoveNext())
{
ServiceObject serviceObject = (ServiceObject) en.Current;
lstServices.Items.Add (serviceObject.DisplayName);
}
}
}

This is the lstClients event handler. Whenever the server admin selects a client's machine name, this event handler is executed and it displays that machine's information.

private void lstServices_SelectedIndexChanged (object sender, System.EventArgs args)
{
ClientInfo ciClient = (ClientInfo) alClientInformation[lstClients.SelectedIndex];

if (rdbProcesses.Checked)
{

ProcessObject processObject = (ProcessObject) ciClient.ProcessList[lstServices.SelectedIndex];
tbServiceName.Text = processObject.StrProcessName;
tbServiceType.Text = processObject.ProcessStartTime;
}
else
{
ServiceObject serviceObject = (ServiceObject) ciClient.ServiceList [lstServices.SelectedIndex];
tbServiceName.Text = serviceObject.DisplayName;
tbServiceType.Text = serviceObject.ServiceType;
tbServiceStatus.Text = serviceObject.Status;
}
}

This event handler executes when the user selects a particular process or service from the list box. It get the relevant information and displays it on screen.

private void
            rdbProcesses_CheckedChanged(object sender, System.EventArgs
            e)
{
if (rdbProcesses.Checked)
{
lbllist.Text = "List
            of Process";
lblname.Text = "Process Name";
lbltype.Text =
            "Process Type";

tbServiceStatus.Visible =
            false;
lblstatus.Visible = false;

if
            (lstClients.SelectedIndex >= 0)
{
ClientInfo ciClient =
            (ClientInfo) alClientInformation
            [lstClients.SelectedIndex];

lstServices.Items.Clear();
IEnumerator
            en = ciClient.ProcessList.GetEnumerator();

while
            (en.MoveNext())
{
ProcessObject processObject =
            (ProcessObject)
            en.Current;
lstServices.Items.Add(processObject.StrProcessName);
}
}
}
}

This event handler switches the on-screen display from service mode into process mode. So, if the user selects the "process" radio button, information about the highlighted machine's process are then shown on the screen.

private void
            rdbServices_CheckedChanged(object sender, System.EventArgs
            e)
{
if (rdbServices.Checked)
{
lbllist.Text = "List of
            Service";
lblname.Text = "Service Name";
lbltype.Text =
            "Service Type";
tbServiceStatus.Visible =
            true;
lblstatus.Visible = true;
if (lstClients.SelectedIndex
            >= 0)
{
ClientInfo ciClient = (ClientInfo)
            alClientInformation[lstClients.SelectedIndex];

lstServices.Items.Clear();
IEnumerator
            en = ciClient.ServiceList.GetEnumerator();
while
            (en.MoveNext())
{
ServiceObject serviceObject =
            (ServiceObject)
            en.Current;
lstServices.Items.Add(serviceObject.DisplayName);
}
}
}
}

This event handler is the opposite of rdbProcesses_CheckedChanged, in that it switches the view from process mode into service mode, as oppose to switching from service to process.

private void tlbClicked(object sender,
            System.Windows.Forms.ToolBarButtonClickEventArgs args)
{
if
            (args.Button.Equals(this.tlbOptions.Buttons[0]))
{
MessageBox.Show("stop
            client");
}
else if
            (args.Button.Equals(this.tlbOptions.Buttons[2]))
{
MessageBox.Show("Pause
            Service");
}
else if
            (args.Button.Equals(this.tlbOptions.Buttons[3]))
{
MessageBox.Show("Resume
            Service");
}
else if
            (args.Button.Equals(this.tlbOptions.Buttons[4]))
{
if
            (selectedClient != null && selectedService !=
            null)
{
bool result = selectedClient.stopService
            selectedService.DisplayName);

if
            (result)
{
selectedService.Status="Stopped";
lstServices_SelectedIndexChanged(sender,
            e);
}
else
MessageBox.Show("Operation Failed","Error
            Message");
}
else
{
MessageBox.Show("No Service
            selected","Alert");
}
}
else if
            (e.Button.Equals(this.tlbOptions.Buttons[6]))
{
MessageBox.Show("Stop
            Process");
}
}

The above method is the event handler for the toolbar. We are only going to discuss one operation here ( stopService ), as this will be enough to give you a good understanding of what is happening.

if (selectedClient != null && selectedService != null)
{
bool result = selectedClient.stopService(selectedService.DisplayName);
if (result)
{
selectedService.Status="Stopped";
lstServices_SelectedIndexChanged(sender, e);
}
else
MessageBox.Show("Operation Failed","Error Message");
}
else
{
MessageBox.Show("No Service selected","Alert");
}
}

Whenever the stopService button is pressed, the code above is executed. It calls the stopService method of the selected client and passes the selected service's name to it. What this stopService method actually does will be discussed in more detail later, for now we are just assuming that if the operation is performed successfully then true is returned, else false is returned.

Class Description: ServerListener

Attributes

Name | Type | Access Modifier | Description
ClientSignal | delegate | Public | Delegate for the new client connection event
OnClientSignal | <<Event>> ClientSignal | Public | Represents the event shown when a new client arrives
Server_Socket | TcpListener | Private | Listens for connections from TCP network clients
iPort | Int | Private | Specifies the port where the Listener is listening
bRunning | Bool | Private | Its used for starting and stopping the Listener

Methods

public serverListener(int
            prt)
{
try
{
iPort=prt;
Server_Socket=new
            TcpListener(iPort);
Server_Socket.Start();

Thread thr=new
            Thread(new
            ThreadStart(server_listen));
thr.Start();
}
catch(Exception
            e)
{
throw new Exception ("Error Occurred in
            Listener");
}

}

This initiates the Listener object and starts it through

Server_Socket.Start();

The difference between initializing and starting the TcpListener's object is that when initializing, the port is initialized/opened, but no operation can be performed at that moment. Actually the TcpListener's object starts working when its start method is called. The start method acts as an entry point for its object. This method also starts a Listener thread which continuously listens for any new clients while working in the background.

public void
            server_listen()
{
while(bRunning)
{
try
{
Socket
            sock=Server_Socket.AcceptSocket ();
ClientInfo clt=new
            ClientInfo(sock);
OnClientSignal(this , new
            serverListenerArgs(clt));
}
catch(Exception e)
{
throw
            new Exception("Error Occurred while listening for clients ");
}
}
}

This method accepts requests from the clients which arise as objects of the socket class.

Socket sock=Server_Socket.AcceptSocket();

In the statement above, the thread goes into a wait mode. This means that this statement will not get completely executed until a client socket arrives. When a new client does arrive, its socket is received into a local variable named sock, and then an object of ClientInfo class is instantiated by passing this socket object. After that, an event is thrown to notify the server class that a new client has arrived.

OnClientSignal(this , new serverListenerArgs(clt));

You may be wondering why we created a serverListenerArgs object while we had a ClientInfo object that we could have passed? The answer to this question is that when throwing an event, the first parameter represents the object which is throwing this event (in this case that would be the serverListener class), and the second parameter must be an object of EventArgs (or any class inherited from the EventArgs ). So to achieve our requirement we created another class called serverListenerArgs and inherited it from EventArgs. We'll explain the serverListenerArgs class later.

public void
            close()
{
bRunning=false;
Server_Socket.Stop();
}

This method is simply used to close the Listener thread.

Class Description: ClientInfo

Attributes

Name | Type | Access Modifier | Description
sConnectedSocket | Socket | Private | This object creates a managed version of an Internet transport service
ClientThread | Thread | Private | it is used for each of the client that arrives
strMachineName | string | Private | The name of the machine of each client
strIPAddress | string | Private | The IP address of each client
iPortNumber | int | Private | Its used for starting and stopping the Listener
strUserName | String | Private | The username of each client
alServiceList | ArrayList | Private |
alProcessList | ArrayList | Private |
bRunning | Bool | Private | Its used for starting and stopping the Listener

Methods

public ClientInfo(Socket
            sock)
{
try
{
sConnectedSocket=sock;

string
            temp = sock.RemoteEndPoint.ToString();
StrIPAddress =
            temp.Substring(0,temp.IndexOf(':'));
iPortNumber =
            Convert.ToInt32(temp.Substring(temp.IndexOf(':')+1));

getRemoteMachineName();
getRemoteUserName();

alServiceList = new ArrayList();
alProcessList = new
            ArrayList();

getProcessList();
getServiceList();

}
catch(Exception
            e)
{
MessageBox.Show(e.Message,"Information");
}
}

This constructor gets all the information from the remote client which this object is handling. For example

getRemoteMachineName();

This method gets the machine name from the client. This constructor also initiates a thread to listen to all the requests that this particular client sends. This means that each and every client which is connected to the server has its own thread which is monitoring its activities. This multithreaded approach to handle clients is used so that we can handle all the clients concurrently.

public void
            run_Thread()
{
try
{
Byte[] bReceive = new
            Byte[1024];

while(bRunning)
{
int i =
            sConnectedSocket.Receive( bReceive);
string
            sBuffer=Encoding.ASCII.GetString(bReceive);
}
}
catch(Exception
            e)
{
MessageBox.Show(e.Message);
}

finally
{
sConnectedSocket.Close();
}
}

This is the entry point for a particular client's thread, and it continuously listens for data coming from the client side.

int i = sConnectedSocket.Receive( bReceive);

Notice that every socket has the ability to send and receive data, to or from the machine which it is representing.

The socket class provides methods to achieve this. The Receive method used in the code above waits until the data arrives and then moves the data into a byte array. There are several overloads present for the Receive method. The integer value that it is returning is the number of bytes that it has received.

string sBuffer=Encoding.ASCII.GetString(bReceive);

We use the above statement to convert the byte stream into a string.

public void
            getRemoteMachineName()
{
string strMachineName = "<command
            type=\"Machine_Name\">";
strMachineName +=
            "</command>$";

byte[] bBuffer = new
            byte[1024];
bBuffer =
            Encoding.ASCII.GetBytes(strMachineName);
sConnectedSocket.Send(bBuffer);

sConnectedSocket.Receive
            (bBuffer);
string s =
            Encoding.ASCII.GetString(bBuffer);

while(sConnectedSocket.Available
            >0 )
{
sConnectedSocket.Receive (bBuffer);
s +=
            Encoding.ASCII.GetString(bBuffer);
}

string
            strResultXML = s.Substring(0,s.IndexOf('$'));

XmlDocument
            doc = new
            XmlDocument();
doc.LoadXml(strResultXML);

XmlNode node
            =doc.FirstChild;
if (node.Attributes.GetNamedItem("type").Value
            == "Machine_Name")
{
XmlNode childNode =
            node.FirstChild;
this.strMachineName =
            childNode.Attributes.GetNamedItem("name").Value;
}
else
throw new Exception("Wrong Packet Recieved");

}

This is the first method where actual client server communication is happening. There are a few things you will need to understand before we go into more detail; in socket communication, stream of bytes are passed because the TCP protocol is unaware of the type of data that is passed. To TCP, any type of data, whether it is an object or any predefined type (such as int, string, etc), is simply a collection of bytes. This is the reason why all the send and receive methods in socket programming uses byte arrays as a buffer to store or retrieve data. Therefore we have used string objects to send and receive, as they can easily be converted into byte arrays. The best medium to produce self-describing data in string format is XML.

In recent years, XML has become the standard way to describe data. Although we could have just passed simple strings and still have achieved all the required results, using XML provides the following benefits:

  • Easy interpretation of data using the System.xml classes.
  • Data is self describing, which means that anyone can easily understand what a particular string is representing. Without the use of XML, only the author of the code would understand the particular data.

Now I that we have explained why XML will be used in our application, let's go back to the getRemoteMachineName method.

string strMachineName = "<command type=\"Machine_Name\">";
strMachineName += "</command>$";

This method is preparing a command, in the form of an XML string, to get a machine name. The character $ acts as an end-of-data identifier. Here the string which we have sent acts as a command for the client, ordering it to reply with its machine name.

bBuffer = Encoding.ASCII.GetBytes(strMachineName);
sConnectedSocket.Send(bBuffer);

These statements convert the XML string into byte form. The send method of the socket object sends the data to the application which this socket is representing.

sConnectedSocket.Receive (bBuffer);
string s = Encoding.ASCII.GetString(bBuffer);

The logic here is that whenever the server sends a command to a client, it expects a response. The response is either the data that was requested, or a notification that the operation requested was performed or not. In this case, the server has sent a command for the machine name - it is now waiting for the client to respond to the request and send back its machine name. How the client responds to this command will be explained in the client code description. The receive method receives the clients response.

while(sConnectedSocket.Available >0 )
{
sConnectedSocket.Receive (bBuffer);
s += Encoding.ASCII.GetString(bBuffer);
}

The limitation with the receive method is that it doesn't receive all the data in one go; instead, it obtains it in chunks. Usually the receive method receives 40 bytes at a time. To overcome this problem, we continue receiving this data until it has all arrived. The Available property of the socket object tells us the amount of data that has yet to be received, in other words, the number of bytes available on the stream.

string strResultXML = s.Substring(0,s.IndexOf('$'));

The line above removes the end-of-string identifier from the string received.

XmlDocument doc = new
            XmlDocument();
doc.LoadXml(strResultXML);

Now we come to the response interpretation part. The string received from the client is also in XML form because of the reasons described earlier. The lines above make an XMLDocument object and load the client's response into it. As we know the format of the response, we can easily parse it:

XmlNode node =doc.FirstChild;
if (node.Attributes.GetNamedItem("type").Value == "Machine_Name")
{
XmlNode childNode = node.FirstChild;
this.strMachineName = childNode.Attributes.GetNamedItem("name").Value;
}
else
throw new Exception("Wrong Packet Recieved");

In the above code, we traverse to the first node and check the type. If it is valid, then it moves to its child node and gets its value, which will be the name of the machine. Otherwise, an exception is thrown.

The description of the getRemoteMachineName method explains in detail how we have achieved client server communication. There are several other ways of doing what we have done, but this approach is much more error-safe and close to standards.

public void
            getRemoteUserName()
{
string strUserName = "<command
            type=\"User_Name\">";
strUserName +=
            "</command>$";

byte[] bBuffer = new
            byte[1024];
bBuffer =
            Encoding.ASCII.GetBytes(strUserName);
sConnectedSocket.Send(bBuffer);

sConnectedSocket.Receive (bBuffer);
string s =
            Encoding.ASCII.GetString(bBuffer);

while(sConnectedSocket.Available
            >0 )
{
sConnectedSocket.Receive (bBuffer);
s +=
            Encoding.ASCII.GetString(bBuffer);
}

string strResultXML =
            s.Substring(0,s.IndexOf('$'));

XmlDocument doc = new
            XmlDocument();
doc.LoadXml(strResultXML);

XmlNode node
            =doc.FirstChild;
if (node.Attributes.GetNamedItem("type").Value
            == "User_Name")
{
XmlNode childNode =
            node.FirstChild;
this.strUserName =
            childNode.Attributes.GetNamedItem("name").Value;
}
else
throw new Exception("Wrong Packet Recieved");
}

The working of this getRemoteUserName method is very similar to that of getRemoteMachineName, so we will not go into its details. This method gets the name of the user who is logged into the client's machine.

public void getProcessList()
{
string
            strProcesslist = "<command
            type=\"Process_List\">";
strProcessList +=
            "</command>$";

byte[] bBuffer = new
            byte[3000];
bBuffer =
            Encoding.ASCII.GetBytes(strProcessList);
sConnectedSocket.Send(bBuffer);

sConnectedSocket.Receive (bBuffer);
string s =
            Encoding.ASCII.GetString(bBuffer);

while(sConnectedSocket.Available
            >0 )
{
sConnectedSocket.Receive (bBuffer);
s +=
            Encoding.ASCII.GetString(bBuffer);
}

string strResultXML =
            s.Substring(0,s.IndexOf('$'));

XmlDocument doc = new
            XmlDocument();
doc.LoadXml(strResultXML);

XmlNode node
            =doc.FirstChild;
if (node.Attributes.GetNamedItem("type").Value
            == "Process_List")
{
XmlNodeList nodeList =
            node.ChildNodes;
IEnumerator en =
            nodeList.GetEnumerator();

while
            (en.MoveNext())
{

XmlNode childNode = (XmlNode)
            en.Current;
string pname=
            childNode.Attributes.GetNamedItem("name").Value;
string ptime =
            childNode.Attributes.GetNamedItem("time").Value;
ProcessObject
            processObject = new ProcessObject(pname,ptime);
strProcessList.Add(processObject);
}
}
else
throw
            new Exception("Wrong Packet Recieved");
}

This getProcessList method obtains the list of processes of the particular client. The command and receive part of this method is the same as in the previous methods, but it differs in the way the response is interpreted. In the other two methods discussed above, the result was a single string such as machine name or username, but in this case the response is a list of processes with several properties. We have defined a class named ProcessObject to represent a process which will be discussed later.

XmlNodeList nodeList = node.ChildNodes;

The above statement gets a list of all the child nodes of the resulting parent node. Each child node describes a single process.

      IEnumerator
            en = nodeList.GetEnumerator();
while
            (en.MoveNext())
{
XmlNode childNode = (XmlNode) en.Current;
string pname=
            childNode.Attributes.GetNamedItem("name").Value;

string ptime
            = childNode.Attributes.GetNamedItem("time").Value;
ProcessObject
            processObject = new ProcessObject(pname,ptime);
strProcessList.Add(processObject);
}

We traverse through each child node and obtain two values. The first one is the name of process and the second one is the time at which the process was started. Once we get both these values we create an object of ProcessObject :

new ProcessObject(pname,ptime);

We then add this newly created abject to the container strProcessList.

public void getServiceList()
{
string
            strServiceList = "<command
            type=\"Service_List\">";
strServiceList +=
            "</command>$";

byte[] bBuffer = new
            byte[3000];
bBuffer =
            Encoding.ASCII.GetBytes(strServiceList);
sConnectedSocket.Send(bBuffer);

sConnectedSocket.Receive (bBuffer);
string s =
            Encoding.ASCII.GetString(bBuffer);

while(sConnectedSocket.Available
            >0 )
{
sConnectedSocket.Receive (bBuffer);
s +=
            Encoding.ASCII.GetString(bBuffer);
}

string strResultXML =
            s.Substring(0,s.IndexOf('$'));

XmlDocument doc = new
            XmlDocument();
doc.LoadXml(strResultXML);

XmlNode node
            =doc.FirstChild;
if (node.Attributes.GetNamedItem("type").Value
            == "Service_List")
{
XmlNodeList nodeList =
            node.ChildNodes;
IEnumerator en =
            nodeList.GetEnumerator();

while
            (en.MoveNext())
{
XmlNode childNode = (XmlNode) en.Current;
string sname=
            childNode.Attributes.GetNamedItem("name").Value;
string stype =
            childNode.Attributes.GetNamedItem("type").Value;
string sstatus =
            childNode.Attributes.GetNamedItem("status").Value;
ServiceObject
            serviceObject = new ServiceObject(sname,stype,sstatus);
strServiceList.Add(serviceObject);
}
}
else
throw
            new Exception("Wrong Packet Recieved");
}

This method is similar to getProcessList; it receives the list of all the windows services running on the client machine. Each service is represented by a ServiceObject class which will be discussed later.

public bool stopService(string
            servicename)
{
string strServiceString = "<command
            type=\"Stop_Service\">";
strServiceString += "<param
            name=\""+servicename+"\"></param>";
strServiceString +=
            "</command>$";

byte[] bBuffer = new
            byte[1024];
bBuffer =
            Encoding.ASCII.GetBytes(strServiceString);
sConnectedSocket.Send(bBuffer);

sConnectedSocket.Receive
            (bBuffer);
string s =
            Encoding.ASCII.GetString(bBuffer);

while(sConnectedSocket.Available
            >0 )
{
sConnectedSocket.Receive (bBuffer);
s +=
            Encoding.ASCII.GetString(bBuffer);
}

string strResultXML =
            s.Substring(0,s.IndexOf('$'));
XmlDocument doc = new
            XmlDocument();
doc.LoadXml(strResultXML);

XmlNode node
            =doc.FirstChild;
if (node.Attributes.GetNamedItem("type").Value
            == "Stop_Service")
{
XmlNode childNode =
            node.FirstChild;
string a =
            childNode.Attributes.GetNamedItem("result").Value;
if (a ==
            "Success")
return true;
else
return
            false;
}
else
{
throw new Exception("Wrong Packet
            Recieved");
return false;
}
}

This method is different from the other methods in the way that it is performed due to the request of a user. Whenever the user needs to stop any service that is running on its machine, it sends the request along with the service name to this method. The sending and receiving part of this method are similar to the previous ones.

XmlNode node =doc.FirstChild;
if (node.Attributes.GetNamedItem("type").Value == "Stop_Service")
{
XmlNode childNode = node.FirstChild;
string a = childNode.Attributes.GetNamedItem("result").Value;
if (a == "Success")
return true;
else
return false;
}
else
{
throw new Exception("Wrong Packet Recieved");
return false;
}

Since this method is not requesting any information, but is asking the client to perform an operation, the response will not be any information but just an acknowledgement which will tell the server whether the requested operation was successfully performed or not.

Class Description: serverListenerArgs

This class is inherited from the EventArgs class. The reason behind using this class is that in event driven programming, when an event is throw,n the parameters passed to the event handler must be through an object of EventArgs class or any of its derived classes. Hence the object of this class is instantiated whenever a listener thread receives a new client and it throws an OnClientSignal event. This class has only one attribute, a ClientInfo object. In short, serverListenerArgs provides a transportation medium for ClientInfo objects to be sent from listener to server class.

Attributes

Name | Type | Access Modifier | Description
ClientInfo | ClientInfo | Private | The client information object which will be passed to the server class.

Class Description: ProcessObject

This class just acts as a user defined type for representing processes. An object of the process class itself has many attributes and methods, but of two of them are of use to us. As we have discussed above, data transferring in Socket programming is restricted in that only byte streams can be sent, so we cannot send objects. What we have done is fetched the information from the Process class object (such as process name and process start time) and sent it through the stream. Since this information wasn't complete enough to make a new Process instance, we have made our own class which is in a sense representative of the process class.

Attributes

Name | Type | Access Modifier | Description
StrStrProcessName | String | Private | Specifies the name of that particular process
StrProcessStartTime | String | Private | Specifies the time the process started

Class Description: ServiceObject

Like the ProcessObject class, ServiceObject is a user defined type to represent Services.

Attributes

Name | Type | Access Modifier | Description
StrDisplayName | String | Private | Specifies the name of that particular service
strServiceType | String | Private | Specifies the type of the service.
strStatus | String | Private | Status of service (Stopped, Started, Paused, etc.)

Client Description

Class Description: netClient

Attributes

Name | Type | Access Modifier | Description
ClientSocket | TcpClient | Private | The main object that connects the client to the server
iServerPort | Int | Private | Specifies the port of the server where the client is to be connected
sServerIP | String | Private | Specifies the IP Address of the server
sMachineName | String | Private | The client's machine name
sUserName | String | Private | The name of the user who has logged onto the client's machine
NetworkStream | NetworkStream | Private | Provides the underlying stream of data for the network access.
Services | ServiceController [] | Private | List of ServiceController instances, each representing a single service available on the clients machine
Processes | Prosess[] | Private | List of processes running on the client's machine
bRunning | Bool | Private | It is used for starting and stopping the client

Methods

public netclient(string ipaddr, int prt)
{
iServerPort=prt;
sServerIP=ipaddr;

Services = new ServiceController[50];
refreshServiceList();

Processes = new Process[20];
refreshProcessList();

sMachineName = SystemInformation.ComputerName;
sUserName = SystemInformation.SUserName;

ClientSocket=new TcpClient();
ClientSocket.Connect(sServerIP,iServerPort);
Console.WriteLine("Client Connected to Server");

ClientSocket.ReceiveBufferSize = 3000;
ClientSocket.SendBufferSize = 3000;
NetworkStream = ClientSocket.GetStream();

if(NetworkStream.CanWrite && NetworkStream.CanRead)
{
try
{
while (bRunning)
{
Byte[] recBytes = new Byte[1024];
int a= NetworkStream.Read(recBytes,0,1024);
string sBuffer = Encoding.ASCII.GetString(recBytes);
sBuffer = sBuffer.Substring(0,sBuffer.IndexOf('$'));

XmlDocument doc = new XmlDocument();
doc.LoadXml(sBuffer);

XmlNode node = doc.FirstChild;
XmlNode childNode;

string sCommandType = node.Attributes.GetNamedItem("type").Value;

switch (sCommandType)
{
case "User_Name":
sendUserName();
break;
case "Machine_Name":

sendMachineName();
break;
case "Service_List":
SendServiceList();
break;
case "Process_List":

SendProcessList();
break;
case "Restart_Client":
restartClient();
break;
case "Shutdown_Client":
shutdownClient();
break;
case "Pause_Service":
childNode = node.FirstChild;

pauseService(childNode.Attributes.GetNamedItem("name").Value);
break;
case "Stop_Service":
childNode = node.FirstChild;

StopService(childNode.Attributes.GetNamedItem("name").Value);
break;
case "Resume_Service":
childNode = node.FirstChild;
resumeService(childNode.Attributes.GetNamedItem("name").Value);
break;
}
}
}
catch(Exception e)
{
Console.WriteLine(e.Message);
}
}
Console.WriteLine("Press any key to continue..");
Console.Read();
}

The netclient method comprises the main functionality of the client. It gathers all the client information, receives all the servers commands, perform actions in accordance to those commands, etc. Now we'll discuss this method in a modular fashion.

iServerPort=prt;
sServerIP=ipaddr;

Services = new ServiceController[50];
refreshServiceList();

Processes = new Process[20];
refreshProcessList();

sMachineName = SystemInformation.ComputerName;
sUserName = SystemInformation.SUserName;

The first thing this method does is to gather all the information about itself and the servers where it has to be connected. The functionality of refreshServiceList and refreshProcessList will be discussed later. SystemInformation is a class from the System.Windows.Forms namespace; it gives all the low level information about the computer. A detailed description of this class is unnecessary with respect to our case study. We have used this class to get the machine and user name.

       ClientSocket=new
            TcpClient();
ClientSocket.Connect(sServerIP,iServerPort);

The above lines are the main part of any client. The first line creates an instance of TcpClient's class. The connect method of this class has several overloads, but the one we have used takes two parameters. The first one is the IP Address of the server, and the second one is the port at which the server is listening for clients. These two parameters must be known before any connection request is sent. The connect method prepares the socket object of the client with information such as IP Address, port number at which the client application is running, the protocol type which is being used for the transmission, and some other information. It makes a packet of them, and sends it through the network to the server whose IP was specified in the parameters, and the application running on that server whose port number was specified.

ClientSocket.ReceiveBufferSize = 3000;
ClientSocket.SendBufferSize = 3000;

Once the client is connected to the server, a few things are done before actually sending or receiving data. The code above sets the size of the send and receive buffer of the client. Every TcpClient instance has two buffers associated with it, one where all the data arrives and the other from where all the data is sent. Different computers have different send and receive rates, and because of this, buffers are used. Buffer sizes are application dependent; large sized buffers are used for the applications in which the amount of data is large.

NetworkStream = ClientSocket.GetStream();

The GetStream method gets an instance of the NetworkStream class. Each TcpClient has its own stream, through which it can send and receive data. By providing separate streams for each TcpClient instance, developers have enforced a level of security, so that if two or more clients are running on same machine each of them get their own data, although all the data is coming from the same medium.

Byte[] recBytes = new Byte[1024];
int a= NetworkStream.Read(recBytes,0,1024);
string sBuffer = Encoding.ASCII.GetString(recBytes);
sBuffer = sBuffer.Substring(0,sBuffer.IndexOf('$'));

As the NetworkStream class is inherited from the Stream class, like all other NetworkStream instances it has the ability to read and write. The above code is where the client listens for commands from the server. The Read method used here waits until data arrives from serve. It has several overloads, but the one we have used has three parameters: the first one is the buffer in which data will be received, the second one is the offset telling it from which position in the byte array it has to start writing data, and the third is the number of bytes to be read. The last two statements of the code perform the same function as in the server i.e. of converting byte array intro string and removing the end-of-string character.

XmlDocument doc = new XmlDocument();
doc.LoadXml(sBuffer);

XmlNode node = doc.FirstChild;
XmlNode childNode;

string sCommandType = node.Attributes.GetNamedItem("type").Value

Once the data has arrived it has to be interpreted - again the same method is applied by parsing the XML string, and getting the type of the command. The last statement gets the type of the command sent.

switch (sCommandType)
{
case "User_Name":
sendUserName();
break;

case "Machine_Name":
sendMachineName();
break;

//Rest of the code
}

Once we have received the type of command, we take the appropriate action. On each type of command we are calling a method to perform the requested action and send a response back to the server.

public void sendMachineName()
{
string
            sMachineNameXML;
sMachineNameXML = "<result
            type=\"Machine_Name\">";
sMachineNameXML += "<params
            name=\"" + sMachineName + "\" />";
sMachineNameXML +=
            "</result>$";

byte[] bBuffer = new
            byte[1024];
bBuffer = Encoding.ASCII.GetBytes(sMachineNameXML);
NetworkStream.Write(bBuffer,0,machineNameXML.Length);
}

The above sendMachineName method is called whenever a request for the Client's machine name arrives. It prepares an XML string with the requested machine name in parameters, and sends this response back to the server. We are just going to take a brief look at the write method. Like the read method, write is also inherited from the stream class, and it also has several overloads available. The one we have used has three parameters: the first one is the reference to the buffer from which data has to be sent, the second one is the offset in the buffer, and the third is the number of bytes to be sent.

public void sendUserName()
{
string
            sUserNameXML ;
sUserNameXML = "<result
            type=\"User_Name\">";
sUserNameXML += "<params name=\"" +
            sUserName + "\" />";
sUserNameXML +=
            "</result>$";
byte[] bBuffer = new byte[1024];
bBuffer =
            Encoding.ASCII.GetBytes(sUserNameXML);
NetworkStream.Write(bBuffer,0,sUserNameXML.Length);
}

This sendUserName method is used to retrieve the user name from the client. Whenever a request for the Client's user name arrives, an XML string is generated that include the requested user name in parameters, and sends this response back to the server.

public void sendProcessList()
{
string
            sProcessXML = "";
sProcessXML += "<result
            type=\"Process_List\">";
IEnumerator en =
            Processes.GetEnumerator();
while (en.MoveNext())
{
Process
            process = (Process) en.Current;
sProcessXML += "<params
            name=\""+process.StrProcessName+"\"
            time=\""+process.StartTime.ToString()+"\"
            />";
}
sProcessXML += "</result>$";
byte[]
            bBuffer = new byte[3000];
bBuffer =
            Encoding.ASCII.GetBytes(sProcessXML);
NetworkStream.Write(bBuffer,0,sProcessXML.Length);

}

The sendProcessList method sends the list of the processes running on the client to server. This method differs to the other methods in the way the response is generated. Since in this method we have to generate a list of processes with their attributes, the XML generating code is a little different. Once a complete response is generated, it is sent back to the server.

public void SendServiceList()
{
string
            sServiceXML = "";
sServiceXML += "<result
            type=\"Service_List\">";
IEnumerator en =
            Services.GetEnumerator();
while
            (en.MoveNext())
{
ServiceController service =
            (ServiceController) en.Current;
sServiceXML += "<params
            name=\""+service.DisplayName+"\"
type=\""+service.ServiceType.ToString()+"\"
status=\""+service.Status.ToString()+"\"
            />";
}
sServiceXML += "</result>$";
byte[] bBuffer
            = new byte[3000];
bBuffer = Encoding.ASCII.GetBytes(sServiceXML);
NetworkStream.Write(bBuffer,0,sServiceXML.Length);
}

The sendServiceList method is more or less like the sendProcessList, the only difference is that instead of the process list, the requested service list of that particular client is sent to the server.

 public bool stopService(string
            serviceName)
{
try
{
IEnumerator en =
            Services.GetEnumerator();
while
            (en.MoveNext())
{
ServiceController service =
            (ServiceController) en.Current;
if (service.DisplayName ==
            serviceName)
service.Stop();
}
refreshServiceList();
string strResultXML;
strResultXML = "<result
            type=\"Stop_Service\">";
strResultXML += "<params
            result=\"Success\"/>";
strResultXML +=
            "</result>$";
byte[] bBuffer = new byte[3000];
bBuffer =
            Encoding.ASCII.GetBytes(strResultXML);
NetworkStream.Write(bBuffer,0,strResultXML.Length);
return
            true;
}
catch(Exception e)
{
string strResultXML =
            "<result type=\"Stop_Service\">";
strResultXML +=
            "<params result=\"Failure\"/>";
strResultXML +=
            "</result>$";
byte[] bBuffer = new byte[3000];
bBuffer =
            Encoding.ASCII.GetBytes(strResultXML);
NetworkStream.Write(bBuffer,0,strResultXML.Length);
return
            false;
}
}

The above method is used to stop a particular service of the client. The name of the service is received in the parameters of the stopService method. The ServiceController class is used here to stop the service, this class is used to connect to and control the existing windows services. Every instance of the service controller class has a Stop method, which stops the service. If the service cannot be stopped, it throws an InvalidOperationException, so in our case, if the service is stopped successfully, we prepare a result message with success as its parameter, else a failure response is sent back to the server.

Designing The User Interface

Our main goal in this application was to demonstrate network programming in C# using sockets, therefore the Graphical User Interface of this application was not the main focus. The server side comprises of a UI, as it is going to perform all the operations, while the client side does not have any UI.

Now we will be discussing each screen, as the operations are performed.

Main Screen

Above is the main screen when the server is executed. This screen contains information on each client that has been logged on. This is the only screen, so all the operations will be performed here. You can find this form in the code download that accompanies this case study. Let's discuss each section.

Name | Type | Description
lbClientsConnected | List Box | This list box contains all the clients that have been connected to the server.
tbComputerName | Text Box | When we select a client from the client connected list, the name of that particular client is displayed here
tbIPAddress | Text Box | When we select a client from the client connected list, the IP Address of the client is displayed here
tbUserName | Text Box | The name of the user of that client is displayed here
tbServiceAccessPoint | Text Box | The port of the server at which the client is connected
tbCommunicationProtocol | Text Box | The protocol that client is using e.g TCP, UDP
lbListofProcesses | List Box | It contain the name of the list of processes that are running on the clients computer
lbListofServices | List Box | It contain the name of the list of services that are available on the clients computer
tbStrProcessName | Text Box | When the process from the List of Process is selected, the name of that process goes here
tbProcessStartTime | Text Box | When the process from the List of Process is selected, the time the process started goes here

Whenever the client is connected to the server, its name is displayed in the Clients Connected list. The server administrator can then observe all the information regarding that client on the screen. The Process List shows all the processes that are running on the client's side, and similarly, the Service List shows all the services available on the client side.

The server can stop the client by pressing the Stop Client button; this button will disconnect the client from the server. The server can also perform operations on the services available from the client, that is, it can pause, resume, or stop the service. This can be done by selecting a service from the Service List and then pressing the respective button. For example, the service can be stopped by selecting it from the Server List and then pressing the Stop Service button. Selecting the process from the process List and then pressing the Stop Process button can only kill the process.

Now well show how a connection is established, and how a service is killed.

The above screen shows the client connected to the server.

The above screen demonstrates what happens when the client is selected. As you can see, all the information described earlier is displayed in the respective boxes. In this screen, the Process list is selected, which means that all the processes running on that particular client are shown.

The above screen is similar to the previous one, the only difference is that the Service List is selected here, which means that all the services available on the client are shown.

In this screen shot, one of the operations is demonstrated. By selecting any service from this list and then pressing any button at the top of the screen, the respective operation is performed. As can be seen, the ASP. NET State Service 's status is set to be stopped (highlighted).

Conclusion

While using .NET, I realized a need to demonstrate the networking concepts in C#. One can certainly read lots about these concepts, but what I wanted was a real application that shows the basic concepts of networking, like client server connection for example, in C#. For this reason, I made up a simple client server application using Sockets and TCP protocol. This application uses the System.Net namespace that includes base classes for TCP server and clients provided by .NET.

The case study illustrates some basic knowledge about .NET technologies, as well as some other technologies such as threads, XML, etc. As the key objective of the case study was to demonstrate Socket programming, a whole section is devoted to the Sockets and TCP concepts. It shows the networking concepts and describes in detail the OSI model, mapping of TCP over OSI model, Sockets, Socket programming, the System.Net namespace, etc. After going through all the technologies, the case study describes how Sockets can be used in C# by creating a small application.

The application is a simple client/server application in which multiple clients connect to a single server through sockets. This application uses TCP for the communication; it is a connection oriented protocol which is highly reliable as it sends acknowledgements for each request and response. The application allows the server to view all the client's information such as its name, its IP Address, the protocol it is using, the port it is connected to, the user's name, etc. The server can also view the processes and services running on the client side. It has the ability to make changes on the client's machine, such as to make the client stop, make the service stop, pause, or resume, or make the process stop.

C#Today - © 2002 WroxPress Limited


Mapping XML to C# objects using reflection

Mapping XML to C# objects using reflection
http://www.csharptoday.com/content.asp?id=1921
Article Rating: 4
Published on October 7, 2002
Abstract
Nowadays XML is the de facto standard for exchanging data between different systems and platforms, so providing a mechanism that allows us to import XML is very useful. In this case study, Giuseppe Naccarato will be using the XmlReader object and reflection to build a powerful general-purpose component for mapping XML to C# objects.
Case Study

Nowadays XML is the de facto standard for exchanging data between different systems and platforms, so providing a mechanism that allows us to import XML is very useful. In this case study, we will be using the XmlReader object and reflection to build a powerful general-purpose component for mapping XML to C# objects.

The XmlReader object is the standard way in which .NET applications read XML documents. Reflection enables a program to inspect and manipulate data. It can be used to find or create types at runtime, invoke methods dynamically, or to discover information about classes and their members. To end the piece, we will be looking at how the component can be used in a real-world situation. This will be an application showing a list of items and their stocks. It can import XML documents, which contain information about customer or purchase orders, in order to add new articles and/or update the stock level information. Developing this example, you can figure out how the proposed component works and why it is more powerful and flexible than .NET XML serialization.

Introduction

The idea is to develop a C# class which imports XML documents following a specified XML-schema. This class, by using the information coming from the XML, will be able to create class-instances using reflection. In addition, the data-members of the class will be filled with data contained in the XML. For example:

<class name="Person">
<field name="firstname" value="Robert" type="string"/>
<field name="surname" value="Hill" type="string"/>
</class>

The C# class, which imports this XML, will create an instance of the Person class where:

person.firstname = "Robert"
person.surname = "Hill"

Obviously that was a very trivial example. In more complex circumstances, the XML document could contain nested classes and array of objects. The developed C# component has to be able to cope properly with these situations as well.

At first glance, importing XML by using reflection might not seem to be the best strategy because the .NET platform already provides serialization for this purpose. Serialization is a very cool C#/.NET feature, and unlike Java, C# provides XML serialization, which serializes and deserializes objects to and from XML. In spite of this, there are some situations where serialization is less than adequate. An example of this is when we have to import information from an external system that is producing data in an XML structure that is different to the one used by .NET serialization.

To solve this problem we have two choices: transformthe source XML into an .NET serialization structure by using XSLT, or to create a component that reads the XML and builds C# objects from it. The first method can be an effective and powerful strategy, but you would have to choose the second way if you are also interested in the data-types. In fact, by using the .NET serialization structure, you can't specify the data-types. This is a significant limitation, as sometimes the data-type is an important issue that we cannot leave out.

This case study will cover many situations in which you have necessarily to import XML whose structure is already designed and can't be changed (perhaps because other systems have been using it for years and cannot be altered). In those situations, it is clear that serialization can't help us at all. In addition, unlike serialization, the component we will be developing can use the standard C# classes without adding any lines of code to make them "serializable".

System Requirements

This system requires Visual Studio .NET 1.0 or later running on Windows 2000 or Windows XP. The case study assumes that the reader has a good understanding of C#, XML and XML schema. An understanding of XmlReader objects and reflection would be beneficial, but not essential. You can find links to other case studies and articles on these topics in the related links section of this case study.

Defining The Problem

The focus of this case study is explaining a way to carry out the data exchange between two systems where the importer is implemented in C#, and the exporter provides data in XML following a specified XML schema. Importing data from external systems is a situation that occurs frequently in the real-world, and there are many different strategies for addressing these problems. Before XML, the data exchange was made by a common data-structure that both systems could understand. Data was written in a file (i.e. CSV files) or streamed in a socket following a specific proprietary protocol. The system importing the data was able to decode that protocol in a way that meant it could update the entities present in its database. To apply this strategy it was necessary to define the common data-structure, develop encoding and decoding algorithms according to the data-structure, and implement the code to import the data to the target system.

Today, thanks to XML, the strategies used to address this problem have become simpler. Now, it is no longer necessary to define the protocol for the data-structure, as that protocol is XML, nor do we need to define the encoding/decoding algorithms. In fact, most modern languages provide modules to read and write XML, for example SAX and DOM, both of which have been developed just for this reason. With .NET you can use the System.Xml namespace, which provides classes like XmlReader and XmlWriter, for reading and writing XML code.

The system reading the XML has to import and create the data according to its own design. Very often, in object-oriented systems, importing data means creating objects. Hence, the developer has to implement code for translating the XML elements in objects. This can be a tough job, because typically each class will have a different 'structure' (i.e. methods and attributes). For example, the Person class is sure to be radically different from the Invoice class. The first one would have fields such as name, lastname, and dateOfBirth. The second one on the other hand would have fields like number, date, and amount. That means that for each class you would have to implement different code to create the objects. We want to avoid that situation.

So, what do we want? We would like to create a C# general-purpose component that can read XML and use it to create C# objects, which belong to any defined class. The component must meet the following criteria:

  • it can create new instances of any predefined class
  • it can import any XML written according to a predefined XML schema
  • no changes must be made to the target class source
  • data-types must be supported.

This is what we will be developing in this case study. Going back to the serialization issue, you should be able to see that serialization does not meet all of the criteria above. Using serialization you can "create new instances of any predefined class", so the first point is met, but you can't "import any XML written according to a predefined XML schema". Serialization uses its own schema that could be incompatible with the one used by the exporting system. It is true that you can allow serialization to read different XML structures by using the C# attributes, but this would mean modifying the source code of the target class, thus meaning the third point would not be met. This is a very important point, as you might not have the option of accessing the source code of the classes (for example, the classes may be defined inside a dll developed by a third party). In addition, serialization does not support data-types at all.

Case Study Structure

The case study is structured as follows:

Component Design and Code

  • XML Parser and XmlReader objects.
  • Reflection and metadata
  • Mapping simple objects
  • Mapping complex objects
  • Mapping array of objects
  • Internal Reference

The sample application

  • UI Design and coding
  • Database Design
  • Importing XML
  • Application Setup

Conclusion

  • Review
  • Limitations and further work
  • Related links

Component Design and Coding

Now that the problem has been defined, let's develop our solution. To do this, we have to know about some important facilities that .NET provides. First, we have to understand the way in which .NET and C# deal with XML, and secondly we need to look at how we can dynamically create C# objects by using reflection.

The idea is to develop a C# class that reads the XML following a predefined XML schema. While it is reading the XML elements, it must be able to create the right C# class and fill its fields with the data coming from the XML. The name of the class, the fields, and the values are written in the XML, thus the component, which uses reflection, can dynamically create the instances and return them.

XML Parser and XmlReader objects

XML plays a fundamental role in the .NET framework. The framework not only provides modules allowing you to use XML in your application, but it uses XML itself for a number of reasons, such as configuration files and source code documentation. In addition, .NET-based technologies like ADO.NET and Web Services work and deal with XML. To allow programmers to use XML, the .NET framework provides the System.Xml namespace, which consists of a set of classes whose main purpose is to read, manipulate, and write XML code. The classes I'm going to use in this case study are System.Xml.XmlReader, and its derived classes such as XmlTextReader and XmlValidatingReader.

The XmlReader class is an abstract class that provides fast, non-cached XML data. XmlTextReader is a concrete implementation of XmlReader that supplies fast text-stream access to XML data. Using XmlTextReader is the standard .NET way of reading the contents of an XML document. The XmlTextReader follows neither the SAX nor the DOM philosophy. However, it is closer to SAX than DOM because like SAX, it parsers the XML and enables applications to process elements in the same moment that they have been read. Unlike DOM, the XmlTextReader object does not create an in-memory object model.

The way XmlTextReader reads the XML document is very similar to the way the contents of the tables in a database are read. The XmlTextReader seeks the XML elements and supplies them to the application together with their attributes. Usually, the content of an XML document can be seen as a tree. The XmlTextReader class implements a visiting algorithm for the XML tree based on the so-called node-first approach. This means that for each XML sub-tree found, the root is visited first followed recursively by all of its children, from first to last. Readings are made in a forward-only and read-only way; therefore, writing is not allowed and the visit cannot return back to nodes already visited.

Using the XmlTextReader class you can easily develop an XML Parser that checks if a document is well-formed, that is whether it respects the XML syntax or not. If you need to check the semantic meaning of an XML document as well, you have to use the XmlValidatingReader class. This is an XmlReader extension that allows us to specify the kind of validation the parser needs. Some of possible choices are: none, DTD, or XML Schema (XSD).

To implement an XML validating parser in C#, first we have to create the XmlTextReader object and the XmlValidatingReader object:

// XmlReader inizializzation
// the
            string filename is the XML file name
XmlTextReader txtReader =
            new XmlTextReader(filename);
XmlValidatingReader xtReader = new
            XmlValidatingReader(txtReader);

As you can see, you need both XmlTextReader and XmlValidatingReader instances, because XmlTextReader does not directly derive from the XmlValidatingReader class. To instantiate an XmlValidatingReader object you have to pass the XmlTextReader as a parameter to the constructor.

Now, you can set the validation type using the ValidationType property:

// Set the ValidationType to XML-Schema (XSD)
xtReader.ValidationType = ValidationType.Schema;

ValidationType is an enumeration. The possible values are: Auto, DTD, None, Schema, and XDR. In this case study we will always use the XML schema validation.

The next step is to define a validation event handler, which is basically a method that will be called by the framework when an error in the XML document occurs. To set a validation event handle we use:

// Set the validation handler
xtReader.ValidationEventHandler +=
new ValidationEventHandler (this.ValidationEventHandle);

The invoked function is this.ValidationEventHandle ; we will see its implementation in a little while.

At this point, we can call the Read method of XmlValidatingReader, which starts to read the document. The method returns each time an XML entity is found. By "entity", I am referring to elements, attributes, or tags. Thus, the xtReader object will contain all the information about the XML entity. The method returns false when there are no more entities, which will happen when the reading operation has ended. In this first example, I invoke the Read method until it returns false. The xtReader object information will not be used:

while (xtReader.Read())
            {
}
xtReader.Close();

The validation event handler implementation is as follows:

public void ValidationEventHandle (object sender, ValidationEventArgs args){
throw new System.Xml.XmlException(args.Message,null);
}

When an error occurs, the validation event handler will throw an exception whose message is derived from the args parameter.

In this case study I'm going to use the following properties and methods of the XmlReader class:

Type | Name | Description
Property | Name | Gets the qualified name of the current node.
Property | Value | Gets the text value of the current node.
Property | NodeType | Gets the type of the current node.
Method | MoveToAttribute | Moves the XmlReader object to the specified attribute.
Property | AttributeCount | Gets the number of attributes for the current element

The XMLImporter

If you assemble the code fragments above in a C# class you can get a simple XML validating parser. We want more though. We have to implement a class that will be used as the basic class for this entire case study. It has to be able to parse an XML document and delegate to a user-defined handler to process the XML entities it finds. What I'm going to develop is an XML parser that follows the design shown in the figure below:

This class diagram shows the XMLImporter class, which is our target, and its relationship with the other classes and interfaces. Basically, the XMLImporter is an XML-schema-based validating parser that reads (via the import method) the XML and calls a user-defined method each time an element requires processing. This method, called processElement, belongs to the interface Handler. As you may see from the diagram, the XmlImporter holds a Header object. The Header interface code is as follows:

public interface Handler {

// processElement is called by the XmlImporter
// each time an XML element is being processed

void processElement(XmlValidatingReader xtReader);

// processObject shall be called by the user
// to process an XML imported root object

void processObject(object obj);
}

The processElement method, as already mentioned, is called from the XMLImporter each time an XML element is found. The xtReader parameter represents the XmlValidatingReader, and stores all the information about the XML element. Consequently, you can implement this method in a way that means you can use the XML element for whatever purpose you wish. The processObject method will be explained in detail later. To briefly explain, it will be invoked after a C# object is created starting from the XML element. By implementing this method you can use the object for the purposes that we set out for the final application earlier in this case study. To use the XmlImporter, we must implement the Handler interface. However, a dummy implementation of Handler is provided; it is the DefaultHandler :

public class DefaultHandler : Handler
            {
public void processElement(XmlValidatingReader
            xtReader){
}
public void processObject(object
            obj){
}
}

The XmlImporter instances using the DefaultHandler are merely XML validating parsers.

In the class diagram you can see that XmlImporter is aggregated to a ValidationEventHandler. Even if this is represented as a Class in the diagram, the implementation is a method, exactly the same as the ValidationEventHandler method we saw in the previous section.

Now, we have the recipe to implement the XmlImporter class. Below is the code:

using System;
using System.Xml;
using
            System.Xml.Schema;

namespace Wrox.Xml.Mapping {

public
            class XmlImporter {

// The user-defined Handler
private
            Handler handler;

// Constructor (using a default
            handler)
public XmlImporter() {
this.handler = new
            DefaultHandler();
}

// Constructor (using a passed
            handler)
public XmlImporter(Handler handler) {
this.handler =
            handler;
}

// Read the XML files
public void
            import(string strFilename ) {

// XmlReader
            inizializzation
XmlTextReader txtReader = new
            XmlTextReader(strFilename);
XmlValidatingReader xtReader = new
            XmlValidatingReader(txtReader);

// Set the ValidationType to
            XML-Schema (XSD)
xtReader.ValidationType =
            ValidationType.Schema;

// Set the validation
            handler
xtReader.ValidationEventHandler +=
new
            ValidationEventHandler (this.ValidationEventHandle);

try
            {
// Parsing
while (xtReader.Read()) {
// Call the
            processElement of the user-defined
            handler
handler.processElement(xtReader);
}
}
catch
            (Exception e){
throw new Exception(e.Message);
}
finally
            {
xtReader.Close();
}
}

// Validation Event
            Handle
public void ValidationEventHandle (object sender,
ValidationEventArgs args) {
throw new
            System.Xml.XmlException(args.Message,null);
}
}
}

The class has two constructors: the first one creates an XmlImporter instance using DefaultHandler, the second one creates an XmlImporter instance using the Handler object passed as parameter. The import method takes the XML file name as input, reads it using the XMLValidatingReader, and for each XML element calls the processElement method of the Header object. Below is a straightforward implementation of Handler, which writes the name of the XML elements and their attributes to the console:

public class MereHandler : Handler
            {
public void processElement(XmlValidatingReader xtReader){
if
            (xtReader.NodeType==XmlNodeType.Element){
Console.WriteLine("Element:
            " + xtReader.Name);
for (int
            i=0;i<xtReader.AttributeCount;i++)
            {
xtReader.MoveToAttribute(i);
Console.WriteLine(" attribute
            {0} = {1}",
xtReader.Name,xtReader.Value);
}
}
public
            void processObject(object obj){
}
}

Notice that it first checks whether the xtReader object contains information about elements, which is indicated when NodeType = XmlNodeType.Element. XmlNodeType is an enumeration, which indicates the type of the XML entity: Element, Attribute, Comment, Document, and so on. Then, the name of the element and all its attributes are written to the console.

The following code fragment instantiates an XmlImporter using a MereHandler object, and calls the import method for the file called example.xml :

xmlImporter = new XmlImporter(new
            MereHandler());
xmlImporter.import("example.xml");

The application will show on the console the elements and the attributes contained in the file.

It should be very clear at this point that by implementing the Header interface, and in particular the processElement method, in the right way, you can transform the XML elements into C# objects. In fact, by using the xtReader object you have all the information you need: the name of the class, the fields, and their values. But how can we create dynamically objects starting from their names and fill them with the right data? The answer to this question is reflection.

Reflection and Metadata

Reflection is a generic term that covers some powerful features available within the .NET Framework. Java programmers have used reflection since Java was introduced. Now, Visual Basic, C#, ASP, and C++ (managed) programmers can also take advantage of reflection. As you will see in this section, by using reflection you can find out information about the types in your program or in other assemblies, and also to read other metadata from assembly manifests. Most of the classes are in the System.Reflection namespace. It's not the purpose of this case study to completely cover all subjects about reflection, but I will introduce the concepts that are useful to us in reaching our target.

Reflection will be used in this case study to provide a set of functionalities allowing us to:

  • create a C# object from a string representing the name of the class
  • discover a field in a C# object from a string representing the name of the field
  • set a value for that field.

The System.Type class helps us to achieve the first objective. A Type instance contains information about a specified C# base type as well as a user defined C# class. A way to get a Type instance is to use the static method GetType. This code fragment shows how you can create a Type object representing the class Person :

Type typeClass = Type.GetType("Person");

The typeClass variable is an instance of Type. By using typeClass we can get metadata containing different kinds of information about the class Person, such as its fields, properties, methods, constructors, etc. By using the Activator class you can even create an instance of Person :

Type typeClass = Type.GetType("Person");
if (typeClass==null)
Console.WriteLine("Class not found");
else
Person p = (Person) Activator.CreateInstance(typeClass);

The code above creates an instance of the Person class by using the GetType method, and then it checks if the returned reference is null. If null is returned then it means that the Person class does not exist, otherwise an instance of Person is created by invoking the static method CreateInstance of the System.Activator class. This system class contains methods to create types of objects locally or remotely, or obtain references to existing remote objects. In this case study, I'm going to use this class just to create the object defined in the XML document. Therefore, we have found a solution for the first objective: we can create C# objects starting with the name of the class.

Through the Type class, we can also get information about the fields contained in the class. Take a look at this code:

Type typeClass = Type.GetType("Person");
FieldInfo fieldInfo = typeClass.GetField("lastname");

The System.Reflection.FieldInfo class has the same meaning for fields as the System.Type class has for classes. An instance of FieldInfo contains information about a field of a class. As shown in the code, you can get a FieldInfo object by using the GetField method of Type. Thus, objective two is also solved: we can discovery a field of a C# class starting with its name.

To complete our objectives, we have to find a way to set the value of a field. This is a very trivial operation: the SetValue method of FieldInfo allows us to set a value for a field belonging to a specified object. The method takes two parameters: the parent object of the field, and the value. The following code shows how to set the value "Hill" to the lastname field of Person :

Type typeClass =
            Type.GetType("Person");
Person p = (Person)
            Activator.CreateInstance(typeClass);
FieldInfo fieldInfo =
            typeClass.GetField("lastname");
fieldInfo.setValue(p,"Hill");

We will be using the concepts covered in this section to map the XML elements to a C# object. In fact, by reading the XML you can get the name of the classes, and the name and the value of the fields as strings. Using this information - together with reflection - you can create the objects and fill the fields with the right values.

Mapping a simple object

We will start by implementing a component that only maps simple objects. A simple object is an object that only contains fields belonging to base types. To map the elements, the component has to know the XML structure; that is how the classes and the fields are represented inside the XML document. An example of the XML file is:

<class
            name="Temperature">
<field name="city" value="London"
            type="string"/>
<field name="dateTime" value="09/08/2002
            12.00" type="time"/>
<field name="temperature" value="23"
            type="short"/>
<field name="scale" value="C"
            type="char"/>
<field name="humidity" value="0.56"
            type="double"/>
</class>

There are two elements: <class> and <field>. The <class> element has the name attribute representing the name of the class. The <field> element has three attributes: name is the name of the field; value is the value of the field; and type that is one of the predefined data-type.

The XML schema describing the XML structure is as follows:

<?xml version="1.0" ?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">

<!-- Type -->
<xs:simpleType name="Type">
<xs:restriction base="xs:string">
<xs:enumeration value="string" />
<xs:enumeration value="integer" />
<xs:enumeration value="short" />
<xs:enumeration value="double" />
<xs:enumeration value="char" />
<xs:enumeration value="boolean" />
<xs:enumeration value="date" />
<xs:enumeration value="time" />
</xs:restriction>
</xs:simpleType>

<!-- Field -->
<xs:complexType name="Field">
<xs:attribute name="name" type="xs:string"/>
<xs:attribute name="value" type="xs:string" />
<xs:attribute name="type" type="Type" />
</xs:complexType>

<!-- Class -->
<xs:complexType name="Class">
<xs:sequence>
<xs:element name="field" type="Field"
minOccurs="0" maxOccurs="unbounded" />
</xs:sequence>
<xs:attribute name="name" type="xs:string" />
</xs:complexType>

<!-- classes -->
<xs:element name="classes">
<xs:complexType>
<xs:sequence>
<xs:element name="class" type="Class" maxOccurs="unbounded" />
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:schema>

As you may notice, this case study considers just eight data-types: string, integer, short, double, char, boolean, date, and time. An XML document, which is of a valid schema, consists of a sequence of <class> elements. A <class> element contains a sequence of <field> elements. The first version of the component we're going to develop only works with XML document that are valid for this schema.

A simple approach we can use to develop the mapping component is to implement the processElement of the Handler interface in a way that it reads the XML, and creates the objects and sets the field values by using reflection. Certainly, that is a perfectly valid solution, but we would not do this directly in the processElement method. Instead, we will divide the code reading the XML from the code creating the objects. For that reason, I thought to introduce the MappingFramework interface. The Handler implementation will have a MappingFramework instance and will delegate the issue of creating objects to it. Thus, we have distinct modules: the Handler, which reads and recognizes the XML elements, and the MappingFramework, which creates the objects and sets the field. The MappingFramework interface is as follows:

public interface MappingFramework {
void
            startClass(string className, string id);
object
            endClass();
void startField(string name, string val, string
            type);
void endField();
void startClassField(string name,
            string id);
void endClassField();
void startArray(string
            name);
void endArray();
void startItem(string
            className);
void endItem();
}

The interface contains five pairs of methods called startXXX and endXXX. For example, startClass will be called by the Handler when a <class> element is found, endClass will be invoked when a </class> element is found. For mapping simple objects, we will only implement the couple of methods related to the <class> and field <elements>. We will discuss the other methods in the next section. An empty implementation of MappingFramework is provided: this is the abstract class EmptyMappingFramework where each method does nothing.

Before starting the implementation of the mapping component for simple objects, I have to tell you something about the data-types. I already said that the component supports data-types, those are the ones indicated into the XML schema. However, those are generic data-types, thus we have to convert them into C# data-types when the field value is set. For this purpose, we will introduce the TypeConverter interface:

public interface TypeConverter {
string convert(string xmlType);
}

The convert method converts the generic-schema-defined data-type into the equivalent C# data-type. This is the implementation (TypeConverterImpl) used in this case study:

public class TypeConverterImpl :
            TypeConverter {
public string convert(string xmlType){
if
            (xmlType.CompareTo("string")==0)
return "System.String";
if
            (xmlType.CompareTo("short")==0)
return "System.Int16";
if
            (xmlType.CompareTo("integer")==0)
return "System.Int32";
if
            (xmlType.CompareTo("double")==0)
return "System.Double";
if
            (xmlType.CompareTo("char")==0)
return "System.Char";
if
            (xmlType.CompareTo("boolean")==0)
return "System.Boolean";
if
            (xmlType.CompareTo("date")==0)
return "System.DateTime";
if
            (xmlType.CompareTo("time")==0)
return
            "System.DateTime";
return null;
}
}

You can even implement your own TypeConverter to modify the data-types that this case study supports, or to add new ones. Each MappingFramework should have a TypeConverter instance.

Now we are ready to implement MappingFramework for the simple objects. These are the first lines of code:

public class MappingFrameworkImpl1 : EmptyMappingFramework {

// The current object
private object currentObj = null;

// The TypeConverter
private TypeConverter converter = new TypeConverterImpl();

The MappingFrameworkImpl1 class extends the EmptyMappingFramework abstract class. It has two private members: currentObj( the current object processed) and converter (a TypeComverterImpl instance).

The implementation of the startClass method is as follows:

public override void startClass(string className, string id) {

// Get the Type
Type typeClass = Type.GetType(className);
if (typeClass==null)
throw new System.Xml.XmlException("Class not found: " + className,null);

// Create the object
currentObj = Activator.CreateInstance(typeClass);
}

The method creates the object instance of the class named className by using reflection features such as Type.GetType and Activator.CreateInstance. The object is stored in the currentObj data member, thus this becomes the current object processed. The id parameter is not used for the simple object.

The implementation of the startField method is:

public override void startField(string name, string val, string type) {

// It Works on the current object
object obj = currentObj;

// Get the Type and FieldInfo
Type typeClass = obj.GetType();
FieldInfo fieldInfo = typeClass.GetField(name);
if (fieldInfo == null)
throw new System.Xml.XmlException
("Field not found: " + obj.GetType() + "." + name,null);

// Get the C# Type using the TypeConverter
string cSharpType = converter.convert(type);
if (fieldInfo.FieldType.ToString().CompareTo(cSharpType)!=0)
throw new System.Xml.XmlException(
"Different type found: " + obj.GetType() + "." + name +
"\n\rXML type: " + type +
"\n\rC# type: " + fieldInfo.FieldType,null);

try {
if (cSharpType.CompareTo("System.String")==0) {
fieldInfo.SetValue(obj,val);
}
if (cSharpType.CompareTo("System.Int32")==0) {
int v = int.Parse(val);
fieldInfo.SetValue(obj,v);
}
if (cSharpType.CompareTo("System.Int16")==0) {
short v = short.Parse(val);
fieldInfo.SetValue(obj,v);
}
if (cSharpType.CompareTo("System.Char")==0) {
char v = val[0];
fieldInfo.SetValue(obj,v);
}
if (cSharpType.CompareTo("System.Double")==0) {
double v = double.Parse(val);
fieldInfo.SetValue(obj,v);
}
if (cSharpType.CompareTo("System.Booelan")==0) {
bool v = bool.Parse(val);
fieldInfo.SetValue(obj,v);
}
if (cSharpType.CompareTo("System.DateTime")==0) {
DateTime v = DateTime.Parse(val);
fieldInfo.SetValue(obj,v);
}
}
catch (Exception) {
throw new System.Xml.XmlException(
"Error in field " + obj.GetType() +
"." + fieldInfo.Name +
"\n\rValue \"" + val +
"\" cannot be contaied in the type " + fieldInfo.FieldType
,null);
}
}

The Handler invokes the startField method when a <field> element is read from the XML. The method has three string parameters:

name | The name of the field (ex: "lastname")
val | The value of the field (ex: "Hill")
type | The generic-schema-defined type of the field (ex: "string")

This method does three things:

  • It creates the FieldInfo instance related to the field represented by name
  • It converts the generic-schema-defined type into a C# data-type by using the TypeConverter
  • It creates the right value (according to the C# data-type) and sets the value for that field

This method is the part of the proposed software that checks the data-types coming from the XML. As you can see, the method throws an exception both when the XML data-type is not the same as the one defined for the C# field, and when the value is not compatible with the data-type of the field.

The last MappingFramework method we implemt is closeClass. Here is the code:

public override object endClass(){
// Retur the current object
return currentObj;
}

This method is called by the Handler when a </class> element is read. This means that the information about the class is finished, thus the method returns the current object that, at this point, should have all the fields filled with the right values.

Now we can start to implement the Handler. We will name the class SimpleObjectHandler. The first lines of the code are:

public abstract class SimpleObjectHandler :
            Handler {

// Mapping Framework
protected MappingFramework
            framework;

// Constructor (using user's Mapping
            Framework)
public SimpleObjectHandler(MappingFramework framework)
            {
this.framework = framework;
}

// This must be
            implemented by derivated classes
public abstract void
            processObject(object obj);

The class is abstract because it does not implement the processObject method, which will be implemented by the concrete class extending SimpleObjectHandler. We will look at this topic later. The only data member of the class is a MappingFramework instance that, as you can see, will be initialized by the constructor.

The most interesting part of the SimpleObjectHandler class is the method processElement. It reads the XML elements and calls the right MappingFramework methods. This is the implementation:

public void
            processElement(XmlValidatingReader xtReader) {

// XML
            Elements
if (xtReader.NodeType==XmlNodeType.Element){

//
            Element is <class>
if (xtReader.Name.CompareTo("class")==0)
            {
xtReader.MoveToAttribute("name");
string className =
            xtReader.Value;
// Call the mapping
            framework
framework.startClass(className,null);
}

//
            Element is <field>
if (xtReader.Name.CompareTo("field")==0)
            {
// Find the name, value and type
            attributes
xtReader.MoveToAttribute("name");
string name =
            xtReader.Value;
xtReader.MoveToAttribute("value");
string val
            = xtReader.Value;
xtReader.MoveToAttribute("type");
string
            type = xtReader.Value;
// Call the mapping
            framework
framework.startField(name,val,type);
}
}

// XML End-Elements
if (xtReader.NodeType ==
            XmlNodeType.EndElement) {
// Element is </class>
if
            (xtReader.Name.CompareTo("class")==0) {
// Call the mapping
            framework
object obj = framework.endClass();
// Call
            overridden
            processObject
processObject(obj);
}
}
}

When the parser reads the <class> element, the method invokes startClass from the mapping framework; the className string is the value of the attribute name. If the read element is <field>, then the startField method is called - the name, val, and type parameters are taken from the respective attributes. If the read element is </class>, then the endClass method is called and then the processObject method is invoked. The class overriding that method could use the imported object in the same way as a normal C# object.

Now that the Handler and the MappingFramework have been implemented, we can carry out a simple program that reads the XML and shows the created objects with its fields. This is the code:

using System;
using System.Xml;
using
            System.Reflection;
using System.Collections;
using
            Wrox.Xml.Mapping;

// Example of using
            SimpleObjectHandler
public class ExampleHandler1:
            SimpleObjectHandler {
public ExampleHandler1() :
base(new
            MappingFrameworkImpl1()) {
}
public override void
            processObject(object obj){
// Show the created
            object
Console.WriteLine(obj); // ToString must be
            overriden
}
}

public class MainProgram {

public
            static void Main (String[] a) {

if
            (a.Length<1){
Console.WriteLine("No
            Params");
return;
}
string filename = a[0];

XmlImporter xmlImporter = new XmlImporter(new
            ExampleHandler1());

try
            {
xmlImporter.import(filename);
}
catch(System.Xml.XmlException
            e) {
Console.WriteLine(e.Message);
}
}
}

The class ExampleHandler1 extends SimpleObjectHandler and implements the abstract processObject method in order to show the imported object (the ToString method must be overridden for the object). The Main program is a console application taking in the XML file name as input. It creates an XMLImporter object, passing to the constructor a new ExampleHandler1 object. Then, it invokes the import method. Let's suppose that there is the following class in the system:

public class Temperature {

public string city;
public DateTime dateTime;
public short temperature;
public char scale;
public double dblHumidity;

public override string ToString() {
return "{" + city + ","
+ dateTime.ToString() + ","
+ temperature + ","
+ scale + ","
+ dblHumidity + "}";
}
}

Let's also suppose that the example-1.xml contains temperature elements like in the XML code we saw at the beginning of this section. The program, when given example-1.xml as input data, will show the imported Temperature objects, as in the following screen shot:

What we have done can work with simple objects. Now we will look at how, by implementing the MappingFramework and the Handler interfaces in a different way, you can get a more powerful component that can map complex objects, array of objects, and internal referenced objects.

Mapping complex object

A complex object is an object containing native type fields and nested objects. In this section, we will reimplement both Handler and MappingFramework in order that the component allows importing complex objects as well. The first thing to do is to define the way we describe the nested object in the XML document. We can modify the structure used for the simple objects, adding a new tag, <classfield>. This means the field is not a base-type, but another class. An example of XML code might be the following:

<class name="Country">
<field name="name" value="Italy" type="string"/>
<classfield name="continent">
<class name="Continent">
<field name="name" value="Europe" type="string"/>
<field name="population" value="320000000" type="integer"/>
</class>
</classfield>
</class>

The Country class has two fields: name and continent. The first one is a string, which is a base-type. The field continent is a <classfield>, hence it represents the Continent class that is nested in Country. To get the schema working with XML, you have to make some changes to the schema you have seen for the simple objects. In particular, you have to add the <classfield> element definition:

<!-- ClassField -->
<xs:complexType name="ClassField">
<xs:sequence>
<xs:element name="class" type="Class" maxOccurs="unbounded" />
</xs:sequence>
<xs:attribute name="name" type="xs:string" />
</xs:complexType>

In addition, you have to change the Class definition so that it can support a sequence of both <field> and <classfield> elements:

<!-- Class -->
<xs:complexType name="Class">
<xs:sequence>
<xs:element name="field" type="Field"
minOccurs="0" maxOccurs="unbounded" />
<xs:element name="classfield" type="ClassField"
minOccurs="0" maxOccurs="unbounded" />
</xs:sequence>
<xs:attribute name="name" type="xs:string" />
</xs:complexType>

You can find the complete XML schema in the support material for this case study - the name of the file is mapping-2.xsd.

To implement the MappingFramework interface for the complex objects, we can start from the one we implemented for the simple objects and try to understand what has to be changed. The main difference is represented by the current object concept (currentObj data member). For supporting nested objects, you cannot use a single member variable to store the object currently processed, because the current object changes each time a nested objects occurs in the document. After a nested object has been processed, the parent object returns to be the current object. Consequently, we need to store the objects in a stack; the current object will always be the element at the top of the stack. Although the .NET Framework provides the Stack class, we can't use it in this context. We need a special version of Stack, which contains these methods:

public void push(object obj) | Pushes an object into the stack
public object pop() | Takes out the object at the top of the stack
public object top(int n) | Returns the object that is -n objects from the top. top(0) // Returns the top top(-1) // Return the object before the top
public bool empty() | Returns true if the stack has no object

The System.Collections.Stack included in .NET does not provide a method similar to top(int n) and, as you will see, we need such method in this situation. The MappingFrameworkImpl2 that extends EmptyMappingFramework starts like this:

public class MappingFrameworkImpl2 : EmptyMappingFramework {

// Stack for the objects
private Stack stackObject = new Stack();

// TypeConverter
private TypeConverter converter = new TypeConverterImpl();

The private member stackObject is the stack where the current objects will be stored.

Let's implement the MappingFramework interface methods. The startClass method now works on the object at the top of the stack that can either be a root or a nested object:

public override void startClass(string
            className, string id) {

// Get the Type
Type typeClass =
            Type.GetType(className);
if (typeClass==null)
throw new
            System.Xml.XmlException("Class not found: " +
            className,null);

// Create a new object
object obj =
            Activator.CreateInstance(typeClass);
stackObject.push(obj);

}

After the object is created, the method puts it in onto the stack. From this moment on, the object is the current object.

The startField method is basically the same as the startField method we implemented for the simple objects. The only difference is that we are now working on the object at the top of the stack instead of working on the currentObject variable:

public override void startField(string name, string val, string type) {

// It works on the stack top-object
object obj = stackObject.top(0);

// ... same implementation of the one for
// the simple objects ...

}

The endClass method is as follows:

public override object endClass(){

object top = stackObject.pop();
if (stackObject.empty())
return top;

return null;
}

The method takes the element from the top of the stack but only returns it when the stack is empty, that is, when all the nested objects have been processed. This means that the endClass method only returns a non-null value if the last object we process is a root object.

For this implementation, we also need to implement the startClassField method of MappingFramework. Each time the parser reads a <classfield> element, the Handle calls startClassField. This method attaches the nested object to the right field of its parent. The method has two parameters: name and id. The name string represents the name of the field; id is not used here. When this method is called, there is the object at the top of the stack that must be assigned to the field specified by name (child variable). The object previously at the top is the object holding the field specified by name (parent variable). What the method does is to set the value of the parent's name field to the object child. Here is the startClassField implementation:

public override void startClassField(string name, string id) {

// The child is the current object...
object child = stackObject.top(0);
// ... and the parent the previuos one
object parent = stackObject.top(-1);

// Get the Type and the FieldInfo
Type typeClass = parent.GetType();
FieldInfo fieldInfo = typeClass.GetField(name);
if (fieldInfo == null)
throw new System.Xml.XmlException
("Field not found: " + parent.GetType() + "." + name,null);

// Attach the child object to the parent
try {
fieldInfo.SetValue(parent,child);
}
catch(Exception) {
throw new System.Xml.XmlException
("Error setting field: " + parent.GetType() + "." + name,null);
}
}

Implementing MappingFramework is not enough; you also have to make some little changes to the Handle implementation. In the support material provided with this case study, you can find the class ComplexObjectHandler implementing Header for supporting complex objects. The implementation is very close to the SimpleObjectHandler implementation. The only difference is that in the processElement, there is a new if statement to check the <classfield> element. In this case, the startClassField method is called. Here is a code fragment:

public void
            processElement(XmlValidatingReader xtReader) {

if
            (xtReader.NodeType==XmlNodeType.Element){

// Element is
            <class>
if (xtReader.Name.CompareTo("class")==0) {
//
            ...

// Element is <field>
if
            (xtReader.Name.CompareTo("field")==0) {
// ...

// Element
            is <classfield>
if
            (xtReader.Name.CompareTo("classfield")==0) {

// Find name
            attribute
while(xtReader.NodeType!=XmlNodeType.Element)
xtReader.Read();
xtReader.MoveToAttribute("name");
string fieldName =
            xtReader.Value;

// Find name attribute (of the nested
            class)
while(xtReader.NodeType!=XmlNodeType.Element)
xtReader.Read();
xtReader.MoveToAttribute("name");
string
            className = xtReader.Value;

// Call the mapping framework
            (creating the nested class)
framework.startClass(className,null);
// Call the mapping framework (attaching the nested
            class)
framework.startClassField(fieldName,null);
}
//
            ...
}

When the parser reads a <classfield> element, the method reads the name attribute and puts it in the fieldName string. According to the XML schema, the next element must be <class>, and so the method reads the name attribute of <class> representing the name of the class, and puts it in the className string. Finally, the startClass and the startClassField methods are invoked.

MappingFrameworkImpl2 and ComplexObjectHandler can be used to implement a concrete Handler that reads the XML and shows the created objects, its fields, and in the proper case, the nested objects:

public class ExampleHandler2: ComplexObjectHandler {
public ExampleHandler2() :
base(new MappingFrameworkImpl2()) {
}
public override void processObject(object obj){
// Show the created object
Console.WriteLine(obj); // ToString must be overriden
}
}

Following what we looked at in the previous section, you could execute a Main program using ExampleHandler2.

Mapping array of object

In this section, we add a new feature to the component to import objects containing array of objects. An example of XML using arrays is as follows:

<class name="Country">
<field name="name" value="Italy" type="string"/>
<array name="majorCities">
<item name="City">
<field name="name" value="Rome" type="string"/>
<field name="county" value="Lazioe" type="string"/>
</item>
<item name="City">
<field name="name" value="Napoli" type="string"/>
<field name="county" value="Campania" type="string"/>
</item>
</array>
</class>

As you can see, there are two new elements: <array> and <item>. The <array> element indicates the field is an array; it has the attribute name, which is the name of the array field. An <array> is a sequence of <item> elements, which have exactly the same structure of the <class> element. The XSD code fragment to define the <array> element is as follows:

<!-- Array -->
<xs:complexType name="Array">
<xs:sequence>
<xs:element name="item" type="Class" maxOccurs="unbounded" />
</xs:sequence>
<xs:attribute name="name" type="xs:string" />
</xs:complexType>

Moreover, the <class> element has to allow <array> elements:

<!-- Class -->
<xs:complexType name="Class">
<xs:sequence>
<xs:element name="field" type="Field"
minOccurs="0" maxOccurs="unbounded" />
<xs:element name="classfield" type="ClassField"
minOccurs="0" maxOccurs="unbounded" />
<xs:element name="array" type="Array"
minOccurs="0" maxOccurs="unbounded" />
</xs:sequence>
<xs:attribute name="name" type="xs:string" />
</xs:complexType>

The complete XML schema is present in the support material of this case study; the name of the file is mapping-3.xsd.

To support arrays you have to implement both a new MappingFramework and a new Handler. The MappingFramework implementation will use two stacks, one for the current objects and the other one for the current arrays. The philosophy behind the stacks is the same as before. The beginning of the class MappingFrameworkImpl3, which implements MappingFrameworks, is as follows:

public class MappingFrameworkImpl3 : EmptyMappingFramework {

// Stack for the objects
private Stack stackObject = new Stack();

// Stack for the arrays
private Stack arrayStack = new Stack();

The implementation of startClass, startField, startFieldClass and endClass are the same as the ones we implemented previously. However, we now need to add the implementation of new methods such as startArray, startItem and endArray.

The Handler calls the startArray method when the parser reads an <array> element. This is the implementation:

public override void startArray(string name)
            {

// Get the top-object from the stack
object obj =
            stackObject.top(0);

// Get the Type and the FieldInfo
Type
            typeClass = obj.GetType();
FieldInfo fieldInfo =
            typeClass.GetField(name);
if (fieldInfo == null)
throw new
            System.Xml.XmlException
("Field not found: " + obj.GetType() +
            "." + name,null);

try {
// Create a new
            ArrayList
ArrayList array = new ArrayList();
// Attach that
            array to the current object
fieldInfo.SetValue(obj,array);
//
            Push the array into the
            array-stack
stackArray.push(array);
}
catch(Exception)
            {
throw new System.Xml.XmlException
("Error setting field: " +
            obj.GetType() + "." + name,null);
}

The name parameter represents the name of the field containing a reference to the array. This field is always an ArrayList ; in fact, I decided to use this structure to map the arrays. The method creates a new empty ArrayList and assigns it to the name field of the current object, which is at the top of the stackObject. In addition, the ArrayList is pushed onto the stackArray.

The Handler calls the method startItem when the parser reads an <item> element. The method creates the object, pushes it on the stackObject, and finally adds the created object to the ArrayList located at the top of the stackArray :

public override void startItem(string
            className) {

// Get the Type
Type typeClass =
            Type.GetType(className);
if (typeClass==null)
throw new
            System.Xml.XmlException("Class not found: " +
            className,null);

// Create a new object
object obj =
            Activator.CreateInstance(typeClass);
stackObject.push(obj);

//
            Add the item to the top-stack-array element
ArrayList array =
            (ArrayList) stackArray.top(0);
array.Add(obj);
}

The endArray method is called by the Handler when the parser finds an </array> element. The implementation is:

public override void endArray()
            {
stackArray.pop();
}

It takes the last ArrayList processed from the stackArray.

ArrayHandler is the implementation of the Handler which works with the simple objects, complex objects, and arrays. An version of the class is shown below, the code shows only the differences with ComplexObjectHandler :

public abstract class ArrayHandler : Handler
            {

// ...

public void
            processElement(XmlValidatingReader xtReader) {

if
            (xtReader.NodeType==XmlNodeType.Element){

// Element is
            <class>
if (xtReader.Name.CompareTo("class")==0) {
//
            ...

// Element is <field>
if
            (xtReader.Name.CompareTo("field")==0) {
// ...

// Element
            is <classfield>
if
            (xtReader.Name.CompareTo("classfield")==0) {
//...

//
            Element is <array>
if (xtReader.Name.CompareTo("array")==0)
            {

// Find name
            attribute
while(xtReader.NodeType!=XmlNodeType.Element)
xtReader.Read();
xtReader.MoveToAttribute("name");
string
            arrayName = xtReader.Value;

// Call the mapping
            framework
framework.startArray(arrayName);
}

// Element
            is <item>
if (xtReader.Name.CompareTo("item")==0) {
//
            Find name attribute
xtReader.MoveToAttribute("name");
// Call
            the mapping framework
framework.startItem(xtReader.Value);
}

}

// XML End-Elements
if (xtReader.NodeType
            == XmlNodeType.EndElement) {

// Element is </classes>
            or </item>
if (xtReader.Name.CompareTo("class")==0
||
            xtReader.Name.CompareTo("item")==0) {
// Call the mapping
            framework
object obj = framework.endClass();
// Call
            overridden processObject
// only for not nested classes
            (obj!=null)
if (obj!=null) processObject(obj);
}

//
            Element is </array>
if
            (xtReader.Name.CompareTo("array")==0) {
// Call the mapping
            framework
framework.endArray();
}
}
}
}

When an <array> element is found, the processElement method gets the name of the field holding the array (nameArray), and calls the startArray method. When an <item> element is found, the method gets the name of the class and invokes the startItem method. The processElement methods treats the </item> element like a </class> element. Eventually, when the parser reads the </array> element, the endArray method is called.

MappingFrameworkImpl3 and ArrayHandler can be used to implement a concrete Handler that reads the XML and shows the created objects, its fields, and if applicable, the nested objects and the arrays:

public class ExampleHandler3: ArrayHandler {
public ExampleHandler3() :
base(new MappingFrameworkImpl3()) {
}
public override void processObject(object obj){
// Show the created object
Console.WriteLine(obj); // ToString must be overriden
}
}

Following on from what we looked at in the "simple object" section, you could execute a Main program using ExampleHandler3.

Internal References

In this section, we're going to look at how to enhance the component so that it can deal with internal references among <class> elements. This will be the final version of the component mapping XML to C# objects. Let's consider the following XML:

<class name="Person",
            id="1000">
<field name="surname" value="Hill"
            type="string"/>
<field name="name" value="Robert"
            type="string"/>
</class>

<class
            name="Company">
<field name="name" value="Hill and Son"
            type="string"/>
<classfield name="chairman"
            id="1000"/>
</class>

When the component reads the Person object, it creates the "Robert Hill" Person object whose id is 1000. Then the component reads the <class name="Company"> element, and creates the "Hill & Son" Company object, whose field chairman is a reference to the Person with id=1000 which was previously created (i.e. "Robert Hill"). The internal references are useful for two main reasons:

  • They allow us to avoid defining the same class lots of times within the same XML document; you can define the class once and then reference it by using the id number;
  • They permit us to carry out one-to-n relationship between class elements.

In the previous example, I wrote the Person class just once. In addition, if "Robert Hill" is the chairman of another company as well, then we can simply add the new company with a reference to the "Robert Hill" Person, like this:

<class name="Company">
<field name="name" value="Hill Ltd" type="string"/>
<classfield name="chairman" id="1000"/>
</class>

Hence, two companies - "Hill and son" and "Hill Ltd" - have the same chairman, "Robert Hill". The component will create two Company objects whose chairman field is a reference to the same Person object, so only one Person object is created! Unfortunately we can't have this feature with the developed component, we have to add the new "internal reference" functionality.

The first step is to adapt the XML schema for supporting this new feature. This is a very simple operation, all we have to do is add the attribute id to the related XSD complex-type of both the <class> and <classfield> elements:

<!-- Add this line to Class and ClassField complex type -->
<xs:attribute name="id" type="xs:string"/>

The id attribute is not mandatory: you specify it only when you want to use the internal references. The complete XSD file is part of the support material for this case study, the name of the file is mapping.xsd.

As usual, first we implement the MappingFramework and then the Handler. The idea is to put the objects having an id into a hash table. Thus, when a <classfield> has the id, it is used to retrieve the right object from the hash table. The MappingFrameworkImpl, the definitive version, is a class implementing MappingFramework. It starts like this:

public class MappingFrameworkImpl : EmptyMappingFramework {

// Stack for the objects
private Stack stackObject = new Stack();

// Stack for the arrays
private Stack stackArray = new Stack();

// Cache for the ID-Objects
private Hashtable idTable = new Hashtable();

As you may notice, the class has the idTable data member, which is the hash table where we put the object with the id.

The startClass must be implemented again. This final version will use the id parameter:

public override void startClass(string
            className, string id) {

// Get the Type
Type typeClass =
            Type.GetType(className);
if (typeClass==null)
throw new
            System.Xml.XmlException("Class not found: " +
            className,null);

// Create a new object
object obj =
            Activator.CreateInstance(typeClass);
stackObject.push(obj);

//
            Store the object whether it has ID
if (id!=null)
try
            {
idTable.Add(id,obj);
}
catch(Exception e){
throw new
            System.Xml.XmlException(e.Message,null);
}
}

You may see that after the creation, the object, if the id is non-null, is put into the hash table.

The startClassField must also be implemented again:

public override void startClassField(string
            name, string id) {

object child = null;
object parent =
            null;

if (id==null){
// XML Element classfield is followed
            by a class definition
// In this case the child is the current
            object...
child = stackObject.top(0);
// ... and the parent
            the previuos one
parent = stackObject.top(-1);
}
else
            {
// XML Element classfield has a class ID
// In this case the
            child in the hashtable...
child = idTable[id];
// ... and the
            parente is the current object
parent = stackObject.top(0);
if
            (child==null)
throw new System.Xml.XmlException("Id not found: "
            + id,null);
}

// Get the Type and the FieldInfo
Type
            typeClass = parent.GetType();
FieldInfo fieldInfo =
            typeClass.GetField(name);
if (fieldInfo == null)
throw new
            System.Xml.XmlException
("Field not found: " + parent.GetType() +
            "." + name,null);

// Attach the child object to the
            parent
try
            {
fieldInfo.SetValue(parent,child);
}
catch(Exception)
            {
throw new System.Xml.XmlException
("Error setting field: " +
            parent.GetType() + "." + name,null);
}
}

If id=null, the method has the same behavior as the startClassField method you have seen in the previous section. When id is not null, the parent object is at the top of the stack and the child object is in the hash table. That's because the <classfield> element, which has id, is not followed by any class element, thus no child object has been created and put into the stack. Instead, the child should be in the hash table, because it has been created by some previous class element having id.

Other methods of MappingFramework are unchanged from the implementation you have seen, in the previous section, for the arrays.

CompleteHandler is the definitive implementation of the Handler, which works with simple objects, complex objects, arrays, and internal references. An amended version of the class is here shown, the code represents the differences with the ArrayHandler implementation:

public abstract class CompleteHandler :
            Handler {

// ...

public void
            processElement(XmlValidatingReader xtReader) {

if
            (xtReader.NodeType==XmlNodeType.Element){

// Element is
            <class>
if (xtReader.Name.CompareTo("class")==0)
            {

// Find name
            attribute
xtReader.MoveToAttribute("name");
string ClassName =
            xtReader.Value;

// Find id
            attribute
xtReader.MoveToAttribute("id");
string id =
            null;
if (xtReader.Name.CompareTo("id")==0)
id =
            xtReader.Value;

// Call the mapping
            framework
framework.startClass(ClassName,id);

}

//
            Element is <field>
if (xtReader.Name.CompareTo("field")==0)
            {
// ...

// Element is <classfield>
if
            (xtReader.Name.CompareTo("classfield")==0) {

// Finds name
            attribute
while(xtReader.NodeType!=XmlNodeType.Element)
xtReader.Read();
xtReader.MoveToAttribute("name");
string fieldName =
            xtReader.Value;

// Finds id
            attribute
xtReader.MoveToAttribute("id");
string idField =
            null;
if (xtReader.Name.CompareTo("id")==0)
idField =
            xtReader.Value;

if (idField==null) {
// id not
            present
// Processe the nested <class>
            element
while(xtReader.NodeType!=XmlNodeType.Element)
xtReader.Read();

//
            Find name attribute (of the nested
            class)
xtReader.MoveToAttribute("name");
string className =
            xtReader.Value;

// Find id attribute (of the nested
            class)
string idClass =
            null;
xtReader.MoveToAttribute("id");
if
            (xtReader.Name.CompareTo("id")==0)
idClass =
            xtReader.Value;

// Call the mapping framework (creating the
            nested class)
framework.startClass(className,idClass);
//
            Call the mapping framework (attaching the nested
            class)
framework.startClassField(fieldName,null);
}
else
            {
// id present
// Call the mapping framework
// lookup
            referred class where id =
            idField
framework.startClassField(fieldName,idField);
}

}

// Element is <array>
if
            (xtReader.Name.CompareTo("array")==0) {
// ...

// Element
            is <item>
if (xtReader.Name.CompareTo("item")==0)
            {
//...

}

// XML End-Elements
if
            (xtReader.NodeType == XmlNodeType.EndElement) {
//...
}
}

As you may notice, even this is an abstract class. This is because the processObject method has not been implemented. Therefore, to use the CompleteHandler, you have to extend it, providing the implementation of processObject. This method depends on the problem you are facing, and by implementing it you can specify how the application has to process the created objects. I deliberately made CompleteHandler abstract so that you must necessarily implement the processObject method - an instance of CompleteHandler without a processObject complete implementation is nonsense.

In the code above, the changed parts are the <class> and <classfield> elements. When the parser reads the <class> element, the method tries to find the id. If it is present, then the id is passed to the startClass method. The situation is a little bit complex when the parser finds a <classfield> element. According to the XML schema, this element can either have the id attribute or be followed by a <class> element. In the first case, the startClassField method is called, passing the field name and the id (see else block). When the id of the <classfield> is not present, then a <class> element follows. Thus, the method gets the name of the class and, if applicable, the id. Finally, it calls the startClass and startClassField methods.

MappingFrameworkImpl and CompleteHandler can be used to implement a concrete Handler that reads the XML and shows the created objects and its fields:

public class ExampleHandler4: ArrayHandler
            {
public ExampleHandler4() :
base(new MappingFrameworkImpl())
            {
}
public override void processObject(object obj){
// Show
            the created object
Console.WriteLine(obj); // ToString must be
            overriden
}
}

Following what was explained in the "simple object" section, you could execute a Main program using ExampleHandler4.

The Sample Application

To show how the developed component can be integrated in a real-word context, I'm going to implement a sample application. This can be seen as a module performing a particular task of a logistics system. The application consists of a form where the warehouse-stock is showed article-by-article. External systems, via XML documents, inform the application that new customer orders or purchase orders have arrived. The application reads the XML, creates the customer orders and/or the purchase orders, updates the stock of the involved articles, and records the warehouse movement. Through the form, the user can monitor the article stocks and the recorded movements.

The final application will look like the one shown below:

Database Design and Structure

The class diagram representing the database of the application is shown the following diagram:

The Order abstract class can have two generalizations: CustomerOrder and PurchaseOrder. The first one is an order coming from a customer requiring some goods. The second one is an order made by the warehouse to a supplier for purchasing some kind of goods. Both types of orders have OrderLine aggregated, which tells us, among other things, what the article is and how many pieces are required. In fact, as you can see, the OrderLine class has an association to the class Article, which is the article that can be purchased or sold. Each article contains a list of Movement objects. The system creates a movement each time the article-stock changes due to a customer order or a purchase order.

The application database consists of in-memory C# objects. We could also permanently store the objects in a database such as SQL Server or Oracle, but that is not the purpose of this case study. The main goal of this piece is to explain how to import data from XML and create objects using it, not how to store objects into a relational database.

The code for the abstract class Order is as follows:

public abstract class Order {
public int number;
public DateTime date;
public ArrayList orderLines;
}

An order consists of a number, a date, and an array of OrderLine objects. There are two generalizations of this abstract class: CustomerOrder and PurchaseOrder, which are implemented this way:

public class CustomerOrder : Order
            {
public string customer;
}

public class PurchaseOrder
            : Order {
public string supplier;
}

As you can see, the only difference is that the first class has a reference to a customer and the second one to a supplier. Obviously, in more complex systems there would be many other differences.

The OrderLine, CustomerOrderLine, and PurchaseOrderLine implementations are as follows:

public abstract class OrderLine {
public
            int number;
public int quantity;
public double
            price;
public Article article;
public abstract int
            getRelativeQuantity();
public abstract string
            getType();
}

public class CustomerOrderLine :
            OrderLine{
public override int getRelativeQuantity(){
return
            -quantity;
}
public override string getType(){
return
            "Customer Order Line ";
}
}

public class
            PurchaseOrderLine : OrderLine{
public override int
            getRelativeQuantity(){
return quantity;
}
public override
            string getType(){
return "Purchase Order
            Line";
}
}

The OrderLine class holds four data members representing the number, the article, the quantity of the article (in pieces), and the price. In addition, the class has two abstract methods that will be implemented by its generalizations.

In fact, the CustomerOrderLine class implements getType in order to return the type of the line as a string, that is "Customer Order Line". The getRelativeQuantity returns -quantity, because for a customer order the goods are leaving the warehouse. Likewise, for the PurchaseOrderLine the getType method returns "Purchase Order Line", and the getRelativeQuantity returns quantity, because in this case the goods are coming in the warehouse.

The Article class is:

public class Article {
public string
            code;
public string description;
public int stock;
public
            ArrayList movements = new ArrayList();
}

An article has a code and a description. The stock member is the quantity currently present in the warehouse. The movements ArrayList contains a list of Movement objects:

public class Movement {
public int
            number;
public OrderLine cause;
}

A warehouse movement consists of a number and a cause, which is the OrderLine that causes the movement.

The class that stores, finds, and manages those business objects is called DataController ; it contains these methods:

// Add an article
public void addArticle(Article article)

// Find an article
public Article lookupArticle(string code)

// Retur all the articles
public ArrayList getAllArticles()

// Find an order
public Order lookupOrder(int number)

// Add an order
public void addOrder(Order order)

// Import objects from an XML file
// It uses the XmlImporter and the CompleteHandler
public void import(string fileName)

You can find the complete implementation of DataController in the support material supplied with this case study.

UI Design and Coding

To obtain a form that looks like the one we saw before, you can follow these steps:

1. Add a new Window Form to the project into the file MainForm.cs.

2. Sets the following properties of the form:

Property | Value
Name | MainFrom
Text | Stock Situation

3. Add the Labels Articles and Movements to the MainForm.

4. Add a ListView named listArticle and put in under the Articles Label.

5. Add a ListView named listMovement and put it under the Movements Label.

6. Add the Label XML File to the MainForm.

7. Add a Text Box next the XML File label and set the property name to "txtXmlFileName".

8. Add a new Button next to the txtXmlFileName Text Box with these properties:

Property | Value
Name | btnImport
Text | Import

The listArticle ListView will show all the articles present in the warehouse and their stocks. When the user clicks on a row, the listMovement will show all the movements related to the selected article. To make the handling of the lists to the MainForm independent, I implemented two classes: ArticleListView and MovementListView. Both implementations are very similar. Here is the code for ArticleListView :

public class ArticleListView {

// The
            ListView
private ListView listView;

// The
            DataController
private DataController controller;

//
            Constructor
public ArticleListView(ListView listView,
            DataController controller){

this.listView =
            listView;
this.controller = controller;

// Set the
            listView proporties
listView.Clear();
listView.View =
            View.Details;
listView.FullRowSelect =
            true;
listView.GridLines = true;

// Column
            Headers
listView.Columns.Add("Article code", -2,
            HorizontalAlignment.Left);
listView.Columns.Add("Description",
            -2, HorizontalAlignment.Left);
listView.Columns.Add("Stock", -2,
            HorizontalAlignment.Left);

// Reload the List
reload();

// Set empty images for the items
ImageList
            imageListSmall = new ImageList();
ImageList imageListLarge = new
            ImageList();
listView.LargeImageList =
            imageListLarge;
listView.SmallImageList =
            imageListSmall;

}

// Add an article to the
            List
public void addArticle(Article article) {
ListViewItem
            item = new
            ListViewItem(article.code,0);
item.SubItems.Add(article.description);
item.SubItems.Add(""+article.stock);
listView.Items.AddRange(new
            ListViewItem[]{item});
}

// Reload the List
public
            void reload() {
listView.Items.Clear();
ArrayList articles =
            controller.getAllArticles();
for (int i=0; i<articles.Count;
            i++)
addArticle((Article)articles[i]);
}
}

The class has the listView data member that is the List View relative to the articles, and controller that is an instance of the DataController class. The constructor accepts as input both listView and controller. It also sets the properties for listView, the column headers, and populates the list. The class has two public methods. The first one is addArticle, which adds an article to the list. The second one is reload, which reloads listView with the articles provided by the controller instance. The MainForm will only access the article List View (listArticle) by an instance of the ArticleListView class (named articleListView), and the movement List View (listMovement) by an instance of MovementListView (named movementListView). You can find the complete implementation of MovementListView with the support material provided with this case study.

The MainForm class has the following data members:

// DataController
private DataController controller = new DataController();
// ListView for the articles
private ArticleListView articleListView;
// ListView for the movements
private MovementListView movementListView;

The MainForm_Load method will create the ArticleListView :

private void MainForm_Load(object sender,
            System.EventArgs e) {
articleListView = new
            ArticleListView(listArticle, controller);
}

When the user selects an article, the listArticle_SelectedIndexChanged method will be invoked. It creates a new MovementListView populated with the movements of the selected article:

private void listArticle_SelectedIndexChanged(object sender, System.EventArgs e) {
// When the user clicks on a ArticleListView item
// a movementListView is created with the movements
// belonging to the selected article
if (listArticle.SelectedItems.Count != 1) return;
string code = listArticle.SelectedItems[0].SubItems[0].Text;
Article article = controller.lookupArticle(code);
movementListView =
new MovementListView(listMovement, controller, article);
}

If you create some articles and run the application, you should obtain something like this:

Importing XML

By means of the txtXmlFileName Text Box, you can specify the XML file to import. Pressing the Import button causes the importing process to start. Below is the button click event for the Import button:

private void btnImport_Click(object sender,
            System.EventArgs e) {
// Import XML and reload the
            ArticleListView
try
            {
controller.import(txtXmlFileName.Text);
articleListView.reload();
}
catch(Exception
            ex){
MessageBox.Show(ex.Message);
}
}

As you may notice, the method calls the import method of the DataController class, and then reloads the articleListView by calling the reload method. If no exception occurs, you should see the imported articles and their movements inside the lists.

The import method of the DataController class is implemented like this:

public void import(string fileName) {
XmlImporter xmlImporter = new XmlImporter(new
            ImportBO(this));
xmlImporter.import(fileName);
}

The method creates a new XmlImporter passing a new ImportBO handler and then calls the import method. The ImportBo handler is a concrete implementation of the CompleteHandler :

class ImportBO : CompleteHandler {

// Use a DataController to deal with the data
private DataController controller;

// Constructor
// MappingFrameworkImpl is used
public ImportBO(DataController controller) :
base(new MappingFrameworkImpl()){
this.controller = controller;
}

// ...to be continued

It has a DataController data member (controller). The constructor takes in the DataController class as input and calls the base constructor, passing a new MappingFrameworkImpl instance.

The processObject method is finally implemented in the following way:

public override void processObject(object
            obj){

// Import an Article
if (obj is Article)
            {
importArticle((Article)obj);
}

// Import an
            Order
if (obj is Order) {
importOrder((Order)
            obj);
}

}

In this application, the imported objects can only be articles or orders. The method delegates the operations to the importArticle and importOrder private methods.

The importArticle method adds a new Article object to the database handled by the DataController. The method is quite smart, as it in fact only adds the article if it is not already present in the database. This is the code:

private void importArticle(Article article) {
Article stockedArticle = controller.lookupArticle(article.code);
if (stockedArticle==null)
controller.addArticle(article);
}

The importOrder method is a little bit more complex. Actually, importing an order means:

  • Checking if the order is already booked;
  • Adding the order to the database;
  • Ceating a new movement for each order line;
  • If needed, adding new articles involved in the order line;
  • Updating the stock.

The code that does all these things is:

private void importOrder(Order order)
            {

Order bookedOrder =
            controller.lookupOrder(order.number);
// Continue only if the
            order does not exist
if (bookedOrder==null) {
// Add the
            order
controller.addOrder(order);
// Processing lines to
            create movements
for (int i=0; i<order.orderLines.Count; i++)
            {
// Get the line
OrderLine line = (OrderLine)
            order.orderLines[i];
// Import the
            article
importArticle(line.article);
// Set line.article to
            the imported article or to
// an already existent article with
            the same code
line.article =
            controller.lookupArticle(line.article.code);
// Add a
            movement
Movement movement = new Movement();
movement.number =
            line.article.movements.Count + 1;
movement.cause =
            line;
line.article.movements.Add(movement);
// Update the
            stock
line.article.stock +=
            movement.cause.getRelativeQuantity();
}
}
}

The application is now complete. The following XML contains three articles and a purchase order:

<!-- firstload.xml -->
<?xml
            version="1.0" encoding="utf-8" ?>

<classes
            xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:noNamespaceSchemaLocation="..\mapping.xsd">

<class
            name="Article" id="01005">
<field name="code" value="01005"
            type="string"/>
<field name="description" value="Sony
            Mobile Phone" type="string"/>
</class>

<class
            name="Article" id="01006">
<field name="code" value="01006"
            type="string"/>
<field name="description" value="Philips TV
            Color" type="string"/>
</class>

<class
            name="Article" id="01007">
<field name="code" value="01007"
            type="string"/>
<field name="description" value="Aiwa
            Video-Tape Recorder"
            type="string"/>
</class>

<class
            name="PurchaseOrder">
<field name="number" value="300"
            type="integer"/>
<field name="supplier" value="SGH Ltd"
            type="string"/>
<field name="date" value="13/08/2002"
            type="date"/>
<array name="orderLines">
<item
            name="PurchaseOrderLine">
<field name="number" value="1"
            type="integer"/>
<field name="quantity" value="5000"
            type="integer"/>
<field name="price" value="59.90"
            type="double"/>
<classfield name="article"
            id="01005"/>
</item>
<item
            name="PurchaseOrderLine">
<field name="number" value="2"
            type="integer"/>
<field name="quantity" value="3500"
            type="integer"/>
<field name="price" value="139.90"
            type="double"/>
<classfield name="article"
            id="01006"/>
</item>
<item
            name="PurchaseOrderLine">
<field name="number" value="3"
            type="integer"/>
<field name="quantity" value="1500"
            type="integer"/>
<field name="price" value="199.90"
            type="double"/>
<classfield name="article"
            id="01007"/>
</item>
</array>
</class>

</classes>

If you import this file, the form should look like the one in the figure below, where you can see the movements related to the article 01005:

Now, you can try to import another file having both customer orders and purchase orders. For example, use the following:

<!-- order.xml -->
<?xml
            version="1.0" encoding="utf-8" ?>

<classes
            xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:noNamespaceSchemaLocation="..\mapping.xsd">

<class
            name="Article" id="01005">
<field name="code" value="01005"
            type="string"/>
<field name="description" value="Sony
            Mobile Phone" type="string"/>
</class>

<class
            name="Article" id="01006">
<field name="code" value="01006"
            type="string"/>
<field name="description" value="Sony TV
            Color" type="string"/>
</class>

<class
            name="Article" id="01007">
<field name="code" value="01007"
            type="string"/>
<field name="description" value="Aiwa
            Video-Tape Recorder"
            type="string"/>
</class>

<class
            name="CustomerOrder">
<field name="number" value="400"
            type="integer"/>
<field name="customer" value="N-B Ltd"
            type="string"/>
<field name="date" value="13/08/2002"
            type="date"/>
<array name="orderLines">
<item
            name="CustomerOrderLine">
<field name="number" value="1"
            type="integer"/>
<field name="quantity" value="50"
            type="integer"/>
<field name="price" value="99.90"
            type="double"/>
<classfield name="article"
            id="01005"/>
</item>
<item
            name="CustomerOrderLine">
<field name="number" value="2"
            type="integer"/>
<field name="quantity" value="100"
            type="integer"/>
<field name="price" value="399.90"
            type="double"/>
<classfield name="article"
            id="01006"/>
</item>
</array>
</class>

<class
            name="CustomerOrder">
<field name="number" value="401"
            type="integer"/>
<field name="customer" value="Hill Ltd"
            type="string"/>
<field name="date" value="14/08/2002"
            type="date"/>
<array name="orderLines">
<item
            name="CustomerOrderLine">
<field name="number" value="1"
            type="integer"/>
<field name="quantity" value="150"
            type="integer"/>
<field name="price" value="99.90"
            type="double"/>
<classfield name="article"
            id="01005"/>
</item>
<item
            name="CustomerOrderLine">
<field name="number" value="2"
            type="integer"/>
<field name="quantity" value="300"
            type="integer"/>
<field name="price" value="299.90"
            type="double"/>
<classfield name="article"
            id="01007"/>
</item>
</array>
</class>

<class
            name="PurchaseOrder">
<field name="number" value="301"
            type="integer"/>
<field name="supplier" value="SGH Ltd"
            type="string"/>
<field name="date" value="16/08/2002"
            type="date"/>
<array name="orderLines">
<item
            name="PurchaseOrderLine">
<field name="number" value="1"
            type="integer"/>
<field name="quantity" value="1000"
            type="integer"/>
<field name="price" value="59.90"
            type="double"/>
<classfield name="article"
            id="01005"/>
</item>
<item
            name="PurchaseOrderLine">
<field name="number" value="2"
            type="integer"/>
<field name="quantity" value="500"
            type="integer"/>
<field name="price" value="239.90"
            type="double"/>
<classfield name="article"
            id="01007"/>
</item>
</array>
</class>

</classes>

It contains two customer orders and a purchase order. Now, the stock situation for article 01005 is changed because 50 pieces have been sold by the customer order 400, and 150 pieces by the customer order 401. Also, the purchase order 301 purchases 1000 new pieces. Hence, the stock is now 5800, as illustrated in this figure:

Application Setup

If you want to try this out for yourself, download the support material for this case study from the C#Today web site and follow the instructions in the readme.txt file included.

Case Study Review

This case study showed you a way to build a component to map XML in C# objects. The end component has the following properties: new instances of whatever predefined class can be imported; only elements written according to a predefined XML schema can be imported; no change to the target-class source-code has to be made in order to make it "importable"; and data-types are supported.

To develop the component we used XmlReader objects, XML schemas, and reflection. We reached the final release of the component in four steps: mapping simple objects, complex objects, arrays, and internal references.

Finally, a real-world example was developed. This showed how to integrate a component into applications. The sample application used the mapping component to import articles and orders from an external system to a logistics .NET-based system.

Any Limitations or Further Work

Obviously in this case study I could not completely cover such a vast subject, but by using simple improvements, the user can create a powerful tool to include in his own systems. Various improvements can be made. For example, the component does not map properties, enumerations, and base-type arrays. However, following these case study-guidelines it should be easy to add those features to the component. Moreover, another improvement might be to add a feature to create XML starting from the C# objects. In this way, two C#-based systems can both export and import XML data in the same format.

Acknowledgments

I wish to thank Marcello Valeri and Valentina Muraglia for helping me to plan out and review this case study. Special thanks are also sent to the C#Today team for their work.

Bibliography

S. Robinson at all, "Professional C# 2nd Edition", Wrox Press 2002.

C#Today - © 2002 WroxPress Limited


MSXML DOM and .NET DOM - A practical comparison of working with the DOM in MSXML 4.0 and C#.NET

MSXML DOM and .NET DOM - A practical comparison of working with the DOM in MSXML 4.0 and C#.NET
http://www.csharptoday.com/content.asp?id=1750
Article Rating: 5
Published on February 25, 2002
Abstract
In this article, Steven Livingstone looks at the techniques used when working with the DOM in the MSXML 4.0 parser and the .NET Framework’s System.XML class. This article is going to look at many common server side XML web development tasks that are needed when using MSXML and .NET’s System.Xml. These will include loading an XML document, navigating the XML document using the DOM, adding and removing XML nodes to and from the document, and working with namespaces within the DOM.
Article

In this article we are going to look at the techniques used when working with the DOM in the MSXML 4.0 parser and the .NET Framework's System.XML class. This article is going to look at many common server side XML web development tasks that are needed when using MSXML and .NET's System.Xml. This will include:

  • Loading an XML document
  • Navigating the XML document using the DOM
  • Adding and removing XML nodes to and from the document
  • Working with namespaces within the DOM

MSXML is now on version 4.0, and MSXML (versions 3.0 and 2.0) is used in most Microsoft based enterprise developments these days, of which the DOM is a major part. C# and .NET are new technologies and it is unlikely that you are using them in production enterprise applications today However, it is important that you understand how it uses XML as it is the future of Microsoft developments over the coming years. With MSXML being a young, but widely implemented technology, and .NET XML being a highly anticipated technology, understanding both can be highly advantageous to any developer.

The migration from older versions to MSXML 4.0 itself won't be as straightforward as it may seem. Many versions of MSXML have come out in the last few years, some with browsers and others as previews or separate downloads. Even when downloaded, you could have versions running side-by-side and two different versions of MSXML that used varying pre-standard and standard languages (such as the older XSL working draft implementation and the newer XSLT 1.0 W3C standard). This has caused a fair amount of confusion, but it is fair to say that MSXML 4.0 is by far the best in terms of implementation, functionality, and standards of any of the MSXML parsers.

Over and above the migration issues of MSXML and XML in .NET, understanding one will help you understand the other as they are based on common industry standards. The article will look at the DOM and show practical samples of how it is implemented in both.

Loading and Navigating a DOM Document

MSXML 4.0

Navigating the DOM in MSXML is probably something most developers who have worked with XML are very used to. Therefore we aren't going to discuss this in much detail here, but rather we will point out the differences in .NET. However, it doesn't hurt to just briefly overview the DOM from an MSXML perspective.

To load an XML source into a DOM Document, three steps must be taken:

  • Use CreateObject to instantiate an instance of a DOMDocument. You can choose between a single threaded or free threaded object by using the respective ProgID's: Msxml2.DOMDocument.4.0 and Msxml2.FreeThreadedDOMDocument.4.0.
  • Decide if you want to load the XML source synchronously (immediately available in the next line of code), or asynchronously (need to check the readyState property before accessing the object). Setting the async property (which is True by default), to True or False, does this.
  • Finally, call the load method, passing it the path to the XML source. If this is Server side code in ASP you must use the Server.MapPath method to correctly map the path to the source.

So the following code would load an XML document:

Set objXMLDoc = Server.CreateObject("Msxml2.DOMDocument.4.0")
objXMLDoc.async = false
bol = objXMLDoc.load(Server.MapPath("someurl.xml")

Notice we create an instance of the Msxml2.DOMDocument.4.0 object which basically provides access to methods of the IXMLDOMDocument2 interface, which is an extension of the IXMLDOMDocument interface (used when you create the object as Msxml.DOMDocument.4.0), supporting schema caching and validation.

The bol variable will be set to the Boolean return value of the load() method signifying whether the XML document was successfully loaded or not. If you want your XML document to be validated against some DTD or schema then you set the ValidateOnParse property to true (which is also the default and so you don't explicitly need to set it - it is used here to purposely illustrate its usage) just before you load the document - if you don't want the document to be validated, then you set this property to false. You may choose not to if you don't want to take the performance hit of validation or if you plan to validate the document dynamically later on.

set objXMLDoc = Server.CreateObject("Msxml2.DOMDocument.4.0") ###
objXMLDoc.async = false
objXMLDoc.ValidateOnParse = true
bol = objXMLDoc.load(Server.MapPath("someuri.xml")

You can then catch any error produced through the parseError property of the DOMDocument as shown below.

if objXMLDoc.parseError.errorCode <> 0 then
Response.Write "The following error occurred: " & _
objXMLDoc.parseError.reason
end if

Although you can do more when loading an XML document, the above is enough to get you started. Let's now look at how to load and work with validation in .NET, which you will find quite different (even for this basic operation). However, you will see that the concepts are very much similar, just that .NET uses a more structured view.

C# and .NET

The XmlDocument class in the System.XmlDocument namespace is very similar to the DOMDocument object that is created when an XML document is loaded. This class gives read/write access to the XML document. Performance can be improved by using the XmlReader class if you require read-only access to the document. One important contrast of the load method of the MSXML DOMDocument object which loads documents asynchronously by default to the .NET XmlDocument load method is that the latter actually loads documents synchronously. Lower level classes in the System.NET namespace can be used to establish asynchronous HTTP connections before the load method is called.

So we can now load an XML document with the following code.

XmlDocument doc = new
            XmlDocument();
doc.load("someuri.xml");

Often we may wish to validate the XML source as we did in the MSXML example above, and to do this we should use the XMLValidatingReader class that does many of the same things as the ValidateOnParse property above. However, to use this class is a little more complicated than simply loading the document as the following code fragment shows (loadDoc.aspx in download):

private void
            LoadDocument()
{
XmlDocument doc = new
            XmlDocument();
XmlTextReader docTextRead =
new
            XmlTextReader(Context.Server.MapPath("../resource/sample.xml"));
XmlValidatingReader
            xmlVal = new
            XmlValidatingReader(docTextRead);
xmlVal.ValidationEventHandler
            += new
ValidationEventHandler(XMLDocumentLoadError);

doc.Load(xmlVal);
ValidationError.Text
            = "The document was successfully loaded.";
}

private void
            XMLDocumentLoadError(object sender, ValidationEventArgs
            args)
{
ValidationError.Text = "The following error occurred :
            " + args.Exception.Message;
}

The LoadDocument() method contains the code to actually load the XML document, and creates an instance of the XmlDocument class. Now, because we want to validate the XML document against some Schema, we must use a separate technique to just load the document using the load() method where there is no validation.

We have to use the following steps to perform validation against some Schema.

  • Load the XML source into an XMLTextReader.
  • Pass that XMLTextReader object into an XMLValidatingReader constructor. This sets us up for allowing validation of the XML document - although the actual validation isn't yet performed.
  • Set the XMLValidatingReader.ValidationEventHandler property to the name of the method you created to handle errors. This sets up our event handler for any validation errors (in our case we use the XMLDocumentLoadError() method).
  • Finally, pass that XMLValidatingReader into the XmlDocument.load method.

The actual error information is accessible through the Exception property, which returns an XmlSchemaException class containing a number of properties and methods to investigate the cause of the problem. We use the Message property which simply returns a string describing the error. Note that when the event handler method is used, any code after the load method which actually causes the document to be parsed and validated will not be run (when validation fails of course!).

You will already be seeing how different the MSXML and .NET techniques are - the concepts are the same, but the actual code is different, sometimes significantly. If you think that was different, navigating the DOM is even more so, and we will look at this now.

Navigating the DOM

As the DOM is a W3C standard, there is not much variation on how you can work with them between the various implementations. The API's are fairly similar and what you can do with them doesn't vary much and so we won't discuss the entire DOM. There are some variations however, and the more important of these are discussed below.

Continuing from the last section where we had managed to load an XML document, we want to now get some information from the document. The two most common ways to do this using the DOM is to use XPath queries (we look at these a little later) or step through the DOMDocument. Both of these techniques are common to MSXML and .NET, and many of the properties and methods are the same, however note how the text string and xml of a given node is accessed differs. In MSXML, the text (returns the concatenated textual content of the node and its children) and xml (returns the XML content of the node and its descendants) properties were used, whereas in .NET these are now InnerText, InnerXml and OuterXml. The first InnerXml returns the XML content that is below the context node, and so excludes the XML of the context node itself. The OuterXml property however includes the XML of the content node. Note also that while the xml property of MSXML was read only, the InnerXml and OuterXml properties of .NET are read/write. An example of this is given in the samples below.

Creating Nodes

When creating a new element or attribute in MSXML you can use one of two methods depending on what you wanted to do, whereas in .NET you always have two ways. The reason is because in MSXML 4.0, if you want to create a new element or attribute, you can use the CreateElement(ElementName) or CreateAttribute(AttributeName) methods, respectively. However, these have no support for namespace qualification and you have to use the CreateNode(NodeType, NodeName, NamespaceURI) method which allows you to define the type of node, the name of the node, and finally the namespace.

In .NET it is a little simpler - you can directly use the CreateElement() or CreateAttribute() (as well as the CreateNode()) methods to do this, as well as being able to specify the prefix you want to use. Therefore, to create an element called "BusinessUnit" in the namespace "uri:business-category" with the prefix "ns" you would do the following:

XmlElement busElem = doc.CreateElement("ns","BusinessUnit",
"uri:business-category");

This will result in an element <ns:BusinessUnit /> being created in memory (we will shortly show what can be done with it). Note that the choice of prefix is effectively irrelevant in XML as they are always expanded to their full URI - you may find it better to try to ensure that the same prefix is used for a given URI in some XML document simply to aid readability.

A Example of Creating Nodes in MSXML and .NET

The following method, WalkDocument(), demonstrates the exact same techniques, but the first (WalkDocument.asp) used MSXML 4.0 and the second (WalkDocument.aspx) uses C#.NET.

Function WalkDocument()
Dim objXMLDoc
Set objXMLDoc = Server.CreateObject("Msxml2.DOMDocument.4.0")
objXMLDoc.async = false
objXMLDoc.validateOnParse = true
objXMLDoc.load(Server.MapPath("../resource/sample.xml"))

strDateOfInvoice = objXMLDoc.documentElement.getAttribute("date")
Rem Get the first <customer> element node
strBusCat = objXMLDoc.documentElement.firstChild.firstChild.firstChild.text

set busElem = objXMLDoc.createNode(1,"ns:BusinessUnit", _
"uri:business-category")
busElem.text = "Business Development"
objXMLDoc.documentElement.firstChild.appendChild(busElem)

strInnerDocXML = objXMLDoc.documentElement.firstChild.xml
strOuterDocXML = objXMLDoc.documentElement.xml
End Function

The following is the same example using C#.

private void
            WalkDocument()
{
XmlDocument doc = new
            XmlDocument();
doc.Load(Context.Server.MapPath("../resource/sample.xml"));

DateOfInvoice.Text
            = doc.DocumentElement.GetAttribute("date");
//Get the first
            <customer> element node
BusCat.Text =
            doc.DocumentElement.FirstChild.FirstChild.FirstChild.InnerText;

XmlElement
            busElem = doc.CreateElement("ns",
            "BusinessUnit",
"uri:business-category");

busElem.InnerText="Business
            Development";
doc.DocumentElement.FirstChild.AppendChild(busElem);

innerdocXML.Text
            = doc.DocumentElement.InnerXml;
outerdocXML.Text =
            doc.DocumentElement.OuterXml;
}

Notice that in both samples, the method for accessing attributes is identical (ignoring case sensitivity), and that the FirstChild property can be used in both to navigate down the tree. However, the .NET implementation uses InnerText in contrast to the text property of the MSXML implementation. Following on from this, the way the "BusinessUnit" element is created is very similar, but in the .NET version the CreateElement() method is used to create the BusinessUnit element in the namespace "uri:business-category" with the prefix "ns". In contrast, the MSXML must use the CreateNode() method to create the element (specified by the parameter value "1") with the namespace but has no option for defining the prefix, and so you specify this directly on the element. Finally both designs use the AppendChild() method to attach the newly created element as a child of the first child element of the document element node. Following this things differ slightly - the .NET version allows you to use the InnerXml and OuterXml to get the XML node and its descendants immediately below the document element, and then the XML including the document element node respectively. However MSXML doesn't have these properties, and you must therefore access the xml property relative to the firstchild of the document element and of the document element respectively.

The output for both is identical as shown in the figure below.

Selection of namespace qualified nodes

MSXML 4.0

When working with MSXML 4.0, you must use the setProperty method of the XMLDOM Object to set the SelectionNamespaces property when you have qualified nodes in the source XML document you intend to select. The actual namespaces should contain a prefix that will be used in the actual XPath queries, and the names of the prefixes DO NOT have to be the same as the name of the prefixes within the XML document itself because all prefixes are ALWAYS expanded to the full URI before any evaluation! However, it is essential that when using these prefixes that you know what URI they will be expanded to so you actually select the nodes you are interested in. Multiple entries should be space delimited. The syntax is as follows:

objXMLDOMDoc.setProperty("SelectionNamespaces","xmlns:")

Note that this method cannot cope with no prefix for the default namespace (as in fact defined in the XPath spec and discussed below) or where there is no namespace, so you must use some prefix within the SelectionNamespace property even if there is no prefix in the source XML document.

<%@ language="VBScript" %>
<%
Dim objXMLDoc
Dim strHTMLCustDisplay
strHTMLCustDisplay=""
Dim strHTMLCountryDisplay
strHTMLCountryDisplay=""
Dim strHTMLBusCatDisplay
strHTMLBusCatDisplay=""
Dim intNodeCounter

Set objXMLDoc = Server.CreateObject("Msxml2.DOMDocument.4.0")
bol = objXMLDoc.load(Server.MapPath("../resource/sample.xml"))
objXMLDoc.setProperty "SelectionNamespaces", _
"xmlns:bc='uri:business-customers' xmlns:ns='uri:business-category'" _
& " xmlns:address='uri:business-addresses'"

'Select all customers nodes from the default namespace
strHTMLCustDisplay = "<h3>Names of customers</h3>"
Set objNodes = objXMLDoc.selectNodes("//bc:customer")
For intNodeCounter = 0 To objNodes.Length - 1
strHTMLCustDisplay = strHTMLCustDisplay & _
objNodes(intNodeCounter).getAttribute("name") & "<BR/>"
Next

'select all customers in a given country address which is Scotland
strHTMLCountryDisplay = "<h3>Names of businesses located in Scotland</h3>"

Set objNodes = objXMLDoc._
selectNodes("//bc:customer[@address:Country='Scotland']")

For intNodeCounter = 0 To objNodes.Length - 1
strHTMLCountryDisplay = strHTMLCountryDisplay _
& objNodes(intNodeCounter).getAttribute("name") & "<BR/>"
Next

'select all the business categories of the customers which are not from
'the above query
strHTMLBusCatDisplay = "<h3>Business Categories of customers" _
& " not located in Scotland</h3>"

Set objNodes = objXMLDoc._
selectNodes("//bc:customer[@address:Country!='Scotland']/ns:BusCat")

For intNodeCounter = 0 To objNodes.Length - 1
strHTMLBusCatDisplay = strHTMLBusCatDisplay _
& objNodes(intNodeCounter).text & "<BR/>"
Next

%>
<html>
<head>
</head>
<body>
<%=strHTMLCustDisplay%>
<%=strHTMLCountryDisplay%>
<%=strHTMLBusCatDisplay%>
</body>
</html>

The output is shown below.

Notice that the SelectionNamespaces property is set to associate the prefixes with the namespaces defined in the source XML document, "bc" being used for the default namespace of the XML document and each prefix and namespace definition separated by a single whitespace.

objXMLDoc.setProperty "SelectionNamespaces", _
"xmlns:bc='uri:business-customers' xmlns:ns='uri:business-category'" _
& " xmlns:address='uri:business-addresses'"

We can then select all the customer element nodes in the document with the XPath query "//bc:customer" qualifying the customer element node in the default namespace "'uri:business-customers" which is associated with the "bc" prefix in the steps above.

Similarly, this query can be filtered to return any Country attributes (in the "uri:business-addresses" namespace associated "address") with the value "Scotland" using the XPath string "//bc:customer[@address:Country='Scotland']".

Finally, an XPath query combining nodes from all namespaces that returns the BusCat elements in the "uri:business-category" namespace could be "//bc:customer[@address:Country!='Scotland']/ns:BusCat".

Let's now look at how to get the exact same results when working with C# and .NET.

C# and .NET

C# uses a very different approach to working with namespaces from that in MSXML 4.0. It has a separate class, XmlNamespaceManager, which stores the namespace prefixes and URI's as strings. It is a lot more powerful that the MXML 4.0 version, allowing you to set (or retrieve) a default namespace, look up prefixes or namespaces, and dynamically add and remove namespace associations as well as other methods. In fact, if you have used the XMLSchemaCache object within MSXML 4.0 you may see a lot more similarities with the XmlNamespaceManager class.

So, when you have no requirements for namespace qualification and want to do a selection, you can simply use the Select method of the XpathNavigator object to return an XPathIterator object.

Before we continue, let's clear up something that confuses many developers when working with namespaces in .NET (and to a lesser extent MSXML 4.0). The .NET documentation says that you can use String.Empty to define the default namespace for your queries, and so the full form would be something like:

xmlmgr.AddNamespace(String.Empty,"uri:business-customers");

However, there is a little more to this. This in fact doesn't do very much other that allow you to later programmatically retrieve the default namespace that was previously programmatically assigned by your code. What it does NOT do is allow you to create XPath queries that attempt to select nodes in the "uri:business-customers" namespace without using prefix in the Xpath query to imply the above "default namespace". This is because the XPath specification (http://www.w3.org/TR/xpath#node-tests) requires that XPath queries that are intended to select nodes within some namespace MUST use a prefix associated with that namespace. We will see what this means in the code sample below.

So, if you have to do qualified XPath selections, you can create an XpathExpression object by compiling an XpathNavigator expression. You then add the relevant namespaces to the XmlNamespaceManager object and use the SetContext method of the XpathExpression object to associate these namespaces with the expressions and you can then go ahead and make qualified XPath queries.

The following code will select the same information as the MSXML 4.0 example above and produce the same output. We will only discuss the main method called DisplayNamespaceInfo() and you can see the rest in the sample code.

private void
            DisplayNamespaceInfo()
{
//Load the XML data
            file.
XmlDocument doc = new
            XmlDocument();
doc.Load(Context.Server.MapPath("../resource/sample.xml"));

XPathNavigator
            objNavDoc = doc.CreateNavigator();

XmlNamespaceManager xmlmgr
            = new XmlNamespaceManager(new
            NameTable());
xmlmgr.AddNamespace("bc","uri:business-customers");
xmlmgr.AddNamespace("ns","uri:business-category");
xmlmgr.AddNamespace("address","uri:business-addresses");

string
            strHTMLCustDisplay = "<h3>Names of
            customers</h3>";
XPathExpression upbaseExpr =
            objNavDoc.Compile("//bc:customer");
upbaseExpr.SetContext(xmlmgr);
XPathNodeIterator upBaseIterator =
            objNavDoc.Select(upbaseExpr);

while
            (upBaseIterator.MoveNext())
{
strHTMLCustDisplay =
            strHTMLCustDisplay +
upBaseIterator.Current.GetAttribute("name","") +
            "<BR/>";
}
CustDisplay.Text =
            strHTMLCustDisplay;

string strHTMLCountryDisplay =
"<h3>Names of businesses located in
            Scotland</h3>";

upbaseExpr =
            objNavDoc.Compile("//bc:customer[@address:Country='Scotland']");
upbaseExpr.SetContext(xmlmgr);
upBaseIterator = objNavDoc.Select(upbaseExpr);

while
            (upBaseIterator.MoveNext())
{
strHTMLCountryDisplay +=
            upBaseIterator.Current.GetAttribute("name","")
+
            "<BR/>";
}
CountryDisplay.Text =
            strHTMLCountryDisplay;

string strHTMLBusCatDisplay =
"<h3>Business Categories of customers not located in
            Scotland</h3>";
upbaseExpr =
            objNavDoc.Compile(
"//bc:customer[@address:Country!='Scotland']/ns:BusCat");

upbaseExpr.SetContext(xmlmgr);
upBaseIterator = objNavDoc.Select(upbaseExpr);

while
            (upBaseIterator.MoveNext())
{
strHTMLBusCatDisplay +=
            upBaseIterator.Current.Value +
            "<BR/>";
}
BusCatDisplay.Text =
            strHTMLBusCatDisplay;
}

Initially the XML document is loaded as a new XmlDocument as discussed earlier. Next, as we intend to query the document using XPath, an XpathNavigator object is created using the CreateNavigator() method of the XmlDocument class.

XPathNavigator objNavDoc = doc.CreateNavigator();

Notice the key use of the XmlNamespaceManager class and the AddNamespace() methods to associate some prefix with the relevant namespace.

XmlNamespaceManager xmlmgr = new XmlNamespaceManager(new NameTable());
xmlmgr.AddNamespace("bc","uri:business-customers");

The NameTable() parameter of the XmlNamespaceManager constructor is a XmlNameTable object which is used to look up prefixes and namespaces.

Later, the XPath expression ("//bc:customer") is compiled and this returns an XPathExpression object whose SetContext() method can then be used to associate the namespaces defined in the XmlNamespaceManager with the compiled expression.

XPathExpression upbaseExpr = objNavDoc.Compile("//bc:customer");
upbaseExpr.SetContext(xmlmgr);

Finally, the actual expression can be executed using the Select() method of the XPathNavigator object with the XPathExpression object as a parameter:

XPathNodeIterator upBaseIterator = objNavDoc.Select(upbaseExpr);

The result is an XPathNodeIterator object containing the nodes selected by the expression, which is then iterated through using the MoveNext() method which will return true when a node exists in the selected node set.

upBaseIterator = objNavDoc.Select(upbaseExpr);

while (upBaseIterator.MoveNext())
{
strHTMLCountryDisplay += upBaseIterator.Current.GetAttribute("name","")
+ "<BR/>";
}
CountryDisplay.Text = strHTMLCountryDisplay;

We get the actual value by accessing the Current property which points to the current node in the node set, and so in the first case (as was done in the MSXML example) the value of the name attribute can be obtained by using the GetAttribute() method, which differs from the MSXML version in that it also takes a second parameter specifying the namespace of the node. The actual value is obtained via the Text property and this is repeated for each XPath query as was discussed in the MSXML example above.

This completes a look at how the Document Object Model (DOM) is used in MSXML and C#.NET. You will have seen a lot of similarities and also quite a few differences - there is a lot to the DOM and the best way you can learn more about it is to try it out.

Look out for upcoming articles on comparisons of MSXML and .NET XML technology implementations, such as Schema validation, SAX and XSLT on http://www.asptoday.com/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq.

Article Information
Author | Steven Livingstone
Technical Editor | Adam Ryland
Project Manager | Helen Cuthill
Reviewers | Sean Schade, Shefali Kulkarni

If you have any questions or comments about this article, please contact the technical editor.

C#Today - © 2002 WroxPress Limited


XML Querying Using .NET and SQL Server 2000

XML Querying Using .NET and SQL Server 2000
http://www.csharptoday.com/content.asp?id=1822
Article Rating: 3.5
Published on February 15, 2002
Abstract
SQL Server 2000 provides a mechanism that allows for the creation of XML views of relational data via the use of mapping schemas. SQL Server 2000 also provides SQLXML managed classes which can be used to perform XPath queries on the aforementioned XML views from .NET applications. Combining the ability to do XPath queries from .NET with SQL Server's ability to define XML Views of relational data means that one can retrieve XML from relational tables with relative ease compared to mechanisms that utilize conventional database access APIs. In this article, Dare Obasanjo and Sanjay Bhatia explore XML, XSLT and SQL Server 2000.
Article

XML has recently become the standard for information interchange amongst businesses, and its usage has spread into several aspects of computing from database systems to networking and programming languages. Due to the growing rise of XML usage in data interchange scenarios, Microsoft has released a package called XML for SQL Server (or SQLXML) that, amongst other features, allows one to create XML views of the relational data stored in the database, and query the XML view using the W3C XPath query language.

One of the most obvious ways to develop a web application using Microsoft technologies is to simply write ASP pages composed of code that access the database and print the results. Though this strategy may be sufficient for most projects, it will not scale when the site gets more complicated. One reason for this inadequacy is coupling. Each piece of code is coupled tightly to both the database and the presentation. For instance, if a developer wants to alter the look and feel of such a website, he or she needs to understand and possibly even modify the database. Similarly, changes to the database may inadvertently adversely affect the code that dynamically generates the website. There is an age-old solution to this problem known as Model-View-Controller. In the web paradigm, the database acts as the model. The presentation code is equivalent to the view, and a middle layer of business logic acts as a controller. An MVC architecture is robust, scalable, and easy for large teams to work with. However, it is also typically a very heavyweight solution since data objects must be created for every table or type of query in the database. Using ASP.NET and SQLXML technologies presents a simpler way to get many of the benefits of a traditional MVC architecture without some of the overhead. Additionally, XPath can be used to interact with the database in a manner that works very well in web applications, sometimes providing more options than typical SQL based applications.

With XML being used as an interface between the database and the presentation layer, both layers are loosely coupled, and thus a change in either one does not necessarily break the other as long as the mapping schema is updated.

In this article a user interface for navigating the Suppliers and Product tables in the Northwind database which ships with SQL Server will be built. The user interface will enable users to obtain a list of suppliers in any given country. Details on suppliers can be obtained by clicking on a hyperlinked supplier name, while details on the products they supply can be obtained by clicking on hyperlinked product names from the supplier details page.

This application will be built primarily using XML technologies including XSLT, XPath and XML Schema without any SQL being involved.

SYSTEM REQUIREMENTS

This application requires the user to have a copy of Visual Studio.NET, SQL Server 2000 (a trial version of which can be obtained at http://www.microsoft.com/sql/evaluation/trial/2000/default.asp?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq) and SQLXML 2.0 (which can be downloaded at http://msdn.microsoft.com/downloads/default.asp?URL=/downloads/sample.asp?url=/msdn-files/027/001/602/msdncompositedoc.xml&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq)

The article assumes a familiarity with C#, as well as with using Visual Studio.NET to create ASP.NET applications. Elementary knowledge of XPath, XSLT and W3C XML schemas would be beneficial but not absolutely necessary to fully understand the contents of this article.

SQLXML XML MAPPING SCHEMAS

One can create XML views of relational data by using the W3C XML Schema Definition (XSD) language to create mapping schemas. A mapping schema is similar to a regular XML schema that describes the constraints and types in an XML document, except for a single difference; mapping schemas contain annotations that describe how to map relational data to XML. To learn more about W3C XML schemas, read the XML Schema Primer available at http://www.w3.org/TR/xmlschema-0/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq

The SQLXML annotations are elements or attributes from the urn:schemas-microsoft-com:mapping-schema namespace. The sql prefix is conventionally used in mapping schemas for elements and attributes from the SQLXML mapping schema namespace.

Below are descriptions the some of the SQLXML annotations that can be used to describe a mapping from tables and columns in the database to XML data. It is assumed in the descriptions below that the prefix xsd is mapped to the W3C XML Schema namespace, http://www.w3.org/2001/XMLSchema?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq .

The sql:relation is used to map relational tables to XML nodes. The value of the sql:relation annotation is the name of table or view that is being mapped to an XML element. Table names that have spaces in them are specified by being placing in square brackets. The target of a sql:relation is typically an xsd:element declaration in the XML schema. The annotation applies to all elements and attributes declared in the complex type definition for the target element unless another sql:relation annotations exists for them. The following are snippets of mapping schemas showing usage of sql:relation:

<xsd:element name="Supplier" sql:relation="Suppliers" type="SupplierType" />
<xsd:element name="order-details" sql:relation="[Order Details]" type="ODtype" />

NOTE: An element with a complex type definition maps to a table or view with the same name in the database unless a mapping is explicitly specified using the sql:relation annotation.

The sql:field is used to map relational columns to XML nodes. The value of the sql:field annotation is the name of the column being mapped to an XML element or attribute. The target of a sql:relation is typically an xsd:element or xsd:attribute declaration in the XML schema.

The following are snippets of mapping schemas showing usage of sql:field:

<xsd:attribute name="supplier-id" type="xsd:integer" sql:field="SupplierID" />
<xsd:element name="Product Name" sql:field="ProductName" type="xsd:string" />

NOTE: An attribute or element with a simple type definition that is a child of an element mapped to a relation maps to a column with the same name unless a mapping is explicitly specified using the sql:field annotation.

The sql:relationship is used to map foreign key relationships between tables to hierarchical child-parent relationships between XML elements. The target of a sql:relationship annotation is typically an xsd:element declaration that has a sql:relation annotation and is part of the complex type definition of an element that also has a sql:relation annotation.

To specify the details of the relationships between the elements, a sql:relationship element must be defined . sql:relationship elements are typically defined in the xsd:appinfo sections of the xsd:annotation elements in the mapping schema. A sql:relationship element has the following attributes:

name: The name of the relationship

parent: The parent table in the relationship. If this attribute is not present, SQLXML can deduce it by examining the hierarchical relationships in the mapping schema.

parent-key: This is used to specify the key(s) in the parent table used in the join operation from which the relationship is created. If multiple keys exist in the parent table then they are all specified, separated by spaces with the entire list enclosed in square brackets.

child: The child table in the relationship.

child-key: This is used to specify the key(s) in the child table used in the join operation from which the relationship is created. If multiple keys exist in the child table then they are all specified, separated by spaces with the entire list enclosed in square brackets.

inverse: This is used to specify whether the primary-key/foreign-key relationship between tables is the inverse of the parent-child relationship between elements or not. The inverse attribute is primarily useful for deletion operations via updategrams so that deletions occur in the proper order.

The following is a snippet of a mapping schema showing the usage of sql:relationship:

<xsd:annotation>
<xsd:appinfo>
< sql:relationship name="SuppliersProducts"

parent="Suppliers"
parent-key="SupplierID"
child="Products"
child-key="SupplierID" />
</xsd:appinfo>
</xsd:annotation>
<xsd:element name="Supplier" sql:relation="Suppliers">
<xsd:complexType>
<xsd:sequence>
<xsd:element name="SupplierID" type="xsd:int" />
<xsd:element name="CompanyName" type="xsd:string" />
<xsd:element name="Product" sql:relation="Products"
sql:relationship="SuppliersProducts" maxOccurs="unbounded">

The sql:key-fields is used to specify the primary key field(s) in a parent relation/element in a hierarchy when the sql:relationship element does not contain enough information about the keys in the relation. The target of a sql:key-fields annotation is typically an xsd:element declaration.

If the sql:key-fields annotation is not used when a relationship does not specify the key fields in the parent relation then the XML returned may possess the incorrect hierarchy of elements.

The following is a snippet of a mapping schema showing the usage of sql:key-fields:

<xsd:annotation>
<xsd:appinfo>
<sql:relationship name="ProductsSuppliers"
parent="Products"
parent-key=SupplierID"
child="Suppliers"
child-key=" SupplierID " />
</xsd:appinfo>
</xsd:annotation>
<xsd:element name=" Product" sql:relation="Products" sql:key-fields="ProductID" >

The sql:hide is used to specify that certain elements or attributes that appear in a mapping schema do not appear in the resulting XML. This is of use when an XPath query needs to be performed against a certain field, but for some reason the recipient of the generated XML is not expected to be privy to the information that was used to perform the query. The possible values of sql:hide are 0, 1, true and false

An example of a usage of sql:hide is a query that retrieves XML data on employees that live in the city of Atlanta, Georgia in the United States by querying for employees whose home phone number begins with "404". In this case it may not be prudent the hand over the phone numbers of all the employees to the recipient of the XML.

The following is a snippet of mapping schema showing usage of sql:hide:

<xsd:element name="Employee" sql:relation="Employees">
<xsd:complexType>
<xsd:attribute name="FirstName" type="xsd:string" />
<xsd:attribute name="LastName" type="xsd:string" />
<xsd:attribute name="HomePhone" type="xsd:string"
sql:hide="true" />
</xsd:complexType>
</xsd:element>

sql:is-constant is used to specify an element that is not mapped to any table, view or column in the database. The target element appears in the generated XML and cannot have any attributes. This is useful in situations where one wants to create a parent element for a series of elements generated from the database. The possible values of sql:is-constant are 0, 1, true and false.

The following is a snippet of mapping schema showing usage of sql:is-constant:

<xsd:annotation>
<xsd:appinfo>
<sql:relationship name="SuppliersProducts"
parent="Suppliers"
parent-key="SupplierID"
child="Products"
child-key="SupplierID" />
</xsd:appinfo>
</xsd:annotation>

<xsd:element name="AllSuppliers" sql:is-constant="true" >
<xsd:complexType>
<xsd:sequence>
<xsd:element name="Supplier" sql:relation="Suppliers" >
<xsd:complexType>
<xsd:sequence>
<xsd:element name="SupplierID" type="xsd:int" />
<xsd:element name="CompanyName" type="xsd:string" />
<xsd:element name="Product" sql:relation="Products" sql:relationship="SuppliersProducts"
maxOccurs="unbounded">

The sql:mapped can be used to specify that an element or attribute declared in the mapping schema does not map to a table, view or column in the database. This is useful in situations where the mapping schema is also used to validate XML from other sources. XML nodes with the sql:mapped annotation do not occur in the generated XML. The target of sql:mapped is typically an xsd:element or xsd:attribute declaration in the XML schema. The possible values of sql:mapped are 0, 1, true and false.

The following is a snippet of mapping schema showing usage of sql:mapped:

<xsd:element name="Employee" sql:relation="Employees" >
<xsd:complexType>
<xsd:attribute name="EmployeeID" type="xsd:string"/>
<xsd:attribute name="FirstName" type="xsd:string" />
<xsd:attribute name="LastName" type="xsd:string" />
<xsd:attribute name="Salary" type="xsd:decimal"
sql:mapped="false" />
</xsd:complexType>
</xsd:element>

The above is not an exhaustive list of SQLXML annotations, but is instead an overview of the more commonly used SQLXML annotations. For a complete description of all of the SQLXML annotation please consult the SQLXML documentation.

CREATING A SQLXML MAPPING SCHEMA

The first step in creating a mapping schema is to write a regular W3C XML Schema file that describes the relational data as XML. To avoid writing the schema from scratch, we used Visual Studio .NET's Data Designer to create a schema by dragging and dropping the Suppliers and Products tables into an XML schema.

Selecting View XML Source from the menu that appears on a right-click displays the following schema:

<?xml version="1.0" encoding="utf-8" ?>
<xsd:schema targetNamespace="http://tempuri.org/XMLSchema.xsd?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq"
elementFormDefault="qualified" xmlns="http://tempuri.org/XMLSchema.xsd"
xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns:msdata="urn:schemas-microsoft-com:xml-msdata">
<xsd:element name="Suppliers">
<xsd:complexType>
<xsd:choice maxOccurs="unbounded">
<xsd:element name="Suppliers">
<xsd:complexType>
<xsd:sequence>
<xsd:element name="SupplierID" msdata:ReadOnly="true" msdata:AutoIncrement="true"
type="xsd:int" />
<xsd:element name="CompanyName" type="xsd:string" />
<xsd:element name="ContactName" type="xsd:string" minOccurs="0" />
<xsd:element name="ContactTitle" type="xsd:string" minOccurs="0" />
<xsd:element name="Address" type="xsd:string" minOccurs="0" />
<xsd:element name="City" type="xsd:string" minOccurs="0" />
<xsd:element name="Region" type="xsd:string" minOccurs="0" />
<xsd:element name="PostalCode" type="xsd:string" minOccurs="0" />
<xsd:element name="Country" type="xsd:string" minOccurs="0" />
<xsd:element name="Phone" type="xsd:string" minOccurs="0" />
<xsd:element name="Fax" type="xsd:string" minOccurs="0" />
<xsd:element name="HomePage" type="xsd:string" minOccurs="0" />
</xsd:sequence>
</xsd:complexType>
</xsd:element>
</xsd:choice>
</xsd:complexType>
<xsd:unique name="DocumentKey1" msdata:PrimaryKey="true">
<xsd:selector xpath=".//Suppliers" />
<xsd:field xpath="SupplierID" />
</xsd:unique>
</xsd:element>
<xsd:element name="Products">
<xsd:complexType>
<xsd:choice maxOccurs="unbounded">
<xsd:element name="Products">
<xsd:complexType>
<xsd:sequence>
<xsd:element name="ProductID" msdata:ReadOnly="true" data:AutoIncrement="true"
type="xsd:int" />
<xsd:element name="ProductName" type="xsd:string" />
<xsd:element name="SupplierID" type="xsd:int" minOccurs="0" />
<xsd:element name="CategoryID" type="xsd:int" minOccurs="0" />
<xsd:element name="QuantityPerUnit" type="xsd:string" minOccurs="0" />
<xsd:element name="UnitPrice" type="xsd:decimal" minOccurs="0" />
<xsd:element name="UnitsInStock" type="xsd:short" minOccurs="0" />
<xsd:element name="UnitsOnOrder" type="xsd:short" minOccurs="0" />
<xsd:element name="ReorderLevel" type="xsd:short" minOccurs="0" />
<xsd:element name="Discontinued" type="xsd:boolean" />
</xsd:sequence>
</xsd:complexType>
</xsd:element>
</xsd:choice>
</xsd:complexType>
<xsd:unique name="DocumentKey11" msdata:PrimaryKey="true">
<xsd:selector xpath=".//Products" />
<xsd:field xpath="ProductID" />
</xsd:unique>
</xsd:element>
</xsd:schema>

The above schema is a good start, but has to be trimmed in a number of ways to become a suitable launching point for a mapping schema. First of all, Product elements need to be made children of the Supplier element so as to properly describe the relationship between both items in XML. Secondly, the xsd:unique constraints are deleted because they are unnecessary in a mapping schema since a mapping schema primarily describes transformations and not constraints. We then mark any fields that may have NULL values in the database as nillable so that they appear in the XML generated from querying the database as elements with their xsi:nil field set to true instead of not appearing at all.

NOTE: Even though XML schema are used to describe the form of the XML that is returned from the database, this does NOT mean that the XML returned will validate against the schema.

For instance, if one of the nillable elements did not have it's nillable attribute set to true, and had a minOccurs value set to 1 or higher, then one would assume incorrectly that at least one instance of that element would occur in the XML generated from querying the database based on the mapping schema regardless of whether the field was NULL or not. What actually would happen is that if the equivalent relational field is NULL and the nilllable attribute is not set to true, then that element does not occur in the returned XML.

Removing the attributes related to Microsoft data types such as msdata:ReadOnly and msdata:AutoIncrement is also a good idea since they are not necessary in the mapping schema.

Finally, the targetNameSpace and elementFormDefault attributes are removed from the root xsd:schema element because SQLXML sometimes behaves unpredictably when a targetNameSpace is specified.

The altered schema is shown below:

<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<xsd:element name="Supplier" type="SupplierType" />
<xsd:complexType name="SupplierType" >
<xsd:sequence>
<xsd:element name="SupplierID" type="xsd:int" />
<xsd:element name="CompanyName" type="xsd:string" />
<xsd:element name="ContactName" type="xsd:string" minOccurs="0" />
<xsd:element name="ContactTitle" type="xsd:string" minOccurs="0" />
<xsd:element name="Address" type="xsd:string" minOccurs="0" />
<xsd:element name="City" type="xsd:string" minOccurs="0" />
<xsd:element name="Region" type="xsd:string" minOccurs="0" nillable="true" />
<xsd:element name="PostalCode" type="xsd:string" minOccurs="0" />
<xsd:element name="Country" type="xsd:string" minOccurs="0" />
<xsd:element name="Phone" type="xsd:string" minOccurs="0" />
<xsd:element name="Fax" type="xsd:string" minOccurs="0" nillable="true" />
<xsd:element name="HomePage" type="xsd:string" minOccurs="0" nillable="true" />
<xsd:element name="Product" maxOccurs="unbounded">
<xsd:complexType>
<xsd:sequence>
<xsd:element name="ProductID" type="xsd:int" />
<xsd:element name="ProductName" type="xsd:string" />
<xsd:element name="SupplierID" type="xsd:int" minOccurs="0" />
<xsd:element name="CategoryID" type="xsd:int" minOccurs="0" />
<xsd:element name="QuantityPerUnit" type="xsd:string" minOccurs="0" />
<xsd:element name="UnitPrice" type="xsd:decimal" minOccurs="0" />
<xsd:element name="UnitsInStock" type="xsd:short" minOccurs="0" />
<xsd:element name="UnitsOnOrder" type="xsd:short" minOccurs="0" />
<xsd:element name="ReorderLevel" type="xsd:short" minOccurs="0" />
<xsd:element name="Discontinued" type="xsd:boolean" />
</xsd:sequence>
</xsd:complexType>
</xsd:element>
</xsd:sequence>
</xsd:complexType>
</xsd:schema>

Once the original schema is made more suitable as a mapping schema, the final step involves adding annotations to the XML schema from the Microsoft mapping schema namespace ; urn:schemas-microsoft-com:mapping-schema. In the mapping schema, the prefix "sql" will be used for the mapping schema namespace. Each element that maps to a relational table, and has a name that is different from that of the relational table, is marked with the sql:relation attribute with its value being set to the name of the relational table. Similarly, each element that maps to a field/column in a relational table and has a different name from the field/column is marked with the sql:field attribute with its value being set to the name of the equivalent field/column in the relational table. In the latter case, the <ProductName> element will be changed to a <Name> element and a sql:field attribute added.

Also an annotation that describes the parent-child relationship between the Supplier and Product elements as a SQL join of the Suppliers and Products tables with the key fields specified is also placed in the mapping schema. The Product element in the schema is marked with the sql:relationship attribute to indicate its relationship to the Supplier element. The completed mapping schema is shown below:

<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns:sql="urn:schemas-microsoft-com:mapping-schema" >
<xsd:annotation>
<xsd:appinfo>
<sql:relationship name="SuppliersProducts"
parent="Suppliers"
parent-key="SupplierID"
child="Products"
child-key="SupplierID" />
</xsd:appinfo>
</xsd:annotation>

<xsd:element name="Supplier" sql:relation="Suppliers" type="SupplierType" />
<xsd:complexType name="SupplierType" >
<xsd:sequence>
<xsd:element name="SupplierID" type="xsd:int" />
<xsd:element name="CompanyName" type="xsd:string" />
<xsd:element name="ContactName" type="xsd:string" minOccurs="0" />
<xsd:element name="ContactTitle" type="xsd:string" minOccurs="0" />
<xsd:element name="Address" type="xsd:string" minOccurs="0" />
<xsd:element name="City" type="xsd:string" minOccurs="0" />
<xsd:element name="Region" type="xsd:string" minOccurs="0" nillable="true" />
<xsd:element name="PostalCode" type="xsd:string" minOccurs="0" />
<xsd:element name="Country" type="xsd:string" minOccurs="0" />
<xsd:element name="Phone" type="xsd:string" minOccurs="0" />
<xsd:element name="Fax" type="xsd:string" minOccurs="0" nillable="true" />
<xsd:element name="HomePage" type="xsd:string" minOccurs="0" nillable="true" />
<xsd:element name="Product" sql:relation="Products" sql:relationship="SuppliersProducts"
maxOccurs="unbounded">
<xsd:complexType>
<xsd:sequence>
<xsd:element name="ProductID" type="xsd:int" />
<xsd:element name="Name" sql:field="ProductName" type="xsd:string" />
<xsd:element name="SupplierID" type="xsd:int" minOccurs="0" />
<xsd:element name="CategoryID" type="xsd:int" minOccurs="0" />
<xsd:element name="QuantityPerUnit" type="xsd:string" minOccurs="0" />
<xsd:element name="UnitPrice" type="xsd:decimal" minOccurs="0" />
<xsd:element name="UnitsInStock" type="xsd:short" minOccurs="0" />
<xsd:element name="UnitsOnOrder" type="xsd:short" minOccurs="0" />
<xsd:element name="ReorderLevel" type="xsd:short" minOccurs="0" />
<xsd:element name="Discontinued" type="xsd:boolean" />
</xsd:sequence>
</xsd:complexType>
</xsd:element>
</xsd:sequence>
</xsd:complexType>
</xsd:schema>

SQLXML XML TEMPLATES

Once a mapping schema has been created, the relational data in the SQL Server Northwind 2000 database can be queried using XPath. XPath is a language for addressing parts of an XML document that utilizes a syntax that resembles hierarchical paths used to address parts of a filesystem or URL. XPath also supports the use of functions for interacting with the selected data from the document. It provides functions for accessing information about document nodes, as well as for the manipulation of strings, numbers, and booleans. XPath uses a compact, non-XML syntax in order to enable its usage of XPath within URIs and XML attribute values (which is useful for other W3C recommendations like XML schema and XSL which use XPath within attributes).

To learn about XPath, read the online tutorial by ZVON at http://www.zvon.org/xxl/XPathTutorial/General/examples.html?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq

Below are the XPath queries that will be used to retrieve XML from the Northwind database:

  • /Supplier: Queries the database for all the suppliers , the suppliers are returned as several <Supplier> elements.
  • /Supplier[SupplierID="foo"]: Queries the database for a supplier with a particular supplier ID, the supplier is returned as a single <Supplier> element.
  • /Supplier/Product[ProductID="bar"]: Queries the database for a product with a particular ProductID, the product is returned as a single <Product> element.

The XML returned by first query (/Supplier) will not be well-formed because it will return several Supplier elements without surrounding them in a root element. To better present the information from the database, an XML template can be used. An XML template is an XML document containing a combination of XPath queries which are <xpath-query> elements from the urn:schemas-microsoft-com:xml-sql namespace and other XML data. When the template is executed, the results of each XPath query replaces each of the <xpath-query> elements in the document.

XML templates are extremely useful in creating complex documents by chaining together multiple XPath queries as well for augmenting data returned by querying the database via queries against a mapping schema.

The following XML template uses the mapping schema created in the previous section to obtain a number of <Supplier> elements from the database and surrounds them in a root element named ROOT:

<ROOT>
<sql:xpath-query mapping-schema="MappingSchema.xml"
xmlns:sql="urn:schemas-microsoft-com:xml-sql">
/Supplier
</sql:xpath-query>
</ROOT>

The following is an excerpt of the XML generated by the XML template:

<ROOT>
<Supplier xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<SupplierID>1</SupplierID>
<CompanyName>Exotic Liquids</CompanyName>
<ContactName>Charlotte Cooper</ContactName>
<ContactTitle>Purchasing Manager</ContactTitle>
<Address>49 Gilbert St.</Address>
<City>London</City>
<Region xsi:nil="true" />
<PostalCode>EC1 4SD</PostalCode>
<Country>UK</Country>
<Phone>(171) 555-2222</Phone>
<Fax xsi:nil="true" />
<HomePage xsi:nil="true" />
<Product>
<ProductID>1</ProductID>
<ProductName>Chai</ProductName>
<QuantityPerUnit>10 boxes x 20 bags</QuantityPerUnit>
<UnitPrice>18</UnitPrice>
<UnitsInStock>39</UnitsInStock>
<UnitsOnOrder>0</UnitsOnOrder>
<ReorderLevel>10</ReorderLevel>
<Discontinued>0</Discontinued>
</Product>
.
</Supplier>
.
</ROOT>

SQLXML MANAGED CLASSES

SQLXML 2.0 ships with three classes for accessing SQLXML functionality from .NET applications, namely the SqlXmlCommand, SqlXmlParameter, and SqlXmlAdapter classes.

SqlXmlCommand contains methods for executing queries or updates against the database using a specified mapping schema.

Constructors:

public SqlXmlCommand (string cnString) Creates a SqlXmlCommand object using the provided ADO or OLEDB connection string.

Methods:

void ExecuteNonQuery() | Used for executing queries that return no results such as updates using Diffgrams or Updategrams.
Stream ExecuteStream() | Executes a query and returns the results in a Stream object.
public void ExecuteToStream (Stream outputStream) | Executes a query and writes the results to the provided Stream.
XmlReader ExecuteXmlReader() | Executes a query and returns the results in an XmlReader object..
SqlXmlParameter CreateParameter() | Creates a SqlXmlParameter object that can be used to pass parameters to a SqlXmlCommand object.
void ClearParameters() | Clears the parameters for the SqlXmlCommand object.

Properties:

bool ClientSideXml | It is possible for SQLXML queries to return rowset data and have the client handle the transformation of this data to XML towards the client. This property when set to true enables this functionality.
string SchemaPath | The name and path to the mapping schema. The path can be absolute or relatve to the current directory.
string XslPath | The name and path to an XSLT stylesheet. The path can be absolute or relative to the current directory
string BasePath | The base path that is used to resolve relative paths in the SchemaPath and XslPath properties as well as the mapping-schema attribute of the XML template used by the class.
string OutputEncoding | The requested encoding for data that will be written to a stream by this class. The default value is UTF-8.
string Namespaces | Used for creating namespace declarations that will be used by the XPath queries executed by this class.
string RootTag | Provides the name for a single root element that will be the parent element of the XML returned by the query. This is useful in situations where the query returns XML that is not well-formed.
string CommandText | This is a string containing the command to execute.
System.IO.Stream CommandStream | This is the stream containing the command to execute. When using CommandStream, only "Template", "UpdateGram" and "DiffGram" CommandType values are supported.
SqlXmlCommandType CommandType | This is indicates the type of command being executed, valid values include "Xpath", "Sql", "TemplateFile", "Template", "UpdateGram" and "DiffGram".

The SqlXmlParameter is used for passing parameters to commands in SqlXmlCommand objects. A SqlXmlParameter cannot be directly instantiated and instead must be created by using the CreateParameter( ) method in the SqlXmlCommand class.

Properties

string Name | The name of the parameter.
object Value | The value of the parameter.

The SqlXmlAdapter provides a bridge between SQLXML operations and the DataSet in ADO.NET. Queries can be executed and their results stored in a DataSet. DataSets obtained in this manner can also be modified and used to update the database.

Constructors

SqlXmlAdapter(SqlXmlCommand cmd) | Initializes the SqlXmlAdapter from a SqlXmlCommand object.
SqlXmlAdapter( string commandText, SqlXmlCommandType cmdType, string connectionString ) | Initializes a SqlXmlAdapter by providing the command in a string, the type of command and the ADO or OLEDB connection string.
SqlXmlAdapter( Stream CommandStream, SqlXmlCommandType cmdType, string connectionString ) | Initializes a SqlXmlAdapter by providing the command in a Stream, the type of command and the ADO or OLEDB connection string.

Methods

void Fill(DataSet ds) | Fills the provided data set with data obtained by executing the command the SqlXmlAdapter was initialized with.
void Update(DataSet ds) | Updates the database from the data in the DataSet.

DISPLAYING THE XML DATA USING ASP.NET WEB FORMS

ASP.NET introduces the concept of web forms, which are combinations of HTML, source code, and HTML GUI components (called controls) that can be used to build web applications. Server-side controls are reusable HTML components that can initiate server-side events and render the HTML dynamically as needed. Server-side controls are .NET classes that are children of the System.Web.UI.WebControls class. Thus ASP.NET pages can be bound to .NET classes, which contain code to handle things like events and output. Server-side controls can access the ASP objects such as Request and Response available via inherited properties. An overview of ASP.NET and server-side controls is available at http://msdn.microsoft.com/vbasic/technical/upgrade/intro.asp?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq.

Although ASP.NET ships with a number of server-side controls, one may want to create a custom control. A custom web control is a class that subclasses the System.Web.UI.UserControl class and includes custom rendering code in the Render(System.Web.UI.HtmlTextWriter writer) method. The primary advantages of custom controls are that they reduce code redundancy across a web application and also provide a clean separation between code and content.

What follows is a description of a custom control that is used as part of a page that allows one to select a list of countries from a drop down list and show a list of suppliers from the Northwind database in that country. The dropdown list of countries will be populated from the database.

To utilize a custom control in an ASP.NET page, two things are needed. The first is to register the control at the top of the page using a page-level directive as shown below:

<%@ Register TagPrefix="user_controls" TagName="SuppliersTable" Src="SuppliersTable.ascx" %>

and the second is to include the control in the web form as one would with a normal HTML component, such as is shown below:

<user_controls:SuppliersTable id="SuppliersTable1" runat="server" />

The runat="server" attribute indicates that the above HTML component is an ASP.Net server-side control.

The SuppliersTable control exists in two places, as a .ascx file that contains the HTML code, and a .cs file that contains the C# code for the SupplierTable class. The .ascx file contains a Control directive whose Inherits attribute makes the SuppliersTable class known to the page compiler. Below is the control directive that would be used in the SuppliersTable.ascx file

<%@ Control Language="c#"
            AutoEventWireup="false"
            Codebehind="SuppliersTable.ascx.cs"
Inherits="SuppliersAndProductsWeb.SuppliersTable"%>

The most important part of the SuppliersTable class is the overridden Render method which is shown below:

protected override void Render(System.Web.UI.HtmlTextWriter writer)
{

try{

// Get country from query string
string selectedCountry = Request.QueryString["country"];

// Start creating html output
Response.Write("<table align=\"left\" border=\"1\" cellpadding=\"0\" cellspacing=\"0\"
style=\"border-collapse: collapse\" bordercolor=\"#777777\" width=\"100%\">");

// Load the document from the results of the XML template
// containing "/Supplier" query
XmlDocument xd = new XmlDocument();
xd.Load(ServerInfo.allDataUrl);

// Use XPath to get every instance of "Country"
XmlNodeList countryNL = xd.SelectNodes("//Country");

Response.Write("<select name=\"country\">");
Response.Write("<option value=\"\"></option>");

// Create a sorted list of countries from the results of
// the XPath query

SortedList countriesList = new SortedList();

for(int cI = 0; cI < countryNL.Count; cI++)
{
string country = countryNL.Item(cI).InnerText;
if(!countriesList.ContainsKey(country))
countriesList.Add(country,country);
}

// Populate the combo with the sorted list of non-repeated countries
for(int cI = 0; cI < countriesList.Count; cI++)
{
string country = countriesList.GetKey(cI).ToString();
string selectedString = country.Equals(selectedCountry) ? " selected " : "";
Response.Write("<option value=\"" + country + "\"" + selectedString + ">" + country +
"</option>");

}
Response.Write("</select>");

Response.Write("<input type=\"submit\" value=\"Ok\"/>");

// Use XPath to query for all suppliers in the selected country
XmlNodeList supplierNL = xd.SelectNodes("/ROOT/Supplier[Country='" + selectedCountry + "']");

Response.Write("<tr bgcolor=\"#777777\">");
Response.Write("<td>Supplier</td>");
Response.Write("<td>Country</td>");
Response.Write("</tr>");

// Process and print out every supplier
for(int i = 0; i < supplierNL.Count; i++)
{
XmlNode supplierNode = supplierNL.Item(i);
XmlNodeList supplierValuesNL = supplierNode.ChildNodes;

Response.Write("<tr>");

// Use XPath to extract specific node values from the DOM
string companyName = supplierNode.SelectSingleNode("CompanyName").InnerText;
string country = supplierNode.SelectSingleNode("Country").InnerText;
string supplierId = supplierNode.SelectSingleNode("SupplierID").InnerText;

// Print out html
Response.Write("<td>");

// create hyperlink on supplier name that invokes
// SupplierDetails.aspx page with SupplierID as a parameter
Response.Write("<a href = \"SupplierDetails.aspx?supplier_id=" + supplierId + "\">");
Response.Write(companyName);
Response.Write("</a>");

Response.Write("</td>");

Response.Write("<td>"+country+"</td>");
Response.Write("</tr>");

}
Response.Write("</table>");

}catch(Exception e){

Response.Write("<h1 align=\"center\">" +
"An error has occurred in processing this page</h1>");

/* send an email to the author of the application */
MailMessage message = new MailMessage();

message.To = "kpako@yahoo.com";
message.From = "Suppliers And Products Web Application";
message.Subject =
"Suppliers Table Web Application Failure: "
+ Request.Path;

message.BodyFormat = MailFormat.Text;

message.Body = "Path: " + Request.Path + "\n\nError: " +
Server.GetLastError().ToString();

//if no SMTP server on localhost then we'll need to specify one
//SmtpMail.SmtpServer = "127.0.0.1"

SmtpMail.Send(message);

}

}/* void Render(System.Web.UI.HtmlTextWriter) */

The above method performs two primary tasks:

  • It populates the drop down list with the list of countries in the database by using an XPath query against the XML view of the Northwind database
  • It generates a table containing information on all suppliers for a selected country and hyperlinks the supplier name to the SupplierDetails.aspx page.

TRANSFORMING THE XML TO HTML WITH XSLT

XSLT stands for eXtensible Stylesheet Language Transformations and is designed to transform one XML document to another text-based format. The XSLT standard supports transformations that result in text, HTML, or XML documents.

XSLT is a declarative or rule-based language in which one describes how the translation is done via the utilization of template rules. Rather than dealing with the low level details like parsing the XML or traversing nodes, one describes how the results of a query get transformed. An XSLT stylesheet typically consists of elements from the XSL namespace (http://www.w3.org/1999/XSL/Transform?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq) which are instructions on how to manipulate the input XML document and output text. For instance, given the following XML to convert to a sorted HTML table:

<People>
<Person first-name="Dave" last-name="Rivers">
<Person first-name="Lucy" last-name="Johnson">
<Person first-name="Mike" last-name="Alexander">
</People>

The XSLT could look like this:

<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<xsl:template match="People">

<table width="100%" bordercolor=black">
<tr><td>Last</td><td>First</td></tr>

<xsl:for-each select="Person" >
<xsl:sort select="@last-name">

<tr>
<td><xsl:value-of select="@last-name"/></td>
<td><xsl:value-of select="@first-name"/></td>
</tr>

</xsl:sort>
</xsl:for-each>
</xsl:template>
</xsl:stylesheet>

The above XSLT document contains a template rule that matches the <People> element, and is executed every time a <People> element occurs in the input XML document. Following this is output text that creates the first row of the HTML table. The xsl:for-each element is a looping construct that [in this instance] iterates through all the <Person> elements. The usage of the xsl:sort element indicates that the loop will iterate through the elements in sorted order by their last name attribute. The body of the loop prints out the values of the last and first name attributes of each <Person> element as a new row in the HTML table.

For more information on XSLT read XSLT By Example tutorials at http://www.objectsbydesign.com/projects/xslt/xslt_by_example.html?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq on the Objects By Design site. The XSLT chapter of the XML Bible is also available online at http://www.ibiblio.org/xml/books/bible2/chapters/ch17.html?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq.

It should be noted that modern browsers such as IE 6.0 (or higher), Mozilla 0.9.7 (or higher), or Netscape 6.2.1 (or higher) can run client side XSLT, which dramatically increases the scalability of an application versus server-side transformations..

DISPLAYING DETAILS OF A PARTICULAR SUPPLIER

XSLT is used by the SupplierDetails.aspx page to transform a specific <Supplier> element obtained by querying the XML view of the database to a suitable HTML representation. The Page_Load(object sender, System.EventArgs e) of the SupplierDetails class queries the XML view of the Northwind database for a Supplier with the SupplierID passed as a parameter to the page.

private void Page_Load(object sender,
            System.EventArgs e)
{

try
{
// Get SupplierID being
            sought after from query string
string supplierId =
            Request.QueryString["supplier_id"];

// Connect to the SQL
            Server database
SqlXmlCommand cmd = new
            SqlXmlCommand(ServerInfo.ConnectionString);

//Specify the
            XPath query
cmd.CommandText = "/Supplier[SupplierID=" +
            supplierId + "]";

//Indicate that the query language is XPath
cmd.CommandType = SqlXmlCommandType.XPath;

//specify
            optional root element for queries that return results
// that are
            not well-formed
cmd.RootTag = "ROOT";

//specify location
            of mapping schema
cmd.SchemaPath =
            ServerInfo.mappingSchema;

// Get the query results as a
            XmlReader
XmlReader Reader;
Reader =
            cmd.ExecuteXmlReader();

// Put the results into a
            DOM
XmlDocument doc = new XmlDocument();
doc.Load(Reader);

//Link back to the main page
//[Probably unnecessary since browser should cache previous
            page]
string supplier_country =
doc.SelectSingleNode("/ROOT/Supplier/Country").InnerText;
Response.Write("<a href=\"Main.aspx?country=" +
            supplier_country +
            "\">");
Response.Write("&lt;&lt;&lt; Back to
            country");
Response.Write("</a><br><br><br>");

//Load
            the XSLT stylesheet that will be used to generate the HTML
XslTransform xslt = new
            XslTransform();
xslt.Load(ServerInfo.supDetailsXslt);

//
            Process the DOM with XSLT and send the results to the
            browser
XmlWriter xw = new
            XmlTextWriter(Response.Output);
xslt.Transform(doc, null,
            xw);

}
catch(Exception ex)
{
Response.Write(ex);
}
}

The XSLT stylesheet used to produce the resulting HTML page is designed with the foreknowledge that some of the columns in the database would have null values (i.e. nil values in the resulting XML).

To deal with null values, a check to see whether the input XML element has an xsi:nil attribute is performed, and only if the attribute does not exist is that row of the HTML table created. The excerpt below from the XSLT file shows how a field that is expected to exist in the resulting XML is treated versus a field that may or may not be null.

<xsl:stylesheet version="1.0"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" >

<xsl:template match="Supplier">
<div>
<table border="1" cellpadding="0" cellspacing="0" style="border-collapse: collapse"
bordercolor="#777777" width="100%">
<tr>
<td bgcolor="#777777" colspan="2">
Company
</td>
</tr>
<tr>
<td>
Name
</td>
<td>
<xsl:value-of select="CompanyName" />
</td>
</tr>
.
<xsl:if test="not(Region[@xsi:nil])">
<tr>
<td>
Region
</td>
<td>
<xsl:value-of select="Region" />
</td>
</tr>
</xsl:if>
.
</xsl:stylesheet>

The stylesheet also contains a section that generates hyperlinks to the ProductDetails.aspx page attached to the names of each product supplied by the selected Supplier.

<xsl:for-each select="Product">
<tr>
<td>
<a href="ProductDetails.aspx?product_id={ProductID}">
<xsl:value-of select="ProductName" />
</a>
</td>
</tr>
</xsl:for-each>

DISPLAYING DETAILS OF A PARTICULAR PRODUCT

XSLT is used by the ProductDetails.aspx page to transform a specific <Product> element obtained by querying the XML view of the database to a suitable HTML representation. The Page_Load(object sender, System.EventArgs e) of the ProductDetails class queries the XML view of the Northwind database for a Product with the ProductID passed as a parameter to the page. There is little difference between the Page_Load method of the SupplierDetails class and that of the ProductDetails class besides the fact that the ProductDetails page uses a different tactic to generate a link to the previous page, and that ProductDetails uses a different stylesheet to transform the XML to HTML:

private void Page_Load(object sender,
            System.EventArgs e)
{

//Link back to the main page
Response.Write("<a href=\"" + Request.UrlReferrer +
            "\">");
Response.Write("&lt;&lt;&lt; Back to
            supplier");
Response.Write("</a><br><br><br>");

try
{
//
            Get the product_id from the query string
string productId =
            Request.QueryString["product_id"];

// Connect to the SQL
            Server database
SqlXmlCommand cmd = new
            SqlXmlCommand(ServerInfo.ConnectionString);

//Specify the
            XPath query
cmd.CommandText = "/Supplier/Product[ProductID=" +
            productId + "]";

//Indicate that the query language is XPath
cmd.CommandType = SqlXmlCommandType.XPath;

//specify
            optional root element for queries that return results
// that are
            not well-formed
cmd.RootTag = "ROOT";

//specify location
            of mapping schema
cmd.SchemaPath =
            ServerInfo.mappingSchema;

// Get the results as a
            XmlReader
XmlReader Reader;
Reader =
            cmd.ExecuteXmlReader();

// Put the results into a
            DOM
XmlDocument doc = new XmlDocument();
doc.Load(Reader);

//Load the XSLT stylesheet that
            will be used to generate the HTML
XslTransform xslt = new
            XslTransform();
xslt.Load(ServerInfo.prodDetailsXslt);

//
            Process the DOM with XSLT and send the results to the
            browser
XmlWriter xw = new
            XmlTextWriter(Response.Output);
xslt.Transform(doc, null,
            xw);

}
catch(Exception ex)
{
Response.Write(ex);
}

}

However the XSLT stylesheet used to display the details for each product is much simpler than that used to display the supplier details since it simply iterates through each <Product> element and prints the name of each child element and its text content.

<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<xsl:template match="Product">
<xsl:for-each select="*">
<tr>
<td>
<xsl:value-of select="local-name()" />
</td>
<td>
<xsl:value-of select="." />
</td>
</tr>
</xsl:for-each>
</xsl:template>
<xsl:template match="*">
<table border="1" cellpadding="0" cellspacing="0" style="border-collapse: collapse"
bordercolor="#777777" width="100%">
<tr>
<td bgcolor="#777777" colspan="2">
Product
</td>
</tr>
<xsl:apply-templates />
</table>
<br />
</xsl:template>
</xsl:stylesheet>

APPLICATION SETUP

  • Follow the directions in the SQLXML 2.0 documentation on creating the nwind virtual directory (http://msdn.microsoft.com/library/default.asp?url=/library/en-us/xmlsql/ac_xml1_2xvd.asp&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq)
  • Create a new Visual C# Project -> ASP.NET Web Application and name it "SuppliersAndProducts" web. Visual Studio will then create a directory on your web server with the same name and initialize some files
  • Unzip the files in the provided SuppliersAndProductsWeb.zip file into a directory of your choice.
  • Select "Add Existing Item" from the Project menu.
  • Select "All (*.*)" from the "Files of type" drop down list.
  • Navigate to the directory that contains the files that were unzipped from the SuppliersAndProductsWeb zip file and add them to the project.
  • Edit the ServerInfo.cs file to conform to the particulars of your machine.
  • Build and Run the project.

CONCLUSION

In this article we've explored how to use W3C XML standards in combination with SQL Server' SQLXML capabilities and ASP.NET to build web applications that expose relational data as XML and presents it to clients as HTML.

This article focused on the ability to query the SQL Server 2000 XML views, but one can also create, update, and delete in the database using SQLXML updategrams and diffgrams. The next article in the series will extend this application by allowing users to add, deleten and modify data from the web interface.

Article Information
Authors | Dare Obasanjo & Sanjay Bhatia
Technical Editor | Adam Ryland
Project Manager | Helen Cuthill
Reviewers | Sean Schade, Saurabh Nandu

If you have any questions or comments about this article, please contact the technical editor.

C#Today - © 2002 WroxPress Limited


Cool Tool for XSLT - Part 2

Cool Tool for XSLT - Part 2
http://www.csharptoday.com/content.asp?id=1819
Article Rating: 4.5
Published on February 12, 2002
Abstract
XSLT, or “Extensible Style Language: Transformations? is a way of transforming XML files from one format to another. XSLT has been in the limelight for a while now, as it has been a popular technology for Web site builders for changing XML data into HTML. XSLT can be a little complicated to understand. In this article, Matthew Reynolds will show you how to build a tool that can help you learn about XSLT and also use XSLT in your applications. In the previous part of this article, we looked at how to build a simple XML editor control and use that control in our application. In this second part, we’ll look in more detail at XSLT and see a few of the basic building blocks that are helpful in day-to-day use.
Article

In this section of our article on XSLT, or rather how to build a cool tool for helping us work with XSLT, we're going to be adapting our control to handle the XML and HTML preview modes and also trying some transformations for the first time.

Using IE

To use Internet Explorer, we have to use COM interop to load and run the Microsoft Web Browser COM ("ActiveX") control. At the time of writing, there is no "managed code" version of IE - i.e. one that runs actually inside of the .NET runtime environment. That's basically because Microsoft hasn't written one yet! A Web browser as sophisticated as IE is represents a considerable investment, so it's not surprising that they expect us to make do with an old school COM component, rather than a nice new .NET component.

In fact, for those of you that don't know this, the Microsoft Web Browser control is IE. The iexplore executable actually acts as a container for this control. When we use this control in our applications, we're actually using Internet Explorer.

The limitation with using IE is that it's geared up to download its own resources from the private network or Internet. It's not particularly easy to give IE a scrap of HTML and ask it to draw it. For this reason, the control is going to create two separate temporary files on the users computer and ask IE to "download" and render those files. We'll have one file for the XML preview, and one for the HTML preview. Technically these files will have the same name, but we'll give one a .xml extension and the other a .html extension as this is the easiest way to give IE instructions on how the two files should be rendered.

What we need to do is create a new User Control called MiniBrowser. Into this control we'll add an instance of the Microsoft Web Browser control. Once we've done that, in the constructor for XmlEditor where we create the TextBox control, we'll also create a MiniBrowser control. We can then alter the ConfigureView method such that the MiniBrowser control is displayed whenever the View or Xml properties are changed.

Create the new User Control now and add a StatusBar and Panel control to it, like this. Set the Panel control's Dock property to Fill.

To use the Microsoft Web Browser control, we have to customize the Toolbox and ask VS .NET to load the control. Right-click on the Toolbox and select Customize Toolbox. Scroll down the list until you find the Microsoft Web Browser control. Check it, and click OK.

The Explorer control will appear somewhere in your Toolbox, most likely right down at the bottom.

Draw an Explorer control onto the panel in the MiniBrowser control. Set its name to axIe, and Dock property to Fill. You won't be able to see the Explorer control once you do this.

We need methods in MiniBrowser that defer to methods on the Explorer control. Specifically, we need to be able to navigate to a given URL, and be able to refresh the page. In addition, we also need to be able to capture the StatusTextChange event off of the Explorer control and update our own status bar text.

Because we're using interop, some of this is a little counter intuitive. First off, when we need to navigate to a URL, we need to call the Navigate2 method of the Explorer control. When we do this, we need to supply a reference to a System.Object that happens to contain the URL that we want to visit. In addition, we also need to supply four blank parameters, using a similar technique. Add this method to MiniBrowser :

public void Navigate(string url)
{
// navigate...
object nullObject = null;
object urlObject = url;
axIe.Navigate2(ref urlObject, ref nullObject,
ref nullObject, ref nullObject, ref nullObject);
}

The RefreshDocument method works in a similar way. However, this time we just supply a single blank parameter:

public void RefreshDocument()
{
object nullObject = null;
axIe.Refresh2(ref nullObject);
}

When we added the Explorer control to the form, VS .NET generated an interop wrapper for us. This wrapper contains new classes that, primarily, allow us to create the control, but also allow us to tap into the events of that control. For example, if we want to create an event handler for the StatusTextChange event, we need to create an instance of AxSHDocVw.DWebBrowserEvents2_StatusTextChangeEventHandler. Add this code the constructor for MiniBrowser :

public MiniBrowser()
{
// This call is
            required by the Windows.Forms Form
            Designer.
InitializeComponent();

// event
            handler...
axIe.StatusTextChange += new
AxSHDocVw.DWebBrowserEvents2_StatusTextChangeEventHandler(IeStatusTextChange);

//
            go to blank...
Navigate("about:blank");
}

The code for IeStatusTextChange looks like this:

protected void IeStatusTextChange(object sender,
AxSHDocVw.DWebBrowserEvents2_StatusTextChangeEvent e)
{
statusbar.Text = e.text;
}

Now that we have the control, we can create an instance of it and display it. To do this, we need private fields in XmlEditor to hold an instance of MiniBrowser and also strings for the temporary filenames. Add the _ie, _xmlPreviewFilename and _htmlPreviewFilename members to XmlEditor :

public class XmlEditor : System.Windows.Forms.UserControl
{
// members...
private TextBox _editor;
private MiniBrowser _ie;
private Control _activeView;
private ViewMode _view = ViewMode.XmlEdit;
private string _xmlPreviewFilename;
private string _htmlPreviewFilename;

In the constructor, we need to create the new MiniBrowser control and use the System.IO.Path class to get the name of a temporary file to use.

public XmlEditor()
{
// This call is
            required by the Windows.Forms Form
            Designer.
InitializeComponent();

// create the editor
            control...
_editor = new TextBox();
_editor.Multiline =
            true;
_editor.WordWrap = false;
_editor.ScrollBars =
            ScrollBars.Both;
_editor.AcceptsReturn =
            true;
_editor.AcceptsTab = true;

// create ie and the temp
            files...
_ie = new MiniBrowser();
_xmlPreviewFilename =
            Path.GetTempFileName();
_htmlPreviewFilename =
            _xmlPreviewFilename + ".html";
_xmlPreviewFilename +=
            ".xml";

// configure the
            view...
ConfigureView();
}

Now we can turn our attention to ConfigureView. When we initially put together this method, we missed out the code that showed the IE control. We first off need to check the View property to see what view we need to show.

// ConfigureView - update the view so that
            we can see the data...
public void ConfigureView()
{
//
            what view do we need to see?
Control requiredView =
            null;
switch(View)
{
case ViewMode.XmlEdit:
requiredView
            = _editor;
break;
case ViewMode.XmlPreview:
requiredView =
            _ie;
break;
case ViewMode.HtmlPreview:
requiredView =
            _ie;
break;

default:
throw new Exception("View mode not
            supported");
}

// do we need to change the
            view?
if(_activeView != requiredView)
{
// clear the
            current view...
panelMain.Controls.Clear();

// show the
            new view...
_activeView = requiredView;
_activeView.Dock =
            DockStyle.Fill;
panelMain.Controls.Add(_activeView);
}

Once we have the view, we need to create the two preview files.

// create the temp
            files...
try
{
FileStream stream = new
            FileStream(_xmlPreviewFilename, FileMode.Create);
StreamWriter
            writer = new
            StreamWriter(stream);
writer.Write(Xml);
writer.Flush();
stream.Close();
stream
            = new FileStream(_htmlPreviewFilename, FileMode.Create);
writer =
            new
            StreamWriter(stream);
writer.Write(Xml);
writer.Flush();
stream.Close();
}
catch(Exception
            ex)
{
MessageBox.Show(ex.GetType().ToString() + ":" +
            ex.Message);
}

Finally, if we're looking at IE, we need to navigate to whichever preview file is appropriate.

// ie?
if(View == ViewMode.XmlPreview)
_ie.Navigate(_xmlPreviewFilename);
if(View == ViewMode.HtmlPreview)
_ie.Navigate(_htmlPreviewFilename);
}

It's bad practice to leave temporary files floating around when an application no longer needs them. What we'll do is adapt the Dispose method of the control such that it will delete the temp files when the object is disposed.

protected override void Dispose( bool
            disposing )
{
if( disposing )
{
if(components !=
            null)
{
components.Dispose();
}
}

// delete temp
            files...
DeleteTempFile(_xmlPreviewFilename);
DeleteTempFile(_htmlPreviewFilename);

base.Dispose(
            disposing );
}

protected void DeleteTempFile(string
            filename)
{
try
{
FileInfo info = new
            FileInfo(filename);
if(info.Exists ==
            true)
info.Delete();
}
catch
{
}
}

You can try this now if you like. Build the project to save the changes and open the Designer for Form1. Change the View property for xmlXml to XmlPreview and you'll see IE.

The View Menu

Being able to change the view mode is really only useful when the application is running. We need to wire up the View toolbar button to allow the user to change the mode that he wants. Ideally we need a drop-down menu available that allows quick access to the appropriate mode.

To do this, open the Designer for XmlEditor and add a new ContextMenu object. Change its Name property to menuView. Add three new items to the list, the names menuViewXmlEdit, menuViewXmlPreview and menuViewHtmlPreview.

Double-click on each of the items and add code to the handlers to set the View property.

private void menuViewXmlEdit_Click(object sender, System.EventArgs e)
{
View = ViewMode.XmlEdit;
}

private void menuViewXmlPreview_Click(object sender, System.EventArgs e)
{
View = ViewMode.XmlPreview;
}

private void menuViewHtmlPreview_Click(object sender, System.EventArgs e)
{
View = ViewMode.HtmlPreview;
}

One more thing needs to be done - we need to attach this ContextMenu control to the buttonView button. Open the Buttons property editor on toolbar and set the DropDownMenu property to menuView and set the Style property to DropDownButton.

Now run the project and you'll be able to change the view using the menu.

Transforming XML

At last we can try and transform some XML using XSLT! The first thing we need is a menu option. If you haven't already done so, add a MainMenu control to Form1 and then add a top-level menu called Transform and a sub-menu item called Transform Document. Set the Name property of this menu item to menuTransformTransform and set the Shortcut property to F5.

The second thing we need is a sample style sheet. Add this to the Lines property of xmlXsl (don't worry about what it does at the moment - we'll look at it later).

<?xml version="1.0"?>
<xsl:stylesheet version="1.0"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform">

<xsl:template match="/Customers">

<ul>
<xsl:for-each select="Customer">
<li>
<xsl:value-of select="Id"/>
</li>
</xsl:for-each>
</ul>

</xsl:template>

</xsl:stylesheet>

The class that we need to use for XSLT transformations is System.Xml.Xsl.XslTransform. The two methods we need are Load (used to load the XSL file) and Transform (used to perform the actual transformation). XslTransform works with XPath documents. XPath is a way of traversing an XML document using a "directory-scheme" style structure. I'll explain this more in a moment once we've done the transformation and we can see what results we actually get back.

An XPath document can be held in the class System.Xml.XPath.XPathDocument. The trick here is to take the value in our Xml property and create an XPathDocument from it. Unlike the XmlValidatingReader class that we saw before, we can't just give XPathDocument a block of text. It either has to load it itself from a network/Internet resource, or it has to get it from a stream. We can use System.IO.MemoryStream to do this. Our next job is to build the CreateMemoryStream method on XmlEditor. This will return a MemoryStream object that we can use as a source of data for the XPathDocument that we need for the transformation.

Creating a populated memory stream requires an extra, non-intuitive step. Because .NET is quite open to the idea of using different encoding standards other than ASCII or UNICODE, we have to use a separate class ( System.Text.ASCIIEncoding) to convert the System.String value to an array of System.Byte objects. Here's the method that should be added to XmlEditor :

public MemoryStream
            CreateMemoryStream()
{
// create the new
            stream...
ASCIIEncoding encoding = new ASCIIEncoding();
return
            new MemoryStream(encoding.GetBytes(Xml));
}

Creating the XPathDocument is simply a matter of getting hold of that stream and passing it to the constructor of the new XPathDocument object, like this:

// CreateXPathDocument - create an XPath
            Document for the data...
public XPathDocument
            CreateXPathDocument()
{
// create the new
            document...
Stream inputStream =
            CreateMemoryStream();
XPathDocument document = new
            XPathDocument(inputStream);
inputStream.Close();

//
            return...
return document;
}

Now that we can get XPathDocument objects back for both the XML and XSL editors, we can look at how do to the transformations. First off, open the Designer for Form1 and add a new Click event handler to the Transform - Transform Document option by double-clicking on it. Add this code:

private void menuTransformTransform_Click(object sender, System.EventArgs e)
{
Transform();
}

The Transform method is responsible for performing the transformation and setting the Xml property of xmlResults. The first thing the method needs to do is to check that the source XML and XSL documents are well formed.

public void Transform()
{
// test the first document?
string message = xmlXml.CheckDocumentWithReport();
if(message != null)
{
MessageBox.Show(
"The XML document is not well formed.\r\n\r\n" +
message, "XSLT");
return;
}
message = xmlXsl.CheckDocumentWithReport();
if(message != null)
{
MessageBox.Show(
"The XSL document is not well formed.\r\n\r\n" +
message, "XSLT");
return;
}

Once that's done, two things can go wrong. One, although the XSL document is well formed, it may not be valid - i.e. XslTransform may not be able to compile it. Two, although the transformation takes place, the results will not be as intended. This is the same deal that we face when writing software everyday - we may either see a compile error, or a run-time error.

The first thing to do is create a new MemoryStream object that the transformation output will be written to. Later, we'll read back the data written into this stream and set the Xml property of XmlResults to this value. We then load the XSL document into the XslTransform object.

// create somewhere to put the
            data...
MemoryStream outputStream = new
            MemoryStream();
try
{
// create the
            transform...
XslTransform transform = new
            XslTransform();

// load...
XPathDocument xslDocument =
xmlXsl.CreateXPathDocument();
transform.Load(xslDocument);

Once we've done that, we try the transformation. We get an XPathDocument back containing the source XML, and then call the Transform method.

// transform...
XPathDocument xmlDocument =
xmlXml.CreateXPathDocument();
transform.Transform(xmlDocument, null, outputStream);

Finally, once we've filled up the stream with the transformation data, we need to make sure all the data that needs to be written has been written by using the Flush method, and then read all the data back out again.

// read the data back and
            display...
outputStream.Flush();
outputStream.Seek(0,
            SeekOrigin.Begin);
StreamReader reader = new
            StreamReader(outputStream);
xmlResults.Xml =
            reader.ReadToEnd();
}

If something goes wrong, we use the static GetReportForException method on XmlEditor and display the information to the user.

catch(Exception ex)
{
// get the
            report...
message = XmlEditor.GetReportForException(
"The
            transformation could not be performed.",
            ex);
MessageBox.Show(message, "XSLT");
}
finally
{
//
            close...
outputStream.Close();
}
}

Run the project now and select the Transform Document option from the menu. You should see a string of XML in the right-hand editor.

Selecting the HTML Preview view shows us what that string of XML would look like if the browser were asked to display it. As it contains HTML tags, it draws it as two items in an unordered list.

"What Happened?"

That looks a little like magic, but what's actually happening?

XSLT can be used to transform XML documents from one format into another. In this case, we've taken an XML document that expresses data and reformatted it into an XML document that in actual fact is an XHTML document. XHTML is an emerging standard that in the long term may replace HTML. It's basically HTML that adheres to XML standards. XML has far more rules than HTML, so when we write XHTML code it tends to be far less ambiguous and therefore far easier to parse than traditional HTML code.

However, all of this is off-topic because virtually all browsers can render XHTML, and for the most part XHTML isn't important to this discussion. The important thing is that we have taken an XML document containing data and transferred it to what you can think of as an "HTML-compatible XML" document. For all intents and purposes, this is HTML.

To understand this, we have to look at the style sheet because this contains all of the instructions. I'll go through it line-by-line.

First off, we have the usual XML declaration that tells us that this is an XML document:

<?xml version="1.0"?>

Secondly, we start a tag defined in the http://www.w3.org/1999/XSL/Transform namespace. This contains the rules for XSL transformations. You can think of it as containing the language used in XSLT:

<xsl:stylesheet version="1.0"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform">

As we work through the document, imagine the document as a folder tree. Each child element is a sub-folder inside the parent element. XPath works on the principle that you have a pointer that points at your location in the tree. When the document is first created, this pointer points before the first data element. In our document, this is the Customers element, which is the top-level element.

The second tag ( xsl:template) we have tells the XSLT compiler that we want to build a template. Usually, whenever you see match or select attributes in XSLT, this is an XPath directive. Here we're saying "match the root-Customers element". The forward-slash is like an absolute path location - i.e. it's at the root of the document:

<xsl:template match="/Customers">

At this point, anything inside the xsl:template will be rendered whenever the compiler comes across the element specified in the match attribute. As there's only one /Customers element, this template will be processed once.

Anything inside the element that isn't an XSLT processing instruction (usually anything that does not begin with xsl:) is copied to the result document. In this case, we're taking a ul start tag over to the results:

<ul>

Next we have a xsl:for-each instruction. This is, as you might expect, a for loop. What we're doing though is telling it to select on Customer. As the XPath pointer is on /Customers, this will be any Customer element inside /Customers. The instructions inside will be executed exactly once per Customer element. Per iteration, the XPath pointer will be moved to the Customer element that's being processed:

<xsl:for-each select="Customer">

Now, a similar deal to before - we copy over an li tag:

<li>

Next, another XSLT instruction. This time, we're using xsl:value-of, which is a way of inserting the value selected into the target document. In this case, we want the value inside the Id element. As the XPath pointer is on Customer at this point, this will be the Id element inside the current Customer element, for example /Customers/Customer/Id :

<xsl:value-of select="Id"/>

After writing the value, we write the end li tag:

</li>

Now we close the xsl:for-each instruction. This is where the loop will either quit, or go back up to the start where the XPath pointer will be positioned at the next Customer element:

</xsl:for-each>

Finally, we close the ul tag (as all the Customer elements would have been processed at this point), close the template, and close the style sheet:

</ul>

</xsl:template>

</xsl:stylesheet>

What Now?

I found that the lynchpin to understanding XSLT was getting to grips with XPath. Once you start imagining the pointer down around the document through the folders and sub-folders, things get a little easier to understand. XSLT is quite dissimilar to traditional programming, which is why a lot of software developers struggle with it. One of the reasons why I built this tool was to make it easy to experiment with XSLT.

Here are some resources that I found useful:

  • http://www.w3schools.com/xsl/xsl_transformation.asp
  • http://www.vbxml.com/xsl/tutorials/intro/default.asp
  • http://hotwired.lycos.com/webmonkey/98/43/index2a.html?tw=authoring

You might also like to check out Michael Kay's "XSLT Programmers Reference". (ISBN 1861005067).

Go experiment! You'll love it!

Summary

In this two-part article we took a look at how to build some cool tools for working with XSLT. In this section, we learned how we can reuse Internet Explorer in our application and get it to render both the XML data in the XmlEditor class and also the HTML data resulting from transformations.

We also saw how to perform a simple transformation and took a look at how XPath works in the context of XSLT.

Article Information
Author | Matthew Reynolds
Technical Editor | Adam Ryland
Project Manager | Helen Cuthill
Reviewers | Vickie Pring, Eddie Chandler

If you have any questions or comments about this article, please contact the technical editor.

C#Today - © 2002 WroxPress Limited


Cool Tool for XSLT - Part 1

Cool Tool for XSLT - Part 1
http://www.csharptoday.com/content.asp?id=1818
Article Rating: 3.5
Published on February 11, 2002
Abstract
XSLT, or “Extensible Style Language: Transformations? is a way of transforming XML files from one format to another. XSLT has been in the limelight for a while now, as it has been a popular technology for Web site builders for changing XML data into HTML. XSLT can be a little complicated to understand. In this article, Matthew Reynolds will show you how to build a tool that can help you learn about XSLT and also use XSLT in your applications. In this first part, we’ll look at how to build a simple XML editor control and use that control in our application. In the second part, we’ll look in more detail at XSLT and see a few of the basic building blocks that are helpful in day-to-day use.
Article

Some of you reading this will be familiar with the concept of XSLT, or "Extensible Style Language: Transformations". For those of you who are not, XSLT is a way of transforming XML files from one format to another. XSLT has two main functions:

  • Changing a XML document used in business exchange from one company's version to another. (For example, renaming elements, moving elements around inside the document, removing elements you don't need, etc).
  • Changing an XML document into a HTML document for display.

XSLT has been in the limelight for a while now, as it has been a popular technology for Web site builders for changing XML data into HTML. Rather than having to write masses of script code to take the database-provided data and turn it into HTML, the data can either be stored as XML files on the database, or some databases (including SQL Server 2000) can be asked to return data as XML. In either case, XSLT can be used to put the data into HTML so that it can be rendered.

XSLT can be a little complicated to understand, mainly because programming in XSLT is not programming in the traditional sense. Most developers work on the principle of writing a method that starts from the top and runs down, making decisions as it goes. XSLT is a bit strange in that you build a file that contains a stylesheet and then ask the XSLT process to apply it to the original XML document and return the new XML document.

Whenever I've done XSLT programming in the past, I've found it a constant process of experimentation - i.e. change a little bit of the stylesheet, try the transformation, change a little more, try again, etc. In this article, I'll be showing you how to build a tool that can help you learn about XSLT and also use XSLT in your applications.

In the first part, we'll look at how to build a simple XML editor control and use that control in our application. In the second part, we'll look in more detail at XSLT and see a few of the basic building blocks that are helpful in day-to-day use.

The "XmlEditor" Control

The first thing we're going to do is to build the XmlEditor control. This control will allow us to edit the XML files. We're not going to see something as sophisticated as the VS .NET XML editor, or the editor in XML Spy (a great tool for working with XML, by the way). In fact, we're going to reuse the TextBox control as the editor control. So that the user can preview the XML or HTML document returned from the transformation, we're also going to host Internet Explorer inside of our control, as this has excellent support for showing XML data as HTML.

Interestingly, IE itself uses XSLT to convert the raw XML data into the HTML view that it uses to present XML data.

To create the control, create a new Visual C# - Windows Application project (we'll roll the control into a Windows Application project rather than a Class Library project to make this demonstration a little easier. If you want to reuse the control in your own projects, simply move the .cs file into a new project. Simple!). Call the new project Transformer.

Laying Out "Form1"

For those of you who can't stand leaving the main form as Form1, change the name of the form now. I typically don't bother for examples like these, as the forms themselves tend to just act as hosts for the controls that we build - i.e. they're not the most important part of the project. I'll leave mine as Form1.

We want to configure the form such that we have the XML file to be transformed in the top left, the XSL in the bottom left and the transformed data on the right hand side. We can do this with a combination of Panel and Splitter controls.

Follow these instructions carefully.

Add a new Panel control to the form. Set its Dock property to Left. Add a splitter control, also making sure that its Dock property is set to Left (these panels are a little hard to see).

Next, add another Panel control to the right-hand side of the form. Set its Dock property to Fill. That gets us the splitter that allows us to change the bias between the left-hand and the right-hand controls.

Now add a new Panel control to the left-hand side. Set its Dock property to Top. Add a Splitter control to the blank area in the bottom-left. The splitter will automatically dock to the left, so change its Dock property to Top. You'll see this:

Finally, add another Panel control to the region in the bottom-left. Set its Dock property to Fill. This gives us the two splitters - one to change the bias between left and right, and another to change the bias between top and bottom on the left only.

Now that we have Form1 configured, we can turn our attention to the XmlEditor control.

Building "XmlEditor"

Add a new User Control to the project. Call it XmlEditor.

XmlEditor will have three view modes - XmlEdit, XmlPreview and HtmlPreview (i've named these modes after the enumeration that we'll use to express them in code.) The XmlEdit mode will use a properly configured TextBox control for its display. XmlPreview and HtmlPreview will use Internet Explorer, contained within the control itself, for its display. When using IE, temporary files will be used to store the data and IE will be asked to display those to the user.

In addition, the editor will also have a toolbar with buttons that allow the user to check whether the XML document is well formed and also to change the view type. A menu will be added to Form1 to allow us to load and save the documents, as well as download the documents from a Web server. (we won't be able to save the documents back to a Web server.)

To the new control, add a ToolBar control and a Panel control. The ToolBar will automatically dock to the top of the control, but you'll need to manually set the Dock property of the Panel to Fill. In addition, set the Name property of the Panel control to panelMain.

The ToolBar will have two buttons: buttonCheck and buttonView. In the download package for this article you'll find some buttons to use (I used a screen capture program to snag these from other programs). To use them, you need to follow these steps:

  • Add an ImageList control to the form. Call it imagelistToolBar. Add the two images to the ImageList by using the Images property.
  • Set the TransparentColor property of the control to Magenta (Bright pink, RGB: FF00FF).
  • Set the ImageList property of the ToolBar to imagelistToolBar.
  • Set the Name property of the ToolBar to toolbar.
  • Add the two buttons. Set the name of the first button to buttonCheck and the name of the second button to buttonView.

You'll get this:

Now, build the project. Add one XmlEditor control to each of the three visible panels on Form1. Set the name of the top-left one to xmlXml, the bottom-left one to xmlXsl and the right-hand one to xmlResults. Set the Dock property of each of them to Fill.

If you run the project, you'll be able to test that the splitter controls do adjust the view properly.

Changing View Modes

To display the editor, we need to implement the View property. This will let us select from the three modes: XmlEdit, XmlPreview and HtmlPreview. In the first instance, we'll only configure XmlEdit.

To support XmlEdit, we need a properly configured TextBox control. We'll build this programmatically, rather than doing it with the editor. First of all, we need private fields in XmlEditor that allow us to hold a reference to this TextBox control and also hold a reference to the currently active control. Plus, we'll need an enumeration that gives the consumer of the control the list of available modes.

Open the code editor for XmlEditor and add these three member variables:

public class XmlEditor : System.Windows.Forms.UserControl
{
// members...
private TextBox _editor;
private Control _activeView;
private ViewMode _view = ViewMode.XmlEdit;

When the control is created, we want to programmatically create the TextBox control. We want the control to have these properties:

  • It must be able to hold multiple lines,
  • It must have vertical and horizontal scroll bars,
  • It must not word wrap,
  • It must accept the Return key and the Tab key,
  • It must use the fixed size, Courier New font (the caller can change this.)

Later we'll also programmatically create the IE control too, but for now just add this code to the constructor:

public XmlEditor()
{
// This call is
            required by the Windows.Forms Form
            Designer.
InitializeComponent();

// create the editor
            control...
_editor = new TextBox();
_editor.Multiline =
            true;
_editor.WordWrap = false;
_editor.ScrollBars =
            ScrollBars.Both;
_editor.AcceptsReturn =
            true;
_editor.AcceptsTab = true;

// configure the
            view...
ConfigureView();
}

ConfigureView will be used whenever the view mode changes - and this most certainly applies when the control is first started. It's responsible for setting up the view so that the data we see reflects the data that the user has chosen.

// ConfigureView - update the view so that
            we can see the data...
public void ConfigureView()
{
//
            what view do we need to see?
Control requiredView =
            null;
switch(View)
{
case ViewMode.XmlEdit:
requiredView
            = _editor;
break;

default:
throw new Exception("View
            mode not supported");
}

// do we need to change the
            view?
if(_activeView != requiredView)
{
// clear the
            current view...
panelMain.Controls.Clear();

// show the
            new view...
_activeView = requiredView;
_activeView.Dock =
            DockStyle.Fill;
panelMain.Controls.Add(_activeView);
}
}

Before we can test this, we need to add the View property. This property needs to be accessible from the Properties window, so we use the Browsable, Category and DesignerSerializationVisibility attributes to configure this.

[Browsable(true), Category("Appearance"),
DesignerSerializationVisibility(DesignerSerializationVisibility.Visible)]
public
            ViewMode View
{
get
{
return
            _view;
}
set
{
_view =
            value;
ConfigureView();
}
}

If you build the project now and open the Designer for Form1, you'll notice that you can indeed see TextBox controls in the editors.

In addition, if you look in the Properties window you'll find the View property. However, do not change this value, as you'll get an exception (we'll fix this later when we add IE to the control).

Editing XML

The next move is to try and edit some XML with the control. If you start the application, you'll notice that you can indeed enter text into the controls. Of course, this text doesn't have to be XML and we're not going to look in this exercise at how to build a sophisticated XML editor like the ones in VS .NET or XML Spy.

What we do need, however, is an Xml property on the control. This will let the caller (and us as we're debugging) enter some XML text into the control.

First off, add this property to XmlEditor. We'll use the Text property of the TextBox control to actually hold the data.

[Browsable(true),
            Category("Appearance"),
DesignerSerializationVisibility(DesignerSerializationVisibility.Visible)]
public
            string Xml
{
get
{
return
            _editor.Text;
}
set
{
_editor.Text =
            value;
ConfigureView();
}
}

Notice how ConfigureView is called when we set the property. We'll need to use this to update the IE view when we get around to putting IE in the control.

After a rebuild, we can set the text in the control from the Properties window. However, this isn't the most useful way to enter XML data, as the edit box is very small and only supports a single line. If we implement a Lines property on the control that returns an array of string objects, VS .NET will give us a larger box to work with. This is simpler than creating our own custom editor for the property.

The Lines property is a little tricky to implement. When we set the Lines property, we need to iterate each line and build a single string, complete with line breaks, and set the Xml property to this new string. When we request the Lines property, we need to break up the string returned in Xml by new-lines (" \n ") codes, but also remove any carriage-return (" \r ") codes. Here's the code:

[Browsable(true),
            Category("Appearance")]
public string[]
            Lines
{
get
{
// blank?
if(Xml == "")
return
            null;

// split up XML by \r codes...
string[] rawLines =
            Xml.Split('\n');
for(int n = 0; n < rawLines.Length;
            n++)
{
// do we need to
            trim?
if(rawLines[n].Substring(rawLines[n].Length - 1, 1) ==
            "\r")
rawLines[n] = rawLines[n].Substring(0, rawLines[n].Length -
            1);
}

// return...
return rawLines;
}
set
{
// anything?
if(value != null)
{
// build up a new
            value for Xml...
StringBuilder builder = new
            StringBuilder();
for(int n = 0; n < value.Length;
            n++)
{
builder.Append(value[n]);
if(n != value.Length -
            1)
builder.Append("\r\n");
}
Xml =
            builder.ToString();
}
else
Xml = "";
}
}

If you build the project, you'll be able to edit the Lines property for the controls. Add this simple XML document to the Lines property of the xmlXml editor:

Checking the XML is Well Formed

Now that we have the document, let's write some code that checks to see if the XML in the editor is well formed. We can use the System.Xml.XmlValidatingReader class to do this for us.

We will be asked to check the code in one of three ways. Firstly, the user can click on the buttonCheck button. If she does this, she'll be expecting to see a message that tells her what is wrong with the XML, if indeed there is anything wrong. Secondly, the developer may want to get a simple Boolean answer to the question, "Is the XML well formed?" Thirdly, the developer may want to ask this question and get an exception back that describes the error in more detail.

We'll add three methods to XmlEditor. CheckDocument will check the document and throw an exception if there is a problem. IsDocumentWellFormed will return True or False, depending on whether it is or isn't. Finally CheckDocumentWithReport will check the document and display a message box telling the user if the document was well formed and if it isn't, why it isn't.

Before you add the code, add namespace import declarations for System.Xml, System.Xml.XPath and System.Xml.Xsl to the top of the code listing.

CheckDocument will simply create an XmlValidatingReader and walk through each node. If it gets to the end without throwing an exception, the document is valid. If it throws an exception, the exception will contain the nature and the location of the problem.

// CheckDocument - check the
            document...
public void
            CheckDocument()
{
XmlValidatingReader reader = null;

//
            walk each node...
try
{
// create a validating
            reader...
reader = new XmlValidatingReader(Xml,
XmlNodeType.Document, null);

//
            walk...
while(true)
{
if(reader.Read() ==
            false)
break;
}
}
catch(Exception ex)
{
//
            rethrow...
throw ex;
}
finally
{
// close the
            reader?
if(reader !=
            null)
reader.Close();
}
}

When we create the XmlValidatingReader, we give it a "document fragment". This happens to be the entire document returned from the Xml property. Once we've done that, we walk each node until we get to the end or something goes wrong. We use the finally clause to close the reader if whatever case occurs.

CheckDocumentWithReport calls this method. If something goes wrong, another method called GetReportForException.

// CheckDocumentWithReport - check the document, and report error...
public string CheckDocumentWithReport()
{
string report = null;
try
{
CheckDocument();
}
catch(Exception ex)
{
// get the report...
report = GetReportForException(
"The document is not well formed.", ex);
}

// return the report...
return report;
}

GetReportForException is a static method, and we'll use this in the next part of the exercise to report on problems when we do the actual XSLT transformations.

// GetReportForException - get the report
            for the exception...
public static string
            GetReportForException(string message, Exception ex)
{
//
            create a builder...
StringBuilder builder = new
            StringBuilder();
builder.Append(message);
builder.Append("\r\n\r\n");

//
            loop
while(ex != null)
{
// do we have an XML
            exception?
if(ex is XmlException)
{
// report the line
            number...
builder.Append("Line:
            ");
builder.Append(((XmlException)ex).LineNumber.ToString());
builder.Append(",
            position:
            ");
builder.Append(((XmlException)ex).LinePosition.ToString());
builder.Append("\r\n");
}
if(ex
            is XsltCompileException)
{
// report the line
            number...
builder.Append("Line:
            ");
builder.Append(((XmlException)ex).LineNumber.ToString());
builder.Append(",
            position:
            ");
builder.Append(((XmlException)ex).LinePosition.ToString());
builder.Append("\r\n");
}
builder.Append(ex.Message);
builder.Append("\r\n\r\n");

//
            next...
ex = ex.InnerException;
}

//
            return...
return builder.ToString();
}

IsDocumentWellFormed is a simple method that relies on exceptions being thrown from CheckDocument if the document is not well formed.

// IsDocumentWellFormed - return
            true/false...
public bool
            IsDocumentWellFormed()
{
try
{
//
            check...
CheckDocument();
}
catch
{
//
            problem...
return false;
}

// ok...
return
            true;
}

Run the project now. If you click the Check button, you should be told that the document is OK. Remove the last </Customers> tag from the document, however, and you should see this:

Validating the XML

Checking that the document is well formed is not the same thing as checking whether or not the document is valid. For this kind of check, a DTD (Document Type Definition) or XSD (XML Schema Definition) file needs to be applied to the document.

We're not going to look at how to do this in this example, but if you do need to check the document against DTD or XSD files, you can use the XmlValidatingReader. To do this, use the Schemas property to add the schemas you want to check against before starting to walk the document nodes.

Menu Options

A typical design pattern for this kind of application is to have a File menu with New, Open, Save and Save As options. To save space, I'm not going to show you how to implement that in this section. However, the download package associated with this article does have this code in place so you can see how this is done.

Next Time

In this section, we took a look at how to build a basic XML Editor that reused the text editing functionality of the TextBox control. We saw how to encapsulate this functionality into a separate control, which we reused three times on a form. We also saw how to build some basic properties into the control and also how we could add functionality to check the well formed-ness of the XML in situ.

Article Information
Author | Matthew Reynolds
Technical Editor | Adam Ryland
Project Manager | Helen Cuthill
Reviewers | Vickie Pring, Eddie Chandler

If you have any questions or comments about this article, please contact the technical editor.

C#Today - © 2002 WroxPress Limited


Introducing XGen XSQL for .NET

Introducing XGen XSQL for .NET
http://www.csharptoday.com/content.asp?id=1811
Article Rating: 4.5
Published on January 31, 2002
Abstract
In today’s article, Paul Looijmans introduces you to XGen, an open-source XML-based XML Generation framework for Microsoft’s .NET platform written entirely in C#. The first extension of this framework is XSQL, a set of tags for generating XML from relational databases. The article contrasts XGen XSQL with SQL Server’s FOR XML solution for generating XML and discusses some of the specifics of both XGen and XSQL.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

Ever since XML became popular a few years ago, its relationship with relational database technology has been the topic of a lot of discussion. XML evangelists tout its flexibility and simplicity, while proponents of the relational database solution point to the dangers of storing information in native XML: information can get scattered throughout the filesystem and there is no proven method to link different XML documents together easily. This leads to a lot of information redundancy. The database proponents further point to the fact that XPath can't match SQL in terms of data selection and (especially) data manipulation capabilities.

The World Wide Web Consortium (W3C) (http://www.w3c.org/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq) is working hard to improve XML, and a lot of the arguments the relational database proponents used in the past are no longer valid. Standards such as XPointer which allows XML documents to be linked together, and the upcoming XQuery standard which defines a more powerful query language for XML, as well as the rise of so-called XML databases, have made XML a more viable solution for storing data.

On the other hand even some of the more hardened XML proponents have had to admit that using a relational database is the best solution for a wide range of problems, so the emergence of XML won't mean that relational databases will disappear anytime soon. Of course the different relational database management systems (RDBMS) like Oracle 9i (http://www.oracle.com/ip/deploy/database/oracle9i/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq), Microsoft SQL Server 2000 (http://www.microsoft.com/sql/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq) and IBM DB2 (http://www-3.ibm.com/software/data/db2/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq) haven't stood still during the rise of XML, and one of the focal points of the development of these databases has been their integration with the XML standard. It's fair to say Oracle is currently ahead of SQL Server 2000 in its XML integration, mainly because Oracle's release schedule is much shorter than Microsoft's: the new release of SQL Server, codenamed Yukon, is not scheduled for release anytime soon, but since the release of SQL Server 2000 Oracle has already released two significant upgrades to their RDBMS software. Microsoft is however making periodic upgrades of the SQL Server 2000 XML capabilities available through the so-called SQLXML Web Releases (http://www.microsoft.com/sql/techinfo/xml/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq).

The SQLXML Web Releases have so far never used the .NET platform for XML-enabling SQL Server 2000 even though the platform has been in public beta for more than a year. This is where XGen, the subject of today's article, comes in: XGen is an open-source framework for generating XML from any data source. It is based on the .NET platform and completely written in C#. Today, I will introduce the XGen framework to you and discuss XSQL, which uses XGen to generate XML from any relational database that supports OLEDB. In a follow-up article, I will discuss the technical implementation of XGen and XSQL more in-depth.

In order to better appreciate the flexibility of XGen XSQL I will start off by briefly discussing the functionality that SQL Server 2000 combined with the Web Releases offers for generating XML from relational databases so I can contrast Microsoft's solution with the XGen XSQL solution.

The SQL Server 2000 solution: FOR XML & Templates

The biggest advantage of exposing information in relational databases as XML is that the information can be used by any program on any platform, from a huge database on a clustered server to a spreadsheet on a Palm computer. XML's simplicity and flexibility combined with the backing of all industry giants has made it the de-facto standard for exchanging information. Another advantage is the ever-growing array of XML-related standards like DOM for programmatically accessing and changing XML, XPath for selecting specific fragments in an XML document, XSLT for transforming XML, XML Schema (XSD) to define XML structure, and many more. When SQL Server 2000 was released, it contained a new way to generate XML from the database. By using the FOR XML clause, the database, in conjunction with the SQL Server OLEDB provider, was able to generate XML. As an example let's say we have the following query:

SELECT * FROM Authors;

To generate XML instead of the usual ADO recordset or ADO.NET DataReader from this query we would have to modify it like this:

SELECT * FROM Authors FOR XML [raw|auto|explicit];

As you can see you have to specify a mode with the FOR XML clause. There are three possibilities:

  • Raw: The most basic way to format your results. All rows in the recordset are transformed to ROW xml elements with attributes matching the column names.
  • Auto: This mode is slightly more advanced than the raw mode. The biggest difference is that the auto mode puts a little more hierarchy into your results when the query is a JOIN query, although you'll have to stick to very strict implicit rules that make it hard to easily use this feature.
  • Explicit: This mode gives you complete freedom over the format of the generated XML. This freedom comes at a high price though: you will have to build a so-called universal table in your query that adheres to strict rules regarding column names and structure. When your XML document is more complex than 2 or 3 levels of nesting writing these FOR XML EXPLICIT queries becomes very cumbersome and the result is very hard to debug and maintain.

The generated XML can be accessed through ADO and ADO.NET, but since XML is often document-centric it is also possible (since SQLXML WebRelease 1) to store the FOR XML query in an XML file that can contain a small set of special tags that can generate XML. Upon request an IIS ISAPI application replaces those special tags with the results of a query. An example of a template is:

<author>
<sql:query xmlns:sql="urn:schemas-microsoft-com:xml-sql">
SELECT TOP 2 * FROM Authors FOR XML RAW
</sql:query>
</authors>

When requested through IIS, the file is processed on the server and would look like this on the client:

<authors>
<row firstname="Stephen" lastname="King"/>
<row firstname="William" lastname="Shakespeare"/>
</authors >

As you can see it's possible to generate XML from SQL Server 2000. So what's the problem, you might ask. The biggest problem is the complex T-SQL code you have to write when you want total control over the way your XML is presented. Like I said before, interoperability with different programs and platforms is a big plus, but this interoperability implies that your XML is formatted in a standard way, often based on XML Schemas or DTDs. The FOR XML EXPLICIT method forces you to build long queries with sometimes complex logic that is very hard to maintain. Another problem is the fact that the FOR XML solution cannot be extended in any way, you're stuck with the RAW, AUTO and EXPLICIT modes. When you have some kind of XML generation task that you want to reuse, you'll have to use the good old copy/paste method, a.k.a. a maintenance nightmare. The XGen framework, which is discussed in the following section, tries to solve these problems.

The XGen solution: XSQL

As mentioned in the introduction, XGen is an open-source framework for generating XML from any data source. XSQL uses the XGen interfaces and base classes to build so-called taghandlers for tags that generate XML from any relational database that supports OLEDB.

Both XGen and XSQL are based on the .NET framework, written in C#, and open-source so you can see how all the different taghandlers are implemented. If there is no taghandler available for your specific needs it's easy to extend the existing classes to add any type of XML generation functionality you might need. This extensibility is definitely one of the strong points of the XGen framework and something that wouldn't be possible in the 'old' ASP/VB-COM world.

Before I continue with the description of some of the technical aspects of XGen and XSQL, it's important to clarify the relationship between the XGen XSQL and Oracle's XSQL. It's fair to say that Oracle XSQL was the inspiration for building the XGen framework and XGen XSQL. After building a very data-intensive application in record time using Oracle XSQL, I decided the .NET platform needed a similar tool for quickly building XML driven applications. Because Oracle XSQL is a closed-source application and is based on a different platform (Java, Servlets, & JDBC), the code base for both applications has very little in common. Also, Oracle XSQL is very much targeted towards the Oracle 8i/9i databases and its PL/SQL language, offering several Oracle-specific tags. Although Oracle XSQL is essentially database-agnostic Oracle doesn't go to great lengths to emphasize this, choosing instead to focus squarely on using XSQL with an Oracle database, which is understandable from their point of view.

XGen XSQL on the other hand offers some tags with advanced relational-XML logic, and since it is based on the XGen framework it offers better extensibility. Furthermore, XGen and XGen XSQL take full advantage of the features the .NET platform offers, including state of the art XML processing and data access capabilities as well as the revolutionary ASP.NET platform. In short, Oracle XSQL and XGen XSQL share their name and a common purpose, namely generating XML from relational databases, but not much else. If you want more information about Oracle XSQL, you can download the Oracle XDK for Java (http://otn.oracle.com/tech/xml/xdk_java/content.html?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq), read this extensive introduction into XSQL (http://download-east.oracle.com/otndoc/oracle9i/901_doc/appdev.901/a88894/adx10xsq.htm?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq), or view the release notes (http://otn.oracle.com/docs/tech/xml/xdk_java/doc_library/Production9i/java/xsql/readme.html?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq) for the latest version, which also includes some examples. You have to become an OTN member (http://otn.oracle.com/membership/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq) before you can access the links.

The XGen XML Generation Framework

Since XSQL is built on top of Xgen, it's important to have a basic understanding of what XGen does. If you want more information about XGen's structure, you will find the documentation at http://www.codeworks.nl/xml/xgen/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq. XGen can be found in the Codeworks.XGen namespace and is made up of three parts:

  • The XGen processor, to process XGen files into XML. Basically all the processor does is walk through the XGen XML file and call a few methods from the handler class associated with the tag or processing instruction in the XGen configuration.
  • Interfaces that a handler must implement so that the XGen processor can call it (either an ITagHandler or an IpiHandler, which is a handler for a processing instruction).
  • XML, like DecodeLexicalParams, which allows you to place a parameter in your XML that is resolved at runtime from an ExecutionContext dependant collection of parameters. For example, when an XSQL page is called through IIS (using an ASP.NET HttpHandler, the ASP.NET alternative to ISAPI applications) the parameters collection contains values from the Session, Cookie, Form and QueryString collections respectively. The XGen syntax for parameters is as follows: SELECT * FROM Authors WHERE id={@id}; The id parameter is inserted at runtime, so when you post a form with an id field containing the value 20, the query executed at runtime would be: SELECT * FROM Authors WHERE id=20; It would be very cumbersome if every handler would have to implement its own parameter parser, that's why it's easier to extend TagHandler, which includes a method to decode parameters, than to implement ITagHandler, which, being an interface, can only contain method and property signatures.

As you can see, the basic XGen architecture is very simple. The XGen processor can be called from any .NET class by simply calling one of the XGenProcessor's overloaded ProcessToXXX methods. The Codeworks.XGen.Web namespace contains the classes to use XGen with ASP.NET: an ASP.NET HttpHandlerFactory, an HttpHandler (not to be confused with an XGen tag- or processing instruction handler) and an XGen specific implementation of the well known System.Web.UI.Page class that adds a Content property through which you can access the processed XGen page. By using the classes in the Codeworks.XGen.Web namespace you can use ASP.NET pages as a stylesheet for your XGen pages. I'll be showing you how to do this in the sample application at the end of the article.

Most of the complexity of generating XML is left to the handlers. In the next section we'll take a look at these handlers by examining an example from XGen XSQL.

XGen XSQL: Generating XML from relational database

XGen XSQL is nothing more than a collection of classes that extend TagHandler and generate XML from SQL statements. The simplest example is the query taghandler whose tag at first glance looks a lot like the SQL Server 2000 template example. In its most simple form an XML file with XSQL tags looks like this:

<results>
<xsql:query xmlns:xsql="www.codeworks.nl/XGen/XSQL/1.0" connection="pubs">
SELECT * FROM Authors;
</xsql:query>
</results>

As you can see, the XML is nearly identical to the SQL Server 2000 template example. There are really only two noteworthy differences:

  • A connection attribute was added. An XSQL Query tag requires either a connection or a connection string attribute identifying which database it should use to execute the query. The connection attribute specified here has the value "pubs", which is linked to a connection string in the XGen configuration file (web.config in a web setting).
  • There is no FOR XML clause in the SELECT statement. Since XSQL is database-agnostic it doesn't use database specific XML generating features like SQL Server's FOR XML. Of course if want to make use of SQL Server's FOR XML feature, you can always write a taghandler that does this yourself.

After processing the XML file, the XGenProcessor returns the following XML:

<results>
<rowset>
<row id="1">
<firstname>Stephen</firstname>
<lastname>King</lastname>
</row>
<row id="2">
<firstname>William</firstname>
<lastname>Shakespeare</lastname>
</row>
</rowset>
</results>

The result is formatted differently than the SQL Server template example: columns are mapped to individual nodes instead of to attributes, the row tag has an id attribute, and a rowset element was added. If this was all XSQL did, it would offer exactly the same functionality as SQL Server 2000 does with only slightly different results.where is the extra power?

First of all, the XSQL Query tag offers an extensive set of attributes that you can use to modify the XML output. To name but a few:

  • The columns-as-attributes attribute allows you to define a comma-separated list of columns that should be added to the row element as attributes instead of as extra nodes, which is the default.
  • The page-size attribute, together with the page attribute, allows you to page through the database results easily.
  • The rowset and row attributes allow you to define the names of the rowset and row elements in the output.
  • The tag-case attribute allows you to set the case for all elements and attributes in the output.

These attributes allow you to format your output XML in a lot of ways and thus allow greater flexibility.

Secondly, if the XSQL Query tag doesn't offer the kind of XML formatting functionality you are looking for, it's easy to build your own TagHandler extension that does. Most of the time all you do is extend the XGen TagHandler base class and override one or both of the methods that that base class offers. Those methods are Initialize and HandleTag, which are both pretty self- explanatory. In the follow-up article I'll walk you through the steps needed to build your own taghandler by looking at one of the standard taghandlers in XSQL.

XSQL Examples

We've seen the simplest XSQL tag in action already, but there are two more advanced tags that really show the power of the XGen framework and XSQL.

A simple example of the hierarchical-query-tag:

<xsql:hierarchical-query xmlns:xsql="www.codeworks.nl/XGen/XSQL/1.0"
connection="pubs">
<xsql:parentquery>
SELECT * FROM Authors;
<xsql:childquery parent-join-column="au_id"
child-join-column="au_id">
SELECT * FROM Titles
</xsql:childquery>
</xsql:parentquery>
</xsql:hierarchical-query>

This tag offers the ability to easily build any level of nesting into your XML output. The hierarchical-query tag offers many of the same formatting attributes as the query tag, including the ability to define which columns to show as attributes. These attributes can be set individually on each level of nesting, providing optimal flexibility.

The second, more advanced tag is my personal favorite because it shows you how a relational database, seen by many XML proponents as too rigidly structured compared to XML, can store information in a hierarchical manner with virtually no limitations, just like XML!

The so-called infinite-hierarchy query has a simple syntax:

<xsql:infinite-hierarchy
            xmlns:xsql="www.codeworks.nl/XGen/XSQL/1.0"
connection="pubs"
parent-column="section_id"
child-column="parent_section"
columns-as-attributes="*"
child-rowset-element="">
SELECT
            * FROM Sections;
</xsql:infinite-hierarchy>

Producing the following XML:

<?xml version="1.0" encoding="utf8" ?>
<rowset>
<row id="1" Name="XML For Dummies">
<row id="2" Name="Introduction">
<row id="3" Name="What is XML?" />
<row id="3" Name="Why Use XML?" />
</row>
<row id="3" Name="XML Syntax">
<row id="4" Name="Element" />
<row id="4" Name="Attribute" />
<row id="4" Name="Comments" />
<row id="4" Name="Processing instruction">
<row id="5" Name="Common processing instructions">
<row id="6" Name="xmlstylesheet" />
</row>
</row>
<row id="6" Name="Entities" />
</row>
</row>
</rowset>

So how does it work? In its most basic form, the query has to contain two columns that contain the parent and the child identifiers. The columns that contain these values must be specified in the XSQL tag. The parent column can be any unique value, most of the time it will be an auto-increment integer column. The child column contains the unique value of its parent row, or NULL if it is a root node. The Access table that this example uses adheres to these simple rules. It contains all sections of a book. Some books will only have chapters, others will have a more complex structure, with paragraphs, etc. The table is included in the sample pubs.mdb database and looks like this:

Using ADO.NET, it's possible to easily build an infinite hierarchy: a hierarchy with arbitrary depths. Some examples of real-world applications of an infinite hierarchy are a filesystem, a threaded message board, or a website structure. If you stick to the simple rules that a parent column must be unique and that the child column contains the unique value of its parent row, then you can easily create XML using the infinite-hierarchy-query taghandler. Of course, there are other ways to create an infinite hierarchy in a relational database, if you want to define other rules for building an infinite hierarchy you can build your own TagHandler extension.

Presentation Options

After XGen has generated your XML, you may want to format it in a certain way for presentation. XGen offers a simple xml-stylesheet processing instruction handler that can transform the generated XML with an XSLT stylesheet. By writing custom processing instruction handlers you can format the results generated by XGen in any way you want. The Codeworks.XGen.Web namespace contains a special processing instruction handler that offers the aforementioned XSLT transformation capabilities, plus the ability to use ASP.NET pages as XML stylesheets. When a visitor requests an XGen page containing the following stylesheet declaration:

<?xml-stylesheet type="asp.net" href="Webform1.aspx" ?>

The XGen page is generated and the ASP.NET page is called to display the results. By extending the CodeBehind page from Codeworks.XGen.Web.UI.AspDotNet.Page instead of System.Web.UI.Page you gain access to a Content property that contains the XGen processing result in the form of an XmlDocument. Also included in the Codeworks.XGen.Web namespace is a set of XSLT-like webcontrols that allow you to easily access the Content property from an ASP.NET page and a TagHandler extension that generates XML containing the request parameters from the Querystring, Form, Cookie and Session collections.

The following code-fragment from an ASP.NET CodeBehind page shows how you can access the contents of the underlying XGen page's result through the Content property:

public class AspStylesheetDemo : Codeworks.XGen.Web.UI.AspDotNet.Page
{
protected System.Web.UI.WebControls.TextBox textBox;

public aspstylesheet()
{
Page.Init += new System.EventHandler(Page_Init);
}

private void Page_Load(object sender, System.EventArgs e)
{
this.textbox.Text = Content.SelectSingleNode("//rowset/row[@id = '1']/name/text()").Value;
}

This powerful feature allows you to completely separate data from presentation. You could use XSLT to transform the data on a page for use in pure XML applications like Web Services and use the ASP.NET stylesheets for presentations in HTML, allowing you to use the full power of the ASP.NET framework, including WebControls for reusable UI components and CodeBehind for a good separation between logic and presentation.

The following very simple example from an ASP.NET page shows you how you can use the webcontrols in the Codeworks.XGen.Web.UI.AspDotNet.Xslt namespace to quickly build data driven websites using XGen and ASP.NET:

<%@ Register TagPrefix="xslt" Namespace="Codeworks.XGen.Web.UI.AspDotNet.Xslt"
Assembly="Codeworks.XGen.Web" %>
<html>
<body>
<table>
<xslt:foreach select="//rowset/row" runat="server">
<tr>
<xslt:foreach select="/" runat="server">
<td>
< xslt:valueof select="."/>
</td>
</xslt:foreach>
</tr>
</xslt:foreach>
</table>
</body>
</html>

As you can see, the combination of ASP.NET and XSLT-like Webcontrols gives you a powerful tool for building very dynamic web pages that can access data from any data source. Webcontrols for conditional logic are also available (xsl:if, xsl:choose, xsl:when and xsl:otherwise). Of course, it's also possible to use the 'traditional' ASP.NET programming model with an ASP.NET page and a CodeBehind class. You could call the XGenProcessor from the CodeBehind class and process an XGen page there, the resulting XmlDocument contains the same XML as the Content property would when you an ASP.NET stylesheet. Again the keywords are flexibility, extensibility, and ease of use.

An XGen XSQL sample application

The sample application contains two sample files each for all three current XSQL taghandlers : a simple version with the bare minimum attributes and a complex version that uses some of the available attributes to format the results. The sample application is based on a modified version of the 'pubs' Access database, which is also included. It also contains examples of how to use the xml-stylesheet processing instruction with an XSLT stylesheet and an ASP.NET stylesheet. For installation instructions see the included readme.txt file. The compiled XGen and XSQL dlls are also included in the download, if you want access to the source files go to http://www.codeworks.nl/xml/xgen?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq.

The sample was written for and compiled on the final release of the .NET framework (http://msdn.microsoft.com/downloads/default.asp?url=/downloads/sample.asp?url=/msdn-files/027/000/976/msdncompositedoc.xml&frame=true&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq) but since you have access to all the sources, it shouldn't be too difficult to recompile the different projects to make the framework compile on the Beta 2 and RC 1 releases of the .NET framework.

Conclusion

I have tried to give you a pretty thorough introduction to XGen and XSQL. I hope the introduction has made you curious enough to try it out. If you think this article was a little light on in-depth code examples, I can assure you this will be generously compensated in the follow-up article, where I will discuss the actual code in XGen and XSQL. Some of the topics I'll discuss in that article are:

  • Writing custom ASP.NET handlers
  • Writing custom configuration section handlers
  • How the Xslt Webcontrols work
  • How to write good documentation using C#'s built in XML documentation tags and how to convert the generated XML into MSDN-library like documentation using the open-source NDOC program.
  • How to write a custom taghandler. If you have any ideas on what kind of taghandler is missing from XSQL at the moment let me know, maybe I'll build it for the upcoming article.
  • The reason for providing separate OLEDB and SQL Server versions of all standard taghandlers and how you can easily build your own database specific versions of the base taghandlers by following a specific inheritance pattern.

As said before, the entire project is open-source. Codeworks believes in the power of the XGen framework, and is dedicated to improving and maintaining it for the future. If you are interested in helping us or if you have any questions or comments, please don't hesitate to contact us (paullooijmans@codeworks.nl), we look forward to hearing from you. You can find all the latest information, documentation and software on XGen and XSQL at http://www.codeworks.nl/xml/xgen?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq.

Article Information
Author | Paul Looijmans
Technical Editor | Adam Ryland
Author Agent | Charlotte Smith
Project Manager | Helen Cuthill
Reviewers | Andrew Krowczyk, Iikka Meriläinen

If you have any questions or comments about this article, please contact the technical editor.

C#Today - © 2002 WroxPress Limited


Remoting in C# - Part 1

Remoting in C# - Part 1
http://www.csharptoday.com/content.asp?id=1798
Article Rating: 4.5
Published on January 15, 2002
Abstract
In large modern applications, business logic and database access is often distributed into multiple tiers of components on separate machines, to take advantage of distributed processing, load balancing, and scalability performance gains. This requires some form of distributed communication technology. Previous technologies that accomplished this task have been Remote Procedures Calls, CORBA, and Distributed COM. The .NET Framework contains a very robust model for distributing objects across process and machine boundaries, called Remoting, and in this article, Zach Greenvoss explores the fundamental concepts used to create distributed objects, host them, and utilize them in a client application. In the next articles we will cover more advanced remoting topics like remoting events, custom channel sinks, and advanced lease management for client–activated objects.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

In modern applications, business logic and database access is often distributed into multiple tiers of components. In small applications these tiers can exist on a single machine, however as applications scale up it becomes increasingly necessary to partition each tier on separate machines to take advantage of distributed processing, load balancing, and scalability performance gains. This requires some form of distributed communication technology. Previous technologies that accomplished this task have been Remote Procedures Calls, CORBA, and Distributed COM. The .NET Framework contains a very robust model for distributing objects across process and machine boundaries, called Remoting, and we will explore the fundamental concepts used to create distributed objects, host them, and utilize them in a client application.

Remoting and How it Differs to Web Services

.NET remoting is a huge topic, and we are only going to cover the basics of how to create and call a distributed object in this article. In a future article we will discuss more advanced topics such as advanced lease management, remoting events, and custom channel sinks. Remoting is used whenever code in different Application Domains must communicate with each other. This can be two separate applications running on the same machine, or two objects communicating between physical tiers in a distributed system. As you will see, the entire remoting system can be configured through XML files, thus enabling easy maintenance and quick changes to the deployment model.

A lot of discussion has centered around Web Services and how they facilitate communication over standard Internet protocols like HTTP and SOAP. Remoting is not a replacement for Web Services: they each have their place in a well designed .NET solution; understanding when to use each will be important in designing .NET applications. Web Services utilize HTTP and SOAP to expose functionality available to any client connected to the Internet. They use a single-call methodology that inherently retains no state. In contrast, remoting is a very rich protocol that can expose objects over HTTP or TCP/IP using either SOAP or Binary encoding for the messages sent between client and server. Remoting is also very extensible, and can be customized to suit your specific application's needs.

Remoting is a great technology to use within the firewall and between tiers in a .NET application (note remoting works in the .NET world only). When interacting with external systems, leveraging the standards that Web Services provide is crucial. However, when using internal components where speed and performance are paramount, remoting with Binary encoding over TCP/IP is probably the best choice. We also have the option of using remoting over HTTP with SOAP encoding, thereby creating an interoperable Web Services-like solution. Again, remember that Remoting can expose components in a rich object-oriented fashion, where Web Services are strictly a stateless single-call technology. There are many options available, and understanding each .NET technology is crucial to choosing the best solution for your project.

Remoting Architecture

The diagram below illustrates the remoting architecture. .NET applications are partitioned into Application Domains, which roughly translate to process boundaries. Remoting is used to communicate across the Application Domain boundaries - whether they are local to the machine or across the network. The server Application Domain hosts the remote object and listens for requests from clients. The client utilizes a proxy object that looks and acts like the actual object, but all calls on this object are forwarded to the server.

Channels are used to connect Application Domains, and most of the work required for remoting is performed within the channel. The .NET Framework ships with two Channel objects: the HTTPChannel and the TCPChannel. Each communicates over the network using a different protocol, depending on the channel type. When a client makes a call on a remote object, the proxy object intercepts the method call and forwards it to the channel. A formatter is used to serialize the message into an appropriate encoding. This could be SOAP or Binary encoding, or it could be a custom designed formatting scheme. An optional number of channel sink objects can also be attached to a channel. Channel sinks are objects that process remoting messages as they pass through a channel. These are a point of developer customization, and logging or filtering channel sinks can be created. The channel then sends the encoded message physically over the network. Depending on the type of channel, this could utilize the HTTP or TCP/IP protocol. Once the message has arrived at the server it passes through a set of server channel sink objects and finally through another formatter to deserialize the message. Then the remoting system interprets the message and calls the correct object method. The return values from this are then sent back along the same channel in the reverse direction.

High-Level Remoting Architecture

Remoting enables you to build an object model that can be distributed across tiers inside or outside your organization. Because of the flexibility it offers, you can have objects exposed internally via TCP/IP channels, and use external objects across the Internet through HTTP channels. Remoting allows you to create a loosely coupled architecture that retains the business logic on the server while still providing client applications with rich functionality and object-oriented development paradigms.

System Requirements

  • .NET Framework Beta 2
  • Visual Studio.NET Beta 2
  • Windows 2000 or Windows XP

Creating a Remotable Object

Building an object that can be called from remote clients is relatively simple. By default, the .NET Framework serializes objects by value. This means that when an object is invoked, it is copied to the local Application Domain and used within the same process. Therefore, instead of running on the server or back-end process, the object is now executing within the client's Application Domain. To enable remoting and to create a distributed object this behavior must be overridden, and to do this an object must inherit from System.MarshalByRefObject. This places the remoting plumbing in place that enables the object to be invoked by a remote proxy and communicate over a remoting channel.

Let's actually create an object that we can call remotely. Our object will contain a method that returns the current Application Domain's name. This will enable us to easily tell if the application is being hosted in a remote server or if it is being used locally. Create a new C# Class Library project in Visual Studio .NET and call it RemoteObjectLib (all projects can be found in the code download for this article).

Add the following code to the class. This is the complete listing of the code required for our remote object.

using System;

namespace
            RemoteObjectLib
{
public class RemoteObject :
            MarshalByRefObject
{
public string GetData()
{
return
            AppDomain.CurrentDomain.FriendlyName;
}
}
}

Note the object inherits from MarshalByRefObject. This is all that is necessary to mark a class as remotable in the .NET Framework. The only method in our object simply returns the name of the current Application Domain it is running within. When we build our client application, we will see that the Application Domain this remote object is running in is different to the client's Application Domain.

Hosting a Remote Object

The remote object must execute inside some form of hosting application. This application runs on the remote server and processes requests from clients for hosted objects. As we will see, the remoting subsystem takes care of almost all these details for the developer.

A developer has many options when building remoting solutions. In addition to the three different types of hosting applications we will discuss next, the type of object activation and remoting protocols used can be customized. This flexibility makes .NET remoting a very powerful technology, but it also makes it very confusing to learn initially. We will walk through the hosting options available, as well as the activation type and protocol options. We will cover each of these in depth, and hopefully clarify the options available with remoting.

1. Custom Application

There are three options to choose from when hosting a remote object. One of these is to build a custom .NET application that will run on the remote machine acting as an object server. This application can be a very simple console based application that must be started manually or it could be a more sophisticated Windows Service that will start automatically and host the remote objects. The .NET Framework enables the creation of Window Service applications very easily, and this task is not as daunting as it might have been in the pre-.NET days. In an enterprise solution this would be the preferred choice. However, we are going to build a simple console server for brevity. I leave it as an exercise for the reader to incorporate the lessons learned in this article into a Windows Service remoting host.

Create a new C# Console client called RemoteServer.

We have to reference the following namespace in the Class1.cs file:

using System.Runtime.Remoting;

This is the namespace for the Remoting API, and it contains a large number of objects and additional namespaces. We need to reference the namespace to use the RemotingConfiguration object later.

Add the following code to the Main method so the complete code looks like this:

using System;
using
            System.Runtime.Remoting;

namespace RemoteServer
{
class
            RemoteServer
{
[STAThread]
static void Main(string[]
            args)
{
Console.WriteLine("Registering
            objects.");

try
{
RemotingConfiguration.Configure("Config.xml");
}
catch(Exception
            ex)
{
Console.WriteLine("Error registering
            Remoting.");
Console.WriteLine(ex.ToString());
Console.ReadLine();
return;
}

Console.WriteLine("Objects
            registered, server
            started...");
Console.ReadLine();

}
}
}

As you can see there really is not much code required for this application. The only line that really does anything interesting is the call to RemotingConfiguration.Configure(). This uses a configuration file to configure the settings for the remoting system. We are going to cover remoting configuration files in detail later in this article, for now just know they provide a way to easily configure the remoting system and register the hosting application to listen for client requests on exposed objects. Of course, we cannot run the application successfully until we have created the configuration file, but since we don't have a client yet anyway this isn't too much of a problem.

All this application does is register the remoting objects in the configuration file, and then wait for requests from clients. Note that there is no code for processing client requests - this is all handled by the .NET Framework and the remoting system. This application simply waits for a user to press enter by using the Console.ReadLine() method.

2. Internet Information Services

The other option available to host remoting objects is IIS. This Web server can manage HTTP requests to remote objects, saving us the problem of creating a hosting application ourselves. In addition, all of the security options and benefits bundled with IIS are available to use for our hosted objects. One important caveat is that objects hosted within IIS can communicate only via the HTTPChannel, the TCP/IP protocol cannot be used. We will cover hosting objects in IIS in a future article, including applying authorization and authentication security to remoting.

3. Component Services

Component Services can also be used to host a remote object. By hosting an object in Component Services we gain access to all of COM+'s features, including transaction management, JIT activation and queued components. These are all available for objects in the .NET Framework, but to use them involves a slightly different process than a simple remote object. To enable access to the COM+ features, an object must derive from ServicedComponent. Once this requirement has been met, various attributes can be applied to the methods and properties to facilitate the Component Services features. As this article is primarily concerned with .NET remoting, we will leave Serviced Components to another article.

Activation Type

The next design decision that must be made is how the server objects will be activated. Two options are available: server-activated objects, or client-activated objects.

  • Server activated objects are controlled by the server application hosting the remote objects. This means the server manages the lifetime of the object and determines when it can be garbage collected.
  • Client activated objects are more like traditional COM components, where the client itself is responsible for the lifetime of the object. .NET does not use reference counting for this, instead a sophisticated lease system exists to control when client-activated objects are no longer in use and can be safely garbage collected.

To complicate matters further, server-activated objects can be one of two types. An object can be a Singleton, which means every call from every client is received and processed by a single object. The other option is SingleCall, which means every call from a client is received and processed by a separate and unique instance of the object. Note that these two options are only available for server-activated objects. The following diagram illustrates how method calls from clients are processed by the two different server-activated activation options.

If you are designing and implementing server-activated objects, care must be taken to avoid keeping state between object method calls. A SingleCall object will not retain any state in the object, as a new object is created every time a request is received. Perhaps more dangerous are Singleton objects, since they will retain state information, but because multiple clients access the same object it is not guaranteed that any data saved to the object is correct. If classic object-oriented state must be maintained in the objects, client-activated objects must be used. Server-activated objects should be used when scalability is paramount, as they utilize fewer resources than client-activated objects.

As mentioned previously, client-activated objects work very much like classic COM components. A client obtains a proxy to an object, and the server maintains that unique object for the client until the client tells the server it no longer requires it. Instead of reference counting, .NET utilizes a time based lease management system that is more efficient and less prone to error. A lease is basically a time period that the server will keep this object alive for this particular client. Every method call on the object will reset the lease time, and before destroying the object the server will poll the client, asking if the object can be garbage collected. This way the client can ensure that the objects it needs remain active on the server. However, if the client is disconnected the server will eventually destroy the objects on its own, thus eliminating a common memory-leak problem with server-utilized resources in previous technologies.

The prime difference between client-activated objects and server-activated objects is that the client-activated objects can safely retain state between method calls. This more closely resembles traditional object-oriented programming, but may not always be the most efficient use of resources. Server-activated objects offer a bigger benefit in terms of performance and scalability, while the client-activated objects provide the ease of use and stateful model many developers are familiar with. As always, the tradeoffs must be weighed, as no single solution will work for every project.

In terms of the code, nothing needs to change in the remote object itself to expose it as either server-activated or client-activated. The only difference is in the hosting application when the object is registered with the remoting subsystem. We will examine how to do this later when we examine remoting configuration files.

Channels and Formatters

Channels are a very important component in the remoting architecture, and the type of channel used is an important design decision. Recall that the channel is the medium of communication between Application Domains and across machine boundaries. All requests and data pass through a channel between the client and the server hosting application. Because of this, the channel objects are based on network protocols that determine how the data will travel over a network between the client and server. The .NET Framework contains two types of channels: one based on the HTTP protocol and one based on raw TCP/IP socket communication.

A design decision must be made to determine if performance is more significant than interoperability. The HTTP channel can send and receive data over any HTTP compliant network, including the Internet and across most firewalls. The TCP/IP channel uses a proprietary protocol that may not be available on all networks, and the ports required may not be open on all firewalls. However, the TCP/IP channel is significantly faster than HTTP, and for internal-only components it is probably an excellent choice.

Channels contain a series of channel sinks, which process the messages passing through the channel. These channel sinks are arranged serially, and when one processes a message, it passes the message to the next in line. The first and last channel sinks also perform special processes. The first is called a formatter, and it encodes the message into a specific format before sending it along the channel. The last channel sink is a transporter, which sends the message along the network to the remote hosting application where the server channel receives it and begins to process it through its sequence of channel sinks. Developers implement channel sinks as a way to plug functionality into the remoting architecture. Message logging, filtering, and encryption are all possible with custom channel sinks.

The channel type determines the transporter object, since an HTTP channel will utilize a HTTP protocol object to send message, while a TCP/IP channel will utilize a TCP/IP protocol object. However, the formatter object is completely independent of the channel. Two formatters are shipped with the .NET Framework: a SOAP formatter and a Binary formatter. Note that this means that a SOAP formatter does not have to be used with an HTTP channel, and a Binary formatter does not have to be used with the TCP/IP channel, although this is the default configuration.

The formatter object serializes messages into a specific encoding. The SOAP formatter creates valid 1.1 SOAP method invocation requests. The Binary formatter creates a proprietary RPC-like invocation packet. This packet is significantly smaller than the SOAP message, and can travel faster over the wire, while the SOAP message can be sent to any system capable of interpreting SOAP 1.1 messages. Again, a design decision must be made between interoperability and performance.

In the next section we will examine configuration files in detail, and we will examine how to expose a remote object via the HTTP or TCP/IP channels with either SOAP or Binary serialization. Also note that the entire Remoting infrastructure, like much of the .NET Framework, is customizable. New channel sinks, formatters, and even channels can be created by developers. This allows us to build a custom distributed object infrastructure, while relying on the .NET remoting system for as little or as much of the gritty details as we desire. It really is a very powerful and flexible system.

XML Configuration Files

While .NET remoting can be programmatically configured, this is not a solution that works well for most situations. As projects grow and requirements change, applications need to be flexible and maintainable. By hardcoding remoting configuration information into the code when a change is made, the code must be recompiled every time - not acceptable in an enterprise solution. A far better solution is to use a remoting configuration file. We have touched on this a few times before now, and we are finally going to cover it in detail.

A remoting configuration file is needed in two places: the server and the client. On the server it registers the remotable object with the remoting system, and configures the correct channel to listen for client requests. On the client this file performs a similar task, except that it registers the object as a proxy, therefore the code it creates is a configured object with a proxy reference to the server. The format of the two configuration files is slightly different, but obviously very similar. These files must be deployed with the application, but can be included as part of the general application's configuration file.

The configuration file contains all the various design decisions we have discussed so far. It is where the activation type, channel, and formatter are specified for the remote object. We register this configuration file with the remoting system by calling RemotingConfiguration.Configure() with the name of the configuration file, as we saw in our console hosting application. An application hosted in IIS does not call this method. Instead the remoting configuration data merely needs to be included as part of the Web.config file.

Now let's get down to actually creating the configuration file. Create a new text file and save it as Config.xml. Note that this file can be called anything, just pass in the correct file name to the call to RemotingConfiguration.Configure(). Add the following six lines to the file:

<configuration>
<system.runtime.remoting>
<application>
</application>
</system.runtime.remoting>
</configuration>

These tags will always exists in the remoting configuration file regardless of other options. This simply specifies the beginning of the remoting section. Note that there are a host of other options available in the configuration schema; please see the .NET Framework documentation for the complete details.

Inside the application tag we are going to place the remotable objects that this application exposes. We create the server configuration file first - we will see in the next section how to create the client configuration file. Add a <service> tag within the <application> tag. This will contain all objects this application exposes. We now have two choices, depending on the activation type of the object.

Server-Activated

The following is the complete <service> section for a server-activated object. The <wellknown> tag is used to denote a server-activated object type. Three mandatory attributes must be included inside this tag. The type attribute specifies the object type to be remoted. It is in the format of Namespace.ObjectName, AssemblyName. Don't forget to include the assembly name after the comma! The objectUri attribute specifies the object's Uniform Resource Identifier. Clients use this when they make requests for this object. The standard practice is to have this be the object name plus .rem or .soap depending on the formatting type (.rem for Binary, .soap for SOAP). The mode attribute can either be the value Singleton or SingleCall. This determines what type of server activation is used with this object. Recall that SingleCall objects are newly created for every method call from a client, while a Singleton object is shared across all client requests and only one object ever exists.

<service>
<wellknown type="RemoteObjectLib.RemoteObject, RemoteObjectLib"
objectUri="RemoteObject.rem"
mode="Singleton"/>
</service>

Client-Activated

The following is the same section for a client-activated object. Note that the tag used in this case is <activated> and the only attribute used is type. This attribute must include the name of the object exposed as well as the assembly name this object is located in.

<service>
<activated type="RemoteObjectLib.RemoteObject, RemoteObjectLib"/>
</service>

Channels

We must now configure the type of channel and formatter used by the hosting application. This can be very simple if you want to use one of the standard configurations, or it can be a bit more complex and involved if you want to develop your own remoting components. We will be using the <channels> tag, which will be added inside the <application> tag. This defines channels to be used with this remoting application. You can specify channels in other places as templates, however this is beyond the scope of this article.

There can be several <channel> tags within the <channels> tag. Each specifies a channel this server application will use to handle remoting requests. Note this means your objects will be available to clients using any listed channel type. This way an internet application can connect to the object over an HTTP Channel, while intranet applications can connect over a TCP/IP channel.

Unless you are designing a custom channel object, most of the configuration files you create will reference one of the existing channel objects, either the TCP/IP Channel or the HTTP Channel. To reference these channels, use the ref attribute in the <channel> tag, using "tcp " or "http " respectively. In addition, a port attribute must be declared to specify on which port to listen for remoting requests. The following illustrates how to configure remoting to listen for request on both TCP/IP and HTTP channels. This section would be included inside the <application> tag.

<channels>
<channel ref="tcp"
            port="6000"/>
<channel ref="http"
            port="8080"/>
</channels>

By default, the TCP/IP channel utilizes the Binary formatter, and the HTTP channel utilizes the SOAP formatter. If this is the behavior you desire, nothing extra needs to be set in the configuration file to achieve this. However, if you would like Binary serialization over HTTP, or SOAP serialization over TCP/IP, a few extra settings must be set.

The <serverProviders> tag can be included within the <channel> tag. This tag is used to define the channel sink providers, which are simply objects that provide access to a channel sink. Recall that a channel can contain any number of optional channel sinks, which can enforce various business rules or logging functionality. In addition to these, which must be created by a developer as none are shipped with the .NET Framework, a single formatter must be specified. This is a special channel sink that appears first in the channel sink chain and formats the message into a specific format. Again, as everything is customizable in the remoting system, it is even possible to build your own formatter sinks to customize the format of .NET messages.

The formatter sink is declared with the <formatter> tag as illustrated below. The ref attribute references a previously defined (or intrinsic) formatter sink. To reference the SOAP formatter, use the "soap " id, and to reference the Binary formatter, use the "binary " id. Again, if you are building and deploying a custom formatter sink then this tag will include type and assembly information for this custom object, but since we are simply referencing the intrinsic formatters all we need is the ref attribute.

<channels>
<channel ref="http" port="8080">
<serverProviders>
<formatter ref="binary"/>
</serverProviders>
</channel>
</channels>

This example section declares an HTTP channel, but uses the binary formatter sink. Again remember that this is the server configuration file; there will be some changes for the client file. The following example illustrates how to use a TCP/IP channel with a SOAP formatter:

<channels>
<channel ref="tcp" port="6000">
<serverProviders>
<formatter ref="soap"/>
</serverProviders>
</channel>
</channels>

Note that any number of <provider> tags can also exist under the <serverProvider> tag. It is with these tags that we define the channel sink chain, and the order they are declared under the <serverProvider> is the order messages pass through them. Again, this is a point of customization to add functionality into the .NET remoting system as messages pass through.

We are now ready to put it all together and build our complete configuration file. As there are a number of different options we can choose, we will only show a single configuration file. However, in the code download for this article can be found a series of configuration files that illustrate all the options covered so far. These should provide an excellent starting point for building your own remoting applications.

<configuration>
<system.runtime.remoting>
<application>
<channels>
<channel ref="tcp" port="6000">
<serverProviders>
<formatter ref="soap"/>
</serverProviders>
</channel>
</channels>
<service>
<wellknown type="RemoteObjectLib.RemoteObject, RemoteObjectLib"
objectUri="RemoteObject.soap"
mode="Singleton"/>
</service>
</application>
</system.runtime.remoting>
</configuration>

Save this file as Config.xml into the same directory as the RemoteServer application. Note that it must exist in the RemoteServer/bin/debug directory, because this is where the application actually runs from. The RemoteObjectLib.dll file also needs to be copied into this same directory. If this is not done the hosting application will not be able to load the correct object, unless the assembly has been placed into the Global Assembly Cache. Try running the RemoteServer application to ensure no exceptions are raised. If everything works correctly, the following screens should appear:

Client Settings

In the next section we will build a simple client to consume the remote object, but first let's look at how to build the client configuration file. This file is used by the client application to register the objects via a proxy, the location of the remote objects, and what channels and formatters to use. This file is very similar to the server version, but there are a few key differences.

In a client configuration file the <service> tag is replaced with a <client> tag, representing the objects that this application will consume remotely. Note that both a <service> and a <client> tag can exist within a configuration file if an application both consumes remote objects as well as exposes objects itself. Inside of the <client> tag can exist <wellknown> tags for server-activated objects, and <activated> tags for client-activated objects in a similar fashion to the server configuration file. The type of activation in the client configuration file must match what has been specified on the server for an object.

If you are using server-activated objects, two attributes must be included within the <wellknown> tag. These are url and type. The url attribute is the complete Uniform Resource Locator address of this object. This must contain the protocol type, server machine name, remote port, and object URI in the format: ProtocolType://ServerName:PortNumber/ObjectURI. The protocol type will be either tcp or http, depending on the type of channel specified. Both the server name and port number must be specified. The object URI is declared in the server configuration file as the objectUri. The type attribute defines the remote object using the same format as the server configuration file: Namespace.ObjectName, AssemblyName. The .NET Framework uses this information to intercept any requests for this object type and forwards them to the local proxy, which in turn pushes messages to the correct remoting channel. The following is an example of a <client> section of a configuration file for our previously exposed object:

<client>
<wellknown type="RemoteObjectLib.RemoteObject, RemoteObjectLib"
url="tcp://localhost:6000/RemoteObject.soap"/>
</client>

If we are consuming a client-activated remote object, the configuration file will be slightly different. Instead of the <wellknown> tag the <activated> tag is used to declare remote objects. Within this tag only a single attribute is available: the type attribute. This again must declare the object type and assembly name. In addition, within the <client> tag, the additional attribute url must be added to reference the remote server. This looks much like the url we discussed previously, however without any object-specific information. An example of a complete <client> section for our remote object is shown below, if it was exposed as a client-activated object.

<client url="tcp://localhost:6000">
<activated type="RemoteObjectLib.RemoteObject, RemoteObjectLib"/>
</client>

Client Channels

Clients must also register what channels they will use to communicate with the server. This is done in exactly the same manner as the server configuration settings with one exception. If custom channel sinks are used they must be declared with the <clientProviders> tag instead of the <serverProviders> tag. The options available are the same, but there must be a distinction between server channel sinks and client channel sinks.

One option that may be used on the client is to specify zero for a port number. This tells the client to use any available port on the client machine to communicate with the server. Servers will most likely never use this setting because clients must have a well-known port to connect to. However clients that only need to make requests on servers do not need to have a hardcoded port number. Often they will simply declare the type of channel they will use and specify zero for the port number to utilize the next available port.

That's it - we can now build our complete client configuration file to communicate with our remote object. The complete file is shown below. Again, due to the many options available this is only one possible configuration. This particular file references the server-activated object that the previous server configuration file exposes. In the download for this article there are a number of other templates to use for client configuration files in various situations. Save this file as ConfigClient.xml; we will use it in the next section.

<configuration>
<system.runtime.remoting>
<application>
<channels>
<channel ref="tcp" port="0">
<clientProviders>
<formatter ref="soap"/>
</clientProviders>
</channel>
</channels>
<client>
<wellknown type="RemoteObjectLib.RemoteObject, RemoteObjectLib"
url="tcp://localhost:6000/RemoteObject.soap"/>
</client>
</application>
</system.runtime.remoting>
</configuration>

Consuming a Remote Object

Now that we have a client configuration file, we can now build a client and see this remoting business in action. One of the requirements of a good distributed object system is that it must be simple for clients to use remote objects. Fortunately, when a remoting configuration file is in place, objects can be created and used in exactly the same manner as standard local objects.

Create a new C# console project in Visual Studio.NET titled RemoteClient.

Add the following code, and don't forget the namespace declarations at the top!

using System;
using System.Runtime.Remoting;
using RemoteObjectLib;

namespace RemoteClient
{
class RemoteClient
{
[STAThread]
static void Main(string[] args)
{
Console.WriteLine("Registering Remoting.");

try
{
RemotingConfiguration.Configure("ConfigClient.xml");
}
catch(Exception ex)
{
Console.WriteLine("Error registering Remoting.");
Console.WriteLine(ex.ToString());
}

Console.WriteLine("Current AppDomain: {0}", AppDomain.CurrentDomain.FriendlyName);

//Create remote object
RemoteObject aRemoteObject = new RemoteObject();
Console.WriteLine("Remote Object's AppDomain: {0}",aRemoteObject.GetData());

//Wait for user to press enter before quitting
Console.ReadLine();
}
}
}

To get this to work we need to add the RemoteObjectLib assembly to this project. Do this by selecting Project->Add Reference. Select Browse and navigate to the RemoteObjectLib.dll file in the RemoteObject/bin/Debug directory. The assembly will be added to the client application, but only as a type library. Remote objects themselves will not run within the client's application domain. This is demonstrated by the output of this application, as the remote object will echo out the Application Domain it is running within. Nonetheless, this illustrates an important issue with .NET remoting: the client application must have the remote assembly. In a future article, I will discuss how to design remoting objects using interfaces so an "empty" assembly can be distributed to the clients and all the functionality can be maintained on the server.

Back to the code itself, the first thing happening is that it registers the client configuration file. Copy the ClientConfig.xml file we created earlier into the RemoteClient/bin/Debug directory with the executable. The application will echo out its current Application Domain name. The application then creates a RemoteObject, which is created remotely on the server, and calls the GetData() method. This returns the Application Domain of the remote object, which will be different to the local client application.

Note that we created the remote object with the new keyword, and other than the call to RemotingConfiguration.Configure(), everything looks like a standard application.

We are now ready to see this system in action. First start up the hosting application. As we have developed the configuration files to reference the local machine, I am assuming this means simply starting the hosting application. If you wish to test this more realistically, simply replace 'localhost' with the name of the remote server in the client configuration file and run the hosting application on the sever. Don't forget the server configuration file and the RemoteObjectLib assembly must be in the same directory as the hosting application.

Once the server hosting application, RemoteServer.exe is started, and it has registered the remoting objects and is awaiting client requests, run RemoteClient.exe. The screen should look like the following:

This application displays the current Application Domain, and then calls the remote object to retrieve the Application Domain of the remote object. The local application is not hosting the object; it is being executed on the server.

Conclusion

.NET remoting is a large topic, and we have only covered the basics, but it is still very extensive. The techniques discussed can be used to build sophisticated distributed object architectures in .NET. Using remoting configuration files makes this an enterprise solution that can adapt quickly to change.

In this article we covered the following topics:

  • What .NET remoting is and how it works
  • What the MarshalByRefObject class is and how it relates to remoting
  • How to create a remoteable object
  • How to host a remote object in a server application
  • How to create remoting configuration files for easy management and maintenance
  • How to consume a remoting object as a client.

In the next article I will cover advanced remoting topics like remoting events, custom channel sinks, and advanced lease management for client-activated objects. Until then, have fun with .NET!

C#Today - © 2002 WroxPress Limited


Building .NET Reporting Solutions with XML and GDI+

Building .NET Reporting Solutions with XML and GDI+
http://www.csharptoday.com/content.asp?id=1796
Article Rating: 4
Published on January 9, 2002
Abstract
Customizing report solutions is far easier today with GDI+ than ever before ?and with the increased ease of use of XML, and the elegant way that data sources can be bound to web controls, we can pretty much tap into any source of data with ease. Combine the two and report solutions can be tailored very easily to suit the need of any business. In this article, Richard Conway creates a C# and ASP.NET application that takes an XML data source and, based on user choices, auomatically creates a bar chart, pie graph, or line chart of the user selected data.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

Whilst the number of changes between Visual Studio 6 and Visual Studio .NET are vast, so are the number of new applications of Visual Studio .NET. One area that has always been the domain of a specialist is graphics programming - well, with GDI+ many of the repugnant graphics API calls have been made redundant. This example brings together a whole heap of .NET technologies and allows us to create graphs on the fly from an XML file.

Customizing report solutions is far easier now in .NET. With the increased ease of use of XML, and the elegant way that data sources can be bound to web controls, we can pretty much tap into any source of data with ease. With GDI+ we have the ability to create charts and graphs with precision plotting without much hassle. Combine the two and report solutions can be tailored very easily to suit the need of any business.

This article consists of a core C# dll, an ASP.NET UI and an XML data source as a backing store for the report. There are many ways to retrieve and use the XML. One common way is a use of the System.Xml namespace. A more elegant solution however is to use attributes and a class hierarchy that matches our XML node hierarchy. XML Serialization is one of the truly remarkable features of the CLR. We can start by introducing a tool called xsd.exe, which can be found in the bin directory of the .NET Framework SDK.

It's worthwhile remembering at this point that the use of XML files is not exclusive. I have chosen it since it is the most robust format for retrieving data. I have used a prescribed XML format, which would need to be followed (as you'll see below). I could, for example, instead of using a file, use a dataset and shape the XML to the prescribed format, or I could use a SQL Server stored procedure and (in version 2000) return a resultset shaping the output to something similar to that which will be presented below.

Using Xml

Here is our sample block of XML data, on which we will report:

Report.xml

<REPORT>
<STATISTICS UNITS="%" YEAR="1998">
<VALUE>2.4</VALUE>
<VALUE>0.1</VALUE>
<VALUE>9.6</VALUE>
<VALUE>0.2</VALUE>
<VALUE>2.1</VALUE>
<VALUE>0.1</VALUE>
<VALUE>0.3</VALUE>
<VALUE>0.4</VALUE>
<VALUE>2.0</VALUE>
<VALUE>1.0</VALUE>
<VALUE>1.1</VALUE>
<VALUE>0.9</VALUE>
</STATISTICS>
.
</REPORT>

This is a simple block of XML, which takes 12 values corresponding to months in the year - we will model our custom server control around these monthly values. Run xsd.exe in the command line:

xsd C:\report.xml

This will produce an .xsd file in the C:\ root directory. Further use of the tool will create a set of classes, which can be populated, serialized, and deserialized directly from the XML document. As follows:

xsd /c C:\report.xsd

Try opening this file in Visual Studio .NET. The results are remarkable - what you'll see is effectively a database schema. In effect this is what the XSD file is. It's a schema (or ruleset) for the XML block we've shown above. Visual Studio .NET displays this in a tabulated form, allowing us to immediately see the relationships between discrete parts of XML (in the same way that a database will allow us to see the primary and foreign key constraints of a table).

The XSD file represents the description (schema) of the XML file - it can be modified to allow the XML to have certain constraints in terms of data types and the way that elements are sequenced within the XML document. Unfortunately the code produced through the Beta 2 version of this tool is not always optimal, but we can use the output of C:\report.cs as a starting point for our project (the RC1 version of this tool corrects many of the problems that were present in previous version).

The following code was produced via the XSD tool:

[System.Xml.Serialization.XmlRootAttribute("REPORT", Namespace="", IsNullable=false)]
public class REPORT
{

/// <remarks/>
[System.Xml.Serialization.XmlElementAttribute("STATISTICS")]
public REPORTSTATISTICS[] Items;
}

This is the top-level class, which contains collections of other classes. It represents the start of an object hierarchy, which is consistent with the parent-child relationship of the XML. The attributes that accompany the code are used to serialize and de-serialize the XML from and to these classes. XmlSerialization classes have been written within the .NET Framework to take advantage of attributes and reflection so these classes will undergo introspection at runtime in order to populate these classes. Unfortunately I don't have enough space to delve into the finer points of XML, but Professional C# (http://www.amazon.com/exec/obidos/ASIN/1861004990/ref=nosim/csharptoday-20/107-5604225-4012523?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq) by Wrox Press provides some fantastic examples in Chapter 15.

/// <remarks/>
public class REPORTSTATISTICS
{

/// <remarks/>
[System.Xml.Serialization.XmlElementAttribute("VALUE", IsNullable=true)]
public REPORTSTATISTICSVALUE[] VALUE;

/// <remarks/>
[System.Xml.Serialization.XmlAttributeAttribute()]
public string UNITS;

/// <remarks/>
[System.Xml.Serialization.XmlAttributeAttribute()]
public string YEAR;
}

/// <remarks/>
public class REPORTSTATISTICSVALUE
{

/// <remarks/>
[System.Xml.Serialization.XmlTextAttribute()]
public string Value;
}
}

This marks the hierarchy that we discussed. In effect the object model looks like the XML structure. We start to see how tightly coupled XML is to the .NET Framework.

Reporting on a Wizard

Step 1

Fig 1 - Step 1 of the Report Wizard

Step one of this Report Wizard uses a simple file upload to get the data from a local file on a client PC to the web server. I can browse to the file, and when selected press continue.

//Read the file contents into a stream
StreamReader stReadReportFile = new StreamReader(Request.Files[0].InputStream, Encoding.UTF8);
//insert into the cache - easiest way to store for short amount of time
Session.Add("fileContents", stReadReportFile.ReadToEnd());
//Let's goto the next wizard page
Server.Transfer("ReportType.aspx");

The above code executes when we press the Continue button. It reads in the file from the ASP.NET Request object which each page (and anything in the context of the Web) has access to. It reads the file in as stream and into a StreamReader object. This is then changed into a string and stored as a Session variable. I, and probably many others reading this, had a great fear of using SessionState in previous versions of ASP. This was most notably because business systems are built on web farms, which can contain more than one web server. A Session is held in memory, which would mean that it would need to be "sticky" to be able to use with confidence, or another means of session management would have to be used with a shared database as a backing store. Well, there is no need to fear the use of SessionState anymore, as SQL Server 2000 can be intrinsically set in ASP.NET to store Session information allowing complete scalability. It's beyond the scope of this article but has an obvious entry in web.config, which must be configured properly. The final statement is a transfer to another page. I prefer using these to Response.Redirect() in ASP as they don't involve a client-server interchange (to get the client to download a different page - other than the requested one).

Step 2

If we consider the page_load event on the next page:

EnableViewState = false;
// Put user code to initialize the page here
//Create the reportextract object and later return the collection that we've created
String entireReport = Convert.ToString(Session["fileContents"]);
ReportExtract reGenXml = new ReportExtract(entireReport);
//Populate a list box with all the years so that one can be chosen
foreach(REPORTSTATISTICS rsPresentYear in reGenXml.ReportStatisticsCollection)
{
lstDropYearList.Items.Add(rsPresentYear.YEAR);
}
//Better to replace the session variable with an object now to save further lookup work
Session.Remove("fileContents");
//Re-add it as the object value
Session.Add("fileContents", reGenXml);

So, from the top down, the page first turns off viewstate since we don't want to remember any page state between requests. This would inadvertently add duplicate entries into the DropDownListBox. We convert the report contents to a string from a session variable (as it is cast to object by default) and create a ReportExtract object to create an object model representation of the XML. We'll look at the deserialization code that it produces a little later.

We can enumerate the collection of REPORTSTATISTICS objects stored in the ReportExtract object and add their YEAR property to the DropDownListBox control which will hold all the textual values of the years. Once this is done, we can remove the contents of the Session variable "fileContents " and reinsert the variable but with a different object - this time a ReportExtract object which has a copy of the REPORT object as it's member variable. When this page loads again, the same code will execute so the ReportExtract class must override the ToString() method of the object. This is interesting as it serializes the XML back in exactly the same form it was before we used the XML element-attribute values to populate the fields of our REPORT object hierarchy.

The constructor for the ReportExtract class contains the following:

//Load the xml into a reader again!! and then deserialize contents into REPORT object
XmlSerializer xsReportDetails = new XmlSerializer(typeof(REPORT));
m_ReportObject = (REPORT)xsReportDetails.Deserialize(new StringReader(xml));

This is a very simple means of moving between a series of objects, which contain collections of other objects and XML. An XmlSerializer object is created (which takes a type argument) and an XML string is deserialized to allow a REPORT object to be created and have its initial state defined by the contents of this document. I'm going to skip a great deal of code throughout this project (for brevity) so I'll focus on the things, which are most important. I would encourage everybody to look at the code distributed with the download to understand how this project works in its entirety.

The ToString() method I mentioned before is overridden as follows to allow a string cast from an object to serialize the object REPORT back to XML.

public override string
            ToString()
{
//Create the familiar serialization
            object
XmlSerializer xsReportDetails = new
            XmlSerializer(typeof(REPORT));
//Create a mem stream to
            temporarily hold it
MemoryStream memReportDetails = new
            MemoryStream();
//Serialize the repomemReportDetailsrt from the
            member variable
xsReportDetails.Serialize(memReportDetails,
            m_ReportObject);
//Read in the stream
memReportDetails.Seek(0, SeekOrigin.Begin);
//use overriden
            method
StreamReader srReportDetails = new
            StreamReader(memReportDetails);
//Return all text
return
            srReportDetails.ReadToEnd();
}

This should have a familiar ring to it now. We create the same XML Serialization object and further create a MemoryStream to hold the contents of the Serialized XML. The contents of the member variable referencing the REPORT object are then placed into the MemoryStream.

We have to reset the Stream pointer back to the beginning in order to read the contents (which is done through StreamReader). The Contents are then returned from the ToString() method in the form of a string.

Fig 2 - This show the second page of the wizard

We ask the report user which graphs he/she wants to generate and for which years. I modeled a file reportfile2.xml, which I've included with the download. It reads the years from this file and displays them on screen (in the DropDownListBox). We have the option of generating a Text Table, a Bar Chart, a Pie Chart, a Line Graph and a Multi-Value line graph, which can be used to compare values over all the years listed in the list box (at least those for which we have data). To finalize this screen we'll be asked what to call our report.

REPORTSTATISTICS rsOurReport = null;
string directoryName = String.Empty;
//Here we can actually generate the report
ReportExtract rsColReportVals = (ReportExtract)Session["fileContents"];
foreach(REPORTSTATISTICS rsChosenReport in rsColReportVals.ReportStatisticsCollection)
{
//Lets find a match if(lstDropYearList.SelectedItem.Text.Equals(rsChosenReport.YEAR))
{
//Add the Create the serverreport directory

When the Continue button is clicked the above code is executed. This will enumerate the REPORT object collection which is a member variable of ReportExtract. This will try to find a year match with the selected year (from the list box).

Directory.CreateDirectory(@"C:\inetpub\wwwroot\ReportDirectory\"+txtReportName.Text+@"\"+rsChosenReport.YEAR+"\\");
directoryName = @"C:\inetpub\wwwroot\ReportDirectory\"+txtReportName.Text+@"\"+rsChosenReport.YEAR+"\\";

When this match is found, a directory is created as above with a path of the web directory followed by the report name and the year. The files are stored under this for accessibility.

ReportGenerator rgReportStats = new ReportGenerator(rsOurReport, chkLineGraph.Checked,
chkBarChart.Checked, chkTable.Checked, chkPieChart.Checked, chkMultiLineGraph.Checked,
rsColReportVals.ReportStatisticsCollection, directoryName);

Finally, a report generation object is created which will aggregate all the decisions made so far and create graphing objects, which use GDI+. We will look at these now.

Creating the Graphs

The Bar Graph

The following creates the bar graph by calling the CreateBarGraph() method, which is invoked on creation of a BarGraph object in the ReportGenerator like so:

BarGraph bgReportGraph = new BarGraph(directory, maxValue, rpReportSolution);

The BarGraph inherits from a class called GenericGraph (as do all other graphs in this project). GenericGraph defines a set of common fields and methods which all graphs use, including heights and offsets (as described below).

CreateBarGraph() is described below - it is responsible for creating the bar graph and applying to it styles and spacing to define what kind of graphic emerges.

Bitmap bmp = new Bitmap((int)gWidth+OFFSET_GRAPHWIDTH,
(int)gHeight+OFFSET_GRAPHHEIGHT,
PixelFormat.Format32bppArgb);
Graphics g = Graphics.FromImage(bmp);
g.FillRegion(Brushes.White, new Region(new Rectangle(0, 0,
(int)gWidth+OFFSET_GRAPHWIDTH,
(int)gHeight+OFFSET_GRAPHHEIGHT)));

The above code demonstrates what little effort is required to create a graphic using GDI+. We create a Bitmap graphic with width and height defined by properties that can be set as attributes on the server control tag (this will be described later). Here we've defined that we also want to create a 32 bit graphic. A Graphics object is created using the static FromImage(bmp) method. This gives us a handle to the underlying graphic object - in essence it will allow us to add all types of shapes and color effects. OFFSET_GRAPHWIDTH and HEIGHT represent the spacing needed to add text to describe the graph axes. The FillRegion() method allows us to choose a brush color and define coordinates which can be used to paint a background color on the graphic. In this case we use white just to be consistent with the web page background color. All of the work is done in the following segment of code.

for(float i = 0; i < totalCount;
            i++)
{
if (i == 0) initWidth = (float)((i + 1) / totalCount)
            * gWidth;

At this point we are looping through the values that we've just put into the table. On the first iteration we calculate the width of each bar by working out each bar as a proportion of the total width of the graph.

float x = (float)((((i + 1) / totalCount) * gWidth) -
(float)(1 / totalCount) * gWidth) + OFFSET_GRAPHWIDTH;

Above is the calculation for the x coordinate of the current bar. The x coordinate is read from the left corner of the graphic - here we have to work out what the accumulated width of all the bars is and then add the offset value, which we set earlier (to allow space for x-axis text descriptions).

float height = (float)(((allValues[(int)i] / maxValue) * gHeight));
float y = (float)(gHeight - height) + OFFSET_GRAPHHEIGHT;

The y value coordinate is somewhat more difficult to calculate. It begins from the top of the graphic and we have to subtract the height of the bar from the total height (and then add the offset to provide space to write text at the top of the graph). The height is calculated by taking the value as a proportion of the previously calculated maximum value (this is the value that we set as the scale maximum i.e. 100% of the height of the graph) - from this value we can work out the height of the bar.

   RectangleF rect = new
            RectangleF(x, y, initWidth,
            height);
g.FillRectangle(Brushes.OliveDrab,
            rect);
g.DrawRectangle(new Pen(Color.Black), rect.X, rect.Y,
            rect.Width, rect.Height);
}

This last piece of code above ends the loop by actually drawing a rectangle on the screen in the place of each bar. A Rectangle object is created using the x & y values calculated before. I've chosen to use an OliveDrab brush to color fill the rectangle, but typically you can use any number of different colors defined as Brushes class constants. The outline of the Rectangle is drawn in black to make the bar stand out on the white background.

private void CreateAxis(float width, float height, Graphics g, string heightToolText)
{
g.DrawLine(new Pen(Color.Black, 2), new Point(OFFSET_GRAPHWIDTH, (int)height+OFFSET_GRAPHHEIGHT),
new Point((int)width+OFFSET_GRAPHWIDTH, (int)height+OFFSET_GRAPHHEIGHT));
g.DrawLine(new Pen(Color.Black, 2), new Point(OFFSET_GRAPHWIDTH, OFFSET_GRAPHHEIGHT),
new Point(OFFSET_GRAPHWIDTH, (int)height+OFFSET_GRAPHHEIGHT));
g.DrawString(heightToolText, new Font("tahoma", 10), Brushes.Black, 0, 10);
g.DrawString(maxValue.ToString(), new Font("tahoma", 10), Brushes.Black, 0, OFFSET_GRAPHHEIGHT+10);
g.DrawString((maxValue/2).ToString(), new Font("tahoma", 10), Brushes.Black, 0, (height/2)+OFFSET_GRAPHHEIGHT);
g.DrawString("0", new Font("tahoma", 10), Brushes.Black, 0, height);
}

The CreateAxis method is called within the previous method using the invocation CreateAxis (gWidth, gHeight, g, "%age"). The width, height, Graphic object and the y-axis text are passed to it. From this information we can draw the graph x and y axis lines by using a black pen, with the width and height having the offset values we discussed previously. In addition it adds the text ' %age ' (in this case) to the top of the graph. The DrawLine() method (of the Graphic object) takes a Pen object argument and two points to be able to effectively draw the line. The DrawString() method takes a string value to write to the graphic and x and y coordinate (it also takes a font object which we've set as 10pt Tahoma). The last 3 lines of code add the uppermost value of the graph, the middle value (half the size of the upper most value) and zero, which defines the lowest value. The code could be modified to allow a negative value to be the lowest - negative values are unnecessary, however, if statistics are adjusted accordingly to compensate (it would also require a little more work to create rectangle bars which are drawn and painted under the x axis as opposed to above).

The bmp is then saved as follows:

graphName = "bargraph.jpg";
bmp.Save(directory+graphName, ImageFormat.Jpeg);

Fig 3 - Bar Graph of XML statistics

The Pie Chart

The pie chart is somewhat easier to model than the bar graph. The CreatePieChart() method does most of the work delegating to a series of private methods, which will be described in detail.

Bitmap bmp = new Bitmap((int)gHeight + 300, (int)gHeight, PixelFormat.Format32bppArgb);
Graphics g = Graphics.FromImage(bmp);
g.FillRegion(Brushes.White, new Region(new Rectangle(0, 0, (int)gHeight+300, (int)gHeight)));

The pie chart is constructed initially like the bar graph. We define a greater offset however (of 300) so that the legend can fit onto graphic. The height is used in this case to define the height and the width, as ideally we want a circular rather than elliptical pie chart.

for(float i = 0; i < totalCount;
            i++)
{
PieBrushes allColour = new
            PieBrushes();
FillArc((allValues[(int)i] / totalAddition), g,
            allColour.getBrush((int)i));
}

We won't explain the PieBrushes struct here, but suffice it to say that it contains 12 colors, which present each month so that each arc on the pie chart can be identified distinctly (the code for the struct can be found in the code downloads section of this article). Calling the FillArc method fills the arc, and we pass it as parameters the proportion of the total circle to fill, the Graphic object, and a unique colored brush. FillArc is shown below:

private void FillArc(float section, Graphics
            g, Brush br)
{
g.FillPie(br, 0, 0, gHeight, gHeight,
            lastAngle, section * 360);
lastAngle += section *
            360;
}

This works out a section as a proportion of the whole circle and plots it within the graphic. The lastAngle variable contains the last angle that the previous pie reached from 0 to 360 degrees. This is used to plot the arc.

Finally we draw the circular border in black pen around the series of connected arcs.

g.DrawEllipse(Pens.Black, 0, 0, gHeight, gHeight);

In the same way as you might find through Microsoft Graph (which is most commonly known through charting with the Excel wizards), I decided that it would be better to create a legend for the pie chart instead of writing on each slice of pie. The legend represents a small block of color (5 pixels by 5 pixels) to the month with which the colored arc represents.

private void CreateLegend(Graphics
            g)
{
PieMonths p = new PieMonths();
PieBrushes allColour =
            new PieBrushes();
g.DrawRectangle(Pens.Black, gHeight + 100, 0,
            100, (gHeight / 3)*2);
for(int i = 0; i < totalCount;
            i++)
{
g.FillRectangle(allColour.getBrush(i), gHeight + 120,
            (i * 20) + 8, 5, 5);
g.DrawString(p.getMonths(i), new
            Font("tahoma", 10), Brushes.Black, gHeight + 130, i *
            20);
}
}

A rectangle is drawn first to allow the legend to be encapsulated in a black border. Then we can get all the values from the PieMonths struct, which contains the month names and the values from the PieBrushes struct. We can then use FillRectangle to fill the 5 by 5 colored rectangle and the DrawString method to write the string name of the month next to the colored rectangle.

This is then saved in the same way as the last graphic and a corresponding HTML image object is created (which we will discuss later).

Fig 4 - Result of using Brushes and colors for a Pie Chart

The Line Graphs

public class LineGraphMulti : LineGraphSingle
{
public LineGraphMulti(ArrayList Reports, string directory, float maxValue,
REPORTSTATISTICS rpStatsVal) : base(directory, maxValue, rpStatsVal)
{
//Create a bitmap graphic which will set the height and width
Bitmap bmp = new Bitmap((int)gWidth+OFFSET_GRAPHWIDTH, (int)gHeight+OFFSET_GRAPHHEIGHT,
PixelFormat.Format32bppArgb);
Graphics g = Graphics.FromImage(bmp);
//Fill the region with rectangle white so that we can define a chart area
g.FillRegion(Brushes.White, new Region(new Rectangle(0, 0, (int)gWidth+OFFSET_GRAPHWIDTH,
(int)gHeight+OFFSET_GRAPHHEIGHT)));

graphName = "multilinegraph.jpg";
int i = 0;

Rather than refer to both line graphs (since their code is very similar) I've decided to just refer to the Multi-Value line graph. This graph is created and has the LineGraphSingle class as its base class (which in turn has GenericGraph). As in all other graphs mentioned thus far, we create a Graphics object and use this to generate the graph, whilst using the graphic to create a jpeg from a Bitmap object.

   foreach(REPORTSTATISTICS
            rpSingleUnit in Reports)
{
this.rpStatsVal =
            rpSingleUnit;
LinePens lpPensCollection = new
            LinePens();
CreateLine(g,
            lpPensCollection.getPen(i));
i++;
}
CreateAxis(gWidth,
            gHeight, g, "%age");
bmp.Save(directory+graphName,
            ImageFormat.Jpeg);
}

LinePens is another struct somewhat similar to PieBrushes. With this we are returned a variety of pens to enable us to select a new pen color based on index from 1 to 12. Here we create the graph axis in the same way as before and save the graphic. The only real variation is the CreateLine method, which is shown below:

public new void CreateLine(Graphics g, Pen
            penColor)
{
float x = (float)(((1 / totalCount) * gWidth) -
            (float)(1 / totalCount) * gWidth) + OFFSET_GRAPHWIDTH;
float
            height = (float)(((float.Parse(rpStatsVal.VALUE[0].Value) /
            maxValue) * gHeight));
float y = (float)(gHeight - height) +
            OFFSET_GRAPHHEIGHT;
Point lastPoint = new Point((int)x,
            (int)y);
//Loop through the total number of values
for(float
            i = 1; i < totalCount; i++)
{
x = (float)((((i + 1) /
            totalCount) * gWidth) - (float)(1 / totalCount) * gWidth) +
            OFFSET_GRAPHWIDTH;
height =
            (float)(((float.Parse(rpStatsVal.VALUE[(int)i].Value) / maxValue) *
            gHeight));
y = (float)(gHeight - height) +
            OFFSET_GRAPHHEIGHT;
Point point = new Point((int)x,
            (int)y);
g.DrawLine(penColor, lastPoint, point);
lastPoint =
            point;
}
}

We've come across the calculations already when discussing bar charts, so won't delve into this again. However, in this line graph example we use Point objects to obtain a point either side of the line and then call DrawLine() on the Graphics object with the information. The lastPoint is recorded in a local variable so that it can be used to plot the first line point in the call to DrawLine(). This method is called as many times (via the LineGraphMulti object) as there are REPORTSTATISTICS objects. Further inspection of the code will reveal this.

Fig 5 - Multiple line graph

Now we've looked at the graphs individually it is time to see how this is all tied together. The majority of the "glue" code - as discussed - is in the ReportGenerator object. It is this that feeds our choices from the web and converts them into useful graphs.

maxValue = 0;
for(int j = 0; j <
            ReportList.Count; j++)
{
for(int i = 0; i <
            ((REPORTSTATISTICS)ReportList[j]).VALUE.Length; i++)
{
//Work
            out the maximum value here using Max function
maxValue =
            Math.Max(float.Parse(((REPORTSTATISTICS)ReportList[j]).VALUE[i].Value),
            maxValue);
}
}
LineGraphMulti mlReportGraph = new
            LineGraphMulti(ReportList, directory, maxValue,
            rpReportSolution);

This code works out the maximum value of the all of the lines in a multiline graph. Even if this was a single line graph, bar graph or pie chart. Since the values we are using are proportional to the largest value, and the y-axis on the graph is scalable, we will need to know the maximum value from our sample of values. The Max function in the Math class is a static method. This is a method that is common to virtually all languages. It simply compares two numbers and returns a copy of the larger. With the multi-line graph this must be tested on all the numbers from the REPORTSTATISTICS collection.

html.Append("<image src=\"");
html.Append(mlReportGraph.graphName);
html.Append("\" border=\"0\"><br>");

With every graph that is created and saved as an image, a StringBuilder class is appended too, so that we have an HTML file containing all the images generated, and an HTML table with all the values in.

Fig 6 - the page index.html showing the table

The HTML is then created as:

html.Append("</body></html>");
FileStream
            fwReportFile = new FileStream(directory+"index.html",
            FileMode.Create, FileAccess.Write);
StreamWriter swReportFile =
            new
            StreamWriter(fwReportFile);
swReportFile.Write(html.ToString());
swReportFile.Close();

using a FileStream object and a StreamWriter object to write the string from the StringBuilder object into a file.

Sometimes it may be necessary to generate reports dynamically and stream them to the browser. Rather than saving them in a physical file we could use the above code to create a MemoryStream and stream the image to the browser. This makes for a quick and fast "on-the-fly" customized image.

<%MemoryStream ms = new MemoryStream();
Bitmap.Save(ms, ImageFormat.Jpeg);

Response.ContentType = "image/jpeg";
Response.BinaryWrite(oStream.ToArray());%>

Conclusion

In this example, I have tried to automate a simple task that many of us have needed to do for our employers. Sometimes we have been unable to this due to time constraints and have had to buy in third party products such as ChartFX. However, many of the new features of Visual Studio .NET enable us to fairly easily create custom solutions like the one just described. One way of extending this would be to use the article on Add-Ins (http://www.csharptoday.com/content.asp?id=1619&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq) to create a wizard enabling us to bind to a data source and develop charts based on any data source. A step further may be to introspect an instance of SQL server to create graphical representations of current data, and use OLAP to provide data to create graphs that could represent future trends.

If you'd like to discuss a broader range of C#Today topics and issues, or if you have any C# related questions you need answers to, please visit our online resource centres at : http://p2p.wrox.com/list.asp?list=csharptoday_discuss&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq and http://p2p.wrox.com/list.asp?list=c_sharp&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq
C#Today - © 2002 WroxPress Limited


Working with XML as an Object Model

Working with XML as an Object Model
http://www.csharptoday.com/content.asp?id=1718
Article Rating: 3.5
Published on December 13, 2001
Abstract
In this article, Salim Naim explores XML, and how we can work with it as an object model. He shows that XML is truly a core technology layer in .NET, while manipulating and working with XML documents with .NET is a natural method built into the .NET Framework, and not an after thought. He covers various techniques, including how to make use of the core functionality provided by the Base Class Libraries in the .NET Framework classes relevant to XML, and also looks at the two abstract base classed XMLReader and XMLWriter, which the XML classes are built upon.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

When working with XML on a Windows platform, we currently have to resort to using Microsoft's MSXML parser, and its W3C compliant programming interfaces like the DOM or SAX for document processing. The Document Object Model (DOM) provides a standardized interpretation of XML documents to applications and scripts. While this standard interpretation works with XML document structures as program structures rather than text streams, consumers can read and manipulate these structures without a detailed understanding of the XML syntax, taking advantage of the DOM APIs built into the parser.

The Simple API of XML (SAX) is a programming interface for applications that enables an efficient method of extracting information from an XML document, without the overhead of the memory tree structure used by the DOM. SAX processes documents in a serial fashion, generating events as it moves through the document, making it memory efficient and suitable for processing large documents where we do not need to manipulate the whole document.

Using either of the above interfaces programmatically is satisfactory for many scenarios that consume/process XML. However, programming componentized solutions that generate XML data is a daunting task. Take the example of developing a DNA business layer object that generates XML data for its clients. Objects of that nature often generate XML documents using primitive string manipulation routines like wsprintf in C++, or string concatenation in VB. The process of generating XML is usually a task completely unrelated to the XML document structure or schema.

With the .NET framework and the XML schema representation of XSD, finally a generic framework is available to both generate and consume XML documents programmatically as an alternative to the standard interpretation provided by the DOM and SAX.

The techniques we discuss in this article will utilize the XML Schema representation (XSD) and the .NET Framework xsd.exe utility to generate .NET classes that wrap the hierarchal representation of an XML document, providing us with a programmable in-memory object model to navigate and manipulate the XML document against.

System Requirements:

  • .NET Framework Beta 2
  • Visual Studio .NET Beta 2

This article assumes that you are familiar with XML and C#.

XML Schema Definition Tool

The XML Schema Definition (XSD) tool (xsd.exe) is a utility that comes as part of the .NET framework. It generates XML schema or .NET classes from XDR, XML or XSD files. This utility can be found in %SystemRoot%\Program Files\Microsoft.NET\FrameworkSDK\Bin and provides many functions based on the input file extension (*.xml, *.xdr, *.xsd, file.dll). A proper explanation of xsd.exe is beyond the scope of this article. We will cover the /classes and /dataset file options for the purposes of this article, but a proper explanation of xsd.exe is beyond our scope - see http://msdn.microsoft.com/library/default.asp?url=/library/en-us/cpguidnf/html/cpconusingxmlschemadefinitiontool.asp&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq for more information.

To get started with xsd.exe and generate a .NET class based on an XSD file, the following command line must be executed:

xsd.exe <schema>.xsd /classes|dataset [/l:CS|VB|JS]

  • The first argument, <schema>.xsd, specifies the XML Schema to use when generating either the class or the dataset; this argument must be an XSD file.
  • The second argument, /classes or /dataset, is the file option to use when generating source code. This option can either be a .NET class /classes or an ADO.NET Dataset /dataset.
  • The last argument, which is optional, is the language used in the generated source code; the default language is C#:

An example command line would be:

xsd simple.xsd /classes

We will see below why this is useful and what we can use it for.

XML Schema Definition

The XML Schema Definition (XSD) is the long awaited markup language that enables us to define an XML document's structure. XSD Schema is composed of a top-level schema element that references the xsd namespace (see http://www.w3.org/2001/XMLSchema?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq). This top level element schema contains type definitions (the simpleType and complexType elements), as well as element and attribute declarations. Besides type, element, and attribute definitions, XSD provides built-in data types for defining data types used in the XML schema.

Note: Additional information on XSD and its elements can be found at XML Schema reference.

Remember our example command line?

xsd simple.xsd /classes

This will generate a C# class wrapper for an XML document based on the simple.xsd schema. See the code below:

simple.cs

using
            System.Xml.Serialization;

[System.Xml.Serialization.XmlRootAttribute(Namespace="http://tempuri.org/Simple.xsd",
            IsNullable=false)]
public class Customer
            {

[System.Xml.Serialization.XmlElementAttribute(IsNullable=false)]
public
            string
            Name;

[System.Xml.Serialization.XmlElementAttribute(IsNullable=false)]
public
            string
            Email;

[System.Xml.Serialization.XmlElementAttribute(IsNullable=false)]
public
            string Password;
}

simple.xsd

A close inspection of this class shows that the XSD Definition tool mapped the Customer element to a Customer class declaration that contains public member variables for each of the elements defined in the XML schema. See the schema below:

<?xml version="1.0" encoding="utf-8" ?>
<xsd:schema id="Simple" targetNamespace="http://tempuri.org/Simple.xsd"
elementFormDefault="qualified" xmlns="http://tempuri.org/Simple.xsd"
xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<xsd:element name="Customer">
<xsd:complexType>
<xsd:sequence>
<xsd:element name="Name" type="xsd:string" />
<xsd:element name="Email" type="xsd:string" />
<xsd:element name="Password" type="xsd:string" />
</xsd:sequence>
</xsd:complexType>
</xsd:element>
</xsd:schema>

simple.xml

This defines a Customer element that contains child elements Name, Email and Password. The Customer element is of a complexType that must contain child elements in the defined sequence. All child elements are of type xsd:string.

This schema below is the schema for simple.xml :

<?xml version="1.0" encoding="utf-16"?>
<Customer>
<Name>Salim Naim</Name>
<Email>email@salimnaim.com</Email>
<Password>secret</Password>
</Customer>

Simple Sample

With a brief understanding of XSD and the .NET Framework utility xsd.exe, we can now delve into a sample. The following sample will define an XML document schema, and then use the XSD definition tool to generate a C# wrapper class for manipulating the document.

The XSD schema we'll use is the Customer document described above. The following command will be executed against this schema to generate the C# wrapper class.

xsd /classes simple.xsd

A close inspection of the generated simple.cs class will show that the XSD Definition tool mapped the Customer element to a Customer class declaration that contains public member variables for each of the elements defined in the XML schema.

The following snippet is from the simple.cs file accompanying the article:

using
            System.Xml.Serialization;

[System.Xml.Serialization.XmlRootAttribute(Namespace="http://tempuri.org/Simple.xsd",
            IsNullable=false)]
public class Customer
            {

[System.Xml.Serialization.XmlElementAttribute(IsNullable=false)]
public
            string
            Name;

[System.Xml.Serialization.XmlElementAttribute(IsNullable=false)]
public
            string
            Email;

[System.Xml.Serialization.XmlElementAttribute(IsNullable=false)]
public
            string Password;
}

The class declaration provided above shows a Customer class with the added customization of attributes from the System.Xml.Serialization namespace.

Attributes are keywords like tagsclasses, which enable you to specify additional information about entities defined in a class. The extra metadata provided by attributes control how the XmlSerializer serializes, or deserializes, an object. The XSD Definition tool (xsd.exe) uses a wealth of metadata available in the XSD schema to generate .NET classes, and preserves this information using attributes.

Note: For additional information on attributes in .NET, check out Kaushal Sanghavi's article on Attributes in .NET (http://www.csharptoday.com/content.asp?id=1567&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq).

Common attributes available when working with classes based on XSD schema via an object model are:

  • XmlRootAttribute: This attribute is an identifier for classes, structures or interfaces and maps to XML root elements.
  • XmlElementAttribute: This attribute, as its name implies, is an identifier between class fields or properties, to XML elements.
  • XmlArrayAttribute: This attribute applies to nested sequences of XML elements. These sequence elements map as fields of type System.Array.

Working with the Wrapper Class

The wrapper class generated by the XSD Definition tool is a class that can be used both to generate XML documents based on the defined schema via serialization, and to consume and manipulate an XML document via the class fields and properties via deserialization.

Generating an XML document is implemented via serialization:

The following snippets are from Simpleclient.cs.

Customer simpleObject = new
            Customer();
simpleObject.Name = "Salim
            Naim";
simpleObject.Email =
            "email@salimnaim.com";
simpleObject.Password =
            "secret";

XmlSerializer xmlSerializer = new
            XmlSerializer(typeof(Customer));
StringWriter stringWriter = new
            StringWriter();

xmlSerializer.Serialize(stringWriter,simpleObject);
textBox1.Text
            = stringWriter.ToString();

The above snippet begins by instantiating an object instance of the Customer class, and then works with the object like a regular data structure. To produce a corresponding XML document based on the defined schema, an XmlSerializer is instantiated along with a StringWriter object to serialize the object as an XML document (the above code snippet will produce the Customer XML document described earlier).

To manipulate and work with the document as an object instance of the class, the XML document must be deserialized back to an object of type Customer:

XmlSerializer xmlSerializer = new
            XmlSerializer(typeof(Customer));
StringReader inputString = new
            StringReader(textBox1.Text);

Customer simpleObject =
            (Customer)xmlSerializer.Deserialize(inputString);
string
            outputString = String.Format("The Customer object contains:\n - Name
            : {0}\n - Email : {1}\n - Password
            :
{2}",simpleObject.Name,simpleObject.Email,simpleObject.Password);

MessageBox.Show(outputString);
MessageBox.Show(outputString);

The above snippet begins by instantiating an instance of the XmlSerializer class and uses the StringReader class to read the XML document. Deserializing the XML document to an object instance of type Customer provides a Customer class representation for working with the XML document without resorting to a parser.

Customer simpleObject = (Customer)xmlSerializer.Deserialize(inputString);

Note: Type and element constraints are all validated during the serialization and deserialization stages by the compiler, and the added metadata is made available to the XmlSerializer class via attributes. For additional information on Serialization in other aspects of .NET, be sure to check Matthew Reynolds' September article on Serialization (http://www.csharptoday.com/content.asp?id=1619&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq).

Validation

So far we have been using the XSD definition tool to generate the C# wrapper class for manipulating an XML document. When type and constraints are broken, validation becomes an important aspect when using XML Serialization / Deserialization as a method for dealing with documents.

When using object-to-XML serialization, or XML-to-object deserialization as a method for manipulating XML documents, validation becomes a topic that must be understood. The XmlSerializer class plays a vital role in bridging data between the CLR and XML with the additional information available in attributes.

Serialization

The Serialization phase is a process that relies on the language compiler for validation when serializing an object instance. Data type to schema type mismatches are resolved with the XSD to CLR data type mapping created, along with the source code from xsd.exe, and are enforced by the compiler. The remaining issue that must be recognized is element existence. The default class wrappers generated by the XSD Definition Tool (xsd.exe) have no method for validation when required elements in the schema are omitted in the object instance being serialized; this can lead to an invalid XML document.

The following snippet is a serialization sample from validationform.cs:

CartCookieType validationObject = new
            CartCookieType();
//an element value
            omitted
//validationObject.Cookie =
            "cookie:29092001-7:10";
validationObject.ExpiryDate =
            DateTime.Now;

XmlSerializer xmlSerializer = new
            XmlSerializer(typeof(CartCookieType));

The above code instantiates an invalid CartCookie XML document upon serialization.

<CartCookie>
<ExpiryDate>2001-10-20+03:00</ExpiryDate>
</CartCookie>

To resolve this, additional logic should be written to validate an object instance, insuring conformance with the document schema. An example is below:

If(validationObject.Cookie.Length > 0
            && .)
{
//Data is Valid
XmlSerializer xmlSerializer
            = new XmlSerializer(typeof(
            CartCookieType));
}
else
{
//Data is inValid, Throw an
            Exception
}

Deserialization

Deserialization is the process of transforming an XML document into an instance of the wrapper class. XmlSerializer.Deserialize() uses the additional metadata stored in the class attributes to validate the XML document against the class declaration.

During Deserialization of data types, document well formedness is validated by the XmlSerializer class. Exceptions are raised if document entities fail against the definition information available in the attributes. Raised exceptions are of type System.Exception, along with an innerException object referencing the detailed source errors.

Note: Document element name and required fields are not validated by the default XmlTextReader used by the XmlSerializer class. A solution to this is to replace the use of the standard Reader, such as StringReader, with a validating XML reader such as XmlValidatingReader prior to Deserialization.

Collections

Real world XML documents present a hierarchal representation of data. So far, the XML document instances that we have spoken about are mapped by the XSD Definition tool as structures, with the schema defined data type resolved as CLR data types. XSD allows XML documents schemas to be defined, specifying the constraints that the document instance must adhere to. XSD elements can specify restrictions on simple and complex element types; these restrictions include: value checking constraints, pattern matching constraints, and element occurrence restrictions.

Note: Value checking constraints and pattern matching constraints are restrictions available in the XSD schema, however, they are not preserved along with the generated class via attributes. This limitation will surely be addressed in future releases of the .NET framework and the System.Xml.Serialization.

Element Occurrence

Elements that exist as part of a collection (e.g. purchase order items) represent the hierarchical aspect of XML. For elements of that nature, the XML Definition tool maps these elements as an Array (System.Array) of the type representing the element.

The following schema is an example of a typical document definition utilizing XSD constraints.

The following extract is from the ShoppingCart.xsd file accompanying this article.

<xsd:schema id="ShoppingCart"
targetNamespace="http://tempuri.org/ShoppingCart.xsd" elementFormDefault="qualified"
xmlns="http://tempuri.org/ShoppingCart.xsd" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<xsd:element name="ShoppingCart" type="ShoppingCartType" />
<xsd:complexType name="ShoppingCartType">
<xsd:sequence>
<xsd:element name="CartID" type="xsd:string" />
<xsd:element name="TimeStamp" type="xsd:date" />
<xsd:element name="items" type="Items" />
</xsd:sequence>
</xsd:complexType>
<xsd:complexType name="Items">
<xsd:sequence>
<xsd:element name="item" minOccurs="0" maxOccurs="unbounded">
<xsd:complexType>
<xsd:sequence>
<xsd:element name="productName" type="xsd:string" />
<xsd:element name="quantity">
<xsd:simpleType>
<xsd:restriction base="xsd:positiveInteger">
<xsd:maxExclusive value="100" />
</xsd:restriction>
</xsd:simpleType>
</xsd:element>
<xsd:element name="Price" type="xsd:decimal" />
</xsd:sequence>
<xsd:attribute name="partNum" type="xsd:string" />
</xsd:complexType>
</xsd:element>
</xsd:sequence>
</xsd:complexType>
</xsd:schema>

The above document schema contains a single root element, ShoppingCart, which is of the complexType ShoppingCartType. The ShoppingCartType is comprised of a CartID element, TimeStamp element, and a collection of cart items. The items element is of complexType (Items). The Items type defines an item element which contains minOccur and maxOccur attributes that indicate that the element behaves as part of a collection of Items. The item element contains an occurrence constraint with a minimum occurrence of zero, and an unbound maximum occurrence. The item element defines a partNum attribute, a productName element, and a quantity element. The quantity element within the item element also contains a restriction, limiting this element's value to only positive integers with an upper limit of a 100.

Generating a class for the above schema using the XSD Definition tool will result in the following class:

The following extract is from the ShoppingCart.cs file, which accompanies this article:

using
            System.Xml.Serialization;

[System.Xml.Serialization.XmlRootAttribute("ShoppingCart",
Namespace="http://tempuri.org/ShoppingCart.xsd",
            IsNullable=false)]
public class ShoppingCartType
            {

[System.Xml.Serialization.XmlElementAttribute(IsNullable=false)]
public
            string
            CartID;

[System.Xml.Serialization.XmlElementAttribute(DataType="date",
            IsNullable=false)]
public System.DateTime
            TimeStamp;

[System.Xml.Serialization.XmlArrayAttribute(IsNullable=false)]
[System.Xml.Serialization.XmlArrayItemAttribute("item",
            IsNullable=false)]
public ItemsItem[] items;
}

public
            class ItemsItem
            {

[System.Xml.Serialization.XmlElementAttribute(IsNullable=false)]
public
            string
            productName;

[System.Xml.Serialization.XmlElementAttribute(DataType="positiveInteger",
IsNullable=false)]
public string quantity;

public
            System.Decimal
            Price;

[System.Xml.Serialization.XmlAttributeAttribute()]
public
            string partNum;
}

The source generated above provides class declarations for the two complexType elements defined in the schema. The ShoppingCartType class represents the ShoppingCart element with its three child elements CartID, TimeStamp, and items. The items field is generated by xsd.exe as an array of type ItemsItem that represents the Items complexType.

Restriction information is lost during source code generation. The quantity elements maxExclusive restriction is partly preserved via the XmlElementAttribute, however, the XSD data type positiveInteger is not correctly mapped to a suitable CLR data type, thus resulting with the default data type mapping of type string.

[System.Xml.Serialization.XmlElementAttribute(DataType="positiveInteger",
            IsNullable=false)]

public string quantity;

The following snippet is a sample of serializing a Collection document:

ShoppingCartType collectionObject = new
            ShoppingCartType();
collectionObject.CartID =
            "cookie:29092001-7:10";
collectionObject.TimeStamp =
            DateTime.Now;

ItemsItem[] itemsArray = new
            ItemsItem[1];

ItemsItem desktop = new ItemsItem();
desktop.partNum = "HW-01-001";
desktop.productName = "HP
            vectra 1GB Desktop";
desktop.quantity = "1";
desktop.Price =
            3000;

//associate the collection
itemsArray[0] =
            desktop;
collectionObject.items =
            itemsArray;

XmlSerializer xmlSerializer = new
            XmlSerializer(typeof(ShoppingCartType));
...

The above snippet is from the CollectionsForm.cs file accompanying the article.

Datasets

So far we have only been discussing the use of .NET classes as a method for manipulating defined XML documents. The XSD Definition tool provides two file options for encapsulation a defined XML document, classes and datasets.

The ADO.NET Dataset provides an in-memory relational representation of data. Hierarchical representation of data is usually represented in XML. These two representations were previously used independently of one another. The .NET Framework correlates these two representations, and provides methods for dealing with the data, using the preferred method of representation. The relationship between Dataset and XmlDataDocument offers great flexibility, providing a single set of data, access to the services around the Dataset such as Web Forms and Win Forms, as well as the suite of XML services such as XSL, XSLT and XPath.

The .NET Framework enables access to both relational and hierarchical data through the Dataset and XmlDataDocument objects. The following sample demonstrates synchronizing a Dataset with an XmlDataDocument to take advantage of the XML services such as XPath and XSLT.

The following snippet provides a hierarchical view of the existing relational data populated in the Dataset:

DataSet dataSet = new
            DataSet();
XmlDataDocument xmlDocument = new
            XmlDataDocument(dataSet);

To take advantage of the .NET Framework XML services like XSLT:

XslTransform xslTransform = new
            XslTransform();
xslTransform.Load("mystylesheet.xsl");

XmlTextWriter
            xmlWriter =
            XmlTextWriter("output.htm",System.Text.Encoding.UTF8);

xslTransform.Transform(xmlDocument,null,xmlWriter)
xmlWriter.Close();

The following snippet is an example of generating a Dataset from an XSD schema using the XSD Definition tool:

e.g. xsd DatasetSample.xsd /dataset

Dataset Sample

The following XSD schema defines a SimpleDataset element that contains three child elements: Name, Email and Password.

The following extract is from the datasetSample.xsd file accompanying the article:

<?xml version="1.0" encoding="utf-8" ?>
<xsd:schema id="DatasetSample" targetNamespace="http://tempuri.org/DatasetSample.xsd"
elementFormDefault="qualified" xmlns="http://tempuri.org/DatasetSample.xsd"
xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<xsd:element name="SimpleDataset">
<xsd:complexType>
<xsd:sequence>
<xsd:element name="Name" type="xsd:string" />
<xsd:element name="Email" type="xsd:string" />
<xsd:element name="Password" type="xsd:string" />
</xsd:sequence>
</xsd:complexType>
</xsd:element>
</xsd:schema>

Generating a dataset for the above schema produces a C# class that inherits from the System.Data.DataSet and implements the required methods.

Working with the Dataset

The following is an example of serializing an XML document from the generated Dataset class.

The following extract is from the DatasetSample.cs file accompanying the article:

DatasetSample datasetSample = new
            DatasetSample();
DataRow dataRow;

dataRow =
            datasetSample.SimpleDataset.NewRow();

dataRow["Name"] =
            "Salim Naim";
dataRow["Email"] =
            "email@salimnaim.com";
dataRow["Password"] =
            "secret";

datasetSample.SimpleDataset.Rows.Add(dataRow);

StringWriter
            stringWriter = new StringWriter();
datasetSample.WriteXml(stringWriter);

To serialize an XML document from the Dataset, data is treated as part of a relational database table. A new Row must be added to the Dataset table, and data must be entered into the row columns. The Serialization is carried out with the WriteXml method of the Dataset class, serializing the document into a StringWriter object.

To Deserialize an XML document and manipulate it as a Dataset table. The XML document must first be read into the Dataset via the ReadXml method, and information is then retrieved by navigating through the rows and columns of the table.

DatasetSample datasetSample = new DatasetSample();
StringReader stringReader = new StringReader(textBox1.Text);

datasetSample.ReadXml(stringReader);

string outputString = String.Format("The Dataset contains:\n - Name : {0}\n - Email : {1}\n - Password : {2}",
datasetSample.SimpleDataset[0].Name,
datasetSample.SimpleDataset[0].Email,
datasetSample.SimpleDataset[0].Password);

Summing it up

As we have seen, XML is truly a core technology layer in .NET, while manipulating and working with XML documents with .NET is a natural method built into the .NET Framework and not an after thought.

The techniques covered in this article make use of the core functionality provided by the Base Class Libraries in the .NET Framework classes relevant to XML. While using the classes or dataset for manipulating XML documents might raise performance concerns for some people, it's reassuring to know that the cores of .the NET Framework XML classes are built upon two abstract base classes XmlReader and XmlWriter. The XmlReader class is heavily used for Deserializing and transforming XML to various representations. This class is a compromise between the DOM and SAX APIs. The model adopted by XmlReader is a streaming, fast, forward only model, similar to SAX, with a much simpler programming interface like the DOM.

C#Today - © 2002 WroxPress Limited


SOAP Support in .NET

SOAP Support in .NET
http://www.csharptoday.com/content.asp?id=1612
Article Rating: 4
Published on July 6, 2001
Abstract
Building distributed applications has challenged programmers for many years now. Building applications that work over heterogeneous platforms is often more involved and complex. The different systems need to agree upon a common data format and representation, which introduces marshalling and serialization support. Further, since this could be running on a public network such as the internet, there can be no assumptions about the hops that the data might take before it reaches its intended destination. Protocols such as RPC, IIOP and DCOM have attempted to solve this, but they usually work well within a controlled private network. We need an extension of these protocols that is widely supported by most platforms and is consistent across heterogeneous systems. The Simple Object Access Protocol (SOAP) attempts to provide such a protocol that is platform agnostic and is open standard. In this article, Kaushal Sanghavi explores how the .NET framework integrates with and utilizes SOAP.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

Building distributed applications has challenged programmers for many years now. Building applications that work over heterogeneous platforms is often more involved and complex. The different systems need to agree upon a common data format and representation, which introduces marshalling and serialization support. Further, since this could be running on a public network such as the internet, there can be no assumptions about the hops that the data might take before it reaches its intended destination. Protocols such as RPC, IIOP and DCOM have attempted to solve this, but they usually work well within a controlled private network. We need an extension of these protocols that is widely supported by most platforms and is consistent across heterogeneous systems. The Simple Object Access Protocol (SOAP) attempts to provide such a protocol that is platform agnostic and is open standard. In this article, we will explore how the .NET framework integrates with and utilizes SOAP.

System Requirements for this article:

  • .NET Beta 2
  • SOAP 2.0 Gold release
  • Windows 2000

DCOM, IIOP, HTTP, SOAP

DCOM, IIOP and Http are examples of protocols for distributed communication that have been around for a while and are relatively mature. These protocols are widely accepted and utilized for server-server communication, and use the request-response paradigm. They use transport protocols like TCP or UDP to pass request and response packets between different servers. However, DCOM and IIOP utilize proprietary protocols for data and object serialization and for authentication services. Thus, every machine that participates in a DCOM or IIOP communication needs to support these protocols. This is where the problem lies; these protocols are platform specific and do not easily interact or integrate. Further, it is extremely difficult to support this over a public internet, since most firewalls block these protocols. Typically, these protocols are used internally within a firewall in a private network. Http, on the other hand, is much better suited for client-server and server-server based communication over a heterogeneous network. Http boasts of ubiquitous support and nearly all firewalls will allow Http traffic. Http is a simple request response protocol that is built on top of TCP and is the de facto protocol of the public internet. However, Http is too simple a protocol to be used directly for building large scale distributed applications; the main issue being that Http lacks standard formats for passing data using RPC calls. Thus, if you have a custom object that is used as a parameter to a RPC call, Http lacks support for representing the serialized object. What is needed is a standard representation for serialized data that can be used in conjunction with Http. XML is such a standard that is gaining wide popularity, and the protocol that uses XML and Http to make RPC calls over distributed public networks is SOAP.

SOAP is a simple lightweight protocol that uses Http as transport and specifies encoding rules using XML. Apart from the encoding rules, SOAP does not require nor does it assume any platform or application semantics. This allows SOAP to be a highly generic protocol that can be supported on any platform and be utilized on a wide variety of applications. A SOAP request is an Http POST request that contains three parts, the SOAP envelope, the encoding rules and the RPC representation. The SOAP envelope contains information about the message being passed, the encoding rules define the rules for dealing with custom objects and the RPC representation defines rules for representing the method requests and responses.

The following is a simple example of a SOAP method request made to a Calculator object (included in the download code), invoking the Add method. We will discuss how this works in the next section:

POST /SimpleCalc HTTP/1.1
Host: 127.0.0.1
Content-Type: text/xml
Content-Length: nnnn
SOAPMethodName: urn:strings-com:ICalculator#Add

<SOAP-ENV:Envelope
xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/"
SOAP-ENV:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"/>

<SOAP-ENV:Body>

<m:Add xmlns:m='urn:strings-com:ICalculator'>
<Operand1>12</Operand1>
<Operand2>23</Operand2>
</m:Add>

</SOAP-ENV:Body>

</SOAP-ENV:Envelope>

This is a sample of what the response would look like:

HTTP/1.1 200 OK
Content-Type: text/xml
Content-Length: nnnn

<SOAP-ENV:Envelope
xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/"
SOAP-ENV:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"/>

<SOAP-ENV:Body>

<m:AddRespose xmlns:m='urn:strings-com:ICalculator'>
<Result>35</Result>
</m:AddResponse>

</SOAP-ENV:Body>

</SOAP-ENV:Envelope>

SOAP in .NET Web Services

Now that we have covered the very basics of SOAP, let us look into how .NET utilizes SOAP. Web Services are the ideal candidates for using SOAP. A web service can be used by any client, running on any platform, since it is based upon the HTTP protocol. Thus, for web services to pass request and response parameters, SOAP is the perfect option for all of the reasons discussed in the above section. The .NET framework provides support for SOAP natively. Thus, a developer building a web service need not even be aware of the fact that the web service internally uses SOAP. Further, the .NET framework provides two tools SoapSuds.exe and WSDL.exe that can be used to generate proxy classes from a web service which generates the code required to handle the required plumbing. Let us take a look at a very simple web service that adds two integers.

namespace WebServiceSample
{
using
            System.Web.Services;

public class Calculator :
            System.Web.Services.WebService
{
[SoapMethod()]
[WebMethod]
public
            int Add (int iOp1, int iOp2)
{
return iOp1 +
            iOp2;
}

}
}

For a class to expose its services as a web service, all it needs to do is derive from the System.Web.Services.WebService class that provides all the plumbing, and then mark its methods using the WebMethod attribute. Further, since we will be using the SOAP protocol (i.e. accessing XML over Http) directly from the client to access this web service, we have to use the SoapMethod attribute as well. Since we won't be using custom namespaces, the request and response namespaces are left blank.

This is all VS.NET requires to generate the SOAP payload corresponding to the method signature. Once a web service is written, it needs to communicate information about itself, so that clients can make calls against the service. This information is the method names, the arguments and return types etc. The .NET framework generates this information automatically in the form of a Web Service Description Language (WSDL) file. The WSDL file can be viewed by passing the "?WSDL" parameter in the query string of the web browser. The WSDL file for the above webmethod is shown below:

<?xml version="1.0" encoding="utf-8"?>
<definitions
xmlns:s="http://www.w3.org/2001/XMLSchema"
xmlns:http="http://schemas.xmlsoap.org/wsdl/http/"
xmlns:mime="http://schemas.xmlsoap.org/wsdl/mime/"
xmlns:tm="http://microsoft.com/wsdl/mime/textMatching/"
xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/"
xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/"
xmlns:s0="http://tempuri.org/"
targetNamespace="http://tempuri.org/"
xmlns="http://schemas.xmlsoap.org/wsdl/">
<types>
<s:schema attributeFormDefault="qualified" elementFormDefault="qualified"
targetNamespace="http://tempuri.org/">
<s:element name="Add">
<s:complexType>
<s:sequence>
<s:element minOccurs="1" maxOccurs="1" name="iOp1"
type="s:int" />
<s:element minOccurs="1" maxOccurs="1" name="iOp2"
type="s:int" />
</s:sequence>
</s:complexType>
</s:element>
<s:element name="AddResponse">
<s:complexType>
<s:sequence>
<s:element minOccurs="1" maxOccurs="1" name="AddResult"
type="s:int" />
</s:sequence>
</s:complexType>
</s:element>
<s:element name="int" type="s:int" />
</s:schema>
</types>
<message name="AddSoapIn">
<part name="parameters" element="s0:Add" />
</message>
<message name="AddSoapOut">
<part name="parameters" element="s0:AddResponse" />
</message>
<message name="AddHttpGetIn">
<part name="iOp1" type="s:string" />
<part name="iOp2" type="s:string" />
</message>
<message name="AddHttpGetOut">
<part name="Body" element="s0:int" />
</message>
<message name="AddHttpPostIn">
<part name="iOp1" type="s:string" />
<part name="iOp2" type="s:string" />
</message>
<message name="AddHttpPostOut">
<part name="Body" element="s0:int" />
</message>
<portType name="CalculatorSoap">
<operation name="Add">
<input message="s0:AddSoapIn" />
<output message="s0:AddSoapOut" />
</operation>
</portType>
<portType name="CalculatorHttpGet">
<operation name="Add">
<input message="s0:AddHttpGetIn" />
<output message="s0:AddHttpGetOut" />
</operation>
</portType>
<portType name="CalculatorHttpPost">
<operation name="Add">
<input message="s0:AddHttpPostIn" /><
<output message="s0:AddHttpPostOut" />
</operation>
</portType>
<binding name="CalculatorSoap" type="s0:CalculatorSoap">
<soap:binding transport='http://schemas.xmlsoap.org/soap/http'
style="document" />
<operation name="Add">
<soap:operation soapAction="http://tempuri.org/Add" style="document" />
<input>
<soap:body use="literal" />
</input>
<output>
<soap:body use="literal" />
</output>
</operation>
</binding>
<binding name="CalculatorHttpGet" type="s0:CalculatorHttpGet">
<http:binding verb="GET" />
<operation name="Add">
<http:operation location="/Add" />
<input>
<http:urlEncoded />
</input>
<output>
<mime:mimeXml part="Body" />
</output>
</operation>
</binding>
<binding name="CalculatorHttpPost" type="s0:CalculatorHttpPost">
<http:binding verb="POST" />
<operation name="Add">
<http:operation location="/Add" />
<input>
<mime:content type="application/x-www-form-urlencoded" />
</input>
<output>
<mime:mimeXml part="Body" />
</output>
</operation>
</binding>
<service name="Calculator">
<port name="CalculatorSoap" binding="s0:CalculatorSoap">
<soap:address location=
"http://localhost/WebServiceSample/Calculator.asmx" />
</port>
<port name="CalculatorHttpGet" binding="s0:CalculatorHttpGet">
<http:address location=
"http://localhost/WebServiceSample/Calculator.asmx" />
</port>
<port name="CalculatorHttpPost" binding="s0:CalculatorHttpPost">
<http:address location=
"http://localhost/WebServiceSample/Calculator.asmx" />
</port>
</service>
</definitions>

Apart from the headers, the WSDL file has 5 main sections, <soap (in\out)> , <httpget> , <httppost (in/out)>, <types> and <service> . These first three sections describe the payload format for the protocols used to access this web service. For example, when the protocol is SOAP, the WSDL specifies that the request action tag will be Add and the response action tag will be. AddRespose. Further, the Soap Address attribute of the Service tag points to the location of the web service file. Similarly, the <httpget> and <httppost> sections define the forms and parameters that will carry the payload when these protocols are used. These sections allow the runtime to generate the payload in the format that is expected by the web service. The <types> section is used to define the types of the data objects that will be used as parameters to the web service calls. For example, the element names iOp1 and iOp2 correspond to a data type of int.

Now that we know what the SOAP calls should look like, let us try to write a simple client application in C# that will attempt to call this web service. This client will create the payload and use HTTP POST to post it to the Calculator webservice. Further, it will parse the response to obtain the results of the add operation. Microsoft provides a utility called the SOAP Toolkit which provides most of the plumbing required to construct and interpret SOAP payloads. We will use this utility to create our sample C# client that will access the Calculator web service. You can download the latest version of this toolkit (gold release 2.0 at the time of publication) from http://msdn.microsoft.com/code/sample.asp?url=/msdn-files/027/001/580/msdncompositedoc.xml&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq.

using System;
using MSSOAPLib;
using
            MSXML2;

public class CalcClient
{
public int
            InvokeSoapAdd (int iOp1, int iOp2)
{
SoapSerializer
            lSerializer;
SoapReader lReader = null;
HttpConnector
            lConnector = new
            HttpConnector();

lConnector.set_Property("EndPointURL",
            mURL);
lConnector.set_Property("SoapAction",
            mSoapAction);

lConnector.Connect
            ();
lConnector.BeginMessage ();

lSerializer = new
            SoapSerializer ();
lSerializer.Init
            (lConnector.InputStream);

lSerializer.startEnvelope
            ("","","");
lSerializer.startBody ("");
string lXML =
            @"<Add xmlns=""http://tempuri.org/""><iOp1>";
lXML +=
            iOp1.ToString();
lXML += "</iOp1><iOp2>";
lXML +=
            iOp2.ToString();
lXML +=
            "</iOp2></Add>";
lSerializer.writeXML
            (lXML);

lSerializer.endBody ();
lSerializer.endEnvelope
            ();

lConnector.EndMessage ();
lReader = new SoapReader
            ();
lReader.Load (lConnector.OutputStream, "");

if
            (lReader.Fault != null && lReader.Fault.text !=
            ""){
string lError = lReader.faultstring.text ;
return
            -1;
}
else{
int lResult =
            Int32.Parse(lReader.RPCResult.text);
return
            lResult;
}
}
private string mURL =
            "http://localhost/WebServiceSample/Calculator.asmx";
private
            string mSoapAction = "http://tempuri.org/Add";
}

As you can see from the above example, it is not trivial to build a SOAP client to access web services. Note that this is in spite of using the SOAP toolkit to help build the XML payload and manage the http connections. The first thing we do in the code above is to create an instance of the HttpConnector object. This object encapsulates the management of http connections. The SoapSerializer and SoapReader are used to create the request payload and interpret the response payload respectively. The serializer object has to create the XML nodes corresponding to the schema expected by the web service. This is a highly error prone and un-maintainable, since the payload has to be identical to what the web service expects, and if the web service changes its signature, the client has to ensure that the serializer is updated to match the new signature.

To avoid writing the SOAP client by hand, the .NET framework provides a utility that reads the SDL file generated by the web service and generates proxy classes that encapsulate all the plumbing required by the client to access the web service. This makes it extremely easy for the client to invoke the web service without having to write pages of code merely to construct and deconstruct the payload. This utility is called WSDL.exe and can be used as follows in the command line to generate the proxy classes.

wsdl /protocol:SOAP /language:CS http://localhost/WebServiceSample/Calculator.asmx?WSDL

The following command produces a C# proxy that wraps calls to the web service. The following is a list of some of the important parameters that can be passed to this utility.

  • Proxy: Generates source code for the classes and methods from the SDL file
  • Protocol: Specifies the protocol that will be used to access the web service. This could be Http-Get, Http-Post or SOAP.
  • Language: Specifies the language in which the proxy will be generated. This could be C#, VB or JS.
  • Path: Specifies the location of the WSDL file. This could point to an actual WSDL file, or to the web service URL with the ?WSDL argument.

The generated proxy class is as follows:

[System.Web.Services.WebServiceBindingAttribute(Name="CalculatorSoap",
namespace="http://tempuri.org/")]
public class Calculator :
System.Web.Services.Protocols.SoapHttpClientProtocol
{
[System.Diagnostics.DebuggerStepThroughAttribute()]
public Calculator() {
this.Url = "http://localhost/WebServiceSample/Calculator.asmx";
}

[System.Diagnostics.DebuggerStepThroughAttribute()]
[System.Web.Services.Protocols.SoapDocumentMethodAttribute
("http://tempuri.org/Add",
Use=System.Web.Services.Description.SoapBindingUse.Literal,
ParameterStyle=System.Web.Services.Protocols.SoapParameterStyle.Wrapped)]

public int Add(int iOp1, int iOp2) {
object[] results = this.Invoke("Add", new object[] {
iOp1,
iOp2});
return ((int)(results[0]));
}

[System.Diagnostics.DebuggerStepThroughAttribute()]
public System.IAsyncResult BeginAdd(int iOp1, int iOp2,
System.AsyncCallback callback, object asyncState) {
return this.BeginInvoke("Add", new object[] {
iOp1,
iOp2}, callback, asyncState);
}

[System.Diagnostics.DebuggerStepThroughAttribute()]
public int EndAdd(System.IAsyncResult asyncResult) {
object[] results = this.EndInvoke(asyncResult);
return ((int)(results[0]));
}
}

The proxy class has a method that wraps the web service methods. Note that the proxy supports both synchronous and asynchronous access of the web service. This is particularly important, since web services could be running on the congested public internet and could be slow to return the data. In this case, the client should not be blocked into making synchronous calls. Instead, the client should use delegates to be notified when the web service call returns.

The generated proxy derives from the SoapClientProtocol class in the System.Web.Services.Protocols namespace. This is the class that contains all the code necessary to create the Http connection and work with XML payloads. Since we specified the protocol to be SOAP, the proxy derives from SoapClientProtocol. If we had specified the protocol to be Http-Get or Http-Post, the proxy would have derived from the HttpClientProtocol class.

Once this proxy class is generated, the client can include the generated proxy in the project and make calls to this class just as if it were a normal class in the local namespace - we can treat the calculator object as if it is on the same machine and invoke the object and make method calls. Thus for the proxy above, the client calls would look like the call below:

protected void button1_Click (object sender,
            System.EventArgs e)
{
Calculator lCalc = new Calculator
            ();
int lResult = lCalc.Add (12,14);
return;
}

Compare this with the code that we wrote using the SOAP toolkit, and you will realize the power and simplicity of WSDL.

The above screenshot shows the calculator that we discuss above. As you can see, it requires the URL of the web service which supports the calculator functionality. The Add functionality uses the CalcClient object to manually create the SOAP packets and send it across the wire. We use the SOAP toolkit objects to help with the basic plumbing support. The Direct Add button uses the Calculator proxy object that is auto generated by the wsdl.exe tool.

SOAP in .NET Remoting

Remoting is the technology that allows objects in different processes or machines to communicate transparently and seamlessly. Remoting can be considered as the infrastructure that provides your applications with the ability to consume a separate business services tier. The biggest advantage of remoting over web services is that remoting is very closely coupled with the .NET runtime. Thus any .NET feature, including, but not limited to, constructors, delegates, interfaces, methods, properties, can be made available by using remoting. Remoting is based on the concepts of channels and formatters, where the channel provides the transport protocol and the formatters are responsible for serializing and deserializing the payload. Given this combination, any object can expose its services as a remoting endpoint to a client object. A thorough discussion of remoting is beyond the scope of this article; please refer to Matt Reynold's excellent article (http://www.asptoday.com/content/articles/20010511.asp?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq) for more details.

The System.Runtime.Remoting and System.Runtime.Remoting.Channels.HTTP namespaces provide classes that allow the creation of Http channels and the System.Runtime.Serialization.Formatters.Soap namespace provides classes for creation of SOAP channel formatters. Using these classes, you can build messages that can be passed between application domains. Let us consider a sample to demonstrate the usage of SOAP in .NET remoting. The following sample code demonstrates how an application in one app domain can register a callback with another application in another app domain.

Let us start with the server code.

namespace CommonRemotingObject
{
public delegate void DataAvailableEvent (string iData);

public class MyEventSource : MarshalByRefObject
{
public void ProcessData (string iData)
{
Console.WriteLine ("Recvd: " + iData);

if (mEvent != null)
mEvent (iData);
return;
}

public event DataAvailableEvent mEvent;
}
}

namespace RemotingServer
{
class RemotingServer
{
public static void Main(string[] args)
{
MyEventSource lSource = new MyEventSource();
System.Type lType = lSource.GetType ();
ChannelServices.RegisterChannel(new HttpChannel(8080));
RemotingConfiguration.RegisterWellKnownServiceType(
lType,
"RemotingServer",
WellKnownObjectMode.Singleton);
Console.WriteLine("Listening ...");
Console.Read ();
}
}
}

The MyEventSource class provides an event that will be fired when the ProcessData method is called. Clients can add their own handlers for this event by registering with this delegate. Further, every object that needs to be marshaled by reference needs to derive from System.MarshalByRefObject. This provides the plumbing needed to remote the object across app domains. In order for the server to provide its services, it needs to be listening on a well known channel. This is the channel clients will use to connect to the server and invoke its services. The ChannelServices class in the System.Runtime.Remoting.Channels namespace provides methods to register an object on a channel. Since we will be using SOAP, we first create an Http channel on an arbitrary port. Then we register our object with the remoting runtime so that the remoting runtime can locate the server object when a client tries to connect to it. Registering with the remoting runtime requires that we pass the assembly and the object that will be used to service the requests. Further, the runtime needs the schema of the methods supported by this object; the .NET runtime can generate schema on the fly from the assembly metadata using WSDL. (We will show you how to manually generate this schema later in this section.) Once the object is registered, it is now listening on the defined port for incoming requests.

Now that we know how the server is setup, let us take a look at the client code to access the server.

namespace RemotingClient
{
public class MyEventSink : MarshalByRefObject
{
public void MyEventHandler (string iData)
{
Console.WriteLine ("Callback: " + iData);
return;
}

public static void Main(String[] args)
{
MyEventSink lSink = new MyEventSink();

ChannelServices.RegisterChannel(new HttpChannel(0));
MyEventSource lSource =
(MyEventSource)Activator.GetObject(
typeof(MyEventSource),
"http://localhost:8080/RemotingServer");

lSource.mEvent +=
new DataAvailableEvent(lSink.MyEventHandler);
lSource.ProcessData("Test Message");
lSource.mEvent -=
new DataAvailableEvent(lSink.MyEventHandler);

Console.Read ();
}
}
}

Since we are using delegates for asynchronous communication with our web service, the client object has to be marshaled across to the server as well, and as a result the client derives from MarshalByRefObject. The client has to register its Http channel and then uses the remoting services to obtain a proxy to the server object. Note that the remoting runtime provides the Activator object that can be used to create objects locally or remotely, or obtain references to existing objects. Since we are using a singleton in our case, all we need is a proxy to the server object that is already running. Once this proxy is obtained, you can invoke methods and properties on this object in the usual manner. Note that in order for the C# compiler to support early binding, you will need to add a reference to the server assembly. If the server is running remotely, you might not have access to this assembly directly. The .NET framework provides a tool to generate the assembly from the WSDL file. This tool is the SoapSuds tool that can be used to generate either a proxy source file or a proxy assembly that your project can reference. The following is the command you would use to create the proxy source and the schema for our RemotingServer. The -url switch points to the wsdl for the server, the -os switch creates a copy of this schema on the local machine, and the -gc switch creates the proxy source.

soapsuds -url:http://localhost:8080/RemotingServer?wsdl -os:Server.xml -gc

Note that in this example, we do not use the proxy that is generated from soapsuds. Instead, we use the Activator object to explicitly locate and obtain a reference to the server object. The generated proxy from using this command looks like the code below. To improve the readability of this code, we have deleted the SOAP attributes declared on the class and the methods.

namespace CommonRemotingObject
{
using System;
using
            System.Runtime.Remoting.Messaging;
using
            System.Runtime.Remoting.Metadata;
using
            System.Runtime.Remoting.Metadata.W3cXsd2001;

public class
            MyEventSource
            :
System.Runtime.Remoting.Services.RemotingClientProxy
{
//
            Constructor
public
            MyEventSource()
{
base.ConfigureProxy(this.GetType(),
"http://66.31.47.252:8080/RemotingServer");
System.Runtime.Remoting.SoapServices.PreLoad
(typeof(CommonRemotingObject.DataAvailableEvent));
}
//
            Class Methods

public void ProcessData(String
            iData)
{
((MyEventSource)
            _tp).ProcessData(iData);
}

public void
            add_mEvent(DataAvailableEvent value)
{
((MyEventSource)
            _tp).add_mEvent(value);
}

public void
            remove_mEvent(DataAvailableEvent value)
{
((MyEventSource)
            _tp).remove_mEvent(value);
}
}
public delegate void
            DataAvailableEvent(String iData);
}

This proxy class can now be included in your client project and used as below.

[Serializable()]
class RemotingClient2 : MarshalByRefObject
{
public void MyEventHandler (string iData)
{
Console.WriteLine ("Callback (remoting client2): "
+ iData);
return;
}

static void Main(string[] args)
{
RemotingClient2 lSink = new RemotingClient2 ();
MyEventSource lSource = new MyEventSource ();

DataAvailableEvent lEvent =
new DataAvailableEvent(lSink.MyEventHandler);

lSource.add_mEvent (lEvent);
lSource.ProcessData("Test Message");
lSource.remove_mEvent (lEvent);
Console.Read ();

}
}

Note that although we do not explicitly make SOAP calls to any object, the remoting runtime takes care of this. All the calls made to the server by the client, and all the callbacks from the server to the client use SOAP over Http.

Note about downloadable code: When trying to run this code for this article, you may experience some HTTP 401 (or 401.3) errors, which are Access Denied errors. Our (yet unproven) theory is that this might be an issue with the Beta 2 code, since similar code works fine under Beta 1. The workaround to this is to ensure that the user id that is being used for "Anonymous Access" for your website has administrator privileges. Depending on your individual settings, all readers may not face this issue, but if you do, please try the above change. Note that this does pose a security risk, so please be sure to revert back to the IUSR_MachineName identity once you have finished exploring the attached code.

Conclusion

As we have discussed in this article, SOAP is deeply integrated into and natively supported by the .NET framework. SOAP is gaining popularity and support from a wide variety of companies, which bodes well for a highly interoperable protocol. While VS .NET and the .NET framework provide excellent support for SOAP which makes the use of SOAP almost transparent for the developer, it will be well worth the effort for all developers to understand the basics of SOAP and how the protocol can be used to build scalable, extensible and highly distributed applications.

Happy .NETing.


Web Site Error Logging and Reporting System

Performance

Performance (Sort by Name)

Creating an ISAPI FIlter HTTP Packet Analyzer
Effective Data Access in C#
Manipulating the Windows 2000 Registry and Event Viewer Using C#
Monitoring IIS Systems
Online CPU Console using a Web Control Library with .NET Security
Performance Monitoring
Runtime Reverse Engineering with System.Diagnostics
Serialization
Sharpening Your C# Unit Testing and Debugging Skills
Thread Pools

Threads, Delegates and Queues
Timing Code in .NET
Ultimate Guide to .NET Object Cleanup - Part I
Ultimate Guide to .NET Object Cleanup - Part II
Ultimate Guide to .NET Object Cleanup - Part III
Using Custom Counters With Performance Monitor in .NET
Web Services Performance and Scalability Design Considerations


Sharpening Your C# Unit Testing and Debugging Skills

Sharpening Your C# Unit Testing and Debugging Skills
http://www.csharptoday.com/content.asp?id=2059
Article Rating: 5
Published on December 23, 2002
Abstract
In this case study, James A Brannan takes us through several techniques that can be used to unit test your code. First, we discusses NUnit, an automated testing framework used to debug and test .NET applications. Next, we take a look at black box testing techniques: specification oriented testing, equivalence partitioning, boundary value analysis, and state transition testing. Finally we explore the white box testing techniques: branch testing, conditional testing, and data definition and use testing.
Case Study

In this case study, we will look at several techniques that can be used to unit test your code. First, we discuss NUnit, an automated testing framework used to debug and test .NET applications. Next, we will take a look at black box testing techniques: specification oriented testing, equivalence partitioning, boundary value analysis, and state transition testing. Finally we will explore the white box testing techniques: branch testing, conditional testing, and data definition and use testing.

Introduction

How do you make your C# components better? Learn more C# programming techniques? Perhaps, but countless studies by software engineers show the single biggest thing you can do to make your components better is unit test them. Unit testing is the testing of an individual component. For example, when using C# after coding a class, you would test it. In this case study, I illustrate several unit testing techniques - black box and white box - you can use to unit test C# components. By the end of this case study, you should understand unit testing C# components using the tool NUnit, and how to use the following testing techniques: specification oriented testing, equivalence partitioning, boundary value analysis, state transition testing, branch testing, conditional testing, and data definition and use testing.

The Problem

The deadline. The closer it gets, the more panicked you become. The more panicked you become, the faster you code. The faster you code, the sloppier you become. The sloppier you become, the harder testing becomes. The harder testing becomes, the more time it takes. The deadline gets closer. And so you forgo unit testing to save time.

Forgoing unit testing just waylays the problem, though. For example, on a large project, when the code makes it to the testing team, now they must find the bugs in the unit. On a small project, or in-house project, the bug might make it into the release. Now, even worse then having testers find the bugs, the users must find the unit's bugs - bugs that might be hidden deep inside an application.

When the tester or user finds a bug, the developer responsible for the unit must usually fix it. So you must now fix the bug you did not fix during development. The bug might now take you days to fix; however, had you found and fixed it when you were coding, the bug might have taken hours or minutes. Forgoing unit testing did not save time.

The Solution

You can save time by unit testing and coding concurrently and purposefully. Unit testing and coding concurrently helps you debug components quicker and smoother. Unit testing purposefully, no matter how informal, is more effective then unit testing haphazardly.

Why unit test concurrently? Unit testing concurrently with development helps prevent you from forgoing unit testing. This is converse to conventional software engineering thinking, which maintains that you should complete the entire unit (for example an assembly) and then unit test the component. Unit testing and developing concurrently helps you find and fix errors immediately, before another interesting feature captures your interest

How do you unit test purposefully? Write the unit test's specifications before writing the unit test. If you are fortunate enough to have a requirements document, or Use Cases, use them to help guide your testing. But, do not aimlessly step through your code in a debugger and consider the first time it runs without exceptions as code complete.

Incrementally debug, unit test, and code. Begin by planning how you intend to implement a component. Then incrementally develop the component, feature by feature. During each increment, first code the feature. Second, plan how you intend to unit test the feature by writing down a few unit tests. Third, write the unit tests and use them to debug and unit test the feature. Repeat these three steps for each feature in the component. Finally, when the component is complete, consider any areas you might have overlooked testing and be sure to unit test them. The following figure illustrates this process.

Note that what I am calling concurrent coding and unit testing, I directly derived from Extreme Programming's (XP) test-driven development philosophy. When practicing XP's test-driven development, testing literally drives development. You can distill test-driven development to three main tenets. First, you code a feature's test(s) before coding the feature. You write the unit test, code the feature, and then run the unit tests on it. When the feature passes the test, the feature is completed. You code no more and no less then that required in passing the test(s). Second, you maintain a testing suite, testing every feature of every component in an application. Ideally, you should automate the test suite. Third, you release no code - even minor enhancements or bug fixes - unless it has associated unit tests that pass when run. Like test-driven development, when concurrently coding and unit testing, you incrementally unit test and code you applications, feature by feature.

Unlike test-driven development, in this case study I write unit tests after writing a feature and not before as XP dictates. If you prefer writing a feature's test before writing the feature, note that the techniques shown here are still valid. Whether you write the test before or after writing a feature does not seem important to me. However, you should note that Extreme Programming enthusiasts emphatically maintain that you write a feature's test before coding the feature. Regardless of whether you write your test immediately before coding a feature or immediately after coding a feature, what seems important to me is that you unit test it thoroughly before coding another feature or enhancing the current feature.

System Requirements

To follow along, you need NUnit 2.0, available at http://www.nunit.org/?WROXEMPTOKEN=981098ZFI2gYTm0YyOIQ6Szs2X. Be sure that you download version 2.0, because it is a total rewrite from version 1.0. You need C# knowledge; I use C# examples that contain parameters, attributes, events, and unsafe code. In this case study, I assume you are already familiar with these C# concepts. You also need some basic familiarity with C pointers and arrays. However, even if you do not fully understand all the C# concepts presented in the code samples, the unit testing concepts should still be clear.

Automated Unit Testing using NUnit

Strict adherence to a regime of coding and unit testing concurrently will do wonders in improving your code. However, regression testing is still problematic. Regression testing tests that new features and modifications to existing features do not break code that passed earlier tests. When practicing the incremental development method in this case study, regression testing could quickly become unwieldy. Every time you incrementally added a new feature, you would have to unit test it and rerun the unit tests for all earlier coded features. What you need is an automated way to test your code.

Driver

One way you can automate testing your code is by using a driver. A driver is code designed to execute a component or function that ordinarily would not be executable on its own. You cannot test an assembly that lacks a main method, for instance.

Consider the following example. Typically, after coding the Tested class you would create a driver to run it.

using System;
namespace ToTest{
class
            Tested
{
public void
            Run()
{
Console.WriteLine("Running.....");
}
}
}

using
            System;
using ToTest;

class MainClass
{
public
            static void Main(string[] args)
{
Tested objTest = new
            Tested();
objTest.Run();
}
}

You would compile both preceding classes and then open them in a debugger (Microsoft's CLR Debugger for example). Next, you would set the executable as the program to debug, set breakpoints, and then click Start on the Debug menu.

You might even add assertions to your driver using System.Diagnostics.Debug.Assert (covered in more detail later in this case study). For example, to the preceding code you might add a read only Name property to Tested and a debug statement to ensure it worked as expected.

private String strName = "tester";
public
            String Name{
get{return this.strName;}
}

System.Diagnostics.Debug.Assert(objTest.Name.Equals("incorrect"),
"Tested class name did not equal ''incorrect.''");

After defining the DEBUG symbol, you would then compile MainClass.

csc /out:test.exe /d:DEBUG /t:exe /r:totest.dll MainClass.cs

Because the DEBUG symbol enables debugging, when MainClass reaches a line containing an assertion, it throws an exception, and displays the Assertion Failed dialog box.

You can sufficiently unit test your code using a simple driver with assertions. If you prefer to write your own drivers, go ahead. The black box and white box unit testing techniques in this case study remain valid. Skip the following section on NUnit and proceed directly to the black box testing section. But before you do, you might want to at least briefly glance at the section on NUnit. A testing framework like NUnit really does make testing your .NET code easier and more robust.

NUnit

NUnit is an open-source testing framework for unit testing .NET applications that makes automated testing easier. Think of NUnit as a framework for writing drivers and tests for C# assemblies.

Test Fixtures and Test

The fundamental two NUnit constructs are the test fixture and the test. A test fixture is a class that contains tests. The only restriction on a test fixture is that it does not override its default constructor. You mark a class as a test fixture via the TestFixture attribute.

[TestFixture]
public class ExampleOneTest{

A test is a method inside a test fixture that tests something. You mark a method as a test via the Test attribute.

[Test]
public void TestGetGreeting()

Consider the following class and associated NUnit unit test.

namespace Examples {
using
            System;

public class ExampleOne {
public String
            GetGreeting(){
return "Hello
            Turkey.";
}
}
}

namespace Examples {
using
            NUnit.Framework;

[TestFixture]
public class
            ExampleOneTest{

[Test]
public void
            TestGetGreeting(){
ExampleOne objExampleOne = new
            ExampleOne();
Assertion.AssertEquals("Hello Turkey.",
            objExampleOne.GetGreeting());
}
}
}

ExampleOne contains one method, GetGreeting. ExampleOneTest is a test fixture that contains one test, TestGetGreeting. TestGetGreeting tests ExampleOne by creating a new ExampleOne instance, calling the GetGreeting method, and ensuring that it equals Hello Turkey.

NUnit GUI Program

NUnit comes with a command-line program and an easy-to-use GUI. To better understand the GUI, let's run ExampleOneTest in the NUnit GUI.

Copy and paste, or re-write, ExampleOne and ExampleOnTest from the preceding code and compile both classes. Be sure to include the NUnit framework assembly (nunit.framework.dll) as a reference. Compile the two classes into one DLL using the SDK C# command-line compiler or your IDE's equivalent.

csc /out:exampleone.dll /t:library /r:nunit.framework.dll ExampleOne.cs ExampleOneTest.cs

After compiling, start the NUnit GUI program. On the File menu, click Open, and then open exampleone.dll. NUnit loads the assembly and creates a hierarchical tree from the fully qualified file name, namespace, test fixtures, and tests.

NUnit places the assembly's file name at the top of the hierarchy, followed by the Examples namespace, the ExampleOneTest test fixture, and the TestGetGreeting test. Select the Examples node in the Tree View, click Run, and NUnit runs all tests in all the fixtures in the Examples namespace, albeit there is only one fixture and one test in this example.

You can include as many tests as desired in a test fixture and as many fixtures as desired in a namespace. If compiled in the same assembly, NUnit creates a suite of tests automatically by examining the assembly's namespace and identifying all test fixtures and tests in that assembly. For example, add the following new class and associated test fixture to your exampleone.dll assembly.

namespace Examples {
using
            System;

public class ExampleTwo {
private String
            strUserName;

public String GetGreeting(){
return "Hello
            Turkey.";
}

public String UserName{
get{return
            this.strUserName;}
set{this.strUserName =
            value;}
}
}
}

namespace Examples {
using
            System;
using NUnit.Framework;

[TestFixture]
public
            class ExampleTwoTest{

ExampleTwo objExampleTwo = new
            ExampleTwo();

[Test]
public void
            TestGetGreeting()
{
Assertion.AssertEquals("Hello Turkey.",
            objExampleTwo.GetGreeting());
}

[Test]
public void
            TestUserName(){
String strName =
            "James";
objExampleTwo.UserName =
            "James";
Assertion.AssertEquals(strName,
            objExampleTwo.UserName);
}
}
}

After recompiling, reload exampleone.dll in NUnit. The ExampleTwoTest test fixture and its two tests, TestGetGreeting and TestUserName, appear in the Tree View. Note that after selecting any node in the hierarchy, clicking Run tests child nodes to the node selected, other nodes do not run.

When NUnit tests fail, the progress bar changes from green (signifying tests passed) to red (signifying one or more tests failed). For example, in the following code, EchoName throws a DivideByZero exception. The ExpectedException attribute indicates that the EchoName test expects an ArgumentException. When you run the tests, the EchoName test fails, and the progress bar turns red, as does the Examples node and the EchoName node.

namespace Examples {
using
            System;
using NUnit.Framework;

[TestFixture]
public
            class ExampleThreeTest{

ExampleThree objExampleThree = new
            ExampleThree();

[Test]
[ExpectedException(typeof(System.ArgumentException))]
public
            void EchoName(){
int test = 0;
int intWillFail =
            4/test;
}

}
}

When a fixture's test fails, both the fixture's node and the failing test's node turn red. Multiple fixtures and multiple tests might fail - in this situation, all failing fixtures and tests turn red.

Attributes

NUnit uses .NET custom attributes to identify NUnit specific functionality. An attribute allows you to define metadata that the CLR retrieves at runtime using reflection. NUnit defines the TestFixture, Test, SetUp, TearDown, ExpectedException, and Ignore attributes. The TestFixture attribute identifies classes that contain NUnit tests. The Test attribute identifies NUnit test methods. SetUp marks the first method run before a test fixture runs any tests, while TearDown marks a method run after a test fixture finishes running tests. Note that both SetUp and TearDown are optional. The Ignore attribute allows you to skip a test when running NUnit tests. Although you could simply remove the Test attribute, Ignore helps you remember that you are not running the test. For example, when running the tests in the NUnit GUI, removing the Test attribute makes the test nonexistent as far as NUnit is concerned. However, when marking the test with Ignore, NUnit changes the ignored node to yellow, making it explicit that you are temporarily skipping the test.

The following code illustrates NUnit's attributes:

namespace Examples {
using
            System;

public class ExampleThree {

public String
            EchoName(String Name){

return
Name;
}
}
}

namespace Examples {
using System;
using NUnit.Framework;

// ---------------------------------------------------------------------
// TestFixture attribute: identifies an NUnit class that contains one or
// more tests.
// ---------------------------------------------------------------------

[TestFixture]
public class ExampleFourTest{
ExampleThree objExampleThree;

// ---------------------------------------------------------------------
// SetUp attribute: identifies the method run when constructing a test
// fixture. This method is run before any tests are run.
//----------------------------------------------------------------------

[SetUp]
public void Init(){
objExampleThree = new ExampleThree();
}

// ---------------------------------------------------------------------
// Test attribute: identifies NUnit test methods.
// ---------------------------------------------------------------------

[Test]
[ExpectedException(typeof(System.DivideByZeroException))]
public void EchoName(){
int test = 0;
int intWillFail = 4/test;
}

// ---------------------------------------------------------------------
// Ignore attribute: notifies NUnit to skip the test.
// ExpectedException attribute: notifies NUnit that the test should
// throw an exception of the specified type. If the test does not
// throw an exception of specified type, it is a failure
// ---------------------------------------------------------------------

[Test]
[Ignore("Don't feel like testing")]
[ExpectedException(typeof(System.ArgumentException))]
public void EchoNameToo(){
int test = 0;
int intWillFail = 4/test;
}

// ---------------------------------------------------------------------
// TearDown attribute: identifies the method run when completing running
// a test fixture. This method is run after all tests are run.
//----------------------------------------------------------------------

[TearDown]
public void Dispose(){
objExampleThree.EchoName("John");
objExampleThree = null;
}
}
}

NUnit's Assertion Class

Inside a test method, you test some functionality using NUnit's Assertion class. You expect a result from exercising some functionality in the feature under test. For example, when NUnit got to the test MyTest, it would then execute the assertion to ensure it was true. If not, NUnit would record the failure.

[Test]
public void MyTest(){
Assertion.Assert("1 does not equal 2.", 1==2);
}

NUnit's Assertion class contains the static methods you use to test expected results. For full coverage of the Assertion class member functions, refer to NUnit's documentation. However, we use several Assertion functions in this case study, and so I will cover them briefly here.

Assert

Assert asserts something is true. For example, the following code fragment asserts that MyFunction returns true.

Assertion.Assert(objMyObject.MyFunction(double dblMathQuestionAnswer))

You can also provide a failure message to Assert that is echoed to the console when the assertion fails.

Assertion.Assert("The test failed.", objMyObject.MyFunction.IsCorrect(double dblMathQuestionAnswer))

AssertEquals

AssertEquals asserts that two values are equal. The following code fragment tests two boolean values, the expected result (parameter one) and the actual result (parameter two).

Assertion.AssertEquals(false, objMyObject.MyFunction.IsCorrect(double
dblMathQuestionAnswer))

You can overload AssertEquals to accept integers, doubles, strings, floats, and objects. You can also provide a failure message to AssertEquals.

Assertion.AssertEquals("The test failed.", false,
objMyObject.MyFunction.IsCorrect(double dblMathQuestionAnswer))

AssertNotNull and AssertNull

AssertNull and AssertNotNull test whether an object is or is not null.

MyObject objMyObject =
            null;
Assertion.AssertNull(objMyObject);
objMyObject = new
            MyObject();
Assertion.AssertNotNull(objMyObject);

As with Assert and AssertEquals, you can also provide a failure message to these functions.

MyObject objMyObject =
            null;
Assertion.AssertNull("Its not null!",
            objMyObject);
objMyObject = new
            MyObject();
Assertion.AssertNotNull("Its null!",
            objMyObject);

AssertSame

AssertSame tests if two object references refer to the same underlying object.

MyObject objMyFirstObject = new MyObject();
MyObject objMySecondObject = objMyFirstObject;
Assertion.AssertSame("Not the same!", objMyFirstObject, objMySecondObject);

NUnit Summary

You now have all the information you need to start using NUnit to unit test your code. Notice that you can incrementally add new test methods to a test fixture without destroying the old tests. You automatically regression test every time you test a new feature. Consider what NUnit does not do, though. NUnit does not provide testing techniques to test your code. Unit testing, though in many respects intuitive, has many different available techniques. These techniques are as important to coding better, if not more, than understanding how to use NUnit. NUnit after all, is only a tool.

Black Box Testing Techniques

Black box testing test expected outputs from known inputs and treats a component's internals as a black box. Black box testing verifies that software meets its specification. However, black box testing does not necessarily validate that a unit functions correctly. For validating the code's functionality, you need white box testing as well.

Strictly speaking, when unit testing concurrently with developing, you are not exclusively black box or white box testing, but, a little of both. For this reason, some testing resources call this testing gray box testing. For example, in the upcoming examples, we know the unit's internals; we are writing the internals as we test, after all. Thus, the dichotomy between black box and white box testing is somewhat false in the upcoming examples.

When unit testing concurrently with development, or following XP's Test Driven development, you employ black box techniques and white box techniques simultaneously. However, for presentation clarity, in this case study I separate the two techniques. This separation is logical, though. When black box testing, you examine inputs and outputs to a component, not the component's internals. Of course, you have an idea, based on the internals, of what the outputs should be, but you are not examining the internals.

Many proven techniques for black box testing exist; in this case study I illustrate four of them:

  • specification oriented testing,
  • equivalence partitioning,
  • boundary value analysis,
  • state transition testing.

Specification oriented testing is the testing you are probably most familiar with; it tests that inputs to a unit yields the expected outputs. For example, if a function is supposed to square a number, then if you give the function the number two as input, the function should return four as output. Equivalence partitioning is a technique used to ensure adequate testing coverage by considering a range of values equal for testing purposes - the goal is to test one or more values from each range of values. Boundary value analysis tests values at a partition's boundaries. Errors are more likely to occur at a boundary, and so you pay special attention to boundary values. Finally, state transition testing derives tests from a component's change in state; the test provides a known event and then tests that the object enters the expected state. Let us consider each testing technique in more detail.

Specification Oriented Testing

Specification oriented testing shows that a feature implements its specification. For example, suppose your boss told you the following.

  • You need to develop a component that returns the greeting "Never pay taxes again <person's name>" or "We will help lower your taxes <person's name>". The component is equally likely to return either greeting.

The following code implements this informal specification:

namespace Greetings {
using
            System;

public class Greeting {

private Random
            objRandom = new Random();

public String GetGreeting(String
            Name){
double dblRandValue =
            this.objRandom.NextDouble();
String strToReturn;
if
            (dblRandValue < .5){
strToReturn = "Never pay taxes again " +
            Name;
}
else
strToReturn = "We will help lower your taxes
            " + Name;

return strToReturn;
}
}
}

The following test fixture tests the preceding component:

namespace Greetings {
using
            System;
using NUnit.Framework;

[TestFixture]
public
            class GreetingTest{

Greeting objGreeting = new
            Greeting();

// Specification oriented test example
            //

[Test]
public void GetGreetingTest()
{
String
            strTheGreeting
            =
objGreeting.GetGreeting("Jerry");

String[]
            objFullGreetings = {"Never pay taxes again Jerry",
"We will help
            lower your taxes
            Jerry"};

Assertion.Assert(strTheGreeting.Equals(objFullGreetings[0])
||
            strTheGreeting.Equals(objFullGreetings[1]));
}
}
}

GetGreetingTest tests that when you provide Jerry as input, it receives either Never pay taxes again Jerry or We will help lower your taxes Jerry. GetGreetingTest tests that the feature (GetGreeting) implements its specification.

As an aside, before continuing, take a moment to reflect on what we did not test. The test ensures the component returns one of two messages - it does not test that GetGreeting is equally likely to return either message. Because dblRandValue is a local variable, we have no easy way to see its value in GetGreetingTest. To test either message's probability, we need to use white box testing techniques, which I cover later.

Equivalence Partitioning

Specification oriented testing ensures a feature fulfills its specification - expected inputs yield expected outputs. However, note that these tests usually only test the simplest cases; sometimes you need more then a single test to adequately test a feature. Functions often accept a range of values as parameters. To adequately test a function like this, you need to test with several values. However, testing all possible values is impractical, if not impossible. Consider the following example:

namespace Greetings {
using
            System;

public class Greeting {

public enum Theme
            {GetMoreBack,PayNothing,PayLess};

// [ GetGreeting excluded
            for brevity ] //

public Theme GetTheme(double
            dblTaxesPaid){
Theme enumTheme = new Theme();

// grouping
            one: -infinity to zero (inclusive) //

if (dblTaxesPaid <=
            0)
enumTheme = Theme.GetMoreBack;

// grouping two:
            (exclusive) zero to 1000 (inclusive) //

if (dblTaxesPaid >
            0 && dblTaxesPaid <= 1000)
enumTheme =
            Theme.PayNothing;

// grouping three: (exclusive) 1000 to +
            infinity //

if (dblTaxesPaid > 1000)
enumTheme =
            Theme.PayLess;

return enumTheme;
}
}
}

Depending upon the tax paid, GetTheme returns GetMoreBack, PayNothing, or PayLess. The range of values GetTheme accepts for dblTaxesPaid ranges from negative infinity to positive infinity. This makes testing all values impossible. Although testing all possible values is impossible, you can test a robust subset of those values using equivalence partitioning.

Equivalence partitioning groups data with similar values in groups, considering all values in a grouping equivalent for testing purposes. For example, in the preceding GetTheme method, there are three natural groupings; values less then or equal to zero, values greater then zero and less then or equal to 1000, and values greater then 1000. We can derive the following tests from the partitions:

  • Test one: enter -20 as input, expected to receive Theme.GetMoreBack.
  • Test two: enter 100 as input, expected to receive Theme.PayNothing.
  • Test three: enter 5000 as input, expected to receive Theme.PayLess.

We write the NUnit tests, adding the GetThemeTestGetMoreBack, GetThemeTestPayNothing, and the GetThemeTestPayLess test methods to GreetingTest.

namespace Greetings {
using
            System;
using NUnit.Framework;

[TestFixture]
public
            class GreetingTest{

Greeting objGreeting = new
            Greeting();

// Test one: enter -20 as input, expected to
            receive
// Theme.GetMoreBack //

[Test]
public void
            GetThemeTestGetMoreBack(){
Assertion.AssertEquals(Greeting.Theme.GetMoreBack,
objGreeting.GetTheme(-20));
}

//
            Test two: enter 100 as input, expected to receive Theme.PayNothing
            //

[Test]
public void
            GetThemeTestPayNothing(){
Assertion.AssertEquals(Greeting.Theme.PayNothing,
objGreeting.GetTheme(100));
}

//
            Test three: enter 5000 as input, expected to receive Theme.PayLess
            //

[Test]
public void
            GetThemeTestPayLess(){
Assertion.AssertEquals(Greeting.Theme.PayLess,
objGreeting.GetTheme(5000));
}

//
            [Specification oriented test example omitted for brevity ]
            //

}
}

GetThemTestGetMoreBack tests that GetTheme returns Theme.GetMoreBack for values less then zero, GetThemeTestPayNothing tests that GetTheme returns Theme.PayNothing for values from zero to 5000, and GetThemeTestPayLess tests that GetTheme returns Theme.PayLess for values greater then 5000.

Boundary Value Analysis

Equivalence partitioning helps ensure adequate test coverage by testing one or more values in each partition. However, errors often occur at a partition's boundary. Boundary value analysis analyzes a partition's boundary values for those errors.

For example, the preceding test cases randomly picked test values to test in each partition. The tests passed; however, these tests overlooked values at the boundary. Suppose we modified the original specification so that values up to 1000, but not including 1000, return Theme.PayNothing, while values over 999 return Theme.PayLess. Although the previous tests still pass, GetTheme has two errors. The first error, the second condition should be all values from zero to 999 not zero to 1000.

if (dblTaxesPaid > 0 && dblTaxesPaid <= 1000) enumTheme = Theme.PayNothing;

The second error, the third condition mistakenly excludes 1000.

if (dblTaxesPaid > 1000) enumTheme = Theme.PayLess;

Tests one, two, and three miss these two errors - these three tests do not use the value 1000 (the boundary value). By testing each partition's boundaries (in this example, the values 0 and 1000) you can catch these errors.

Boundary value analysis involves three steps. First, determine the boundary values. Second, write the test specifications based upon those values.

  • Test four: enter zero as input, expected to receive Theme.GetMoreBack.
  • Test five: enter 1000 as input, expected to receive Theme.PayLess.

Third, write the NUnit tests that test the boundary values:

namespace Greetings {
using
            System;
using NUnit.Framework;

[TestFixture]
public
            class GreetingTest{

Greeting objGreeting = new
            Greeting();

// boundary value analysis tests example
            //

[Test]
public void
            GetThemeTestGetMoreBackBoundary(){
Assertion.AssertEquals(Greeting.Theme.GetMoreBack,
objGreeting.GetTheme(0));
}

[Test]
public
            void
            GetThemeTestPayLessBoundary(){
Assertion.AssertEquals(Greeting.Theme.PayLess,
objGreeting.GetTheme(1000));
}

.
            . .

}
}

The GetThemeTestPayLessBoundary test fails. But after fixing the second condition and third condition in GetTheme, as in the following code, the test passes.

    public Theme
            GetTheme(double dblTaxesPaid){
Theme enumTheme = new
            Theme();

// grouping one: -infinity to zero (inclusive)
            //

if (dblTaxesPaid <= 0)
enumTheme =
            Theme.GetMoreBack;

// grouping two: (exclusive) zero to 1000
            (exclusive) //

if (dblTaxesPaid > 0 &&
            dblTaxesPaid < 1000)
enumTheme = Theme.PayNothing;

//
            grouping three: (exclusive) 1000 to + infinity //

if
            (dblTaxesPaid >= 1000)
enumTheme =
            Theme.PayLess;

return enumTheme;
}

Boundary value analysis does not always have to test explicit partitions as in the last example. Data partitions can occur in less obvious places. For example, in the following method, there are two partitions - dates greater then the current date and dates less then or equal to the current date. Suppose we added the following function to the Greeting class.

    public bool
            ValidDate(DateTime objDate){

// note: this code is
            incorrect
bool blnValid = false;
if
            (!(DateTime.Compare(objDate, System.DateTime.Now) >
            -1))
blnValid = true;

return blnValid;
}

ValidDate determines if a date is valid. If the date is before the system date, the function returns true. Conversely, if the date is on or after the system date, the function returns false.

Please note a subtle potential bug in the code: if the DateTime passed to ValidDate is initialized using a date with no time, and that date is the same date as System.DateTime, then the method returns false. Although technically correct, this is not the intended behavior - we want two dates with the same date to be equal. This false value occurs because System.DateTime.Now initializes to the current time on the current date while the DateTime passed to the function initializes to midnight.

Using specification oriented testing or equivalence partitioning would probably miss this bug, because we would test one value less than System.DateTime.Now and one value greater then System.DateTime.Now. The following test passes, for example.

[Test]
public void ValidDateTest(){
Assertion.Assert(objGreeting.ValidDate(new DateTime(1965,10,02)));
}

However, if we consider the boundary, where both dates are equal, the following test fails.

[Test]
public void ValidDateTest (){

// should fail so negate to true to pass
// Note: change to current system date if following along at home.

Assertion.Assert(!(objGreeting.ValidDate(new DateTime(2002,10,26))));
}

Now suppose that after running the preceding test, we caught our mistake and changed ValidDate to the following:

    public bool
            ValidDate(DateTime objDate){
bool blnValid = false;
if
            (!(DateTime.Compare(objDate.Date, System.DateTime.Now.Date) >
            -1))
blnValid = true;

return blnValid;
}

ValidDate is now comparing the date parts of two DateTime classes and not the date and time.

State Transition Testing

Sometimes you can modeling a feature's behavior using a state transition diagram. A state transition diagram models a finite state machine. A finite state machine is an idealized model of a system where the system can usually only be in one state at a time. For example, a light can be on or it can be off, but not both. Suppose we needed to add a new feature to Greeting that allows a user to win when the class receives an external event signifying Greeting should allow winners. The following diagram models the necessary behavior.

One way we can implement the preceding requirement is by using an event. The following code adds an event handler to the Greeting class:

namespace Greetings {
using
            System;

public class Greeting {

private bool blnWinner
            = false;

// property indicating if user is a winner
            //

public bool Winner{
get{return
            this.blnWinner;}
}

// method that captures a let win event
            //

public void LetWinEventNotifier(Object sender, bool
            blnWin){
this.blnWinner =
            blnWin;
System.Console.WriteLine("called");
}

. .
            .

}
}

To test the event handler, we add a delegate, an event handler, and a method to register the event handler and test the handler in the Greeting class:

namespace Greetings {
using
            System;
using NUnit.Framework;

public delegate void
            LetWinEventHandler (Object source, bool
            blnWin);

[TestFixture]
public class
            GreetingTest{

Greeting objGreeting = new Greeting();

.
            . .

public event LetWinEventHandler
            LetWinEvent;

[Test]
public void
            LetWinHandledTest(){
this.LetWinEvent +=
            new
LetWinEventHandler(objGreeting.LetWinEventNotifier);
Assertion.Assert(!objGreeting.Winner);
LetWinEvent(this,true);
Assertion.Assert(objGreeting.Winner);
LetWinEvent(this,false);
Assertion.Assert(!objGreeting.Winner);
}
}
}

After declaring a delegate with the same method signature as LetWinEventNotifier in the Greeting class, the GreetingTest fixture declares a LetWinEvent event. The LetWinHandledTest registers the Greeting class's LetWinEventNotifier and then tests that it receives the events and reacts correctly.

White Box Testing Techniques

White box testing tests a unit's internals, focusing on conditional statements, looping, and other structural concerns. While black box testing ignores a unit's internals, white box testing uses this knowledge of a unit's internals to further test a component. When black box testing, you were testing that inputs yielded expected outputs, now, you are testing that the component's internals function correctly.

Assertions and Debug Statements

By using assertions and debugging statements, you can add code that is executed when you compile your application in debug mode, but not when you compile it for release. When writing .NET applications, the Debug class implements both these concepts. Although there are many different ways you can use the Debug class, in this case study we use it to write debugging to a StringWriter. We then test the StringWriter for certain sub-strings in our NUnit test fixture. This use should become clearer after the examples in the next two sections.

Assertions

C# supports assertions for debugging your code through the Assert method in the System.Diagnostics.Debug class. The Assert method tests conditions that should be true. If an assertion is not true then it opens an Assertion Failed dialog box.

When using assertions, the compiler only enables assertions when you compile the code in debug mode. When compiled in release mode, the compiler disables assertion statements. You must compile the code with the \d:debug switch.

Having an Assertion Failed dialog box open when running automated unit tests is unacceptable (someone has to click Ok, Abort, or Cancel). We need to obtain the information another way. We can change this behavior by replacing the Debug class's listeners with our own. For more on the Listener class, refer to the .NET SDK documentation. Consider the following example:

using System;
using ToTest;
using
            System.Diagnostics;
using System.IO;

public class
            MainClass
{
public static void Main(string[] args)
{
//
            create a stringwriter and trace listener to write debug messages
            to

StringWriter objWriter = new
            StringWriter();
TextWriterTraceListener myTextListener =
            new
TextWriterTraceListener(objWriter);
Debug.Listeners.Clear();
Debug.Listeners.Add(myTextListener);
Tested
            objTest = new
            Tested();

Debug.Assert(objTest.Name.Equals("incorrect"),
"Assertion failed:Main.cs:line
            17");

Console.WriteLine(objWriter.ToString());
objWriter.Close();
myTextListener.Close();
}
}

The preceding code replaces the Debug class's DefaultTraceListener with a TextWriterTraceListener that writes to a StringWriter, which then echoes the string to the console.

Fail:Assertion failed:Main.cs:line 17

Instead of echoing to a string, though, we can return values to a NUnit test, allowing the tests to see inside the component. Consider the following component and associated NUnit test fixture.

namespace AssertsAndDebug{
using
            System;

public class TestedClass {
public bool
            ReturnFalse(){
System.Diagnostics.Debug.Assert(false,
"Assertion
            Failed:TestClass.cs:line 7");
return
            false;
}
}
}

namespace AssertsAndDebug {
using
            System;
using NUnit.Framework;
using
            System.Diagnostics;
using
            System.IO;

[TestFixture]
public class TestedClassTest
            {

private TestedClass objTestedClass = new
            TestedClass();
private StringWriter objWriter = new
            StringWriter();
private TextWriterTraceListener
            objTextListener;

[SetUp]
public void
            Init(){
objTextListener = new
            TextWriterTraceListener(objWriter);
Debug.Listeners.Clear();
Debug.Listeners.Add(objTextListener);
}

private
            void ReInitializeWriters(){
objWriter.Close();
objWriter = new
            StringWriter();
objTextListener.Writer =
            objWriter;
}

[Test]
public void
            TestReturn(){
this.ReInitializeWriters();
objTestedClass.ReturnFalse();

if
            (objWriter.ToString().Trim().Length >
            0)
Assertion.Fail(objWriter.ToString().Trim());
}

[Test]
public
            void
            TestReturnTwo(){
this.ReInitializeWriters();
objTestedClass.ReturnFalse();
if
            (objWriter.ToString().Trim().Length >
            0)
Assertion.Fail(objWriter.ToString().Trim());
}

[TearDown]
public
            void
            Dispose(){
objWriter.Close();
objTextListener.Close();
}
}
}

When we load the assembly in the NUnit GUI, the tests fail. The reason they fail is that an assertion fails in the AssertsAndDebug class. The StringWriter returns the string from the StringWriter and Debug.Assert in TestedClass fails.

Now before you object, yes this is a "low-rent" way to test a unit's internals. Writing debugging output to a string and then doing substring comparisons to test for certain values is not a high-tech solution. But it works and you can implement it easily.

Certainly there are other ways to obtain the same results as assertions and debugging statements; if you have a better way to do the same thing, use that technique. Better yet, post it to this article's discussion board. Use whatever technique you prefer; however, what is important is that you internally record what is happening inside your component and then test what happened in your NUnit tests.

Branch Testing

When branch testing, you examine the paths through a component. For example, consider the following component. The paths are marked A, B, C, and D.

namespace Greetings {
using
            System;
using System.Diagnostics;

public class Greeting
            {

private bool blnIsEighteen = false;
private static int
            WINNUMBER = 7;

public double GetCashOut(int winningNumber,
            bool tax, bool is18){
this.blnIsEighteen = is18;
double
            winnings = 0;
if (this.blnIsEighteen){
if
            (IsWinner(winningNumber)){
A. if (tax) winnings =
            ReturnCashOutTaxed();
B. else winnings =
            ReturnCashOutNotTaxed();
}
C. else winnings =
            ReturnCashDeducted();
}
D. else throw new Exception("Must be
            18.:line 23:Greeting.cs");

return
            winnings;
}

private bool IsWinner(int
            winningNumber){
if (winningNumber == WINNUMBER)
            {

return true;
}
return false;
}

A.
            private double
            ReturnCashOutTaxed(){
Debug.Assert(this.blnIsEighteen, "Not
            eighteen:line
            41:Greeting.cs");
Debug.WriteLine("in_ReturnCashOutTaxed");
return
            DetermineWinning() * .3;
}

B. private double
            ReturnCashOutNotTaxed(){
Debug.Assert(this.blnIsEighteen, "Not
            eighteen:line
            47:Greeting.cs");
Debug.WriteLine("in_ReturnCashOutNotTaxed");
return
            DetermineWinning();
}

C. private double
            ReturnCashDeducted(){
Debug.Assert(this.blnIsEighteen, "Not
            eighteen: line
            53:Greeting.cs");
Debug.WriteLine("in_ReturnCashDeducted");
return
            -10;
}

private double DetermineWinning(){
Random
            objRandom = new Random();
double dblRandom =
            objRandom.NextDouble();
Debug.WriteLine("<random>" +
            dblRandom + "</random>" );
return dblRandom *
            10000;
}
}
}

When the user has the winning number and must pay taxes, Greeting follows path A. When the user has the winning number and is not required to pay taxes, Greeting follows path B. When the user does not have the winning number, Greeting follows path C. Finally, if the user is under eighteen, Greeting follows path D. We can glean the following four tests from these paths:

  • Test one: Call GetCashOut(7,true,true), expected to not receive any output from failed assertions. Expect to call ReturnCashOutTaxed method in Greeting. Expect the random number multiplied by 10000 to equal the returned number from GetCashOut.
  • Test two: Call GetCashOut(7,false,true), expected to not receive any output from failed assertions. Expect to call ReturnCashOutNotTaxed method in Greeting. Expect the random number multiplied by 10000 to equal the returned number from GetCashOut.
  • Test three: Call GetCashOut(6,false,true), expected to not receive any output from failed assertions. Note that 6 is just an arbitrarily chosen number, 7 is the winning number. Expect to call ReturnCashDeducted method in Greeting. Expect number returned from GetCashOut to equal -10.
  • Test four: Call GetCashOut(6,false,false), expected to receive an exception.

Greeting only writes "line" to the listener when an assertion evaluates to false. So in the tests, if a test finds "line", we know an assertion evaluated to false, and the test fails.

Assertion.Assert(strDebugOut, strDebugOut.IndexOf("line") < 0);

Notice that we cannot test the returned value by using black box testing techniques. Testing the returned value requires knowing the internal random value. Instead, in the preceding code, we remedy this problem by using Debug.WriteLine.

Debug.WriteLine("<random>" + dblRandom + "</random>" );

Then, when writing the test, you can write code similar to the following to obtain an internal value:

String strWinning =
            strDebugOut.Substring(
strDebugOut.IndexOf("<random>")+8,
strDebugOut.IndexOf("</random>") -
(strDebugOut.IndexOf("<random>")+8) );

Finally, note that you can also test the branch taken in a component by using Debug.WriteLine.

Debug.WriteLine("in_ReturnCashDeducted");

Then, assert that the string is in the debug output in your tests.

Assertion.Assert("Path wrong?",
strDebugOut.IndexOf("in_ReturnedCashOutTaxed") > 0);

The following code illustrates these techniques by testing the Greeting class:

namespace Greetings {
using
            System;
using NUnit.Framework;
using System.IO;
using
            System.Diagnostics;

[TestFixture]
public class
            classGreetingTest{
Greeting objGreeting = new
            Greeting();

//
            ----------------------------------------------------- //
// Test
            Path A - Test one //
//
            -----------------------------------------------------
            //

[Test]
public void
            GetCashOutTestPathA(){

StringWriter objWriter = new
            StringWriter();
TextWriterTraceListener objListener =
            new
TextWriterTraceListener(objWriter);

Debug.Listeners.Clear();
Debug.Listeners.Add(objListener);

Assertion.Assert(objGreeting.GetCashOut(7,true,true)
            > 0);

String strDebugOut = objWriter.ToString();

//
            Test that no Debug.Assert statements were false
            //

Assertion.Assert(strDebugOut, strDebugOut.IndexOf("line")
            < 0);
// Test path A taken //

Assertion.Assert("Path
            wrong?",
strDebugOut.IndexOf("in_ReturnedCashOutTaxed") >
            0);

objWriter.Close();
objListener.Close();
}

[Test]
public
            void GetCashOutTestTheAmountPathA(){

StringWriter objWriter =
            new StringWriter();
TextWriterTraceListener objListener =
            new
TextWriterTraceListener(objWriter);

Debug.Listeners.Clear();
Debug.Listeners.Add(objListener);

//
            Get winnings - user is winner, pays tax, and is over 18
            //

double amount =
            objGreeting.GetCashOut(7,true,true);

String strDebugOut =
            objWriter.ToString();

// get the random number from Debug
            //

String strWinning =
            strDebugOut.Substring(
strDebugOut.IndexOf("<random>")+8,
strDebugOut.IndexOf("</random>") -
(strDebugOut.IndexOf("<random>")+8) );

// Test that
            the amount returned is what was expected
            //

Assertion.AssertEquals("amount not equal to what was
            expected",
System.Convert.ToDouble(strWinning) * 1000,
            amount);
}

//
            ----------------------------------------------------- //
// Test
            Path B - Test two //
//
            -----------------------------------------------------
            //

[Test]
public void
            GetCashOutTestPathB(){

StringWriter objWriter = new
            StringWriter();
TextWriterTraceListener objListener =
            new
TextWriterTraceListener(objWriter);

Debug.Listeners.Clear();
Debug.Listeners.Add(objListener);

Assertion.Assert(objGreeting.GetCashOut(7,false,true)
            > 0);

String strDebugOut = objWriter.ToString();

//
            Test that no Debug.Assert statements were false
            //

Assertion.Assert(strDebugOut, strDebugOut.IndexOf("line")
            < 0);

// Test path B taken
            //

Assertion.Assert("Path
            wrong?",
strDebugOut.IndexOf("in_ReturnedCashOutNotTaxed") >
            0);

objWriter.Close();
objListener.Close();
}

[Test]
public
            void GetCashOutTestTheAmountPathB(){

StringWriter objWriter =
            new StringWriter();
TextWriterTraceListener objListener =
            new
TextWriterTraceListener(objWriter);

Debug.Listeners.Clear();
Debug.Listeners.Add(objListener);

//
            Get winnings - user is winner, doesn't pay tax, and is over 18
            //

double amount =
            objGreeting.GetCashOut(7,false,true);

String strDebugOut =
            objWriter.ToString();

// get the random number from Debug
            //

String strWinning =
            strDebugOut.Substring(
strDebugOut.IndexOf("<random>")+8,
strDebugOut.IndexOf("</random>") -
(strDebugOut.IndexOf("<random>")+8) );

// Test that
            the amount returned is what was expected
            //

Assertion.AssertEquals("amount not equal to what was
            expected",
System.Convert.ToDouble(strWinning) * 1000,
            amount);
}

//
            ----------------------------------------------------- //
// Test
            Path C - Test three //
//
            -----------------------------------------------------
            //

[Test]
public void
            GetCashOutTestPathC(){

StringWriter objWriter = new
            StringWriter();
TextWriterTraceListener objListener =
            new
TextWriterTraceListener(objWriter);

Debug.Listeners.Clear();
Debug.Listeners.Add(objListener);

Assertion.Assert(objGreeting.GetCashOut(6,false,true)
            > 0);

// Test path C taken
            //

Assertion.Assert("Path
            wrong?",
strDebugOut.IndexOf("in_ReturnedCashDeducted") >
            0);

String strDebugOut = objWriter.ToString();

// Test
            that no Debug.Assert statements were false
            //

Assertion.Assert(strDebugOut, strDebugOut.IndexOf("line")
            <
            0);

objWriter.Close();
objListener.Close();
}

[Test]
public
            void GetCashOutTestTheAmountPathC(){

StringWriter objWriter =
            new StringWriter();
TextWriterTraceListener objListener =
            new
TextWriterTraceListener(objWriter);

Debug.Listeners.Clear();
Debug.Listeners.Add(objListener);

//
            Get winnings - user is not winner, doesn't pay tax, is over 18
            //

double amount =
            objGreeting.GetCashOut(6,false,true);

// Test that the amount
            returned is what was expected
            //

Assertion.AssertEquals("amount not equal to what was
            expected", -10,
amount);
}

//
            ----------------------------------------------------- //
// Test
            Path D - Test four //
//
            -----------------------------------------------------
            //

[Test]
[ExpectedException(typeof(System.Exception))]
public
            void Under18Test(){
double amount =
            objGreeting.GetCashOut(6,false,false);
}

}
}

Conditional Testing

While branch testing focuses on ensuring a program follows a particular path, conditional testing focuses on the condition that might have led to the path taken. You use conditional testing to test complex logic. For example, suppose we had to develop a simple component that simulated a baby's decision making when deciding to eat food, play with the food, or throw the food:

  • If food is bad, baby is angry, and mom looks like she might give a spanking if baby is bad - refuse food.
  • If food is bad, baby is angry, and mom does not look like she might give a spanking if baby is bad - throw bowl.
  • If food is bad, baby is mischievous, and mom looks like she might give a spanking if baby is bad - eat food.
  • If food is bad, baby is mischievous, and mom does not look like she might give a spanking if baby is bad - refuse food.
  • If food is bad and baby is tired - sleep.
  • If food is good - eat food.

We can model these specifications using a decision tree:

We can also model the logical conditions using a truth table.

 | Mood:Angry | Mood:Mischievous | Mood:Tired | Delicious | Mom will spank
Throw Bowl | true | false | false | false | false
Refuse Food | true | false | false | false | true
Refuse Food | false | true | false | false | false
Sleep | false | false | true | false | na
Eat | false | true | false | false | true
Eat | na | na | na | true | na

If baby throws his or her bowl, then we know the following is true:

enumBabyReaction == BabyReaction.ThrowBowl && mood == BabyMood.Angry && !blnFoodGood && !blnMomSpank

If the baby refuses food, then we know the following is true:

enumBabyReaction == BabyReaction.RefuseFood
            && ((mood == BabyMood.Angry &&
blnMomSpank) ||
            (mood == BabyMood.Mischievous && !blnMomSpank))
            &&
!blnFoodGood

If baby eats food, then we know the following is true:

enumBabyReaction == BabyReaction.EatFood && ( blnFoodGood || (mood ==
BabyMood.Mischievous && blnMomSpank))

The complete Baby class follows:

namespace AssertsAndDebug {
using System;
using System.Diagnostics;

public class Baby {

public enum BabyReaction{ThrowBowl,EatFood,RefuseFood,Sleep};
public enum BabyMood{Angry,Mischievous,Tired};

private BabyMood mood;

public BabyMood Mood{
set{mood = value;}
}

private bool blnMomSpank;

public bool MomSpank{
set{blnMomSpank = value;}
}

// Determine baby's reaction based upon several factors //

public BabyReaction GetFood(bool blnFoodGood){

BabyReaction enumBabyReaction = BabyReaction.EatFood;

if (blnFoodGood) enumBabyReaction = BabyReaction.EatFood;
else{

switch (mood) {
case BabyMood.Angry :
if (blnMomSpank) enumBabyReaction = BabyReaction.RefuseFood;
else enumBabyReaction = BabyReaction.ThrowBowl;
break;
case BabyMood.Mischievous :
if (!blnMomSpank) enumBabyReaction = BabyReaction.RefuseFood;
break;
case BabyMood.Tired :
enumBabyReaction = BabyReaction.Sleep;
break;
}

// note: using fact that this code is only executed when in
// debug mode. be carefull for side-effects

Debug.Assert(this.BabyReactionTest(enumBabyReaction,
blnFoodGood),"Failed BabyReactionTest Baby.cs line 38");

return enumBabyReaction;
}

// Test that ensures the conditions are correct for baby's reaction //

private bool BabyReactionTest(BabyReaction enumBabyReaction,
bool blnFoodGood){

bool passed = false;

if (enumBabyReaction == BabyReaction.ThrowBowl
&& mood == BabyMood.Angry
&& !blnFoodGood
&& !blnMomSpank
)
passed = true;
else if (
enumBabyReaction == BabyReaction.RefuseFood
&& (
(mood == BabyMood.Angry && blnMomSpank)
||
(mood == BabyMood.Mischievous && !blnMomSpank)
)
&& !blnFoodGood
)
passed = true;
else if (enumBabyReaction == BabyReaction.EatFood
&& ( blnFoodGood
||
(mood == BabyMood.Mischievous && blnMomSpank)
)
)
passed = true;
else if (enumBabyReaction == BabyReaction.Sleep
&& mood == BabyMood.Tired
)
passed = true;

return passed;
}

}
}

When unit testing the GetFood method, you need to ensure every logical possible action is invoked. To do this, we need to understand what causes different reactions. For example, when the food is bad, mommy will not spank, and baby is angry, GetFood returns ThrowBowl. The following test initializes the baby component so that GetFood returns ThrowBowl :

[Test]
public void
            TestThrowBowl(){
this.ReInitializeWriters();
this.objBaby.MomSpank
            = false;
this.objBaby.Mood =
            Baby.BabyMood.Angry;
this.objBaby.GetFood(false);

String
            test = objWriter.ToString();
if
            (objWriter.ToString().Trim().Length >
            0){
Assertion.Fail(objWriter.ToString().Trim());
}
}

The following code tests every possible Reaction from GetFood :

namespace AssertsAndDebug {
using
            System;
using NUnit.Framework;
using
            System.Diagnostics;
using
            System.IO;

[TestFixture]
public class BabyTest
            {

private Baby objBaby = new Baby();
private StringWriter
            objWriter = new StringWriter();
private TextWriterTraceListener
            objTextListener;

[SetUp]
public void
            Init(){
objTextListener = new
            TextWriterTraceListener(objWriter);
Debug.Listeners.Clear();
Debug.Listeners.Add(objTextListener);
}

private
            void ReInitializeWriters(){
objWriter.Close();
objWriter = new
            StringWriter();
objTextListener.Writer =
            objWriter;
}

[Test]
public void
            TestThrowBowl(){
this.ReInitializeWriters();
this.objBaby.MomSpank
            = false;
this.objBaby.Mood =
            Baby.BabyMood.Angry;
this.objBaby.GetFood(false);

String
            test = objWriter.ToString();
if
            (objWriter.ToString().Trim().Length >
            0)
{
Assertion.Fail(objWriter.ToString().Trim());
}
}

[Test]
public
            void
            TestRefuseFood(){
this.ReInitializeWriters();

this.objBaby.MomSpank
            = true;
this.objBaby.Mood =
            Baby.BabyMood.Angry;
this.objBaby.GetFood(false);

this.objBaby.MomSpank
            = false;
this.objBaby.Mood =
            Baby.BabyMood.Mischievous;
this.objBaby.GetFood(false);

String
            test = objWriter.ToString();
if
            (objWriter.ToString().Trim().Length >
            0)
{
Assertion.Fail(objWriter.ToString().Trim());
}
}

[Test]
public
            void
            TestEat(){
this.ReInitializeWriters();

this.objBaby.GetFood(true);

String
            test = objWriter.ToString();
if
            (objWriter.ToString().Trim().Length >
            0)
{
Assertion.Fail(objWriter.ToString().Trim());
}
}

[Test]
public
            void
            TestSleep(){
this.ReInitializeWriters();

this.objBaby.MomSpank
            = false;
this.objBaby.Mood =
            Baby.BabyMood.Tired;
this.objBaby.GetFood(false);

String
            test = objWriter.ToString();
if
            (objWriter.ToString().Trim().Length >
            0)
{
Assertion.Fail(objWriter.ToString().Trim());
}
}

[TearDown]
public
            void
            Dispose(){
objWriter.Close();
objTextListener.Close();
}
}
}

Looping constructs like while loops also use logical conditions to determine when to stop looping. Suppose you have a component that allows you to drain a bucket. However, you can only drain it to a minimum level of 25 units and you only have 100 "cycles" that you can drain the bucket.

this.level > intDesiredLevel && this.level > MINACCEPTABLE && intCycles < MAXCYCLES

Thus, we know that after starting the drain action, it will only stop when the following condition evaluates to true.

(this.level == intDesiredLevel)|| (this.level == MINACCEPTABLE) || intCycles == MAXCYCLES)

Knowing this limitation, we can write an assertion that tests if this condition is true when draining stops.

namespace Examples {
using
            System;
using System.Diagnostics;

public class Drainer
            {

private int level;
private static int MAXCYCLES =
            100;
private static int MINACCEPTABLE = 25;

public
            Drainer(int level){
if (level < MINACCEPTABLE)
this.level =
            MINACCEPTABLE;
else
this.level = level;
}

public
            void Drain(int intDesiredLevel){
int intCycles =
            0;
Console.WriteLine("Draining");
while(this.level >
            intDesiredLevel &&
this.level > MINACCEPTABLE
            &&
intCycles <
            MAXCYCLES)
{
this.level--;
intCycles++;
Console.Write(".");
}
Debug.Assert((this.level
            == intDesiredLevel)||
(this.level == MINACCEPTABLE) ||
            (intCycles == MAXCYCLES),
"Assertion did not hold. Drainer.cs
            line 26.");

Debug.Write("<level>" + this.level +
            "</level>");
Debug.Write("<cycles>" + intCycles +
            "</cycles>");
}
}
}

We can then write the test. Using the same technique as before, we write a unit test that tests the debug output for the "line" string. To adequately cover the while loop, we must consider three conditions, or, the following three test cases:

  • Test one: initialize level to 900, set drain level to 400, expect the cycles to equal 100.
  • Test two: initialize level to 100, set drain level to 96, expect the level to equal 96.
  • Test three: initialize level to 27, set drain level to 12, expect the level to equal 25.

DrainTest implements these tests in the following NUnit test fixture:

namespace Examples {
using
            System.Diagnostics;
using System.IO;
using
            NUnit.Framework;

[TestFixture]
public class
            DrainerTest{

StringWriter
            objWriter;
TextWriterTraceListener
            objTextWriterTraceListener;

[SetUp]
public void
            Init(){
objWriter = new
            StringWriter();
objTextWriterTraceListener =
new
            TextWriterTraceListener(objWriter);

Debug.Listeners.Clear();
Debug.Listeners.Add(objTextWriterTraceListener);
}

private
            void ReInitialize(){
objWriter = new
            StringWriter();
objTextWriterTraceListener.Writer =
            objWriter;
}

// Test exit from maximum cycles being
            reached //

[Test]
public void
            TestDrainMaxCycles()
{
Drainer objDrainer = new
            Drainer(900);
objDrainer.Drain(400);
Assertion.Assert(objWriter.ToString().Trim(),
objWriter.ToString().IndexOf("line")
            <
            0);
Assertion.Assert(objWriter.ToString().Trim(),
objWriter.ToString().IndexOf("<cycles>100</cycles>")
            > -1);

}

// Test exit from desired level being
            reached //

[Test]
public void
            TestDesiredLevel(){
this.ReInitialize();
Drainer objDrainer =
            new
            Drainer(100);
objDrainer.Drain(96);
Assertion.Assert(objWriter.ToString().Trim(),
objWriter.ToString().IndexOf("line")
            <
            0);
Assertion.Assert(objWriter.ToString().Trim(),
objWriter.ToString().IndexOf("<level>96</level>")
            > -1);
}

// Test exit from minimum level being reached
            //

[Test]
public void
            TestMinLevel(){
this.ReInitialize();
Drainer objDrainer = new
            Drainer(27);
objDrainer.Drain(12);
Assertion.Assert(objWriter.ToString().Trim(),
objWriter.ToString().IndexOf("line")
            <
            0);
Assertion.Assert(objWriter.ToString().Trim(),
objWriter.ToString().IndexOf("<level>25</level>")
            > -1);
}
}
}

Data Definition and Use Testing

The data definition and use testing test is a white box testing technique that tests a data's definition and its later use. Using a watch in a debugger, for example, is an example of data definition and use testing. You test the variable's initialization and then you test its different values at different points in the application. The goal is writing tests that drive the program to initialize and then use data in certain ways. You are consciously testing how your program uses different variables.

Now, before we continue with an example, no, you would never want to program like this at home. Yes, using unsafe code, character arrays, and pointers like this in C# is irresponsible, at best. And no, the "fixes" described below are not really fixes at all. But the following example illustrates data definition and use testing, and why it is important, so suspend disbelief. Consider the following unsafe code example.

namespace Examples {
using
            System;
using System.Diagnostics;
using
            System.IO;

public class UnsafeExample {

// copy
            characters from one character array to another
// note that if
            destination is longer then source, and
// destination has
            characters already in it, those chars
// remain. i.e.
// src:
            test
// dest: hey now
// dest after copying:
            testnow

private static unsafe void DoSomething(char[] src,
            char[] dst){
int srcLen = src.Length;
int dstLen =
            dst.Length;

fixed (char* pSrc = src, pDst = dst){
char*
            ps = pSrc;
char* pd = pDst;

for (int i = 0; i < srcLen;
            i++){
*pd =
            *ps;
pd++;
ps++;
}

}

}

public static
            StringWriter DoIT (){

char[] initial = {'Y','O','U', '
            ','T','U','R','K','E','Y'};;
char[] outputMessage = {'Y','O','U',
            ' ','A','R','E',' ','1','!',
'
            ','W','R','I','T','E','R'};

DoSomething(initial,outputMessage);

StringWriter
            objWriter = new StringWriter();

// note: error here - the
            length of the string
// in the stringwriter is 17 "YOU
            TURKEY\0\0\0\0\0\0\0"
// note that \0 is a null char in C
            language

for(int i=0; i < outputMessage.Length; ++i)
objWriter.Write(outputMessage[i]);

return
            objWriter;
}

}
}

The UnsafeExample has two methods, DoSomething and DoIT. DoSomething takes two character arrays as parameters. The first array is the source, while the second is the destination. DoIt initializes two character arrays and then passes them to DoSomething. DoSomething uses pointers to copy the source to the destination. DoIT then copies the destination character array to a StringWriter and returns the StringWriter.

But note the glaring error in the preceeding code - src only overwrites the first ten characters of dst, leaving the remaining seven characters intact. Instead of YOU TURKEY, we obtain YOU TURKEY WRITER. We need to ensure that dst only contains the characters from src. One way we can do this is by checking that src always ends with null.

private static unsafe void
            DoSomething(char[] src, char[] dst){
int srcLen =
            src.Length;
int dstLen = dst.Length;

// assert that the
            last character of src is null string
            terminator

Debug.Assert(src[src.Length-1] == '\0',
"Last
            char not a null. UnsafeExample.cs:line 13");

// we would add
            exception here //

fixed (char* pSrc = src, pDst =
            dst){
char* ps = pSrc;
char* pd = pDst;

for (int i = 0;
            i < srcLen; i++){
*pd = *ps;
pd++;
ps++;
}
}

However, even after fixing DoSomething, there remains an error in DoIT. The outputMessage character array is 17 characters long. If initial is terminated with a \0, then when initial overwrites outputMessage in DoSomething, the first ten characters are overwritten while the remaining seven characters are null. A NUnit assertion like the following would fail.

Assertion.AssertEquals("YOU TURKEY", strMessage);

The string is YOU TURKEY\0\0\0\0\0\0\0 not YOU TURKEY. To fix this problem, only copy outputMessage to the first null.

public static StringWriter DoIT (){
char[] initial = {'Y','O','U', ' ','T','U','R','K','E','Y','\0'};;
char[] outputMessage = {'Y','O','U', ' ','A','R','E',' ','1','!',
' ','W','R','I','T','E','R'};

DoSomething(initial,outputMessage);

StringWriter objWriter = new StringWriter();

// only copy up to the first null

for(int i=0; outputMessage[i] != '\0'; ++i)
objWriter.Write(outputMessage[i]);

return objWriter;
}

We can now write the NUnit test and it passes, albeit the code it is testing is something you would never want to do in the first place.

namespace Examples {
using
            System.Diagnostics;
using System.IO;
using System;
using
            NUnit.Framework;

[TestFixture]
public class
            UnsafeExampleTest{

StringWriter
            objWriter;
TextWriterTraceListener
            objTextWriterTraceListener;

[SetUp]
public void
            Init(){
objWriter = new
            StringWriter();
objTextWriterTraceListener = new
            TextWriterTraceListener(objWriter);

Debug.Listeners.Clear();
Debug.Listeners.Add(objTextWriterTraceListener);
}

private
            void ReInitialize(){
objWriter = new
            StringWriter();
objTextWriterTraceListener.Writer =
            objWriter;
}

[Test]
public void UnsafeTest
            (){
String strMessage =
            UnsafeExample.DoIT().ToString().Trim();
Assertion.Assert(objWriter.ToString().Trim(),
objWriter.ToString().IndexOf("line")
            < 0);
Assertion.AssertEquals("YOU TURKEY",
            strMessage);
}
}
}

In the preceding code, notice how we analyzed and fixed the problems. We first noticed that the code did not initialize initial (which became src) with a \0 as the last character. We next found that when the code used dst and src, it did so incorrectly. Instead of overwriting dst with all of src, it only overwrote the first ten characters. After fixing this problem, we then noticed that the remaining characters in dst were overwritten with nulls. We did not obtain src, but instead, obtained src with trailing nulls. We fixed that problem and the function worked as expected.

Summary

Concurrent unit testing and coding will help you write robust components. A tool like NUnit facilitates writing new unit tests and automating regression testing. Although the distinction between black box testing and white box testing is unclear when discussing unit testing, there are several black box and white box testing techniques you can employ to unit test your code. When black box testing; specification oriented testing, equivalence partitioning, boundary value analysis, and state transition testing are just four of many available testing techniques. When white box testing; branch testing, conditional testing, and data definition and use testing are three of many available testing techniques.

Unit testing your code using proven techniques is more productive then random unit testing. Black box tests test that for given inputs you obtain expected outputs. When the output does not match the value expected, the test fails. White box testing tests that a unit's internals function correctly. By adding assertions and debugging statements, you test that certain conditions are true as the unit runs. There are many ways you can use assertions and debugging statements. This case study does not portend to be a discussion of the best way to use assertions and debugging statements. However, it does illustrate that you should use them so that their output is available to NUnit tests if you want to automate your testing.

C#Today - © 2002 WroxPress Limited


Effective Data Access in C#

Effective Data Access in C#
http://www.csharptoday.com/content.asp?id=2045
Article Rating: 4.5
Published on December 9, 2002
Abstract
The vast majority of applications we design today require database access. It is true to say that the database layer of any application is where the performance of an application can be either seriously enhanced or severely hindered. With the advent of .NET, a whole new paradigm for data access has evolved, but many people are still employing architectures that worked for COM or other technologies, without looking at the benefits to be gained from the new features available in the .NET Framework. During this case study, Rob Harrop looks at how we can create an object oriented framework to represent our data and an efficient, lightweight framework for reading from and writing to the database.
Case Study

This case study highlights a variety of techniques to design and create an effective, efficient, and manageable data access API using C#.

Introduction

The vast majority of applications we design today whether they are process management applications, e-commerce applications, or any other enterprise application require database access. It is also true to say that the database layer of any application is where the performance of an application can be either seriously enhanced or severely hindered.

With the advent of .NET a whole new paradigm for data access has evolved, but many people are still employing architectures that worked for COM or other technologies, without looking at the benefits to be gained from the new features available in the .NET Framework.

During this case study I will look at how we can create an object oriented framework to represent our data and an efficient, light-weight framework for reading from and writing to the database.

System Requirements

It goes without saying that the .NET Framework is a mandatory requirement of this case study. In addition, you will also need SQL Server 2000 and Internet Information Services with ASP.NET running and configured.

I used Visual Studio .NET to create this application and the solution file is available in the download. You can however edit the source with any text editor and compile it using the command line compiler from the SDK

Defining the Problem

To highlight the problem of data access I am going to create an almost fully featured on-line DVD store. I say "almost" given that we won't actually be selling real DVD's. Thank you to Amazon.com for the information and pictures of the DVD boxes.

Case Study Structure

I am big supporter of the think-before-you-act methodology of software design, so for the purposes of this case study we are going to create a fairly rigorous design before we get down to any real coding.

The case study will have the following structure:

  • Analyzing our problem
  • Database Design and Structure
  • Mapping Business Entities to Tables
  • Stored Procedures
  • Transaction Processing
  • Component Design and Coding
  • Modeling our data
  • Accessing Data
  • Inserting, Updating and Deleting Data
  • UI Design and Coding
  • Creating the catalogue
  • Creating the checkout

Analyzing the Problem

When designing any application it is useful to have a thorough understanding of what the application is meant to do. With this understanding, we can identify the key entities of our application. It is these entities that will form the backbone of our application data.

Defining Application Entities

With the DVD Store we can simply state that we wish for our customers to be able to purchase films in DVD format. From that simple statement we have identified the following entities:

  • Customer
  • DVD Film
  • Order

Now we can also state that we want our customers to be able to view the stars of any DVD film on the web site, and also search for a film starring a particular actor or actress. We have now defined another entity, the Actor.

From this simple exercise we have built an understanding of the application and its requirements. In the statements above we have also managed to define the relationships between our entities:

  • A Customer places an Order
  • An Order contains DVD Films
  • An Actor stars in a DVD Film

Upon closer inspection, we see that these definitions aren't entirely accurate. For instance an Actor, unless they are a particularly bad actor, will star in more than one DVD Film, and a DVD Film will rarely have just one star.

In addition to this, we are hoping that our DVD Store will be successful and that Customers will place Orders for more than one DVD Film, and indeed that they will return to the web site and place further Orders.

So our definition of entities and their relationships becomes:

  • A Customer places zero or more Orders
  • An Order may contain one or more DVD Films
  • A DVD Film may star one or more Actors
  • An Actor can star in zero or more DVD Films

Defining our Actions

Having defined the entities in our application, we now need to define how these entities will interact with each other.

When defining our entities, we have already defined some of our actions, such as Customer places an Order. We also need to define other actions that can take place within the application scope.

Because our Customers will have to provide us with their personal details, they will have to register their details on the web site. Once a customer has registered, we need to be able to retrieve their details to validate them at login time and also to generate an order confirmation e-mail.

Also, the web site won't be much use without search and catalogue facilities; specifically we want to:

  • Display the three most recent additions to the catalogue
  • Enable users to perform a keyword search for a DVD
  • Enable users to view the full details of a DVD along with a list of the stars
  • Enable users to view the full catalogue
  • Enable users to search for a DVD starring a particular Actor

Database Design and Structure

Taking our entity and relationship definitions we can design our database.

Err on the side of storing data

When defining relationships, and in the interest of maintaining a fully normalized database, it is recommended that the same data is not stored in more than one place. This process leads to the least amount of data repetition but sometimes data needs to be repeated and this case study highlights a perfect example.

We are going to be storing DVD Films (Titles) in our database along with their current price. Also, we are going to be storing Orders along with a list of the Titles ordered. It is tempting then to design our database thus:

In the example above we are saying that each Order can contain one or more OrderItems and that each OrderItem is linked to one Title. At first glance this looks fine, but when management turn around and say 'OK, now we want to see the total value of all our Orders' how can you accurately show that information given that the value in the SalePrice field of the Titles table may not be the same as is was when the Order was placed? What we need is an arrangement like this:

Here we have added a UnitPrice field to the OrderItems table. Now we can determine the price of that Title as of the date the order was placed.

There are plenty of situations where information may be required in the future, and it requires careful analysis as to whether you store that information separately from its original entity as in the above example. As the title of this section implies, you should err on the side of storing data; you never know when it might be needed!

Defining Primary Keys

When defining key for a table it is often the case that the data will have a natural key. In the case of the Customers table, the e-mail address field would be a perfectly good key field, but I have opted to use a simple 32 bit identity field. The reason for this is merely down to performance; 32-bit computers are inherently more proficient at dealing with 32 bit integer values than they are with say a variable length string, so indexing and searching on a 32 bit value will be faster. Also, we are inevitably passing less data across the network when we do a query based on a 32 bit integer than with a variable length string.

You may not be convinced that the saving on internal network bandwidth will be that great, but consider the situation where you output a list of items to a browser with a link to more details for each item. The link will contain (more than likely) the primary key of that record. Using an integer will mean sending less data to the browser than if we were to send a variable length string.

Many-to-Many Relationships

Most applications will at some point define entities that have a many-to-many relationship. For example, in our application, we have defined that an Actor may star in many DVD Films and each DVD Film may contain many Actors.

It is simple to model this in a relational database using a link table, but one thing I must point out is that the link table does not constitute an entity. We can model the many-to-many relationship directly within our code without having to create a new entity. This is extremely bad coding practice, but you would be surprised at how many people actually code like this.

The Schema

Having analyzed our entities and relationships, we can now create our design. I have used the SQL Server built-in modeling tools to draw this diagram, but any database modeling tool can be used.

You will notice also that I have added a further entity, Address, to the database design. We want our customers to be able to save their address details for when they pay us a return visit. Also we want to allow customers to have more than one address configured.

Because we are not going to allow customers to delete or modify addresses once they are added to the system, I have used foreign keys to reference the ShipTo and BillTo addresses.

In the Titles table you will notice that I have included a field called BoxShot which stores a path to a photo of the DVD. This method is preferable to storing the data in the database, as you will not need to stream the data in and out of the database server.

Stored Procedures

During our analysis we defined a distinct set of actions required to make the web site function as required. We can match these actions to the data they will manipulate, and we can perform the data manipulation.

Managing Transactions

We could opt to manage our transactions within our stored procedure logic, but this is an unwise choice as it does not facilitate easy changes to our transaction model. If we manage transactions within our data access component, we can change how these transactions are managed without affecting our stored procedures.

Retrieving Identity Values

We are making extensive use of identity values in our tables, and we will need to return the value of an identity column from an INSERT procedure on the vast majority of our tables. We can do this in one of two ways:

  • Using Output Parameters
  • Using the ExecuteScalar method of the SqlCommand object

When using Output Parameters we would specify something like SET @CustomerId = SCOPE_IDENTITY() at the end of our stored procedure, whereas with ExecuteScalar we would code something like SELECT SCOPE_IDENTITY(). Our application will make use of the ExecuteScalar method for reasons that will become apparent when we code the data access components.

From a performance point of view, there is no real difference in using either method as these graphs from a recent MSDN article show:

Scope of Logic

Generally, when creating stored procedures, it is best to stray away from putting anything but the bare minimum of logic within our procedures. We need to keep the database server free from having to take care of logic that could easily be delegated to a business logic component running on a separate server.

For this case study our stored procedures only retrieve, insert, or update data using the values supplied by the data access components. No logic, other than that required to perform a successful SELECT, INSERT, or UPDATE will take place within stored procedures.

This has the additional benefit of making a move to another database vendor simpler than it would be if all the logic in the stored procedures had to be re-implemented on another database server

Customer Account Procedures

During our analysis we decided that a customer should be able to register their details on the web site and that we need to be able to retrieve those details at a later date. We can model this in two stored procedures:

CREATE PROCEDURE [InsertCustomer]
(@FirstName [varchar](50),
@LastName [varchar](50),
@EmailAddress [varchar](50),
@Password [varchar](50))

AS INSERT INTO [dvdstore].[dbo].[Customers]
( [FirstName],
[LastName],
[EmailAddress],
[Password])

VALUES
( @FirstName,
@LastName,
@EmailAddress,
@Password)

SELECT SCOPE_IDENTITY()

This stored procedure is the fairly standard form of an insert procedure, returning the inserted identity value with a simple SELECT statement.

Next we need to retrieve the Customer details from the database. This will need to be done when the Customer logs in after supplying an e-mail address and password, and also by the application when we display customer details on screen or when we create the order confirmation e-mail. To do this, we will create a simple procedure to retrieve a customer based on the e-mail address. We can verify the password in our code.

CREATE PROCEDURE GetCustomerByEmailAddress
@EmailAddress [varchar] (50)
AS

SELECT AccountId, FirstName, LastName, EmailAddress, [Password] FROM Customers WHERE EmailAddress = @EmailAddress

We mustn't forget that in addition to the basic account details for a Customer, we will need to retrieve the Addresses for that Customer and also allow a Customer to insert a new Address.

Inserting the Address is a simple affair; note that this time the identity is not returned from the stored procedure, as we will not need to use it immediately after an INSERT has been performed:

CREATE PROCEDURE [InsertAddress]
(@AccountId [int],
@Address1 [varchar](50),
@Address2 [varchar](50),
@Address3 [varchar](50),
@City [varchar](50),
@County [varchar](50),
@PostCode [varchar](50))

AS INSERT INTO [dvdstore].[dbo].[Addresses]
( [AccountId],
[Address1],
[Address2],
[Address3],
[City],
[County],
[PostCode])

VALUES
( @AccountId,
@Address1,
@Address2,
@Address3,
@City,
@County,
@PostCode)

Retrieving Addresses for a Customer is simple; we require one input parameter, the Customer's AccountId, and we return the full set of associated Addresses:

CREATE PROCEDURE GetCustomerAddresses
@CustomerAccountId [int]
AS

SELECT AddressId, AccountId, Address1, Address2, Address3, City, County, PostCode
FROM Addresses WHERE AccountId = @CustomerAccountId

That covers the functionality required to enable Customers to register with site, log in, and add new addresses.

Providing Search and Catalogue Functionality

We also defined a set of searches and catalogue functions we wanted to use on the site. The simplest of these allows the Customer to view all the DVDs in the catalogue:

CREATE PROCEDURE GetAllTitles AS

SELECT TitleId, [Name], [Description], BoxShot, SalePrice, AddedOn FROM Titles

Showing the latest three Titles to be added to the database on the home page is not difficult. We use the AddedOn field to determine which three Titles were added last, and then return them:

CREATE PROCEDURE GetLatestAdditions AS

SELECT TOP 3 TitleId, [Name], [Description], [BoxShot], [SalePrice], [AddedOn] FROM Titles ORDER BY AddedOn DESC

Allowing the Customer to search for Titles by Actor requires two procedures: one so that we can display a list of Actors to prevent the Customer from having to type in an Actors name, and one to search based on an ActorId :

CREATE PROCEDURE GetAllActors AS

SELECT ActorId, FirstName, LastName PhotoPath FROM Actors

CREATE PROCEDURE GetTitlesByActor
@ActorId [int]
AS

SELECT t.TitleId, t.[Name], t.[Description], t.BoxShot, t.SalePrice, t.AddedOn
FROM Titles t INNER JOIN TitleActors ta ON t.TitleId = ta.TitleId WHERE ta.ActorId = @ActorId

To be able to display a full summary of a Title we will need to be able to retrieve a Title by TitleId and also retrieve a set of Actors by TitleId :

CREATE PROCEDURE GetTitleById
@TitleId [int]
AS

SELECT TitleId, [Name], [Description], BoxShot, SalePrice FROM Titles WHERE TitleId = @TitleId

CREATE PROCEDURE GetActorsByTitle
@TitleId [int]
AS

SELECT a.ActorId, a.FirstName, a.LastName, a.PhotoPath
FROM Actors a INNER JOIN TitleActors ta ON a.ActorId = ta.ActorId WHERE ta.TitleId = @TitleId

We also wanted the Customer to be able to perform a keyword search on the catalogue. Because I want to reduce the amount of processing the database server has to do, and to illustrate the flexibility of the data access framework we are about to build, I will roll up a custom SQL statement for the supplied keywords in the data access tier and issue the command to the database server.

Creating Orders in the Database

The only procedures that are left to create are those to allow us to enter an Order and its associated OrderItems into the database. For this we will create two procedures: one to insert the Order, the other to insert the OrderItems. We will then link these together in the data access component and wrap them in a transaction. The code for these procedures is fairly simple:

CREATE PROCEDURE [InsertOrder]
(@OrderedBy [int],
@OrderedOn [datetime],
@ShipTo [int],
@BillTo [int])

AS INSERT INTO [dvdstore].[dbo].[Orders]
( [OrderedBy],
[OrderedOn],
[ShipTo],
[BillTo])

VALUES
( @OrderedBy,
@OrderedOn,
@ShipTo,
@BillTo)

SELECT SCOPE_IDENTITY()

Having inserted the Order, the identity is returned and we can execute the next procedure to insert OrderItems :

CREATE PROCEDURE [InsertOrderItem]
(@OrderId [int],
@TitleId [int],
@Quantity [int],
@UnitPrice [money])

AS INSERT INTO [dvdstore].[dbo].[OrderItems]
( [OrderId],
[TitleId],
[Quantity],
[UnitPrice])

VALUES
( @OrderId,
@TitleId,
@Quantity,
@UnitPrice)

With that, we have finished the database design. Now we need to create the components to represent and access this data.

Component Design and Coding

For our component architecture we want to accomplish two things. Firstly we need a way to represent our entity data, and secondly we need to be able to read and write the entity data to and from the database.

Representing our Entities

When we are representing our entities in code we have a few more choices than we do with the database. We could use either of the .NET provided methods, the DataReader or the DataSet, or we could create our own entity classes.

What .NET Offers

Out of the box .NET makes available the DataReader and DataSet classes to describe our data. The DataReader is a streaming model, which maintains a connection to the database for the duration of the read process. DataReader is ideal for fast, forward-only access to a set of data, but it is not suited to being accessed as a remote object, and does not support any form of disconnected access. DataSet on the other hand is an in-memory representation of the data. In fact, a DataSet is an XML representation that is created internally using DataReader to query the database. The problem with DataSet is that it is often too much of a heavyweight approach. The DataSet is quite large when serialized and sent over the network (in a remoting situation); also the API is quite general and can produce unmanageable code, for instance:

// get a DataSet
DataSet myDataSet = GetDataSet(); // this function returns a DataSet

// access a field
int customerId = (int)myDataSetSet.Tables[0].Rows[0]["CustomerId"];

As you can see, accessing a field of a DataSet can be tedious and it is not strongly typed, which could prove to be a problem if you are coding a library for another programmer to access. What we need is an API that supports something similar to this:

// get a CustomerSet
CustomerSet customers = GetAllCustomers();

// get the id
Customer customer = customers[0];
int customerId = customer.CustomerId;

// or
customerId = customers[0].CustomerId;

As you can see, the API defined above is much simpler to access and manage. The CustomerSet class could be created in a variety of ways, one of which is to inherit from DataSet ; however, we are still retaining heavyweight features of the DataSet which we may not use. The other option is to model a set of classes to represent the data and load them from the database using DataReader, which is exactly what .NET does for DataSet.

BusinessEntity and BusinessEntitySet

For our solution we are going to create two abstract classes, BusinessEntity and BusinessEntitySet, which will act as generic representations of a single entity and a set of entities respectively. To ensure we can use these as we would a DataSet, we need to build support for the following functionality:

  • Data binding to controls such as DataGrid and DataList
  • Sorting
  • Serialization so we can use the entities in a remote environment

Building the BusinessEntity Class

The BusinessEntity class must be able to store an arbitrary number of fields and to be serialized for transport across a network. The classes that derive from BusinessEntity will implement the actual public properties such as CustomerId.

Firstly we need to create the class itself and mark it as Serializable. Also we want to implement the ISerializable interface to implement our own custom serialization methods. I recommend against relying on the serialization support provided by the framework. This relies on reflection and is fairly slow compared with a custom implementation.

using System;
using System.IO;
using
            System.Text;
using System.Runtime.Serialization;
using
            System.Xml;

namespace
            Wrox.DvdStore.Common
{
[Serializable]
public abstract class
            BusinessEntity : ISerializable
{
// object array to store the
            field data
private object[] fields;

public
            BusinessEntity()
{
// default
            constructor
}

}
}

For our implementation of ISerializable we will simple serialize and de-serialize the fields array. This ensures the object graph will be fairly small.

#region Implementation of ISerializable
public void GetObjectData(SerializationInfo info, StreamingContext context)
{
// add the fields array to the object graph
info.AddValue("Fields", fields);
}
public BusinessEntity(SerializationInfo info, StreamingContext context)
{
// read the fields array from the object graph
fields = (object[])info.GetValue("Fields", typeof(object[]));
}
#endregion

Also we want our derived classes to be able use typed accessors to the data in the fields array. To do this we simply create a set of GetXXX methods for the types we want:

#region Typed Field Accessors

// return a Byte at field index
protected byte GetByte(int index)
{
return Convert.ToByte(this[index]);
}

// return an Int16 at field index
protected short GetInt16(int index)
{
return Convert.ToInt16(this[index]);
}

// return an Int32 at field index
protected int GetInt32(int index)
{
return Convert.ToInt32(this[index]);
}

// return an Int64 at field index
protected long GetInt64(int index)
{
return Convert.ToInt64(this[index]);
}

// return a Decimal at field index
protected decimal GetDecimal(int index)
{
return Convert.ToDecimal(this[index]);
}

// return a Single(float) at field index
protected float GetSingle(int index)
{
return Convert.ToSingle(this[index]);
}

// return a Double at field index
protected double GetDouble(int index)
{
return Convert.ToDouble(this[index]);
}

// return a DateTime at field index
protected DateTime GetDateTime(int index)
{
return Convert.ToDateTime(this[index]);
}

// return a Boolean at field index
protected bool GetBoolean(int index)
{
return Convert.ToBoolean(this[index]);
}

// return a Char at field index
protected char GetChar(int index)
{
return Convert.ToChar(this[index]);
}

// return a String at field index
protected string GetString(int index)
{
return Convert.ToString(this[index]);
}

// return a Guid at field index
protected Guid GetGuid(int index)
{
if((this[index] == null) || (this[index] != DBNull.Value))
return (Guid)this[index];
else
return Guid.Empty;
}

// return an Object at field index
protected object GetValue(int index)
{
return this[index];
}
// set the value of field index
protected void SetValue(int index, object value)
{
this[index] = value;
}
#endregion

That will allow for pretty much any data type you will need to be read from the fields of the BusinessEntity.

Using System.Activator

When we are creating a BusinessEntity will use the System.Activator class to create an instance of the derived class. For instance, we could have code such as:

BusinessEntity entity = (BusinessEntity)Activator.CreateInstance(typeof(Customer));

This will actually create a class of type Customer ; this is very useful given that we can create one method to load any class that is derived from BusinessEntity from the database.

Creating a BusinessEntity from the database

Now we need to specify a way in which we can create a BusinessEntity from the objects in a SqlDataReader. The SqlDataReader class has a method GetValues(object data) which will populate an object array with fields of it current row. We need to create two methods that will be called from BusinessEntitySet to create a BusinessEntity of a particular type and populate the field data. This method will be called by BusinessEntitySet to create a BusinessEntity for every row in the SqlDataReader.

public static BusinessEntity FromData(Type
            entityType, object[] data)
{
// create BusinessEntity of type
            entityType
BusinessEntity entity =
            (BusinessEntity)Activator.CreateInstance(entityType);

// Set
            the field data
entity.SetFields(data);

// return
return
            entity;
}

// only accessible from derived objects and
            objects in the
// same assembly
protected internal void
            SetFields(object[] data)
{
fields = data;
}

When we are creating a class derived from BusinessEntity without a DataReader we will need to be able to specify the number of fields to store in the fields array, and we also need to get direct to a particular object in the fields array, so we can add two protected members to the class to allow this:

// set the number of elemeents
// to be
            held by the fields array
protected void SetFieldCount(int
            count)
{
if(fields == null)
{
fields = new
            object[count];
}
}

// access an element within the
            fields array
protected object this[int
            index]
{
get
{
return
            fields[index];
}
set
{
fields[index] =
            value;
}
}

That's all there is to the BusinessEntity class. Any entity specific code will be implemented in that entities class, but the entity will not have to worry about storing or serializing that data. The next class we will need to implement is the abstract BusinessEntitySet class. This will be a generic collection class for holding classes derived from BusinessEntity.

Storing the Entities

The first thing to look at is how to store the entities for this entity set. In this implementation I have used an ArrayList class as this provides a pre-built implementation of most of the interfaces we will need for this class and it is very efficient.

Serializing a BusinessEntitySet

In addition to serializing a BusinessEntity, we need to be able to serialize the BusinessEntitySet if we are going to be using sets of data in a remote environment. This isn't difficult, but taking the time to implement our own serialization methods will greatly improve the performance of serialization which is key to creating a good remoting framework.

#region Implementation of ISerializable
public virtual void GetObjectData(SerializationInfo info, StreamingContext context)
{
// add the entities ArrayList to the object graph
info.AddValue("Entities", entities);
}

public BusinessEntitySet(SerializationInfo info, StreamingContext context)
{
// get the entities ArrayList from the object graph
entities = (ArrayList)info.GetValue("Entities", typeof(ArrayList));
}
#endregion

As you can see, that is not too difficult, but this highlights the reason why I chose to manually implement the collection features rather than inherit from CollectionBase. If I had inherited from CollectionBase, the ArrayList typed member InnerList is read-only, meaning I would have had to create a temporary ArrayList in memory and then manually copy each entity from one ArrayList to the other. The performance degradation of this method should be fairly obvious.

To highlight the performance benefits of this serialization framework, I did a few tests against the .NET DataSet provided serialization. I used the Products table of the Northwind database which contains 77 records on my installation. My results were:

 | Time for 100 Serializations (seconds) | File Size (KB)
.NET DataSet | 1.53 | 31
BusinessEntitySet | 0.74 | 9

As you can see this framework produces smaller object graphs faster than the .NET implementation. It is interesting to note that I also did an implementation of this where each BusinessEntity maintained its own internal member variables and performed its own serialization such as:

Public void GetObjectData(SerializationInfo info, StreamingContext context){
// add the customer id field
info.AddValue("CustomerId", customerId);

// add the customer name field
info.AddValue("Name", name);

// and so on.
}

This implementation is substantially slower at serialization than both .NET and the implementation detailed here. If you choose to create a framework like this, you should mimic what the DataSet class does for serialization: use the XmlTextWriter class to create an XML based representation of the data. This method, if implemented correctly, will give performance figures comparable to that of .NET

Specifying the BusinessEntity type

For our loading framework to work effectively, the classes derived from BusinessEntitySet must specify the type of BusinessEntity represented. This is done using a protected constructor which is the only constructor available for this class.

// the type of BusinessEntity represented
private Type entityType;

// derived class provide the type of the entity they
// are representing
protected BusinessEntitySet(Type entityType)
{
this.entityType = entityType;
}

Loading the BusinessEntitySet from the database

To enable us to the load BusinessEntitySets of any type from the database we will create a static method on the BusinessEntitySet that returns a BusinessEntitySet of a certain type. In this method we use System.Activator to create an instance of the class that is derived from BusinessEntitySet, such as CustomerSet or TitleSet. Similar to BusinessEntity, this will allow us to create one method to load any derived type of BusinessEntitySet from the database

public static BusinessEntitySet
            FromDataReader(SqlDataReader reader, Type entitySetType)
{
//
            create business entity set
BusinessEntitySet be =
            (BusinessEntitySet)Activator.CreateInstance(entitySetType);

//
            find out how many fields we have
int fieldCount =
            reader.FieldCount;

// create a holder for the field
            data
object[] data = null;

while(reader.Read())
{
//
            get the data for this row
data = new
            object[fieldCount];

// populate individual field
            parts
reader.GetValues(data);

// create a business
            entity
BusinessEntity en = BusinessEntity.FromData(be.EntityType,
            data);
be.Entities.Add(en);
}

// close the
            reader
reader.Close();

return be;
}

Performance of loading data

I also tested the loading of data using this method versus the .NET DataSet/DataAdapter method, using the same data as when testing the serialization. Here are my findings:

 | Time for 1000 Queries (seconds)
.NET DataSet/DataAdapter | 7.21
BusinessEntitySet | 5.95

As you can see from the performance figures, the querying performance of this code is substantially faster than that of the basic .NET DataSet/DataAdapter combination.

Supporting binding and enumeration

We want to be able to bind our BusinessEntitySets to controls such as a DataGrid as we would with a DataSet, and also to be able to use the familiar foreach construct to loop through thus:

   // get a
            CustomerSet
CustomerSet customers =
            GetCustomers();

foreach(Customer customer in
            customers)
{
Console.WriteLine(customer.Name);
}

Supporting enumerations is a simple case of implementing the IEnumerable interface, and since we only need to enumerate the entities ArrayList, we can just delegate the IEnumerable::GetEnumerator() call to the entities ArrayList.

#region Implementation of IEnumerable
public System.Collections.IEnumerator GetEnumerator()
{
return entities.GetEnumerator();
}
#endregion

To support binding we just need to implement the IList interface that also requires that we implement the ICollection and IEnumerable interfaces. At data binding time, the data bound control will use the IList interface to loop through the objects in the collection. So, we could simply bind a BusinessEntitySet to a DataGrid and we are away. In order to support columnar binding such as the DataGrid, we don't actually have to do anything other than define our public properties on our BusinessEntity derived classes. The .NET Framework will allow us to bind the columns of a DataGrid to these properties.

In our case, the ArrayList already implements IList, so we can simply defer all our calls for this interface to the entities ArrayList.

#region Implementation of IList
public void RemoveAt(int index)
{
entities.RemoveAt(index);
}

public void Insert(int index, object value)
{
entities.Insert(index, value);
}

public void Remove(object value)
{
entities.Remove(value);
}

public bool Contains(object value)
{
return entities.Contains(value);
}

public void Clear()
{
entities.Clear();
}

public int IndexOf(object value)
{
return entities.IndexOf(value);
}

public int Add(object value)
{
return entities.Add(value);
}

public bool IsReadOnly
{
get
{
return entities.IsReadOnly;
}
}

public object this[int index]
{
get
{
return entities[index];
}
set
{
entities[index] = value;
}
}

public bool IsFixedSize
{
get
{
return entities.IsFixedSize;
}
}
#endregion

Implementing Sort Functionality

For the sort functionality, I chose to defer the actual implementation of sorting to the derived classes. I had actually implemented a reflection-based method; however this was extremely slow, and didn't really match the performance of the .NET DataSet.

For sort functionality I wanted to be able to specify a sort expression such as CustomerId DESC and have the BusinessEntitySet perform a sort based on the supplied parameters. I also needed to know whether or not the derived class would support sorting.

I decided that I would use the IComparer interface to implement sorting. This interface has one method, Compare(object x, object y), which the implementing class uses to compare objects x and y. From this, I could determine whether a BusinessEntitySet derived class supported sorting by determining whether or not it implemented IComparer. So I added this simple method to BusinessEntitySet :

// does this entity support
            sorting?
public bool SupportsSort
{
get
{
return
            (this is IComparer);
}
}

Now that I can know whether or not my derived class supports sorting, I can write a simple sort function to parse out the sort expression supplied by the client. I also added a protected member variable for the SortColumn and a bool value, SortAscending, to flag the sort direction.

// get the sort column as set by the sort
            operation
protected string SortColumn
{
get
{
return
            sortColumn;
}
}

// sort in ascending
            order?
protected bool SortAscending
{
get
{
return
            sortAscending;
}
}

The actual sorting is carried out by passing a reference to IComparer, which is the BusinessEntitySet derived class, to the entities ArrayList Sort method. This in turn determines the sort order of two objects by calling the Compare method on the BusinessEntitySet derived class that implement IComparer.

public void Sort(string sort)
{
//
            check if we support sort
if(SupportsSort)
{
// parse the
            sort direction
if(sort.EndsWith("DESC"))
sortAscending =
            false;
else
sortAscending = true;

// get the sort
            column
if(sortAscending)
sortColumn = sort.Substring(0,
            sort.Length - 4);
else
sortColumn = sort.Substring(0,
            sort.Length - 5);

// sort the ArrayList using the
            IComparer
// implementation of the the derived
            class
entities.Sort((IComparer)this);
}
else
{
// the
            derived class does not implement IComparer, so we
// cannot sort
            the BusinessEntities
throw new
            NotImplementedException(String.Format("Class {0} does not support
            sorting.", this.GetType().Name));
}
}

One last thing we will need to do is add a set of methods that the derived classes can use to compare certain field types. The Int16, Int32, Int64, Single and Double types do not have a Compare method, so we will manually implement them here for our derived classes to use:

   protected int
            CompareInt16(short a, short b)
{
// a is less than b
if(a
            < b)
return -1;

// a equals b
if(a == b)
return
            0;

// a is greater than b
return 1;
}

protected
            int CompareInt32(int a, int b)
{
// a is less than b
if(a
            < b)
return -1;

// a equals b
if(a == b)
return
            0;

// a is greater than b
return 1;
}

protected
            int CompareInt64(long a, long b)
{
// a is less than b
if(a
            < b)
return -1;

// a equals b
if(a == b)
return
            0;

// a is greater than b
return 1;
}

protected
            int CompareSingle(float a, float b)
{
// a is less than
            b
if(a < b)
return -1;

// a equals b
if(a ==
            b)
return 0;

// a is greater than b
return
            1;
}

protected int CompareDouble(double a, double
            b)
{
// a is less than b
if(a < b)
return
            -1;

// a equals b
if(a == b)
return 0;

// a is
            greater than b
return 1;
}

We will look at the actual implementation of the Compare method later, when we create classes derived from BusinessEntitySet. For now, that covers the implementation of our data description code. To recap, we can:

  • Load a set of entities from the database using a SqlDataReader
  • Serialize these entities for transport across a network
  • Bind the entities to data bound controls such as DataGrid and DataList
  • Sort (almost) a set of entities

Accessing Data

Now that we have created a method for representing our data in an object oriented manner we can sort, we need to look at how we are going to get our data into and out of the database. Essentially we have three needs:

  • Execute a query that returns a BusinessEntitySet from the database
  • Execute a query that returns a scalar value from the database
  • Execute a query that returns no value at all

In addition, we need to be able to execute these methods as part of a transaction.

Transactions

For those of you who aren't familiar with transactions, a transaction is set of procedures that either complete or fail as a whole. If we have a set of procedures we want to guarantee will either work all together or fail together we use a transaction.

A good example of where this is useful is when we need to insert a row into the Orders table and then insert a set of rows into the OrderItems table. If we had inserted the row into the Orders table and the server crashed we would have an Order for zero items that leads to inconsistent data.

We can wrap procedures such as this into a transaction to ensure that we keep the integrity of our data.

Options for Transactions

We have three main options for implementing transactions:

  • Implement our own transaction manager
  • Use the SqlTransaction class
  • Use COM+ Enterprise Services

Implementing our own transaction manager is impractical, and quite pointless given the amount of support for transactions the .NET platform offers.

COM+ Enterprise Services

COM+ Enterprise Services offers a flexible transaction model that allows for multiple resource managers such as databases and message queues to participate in transactions distributed across multiple machines. The problem with implementing this in a .NET application is that this is often overkill when we need a simple transaction to span one of more tables in one database.

Another drawback of using COM+ from .NET is the performance overhead. Although you can choose whether a class supports or requires a new transaction, the main performance degradation comes from how you have to implement a COM+ object in .NET.

To use COM+ services, you must derive from ServicedComponent, which in turn derives from ContextBoundObject. ContextBoundObject derives from MarshalByRefObject, allowing for a COM+ object to be remoted; as it also derives from ContextBoundObject there is an extra level of marshalling that must occur regardless of whether a component is remote or local. Any method call to a ContextBoundObject takes extra time than to a standard MarshalByRefObject.

Using SqlTransaction

Unless you specifically need to use multiple resource managers, or transaction that are spread across multiple machines, I would recommended sticking with the SqlTransaction class, as this offers performance close to that of embedding BEGIN TRANSACTION and COMMIT TRANSACTION statements in your stored procedures. In fact the SqlTransaction class merely echoes these commands to the database server. Our solution will utilize SqlTransaction to manage all transactions, as we have no need to interface with other resource managers.

Dynamically Loading Stored Procedure Parameters

Within this solution, I wanted to make it easy to add and edit stored procedures to the code without having to re-code the parameter initialization code each time around. To aid this, the parameters for a stored procedure are loaded at runtime and then cached for later retrieval. The performance hit you get from querying the database for the parameter set is outweighed by the ease of maintenance and the fact that each access to a stored procedure after the first retrieves the parameters from the cache, which is quicker than rolling them up each time.

The ParameterCache class

To build the parameter cache, I put a simple wrapper around a thread synchronized Hashtable, using the concatenation of the connection string and stored procedure name as the key.

// create a thread safe cache
private static Hashtable cache = Hashtable.Synchronized(new Hashtable());

.

// cache parameters for a stored procedure on the supplied connection
private static void Cache(string connectionString, string procedureName, params SqlParameter[] parameters)
{
// build up a key to hash the data
string hashKey = String.Format("{0}:{1}", connectionString, procedureName);

// add to the cache
cache[hashKey] = parameters;
}

Next I used the DeriveParameters method of the SqlCommandBuilder class to get the set of parameters for a particular stored procedure. The DeriveParmaters method will always return the parameter @RETURN_VALUE at index 0; I generally never use this parameter, so I have included a Boolean flag to remove this parameter from the collection.

private static SqlParameter[] DeriveParameters(string connectionString, string procedureName, bool includeReturnParameter)
{
using(SqlConnection connection = new SqlConnection(connectionString))
using(SqlCommand command = new SqlCommand(procedureName, connection))
{
// open the connection
connection.Open();

// set up the command
command.CommandType = CommandType.StoredProcedure;

// get the parameter set from the database server
SqlCommandBuilder.DeriveParameters(command);

// do we need the return parmeter??
if(!includeReturnParameter)
{
command.Parameters.RemoveAt(0); // @RETURN_VALUE is always at 0
}

// create the output parameter set
SqlParameter[] outParameters = new SqlParameter[command.Parameters.Count];
command.Parameters.CopyTo(outParameters, 0);

// return
return outParameters;
}
}

As you'll notice, the method above is private. I put two public wrapper methods around this function that will retrieve the parameters from the cache should they exist there:

public static SqlParameter[] GetParameters(string connectionString, string procedureName)
{
return GetParameters(connectionString, procedureName, false);
}

public static SqlParameter[] GetParameters(string connectionString, string procedureName, bool includeReturnParameter)
{
// build up a key to hash the data
string hashKey = String.Format("{0}:{1}", connectionString, procedureName);

// try to get the parameters out of the cache
SqlParameter[] parameters = (SqlParameter[])cache[hashKey];

if(parameters == null)
{
Console.WriteLine("Getting Parameters");
// get the parameters
parameters = DeriveParameters(connectionString, procedureName, includeReturnParameter);

// cache for next time
Cache(connectionString, procedureName, parameters);
}

// return
return CloneParameters(parameters);
}

You'll notice the last line makes a call to the CloneParameters method. We don't want any changes that are made to the SqlParameter array to be reflected in the cache, so we send a cloned set of parameters to the client, not the actual set in the cache:

private static SqlParameter[]
            CloneParameters(SqlParameter[] parameters)
{
// create a new
            parameter array
SqlParameter[] clonedParameters = new
            SqlParameter[parameters.Length];

// clone
            parameters
for(int x = 0; x < parameters.Length;
            x++)
{
clonedParameters[x] =
            (SqlParameter)((ICloneable)parameters[x]).Clone();
}

//
            return
return clonedParameters;
}

The DataHelper class

I want all my data access classes to be able to use a core set of querying features to perform the three basic actions that we defined above with or without a transaction.

I want the data access classes to be able to create, commit, and rollback the transactions, but I need the DataHelper to be able to enlist the commands it builds in these transactions

To do this, I will need four methods for each of the three actions: two for executing a stored procedure with or without a transaction, and two for executing a text command with or without a transaction.

Preparing a Command and Setting Parameter Values

All the methods in the DataHelper class that execute a command, as opposed to those that are simply a pass-through to another method, use the PrepareCommand method. This will configure the SqlTransaction, attach the SqlParameter, and also prevent the derived parameters from trying to insert.

The ExecuteNonQuery Methods

We can use ExecteNonQuery to explicitly state that we do not want any data returned. The methods are fairly simple, and highlight useful patterns for managing connections. All public methods pass through to one private method, that creates and sets up the SqlCommand object using PrepareCommand, and then executes the resulting SqlCommand :

We then have two public methods for executing textual commands with or without a transaction:

public static void ExecuteNonQuery(SqlTransaction transaction, CommandType commandType, string commandText)
{
// execute
ExecuteNonQuery(transaction.Connection, transaction, commandType, commandText, null);
}

public static void ExecuteNonQuery(string connectionString, CommandType commandType, string commandText)
{
// create the connection
using(SqlConnection connection = new SqlConnection(connectionString))
{
// open the connection
connection.Open();

// execute
ExecuteNonQuery(connection, null, commandType, commandText, null);
}
}

When we are executing a stored procedure we receive the parameters as an object[]. Note that the method signatures define the use of the params keyword to denote that each object can be passed as a separate argument such as:

// execute a stored procedure
ExecuteNonQuery("server=(local);integrated security=sspi;database=dvdstore", "UpdateCustomer", 1, "Bill", "Gates');

Notice how we pass the parameters as individual objects. Using the params keyword gives the client programmer flexibility in how the method will be used.

Executing a stored procedure requires a few more steps to ensure that the parameter values that are passed are valid before assigning them to the appropriate SqlParameter object with a call to AssignValuesToParameters :

private static void AssignValuesToParameters(object[] values, SqlParameter[] parameters)
{
// check we have both
if((values == null) || (parameters == null))
return;

// check we same the number of values as parameters
if(values.Length != parameters.Length)
return;

// now set the values
for(int x = 0; x < values.Length; x++)
{
parameters[x].Value = values[x];
}
}

public static void ExecuteNonQuery(string connectionString, string procedureName, params object[] parameterValues)
{
// get the parameters if we have any values
SqlParameter[] parameters = null;

if((parameterValues != null) && (parameterValues.Length > 0))
{
// get the parameters
parameters = ParameterCache.GetParameters(connectionString, procedureName);

// assign the values
AssignValuesToParameters(parameterValues, parameters);
}

// create the connection
using(SqlConnection connection = new SqlConnection(connectionString))
{
// open the connection
connection.Open();

// execute
ExecuteNonQuery(connection, null, CommandType.StoredProcedure, procedureName, parameters);
}
}

public static void ExecuteNonQuery(SqlTransaction transaction, string procedureName, params object[] parameterValues)
{
// get the parameters if we have any values
SqlParameter[] parameters = null;

if((parameterValues != null) && (parameterValues.Length > 0))
{
// get the parameters
parameters = ParameterCache.GetParameters(transaction.Connection.ConnectionString, procedureName);

// assign the values
AssignValuesToParameters(parameterValues, parameters);
}

// execute
ExecuteNonQuery(transaction.Connection, transaction, CommandType.StoredProcedure, procedureName, parameters);
}

Note that when we are passing a SqlTransaction object instead of a connection string, we use the connection string associated with the SqlTransaction. An important point to notice here is that by default, the connection string security information will not be stored in the SqlTransaction object, so when the ParameterCache class attempts to connect it will be unable to do so. To get around this, we simple add persist security info=true to our connection string and the login details will be saved.

The ExecuteScalar Methods

The ExecuteScalar methods are almost identical to the ExecuteNonQuery methods, apart from the use of an object placeholder for the return value.

private static object ExecuteScalar(SqlConnection connection, SqlTransaction transaction, CommandType commandType,
string commandText, params SqlParameter[] parameters)
{
// create the command
SqlCommand command = new SqlCommand();

// prep the command
PrepareCommand(command, connection, transaction, commandType, commandText, parameters);

// execute
object returnValue = command.ExecuteScalar();

// clear the parameters
command.Parameters.Clear();

// return
return returnValue;
}

public static object ExecuteScalar(string connectionString, CommandType commandType, string commandText)
{
// create the connection
using(SqlConnection connection = new SqlConnection(connectionString))
{
// open the connection
connection.Open();

// set the return value
return ExecuteScalar(connection, null, commandType, commandText, null);
}
}

public static object ExecuteScalar(string connectionString, string procedureName, params object[] parameterValues)
{
// get the parameters if we have any values
SqlParameter[] parameters = null;

if((parameterValues != null) && (parameterValues.Length > 0))
{
// get the parameters
parameters = ParameterCache.GetParameters(connectionString, procedureName);

// assign the values
AssignValuesToParameters(parameterValues, parameters);
}

// create the connection
using(SqlConnection connection = new SqlConnection(connectionString))
{
// open the connection
connection.Open();

// return
return ExecuteScalar(connection, null, CommandType.StoredProcedure, procedureName, parameters);
}
}

public static object ExecuteScalar(SqlTransaction transaction, CommandType commandType, string commandText)
{
return ExecuteScalar(transaction.Connection, transaction, commandType, commandText, null);
}

public static object ExecuteScalar(SqlTransaction transaction, string procedureName, params object[] parameterValues)
{
// get the parameters if we have any values
SqlParameter[] parameters = null;

if((parameterValues != null) && (parameterValues.Length > 0))
{
// get the parameters
parameters = ParameterCache.GetParameters(transaction.Connection.ConnectionString, procedureName);

// assign the values
AssignValuesToParameters(parameterValues, parameters);
}

return ExecuteScalar(transaction.Connection, transaction, CommandType.StoredProcedure, procedureName, parameters);
}

The same rules for connection strings and transactions apply here as it did in the ExecuteNonQuery method.

The ExecuteBusinessEntitySet Methods

This method is slightly more complex in the way that we manage our connections because we need to keep the connection alive long enough to populate the BusinessEntitySet. You will also notice that the methods accept an extra parameter, a Type value indicating the BusinessEntitySet derived class that System.Activator should instantiate.

The actual code for populating a BusinessEntitySet is taken care of in the BusinessEntitySet::FromDataReader method, so all we need to do is worry about getting the SqlDataReader and passing it to the BusinessEntitySet. The connection will be closed during the call to BusinessEntitySet::FromDataReader.

private static BusinessEntitySet ExecuteBusinessEntitySet(SqlConnection connection, SqlTransaction transaction,
CommandType commandType, string commandText, Type entitySetType, params SqlParameter[] parameters)
{
// create the command
SqlCommand command = new SqlCommand();

// prep the command
PrepareCommand(command, connection, transaction, commandType, commandText, parameters);

// execute
SqlDataReader reader = command.ExecuteReader();

// clear the parameters
command.Parameters.Clear();

// create the return value
BusinessEntitySet entitySet = BusinessEntitySet.FromDataReader(reader, entitySetType);

// BusinessEntitySet::FromDataReader closes and destroys the connection

// return
return entitySet;
}

public static BusinessEntitySet ExecuteBusinessEntitySet(string connectionString, CommandType commandType, string commandText,
Type entitySetType)
{
// create the connection
using(SqlConnection connection = new SqlConnection(connectionString))
{
connection.Open();

return ExecuteBusinessEntitySet(connection, null, commandType, commandText, entitySetType, null);
}
}

public static BusinessEntitySet ExecuteBusinessEntitySet(string connectionString, string procedureName, Type entitySetType,
params object[] parameterValues)
{
// get the parameters if we have any values
SqlParameter[] parameters = null;

if((parameterValues != null) && (parameterValues.Length > 0))
{
// get the parameters
parameters = ParameterCache.GetParameters(connectionString, procedureName);

// assign the values
AssignValuesToParameters(parameterValues, parameters);
}

// create the connection
using(SqlConnection connection = new SqlConnection(connectionString))
{
connection.Open();

return ExecuteBusinessEntitySet(connection, null, CommandType.StoredProcedure, procedureName, entitySetType, parameters);
}
}

public static BusinessEntitySet ExecuteBusinessEntitySet(SqlTransaction transaction, CommandType commandType, string commandText,
Type entitySetType)
{
// execute
return ExecuteBusinessEntitySet(transaction.Connection, transaction, commandType, commandText, entitySetType, null);
}

public static BusinessEntitySet ExecuteBusinessEntitySet(SqlTransaction transaction, string procedureName, Type entitySetType,
params object[] parameterValues)
{
// get the parameters if we have any values
SqlParameter[] parameters = null;

if((parameterValues != null) && (parameterValues.Length > 0))
{
// get the parameters
parameters = ParameterCache.GetParameters(transaction.Connection.ConnectionString, procedureName);

// assign the values
AssignValuesToParameters(parameterValues, parameters);
}

return ExecuteBusinessEntitySet(transaction.Connection, transaction, CommandType.StoredProcedure, procedureName, entitySetType,
parameters);
}

Creating our BusinessEntities

Now that we have the framework to represent our data and to read from and write to the database, we can start to build our BusinessEntity classes. We will need classes for:

  • Title
  • Actor
  • Order
  • OrderItem
  • Customer

I won't go over every part of every BusinessEntity as they are very similar. I will simply address how we create a BusinessEntity, what is required to support serialization, and how to represent relationships between some of the BusinessEntities.

The Title BusinessEntity

The first thing we need to do is create the class and inherit from BusinessEntity. We should also mark the class with the Serializable attribute so that we can serialize it.

[Serializable]
public class Title :
            BusinessEntity
{

// field count constant
private
            const int FieldCount = 5;

public Title()
{
// set the
            internal data
            array
SetFieldCount(FieldCount);
}

}

To finish adding support for serialization we will need to add the protected constructor required by the ISerializable framework:

protected Title(SerializationInfo info,
            StreamingContext context) : base(info, context)
{
//
            Constructor for deserializing
}

Now that we have the basic BusinessEntity in place, we can add in the public properties that represent the fields for this entity. First we start by adding private constants to represent the index of each field. Doing so means that we can easily update this BusinessEntity if the underlying field order should change:

// field index constants
private const int TitleIdField = 0;
private const int NameField = 1;
private const int DescriptionField = 2;
private const int BoxShotField = 3;
private const int SalePriceField = 4;

Now we create the public properties. Note that I am using the GetXXX and SetValue methods from the BusinessEntity to get typed access to the fields.

public int TitleId
{
get
{
return GetInt32(TitleIdField);
}
set
{
SetValue(TitleIdField, value);
}
}

public string Name
{
get
{
return GetString(NameField);
}
set
{
SetValue(NameField, value);
}
}

public string Description
{
get
{
return GetString(DescriptionField);
}
set
{
SetValue(DescriptionField, value);
}
}

public string BoxShot
{
get
{
return GetString(BoxShotField);
}
set
{
SetValue(BoxShotField, value);
}
}

public decimal SalePrice
{
get
{
return GetDecimal(SalePriceField);
}
set
{
SetValue(SalePriceField, value);
}
}

With that done, our Title entity is complete. As you can see, once the framework is in place, creating new entities is actually very easy and the performance benefits are quite large.

The Order and OrderItem BusinessEntities

As I said earlier, I won't cover the code for every single entity, as they all follow the same basic pattern as the Title entity. However, for the Order entity, I have added an extra public property called Items. This property accesses an OrderItemSet that will hold all the OrderItems for this Order.

private OrderItemSet orderItems;

//
            Store the items for this Order
public OrderItemSet
            Items
{
get
{
if(orderItems == null)
orderItems = new
            OrderItemSet();

return
            orderItems;
}
set
{
orderItems =
            value;
}
}

Structuring the class this way means that I can accurately represent the Order and its OrderItems, and I can pass the Order object to the data access components for entry in the database.

Also I will have to extend the serialization of this class to include the OrderItemSet. This is easily done, but be careful to remember to override the serialization methods on the base class so that the rest of the entity gets serialized.

// constructor for serialization
protected Order(SerializationInfo info, StreamingContext context) : base(info,context)
{
// now deserialise the order items
try
{
orderItems = (OrderItemSet)info.GetValue("OrderItems", typeof(OrderItemSet));
}
catch
{
// was not specified in the serilization code
// or the value was null
orderItems = null;
}
}

// override the default serialization behaviour
// and add the orderItesm object to the object
// graph in addition to the basic field set
public override void GetObjectData(SerializationInfo info, StreamingContext context)
{
// make sure the base
// class can serialize first
base.GetObjectData(info, context);

// now add the orderItems
info.AddValue("OrderItems", orderItems);
}

The Customer BusinessEntity

Similar to the Order and OrderItem relationship, I have added another public property to the Customer entity for the Addresses of that entity. The code is similar to that of the Order entity, but here it is as a recap:

[Serializable]
public class Customer : BusinessEntity
{
// field index constants
private const int AccountIdField = 0;
private const int FirstNameField = 1;
private const int LastNameField = 2;
private const int EmailAddressField = 3;
private const int PasswordField = 4;

private AddressSet addresses;

public Customer()
{
// set up the number of fields
SetFieldCount(5);
}

// override the default behaviour for de-serialization and read the
// addresses object from the object graph
protected Customer(SerializationInfo info, StreamingContext context) : base(info, context)
{
try
{
addresses = (AddressSet)info.GetValue("Addresses", typeof(AddressSet));
}
catch
{
// there was an error in reading the addresses
// object. It was probably null.
addresses = null;
}
}

// override the default behaviour for serialization and add the
// addresses object too the object graph
public override void GetObjectData(SerializationInfo info, StreamingContext context)
{
// all the base method
// to serialize the fields
base.GetObjectData(info, context);

// now add the addresses object
info.AddValue("Addresses", addresses);
}

#region Fields

public int AccountId
{
get
{
return GetInt32(AccountIdField);
}
set
{
SetValue(AccountIdField, value);
}
}

public string FirstName
{
get
{
return GetString(FirstNameField);
}
set
{
SetValue(FirstNameField, value);
}
}

public string LastName
{
get
{
return GetString(LastNameField);
}
set
{
SetValue(LastNameField, value);
}
}

public string EmailAddress
{
get
{
return GetString(EmailAddressField);
}
set
{
SetValue(EmailAddressField, value);
}
}

public string Password
{
get
{
return GetString(PasswordField);
}
set
{
SetValue(PasswordField, value);
}
}

#endregion

// addresses
public AddressSet Addresses
{
get
{
if(addresses == null)
addresses = new AddressSet();

return addresses;
}
}
}

Creating the BusinessEntitySets

With our BusinessEntities in place we can now create our fully typed BusinessEntitySets. The code for this is fairly trivial. As such, I will only show the code for one to highlight what we need to do and also how to implement sorting.

The first thing we need is the class, remembering to mark the class as Serializable, and also to inherit from BusinessEntitySet and implement IComparer (if we are supporting sorting).

[Serializable]
public class TitleSet : BusinessEntitySet, System.Collections.IComparer
{
// . class code here .
}

The BusinessEntitySet class has a protected constructor that requires we pass the Type of the BusinessEntity this class is representing:

// store the entity type we are
            representing
private static Type BusinessEntityType =
            typeof(Title);

public TitleSet() : base(BusinessEntityType)
            // call the base constructor to set entity
            type
{

}

Next we should remember to implement the protected constructor as required by the ISerializable interface:

// serialization constructor - delegate to
            base class
protected TitleSet(SerializationInfo info,
            StreamingContext context) : base(info,context)
{
}

With that done, we can now create methods to add and access type BusinessEntities from the collection:

// add a new Title To the Collection
public void Add(Title title)
{
this.Entities.Add(title);
}

// get a Title at the supplied index
public new Title this[int index]
{
get
{
return (Title)this.Entities[index];
}
}

Note here that I have accessed the Entities ArrayList directly rather than calling base.Add(), as this would result in a StackOverflowException being thrown at runtime. Now we have all the functionality that we need, except sorting.

Implementing Sorting

Because our framework use the IComparer interface to implement its sorting, all we need to do is provide an implementation of the IComparer interface on every BusinessEntitySet we need to be able to sort.

IComparer has one method, Compare, which is passed two objects, x and y, for it to compare. Because of the way our framework is coded, we can be fairly sure that the objects being passed will be the BusinessEntity type represented by this BusinessEntitySet, but we will check anyway. The Compare method expects an integer return parameter signaling whether the objects are equal, whether x is greater than y or whether y is greater than x. Most of the primitive types in .NET have a built in Compare method, but we have specified methods for those that don't in the BusinessEntitySet class.

#region Implementation of
            IComparer
public int Compare(object x, object y)
{
// check
            that the passed objects are of the correct type
if(!(x is Title)
            || !(y is Title))
throw new InvalidOperationException("Incorrect
            Object Types");

// now cast the passed object to Title
            switching the order
// if we are sorting descending
Title
            tx;
Title ty;

if(SortAscending)
{
tx =
            (Title)x;
ty = (Title)y;
}
else
{
tx =
            (Title)y;
ty = (Title)x;
}

Notice how we assign the objects x and y to the Titles ix and iy, based on the specified sort order. Now we can perform a sort based on the column specified using a simple switch statement:

// now compare based on the specified column
switch(SortColumn)
{
case "TitleId":
return CompareInt32(tx.TitleId, ty.TitleId);
case "Name":
return String.Compare(tx.Name, ty.Name);
case "Description":
return String.Compare(tx.Name, ty.Name);
case "SalePrice":
return Decimal.Compare(tx.SalePrice, ty.SalePrice);
default:
throw new InvalidOperationException(String.Format("Can not sort using column {0}", SortColumn));
}
}
#endregion

As you can see, implementing sort functionality is fairly simple and quite flexible, given that we can restrict which BusinessEntitySets are sortable and on which properties they can be sorted.

Data Access Components

Now that we have a full set of BusinessEntities and BusinessEntitySets, we can start to code our data access components. We will need three data access components:

  • Customers
  • Actors
  • Titles
  • Orders

Note that I will not be creating a component for OrderItems or Addresses. This is because the OrderItem entity and the Address entity really only exists as sub-entities to Order and Customer respectively, and are always accessed in the context of the parent entity

The Customers Component

Earlier on we created stored procedures to allow us to add a Customer to the database, add their addresses into the database, and also to retrieve a Customer by e-mail address. For our Customer component we will specify three methods:

  • InsertCustomer(Customer customer)
  • GetCustomer(string emailAddress)
  • InsertAddress(Customer customer, Address address)
  • GetAddresses(Customer customer)

The InsertCustomer Method

We start the method by creating a SqlConnection and beginning a SqlTransaction on that connection. When inserting a Customer, I also want to ensure that they add at least one address, so we will check to see if the AddressSet for this customer contains an Address. If so, we proceed, otherwise an exception is thrown:

// add a new customer to the database
public int InsertCustomer(Customer account)
{
// create a transaction for this
// operation as we will be setting up an address

using(SqlConnection connection = new SqlConnection(DataApplication.DvdStore.ConnectionString))
{
// open the connection
connection.Open();

// get the transaction
SqlTransaction transaction = connection.BeginTransaction();

try
{
// check for supplied address
if(account.Addresses.Count < 1)
throw new InvalidOperationException("Each account MUST contain at least one address");

Now we start our transaction, by calling the InsertCustomer stored procedure with the ExecuteScalar method of our DataHelper class:

// insert the customer first
int customerId = Convert.ToInt32(DataHelper.ExecuteScalar(transaction, "InsertCustomer", account.FirstName,
account.LastName, account.EmailAddress, account.Password));

This method will return the AccountId of the newly inserted Customer. Note that I have used Convert.ToIn32() rather than a cast to Int32. This is because the identity field is actually returned as a Decimal and cannot be cast to Int32.

Once the Customer is successfully inserted into the database, I can then call the InsertAddress procedure, passing in the same SqlTransaction object as I did for the previous procedure call.

// add the address
Address address = account.Addresses[0];

DataHelper.ExecuteNonQuery(transaction, "InsertAddress", customerId, address.Address1, address.Address2, address.Address3,
address.City, address.County, address.PostCode);

Now, providing that no exceptions were generated, we commit the transaction and return the newly inserted CustomerId to the caller. However if an exception is thrown it will be caught by the catch block and re-thrown as soon as the transaction has been rolled back. This ensures that no Customer can be entered onto the database without an Address.

// commit the transaction
transaction.Commit();

// return the id of the new customer
return customerId;
}
catch
{
// rollback the transaction
transaction.Rollback();

// re-throw the error
throw;
}

The finally block will ensure that any open connections are closed despite any exceptions that were thrown. You should also note that the code in the finally block will execute before the method call returns as long as the return statement occurs within either the try or catch block.

finally
{
// close the
            connection
connection.Close();

// close the
            connection
connection.Dispose();
}
}
}

The GetCustomer method

The GetCustomer is quite simple, as it does not require a transaction, but we want it to return a Customer not a CustomerSet, so we need some logic to do this:

// get the customer with the specified email address or return null
public Customer GetCustomer(string emailAddress)
{
// hold all returned records
CustomerSet customers = (CustomerSet)DataHelper.ExecuteBusinessEntitySet(DataApplication.DvdStore.ConnectionString,
"GetCustomerAccountByEmailAddress", typeof(CustomerSet), emailAddress);

if(customers.Count > 0)
return customers[0];
else
return null;
}

As you can see, we get a temporary CustomerSet based on the query and then we return null if the CustomerSet is empty, or the first Customer in the set otherwise.

The InsertAddress Method

InsertAddress is very similar to the InsertCustomer method other than we don't use a transaction this time around:

public void InsertAddress(Customer customer, Address address)
{
// execute the command
DataHelper.ExecuteNonQuery(DataApplication.DvdStore.ConnectionString, "InsertAddress", customer.AccountId, address.Address1,
address.Address2, address.Address3, address.City, address.County, address.PostCode);
}

You will see that the AccountId for this address is determined from the Customer object passed into the method. I prefer to pass entities rather pass say, an Int32, as it gives me more flexibility should the code change, as I don't need to change the method signatures.

Now that we have all the methods we will require for the Customer entity, let's look at something more complicated, the Orders component.

The GetAddresses Method

GetAddresses performs a simple SELECT statement on the Addresses table, to return all the Addressess for a particular Customer:

// return the addresses for a particular customer
public AddressSet GetAddresses(int customerAccountId)
{
return (AddressSet)DataHelper.ExecuteBusinessEntitySet(DataApplication.DvdStore.ConnectionString, "GetCustomerAddresses",
typeof(AddressSet), customerAccountId);
}

The Orders Component

The Orders component has only one method, InsertOrder, but this method is fairly complex. The first thing we do is verify that the Order has a least one OrderItem :

public int Insert(Order order)
{
// check that the order contains at least one item
if(order.Items.Count == 0)
throw new InvalidOperationException("An Order must contain at least one OrderItem");

If this is not the case, an exception is thrown, otherwise we create the connection and a transaction and we call the InsertOrder procedure with ExecuteScalar to retrieve the id of the Order.

// create the connection
SqlConnection connection = new SqlConnection(DataApplication.DvdStore.ConnectionString);

// open the connection
connection.Open();

// create a transaction
SqlTransaction transaction = connection.BeginTransaction();

try
{
// insert the order
int orderId = Convert.ToInt32(DataHelper.ExecuteScalar(transaction, "InsertOrder", order.OrderedBy, order.OrderedOn,
order.ShipTo, order.BillTo));

Now we loop through each individual OrderItem and add this to the database as part of this transaction.

// now insert the order lines
foreach(OrderItem item in order.Items)
{
DataHelper.ExecuteNonQuery(transaction, "InsertOrderItem", orderId, item.TitleId, item.Quantity, item.UnitPrice);
}

Providing no exceptions are thrown, the transaction is committed and the method returns the new OrderId. Any exceptions are caught in the catch block and the transaction is rolled back. As always, the connection is closed by the finally block:

   // return the order
            id
return orderId;
}
catch
{
// rollback the
            transaction
transaction.Rollback();

// re-throw the
            exception
throw;
}
finally
{
// close the
            connection
connection.Close();

//
            dispose
connection.Dispose();
}
}

That covers the Orders component. I want to look briefly at how we can use a text based SQL command with the DataHelper class by looking at the FindByKeywords method of the Titles class, but the rest of the methods are fairly similar to what we have covered and are just included for completeness.

Using Text Based SQL Commands

The FindByKeyword method accepts an array of strings representing the keywords to search for. We use the StringBuilder class to build the command, searching both the Name and Description fields for any instance of any of the keywords.

// perform a keyword search on the titles table

public TitleSet FindByKeywords(string[] keywords)
{
// create a StringBulider to hold the
// command data
StringBuilder sb = new StringBuilder();

// add the basic part of the command
sb.Append("SELECT TitleId, [Name], [Description], BoxShot, SalePrice, AddedOn
FROM Titles WHERE ");

// now add the keyword to the command
foreach(string keyword in keywords)
{
// search the name column
sb.AppendFormat("[Name] LIKE '%{0}%' ", keyword);

// serach the description column
sb.AppendFormat("OR [Description] LIKE '%{0}%' OR ", keyword);
}

// we have OR trailing at the end. remove it
sb.Remove(sb.Length - 4, 4);

.

We then use the text command version of ExecuteBusinessEntitySet to return a TitleSet containing Titles matching the query:

// now execute the command and return
return (TitleSet)DataHelper.ExecuteBusinessEntitySet(DataApplication.DvdStore.ConnectionString, CommandType.Text, sb.ToString(),
typeof(TitleSet));
}

UI Design and Coding

Now that we have all the data access components and BusinessEntities created, we can begin to put together our user interface. For the user interface we will create a simple ASP.NET application that we take Orders for our DVDs.

The Search Box Control

I want to display a keyword search box on every page. To do this I will create an ASP.NET User Control. For this control I simply add a TextBox called keywords and an ImageButton called doSearch. These will control and then link up the Click event of the doSearch button to forward the request to the Search Results page, which will perform the query and display the results.

The Shopping Cart Class

This class is a simple wrapper around an OrderItemSet that is stored in the current HttpSessionState object for each Customer:

public class Cart
{
public
            Cart()
{
// do nothing
}

// get access to the
            current HttpSession
private static HttpSessionState
            Session
{
get
{
return
            HttpContext.Current.Session;
}
}

public static
            OrderItemSet Items
{
get
{
if(Session["ITEMS"] ==
            null)
Session["ITEMS"] = new OrderItemSet();

return
            (OrderItemSet)Session["ITEMS"];
}
set
{
Session["ITEMS"]
            = value;
}
}

public static void AddItem(Title title,
            int quantity)
{
// add to the order
OrderItem
            item;

// check if we have an existing one
item =
            Items.GetByTitle(title.TitleId);

// did we get
            one?
if(item == null)
{
item = new
            OrderItem();
item.TitleId = title.TitleId;
item.TitleName =
            title.Name;
item.TitleDescription =
            title.Description;
item.UnitPrice =
            title.SalePrice;
item.Quantity =
            quantity;

Items.Add(item);
}
else
{
// increase
            the quantity
item.Quantity +=
quantity;
}
}
}

This will allow us to bind the contents of the cart directly to a new Order object when the Customer wants to complete the Order. I also added a new method to the OrderItemSet class to allow us to find an OrderItem by the TitleId so we could see if a Title already existed in the cart, and increase the quantity of this OrderItem rather than create a new one:

// get a title from the order by
//
            TitleId or return null
public OrderItem GetByTitle(int
            titleId)
{
for(int x = 0; x < this.Entities.Count;
            x++)
{
if(this[x].TitleId == titleId)
return
            this[x];
}

return null;
}

Also I added two new member variables to the OrderItem class to store the TitleName and TitleDescription. This is mainly for the purposes of displaying the shopping cart so we don't have to query the database for the name of Title in the cart. This also highlights the flexibility of writing our own custom BusinessEntity classes as we are not tied to the database schema, as we would have been with a DataSet.

The Home Page

Earlier on we specified that we wanted to display the latest three DVDs added to the database on the home page. I want to display the DVDs in a list similar to Amazon so I will add a DataList to my home page and name it dvdList :

In the Page_Load event I will now use my data access component to get the latest three DVDs from the database and bind them to the DataList :

if(!Page.IsPostBack)
{
// bind the
            list to show all the dvds available
TitleSet titles = (new
            Titles()).GetLatest();
dvdList.DataSource =
            titles;
dvdList.DataBind();
}

My DataList template contains a link to the DVD Details page for each DVD and also an "Add to Cart" button that fires the DataList's ItemCommand event. The following code will determine which item was clicked and add it to the cart.

private void dvdList_ItemCommand(object
            source, System.Web.UI.WebControls.DataListCommandEventArgs
            e)
{
if(e.CommandName == "AddToCart" &&
            e.CommandArgument != null)
{
// get the TitleId
int titleId
            = Int32.Parse(e.CommandArgument.ToString());

// get the Title
            from the database
Title title = (new
            Titles()).GetById(titleId);

// add a new cart
            item
Cart.AddItem(title, 1);

// show the cart
            page
Response.Redirect("Cart.aspx");
}
}

The DVD Details Page

Here we retrieve a Title object by id from the database and the list of Actors associated with this Title. This data is then bound to a set of Label controls on the page:

// bind the text labels
titleName.Text =
            title.Name;
titleDescription.Text =
            title.Description;
titlePrice.Text = String.Format("£{0:F}",
            title.SalePrice);

// set the image url
boxShot.ImageUrl =
            title.BoxShot;

The Actor list is bound to a Literal control. I have set it to output a set of links that will direct the user to the list of that Actor's films.

// get the list of actors for this
            title
ActorSet actors = (new
            Actors()).GetByTitle(title);

// sort based on last
            name
actors.Sort("LastName ASC");

// now output with links
            to enable the user to search based on this actor
foreach(Actor
            actor in actors)
{
actorSet.Text += String.Format("<a
            href=\"SearchResults.aspx?ACTOR={0}\">{1}</a> , ",
actor.ActorId, actor.DisplayName);
}

// remove the
            trailing comma
actorSet.Text =
            actorSet.Text.Remove(actorSet.Text.Length - 2, 2);

The Search by Actor Page

This page retrieves the full list of Actors from the database and binds them to a DropDownList control, which has the AutoPostBack property set to True, causing the SelectedIndexChanged event to fire automatically each time a user changes the selection:

// load in the actor list one first
            load
if(!Page.IsPostBack)
{
// get all the
            actors
ActorSet actors = (new Actors()).GetAll();

// sort
            based on last name
actors.Sort("LastName ASC");

// now
            bind to the list
actorList.DataSource =
            actors;
actorList.DataTextField =
            "DisplayName";
actorList.DataValueField =
            "ActorId";
actorList.DataBind();

// set the first item to
            say select actor
actorList.Items.Insert(0, "Choose an actor");
}

When the SelectedIndexChanged event of the DropDownList is fired, the ID of the currently selected Actor is passed to the Search Results Page:

private void
            actorList_SelectedIndexChanged(object sender, System.EventArgs
            e)
{
// get the Actor selected
if(actorList.SelectedItem !=
            null)
{
// try to get the id of the actor
try
{
//
            get the id
int actorId =
            Int32.Parse(actorList.SelectedItem.Value);

// redirect to
            search results
Response.Redirect("SearchResults.aspx?ACTOR=" +
            actorId.ToString());
}
catch
{
// do nothing
// let
            customer selected again
}
}
}

The Search Results Page

Firstly, this page will determine whether the search is a keyword search or a "by Actor" search, by looking for certain keys in the Request.QueryString collection:

// store the results of either search
            here
protected TitleSet
            dataSource;

if(!Page.IsPostBack)
{
// we need to
            determine what the search criteria is
// if the QueryString
            contains the key KEYWORDS then we need to perform
// a keyword
            search. If the QuertyString contains the key ACTOR then we search by
            Actor

// check for
            KEYWORDS
if(Request.QueryString["KEYWORDS"] != null)
{
//
            get the keywords
string[] keywords =
            Request.QueryString["KEYWORDS"].Split(' ');

// now set the
            data source based on this query
dataSource = (new
            Titles()).FindByKeywords(keywords);

// update the
            view
UpdateView();
}
else if(Request.QueryString["ACTOR"]
            != null)
{
// see if the key contains a Int32 as a
            value
try
{
// get the id
int actorId =
            Int32.Parse(Request.QueryString["ACTOR"]);

// create the
            actor object
Actor actor = new Actor();
actor.ActorId =
            actorId;

// set the data source
dataSource = (new
            Titles()).GetByActor(actor);

// update
            view
UpdateView();
}
catch(FormatException ex)
{
//
            was not an integer
// set the datasource to null
// so no
            records are displayed
dataSource =
            null;
}
}
else
{
// no valid search parameters
//
            send the user to the home
            page
Response.Redirect("default.aspx");
}
}

Once the appropriate search is performed, the resulting TitleSet is bound to a DataList with an ItemTemplate similar to that of the DataList from the homepage:

// bind the current dataSource object to the
            display list
private void UpdateView()
{
if(dataSource ==
            null || dataSource.Count == 0)
{
// flag no records
            found
resultLabel.Text = "<br>No titles were found that
            match your criteria";
// hide the list
dvdList.Visible =
            false;
}
else
{
// bind the data to the
            list
dvdList.DataSource =
            dataSource;
dvdList.DataBind();
}
}

Again we capture the ItemCommand of the DataList when an Add to Cart button is clicked, and we add the appropriate Title to the cart:

private void dvdList_ItemCommand(object
            source, System.Web.UI.WebControls.DataListCommandEventArgs
            e)
{
if(e.CommandName == "AddToCart" &&
            e.CommandArgument != null)
{
// get the TitleId
int titleId
            = Int32.Parse(e.CommandArgument.ToString());

// get the Title
            from the database
Title title = (new
            Titles()).GetById(titleId);

// add a new cart
            item
Cart.AddItem(title, 1);

// show the cart
            page
Response.Redirect("Cart.aspx");
}
}

The Cart Page

The Customer can view the contents of their cart at any time from this page. The OrderItemSet from that is stored in the current HttpSessionState, and is bound to a DataGrid to allow the Customer to view the contents of their cart.

private void Page_Load(object sender,
            System.EventArgs
            e)
{
if(!Page.IsPostBack)
{
UpdateView();
}
}

private
            void UpdateView()
{
if(Cart.Items.Count >
            0)
{
cartGrid.DataSource =
            Cart.Items;
cartGrid.DataBind();
}
else
{
// hide the
            grid
cartTable.Visible = false;

// set "no items"
            message
errorMessage.Text = "There are no items in your
            cart.";
}
}

The Customer is able to change the quantity of each Title required, and the changes are saved to the cart when the Update button is clicked:

private void updateCartButton_Click(object
            sender, System.Web.UI.ImageClickEventArgs e)
{
// loop through
            the data grid and get the
// new quantities
for(int index =
            0; index < cartGrid.Items.Count; index++)
{
// get the
            OrderItem
OrderItem item = Cart.Items[index];

// find the
            quantity
TextBox quantityBox =
            (TextBox)cartGrid.Items[index].Cells[2].Controls[1];

// see
            if the value is a number
try
{
// get the quantity
int
            newQuantity = Int32.Parse(quantityBox.Text);

if(newQuantity
            == 0) // remove the item from the
            order
{
Cart.Items.Remove(item);
}
else
{
//
            update the order item
item.Quantity =
            newQuantity;
}
}
catch(FormatException ex) // catch non
            numeric values
{
// do nothing for this item
// leave the
            quantity as is
}
}

// update the
            view
UpdateView();
}

The Login Page

The login page allows a Customer who has previously registered their details on the site to log in to finish their Order. The page consists of two TextBox controls for the e-mail address and password of the Customer. When the Customer clicks the Login button, we retrieve the Customers details using their e-mail address. If the e-mail yields a valid Customer object we hash the value in the password TextBox using the FormsAuthentication::HashPasswordForStoringInConfigFile method and compare this with the value in the Customer object:

private void loginButton_Click(object sender, System.Web.UI.ImageClickEventArgs e)
{
// get the account for this username
Customer account = (new Customers()).GetCustomer(emailAddress.Text);

// see if we have an account
if(account == null) // invalid user name
{
errorMessage.Text = "Sorry, the username or password you supplied is invalid. Account Invalid. Try again";
}
else
{
// try to match the password
string hashedPassword = FormsAuthentication.HashPasswordForStoringInConfigFile(password.Text, "SHA1");

If the credentials match, we set the authentication cookie and forward the Customer to the Checkout page, otherwise we display an error message.

if(account.Password ==
            hashedPassword)
{
// set the authentication cookie
// and
            redirect to requested
            page
FormsAuthentication.RedirectFromLoginPage(account.EmailAddress,
            true);
}
else // invalid password
{
errorMessage.Text =
            "Sorry, the username or password you supplied is invalid.Try
            again";
}
}
}

The Join Page

The Join page simply captures the Customer's data. We check to see if the e-mail address supplied has already been used and flag an error if so. We also hash the password in SHA1 format using the FormsAuthentication::HashPasswordForStoringInConfigFile method. Once the Customer has successfully registered, we forward them back to the Login screen:

private void joinMeButton_Click(object
            sender, System.Web.UI.ImageClickEventArgs e)
{
// check to see
            if this email address is used
Customer existingCustomer = (new
            Customers()).GetCustomer(emailAddress.Text);

if(existingCustomer
            != null) // customer exists
{
errorMessage.Text =
            "<br>That e-mail address is already used. Try
            another<br>";
return;
}

// create the
            CustomerAccount
Customer account = new Customer();

// set
            the details
account.FirstName =
            firstName.Text;
account.LastName =
            lastName.Text;
account.EmailAddress =
            emailAddress.Text;
account.Password =
            FormsAuthentication.HashPasswordForStoringInConfigFile(password.Text,
            "SHA1");

// create the address
Address address = new
            Address();
address.Address1 = address1.Text;
address.Address2
            = address2.Text;
address.Address3 =
            address3.Text;
address.City = city.Text;
address.County =
            county.Text;
address.PostCode =
            postCode.Text;

account.Addresses.Add(address);

// add
            the account to the database

try
{
// insert the
            customer
(new Customers()).InsertCustomer(account);

// the
            person was trying to check
            out
Response.Redirect("Checkout.aspx");
}
catch(Exception
            ex)
{
errorMessage.Text = String.Format("<br>There was
            an error<br>{0}<br>",
            ex.Message);
}

}

The Checkout Page

When loading the Checkout page we get the Customers configured Addresses and bind them to two DropDownLists, allowing the Customer to specify a delivery and billing address for the Order :

private void Page_Load(object sender,
            System.EventArgs e)
{
// get the current account from the
            session
account = (new
            Customers()).GetCustomer(User.Identity.Name);

if(!Page.IsPostBack)
{
//
            bind the address boxes
AddressSet addresses = (new
            Customers()).GetAddresses(account.AccountId);

shipToAddress.DataSource
            = addresses;
shipToAddress.DataTextField =
            "FullAddress";
shipToAddress.DataValueField =
            "AddressId";
shipToAddress.DataBind();

billToAddress.DataSource
            = addresses;
billToAddress.DataTextField =
            "FullAddress";
billToAddress.DataValueField =
            "AddressId";
billToAddress.DataBind();
}
}

When the Complete Order button is clicked, we set the properties on the Order and attach the OrderItems from the cart to the new Order object before calling Orders::InsertOrder :

private void completeOrderButton_Click(object sender, System.EventArgs e)
{
// get the formatted credit card number
// a real world application would do some form of credit
// card verification here

// add the order to the database
Order order = new Order();
order.OrderedBy = account.AccountId;
order.OrderedOn = DateTime.Now;
order.BillTo = Int32.Parse(billToAddress.SelectedItem.Value);
order.ShipTo = Int32.Parse(shipToAddress.SelectedItem.Value);

// attach the order item from the cart to this order
order.Items = Cart.Items;

try
{
// insert the order
Orders orderInterface = new Orders();
order.OrderId = orderInterface.Insert(order);

// create an email to the Customer
MailMessage message = new MailMessage();
message.To = User.Identity.Name; // get the user email from the current Principal
message.From = "dvd@wrox.com";
message.Subject = "Order From Wrox DvdStore";

// set the mail body
message.Body = GetMessageBody(order);

// send the mail
SmtpMail.Send(message);

// hide the checkout table
checkoutTable.Visible = false;

// ouput order complete message
actionLabel.Text = String.Format("You order is complete<br>Your order number is {0}", order.OrderId);

// clear the cart
Cart.Items.Clear();

}
catch(Exception ex)
{
Response.Write(ex.Message);
}
}

private string GetMessageBody(Order order)
{
// create a StringBuilder
StringBuilder sb = new StringBuilder();

// header line
sb.Append("Order Confirmation - Wrox DVD Store\r\n");

// order info
sb.AppendFormat("Order Number {0}\r\n", order.OrderId);
sb.AppendFormat("Ordered On {0}\r\n", order.OrderedOn);
sb.AppendFormat("Order Total £{0:F}\r\n", order.Items.GetTotal());

// write the order items
sb.Append("\r\n\r\n"); // write some blank lines

sb.AppendFormat("{0}|{1}|{2}|{3}\r\n", "Title".PadRight(50, ' ' ), " Quantity ", " Unit Price ", " Sub Total ");
sb.Append(new String('-', 90) + "\r\n");

foreach(OrderItem item in order.Items)
{
sb.Append(item.TitleName.PadRight(50, ' ') + "|");
sb.Append(String.Format("{0}", item.Quantity).PadLeft(10, ' ') + "|");
sb.Append(String.Format("£{0:F}", item.UnitPrice).PadLeft(12, ' ') + "|");
sb.Append(String.Format("£{0:F}", item.SubTotal).PadLeft(10, ' ') + "\r\n");
}

// return
return sb.ToString();
}

When the Order is inserted, we send the Customer a confirmation e-mail and show the order number of the newly created Order.

The Sample Application

The Sample Application is very self explanatory for anyone who has bought anything from the Internet before; it really serves to highlight how you can leverage a powerful data access and description framework to speed application development, increase performance, and create simple business tier and presentation tier code.

Application Setup

To setup the application, run the MSI file supplied in the download to install the web application to your server. Once you have done that, simply run the supplied SQL Script, which will create the database and stored procedures as well a creating some sample data to get you started.

You should also change the following section in the web.config file to reflect your environment:

<appSettings>
<add key="EmailServer" value="smtp.nildram.co.uk"/>
<add key="ConnectionString" value="server=(local);uid=sa;pwd=potter1290;database=dvdstore;persist security info=true"/>
</appSettings>

Case Study Review

In this case study we have covered how we can build a flexible, extensible and scalable extension to the supplied .NET data classes.

We have looked a serialization in depth, and I will recap that using the default, reflection-based serialization framework is a serious detriment to the performance of your applications.

When looking at transactions we addressed a coding pattern to ensure that SqlTransactions rollback successfully and the SqlConnection objects are always closed. We also looked at the performance implications of using COM+ Enterprise Services. While I don't deter you from getting to grips with the COM+ Enterprise Services, I do recommend that you use them only when circumstances force you to. If you have to use COM+ Enterprise Services for you transactions, get into the habit of separating non-transacted methods out into a class separate from the transacted ones. This way these methods don't suffer the performance hit associated with deriving from ServicedComponent.

We also looked briefly at the sorting functionality provided by the .NET Framework and how a simple implementation of the IComparer interface can provide your application with powerful sorting functions.

For me, the two main benefits I got from this project was my research into the SqlCommandBuilder class. The code that loads the parameters dynamically from the database at runtime has saved me and my team huge amounts of development time, time that we can better spend on the business logic of the application. Secondly, when I was looking at how Microsoft had implemented the .NET data classes, I built up a great understanding of IL. I would recommend anyone who is serious about developing with .NET to get to grips with IL as it allows you to get a good grasp of the under-the-hood operations of .NET

Any Limitations or Further Work

As for limitations of the framework, I suppose that really, you are only limited by how much you choose to extend and expand on the supplied functionality.

If you are keen to extend this framework as I have done, then I would look at creating an XML Serialization framework so you can send the objects to applications and other platforms. I have also built a VS.NET add-in that will auto-generate BusinessEntities and BusinessEntitySet for a specific table, in addition to generating the Insert, Update, Delete, Select by Id and Select by Foreign Key stored procedures and C# code.

Also, I am currently working on an extension to the framework to allow behaviour similar to the "Field Groups" behaviour in Enterprise Java Beans. With this you specify a set of fields that are a group. The first time one is accessed, you load the set from the database. This works for updates as well, and can be useful for tables that have large binary or text fields.

C#Today - © 2002 WroxPress Limited


Threads, Delegates and Queues

Threads, Delegates and Queues
http://www.csharptoday.com/content.asp?id=1865
Article Rating: 5
Published on September 9, 2002
Abstract
Multithreaded applications are tough to design and even tougher to debug, but the rewards are definitely worth the effort. In .NET we now have a new set of thread classes which change the landscape somewhat for programming veterans, and gives novices an easier footing. This easier ground still has traps for the unwary. Veterans who are accustomed to the Win32 design of posting messages to threads will find they have some work to transition that model into .NET; novices will find creating threads is easy ?but threading issues such as data access conflicts or deadlocks are still present and need to be avoided. In this case study by Scot Boyd we will create a simple multi-threaded application, demonstrating different techniques of creating and using threads, and explore the design choices intended to avoid some of the problems a multi-threaded application can encounter. We will study how one thread can send messages to another, going from simple to robust techniques; first using delegates, then by using queues from the .NET Framework Classes, before finally exploring how to use the Microsoft Message Queue product.
Case Study

One of the many issues that programmers of multi-threaded applications have to address is the communication of data and events between threads. This case study will demonstrate how delegates can be used to send a simple message from thread to thread, briefly examine some of the problems you may encounter using this technique under heavy loads, and show you how to use System.Collection.Queue and the Microsoft Message Queue Service (MSMQ) to alleviate some of those problems.

Introduction

Multithreaded applications are tough to design and even tougher to debug, but the rewards are definitely worth the effort. In .NET we now have a new set of thread classes which change the landscape somewhat for programming veterans, and gives novices an easier footing. This easier ground still has traps for the unwary. Veterans who are accustomed to the Win32 design of posting messages to threads will find they have some work to transition that model into .NET; novices will find creating threads is easy ?but threading issues such as data access conflicts or deadlocks are still present and need to be avoided.

In this case study we will create a simple multi-threaded application, demonstrating different techniques of creating and using threads, and explore the design choices intended to avoid some of the problems a multi-threaded application can encounter. We will study how one thread can send messages to another, going from simple to robust techniques; first using delegates, then by using queues from the .NET Framework Classes, before finally exploring how to use the Microsoft Message Queue product.

System Requirements

To get the most out of this case study, the reader should already be familiar with the concepts of asynchronous communication and multithreading. We will introduce delegates and queues in .NET so prior familiarity is not required. The reader should also be aware of basic .NET concepts and the C# language.

Defining The Problem

Using multiple threads in an application can provide many benefits to the programmer. On multi-CPU machines, an application using threads will see them spread across the system CPUs, increasing overall performance. Even on a single-CPU machine, multithreading has considerable benefits - it allows you to run a lengthy section of code in the background while allowing access to the rest of your application, or to setup code which waits for the right time to act. Multithreading can keep your UI responsive and snappy, making your application respond to the user’s wishes like a sports car instead of a dump truck. Multithreading, however, is not a universal panacea; if your application exists solely to calculate pi to the nth place, then one thread will do.

Programmers who want to take advantage of multiple threads are faced with the troublesome task of managing communications between those threads. It’s inadvisable to simply call into a class being used by another thread as if the class belonged to you. Problems spring up immediately when two threads try to access the same data, or else get blocked waiting for a previous thread to finish. When two threads compete to access the same data, corruption is usually the result, and blocking a client thread for any length of time quickly degrades the performance benefits of multithreading.

The most efficient methods for communication between threads revolve around the concept of asynchronous communication, or simply messaging. By quickly sending a message to another thread (or a class being run by another thread) the client threads can get on with their business and the server thread can process its messages without waiting for class properties or methods to become unblocked. For encapsulation purists, messaging also provides a method of communication where knowledge of the destination class is unnecessary.

Case Study Structure

This case study has three major parts:

  • Delegates ?How to define, create, and invoke delegates between threads to send a message.
  • Queues ?Creating a managed queue and using it to send and receive messages.
  • MSMQ ?Accessing the MSMQ service and using their queues to send and receive messages.

Defining The Solution

In this case study we will explore three ways to send messages: delegates, Queues from the .NET Framework, and Microsoft Message Queue Service. A delegate-based solution does not provide true messaging so much as a message-like abstraction that delivers quite a few benefits for a very low overhead. Message queues are the professional strength way to trade more overhead for the benefits of passing true messages between threads, and we will demonstrate how to use them.

For our example I’ve chosen to imitate a program used in the Interactive Television world up until fairly recently. Interactive Television (ITV) is a fusion of web and television content, which can be viewed by some set-top boxes today (including Microsoft’s WebTV and UltimateTV products). The most common use of ITV in the US is sending URLs of specially designed Web pages (called ‘triggers? in unused areas of the video signal. The set-top box retrieves these pages from the Internet and displays them along with the video content. In order to make this happen, the broadcaster needs a program designed to send these triggers at specific times using a trigger inserter, running on the broadcaster’s local network. My sample stops short of actually contacting the inserter equipment.

Component Design and Coding

This program can be viewed as the interaction of three different types of threads. Each running thread in this application is closely linked to a particular class instance. First, we have the Form1 class, our basic UI. It is run on a thread created by the Application.Run() call found in the Main() function. This thread spawns the other two types of threads in our application, using the ServerThread and ClientThread classes. ServerThread is our main ‘worker?thread, i.e. it does the work. ClientThread is used as a delay timer, taking data intended for ServerThread and waiting for the right time to deliver that data.

In this case study we only begin to explore threading in .NET using the Thread class, and basic synchronization using the lock () statement. There are more sophisticated structures a programmer comfortable with threading basics will want to explore in .NET - the ThreadPool class which allows you to spawn multiple threads and manages their performance on your behalf, and synchronization classes including the Monitor, Mutex, and ReaderWriterLock classes. All of the above classes reside in the System.Threading namespace.

Version 1 ?Delegate Based Communication Between Threads

The following sections describe how we design an initial version of our ITV Trigger application using a delegate to communicate between ClientThread and ServerThread. Delegates provide a quick, easily encapsulated, and strongly typed way to pass messages between classes unknown to each other, and only incur a low overhead. This structure uses the same form as version 2, and is outlined in the UI Design and Coding section.

Defining ServerThread

Let’s look at the definition of ServerThread.DoWork(), where the main work of the ServerThread class is accomplished. All the code shown in this case study is included with the sample projects available for download.

public void
            DoWork()
{
while(bGo)
{
Console.Write(DateTime.Now.Second);

//
            check trigger state
if (aCount >= 0)
{
string
            trig;
lock(this) // secure the data for this thread
{
trig
            = aTriggers[aCount--];
}
// send latest trigger if
            possible
Console.WriteLine(trig);
}

// sleep a bit -
            simulate transmission delay between app and ITV
            inserter
System.Threading.Thread.Sleep(1000);
}
}

Because DoWork() is called on a new thread, we need some mechanism to ‘trap?the thread into continuing to work until we’re ready to shut down. When the thread exits the procedure, it will die. So, the first order of business is a while loop we can easily control. In this case, the thread continues looping as long as bGo, a private property in our class, is true. A public Stop() method is used to set bGo to false when we want to shut down this thread.

The Console.Write call at the top of the loop is a simple diagnostic to show that our server thread is ticking over. The next two lines are where the real action occurs. aTriggers is an array of strings designed to hold pending ITV triggers. aCount is the number of waiting triggers minus one, so it can do double duty as an index into the array. If a trigger is waiting, we put in a temporary variable, then show it to the console. This is where a real ITV trigger inserter would use Windows Sockets to activate a connection with the inserter hardware and broadcast the trigger. That would take time, so we use a call to Sleep() to simulate one second of delay. One second is the time it takes to contact an ITV trigger inserter mechanism on the broadcaster’s local network and insert the trigger into the video data. After we wake up, we take another turn around the block, unless bGo has been changed to false.

Note that DoWork() is not a static function. This is because ServerThread holds its own working data, and the simplest way to do that is to create properties to hold the data. Where do we get the data? ServerThread has a function named SendTrigger, which is called by other threads:

public int SendTrigger(String sTrigger)
{
if (aCount >= 9)
return 10; // Not throwing exception because it's not the caller's fault. Probably

lock(this)
{
aTriggers[++aCount] = sTrigger;
}
return aCount;
}

Our array of triggers is only ten elements wide. This is because it takes a second to send a trigger, so if we have ten triggers waiting we are ten seconds behind in transmitting them. So our first step is to make sure we don’t overflow the array, by checking aCount. You could throw an exception here, but I like to use them only when absolutely necessary since they are very disruptive to program flow and this problem was not really caused by the calling thread. If we have room we quickly store the data and return.

Why don’t we just use the calling thread to contact the ITV inserter? Why are we caching the data and releasing the thread so quickly? The answer to these two questions illustrates the first problem a multi-threaded application is likely to run into: blocking calling threads.

You may have noticed a block in both of these methods within a lock () statement. Lock () is used for synchronized access to objects. Synchronized access simply means that only one thread at a time can access the protected data. Because the thread in both cases is using an instance of a class, we use this to close off access to all members of the ServerThread instance. Access is restricted to the current thread until the end of the code block.

Without these locks you run the risk of encountering a class of bug called a race condition. A race condition occurs when two threads access the same data at the same time - causing a ‘race?to see which thread wins ?and usually you end up with data corruption. Race conditions are the front-runner for the Worst Bug Ever Award. They are often intermittent, making them hard to detect in a controlled test; the effect of a single race condition can range from mild to disastrous in any given encounter; and if you can’t find the bug, the only remedy is to rewrite sections of code hoping to fix it by accident.

A lock (any data synchronization) can cause two other classes of bugs: a blocked thread or a deadlock. A thread becomes blocked when it’s waiting for another thread to release a lock before it can proceed. This kind of bug doesn’t cause data corruption, but it induces a drag on performance that cancels out some of the benefits of using a thread in the first place. In the worst case, a block can last indefinitely, preventing your application from finishing. A deadlock is actually deadly ?two threads become blocked indefinitely by each other, and neither can continue.

public string strA;
public string
            strB;

public void
            Forward()
{
lock(strA)
{
Console.WriteLine("Forward has
            A");
Thread.Sleep(1000);
lock(strB)
{
Console.WriteLine("Forward has
            B");
}
}
}

public void
            Reverse()
{
lock(strB)
{
Console.WriteLine("Reverse has
            B");
Thread.Sleep(1000);
lock(strA)
{
Console.WriteLine("Reverse has
            A");
}
}
}

In the sample code above, I’ve contrived a situation where a deadlock is sure to occur. In a situation where one thread calls Forward within one second of another thread calling Reverse, the two threads will run headlong into each other. This is because they’ve each grabbed a lock the other one needs in order to continue.

The advice on how to fix this situation bring to mind the joke where a man tells a doctor "It hurts when I do this", and the doctor replies "So don’t do that." There are a couple of ways to fix this situation: don’t nest lock () statements, or take your locks in the same order. Another approach is to try and take locks one a time (this isn’t a useful option if you’re handling transactions) or to put your thread-accessible data in one class and then lock that ‘master?object instead of all the data objects separately. As a final option, you can roll your own locks using the System.Threading.Monitor class.

Above all, to avoid blocked threads and deadlocks, try to keep your lock () statements as short and as fast as possible. You still want them to encompass all the data accesses that are related to each other ?transaction programmers will recognize this as the concept of Atomicity.

Creating ServerThread

// The next two lines create an instance of ServerThread (so we can store pent-up Triggers)
// and create a Thread for that instance
instance = new ServerThread();
Thread workThread = new Thread(new ThreadStart(instance.DoWork));

// Now we start our thread before popping up the Form
workThread.Start();

Calling a new thread is a three step process. First you create a ThreadStart object. The ThreadStart object needs a function when you construct it. This is the function that will eventually be called when the thread is started. Next you create a thread, passing your ThreadStart object in the constructor. Finally, when you are ready to kick off your thread, you call Start () on the Thread object you’ve created.

It’s important to note that the function you pass to ThreadStart cannot have any parameters. This isn’t a problem for ServerThread, but it will become a problem later.

Shutting Down ServerThread

In order to provide for the orderly shutdown of the ServerThread, I have added a public method Stop () to the class. This function sets an internal Boolean value bGo to false; every time the thread in the DoWork () method finishes the while loop, it checks the status of bGo to see if it should continue.

public bool Stop()
{
bool retval =
            bGo;
bGo = false;
return retval;
}

For this function, we don’t use the lock on bGo because even in a race condition, both threads will be setting it to false and both threads want the value to be false. Retval is local to the calling thread’s memory, so a second calling thread won’t switch another thread’s return value before the function returns. It would be possible for two threads, calling Stop () at the same time, to both get return values of true. This indicates to both that the thread was running, which is true.

Another way to stop a thread in progress is to call the Abort () function on the Thread class. This raises a ThreadAbortException in the working thread. By implementing Stop () rather than expecting clients to call Abort (), I decide when the thread stops executing and can provide for a more orderly shutdown than an exception handler might be able to provide. If the Thread class was not declared as sealed (cannot be derived) I would probably have used Thread as my base class and overridden Abort () to provide an orderly shutdown.

Calling ServerThread.SendTrigger()

Delegates (defined as System.Delegate) are an interesting feature of .NET. One way to describe them is as ‘object oriented function pointers.? Another description is ‘type safe callbacks? They are classes that are declared by programmers to call a function of a particular type, and only that type. In a sense, when you declare a delegate you are really giving a type definition to a function call. Once a delegate has been defined in this way, an instance of it can be created which refers to a function of the proper type. Consider this definition:

public delegate int SendTriggerDelegate(String sTrigger);

Now take a look at the definition of SendTrigger above. I’ll wait.

Even though the delegate is named after the procedure I intend to use it with, a delegate with this definition will bind to any procedure that takes a String as its only parameter and returns an integer. Creating the delegate is straightforward:

ServerThread instance = new
            ServerThread();
SendTriggerDelegate d;
d = new
            SendTriggerDelegate(instance.SendTrigger);

Now, any time I want to call SendTrigger () on this instance of ServerThread, I can write the following code:

d("http://www.wrox.com/show/title.htm");

More importantly, I have declared a simple short type that can be used to send triggers to any destination object I create. I can give this object to anything that understands the SendTriggerDelegate type, and they can call ServerThread even if they don’t understand the ServerThread type. Only the class that creates the new delegate needs to know the type of the destination class.

ClientThread accepts a delegate rather than a pointer to the ServerThread class for this reason. It gives me a layer of indirection between the two classes that is similar to messaging.

Delegates can serve many purposes in .NET. For instance, delegates are used to indicate functions to be run by the ThreadPool class, and multicast delegates are used to deliver local events (the EventHandler class is a delegate class). Our use of delegates to isolate the ServerThread class is just a single example of the utility of delegates.

Defining ClientThread

Once ClientThread is started, the work of the thread is incredibly simple:

public static void SendTriggerAtTime(SendTriggerDelegate funcd, DateTime dtTime, String sTrigger)
{
// Sleep until the trigger comes due
Thread.Sleep(dtTime - DateTime.Now);
func(sTrigger);
}

ClientThread takes a SendTriggerDelegate and waits until the time indicated by dtTime. Then it makes a call the delegate with the given trigger string. Unlike ServerThread.DoWork(), this function is marked static, so that it doesn’t require an instance of the ClientThread class to run. ClientThread doesn’t really need instance data in this case as long as I can pass parameters, and because there is no ceiling on the number of ClientThread threads this saves a little overhead.

Starting ClientThread

Now we hit a snag. Thread.Start() calls a procedure given to the ThreadStart object’s constructor ?and that procedure can have no parameters. So what do I do when I want to start a function with parameters? It turns out the folks at Microsoft have a shell game to get around this limitation. Let’s play along and see if we can keep our eye on the ball.

First, how do we give parameters to a procedure that can’t accept parameters? We can create an object, set a few properties, and call a parameterless function that will refer back to those properties. Let’s look at this example:

// This class wraps the paramaters the form
            will be sending to the client thread
private class
            ClientThreadParams
{
// These three values become parameters
            to SendTriggerAtTime
public SendTriggerDelegate d1;
public
            DateTime dtTime;
public String sTrigger;

// This method is
            called by ThreadStart(), and calls the client thread
            SendTriggerAtTime in turn
public void CallClientThread()
{
ClientThread.SendTriggerAtTime(d1, dtTime,
            sTrigger);
}
}

Each parameter that SendTriggerAtTime() requires is represented by a property, and CallClientThread() uses those properties to call SendTriggerAtTime(). Using properties in this manner is impure from an object-oriented programming standpoint, mostly because the code doesn’t make it obvious where the property data is coming from.

Second, in order to start the client thread we need to create a new instance of ClientThreadParams and pass a reference to CallClientThread() to the ThreadStart constructor. For convenience, I’ve wrapped this sequence within another static function named BeginThread() :

public static void
            BeginThread(SendTriggerDelegate d, DateTime dtTime, String
            sTrigger)
{
ClientThreadParams param = new
            ClientThreadParams();
param.d1 = d;
param.dtTime =
            dtTime;
param.sTrigger = sTrigger;

Thread t = new
            Thread(new
            ThreadStart(param.CallClientThread));
t.Start();
}

Now whenever I want a thread to spawn and wait for a given time before submitting a trigger to the SendTrigger() function, I just call BeginThread() with a SendTriggerDelegate. Since the thread only manipulates private data I don’t have any synchronization worries.

Pros & Cons in this version:

  • Pros: A single function for thread access limits the number of ways we can get into trouble.
  • Pros: Locking access to aTriggers and aCount protects us from race conditions.
  • Pros: Only one lock() protects us from a deadlock.
  • Con: Use of an array for incoming data. The manner in which we use the aTriggers array makes it a LIFO (last in, first out) store. A collection type with FIFO (first in, first out) access would be preferable.
  • Con: Use of locks: Synchronizing our own data members is a fairly hardcore approach. It would be nice if we didn’t have to worry about synchronizing client threads.
  • Con: Instantiating a thread with parameters: ClientThreadParams is a little sloppy ?there might be a better way to structure that class.
  • Con: Trigger data is not encapsulated enough: Right now we are using raw strings for triggers. If we want to expand the type of data we use for triggers it will require a lot of class updates to accommodate that change.

Version 2 ?Using a System.Collections.Queue as a Message Receipt Mechanism

The next three sections show how a publicly exposed data member of the type System.Collections.Queue can be used as a messaging destination for a Thread. A Queue is a FIFO (first in, first out) data store that is better suited to our application, and it provides its own synchronization mechanism for clients to use. Arrays can be used in a FIFO manner, but we would have to write our own synchronized access functions that the Queue class provides for free.

This structure uses the same form as version 1, and is outlined in the UI Design and Coding section.

Adding a Queue to ServerThread

Instead of the string array (aTriggers) and array size variable (aCount) we used earlier, we declare a variable named MsgQueue of the Systems.Collection.Queue type. Note that unlike an array, the Queue does not need to be declared in conjunction with the type it is designed to hold.

public Queue MsgQueue; // holds TriggerMsg objects

Modifying DoWork () is relatively simple. Inside the while loop, instead of checking the value of aCount to see if we have messages waiting, we check the Count property of our MsgQueue object. Note that we do this without synchronizing thread access. Since we are the only thread dequeuing messages, the only race condition we have to worry about is if Count is being incremented as we are accessing it. In that case, if we get a zero result we will miss the new message until we come around the loop again ?that doesn’t present a problem. If we do get a positive count, then we will still be okay as long as we call Dequeue () in a thread safe manner. Providing, of course, that the other thread plays by the rules and uses a synchronized copy of the queue when calling Enqueue ().

In the previous version we synchronized access to our object by passing our this pointer to lock (). The Queue class provides it’s own synchronized implementation, freeing us from needing to worry about lock () ourselves. Instead we get a safe copy of our MsgQueue object by calling the Synchronized () method. Using a synchronized version of the queue is safe for every operation except enumeration (see the definition of Queue.SyncRoot in the .NET SDK for more information on enumeration and using lock () with queues).

if (MsgQueue.Count > 0)
{
// get a
            handle to a securable queue
Queue SyncQueue =
            Queue.Synchronized(MsgQueue);
TriggerMsg msg = (TriggerMsg)
            SyncQueue.Dequeue();
Console.WriteLine(msg.Trigger);
};

One last thing: because Dequeue returns an object, we have to cast that object to our message type. TriggerMsg is a simple class with a string property named, surprisingly enough, Trigger:

public class TriggerMsg
{
public String Trigger;
}

Putting Messages into the Queue : ClientThread

With SendTriggerAtTime, we swap the SendTriggerDelegate for a pointer to our destination queue, and the simple string trigger for a TriggerMsg. The message class gives us a little more type encapsulation for the parameters, and using a Queue maintains the encapsulation from ServerThread that using a delegate gives us.

public static void SendTriggerAtTime(Queue destQueue, DateTime dtTime, TriggerMsg msg)
{
// Sleep until the right time
Thread.Sleep(dtTime - DateTime.Now);

// Get a thread-safe handle to the queue before adding our message to it
Queue SyncQueue = Queue.Synchronized(destQueue);
SyncQueue.Enqueue(msg);
}

One important change is that because the ServerThread implementation handled synchronization in its SendTrigger method, the responsibility for synchronization now falls to ClientThread as we move to a queue implementation. For this reason we grab a synchronized copy of the queue before calling Enqueue () and passing the message we were given. In this case, we are relying on our clients to synchronize along with us; this is fine if we are writing the client code ourselves, but isn’t a good idea if you are writing a library or a reusable component. In that case, you’ll want to wrap Enqueue () and make sure it’s synchronized, similar to the way we wrapped the array access in the first version.

Cleaning up ClientThreadParams

In the previous version we invoked ClientThread using a class called ClientThreadParams. ClientThreadParams had public properties which had to be set before CallClientThread () was called. This isn’t absolutely the best coding practice ?even great programmers can forget a detail like this. We can fix it by marking the properties private, and creating a mandatory constructor. This ensures that a programmer won’t forget to set the properties, but not that they won’t pass garbage data. Verifying the parameters is an exercise left to the reader.

private class ClientThreadParams
{
public ClientThreadParams(Queue destination, DateTime time, TriggerMsg message)
{
destQueue = destination;
dtTime = time;
msg = message;
}

// These three values become parameters to SendTriggerAtTime
private Queue destQueue;
private DateTime dtTime;
private TriggerMsg msg;

// This method is called by ThreadStart(), and calls the client thread SendTriggerAtTime in turn
public void CallClientThread()
{
ClientThread.SendTriggerAtTime(destQueue, dtTime, msg);
}
}

Oh, and I updated it to pass a Queue and a TriggerMsg instead of a delegate and a string. BeginThread (not shown) is similarly modified.

Pros & Cons in this version:

  • Pro: By using a Queue for incoming messages we service the triggers in the same order they are submitted (first in, first out).
  • Pro: By using a Queue we don’t have to worry about Thread Synchronization in ServerThread except to call Synchronize to get a safe copy of the queue.
  • Pro: By wrapping the data in the TriggerMsg class we are able to change our data format without revisiting ClientThread ’s code (aside from a recompile).
  • Pro: Removing the public data members from ClientThreadParams and providing a mandatory parameterized constructor for the class gives us a guarantee that the values will be populated before CallClientThread() is executed.
  • Con: ServerThread is trusting the client to synchronize queue access.
  • Con: We are limited to the scalability of the Queue class, which is an X factor..

Version 3 ?Using a Microsoft Message Queue (MSMQ) as a Message Receipt Mechanism

These final sections detail how to transform this application to use the MSMQ product via the objects in the System.Messaging namespace. MSMQ is a robust system providing for remote access, offline storage of messages, bridging to heterogeneous networks (including UNIX, MVS, OS/2, AS/400, Unisys, and VMS systems), message priorities and a host of other features.

This structure uses an updated version of the form from the previous sections, and is outlined in the UI Design and Coding section. Be sure to check the Application Setup section for installation instructions for the Microsoft Message Queuing Services.

Messaging and MSMQ: A Brief Overview

Messaging is a communications paradigm and a programming methodology at the same time. A Message is, in its purest form, a self-contained object (or data package) sent from one source to a recipient. Most messages are sent for one of two reasons: either to advise the recipient of an event or action, or to request an action or data from the recipient. Messages should be designed to be as small as possible while still being complete. Completeness comes before size - you want to make sure the recipient receives all the information they need so that your message doesn’t require them to send other messages before taking action. Smaller messages are processed more quickly and less likely to encounter transmission errors. A message should also be independent ?understandable as a single entity, so that the recipient doesn’t need to know a previous context or message in order to act on it. Lastly, a message recipient should be designed to handle messages in a random order. Just as a message should be independent, the response to any message should be independent of what messages have already been received.

Like most rules, these can be broken and the world will still keep turning. Some messages will necessarily break these ‘rules?(especially size-wise). If you can manage to follow these rules the design and operation of your system becomes easier. When the messages are complete and the actions taken in response are independent, the system handles uncertainty surprisingly well.

Sometimes the rules must be broken, such as when you have to send data sequentially in multiple messages. Professional messaging systems (BizTalk, IBM MQ Series, etc) can compensate for this by offering Qualities of Service. A Quality of Service (QoS) is a guarantee by the messaging system about the way it will handle messages. When sequential messages are used, the system may request a queue with a QoS guaranteeing that messages will be delivered in the same order they are submitted. MSMQ offers other qualities of service, including persistence (storing the message on disk), security (e.g. authentication), and priorities.

We won’t be exploring advanced features of MSMQ in this case study. Instead, we’re just going to learn how to use a basic MSMQ queue. In order to accomplish this, we’ll explore a few of the System.Messaging classes, including the MessageQueue class which represents a local or remote MSMQ queue, and the Message class which represents the individual messages you will send or receive. Accessing the MSMQ queues are easy, but there is a cost associated with them. The queues are maintained by an unmanaged service in unmanaged memory, so your messages will be delayed as they are copied ("thunked") across the managed/unmanaged boundary. This delay might be noticeable if you use the queue heavily in a single managed process, but will not be a factor if you use the queue to send messages between processes or machines.

Enqueuing an MSMQ Message: ClientThread

In this version the SendTriggerAtTime function in ClientThread grows substantially although the basic job is pretty simple: put a TriggerMsg into the designated queue. The first step is to initialize a System.Messaging.MessageQueue object to use the queue named in the destQueue parameter. If we don’t make sure the queue exists first, we’ll get an exception (MSMQ defines over 100 reasons to throw an exception in this API, so you will encounter them quite often). That makes this piece of code our first order of business:

try
{
if
            (!MessageQueue.Exists(destQueue))
{
// Create the queue if it
            does not exist.
MessageQueue myNewPublicQueue =
MessageQueue.Create(destQueue);
}
}

The catch block isn’t shown - for this application, I just popup an error dialog (except for one handler that we’ll see later). This code is pretty straightforward - if the queue doesn’t exist, we create it. Queue names follow a particular nomenclature, depending on the type of queue you’re trying to use. There are two major queue types you will be dealing with. Public queues, which are accessible to MSMQ installations operating in a domain with an MSMQ server, follow the syntax of MachineName\QueueName. Public queues are replicated across the domain. Private queues use the syntax MachineName\Private$\QueueName. They are available on both domain and workgroup (single-machine or non-Active Directory) installations, but if they are on a domain they are not replicated. The MachineName can be specified as .\ for the local machine, or omitted entirely.

In MSMQ it doesn’t matter who creates the queue initially, as long as both the sender and receiver of messages are using the same queue address. It’s entirely possible for a sender to create a queue, put messages into the queue, and shutdown before the receiver even starts. That receiver can then pull the messages off the queue even though the client is shutdown - they are stored by MSMQ until retrieved. Messages can be set to expire while waiting for retrieval - you should check the SDK documentation for the MessageQueue.DefaultPropertiesToSend.TimeToBeReceived property for more information. By default messages never expire. Administrators can remove messages manually if necessary.

The next bit of code is pretty straightforward. Now that the queue definitely exists, we can create a new MessageQueue object and point it to our destination queue (destQueue).

try
{
// The XML formatter is default
            here
MessageQueue q = new
            MessageQueue(destQueue);
q.Send(msg);
}

There are four basic ways to determine the binary form a message will take when it’s in the queue. There are three formatters that are provided automatically in the System.Messaging namespace: XMLMessageFormatter, ActiveXMessageFormatter, and BinaryMessageFormatter. The XMLMessageFormatter is the default for a newly created MessageQueue object. It will take any managed object passed to Send () and serialize it to XML before putting it into the queue. Receive () is not so generous, as we will see later.

ActiveXMessageFormatter is used for interoperability with the ActiveX version of the MSMQ API, which serializes messages based on the IPersistStream interface from COM. The BinaryMessageFormatter renders messages to binary format instead of XML in order to provide a performance boost. The fourth method is the tried and tested do-it-yourself model where you create your own implementation of the IMessageFormatter interface.

Receiving an MSMQ Message : ServerThread

DoWork () has to undergo a few changes in order to accommodate using MSMQ. First, we have to setup the XMLMessageFormatter to receive TriggerMsg objects. While MessageQueue.Send () with the XMLMessageFormatter will serialize any type it comes in contact with, Receive () has to be primed with either the type declarations or type names of any class you expect to receive when you dequeue a message.

try
{
if (!MessageQueue.Exists(QueueName))
MessageQueue.Create(QueueName);
MsgQueue = new MessageQueue(QueueName);
XmlMessageFormatter xmlmf = new XmlMessageFormatter(new Type[]{typeof(TriggerMsg)});
MsgQueue.Formatter = xmlmf;
}

In the code above we use types directly instead of type names, specifying the TriggerMsg class as the only message payload we expect. In MSMQ, the Message is made up of dozens of properties, the data received by Send () is serialized into the Body property. The XMLMessageFormatter will automatically deserialize the Body to whatever type or types we tell it to ?in this case, TriggerMsg. Note that the XMLMessageFormatter constructor takes an array of types even though we only use one in this example.

You will want to take caution using this formatter to serialize objects. If the message class is derived from many other classes, either directly or through ‘deep?inheritance (class D inherits C inherits B, etc) your message could be much fatter than necessary. Declaring your message class separately is a safe bet in this regard.

The next major revision comes from within the while loop used to dequeue messages.

try
{
System.Messaging.Message
            TriggerMessage = MsgQueue.Receive(new
            TimeSpan(10000));
TriggerMsg msg = (TriggerMsg)
            TriggerMessage.Body;
StatusBar.Text = msg.Trigger;
}
catch
            (MessageQueueException e)
{
// Handle no message arriving in
            the queue.
if (e.MessageQueueErrorCode ==
            MessageQueueErrorCode.IOTimeout)
{
// do nothing, this is not
            fatal
}

?br> }

MessageQueue.Receive () is a synchronous call but it will take a TimeSpan as a maximum amount of time to wait before returning. If Receive () does exceed its allotted time span, it will throw an exception instead of returning normally ?this means Receive () pretty much needs its own try block in which to work. BeginReceive () provides an asynchronous method of retrieving Messages. This would be preferable if we were receiving messages on the Forms thread, but since we have already dedicated a thread to dequeuing messages our thread has nothing better to do than to give Receive () a second to wait. We catch the exception inside the while loop so that we can come back around and try again.

When Receive () does return properly it gives us the entire MSMQ Message and not just the payload. We access the payload through the Body property, casting it to one of the types we passed to the XMLMessageFormatter earlier. In the Version 3 form we show the trigger text on a new StatusBar provided when ServerThread is constructed, instead of the console.

Once we leave the while loop we go ahead and delete the queue as our last action:

try
{
if
            (MessageQueue.Exists(QueueName))
MessageQueue.Delete(QueueName);
}

This is not strictly necessary, it’s even inadvisable in a real-world application since you will delete any waiting messages. For the purposes of this sample it is better to clean up behind ourselves so that we don’t end up with a proliferation of dead queues. Under normal circumstances Queue names are fixed at design-time so orphaned queues are rare. Delete () will throw an exception if the queue doesn’t exist already, so we need to check that before calling Delete () to ensure an orderly shutdown.

Pros & Cons in this version:

  • Pro: Using MSMQ preserves FIFO order of delivered messages.
  • Pro: MSMQ continues to handle Thread Synchronization for us.
  • Pro: MSMQ accepts .NET classes as message types quite painlessly.
  • Pro: Using MSMQ gives us options to increase our program’s scalability beyond the capabilities of the Queue collection class.
  • Con: Using MSMQ demands careful exception handling.
  • Con: Higher program overhead accessing MSMQ queues.
  • Con: MSMQ has to be installed by System Administrators on any machine that will utilize it, and will require some configuration for a Domain.

UI Design and Coding

The form used in this case study is mainly designed to spawn the client and server threads. It will load a file as a list of triggers and the interval to wait before sending the trigger. The intervals do not have to be in order. When the Send Triggers button is pushed, the form spawns a client thread for each trigger, which then waits for the appropriate time to send that trigger to the server thread.

In the second major version of the form (the third version of our application) some controls have been added to set an MSMQ queue name and to switch between operating modes.

Form Structure, Version 1 & 2

Our initial form has three major controls: a Text box, a File Open button (marked as an ellipsis), and a Send Triggers button. The File Open button pops up a standard forms dialog that allows the user to select a file. This file is read directly into the edit box, and should consist of a number of seconds before a trigger is needed, then the trigger itself with a tab in between. Triggers and their counts are separated by newlines. Send Triggers reads the data from the edit box and spawns a thread for each to wait for the right time to send the given trigger.

SendTriggers_Click

This function takes the contents of the text box and spawns multiple client threads using the ClientThread.BeginThread() function. The delegate points to an instance of ServerThread -the delegate itself is reused since the garbage collector .NET uses will dispose of it automatically when all the client threads finish. A DateTime object is created using the static DateTime.Now() and DateTime.AddSeconds() methods together with a string-to-double conversion using the Convert class. Note that the for loop I used here iterates twice within the loop, since Triggers.GetLength() is expected to return an even number of strings (seconds and triggers). The string format is <seconds>\t<trigger>\n (e.g. "10 http://www.wrox.com/show.htm"). For more details see the Application Setup section.

private void SendTriggers_Click(object
            sender, System.EventArgs e)
{
// This creates an array of
            numbers of triggers, alternately
String[] Triggers =
            TriggerList.Text.Split(new Char[] {'\t', '\n'});

// We'll
            reuse this delegate; it will die when all the treads drop their
            references
SendTriggerDelegate d = new
            SendTriggerDelegate(instance.SendTrigger);

for (int i = 0; i
            < Triggers.GetLength(0);)
{
// Send the delegate, the time
            index from now, and the trigger
ClientThread.BeginThread(d,
            DateTime.Now.AddSeconds(Convert.ToDouble(Triggers[i++])),
            Triggers[i++]);
}
}

In Version 2, this final call to BeginThread changes slightly as we replace the delegate d with the destination queue:

ClientThread.BeginThread(destQueue, dt, msg);

Form Structure, Version 3

This new form adds three major new elements. First, it allows us to switch modes: "Client Only" mode allows you to load trigger files and send triggers to the specified queue; "Server Only" mode dequeues triggers from the specified queue; and the default "Single App" mode performs both functions just as the Version 1 & Version 2 forms did.

The next major element is the combination of the edit box (labeled "Queue:") and the Set button. The box remains read-only until the Set button is clicked, when the edit box is enabled so you can edit the name of the queue you are using. Clicking Set again resets the ServerThread, if it is running, and any subsequent ClientThread s will use this queue name as well.

Lastly I have added a Status Bar that is updated by ServerThread when a trigger is dequeued. This was an easy way to see what’s going on without the Console.

The Sample Application

The simplest way to start TDQ is to use the "Shortcut to TDQ.exe" shortcut located in the TDQ Project top level folder of the included files. This shortcut invokes the executable from the Release directory of the TDQv3 project. Any number of instances of this application can run simultaneously in Single App mode (although they shouldn’t reference the same queue when they do so!) or in a mixture of Client Only and Server Only mode. Multiple Client Only instances can send triggers to a single Server Only instance that uses the same queue.

Single App mode

The default operating mode of this application is "Single App" (single application) mode. In this mode the application runs a server thread dequeuing messages from the given queue, and as many client threads as necessary when the Send Triggers button is pressed. In order to use Send Triggers you first need to load a file specifying one or more triggers. Files are loaded by pressing the button marked with an ellipsis (?, and two files are provided: triggers.txt and stresstriggers.txt. Once a file has been loaded, the Send Triggers button becomes available. When that button is pressed, one client thread per trigger is created and that thread waits until the time for the trigger has elapsed. At that time, a message is deposited into the MSMQ queue specified. The server thread will remove messages from the same queue each second and post the trigger name to the status bar at the bottom.

The name of the queue can be changed by pressing the ‘Set?button, modifying the queue name, and pressing ‘Set?when finished. At this point the server thread will reset and begin dequeuing the new queue.

Client/Server Only modes

In Client Only mode, files can be loaded and the Send Triggers button will operate normally. Triggers will accumulate in the MSMQ queue unless another version of this application is running in either Single App or Server Only mode.

In Server Only mode, the ellipsis button and the Send Triggers button are disabled. At startup, a server thread is created to dequeue messages from the specified MSMQ queue. The queue name can be changed using the Set button. If the queue is changed, the server thread will reset and begin dequeuing from the new queue. When a message is dequeued, the trigger string will be displayed in the status bar.

Using the MSMQ Queue Manager

There are two simple ways you can monitor the contents of an MSMQ queue. If you are running the Visual Studio environment you may have a Server Explorer interface available (it is not included with the Standard editions of Visual Studio). Under this interface you can examine various aspects of servers you are connected into, including events, performance info, services, SQL processes, and MSMQ queues. You can select a queue to monitor by navigating to it and clicking on it in this interface.

A similar interface is available from the Control Panel. Inside the Control Panel, open the Computer Management applet and navigate through the ‘Service and Applications?section to the ‘Message Queuing?section, and then to the queue you want to monitor.

If you run the application in Client Only mode, load a file and press Send Trigger, you can watch messages as they accumulate in the queue using either of the methods above.

Application Setup

The sample code for all three versions of TDQ is provided in the zip file named TDQ.zip. Under the ‘TDQ Project?directory each version of the project is provided in its own subdirectory. For convenience, a shortcut to the TDQv3 project is also provided. That project also has links to the two previous versions.

In order for TDQ version 3 to work, you will need to have MSMQ and the Microsoft .NET Framework installed and running.

Installing the Microsoft .NET Framework

These samples require the Microsoft .NET Framework to operate. To install the Microsoft .NET Framework navigate to the following URL: http://msdn.microsoft.com/downloads/default.asp?url=/downloads/sample.asp?url=/msdn-files/027/001/829/msdncompositedoc.xml&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq. You should also install the Service Pack 2 available on this same page.

Installing MSMQ

MSMQ is included with Windows 2000 and Windows XP. To install, login with Administrator credentials and then go to Control Panel. In Control Panel, click Add/Remove Programs and select the Add/Remove Windows Components button. In the Windows Components Wizard scroll down the Components box to reveal "Message Queuing Services". Check the box next to "Message Queuing Services" if it is not already checked. Click ‘Next?and follow any instructions Setup presents. To install MSMQ 1.0 on Windows NT 4.0, Windows 98, or Windows ME, use the Windows NT Server 4.0 Option Pack: http://www.microsoft.com/ntserver/nts/downloads/recommended/NT4OptPk/default.asp.

During installation you may be presented with the choice between installing an MSMQ server, independent client, or dependant client. Servers are MSMQ controllers for an entire domain, and require Active Directory. Independent clients host their own queues and work if connected to a domain or separate. Dependant clients require an MSMQ server on the domain, and access queues stored on other machines. If in doubt, select Independent Client during setup.

NOTE: If you have MSMQ installed properly on your Windows domain, you will be able to specify queues on other machines as the destination queue in TDQv3. This scenario has not been tested.

Trigger file format

There are two trigger files included in this sample: triggers.txt and stresstriggers.txt. Triggers.txt shows a normal progression of ITV triggers, although slightly accelerated. Stresstriggers.txt fires four triggers at once in order to create a race condition for testing. The format of these files is: <#seconds><tab><trigger><newline> on each line of the file. ?seconds?indicates how long until the trigger is supposed to fire, and ‘trigger?is the text to be sent (an URL to a properly formatted website in the ITV world, although for the purpose of this sample any text will do).

Case Study Review

Programmers use threads for a number of reasons. A thread dedicated to the user interface makes that interface responsive while the real work happens on another thread. Long processes or intense calculations can benefit from their own thread as long as your application can perform other useful tasks in the meantime.

These benefits carry a price. Two or more threads accessing the same data at the same moment will cause unpredictable behaviors - bugs ranging from minor to serious. This is the worse kind of threading bug ?a race condition. Sometimes impossible to duplicate, a programmer faced with fixing these bugs will begin to wonder if the benefits of threading are worth the pain of debugging.

There are two major ways for a programmer to inoculate themselves against these bugs: synchronizing data manually via the lock() command, or running objects in their own synchronization context, where the .NET Execution Engine takes over synchronization tasks at the cost of locking your entire object as any thread enters it.

In this example we see how to lock data and how to use synchronizable objects (in this case, System.Collections.Queue) safely with multiple threads. We have to keep in mind that locking up data causes another potential threading bug ?the deadlock. In a deadlock situation, Thread A takes a lock while accessing data, then waits for data locked by Thread B. The problem arises if, under these circumstances, Thread B is waiting on the data that Thread A has locked. In this case both threads freeze and cannot resume (hence DEADlock). Being aware that kind of bug exists is the best way to prevent it.

One method of keeping threads from interfering with each other directly is to use messaging to communicate between them. By encapsulating data in messages you can keep threads from fighting over that data to some extent. Using messaging properly is also a good way to reduce the complexity of an application - threads that don’t care in what order messages are sent become more resilient in unpredictable situations.

What is Messaging, exactly? A Message, you’ll recall, is a self-contained object or data package sent as indication of an event or a command request. Ideally a message is as small as possible, has all the data the recipient needs, and is not dependant on knowledge of some prior state or sequence.

In our first thread example, rather than creating a message object and a receptacle for that object, we opt instead for a method abstracted by a delegate. A delegate is an object-oriented function pointer that allows you to create an object (the delegate) representing a method or static function. This delegate can be linked to any method fitting the defined profile: once attached to a method it can be passed to another object without revealing the source of the final method. This encapsulation gives you a way to mimic messaging to some extent. If the destination method is relatively stateless (making each method invocation work independently of other invocations) and the data passed with the delegate is complete, you can get a lot of the benefits of messaging with little overhead. Each declared delegate type can be treated as a message type and each instance of a delegate can be treated as the destination of a message.

If you want to move beyond delegates, the best receptacle for messages is a queue. In our second example, we defined a message type and used a System.Collections.Queue object as our message destination. Passing the queue as a destination still allows us to encapsulate the destination from the message sources. Queues also provide their own synchronization mechanism, preventing the scourge of deadlocks, and will automatically expand as message traffic increases. Sending messages to a thread using a queue is also a technique that Win32 programmers should be familiar with (ah, the glory days of PostMessage()).

Professional strength messaging offers many benefits over a managed queue. Most, but not all, of these benefits revolve around the idea of Quality of Service. A Quality of Service (QoS) is a guarantee from the messaging system about how it will handle your message. First-in first-out (FIFO) access to a queue can be considered a Quality of Service. Violating FIFO in order to service messages with a higher priority first is also a Quality of Service.

Microsoft Message Queue Service (MSMQ) is a professional messaging product included with Windows 2000 and Windows XP, accessible via the classes defined in the System.Messaging namespace. MSMQ offers message routing across a domain, priorities for messages, persistence of messages (the queue can exist even when the programs using it do not) and other qualities of service.

Using MSMQ does cost some overhead: the service runs in an unmanaged space so each call is going to cause some thunks (managed/unmanaged transitions). You will also need to pay some attention to how managed data is formatted going into and out of the queue ?in our last example we used the built-in XMLFormatter class to handle this task for us as we demonstrated how to send and receive messages using an MSMQ queue.

Any Limitations or Further Work

Turning this application into a real trigger inserter would require a few extra steps. There are actually two types of triggers; Transport A (closed captioned) and Transport B (bit-streamed). For a real world application the following steps would have to be taken:

  • Transport A triggers require a computed checksum.
  • Transport A triggers allow for a display title and other optional instructions.
  • A new file format and error handling in case the file is corrupt.
  • Support for Transport B.
  • A UI for specifying an inserter to call.
  • A feedback UI for the client would be optional.
C#Today - © 2002 WroxPress Limited


Runtime Reverse Engineering with System.Diagnostics

Runtime Reverse Engineering with System.Diagnostics
http://www.csharptoday.com/content.asp?id=1775
Article Rating: not yet rated
Published on May 27, 2002
Abstract
In this case study, Tony Loton looks at how you can trace the execution of a .NET application to find out which methods are invoked, on which objects, by which objects, and in what order; so you'll discover how your application really behaves at runtime. You'll learn how to turn tracing on and off, and how to configure the tracing behavior outside of the application. As the logical conclusion to this work we'll build a UML sequence diagram from an application trace, which could be used for comparison against an original dynamic model design or simply as an aesthetic form of presentation.
Case Study

In this case study we'll look at how you can trace the execution of a .NET application to find out which methods are invoked, on which objects, by which objects, and in what order; so you'll discover how your application really behaves at runtime.

You'll learn how to turn tracing on and off, and how to configure the tracing behavior outside of the application. Though we'll be using C# as the implementation language, our use of the System.Diagnostics classes should be equally applicable to any .NET language.

As the logical conclusion to this work we'll build a UML sequence diagram from an application trace, which could be used for comparison against an original dynamic model design or simply as an aesthetic form of presentation.

Introduction

In a previous case study, "C# to UML Reverse Engineering" (http://www.csharptoday.com/content.asp?id=1742&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq), I discussed techniques for reverse engineering a C# source file or any .NET assembly into a UML class diagram. I ended that case study by saying this.

"As implemented in many modeling tools, the term reverse engineering can be taken in a limited sense to mean static reverse engineering, and that's what we've dealt with in this case study. Start with a non-executing binary or source code file and end up with a static UML class diagram.

UML is not limited to static modeling and the UML notation supports dynamic modeling of the execution behavior of applications. I'm thinking here of sequence diagrams and collaboration diagrams, which you normally draw up-front to describe how you expect the application to behave. Wouldn't it be good if you could reverse engineer those diagrams too, to show how the application really does behave when you run it?"

In the discussion that follows we'll look at doing exactly that, and if you're not a UML fan don't tune out just yet. While a UML sequence diagram is the logical conclusion to this work, I'm sure you'll pick up something useful along the way in terms application profiling in general. no UML required.

Judging by the responses to a Java article that I wrote on the same subject some time ago, this topic should be interesting to a wide audience, so here goes.

System Requirements

You'll need the .NET Framework SDK that's available for free from the Microsoft web site at http://msdn.microsoft.com/netframework/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq, or the Visual Studio .NET that's not. Either way, you'll need to be able to launch a command window that's aware of the .NET environment, which VS.NET users can do by choosing the Microsoft Visual Studio .NET / Visual Studio .NET Tools / Visual Studio .NET Command Prompt program from the Windows Start menu.

If you'd like to take advantage of the Rational Rose script that I'll offer for rendering the results in UML, you'll need a copy of Rational Rose '98 or later. You could try picking up a trial copy from http://www.rational.com/tryit/rose/index.jsp?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq.

Defining The Problem

The problem I've set myself is that of capturing information from a running application so that we can trace which operations have been called, on which objects, in what order. That is, to come up with a trace like this:

which when (optionally) rendered as UML would be a sequence diagram like this:

Regardless of the representation used, what this information tells us is how the application really behaves at run time, so that you can compare that behavior with your original dynamic model. You did produce one at design time, didn't you? Alternatively it could prove to be a very handy technique for debugging.

Case Study Structure

  • In the "Defining the Solution" section we'll define the solution as comprising some instrumentation code, a compilation flag, and an application configuration file. Plus an optional Rational Rose script to build the UML sequence diagram.
  • In the "Component Design and Coding" section we'll step through the code of a program to insert instrumentation automatically into any application, and we'll look at the code for the Rose script.
  • In the "Sample Application" section I'll offer a simple Banking program, which we'll trace and for which I'll present the trace results.

Defining The Solution

It's not the first time I've looked at this kind of problem. The first time resulted in an article for Java Developer's Journal (March 2001) entitled "Using the Java Platform Debugger Architecture" and the second time resulted in chapter 10 "Servlet Debugging Techniques" of the Wrox Press "Professional Java Servlets 2.3" book (http://www.wrox.com/ACON11.asp?WROXEMPTOKEN=141193ZxsCGPbA0ntBAQn4Emvg&ISBN=186100561X). In the first case, the technology employed was the Java Platform Debugger Architecture (JPDA), and in the second case it was Servlet Filters and Event Listeners.

For the .NET solution we'll use two classes from the .NET Framework System.Diagnostics namespace: System.Diagnostics.Trace and System.Diagnostics.StackTrace. We'll instrument an application by inserting a few lines of code at the start of each method:

// ** START INSTRUMENTATION **

Get a stack trace that tells us where we are in the running application:

StackTrace stackTrace=new StackTrace();

The top stack frame will tell us which object / method is executing, and the next stack frame will tell us which (other) object invoked the method:

StackFrame topFrame=stackTrace.GetFrame(0);
StackFrame nextFrame=stackTrace.GetFrame(1);

Assume no calling object unless there really is a second stack frame, in which case we'll record the class (declaring type) of the calling object:

String __callingObject=":";
if (nextFrame!=null) __callee=":"+nextFrame.GetMethod().DeclaringType;

Now write out the calling object, the method that was called (i.e. the method into which this code has been inserted), and the class of the current object:

System.Diagnostics.Trace.Write(__callee);
System.Diagnostics.Trace.Write("|"+topFrame.GetMethod());
System.Diagnostics.Trace.WriteLine("|:"+topFrame.GetMethod().DeclaringType);

//
            ** END INSTRUMENTATION **

The result will be output as follows, with pipe-separated entries for the calling class (none on the first line), the method invoked, and the receiving class. Don't worry about the colon ":" characters just yet, which are there to allow for a future enhancement in which we would specify each object as instanceID:ClassName rather than just :ClassName.

:|Void Main(System.String[])|:Banking.Bank
:Banking.CurrentAccount|Void .ctor()|:Banking.Account
:Banking.Bank|Void .ctor()|:Banking.CurrentAccount
:Banking.CurrentAccount|Void .ctor()|:Banking.Account
:Banking.Bank|Void .ctor()|:Banking.CurrentAccount
:Banking.Bank|Double directdebit(Double)|:Banking.CurrentAccount
:Banking.SavingsAccount|Void .ctor()|:Banking.Account
:Banking.Bank|Void .ctor()|:Banking.SavingsAccount
:Banking.Bank|Double applyInterest(Double)|:Banking.SavingsAccount

But we won't get that at all unless we compile the instrumented application with the /d:TRACE option with this command:

> csc /d:TRACE Banking.cs

.and have a supporting application configuration file (named Banking.exe.config) with the following content:

<configuration>
<system.diagnostics>
<trace autoflush="true" indentsize="4">
<listeners>
<add name="myListener"
type="System.Diagnostics.TextWriterTraceListener"
initializeData=".\myListener.log" />
<remove type="System.Diagnostics.DefaultTraceListener"/>
</listeners>
</trace>
</system.diagnostics>
</configuration>

In that file we're telling the .NET Runtime to add a new listener for trace events, of type System.Diagnostics.TextWriterTraceListener, with the output destination specified as the file myListener.log. This listener will replace the default listener, which we've asked to be removed above.

So that's the basis of our solution comprising some instrumentation code, compilation with the d:/TRACE flag, and a supporting configuration file. Before moving on to the coding and demonstration I'll tell you a little more about each of those and about the optional Rational Rose scripting.

Instrumentation with System.Diagnostics classes

The "Introduction to Instrumentation and Tracing" within the "Visual Basic and Visual C# Concepts" section of the .NET documentation defines instrumentation as the ability for an application to incorporate:

  • Code tracing
  • Debugging
  • Peformance counters
  • Event logs

Those kinds of instrumentation are supported by the classes of the System.Diagnostics namespace. We'll be concentrating on code tracing and therefore the System.Diagnostics.Trace class. That said, the techniques discussed here could be extended to debugging, performance counters, and event logs ; also thanks to the Trace class, or by incorporating the Debug class from the same namespace.

The Trace and Debug classes have various static methods that allow you to write out messages to the output window, a text log file, or some other destination. Some of these methods are conditional, such as Trace.Assert(), and some are unconditional such as Trace.Write(). The code listed in the previous section uses the latter.

Trace output is not written out directly, but is intercepted by one or more trace listeners. You can add trace listeners into the Listeners collection, to supplement the DefaultTraceListener or to replace it completely. In code you could replace the default listener with new listener like this:

Trace.Listeners.Clear();
Trace.Listeners.Add(new TextWriterTraceListener(Console.Out));

In our solution we're not doing that because it would couple the tracing behavior too closely to the application code. Instead we're configuring the trace listener(s) via an application configuration file.

Application Configuration Files

Configuration files are XML files that can be used to change settings without recompiling applications. For an executable-hosted application (which is what my sample Banking application is) the configuration file is located in the same directory as the application. It has the same name plus a .config extension.

Banking.exe is the application.

Banking.exe.config is the corresponding configuration file.

As background information I'll tell you that configuration files for ASP.NET hosted applications follow a different scheme. In that case there will be a set of configuration files located at various levels in the URL path of the application. For an Internet Explorer application, the relevant configuration file will be specified by the <link> tag. You can read more about the various kinds of configuration files, for the various kinds of applications, at ms-help://MS.VSCC/MS.MSDNVS/cpguide/html/cpconapplicationconfigurationfiles.htm.

Because it is a standard XML file, an application configuration file has a schema to which the contents must conform. For Debug and Trace Settings, the schema elements (from ms-help://MS.VSCC/MS.MSDNVS/cpgenref/html/gngrftracedebugsettingsschema.htm in the .NET help documentation) are:

Element | Description
<add> element for the <listeners> element | Adds a listener to the Listeners collection.
<add> element for the <switches> element | Specifies the level where a trace switch is set.
<assert> | Specifies whether to display a message box when you call the Debug.Assert method; also specifies the name of the file to write messages to.
<listeners> | Specifies a listener that collects, stores, and routes messages.
<remove> | Removes a listener from the Listeners collection.
<swithces> | Contains trace switches and the level where the trace switches are set.
<system.diagnostics> | Specifies trace listeners that collect, store, and route messages and the level where a trace switch is set.
<trace> | Contains listeners that collect, store, and route tracing messages.

In our solution we're making use of the <system.diagnostics>, <trace>, <listeners>, <add> and <remove> elements. We're not using <switches> in our solution, so for completeness I'll just make you aware of what a trace switch is.

The System.Diagnostics.TraceSwitch class provides a multilevel switch to control tracing and debug output without recompiling your code, and you can use the trace levels to filter out messages based on their importance. Switches may be added and removed in the application configuration file, as with listeners.

In summary, the purpose of an application configuration file is to allow certain behaviors - including tracing - to be changed without requiring the application to be recompiled.

Compiling with the /d:TRACE flag

To enable tracing in a C# application you can either add a preprocessor directive #define TRACE to the top of the source file or add the d:/TRACE flag to the compiler command line. In this case study we're doing the latter, and you might like to try compiling with and without that flag to see the effect.

If you're working with Visual Basic you can enable tracing in a similar way by adding the /d:TRACE=True flag to the compiler command line.

If you preference is to compile from Visual Studio .NET itself, rather than from the command line, it's worth keeping in mind that by default the IDE compilation has tracing enabled. So unless you change it, you're released application could well be pumping out tracing information that you don't want it to. You can change this behavior via the Project Properties dialog, in the Configuration Properties / Build / Code Generation entry.

Rational Rose Scripting

Rational Rose has been one of the most widely used Object Modeling CASE tools in recent years, and as far back as I can remember it has had a scripting language that provides programmatic access to the Rose application and the model(s) housed within it.

The scripting language is called Summit BasicScript, and if you're familiar with early versions of Visual Basic (many visitors to this site, I suppose) you'll be quite comfortable with it. Access to the UML model (actually the meta-model) is provided by the Rose Extensibility Interface (REI) classes that represent the model elements - classes, methods, and so on - as well as the UML diagrams.

We'll make use of those classes via the scripting language, but the same set of classes is available for manipulation from your favorite Microsoft language via the OLE/COM objects of Rose Automation. If that's your preference, it shouldn't be too difficult to convert the script to a VB program, or for that matter even a C# program.

You can find more information about the scripting language and the extensibility classes in the Rose Entensibility Interface / Summit BasicScript Reference and Rose Extensibility Interface / Rose Extensibility Reference Rational Rose help topics.

Component Design and Coding

As described above, the key to our solution will be a few lines of instrumentation code inserted at the start of each method in the original program. Of course, you don't want to go through your program pasting that code into each and every method do you? So we'll automate it with a C# program.

We'll then implement a Rational Rose script to import the trace results and to build a UML sequence diagram reflecting the sequence of interactions between objects - i.e. the scenario - of the running program. That part's not compulsory, but quite interesting.

Automating Insertion of the Instrumentation Code

The following code of the InsertInstrumentation class will be used to read in any C# program and write it out again with instrumentation code included.

using System;

We'll make use of System.IO so as to read the original source file, and System.Text.RegularExpressions to help with parsing of that file. If you've read other articles that I've written you'll have been expecting regular expressions to appear somewhere, and - if not - you can get up to speed with them by reading my article "Introducing .NET Regular Expressions with C#" at http://www.csharptoday.com/content.asp?id=1786&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq.

using System.IO;
using System.Text.RegularExpressions;

Our class will be called InsertInstrumentation, housed within the reverse namespace:

namespace reverse
{
public class InsertInstrumentation
{

In the main method we open the original source file and call the instrument() method to insert the appropriate instrumentation:

        public
            static void Main(string[] args)
{
String
            inputString="";
String inputLine;

StreamReader sr =
            File.OpenText(args[0]);

while
            ((inputLine=sr.ReadLine())!=null)
inputString=inputString+"\n"+inputLine;

sr.Close();

instrument(inputString);
}

The instrument() method

The instrument() method is where we read the text of the input file, then write it out again with the instrumentation code inserted into each method:

private static void instrument(String text)
{

Our output file will contain modified source code with instrumentation. That output will only compile if we start by importing the System.Diagnostics namespace classes:

System.Console.WriteLine("using System.Diagnostics;");

As we go, we'll be keeping track of the previous match of the regular expression:

Match previousMatch=null;

Here we set up the regular expression itself. In a nutshell it matches each method declaration, which it reads into a named group (think of it as a variable) called "<method>", and any preceding code which is read into a named group "<priorcode>".

string regex=@"(?s)(?<priorcode>.*?)"
+@"(?<method>(public|private)"
+@"\s+[\w\s]*\w+\s*\(.*?\)\s*\{)";
Regex methodRegex=new Regex(regex);

That regular expression is not perfect because it handles only those methods that are declared explicitly as public or private. Once you become familiar with regular expressions - perhaps via my earlier article - you may wish to revisit this to account for protected and internal methods, plus those with no explicit access modifier.

Now we'll match the regular expression against the input text (the original source code) and step through the matching portions:

            Match
            methodMatch = methodRegex.Match(text);

for ( ;
            methodMatch.Success;
methodMatch =
            methodMatch.NextMatch())
{

For each match, we'll pick out the priorcode and the method declaration, and we'll write out the priorcode:

string priorcode=""+methodMatch.Groups["priorcode"];
string method=""+methodMatch.Groups["method"];
System.Console.WriteLine(priorcode);

Next we write out the method declaration along with the instrumentation code, via a separate method named writeInstrumentation(). We record this match as previousMatch and go round again:

writeInstrumentation(method);
previousMatch=methodMatch;
}

Because of how the regular expression works, we'll be left with some code at the end that we've not yet written out. Here we tidy up by writing the remaining code:

            if
            (previousMatch!=null && previousMatch.Success)
{
int
            remainingCodeStart=previousMatch.Index
+previousMatch.Length;
int
            remainingCodeLength=text.Length
-remainingCodeStart;
System.Console.WriteLine(text.Substring(remainingCodeStart
,remainingCodeLength));
}
}

You will have noticed in the above code that - upon discovering each match of the regular expression - we call a writeInstrumentation() method to actually write out the instrumentation code.

The writeInstrumentation() method

Here is that method, which takes in the method signature text as its sole parameter:

private static void writeInstrumentation(string method)
{

We write out the method signature itself as part of the modified source code, along with a comment marking the start of the instrumentation:

System.Console.WriteLine(method);
System.Console.WriteLine("\n// ** START INSTRUMENTATION **");

Now we write the instrumentation code that was explained previously in the Defining the Solution section:

System.Console.WriteLine(
"StackTrace stackTrace=new StackTrace();");
System.Console.WriteLine(
"StackFrame topFrame=stackTrace.GetFrame(0);");
System.Console.WriteLine(
"StackFrame nextFrame=stackTrace.GetFrame(1);");

System.Console.WriteLine("String __callingObject=\":\";");
System.Console.WriteLine(
"if (nextFrame!=null) "
+"__callingObject=\":\"+nextFrame.GetMethod().DeclaringType;");

System.Console.WriteLine(
"System.Diagnostics.Trace.Write(__callingObject);");
System.Console.WriteLine( "System.Diagnostics.Trace.Write(\"|\"+topFrame.GetMethod());");
System.Console.WriteLine(
"System.Diagnostics.Trace.WriteLine(\"|:\"+topFrame.GetMethod().DeclaringType);");

And we mark the end of the instrumentation with a comment:

            System.Console.WriteLine("\n//
            ** END INSTRUMENTATION **");
}

}
}

Remember that the result will be the original source file unchanged, save for the inclusion of instrumentation code at the start of every method (including constructors) like this:

public double applyInterest(double
            rate)
{

// ** START INSTRUMENTATION **
StackTrace
            stackTrace=new StackTrace();
StackFrame
            topFrame=stackTrace.GetFrame(0);
StackFrame
            nextFrame=stackTrace.GetFrame(1);
String
            __callingObject=":";
if (nextFrame!=null)
            __callingObject=":"+nextFrame.GetMethod().DeclaringType;
System.Diagnostics.Trace.Write(__callingObject);
System.Diagnostics.Trace.Write("|"+topFrame.GetMethod());
System.Diagnostics.Trace.WriteLine("|:"+topFrame.GetMethod().DeclaringType);
//
            ** END INSTRUMENTATION **

Balance=Balance*rate;
return
            Balance;
}
}

When we compile and run the instrumented application, as I'll show later, we'll get the logged output shown previously in Defining the Solution. That application trace may be further processed or displayed in a number of ways, one of which is as a UML sequence diagram in Rational Rose.

The Rational Rose Script (importScenario.ebs)

If you're using Rational Rose you'll no doubt find this script very interesting, and - if not - you may skip this discussion. But if you're working with another UML modeling tool you might just want to look through it anyway as it will give you some ideas about what you could do via the scripting language of your own tool.

In the Main subroutine we ask for an input file (the log file produced by the instrumentation) via a file dialog. We then pass the file name into the ImportScenario subroutine:

Sub Main
FileName$ = OpenFileName$ ("Import", "All files:*.*")
If FileName$ <> "" Then ImportScenario FileName$
End Sub

The ImportScenario subroutine follows. I've used the term scenario because, in UML terminology, a scenario is what we'll be presenting as a sequence diagram. You can think of a scenario as a usage pattern for application under investigation:

Sub ImportScenario (FileName As String)

Here we get hold of the current Rational Rose model, so that we can add to it:

Dim MyModel As Model
Set MyModel = RoseApp.CurrentModel

We'll be creating a new scenario (sequence) diagram with the same name as the log file containing the application trace, e.g. myListener.log or - more meaningfully - Banking.log, but only if that scenario does not already exist:

scenarioID = MyModel.RootCategory.ScenarioDiagrams.FindFirst (FileName)
Dim sDiagram As ScenarioDiagram
If scenarioID <> 0 Then
Set sDiagram = MyModel.RootCategory.ScenarioDiagrams.GetAt (scenarioID)
Else
Set sDiagram = MyModel.RootCategory.AddScenarioDiagram (FileName,1)
End If

Our sequence diagram will comprise object instances that exchange messages, with each instance having a unique ID. These variables will be used to keep track of the instances.

Dim instanceIDs() As String
ReDim instanceIDs(1)
instanceIDs(1)=""
Dim theInstances As New ObjectInstanceCollection

Note that our output log file supports the notion of sender and receiver objects of the form instanceID:ClassName, but in this case study we're actually not recording unique instance IDs for each object. The log file contents will always refer to objects as :ClassName and the resulting sequence diagram will be as shown earlier, with anonymous instances as preferred by many object modelers. If we improve the instrumentation in future to include instance IDs, this script will be ready.

Each message in our sequence diagram will have an ordinal sequence number, so here we start with number 1:

seq%=1

We open the viewport window to report the progress of the script and we open the input file for read-only access:

Viewport.Open
Open FileName$ For Input Access Read As #1

Now we step through the file, reading each line as we go:

While Not EOF (1)
Line Input #1, inString$

Each entry in the file will contain three pipe-separate fields: the sender (caller), the message (method), and the receiver (callee):

caller$=Item$(inString$,1,1,"|")
callee$=Item$(inString$,3,3,"|")
method$=Item$(inString$,2,2,"|")

The caller and callee objects could be identified by an instance ID and a class name, separated by a colon. Remember that for this case study, the instance IDs will actually be blank, but the script will take them if they're there:

callerID$=Item$(caller$,1,1,":")
callerClass$=Item$(caller$,2,2,":")
calleeID$=Item$(callee$,1,1,":")
calleeClass$=Item$(callee$,2,2,":")

For simplicity we'll put the two classes into the root category (root package) of the model, if they don't already exist there. A more advanced script could build a category (package) hierarchy reflecting the namespaces of the classes:

Dim thisCat As Category
Set thisCat=MyModel.RootCategory

If thisCat.GetAllClasses().FindFirst(callerClass$) = 0 Then
Set thisClass=thisCat.AddClass (callerClass$)
End If

If thisCat.GetAllClasses().FindFirst(calleeClass$) = 0 Then
Set thisClass=thisCat.AddClass (calleeClass$)
End If

Each object instance will be uniquely identified by a combination of its ID (not present in our log file) and its class, plus we'll use two variables to hold the caller and callee instances respectively.

callerInstanceID$=callerID$+":"+callerClass$
calleeInstanceID$=calleeID$+":"+calleeClass$
Dim callerInstance As ObjectInstance
Dim calleeInstance As ObjectInstance

We'll look to see if the caller instance has been recorded in the instanceIDs list and is therefore already on the diagram. We don't want to add a new object timeline every time this instance is the sender of a message:

foundCallerPos%=0
size%=UBound(instanceIDs)
For pos%=1 To size%
If instanceIDs(pos%)=callerInstanceID$ Then
foundCallerPos%=pos%
End If
Next pos%

If the caller instance is not already on the diagram, we add it to the diagram and also to the list of known instances:

If foundCallerPos% = 0 Then
Set callerInstance=sDiagram.addInstance(callerID$,callerClass$)
ReDim Preserve instanceIDs(size%+1)
instanceIDs(size%+1)=callerInstanceID$
foundCallerPos%=size%+1
theInstances.Add callerInstance
End If

Same again for the callee instance. Add it to the diagram if it's not already there:

foundCalleePos%=0
size%=UBound(instanceIDs)
For pos%=1 To size%
If instanceIDs(pos%)=calleeInstanceID$ Then
foundCalleePos%=pos%
End If
Next pos%
If foundCalleePos% = 0 Then
Set calleeInstance=sDiagram.addInstance(calleeID$,calleeClass$)
ReDim Preserve instanceIDs(size%+1)
instanceIDs(size%+1)=calleeInstanceID$
foundCalleePos%=size%+1
theInstances.Add calleeInstance
End If

Make sure we have a handle on the sequence diagram representations of the caller - and callee - instances, then add a message (the method signature) from one to the other:

Set callerInstance=theInstances.GetAt(foundCallerPos%-1)
Set calleeInstance=theInstances.GetAt(foundCalleePos%-1)

Dim theMessage As Message
Set theMessage=sDiagram.CreateMessage(method$,callerInstance, calleeInstance, seq%)

Give the user some feedback in the viewport window, increase the sequence number for next time around, and complete the loop:

Print callerClass$,"---",method$,"-->",calleeClass$
seq%=seq%+1
Wend

Finally, close the input file nicely and end the subroutine:

Close #1
End Sub

Sample Application

In this section we'll take a sample application, instrument it, and view the trace results as a simple list. Then we'll reverse engineer a sequence diagram representation using the Rose script.

You'll find everything you need in the download, which contains the following files. Some of them will be obvious immediately and the purpose of the others will become clear as you read on.

Download Contents

File Banking.cs contains the source code for the sample application (see below).

File Banking.exe is a compiled version of the sample application.

File Banking.save is a spare copy of the Banking.cs source code, just in case.

File importScenario.ebs is the Rational Rose script that builds the UML sequence diagram.

File InsertInstrumentation.cs is the source code of the program to add instrumentation.

File InsertInstrumentation.exe is the compiled version.

File instrumented\Banking.cs will be the instrumented version of the sample application.

File instrumented\Banking.exe is the compiled version.

File instrumented\Banking.exe.config is the application configuration file for the sample application.

File instrumented\myListener.log is where the trace results will be written.

File build.bat contains the command csc /d:TRACE Banking.cs to compile the instrumented sample application.

File sampletraces\BankingTrace.mdl represents the sample trace as a Rational Rose model.

File sampletraces\BankingTrace.xls represents the sample trace as an Excel speadsheet.

File sampletrace\BankingTrace.xml represents the sample trace as an XML file, for Internet Explorer.

Banking Application Code

If you've read my previous (Static) Reverse Engineering case study at http://www.csharptoday.com/content.asp?id=1742&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq you'll be familiar with my sample Banking program. I've changed it slightly for this case study and the latest version is provided in file Banking.cs of the code download.

I wont give a blow-by-blow account of every single line of code because that's not important, but here is the complete code with a few useful pointers.

using System;

namespace Banking
{

The Account class will be a base class for various types of account, such as CurrentAccount and SavingsAccount. We won't be concerned with the member variables and properties but the constructor will be instrumented:

    public class
            Account
{
private static int
            _nextAccountNumber;

private int _accountNumber;
private
            double _balance;

public
            Account()
{
_accountNumber=_nextAccountNumber;
_nextAccountNumber++;
}

public
            int AccountNumber
{
get { return _accountNumber;
            }
}

public double Balance
{
get { return _balance;
            }
set { _balance=value; }
}
}

The CurrentAccount class extends Account. It will have its constructor and its directdebit() method instrumented:

    public class
            CurrentAccount : Account
{
public
            CurrentAccount()
{
}

public double directdebit(double
            amount)
{
Balance=Balance-amount;
return
            Balance;
}
}

The SavingsAccount class extends Account. It will have its constructor and its applyInterest() method instrumented:

    public class
            SavingsAccount : Account
{
public
            SavingsAccount()
{
}

public double applyInterest(double
            rate)
{
Balance=Balance*rate;
return
            Balance;
}
}

The Bank class has a Main() method to drive the other classes through a typical sequence of events, i.e. a scenario:

public class Bank
{
public static void Main(string[] args)
{
// Current Account #0
CurrentAccount currentAccount=new CurrentAccount();
currentAccount.Balance=100;
System.Console.WriteLine("Current Account "
+currentAccount.AccountNumber
+" has balance "+currentAccount.Balance);

// Current Account #1
currentAccount=new CurrentAccount();
currentAccount.Balance=100;
currentAccount.directdebit(50);
System.Console.WriteLine("Current Account "
+currentAccount.AccountNumber
+" has balance "+currentAccount.Balance);

// Savings Account #1
SavingsAccount savingsAccount=new SavingsAccount();
savingsAccount.Balance=1000;
savingsAccount.applyInterest(1.1);
System.Console.WriteLine("Savings Account "
+savingsAccount.AccountNumber
+" has balance "+savingsAccount.Balance);
}
}
}

Just to clarify, the sequence of events is:

  • Create a CurrentAccount.
  • Create another CurrentAccount and call its directdebit() method.
  • Create a SavingsAccount and call its applyInterest() method.

Instrumenting the application

If you don't have an InsertInstrumentation.exe file, create one by compiling the InsertInstrumentation.cs C# program:

> csc InsertInstrumentation

Now run the InsertInstrumentation program against the Banking.cs source file, taking care to divert the output into the \instrumented directory.

> InsertInstrumentation Banking.cs > instrumented\Banking.cs

For convenience, if you accidentally overwrite the original Banking.cs file with the instrumented version (yes, I've done that!) you should be able to recover by renaming file Banking.save to be Banking.cs.

You might like to take a look at the instrumented version of the application source code before moving on.

Compiling the application

Compile the application in the \instrumented directory with tracing enabled, like this:

> cd instrumented
> csc /d:TRACE Banking.cs

At this point it's worth noting that the \instrumented directory contains a file called Banking.exe.config, which was not generated by the compilation but written in advance by me. You can use my file as the starting point of a configuration file of your own applications by simply renaming it to YourApplication.exe.config. I explained the contents of the file earlier, but as a reminder here it is again:

<configuration>
<system.diagnostics>
<trace autoflush="true" indentsize="4">
<listeners>
<add name="myListener"
type="System.Diagnostics.TextWriterTraceListener"
initializeData=".\myListener.log" />
<remove type="System.Diagnostics.DefaultTraceListener"/>
</listeners>
</trace>
</system.diagnostics>
</configuration>

Note that this application configuration file specified the file myListener.log to take the output of the TextWriterTraceListener.

Running the application

Now we can run the instrumented application and see its output:

> Banking
Current Account 0 has balance 100
Current Account 1 has balance 50
Savings Account 2 has balance 1100

For this case study we're not interested in the output from the program itself, but in the contents of the myListener.log file which are:

:|Void Main(System.String[])|:Banking.Bank
:Banking.CurrentAccount|Void .ctor()|:Banking.Account
:Banking.Bank|Void .ctor()|:Banking.CurrentAccount
:Banking.CurrentAccount|Void .ctor()|:Banking.Account
:Banking.Bank|Void .ctor()|:Banking.CurrentAccount
:Banking.Bank|Double directdebit(Double)|:Banking.CurrentAccount
:Banking.SavingsAccount|Void .ctor()|:Banking.Account
:Banking.Bank|Void .ctor()|:Banking.SavingsAccount
:Banking.Bank|Double applyInterest(Double)|:Banking.SavingsAccount

That shows a complete trace of the constructors and other methods that have been invoked during the execution of the application. Each line shows the calling object (blank on the first line), the message that was sent (or method invoked), and the receiving object.

It's not the most eye catching representation, so to make it clearer I simply opened the log file in Microsoft Excel and applied a little formatting to the columns to give:

In the Case Study Review section I'll offer some more thoughts on presenting and analysing the application trace.

Building the UML Sequence Diagram

If you have access to Rational Rose you can use my script to build a real UML sequence diagram for the scenario represented by the application trace.

First, launch Rose and use the Tools / Open Script menu option to open the importScenario script; or simply double-click the script file to launch Rose with the script already loaded. You can then run the script by pressing the green "Start" button.

Choose the file that contains the logged trace results, i.e. myListener.log:

Then watch the messages that appear in the Viewport window:

Finally you will discover that the Rose model browser contains a set of classes, and a new sequence diagram representing the application trace, as shown here:

That completes the sample application, but did you notice anything odd about the sequence of events? According to that trace it looks as though the Banking.CurrentAccount calls its superclass Banking.Account constructor before it has had its own constructor invoked. This pattern is repeated for every subclass / super class construction and there's not much we can do about it.

To understand what's going on here you have to imagine that the subclass constructor as an implicit first line of code that invokes the constructor of the superclass, and which --in effect - causes the superclass instrumentation to be executed before its own instrumentation. This following diagram might help:

Apart from that, the sequence of events is ordered as you would expect for the sample application.

Case Study Review

In this case study we've investigated a technique for instrumenting a C# application, which allows us to capture the runtime - i.e. dynamic - behavior of the application. The resulting application trace may be useful as a teaching aid (to see demonstrate objects really interact), and for debugging or performance monitoring (if we include method timings in the instrumentation code).

Ultimately it allows us to reverse engineer a UML sequence diagram to complement the reverse engineered class diagrams demonstrated in my previous case study http://www.csharptoday.com/content.asp?id=1742&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq. In a typical project you may well draw sequence diagrams for your applications up-front to describe the intended behavior, and now we have a way of validating those diagrams against the actual behavior during testing.

For the instrumentation we used mainly the System.Diagnostics.Trace class, and the System.Diagnotics.StackTrace class to discover the calling (caller) and called (callee) objects. We automated the insertion of instrumentation code into every method by implementing the InsertInstrumentation program. Through that program I demonstrated a practical use for regular expressions.

We looked at how tracing could be enabled and disabled with the /d:TRACE compiler flag and how the tracing behavior could be configured outside of the target application in an application configuration file.

With the help of a Rational Rose script I showed how the end results really could be rendered in UML, and finally I used the sample sequence diagram to highlight an interesting feature about the order of class construction.

During the review phase for this case study, one of the reviewers made a very interesting point that this approach should be a method of last resort. Surely it would be better to insert trace statements as you build an application rather than as an afterthought? My own opinion is that it would be less flexible, i.e. hard-coded, and possibly inconsistent across the application, but it's certainly something you should be thinking about as part of the design.

Notwithstanding the recommendations I've given in the next section, the important messages to take away from this case study are:

  • You can capture information about the behavior of your application as it runs, i.e. you can capture an application trace, using the classes of the System.Diagnostics namespace and the techniques described here. But there may be other ways, mentioned in "Non-invasive Tracing" below.
  • The information you capture may be useful for debugging purposes, for measuring performance (if we included counters in the instrumentation), or simply as a learning tool. Remember the order-of-construction effect that was highlighted in the example?
  • Ultimately you can reverse engineer the runtime (dynamic) behavior of an application and present it as UML; to complement any static reverse engineeringyou may do, or to validate the application designer's original dynamic model.

Remember that although we've used C# in this case study, in .NET all languages are equal. So it shouldn't be too difficult to adapt this approach to work with Visual Basic, C++ or even J# applications.

Limitations and Further Work

In this case study we've made a good start on the problem of reverse engineering the behavior of an application as it runs. In principle it works, and there are a few improvements that I can think of that you might like to work on.

Intermediate File Format

I kept the intermediate file format quite simple, mainly because my Rose scripting skills are not up to parsing a more complex structure such as XML. But I do think that XML would be a sensible choice for the intermediate results, so its something you might like to think about. I envisage an XML file like this:

<trace>
<interaction seq="1">
<sender>:</sender>
<message>Void Main(System.String[])</message>
<receiver>:Banking.Bank</receiver>
</interaction>

<interaction seq="2">
<sender>:Banking.CurrentAccount</sender>
<message>Void .ctor()</message>
<receiver>:Banking.Account</receiver>
</interaction>
</trace>

The rendering in Internet Explorer would be:

Processing the Results

We've processed the trace results by loading them into an Excel spreadsheet and by constructing a UML sequence diagram in Rational Rose. By writing out the trace as XML (above) you could open them up to a wider range of analysis or presentation tools, or maybe write a dedicated trace viewer yourself.

Remember that we've decoupled the trace handling from the application itself through the use of an application configuration file. In that file we specified a TextWriterTraceListener for writing the results to a log file, which replaced the default trace listener that would write the results as console output.

You can write your own TraceListener classes and add them in the same way, thus opening up a whole new set of possibilities. What about a trace listener that sends out each trace event as a SOAP message to a remote real-time debugging or profiling tool?

Alternative Instrumentation

As implemented here we've made use of the System.Diagnostics.StackTrace class to obtain information on the calling class, the called class, and the method that was invoked, and we've written that information out via the System.Diagnostics.Trace.Write() method. In a typical method, the instrumentation code looks like this:

public double applyInterest(double
            rate)
{

// ** START INSTRUMENTATION **
StackTrace
            stackTrace=new StackTrace();
StackFrame
            topFrame=stackTrace.GetFrame(0);
StackFrame
            nextFrame=stackTrace.GetFrame(1);
String
            __callingObject=":";
if (nextFrame!=null)
            __callingObject=":"+nextFrame.GetMethod().DeclaringType;
System.Diagnostics.Trace.Write(__callingObject);
System.Diagnostics.Trace.Write("|"+topFrame.GetMethod());
System.Diagnostics.Trace.WriteLine("|:"+topFrame.GetMethod().DeclaringType);
//
            ** END INSTRUMENTATION **

In fact, the instrumentation code looks exactly like that, regardless of the method it's in.

Now think about an alternative instrumentation that is method-specific and which therefore sends out some method related information. like the parameter values. I'm thinking about a line of code like this inserted into the method above.

System.Diagnostics.Trace.WriteLine(
"|applyInterest( rate="+rate+" )");
);

That information would be quite useful and you might like to think about inserting such lines of code via the InsertInstrumentation program. I think the contents of the regular expression's <method> group will tell you everything you need to know about the parameter names.

Finally in terms of improved instrumentation, you could think about how to identify the actual object instance on which the method has been invoked; not just the class. Remember that the Rational Rose script can cope with individually-identifiable instances when building the sequence diagram.

Non-Invasive Tracing

The approach I've described decouples the tracing behavior from the application code thanks to the application configuration file. The tracing can be enabled or disabled at the flick of a compiler switch, and a lot of typing is saved by the automated insertion of the instrumentation code.

All that said, this approach is essentially invasive; which means we must modify the original application code. In contrast, the runtime reverse engineering solutions that I devised using Java were non-invasive, using the Java Platform Debugger Architecture (JPDA) for applications in general and using sevlet filters and event listeners for web applications specifically.

Non-invasive techniques for tracing of .NET applications are worth investigating. You could try driving the Machine Debug Manager using DCOM, or maybe take advantage of Windows Management Instrumentation (WMI). I'm no expert on the latter, but it looks promising and you can read Richard Conway's article on that technology at http://www.csharptoday.com/content.asp?id=1807&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq. Either way, my past experiences tell me that by using these non-invasive techniques it could be much more difficult to capture the method-specific information that I described above.

Case Study Information
Author | Tony Loton
Technical Editor | Adam Ryland
Project Manager | Helen Cuthill
Reviewers | Andrew Krowczyk, Saurabh Nandu, John Hodgson, Adam Ryland

If you have any questions or comments about this case study, or suggestions for further pieces, please contact the technical editor. If you wish to talk about the case study with other subscribers, please visit our forum here.

C#Today - © 2002 WroxPress Limited


Timing Code in .NET

Timing Code in .NET
http://www.csharptoday.com/content.asp?id=1800
Article Rating: 4
Published on January 17, 2002
Abstract
In many situations it is useful to know how long it takes for some lines of code to execute in order to find potential bottlenecks in applications. Unfortunately, the native timers available under .NET are not suited for high-resolution timing. Instead, it is possible to use the interop services in .NET to access the native Win32 API and use the timing functions provided there. In this article, Alex Lehmberg will show how to time code using the Win32 API, and will illustrate the uses of timing. In order to do that, he will make an XSLT transformation of an XML file using both the native .NET functions and the ordinary MSXML 3.0 to compare which one is faster.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

Despite the overwhelming number of classes built into the .NET class libraries, it is still possible to find areas where developers might miss some functionality. One of the things missing in native .NET is the ability to work with high-resolution timers. The only native timer built into .NET is meant to be used in programs where you want a certain event to be raised at some predefined intervals. As a consequence, the resolution is measured in seconds, which is totally inadequate if you have the need to time single lines of code.

There is no need to despair, as the interoperability functions in .NET makes it possible to call functions in the native Win32 API. The Performance Counters in the System.Diagnostics namespace uses the interoperability functions to provide high-resolution timing, and the same principles can be used to write our own High Resolution Timer.

To make it easier to work with high-resolution timers in .NET, we want to make a timer class that calls the Win32 functions under the covers. We will then use the class in our Windows Forms application to make it possible to time XSL transformations. In .NET, there are two roads you can take when an XSL transformation is needed. The simplest and most logical is to use the System.Xml.Xsl namespace. Alternatively it is possible to use the standalone XML parser MSXML 3.0 - all that is needed is a reference to the component in your VS.NET Project. There are several reasons why you might want to use the "old" msxml.dll. One reason could be that your organization has a lot of legacy code that uses msxml.dll. Another, and possibly more interesting reason, could be that msxml.dll could turn out faster than the functions provided by .NET.

API overhead

As mentioned before, the .NET framework provides us with two options when we want to time code. The first and perhaps easiest way is to use the performance counter in the System.Diagnostics namespace, but it is also possible to do all the interoperability code yourself. Given the DllImport attribute, it isn't very much work, and the advantage of this approach is that you know precisely what is going on when you time code. The same cannot be said when using the built in performance counters, and it turns out that the API overhead of using the performance counter classes is substantially higher than our homegrown method.

To illustrate this point, I wrote a small application designed to time the overhead when using the two different approaches. The result can be seen in the screenshot below:

The numbers on the left are made using the performance counter, while the ones on the right are made using the High Resolution Timer class that we write in this article. A difference of 3 against 1704 is quite a difference, and even if we could make the overhead of calling the performance counter smaller, there is still the advantage with our own class that we know exactly know what it does. The numbers are in performance counter units, and have to be divided by the frequency to obtain the result in seconds.

Regardless of the approach you choose, it only serves to illustrate the enormous flexibility of the .NET framework. It is very common that similar result can be achieved in many different ways.

System Requirements

The article is based on the first Release Candidate of Visual Studio .NET (Framework version 1.0.3328). The program will not work as intended if used on non NT based systems (Windows 98/ME) as the high-resolution timer doesn't work. It is required that MXSML 3.0 is installed on the machine - it can be downloaded from Microsoft's website (http://www.microsoft.com/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq). No knowledge of XML or XSL is required, as sample files are included in the download. The emphasis is on timing and not on XML/XSL.

Defining Problem

When we want to roll our own timing functions we need to examine the possibilities using Win32, and it turns out that all our needs are served by only two functions residing in kernel32.dll. The functions are called QueryPerformanceCounter and QueryPerformanceFrequency, and we will discuss both functions in detail shortly.

Our Visual Studio Solution is made up of two separate projects. The first project contains the timer class, and the second project is our Windows Form application that makes it possible to time XSL-transformations. The timer class is generic and can be used in any application where timing is required, but the code using the timer class will have to be hard coded every time to make sure that the right sections of code are timed. If a developer wanted to time a loop doing some work he wouldn't have much use for the XSL-transformations included in the sample.

Defining Solution

Component Design

As already mentioned, the two functions we need to implement a high-resolution timer are QueryPerformanceCounter and QueryPerformanceFrequency. QueryPerformanceFrequency is used to retrieve the resolution of the timer in the system. If the function returns zero, it means that no timers are supported on the system, otherwise the number returned is the number of counts per seconds, or in other words the frequency. Any subsequent calls to QueryPerformanceFrequency will return the same value, as the frequency remains constant once the system is started. Because of that, it is only necessary to call the function once during the timer's lifetime.

The second function, called QueryPerformanceCounter, is the one doing the actual timing. When a call is made to QueryPerformanceCounter, the current value of the high-resolution performance counter is returned. If a call returns zero, it means that an error has arisen. To calculate the time elapsed between two calls to QueryPerformanceCounter, you only need to subtract the two values and divide them by the frequency to get the time in seconds. If the first call to QueryPerformanceCounter returns 40,000, the second one returns 170,000, and the call to QueryPerformanceFrequency returns 600,000, then the time elapsed is about 0.22 seconds ((170,000-30,000)/600,000).

The first four lines of our high-resolution timer class imports the functions for use in .NET using the attribute DllImport :

[DllImport("kernel32.dll")]
extern static short QueryPerformanceCounter(ref long x);
[DllImport("kernel32.dll")]
extern static short QueryPerformanceFrequency(ref long x);

The DllImport attribute tells the compiler that the method carrying the attribute is implemented in an unmanaged DLL - in this case kernel32.dll. It is possible to use the DllImport attribute to customize several aspects of the import process, but in this case only the name of the DLL is needed. When using the DllImport attribute, it is necessary to include the namespace System.Runtime.InteropServices, otherwise the code will not compile. The constructor is quite simple and performs two tasks. It initializes the private variables start and stop to 0, and it calls QueryPerformanceFrequency to retrieve the number of counts per second.

public
            HighResTimer()
{
QueryPerformanceFrequency(ref
            freq);

start = stop = 0;
}

As mentioned earlier, the frequency is zero if the high-resolution counter is not supported, and the method supported() checks the variable freq and returns true if the counter is supported, and false otherwise:

public bool Supported()
{
if(freq !=
            0)
return true;
else
return false;
}

The methods Start() and Stop() are called just before the code to time starts, and just after, respectively. Apart from using different variables, they are identical, and they both call the imported function QueryPerformanceCounter to retrieve the current value of the counter. It would arguably be more logical to have only one timing method which could have been named something like Mark(), but it is easier to think in terms of Start() and Stop(). When the timing is done the user can call the method SecondsElapsed() to get the time in seconds:

public double SecondsElapsed()
{
return (stop-start) *1.0 / freq;
}

The last method provided in the HighResTimer class is called QueryTimer() and it returns the resolution in seconds. Although there are no methods designed to return split times, it is possible to do this by calling the Stop() method repeatedly. If the SecondsElapsed() method is called after every call to Stop() then the accumulated time since the Start() method was called is returned.

UI Design

The application is made up of a single Windows Form where the user is able to choose which XML and XSL files to use in the transformation, what kind of transformation is done, and how many iterations should be timed. Almost all significant code is executed when one of the four buttons are pressed.

The first thing to do when setting up a transformation is to pick the files to be used, and to that end we use an OpenFileDialog :

private void btnXML_Click(object sender, System.EventArgs e)
{
if(fileXML.ShowDialog() == DialogResult.OK)
tbXML.Text = fileXML.FileName;
}

When that is done, it is possible to choose whether to do a .NET or a msxml.dll transformation by checking the appropriate radio button. The last thing to do before timing can begin is to decide how many iterations should be tested. The default value is 1, but the more iterations that are done, the more representative the final result will be. It is probably a good idea to start with a low number of iterations to find out how long it takes, otherwise you may have to wait several minutes for the timing to end.

There are two buttons in the lower part of the form called Start Timing and Query Timer. The button called Query Timer gets the resolution of the high-resolution timer and displays the value in the textbox called Resolution. The handler for the button is shown below:

private void btnQuery_Click(object sender,
            System.EventArgs e)
{
HighResTimer testTimer = new
            HighResTimer();
double Res;
if(testTimer.Supported())
{
Res = testTimer.QueryTimer();
tbRes.Text =
            Res.ToString();
}
else
MessageBox.Show("High Resolution
            timer not supported");
}

You will notice that the same value is returned every time the button is pressed, as the resolution remains the same when the system is running.

The actual timing is done when the button Start Timing is pressed. The first thing that happens is that the value entered into the Iterations textbox is converted to a short. If that is not possible, an exception is thrown and an error message is displayed:

try
{
iters = Convert.ToInt16(tbIter.Text);
}
catch
{
MessageBox.Show("Wrong format in Iterations TextBox");
}

After that, the program checks to see if the timer is supported on the system, and if so the code branches according to what type of transformation was chosen. I will only show the code for transformations using .NET, as the principle is the same:

XPathDocument doc = new
            XPathDocument(fileXML.FileName);
XslTransform myTransform = new
            XslTransform();
myTransform.Load(fileXSL.FileName);
MemoryStream store = new MemoryStream();
XPathNavigator nav
            =((IXPathNavigable)doc).CreateNavigator();
myTimer.Start();
for(i=0;
            i<iters;i++)
myTransform.Transform(nav,null,store);
myTimer.Stop();

The first four lines set up the transformation that we don't want to time. Then the timer is started, and the loop runs making a transformation for every iteration. When the loop is finished, the timer is stopped. In this case we store the result of the transformation in a memory stream, but other options are available. It should be noted that the way transformations are done are different between .NET and msxml.dll, and any conclusion as to which one is faster should be drawn with care. The reason is that the libraries are different in structure, which makes it hard to make precise comparisons.

If the measurement of time has to be as precise as possible, one thing should be changed in the code. As it is now, the time it takes to call the methods Stop() and Start() are included in the final result. If the timed code sections executes very fast you might want to isolate the time from the overhead of calling the timer methods. To do so, all you have to do is to call Start() and Stop() after each other and subtract the value from the final result. Below is the corresponding code to do transformations using msxml:

IXMLDOMDocument2 doc = new
            DOMDocument30();
doc.load(fileXML.FileName);

IXMLDOMDocument2
            transform = new
            FreeThreadedDOMDocument30();
transform.load(fileXSL.FileName);

IXSLTemplate
            template = new XSLTemplate30();
template.stylesheet =
            transform;
IXSLProcessor processor =
            template.createProcessor();
processor.input =
            doc;
myTimer.Start();
for(i=0; i<iters;
            i++)
processor.transform();
myTimer.Stop();

Another thing to note is that the first transformation is always slower than any subsequent ones. Why that is I'm not sure, but the most obvious explanation is that all the JIT compilation and setup is done the first time (I would like to hear any other explanations!).

When the timing is done, the total time elapsed and the average time for each iteration is calculated and shown in the textboxes using the code:

Total = myTimer.SecondsElapsed();
try
{
Iter = Convert.ToDouble(tbIter.Text);
IterRes = Total/Iter;
tbTotal.Text = Total.ToString();
tbAverage.Text = IterRes.ToString();
}

The Sample Application

The screenshot above shows the result of timing a transformation using the XSL framework in .NET. The test was done on a machine with a Celeron 466 processor and 256 MB RAM.

It is important that the computer doing the timing is not doing any heavy processing in the background, because that would influence the results and make any comparisons harder.

Below is a screenshot doing the same transformation using msxml.dll. It is evident that the transformation is done faster, but it is hard to compare directly given the difference in structure of the libraries. In real applications, the easiest approach would be to implement XSL-transformations using both libraries. If major differences emerge in the timing, go for the fastest. Under normal circumstances most developers would choose to stay with the classes in .NET, but the timing makes it possible to make educated decisions. At any rate, the timing can be used in all cases where doubt exist whether one solution is faster than another.

Application Setup

The download is a Visual Studio .NET solution containing two projects, named Timing (the class library) and Time GUI (the sample application). Included is a folder containing a two sample files to use in transformation. The content of the XML file is in Danish, but should work just the same.

Any Limitations or Further Work

It should be noted that error handling in the code could be better, but the most common mistakes should be caught. One useful addition to the sample application would be to display the result of the transformation in a textbox for users to see. The difference in time between transformations in .NET and msxml.dll could be displayed directly. Also, database support could be useful.

Conclusion

I have shown how to use the functions in the native Win32 API when the .NET framework doesn't deliver what we want. We have built a high-resolution timer that can be used to time fragments of code, and we have demonstrated the timer in a sample application used to time XSL transformations.

If you'd like to discuss a broader range of C#Today topics and issues, or if you have any C# related questions you need answers to, please visit our online resource centres at : http://p2p.wrox.com/list.asp?list=csharptoday_discuss&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq and http://p2p.wrox.com/list.asp?list=c_sharp&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq
C#Today - © 2002 WroxPress Limited


Ultimate Guide to .NET Object Cleanup, Part III

Ultimate Guide to .NET Object Cleanup, Part III
http://www.csharptoday.com/content.asp?id=1793
Article Rating: 4.5
Published on January 4, 2002
Abstract
In the final part of this 3-part article, Jan Narkiewicz discusses the Win32 functions that cause a process/thread to exit (self-inflicted termination) or terminated (externally-inflicted termination). These functions include: ExitThread, ExitProcess, TerminateThread, and TerminateProcess. Each of these functions will be reviewed and presented in the context of how they affect .NET’s try/finally construct, using keyword, and an object’s Finalize method.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

This article is the final installment of a three-part series on .NET object cleanup. The first installment (http://www.csharptoday.com/content.asp?id=1765&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq) of this article discussed:

  • try/finally - the try/finally construct was reviewed
  • Finalize method - the Finalize method (called with Garbage Collection cleans up an object) was reviewed.
  • using keyword - C# intrinsic mechanism for disposing of objects by insuring IDisposable's Dispose method is called for said objects.

The second part of this series of articles (http://www.csharptoday.com/content.asp?id=1766&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq) discussed .NET process and thread termination. Specifically it presented the various mechanisms used by .NET in the cleanup of objects. Environment.Exit, Thread.Abort, Forms.Exit, Forms.ExitThread, Process.CloseMainWindow, and Process.Kill were all presented. These mechanisms were evaluated in the context of try/finally, using keyword and Finalize methods, and whether they prevented the aforementioned cleanup mechanisms from executing.

This section of the article discusses the Win32 functions that cause a process/thread to exit (self-inflicted termination) or terminate (externally-inflicted termination). These functions include: ExitThread, ExitProcess, TerminateThread, and TerminateProcess. Each of these functions will be reviewed and presented in the context of how they affect .NET's try/finally construct, using keyword and an object's Finalize method.

All application and class libraries discussed in this section are part of the SWPAllWin32API solution.

Process and Thread Termination within Win32

Before .NET, when Win32 were widespread, there were two mechanisms used by C++ developers to ensure object cleanup took place:

  • __try/__finally - the __try/__finally construct represented an extension to the C/C++ language. The behavior of this constructor directly parallels C#'s try/finally. No matter how the __try block is exited, the __finally block is guaranteed to execute. This approach could not be used in the same method as objects because destructors and __try/__finally are incompatible.
  • Destructor - C++ destructors also guaranteed objects cleaned up. Remember that in C++ the destructor takes place immediately when the object goes out of scope while in C# the destructor is subject to the whims of garbage collection.

Both __try/__finally and the C++ destructor had certain problem areas. These problem areas were functions that caused the __finally block not to execute and the C++ destructor not to be called. These included: ExitThread, ExitProcess, TerminateThread, and TerminateProcess. Given that the .NET Framework runs on top of Win32 (for the present time at least) it is possible that these functions can also thwart the cleanup efforts of the C#'s try/finally, using statement and the Finalize method. The purpose of this section is to determine whether or not these Win32 functions cause the finally block not to be called, the using statement not to dispose of objects, and the Finalize method not to execute.

The namespace that supports calling legacy functions (including Win32 functions) from .NET is System.Runtime.InteropServices. There will be times that C# developers must call Win32 functions. The examples shown here (using the likes of ExitThread, ExitProcess, TerminateThread, and TerminateProcess) are not advocating that C# developers call these functions. Instead these examples serve to demonstrate what happens if an external application or legacy DLL being accessed triggers off these functions.

The following excerpt, the SWPWin32 class from class library SWPWin32API, demonstrates using elements of System.Runtime.InteropServices to access legacy Windows functions:

using System;
using
            System.Runtime.InteropServices;

namespace SWPWin32API
{
public class
            SWPWin32
{
[DllImport("Kernel32.dll")]
public static extern
bool TerminateThread(int handle, int
            exitCode);
}
}

The SWPWin32API class is used to expose each Win32 method used to evaluate exactly what try/finally, the using statement and the Finalize method guarantee with respect to clean up. In the previous code snippet, notice that the TerminateThread method is specified as extern. Crafty C# developers recognize that methods marked as extern are implemented externally. Where this method is implemented should be clear from the attribute specified before TerminateThread is declared: [DllImport("Kernel32.dll")]. Yes, the Kernel32.dll a rather critical part of the Win32 API. This DLL just so happens to contain the TerminateThread function in addition to functions ExitThread, ExitProcess, and TerminateProcess. The prototype specified for the TerminateThread method corresponds to the prototype for the TerminateThread function found in Kernel32.dll (see winbase.h).

Microsoft should be congratulated. The winbase.h header file is found in the Visual Studio .NET era under: \Program Files\Microsoft Visual Studio .NET\Vc7\PlatformSDK\Include. Traditionally Microsoft has mixed the C/C++ headers (language specific headers) with operating system specific headers such as winbase.h. The Vc7\PlatformSDK\Include directory now contains the operating system specific headers, leaving Vc7\Include to contain the C/C++ specific headers. When interoperating with legacy Windows functions, the files found under Vc7\PlatformSDK\Include are an indispensable reference.

The DllImport attribute in the previous code snippet from the SWPWin32 class corresponds to the DllImportAttribute class found in System.Runtime.InteropServices. Specifying this attribute means that the method is actually implemented in the specified, unmanaged DLL (a.k.a. Kernel32.dll). The DllImport attribute also facilitates having the method name differ from the function name, supporting different character sets including Unicode, specifying different calling conventions and interacting with Win32's GetLastError.

In addition to the Win32 functions previously discussed, several others are exposed as part of the SWPWin32 class:

  • GetCurrentThreadId - retrieves the ID of the currently executing thread. This thread ID can then be passed to OpenThread in order to retrieve the thread's process specific handle (a Win32 HANDLE).
  • OpenThread - given a thread ID, returns a process specific handle to the thread. This handle can be passed to functions such as TerminateThread in order to terminate the thread specified by the handle.
  • GetCurrentProcessId - retrieves the ID of the current process. This process ID can then be passed to OpenProcess in order to retrieve a handle (a Win32 HANDLE) to the process specified by the process ID.
  • OpenProcess - given a process ID, returns a handle to the process. This handle can be passed to functions such as TerminateProcess in order to terminate the process specified by the handle. When the process is terminated all threads in said process also terminate.
  • CloseHandle - closes a Win32 HANDLE such as a handle opened by OpenProcess or OpenThread.

Reading the previous description of each function, it should be clear that terminating a thread or process takes a few steps. For example to terminate one process from a different process requires that:

  • The terminating process receives process ID for process to be terminated (i.e. the ID the process to be terminated retrieves using GetCurrentProcessId).
  • Process ID is passed to OpenProcess that in turn returns a handle. Make sure to specify the PROCESS_TERMINATE flag when calling OpenProcess so that the handle returned has terminate-process rights.
  • Call TerminateProcess specifying the handle returned from OpenProcess.
  • Call CloseHandle to cleanup any resources associated with the handle returned by OpenProcess.

The SWPWin32 class exposes a method, SWPTerminateProcess, that terminates a process in the manner just described:

    // Taken from
            winnt.h
public const int PROCESS_TERMINATE = 0x0001;

static public bool SWPTerminateProcess(int
            procID)
{
int h = OpenProcess(PROCESS_TERMINATE,
false,
procID);
if (h == 0)
{
return
            false;
}

try
{
if (!TerminateProcess(h,
            0))
{
return false;
}
}

finally
{
if (h !=
            0)
{
CloseHandle(h);
h = 0;
}
}

return
            true;
}

A similar helper method (SWPTerminateThread) is provided in order to insure that a thread terminates given said thread's ID. The steps required to terminate a thread mirror the steps required to terminate a process. These thread termination steps are as follows:

  • Terminating thread receives thread ID for thread to be terminated (i.e. the ID the thread to be terminated retrieves using GetCurrentThreadId).
  • Thread ID is passed to OpenThread that returns a handle. Make sure to specify the THREAD_TERMINATE flag when calling OpenThread so that the handle returned has terminate-thread rights.
  • Call TerminateThread specifying the handle returned from OpenThread.
  • Call CloseHandle to cleanup any resources associated with the handle returned by OpenThread.

The prototype for the thread terminating function, SWPTerminateThread, is as follows:

static public bool SWPTerminateThread(int threadID)

try/finally and Win32 Process and Thread Termination

The console application SWPFinallySacLamb was developed in order to determine if Win32's TerminateThread and TerminateProcess affect try/finally. SWPFinallySacLamb also serves to demonstrate the use of GetCurrentThreadId and GetCurrentProcessId, and is implemented as follows:

using System;
using System.Windows.Forms;
using SWPWin32API;

namespace SWPFinallySacLamb
{
class SWPFinallyBlock
{
[STAThread]
static void Main(string[] args)
{
try
{
MessageBox.Show("Blocking (ProcessID: " +
SWPWin32.GetCurrentProcessId() +
", ThreadID: " +
SWPWin32.GetCurrentThreadId() +
")");
}

finally
{
Console.WriteLine("In finally");
}
}
}
}

The previous application blocks using a message box (MessageBox.Show). This message box block thus allows time for Win32's TerminateThread and TerminateProcess to be run from an external process. If the text "In finally" is written to the console then the finally block executed in spite of either TerminateThread or TerminateProcess being called. The message box displays its own current thread ID and current process ID because these are used to create the handles used by Win32's TerminateThread and TerminateProcess in order to terminate the thread and the process respectively. It is recognized the Id property of the .NET Process class could have been used to retrieve the ID of the process. There is no corresponding Id property of the .NET Thread class so GetCurrentThreadId and GetCurrentProcessId were used for consistency.

An example of the message box displayed by SWPFinallySacLamb is:

The SWPTermProcess console application is used to terminate external application such as SWPFinallySacLamp. The SWPTermProcess application takes a single command-line argument corresponding to a process ID. The process associated with this ID is then terminated using, SWPWin32.TerminateProcess (the SWPWin32 wrapper for the TerminateProcess function). The SWPTermProcess application is implemented as follows:

using System;
using
            System.Globalization;
using SWPWin32API;

namespace
            SWPTermProcess
{
class
            SWPTerminateProcess
{
[STAThread]
static void Main(string[]
            args)
{
if (args.Length !=
            1)
{
Console.WriteLine("Usage: SWPTermProcess
            ProcessID");
return
            ;
}

SWPWin32.SWPTerminateProcess(Int32.Parse(args[0]));
            }
}
}

The only bit of excitement in the SWPTermProcess application is the use of the Int32 structure's Parse method. The Parse method converts a string to an integer.

When SWPTermProcess is run specifying the process ID of an SWPFinallySacLamb process, the SWPFinallySacLamb process terminates, and the finally clause inside SWPFinallySacLamb is not executed. The proof is that the text "In finally" is never displayed to the console window in which SWPFinallySacLamb was launched.

The SWPTermThread console application is similar to the SWPTermProcess application. While the SWPTermProcess application terminates a process given a process ID, the SWPTermThread application terminates a thread given the thread ID. Suffice it to say that the implementation of SWPTermThread is nearly identical to SWPTermProcess save that SWPTermThread calls SWPWin32.TerminateThread in order to terminate the specified thread. This method in turn calls Win32's TerminateThread function.

When SWPTermThread is run specifying the thread ID of SWPFinallySacLamb 's primary thread, the finally clause inside SWPFinallySacLamb is not executed, but the process does not terminate. Garbage Collection runs in a separate background thread. This thread does not terminate, and hence although the message box is no longer displayed, the application SWPFinallySacLamb keeps running. When the finally block executes, it executes in the same thread as the try block. For this reason, we know the TerminateThread prevents the finally block from executing. Since "In finally" was never displayed and the thread was terminated, the finally block is kaput.

try/finally and Exiting Win32 Processes and Threads

It has already been discussed that the ExitProcess function terminates a Win32 process without calling clean up code (__try/__finally or C++ destructors), and similarly the ExitThread function exits a thread without calling the same manner of clean up code. With Win32 presently being the underpinnings of the .NET Framework, it remains to be explored just how ExitProcess and ExitThread work and play with C#'s try/finally.

The SWPWin32API assembly's SWPWin32 class exposes methods ExitProcess and ExitThread that correspond to their Win32 namesakes. The C# console application SWPExThread contains the following code, which demonstrates in what manner ExitThread interacts with C#'s try/finally. This code consists of a try region that contains a call to ExitThread. Inside the finally block, a Console.WriteLine displays a string "In finally" in the event that the finally region is executed even after Win32's ExitThread is called.

The code associated with the SWPExThread console application is as follows:

class SWPExitThread
{
[STAThread]
static void Main(string[] args)
{
try
{
Console.WriteLine("Before -- SWPWin32.ExitThread()");
// Exit thread with exit code: 0
SWPWin32.ExitThread(0);
// This will never get executed
Console.WriteLine("After -- SWPWin32.ExitThread()");
}

finally
{
Console.WriteLine("In finally");
}
}
}

As it turns out, when the SWPExThread application is run, the "In finally" string is never displayed. This means that the thread terminated without calling the code in the finally region. In spite of calling ExitThread, the process (as shown in the previous code snippet) does not terminate. Remember again, the Garbage Collection for a console application runs in a separate thread. The ExitThread function only terminated what can be thought of as the process's primary thread. All other threads are still left running.

The SWPExProcess console application looks like SWPExThread. The goal of the SWPExProcess application is to determine if the finally block will be called in spite of executing ExitProcess. The code associated with SWPExProcess is as follows:

class SWPExitProcess
{
[STAThread]
static void Main(string[] args)
{
try
{
Console.WriteLine("Before -- SWPWin32.ExitProcess()");
// Exit process with exit code: 0
SWPWin32.ExitProcess(0);
// This will never get executed
Console.WriteLine("After -- SWPWin32.ExitProcess()");
}

finally
{
Console.WriteLine("In finally");
}
}
}

The ExitProcess function causes the code in the finally block not to execute.

Finalize Methods and Win32 Process and Thread Termination

A Finalize method was added to the SWPFinallySacLamb console application in order to:

  • Demonstrate whether or not object finalization still takes place if a process is terminated using TerminateProcess.
  • Demonstrate whether or not object finalization still takes place if a thread is terminated using TerminateThread.

The freshly gussied up version of the SWPFinallySacLamb console application is as follows:

    class
            SWPFinallyBlock
{
~SWPFinallyBlock()
{
Console.WriteLine("In
            finalize for SWPFinallyBlock");
}

[STAThread]
static
            void Main(string[] args)
{
// This object will be
            finalized
SWPFinallyBlock finallyBlock = new
            SWPFinallyBlock();
try
{
MessageBox.Show("Blocking
            (ProcessID: " +
SWPWin32.GetCurrentProcessId() +
", ThreadID:
            " +
SWPWin32.GetCurrentThreadId() +
")");
}

finally
{
Console.WriteLine("In
            finally");
}
}
}

The question remains, does the WriteLine inside the Finalize method of the SWPFinallyBlock class get called if:

  • The SWPTermProcess console application calls Win32's TerminateProcess and terminates the SWPFinallySacLamb process?
  • The SWPTermThread console application calls Win32's TerminateThread and terminates the primary thread of the SWPFinallySacLamb process?

The results as to whether or not the Finalize method is called, hence displaying "In finalize for SWPFinallyBlock" is as follows:

  • SWPTermProcess - the SWPFinallySacLamb process is terminated and the object is not finalized because the string inside the Finalize method was not displayed.
  • SWPTermThread - the SWPFinallySacLamb process's primary thread is terminated and the object is not finalized because the string inside the Finalize method was not displayed. The process itself is not terminated because at least one more thread remains running in the process (likely the thread associated with Garbage Collection).

The case of the process's primary thread being terminated deserves a bit more discussion. If the process whose primary thread has been terminated is able to terminate using a finalize-friendly mechanism such as Environment.Exit, the objects will be finalized. In the case of SWPFinallySacLamb, there is no mechanism built in that provides a way for the process to exit.

There is a bit of a conundrum in the Finalize method (~SWPFinally) of the SWPFinally class that does merit discussion. When this Finalize method is called, it presumes that the Console.Out property of the Console class has not been cleaned up. If the Out property has not been cleaned up by Garbage Collection when the application exits, it is still permissible to call Console.Out.Writeline. Garbage Collection does not guarantee that at "application finalization" any object will still exist. It was a bit presumptuous (dangerous) of the code to assume that any method/property of the Console class would available because the Console instance may have been finalized.

During Beta 1, attempting to call Console.Out or MessageBox.Show resulted in an access violation. This is because the underling object associated with Console.Out or MessageBox.Show had been cleaned up already if the Finalize method was being triggered as part of final application cleanup. As of Beta 2 and onwards such common place output mechanisms are still active when called during an object's finalization at application exit. For sample application such as those presented this behavior can be relied upon. In a more complicated application, such assumptions should not be made.

Finalize Methods and Exiting Win32 Processes and Threads

In order to test whether an object will be finalized in the event that ExitProcess was called, the SWPExProcess application was tweaked as follows:

  • A Finalize method was added to the SWPExitProcess class and inside this method WriteLine the string "In Finalize of SWPExitProcess".
  • An instance of the SWPExitProcess class was created in the Main method. This instance should be finalized at application exit.

Recall that SWPExProcess calls ExitThread inside of a try block. The idea is to see if the code in the finally region will be executed in spite of ExitThread (finally was not called). Given the previous changes just made to SWPExProcess, it remains to be seen if the object of type SWPExitProcess class will be finalized at application exit. As it turns out the string "In Finalize of SWPExitProcess", is never displayed, and hence the object is not finalized. Calling Win32's ExitProcess function clearly prevents object finalization.

As SWPExProcess is used to test ExitProcess, SWPExThread is used to test ExitThread. For this reason, the following changes were made to the SWPExThread application:

  • A Finalize method was added to the SWPExitThread class and inside this method WriteLine the string "In Finalize of SWPExitThread".
  • An instance of the SWPExitThread class was created in the Main method. This instance should be finalized at application exit.

When ExitThread is called, the SWPExitThread class's Finalize method is not called. This is because the primary thread terminates. The application does not exit because of the thread handling Garbage Collection. As was demonstrated in a variety of scenarios, the object could be finalized if the SWPExThread application had a means by which to exit.

Using Statement and Exiting Win32 Processes and Threads

By this time, code such as the following section from SWPExThreadWithUsing should appear quite commonplace:

    [STAThread]
static
            void Main(string[] args)
{
SWPExposeDispose exposeDisplose =
            null;

using (exposeDisplose = new
            SWPExposeDispose())
{
Console.WriteLine("Before --
            SWPWin32.ExitThread()");
// Exit thread with exit code:
            0
SWPWin32.ExitThread(0);
// This will never get
            executed
Console.WriteLine("After --
            SWPWin32.ExitThread()");
}
}

The previous code leverages the SWPExposeDispose class from the SWPFinalizeImpl class library. As its name says, the SWPExposeDispose class exposes the IDisposable interface, complete with the Dispose method. The SWPWin32API assembly's SWPWin32 class is used to provide an implementation of Win32's ExitThread via the ExitThread method of the SWPWin32 API class. When the SWPExThreadWithUsing class is run, the LastGasp.log file is not created, and hence the Dispose method of the SWPExposeDispose instance is not called, but at the same time the application does not exit. To quote a previous section of an article, "This means that the thread terminated without calling the code in the finally region. In spite of calling ExitThread, the process (as shown in the previous code snippet) does not terminate. Remember again the Garbage Collection for a console application runs in a separate thread. The ExitThread function only terminated what can be thought of as the process's primary thread. All other threads are still left running."

The SWPExThreadWithUsing console application is the near-clone of SWPExProcessWithUsing. For the later console application, ExitProcess is called within the using block rather than ExitThread as in the former. When SWPExProcessWithUsing is run, the instance of the SWPExposeDispose class is not disposed of (Dispose is not called) and hence Win32's ExitProcess prevents using from disposing of an object. The source code for SWPExProcessWithUsing can be found in the downloadable code associated with the article.

The SWPUsingSacLamp console application (see below) facilitates examining how the using statement works, and plays with the likes of Win32's TerminateThread and TerminateProcess:

class SWPUsingBlock
{
[STAThread]
static void Main(string[] args)
{
SWPExposeDispose exposeDisplose = null;

using (exposeDisplose = new SWPExposeDispose())
{
MessageBox.Show("Blocking (ProcessID: " +
SWPWin32.GetCurrentProcessId() +
", ThreadID: " +
SWPWin32.GetCurrentThreadId() +
")");
}
}
}

The SWPUsingSacLamb console application (see above) blocks until it is terminated (like TerminateThread and TerminateProcess). Recall that the aforementioned Win32 termination functions are called externally as follows:

  • TerminateProcess - the SWPTermProcess console application can be used to terminate SWPUsingSacLamb. When this event occurs, the using statement in the SWPUsingSacLamb application does not call Dispose, and hence Win32's TerminateProcess prevents using from performing object disposal.
  • TerminateThread - the SWPTermThread console application can be used to terminate SWPUsingSacLamb. When this event occurs, the using statement in the SWPUsingSacLamb application does not call Dispose, but there is a caveat to this behavior.

The behavior of using and TerminateThread is the same as was shown for finally with ExitThread, using with ExitThread, and finally with TerminateThread. What is this behavior? Basically the SWPUsingSacLamb application does not exit because the Garbage Collection thread still lingers in the background, and hence officially using has not failed to Dispose. Still, using is a per-thread task, and even though the application fails to exit, Dispose will never be called for the instance of the SWPExposeDispose class associated with the using statement.

Conclusions

With respect to the behavior of the C# cleanup mechanism and Win32 thread/process clean up functions we have looked at, here is a summary:

DLL | Function | finally | Finalize method | Using statement
kernel32.dll | ExitProcess | Not Executed | Not Executed | Not Executed
kernel32.dll | ExitThread | Not Executed | Not Executed | Not Executed
kernel32.dll | TerminateProcess | Not Executed | Not Executed | Not Executed
kernel32.dll | TerminateThread | Not Executed | Not Executed | Not Executed

Guarantees are few and far between when a .NET application encounters the likes of ExitThread, ExitProcess, TerminateThread, and TerminateProcess. Potentially, it is possible for finalization to take place in spite of ExitThread or TerminateThread. Regardless, the Win32 cleanup functions ExitThread, ExitProcess, TerminateThread, and TerminateProcess should be avoided in order to ensure that finally regions are executed and object finalization takes place. The bigger problem with these functions is not when your application calls them, but when a third-party calls them for you.

Recall from the first part of this article, the following table demonstrating how try/finally, Finalize methods, and the using statement affect .NET process and thread termination mechanisms:

Namespace | Class | Method | finally | Finalize Method | using statement
System | Environment | Exit | Not Executed | Executed | Not Executed
System.Threading | Thread | Abort | Executed | Not Executed | Executed
System.Windows.Forms | Application | Exit | Executed | Executed | Executed
System.Windows.Forms | Application | ExitThread | Executed | Executed | Executed
System.Diagnostics | Process | CloseMainWindow | Executed | Executed | Executed
System.Diagnostics | Process | Kill | Not Executed | Not Executed | Not Executed
n/a | n/a | Unhandled exception | Executed | Not Executed | Executed

During this article several references were made to the idea that Garbage Collection is run in a separate thread. In order to understand how/when garbage collection is run in a separate thread take a look at "Professional .NET Framework" by Jan D. Narkiewicz, et al (Wrox Press, ISBN: 1861005563). A full description of this text can be found at: http://www.wrox.com/Books/Book_Details.asp?sub_section=1&isbn=1861005563&subject=%2ENET&subject_id=63&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq

Chapter 5 of this book discusses the difference between .NET runtime hosts. ASP.NET applications for example are optimized for running on servers, and hence do not run Garbage Collection in a separate thread. Console applications and Windows Forms applications are optimized for interacting with humans, and hence Garbage Collection is run in a separate thread. In technical terms, ASP.NET applications run in a "server" runtime host while console applications and Windows Forms applications run in a "workspace" runtime host. Specifying the runtime host in which an application runs (server versus workstation) dictates the behavior of Garbage Collection.

Given that Garbage Collection can run for certain types of applications in the same thread, a few more experiments could be performed with ExitThread and TerminateThread (will these Win32 functions cause the application to exit and in event of such an exit will the objects be finalized?). These experiments are an exercise left to the reader. The whole premise of this article is to demonstrate the use of such experiments in order to show just how reliable object clean up is. Sound .NET development reflects an understanding of the limitations of the .NET Framework, and the limitations of .NET when used in conjunction with its system underpinnings - the Win32 API.

This rather detailed three-part article has a flaw - although a slew of .NET and Win32 process and thread clean methods were discussed, there are more scenarios to be explored. Adventurous readers (or maybe another article at a later date) should explore how the following termination mechanisms affect try/finalize, using statement or the Finalize method:

  • Service Shutdown for Windows Services
  • CTRL-C for Console Applications
  • User logging off for applications running in the interactive desktop
  • System shutdown
  • Process termination from Task Manager

    If you'd like to discuss a broader range of C#Today topics and issues, or if you have any C# related questions you need answers to, please visit our online resource centres at : http://p2p.wrox.com/list.asp?list=csharptoday_discuss&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq and http://p2p.wrox.com/list.asp?list=c_sharp&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq
C#Today - © 2002 WroxPress Limited


Ultimate Guide to .NET Object Cleanup, Part II

Ultimate Guide to .NET Object Cleanup, Part II
http://www.csharptoday.com/content.asp?id=1792
Article Rating: 5
Published on January 3, 2002
Abstract
In part 2 of this 3-part article, Jan Narkiewicz evaluates the effectiveness of the C# object cleanup techniques used in part 1 (try/finally, the using statement, and the Finalize methods) when they are used in conjunction with .NET thread and process cleanup methods. A series of sample C# applications are developed throughout this article in order to demonstrate each approach to object cleanup, and how it is affected by .NET thread and process termination.
Article

This is the second part of a three-part series that discusses .NET object cleanup. In the first installment in this series (http://www.csharptoday.com/content.asp?id=1765&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq), an overview was presented of the techniques employed by C# in order to ensure objects cleanup: try/finally, the using statement, and Finalize methods. Part one in the series also reviewed the methods .NET uses to terminate threads and processes. The Win32 functions that exit and terminate threads/processes were also presented.

This second part evaluates the effectiveness of the previous mentioned C# object cleanup techniques when they are used in conjunction with .NET thread and process clean up methods. A series of sample C# applications are developed throughout out this article in order to demonstrate each approach to object cleanup and how it is affected by .NET thread and process termination. The basic idea is to determine whether the likes of try/finally, the using statement, and Finalize methods fail to clean up properly when used in conjunction with the following .NET methods that terminate a process: Environment.Exit, Application.Exit, Process.Kill, and Process.CloseMainWindow. Similarly the success or failure of each approach to C# object cleanup is evaluated in conjunction with the following .NET methods that terminate a thread: Thread.Abort, and Application.ExitThread.

Without further ado, lets evaluate C# object cleanup in conjunction with .NET process and thread termination methods.

Process and Thread Termination within .NET and try/finally

Determining if the finally region is executed is a simple process. For example, the following snippet from the SWPEnvExit console application will display "In finally" if the code in the finally region is executed:

    try
{
Environment.Exit(0);
}

finally
{
Console.WriteLine("In
            finally");
}

The reason the finally region in the previous code snippet may not be called is because the Environment.Exit method was called in the try region. The text "In finally" is not displayed which means Environment.Exit causes the code in the finally region not to execute. The documentation for this method can be found at: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/cpref/html/frlrfsystemenvironmentclassexittopic.asp&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq

This documentation does not mention the important caveat, namely that Environment.Exit causes the code in the finally region not to execute.

Thread Class and Finally

The Thread.Abort method causes a thread to terminate due to the raising of a special exception, ThreadAbortException. The ThreadAbortException exception is special because even if it is caught (using catch) it is raised again after the catch region is executed. To quote MSDN, "When this exception is raised, the runtime executes all the finally blocks before killing the thread."

The SWPThAbort console application is provided to verify the veracity of the previous quote from MSDN.

The SWPThAbort application does call its finally region even though Thread.Abort is called. This makes sense given that Thread.Abort raises an exception. After all, the finally region was designed to execute when an exception was thrown.

Application Class and Finally

The Application class of the System.Windows.Forms namespace exposes the following methods that terminate a process:

  • Exit: used with Windows Forms applications, this method causes said applications to exit because the Application class's Run method returns. The SWPAppExit Windows Forms application demonstrates the Exit method being called.
  • ExitThread - used with Windows Forms applications, this method causes the calling thread to exit and ceases displaying all windows associated with said thread. The SWPAppExit Windows Forms application also demonstrates the ExitThread method being called.

In order to understand how the Application class's Exit method works, it is important to recognize the significance of the Application class's Run method. Within the SWPAppExit Windows Forms application, the Run method is used as follows:

static void Main()
{
Application.Run(new SWPApplicationExit());
}

The Windows Form associated with the SWPApplicationExit class (a class derived from the Form class) is displayed when it is instantiated and passed as parameter to the Run method. The responsibilities of the Run method include displaying the Form and running a message loop that will be used to handle the messages sent to the Form. The Exit method causes the windows displayed by the Form to exit and causes Run to return. For most Windows Forms applications causing Run to return causes the application to exit.

The code inside the SWPApplication that demonstrates Application.Exit is found in the buttonAppExitFinally_Click method. The aforementioned method is called when a button labeled "App. Exit Finally" is pressed. The code associated with this method is as follows:

private void buttonAppExitFinally_Click(object sender,
System.EventArgs e)
{
try
{
Application.Exit();
}
finally
{
Trace.WriteLine("In finally");
}
}

As it turns out, when the Application.Exit method is called in the previous code snippet, the code in the finally region is executed.

A button labeled "App. ExitThread Finally" is provided within the SWPApplication in order to demonstrate the Application.ExitThread method in action. This code is as follows:

private void buttonAppExitThreadFinally_Click(object sender,
System.EventArgs e)
{
try
{
Application.ExitThread();
}
finally
{
Trace.WriteLine("In finally");
}
}

It should be recognized that the code in the finally block is executed after the button AppExitThreadFinally button is clicked. This observation means that Application.ExitThread does not impede the execution of a finally region.

Process Class and Finally

The Process class of the System.Diagnostics namespace exposes the following methods that terminate a process:

  • CloseMainWindow: closes a user interface application thus causing the application (the process) to exit. The SWPProcCloseMainWindow Windows forms application demonstrates the CloseMainWindow method being called.
  • Kill - stops a process immediately. The SWPProcKill console application demonstrates the Kill method being called.

Both the Kill and CloseMainWindows methods are not static and hence an instance of the Process class is needed in order to call these methods. The Process class's GetCurrentProcess method is used to retrieve the Process instance associated with the process in which this method is called. An example of the GetCurrentProcess method in action is as follows:

Process currentProc = Process.GetCurrentProcess();

The code that calls CloseMainWindow is associated with the click event received by a button (the button labeled "Proc. CloseMainWin Finally") in the SWPProcCloseMainWindow application. The method for handling a click of the "Proc. CloseMainWin Finally" button is as follows:

private void buttonProcCloseMainWinFinally_Click(object sender,
System.EventArgs e)
{
try
{
Process currentProcess = Process.GetCurrentProcess();

currentProcess.CloseMainWindow();
}

finally
{
Trace.WriteLine("In finally");
}
}

The "In finally" text is displayed in Visual Studio .NET's output window after the CloseMainWindow method is called. This means that CloseMainWindow does not prevent the code in the finally region from executing. Actually, the finally region was originally written as follows:

finally
{
MessageBox.Show("In finally");
}

The message box in the previous finally region is not displayed. This has to do with how Windows messages are handled. When the CloseMainWindow method is called, a message to terminate the SWPProcCloseMainWindows application is queued. Before a message box can be displayed, a variety of messages are queued. Since the close application message was queued first (a queue is a First-In-First-Out data structure), this message is handled before any messages are handled associated with displaying the message box.

The SWPProcKill console application is extremely simple. The code associated with this application can be found with the code downloaded with this article. Just like every other try/finally experiment the finally block inside SWPProcKill attempts to display the string "In finally" using a Console.WriteLine in the finally block. As it turns out, the "In finally" text is not displayed by the SWPProcKill application, and hence the Process.Kill method does prevent the finally block from being executed. This behavior is not surprising because MSDN alludes to this immediacy with respect to process exit. To quote MSDN's documentation on the Kill method, "Immediately stops the associated process."

Process and Thread Termination within .NET and Finalize

Verifying if a class's Finalize method is called can be done quite simply. Consider the case of determining if the Finalize method is called in spite of calling Environment.Exit. Previously the SWPFinalize class was introduced. This class contained a Finalize method that wrote to a file. In order to make a variation of this class available to a wide variety of applications, it has been placed in the SWPFinalizeImpl assembly. The new version of this class is called SWPFinalizeV2 and it is implemented as follows:

using System;
using
            System.IO;

namespace SWPFinalizeImpl
{
public class
            SWPFinalizeV2
{
~SWPFinalizeV2()
{
FileStream file =
new FileStream("LastGasp.log",
FileMode.CreateNew,
FileAccess.Write,
FileShare.None);
StreamWriter
            fileWriter = new StreamWriter(file);

fileWriter.WriteLine("In
            finalize");
fileWriter.Close();
file.Close();
}
}
}

The basic premise of the previous code snippet is to perform the following tasks in the class's Finalize method:

  • Create a file named LastGasp.log
  • Write the string "In finalize" to the file
  • Close the file.

The class SWPFinalizeV2 can be used in conjunction with the SWPEnvExit console application. Recall that this was used to demonstrate whether or not Environment.Exit interfered with the calling of a finally block. The tweaked version of the SWPEnvExit application is as follows:

using System;
using
            SWPFinalizeImpl;

namespace SWPEnvExit
{
class
            SWPEnvironmentExit
{
[STAThread]
static void Main(string[]
            args)
{
SWPFinalizeV2 goOutOfScope = new
            SWPFinalizeV2();

try
{
Environment.Exit(0);
}

finally
{
Console.WriteLine("In
            finally");
}
}
}
}

Notice that the previous code specifies using SWPFianlizeImpl in order to simplify using the SWPFinalizeV2 class. A reference was created between the SWPFinalizeImpl class library and SWPEnvExit application. The previous code snippet does create and instance of the SWPFinalizeV2 class:

SWPFinalizeV2 goOutOfScope = new SWPFinalizeV2();

The purpose of the goOutOfScope variable is to go out of scope and hence demonstrate whether or not Environment.Exit interferes with object finalization. After Environment.Exit is called, the news in this regard is good. The LastGasp.log file is created containing the text "In finalize." This means that Environment.Exit does not interfere with object finalization. This news is all the more significant because Environment.Exit did interfere with executing the code in a finally block.

What remains is to determine how the other .NET process and thread termination methods behave with respect to object finalization. The premise used to determine this will be to take the previous discussed test applications (SWPThAbort, SWPAppExit, etc.) and add code that creates an instance of an SWPFinalizeV2 object. The presence of the LastGasp.log file will determine if given method interferes with object finalization. Rather than cause excessive ennui for the reader by demonstrating these rather trivial code modifications, a summary of the results will be presented. The summary as to how these other thread/process termination methods behave with regards to calling the Finalize method is as follows:

  • Application.Exit - the Finalize method for the SWPFinalizeV2 instance is executed.
  • Application.ExitThread - the Finalize method for the SWPFinalizeV2 instance is executed.
  • Process.Kill - the Finalize method for the SWPFinalizeV2 instance is not executed.
  • Process.CloseMainWindow - the Finalize method for the SWPFinalizeV2 instance is executed.
  • Thread.Abort - the Finalize method for the SWPFinalizeV2 instance is not executed.
  • Environment.Exit - as was demonstrated using source code, the Finalize method for the SWPFinalizeV2 instance is executed.

The case of Thread.Abort deserves a bit more consideration. Basically an unhandled exception (ThreadAbortException) results in object finalization not taking place. Does the same behavior hold true for all exceptions or only exceptions of type ThreadAbortException ? The SWPUnhandledException console application basically throws an unhandled exception of type Exception. Before this exception is raised, an instance of SWPFinalizeV2 is created. The rationale for creating this instance is to determine if its Finalize method will be called. The code associated with the SWPUnhandledException is as follows:

using System;
using
            SWPFinalizeImpl;

namespace
            SWPUnhandledException
{
class
            SWPUnhandledException
{
[STAThread]
static void
            Main(string[] args)
{
SWPFinalizeV2 goOutOfScope = new
            SWPFinalizeV2();

throw new Exception("Throw an unhandled
            exception");
}
}
}

After the SWPUnhandledException completes, there is no LastGasp.log file created. Clearly, the Finalize method of the SWPFinalizeV2 instance, goOutOfScope, is not called.

The SWPUnhandledExceptionWithUsing evaluates the behavior of the using statement in conjunction with an unhandled exception. The code is fundamentally the same as SWPUnhandledException so minutae-oriented readers should download this application with the source code for this article. What the SWPUnhandledExceptionWithUsing application demonstrates, because the LastGasp.log file is created, is that the using statement works even if an unhandled exception is generated.

Process and Thread Termination within .NET and the Using Statement

Before the relative merits of the using statement can be ascertained, a class is needed that implements the IDisposable interface and hence exposes the Dispose method. For this reason, the SWPExposeDispose class was added to the SWPFinalizeImpl assembly. The SWPExposeDispose class implements IDisposable and hence exposes the Displose method. This class is implemented as follows:

public class SWPExposeDispose : IDisposable
{
private bool bIsDisposed = false;

public void Dispose()
{
if (!bIsDisposed)
{
bIsDisposed = true;
Garbage Collection.SuppressFinalize(this);
FileStream file =
new FileStream("LastGasp.log",
FileMode.CreateNew,
FileAccess.Write,
FileShare.None);
StreamWriter fileWriter = new StreamWriter(file);

fileWriter.WriteLine("In Dispose");
fileWriter.Close();
file.Close();
}
}

~SWPExposeDispose()
{
Dispose();
}
}

The implementation of the SWPExposeDispose class mirrors that of our Finalize demonstration class, SWPFinalizeV2. For the SWPExposeDispose class, the "extra cleanup step" (writing "In Dispose" to a file) is performed in the Dispose method. A Boolean data member, bIsDisposed, is used to ensure that the Dispose method performs its "extra cleanup step" only once (only act if bIsDisposed if false). The Finalize method (~SWPExposeDispose) of the SWPExposeDispose class does call the Dispose method.

The Dispose method does call the Garbage Collection class's SupressFinalize method. The idea of this method is to suppress object finalization. This means that if the object is disposed of (the Dispose method is called), the object is removed from Garbage Collection's finalize queue. The finalize queue contains all objects that must be finalized when Garbage Collection cleans up an object. After Dispose is called, an instance of SWPExposeDispose will not be finalized by garbage collection due to the calling of SupressFinalize.

In order to demonstrate how SWPExposeDispose works, consider the following methods from the SWPApplication windows forms application:

private void buttonAppExitThreadUsing_Click(object sender,
System.EventArgs e)
{
SWPExposeDispose exposeDisplose = null;

using (exposeDisplose = new SWPExposeDispose())
{
Application.Exit();
}
}

private void buttonAppExitUsing_Click(object sender, System.EventArgs e)
{
SWPExposeDispose exposeDisplose = null;

using (exposeDisplose = new SWPExposeDispose())
{
Application.ExitThread();
}
}

The previous methods are called when the button labeled "App.Exit Using" and the button labeled, "App.ExitThread Using" are clicked respectively. The buttonAppExitUsing_Click method contains:

  • A using statement in which an instance of an SWPExposeDispose is created

using (exposeDisplose = new SWPExposeDispose())

  • A call to Appplication.Exit inside the brackets dictating the scope of the using statement.

So if the Finalize method is called even though Application.Exit was called inside the brackets of the using region we will be happy. The Application.Exit method does not prevent the constructs associated with the using statement from performing object cleanup by calling the Dispose method of the instance of a SWPExposeDispose specified within the using statement's brackets.

The buttonAppExitThreadUsing_Click method was also demonstrated in the previous code snippet. The instance of the SWPExposeDispose specified within the using statement's brackets was cleaned up in spite of Appplication.Exit being called inside the using region.

A couple of lines of code are required in order to perform each experiment pertaining to evaluating if the using statement successfully calls Dispose in spite of a .NET thread/process termination method. Rather than present this rather uninteresting exercise in software development, the following recap is provided:

  • Environment.Exit - the SWPApplication Windows Forms application demonstrates that Dispose is called in spite of the Environment.Exit method being called inside the brackets associated with a using statement.
  • Process.Kill - the SWPProcKillWithUsing console application demonstrates that Dispose is NOT called in spite of the Process.Kill method being called inside the brackets associated with a using statement.
  • Process.CloseMainWindow - the SWPProcCloseMainWindow console application demonstrates that Dispose is called in spite of the Process. CloseMainWindow method being called inside the brackets associated with a using statement.
  • Thread.Abort - the SWPThAbortWithUsing console application demonstrates that Dispose is called in spite of the Thread.Abort method being called inside the brackets associated with a using statement.
  • Application.Exit - the SWPApplication Windows Forms application demonstrates that Dispose is called in spite of the Application.Exit method being called inside the brackets associated with a using statement.
  • Application.ExitThread - the SWPApplication Windows Forms application demonstrates that Dispose is called in spite of the Application.ExitThread method being called inside the brackets associated with a using statement.

Conclusions

A table summarizing the results of the experiments is follows:

Namespace | Class | Method | finally | Finalize method | using statement
System | Environment | Exit | Not Executed | Executed | Not Executed
System.Threading | Thread | Abort | Executed | Not Executed | Executed
System.Windows.Forms | Application | Exit | Executed | Executed | Executed
System.Windows.Forms | Application | ExitThread | Executed | Executed | Executed
System.Diagnostics | Process | CloseMainWindow | Executed | Executed | Executed
System.Diagnostics | Process | Kill | Not Executed | Not Executed | Not Executed
n/a | n/a | Unhandled exception | Executed | Not Executed | Executed

The previous table demonstrates there are some coding situations that should be avoided. These situations to avoid include:

  • Do not allow unhandled exceptions of any kind. Unhandled exceptions circumvent the cleanup of objects by Garbage Collection. Remember that Thread.Abort is equivalent to an unhandled exception.
  • Do not call Thread.Abort unless you have a good reason (see previous bullet)
  • Avoid calling Process.Kill. This method kills mercilessly, where mercilessly means that no finalize regions are executed and no Finalize methods are executed.

If the previous situations are avoided, the Garbage Collection will always finalize objects. Yes, play by some rules and be guaranteed that every object will be cleaned up. The basic rule is, that if an object can be finalized it will likely be cleaned up.

The guarantees of calling finally and executing the Finalize method reviewed thus far have been all .NET related. Not explored (at least not yet) is how object cleanup fairs in the face of Win32's ExitThread, ExitProcess, TerminateThread, and TerminateProcess. The behavior of try/finally and finalization will be presented in the final part of this article.

C#Today - © 2002 WroxPress Limited


Ultimate Guide to .NET Object Cleanup, Part I

Ultimate Guide to .NET Object Cleanup, Part I
http://www.csharptoday.com/content.asp?id=1791
Article Rating: 4.5
Published on January 2, 2002
Abstract
In part 1 of this 3-part article, Jan Narkiewicz looks at .NET object cleanup. He presents an overview of the techniques employed by C# in order to ensure object cleanup: try/finally, the using statement, and Finalize methods. In this article he also reviews the methods .NET uses to terminate threads and processes. The Win32 functions that exit and terminate threads/processes are also examined.
Article

In the renowned art of software engineering there is often a need to guarantee a particular task is performed. For example, if a special file is being generated, the final step might be to write a CRC to the file's header. If this step does not take place, then the generated file is considered to be invalid (no CRC, no valid file). The constructs provided by the Common Language Runtime (CLR) that can be used to guarantee a task is executed include:

  • try/finally - the finally block of the try/finally construct provides a region of code that is guaranteed to be executed.
  • Finalize method (a destructor) - when Garbage Collection (GARBAGE COLLECTION) cleans up an object, it calls the Finalize method for the object (unless calling the Finalize method has explicitly been suppressed).
  • using statement - objects that implement the IDisposable interface from the System namespace can be automatically disposed with the help of the using statement. The basic idea is to specify an object or objects in conjunction with the using statement, and when the using statement goes out of scope, the objects are disposed of (their Dispose method is executed).

Throughout this article, the term "Finalize method" will be used rather than destructor. In C++, destructors are executed immediately when an object goes out of scope. In C#, a destructor (Finalize method) is called once Garbage Collection gets around to cleaning up an object. Because of this significant difference in behavior, the term "Finalize method" is preferred.

C# is capable of exploiting the using statement, try/finally and/or a Finalize method to ensure a task or tasks takes place, but what is this guarantee worth? Under what condition does the finally region fail to execute, or might Garbage Collection fail to finalize an object (a.k.a. when doesn't Garbage Collection call an object's Finalize method)?

This article evaluates the different mechanisms that can be used to terminate a thread and a process within .NET and within Windows. The effect each such mechanism has on try/finally and the Finalize method will be presented. Amongst this arsenal of thread and process termination methods are the .NET Framework's:

  • Namespace: System, Class: Environment, Method: Exit - terminates the process in which this method is called.
  • Namespace: System.Threading, Class: Thread, Method: Abort - triggers a special exception that potentially leads to a thread's termination.
  • Namespace: System.Windows.Forms, Class: Application, Method: Exit - causes a Windows Forms application (process) to exit once all queued messages have been processed. To be specific, the Application.Run method returns, and for most Windows Forms applications this causes the application to exit (a.k.a. the process to terminate).
  • Namespace: System.Windows.Forms, Class: Application, Method: ExitThread - closes the thread calling this method and any windows associated with said thread.
  • Namespace: System.Diagnostics, Class: Process, Method: Kill - kills the process specified by the Process instance.
  • Namespace: System.Diagnostics, Class: Process, Method: CloseMainWindow - closes a process running a user interface by sending a close message to the process's main window.

The aforementioned methods are not the only culprits that can potentially interfere with try/finally and Finalize methods. Win32 functions can potentially cause try/finally and the Finalize method to misbehave. The Win32 functions that may cause finally blocks or the using statement's object disposal or Finalize methods not to execute include:

  • ExitThread - causes the calling thread to exit immediately.
  • ExitProcess - causes the process in which this function is called to exit immediately.
  • TerminateThread - causes the specified thread to terminate where the specified thread is passed using a Windows handle (type, HANDLE).
  • TerminateProcess - causes the specified process to terminate where the specified process is passed using a Windows handle (type, HANDLE).

As you may have guessed, this article presents an overview of try/finally and the Finalize method. In order to fully understand the Finalize method, certain tenets of Garbage Collection must also be discussed. Both the .NET and Win32 mechanisms for cleaning up threads and processes will also be presented. In order to access Win32 functions, elements of the System.Runtime.InteropServices namespace must be utilized. This interaction with the System.Runtime.InteropServices namespace includes the use of attributes. Each cleanup scenario discussed is presented using a series of examples that demonstrate the limitations of each approach (try/finally or Finalize method) to object cleanup.

The basic idea of this article is to fill in the two right hand columns of the following tables. The first table (which follows) contains a "finally" and a "Finalize method" column. These columns will contain the results as to whether or not a particular .NET method interferes with the code in the finally block, the object disposal associated with a using statement, or with Garbage Collection calling an object's Finalize method:

Namespace | Class | Method | finally | Finalize method | Using statement
System | Environment | Exit | Yes or No | Yes or No | Yes or No
System.Threading | Thread | Abort | Yes or No | Yes or No | Yes or No
System.Windows.Forms | Application | Exit | Yes or No | Yes or No | Yes or No
System.Windows.Forms | Application | ExitThread | Yes or No | Yes or No | Yes or No
System.Diagnostics | Process | CloseMainWindow | Yes or No | Yes or No | Yes or No
System.Diagnostics | Process | Kill | Yes or No | Yes or No | Yes or No

The completed version of the previous table is contained in the conclusion of part two of this article.

The second table (which follows) contains a "finally" and a "Finalize method" column. These columns will contain the results as to whether or not a particular Win32 function interferes with the code in the finally block, or with Garbage Collection calling an object's Finalize method:

DLL | Function | finally | Finalize method | using statement
kernel32.dll | ExitProcess | Yes or No | Yes or No | Yes or No
kernel32.dll | ExitThread | Yes or No | Yes or No | Yes or No
kernel32.dll | TerminateProcess | Yes or No | Yes or No | Yes or No
kernel32.dll | TerminateThread | Yes or No | Yes or No | Yes or No

The completed version of the previous table is contained in the conclusion of part three of this article.

The code in this article is developed in C# using Visual Studio .NET, Enterprise Edition, Release Candidate 3. The basic organization of this multi-part article is:

  • Part I (current part of article) - presents a comprehensive overview of .NET object cleanup using C#. This includes a review of try/finally, the using statement and object finalization.
  • Part II - presents an overview of .NET process and thread termination mechanisms. This includes a discussion of how such termination mechanisms affect each form of .NET object cleanup.
  • Part III - presents an overview of Win32 process and thread termination mechanisms. This overview includes a discussion of how such termination mechanisms affect each form of .NET object cleanup.

It should be noted that the concepts addressed in this article are presented as part of University of California at Berkeley Extension's "C# Programming II" and "VB.NET Programming II" courses. This material is used with permission of Software Pronto Inc., which is responsible for teaching the aforementioned courses. Information regarding these courses can be found at:

  • C# Programming II: http://www.unex.berkeley.edu/cat/323543.html?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq
  • VB.NET Programming II: http://www.unex.berkeley.edu/cat/323600.html?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq

Review of .NET Cleanup

Readers familiar with the nuances of Garbage Collection, object finalization, and try/finally can skip this section and its subsections.

The Finally block of a try/finally construct and a Finalize method both ensure that designated code (possibly clean up code) is executed. According to MSDN, the using statement performs object clean up (disposal). Actually, the using statement, in conjunction with each object's Dispose method, can ensure that any bit of designated code is executed. What distinguishes these mechanisms from each other is what happens when this cleaned up code is executed. When the thread of execution leaves the try region of a try/finally construct, the code inside the finally region is executed immediately. Similarly, when a using statement goes out of scope, all objects associated with the using statement are disposed of. This immediacy differs from when an object that goes out of scope can be Garbage Collected. Garbage Collection will eventually cleanup each object that goes out of scope. Although Garbage Collection can be explicitly triggered (the GARBAGE COLLECTION class's Collect method), Garbage Collection is typically a non-deterministic process. Garbage Collection kicks off when a certain threshold of memory is exceeded (resources become scarce) or if the process terminates, and garbage collection hence finalizes every object. As of Beta 2 of .NET, garbage collection, and hence the calling of each object's Finalize method, is guaranteed for all objects within a process. This behavior was not guaranteed in Beta 1 of the .NET Framework.

Finalize Review

When Garbage Collection cleans up an object (finalizes an object) it calls said object's Finalize method. In C#, a Finalize method is specified in the following form:

class ClassName
{
~ClassName() // destructor a.k.a. a Finalize method
{
// clean up code here
}
}

The managed heap contains a current offset pointer. This pointer refers to the next location in the managed heap in which memory will be allocated. Each time memory is allocated, this pointer is incremented by the size of the memory allocation. When the current offset pointer exceeds a certain threshold, Garbage Collection is initiated. If the current offset pointer threshold is not exceeded then Garbage Collection is not triggered and hence any object that has gone out of scope is not finalized (the Finalize method is not called). To not exceed this threshold means that memory is not being allocated. Yes, Garbage Collection can be triggered programmatically. Yes, the managed heap is a bit more complicated than described (per-thread heaps, a large memory heap, finalization queue, etc.), but the basic idea is that objects are not cleaned up immediately when they go out of scope.

The SWPFinalizeDemo console application demonstrates a scenario in which the Finalize method is not called. This application contains a class, SWPFinalize. This class contains a Finalize method (e.g. ~SWPFinalize) which opens a file (new StreamWriter(fileName)), writes a piece of critical information (file.WriteLine), and closes the file (file.Close). The Finalize method performs one more crucial step. The SWPFinalize class contains a static data member, done. When the Finalize method is called, done is set to true. The Finalize method of the SWPFinalize class is as follows:

~SWPFinalize()
{
StreamWriter file = new StreamWriter(Path.GetTempFileName());

file.WriteLine("Some piece of critical information");

file.Close();
done = true;
}

The done data member is used by a while loop. This while loop resides in the Main method. The Main method first calls the SWPOutOfScope method. This creates an instance of type SWPFinalize, then returns. This means that the object has gone out of scope (becomes inaccessible) and is hence a candidate for Garbage Collection. The code associated with the SWPOutOfScope method is as follows:

static void SWPOutOfScope()
{
SWPFinalize finalize = new SWPFinalize();
} // instance, finalize, is out of scope as of now

Once the Main method has called the SWPOutOfScope method, a while loop is executed. This while loop loops until that done data member is set to true. The Main method is implemented as follows:

static void Main(string[] args) { SWPOutOfScope(); while (!done); }

The source code of the SWPFinalizeDemo console application is as
            follows:

using System;
using
            System.IO;

namespace SWPFinalizeDemo
{
class
            SWPFinalize
{
string fileName;
static bool done =
            false;

SWPFinalize(string fileName)
{
this.fileName =
            fileName;
}

~SWPFinalize()
{
StreamWriter file = new
            StreamWriter(fileName);

file.WriteLine("Some piece of
            critical information");
file.Close();
done =
            true;
}

static void SWPOutOfScope()
{
SWPFinalize
            finalize = new
            SWPFinalize(Path.GetTempFileName());
}

[STAThread]
static
            void Main(string[] args)
{
SWPOutOfScope();
while
            (!done);
}
}
}

When the SWPFinalizeDemo
            application is run, it runs forever. This is because this
            application reaches a point in its execution at which it will
            allocate no more memory. Because no more memory is allocated,
            resources do not become scarce and Garbage Collection is not
            initiated. Because Garbage Collection is not run then the Finalize method is not called, and hence the
            done data member is never set to true
            (the condition that allows the program to terminate).

This example is not meant to be practical because most
            applications continuously allocate memory. Still, there are
            applications that allocate memory up to a certain point and then
            allocate no more. This will result in a situation such as that just
            presented in the SWPFinalizeDemo
            application. Certain objects will not be finalized until the
            application exits (thus initiating finalization for all objects) or
            until Garbage Collection is called programmatically (the Garbage Collection class's Collect method).

Try/Finally Review

The basic premise of the try/finally construct is that no matter
            how the try block is exited, the code in the finally block will
            always be executed (or so it is hoped). A try/finally looks like
            this:

    try
{
//
            Doesn't matter how you exit: throw, break, continue, return
//
            goto - finally block is always
            executed
}

finally
{
// always executed (so we
            hope)
}

The SWPTryFinallyDemo application
            demonstrates the finally block always being executed regardless of
            how the try region is exited. This application exposes an
            enumeration, HowToLeaveTry. This
            enumeration allows each way to leave a try/finally region to be
            specified. The HowToLeaveTry
            enumeration is declared as follows:

    enum
            HowToLeaveTry
{
ViaFallThrough,
ViaGoto,
ViaReturn,
ViaException
}

The SWPShowTryFinally method
            handles each enumeration value of the HowToLeaveTry enumeration. The mechanisms
            used to exit the try region are self-documenting. If the enumeration
            value of ViaGoto is specified, then
            the try region is exited using a goto
            statement. If the enumeration value of ViaReturn is specified, then the try region
            is exited using a return statement.
            The permissible ways to leave the try region include: return, goto,
            exception, break, continue or simply falling out the
            bottom.

The code from the SWPTryFinallyDemo
            application is as follows:

    static void
            SWPShowTryFinally(HowToLeaveTry
            howToLeaveTry)
{
try
{
switch(howToLeaveTry)
{
case
            HowToLeaveTry.ViaException:
throw new Exception("Dummy
            exception");

case
            HowToLeaveTry.ViaReturn:
return;

case
            HowToLeaveTry.ViaGoto:
goto LetsGo;

case
            HowToLeaveTry.ViaFallThrough:
break;
}
}

catch(Exception
            ex)
{
// ingore
            this
}

finally
{
Console.WriteLine("In
            Finally");
}
LetsGo:
return;
}

The basic idea behind the previous code is that regardless of the
            value associated with the parameter, HowToLeaveTry, the code in the finally
            region is executed. The code in this region simply displays the text
            "In Finally" to the output console
            window.

Meticulous readers may have noticed that the SWPShowTryFinally method does not
            demonstrate how to leave a try block using either break or continue. The support code that comes with
            this article includes examples of leaving a try block using break and continue.

Using Statement and IDisposable Review

The using statement is associated
            with a number of objects. The using
            statement takes the following form:

using (objects-to-clean-up)
{
} //
            using out of scope. Dispose called for each object

  • objects-to-clean-up -- list of objects disposed of when the
              using statement goes out of scope.
              The objects-to-clean-up is a list of one or more objects where
              multiple objects are separated by a comma. Each object in
              objects-to-clean-up takes the following form:

(expression | type identifier =
            initializer)

  • expression -- code that creates an object. Said object must
              implement the IDisposable interface
              since the Dispose method is called once the using block goes out
              of scope.

  • type -- type created by "initializer". Said type must
              implement the IDisposable interface.

  • identifier -- name of the instance to create.

  • initializer -- a class factory, new operator or mechanism that that
              creates an object.

Each object associated with a using
            statement must implement the IDisposable interface (found in the System namespace), which is defined as
            follows:

public interface IDisposable
{
public
            void Dispose();
}

Why this interface, that only exposes but a single method, the
            Dispose method? The idea is that each
            object associated with the using
            statement will have its Dispose method
            called when the using statement goes
            out of scope. MSDN specifies that the IDisposable interface and its Dispose method provide a uniform mechanism
            with which an object can release unmanaged resources. Clearly the
            Dispose method can perform any action
            required of an object. Garbage Collection recognizes IDisposable, and will hence call the Dispose method as part of object
            finalization. Recall that an object's Finalize method could not be called
            explicitly. The Dispose method does
            not suffer this limitation. This means that both the consumer of an
            object and Garbage Collection can surrender unmanaged resources
            using IDisposable 's Dispose method.

Clearly the using statement only
            makes sense to include as part of C# if implementing the IDisposable interface is commonplace. The
            litany of classes that implement the IDisposable interface include AsymmetricAlgorithm, BinaryReader, BinaryWriter, Brush, CacheDependency, Component, ComponentDesigner, Container, Control, CryptoAPITransform, Cursor, and CustomLineCap. The preceding list only goes
            up to the letter 'C'. If the classes from 'D' to 'Z' were mentioned
            with respect to IDisposable, the list
            would be monstrous.

The FileStream (from the System.IO namespace) is yet another class
            that implements IDisposable. For this
            reason, this class will be used to demonstrate the using statement in action. To understand how
            to use the using statement, consider
            the following snippet from the console application SWPUsingDemo:

string fileName =
            Path.GetTempFileName(); // get a tempory file name

if
            (File.Exists(fileName))

{
File.Delete(fileName);
}

// fileIO opens a file
            that is not shared
// (FileShare.None). This means that any
            future
// attempt to open file will fail unless
// fileIO has
            closed the file
using (FileStream fileIO =
new
            FileStream(fileName,

FileMode.CreateNew,
FileAccess.ReadWrite,
FileShare.None))
{
}
            // fileIO.Dispose() called here which closes
// the
            file

// This succeeds only if fileIO has called its Close

// method which is called as part of
            fileIO.Dispose().
FileStream fileIO2 =
new
            FileStream(fileName,

FileMode.Open,
FileAccess.Read,
FileShare.None);

In the previous snippet, FileStream
            fileIO creates and opens a file specified by fileName with the FileShare set to None. This means no other instance of the
            file can be opened, because there is no sharing. Once fileIO closes the file by calling the Close method, other instances of the file
            can be opened. Clearly the FileStream
            fileIO2 opens an instance of the same file after the using statement goes out of scope. The
            constructor of fileIO2 successfully
            opens the file specified by fileName,
            which means that the using statement
            disposes of the instance fileIO by
            calling Dispose, which in turn closes
            fileIO.

Conclusions

Thus far we have discussed the mechanisms C# uses in order to
            ensure object cleanup occurs (try/finally, Finalize and the using statement). Clearly, parts 2 and 3 of
            this article is where the raw, unadulterated excitement takes place.
            Part 2 presents each mechanism that can be used by .NET to cleanup
            up a thread (Thread.Abort and Application.ExitThread) or a process (Environment.Exit, Application.Exit, Process.CloseMainWindow, and Process.Kill). As part of this presentation,
            the behavior of each such cleanup mechanism is presented in the
            context of how it affects try/finally, Finalize and the using statement. A series of concise
            examples are used to demonstrate the behavior of each thread/process
            method used in conjunction with C# object cleanup.

Part 3 of this article mimics the pattern of Part 2. This
            includes the use of a series of concise examples that serve to
            demonstrate the concepts presented in Part 3 of this article. This
            third portion of the article reviews Win32's process and thread
            cleanup functions: TerminateThread,
            TerminateProcess, ExitThread, and ExitProcess. The aforementioned functions
            are evaluated with respect to how they influence try/finally, Finalize and the using statement. This third portion of the
            article reviews the best approaches that can be taken to insure that
            objects cleanup fully, and makes note as to which .NET and Win32
            process/thread cleanup mechanisms should not used because of their
            interference with object cleanup.

                If you'd like to discuss a broader
                  range of C#Today topics and issues, or if you have any C#
                  related questions you need answers to, please visit our online
                  resource centres at : http://p2p.wrox.com/list.asp?list=csharptoday_discuss&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq
                  and http://p2p.wrox.com/list.asp?list=c_sharp&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq

     Brought to you by ownSky!
 C#Today - © 2002 WroxPress
Limited


Creating an ISAPI FIlter HTTP Packet Analyzer

Creating an ISAPI FIlter HTTP Packet Analyzer
http://www.csharptoday.com/content.asp?id=1719
Article Rating: 4.5
Published on December 14, 2001
Abstract
Part of the job as a software engineer is to design, code and debug HTTP/SOAP application systems. Normally, one would use Windows 2000’s Network Monitoring tool to analyze network packets to and from a machine, but this is a hassle to do every time, because by default this tool captures all network packets (including Ethernet, ATM etc) when all we may want is HTTP and the layer above that. To do this in the Network Monitor, we have to filter to only to view HTTP packets, and even then these raw packets are in hexadecimal, which is hard to read or diagnose. To solve this problem, Kenny Lim shows us how to build a tool to monitor HTTP packets.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

Part of my job as a software engineer is designing, coding and debugging my HTTP/SOAP application systems. Normally, I will use Windows 2000's Network Monitoring tool to analyze network packets to and from my machine. This is a hassle to do every time, because by default this tool will capture all network packets (including Ethernet, ATM etc) and all I want is HTTP and the layer above that. To do this in the Network Monitor, I have to filter to only to view HTTP packets, and another problem is that the raw packet (which is in hexadecimal) is hard to read or diagnose.

To solve my problem, I have designed a small analyzing system to analyze and debug my development process. There are two parts to this program:

  • An Internet Server API (ISAPI) filter written in unmanaged Visual C++ (I call this RawHTTP.dll).
  • A smart client program written in .NET Framework using C# programming language (I call this LogAnalyser.exe).

With this tool, I will have a SYSTEM TRAY icon on my machine, which will blink when a new HTTP packet is captured. Then I click on this icon, a window will be spawned and will list HTTP or SOAP packets in summary showing HTTP headers, date-time, sessions, source IP address and so on (Figure 1). If I want to look at a packet in more detail, I drill down into it by selecting a row (Figure 2).

Figure 1

Figure 2

System Requirements

This program was compiled with Microsoft Visual Studio.NET Beta 2 and run in Windows XP Professional. The C++ ISAPI (RawHTTP) is run on IIS 5.0.

The client program (LogAnalyzer) requires the .NET Framework Beta 2, and is written in C#.

Defining the Problem (Server Side)

This tool listens to all HTTP packets coming into and serving out from IIS. To do this, we have two options:

  • Use an ISAPI Filter (Native Code)
  • Use an ASP.NET HttpModule

After some fact finding, I found out that HttpModule or HttpHandler will not be able to listen to all the extensions and requests that are invalid. It is possible in IIS 6.0 by mapping all extensions to an HttpModule, but this will require all extension to be mapped to ASP.NET. This will break existing ASP or other CGI applications. Knowing this, I decided to write an ISAPI filter for this task.

ISAPI Filter (RawHTTP)

An ISAPI filter is nothing more than a conventional Win32 DLL. A DLL that work as ISAPI filter must export 2 functions - which are as follows:

GetFilterVersion

This function is the entry-point function called by IIS on this ISAPI filter. This function must be preset for the filter to work properly. Normally, this is a very good stage to add an initialization routine.

HttpFilterProc

IIS will call this function whenever the event we are interested happens. IIS will pass certain pointers to data structure to this function.

ISAPI filters are the lowest level in writing web applications, and certainly ISAPI can deliver the highest performance if carefully designed. See figure 3 for a simple illustration of where the ISAPI filter fits within IIS. When a packet comes into IIS, it will go through the ISAPI filters first, from the top of the list to the bottom. Any filter can change or discard the packet so that it won't be seen by any ISAPI filter or extension listed after it.

ISAPI extensions (also known as ISAPI applications in the IIS Manager), will not, unlike the filters, intercept every packet. They will be called only if the client browser explicitly calls it (i.e. http://localhost/isapi/myextension.dll?param=something), or if its mapped extension is being called. For example, any call to .ASP will call the asp.dll to process.

Figure 3

There are a few events that can be registered so that IIS will callback to your routine; IIS events range from controlling IIS logging, performing custom authentication, to reading raw HTTP packets. IIS events also happen in a sequence known as event notification order. For example, SF_NOTIFY_READ_RAW_DATA will happen before SF_NOTIFY_PREPROC_HEADERS. For a complete list of the event notification order for ISAPI filters, check out http://msdn.microsoft.com/library/default.asp?url=/library/en-us/iisref/html/psdk/asp/isgu744y.asp&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq.

For our filter code, we will only need 2 notifications from IIS:

  • SF_NOTIFY_READ_RAW_DATA
  • SF_NOTIFY_SEND_RAW_DATA.

SF_NOTIFY_READ_RAW_DATA is the first event notification that will happen when IIS receives the raw HTTP packet from a client browser or Internet application. SF_NOTIFY_SEND_RAW_DATA is the last event notification from IIS that happens when IIS sends HTTP packets generated by server code.

Overall Flowchart for the ISAPI (RawHTTP)

Start coding the ISAPI Filter

Start Visual Studio .NET and choose Visual C++ Projects | Win32 Project, and enter the name of your project (Figure 4). We will then see Figure 5: choose the Application Settings tab and choose DLL. Click Finish and Visual Studio.NET will generate a blank template.

Figure 4

Figure 5

Next add a DEF file into the project from Solution Explorer and name it RawHttp.DEF. The content of the file should be the following:

LIBRARY RawHttp
EXPORTS
GetFilterVersion
HttpFilterProc

This file defines the function that will be exported from this DLL. These are the two entry points required for it to be an ISAPI filter. In the dialog box above, Visual Studio actually has a wizard to generate the EXPORT function automatically. I'm not using this because it won't work in my case. The wizard will generate the __declspec(dllexport) keyword in front of our function declaration. However, ISAPI functions are declared as __stdcall. The compiler won't let us have both. To work around this, we use a DEF file.

Since we have promised to expose these two functions, let's code it. To begin, add a header (.h) file to the project, with the following content:

extern "C"
{
BOOL WINAPI GetFilterVersion(HTTP_FILTER_VERSION *pVersion);
DWORD WINAPI HttpFilterProc(HTTP_FILTER_CONTEXT *pfc, DWORD dwNotificationType, LPVOID pvNotification);
}

The above code is the function prototypes definition. Now, open RawHTTP.cpp (which has been generated by the VC++ Wizard) to add the following implementation code:

GetFilterVersion

BOOL WINAPI
            GetFilterVersion(HTTP_FILTER_VERSION *pVersion)
{
// Register
            Notifications
pVersion->dwFlags = SF_NOTIFY_ORDER_DEFAULT
            |
SF_NOTIFY_READ_RAW_DATA |
SF_NOTIFY_SEND_RAW_DATA;

//
            Setting Version and Description
pVersion->dwFilterVersion =
            HTTP_FILTER_REVISION;
strcpy(pVersion->lpszFilterDesc,
            "RawHTTP");

// Constructing Logger
pLogger = new
            Logger();

return TRUE;
}

This function will register two of the required IIS notifications, set the ISAPI version and description, and finally instantiate a global instance of Logger (Logger is a generic C++ class). The SF_NOTIFY_ORDER_DEFAULT tells IIS to load this ISAPI filter with the default order, which is Low priority in IIS 5.0. If you want to change the priority so that your filter can be called earlier, you can change this flag to SF_NOTIFY_ORDER_HIGH. When the correct notification is received by IIS, IIS will point to the HttpFilterProc function, which is shown below:

HttpFilterProc

DWORD WINAPI HttpFilterProc(HTTP_FILTER_CONTEXT *pFilterContext, DWORD dwNotificationType, LPVOID pvNotification)
{
DWORD dwReturn = SF_STATUS_REQ_NEXT_NOTIFICATION;

switch(dwNotificationType)
{
case SF_NOTIFY_READ_RAW_DATA:
dwReturn = pLogger->ProcessMessage(pFilterContext,
(HTTP_FILTER_RAW_DATA *)pvNotification, Logger::READ);
break;

case SF_NOTIFY_SEND_RAW_DATA:
dwReturn = pLogger->ProcessMessage(pFilterContext,
(HTTP_FILTER_RAW_DATA *)pvNotification, Logger::WRITE);
break;

default: break;
}
return dwReturn;
}

In this function, we have a switch statement to check if the packet is SF_NOTIFY_SEND_RAW_DATA or SF_NOTIFY_READ_RAW_DATA. It will then pass to the Logger class to perform more work.

Logger Class

Our Logger class will be used by this ISAPI to perform all the work. To add a generic class to the project, in Visual Studio .NET, right click on the project and add a class. Choose Generic C++ Class and Open (Figure 6).

Figure 6

The next dialog wizard will appear (Figure 7), enter Logger into the class name field and click Finish. This will generate the skeleton code of Logger.cpp and Logger.h.

Figure 7

Logger.h

The first section of this header file defines the constants for the class. Hard-coding constant values in the source code is a good example. In a real-world application, it is better to have all the constants loaded up from a configuration file.

#define OFFSET_IDENTIFIER 5
#define OFFSET_START 6
#define BLOCK_SIZE 100
#define GUID_SIZE 36
#define LOCALHOST "127.0.0.1"
#define COMMAND_IDENTIFIER "HGI8H"
#define COMMAND_START_IDENTIFIER '*'
#define COMMAND_STOP_IDENTIFIER 'B'
#define COMMAND_START_PATH "C:\\HTTPRAW\\START.KEY"
#define COMMAND_STOP_PATH "C:\\HTTPRAW\\STOP.KEY"
#define LOG_PATH "C:\\HTTPRAW\\LOG.TXT"

Next we have two structures that will hold valuable information for the class. The ContextStore structure is used to store a session number that relates to a session in IIS. Since IIS can use different threads to service the same request and response, this SessionNo will be the key relating all the conversations for a request-response from the browser/application to IIS and back.

InfoStore is the structure that stores information to be logged. Information such as source IP address, datetime, request direction, and HTTP headers are stored in this structure. In our InfoStore structure, we have a constructor and destructor for the structure. In the constructor, we make sure we have initialized our variable. In the destructor, we will free the memory allocated.

typedef struct ContextStore { DWORD
            SessionNo; } SContextStore;

typedef struct InfoStore
{
const static int c_IPAddressLen = 20;
const static int
            c_DetailTimeLen = 20;
const static int c_MillisecondLen =
            5;
const static int c_SessionNoLen = 20;
const static int
            c_DirectionLen = 10;

InfoStore(){
memset(m_IPAddress,
            NULL, c_IPAddressLen);
memset(m_DetailTime, NULL,
            c_DetailTimeLen);
memset(m_Millisecond, NULL,
            c_MillisecondLen);
memset(m_SessionNo, NULL,
            c_SessionNoLen);
memset(m_Direction, NULL,
            c_DirectionLen);
m_Buffer =
            NULL;
}

~InfoStore()
{
if(m_Buffer)
delete []
            m_Buffer;
}
char m_IPAddress[c_IPAddressLen];
char
            m_DetailTime[c_DetailTimeLen];
char
            m_Millisecond[c_MillisecondLen];
char
            m_SessionNo[c_SessionNoLen];
char
            m_Direction[c_DirectionLen];
char * m_Buffer;
long
            m_BufferLen;
} SInfoStore;

The following is the Logger class definition. The Critical Section (m_Section) is used to control synchronization of multiple threads, to stop them from corrupting the log file. Synchronization is needed because ISAPIs are implicitly multi-thread. FilterAction is a C++ enumeration to be passed to a function and differentiate between READ and WRITE. The member field m_IsOperating stores the state if the logging is turned off or on. The member functions will be explained in more detail.

class Logger
{
public:
Logger();
~Logger();

enum FilterAction { READ, WRITE };
DWORD ProcessMessage(HTTP_FILTER_CONTEXT *pFilterContext,
HTTP_FILTER_RAW_DATA *pFilterRawData, FilterAction f);

private:
BOOL CheckHttpCommand(SInfoStore *pInfo);
char * ReadContent(char *pFileName);
SContextStore * ReadContextStore(HTTP_FILTER_CONTEXT
*pFilterContext, FilterAction f);
BOOL ReadInformation(SInfoStore *pInfo, HTTP_FILTER_CONTEXT
*pFilterContext, FilterAction f);
BOOL WriteLog(SInfoStore *pInfo);
BOOL WriteDetailLog(SInfoStore *pInfo);
DWORD ReadRequestPosition(SInfoStore *pInfo);

CRITICAL_SECTION m_Section;
BOOL m_IsOperating;
};

Logger.cpp

Constructor & Destructor

When the Logger is instantiated, the constructor will be called. In the constructor, we initialize the Critical Section to be used as a synchronization object for all the servicing threads. When the object is done, the destructor will be called. When this happens, it will delete the same Critical Section.

#include
            "stdafx.h"

Logger::Logger()
{
// if you don't want your
            ISAPI to be run by default, uncomment the
// following
            statement.
// m_IsOperating =
            false;
::InitializeCriticalSection(&m_Section);
}

Logger::~Logger()
{
::DeleteCriticalSection(&m_Section);
}

Logger::ProcessMessage

ProcessMessage is the member function that is called from HttpFilterProc. This method will be called regardless of READ or SEND notification. To differentiate the caller context, we use a FilterAction enumeration. Along with this enumeration, this method accepts the standard structure from IIS.

DWORD Logger::ProcessMessage(HTTP_FILTER_CONTEXT *pFilterContext, HTTP_FILTER_RAW_DATA *pFilterRawData, FilterAction f)

{
DWORD dwTemp;
DWORD dwReturn = SF_STATUS_REQ_NEXT_NOTIFICATION;
// Code Continues below .
}

Part of the logic of ProcessMessage is to instantiate a SInfoStore structure (look at logger.h), this will store information about the packet.

Next we attempt to create a block of memory to store the incoming data. We won't define a fixed size array to store this, because we want to make sure we have no buffer overrun problem in this project. Buffer overrun is the number one cause of problems like downtime or hacking. When someone knows that there is a flaw in the buffer management (for example, in using a fixed size array), they can transmit more bytes to the server and ultimately take over the IIS process (inetinfo.exe) by injecting malicious code.

Check if the memory allocation is successful; if not, abort the session. If the allocation is ok, we can copy the packet buffer and source IP address into our structure and then check if it is a command from our smart client, LogAnalyzer (for more information about this routine, look into the CheckHttpCommand member function).

If the packet is not a command, the code will check whether the current state (m_IsOperating) is on or off. If it's on, we can proceed to read the remaining information like datetime and session into our structure (SInfoStore). With all the information populated into the structure, we can then write the information to the log file. After writing to the log file, we can leave the critical section so that other threads can get into the log file and write as well. Finally, we de-allocate the structure (SInfoStore), which will then call its destructor.

SInfoStore *pInfo = new
            SInfoStore();
if(pInfo)
{
pInfo->m_Buffer = new
            char[pFilterRawData->cbInBuffer+1];
if(pInfo->m_Buffer)
{
memcpy(pInfo->m_Buffer,
            pFilterRawData->pvInData,
pFilterRawData->cbInBuffer);
pInfo->m_Buffer[pFilterRawData->cbInData]
            = NULL;
pInfo->m_BufferLen =
            pFilterRawData->cbInData;

::EnterCriticalSection(&m_Section);

dwTemp
            =
            pInfo->c_IPAddressLen;
pFilterContext->GetServerVariable(pFilterContext,
            "REMOTE_ADDR",
pInfo->m_IPAddress,
            &dwTemp);
if(!this->CheckHttpCommand(pInfo)){
if(this->m_IsOperating){
this->ReadInformation(pInfo,
            pFilterContext,
            f);
this->WriteLog(pInfo);
}
}
else
dwReturn =
            SF_STATUS_REQ_FINISHED;

::LeaveCriticalSection(&m_Section);

}
}
delete
            pInfo;
return dwReturn;
}

Logger::CheckHttpCommand

One of the features of our tool is the ability to control the filter from a smart client (LogAnalyzer). For example, we can turn off the packet logging when we're not testing a web application. To achieve this, the ProcessMessage member function will call CheckHttpCommand for every packet that comes in. The algorithm is simple. We first check if the request is coming from localhost. If it's not, return immediately. If it is from localhost, the function will then check if an identifier exists in the HTTP header (COMMAND_IDENTIFIER). If this string exists in the packet, it will go further down to check if it's a START (COMMAND_START_IDENTIFIER) command or STOP command (COMMAND_STOP_IDENTIFIER). Finally it will check if the key is matching correctly. The mechanism used for the key is that the client will generate a GUID at the time of the request (START or STOP) and then write the GUID into a file and send it as part of the request command. This method will compare the GUID from the file and the GUID sent up with the request. If they match, the command is authenticated.

BOOL Logger::CheckHttpCommand(SInfoStore
            *pInfo)
{
BOOL bReturn =
            FALSE;
if(strcmp(pInfo->m_IPAddress,
            LOCALHOST)==0)
{
{
char char *pBlock = new
            char[BLOCK_SIZE+1];
if(pBlock){
if(pBlock)
{
strncpy(pBlock,
            pInfo->m_Buffer, BLOCK_SIZE);
pBlock[BLOCK_SIZE] =
            NULL;
char *p = ::strstr(pBlock,
            COMMAND_IDENTIFIER);
if(p){
char *pKey;
char *pPos =
            p+6;
if(*(p+5) ==
            COMMAND_START_IDENTIFIER){
if(p)
{
char *pKey;
char
            *pPos = p+OFFSET_START;
if(*(p+OFFSET_IDENTIFIER) ==
            COMMAND_START_IDENTIFIER)
{
pKey =
            this->ReadContent(COMMAND_START_PATH);
if(pKey){
if(pKey)
{
if(::strncmp(pKey,
            pPos, GUID_SIZE) == 0){
this->m_IsOperating = TRUE;
bReturn
            = TRUE;
}
delete[] pKey;
}
}
else if(*(p+5) ==
            COMMAND_STOP_IDENTIFIER){
{
this->m_IsOperating =
            TRUE;
bReturn = TRUE;
}
delete[] pKey;
}
}
else
            if(*(p+OFFSET_IDENTIFIER) == COMMAND_STOP_IDENTIFIER)
{
pKey
            pKey pKey = this->ReadContent(COMMAND_STOP_PATH);
if(pKey){
            if(pKey)
{
if(::strncmp(pKey, if(::strncmp(pKey, pPos,
            GUID_SIZE) == 0){
0)
this->m_IsOperating =
            FALSE;
bReturn = TRUE;
}

delete[]
            pKey;
}
}
}
delete[]
            pBlock;
}
}
{
this->m_IsOperating = FALSE;
bReturn
            = TRUE;
}
delete[] pKey;
}
}
}
delete[]
            pBlock;
}
}
return return bReturn;
}

Logger::ReadContent

ReadContent is just a helper function that will return the content of a file. This function is use to fetch the GUID from the file.

char * Logger::ReadContent(char
            *pFileName)
{
FILE *fptr = fopen(pFileName,
            "r");
if(fptr){
char *pKey = new
            char[GUID_SIZE+1];
if(pKey)
{
fgets(pKey, GUID_SIZE+1,
            fptr);
pKey[GUID_SIZE] = NULL;
fclose(fptr);
return
            pKey;
}
}
return NULL;
}

Logger::ReadContextStore

ReadContextStore is invoked by ReadInformation. The purpose of this method is to generate our session number when it is a request (READ). To do so, we have to use AllocMem from ISAPI to allocate memory for our data. When we use AllocMem, IIS will handle the memory management for the allocated memory. It will then stuff into pFilterContext. This way, we can set the context (session number) throughout the same request-response. If it's a reply from the IIS (SEND), we can just grab the information from pFilterContext.

SContextStore *
            Logger::ReadContextStore(HTTP_FILTER_CONTEXT
*pFilterContext,
            FilterAction f)
{
SContextStore *pContext = NULL;
if(f ==
            FilterAction::READ){
if(f == FilterAction::READ)
{
pContext
            = (SContextStore *)pFilterContext->AllocMem(pFilterContext,
(pFilterContext, sizeof(SContextStore),
            0);
pContext->SessionNo =
            ::GetTickCount();
pContext->CountNo =
            1;
pFilterContext->pFilterContext =
            pContext;
}
else
{
pContext = (SContextStore
            *)pFilterContext->pFilterContext;
pContext->CountNo =
            pContext->CountNo+1;
pFilterContext->pFilterContext =
            pContext;
}
return pContext;
}

Logger::ReadInformation

ReadInformation will be called by ProcessMessage to populate the SInfoStore structure:

BOOL Logger::ReadInformation(SInfoStore
            *pInfo, HTTP_FILTER_CONTEXT
*pFilterContext, FilterAction
            f)
{
SYSTEMTIME
            t;
::GetSystemTime(&t);
::sprintf(pInfo->m_DetailTime,
            "%02d:%02d:%02d.%d",
t.wHour, t.wMinute, t.wSecond,
            t.wMilliseconds);
::sprintf(pInfo->m_Millisecond, "%d%c",
            t.wMilliseconds, NULL);
::sprintf(pInfo->m_SessionNo, "%d",
            SYSTEMTIME t;
SContextStore *pContext =
            this->ReadContextStore(pFilterContext,
            f)->SessionNo);
f);
::GetSystemTime(&t);
::sprintf(pInfo->m_DetailTime,
            "%02d:%02d:%02d.%d%d", t.wHour, t.wMinute,
t.wSecond,
            t.wMilliseconds,
            pContext->CountNo);
::sprintf(pInfo->m_Millisecond,
            "%d%d%c", t.wMilliseconds, pContext->CountNo,
            NULL);
::sprintf(pInfo->m_SessionNo, "%d",
            pContext->SessionNo);
if(f if(f ==
            FilterAction::READ)
strcpy(pInfo->m_Direction,
            "Incoming");
else
strcpy(pInfo->m_Direction,
            "Outgoing");
return TRUE;
}
strcpy(pInfo->m_Direction,
            "Incoming");
else
strcpy(pInfo->m_Direction,
            "Outgoing");
return TRUE;
}

Logger::WriteLog

WriteLog is used to append to the log file. The log file is created in shared mode, and we set the file pointer to the end before we begin writing to it. In the body, we use the SInfoStore structure and write XML element strings into the log file. However, this function will just log summary information. The generated XML file will be loaded up by the client program as a data grid. The raw detail packet information is stored in the detail file by WriteDetailFile (below):

BOOL Logger::WriteLog(SInfoStore
            *pInfo)
{
DWORD dwTemp;
HANDLE hFile =
            ::CreateFile(LOG_PATH, GENERIC_WRITE,
FILE_SHARE_READ, NULL,
            OPEN_ALWAYS, FILE_ATTRIBUTE_NORMAL, NULL);
if(hFile !=
            INVALID_HANDLE_VALUE){
::SetFilePointer(hFile, 0, 0,
            FILE_END)

::WriteFile(hFile, "<call>", 6, &dwTemp,
            NULL);

::WriteFile(hFile, "<session>", 9, &dwTemp,
            NULL);
::WriteFile(hFile, pInfo->m_SessionNo,
strlen(pInfo->m_SessionNo), &dwTemp,
            NULL);
::WriteFile(hFile, "</session>", 10, &dwTemp,
            NULL);

::WriteFile(hFile, "<time>", 6, &dwTemp,
            NULL);
::WriteFile(hFile, pInfo->m_DetailTime,
strlen(pInfo->m_DetailTime), &dwTemp,
            NULL);
::WriteFile(hFile, "</time>", 7, &dwTemp,
            NULL);

::WriteFile(hFile, "<ip>", 4, &dwTemp,
            NULL);
::WriteFile(hFile, pInfo->m_IPAddress,
strlen(pInfo->m_IPAddress), &dwTemp,
            NULL);
::WriteFile(hFile, "</ip>", 5, &dwTemp,
            NULL);

::WriteFile(hFile, "<dir>", 5, &dwTemp,
            NULL);
::WriteFile(hFile, pInfo->m_Direction,
strlen(pInfo->m_Direction), &dwTemp,
            NULL);
::WriteFile(hFile, "</dir>", 6, &dwTemp,
            NULL);

DWORD dwPos =
            this->ReadRequestPosition(pInfo);
if(dwPos ==
            0)
{
::WriteFile(hFile, "<request>", 9, &dwTemp,
            NULL);
::WriteFile(hFile, "</request>", 10, &dwTemp,
            NULL);
this->WriteDetailLog(pInfo);
}
else
{
::WriteFile(hFile,
            "<request><![CDATA[", 18, &dwTemp,
            NULL);
::WriteFile(hFile, pInfo->m_Buffer, dwPos, &dwTemp,
            NULL);
::WriteFile(hFile, "]]></request>", 13,
            &dwTemp,
            NULL);
this->WriteDetailLog(pInfo);
}

::WriteFile(hFile,
            "</call>", 7, &dwTemp,
            NULL);

CloseHandle(hFile);
return TRUE;
}
return
            FALSE;
}

BOOL Logger::WriteDetailLog(SInfoStore
            *pInfo)
{
BOOL bReturn = FALSE;
DWORD dwTemp;
char
            *pDetailFile = new char[MAX_PATH];
if(pDetailFile)
            {
memst(pDetailFile, 0, MAX_PATH);
sprintf(pDetailFile,
            "C:\\HTTPRAW\\%s%s%s.txt",
            if(pDetailFile)
{
memset(pDetailFile, 0,
            MAX_PATH);
sprintf(pDetailFile, "C:\\HTTPRAW\\%s%s%s.txt",
            pInfo->m_SessionNo,
pInfo->m_Millisecond,
            pInfo->m_Direction);
HANDLE hDetail = CreateFile(pDetailFile,
            GENERIC_WRITE, FILE_SHARE_READ,
NULL, OPEN_ALWAYS,
            FILE_ATTRIBUTE_NORMAL, NULL);
if(hDetail){
WriteFile(hDetail,
            pInfo->m_Buffer, (DWORD)pInfo->m_BufferLen,
            (DWORD)strlen(pInfo->m_Buffer), &dwTemp,
            NULL);
&dwTemp, NULL);
CloseHandle(hDetail);
bReturn =
            TRUE;
}
delete [] pDetailFile;
}
return
            bReturn;
}
if(hDetail)
{
WriteFile(hDetail,
            pInfo->m_Buffer, (DWORD)pInfo->m_BufferLen, &dwTemp,
            NULL);
CloseHandle(hDetail);
bReturn = TRUE;
}
delete []
            pDetailFile;
}
return bReturn;
}

StdAfx.h

Finally, be sure to add the required header file to the project before compiling. Httpfilt.h is required for ISAPI related functions. String.h is required for string manipulation function such as sprintf.

#pragma once

// Exclude rarely-used stuff from Windows headers
#define WIN32_LEAN_AND_MEAN

// Windows Header Files:
#include <windows.h>
#include <httpfilt.h>
#include <string.h>
#include <stdio.h>
#include "Logger.h"

Compiling & Deploying the DLL

Now all the source code is in place, we can build our DLL. By default, Visual Studio will generate the DLL in DEBUG mode, which is fine, as we may wish to debug our ISAPI. For more information about debugging ISAPI, check out http://support.microsoft.com/default.aspx?scid=kb;EN-US;Q183480&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq.

After building the project, we'll have RawHttp.dll in a debug directory. The next step is to install this DLL in our ISAPI. To do this, open up IIS Manager, right click on the server name, and choose Properties (Figure 8).

In the IIS Master Properties dialog box, choose Edit and then the ISAPI filter tab (Figure 9). Click the Add button and specify the ISAPI name and path to our DLL (Figure 10).

Figure 8

Figure 9

Figure 10

Once we have installed the ISAPI filter, open a command window and run iisreset. This will restart the IIS service and load up the RawHTTP.dll. Once we have done this, create a folder c:\HttpRaw. This is the directory used by this tool.

Now we can run a browser or some other Web Stress utility like WAST to hit our web site. One way that I use to check for memory leaks in my DLL is to use the Performance Monitor, select inetinfo.exe, and choose the Private Bytes counter. By having a stress utility run over the course of a day or two, I can monitor for a memory leak happening.

Windows Form Client (HTTPAnalyzer)

The ISAPI Filter will log summary and detail packet information into files. In this section, we will explore how we are going to utilize those log files. The LogAnalyzer program is a Visual Studio .NET Windows Forms solution consisting of 2 forms. The main form is called FORM1 (Figure 1) and the dialog form is called FORM2 (Figure 2).

FORM1

FORM1 is the main startup window for the client program. This window shows summary HTTP packet information in the log files. To start explaining this window, we should start with the Load event.

Form1_Load Event

The Load event is a good place to initialize resources. In this event, we load icon files into Icon objects. This icon will be used to display the icon on the system tray area:

      private
            void Form1_Load(object sender, System.EventArgs e)
{
//
            loading up icon files to be used in the system tray.
oIconNormal
            = new Icon(Application.StartupPath + "\\icon0.ico");
oIconChange1 = new Icon(Application.StartupPath +
            "\\icon1.ico");
oIconChange2 = new Icon(Application.StartupPath +
            "\\icon2.ico");
oIconDisable = new Icon(Application.StartupPath +
            "\\icon3.ico");

In this section, we instantiate a FileSystemWatcher object. The FileSystemWatcher is used to notify the client program when certain filesystem events like file size changes happen. In our case, we tell it to monitor for log.txt file (the log file that our RawHttp ISAPI writes to) and tell it to notify our callback function (OnChanged) when an event happens.

         //
            registering file system watcher.
oFSWatch = new
            FileSystemWatcher();
oFSWatch.Path =
            sCfgPath;
oFSWatch.NotifyFilter =
            NotifyFilters.LastWrite;
oFSWatch.Filter =
            sCfgFileName;
oFSWatch.Changed += new
            FileSystemEventHandler(OnChanged);
oFSWatch.EnableRaisingEvents =
            true;

In this section, we set up the system tray area. We instantiate the NotifyIcon object, setting up the default icon, text, and add an event handler (OnClick). We also create a context (or right click) menu and add a bunch of menu items. Each menu item has an event handler associated with it:

         //
            setting up NotifyIcon and context menu.
tray = new
            NotifyIcon();
ContextMenu ctx = new ContextMenu();
oMenuStatus
            = new MenuItem("Disable Service", new
            EventHandler(OnStatus));
ctx.MenuItems.Add(oMenuStatus);
ctx.MenuItems.Add(new
            MenuItem("Refresh", new
            EventHandler(OnRefresh)));
ctx.MenuItems.Add(new
            MenuItem("Clear/Reset", new
            EventHandler(OnClear)));
ctx.MenuItems.Add(new
            MenuItem("-"));
ctx.MenuItems.Add(new MenuItem("Exit", new
            EventHandler(OnExit)));
tray.ContextMenu = ctx;
tray.Icon =
            oIconNormal;
tray.Text = "HTTP Analyser, no
            packet";
tray.Click += new EventHandler(OnClick);
tray.Visible
            = true;
oBtnStatus.Text = oMenuStatus.Text;

In this section, we send an HTTP GET request to our HttpRaw ISAPI filter. This is a command packet to tell HttpRaw to start logging now the client analyzer has started. To send the HTTP request, we use the WebRequest class. PrepareStartKey is a method that will both create a start GUID and return it. The catch block is empty because the ISAPI won't return status code 200 for the GET request. Therefore, GetResponse will always throw an exception.

         try
{
WebRequest
            req = WebRequest.Create(sStartURL +
            PrepareStartKey());
req.GetResponse();
}
// the catch block
            is empty because ISAPI will never return any response for
            command.
catch(Exception ex){}

In this section, we are setting some properties for the DataGrid object. At the end of this event, we call Reload.

         //
            setting some properties for
            DataGrid.
oDataGrid.PreferredColumnWidth =
            150;
oDataGrid.ReadOnly = true;
oDataGrid.ParentRowsVisible =
            false;
this.Reload();
}

Reload()

The purpose of Reload is to populate the DataGrid in FORM1. To do this, we use data binding. This method is called at the end of the form1_load event, the Refresh menu item and the Refresh button:

private void Reload()
{
try
{

The first step to check in this method is whether log.txt exists or not. If this file does not exist, there's no point proceeding because there's no log file to use.

if(File.Exists("c:\\httpraw\\log.txt"))
{

When we know log.txt exists, we create a memory stream (256KB) as a buffer to hold the final XML used to populate the DataSet object. All the code does is insert a <root> element before and after those XML elements from log.txt.

               DataSet
            oDoc = new DataSet();
MemoryStream ms = new
            MemoryStream(MEMORYSIZE); // 256KB memory
StreamWriter s = new
            StreamWriter(ms);

StreamReader oStreamXml = new
            StreamReader("c:\\httpraw\\log.txt");
s.Write("<root>");
s.Write(oStreamXml.ReadToEnd());
oStreamXml.Close();
oStreamXml
            = null;
s.Write("</root>");
s.Flush();

To ensure that the memory stream can be read into the DataSet, we have to set the position to 0. This will set the internal pointer of the memory stream to the start position. Next, we reading the DataSet object and bind the DataSet with the DataGrid object. This will display the summary HTTP packet in the DataGrid.

               ms.Position
            = 0;
oStreamXml = new
            StreamReader(ms);
oDoc.ReadXml(oStreamXml);
oStreamXml.Close();
oStreamXml
            = null;
oDataGrid.DataSource =
            oDoc;
oDataGrid.NavigateTo(0,"call");
}
}
catch(Exception
            e)
{
MessageBox.Show("System Error or ISAPI not
            running.");
this.Dispose();
}
}

OnChanged

The OnChanged method will be called when the log.txt is changed. This method updates the text in the window to tell the user that a new packet has arrived. In addition, we animate an icon to get attention from the user.

   private void
            OnChanged(object source, FileSystemEventArgs e)
{
tray.Icon =
            oIconChange1;
tray.Text = "Last HTTP capture : " +
            DateTime.Now.ToLongTimeString();
oStatusBar.Text = "New Data
            Available, click Refresh. " +
            tray.Text;
Thread.Sleep(ANIMATIONDELAY); // 25 ms
tray.Icon =
            oIconChange2;
Thread.Sleep(ANIMATIONDELAY); // 25 ms
tray.Icon
            = oIconNormal;
}

oBtnClear_Click

This event handler will be fired when the user click on the CLEAR button. This event handler will enumerate through all the .txt files and delete them from the c:\httpraw folder. With this functionality, the working folder can be cleaned up when the user decides to do so. After calling Delete() on each FileInfo object, we'll refresh the DataGrid:

   private void
            oBtnClear_Click(object sender, System.EventArgs
            e)
{
try
{
DirectoryInfo di = new
            DirectoryInfo(@"c:\httpraw\");
FileInfo[] fi =
            di.GetFiles("*.txt");
int nLength = fi.Length;
for(int
            i=0;i<nLength;i++)
fi[i].Delete();
oDataGrid.DataSource =
            null;
oDataGrid.Refresh();
}
catch(Exception
            ex)
{
MessageBox.Show("Error : Try again later, some files are
            being used by other process.");
}
}

FORM2

FORM2 is a dialog box that will be created when the user click on a row of summary packet to drill down. FORM2 is just a content viewer for the detail packet information. There are a few tabs in FORM2, each tab showing different encoding for the same data. Here are a few important methods in FORM2.

GetPacketContent

The purpose of this method is to read the content from a detailed log file (written by HttpRaw) and display it in the default tab (ASCII). This method will start to read the file using BinaryReader. Next the binary byte will be converted to a character and appended to StringBuilder. When all the bytes from the BinaryReader are converted, it will show the string in a text box (oTextAscii). Please note that because the TextBox control cannot display the character NULL, this method removes any NULL character in the string. This is a limitation of this method.

     private void
            GetPacketContent(string sFileName)
{
try
{
// opening
            the detailed log file.
FileStream fs = new
            FileStream("c:\\httpraw\\" + sFileName, FileMode.Open,
            FileAccess.Read);
if(fs!=null)
{
BinaryReader bs = new
            BinaryReader(fs);
// only handle first 256kb of
            content.
StringBuilder sBuilder = new
            StringBuilder(READSIZE);
m_InternalBytes =
            bs.ReadBytes(READSIZE);
int nLen = m_InternalBytes.Length;
//
            for loop to ignore character NULL and append to
            StringBuilder.
for(int
            i=0;i<nLen;i++)
if(m_InternalBytes[i] !=
            0)
sBuilder.Append(Convert.ToChar(m_InternalBytes[i]));
//
            Display the String from StringBuilder to the
            TextBox.
oTextAscii.Text =
            sBuilder.ToString();
bs.Close();
fs.Close();
}
else
oTextAscii.Text
            = "No detail packet for this record.";
}
catch(Exception
            e)
{
oTextAscii.Text = "No detail packet for this
            record.";
}
}

SelecedtIndexChanged

This event handler will be called when the user changes from one tab to another. This event handler will convert from ASCII text into various types of format to display in a text box.

   private void
            tabControl1_SelectedIndexChanged(object sender, System.EventArgs
            e)
{
// Hexadecimal Tab
if(tabControl1.SelectedIndex ==
            1)
{
string sTemp;
char [] c =
            oTextAscii.Text.ToCharArray();
StringBuilder sBuilder = new
            StringBuilder();
for(int i=0;i<c.Length;i++)
{
//
            Convert string to hexadecimal (base 16)
sTemp =
            Convert.ToString(Convert.ToInt32(c[i]), 16);
if(sTemp.Length ==
            1)
sBuilder.Append("0");
sBuilder.Append(sTemp);
sBuilder.Append("
            ");
}
oTextHex.Text = sBuilder.ToString();
}
// Decimal
            Tab
else if(tabControl1.SelectedIndex == 2)
{
string
            sTemp;
char [] c =
            oTextAscii.Text.ToCharArray();
StringBuilder sBuilder = new
            StringBuilder();
for(int i=0;i<c.Length;i++)
{
//
            Convert string to decimal (base 10)
sTemp =
            Convert.ToString(Convert.ToInt32(c[i]), 10);
if(sTemp.Length ==
            1)
sBuilder.Append("00");
else if(sTemp.Length ==
            2)
sBuilder.Append("0");
sBuilder.Append(sTemp);
sBuilder.Append("
            ");
}
oTextDecimal.Text = sBuilder.ToString();
}
//
            Base64 Tab
else if(tabControl1.SelectedIndex == 3)
{
//
            Convert string to Base64 string
oTextBase64.Text =
            Convert.ToBase64String(m_InternalBytes);
}
}

Limitations and Future Work

The design of this filter is to be used as a personal HTTP/SOAP packet analyzer. This work is not intended to be installed on any production environment, because it logs a lot of details about all incoming and outgoing packets.

Secondly, we use the file system to store XML documents about the HTTP packets. To improve performance, we could use a named pipe to stream data from the ISAPI filter to the client program without I/O to the disk. Writing to memory is much faster than to disk.

Conclusion

The motivation of this tool is to make the debugging and analysis of HTTP or SOAP packets much easier than it is today. By having this tool, I can spot a problem much earlier at the packet level. In addition, this tool allows me to know which culprit machine is sending those "Code Red" packets to my computer ;-)

If you'd like to discuss a broader range of C#Today topics and issues, or if you have any C# related questions you need answers to, please visit our online resource centres at : http://p2p.wrox.com/list.asp?list=csharptoday_discuss&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq and http://p2p.wrox.com/list.asp?list=c_sharp&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq
C#Today - © 2002 WroxPress Limited


Manipulating the Windows 2000 Registry and Event Viewer Using C#

Manipulating the Windows 2000 Registry and Event Viewer Using C#
http://www.csharptoday.com/content.asp?id=1716
Article Rating: 3.5
Published on December 11, 2001
Abstract
The Windows Operating System provides a lot of flexibility for programming against the various services offered by it. We can develop applications to manipulate the OS’s various services like the registry and EventViewer. In this article, Sandra Gopikrishna focuses on the following topics - writing to and reading from the system registry by employing the classes of the Microsoft.Win32 namespace, and interacting with the Windows 2000 event logs such as the application, security and system logs by employing the classes of the System.Diagnostics namespace.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

The Windows Operating System provides a lot of flexibility for programming against the various services offered by it. We can develop applications to manipulate the OS's various services like the registry and EventViewer. The .NET framework ships with a bunch of classes that can make the programming of system services a cakewalk for the developer. In this article, we focus on the following topics:

  • Writing to and reading from the system registry by employing the classes of the Microsoft.Win32 namespace.
  • Interacting with the Windows 2000 event logs such as the application, security and system logs by employing the classes of the System.Diagnostics namespace.

Software Requirements

The examples in this article are coded in C# and makes use of the following software:

  • Windows 2000.
  • .NET Framework SDK BETA 2.
  • Visual Studio.NET BETA 2 (Optional).

Registry and .NET

The Windows registry used to be the central repository for the COM Programmer. It houses a wide range of metadata ranging from user settings to registration details of COM Components. In the case of .NET, the use of the system registry is minimal. However for interoperating with COM/COM+ components, we need to rely on system registry. The process of accessing the registry is made simple and straightforward in .NET. It should be noted that in COM and COM+ the registry is accessed by employing the raw WIN32 API, which is pretty complex.

Interacting with System Registry

The .NET Framework includes the namespace Microsoft.Win32 that makes the process of registry manipulation a cakewalk for the developer. By making use of classes present in the Microsoft.Win32 namespace, the developer can do registry manipulations such as writing into it and reading from it on the fly.

Class Name | Functionality
Registry | This type encapsulates the windows registry and all its associated hives.
RegistryKey | This type allows the developer to insert, update and remove the information stored in the registry.
RegistryHive | This enumerates each of the hives present in the registry.

As evident from the above table the RegistryKey type of the Microsoft.Win32 namespace plays a key role in interacting with the system registry. The following table explores some of the important properties and methods of the RegistryKey type.

Property/Method | Functionality
Name(property) | This property retrieves the name of the key.
CreateSubKey(method) | This method creates a new subkey or opens up an existing one.
SetValue(method) | This method sets the specified value.
GetValue(method) | This method retrieves the specified value.
DeleteSubKey(method) | This method deletes the specified subkey. It should be noted for deleting the child subkeys we have to use another property called DeleteSubKeyTree.

Example Application

The following example application demonstrates interaction with the system registry, by making use of the RegistryKey class of the Microsoft.Win32 namespace .The goal of this example is to allow the user to save their application preferences such as Background color, font-family and font-color into the registry, to retrieve them when required, and finally to delete them when required. The application consists of a windows form with the following primary components laid over it:

RichTextBox Control

The text to be formatted is entered in the RichTextBox control present in the windows form.

ListBox Controls

The font-family, font-color and background color of the RichTextBox control is selected from the list boxes present in the form.

Menu and menu items

The application contains two menus: 'UserSettings' and 'Exit'.

The 'UserSettings' Menu has the following menu items present:

Load From Registry

This menu item contains the functionality for loading the font-family, font-color and Background color for the Rich textbox present in the Windows form

Save To registry

This menu item contains the functionality for saving the font -color, background color and font -family of text contained in the RichTextBox to the system registry.

Remove From registry

This menu item clears off the information pertaining to font color, font family and background color from the system registry.

Exit

The Exit menu contains the menu item 'Exit Application', which has the functionality for exiting from the application.

Building the Example

  • Open Visual Studio .NET.
  • Click the File| New | Project | Visual C# | Windows Application.
  • Give a suitable name for the project, like regmanipulate.
  • Clicking the next button creates a Windows form based project.
  • Layout the menus, RichTextBox and listbox controls on the windows form as shown in the following screenshot. Refer to the project present in the download for the properties of each control.

  • Right click on the form control and select the View code item from the resulting menu.
  • In the code, under the namespace section, import the namespace Microsoft.Win32 that is required for registry manipulation as indicated below:

//Required for manipulating the System Registry
using Microsoft.Win32;

  • The ListBox controls enable the user to change the font-color, font-family and text color of the text appearing in the RichTextBox control present in the form. The changes are reflected in the RichTextBox as and when the user indicates his choice through the listbox controls, which happens by triggering the SelectedIndexChanged event of the corresponding ListBox control.
  • The SelectedIndexChanged event of the listbox associated with the font-family contains the functionality for changing the font-family of the RichTextBox control. This is achieved by employing the following code block. The SelectedIndexChanged event is triggered whenever any item is selected from the listbox control.

private void
            cmbFontFamily_SelectedIndexChanged(object sender, System.EventArgs
            e)
{
//Set the Font Family property of the richtextbox control
            as
//indicated in the corresponding listbox
richTextBox1.Font = new Font(new
            FontFamily(cmbFontFamily.SelectedItem.ToString()),12);
}

In the above code we are instantiating a new font type with the family indicated by the user through the listbox associated with the font-family present in the form. The constructor of the font type takes 2 parameters; the first is the font family, and second is the font size (hard coded as 12 in this application). The font object thus created is then assigned to the 'Font' property of the RichTextBox control.

  • The SelectedIndexChanged event of the listbox associated with the background color contains the functionality for changing background color of the RichTextBox control. This is achieved by employing the following code block:

private void cmbBgColor_SelectedIndexChanged(object sender, System.EventArgs e)
{
//Set the BackColor property of the richtextbox control as
//indicated in the corresponding listbox
richTextBox1.BackColor = Color.FromName(cmbBgColor.Text);
}

In the above code the color type is created from the selection specified in the listbox control associated with Background color and by employing the FromName method of the type Color. This Color type instance is then assigned to the BackColor property of the RichTextBox control.

  • The SelectedIndexChanged event of the listbox associated with the Text color contains the functionality for changing the color of the text in the RichTextBox control. This is achieved by employing the following code block:

private void cmbTxtColor_SelectedIndexChanged(object sender, System.EventArgs e)
{
//set the richtextbox forecolor property as indicated in
//corresponding listbox
richTextBox1.ForeColor = Color.FromName(cmbTxtColor.Text );
}

In the above code an instance of color type is obtained by employing the FromName method of the Color class with the specified color in the listbox control associated with the Text color. This instance is then assigned to the ForeColor property of the RichTextBox control.

  • The UserSettings menu contains the functionality for writing into system registry, reading from it and deleting from the registry.
  • The Load From The Registry menu item of the UserSettings menu retrieves the font-color, font-family and background color to be applied to the RichTextBox control by calling the method loadFromReg().
  • In the loadFromReg method, a registrykey is opened under the CurrentUser section of the system registry employing the RegistryKey class of the Microsoft.Win32 Namespace as indicated by the following code block:

//Create a Registry key under CurrentUser section
RegistryKey myregkey = Registry.CurrentUser;

  • The subkey myappsettings is opened under the software section of the system registry. This is achieved by employing the OpenSubKey method of the RegistryKey type. The OpenSubKey method takes two parameters: the first one is the subkey to be opened, and the second parameter is a Boolean indicating whether the opened subkey is writeable or not. The following code block demonstrate this process:

//Open the key myappsettings under the software section of the registry
myregkey = myregkey.OpenSubKey("Software\\myappsettings", true);

  • Under the Subkey myappsettings the keys stfntfam, stbgcolor, stforecolor are searched and if they are present the values are retrieved from them and applied to the RichTextBox control as indicated by the following block of code. The keys stores the following information:
  • stfntfam stores the font-family.
  • stbgcolor stores the background color for the richtextbox control.
  • stforecolor stores the font color.
  • The values are retrieved from these keys by employing the GetValue method of the RegistryKey class. The GetValue method takes two parameters: the first one is the key name and the second parameter being the type of the key. The type of key is obtained by employing the GetType() method on the corresponding local variable that is used to store the setting retrieved from the system registry. The following block of code indicates this:

try
{
//Retrieve the font-family and apply to the richtextbox control
string storedfontfamily = "";
storedfontfamily = (string)myregkey.GetValue("stfntfam",storedfontfamily.GetType() );
richTextBox1.Font = new Font(new FontFamily(storedfontfamily),12);
//Retrieve the backgroundcolor and apply to the richtextbox control
string storedbgcolor = "";
storedbgcolor = (string)myregkey.GetValue("stbgcolor",storedbgcolor.GetType());
richTextBox1.BackColor = Color.FromName (storedbgcolor);
//Retrieve the fore color and apply to the richtextbox control
string storedforecolor = "";
storedforecolor = (string)myregkey.GetValue("stforecolor",storedforecolor.GetType());
richTextBox1.ForeColor = Color.FromName(storedforecolor);
}
catch(Exception e)
{
//Write the exception message to the richtextbox control
richTextBox1.Text = e.Message;
}

  • The Save To Registry menu item of the UserSettings menu saves the font-color, font-family and background color indicated in the corresponding listbox controls into the registry by making use of the function loadIntoReg().
  • The method loadIntoReg() uses the SetValue() method of the RegistryKey class and loads the values indicated in the listboxes to the registry. The SetValue method takes two parameters; the first one being the key name, and the second being the value to be set for the key. The following code block demonstrates this:

private void loadIntoReg()
{
//Create the registry key
RegistryKey myregkey = Registry.CurrentUser;
myregkey = myregkey.CreateSubKey("Software\\myappsettings");

//store the text color to the registry under the name 'stfntname'
myregkey.SetValue("stfntfam",cmbFontFamily.Text );

//store the background color to the registry under the name 'stbgcolor'
myregkey.SetValue("stbgcolor",cmbBgColor.Text );

//store the fore color to the registry under the name 'stforecolor'
myregkey.SetValue("stforecolor",cmbTxtColor.Text);

}

  • After the Save To The Registry menu item is clicked the user settings are entered into the registry as shown in the screen shot below

  • The menu item of the UserSettings menu removes the font-color, font-family and background color that are stored in the registry by making use of the method delFromReg().
  • The method delFromReg() employs the DeleteSubKeyTree() method of the RegistryKey class and removes all the stored settings under the key myappsettings from the registry. The method DeleteSubKeyTree() expects the key to be deleted to be passed on as a parameter to it.

public void delFromReg()
{
//Open the
            subkey to be deleted
RegistryKey myregkey =
            Registry.CurrentUser;
myregkey =
            myregkey.OpenSubKey("Software",true);

try
{
//Delete
            the subkey and all it's
            children
myregkey.DeleteSubKeyTree("myappsettings");
}
catch(Exception e)
{
//echo the exception message to
            richtextbox control
richTextBox1.Text =
            e.Message;
}
}

  • The Exit Application menu item of the Exit menu exits the application by employing the following piece of code.

Application.Exit() ;

This completes the first section of the article. In this section we have implemented the concepts of programmatically accessing the system registry in .NET by reading, writing and deleting user settings indicated in an application. The next section of the article focuses on manipulating the windows 2000 event log.

Manipulating the Windows 2000 Event Log

What are event logs?

Windows 2000 Event logs enable us to monitor the various events related to security ,system and application programs occurring in the operating system. The event log service is started automatically when Windows 2000 is started up. Because the event log is maintained by the system and can be accessed easily by Windows 2000's built in the event viewer application, it makes sense to use this from within our own applications to log events, rather than plain text files. Windows 2000 comes with three event logs:

System Log

The System log records all the events logged by the system components of the Windows 2000 operating system. The event type that needs to be logged is determined by the operating system. Example: When a device driver fails, the event is logged in the System Log.

Application Log

The Application log records events pertaining to application programs. The Application program decides the events that need to be monitored. Example: Failure of Database operations can be logged in the Application log.

Security Log

The Security log records all the events pertaining to security events, such logon attempts, and events related to resource use like creating, opening, or deleting files. Example: Invalid logon into a system makes an entry into security log.

Viewing event logs

Windows 2000 provides us with an application known as the event viewer, with which we can view all the event logs present in the system. The Event Viewer displays the information such as date, time, source, event type, category, Event ID, user account, and computer name for each log record available in the event log.

Manipulating event logs from the .NET Framework

The .NET Framework's System.Diagnostics namespace contains the classes necessary for manipulating event logs. It also provides the necessary classes for creating and manipulating our own customized event logs. The EventLog class of Systm.Diagnostics encapsulates all the functionality for interacting with the windows event logs. The following are some of the important methods of the EventLog class:

Method | Description
CreateEventSource | establishes the application as an event source.
GetEventLogs | creates an array of event logs.
WriteEntry | inserts an entry in the specified event log
Clear | clears the event log

The following are some of the important properties of the EventLog class:

Property | Description
Log | Specifies the log to be manipulated. It can take the values of 'Application' , 'System', 'Security' or any other name in case of customized logs.
Source | This specifies the application name to register and use when writing to event log.

Example Application

The following application demonstrates the method of manipulating the event logs of the windows 2000 employing the EventLog class of the System.Diagnostics namespace.

The application enables us to do the following things:

  • Read from and write to the application log
  • Read from the system log
  • Read from the security log
  • Read, write and delete a customized log created by the program

Components of the application

The application comprises of a windows form with the following controls laid over it:

RichTextBoxControl

The content to be written to the event log and retrieved from the event log is contained in the RichTextBox control. It also displays the error message in the case of any exception occurring in the application.

Menus with Menu Items

The application comprises of the following menus:

Application Log

This contains the menu items for reading and writing into application event log

Security Log

Contains the menu item for reading the security event log.

System Log

Contains the menu items for reading from the system event log.

Custom Log

Contains the menu items for creating, reading and deleting our own event log.

Exit

Contains the functionality for exiting from the application.

Building the application

  • Open Visual Studio .NET.
  • Click File | New | Project | Visual C# | Windows Application.
  • Call the project ManipEventLog.
  • Layout the various controls on the form as shown in the screenshot. Refer to the ManipEventLog project in the download section for looking into the individual properties for each of the control.

  • Right click on the form control and select View code.
  • In the code, under the namespace section, import the namespace System.Diagnostics:

//Required for manipulating the windows 2000 event logs
using System.Diagnostics;

Declare and define the EventLog class in the class Form1 generated by the Visual Studio .NET as indicated by the following:

//Declare and define EventLog type

public EventLog mylog = new EventLog();

  • Let's discuss the functionality of each of the menus, which contain the logic for interacting with the event logs.
  • The Application Log menu has the functionality for interacting with the system Application Log.
  • The menu item Read From Log invokes the method readfromlog, which retrieves and displays all the entries of the Application Log in the RichTextBox control. The following code demonstrates this:

private void menuItem2_Click(object sender, System.EventArgs e)
{
readfromlog("Application", "Text" );
}

  • The Readfromlog method takes two arguments; the first is log type, and the second is the source. For interacting with the Application Log, the log type should be specified as 'Application' and source should be specified as 'Text'. The readfromlog method is explained by the following code:

public void readfromlog(string logtype,
            string logsrc)
{
//Specify the Log type
mylog.Log =
            logtype;

//Specify the Log Source
mylog.Source =
            logsrc;

//Retrieve all the messages from the specified
            log
int i = 0;
for(i
            =0;i<mylog.Entries.Count;i++)
{
//display these messages in
            the RichTextbox control
richTextBox1.Text = richTextBox1.Text +
            "\r\n" + mylog.Entries[i].Message
            ;
}

if(mylog.Entries.Count == 0
            )
{
richTextBox1.Text = "No Entries
            Found";
}
}

In the above method, the Log and Source properties are specified for the log, and then all the entries present in the log are retrieved by the Entries property of the EventLog instance. Finally all the entries are displayed in the RichTextBox control of the form. In the case of no entries in the log an appropriate message is displayed in the RichTextBox control.

  • The menu item Write To Log invokes the method writetolog, which writes the content of the RichTextBox control to the Application Log, as shown below:

private void menuItem3_Click(object sender, System.EventArgs e)
{
writetolog("Application", "Text");
}

  • The writetolog method takes two arguments; the first being log type, and the second the source. For writing to the Application Log, the log type should be specified as 'Application' and source should be specified as 'Text'. The WriteEntry method of the eventlog has the following structure:

public static void WriteEntry(
string source,
string message,
EventLogEntryType type,
int eventID
);

In the above format, the third parameter EventLogEntryType refers to the nature of the log. It takes the following 5 values:

Value | Description
EventLogEntryType .Error | Indicates an error event.
EventLogEntryType .FailureAudit | Indicates a failure audit event.
EventLogEntryType .Information | Indicates an information event.
EventLogEntryType .SuccessAudit | Indicates a success audit event.
EventLogEntryType .Warning | Indicates a warning event.

If no value is specified for the parameter EventLogEntryType it defaults to Information type. In this article we assume the log entries to be of Information type.

  • The writetolog method is shown in the following code:

public void writetolog(string logtype, string logsrc)
{
//Specify the Log Type
mylog.Log = logtype;

//Specify the Log Source
mylog.Source = logsrc;

//Write to the specified Log
mylog.WriteEntry(richTextBox1.Text);

}

  • In the above method, the Log and Source property are specified for the log and then the content of the RichTextBox control is written into the application log by employing the WriteEntry() method of the EventLog class.
  • The following screenshot displays the entries that are made into the application log by our application:

  • The security and system logs are accessed in a similar way, except that we can only read from these logs.

For the security log:

private void menuItem5_Click(object sender, System.EventArgs e)
{
readfromlog("Security", "Security" );
}

For the system log:

private void menuItem8_Click(object sender, System.EventArgs e)
{
readfromlog("System","System");
}

the custom log is a little different as we have to create it.

  • The Custom Log menu contains the functionality for creating a customized event log and for performing read and delete operations on it.
  • The Write To Log menu item of the Custom Log menu, on clicking, creates an event log with the name mycustomlog and writes the content of the RichTextBox control into it. The following piece of code demonstrates this process.

   private void
            menuItem11_Click(object sender, System.EventArgs
            e)
{
mylog.Log =
            "mycustomlog";
if(!EventLog.SourceExists("myeventsource"))
{
EventLog.CreateEventSource("mycustomlog",
            "myeventsource");
}

//Create an EventLog instance and
            assign its source.
mylog.Source = "myeventsource";

//Write
            an informational entry to the event log.
mylog.WriteEntry(richTextBox1.Text );

}

  • In the above code the Log property of the EventLog instance is set to mycustomlog, then a source "myeventsource" is registered for "mycustomlog" using the CreateEventSource method of the EventLog class. The content of the RichTextBox control is written into the created event log 'mycustomlog' by employing the WriteEntry method of the EventLog class.
  • The following screen shot displays the event viewer application with the event log 'mycustomlog' created by our application.

  • The Read From Log menu item of the Custom Log menu reads all the entries of the event log mycustomlog created in the previous step and displays it in the RichTextBox control. The following code demonstrates this:

   private void
            menuItem13_Click(object sender, System.EventArgs e)
{
//Read
            all the items from mylog
mylog.Log = "mycustomlog";
int i =
            0;
for(i
            =0;i<mylog.Entries.Count;i++)
{
richTextBox1.Text =
            richTextBox1.Text + "\r\n" + mylog.Entries[i].Message
            ;
}
if(mylog.Entries.Count == 0 )
{
richTextBox1.Text =
            "No Entries Found";
}

}

  • In the above code, the Log property of the EventLog instance is set to mycustomlog, then all the entries in mycustomlog are retrieved using the Entries property of the EventLog class and written into the RichTextBox control present in the windows form.
  • The Clear Log menu item of the Custom Log menu clears all the entries in mycustomlog. The following code demonstrates this:

private void menuItem14_Click(object sender, System.EventArgs e)
{
mylog.Log = "mycustomlog";
mylog.Clear();
}

In the above code the Log property is specified as mycustomlog and Clear() method is invoked which clears all the entries from the mycustomlog.

  • Clicking the Exit Application menu item of the Exit menu exits the application by employing the following:

private void menuItem9_Click_1(object sender, System.EventArgs e)
{
Application.Exit();
}

The Exit() method of the Application class exits the application. This completes the explanation of the second part of the article.

Conclusion

The .NET Framework provides a robust and flexible architecture for interacting with the various system services. The COM programmer used to rely heavily on the raw WIN 32 API for accessing low-level system services like the registry. However with the advent of .NET framework there was a dramatic change in the approach - not only in the ease of use of manipulating the registry, but also in the way we can store configuration information in config.xml files. In .NET, all the functionality for accessing the low-level system services are exposed in the form of clean APIs, making the task of application development much simpler for the developer.

If you'd like to discuss a broader range of C#Today topics and issues, or if you have any C# related questions you need answers to, please visit our online resource centres at : http://p2p.wrox.com/list.asp?list=csharptoday_discuss&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq and http://p2p.wrox.com/list.asp?list=c_sharp&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq
C#Today - © 2002 WroxPress Limited


Online CPU Console using a Web Control Library with .NET Security

Online CPU Console using a Web Control Library with .NET Security
http://www.csharptoday.com/content.asp?id=1706
Article Rating: 4
Published on November 26, 2001
Abstract
Administering applications and servers when not connected to the network can be a nightmare, especially when only a few people manage the application. Just imagine going out for an evening on the town and then you’re paged at one o’clock in the morning with an alarming message that your application has gone down. For companies without Virtual Private Networks (VPN) the task is even more difficult because now you must come to the office to rescue your application. Microsoft’s .NET framework provides easily accessible assemblies to read and modify imperative information about any given machine, and most importantly it also provides backwards compatibility to read and modify machines, which are not .NET framework enabled. In this article, Eric Rockenbach shows us how to create a CPU Console administration tool and appropriate secuirty scheme for any machine on our network.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

When not connected to the network, administering applications and servers can be a nightmare, especially when only a few people manage the application. Just imagine going out for an evening on the town and then you're paged at one o'clock in the morning with an alarming message that your application has gone down. For companies without Virtual Private Networks (VPN) the task is even more difficult because now you must come to the office to rescue your application (VPN allows remote access of machines when both machines have the correct software installed). Microsoft's .NET framework provides easily accessible assemblies to read and modify imperative information about any given machine, and most importantly, it also provides backwards compatibility to read and modify machines which are not .NET framework enabled. In this article we see how to create a CPU Console administration tool and appropriate security scheme for any machine on our network.

System Requirements

  • Server: Windows 2000, IIS 5.0, .NET Framework, (Server Hosting the CPU Console application)
  • Machine to be checked: Windows 2000, NT
  • User's workstation: IE 5.0+, Admin Group of Machine, Admin Group of Server, Win 2000 Login account

Assumptions

This article is designed to combine different elements of the .NET framework to meet an important business challenge. We do not assume exposure to the assemblies mentioned, but having exposure to the .NET framework will be helpful, especially when expanding on this article's functionality.

Designing an Online CPU Console

Implementing a CPU Console over an entire network can be challenging - not only because of the cost of license fees for every machine, but also because the software must be installed on possible mission critical machines, in which domain administrators might not be allowed to install un-trusted software. In addition, the console must be open to the web, which means authentication and SSL on every machine, plus an open port, which is not a secure configuration to say the least. The challenge is to keep servers protected, yet still be able to administer these over the web for disconnected users.

The sample application we develop demonstrates how the .NET framework provides an elaborate library for administering events, processes, and services. .NET handles the majority of the low level code that developers write to view system resources. We utilize .NET's ability to use the installed framework on one machine to gain access and information about another machine that does not have the .NET framework installed.

One of the most impressive features of the .NET framework is its extensive robust security model. Many of .NET's classes now provide properties that we can "get" or "set" to administer runtime security. .NET also houses a complete assembly System.Security, which features nothing but security settings and preferences. Our CPU Console application will capitalize on proper and efficient usage of Windows 2000 role based security in conjunction with .NET's inherent impersonation based security. We will authenticate the user in IIS utilizing their Windows 2000 account and domain, but you will still have to encrypt data from your browser through conventional SSL methods.

Microsoft's ASP.NET Authenticated Impersonation Security Model.

A Web Control Library (a library of custom ASPX controls) is preferred because of its excellent reusability benefits.

Building an Online CPU Console

Users may have a variety of domains and machines that they need to connect to. Consider what features will integrate best over a web interface, given that time may be of the essence to users.

Our online CPU Console consists of two projects: a Web Control Library, which houses the core classes, and an ASP.NET Web Application, which calls the Web Control Library .dll. The Web Control Library contains three custom web control class files: services, processes and events. ASP.NET Web Application contains three web forms that call the custom web controls.

Let's take a look at the architecture of the code available for download.

WROXControlLib

Our Web control library contains ready-to-use custom web controls which in turn access events, processes, and services of any PC.

  • eventlog.cs - Custom Control displaying all events in any given log (application, system, or security), and replicates the Windows Event Viewer.
  • process.cs - Custom Control displaying all processes currently running on a machine, and also stops processes chosen by the client. Process.cs replicates the process panel of the task manager.
  • services.cs - Custom Control displaying all services installed on any machine. Similar to Windows Service Console, services.cs also allows a user to start and stop a service.

WROXCPUConsole

An ASP.NET Web Application that requests custom web controls in the WROXControlLib.dll. Included in this project are the security settings allowing impersonation after IIS user authentication.

  • vw_eventlog.aspx -requests the eventlog.cs custom control.
  • vw_process.aspx -requests the process.cs custom control.
  • vw_service.aspx -requests the services.cs custom control.
  • web.config - Web configuration file supports ASP.NET's Impersonation based identity and windows authentication.

WROXControlLib

Step one is to set up a Web Control Library. Open Visual Studio .NET and choose a Web Control Library project. Add three new Custom Web Controls to the project named evenlog.cs, process.cs, and services.cs. Add System.ServiceProcess as a reference to the project by right-clicking References in the Solutions Explorer.

eventlog.cs

Events are accessed through the System.Diagnostics assembly. Use this assembly by making a reference at the top of your class.

Write to the event log by calling EventLog.WriteEntry(). The WriteEntry() method provides many parameters such as the type of event (info, error, etc.), the source of the event (our assembly), and the message of the event.

Read from the event log by calling EventLog().Entries to obtain the collection of events from the specified machine. This method may be called with parameters such as log type (application, system, security), and machine name.

The code below requests the collection of events and loops through them with a for statement. Notice how the collection is called with a machine and log type, that is set by the client application. The collection returns the earliest event first by default; the log must be turned around to view the newest event first. Reverse the order by iterating through the collection backwards. Print the content out to the client application using the HTMLTextWriter. HTMLTextWriter is the default parameter passed into a custom control. Let's take a look at the code for eventlog.cs:

using System;
using System.Web.UI;
using System.Web.UI.WebControls;
using System.ComponentModel;
using System.Diagnostics; //Add This Reference

public class eventlog :
            System.Web.UI.WebControls.WebControl
{
private string logType
            = "Application";
private string machine = "";

//Machine is
            a Property that is Set by the Client
public string
            Machine
{
get
{
return
            machine;
}

set
{
machine =
            value;
}
}

protected override void
            Render(HtmlTextWriter doc)
{
//Call the Collection of
            Events
EventLogEntryCollection objEventCol = new
            EventLog(logType,machine).Entries;

//Loop Through the
            Collection of Events
for(int
            iCount<objEventCol.Count;iCount=objEventCol.Count; iCount>=0;
            iCount++)
{

//Write to the
            Document
doc.Write(objEventCol[i].EntryType.ToString() + " -
            ");
doc.Write(objEventCol[i].Source +
            "<br>");
doc.Write(objEventCol[i].Message +
            "<p>");
}
}
}

process.cs

Processes may be viewed and stopped using the System.Diagnostics assembly. After referencing the assembly at the top of the class file, call Process.GetProcesses() to retrieve the collection of running processes. Specify which machine these processes are viewed from by setting the machine name parameter.

Processes can be completely stopped by calling the Kill() method. A reference to each thread that a Process is running can also be obtained. Users can also be allowed to stop specific threads as an alternative to stopping the entire process, but this is an option to use with caution as the user may not be completely familiar with all the internal workings of that application.

The code below requests the collection of processes and loops through them with a foreach statement. Notice how the collection is called by a machine that is set by the client application. The contents are then printed out to the client application using the HTMLTextWriter. Let's take a look at the code for process.cs:

using System;
using System.Web.UI;
using System.Web.UI.WebControls;
using System.ComponentModel;
using System.Diagnostics; //Add This Reference

public class process :
            System.Web.UI.WebControls.WebControl
{
private string machine
            = "";
private string stopProcess = "";

//Machine is a
            Property that is Set by the Client
public string
            Machine
{
get
{
return
            machine;
}

set
{
machine =
            value;
}
}

protected override void
            Render(HtmlTextWriter doc)
{
//Loop Through the Collection of
            Processes
foreach(Process objP in
            Process.GetProcesses(machine))
{
//Write to the
            Document
doc.Write(objP.ProcessName + " -
            ");
doc.Write(objP.Threads.Count.ToString() + " -
            ");
doc.Write(objP.TotalProcessorTime.ToString() +
            "<br>");
}
}
}

Notice in the code that the Kill() method is called to completely stop a process while looping through the collection.

foreach(Process objP in
            Process.GetProcesses(machine))
{
if(objP.ProcessName.Trim() ==
            stopProcess.Trim())
{
//Stops the
            Process
objP.Kill();
}
}

service.cs

Services can be viewed, stopped, started, paused, and resumed using the System.ServiceProcess assembly. Reference the assembly at the top of the class file and call ServiceController.GetServices() to receive a collection of the services installed on any given machine.

Services may be started by calling the Start() method, and stopped by calling the Stop() method.

In the code below, we request the collection of services, and loop through them with a foreach statement. Notice how the collection is called with a machine that is set by the client application. We print the content out to the client application using the HTMLTextWriter. Let's take a look at the code for service.cs:

using System;
using
            System.Web.UI;
using System.Web.UI.WebControls;
using
            System.ComponentModel;
using System.ServiceProcess; //Add This
            Reference

public class services :
            System.Web.UI.WebControls.WebControl
{
private string machine
            = "";
private string stopService = "";
private string
            startService = "";

//Machine is a Property that is Set by the
            Client
public string Machine
{
get
{
return
            machine;
}

set
{
machine =
            value;
}
}

protected override void
            Render(HtmlTextWriter doc)
{
//Loop Through the Collection of
            Services
foreach(ServiceController objSP in
            ServiceController.GetServices(machine))
{
//Write to the
            Document
doc.Write(objSP.DisplayName + " -
            ");
doc.Write(objSP.Status.ToString() + " -
            ");
doc.Write(objSP.ServiceType.ToString() +
            "<br>");
}
}
}

Note that the Stop() method is called to stop a service, and the Start() method is called to start a service while looping through the collection.

foreach(ServiceController objSP in
            ServiceController.GetServices(machine))
{
if(objSP.DisplayName.Trim()
            == stopService.Trim())
{
//Stop a
            Service
objSP.Stop();
}

if(objSP.DisplayName.Trim() ==
            startService.Trim())
{
/Start a
            Service
objSP.Start();
}
}

WROXCPUConsole

Now that our Web Control Library is finished, we continue by building the web interface for this application. Open Visual studio and choose an ASP.NET Web Application project. Add the following three Forms: vw_EventLog.aspx, vw_Process.aspx, and vw_Service.aspx.

Note that the .aspx files call a register tag at the top of the document, containing the namespace, assembly name, and tag prefix of the WROXCPUControl.dll.

vw_EventLog.aspx

Call the event viewer control by the tag prefix colon and the class name of eventlog.cs. Note that we set a property from the query string in the code behind file.

Display vw_EventLog.aspx file

< !-- Register the WROXControlLib Assembly -- >
<%@ Register TagPrefix="CPU" NameSpace="WROXCPUCONSOLE.ControlLib" Assembly="WROXControlLib" %>

<HTML>
<BODY>
<!-Call the Event Log Custom Control -->
<CPU:EventLog id="eventlog1"
runat="server"
Machine="machine">
</CPU:EventLog>
</BODY>
<HTML>

Code Behind vw_EventLog.aspx.cs file

public class vw_EventLog :
            System.Web.UI.Page
{
//Event Log Custom Control
protected
            WROXCPUCONSOLE.ControlLib.eventlog eventlog1;

private void
            Page_Load(object sender, System.EventArgs e)
{
//Set the
            machine property from the Query String
eventlog1.Machine =
            Request.QueryString["machine"].ToString();
}
}

Event Log Application Available in the Code Download

vw_Process.aspx

Call the process panel control by the tag prefix colon and the class name of process.cs. Note that we set a property from the query string in the code behind file.

Display vw_Process.aspx file

< !-- Register the WROXControlLib Assembly -- >
<%@ Register TagPrefix="CPU" NameSpace="WROXCPUCONSOLE.ControlLib" Assembly="WROXControlLib" %>

<HTML>
<BODY>
<!-Call the Process Custom Control -->
<CPU:PROCESS id="processes"
runat="server"
Machine="machine">
</CPU:PROCESS>
</BODY>
<HTML>

Code Behind vw_Process.aspx.cs file :

public class vw_Process :
            System.Web.UI.Page
{
//Process Custom Control
protected
            WROXCPUCONSOLE.ControlLib.process processes;

private void
            Page_Load(object sender, System.EventArgs e)
{
//Set the
            machine property from the Query String
processes.Machine =
            Request.QueryString["machine"].ToString();
}
}

Process Panel Application Available in the Code Download

vw_Service.aspx

Call the service panel control by the tag prefix colon and the class name of services.cs. Note that we set a property from the query string in the code behind file.

Display .aspx file

< !-- Register the WROXControlLib Assembly -- >
<%@ Register TagPrefix="CPU" NameSpace="WROXCPUCONSOLE.ControlLib" Assembly="WROXControlLib" %>

<HTML>
<BODY>
<!-Call the Service Custom Control -->
<CPU:SERVICES id="service"
runat="server"
Machine="machine">
</CPU:SERVICES>
</BODY>
<HTML>

Code Behind .aspx.cs file:

public class vw_Service :
            System.Web.UI.Page
{
//Service Custom Control
protected
            WROXCPUCONSOLE.ControlLib.services service;

private void
            Page_Load(object sender, System.EventArgs e)
{
//Set the
            machine property from the Query String
service.Machine =
            Request.QueryString["machine"].ToString();
}
}

Service Panel Application Available in the Code Download

Web.config

We tie the CPU Console together using the web.config file; here security is set for the entire application and any assemblies this application will execute. We impersonate the client's identity to call the assemblies, which in turn uses the client's identity to call the event log, processes, and services on any machine the user has access rights to. By setting the authentication type to Windows, ASP.NET will automatically retrieve the username and password when the user logs in. Web.config file should contain the following elements.

<configuration>
<system.web>
<authentication mode="Windows" />
<identity impersonate="true" />
</system.web>
</configuration>

IIS Configuration

The final step is to configure IIS. In order to configure IIS correctly, and separate the release version and development versions, build the release version of our ASP.NET Web Application in the Program Files folder instead of the Inetpub folder.

Open Internet Services Manager, right-click on the default web site. Choose New | Virtual Directory. Click Next, and enter an alias, such as CPUConsoleApp (do not choose the same name of the ASP.NET Web Application). Browse to the ASP.NET Web Application that was moved out of inetpub\wwwroot. Click finish to complete the setup.

The new Virtual Directory will now be viewable; right-click and select properties. Click Directory Security | Edit. Uncheck all the boxes except for Basic Authentication, and click the Edit button besides Basic Authentication. Type a backslash ("\ ") in the Domain text area. The backslash allows users of the console to control multiple domains. Click OK on each of the three forms open to complete the configuration. Stop and start IIS to ensure proper configuration settings before viewing the application. Users must be in the group of administrators on the server and the machine they connect to.

IIS Authentication Configuration

The Web Application will be viewable by opening Internet Explorer and typing the following URL into the address bar.

http://localhost/VirtualDirectory/ASP/vw_Default.aspx?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq

If IIS is configured correctly, there will be a prompt to log in before entering the web application. Use a Windows 2000 Account Domain\UserName for the User Name input, and the password of the account.

Login Prompt on Entrance of the Web Application

Conclusion

Our completed online CPU Console is a multi-tiered, robust, reusable application that is easily modifiable, and serves an important business purpose. In the downloadable code, connect to the vw_default.aspx web form in the ASP folder to allow the user to select a machine name.

By using the Online CPU Console, one can easily administer services, events and processes of any machine, even those machines that are not .NET framework enabled.

C#Today - © 2002 WroxPress Limited


Serialization

Serialization
http://www.csharptoday.com/content.asp?id=1576
Article Rating: 3.5
Published on September 17, 2001
Abstract
One of the most fundamental jobs for a piece of software is saving a document and loading it back up again. This process has become known as “serialization? and .NET has strong support for this. In this article, Matthew Reynolds takes a look at how to save and load documents for a simple WinForms application. As part of the work, he introduces a reference implementation for properly handling the creating, saving and opening of documents as laid down by the Windows guidelines.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

One of the most fundamental jobs for a piece of software is saving a document (the application's / user's data) and loading it back again at a later time. The software that doesn't need to do this is relatively rare. This process has become known as "serialization", and .NET has strong support for this.

In this article we'll be taking a look at how to save and load documents for a simple WinForms application. (These techniques will work equally well for ASP.NET sites and other applications that don't use WinForms, though). As part of the work, I'll introduce a reference implementation for properly handling the creating, saving and opening of documents as laid down by the Windows guidelines.

Our Document

In this exercise, we'll build a basic text document containing the text, the title of the work, and the author's name. We'll then demonstrate how we can save this information out to a file on the disk. First though, we need to build our client application.

Using Visual Studio, create a new Visual C# - Windows Application project. Call it Serialization Demo.

When the Designer for Form1 appears, add three TextBox controls.

  • Set the Name property of the first control to textTitle. Set the Anchor property to Top, Left, Right . Clear the Text property.
  • Set the Name property of the second control to textAuthor. Set the Anchor property to Top , Left, Right . Clear the Text property.
  • Set the Name property of the third control to textText. Set the Anchor property to Top , Bottom, Left, Right . Clear the Text property. Set the MultiLine property to True and the ScrollBars property to Vertical.

We'll need a basic menu for our application. Draw a new MainMenu control onto the form. Use the Menu Designer to add a top-level File option, and New, Open, Save, Save As and Exit options. Change the Name property of each option to menuFileNew, menuFileOpen, and so on.

Using the Properties window, bind the Menu property of Form1 to the MainMenu control you just created.

We'll store the contents of the document in an object of class Document. As well as having properties to store the title, author and text of the document, this class will also need properties to store the filename, and whether or not the document has changed since it was created or opened.

Using Solution Explorer, create a new class called Document. Add these members:

   public class
            Document
{
// members...
public String Filename;
public
            Boolean Dirty;
private String _title;
private String
            _author;
private String _text;

The Dirty member will be used to keep track of whether or not the document needs saving. Whenever the Title, Author or Text properties are changed, we'll set this to True. That way, when it's time to close the application, or the user attempts to create a new document or open an existing document, we know if we're supposed to prompt them to make sure they don't inadvertently lose data.

Add these properties:

   public String
            Title
{
get
{
return
            _title;
}
set
{
if(_title != value)
{
_title =
            value;
Dirty = true;
}
}
}

public String
            Author
{
get
{
return
            _author;
}
set
{
if(_author != value)
{
_author =
            value;
Dirty = true;
}
}
}

public String
            Text
{
get
{
return _text;
}
set
{
if(_text
            != value)
{
_text = value;
Dirty =
            true;
}
}
}

When the user asks to save the document, we need to determine if they need to provide a new filename. If the Filename member is empty, we are "untitled". Add this method to tell the caller whether or not the document is untitled:

   public Boolean
            IsUntitled()
{
if(Filename == null)
return
            true;
else
return false;
}

Let's now add code to Form1 that allows us to create new documents. We'll create a new document whenever the application starts. This is an expected function of desktop software - most Microsoft Office applications, for example, will create a new document when they start. We'll need to add a member to Form1 that will hold the current document object. Open the code editor for Form1 and add the _document member.

public class Form1 : System.Windows.Forms.Form
{
// members...
private Document _document;

Then, add this method that will create a new document.

   // CreateNewDocument -
            create a new document...
public void
            CreateNewDocument()
{
// check save...
if(CheckSave() ==
            false)
return;

// create a new document...
Document =
            new Document();
Document.Dirty =
            false;
UpdateCaption();
}

The CheckSave method is important for our reference implementation, and we'll implement this fully later. For now, add this stub implementation:

   // CheckSave - prompt the
            user to save the changes...
public Boolean
            CheckSave()
{
return true;
}

Whenever the underlying document object changes, we'll need to update the view. Add this property to Form1.

   // Document
            property...
public Document Document
{
get
{
return
            _document;
}
set
{
// set it...
_document =
            value;

//
            update...
UpdateCaption();
UpdateView();
}
}

UpdateCaption will be responsible for updating the application caption to show the filename of the document, and to indicate whether or not the document needs saving. When the document does need saving, an asterisk will appear next to the filename. This is how Visual Studio.NET and previous versions indicated that a file needed saving, but you should be aware that it's very unusual to find productivity software, like Microsoft Office, that uses this notation. Usually, the user is not informed visually when a document has entered a "dirty" state.

   // UpdateCaption - update
            the view...
public void UpdateCaption()
{
// clear the
            caption...
String newCaption = "";

// do we have a
            document?
if(Document != null)
{
//
            untitled?
if(Document.IsUntitled() == true)
newCaption =
            "Untitled";
else
newCaption = new
            System.IO.FileInfo(Document.Filename).Name;

//
            dirty?
if(Document.Dirty == true)
newCaption += "
            *";

// space...
newCaption += " - ";
}

// add
            the application name...
newCaption += "Serialization
            Demo";

// changed?
if(this.Text !=
            newCaption)
this.Text = newCaption;
}

UpdateView will be responsible for updating the text boxes on the form to reflect the document that has either been created or been opened. Add this method:

   // UpdateView - update the
            document after load/new...
public void UpdateView()
{
//
            clear...
textTitle.Text = "";
textAuthor.Text =
            "";
textText.Text = "";

// do we have a
            document?
if(Document != null)
{
textTitle.Text =
            Document.Title;
textAuthor.Text =
            Document.Author;
textText.Text =
            Document.Text;
}
}

When the Load method on the form is fired, we need to call CreateNewDocument. Flip over to the Designer for Form1 and double-click on the background of the form. Add this code:

   private void
            Form1_Load(object sender, System.EventArgs e)
{
// create a
            document...
CreateNewDocument();
}

For neatness, we should wire up the New menu option. Flip over to the Designer and double-click on the New menu option on the form. Add this code:

   private void
            menuFileNew_Click(object sender, System.EventArgs e)
{
//
            create...
CreateNewDocument();
}

Flip over to the designer and double-click on textTitle. Add this code:

   private void
            textTitle_TextChanged(object sender, System.EventArgs e)
{
//
            update...
if(Document != null)
{
Document.Title =
            textTitle.Text;
UpdateCaption();
}
}

Do the same with textAuthor:

   private void
            textAuthor_TextChanged(object sender, System.EventArgs e)
{
//
            update... if(Document != null)
{
Document.Author =
            textAuthor.Text;
UpdateCaption();
}
}

And also with textText:

   private void
            textText_TextChanged(object sender, System.EventArgs e)
{
//
            update...
if(Document != null)
{
Document.Text =
            textText.Text;
UpdateCaption();
}
}

If you run the project and add details to the document, the caption will update itself. Behind the scenes, the Document object will be updated.

Saving the Document

Now that we can store the document in memory and allow the user to make changes to it, we can turn our attention to saving the document.

Flip over to the Designer and double-click on the Save menu option on the form. Add this code:

   private void
            menuFileSave_Click(object sender, System.EventArgs e)
{
//
            save the document...
SaveDocument();
}

Once more, flip back to the Designer and double-click on the Save As menu option. Add this code:

   private void
            menuFileSaveAs_Click(object sender, System.EventArgs e)
{
//
            save it...
SaveDocumentAs();
}

Now, create the SaveDocument method. We'll call this whenever we need to save the document, but it will have the intelligence to determine if the document is untitled and, if it is, it will prompt the user for a filename:

   // SaveDocument - save the
            document to disk...
public Boolean SaveDocument()
{
// do
            we have a document?
if(Document == null)
return
            true;

// if the document
            untitled?
if(Document.IsUntitled() == true)
return
            SaveDocumentAs();

// save the
            document...
Document.Save();

// update the
            caption...
UpdateCaption();

// return...
return
            true;
}

SaveDocumentAs will need a SaveFileDialog control available. Flip over to the Designer and add a new SaveFileDialog control. Change its Name property to dialogSaveFile.

Open the code editor again and at the top of the class definition add the FileDialogFilter constant:

public class Form1 : System.Windows.Forms.Form
{
// members...
private Document _document;

// const...
private String FileDialogFilter =
document Files (*.myDoc)|*.myDoc|All Files (*.*)|*.*||";

Now add this method:

   // SaveDocumentAs - prompt
            for a filename and then save...
public Boolean
            SaveDocumentAs()
{
// do we have a document?
if(Document ==
            null)
return true;

// show the
            dialog...
dialogSaveFile.Filter =
            FileDialogFilter;
if(dialogSaveFile.ShowDialog() ==
            DialogResult.Cancel)
return false;

// set the
            filename...
Document.Filename =
            dialogSaveFile.FileName;
Document.Save();

//
            update...
UpdateCaption();

// return...
return
            true;
}

Serializing Documents

Of course, we need to implement the Save method and this is where things start to get interesting.

Serialization in .NET is done using the System.Runtime.Serialization namespace. Specifically, this namespace contains an interface called IFormatter. This interface contains methods called Serialize and Deserialize that can be used to save data to, and load data from, a stream. In our case, we're going to use a file stream, but there's no reason why you couldn't use a network stream, a memory stream, or a stream of your own devising.

To serialize the data, we're going to be using .NET's "binary formatter". This is implemented in the astonishingly named System.Runtime.Serialization.Formatters.Binary.BinaryFormatter class. This formatter takes whatever data you pass in and turns it into binary format. This binary format matches how the data is actually stored in memory. For string data this looks pretty sensible - send the string Disraeli and you'll see the word Disraeli stored in the target file. However, strings are very much the exception to the rule. Numeric values are stored in the computer's memory as arrays of bits. An int is stored as 32-bits, or four bytes. So, if you look in the file, rather than seeing 18000 you'll see four characters that when converted, turn into the binary representation of 18000.

Binary formatting is great because it is very small and very fast to save and load, as rather than having to convert from an internal storage format to an external storage format, the way the data is stored in memory can be used directly. The downside to this is that it becomes hard to port the data to another platform, but for the vast majority of you this is never going to be a problem.

To handle serialization, we have to implement System.Runtime.Serialization.ISerializable. This interface defines a single method called GetObjectData. The formatter will call this method whenever it requires the object to serialize its data to the stream. We also have to add the Serializable attribute to the class, otherwise the formatter won't even attempt to work with it.

At the top of the code listing for Document, add these namespace import declarations:

using System;
using System.IO;
using System.Runtime.Serialization;
using System.Runtime.Serialization.Formatters.Binary;

Next, make Document implement ISerializable, and add the Serializable attribute, like this:

[Serializable]
public class Document : ISerializable
{

As we mentioned before, when we tell Document that it implements this interface we must add an implementation for GetObjectData. Add this code:

// GetObjectData - serialize our data...
protected void GetObjectData(SerializationInfo info,
StreamingContext context)
{
// save the data...
info.AddValue("Title", Title);
info.AddValue("Author", Author);
info.AddValue("Text", Text);
}

In this instance, info provides the method by which we supply information to the formatter. The AddValue method takes pretty much any value you could want, but it does require a name. This name will be used when the data is de-serialized when the document is loaded again.

When building Save and Load methods, it's a good idea to create them in a way that makes it easy for developers who use your classes to overload them. Here's the first version, which takes no parameters:

   public void
            Save()
{
Save(Filename);
}

Here's the second, which takes a filename. We delete the file if it already exists, and create a new FileStream object:

   public void Save(String
            filename)
{
// delete the file...
FileInfo info = new
            FileInfo(filename);
if(info.Exists())
info.Delete();

//
            create a stream...
FileStream stream = new FileStream(filename,
            FileMode.Create);
Save(stream);
stream.Close();
}

The third version takes a stream and creates a new BinaryFormatter object:

public void Save(Stream stream)
{
// create a formatter...
BinaryFormatter formatter = new BinaryFormatter();
Save(stream, formatter);
}

The final version takes both a stream, and an object that implements IFormatter:

public void Save(Stream stream, IFormatter formatter)
{
// save it...
formatter.Serialize(stream, this);

// flag it...
Dirty = false;
}

Run the project and you should be able to save the document by using the File - Save menu option. The caption will reflect the name of the document when you save it.

If you look in the folder, you'll now be able to see a doc1.myDoc file. If you double-click on the file you'll be prompted for an application to open the document with. Select Microsoft Visual Studio.NET 7.0. Visual Studio's binary file editor will present the data to you. Although it will appear mangled, you can see the text of the document embedded into the file:

Loading the Document

Loading the document, or de-serialization, is just as easy. Once you've created a formatter, you just call Deserialize. However, rather than calling an equivalent of GetObjectData, the formatter will attempt to call a special de-serialization constructor that you need to implement. This constructor should be implemented as "protected" to prevent callers from inadvertently using it.

If the project is running, close it. Open the Designer for Form1. Draw on a new OpenFileDialog control and change its Name property to dialogOpenFile.

Next, double-click on the Open menu option on the Form. Add this code:

   // OpenDocument - open a
            document...
public void OpenDocument()
{
// check
            save...
if(CheckSave() == false)
return;

// show the
            dialog...
dialogOpenFile.Filter =
            FileDialogFilter;
if(dialogOpenFile.ShowDialog() ==
            DialogResult.Cancel)
return;

// open it...
Document =
            Document.Load(dialogOpenFile.FileName);
}

Load methods are best implemented as static methods, as it provides a natural way to instantiate an instance of the class while keeping the name descriptive. For example, if we created a constructor that took a filename, we don't know from the name if the class intends to de-serialize the data at this point, or if it's just altering its internal state to hold the filename. By using an explicit "Load" method, the developer has more of an idea what is going on. In this case, we'll provide a filename and expect to get a Document object back in return. In fact, we're going to have a couple of versions of this method just as we did with Save.

Open the code editor for Document. Add this method:

   // Load - load the
            document...
public static Document Load(String
            filename)
{
// create a stream and load it...
FileStream
            stream = new FileStream(filename, FileMode.Open);
Document
            newDocument = Load(stream);
stream.Close();

// set the
            filename...
newDocument.Filename = filename;

//
            return...
return newDocument;
}

Then add this version:

   public static Document
            Load(Stream stream)
{
// create a formatter and
            load...
BinaryFormatter formatter = new
            BinaryFormatter();
return Load(stream,
            formatter);
}

And also add this version:

   public static Document
            Load(Stream stream, IFormatter formatter)
{
// return
            it...
return
            (Document)formatter.Deserialize(stream);
}

As we mentioned before, the de-serialization process relies on a protected constructor that does the reverse of GetObjectData. Add this code:

protected Document(SerializationInfo info, StreamingContext context)
{
// load it...
_title = info.GetString("Title");
_author = info.GetString("Author");
_text = info.GetString("Text");
}

You can see from that code how simple de-serialization is. If you need to de-serialize other data types, there are plenty of Get methods. If the type you want is not directly supported, use GetValue, and cast the return value to whatever data type you need.

Run the project. You should be able to load the document back into the editor.

Implementing CheckSave

At this point you should have an understanding of how to save and load document data using the serialization features in .NET. To complete this application, we need to implement CheckSave.

You might have guessed that CheckSave is a method that we call whenever we need to save the document. If the method returns True, the user has either saved the changes or explicitly discarded the changes. If the method returns False, whatever process is being called (either "new" or "open") should be cancelled.

Here's the code for CheckSave:

   // CheckSave - prompt the
            user to save the changes...
public Boolean CheckSave()
{
//
            do we have a document?
if(Document == null)
return
            true;

// do we need to save it?
if(Document.Dirty ==
            false)
return true;

// save changes?
DialogResult
            result = MessageBox.Show("Do you want to save changes to
this
            document?", "Serialization Demo",
            MessageBoxButtons.YesNoCancel,
MessageBoxIcon.Question);

//
            cancel?
if(result == DialogResult.Cancel)
return
            false;

// save?
if(result == DialogResult.Yes)
return
            SaveDocument();

// return...
return true;
}

So, if the document is not dirty, we say, "OK, continue". If the document is dirty, we present a message box. The user can select Yes, No or Cancel. Cancel is self-explanatory, and No will cause the method to return, signaling that whatever called it should proceed.

On the other hand, if Yes was selected, SaveDocument will be called. If the document is untitled we'll present the Save File dialog. At this point the user could click Cancel, and so we have to cascade the False back to whoever called CheckSave. Looking at the code for SaveDocumentAs, you can see this in action:

   // SaveDocumentAs - prompt
            for a filename and then save...
public Boolean
            SaveDocumentAs()
{
// do we have a document?
if(Document ==
            null)
return true;

// show the
            dialog...
dialogSaveFile.Filter =
            FileDialogFilter;
if(dialogSaveFile.ShowDialog() ==
            DialogResult.Cancel)
return false;

// set the
            filename...
Document.Filename =
            dialogSaveFile.FileName;
Document.Save();

//
            update...
UpdateCaption();

// return...
return
            true;
}

CheckSave needs to be called when the application is being closed, primarily to allow the user a chance to cancel the close or to save the changes. Add this method:

   private void
            Form_Closing(object sender, CancelEventArgs e)
{
// check
            save...
if(CheckSave() == false)
e.Cancel =
            true;
}

Also, wire in the event in the constructor for Form1:

   public
            Form1()
{
//
// Required for Windows Form Designer
            support
//
InitializeComponent();

//
            cancel...
this.Closing += new
            CancelEventHandler(Form_Closing);
}

Now if you run the project and attempt to close the application without saving changes, you'll be prompted:

Saving in Simple Object Access Protocol ("SOAP")

In this article, we only examined the use of BinaryFormatter. .NET also comes supplied with SoapFormatter, which you can find in the separate System.Runtime.Serialization.dll assembly.

Serialization is heavily related to .NET Remoting, which in turn is heavily related to Web services. Whenever a client has to send an object to a Web service, it in fact uses the same serialization paradigm that we've seen here, but uses the SoapFormatter. This formatter is, obviously, capable of serializing the data to SOAP envelopes rather than into a binary stream as we've seen.

Summary

In this article we took a look at how the serialization functionality in .NET can be used to create an application capable of saving and loading its own document to a stream. We also presented a reference implementation for handling the user interface on document-centric applications, i.e. we prompted to save changes at the correct point, provided appropriate menu items, presented the filename in the window caption and so on.

C#Today - © 2002 WroxPress Limited


Performance Monitoring

Performance Monitoring
http://www.csharptoday.com/content.asp?id=1627
Article Rating: 4
Published on August 15, 2001
Abstract
Since the original version of Windows NT, system software such as IIS, SQL Server, MSMQ and others have reported runtime information through the “Performance Monitor? This utility gives system administrators a quick view of the current utilization of the system, from things like the amount of processor power and memory being used to the number of database requests each second. Creating Perfmon counters in your own applications shows system administrators what your application is doing and can also provide some pretty useful business analysis data too. For example, on a Web site built with ASP.NET, you can create a counter that reports the number of orders placed each minute, or the number of times a particular page has been hit. For server applications, you can report things like the number of clients currently connected or the amount of time spent processing particular units of work. In this article, Matthew Reynolds looks at how performance counters work in .NET and creates a reusable library that makes using performance counters a little easier.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

Since the original version of Windows NT, system software such as IIS, SQL Server, MSMQ and others have reported runtime information through the "Performance Monitor". This utility gives system administrators a quick view of the current utilization of the system, from things like the amount of processor power and memory being used to the number of database requests each second.

The Performance Monitor (also known as "Perfmon") works by using "counters". An application can register counters (such as "number of connected clients", "time spent processing work items") and can update the values in these counters as it runs. The system administrator can then connect to the Perfmon application and display one or more of these counters on a graph.

When you're using Perfmon, you might find the "Highlight" function useful. This highlights the selected counter making it stand out from the rest. Click the "lightbulb" on the toolbar to turn it on.

Creating Perfmon counters in your own applications shows system administrators what your application is doing and can also provide some pretty useful business analysis data too. For example, on a Web site built with ASP.NET, you can create a counter that reports the number of orders placed each minute, or the number of times a particular page has been hit. For server applications, you can report things like the number of clients currently connected or the amount of time spent processing particular units of work.

In this article, we're going to take a look at how performance counters work in .NET and create a reusable library that makes using performance counters a little easier. This code was created with .NET Beta 2 Build 9254.

The Sample Application

As building a service application just for this example is a little complicated, we're going to build a C# desktop application that simulates up to 32 clients connecting and doing some work.

To get started, create a new Visual C# | Windows Application project and call it PerfMon Demo.

When the Form Designer form Form1 appears, draw on a TrackBar control and a Label control like this. Set the Name property of the TrackBar control to trackNumClients and the Name property of the Label control to labelNumClients.

Next, set the Maximum property of the TrackBar control to 32 and change the Value property to 1.

You'll also need to add a Timer control to the form. Call it timerUpdateClientCount and set its Interval property to 250.

Using Solution Explorer, create a new class called Client. Include the System.Threading namespace by adding this declaration to the top of the code listing:

using System;
using System.Threading;

What we're going to do is configure Client so that it creates and manages the lifetime of its own thread. These threads will periodically "wake up" and so some work, but most of the time it will be sleeping. This tends to be a highly encapsulated method for working with threads and makes it easy for consumers of the threads to use. Add these members to the definition of Client:

   public class
            Client
{
// members...
private Thread _thread;
private
            Boolean _isCancelled;

public
            Client()
{
}
}

Next, add this method to Client:

      //
            ThreadEntryPoint - the thread starts here...
private void
            ThreadEntryPoint()
{
// start a loop...
ManualResetEvent
            pauseEvent = new ManualResetEvent(false);
while(true)
{
//
            sleep...
pauseEvent.WaitOne(250);

// are we
            cancelled?
if(_isCancelled == true)
break;

// pretend
            to do some work...

}
}

Next, add this method that will start the thread:

      //
            SpinUp - start the thread...
public void SpinUp()
{
// are
            we running?
if(_thread == null)
{
// create a new
            thread...
ThreadStart threadStart =
new
            ThreadStart(this.ThreadEntryPoint);
_thread = new
            Thread(threadStart);

// run the
            thread...
_thread.Start();
}
}

Then, add this method that will stop the thread when needed:

      //
            SpinDown - stop the thread...
public void SpinDown()
{
//
            are we running?
if(_thread != null)
{
// cancel
            it...
_isCancelled = true;

//
            wait...
_thread.Join();

// delete...
_thread =
            null;
}
}

Those three methods will handle the lifetime of the client. Basically, the thread will sleep for quarter of a second, wake up, go back to sleep for quarter of a second and so on until _isCancelled eventually gets set to True. At this point the thread will drop out of the infinite loop and will end.

To manage the threads themselves, we need to hold a list of them in Form1. The best way to hold the list is to use a collection of some sort and, for simplicity we'll use a System.ArrayList. Open the code editor for Form1 and add this member:

public class Form1 : System.Windows.Forms.Form
{
// members...
private ArrayList _clients = new ArrayList();

The principle behind our UI is that we can adjust the slider up or down to change the number of running clients. When this happens, we'll need a method the can either add new clients or stop existing ones from running. As you know, each client uses a single thread so this will have the effect of increasing and decreasing the number of running threads. Add this method to Form1:

// AdjustClientCount - set the number of running clients...
public void AdjustClientCount(int newCount)
{
// set a wait cursor...
Cursor = Cursors.WaitCursor;

As the process of stopping clients is not instantaneous, to create a good user experience it's best to display an hourglass cursor when we're going this.

The first thing we check for is to see if we've asked for more clients than we already have:

// do we need to go up or down?
if(newCount > _clients.Count)
{

If we have we create a For loop that counts from the current number of clients to one less than the number required. Per iteration, we create a new client, call SpinUp to tell is to create the thread and add it to the ArrayList.

            //
            add new clients for as long as it takes...
for(int n =
            _clients.Count; n < newCount; n++)
{
// create
            one...
Client newClient = new
            Client();
newClient.SpinUp();
_clients.Add(newClient);
}
}
else
{

Alternatively, we might have asked for less clients than we have at the moment, in which case we do the opposite, i.e. -we want to count back from the number we do have until we reach the number that we want.

            if(newCount
            < _clients.Count)
{
// remove clients from the
            list...
for(int n = _clients.Count - 1; n >= newCount;
            n--)
{
// get the client and stop it...
Client stopClient =
            (Client)_clients[n];
stopClient.SpinDown();

// remove
            it...
_clients.Remove(stopClient);
}
}
}

Once we've adjusted the list, we update the label control to report back to the user just how many clients we have running and set the cursor back to normal.

// report the number...
labelNumClients.Text = _clients.Count.ToString();

// reset the cursor...
Cursor = Cursors.Default;
}

When the application starts, we want to set the number of clients to 1, so add this code to Form1:

private void Form1_Load(object sender, System.EventArgs e)
{
AdjustClientCount(1);
}

And, as we know, when the slider is moved we want to move the number of clients too. Flip back to the Form Designer and double-click on the slider. Add this code to the new Scroll event handler:

private void trackNumClients_Scroll(object sender, System.EventArgs e)
{
// reset the timer...
timerUpdateClientCount.Enabled = false;
timerUpdateClientCount.Enabled = true;
}

The reason why we toggle the Enabled property to False and then back to True is that this will have the effect of resetting the timer so that it will fire quarter of a second after the user has stopped sliding the scroll bar. This will make the UI work more smoothly as it reduces the tendency for the application to "stall" while the client count is being adjusted.

Once more, flip back to the Form Designer and double-click on the Timer control. Add this code:

private void timerUpdateClientCount_Tick(object sender, System.EventArgs e)
{
// disable the timer...
timerUpdateClientCount.Enabled = false;

// adjust the clients...
AdjustClientCount(trackNumClients.Value);
}

Finally, unless we properly close all of our threads, our application will not stop when we close the main window. Add this code:

       private
            void Form1_Closed(object sender, System.EventArgs e)
{
// go
            through...
foreach(Client client in
            _clients)
client.SpinDown();
}

Obviously, this method will not be called unless we hook in the event handler. Expand the Windows Form Designer generated code region and find the InitializeComponent method. Add a Closed event handler like this: (I've omitted some code here for brevity).

         this.Name
            = "Form1";
this.StartPosition =
            System.Windows.Forms.FormStartPosition.CenterScreen;
this.Text =
            "PerfMon Demo";
this.Load += new
            System.EventHandler(this.Form1_Load);
this.Closed += new
            System.EventHandler(this.Form1_Closed);

You can try running the application at this point, but it won't do much as we haven't looked at integrating our application with Perfmon yet! What we need to do now is turn our attention to physically creating the performance counters and writing code that will allow us to look them up.

Creating Performance Counters

Using the Framework to create performance counters is pretty straightforward. However, in this section I will be providing a set of classes for managing performance counting in your own application that's a little easier to use than the basic classes included in the Framework.

We're in a position where we have a number of threads that all need to update the same counters. (In the first test run of the application, we're going to have a single counter called NumClients that reports the number of Client objects running in a thread). What we'll do is create a class called MyCounters that has a read-only property called NumClients and we'll use that property to tell Perfmon when a new client starts or when an existing client stops.

In order to let Perfmon know what counters are available to system administrators using the Perfmon application, we have to register each counter. We can either do this each time the application starts or, more reasonably, the first time the application runs after it has been installed.

.NET allows you to group your counters into "categories". The Perfmon application in Windows 2000 confusingly calls these categories "objects" so I can only assume in later version of Windows this will be changed in line with the .NET naming convention.

To run Perfmon, click the Start button and select Programs | Administrative Tools | Performance. Alternatively, select Start | Run, enter Perfmon and click OK. Both have the same effect.

When Perfmon starts, click the gray "plus sign" icon on the toolbar. This lets you add new counters to the graph. The Performance object list (shown here as .NET CLR Jit) is the category.

When we add our new categories, our category will appear in this Performance object list.

Creating our Classes

As I mentioned before, we're going to abstract the .NET Perfmon functionality away into our own classes. In fact, we'll use two: Counter and CounterCollection.

Using Solution Explorer, create a new class called Counter. Add this namespace import declaration to the top of the code:

using System;
using System.Diagnostics;

Next, add these members:

/// <summary>
/// Summary description for Counter.
/// </summary>
public class Counter
{
// members...
public String CategoryName;
public String Name;
public String HelpText;
public PerformanceCounterType Type;
private PerformanceCounter _counter;

That's all we need to do for a moment, so using Solution Explorer create a new class called CounterCollection. Add this namespace import declaration:

using System;
using System.Diagnostics;

This new class needs to inherit from System.Collections.CollectionBase, like this:

/// <summary>
/// Summary description for CounterCollection.
/// </summary>
public class CounterCollection : System.Collections.CollectionBase

Inheriting from CollectionBase in this way is a .NET best practice for creating strongly typed collections. Collections like these are far easier for developers to use than more general types than ArrayList or Collection for the simple reason that the developer can guarantee that she will receive an object of a certain type, whereas with general purpose collections each time a value is drawn from the collection it has to be case to the appropriate type and any exceptions caught and handled.

When inheriting from CollectionBase, it's convention to add three methods: Add, Remove and Item. In C#, Item becomes the default indexer property for the class. Add this method to CounterCollection:

// Add - add a counter...
public void Add(Counter newCounter)
{
List.Add(newCounter);
}

To make our lives easier later on, we'll need another version of the same method:

public void Add(String categoryName,
            String name, String helpText,
PerformanceCounterType
            type)
{
// create a new counter...
Counter counter = new
            Counter();
counter.Name = name;
counter.HelpText =
            helpText;
counter.Type = type;

// add
            it...
Add(counter);
}

The CollectionBase class defines a protected property called List that returns an ArrayList. All we have to do with Add is pass the Counter object over to the List.

Next, add this method to define a default indexer on the collection:

public Counter this[int index]
{
get
{
return (Counter)List[index];
}
set
{
List[index] = value;
}
}

Later, we'll find that it's useful to be able to find a counter by name. Add this version of the indexer that will look through each of the items in the list for ones with a matching Name property.

      public
            Counter this[String name]
{
get
{
// find the item in
            the list...
foreach(Counter counter in InnerList)
{
//
            match?
if(String.Compare(counter.Name, name, false) ==
            0)
return counter;
}

// return nothing...
return
            null;
}
}

One wrinkle about CollectionBase - when we want to iterate through our own collection, we have to use InnerList, not List.

Now, using Solution Explorer, create a new class called MyCounters. Add this namespace import declaration:

using System;
using System.Diagnostics;

Next, add this static field and this string constant:

/// <summary>
/// Summary description for MyCounters.
/// </summary>
public class MyCounters
{
// members...
public static CounterCollection Counters;
public const String CategoryName = "MyService";

By making this field static, from our Client object we'll be able to easily access the Counters collection and through that physically make adjustments to the counter viewable from the Perfmon application.

Next, add this method to MyCounters that will create the new collection and populate it with a single counter called NumClients.

      //
            RegisterCounters - register the counters...
public static void
            RegisterCounters()
{
RegisterCounters(false);
}

public
            static void RegisterCounters(Boolean forceCreate)
{

The first thing we do is to create a new CounterCollection object.

// create the collection...
Counters = new CounterCollection();

Once we have that, we can register the counter. We need to provide the name of the category (which we store in a constant called CategoryName), the name of the counter itself and the type of counter. We have quite a few counters to choose from, but this is one of the most common. NumberOfItems32 defines a 32-bit integer number that keeps a basic count. You can learn about the other counters from the MSDN library by looking up PerformanceCounterType enumeration in the index.

// add the counters...
Counters.Add(CategoryName, "NumClients",
"Reports the number of running clients.",
PerformanceCounterType.NumberOfItems32);

Although we can keep a list of performance counters in our own objects, that's useless unless we have a way to tell Windows and Perfmon about them. Our CreateCounters method that we're yet to build will do this for us. However, it makes sense to only go through this stage if the counters don't already exist, as it's inefficient to delete them and recreate them each time.

// do we need to install the counters?
if(forceCreate == false)
{
// do they exist?
if(PerformanceCounterCategory.Exists(CategoryName) == false)
CreateCounters();
}
else
CreateCounters();
}

If we do call CreateCounters, the first thing we try is to delete them:

      //
            CreateCounters - create the counters on the system...
public
            static void CreateCounters()
{
// try to delete
            them...
if(PerformanceCounterCategory.Exists(CategoryName) ==
            true)
PerformanceCounterCategory.Delete(CategoryName);

Creating the counters is just a matter of calling another static method on the PerformanceCounterCategory class just like we did with Exists and Delete. However, before we can do this we need to create a collection of Framework objects from our collection of objects.

         //
            create another collection to register them
            in...
CounterCreationDataCollection list = new
            CounterCreationDataCollection();

// go through each
            one...
foreach(Counter counter in Counters)
{
// create
            it...
CounterCreationData data = new
            CounterCreationData();
data.CounterName =
            counter.Name;
data.CounterHelp =
            counter.HelpText;
data.CounterType =
            counter.Type;
list.Add(data);
}

If this seems like a waste of time abstracting away from the Framework when the Framework looks similar to our objects - hold tight! We'll get there in a minute.

Once the Framework collection is up and running, we can call Create and register them.

// register the category...
PerformanceCounterCategory.Create(CategoryName, "", list);
}

Go back to the code editor for Form1 and before calling AdjustClientCount, call the static RegisterCounters method:

      private
            void Form1_Load(object sender, System.EventArgs e)
{
//
            register the counters...
MyCounters.RegisterCounters();

//
            create the first client...
AdjustClientCount(1);
}

Try running the project now and not much will happen. However, if you open Perfmon you should be able to see that the new MyService category and NumClients counter have both been registered. (If Perfmon is running, you'll have to close it and restart it before you can see the counter.)

Updating Counter Values

To update counter values, all we have to do is get hold of a System.Diagnostics.PerformanceCounter object that represents the counter want to update.

Open the code editor for Client. Add the private _counter method as shown here:

/// <summary>
/// Summary description for Counter.
/// </summary>
public class Counter
{
// members...
public String CategoryName;
public String Name;
public String HelpText;
public PerformanceCounterType Type;
private PerformanceCounter _counter;

Next, add this read-only property that can return an instance of a System.Diagnostics.PerformanceCounter object:

      public
            PerformanceCounter PerfCounter
{
get
{
// do we have
            it?
if(_counter == null)
_counter = new
            PerformanceCounter(CategoryName, Name, false);

// return
            it...
return _counter;
}
}

The trick now is to increment the NumClients counter when the thread first reaches the ThreadEntryPoint method and decrement it again just before it leaves. That way, we'll get an accurate reflection of the number of threads currently working. Open the code editor for Client and add a these two Perfmon calls to the top and bottom of ThreadEntryPoint:

      //
            ThreadEntryPoint - the thread starts here...
private void
            ThreadEntryPoint()
{
// tell perfmon about
            us...
MyCounters.Counters["NumClients"].PerfCounter.Increment();

//
            start a loop...
ManualResetEvent pauseEvent = new
            ManualResetEvent(false);
while(true)
{
//
            sleep...
pauseEvent.WaitOne(250, false);

// are we
            cancelled?
if(_isCancelled == true)
break;

// pretend
            to do some work...

}

// tell perfmon we've gone
            away...
MyCounters.Counters["NumClients"].PerfCounter.Decrement();
}

Now run the project and separately stop and restart Perfmon. Click the "plus" button to add a new counter and add the MyService | NumClients counter. The counter should start at 1 and as you adjust the counter should move between 0 and 32.

If you close the application, you'll notice that the number of clients has to fall to 0 before the application will stop. That's because on the Closed event, we individually spin down all of the running clients before relinquishing control back to Windows.

Improving Our Approach

You'll notice that when we want to adjust the counter we use a line like this:

// tell perfmon about us...
MyCounters.Counters["NumClients"].PerfCounter.Increment();

You can see there that we're using an indexer property on Counters and looking up the counter by name. That's not exactly the most efficient way of doing things, so ideally we need to find a better way.

Open the code editor for MyCounters and add this private member:

/// <summary>
/// Summary description for MyCounters.
/// </summary>
public class MyCounters
{
// members...
public static CounterCollection Counters;
public const String CategoryName = "MyService";
private static PerformanceCounter _numClients;

Next, add this property:

// NumClients - return the number of clients...
public static PerformanceCounter NumClients
{
get
{
if(_numClients == null)
_numClients = Counters["NumClients"].PerfCounter;
return _numClients;
}
}

Now when we want to use the counter we should reference the property rather than using the indexer. This will make things far more efficient, especially if we have a number of counters because having a reference and using it is far quicker than looking up the reference each time you want to use it. Open the code editor for Client change the code to use the new property:

      //
            ThreadEntryPoint - the thread starts here...
private void
            ThreadEntryPoint()
{
// tell perfmon about
            us...
MyCounters.NumClients.Increment();

// start a
            loop...
ManualResetEvent pauseEvent = new
            ManualResetEvent(false);
while(true)
{
//
            sleep...
pauseEvent.WaitOne(250, false);

// are we
            cancelled?
if(_isCancelled == true)
break;

// pretend
            to do some work...

}

// tell perfmon we've gone
            away...
MyCounters.NumClients.Decrement();
}

Now whenever we want to get at the counter we use a single static property on a class rather than having to roll through the entire list matching by name. The first time NumClients is called, the value stored in the appropriate Counter.PerfCounter property is cached for fast access.

Adding a "Rate" Counter

We've seen how to use the most common type of counter: NumberOfItems32. Another very common type of counter is a "rate" counter. These counters are useful for discovering how many times a particular operation or process is happening in a single second. In this section, we'll add another counter to our application that can report the rate of operations across all of the threads in a given second.

Open the CodeEditor for MyCounters and find the RegisterCounters method. Add an additional call to Add that will create our new counter.

public static void RegisterCounters(Boolean forceCreate)
{
// create the collection...
Counters = new CounterCollection();

// add the counters...
Counters.Add(CategoryName, "NumClients",
"Reports the number of running clients.",
PerformanceCounterType.NumberOfItems32);
Counters.Add(CategoryName, "RateOfWork",
"Reports the number of work items done each second.",
PerformanceCounterType.RateOfCountsPerSecond32);

// do we need to install the counters?
if(forceCreate == false)
{
// do they exist?
if(PerformanceCounterCategory.Exists(CategoryName) == false)
CreateCounters();
}
else
CreateCounters();
}

One wrinkle with the Perfmon implementation on Windows is that you can't just add new counters to an already defined category. If we want to add a new counter (as we do here) we have to delete the entire category and recreate it. Luckily, we can do this quite easily by calling RegisterCounters and settings its forceCreate parameter to True as this deletes and recreates the counters irrespective of whether or not they were already there.

Open the code editor for Form1 and make this change to the Load event handle:

      private
            void Form1_Load(object sender, System.EventArgs e)
{
//
            register the
            counters...
MyCounters.RegisterCounters(true);

// create
            the first client...
AdjustClientCount(1);
}

To use the counter, we can either add a new read-only property like we did with NumClients, or we can use the indexer. For sake of simplicity, we'll use the indexer but only once within the lifetime of the thread. Open the code editor for Client and add this code to get the performance counter and call Increment:

      //
            ThreadEntryPoint - the thread starts here...
private void
            ThreadEntryPoint()
{
// tell perfmon about
            us...
MyCounters.NumClients.Increment();

// get the wait
            counter...
System.Diagnostics.PerformanceCounter rateOfWork =
MyCounters.Counters["RateOfWork"].PerfCounter;

// start a
            loop...
ManualResetEvent pauseEvent = new
            ManualResetEvent(false);
while(true)
{
//
            sleep...
pauseEvent.WaitOne(250, false);

// are we
            cancelled?
if(_isCancelled == true)
break;

// pretend
            to do some work...
rateOfWork.Increment();
}

// tell
            perfmon we've gone
            away...
MyCounters.NumClients.Decrement();
}

Part of the appeal of the Perfmon implementation is that Perfmon itself deals with the logic behind the counters. In this instance, we're constantly incrementing the rateOfWork counter but Perfmon will automatically turn this into a rate of change counter.

To test it, run up the application and stop and restart Perfmon. Add both counters and you'll see that we now have two lines.

You'll notice here that the RateOfWork counter is always four times the current value for NumClients. This makes sense as we've told Client to wait for one quarter of a second per iteration of the loop, therefore our rate of work is always going to be "four times the current number of clients".

"RawValue" and Thread Safety

In our two counter examples we've consistently used Increment and Decrement to adjust the counters by 1 on each call. We can alternatively use IncrementBy to adjust the counter by more than one per call.

All three of these methods have been designed to be thread safe, which is why we can use the same PerformanceCounter object from multiple instances of Client with impunity. However, there is a property called RawValue on PerformanceCounter that can set the counter to a specific value. This property is not thread safe, so you should be careful should you have cause to use it. (Although in most cases it is rare that you would want to use it).

On the other hand, our CounterCollection is not thread safe, so in this section we'll look at how we can tweak CounterCollection so that it is.

First of all, we'll need a System.Threading.ReaderWriterLock object. This will let us protect access to the List property from our Add methods and indexer properties. Add this member to CounterCollection:

/// <summary>
/// Summary description for CounterCollection.
/// </summary>
public class CounterCollection : System.Collections.CollectionBase
{
// members...
private System.Threading.ReaderWriterLock _lock =
new System.Threading.ReaderWriterLock();

With the lock available as a private member, all we have to do is use it from in our Add method. Make this change:

      // Add -
            add a counter...
public void Add(Counter
            newCounter)
{
_lock.AcquireWriterLock(-1);
List.Add(newCounter);
_lock.ReleaseWriterLock();
}

The adjustments to the indexer properties are a little more complicated as we have to take care not to leave each property without releasing the lock that we've acquired. Make this change to the first indexer property:

      public
            Counter this[int index]
{
get
{
// find the
            item...
_lock.AcquireReaderLock(-1);
Counter found =
            (Counter)List[index];
_lock.ReleaseReaderLock();

// return
            the item...
return found;
}
set
{
// we need to lock
            on set too...
_lock.AcquireWriterLock(-1);
List[index] =
            value;
_lock.ReleaseWriterLock();
}
}

Notice that we have to acquire a lock in the set portion of this property too.

Next, make this change to the other version of the indexer property:

      public
            Counter this[String name]
{
get
{
//
            lock...
_lock.AcquireReaderLock(-1);

// find the item in
            the list...
Counter found = null;
foreach(Counter counter in
            InnerList)
{
// match?
if(String.Compare(counter.Name,
            name, false) == 0)
{
found =
            counter;
break;
}
}

//
            unlock...
_lock.ReleaseReaderLock();

// return what we
            found...
return found;
}
}

If you run the application again you'll notice that everything still works as it did before. However, this time you can be confident that if someone changes the CounterCollection object while you're using it you won't suffer any ill effects.

Server Explorer in Visual Studio.NET

Those of you reading the Perfmon documentation in MSDN may notice that Visual Studio.NET offers a view of the registered performance counters through the Server Explorer. I've steered you away from using this functionality because at the time of writing in July 2001, this functionality was pretty buggy. In particular, newly registered categories were not reflected in the Server Explorer view.

If you're reading this article after the next major release of VS.NET, try this functionality again by selecting View | Server Explorer from the VS.NET menu. You might find it useful!

Summary

In this article we've taken a look at how to use performance counters in your own application. Performance counters are of particular use to those of you building server applications, especially those destined to run as a Windows service. However, there is some scope for clever use of performance counters in your ASP.NET Web sites or even Web services.

C#Today - © 2002 WroxPress Limited


Using Custom Counters With Performance Monitor in .NET

Using Custom Counters With Performance Monitor in .NET
http://www.csharptoday.com/content.asp?id=1616
Article Rating: 4
Published on July 30, 2001
Abstract
Applications are getting more and more complex and bloated with time. An application could consist of numerous components, not all of which would be developed by the same company. In this plug and play paradigm, detecting and monitoring the system in real time offers many benefits - allowing an administrator or automated tool the ability to observe the system’s functioning and performance in real time and make decisions based on the observed data. To observe and collect application data that is useful in analyzing this real time system performance, Windows 2000 ships with a tool called the System (Performance) Monitor. However, interfacing with the performance monitor was one of the biggest challenges a developer faced. The interface was complex and required usage of shared memory and interprocess communication which was prone to error and the cause of numerous bugs. Nowadays though, Visual Studio .NET and the .NET framework make integration with the Performance Monitor trivial. In this article, Kaushal Sanghavi explores the support the .NET framework provides for interfacing with Performance Monitor.
Article

Introduction

Applications are getting more and more complex and bloated with time. An application could consist of numerous components, not all of which would be developed by the same company. In this plug and play paradigm, detecting and monitoring the system in real time offers many benefits. This allows an administrator or an automated tool the ability to observe the system's functioning and performance in real time and make decisions based on the observed data. For example, if an administrator realizes that the system page file utilization is higher than normal when a particular applications runs, it could point to the need to add more physical memory to the system. Another example could be the ASP and IIS performance data; if the average waiting time for a request is anything greater than a trivial amount, it could point to some serious bottlenecks in the system architecture, infrastructure, or both. To observe and collect application data that is useful in analyzing this real time system performance, Windows NT shipped with a tool called the Performance Monitor. (This tool is called System Monitor under Windows 2000; in this article we will use the term Performance Monitor to refer to both the Performance Monitor in NT and the System Monitor in Windows 2000). This is a graphical tool that captures performance data and can plot various graphs and can record data for future playback. However, under Windows NT and even under Windows 2000, interfacing with the performance monitor was one of the biggest challenges a developer faced. The interface was complex and required usage of shared memory and interprocess communication, which was prone to error and the cause of numerous bugs. However, Visual Studio .NET and the .NET framework make integration with the Performance Monitor trivial. In this article, we will explore the support the .NET framework provides for interfacing with Performance Monitor.

Performance Monitor Basics

The Performance Monitor is a very simple tool whose sole function is to collect data from applications that have registered themselves with the tool. Once an application registers itself, Performance Monitor simply polls the application periodically, depending on its sampling interval, to obtain the latest value of the data that the application wants to communicate. Performance Monitor itself enforces no restrictions or requirements on the application developer. Any application that presents the data in a certain format can easily integrate with Performance Monitor.

The Performance Monitor uses the term Categories and Counters to represent data. A Counter is the actual data being collected and a Category is a logical grouping of related counters. For example, ASP is an existing Category that contains a whole bunch of counters like Current Session, Total Requests, and Script errors to name a few. An application can separate its data into multiple categories, each of which can consist of multiple counters. The following screenshot displays some of the counters that belong to the ASP category. Furthermore, Performance Monitor can collect data not only on the local machine but also on any machine running on the network. (Of course, in order to connect to a remote machine, you need to have the necessary permissions on the remote machine).

Certain applications run in a singleton mode. A singleton mode implies that only one instance of the application can be active at any given point of time. For example, a machine can have only one instance of IIS running at any given time and consequently only one instance of the ASP runtime. Even if there are multiple users connecting to the same web server, a single instance of the ASP runtime is active. However, there are certain applications that allow multiple instances to be active. In this case, it is important to collect performance data separately for each running instance. Performance Monitor allows this by providing a list of active instances that a user can chose from. For example, the following screenshot displays the Print Queue object, which offers information about print jobs. As you can see, the Fax machine and the HP Office Jet printer both share this print queue and as a result are displayed as separate instances. Performance Monitor offers the ability to capture the data as a single unit consisting of all the instances, or on a per instance basis.

As we mentioned above, in order for Performance Monitor to collect data from an application, the application needs to register itself with the tool. Once the application is registered, Performance Monitor can query the application for the different categories and counters that it supports, and once a counter is chosen by the user, it can poll the application periodically to obtain the value of the counter data in real time. All this information is communicated by the application via the Windows registry:

Any application that needs to integrate with Performance Monitor needs to create an entry in the registry under the HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\ApplicationName\Performance key. When Performance Monitor starts up, it reads all the values in the registry under the HKLM\System\CurrentControlSet\Services branch that have defined a Performance sub key. This gives the tool a list of applications that support the collection of performance data. Further, the values under the key specify the application DLL that will be responsible for communicating with Performance Monitor and the various functions that should be called by the tool to collect performance data. The following is a brief description of these values.

  • Library: This is the DLL that Performance Monitor will load into its address space to obtain performance data. This DLL will be responsible for setting up an interprocess communication channel between Performance Monitor and your application which will be used to pass the actual counter values. This is necessary since your application and Performance Monitor run in separate processes and thus Performance Monitor cannot make direct method calls into your application. The most common form of this IPC is via the use of shared memory segments. Note that every application that needs to integrate with Performance Monitor needs to do this.
  • Open: This is the function that is called when Performance Monitor is starting up. This is called exactly once and should be used to initialize your application's performance data collector DLL.
  • Collect: This is the function that performs the real work. This is called by Performance Monitor periodically to collect the performance data. This is the function that should create the data structures that represent the categories, counters and instances that are supported by this application. You have to be careful when you write this function, because a crash in this function will bring Performance Monitor down.
  • Close: This is the function that is called when Performance Monitor is shutting down. This is called exactly once and should be used to release resources and perform general cleanup.

Integrating with Performance Monitor

An application can integrate with Performance Monitor in two ways. It can use existing performance counters and can read their values or update their values, or it can define custom counters that are specific to the application. In this section, we will discuss how Visual Studio .NET allows you to use existing counters and in the next section, we will describe how to define custom categories and counters and manipulate these counters.

With Visual Studio .NET, obtaining a reference to an existing counter is as simple as dragging and dropping an instance of the PerformanceCounter object from the components tab. This creates an instance of the System.Diagnostics.PerformanceCounter class. This is the class that encapsulates most of the functionality that is required to interface with Performance Monitor. Once this object is dragged and dropped onto your form, you can specify the category, counter and instance that this component will represent. Once this is done, you can use this component to interact with this performance counter.

As the above screenshot shows, we have added an instance of the performance counter object from the Toolbox to our form. Further, we set the properties on this component that specify the performance counter that this object will represent. For this example, we specify that the counter object will represent the Application Restarts counter from the ASP.NET category on the local machine. Since there is one single instance of the ASP.NET runtime, we do not specify the instance name. Further, since this is an existing counter (as opposed to a custom counter), we set the ReadOnly property to be true. This is because we only need to read the value of the counter; we do need want to update its value. Reading this value is as simple as invoking a single method on this object as show below:

private void btnPerfmon_Click(object sender, System.EventArgs e)
{
txtAppRestarts.Text = perfAppRestart.NextValue ().ToString ();
}

This displays the number of time the ASP .NET application has been restarted. Note that since this is not a custom counter, we will not be able to update this counter value. This is a key point to note, since if we were able to update this counter, it could result in spurious data being collected by Performance Monitor. Since only the application that owns the counter possesses knowledge to update it, the .NET runtime will prevent any other application from manipulating counters it doesn't own. If this were not true, then a malicious application would be able to update any counter with any value! Attempting to do this results in an exception as shown below.

private void btnPerfmon_Click(object sender, System.EventArgs e)
{
txtAppRestarts.Text = perfAppRestart.NextValue ().ToString ();
perfAppRestart.Increment ();
}

Note: We could of course use Performance Monitor without adding performance counters to our code, but in this case we only see the predefined .NET counters - without custom code we cannot monitor how much memory our application consumes, how many exceptions it has thrown, how many Interop calls it has made, etc. So now that we have seen how to use VS.NET to add performance counters to our applications, let us discuss how to add custom counters that represent our application data.

Custom Performance Counters

Before we start discussing custom counters, let us look into the classes that the .NET framework supports for the creation and manipulation of custom counters. These classes belong to the System.Diagnostics namespace.

  • PerformanceCounter: This is the class that represents a performance counter. This class can be used to read and update performance counters. However, you can only update custom counters, as we discussed above. The following are some of the important methods and properties that this class supports.
  • CategoryName: This specifies the category that this counter belongs to.
  • CounterHelp: This specifies a description of this particular counter. This is a useful property and should be used to communicate as much information to the user as possible. The following screenshot displays the help text for the Application Restart counter.

  • CounterName: This specifies the name of the counter.
  • CounterType: This specifies the type of the counter. A counter could be a simple numeric value or it could be represent a rate or average of the data being collected.
  • Decrement, Increment, IncrementBy: These methods are used to update the counter value for custom counters.
  • NextValue: This method is used to read the value of any given counter at this point in time.
  • PerformanceCounterCategory: This is the class that represents a category of related performance counters. This class supports methods for adding, updating and deleting counter categories.
  • Create: This creates a custom performance counter category and adds one or more counters to this category. This is the method that is responsible for creating the registry entries that we discussed earlier in this article.
  • Delete: This deletes a custom performance counter category. Note that currently, this method does not support the deletion of individual counters from a category.
  • GetCounters: This retrieves a list of counters that belong to this category.

Now that we have covered some of the basic classes and methods that are used to support the Performance Monitor, let us discuss how to build custom performance counters. Adding a custom counter is as simple as calling the Create method as we discussed above, and then calling methods on the PerformanceCounter class to update the value of the counter. We will build a simple calculator object that adds, subtracts, multiplies and divides two numbers. Further, we will add counters to this calculator object that monitors the operation on the calculator like the number of operations performed and the number of errors that were generated. These errors could be due to incorrect input (text instead of numeric values) or divide by 0 or overflow operations.

Let us start by looking at the method that creates the performance counters:

private void CreatePerfCounters ()
{
if (PerformanceCounterCategory.Exists(mCategoryName))
return;

CounterCreationData[] lCounterData = new CounterCreationData[3];
lCounterData[0] = new CounterCreationData();
lCounterData[1] = new CounterCreationData();
lCounterData[2] = new CounterCreationData();

lCounterData[0].CounterName = mTotalOpsCounter;
lCounterData[0].CounterType = PerformanceCounterType.NumberOfItems32;
lCounterData[0].CounterHelp = mTotalOpsCounterHelp;

lCounterData[1].CounterName = mTotalErrorsCounter;
lCounterData[1].CounterType = PerformanceCounterType.NumberOfItems32;
lCounterData[1].CounterHelp = mTotalErrorsCounterHelp;

lCounterData[2].CounterName = mTotalDiv0ErrorCounter;
lCounterData[2].CounterType = PerformanceCounterType.NumberOfItems32;
lCounterData[2].CounterHelp = mTotalDiv0ErrorCounterHelp;

CounterCreationDataCollection lCollection = new
CounterCreationDataCollection (lCounterData);
PerformanceCounterCategory.Create(mCategoryName, null, lCollection);
}

private string mCategoryName = "ASPToday Calculator";

private string mTotalOpsCounter = "Total Operations";
private string mTotalErrorsCounter = "Total Errors";
private string mTotalDiv0ErrorCounter = "Divide by 0 Errors";

private string mTotalOpsCounterHelp =
"This counter specifies the total number of operations that have occurred.
This includes all operations that resulted in errors.";

private string mTotalErrorsCounterHelp =
"This counter represents the total number of errors that have occurred.
These errors could be due to incorrect input,
or an overflow or divide by 0 errors.";

private string mTotalDiv0ErrorCounterHelp =
"This counter represents the total number of Divide By 0 errors.";

We start by checking if the performance category already exists. If it does, we do not need to create the counters all over again, so we return without doing any work. However, if the category does not exist, then we need to create the custom category and counters. As you can see from the code block above, we will create a category named ASPToday Calculator that will have three custom counters. These counters will reflect the total operations that have been performed by the calculator over its lifetime, the total number of errors that have occurred and the total number of divide by 0 errors. We use the static Create method from the PerformanceCounterCategory class to create the custom counters. The CounterCreationData is the class that represents each individual counter. In order to set up each counter, we specify three properties for the counter: the name of the counter, the type of the counter and the help string that is associated with each counter. This collection is passed to the Create method that will create the category and the counters.

Once this method is invoked, the custom counters are created and can be verified by checking the registry. The following is the screenshot that displays the Performance sub key for the ASPToday Calculator key and its values. As you can see the Library value does not point to our assembly, instead it points to the netfxperf assembly which is a .NET framework assembly that manages the plumbing for interfacing with the Performance Monitor. As you can see, all that we need to do is call methods on the .NET classes; we do not need to worry about setting up the communication with Performance Monitor.

public
            Calculator()
{
CreatePerfCounters ();

mTotalOps = new
            PerformanceCounter
(mCategoryName, mTotalOpsCounter,
            false);

mTotalErrors = new PerformanceCounter
(mCategoryName, mTotalErrorsCounter,
            false);

mTotalDiv0Errors = new PerformanceCounter
(mCategoryName, mTotalDiv0ErrorCounter, false);
}

Once the counters are created, we need to obtain a reference to each of the three custom counters so that we can update their values. Doing this is as simple as creating an instance of the PerformanceCounter class and passing in the category name and the counter name. The third argument specifies that this counter will not be read only; since we will be updating the counter values, we have to specify this. The default is that the counters are read only and any attempt to update them will result in an exception. Note that in this example, we manually create an instance of the PerformanceCounter class. Another way of doing this would be to drag and drop three instances of the PerformanceCounter object from the VS.NET IDE's Toolbox. However, since we are not guaranteed that the counters will exist on the machine on which this is run, the manual method is safer. (Performance Monitors can be installed using custom .NET installation components as well, but this topic will be covered in a different article).

Once these counters have been created and initialized, we can update them by calling the Increment and Decrement method on these counters. In our example, we are keeping a running count of a total and so we will never use the Decrement method; however, this method is identical to the Increment method.

public int Add (string iOp1, string
            iOp2)
{
try{
mTotalOps.Increment ();
int lResult =
            Int32.Parse(iOp1) + Int32.Parse(iOp2);
return
            lResult;
}
catch (Exception){
mTotalErrors.Increment
            ();
return -1;
}
}

public int Subtract (string iOp1,
            string iOp2)
{
try{
mTotalOps.Increment ();
int lResult
            = Int32.Parse(iOp1) - Int32.Parse(iOp2);
return
            lResult;
}
catch (Exception){
mTotalErrors.Increment
            ();
return -1;
}
}

public int Multiply (string iOp1,
            string iOp2)
{
try{
mTotalOps.Increment ();
int lResult
            = Int32.Parse(iOp1) * Int32.Parse(iOp2);
return
            lResult;
}
catch (Exception){
mTotalErrors.Increment
            ();
return -1;
}
}

public float Divide (string iOp1,
            string iOp2)
{
try{
mTotalOps.Increment ();

if
            (Int32.Parse(iOp2) == 0){
mTotalErrors.Increment
            ();
mTotalDiv0Errors.Increment ();
return
            -1;
}

float lResult = Int32.Parse(iOp1) /
            Int32.Parse(iOp2);
return lResult;
}
catch
            (Exception){
mTotalErrors.Increment ();
return
            -1;
}
}

As you can see from the above code block, we update the Total Operations counter in each method, and we increment the error counters whenever an exception is thrown. The following is a screen shot of our simple calculator.

Once this is run and a few operations are attempted, the following graph from Performance Monitor is obtained. As you can see, it plots the values of the counters in real time.

Conclusion

Monitoring the runtime statistics of any system offers vital clues towards debugging, troubleshooting and tuning the system. This is even more important once the system is deployed and the original developers and code debugging tools are not usually available at the deployment site. The Performance Monitor is a great tool that can be easily integrated into any system and allows for monitoring of any component in your application. However, there is a small overhead to Performance Monitor, since your application will have to collect data and communicate this to Performance Monitor. However, under most situations, this overhead is minimal and the benefits of monitoring your application far outweigh the performance hit. Further, the .NET framework makes it extremely simple and easy for developers to add this support to their applications.

Happy .NETing.

C#Today - © 2002 WroxPress Limited


Web Site Error Logging and Reporting System

Security

Security (Sort by Name)

A Suite of Cryptographic Objects Part 2 Implementing Key Exchange Over a Network
C# Browser History Monitoring Application - Part 1
Code Access Security
Cryptographic Objects in C# pt_ I
Custom Permissions
Custom Security Implementation
Explaining the System.Security Cryptography Namespace
Online CPU Console using a Web Control Library with .NET Security
Protecting Your Code - Reflection, Obfuscation and Encryption - Part 1
Protecting Your Code - Reflection, Obfuscation and Encryption - Part 2

Role Based Security
Secure Authentication Using a C# Implementation of the Zero Knowledge Proof


Protecting Your Code - Reflection, Obfuscation and Encryption - Part 2

Protecting Your Code - Reflection, Obfuscation and Encryption - Part 2
http://www.csharptoday.com/content.asp?id=1855
Article Rating: 4.5
Published on August 26, 2002
Abstract
The first part of this case study by Roger Rowland examined how reflection and CodeDOM could be used together to reverse engineer a compiled .NET assembly into C# source code. This second part looks at the implications of this, and explores ways of attacking the problem. We’ll look in detail at code obfuscators ?how they work, what are their strengths and weaknesses, we’ll discuss alternative methods of securing sensitive code, and finally develop an example application demonstrating an alternative technique for making life more difficult for the casual hacker.
Case Study

The first part of this case study (here) examined how reflection and CodeDOM could be used together to reverse engineer a compiled .NET assembly into C# source code. This second part looks at the implications of this, and explores ways of attacking the problem. We'll look in detail at code obfuscators - how they work, what are their strengths and weaknesses, we'll discuss alternative methods of securing sensitive code, and finally develop an example application demonstrating an alternative technique for making life more difficult for the casual hacker.

Introduction

Before the advent of .NET, Win32 executables were hard to reverse engineer. You had to have a certain level of skill and knowledge and be very familiar with assembly code and operating system internals. It wasn't a game that everyone could play. Over the years, a variety of techniques have been invented, and third-party tools developed, to make it even harder for the hackers and crackers. In essence there already exists some sort of equilibrium where the good guys and bad guys use quite sophisticated techniques to outwit each other, while the rest of the world just gets on with the job and takes very little notice.

With .NET, the situation has changed somewhat. Now, you don't need a great deal of knowledge and expertise to get inside someone else's code. We proved that in the first part of this case study. Now anyone with a copy of ILDasm, let alone any of the commercial .NET decompilers, can look over your assemblies and reverse engineer your designs and source code. Suddenly, your software licensing code, copy protection mechanisms, and proprietary business logic are much more open to the common gaze - whether it's legal to do so or not. Anyone can peruse the details of your software for whatever reason they like - be it an attempt to debug their own interface with your application or to search for security flaws or loopholes. This must be enough to make many people pause for thought.

In this case study we're first going to take a look at the different options available for protecting your code. Then we'll look in more detail at obfuscators. These are tools designed to make the information retrievable from an assembly more difficult to interpret. We'll see in principle how they manipulate the metadata in a .NET assembly, and look at the benefits and any side effects the use of such tools may bring. We'll go on to explore encryption by taking a look at what's available as standard within the .NET framework class library (FCL) and finally develop an alternative technique to try and push those goalposts a little further out of reach.

System Requirements

This system requires Visual Studio .NET V1.0 or later running on Windows 2000 or Windows XP. The case study assumes that the reader has a good understanding of C# and some familiarity with encryption terminology.

Case Study Structure

After a very brief recap on the ground we covered in part one of this case study, we'll look at the options available for protecting compiled code from decompilers - primarily by obfuscation. Then we'll finish up by demonstrating an alternative technique using just pure C# and the FCL to add one more weapon to the arsenal.

Recapitulating on Reflection

In part one of this case study we developed a simple .NET decompiler, which took a compiled assembly and reverse engineered this back into understandable C# source code using just the System.Reflection namespace together with the System.CodeDOM code-generating classes. We showed how this was possible due to the comprehensive nature of the metadata stored in every .NET assembly and that it did not take a great deal of time or ingenuity to do so.

The real dilemma is that this situation arises as a natural consequence of the techniques Microsoft used to bring about some of the major advantages of .NET. In order to tackle things like the long-standing "DLL hell" problem, binary compatibility issues with COM, and to provide language interoperability in an object-oriented environment, it almost became inevitable that we would end up with completely self-describing components. Which is exactly what we've got. The JIT (Just In Time) compiler, which forms part of the .NET framework, uses the metadata held in .NET executables to achieve early-binding without having to worry about the exact order of methods in a vtable or the memory footprint of an object. This allows applications to adapt at run time when updated versions of components are installed and is nicely complemented by the side-by-side versioning capability of the CLR. I have to say that, in my opinion, it is an entirely logical solution and I really like it. It's certainly not going to go away.

So, is this something we're stuck with? Do we have to accept that it will now be easier than ever before for our applications to be reverse engineered? After all, this is not a new problem - Java developers have always been faced with this issue and the world still turns. Is it just that it's different to what we've been used to? Should we be using .NET just for server-side development? If so, why is there such a wealth of support in .NET for deployment and end-user security? How can we migrate our existing applications to the .NET platform and make sure we retain control of our intellectual property?

Well, the answer is that there are a number of options. The word of the day here is compromise. It's impossible to prevent reverse engineering of compiled code - all we can do is make it harder. As long as we accept that, there's a lot that can be done to maintain the equilibrium. That's what this second part of the case study is all about

Protecting Intellectual Property

The following sections will describe some of the current options available for protecting the intellectual property contained in compiled .NET assemblies. This is not intended to be a review or comparison of particular products or techniques. The purpose of this section is to give you as much detail as I can on the technical issues involved in protecting your code in order to help you decide on the best policy for your individual requirements. We will cover the following areas:

  • Use of unmanaged code
  • Native code generation
  • Code obfuscation
  • Other solutions

Use of Unmanaged Code

This is perhaps the most obvious solution. At one end of the spectrum, we forget the .NET environment and develop all of our applications as unmanaged code using C++. At the other end, we identify a couple of important subroutines and implement them in an unmanaged DLL, using P/Invoke to call into these from our managed code as required. Alternatively, we could use the managed C++ compiler to produce an assembly containing both managed and unmanaged code.

At first sight this might seem like a nice low-impact solution. We probably already have the necessary C++ (or whatever) skills to write and maintain the unmanaged portions, and it might seem like a gentle way of dipping a toe in the .NET waters by redeveloping non-critical parts of a system first and proceeding in an incremental manner. Of course, we need to be happy that some portions of our system will still be easy to decompile and so not all applications will be suitable. However, this approach really just postpones the issue and there are other disadvantages. There are no real killers but you nevertheless need to be aware of them.

The ability to call unmanaged code from .NET requires a highly trusted security status, which may mean your application has to have more permissions than it really needs. As .NET does not provide for restricted delegation of authority, this interface into unmanaged code presents an ideal point of attack for viruses and trojans to use your application for their own undesirable purposes. At the point where a trusted assembly calls into unmanaged code, the unmanaged code inherits the same permissions as the caller and, similarly, any executable invoked in turn by this unmanaged code also inherits the original permissions. It would be quite easy for someone to insert a call into your unmanaged dll, which would execute any program of their choice with a highly-trusted status. It is not possible in the .NET environment to restrict this delegation of authority to particular modules and this is a potential security weakness.

Using unmanaged code inevitably ties you down to a particular implementation platform. This is not a problem just yet because there's no established cross-platform support for .NET. However, if and when this happens you immediately lose out on a big potential benefit.

If you're using P/Invoke to access the unmanaged code in your application, there is a performance hit. How much of a problem this causes will depend on how often you call into the unmanaged code and how much data needs to be marshalled across the interoperation boundary.

Maybe an obvious point, but one that shouldn't be missed, is that you will need to retain the necessary skills to support and maintain the unmanaged portions of code and also ensure you have a suitable development environment to do so. I'm not suggesting for a moment that C# will replace C++ overnight (or even very soon) but I am reminded of the very large daily rates on offer for "legacy" Cobol programmers in the run up to Y2K - maybe an extreme example but thinking ahead is not as common as 20/20 hindsight.

Ok, and of course unmanaged code can also be reverse engineered - a counter often espoused on the newsgroups and, in my opinion, not particularly helpful. This is of course exactly where we were before .NET and we're just trying hard not to move too far in the wrong direction. I don't think that's unreasonable.

In essence, use of unmanaged code to protect intellectual property is a viable option for some people in some situations, but it's not for everyone.

Native Code Generation

If you frequent the .NET newsgroups, you'll see that some people have asked whether the use of the ngen.exe utility, which is part of the .NET Framework SDK, might be employed to prevent people decompiling your assemblies. The short answer to this is really no. The ngen utility is specifically designed to allow you to pre-JIT an assembly on a user's computer at the time the application is installed. It is not designed to protect your assembly from reverse engineering.

What ngen does is to compile your assembly to native code during the installation process and to store this in the Native Image Cache, which is part of the Global Assembly Cache (GAC). This is an entirely worthwhile and reasonable exercise and will reduce application load and run times on the target system. The crux of the matter is that it has to run at install time - you can't just copy and paste a ngen image out of the GAC. In any case, the native image won't run without the CLR, and the CLR still needs access to the original assembly, complete with metadata, for type resolution, reflection and version checking.

Just attempting to use ngen to produce a native image before distribution to the end user is fraught with problems. Not only do you need to do this on a machine with the same processor and operating system but you also need to consider a host of other dependencies. You need to know the exact identities of any other assemblies you bind to, which security policy the code should run under, and what version of the CLR is on the end user's machine. Not a simple task! If you can identify all of these, then any subsequent change on the user's machine that affects dependent assemblies may result in the CLR disregarding your native image and reverting to JIT - for which it needs the IL in the original assembly.

Some documentation suggests that pre-JITted assemblies are not compiled with the same optimisations possible with a true JIT on the target platform. I don't have any hard evidence about this, but it's certainly true that the operating system and processor make a big difference. For example, Thread Local Storage (TLS) is optimised for a particular OS, and some processor instructions (e.g. CMOV) can be used on a Pentium 4 but not a Pentium 3. These differences definitely exist but are not comprehensively documented and are of course subject to change. Still keen?

The documentation for ngen states "If Ngen.exe encounters any methods in an assembly that it cannot generate, it excludes them from the native image. When the runtime executes this assembly, it will revert to JIT compilation for the methods that were not included in the native image." So there's not even any guarantee that you can completely pre-compile your assembly anyway. And, assuming it has pre-compiled, the ngen image will contain data structures defined and used in the CLR which could change in later versions - you may find that a .NET framework service pack renders your distributed image invalid. The CLR won't care, it will again revert to JIT and carry on regardless - assuming, that is, that you have also made the original assembly available, which somewhat defeats the object.

There are so many things which make this option a non-starter that you need to be really sure of what you're doing and the likely effects before you attempt to do this. If you want my advice - don't go there. Maybe look at a third-party obfuscator instead.

Acknowledgements

In preparing this part of the case study, I would like to extend my thanks to the following individuals and their respective organisations for their valuable time and information. Without exception, they have been more than happy to discuss technical details of their products and so made the preparation of this case study both satisfying and informative.

Brent Rector (http://www.wiseowl.com/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq) - Demeanor for .NET, Enterprise Edition

Daniel Jebaraj (http://www.syncfusion.com/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq) - Essential Obfuscator

Dr. Huihong Luo (http://www.remotesoft.com/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq) - Remotesoft Obfuscator and .NET Protector

Gabriel Torok (http://www.preemptive.com/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq) - Dotfuscator

Obfuscation

Already there are a number of commercial .NET obfuscators. I've looked at four of the most well-known in researching this case study and have tried to present a composite view of the technology on offer. There has obviously been a great deal of thought and effort behind the development of these products, with some useful innovations coming to light. I don't doubt that they have all generated a high level of interest from the .NET community and will continue to do so as the .NET movement spreads.

In this section, which is quite detailed, we'll look first at the principles underlying obfuscation before going into more depth as we see exactly how these products go about their job. This section will conclude with a discussion of the strengths and weaknesses of obfuscation in principle.

What is Obfuscation?

The Oxford English Dictionary defines obfuscate like so.

"obfuscate - 1. obscure or confuse (a mind, topic, etc.). 2. stupefy, bewilder"

In the context that we're using the word, this is exactly what we mean. Obfuscation is not about preventing someone from reverse engineering your code, it's all about making the information they get out much more difficult to understand.

The primary method employed by obfuscators is to rename symbols in the metadata. By symbols, I mean the names of your classes, methods, member fields, etc. This has absolutely no effect on the functionality of your code - in these terms it still runs as it always did, the CLR still sees it as a valid .NET assembly and it can still (in theory) be decompiled just as easily. It's just that - to a human reader - it's much more difficult to make any sense out of it when all of the identifiers are meaningless.

Most of the commercial obfuscators try to throw away as much information as possible without compromising the function of your code in any way. This is known as a lossy transformation and the extent to which this can be done might surprise you. Also, the effect that the loss of these vital clues has on the ability of someone to understand your code should not be underestimated. Mind you, it's not just humans who need to be fooled - it's not so easy to confuse a decompiler by using duplicate names and apparent absence of context. Some of these more psychological obfuscatory techniques can be mitigated during decompilation so there's a real battle here to produce something that is less comprehensible both to humans and machines and yet that still has to run as it always did. A tough challenge.

So let's see how it's done.

Obfuscation in Practice

Some .NET obfuscators - certainly early ones - adopted a technique that has become known as round-tripping. This involves dumping an assembly to an IL source file (using ILDasm), making changes to this source file, and finally recompiling with ILasm. It's fair to say that this technique has some major drawbacks and is not used by any of the commercial obfuscators I looked at. The biggest problem with this is the inability to handle mixed images - those that contain both managed and unmanaged code. This includes almost every assembly developed with the Visual C++ .NET compiler.

The problem is that ILDasm just ignores any code that is not IL - if you disassemble a mixed image .NET assembly using ILDasm, any native code functions will be left out of the dumped source file. This is really a problem for applications developed with the managed C++ compiler. In this environment, it is possible (and likely) that there will be both managed and unmanaged code in the same assembly, so any attempt to decompile, obfuscate, and recompile with ILasm will result in an incomplete assembly.

Mainly for this reason, all of the .NET obfuscators examined here do not employ round-tripping but operate directly on the metadata in a .NET assembly, which, in my opinion, is the right thing to do.

In addition to favouring direct obfuscation of .NET binaries, it seems that most commercial obfuscators are moving feature-wise in the same direction. There are differences in terminology and undoubtedly implementation, but there are a number of common features we can examine. These are:

  • Symbol Obfuscation
  • Metadata Obfuscation
  • Control Flow Obfuscation
  • String Encryption

I'll describe each of these in turn and then cover a few of the additional features on offer.

Symbol Obfuscation

Symbol obfuscation affects the names of the types and members defined in an assembly and, in some cases, types referenced in other assemblies. To understand this better, we need to see what the internal structure of a .NET assembly is like. We'll do this in general terms, not being too pedantic about the details. If you are interested in looking deeper then consult the Word document Partition II Metadata.doc, which is installed along with the framework SDK.

Figure 1 shows, in schematic form, the major parts of a compiled assembly (an exe or a dll). At the head of the file are conventional PE (Portable Executable) headers. These identify the file type to the operating system and allow it to be loaded for execution. Apart from the mechanism used to ensure that this executable is handled by the CLR, this is essentially no different to any executable binary and is of little interest to us for the purposes of obfuscation.

Figure 1 - Inside an Assembly

Following the PE headers is the CLI header (Common Language Infrastructure). The CLI header contains all of the runtime-specific data (e.g. CLR version, strong name signature) and includes pointers to the metadata and the method definition of the assembly's entry point. This is followed by the metadata and IL code itself - the section labelled CLI Data in Figure 1 - this is the section we're really interested in, so we'll look a little closer inside.

The metadata (as I'll generically refer to this section of the assembly) contains a number of tables and some heaps. The tables are of three types - there is a set of tables that describe the types and members defined in the assembly, a set of tables that list the types and members referenced by the assembly, and a set of manifest tables that basically contains the names of the files that are part of the assembly. The latter primarily concerns multi-file assemblies, where an assembly can be logically composed of a number of physical modules.

The heaps include the string heap and the blob heap. These heaps are used to store various information and are referenced by pointers in the metadata tables. For example, the names of all methods, fields, properties, etc. are held in the string heap, and the values of compile-time constants, assembly hash value, public keys, etc. are held in the blob heap.

Figure 2 shows this schematically.

Figure 2 - Inside the Metadata

The items we're most interested in for performing symbol obfuscation are the string heap and the definition tables. Some obfuscators allow a number of interdependent assemblies to be obfuscated as a logical unit, in which case some of the information in the reference tables is also changed. We'll just look at the more simple case here, as the principle is the same.

Out of the many definition tables held in the metadata, three of the most relevant for performing symbol obfuscation are the following:

Table Name | Description
TypeDef | Contains one row for each type defined in the assembly. Indicates such things as the accessibility (public, private, etc.) and the base type, and contains pointers to the methods, fields, etc. defined in the relevant metadata table. The type's name and its namespace are indicated by pointers into the string heap.
FieldDef | Contains one row for every field defined in the assembly. Indicates the field type and accessibility and holds the field name as a pointer into the string heap.
MethodDef | Contains one row for each method defined in the assembly. A pointer to the string heap provides the name and a pointer to the blob heap provides the signature. In addition to the accessibility and other flags, there will be a pointer to the IL code for the method and pointers to parameters in the ParamDef table

You can see how each table is organised to keep the size of the assembly to a minimum. The use of the string heap plays a big part in this as each unique string is held only once. If, in our source code, we used the same identifier for a method, field, parameter, property, event, etc. then the compiler just puts one string in the heap and ensures that all relevant rows in the definition tables contain a pointer to it. This technique extends to the reference tables too - if we call a method on a type in a referenced assembly that happens to have the same name as a member in our own assembly, then we still only hold the name in one entry on the string heap.

You might possibly begin to see how the process of symbol obfuscation is assisted by this structure. Especially when you consider that the actual names used in the source code only need to remain visible for certain purposes. After all, the compiled IL code has already been processed to ensure that any types, methods, fields, etc. that it uses are resolved into pointers to the correct row in the relevant metadata table. Once it's been compiled (to the IL stage), the original names are largely redundant. We only need to be careful about situations where the name still matters. These include the following:

Any public types, methods, fields, properties, events, etc. defined in our assembly. There's not much point in exposing a public type and then changing its name!

Specific places in code where types are referenced by name - for example by using the reflection classes to create an instance of a type (effectively, late binding, or dynamic object creation).

Serialization, and therefore remoting. By default, the process of serializing to and from XML uses the names of members (determined at run time) as element names. If, at run time, your applications have obfuscated versions of the original names then this will not work correctly.

Apart from the above cases, the majority of private members defined in an assembly can be safely renamed without affecting the operation of the code in any way. The way that an obfuscator will do this is by manipulating values in the string heap and changing pointers in the definition tables to make the resulting picture appear nothing like your original source code.

Most obfuscators start by replacing meaningful names with single-character identifiers and re-use these same names as much as possible while still keeping the assembly logically the same as far as the CLR is concerned. This means that method names may be heavily overloaded - as long as no two methods have the same signature, they may as well share the same name - even if this makes no sense as far as the application design goes. That's the whole point. It's not just the original names that are lost, but the picture of which methods are genuinely overloaded. This all helps to confuse matters if the assembly is decompiled.

This obfuscation can be performed on a type definition or any of its members - not just the methods. As long as the resulting assembly is still valid to the CLR it doesn't make any difference. For example, consider the following class definition:

Figure 3 - Before Symbol Obfuscation

Figure 3 shows a picture of the content of the definition tables and string heap, which might have resulted from building this C# class. The class and its members are entered by the C# compiler into the relevant definition tables in the assembly's metadata. The string heap contains all of the member identifiers, and the rows in the definition tables reference these using pointers.

Figure 4 shows how the picture may appear after obfuscation.

Figure 4 - After Symbol Obfuscation

You can see that there's been no serious change made to the metadata - all of the original definitions are untouched apart from the pointers to the string heap. The string heap itself has been trimmed and this is one reason why assemblies physically shrink following obfuscation.

While still making sense to the CLR, any attempt to use the information now contained in the metadata to decompile the class would result in the following (invalid) C# source code:

private class a
{
string a;
int
            a;
a a;
a[] a() {.}
}

This is obviously much less understandable to any human reader. However, note that the structure of the types defined in the assembly is still visible and class hierarchies can still be derived - although some obfuscators do attempt to randomise things by placing related classes into different namespaces and vice versa (we saw in part one of this case study that the namespace used when declaring a type is held in the metadata but is not required by the runtime, so there is no adverse effect on the function of an assembly obfuscated in this way).

Notice that we have done nothing that affects the IL code which implements each method body, so the application will function exactly as it did before obfuscation.

Metadata Obfuscation

There is some information held in metadata definition tables that is not really necessary at runtime. In particular, this is information about properties and events. In the first part of this case study, we discovered exactly how these constructs are implemented in a compiled assembly so this should not be too surprising (unless of course you didn't read the first part!).

The get and set accessors of a property are implemented just like any other method, and any use of properties in the remainder of your assembly's code is resolved into the appropriate method call at compile time (that's at the C# compiler stage, not the JIT compiler). Similarly, any code that registers or unregisters interest in an event is resolved into the corresponding add, remove or raise method call and the remaining work is taken care of by the compiler-generated MulticastDelegate.

This means that there is no need to treat properties or events as special cases at runtime. However, the metadata includes a number of tables that serve to identify the properties and events defined in the assembly and to associate these with their respective MethodDefs. This makes it easy - via reflection - to identify which methods handle which properties, and this information could be useful to a hacker (it was certainly useful for our decompiler in part one). Remember, the presence of these tables does not directly affect runtime behaviour.

Figure 5 shows how a number of metadata tables are used to associate events with a type. The EventMap table allows a number of events in the Event table to be linked with their defining type. For each such event, the MethodSemantics table identifies the relevant entries in the MethodDef table for those methods that implement the event. This rather convoluted linking allows the exact type of method (add handler, remove handler, raise event) to be identified as well as the method itself. The situation is similar for properties:

Figure 5 - Event Information in Metadata

So, while this information is very useful for the Intellisense prompting of the development environment, and certainly useful for a decompiler, it's completely disregarded by the CLR at runtime. More than one of the commercial obfuscators available remove this redundant information completely - thus permanently losing the relationships between these methods. As the method names will also be subjected to symbol obfuscation, there's no foolproof way of rebuilding these relationships during decompilation. It all muddies the waters further.

One of the more controversial obfuscation techniques coming into play does much more than we've seen so far. I gather that this is not something that everyone will be comfortable with, and I expect it will be implemented as an optional obfuscation technique. This is because it affects the IL code itself and potentially the performance and/or functionality of the application. We'll discuss this next.

Control Flow Obfuscation

As the name suggests, control flow obfuscation involves making it more difficult to follow the flow of control through your code. At the time of writing, it's not so easy to get a good idea of the techniques being employed as a number of the commercial obfuscator vendors are still in the process of developing this feature. What I can do is to describe in general terms some of the known techniques, but bear in mind that this is a developing area.

If you've ever tried to work your way through a bad developer's spaghetti-code, you will appreciate how much confusion can be generated by adding a little spaghetti into a previously well-structured application. The major difficulty here is obviously being able to do this without affecting the logical functioning of the application. The techniques I've identified as possible candidates for use in an obfuscator are as follows:

  • Inlining
  • Outlining
  • Code synthesis

Inlining - is exactly what you expect. This involves replacing a method call with the contents of the called method. This is something quite often performed anyway during JIT compilation. It's quite likely that the bodies of simple get and set accessor methods will be inlined by the JIT compiler to overcome the performance overhead associated with the use of properties. Apart from possibly bloating code by inlining a large method in multiple places, there's no real danger of producing unwanted side-effects by inlining IL code to produce larger, more monolithic methods. This effectively removes structure from the code and hinders comprehension.

Outlining - is the opposite of inlining. This means removing a section of code from a routine into a new method and inserting a method call in its place. This technique is more normally associated with profile-based optimisers, where less-frequently-accessed blocks of code (often error handlers) are outlined from the body of a routine to speed execution through the most likely path. When used in an obfuscatory fashion, outlining creates superfluous structure in decompiled code, so adding to the confusion. Like inlining, outlining is pretty safe functionality-wise, although as an obfuscator would probably not perform a profiling exercise, it's equally likely that outlining could hinder as well as help performance.

Code synthesis - is where code is genuinely changed in the assembly and is, in my opinion, the most difficult to assess in terms of unwanted side effects. It's true from a theoretical view that there exist code constructs that are logically the same yet can be expressed using different expressions. But, as IL code doesn't retain these original higher-level constructs, it's difficult to see how this can be used to confuse a decompiler. Consider the following class:

class Class1
{
private void
            forloop()
{
for (int i = 0; i < 10;
            i++)
Console.WriteLine(i);
}

private void
            whileloop()
{
int i = 0;
while (i <
            10)
{
Console.WriteLine(i);
i++;
}
}

These two methods - forloop() and whileloop() - are functionally identical. If you compile this class with the C# compiler and examine the generated IL using ILDasm, you will see that they produce exactly the same output, byte for byte. See Figure 6 for an example:

Figure 6 - IL Code "for" loop

So how does a decompiler decide whether this should be reverse engineered to a for loop or a while loop?

The answer is that it doesn't. If you decompile the above assembly using Anakrino, both methods will appear as while loops. If you decompile using Remotesoft's Salamander, both will appear as for loops. So what exactly is my point?

My point is that, although IL is a much higher-level representation of the original source code than native x86 assembler, it still appears to be at too low a level for an obfuscator to reliably identify patterns in the code that might be misinterpreted by a decompiler. Don't misunderstand me here - I'm not saying that it can't be done, I'm just having trouble reconciling the need for a provably equivalent code change with a result that is in some way obfuscated. Nobody will want to gamble that their code might work incorrectly following obfuscation, so it has to be provable in theory that this can't happen. Given that this is indeed true, then the obfuscated code must still stand a fair chance of being abstracted correctly by a decompiler. Whether, in a situation like that above, we get a for loop or a while loop is irrelevant - someone can still understand it. If, on the other hand, we end up with something that does fool the decompiler, then the chances are that it will also fool (or mislead) the JIT compiler and we may end up with a less than optimal native code version. It has been suggested to me that JIT compilers are optimised for the IL code patterns generated by code compilers such as C# and VB.NET and this doesn't seem an unreasonable assumption.

I think I will keep a keen eye on developments in this area and would welcome any feedback which might help resolve the issue. In the meantime, let's change tack slightly and look at another widely used obfuscation technique.

String Encryption

You shouldn't be surprised to hear that the metadata holds any string literals used in your code. This is no different in principle to an x86 executable, and it's already quite easy to look for string references with a Win32 disassembler to identify a particular section of code for deeper study. If someone can identify the exact method that displays the string "Invalid license code - please try again", then it probably doesn't matter too much that the name of the method has been changed by an obfuscator.

Most of the obfuscators I looked at offer the ability to encrypt any such literals in the assembly. It's almost ironic that the accessible nature of the .NET assembly, which has been working against us all along, now makes it relatively easy to identify all of the string literals, find the methods that use them, and insert a call to a procedure that will decrypt the string at runtime. We can even add the classes and methods to do this right into the existing file without breaking anything.

Yes, this has a slight performance hit (we're talking compromise remember) but it seems crazy to go to the trouble of obfuscating all of the symbols in your assembly if you intend to leave a bunch of signposts around that make navigating to the correct method a complete doddle. Ok, suggestive clues in string literals won't be available in all cases, but it's another item of information than can be hidden, so why not hide it?

That's all that can be said about the most common features of all serious commercial .NET obfuscators. The final section in this topic will cover a few of the innovations introduced by various vendors to make their product more useful or attractive. If you'd like to obtain further details of any of these obfuscators, follow the links at the end of this case study.

Other Features

Before I start this section, I have to state that I have no financial interest in any of the products mentioned (I should be so lucky). I have attempted to remain objective and give a composite view of the technology on offer as this is the most important factor in deciding what's best for your situation. There are other factors to consider but this is not the place to discuss them.

The following features may be provided by one, more, or all of the listed products and I've deliberately not made mention of who offers what. Some features may still be on the wish-list. While I have my own personal view on the most useful features, I earn my living by developing software and writing technical articles, not by making sales pitches so I'll leave this to your own judgement!

In no particular order, the features described in this section are as follows:

  • Multiple Assembly Obfuscation
  • Incremental Obfuscation
  • Overload Induction
  • Selective Exclusion
  • Extensibility
  • Assembly Size Reduction
  • ILDASM Protection
  • Code Optimisation
  • Data Obfuscation

Multiple Assembly Obfuscation - covers the situation where you might have a number of interdependent assemblies, which together may be considered an application unit. Although each individual assembly may define public types, if these are intended to be used just by other assemblies in the same application, then they may undergo symbol obfuscation within the application unit. This means that not only the metadata definition tables are obfuscated, but also the reference tables.

Incremental Obfuscation - this assists multiple assembly obfuscation where a development team may be working on a number of dependent assemblies independently. Incremental obfuscation allows mapping tables to be maintained containing the pre- and post- obfuscated names of the members in an assembly so that a dependent assembly may be subsequently obfuscated and yet still maintain referential integrity without the need to rebuild and re-obfuscate the complete application. This is particularly effective when distributing updated components.

Overload Induction - this takes method overloading to a new level. This patented algorithm is already used in the Java environment and has now been migrated to .NET. Using this technique, method identifiers are maximally overloaded after an exhaustive scope analysis. This makes it much more difficult for a human reader to determine the true target of any method call. Generally, this technique results in a larger number of methods sharing a smaller number of names.

Selective Exclusion - this is an essential feature. In order to allow assemblies to expose a public interface, allow reflection, or undergo serialization, there needs to be some method whereby individual types and/or members can be excluded from the obfuscation process. Some obfuscators allow specification of XML configuration files to assist this exclusion, others allow custom attributes to be defined in the source code. Many products also automatically recognise circumstances where obfuscation should not be applied and either handle this intelligently or flag a warning.

Extensibility - only one of the obfuscators I looked at offers this facility. This allows the development of plug-ins to the obfuscator, which can define customised rules controlling symbol obfuscation.

Assembly Size Reduction - is an inevitable result of symbol obfuscation. If shorter and fewer strings are used in the string heap then the size of the file will be smaller. Removing other redundant metadata (properties and events) also helps keep the size down. This probably provides more of a benefit for deployment rather than performance, especially if components are delivered over the internet - either as ActiveX components, trickle-fed applications, or as a downloadable installation. Generally, any decrease in size is a good thing.

ILDASM Protection - a feature in some products allows the obfuscator to emit some invalid IL code into the assembly. Although this doesn't affect the existing code, it has the side-effect of crashing ILDasm when used to examine it. This is more likely due to a bit of poor coding in ILDasm than any true innovation on the part of the obfuscator but every little helps. Unfortunately this technique is not so effective with commercial decompilers and can't be used in all types of assembly.

Code Optimisation - you might wonder how your code could be optimised even further - having spent so long writing and testing it and then having it compiled not once but twice, by the C# or VB.NET compiler and then the JIT. Apparently though there is still room for improvement. This is another area where I'm still a little sceptical and it only features in the "coming soon" category of some products. From what I understand, it is possible to identify redundant methods and unreachable code and to restructure data to reduce size and improve performance. Another area to keep an eye on.

Data Obfuscation - this is as yet a non-existent feature, but it has been proposed as a potential technique by at least one of the obfuscator vendors. This technique involves obfuscating not the names within an assembly but the way in which data is represented. For example, converting certain local variables into global variables or changing multi-dimensional arrays into single-dimension arrays. I can see some value in this so it's yet another area to keep a watch over.

Strengths and Weaknesses

We've seen in principle what can be achieved by obfuscation and we already banged the point home earlier that there is no way to prevent reverse engineering at some level. So what are the benefits we can gain by using an obfuscator?

Well, first off, it's easy to do. You don't need to change the way you write software and your developers don't need any additional training. Maybe changing the way you design your systems could maximise the degree of obfuscation possible but in general terms you don't need to think too hard.

Fitting an obfuscator into your development environment is pretty easy. Some products offer just a command-line tool, which can be invoked in a post-build step, while others offer a product that integrates with the Visual Studio .NET IDE and allows you see the effect of the obfuscation while you work. Some provide both. Also, team-based development is assisted by features like incremental obfuscation.

If you've invested a great deal of time and money in developing your software and you want to keep a competitive advantage then you'll probably agree that a commercial obfuscator is very good value for money. Compared to the normal level of expenditure on software development tools, enterprise level obfuscation is still cheap.

There's no effect on deployment. You don't need to distribute any runtime code or libraries as everything happens within the assembly itself. You can distribute it just like any other assembly and it may even be a little smaller.

It definitely works. Try one yourself and you'll see that the results you get back from a decompiler are more difficult to understand. If your application is large or complex beforehand, this additional confusion can thwart many human hackers. Almost certainly a decompiler will not be able to produce valid compilable code following obfuscation. Don't forget though that it's not just humans that need to be confused - decompilers can't be baffled by psychological tricks.

In short, an obfuscator will deter casual hackers and it will buy you time if someone is really determined to reverse engineer your code. How many people are deterred and how much time you can buy are dependent on a couple of weaknesses in this technique. Not weaknesses in the products, but in the principles.

The first problem is that you often can't obfuscate everything. If you expose a public interface or use serialization or remoting, then these parts of your code won't work if they are obfuscated. Also, inheritance constraints may mean that some identifiers - mostly virtual methods - cannot be renamed (for example, you cannot rename an overridden ToString() method).

The second problem is that the IL code is still available in the obfuscated assembly. All that's happened is that it is more difficult to examine this in the correct context. If the part of your assembly you need to protect is a license code validation, for example, this is probably an important issue. All that a hacker needs to do is to find the method(s) that handle this validation and then examine the IL code with ILDasm or a commercial decompiler. They won't need to understand the whole application, they just need to pinpoint a relatively small area of interest and, once they've done so, obfuscation can only help so much.

This last point brings us nicely to a slightly different product, which has a new way of tackling this particular problem.

Other Solutions

One recently-released product that deserves a mention is Remotesoft's .NET Protector. This is not an obfuscator - it doesn't change any of the metadata in the assembly. It is primarily concerned with protecting IL method bodies from decompilation.

As it doesn't tamper with the metadata, it has none of the drawbacks mentioned earlier. It works fine with reflection, serialization and remoting, and the public members of the assembly need no special treatment. Also, it requires no additional security privileges, is compatible with both pure .NET images and mixed images, and performs as well as a pre-JITted assembly. The IL ordinarily contained in the assembly is replaced by what the vendor describes as pseudo-native code, which is neither IL nor native and which is encrypted for good measure. The only apparent limitation is that, at present, support is limited to the Windows platform and the one downside is that the pseudo-native code is typically larger than the original IL.

So far, this sounds pretty good. The Windows platform constraint is bit of a non-problem at the moment as this will probably be the platform of choice for an awful lot of serious .NET development in the foreseeable future. The ability to protect decompilation of the IL while exposing all of the original metadata is the complete converse of the way an obfuscator works and yet it seems perfectly sensible. Where an obfuscator will try to make it difficult for you to find your way around a decompiled assembly so that you hopefully never get to the sections of code you're interested in, the .NET Protector effectively allows you full visibility of the structure of the types in an assembly, confident in the fact that this is much as you'll ever get.

At runtime, the pseudo-native code is decrypted and transformed into true native format and then is executed by the CLR as normal. According to the vendor, the performance is the same as a pre-JITted image although native-related invocations, such as P/Invoke and COM Interop should be faster. In addition to protecting IL code, the .NET Protector also allows string encryption, resource encryption and the option of runtime password protection.

At the time of writing, I don't have any technical details of how the product works. What I do know is that the pseudo-native code is pretty much just native code and that the encryption stage is not crucial to keeping the IL protected. This would suggest that there must be at least a partial compilation of IL during the protection process and either the vendors have created their own IL compiler or they utilise existing facilities available in the .NET runtime. Certainly the claim that performance matches a pre-JITted image supports the latter, but I really have no hard facts, just guesses. The .NET framework includes JIT compilers for all CPU configurations so the limitation of Windows as the development and deployment platform may suggest that these are used at some point in the process. Again, I don't know. I haven't been able to confirm whether or not the JIT compiler is used at all at runtime but, if not, then the decrypted pseudo-native code must be completely valid as far as the CLR is concerned as the protected .NET assembly is not only valid according to the Microsoft .NET specification but is also fully verifiable. It's also not clear whether there is a runtime component to .NET Protector, although I can certainly envisage situations where this would be helpful.

This is an intriguing solution to a difficult problem and I think that this product, if it really does all it claims, would be a more secure method of protecting an assembly from decompilation. I shall certainly be monitoring this area with great interest.

If you're now completely bewildered by the options available for protecting your code, we're going to make it worse by describing yet another technique. This idea is a product of my own devious mind and uses just native C# and the FCL - I've not seen it used elsewhere so this may be of interest in its own right or it may generate ideas for alternative solutions. Here goes ...

The Sample Application

Our goal is to make it as difficult as we can for someone to inspect our assemblies by accessing the metadata. The way they do this may be by using reflection (as we discovered in part one of this case study), by using a commercial decompiler or simply by inspecting the assembly with ILDasm. In all cases, we need to hide this information in some way that makes it less accessible to the more casual hacker.

I'll say again that, in tackling this problem, all we're really doing is reaching a compromise. There's no way we can absolutely prevent someone reverse engineering our assembly if they're really determined to do so. The only way to be 100 percent protected is to never release your compiled assemblies into the public domain (i.e. keep everything server-side) but obviously this isn't realistic for every case. So, given that some bad guys have a copy of your compiled code then, given enough time and expertise, they will be able to figure out exactly what it does and how it does it. This has always been true and always will be. If your code can't be reverse engineered, then it also won't run on a computer.

The compromise is based on the importance we place on keeping our code secure and the amount of time and trouble we are prepared to invest to make this happen. Earlier we discussed obfuscators and alternative techniques such as using unmanaged code for implementing particular parts of a system. This is fine, but it still means that we have to accept the associated overheads. Obviously, if we're using unmanaged code, we can't take such great advantage of the numerous benefits the .NET development environment provides to a developer. We have to retain skills in these areas and accept any consequential costs. There will always be some sort of balance to be made.

What I want to try to achieve is, at least for some cases, a better compromise and I believe this may be possible by the judicious use of encryption.

.NET Cryptography

Within the FCL resides the System.Security.Cryptography namespace. This allows programmatic access to a number of cryptographic services through a relatively simple interface. It is certainly easier to use than the underlying Crypto API, which was first introduced by Microsoft in 1996. Unlike this API, the programming model exposed by the System.Security.Cryptography classes is stream-based. For most cryptographic transformations (encryption, decryption, hashing), the CryptoStream class, which derives from System.IO.Stream, is used. This takes care of all necessary buffering and makes it very easy to stream data from other sources (file streams, memory streams, network, etc.) through an encryptor or decryptor to achieve the desired result. We'll see this in action shortly.

The System.Security.Cryptography namespace supports a number of common algorithms covering three basic cryptographic techniques:

  • Symmetric cryptography
  • Asymmetric cryptography
  • Hashing

Symmetric cryptography is sometimes called secret key encryption. This is where the encryption algorithm requires the same key for both encryption and decryption. The advantage of this is that these algorithms can be made very fast and efficient. However, the primary weakness is the problem of key exchange. The mechanism for ensuring that both the sender and receiver of an encrypted message have the same key and that it remains secure poses a number of logistical problems, particularly where the sender and receiver never physically meet. This was the prime motivation for the development of asymmetric algorithms.

Asymmetric cryptography is also known as public key encryption. In this technique, the key used to decrypt a message (the private key) is different from the key used to encrypt the message (the public key). The public and private keys are mathematically related, but it is computationally infeasible to derive the private key from the public key. In use, the sender of a message would first use the intended recipient's public key to encrypt the content and the recipient would use his matching (but secret) private key to decrypt the message. As the roles of the public and private keys may also be reversed, a variation of this technique is used to produce a digital signature. If a sender encrypts a message using his private key, then any and all recipients should be able to decrypt it using the sender's public key. As the sender is the only person in possession of the private key, he is the only person who could encrypt the message in this way and a successful decryption therefore implies that the message did indeed originate from him. What's more, he can't deny it (this is known as non-repudiation). Thus, an item of data encrypted with a private key is called a digital signature. A common practice is to use a message digest as the item of data to be encrypted - read on.

Hashing doesn't per se involve any encryption of the message. It is a technique whereby the message content is used to produce a fixed-length digital fingerprint, also known as a message digest. This has the property that the same process applied to the same message will produce the same digest, so that if any small change is made to the message content, the message digest will also change. Furthermore, it is highly unlikely that any two different messages would produce the same digest. As described above, when producing a digital signature, the original content of the message is first used to produce a message digest. This is then encrypted using the originator's private key and transmitted along with the message. The message itself need not be encrypted, but the accompanying digital signature, which may be decrypted using the originator's public key and compared with a recomputed message digest, ensures that the message really was from the originator and that it had not been tampered with. Algorithms employed to produce a digest are known as hash functions.

The .NET framework class library contains support for the common symmetric (DES, Triple-DES, RC2, Rijndael), asymmetric (RSA, DSA) and hash algorithms (MD5, SHA1, SHA256, SHA384, SHA512) in the System.Security.Cryptography namespace. In addition, the namespace System.Security.Cryptography.XML contains a full implementation of the W3C standard for digitally signing XML. The majority of the .NET cryptography algorithms are implemented as managed wrappers over the Crypto API although there are a handful that are implemented directly in managed code. The former tend to have class names ending with "CryptoServiceProvider" (e.g. RSACryptoServiceProvider, DESCryptoServiceProvider) while the latter have class names ending with "Managed" (e.g. RijndaelManaged, SHA1Managed).

Outline Design

What I intend to do is really rather simple. The idea is that the public and private aspects of the application you wish to protect are developed either as a number of interdependent assemblies or as separate modules within an assembly. This is quite possibly a natural design choice in many applications anyway and is in no way counterintuitive. This type of design fits well with a number of standard design patterns, for example the Adapter, Bridge, Façade and Proxy structural patterns, which all serve to promote a weak coupling between a subsystem and its clients. We can take advantage of these situations to physically remove the private implementation of a public interface into a separate assembly. This makes it easier to secure.

The use of modules, allowing the development of multi-file assemblies, requires command line compilation. There's currently no support in Visual Studio .NET for this type of development so, for this reason, the example I'll show will use a Windows Forms .exe as the public part of the application and a .NET class library (a DLL containing metadata and IL) as the private part. In fact, we'll use the application we developed in the first part of this case study as our target. This was the reflection-based decompiler, DeComp. We're going to try and protect our decompiler from decompilation!

The original design resulted in a single .NET assembly containing the main classes we developed to undertake the decompilation. These were Generate, a class derived from System.Windows.Forms.Form to which we added a number of private methods used during decompilation, and ILReader the class we used to decompile the IL code for each method body. There were also a small number of additional classes associated with and supporting ILReader. This placed everything in the one assembly and so made everything visible through the same metadata.

What we will do is to remove the private methods we added to Generate into a new class, called Decompiler, and place this new class along with ILReader and its supporting classes into a separate class library (a .NET dll). The modified Generate class, which will still descend from System.Windows.Forms.Form will still handle the UI interface but will invoke an instance of Decompiler to do the dirty work. We'll go through this is more detail later but for now you just need to appreciate the reason for splitting the application this way (apart from producing a better design of course!).

The end effect is that the code we want to protect should be physically contained in its own file (be it a class library or a .NET module), which we can distribute in an encrypted form. We can then take advantage of the way the CLR attempts to find and load these files at runtime to decrypt the assembly on the fly and ensure that the decrypted version only exists in memory while the host application is running. This immediately raises the stakes and requires a lot more work on the part of a would-be hacker to identify, isolate, decrypt and inspect the protected code. Like I said before, this is in no way foolproof (I'll discuss the limitations later) but it does take a lot of players out of the game and doesn't place too much overhead on the part of the developer to implement. We also don't lose any of the benefits of the .NET development environment nor do we need to perform any obfuscation (although we still could).

The CLR really helps us here because it allows us to trap missing modules and/or assemblies programmatically at run time and to write our own code to supply the missing item as the return value of an event handler. This is not something we could do before .NET. If a Win32 executable indicated a dependence on a particular library (dll) and this library could not be found by the operating system when the application was loaded, then we were faced with an unrecoverable error. The only way around this was to use LoadLibrary() and GetProcAddress() to provide dynamic linking at run time. Not only is this more messy to code but it also hides the true dependencies of an executable and vastly complicates version control.

Before we go into the details of reworking the design of our decompiler, we need to develop a utility to allow us to encrypt a compiled assembly. We'll do that now.

Encrypting an Assembly

For the situation outlined above, the use of a symmetric encryption algorithm is the obvious choice. This does have the drawback that, to be really secure, we need to keep the key secret. As we will be decrypting the assembly on an end user's desktop at runtime, there's really no way to achieve this, especially as we'll be performing this part of the operation from a straightforward unencrypted, unobfuscated assembly, which can be freely examined by ILDasm.

But never forget that we aren't aiming for the unattainable 100% protection - we're only making it a little harder. The way to keep this as safe as possible is to be more inventive with the way we get hold of the secret key at runtime. At one end of the spectrum, we might have a string constant nicely visible in the metadata with a name like secretkey, while at the other we might supply the key over a secure internet channel at runtime and make sure we authenticate the user before doing so. We could even supply the encrypted assembly over the internet at runtime and so make it even more inconvenient to decrypt. Whatever happens, it will always be possible for someone with the right knowledge to obtain the key and so decrypt our assembly, so all I'll do here is not make it so obvious.

When the CLR decides that it can't find an assembly, it fires an event and passes in a string argument containing the full name of the missing assembly. This includes the name, version number and culture and more or less uniquely identifies the assembly. For convenience then, and to avoid hardcoding some form of password in the metadata, we'll use this full assembly name as the basis for our encryption key.

We could just use the string itself (or part of it) as a key, but this would be naïve and is not something that would ever be done in a real world situation. What usually happens is that a character-based password is used to derive the actual encryption key, which is of a pseudo-random nature. We can do this using the PasswordDeriveBytes class, which basically takes an incoming character-based password and transforms it into a strong key.

The algorithm is deterministic, so although the output is pseudo-random, it will produce the same result when presented with the same input. This is essential if we ever want to decrypt the data using the original password! There are a number of ways to use this class but the way we'll use it here is to pass in our variable length character-based password in the constructor, then use the GetBytes() method to retrieve a pseudo-random byte array of a fixed length. So, assuming we have the full assembly name in the string fullname, we can get a 192 bit key like this:

PasswordDeriveBytes db = new PasswordDeriveBytes(fullname, null);
byte[] namekey= db.GetBytes(24);

Note that 24 bytes equals 192 bits (24 x 8) - I'm sure you get the idea.

Let's put this into practice. From Visual Studio .NET, create a new Windows Form project called EncryptAssembly. Rename the Form1 class to MainForm, add a MainMenu, an OpenFileDialog, and a SaveFileDialog. Name the dialogs dlgopen and dlgsave respectively. I prefer to build a Windows Forms application just for the convenience of using these dialogs to enter the input and output filenames, but we could just as easily build a console application and accept the filenames as arguments on the command line. Maybe I'm getting too lazy.

So this is just a simple GUI to wrap our encryption method. I've added a label with some descriptive text and also added Open and Exit options to the main menu. The result is shown in Figure 7.

Add a click event handler for the File/Open menu option and present the open and save file dialogs in turn:

private void menuFileOpen_Click(object
            sender, System.EventArgs e)
{
// prompt for file to
            encrypt
if (dlgopen.ShowDialog() == DialogResult.OK)
{
//
            prompt for file to save result
if (dlgsave.ShowDialog() ==
            DialogResult.OK)
{
Encrypt(dlgopen.FileName,
            dlgsave.FileName);
MessageBox.Show("Encryption
            completed");
}
}
}

Figure 7 - Encrypt Assembly UI

Also add a handler for the File/Exit option to close the application:

private void menuFileExit_Click(object sender, System.EventArgs e)
{
// exit application
this.Close();
}

Now we can write the Encrypt() method, which does all of the work. We'll need to import a few namespaces:

using System.IO;
using System.Security;
using System.Security.Cryptography;
using System.Text;
using System.Reflection;

Then start the Encrypt() method as follows:

private void Encrypt(string fromfile, string tofile)
{
// attempt to encrypt given assembly
string fullname;
ICryptoTransform desencrypt;

The ICryptoTransform object will be used to hold an interface to one of the .NET cryptography symmetric encryption classes. In this example we'll use the Triple-DES algorithm, but as the above interface defines the basic operation of any cryptographic transformation, we could plug in a different algorithm pretty easily.

Next we want to load the assembly we're going to encrypt. We do this for two reasons - first, to make sure it really is an assembly, and second, to get the full name, which we need to generate the key:

// check that input file is an
            assembly
try
{
// load assembly and retrieve full
            name
Assembly assembly = Assembly.LoadFrom(fromfile);
fullname
            = assembly.FullName;
}
catch(Exception ex)
{
// failed
            to load assembly
throw new Exception("Unable to load assembly - "
            +
fromfile + "\n" + ex.ToString());
}

We can use PasswordDeriveBytes to get the strong key and also a 64 bit initialisation vector (IV):

// create a 192 bit key and 64 bit iv using the full assembly name
PasswordDeriveBytes db = new PasswordDeriveBytes(fullname, null);
byte[] namekey= db.GetBytes(24);
byte[] nameiv= db.GetBytes(8);

The initialisation vector is a random number used as a starting point in the encryption process. If we used the same key to encrypt the same data, but with a different initialisation vector, we would get two different results. Effectively, it adds a little more randomness into the process.

One of the reasons for using an IV is when encrypting a known file type (e.g. a Microsoft Word document or a PE file) where the first few bytes of the unencrypted file might be in some way predictable. In this situation, knowing what the first few bytes are likely to be gives an attacker a foothold when trying to decipher the data. By using an IV, these bytes are encrypted in a much less predictable way and make it considerably more difficult for the encryption key to be discovered. It's not really relevant to our scenario because it would be much easier for the key to be obtained by reverse engineering the decryption process at runtime than by undertaking a conventional cryptographic attack. I've used it here for good order and it's useful to know for other situations.

The next thing we do is to select which encryption algorithm we want to use. This is very simple using the classes provided by the System.Security.Cryptography namespace, all of which provide "safe" defaults for any uninitialised members. There's no danger that you'll inadvertently produce a weak encryption by using these classes - they will generate a random encryption key and initialisation vector for you if you fail to supply them. However, if you also fail to save these generated values, you may never be able to decrypt your own data!

This is how we create an encryptor for the Triple-DES algorithm:

try
{
// create a tripleDES encryptor
TripleDESCryptoServiceProvider DES3 =
new TripleDESCryptoServiceProvider();
DES3.Key = namekey;
DES3.IV = nameiv;
desencrypt = DES3.CreateEncryptor();
}
catch(Exception ex)
{
throw new Exception("Unable to create encryptor - " +
ex.ToString());
}

I've used a try/catch block mainly because different algorithms accept different sized keys. If we decided to change the algorithm and forget to adjust the key length, the above code will trap and report the error. The valid key sizes for each cryptography algorithm are not terribly well explained in the .NET framework documentation so the best place to go for this information is to examine the LegalKeySizes property of the encryption class itself.

This property is an array of KeySizes, each giving the minimum and maximum key sizes (in bits) together with what is known as a skip size. This is a fixed increment between valid key sizes. For example, the RC2 algorithm implemented by the RC2CryptoServiceProvider class has a minimum key size of 40, a maximum key size of 128, and a skip size of 8. This means that the valid key sizes are 40, 48, 56, 64, . 120, 128 bits.

Getting back to business, we've now got all we need to encrypt the assembly. All we do is to stream it from the input file into a byte array in memory, then stream it back from memory to the output file through the encryptor:

// encrypt fromfile into tofile
try
{
// read input file into byte array
FileStream fsInput = new FileStream(fromfile, FileMode.Open,
FileAccess.Read);
byte[] bytearrayinput = new byte[fsInput.Length - 1];
fsInput.Read(bytearrayinput, 0, bytearrayinput.Length);
fsInput.Close();

// write encrypted version
FileStream fsEncrypted =
new FileStream(tofile, FileMode.Create, FileAccess.Write);
CryptoStream cryptostream =
new CryptoStream(fsEncrypted, desencrypt, CryptoStreamMode.Write);
cryptostream.Write(bytearrayinput, 0, bytearrayinput.Length);
cryptostream.Close();
fsEncrypted.Close();
}
catch(Exception ex)
{
// catch error during file handling
throw new Exception("Error encrypting file - " +
ex.ToString());
}
}

You see how we use a CryptoStream to do this? We first create a normal FileStream giving access to our output file, then we supply this FileStream together with the ICryptoTransform encryptor to the CryptoStream constructor. As far as our code is concerned, this works just like a normal stream so we just write our byte array out to it and then tidy up. Behind the scenes, the CryptoStream uses the algorithm we supplied to encrypt the data and then uses the FileStream we supplied to write the encrypted version out. It's difficult to see how this could be any simpler. You'll see later how the reverse process is just as simple, we can use any of the methods available for streams and just plug in an encryptor or a decryptor and pump our data in or out as we please. The rest gets handled automatically and we hardly need to think about it.

Using an Encrypted Assembly

Now we'll rework the design of the DeComp decompiler according to the earlier outline design. We'll create a new solution for this and call it CSDecomp (for C# Decompiler). The solution will contain a Windows Forms application, also called CSDecomp, and a C# class library called CSDLibrary (see Figure 8).

I don't intend to regurgitate the details of DeComp, which we covered in great detail in part one of this case study. If you don't have access to the download material from part one, and want to build the new version as we go, then you'll need to get hold of it now. Otherwise, the complete source for this version of the decompiler is available with the download material for this case study.

Figure 8 - C# Decompiler

Add a reference to CSDLibrary from CSDecomp and add the following using statements to the latter:

using System.Reflection;
using System.IO;
using System.Security;
using System.Security.Cryptography;
using CSDLibrary;

As before, in CSDecomp, rename the Form1 class to Generate and save the project. Then, in CSDLibrary, rename the default Class1 to Decompiler and add the ILReader classes in the existing Reader.cs source file.

Remember from the outline design that what we need to do is to move anything we want to protect into a separate assembly. Figure 9 shows the situation we had with DeComp, where everything was accessible in one assembly, while Figure 10 shows the picture we're aiming for now:

Figure 9 - DeComp original design

Now, copy the declarations for the following member fields from DeComp into the Decompiler class in CSDLibrary :

private CodeCompileUnit codeunit;
private ILReader reader;

Next, add another private member field that we'll use to hold the decompiled source:

private string decompiledsource;

Then copy in the following private methods from DeComp :

private CodeAttributeDeclarationCollection getCustomAttributes(object[] attrs)
private bool isMethodDefined(MethodInfo meth, Type type)
private MemberAttributes getMemberAttributes(MethodInfo info)
private MemberAttributes getMemberAttributes(ConstructorInfo info)
private MemberAttributes getMemberAttributes(FieldInfo info)
private CodeParameterDeclarationExpressionCollection
getParameterDeclarations(ParameterInfo[] infoarray)
private void addType(Type aType, CodeTypeDeclaration owner)
private void addConstructor(ConstructorInfo info, CodeTypeDeclaration owner)
private void addEvent(EventInfo info, CodeTypeDeclaration owner)
private void addField(FieldInfo info, CodeTypeDeclaration owner)
private void addMethod(MethodInfo info, CodeTypeDeclaration owner)
private string getArgumentList(ParameterInfo[] parms)
private void addProperty(PropertyInfo info, CodeTypeDeclaration owner)
private CodeNamespace findNamespace(string name)
private CodeCommentStatement[] GetILForMethod(MethodBody methodBody)

Figure 10 - CSDecomp amended design

That gets the bulk of the code into the new class library. Add the following imports for the namespaces we're using:

using System.CodeDom;
using System.CodeDom.Compiler;
using System.IO;
using System.Text;
using Microsoft.CSharp;
using System.Reflection;
using System.Reflection.Emit;
using System.Reflection.ILReader;

Then add a read-only property to allow the string of decompiled source code to be retrieved:

public string Source
{
// read-only
            property for decompiled source
get
{
return
            decompiledsource;
}
}

To finish off the Decompiler class, we'll provide a constructor that accepts a fully qualified filename to the assembly which needs to be decompiled. The constructor will then undertake the decompilation at once.

This is perhaps not the way we would normally design a constructor. Ordinarily, we would want the constructor to be as short and fast as possible, limiting its activities to simple member initialisation. So we would probably have defined a parameterless constructor and then provided a second public method to perform the actual decompilation. I've deliberately designed the constructor this way so that the absolute minimum amount of information is held in the referencing assembly.

If we're trying to keep as much information as possible away from the public gaze, we can slightly compromise the ideal design to this end (note the word slightly !). This way, all that is visible from the referencing assembly is the presence of a class called Decompiler, which has a property called Source. Nothing else is visible.

So the constructor is as follows:

public Decompiler(string
            filename)
{
// create compile unit and load assembly for
            reflection
codeunit = new CodeCompileUnit();
Assembly
            mainAssembly = Assembly.LoadFrom(filename);

// load for
            ILReader
reader = new ILReader(mainAssembly.GetModules()[0], new
            AssemblyLoader());

// reflect over all types in the
            assembly
Type[] allTypes =
            mainAssembly.GetTypes();
foreach(Type t in allTypes)
{
//
            add top-level types to codedom compile unit
if (t.MemberType ==
            MemberTypes.TypeInfo)
addType(t, null);
}

// create
            code generator
CSharpCodeProvider cs = new
            CSharpCodeProvider();
ICodeGenerator csGen =
            cs.CreateGenerator();
CodeGeneratorOptions options = new
            CodeGeneratorOptions();
options.BracingStyle = "C";

//
            generate code and save result
StringWriter sw = new
            StringWriter();
csGen.GenerateCodeFromCompileUnit(codeunit, sw,
            options);
decompiledsource = sw.ToString();
}

The above code is basically similar to the code we previously had in the File/Open menu handler in the DeComp project.

To finish off the CSDLibrary assembly, we just need to add the AssemblyLoader class into the CSDLibrary namespace. This class was also explained in part one of the case study so I won't repeat that here:

class AssemblyLoader : IAssemblyLoader
{
public Assembly Load(string assemblyName)
{
return Assembly.Load(assemblyName);
}

public Assembly LoadFrom(string fileName)
{
return Assembly.LoadFrom(fileName);
}
}

Now, we can code the public part of the application in the Windows Forms project CSDecomp. This will have the same UI as the original DeComp, basically a menu, a file open dialog, and a text box to display the result.

By using the Decompiler class from CSDLibrary, the File/Open menu handler is much simpler:

private void menuFileOpen_Click(object
            sender, System.EventArgs e)
{
// open file and decompile
if
            (dlgopen.ShowDialog() == DialogResult.OK)
{
textBox1.Text =
            "";
Decompiler decomp = new
            Decompiler(dlgopen.FileName);
textBox1.Text =
            decomp.Source;
}
}

This is all we need just now to be able to build and test the project, which - take it from me - works just fine. Again, I won't repeat the testing we performed in part one.

This is where we can add the code needed to locate and decrypt the encrypted version of CSDLibrary.dll that we'll distribute with the finished application. Apart from the additional code that I'm going to show you in a moment, we don't need to change the way we develop the application at all. We can continue to make changes both to CSDecomp and CSDLibrary and the only additional step needed will be to use EncryptAssembly on the release version of CSDLibrary.dll before we distribute the application.

So at this point, run EncryptAssembly, select CSDLibrary.dll as the input file, and save the encrypted version in a file named CSDLibrary.enc. You can then delete the original dll to make sure we're really decrypting the hidden version at runtime. Before this, we need to make some more changes.

The first change we'll make is to add an event handler to catch the situation where the CLR cannot find the referenced assembly. We want to do this before we attempt to reference any of the types in the missing assembly, so a good place to do this is in the Main() method:

[STAThread]
static void Main()
{
// trap unresolved assembly references
AppDomain appdom
            = AppDomain.CurrentDomain;
appdom.AssemblyResolve += new
            ResolveEventHandler(MyResolveHandler);

// show main
            form
Application.Run(new Generate());
}

If, as shown, we're trapping a missing assembly, we register a handler for the AssemblyResolve event in the current AppDomain (as shown). For a multi-file assembly, we could handle an encrypted module in a similar manner by trapping the ModuleResolve event of the assembly.

The event handler itself is another static method, which starts as follows:

static Assembly MyResolveHandler(Object sender, ResolveEventArgs e)
{
// handle missing assembly
string path = Path.GetDirectoryName(Application.ExecutablePath);
string filename = path + @"\CSDLibrary.enc";

Here, we build the fully qualified filename for the encrypted version of the missing assembly. It's worth noting that, as we get the full name of the missing assembly in the ResolveEventArgs parameter, we could have used this to derive the filename of the encrypted version rather than hardcoding the name as above. Indeed, if we had multiple encrypted assemblies, this would be an ideal way to do it as the same routine could handle any missing assembly without change.

I've kept things as simple as possible for our single encrypted assembly but there's plenty of scope for including additional complications at this point if you wish. One option might be to hold encrypted assemblies as embedded resources rather than as separate physical files. This would introduce one more hurdle to be overcome for someone trying to access the encrypted code.

Now that we've got the path to the encrypted file and the full name of the missing assembly, we can reconstruct the encryption key and initialisation vector that we used during encryption:

// recreate 192 bit key and 64 bit iv using the full assembly name
PasswordDeriveBytes db = new PasswordDeriveBytes(e.Name, null);
byte[] namekey= db.GetBytes(24);
byte[] nameiv= db.GetBytes(8);

In a similar manner to the encryption process, we create a Triple-DES decryptor using these values:

// create a
            decryptor
TripleDESCryptoServiceProvider DES3 = new
            TripleDESCryptoServiceProvider();
DES3.Key = namekey;
DES3.IV
            = nameiv;
ICryptoTransform desdecrypt =
            DES3.CreateDecryptor();

Decrypting the assembly is then a simple matter of creating a CryptoStream and reading the decrypted data from the file straight into memory in a byte array:

// decrypt file into a byte array
FileStream fsInput =
new FileStream(filename, FileMode.Open, FileAccess.Read);
CryptoStream cryptostream =
new CryptoStream(fsInput, desdecrypt, CryptoStreamMode.Read);
byte[] bytearrayinput = new byte[fsInput.Length - 1];
cryptostream.Read(bytearrayinput, 0, bytearrayinput.Length);
cryptostream.Close();

Like we did during encryption, we define an underlying FileStream to get the source data, combine this with our decryption algorithm to get a CryptoStream, then just read from the CryptoStream to retrieve and decrypt the data in one. So we end up with a decrypted version of CSDLibrary.dll in a memory array.

The static Load() method of the Assembly class conveniently has an overload that accepts a byte array as input, so we can recreate the missing library as an Assembly object and return it as the result of the event handler:

// load assembly from byte array
Assembly assembly = Assembly.Load(bytearrayinput);
return assembly;
}

And that's it. The CLR now has the assembly in memory, just as if it had loaded it itself, the additional working memory we needed for decryption will be reclaimed by the garbage collector, and all we've added by way of an overhead is the one-off performance hit while we performed the actual decryption:

Figure 11 - ILDasm references to hidden assembly

The private assembly never exists in a decrypted format except in memory and so the best anyone could do with a decompiler is to retrieve details of the public part of the application referenced in CSDecomp.exe. This doesn't give much away at all, as the output from ILDasm shows in Figure 11.

To get any more than this, it would be necessary to disassemble the MyResolveHandler() method and write a similar method in a new application to perform the decryption using this same technique. This is still well within the capability of a determined hacker but takes more than a quick browse with a decompiler or ILDasm and is quite easy to complicate further.

I already mentioned the possibility of alternative methods of key exchange - maybe the key could be supplied at runtime by user input, internet access, etc. Perhaps the decrypted assembly could also be made less accessible - embedded as a resource in the executable, or again supplied at runtime over an internet connection. There are a number of possibilities.

We could even take the trouble to use a commercial obfuscator in addition to the above and so place even more obstacles in the way. This is the same sort of game that's already played between software developers and hackers (for the want of a better word - I mean anyone who is intent on reverse engineering your code for whatever reason).

Application Setup

If you want to try this out for yourself, download the support material for this case study from the C# Today web site and follow the instructions in the ReadMe.txt file included. The download contains the CSDecomp project and the CSDLibrary class library described above as well as the EncryptAssembly source code.

Limitations

We discussed the limitations of obfuscation and other techniques earlier. This section covers the limitations of using an encrypted assembly as described above.

The main problem is that the code that locates and decrypts the "hidden" assembly is held in a conventional .NET executable and so is easy to locate and decompile. It would be possible to use an obfuscator to make this more difficult, but it would probably be better to implement this method in unmanaged code. The ability to decompile the decryption routine in the above example means that it becomes clear exactly where the encrypted assembly is located and what key was used to encrypt it.

A stronger method would be to ensure that the key could not be determined just by examining the static code. If the key was instead received via a network or internet connection, or even provided by a call into unmanaged code, then this would necessitate some dynamic tracing of the process at runtime in order for the key to be found. While still possible, this would require a higher level of expertise and possibly some additional software.

Case Study Review

In part one of this case study, we showed how vulnerable an unprotected .NET assembly is to reverse engineering and decompilation. The intention of this part of the case study was to examine a range of options that might be used to mitigate the effects of this exposure. We looked at the use of unmanaged code, native code generation and commercial obfuscators, then went on to develop an alternative technique using encryption.

Somewhere in this range of options - either singly or in combination - there is probably a solution for anyone concerned with protecting their investment in intellectual property. We already know (from the number of times I've repeated it!) that it is not possible to absolutely prevent reverse engineering as long as you distribute executable code. But the amount of information contained in the metadata of a .NET assembly, together with the high-level nature of IL code, has caused a lot of concern and the absence of any inbuilt method in the CLR to encrypt or otherwise hide the details of a method's implementation has been viewed as a major shortcoming of .NET.

Personally, I'm not yet sure of the real importance of this problem. I'm convinced that a lot of the adverse reaction has been a knee-jerk response to the sudden change brought about by these accessible, self-describing components and, in many cases, there's probably little real danger of someone losing out financially due to reverse engineering attacks. In the Win32 world we're just not used to our source code being publicly exposed and never have been. It's perhaps a bit of a shock to have the curtains pulled away to reveal your every coding secret in its naked glory. This is quite a contrast to traditionally open environments like the various flavours of Unix, yet these are still commercially successful.

Looking at the reality, there are not many categories of application that are genuinely sensitive to this exposure. Taking a different tack, we can look around at any number of open source developments that mirror popular commercial products and wonder how much of this freely available code implements techniques that are considered trade secrets by the originator. Does this exposure have an effect? Are there just normal market forces coming into play, where commercial advantage is really achieved by bringing out new features rather than copying someone else's old ones? Why would anyone really take the trouble to reverse engineer your product?

Well, when writing this case study, I really wanted to get inside some of the commercial obfuscators. I wanted to know in detail how they worked, how they might be beaten by a decompiler. But did I launch into a major reverse engineering exercise? No - I just sent e-mails to the companies concerned and asked them to tell me how their products work. I looked at their web sites, downloaded FAQ's and whitepapers. I didn't need to look at their source code - they were all happy enough to explain it to me!

This is perhaps a valid point. Consider encryption, where one might think it's crucial to keep the details of encryption and key generation algorithms out of the clutches of the bad guys. If you do think that, then you're 100% wrong. The very best cryptographic techniques are completely open and subject to scrutiny from experts worldwide - they have to be if they are to be recognised for what they are. If someone offers you "new" encryption software with a "secret" algorithm, avoid it. For these products, using a well-known and trusted algorithm, providing technical documentation explaining their implementation, and making the source code available are all signs that a vendor is confident about its product's security.

Ok, so I'm tending to talk about the ends of the spectrum and most people will fall somewhere in the middle. Yes, there are portions of code that need to be kept private, and this is not because they use weak algorithms or are poorly designed. You should now have a much better idea of the options available for keeping private things private, but my last and most important point is this - before you spend a lot of time and effort preventing someone reverse engineering your systems, take the time to really consider exactly how much harm they can do anyway.

It might save you a few headaches.

Sleep well - and don't have nightmares ..

Case Study Information
Author | Roger Rowland
Editor | Adam Ryland
Reviewers | Benny Mathew, Sean Schade, Dinar Dalvi

If you have any questions or comments about this case study, or suggestions for further pieces, please contact the technical editor. If you wish to talk about the case study with other subscribers, please visit our forum here.

C#Today - © 2002 WroxPress Limited


Protecting Your Code - Reflection, Obfuscation and Encryption - Part 1

Protecting Your Code - Reflection, Obfuscation and Encryption - Part 1
http://www.csharptoday.com/content.asp?id=1852
Article Rating: 5
Published on August 5, 2002
Abstract
The System.Reflection namespace provides the means whereby the public and private members of a type’s implementation may be accessed from the metadata held within every .NET assembly. There is no way to prevent this, and along with the accessible nature of intermediate language (IL) code, this has been the cause of much concern when considering the protection of intellectual property. In the first of two parts, this case study by Roger Rowland examines just how far reflection alone can be used to reverse engineer a compiled .NET assembly into C# source code. In the second part of this series, we will examine exactly what can be done to protect sensitive code from prying eyes.
Case Study

The System.Reflection namespace provides the means whereby the public and private members of a type’s implementation may be accessed from the metadata held within every .NET assembly. There is no way to prevent this and, along with the accessible nature of intermediate language (IL) code, this has been the cause of much concern when considering the protection of intellectual property. In the first of two parts, this case study examines just how far reflection alone can be used to reverse engineer a compiled .NET assembly into C# source code. In the second part of this series, we will examine exactly what can be done to protect sensitive code from prying eyes.

Introduction

Reflection is a major aspect of .NET. It allows dynamic discovery of the types contained in an assembly along with the fields, properties and methods defined therein. It underlies many of the advantages of the .NET framework?late binding to objects in dynamically loaded assemblies, type browsers and Intellisense prompting during development, serialization and hence ?indirectly ?remoting. It is reflection that facilitates component reuse in a much more integrated fashion than COM ever did. Certainly the reflection capability built into .NET brings many advantages.

The downside of this is that the facilities available through the System.Reflection namespace provide a much easier way than ever before for other people to examine the structure of your applications by querying the metadata that is contained in the manifest held within every .NET assembly. It’s not just the public types and members which could come under such scrutiny ? private, internal and protected fields, properties and methods can all be examined through reflection, and the Common Language Runtime (CLR) provides no mechanism for prohibiting this.

If you are familiar with ILDasm.exe ?the Microsoft Intermediate Language (IL) disassembler ?you will already be aware how much information is available. You may not be aware, however, exactly how easy it can be to render this information into a format much more digestible to a developer.

In this case study we’ll develop a simple decompiler, which uses Reflection and CodeDOM together to allow a .NET assembly to be reverse engineered into C# source code. As a final twist, we’ll fill in the decompiled method bodies with disassembled IL code as comment lines. This is about all we need to do to produce a C# source file that, if not directly compilable, would be easily comprehensible to any moderately experienced developer. All of this using standard .NET tools, pure C# code, and starting with nothing more than a .NET assembly binary. Just like the ones you distribute with your applications.

Worried? You should be.

.NET Decompilers

There are already a number of .NET decompilers on the market (for example, the Salamander decompiler at http://www.remotesoft.com/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq). Often, these are marketed alongside obfuscators, which make the task of understanding decompiled code much harder. We’ll take a closer look at obfuscators in part two of this case study, as there’s a lot to discuss. These commercial decompilers typically operate directly at the IL level, which makes them conceptually different from the approach we’ll be taking but, while the results can be that much more impressive, you’ll see enough to realise that it doesn’t need a great deal of time, effort, expertise or expense for a hacker to get inside your code.

Background Reading

Previously on C# Today, the use of reflection for reverse engineering has been touched on in C# to UML Reverse Engineering. Also, using CodeDOM to generate source code has been covered in Building a Forms Generator Using Code DOM and Generating Design Patterns Using CodeDOM. If you haven’t read these articles then it may be an idea to do so now as this will help your understanding of what’s to follow.

System Requirements

This system requires Visual Studio .NET V1.0 or later running on Windows 2000 or Windows XP. The case study assumes that the reader has a good understanding of C# and some familiarity with CodeDOM.

Defining The Problem

The problem here is not so much one of design as one of discovery. The System.Reflection namespace contains many classes that allow the types contained in an existing assembly to be inspected. The System.CodeDOM namespace likewise contains many classes that allow types to be defined in sufficient detail to automatically generate C# or VB.NET source code describing these types.

The problem here is that neither of these namespaces is particularly well documented.

The System.Reflection namespace (and the more specialised System.Reflection.Emit namespace) contain some types and some type members specifically for use by compiler developers, which would not normally be used by application developers. The framework documentation doesn’t flag any distinctions between these types and this makes it somewhat confusing in places. The documentation for System.CodeDOM is also difficult to digest and can be confusing to an application developer due to the different nomenclature used. Where a developer might think of protected or internal modifiers for a type member, the CodeDOM documentation uses the CLR attributes Family and Assembly respectively. Believe me, it’s not hard to get lost in the mass of CodeDOM classes, properties, methods, collections and enumerations.

The following table shows some of the differences in nomenclature between CLR, C# and VB.NET terms:

CLR Term | C# Term | VB.NET Term
Private | Private | Private
Family | Protected | Protected
FamilyAndAssembly | (not supported) | (not supported)
Assembly | Internal | Friend
FamilyOrAssembly | Protected internal | Protected friend
Public | Public | Public
Abstract | Abstract | MustInherit / MustOverride
Final | Sealed | NotInheritable / NotOverridable
Static | Static | Shared
InitOnly | Readonly | ReadOnly
Virtual | Virtual | Overridable
NewSlot | New | Shadows
ReuseSlot | override | Overrides

Case Study Structure

The sample application we’ll develop will have a simple user interface, just enough to allow us to select an assembly to decompile and to display the resulting source code. I’ll briefly mention the UI design and then go into the detail where we get to grips with the Reflection and CodeDOM classes. Then we’ll dive a little deeper into the metadata to see how we can extract the IL code for each method body, after which we’ll put our decompiler through its paces before discussing what we’ve learnt and looking ahead to see what can be done about it!

  • The Sample Application
  • UI Design
  • Coding
  • Loading an Assembly
  • Types and Nested Types
  • Events
  • Fields
  • Methods
  • Constructors
  • Properties
  • Parameter Declarations
  • Custom Attributes
  • Methods ?Body Snatching!
  • Testing the Decompiler
  • Application Setup
  • Limitations
  • Case Study Review

The Sample Application

The sample application will be a Windows Forms application named DeComp. From Visual Studio .NET, create a new C# Windows Forms project and rename the Form1 class to Generate.

UI Design

Using the Forms Designer, resize the Generate form to 584 x 413, add a MainMenu and an OpenFileDialog, naming the latter dlgopen. Next add a TextBox and set the Dock property to Full, the Font to Courier New 9pt, set MultiLine and ReadOnly to True, WordWrap to False and ScrollBars to Both. The resulting design should look something like Figure 1 below.

Figure 1 - UI Design

On the MainMenu, add a File menu item containing the options Open and Exit. Add Click event handlers for these two items and complete the click handler for the exit option as follows:

private void menuFileExit_Click(object sender, System.EventArgs e)
{
// exit
this.Close();
}

None of this should have caused you any problems, so we’ll get straight to the details now and start coding the decompiler.

Coding

The first thing we’ll do is to import some of the namespaces we’ll be using,

using System.CodeDom;
using System.CodeDom.Compiler;
using System.IO;
using System.Text;
using Microsoft.CSharp;
using System.Reflection;
using System.Reflection.Emit;

and we’ll also declare a private field to hold the CodeDOM code unit to contain all of our generated declarations:

private CodeCompileUnit codeunit;

The following sections describe the remainder of the coding, which is not challenging, but there is a lot of it. In general, we’ll cover the following sequence of steps, most of which involve converting information obtained from reflection into equivalent CodeDOM declarations.

  • Loading an assembly and examining the types defined therein.
  • Declaring types (classes, interfaces, structs and enums).
  • Declaring events.
  • Member fields (private, public, static, const, initialised)
  • Method declarations.
  • Constructors (private, public, instance, static)
  • Properties and type indexers
  • Parameter lists
  • Custom attributes
  • Decompiling IL method bodies

We’ll start with the way we load an assembly from disk and examine the types defined within.

Loading an Assembly

The main processing will be initiated when the user selects File/Open from the menu. So, in the Click event handler for the Open option on the menu, display the OpenFileDialog to allow the user to select a .NET assembly (either an .exe or a .dll) for decompilation.

private void menuFileOpen_Click(object sender, System.EventArgs e)
{
// open an existing .net assembly
if (dlgOpen.ShowDialog() == DialogResult.OK)
{

Having acquired a filename, we can load the assembly by using the static method Assembly.LoadFrom(). This makes sure we load the assembly from the file selected by the user. If we used the Load() method, then the same algorithm used by the CLR to implicitly load an assembly would be used. This means, that the assembly may instead be loaded from the global assembly cache (GAC) or from the application path of the decompiler, and so we may not get exactly what we’re aiming for.

We can reinitialise the CodeCompileUnit at the same time we load a new assembly, so the menuFileOpen_Click() method continues as follows:

// create compile unit and load assembly for reflection
codeunit = new CodeCompileUnit();
Assembly mainAssembly = Assembly.LoadFrom(dlgOpen.FileName);

Having loaded the assembly, we reflect over all the types contained therein and add these to our CodeDOM definitions.

// reflect over all types in the
            assembly
Type[] allTypes =
            mainAssembly.GetTypes();
foreach(Type t in allTypes)
{
//
            add top-level types to codedom compile unit
if (t.MemberType ==
            MemberTypes.TypeInfo)
addType (t, null);
}

You’ll see that the bulk of the processing is carried out by the addType() method, which we’ll cover in a moment. For now, we can complete the above method by assuming we’ve reflected over the complete assembly and added all of our CodeDOM definitions to the codeunit.

All that remains then is to generate the decompiled C# source code and display it in the TextBox on the main form:

      //
            create code generator
CSharpCodeProvider cs = new
            CSharpCodeProvider();
ICodeGenerator csGen =
            cs.CreateGenerator();
CodeGeneratorOptions options = new
            CodeGeneratorOptions();
options.BracingStyle = "C";

//
            generate code and show result
StringWriter sw = new
            StringWriter();
csGen.GenerateCodeFromCompileUnit(codeunit, sw,
            options);
textBox1.Text = sw.ToString();
}
}

This is pretty straightforward, although we really need some exception handling in here as it’s quite possible for the code generator to choke on an invalid declaration if we’ve not been careful. The above code has been amended for clarity - the download material for this article contains a more robust version.

Types and Nested Types

Now for the addType() method. The purpose of this method is to add a CodeDOM type declaration for a type we’ve retrieved from the Assembly using reflection.

As it’s possible for types to be nested, this method may be called recursively to add a type declaration to an existing CodeDOM type. For this reason, the method takes an optional argument specifying the owning CodeDOM type declaration. If this is null (as in the above code), then the new type is added directly to the CodeDOM namespace. Like this:

private void addType(Type aType,
            CodeTypeDeclaration owner)
{
// build a new CodeDOM type
            declaration
CodeTypeDeclaration newtype = new
            CodeTypeDeclaration(aType.Name);
if (owner == null)
{
// if
            owner is null, add type definition to its namespace
CodeNamespace
            ns =
            findNamespace(aType.Namespace);
ns.Types.Add(newtype);
}
else
{
//
            otherwise, add a nested
            type
owner.Members.Add(newtype);
}

Namespaces actually mean absolutely nothing to the CLR, they exist purely as an aid to the developer to organise his/her code more effectively and to allow the compiler to resolve type names coded in a shorthand syntax.

However, it’s very convenient that the namespace is held in the metadata describing each type so we can create a CodeDOM namespace with the correct name when we decompile the assembly. This brings added clarity to our decompiled code and so makes it easier for us to interpret the developer’s original design.

As every type we come across in the assembly will have a Namespace property, we need to make sure that we don’t go creating a new CodeDOM namespace declaration for each type we find that was defined in it. We need a helper method to check if we’ve already created that namespace and to pass back a reference if so.

That’s the purpose of the findNamespace() method invoked above. It’s defined as follows:

private CodeNamespace findNamespace (string
            name)
{
// return CodeDOM namespace with given name
int
            index = -1;

// find namespace if already declared
for (int
            i = 0; i < codeunit.Namespaces.Count; i++)
{
// check by
            name
if (codeunit.Namespaces[i].Name == name)
{
// set
            index and exit loop
index = i;
break;
}
}

// add
            namespace to unit or use existing
if (index == -1)
{
index
            = codeunit.Namespaces.Add(new CodeNamespace(name));
}

//
            return namespace
return
            codeunit.Namespaces[index];
}

This method looks through all of the namespaces already defined within the CodeCompileUnit, attempting to find a match by name. If a match is not found, the namespace is added and, in either case, a reference to the correct CodeDOM namespace is returned.

Now, let's return to the addType() method. By now we’ve created a CodeDOM type declaration for the type, we’ve added it to the correct CodeDOM namespace, and the namespace has been added to the CodeDOM CodeCompileUnit referenced by the private member field codeunit. Everything is in place for us to generate source code once we’ve finished defining the type.

We can continue to fill in the details of our new type by adding some basic information available from the metadata:

// fill in basic details of this type
newtype.TypeAttributes = aType.Attributes;
newtype.IsClass = aType.IsClass;
newtype.IsEnum = aType.IsEnum;
newtype.IsInterface = aType.IsInterface;

These properties identify whether our type is a class, enumeration, interface, etc. and also determine the accessibility (public, private, protected, etc.). The latter is achieved by setting the CodeDOM TypeAttributes directly from the Attributes obtained by reflection. According to the documentation, this is the only compatible mapping ?you’ll see later that we need to do a little more work to map other attributes between CodeDOM and reflection enumerations.

Now we’ve reached this point, we need to be a little more careful and maybe need a deeper understanding of how the CLR view differs from the familiar C# perspective. First off, let’s look a little more closely at how an enum is defined in an assembly.

Enumerated Types

If you create an enum in a C# project, then examine the compiled assembly using ILDasm, you’ll see what I mean. The following C# declaration,

public enum NormalByteEnum : byte
{
ByteEnum1 = 1,
ByteEnum2 = 2
}

looks like this from ILDasm (see Figure 2) ?/p>

Figure 2 - Enum in ILDasm

You can see that an enum is really defined as a class descending from System.Enum and containing static fields for each entry in the enumeration (initialised to their respective values). Furthermore, the actual type of the enumeration (we defined it as a byte in our example above) is recorded against a field named value__, which has the attributes SpecialName and RTSpecialName.

We need this information to define the enum correctly using CodeDOM because we only have one class ?CodeTypeDeclaration ?to handle classes, enumerations, structs and interfaces. In this case, the important thing to do is to define the correct base class ?not System.Enum as you might expect, the enumeration is indicated instead by the IsEnum property, by the type of the value__ field.

We can do this like so:

   // handle enum and
            struct
if (newtype.IsEnum)
{
// for enums, get type of
            field
FieldInfo finf =
            aType.GetField("value__");
newtype.BaseTypes.Add(finf.FieldType);
}

Now we can continue and set the base type for any non-enum types:

   else
{
// flag
            struct if required
newtype.IsStruct =
            aType.IsValueType;

// only add a non-default base type
if
            (aType.BaseType != null)
{
if
            (!aType.BaseType.Equals(typeof(object))
            &&
!aType.BaseType.Equals(typeof(System.ValueType)))
{
newtype.BaseTypes.Add(aType.BaseType);
}
}
}

Notice we also set a flag here to differentiate classes and structs and we only bother to add a base type if its not one of the defaults (System.Object for a class or System.ValueType for a struct). This is not strictly necessary but it does make our generated code look more like the original if we leave these as implied derivations.

Remember that in C#, a class can only have at most one base class, so while that takes care of enums and structs, we still need to handle interfaces. If we’re dealing with a class or an interface then this could also inherit from one or more other interfaces, and we need to know this to produce the correct CodeDOM declaration. We can find out what interfaces a type inherits from via the Type.GetInterfaces() method, but in doing so there’s another small hurdle to overcome.

The problem is that the GetInterfaces() method will return all interfaces inherited by the type we’re examining and any inherited from its base class, and so on right up the class hierarchy! Whereas we only want to know about any interfaces for which our type actually implements methods, that is, just the interfaces it directly inherits.

We can find this out in a roundabout way by using another method defined on the Type class, GetInterfaceMap(). The interface map for a type contains an array of target methods for a particular interface, which represent the actual methods that implement the interface at this point in the class hierarchy. By checking all of the target methods for all of the interfaces inherited by the type we’re examining, we can find out if any of those methods are declared in that type, and so work out which interfaces are directly inherited.

If you followed that, you’ll follow this next piece of code. If you didn’t follow it, then perhaps looking at the code may make it easier. What we do here is first make sure we’re dealing with a class or an interface ?we’re not interested in anything else. Then we get an array of the interfaces inherited by that type. Next, for each of these interfaces, we get an array of the actual methods that implement that interface. After this, for each method, we check the DeclaringType to see if it matches the type we’re examining. If any method does, then we add the interface to the collection of CodeDOM base types for the type declaration before moving on and checking the next interface. And, yes, I wish it were more simple too, but it’s not. Here’s the code:

   // add implemented
            interfaces
if (newtype.IsClass || newtype.IsInterface)
{
//
            add any inherited interfaces
Type[] ifaces =
            aType.GetInterfaces();
foreach(Type iface in ifaces)
{
//
            add interface if this type implements it
            directly
InterfaceMapping imap =
            aType.GetInterfaceMap(iface);
MethodInfo[] meths =
            imap.TargetMethods;
foreach(MethodInfo meth in meths)
{
if
            (meth.DeclaringType.Equals(aType))
{
newtype.BaseTypes.Add(iface);
break;
}
}
}
}

The code actually takes up much less space than the preceding explanation and is probably easier to understand!

Almost the last thing we do to complete our declaration of the reflected type is to copy over any custom attributes. Now almost anything can have custom attributes, so I’ve employed another helper method to do this and I’ll detail this later. For now, we can just add a couple more lines to the addType() method.

// add custom attributes
object[] attrs = aType.GetCustomAttributes(false);
newtype.CustomAttributes.AddRange(getCustomAttributes(attrs));

So we’ve now defined the basic details for the reflected type in CodeDOM terms. We’ve got the name, accessibility and any custom attributes, and we’ve made sure we’ve got the correct base classes and that we’ve taken care of anything particular to classes, structs, enums and interfaces. Now we can dive inside this type and look to see what members it contains.

Still using the Type class to get these details from the assembly, we can retrieve an array of all members declared within this type.

// get members for this type
MemberInfo[] allMembers = aType.GetMembers(BindingFlags.DeclaredOnly |
BindingFlags.NonPublic | BindingFlags.Public | BindingFlags.Instance |
BindingFlags.Static);

Note the use of the BindingFlags argument. This allows us to specify exactly what sort of members we’re looking for. The above combination will retrieve all public and private, static and instance members. Using BindingFlags.DeclaredOnly allows us to filter out any inherited members so that we only get details of members actually declared by the type rather than those inherited. It needs to be said that if there were some similar mechanism for retrieving interfaces this would have saved us a lot of trouble earlier!

We can then process the array of MemberInfo objects and invoke helper methods to process each one according to their member type. For most of these, we recast the MemberInfo to something more specific to allow us access to the details we need. Note here, that if we come across a nested type, we call addType() recursively to include a complete type declaration within the current CodeDOM type declaration. The following switch statement identifies the member type and passes control to the relevant helper method:

// process members according to member type
foreach(MemberInfo mem in allMembers)
{
switch (mem.MemberType)
{
case MemberTypes.NestedType:
case MemberTypes.TypeInfo:
addType((Type)mem, newtype);
break;
case MemberTypes.Event:
addEvent((EventInfo)mem, newtype);
break;
case MemberTypes.Field:
addField((FieldInfo)mem, newtype);
break;
case MemberTypes.Method:
addMethod((MethodInfo)mem, newtype);
break;
case MemberTypes.Constructor:
addConstructor((ConstructorInfo)mem, newtype);
break;
case MemberTypes.Property:
addProperty((PropertyInfo)mem, newtype);
break;
default:
MessageBox.Show("Unknown type - " +
mem.MemberType.ToString());
break;
}
}
}

That completes the addType() method. Now we’ll examine each of the helper methods in turn.

Events

Events are probably the most straightforward class members to decompile. Just to recap, the addEvent() method is invoked with two parameters ?the first is an EventInfo object from the System.Reflection namespace, which contains details of an event in the type we’re currently decompiling. The second parameter is a reference to the CodeDOM type declaration into which we need to add a matching declaration for this event.

The CodeDOM class that declares an event is CodeMemberEvent. Apart from adding any custom attributes (using the same helper we invoked earlier), about all we need to find out is the name of the event and the type of the delegate needed to trap the event. Both of these can be found directly from the EventInfo object in the Name and EventHandlerType properties respectively, so the addEvent() method is really simple:

private void addEvent(EventInfo info,
            CodeTypeDeclaration owner)
{
// add event to given
            type
CodeMemberEvent newevent = new
            CodeMemberEvent();
newevent.Name = info.Name;
newevent.Type =
            new CodeTypeReference(info.EventHandlerType);

object[]
            cusattr =
            info.GetCustomAttributes(false);
newevent.CustomAttributes.AddRange(getCustomAttributes(cusattr));

//
            add event to owner
owner.Members.Add(newevent);
}

We’ve not heard the last of events though, so you may have a few surprises later!

Fields

Fields are not as simple as events, because we have to take special care with enumerations and indeed any field defined with an initial value. By this I really mean constants. Normal instance fields which are declared in C# with an initial value are not described in this way in the metadata. Instead, the compiler inserts the relevant initialisation code into the body of the constructor so we cannot regenerate the exact same syntax when we decompile. However, we don’t end up losing this information, as we’ll see later.

We also need to make sure we don’t output a field declaration for the special value__ field we discovered hiding in an enumeration. Fortunately, we can find this (and any as yet undiscovered special fields) by examining the FieldInfo.Attributes property to see if the SpecialName flag is set.

The addField() method starts like this:

private void addField(FieldInfo info, CodeTypeDeclaration owner)
{
// add field details to given type

// don't add special fields
if ((info.Attributes & FieldAttributes.SpecialName) != 0)
return;

Then we create a new CodeDOM definition for the field and set the basic attributes:

// create CodeDOM field definition
CodeMemberField newField = new CodeMemberField(info.FieldType, info.Name);
newField.Attributes = getMemberAttributes(info);

I’ve created an overloaded method called getMemberAttributes(), which I’ll use to map the basic attributes between reflected classes (EventInfo, FieldInfo, PropertyInfo, etc.) and the corresponding CodeDOM classes (CodeMemberEvent, CodeMemberField, CodeMemberProperty, etc.). Unlike the simple mapping we had between Attributes and TypeAttributes at the class level, these enumerations ?while basically conveying the same information ?do not necessarily have the same internal values. We have to test each attribute separately and decide whether or not to set the corresponding bit flag in the equivalent CodeDOM enumeration.

This is the getMemberAttributes() method that handles fields:

private MemberAttributes
            getMemberAttributes(FieldInfo info)
{
// convert Reflection
            FieldInfo attributes to CodeDOM
MemberAttributes attrDom =
            0;

// map attributes between types
if (info.IsAssembly)
            attrDom |= MemberAttributes.Assembly;
if (info.IsFamily) attrDom
            |= MemberAttributes.Family;
if (info.IsFamilyOrAssembly) attrDom
            |= MemberAttributes.FamilyOrAssembly;
if (info.IsPrivate) attrDom
            |= MemberAttributes.Private;
if (info.IsPublic) attrDom |=
            MemberAttributes.Public;
if (info.IsLiteral)
attrDom |=
            MemberAttributes.Const;
else if (info.IsStatic)
attrDom |=
            MemberAttributes.Static;

// return CodeDOM
            enumeration
return attrDom;
}

This is something to be careful with, and at first it gave me all sorts of problems. You’ll note that I have coded this method so that a constant field will never be also marked as static (even though it is in the metadata ?take a look at one with ILDasm if you don’t believe me). The reason for this is that, in CodeDOM terms, declaring a static const field doesn’t make any more sense than it does to the C# compiler. Whereas the C# compiler will throw out an error, CodeDOM just refuses to output either the static or the const keywords if you define both. This is not documented anywhere and although ?knowing the answer ?it makes some sort of sense, it does mean that mapping these attributes is not as simple as finding the matching meaning and flipping the bit.

Also here we find out that there is apparently no way to declare a readonly field using CodeDOM. We know from reflection if we’ve got such a field as the InitOnly flag is set in the Attributes property of the FieldInfo object, but there’s no corresponding setting in the CodeDOM MemberAttributes enumeration. This is annoying but not critical to our understanding of the decompiled code.

Having mapped the attributes, we now need to handle the special case where a field is marked as Literal in the reflection class. This will happen for constants and enumerations, both of which need to have an initial value specified. I’ll show you the code first and then talk through it:

// handle field initialisation
if (info.IsLiteral)
{
// get initial value
object val = info.GetValue(null);

// make sure we use a valid primitive type
switch(Convert.GetTypeCode(val))
{
case TypeCode.Empty:
break;
case TypeCode.Double:
case TypeCode.Single:
val = System.Convert.ChangeType(val, typeof(System.Double));
break;
case TypeCode.String:
val = System.Convert.ChangeType(val, typeof(System.String));
break;
default:
val = System.Convert.ChangeType(val, typeof(System.Int64));
break;
}

// create initial expression
newField.InitExpression = new CodePrimitiveExpression(val);
}

There’s no problem identifying literals, nor in obtaining the initial value. This is all held safely in the metadata and the reflection classes allow us to query this information with ease.

The problem here is really with CodeDOM.

Whenever we use the CodePrimitiveExpression class to create an expression containing a primitive value, the CSharpCodeProvider actually expects this to be not just a primitive type, but a CLS compliant data type. If it’s not, it will throw an exception when we try to generate code. This means for instance that we cannot use any unsigned numeric types to create a CodePrimitiveExpression even though these are valid C# primitive data types.

This is not too much of a problem to overcome, but it’s really something that shouldn’t be necessary. The switch statement above examines the TypeCode of the initial value (which must have been a valid primitive type or the assembly wouldn’t have compiled) and recasts the object to a compatible CLS compliant type. This avoids any problems with the code provider, but it does feel like a bit of a kludge. After all, we’re only adding a literal value into the source code, we’re not leaving anything around that implies a data type.

Having initialised any literals, all that remains is to define any custom attributes then add the field to the owning CodeDOM type declaration. As before, we use the getCustomAttributes() helper to do this and this will be explained fully later.

   // add custom
            attributes
object[] cusattr =
            info.GetCustomAttributes(false);
newField.CustomAttributes.AddRange(getCustomAttributes(cusattr));

//
            add field to owning
            type
owner.Members.Add(newField);
}

Methods

Methods can be reflected quite nicely. The main thing we can’t currently do from the System.Reflection namespace is to access the IL code in the method body. However, there is an alternative way to do this that I’ll talk about later.

One other thing we need to be careful of with methods is that we don’t extract any methods generated by the compiler ?similar to the value__ field in enumerations. One example of when this happens is when a class uses properties ?the get and set accessors are actually compiled as separate methods in the assembly and we can see this by using ILDasm as before.

Create a C# class library and define a simple property:

public class Test
{
private int
            prop;

public int Prop
{
get { return prop; }
set {
            prop = value; }
}
}

Compile the project and then examine it using ILDasm. You’ll see that the compiler has generated two member methods by prefixing the property name with get_ and set_ respectively (see Figure 3).

Figure 3 - Accessor Methods

As with the value__ field in an enumeration, these methods have the SpecialName flag set in their Attribute property, so we can check this during decompilation and just ignore these methods. We could of course just carry on and decompile them, but we’ll be dealing with properties as a separate issue so we don’t actually gain anything in terms of nicely decompiled code.

The addMethod() method therefore begins as follows:

private void addMethod(MethodInfo info, CodeTypeDeclaration owner)
{
// add a member method

// don't add special methods
if ((info.Attributes & MethodAttributes.SpecialName) != 0)
return;

// create CodeDOM method declaration
CodeMemberMethod newmeth = new CodeMemberMethod();
newmeth.Name = info.Name;
newmeth.Attributes = getMemberAttributes(info);

As with fields, we use an overload of getMemberAttributes() to map the reflection attributes to the corresponding CodeDOM versions. Again, we have to introduce some extra logic into this mapping process to make sure we don’t end up with any invalid combinations of flags in CodeDOM.

The problem this time is with virtual methods. If a member is not flagged as Virtual in the reflection class MethodInfo, then it must be flagged as Final in the corresponding CodeDOM declaration. If a member is flagged as Virtual, then we need to establish if this is an override of an inherited method, a new virtual method that hides an inherited method, or a genuine virtual method in the declaring class.

We start like so:

private MemberAttributes
            getMemberAttributes(MethodInfo info)
{
// convert Reflection
            MethodInfo attributes to CodeDOM
MemberAttributes attrDom =
            0;

// map attributes between types
if (info.IsAssembly)
            attrDom |= MemberAttributes.Assembly;
if (info.IsFamily) attrDom
            |= MemberAttributes.Family;
if (info.IsFamilyOrAssembly) attrDom
            |= MemberAttributes.FamilyOrAssembly;
if (info.IsPrivate) attrDom
            |= MemberAttributes.Private;
if (info.IsPublic) attrDom |=
            MemberAttributes.Public;
if (info.IsStatic) attrDom |=
            MemberAttributes.Static;
if (info.IsAbstract) attrDom |=
            MemberAttributes.Abstract;

This gets details of the accessibility of the method. Then we can examine the Virtual flag and take appropriate action:

   // check virtual
            methods
if (info.IsVirtual)
{
// see if virtual method was
            defined in a base class
if (isMethodDefined(info,
            info.ReflectedType.BaseType))
{
// if so, check for new or
            override
if ((info.Attributes & MethodAttributes.NewSlot) !=
            0)
attrDom |= MemberAttributes.New;
else
attrDom |=
            MemberAttributes.Override;
}
}
else
{
// if not
            virtual, make sure final is set
attrDom |=
            MemberAttributes.Final;
}

// return CodeDOM
            enumeration
return attrDom;
}

The reflected attributes we need to examine to distinguish between an override and a new method are NewSlot and ReuseSlot. These flags indicate whether the method takes a new slot in the vtable or reuses an existing slot (the default action). As these are mutually exclusive, we need only test one of them to see if this is an override or not. However, we only need to make this distinction when the method is declared in a base class, hence the initial test in the code above. This is more messy than I’d like, but we can’t tell the difference between a new virtual method (that hides a base class method) and a genuinely new virtual method in the derived class just by examining the reflected attributes! We’re forced to examine the base class to see if such a method is also defined there. This is done by the isMethodDefined() helper, which takes the MethodInfo of the method to match and the Type of the class to match it in. It uses Type.GetMethod() to search for a method with the same name and the same number, order and type of parameters. Like this:

private bool isMethodDefined(MethodInfo meth, Type type)
{
// return true if a matching method is defined on type
if (type == null) return false;

// create array of parameter types
ParameterInfo[] parms = meth.GetParameters();
Type[] parmtypes = new Type[parms.Length];
for(int i = 0; i < parms.Length; i++)
parmtypes[i] = parms[i].ParameterType;

// set binding flags
BindingFlags bindingattr =
BindingFlags.ExactBinding | BindingFlags.NonPublic |
BindingFlags.Public | BindingFlags.Instance;

// attempt to find method with same name and parms
MethodInfo match = type.GetMethod(meth.Name, bindingattr,
null, parmtypes, null);

// return result
return (match != null);
}

Right, we’ve now got the method’s attributes converted correctly so we can carry on with the addMethod() helper and add any custom attributes.

object[] cusattr = info.GetCustomAttributes(false);
newmeth.CustomAttributes.AddRange(getCustomAttributes(cusattr));

Then we can find the parameter list for this method and build matching CodeDOM declarations. As we’ll also need to do this for other members, we’ll use another helper method ?getParameterDeclarations() ?which I’ll describe later.

newmeth.Parameters.AddRange(getParameterDeclarations(info.GetParameters()));

The last thing we need to do is to set the return type of the method and then add it to the owning CodeDOM type declaration:

   newmeth.ReturnType = new
            CodeTypeReference(info.ReturnType);

// add method to owning
            type
owner.Members.Add(newmeth);
}

We’ll come back to the addMethod() helper later when we deal with the method body. This is also something we need to do for constructors and properties so we’ll cover the basics of these first.

Constructors

As far as the CLR is concerned, constructors come in two flavours. There’s the normal instance constructor, which is flagged in the metadata with the SpecialName attribute and given the name .ctor. Also, there may optionally be a type constructor (also known as a static constructor, class constructor or type initialiser). This also has the SpecialName attribute, but this time with a name of .cctor.

These constructors have different CodeDOM declarations. An instance constructor is declared using CodeConstructor and a type constructor is declared using CodeTypeConstructor, both of which descend from CodeMemberMethod. We can check the constructor flavour up front because, besides having the special name .cctor, a type constructor is also flagged as static.

private void addConstructor(ConstructorInfo
            info, CodeTypeDeclaration owner)
{
// add a constructor to the
            given type
CodeMemberMethod newcons;

// check for a class
            constructor
if (info.IsStatic)
newcons = new
            CodeTypeConstructor();
else
newcons = new
            CodeConstructor();

Then we can set the name, attributes and parameters:

   // set name, attributes
            and parameters
newcons.Name = info.Name;
newcons.Attributes =
            getMemberAttributes(info);
object[] cusattr =
            info.GetCustomAttributes(false);
newcons.CustomAttributes.AddRange(getCustomAttributes(cusattr));
newcons.Parameters.AddRange(getParameterDeclarations(info.GetParameters()));

//
            add constructor definition to
            owner
owner.Members.Add(newcons);
}

A type constructor never actually has any parameters, but the above coding is clean and simple and there’s no real harm in performance terms by attempting to process an empty collection.

We’re using the still-to-be-defined methods getCustomAttributes() and getParameterDeclarations() here and also another overload of getMemberAttributes().

private MemberAttributes
            getMemberAttributes(ConstructorInfo info)
{
// convert
            Reflection ConstructorInfo attributes to CodeDOM
MemberAttributes
            attrDom = 0;

// map attributes between types
if
            (info.IsAssembly) attrDom |= MemberAttributes.Assembly;
if
            (info.IsFamily) attrDom |= MemberAttributes.Family;
if
            (info.IsFamilyOrAssembly) attrDom |=
            MemberAttributes.FamilyOrAssembly;
if (info.IsPrivate) attrDom |=
            MemberAttributes.Private;
if (info.IsPublic) attrDom |=
            MemberAttributes.Public;
if (info.IsStatic) attrDom |=
            MemberAttributes.Static;

// return CodeDOM
            enumeration
return attrDom;
}

Ideally, these attributes need to be mapped slightly differently into CodeDOM to avoid unexpected side effects, although we could have just use the same mapping as we did for a normal method. There are no conflicting attributes between constructors and methods, but the additional code we have for handling virtual methods clouds the issue a little and I believe it’s best to keep these mappings separate.

Properties

We touched on properties earlier, where we saw how the accessor methods are held in the assembly as specially-named members. There’s another member type that’s held in an assembly as a property ?a type indexer.

Unfortunately, although we can see these indexers as properties using reflection, we cannot reconstruct the C# source code using the CodeDOM property class, CodeMemberProperty. This just doesn’t have the ability to represent an indexer and there is no specialised CodeDOM class that does. So what can we do?

Well, we can cheat.

CodeDOM doesn’t actually validate the source code it generates, and we can make it happily spit out any invalid member names we like. This helps us here because, if we remind ourselves what an ordinary property looks like:

public int NormalProperty
{
get { //
            TODO }
set { // TODO }
}

and then what a type indexer looks like:

public bool this[SomeParameters]
{
get { // TODO }
set { // TODO }
}

Then we can just build a property named in the above format (i.e. " this[parameter list] ") and supply this to CodeDOM in the normal way to reconstruct the desired code!

So let’s start building a property from the reflected PropertyInfo. The first thing we do is to get the accessibility information (public, private, etc.), which, for a property, is actually held against the SpecialName accessor methods (either one will do). We can get hold of these easily enough like this:

private void addProperty(PropertyInfo info, CodeTypeDeclaration owner)
{
// add a property to the given type
CodeMemberProperty newprop = new CodeMemberProperty();

// get accessibility from get/set methods
MethodInfo[] methinfo = info.GetAccessors(true);
if (methinfo.Length > 0)
newprop.Attributes = getMemberAttributes(methinfo[0]);

By using the GetAccessors() method, we don’t care whether this is a read only or write only property, we will always get at least one of these.

Next, we add any custom attributes using our ubiquitous helper method getCustomAttributes().

// add custom attributes
object[] cusattr = info.GetCustomAttributes(false);
newprop.CustomAttributes.AddRange(getCustomAttributes(cusattr));

Then we can set more basic information about the property ?its name, type and whether it is read only or write only. All of this can be copied straight from the PropertyInfo object we’ve obtained using reflection.

   // set basic
            details
newprop.Name = info.Name;
newprop.Type = new
            CodeTypeReference(info.PropertyType);
newprop.HasGet =
            info.CanRead;
newprop.HasSet = info.CanWrite;

Then we check to see if this is a type indexer or not. In C#, a type indexer is normally given the name Item. We could test this name against the PropertyInfo but, as it is possible for a developer to have forced the C# compiler to use a different name (by applying the System.Runtime.CompilerServices.IndexerName custom attribute), there’s a much simpler way. For an indexer, the get accessor method will have a parameter list. If it doesn’t have one, then we can safely assume that we have a genuine property. We can get the parameter list for the get accessor like this:

// check for a type indexer
MethodInfo getmeth = info.GetGetMethod(true);
ParameterInfo[] parms = getmeth.GetParameters();

Then we can check for a type indexer and, if necessary, assign our carefully constructed name:

   // if get accessor has
            parameters, this is an indexer
if (parms.Length >
            0)
{
// derive new name based on parameters
newprop.Name =
            "this[" + getArgumentList(parms) + "]";
}
// add property to
            owning type
owner.Members.Add(newprop);
}

The getArgumentList() method just loops through the parameters and builds a comma delimited string in conventional C# format.

private string
            getArgumentList(ParameterInfo[] parms)
{
// build string with
            any parameter details
StringBuilder sb = new
            StringBuilder();
bool comma = false;
foreach(ParameterInfo
            parm in parms)
{
// add comma if not first in list
if
            (comma) sb.Append(", ");

// add parameter type and
            name
sb.Append(parm.ParameterType.ToString());
sb.Append("
            ");
sb.Append(parm.Name);
comma = true;
}

// return
            result
return sb.ToString();
}

This takes care of both properties and type indexers. We’ll just clear up a couple of loose ends before looking at extracting details of method bodies, so here goes with parameters and custom attributes?/p>

Parameter Declarations

Parameters may be declared on methods or constructors. When we dealt with member types, we invoked a helper method ?getParameterDeclarations(), which accepted a ParameterInfo array (from reflection) and returned a CodeParameterDeclarationExpressionCollection for CodeDOM (that must be one of the longest class names in the FCL!). This method is defined as follows:

private CodeParameterDeclarationExpressionCollection getParameterDeclarations(ParameterInfo[] infoarray)
{
// convert Reflection ParameterInfo to CodeDOM
CodeParameterDeclarationExpressionCollection parms =
new CodeParameterDeclarationExpressionCollection();
foreach(ParameterInfo info in infoarray)
{
// create a new CodeDOM parameter
CodeParameterDeclarationExpression parm =
new CodeParameterDeclarationExpression();

// copy declaration from reflected info
parm.Name = info.Name;
parm.CustomAttributes.AddRange(getCustomAttributes(info.GetCustomAttributes(false)));
parm.Type = new CodeTypeReference(info.ParameterType);
if (info.IsIn)
{
// check for ref parameter
if (info.ParameterType.IsByRef)
parm.Direction = FieldDirection.Ref;
else
parm.Direction = FieldDirection.In;
}
else if (info.IsOut) parm.Direction = FieldDirection.Out;

// add parameter to collection
parms.Add(parm);
}

// return CodeDOM collection
return parms;
}

There’s not a great deal of explanation needed here. Each parameter in an argument list must have a type and a name and may optionally have any number of custom attributes and/or a field direction qualifier. All of this we can get from the ParameterInfo so we just need to create a CodeParameterDeclarationExpression for each parameter, copying the various properties and then add it to the collection.

Here we again use the getCustomAttributes() method to map custom attributes between the reflection and CodeDOM classes, so we’ll cover that next.

Custom Attributes

Custom attributes cause a lot of problems. Two of these are pretty big ones.

First, not everything you might think of as a custom attribute is actually treated as such by the C# compiler or the CLR. There are a number of common attributes that get special treatment during compilation and are actually encoded in the metadata as bit flags rather than as serialised attribute classes. This means that they are not visible through the normal reflection mechanism using the GetCustomAttributes() method. Some of these (but not all) are visible through additional properties held in MemberInfo, FieldInfo or MethodInfo objects, but they’re still not easy to get at. The sort of attributes I’m talking about here are things like [Serializable], [DllImport], [StructLayout], etc. so they are quite commonly used. They are referred to in the documentation as pseudo-custom attributes.

The second problem is that, even for genuine custom attributes, it’s not easy to reconstruct the correct source code to produce the attribute based on the information in the metadata. What happens when you define a custom attribute (not a pseudo one) in an application is that, at compile time, an instance of the attribute class is constructed using the parameters you supply in the source ?these may be a combination of constructor arguments and/or public fields/properties. Then the compiler serialises the initialised attribute instance straight out into the metadata. When the assembly is subsequently loaded, these custom attributes are deserialised and so can be accessed and have their properties tested to provide all the functionality for which they are intended. This is great but it doesn’t help us reverse engineer the source code! We don’t actually want an initialised attribute, we want to know what arguments to supply to the attribute constructor to produce the same effect.

There are so many pitfalls here that it’s difficult to know what (if anything) to do. For a large number of cases, it may not be relevant that an assembly employs custom attributes if all we want to do is examine the general structure of the decompiled assembly. For some of the pseudo-custom attributes, we could check the IsSerializable, IsExplicitLayout, IsSequentialLayout, properties of a reflected type and then attempt to reconstruct some of these more well-known ones by adding CodeAttributeDeclaration objects to our CodeDOM model.

I want to keep this decompiler as clean and uncluttered as possible while still demonstrating how reflection can be used as a reverse engineering tool. The line I’m going to take here is to overlook the pseudo-custom attributes but to make you aware that we could glean some of this information from the metadata with a little more effort. It would make our code messier and it may be better to use ILDasm (with the /ADV command line switch) to dump this information. I’m also going to pass over the much more difficult problem of reconstructing a correct parameter list in the call to a custom attribute constructor and just try and give a little bit of an edge by popping out the names of any custom attributes the designers of an assembly have used and to show where they appear. This is exactly what the getCustomAttributes() helper method does:

private CodeAttributeDeclarationCollection
            getCustomAttributes(object[] attrs)
{
// convert Reflection
            custom attributes to CodeDOM
CodeAttributeDeclarationCollection
            attrsDom =
new CodeAttributeDeclarationCollection();

//
            process all incoming attributes
foreach(object attr in
            attrs)
{
// create a CodeDOM
            version
CodeAttributeDeclaration attrDom = new
            CodeAttributeDeclaration();
attrDom.Name =
            attr.ToString();

// add it to the
            collection
attrsDom.Add(attrDom);
}

// return CodeDOM
            collection
return attrsDom;
}

The reflected attributes are presented as an array of objects (although they should all descend from System.Attribute, the reflection classes are able to return any non-CLS compliant attributes too), which is processed in a simple manner to retrieve the names and to produce a matching CodeDOM collection for the relevant declaration.

I’m going to leave that one alone now as we’ve got something much more rewarding to get stuck into.

Methods ?Body Snatching!

If, when you installed the .NET Framework SDK, you didn’t have a browse around the various folders created, then you may not be aware what a wealth of information can be found.

Under the Framework SDK root folder is <FrameworkSDK>\Tool Developers Guide\docs, which contains a number of MS Word documents describing in great detail the structure, layout and content of assembly manifests, the IL instruction code set, the content and meaning of all manner of metadata tables and, in the document Partition II Metadata.doc, you will find (in Chapter 24) a complete description of the layout of a Portable Executable (PE) file (that’s a DLL or an EXE), giving you everything you need to know to navigate to the manifest and locate the pointers to raw IL code bytes for the body of every method defined in the assembly.

If you take a look at that document now and feel a little daunted, you’re in good company ?so did I! But, nevertheless, I realised that I could find the information I needed programmatically by loading the DLL or EXE under examination into an array of raw bytes (rather than by using the Assembly loader we used earlier) and walk my way through this data to find what I needed.

Now, there are also documents in the same folder, which describe an arguably easier way to get at the metadata, and one might suggest that we could have used these when developing our decompiler. But these functions, which provide an unmanaged API, are intended for use by compilers and loaders and provide a very fast but low-level COM interface to both inspect and emit structures in metadata. With its heavy use of pointers and unmanaged data types, the API is not amenable to use from a native C# application ?it’s intended to be used from C++. As the primary purpose of this case study is to demonstrate how easy it is to decompile an assembly into a digestible form without resorting to low level tools, I made a conscious effort to disregard the unmanaged API in favour of a more laborious parsing of the metadata.

Having resolved to adopt this approach, I first took time out to surf the Internet and see if I could find any information to save me a little time. Maybe someone had already tried this and identified a way to accumulate offsets so that the relevant information could be pinpointed more quickly. There wasn’t much point in treading through individual bytes of raw metadata if there was a way to skip over large chunks. Although hopeful of finding something useful, I wasn’t expecting to be quite so fortunate in my quest.

From the web site of an obviously talented and capable developer by the name of Lutz Roeder (see http://www.aisto.com/roeder/dotnet?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq), I found the following.

Managed Reflection IL Reader for .NET

"A class library that extends the existing System.Reflection model with functions for accessing the IL bytecode by reading information out of the corresponding Module file."

Yes, Lutz ?who currently works for Microsoft’s Developer Platform Division ?had produced a C# class library to do exactly what I needed and, what’s more, the C# source code for this library was available for download from the above web site. After a brief correspondence by e-mail, Lutz has kindly agreed to allow me to use this class as part of my sample application for this case study. This is on the understanding that any bugs or technical problems with the class should be fired in my direction. As I have had to make some changes to the original code in order to integrate it with the decompiler, I think this is only fair!

The library consists of a C# source file, Reader.cs, which contains the class ILReader defined in a namespace System.Reflection.ILReader. I don’t intend to describe this class in great detail, really just to show you how it can be integrated into our decompiler to provide the icing on the cake. We can already produce a comprehensive decompilation of the structure of the types defined in an assembly using just the features built into the FCL itself, but the chance to pop in the IL code for each method body in the correct place was just too good to pass up! This information could in any case be obtained from ILDasm, just not in such a convenient manner.

After adding the Reader.cs source file to the DeComp project, we can access the ILReader using only a few additional lines of code. First, add a using statement:

using System.Reflection.ILReader;

Then add a private member field to the Generate class to hold an instance of the reader:

private ILReader reader;

The Reader.cs library defines an interface to be used for loading an assembly. Following the example from the IL reader download, we can add a class to the DeComp project that implements this interface as follows:

class AssemblyLoader : IAssemblyLoader
{
public Assembly Load(string assemblyName)
{
return Assembly.Load(assemblyName);
}

public Assembly LoadFrom(string fileName)
{
return Assembly.LoadFrom(fileName);
}
}

Then we need to amend the existing File / Open menu handler to create and initialise an ILReader just after we load the assembly as shown below:

private void menuFileOpen_Click(object sender, System.EventArgs e)
{
// open an existing csharp source file
if (dlgOpen.ShowDialog() == DialogResult.OK)
{
// create compile unit and load assembly for reflection
codeunit = new CodeCompileUnit();
Assembly mainAssembly = Assembly.LoadFrom(dlgOpen.FileName);

// load for ILReader
reader = new ILReader(mainAssembly.GetModules()[0],
new AssemblyLoader());

This will load a copy of the first module in the assembly (that will be the one containing the manifest) into the ILReader class. If you browse the source code for this class and look through the Microsoft documentation describing the layout of a PE file, you will see in essence what is happening here and how much work it saves us.

ILReader exposes a public method ?GetMethodBody() ?that returns a MethodBody for a given method. MethodBody implements an IEnumerable interface, which allows us to use the foreach syntax to access the IL instructions in the method body. So once we have instantiated an ILReader, we just need to add a couple of lines to the addMethod() helper to retrieve the method body and add it to the CodeDOM method definition as comments. I’ve used another helper method, GetILForMethod(), which does just that.

// add IL as comment block
MethodBody methodBody = reader.GetMethodBody(info);
if (methodBody != null)
newmeth.Statements.AddRange(GetILForMethod(methodBody));

We can do likewise to retrieve the body of instance and type constructors by adding the following lines to our addConstructor() method:

// add IL as comment block
MethodBody methodBody = reader.GetMethodBody(info);
if (methodBody != null)
newcons.Statements.AddRange(GetILForMethod(methodBody));

This leaves just properties (and therefore type indexers). These are slightly different as we potentially have two method bodies to retrieve ?one for the get accessor and one for the set accessor.

The PropertyInfo we get from reflection allows us to access the accessor methods as MethodInfo objects by way of the GetGetMethod() and GetSetMethod() methods respectively. We don’t need to worry about finding these SpecialName methods ourselves and we also don’t care whether we have a genuine property or a type indexer. All we need to do is to check for a read-only or write-only property so that we don’t needlessly attempt to access a non-existent method. Add the following code to the addProperty() method:

   // add IL for get as
            comment block
if (info.CanRead)
{
MethodInfo getinfo =
            info.GetGetMethod(true);
MethodBody methodBody =
            reader.GetMethodBody(getinfo);
if (methodBody !=
            null)
newprop.GetStatements.AddRange(GetILForMethod(methodBody));
}

//
            add IL for set as comment block
if
            (info.CanWrite)
{
MethodInfo setinfo =
            info.GetSetMethod(true);
MethodBody methodBody =
            reader.GetMethodBody(setinfo);
if (methodBody !=
            null)
newprop.SetStatements.AddRange(GetILForMethod(methodBody));
}

To finally complete the decompiler, we can now define the method GetILForMethod(), which returns the decoded IL method body as an array of CodeCommentStatement objects. I have adapted this routine from Lutz Roeder’s example code and in general it works like this:

  • IL instructions are written as text lines to a StringWriter.
  • The StringWriter is converted to an array of strings.
  • The string array is used to build a CodeCommentStatement array.

We start off by declaring a StringWriter and adding some basic details about the method:

private CodeCommentStatement[]
            GetILForMethod(MethodBody methodBody)
{
// return method body
            as decoded IL in CodeDOM comments
StringWriter sw = new
            StringWriter();

Sw.NewLine = "\n";
sw.WriteLine(".maxstack
            " + methodBody.MaxStack);

// list local variables
Type[]
            locals = methodBody.Locals;
if ((locals != null) &&
            (locals.Length > 0))
{
sw.Write(".locals (");
for (int i
            = 0; i < locals.Length;
            i++)
{
sw.Write(locals[i].ToString() + " V_" + i);
if (i !=
            (locals.Length - 1)) sw.Write(",
            ");
}
sw.WriteLine(")");
}

In a similar manner to ILDasm, a list of local variables is produced. Unlike ILDasm, however, ILReader always assigns new variable names. In debug builds, the original variable names can be obtained from the metadata but it is not a straightforward task to retrieve these. In short, ILDasm does it but we don't.

Next, details of exception handlers are listed. These are held in one or more separate data sections following the method body in the assembly and are extracted by ILReader into an array of ExceptionHandler objects. The details of the IL code lines covered by each exception handler are listed at the top of the method body following the list of local variables:

// exception handlers
ExceptionHandler[]
            exceptions = methodBody.Exceptions;
if (exceptions !=
            null)
{
foreach (ExceptionHandler handler in
            exceptions)
{
sw.Write(".try L_" +
            handler.Try.Offset.ToString("x4"));
sw.Write(" to L_" +
(handler.Try.Offset + handler.Try.Length).ToString("x4") + "
            ");

if (handler is Catch)
{
Catch catchHandler =
            (Catch) handler;
sw.Write("catch " +
            catchHandler.Type.ToString());
}

if (handler is
            Filter)
{
Filter filterHandler = (Filter)
            handler;
sw.Write("filter L_" +
filterHandler.Expression.ToString("x4"));
}

if
            (handler is Finally) sw.Write("finally");
if (handler is Fault)
            sw.Write("fault");

sw.Write(" L_" +
            handler.Handler.Offset.ToString("x4"));
sw.WriteLine(" to L_" +
(handler.Handler.Offset +
            handler.Handler.Length).ToString("x4"));
}
}

The IL instructions are written next. These are formatted according to the instruction type and OpCode using a switch statement:

// instructions
foreach (Instruction instruction in methodBody)
{
OpCode opCode = instruction.OpCode;
Object operand = instruction.Operand;
Byte[] operandData = instruction.OperandData;

sw.Write("L_" + instruction.Offset.ToString("x4") + ": ");
sw.Write(opCode.Name + " ");

if (operand != null)
{
switch (opCode.OperandType)
{
case OperandType.InlineNone:
break;

case OperandType.ShortInlineBrTarget:
case OperandType.InlineBrTarget:
int target = (int) operand;
sw.Write("L_" + target.ToString("x4"));
break;

case OperandType.ShortInlineI:
case OperandType.InlineI:
case OperandType.InlineI8:
case OperandType.ShortInlineR:
case OperandType.InlineR:
sw.Write(operand.ToString());
break;

case OperandType.InlineString:
sw.Write("\"" + operand.ToString() + "\"");
break;

case OperandType.ShortInlineVar:
case OperandType.InlineVar:
if (operand is int)
sw.Write("V_" + operand.ToString());
else if (operand is ParameterInfo)
{
ParameterInfo parameterInfo =
(ParameterInfo) operand;
sw.Write((parameterInfo.Name != null) ?
parameterInfo.Name : ("A_" + parameterInfo.Position));
}
break;

case OperandType.InlineSwitch:
sw.Write("(");
int[] targets = (int[]) operand;
for (int i = 0; i < targets.Length; i++)
{
if (i != 0) sw.Write(", ");
sw.Write("L_" + targets[i].ToString("x4"));
}
sw.Write(")");
break;

case OperandType.InlineSig:
case OperandType.InlineMethod:
case OperandType.InlineField:
case OperandType.InlineType:
case OperandType.InlineTok:
if (operand is Type)
sw.Write((Type) operand);
else if (operand is MemberInfo)
{
MemberInfo memberInfo = (MemberInfo) operand;
if (memberInfo.DeclaringType != null)
{
sw.Write(memberInfo.DeclaringType);
sw.Write("::");
}
sw.Write(memberInfo);
}
else
throw new Exception();
break;
}
}
else
{
if (operandData != null)
{
sw.Write("null // " + instruction.OpCode.OperandType + " ");
foreach (Byte b in operandData)
sw.Write(b.ToString("X2") + " ");
}
}

sw.WriteLine();
}

When we reach this point, we have the decoded IL method body in string format (in the StringWriter), with individual lines delimited by newline characters.

To convert these to an array of strings, we can split on the newline delimiter:

// convert to string
string ilcode = sw.ToString();

// split lines into string array
string[] lines = ilcode.Split(new char[] {'\n'});

Then we create an array of CodeCommentStatement, which will be returned as the result of the method:

// create an array of comment statements
CodeCommentStatement[] states = new CodeCommentStatement[lines.Length];

Finally, we build a CodeCommentStatement from each string, add it to the above array and return the result:

   // add strings as comment
            lines
for (int i = 0; i < lines.Length; i++)
{
states[i]
            = new CodeCommentStatement(lines[i]);
}

// return
            array
return states;
}

At last, this completes the reflection-based decompiler. We can give it a test?/p>

Testing the Decompiler

With the download material for this article, I have included a project ?TestDecomp ?that contains all of the following code. This project is a C# class library in which I have coded all of the structures we need to test the various aspects of the decompiler. In this section, I will go through each test, showing the original source code together with the decompiled version. This will not show the TestDecomp source as if I have compiled each version, and it will be in no particular order. I’ll just extract relevant elements from the complete decompilation in order to illustrate particular items of interest. It’s easier to highlight different features this way than by presenting you with two three-page source files!

The first thing to check is that we can decompile the general structure of an assembly in terms of its constituent classes. The source code to do this includes conventional classes, an abstract class and a nested class. The original definitions (edited for clarity) are as follows:

namespace TestDecomp
{
public class
            ClassWithStaticConstructor { }

public abstract class
            AbstractClass { }

public class DescendsFromAbstractClass :
            AbstractClass { }

public class TestClass :
            DescendsFromAbstractClass
{
private class PrivateNestedClass {
            }

public class PublicNestedClass { }
}
}

When the compiled class library is decompiled from the resulting DLL, we get the following source code back (again, edited for clarity) ?see Figure 4 for a screenshot:

namespace TestDecomp
{
public class
            ClassWithStaticConstructor { }

public abstract class
            AbstractClass { }

public class DescendsFromAbstractClass :
            TestDecomp.AbstractClass {
            }

[System.Reflection.DefaultMemberAttribute()]
public
            class TestClass :
            TestDecomp.DescendsFromAbstractClass
{
private class
            PrivateNestedClass { }

public class PublicNestedClass {
            }
}
}

Figure 4 - Testing the Decompiler

You can see we got everything we expected, even in the same order as originally defined (though this is not guaranteed), but with one additional line. The DefaultMemberAttribute has been automatically applied to the TestClass type. This is because (edited out of the above example) I have defined a type indexer in this class. Discussing why the compiler does this is a little out of scope right here but it’s the first of several indications that, in terms of source code, our decompiler actually appears to get back more information than went in! What we’re exposing here is code that the compiler has added on our behalf to implement our source code in an assembly suitable for the CLR. We touched on this when we discovered how the set and get accessor methods are compiled from a property definition and we’ll see more in a while. Just staying with properties and type indexers for a moment, these are the original definitions within TestClass :

public int publicproperty
{
get {
            return privateintfield; }
set { privateintfield = value;
            }
}

public bool this[int bitnumber]
{
get { return
            (bitnumber == 1); }
set { if (value) privateintfield = 1;
            }
}

And this is the decompiled equivalent. First the property:

public int
            publicproperty
{
get
{
// .maxstack 1
// L_0000:
            ldarg.0
// L_0001: ldfld TestDecomp.TestClass::Int32
            privateintfield
// L_0006: ret
}
set
{
// .maxstack
            2
// L_0000: ldarg.0
// L_0001: ldarg.1
// L_0002: stfld
            TestDecomp.TestClass::Int32 privateintfield
// L_0007: ret
}
}

Then the type indexer:

public bool this[System.Int32
            bitnumber]
{
get
{
// .maxstack 2
// L_0000: ldarg.1
// L_0001: ldc.i4.1
// L_0002: ceq
// L_0004: ret
}
set
{
// .maxstack 2
// L_0000: ldarg.2
//
            L_0001: brfalse.s L_000a
// L_0003: ldarg.0
// L_0004:
            ldc.i4.1
// L_0005: stfld TestDecomp.TestClass::Int32
            privateintfield
// L_000a: ret
}
}

You see in both cases how it would not take a great deal of effort to understand the operation of these (albeit simple) accessor methods.

Let’s make sure we handle class declarations and constructors correctly. The ClassWithStaticConstructor is exactly that. It’s defined in TestDecomp like this:

public class ClassWithStaticConstructor
{
public static string staticstring;

static ClassWithStaticConstructor()
{
staticstring = "test";
}

And it decompiles like this:

public class
            ClassWithStaticConstructor
{
public static string
            staticstring;

static ClassWithStaticConstructor()
{
//
            .maxstack 1
// L_0000: ldstr "test"
// L_0005: stsfld
            TestDecomp.ClassWithStaticConstructor::System.String
            staticstring
// L_000a: ret
}

public
            ClassWithStaticConstructor()
{
// .maxstack 1
// L_0000:
            ldarg.0
// L_0001: call System.Object::Void .ctor()
//
            L_0006: ret
}

We can see here that we have decompiled a static constructor and an instance constructor in the same class. Looking at the original source, we see we didn’t actually define an instance constructor ?this again has been generated automatically by the C# compiler, and you can probably deduce from the constructor body that, as we’d expect, it just calls the base class constructor (i.e. from System.Object).

Moving on, we can add some enumerations to this class. We’ll define one that uses bit flags and one that doesn’t. The original definitions are:

[Flags]
public enum BitflagUintEnum : uint
{
UintEnum1 = 0x00000001,
UintEnum2 = 0x00000002,
UintEnum4 = 0x00000004,
UintEnum8 = 0x00000008
}

public enum NormalByteEnum : byte
{
ByteEnum1 = 1,
ByteEnum2 = 2
}

Which decompiles almost verbatim to the following:

[System.FlagsAttribute()]
public enum BitflagUintEnum : System.UInt32
{
UintEnum1 = 1,
UintEnum2 = 2,
UintEnum4 = 4,
UintEnum8 = 8,
}

public enum NormalByteEnum : System.Byte
{
ByteEnum1 = 1,
ByteEnum2 = 2,
}

All that’s happened here is that the type names are now fully-qualified genuine names rather than the C# aliases uint and byte. The initial values have also been correctly regenerated.

We’ll make sure we can handle struct’s as well as classes and enums. So, still within the same class, we’ll add the following original definitions:

private struct PrivateStruct
{
public
            int nSize;
public int nType;

public PrivateStruct(int
            size, int type)
{
nSize = size;
nType =
            type;
}
}

Which we find will decompile successfully as follows:

private struct PrivateStruct
{
public
            int nSize;
public int nType;

public PrivateStruct(int
            size, int type)
{
// .maxstack 2
// L_0000: ldarg.0
//
            L_0001: ldarg.1
// L_0002: stfld
            TestDecomp.ClassWithStaticConstructor+PrivateStruct::Int32
            nSize
// L_0007: ldarg.0
// L_0008: ldarg.2
// L_0009:
            stfld TestDecomp.ClassWithStaticConstructor+PrivateStruct::Int32
            nType
// L_000e: ret
}
}

Here again we can see the benefit of providing a decompiled IL body right in the generated source code.

Moving back briefly to the class level, we need to check that the abstract/virtual/override/new scenarios all work correctly. So in the TestDecomp project, we’ll define an abstract class and a class that is derived from it, with two method overrides.

I’ve added a nonsensical statement in the overridden method bodies just so we can check what action the compiler will take.

The original code is:

public abstract class
            AbstractClass
{
public AbstractClass()
{
}

public
            abstract void abstractmethod();
public abstract void
            anotherabstractmethod();
}

public class
            DescendsFromAbstractClass : AbstractClass
{
public override
            void abstractmethod()
{
int x = 0;
}

public override
            void anotherabstractmethod()
{
int x =
0;
}
}

And the decompiled version:

public abstract class
            AbstractClass
{
public AbstractClass()
{
// .maxstack
            1
// L_0000: ldarg.0
// L_0001: call System.Object::Void
            .ctor()
// L_0006: ret
}

public abstract void
            anotherabstractmethod();
public abstract void
            abstractmethod();
}

public class DescendsFromAbstractClass
            : TestDecomp.AbstractClass
{
public
            DescendsFromAbstractClass()
{
// .maxstack 1
// L_0000:
            ldarg.0
// L_0001: call TestDecomp.AbstractClass::Void
            .ctor()
// L_0006: ret
}

public override void
            anotherabstractmethod()
{
// .maxstack 0
// L_0000: ret
}

public override void abstractmethod()
{
//
            .maxstack 0
// L_0000: ret
}
}

Notice here how the compiler has created a default constructor for DescendsFromAbstractClass, this time calling the correct base class constructor. The compiler has also optimised out our nonsensical method bodies! This is because I have decompiled the Release version of TestDecomp. Decompiling the Debug version retrieves the non-optimised nonsensical methods just as we wrote them. For example:

    public override void
            anotherabstractmethod()
{
// .maxstack 1
// .locals
            (System.Int32 V_0)
// L_0000: ldc.i4.0
// L_0001: stloc.0
// L_0002: ret
}

This is a case where the decompiler may sometimes give us less than we expected! This is a point to bear in mind when testing the decompiler although it can also give some useful insights into your own code when you can examine what the compiler considers important during its optimisation phase.

Moving on to the TestClass definition, we can check that we are able to decompile member fields correctly by declaring all manner of combinations of access modifiers. Again, the original is:

public class TestClass : DescendsFromAbstractClass
{
public int[] publicintarray;
private const string privateconststring = "pcstring";
public int publicintfield;
public static float publicstaticfloatfield;
public const float publicconstfloatfield = 10.5f;
public readonly int publicreadonlyint = 1;
protected int protectedintfield;
protected static int protectedstaticintfield;
protected const int protectedconstintfield = 1;
private int privateintfield;
private static int privatestaticintfield;
private const int privateconstintfield = 1;
internal int internalintfield;
internal static int internalstaticintfield;
internal const int internalconstintfield = 1;
protected internal int protectedinternalintfield;
protected internal static int protectedinternalstaticintfield;
protected internal const int protectedinternalconstintfield = 1;

Followed by the decompiled version:

[System.Reflection.DefaultMemberAttribute()]
public class TestClass : TestDecomp.DescendsFromAbstractClass
{
public int[] publicintarray;
public int publicintfield;
public int publicreadonlyint;
protected int protectedintfield;
private int privateintfield;
internal int internalintfield;
protected internal int protectedinternalintfield;
public static System.Single publicstaticfloatfield;
protected static int protectedstaticintfield;
private static int privatestaticintfield;
internal static int internalstaticintfield;
protected internal static int protectedinternalstaticintfield;
private const string privateconststring = "pcstring";
public const System.Single publicconstfloatfield = 10.5;
protected const int protectedconstintfield = 1;
private const int privateconstintfield = 1;
internal const int internalconstintfield = 1;
protected internal const int protectedinternalconstintfield = 1;

Looking through and comparing these definitions you’ll see that the only failure has been with the readonly field publicreadonlyint. As discussed earlier, although we know from the reflected attributes that the field is readonly, there is no way declare such a field using CodeDOM. This appears not to be the only problem though ?it looks like we’ve lost the initial value too! Well, yes and no, all becomes clear when we consider the constructor for this class. First, the original:

public TestClass(int
            intfield)
{
publicintfield = intfield;
}

Then the decompiled equivalent:

public TestClass(int intfield)
{
//
            .maxstack 2
// L_0000: ldarg.0
// L_0001: ldc.i4.1
//
            L_0002: stfld TestDecomp.TestClass::Int32 publicreadonlyint
//
            L_0007: ldarg.0
// L_0008: call
            TestDecomp.DescendsFromAbstractClass::Void .ctor()
// L_000d:
            ldarg.0
// L_000e: ldarg.1
// L_000f: stfld
            TestDecomp.TestClass::Int32 publicintfield
// L_0014: ret
}

Notice the first three lines in the method body? As I mentioned earlier, this is what the C# compiler does with initialised instance fields. All the initialisation code is moved to the front of the instance constructor. As we saw before, there will always be a constructor, even if we don’t define one, and the compiler always initialises any non-constant fields in this way.

Earlier, we started checking that we could correctly reconstruct overridden methods from abstract base classes. As TestClass sits at the bottom of the previous hierarchy, we can also check that we can detect hidden virtual methods. In TestClass, we’ll define a smattering of methods (bodies omitted):

public override void abstractmethod() {
            }
public virtual new void anotherabstractmethod() { }
public
            void publicvoidmethod(int intparm) { }
public static void
            publicstaticmethod(int intparm) { }
public virtual void
            publicvirtualmethod(int intparm) { }
public int
            publicintmethod(ref int refintparm) { }

Then we can check each of these in the decompiled version:

public override void abstractmethod() {
            }
public new virtual void anotherabstractmethod() { }
public
            void publicvoidmethod(int intparm) { }
public static void
            publicstaticmethod(int intparm) { }
public virtual void
            publicvirtualmethod(int intparm) { }
public int
            publicintmethod(System.Int32& refintparm) { }

Which seems to work fine. We’re handling the virtual, new and override methods correctly and recreating the correct method parameters and types. We’ve even detected the ref parameter, though CodeDOM has rendered the source in a slightly different format.

In TestClass, we had defined two nested classes. For the first of these ?PrivateNestedClass ?we can include some exception handling in the constructor to make sure we decompile that in an understandable manner. So the original source is like this:

private class
            PrivateNestedClass
{
private string privatestringfield =
            "test";
private float privatefloatfield;

public
            PrivateNestedClass(string s, float f)
{
float localfloat =
            1.5f;
try
{
privatestringfield = s;
}
catch(Exception
            ex)
{
throw new
            ArgumentException("Error");
}
finally
{
privatefloatfield
            = f * localfloat;
}
}
}

I’ve added a couple of member fields to make sure these get output at the right level in the nested class and also introduced some different data types and multiple parameters.

The decompiled version of this class appears as follows:

private class
            PrivateNestedClass
{
private string
            privatestringfield;
private System.Single
            privatefloatfield;

public PrivateNestedClass(string s,
            System.Single f)
{
// .maxstack 3
// .locals (System.Single
            V_0)
// .try L_0017 to L_0020 catch System.Exception L_0020 to
            L_002c
// .try L_0017 to L_002e finally L_002e to L_0038
//
            L_0000: ldarg.0
// L_0001: ldstr "test"
// L_0006: stfld
            TestDecomp.TestClass+PrivateNestedClass::System.String
            privatestringfield
// L_000b: ldarg.0
// L_000c: call
            System.Object::Void .ctor()
// L_0011: ldc.r4 1.5
// L_0016:
            stloc.0
// L_0017: ldarg.0
// L_0018: ldarg.1
// L_0019:
            stfld TestDecomp.TestClass+PrivateNestedClass::System.String
            privatestringfield
// L_001e: leave.s L_002c
// L_0020: pop
// L_0021: ldstr "Error"
// L_0026: newobj
            System.ArgumentException::Void .ctor(System.String)
// L_002b:
            throw
// L_002c: leave.s L_0038
// L_002e: ldarg.0
//
            L_002f: ldarg.2
// L_0030: ldloc.0
// L_0031: mul
//
            L_0032: stfld TestDecomp.TestClass+PrivateNestedClass::Single
            privatefloatfield
// L_0037: endfinally
// L_0038: ret
}
}

Without reiterating too many details, you’ll see that this is a fair decompilation of the original code. Notice how the line numbers covered by the try and catch blocks are listed at the start of the decompiled method body.

The last thing to check is events. For this, we’ll use the other nested class ?PublicNestedClass ?and just define a delegate and an event:

public class PublicNestedClass
{
public delegate void publicdelegate(object sender, EventArgs e);
public event publicdelegate publicevent;
}

This should be pretty straightforward, the event member type was one of the first (and easiest) we came across. However, when we come to decompile this class, as you’ve probably already guessed, we get much more than we bargained for:

public class PublicNestedClass
{
private TestDecomp.TestClass.PublicNestedClass.publicdelegate publicevent;

public PublicNestedClass()
{
// .maxstack 1
// L_0000: ldarg.0
// L_0001: call System.Object::Void .ctor()
// L_0006: ret
}

private event TestDecomp.TestClass.PublicNestedClass.publicdelegate publicevent;

public sealed class publicdelegate : System.MulticastDelegate
{

public publicdelegate(object @object, System.IntPtr method)
{
}

public virtual void EndInvoke(System.IAsyncResult result)
{
}

public virtual System.IAsyncResult BeginInvoke(object sender,
System.EventArgs e, System.AsyncCallback callback, object @object)
{
}

public virtual void Invoke(object sender, System.EventArgs e)
{
}
}
}

If you’ve read any of the .NET framework documentation covering events and delegates then perhaps this will not be so much of a surprise. In any case, let’s pick it apart and see what the compiler actually did when it compiled our original two lines of code.

First, ignore the automatically generated constructor. We’ve covered that already and it has nothing to do with the event or delegate. However, the following line,

public event publicdelegate publicevent;

actually caused the compiler to generate three things. Two of these we don’t see in the decompiled code, they are methods with the SpecialName attribute set (we deliberately ignored these, remember?). They are named add_publicevent() and remove_publicevent() and are the methods used to allow objects to register and unregister interest in the event.

You can see these methods using ILDasm as before (see Figure 5):

Figure 5 - Event Methods

The third thing generated by the event declaration is a private field named after the delegate ?shown by this line in the decompiled source:

private event TestDecomp.TestClass.PublicNestedClass.publicdelegate publicevent;

You’ll see that this field has a data type of publicdelegate, so what exactly is this type? Well you can see from the decompiled code and from Figure 5 that publicdelegate is a complete nested class definition. The class is derived from System.MulticastDelegate (as are all delegate types) and has four methods defined. It’s marked as public because we defined our delegate as public. Things become clearer when you realise now why a delegate is the only C# entity that can be declared at global scope ?this is because it’s actually a class, so we’re not really breaking any rules.

The four methods ?the constructor, Invoke, BeginInvoke and EndInvoke ?handle the mechanics of invoking call-back methods in objects who have registered an interest in the corresponding event. This is too deep a subject to go into here, the real point being the appreciation that decompiling a .NET assembly could end up telling you as much about the way the C# compiler and the CLR go about their business as it does about the application code itself.

You may also notice that there are no method bodies in the delegate class. This is because they are marked as RunTime code and so cannot be accessed in the same way as our other method bodies. If you examine the ILReader class, you will see that it specifically ignores certain types of method, RunTime being just one of them. The others are Native and OPTIL, representing native code (e.g. x86 instructions) and parts of the .NET framework infrastructure respectively.

We could have produced a cleaner decompilation of the delegate declaration ?now we know how it’s implemented ?by using the parameter list from the delegate’s Invoke() method to create a CodeDOM CodeMemberDelegate and so recreate the original source code. However, we would still need to filter out both the nested delegate class and the private event field ?neither of which are marked in any special way in the metadata, thus making our task a little more complicated. Overall I think we have a pretty good decompiler that has the side effect of opening our eyes a little to what’s going on under the hood.

Application Setup

If you want to try this out for yourself, download the support material for this case study from the C#Today web site and follow the instructions in the ReadMe.txt file included.

The download contains the DeComp project and the TestDecomp class library used for testing.

Limitations

The decompiler suffers from some limitations. These fall into three categories: limitations of reflection, limitations of CodeDOM, and technical limitations.

Limitations of reflection include the inability to get directly to the IL code for a method. We needed to go to some lengths for a workaround, although this was still possible without needing to drop down into low level tools. Also, and perhaps not relevant for C# assemblies, there is currently no way using reflection that we can retrieve default values for optional parameters. A relatively minor problem.

Limitations of CodeDOM are probably a little more serious. For instance, we cannot define a readonly field using CodeDOM. Neither can we define a type indexer. For the latter, although we came up with a workaround, the effect of this is that we limited ourselves to C# as the decompilation language. If it were not for our custom treatment of type indexers, we could have offered a choice of C# or VB.NET code as the output from the decompiler. To do so now would require more workarounds and so a less optimal solution.

What I’ve termed technical problems are probably the most difficult to overcome. To reproduce native C# source code in as near a match as possible to the original is hard. Here, we have to fight the compiler itself to reverse engineer events and delegates from its automatically generated classes, fields and methods. We also have a big job on our hands to rebuild the correct constructor arguments for custom attributes. When it comes to compiler optimisation we have a slightly subtler problem. If the objective is to reverse engineer the exact source code used during compilation then we’re almost certainly lost ?in an assembly of any size, the compiler will have removed redundant code, altered parameter lists, inlined code, and a whole host of other things that really mean that information has been thrown away during compilation. This is not information needed to make the assembly operate as designed, just information that may have provided additional clues as to the original nature of the source. This may or may not be a problem depending on one’s motives for decompiling in the first place.

Case Study Review

This case study set out to reverse engineer compiled .NET assemblies into C# source code. We tried to keep things as simple as possible and to keep the solution within the bounds of the C# language and the framework class library (FCL). The objective was not to demonstrate how clever we can be, but rather how little we need to know in order to glean a great deal of information about an assembly ?much more than we ever could with pre-.NET executables.

While we did have to dig a little deeper than intended to produce decompiled IL method bodies, I think we succeeded in these aims. If what you’ve discovered has given you cause to think about the safety of your own intellectual property, then I have achieved what I set out to do.

The second part of this case study will look at ways of attacking this problem. We’ll look in detail at code obfuscators?how they work, what are their strengths and weaknesses, we’ll discuss alternative methods of securing sensitive code, and finally develop an example application demonstrating at least one technique for protecting your valuable code from prying eyes.

Case Study Information
Author | Roger Rowland
Editor | Adam Ryland
Reviewers | Kaushal Sanghavi, Owen Cutajar, Sean Schade

If you have any questions or comments about this case study, or suggestions for further pieces, please contact the technical editor. If you wish to talk about the case study with other subscribers, please visit our forum here.

C#Today - © 2002 WroxPress Limited


Explaining the System.Security Cryptography Namespace

Explaining the System.Security Cryptography Namespace
http://www.csharptoday.com/content.asp?id=1807
Article Rating: 4
Published on January 25, 2002
Abstract
In this article, Prasad DV looks at the advantages of using the useful System.Security.Cryptography namespace in the Microsoft .NET Framework ?especially the Public Private Key-Pair algorithm called RSA. The author goes on to show how a secure commercial web site can be written with ease using the Framework, and the important security considerations involved.
Article

Security is a matter of great concern for web developers. If the World Wide Web is to continue its natural progression from where it is today, it is up to us, web developers, to address the security related issues - like security of communications between browsers, security of web storage of sensitive information, and proper authentication. Microsoft's .NET Framework has given the web developer several tools and technologies in this regard. These tools are contained with in the System.Security.* namespaces of the framework. Among these, the web developers can use the class library available under the namespace System.Security.Cryptography for ensuring encrypted transmission of sensitive data and their storage. However, there is not much by way of documentation either in the SDK or in several published articles on this topic.

This article will present a simple overview of the cryptography namespace and demonstrate how we can use the RSA and DES encryption technologies in a practical scenario. The article will throw light on how one can use asymmetric key encryption algorithm by separation of the public key from the key pair and use it for one-way encryption.

System Requirements

Server:

  • Windows NT, Win2K, Win XP with IIS
  • Microsoft .NET Framework Beta 2
  • Microsoft SQL2000 server

Client

  • Win9x / Win NT and above
  • Microsoft.NET Runtime
  • Internet Explorer 5.5 or above

Defining The Problem

At present, commercial websites that do business on the Internet and require financially sensitive information like credit card numbers operate from web servers which are enabled with Secure Socket Layer (SSL). In this scheme of things, the server obtains a certificate called an X509 certificate from a trusted Certification Authority. A public-key certificate is a digitally signed statement from one entity, saying that the public key (and some other information) of another entity has some specific value.

A certificate is the standard way to pass around public or symmetric keys. The message format for a certificate includes a digital signature from a Certificate Authority. Trusting a Certificate Authority in effect means you trust that any certificates created by that authority properly identify the owner for whom the certificate was issued. Therefore the private key that corresponds to the public key in the certificate is deemed to be known only by the specified owner. The most common way for representing certificates is to use a standard format referred to as X.509.

Probably the most widely visible application of X.509 certificates today is in web browsers (such as Netscape Navigator and Microsoft Internet Explorer) that support the SSL protocol. SSL (Secure Socket Layer) is a security protocol that provides privacy and authentication for your network traffic. These browsers can only use this protocol with web servers that support SSL.

Other technologies that rely on X.509 certificates include:

  • Various code-signing schemes, such as signed Java ARchives, and Microsoft Authenticode.
  • Various secure E-Mail standards, such as PEM and S/MIME. E-Commerce protocols, such as SET.

Although you can create an X509 Certificate yourself (in fact .NET Framework provides you with a tool to do so) the more prudent approach would be to get it from a Certificate Authority. However, .NET Framework enables one to do encryption and decryption of data between the browser and the server, without using the Secure Socket Layer.

Away from certificates, at the other end of cryptography lies encryption. Encryption comes in two flavors - Symmetric and Asymmetric. These are the two common categories of encryption schemes, both involving keys:

  • Symmetric encryption schemes use the same key to both encrypt and decrypt a message. A symmetric key is often called a session key.
  • Asymmetric encryption schemes have two different keys: a public key and a private key. A message encrypted with a public key can only be decrypted with the private key. Private/Public key encryption mechanisms provide a lot of power for performing authentication and insuring data integrity.

The common language runtime uses a stream-oriented design for implementing symmetric algorithms and hash algorithms. The core of this design is CryptoStream, a class that derives from the Stream class. Stream-based cryptographic objects all support a single standard interface (CryptoStream) for handling the data transfer portion of the object. Because all of the objects are built on a standard interface, you can chain together multiple objects (such as a hash object followed by an encryption object), and you can perform multiple operations on the data without needing any intermediate storage for it. The streaming model also allows you to build objects from smaller objects. For example, a combined encryption and hash algorithm can be viewed as a single stream object, even though this object might be built from a set of stream objects.

Typically, private-key algorithms, called block ciphers, are used to encrypt one block of data at a time. Block ciphers (like RC2, DES, TrippleDES, and Rijndael) cryptographically transform an input block of n bytes into an output block of encrypted bytes. If you want to encrypt or decrypt a sequence of bytes, you have to do it block by block. Because the size of n is small (n = 8 bytes for RC2, DES, and TripleDES; n = 16 [the default], n = 24, or n = 32 bytes for Rijndael), values larger than n have to be encrypted one block at a time.

The block cipher or Symmetric Algorithm classes provided in the base class library use a chaining mode called Cipher Block Chaining (CBC), which uses a key and an Initialization Vector (IV) to perform cryptographic transformations on data. For a given private key k, a simple block cipher that does not use an initialization vector will encrypt the same input block of plaintext into the same output block of ciphertext. If you have duplicate blocks within your plaintext stream, you will have duplicate blocks within your ciphertext stream. If an unauthorized user knows anything about the structure of a block of your plaintext, she can use that information to decipher the known ciphertext block and possibly recover your key. To combat this problem, information from the previous block is mixed into the process of encrypting the next block. Thus, the output of two identical plaintext blocks is different. Because this technique uses the previous block to encrypt the next block, an IV is used to encrypt the first block of data. Using this system, common message headers that might be known to an unauthorized user cannot be used to reverse engineer a key.

Use of a symmetric algorithm requires that only the sender and the receiver know the secret key. In the .NET Framwork, DESCryptoServiceProvider, RC2CryptoServiceProvider, and TripleDESCryptoServiceProvider are implementations of symmetric algorithms. However, in a web scenario when the customer and the shop owners are complete strangers living in two different parts of the globe, there is no way of having a common secret key, unless it is sent through the web.

The .NET Framework provides the following classes that implement public-key encryption algorithms:

DSACryptoServiceProvider

RSACryptoServiceProvider

In the .NET Framework, there are "abstract classes" that define cryptographic services in an abstract fashion (without a concrete implementation). Apart from these abstract classes, the Framework also provides limited implementation of these abstract classes. A cryptographic service is always associated with a particular algorithm or type, and it either provides cryptographic operations (like those for digital signatures or message digests), or generates or supplies the cryptographic material (keys or parameters) required for cryptographic operations.

Typically, take the case of a shopping web site. When a new user registers with the web site by providing his personal information, he is also required to give out his credit card number to the store owners. If this transmission is done using the open HTTP channel, any one can snoop on the TCP Ports using listeners and get the card information. After the card information reaches the store owners, they typically store the information in a database table. Although the databases are structured to allow only authenticated users, since they store a vast volume of data, there are many people who have access to these data tables. They will be able to read the open data in the tables with tools like Query Analyzer.

In this situation, System.Security.Cryptography classes can be used to ensure encrypted transmission of such sensitive data and their encrypted storage at every point - except at the bank. The following diagram illustrates such a scenario.

Fig 1: The Scheme of encryption.

Let's examine the case of an imaginary garment shop - say "UShopSecure.com" - and an imaginary bank "SecureBank.com" The shop's website is designed like any other commercial website - akin to Microsoft's ASP.NET show-case "www.IBuySpy.com". Therefore, we need not go into detail regarding the architecture of the website, which can be read from the documentation provided by the IBuySpy site.

The site consists of a default "Welcome" page, a "Log-in" page, a "Registration Page", a Products Showcase page, individual product pages, and a shopping cart mechanism. The entire site is data driven and uses SQL Tables, of which the Customers table handles the customer information.

The SecureBank - the imaginary bank - exposes a web-service called CardService to enable online authentication of credit cards issued by it. This web service is expected to use secure communication of the card numbers. We will be examining the aspects governing web security in such a scenario.

Defining The Solution

The Design Consideration of our solution are as follows:

  • The .NET Architecture provides hosting of assemblies on Internet Explorer 5.5. The HTML document can provide information on which assemblies to download, the assemblies' locations, and the location of a configuration file that can provide additional information. The CLR takes away the task of registering such assemblies with the System Registry. Therefore, encryption at the client end will be achieved by taking advantage of this mechanism. (Note: code deployed from the Internet typically has the default Internet permissions set by security policy, which allow code to perform only a limited set of functions)
  • Web Service Security is in an evolving phase. The Web Services Security Language draft that defines security mechanisms for SOAP messages relies heavily on work done for XML messages in general. The XML-Signature Syntax and Processing proposed recommendation by the World Wide Web Consortium (W3C) defines a mechanism for creating and verifying digital signatures for all or part of an XML message. It also indicates an XML-compliant mechanism for holding a digital signature within an XML document. Therefore, we shall design our solution in such a way that its concepts can be used to make it W3C Standards compliant as and when such standards evolve.
  • The RSA Cryptography implementation in the .NET Framework poses a limitation - the RSACryptoServiceProvider class, the only implementation of the RSA, restricts us to 8 bytes for the length of the data to be encrypted. However, there is no such restriction with regard to the DES Encryption implementation. Therefore, we will be constrained to use the RSA only for exchanging the common DES Key, while DES itself will be used for encryption of the actual data.

Database Design

The database of the UShopSecure website consists of several tables of which only the customers table is of any interest to us - as it contains the card information.

Fig 2: 'customers' Table structure

The CardNo field has been given a length of 150 bytes - in order to enable us to store encrypted card numbers since encryption can increase the size of the data.

The customers table is encapsulated in a class object called customersDB, found in the file customersDB.cs in the code download. This class object exposes the required public methods as a business object and implements the database access within itself.

Database structure

We will also be using some stored procedures in order to facilitate our database operations. The CustomerAdd procedure adds a new customer into the database and returns the CustomerID as an output value. The CustomerDetail procedure takes the CustomerID as the parameter and returns the customer details. The UpdateCardInfo takes the credit card information and updates the table with these. For simplicity, we are not adding constraints like uniqueness of Names, Email data, etc., and the table will take multiple entries for the same person.

The secure bank uses the SecureBank database and its Accounts table is structured as follows:

Figure 2(a) : The 'accounts' data table

The accounts database is exposed to the code in the form of a business object, accountsDB - a class object that encapsulates data access to the table through public methods.

The SecureBank also exposes a webservice, CardService.Service1. The web service comprises of two web methods, GetPublicKey and VerifyCardInfo. When the web service application is started, by its first invoke, the constructor creates an RSACryptoServicePRovider object and saves it in its applications scope under the name ' key', with the following code:

RSACryptoServiceProvider key;
public
            Service1()
{
InitializeComponent();
if (Application["key"]
            == null)
{
Application["key"] = new
            RSACryptoServiceProvider();
}
key =
            (RSACryptoServiceProvider)Application["key"];

}

The GetPublicKey web method returns an XmlString value representing the Public Key extracted from the RSA object saved earlier in the Application Scope. This is done by the following code:

[WebMethod]
public String
            GetPublicKey()
{
try
{
return key.ToXmlString(false);
}
catch (Exception e)
{
return
            e.ToString();
}
}

The "false" value to the includePrivateParameters parameter in the RSACryptoServiceProvider's ToXmlString method ensures that the XmlString given out represents only the Public Key of the RSA Key pair. On the other hand, if the value to this parameter is true, the method will spill the beans - giving away the Public and Private Keys!

The ValidateCard method exposed by the CardService takes the DES Encrypted Card Number as an UrlEncoded String value, and a Byte Array representing the encrypted DES Key value - which should have been encrypted using the PublicKey obtained from the earlier web method - with which the Card Number is encrypted. The code for the method is as below:

[WebMethod]
public bool
            ValidateCard(String RSACypherData, Byte[] Temp)
{
byte[] bIV =
            { 0, 0, 0, 0, 0, 0, 0, 0 };
Byte[] _SessionKey =
            key.Decrypt(Temp, true);
DESCryptoServiceProvider des = new
            DESCryptoServiceProvider();
des.IV = bIV;
des.Key =
            _SessionKey;
ICryptoTransform cc = des.CreateDecryptor();
Temp
            = null;
Temp =
            HttpUtility.UrlDecodeToBytes(RSACypherData);
System.Text.ASCIIEncoding
            textconverter = new System.Text.ASCIIEncoding();
String
            _CardNumber = textconverter.GetString(cc.TransformFinalBlock
            (Temp,0,Temp.GetLength(0)));
AccountsDB cust = new
            AccountsDB();
return
            cust.CardExists(_CardNumber);
}

First - a byte array called bIV is used as an Initialization Vector - which is an added security measure in addition to the Key Value for encryption. This array is initialized to contain all zeros. However, we can change these values provided such changes are hard coded into the design at the place of Decryption also.

Next - the key property created earlier by us, when the application was started, is used by us to decrypt the SessionKey, using the RSA object's Decrypt method. Here, we give the value true to the parameter includePrivateParameters, as we should do the decryption using the Private Key also.

Next - a DESCrypto object is created using the IV and the SessionKey value extracted by us.

In order to decrypt the credit card number using the DESKey, a DESDecryptor object, which is an instance of the IcryptoTransform, is created from the DESKey. This encryptor has a TransformFinalBlock method that takes the data to be decrypted and returns a byte array containing the Decrypted value of the credit card. This byte array is transformed into a string value by instantiating a System.Text.ASCIIEncoding object and invoking its GetString method. This string value is then passed on to the customerDB's CardExists method together with the customer's name, which is used to look through the database to see if there is a credit card corresponding to the card number, and returns a boolean value. For the sake of simplicity, checking of the card's validity date and its card type have been omitted. However, it will be a breeze to add these to the web method.

Component Design

For the client side encryption, we shall develop a simple 'dll' file based component assembly called CryptoUtil. The component exposes four public properties: CardNumber that takes the plain card number as a string; PublicKey that takes the HtmlEncoded XML string containing the public key of the server; SessionKey that returns the Encrypted Sessions key produced under the DES Algorithm and encrypted using the RSA algorithm; and CypherData that returns the encrypted card number encrypted using the DES algorithm. Both the latter two return values are UrlEncoded, so that they do not get altered in Http transmission. The System.Web.HttpUtility class contains the URLEncode method.

The component assembly is delivered to the client using the <OBJECT> tag, which is placed within the Register.aspx page as follows:

<OBJECT id="Encryptor" classid =
"http:CryptoUtil.dll#SecureStore.CryptoUtil"></OBJECT>

The classid property of the object tag specifies that the object is to be obtained from the server through Http, and its dll file name besides the unqualified class name.

Component Coding

In order to avoid calling methods within such embedded components, the method calls are done within the control, as soon as the user places the input properties. This is done as follows:

private String _CardNumber =
            "";
public String CardNumber
{
get {return
            this._CardNumber;}
set
{
_CardNumber = value;
if
            (_PublicKey!="") this.EncryptData();
}
}

You may note that when the property is set, it checks if the other required value _PublicKey is also set, and then calls the EncryptData method. The method works as follows:

public void EncryptData()
{
// Obtain
            the RSACrypto Key using the Public key given to us
byte[] bIV = {
            0, 0, 0, 0, 0, 0, 0, 0 };
key.FromXmlString(_PublicKey);
//
            Obtain the Session Key from the RSAEncryption
// Generate a
            SessionKey using DES Algoritham
DESCryptoServiceProvider des =
            new DESCryptoServiceProvider();
des.GenerateKey();
des.IV =
            bIV;
Byte[] SessionKey = des.Key;
// Encrypt the Credit Card
            No using the DES Algoritham
ICryptoTransform cc =
            des.CreateEncryptor();
Byte[] _CardNo =
            ConvertStringToByteArray(_CardNumber);
Byte[] EncodedCardNo =
            cc.TransformFinalBlock(_CardNo, 0, _CardNo.GetLength(0));
//
            Encrypt the Session Key Using the RSA Algoritham
Byte[]
            EncodedSessionKey = key.Encrypt(SessionKey, false);
_SessionKey =
            HttpUtility.UrlEncode(EncodedSessionKey);
_CypherData =
            HttpUtility.UrlEncode(EncodedCardNo);
}

The process of encryption in this method is exactly the opposite of the decryption process shown in the CardService's VerifyCard method:

  • First - a byte array called bIV is created and initialized with all zeros, exactly the same value as found in the Bank's web method.
  • Next - the RSAKey is created from the Public Key XML PublicKey String using the RSACryptoServiceProvider.FromXmlString method. Any data encrypted from this key can be decrypted only by the person holding the correct RSA KeyPair corresponding to the PublicKey used.
  • Next - a DESCrypto object is created using the IV.
  • The Key value is extracted from the DESCrypto object, in order to be used as the session key. This session key must be provided to the user of the data so that he can decrypt the data encrypted using this particular instance of the DESKey.
  • In order to encrypt the credit card number using the DESKey, a DESEncryptor object, which is an instance of the ICryptoTransform, is created from the DESKey. This Encryptor has a TransformFinalBlock method that takes the data to be encrypted and returns a byte array containing the encrypted value of the credit card.
  • For transmitting this byte array in a web scenario, we need to convert it into a string. To do that, we take recourse to the HttpUtility object's UrlEncode method. Other options, like the GetString methods within System.Text will not work in a web scenario, since these encoding schemes are based on ASCII, Unicode, Base8, or Base16 characters, and cannot correctly encrypt and decrypt escape characters.

UI Design

In a web project, the UI design comes in the form of Web Page design. Our UShopSecure web site has several web pages, of which Register.aspx is the one important to us, as it concerns Cryptography. The design of the page consists of two ASP.NET User components called Header.ascx and Menu.ascx. These components are consistently used in the site for uniformity of appearance. The Register page includes five text boxes for user name, email, password, password-confirmation and credit card number. Two hidden fields called KeyValue and Cypherdata are placed by us in the web page to enable us obtain the encrypted data back from the user. Appropriate field validator controls are included in the design to ensure data integrity. The submit button is attached to a CustomValidator control, which directs the code execution to a client side function called Encrypt () when the user presses the Submit button. This client side javascript function interacts with the CryptoUtil object placed by us within the Menu user component using the <OBJECT> tag, as explained earlier, as follows:

<SCRIPT
            language="javascript">
function Encrypt() {
if
            (document.all.txtCardNo.value=='') return true;
document.all.Encryptor.PublicKey = ' <here the value is hard
            coded by the ASP.NET>';
document.all.Encryptor.CardNumber =
            document.all.txtCardNo.value;
document.all.KeyValue.value =
            document.all.Encryptor.SessionKey;
document.all.Cypherdata.value
            = document.all.Encryptor.CypherData;
document.all.txtCardNo.value = '';
return true;
}
</SCRIPT>

All that the javascript function does is to place the value, if any, typed by the user in the text box txtCardNo into the CardNumber property field of the Encryptor object, and then read the values found in the SessionKey and CypherData property fields of the Encryptor object and place them into the two hidden fields KeyValue and Cypherdata. It also deletes the actual card number in the text box by replacing its value with an empty string.

The above code is placed within the body of the web page in code within the ASPX page's Page_Load event handler. This is done by writing all the text of the javascript function into a StringBuilder object called sb, and placing it within the web page through a HtmlGenericControl, which again is placed inside a PlaceHolder control called PH, as follows:

   HtmlGenericControl
            obj;
obj = new
            HtmlGenericControl("SCRIPT");
obj.Attributes.Add("language",
            "javascript");
obj.InnerText =
            sb;
this.FindControl("PH").Controls.Add(obj);

UI Code

In the server side, after the user presses the Submit button and the client side encryption takes place, the btnSubmit_Click event handler takes control. The following code passes the value returned by the user in the KeyValue hidden input to a private method DecryptKeyInfo, which uses the RSAKey stored in the Application object of the web application:

String KeyNumber = "";
if (KeyValue.Value
            != "")
{
KeyNumber = this.DecryptKeyInfo(KeyValue.Value);
}

The code of the DecryptKeyInfo is similar to the first few lines of code contained in the verifyCardInfo web method we saw in the CardService web service.

After obtaining the decrypted key value, the btnSubmit_Click handler calls another private function, VerifyCardInfo, whose code reads:

bool VerifyCardInfo (String CardNumber,
            String KeyValue)
{
Service1 service = new
            SecureStore.localhost.Service1();
String RSAKey =
            service.GetPublicKey();
RSACryptoServiceProvider NewRSA = new
            RSACryptoServiceProvider();
NewRSA.FromXmlString(RSAKey);
Byte[]
            SessionKey = HttpUtility.UrlDecodeToBytes(KeyValue);
Byte[]
            EncodedSessionKey = NewRSA.Encrypt(SessionKey, false);
String
            _CypherData = CardNumber;
bool RetVal =
            service.ValidateCard(_CypherData, EncodedSessionKey);
return
            RetVal;
}

Here you may note the reference to the SecureStore.localhost.Service1 class object. It is nothing but the proxy class of the CardService web service obtained using the wsdl.exe command line utility. The above function first obtains the RSA PublicKey of the web service by invoking the GetPublicKey method upon the proxy object. It then encrypts the KeyValue decrypted earlier, now using the Public Key of the web service. Finally it calls the VerifyCardInfo method of the Web Service through its proxy to obtain the validity of the card.

Now, we once again return to the btnSubmit_click event handler. If the verification with the bank fails, the page is returned to the user with the message " Invalid Card Number ". Otherwise, the other details like the card type and card validity date are extracted from the form and stored into the database. It is of particular interest to note that the information concerning the card number stored into the database is only the encrypted data. For example, the following figure shows the contents of the form typed in by a user and the values one may see in the database of the Store.

The Sample Application

The zip file download for this article contains the source code for the entire website "UshsopSecure.com", and the "CardService" web service exposed by the "SecureBank".

Application Setup

  • Unzip the files to your hard disk. As the file paths have been specified within the zip file, the unzipping action will create two directory structures, one for the UShopSecure and one for the CardService.
  • Run the UShopSecure.bat file contained within the UShopSecure Directory. The setup has been scripted to take care of almost everything, including configuring the IIS for the virtual directory, etc, except the setting up of the databases filling some initial data into the required data tables.
  • Open the SQL Server and create two databases with the names SecureStore and SecureBank, and run the 5 .sql files in both the newly created directories using the SQL Server's Query Analyzer.
  • Open the web.config files in each of the two directories and type in your SQL username and password in the configuration string placed within the appSettings tag.
  • Just open your browser and type "localhost/UshopSecure". Initially, the credit card number "1111" will be validated by the bank against the author's name, 'Prasad DV'. To validate other numbers, you must manually update the accounts table of the SecureBank database.

Any Limitations or Further Work

The credit card validation is done only upon the name and the card number. In reality, it needs to be done based on the validity date and the card type. The store site does not include code for updating the card information in case the user does not give the card particulars at the first instance. Necessary code needs to be written for this.

Conclusion

The Microsoft .NET platform provides solution to secure transmission of data through encryption and decryption. This is indeed, a very powerful tool in the hands of a web developer. However, in wrong hands, this technology can be used for harmful purposes. In the RC and the final version of the Framework, the ability to encrypt and decrypt using the key pair algorithm has been restricted to platforms having High Encryption pack installed - thus making these features unavailable for certain countries of the world listed in the US Export Regulations. Still, it will be worthwhile for the programmers to make hay while the sun shines!

Article Information
Author | Prasad DV
Technical Editor | Adam Ryland
Author Agent | Charlotte Smith
Project Manager | Helen Cuthill
Reviewers | Iikka Meriläinen, Kaushal Sanghavi

If you have any questions or comments about this article, please contact the technical editor.

C#Today - © 2002 WroxPress Limited


Secure Authentication Using a C# Implementation of the Zero Knowledge Proof

Secure Authentication Using a C# Implementation of the Zero Knowledge Proof
http://www.csharptoday.com/content.asp?id=1795
Article Rating: 3.5
Published on January 10, 2002
Abstract
The proliferation of far-flung distributed processing calls for enterprise architects to be familiar with the various methods of authentication. Zero Knowledge Proofs that use a cryptographic hash are lightweight alternatives to authentication methods that use public key cryptography. The .NET framework makes an implementation of a zero knowledge protocol easy. In this article, Walter F Schoellmann takes a look at Zero Proof Knowledge, and creates an application to help the reader understand the concepts involved.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

.NET makes far-flung distributed processing a real possibility. While .NET provides a rich framework of enabling technologies, the responsibility is on us to recognize the issues and design patterns of distributed processing and to be able to implement them using the framework. A primary issue of distributed systems is the authentication of users and system components. Authentication, privacy, integrity and non-repudiation are the domain of modern cryptography. Cryptography provides several ways to implement authentication based on a mutual secret. This article demonstrates how components can prove they know a mutual secret without revealing or compromising it in the process. In cryptographic circles, this concept is known as the Zero Knowledge Proof. A Zero Knowledge Proof may seem like an impossible notion, but the scheme is really pretty simple, easy to implement, and very useful in distributed applications.

Introduction

It is challenging to prove that you know a secret without revealing or compromising it in the process. Consider this dialog between Sergeant Schultz and Colonel Hogan in a fictional episode of Hogan's Heroes:

Sgt. Schultz: "The war will be over soon, Der Fuhrer is planning a surprise attack on your comrades."

Col. Hogan: "Schultz, I don't believe you. Der Fuerher would never trust you with that information."

Sgt. Schultz: "Oh yes he would and he did. I am his trusted confidant."

Col. Hogan: "It isn't true."

Sgt. Schultz: "Oh yes it is."

Col. Hogan: "Is not."

Sgt. Schultz: "Is too."

Col. Hogan: "Prove it, if you can."

Sgt. Schultz: "The attack is on Washington, D.C. next Wednesday at 9:00 a.m."

Col. Hogan: "Well Schultz, I guess your right. You must be Der Fuhrer's trusted confidant. I'm impressed."

It is easy to make fun of Sergeant Schultz for his inability to prove that he knew the secret with revealing it, but the sad fact is that the common proof of something is to reveal it. When your web application needs to know who the user is, do you require the user to state his name and reveal his password?

The Zero Knowledge Proof is a cryptographic protocol that makes a proof of knowledge possible without disclosure. The Zero Knowledge Proof is a design pattern worthy of architectural attention and space in your professional toolbox.

Authentication in Distributed Systems

A few short years ago I was working for a .com company that had the vision to propose a new form of postage to the United States Postal Service. It was a fantastic concept! We wanted to give anyone the ability to purchase postage over the Internet, and to print the postage right on his or her own envelopes from a personal computer. We were sure we would all be rich. After all, we had a replacement for the ubiquitous "postage meter". The Postal Service liked the idea, but as you can imagine, they were concerned about fraud. They wanted to know how they would authenticate the printed postage, and how we would authenticate the users of the system, the postage accounts, and the system components themselves. Internet Postage is a reality today, a part of .com history.

If you are a enterprise architect, or want to be, these are the kinds of authentication challenges that you can be expected to face as enabling technologies, like .NET, make far-flung distributed processing a real possibility. Modern cryptography provides many design patterns to address the pervasive problem of authentication. We made use of several in our Internet Postage solution. While I recommend that you become familiar with as many of these design patterns as possible, the focus of this article is to discuss the concept of the Zero-Knowledge Proof. For a thorough, yet approachable presentation of cryptography and authentication techniques, check out "Applied Cryptography", by Bruce Schneier (http://www.amazon.com/exec/obidos/ASIN/0471117099/qid=1009794901/sr=1-1/ref=sr_1_11_1/107-4361548-5072512?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq).

The architectural shift to the web services paradigm means that application designers can no longer count on their objects being packaged together, run in physically controlled environments, and handled by trusted parties. The user of an application is just as likely to be another application. After all, that's what the web services paradigm is all about. Worms, viruses, and impostors unable to authenticate themselves as valid users of an order fulfillment web service, for example, could be denied requests.

Not all the challenges that lie ahead are as esoteric as my experience with Internet Postage. Consider how similar it would be today for an entertainment company to offer tickets to concerts or other live events over the Internet. The purchasers could print the tickets using their personal computers and have the tickets authenticated as they enter the event. Also, consider a much simpler example of a web site that requires a user to enter a name and a password to validate his or her identity. Before .NET, most developers simply placed a login page in a separate site directory and enabled https (Secure Socket Layer), so that the name and password would be encrypted before they were sent to the web server. Is this familiar approach the best approach? What is the computational cost of SSL? What are alternative methods of authentication in .NET?

There are a number of authentication methods provided by the Microsoft .NET platform available to application architects:

  • Windows: Basic, digest, or Integrated Windows Authentication (NTLM, Kerberos)
  • Microsoft Passport Authentication
  • Forms
  • Client Certificates

These methods are specific Microsoft implementations of some of the authentication design patterns found in cryptography. Most of these methods use public key encryption with weighty computational requirements. This is reason enough for architects to become familiar with them. Far-flung distributed processing is likely to have need of lighter weight authentication protocols such as Zero Knowledge Proofs. The best solution may be to develop your own application-defined authentication method.

The Nature of the Zero Knowledge Proof

When you hear the words "Open sesame", do you think of Ali Baba and the Forty Thieves? If not you definitely need to read up on your fairy tales! "Open sesame" was the secret phrase that would open a door in a rock that formed the entrance to the cave of the Forty Thieves. Inside the cave they kept the spoils of their thievery. By chance, Ali Baba overheard the secret phrase as the Captain of the thieves spoke it to gain entrance to the cave when they returned from one of their many heists.

I'll now take a little poetic license with the story for the sake of illustrating the Zero Knowledge Proof protocol. As my version of the story goes, Ali Baba used his knowledge of the secret phrase to enter the cave from time to time to supplement his meager income, for Ali Baba was a very poor man. Now as fate would have it, Ali Baba had a brother named Cassim, who noticed that Ali Baba was suddenly able to live beyond his means. Cassim, being a caring and moral man, was concerned that Ali Baba's recent lifestyle change was the result of some disreputable behavior on the part of Ali. When Cassim approached Ali Baba and challenged him about the source of his funds, Ali Baba told his brother about the cave of the Forty Thieves. Of course, he left out the actual secret phrase that opened the door. While Cassim's integrity was beyond reproach, Ali Baba's wasn't. Cassim was skeptical about Ali Baba's story, and threatened to turn him over to the authorities if he could not prove that his story was true. Ali Baba did not want to reveal the secret phrase that would open the door, but he also did not want to be handed over to the authorities. Ali Baba, being a practical man, sought the advice of his wife Morginia. She immediately recognized the applicability of a Zero Knowledge Proof to satisfy Cassim. She explained the protocol to Ali Baba, who promptly went back to Cassim to prove he was in possession of the secret phrase. The dialogue between the two brothers went like this:

Ali Baba: "You know that I am poor and do not have many possessions."

Cassim: "Yes."

Ali Baba: "There are many items in the cave and I am willing to bring an item from the cave and show it to you to prove that I know the secret password."

Cassim: "Though you are a man of few possessions showing me one item will not convince me."

Ali Baba: "I am willing to go to the cave and bring an item as many times as necessary until you are convinced."

Cassim agreed and dispatched Ali Baba to fetch an item. To his surprise, Ali returned with an item as ordered. Cassim, not easily convinced, dispatched Ali Baba forty-eight times to retrieve items until he was convinced that Ali Baba's story was true. Ali's wife had given him a Zero Knowledge Proof that made it possible to convince his brother not to hand him over to the authorities. Ali Baba proved to his brother's satisfaction that he indeed knew the secret phrase without having to reveal it to him.

This version of the Ali Baba fairy tale illustrates the nature of the Zero Knowledge Proof. First and foremost, a Zero Knowledge Proof is a proof of confidence, not a conclusive proof. Much of psychological testing is a proof of confidence. Suppose that Ali Baba also claimed to be psychic, and sought to prove it to Cassim. Cassim thinks of a number and asks Ali Baba whether the number is even or odd. Ali Baba has a one in two chance of answering correctly without having any psychic powers. The chance of Ali Baba answering correctly twice is one in four. Cassim does not believe that a one in four chance is conclusive. The chance of Ali Baba answering correctly (n) times is one in 2n. Cassim would believe Ali Baba to be psychic if he were able to correctly answer Cassim sixteen times (1 in 65,536). A trained psychologist may demand even more iterations of the protocol, but sooner or later would come to the same conclusion, as long as Ali Baba successfully answered. Secondly, a Zero Knowledge Proof conveys no knowledge in the process. This is best illustrated by the fact that in both proofs, Cassim knows no more after the proof than he knew before the proof. The proof provided a gradual accumulation of confidence, not an accumulation of knowledge. The third characteristic of a Zero Knowledge Proof is that it is not possible to convince a third party of the proof's validity. Even though Cassim was convinced that Ali Baba knew the secret phrase and was a psychic, he had no objective evidence from the proof with which to convince anyone else. The final characteristic of a Zero Knowledge Proof is that it is an iterative lightweight protocol.

Zero Knowledge Proof Characteristics

  • A Zero Knowledge Proof is a proof of confidence not a conclusive proof.
  • No knowledge is conveyed in a Zero Knowledge Proof.
  • It is not possible to convince a third party of a Zero Knowledge Proof 's validity.
  • A Zero Knowledge Proof is an iterative lightweight protocol.

A Computational Example of the Zero Knowledge Proof

The previous Ali Baba examples illustrate the nature of the Zero Knowledge Proof, but they do not lend themselves to a code implementation. To see how to implement the Zero Knowledge Proof in code, let's consider the situation where Ali Baba and his wife Morginia share a mutual secret (the secret phrase to open the cave door). As the years go by, Ali Baba is concerned that Morginia may have forgotten the secret, and wants to verify her knowledge in case he should die. Ali Baba remembers that the way he came to know the secret was by overhearing it. He doesn't want Morginia to reveal the secret for fear that it will be overheard again. So Ali Baba relies on the ingenuity of his wife to define another Zero Knowledge Proof protocol. She described the protocol to him like this:

Morginia: "Ali Baba, you choose a number at random and tell it to me."

Ali Baba: "Ok."

Morginia: "I'll take your number and combine it with the secret phrase. Then I'll input the combination to a one-way cryptographic hash function and tell you the result."

Ali Baba: "I get it, i should do the same. I'll combine my number with the secret phrase, as I know it. Then, as you did, I'll use the same one-way cryptographic hash function. Then I'll compare my result with the one that you told me."

Morginia: "That's right. If the results don't match, I've forgotten the secret."

Ali Baba: "Ok, but if they do match does that mean you know the secret?"

Morginia: "No, but you should have a higher confidence that I know the secret. You can repeat the process as many times as you wish until your confidence is your proof."

Ali Baba: "Alright, which one-way cryptographic hash function shall we use?"

Morginia: "Let's use the Secure Hash Algorithm (SHA). It produces a result that is 160 bits long. The chance of me producing a matching hash result with the wrong secret phrase is 1 in 280."

Ali Baba: "Awesome, I will be confident that you know the secret after just one iteration of the protocol."

Morginia Uses C# To Convince Ali Baba She Knows the Secret Phrase

To implement Morginia's Zero Knowledge Proof that she knows the secret phrase, we start by creating a new C# Windows Application Project with the name Ali_Baba_and_Morginia. Now we add an Ali_Baba class to the project with a GetChallenge() method.

Ali Baba's GetChallenge() method generates the random number that is returned as a challenge for Morginia. The random number is also saved in a private class field for use later when Morginia provides a response for verification. The purpose of the random number is to produce a different response from Morginia each time she is challenged by Ali Baba. This technique makes the protocol unpredictable to hackers, and more difficult to determine the secret or for generating a matching hash result. The larger the range of random numbers the better.

I recommend the use of a Globally Unique Identifier (GUID) as the random number. Its large size (128 bits) makes it excellent. I also like to convert it to a string so it is easy to display:

private string m_stringChallenge;
public
            string GetChallenge()
{
//
// I like to use a Globally
            Unique Identifier (GUID)
// as a random number but any random
            number is fine.
//
// It is also my preference to convert the
            GUID to
// a string for easy display.
//
// We save the
            random number to a private class field
// because we need it
            later to verify Morginia's
// response.
//
m_
            stringChallenge =Guid.NewGuid().ToString();
return m_
            stringChallenge;
}

Next instantiate an Ali_Baba object in the application form's constructor. There is nothing creative here:

private Ali_Baba m_ali_baba;
public
            Form1()
{
//
// Required for Windows Form Designer
            support
//
InitializeComponent();

//
// TODO: Add
            any constructor code after InitializeComponent call
//
//
            Instantiate an Ali_Baba object for the form.
//
m_ali_baba=new
            Ali_Baba();
}

Now add a button, textbox, and label to the form:

Use the button's Click event to provide a way to invoke Ali Baba's GetChallenge() method and display the result in the textbox:

private void buttonChallenge_Click(object
            sender, System.EventArgs e)
{
//
// Invoke Ali Baba's
            GetChallenge() method and display
// the
            result.
//
textboxChallenge.Text=m_ali_baba.GetChallenge();
}

Ok, test the application and verify that Ali Baba generates a new random number each time the button is clicked:

Now add a Morginia class to the project with a GetResponse() method. The Morginia class needs to use the System.Text and System.Security.Cryptography namespaces. The System.Text namespace provides ASCII encoding to convert a Unicode string to a byte array that the .NET class library Secure Hash Algorithm takes as input. The Secure Hash Algorithm classes are located in the System.Security.Cryptography namespace.

The .NET Framework Class Library supports implementations for the SHA1 and MD5 hash algorithms (don't be confused by the "1" tacked on to the SHA acronym! It represents a fix to the problem found in the original published algorithm shortly after it was released. No one ever uses the original publication). The SHA1 Class and the MD5 Class wrap the Microsoft CrytoAPI legacy Dlls. Those implementations are unmanaged code. The same namespace provides CLR managed implementations for the SHA1 algorithm (SHA1Managed Class) and three derivatives (SHA256Managed Class, SHA384Managed Class and SHA512Managed Class). The three derivatives provide larger hash results (256 bits, 384 bits, and 512 bits respectively) then the 160 bits of SHA1. All implementations are easy to use by simply creating an instance of the class and invoking its ComputeHash() method.

Remember the chance of producing a matching hash result with the wrong secret phrase using SHA1 is 1 in 280. Using larger hash results is overkill for this simple application, and just increases the computational weight of the protocol. So I recommend the use of the SHA1Managed Class.

The ComputeHash() takes a byte array as input, and the hash result is a byte array. So, we use the GetBytes() method of the ASCII encoding class to convert the concatenated challenge and secret phrase Unicode string to a byte array. The ToBase64String() of the Convert class, found in the System namespace, is used to convert the hash result byte array into a web compatible Base64 string:

using System.Text;
using
            System.Security.Cryptography;
//
// For the purpose of this
            simple demonstration I am
// leaving the "Secret Phrase" exposed
            but this is not
// suitable for any real authentication
            application.
//
private const string
            m_stringSecretPhrase="Open Sesame";
public string
            GetResponse(string a_stringChallenge)
{
//
// First merge
            the Challenge string with the Secret Phrase.
//
// Next
            convert the combined Unicode string to an ASCII
// encoded byte
            array.
//
// Then create an SHA1Managed object and invoke
            the
// ComputeHash method to get the Response.
//
//
            Finally convert the Response byte array to a Base64
// encoded
            string.
//
byte[]
            byteMerge=Encoding.ASCII.GetBytes(
a_stringChallenge+m_stringSecretPhrase);
SHA1Managed
            sha1managed=new SHA1Managed();
byte[]
            byteResponse=sha1managed.ComputeHash(byteMerge);
return
            Convert.ToBase64String(byteResponse);
}

Next, as was done with Ali_Baba, instantiate a Morginia object in the application form's constructor:

private Ali_Baba m_ali_baba;
private
            Morginia m_morginia;
public Form1()
{
//
// Required for
            Windows Form Designer
            support
//
InitializeComponent();

//
// TODO: Add
            any constructor code after InitializeComponent call
//
//
            Instantiate an Ali_Baba object for the form.
//
m_ali_baba=new
            Ali_Baba();
//
// Instantiate a Morginia object for the
            form.
//
m_morginia=new Morginia();
}

Now add another button, textbox, and label to the form to expose Morginia's response:

Use the button's Click event to provide a way to invoke Morginia's GetResponse() method and display the result:

private void buttonResponse_Click(object
            sender, System.EventArgs e)
{
//
// Invoke Morginia's
            GetResponse() method and display
// the
            result.
//
textboxResponse.Text=m_morginia.GetResponse(textboxChallenge.Text);
}

Now the application can be tested to display Morginia's Response to Ali Baba's challenge. Try it by clicking the "Challenge" button, and then the "Response" button:

Next, add a GetVerification() method to the Ali_Baba class. The class needs to use the System.Text and System.Security.Cryptography namespaces, as did Morginia. The same procedure used by Morginia to generate the response must be used by Ali_Baba to verify the result; therefore, the code is the same, and needs no additional explanation. Only the comparison of the two hash values and the return of the result need to be added:

using System.Text;
using
            System.Security.Cryptography;
//
// For the purpose of this
            simple demonstration I am
// leaving the "Secret Phrase" exposed
            but this is not
// suitable for any real authentication
            application.
//
private const string
            m_stringSecretPhrase="Open Sesame";
public string
            GetVerification(string a_stringResponse)
{
//
// First
            merge the saved Challenge string with the Secret Phrase.
//
//
            Next convert the combined Unicode string to an ASCII
// encoded
            byte array.
//
// Then create an SHA1Managed object and invoke
            the
// ComputeHash method to get the Response.
//
//
            Finally convert the Response byte array to a Base64
// encoded
            string and compare it to the one provided by
//
            Morginia.
//
byte[]
            byteMerge=Encoding.ASCII.GetBytes(
m_stringChallenge+m_stringSecretPhrase);
SHA1Managed
            sha1managed=new SHA1Managed();
byte[]
            byteResponse=sha1managed.ComputeHash(byteMerge);
if
            (a_stringResponse==Convert.ToBase64String(byteResponse))
return
            "Morginia Does Know";
else
return "Morginia Doesn't
            Know";
}

As done twice before, add another button, textbox, and label to the form to expose Ali Baba's verification:

Use the button's Click event to provide a way to invoke Ali Baba's GetVerification() method, and display the result in the textbox:

private void buttonVerification_Click(object
            sender, System.EventArgs e)
{
//
// Invoke Ali Baba's
            GetVerification() method and display
// the
            result.
//
textboxVerification.Text=m_ali_baba.GetVerification(textboxResponse.Text);
}

Now the entire Zero Knowledge Proof protocol can be tested. Try it by clicking the "Challenge" button, the "Response" button, and finally the "Verification" button:

Make sure that Ali Baba can detect an incorrect response from Morginia. Modify Morginia's response in the textbox and click the "Verification" button again. When I did it, I simply changed the lowercase "d" to an uppercase "D". Ali Baba caught the difference!

As one final test, Morginia's secret phrase should be modified to reflect the situation where she did forget it. Change the phrase to "Open I say", and verify that Ali Baba can detect an incorrect response.

private const string m_stringSecretPhrase="Open I say";

The Zero Knowledge Proof again provides Ali Baba with clear evidence that Morginia has forgotten the secret phrase.

Conclusions

The proliferation of far-flung distributed processing calls for enterprise architects to be familiar with the various methods of authentication. Zero Knowledge Proofs that use a cryptographic hash are lightweight alternatives to authentication methods that use public key cryptography. The .NET framework makes an implementation of a zero knowledge protocol easy.

If you'd like to discuss a broader range of C#Today topics and issues, or if you have any C# related questions you need answers to, please visit our online resource centres at : http://p2p.wrox.com/list.asp?list=csharptoday_discuss&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq and http://p2p.wrox.com/list.asp?list=c_sharp&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq
C#Today - © 2002 WroxPress Limited


C# Browser History Monitoring Application (Part 1)

C# Browser History Monitoring Application (Part 1)
http://www.csharptoday.com/content.asp?id=1714
Article Rating: 4
Published on December 6, 2001
Abstract
The application developed by Jan Narkiewicz in this article monitors the browser history file of a currently logged in user. As the user surfs the web, the contents of their browser history folder are mirrored to a secure location. Inside the mirrored location (only accessible to administrators on the machine) all additions and updates to the user’s browser history folder are recorded. Deletions are not recorded. The idea is to monitor a user’s web surfing habits without letting them know their surfing habits are being monitored.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

The windows service application developed in this article (SWPWebWatcherService.exe) monitors the browser history file of a user currently logged in to the machine the service is running on. As the user surfs the web, the contents of their browser history folder are mirrored to a secure location. Inside the mirrored location (only accessible to administrators on the machine) all additions and updates to the user's browser history folder are recorded. Deletions are not recorded. The idea is to monitor a user's web surfing habits without letting them know their surfing habits are being monitored.

This application should only be used in situations where the legal and ethical consequences are taken into account. For example, you may want to monitor someone using your computer whom you suspect is accessing illegal websites.

The SWPWebWatcherService.exe application runs as a windows service. Services are visible using Task Manager (CTRL-ALT-DELETE and click the Task Manager button) but are not visible as programs running on the user's desktop. More information on services can be found at http://www.csharptoday.com/content.asp?id=1538&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq.

The .NET technology used to implement the SWPWebWatcherService.exe application includes classes from the following namespaces:

  • System.ServiceProcess - this namespace exposes classes that support the development of services. These classes include ServiceBase that exposes the ability to start, stop, pause and continue a services' execution. The installation/uninstall of services is facilitated using the classes, ServiceInstaller and ServiceProcessInstaller.
  • The ServiceProcessInstaller class supports the installation of a process the runs one or more services. A ServiceInstaller class is required per-service running within a process as this class configures the behavior of an individual service.
  • System.IO - this namespace provides classes for reading/writing files and data streams both synchronously and asynchronously. The classes exposed also allow the contents of a directory to be monitored (using the FileSystemWatcher) and for the contents of directories to be determined (using DirectoryInfo). The File class exposes methods that can move and copy files (the contents of our Browser History Folder). This namespace's Path class provides methods that manipulate file paths including the handing of the directory portion of the path, the root, the file name, and the extension.
  • Microsoft.Win32 - this namespace exposes classes that handle operating system events and that can manipulate the registry. Information such as where a user's browser history folder resides is read from the registry using the RegistryKey class.

This SWPWebWatcherService.exe application runs as a service and therefore will only run on Windows NT, Windows 2000 or Windows XP. All modules/assemblies developed for this application are prefixed by SWP (shorthand for Software Pronto, Inc. - this prefix is to used ensure that no code developed here is confused with .NET namespaces, classes, methods or properties).

As part of the Browser History Monitor application, a Winlogon Notification Package called SWPWinLoginPkg.dll is developed and installed. This Win32 DLL (a Windows DLL and not a .NET assembly) exposes a set of functions that receive logon notifications from the security subsystem. It is actually the security subsystem that is responsible for loading and unloading this DLL when the system starts up and shuts down. This DLL is not a .NET application but instead is implemented as an unmanaged C++ application. We use C++ to write the Winlogon Notification Package DLL due to its support for unmanaged code.

The basic architecture for the Browser History Monitoring application is as follows:

Under Windows 2000, services are administered using the services snap-in that runs under Microsoft Management Console (MMC). This can be launched by accessing Control Panel | Administrative Tools | Services. Once the SWPWebWatcherService service is fully installed the Services tool appears as follows:

This article is divided into two parts. The first part here shows how to implement the service that monitors a user's history folder. The second part (tomorrow) shows how to implement a Windows Login Notification DLL.

Developing the Service

In Visual Studio 6.0, it took a bit of development effort to create a service. Nowadays, Visual Studio .NET handles most of the service-related drudgery for us. Selecting File | New | Project | Windows Service creates a ready-made service application. Call the service SWPWebWatcherSerivce:

Rename Server1.cs as SWPWebWatcher.cs, and rename the class name Service1 as SWPWebWatcher.

Creating an Installer and Installing the Service

The installer for the SWPWebWatcherService service is created by selecting Visual Studio .NET's Design view. Right clicking on the Design view displays the following context menu:

Select the Add Installer menu item and we get a service installer:

To keep the names consistent, the file ProjectInstaller.cs is changed to SWPProjectInstaller.cs and the ProjectInstaller class implemented inside this source file is renamed SWPProjectInstaller.

More important than name consistency is that the SWPProjectInstaller class comes with two data members of type, ServiceProcessInstaller and ServiceInstaller:

private System.ServiceProcess.ServiceProcessInstaller serviceProcessInstaller1;
private System.ServiceProcess.ServiceInstaller serviceInstaller1;

The ServiceProcessInstaller class writes specific registry entries for each service running in the same application. These registry settings specify the account type (user, local server, local administrator, etc), username and password for the service. The ServiceProcessInstaller class is derived from the base class, Installer, and hence ServiceProcessInstaller works with .NET installation utilities such as InstallUtil.exe.

The ServiceInstaller class writes specific registry entries for our service. These registry settings specify how a service starts (manually, automatically or disabled) and what other services the installed service is dependant on. The ServiceInstaller class is also derived from Installer.

The registry keys setup by ServiceProcessInstaller and ServiceInstaller are found under:

HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\servicename

where servicename is the name of the service install, SWPWebWatcher.

Displaying the properties for serviceInstaller1 allows us to configure the service:

Change the DisplayName property to SWP Web Watcher, and the ServiceName to SWPWebWatcher. The SWPWebWatcherService service is started by a Winlogon notification DLL when certain users log on, so leave the StartType as Manual. The code created by Visual Studio .NET to set these properties for the serviceProcessInstall1 instance is found in the SWPProjectInstaller class's InitializeComponents method:

this.serviceInstaller1.DisplayName = "SWP Web Watcher";
this.serviceInstaller1.ServiceName = "SWPWebWatcher";

The ServicesDependedOn property is not set because our SWPWebWatcherService does not depend on another service.

The previous properties set programmatically by ServiceInstaller can also be set manually (once the service is installed) using the Services administrative tool.

The properties for the ServiceProcessInstaller instance, serviceProcessInstaller1, can also be configured in Visual Studio .NET. The properties for this data member are as follows:

The Account property in the screenshot above is set to User. The Username and Password properties correspond to the username (.\Administrator, the local administrative account) and password (1234) of the user account used by the service. The Account property could also have been set to a variety of different other accounts (LocalService, LocalSystem, and NetworkService). The LocalSystem (local system account) setting on most machines is equivalent to specifying a user account of username, .\Administrator. The code created by Visual Studio .NET to set these properties for the serviceProcessInstall1 instance is found in the SWPProjectInstaller class's InitializeComponents method:

this.serviceProcessInstaller1.Password = "1234";
this.serviceProcessInstaller1.Username = ".\Administrator";

In the properties window, settings for the Username and Password could have been omitted. In this case, the installer utility (installutil.exe) would query for username and password as part of installation. installutil.exe will only execute when the appropriate paths and environmental variables are set up. These settings can be setup for the console window by executing either the vsvars32.bat batch file or the vcvars32.bat that ship with Visual Studio .NET. An alternative to creating a console window that is property configured is to launch a console window via the Start Menu | Programs | Microsoft Visual Studio.NET 7.0 | Visual Studio.NET Tools | Visual Studio.NET Command Prompt.

Querying the Registry

Before the browser history folder can be monitored for a particular user, it must be determined which user is logged in and it must be determined where that user's history folder resides. To find this information, the registry is queried.

Before the registry can be queried, it simplifies coding to place a new using directive at the top of the SWPWebWatcher.cs source file in which the registry will be manipulated.

using.Microsoft.Win32;

Contained in this namespace are the Registry and RegistryKey classes. The Registry class contains fields corresponding to specific root level keys in the registry. These fields include the CurrentConfig field that accesses HKEY_CURRENT_CONFIG, the CurrentUser field that accesses HKEY_CURRENT_USER, etc. The RegistryKey class provides functionality to create, delete, modify and query keys and sub-keys within the registry. The namespace Microsoft.Win32 is so named because, although the registry can be accessed via .NET, the registry is not part of .NET. The basic distinction is that (in theory) .NET is platform independent, but clearly the registry in platform dependant.

The keys retrieved within the registry are found under HKCU (HKEY_CURRENT_USER) or in .NET terms, Registry.CurrentUser. The path used to retrieve the keys and the code used to open this key are as follows:

string subKeyStrExplorer = @"Software\Microsoft\Windows\CurrentVersion\Explorer";
RegistryKey subKeyExplorer = Registry.CurrentUser.OpenSubKey(subKeyStrExplorer);

Basically when Windows Explorer (the shell) is run for a particular user, there are settings specific to that user. These settings include the user's username and the location of the user's browser history directory. The specify entry under the previous path that holds the user name is, Logon User Name. The specific value for this key can be retrieved as follows using the previously opened registry key instance, subKeyExplorer:

string keyLogonUserName = "Logon User Name";
string userName = subKeyExplorer.GetValue(keyLogonUserName).ToString();

The Shell Folders key resides under .\CurrentVersion\Explorer. This sub-key contains an entry, History. This entry contains the directory in which the user's browser history is stored. The browser history folder for a user is retrieved as follows:

RegistryKey subKeyShellFldrs = subKeyExplorer.OpenSubKey("Shell Folders");
string historyFolder = (subKeyShellFldrs.GetValue("History")).ToString();

All the previously presented code is implemented in the method SWPLoggedOnGetUserInfo. This method is as follows:

static private void SWPLoggedOnGetUserInfo(out string historyFolder, out string userName)
{
// Get the value for directory to watch from system registry
RegistryKey hkcu = Registry.CurrentUser;
string subKeyStrExplorer =
@"Software\Microsoft\Windows\CurrentVersion\Explorer";
RegistryKey subKeyExplorer = hkcu.OpenSubKey(subKeyStrExplorer);
string keyLogonUserName = "Logon User Name";

if (subKeyExplorer.GetValue(keyLogonUserName) != null)
{
userName = subKeyExplorer.GetValue(keyLogonUserName).ToString();
}
else
{
userName = "DefaultUser";
}

RegistryKey subKeyShellFldrs =
subKeyExplorer.OpenSubKey("Shell Folders");

historyFolder = (subKeyShellFldrs.GetValue("History")).ToString();
subKeyShellFldrs.Close();
subKeyExplorer.Close();
}

The history folder retrieved for the user by SWPLoggedOnGetUserInfo on a Windows 2000 machine typically takes the form of:

C:\Documents and Settings\AnyUserName\Local Settings\History

System.IO.Path

Before the history file is monitored, we need to discuss the Path class. It simplifies coding to place a new using directive at the top of the SWPWebWatcher.cs source file in which the history file is monitored.

using.System.IO;

There is no constructor for the Path class, meaning that this class can never be instantiated. Instead the Path class provides a set of methods that manipulates file paths. For legacy Win32 developers, .NET's Path class provides much of the same functionality as provided by ShlWapi.dll. The Path class's methods used in the SWPWebWatcherService application include:

  • string Path.GetDirectoryName(string path) - given a path (parameter, path) retrieves and returns the directory portion of the path. This method raises the ArgumentException if the path is malformed.
  • string Path.GetFileName(string path) - given a path (parameter, path) retrieves and returns the file name portion of the path. This method raises the ArgumentException if the path is malformed.
  • string Path.Combine(string upperPath, string lowerPath) - concatenates two paths (upperPath and lowerPath) and returns the result. If needed a path separator (Path.PathSeparator) is placed between the concatenated paths. This method raises the ArgumentException if the either path passed as a parameter is malformed.

System.IO.Directory

Within the monitoring application, the Directory class's CreateDirectory method is used extensively to create directories. The prototype for this method is as follows:

public static DirectoryInfo CreateDirectory(string path);

Certain behaviors of the CreateDirectory class are exploited by the application, specifically:

  • If the path already exists, CreateDirectory succeeds and takes no action. This means that it is not necessary to test if a directory exists before creating it. For example on most Windows NT and Windows 2000 installations the following would be a noop for CreateDirectory because c:\WinNT already exists:

Directory.CreateDirectory(@"c:\WinNT");

  • If multiple directories in the path do not exist, CreateDirectory, creates each directory. To understand this concept, consider the following path, c:\exists\notexist1\notexist2. The directory c:\exists exists but c:\exists\notexist1 does not exist and consequently c:\exists\notexist1\notexist2 does not exist. Calling CreateDirectory as follows will create both c:\exists\notexist1 and c:\exists\notexist1\notexist2:

Directory.CreateDirectory(@"c:\exists\notexist1\notexist2");

Monitoring the History File

The SWPWebWatcher class's data members that are involved with monitoring the history file are as follows:

private FileSystemWatcher historyWatcher = null;
private string destDirName = null;

The historyWatcher is the FileSystemWatcher instance that actually monitors the browser history folder. The crucial property of the FileSystemWatcher is the Path (type, string) which is the fully qualified path being monitored (the user's browser history file). The destDirName is the location in which the mirror of the monitor user's browser history folder is copied. The method responsible for setting up the aforementioned data members is SWPStartupMonitors.

The first bit of code in SWPStartupMonitors ensures that the destination directory data member, destDirName, is set and that this directory exists:

private void SWPStartupMonitors(string historyFolder, string userName)
{
destDirName = Path.Combine(@"c:\MockHistory", userName);
Directory.CreateDirectory(destDirName);
// rest of SWPStartupMonitors to be implemented here (see below)
}

The remainder of the SWPStartupMonitors method sets up the FileSystemWatcher historyWatcher data member in order to monitor the user's browser history folder. The first step in this process (after this class is created) is to assign a directory path to the FileSytemWatcher's Path property (historyWatcher.Path = historyFolder). After this, the types of modifications to the history folder that generate notifications must be specified. The flavors of notification that will generate events are specified using the FileSystemWatcher 's NotifyFilter property. This property is set to the bitwise OR'ing of NotifyFilters.DirectoryName (modified directory name generates event), NotifyFilters.FileName (modified file name generates event), and NotifyFilters.LastWrite (change to directory and/or file's last write time generates event). The filter methods must be associated with each event's delegate (e.g. historyWatcher.Changed += new FileSystemEventHandler(this.historyWatcher_Changed);). Setting the IncludeSubdirectories property to true causes notifications to also be generated for subdirectories and setting EnableRaisingEvents to true ultimately enables the raising of notification when the specified changes are made to the user's browser history file. The code from the SWPStartupMonitors method that sets up the FileSystemWatcher instance is as follows:

historyWatcher = new FileSystemWatcher();
historyWatcher.Path = historyFolder;
historyWatcher.NotifyFilter = NotifyFilters.DirectoryName |
NotifyFilters.FileName |
NotifyFilters.LastWrite;
historyWatcher.Changed += new FileSystemEventHandler(this.historyWatcher_Changed);
historyWatcher.Renamed += new RenamedEventHandler(this.historyWatcher_Renamed);
historyWatcher.Created += new FileSystemEventHandler(this.historyWatcher_Created);
historyWatcher.IncludeSubdirectories = true;
// If we don't do the following, we get no message notifications
historyWatcher.EnableRaisingEvents = true;

From the previous code snippet it should be clear that the methods used to handle the notifications raised by the historyWatcher FileSystemMonitor instance are historyWatcher_Changed, historyWatcher_Renamed, and historyWatcher_Created.

The SWPStartupMonitors method in its entirety and data members of note are as follows:

private FileSystemWatcher historyWatcher = null;

private string destDirName = null;

private void SWPStartupMonitors(string historyFolder, string userName)
{
destDirName = Path.Combine(@"c:\MockHistory", userName);
Directory.CreateDirectory(destDirName);

historyWatcher = new FileSystemWatcher();
historyWatcher.Path = historyFolder;
historyWatcher.NotifyFilter = NotifyFilters.DirectoryName |
NotifyFilters.FileName |
NotifyFilters.LastWrite;
historyWatcher.Changed += new FileSystemEventHandler(this.historyWatcher_Changed);
historyWatcher.Renamed += new RenamedEventHandler(this.historyWatcher_Renamed);
historyWatcher.Created += new FileSystemEventHandler(this.historyWatcher_Created);
historyWatcher.IncludeSubdirectories = true;
// If we don't do the following, we get no message notifications
historyWatcher.EnableRaisingEvents = true;
}

FileSystemWatcher Notification Handlers

The methods historyWatcher_Changed, historyWatcher_Renamed, and historyWatcher_Created are called when specific notifications are raised by the FileSystemWatcher instance, historyWatcher. The notifications raised correspond to changes made to the user's browser history folder, it sub-folders and the files contained in these folders.

When a file or directory path changes, the notification is handled by the historyWatcher_Changed method. This method basically takes SourcePath\NewStuff and ensures that it exists under Destpath\NewStuff. If the source that was changed was a file then 1) the directory fully qualifying the file in the destination is created (Directory.CreateDirectory) and 2) the file is copied from source to destination (File.Copy). If the source that was changed was a directory then this method simply makes sure the directory exists in the destination (Directory.CreateDirectory).

The event-specific parameter associated with the notification is of type FileSystemEventArgs. This class contains a pertinent property, Path. The Path property corresponds to the file/directory that was changed. The code implementing the historyWatcher_Changed method is as follows:

private void historyWatcher_Changed(object sender, FileSystemEventArgs e)
{
// this returns TheRest based on SourceDir\TheRest
string theRest = e.FullPath.Substring(historyWatcher.Path.Length + 1);
string changedDest = Path.Combine(destDirName, theRest);

if (File.Exists(e.FullPath)) // Is it changed file?
{
Directory.CreateDirectory(Path.GetDirectoryName(changedDest));
File.Copy(e.FullPath,
changedDest,
true);
}

else // changed directory -- noop but do make sure it exists
{
Directory.CreateDirectory(changedDest);
}
}

When a file or directory is created the behavior is identical to the case where a file or directory is changed. For this reason the method that handles the creation event, historyWatcher_Created, simply calls historyWatcher_Changed. The code implementing historyWatcher_Created is as follows:

private void historyWatcher_Created(object sender, FileSystemEventArgs e)
{
// For now Created and Changed behave the same. We will keep the
// historyWatcher_Created method just in case the Created
// and Changed cases are no longer handled in the same manner.
historyWatcher_Changed(sender, e);
}

For those paying close attention, the historyWatcher_Created method is called at midnight when the history file corresponding to the new day is created.

When a file or directory is renamed, the notification is handled by the historyWatcher_Rename method. The event-specific parameter associated with the notification is of type FileSystemEventArgs. The Path property of this class has already been discussed by for the case of the rename event; Path corresponds to the newly named file/directory that was changed. The original name of the file/directory is specified using the FileSystemEventArgs class's OldFullPath property.

Renaming is a bit more difficult than creating or changing because the file/directory to rename may not have been mirrored yet. If this is the case, then the file is copied or the directory name is created. If the original file/directory already exists then the file is renamed (File.Move) or the directory is renamed (Directory.Move). The basic shell of the historyWatcher_Rename method behaves as just described:

if File.Exists(Source Path)
if (File.Exists(Dest Path)) // existing file so rename
File.Move
else // create since file does not exist
File.Copy
else // it is a directory
if (Directory.Exists(Dest Path))
Directory.Move
else // create since directory does not exist
Directory.CreateDirectory

The code implementing the historyWatcher_Rename method is as follows:

private void historyWatcher_Renamed(object sender, RenamedEventArgs e)
{
// this returns TheRest based on SourceDir\TheRest
string newSubPath =
e.FullPath.Substring(historyWatcher.Path.Length + 1);
string oldSubPath =
e.OldFullPath.Substring(historyWatcher.Path.Length + 1);
string oldDest = Path.Combine(destDirName, oldSubPath);
string newDest = Path.Combine(destDirName, newSubPath);

// Is it a renamed file?
if (File.Exists(e.FullPath))
{
if (File.Exists(oldDest)) // existing file so rename
{
File.Move(Path.Combine(destDirName, oldSubPath),
Path.Combine(destDirName, newSubPath));
}
else // file not mirrored -- yet
{
// Make sure file path exists
Directory.CreateDirectory(Path.GetDirectoryName(oldDest));
// file has yet to be mirrored so rather than a rename it,
// copy it
File.Copy(e.FullPath,
Path.Combine(destDirName, newSubPath),
false);
}
}

else // renamed directory
{
// dest was already mirrored so it is a directory rename
if (Directory.Exists(oldDest))
{
Directory.Move(oldDest, newDest);
}

else // dest not mirrored yet so create from scratch
{
Directory.CreateDirectory(newDest);
}
}
}

Starting and Stopping Monitoring

SWPWebWatcherService is provided (thanks to Visual Studio .NET's automatically generated code) with an OnStart method to start the service and an OnStop method to stop it. The code to initiate monitoring the history file should obviously be placed in OnStart, and the code to cease the monitoring of the history file should be placed in OnStop.

The OnStart method makes use of the following previously discussed methods:

  • SWPLoggedOnGetUserInfo - retrieves the history file and user name for the currently logged in interactive user.
  • SWPStartupMonitors - initiates the monitoring of the history directory folder for the currently logged in interactive user.

The only other code in OnStart, besides the two previously mentioned methods, is a switch statement. This switch statement examines the user name of the currently logged in interactive user and only enables history folder monitoring for specific users. The bShouldMonitor data member of SWPWebWatcher class is set to true when monitoring is being performed for a user.

bool bShouldMonitor = false;

protected override void OnStart(string[] args)
{
string userName;
string historyFolder;

SWPLoggedOnGetUserInfo(out historyFolder, out userName);
switch (userName)
{
case "UserNameForSpiedOnUser":
case "MyLittleBrother": //
bShouldMonitor = true;
break;
default:
}

if (bShouldMonitor)
{
SWPStartupMonitors(historyFolder, userName);
}
}

The bShouldMonitor data member may appear to only be used in the context of the OnStart method. Actually the bShouldMonitor data member is used as part of a backdoor that allows the SWPWebWatcherService application to be run by the user as just a process rather then a process acting as a service. Running as "just a process" simplifies the debugging of the service.

For those new to services, recognize that if no user is logged into a machine the services can still be running away merrily in the background. This is because these services are run in a region of windows that is still active when no user is logged in. This area of Windows is a Windows station specifically reserved for services. When a user logs on, an interactive Windows station is created and all user applications run (for the most part) in the interactive desktop created by the user's Windows station. The interactive Windows station is the area of windows where the interactive user's applications run. For example Outlook, Word, Visual Studio .NET typically run on the interactive desktop since they are being run by an interactive user. When the user logs out, the interactive desktop and interactive Windows station is cleaned up and all the user's applications cease to run (Word, Outlook, and Visual Studio .NET exit). We allow SWPWebWatcherService to run as in the interactive desktop is to simplify debugging. By providing a single command-line parameter, SWPWebWatcherService will not run as a service and therefore can be simply debugged by Visual Studio .NET. If SWPWebWatcherService was running a service it could still be debugged but Visual Studio .NET would have to attach to the services. One mechanism that can be used to attach to a service for purposes of debugging is through the Debug | Processes menu item.

The Main method of the SWPWebWatcherService checks the number of command-line arguments, and if at least one argument is provided simply runs this application interactively (i.e. if (args.Length > 0)). The code inside the modified Main method that handles this is as follows:

static void Main(string[] args)
{
// providing a command-line arg gives us a back door so we can debug
if (args.Length > 0)
{
SWPWebWatcher webWatcher = new SWPWebWatcher();

webWatcher.bShouldMonitor = true; // monitor regardless of user
webWatcher.OnStart(args);
MessageBox.Show("Close MessageBox to continue");
return ; // terminate here otherwise we'd run as a services
}

// run SWPWebWatcher as a service here
// The code below was generatd by Visual Studio.NET
System.ServiceProcess.ServiceBase[] ServicesToRun;

ServicesToRun = new System.ServiceProcess.ServiceBase[]
{ new SWPWebWatcher() };
System.ServiceProcess.ServiceBase.Run(ServicesToRun);
}

The previous code uses the MessageBox class. This class was exposed adding a reference to the System.Windows.Forms.dll assembly and by placing a user System.Windows.Forms at the top of the source file in which the Main method resides.

The OnStop method is called when the service control manager stops the SWPWebWatcherService service. The primary duties of this method are to cleanup the FileSystemWatcher instance (instance, historyWatcher) used to monitor the browser history folder and to clean up the DirectoryInfo data member that specifies the destination directory in which the history is mirrored (instance, copiedDir). In keeping with these primary duties, the OnStop method is as follows:

protected override void
            OnStop()
{
historyWatcher.Dispose();
historyWatcher =
            null;
}

Installing the Service

To install the executable, SWPWebWatcherService.exe, as a service using installutil.exe, the following command-line is executed from the command prompt.

D:\SWPWebWatcherService\bin\Debug>installutil SWPWebWatcherService.exe

The file extension (*.exe) is required in order for the previous command to execute properly. Once the previous command-line has been successfully executed, the service is installed and can be managed using the service administrative snap-in.

To uninstall the service, run the installutil.exe utility with the /U option (uninstall option). Uninstalling the service is performed as follows from the command prompt:

D:\SWPWebWatcherService\bin\Debug>installutil /U SWPWebWatcherService.exe

Further Work

The Browser History Monitor is not designed to run all the time or even for every user on a machine. For this reason the service that performs the monitoring is only started when a user logs on and is stopped when a user logs off. This starting and stopping could be easily extended to start and stop the service only for certain users. The Windows security API exposes a style of DLL that can detect and take action on user logon or logoff. Such DLL's are referred to as Winlogon Notification Packages. MSDN provides a detailed write up on Winlogon Notification Packages under: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/security/hh/logauth/notify_9r51.asp&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq.

Conclusion

Thus far the SWPWebWatcherService application does a good job as a standalone application. The second part of this article demonstrates the development of a WinLogon Notification DLL that will be used to start the SWPWebWatcherService service when a user logon notification is received, and stop this service when a logout notification is received. This DLL not only demonstrates how to develop a WinLogon Notification DLL, but also shows how to develop with Visual Studio .NET and unmanaged code. This DLL must be developed as an unmanaged application, so C++ in conjunction with Visual Studio .NET was selected for development. The second part of this article shows some practical circumstances under which the managed C# world must interact with unmanaged code.

The second part will also show how to use and view the results generated by the browser monitoring application.

C#Today - © 2002 WroxPress Limited


Custom Security Implementation
http://www.csharptoday.com/content.asp?id=1745
Article Rating: 5
Published on November 15, 2001
Abstract
In the previous articles in this series, we have discussed the security architectures and schemes available to developers in the .NET framework. Code access security and role based security are the two main types of security available to developers. In this article, Kaushal Sanghavi explores .NET’s support for building a custom security scheme by implementing our own identity and principal classes that rely on user information stored in a database. In the previous article on Custom Security Permissions, we focused on Code Access Security, while in this article we focus on extending .NET’s support for Role Based Security.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

In the previous articles in this series, we have discussed the security architectures and schemes available to developers in the .NET framework. Code access security and role based security are the two main types of security available to developers. Code access security deals with assigning permissions to code, which allows it to access protected resources like disk files or the registry, or carry out certain operations like accessing unmanaged code. Role based security deals with identifying the user who is attempting to execute the code, and allows or disallows certain operations based on the privileges that the user possesses. Code access security is built around the notion of permission objects, which control who can access a protected resource, and what level of access is granted. These permissions are usually based on a permission policy that is specified by administrators on a given machine or domain. Role based security, on the other hand, also deals with permissions, but to a lesser extent. The main concept behind role based security is that of identity. Role based security makes decisions based on the identity of the user who is attempting to execute code. Identities are closely tied to principals, which contain information about the roles a user belongs to. The most common type of identity and principal objects, that have been used since the early days of Windows, have been the domain users and the built in Windows user groups like Administrators, Power Users, etc. These groups, or accounts, were used to grant or deny access to files, network shares, and even databases, that were hosted on the same, or a trusted, domain. The Windows domain controller was the master server for all security requests, and would authenticate and authorize access. While this scheme worked well, it was too limited and closely tied to the Windows user accounts. A lot of enterprise applications stored user accounts in databases and even on different systems like Unix. The .NET framework allows an extensible architecture that lets developers build their own principals and identities, and use them with .NET role based security. In this article, we will explore .NET's support for building a custom security scheme by implementing our own identity and principal classes that rely on user information stored in a database.

In our previous article on Custom Security Permissions (http://www.csharptoday.com/content.asp?id=1668&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq), we defined custom permissions that can be used to augment existing permissions. That article focused on Code Access Security, while this article focuses on extending .NET's support for Role Based Security. Please refer to earlier articles in this series for a complete discussion of code access security and role based security.

Note that this article is based on Visual Studio .NET Beta 2, SQL Server 2000 and Windows 2000 (all versions).

Principals and Identities

Role based security is based on Identities and Principals. An identity represents an authenticated user and a principal represents the security context of this user, including that user's identity and the roles that the user belongs to. Every thread executes under a specific principal, and code that relies on role based security uses this principal to check for user authorization. Thus, if a component allows access only to users belonging to a specific role, every access to the component would result in the component looking up the current thread's principal object and determining if the specific role was found.

The System.Security.Principal namespace declares two interfaces, IIdentity and IPrincipal. These interfaces are implemented by classes that provide a role based security scheme for authorization. Currently, the .NET framework provides two implementations of these interfaces.

  • WindowsIdentity, WindowsPrincipal : These classes are used for authorizing users based on their Windows domain accounts.
  • GenericIdentity : These classes are used for authorizing users based on user names that are not related to Windows domain accounts.

The IIdentity interface declares the following properties:

  • AuthenticationType: This property returns a string that describes the type of authentication that is in place. The value for this property will be defined by each individual application.
  • IsAuthenticated: This property returns a value that indicates whether the current user has been authenticated or not.
  • Name: This property returns the user name of the user represented by this identity.

The IPrincipal interface declares the following properties and methods:

  • Identity: This property returns the identity object that is associated with this principal.
  • IsInRole: This method returns a value that indicates whether the user represented by this principal belongs to a specific role.

The default GenericPrincipal and WindowsPrincipal classes provided by the .NET framework are sufficient to handle most security schemes. However, when these default implementations do not satisfy your application's security requirements, it is possible to implement classes that implement IIdentity and IPrincipal, and use these for authentication and authorization.

Now that we have discussed the basics of identities and principals, let us discuss a custom security implementation by defining our own identity and principal classes.

Custom Security Implementation

In this article, we will focus on building a database backed security scheme, where user credentials are stored in a user table in a database, and another table stores the roles that a given user belongs to. We will use Microsoft SQL Server 2000 as our database, but this could easily be expanded to any other database on any platform.

The following screen shot shows the table schema. The UserTable stores user ids and passwords for authenticated users, the Roles table is a lookup table for all possible roles for the application, and the UserRoles table stores the roles that a user belongs to.

The following three screen shots show some sample data that we will be using for the rest of this article. We have four users and three roles in our system. Note that in this sample we are storing clear text passwords in the user table - in a production environment, passwords should be encrypted for obvious reasons!

A high-level architecture diagram for our custom security implementation is shown below. We have a login class called LoginServer that will be used to authenticate users against the user table from the above screen shots. Once a user is authenticated, the LoginServer class creates an identity object and a principal object for the authenticated user. The login server reads the UserRoles table and passes in an array of roles associated with the authenticated user to the principal object. This list of roles is referred to when a component calls the IsInRole method of the principal object.

Let us start by looking at the LoginServer class first. This class implements a single public method, Login, which takes in a user name and password and authenticates the user. If the credentials are valid, the roles associated with the user are read from the database, and the custom identity and principal objects are created. The existing principal object (which could be any principal associated with the thread) is then replaced by the custom principal object.

The Login method is shown below:

public bool Login (string iUserId, string
            iPassword)
{
try{
// establish a
            connection
SqlConnection lConn = new SqlConnection
("server=localhost;uid=sa;pwd=sa;database=CSharpSecurity;");

//
            validate user credentials passed by a login client
string lSqlStr
            = "select * from UserTable where UserId = '" +
iUserId + "' and
            Password = '" + iPassword + "'";

SqlCommand lCommand = new
            SqlCommand (lSqlStr, lConn);
lConn.Open ();

// if user is
            invalid, don't set a principal
SqlDataReader lReader =
            lCommand.ExecuteReader ();
if (!lReader.Read
            ()){
Thread.CurrentPrincipal = null;
return false;
}
lReader.Close ();

// get roles for a valid
            user
lSqlStr = "select Role from UserRoles where UserId = '" +
iUserId + "'";
lCommand.CommandText = lSqlStr;
lReader =
            lCommand.ExecuteReader ();

ArrayList lRolesArr = new
            ArrayList ();
while (lReader.Read ())
lRolesArr.Add
            (lReader.GetString (0));

string[] lRoles =
            (string[])lRolesArr.ToArray (typeof(string));
lConn.Close
            ();

// add the roles to the custom
            principal
CreateIdentityPrincipal (iUserId, lRoles);
return
            true;
}
catch (Exception lExcep){
MessageBox.Show
            (lExcep.Message);
return false;
}
}

Since we are using SQL Server as our database, we use the ADO .NET classes from the System.Data.SqlClient namespace to access the database tables (for a detailed discussion on ADO .NET and the SqlClient namespace, please refer to http://www.asptoday.com/content/articles/20000925.asp?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq). We start by creating an SqlConnection object that contains the connection string to connect to the database. Our sample database is called CSharpSecurity, and we use the infamous sa - sa credentials to access the database. Readers should change this connection string to reflect their machine settings. Once a connection is established to the database, we look for the user id - password combination that was passed to this method in the UserTable. If a match is found, then the user is authenticated, and we use the user name to retrieve all the associated roles from the UserRoles table. We use the SqlDataReader object to iterate through the returned rows, using an ArrayList object from the System.Collections namespace to store these values. These are then converted into a string array that will be passed to the custom principal object. The CreateIdentityPrincipal method is a private method that creates the custom identity and principal objects and attaches it to the current thread.

The CreateIdentityPrincipal is shown below:

private void CreateIdentityPrincipal (string iUserName, string[] iRoles)
{
CustomIdentity lIdentity = new CustomIdentity (iUserName, true);
CustomPrincipal lPrincipal = new CustomPrincipal (lIdentity, iRoles);

Thread.CurrentPrincipal = lPrincipal;
}

Identity objects need a user name and a flag indicating whether the user is authenticated or not, and principal objects need the underlying identity object and a list of roles that the user belongs to. This data is passed by the above method and the custom identity and principal objects are created. Once the custom principal object is created, we attach it to the currently executing thread, by calling the static CurrentPrincipal property on the System.Threading.Thread class. This principal will now be associated with all the calls that are invoked on this particular thread. This is a very important point. Once a principal is associated with a thread, it stays attached to the thread until another principal object replaces it, or the thread terminates. Thus, when an authenticated user logs off the system, or the thread is assigned to a different user, it is very important to update the thread's principal object. For example, consider the scenario where a thread is associated with user "abc", and another user "xyz" attempts to log on to the system on the same thread. If the principal is not updated for the second login, the user "xyz" will still be referring to user "abc"'s principal. Note that in our code for the Login method, if a user login is unsuccessful, we set the thread's principal to null as shown below. This ensures that an invalid user will not be able to use a previously authenticated user's principal to gain invalid access to the system.

SqlDataReader lReader =
            lCommand.ExecuteReader ();
if (!lReader.Read
            ()){
Thread.CurrentPrincipal = null;
return false;
}
lReader.Close ();

A thread always requires a principal associated with it. Setting a thread's principal to null can affect the thread's internals, and as a result, the .NET framework replaces a null principal by an empty instance of the GenericPrincipal class, as you can see below. This has the same effect as resetting security for the thread to deny all access. The following screen shot was captured by passing in an invalid set of credentials and watching the thread's principal in the Visual Studio .NET debugger. A null principal is replaced by a GenericPrincipal and GenericIdentity pair, and the IsAuthenticated property of the identity object is set to false. This ensures that an invalid user will not be able to bypass any role based security checks.

Let us now look at the custom identity class shown below:

internal class CustomIdentity :
            IIdentity
{
public CustomIdentity (string iUserName, bool
            iIsAuthenticated)
{
mUserName = iUserName;
mIsAuthenticated
            = iIsAuthenticated;
}

public string
            AuthenticationType
{
get{
return "Custom
            Authentication";
}
}

public bool
            IsAuthenticated
{
get{
return true;
}
}

public
            string Name
{
get{
return
            mUserName;
}
}

private string mUserName =
            null;
private bool mIsAuthenticated = false;
}

The CustomIdentity class implements the IIdentity interface and the three properties associated with this interface. This is a fairly straightforward class, where the LoginServer passes the property values to this class's constructor, and the class's properties expose these values.

The custom principal class is fairly simple too, and is shown below:

internal class CustomPrincipal :
            IPrincipal
{
public CustomPrincipal (IIdentity iIdentity,
            string[] iRoles)
{
mIdentity = iIdentity;
mRoles =
            iRoles;
Array.Sort (mRoles);
}

public IIdentity
            Identity
{
get{
return mIdentity;
}
}

public
            bool IsInRole (string iRole)
{
return !(Array.BinarySearch
            (mRoles, iRole) < 0);
}

private IIdentity mIdentity =
            null;
private string[] mRoles = null;
}

The CustomPrincipal class implements the IPrincipal interface and the single property and method associated with this interface. The constructor of this class takes in a reference to the underlying identity object, and a string array of associated roles. We sort this array so that we can use the BinarySearch method from the System.Array class for a fast lookup of a role. The IsInRole method calls this method to determine if a given role is present in the principal's set of roles.

An important point to note with these custom objects is these classes's protection levels. The CustomIdentity and CustomPrincipal classes are not public classes like the LoginServer; instead we use the internal access qualifier to ensure that these classes can be accessed only by the LoginServer. The internal qualifier permits access to a class only by other classes in the same assembly. This is important to protect the identity and principal objects from other clients. If these were public classes, a component could directly invoke methods on these classes and bypass the LoginServer class altogether. It could then pass any role it wanted to the CustomPrincipal object and have these roles associated with the thread's principal. In order to ensure that these classes can be instantiated and invoked only if the LoginServer's Login method succeeds, we make these classes internal to the assembly. Note that other assemblies can still access the custom principal, by using the IPrincipal and IIdentity interfaces. Thus, read access is allowed, but invalid creation is avoided.

Once a component calls the Login method on the LoginServer, it can use role based security to check for role membership for any given user, as shown below. Before allowing access to a resource, the code checks whether the current identity object is authenticated, and then it uses the principal's IsInRole method to determine role membership.

private void btnLogin_Click(object sender, System.EventArgs e)
{
// try to login a user, if successful, the custom principal
// will be created and associated with the thread
LoginServer lLoginServer = new LoginServer ();
if (!lLoginServer.Login (txtUserId.Text, txtPwd.Text))
MessageBox.Show ("Login Failed.", "Custom Security");
}

private void btnChairman_Click(object sender, System.EventArgs e)
{
IPrincipal lPrincipal = Thread.CurrentPrincipal;
IIdentity lIdentity = lPrincipal.Identity;
string lName = lIdentity.Name;

// use custom role based security to grant or deny access
if (lIdentity.IsAuthenticated && lPrincipal.IsInRole ("Chairman"))
MessageBox.Show (lName + " is a Chairman, Access Granted.",
"Custom Security");
else
MessageBox.Show (lName + " IS NOT a Chairman, Access DENIED.",
"Custom Security");
}

Attached Sample Code

The attached sample code consists of two projects, SecurityClient and SecurityServer, and an Access database, CSharpSecurity.mdb.

The SecurityServer project contains the LoginServer, CustomIdentity and CustomPrincipal class implementations, and the SecurityClient project contains a Windows Forms application that uses these classes, as shown below. The CSharpSecurity.mdb database contains all the exported SQL Server tables and the sample data. Readers should create a SQL Server database and import this Access mdb file into the SQL Server database.

The above screen shot shows our sample application that uses custom security. Once a user id and password are entered, and the Login button is clicked, the user is authenticated against the SQL Server tables. If the credentials are valid, the custom principal objects are created and associated with the thread. The sample defines three roles, and the three group boxes represent each of these three roles. Based on role membership, access will be granted or denied as shown in the following screen shots.

Please follow the steps outlined below to set up the sample application on your machine.

  1. Create a new SQL Server database called CSharpSecurity.
  2. Use SQL Server DTS to import the contents of the CSharpSecurity.mdb file into this table.
  3. Update the connection string in the Login method of the LoginServer class in the SecurityServer.cs file to use the appropriate credentials on your local machine.

Conclusion

Security has always been a complex and intimidating aspect of software development on Windows. The .NET framework, however, supports a very open and extensible scheme that makes a custom security implementation very easy, as we demonstrated in this article. For most developers, the built in security schemes should suffice, and developers building custom security schemes should tread carefully. In spite of the ease of development with .NET, security done wrong can haunt you for a long long time.

Happy .NETing.

If you'd like to discuss a broader range of C#Today topics and issues, or if you have any C# related questions you need answers to, please visit our online resource centres at : http://p2p.wrox.com/list.asp?list=csharptoday_discuss&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq and http://p2p.wrox.com/list.asp?list=c_sharp&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq
C#Today - © 2002 WroxPress Limited


Custom Permissions
http://www.csharptoday.com/content.asp?id=1694
Article Rating: 5
Published on November 14, 2001
Abstract
Any multi user system needs to have a concept of security to protect one user’s data from another. This security element could be anything from granting or denying a user access to a file, a network share, or certain entities in a database. These security mechanisms are controlled via the use of Permissions. Before the advent of .NET, using permissions in application code was hard, but now the framework allows a very open and extensible permission architecture. In this article, the third of the week's security-based articles, Kaushal Sanghavi explores the support that the .NET framework offers for working with permissions, and demonstrate how developers can design their own custom permissions to control access to protected data.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

Different variations of security have existed from the very start of operating systems; at one end of the spectrum was Microsoft DOS with absolutely no built in security, and at the other end are the current versions of Windows, that support sophisticated security mechanisms such as PKI and Kerberos. This security element could be anything from granting or denying a user access to a file, a network share, or certain entities in a database. These security mechanisms are controlled via the use of Permissions that are explicitly granted or denied to users. Before the advent of .NET, using permissions in application code was non-trivial. Furthermore, it was impossible for a component to explicitly specify the permissions it required to operate. For example, a component that accessed a file assumed that it had access to the file, and if the required permission was not held, the component would then have to handle the error raised by the operating system. This not only lead to involved error handling, but also was a cause of confusion for administrators, who rarely knew what set of permissions a component required in order to execute flawlessly. The .NET framework allows a very open and extensible permission architecture that is tied closely to Code Access Security (and to a lesser extent, Role Based Security). We have covered code access and role based security in previous articles (http://www.csharptoday.com/content.asp?id=1717&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq); in this article, we will explore the support that the .NET framework offers for working with permissions, and demonstrate how developers can design their own custom permissions to control access to protected data.

Note that this article is based on Visual Studio .NET Beta 2 and Windows 2000 (all versions).

Permissions

Permissions are central to the .NET framework's concept of security. Any code that is executed under the .NET runtime needs to possess permissions to execute. This permission is granted to the code via the policy settings specified by an administrator on any given machine. These policy settings control the level of access any component enjoys, and could be based on the identity that the code is executing under, or the origin of the code (is it downloaded from the internet, is it signed by a trusted authority, etc). Once the security policies are in place, code that executes under the CLR has to abide by these policies. Very often, designers of a component know the permissions that their component would need in order to execute successfully. For example, if the component accesses operating system objects like files or the registry, the component will need access to these objects if it is to be able to function. In such a scenario, a component can request the desired permissions in advance, via the use of attributes embedded in its metadata, so that the runtime can determine whether these permissions can be granted to the code or not. If the permissions cannot be granted, the runtime will not load the component, and this results in much better error handling than if a component were to fail in the middle of its execution due to unavailable permissions. Even if permissions are not explicitly requested, the .NET runtime will ensure that all access to protected resources is verified.

For a complete discussion on permissions, and how permissions can be granted or denied in code, please refer to our article on Code Access Security (http://www.csharptoday.com/content.asp?id=1717&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq). The rest of this article will focus on designing custom permissions that can be used to augment the default permission classes that are provided by the .NET framework.

Custom Permissions

The System.Security.Permissions namespace provides classes that represent permissions to control some of the commonly accessed resources. This namespace provides classes for protecting file IO, registry access, access to environment variables, etc. These permissions are usually sufficient to protect the most common types of system objects. However, depending on the application, developers may need to extend this support to protect access to objects that depend on the individual application's business logic. In addition to protecting types of objects not covered by the .NET framework, developers often need to fine tune the existing support. For example, an application might want to grant access to the local registry while denying access to registries on remote machines, or an application might want to control access to certain portions of a file instead of the entire file. Any situation where the granularity of control differs from that provided by the .NET framework calls for a custom permission.

Implementing a custom permission is fairly simple, and involves the implementation of only a couple of interfaces. However, custom permissions should be designed very carefully if they are to prove effective. For example, assume we have a custom permission that controls access to a table in a database. The data access component could check for this permission before it allows a caller to execute queries against this table. However, there is no guarantee that a caller will use this data access component to communicate with the database; if the database itself doesn't have the appropriate security set up, there is nothing stopping a developer from using ADO.NET to bypass the data access component and directly access the tables in the database. Thus, when designing a custom permission, care has to be taken that all other forms of access to the element being protected by the custom permission are at least as restrictive as the custom permission.

Any class that implements a custom permission needs to implement the IPermission and IUnrestrictedPermission interfaces. The IPermission interface contains methods that deal with manipulating the custom permission, and the IUnrestrictedPermission interface contains a single method that indicates whether unrestricted access is allowed to the protected object. These methods are called imperatively by the code that tries to access the protected resource. Custom permissions can be used declaratively as well, by implementing an attribute class that derives from the CodeAccessSecurityAttribute class.

Now that we have covered the basics of permissions, and also introduced custom permissions, the rest of this article will focus on implementing a custom permission class. This class will augment the FileIOPermission class that controls access to disk files. One notable shortcoming of the FileIOPermission class is that a single permission is used to control write access to a file, and creation and deletion of a file. Very often, applications need a finer grain of control, where certain users are allowed full access, but certain users are only allowed to write to existing files and are not allowed to create or delete files. The custom permission class that we build will treat writing, creation and deletion of a file as independent permissions that can be granted or denied. Thus, certain users can be granted only a write permission, while more trusted users can be allowed to create and delete files.

Custom Permissions Class

There are two ways in which a custom permission class can be implemented. One way of doing this is by implementing the IPermission and IUnrestrictedPermission interfaces. A slightly easier way is to have the custom permission class inherit from the CodeAccessPermission class in the System.Security namespace. CodeAccessPermission implements IPermission, so some of its methods can be reused. There is one important difference between using CodeAccessPermission and IPermission directly to implement a custom permission. The CodeAccessPermission performs a stack walk for every permission that it checks. A stack walk implies looking at the entire call stack and verifying that each assembly in the call chain has access to the protected resource. If a single assembly does not have access, the request is rejected. Note that this is a very secure way of implementing permissions, but stack walks are fairly expensive, and thus this is not the most performant method. By implementing IPermission directly on the other hand, a stack walk can be avoided.

Since we are building a permission that will control creation and deletion of files, we will need to perform a stack walk to ensure that all assemblies in the call stack have the required permission. We will thus build our permission class by deriving from the CodeAccessPermission class. This class will have three properties that control whether files can be written to, created or deleted. For the sake of simplicity and clarity, this permission will be granted to all files and folders on a machine. Thus, if an assembly has the Create permission, it can create files in any folder on the local machine. In a more realistic example, this would be controlled at a folder level, so that files can be created in some folders and cannot be created in other folders. However, this would require using hash tables to keep track of all the folders and their associated permissions, and might prove to be distracting for the readers.

The class that implements a custom permission needs to implement the following methods.

  • Copy: Copies the current permission object onto a new one.
  • Intersect: Returns an object that is the intersection of two permission objects.
  • IsSubsetOf: Checks if one permission object is a subset of another.
  • ToXml: Creates an XML representation of the permission object.
  • FromXml: Creates a permission object from its XML representation. The ToXml and FromXml methods are implemented so that the permission object can be persisted.

Our permission class will implement the above five methods and the IsUnrestricted method from the IUnrestrictedPermission interface. Further, the permission class will contain three boolean members that represent write, create and delete access for the object. Another boolean member will hold the unrestricted value for the IsUnrestricted method.

public class CustomPerm:
            CodeAccessPermission, IUnrestrictedPermission
{
public
            CustomPerm(PermissionState iPermState)
{
mUnrestricted =
            (iPermState == PermissionState.Unrestricted);
}

public
            CustomPerm(bool iWritePerm, bool iCreatePerm, bool
            iDeletePerm)
{
mWritePerm = iWritePerm;
mCreatePerm =
            iCreatePerm;
mDeletePerm = iDeletePerm;
}

public bool
            IsUnrestricted()
{
return mUnrestricted;
}

public
            override IPermission Copy()
{
CustomPerm lCopy = new
            CustomPerm (PermissionState.None);

// copy all the fields
            over
lCopy.mUnrestricted =
            this.mUnrestricted;
lCopy.mWritePerm =
            this.mWritePerm;
lCopy.mCreatePerm =
            this.mCreatePerm;
lCopy.mDeletePerm =
            this.mDeletePerm;

return lCopy;
}

public override
            IPermission Intersect(IPermission iTarget)
{
if (iTarget ==
            null)
return null;

CustomPerm lResult = new CustomPerm
            (PermissionState.None);
CustomPerm lTarget =
            (CustomPerm)iTarget;

// create an intersection set of the
            objects
lResult.mUnrestricted = (this.mUnrestricted == true
            &&
lTarget.mUnrestricted == true);
lResult.mWritePerm
            = (this.mWritePerm == true &&
lTarget.mWritePerm ==
            true);
lResult.mCreatePerm = (this.mCreatePerm == true &&
lTarget.mCreatePerm == true);
lResult.mDeletePerm =
            (this.mDeletePerm == true &&
lTarget.mDeletePerm ==
            true);

return lResult;
}

public override bool
            IsSubsetOf(IPermission iTarget)
{
if (iTarget ==
            null)
return false;

CustomPerm lTarget =
            (CustomPerm)iTarget;

return (this.mUnrestricted ==
            lTarget.mUnrestricted &&
this.mWritePerm ==
            lTarget.mWritePerm &&
this.mCreatePerm ==
            lTarget.mCreatePerm &&
this.mDeletePerm ==
            lTarget.mDeletePerm);
}

public override SecurityElement
            ToXml()
{
SecurityElement lElem = new
            SecurityElement("CustomPerm");

// the schema can be whatever
            meets the requirements
lElem.AddAttribute ("Unrestricted",
            mUnrestricted.ToString());
lElem.AddAttribute ("WritePerm",
            mWritePerm.ToString());
lElem.AddAttribute ("CreatePerm",
            mCreatePerm.ToString());
lElem.AddAttribute ("DeletePerm",
            mDeletePerm.ToString());

return lElem;
}

public
            override void FromXml(SecurityElement iElement)
{
string lVal
            = iElement.Attribute("Unrestricted");
if (lVal !=
            null)
this.mUnrestricted = Convert.ToBoolean (lVal);

lVal
            = iElement.Attribute("WritePerm");
if (lVal !=
            null)
this.mWritePerm = Convert.ToBoolean (lVal);

lVal =
            iElement.Attribute("CreatePerm");
if (lVal !=
            null)
this.mCreatePerm = Convert.ToBoolean (lVal);

lVal =
            iElement.Attribute("DeletePerm");
if (lVal !=
            null)
this.mDeletePerm = Convert.ToBoolean
            (lVal);
}

private bool mUnrestricted = false;
private
            bool mWritePerm = false;
private bool mCreatePerm =
            false;
private bool mDeletePerm = false;
}

The above class represents our custom permission class. The crux of this class is the three boolean member variables that represent the Write, Create and Delete permissions for files on the local machine. The constructor takes in values for these three permissions and sets the state of its member variables. The Copy method simply creates a new instance of this class as a copy of the existing object, the Intersect method compares all the member variables and creates a new intersection object, and the IsSubsetOf method returns true if the passed object is identical to the current object, otherwise false.

The ToXml method serializes the object into XML that has the following format, and the FromXml method reads this serialized XML and sets the value of the member variables based on the XML attributes. This allows the permission object to be serialized to disk and then recreated from the persisted XML.

<CustomPerm WritePerm=true CreatePerm=false DeletePerm=true>

This is all that is required to implement a custom permission. A security policy will now set the values for the specific permissions, and the component that handles access to files can demand this permission from its callers. This component that serves requests for file operations is shown below. The code below only shows the CreateFile method for simplicity - please refer to the attached code for the full class implementation.

public class
            FileRequestServer
{
public void CreateFile (string iFileName,
            bool iCreatePerm)
{
PermissionSet lPS = new PermissionSet
            (PermissionState.None);
FileIOPermission lFilePerm = new
            FileIOPermission
(PermissionState.Unrestricted);
CustomPerm
            lPerm = new CustomPerm (false, iCreatePerm,
            false);
lPS.AddPermission (lPerm);
lPS.AddPermission
            (lFilePerm);
lPS.PermitOnly ();

Create (iFileName);
}

private void Create (string
            iFileName)
{
CustomPerm lPerm = new CustomPerm (false, true,
            false);
lPerm.Demand ();

File.Create
            (iFileName);
}
}

The FileRequestServer is the component that will handle requests to process file IO. The CreateFile method takes in the name of the file that needs to be created, and also a boolean variable that represents whether the Create permission is granted to the code or not. In our sample, for the sake of simplicity, we will allow the user to pass in this value, but ideally this would be read from an administrative policy. A discussion of security policy implementation is beyond the scope of this article. The CreateFile method sets up the permission set based on the custom permission value; it also adds the FileIOPermission since this will be required to create a file. Once this permission set is created, it calls the private Create method that invokes the Demand method on our custom permission object. If the Create permission is granted, this demand will succeed, and if the permission has not been granted, a SecurityException is thrown.

A key point to note here is that all file IO has to happen through the FileRequestServer for the custom permissions to be effective. The following code block shows how the FileRequestServer object can be used to create a file. Again note that we pass in a user selected value for the Create permission, realistically this would be specified in a security policy.

private void btnCreate_Click(object sender,
            System.EventArgs e)
{
FileRequestServer lServer = new
            FileRequestServer ();
lServer.CreateFile (txtFile.Text,
            chkCreate.Checked);
}

Implementing Declarative Permissions

In the above section, we discussed how custom permissions can be demanded imperatively. Permissions can be used declaratively as well, via the use of attributes. In order to use a custom permission declaratively, we need to declare an attribute class that derives from the CodeAccessSecurityAttribute class. This class needs to implement a single method CreatePermission, which will create an instance of the underlying custom permission object. Furthermore, this class can also support properties that may be used to specify values for member data. The following code block shows the CustomPermAttribute class, which can be used with our custom permissions class from the above section for declarative support. This class supports three properties for the individual permissions.

[AttributeUsageAttribute(AttributeTargets.All)]
public
            class CustomPermAttribute:
            CodeAccessSecurityAttribute
{
public
            CustomPermAttribute(SecurityAction iAction): base
            (iAction)
{}

public bool Write
{
get{
return
            mWritePerm; }
set{
mWritePerm = value;
}
}

public
            bool Create
{
get{
return
            mCreatePerm;
}
set{
mCreatePerm =
            value;
}
}

public bool Delete
{
get{
return
            mDeletePerm;
}
set{
mDeletePerm =
            value;
}
}

public override IPermission
            CreatePermission()
{
return new CustomPerm (mWritePerm,
            mCreatePerm, mDeletePerm);
}

private bool mWritePerm =
            false;
private bool mCreatePerm = false;
private bool
            mDeletePerm = false;
}

The following code block shows how our custom permission can be used declaratively.

[CustomPerm (SecurityAction.Demand, Create=true)]
public void DeclarativeCreateAllow (string iFileName)
{.}

Attached Sample Code

The attached sample code contains two projects, PermissionObjects and PermissionClient. The PermissionObjects project contains the custom permission class and the FileRequestServer class. The PermissionClient project contains the client code that will use the FileRequestServer object to carry out file operations. The following screen shot displays the sample application. By checking the three check boxes, you can enable the permissions. If a permission is not enabled, and an operation is attempted, a security exception will be thrown.

If the check boxes are checked, access is granted, and the call succeeds. However, if the check boxes are unchecked, access is denied and a security exception, as shown in the following screen shot is raised.

Conclusion

Permissions play a very important role in any security implementation. Before the .NET framework, developers had very little choice in implementing custom security schemes and permissions, and were often limited to using Windows accounts and domain security privileges to build their applications. The .NET framework introduces a very clean and extensible model for building custom security schemes and designing and implementing custom permissions is fairly simple, as we have shown in this article.

In the next article in this series, we will take a look at Custom Security Implementation.

Happy .NETing.

If you'd like to discuss a broader range of C#Today topics and issues, or if you have any C# related questions you need answers to, please visit our online resource centres at: http://p2p.wrox.com/list.asp?list=csharptoday_discuss&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq and http://p2p.wrox.com/list.asp?list=c_sharp&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq
C#Today - © 2002 WroxPress Limited


Role Based Security
http://www.csharptoday.com/content.asp?id=1697
Article Rating: 4.5
Published on November 13, 2001
Abstract
One of the most common ways of handling authorization is via a technique known as Role Based Security (other techniques use individual user information to grant or deny access). Different variations of role based security have existed in Windows for a long time now. Windows NT and Windows 2000 define built in user groups, such as Administrators, Power Users, etc, and users are granted access to files and network shares based on their Windows group memberships. MTS and COM+ introduced role based security at the application development level, and the .NET framework builds upon this. In this article, Kaushal Sanghavi explores .NET’s support for role based security and how it can be used to build a flexible authorization scheme.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

User security is broadly considered to comprise of two functional areas; authentication and authorization (encryption is also a key unit of security, but that is at the data level). Authentication deals with a user identifying herself with the right security credentials. These credentials will usually be a user name and password combination, but we can also use Digital Certificates, XML Signatures, and Smart Cards. Authentication deals with identifying users, and only letting users who possess valid credentials into the system. Authorization is the step that occurs after authentication, and deals with allowing users access to selected parts of the system to which they have authorized access. For example, a system may have many different authenticated users, but a user with Administrator permissions is likely to be authorized to access more areas than a user with Guest permissions. One of the most common forms of handling authorization is via a technique known as Role Based Security (other techniques use individual user information to grant or deny access). Different variations of role based security have existed in Windows for a long time now. Windows NT and Windows 2000 define built in user groups, such as Administrators, Power Users, etc, and users are granted access to files and network shares based on their Windows group memberships. MTS and COM+ introduced role based security at the application development level, and the .NET framework builds upon this. In this article, we will explore .NET's support for role based security, and how it can be used to build a flexible authorization scheme. Note that although role based security in .NET is very similar to role based security in COM+, we will leave the integration of tradition COM+ security and .NET security for a later article in this series.

Note that this article is based on Visual Studio .NET Beta 2 and Windows 2000 (all versions).

Role Based Security

Role based security, arguably, is very easy to implement. We could write an application with hard coded user strings and a multitude of "if-else" statements that branch code, based on the user name. Thus, user "xyz" would run one version of a stored procedure and user "abc" would run another version that was tailored to the user's access levels. While this seems to be a fairly simple thing to do, maintaining this code is impractical. Every time a new user is added, or a user's access level changes, a code change will be required. While this is a quick solution for a small system or prototype, this is not a scalable or maintainable option. The configuration of security roles and access levels needs to be separated from the code for the solution to be scalable and flexible. COM+ allows exactly this, by letting administrators set up roles that represent common groups of users, and then allowing access to certain components by users belonging to specific roles. For example, a component that deals with payroll records in an organization can be set so that it is only accessible to users who belong to the "Payroll" group.

The following Component Services sample screenshot shows how we would set this up:

Thus, administrators can specify different roles for a given COM+ application and associate users with roles, and roles with components. Since this whole process is declarative and is administered from a MMC snap-in (or programmatically with the COM+ admin objects), maintaining this system is easy, and does not involve code changes. In fact, the whole security architecture is transparent to the component, and is handled by the runtime. Role based security can be configured externally, simply by defining roles and then granting access to components (or interfaces supported by the component) by specific roles. This is a major feature of COM+ that is used by many enterprise applications.

Declarative role based security, as described above, is a great feature, but as you can see from the above screen shots, declarative security in Component Services works at a component level. A role is granted access to a component on an all-or-nothing basis. Users in a role can either access all the public methods in a component, or can access none at all. This works well when the component boundaries and role boundaries are completely aligned. Declarative security, however, does not work when different users need different kinds of access to a particular component. Consider the example of a national retail chain, and a component that shows sales data for a given set of stores. The same component will be used by a district manager and by a regional manager, but the SQL statement executed will be different based on the organizational hierarchy. In this case, it is not practical to build separate components, one that will return sales data for a region and another that returns sales data for a district.

In such situations, the developer has no choice but to resort to imperative security, wherein the security checks are embedded within the code. The component will have to check the roles that the user belongs to, and then execute a branch of code based on the specific user role. Note that this is not as bad as the scenario we explained at the beginning of this section. Typically, an enterprise will define fewer roles compared to hundreds or thousands of individual users. Further, while users in a role may change over time, the access granted to a particular role rarely changes. As a result, developers don't face a huge maintenance problem by using imperative role based security. The following code block shows how roles can be used with imperative security. We will discuss this in detail later in this article.

if (ContextUtil.IsSecurityEnabled)
{
if (ContextUtil.IsCallerInRole("DistrictManagers"))
GetDistrictManagerSalesData ();

else if (ContextUtil.IsCallerInRole("RegionalManagers"))
GetRegionalManagerSalesData ();
}

So far, our discussion has focused on COM+, but the behavior is identical for .NET components. .NET allows developers to build serviced components, which are managed components that can be hosted by COM+ applications, and consequently, enjoy all the features of COM+ applications.

Differences Between COM+ and .NET

There is, however, one important distinction between COM+ and .NET where role based security is concerned. Under COM+, the users belonging to a particular role were mapped to Windows domain accounts. Thus, if a user needed to be added to a particular role, that user needed to have a Windows account in the domain where the application would run. If you had a database full of users for a COM+ application, using role based security would be extremely difficult. It is very different in .NET, where the .NET framework does not force developers to use Windows accounts as the sole form of users. While Windows accounts are supported, simple user name - password based accounts are also supported, as are custom implementations of user accounts. This offers much more flexibility to developers for their authentication and authorization schemes.

The rest of the article will focus on the default .NET implementation of role based security. Custom implementation of role based security is covered in a separate article.

Principals and Identities

Role based security in .NET is based on Principals and Identities.

An identity encapsulates information about the authenticated user. This information includes the user name and the type of authentication that is used.

A principal encapsulates information about the security context, such as the user identity and user roles. A principal object contains a nested identity object, and the .NET runtime uses these objects to provide role based security. For a user session, the runtime maintains a principal object containing information about the roles that the user belongs to, and also allows access to the underlying identity object. Note that a principal object and an identity object always go in pairs, and any security implementation consists of both objects. By default, the .NET framework supports the following two types of principal and identity objects:

  • Windows: This scheme is used for authorizing users based on their Windows domain accounts.
  • Generic: This scheme is used for authorizing users based on simple user names that are not related to Windows domain accounts.

The .NET runtime maintains a context, which is very similar to the object context maintained by MTS and COM+, and contains information about the principal and identity objects that are associated with the current invocation. These objects are made available to the callee so that the caller can be authorized either declaratively or imperatively, as discussed in the next section.

Imperative and Declarative Security

The .NET framework allows a component to check a caller's role before allowing the call to succeed. This check can be performed either imperatively or declaratively. The declarative check is via the use of .NET attributes and is useful when the check is to be performed at a method level. The imperative check is via the use of the PrincipalPermission class from the System.Security.Permissions namespace, which allows you to demand specific user names or roles.

Let us look at both these methods of implementing role based security. We will cover imperative security first. Let us consider a simple example where we have two users, one is a district manager, and another is a regional manager. The district manager can only access the district manager's method, and the regional manager can only access the regional manager's method.

The following code block shows the secure component that will enforce this role based security.

public class ImperativeSecureComponent
{
public void DistrictManagerMethod ()
{
PrincipalPermission lPerm = new PrincipalPermission (null, "DistrictManagers");

try
{
// ensure that only district managers can access this
lPerm.Demand ();
}
catch (Exception lException)
{
MessageBox.Show (lException.Message, "DistrictManagerMethod exception");
return;
}

MessageBox.Show ("Access granted to DistrictManagerMethod", "DistrictManagerMethod");
return;
}

public void RegionalManagerMethod ()
{
PrincipalPermission lPerm = new PrincipalPermission (null, "RegionalManagers");

try
{
// ensure that only regional managers can access this
lPerm.Demand ();
}
catch (Exception lException)
{
MessageBox.Show (lException.Message, "RegionalManagerMethod exception");
return;
}

MessageBox.Show ("Access granted to RegionalManagerMethod", "RegionalManagerMethod");
return;
}
}

The above code shows a simple component that has two public methods, the DistrictManagerMethod and the RegionalManagerMethod. As the name suggests, only users who belong to the DistrictManagers role can access the DistrictManagerMethod, and only users belonging to the RegionalManagers role can access the RegionalManagerMethod. Enforcing this check is as simple as creating a new instance of the PrincipalPermission object. The constructor for this class takes two parameters, namely the name of the user and the role of the user associated with this permission. Since any user can call the constructor in the specific role, the first parameter is null, while the second parameter contains the name of the required role. Once this permission object is created, we call the Demand method (analogous to code based security) on this object to enforce a security check. If the user calling this method belongs to the demanded role, this method succeeds, and if the user does not belong to the role, a SecurityException is thrown, which can be handled by the component to deny access to the caller.

In this example we associate a method with a single role. If a method were to be accessible from multiple roles, the PrincipalPermission object supports a Union method that can be used to return a permission object that represents the union of multiple permission objects. Similarly, there is an Intersect method as well.

The following code block shows how the DistrictManagerMethod can be modified to allow access to both district mangers and regional managers. The "DistrictManagers" and "RegionalManagers" roles can be read from a Roles table in a database. This could be a simple table that holds all available roles and their levels of access.

public void DistrictManagerMethod
            ()
{

PrincipalPermission lPerm = new PrincipalPermission
            (null, "DistrictManagers");
PrincipalPermission lPerm2 = new
            PrincipalPermission (null,
            "RegionalManagers");

System.Security.IPermission lIP =
            lPerm.Union (lPerm2);
lIP.Demand ();

MessageBox.Show
            ("Access granted to DistrictManagerMethod",
            "DistrictManagerMethod");
return;
}

Note that in this example, we don't call the ContextUtil.IsCallerInRole directly; this is automatically handled by the PrincipalPermission object. The main motivation for using this object instead of calling the IsCallerInRole method is that the PrincipalPermission object usage is consistent with the code access security mechanism.

Declarative security checks are less verbose, as shown below:

public class DeclarativeSecureComponent
{
[PrincipalPermission (SecurityAction.Demand, Role="DistrictManagers")]
public void DistrictManagerMethod ()
{
MessageBox.Show ("Access granted to DistrictManagerMethod", "DistrictManagerMethod");
return;
}

[PrincipalPermission (SecurityAction.Demand, Role="RegionalManagers")]
public void RegionalManagerMethod ()
{
MessageBox.Show ("Access granted to RegionalManagerMethod",
"RegionalManagerMethod");
return;
}
}

Declarative security uses attributes to specify the roles to be required from the caller. If the caller does not belong to the specified role, a SecurityException is thrown.

Difference Between Declarative and Imperative Security

There are two main differences between the use of declarative security and imperative security. In declarative security, the roles are essentially hard coded at design time, while in imperative security, these can be read from an external source such as a database or a config file. While config files can be used for prototypes or very simple applications, databases should be the repository of choice for roles. Further, with declarative security, the granularity of the access check is a method, while with imperative security, the granularity is controlled by the developer.

Now that we have discussed how a component can enforce security checks, let us see how a caller of such a component can communicate the roles it belongs to. This is shown below:

private void SetupPrincipal
            ()
{
string lUserName = cbUserName.SelectedItem.ToString
            ();
GenericIdentity lIdentity = new GenericIdentity
            (lUserName);

string[] lDMRoles =
            {"DistrictManagers"};
string[] lRMRoles =
            {"RegionalManagers"};

GenericPrincipal lPrincipal;

if
            (lUserName.Equals ("Alex DM"))
lPrincipal = new
            GenericPrincipal(lIdentity, lDMRoles);

if (lUserName.Equals
            ("Tony RM"))
lPrincipal = new GenericPrincipal(lIdentity,
            lRMRoles);

Thread.CurrentPrincipal =
            lPrincipal;
}

We first create an identity for the user, and then, based on the user name, associate the user name and a role to create a principal. This principal is then attached to the current thread object so that this can be accessed by all downstream components. This is a simple example to demonstrate a concept, and so we have hard coded the association of user names and roles. In a more realistic example, this association would come from a database or an external file. Hard coding it in the code is neither scalable nor flexible as we discussed in the beginning of this article. We will cover storing users and roles in a database in further articles in this series.

Note that in this article, we have discussed role based security from a middle tier perspective, and thus we talk about storing roles and users in a database or a config file. For ASP .NET applications, it is possible to use the web.config file to store this information, but a more generic and scalable solution would be to use a database. The table schemas for this database would be dependent on the application that is being developed, but at the least, it should have a user table that stores user information, a role table that stores information about existing roles and the level of access associated with each role, and a mapping table that maps users to roles. This mapping table is necessary since, a single user could belong to many roles. Once this schema is designed, a simple administrative utility can be designed to add and update users and roles to the system.

Attached Sample Code

The attached sample code contains all the code discussed in the earlier section. The following screen shot shows the sample application. There are two users, Alex who is a district manager, and Tony who is a regional manager. The Imperative and Declarative buttons execute the two different ways of enforcing role based security. Depending on the selected user, one of the methods will succeed and the other will fail with a SecurityException.

Conclusion

Authorization is an integral part of any security scheme, and role based security allows for a highly extensible and scalable solution. Role based security fits in very well with IIS and Windows based security, and in a major enhancement over COM+, the .NET framework allows custom implementations of role based security. This should allow developers to choose from an existing implementation of role based security or custom build their own implementations. Either way, .NET's support for role based security should go a long way in helping developers achieve their security objectives.

Happy .NETing.

C#Today - © 2002 WroxPress Limited


Code Access Security
http://www.csharptoday.com/content.asp?id=1743
Article Rating: 4.5
Published on November 12, 2001
Abstract
With the .NET strategy evangelizing loosely coupled component development, the designers of .NET had to come up with security strategies that helped to control these potentially dangerous components. The .NET framework introduces two main models for securing your application: code based security, and role based security. In this article, the first of a week-long topic on security, Kaushal Sanghavi discusses the .NET framework’s for security, and demonstrates how we can use permissions in code. Later in the week, he will also tackle custom security permissions, custom security implementation, and integrating COM+ and .NET security schemes.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

The development world is slowly moving away from large monolith applications that were written by a single entity and executed in a closed controlled environment. Gone are the days when a company would build all its components and then package them into a small number of binaries that would only be used in one application. Monolith programming has slowly made way for distributed component based development, where components are developed as self-contained units that perform a well known function, and can be used by many applications. There are numerous independent software vendors and component vendors who build and publish components that can be used to build applications. While this kind of development provides a great architecture for building very loosely coupled (component based) systems, there are inherent risks to this approach. Since your application is now built using third party components (possibly including freeware components), how well can these components be trusted? Even if reliable vendors have developed these components, what is the guarantee that a malicious component isn't intercepting and spoofing calls to the actual component on a target machine that has been compromised? This issue becomes even more severe with the downloading of components and controls from Internet sites that may not be completely reliable. With the .NET strategy evangelizing loosely coupled component development, the designers of .NET had to come up with security strategies that helped to control these potentially dangerous components. The .NET framework introduces two main models for securing your application: code based security, and role based security. Code based security is based on the fact that the executing code explicitly needs to request and be granted permissions to perform certain actions, and role based security is based on the identity of the principal executing the code. Variations of role-based security have been around since MTS and COM+, but code based security is relatively new. Code based security coupled with administrative policies that govern it is called evidence based security. In this article, we will discuss the .NET framework's support for code and evidence based security, and demonstrates how we can use permissions in code. Unless otherwise specified, we will use the term Code Access Security to refer to code and evidence based security.

Note that this article is based on Visual Studio .NET Beta 2 and Windows 2000.

Code Access Security

Traditional security policies and controls have tended to be role based. Certain user accounts have certain privileges, and code that executes under that user account inherits all the privileges of the account. This has the advantage of being very simple to maintain: simply create groups of users with similar privileges, and assign the privileges to the group. This worked well in a controlled environment, where all the code that executed under a user account came from a trusted source and did not pose any security risks. However, in today's world, where code can land up on a machine from various sources including the Internet, this policy leaves a lot to be desired. If a user with sufficient privileges inadvertently executes malicious code, the code can cause serious damage not only to the user's local system, but it can also have far reaching effects across the entire network. In order to prevent this, and offer a tighter security scheme, the .NET framework introduces the concept of Code Access Security, wherein the code itself needs to have certain privileges, irrespective of the account that the code runs under. This privilege could be granted by an administrator based on the origin of the code, whether or not the code is signed and has a strong name, what kind of permissions the code is requesting, and so forth. Under this scheme, the code needs to request permission before accessing certain resources like the hard disk, the network, or the registry, to name but a few. The CLR at runtime will determine whether the code qualifies for the requested privileges, and if it does, the call succeeds. However, if the CLR, based on a security policy, decides to deny the request, then an exception of type SecurityException is thrown, and the code is not allowed to execute. This security exception is analogous to the infamous Error 5: Access Denied error in the Win32 days, and can be caught using the standard try-catch blocks.

The CLR associates permissions at an assembly level. It is not possible to write a security policy that grants a privilege to a class in an assembly, and then denies it to another class in the same assembly. When a method in an assembly attempts to access a resource, or to perform a privileged operation, the CLR performs a stack walk to determine whether all the assemblies in the current call stack have permission to access the resource or perform the operation. This is a key point in understanding code access security, since without the stack walk security could easily be compromised.

For example, consider the call stack as shown in the diagram below, where method A1 in assembly A calls A2, which in turn calls B1, which calls C1, which finally calls D1. P1, P2, and P3 represent three permissions that can be granted to an assembly. Thus, assembly A has permissions P2 and P3, B has P1 and P3, and so forth.

As you can see from the above diagram, permission P3 has been granted to every assembly. Thus, if method D1, the last method in the call stack, attempts to perform an action that requires permission P3, access will be granted since each assembly in the call stack has the required permission. However, if D1 attempts an action requiring P2, access will be denied, in spite of the assembly having the required permission, since the calling assembly C does not have this permission.

This way, a malicious assembly cannot use a trusted assembly to perform privileged operations. Where permissions are concerned, merely having the permission does not guarantee access to the resource. The most restrictive policy is adopted, and if any assembly in the call stack lacks a permission, the entire call stack is denied that permission.

Permissions

As we discussed in the previous section, the entire code access security mechanism is based on the concept of permissions. Permissions can be requested, denied, granted and revoked by assemblies and security policies. Before we discuss how permissions can be used by assemblies to ensure security, let us look at some of the permissions that the .NET framework defines. Note that these are only a few common permissions; please refer to the Links section for a complete list of permissions. Furthermore, developers can define and implement their own permissions. A discussion on the implementation of custom permissions is beyond the scope of this article. The following sections illustrate how permissions are used in code.

  • EnvironmentPermission: This controls the ability to query and modify the environment variables on the local machine. This includes both system-wide and user level variables.
  • FileIOPermission: This controls the ability to access files and folders on the local machine and on the network. This is a key permission, since if a component cannot access any files, their ability to cause damage is severely restricted. This controls Read, Write and Append access.
  • MessageQueuePermission: This controls the ability to read and write messages to MSMQ. This is useful in preventing malicious code from spoofing or peeking at sensitive messages in a public or private queue.
  • RegistryPermission: This controls the ability to read and write values or keys into the Windows registry. This is an important permission because invalid access to the registry can cause serious damage to the system.
  • SecurityPermission: This permission groups together certain system permissions like the ability to control threads and policies. The most important permission that this type controls is the ability to access unmanaged code.
  • Note that all this discussion of permissions and security applies only to managed code. Since all this security is handled by the CLR, it can only be enforced in managed code. The minute your managed code makes a call into unmanaged code, all bets are off, and the unmanaged code can do whatever it decides to. As a result, permission to access unmanaged code should be granted very judiciously.

In addition to these permissions, there are many other predefined permissions that control access to databases, isolated storage, directory services, printing, DNS, networks, etc.

These permissions that are defined in the CLR are all represented by permission classes that control access to the particular permission. All code access permissions derive from the CodeAccessSecurity class in the System.Security namespace. The following are some of the important methods of this class:

  • Assert: As we discussed earlier in the article, the CLR performs a stack walk for checking every permission granted to every assembly in the call stack. Walking up a stack is not very efficient and can cause performance problems in the code. If that assembly developer decides that performance is more important than security, or if the code is guaranteed to execute in a safe environment, the developer can turn off the stack walk. This is done by calling Assert on the permission for which a stack walk is not necessary. Note that this can be a potential security risk and permissions should be asserted very carefully. Further, only assemblies that have been granted the
  • SecurityPermission can Assert permissions.
  • Deny: Under certain situations (such as when an assembly wants to limit liability), an assembly might want to deny a permission even if an administrator has granted it. This can be done by calling Deny on the specific permission, and this denies callers in the call stack from accessing the resource protected by this permission, even if the caller has been explicitly granted the permission. Note that this holds true only for calls made into the assembly denying the permission. As soon as this method completes execution, and the assembly is removed from the call stack, the permissions of the other assemblies revert back to normal. This is useful when an assembly wants to override all security policies and completely avoid access to a resource. As we mentioned earlier, since the CLR adopts the most restrictive policy, this denial is allowed.
  • RevertAll: This reverts an assembly's permission back to default. This is used when the default set of permissions has been overridden by the use of an Assert or a Deny call.

The .NET framework provides these methods so that assemblies can override some of the defaults that are provided by the CLR.

Imperative Security

Permissions can be requested and denied, as we discussed in the previous section. An assembly can perform these requests in two ways. The first approach is called Imperative Security, where the assembly uses the CodeAccessSecurity class to obtain or manipulate a permission. This is done by creating instances of the different permission classes, and using a permission set to group these permissions together. A permission set is represented by the PermissionSet class, and it represents a collection of permissions. You can add or remove individual permissions to or from a permission set, and then use the PermissionSet methods to demand or modify these permissions. Imperative security should be used by assemblies when the permission sets are dynamic and are unknown at design time.

The following code block shows how we can use the PermissionSet class to request and deny permissions.

public void ProcessRequest (string iFile,
            string iEnv)
{
PermissionSet lPS = new PermissionSet
            (PermissionState.None);

if (mFilePerm){
FileIOPermission
            lFP = new FileIOPermission
(FileIOPermissionAccess.Read,
            iFile);
lPS.AddPermission (lFP);
}

if
            (mEnvPerm){
EnvironmentPermission lEP = new
            EnvironmentPermission
(EnvironmentPermissionAccess.Read,
            iEnv);
lPS.AddPermission (lEP);
}

lPS.PermitOnly
            ();

ReadFile (iFile);
ReadEnv
            (iEnv);
return;
}

The above code block shows a method ProcessRequest that needs to read both a file and an environmental variable. In order to read a file, the code requires the FileIOPermission, and in order to read the environmental variables, the code requires the EnvironmentPermission. Since we allow the user to grant or deny these permissions, we use conditional blocks to create these permissions and add it to the permission set. The permission set as we discussed earlier, is an instance of the PermissionSet class. During construction, we create an empty permission set by using the PermissionState.None flag. Next, we call the PermitOnly method on the permission set, which ensures that only the permissions that have been explicitly added to the set will be available. Any permission that is not a part of the set will be automatically denied. Further, when we create the FileIOPermission and the EnvironmentPermission objects, we specify what object this permission is being requested for. This is important, since a permission might be available for a particular resource, but can be denied for another. Once this code is executed, the ReadFile and ReadEnv will succeed or fail depending on whether the permissions have been granted or not.

The ReadFile and ReadEnv methods are shown below. These methods are fairly simple, and do not contain any security code. If the necessary permissions are available, the code will execute, and if not, the catch block will catch an instance of type SecurityException.

private void ReadFile (string
            iFile)
{
FileStream lStream;
try{
lStream = new
            FileStream
(iFile, FileMode.Open,
            FileAccess.Read);
}
catch (Exception
            lException){
MessageBox.Show (lException.Message, "File
            Error");
return;
}

int lFileLen =
            (int)lStream.Length;
byte[] lBytes = new byte
            [lFileLen+1];
lStream.Read (lBytes, 0, lFileLen);

string
            lDataRead;
lDataRead = System.Text.Encoding.ASCII.GetString
            (lBytes);
MessageBox.Show (lDataRead, iFile);
lStream.Close
            ();

return;
}

private void ReadEnv (string
            iEnv)
{
try{
string lEnvVal =
            Environment.GetEnvironmentVariable (iEnv);
if (lEnvVal ==
            null)
lEnvVal = "Not Set";
MessageBox.Show (lEnvVal,
            iEnv);
}
catch (Exception lException){
MessageBox.Show
            (lException.Message, "Env
            Error");
}

return;
}

As you can see from this discussion, imperative security is fairly simple. All you need to do is create an instance of the PermissionSet class, and add the appropriate permission objects to this set. Once this is done, you can request or deny some or all of these permissions. If any of the calls fail, an exception is thrown that will contain information about the error.

Declarative Security

Declarative security use Attributes to request, deny or override the required permissions. These security attributes can be specified at the assembly, class, or method level, and control the permissions at that level. The runtime behavior, as far as permissions and security is concerned, is identical for declarative security and imperative security. All the attributes for code access permissions derive from the CodeAccessSecurityAttribute. Similar to the PermissionSet, you can use these attributes to grant or deny access to individual permissions, as shown below.

[FileIOPermission (SecurityAction.Deny, Unrestricted=true)]
[EnvironmentPermission (SecurityAction.Deny, Unrestricted=true)]
public void ProcessRequestDenyAll (string iFile, string iEnv)
{
ReadFile (iFile);
ReadEnv (iEnv);
return;
}

[FileIOPermission (SecurityAction.Assert, Unrestricted=true)]
[EnvironmentPermission (SecurityAction.Assert, Unrestricted=true)]
public void ProcessRequestAllowAll (string iFile, string iEnv)
{
ReadFile (iFile);
ReadEnv (iEnv);
return;
}

As you can see from the above code block, we use FileIOPermissionAttribute and EnvironmentPermissionAttribute to assert and deny permissions. The Unrestricted parameter specifies that the permission applies to all types of files and environmental variables. If we wanted to restrict access to a particular file or environmental variable, we can use the following code. However, note that these values have to be specified at design time and not at run time as in imperative security.

[EnvironmentPermission (SecurityAction.Deny,
            Read="USERNAME")]
public void DenyLogonUser (string iFile, string
            iEnv)
{
ReadEnv (iEnv);
return;
}

The above code block explicitly denies access to the USERNAME environmental variable; access to other variables will be determined by the security policy.

The advantage of declarative security over imperative security is that all the permission data gets included in the metadata of the assembly, and can be read by a utility tool using reflection. This can be a great way to compile statistics or other information about assemblies and their usage of permissions. This is not possible with imperative security. However, imperative security has the advantage of being able to specify the permissions at run time based on conditions. In declarative security, all the settings have to be specified at design time.

The Attached Sample Code

The attached sample code contains one example each of imperative and declarative security, as we discussed in the above sections. The following is a screen shot of the attached application. The checkboxes represent whether the permission should be granted or not.

If the permission is granted, the code attempts to read the specified file and value of the environmental variable, and displays it in a message box. If the permission is not granted, a security exception is thrown as shown in the following screen shots.

Conclusion

Security is a critical part of any system design. Traditionally, developers had very little control over how security was implemented and enforced. However, with the .NET framework and the CLR, developers have far greater control over security, and since developers know the code they write best, this is a great benefit for software development. Further, as we have seen in this article, the code to handle security is very simple. There are a few classes and a few methods, and coding permissions is not rocket science. The key is how to design a solid optimal permission scheme that requests only the needed permissions, and denies all other permissions. Once developers figure out the optimal set of permissions for their assemblies, coding it should be trivial.

Happy .NETing.

If you'd like to discuss a broader range of C#Today topics and issues, or if you have any C# related questions you need answers to, please visit our online resource centres at : http://p2p.wrox.com/list.asp?list=csharptoday_discuss&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq and http://p2p.wrox.com/list.asp?list=c_sharp&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq
C#Today - © 2002 WroxPress Limited


A Suite of Cryptographic Objects Part 2: Implementing Key Exchange Over a Network

A Suite of Cryptographic Objects Part 2: Implementing Key Exchange Over a Network
http://www.csharptoday.com/content.asp?id=1566
Article Rating: 4
Published on August 30, 2001
Abstract
Throughout the whole history of Cryptography one element has continually created the most difficulty: Key Exchange. In this article, Richard Conway looks at the idea and implementation of Key Exchange over a network, bringing together the elements that we have discussed so far in the previous article (with respect to symmetric key cryptography) and also introduces concepts of public key cryptography. We demonstrate the use of this by creating a secure chat application.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

Throughout the whole history of Cryptography one element has continually created the most difficulty: Key Exchange. In this article, we will look at the idea of Key Exchange. This will bring together all the elements that we have discussed so far in the previous article (with respect to symmetric key cryptography) and also introduce concepts of public key cryptography. We will finally discuss the use of cryptography over a network.

To begin, let's reiterate the use of a session key. In the first part of this article we looked at cryptographic algorithms such as DES and RC2 which use a single key to both encrypt and decrypt data. This enabled us to encrypt file contents or streams of data and then to decrypt this same data. This is clearly very useful, but what if someone were to steal the key? This would render any encrypted items (i.e. possible network streams) vulnerable.

This problem was solved over 25 years ago by Whitfield Diffie and Martin Hellman, and implemented by Rivest, Shamir and Adelman. They created RSA, one of the most widely used encryption algorithms in the world.

Conceptually very different from its counterpart symmetric key cryptography - public key cryptography involves the use of two keys (while RC2 is a common algorithm used in symmetric key cryptography RSA is used in public key cryptography). One of these keys is kept secret (and so called the private or secret key), the other is publicly available.

If Bob gives Alice a public key which is the paired key to his secret key, Alice can now send a message to Bob encrypting the message with Bob's public key. Only Bob will now be able to decrypt the message with his secret key and get back to the original plaintext. Therefore Bob is now able to hand out his public key to any number of people and receive messages from them. This opens up a whole series of problems (the following is mentioned briefly but is beyond the scope of this article - a fuller explanation can be found in Applied Cryptography by Bruce Schneier). It is possible that somebody can now assume Alice's identity and send a message to Bob. If Alice signed all her messages with her secret key then Bob could first use Alice's public key to check the authenticity of the message sender and then decrypt the original message using his secret key to get back to the plain text. In effect Alice has just sent a verifiable digital signature with the encrypted message.

Public key cryptography works by producing two keys where one cannot be derived from the other to encrypt and decrypt messages. In the real world however, this is impractical - the RSA algorithm, which is an implementation of public key cryptography, uses keys of a very large size to encrypt and decrypt data which means that simple messages take a great deal of time to be transferred. This is where the idea of key exchange becomes very important. If Bob sends Alice his public key and Alice then generates a symmetric key then the key can be decrypted by Bob and used to encrypt and decrypt future messages. At this point the key has been successfully exchanged and Bob is the only person able to decrypt the symmetric key while it is being sent (unless someone has stolen his secret key and is listening in on the exchange). This is explained in slightly more detail in the bullet points below:

  • Alice creates an asymmetric key pair (public/private key pair) using the RSA algorithm.
  • Alice keeps her private key secret from the rest of the world.
  • She sends Bob her public key over the internet - Bob receives the public key sent to him by Alice but while in transit Eve also takes a copy of the public key.
  • Bob generates a symmetric key (using the DES algorithm).
  • Bob then encrypts this key with the public key sent to him by Alice.
  • He then sends the encrypted key to Alice over the internet.
  • Eve sneaks a peek at the message but as she doesn't have the private key she will not be able to decrypt the encrypted symmetric key.
  • Alice receives the encrypted symmetric key (which can now be called a session key) and uses her private key to decrypt the message.
  • Alice and Bob now have the session key and are able to communicate securely by encrypting all their messages with it. If Eve intercepts a message in transit she will be unable to decrypt it (not having the session key).

One common use of the above idea of key exchange is SSL (Secure Sockets Layer). Whenever you see https in the address line it refers to the fact that the browser has downloaded a public key (digital certificate) - a key exchange then ensues, normally using the RC2 algorithm that we covered in the last article. Then all further communications are encrypted using this key.

To illustrate all the above examples I've written a very simple secure chat program - based on the Unix version of talk but using key exchange and encrypted streams. This code is not production ready, but is used as an example to illustrate encrypted streams and key exchange.

Creating a project

Firstly create a new Windows Forms project and follow the layout in the diagram below.

The following directives were used.

using System;
using System.Drawing;
using System.Collections;
using System.ComponentModel;
using System.Windows.Forms;
using System.Net;
using System.Net.Sockets;
using System.Security.Cryptography;
using System.IO;
using System.Threading;
using System.Text;

The first five are necessary since we need to use objects from this namespace for our windows forms application. The System.Net and Sockets imports are used because of the need to create sockets and network streams for communication across a network. The System.Security.Cryptography namespace we met in the last article and we'll be discussing some of the more relevant classes we can use later. System.IO for IO and filehandling. System.Threading is imperative and again later we will be taking a cursory look at threading issues and why multiple threads are extremely important in networked applications. System.Text exposes the ever useful Encoding object which enables us to change text formats and storage types.

Screenshot of securechat program

Exporting a key

The first piece of code we'll look at takes place when the user clicks the export button. This creates a key pair (a private key and public key) and then stores the xml representation of the key pair in an xml file in c:\ root.

private void butExportKey_Click(object sender, System.EventArgs e)
{
try
{
RSACryptoServiceProvider rsaExport = new RSACryptoServiceProvider(512);

RSACryptoServiceProvider(512) will create a random key pair 512 bits in length as above. This will effectively contain all the information needed to create a public and private key pair. These keys are invisible to the programmer but as we have seen above all of the information necessary to generate the keys "on the fly" are apparent in the xml.

string sKeyXMLVal = rsaExport.ToXmlString(true);

We can then use this object and format the contents into an xml string.

FileStream fOut = new FileStream("C:\\key.xml", FileMode.CreateNew, FileAccess.ReadWrite);

This string is then written to a file called key.xml as shown above. When the import button is pressed on the form it expects to find this file so that it can deserialize it and recreate the RSA object.

      StreamWriter
            swOut = new
            StreamWriter(fOut);

swOut.Write(sKeyXMLVal);
swOut.Flush();
swOut.Close();

sbSecureChat.Text
            = "Key object serialized to key.xml
            successfully";
}
catch(Exception ex)
{
sbSecureChat.Text
            = "An Error has
            occurred";
MessageBox.Show(ex.Message);
this.Close();
}
}

In effect this is serialization of the RSA key object because the string can be reloaded at any time and the key pair can be derived. This is what the xml looks like:

<RSAKeyValue> <Modulus>
ut4IR+aH/7zDqRit1v1pl0ylIw0tlW7oj8ERcfDtpYq8/xiizY/w2X5DfcHEkSe/NZ1XqSsS4BKiIyhm9RXT4w==
</Modulus>
<Exponent>AQAB</Exponent>
<P>9I5Nu8tjE1mCHS/n572BSJOpwjpqZQAEoG1OBxNDg3M=</P>
<Q>w5yjq4vf1hY1rVErPyfXPJGrxXNF/LYSyx0OEVtXsdE=</Q>
<DP>phmA8mvRfvLsTKac++jaRlEEm8bRPIgmnE+hD20mDcE=</DP>
<DQ>qn7C4Wi/JnihPHl8+kUdgztsvmLk44lHBCZE9rfozaE=</DQ>
<InverseQ>gASLbM5qeVCrE0diNTrH0JLnIGH2arDpORCOHolQ8SE=</InverseQ>
<D>Re3C8Y0uhaBRJB1tKJAKjKkM/zH2Rf6MHwekDJyZgiHlYbMQyHOI+6dfLUf3Dw1scB0/kp9WXxx4UPPbyu1mIQ==</D>
</RSAKeyValue>

This xml looks confusing. It is. The RSA algorithm involves the use of very large prime numbers p and q and generates the two keys from them using the information derived from these numbers. Those who read my last article can see that this has been encoded using Base64 to avoid any of the real data being visible (Base64 strings are distinctive because of the equals sign at the end).

Importing a key

All the code below is written within a try-catch block. The finally block is executed whether the code generates an exception or not. The finally block in all segments of code is just written to allow the button to be greyed out. I have closed the form in the catch block but if the exception is something that can be recovered from, then the exception type can be tested and the code can continue to execute if required.

private void butImportKey_Click(object
            sender, System.EventArgs e)
{
try
{
rsaImport = new
            RSACryptoServiceProvider();
FileStream fIn = new
            FileStream("C:\\key.xml", FileMode.Open,
            FileAccess.Read);
StreamReader swIn = new
            StreamReader(fIn);
rsaImport.FromXmlString(swIn.ReadToEnd());
swIn.Close();

This is the corresponding opposite method to the export method. The key file is read using a stream to read all the content. Then the xml is loaded into the rsa object using the FromXmlString(string) method.

      sbSecureChat.Text
            = "Key object retrieved from key.xml successfully";
butGenerateKeyPair.Enabled = true;

}
catch(Exception
            ex)
{
sbSecureChat.Text = "An Error has
            occurred";
MessageBox.Show(ex.Message);
this.Close();
}
finally
{
butImportKey.Enabled
            = false;
}

Now that the object has it's correct state the other chat program is contacted over network port 4002.

   try
{
mIpe = new
            IPEndPoint(IPAddress.Parse("127.0.0.1"), 4002);
mTcp = new
            TcpClient();
mTcp.Connect(mIpe);
mNet = mTcp.GetStream();
}
catch(Exception
            ex)
{
MessageBox.Show(ex.Message);
this.Close();
}
}

The above code creates an IP Endpoint - which is effectively an IP address and a port specified as a data type(object). IPAddress.Parse("") takes a string a argument and converts it into a corresponding IP address object. A TcpClient() object is used to bind to a listener socket (which we'll describe later) on the other chat program and then a connection is made using the IP Endpoint information. A stream is created using the GetStream() method of the TcpClient object and this allows us to pass continuous messages using stream writes and flushes.

mIpe = new
            IPEndPoint(IPAddress.Parse("127.0.0.1"), 4002);
mTcp = new
            TcpClient();
mTcp.Connect(mIpe);
mNet = mTcp.GetStream();

Using Key Exchange

Let's look at the whole picture now. Until now we've discussed a little about the RSA algorithm, now though we'll use key exchange to show how .NET has inbuilt objects to allow key exchange to take place with the minimal of effort. The windows form chat program is supposed to be able to communicate over a network. It can also be used in loopback to test the concept of key exchange. The IP address 127.0.0.1 is the local IP address that refers to the local machine (some of you will recognize this from local testing of ASP pages by typing wither localhost or 127.0.0.1 into the address bar of your web browser).

This program was written using a networking approach of listeners and client connections via sockets. There are number of reasons I chose this approach. The first is simply that this is archetypal. There are a multitude of chat clients that have been written in java to show how key exchange can be used to facilitate secure communication - I wanted to show that it was just as easy - in fact easier - to write this in C#. There are also a number of non-secure chat programs which use simple client-server chat. There is code online to demonstrate client-server chat using C# remoting technologies but none using basic socket programming. Another strong reason for taking a very simple approach and using two clients instead of many clients and a server is the way that the formatting objects work. The xml file will allow the key exchange formatting objects to derive both the public and private key information which is not traditional. One party usually has both the private and public key and the other only the public key therefore using this approach it is technically impossible to send the public key alone to another party - effectively both keys will be sent initially so both parties must trust one another to withhold key information from other parties to avoid illicit impersonation and possible interception of messages. Therefore in this application key exchange will take place as it normally does but each party initially has both the public key and private key. This doesn't affect key exchange but does make the process of secure exchange of information less secure for obvious reasons.

private void butGenerateKeyPair_Click(object sender, System.EventArgs e)
{
byte[] session_key = { 10, 20, 30, 40, 50, 60, 70, 80 };
desKey.Key = session_key;
RSAOAEPKeyExchangeFormatter r = new RSAOAEPKeyExchangeFormatter(new RSACryptoServiceProvider());

In the above example we have defined a DES key. This is not in fact a true DES key as the DES algorithm generates a key based on certain criteria, but nevertheless we can use this key to transfer as a session key. The RSAOAEPKeyExchangeFormatter object does all the work for us using the public key to encrypt our makeshift session key. Note that to generate a random DES key, use DESCryptoServiceProvider.GenerateKey() and then use the Key property to retrieve the key value.

r.SetKey(rsaImport);

For both chat clients we use the SetKey(AsymmetricAlgorithm) method which can take RSA (or any other Asymmetric algorithm). rsaImport is a fully populated object containing the key pair.

   IPEndPoint ipE = new
            IPEndPoint(IPAddress.Parse("127.0.0.1"), 2002);
TcpClient t = new
            TcpClient(ipE);
NetworkStream n = t.GetStream();

Then an IP endpoint is created in loopback on port 2002 and the TcpClient object created. We get a stream object with which we can keep sending data and eventually we will write the encrypted contents to this network stream.

byte[] bStreamExchange = r.CreateKeyExchange(session_key);
n.Write(bStreamExchange, 0, bStreamExchange.Length);
}

The key is actually encrypted using the public key using the CreateKeyExchange method of the RSAOAEPKeyExchangeFormatter object. This doesn't necessarily have to be a session key, it can be any data. However, this method will return a byte array which enables us to write to the network stream.

Note. In the above snippet of code it is assumed that you have already clicked the import key button - in fact the key exchange button is disabled if the import key button has not yet been pressed.

Listening via threads

Rather briefly we'll touch on the subject of threads.... A program written in VB 6 would contain only 1 thread (called main), this is the entry point to the application. There are times however, when it is necessary to have more than 1 thread. If an IO operation is occurring then the program will effectively be locked up with only one thread. Therefore we write code which takes advantage of multithreading capabilities. All .NET languages support multithreading. We have had to use threads with the chat client to allow a separate thread of execution to listen on a certain port without affecting the needs of the rest of the program.

Thread tdListen = new Thread(new
            ThreadStart(this.ThreadTcpListener));
Thread tdKeyExchange = new
            Thread(new
            ThreadStart(this.ThreadKeyExchangeListener));
tdKeyExchange.Start();
tdListen.Start();

The above code shows how it is possible to create a thread. Simply create the thread object and pass this thread object a ThreadStart object. ThreadStart 's constructor is a delegate which enables us to essentially create a "safe" pointer to a method. A delegate is a new type in C# which enables programmers to pass a method to another method as an argument (i.e. effectively allows a method to be referred to in the same way as a primitive). They are exceptionally useful and are used naturally within .NET for Event Handling and Thread operations (described below). The method passed to the ThreadStart delegate must return void and also not take any arguments of any type. The two arguments to ThreadStart are method names which exist in the current object. The start method is called on each of these two threads to allow them to execute. A thread exists so that many operations within many programs can be effectively run simultaneously. Each thread has a quantum (a unit of time - which is normally very short) and must switch (context switch) to another thread when that time has elapsed. In this way the processor (which can only execute one set of instructions at once) can then "swap in" different pieces of code to execute. Threading is not covered in great detail here as many of the problems and implementations associated with threads are not relevant to this example. The above code is called from the constructor of the form.

void
            ThreadKeyExchangeListener()
{
try
{
TcpListener
            tcpListen = new
            TcpListener(2003);
tcpListen.Start();

Socket s =
            tcpListen.AcceptSocket();
int iLenKeyExchange =
            s.Receive(bTcpData, 0, bTcpData.Length,
            SocketFlags.None);

RSAOAEPKeyExchangeDeformatter d = new
            RSAOAEPKeyExchangeDeformatter(rsaImport);
desKey.Key =
            d.DecryptKeyExchange(bTcpData);
s.Close();
butImportKey.Enabled
            = false;
}
catch(Exception
            e)
{
MessageBox.Show(e.Message);
this.Close();
}
finally
{
Thread.CurrentThread.Abort();
}
}

This method effectively runs in a different thread. Here it creates a TcpListener object on a particular port and then tells the listener to start listening using the Start() method. The AcceptSocket() method is called which returns a socket (which will represent a socket - a connection on that particular port). This socket once connected will send the key data encrypted. The method will stop executing and trigger an event later when a client requests a socket connection.

      int
            iLenKeyExchange = s.Receive(bTcpData, 0, bTcpData.Length,
            SocketFlags.None);

RSAOAEPKeyExchangeDeformatter d = new
            RSAOAEPKeyExchangeDeformatter(rsaImport);

The Receive() method will receive any data sent in the form of a byte array. In this case we know that it is going to be the key data so we can load rsaImport (which is the RSA object already having key state data) into the RSAOAEPKeyExchangeDeformatter object.

desKey.Key = d.DecryptKeyExchange(bTcpData);

We use the DecryptKeyExchange(byte[]) method passing it the byte array. This will now use the private key to decrypt the stream data allowing us to have access to the underlying data (the DES key). This key value is then set to the Key property of the DESCryptoServiceProvider object enabling us to encrypt text. The two chat programs now have the matching key.

      s.Close();
butImportKey.Enabled
            = false;
}
catch(Exception
            e)
{
MessageBox.Show(e.Message);
this.Close();
}
finally
{
Thread.CurrentThread.Abort();
}
}

As we no longer need this thread (since we have successfully transferred the key) we can destroy it. This is done using the code in the finally block. Thread.CurrentThread.Abort() allows us to abort the thread we are using and free up the resources that it has locked up.

void
            ThreadTcpListener()
{
try
{
byte[] bTcpData = new
            byte[4096];
TcpListener tcpListen = new
            TcpListener(4003);
tcpListen.Start();

The above section of code represents the second thread where another TcpListener object is created and made to listen on network port 4003.

string sTcpData = String.Empty;
/*while(!isKeyLoaded)
{
sbSecureChat.Text = "Session key not loaded yet";
Thread.Sleep(TimeSpan.FromSeconds(3));
}*/

while(true)
{
Socket s = tcpListen.AcceptSocket();

A socket is created within the while loop and the inner loop sits and waits until data is received (this is why a separate thread is needed here - as otherwise it would stop the rest of the program from executing until it had received data).

         while(0!=s.Receive(bTcpData,
            0, bTcpData.Length, SocketFlags.None))
{
byte[] key =
            desKey.Key;
byte[] iv = desKey.IV;
MemoryStream mStream1 = new
            MemoryStream();
mStream1.Position = 0;
ICryptoTransform
            iCrypto = desKey.CreateDecryptor();
iCrypto1 =
            desKey.CreateDecryptor(key, iv);

CryptoStream cCrypto = new
            CryptoStream(m, iCrypto,cCrypto1 =
new CryptoStream(mStream1,
            iCrypto1, CryptoStreamMode.Write);
cCrypto.Write(bTcpData, 0,
            iReceive);
cCrypto.FlushFinalBlock();
cCrypto.Close();
byte[]
            bTemp = new Byte[4096];
int iTemp = m.Read(bTemp, 0,
            bTemp.Length);
sTcpData = Encoding.Default.GetString(bTemp, 0,
            bTemp.Length);
lstChat.Items.Add(sTcpData+"\n");cCrypto1.Write(bTcpData,
            0,
            iReceive);
cCrypto1.FlushFinalBlock();
cCrypto1.Close();
sTcpData
            = Encoding.Default.GetString(mStream1.ToArray());
lstChat.Items.Add("RICHARD: "+sTcpData);

A MemoryStream object is created followed by an ICryptoTransform interface and these are passed as arguments to the CryptoStream object. Anything written to the CryptoStream will be effectively written to the stream ( MemoryStream) given as an argument to the CryptoStream constructor. Once encrypted (or decrypted in this case) mStream1.ToArray() then returns a byte array of the contents that have had the key applied to it.

         }
//s.Listen(5);
sTcpData
            = String.Empty;
}
}
catch(Exception
            e)
{
MessageBox.Show(e.Message);
this.Close();
}
finally
{
Thread.CurrentThread.Abort();
}
}

For a thorough explanation of the CryptoStream object please refer to part 1 of this article. Suffice it to say that the data is decrypted and then converted from a byte array to a string where it is added as a listitem to the listbox on the form.

try
{
MemoryStream mStream = new
            MemoryStream();
ICryptoTransform iCrypto =
            desKey.CreateEncryptor();
CryptoStream cCrypto = new
            CryptoStream(mStream, iCrypto, CryptoStreamMode.Write);
byte[]
            bTemp =
            Encoding.Default.GetBytes(txtChat.Text);
cCrypto.Write(bTemp, 0,
            bTemp.Length);
cCrypto.FlushFinalBlock();
cCrypto.Close();
mNet.Write(mStream.ToArray(),
            0, mStream.ToArray().Length);
}
catch(Exception
            ex)
{
MessageBox.Show(ex.Message);
this.Close();
}

The above section of code is the opposite to what we have just discussed. This code executes when the user presses the send message button. Whatever, at this time, is in the textbox is then written to the cryptostream object and then later on written to the networkstream.

Conclusion

For those java or C++ programmers that have used networking and sockets in the past, you will notice that this is not an archetypal example. We would usually have a network client and a network server, or many clients and a network chat server in much the same way as IRC works. This example can be modified to create this effect allowing for key exchange but it doesn't lend itself well to this as the public key (and private key) or in this case the xml file containing the key object data has to be given to all the clients. This program can be used across a network or used in loopback as I have done.

In the case of loopback it is necessary to compile two versions of the chat program that listen on different ports. I used 2002 and 4002 on one and 2003 and 4003 on the other. This will avoid an exception as only one tcplistener object can be created per port per machine. A compiled version of both chat clients and their source as well as a sample xml key file is available for download with this article.

To use the program in loopback start each chat client.

  • generate a copy of the key.xml file in the C: root by pressing the export key button on one of the clients.
  • Next click on the import buttons on both clients. Both buttons on the chat clients should now be disabled.
  • On either of the chat clients click the button marked "Generate key pair and use key exchange for stored des key ". The button on the chat client in focus should now be disabled and so should the other client's button.
  • On either form type in the textbox any text and press send message.

The message should then appear in the listbox on both forms. The communication between the forms should be instantaneous. Behind the scenes though the data is being encrypted once sent from a network port to a different network port. This can be verified if a packet sniffer - such as MS Netmon is used to analyze the TCP packet data which will only show the encrypted output (either this or debugging the code line by line to see where the information is transformed and transformed back again). Note: When the files from the example application are unzipped make sure that key.xml is in the root of drive C: or generate another copy of this file in the C: root by pressing the export key button on one of the clients.

C#Today - © 2002 WroxPress Limited


Cryptographic Objects in c# pt. I

Cryptographic Objects in c# pt. I
http://www.csharptoday.com/content.asp?id=1640
Article Rating: 4
Published on August 23, 2001
Abstract
The days where developers have to fiddle about with complex APIs and byte lengths of plain text are now over, at least for the .NET developer. Previously, all documentation for the Crypto API was in C, and VB developers who may well have needed to use the Crypto API had to struggle working out API declarations for use within their VB/COM applications. The .NET framework provides a comprehensive library of cryptographic classes which can be used for anything from hashing passwords, encrypting file contents, to implementing symmetric key exchange sessions over the internet. In the following two articles, Richard Conway shows us the use of these cryptographic objects and simplifies some crypto terminology.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

The days where developers have to fiddle about with complex APIs and byte lengths of plain text are now over, at least for the .NET developer. Previously, all documentation for the Crypto API was in C, and VB developers who may well have needed to use the Crypto API had to struggle working out API declarations for use within their VB/COM applications.

The .NET framework provides a comprehensive library of cryptographic classes which can be used for anything from hashing passwords, encrypting file contents, to implementing symmetric key exchange sessions over the internet. In the following two articles we'll study the use of these cryptographic objects and simplify some crypto terminology.

Hashing functions

The following program has two classes: one to hash passwords and the other to encrypt files or streams. Message digests are implemented in .NET in two forms: MD5 and SHA1. Both are equally useful, MD5 returns 16 bytes based on any input content and this is unique for that content. SHA1 returns 20 bytes. These digests can be used as follows:

  • Verifying the contents of a file being transferred by computing the digest based on the file contents - in this way if the file contents have changed there will be a discrepancy which will be apparent in the digest calculated previously to the current digest.
  • Using a username/password to provide a secure login mechanism. The user would type these two into a login prompt and the concatenation of the two words would be hashed and compared to a stored hash value to determine authentication.

Hashing is especially useful for controlling access to resources on a website. As such ASP.NET can be used in conjunction with the C# objects below to create a user repository in a database. First the user enters a username and a password. These two strings are then concatenated and the result will be passed to an MD5 or SHA1 object. A unique hash would be computed and this hash could be compared to one that was stored earlier in the database when the user account was created. The simplicity of this scheme is that the user is the only one who needs to know the password. A malicious person viewing the contents of the database will see the username with the password stored next to the username as a hash - they will never know more than 50% of what is required to use the account. In effect all details can be hashed including the credit card number with the username for secondary access if someone has forgotten their password.

After a successful database hash is matched with one computed on the fly then a token can be given which allows access to certain resources on the website. In effect this is similar to how Microsoft have implemented hashing from within the web.config page (which is one of the two key pages now to any ASP.NET site along with the global.asax).

This is a section of a web.config page:

<configuration>
<security>
<authentication mode="Cookie">
<cookie cookie="token" loginurl="login.aspx" decryptionkey="autogenerate">
<credentials passwordformat="SHA1">
<user name="newuser" password="UEHRYF83478FHKS6RT45" />
</credentials>
</cookie>
</authentication>
<authorization>
<deny users="?" />
</authorization>
</security>
</configuration>

The section that interests us at the moment is the line:

<credentials passwordformat="SHA1">

This means that all passwords will have SHA1 hashes computed and stored for them. The token discussed in the previous paragraphs is given to the web user in the form of a cookie. We won't be discussing this here but the CookieAuthentication object has static methods that allow it to control the login process.

The hashing object wrappers below can be used to gain greater clarity in the hashing process and possibly to replace use of the web.config to do the same. You can, as you will see below, hash any number of things (including file contents); hashing multiple strings can be done by just using string concatenation followed by computation of the hash itself.

Encryption

The second class is used to generate symmetric keys. The algorithms used are DES, Triple DES and RC2 - these keys can be used to encrypt any volume of data. A key is generated by any of the 3 above algorithms and is then used to encrypt some plaintext. The result is called cipher text. Symmetric encryption can be used for the following:

  • Encrypt the contents of a file
  • Encrypt the contents of stream (i.e. a data stream being transmitted over the internet).

The above key algorithms are just used to generate the key itself. The encryption is done by XORing the key bits to the bits of the plaintext. XOR is used as the key can be XORed again to the cipher text to return to the original plaintext. Cipher block chaining is used to add the bits of the previous cipher text to the plaintext following. In this way there will be no repetition in the cipher text and the key won't be able to be derived by looking at the similarities within the cipher text itself.

From the diagram you can see that the plaintext is encrypted in the first block (denoted by gray) and the resulting cipher text is Xored with the plaintext from the next block (denoted by yellow) before encryption.

Xor is a binary operation that can be done on a bit. A Xor truth table would look like the following:

Input bits | Result
00 | 0
10 | 1
01 | 1
11 | 0

Unlike a conventional Or operation it produces true when either input bit is 1 but not both. As all text characters can be represented in ASCII (an 8 bit notation) it is the bits in the ASCII representation of the character that are XORed with the bits of the key to provide the results given in the truth table above. As you can see above, if a one bit key (1) encrypts a one bit plaintext 1 the result would be 0. To decrypt this all you would need to do is the XOR operation again so 0 Xor 1 would be 1 thus arriving at the original plaintext.

To begin the project a reference must be set in the project pane to the System.Security.Cryptography namespace.

Then to import the Crypto library in the System.Security.Cryptography namespace to avoid prefixing the cryptographic objects:

using System.Security.Cryptography;

This namespace has a wealth of uses. Some we will cover in this article and others we'll save for the future. It contains implementations of RSA which will be used in part II to create a key-exchange program. Most of you will be familiar with the internet use of SSL. Well, effectively SSL uses Private/Public key cryptography, i.e. a digital certificate (issued by a certified authority) to allow a web browser to generate a symmetric key and send data securely to the webserver (so that only the browser and server know the key). This whole processor is called key-exchange and objects exist in the Cryptography namespace to allow for the formatting of these keys and implementations of the key-exchange. Some support classes exist to allow for the generation and maintenance of keys also, including classes which allow changeable bit padding for encryption and other classes that allow one to customize encryption for the task at hand.

We will also have to reference the namespace System.Text specifically because we will need to use the Encoding objects static methods to convert between strings and byte arrays. We will also refer to System.IO since we will be reading and writing to files.

using System.Text;
using System.IO;

These base classes can now be accessed without the need for the namespace prefix e.g.

File.Exists("temp.txt");

Or

FileStream f = FileStream("temp.txt", FileMode.Create, FileAccess.Write);

These two classes are found within the System.IO namespace and are invaluable for reading and writing byte array to files. As such they are used throughout the code to read from plaintext files and to write out to cipher text files.

Hashing class

public class
            HashLoginCredentials
{
public enum HashTypeDef { MD5, SHA1
            }

bool isSha = false;
string[] gCredentials =
            {};
string plain = String.Empty;
byte[] hashed = {};
byte[]
            bHashResult = new byte[21];

public
            HashLoginCredentials(HashTypeDef htd, params string[]
            sCredentials)
{
if(htd==HashTypeDef.SHA1)
{
isSha =
            true;
}
gCredentials = sCredentials;
}
public string
            Hash(bool
            isBase64)
{
if(isSha)
{
SHA1CryptoServiceProvider sha1 =
            new SHA1CryptoServiceProvider();
for(int i = 0;
            i<gCredentials.Length; i++) plain += gCredentials[i];
byte[]
            bSha1Hash =
            Encoding.ASCII.GetBytes(plain);
//sha1.TransformBlock(bSha1Hash,
            0, bSha1Hash.Length, bHashResult, 0);
bHashResult =
            sha1.ComputeHash(bSha1Hash);
hashed =
            bHashResult;
}
else
{
MD5CryptoServiceProvider md5 = new
            MD5CryptoServiceProvider();
for(int i = 0;
            i<gCredentials.Length; i++) plain += gCredentials[i];
byte[]
            bMD5Hash =
            Encoding.ASCII.GetBytes(plain);
//md5.TransformBlock(bMD5Hash, 0,
            bMD5Hash.Length, bHashResult, 0);
bHashResult =
            md5.ComputeHash(bMD5Hash);
hashed =
            bHashResult;
}
if(isBase64)
{
return
            Convert.ToBase64String(hashed);
}
else
{
return
            Encoding.ASCII.GetString(hashed);
}
}
}

This -is a very simple wrapper class for hashing but immensely useful. As mentioned before any text can have a unique message digest. The two implementations in .NET used here are MD5 and SHA1 which are the most common. If we take a more detailed look at the cryptographic hashing objects themselves we begin with the SHA1CryptoServiceProvider object, we can use the ComputeHash(byte[]) method to put any amount of data into the hashing object. This method returns the value of the hash in the form of a byte array. In this implementation a string is returned from this hash wrapper object instead of a byte array, simply because this can be written to a database table very easily for a particular user or in an XML file as I've done in the ASP.NET implementation of this object.

Convert.ToBase64String(hashed);

The above line of code is very useful. A static method of the Convert object enables us to pass a byte array and retrieve a Base64 encoded string. Base64 defines a character set of upper and lower case alphabet characters, digits 0-9 and a plus sign, a slash and a space. This means that if any piece of encrypted text or hashed text is converted into this from ASCII text it will always be viewable and printable. A Base64 string always has an equals sign at the end (to show that it is Base64). See the code below:

string temp_test = String.Copy("ABCDE&^%$!@");

which produces as a Base64 output:

QUJDREUmXiUkIUA=

Using the encoding object is also very useful. It can be used to convert characters to ASCII from Unicode or vice-versa. It can be used to get the default system encoding and translate between strings and byte arrays or char arrays etc.

Encoding.ASCII.GetString(hashed);

Encryption classes

There are three internal classes which map onto the 3 key algorithms discussed above; one of them looks as follows:

internal class GenerateDesKey :
            SymDefKey
{
private byte[] desIVVal = {
10, 20, 30, 40, 50,
            60, 70, 80
};
byte[] desKeyVal = {};

public
            GenerateDesKey()
{
DESCryptoServiceProvider desKey = new
            DESCryptoServiceProvider();
desKey.GenerateKey();
desKeyVal =
            desKey.Key;
desKey = null;
}
public byte[] Key()
            {
//Returns key value
return desKeyVal;
}
public byte[]
            IV()
{
//Returns IV Value
return
            desIVVal;
}
}

The value IV is used to store an Initialization Vector, which is used to make it a little more complicated to break the encryption. The IV is added to the plaintext block to be encrypted using CBC making it slightly harder to crack.

This is a simple class which implements a common interface: SymDefKey so that Key() and IV() methods can be defined on all these wrapper classes. The DESCryptoServiceProvider is used here to create a DES key (DES keys are normally 56 bits long). As can be seen the GenerateKey() method is used to generate a key within the DESCryptoServiceProvider object and it can be retrieved through the Key property. IV's are only really used for DES now since key lengths are small and DES keys have been found via brute force and decryptions have followed. With Triple DES (DES3) we have 168 bit keys (or three 56 bit keys) which are used to encrypt, decrypt and re-encrypt the plaintext - Essentially this is an extension of DES using the same key sizes but to perform operations on the plaintext 3 times. This is the most common key used and is far more secure than DES. However, to use this the high encryption pack needs to be installed on Windows. This can be found on the Microsoft website at http://www.microsoft.com/windows2000/downloads/recommended/encryption/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq/.

case KeyEncryptAlgorithm.DES:
keyAlg = (SymDefKey) new GenerateDesKey();
keyTemp = KeyEncryptAlgorithm.DES;
break;
//Create an instance of internal DES3key class
case KeyEncryptAlgorithm.DES3:
keyAlg = (SymDefKey) new GenerateDes3Key();
keyTemp = KeyEncryptAlgorithm.DES3;
break;
//Create an instance of internal RC2 class
case KeyEncryptAlgorithm.RC2:
keyAlg = (SymDefKey) new GenerateRC2Key();
keyTemp = KeyEncryptAlgorithm.RC2;
break;

The above segment of code shows that the class defined above casts to the interface type that it implements (SymDefKey) - all of these wrapper classes implement the methods Key and IV which enable the objects which we'll talk through later on to retrieve and use to get do the actual work (encryption or decryption). This code exists in the object constructor.

A boolean flag is set to enable the object to work out whether the user has passed a file name (which must exist) or a string - It will check first if the file exists, if not then it will assume that the user has passed a string needs to be encrypted. This is set after the file name is tested for existence.

If the file does exist then the following code is executed:

if(!fileNameOrTextToEncrypt.EndsWith(".enc"))
{
//Calculate
            output name of encryption file
outName =
            fileNameOrTextToEncrypt.Substring(0, fileNameOrTextToEncrypt.Length
            - 3)+"enc";
}
else
{
//Calculate output name of
            decrypted file
outName = fileNameOrTextToEncrypt.Substring(0,
            fileNameOrTextToEncrypt.Length - 3)+"tmp";
bToBeEncrypted =
            false;
}

This forms part of the Encrypt() method. It checks to see if the filename being passed ends with .enc. If so then the method will create an output file of the same name in the same directory ending with .tmp which will contain the decrypted output from the .enc file.

ReadKeyFileContents(ref keyVal, ref IVVal, outName);

Assuming the latter, then the program will be unable to proceed and will generate an exception if it is unable to find a .key file in the current directory containing a key and an IV value. This ReadFileContents() method resides within a try-catch block ensuring that the file will have to be present for decryption.

Using ICryptoTransform interfaces and Cryptographic objects

In the following code we will see how the actual encryption is done. Formerly (in beta 1) we would have defined a class which implemented ICryptoStream to allow us to write to any stream - this could then be custom defined. The authors of the .NET Crypto libraries have simplified this even further in Beta 2 by allowing ICryptoTransform interfaces to be created that represent either encryption or decryption. This contains all the key and IV information.

try
{
//Here we create the decryption
            objects but if the key file doesn't
//exist, automatically throw
            an exception again

outName =
            fileNameOrTextToEncrypt.Substring(0, fileNameOrTextToEncrypt.Length
            - 3)+"key";

ReadKeyFileContents(ref keyVal, ref IVVal,
            outName);
ICryptoTransform streamDecrypt =
            d.CreateDecryptor(keyVal, IVVal);

CryptoStream csDecrypt =
            new CryptoStream(fOut, streamDecrypt, CryptoStreamMode.Write);

len = 0;
lenCount = 0;
byte[] bOutputDecrypt = new
            byte[4096];
while (lenCount < fileLength) {
len =
            fIn.Read(bContents,0,4096);
csDecrypt.Write(bContents,0,len);
lenCount
            = lenCount +
            len;
}
csDecrypt.FlushFinalBlock();
csDecrypt.Close();
}
catch(Exception e)
{
throw e;
}

Some of the above file handling code might be familiar to .NET aficionados. The CryptoStream object is created and takes as its arguments the ICryptoTransform interface (which is our Decryptor and contains the key and IV that we need). It also needs a valid stream object. In the above code it is a FileStream although this is because we are going to write to a file. In a later piece of code we use a MemoryStream in the same place to write to memory and retrieve our encrypted output as a byte array. However, it just needs to be a stream which means that it can be written to any networking stream object too. In the while loop we read from one file and then write to the CryptoStream (effectively in this case another file). The Stream can be written to any number of times, then FlushFinalBlock() is called followed by Close(), which will flush the stream buffer. Close releases all handles (in this case to files).

MemoryStream mStream = new
            MemoryStream();
mStream.Position =
            0;

if(!bToBeEncrypted)
{
ICryptoTransform streamDecrypt
            = d.CreateDecryptor(keyVal, IVVal);
CryptoStream csEncryptString
            = new CryptoStream(mStream, streamDecrypt,
            CryptoStreamMode.Write);
csEncryptString.Write(Encoding.Default.GetBytes(fileNameOrTextToEncrypt),
            0,
Encoding.Default.GetBytes(fileNameOrTextToEncrypt).Length);

csEncryptString.FlushFinalBlock();
csEncryptString.Close();
}

As can be seen from the above implementation a MemoryStream can be used in place of a FileStream. This exemplifies the idea that any stream can be used. A MemoryStream is useful in this effect as an area of memory can be written to as a stream and later discarded when the contents are retrieved into an array or string.

private void CreateKeyFile(File
            f)
{
string outNameKey = f.FullName.Substring(0,
            f.FullName.Length - 3)+"key";
FileStream fKey = new
            FileStream(outNameKey, FileMode.Create,
            FileAccess.Write);
StreamWriter sKey = new
            StreamWriter(fKey);
sKey.WriteLine(Convert.ToBase64String(keyAlg.Key()));
sKey.WriteLine(Convert.ToBase64String(keyAlg.IV()));
sKey.Flush();
sKey.Close();
fKey.Close();
}

After the first encryption is done then the key file is written in the same directory as the plaintext and the cipher text output. The above method is used to write a Base64 encoded version of the key and the IV to a file. This can again be extended to use a key file which contains multiple keys.

I've written this article into a .dll project which can be used generically within a project. The following sample code (ConsoleClient.cs) explains how to use the objects contained therein:

try
{
//First file encryption or decryption
CryptoWrapper.EncryptStringOrFile e = new EncryptStringOrFile(EncryptStringOrFile.KeyEncryptAlgorithm.DES,
"C:\\temp.enc");
string temp = e.Encrypt();
//Then hash passwords ans usernames
CryptoWrapper.HashLoginCredentials h = new HashLoginCredentials(HashLoginCredentials.HashTypeDef.SHA1,
"username", "password");
string hash = h.Hash(true);
//Then encrypt a stream
CryptoWrapper.EncryptStringOrFile e1 = new EncryptStringOrFile(EncryptStringOrFile.KeyEncryptAlgorithm.DES,
"EncryptOrDecryptThisText");
string temp1 = e1.Encrypt();

Console.WriteLine("filename is: "+temp);
Console.WriteLine("hash is: "+hash);
Console.WriteLine("Encrypted text is: "+temp1);
Console.WriteLine("\r\nPress any key to exit...");
Console.Read();
return 0;
}
catch(Exception e)
{
Console.WriteLine(e.StackTrace);
return 0;
}

Here is an example of the encrypted output from the above main() method:

In the first line "temp.enc " is passed to the EncryptFileOrString object and then the encrypt method is called which looks for the key file in the respective directory and decrypts the file into "temp.tmp ". It returns the name of the output file is successful.

In the second operation a HashLoginCredentials object is created and it is instructed to use a SHA1 hash, username and password is passed into this constructor and the resultant hash is returned.

In the final operation a DES algorithm is used to encrypt some plaintext. The cipher text is returned to the console in the form of a string. The only prerequisite is for a file called C:\temp.key to exist so that key information can used to encrypt and decrypt plain and cipher text respectively.

A test harness for the .dll has been provided in ASP.NET. After extracting the .zip file, typing http://localhost/CryptoWrapper/login.aspx?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq (assuming you have installed a webserver) should present the following screen:

There is a readme.txt file to accompany the test harness which should be read prior to using the page. However, code from the test harness can be easily adapted into an application to allow, amongst other things, logins and password hashing from which can be stored and retrieved from an XML file and also applications that allow files to be uploaded but encrypt them on the server to allow a greater level of security.

Conclusion

We have see how to store and retrieve encrypted data from a filesystem using the examples of an encrypted file in the C:\Temp directory and encrypted passwords. In the next article in this series, we will look at encryption of data over a network.

C#Today - © 2002 WroxPress Limited


Web Site Error Logging and Reporting System

Data Access, ADO.NET

Data Access, ADO.NET (Sort by Name)

.NET Data Access Primer - Part 1
.NET Data Access Primer - Part 2
.NET Data Access Primer - Part 3
.NET Data Access Primer - Part 4
.NET Web Data ToolKit
A .NET Data Provider
Building a Database-Driven Guestbook with C# and ASP.NET
Building an Address book application using Windows Forms and ADO.NET
Building an ASP.NET Mailing List
Creating Data Bound Controls

Creating Data-Driven Enterprise Applications using C# and SQL Server
Developing a Universal Data Access Layer leveraging ADO.NET, C# and Factory Design Pattern
Effective Data Access in C#
Mapping Windows Forms to A Relational Database - Part 1
Mapping Windows Forms to A Relational Database - Part 2
Working with Web Data in C#


Creating Data-Driven Enterprise Applications using C# and SQL Server

Creating Data-Driven Enterprise Applications using C# and SQL Server
http://www.csharptoday.com/content.asp?id=1955
Article Rating: 5
Published on October 28, 2002
Abstract
This case study by Mark Plummer describes how to go about designing and implementing the data tier of an enterprise application using C# and SQL Server. It concentrates on a single database server scenario (although multiple server implementation is also looked at along the way), and bring together the new features of .NET with proven methodologies to create a high performance, scalable transactional data tier.
Case Study

This case study describes how to go about designing and implementing the data tier of an enterprise application using C# and SQL Server. It will concentrate on a single database server scenario (although multiple server implementation is also looked at along the way), and bring together the new features of .NET with proven methodologies to create a high performance, scalable transactional data tier.

Introduction

Most enterprise applications interact with a database. Very few of these enterprise applications are the same, but the database interaction is strikingly similar in each case. Yet many of these applications suffer from poor performance, low scalability, inability to use transactions correctly, and generally poor design. This case study will set out to show the reader what developers have been doing wrong, and what can be done to remedy these mistakes.

Several methodologies have emerged in recent years, but it is only fairly recently that one has emerged on top. Unfortunately, this is only described in terms of complex VC++, so I would like to 'convert' the relevant ideas to C# and the .NET environment.

What the reader will gain from this piece is an easy-to-implement methodology that can be applied over and over to reap the rewards of performance, scalability, and transactional support, and exert a sense of uniformity over his or her data tier.

System Requirements

SQL Server 2000 and the .NET framework are mandatory. A good C# editor like Visual Studio .NET is highly recommended. A good understanding of ADO or ADO.NET would help, although a lot will be explained as we go along. As this is a C# case study, an intermediary knowledge of C# is required. A general understanding of databases and database theory is highly recommended.

Defining The Problem

Almost every data tier that I have looked at has been designed and implemented differently, and that is from a methodology that is almost eight years old. With the recent emergence of .NET and its new way of approaching data access, the amount of permutations of data tier design and implementation is likely to be extremely high. These permutations will lead to inconsistencies in performance, scalability, transactional support, and integration with other systems

The Business

When a firm wants to create, extend or update just about anything in IT, they will generally make use of a business analyst. The business analyst's role is to extract the requirements of the business and business users and present them in a structured and logical manner. Although a lot of them are, a business analyst is not required to be highly technical. Add to that the fact that most business users are not technical experts and do not think about their job in a technical or pedantic manner like techies do, and you create a potential situation where you have a logical business document going straight to the programmers.

Add to this the fact that many concepts common to business people are either unknown or incomplete to the technical people. For example, assume the business states that: "All employee's addresses are to be recorded". Does this mean that an employee can have more than one address? How many fields does the business want in the address? What are their names? Is the post code a required field? If so, what happens if an employee does not know his post code? Can he save the rest of the details in the interim? As you can see, even a simple concept such as an address can have many questions attached to it from a technical perspective that can easily go unnoticed from a business perspective. Thus:

"All employee's addresses are to be recorded",

can easily turn into (probably after many versions and iterations):

Definition: Address

An address records a unique place in the world and consists of the following fields:

Address Name (50 character Unicode string)
Flat Number (10 character nullable Unicode string - [E.G. 52A])
House or Flat Name (50 character nullable Unicode string)
Street Number (10 character nullable Unicode string - [E.G. 52A])
Street Name (50 character Unicode string)
Locality (50 character nullable Unicode string)
City (50 character Unicode string)
County (50 character nullable Unicode string)
Country (50 character Unicode string)
Post Code (10 character Unicode string)

Rule: Employee addresses

Employee's addresses are to be captured as soon as their postcode and street name
are known. An employee may have zero, one, or more addresses and each must be named.
(e.g Residential Address, Postal Address, etc)

What is often lacking in these situations is a technical analyst - someone who can take the business analyst's documents and translate it into technical documents for the IT department. This technical analyst can save many round trips from the IT department to the business users and back again by accompanying the business analyst where appropriate. Better still is to have a person who can play both roles.

What I am trying to demonstrate here is the need to be pedantic and thorough during the design phase as it will not only save you money, but it will save you time as well.

The Different ways of Storing and Retrieving Data

Communication with a database can be broken down into two steps. Firstly, you send some sort of request to the database, and secondly, you optionally get some data returned to you. The data that is returned (apart from any error codes) can be a combination of a return value, a set of zero or more output parameters, and a set of zero or more result sets (also known as record sets, record arrays, data sets, or tables).

In the past you had to explicitly manage a raw memory buffer using C/C++ via the ODBC and later the OLEDB libraries to get information to and from the database. Microsoft has, during the years, provided wrappers for this in the form of DAO, RDO, ADO, and now ADO.NET, but in essence they are merely handling the buffers behind the scenes. You achieve ease of use in return for a minimal loss of performance.

The fastest way to get data back from a database is by using output parameters and/or the return value. The return value is simply a mandatory integer output parameter that accompanies each and every stored procedure. It has historically been used to return error codes to the caller, or the number of rows affected by the last database operation. It is a highly underused parameter; it is either being ignored by developers as an error code, or it is being ignored when returning the number of rows affected by the last database operation. (If you wish to increase performance you can tell SQL Server not to keep returning the number of rows affected - and hence make use of the return value yourself - by using SET NOCOUNT ON. To reverse this, use SET NOCOUNT OFF.)

Since the return value is a 'built-in' parameter, it should offer a slight performance advantage over an output parameter. Otherwise, the return value and output parameters of a stored procedure can be compared to the return value and output parameters of a normal function in terms of performance. The disadvantage of using the return value, however, is that it is not plainly obvious to spot it in the data tier or the header of the stored procedure.

Returning data in the form of a result set is considerably slower than using output parameters. (I have seen people use multiple result sets to get back two integer values, and they wondered why their application was slow!) Later in this case study I will provide some metrics (from MSDN) to show the performance comparisons of the various ways of retrieving data.

If you need to perform several operations and only have those operations saved permanently if all of them succeed, you will need to use a transaction. Using a transaction, you can modify the database as much as you want, and as soon as something goes wrong, you can tell the database to undo everything that you have done. Transactions help to maintain the integrity of the data, and help to adhere to the well-known ACID test (Atomicity, Consistency, Isolation and Durability).

Generally, Atomicity refers to the fact that you are attempting an atomic unit of work, which means that from a business point of view the transaction's work cannot and should not be able to be broken down.

Consistency means that each transaction will leave the data in a consistent state and so data integrity is preserved.

Isolation refers to the fact that one transaction should not and will not be affected by another transaction happening at the same time. They are logically ignorant of each other.

Finally, Durability refers to the fact that if a transaction succeeds, it is guaranteed to persist itself, even if the machine crashes. Special log files are used for this.

Transactions are a must for enterprise applications as data integrity is essential for accounting, legal, and service reasons.

The History of the Data Tier

In the Beginning...

Data tiers never used to exist. Instead, there was just a mammoth application that contained the data store, the data retrieval code, the business logic, and the presentation code. With the advent of ODBC, some application developers began separating the database from the application. Even then, a lot of application developers kept vast amounts of business logic embedded in the database.

Tiers Emerge

With the emergence of Microsoft DNA (i.e. n-tier architecture) the application was divided into a presentation tier, a business logic tier, a data tier, and the database. This methodology has largely stuck, albeit with a few minor changes, which we will look at shortly.

With the database now not guaranteed to be in the same process, nor on the same machine as the data tier, a connection is required. This is essentially an object residing on the data tier machine loaded from the database library (located on the data tier machine) that can locate the database via a connection string and talk to it over the network (or internet).

Connections are expensive, both in resources and in monetary terms. Having one connection per client for thousands of internet users connecting to your database is not advisable from a performance or budgetary perspective. Additionally, having the connection wait around while the user is idle is simply a waste of resources. For these reasons, Microsoft implemented connection pooling, whereby closing a connection simply returned that connection to a pool. When you then create a new connection, the pool will be searched first, looking for a match against your connection string (an exact character match is required!). Developers were then encouraged to close connections as soon as possible to gain performance and scalability.

MTS is born

With regards to transactions, originally there was limited library support and the onus was on the developer to create their own complex frameworks. The arrival of Microsoft Transaction Server (MTS) was heralded as the best thing since sliced bread, and many people jumped on the bandwagon. MTS was easy to use in the sense that there was no complex parameter passing required to keep track of transactions. You merely called a method on a global object and effectively set none, one, or both of two bits: a 'done' and a 'happy' bit. You set the 'happy' bit if you deemed that your method had succeeded, and set the 'done' bit to say that you were finished. MTS took care of the rest. When the 'root object' (the object at the beginning of the call chain) exited, MTS would intercept it, check the 'done' and 'happy' bits, and commit or abort (rollback) the transaction accordingly.

COM+ takes over

COM+ extended MTS by using attributes (yes, attributes). From the Component Services Explorer (CSE), you as the administrator (or developer) could set the transactional requirements of a class, and could also set whether the 'done' bit is set on return of a method.

It is important to note that these transactions used by COM+ and MTS are distributed transactions and not local transactions. This means that they require a 'transaction coordinator', like Microsoft's Distributed Transaction Coordinator (MS DTC) to coordinate the transaction between several different databases (or other XA compliant devices), even if there is only one database involved in the transaction.

COM+ also completed the unfinished parts of MTS, namely object pooling. With object pooling, COM+ stores a pool of objects rather than destroying and recreating them. When you need a new object, COM+ will first look in the pool for one. It was discovered by Microsoft that using object pooling with objects that contained a connection was twice as fast as using connection pooling, so a lot of C++ programmers now use object pooling for their connections (note that object pooling is not available in VB6). The only 'extra' to watch out for is that you will have to manually enlist your newly activated object in a transaction if you require its participation, as COM+ cannot do this for you automatically. However, COM+ does store deactivated objects that were in a transaction in their own 'mini pool' and will hand you one of these first if the same transaction requires a new object of that type. (Thank you to Joe Long at Microsoft for pointing that one out to me.)

.NET Arrives

The .NET framework library lets you use all of the COM+ Services by providing attributes and the classes in the EnterpriseServices namespace. It also provides OLE DB support as well as native SQL Server classes via the OledbClient and SqlClient namespaces respectively (part of ADO.NET).

SqlClient talks directly to SQL Server through dbnetlib.dll using the TDS protocol (Tabular Data Stream) and is the fastest (and recommended) way to communicate with SQL Server. It is handy to remember that ADO.NET datasets are completely disconnected, whereas its predecessor's recordsets generally were not, or were tricky to implement. It is useful to note that the .NET framework library does not seem to support manual enlistment into distributed transactions. For that, one would have to use PInvoke or COM InterOp.

SqlClient In-Depth

Armed with Microsoft's IL Disassembler (ILDasm.exe), I decided to forage around the guts of the SqlClient namespace in System.Data.dll. To summarize my findings: SqlClient basically uses the SqlConnection class to connect to the database and to begin and end transactions, the SqlCommand class to capture the user's requests for data in a structured manner, and the SqlTransaction class to handle committing or rolling back the transaction. Several other classes exist (like the SqlDataReader class) to turn an internal buffer of returned data into an easily used object.

When you begin a transaction using the SqlConnection class' BeginTransaction method, all that happens under the covers is a ' SET ISOLATION LEVEL XXX; BEGIN TRANSACTION ' T-SQL statement is sent to SQL Server, where XXX is the isolation level that you have set, or 'Read Committed' as the default. (The method does make sure that the connection is still open, that no existing SqlDataReader classes are attached (as they block the connection), and that no old or existing transaction exists).

When you call any of the ExecXXX methods on the SqlCommand class, they all internally call ExecuteReader, which uses an internal class called TdsParser to turn all the nicely structured information that you have placed into the SqlCommand object into a buffer to send to SQL Server. If data is returned, it is then placed into another buffer to be used by one of the SqlClient data container classes if required. It is useful to note that output parameters and return values are placed at the end of the buffer, so if any result sets are returned, they must be read first before these values can become available (they will appear as null until this time).

The SqlTransaction class merely emits ' COMMIT TRANSACTION ' and ' ROLLBACK TRANSACTION ' T-SQL statements to SQL Server when the Commit or RollBack methods are called respectively. Since other parts of the SqlClient namespace might roll the transaction back internally if an error occurs, this Rollback method could throw an exception if you call it after the transaction has been rolled back internally. If this occurs, you are guaranteed that the transaction has been rolled back.

If you wish to participate in a transaction, you will need a reference to the SqlTransaction object that you got from calling BeginTransaction on the SqlConnection object, and you will need a reference to that SqlConnection class. Armed with these, you can use a SqlCommand object (a new or existing one) by setting its Connection and Transaction properties to the two references you obtained above.

The Past Mistakes of the Data Tier

Before distributed programming took off, there was sparse information, methodology, or metrics available about the implications the design of a data tier had on the performance and scalability of the application. Good object orientated (OO) application programmers were often given the task of designing n-tier distributed applications, and were put in charge of, amongst other things, the data tier. Having no previous commercial experience of databases, and armed only with their years of server development experience, they set off to design an elegant data tier. They were treading on new ground, and so a few (now well recognized) common mistakes emerged:

Get Next

Linked lists are a common structure in programming; they are a structure that has a pointer to both its current data and next element. With that, one could write an elegant loop that would do something with the data, get the next element in the list, and start again. Some programmers used this technique as an elegant looking solution to the data tier and designed the database tables in the same way, not realizing that the databases had been designed, and hence optimized, for sets and set operations. The result was an exponential increase in the amount of trips to the database, which in turn had a huge negative effect on performance.

Sticking with strict OO

At the time, many OO programmers held OO above all else. They lived by it, and they would not compromise. For example, if they had n database queries to call in a row, and none of the queries returned any results, a batch job could be created in one string to send to the database. This would mean one trip to the database. The OO programmers, on the other hand, would stick to their guns that they must maintain their structure and set each one of the queries up as a separate class each with its own method to run itself. The result would be n trips to the database instead of one.

Using a bazooka to kill a fly

When MTS (and later COM+) came out, it was seen as the way to handle transactions. It could handle transactions involving any number of different databases (and other devices too) using a device called a DTC (Distributed Transaction Coordinator). This was fine if you were using more than one database, but most businesses weren't. So what was the impact of DTC on an application using only one database? It actually decreased performance!

DTC uses a methodology called 'Two-Phase-Commit' where each participant records a log of what it would actually do were it to commit. If all the participants succeed, then DTC tells each participant to commit what they have logged. With an application consisting of only one database, this is effectively like doing the transaction twice, consuming time and resources along the way!

Failing to keep order

One mistake made by data tier implementers is to treat each stored procedure as a separate entity, and not look at them as a whole. The result is that resources (tables, etc) get acquired in many different orders. This can lead to deadlocks where stored procedure A has acquired resource X and is waiting on resource Y, because stored procedure B has acquired resource Y and is waiting on resource X. The database then has to intervene and destroy one of the transactions. In the mean time, unnecessary resources have been held for a long time period, decreasing performance and scalability.

Isolation Level versus Performance

The default isolation level for MTS and COM+ is Serializable. The added guarantee that this level offers over the others is that it will not allow Phantoms. In other words, this isolation level will not allow the situation where the transaction reads a set of records twice and gets different results because another transaction inserted or deleted records from this set.

SQL server uses a concept called 'Key Range' locks to enforce this level. It effectively takes the largest and smallest primary key values of your set and locks all the possible records in between, whether or not they are in your set or even exist!

If your primary key is an identity column (which is merely an internal id and has no real business bearing) and a transaction brought back ID numbers 1 and 999, then all records with ids greater than zero and less than 1000 would be locked. This would have a severe impact on the performance and scalability of your application, as this is not what you intended in the first place.

A caveat

A caveat to this is the fact that reading data often requires a higher degree of isolation than writing data. Citing an example from Tim Ewald's Transactional COM+ (pg 260):

You have two tables participating in a parent-child relationship. The parent table stores order details and the child table stores line items for the order. A foreign key constraint is in place to maintain data integrity.

You have a transaction (A) that:

  • Deletes all the line items for an order
  • Deletes that order

The order of deletion must be so because of the foreign key constraint. Next you have another transaction (B) that:

  • Reads all the line items for an order
  • Reads that order

The order of reading must be so to avoid deadlocks.

Now if, for the same order, transaction B executes step 1, then transaction A executes step 1 and step 2, then transaction B executes 2, you will end up with transaction B returning line items but no order. What you actually need to do is to wrap transaction B in a Repeatable Read Isolation Level. Many developers have failed to realize this.

Identity Columns and Clustered Indexes

Index design, which some see more as an art than a science, was largely left until complaints came in about performance. Many developers simply stuck with the defaults or made all their identity columns the primary key using a clustered index. What this means is that the physical data on disk gets stored in that order. Since identity columns are simply auto incrementing numbers, every new row will get inserted at the end of the table. Although this sounds like a good idea, it can actually be quite harmful with respect to performance. If one client is trying to insert a few rows at a time, all this activity is taking place at the same place in the physical table, and SQL Server would then try to optimize by locking the entire page for the client. Any other client would then be blocked, which can have a snowball effect on the server.

Long Running Transactions

When a transaction acquires a lock on a resource (be it a row, page, table, etc) it effectively blocks all other transactions from using it, causing them to wait. The number of permutations of waiting dependencies that even a small number of transactions can have can easily grow exponentially. It is therefore crucial not to hold onto a resource for any more time than is absolutely necessary. Yet many developers have had to wait for transactions while the application is waiting on user input (which in CPU time is almost an eternity). If the user went to lunch, the backlog of transactions waiting on transactions waiting on transaction would surely cause the entire system to grind to a halt, if not crash. The best that could happen is that the transaction would timeout and be rolled back.

Suggestions and Improvements to Date

Through the years, many developers, authors, and vendors have come up with a set of suggestions and metrics to try and guide us in the direction of performance and scalability. We will now take a look through a few of the more common suggestions.

Stored Procedures are good

To keep in with interface programming (the current recommended way to go), as well as for performance reasons, all data access should be made via stored procedures. SQL Server pre-compiles the stored procedures and creates an execution plan the first time they are run. This execution plan is then used on subsequent requests.

Also, if you know that all of your database access is done through the stored procedures, then that is the only place you need to update if you change your table structure. If you make sure that the stored procedure names, the parameter names and order, and the result set's column names (aliases) and order are maintained, you should not break any existing code. Once you have created a stored procedure, test it and try to optimize it for peak performance. Also look at it in conjunction with ALL the other stored procedures that you have created. Is the order of resource acquisition the same as any other references to these resources?

COM+ Declarative Transactions are slow

For single database systems, do not use COM+ declarative transactions. Either write the transactional support into your stored procedures, or, if that will become too complex, use some kind of code framework to start, commit, or rollback the transactions (this is what we are going to be doing!). This should increase your performance and help scalability. It is interesting to note how this has gone full circle in recent years.

Make chunky not chatty calls and minimize round trips

If you don't require any return values, group database calls together and submit them as one batch. Remember that databases are designed, and hence optimized, for sets and set operations, so design your tables and stored procedures with that in mind. Making one call to the database to get 100 rows is exponentially better than making 100 calls to each get one row, or even to return output parameters.

Pass Data by Value

In the data server, you basically have to get the data into and out of a buffer, so having further marshalling involved to get each piece of data is wasteful. Pass the data into the data tier by value, and then into the buffer by value. Optimally you would want to do this in one step.

What I prefer to do is to create a struct that mimics the return value, the input and output parameters, and the output columns, and pass that in and out of my methods on the data tier. That way, if someone else wants to add another field to that struct (and we are using the same stuct for business reasons) it will not affect my code. It also has the added bonus that if I were to want to add a similar field, I would go to the definition of the struct and see that it has already been added. This would lead me to look at code using the new field to see if I could use the existing code, rather than duplicate it somewhere else.

Also, if your COM+ data tier classes can be loaded in the same context as your business tier classes, then they should be. This can be done in COM+ by marking your data tier assembly as a Library Application instead of a Server Application. This will eliminate unnecessary marshalling. If a parameter only returns data, mark it as an out parameter in C#, as this will eliminate that parameter having to be marshaled into the method.

Pool Connections or, if possible, objects

Object pooling has been demonstrated by Microsoft to be twice as fast as connection pooling. Unfortunately, this functionality deals with distributed transactions, where multiple connections participate in the same transaction. As we will not be using distributed transactions, we will be implementing a different (and new) approach.

Optimize your Indexes

One of the major factors in database performance is the index. An index can mean the difference between 0.001 seconds and 100 seconds. Index design is a complex topic, and some people would rate it as more of an art than a science, but it is essential for your application to perform well.

Indexes in SQL Server are divided into clustered and non-clustered indexes. Clustered indexes actually store the physical data in the table in that order, hence there can only be one clustered index per table. A non-clustered index is simply another internal table holding the index (in order) with an internal row id of the table.

If you will be retrieving single rows by their primary key, then it is best not to make that key a clustered index. A clustered index serves its purpose well if you will be searching on a range of values or will be returning a set of values ordered in the same way as the clustered index.

Acquire resources in order

If you try to follow the pattern of acquiring resources in the same order in different transactions, you will minimize the probability of getting deadlocks.

Isolation is a matter of taste

Choose the isolation level that is correct for the transaction it will be applied to. Keep in mind that reading often requires a higher degree of isolation.

Retrieve Data in the correct way

Microsoft recommends using a data reader over a dataset if you do not need the data lying around (MSDN Article - Performance Tips and Tricks in .NET Applications) and using SqlClient over ODBC. The following performance metrics (higher numbers are better) show this:

 | ADO (ODBC) | SQL
DataSet | 801 | 2507
DataReader | 1083 | 4585

Microsoft also recommends (only if you are using some form of connection pooling - which we are) using output parameters over returning result sets (MSDN Article - .NET Data Access Architecture Guide), as the former can outperform the latter by up to 30% in high-stress conditions (200+ simultaneous connections).

The Proposed Solution

What is propose is a set of .NET classes that will interact with each other to provide a data tier that will adhere to all the guidelines suggested in this case study. It must be structured enough to allow maximum code re-use, but not at the cost of performance nor scalability.

The Solution

Our enterprise application consists of a set of employees, each of whom can have one or more addresses. We will need to capture new employees details, as well as update or delete employees. We will also input some test data for a demonstration of our 'framework'.

Database Design and Structure

The Database

We will need to refer to this database later in the code, so we will name it now.

CREATE DATABASE data_tier
GO
USE data_tier
GO

The Tables

Let us start with the address table. An address will have, in addition to the logical description at the beginning of this case study, a unique internal id to distinguish between addresses.

CREATE TABLE [dbo].[address] (
[id] [int] IDENTITY (1, 1) NOT NULL,
[name] [nvarchar] (50) NOT NULL,
[flat_number] [nvarchar] (10) NULL,
[flat_house_name] [nvarchar] (50) NULL,
[street_number] [nvarchar] (10) NULL,
[street_name] [nvarchar] (50) NULL,
[locality] [nvarchar] (50) NULL,
[city] [nvarchar] (50) NULL,
[county] [nvarchar] (50) NULL,
[country] [nvarchar] (50) NOT NULL,
[post_code] [nvarchar] (10) NOT NULL
) ON [PRIMARY]
GO
CREATE CLUSTERED INDEX [IX_address] ON
[dbo].[address]([post_code]) ON [PRIMARY]
GO
ALTER TABLE [dbo].[address] WITH NOCHECK ADD
CONSTRAINT [PK_address] PRIMARY KEY NONCLUSTERED
([id]) ON [PRIMARY]
GO

A clustered index has been placed on the post_code column. This forces the data in the table to be stored in this order, making searches on partial post codes extremely fast. The primary key has been made a non-clustered index on the id column, as generally only single rows will be retrieved using this primary key.

Next we have the employee table. This will record the fist name, last name, and the date of birth of the employee, as well as an internal id to distinguish between employees.

CREATE TABLE [dbo].[employee] (
[id] [int] IDENTITY (1, 1) NOT NULL,
[first_name] [nvarchar] (50) NOT NULL,
[last_name] [nvarchar] (50) NOT NULL,
[date_of_birth] [datetime] NOT NULL
) ON [PRIMARY]
GO

CREATE CLUSTERED INDEX [IX_employee] ON
[dbo].[customer]([last_name]) ON [PRIMARY]
GO

ALTER TABLE [dbo].[employee] WITH NOCHECK ADD
CONSTRAINT [PK_employee] PRIMARY KEY NONCLUSTERED
([id]) ON [PRIMARY]
GO

A clustered index has been placed on the last_name column. This will sort the data by that column and be fast for employee searches. The primary key has been made a non-clustered index on the id column, as generally only single rows will be retrieved using this primary key.

Finally, we need a table to record which addresses belong to which employee.

CREATE TABLE [dbo].[employee_address] (
[employee_id] [int] NOT NULL,
[address_id] [int] NOT NULL
) ON [PRIMARY]
GO

ALTER TABLE [dbo].[employee_address] WITH NOCHECK ADD
CONSTRAINT [PK_employee_address] PRIMARY KEY CLUSTERED
([employee_id],[address_id]) ON [PRIMARY]
GO

A clustered index (which is also the primary key) has simply been placed on both columns. This has been done with the employee_id first to physically sort the data by employee_id and then address_id. The reason for placing both columns into the index is that once you have found the value in the index, you already have all the information that you need, and do not have to make an extra traversal over the table to retrieve the address_id.

The Stored Procedures

To access these tables, we will define a set of stored procedures. We will need to be able to create a new employee, update an existing one, and delete an existing one. We also want to provide the functionality to create, update or delete an employee's address. Finally, we will provide a search mechanism.

CREATE PROCEDURE usp_get_employee
(@id int,
@first_name nvarchar(50) OUT,
@last_name nvarchar(50) OUT,
@date_of_birth datetime OUT)
AS
BEGIN
SELECT id as id,
name as name,
flat_number as flat_number,
flat_house_name as flat_house_name,
street_number as street_number,
street_name as street_name,
locality as locality,
city as city,
county as county,
country as country,
post_code as post_code
FROM address a
JOIN employee_address ea ON a.id = ea.address_id
WHERE employee_id = @id

SELECT @first_name = first_name,
@last_name = last_name,
@date_of_birth = date_of_birth
FROM employee
WHERE id = @id
END
GO

This stored procedure assumes that the addresses are always required when retrieving an employee. Although this may not always be the case, it has been included here to demonstrate several points.

Firstly, it has eliminated round trips to the database. Some OO programmers would still prefer to get an array of address ids, pass each one to an address object, and get the address object to populate itself from the database. Secondly, it shows that combining output parameters and result sets should not be shied away from. Thirdly, it shows the strict use of interface programming. Everything coming from the database is named using the 'as' keyword, including the column names. This leave the database developer free to change his internal structure as his sees fit. He could rename the post_code column to zip_code, and all he would need to do is to correct the stored procedure. No data tier code would need to be changed. Finally, it shows the order of selection. We select the addresses first and then the employee, as this is the order in which we would delete them. Doing this minimizes the chances of a deadlock.

The other stored procedures to be used by the application are presented below.

CREATE PROCEDURE usp_create_employee
(@first_name nvarchar(50),
@last_name nvarchar(50),
@date_of_birth datetime,
@id int OUT)
AS
BEGIN
INSERT INTO employee (first_name, last_name, date_of_birth)
VALUES(@first_name, @last_name, @date_of_birth)
SET @id = @@identity
END
GO

CREATE PROCEDURE usp_update_employee
(@id int,
@first_name nvarchar(50),
@last_name nvarchar(50),
@date_of_birth datetime)
AS
BEGIN
UPDATE employee
SET first_name = @first_name,
last_name = @last_name,
date_of_birth = @date_of_birth
WHERE id = @id
END
GO

CREATE PROCEDURE usp_delete_employee
(@id int)
AS
BEGIN
-- 1. Delete all the employees adresses
DELETE address FROM address a
JOIN employee_address ea ON a.id = ea.address_id
WHERE ea.employee_id = @id
-- 2. Delete the Mapping
DELETE employee_address
WHERE employee_id = @id
-- 3. Delete the Employee
DELETE employee
WHERE id = @id
END
GO

An employee can be created, updated, or deleted. Creating an employee returns an id back to the caller.

CREATE PROCEDURE usp_create_employee_address
(@employee_id int,
@address_name nvarchar(50),
@flat_number nvarchar(10),
@flat_house_name nvarchar(50),
@street_number nvarchar(10),
@street_name nvarchar(50),
@locality nvarchar(50),
@city nvarchar(50),
@county nvarchar(50),
@country nvarchar(50),
@post_code nvarchar(10),
@id int OUT)
AS
BEGIN
INSERT INTO address (name, flat_number, flat_house_name,
street_number, street_name, locality,
city, county, country, post_code)
VALUES (@address_name, @flat_number, @flat_house_name,
@street_number, @street_name, @locality,
@city, @county, @country, @post_code)
SET @id = @@Identity
INSERT INTO employee_address (employee_id, address_id)
VALUES (@employee_id, @id)
END
GO

CREATE PROCEDURE usp_update_employee_address
(@id int,
@address_name nvarchar(50),
@flat_number nvarchar(10),
@flat_house_name nvarchar(50),
@street_number nvarchar(10),
@street_name nvarchar(50),
@locality nvarchar(50),
@city nvarchar(50),
@county nvarchar(50),
@country nvarchar(50),
@post_code nvarchar(10))
AS
BEGIN
UPDATE address
SET name = @address_name,
flat_number = @flat_number,
flat_house_name = @flat_house_name,
street_number = @street_number,
street_name = @street_name,
locality = @locality,
city = @city,
county = @county,
country = @country,
post_code = @post_code
WHERE id = @id
END
GO

CREATE PROCEDURE usp_delete_employee_addresses
(@id int)
AS
BEGIN
DELETE address FROM address a
JOIN employee_address ea ON a.id = ea.address_id
WHERE ea.employee_id = @id
DELETE employee_address
WHERE employee_id = @id
END
GO

One interesting thing to note is that the addresses of the employee are deleted in the stored procedure that deletes the employee. Some people may disagree with this approach, claiming that these are business rules. I would argue that these are data-centric rules, as an employee address record cannot exist in the database without an employee, making this a table relationship rule.

The Test Data

DECLARE @employee_id int
DECLARE @address_id int
EXEC usp_create_employee 'Fred','Smith','1972-12-05 12:12:12.007',
@employee_id OUT
EXEC usp_create_employee_address @employee_id, 'Personal1', '1',
'Tennis Court', '1', 'Wimbledon Ave', 'Wimbledon', 'London', 'London',
'UK', 'SW19', @address_id OUT
EXEC usp_create_employee_address @employee_id, 'Personal2', '1',
'Tennis Court', '1', 'Wimbledon Ave', 'Wimbledon', 'London', 'London',
'UK', 'SW19', @address_id OUT
EXEC usp_create_employee_address @employee_id, 'Personal3', '1',
'Tennis Court', '1', 'Wimbledon Ave', 'Wimbledon', 'London', 'London',
'ZZ', 'SW19', @address_id OUT
EXEC usp_create_employee_address @employee_id, 'Personal4', '1',
'Tennis Court', '1', 'Wimbledon Ave', 'Wimbledon', 'London', 'London',
'ZZ', 'SW19', @address_id OUT

The test data is used in conjunction with the business tier to show the benefits of using this methodology.

Data Tier Design and Coding

For this demonstration, we will be creating a C# class library (dll file). We shall call the class library (and its namespace) DataTier.

Of the COM+ Component Services, we will be using a connection strings and object pooling. Using COM+ Component Services will require us to reference the System.EnterpriseServices library, and will require our assembly to be Strong Named. Therefore we will require a Strong Name Key (.snk) file.

You can either use the .snk file that is supplied, or you can create your own with Microsoft's sn.exe utility, which comes with the Framework SDK (found in the bin folder). From the command prompt, type: sn -k <file name> where file name is the name you wish to give to your strong name key file.

The Assembly Info

Once you have a .snk file, you need to reference it. In the AsseblyInfo.cs file, you will need to insert the following:

[assembly: AssemblyKeyFileAttribute(@"yourDirectory\youFile.snk")]

where yourDirectory is the directory the .snk file is in, and yourFile is the name of it.

Since we require the data tier to be in the same context as the business tier, we must mark it as being a Library Application. This is done by placing the following line in the AssemblyInfo.cs file:

[assembly: ApplicationActivation(ActivationOption.Library)]

We will also need to provide the COM+ Application name and ID that we wish to belong to, and hence be registered for. This is done by inserting the following in the AssemblyInfo.cs file:

[assembly: AssemblyDelaySign(false)]
[assembly: ApplicationName("DataTier")]
[assembly: ApplicationID("78BF86F2-E008-44c7-B37F-762FC1A27909")]

For this to compile, you will need to place the following at the top of the AssemblyInfo.cs file with the other using directives.

using System.EnterpriseServices;

The Data Types

We start off by defining the internal structures that we will be using. They will entirely consist of primitive types in order to act as a bridge between the business tier and the database.

using System;

namespace
            DataTier
{
public struct InternalEmployee
{
public Int32
            ID;
public string FirstName;
public string LastName;
public
            DateTime DateOfBith;
}
public struct
            InternalAddress
{
public Int32 ID;
public string
            Name;
public string FlatNumber;
public string
            FlatHouseName;
public string StreetNumber;
public string
            StreetName;
public string Locality;
public string
            City;
public string County;
public string Country;
public
            string PostCode;
}
}

The DataServer Class

We start the file off with the using directives:

using System;
using System.Data;
using System.Data.SqlClient;
using System.EnterpriseServices; // ServicedComponent and Attributes
using System.Text; // StringBuilder

namespace DataTier
{
[ObjectPooling(true)]
[MustRunInClientContext(true)]
[ConstructionEnabled(Default="initial catalog=data_tier;data
source=localhost;User Id=sa")]
public class DataServer : ServicedComponent
{
public DataServer(){}

Note that we are referencing the System.EnterpriseServices library as we are deriving our DataServer class from ServicedComponent and using various attributes from that namespace. We will be using the StringBuilder class and hence need to reference the System.Text namespace.

The ObjectPooling attribute declares our intention to use object pooling, as we have supplied it with a value of true. The minimum and maximum pool sizes could also be declared here for fine grain control. The minimum pool size is useful if you require COM+ to pre-populate a number of objects on startup.

The MustRunInClientContext attribute is used in conjunction with the attribute we declare in the AssemblyInfo.cs file to load the assembly as a Library Application. The ConstructionEnabled attribute is used to declare our intention to use construction strings and to provide a default construction string in case one is not set in the CSE. I have used the default 'sa' database user for our application. For a commercial version, it is much more secure to remove all security from the connection string and use the CSE to let the component run under a particular Windows User. SQL Server can then be set up to accept this user and assign privileges.

Our class finally derives from Serviced Component to ensure that all the necessary COM+ plumbing will be wired up for us.

Next we declare our private member variables:

private Select m_rSelect;
private Update m_rUpdate;
private Processor m_rProcessor;
private string m_sCnn;
private SqlConnection m_rCnn;
private SqlCommand m_rCmd;
private SqlCommand m_rInternalCommand;
private object m_rRoot;
private TransactionStatus m_eStatus;
private StringBuilder m_sbParsedCommands;
private IsolationLevel m_eIsolationLevel;

We will address the purpose of each of these member variables when they become relevant. For now we will press on.

The Static and Constant Helper Functions

We will now define several 'Helper' functions and constants for use with the Data Tier:

public const Int32 NameTextSize = 50;
public const Int32 NumberTextSize = 10;
public static string GetNVarChar(string Input)
{
if(Input==null)
return "NULL";
Input = Input.Replace("'","''");
return "'" + Input + "'";
}
public static string GetDateTime(DateTime Input)
{
if(Input==DateTime.MinValue)
return "NULL";
// More accurate than SqlClient
return "'" + Input.ToString("yyyy-MM-dd hh:mm:ss.fff") + "'";
}
public enum TransactionStatus
{
Nothing = 0,
BeginTransaction = 1,
InTransaction = 2
}

The NameTextSize and NumberTextSize help us store, in one place, the lengths of our database strings. As we will be referencing them many times in our Data Tier code, if we need to change any of them, only one change will be needed.

Part of our design is the 'flattening out' of our commands, i.e. creating a stream of commands in a string. We will need a function to make sure that text fields are returned in the proper format. We will also need to convert DateTime variables into strings. Currently SqlClient chops off the seconds and fractions of seconds when doing this. I prefer to leave them in. Our GetNVarChar and GetDateTime methods fulfill these requirements.

We will also need to record the intention of using a transaction so I have created an enumeration to store the 'logical' status of the transaction. This is crucial in order to start a transaction as late as possible to minimize the time that the transaction is actually running.

COM+ Services

The first COM+ Service that we have declared our support of is the construction string. We therefore need to override the Construct method, which COM+ guarantees will be called immediately after the constructor and will be passed in the construction string in the CSE, or the default declared in the attribute (ConstructionEnabled) if none exists.

protected override void Construct(string
            constructString)
{
m_sCnn = constructString;
m_rCnn = new
            SqlConnection(m_sCnn);
m_rCnn.Open();
m_rCmd = new
            SqlCommand();
m_rCmd.CommandType =
            CommandType.StoredProcedure;
m_rCmd.Connection =
            m_rCnn;
m_rInternalCommand = new
            SqlCommand();
m_rInternalCommand.CommandType =
            CommandType.Text;
m_rInternalCommand.Connection =
            m_rCnn;
m_rSelect= new Select(this, m_rCmd);
m_rUpdate = new
            Update(this, m_rCmd);
m_rProcessor = new Processor(this,
            m_rCmd);
}

Here we stored the construction string (you will see why later) and a connection that we open. We then create two command objects pointing to the same connection. One is a public command used for commands that return data to the client by means of a stored procedure, and one is an internal command used to execute buffered commands that do not return data. Although the documentation in MSDN states that this is a public method, it is in fact a protected method and will not compile unless set as such.

If an error occurs, it is wise to log it somewhere (as users will not remember technical error information) and to throw the error up to be handled by the business tier. I have removed all error handling that is not part of the logic I wish to demonstrate, for the sake of brevity.

The next COM+ Service that we have declared our support for is object pooling. When an instance of the DataServer class is requested, COM+ will first look in the object pool to see if one already exists that is not in use. If there is an instance of DataServer in the pool, COM+ removes it from the pool, calls Activate on it, and releases it to the client. If the pool is empty, COM+ will create a new DataServer object, and will still call the activate method before releasing it to the client.

protected override void
            Activate()
{
m_eStatus =
            TransactionStatus.Nothing;
m_sbParsedCommands = new
            StringBuilder();
if(m_rCnn.State!=ConnectionState.Open)
{
m_rCnn.Close();
this.Construct(m_sCnn);
}
else
            if(m_rCmd.Transaction!=null)
{
this.Rollback();
}
}

When a DataServer object is activated, it will reset its internal transaction status to nothing and create a new stringbuilder object to store the buffered commands. It will also make sure that the connection has not become corrupt and that no lingering transaction exists.

When an object that implements IObjectControl (i.e. when it declares its support for object pooling with the ObjectPooling attribute) is about to be disposed, COM+ guarantees to call the Deactivate method on it.

protected override void
            Deactivate()
{
this.Rollback();
}

Here we simply call our RollBack method, which, as we will learn later, will Rollback any lingering transactions, if they exist.

After COM+ has called the Deactivate method it calls the CanBePooled method. If this returns true, then COM+ will place this object on the pool, otherwise it will let the object die gracefully.

Internal Helper Methods

The DataServer class has several internal helper methods to get the infrastructure to work smoothly. I will discuss each in turn.

internal bool InTransaction()
{
return
            m_rCmd!=null &&
m_rCmd.Transaction!=null
            &&
m_rCmd.Transaction.Connection!=null;
}

The InTransaction method checks to see whether the DataServer is currently inside an actual transaction. This is done in SqlClient by checking the Connection property of the Transaction object.

internal void BeginTransaction(object
            Caller, IsolationLevel Isolation)
{
if(m_eStatus ==
            TransactionStatus.Nothing)
{
m_rRoot=Caller;
m_eStatus =
            TransactionStatus.BeginTransaction;
}
if(m_eIsolationLevel<Isolation)
m_eIsolationLevel
            = Isolation;
}

The BeginTransaction is used to record the intention of beginning a transaction, without beginning one. This is done to start the transaction as late as possible, to minimize its lifetime. It also records the root caller to know when to commit, as well as the intended isolation level, which it will check against previous requests, and select the highest.

internal void AppendCommand(object Caller, string SQL,
IsolationLevel Isolation)
{
m_sbParsedCommands.Append(" " + SQL + " \n");
this.BeginTransaction(Caller, Isolation);
}

The AppendCommand method is used for commands that do not return data. These commands can be stored (in a stream or string) and sent to the database as one batch, thus minimizing round-trips. It is assumed that any operation not returning data must be doing something to the database and hence will require a transaction. The isolation level can be specified.

internal void
            FlushBuffer()
{
if(m_sbParsedCommands.Length>0)
{
if(m_eStatus!=TransactionStatus.InTransaction)
{
m_rCmd.Transaction
            =
m_rCnn.BeginTransaction(m_eIsolationLevel);
m_eStatus=TransactionStatus.InTransaction;
}
m_rInternalCommand.CommandText
            = m_sbParsedCommands.ToString();
m_rInternalCommand.Transaction =
            m_rCmd.Transaction;
m_sbParsedCommands.Remove(0,
            m_sbParsedCommands.Length-1);
m_rInternalCommand.ExecuteNonQuery();
}
}

The FlushBuffer method is used for sending the buffered commands, if any, to the database. This will be done either when the root object is about to commit, or when a command that returns data needs to be executed. This method creates a physical transaction at the appropriate isolation level, marks the DataServer object as being in a transaction, executes the buffered commands on the internal command object, and clears the buffer.

internal void
            Rollback()
{
try
{
if(InTransaction())
m_rCmd.Transaction.Rollback();
}
catch
{
/*
            Microsoft could have already rolled back */
/* Could have done a
            IF @@TRANCOUNT > 0 ROLLBACK TRANS*/
}
finally{m_eStatus =
            TransactionStatus.Nothing;}
}

The Rollback method is used, surprisingly, for rolling back transactions. It will first check to see if the DataServer object is in a transaction, and if so, it will roll it back. It is interesting to note that I have placed this logic inside a try block. This is done because several of the SqlClient classes will actually rollback the transaction for you. Thus calling Rollback will result in a SQL error as there is no transaction to rollback. One suggestion that I have seen is to send an "IF @@TRANCOUNT > 0 ROLLBACK TRANS " command down the wire. While this will prevent an error, it will not do all the things that SqlConnection.Rollback does behind the scenes for you. Microsoft suggest that you merely catch any error and ignore it. The finally block makes sure that the status is reset.

internal void
            Commit()
{
try
{
this.FlushBuffer();
if(InTransaction())
m_rCmd.Transaction.Commit();
}
catch
{
Rollback();
throw
            new Exception("Transaction Aborted");
}
finally{m_eStatus =
            TransactionStatus.Nothing;}
}

The Commit method is used, surprisingly, for committing our transaction. It will first flush the buffer of any pending commands. If an error occurs, it will attempt to roll the transaction back, and then raise an exception to the client. The finally block makes sure that the status is reset.

internal void ExecuteNonQuery(object Caller,
            bool WithTransaction,IsolationLevel
            Isolation)
{
if(WithTransaction)
this.BeginTransaction(Caller,
            Isolation);
FlushBuffer();
if(WithTransaction &&
            m_eStatus!=TransactionStatus.InTransaction)
{
m_rCmd.Transaction
            =
m_rCnn.BeginTransaction(m_eIsolationLevel);
m_eStatus=TransactionStatus.InTransaction;
}
m_rCmd.ExecuteNonQuery();
}

The ExecuteNonQuery method is used to execute a command that does not return a dataset, but cannot be placed in the buffer as it returns either output parameters or a return value. A transaction is optionally started (if one does not exist), the buffer is flushed, and the command is executed.

internal SqlDataReader ExecuteReader(object Caller, bool WithTransaction, IsolationLevel Isolation)
{
if(WithTransaction)
this.BeginTransaction(Caller, Isolation);
FlushBuffer();
if(WithTransaction && m_eStatus!=TransactionStatus.InTransaction)
{
m_rCmd.Transaction =
m_rCnn.BeginTransaction(m_eIsolationLevel);
m_eStatus=TransactionStatus.InTransaction;
}
return m_rCmd.ExecuteReader();
}

The ExecuteReader method is used to execute a command that returns a DataReader. A transaction is started if necessary, the buffer is flushed, and the command is executed.

The Public Methods and Properties

public void
            SetAbort()
{
this.Rollback();
}
public void
            SetComplete(object Caller)
{
if(object.ReferenceEquals(Caller,
            m_rRoot))
this.Commit();
}
public Select Select{get{return
            m_rSelect;}}
public Update Update{get{return
            m_rUpdate;}}
public Processor Processor{get{return
            m_rProcessor;}}

The SetAbort method is provided for the client to abort the transaction. A database transaction may not even have started physically, but this method guarantees to rollback any database activity. No exception will be thrown by SetAbort (this is guaranteed), and the code calling SetAbort would already know that an error has occurred, so would not want to handle an exception anyway.

The SetComplete method will commit the transaction if the caller is the root caller.

The Select, Update and Processor properties simply provide access to our objects containing the data access code.

Well, that is it for our DataServer class. We will now see how our data accessing classes will interact with this class.

The Processor Class

The Processor class is the class in which we will store all data access code that will not be returning any values.

using System;
using System.Data;
using
            System.Data.SqlClient;
using System.Text;

namespace
            DataTier
{
public class Processor
{
private SqlCommand
            m_rCmd;
private DataServer m_rParent;
internal
            Processor(DataServer Parent, SqlCommand SqlCommand)
{
m_rCmd =
            SqlCommand;
m_rParent = Parent;
}

We will be making use of the StringBuilder class so we have referenced the System.Text namespace. We hold a reference to the DataServer class (our parent) and mark the constructor as internal (similar to VB's PublicNotCreateable class attribute) to make sure that only this assembly can create instances of this class. The SqlCommand object that we are holding a reference to is the public SqlCommand object of the DataServer class.

public void UpdateEmployee(object Caller, InternalEmployee Employee)
{
StringBuilder sb = new StringBuilder("EXEC usp_update_employee ");
sb.Append(Employee.ID);
sb.Append("," + DataServer.GetNVarChar(Employee.FirstName));
sb.Append("," + DataServer.GetNVarChar(Employee.LastName));
sb.Append("," + DataServer.GetDateTime(Employee.DateOfBith));
m_rParent.AppendCommand(Caller, sb.ToString(),
IsolationLevel.ReadCommitted);
}

The UpdateEmployee method takes a vtEmployee structure and the caller and creates an SQL command that it sends to the DataServer 's buffer. It uses the static GetNVarChar method to encode the text and the static GetDateTime method to encode the date. This is where the isolation level decision is made.

public void DeleteEmployee(object Caller, Int32 EmployeeID)
{
StringBuilder sb = new StringBuilder("EXEC usp_delete_employee ");
sb.Append(EmployeeID);
m_rParent.AppendCommand(Caller, sb.ToString(),
IsolationLevel.ReadCommitted);
}
public void UpdateAddress(object Caller, InternalAddress Address)
{
StringBuilder sb =
new StringBuilder("EXEC usp_update_employee_address ");
sb.Append(Address.ID);
sb.Append("," + DataServer.GetNVarChar(Address.Name));
sb.Append("," + DataServer.GetNVarChar(Address.FlatNumber));
sb.Append("," + DataServer.GetNVarChar(Address.FlatHouseName));
sb.Append("," + DataServer.GetNVarChar(Address.StreetNumber));
sb.Append("," + DataServer.GetNVarChar(Address.StreetName));
sb.Append("," + DataServer.GetNVarChar(Address.Locality));
sb.Append("," + DataServer.GetNVarChar(Address.City));
sb.Append("," + DataServer.GetNVarChar(Address.County));
sb.Append("," + DataServer.GetNVarChar(Address.Country));
sb.Append("," + DataServer.GetNVarChar(Address.PostCode));
m_rParent.AppendCommand(Caller, sb.ToString(),
IsolationLevel.ReadCommitted);
}
public void DeleteAddress(object Caller, Int32 AddressID)
{
StringBuilder sb =
new StringBuilder("EXEC usp_delete_employee_addresses ");
sb.Append(AddressID);
m_rParent.AppendCommand(Caller, sb.ToString(),
IsolationLevel.ReadCommitted);
}

The DeleteEmployee, UpdateAddress and DeleteAddress methods use the same methodology to complete their tasks.

The Update Class

The Update class is the class in which we will store all data access code that will modify the database and return data as well.

using System;
using System.Data;
using System.Data.SqlClient;
using System.Data.SqlTypes; // SqlDateTime.Null

namespace DataTier
{
public class Update
{
private SqlCommand m_rCmd;
private DataServer m_rParent;
internal Update(DataServer Parent, SqlCommand SqlCommand)
{
m_rCmd = SqlCommand;
m_rParent = Parent;
}

We will be making use of the SqlDateTime.Null value for DateTime fields so we have referenced the System.Data.SqlTypes namespace. We hold a reference to the DataServer class (our parent) and mark the constructor as internal. The SqlCommand object that we are holding a reference to is the public SqlCommand object of the DataServer class.

public Int32 CreateEmployee(object Caller, InternalEmployee Employee)
{
m_rCmd.Parameters.Clear();
m_rCmd.CommandText = "usp_create_employee";
m_rCmd.Parameters.Add("@first_name", SqlDbType.NVarChar,
DataServer.NameTextSize).Value =
Employee.FirstName;
m_rCmd.Parameters.Add("@last_name", SqlDbType.NVarChar,
DataServer.NameTextSize).Value =
Employee.LastName;
m_rCmd.Parameters.Add("@date_of_birth", SqlDbType.DateTime).Value =
Employee.DateOfBith==DateTime.MinValue?
SqlDateTime.Null:Employee.DateOfBith;
m_rCmd.Parameters.Add("@id", SqlDbType.Int).Direction =
ParameterDirection.Output;
m_rParent.ExecuteNonQuery(Caller, true, IsolationLevel.ReadCommitted);
return m_rCmd.Parameters["@id"].Value==DBNull.Value?
0:(Int32)m_rCmd.Parameters["@id"].Value;
}

The CreateEmployee method takes a vtEmployee structure and the caller and builds the command on the DataServer 's public command object. It then calls ExecuteNonQuery on the DataServer, passing in the caller, the desire for a transaction, and the isolation level. Finally it extracts the output parameter and returns it.

public Int32 CreateAddress(object Caller, Int32 EmployeeID, InternalAddress Address)
{
m_rCmd.Parameters.Clear();
m_rCmd.CommandText = "usp_create_employee_address";
m_rCmd.Parameters.Add("@employee_id",
SqlDbType.Int).Value = EmployeeID;
m_rCmd.Parameters.Add("@address_name",
SqlDbType.NVarChar,
DataServer.NameTextSize).Value = Address.Name;
m_rCmd.Parameters.Add("@flat_number",
SqlDbType.NVarChar,
DataServer.NumberTextSize).Value =
Address.FlatNumber;
m_rCmd.Parameters.Add("@flat_house_name",
SqlDbType.NVarChar,
DataServer.NameTextSize).Value =
Address.FlatHouseName;
m_rCmd.Parameters.Add("@street_number",
SqlDbType.NVarChar,
DataServer.NumberTextSize).Value =
Address.StreetNumber;
m_rCmd.Parameters.Add("@street_name",
SqlDbType.NVarChar,
DataServer.NameTextSize).Value =
Address.StreetName;
m_rCmd.Parameters.Add("@locality",
SqlDbType.NVarChar,
DataServer.NameTextSize).Value =
Address.Locality;
m_rCmd.Parameters.Add("@city",
SqlDbType.NVarChar,
DataServer.NameTextSize).Value =
Address.sCity;
m_rCmd.Parameters.Add("@county",
SqlDbType.NVarChar,
DataServer.NameTextSize).Value =
Address.County;
m_rCmd.Parameters.Add("@country",
SqlDbType.NVarChar,
DataServer.NameTextSize).Value =
Address.Country;
m_rCmd.Parameters.Add("@post_code",
SqlDbType.NVarChar,
DataServer.NumberTextSize).Value =
Address.PostCode;
m_rCmd.Parameters.Add("@id", SqlDbType.Int).Direction =
ParameterDirection.Output;
m_rParent.ExecuteNonQuery(Caller, true, IsolationLevel.ReadCommitted);
return m_rCmd.Parameters["@id"].Value==DBNull.Value?
0:(Int32)m_rCmd.Parameters["@id"].Value;
}

The CreateAddress method works in exactly the same manner.

The Select Class

The Select class is the class in which we will store all data access code that will return data.

using System;
using System.Data;
using
            System.Data.SqlClient;
using System.Collections;

namespace
            DataTier
{
public class Select
{
private SqlCommand
            m_rCmd;
private DataServer m_rParent;
internal
            Select(DataServer Parent, SqlCommand SqlCommand)
{
m_rCmd =
            SqlCommand;
m_rParent = Parent;
}

We will be making use of the ArrayList class for our collections so we have referenced the System.Collections namespace. We hold a reference to the DataServer class (our parent) and mark the constructor as internal. The SqlCommand object that we are holding a reference to is the public SqlCommand object of the DataServer class.

public vtEmployee GetEmployee(object Caller, Int32 EmployeeID, out ArrayList Addresses)
{
InternalEmployee vEmployee = new InternalEmployee();
vEmployee.ID = EmployeeID;
m_rCmd.Parameters.Clear();
m_rCmd.CommandText = "usp_get_employee";
m_rCmd.Parameters.Add("@id", SqlDbType.Int).Value = vEmployee.ID;
m_rCmd.Parameters.Add("@first_name",
SqlDbType.NVarChar,
DataServer.NameTextSize).Direction =
ParameterDirection.Output;
m_rCmd.Parameters.Add("@last_name",
SqlDbType.NVarChar,
DataServer.NameTextSize).Direction =
ParameterDirection.Output;
m_rCmd.Parameters.Add("@date_of_birth",
SqlDbType.DateTime).Direction =
ParameterDirection.Output;
// We must read the data BEFORE we can get the Output Parameters
System.Data.SqlClient.SqlDataReader rReader =
m_rParent.ExecuteReader(Caller, false,
IsolationLevel.RepeatableRead);
Addresses = new ArrayList();
while(rReader.Read())
{
InternalAddress vAddress = new InternalAddress();
vAddress.ID = rReader.GetInt32(0);
vAddress.Name = rReader.GetString(1);
if(!rReader.IsDBNull(2))
vAddress.FlatNumber = rReader.GetString(2);
if(!rReader.IsDBNull(3))
vAddress.FlatHouseName = rReader.GetString(3);
if(!rReader.IsDBNull(4))
vAddress.StreetNumber = rReader.GetString(4);
if(!rReader.IsDBNull(5))
vAddress.StreetName = rReader.GetString(5);
if(!rReader.IsDBNull(6))
vAddress.Locality = rReader.GetString(6);
if(!rReader.IsDBNull(7))
vAddress.City = rReader.GetString(7);
if(!rReader.IsDBNull(8))
vAddress.County= rReader.GetString(8);
vAddress.Country = rReader.GetString(9);
vAddress.PostCode = rReader.GetString(10);
Addresses.Add(vAddress);
}
// Close the Reader to free up the Connection
rReader.Close();
// Output Parameters are now available
vEmployee.FirstName =
(m_rCmd.Parameters["@first_name"].Value != DBNull.Value)?
(string)m_rCmd.Parameters["@first_name"].Value : "" ;
vEmployee.LastName =
(m_rCmd.Parameters["@last_name"].Value != DBNull.Value)?
(string)m_rCmd.Parameters["@last_name"].Value : "" ;
vEmployee.DateOfBith =
(m_rCmd.Parameters["@date_of_birth"].Value != DBNull.Value)?
(DateTime)m_rCmd.Parameters["@date_of_birth"].Value :
DateTime.MinValue;
return vEmployee;
}

The GetEmployee method takes the Employee's ID and the caller and builds the command on the DataServer 's public command object. It then calls ExecuteReader on the DataServer, passing in the caller, the desire for a transaction, and the isolation level. It then reads the list of addresses into an ArrayList and then extracts the output parameters. It then closes the reader to free up the connection and returns.

Business Tier Design and Coding

For this demonstration, we will be creating a C# console application (.exe file) to mimic the business tier. We shall call the console application (and its namespace) DataTierClient. We will create a class called Utility to mimic a typical business tier class and let the main procedure mimic a client.

using System;
using System.Collections;
using DataTier;

namespace DataTierClient
{
public class Utility
{
public Utility(){}

Nothing new here! We are referencing the DataTier and System.Collections namespaces.

public void CleanAddresses(Int32
            EmployeeID)
{
DataServer Data = new DataServer();
ArrayList
            aAddresses = null;
InternalEmployee vEmployee
            =
Data.Select.GetEmployee(this, EmployeeID, out
            aAddresses);
for(int
            i=0;i<aAddresses.Count;++i)
{
InternalAddress vAddress =
            (InternalAddress)aAddresses[i];
if(vAddress.Country.StartsWith("Z"))
Data.Processor.DeleteAddress(this,
            vAddress.ID);
else
            if(vAddress.Country=="UK")
{
vAddress.Country = "United
            Kingdom";
Data.Processor.UpdateAddress(this,
            vAddress);
}
}
}

The CleanAddresses method is used to implement the following business task: given an employee ID, iterate through all of his or her addresses and either delete it if the country begins with a Z or change the country to "United Kingdom" if it currently reads "UK".

using System;

namespace
            DataTierClient
{
class StartClass
{
///
            <summary>
/// The main entry point for the
            application.
/// </summary>
[STAThread]
static void
            Main(string[] args)
{
Utility rUtility = new
            Utility();
rUtility.CleanAddresses(1);
}
}
}

The Main method is used to mimic a client request coming in for our pretend business server.

The Sample Application

There is not much to see visually with this application. It is, after all, a Data Tier application. However, that said, one can gain a great deal out of running a trace against the database using the Profiler tool that ships with SQL server. That should demonstrate what is happening.

Application Setup

After running the SQL Script file to set up the database, simply create two instances of VS.NET and load the DataTierClient and DataTier solutions respectively. Then simply step through the DataTierClient application in debug mode (it is assumed that the database is residing on the same machine as the VS.NET projects and that the 'sa' password is blank. If not, please refer to the ReadMe.txt file).

Case Study Review

This case study has set out to suggest a methodology to use when implementing a single-server data tier. It has investigated past mistakes and technologies and tried to amalgamate the information into a technical framework.

The various methods of retrieving data are discussed. These include using output parameters (including the return value) and result sets, as well as the two methods of reading the result set; namely by using a data reader or populating a data set. Microsoft's metrics are provided to compare these methods.

Next, past mistakes made by developers were investigated and explained, and the most commonly accepted solutions and suggestions followed on from this. Examples included not applying database theory and principles (like using indexes and set operations), letting in network latency (round-tripping) and not understanding locking and blocking principles. A solution was then suggested. This solution incorporated the technologies and avoided the mistakes already discussed.

We then created a database and some tables and stored procedures to demonstrate our solution. Several interesting and important peripheral facts are discussed and demonstrated here with respect to the choice of indexes and the layout of the stored procedures, for example we look at the reason why we have chosen a non-clustered over a clustered index. While not directly related to our framework, they are crucial to a good data tier and are included for completeness.

Of the COM+ Services, we used construction strings, object pooling, and client context loading, as the former is useful for administration and the latter two will increase our performance and scalability.

The DataServer class that we create is nothing by itself, and needs to interact with the Select, Update and Processor classes. We use communication by internal methods to create a flexible framework that will minimize roundtrips to the database, yet control transaction logic.

The business tier can then use this DataServer class whilst being relatively ignorant of the data logic below. This approach, I feel, will give the business tier programmer flexibility whilst not reducing performance nor scalability. It can actually enhance it in some cases.

What the reader (hopefully) is left with is a good methodology and the information of SqlClient to help him or her utilize it efficiently.

Any Limitations or Further Work

There are a few limitations of this approach. One limitation is the fact that the DataServer class needs to be passed around, which is cumbersome and inelegant. A better approach would be to 'hide' it in the context or call sink (that would be another article in itself!).

Another limitation is that the root object cannot call methods on itself, otherwise it would commit transactions prematurely. It would be nice to have the root object be known by the call chain. An observation on this methodology is the fact that once a transaction has begun, the isolation level cannot be changed, but this would be the case anyway if one were to code it by hand.

Further work on this would be to hide the DataServer object and let the other data tier classes use attributes to indicate their transactional requirements (like declarative transactions in COM+ Enterprise Services). Interception would then be used to interact and control the transaction(s).

Another piece of performance-enhancing work would be to somehow map a structure that you wish the output data to reside in to the output buffer in the internal TdsParser object. This would eliminate an extra copy operation for ALL the data returned. This approach would be similar to the OLE DB Consumer templates provided in ATL VC++. This again would probably be an article in itself!

C#Today - © 2002 WroxPress Limited


A .NET Data Provider

A .NET Data Provider
http://www.csharptoday.com/content.asp?id=1760
Article Rating: not yet rated
Published on March 25, 2002
Abstract
In this case study, Todd Clemeston covers what a .NET data provider is and examines the reasons why you might consider writing one. Todd also summarizes the guidelines that a custom provider needs to follow in order to “play nicely?in the .NET framework. Once the basics are covered, Todd will walk you through the implementation of a basic data provider that supports querying of the Windows Management Instrumentation (WMI) API. To round out the piece, Todd will demonstrate the use of the custom data provider by covering a simple.NET Windows application that displays the results of WMI queries in a ListView.
Case Study

In this case study, Todd Clemetson covers what a .NET data provider is and examines the reasons why you might consider writing one. Todd also summarizes the guidelines that a custom provider needs to follow in order to "play nicely" in the .NET framework. Once the basics are covered, Todd will walk you through the implementation of a basic data provider that supports querying of the Windows Management Instrumentation (WMI) API. To round out the case study, Todd will demonstrate the use of the custom data provider by covering a simple.NET Windows application that displays the results of WMI queries in a ListView.

Before continuing please take note of the following environmental requirements for the purposes of this discussion, as well as for working with the sample code accompanying the case study:

  • .NET RTM
  • Visual Studio .NET RTM (or a text editor)

What is a .NET Data Provider?

A .NET data provider is a set of specialized classes that implement standard ADO.NET interfaces and serve as a bridge between a data source and .NET. Two data providers are included with the .NET framework: SQL Server and OLE DB. The SQL Server data provider connects to SQL Server Versions 7.0 and 2000, and the OLE DB data provider connects to OLE DB data sources. Additionally, Microsoft has made an ODBC data provider available that you may download. As its name implies, this data provider connects to ODBC based data sources. If you wish to download this provider, there is a link at the end of this case study.

Why would you ever need to write your own data provider within .NET? With the addition of the ODBC data provider mentioned above and the various XML and File IO classes that .NET provides, it is uncommon to find yourself in a situation where writing a custom data provider is the only solution. One major reason you might be forced into writing a custom data provider is if the .NET framework does not supply a data provider for a particular data source. In this case, the underlying data source technology may be old or so new that one hasn't been provided (maybe you are the designer of the data source!) A more interesting reason for writing your own data provider might be that you want all data access to go through a set of standard interfaces (i.e. ADO.NET interfaces). Writing a .NET Data Provider to encapsulate a data source allows your data access code to work with standard interfaces and not be the least bit concerned that the underlying data source is Joe's 3rd party data store and not SQL Server or Oracle. Along the same lines, having a standard set of interfaces also plays nicely for those .NET GUI components that integrate with ADO.NET providers (the DataGrid for example). Finally, creating a .NET Data Provider for a data source also helps you integrate with 3rd party products that use ADO.NET such as Crystal Reports for .NET, etc.

Guidelines for a .NET Data Provider

There really is no comprehensive specification on creating data providers within .NET. The help that ships with the RTM (see "Getting Started with a .NET Data Provider Implementation within VS.NET") defines the ADO.NET interfaces, offers suggestions for overloaded constructors, and generally gets you off to a good start. It also provides source code that implements a basic provider that may help you get the gist of what each method and property is intended to do. I have found one other article on the internet that describes how to write a custom .NET data provider. You may find the link to it at the end of this case study. Additionally, Wrox has published a book entitled "Professional ADO.NET" that does a good job covering .NET data provider construction. My recommendation is to take the "shotgun" approach and read as much material on the subject as possible. Study the help files that ship within .NET, visit the links at the end of this case study, and of course read this case study!

In the next few sections, I will attempt to provide some general guidelines to follow when writing .NET data providers. The intent of these sections is not to regurgitate the help that ships with .NET. Instead these sections highlight some guidelines that are of the most use. When we get down to the details of implementing a custom provider, I will go into specifics that you probably won't glean from the .NET help.

Namespaces

Let's start off with a relatively simple guideline. Actually, this should really be a requirement: data providers should use a unique namespace. In other words, don't put your custom .NET data provider in another data provider's namespace. For reference, the ADO.NET namespaces are listed below:

Namespace | Description
System.Data | Defines common interfaces for ADO.NET
System. Data. Common | Classes that are common across the .NET data providers
System. Data. SqlClient | Classes specific to the SQL Server data provider
System. Data. OleDb | Classes specific to the OLE DB data provider

Additionally, namespaces should follow the CompanyName.TechnologyName namespace documented in the .NET help. The WMI data provider we will write in this case study will use the CsharpToday.DataProviders.WmiData namespace.

Minimal Interfaces

The minimum set of interfaces that a .NET data provider should implement are listed below. We will cover each one of these interfaces in detail when we get into the source code that accompanies this case study.

Interface Name | Description
IDbConnection | A connection to a data source.
IDbCommand | A command or query used against a data source.
IDataReader | Represents forward - only/read-only access to data from the underlying data source.
IDbDataAdapter | Provides the link between a data source and a .NET Dataset.

Interfaces play an important role in flexibility. Writing code against interfaces gives programmers the ability to swap out chunks of code with little or no ill-effects to the source code using those interfaces. The power and flexibility of interfaces is what drives the requirement that .NET data providers implement standard ADO.NET interfaces. This requirement affords developers using your provider the choice of writing code against the standard ADO.NET interfaces listed above, or against the concrete classes that a particular data provider offers (SqlConnection, SqlCommand, etc in the case of the SQL Server Data Provider). In cases where the data access layer within a framework must support a number of current and future data source technologies, it makes sense for the layer to work with the interfaces supplied by ADO. NET. Other projects may not require this much flexibility. The nature of the application may dictate a single database platform, and the programmer may be safe to code against concrete classes. In summary, the requirement that all data providers implement a standard set of interfaces is important in that it leaves the decision of flexibility up to the developer using your provider.

Interface "No-ops"

You are not forced to implement every property and method that an ADO.NET interface specifies. For instance, if the implementation of your data provider will not support the concept of a Transaction, it is acceptable to throw a NotSupportedException. It is up to the author of the data provider to implement methods that make sense and to take appropriate action when methods and properties don't apply. The bottom line: use common sense ! As you will see in subsequent sections, the data provider that we will examine in this case study throws exceptions for areas that don't apply to the WMI API. When we encounter these areas, I will provide you with an explanation of what you would have to do to implement the specific functionality.

Compiling

We will close the guideline section with a relatively simple one. Each Data Provider should exist in its own assembly. Additionally, in order to implement the required interfaces, each assembly should reference the System.Data.dll. If you use VS.NET, simply using the Class library wizard helps you adhere to this guideline as it references this assembly.

Creating a Custom Provider

So far we have covered the basics of .NET data providers. You now know what type of data providers are included in the .NET framework, you understand the two major reasons for writing a custom data provider, and you had a quick overview of the major guidelines to keep in mind when implementing a data provider. With the basics out of the way, we are almost ready to take a look at a sample data provider. Before doing so, let's examine the process of writing a provider from a very high level by breaking the process into the following steps:

  1. Select Interfaces and determine applicability of Functionality - Aside from the standard interfaces we covered earlier, there a few other interfaces you might wish to implement (e.g. IDbDataParameter). Examine your requirements and determine the interfaces you need to implement. Within each interface, highlight methods applicable to your provider and note methods or properties that do not apply.
  2. Determine a namespace - Determine what namespace your data provider will occupy.
  3. Create the Assembly - This is somewhat of a no-brainer. Simply use the VS.NET IDE to create a class library. By default the wizard includes a reference to System.Data.dll.
  4. Implement the interfaces - In this step, you write the necessary code to implement the interfaces you selected in step 2. A few words of wisdom follow. If you are the type of programmer who compiles often and can't stand compiler errors as you code, consider providing "thin" implementations of all the interfaces before you start any hardcore coding. This suppresses compiler errors, for instance, when implementing the IDbCommand class which makes reference to the Connection class. Also, after providing a "thin" implementation of each interface, it generally makes sense to implement the interfaces in the following order: IDbConnection, IDbCommand, IDataReader, IDbDataAdapter, however it really doesn't make a difference which order you implement them in.
  5. Test - Had to throw this one in here. Be sure to test your data provider extensively. I recommend installing NUnit which is a freely available unit-testing framework from SourceForge. I provide a link to NUnit at the end of the case study.

The remainder of this case study focuses on the creation of a custom data provider that wraps a piece of the WMI API for querying and retrieving WMI results. Once we cover implementing the provider, we will look at a sample Windows application that uses the WMI provider to query a server. I kept a few things in mind when choosing a data source technology for this case study's custom data provider: I wanted to make sure I didn't use an example that another author has used, and I wanted to stick with a data source technology that was relatively simple to understand and didn't take away the focus of the case study. The WMI API fits both of these requirements. One of the disadvantages of choosing WMI as a data source is that it may make some aspects of creating an ADO.NET data provider seem simpler than they really are. I will speak more on this issue when we implement the IDataReader interface. One last note: the WMI provider we will create requires a reference to the Microsoft WMI Scripting V1.1 Library. If you don't have this COM object on your machine, you may download it from Microsoft's web site. A link to downloading the WMI SDK is provided at the end of this case study.

What is WMI?

In short, WMI is a management architecture that lets you remotely manage computer systems. Some of the things you can do with WMI that pertain to the WMI provider we will examine include listing all of the processes running on a remote machine or listing all the NTFS partitions on a server that have a specific percentage of free disk space. You may also perform operations such as starting/stopping services or rebooting machines. There is much more to the WMI API than I would care to go into at this point. If you would like more details on WMI, follow the WMI link at the bottom of this case study. The case study describes WMI and its architecture. Alternatively, you can read the case study "WMI for .NET" on C#Today (http://www.csharptoday.com/content.asp?id=1807&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq). Now that you have a basic understanding of WMI, let's examine some fictitious criteria that our WMI provider must meet:

  • Connect - The WMI data provider must be able to connect to a server that supports WMI. We will create a class named WmiConnection that manages this connection. This class will implement the ADO.NET IDbConnection interface.
  • Query - The custom provider must also support querying a server for WMI data. We will create a class named WmiCommand that encapsulates a WMI Query. This class will implement the ADO.NET IDbCommand interface.
  • Moving through Results - We also need a mechanism for calling programs to iterate through the result set that is returned from WMI. We will create a WmiDataReader class to handle this functionality. This class will implement the ADO.NET IDataReader interface.
  • Must support the .NET Dataset - The provider must support populating a dataset with the results of a WMI query. The WmiDataAdapter class will handle this requirement. This class will implement the ADO.NET IDataAdapter interface.
  • Starting/Stopping a Service - For WMI queries against Services running on a WIN32 server (i.e. SELECT * FROM WIN32_SERVICE ) we want users of the provider to be able to change the State of the service and effectively stop or start a service. The WmiDataAdapter class will handle this piece of functionality through the standard Update method exposed within the IDataAdapter interface.

Now that we have defined the functionality for the WMI provider, let's dive into some code. The first interface we will implement is the IDbConnection interface.

WmiConnection

As mentioned earlier, the IDbConnection interface defines a connection to a data source. We will implement this interface through a class called WmiConnection. The WmiConnection class manages the connection to a server that supports WMI. The details of the IDbConnection interface are depicted below. As we implement each method and property, I will provide a description and walk you through the accompanying source code.

In the guidelines section at the beginning of this case study, I mentioned that it is safe to throw exceptions when interface methods are not applicable. We will see this guideline in action within the WmiConnection class because it doesn't have the concept of a timeout, a database, or transactions. As outlined earlier, we will handle these situations by throwing a NotSupportedException. In cases where properties are not applicable, we will perform some "magic" by returning a consistent value and implementing an empty Set method.

Let's take a look at a partial class definition for WmiConnection:

public class WmiConnection : IDbConnection
{
/// <summary>
/// Represents the state of the WmiConnection.
/// Possible values are: Open, Closed, Broken,
/// Connecting, Executing and Fetching.
/// </summary>
private ConnectionState _connectionState;
/// <summary>
/// The connection string used to connect to
/// the server. Although I named this
/// connection string, in our case it is the name of the server
/// that we will be connecting to in order
/// to issue WMI queries.
/// </summary>
private string _connectionString;
/// <summary>
/// Used to connect to a server that can
/// process WMI requests.
/// </summary>
private WbemScripting.SWbemLocator _locator;
/// <summary>
/// WMI object that represents an authenticated
/// connection to a namespace. This object
/// is exposed as an internal property for
/// Reader and Adapter classes to use for
/// the purpose of executing WMI queries.
/// </summary>
private WbemScripting.ISWbemServices _services;
.
.
.

}

Note that the definition of WmiConnection includes implementing the IDbConnection interface. For a description of each member variable used within the Connection class, examine the comments that precede each variable declaration. Let's examine the properties that the WmiConnection class contains.

Properties

This section documents the properties that make up the WmiConnection class. Properties that have IDbConnection in parenthesis indicate that the IDbConnection interface dictates the specific implementation.

State (IDbConnection) - The State property describes the current state of the connection through the ConnectionState enumeration. The different states in this enumeration are listed below:

Value | Description
Broken | The underlying connection is broken and may only occur once the connection is open.
Closed | The underlying connection to the data source is closed.
Connecting | The connection to the data source is in progress.
Executing | The connection to the data source is in the process of executing a command.
Fetching | The connection is in the process of retrieving data from the data source.
Open | The underlying connection to the data source is open.

Note that this property is read-only as it only implements the get function. The underlying _connectionState variable is set internally by the connection object only. We certainly don't want code outside of the connection class modifying the state of the connection.

public ConnectionState State
{
get
{
return _connectionState;
}
}

ConnectionString (IDbConnection) - The ConnectionString property defines how to
            connect to a particular data source. If you have written code to
            access a RDBMS such as SQL server or Oracle, you most likely have
            had to specify a connection string in order to connection to the
            database server. I chose to keep the connection string simple for
            the WMI data provider. It is simply used to specify the name of the
            server to connect to. Note that we could extend this property to
            support a semi-colon separated list of parameters which also
            includes the user name and password. In this case we would have then
            had to parse the connection string when a calling program invoked
            the connection's Open method. Note
            that this property is exposed as a read/write property.

public string
            ConnectionString
{
get
{
return
            _connectionString;
}
set
{
_connectionString =
            value;
}
}

ConnectionTimeout (IDbConnection) - The ConnectionTimeout property represents the
            amount of time that expires before a connection attempt fails and an
            error is generated. Although the ConnectionTimeout property does not apply to
            the WmiConnection connection class, we
            still must provide a body for it. Here we are just returning 0 when
            the value is requested. The set implementation below has an empty
            body and does not maintain any state.

public int
            ConnectionTimeout
{
get
{
return 0;
}
set
{
}
}

Database (IDbConnection) -
            The Database property is the name of the database being referenced
            by the connection. The IDbConnection
            interface defines this property as read-only. This property is
            applicable to relational databases, but is not applicable to the WMI
            provider. The implementation will simply return an empty string.

public string
            Database
{
get
{
return "";
}
}

WmiServices - The WmiServices property is not part of the
            IDbConnection interface, but needs to
            be exposed by the WmiConnection class
            so that other WMI data provider classes (namely the WmiCommand and WmiDataAdapter) can leverage it to query for
            WMI data. The WmiServices property is declared as the ISWbemServices interface. This interface
            exposes a method called ExecQuery that
            takes a WMI query and returns a result set. We will cover this
            method and the result set in more detail later on. Note that this
            property is marked as internal within the scope of the WmiConnection class. The internal access
            modifier makes a member publicly available to all code within an
            assembly, but hides it from classes outside of the assembly. We
            don't want the connection class making the underlying WMI API
            available.

internal ISWbemServices
            WmiServices
{
get
{
return
            _services;
}
}

Methods

The following methods define the behavior of the WmiConnection class. In the case of the
            properties above, methods marked with the IDbConnection interface in parenthesis imply
            that the method is a requirement of the IDbConnection interface.

CreateCommand (IDbConnection) - The CreateCommand method is essentially a
            factory method that creates the appropriate Command object associated with a provider
            connection class. A factory method is a creational design pattern
            that is responsible for creating and returning objects that derive
            from a common abstract type. The WMI provider's CreateCommand method creates and returns
            objects that implement the IDbCommand
            interface. Even more important is the fact that the CreateCommand
            method returns an instance of a WmiCommand object, the concrete command
            object that the WMI provider knows how to work with. We will dive
            into the details of the WmiCommand
            class in the next major section. Here is the CreateCommand implementation for the WMI
            provider:

public IDbCommand
            CreateCommand()
{
return new WmiCommand();
}

Open (IDbConnection) - The
            IDbConnection Open method is responsible for opening up a
            connection to a data source given the ConnectionString property of the Connection object. The WmiConnection. Open () method below connects to a server
            that will respond to WMI requests. I kept the call to the WMI
            locator object method ConnectServer
            simple and only pass in the name of server by passing the entire
            connection string. The implementation for the Open method is shown below:

public void Open()
{
if(this.State ==
            ConnectionState.Open)
{
return;
}
if(ConnectionString !=
            "")
{
_services =
_locator.ConnectServer(ConnectionString,
null, null, null, null, null,
0, null);
this._connectionState =
            ConnectionState.Open;
}
else
{
throw new
            Exception("Invalid Connection String.");
}
}

This method first checks the state of the connection before
            opening it. If the calling program inadvertently tried opening a
            connection that is already open, we don't want to run through the
            process of re-connecting. Depending upon your implementation, you
            may wish to throw an exception, refresh the connection by simply
            closing and opening the connection, or do nothing at all since it is
            already open. Once the Open method
            verifies that the connection string is valid, it connects to the
            server and sets the state of the connection to open. Here is the
            signature to the ConnectServer
            method:

ConnectServer(Server, Namespace, User,
            Password, Locale, Authority, SecurityFlags,
            NamedValueSet)

Argument | Description
Server | Specifies the name of the computer to connect to.
Namespace | Specifies the namespace with which you log on.
User | The user name to use when connecting to the server.
Password | The password to use when connecting to the machine.
Locale | Specifies the locale from which information should be retrieved.
Authority | Specifies the authentication to be used. Should only be used with connections to remote WMI servers.
SecurityFlags | Allows you to pass flag values to ConnectServer. Passing 0 tells the ConnectServer method to return only after a connection to the server has been made.

Close (IDbConnection) - The
            IDbConnection.Close method must
            release resources that are associated with the connection. The WMI
            provider's simple implementation sets the services and locator
            objects to null in addition to setting the state of the connection
            to Closed.

public void Close()
{
_services =
            null;
_locator = null;
this._connectionState =
            ConnectionState.Closed;
}

Constructors -At a minimum, the Connection class should implement a default
            constructor in addition to an overloaded one that takes a string
            that represents the connection string of the connection. Both
            constructors should also be sure to initialize the ConnectionState to Closed. Here is the implementations for
            WmiConnection 's two constructors:

public WmiConnection
            ()
{
_connectionState =
            ConnectionState.Closed;
}

public WmiConnection(string
            strConnectionString)
{
_connectionState =
            ConnectionState.Closed;
this.ConnectionString =
            strConnectionString;
}

Unsupported Methods - The WMI provider will not be
            supporting transactions or the concept of changing a database with
            them. Because of this, it is important to throw NotSupportException's for these methods to
            let programs calling the provider know this is the case. I will,
            however, cover the intent of these methods below.

The BeginTransaction method is
            responsible for creating and returning a data provider specific
            transaction object for starting a database transaction. The
            transaction object that gets returned from the BeginTransaction method must support the
            IDbTransaction interface shown
            below:

If you need to support Transactions in your provider, you would
            also need to provide a provider specific implementation of the IDbTransaction interface. The Connection property maintains a reference to
            the data provider specific connection object. The Isolation level is
            an enumeration that describes how locking behavior will occur within
            the transaction. For the different types of isolation levels, look
            up IsolationLevel in the .NET help files. As you probably guessed,
            the Commit method commits the
            transaction and the Rollback method
            rolls the transaction back. If you need to have support in your
            custom provider, you will also need to provide an implementation for
            the Transaction property of your command object.

The other method that the WmiConnection class will not support is the
            ChangeDatabase method; WMI does not
            have the concept of a database. The purpose of the ChangeDatabase method is to change the
            database for a connection object that already has an open
            connection. Although this method really doesn't apply, we could
            extend the WMI provider and equate the ChangeDatabase method to changing the server
            name that we connect to. Under this scenario, the provider would
            have to actually disconnect from the existing server and
            re-establish its connection to the new server.

WmiCommand

Now that we have a method for connecting to a server, we will
            move onto the next requirement which calls for querying a server
            using the WMI Query language. We will implement the WMI query
            functionality by implementing a class called WmiCommand. This class implements the IDbCommand interface. A Command in ADO.NET
            represents a request to a data source. The SqlClient data provider
            in .NET, for instance, has a SqlCommand class that encapsulates the
            concept of a SQL command. The WmiCommand object will encapsulate a WMI SQL
            command. The IDbCommand interface is
            shown in the picture below:

Here is a partial class definition of the WmiCommand that shows the member variables
            of the class. The comments within the class definition describe what
            each member does.

public class WmiCommand :
            IDbCommand
{
/// <summary>
/// The WMI SQL used to
            obtain data.
/// </summary>
private string
            _wmiCommand;
/// <summary>
/// This represents how the
            results
/// of a command will be applied to a DataRow
///
            when the Update method of the DataAdapter
/// is invoked.
///
            </summary>
private UpdateRowSource _updRowSource;
///
            <summary>
/// Represents a connection to a server
///
            that serves up WMI data.
/// </summary>
private
            WmiConnection _connection;
.
.
.
}

Properties

The following is a list of the properties that the WmiCommand class contains.

CommandText (IDbCommand) -
            The CommandText property represents
            the actual query string that gets executed against the data source.
            In the case of the SQL Server and OLE DB data providers, this
            property represents the SQL to be executed against a particular
            RDBMS. The CommandText property for
            the WMI provider is the actual WMI SQL query that will be executed
            against the server.

public string
            CommandText
{
get
{
return
            _wmiCommand;
}
set
{
_wmiCommand =
            value;
}
}

CommandTimeout (IDbCommand)
            - Represents the amount of time to wait before stopping the
            execution of a command. We are not going to support the concept of a
            timeout, so we just simply return 0 to the caller and provide an
            empty set function.

public int
            CommandTimeout
{
get
{
return
            0;
}
set
{
}
}

CommandType (IDbCommand) -
            The CommandType property represents
            the type of command to be executed through the CommandType enumeration. The CommandType enumeration defines the
            following possible types: StoredProcedure, TableDirect, and Text. The data providers in the .NET
            framework support all three of these enumerations. Setting this
            property to StoredProcedure specifies
            that the CommandText property is the
            name of a stored procedure. Setting this property to TableDirect specifies that the CommandText property is the name of a table
            in a database. Setting the property to Text specifies that the CommandText is a query (e.g. SQL or WQL in
            the case of the WMI provider). Because we are only supporting the
            Text option, the implementation below always returns text and only
            supports the setting of text.

public CommandType
            CommandType
{
get
{
return
            CommandType.Text;
}
set
{
if(value !=
            CommandType.Text)
{
throw new NotSupportedException("This
            provider only supports text commands.");
}
}
}

Connection (IDbCommand) -
            The Connection property is simply the
            connection that the Command object is
            associated with. Its implementation is below:

public IDbConnection
            Connection
{
get
{
return
            _connection;
}
set
{
_connection =
            (WmiConnection)value;
}
}

Parameters - The Parameters represents a collection of
            IDataParameter objects used to specify
            the parameters of a SQL statement or stored procedure. The WMI
            custom provider will not support parameters, so it just simply
            returns null in its implementation. If you need to implement
            parameters, you must create a ParameterCollection class that implements
            the IDataParameterCollection. You will
            also need to create your own DataParameter class implementing the IDataParameter interface.

public IDataParameterCollection
            Parameters
{
get
{
return null;
}
}

Transaction (IDbCommand) -
            The Transaction property represents the transaction which the
            command will execute under. The WmiCommand does not support transactions, so
            we simply return null when asked for a transaction and throw an
            exception when the caller attempts to specify a transaction.

public IDbTransaction
            Transaction
{
get
{
return
            null;
}
set
{
throw
new NotSupportedException("The
            WmiCommand object doesn't " +
"support
            transactions.");
}
}

UpdateRowSource (IDbCommand)
            - Lets a calling program specify the method with which values
            returned from a data source are applied to a row within an ADO.NET
            Dataset. This property comes into play within the Update method of the DataAdapter. The WMI
            provider doesn't support return (or output) values in its Adapter's
            Update method, so the enumeration is
            set to None in
            the constructor and may not be changed. If a calling program
            attempts to change this to any other value, the provider will throw
            an exception.

For the curious, the following table describes the different
            values of the UpdateRowSource
            enumeration, and how they affect the behavior of a command used with
            a DataAdapter:

Name | Description
Both | The output parameters and the first row returned get mapped to the changed row in the Dataset.
FirstReturnedRecord | Data in the first returned row is mapped to the changed row in the DataSet.
None | Returned parameters or rows are ignored.
Output Parameters | Output parameters are mapped to the changed row in the DataSet.

public UpdateRowSource
            UpdatedRowSource
{
get
{
return
            _updRowSource;
}
set
{
if(value !=
            UpdateRowSource.None)
{
throw
new
            NotSupportedException("The WmiCommand " +
"object only supports
            None as an " +
"UpdateRowSource
            type.");
}
}
}

WmiCommand Methods

There are really only two methods of importance for the WmiCommand class. The rest of the methods
            don't apply to the WMI provider, although I will go over them so you
            understand what each means.

ExecuteReader (IDbCommand) -
            ExecuteReader is a factory method that
            returns a DataReader associated with
            the Command class. Remember than an
            IDataReader provides forward-only
            access to data from a data source. The ExecuteReader implementation is shown
            below:

public IDataReader
            ExecuteReader()
{
if(this.Connection == null)
{
throw
            new Exception("Cannot execute reader. The WmiConnection "
            +
"object is not valid.");
}
if(this.Connection.State !=
            ConnectionState.Open)
{
throw new Exception("Cannot execute
            reader. The connection " +
"to the server is
            invalid.");
}
// create a reader
WmiDataReader reader =

new WmiDataReader(this._connection);
// the object set that
            is returned as a result of the
// query.
ISWbemObjectSet
            ObjSet = null;
// execute the query as WMI Query Language

ObjSet =
            _connection.WmiServices.ExecQuery(_queryCommand,
"WQL", 0x10,
            null);
// set the readers data to the object set
reader.Data =
            ObjSet;
// return the reader to the caller
return
            reader;

}

The above code is not overly complex. After verifying that the
            connection is valid, the method creates a WmiDataReader object, executes the WMI
            Query, and sets the WmiDataReader.Data
            property to the result set of the WMI query. We will cover
            the details of the DataReader
            implementation in the next major section, but essentially the DataReader will use the Data property to iterate through the result
            set.

The WMI ExecQuery method
            highlighted in the code above takes the following four
arguments:

  1. WMIQuery - The first specifies the WMI query to execute. The
              syntax of this query follows WQL (WMI SQL), which is a subset of
              the ANSI standard SQL language with WMI-specific extensions.

  2. QueryLanguage - The second argument is the query language to
              use, and must always be "WQL" if anything is specified.

  3. QueryBehavior - The third argument specifies the behavior of
              the query and determines if this call returns immediately. We have
              specified WbemFlagReturnImmediately
              (0x10) for this option.

  4. NamedValueSet - The final argument is NamedValueSet. We have left this as null,
              but you can pass a SwbemNamedValueSet object to give
              contextual information that can be used by a provider servicing
              the request. If a provider supports this argument, its
              documentation will have specified the values, data types, allowed
              values, and semantics.

I mentioned earlier that using WMI as the data source made some
            aspects of authoring a data provider seem simpler than they normally
            are. This is indeed the case for the ExecuteReader() method. When implementing
            the ExecuteReader method, it is common
            to have to write somewhat complex parsing logic to break a query
            apart. In the implementation above, we simply forward the query to
            the WMI COM object.

ExecuteScalar (IDbCommand) -
            The IDbCommand. ExecuteScalar method is meant to be an
            efficient means to access data that is in the form of a single value
            within a single row. A good example of a situation where this method
            is helpful is in the case where a record count from a SQL select
            statement is required. The result of a SELECT
            COUNT(*) FROM Table1 SQL query returns a single column and
            single value. The ExecuteScalar method
            is meant to provide quick access to these types of queries. WMI does
            not support a count type query, but for purposes of this case study,
            I have provided an implementation that invokes the ExecuteReader method above, and returns the
            first column of the first row. Admittedly, this is inefficient, but
            it illustrates the intent of the method.

public object
            ExecuteScalar()
{
if(this.Connection == null)
{
throw
            new Exception("Cannot execute reader. The WmiConnection "
            +
"object is not valid.");
}
if(this.Connection.State !=
            ConnectionState.Open)
{
throw new Exception("Cannot execute
            reader. The connection " +
"to the server is
            invalid.");
}
// execute the reader
IDataReader dr =
            this.ExecuteReader();
dr.Read();
// return the first column
            for the first
// row in the resultset.
return
            dr[0];
}

The following methods throws NotSupportExceptions because WMI doesn't
            support the particular functionality:

  • Cancel - You would provide an implementation for this method
              if you wanted to give a calling program the ability to cancel the
              execution of a command.

  • CreateParameter - If your data provider supports Data
              parameters, you would override this method to create a data
              provider specific instance of an IDataParameter object.

  • ExecuteNonQuery - You would provide an implementation of this
              method if your data provider supported adding, updating, and/or
              deleting of data from your data source. The SQL and OLE DB data
              providers, for instance, use this method to execute INSERT and
              UPDATE SQL statements.

  • Prepare - This method only applies when the CommandType property is set to StoredProcedure. If your data provider
              supports stored procedures, you would write the necessary code to
              compile the command for your data source.

WmiDataReader

The purpose of an ADO.NET DataReader is to provide efficient
            forward-only/read-only access to the results of a command. The DataReader is also responsible for mapping
            the data types of the source data to the Common Language Runtime
            (CLR) data types. The WMI provider's implementation of the
            DataReader is accomplished via the WmiDataReader class. The WmiDataReader class provides access to the
            result set of a WMI query command. Recall that the WmiCommand. ExecuteReader() method executes a WMI Query,
            creates a WmiDataReader, and sets the
            internal Data property of the WmiDataReader to the result set of the
            query. The WmiDataReader implements
            interface methods allowing a calling program to iterate through the
            WMI result set. Note that the WmiDataReader class implements the IDataReader interface. This interface in
            turn implements the IDataRecord
            interface. The IDataReader interface
            is shown below. It consists of a few properties and methods.

The IDataRecord interface (which
            the IDataReader interface implements)
            is comprised of a very large number of "Getter" functions, aimed at
            retrieving individual column values from the current record in the
            result set. A partial interface definition is shown below to give
            you an idea of what this interface looks like:

Here is a partial class definition for the WmiDataReader class, with descriptions for
            each member variable:

public class WmiDataReader:
            IDataReader
{
/// <summary>
/// The
            connection associated with the reader
///
            </summary>
private WmiConnection _connection;
///
            <summary>
/// Holds the number of rows retrieved from the
            last query.
/// </summary>
private int _numRows;
///
            <summary>
/// The resultset of the WMI query.
///
            </summary>
ISWbemObjectSet _data = null;
///
            <summary>
/// The enumerator of the object set.
///
            </summary>
IEnumerator _wmiEnumerator;
///
            <summary>
/// used to map the names of the columns to
            indexes. WMI supports
/// looking up by names, but we need an
            efficient way of looking
/// up column values by index. This
            array list holds this lookup.
/// </summary>
ArrayList
            _indexLookup = new ArrayList();
/// <summary>
///
            reference to the current record in the WMI resultset.
///
            </summary>
ISWbemObject _currentObject = null;
///
            <summary>
/// reference to the property set (i.e collection
            of columns)
/// </summary>
ISWbemPropertySet
            _currentPropertySet = null;
.
.
.
}

Properties

Let's first take a look at the properties of the WmiDataReader class. They are listed
            below.

Data - You got a sneak preview of this property when we explored
            the WmiCommand.ExecuteReader() method.
            This property is marked internal so that the WmiCommand object can set its value after
            executing the WMI query. Recall that the Data property is of the
            type ISWbemObjectSet. You can think of
            the ISWbemObjectSet interface as a
            result set. This object exposes an enumerator that WmiDataReader will utilize to move forward
            through the result set. The implementation of the Data property in the next code block sets
            the private _ data member variable to
            the value passed in. It then sets the _ wmiEnumerator member variable to the ISWbemObjectSet's enumerator followed by
            setting the _ numRows member variable
            to the number of rows in the result set. This private variable will
            be accessed via the RecordsAffected
            property of the DataReader.

internal ISWbemObjectSet
            Data
{
set
{
_data = value;
// we need to keep track
            of the enumerator
// for the object set.
this._wmiEnumerator =
            _data.GetEnumerator();
// set the number of rows to the
            count.
this._numRows = _data.Count;
// build the index lookup
            table.
this.BuildIndexLookup();
}
}

Accessing actual column values within ISWbemObjectSet is accomplished by
            specifying a column name. The IDataReader
            interface, however, must provide access to these values by
            name as well as and by an integer-based index. To support looking up
            values through an index, the WMI provider creates an ArrayList named _ indexLookup that contains the ordered names
            of the columns in the WMI result set. The BuildIndexLookup method that gets invoked
            below builds this list. Methods within the WmiDataReader class will leverage this ArrayList by passing in an index to get a
            column name, and using the column name to get at the data. Here is
            the code for the BuildIndexLookup()
            method:

private void
            BuildIndexLookup()
{
ISWbemObject WbemObj =
            null;
ISWbemPropertySet WbemPropSet = null;

WbemObj =
            this.GetFirstObjectInResultSet();
WbemPropSet =
            WbemObj.Properties_;
// grab the enumerator for the
            properties
IEnumerator e = WbemPropSet.GetEnumerator();
//
            Advance the enumerator's cursor until we reach
// the last
            item
while (e.MoveNext())
{
// grab the current
            property
ISWbemProperty ob = (ISWbemProperty)e.Current;
// add
            the property name to the
            index
_indexLookup.Add(ob.Name);
}

}

The code above takes the first result, grabs the property
            collection, and adds an entry to an array that simply stores the
            name of the property.

RecordsAffected (IDataReader) - Returns the number of rows
            affected by the Command. The number of
            rows may be the result of a SELECT, UPDATE, DELETE, or INSERT query.
            With regards to the WMI provider, this property will be the number
            of records in the result set as represented by the _numRows member variable.

public int
            RecordsAffected
{
get
{
return
            _numRows;
}
set
{
}
}

FieldCount (IDataReader) -
            Returns the number of fields that are in the result set. The WMI
            provider implementation of this property simply forwards the call to
            the WMI PropertySet object's count, as
            shown below:

public int
            FieldCount
{
get
{
return
            _currentPropertySet.Count;
}
}

Depth (IDataReader) - This
            is the nesting depth for the current row. This property applies to
            data providers where queries return multidimensional or hierarchical
            results. The WmiDataReader does not
            support this concept, so the implementation of the Depth property returns 0, and contains no
            logic in the "setter" function.

public int Depth
{
get
{
return
            0;
}
set
{
}
}

IsClosed (IDataReader) -
            Returns a Boolean specifying if the DataReader is closed. The WMI provider in
            this example does not support the concept of being closed so it
            always returns false.

public bool
            IsClosed
{
get
{
return
            false;
}
set
{
}
}

Methods

Constructors - The WmiDataReader contains two constructors.
            Both initialize the _ numRows member
            variable to 0. The overloaded constructor takes a WmiConnection as a parameter and sets the
            internal connection property. Note that both constructors are marked
            internal. Users must not be able to construct readers. Only the
            Command object must create and return
            a reader. Here are the constructor definitions:

internal WmiDataReader()
{
_numRows =
            0;
}

internal WmiDataReader(WmiConnection
            connection)
{
_connection = connection;
_numRows =
            0;
}

Read (IDataReader) - The
            Read method is responsible for moving
            the DataReader to the next record in
            the result set. It returns false once it reaches the end of the
            result set, and true otherwise. The implementation below advances
            the enumerator of the result set. If it's not at the end of the
            results, it sets the _ currentObject
            and _ currentPropertySet variables. If
            it has reached the end, it sets the objects to null.

Here is the implementation:

public bool Read()
{
// move to the
            next result
bool bRet = _wmiEnumerator.MoveNext();
if(bRet ==
            true)
{
// make sure the current object points
// to the
            current item on the enumerator
_currentObject = (ISWbemObject)
            _wmiEnumerator.Current;
// also set the propertyset member
            variable
_currentPropertySet = _currentObject.Properties_;
//
            return the result of the movenext
            operation
}
else
{
_currentObject =
            null;
_currentPropertySet = null;
}
return
            bRet;
}

GetValue (IDataReader) - The
            GetValue method is part of the IDataRecord interface definition. It takes
            an integer as a parameter, and returns the value of a column for the
            given record as a .NET object. The WMI provider's implementation
            verifies that the result set contains data, verifies that the index
            is valid, looks up the column name based on the index, and invokes a
            custom method called GetCurrentValueByName().

public Object GetValue(int
            i)
{
if(_numRows == 0)
{
throw new
            IndexOutOfRangeException("Must first call read");
}
if(i <
            0 || i > this._currentPropertySet.Count)
{
throw new
            IndexOutOfRangeException("No such column");
}
string
            strFieldName =
(string)_indexLookup[i];

return
            this.GetCurrentValueByName(strFieldName);
}

The GetCurrentValueByName method
            simply returns the column value given the name passed to it. Its
            definition is shown below:

private string
            GetCurrentValueByName(string strName)
{
return
            _currentPropertySet.Item(strName,
            0).get_Value().ToString();
}

GetXXX Functions (IDataRecord) -Users of a DataReader may also obtain strongly typed
            values from the result set by invoking one of the many GetXXX methods that make up the IDataRecord interface. Recall that the WmiDataReader class implements the IDataReader interface, which also implements
            the IDataRecord interface. Listing the
            implementations of all of the IDataRecord methods would be a bit mundane,
            but I have provided two implementations of the GetXXX methods below. In general the GetXXX methods invoke the GetValue method we implemented earlier and
            cast the return value to the appropriate type.

public string GetString(int
            i)
{
return this.GetValue(i).ToString();
}

public
            bool GetBoolean(int i)
{
return
            (bool)this.GetValue(i);
}

this[parameter] (IDataReader) -The IDataReader interface also defines two
            separate indexers which allows the implementing class to be treated
            as if it were a collection. There are two implementations of the
            indexer. The first provides access to column values by name; the
            second by an integer index. Implementations for the WMI provider of
            these two indexers follow:

public object this[string
            name]
{
get
{
return
            this.GetCurrentValueByName(name);
}
}

Here is the integer based implementation of the indexer:

public object this[int
            i]
{
get
{
return
this.GetValue(i);
}
}

GetFieldType (IDataReader) -
            The GetFieldType method returns the
            CLR type of the data item at the specified index within the current
            record. WMI uses Common Information Model (CIM) data types to
            represent its data. For more information on CIM, navigate to the CIM
            link found at the end of this case study. It is up to the WmiDataReader to translate CIM to CLR data
            types. After grabbing the property at the specific index, the method
            invokes the ConvertCIMToCLR method
            whose implementation follows:

public Type GetFieldType(int
            i)
{
ISWbemProperty p =
            _currentPropertySet.Item(_indexLookup[i].ToString(),0);

return
            ConvertCIMToCLR(p.CIMType);
}

To simplify converting types, the WmiDataReader simply converts all data types
            to the CLR string data type as shown below. If we wanted to
            elaborate on this further, we could create a switch statement that
            provided a mapping for each CIM type.

private Type ConvertCIMToCLR(WbemCimtypeEnum
            en)
{
return typeof(string);
}

GetName (IDataReader) - The
            GetName method returns the name of the
            field within the result set given the index passed in. Its
            implementation is shown below. Note that it must use the _ indexLookup ArrayList because the WMI PropertySet object looks up column names by
            name. Remember that all the _ indexLookup array does is allow for
            efficient lookup of column names by index.

public string GetName(int i)
{
// grab
            the object given the index passed in
ISWbemProperty ob =

_currentPropertySet.Item(_indexLookup[i].ToString(),0);
//
            return the property name
return ob.Name;
}

GetValues (IDataReader) -
            The GetValues method populates an
            array of objects passed into the method with the data from the
            current record, where each column value represents an object. Note
            that the caller of this method must allocate enough space in the
            object array to satisfy the number of columns. The WMI provider's
            implementation for this method is shown below:

public int GetValues(object[]
            values)
{
// make sure we have a valid
// number of
            properties
if(_currentPropertySet.Count >
            0)
{
if(values.Length < this.FieldCount)
{
throw new
            Exception("You must allocate the " +
"object array with enough
            room to " +
"hold the number of fields in the " +
"row
            set.");
}

int iIndex = 0;
// iterate through the list
            of properties
foreach(ISWbemProperty p in
            _currentPropertySet)
{
// set the value objects to
// the
            value
values[iIndex] = p.get_Value();
}
// return the
            number of values
return values.Length;
}
else
{
// if
            we go here, there weren't any
// properties
values =
            null;
return 0;
}
}

GetOrdinal (IDataReader) -
            The GetOrdinal method takes a name of
            a column and returns the index at which the column name is located.
            The WmiDataReader implementation uses
            the IndexOf method of the string class to provide this functionality
            as shown below:

public int GetOrdinal(string
            name)
{
return _indexLookup.IndexOf(name);
}

Close (IDataReader) - The
            WMI implementation does not have a concept of closing a DataReader. The SQL Server and OLE DB
            providers within .NET only allow one DataReader to be open for each connection,
            and the DataReader must be explicitly
            closed after the programmer is done.

GetSchemaTable (IDataReader)
            - GetSchemaTable returns a DataTable populated with metadata about the
            columns of the reader. This method is useful in that users may
            generically execute and display the results of a query by populating
            a grid full of columns not known at design time. We will create a
            Windows application that utilizes this method to populate the
            headings of a ListView control with column names. The GetSchemaTable implementation below creates
            a table called WMIMetaData, defines a
            ColumnName and DataType column within the table, and
            creates a row for each column in the result set.

public DataTable GetSchemaTable()
{
//
            create a datatable
DataTable dt = new
            DataTable("WMIMetaData");
// add the schema
            columns
dt.Columns.Add("ColumnName",
            typeof(string));
dt.Columns.Add("DataType", typeof(Type));
//
            iterate through the list of properties
foreach(ISWbemProperty p
            in _currentPropertySet)
{
// add a new row
DataRow dr =
            dt.NewRow();
// set the name of the colum and
// the data type
            properties
dr["ColumnName"] = p.Name;
dr["DataType"] =
            this.ConvertCIMToCLR(p.CIMType);
// add the row to the
            table
dt.Rows.Add(dr);
}
return dt;
}

GetDataTypeName (IDataReader) - This method returns the name
            of the data type for the specified field index. The implementation
            below looks up the property object based upon the index, converts
            the data type to a CLR type, and returns the string representation
            of the type.

GetDataTypeName(int i)
{
// grab the
            object given the index passed in
ISWbemProperty ob =
_currentPropertySet.Item(_indexLookup[i].ToString(),0);
//
            determine the CLR type.
Type t =
            ConvertCIMToCLR(ob.CIMType);
// return the name of the
            type
return t.ToString();
}

NextResult (IDataReader) -
            NextResult is applicable to those data
            providers that support multiple result sets. The SQL Client data
            provider supports multiple result sets when SQL statements are
            separated by a semi-colon. Invoking this method tells the DataReader to point to the next result set.
            The WMI provider will not support this concept, so this method
            throws a NotSupportedException.

GetData (IDataReader) - The
            GetData method is used to expose
            nested tables and other hierarchical data. Both data providers
            within .NET do not support this method. We are not implementing this
            method either.

WmiDataAdapter

A .NET DataAdapter provides the
            link between a data source and a .NET Dataset. I assume you have a
            basic understanding of what a .NET Dataset is. If you need further
            details, please see the following article on CSharpToday: http://www.csharptoday.com/content.asp?id=1568&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq.
            The SQL Server and OLE DB implementations of the DataAdapter within the .NET framework
            support populating a Dataset with data
            from a data source in addition to updating a data source given
            changes to the Dataset. Updating of
            the data source is accomplished via the DataAdapter's Update method. This is a relatively easy
            concept to grasp in regards to a database. A typical sequence of
            events might be:

  • Populate a dataset using an adapter

  • Make modifications to a data set by modifying, adding, or
              deleting rows, and

  • invoke the DataAdaper's Update
              method to synchronize the changes back to the data source.

However, WMI's WQL only supports read-only access to the WMI
            data, so we are a limited in what we can do in this regard. However,
            the WMI provider will support the concept of Updating WMI data by
            adding logic to the Adapter's update
            method that inspects the State of a service (e.g., Stopped or
            Started) and stops or starts the service.

The WmiDataAdapter class implements
            the IDbDataAdapter interface. This
            interface derives from the IDataAdapter interface to create the
            following interface definition:

Properties

SelectCommand (IDataAdapter)
            - This is the command that the DataAdapter uses to populate the Dataset. The WMI provider's implementation
            represents this property as an instance of a WmiCommand object:

public WmiCommand
            Command
{
get
{
return
            _command;
}
set
{
_command =
value;
}
}

UpdateCommand, DeleteCommand and InsertCommand (IDataAdapter) - The WMI provider only
            supports the concept of the SelectCommand so we have empty
            implementations of these properties to satisfy the interface. This
            interface is typically geared toward a RDBMS type data provider. I
            could have chosen to just implement the IDataAdapter interface which would remove
            these four commands, but I wanted to illustrate the various commands
            associated with an adapter.

TableMappings (IDataAdapter)
            - This property specifies how a table in the data source maps to a
            DataTable within a DataSet. This allows a program to
            essentially hide the underlying physical table names and exposes
            names within a DataSet that are more "user friendly" to a user or
            calling program. It allows for a physical table named COMP_SALES to
            be represented in a DataSet as Company Sales. Additionally, each
            TableMapping instance contains a
            reference to a collection of DataColumnMapping objects that offer a
            similar service. With a DataColumnMapping, a physical column named
            COMPANY_ID could be represented in a DataSet as Company ID. If you
            need to support TableMappings in a
            provider, you must examine them when you populate a DataSet and when
            callers issue the Update method. We
            will cover these methods later.

MissingMappingAction (IDataAdapter) - This property is an
            enumeration that defines the action taken when a specified table or
            column mapping (e.g. DataTableMapping,
            DataColumnMapping) is missing from the source table or source
            column. Recall our discussion in the previous section and assume
            that a calling program attempts to populate a DataSet. In setting up
            the DataAdapter, the calling program has specified mappings one of
            which maps the COMP_SALES physical table to the Company Sales table
            in a DataSet. If the COMP_SALES table doesn't exist in the database,
            the data provider needs to know what action to take. The MissingMappingAction property specifies this
            action through the MissingMappingAction enumerator which
            provides for the following actions:

  • Error - Throwing an exception

  • Ignore - Ignoring the source tables and columns that failed to
              map. This results in some source data not making it in the Dataset

  • Passthrough - Creating the DataTable and DataColumn that failed to map.

Since the WMI provider does not support mappings, the MissingMappingAction always returns Passthrough (which creates the tables and
            columns in the dataset). The set operation throws an exception for
            any other type but Passthrough.

public MissingMappingAction
            MissingMappingAction
{
get
{
return
            MissingMappingAction.Passthrough;
}
set
{
}
}

MissingSchemaAction (IDataAdapter)
            - While the MissingMappingAction
            specifies the action to be taken when an expected table or column is
            missing from a physical data source, the MissingSchemaAction property specifies the
            action to be taken when a DataTable or
            DataColumn is missing from the schema
            within a Dataset. Possible actions
            include:

  • Add - Adding the necessary columns to the schema

  • AddWithKey - Adding the columns with primary key information

  • Error - Throwing an exception

  • Ignore Ignoring the extra columns

Our implementation only supports the Add member of the enumeration.

public MissingSchemaAction
            MissingSchemaAction
{
get
{
return
            MissingSchemaAction.Add;
}
set
{
}
}

Methods

Constructor - Aside from the default constructor, there
            are three additional constructors that make up the adapter. Their
            signatures and descriptions are below:

// sets the Command property of the
            adapter
public WmiDataAdapter(WmiCommand comm)
// sets the
            Command property of the adapter and also sets the WmiCommand's //
            Connection property.
public WmiDataAdapter(WmiCommand comm,
            WmiConnection conn)
// creates a new WmiCommand and WmiConnection
            objects.
public WmiDataAdapter(string strCommand, string
            strConnection)

Fill (IDbDataAdapter) - This
            method Inserts or Updates rows in the Dataset to match those within the data
            source. The WmiDataAdapter.Fill method
            is shown in the next code block. Its first job is to make sure the
            command object is valid. Note that this method also keeps track of
            the ConnectionState. If the connection
            is closed, it must open the connection and set a flag (bMustCloseConnection) so that the connection
            will be left in its initial state once the Fill operation is complete. It then proceeds
            to fill the schema in the Dataset and
            grabs a reference to the table based upon the table name. It then
            creates a WmiDataReader and iterates
            through it adding a new row to the table for each row in the reader.
            Note the call to AcceptChanges at the
            end of the Fill method. The call to
            AcceptChanges is vital in order to
            keep the state of each row within the DataSet in a consistent state.
            Internally, the DataSet keeps state information for columns within
            each row. It does this through the RowState property associated with each Row.
            The RowState property returns the
            DataRowState which specifies if a row has been Added, Deleted, Detached, Modified, or
            Unchanged. As the fill method adds records to the DataSet,
            each row's RowState is set to the
            Added state. The AcceptChanges method highlighted in the code
            below accepts these inserted rows and changes all of the row's
            RowState to Unchanged. Why is this important? If we removed
            this line and a program was to make modifications to rows inside the
            DataSet, the state will remain marked as Added. Now assume a program
            wants to invoke the Adapter's Update
            method to update the data source with their modifications. The Update method would query the DataSet to
            find out the state of its Data and erroneously determine that a
            bunch of rows have been inserted (because the RowState is set to Added). If the WMI provider supported adding
            rows in this scenario, it would end up duplicating all of the rows
            that the Fill method added. The AcceptChanges puts all rows in a "pristine"
            state by setting the RowState to Unchanged. The importance of this method
            will become clearer when we cover the Update method. Here is the source code to
            the Fill method:

public int Fill(DataSet ds)
{
// make
            sure the command is good
if(_command == null ||
            _command.CommandText.Trim() == "")
{
throw new
            Exception("Cannot fill without receive command");
}
// if the
            connection is closed, we must open it.
// We also need to make
            sure we close it before
// leaving this method.
bool
            bMustCloseConnection = false;
if(_command.Connection.State !=
            ConnectionState.Open)
{
_command.Connection.Open();
bMustCloseConnection
            = true;
}
// fill the schema for this
            query
this.FillSchema(ds, SchemaType.Mapped);
// create the
            table
DataTable dt = ds.Tables[this.CommandTableName];
//
            create a data reader
WmiDataReader dr =
(WmiDataReader)_command.ExecuteReader();

// iterate
            through the reader
while(dr.Read())
{
// create a new
            row
DataRow newRow = dt.NewRow();
// set all of the
            fields
for(int i = 0; i < dr.FieldCount;
            i++)
{
newRow[i] = dr[i];
}
// add the row to the
            table
dt.Rows.Add(newRow);
}
// accept all of the
            changes
ds.AcceptChanges();
// if we had to open up the
            connection to
// perform this operation then we must
// close
            it.
if(bMustCloseConnection)
{
_command.Connection.Close();
}
//
            return the number of rows affected
return
            dr.RecordsAffected;
}

FillSchema (IDbDataAdapter)
            -This method is responsible for creating a table in the DataSet and configuring the schema to match
            the source table based on the SchemaType passed in. The SchemaType may specify Mapped or Source. Mapped directs the adapter to use
            table mappings when creating the schema. WmiDataAdapter doesn't inspect the SchemaType, and implements the SchemaType.Source method that uses the
            source data to specifying the schema. The result is that the table
            and columns names inside the Dataset
            will map directly to the table and column names in the data source.

In the implementation below, the FillSchema method creates a table that maps
            to the source table (removing it if it already exists). It then
            iterates through the set of properties adding a column for each
            column in the result set.

public DataTable[] FillSchema(DataSet ds,
            SchemaType schemaType)
{
// need the connection for this
            adapter
WmiConnection conn =
            (WmiConnection)_command.Connection;
// we need to object
            set
ISWbemObjectSet ObjSet = null;
ISWbemObject WbemObj =
            null;
ISWbemPropertySet WbemPropSet = null;
// unfortunately
            we need to execute the query
// to get the metadata information
            necessary
// to fillin the schema
ObjSet = conn
            .WmiServices.ExecQuery(_command.CommandText,
"WQL", 0x10,
            null);
// create 1 table (for this version we will
// only
            support single select statements
DataTable []dt = new
            DataTable[1];
// new and create the table with the table
            name
string strTableName = this.CommandTableName;
// if the
            dataset contains a definition for this
// table name, remove
            it.
if(ds.Tables.Contains(strTableName))
{
ds.Tables.Remove(strTableName);
}

dt[0]
            = new DataTable(strTableName);
// grab the enumerator for the
            object set
IEnumerator obEnum = ObjSet.GetEnumerator();
//
            grab the current object
obEnum.MoveNext();
WbemObj =
            (ISWbemObject)obEnum.Current;
// grab the
            properties
WbemPropSet = WbemObj.Properties_;
// grab the
            property enumerator
IEnumerator ePropSet =
            WbemPropSet.GetEnumerator();
// Advance the enumerator's cursor
            until we reach
// the last item
while (ePropSet.MoveNext())
{
DataColumn dc = new DataColumn();
// Cast the Current
            item to the proper type
ISWbemProperty ob =
            (ISWbemProperty)ePropSet.Current;
// set the column
            name
dc.ColumnName = ob.Name;
dc.DataType =
            ConvertCIMToCLR(ob.CIMType);
// add this column to the
            table
dt[0].Columns.Add(dc);
}

// add this table to the
            dataset
ds.Tables.Add(dt[0]);
// return the table
return
            dt;

}

Update (IDbDataAdapter) -The
            Update method is responsible for
            updating a physical data source with changes made to the DataSet
            that was populated by the adapter. The WMI implementation of this
            method will meet the final requirement of the data provider by
            supporting stopping and starting of Window's services. The sequence
            of events might unfold as follows:

  1. A calling program uses the WMI adapter to fill the Dataset
              with the following WMI query: SELECT * FROM WIN32_SERVICE

  2. The adapter obliges this request and creates a DataTable
              called WIN32_SERVICE within the DataSet and populates it with a
              list of Windows services

  3. The calling program uses the DataSet select command and
              obtains the row containing the World Wide Web Publishing Service
              (IIS) and sets the State field from "Running" to "Stopped."

  4. The calling program invokes the Adapter's Update method.

  5. Finally the Adapter examines the changed State column for the
              World Wide Web Publishing service and uses the .NET
              ServiceController class to stop the service.

Now that you have an understanding of what the WMI provider's
            update method will do, let's look at its implementation:

public int Update(DataSet
            dataSet)
{
// as a special case to demonstrate
// updating
            functionality, we
// are going to demonstrate what
//
            happens when someone
// updates the state of a
            service
if(this.CommandTableName.ToUpper()
==
            "WIN32_SERVICE")
{
// this call gets a list of rows that
// were modified. These rows
// contain current values
            (after changes)
DataRow[] modifiedRows =
dataSet.Tables[CommandTableName].
Select("","",DataViewRowState.ModifiedCurrent);

//
            iterate through each modified row
foreach(DataRow dr in
            modifiedRows)
{
string previousServiceState =
dr["State",
            DataRowVersion.Original].ToString();

// make sure there was a
            change in the states. Someone
// could have made modifications to
            Data column values
// besides the State column. This line makes
            sure
// that the State column for the service row
// has
            indeed changed.
if(dr["State"].ToString() !=
            previousServiceState)
{
// create the Service controller
// associated with the
// service and system
            name
ServiceController sc =
new
            ServiceController(dr["Caption"].ToString(),
dr["SystemName"].ToString());

// this logic checks the
            modified rows
// state and either stops
// or starts the
            service.
if(dr["State"].ToString() ==
            "Stopped")
{
sc.Stop();
}
else if(dr["State"].ToString()
            == "Running")
{
sc.Start();
}
}
}
// return the
            number of modified rows
return
            modifiedRows.Length;
}
else
{
return
            0;
}
}

Since the provider only keys in on updates for WIN32_SERVICE
            objects, the method first verifies that the WMI Table name is
            WIN32_SERVICE . It then proceeds to grab all of the modified rows
            into a DataRow collection called modifiedRows by utilizing the
            DataTable's Select method and
            specifying the ModifiedCurrent DataViewRowState. The DataViewRowState ModifiedCurrent value informs the Select method to only return a list of
            current records that have been modified. Once the method has a list
            of rows that were modified, it iterates through each row, grabbing
            the previous value of the State column (remember this column
            indicates whether the service is running or not). If the calling
            program changed the value, the Update
            method then sets up the ServiceController object the correct
            information and starts or stops the service.

Using The Custom WMI Provider

Now that we have covered all of the properties and methods for
            the four interfaces, let's examine a quick application that tests it
            out. Included with the source code of the data provider is a Windows
            based application that allows you to type in a WMI query and
            populate a ListView control with the results. The screenshot of the
            application below shows the results from a "SELECT * FROM
            WIN32_PROCESS" query on my local machine.

The Server TextBox in the screen
            shot above allows a user to specify a server to execute queries
            against. The WmiQuery TextBox takes
            the specific WMI SQL query. Once the user has completed these two
            TextBoxes they may click the Execute
            button to execute the WMI query. The Stop/Start a Service button pertains to
            WIN32_SERVICE WMI queries. Once a WIN32_SERVICE query has been
            executed (i.e. SELECT * FROM WIN32_SERVICE), the user may select a
            service from the results grid and change the service's status from
            Running to Stopped and vice versa. The Clear button clears the Results
ListView.

There are quite a few WMI queries to choose from. Here are a few
            more for you to play around with:

QUERY | Description
SELECT * FROM Win32_service | Obtains a list of services running on the target machine.
SELECT * FROM Win32_LogicalMemoryConfiguration | Obtains one record listing details of the logical memory configuration from the target machine.
SELECT * from win32_LogicalProgramGroup | Retrieves a listing of the logical program groups on the target machine.
SELECT * from win32_operatingsystem | Obtains information about the operating system of the target machine.

You may also replace the * from the SELECT queries above with
            specific column names if you wish.

When you click on the Execute
            button, the button_click event is
            triggered, as shown in the next code block:

private void btnExecutDS_Click(object
            sender, System.EventArgs e)
{
// make sure the UI is completed
            correctly
if(tbWmiQuery.Text.Trim() ==
            "")
{
MessageBox.Show("Query is
            invalid.");
return;
}
if(tbServer.Text.Trim() ==
            "")
{
MessageBox.Show("Server must be
            specified.");
return;
}
try
{
this.ClearListView();
//
            create a WMI Connection with the
// connection
            string
WmiConnection conn = new
            WmiConnection();
conn.ConnectionString =
            tbServer.Text;
conn.Open();
// make sure the connection is
            open
if(conn.State !=
            ConnectionState.Open)
{
MessageBox.Show("Could not connect to
            {0}",
tbServer.Text);
return;
}
// create a new command
            with the WmiQuery
WmiCommand comm = new
            WmiCommand();
comm.Connection = conn;
comm.CommandText =
            tbWmiQuery.Text;
// Execute the command. We will be working
            with
// a reader
WmiDataAdapter wda = new
            WmiDataAdapter();
wda.Command = comm;
DataSet ds = new
            DataSet();

wda.Fill(ds);

this.CreateResultColumns(ds.Tables[0]);
this.PopulateListView(ds.Tables[0]);

}
catch(Exception
            ex)
{
MessageBox.Show(ex.Message);
}

}

After verifying the UI controls the code, it clears the ListView
            control, creates a connection to the server, and creates a WmiCommand and WmiDataAdapter object. It then populates a
            Dataset using the Adapter. Once the
            Dataset is populated, the method uses
            the DataSet to populate the ListView
            with columns and rows.

The CreateResultColumns method
            shown below creates a ListView column for each DataColumn in the DataTable:

private void CreateResultColumns(DataTable
            dt)
{
// the schema returned us the name and
// type of
            each of the columns.
// this routine creates ListView
            Columns
// for each column returned
foreach(DataColumn dc in
            dt.Columns)
{
lvResults.Columns.Add(dc.ColumnName,
100,
HorizontalAlignment.Left);
}
}

The PopulateListView method
            iterates through each row in the table and adds each column within
            the row to the result set.

private void PopulateListView(DataTable
            dt)
{
foreach(DataRow dr in dt.Rows)
{
ListViewItem lvi
            = new ListViewItem();
for(int i = 0; i < dt.Columns.Count;
            i++)
{
// itearate through each of the fields
// in the
            result set to continue
// updating the ListViewItem
if(i ==
            0)
{
lvi.Text =
            dr[i].ToString();
}
else
{
lvi.SubItems.Add(
            dr[i].ToString());
}
// add the results to the
//
            ListView
}
lvResults.Items.Add(lvi);
}

}

Additionally, the program shown in the screen shot above
            exercises the DataAdapter's update which allows you to start or stop
            a Windows service. The Windows application code that exercises this
            functionality is shown below and assumes the DataSet contains the
            results of a WIN32_SERVICE result set. After issuing a WIN32_SERVICE
            query, the user may select a service and click the Start/Stop a Service button. The following
            code gets executed when the user clicks the Stop/Start a Service button:

string strServiceName =
            this.GetCaptionItemText();

if(strServiceName ==
            "")
{
MessageBox.Show("Operation failed. Was your last "
            +
"query against the WIN32_SERVICE object? " +
"You must
            execute this query and then " +
"click the Start/Stop Service
            button.");
return;

}

if(_wmiDataSet != null
            && _wmiDataSet.Tables.Count > 0)
{
// look for the
            service with the correct caption

DataRow []dr =
            _wmiDataSet.Tables[0].
Select("Caption='" + strServiceName +
            "'",
"", DataViewRowState.CurrentRows);

// make sure we
            found a row
if(dr != null && dr.Length == 1)
{
//
            editing of the row begins
dr[0].BeginEdit();
// set the state

if(dr[0]["State"].ToString() ==
            "Running")
{
dr[0]["State"] =
            "Stopped";
}
else
{
dr[0]["State"] =
            "Running";
}
// editing of the row
            ends
dr[0].EndEdit();
// let the adapter handle the stopping
            or starting
// of the
            service
_wmiDataAdapter.Update(_wmiDataSet);

}
else
{
MessageBox.Show("Operation
            failed. Was your last " +
"query against the WIN32_SERVICE
            object? " +
"You must execute this query and then " +
"click
            the Start/Stop Service
            button.");

}
}
else
{
MessageBox.Show("Operation
            failed. Was your last " +
"query against the WIN32_SERVICE
            object? " +
"You must execute this query and then " +
"click
            the Start/Stop Service button.");
}

The code above first retrieves the value of the caption column
            within the ListView by invoking the GetCaptionItemText method. We will examine
            the code for this method next. The value of the caption column
            represents the name of the Service that the user selected. After
            retrieving the Service name, the code then uses the DataTable.Select method to locate the
            correct row in the DataSet. Once it has a valid row, the code then
            edits the row, toggles the State of the service and invokes the
            adapter's Update method. The Adapter
            takes over and executes our Update
            code which in turn starts or stops the service.

Here is the code for the GetCaptionItemText method:

private string
            GetCaptionItemText()
{
if(lvResults.SelectedItems.Count >
            0)
{
foreach(ColumnHeader ch in
            lvResults.Columns)
{
if(ch.Text == "Caption")
{
return
            lvResults.SelectedItems[0].
SubItems[ch.Index].Text;
}
}
}
return "";
}

Case Study Review

This case study covered quite a bit of material. We examined what
            a .NET data provider is, and explored the major reasons you might
            create one. We also touched on some of the guidelines to keep in
            mind when writing a provider. After a quick introduction to WMI, we
            proceeded to cover the implementation of the four major interfaces
            for ADO.NET providers. We completed the case study by walking
            through a simple Windows application that used the custom WMI data
            provider to retrieve and Update WMI data.

Case Study Information
Author | Todd Clemeston
Technical Editor | Adam Ryland
Project Manager | Helen Cuthill
Reviewers | Andrew Krowczyk, Saurabh Nandu, Slavomir Furman, Adam Ryland

If you have any questions or comments about this case study,
            or suggestions for further pieces, please contact the technical
            editor. If you wish to talk about the case study with other
            subscribers, please visit our forum here.

     Brought to you by ownSky!
 C#Today - © 2002 WroxPress
Limited


.NET Web Data ToolKit

.NET Web Data ToolKit
http://www.csharptoday.com/content.asp?id=1831
Article Rating: 4
Published on March 11, 2002
Abstract
Think about how useful it would be to have a toolkit that allows you to SELECT nuggets of information FROM live web pages as though selecting from tables in a relational database…using industry-standard SQL. Imagine if you could capture the results into an ADO.NET DataTable, allowing you to treat web data like any other data source. This Case Study by Tony Loton describes the problems encountered - and solutions devised - during the design and implementation of just such a toolkit.
Case Study

FEEDBACK: We would love to hear what you think of this case study, and what topics you would like to see covered in future. Please take a few moments to mail the Editor to give us your opinions.

Think about how useful it would be to have a toolkit that allows you to SELECT nuggets of information FROM live web pages as though selecting from tables in a relational database.using industry-standard SQL. Imagine if you could capture the results into an ADO.NET DataTable, allowing you to treat web data like any other data source. This Case Study describes the problems encountered -- and solutions devised -- during the design and implementation of just such a toolkit.

So that you know where we're heading, the application we'll be discussing takes a web page like this:

...and processes SQL statements like this:

SELECT .table[1].tr[0].td[1].table[1].tr[1].td[0].table[0].tr[*].td[0].a[0].text[0] AS title,
.table[1].tr[0].td[1].table[1].tr[1].td[0].table[0].tr[*].td[1].text[0] AS author,
.table[1].tr[0].td[1].table[1].tr[1].td[0].table[0].tr[*].td[2].text[0] AS isbn
FROM http://www.wrox.com/Books/Books.asp?section=1_1
WHERE title LIKE 'Professional*'
ORDER BY isbn ASC

...to produce an ADO.NET DataTable like this:

As a taste of what is to come, notice that in the SQL statement we're selecting sets of HTML tagged elements as though selecting fields of a relational table, and we're taking the data from a web page URL in the same way that we would take data from a table.

There will come a time - no doubt - when we won't need to do this because every online information provider will have a full web service using XML and SOAP. That day has not yet come, so let's press on.

Contents

Section
Scope Of The Case Study
Challenge 1: Retrieving Web Data via HTTP Submissions
Challenge 2: Parsing HTML and / or XML Elements
Challenge 3: Selecting Elements
Challenge 4: Flattening the Tag Hierarchy to form a Relational Table
Challenge 5: Supporting Structured Query Language
Case Study Review
Potential Applications

Background to the idea

It all started with my original article "Accessing the World's Biggest Database ", published on the JavaWorld web site in March 2001 (http://www.javaworld.com/javaworld/jw-03-2001/jw-0316-webdb.html?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq). That idea manifested itself in the WebDataKit commercial implementation for the Java 2 platform, an early version of which was written up as a case study in "Professional Java Data" (Wrox Press, ISBN 1-8610041-0-9).

With the Java implementation quite stable, I then turned my attention to the Microsoft Platform, and first to the web query support provided out-of-the-box with Microsoft Excel. I mentioned the Excel approach in my "Working with Web Data in C# " article (http://www.csharptoday.com/content.asp?id=1782&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq). If you're looking for a really easy way to get web data into a spreadsheet for further processing, it's well worth a look. In that article, I also introduced the basic mechanisms for extracting web data into a C# program on the .NET framework.

At the time of writing, the commercial WebDataKit for .NET is under development as a beta product. You can find out more about that, and try it out, at http://www.lotontech.com/wdkDotNet.html?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq.

Now that you know where the idea came from, and what progress has been made so far, let me now tell you about what we'll cover in the case study.

Scope of the Case Study

It's not practical to give a line-by-line account of the complete code, and in any case, it was written as the basis of a commercial product rather than as a teaching aid. and so is not too pretty!!! The underlying ideas are more important than the code itself, so think of this case as a recipe - rather than a blueprint -- for creating a Web Data Toolkit. We'll discuss some of the key decisions that were made during the design and implementation, and in that context you'll find out how I tackled the problems of how to...

  • Extract data from web pages stored on live web servers or as files on disk.
  • Make individual HTML and XML elements uniquely addressable.
  • Select elements using wildcards and regular expressions, to pick out related sets.
  • Flatten a hierarchic HTML or XML structure to a flat relational structure.
  • Parse SQL statements and use SQL to query web pages a though they were relational database tables.

From a programming point of view, you'll see some interesting uses for the WebRequest, Regex, and DataTable classes of the .NET framework, plus other classes from these namespaces:

  • System.Net
  • System.IO
  • System.Text.RegularExpressions
  • System.Data

At the end I'll provide some code that you can run so you can see that it really does work.

Design and Implementation Decisions

I'll present the design and implementation decisions as a series of challenges that were faced during the development of the Web Data Toolkit. For each one, I'll present the essence of the algorithmic solution or .NET technical solution that met the challenge.

I've provided the complete source code as it stands at the time of this case study. It's warts-and-all source code, included for reference only and not intended for further distribution. Where appropriate in the discussion that follows, I've included some pointers into the source files.

Challenge 1: Retrieving Web Data via HTTP Submissions

Fundamental to the rest of the toolkit is the ability to fetch the text content from a web URL (beginning http://), or a file URL (beginner file://). For web URLs, we might need to pass some parameters in order to specify exactly the response we want, and such parameters will generally be appended to the end of the URL like this.

http://www.wrox.com/Books/Books.asp?section=1_1

In this case, we're assuming that the web site will be expecting requests via the HTTP GET method. For sites expecting requests via the POST method, the URL would be.

http://www.wrox.com/Books/Books.asp

and we'd send the parameter section=1_1 as a separate stream of data.

Our solution must support one of those submission methods, or it could support both, because it is possible to append parameters to a URL and post some data at the same time.

The following method, which you could easily incorporate into your own programs, makes use of the .NET System.Net.WebRequest and System.Net.WebResponse classes to issue a HTTP submission and to capture the entire response as a single text string.

First, pass in a URL (which might have parameters appended), some additional text to post (which is optional), and a HTTP method (which may be "GET" or "POST"):

public static String getTextFromURL (String url, String postText, String method)
{

Create a WebRequest from the URL and set its method:

WebRequest request = WebRequest.Create
            (url);
request.Method = method;

Enter this section of code only if the method is "POST", and only if it is a web-based URL, capable of being cast to HttpWebRequest because it begins with http://:

    if
            (method.Equals("POST") && (request is
            HttpWebRequest))
{
request = (HttpWebRequest)
            request;
request.ContentType =
            "application/x-www-form-urlencoded";

byte [] bytes =
            null;
bytes = System.Text.Encoding.ASCII.GetBytes
            (postText);
request.ContentLength = bytes.Length;
Stream
            outputStream = request.GetRequestStream ();
outputStream.Write
            (bytes, 0,
            bytes.Length);

outputStream.Close();
}

For all GET requests, and requests to file URLs (beginning file://), we would not be able to post extra text using the code above, and attempting to do so would raise an exception. You should also be aware that for the text to be posted correctly, the ContentType of the WebRequest must be set as shown.

Whether we post additional text or not, we will need to get hold of the HTTP response:

WebResponse response = request.GetResponse();
Stream responseStream = response.GetResponseStream();

...and read the text:

StreamReader reader = new StreamReader (responseStream,System.Text.Encoding.Default);

String text=reader.ReadToEnd ();

Finally, we'll close and return the text.

reader.Close();

return text;
}

For further reading you are invited to look at the real getTextFromUrl(.) method in the WebDataEngine.cs source file.

That's quite a useful method, and not too different from my original Java solution that used the java.net.URLConnection and java.io.InputStream classes.

Challenge 2: Parsing HTML and / or XML Elements

In order to pick out nuggets of information from web pages, using structured queries or otherwise, those elements must be made uniquely addressable. For each piece of text enclosed within HTML or XML tags, I'd like to label the position of that text within the tag hierarchy. Taking as a basis the web screenshot shown earlier, I'd like to come up with something like this:

Notice how each piece of text on the page is listed in the form:

KEY (unique position) = CONTENT (the text)

In order to support an algorithm that traverses the hierarchy of tags and builds element keys by counting the tags at each level, we need an underlying mechanism for picking out (parsing) the tag- and text- tokens in a string such as:

"<HTML><HEAD><TITLE>Sample HTML
Table</TITLE></HEAD><BODY><TABLE><TR><TD>row0,col0</TD><TD>row0,col1</TD></TR>
<TR><TD>row1,col0</TD><TD>row1,col1</TD></TR></TABLE></BODY></HTML>"

At this point I made a major departure from the original Java implementation. In that version, I had taken advantage of classes from the javax.swing.text.html.HTMLEditorKit and org.xml.sax packages to parse the HTML or XML tags. At first glance I saw no equivalent parser classes in the .NET framework, so I wrote a very basic tag parser (and I mean very basic!) myself.

I then decided it was a much better approach anyway because it runs quicker, is not dependent on any third-party (e.g. SAX) class library, and is useful for parsing any kind of text embedded within <starttag> and </endtag> pairs. For this application we don't even need to ensure the validity of the HTML or XML structure, as long as there is some kind of structure. However, it's not completely problem-free though as you'll see at the end.

The Tag Parser has a method named getTagOrText(.) that simply returns the next tag or inter-tag text from the input string provided by the getTextFromURL(.) method shown earlier. It's called in a loop like this.

while (
            (tagOrText=getTagOrText(taggedText,pos))!=null)
{
// Do
            something with the tag or the text
// then set the parsing
            position for next time
            around.

pos=pos+tagOrText.Length;
}

In a nutshell, the getTagOrText(.) method takes anything between < and > as a tag, and anything between > and < as text.

The comment // Do something with the tag or the text is where we would keep track of our position within the embedded tag hierarchy (by maintaining a tag path of the form .table[0].tr[1].td[3]) as we meet each new tag, or where we would add a new text element at the current position.

That description should suffice to paint a picture of what happens. The code itself is somewhat convoluted and distracting, and shows off nothing in terms of the .NET framework, so we won't dig too deep except to point out how we deal with tag attributes and non-structural HTML tags.

Tag Attributes

If you look back at the figure shown earlier, you will see that besides capturing the text content from between tags we're also capturing the values of attributes that appear within the tags themselves. Attribute entries appear like this with an @ symbol denoting the name of the attribute:

.table[1].tr[0].td[1].table[1].tr[1].td[0].table[0].tr[2].@bgColor = white
.table[1].tr[0].td[1].table[1].tr[1].td[0].table[0].tr[2].@height = 15

In this case, there are two attributes with name bgColor / value white and name height / value 15 respectively, extracted from a tag like this:

<TR bgColor=white height=15>

Given a tag like that one, the following code segment picks out the attribute name / value pairs using the .NET Regular Expressions.

Regex attributeRegex=new Regex("(?<name>\\w+)\\s*=\\s*(?<val>(?<q>\"?).+?\\k<q>)(?=(\\s+|\\s*>))");

Match attributeMatch = attributeRegex.Match(fullTagOrText);

for ( ; attributeMatch.Success; attributeMatch = attributeMatch.NextMatch())
{
String name=""+attributeMatch.Groups["name"];
String val=""+attributeMatch.Groups["val"];

// Now that we have the attribute name and value, so we would do
// something with them.
}

In that code we first created an instance of the Regex class to hold our regular expression pattern (the awesome looking text string). Note that the regular expression engine interprets certain escape sequences - such as \s for space character - and we must prepend an additional \ (to give \\s) to prevent the C# compiler from attempting to interpret those escape sequences itself.

We use an instance of the Match class to match our regular expressions against the tag text, and we step through the matches.of which there may be more than one. Within the loop we pick out the text segment that our regular expression matched as an attribute name (look for "<name>" in the regular expression string), and the text segment that our regular expression matched as the attribute value (look for "<value>" in the regular expression string).

Now you'll be wondering about the regular expression string itself. A regular expression is a text pattern that is superimposed onto an input string to see if any segments of the string match the pattern. I'll explain the regular expression used in code here, but if you want to know more about regular expressions in general it's worth taking a look at my article "Introducing .NET Regular Expression with C# " at http://www.csharptoday.com/content.asp?id=1786&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq.

The following table describes the pattern we're looking for. You should be able to relate each entry back to the original regular expression string.

Regular Expression | Explanation | ...in plain English
(?<name>\\w+) | Match one of more word \\w+ characters captured into a group called <name> | The attributed name
\\s*=\\s* | Followed by zero or more spaces \\s* followed by an equals sign = then zero or more spaces \\s | An equals sign (with spaces either side)
(?<val> | Followed by a group called <val> that captures... |
(?<q>\"?) | A quote \" character that's optional ? and which is captured into a group called <q> | An optional quote character
.+? | Followed by any character(s) . repeated at least once but as few times as possible +? | The attribute value
\\k<q> | Followed by the next character matching the back referenced \\k character that was previously captured into group <q> | A closing quote if there was an opening one
) | end of <val> group |
(?=(\\s+|\\s*>)) | Followed by a look ahead group of one or more spaces \\s+ or | zero or more spaces and an end tag brace \\s*> | Some spaces or the tag's closing brace up ahead

To make it a little clearer, here's how it maps twice onto a sample tag with attributes.

Actually, for that sample tag, the regular expression could be made somewhat simpler because there are no quotes surrounding the attribute values and no spaces either side of the equals signs. But, we need to allow for possible variations in syntax, and in that respect we could have made it even more complex to handle -- for example -- attribute values enclosed within single quotes.

If you think it looks complicated, rest assured that it's easy when you know how, and it's certainly more compact - and adaptable - than writing pure C# code to look for the same patterns. As I said before, you can look at my earlier article if you want to learn more about Regular Expressions.

For your information, in the original Java implementation I could not take advantage of the JDK1.4 regular expression classes, as they had not yet arrived. Fortunately, the Java API for XML Processing (JAXP) provided a callback method that was invoked for each tag encountered, and which included a list of attribute name / value pairs as a parameter. So I simply had to write an implementation for this method signature:

public void handleStartTag(HTML.Tag tag, MutableAttributeSet attributes, int pos)

Non-Structural HTML Tags

Now let's think again about the keys allocated to individual text elements according to their positions within the source page. In the following example, you'll notice that I've chosen to ignore certain HTML tags when forming the key.

.table[1].tr[0].td[1].table[1].tr[1].td[0].table[0].tr[2].text[0]

The full path to that element would really be something like this:

.html[0].body[0].table[1].tr[0].td[1].table[1].tr[1].td[0].table[0].tr[2].b[0].text[0]

The bold text shows which tags I've chosen to ignore because they increase the size of the key unnecessarily while at the same time.

  • not adding to its uniqueness, in the case of .html[0].body[0] that would be shared by all elements
  • or splitting text elements unnecessarily, as would non-structural tags like the <b> (i.e. .b[0] ) bold tag.

To reduce the length of the keys and to ensure only structural HTML tags are included, I pass two arrays into the TagParser, one for matched HTML tags (those that have a matching end tag) and the other for empty - or simple -- tags (i.e. those with no closing tag). Any tag found in the input text that is not in one of my lists will not be added to an element's key. That's possible because HTML has a finite set of tags and we know what they are. For XML files we'll have no special knowledge, so all tags will be used in forming keys.

Here is the code segment that sets up the arrays of structural HTML tags for the TagParser constructor.

String[] matchedHTMLTags=new String[] {"<table>","<tr>","<th>","<td>","<a>"};

String[] emptyHTMLTags=new String[] {"<img>"};

TagParser tagParser=new TagParser(matchedHTMLTags,emptyHTMLTags);

At this point it's worth noting that although the examples in this case study concentrate on HTML tables, the inclusion of the <a> and <img> tags in those lists means that even in a table-free page you could still pick out all the links (.a[*].@href for the URLs, and .a[*].text[0] for the link texts) or images (.img[*]) as column data. By adding more tags to the lists, you could pick out elements from between other tag combinations, like <div> and <span>.

We run the parser via its getElements(.) method, passing in the source text from the web page, and we return the result to whatever called this code.

return tagParser.getElements(taggedText);

The end result is an ArrayList containing instances of a TaggedElement class, having two read-only properties -- a key and content - plus a read/write property selectSpec.

public class TaggedElement
{
private
            String elementKey;
private String elementContent;
private
            String elementSelectSpec;

public TaggedElement(String key,
            String
            content)
{
this.elementKey=key;
this.elementContent=content;
}

public
            string key
{
get { return elementKey; }
}

public
            string content
{
get { return elementContent;
            }
}

public string selectSpec
{
get { return
            elementSelectSpec; }
set { elementSelectSpec=value;
            }
}
}

The purpose of the key and content properties should be obvious, but you might be wondering about the selectSpec property. All will become clear as we move on.

For further reading you are invited to look at the real getTagOrText(.) and getElements(.) methods of the TagParser.cs source file.

Challenge 3: Selecting Elements

Having cracked the problem of retrieving a set of uniquely addressable elements comprising the source web page, we then needed a way of picking out a subset of elements that we're interested in. From the book listing on the Wrox web site we might be interested only in the book titles.

The following screen shot shows a breakdown of the elements at the Wrox site. You'll notice from the URL that for convenience we're actually looking at a saved version of the page, which serves to demonstrate the support for the file:// protocol in addition to the http:// protocol (see Challenge 1). That saved version has been included in the download bundle so that you'll be able to try it out exactly as I have.

Now look what happens if we place some filter text into the Key Filter box, to pick out only book titles:

The format of the elements keys, and the format of the key filter used to select them, should be intuitive enough if you're used to hand-coding HTML. Later you'll see how our filters will be given alias names to make them even more meaningful. XML tags will generally have semantically significant names rather than the syntactically significant names of the HTML tags, so when dealing with XML source data you'll see more meaningful element keys like .PLAY[0].ACT[0].TITLE[0].

Implementing that functionality meant matching the Key Filter against the key property of each TaggedElement in the list. Another job for regular expressions, but the problem is that specifying a key filter using full regular expression notation would require a less-than-friendly string like this:

\.table\[1\]\.tr\[0\]\.td\[1\]\.table\[1\]\.tr\[1\]\.td\[0\]\.table\[0\]\.tr\[\d+\]\.td\[0\]\.a\[0\]\.text\[0\]

The backslash \ character preceding each dot, open square brace, and close square brace, would be necessary for the Regex engine to treat those as literal characters rather than escape sequences with special meanings. And the \d+ sequence in that regular expression, meaning "match one or more decimal digits", would be used in place of the simple wildcard * characters.

Clearly users would not want to enter such a convoluted string, so we use the simpler syntax shown in the screenshots. The first step in our method to pick out matching elements therefore converts the simpler syntax (the Key Filter, passed as the selectSpec parameter in code) into the full regular expression syntax:

public static ArrayList getSelectedElements(ArrayList originalElements, String selectSpec)
{
String filter=selectSpec;

filter = filter.Replace("[", @"\[");
filter = filter.Replace("]", @"\]");
filter = filter.Replace(@"\[*\]", @"\[\d+\]");
filter = filter.Replace(".", @"\.");

filter = "^"+filter+"$";

At this point we have a fully formed regular expression, with a ^ prepended and a $ appended to ensure that the pattern is matched against the whole of the key property from start to finish (otherwise a shorter selectSpec could match the front portion or end portion of an element's key, or any portion in between).

Now we create an ArrayList to hold the subset of selected elements, we step through the complete set of tagged elements, and we add each element to the list only if the regular expression matches the key property. We also set the selectSpec property of each tagged element so that later we'll know what key filter - in simple non-regex syntax - was used to select it.

ArrayList filteredElements=new
            ArrayList();

Regex filterRegex = new Regex(filter);

foreach (TaggedElement element in originalElements)
{
Match theMatch=null;

theMatch =
            filterRegex.Match(element.key);

if
            (theMatch.Success)
{
filteredElements.Add(element);
element.selectSpec=selectSpec;
}
}

return
            filteredElements;
}

For further reading you are invited to look at the real getSelectedElements(.) method in the WebDataEngine.cs source file.

Another good use for regular expressions, and now we have the basis of the useful GUI application shown in the screenshots. The WebFilterViewer allows us to experiment by picking out various groups of elements using key filters (or selectSpecs, if you prefer).

Challenge 4: Flattening the Tag Hierarchy to form a Relational Table

We now have the ability to select a set of elements from a web page into a list, or - to put in another way - into a column. The next challenge was to select several columns of data into the rows of a table.

At first glance this is quite straightforward. We could easily devise key filters to pick out the Author and ISBN details from the Wrox page, in addition to the Title elements just demonstrated. And because all of those elements are contained within a regular HTML table, the three lists (or columns) will be identical in length so easy to combine. To remind you, here's what the source page looks like:

But not all tables are regular, and it's even possible to have tables within tables as you can see in this simple page.

The WebFilterViewer (shown next) shows how the elements are arranged in the HTML hierarchy, and it allows us to formulate the selectSpecs to pick out data in each column.

Use selectSpec .table[0].tr[*].td[0].text[0] for author elements...

T. Loton

A. Nother

Use selectSpec .table[0].tr[*].td[1].table[0].tr[*].td[0].text[0] for title elements...

Professional Java Data

Professional Java Servlets

Professional Java Things

Use selectSpec .table[0].tr[*].td[1].table[0].tr[*].td[1].text[0] for ISBN elements...

ISBN 1234567

ISBN 1234568

ISBN 1234569

Selecting author elements would return two entries, whereas selecting title or ISBN elements would each return three entries. In such cases we need to flatten the hierarchy, repeating elements as necessary to give a relational table. like:

T. Loton | Professional Java Data | ISBN 1234567
T. Loton | Professional Java Servlets | ISBN 1234568
A. Nother | Professional Java Things | ISBN 1234569

To show how we solve the flattening problem, take a look at the following table in which I've listed the elements selected into the first (author) column combined in every possible combination with the elements selected into the second (title) column. For simplicity I've left out the final (ISBN) column.

Author selected with .table[0].tr[*].td[0].text[0] gives | Title selected with .table[0].tr[*].td[1].table[0].tr[*].td[0].text[0] gives
T. Loton with key .table[0].tr[0].td[0].text[0] | Professional Java Data with key .table[0].tr[0].td[1].table[0].tr[0].td[0].text[0]
T. Loton with key .table[0].tr[0].td[0].text[0] | Professional Java Servlets with key .table[0].tr[0].td[1].table[0].tr[1].td[0].text[0]
T. Loton with key .table[0].tr[0].td[0].text[0] | Professional Java Things with key .table[0].tr[1].td[1].table[0].tr[0].td[0].text[0]
A. Nother with key .table[0].tr[1].td[0].text[0] | Professional Java Data with key .table[0].tr[0].td[1].table[0].tr[0].td[0].text[0]
A. Nother with key .table[0].tr[1].td[0].text[0] | Professional Java Servlets with key .table[0].tr[0].td[1].table[0].tr[1].td[0].text[0]
A. Nother with key .table[0].tr[1].td[0].text[0] | Professional Java Things with key .table[0].tr[1].td[1].table[0].tr[0].td[0].text[0]

Clearly not all of those candidate rows make sense, for example "A. Nother" combined with "Professional Java Data", so the next task is to take out the ones that don't make sense. In the following table I've replaced any concrete indexes (between square braces) in the elements keys with * characters if those indexes were fixed in the original selectSpec. Conversely, I've retained any indexes in the element keys if there was a corresponding wildcard * character in the selectSpec.

Author selected with .table[0].tr[*].td[0].text[0] gives | Title selected with .table[0].tr[*].td[1].table[0].tr[*].td[0].text[0] gives
T. Loton with key .table[*].tr[0].td[*] | Professional Java Data with key .table[*].tr[0].td[*] .table[*].tr[0].td[*]
T. Loton with key .table[*].tr[0].td[*] | Professional Java Servlets with key .table[*].tr[0].td[*] .table[*].tr[1].td[*]
T. Loton with key .table[*].tr[0].td[*] | Professional Java Things with key .table[*].tr[1].td[*] .table[*].tr[0].td[*]
A. Nother with key .table[*].tr[1].td[*] | Professional Java Data with key .table[*].tr[0].td[*] .table[*].tr[0].td[*]
A. Nother with key .table[*].tr[1].td[*] | Professional Java Servlets with key .table[*].tr[0].td[*] .table[*].tr[1].td[*]
A. Nother with key .table[*].tr[1].td[*] | Professional Java Things with key .table[*].tr[1].td[*] .table[*].tr[0].td[*]

You will see that, up to a point, the keys on the left now match the keys on the right. Except that some of them don't because the indexes do not match up as I've shown in bold. Removing those non-matches leaves the following table, which is exactly what we want.

T. Loton with key .table[*].tr[0].td[*] | Professional Java Data with key .table[*].tr[0].td[*] .table[*].tr[0].td[*]
T. Loton with key .table[*].tr[0].td[*] | Professional Java Servlets with key .table[*].tr[0].td[*] .table[*].tr[1].td[*]
A. Nother with key .table[*].tr[1].td[*] | Professional Java Things with key .table[*].tr[1].td[*] .table[*].tr[0].td[*]

Retained elements are said to be in the same context, the context being defined by the modified versions of the keys that we compared.

I implemented that flattening algorithm in code and extended it to handle any number of selected columns. The code's not too pretty but essentially all it does is.

  • Form a cross-product of every element selected into column 1, combined with every element selected into column 2, combined with every element selected into column 3. and so on.
  • Check that the context of every element in each row is consistent with the context of every other element in the same row.

Determining the context of each selected element depends on knowing the selectSpec that was used to select the element in the first place. That's why the TaggedElement class had a selectSpec property that was set in Challenge 3.

Returning rows as a .NET DataTable

As we're building a relational table, it made sense to deliver the end result as a .NET System.Data.DataTable instance, thus making the data extracted from web pages accessible in the same way as any other .NET data source, and - as you'll see later - to facilitate querying of the web data using conventional Structured Query Language.

The following getSelectedRows(.) method takes in a list of names, one for each column, which will ultimately be derived from the column aliases specified in a SQL statement. The separate ArrayLists of elements for each column, as returned by the getSelectedElements(.) method in Challenge 3, become items of a containing ArrayList passed in as a second parameter named disjointColumns.

public static DataTable getSelectedRows(ArrayList columnNames, ArrayList disjointColumns)
{

Just in case the content of the second parameter is not clear, I'll tell you that each entry in the disjointColumns list will itself be an ArrayList of elements selected with one of the selectSpecs. So in our example, disjointColumns[0] would be an ArrayList containing ["T. Loton", "A. Nother"], and disjointColumns[1] would be an ArrayList containing ["Professional Java Data", "Professional Java Servlets", "Professional Java Things"].

Now we create a .NET DataTable to hold the final results and added several column definitions to it:

int
            columnCount=disjointColumns.Count;

DataTable finalTable=new
            DataTable();

foreach (String columnName in
            columnNames)
{
DataColumn newColumn = new DataColumn();
newColumn.DataType =
            System.Type.GetType("WebDataToolKit.TaggedElement");
newColumn.AllowDBNull = true;
newColumn.Caption =
            ""+columnName;
newColumn.ColumnName =
            ""+columnName;
finalTable.Columns.Add(newColumn);
}

We then run through the algorithm described earlier to combine column elements in all combinations, omitting the combinations that don't make sense.

For further reading you are invited to look at the real getSelectedRows(.) method in the WebDataEngine.cs source file.

If you're not inspired to look through the source code for that complex algorithm - and I don't blame you - you might nonetheless be interested to know that the general procedure for adding a (valid) row to the DataTable is.

DataRow newRow = finalTable.NewRow();
newRow[0] = someData
newRow[1] = someMoreData
finalTable.Rows.Add(newRow);

Setting the column DataTypes

You will have noticed that initially we set the type of each column to be our own class TaggedElement. We're taking input as ArrayList s of TaggedElement s and we need the elements in that form throughout the algorithm so that we have access to each element's key, content and selectSpec. As a final step, we copy the whole table to a new table with more appropriate data types.

HTML is a text markup language, thus every element will be extracted as a text string like "T. Loton", "13.99", or "01-01-2002" and the default type for each column would be string. Though there are no explicit data types in HTML, we can infer that the first of those really is a text string, the second is a float or double, and the third is a date. Thus when we form the final table, we set each column's data type to:

  • System.DateTime if all entries in the column represent a date
  • System.Double if all entries in the column represent a real (floating point) number
  • String otherwise

We determine the implicit type of each element using a series of try.catch blocks as follows:

bool isDate=false;
bool
            isDouble=false;

DateTime dateValue=new DateTime();
Double
            doubleValue=Double.NaN;

try
{
dateValue=Convert.ToDateTime(thisContent);
isDate=true;
}
catch(Exception
            ex){}

try
{
doubleValue=Convert.ToDouble(thisContent);
isDouble=true;
}
catch(Exception
            ex){}

At this point we'll know whether the element is isDate or isDouble (or else default String). If all the other elements in the same column come out the same, we reset the type of that column to that type. The end result is a representation of the original web data in the form of a DataTable with appropriately typed columns, indistinguishable from any other DataTable of whatever source.

You might be concerned about the performance implications of the code just shown, and you'd be right, but the benefit in doing this will become clear when we consider SQL in the next challenge. Look out for the "Data Types Revisited" section!

Uses for the DataTable representation

As with any other DataTable, we can bind it to a System.Windows.Forms.DataGrid form element with code like this:

System.Windows.Forms.DataGrid sqlDataGrid
= new System.Windows.Forms.DataGrid();
sqlDataGrid.DataSource=ourDataTable;

So very little effort is required to visualize the data thus:

For further reading you are invited to look at the SqlResultsForm.cs source file.

Furthermore, it would be possible to construct a .NET DataSet by combining one or more DataTables originating from different web sources. Thus we really could treat the World Wide Web as a huge database in which the individual web pages are the tables. What makes the .NET DataSet particularly interesting is the fact that we could convert easily to an XML representation with a full in-line schema, using this single line of code:

string xmlWithSchema = ourDataSet.GetXmlSchema();

For those XML gurus amongst you, how about using that representation for XPATH queries or for transformations using XSLT?

Challenge 5: Supporting Structured Query Language

It's no accident that the previous screenshot was titled "SQL Results" as that's what we'll consider next. Remember what I said in the introduction?

Think about how useful it would be to have a toolkit that allows you to SELECT nuggets of information FROM live web pages as though selecting from tables in a relational database.using industry-standard SQL.

At this point in the development, I was well on the way to achieving that aim because we have the ability to select elements of a web page into columns of data, to be combined with other columns of data in a complete relational table. In a nutshell we now have the ability to run an SQL statement of the form:

SELECT .table[0].tr[*].td[0].text[0] AS author
, .table[0].tr[*].td[1].table[0].tr[*].td[0].text[0] AS title
, .table[0].tr[*].td[1].table[0].tr[*].td[1].text[0] AS isbn
FROM file://C:/XML/WroxEmbedTables.html

Each part of the SELECT clause comprises a key filter -- called a selectSpec in code, remember? -- to pass to the getSelectedElements(.) method, along with a column alias to pass through to the getSelectedRows(.) method. The FROM clause has a URL to pass into the getTextFromURL(.) method. Of course we'd have to parse the text of the SQL statement to pick out the various parts, and you won't be surprised to hear that for that job I once again used regular expressions!

To support a fuller SQL syntax we need to allow statements that include WHERE and ORDER BY clauses. Here is such a statement that refers to the Wrox New Releases web page:

SELECT .table[1].tr[0].td[1].table[1].tr[1].td[0].table[0].tr[*].td[0].a[0].text[0] AS title
, .table[1].tr[0].td[1].table[1].tr[1].td[0].table[0].tr[*].td[1].text[0] AS author
, .table[1].tr[0].td[1].table[1].tr[1].td[0].table[0].tr[*].td[2].text[0] AS isbn
FROM file://C:/XML/WroxNewReleases.htm
WHERE title LIKE 'Professional*'
ORDER BY isbn ASC

With that functionality in place we'll have completed the toolkit, so here's how I did it.

Supporting WHERE and ORDER BY

In the original Java implementation, I implemented from scratch the functionality to parse out WHERE and ORDER BY clauses, to compare each row against the WHERE clause, and to sort according to the ORDER BY clause. Not as simple as it sounds because, typically, WHERE clauses support AND / OR / NOT operators and ORDER BY clauses allow sorting by any combination of columns in ascending or descending order.

The .NET framework provides a very useful shortcut here thanks to the fact that we've already assembled our results in a DataTable. This class has a Select(.) method that allows a SQL WHERE clause to be passed straight in, along with an ORDER BY clause. So for our example, if we set a couple of variables as follows.

String whereClause = "title LIKE
            'Professional*'";
String whereClause = "isbn ASC";

We can then call the following method to get a new version of our DataTable, filtered and sorted as appropriate:

public static DataTable
            filterAndSort(DataTable results, String whereClause, String
            orderClause)
{
DataTable
            newResults=results.Clone();
newResults.Clear();

DataRow[]
            rows=results.Select(whereClause,orderClause);
foreach (DataRow
            row in rows)
{
DataRow
            newRow=newResults.NewRow();
newRow.ItemArray=row.ItemArray;

newResults.Rows.Add(newRow);
}

return
            newResults;
}

That shows off a really neat feature of the .NET DataTable, and it saves on a lot of code compared with the Java implementation, especially when you consider that we could be dealing with a SQL statement ending thus:

WHERE (title LIKE 'Professional*' OR author LIKE 'Matthew*') AND isbn >='1861005'
ORDER BY author ASC, title DESC

No matter how complicated our WHERE and ORDER BY clauses become, it's all handled effortlessly by the DataTable itself.

Data Types Revisited

Remember that when building the DataTable in the first place we took care to set the data types of the columns to more appropriate types than the default String type? That was to make the results of WHERE and ORDER BY clauses more intuitive.

When comparing values of - for example - a "percentage change" column, we'd like a comparison 20 > 100 to evaluate to false, as it will if we set the column a numeric type like System.Double. If we left the column type as System.String the same expression would evaluate to true as a consequence of a lexical comparison. A similar situation exists with dates, which would compare differently as strings than as true dates.

For further reading you are invited to look at the real filterAndSort(.) method in the SqlEngine.cs source file.

That completes the discussion of the design and implementation decisions that were taken, and we can now take the toolkit for a test run!

Proof of the Pudding

I'll give you some software to try out, so that you can see how these ideas really do hang together. I've provided an early version of part of the WebDataKit for .NET beta release that was compiled and tested using the .NET Framework final release.

Compiling the Source Code

As stated earlier, I've supplied - for reference only -- the complete source code as it stands for this case study. I've also included saved versions of some sample HTML and XML pages used in the previous discussion.

You can compile the code using this command:

csc WebFilterViewer.cs SqlResultsForm.cs WebDataEngine.cs SqlEngine.cs TagParser.cs

An executable file WebFilterViewer.exe will be produced as output, and don't worry if during the compilation you get a few warnings of the kind:

WebDataEngine.cs(219,9): warning CS0168: The variable 'ex' is declared but never used

Example use of the WebFilterViewer

In the following screenshot you will see that I've launched the WebFilterViewer, now enhanced with a text field to take an SQL statement. I've entered a URL, in this case file-based, into the URL textbox and I've submitted the HTTP request by pressing the Submit button.

This ListBox directly below the Submit button lists each HTML element of the page in the form key = content, and the element keys can be used to build a SQL statement as shown in the lower text box. For convenience, if you click an item in the list its full key is appended to the text of the SQL statement. Saves a lot of typing, believe me!

In that example, I've picked out a line referring to a book title and I've modified its key to include wildcard *s that pick out all similar items -- i.e. all book titles - as my first selectSpec. For the second selectSpec, I'm picking out author names from a higher level of the HTML hierarchy, again with a strategically placed wildcard *. Thanks to the flattening technique described in Challenge 4, each book title will be combined with the correct author as you'll see next.

Upon pressing the Run SQL button the following form - with an embedded DataGrid - is displayed.

With that example showing how straightforward it is to use the toolkit, you are invited to try it out for yourself using any http:// or file:// based HTML or XML file as a data source. When you've experimented with a few HTML files you could try it out with an XML file like the "Two Gentlemen of Verona" sample file. Here's an example.

The SQL Statement.

The Results..

Thanks to the simple TagParser, XML is supported solely because it has text embedded between <starttag> and </endtag> combinations. This means that no DTD, Schema, or other supporting information is needed and all we need is the .xml file itself.

Case Study Review

Now let's look back over the case study to review the design and implementation issues that have been discussed in relation to the .NET Web Data Toolkit. At the outset I told you that we would look at how to:

  • Extract data from live web pages or files on disk.
  • Make individual HTML and XML elements uniquely addressable.
  • Select elements using wildcards and regular expressions.
  • Flatten a hierarchic HTML or XML structure to a flat relational structure.
  • Parse SQL statements and use SQL to query web pages a though they were relational database tables.

Those issues were presented as a series of challenges to be addressed. I'll list those challenges again and remind you of the solution for each one.

Challenge 1: Retrieving Web Data via HTTP Submissions

The ability to fetch source data from web sites by simulating HTTP GET or POST submissions was the foundation on which everything else was built. We met that challenge with the help of the .NET System.Net.WebRequest and System.Net.WebResponse classes, and I listed a getTextFromURL(.) method that you could use as the basis of any application that needs to fetch web data.

Challenge 2: Parsing HTML and / or XML Elements

The next challenge was to parse the resulting page to make each HTML, XML, or in fact any <starttag> </endtag> delimited element, uniquely addressable via a key value. For that task I used a custom-built TagParser class that will work with any kind of tagged input, but which has a limited knowledge of HTML tags. The use of regular expressions was demonstrated as a mechanism for picking out attribute name / value pairs from within a tag, and the end result of the whole process - as implemented by a getElements(.) method - was a series of TaggedElement instances having key and content properties of the form:

KEY | CONTENT
.table[1].tr[0].td[1].table[1].tr[1].td[0].table[0].tr[2].@bgColor | White

Challenge 3: Selecting Elements

Having a set of individually addressable elements comprising the whole page, we next wanted to pick out sets of elements with similar keys; for example to pick out all book titles with keys matching the pattern .table[1].tr[0].td[1].table[1].tr[1].td[0].table[0].tr[*].td[0].a[0].text[0]. For that pattern-matching task we once again used regular expressions, having first converted the simpler pattern syntax to the full regular expression syntax thus:

\.table\[1\]\.tr\[0\]\.td\[1\]\.table\[1\]\.tr\[1\]\.td\[0\]\.table\[0\]\.tr\[\d+\]\.td\[0\]\.a\[0\]\.text\[0\]

In preparation for SQL support we termed the pattern a selectSpec and passed this in to a getSelectedElements(.) method along with the full list of tagged elements. That method returned a subset of elements that matched the selectSpec.

Challenge 4: Flattening the Tag Hierarchy to form a Relational Table

Combining results from separate selectSpec s into a single relational table would allow us to treat the web data in the same way as any other data source, and present it as such, so that was our next challenge. We adopted a flattening technique to ensure that sets of selected elements were combined into table rows in sensible combinations, even in cases where the source data was not presented originally in the form of a regular table.

We returned the results of the getSelectedRows(.) method as a .NET System.Data.DataTable that could be displayed directly in a System.Windows.Forms.DataGrid.

Challenge 5: Supporting Structured Query Language

The completion of Challenge 4 provided support for SQL statements to SELECT tagged elements FROM web pages URLs as though selecting column data from traditional relational tables, with the results being returned in the form of a relational table. To support SQL syntax more fully we needed to handle WHERE clauses and ORDER BY clauses, which we did by taking advantage of the DataTable.Select(.) method that we wrapped up in our own filterAndSort(.) method.

Limitations and Future Enhancements

As described in this case study, the key challenges have been met and the fundamental pieces are in place for the beta version of the .NET WebDataKit. However, it's not yet industrial-strength and has a few shortcomings compared with the Java implementation. Here I'll describe some of the limitations of the case study software and some of the work yet to be done for the beta release.

Parsing non-well-formed HTML and XML

I'd been using a saved version of the Wrox New Releases web page for some time as a basis for my testing, and when I returned to the live online version I noticed some odd behavior that manifested itself in element keys of the form:

.table[3].tr[0].td[1].table[0].tr[1].td[0].table[0].tr[0].td[0].table[0].tr[2].td[0].table[0].tr[0].td[0].
table[0].tr[1].tr[0].tr[0].td[9].text[0]

I've highlighted the offending portion, showing that what's wrong is the repeated occurrence of tr[] entries in the key. That should not be possible with a well-formed HTML file and indeed the problem lay in the fact that certain portions of the page were not well-formed to the extent that some of the <tr> tags were not closed off by a corresponding </tr> tag as you can see in this HTML fragment taken from the page.

<tr class="conBgColorLightGrey">
<td class="conHXXSBlack"><a class="conHXXSBlack"
href="ACON11.asp&ISBN=1861006152">Beginning ASP.NET using C#</span>&nbsp;</td>
<td class="conHXXSBlack">David Sussman et al&nbsp;</td>
<td><span class="conHXXSBlack" style="text-decoration:none;">1861006152</span></td>
<td><a class="conCodeDownloadLink"
href="ACON12.asp&ISBN=1861006152">Download</a></td>
<tr>
<td class="conHXXSBlack"><a class="conHXXSBlack"
href="ACON11.asp&ISBN=1861005555">Beginning Visual Basic .NET
Databases</span>&nbsp;</td>
<td class="conHXXSBlack">Matthew Reynolds et al&nbsp;</td>
<td><span class="conHXXSBlack" style="text-decoration:none;">1861005555</span></td>
<td><a class="conCodeDownloadLink"
href="ACON12.asp&ISBN=1861005555">Download</a></td>

Mid way through that HTML source a <tr> tag follows directly from a </td> tag where as we would expect to see a </tr> tag in between. Thus the TagParser was fooled into thinking that a new row had started before the previous row had ended.

Clearly we need to give the home-grown TagParser the intelligence to cope with not-so-well-formed HTML, perhaps by inserting a phantom </tr> into the text where we would normally expect to find one. For the record, the original Java implementation was unaffected by such problems because the parsing was performed by the HTML parser provided by the Java Runtime itself, which was designed to make allowances for sloppy HTML.

Impact of web site changes

The previous limitation came to light because the format of the Wrox web page had changed, and you might already be thinking about how this approach is affected by web site changes. In my experience of working with these tools, web pages are typically good for about six months before they change in format and hence break the SQL. A reworking every six months might be fine for your application, but what if you distribute applications based on this technology to other people? Well that's the whole point of using SQL as it allows you to store the page-format-dependent queries separately from the code itself, possibly in a text file. Whenever a page changes, you re-write the SQL to return the same number of columns, then simply distribute the new statement as a "patch" with no new code and no recompilation required.

Selections from multiple pages

This implementation is limited to the selection of data from one page at a time. In principle, it's possible to select data from multiple pages as you would select from multiple tables, using SQL joins. Indeed that's what the Java implementation allows through SQL statements of the form:

SELECT allbooks.table[1].tr[0].td[1].table[1].tr[1].td[0].table[0].tr[*]
.td[0].a[0].text[0] AS allbooks.title
, allbooks.table[1].tr[0].td[1].table[1].tr[1].td[0].table[0].tr[*].td[1]
.text[0] AS author
, topten.table[1].tr[0].td[2].table[0].tr[8].td[0].table[0].tr[*].td[1]
.a[0].text[0] AS topten.title

FROM http://www.wrox.com/Books/Books.asp?section=11_2 AS allbooks
, http://www.wrox.com/ AS topten

WHERE allbooks.title=topten.title

In that statement you can see how multiple pages may be specified in the FROM clause, with each page being aliased to distinguish the elements in the SELECT clause. That functionality will be added into the beta release to bring it in line with the Java implementation.

If you'd like to think about it yourself, I can tell you that elements from the first page should be combined with elements from the second page in a full cross-product as though they automatically belong in the same context. The sensible combinations would then be retained solely as a result of the WHERE clause that equates a value in the first table with a corresponding value in the second table.akin to a primary key / foreign key relationship. That's exactly how a traditional relational join works, and it depends on there being some values in common between the two pages to be joined.

Stateful Pages and Authentication

In this case study we've dealt with selecting data from a static page or from a dynamic page resulting from a single HTTP GET or POST submission. In theory, the idea could be extended to a solution for driving any online application - comprising multiple pages -- automatically. I'm thinking here of writing a program (if you dare) to drive the internet service provided by your bank.

For any such application, we'd need to submit multiple HTTP requests to step through the various pages, taking care that the state was maintained as required by the application through cookies, URL re-writing, or some other technique. We might need also to pass through an authentication stage before gaining access to the application at all. Something for you to think about if you'd like to take this to its logical conclusion, but I'll admit to not being able to help you out just yet.

What I can tell you is that as far as SSL goes, it's possible to submit requests to https:// data sources with no additional code whatsoever, which was not the case with Java until the arrival of the JDK 1.4. Try it out by typing the following URL into the WebFilterViewer: https://www.verisign.com/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq

Potential Applications

As I've taken a single example throughout, I might have left you with the impression that this toolkit is only useful for obtaining a list of books from the Wrox web site. So let me give you some more food for thought.

The majority of developers using the original Java WebDataKit are working with financial web sites such as Yahoo!Finance at http://finance.yahoo.com/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq and FT MarketWatch at http://mwprices.ft.com/custom/ft-com/quotechartnews.asp?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq, to extract and interpret up-to-the-minute share price information. In fact, I developed the toolkit originally to automate aspects of my amateur investing hobby. Subsequently I've used the software for sending SMS messages to mobile phones via a web site that provides such a service, and for querying of on-line timetables as you can see from this screenshot of the original Java tool.

Notice how in that SQL statement I could change the posted parameters (highlighted) to capture data from a journey between any two UK stations on any date.

What each of those applications has in common is the need to send some parameter data to a web site using the GET or POST method, then to capture the response - a simple SMS acknowledgement, complete share price information, or a set of train times as HTML or XML -- for further processing or local storage. For some of you the phrase screen scraping might spring to mind, and as such you can think of any web site as a service provider and this toolkit as a mechanism for automating the use of those online services. no SOAP required.

One final thought is that you might like to think about gathering up-to-the-minute information from a range of web sites to be combined into a single cockpit application, or even your own online portal.

Case Study Information
Author | Tony Loton
Technical Editor | Adam Ryland
Project Manager | Helen Cuthill
Reviewers | Andrew Krowczyk, Saurabh Nandu, Adam Ryland

If you have any questions or comments about this case study, or suggestions for further pieces, please contact the technical editor. If you wish to talk about the case study with other subscribers, please visit our forum here.

C#Today - © 2002 WroxPress Limited


Creating Data Bound Controls

Creating Data Bound Controls
http://www.csharptoday.com/content.asp?id=1813
Article Rating: 4
Published on February 4, 2002
Abstract
In this article, Matthew Reynolds takes a look at how to build Windows Forms controls that support data binding. You’re probably already familiar with how to use data binding with Windows Forms controls supplied with the Framework, but what you may not be familiar with is how to create your own controls that have this functionality. This article will show you how.
Article

In this article, we're going to be taking a look at how to build our own Windows Forms controls that support data binding.

You're probably already familiar with how to use data binding with Windows Forms controls supplied with the Framework (check out http://www.csharptoday.com/content.asp?id=1701&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq for an overview of Data Binding techniques). What you may not be familiar with is how to create your own controls that have this functionality.

In this article we're going to create a new control derived from System.Windows.Forms.ListView . This control will support a new property (among others) called DataSource, which is able to accept a number of different data types. The list will then be populated with the data that it has been given, e.g:

The Problem

The problem that all data binding controls face is that the control itself has to be able to handle a wide variety of data. It's no use having a control that can accept and display a DataSet but not an array, or one that can accept a System.Collections.Hashtable but not a System.Collections.NameValueCollection object.

Thankfully, the Framework defines a number of collections that are extremely useful for managing lists. Whenever we're given an object to bind to, we can check to see if the object supports a number of known types and do something different depending on the type that we get back.

If we look at the definition for System.Array (the class from which all array types are inherited), you'll notice that it implements a number of interfaces:

public abstract class Array : ICloneable, IList, ICollection, IEnumerable

For our purposes, System.ICloneable isn't of much interest. This interface simply allows us to make a copy of the array. However, the other three are interesting.

System.Collections.IEnumerable is implemented on objects that support lists of data. It's used by C# and other managed languages to implement foreach loops, e.g:

foreach(object data in
            myList){
.}

System.Collections.ICollection also implements IEnumerable , meaning that any object supporting ICollection also has to implement IEnumerable . This interface supports extra properties to support thread-safe access, and a Count property to return the size of the collection.

System.Collections.IList also implements ICollection , meaning that any object supporting IList also implements ICollection , and therefore IEnumerable . This interface supports the indexer (implemented through the Item property) to access list elements arbitrarily, and also methods to manipulate the elements of the list, for example Add , Remove , Clear , etc.

In order to add basic information to the list, realistically the only thing that we need to do is look for objects that support IEnumerable . Once we have that, we can configure a foreach loop and add the data to the list. In the first example below, that's exactly what we're going to do.

However, looking for IEnumerable isn't the best approach in all cases. Like the DataGrid control, a common requirement of our new control derived from ListView is to show data contained within a DataSet, or more properly, contained within a DataTable contained within a DataSet.

DataTable does not support any of the interfaces that we have just discussed. However, it does support an interface called System.ComponentModel.IListSource . This interface defines a method called GetList that returns an IList -implementing object. So, if we do have a DataTable, we can find an IList , and therefore we can find an IEnumerable . However, DataTable actually contains rich data expressed in a tabular format. When we have a DataTable, what we want to do is iterate the columns and create a new column in the header of the ListView for each column (this is shown in the first screenshot of this article, shown above.)

There is no interface in the Framework that expressly defines tabular data. When working with a DataTable, what we'll do instead is use a separate class that understands how to work exclusively with DataTable objects. In fact, we'll have one generic class called BoundListViewBinder that understands the basic collection interfaces that we've just discussed, and inherit a new class from this called DataTableBoundListViewBinder and overload methods so that it knows how to work with the special DataTable case.

The Application

For this exercise, we're going to create two separate projects. One will be a Windows Application project and will define a basic Form class that we can paint the new control on. The other project will be a Windows Control Library project and will contain the new ListView class and the support classes.

To kick off, create a new Windows Application project. Call it InfoSchemaViewer (the name will become apparent later).

Next, into the same solution, add a new Windows Control Library project by right-clicking on the Solution 'InfoSchemaViewer' entry in Solution Explorer and selecting Add - New Project . Call the new project DataBoundControls .

To the new DataBoundControls project, add a new User Control by right-clicking on the project and selecting Add - Add User Control in Solution Explorer. Call it BoundListView . Open the code editor for the class, and set it to inherit from System.Windows.Forms.ListView instead of System.Windows.Forms.UserControl .

public class BoundListView : System.Windows.Forms.ListView
{

As we mentioned before, we're going to implement the code that actually does the binding in a separate class called BoundListViewBinder . The default implementation of this will simply look for IEnumerable , and we'll be able to derive new classes from this and use it to bind to specific types of data, like the DataTable. Now add a new class to the project called BoundListViewBinder .

In the first instance, this class just needs to be tweaked slightly to include a private member to hold a reference back to the appropriate BoundListView . We also need to replace the constructor with one that requires a BoundListView , and add a read-only property. It will look like this:

public class BoundListViewBinder
{
//
            members...
private BoundListView _boundListView;

public
            BoundListViewBinder(BoundListView boundListView)
{
//
            store...
_boundListView = boundListView;
}

//
            BoundListView property...
public BoundListView
            BoundListView
{
get
{
return
            _boundListView;
}
}
}

The BoundListViewBinder instance that we're using will either be explicitly supplied by the consumer of the object (in the case that the developer builds his or her own BoundListViewBinder to suit his or her needs), or will be chosen by the BoundListView control itself. This will be done through the Binder property of BoundListView . If this property is null, it will choose one, otherwise it will use the one that has been set.

As well as storing this Binder property, we also need a field to hold the data itself. Add these members to BoundListView :

public class BoundListView : System.Windows.Forms.ListView
{
// members...
private object _dataSource;
private BoundListViewBinder _binder;

Also, add these methods. We'll add the references DataBind method in a moment.

// Binder property...
public
            BoundListViewBinder Binder
{
get
{
return
            _binder;
}
set
{
// set it...
_binder =
            value;

// update...
DataBind();
}
}

//
            DataSource property...
public object
            DataSource
{
get
{
return
            _dataSource;
}
set
{
// set it...
_dataSource =
            value;

// update...
DataBind();
}
}

By default, the ListView control will want to use the "large icon" view as its display mode. We want the "details" view to be the default view mode. Find the constructor for BoundListView and add some code to set the View property to View.Details :

public BoundListView()
{
// This call
            is required by the Windows.Forms Form
            Designer.
InitializeComponent();

// set the default
            properties...
View = View.Details;
}

For now, we don't want to implement DataBind as we want to ensure we can create an instance of this control and use it. Add this stub method that we'll add some functionality to later:

// DataBind method...
protected void
            DataBind()
{
}

At this point, open the Designer for Form1. In the Toolbox, you should find the BoundListViewControl .

Draw a new BoundListView control onto Form1 . Make it occupy most of the available space and set the Anchor property to Top, Left , Bottom , Right.

At this point, because we can see the control, we've proven that the control is working. Now all we need to do is implement the DataBind method.

Implementing "DataBind"

In the first instance, we want to get the control to bind to a standard array of data. System.Array (the class from which all array types are derived), and as we've already seen this class supports these interfaces:

public abstract class Array : ICloneable, IList, ICollection,
IEnumerable

IEnumerable is the one we want to use here, as we just want a basic list of data. We'll add new rows to the ListView for each item in the array. As the array of data has one dimension (as opposed to a table of data, which has two dimensions - rows and columns), we only need to add a single column to the BoundListView .

Adding data to the list view will be a two-phase affair, implemented in two separate methods. UpdateColumns will create the new columns that the data requires, and as I just mentioned, in the case of an array this will be a single column. UpdateData will iterate the data and physically add the rows.

Adding the column is easy. We create a new System.Windows.Forms.ColumnHeader object, give it a default Text value, and also set the Width property to be equal to the width of the BoundListView control that it's associated with. Add this code to BoundListViewBinder :

// UpdateColumns method...
public virtual
            void UpdateColumns(object dataSource)
{
//
            update...
ColumnHeader header = new
            ColumnHeader();
header.Text = "Data";
header.Width =
            BoundListView.ClientSize.Width;
BoundListView.Columns.Add(header);
}

When it comes to UpdateData , things get a little more complicated. We want to implement multiple overloaded methods to handle the basic interface and object types that the control handles by default. The basic entry point for this method will take a System.Object as it's parameter and attempt to de-reference a more appropriate type from that point. In this case, we just want it to check to see if the object implements IEnumerable and, if it does, pass off to another method. Add this code to BoundListViewBinder :

// UpdateData method...
public virtual
            void UpdateData(object dataSource)
{
// get an interface that
            we can work with...
IEnumerable enumerable =
            null;
if(dataSource is IEnumerable)
enumerable =
            (IEnumerable)dataSource;

// can we do this?
if(enumerable
            != null)
{
UpdateData(enumerable);
return;
}

//
            exception...
throw new Exception("Data source cannot be
            bound");
}

Here's the implementation for the first overloaded version of UpdateData . Notice how we use a simple foreach loop to walk to the data in the object. In fact, we could call foreach on the object directly without having first converted to an IEnumerable type. However, we do it here because we want to handle the exception. If we know the object definitely supports IEnumerable , we'll definitely be able to call foreach . If we don't have this knowledge, an exception could be thrown. As we'll see later, checking to see if the object supports IEnumerable explicitly gives us the option of using an alternate overloaded version of UpdateData to handle a different kind of data.

protected void UpdateData(IEnumerable
            enumerable)
{
// get the enumerator...
foreach(object data
            in enumerable)
{
// add it to the
            list...
AddData(data);
}
}

Next, here's the basic AddData method:

protected void AddData(object
            data)
{
BoundListView.Items.Add(data.ToString());
}

At this point we can start to implement the DataBind method. Open the code editor for Form1 , and add this code to the Load event handler (if you don't have a Load event handler, open the Designer for the form and double-click on the background).

private void Form1_Load(object sender, System.EventArgs e)
{
// create an array and bind it...
string[] names = { "Alex", "Zoe", "Sarah", "Len", "Darren" };
boundListView.DataSource = names;
}

When the DataSource property is set, BoundListView will set its internal _dataSource field and also call the DataBind method. At this point, we need to choose a BoundListViewBinder derived object (or just the object itself) to do the actual binding. Earlier we mentioned that we can either specifically give it an object that understands the kind of data we are binding, or we can choose one based on what we know about the data. We'll create an instance of BoundListViewBinder as a default. Here's the implementation for DataBind :

// DataBind method...
protected void
            DataBind()
{
// clear...
Clear();

// do we have any
            data?
if(DataSource == null)
return;

// choose a binder
            based on what we know...
BoundListViewBinder useBinder =
            Binder;
if(useBinder == null)
{
// still
            nothing?
if(useBinder == null)
useBinder = new
            BoundListViewBinder(this);
}

//
            update...
useBinder.UpdateColumns(DataSource);
useBinder.UpdateData(DataSource);
}

You can see there how we use the Binder property to retrieve the BoundListViewBinder that we require. If we don't have one, we create a new one (the odd looking if structure in that code snippet will be explained later on). Once we have created the default, we call UpdateColumns and UpdateData .

Run the project now and you should see this:

Notice how the column header exactly fits the available space inside the control.

Binding to DataTable Objects

We've proven now that we can bind to simple array data. What happens if we try and bind to a table in a DataSet, or rather to a DataTable. The data that we're going to use will be drawn directly from a SQL Server, and I'll be introducing the "INFORMATION_SCHEMA" views into the discussion and using those.

As you may know, SQL Server uses internal tables in the master database to store the structure of the databases that it has access to. The structure of the information is relatively complex, and optimized to the way that SQL Server needs to retrieve and work with this data. However, we can extract more easily understood data from a set of views known as the "INFORMATION_SCHEMA" views. For example, "INFORMATION_SCHEMA.TABLES" returns a list of all the tables in the database. "INFORMATION_SCHEMA.COLUMNS" returns a list of all the columns. Some of the views are a little more obscure. By default, the list as defined in the T-SQL documentation includes some views that actually are not installed into SQL Server in a standard installation. However, here is a list of the views that are installed by default:

  • INFORMATION_SCHEMA.Check_Constraints - shows the CHECK constraints in the database,
  • INFORMATION_SCHEMA.Column_Domain_Usage - shows which columns use which user-defined data types,
  • INFORMATION_SCHEMA.Column_Privileges - shows which privileges are assigned to columns,
  • INFORMATION_SCHEMA.Columns - shows the columns in all tables,
  • INFORMATION_SCHEMA.Constraint_Column_Usage - shows the constraints on a per-column basis,
  • INFORMATION_SCHEMA.Constraint_Table_Usage - shows the constraints on a per-table basis,
  • INFORMATION_SCHEMA.Domain_Constraints - shows the rules for user-defined data types,
  • INFORMATION_SCHEMA.Domains - shows the user-defined data types,
  • INFORMATION_SCHEMA.Key_Column_Usage - shows which columns are the key columns in the tables,
  • INFORMATION_SCHEMA.Referential_Contraints - shows a list of the foreign-key/primary-key relationships,
  • INFORMATION_SCHEMA.Schemata - shows a list of all available databases,
  • INFORMATION_SCHEMA.Table_Constraints - shows the constraints against a table,
  • INFORMATION_SCHEMA.Table_Privileges - shows the privileges against a table.
  • INFORMATION_SCHEMA.Tables - shows all tables,
  • INFORMATION_SCHEMA.View_Column_Usage - shows which columns are used by which views,
  • INFORMATION_SCHEMA.View_Table_Usage - shows which tables are used by which views,
  • INFORMATION_SCHEMA.Views - shows all views.

First off, let's add a new drop-down list to Form1 to allow the user to select one of these views. Open the Designer and add a new ComboBox control. Set its Name property to comboViews and its Anchor property to Top, Left , Right. Set its DropDownStyle property to DropDownList .

We can add the list of views to the control by editing the Items property. Add the views you're interested in ( INFORMATION_SCHEMA.Tables and INFORMATION_SCHEMA.Columns are the best two to add if you're not into the idea of adding all of them).

Close the Collection Editor and double-click on the ComboBox control to create a SelectedIndexChanged event handler. The event handler needs to have some pretty ordinary code that connects to the database and populates the DataSet. After we've populated the DataSet, we set the BoundListView control's DataSource property with the DataTable .

private void
            comboViews_SelectedIndexChanged(object sender,
System.EventArgs
            e)
{
DataSet dataset = null;
SqlConnection connection =
            null;
try
{
// connect...
connection = new
            SqlConnection(ConnectionString);
connection.Open();

//
            run...
SqlDataAdapter adapter = new SqlDataAdapter(
"select *
            from " + comboViews.SelectedItem, connection);
dataset = new
            DataSet();
adapter.Fill(dataset);
adapter.Dispose();

//
            close...
connection.Close();
}
catch(Exception
            ex)
{
MessageBox.Show(ex.GetType().ToString() + ":" +
ex.Message);
return;
}
finally
{
if(connection
            != null)
connection.Close();
}

//
            bind...
boundListView.DataSource =
            dataset.Tables[0];
}

Run the project and select one of the views from the list. You'll see this exception:

What's happening here is that System.Data.DataTable does not support IEnumerable . In fact, it supports:

public class DataTable : MarshalByValueComponent, IListSource,
ISupportInitialize, ISerializable

System.Component.IListSource is the interesting one here. The fact that this interface is supported tells us that the DataTable does support a list of items, but that the object has to do some extra work to return an IList -supporting object back to us. If we test for this interface, we can call the GetList method on it and get an IList -supporting object. As IList inherits from IEnumerable , we'll be able to use our existing overloaded UpdateData method.

Add this code to the UpdateData method of BoundListViewBinder to check for the IListSource interface and call an overloaded version of UpdateData :

// UpdateData method...
public virtual
            void UpdateData(object dataSource)
{
// get an interface that
            we can work with...
IEnumerable enumerable =
            null;
if(dataSource is IEnumerable)
enumerable =
            (IEnumerable)dataSource;

// can we do this?
if(enumerable
            != null)
{
UpdateData(enumerable);
return;
}

//
            does it support IListSource?
IListSource listSource =
            null;
if(dataSource is IListSource)
listSource =
            (IListSource)dataSource;

// this one?
if(listSource !=
            null)
{
UpdateData(listSource);
return;
}

//
            exception...
throw new Exception("Data source cannot be
            bound");
}

We need to provide another version of UpdateData . This version accepts an IListSource :

protected void UpdateData(IListSource
            listSource)
{
// get a list...
IList list =
            listSource.GetList();
if(list !=
            null)
{
UpdateData((IEnumerable)list);
return;
}

//
            otherwise...
throw new Exception("IListSource did not return an
            IList");
}

If you run the project now, you won't see the exception:

Creating "DataTableBoundListViewBinder"

What we have to do now is build another BoundListViewBinder class that works with DataTable objects. This class will understand how to iterate the columns in the table, and how to properly populate new ListViewItem objects as they are added to the view to give our tabular view.

Create a new class called DataTableBoundListViewBinder and inherit it from BoundListViewBinder . You'll also need to explicitly provide a new constructor that calls the base class' constructor and remove the existing parameterless constructor that VS .NET creates for us.

public class DataTableBoundListViewBinder : BoundListViewBinder
{
public DataTableBoundListViewBinder(BoundListView boundListView)
: base(boundListView)
{
}

To process the columns, we just walk the Columns collection of the table and create new ColumnHeader objects for each. Add this code to DataTableBoundListViewBinder :

public override void UpdateColumns(object
            dataSource)
{
// go through the columns...
DataTable table
            = (DataTable)dataSource;
foreach(DataColumn column in
            table.Columns)
{
// create a new header...
ColumnHeader
            header = new ColumnHeader();
header.Text =
            column.ColumnName;
header.Width = 100;

// add
            it...
BoundListView.Columns.Add(header);
}
}

The new version of UpdateData is similar. With every iteration of the Rows collection of the table, we add a new DataRowListViewItem to the list (we'll build this new class in a moment).

public override void UpdateData(object dataSource)
{
// do each row...
DataTable table = (DataTable)dataSource;
for(int n = 0; n < table.Rows.Count; n++)
BoundListView.Items.Add(new
DataRowListViewItem(table.Rows[n]));
}

DataRowListViewItem is derived from System.Windows.Forms.ListViewItem . What we do here is iterate all of the available columns and update the Text property to change the data for the first ("zeroth") column and add new SubItems for each additional column. Create a new class called DataRowListViewItem , set it to derive from ListViewItem , remove the existing constructor, and replace it with this one:

public class DataRowListViewItem :
            ListViewItem
{
// constructor...
public
            DataRowListViewItem(DataRow row)
{
// add the items...
int
            numColumns = row.Table.Columns.Count;
for(int n = 0; n <
            numColumns; n++)
{
// first?
if(n == 0)
this.Text =
            row[n].ToString();
else
this.SubItems.Add(row[n].ToString());
}
}
}

Now all that remains is to get BoundListView to use DataTableBoundListViewBinder rather than BoundListViewBinder . We can do this by adding a test to the DataBind method:

// DataBind method...
protected void
            DataBind()
{
// clear...
Clear();

// do we have any
            data?
if(DataSource == null)
return;

// choose a binder
            based on what we know...
BoundListViewBinder useBinder =
            Binder;
if(useBinder == null)
{
// test for types that we
            know about...
if(DataSource is DataTable)
useBinder =
new
            DataTableBoundListViewBinder(this);

// still
            nothing?
if(useBinder == null)
useBinder = new
            BoundListViewBinder(this);
}

//
            update...
useBinder.UpdateColumns(DataSource);
useBinder.UpdateData(DataSource);
}

Remember that if we want to build our own classes to support different data types, we can explicitly tell BoundListView to use them by setting the Binder property. What we've done here is add an automatic action to BoundListView when we're given a DataTable .

Run the project now and you should see this:

Summary

There you go! A version of the ListView control that can display a DataTable, or an array of data automatically.

Although the ListView control does handle data binding through the DataBindings property available to all controls, what we've done here is given the ListView the ability to bind to a list of data with a single call - something that it could not do originally. This article is a great demonstration of how we can reuse and extend the Framework's standard controls and how we can work with lists of data.

Article Information
Author | Matthew Reynolds
Technical Editor | Adam Ryland
Author Agent | Charlotte Smith
Project Manager | Helen Cuthill
Reviewers | Andrew Krowczyk, Susan Connery

If you have any questions or comments about this article, please contact the technical editor.

C#Today - © 2002 WroxPress Limited


Working with Web Data in C#

Working with Web Data in C#
http://www.csharptoday.com/content.asp?id=1808
Article Rating: 5
Published on January 28, 2002
Abstract
Think of the World Wide Web as a giant database management system in which the web sites are the databases and the web pages are the database tables. Now think about how useful it would be to gather data from those tables in a form that could be processed, queried, and combined with other, otherwise unrelated, data in new combinations…using C# and the .NET framework. In this article, Tony Loton looks at how to: drive HTTP GET and POST requests to web sites using the WebRequest and Stream classes, capture the responses via the WebResponse class, and pick out specific nuggets of information from a web page using regular expressions and the Regex class. The end result is a small but useful engine and a demonstration application, that goes a long way towards allowing data to be selected from web pages as though selecting from relational database tables, using SQL syntax.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

Think of the World Wide Web as a giant database management system in which the web sites are the databases and the web pages are the database tables. Now think about how useful it would be to gather data from those tables in a form that could be processed, queried, and combined with other -- otherwise unrelated -- data in new combinations.using C# and the .NET framework.

In this article you will learn how to:

  • Drive HTTP GET and POST requests to web sites using the WebRequest and Stream classes.
  • Capture the response - containing HTML or XML data - via the WebResponse class.
  • Pick out specific nuggets of information from a web page using regular expressions and the Regex class.

The end result will be a small but useful engine and a demonstration application, that goes a long way towards allowing data to be selected from web pages as though selecting from relational database tables, using SQL syntax.

Introduction

The seeds of this idea were sown in my original article "Accessing the World's Biggest Database" published on the JavaWorld web site in March 2001 at the URL

http://www.javaworld.com/javaworld/jw-03-2001/jw-0316-webdb.html?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq. That idea manifested itself in the WebDataKit commercial implementation, an early version of which was written up as a case study in "Professional Java Data" (Wrox Press). If this article whets your appetite for finding out how the idea originated, or how to take it further, those will be useful sources of information.

Having tested out the principles with Java, I decided to turn my attention to the Microsoft platform, first using - believe it or not - plain old (actually plain new) Microsoft Excel and then using C# and .NET. In this article we'll concentrate on the C# / .NET approach, but I'll use Excel as a handy way to introduce the problem we're looking to solve.

System Requirements

To write this article I used the Visual Studio.NET beta 2 release, and for the (optional) Microsoft Excel trials I used Excel 2002 that came bundled with Office XP.

I'll assume a basic knowledge of the Visual Studio.NET environment and the C# language, to a level at which you can compile and run an application if I provide the source (.cs) files.

Defining Problem

As a regular author and reviewer for Wrox, and a regular reader of Wrox books, I'm always interested to find out new titles will be appearing soon, which I can discover by visiting this URL (http://www.wrox.com/Books/Books.asp?section=1_2&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq):

Currently I'm interested to know what .NET books will be appearing in the near future, but I'm too lazy to visit the site and trawl through the list manually. Wouldn't it be great if I had a program, a software agent, which would visit the web page and tell me if any .NET books were listed?

The shortest route to automation of that task is to use the support for Web Queries provided by Microsoft Excel, so we'll look briefly at that first.

Excel Web Queries

If you have a suitable version of Excel (see System Requirements) you can try out a very simple web query by typing in and running this basic macro:

Sub WroxFutureTitles()

Set thisSheet = Workbooks(1).Worksheets(1)
Set theseResults = thisSheet.QueryTables _
.Add(Connection:="URL;http://www.wrox.com/Books/Books.asp?section=1_2", _
Destination:=thisSheet.Cells(1, 1))
With theseResults
.WebSelectionType = xlEntirePage
.WebFormatting = xlNone
.Refresh
End With

End Sub

The end result is a worksheet containing the HTML text elements comprising the page, each element being uniquely addressable via a cell reference. In the following screenshot you can see that the new book titles appear in cells D8:D23. I'm sure that for some of you it would be quite trivial to enhance the original macro to traverse column D looking for cells containing the text ".NET"; thus achieving my original objective.

At this point, start to think about some of the other potential applications. What about a macro that visits a financial web site to pick out the latest share prices for the stocks in your portfolio, with an enhancement to highlight the biggest risers? If you have a potential application in mind, and you're comfortable using VBA macros, feel free to duck out now to experiment. After which, I hope you'll come back!

If you're staying with me, or you've just come back (nice to see you again), we'll now look at how to solve the same problem using C# and .NET. In a nutshell, our C# solution must:

  • Allow text content to be captured from web sites.
  • Make each HTML (or XML) element uniquely addressable.

And for good measure

  • Allow sets of elements to be picked out if they match certain criteria (like containing the text ".NET").

Defining Solution

One approach would be to build on the functionality provided by Excel, perhaps by driving the Excel application itself - via its COM interfaces - from a C# program, or by treating an Excel spreadsheet with an embedded web query as an ODBC data source to be read via the .NET OleDBDataAdapter.

I failed on both counts. In the former case, this was probably because my COM skills are so rusty after spending several years with my head in Java. In the latter case, it was as a result of this dialog message:

Yes I did try to reuse it again later with OleDbDataAdapter, to no avail, and later I noticed the following statement in the .NET Data Providers section of the .Net Framework Developer's Guide :

The OLE DB .NET Data Provider does not work with the OLE DB Provider for ODBC (MSDASQL).

In any event, it turns out that piggy-backing Excel is perhaps not the best approach. It limits the user base for my killer applications (yes, I have plans) to those users with Excel, and it restricts the deployment location to the client tier only.

One thing I'd really like to do is to capture data from various third-party web sites into my own web tier, then combine the data and re-present it in the form of a portal based on ASP pages and / or Web Forms. So, in this article, we'll aim for a self-contained C# / .NET solution that will potentially run on any tier without unnecessary dependencies, as does my original Java implementation.

Component Design

Our requirements will be handled by a single class, called WebDataEngine, which will live within the webdata namespace. Here's how the WebDataEngine.cs file begins, with a set of using statements, the class declaration, and a parameterless constructor :

using System;

using System.Collections;
using System.Net;
using System.IO;
using System.Text;
using System.Text.RegularExpressions;

namespace webdata
{
public class WebDataEngine
{

public WebDataEngine() {}

The class will have just three (static) methods to perform these three functions:

  • getTextFromURL(.) - Submit a HTTP request to a URL and capture the response.
  • getElements(.) - Return a list of uniquely adressable HTML tagged elements.
  • getFilteredElements(.) - Filter the elements using regular expressions.

Typically these three methods will be invoked in sequence, each taking input from the previous one.

Submitting a HTTP request and capturing the response

Most requests for web pages are issued using the HTTP GET method. The GET method allows us to specify the URL of the page and - in the case of a dynamic (JSP, ASP or CGI) page - a set of query parameters appended to the URL.

http://www.somesite.com/somewhere/somepage.html?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq (for a static page)

or

http://www.somesite.com/somesearch?searchword=someword&numresults=10&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq (for a dynamic page)

Some sites require dynamic pages to be requested via the HTTP POST method, which hides the query parameters in the URL and passes them separately as a stream of data. You can determine which method is required, and what parameter values must be passed, by analysing the HTML form tags.

The following HTML form definition (simplified) was extracted from the URL http://www.wrox.com/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq and it tells me that we may submit a HTTP POST request to the URL http://www.wrox.com/Search/Search.asp?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq, passing the string textfield=mysearchterm as the POSTed stream of data.

<form method=" post " action=" /Search/Search.asp " id=form1
            name=form1>
.
<input type="text" name=" textfield " size="15"
            maxlength="50">
<input type="image"
            src="/Includes/images/search.gif" name="submit" id=submit border="0"
            width="120" height="12">
.
</form>

Incidentally, if the method had been "get", we would in that case be able to submit a HTTP GET request via the URL http://www.wrox.com/Search/Search.asp?textfield=mysearchterm&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq.

Our engine will support both submission methods for getting text content from a URL. So, the parameters to the getTextFromURL(.) method will be the URL (which may have query parameters appended), the postText (which is optional and only relevant to POST submissions), and a string indicating which method should be used - "GET" or "POST":

public static String getTextFromURL (String url, String postText
, String method)
{

The first step in this method creates a WebRequest and sets its HTTP submission method:

WebRequest request = WebRequest.Create (url);
request.Method = method;

If the method is "POST" we obtain a Stream to which we can write the postText:

   if (method.Equals("POST")
            && (request is HttpWebRequest))
{
request =
            (HttpWebRequest) request;
request.ContentType =
            "application/x-www-form-urlencoded";
request.ContentLength =
            postText.Length;

Stream outputStream =
            request.GetRequestStream ();
StreamWriter writer=new
            StreamWriter(outputStream);

writer.Write(postText);

writer.Close();
outputStream.Close
            ();
}

Did you notice that we tested the type of the request, which was created by the call to WebRequest.Create(url), before we posted the postText ? All URLs beginning http:// or https:// will result in a HttpWebRequest instance (subclass of WebRequest) that allows data to be posted. However, if we supply a URL beginning file:// relating to a file held on disk, the resulting WebRequest will not be of type HttpWebRequest, and will not support the posting of data.

Whether we post data or not, we must capture the response. For simplicity we're reading the entire response - i.e. the full HTML page - into a text string and returning this to the caller:

WebResponse response = request.GetResponse();
Stream responseStream = response.GetResponseStream();
StreamReader reader = new StreamReader
(responseStream,System.Text.Encoding.Default);

String text=reader.ReadToEnd ();

reader.Close();

return text;
}

Now that we have a way of getting hold of the content of a web page, we need to make each of the elements within the page individually addressable, so that we can pick out just the ones we want.

Making HTML elements uniquely addressable

The getElements(.) method will take in a string containing the entire content of a web page. It will give back a list of elements (smaller text strings), each having a unique key or address of the element within the page. Notice that this method simply delegates to an additional class called TagParser :

public static ArrayList getElements(String taggedText)
{
String[] htmlTags=new String[] {"<html>"
,"<table>"
,"<tr>"
,"<th>"
,"<td>"
};

TagParser tagParser=new TagParser(htmlTags);

return tagParser.getElements(taggedText);
}

We won't be distracted by the code of the TagParser class, because that class simply encapsulates an algorithm that won't teach anything new about C# or .NET. It's not even a well-written algorithm, rather a quick-and-dirty approach to picking out text from within tags so as to move this article along. We'll treat it as a black box that is easily replaceable and for which we know the inputs and outputs.

Inputs and Outputs of the TagParser

Here is a very simple extract of a HTML page:

<html>
<body>
<table>
<tr><td>Wrox</td><td>Press</td></tr>
<tr><td>Tony</td><td>Loton</td></tr>
</table>
</body>
</html>

We would request this via the getTextFromURL(.) method, and supply to the TagParser as a single string:

<html><body><table><tr><td>Wrox</td><td>Press</td></tr><tr><td>Tony</td><td>Loton</td></tr></table></body></html>

As output, we'll get an ArrayList of TaggedElement s, key-content pairs of the form:

.html#0.table#0.tr#0.td#0 = Wrox
.html#0.table#0.tr#0.td#1 = Press
.html#0.table#0.tr#1.td#0 = Tony
.html#0.table#0.tr#1.td#1 = Loton

The text string "Tony" is (reading backwards) contained within the 0th table data tag (.td#0), of the 1st table row (.tr#1), of the 0th table (.table#0) of the HTML page. Those keys are like cell references in Excel, shown earlier, but now not limited to two dimensions.

Notice that the tags used to formulate the element keys are limited to those specified in the htmlTags array within the getElements(.) method listed above. HTML supports many tags - like <b> (bold) - that have no structural significance, and would make the keys unnecessarily long, so we ignore them. In fact, in that example we even ignored the <body> tag by not including it in the htmlTags array.

Finally, you'll be interested to know that the resulting ArrayList contains key-value pairs encapsulated in instances of a TaggedElement class that has the following definition. Notice the use of read-only C# key and content properties here:

public class TaggedElement
{
private
            String elementKey;
private String elementContent;

public
            TaggedElement(String key, String
            content)
{
this.elementKey=key;
this.elementContent=content;
}

public
            string key
{
get { return elementKey; }
}

public
            string content
{
get { return elementContent;
            }
}
}

So that describes the inputs and outputs of the TagParser, which are exactly the same as the inputs and ouputs of the getElements(.) method that wraps it.

Filtering with regular expressions

Each element is individually addressable via its key, and now we just need a mechanism for picking out the single elements, or groups, that we're interested in. We'll use the .NET Regular Expressions feature for this.

A full coverage of regular expressions is beyond the scope of this article, but you will learn a little by example when we look at the sample application later.

The final method of our WebDataEngine class, shown next, takes in a complete list of TaggedElements - called originalElements - obtained from the previous getElements(.) method. It also takes in a regular expression - called filter - as a string that will be used to filter the original elements.

From the filter string we will create at Regex instance, which will be matched against each element's key or content (according to the setting of the contentFilter flag) in turn. The matching elements will be appended to a new ArrayList named filteredElements.

public static ArrayList
            getFilteredElements(ArrayList originalElements, String filter, bool
            contentFilter)
{
ArrayList filteredElements=new
            ArrayList();

Regex filterRegex = new Regex(filter);

foreach (TaggedElement element in originalElements)
{
Match theMatch=null;

if (contentFilter) theMatch =
            filterRegex.Match(element.content);
else theMatch =
            filterRegex.Match(element.key);

if (theMatch.Success)
            filteredElements.Add(element);
}

return
            filteredElements;
}

UI Design

To test-drive the WebDataEngine we'll use a form that allows you to enter:

  • The URL of the source web page
  • Some additional text to post (optional)
  • A (regular expression) filter for element keys
  • A (regular expression) filter for element contents

With a little slight of hand I've set out the fields of this WebDataForm to capture the required values in such a way, and with appropriate labels, to make the jump to something approaching SQL. Remember that one of my ambitions was to use SQL syntax to select data from web pages?

This maps onto the parameters of the three WebDataEngine methods as follows:

SELECT keyFilter AS field
FROM url WITHPOST postText
WHERE field MATCHES contentFilter

UI Code

The code for the WebDataForm is provided for download, but is not essential to the plot, as it simply calls the methods of the WebDataEngine - in turn - in response to the user pressing the Run button, then adds the resuting elements into the list box. Here is the essential code:

String method="GET";
if
            (postBox.Text.Length>0) method="POST";

String
            taggedText=webdata.WebDataEngine.getTextFromURL(urlBox.Text
,postBox.Text,method);

ArrayList
            elements=webdata.WebDataEngine.getElements(taggedText);

elements=webdata.WebDataEngine.getFilteredElements(elements
,keyFilterBox.Text,false);

elements=webdata.WebDataEngine.getFilteredElements(elements
,contentFilterBox.Text,true);

The Sample Application

As a demonstration of how to use this sample front-end to the WebDataEngine, let's go back to the original problem that we solved using Excel. That problem was to pick out the latest .NET books from the Wrox web site:

Using default regular expression filters of \w* (match anything), we get:

Next you will see what happens if we add an appropriate key filter in the SELECT clause:

That keyFilter entry in the SELECT field should have been self-explanatory up to the final few characters, which is where the magic happens. Here are those final characters:

.tr#\d+.td#0$

The text \d+ is regular expression syntax for matching one or more decimal digits, and the text $ is regular expression syntax for saying the the pattern must match right up to the end of the line.

So, now we have a list containing just the book titles from that page, but remember that we want to know specifically about the .NET books. So, we add a (regular expression) content filter in the WHERE.MATCHES field, like this:

Problem solved, but keep in mind the fact that the WebDataEngine is the essential component and this WebDataForm is just an example front-end. Now that we have the ability to capture web data into our C# programs, we need not be limited by simple displaying the results in a list box. Why not load the data into a database for later processing, or re-present the filtered results - along with data from other pages - as a summary web page on your local intranet?

Application Setup

Everything you need is provided in the CSharpWebData project. Just load this project into Visual Studio .NET, build, and run the WebDataForm.

Review and Further Work

Our WebDataEngine has reduced the problem of selecting the latest .NET books from the Wrox site down to the problem of devising an SQL statement like this:

SELECT .html#0.table#1.tr#0.td#1.table#1.tr#1.td#0.table#0.tr#\d+.td#0$ AS field
FROM http://www.wrox.com/Books/Books.asp?section=1_2
WHERE field MATCHES \w*.NET\w*

In solving that problem we've created a handy WebDataEngine that may be used in a great many applications, based on the idea of extracting useful data automatically from the World Wide Web. Think about portals, web agents, share price trackers, and the like.

The ideas I've discussed are not brand new with this article, and my coverage of the C# / .NET implementation reflects the work that I've done previously with Java. The Java work went much further and it provides some interesting ideas for me (or you) to tackle next.

Futher Work

Here are some of the ideas I have for building on the work done here.

HTML and XML Parsing

For convenience, I put together a quick-and-dirty parser, the TagParser class, to strip out the text between HTML tags, and to form unique keys by numbering the tags and each level of the HTML hierarchy. Did I mention that it works with XML too? Take a look at the following screenshot in which I examine the Two Gentlemen of Verona example XML file to pick out all the Acts / Scenes in which the character "SILVIA" speaks:

Looks impressive, but this parser has no comprehension of what constitutes valid HTML or XML content, and it operates on any text file that simply contains <starttag> and </endtag> combinations. I hid the code, and wrapped the functionality up in the getElements(.) method of the WebDataEngine to make it easily replaceable.

Replacing that parsing code seems a a logical next step, and - for the record - in the Java implementation I used the HTML Parser provided in the java.swing package and the SAX XML Parser provided by the Java API for XML Parsing.

I've not yet looked too deeply into the .NET support for XML parsing, but I think the XMLReader class would be a good starting point. I'd expect it to cope with XHTML, but for traditional HTML you might just need to go back to a home-grown parser like the one I've used. I dare say you could build one using regular expressions!

Improved SQL Support

I introduced the idea of using SQL as the syntax for extracting web data by arranging the fields of the WebDataForm as a SQL-like template.

Taking this to its logical conclusion - as I did with Java - we would end up with a full SQL engine that accepts queries as free form text. It would allow data to be selected from multiple pages (think of them as tables) and would join together the results to form new row sets.

To whet your apetite, take a look at the following SQL statement that selects the titles and authors (again, from the URL http://www.wrox.com/Books/Books.asp?section=11_2&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq) of those books that appear in the top ten list at the URL http://www.wrox.com/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq. Note the slight difference in syntax used by the Java implementation:

SELECT allbooks.table[1].tr[0].td[1].table[1].tr[1].td[0].table[0].tr[*]
.td[0].a[0].text[0] AS allbooks.title
, allbooks.table[1].tr[0].td[1].table[1].tr[1].td[0].table[0].tr[*].td[1]
.text[0] AS author
, topten.table[1].tr[0].td[2].table[0].tr[8].td[0].table[0].tr[*].td[1]
.a[0].text[0] AS topten.title

FROM http://www.wrox.com/Books/Books.asp?section=11_2 AS allbooks
, http://www.wrox.com/ AS topten

WHERE allbooks.title=topten.title

To prove that it really is possible to do that, here's a screenshot showing the query running in the Java implementation:

You might be interested to know that "Professional Java Data" (Wrox Press) contains a Case Study chapter describing how something like that can be built using Java. The ideas should be readily convertible into C#.

Conclusion

We started by looking at a problem that could be solved by extracting data from live web pages. That problem was "how to find out which Wrox future titles relate to the .NET platform", and we discussed briefly how that problem could be solved using nothing more sophisticated than Microsoft Excel and a little VBA. A perfectly valid approach if you prefer.

For those of you more interested in C# and the .NET framework (most of you I guess), we stepped through a solution based on the implementation of a useful WebDataEngine class that has just three static methods. In this class we made use of:

  • the WebRequest class for issuing HTTP requests to web sites
  • the WebResponse class for capturing the HTTP response
  • the Regex class for performing pattern matching on character strings using regular expressions

Because these are .NET classes, not special to C#, there's no reason why you couldn't apply exactly the same techniques using Visual Basic or C++.

A sample front-end was demonstrated in the form of the WebDataForm, which we treated as a template for running basic SQL queries against web sources.

Finally I told you about some of the enhancements that I plan for this C# code, to bring it in line with the Java implementation. Maybe you'll beat me to it!

Article Information
Author | Tony Loton
Technical Editor | Adam Ryland
Author Agent | Charlotte Smith
Project Manager | Helen Cuthill
Reviewers | Iikka Meriläinen

If you have any questions or comments about this article, please contact the technical editor.

C#Today - © 2002 WroxPress Limited


Mapping Windows Forms to A Relational Database Part 2

Mapping Windows Forms to A Relational Database Part 2
http://www.csharptoday.com/content.asp?id=1700
Article Rating: 4.5
Published on January 8, 2002
Abstract
The purpose of this article series is to create an application for saving time and effort in mapping relational databases. In this installment, Levent Camlibel focuses on generating Windows Forms and C# objects, using the custom objects developed in the first part of the article.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

The purpose of this article series is to create an application for saving time and effort in mapping relational databases. In this final part we will focus on generating Windows Forms and C# objects, using the custom objects developed in the first part of the article.

System Requirements

This software requires Visual Studio .NET Beta 2 or later to compile, and Microsoft SQL Server or MSDE for database access.

Project Structure

There are three Visual Studio .NET projects and three namespaces in the project group. These are:

  • Mapping.CustomObjects: This namespace is for custom objects and will be explained in detail later.
  • Mapping.WinForms: This namespace is for the project that generates Windows Forms.
  • Mapping.ClassObjects: This namespace is for the project that generates C# database related objects

Before getting into a discussion about generating Windows Forms and C# objects, let us recap what we covered yesterday on custom objects.

Custom Objects

  • The Database object can be considered as DataSet object,
  • The Table object can be considered as DataTable and
  • The Column object can be considered as DataColumn in .NET framework.
  • Databases, Tables and Columns objects can be considered as Collections of above objects.

For the discussion of .NET DataSet versus custom objects, please refer to the first part of the article.

Article Structure

  • Generating Windows Forms
  • Generating C# Objects

Windows Forms are the reflection of a table in a relational database. Let's see a Windows Form generated by this application:

The title of the Windows Form is taken from the name of a database, such as "USERS". Each Label control in the Windows Form reflects one field in the database table. We will cover how to parse database fields into the right Windows Forms controls, such as TextBox, DateTimePicker, or CheckBox. In addition, you will see three Button Windows controls for inserting, updating, and reporting data. These buttons will initiate the right C# objects to insert, update, and report data. Now we have an idea about what we want to accomplish, let's start with the subject of generating Windows Forms.

Generating Windows Forms

A Windows Form control, either a Textbox, Label, DateTimePicker, or CheckBox, will map to each field in a relational database. We map each database field to the right Windows Forms control based on the data type of the field. For example, if the data type is System.DateTime, we drag and drop a DateTimePicker on the Windows Form.

Let's look at the code and start with the main part of generating Windows Forms (WinForms.cs):

namespace Mapping.WinForms
{
public class WinForms
{
protected String connectionString;
protected StreamWriter streamWrite;
protected FileStream streamFile;
protected int PointX;
protected int PointY;
protected String className;
protected Databases dataBaseCollection;
public static string ONE_TAB ="\t";
public static string TWO_TAB ="\t\t";
public static string THREE_TAB ="\t\t\t";
public static string FOUR_TAB ="\t\t\t\t";
public static string FIVE_TAB ="\t\t\t\t\t";
public static string SIX_TAB ="\t\t\t\t\t\t";
public static string NINE_TAB ="\t\t\t\t\t\t\t\t\t";

public WinForms(Databases DataBaseCollection, String ConnectionString)
{
this.dataBaseCollection=DataBaseCollection;
this.connectionString=ConnectionString;
}

The WinForms class is the main class used to generate Windows Forms. The above code gives some idea of how it is initiated, and what the instance variables are.

  • connectionString: holds the connection string for database access
  • streamWrite: an instance of StreamWriter object in order to write code to files
  • streamFile: an instance of FileStream object in order to define the file access
  • PointX: holds horizontal distance (points)
  • PointY: holds vertical distance
  • className: holds Table name in a relational database, and is used to name windows forms and class files.
  • dataBaseCollection: an instance of Databases custom object. This object holds all information that is parsed from a relational database. We parsed it in the first part of the article.
  • ONE_TAB and others: used to indent code for easy reading.

When the user initiates this object, the user has to provide two arguments, DataBaseCollection, and ConnectionString. DatabaseCollection must have all the information of the relational database, otherwise the software will not be able to generate Windows Forms and C# objects.

After initiating the WinForms object, the user can use the following function to generate Windows Forms.

public void CreateWinForms(String FileLocation, String Namespace, String DataNameSpace)

This function requires three arguments:

  • FileLocation: a folder name in which the user can save Windows Forms. For example, C:\projects,
  • Namespace: When the application generates Windows Forms, it will add this value as the namespace of the Windows Forms.
  • DataNameSpace: This will be the namespace declaration of C# objects and is also used for referencing C# objects in your Windows Forms.

The following code is taken from the CreateWinForms function in WinForms.cs. When we generate Windows Forms, we use the following objects:

  • WindowsFormDesignerGeneratedCodes Object: This object generates the pre-created code in the Windows Forms, similar to the code we see in the "+ Windows Form Designer generated code" section of a Windows Form. We will look at this object in detail later.
  • CreateInstanceVariables Object: This object is responsible for generating instance variables for the Windows Forms.
  • WinFormFunctions Object: This object is responsible for generating Insert, Update, and Report functions that are linked with C# database layer objects.

WinForms.cs continues:

WindowsFormDesignerGeneratedCodes
            windesGene= new
            WindowsFormDesignerGeneratedCodes();
CreateInstanceVariables
            instVar=new CreateInstanceVariables();
WinFormFunctions
            winFunctions =new
            WinFormFunctions(this.connectionString);

for(int i=0; i<
            this.dataBaseCollection[0].Tables.Count;
            i++)
{
className=this.dataBaseCollection[0].Tables[i].TableName;

///check
            whether file location has \ at the end of filelocation
            string
///if it is not there, append the string with "\"
            character

FileLocation=FileLocation.Trim();
if(FileLocation.Substring(FileLocation.Length-1,1)!="\\")
FileLocation+="\\";

///if
            the filelocation is not exist
///create one, here
if(!Directory.Exists(FileLocation))
{
Directory.CreateDirectory(FileLocation);
}
try
{
streamFile
            = new FileStream(FileLocation+"frm"+className+".cs",
            FileMode.Create, FileAccess.Write);

streamWrite = new
            StreamWriter(streamFile);

streamWrite.WriteLine("using
            System;");
streamWrite.WriteLine("using
            System.Data;");
streamWrite.WriteLine("using
            System.Drawing;");
streamWrite.WriteLine("using
            System.Collections;");
streamWrite.WriteLine("using
            System.ComponentModel;");
streamWrite.WriteLine("using
            System.Windows.Forms;");
streamWrite.WriteLine("using
            "+DataNameSpace+";");
streamWrite.WriteLine();
streamWrite.WriteLine("namespace
            "+Namespace);
streamWrite.WriteLine("{");
streamWrite.WriteLine(ONE_TAB+"public
            class frm"+className.Replace(" ", "_")+":
            System.Windows.Forms.Form");
streamWrite.WriteLine(ONE_TAB+"{");

Let's explain the code above.

for(int i=0; i< this.dataBaseCollection[0].Tables.Count; i++)

This loop iterates all tables in the databaseCollection custom object. We need this because we want to generate one Windows Form for each database table.

FileLocation=FileLocation.Trim();
if(FileLocation.Substring(FileLocation.Length-1,1)!="\\")
FileLocation+="\\";

if(!Directory.Exists(FileLocation))
{
Directory.CreateDirectory(FileLocation);
}

The above code checks the validity of the file location. If the folder does not exist, we create one. The next step is to instantiate the FileStream and StreamWriter objects by using:

streamFile = new
            FileStream(FileLocation+"frm"+className.Replace(" ", "_")+".cs",
            FileMode.Create, FileAccess.Write);

streamWrite = new
            StreamWriter(streamFile);

We use the "frm" suffix and name of a database table to name the Windows Forms. The following code ensures that there is no space in table name. For example, the table name might be Order Details:

className.Replace(" ", "_")+".cs"

The result will be a file called frmOrder_Details.cs

The rest of the code writes necessary namespace declarations in a Windows Form. The example output of generated code might be:

using System;
using System.Data;
using System.Drawing;
using System.Collections;
using System.ComponentModel;
using System.Windows.Forms;
using Camlibel.Data; // database layer namespace(this namespace is entered when the user configures the application).

namespace Camlibel.WinForms //This is also entered by the user
{
public class frmOrder_Details: System.Windows.Forms.Form
{

After this point, it is time to generate instance variables using the CreateInstanceVariables object. The following code will call the function WriteInstanceVariables of this object.

instVar.WriteInstanceVariables(ref streamWrite,this.dataBaseCollection[0].Tables[i].Columns);

As you see, we pass StreamWriter object as reference, and Columns custom object of a particular database table. We pass StreamWriter object as reference because we want to write the code into the same file defined in the WinForms object.

public void WriteInstanceVariables(ref StreamWriter streamWrite, Columns ColumnCollection)
{
for(int i=0;i<ColumnCollection.Count;i++)
{
streamWrite.Write(WinForms.TWO_TAB+"private ");
streamWrite.Write("System.Windows.Forms.Label ");
streamWrite.WriteLine("lbl"+ColumnCollection[i].ColumnName.Replace(" ","_")+";");

if(ColumnCollection[i].Type=="System.DateTime")
{
streamWrite.Write(WinForms.TWO_TAB+"private ");
streamWrite.Write("System.Windows.Forms.DateTimePicker ");
streamWrite.WriteLine("dt"+ColumnCollection[i].ColumnName.Replace(" ","_")+";");
}
else if(ColumnCollection[i].Type=="System.Boolean")
{
streamWrite.Write(WinForms.TWO_TAB+"private ");
streamWrite.Write("System.Windows.Forms.CheckBox ");
streamWrite.WriteLine("ch"+ColumnCollection[i].ColumnName.Replace(" ","_")+";");
}
else
{
streamWrite.Write(WinForms.TWO_TAB+"private ");
streamWrite.Write("System.Windows.Forms.TextBox ");
streamWrite.WriteLine("txt"+ColumnCollection[i].ColumnName.Replace(" ","_")+";");
}
streamWrite.WriteLine();
}
streamWrite.WriteLine(WinForms.TWO_TAB+"private System.Windows.Forms.Button btnInsert;");
streamWrite.WriteLine(WinForms.TWO_TAB+"private System.Windows.Forms.Button btnUpdate;");
streamWrite.WriteLine(WinForms.TWO_TAB+"private System.Windows.Forms.Button btnReport;");
}

As you see from the code above, we generate one of three different Windows Form controls depending upon the data type of the database field.

1- If the data type is DateTime, we generate code to declare a DateTimePicker Windows Form control. Moreover, we used a "dt" suffix in order to name the control. WinForms.TWO_TAB adds two tabs space to the line.

streamWrite.Write(WinForms.TWO_TAB+"private ");
streamWrite.Write("System.Windows.Forms.DateTimePicker ");
streamWrite.WriteLine("dt"+ColumnCollection[i].ColumnName.Replace(" ","_")+";");

For example, the above code generates the output below:

private System.Windows.Forms.DateTimePicker dtOrderDate;

2- If the data type is Boolean, we generate a code to declare a CheckBox Windows Form control. The naming convention is "ch".

3- If the data type is not DateTime or Boolean, we generate a Textbox Windows Form control. The naming convention is "txt". For example:

private System.Windows.Forms.TextBox txtFirstName;
private System.Windows.Forms.CheckBox chStatus;

Moreover, every database field will have a Label Windows Form control next to them. The naming convention is "lbl". However, this application does not handle Byte[] types such as variants, or pictures. After parsing columns to instance variables, we add three Button Windows Form controls for inserting, updating and reporting purposes.

After this point, we need to write code to generate the following:

public
            Form1()
{
InitializeComponent();
}
protected override
            void Dispose( bool disposing )
{
if( disposing )
{
if
            (components != null)
{
components.Dispose();
}
}
base.Dispose( disposing
            );
}

If you look at the WinForms.cs, you will see the code to do this:

streamWrite.WriteLine(TWO_TAB+"private
            System.ComponentModel.Container components =
            null;");
streamWrite.WriteLine(TWO_TAB+"public
            frm"+className.Replace("
            ","_")+"()");
streamWrite.WriteLine(TWO_TAB+"{");
streamWrite.WriteLine(THREE_TAB+"InitializeComponent();");
streamWrite.WriteLine(TWO_TAB+"}");
streamWrite.WriteLine(TWO_TAB+"protected
            override void Dispose( bool disposing
            )");
streamWrite.WriteLine(TWO_TAB+"{");
streamWrite.WriteLine(THREE_TAB+"if(
            disposing
            )");
streamWrite.WriteLine(THREE_TAB+"{");
streamWrite.WriteLine(FOUR_TAB+"if(components
            !=
            null)");
streamWrite.WriteLine(FOUR_TAB+"{");
streamWrite.WriteLine(FIVE_TAB+"components.Dispose();");
streamWrite.WriteLine(FOUR_TAB+"}");
streamWrite.WriteLine(THREE_TAB+"}");
streamWrite.WriteLine(THREE_TAB+"base.Dispose(disposing);");
streamWrite.WriteLine(TWO_TAB+"}");

Now we are ready to generate code for initializing components. To do this, we use the following function that requires the table name (classname), StreamWriter instance by reference, and Columns custom object that consists of all necessary information.

windesGene.WriteDesignerGeneratedCodes(this.className,ref streamWrite,this.dataBaseCollection[0].Tables[i].Columns);

Let's see what happens in this function (in WindowsFormDesignerGeneratedCodes.cs):

public void WriteDesignerGeneratedCodes(String ClassName, ref StreamWriter streamWrite, Columns ColumnCollection)
{
this.className=ClassName;
StringBuilder strRow= new StringBuilder();
streamWrite.WriteLine(WinForms.TWO_TAB+"#region Windows Form Designer Generated Code");
streamWrite.WriteLine(WinForms.TWO_TAB+"private void InitializeComponent()");
streamWrite.WriteLine(WinForms.TWO_TAB+"{");

The above code generates the following output:

#region Windows Form Designer generated code

private void InitializeComponent()
{

As you see, we declare strRow as StringBuilder. I have used this variable to hold the class names (table names) in order to generate the following code output:

this.Controls.AddRange(new System.Windows.Forms.Control[] {
this.label3,
this.txtDataNameSpace,
this.btnCode,
this.txtnameSpace,
this.NameSpace,
this.txtPassword,
this.txtUserName,
this.label6,
this.label5,
this.btnWinForms,
this.txtLocation,
this.label4,
this.txtDataBase,
this.label2,
this.treeTables,
this.txtServer,
this.label1,
this.btnGetTables});

Let's continue. It is now time to generate code for initializing components on the form. To do this, we use a for loop to iterates the columns custom object:

for(int i=0;i<ColumnCollection.Count;
            i++)
{
streamWrite.Write(WinForms.THREE_TAB+"this.");
streamWrite.Write("lbl"+ColumnCollection[i].ColumnName.Replace("
            ","_")+"=");
streamWrite.WriteLine("new
            System.Windows.Forms.Label();");

if(ColumnCollection[i].Type=="System.DateTime")
{
streamWrite.Write(WinForms.THREE_TAB+"this.");
streamWrite.Write("dt"+ColumnCollection[i].ColumnName.Replace("
            ","_")+"=");
streamWrite.WriteLine("new
            System.Windows.Forms.DateTimePicker();");
}
else
            if(ColumnCollection[i].Type=="System.Boolean")
{
streamWrite.Write(WinForms.THREE_TAB+"this.");
streamWrite.Write("ch"+ColumnCollection[i].ColumnName.Replace("
            ","_")+"=");
streamWrite.WriteLine("new
            System.Windows.Forms.CheckBox();");
}
else
{
streamWrite.Write(WinForms.THREE_TAB+"this.");
streamWrite.Write("txt"+ColumnCollection[i].ColumnName.Replace("
            ","_")+"=");
streamWrite.WriteLine("new
            System.Windows.Forms.TextBox();");
}
}
streamWrite.WriteLine(WinForms.THREE_TAB+"this.btnInsert
            = new
            System.Windows.Forms.Button();");
streamWrite.WriteLine(WinForms.THREE_TAB+"this.btnUpdate
            = new
            System.Windows.Forms.Button();");
streamWrite.WriteLine(WinForms.THREE_TAB+"this.btnReport
            = new System.Windows.Forms.Button();");

The code above generates, for example, the following component-initializing code:

this.txtDataBase = new
            System.Windows.Forms.TextBox();
this.txtLocation = new
            System.Windows.Forms.TextBox();
this.txtServer = new
            System.Windows.Forms.TextBox();
this.btnWinForms = new
            System.Windows.Forms.Button();

After initializing the code, we have to put these components on the form. The properties of a component might change based on the type of the component. The following code designs Windows Forms and puts the necessary components on it.

for(int i=0;i<ColumnCollection.Count;
            i++)
{

if(Count %
            2==0)
{
PointY+=32;
PointX=8;
}
strRow.Append(WinForms.NINE_TAB+"lbl"+ColumnCollection[i].ColumnName.Replace("
            ","_")+",\r\n");

streamWrite.Write(WinForms.THREE_TAB+"this.");
streamWrite.Write("lbl"+ColumnCollection[i].ColumnName.Replace("
            ","_")+".");
streamWrite.WriteLine("Location=new
            System.Drawing.Point("+PointX+","+PointY+");");
streamWrite.Write(WinForms.THREE_TAB+"this.");
streamWrite.Write("lbl"+ColumnCollection[i].ColumnName.Replace("
            ","_")+".Text=");
streamWrite.WriteLine("\""+ColumnCollection[i].ColumnName.Replace("
            ","_")+"\";");
streamWrite.Write(WinForms.THREE_TAB+"this.");
streamWrite.Write("lbl"+ColumnCollection[i].ColumnName.Replace("
            ","_")+".Name=\"");
streamWrite.WriteLine("lbl"+ColumnCollection[i].ColumnName.Replace("
            ","_")+"\";");
streamWrite.WriteLine();
PointX+=100;
if(ColumnCollection[i].Type=="System.DateTime")
{
strRow.Append(WinForms.NINE_TAB+"dt"+ColumnCollection[i].ColumnName.Replace("
            ","_")+",\r\n");
streamWrite.Write(WinForms.THREE_TAB+"this.");
streamWrite.Write("dt"+ColumnCollection[i].ColumnName.Replace("
            ","_")+".");
streamWrite.WriteLine("Location=new
            System.Drawing.Point("+PointX+","+PointY+");");
streamWrite.Write(WinForms.THREE_TAB+"this.");
streamWrite.Write("dt"+ColumnCollection[i].ColumnName.Replace("
            ","_")+".Name=\"");
streamWrite.WriteLine("dt"+ColumnCollection[i].ColumnName.Replace("
            ","_")+"\";");
streamWrite.Write(WinForms.THREE_TAB+"this.");
streamWrite.Write("dt"+ColumnCollection[i].ColumnName.Replace("
            ","_")+".Size=");
streamWrite.WriteLine("new
            System.Drawing.Size(100,
            20);");
streamWrite.Write(WinForms.THREE_TAB+"this.");
streamWrite.Write("dt"+ColumnCollection[i].ColumnName.Replace("
            ","_")+".Format=");
streamWrite.WriteLine("System.Windows.Forms.DateTimePickerFormat.Short;");
}
else
            if(ColumnCollection[i].Type=="System.Boolean")
{
strRow.Append(WinForms.NINE_TAB+"ch"+ColumnCollection[i].ColumnName.Replace("
            ","_")+",\r\n");
streamWrite.Write(WinForms.THREE_TAB+"this.");
streamWrite.Write("ch"+ColumnCollection[i].ColumnName.Replace("
            ","_")+".");
streamWrite.WriteLine("Location=new
            System.Drawing.Point("+PointX+","+PointY+");");
streamWrite.Write(WinForms.THREE_TAB+"this.");
streamWrite.Write("ch"+ColumnCollection[i].ColumnName.Replace("
            ","_")+".Name=\"");
streamWrite.WriteLine("ch"+ColumnCollection[i].ColumnName.Replace("
            ","_")+"\";");
streamWrite.Write(WinForms.THREE_TAB+"this.");
streamWrite.Write("ch"+ColumnCollection[i].ColumnName.Replace("
            ","_")+".FlatStyle=");
streamWrite.WriteLine("System.Windows.Forms.FlatStyle.Popup;");
}
else
{
strRow.Append(WinForms.NINE_TAB+"txt"+ColumnCollection[i].ColumnName.Replace("
            ","_")+",\r\n");
streamWrite.Write(WinForms.THREE_TAB+"this.");
streamWrite.Write("txt"+ColumnCollection[i].ColumnName.Replace("
            ","_")+".");
streamWrite.WriteLine("BorderStyle =
            System.Windows.Forms.BorderStyle.FixedSingle;");
streamWrite.Write(WinForms.THREE_TAB+"this.");
streamWrite.Write("txt"+ColumnCollection[i].ColumnName.Replace("
            ","_")+".");
streamWrite.WriteLine("Location=new
            System.Drawing.Point("+PointX+","+PointY+");");
streamWrite.Write(WinForms.THREE_TAB+"this.");
streamWrite.Write("txt"+ColumnCollection[i].ColumnName.Replace("
            ","_")+".Name=\"");
streamWrite.WriteLine("txt"+ColumnCollection[i].ColumnName.Replace("
            ","_")+"\";");

if(ColumnCollection[i].Type=="System.String")
{
streamWrite.Write(WinForms.THREE_TAB+"this.");
streamWrite.Write("txt"+ColumnCollection[i].ColumnName.Replace("
            ","_")+".MaxLength=");
streamWrite.WriteLine(ColumnCollection[i].Length+";");
streamWrite.WriteLine();
}
else
{
streamWrite.WriteLine();
}
}
PointX+=150;
Count++;
}

DateTimePicker, Label, TextBox and CheckBox might have different properties. Because of that, we use If statements to separate components. The code generates, for example, the following output.

this.txtnameSpace.BorderStyle =System.Windows.Forms.BorderStyle.FixedSingle;
this.txtnameSpace.Location = new System.Drawing.Point(376, 136);
this.txtnameSpace.Name = "txtnameSpace";
this.txtnameSpace.TabIndex = 20;
this.txtnameSpace.Text = "Camlibel.WinForms";

So now, we have completed the basic Windows Form design. But we want to add functionality to insert, update, and report to the database. Because of that, we create another C# object for this task in WinFormFunctions.cs. The following code is taken from WinForms.cs:

winFunctions.WriteInsertFunction(ref streamWrite, this.dataBaseCollection[0].Tables[i].Columns,this.className);

winFunctions.WriteReportFunction(ref streamWrite, this.dataBaseCollection[0].Tables[i].Columns,this.className);

winFunctions.WriteUpdateFunction(ref streamWrite, this.dataBaseCollection[0].Tables[i].Columns,this.className);

These functions will generate insert, update, and report functions. Let's see the code behind it:

public void WriteInsertFunction(ref StreamWriter streamWrite, Columns ColumnCollection,String className)
{
String forwardValue=ParseTypesForInsert(ColumnCollection);
streamWrite.WriteLine(WinForms.TWO_TAB+"private void btnInsert_Click(object sender, System.EventArgs e)");
streamWrite.WriteLine(WinForms.TWO_TAB + "{");
streamWrite.WriteLine();
streamWrite.WriteLine(WinForms.THREE_TAB+"try");
streamWrite.WriteLine(WinForms.THREE_TAB+"{");
streamWrite.WriteLine(WinForms.FOUR_TAB +"cls"+className.Replace(" ", "_")+
" _"+className.Replace(" ", "_")+"= new "+ "cls"+className.Replace(" ", "_")+
"(@\""+this.connectionString+"\");");
streamWrite.WriteLine(WinForms.FOUR_TAB +" _"+className.Replace(" ", "_")+
".Insert("+forwardValue+");");

streamWrite.WriteLine(WinForms.THREE_TAB+"}");
streamWrite.WriteLine(WinForms.THREE_TAB+"catch(Exception ex)");
streamWrite.WriteLine(WinForms.THREE_TAB+"{");
streamWrite.WriteLine(WinForms.FOUR_TAB+"throw ex;");
streamWrite.WriteLine(WinForms.THREE_TAB+"}");
streamWrite.WriteLine(WinForms.TWO_TAB+"}");
}

The output might look like the following:

private void btnInsert_Click(object sender,
            System.EventArgs e)
{
try
{
clsFirmPermission
            _FirmPermission= new
            clsFirmPermission(@"server=(local)\vste;uid=sa;pwd=;database=supplynet;");
_FirmPermission.Insert(Int32.Parse(txtFirm_ID.Text),
            Int32.Parse(txtPermission_ID.Text),
Int32.Parse(txtSNET_ID.Text),
            txtPermission_Value.Text);
}
catch(Exception ex)
{
throw
            ex;
}
}

As can be seen, during this process, the code will instantiate the database layer object for inserting data. After that, it will call Insert function with parsed values. So, let us see how to handle the parsing. The following code will parse the value based on data type:

private String ParseTypesForInsert(Columns
            ColumnCollection)
{
int Col=0;
String
            forwardValue="";
for(int
            i=0;i<ColumnCollection.Count;i++)
{
Col++;
if(Col == 4)
{
forwardValue+="\r\n"
            +WinForms.FIVE_TAB;
Col=0;
}
if(ColumnCollection[i].AutoIncrement==false)
{
switch
            (ColumnCollection[i].Type)
{
case
            "System.String":
forwardValue+="txt"+ColumnCollection[i].ColumnName.Replace("
            ","_")+".Text, ";
break;
case
            "System.Int16":
forwardValue+="Int16.Parse(txt"+ColumnCollection[i].ColumnName.Replace("
            ","_")+".Text), ";
break;
case
            "System.Int32":
forwardValue+="Int32.Parse(txt"+ColumnCollection[i].ColumnName.Replace("
            ","_")+".Text), ";
break;
case
            "System.Int64":
forwardValue+="Int64.Parse(txt"+ColumnCollection[i].ColumnName.Replace("
            ","_")+".Text), ";
break;
case
            "System.DateTime":
forwardValue+="DateTime.Parse(dt"+ColumnCollection[i].ColumnName.Replace("
            ","_")+".Text), ";
break;
case
            "System.Decimal":
forwardValue+="Decimal.Parse(txt"+ColumnCollection[i].ColumnName.Replace("
            ","_")+".Text), ";
break;
case
            "System.Double":
forwardValue+="Double.Parse(txt"+ColumnCollection[i].ColumnName.Replace("
            ","_")+".Text), ";
break;
case
            "System.Byte[]":
forwardValue+="Byte.Parse(txt"+ColumnCollection[i].ColumnName.Replace("
            ","_")+".Text), ";
break;
case
            "System.Single":
forwardValue+="Single.Parse(txt"+ColumnCollection[i].ColumnName.Replace("
            ","_")+".Text), ";
break;
case
            "System.Boolean":
forwardValue+="ch"+ColumnCollection[i].ColumnName.Replace("
            ","_")+".Checked, ";
break;

default:
throw new
            Exception("Type is
            unknown");
}
}
}
forwardValue=forwardValue.Remove(forwardValue.Length-1,1);

if(forwardValue.TrimEnd().LastIndexOf(",")==forwardValue.TrimEnd().Length-1)
{
forwardValue=forwardValue.TrimEnd().Remove(forwardValue.TrimEnd().Length-1,1);
}
return
            forwardValue;
}

We have now completed the Windows Forms. Now we will look at the C# objects, which are the database access layer objects.

Generating C# Objects

Why do we need C# objects? C# objects will allow Windows Forms to access and manipulate Microsoft SQL database. In other words, the C# objects' task is to provide a channel between Windows Forms and Microsoft SQL Server. This part of the project consists of:

  • Class Mapping
  • Class Functions
  • Class Instance Variables
  • Class Properties

We will see how we can parse a relational database into a C# class file. It follows the same logic as parsing to Windows Forms. The ClassMapping object has a CreateCSharp function which generates a C# class file. Here is a code snip:

public void CreateCSharp(String
            FileLocation, String Namespace)
{
ClassInstanceVariables
            instVar= new ClassInstanceVariables();
ClassProperties clsPro=
            new ClassProperties();
ClassFunctions clsFunc= new
            ClassFunctions();

for(int i=0; i<
            this.dataBaseCollection[0].Tables.Count;
            i++)
{
this.className=this.dataBaseCollection[0].Tables[i].TableName;

///check
            whether file location has \ at the end of filelocation
            string
///if it is not there, append the string with "\"
            character

FileLocation=FileLocation.Trim();
if(FileLocation.Substring(FileLocation.Length-1,1)!="\\")
FileLocation+="\\";

fs
            = new FileStream(FileLocation+"cls"+this.className.Replace(" ",
            "_")+".cs", FileMode.Create, FileAccess.Write);

streamWrite =
            new StreamWriter(fs);

StringBuilder insertFunction= new
            StringBuilder("public void Insert(");
StringBuilder
            updateFunction= new StringBuilder("public void
            Update(");
StringBuilder insertStatement= new
            StringBuilder("INSERT INTO "+"["+this.className+"]
            (");
StringBuilder updateStatement= new StringBuilder("UPDATE
            "+"["+this.className+"] SET ");
StringBuilder
            insertStatementValue= new StringBuilder();
ArrayList PrimaryKeys
            = new ArrayList();
ArrayList AutoIncrement = new
            ArrayList();

streamWrite.WriteLine("using
            System;");
streamWrite.WriteLine("using
            System.Text;");
streamWrite.WriteLine("using
            System.Data;");
streamWrite.WriteLine("using
            System.Data.SqlClient;");
streamWrite.WriteLine();

streamWrite.WriteLine("namespace
            "+Namespace);
streamWrite.WriteLine("{");
streamWrite.WriteLine(ONE_TAB+"public
            class cls"+this.className.Replace(" ",
            "_"));
streamWrite.WriteLine(ONE_TAB+"{");

//
            CreateInstanceVariables();
instVar.WriteInstanceVariables(ref
            streamWrite,this.dataBaseCollection[0].Tables[i].Columns);

streamWrite.WriteLine(TWO_TAB+"public
            cls"+this.className.Replace(" ", "_")+"(String
            ConnectionString)");
streamWrite.WriteLine(TWO_TAB+"{");
streamWrite.WriteLine(THREE_TAB+"this.connString=ConnectionString;");
streamWrite.WriteLine(TWO_TAB+"}");
streamWrite.WriteLine();

clsPro.WriteProperties(refstreamWrite,this.dataBaseCollection[0].Tables[i].Columns,
ref insertFunction,ref insertStatement, ref
            insertStatementValue,
ref updateFunction,ref updateStatement, ref
            PrimaryKeys,
ref
            AutoIncrement);

if(clsPro.PrimaryKeyCount>0)
{
updateFunction.Remove(updateFunction.Length-2,2);
updateStatement.Remove(updateStatement.Length-2,2);
updateFunction.Append(")");
clsFunc.WriteUpdateFunction(ref
            streamWrite,updateFunction,updateStatement,PrimaryKeys
            );
}
insertFunction.Remove(insertFunction.Length-2,2);

insertStatement.Remove(insertStatement.Length-2,2);

insertStatementValue.Remove(insertStatementValue.Length-(insertStatementValue.Length-
insertStatementValue.ToString().LastIndexOf(",")),
insertStatementValue.Length-insertStatementValue.ToString().LastIndexOf(","));
insertFunction.Append(")");

clsFunc.WriteInsertFunction(ref
            streamWrite,insertFunction,
            insertStatement,insertStatementValue);
clsFunc.WriteReportFunction(ref
            streamWrite);
streamWrite.WriteLine(ONE_TAB+"}");
streamWrite.WriteLine("}");

//
            Close the writer and underlying file.
streamWrite.Close();
}

Let me explain the following variables:

  • UpdateFunction
  • UpdateStatement
  • InsertStatementValue
  • InsertFunction
  • insertStatement

UpdateFunction Variable

This variable holds information for generating the following output:

public void Update(System.Int32 Contract_ID, System.Int32 Supplier_Product_ID, System.Decimal Contract_Price)

UpdateStatement Variable

This variable holds information for generating the following code: for example,

Customer is the table name here:

UPDATE [Contract_Product] SET
            [Contract_ID]="+Contract_ID+",
            [Supplier_Product_ID]="+Supplier_Product_ID+",
[Contract_Price]="+Contract_Price+"";
" WHERE
            [Contract_ID]="+Contract_ID+" AND "+"
            [Supplier_Product_ID]="+Supplier_Product_ID;

We put class names into "[ ]" because some table names might contain spaces.

InsertFunction Variable

This variable holds information for generating the following code: For example:

public void Insert(System.Int32 Contract_ID, System.Int32 Supplier_Product_ID, System.Decimal Contract_Price)

insertStatement Variable

This variable holds information for generating the following code: For example:

SQLStatement="INSERT INTO [Contract_Product] (Contract_ID, Supplier_Product_ID, Contract_Price)";

InsertStatementValue Variable

This variable holds information for generating code similar to the following:

strInsert.Append(SQLStatement+" VALUES
            (");
strInsert.Append(Contract_ID+",");
strInsert.Append(Supplier_Product_ID+",");
strInsert.Append(Contract_Price+")");

When we update the database, we do so based on primary keys. There is no way to update the database based on some criteria unless there is one more primary key. Therefore the following code makes sure that the C# object will have update function if there is one or more primary key in the table:

if(clsPro.PrimaryKeyCount>0)
{
}

Otherwise, the C# object will not have any update function.

In this function, we call the following code to generate instance variables in C# object:

instVar.WriteInstanceVariables(ref streamWrite,this.dataBaseCollection[0].Tables[i].Columns);

public void WriteInstanceVariables(ref StreamWriter streamWrite, Columns ColumnCollection)
{
for(int i=0;i<ColumnCollection.Count;i++)
{
string st=ColumnCollection[i].Type;
streamWrite.Write(ClassMapping.TWO_TAB+"private ");
streamWrite.Write(st);
streamWrite.WriteLine(" _"+ColumnCollection[i].ColumnName.Replace(" ","_")+";");
}
streamWrite.WriteLine(ClassMapping.TWO_TAB+"private String connString;");
streamWrite.WriteLine();
}

The above code will generate output similar to the following:

private System.Int32 _Contract_ID;
private System.Int32 _Supplier_Product_ID;
private System.Decimal _Contract_Price;

As you see, instance variables start with underscore character "_" plus field name.

After generating instance variables, it is time to generate object properties. The following code will generate the necessary output:

clsPro.WriteProperties(ref streamWrite,this.dataBaseCollection[0].Tables[i].Columns,
ref insertFunction,ref insertStatement, ref insertStatementValue,
ref updateFunction,ref updateStatement, ref PrimaryKeys, ref AutoIncrement);

public void WriteProperties(ref StreamWriter streamWrite, Columns ColumnCollection,
ref StringBuilder insertFunction,ref StringBuilder insertStatement,
ref StringBuilder insertStatementValue, ref StringBuilder updateFunction,
ref StringBuilder updateStatement, ref ArrayList PrimaryKeys, ref ArrayList AutoIncrement )
{
int CountKey=0;
for(int i=0;i<ColumnCollection.Count;i++)
{
string st=ColumnCollection[i].Type;
streamWrite.Write(ClassMapping.TWO_TAB+"public ");
streamWrite.Write(st);
streamWrite.WriteLine(" "+ColumnCollection[i].ColumnName);
streamWrite.WriteLine(ClassMapping.TWO_TAB+"{");
streamWrite.WriteLine(ClassMapping.THREE_TAB+"get {return this._"+
ColumnCollection[i].ColumnName.Replace(" ","_")+ ";}");
streamWrite.WriteLine(ClassMapping.THREE_TAB+"set { _"+
ColumnCollection[i].ColumnName.Replace(" ", "_")+"=value; }");

streamWrite.WriteLine(ClassMapping.TWO_TAB+"}");

if (ColumnCollection[i].PrimaryKey==true)
{
CountKey++;
PrimaryKeys.Add(ColumnCollection[i].ColumnName);
}

updateFunction.Append(st +" "+ColumnCollection[i].ColumnName.Replace(" ", "_")+", ");
updateStatement.Append("["+ColumnCollection[i].ColumnName+"]="+
AppendUpdateValues(st,ColumnCollection[i].ColumnName.Replace(" ", "_")));

if(0==CountKey)
{
this.CountKeys=0;
}
else
{
this.CountKeys=CountKey;
}
if(ColumnCollection[i].AutoIncrement==true)
{
AutoIncrement.Add(ColumnCollection[i].ColumnName);
}
else
{
insertFunction.Append(st +" "+ColumnCollection[i].ColumnName.Replace(" ", "_")+", ");
insertStatement.Append(ColumnCollection[i].ColumnName.Replace(" ", "_")+", ");
insertStatementValue.Append(AppendInsertValues(st,
ColumnCollection[i].ColumnName.Replace(" ", "_")));
}
}
}

The above code will generate output similar to the following:

public System.Int32 Contract_ID
{
get
            {return this._Contract_ID;}
set { _Contract_ID=value;
            }
}
public System.Int32 Supplier_Product_ID
{
get
            {return this._Supplier_Product_ID;}
set {
            _Supplier_Product_ID=value; }
}
public System.Decimal
            Contract_Price
{
get {return this._Contract_Price;}
set {
            _Contract_Price=value; }
}

I believe the following code requires explanation:

if(ColumnCollection[i].AutoIncrement==true)
{
AutoIncrement.Add(ColumnCollection[i].ColumnName);
}
else
{
insertFunction.Append(st
            +" "+ColumnCollection[i].ColumnName.Replace(" ", "_")+",
            ");
insertStatement.Append(ColumnCollection[i].ColumnName.Replace("
            ", "_")+",
            ");
insertStatementValue.Append(AppendInsertValues(st,
ColumnCollection[i].ColumnName.Replace("
            ", "_")));

The above code ensures that if the field is autoincremented, the insert function will not have this field. If the field is not autoincremented, the insert function will have this field. We need this so that we do not attempt to insert data to an autoincremented field (doing so would return a database error).

It is time to generate the data access functions. The following code can generate the insert function:

public void WriteInsertFunction(ref
            StreamWriter streamWrite, StringBuilder strInsertFunction,
StringBuilder insertStatement,StringBuilder
            insertStatementValues)
{
streamWrite.WriteLine(ClassMapping.TWO_TAB+strInsertFunction.ToString());
streamWrite.WriteLine(ClassMapping.TWO_TAB+"{");

streamWrite.WriteLine(ClassMapping.THREE_TAB+"SqlConnection
            conn= new
            SqlConnection(this.connString);");
streamWrite.WriteLine(ClassMapping.THREE_TAB+"SqlCommand
            comm= new
            SqlCommand();");
streamWrite.WriteLine();
streamWrite.WriteLine(ClassMapping.THREE_TAB+"StringBuilder
            strInsert=new
            StringBuilder();");
streamWrite.WriteLine(ClassMapping.THREE_TAB+"String
            SQLStatement;");
streamWrite.WriteLine();
streamWrite.WriteLine(ClassMapping.THREE_TAB+"SQLStatement=\""+
            insertStatement.ToString()+")\";");
streamWrite.WriteLine(ClassMapping.THREE_TAB+"strInsert.Append(SQLStatement"+"+\"
            VALUES
            (\");");
streamWrite.WriteLine(ClassMapping.THREE_TAB+insertStatementValues.ToString()+")\");");

streamWrite.WriteLine();
streamWrite.WriteLine(ClassMapping.THREE_TAB+"try");
streamWrite.WriteLine(ClassMapping.THREE_TAB+"{");
streamWrite.WriteLine(ClassMapping.FOUR_TAB+"comm.CommandText=strInsert.ToString();");
streamWrite.WriteLine(ClassMapping.FOUR_TAB+"comm.ExecuteNonQuery();");
streamWrite.WriteLine(ClassMapping.THREE_TAB+"}");
streamWrite.WriteLine(ClassMapping.THREE_TAB+"catch(Exception
            ex)");
streamWrite.WriteLine(ClassMapping.THREE_TAB+"{");
streamWrite.WriteLine(ClassMapping.FOUR_TAB+"throw
            ex;");
streamWrite.WriteLine(ClassMapping.THREE_TAB+"}");
streamWrite.WriteLine(ClassMapping.TWO_TAB+"}");
}

The above code will generate output similar to the following:

public void Insert(System.Int32 Contract_ID,
            System.Int32 Supplier_Product_ID, System.Decimal
            Contract_Price)
{
SqlConnection conn= new
            SqlConnection(this.connString);
SqlCommand comm= new
            SqlCommand();

StringBuilder strInsert=new
            StringBuilder();
String SQLStatement;

SQLStatement="INSERT
            INTO [Contract_Product] (Contract_ID, Supplier_Product_ID,
            Contract_Price)";

strInsert.Append(SQLStatement+" VALUES
            (");

strInsert.Append(Contract_ID+",");
strInsert.Append(Supplier_Product_ID+",");
strInsert.Append(Contract_Price+")");

try
{
comm.CommandText=strInsert.ToString();
comm.ExecuteNonQuery();
}
catch(Exception
            ex)
{
throw ex;
}
}

So far, we have developed Windows Forms and C# objects with functions, properties, instance variables and so on. Now let's look at the sample application that ties everything together.

The Sample Application

I have developed a small application in Visual Studio .NET RC environment. In this solution, there are three different projects:

  • Project for custom objects
  • Project for Generating Windows Forms
  • Project for Generating C# Objects.

The solution file is in the DataWinForms folder. After starting the application, you must configure the following for your own system:

  • Server Name: the name of SQL Server or IP number
  • Database Name: the name of Database in the SQL server
  • UserName:
  • Password:
  • File Location: the folder such as c:\project, c:\temp
  • Namespace: the namespace for Windows Forms
  • Data Layer Namespace: the namespace for C# objects

After configuring the necessary information, click on "Get Tables". This will provide data access, parsing the database into our custom objects. When this is done, it will let you know about it. You can see at the left side of the screen that the database tables are already parsed into custom objects.

When you click, for example, on Order Table, you will see three fields: Order_ID, Contract_ID, and Order_Date. When you click on these fields, you will see the properties of these fields such as data type, length, PrimaryKey and so on.

After parsing database into custom objects, click on "Generate Windows Forms", this will generate Windows Forms, and clicking on "Generate C# Class Files" will generate C# objects for you. I generated Windows Forms and C# objects in to C:\temp5\ folder. The following figure show how it might look:

Now, after generating Windows Forms and C# objects, we can add these files to our Visual Studio .NET projects.

How can we do that? Let's assume that we have a new C# Windows Application project in Visual Studio .NET. To add these generated files into your project, go to Project | Add Existing Item. It might look like the following:

Select all the files, and click on Open. You will see all files in the project Solution Explorer window. Now what? We have to create menus for the Windows Forms. We do not need to do so for the C# objects because they are already connected to the Windows Forms. That's all. Have fun!

Conclusion

We have developed software which generates Windows Forms and C# objects, by using .NET objects, custom objects, and Microsoft SQL Server 2000. This application can be extended for other databases such as Oracle, MySQL, DB2 and so on.

If you'd like to discuss a broader range of C#Today topics and issues, or if you have any C# related questions you need answers to, please visit our online resource centres at : http://p2p.wrox.com/list.asp?list=csharptoday_discuss&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq and http://p2p.wrox.com/list.asp?list=c_sharp&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq
C#Today - © 2002 WroxPress Limited


Building an Address book application using Windows Forms and ADO.NET

Building an Address book application using Windows Forms and ADO.NET
http://www.csharptoday.com/content.asp?id=1636
Article Rating: 4.5
Published on August 22, 2001
Abstract
In this article, Jim Joy introduces the basic concepts of developing applications for the Windows Platform, deals with database accessing methodologues, and also takes a look at web services. All of these are talked about while constructing an address book application, written in C#.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

Introduction

The .NET technology framework introduced by Microsoft is becoming the talk of the town amongst an increasingly large community of programmers. Windows Forms, ASP.NET, ADO.NET, Web Services, etc, are becoming modern day buzzwords. The .NET framework provides the programmers with a powerful set of tools and technologies, which make development of enterprise wide, distributed applications much easier than the traditional ways.

In this article, I've attempted to introduce the developers to the basic concepts of developing applications for the Windows Platform. This article lays the groundwork for creating Windows applications using the Windows.Forms namespace provided by the .NET framework. I've also dealt with database accessing methodologies (SQL Server, in particular) using ADO.NET. Finally, I touch up a little on web services, which is where the Internet development community seems to move, and provide a few examples that illustrate how easy and effective is the development of .NET web services.

I've used C# as the language of my choice, but since the Common Language Runtime (CLR) and its associated Common Type System (CTS) of the .NET framework eliminates the much debated language barrier issues, it's up to you, the programmer, to re-write any parts of this application in the language of your choice (VB.NET, JScript or even COBOL).

The address book application helps you to organize details of your contacts in an orderly way. An SQL Server 7.0 database is used to house the contact details.

User Interface

The address book application owes a heavy debt to the System.Windows.Forms namespace for the user interface design. The Windows.Forms namespace (formerly the WinForms namespace in Beta 1.0) houses a large number of classes that define all the Windows user interface widgets like Forms, Buttons, Labels, TextBoxes, ListViews, TreeViews, Menus, etc.

Top of Form

The address book application has four Forms, all of which are derived from the Form class found in the System.Windows.Forms namespace. These four forms use a wide variety of widgets found in the System.Windows.Forms namespace.

Splash.cs - This form serves as the splash screen (startup screen) for the address book application.

AddressList.cs - This form gives the user, the list of addresses stored in his / her address book, and provides options to add, edit, delete and search addresses.

AddressEntry.cs - The user adds / edits address entries into the address book using this form.

SearchAddress.cs - This form provides the user with an option to search addresses using a nickname.

Now, let's explore the construction of these forms and the tasks they perform in detail.

Splash.cs - The Startup Screen

The splash.cs defines the mySplash form class that serves as the "splash screen" for the address book application and has two LinkLabels for navigation purposes. LinkLabels are defined in the System.Windows.Forms namespace and provides the user with a HTML hyperlink like look and feel.

The first step is to create the form. I've derived the mySplash class from the System.Windows.Forms.Form class as follows.

using System.Windows.Forms
..
Public class mySplash : Form //deriving mySplash from the Systems.Windows.Forms.Form class
{
...
public mySplash()
{

}
}

The mySplash() constructor defined in the mySplash class will be triggered when an object of the same class is created. Setting the different properties and methods that the Form class exposes like the StartPosition property, the FormBorderStyle property, etc, can change the appearance of the form.

this.FormBorderStyle = FormBorderStyle.FixedDialog;
this.StartPosition = FormStartPosition.CenterScreen;

( FormBorderStyle and FormStartPosition are enumerations defined in the System.Windows.Forms namespace.)

Within the mySplash class, there are 2 LinkLabel objects (derived from System.Windows.Forms.LinkLabel class), which are instantiated as follows:

LinkLabel myStart;
LinkLabel
            myExit;
myStart = new LinkLabel();
myExit = new
            LinkLabel();

The LinkLabel myStart displays " Continue ", and clicking this will take the user to the next form of the application. To accomplish this, a method should be attached to the "Click" event of the LinkLabel so that when a user clicks on the LinkLabel, this method is executed. In order for an object to respond to an event, you must provide an event-handling method, known as an event handler, that executes the desired functionality in response to the event. You must register the event handler with the event source.

Handling Events

Delegates manage event handling in Windows Forms, and they are classes used to build event-handling mechanisms. They are like function pointers (those of you who are C++ gurus will know!), but are object-oriented, enforce type-safety, and are secure. A delegate points to an object and has references to one or more methods within the object. The event model used by Windows Forms uses delegates to bind events to the methods used to handle them. The delegate allows other classes to register for event notification by specifying a handler method. When the event occurs, the delegate calls the bound method. This bound method is called as an event handler, and when the event is raised, the code within the event handler is executed. Each event handler provides two parameters that allow you to handle the event.

Event handlers for the LinkLabel controls are defined as follows:

myStart.Click += new
            EventHandler(onContinue);
myExit.Click += new
            EventHandler(onExit);

The onContinue and onExit methods, which are the event handlers, are defined as follows:

protected void onContinue(Object mySender , EventArgs myArgs)
{
...
}

protected void onExit(Object mySender , EventArgs myArgs)
{
...
}

Clicking on the myStart LinkLabel fires up the onContinue method, which creates an instance of the AddressList class, which will be described in the next section.

Now that we have defined the user interface for the splash screen by defining the form class and it's associated controls, let's look at how we create an instance of the form and get it up and running. The mySplash class contains a method named Main , which serves as the entry point of our application, where the program control starts and ends. The Main() method calls the static Run() method of the Application class, which is used to start and stop applications. The Run() method is called to start an application message loop, and it also creates an instance of the mySplash class and makes it visible on the screen.

Public static void
            Main()
{
Application.Run(new mySplash());
}

AddressList.cs - The Address List Screen

This form is used to display the list of addresses managed by the address book application. The AddressList class, which is derived from the System.Windows.Forms.Form class, defines the form and it's associated controls. The list of addresses is displayed using a ListView control. This form also provides the users with buttons for adding, editing, deleting and searching addresses.

Now, let's look at the way in which the buttons and column headers are created and used in this form. The simplest way to create and use a control on a form is to simply instantiate the class and set it's different properties to suite your choice. But, if there are a number of controls that should have similar properties (for example, if it's a general practice to have the same size and fonts for all buttons in a form), doing this way can lead to a lot of unnecessary code.

The best way out is to use inheritance, which involves deriving your own class, setting the common properties, and then instantiating the class. That's exactly what is done here. For example:

Class AddressButton:Button
{
public
            AddressButton(String myText)
{
this.Text =
            myText;
}

public void SetParams(Size mySize , Point
            myPoint , int myTabIndex)
{
this.Size =
            mySize;
this.Location = myPoint;
this.TabIndex =
            myTabIndex;
}
}

Here, the AddressButton class is derived from the System.Windows.Forms.Button class and has a constructor, which sets the Text property of the button, and a method named SetParams that sets the Size , Location and TabIndex properties of the button. Now creating the button is as simple as instantiating the AddressButton .

private AddressButton
            btnAddButton;
btnAddButton = new
            AddressButton("Add");
btnAddButton.SetParams(new Size(88,32) ,
            new Point(320,16) , 1);
btnAddButton.Click += new
            EventHandler(onAdd);
this.Controls.Add(btnAddButton);

The above code creates the "Add" button and adds it to the Controls collection of the AddressList form so that it becomes a part of the form. A similar approach has been followed for creating the column headers for the List View too. The AddressHeader class derived from the ColumnHeader class has a constructor that sets the Text , Width and TextAlign properties of the column header.

The ListView class defined in the Windows.Forms namespace provides a number of properties for controlling the appearance of the ListView. A few of them have been used in our application like FullRowSelect , GridLines , HideSelection , Sorting , View , AllowColumnReorder , etc. The column headers are attached to the ListView by using the AddRange method of the Columns collection of the ListView class. There are other ways also to add column headers. For example, you can use the Add method of the ColumnHeaders collection to add column headers one at a time. The AddRange method is useful in this scenario as it allows adding multiple column headers at once. Also, please note how event handlers are attached to the different controls as described in the previous section.

Data Access

Next comes another interesting part of the .NET framework, accessing databases. The .NET framework comes with a new flavor of the "already tasty" ADO, called ADO.NET. A description of the features of ADO.NET is far beyond the scope of this article. ADO.NET provides data access in two flavors, which are the System.Data.SqlClient namespace which is specific to MS SQL Server, and the System.Data.OleDb namespace which is the OLE DB .NET managed data provider for all the OLE DB providers that you may have installed. The SQL Server .NET Data Provider uses its own protocol to communicate with SQL Server. It is lightweight, and the performance is commendable as it accesses an SQL Server data source directly without adding an OLE DB layer. Our address book application uses an SQL Server 7.0 database, so it was natural that the SqlClient library was chosen.

The LoadEntries method is responsible for reading data from the database and loading the ListView with this data. For establishing a connection to the SQL Server database, an object of the SqlConnection class named myConnection is created. The default constructor of the SqlConnection class takes one argument - the connection string. Once the object is created, we have to explicitly open the connection by calling the Open() method of the myConnection object.

Before diving into the details of data access, let's take a brief look at the error/exception handling that C# provides. C# has built-in support for handling exceptions which may occur during the execution of your program. These exceptions are handled by code that is outside the normal control flow. The try , throw , catch , and finally keywords are used to implement exception handling.

SqlConnection myConnection = new
            SqlConnection(myConnString);
String myQuery = "SELECT * FROM
            AddressBook ORDER BY
            Nickname";
try
{
myConnection.Open();
...
}
Catch(SqlException
            myEx)
{
MessageBox.Show(myExcept.Message + " Could not read
            data" , "Address Book 1.0" , MessageBoxButtons.OK ,
MessageBoxIcon.Error);
}
finally
{
myConnection.Close();
}

Note how a connection is established to the SQL server. The connection string is available in the string myConnString , and this string is passed as the argument to the SqlConnection constructor. Next, a connection to the database is established inside a try block so that if the connection fails, the SqlException raised can be caught in the catch block. Finally, the " finally " block is executed which closes the connection object. The finally block is used to perform some clean-up jobs, such as closing database connections, etc, regardless of whether an exception is raised or not. Code in the finally block is always executed, so you can safely place clean-up code here so that the memory used by your code is neat, regardless of whether an exception occurs or not!

Apart from system generated errors like the one that we saw above, your code can also raise exceptions when an exceptional condition is encountered. This is done by using the throw statement.

Once a connection to the database is established, the next step is to read the data that we are interested in. Obviously, executing an SQL statement against the table(s) of our choice does this. But how do we do this?

The SqlCommand object is used to represent an SQL statement or a stored procedure that needs to be executed against a selected SQL server database. As with the SqlConnection object, the SqlCommand object is also a core component of the SQL .NET data provider. The SqlCommand object enables your code to execute database commands to return results, modify existing data, execute stored procedures, and send or retrieve parameters.

SqlCommand myCommand = new
            SqlCommand(myQuery , myConnection);
SqlDataReader myReader =
            myCommand.ExecuteReader();

while(myReader.Read())
{
ListViewItem
            myItem = new ListViewItem(myReader.GetString(1));//Fetching the
            NickName
myItem.ForeColor =
            Color.Brown;
myItem.SubItems.Add(myReader.GetString(2));//Fetching
            the
            FirstName
...
lvwAddressList.Items.Add(myItem);
}
myReader.Close();
myCommand.Dispose();

The SqlCommand constructor takes in the SQL statement and the active SqlConnection as the arguments. The results of the SQL statement executed against the database by the SqlCommand object is retrieved into an SqlDataReader object. The SqlDataReader provides a high-performance, forward-only, read-only data stream from the data source (in this case, our SQL Server database). The ExecuteReader() method of the SqlCommand object returns forward-only, read-only rows from the database into the myReader SqlDataReader object. Individual columns of data are obtained from the SqlDataReader as strings by using the GetString() method by passing the zero-based column ordinal. This data is filled into the ListView object.

Clicking on the "Add" button fires up the onAdd event handler and enables the user to add an entry into the address book. This will show up the AddressEntry form to the user and once the user fills in all the details and clicks the "Save" button, this data is written into the database. For writing the data, a connection is again opened to the database and an "INSERT" SQL statement is passed to the database by using an SqlCommand object. The ExecuteNonQuery() method of the SqlCommand class is used to execute Transact-SQL commands such as UPDATE, INSERT, DELETE and SET statements.

SqlConnection myConnection = new
            SqlConnection(myConnString);
String myQuery = "INSERT INTO
            AddressBook ...";
SqlCommand myCommand = new SqlCommand(myQuery ,
            myConnection);
try
{
myConnection.Open();
myCommand.ExecuteNonQuery();
ListViewItem
            newItem = new
            ListViewItem(myAddressEntry.NickName);
}
catch(SqlException
            myExcept)
{
MessageBox.Show(myExcept.Message + " Error adding
            data" , "Address Book 1.0" , MessageBoxButtons.OK ,
MessageBoxIcon.Error);
}
finally
{
myCommand.Dispose();
myConnection.Close();
}

Similarly, in the case of editing and deleting entries from the address book, the SqlCommand object is used to update the data in the database with its ExecuteNonQuery() method. In the case of edit, this is done by executing an UPDATE statement and in the case of a delete, a DELETE statement is used.

Now, let's look at the AddressEntry form, which provides the user interface for adding and editing entries in the address book.

AddressEntry.cs - Adding and editing addresses

This form is used to provide the users with an interface to add and edit addresses in the address book. The AddressEntry form defines the form and its associated controls. Note that as in the AddressList form, controls with a similar nature are defined using inheritance. The EntryButton , EntryLabel , and the EntryText classes are inherited from the Button , Label , and TextBox classes respectively.

An important highlight in this form is the use of properties. Properties are named members of classes, structures, and interfaces. They provide a flexible mechanism to read, write, or compute the values of private fields through accessors. The accessor of a property contains the statements for reading/computing the property value, or writing the property value. The get accessor is used to read the value of a property, and the set accessor is used to write the value of a property. The AddressEntry class has five properties, which are NickName , FirstName , LastName , Email and PhoneNumber . These properties are read and assigned using their respective get and set accessors.

public String NickName
{
get
{
return txtNickName.Text;
}
set
{
txtNickName.Text = value;
}
}

These properties thus will contain the value of the designated Textboxes. In the AddressEntry class, these properties are used for adding and editing address entries.

lvwAddressList.FocusedItem.SubItems.Add(myAddressEntry.FirstName);

Also, see how validations are performed in this form. The AddressEntry class has two buttons btnSaveButton and btnCancelButton . The DialogReuslt property of the btnCanelButton is set to DialogResult.OK , and also the AcceptButton property of the AddressEntry class is set to the btnSaveButton button. This will cause the form to close when the user clicks on the "Save" button and thus the code in the remaining section of the onAdd and onEdit event handler will be executed. However, if the user leaves the nickname field blank and clocks on the " Save " button for example, a warning message is displayed to the user, and then the DialogResult property is set to DialogResult.None so that the form does not close until the user fills up all the required fields.

protected void onSave(Object mySender , EventArgs myArgs)
{
if(txtNickName.Text.Trim() == "")
{
MessageBox.Show("Please enter the nickname", "AddressBook 1.0", MessageBoxButtons.OK ,
MessageBoxIcon.Error);
txtNickName.Focus();
this.DialogResult = DialogResult.None;
}
...
}

Searching Addresses - The Web Service Way!

Now, we come to final section of the address book application, a simple search utility. This search utility is invoked by clicking the "Search" button in the AddressList form. The SearchAddress form is then displayed to the user. The search utility is implemented using a web service.

A Web Service provides a particular element of functionality, such as business logic, and is accessible to systems on different operating platforms by using XML and HTTP. A Web Service can be used either internally by a single application, or exposed externally over the Internet for use by any number of applications. Because it is accessible through a standard interface, a Web Service allows diverse systems to work together as a single entity.

Web services are built using ASP.NET. A web service, in simple terms, consists of one or methods designated as WebMethods, which implement the processing logic intended to be performed by the web service. The description of a web service (about the various methods of the web service and how to use them, etc), called the service description, is expressed as an XML document written in an XML grammar called WSDL (Web Service Description Language). The WSDL for a web service defines the format of messages the web service understands. The service description is nothing but an agreement that defines the behavior of a web service and informs a client how to use the service.

We have one web service in our address book application and it is defined in the AddressUtils.asmx file. Our web service is named as AddressUtilitiesService and provides the following functionalities.

  • Returns a DataSet populated with the details of an address entry after accepting a nickname.
  • Returns a DataSet populated with the nicknames of all the address entries in the database.

The AddressUtilitiesService has two web methods, which are GetEntryDetails() and GetDetails() .

[WebService(Description = "Methods for accessing Address Book database entries"
,Namespace = "http://webservices.jim.com" , Name = "AddressUtilitiesService")]

public class AddressUtils : WebService
{
[WebMethod(Description = "Gets all the details for the specified nickname from the database and returns a DataSet")]
//returns a dataset with the details of the passed nickname
public DataSet GetEntryDetails(String tmpnickname)
{
...
}

[WebMethod(Description = "Gets all the nicknames from the database and returns a DataSet")]
//returns a dataset with all the nicknames
public DataSet Getnicknames()
{
.
}
}

All web services are derived from the System.Web.Services.WebService class. The methods that are to be exposed by the web service are declared as " WebMethod "s. You can give a name to your web service by using the Name attribute of the web service declaration. Please note that any method in a web service can be exposed as a web method by adding the WebMethod directive to it.

Once the web service is written, we should generate a proxy class that represents the web service so that our address book application can make use of the service. The proxy class acts as an agent / representative of the web service and forwards all calls made to the proxy class to the correct web service. With a proxy class, a client can call a method of the proxy class, which in turn does the necessary tasks to pass the call over the wire to call the desired method defined in the web service.

The WSDL utility is used to generate proxy classes for web services. For creating a proxy class for our web service named " AddressUtilitiesService ", open a command window and type in the following

WSDL <url-of-AddressUtils.asmx-file>

Please note that the AddressUtils.asmx is an ASP.Net component and should be treated as a web application. This means that you have to create a virtual directory for the AddressUtils.asmx file location and map it. For example, if you have created a virtual directory named AddressBook and the AddressUtils.asmx file is in this directory, then to generate the proxy use the following command at the command prompt:

WSDL http://localhost/AddressBook/AddressUtils.asmx

This will generate the proxy class named AddressUtilitiesService.cs (the default language is C#, you can change it by configuring the various options give by the WSDL tool). Note that the class file generated has the name of the web service and NOT the name of the .asmx file.

Now that we have the proxy class for the web service, using it is pretty straightforward, it's just like using any class. Instantiate it and call the methods at will! Can't be simpler than this, right?

AddressUtilitiesService myClient = new
            AddressUtilitiesService();
//returns a data set containing the
            details for the selected nickname
DataSet myDataSet =
            myClient.GetEntryDetails(cboNickNames.SelectedItem.ToString());

The above code will return the details of the address entry identified by the nickname passed to the web method GetEntryDetails() .

The SearchAddress.cs defines the form and it's associated controls for implementing the user interface for searching addresses. The SearchAddress constructor uses the GetNickNames() web method and fills up the cboNickNames combo box with the results.

DataSet myDataSet =
            myClient.GetNickNames();
foreach(DataRow myRow in
            myDataSet.Tables[0].Rows)
{
foreach(DataColumn myColumn in
            myDataSet.Tables[0].Columns)
{
cboNickNames.Items.Add(myRow[myColumn]);
}
}

The DataSet is an in-memory representation of data that follows a relational data model, regardless of the source. The DataSet represents a complete set of data including related tables, constraints, and relationships among the tables. The methods and objects in a DataSet are consistent with that in the RDBMS model. The DataSet consists of a collection of DataTable objects that represent the tables in the DataSet . The DataRelation object of the DataSet represents the relationship between tables. The DataTable consists of DataRows and DataColumns . A DataRow represents a single row of data and a DataColumn represents a single field of data.

Clicking the "Search" button after selecting a nickname in the combo box will fire up the onSearch event handler, which in turn invokes the GetEntryDetails() web method of our web service. This method, as mentioned earlier, will return the details of the address entry as a DataSet .

Running the Address Book Application

I used the beta 2 version of the .NET framework SDK for developing this address book and not Visual Studio.NET beta 2.0. The reason I chose not to use VS.NET was that I wanted to write the code for creating the user interface myself, and not by the visual tools provided by VS.NET. In that way, I was able to learn more about the Windows Forms library and it's constituent classes.

For running the address book, you need to have a copy of the .NET framework SDK beta 2.0 installed in your system. Kindly note that beta 1.0 or the PDC version will not help. Once you have downloaded the five files ( Splash.cs , AddressList.cs , AddressEntry.cs , SearchAddress.cs and AddressUtils.asmx ), the first step is to create a virtual directory for our web service and host the AddressUtils.asmx file there. Then, using the WSDL utility, generate a proxy class for the web service. If you run the WSDL utility from the directory where you have your four source files ( Splash.cs , AddressList.cs , AddressEntry.cs and SearchAddress.cs ) files, the proxy class will also be created in the same directory.

C:\your-directory> WSDL <url-of-AddressUtils.asmx>

Once the proxy is created (you'll see a class file named AddressUtilitiesService.cs ), compile the application using the command line option of the C# compiler:

csc /out:AddressBook.exe Splash.cs AddressList.cs AddressEntry.cs SearchAddress.cs AddressUtilitiesService.cs

This generates the executable file AddressBook.exe . The scripts for creating the database are also included in the zip file. Note that I've used the name of the database as AddressBook , so if you wish to change this, make the appropriate changes in the connection strings used across the application.

As far the database is concerned, the creation is pretty simple. Create an SQL server database named " AddressBook " and then run the AddressBook.sql file (which you can download along with the code) against the newly created database using the SQL Server Query analyzer.

Conclusion

I've tried my best to give an introduction about the various technologies offered by the .NET framework. As you see, Windows Forms is the new API for developing applications for the Windows platform, based on .NET. Windows Forms Framework provides the developer with a set of fully object-oriented classes that enable you to write rich Windows applications. Additionally, Windows Forms can act as the local user interface in an n-tier distributed solution (an example is demonstrated in our address book, in the usage of a web service).

ADO.Net improves data access to a large extent by providing a number of evolutionary improvements over ADO including interoperability, maintainability, performance etc.

Web services are taking the Internet programming world into a new era of distributed application development. Web services deliver interoperability on a new level never seen before, and it's only a matter of time before they will become the fundamental technology that will bind diverse applications across the globe.

Further reading

April, May, June, July, August issues of the MSDN magazine

C#Today - © 2002 WroxPress Limited


.NET Data Access Primer Part 4

.NET Data Access Primer Part 4
http://www.csharptoday.com/content.asp?id=1550
Article Rating: 4
Published on August 16, 2001
Abstract
In previous articles in this series, Rey Nunez has covered basic ADO.NET functionality and introduced us to DataLists and DataSets. In this article, he covers the key ADO.NET and SQL concepts, components and methods to enable users to search for information in a data store.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

This article is the fourth of a series on how to retrieve and update information stored in databases. In this session we cover the key ADO.NET concepts, components and methods to enable users to search for information in a data store.

Searching for information

The simplest way to find data is to provide a full table-view where the user can browse thru the entire set of records, and simply use the client's Find dialog box to get to the item in question. Whilst that may suffice for small data sets contained within the page, it is not efficient in real-world applications. In this article we look at the different ways of letting users dynamically search for information that is stored externally, whether to look for a particular value, one record, or a group of records.

Basically, we provide a search form wherein users can type a keyword to find, and in the form submit handler select the records that match the specified criteria, then test and display the results of the search.

There are mainly two ways to filter data. One is to filter data at the database command level, by using a WHERE clause on your query. The other is to filter data after it is in the DataSet, by using an ADO.NET filter expression. Both methods can be used to do either:

  • an "exact seek" to isolate a specific record
  • or a "broad search" to retrieve a set of records that meet criteria the user specifies.

In a while, we shall show interactive samples using both methods.

Comparison Operators

The result of any search method depends on how the search comparison is done, or how the match to look for is evaluated, based on the following operators:

Operator | Comparison
= | Equal to
LIKE | SQL LIKE
!=, <> | Not equal to
> | More than
>= | More than or equal to
< | Less than
<= | Less than or equal to

NOTE: NOT ALL data source providers support ALL the comparative operators. Always test to ensure whether a specific provider supports the comparator.

Basically, equal (=) is best used for a "strict" search, such as when you need the user to provide an exact search string to match. To succeed, the input string must match an exact value in the lookup field. This type of search is mainly employed in user authentication routines, where both the user's name and password are required to exactly match their database entries.

The rest of the operators are better suited to retrieving groups of records that match a given criteria. The LIKE operator, in particular, is best used for "power search" functionality, where we need to look up some value ANYWHERE within the search field, like to find, for example, any product with the words "cool" or "rock" or whatever in the product's description.

This is made possible by using the LIKE operator with "wild cards" - special characters that the string value may contain - to specify one or more occurrences of ANY character. SQL supports the percent sign (%), while ADO.NET filter expressions support both the asterisk (*) as well as the percent sign (%)for wild cards. The use of wild cards is covered later.

Searching for a Single Record at the Database Command Level

Here we look at a sample of using SQL to query a "Users" table and return results for a single record. In this exercise, enter me@me.com for the user email and ado.net for the password.

Show me | View Source

Notice the inclusion of a RequiredFieldValidator control to test if the input fields are left blank, and a RegularExpressionValidator to check the format of the email address in the user field. If the page is displayed in a browser that is ECMAScript 1.2 compliant (supports dynamic HTML, such as IE4+, NN6+), the validation controls can perform client-side validation to provide immediate feedback to the user. The validation check is still performed in server code if the browser does not support DHMTL or if client-side scripting is disabled. No code changes are required to enable either scenario; the logic required to handle the checks on both basic and DHTML-compatible browsers is built into the controls themselves.

When the form is sent for processing, we first check the page's IsValid property. Recall that this property is an aggregate of the IsValid properties of all the validators on the page - if any validation control has detected a failure, the page's IsValid property is set to false, and the page is simply redisplayed. Only if all controls on the page are valid will we go ahead.

Example using System.Data.OleDb

In the OleDb sample above, as well as in some of the previous samples shown in this series, the page logic is encapsulated entirely within its Page_Load event handler. The Page_Load event occurs each time the page is accessed by a client browser.

The code in the Page_Load handler is wrapped in a conditional code block that only executes if Page.IsPostBack. The IsPostBack property provides an easy way to determine whether the page is being accessed the first time, or whether it is being accessed as a result of a user clicking on a control that causes the page to "post back" to the server.

The user input is obtained using the Forms collection of the page's Request object. The Forms collection contains the name/value pair of each control contained within a <form> element posted from a client during an HTTP request. We pass these values as arguments to our Select query.

protected void Page_Load(Object Src,
            EventArgs E) {
if (IsPostBack) {
if (Page.IsValid) {
String
            myQuery = "select UserID from users where UserEmail='" +
            Request.Form["UserEmail"] +
"' AND Password='" +
            Request.Form["Password"] + "'";
. . .

The query selects the UserID whose values in the lookup fields (UserEmail and Password) equally match the given strings. Next, we need some way to contain the result of the query. Here we create an instance of a DataReader.

         OleDbConnection
            myConn = new OleDbConnection("Provider=Microsoft.Jet.OleDb.4.0;Data
            Source=" +
Server.MapPath("/devguide/shared/data/guests.mdb"));
OleDbCommand
            myCommand = new OleDbCommand(myQuery,
            myConn);
myConn.Open();
OleDbDataReader myReader =
            myCommand.ExecuteReader();

Then we set up a condition to check if the DataReader indeed contains the selected row upon opening. If not, it means the query returned empty, hence, the authentication has failed.

         if
            (myReader.Read()) {
// do your thing when okay here
}
else
            {
// do your thing when reader is empty here
}

Whilst using if (myReader.Read()) may seem "unconventional", in the sense that it's use is "undocumented" in the .NET SDK, it works as a conditional IF ONLY to check if a reader contains at least one row of data.

The .NET documentation states that the Read method "advances the data reader to the next record. The default positioning is prior to the first record, therefore you must call Read before accessing any data."

But recall that a DataReader can only be built via a command object's ExecuteReader method, which relies on a query that may, as in this case, return no records. For whatever reason, the .NET SDK does not include a single property or method to simply indicate whether a DataReader contains any record at all. In fact, if you attempt to bind any of the list controls (DataGrid, DataList, Repeater) to an empty reader, you get an exception.

On the other hand, executing Read returns a Boolean value: true if another record can be read; otherwise, false. Hence, using if (myReader.Read())...else just after a DataReader is returned handles both situations logically: check if a record can be read, and at the same time, if so, move the reader to the first record (to access any values that may be needed).

Notice in the above sample that entries other than me@me.com for user email and ado.net for password return an empty reader. This method is highly accurate, though is case-insensitive. You can, however, define a custom function to do case-sensitive comparison (for MS Access and other OleDb providers), but that is beyond the scope of this exercise. For SQL Server 7 and later, case-sensitivity for search and sort operations is set at the software installation level. In contrast, case-sensitive searching can easily be enabled at the DataSet level, as we shall later see.

Example using System.Data.SqlClient

If user credentials are stored in SQL Server, an alternative is to use an internally stored procedure to verify the email and password values against those stored in the database. This offers a slight performance advantage in that there is no overhead required to return the row. Parameters are used instead.

The following illustrates the typical authentication step using a stored procedure, assuming a Users table with the minimum fields UserID, UserEmail, and Password. The procedure returns the UserID if successful, and 0 (zero) if not.

Create Procedure UserLogin (
@UserEmail nvarchar(50),
@Password nvarchar(20),
@UserID int OUTPUT
)
As
SELECT @UserID=UserID FROM Users
WHERE UserEmail=@UserEmail AND Password=@Password
IF @@Rowcount < 1
SELECT @UserID=0

The code to implement the call to the procedure is defined as follows:

   if (Page.IsValid)
            {
SqlConnection myConn = new SqlConnection("your connection
            string here");
SqlCommand myCmd = new SqlCommand("UserLogin",
            myConn);

// declare the Command as a stored
            procedure
myCmd.CommandType =
            CommandType.StoredProcedure;

// define the parameters that
            are to be passed to the
            procedure
myCmd.Parameters.Add("@UserEmail",
            SqlDataType.NVarChar, 50).Value =
UserEmail.Value;
myCmd.Parameters.Add("@Password",
            SqlDataType.NVarChar, 50).Value =
Password.Value;
myCmd.Parameters.Add("@UserID",
            SqlDataType.Int, 4).Direction =
ParameterDirection.Output;

try
            {
myConn.Open();
myCmd.ExecuteNonQuery();
}
catch
            (Exception e){
// if an error occurs, pass the exception
            up
throw e;
}
finally {
if (myConn.State ==
            ConnectionState.Open)
myConn.Close();
}

if (@UserID
            > 0)
// ... do your thing when okay here
else
//
            ... do your thing when @UserID is zero here
}

For more information, see Using Stored Procedures with a Command and Input and Output Parameters, and Return Values.

Searching for a Single Record at the DataSet Level

The following example is similar to the above, but does the record selection at the DataSet level. Try using different case settings for the user email and password and see the results.

Show me | View Source

The DataTable object includes a Select method that returns an array of DataRow objects that match specified criteria. To construct, you declare an array variable of type DataRow[] to contain the results and, at the same time, invoke the method.

DataRow[] foundRows = myTable. Select ("UserEmail='" +
Request.Form["UserEmail"] +
"' AND Password='" +
Request.Form["Password"] + "'");

The Select method enables users to retrieve rows based on a filter expression, and optionally, a sort order and the DataViewRowState (ex. only added, modified or deleted rows). In the sample, we use only a filter expression that specifies the row(s) to return based on DataColumn values.

Filter expressions in ADO.NET are essentially a subset of ANSI-SQL, using the same language rules as an SQL Select query. Notice that the above filter uses the same expression as the WHERE clause used in the earlier example. For details of valid expressions you can use in ADO.NET, please refer to the DataColumn.Expression Property.

Then to test if the returned array contains any data, you simply check the array's Length property:

         if
            (foundRows.Length > 0) {
// ... do your thing when okay
            here
}
else {
// do your thing when array is empty
            here
}

To specify whether string comparisons within DataTable objects are case-sensitive, you simply set the DataSet's CaseSensitive property to true.

DataSet myDataSet = new DataSet();
myAdapter.Fill(myDataSet, "Users");
myDataSet.CaseSensitive = true;

Whilst in the above samples a message is merely shown to confirm the query was successful, the typical logic redirects the user to another page when so.

if (myReader.Read())
Response.Redirect("http://www.asptoday.com");

One other way to search for a single record in a DataSet is via the Find method as described in .NET Data Access Primer: Part 3. Recall that Find works in conjunction with a primary key, so in that case, the UserEmail and Password columns must have been set to work as such at the database design level.

Of course, user authentication entails a bit more than what has been discussed, especially when users need to perform certain restricted tasks across pages of an application. But the gist remains, commonly using the returned UserID as a session-wide token. Typical scenarios involve several schemes, this author's preference being ASP.NET encrypted cookie authentication, particularly for shopping cart apps, which we shall cover separately.

Searching for Multiple Records at the Database Command Level

Now let's see how queries and filters can be used in a "power search" to retrieve a matching set of records.

Wild Cards

"Wild card " characters act as placeholders for other characters when you are specifying a value you want to find. You can use wild cards in a query or filter expression to search for specific records in a table where a given field contains certain characters, and you:

  • know only part of the value.
  • want to find values that start or end with a specific letter or match a certain pattern.

In general, SQL uses the percent character (%) as a wild card character. In addition, ADO.NET filter expressions can use the asterisk (*).

To search for a match ANYWHERE within the lookup field, you can include a wild card at the beginning and end of the search string. For example:

String myQuery = "SELECT Description FROM Products WHERE Description LIKE '%dura%'"

This searches for all entries in the Description field with the string "dura" in them, and returns the records with the value found anywhere in the field.

To look for a match from the start of the lookup field, or to return all entries with the search string as the FIRST letters of the field's value, include a wild card only at the end of the search string.

String myQuery = "SELECT Description FROM Products WHERE Description LIKE 'dura%'"

To return all entries with the search string as the LAST letters of the field's value, you format the query as follows:

String myQuery = "SELECT Description FROM Products WHERE Description LIKE '%dura'"

Now let's see that in action, using any given string from a search form. Because here we are providing a "universal" search, a little ploy I do in these cases is to include the wild card at both ends of the search string when defining the query.

String myQuery = "select Type, Brand, Model, Description,
Price from Products
where Description like '%" +
Request.Form["strSearch"] + "%'
order by Brand, Price";

This way, I do not have to rely on the user having to set the format of the search string. Of course, this is not a fail-safe method, particularly in cases where the user is looking for a value specifically at the start or end of the field. Yet, it does the job remarkably and can search for any given string anywhere within any given field. To see is to believe, so to test, type a string or any portion thereof that describes the product you want to find (ex. food, comf, tool, power, etc.).

Show me | View Source

When the form is sent, we create an instance of a DataReader to contain the results of the query, and bind this to a DataGrid control.

   if (IsPostBack)
            {
String myQuery = "select Type, Brand, Model, Description,
Price from Products
where Description like '%" +
Request.Form["strSearch"] + "%'
order by Brand,
            Price";
OleDbConnection myConn = new
            OleDbConnection("Provider=Microsoft.Jet.OleDb.4.0;Data Source=" +
Server.MapPath("/devguide/shared/data/gear.mdb"));
OleDbCommand
            myCommand = new OleDbCommand(myQuery,
            myConn);
myConn.Open();
DataReader myReader =
            myCommand.ExecuteReader();

Now we have to provide some way to check if the query indeed returns a result set, and bind the reader to the grid only when so; if not, the page will throw an exception. To get around that, we wrap the below portion of the code in a try...catch block.

      try
            {
myDataGrid.DataSource =
            myReader;
myDataGrid.DataBind();
int found =
            myDataGrid.Items.Count;
message.InnerText = "... " + found + "
            records found in Products Table
where Description contains '" +
            Request.Form["strSearch"] + "'";
myDataGrid.Visible =
            true;
}
catch(Exception e) {
message.InnerText = "Sorry, no
            match found. Please try again.";
myDataGrid.Visible =
            false;
}

In Developing High-Performance ASP.NET Applications, Microsoft recommends NOT to rely on exceptions as a way to control normal program flow, IF it is possible to detect in code a condition that would cause an exception. But in this situation, using if (myReader.Read()) to test would fail because invoking Read before binding the reader to the grid returns one item less in the DataGrid than the actual number of records returned by the query. Recall that a DataReader supplies a forward-only stream of data, and once a row has been read, you cannot get back to it. In other words, what binds to the grid then is only what's left in the reader (one less). Try it.

Specifying Multiple Conditions

You can include as many conditions as needed in a query, connecting them with the "AND " or "OR " operators. This greatly expands the search functionality by allowing, for example, searching for the given string in two fields.

String myQuery = "SELECT Type, Brand, Model, Description, Price FROM Products
WHERE Description LIKE '%" + Request.Form["strSearch"] + "%'
OR Model LIKE '%" + Request.Form["strSearch"] + "%'
ORDER BY Brand, Price";

"AND " trims the search criterion as both conditions must be met, while "OR " broadens the possibilities further. You can also use the "NOT " operator to reverse the value of a logical expression, for example:

String myQuery = "SELECT . . . WHERE State='CA' AND NOT City='Los Angeles'";

In cases where you need to refer to the search string repeatedly, you can assign this to a variable, and refer to the variable instead. This is more for code simplification rather than a performance issue in .NET, since the entire page is compiled once and all object references are handled automatically.

String fnd = Request.Form["strSearch"];
String myQuery = "SELECT . . . WHERE Description LIKE '%" + fnd + "%'
OR Model LIKE '%" + fnd + "%' ORDER BY . . .";

Keep in mind the logical placement of quotes when using object references in queries, more so when used with multiple conditions, different operators and mixed data types, or the query won't work. The general format is:

"SELECT . . . WHERE fldName1='" + var1 + "' OR fldName2 LIKE '%" + var1 + "%'
AND fldName3<=" + var2;

where var2 in this case is a numeric field, meaning it must not have quotes. Later in this series, we shall devote an entire article on SQL (Structured Query Language) basics as used in ADO.NET.

Searching for Multiple Records at the DataSet Level

The following example is similar to the above, but does the record selection at the DataSet level. The logic is basically identical, except that the method applies a "filter condition" to the DataSet to retrieve the search results. Also in this sample, the result set is displayed in a templated <asp:DataList> control instead.

Show me | View Source

ADO.NET includes the DataView class, which allows us to selectively "screen out" records in a DataSet object. A DataView may be likened to "saved views" in MS Access or stored procedures that return specific result sets in SQL Server. A DataView can be filtered using the same language rules as an SQL Select query, but the filter is dynamic. For instance, if a row is added to the data, and it qualifies with the filter, it is visible in the view. Likewise, filtered views can be sorted as well. To set up a DataView, you construct it by handing it a DataTable from the DataSet:

DataView myView = new DataView(myDataSet.Tables["Products"]);

Next, you set up the filter via the DataView object's RowFilter property, using an expression similar to that used in the WHERE clause of the SQL search example above.

myView.RowFilter = "Description LIKE '%" + Request.Form["strSearch"] + "%'";

To set up a sort, you can use a comma-delimited list of DataColumn names, followed by the keywords ASC (default and optional) or DESC, to specify whether each field is sorted in ascending or descending order. The data is not physically rearranged, but is simply accessed in the sorted order.

myView.Sort = "Brand, Price DESC";

In the same way as in a query, you can include as many conditions as needed in a filter, connecting them with the "AND " or "OR " operator.

Likewise, you have to provide some way to determine if the filter in fact returns a result set. Again, the .NET SDK does not include a specific property or method to check whether a filtered DataView contains data. In this case, we simply test the Items.Count property of the DataList, after invoking DataBind():

      myDataList.DataSource
            = myView;
myDataList.DataBind();

int found =
            myDataList.Items.Count;
if (found > 0) {
message.InnerText
            = "... " + found +
" records found in Products Table where
            Description contains '" +
Request.Form["strSearch"] +
            "'";
myDataList.Visible = true;
}
else
            {
message.InnerText = "Sorry, no match found. Please try
            again.";
myDataList.Visible = false;
}

Conclusion

Well, that's about it for this session. Hopefully again, this article has at least helped enhance your acquaintance with ADO.NET, particularly on the different ways to search a database for information, and to effectively test and display the results. By mixing in code shown in various examples in this series, such as sorting, selecting and displaying the selected item in the same or different page, you can adapt the above concepts to serve as a front-end in almost any type of e-retail application.

In the download file, you will run into some code whose annotations have been omitted here for simplicity (and space constraints). These pertain mostly to performance tuning, particularly output caching, maintaining state and so forth, the details of which are aptly covered in Developing High-Performance ASP.NET Applications.

And, due to fair trade practices (and bandwidth) restrictions, I am unfortunately not at liberty to include the outdoor gear database (2.5 Mb including over 250 images) with the download. You can, of course, run tests with the samplers using any similar data source.

In .NET Data Access Primer Part V, we shall look into the ADO.NET concepts and methods to enable users to move around the records in a data source. If you haven't done so, I suggest you start with Part I to help you "get connected".

C#Today - © 2002 WroxPress Limited


Building an ASP.NET Mailing List

Building an ASP.NET Mailing List
http://www.csharptoday.com/content.asp?id=1634
Article Rating: 4.5
Published on August 14, 2001
Abstract
In his first article, Doug Seven showed how a guestbook could be made in ASP.NET and C#. In this follow-up, he looks at creating a simple mailing list tool. This uses a SQL Server table to store the names and e-mail addresses of the subscribers, and also provides the ability to send an e-mail to the entire list. This article is bundled with Doug's Guestbook article as a ?2 e-doc available from http://www.amazon.com/exec/obidos/ASIN/B00005TZ91/ref=nosim/wrox-20.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

In my last article I showed you how to build a guestbook using ASP.NET and C#. In this follow-up article I will show you how to build a mailing list in ASP.NET and C#. The mailing list will have two parts to it. The first part is a form where visitors to your Web site can sign-up for the mailing list. The second part is a Web Form that you can use to send an e-mail, either in text or HTML format, to all of the subscribers on your mailing list.

For the code in this article you will need:

  • .NET Framework Beta 2
  • Windows 2000 with IIS 5.0
  • Internet Explorer 5.5
  • SQL Server 7.0 or 2000
  • A text editor, such as Notepad, TextPad, or Visual Studio.NET Beta 2

Building the Data Structure

The mailing list application uses an SQL Server table to hold the name and e-mail addresses of your subscribers. Visitors to your Web site can sign-up to your mailing list by entering their information in a form, and submitting it. When it is submitted, the name and e-mail address is added to the table using a stored procedure. In this stored procedure you will first check to see if the user is already in the table, and if they are, you will display a message telling them that they are already subscribed, and not re-enter the data. This will prevent duplicate addresses from being added to your mailing list. If their e-mail address is not found in the table, you will add their name and e-mail address to the mailing list table and display a message indicating that they have been subscribed.

All of the data access is abstracted to a data layer class, named MailingListData. The separation of the application layers is illustrated here:

To begin, create a new table in your SQL Server database named "MailingList ".

  1. Open the SQL Server Enterprise Manager
  2. Navigate to your database
  3. Right-click on the "Tables " node and select "New Table "
  4. Create two columns
  5. Email
    varchar 255
    Primary Key
    Allow Nulls = false
  6. Name
    varchar 200
    Allow Nulls = true
  7. Save the table as "MailingList "

In Enterprise Manager Design Mode, the table should look like this:

The email column will be used as the Primary Key. This will be used to ensure that each e-mail address entered is a unique address, helping to prevent duplicate entries.

Configuration Settings

The SqlConnection s in the MailingListData class are constructed using ConfigurationSettings.AppSettings["conString"] in place of their connection string value. This retrieves a key/value pair from the <appSettings> element in your Web.config file. The <appSetting> element is an optional element that you can add to the configuration file that allows you to define application-wide key/value pairs, such as database connection strings, in a centralized location. Since you don't have the conString key/value pair in the configuration file yet, you will have to add it. Add the following element to your configuration file, inside the <configuration> element.

<appSettings>
<add key="conString"
value="server=localhost;database=ASPToday;uid=sa;pwd=;" />

</appSettings>

The Stored Procedure

Before you build the sign-up Web Form, you need to build the JoinMailingList stored procedure. In the SQL Server Enterprise Manager:

  1. Navigate to your database
  2. Right-click on the "Stored Procedures " node and select "New Stored Procedure "
  3. Enter the following stored procedure:

CREATE PROCEDURE [JoinMailingList]
@visitorName varchar (200),
@email varchar (255),
@result bit output
AS
IF (SELECT COUNT(*) FROM Mailinglist WHERE [Email] = @email) > 0
BEGIN
SET @result = 0
END
ELSE
BEGIN
INSERT INTO Mailinglist
(
[Email],
[Name]
)
VALUES
(
@email,
@visitorName
)
SET @result = 1
END
GO

In the JoinMailingList stored procedure you do two things. First you use the SELECT COUNT(*) function to see if there are any records in the table that already have the e-mail address being submitted. If a record exists, you return an output parameter, @result, with the value 0 (failed). If a record does not exist, you use an INSERT command to add the submitted data to the table, and return 1 (succeeded) in the @result output parameter.

The MailingListData Class Structure

The MailingListData class is the center point of the application. This is where data input by the user will be passed for entry into the database, and where data from the database will be retrieved and passed back to the presentation layer.

Create a new file named MailingListData.cs. Begin building this class by adding the following using statements and class declaration:

using System;
using System.Web;
using System.Web.UI;
using System.Web.UI.HtmlControls;
using System.Web.UI.WebControls;
using System.Data;
using System.Data.SqlClient;
using System.Web.Mail;
using System.Configuration;

public class MailingListData
{

}

In this class you will create two methods, SignUp and SendMail. The first will be used to take in the name and e-mail address of the Web site visitor and add it to the database using the JoinMailingList stored procedure.

The SignUp Method

Add the following method inside the class declaration:

public string SignUp(string visitorName, string email)
{

using System;
using System.Web;
using
            System.Web.UI;
using System.Web.UI.HtmlControls;
using
            System.Web.UI.WebControls;
using System.Data;
using
            System.Data.SqlClient;

public class Signup :
            Page
{
protected TextBox Name;
protected TextBox
            Email;
protected Label ResultMsg;

protected void
            Submit_Click(object sender, EventArgs
            e)
{
if(Page.IsValid)
{
SqlConnection con = new
SqlConnection("server=localhost;database=ASPToday;uid=sa;pwd=;");
SqlConnection(ConfigurationSettings.AppSettings["conString"]);
SqlCommand
            cmd = new SqlCommand("JoinMailingList", con);
cmd.CommandType =
            CommandType.StoredProcedure;
            cmd.Parameters.Add(new
SqlParameter("@visitorName",
            SqlDbType.VarChar, 200));
cmd.Parameters.Add(new
            SqlParameter("@email", SqlDbType.VarChar,
            255));
cmd.Parameters.Add(new SqlParameter("@result",
            SqlDbType.Bit));

cmd.Parameters["@visitorName"].Value =
            visitorName;
cmd.Parameters["@email"].Value =
            email;
cmd.Parameters["@result"].Direction =
            ParameterDirection.Output;

try
{
con.Open();
cmd.ExecuteNonQuery();
con.Close();

if(Convert.ToBoolean(cmd.Parameters["@result"].Value))
{
return
            " - You have been subscribed to the mailing
            list.";
}else{
return " - You are already subscribed to the
            mailing list.";
}
}
}
}catch
{
return "There was
            an error processing your sign-up. Please try
            again.";
}
}

In the Signup method you take the name and e-mail address as input arguments, and then create a SqlConnection and a SqlCommand to execute the JoinMailingList stored procedure. You create three SqlParameters for the SqlCommand: @visitorName and @email, which are input parameters, and @result, which is an output parameter. The SqlDbType enumeration for each parameter is set to the data type expected by the stored procedure. The values for the @visitorName and @email parameters are set using the values of the two method arguments.

Once all of the values are set, the SqlConnection is opened, and the SqlCommand is executed using the ExecuteNonQuery method. This method is used when no result set is expected, such as with an INSERT or UPDATE command. Once the SqlCommand has executed, you check the value of the @result output parameter. This parameter returns a bit data type (0 or 1). You use the Convert class to convert the value to a boolean data type (0 = false, 1 = true). If @result returns false, the method returns a message indicating that the visitor is already a subscriber. In the event of an error, the catch handler returns a user-friendly message. If the @result parameter returns true then a message is returned indicating that the visitor has been added to the mailing list, as shown below.

The SendMail Method

Add the following method inside the class declaration:

public void SendMail(HtmlForm _form, string
            fromAddress,
string subject, string message, bool
            isHtml)
{

using System;
using System.Web;
using
            System.Web.UI;
using System.Web.UI.HtmlControls;
using
            System.Web.UI.WebControls;
using System.Data;
using
            System.Data.SqlClient;
using System.Web.Mail;

public class
            SendMail : Page
{
protected TextBox Message;
protected
            TextBox Subject;
protected TextBox From;
protected CheckBox
            Html;
protected HtmlForm myForm;

protected void
            Submit_Click(object sender, EventArgs
            e)
{
if(Page.IsValid)
{
MailMessage mm = new
            MailMessage();
mm.From = From.Text;fromAddress;
mm.Subject =
            Subject.Text;subject;

if(Html.Checked)if(isHtml)
{
mm.BodyFormat
            = MailFormat.Html;
}else{
mm.BodyFormat =
            MailFormat.Text;
}

}

SqlConnection con = new
SqlConnection("server=localhost;database=ASPToday;uid=sa;pwd=;");
SqlConnection(ConfigurationSettings.AppSettings["conString"]);
SqlCommand
            cmd = new SqlCommand("SELECT * FROM MailingList",
            con);

try
{
con.Open();
SqlDataReader reader =
            cmd.ExecuteReader();

while(reader.Read())
{
mm.To =
            (string)reader["email"];
(string)reader["name"];
mm.Body =
            message.Replace("<--Name-->",
            (string)reader["name"]);
SmtpMail.Send(mm);
_form.Controls.Add(new
LiteralControl("Mail sent to: " + mm.To +
            "<br>"));
Response.Flush();
}
//Flush the page output
            to the
            browser
HttpContext.Current.Response.Flush();
}

con.Close();
}
catch
{
//Add
            error handling
}
}

In the SendMail you take in five arguments: _form, which is a reference to the HtmlForm in the calling Web Form; fromAddress, which is the e-mail address that will appear in the form field of the e-mail; subject, which is the value that will appear in the Subject field of the e-mail; message, which is the body of the e-mail you are sending; isHtml, which is a boolean value indicating if the e-mail should be sent as text, or HTML.

In the SendMail method you create an instance of the System.Web.Mail.MailMessage class. This class instance represents the e-mail that gets sent. You set the MailMessage.From and MailMessage.Subject properties to the values the fromAddress and subject arguments passed into the method. If the isHtml argument is true you set the MailMessage.BodyFormat property to the MailFormat.Html enumeration, otherwise you set it to MailFormat.Text.

Once the basic MailMessage is set up, you create a SqlConnection and a SqlCommand to connect to your database and select all of the records from the MailingList table. You use the SqlCommand.ExecuteReader method to return the result set in a SqlDataReader, which creates a data stream from the database. This is the most memory efficient way to return data that is going to be iterated through from beginning to end. A DataReader returns a basic forward-only, read-only stream of data.

Using the SqlDataReader.Read method, you iterate through the records in the result set. For each record, you assign the email column value to the MailMessage.To property.

The last thing to do with the MailMessage before sending it is to set the Body property. This is set using the String.Replace("<--Name-->", (string)reader["name"]) method. This method creates a new string value, replacing any instances of <--Name--> with the value of the name column value from the SqlDataReader. This is how your e-mail gets personalized. Anywhere that the string <--Name--> appears in the message, the name of the subscriber will appear.

To send the MailMessage, you use the System.Web.Mail.SmtpMail.Send method. This method takes the MailMessage object as its argument. For confirmation purposes, you add a new LiteralControl to the HtmlForm referenced in the _form argument. This LiteralControl is just a text string indicating that the e-mail was sent to the specific recipient. You use the HttpContext.Current.Response.Flush method to flush, or render, the output to the client as each MailMessage is sent. Other than providing a confirmation for you, this also enables you to send the e-mail to a large list of subscribers without the Web request timing out. The Response.Flush method maintains the connection between the browser and the server as it flushes the output to the browser after each MailMessage is sent. If you did not do this, and your MailingList table had a few hundred records in it, the Web request may timeout before all of the MailMessage s are sent. Using Response.Flush will prevent the Web request from timing out.

Lets start putting this all together.

Building the Presentation Layer

The signup.aspx Web Form is a fairly simple form. It is two text boxes, one for the visitor's name, one for their e-mail address, and a submit button. The signup.aspx Web Form uses a code behind class file named signup.aspx.cs.

Building the Sign-Up Web Form

First create the Web Form by creating a file named signup.aspx, and add the following code:

<%@ Page Language="C#" INHERITS="Signup" %>
<html>
<body>
<form runat="server">
<table border="0" bordercolor="Tan"
CellPadding="4" CellSpacing="0"
Width="100%">
<tr bgcolor="Maroon">
<td valign="top" colspan="2"
style="font: bold 10pt Verdana; color:white;">
Join the Mailing List <asp:Label runat="server" id="ResultMsg" />
</td>
</tr>
<tr>
<td valign="top"
style="font: bold 9pt Verdana">
Name:
</td>
<td Width="100%" NOWRAP>
<asp:TextBox runat="server" id="Name"
Width="400"
Font-Size="9pt" Font-Name="Verdana"
BorderStyle="Solid" BorderWidth="1" />
<asp:RequiredFieldValidator runat="server"
ControlToValidate="Name"
ErrorMessage="*required"
Font-Size="9pt" Font-Name="Verdana" />
</td>
</tr>
<tr>
<td valign="top"
style="font: bold 9pt Verdana" NOWRAP>
E-Mail:
</td>
<td Width="100%">
<asp:TextBox runat="server" id="Email"
Width="400"
Font-Size="9pt" Font-Name="Verdana"
BorderStyle="Solid" BorderWidth="1" />
<asp:RequiredFieldValidator runat="server"
ControlToValidate="Email"
ErrorMessage="*required"
Font-Size="9pt" Font-Name="Verdana" />
<asp:RegularExpressionValidator
runat="server"
ControlToValidate="Email"
Display="Dynamic"
ValidationExpression="[\w\x2E\x2D]{2,}\x40{1}[\w\x2E\x2D]{2,}\x2E{1}[\w\x2E\x2D]{2,}"
ErrorMessage="Use the format name@server.com"
Font-Size="9pt" Font-Name="Verdana" />
</td>
</tr>
<tr>
<td></td>
<td Width="100%">
<asp:Button runat="server"
Text="Join Mailing List"
Width="400"
Font-Size="9pt" Font-Name="Verdana"
BorderStyle="Solid" BorderWidth="1"
BorderColor="Tan" BackColor="Maroon"
ForeColor="White" Font-Bold="True"
OnClick="Submit_Click" />
</td>
</tr>
</table>
</form>
</body>
</html>

In the signup.aspx Web Form you create a simple table and include the two text boxes, plus a RequiredFieldValidator for each text box. The RequiredFieldValidators are used to ensure that a value is submitted for each text box. You set the ControlToValidate property for each RequiredFieldValidator to the respective TextBox control's ID property. The Email control also has a RegularExpressionValidator associated to it. This uses a regular expression to validate the e-mail address and ensure that it is well formed (i.e. a valid format for an e-mail address).

The Button control's OnClick property indicates that when the button is clicked, the Submit_Click event handler should fire. This event handler is in the code behind class for the Web Form. The code for the signup.aspx.cs file is shown here:

using System;
using System.Web;
using
            System.Web.UI;
using System.Web.UI.HtmlControls;
using
            System.Web.UI.WebControls;

public class Signup :
            Page
{
protected TextBox Name;
protected TextBox
            Email;
protected Label ResultMsg;

protected void
            Submit_Click(object sender, EventArgs
            e)
{
if(Page.IsValid)
{
MailingListData mld = new
            MailingListData();
ResultMsg.Text = mld.SignUp(Name.Text.Trim(),
            Email.Text.Trim());
}
}
}

In the code behind class for the signup.aspx Web Form you map three variables to the server controls in the Web Form. You create one event handler method, Submit_Click. In the Submit_Click method you check to ensure the page is valid, and if it is, you call the MailingListData.SignUp method. When you call the method, you pass the Name.Text and Email.Text values in as arguments. Notice that the Trim() method of the String class is used on each. This is to remove any white space from the beginning and end of the string values in the text boxes. This ensures "clean" values (values with no extraneous white space) are entered into the database. The return value from the method call is set to the Text property of the ResultMsg Label control.

Now when a visitor enters their name and e-mail address and clicks on the button, their name and e-mail address are inserted into the MailingList table, and a message is rendered to the page, as seen here:

The ePony Express

Once you have entries in your MailingList table, you can send bulk mail such as a newsletter to the entire list. This is done with an administrative page named sendmail.aspx. The sendmail.aspx Web Form uses three TextBox controls (subject, from and message), a CheckBox control (HTML format) and a Button control (Send Mail). The mail is sent by creating a System.Web.Mail.MailMessage object, and then sending it to each entry in the MailingList table. The Web Form will look like the page seen here:

Like the signup.aspx Web Form, the sendmail.aspx Web Form uses a code behind class to separate the code and content. The code for the Web Form (sendmail.aspx) is shown here:

<%@ Page Language="C#" INHERITS="SendMail" SRC="SendMail.aspx.cs" %>
<html>
<body style="font: 9pt Verdana">
<form runat="server" id="myForm">
<table border="0" bordercolor="Tan"
CellPadding="4" CellSpacing="0"
Width="100%">
<tr bgcolor="Maroon">
<td valign="top" colspan="2"
style="font: bold 10pt Verdana; color:white;">
Send Mail to the Mailing List
</td>
</tr>
<tr>
<td valign="top"
style="font: bold 9pt Verdana" NOWRAP>
Send Mail As HTML:
</td>
<td Width="100%" style="font: 9pt Verdana">
<asp:CheckBox runat="server" id="Html" />
</td>
</tr>
<tr>
<td valign="top"
style="font: bold 9pt Verdana" NOWRAP>
Subject:
</td>
<td Width="100%" style="font: 9pt Verdana">
<asp:TextBox runat="server" id="Subject"
Width="400"
Font-Size="9pt" Font-Name="Verdana"
BorderStyle="Solid" BorderWidth="1" />
</td>
</tr>
<tr>
<td valign="top"
style="font: bold 9pt Verdana" NOWRAP>
From:
</td>
<td Width="100%" style="font: 9pt Verdana">
<asp:TextBox runat="server" id="From"
Width="400"
Font-Size="9pt" Font-Name="Verdana"
BorderStyle="Solid" BorderWidth="1" />
</td>
</tr>
<tr>
<td valign="top"
style="font: bold 9pt Verdana" NOWRAP>
Message:
</td>
<td Width="100%" style="font: 9pt Verdana">
Put <font color="#CC3300"><b>&lt;--Name--&gt;</b></font>
where you want the name of the recipient in the e-mail.<br>
<asp:TextBox runat="server" id="Message"
Width="400"
TextMode="Multiline" Rows="5"
Font-Size="9pt" Font-Name="Verdana"
BorderStyle="Solid" BorderWidth="1" />
</td>
</tr>
<tr>
<td></td>
<td Width="100%">
<asp:Button runat="server"
Text="Send Mail"
Width="400"
Font-Size="9pt" Font-Name="Verdana"
BorderStyle="Solid" BorderWidth="1"
BorderColor="Tan" BackColor="Maroon"
ForeColor="White" Font-Bold="True"
OnClick="Submit_Click" />
</td>
</tr>
</table>
</form>
</body>
</html>

In the Web Form you simple create a basic HTML table layout. You make three TextBox controls, one for the subject of the e-mail, one for the from-address, and one for the actual message. You also create a CheckBox control named Html. This CheckBox control will be used to allow you to specify that the e-mail should be sent as HTML format. If the CheckBox is not checked, the e-mail will be sent as plain text. The last control is a Button control with its OnClick property set to Submit_Click. Like the signup.aspx Web Form, this property specifies the name of the event handler to fire when the button is clicked.

In the SendMail code behind class you create the Submit_Click event handler. In this event handler you call the MailingListData.SendMail method to send the e-mail to each recipient. The code for the SendMail code behind class is shown here:

using System;
using System.Web;
using
            System.Web.UI;
using System.Web.UI.HtmlControls;
using
            System.Web.UI.WebControls;
using System.Data;
using
            System.Data.SqlClient;
using System.Web.Mail;

public class
            SendMail : Page
{
protected TextBox Message;
protected
            TextBox Subject;
protected TextBox From;
protected CheckBox
            Html;
protected HtmlForm myForm;

protected void
            ClearForm()
{
//Clear the form fields
Subject.Text =
            "";
From.Text = "";
Message.Text = "";
Html.Checked =
            false;
}

protected void Submit_Click(object sender,
            EventArgs e)
{
if(Page.IsValid)
{
MailingListData mld =
            new MailingListData();
mld.SendMail(myForm, From.Text.Trim(),
Subject.Text.Trim(), Message.Text.Trim(),
Html.Checked);
ClearForm();
}
}
}

In the sendmail.aspx.cs code behind class you create variables to map to the server controls on the Web Form, and the HtmlForm element. In the Submit_Click event handler method you check to ensure the page is valid, then call the MailingListData.SendMail method. You pass all of the server control values and a reference to the HtmlForm element into the SendMail method.

Running the Application

Now that all of the class files are written, your application is complete. The only thing left to do is compile the classes into a DLL and put it in the \bin directory of your application. If you are using Visual Studio.NET you can do this by selecting Build from the Build menu. If you are not using Visual Studio.NET then you can use a command line compiler.

Open a command prompt and navigate to the directory that your class files are in using the CD command. The CD command will look something like this:

C:> CD C:\Inetpub\wwwroot\ASPToday\MailingList

Once there, use the csc.exe compile by executing the following command:

csc.exe /target:library /out:ASPToday.dll *.cs /r:System.dll /r:System.Data.dll

This will compile all of the .cs files in the current directory into a DLL named ASPToday.dll. Move this DLL to the \bin directory of your application, and you are ready to go.

Navigate to the sigunp.aspx Web Form and enter your e-mail address. Then, navigate to the sendmail.aspx Web Form. By filling out the sendmail.aspx Web Form as seen here:

Each recipient will receive a text e-mail like this:

If you fill out the sendmail.aspx Web Form, indicating the e-mail should be sent using HTML format, and then put HTML tags in the message TextBox like this:

Each recipient will then receive an e-mail that looks like this:

Summary

In this article I showed you how to build a simple mailing list tool for your Web application. This tool uses a SQL Server table to store the names and e-mail addresses of subscribers to your mailing list. Visitors to your Web site can subscribe to your mailing list by filling out a form and submitting it. Their name and e-mail address are inserted into the database. When you want to send an e-mail you fill out a form, and using the System.Web.Mail namespace, create a MailMessage and send it to each recipient. This is a fairly simple mailing list tool, but has its limitations. For instance, if there are any more than a few hundred subscribers, sending all the mails will take a while and you will also have to leave your browser open while all the messages are sent. For larger mailing lists, this functionality can be converted to a Web Service, and an asynchronous "fire-and-forget" call can be made using a proxy client. The real purpose of this article is to show you the basic structure for building a simple mailing list tool. As your Web application grows, you may need to extend this functionality. This application can be extended to include a number of features, such as a preview Web Form, or a Web Form for sending test e-mails to yourself, etc.

C#Today - © 2002 WroxPress Limited


Developing a Universal Data Access Layer leveraging ADO.NET, C# and Factory Design Pattern

Developing a Universal Data Access Layer leveraging ADO.NET, C# and Factory Design Pattern
http://www.csharptoday.com/content.asp?id=1626
Article Rating: 4
Published on August 13, 2001
Abstract
In this article Naveed Zaheer shows us how to build a thin universal data access layer at the top of ADO.NET using Design Patterns and .NET Framework. This component layer, which will be based on ADO.NET, will provide access to OLE DB Managed Providers as well as SQL Server Managed Providers through a single interface. It will be extensible so that in the future we can add access to other Managed providers when they become available.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

Ideally, every software application will access the data from all possible data sources in the same manner. Every application will be able to manipulate data in every possible way. But we do not live in a perfect world. There are limitations on how we can or cannot access certain types of data. In the past few years Microsoft's ActiveX Data Object (ADO) has revolutionized data access mechanism. With the arrival of .NET, comes ADO.NET, which truly is the next generation of the ADO data species. In this article, readers will learn how to build a thin universal data access layer at the top of ADO.NET using Design Patterns and .NET Framework. This component layer, which will be based on ADO.NET, will provide access to OLE DB Managed Providers as well as SQL Server Managed Providers through a single interface. It will be extensible so that in the future we can add access to other Managed providers when they become available.

This article does not require a prior knowledge of Design Patterns. However, familiarity with Object Oriented Design and Analysis techniques is recommended. A brief overview of Design Patterns is covered later in the article. The discussion on n-tier (Presentation Tier, Business Tier, Data Tier) design and architecture is beyond the scope of this article. It assumes that readers have some familiarity with n-tier design.

An Overview of Data Access Strategies

Before we dive into ADO.NET, let's go through a brief overview of history of data access on Microsoft platforms including Windows and DOS. It all started with the arrival of ODBC (Open Database Connectivity). ODBC provided a single API-style interface to access heterogeneous data sources. Components called ODBC drivers provided the abstraction layer between databases and ODBC interface. Theoretically an application could switch to any database without modifying a single line of data access code for which an ODBC driver was available. Because of the API-style interface the use of ODBC was much more rampant in C/C++ applications than in Visual Basic applications. Then came a slew of data access frameworks such as DAO (Data Access Objects) and RDO (Remote Data Objects). DAO basically came with Visual Basic to supply developers with a data access class library for Microsoft's own desktop database named "Access". On the other hand RDO just provided an object-wrapper for ODBC. Although each of these solutions was effective for accessing certain types of data source, none provided a comprehensive solution. The arrival of ADO (Active Data Objects) a few years ago provided a far from elegant, but better solution to common data access problems. ADO is a user-friendly object wrapper around Microsoft's data access technology OLE DB. This object model provides uniform access relational as well as non-relational data. With the many great features it provides, the best one is a single interface to access all type of data sources. Just like ODBC drivers for ODBC, you have to develop OLE DB providers as an abstraction layer between database and OLE DB. From ODBC to ADO one theme was consistent i.e. a single interface to access different types of data sources. Although it was useful to switch between data sources without making many changes to the data access layer, in many cases it took away a chunk from performance. The availability of OLE DB provider for SQL Server alleviated the problem a little bit, but still performance was the issue. Because now it was possible to access SQL Server data by using this native OLE DB provider instead of OLE Db Provider for ODBC.

Next Generation of Data Access - ADO.NET

With the arrival of .NET came ADO.NET. ADO.NET is not here to replace classic ADO, but to offer a different approach to data access. Classic ADO offered disconnected recordsets but still most of the data access was in connected mode. What ADO.NET offers is a disconnected approach. Also, to boost performance it offers a separate, Managed Provider for different types of data sources. The idea is to take advantage of native database capabilities, instead of building a single interface across all backends based upon least common denominator. At the time of writing only three managed providers i.e. MS SQL Server, OLE DB and ODBC are available. Manager Provider for SQL Server goes directly through TDS (Tabular Data Stream), which is SQL Server's native data interface. It is only supported for SQL Server 7.0 or above. This way if you are accessing data from MS SQL Server, you get a big performance boost compared to OLE DB provider. Now we have separate classes to deal with when accessing data for OLE DB and MS SQL Server data sources. Suppose that you are using OLE DB data provider to access data from an Oracle database for your .NET application. In the meantime, an ADO.NET Managed Provider for Oracle native data interface becomes available from Microsoft or a 3rd party. Now to use the new Manager Provider, you may have to alter all your data access code.

Many businesses are running a mixture of different databases. Imagine a company's e-Commerce site for example. It may have a variety of databases that it uses for different purposes. For example, DB2 database, where its entire inventory has stored; Oracle database, where it's entire customer data is stored; and MS SQL Server database where it is planning to migrate in the future. What if we can have a single interface to access all these Managed Providers without sacrificing performance? This interface should be neither too general nor too specific. Whenever possible, this interface should try to expose the native Provider functionality instead of providing a wrapper for it. It should cover the entire common functionality supported by all the Manager Providers. The abstraction layer it provides should not be so thick that it degrades performance.

There are different ways to solve this problem. One way is to have a single class where each method has a switch-case statement to handle different data providers. Sample code for this solution is shown below:

public override void Execute(ProviderType ctProviderType,CommandType ctStatement, string strSQL)
{
switch(ctProviderType)
{
case ProviderType.MSSqlClient:
.
//Do Stuff Here for SqlClient
.
objSqlCommand.ExecuteNonQuery()
break;

case ProviderType.MSOleDb:
.
//Do Stuff Here for OleDb
.
objOleDbCommand.ExecuteNonQuery()
break;

default:
//Do Nothing
break;
}

}

However, this is not an elegant solution, because it will make code unnecessarily complex. Each time a new Managed Provider becomes available you will have to modify that class. After you make a change for one Managed Provider, you will have to regression test the component against all Managed Providers just to make sure you did not break anything.

Overview of Design Patterns

Let's see how Design Patterns can help us solving this problem. However, before we do that let's get a brief look at what Design Patterns are. Design Patterns are the robust solutions of frequently occurring problems, which evolve, because of object-oriented development, over a period of decades. These are not new techniques, which were never used before. You may have been using Design Patterns before without any knowledge of doing so. For example if you have programmed with Visual C++ MFC you must have used Document/View Architecture, which itself is based on a Design Pattern. If you have programmed with ATL (Active Template Library) or STL (Standard Template Library) then you must have used templates. A template is a type of Behavioral Design Pattern. In COM, design of interface IClassFactory is based on Factory Design Pattern. Singleton is a common Design Patten, which is used very often. For a detailed discussion about Design Patterns, please refer to the book mentioned in the Helpful Links section later in this article. Design Patterns are divided into three categories:

  • 1) Creational: Design Patterns in this category deal with object instantiation. Factory and Singleton belong to this category.
  • 2) Structural: Design Patterns in this category deal with object and class composition. Adapter and Facade belong to this category.
  • 3) Behavioral: Design Patterns in this category deal with algorithms and assigning object responsibilities. Mediator and Command belong to this category.

Factory Design Pattern

Sometimes an application or a framework does not know at runtime what kind of objects it has to create at runtime. It may only know about the abstract class for those objects. However, these abstract classes or interfaces cannot be instantiated. In other words, application only knows when to create the object but it does not know the type of the object to create. Factory Design Pattern does solve this problem. Factory Design Pattern Method is a Creational Design Pattern. The purpose of Factory method is to create objects. It helps us design an interface that creates objects of appropriate subclasses at runtime. It provides loose coupling eliminating the need to tie application specific classes together.

Factory Design Pattern is defined as "Define an interface for creating an object, but let the subclasses decide which class to instantiate." The Factory method lets a class defer instantiation to subclasses. The Factory Method lets a class defer instantiation to subclasses in the book Design Patterns.

In simple words, Factory Method lets you decide at runtime which subclasses of an abstract base class will be instantiated.

There are different implementations of Factory Method. One particular implementation that we are interested in is "Parameterized Factory method". In this implementation Factory Method is used to create multiple kinds of objects. It accepts a parameter that identifies which type of object to create. All the objects created implement the same interface or subclasses from the same abstract base class. Consider the following example:

public class Creator
{
public static int gintComicBook = 0;
public static int gintProgrammingBook = 1;
public static int gintFictionBook = 2;
public Creator()
{
}
public Book CreateBook(int iBookType)
{
switch(iBookType)
{
case gintComicBook:
return new ComicBook();

case gintProgrammingBook:
return new ProgrammingBook();

case gintFictionBook:
return new FictionBook();

};
}
}

In the code above function, CreateBook of class Creator takes an integer as a parameter. Based on that parameter it decides what type of object it is going to create. All the three classes i.e. ComicBook, ProgrammingBook, FictionBook, derive from the same base class Book.

Data Access Layer Implementation

Let's see that how we can use the Parameterized Factory Method implementation to create a single interface for all of the Managed Providers using Factory Method. First, we will have to have a Creator class with a function like CreateBook. This function will take a parameter identifying what type of Managed Provider object it has to create. For our implementation, class DataManager is our Creator class. Its function GetDBAccess, which has two overloads, behaves the same way as CreateBook behaves in the above example. Following is the partial code for DataManager class:

public static DBAccess GetDBAccessor(ProviderType ctProviderType,string strConnectionInfo)
{
DBAccess objDBAccess;
switch(ctProviderType)
{
case ProviderType.MSSqlClient:
objDBAccess = new DBAccessSQL(strConnectionInfo);
break;

case ProviderType.MSOleDb:
objDBAccess = new DBAccessOleDb(strConnectionInfo);
break;

default:
objDBAccess = null;
break;
}
return objDBAccess;
}
public static DBAccess GetDBAccessor(ProviderType ctProviderType)
{
return GetDBAccessor(ctProviderType,"");
}

As you can see from the code shown above, both overloads of function GetDBAccess take a parameter of type enumeration ProviderType. Based on the value of that parameter, it is decided from that object which class will be instantiated. Both classes i.e. DBAccessOleDb, DBAccessSQL, derive from the same base class DBAccess. DBAccess is the abstract base class that defines an interface common to all Managed Providers. Following is the partial code shown for the base class DBAccess:

public abstract class DBAccess : IDisposable
{
protected string mstrConnectionInfo;
//public properties
public abstract string ConnectionInfo
{
set;
get;
}
public abstract bool InTransaction
{
get;
}
//public functions
public abstract void BeginTransaction();
public abstract void CommitTransaction();
public abstract void AbortTransaction();
public abstract void Execute(CommandType ctStatement,string strSQL);
public abstract void Execute(CommandType ctStatement,string
strSQL,string strTable,out DataSet objDataSet);
public abstract void Execute(CommandType ctStatement,string
strSQL,out IDataReader objDataReader);
public abstract void SetParameter(string strName, ParameterDirection
pdDirection, ParameterType ptType, int intSize, object objValue);
public abstract void SetParameter(string strName, ParameterDirection
pdDirection, ParameterType ptType, int intSize);
public abstract void SetParameter(string strName, ParameterDirection
pdDirection, ParameterType ptType);
public abstract void ClearParameters();
public abstract void GetParameterValue(string strName, out object
objValue);
public abstract void GetParameterValue(string strName, out string
strValue);
public abstract void GetParameterValue(string strName, out short
shtValue);
public abstract void GetParameterValue(string strName, out int
intValue);
public abstract void GetParameterValue(string strName, out Double
dblValue);
public abstract void GetParameterValue(string strName, out Decimal
decValue);
public abstract void GetParameterValue(string strName, out long
lngValue);
public abstract void GetParameterValue(string strName, out DateTime
dateValue);
public virtual void Dispose()
{
}
}

One interesting thing about this class is that it implements IDisposable interface. This interface only has one method Dispose, for which implementation has to be provided by the class, which implements that interface. As we know, in .NET Framework, Garbage Collection handles the object cleanup. However, it is recommended that if you are dealing with precious system resources, such as database connections, in your class then you should expose a Dispose method. In that method you should properly cleanup those resources. Applications using the objects of your class should call this method to properly and timely free the system resources.

As we know that classes derived from DBAccess encapsulates important system resources such as database connection. It is possible in some cases that an object of type DBAccess is de-referenced but Garbage Collector may take some time to collect. In that case, the database connection that is encapsulated by the subclass of DBAccess will not be freed until the Garbage collection takes place. This is not a desirable behavior and may consume resources on the database sever. That's where use of Dispose method comes into the picture. We can put all the database connection cleanup code in the Dispose method. An application that is using an object of class DBAccess will call Dispose to free up any database resources. Note that classes such as DBAccessSQL, which is derived from DBAccess implements IDisposable inherently because it is implemented in its base class.

As you can see none of the functionality in the abstract base class DBAccess is provider specific. All the properties and functions of base class DBAccess, which are declared abstract, will require implementation in the derived classes. All the provider specific functionality is implemented in the derived classes. It will be a lot clearer when we start looking at the different subclasses of DBAccess class. Let's look at the code shown below for DBAccessSQL:

internal class DBAccessSQL :
            DBAccess
{
private SqlConnection mobjConnection =
            null;
private SqlCommand mobjCommand = null;
private
            SqlDataAdapter mobjDataAdapter = null;
private SqlTransaction
            mobjTransaction = null;
internal DBAccessSQL(string
            strConnectionInfo)
{
mstrConnectionInfo =
            strConnectionInfo;
mobjCommand = new
            SqlCommand();
mobjDataAdapter = new
            SqlDataAdapter();
}
internal
            DBAccessSQL()
{
mstrConnectionInfo = "";
mobjCommand = new
            SqlCommand();
mobjDataAdapter = new
            SqlDataAdapter();
}

Class DBAccessSQL is derived from DBAccess. Access modified for class DBAccess is "public" while access modifier for DBAccessSQL is "internal". Access modifier "internal" makes sure that this class is not visible outside its own namespace. Therefore, when we build DataManager component as a DLL, consumers will not be able to create the instances of class DBAccessSQL directly. They will only be able to create them through the creator class DataManager. That is exactly what we want. In addition, DBAccessSQL declares objects of type SqlConnection, SqlCommand and SqlDataAdapter as data member. Data members of the class mobjCommand and mobjDataAdapters are initialized in the object's constructor. You can set value of data member mstConnectionInfo in different ways. One way is that when you call GetDBAccess method of class DataManager, you can pass connection string as one of the parameters. Another way to set this information is by calling property ConnectioInfo of class DBAccessSQL. Whatever way you set this property, you have to do it before calling functions Execute or BeginTransaction. Otherwise, an SQLException will be thrown by the Managed Provider.

One benefit that we get from building a wrapper around the ADO.NET data access functionality is that we are able to hide complex implementation details from the consumer of the component. For example, you know that to get a DataSet build based on the results returned from a query, we have to use the Command object of DataAdapter class. For a query that returns no results, we can use Command object directly. As you can see from the code listed above, function Execute has four overloads. Each of them returns a different type of object as out parameters. These overloads of function Execute hide the complex details on how different data access objects such as DataReader or DataSet was created.

public override void SetParameter(string strName, ParameterDirection pdDirection,
ParameterType ptType)
{
try
{
SqlParameter objParameter = mobjCommand.Parameters.Add(strName,
GetSqlParameterType(ptType));
objParameter.Direction = pdDirection;
}

//Exception Handling Code Here
}

Code for one of the overloads of function SetParameter is shown above. Overloads of the function SetParameter are used to add parameter to the Parameters collection of Command object for the query or the stored procedure to be executed. As you can see, function SetParameter does not take any of the Provider dependent parameter types as parameter. It takes enumeration ParameterType declared in file DataManager.cs as a parameter for this function. This enumeration abstracts out the differences between the parameters types used by different Managed Providers. It is the responsibility of the derived class of DBAccess to convert it to Provider-Specific parameter type. On thing to note here is that we always add a parameter to the Parameters collection of mobjCommand data member. So, what happens when the overload of function Execute is called that uses data member mobjDataAdapter ? In that case function BindParameters is called by the Execute function, which copies parameters from data member mobjCommand's Parameters collection to the Parameters collection of mobjDataAdapter's SelectCommand property. Listing for function BindParameter is shown below:

private void BindParameters (SqlCommand
            objCommand)
{
if (mobjCommand.Parameters.Count >
            0)
{
foreach(SqlParameter objParameter in
            mobjCommand.Parameters)
{
SqlParameter objNewParameter =
objCommand.Parameters.Add(objParameter.ParameterName,objParameter.SqlDbType,objParameter.Size);
objNewParameter.Value
            = objParameter.Value;
objNewParameter.Direction =
            objParameter.Direction;
}
}
}

Now let's look at the code for some of the more interesting functions of class DBAccessSQL. Following is the code for one of the overloads of function Execute:

public override void Execute(CommandType
            ctStatement, string
            strSQL)
{
try
{
CreateConnection();
SetCommandData(mobjCommand,ctStatement,strSQL);
mobjCommand.ExecuteNonQuery();
}

//Exception
            Handling Code Here
}

This function is pretty simple. Parameters to this function are enumeration CommandType, which may be a stored procedure, name of a table or an SQL query and the statement text itself. This function first calls CreateConnection. Function CreateConnection is responsible for creating and opening the database connection if one does not exists. It does so by checking data member mobjConnection. If data member mobjConnection is not null and its state is Open, this function does not do anything. Otherwise, it creates and opens a new SQLConnection object and assigns it to data member mobjConnection. As said before, it is recommended that as soon as you are done with an instance of class DBAccessSQL, you should call its Dispose method. Dispose method takes care of closing the database connection. The reason why it is done this way is to follow the principle of getting a system resource as late as possible and releasing it as early as possible once it is no longer required. This way no one holds precious system resources for too long and their availability is maximized. A complete example on the sequence of how different methods of this class should be called will be explained in a later section.

Finally, function GetParameterValue that has eight overloads is used to extract the value of parameters returned after executing a stored procedure. Each overload returns a parameter of a specific type as out parameter. Function ClearParameters just clears out all the parameters that are set by calling function SetParameter. If there is no parameter set by calling function SetParameter, this function does not do anything. As mentioned earlier, overloads for function GetParameterValue are used to extract the value of the parameters returned by the stored procedure. As you can see from the code shown below for one of the overloads of this function, it simply gets the parameter value from mobjCommand's Parameters collection.

public override void GetParameterValue(string strName, out object objValue)
{
try
{
objValue = mobjCommand.Parameters[strName].Value;
}

//Exception Handling Code Here
}

Testing

In the code download there is an application called DataManagerTest that we will use to test this component. We will use the pubs database that comes with MS SQL Server as our test database. As you can see from the screenshot below, that each buttons is to test a specific functionality.

First let me show you how we will use this component. The partial code for one of the button onclick event is shown below:

DBAccess objABAccess =
            DataManager.GetDBAccessor(ptType,mstrConnectionInfo);
objABAccess.SetParameter("@JobId",ParameterDirection.Input,ParameterType.dmPtSmallInt,2,14);
objABAccess.SetParameter("@JobDesc",ParameterDirection.Output,ParameterType.dmPtVarChar,50);
objABAccess.Execute(CommandType.StoredProcedure,strSQL);
objABAccess.GetParameterValue("@JobDesc",out
            strJobDesc);
objABAccess.Dispose();

As you can see, we first get a reference of a new objDBAccess component by calling GetDBAccessor method of DataManager with proper ProviderType enumeration value. Then parameters are assigned to that component by calling function SetParameter. The next step is to call Execute method with proper CommandType and SQL text. If the command type is stored procedure and the stored procedure has any output parameters, you can use overloads of function GetParameterValue to extract their values. Finally, once you are done with the DBAccess object then it is highly recommended that you call its Dispose method to cleanup system resources in a timely manner. If you want to reuse the same DBAccess object then do not call its Dispose method. However, make sure to call its function ClearParameters to safely remove all the parameters that were assigned to this object before. You should call ClearParameters before you do anything else with the DBAccess object. Partial code for this DBAccess reuse is shown below:

DBAccess objABAccess = DataManager.GetDBAccessor(ptType,mstrConnectionInfo);
//Code for SetParameters
//Code to Call Execute
//Code to Get Parametrs or Deal with Data Set
//Make Sure you call Clear Paramaetrs before you do anything
objABAccess.ClearParameters()
//Code to Reuse the same object
//Code to SetParameters
//Code to Call Execute
//Code to Get Parametrs or Deal with Data Set
//Once you are done then call Dispose
objABAccess.Dispose();

Tools and Technologies Used

We will be using the following tools and technologies to build this component.

  • . NET Framework Beta 2
  • Visual Studio .NET Beta 2
  • Windows 2000 Server or Professional
  • Internet Information Server 5.0
  • C#
  • SQL Sever 7.0 or 2000

Conclusion

This completes our discussion regarding this article. DBAccessOleDb is the other class that is derived from DBAccess. It works pretty much the same was as DBAccessSQL. There is some functionality, which is only available in Managed Provider for SQL Server. To support that kind of functionality you have two options: One, that you can somehow implement that functionality yourself, the other being to throw a proper exception. Some other features that we may want to implement later are bulk updates using DataSet and DataAdapter.

One other thing to note here is how straightforward Object Oriented development is with .NET Framework. With classic COM, only interface inheritance was possible. So to develop an application based on complex Object Oriented designs you had to improvise and use roundabout techniques. However, with .NET Framework both interface inheritance and implementation inheritance are available. This way any type of Object Oriented implementation is possible using the .NET Framework. You are not forced to convert your OO classes to interfaces to make things work, as you had to do in the case of classic COM. In addition, you can take advantage of different OO techniques without any problem.

C#Today - © 2002 WroxPress Limited


Building a Database-Driven Guestbook with C# and ASP.NET

Building a Database-Driven Guestbook with C# and ASP.NET
http://www.csharptoday.com/content.asp?id=1546
Article Rating: 3.5
Published on August 7, 2001
Abstract
In this article, Doug Seven focuses on building a guestbook for a Web site. In the next part he will concentrate on building a mailing list as well. Both of these tools will be implemented using some common principals ?a functional user interface, database access, and efficient business logic. This article is bundled with Doug's Mailing List article as a ?2 e-doc available from http://www.amazon.com/exec/obidos/ASIN/B00005TZ91/ref=nosim/wrox-20.
Article

This is the first of two articles about building useful visitor features for your Web application. This article will focus on building a guestbook for your Web site, and the next article will focus on building a mailing list. Both of these tools can be implemented using some common principals - functional user interface, database access, and efficient business logic.

This article will teach you how, using ASP.NET and C#, you can build a guestbook that displays entries in chronological order in a paging-enabled DataGrid. The guestbook entries will be inserted into a SQL Server database. The DataGrid will enable custom paging, allowing you to retrieve only the number of entries you need to display from the database, making the most efficient use of memory on the server. When you're done, your guestbook will look something like this:

For the code in this article you will need:

  • .NET Framework Beta 2
  • Windows 2000 with IIS 5.0
  • Internet Explorer 5.5
  • SQL Server7.0 or 2000
  • A text editor, such as Notepad, TextPad, or Visual Studio.NET Beta 2

Building the Data Structure

The guestbook will provide a user interface (a form) where visitors to your Web site can enter their message and click a button to submit it. When the button is clicked, the data will be inserted into an SQL Server table using a stored procedure. Each time the page is rendered, including after a post, the data will be retrieved from the table using another stored procedure, and rendered in a DataGrid. All of the data access is abstracted to a data layer class, named GuestbookData. The separation of the application layers is illustrated here:

To begin, you will need to build the table in your SQL Server database. For simplicities sake, name the table "guestbook." The table should have six columns: id, title, visitorName, email, entryDate, and message. The id column is the Primary Key, and should be set up as an Identity column with a Seed value of 1 and an Increment value of 1. In the SQL Server Enterprise Manager, the table will look like this:

Building the Data Access Layer

Since we have started with the backend of the application, the database, we will continue working our way outward. The data access layer (DAL) class, GuestbookData, will expose the two methods we need to make the guestbook work, getGuestbookData and addGuestbookData.

Using a DAL class we abstract the exchange between our application and the database away from the presentation layer, making it more object oriented and the code more reusable. This is good programming practice. The DAL class will be compiled into our application's DLL and be available to any other class, including a Page class (such as a Web Form), at any time.

Configuration Settings

The SqlConnection s in the GuestbookData class are constructed using ConfigurationSettings.AppSettings["conString"] in place of their connection string value. This retrieves a key/value pair from the <appSettings> element in your Web.config file. The <appSetting> element is an optional element that you can add to the configuration file that allows you to define application-wide key/value pairs, such as database connection strings, in a centralized location. Since you don't have the conString key/value pair in the configuration file yet, you will have to add it. Add the following element to your configuration file, inside the <configuration> element.

<appSettings>
<add key="conString"
value="server=localhost;database=guestbook;uid=sa;pwd=;" />
</appSettings>

The Stored Procedures

You need to create the AddMessageToGuestbook stored procedure in your database. To do this, add the following code to your database:

CREATE PROCEDURE [AddMessageToGuesbook]
@title ntext,
@visitorName nvarchar (100),
@email nvarchar (255),
@message ntext
AS
INSERT INTO
guestbook
(
title,
visitorName,
email,
EntryDate,
message
)
VALUES
(
@title,
@visitorName,
@email,
GETDATE(),
@message
)
GO

This stored procedure enters all of the supplied fields into the guestbook table. The only fields not supplied are the ID field, which is an identity or auto increment field (and not used in this stored procedure), and the EntryDate field, which is supplied using the SQL GETDATE() function.

To display the guestbook entries you will use a DataGrid and bind the records from the guestbook table to it. Now, if everything goes well you will have thousands of entries in your guestbook. The last thing you want is to retrieve all of those records every time the page is rendered. Imagine if one hundred people viewed the page all at the same time. If you have 1,000 records in the table, and you retrieve them all for each request, that is 100,000 records you are putting into memory. Now image 1,000 people viewing the guestbook at once, and 10,000 records in the table. To eliminate this memory drain we will use custom paging in the DataGrid, and only retrieve exactly the records you need to display.

To enable the custom-paging scenario you need to create a stored procedure that retrieves only one page of data at a time, as well as a value indicating how many records there are. The page is defined by the DataGrid's PageSize property. For argument's sake, lets say the PageSize property is set to 5. This indicates that five records will be displayed on each DataGrid page. If there are more than five records in the table, page number navigation links will be rendered to the Web page. For the stored procedure you will need to find out how many total records there are, and then retrieve only the records you need to display for the requested page. To do this, add the following stored procedure to the database:

CREATE PROCEDURE [GetGuestbookEntries]
@pagesize int,
@pagenumber int,
@totalrecords int output
AS
DECLARE @startrecord int
SELECT @totalrecords = COUNT(*) FROM Guestbook
SET @startrecord = @totalrecords - (@pagesize * @pagenumber)
SELECT * FROM Guestbook
WHERE
[id] >= @startrecord
AND
[id] <= @startrecord + @pagesize
ORDER BY
entryDate DESC
GO

One point to note is that this stored procedure assumes that all of the records in the database are numbered sequentially beginning at 1, without skipping any numbers, as they are in the sample we are building. If any records are deleted, or numbers are skipped, this stored procedure will not work as expected. If you anticipate this scenario happening, you could consider using a SQL temporary table to retrieve the records and renumber them.

The GetGuestbookEntries stored procedure takes two input parameters and returns a result set and one output parameter. The two input parameters are @pagesize, which is how many records are being displayed per page, and @pagenumber, which is the page number to be displayed. For example, if you are showing five records per page, and you are displaying the 3rd page, then @pagesize is 5 and @pagenumber is 3. The output parameter, @totalrecords, returns the total record count from the table. You will use this number to determine how many pages there are for displaying the navigation links.

In the stored procedure you create another variable, @startrecord, which is used to determine which record number (using the id value) is the first record to be selected. The @startrecord is the total number of records, minus the page size, multiplied by the page number. For example, if there are 100 records, and you are showing five records per page, and you are showing page three, then the @startrecord is 100-(5*3), or 85.

You then select the number of records you are showing on one page (5 in our example), where the id value is greater than or equal to the @startrecord value and less than or equal to the @startrecord plus the page size. In our example, that would be records 85 through 90. These records are then ordered by the entryDate in descending order. This method will return one page of records, ensuring that page one will have the newest records (100-(5*1) = records 95 through 100), and each page will list the entries from newest to oldest.

The GuestbookData Class Structure

Create a new file named GuestbookData.cs. Begin building this class by adding the following code, using statements and class declarations:

using System;
using System.Web;
using System.Data;
using System.Data.SqlClient;
using System.Configuration;

public class GuestbookData
{
}

Inside the class you will expose one public property, VirtualItemCount. This property will be used to enable the paging functionality on the DataGrid. I'll get more into that in a while, but for now add this line of code inside the class declaration:

public int VirtualItemCount;

The addGuestbookData Method

The first method you are going to build is the addGuestbookData. This method will take in the name of the Web site visitor, their e-mail address, the title of their message, and the body of the message as its arguments. These will all be string values. In the method you will connect to the SQL database and execute a stored procedure. Add the following code inside the class declaration:

public void addGuestbookData(string visitorName, string Email, string Title, string Message)
{
SqlConnection con = new SqlConnection(ConfigurationSettings.AppSettings["conString"]);

public class Guestbook : Page
{
protected DataGrid entryGrid;
protected TextBox Name;
protected TextBox Email;
protected TextBox Title;
protected TextBox Message;

protected void Page_Load(object sender, EventArgs e)
{
if(!Page.IsPostBack)
{
BindGrid();
}
}

protected void Submit_Click(object sender, EventArgs e)
{
if(Page.IsValid)
{
SqlCommand cmd = new SqlCommand("AddMessageToGuesbook", con);
cmd.CommandType = CommandType.StoredProcedure;
cmd.Parameters.Add(new SqlParameter("@visitorName", SqlDbType.NVarChar));
cmd.Parameters.Add(new SqlParameter("@email", SqlDbType.NVarChar));
cmd.Parameters.Add(new SqlParameter("@title", SqlDbType.NText));
cmd.Parameters.Add(new SqlParameter("@message", SqlDbType.NText));

cmd.Parameters["@visitorName"].Value = visitorName;
cmd.Parameters["@email"].Value = Email;
cmd.Parameters["@title"].Value = Title;
cmd.Parameters["@message"].Value = Message;

try
{
con.Open();
cmd.ExecuteNonQuery();
con.Close();
}
catch
{
//Add error handling
}
}

The addGuestbookData uses a System.Data.SqlClient.SqlCommand object to execute a stored procedure. The stored procedure, AddMessageToGuestbook, takes four input parameters, @visitorName, @email, @title, and @message. The parameters are created in the command's parameters collection. Each parameter has its value assigned using the arguments passed into the method call.

Once all of the parameters are created and their values are set, you open the connection and use the ExecuteNonQuery method of the SqlCommand to execute the stored procedure. The ExecuteNonQuery method is used when a result set is not expected, such as in an INSERT command.

The getGuestbookData Method

Now we shall move onto BindGrid. We shall build the getGuestbookData method. This method is designed to return the records for one page of data to display in the DataGrid. I will explain how the paging functionality works in a while, but for now let's focus on this method. Add the following code to the Guestbook.cs code behind file, inside the class declaration:

protected void BindGrid()
{
//Make
            sure the form is cleared
Name.Text = Email.Text =
            "";
Title.Text = "";
Message.Text = "";
public DataSet
            getGuestbookData(int PageSize, int PageNumber)
{
//Create all
            the data objects
DataSet myDataSet = new
            DataSet();
SqlConnection con = new
SqlConnection(ConfigurationSettings.AppSettings["conString"]);
SqlConnection("server=localhost;database=ASPToday;uid=sa;pwd=;");
SqlDataAdapter
            cmd = new SqlDataAdapter("GetGuestbookEntries",
            con);
cmd.SelectCommand.CommandType =
            CommandType.StoredProcedure;
cmd.SelectCommand.Parameters.Add(new
SqlParameter("@pagesize",
            SqlDbType.Int));
            cmd.SelectCommand.Parameters.Add(new
SqlParameter("@pagenumber",
            SqlDbType.Int));
            cmd.SelectCommand.Parameters.Add(new
SqlParameter("@totalrecords",
            SqlDbType.Int));

cmd.SelectCommand.Parameters["@pagesize"].Value
            = PageSize;
cmd.SelectCommand.Parameters["@pagenumber"].Value =
            PageNumber;
cmd.SelectCommand.Parameters["@totalrecords"].Direction
            = ParameterDirection.Output;

//Fill the
            DataSet
cmd.Fill(myDataSet, "Messages");

//Set the
            VirtualItemCount to render the correct number of navigation
            links
VirtualItemCount =
            Convert.ToInt32(cmd.SelectCommand.Parameters["@totalrecords"].Value);

entryGrid.DataBind();
}
return
            myDataSet;
}

The getGuestbookData method takes in two arguments, the page size (number of records per displayed page), and the requested page number. The data is retrieved using an SqlDataAdapter, and it is filled into a DataTable in a DataSet. You do this because you need to access the @totalrecords output parameter before the DataGrid is bound. If you were to use a SqlDataReader, then the output parameter would not be accessible until all of the records in the SqlDataReader have been read, which is too late. You need the @totalrecords value to set the DataGrid.VirtualItemCount, which the DataGrid uses in conjunction with the PageSize property to determine how many page navigation links to render.

Executing the SqlDataAdapter.Fill method opens the connection to the database and returns the result set into a new DataTable in the specified DataSet. In the code sample you create a new DataTable named "Messages." Next you set the DataGrid.VirtualItemCountVirtualItemCount property of the GuestbookData class to the value of the @totalrecords parameter. This is the total number of records in the guestbook table. Lastly you return the DataSet to the calling object.

Building the Presentation Layer

With the DAL class built, you can begin building the input interface for the guestbook. The input interface is made up of four TextBox es and a Button. Create a new Web Form named guesbook.aspx, and then add the following code:

<%@ Page Language="C#" INHERITS="Guestbook" %>
<html>
<body>
<form runat="server">
<table border="0" bordercolor="Tan"
cellpadding="4" cellspacing="0"
width="100%">
<tr bgcolor="Maroon">
<td valign="top" colspan="2"
style="font: bold 10pt Verdana; color:white;">
Add your Guesbook Entry:
</td>
</tr>
<tr>
<td valign="top"
style="font: bold 9pt Verdana">
Name:
</td>
<td Width="100%">
<asp:TextBox runat="server" id="Name"
Width="400"
Font-Size="9pt" Font-Name="Verdana"
BorderStyle="Solid" BorderWidth="1" />
<asp:RequiredFieldValidator runat="server"
ControlToValidate="Name"
ErrorMessage="*required"
Font-Size="9pt" Font-Name="Verdana" />
</td>
</tr>
<tr>
<td valign="top"
style="font: bold 9pt Verdana">
E-Mail:
</td>
<td Width="100%">
<asp:TextBox runat="server" id="Email"
Width="400"
Font-Size="9pt" Font-Name="Verdana"
BorderStyle="Solid" BorderWidth="1" />
<asp:RequiredFieldValidator runat="server"
ControlToValidate="Email"
ErrorMessage="*required"
Font-Size="9pt" Font-Name="Verdana" />
</td>
</tr>
<tr>
<td valign="top"
style="font: bold 9pt Verdana">
Title:
</td>
<td Width="100%">
<asp:TextBox runat="server" id="Title"
Width="400"
Font-Size="9pt" Font-Name="Verdana"
BorderStyle="Solid" BorderWidth="1" />
<asp:RequiredFieldValidator runat="server"
ControlToValidate="Title"
ErrorMessage="*required"
Font-Size="9pt" Font-Name="Verdana" />
</td>
</tr>
<tr>
<td valign="top"
style="font: bold 9pt Verdana">
Message:
</td>
<td Width="100%">
<asp:TextBox runat="server" id="Message"
TextMode="Multiline" Rows="4"
Width="400"
Font-Size="9pt" Font-Name="Verdana"
BorderStyle="Solid" BorderWidth="1" />
<asp:RequiredFieldValidator runat="server"
ControlToValidate="Message"
ErrorMessage="*required"
Font-Size="9pt" Font-Name="Verdana" />
</td>
</tr>
<tr>
<td></td>
<td Width="100%">
<asp:Button runat="server"
Text="Click Here to Add Your Entry"
Width="400"
Font-Size="9pt" Font-Name="Verdana"
BorderStyle="Solid" BorderWidth="1"
BorderColor="Tan" BackColor="Maroon"
ForeColor="White" Font-Bold="True"
OnClick="Submit_Click" />
</td>
</tr>
</table>

The preceding code sets up the input portion of the Web Form. This includes four TextBox es for the visitor to enter their name, e-mail address, the message title and the message. The Message TextBox uses the TextMode property and the Row property to create a <textarea> HTML element. Each TextBox control has a RequiredFieldValidator associated with it. The validator control allows you to require that the Web site visitor fills out all the TexBox es. The Button control uses the OnClick property to define the event handler, Submit_Click, that should be fired when the button is clicked.

Just below the input portion of the Web form, you will use a DataGrid to display the guestbook entries. Add the following code:

<asp:DataGrid runat="server" id="entryGrid"
AutogenerateColumns="False"
Width="100%"
BorderColor="Tan" BorderWidth="1"
Gridlines="Horizontal"
Cellpadding="4" CellSpacing="0"
HeaderStyle-Font-Name="Verdana"
HeaderStyle-Font-Size="10pt"
HeaderStyle-Font-Bold="True"
HeaderStyle-BackColor="Maroon"
HeaderStyle-ForeColor="White"
ItemStyle-Font-Name="Verdana"
ItemStyle-Font-Size="9pt"
AlternatingItemStyle-BackColor="Tan"
AllowPaging="True"
PageSize="5"
AllowCustomPaging="True"
OnPageIndexChanged="entryGrid_PageIndexChanged" >
<PagerStyle Mode="NumericPages"
Font-Name="Verdana"
Font-Size="9pt"
Font-Bold="True"
HorizontalAlign="Center"
Position="TopAndBottom" >
</PagerStyle>
<Columns>
<asp:TemplateColumn HeaderText="Guestbook Entires">
<ItemTemplate>
<b><%# DataBinder.Eval(Container.DataItem, "title") %></b>
- <%# DataBinder.Eval(Container.DataItem, "entryDate", "{0:f}") %><br>
Submitted By: <%# DataBinder.Eval(Container.DataItem, "visitorName") %>
- <a href="mailto:<%# DataBinder.Eval(Container.DataItem, "email") %>">
<%# DataBinder.Eval(Container.DataItem, "email") %></a><br>
<%# DataBinder.Eval(Container.DataItem, "message") %>
</ItemTemplate>
</asp:TemplateColumn>
</Columns>
</asp:DataGrid>
</form>
</body>
</html>

A DataGrid, by default, will render a basic HTML table with one table row for each record displayed. You can bind any object that supports the IEnumerable, such as a DataTable or an Array. The DataGrid will automatically generate rows and columns for all of the data in its data source. This is called data binding. It is out of the scope of this article to fully explain data binding. The basic concept is that a data source is provided, such as a DataTable, and bound to the DataGrid, which results in an HTML table representing all of the data.

Your DataGrid uses a number of properties to customize its look and feel, as well as some properties to set up the paging format. The AutogenerateColumns property defines whether the DataGrid will generate the columns of the table based on the data source, or if you will create the columns using the various DataGridColumn objects, such as the BoundColumn, or TemplateColumn. The following style properties are used to customize the layout with things like color, font, etc. The four paging-related properties are used to set up the paging structure of the DataGrid. I'll cover that shortly, but first let's look at how the OnClick event of the Button at the top of the Web form is handled.

The Web Form Code Behind Class

The guestbook.aspx Web Form uses code behind technology to separate the programmatic code and the page layout. Create a file named Guestbook.cs, this will be the code behind the file. I chose this name since it is the same name that Visual Studio.NET would automatically create for the guestbook.aspx Web Form.

The VirtualItemCount property is used in conjunction with the PageSize property to determine how many page navigation links to render. If the VirtualItemCount is 100, and the PageSize is 10, then page navigation links for 10 pages will be rendered (100/10 = 10). Once the VirtualItemCount property is set, you set the Messages table as the DataGrid.DataSource, and call the DataBind method.

Can You Handle Paging?

The last thing you have to do before you can use the guestbook is add an event handler for the DataGrid.PageIndexChanged event. This event is fired whenever one of the page navigation links is clicked. In the PageIndexChanged event handler you set the CurrentPageIndex of the DataGrid to the NewPageIndex property of the DataGridPageChangedEventArgs. Add the following event handler to the Guestbook.cs code behind the file.

protected voidentryGrid_PageIndexChanged(object sender,DataGridPageChangedEventArgs e)
{
entryGrid.CurrentPageIndex = e.NewPageIndex;
BindGrid();
}

In the DataGrid server control you set the OnPageIndexChanged property to "entryGrid_PageIndexChanged". This will call the newly created event handler whenever a page navigation link is clicked. When the event handler fires, the DataGrid.CurrentPageIndex property will be set to the value of the DataGridPageChangedEventArgs.NewPageIndex, which is the page index for the page number that was clicked on. Once the CurrentPageIndex is set, you call the BindGrid method, and the data is retrieved, the DataGrid is bound, and finally the page is rendered to the client.

Add the following code to the code behind file:

using System;
using System.Web;
using
            System.Web.UI;
using System.Web.UI.HtmlControls;
using
            System.Web.UI.WebControls;
using System.Data;

public class
            Guestbook : Page
{
protected DataGrid entryGrid;
protected
            TextBox Name;
protected TextBox Email;
protected TextBox
            Title;
protected TextBox Message;

protected void
            Page_Load(object sender, EventArgs
            e)
{
if(!Page.IsPostBack)
{
BindGrid();
}
}

protected
            void ClearForm()
{
//Make sure the form is
            cleared
Name.Text = "";
Email.Text = "";
Title.Text =
            "";
Message.Text = "";
}
}

The code behind class Guestbook inherits from the System.Page class, which gives it all the functionality of the Page class, such as the Page_Load event. Just after the class decla ration you declare five variables. These variables directly map to objects in the guestbook.aspx file. This maps these objects to the class so you can access them programmatically.

In the Page_Load event handler you check to see if the current request is a postback. If it is not a postback then you call the BindData method, which you will build shortly.

The other method in the preceding code is called ClearForm. This will be used to clear all of the TextBoxes after a user's post is submitted.

Handling the Submit Click

The Submit_Click event handler is fired whenever the Button control in the Web Form is clicked. This is because you mapped the Button.OnClick property to a method named Submit_Click. Add the following code inside the class declaration of the code behind class:

  protected void
            Submit_Click(object sender, EventArgs
            e)
{
if(Page.IsValid)
{
GuestbookData gbDal = new
            GuestbookData();
gbDal.addGuestbookData(Name.Text.Trim(),
Email.Text.Trim(),
Title.Text.Trim(),
Message.Text.Trim());
BindGrid();
}
}

In the Submit_Click event handler you first check to see if the page is valid. This will evaluate true if all of the TextBoxes have something in them. If one or more are empty, the RequiredFieldValidators will return false, and the Page.IsValid property will be false. If the Page is valid then you create an instance of the GuestbookData DAL class you created earlier. You then call the addGuesbookData method, passing in the values from the TextBoxes. After the data has been entered in the database, you call the BindData method. This ensures that the new data is entered into the database before the DataGrid is bound.

Binding Data One Page at a Time

Now we shall look at the BindGrid method. Add the following code to the Guestbook.cs code behind file, inside the class declaration:

  protected void
            BindGrid()
{
ClearForm();
//Instantiate the DAL
            object
GuestbookData gbDal = new GuestbookData();
//Set the
            data source and bind the grid
entryGrid.DataSource =
            gbDal.getGuestbookData(entryGrid.PageSize,
entryGrid.CurrentPageIndex + 1);
entryGrid.VirtualItemCount =
            gbDal.VirtualItemCount;
entryGrid.DataBind();
}

In the BindGrid method you do two things. You call ClearForm to clear all of the TextBoxes so the form starts fresh every time, and you bind the DataGrid to the result of calling the getGuestbookData method.

The PageSize argument of the getGuestbookData method is set from the DataGrid.PageSize value, and the PageNumber argument is set from the DataGrid.CurrentPageIndex + 1. The CurrentPageIndex is a zero-based index; the CurrentPageIndex for page one is 0, the CurrentPageIndex for page two is 1, and so on. To set the PageNumber argument you add 1 to the CurrentPageIndex so page one is 1, page two is 2, and so on.

The DataGrid.DataSource property defines where the DataGrid will find the data to bind. Setting this to the result of calling getGuestbookData will cause that method to fire, and return a DataSet. The DataGrid will default to the first DataTable in the DataSet.Tables collection. In this case, the getGuestbookData method returns a DataSet with only one DataTable in it, so that is what the DataGrid will use as its data source.

The DataGrid.VirtualItemCount property is used in conjunction with the PageSize property to determine how many page navigation links to render. If the VirtualItemCount is 100, and the PageSize is 10, then page navigation links for 10 pages will be rendered (100/10 = 10). When the getGuestbookData method was called, the GuestbookData.VirtualItemCount property was set to the total number of records in the guestbook table. Once the VirtualItemCount property is set, you set the Messages table as the DataGrid.DataSource, and call the DataBind method. The DataGrid is bound to the data source, which is one page of data, and page navigation links are rendered based on the PageSize property and the VirtualItemCount property.

Can You Handle Paging?

The last thing you have to do before you can use the guestbook is add an event handler for the DataGrid.PageIndexChanged event. This event is fired whenever one of the page navigation links is clicked. In the PageIndexChanged event handler you set the CurrentPageIndex of the DataGrid to the NewPageIndex property of the DataGridPageChangedEventArgs. Add the following event handler to the Guestbook.cs code behind file.

protected voidentryGrid_PageIndexChanged(object sender,DataGridPageChangedEventArgs e)
{
entryGrid.CurrentPageIndex = e.NewPageIndex;
BindGrid();
}

In the DataGrid server control you set the OnPageIndexChanged property to "entryGrid_PageIndexChanged". This will call the newly created event handler whenever a page navigation link is clicked. When the event handler fires, the DataGrid.CurrentPageIndex property will be set to the value of the DataGridPageChangedEventArgs.NewPageIndex, which is the page index for the page number that was clicked on. Once the CurrentPageIndex is set, you call the BindGrid method, and the data is retrieved, the DataGrid is bound, and the page is rendered to the client.

Running the Application

Now that all of the class files are written, your application is done. The only thing left to do is compile the classes into a DLL and put it in the \bin directory of your application. If you are using Visual Studio.NET you can do this by selecting Build from the Build menu. If you are not using Visual Studio.NET you can use a command line compiler.

Open a command prompt and navigate to the directory that your class files are in using the CD command. The CD command will look something like this:

C:> CD C:\Inetpub\wwwroot\ASPToday\Guestbook

Once there, use the csc.exe compile by executing the following command:

csc.exe /target:library /out:ASPToday.dll *.cs /r:System.dll /r:System.Data.dll

This will compile all of the .cs files in the current directory into a DLL named ASPToday.dll. Move this DLL to the \bin directory of your application, and now you are ready to go.

Summary

In this article I showed you how to build a functional and efficient guestbook for your Web application. The guestbook uses a DataGrid with custom paging to ensure that only the necessary records are retrieved from the database and displayed in the browser for the visitor. In the following articles I will show you how to build a mailing list and use the System.Web.Mail namespace to send e-mail, such as a newsletter, to a list of recipients in a database. In the third installment, I will show you how to build a message board where visitors to your Web site can post and reply to messages.

C#Today - © 2002 WroxPress Limited


.NET Data Access Primer: Part 3

.NET Data Access Primer: Part 3
http://www.csharptoday.com/content.asp?id=1544
Article Rating: 4.5
Published on August 2, 2001
Abstract
This article is the third of a series on how to retrieve and update information stored in databases. In Part II, we covered how to update data at the database command level, using SQL append, update and delete queries. In this session Rey Nuñez focuses on the key ADO.NET concepts, components and methods to enable users to add, edit and delete information at the DataSet level.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

This article is the third of a series on how to retrieve and update information stored in databases. In Part II, we covered how to update data at the database command level, using SQL append, update and delete queries. In this session we focus on the key ADO.NET concepts, components and methods to enable users to add, edit and delete information at the DataSet level. To help better relate to the samplers, let's first take a closer look at how data is modeled in an ADO.NET DataSet.

Accessing Data in a DataSet Object

The centerpiece of the ADO.NET architecture is the DataSet class. The DataSet represents a complete set of 'disconnected' memory-resident data, structured much like a simple database, including tables, columns, rows, constraints, and relationships among the tables.

Each DataSet can contain multiple DataTable objects, with each DataTable containing data from a single data source, such as MS SQL Server. Access to each table is done via the DataSet object's Tables collection, which comprises the list of all DataTable objects in the set. We can refer to each DataTable in the DataSet by using either the table name:

myDataSet.Tables["Products"]

or by using the ordinal index of the table item in the collection, which starts counting from zero (is zero-based ), in the order the tables were added to the set. For example, to refer to the first DataTable in the DataSet:

myDataSet.Tables[0]

Each DataTable in turn contains a DataColumnCollection that is accessed via the table's Columns property. This comprises the list of all DataColumn objects for that table, which determines the schema of each DataTable. When a DataSet is used with an existing database, this collection will automatically contain a set of columns representing each selected field in the data source. For example, passing the following query to a DataAdapter to fill a DataSet (see Part I):

      String
            myQuery = "SELECT * FROM Products";
DataSet myDataSet = new
            DataSet();
OleDbDataAdapter myAdapter = new
            OleDbDataAdapter(myQuery,
            myConnection);
myAdapter.Fill(myDataSet, "Products");

results in a "Products" DataTable that will have the same schema or collection of columns as the table in the data source. In the same manner, we can refer to each DataColumn in the DataTable by using either the column name:

myDataSet.Tables["Products"].Columns["ProductID"]

or by using the ordinal index of the column item in the collection, which also is zero-based, in the order the columns were added to the table. For example, to refer to the second DataColumn in the specified DataTable:

myDataSet.Tables["Products"].Columns[1]

Each DataTable also contains a DataRowCollection that is accessed via the table's Rows property, which comprises the list of all DataRow objects that belong to that table. Each DataRow represents a record (a row of data) in the data source. We can refer to each DataRow by using the row item's ordinal index, which likewise is zero-based, in the order the records were added to the table. For example, to refer to the third DataRow in the "Products" table:

myDataSet.Tables["Products"].Rows[2]

Together, these two collections allow us to access the actual contents of a DataTable. To retrieve, evaluate and manipulate the values in the table, you simply specify the DataRow item, along with the DataColumn index or name. Typical hierarchical notation is:

myDataSet.Tables["tableName"].Rows[i] ["columnName"]

where [i] stands for the row item index. To simplify object references, you can store objects as named instances of a class, as in:

DataTable my able = myDataSet.Tables["Products"];

From then on, you can use these names instead in your code. For example, to populate form input controls with data from the first DataRow in the "Products" table (using the named table object above):

DataRow thisRow = myTable.Rows[0];
fldCategory.Value = thisRow["Category"];
fldType.Value = thisRow["Type"];
fldBrand.Value = thisRow["Brand"];
fldModel.Value = thisRow["Model"];
fldPrice.Value = thisRow["Price"].ToString();

Now let's see how we can apply all that in practical scenarios. The following sections assume that you have read and gone through the exercises in .NET Data Access 101: Part II. The discussions here focus mainly on the changes relevant to the samplers demonstrated in that article. Unlike in the methods previously described, though, the main distinction is that an instance of a DataSet must exist BEFORE any update can be performed.

Adding a new row of data

On the whole, to enable users to add a row of data into the DataSet, we provide a simple input form to the page (as described in Part II), and in the form submit event handler do the following steps:

1. Declare a new variable as type DataRow.

DataRow newRecord = myDataSet.Tables["Messages"].NewRow();

A new DataRow object is returned by invoking the DataTable object's NewRow() method. The DataTable then creates a blank DataRow object based on the schema of the table, as defined by its DataColumnCollection.

Assign each DataColumn the corresponding form input value, using either the column's index or name.

newRecord["MessageDate"] = msgDate.Value;
newRecord["MessageFrom"] = msgFrom.Value;
newRecord["Email"] = msgEmail.Value;
newRecord["MessageSubject"] = msgSubject.Value;
newRecord["MessageBody"] = msgBody.Value;

2.Then invoke the Add method on the Rows collection of the DataTable.

myDataSet.Tables["Messages"].Rows.Add(newRecord);

The below sample illustrates this code in action.

Show me | View Source

The addMessage method assumes an already open DataSet, which we implement in the Page_Load event handler at post-back. This way, the connection is made only when the user submits the form.

protected void Page_Load(Object Src,
            EventArgs E) {
if (IsPostBack) {
String strQuery = "SELECT
            MessageDate, MessageFrom, Email, MessageSubject, MessageBody FROM
            Messages";
OleDbConnection myConn = new
            OleDbConnection("Provider=Microsoft.Jet.OleDb.4.0;Data Source="
            +
Server.MapPath("/devguide/shared/data/guests.mdb"));
adapter
            = new OleDbDataAdapter(strQuery, myConn);
myDataSet = new
            DataSet();
adapter.Fill(myDataSet, "Messages");
. .
            .
}

Note in the addMessage method that the changes apply only at the DataSet level, and are reconciled with the source data only when the Update method of the DataAdapter that populated the DataSet is explicitly called. This allows for the addition of multiple rows (such as in a shopping cart app) before committing the appended rows back to the data source.

adapter.Update(myDataSet, "Messages");

The Update method in this case takes as arguments an instance of the DataSet that contains the changes, and the DataTable name or object from which to retrieve the changes. When the Update method is called, the DataAdapter analyzes the changes that have been made, and executes the appropriate command (INSERT, UPDATE, or DELETE) to resolve changes to the underlying data store. But note in the sample the absence of an explicit INSERT command. Instead we use the OleDbCommandBuilder to auto-generate the command. We'll get back to this soon after we look at how to edit and delete rows in the DataSet .

Editing an existing row of data

Now let's look at the steps to allow users to modify an existing row of data. These are basically the same as when adding a new record, except in this case you need some way to let the users choose which record to update. In the same manner (as described in Part II), we use a DataGrid with a Hyperlink Column for the selectable field, but how the record is actually selected is different. Recall that here we need to select a row from an instance of a DataSet, not from the underlying data store.

Show me | View Source

Here is a brief rundown of the page's main components.

An <asp:Panel> control (named loadPanel) displays the list of messages to select from using a templated <asp:DataGrid> control. The DataGrid contains a HyperLinkColumn that presents the data from the MessageFrom column as HyperLink controls. When a user clicks a hyperlink in this column, the ID of the record to edit is passed back to the page as a query string argument. For example, the URL that would appear in the browser's address bar when a user clicks on a link would look like:

edit.aspx?id=25

The Page_Load event handler obtains the specified MessageID using the QueryString collection of the page's Request object. This collection contains all query strings (the portion in the URL after the ?) sent from a client during an HTTP request, and is commonly the API via which page developers access arguments when doing page to page navigation transfers. Following are the steps to get to the specified record:

  • Because query string arguments are coerced into type String, we first need to convert the given MessageID back into an Integer (see table's structure in Part II).

int id = Int32.Parse(Request.QueryString["id"]);

  • We then retrieve the record from the DataSet using the Find() method of the DataTable's DataRowCollection, passing it the given id.

thisRecord = myTable.Rows.Find(id);

  • Then we call the getMessage() method to populate the form input controls with corresponding values from each DataColumn in the found row.

public void getMessage() {
msgDate.Value
            = thisRecord["MessageDate"].ToString();
msgFrom.Value =
            thisRecord["MessageFrom"].ToString();
msgEmail.Value =
            thisRecord["Email"].ToString();
msgSubject.Value =
            thisRecord["MessageSubject"].ToString();
msgBody.Value =
            thisRecord["MessageBody"].ToString();
editPanel.DataBind();
}

The form is then rendered in the <asp:Panel> control (named editPanel ) wherein the user can modify the values. When the form is submitted, control is passed to the updateMessage method that first assigns the modified form input values to each corresponding DataColumn , and then updates the data source.

public void updateMessage(Object sender,
            EventArgs E) {
if (Page.IsValid) {
thisRecord["MessageFrom"] =
            msgFrom.Value;
thisRecord["MessageSubject"] =
            msgSubject.Value;
thisRecord["Email"] =
            msgEmail.Value;
thisRecord["MessageBody"] =
            msgBody.Value;
adapter.Update(myTable);
}
postPanel.DataBind();
}

The Update method in this case takes as an argument the DataTable that contains the changes. The modified values are then displayed in the <asp:Panel> control named postPanel.

On Using Find to Retrieve Key Values

The general rule in better database design is that every table must have a column, or group of columns, that uniquely identifies each row in the table. This column (or columns) is called the primary key.

Basically, to make a column work as a primary key, you specify two column properties at the database design level: the values within the column cannot be null and must be unique. In other words, the column cannot be blank and no two values in the column can be the same. This ensures that no two records can ever share a primary key.

In certain cases, a primary key can be made up of two or more columns. This is usually the case when search operations on the data need to be done on a group of columns. A two-column primary key might consist of the "FirstName " and "LastName " columns, for example, to distinctly identify the records of, say, John Doe and John Dow.

The Find method used in the above example gets a DataRow specified by the primary key value - in other words, you cannot use Find with a DataTable that has no primary key. Find will throw a MissingPrimaryKey exception when used to access a row in a table that has no primary key.

Whilst it is possible to programmatically add a unique constraint to a DataColumn and assign that as a DataTable's primary key at the DataSet level, common database sense dictates specifying a primary key at the database design level. Our sample table uses the MessageID column as a single-column primary key.

On the other hand, the Fill method of the DataAdapter fills a DataSet only with table columns and rows from a data source. While constraints are commonly set at the data source, the Fill method does not automatically add this schema information to the DataSet.

To populate a DataSet with existing constraint information from a data source, you can either call the FillSchema method of the DataAdapter, or set the MissingSchemaAction property of the Fill method to AddWithKey before calling Fill. Adding schema information to a DataSet BEFORE filling it with data ensures that primary key constraints will be included with the DataTable objects in the DataSet. The Page_Load event handler of the example above illustrates how this is done:

protected void Page_Load(Object Src,
            EventArgs E) {
OleDbConnection myConn = new
            OleDbConnection("Provider=Microsoft.Jet.OleDb.4.0;Data Source=" +
Server.MapPath("/devguide/shared/data/guests.mdb"));
adapter
            = new OleDbDataAdapter("SELECT * FROM Messages ORDER BY
            MessageFrom", myConn);
myDataSet = new
            DataSet();
adapter.FillSchema(myDataSet, SchemaType.Source,
            "Messages");
adapter.Fill(myDataSet, "Messages");
. .
            .

And, in case you need to programmatically determine the schema of your data source . . .

The following SQL Server .NET Data Provider example selects the tables from an informational view provided by SQL Server and populates a DataTable using a DataAdapter that we render in a DataGrid.

Show me | View Source

The following OLE DB .NET Data Provider example uses the OleDbConnection object's GetOleDbSchemaTable method to return a DataTable with the schema information.

Show me | View Source

For details, see Obtaining Schema Information from a Database.

Deleting an existing row

Now let's look at the steps to allow a row to be removed from a DataSet. In the same manner as when editing a record, we let the user choose which record to delete, except in this case, we need not provide an entry form. To confirm that the selection is correct, though, we render the record's data and provide a form with just Delete and Cancel buttons.

Show me | View Source

ADO.NET provides two methods to delete a DataRow from a DataTable object: the Remove method of the DataRowCollection object, and the Delete method of the DataRow object.

The Remove method of the DataRowCollection takes a DataRow as an argument and removes it from the collection as in the following example.

myTable.Rows.Remove(thisRecord);

In contrast, the below example demonstrates how we call the Delete method on a DataRow, and then invoke Update to save the changes to the data source (as done in the sample).

thisRecord.Delete();
adapter.Update(myTable);

While the Remove method deletes a DataRow from the DataRowCollection, the Delete method only marks the row for deletion. By using Delete, you can programmatically check which rows are marked for deletion before actually removing them. When a row is marked for deletion, its RowState property is set to "Deleted". The actual removal of the row (in the associated data source) occurs only when the application invokes the DataAdapter's Update method. To accept changes while working at the DataSet level, the application can call the DataTable's AcceptChanges method.

Note that if you actually "remove" the row from the collection, the DataAdapter will not see that row when you update the data source. Thus it will not execute any command for that row. However, if the row is marked as "Deleted", the DeleteCommand of the DataAdapter (or generated by the CommandBuilder) will be invoked (see below for details).

On Reconciling Changes to the Data Source

As earlier mentioned, the Update method of the DataAdapter (used to populate a DataSet) must explicitly be called to resolve changes from the DataSet back to its data source.

When the Update method is invoked, the DataAdapter examines the state of each row, analyzing what changes have been made, and executes the required INSERT, UPDATE, or DELETE statements to the associated database. The syntax of the commands must be explicitly set (at design-time and, where possible, through the use of stored procedures) before calling Update. If Update is called and the appropriate command does not exist for a given update (for example, no InsertCommand for rows added to the DataSet), an exception will be thrown.

If, however, the DataTable in question maps to or is generated from a single database table, you can take advantage of the CommandBuilder class to automatically generate these commands. This simplifies and reduces the code required to perform INSERT, UDPATE, and DELETE operations.

In order for automatic command generation to work, at least the SelectCommand property of the DataAdapter must have been set. This is done either via the SELECT query used by the DataAdapter to fill the DataSet, or by explicitly setting the command text of the property. Then, any additional SQL statements that are not set are generated by the CommandBuilder. The DataTable schema retrieved by the SelectCommand determines the syntax of the automatically generated INSERT, UPDATE, and DELETE statements.

Note that the SelectCommand must also return at least one primary key or unique column, else an InvalidOperation exception is thrown and the commands are not generated. In other words, the CommandBuilder will work only with a Datable that has a primary key.

Using the CommandBuilder

ADO.NET provides a CommandBuilder object for both the SQL Server .NET Data Provider and the OLE DB .NET Data Provider. The above examples all use an OleDbCommandBuilder to auto-generate the needed SQL statements. Below shows how a new instance of the OleDbCommandBuilder is initialized with the associated OleDbDataAdapter object, as done in the samples.

   adapter = new
            OleDbDataAdapter("SELECT * FROM Messages ORDER BY MessageFrom",
            myConn);
myDataSet = new
            DataSet();
adapter.FillSchema(myDataSet, SchemaType.Source,
            "Messages");
adapter.Fill(myDataSet,
            "Messages");
OleDbCommandBuilder cmdBuilder = new
            OleDbCommandBuilder(adapter);

You can only associate one DataAdapter with one CommandBuilder object at any one time.

To generate INSERT, UPDATE, or DELETE statements, the CommandBuilder uses the properties of the DataAdapter's SelectCommand to automatically retrieve the required set of metadata. It is important to note that whenever the SELECT statement associated with the CommandBuilder changes, the application must refresh the schema information used to generate the commands by invoking the RefreshSchema method.

The CommandBuilder also uses the Connection, CommandTimeout, and Transaction properties referenced by the SelectCommand. Again, RefreshSchema must be invoked to update the metadata whenever any of these properties are modified. Otherwise, the generated InsertCommand, UpdateCommand, and DeleteCommand properties would still retain their previous values, and the Update method will fail.

Conclusion

Well, that's about it for this session. Hopefully at least, you've gained insight on the ways to add, edit or delete records at the DataSet level. Again, whilst seemingly basic, the methods described here can be usable to any application with similar data-based needs.

Which to use when, whether to invoke methods at the database command level or at the DataSet level, would of course depend on the purpose of each application. In general, the methods described here are better suited to scenarios with longer-running applications that need to work with disconnected DataSets, such as in a Web Service.

In a typical scenario, a client makes a request to a Web Service, and a response to the request populates a DataSet from a database using a DataAdapter, returning the DataSet to the client. The client can then view and make modifications to the data. When done, the client passes the modified DataSet back to the Web Service, which then uses the DataAdapter to reconcile the changes in the returned DataSet with the source database. The Web Service may then return a DataSet that reflects the current values in the database to confirm. Because a DataSet is stateless, it can be safely passed between the server and the client without tying up server resources such as database connections.

Additionally, since a DataSet is transmitted as an XML stream, non-ADO.NET clients can use the same Web Service as that consumed by ADO.NET clients. Likewise, ADO.NET clients can easily interact with non-ADO.NET Web Services by sending any client DataSet to the Web Service as XML data and by consuming any XML returned from the Web Service as a DataSet.

In .NET Data Access 101: Part IV, we shall look into the ADO.NET concepts and methods to enable users to search for information in a data store. If you haven't done so, I suggest you start with Part I to help you "get connected".

C#Today - © 2002 WroxPress Limited


.NET Data Access Primer: Part 2

.NET Data Access Primer: Part 2
http://www.csharptoday.com/content.asp?id=1596
Article Rating: 3.5
Published on June 26, 2001
Abstract
Whilst HTML and ASP alone may suffice when building basic interactive Web pages, the majority of dynamic, information-based Web applications require that a developer be able to work with databases. In this series, Rey Nuñez aims to help one learn how to develop active server pages in the .NET platform that are used to retrieve and update information stored in databases. The subject matter delves specifically on the key concepts and methods geared towards the construction of data-bound Web Forms (ASP.NET pages) using Web Forms controls, ADO.NET and the C# Language Specification.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

This article is the second of a series on how to retrieve and update information stored in databases. In this session we cover the key ADO.NET concepts, components and methods to enable users to add, edit, and delete information in a data store. To help better appreciate the tutorials, here we shall connect to a simple Access 2000 database named " Guests ", using a table named " Messages ". For your guidance, the table is structured as follows:

Field Name | Data Type | Description
MessageID | AutoNumber | Unique message identifier (for sorting purposes)
MessageDate | Date/Time | Stores the date / time when the message was created
MessageFrom | Text | Identifies the person leaving the message
Email | Text | Stores the email address of the person leaving the message
MessageSubject | Text | Identifies the subject/content of the message
MessageBody | Memo | Stores the body of the message

Adding a new record

Basically, to add a row of data into the database, we provide a simple input form to the page, and execute an SQL Insert command in the form submit event handler. The sample below illustrates how a new record can be added to our data source.

Show me

First, we create an entry form consisting of the set of fields we need to collect data from. To prevent accidental violation of the table's field constraints, we provide for checking the required input values using the validator controls included with ASP.NET. The RequiredFieldvalidator ensures that the compulsory fields are filled up, while the RegularExpressionValidator is used for checking the format of the email field. The ValidationSummary control displays a summary of the validation errors for all of the validators on a page.

<form runat="server">

<input type=hidden id="msgDate" runat="server">

<table width=85% cellspacing=1 cellpadding=3 border=0>
<tr>
<td>Your name:</td>
<td><input id="msgFrom" runat="server">
<asp:requiredfieldvalidator
controltovalidate="msgFrom"
errormessage="Sender's name cannot be blank. "
runat=server>*
</asp:requiredfieldvalidator></td></tr>
<tr>
<td>Your email address:</td>
<td><input id="msgEmail" runat="server">
<asp:regularexpressionvalidator
controltovalidate="msgEmail"
validationexpression="^[\w-]+@[\w-]+\.(com|net|org|edu|mil)$"
errormessage="Please enter a valid e-mail address. "
runat=server>*
</asp:regularexpressionvalidator></td></tr>
<tr>
<td>Your subject:</td>
<td><input id="msgSubject" runat="server"></td></tr>
<tr>
<td>Your message:</td>
<td><textarea id="msgBody" rows=5 cols=35 runat="server"></textarea></td></tr>
</table>
<p><input type="submit" value="Add New" runat="server" onserverclick="addMessage">
<input type="reset" value="Cancel" onclick="location.replace('http://www.asptoday.com')"></p>

<asp:validationsummary runat="server"
displaymode="SingleParagraph"
showmessagebox=true />

</form>

The structure of the form is fairly straightforward, having one input control (each designated with a unique identifier) for each of the fields in the table that we need to fill up. Note that we do not provide input for MessageID , which increments automatically, and MessageDate , which we fill in programmatically via an <input type=hidden id="msgDate"> control at the top of the form. This will contain the current DateTime value on post back, as shown below:

   if (IsPostBack)
            {
msgDate.Value = DateTime.Now.Format("g",null);
. .
            .

DateTime.Now returns a DateTime type representing the current date and time that depends on the system timer. Format() converts this DateTime to a String with the specified format character and culture info. The method defaults to the general ( "g" ') format and current culture if null or no arguments are passed.

Notice also how we attach the validator controls to the input controls whose values need to be verified:

<input id="msgFrom" runat="server">
<asp: requiredfieldvalidator
controltovalidate="msgFrom"
. . .
<input id="msgEmail" runat="server">
<asp: regularexpressionvalidator
controltovalidate="msgEmail"
. . .

When the form is posted, control is passed to the designated form submit handler, in this case addMessage() , which gathers the field names and values in the form's data set. After validation, these field names and values are added as a new row into the specified table via a SQL Insert command. While in certain cases it is possible to pass the form's input values directly, especially when all values are of type String , here I've opted to use the ADOCommand object's Parameters collection to populate the insert command's values. This ensures that the values passed are strictly matched to the corresponding field types (and optionally, field sizes as well) in the data source. The handler is defined as follows:

public void addMessage(Object sender, EventArgs E) {
if (Page.IsValid) {
String insertCmd = "INSERT INTO messages (MessageDate, MessageFrom, Email, MessageSubject, MessageBody)" & _
" VALUES (Date, From, Mail, Subject, Body)";
ADOCommand myCommand = new ADOCommand(insertCmd, myConnection);

myCommand.Parameters.Add(new ADOParameter("Date", ADODBType.Date));
myCommand.Parameters["Date"].Value = msgDate.Value;
myCommand.Parameters.Add(new ADOParameter("From", ADODBType.Char));
myCommand.Parameters["From"].Value = msgFrom.Value;
myCommand.Parameters.Add(new ADOParameter("Mail", ADODBType.Char));
myCommand.Parameters["Mail"].Value = msgEmail.Value;
myCommand.Parameters.Add(new ADOParameter("Subject", ADODBType.Char));
myCommand.Parameters["Subject"].Value = msgSubject.Value;
myCommand.Parameters.Add(new ADOParameter("Body", ADODBType.VarChar));
myCommand.Parameters["Body"].Value = msgBody.Value;

myCommand.ActiveConnection.Open();
myCommand.ExecuteNonQuery();
myCommand.ActiveConnection.Close();
}
BindPanel();
}

Note that the insert occurs only if the page passes the conditional check.

if (Page.IsValid) {
. . .
}

In a page with validation controls, the Page.IsValid property is set after all validation controls have been processed. If any of the controls reveals that a validation check has failed, the entire page is set to invalid . The condition we set-up then evaluates to false , and no insert happens. If a validation control is in error, the specified message (ex. errormessage="Sender's name cannot be blank. " ) displays in the ValidationSummary control set somewhere on the page.

Now let's take a closer look at how this Parameters collection is used to feed values to the insert command. In essence, for each value that we need to pass to the table, we add a new ADOParameter to the collection that specifies the parameter's name and data type :

myCommand.Parameters.Add(new ADOParameter("Date", ADODBType.Date));
. . .
myCommand.Parameters.Add(new ADOParameter("Subject", ADODBType.Char));

Then we set the Value property of each parameter to the value of the corresponding input control in the form.

myCommand.Parameters["Date"].Value = msgDate.Value;
. . .
myCommand.Parameters["Subject"].Value = msgSubject.Value;

The names to be specified in each ADOParameter constructor must be the names of the parameters used in the insert command ( Date, From, Mail, Subject, Body below):

String insertCmd="INSERT INTO messages (MessageDate, MessageFrom, Email, MessageSubject, MessageBody)" & _
" VALUES ( Date, From, Mail, Subject, Body )";

Then to perform the insert:

myCommand.ActiveConnection.Open();
myCommand.ExecuteNonQuery();
myCommand.ActiveConnection.Close();

When performing commands that do not require data to be returned (such as inserts, updates, and deletes), we activate the ADOCommand (or SQLCommand) by calling the ExecuteNonQuery () method. (For queries that return a data set, you must issue the Execute() method instead). Notice that the connection must be explicitly opened (and closed) when we use an instance of the Command class (the DataSetCommand automatically handles the connection for you).

We can also execute the insert command inside of a try . . . catch block to trap for other requisites, for instance, in odd cases where we let users input a primary key, and the key for the inserted row already exists.

      myCommand.ActiveConnection.Open();

try
            {
myCommand.ExecuteNonQuery();
}
catch (ADOException
            e){
for (int i=0; i < e.Errors.Count; i++){
// loop thru
            the Errors collection and display a
            message
}
}
myCommand.ActiveConnection.Close();

It is important to close the Connection before the page finishes executing. If not, you may inadvertently exhaust the connection limit while waiting for the page instances to be garbage-collected routinely.

Note that because we post back to the same page, we need some way of laying out the page structure since the user interfaces we have to provide at page load and at post back are different. With traditional ASP, we can typically intersperse HTML within ASP script blocks, and while still possible with the .NET page framework, ASP.NET provides a more efficient way of handling page layout with the Panel server control.

The <asp:panel> control provides a container for other controls; this control is rendered as an HTML <DIV> element. It is especially useful when you wish to generate controls programmatically or to hide and show a group of controls. Our sample illustrates use of two Panel controls to contain the separate interfaces for page load and post back. We simply switch the respective control's Visible property to either true or false.

The custom BindPanel() method is used to update the data bound to controls in the post back panel. The DataGrid is optional; it is used here to show that the record had indeed been added to the DataSet, and thus renders in the grid.

Click to view the complete source [C#] for the insert sample above.

Editing an existing record

Now let's look at the steps to allow users to modify an existing record. These are basically the same as when adding a new record, except in this case you need some way to let the users choose which record to update.

You can either provide an HTMLSelect control populated with values from a given field, as described in .NET Data Access Part I: Dynamic Queries, or use a DataGrid with a Hyperlink Column for the selectable field.

Show me

There may be other ways, but here's how it's done in the sample. First, we organize the page layout (and sequence) into three Panel controls: one to contain the interface for user selection, the next for the editing interface, and the last for the postback view. Likewise, we structure the page load handler to show/hide the matching panels and call the corresponding methods.

<script language="C#"
            runat="server">
ADOConnection myConnection;
String
            id;

protected void Page_Load(Object Src, EventArgs E) {
id
            = Request.QueryString["id"];
myConnection = new
            ADOConnection("Provider=Microsoft.Jet.OLEDB.4.0;Data Source="
+
            Server.MapPath("/devguide/shared/data/guests.mdb"));
if
            (IsPostBack) {
loadPanel.Visible = false;
editPanel.Visible =
            false;
postPanel.Visible = true;

}
else if
            (Request.QueryString.Count>0)
            {
getMessage(Request.QueryString["id"]);
loadPanel.Visible =
            false;
editPanel.Visible = true;
postPanel.Visible =
            false;
}
else {
BindGrid();
loadPanel.Visible =
            true;
editPanel.Visible = false;
postPanel.Visible =
            false;
}
}

// other method blocks here for getting and
            updating the row data

</script>

By default, the DataGrid generates a bound column for each field in the data source ( AutoGenerateColumns = true). Each field in the data is rendered in a separate column, in the order it occurs in the data. Field names appear in the grid's column headers, and values are rendered in text labels. A default format is applied to non-string values. The grid also supports other column types that render differently:

Column Name | Description
Bound Column | Lets you control the order and rendering of the columns.
Button Column | Bubbles a user command from within a row to an event handler on the Grid.
Edit Command Column | Displays Edit, Update, and Cancel links in response to changes in the DataGrid's EditItemIndex property.
HyperLink Column | Presents the bound data as HyperLink controls.
Template Column | Lets you control which controls are rendered in the column.

Any of the column types may be used together, as can be seen in the below snippet, which shows how the DataGrid is declared in the sample:

<asp:datagrid id="myDataGrid" runat="server"
. . .
autogeneratecolumns="false">

<property name="Columns">
<asp:boundcolumn headertext="Date"
datafield="MessageDate"
dataformatstring="{0:g}" />
<asp:hyperlinkcolumn
headertext="From"
datanavigateurlfield="MessageID"
datanavigateurlformatstring="update.aspx?id={0}"
datatextfield="MessageFrom" />
<asp:boundcolumn headertext="Subject"
datafield="MessageSubject" />
</property>
</asp:datagrid>

The BoundColumn represents a column bound to a data field, which is set by the DataField property. The DataFormatString property (used in the MessageDate column) sets the display format of data in the column and is specified as:

dataformatstring="{0:g}"

The data format string consists of two parts separated by a colon in the form {X:Xxx}. The first X (before the semicolon) specifies the parameter number in a zero-based list of parameters, which can only be "0" (zero) since there is only one value for each cell. The second X (after the semicolon) specifies the format in which to display the value according to the standard .NET formatting conventions. The small xx specifies the number of significant digits or decimal places to display the value (applies to numeric fields). The entire string is enclosed in curly brackets to denote that this is a format string and not a literal string. Any text outside of the curly brackets will be displayed as literal text.

The HyperLinkColumn creates a column within the DataGrid containing hyperlinks that navigate to specified URLs. The DataTextField property sets the field in the DataSource that will be used as the source of the hyperlinks' text. The DataNavigateUrlField property sets the field in the DataSource that provides the URL of the page to navigate to. In this case, we set this to the MessageID field. The DataNavigateUrlFormatString property sets the formatting applied to the resulting hyperlink. The below snippet show how to set this property:

datanavigateurlformatstring="update.aspx?id={0}"

As you may have probably noticed (in the browser's status bar), the value of the MessageID field appears in place of the {0} when a user hovers over or clicks the hyperlink. This value is then passed as an argument of the Request.QueryString that we then use to specify which record to edit on the succeeding page load.

. . .
else if (Request.QueryString.Count>0) {
getMessage(Request.QueryString["id"]);
loadPanel.Visible = false;
editPanel.Visible = true;
postPanel.Visible = false;
}

The getMessage() method is used to pick up the bits of data from the selected row and fill up the controls in the edit form. The below code shows how we define this method:

public void getMessage(String id) {
ADODataSetCommand myCommand = new ADODataSetCommand("SELECT * FROM Messages WHERE MessageID=" + id, myConnection);
DataSet ds = new DataSet();
myCommand.FillDataSet(ds, "msgDetails");
DataRowView editView = ds.Tables["msgDetails"].DefaultView[0];

msgDate.Value = editView["MessageDate"].ToString();
msgFrom.Value = editView["MessageFrom"].ToString();
msgEmail.Value = editView["Email"].ToString();
msgSubject.Value = editView["MessageSubject"].ToString();
msgBody.Value = editView["MessageBody"].ToString();
editPanel.DataBind();
}

As usual, to gain access to the data source, we create and fill a DataSet, this time with the results of a select query that returns only the data for the chosen record.

"SELECT * FROM Messages WHERE MessageID=" + id

Note that we also create and use an instance of the DataRowView class to simplify reference to the row of data when initially populating the HTMLInput controls in our edit form. The form is generally the same as in the insert example, except that the controls are filled with a representation of the source data.

Likewise, when the form is sent, control is passed to the submit handler updateMessage() , which collects the form's data set, and after passing validation, executes an SQL Update command to save the changes back to the record in the table. In the same manner, we use the ADOCommand object's Parameters collection to populate the update command's values.

public void updateMessage(Object sender,
            EventArgs E) {
if (Page.IsValid) {
String updateCmd = "UPDATE
            messages SET MessageFrom=From, Email=Mail, " &
            _
MessageSubject=Subject, MessageBody=Body WHERE MessageID=" +
            id;
ADOCommand myCommand = new ADOCommand(updateCmd,
            myConnection);

This acts as if the user pressed the browser's back button.

myCommand.Parameters.Add(new ADOParameter ("From", ADODBType.Char));

myCommand.Parameters["From"].Value = msgFrom.Value;

myCommand.Parameters.Add(new ADOParameter("Mail", ADODBType.Char));

myCommand.Parameters["Mail"].Value = msgEmail.Value;

myCommand.Parameters.Add(new ADOParameter("Subject", ADODBType.Char));

myCommand.Parameters["Subject"].Value = msgSubject.Value;

myCommand.Parameters.Add(new ADOParameter("Body", ADODBType.VarChar));

myCommand.Parameters["Body"].Value = msgBody.Value;

myCommand.ActiveConnection.Open();

myCommand.ExecuteNonQuery();

myCommand.ActiveConnection.Close();

}

postPanel.DataBind();

}

You should Click to view the complete source [C#] for the update sample above.

Note the use of some client-side JavaScript in certain parts of the code, specifically in the Cancel and Back To buttons. The Cancel button in the edit form calls a self.location.replace that essentially forces a new page load without any querystring parameter. The Back To Edit button in the post back panel calls a history.go(-1) that essential s is intended to simplify basic page navigation that is independent of, and does not affect, the server-side scripts. But recall that these may not function in browsers that do not support, or have disabled support for, client-side scripts.

Deleting an existing record

Now let's look at the steps to allow a record to be removed from a data source.

Similarly, as when editing a record, we let the user choose which record to delete, except in this case, we need not provide an entry form. To confirm that the selection is correct, though, we render the record's data and provide a form with just Delete and Cancel buttons.

As in the insert and update methods, the Delete button (which is really a submit button), actually posts back to the same page, but here we simply issue an SQL Delete command.

Show me

In C#, as well as in most programming languages, global variables, or page members that are to be made publicly available to any procedure on the page, must be declared after the main <script> tag and outside of any function block.

<script language="C#" runat="server">
ADOConnection myConnection;
String id, msgDate, msgFrom, msgEmail, msgSubject, msgBody;

protected void Page_Load(Object Src, EventArgs E) {
. . .
}

// other method blocks here
. . .
</script>

This makes the object definitions public, the purpose of which becomes evident when we need to bind to these objects somewhere in the page:

<table cellspacing=1 cellpadding=5 border=0>
<tr><td>Date:</td><td><b><%# msgDate %></b></td></tr>
<tr><td>Name:</td><td><b><%# msgFrom %></b></td></tr>
<tr><td>Email:</td><td><b><%# msgEmail %></b></td></tr>
<tr><td>Subject:</td><td><b><%# msgSubject %></b></td></tr>
<tr><td>Message:</td><td><b><%# msgBody %></b></td></tr>
</table>

Click to view the complete source [C#] for the delete sample above.

When working with larger databases, the DataGrid may prove unfit for providing the selection interface, as users would then have to scroll thru a lengthy list. Alternatively, we can provide first for selecting a subset of records from the data source, and then provide for selecting an item from that DataSet. Here we illustrate a variation of the dynamic DataList example shown in .NET Data Access: Part I, linking to the selected item's description along with a mock order form, all in one page. In this sample, though, I'd rather that you do not modify our sample database, so here we just display the data for the selection.

Show me

Conclusion

Well, that's about it for this session. Hopefully at least, you've gained insight on some of the ways to add, edit or delete records from a database in the .NET framework. Whilst outwardly basic, the methods described here can be usable to any application with similar data-based needs.

Whenever testing any update application, it is good practice to maintain backup copies of your data at all times. If you inadvertently update or delete the wrong records, you cannot undo the operation.

Again of course, the samples provided can further be improved on performance grounds. Because here we use separate pages to clearly show each method, we need to redundantly open and close the database. Ideally, table update methods are grouped onto one page, which can have Add, Edit and Delete buttons, as well as provide for search, paging and/or record navigation. This way, opening and closing the connection is done only once.

In .NET Data Access Primer Part III , we shall look into the ADO.NET concepts and methods to enable users to search for information and move around the records in a data store. If you haven't done so, I suggest you start with Part I to help you "get connected".

C#Today - © 2002 WroxPress Limited


.NET Data Access Primer

.NET Data Access Primer
http://www.csharptoday.com/content.asp?id=1594
Article Rating: 3
Published on June 25, 2001
Abstract
Whilst HTML and ASP alone may suffice when building basic interactive Web pages, the majority of dynamic, information-based Web applications require that a developer be able to work with databases. In this series, Rey Nuñez aims to help one learn how to develop active server pages in the .NET platform that are used to retrieve and update information stored in databases. The subject matter delves specifically on the key concepts and methods geared towards the construction of data-bound Web Forms (ASP.NET pages) using Web Forms controls, ADO.NET and the C# Language Specification.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

Whilst HTML and ASP alone may suffice when building basic interactive Web pages, the majority of dynamic, information-based Web applications require that a developer be able to work with databases.

This series aims to help one learn how to develop active server pages in the .NET platform that are used to retrieve and update information stored in databases. The subject matter delves specifically on the key concepts and methods geared towards the construction of data-bound Web Forms (ASP.NET pages) using Web Forms controls, ADO.NET and the C# Language Specification.

The articles in this series assume some familiarity with database fundamentals and SQL. For information on Web Forms basics, please refer to ASP.NET Web Forms: An Overview. Since most of the commentaries in this series essentially involve ADO, let's start with a quick backgrounder on that.

ADO.NET in Brief

ADO.NET is the next generation of Microsoft ActiveX Data Objects (ADO), an applications programming interface (API) for accessing and manipulating data in the Microsoft .NET Framework.

ADO.NET can be used in any consumer application that needs to connect to, and communicate with, any OLE DB-compliant data source, such as Microsoft SQL Server. The ADO.NET architecture enables you to build components that efficiently manage data from multiple data sources. In a disconnected scenario such as the Internet, ADO.NET provides the tools to request, modify, and reconcile data in multi-tier systems.

Whilst ADO.NET retains some of the key concepts evolved from previous ADO models, it has been largely expanded to provide access to structured data from diverse sources such as a flat text file, relational data from database management systems, or hierarchical XML data. All of this is accommodated in a consistent, standardized programming model.

This section is meant to introduce the key features of ADO.NET in brief, particularly on accessing data in relational database management systems (RDBMS). Accessing XML data will be the subject of a separate article in this series.

Managed Providers

The .NET framework data-access API provides two 'managed' providers for administering connections with a data source: the SQL Server Managed Provider and the ADO Managed Provider. A managed provider describes a collection of classes used to enable interaction with a data store (typically a database) in the managed runtime environment.

Each managed provider consists of the set of interfaces that implement access to a specific data source, using either the System.Data.SQL or System.Data.ADO class libraries respectively. The SQL classes are used to link directly to MS SQL Server data, while the ADO classes are for use with any OLEDB-compliant data provider.

  • The SQL Server Managed Provider is designed to work with Microsoft SQL Server (version 7.0 and later) databases. It uses a private protocol called Tabular Data Stream (TDS) to mediate communications directly between the server database and the representation of that data in the client application. It does not use OLE DB, ADO or ODBC.
  • The ADO Managed Provider is designed to work with the following OLE DB providers:
    Driver | Provider
    SQLOLEDB | SQL OLE DB Provider
    MSDAORA | Oracle OLE DB Provider
    JOLT | Jet OLE DB Provider
    MSDASQL/SQLServer ODBC | SQL Server ODBC Driver via OLE DB for ODBC Provider
    MSDASQL/Jet ODBC | Jet ODBC Driver via OLE DB Provider for ODBC Provider

To be able to use the managed providers that come with the .NET Framework, the following namespaces need to be included in the .aspx page.

For the SQL Server Managed Provider:

<%@ Import Namespace="System.Data" %>
<%@ Import Namespace="System.Data.SQL" %>

For the ADO Managed Provider:

<%@ Import Namespace="System.Data" %>
<%@ Import Namespace="System.Data.ADO" %>

The System.Data namespace consists mostly of the classes that constitute the ADO.NET architecture that are common to both providers. Together, these cover the entire set of interfaces for creating distributed, data-sharing .NET applications.

NOTE: While downlevel versions of MS SQL Server still are accessible via the ADO Managed Provider (System.Data.ADO namespace) using the SQLOLEDB or MSDASQL/SQLServer ODBC drivers (shown in above table), Microsoft specifies using the SQL Server Managed Provider (System.Data.SQL namespace) for versions 7.0 and later to increase overall performance. For particulars, see SQL Server 2000 Architecture.

Core ADO.NET

The following sections introduce the core ADO.NET components that are available in each managed provider for accessing data in relational database management systems:

  • Connections - for connecting to and managing transactions against a database.
  • Commands - for issuing SQL Commands against a database.
  • DataReaders - for directly reading data off a stream.
  • DataSets and DataSetCommands - for storing and manipulating data in a memory-resident data store.

Basically, the core ADO.NET functionalities may be summarized as follows:

A Connection object establishes the link between a Web page and a database. Commands are issued to the database provider and the result sets are returned in the form of streams that travel over these connections. The result sets can then be read by DataReaders for fast, efficient display of data, or be stored into memory-resident DataSet objects, which can then be manipulated by DataSetCommands, to let users access and manipulate the records in the DataSet. Developers can then use the methods built-in the DataSet to reconcile the DataSet information with the underlying data source.

Connections

Microsoft provides two Connection objects with the .NET Framework to establish a link to a specific data store: the SQLConnection and the ADOConnection. Connections can be opened explicitly by calling the Open method on the established connection. The following code snippets demonstrate how to establish and open a connection using each provider.

SQLConnection

String connectionString = "server=localhost;
            uid=sa; pwd=; database=northwind";
SQLConnection myConn = new
            SQLConnection(connectionString);
myConn.Open();

ADOConnection

String connectionString =
            Provider=Microsoft.Jet.OLEDB.4.0; Data
            Source=Northwind.mdb"
ADOConnection myConn = new
            ADOConnection(connectionString);
myConn.Open();

Note that while using the new constructor (when creating an instance of a class) is no longer a performance issue in .NET, which automatically handles object layout and references to objects, releasing them when they are no longer used, it still is good practice so as not to exhaust system limits while waiting for the instances to be garbage-collected.

Commands

Having established a Connection, you next need to be able to execute statements against the database. The simplest and most direct route for this is through the ADOCommand and SQLCommand objects.

A Command object is used to give an instruction to the provider on what to do with the information from a database. A command typically declares an SQL statement that either performs a select query to return a result set, an action query to update (add, edit, or delete) records from the database, or create and modify table structures. A command can also pass parameters and return values.

Typical examples are given in the table below:

Query | Example
Simple Select | SELECT * from Products WHERE ProductType='Boots';
Join Select | SELECT * from Orders O, Products P WHERE O.Product_ID=P.Product_ID;
Insert | INSERT into Employees VALUES ('123-45-6789', 'Doe', 'John', ' Development Lead ');
Update | UPDATE Employees SET Title='Development Lead' WHERE LastName='Doe';
Delete | DELETE from Employees WHERE Productivity < 10;

Commands can be explicitly stated, or issued as a call to a procedure stored in the database itself. The following code snippets demonstrate how to issue a select statement against the established connection.

SQLCommand

String SQLStmt = " SELECT * FROM
            Customers";
SQLCommand myCommand = new SQLCommand(SQLStmt,
            myConn);

ADOCommand

String SQLStmt = " SELECT * FROM
            Customers";
ADOCommand myCommand = new ADOCommand(SQLStmt,
            myConn);

DataReaders

When working with large amounts of data, holding memory open can become a performance issue. For example, attempting to read 1,000 rows out of a database causes a traditional ADO Recordset object to allocate and maintain memory for those 1,000 rows for the lifetime of the connection. If 1,000 users do this against the same machine at the same time, memory usage becomes critical.

To resolve this, the .NET framework includes the DataReader object, which is a read-only, forward-only stream returned from the database. Only one record at a time is ever present in memory.

The DataReader interface supports data from a variety of sources, such as relational and hierarchical data. DataReaders are appropriate when the need is to simply read the results after executing a command to a data source.

The following code snippets demonstrate how to declare a variable that refers to an instance of a DataReader object. The DataReader object will contain the results of the Command object when it is executed. The object in which the command will be used must have already been created and passed as an argument (parameter) when calling the Command object's Execute method. Continuing from the examples above:

SQLDataReader

SQLDataReader myReader = null;
myCommand.Execute(out myReader);

ADODataReader

ADODataReader myReader = null;
myCommand.Execute(out myReader);

You step through the DataReader in a very simple format:

While (myReader.Read()) {
// do your thing with the current row here
}

The below example shows in action what we have discussed so far: establish a connection to an SQL Server data source, issue a select command against the connection, instantiate a DataReader to contain the results of the command, then iterate thru the DataReader to render the data.

Show me

The following shows the complete code [C#] for the example above, using the Northwind database sample installed with SQL Server 7.0.

<%@ Import Namespace="System.Data"
            %>
<%@ Import Namespace="System.Data.SQL"
            %>

<html>
<head>
<title>Simple
            DataReader Sample</title>
<link rel="stylesheet"
            href="../../shared/demos.css">

<script language="C#"
            runat="server">
public SQLDataReader myReader;
public
            String html;

protected void Page_Load(Object Src, EventArgs E
            ) {
SQLConnection mySQLConnection = new
            SQLConnection("server=localhost;uid=sa;pwd=;database=northwind");
SQLCommand
            mySQLCommand = new SQLCommand("select * from customers",
            mySQLConnection);

try
            {
mySQLConnection.Open();
mySQLCommand.Execute(out
            myReader);

html="<h3>Simple DataReader
            Example</h3>";
html+="<Table>";
html+="<TR>";
html+="<TD><B>Customer
            ID</B> &nbsp;
            </TD>";
html+="<TD><B>Company
            Name</B></TD>";
html+="</TR>";

while
            (myReader.Read()) {
html+="<TR>";
html+="<TD>" +
            myReader["CustomerID"].ToString() +
            "</TD>";
html+="<TD>" +
            myReader["CompanyName"].ToString() +
            "</TD>";
html+="</TR>";
}
html+="</Table>";
}
catch(Exception
            e) {
html=e.ToString();
}
finally
            {
myReader.Close();
mySQLConnection.Close();
}
Response.Write(html);
}
</script>
</head>

<body>
</body>
</html>

Note that the actual rendering block is contained within the try ..catch statement. This provides some way to handle whatever exception may arise due to the connection. Code in the finally block is always executed, regardless of whether it is the try or the catch block that is run, so it becomes the logical place to close the reader and the connection.

Note also how the value of a field in the DataReader is easily accessed and rendered.

myReader["CustomerID"].ToString()

Now let's take that a step further. We can bind a SQLDataReader directly to a DataGrid, one of the available list-bound ASP.NET Web Forms controls. The DataGrid renders a table containing the SQL data. In this case we are only displaying data, so the forward-only nature of the DataReader is perfectly suited to this scenario, and we benefit from the performance boost that a DataReader provides.

Show me

The following shows the complete code [C#] for the example above, using the Pubs database sample installed with SQL Server 7.0.

<%@ Import Namespace="System.Data" %>
<%@ Import Namespace="System.Data.SQL" %>

<html>
<head>
<title>Binding a DataReader to a DataGrid Control</title>
<style>
body {font:10pt verdana}
</style>

<script language="C#" runat="server">
protected void Page_Load(Object Src, EventArgs E) {
SQLConnection myConnection = new SQLConnection("server=localhost;uid=sa;pwd=;database=pubs");
newSQLCommand("select String myQuery =
"select au_lname as LastName, au_fname as FirstName, Address, City, State, Zip from Authors";
SQLCommand(myQuery, myConnection);
SQLDataReader dr;
try {
myConnection.Open();
myCommand.Execute(out dr);
myDataGrid.DataSource = dr;
myDataGrid.DataBind();
}
catch(Exception e) {
Response.Write(e.ToString());
}
finally {
myConnection.Close();
}
}
</script>
</head>

<body>
<h3>Binding a DataReader to a DataGrid Control</h3>

<center>
<asp:datagrid id="myDataGrid" runat="server"
width="92%"
backcolor="#eee"
cellpadding=5
cellspacing=0
font-size="9pt"
headerstyle-backcolor="slategray"
headerstyle-forecolor="beige"
headerstyle-font-size="10pt"
headerstyle-height=25
/>
</center>

</body>
</html>

The DataGrid control supports a DataSource property that takes an Enumerable or Collection type. We can set this to an instance of the DataReader class, which contains the result of an SQL query. After setting the DataSource property, we call DataBind() to populate the control.

myDataGrid.DataSource = dr;
myDataGrid.DataBind();

For more information on binding ASP.NET list controls to a data source, please refer to Data Binding Expression Syntax. In the next section, we shall look at an entirely new interface included in ADO.NET for storing and manipulating data from a data store.

DataSets and DataSetCommands

The DataSet object represents an in-memory cache of data that behaves much like a database, containing tables, columns, rows, relationships, constraints, and data. The important distinction is that a DataSet can represent data in a consistent programming model regardless of the type of data source, be it data from a database, an XML file, from code, or user input.

Changes to the DataSet are tracked as they are made, much like tracking changes in a document. The DataSet includes a GetChanges() method that provides output in the form of another DataSet, which can then be used by DataSetCommands (or other objects) to update the source of the data.

Now let's see how this is done.

DataSet Basics

There are generally two ways to employ a DataSet object:

  • with an existing database management system
  • creating a new DataSet programmatically

In the first case, you use data from an existing database management system, such as SQL Server, to build and fill tables in the DataSet. This method is suitable when you need to obtain and manipulate data that is readily available in a database.

In the second case, you can programmatically create a DataSet, add DataTable objects to it, and then create DataRelation objects to link each table. The programmatic method is suitable when you need to create a DataSet at run time to capture data generated by an application.

In this article, we focus on the first method.

Using a DataSet With Existing Data

In a typical implementation, the basic steps would be to define the connection, construct a DataSetCommand, create an instance of the DataSet class, then build and fill each table in the DataSet with data from the source DBMS via the FillDataSet() method of the DataSetCommand.

Getting Data from a Database

The basic function of a DataSetCommand is to pull data off a data store and push it into a DataTable within the DataSet. To accomplish this, we create and pass it two arguments: the CommandText to execute (typically an SQL Select statement), and the Connection parameters.

SQLConnection myConnection = new SQLConnection("server=localhost;uid=sa;pwd=;database=pubs");
String myQuery =
"select au_lname as LastName, au_fname as FirstName, Address, City, State, Zip from Authors",Authors";
myConnection);
SQLDataSetCommand(myQuery, myConnection);

Next, we create and populate a DataSet with the results from the query by calling the FillDataSet() method of the DataSetCommand object:

   DataSet ds = new
            DataSet();
myCommand.FillDataSet(ds, "Authors");

The DataSet now holds the results of the query. In fact, the DataSet can hold results from multiple queries and even relate them, which we shall later see.

Notice that the FillDataSet method expects two arguments: the DataSet object, and the name of the table to fill in the DataSet. If that table does not exist, it is created for you.

Now let's see how the DataSet can be easily bound to a DataGrid control:

<%@ Import Namespace="System.Data" %>
<%@ Import Namespace="System.Data.SQL" %>

<html>
<head>
<title>Binding a DataSet to a DataGrid Control</title>
<style>
body {font:10pt verdana}
</style>

<script language="C#" runat="server">
protected void Page_Load(Object Src, EventArgs E) {
SQLConnection myConnection = new SQLConnection("server=localhost;uid=sa;pwd=;database=pubs");
String myQuery =
"select au_lname as LastName, au_fname as FirstName, Address, City, State, Zip from Authors",Authors";
SQLDataSetCommand myCommand = new SQLDataSetCommand(myQuery, myConnection);
DataSet ds = new DataSet();
myCommand.FillDataSet(ds, "Authors");

myDataGrid.DataSource = ds.Tables["Authors"].DefaultView;
myDataGrid.DataBind();
}
</script>
</head>

<body>
<h3>Binding a DataSet to a DataGrid Control</h3>

<center>
<asp:datagrid id="myDataGrid" runat="server"
width="92%"
backcolor="#eee"
cellpadding=5
cellspacing=0
font-size="9pt"
headerstyle-backcolor="slategray"
headerstyle-forecolor="beige"
headerstyle-font-size="10pt"
headerstyle-height=25
/>
</center>

</body>
</html>

Show me

As in the SQLDataReader sample above, we set the DataGrid's DataSource property, this time to the DataSet's DefaultView property.

myDataGrid.DataSource = ds.Tables["Authors"].DefaultView;
myDataGrid.DataBind();

The DefaultView represents the current state of the DataSet, including any changes that have been made by application code (row deletes or value changes, for example). In the same manner, after setting the DataSource property, we call DataBind() to populate the control.

Notice the absence of a call to open the connection. Unlike in the previous examples, the connection need not explicitly be opened when we use the DataSetCommand. It automatically handles that, which makes trapping for system exceptions (such as failed connections) less critical. In fact, one of the major performance enhancements the .NET framework provides is its managed execution environment, which, aside from administering code execution, provides runtime services that significantly improve code implementation, such as automatic resource optimization, runtime profiling, automatic memory management, and enhanced exception handling.

Still, though, you can always provide custom exception-handling routines to trap data-related errors at runtime (typically triggered via try..catch blocks). For simplicity, though, code for these routines will gradually be treated later in the series.

Note that while the data in the grid may seem no different from the earlier SQLDataReader sample, the main distinction is that once the DataSet is loaded, you can let users modify the data, and the DataSet will track the changes. More about that soon, but for now, let's look into some other ways to display the information from a DataSet.

Templated Controls

While the DataGrid is suitable for scenarios where a grid-like representation of data is appropriate, many times the presentation of data needs to be mu ch more flexible. ASP.NET offers two other controls, DataList and Repeater, which give the developer greater customization over the rendering of list-bound data. These controls have no default rendering of their own; the way data is rendered is completely determined by the developer's implementation of the control's templates, which describe how to present the data items.

The Repeater control just iterates over the bound data, rendering the ItemTemplate once for each item in the DataSource collection. It does not render anything besides the elements contained in its templates.

While the Repeater is suitable for general-purpose listings, the DataList provides added features for controlling the layout of the list. Unlike the Repeater, the DataList renders elements outside of the template definition, such as table rows, cells and spans containing style attributes, to enable richer formatting. For example, DataList supports RepeatColumns and RepeatDirection properties that specify how the ItemTemplates are laid out, i.e. whether data should be rendered in multiple columns, and in which direction (vertical or horizontal) to render the data items.

In ASP.NET Web Forms: An Overview - Part 2, we presented samples with complete source code demonstrating both controls, and in brief covered how to bind list control items to a data source. In the following section, we tackle a variation of using the DataList control.

Dynamic queries

Here we explore a method that can retrieve specific data sets based on a variable query constraint. This is extremely useful in cases where the query condition is initially unknown. This sample demonstrates dynamic queries in action, where the user decides on which records to display. Also here we look at how to modify the data displayed in a DataList using the value posted from an ASP.NET selectable dropdown list control.

Show me

This is made possible by providing the variable to the SQL query string that retrieves the data set.

String selectCmd = "select title_id, title, pub_id, type, price from Titles where type=@Type";

The SQLDataSetCommand maintains a Parameters collection that can be used to replace variable identifiers with values (denoted by a @ in front of the variable name). We add a new SQLParameter to this collection that specifies the name, type, and size of the parameter, and then set its Value property to the value of the select.

myCommand.SelectCommand.Parameters.Add(new SQLParameter("@Type", & _
SQLDataType.VarChar, 20));
myCommand.SelectCommand.Parameters["@Type"].Value = mySelect.Value;

Because the HtmlSelect control also supports a Collection DataSource property, we can use another select query to dynamically populate the select box, which guarantees the database and UI are always in sync. Below illustrates how to bind a data source to the HtmlSelect control. The data source in this case is a DataSet that is defined in the Page_Load event handler.

The id property of the HtmlSelect is set to mySelect. When the page loads the first time (not postback), a DataSet object containing the results of a select query is created, and then the select control's DataSource property is assigned to use this DataSet object. Finally, to bind the values in the DataSet to the control itself, the control's DataBind() method is called.

   if (!IsPostBack)
            {
SQLDataSetCommand myCommand = & _
new
            SQLDataSetCommand("select distinct Type from Titles",
            myConnection);
DataSet ds = new
            DataSet();
myCommand.FillDataSet(ds,
            "Types");
mySelect.DataSource =
            ds.Tables["Types"].DefaultView;
mySelect.DataBind();
}

Note that the above snippet must be enclosed within a script block that is run only at page load, and only if the load is not from a postback, because we need to do this only once. The following shows the complete code [C#] for the example above:

<%@ Import Namespace="System.Data" %>
<%@ Import Namespace="System.Data.SQL" %>

<html>
<head>
<title>Binding a Parameterized Select to a DataList Control</title>
<style>
body, div, table {font:10pt verdana}
</style>

<script language="C#" runat="server">
SQLConnection myConnection;

protected void Page_Load(Object Src, EventArgs E) {
myConnection = new SQLConnection("server=localhost;uid=sa;pwd=;database=pubs");

if (!IsPostBack) {
SQLDataSetCommand myCommand = new SQLDataSetCommand("select distinct Type from Titles", myConnection);
DataSet ds = new DataSet();
myCommand.FillDataSet(ds, "Types");
mySelect.DataSource = ds.Tables["Types"].DefaultView;
mySelect.DataBind();
}
}

public void GetTitles_Click(Object sender, EventArgs E) {
String selectCmd = "select title_id, title, pub_id, type, price from Titles where type=@Type";
myConnection = new SQLConnection("server=localhost;uid=sa;pwd=;database=pubs");
SQLDataSetCommand myCommand = new SQLDataSetCommand(selectCmd, myConnection);
myCommand.SelectCommand.Parameters.Add(new SQLParameter("@Type", SQLDataType.VarChar, 20));
myCommand.SelectCommand.Parameters["@Type"].Value = mySelect.Value;
DataSet ds = new DataSet();
myCommand.FillDataSet(ds, "Titles");
myDataList.DataSource = ds.Tables["Titles"].DefaultView;
myDataList.DataBind();
}
</script>
</head>

<body>
<h3>Binding a Parameterized Select to a DataList Control</h3>

<center>
<form runat="server">

<p>Select a subject category:

<select id="mySelect" datatextfield="Type" runat="server"/>

<input type="submit" OnServerClick="GetTitles_Click" value="Get Titles" runat="server"/><p>

<asp:datalist id="myDataList" repeatcolumns="2" runat="server">

<template name="itemtemplate">
<table cellpadding=10>
<tr>
<td valign="top">
<img align="top" src='<%# DataBinder.Eval(Container.DataItem,
"title_id", "/devguide/shared/images/title-{0}.gif") %>' >
</td>

<td valign="top">
<b>Title:</b> <%# DataBinder.Eval(Container.DataItem, "title") %><br>
<b>Category:</b> <%# DataBinder.Eval(Container.DataItem, "type") %><br>
<b>Publisher ID:</b> <%# DataBinder.Eval(Container.DataItem, "pub_id") %><br>
<b>Price:</b> <%# DataBinder.Eval(Container.DataItem, "price", "$ {0}") %>

<p><a href='<%# DataBinder.Eval(Container.DataItem, "title_id",
"purchase.aspx?titleid={0}") %>' >
<img border="0" src="/devguide/shared/images/purchase_book.gif"></a>
</td></tr>
</table>

</template>

</asp:datalist>

</form>
</center>

</body>
</html>

Now let's see a similar version this time using System.Data.ADO classes linked to the AdvWorks sample database installed with MS Access 2000.

Show me

The logic is generally the same, except of course in the class declarations and in the connection string.

ADOConnection myConnection = new ADOConnection("Provider=Microsoft.Jet.OLEDB.4.0;
Data Source=" + Server.MapPath("/devguide/shared/data/dbtutor.mdb"));

Note the use of Server.MapPath to expose the full path of the data source's location. This is only necessary, as in this case, when the database is stored in a different folder from where the requesting page is. Note also that because the images URLs are part of the DataSet (as values stored in the ProductImageURL field), we can easily bind the source of each record's image as follows:

<img align="top" src='<%# DataBinder.Eval(Container.DataItem, "ProductImageURL") %>' >

Note also how databinding expressions can simply be laid out with inline text (such as Code: and the 'US' characters before the $ sign). Also, HTML formatting is done via Cascading Style Sheets (CSS). This way, the presentation can flexibly be controlled; varied element styles (such as font, color, size, etc.) can easily be applied by simple changes to the style sheet. Below shows the complete code for the sample above.

<%@ Import Namespace="System.Data" %>
<%@ Import Namespace="System.Data.ADO" %>

<html>
<head>
<title>Binding a Parameterized Select to a DataList Control</title>
<style>
body, div, table {font:10pt verdana}
.ordcode {font:bold 9pt arial; color:navy}
</style>

<script language="C#" runat="server">
ADOConnection myConnection;

protected void Page_Load(Object Src, EventArgs E) {
myConnection = new ADOConnection("Provider=Microsoft.Jet.OLEDB.4.0;Data Source="
+ Server.MapPath("/devguide/shared/data/dbtutor.mdb"));

if (!IsPostBack) {
ADODataSetCommand myCommand = new ADODataSetCommand("select distinct ProductType from Products", myConnection);
DataSet ds = new DataSet();
myCommand.FillDataSet(ds, "Types");
mySelect.DataSource = ds.Tables["Types"].DefaultView;
mySelect.DataBind();
}
}

public void GetProducts_Click(Object sender, EventArgs E) {
String selectCmd = "SELECT ProductType, ProductCode, ProductName, ProductDescription, ProductImageURL, UnitPrice FROM
Products where ProductType=@ProductType";
myConnection = new ADOConnection("Provider=Microsoft.Jet.OLEDB.4.0;Data Source="
+ Server.MapPath("/devguide/shared/data/dbtutor.mdb"));
ADODataSetCommand myCommand = new ADODataSetCommand(selectCmd, myConnection);
myCommand.SelectCommand.Parameters.Add(new ADOParameter("@ProductType", ADODBType.Char));
myCommand.SelectCommand.Parameters["@ProductType"].Value = mySelect.Value;
DataSet ds = new DataSet();
myCommand.FillDataSet(ds, "Products");
myDataList.DataSource = ds.Tables["Products"].DefaultView;
myDataList.DataBind();

if (IsPostBack)
footer.InnerHtml =
"<hr width=92% size=1 color='red'><small>Catalog photos courtesy of Recreational Equipment Incorporated.
<br>&copy; 2000 Rey Nu&ntilde;ez and Adventure Works</small>";
}
</script>
</head>

<body topmargin=0 leftmargin=0 marginwidth=0 marginheight=0>
<center>
<form runat="server">
<table>
<tr>
<td><img src="../../shared/images/advworks/aw_logo.jpg"
width=344 height=146 alt="Gear Up with Adventure Works!" border=0></td>
<td align="center">
<p><small>Adventure Works gear has been tested in the most extreme environments on earth,
from the 8,000-meter peaks of the Himalayas to the sub-arctic giants of Alaska.</small></p>

<p>Select Gear: <select id="mySelect" datatextfield="ProductType" runat="server"/>

<input type="submit" OnServerClick="GetProducts_Click" value="Get Gear" runat="server"/></td></tr>
</table>

<hr width=92% size=1 color="red">

<asp:datalist id="myDataList" repeatcolumns="2" runat="server">

<template name="itemtemplate">
<table cellpadding=10>
<tr valign="top">
<td><img align="top"
src='<%# DataBinder.Eval(Container.DataItem, "ProductImageURL") %>' >
</td>
<td>
<b><%# DataBinder.Eval(Container.DataItem, "ProductName") %></b><br>
<%# DataBinder.Eval(Container.DataItem, "ProductDescription") %>
<p class="ordcode">Code:
<%# DataBinder.Eval(Container.DataItem, "ProductCode") %>
US<%# DataBinder.Eval(Container.DataItem, "UnitPrice", "$ {0}") %></p>
</td></tr>
</table>

</template>

</asp:datalist>

<p id="footer" runat="server"></p>
</form>
</center>

</body>
</html>

Sorting Data

A common requirement in many applications is the ability to sort the order in which the data is displayed. While the DataGrid does not include a method to explicitly sort its data, it does provide a way to call a handler that may then be coded to change the sorting of the data bound to the grid.

Data in a grid is usually sorted by clicking the header of the column you wish to sort. To enable sorting in a DataGrid, you set AllowSorting=true. When enabled, the grid renders LinkButtons in the header for each column. When any button is clicked, the grid's OnSortCommand event is thrown, which you can then easily handle in your code. This example shows how to implement basic sorting in a DataGrid.

<form
            runat="server">
<asp:DataGrid id="myDataGrid"
            AllowSorting="true"
OnSortCommand="myDataGrid_Sort"
            runat="server" />
</form>

We define an event handler method (herein named myDataGrid_Sort) that we delegate to receive control when the OnSortComand fires. Since a DataGrid always displays the data in the same order it occurs in the source, the typical logic is to sort the data source, then rebind the data to the grid.

Below shows a portion of the method that issues a Sort command back to the grid. When the client clicks a column link, the name of the column is passed as a SortField property on the DataGridSortCommandEventArgs parameter, which we then use to set the Sort property of the data bound to the grid.

<script>
protected void myDataGrid_Sort(Object Src, DataGridSortCommandEventArgs E) {
...
DataSet ds = new DataSet();
myCommand.FillDataSet(ds, "Authors");
DataView myDataView = ds.Tables["Authors"].DefaultView;
myDataView.Sort = sortfield;
myDataGrid.DataSource = myDataView;
myDataGrid.DataBind();
}
</script>

Notice here that we use an instance of a DataView class instead as the DataSource of the grid. The DataView represents a bindable, customized view of a DataTable for sorting, filtering, searching, editing, and navigation.

Show me

Below is the complete source [C#] for the sample above, using different views of the Authors table in the Pubs database.

<%@ Import Namespace="System.Data" %>
<%@ Import Namespace="System.Data.SQL" %>

<html>
<head>
<title>Sorting Data in a DataGrid Control</title>
<style>
body {font:10pt verdana}
</style>

<script language="C#" runat="server">
SQLConnection myConnection;

protected void Page_Load(Object Src, EventArgs E) {
myConnection = new SQLConnection("server=localhost;uid=sa;pwd=;database=pubs");
if (!IsPostBack) BindGrid("AuthorID");
}

protected void myDataGrid_Sort(Object Src, DataGridSortCommandEventArgs E) {
BindGrid(E.SortField);
}

public void BindGrid(String sortfield) {
String myQuery = "select au_id as AuthorID, au_lname as LastName, " & _
" au_fname as FirstName, Address, City, State, Zip from Authors";
SQLDataSetCommand myCommand = new SQLDataSetCommand(myQuery, myConnection);
DataSet ds = new DataSet();
myCommand.FillDataSet(ds, "Authors");
DataView myDataView = ds.Tables["Authors"].DefaultView;
myDataView.Sort = sortfield;
myDataGrid.DataSource = myDataView;
myDataGrid.DataBind();
}
</script>
</head>

<body>
<h3>Sorting Data in a DataGrid Control</h3>

<center>
<form runat="server">

<asp:datagrid id="myDataGrid" runat="server"
OnSortCommand="myDataGrid_Sort"
backcolor="#eee"
cellpadding=5
cellspacing=0
font-size="9pt"
headerstyle-backcolor="slategray"
headerstyle-forecolor="beige"
headerstyle-font-size="10pt"
headerstyle-height=25
allowsorting="true"
/>

</form>
</center>

</body>
</html>

Well, that's about it for this session. The samplers presented in this section involve simple, yet powerful features that can be used in a variety of data-based apps. The pages can serve as reusable templates that can be adopted for most uses, simply by changing the data source. Of course, they can further be enhanced, but I wouldn't want to spoil the fun, so I'll leave that up to you.

In .NET Data Access 101: Part II, we shall cover how to let users add, edit, and delete information in a DataSet, check for errors, and merge the changes from the DataSet back into the data source.

Download

If you already have the download file provided in the ASP.NET Web Forms article, this file would update the existing set-up installed in your local drive. If not, you simply get the added bonus of that package here.

Related Links:

System.Data.ADO Namespace
http://msdn.microsoft.com/library/dotnet/cpref/frlrfSystemDataADO.htm

System.Data.SQL Namespace
http://msdn.microsoft.com/library/dotnet/cpref/frlrfSystemDataSQL.htm

Data Binding Expression Syntax
http://msdn.microsoft.com/library/dotnet/cpguide/cpcondatabindingexpressionsyntax.htm

Cutting Edge: Server-side ASP.NET Data Binding
http://msdn.microsoft.com/library/periodic/period01/cutting0103.htm

Data Binding Expression Syntax
http://msdn.microsoft.com/library/dotnet/cpguide/cpcondatabindingexpressionsyntax.htm?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq

C#Today - © 2002 WroxPress Limited


Web Site Error Logging and Reporting System

SOAP, Data Transfer

SOAP, Data Transfer (Sort by Name)

C# and Java Remoting Part 1 - a C# Remoting Primer
C# Web Component Communication in .Net using Events - Part I
COM+ and C#
Cool Tool for XSLT - Part 1
Cool Tool for XSLT - Part 2
Creating an ISAPI FIlter HTTP Packet Analyzer
Data Transformation Engine
Debugging Remoting Transactions
Interoperating Java, VB6, and .NET Web Services
Network Programming in C#

SOAP Support in .NET
Transforming Formats with XSLT and .NET Object
Using Remote Access Services in C#
Working with Web Data in C#
Writing a Messenger Application Using C# - Part 1
Writing a Messenger Application using C# - Part 2


Data Transformation Engine

Data Transformation Engine
http://www.csharptoday.com/content.asp?id=1853
Article Rating: 5
Published on August 12, 2002
Abstract
Numerous software applications require data to be imported from an external source and translated into a format that the application knows how to deal with. In this case study, Catalin Tomescu explains the problems he encountered while working on a real-world project some time ago. The application was the direct need of an insurance company to import service contracts from different clients into their database. The format of the data sent by these clients was very different depending on the system they had implemented at their end. Some clients sent the data in XML format, others in CSV file, etc. Because of this, Catalin's team was forced to create a system that would allow them to import the data into the database. In this case study he’ll show you the solution to this problem. We’ll look at the framework for data transformations that was built which is capable of taking data from various sources and transforming it into a wide range of types.
Case Study

At some point in its life cycle, all software applications will need to process some kind of data. Data can represent any information that the application needs to process in order to fulfill the goal it was built for. Unless this data is internally generated by the software application itself, it must be imported from an external source and translated into a format that the application knows how to deal with.

To be more specific, we'll explain the problems we encountered while working on a project some time ago. The application we were building was a direct need of an insurance company to import service contracts from different clients into their database. The format of the data sent by these clients was very different depending on the system they had implemented at their end. Some clients agreed to send the data in XML format, others in CSV file or fixed width files, others using files with a proprietary format. Because of this, we were forced to create a system that would allow us to import the data into our database. We started to use DTS, but soon realized that as the number of clients grew, so would the range of data formats we would have to deal with. DTS was far from the perfect solution, so we had to search for another solution.

In this case study we'll show you a solution to this problem. We'll show you a framework for data transformations that we built and which is capable of taking data from various sources and transforming it into a wide range of types.

Introduction

Data transformation involves a source and one or more destinations. The format of the data in the source and destinations can be a text file (CSV, fixed width, or proprietary format), a file containing records with different format, an XML file, data existing in a database, or anything else. Turning the source data to the specific format that an application needs it in can be a painful job sometimes. Often this task requires developers to write a transformer that will make the translation from the source format to the destination format.

This is not a new problem, and this is not the first attempt to solve it. Let's examine one of the existing solutions to the problem. With version 7.0 SQL Server comes a package called Data Transformation Services that was specifically created to solve these kinds of problems. Data Transformation Services (DTS) provides a set of tools that let you extract, transform, and consolidate data from disparate sources into single or multiple destinations supported by DTS connectivity. By using DTS tools to graphically build DTS packages, or by programming a package with the DTS object model, you can create custom data movement solutions tailored to the specialized business needs of your organization. A DTS package is an organized collection of connections, DTS tasks, DTS transformations, and workflow constraints assembled either with a DTS tool or programmatically.

For most situations, DTS is sufficient, but there are situations where the problem is much more complex and DTS is not able to address it. For example, one problem is that there are no connectors to XML files or files with formats other than CSV or fixed width.

To overcome the situations when DTS can't be used, we designed and build a Data Transformation Engine that will address existing issues.

System Requirements

To compile and run the code included in this article you need:

  • Visual Studio .NET Professional or any other edition
  • SQL Server 7.0 or 2000 can be used if you want to test the database connector. You can also choose the lighter version of SQL Server, which is MSDE. MSDE is distributed free by the Microsoft Corporation and can be downloaded from their web site.
  • Any XML editor or a simple text editor for creating XML files

The case study also assumes you have a fairly good understanding of the C# language and .NET Framework. Experience with XML, OO principles and design patterns are a plus in understanding this piece.

Defining The Problem

Whether we are talking about financing solutions or engineering packages, both processes need some kind of data to be able to produce results. The driving source of information is financial statements, accounting statements, or sensor readings. Very often this data it's not available in exactly the desired format and requires some kind of transformation. Financial statements can be transmitted via the Internet in comma-separated text files, it can be extracted from a database, or it can be sent as an XML file through a web service.

As I mentioned earlier, a possible solution for this problem would be to use Data Transformation Services available as a package in SQL Server 7.0 and above. We could use DTS in this way:

  • Create a package in SQL Server that will import the specified source data into the specified destination. Each DTS package contains one or more steps that are executed when the package is run. When executed, the package connects to the correct data sources, copies data and database objects, transforms data, and notifies other users or processes of events.
  • Run the package manually or automate this task through SQL Task Scheduler.

DTS is a good choice for solving simple and medium difficulty problems. For example, the number of formats available in DTS for describing data sources and data destinations does not cover some of the new technologies like XML. Another drawback is that DTS requires SQL Server to be present on the computer in order to store and schedule DTS packages. Earlier versions of DTS were even more restrictive in respect to the tasks that were available to use in a package.

Defining The Solution

Data Transformation Engine, the framework that we designed, expands the list of possible data types for describing sources and destinations available in most of the existing solutions. In addition to the standard list (CVS, fixed width files, databases, Excel) DTE is capable of creating and reading XML files, files with multiple types of records.

What is more important about DTE is that it was designed to allow new sources and destinations to be created and plugged into the system very easily. This allows us, or any other developer, to add data sources as they are needed.

Component Design and Coding

This section will focus on explaining how the DTE framework was designed, as well as going through implementation details. The implementation is entirely done using C# language, but because .NET is a cross language environment you can choose to rewrite parts of the application in your language of choice. Similarly, extending this framework can be done in any of .NET's supported languages.

The DTE solution contains four projects:

The DTE project contains the controlling objects in the framework.

The DTECommon project contains interface definitions. We chose to separate the interface definitions from the rest of the framework implementation so that it can be used in extending the set of data sources and destinations. This way we can keep the size of the library very small. More details about this will be given in the next sections.

The DTEStdDataFactory project contains the implementation for most used data sources and destinations. More implementation details about this will be given in the next sections.

The Test project contains the source code of the example application included in this case study.

Design Decisions

When we designed the DTE we started with analyzing a transformation process. Basically what happens is that source records of data are transformed into destination records and pushed to the data destination. This is the first design decision we made for our framework:

Data sources and destination will treat data as records composed of fields.

We used this approach because the practice has proved that in most of the cases the data will be seen as records of information that will be accessed sequentially for either reading or writing.

This process is captured in the following figure:

We designed DTE starting with this simple idea. The central part of the DTE is the processing engine. The processing engine controls the flow of transformation tasks and has the following major tasks:

1.Read transformation descriptor file

2.Create source and destination(s) objects based on the information in the descriptor

3.Open source and destination(s)

4.While records exist in source

...4.1Decrypt raw data and create source record

...4.2Choose destination record

...4.3Perform data transformation using field mapping matrix

...4.4Write destination record

5.Close data source and data destination(s)

Though the overall design is good, it comes with a limitation: if a client application needs to process only a subset of the data in the source storage, DTE will still parse the whole set of source data. Implementing a filter mechanism is relatively easy with databases, but becomes more complicated with text files. DTE allows client application to select a subset of records from the source database but has no provision for filters when it comes to text files.

We mentioned in the previous sections that the DTE framework is able to deal with a wide range of sources and destinations. To achieve this, we used a key element in the design of DTE. In the following section you'll see how we made use of design patterns to achieve a flexible, extensible, and reusable set of components.

Managing Sources and Destinations

The second design decision we made was:

Use the Abstract Factory pattern to manage sources and destinations.

This allows us to maintain the same interface in the processing engine while extending the types of sources and destinations available to users. The Abstract Pattern allows the DT Engine to create instances of objects from different families of sources and destinations without specifying their concrete classes.

The architecture of the DT Engine with respect to this topic is outlined in the following UML class diagram:

The implementation of the Abstract Factory pattern uses interfaces and inheritance. Of course, each programming language has its own ways to create or simulate interfaces and deal with inheritance.

As you can see from the above figure, we defined a set of interfaces for data source, data destination and corresponding records.

From these interfaces we'll derive concrete classes for each type of source/destination we decide to implement. By default, the DTE framework comes with the following sets of hierarchies: CVS, Fixed Width Fields, Database, XML and Record Based Files. Each of these packages is contained in a separate C# namespace to be manipulated easily. We'll discuss details of their implementation later.

The breakdown in physical packages is shown in the following UML structure diagram:

The DTEApplication object is designed to use the IDataFactory interface to create data sources, data destinations, and record objects. To be able to better understand how this works, let's get into the implementation details of the DTEApplication object.

DTE application

DTEApplication is the main controlling object of the framework and it is located in the DTE project. The main purpose of this object is to control the flow of the transformation process. The transformation process can basically be broken down into three phases: initialization, transformation process, and cleanup. This can easily be seen by looking at the StartTrxProcess method of the object.

The Initialization phase consist of:

  • Read transformation descriptor file
  • Create data source and destinations
  • Assign initialization properties to source and each destination object

Client applications describe a transformation process by writing a descriptor file, which must be in XML format, and have following structure:

<?xml version="1.0"?>
<transformation>
<source mode="" name="">
<property name="Data Source"> </property>
<property name="Configuration File"> </property>
</source>

As you may already have guessed, source element describes the source of data used in the transformation process. The source element contains all the attributes that are necessary to locate and use a file, database table, or any other type of source. Mode and name attributes are mandatory and the framework will generate an error if they are not present:

  • The Mode attribute defines the type of data source. Valid values for this attribute are defined in the DataFactories section of the DTE settings file. Here is the current list: CHAR_SEPARATED_VALUES, FIXED_WIDTH, XML, DATABASE, and RECORD_BASED.
  • The Name attribute must be uniquely defined among source and destination names. This is just an identifier and doesn't need to be tied to the actual source.

The Property element defines a specific property for the data source. There are two mandatory properties for any data source: Data Source and Configuration File.

  • Data Source property can be:
  • The name and path to the source file. You can have something like this: \\machinename\folder\mysource.txt or c:\myfiles\source.txt.
  • A SELECT SQL statement that gathers records from one or more tables.
  • Anything else that describes the source of data in your specific data type. For example, I can imagine someone writing a connector to access registry information in which case this property contains a registry key.
  • Configuration File property represents the path to a file that contains the descriptor for specified data source. We chose not to include the descriptor in the same XML file so that you can reuse the source descriptor file somewhere else. The descriptor is specific for each source and we'll discuss them later.

The software allows a programmer to extend this list of properties with additional items as they are needed by a data source. Such an example is the Database connector, which defines a new property called "Connection String" to capture the information required to access data in a database.

<destination mode="" name="">
<property name="Data Source"> </property>
<property name="Configuration File"> </property>
</destination>

A destination element is used to describe a data destination. A unique feature of DTE is that a transformation process may contain multiple destinations. Imagine the following example: a file with POS sales and returns that needs to be imported in a database. Sales must go to the Sales table and returns must go to the Restock table. This process will contain two destination elements, one for each table. All attributes and property tags have the same meaning, as for the source.

<client>
<assembly_name>MyClient.dll</assembly_name>
<object_name>MyClient.MyProcesserEvents</object_name>
</client>

The Client element is used to indicate a notification object. This notification object is called during the transformation process to give a chance for the client to dynamically make decisions at run-time. The notification object must implement the IDTENotifications interface that is defined as follows:

public interface IDTENotifications
{
bool CheckProperties (string sDataName, ref Hashtable colProperties);
string GetSrcRecordIDFromRawData (object RawData);
string GetDestRecordID(ISourceRecord srcRecord);
bool BeforeApplyTransformation (ISourceRecord srcRecord);
bool BeforeComit(ref IDestinationRecord destRecord);
}

The CheckProperties method allows client objects to modify properties before they are applied to the source or destination objects.

The GetSrcRecordIDFFromRawData method is called when the source supports multiple record types and the data source is not capable of automatically determining which record type to use. This method is defined for future versions because current versions of the framework support only sources with records that can be automatically determined.

The GetDestRecordID method has the same meaning as above, but for data destinations.

The BeforeApplyTransformation method is called just before the DTE populates the destination columns from the source record. Calling this method gives the client object a chance to modify the source fields, perhaps to apply more transformations.

The BeforeCommit method is called after applying transformations, but before persisting the destination record.

<mappings source_rec_id=" " dest_rec_id="
            " dest_name=" ">
<map destination_column_index="0"
            source_column_index="0"/>
<map destination_column_index="1"
            source_column_index="2"/>
<map destination_column_index="2"
            source_column_index="1"/>
<map destination_column_index="3"
            source_column_index="4"/>
<map destination_column_index="4"
            source_column_index="3"/>
</mappings>
</transformation>

The Mappings element describes the transformations that need to be applied between source record and destination record. There could be more than one mappings element depending on how many destination records exist in each destination:

  • Source_Rec_ID attribute represents the id of the source record and this should match one of the record ids in the data source descriptor file.
  • Dest_Rec_Id attribute represents which destination record to use when making the mapping.
  • Dest_Name represents the name of a destination defined in a Destination element.

A Mappings element contains a list of Map elements that indicates a field-to-field correspondence. The source and destination elements can be indicated using indexes or names. However, the use of names comes with a performance trade-off, as there is an additional function call to determine the index of specified column name in the array of filed definitions. The function that accomplishes that is named GetFieldIndexByName and you'll find it included in each record implementation.

Here is the source code of this function:

private int GetFieldIndexByName (string
            sFieldName)
{
for (int i=0; i<_fieldInfo.Length;
            i++)
{
if (_fieldInfo[i].Name.Equals(sFieldName))
return
            i;
}

return -1;
}

The DTEApplication object uses the DTETrxDescriptorReader object to read a transformation descriptor. The Load method takes a file name as a parameter and searches for each key element in the XML document. To process mapping elements, the Load methods uses a FieldMappingReader object to create a FieldMappingCollection object that is the internal representation of a XML mappings element.

The implementation of these two classes is fairly straightforward. We use the System.Xml namespace to parse an XML file and use XSLT to find a particular XML node or collection of nodes. To achieve this, we use the XmlDocument.SelectSingleNode and XmlDocument.SelectNodes methods respectively. Both methods take an XSLT search path as a parameter.

Here are few code excerpts:

XmlNodeList list = root.SelectNodes("destination");
XmlNodeList mapList = root.SelectNodes("mappings");
node = root.SelectSingleNode("client");

Now that we've seen what the initialization step involves, let's move to the next phase: create data sources and destinations. Once the descriptor is parsed, DTE knows what type of data sources and destinations are involved in the process and can create an instance of each.

As we mentioned earlier, DTE uses the IDataFactory interface to create the source and destination objects from specific data factories. The first step is to obtain a reference to the data factory corresponding to each source and destination. To do that DTE calls a private method GetDataFactoryFromMode that takes a string parameter representing the type source or destination to retrieve the factory for. A DTE configuration file maintains a list with active data factories and it can be updated with new data factories as they are available to DTE. The structure of this configuration file is:

<DataFactories>
<DataFactory mode="">
<Assembly> </Assembly>
<Class> </Class>
</DataFactory>
</DataFactories>

One or multiple DataFactory elements describe an active factory by defining a corresponding assembly and class. When a DTEApplication object instance is created, it loads the information in this file and makes it available to the transformation descriptor reader object.

Here is the source code for the GetDataFactoryFromMode method:

private IDataFactory GetDataFactoryFromMode
            (string mode)
{
IDataFactory factory = null;
FactoryInfo
            factoryInfo = null;

try
{
factoryInfo =
            _factoryReader.GetFactoryInfoByMode(mode);
_assembly =
            Assembly.Load(factoryInfo.Assembly);
factory =
            (IDataFactory)_assembly.CreateInstance(factoryInfo.Class);
}
catch (Exception e)
{
Trace.WriteLine(e);
factory =
            null;
}

return factory;
}

After DTE creates data sources and destinations, the next step is to assign properties read from the descriptor to each source and destination. DTE first notifies the client object, if it exists, that it is about to assign properties, and if any changes are made to the properties they are captured before.

Here is the source code from the private TrxInitialization method:

Hashtable tmp;
if (_descriptorReader.NotificationInterface != null)
{
for (int i = 0; i < _descriptorReader.DestinationDescriptors.Count; i++)
{
tmp =
((DataTrxDescriptor)_descriptorReader.DestinationDescriptors[i]).GetAllProperties();

// if user changes any of the properties send new properties back to
// the descriptor to be processed.
if (_descriptorReader.NotificationInterface.CheckProperties(
((DataTrxDescriptor)_descriptorReader.DestinationDescriptors[i]).GetName(),
ref tmp))
((IDataDestination)_descriptorReader.Destinations[((DataTrxDescriptor)_descriptorReader.
DestinationDescriptors[i]).GetName()]).SetProperties(tmp);
}

tmp = ((DataTrxDescriptor)_descriptorReader.SourceDescriptor).GetAllProperties();

if (_descriptorReader.NotificationInterface.CheckProperties(
((DataTrxDescriptor)_descriptorReader.SourceDescriptor).GetName(), ref tmp))
_descriptorReader.Source.SetProperties(tmp);
}

This was the last step in the initialization phase, and DTE has now got all the information necessary to step into the second phase: the actual transformation process. But before going into this second phase, we'll walk you through the design and implementation of standard types of data sources and destination.

Standard Data Sources and Destinations

DTE is shipped with a set of the most-used types of data sources and destinations: CSV files, fixed width files, XML files, database (only SQL Server and MSDE are supported), and a record base file. In DTE terminology we call them "connectors". We chose these connectors because:

  • CSV and fixed width are the most used types of text files.
  • XML is a new technology that has gained a lot of popularity in the industry, and so there is a very good chance that you'll need to work with such files.
  • Database is probably the best choice when it comes to manipulating large amounts of information.
  • Record based files are also a very popular choice mostly in legacy systems like mainframe and AS/400.

Of course we could have added a connector for Excel and other popular Office types of files, but this is a relatively easy task given the standard set.

Text based connectors like CSV, Fixed width and Record Based File are designed using a basic type of file that is capable of handling records of information stored in fields or columns. In this section we'll show you the design for a single connector (CSV Connector) and walk you through the implementation of the others. The design for all other connectors is identical and should not pose too many difficulties in this case study.

The architecture of the CSV Connector is described in the following UML Class Diagram:

Let's analyze for a second this architecture. As you can see, we use the interfaces defined earlier, IDataSource and IDataDestiantion, to create sources and destinations, and ISourceRecord and IDestinationRecord for corresponding records. If we take a closer look at IDataSource and IDataDestination, one may say they are almost identical so we could have put all the code in the same class and spare a class definition. That might be true, but we thought that in 99% of the cases the data source and destination are different types so, loading a class instance with both implementations when we only need one part, source or destination, doesn't make sense.

This separation is also beneficial from another perspective. If a client application doesn't need, let's say, a destination (e.g. in a scenario with a mainframe source where we only export data), the developer will choose to implement only a MainframeSource class to only extract data from mainframe. The same consideration applies in the design of records, where we separate the source record and destination record classes.

The BaseFile is a class that prolongs the abstraction of a text file in our architecture. This design was preferred over the use of .NET's TextReader and TextWrites classes because we might want to change the foundation of handling files in the future. The implementation is simple enough and does not require any additional comments.

The BaseFileWithFields class is designed as a placeholder for a collection of fields with string values. This class is primarily used by corresponding record objects in text file connectors and is necessary to reduce the duplication of code in each concrete record object. The implementation is simple and includes methods to load and get the value of a field or for a collection of fields.

CSV Connector

The CSVSource class implements the IDataSource interface and derives from the BaseFile class. Let's analyze the source code of this class:

/// <summary>
/// CSVSource is the
            implementation of IDataSource for CVS connector.
///
            </summary>
public class CSVSource : BaseFile, IDisposable,
            IDataSource
{
private string _sFileName,
            _sConfigFileName;
private ISourceRecord _record;
private
            string _recordID = null;
private bool _eof = false;

public
            CSVSource()
{
_record = new
            CSVSourceRecord();
}

The class constructor creates a new instance of the CSVSourceRecord that will be used to handle records from a file with character separated value records.

Below is the implementation for methods in the IDataSource interface:

// IDataSource methods
public bool MultipleRecordTypesSupported ()
{
return false;
}

Because this type of file has only one type of record type, the MultipleRecordTypesSupported method returns false.

public bool LoadDescriptor (string DescriptorFile)
{
return _record.LoadDescriptor(DescriptorFile);
}

The LoadDescriptor method delegates the call to the record object that in fact is able to digest the information in the descriptor file. Some of the information might be useful to the source object, but most of it helps the record object to parse raw data.

Let's analyze a sample descriptor file:

<?xml version="1.0"?>
<descriptor type="CHAR_SEPARATED_VALUES" separation_char="|" rec_id="ReturnDst">
<column name="TRANSACTION_ID"/>
<column name="SKU_NUMBER"/>
<column name="QUANTITY"/>
</descriptor>

As you may notice, the type of the descriptor is set to CHAR_SEPARATED_VALUES, which indicates that this is a CSV descriptor. The separation_char attribute indicates the character that will be used to split the raw data of information, a pipe ("|") character in this example. The record_id attribute defines this record's identifier. This attribute is very important as it plays a critical role in defining field mappings. The three column elements define the fields in this sample CSV file:

public bool MoveNext()
{
bool ret =
            false;

if (!_eof && base.ReadLine())
ret =
            _record.ParseFieldsFromRawData(base.line);
else
_eof =
            true;

return ret;
}

MoveNext as the name suggests, is used to move to the next record in the data source. In this case we read lines from the source file as long as the end of the file is not reached. To advance to the next line we call the ReadLine method on the base class BaseFile. If we get a positive result, we call the ParseFieldsFromRawData method on the record object, which populates each field in the source record with the value in the file.

public new bool Close()
{
return base.Close();
}

Close delegates the call to the BaseFile. As in the case of LoadDescriptor we use the base keyword to call methods on the base class. The Base keyword is a new feature in .NET's C# language where only one base class is allowed per class declaration. This restriction makes it safe to identify the base class in the current inheritance operation.

public bool Open()
{
bool ret =
            true;

if (LoadDescriptor(_sConfigFileName) == false)
ret =
            false;
else
{
ret = Open(_sFileName,
            OpenTypeEnum.OT_READ);
if (ret)
_recordID =
            _record.GetRecordID();
}

_eof = !ret;

return
            ret;
}

The Open method is a good place to load the data source descriptor. If the call is successful, we use the Open method on the BaseFile class. If this operation succeeds, the record name is stored in a local variable.

public void SetProperties (Hashtable props)
{
_sFileName =
            (string)props[DTEStaticPropertiesEnum.DTE_PROP_DATASOURCE];
//
            file name
_sConfigFileName =
            (string)props[CSVConnectorPropertiesEnum.CSV_PROP_CONFIG_FILE];
// configuration file
}

If you remember when we discussed data properties, the final step in the process of assigning properties to source and destinations was to call the SetProperties method to achieve this goal. The implementation stored the value of two important properties in local variables for later use.

public string GetCurrentRecordID
            ()
{
return _recordID;
}

public void
            SetCurrentRecordID (string recordID)
{
_recordID =
            recordID;
}

public ISourceRecord
            GetCurrentRecord()
{
return (_eof ? _null :
            record);
}
}

The GetCurrentRecord function returns the instance of ISourceRecord stored in the class when the file has not reached the end.

As you can see, implementing the source part of the CSV connector is not very complicated and is made easy using supporting classes. Let's now examine the implementation of the CSVSourceRecord class:

namespace DTEConnectorFactory.CSV
{
/// <summary>
/// Implementation of ISourceRecord for CVS connector.
/// </summary>
public class CSVSourceRecord : BaseFileWithFields, ISourceRecord
{
private char _separator;
private string _recordID = null;
private string _rawData = null;

// ISourceRecord methods

//code ommited

The ParseFieldFromRawData is the method that transforms raw data, in our case a text line in the source file, into corresponding fields of the CSV connector by splitting the string with the character indicated in the source descriptor file.

public bool ParseFieldsFromRawData (object sLine)
{
try
{
_rawData = (string)sLine;

base.FieldValue = _rawData.Split(_separator,
base.FieldValue.Length);

return true;
}
catch (Exception e)
{
Trace.WriteLine(e);
return false;
}
}

The LoadDescriptor method has two definitions, one that takes a string as a parameter and second one that takes an XmlElement as a parameter. The second method is useful, as we'll see later, for record-based connectors. Both methods use an instance of the CSVDescriptorReader class to read the CSV source descriptor and retrieve some of the information that is needed in this class like, such as information about fields, field separator, and record name.

public bool LoadDescriptor (string
            DescriptorFile)
{
CSVDescriptorReader configReader = new
            CSVDescriptorReader();

if
            (!configReader.Load(DescriptorFile))
return
            false;

_separator = new
            char[configReader.SeparationChar.Length];
configReader.SeparationChar.CopyTo(0,
            _separator, 0,
configReader.SeparationChar.Length);

_recordID =
            configReader.RecordID;
base.InitializeNoOfFields(configReader.Columns.Length);

return
            true;
}

The CSVDescriptorReader class is designed to parse a descriptor for a CSV connector. Each descriptor is unique because it contains information of a particular data source and therefore it is not possible to write a generic reader. In this particular case, the CSV descriptor contains a list of columns in a record and the character used as field separator. More details about each descriptor are presented in the next section.

Next we'll present the CSVDestination and CSVDestinationRecord classes. Destinations implement the IDataDestination interface, and CSVDestination is no exception. The primary goal of a destination is to save a record to the specified location. This is accomplished with the WriteRecord method, which takes as a parameter an object that implements the IDestinationRecord interface. The implementation of WriteRecord is particular to each connector and we'll outline the differences as we move on.

Let's take a look at this method in CSVDestination.

public bool WriteRecord (IDestinationRecord record)
{
base.line = (string)record.CollateRecordFromColumns();
return base.WriteLine();
}

We designed the method this way keeping in mind that records in different destinations have different formats. For example a record for a CSV file is a list of columns joined using a separator character, a record in an XML file is an Xml element, and a record in a database can be created using an INSERT SQL statement.

Therefore, the CollateRecordFromColumns method returns an Object value. In .NET all classes derive from the Object class and we can take advantage of this and rely on this lowest common denominator to achieve a more generic code. Using a cast in the connector we can quickly convert the Object value to the original format.

CSVDestination converts the returned value into a string representing the line of text to be written in the CSV destination file, and performs a write operation using the WriteLine method in BaseFile class.

Let's pause for a moment to analyze the CollateRecordFromColumns method. In the case of the CSV connector, the line of text that needs to be written is built simply by calling String.Join. The String class has a rich set of methods and one of them is Join, which is exactly what we need to build the CSV record.

public object CollateRecordFromColumns()
{
return String.Join(_separator, base.FieldValue);
}

The rest of the methods in CSVDestination and CSVDestinationRecord are simple enough to be omitted from this presentation.

The next text-based connector is for text files with fixed width fields, which is probably the next most used text file format.

Fixed Width Connector

Let's start by analyzing a sample descriptor for this type of connector:

<?xml
            version="1.0"?>
<descriptor type="FIXED_WIDTH"
            rec_id="POSRecord">
<column name="StoreID" start="0"
            width="8" paddingType="none"/>
<column name="SKUNumber"
            start="8" width="6" paddingType="left" paddingChar="
            "/>
<column name="PurchaseDateTime" start="14" width="10"
            paddingType="none"/>
<column name="ProductDesc" start="24"
            width="30" paddingType="none"/>
<column name="SalePrice"
            start="54" width="10" paddingType="right"
            paddingChar="0"/>
</descriptor>

The type attribute contains the identifier for the fixed width connector, FIXED_WIDTH. The record identifier is contained in the rec_id attribute. Column elements contain information about fields in the file like name starting position (zero based), width of each field, and padding information: type and character. The starting position of the columns don't need to be in sequence, and it doesn't have to cover 100% a physical record. For example, given the above example, we might decide to omit one of the fields from the sequence. Let's say we don't need the product description in our application, so we decide to delete this column element from definition. This will not cause problems for DTE, and it will simply ignore fields that are not described in a column element.

The implementation of FWFSource and FWFDestination are very similar to FWFSourceRecord and FWFDestinationRecord respectively with CSV, except with the way that fields are parsed from raw source data and the way a record is built from fields. The implementation for these two different methods is provided below.

The ParseFieldsFromRawData method uses the information from the descriptor, where each field starts, and how many characters it has, to get the value of each field.

public bool ParseFieldsFromRawData (object sLine)
{
_rawData = (string)sLine;

try
{
for (int i=0; i < _fieldInfo.Length; i++)
base.FieldValue[i] =
_rawData.Substring(_fieldInfo[i].Start, _fieldInfo[i].Width);
}
catch (Exception e)
{
Trace.WriteLine(e);
return false;
}

return true;
}

For this to work correctly it is very important to have correct information in the descriptor file. If the raw data information is not exactly matching the pattern in the descriptor, the function returns false.

In the FWFDestinationRecord class we have two methods that help to build the record data. In the case of a file with Fixed Width Fields the record is a string.

public object CollateRecordFromColumns()
{
string tmpLine = System.String.Empty;
int tmp = 0;

for (int i=0; i < _fieldInfo.Length; i++)
{
if (tmp < _fieldInfo[i].Start)
{
tmpLine += new String(' ', _fieldInfo[i].Start - tmp);
}
tmpLine += MakeFieldWidth(i);
tmp += _fieldInfo[i].Width;
}

return tmpLine;
}

// private functions
private string MakeFieldWidth (int index)
{
FWFField _info = _fieldInfo[index];

if (FieldValue[index] == null)
return new String(_info.PaddingChar[0], _info.Width);

if (FieldValue[index].Length > _info.Width)
return (FieldValue[index]).Substring(0, _info.Width);
else if (FieldValue[index].Length < _info.Width)
if (_info.PaddingType == PaddingTypeEnum.PADDING_TYPE_RIGHT)
return (FieldValue[index].PadRight(_info.Width, _info.PaddingChar[0]));
else if (_info.PaddingType == PaddingTypeEnum.PADDING_TYPE_LEFT)
return (FieldValue[index].PadLeft(_info.Width, _info.PaddingChar[0]));
else if (_info.PaddingType == PaddingTypeEnum.PADDING_TYPE_NONE)
return (FieldValue[index].PadRight(_info.Width, ' '));

return FieldValue[index];
}

A helper function MakeFieldWidth is implemented to bring a field to the correct length. This is done using the information in the descriptor file. As we'll see in the next section the descriptor contains all the necessary information about fields. Again we make use of the built-in functions of the String class to left or right pad a string with given characters.

Next we'll take a look at the record based connector. This type of connector is very useful when we have to read or create a source file with multiple types of records. The connector is built to recognize the type of record in the source file by looking at a specific sequence of characters, or by interrogating the notification client object. A limitation of the current version is that only two types of record types are allowed: CSV and FWF.

In the RecordBasedSource class the most interesting piece of code is the MoveNext method:

public bool MoveNext()
{
bool ret = true;
string tmpRawData = null;
RecordIdentifierValueToIDMapping tmpMap;

try
{

We use BaseFile methods to read the next line of text from the file. If the operation succeeds, the next step is to identity what record type we just read. For this, we first get the sequence of characters as indicated in the descriptor file and lookup in the map to find the correct record name. All the information is provided in the descriptor file.

if (!_eof && !base.ReadLine())
{
ret = false;
_eof = true;
}
else
{
// get record ID
_currRecordID =
base.line.Substring(_recID.Start, _recID.Length);

tmpMap = (RecordIdentifierValueToIDMapping)_recIdValueRecIDMappings[_currRecordID];
_currRecordID = tmpMap.RecordID;

// get record object from list
_currRecord = (ISourceRecord)_records[_currRecordID];

After we have the record name we can move to parsing the fields from the raw data. There is one more step before we can do this, which is to eliminate the record identification sequence from the raw data. We call ParseFieldsFromRawData on the identified record object instance and if the source record needs to store the value of the identifier in a specific column we put the value in place.

            //
            parse fields from data source
tmpRawData = (_recID.Start > 0 ?
            base.line.Substring(0, _recID.Start - 1) :
String.Empty) +
            base.line.Substring(_recID.Start + _recID.Length);

ret =
            _currRecord.ParseFieldsFromRawData(tmpRawData);

if (ret
            && tmpMap.DestColumnIndex >
            -1)
_currRecord.SetColumnValue(tmpMap.DestColumnIndex,
            tmpMap.Identifier);
}
}
catch (Exception
            e)
{
Trace.WriteLine(e);
ret = false;
}

return
            ret;
}

You might have already noticed that this connector namespace has no Source and Destination records defined, because we use either CVS's or FWF's for corresponding records.

XML Connector

Next, we'll take a look at the XML connector, which is a little bit more complex than previous ones. When we designed the XML connector we began by analyzing many types of files to observe a common pattern. What we noticed is that in most situations the elements with significant information are located immediately under the root element or somewhere in the heart of the file. Also, the structure of the record can be simple or complex, meaning that the elements of the record are located immediately under record tag or there is an additional XSL path to it.

The following diagram depicts a sample file:

<?xml version="1.0"?>
<x>
<x1>
<y> <!- Record starts here -->
<z>
<a>field 11</a>
<b>field 12</b>
<c>
<d>field 13</d>
</c>
</z>
</y> <!- Records ends here -->
<y> <!- Record starts here -->
<z>
<a>field 21</a>
<b>field 22</b>
<c>
<d>field 23</d>
</c>
</z>
</y> <!- Records ends here -->
<y> <!- Records starts here -->
<z>
<a>field 31</a>
<b>field 22</b>
<c>
<d>field 33</d>
</c>
</z>
</y> <!- Records ends here -->
</x1>
</x>

As you can see in this example, the path to the list of records is x/x1, and each record has three fields. The third field has a more complex structure with an outer element c and inner one d. Also the record has two levels to actual field elements: y/z.

Let's now examine the descriptor file to better understand how it works:

<?xml version="1.0"?>
<descriptor type="XML" rec_id="XMLRecord">
<record_path>//x/x1</record_path>
<record_tag>y/z</record_tag>
<columns>
<column name="DP_STREAM_NO">a</column>
<column name="CONTRACT">b</column>
<column name="PURCHASE_DATE">c/d</column>
</columns>
</descriptor>

The Record_Path element describes the XPath to the list of records. In our example, records are enclosed in element y which is located under element x1, so the XPath to y is //x/x1. The Record_Tag element contains the XPath inside the record structure to the element where the field nodes start. In our sample, record fields are located inside z element, and the XPath relative to a record root is y/z. A series of column elements describe the XPath of each field relative to the root element in the report_tag element. In our example, the record has three fields described in nodes a, b and d. The node d is located under c so the relative XPath is c/d.

The XML connector we built can create and read these kind of XML files. The structure of the XML file we want to read can be more complex that this, and we intend to extend the capabilities of this connector, but in the majority of cases this is good enough.

The basic idea behind XmlSource is to create a list with source elements and then fetch them as the client application calls the MoveNext method. Let's see the Open method:

public bool Open ()
{
bool ret =
            true;

try
{
if (LoadDescriptor(_sConfigFileName) ==
            false)
return false;

_recordID =
            _record.GetRecordID();
_index = -1;

_doc = new
            XmlDocument();
_doc.Load(_sFileName);
_root =
            _doc.DocumentElement;
_lines =
            _root.SelectNodes(((XmlSourceRecord)_record).RecordPath + "/"
+
            ((XmlSourceRecord)_record).RecordTag);

if (_lines.Count ==
            0)
_eof = true;
else
_eof = false;
}
catch
            (Exception e)
{
Trace.WriteLine(e);
ret =
            false;
}

return ret;
}

After loading the descriptor file we can open the source XML file, and using XSLT we grab the list of elements with records that we are interested in. The information about where these records are located is maintained in the descriptor file. The list of XML elements with records is maintained and we'll fetch them when requested.

The MoveNext method is shown next:

public bool MoveNext ()
{
if (_eof
            && _index + 1 > _lines.Count - 1)
{
_eof =
            true;
return (!_eof);
}
else
return
            _record.ParseFieldsFromRawData(_lines.Item(++_index));
}

We use a variable to maintain the current position in the list, very much like a cursor pointer in a database recordset. When there are no more elements we flag the EOF variable.

Moving to the XmlSourceRecord class we analyze the ParseFieldsFromRawData method:

public bool ParseFieldsFromRawData (object
            oRawData)
{
XmlNode _tmp = null;
XmlNode aNode =
            (XmlNode)oRawData;
bool ret = true;

try
{
for (int
            i=0; i < _fieldInfo.Length; i++)
{
_tmp =
            aNode.SelectSingleNode(_fieldInfo[i].ColumnTag);
if (_tmp !=
            null)
base.SetField(i, _tmp.InnerText);
}
}
catch
            (Exception e)
{
Trace.WriteLine(e);
ret =
            false;
}

return ret;
}

Raw data in this case is an XmlNode object that is passed from the MoveNext method. We then use the information in the descriptor about the record structure and parse each field value.

Moving forward to the XmlDestination and XmlSourceDestination classes, notice how we build the record object. In our case, this is the XmlNode containing all fields. Building the XmlNode requires us to build the whole record structure while we populate field values.

public object
            CollateRecordFromColumns()
{
string[] elem;
XmlNode node =
            null, rec = null, newRecord = null;
int i = 0;

// create
            record
elem = _recordTag.Split(new char[] {'/'});
node =
            _recs;

for (int idx = 0 ; idx < elem.Length; idx++)
{
node =
            node.AppendChild(_doc.CreateElement(elem[idx]));
if (i ==
            0)
newRecord = node.Clone();
}

// for safety reasons
            we'll check if new records was created
if (newRecord ==
            null)
newRecord = node.Clone();

rec = node;

// add
            record fields
for (int idx = 0 ; idx < _fieldInfo.Length;
            idx++)
{
elem = _fieldInfo[idx].ColumnTag.Split(new char[]
            {'/'});
i = 0;
node = rec;
while (i < elem.Length)
node =
            node.AppendChild(_doc.CreateElement(elem[i++]));

node.InnerText
            = _file.GetField(idx);
}

return
newRecord;
}

After the record is built, we return the XmlNode object to the WriteMethod function that adds it to the existing list of records. The XmlDestination object maintains an instance of XmlDocument with all the current records until the Close method is called, when it saves it on disk.

The so-called header part of the XML file, x/x1 in our sample file, is created when the destination file is opened. Notice the Open method:

public bool Open()
{
if (LoadDescriptor(_sConfigFileName) == false)
return false;
else
_recordID = _record.GetRecordID();

// create the XmlPath to the records
string[] elem = ((XmlDestinationRecord)_record).RecordPath.Split(new char[] {'/'});
string sXML = "<?xml version=\"1.0\"?>";

Stack stack = new Stack(elem.Length);

As you can see, we've made use of Stack class. Stack is a structure that follows the rule FILO, First In Last Out. This means that elements are stacked in the same order they are pushed in, and extracted in reverse order. You can think of it as a box where books are stored. To reach the first stored book in the box you must first extract all the books in reverse order.

In .NET the stack is implemented as a collection with two important methods, Push and Pop. Push adds a new element to the stack and Pop extracts last added element. The Stack object can be created with an initial limit, but the capacity grows as needed.

// create the structure to the record
int
            i = -1;
while (++i < elem.Length)
if
            (!elem[i].Equals(String.Empty))
{
sXML += "<" + elem[i] +
            ">";
stack.Push("</" + elem[i] + ">");
}

while
            (stack.Count>0)
sXML += (string)stack.Pop();

_doc =
            new XmlDocument();
_doc.LoadXml(sXML);
_root =
            _doc.DocumentElement;
_recs =
            _root.SelectSingleNode(((XmlDestinationRecord)_record).RecordPath);
((XmlDestinationRecord)_record).SetOutterElements(_doc,
            _recs);

return true;
}

We loop through the list of outer elements and create each one of them. The last one created, the most inner one, serves as a base for adding record elements and we store it in a variable _recs.

Database Connector

The last connector on our list, but probably the most used one, is Database.

Let's examine a descriptor example:

<?xml version="1.0"
            encoding="UTF-8"?>
<descriptor type="DATABASE"
            rec_id="POSSaleDBRec">
<column name="TransactionID"
            dataType="String" length="20"/>
<column name="ProductID"
            dataType="String" length="10"/>
<column name="Quantity"
            dataType="Int32"/>
<column name="SaleDate"
            dataType="DateTime"/>
</descriptor>

The column elements describe columns in the source recordset. The dataType and length attributes help you describe the type of fields that the source recordset contains.

Following the same pattern as before, we implemented DBSource and DBDestination classes just like we did with the DBSourceRecord and DBDestinationRecord classes. The only difference in the Source is that we use the System.SqlClient namespace to get a set of records from the specified database and store them in a SqlDataReader object. The source record is implemented as a collection that holds objects of type Object, so we cannot use the BaseFile class any more.

Here is the implementation for the ParseFieldsFromRawData method:

public bool ParseFieldsFromRawData (object
            rawData)
{
try
{
_rawData =
            (SqlDataReader)rawData;

// read fields from data reader based
            on descriptor
for (int i=0; i < _dbFieldValues.Length;
            i++)
_dbFieldValues[i] = _rawData.GetValue(i);

return
            true;
}
catch (Exception e)
{
Trace.WriteLine(e);
return false;
}
}

The implementation for the Destination object is also quite simple because persisting a record involves nothing more than executing an INSERT SQL statement. The INSERT SQL is returned by the DBDestinationRecord object. Here is the implementation of WriteRecord :

public bool WriteRecord (IDestinationRecord record)
{
string sInsertSQL = (string)_record.CollateRecordFromColumns();
_dbHelper.RunSQL(sInsertSQL);
return true;
}

The implementation for CollateRecordFromColumns is listed below:

public object
            CollateRecordFromColumns()
{
string sInsertSQL = "INSERT INTO
            " + _sTableName + "(";
string sValue = null;

for (int i=0;
            i < _fieldInfo.Length; i++)
sInsertSQL += (i==0 ? String.Empty
            : ",") + _fieldInfo[i].Name;

sInsertSQL += ")
            VALUES(";

for (int i=0; i < _fieldInfo.Length;
            i++)
{
sValue = Convert.ToString(_dbFieldValues[i]);
if
            (_fieldInfo[i].Length != -1 && sValue.Length >
            _fieldInfo[i].Length)
sValue = sValue.Substring(0,
            _fieldInfo[i].Length);

if (_fieldInfo[i].DataType ==
            FieldDataType.FIELD_TYPE_STRING ||
_fieldInfo[i].DataType ==
            FieldDataType.FIELD_TYPE_DATETIME)
sValue = "'" + sValue +
            "'";

sInsertSQL += (i==0 ? String.Empty : ",") +
            sValue;
}

sInsertSQL += ")";

return
            sInsertSQL;
}

We basically loop through the list of fields and build an SQL statement like this: INSERT INTO (columns) VALUES (values). We use the information in the descriptor to check the type of each column because string and date fields need to be enclosed between ' and ' characters.

The DBDestinationRecord class has a slightly more complicated implementation for the SetColumnValue function. Because the database can hold multiple types of data, we need to convert incoming values to destination data type. This is accomplished by using the System.Reflection namespace to dynamically invoke a function in a class.

The Convert class has a set of static functions that perform conversions from one type to another. If the type of passed parameter is different from the destination type we call the appropriate function in the Convert class. We use the Invoke method of the Type class, making sure to set the correct binding parameters. In this case, BindingFlags.Public, BindingFlags.InvokeMethod and BindingFlags.Static are required.

public void SetColumnValue (int index, object colValue)
{
string sDataType = colValue.GetType().FullName;
object result = null;

// need to translate to destination data type
Type type = typeof (Convert);

if (!sDataType.EndsWith(_fieldInfo[index].DataType))
{
result = type.InvokeMember("To" + _fieldInfo[index].DataType,
BindingFlags.Public | BindingFlags.InvokeMethod | BindingFlags.Static,
null, null, new Object[] { colValue });
_dbFieldValues[index] = result;
}
else
_dbFieldValues[index] = colValue;
}

All database related operations like, open connection, execute SQL statement, create a command object, create and load a set of records, are separated in the DatabaseHelper class.

The following diagram depicts the relationship between transformation descriptor elements, record descriptor elements, and DTE to help you better understands the existing relations between key elements.

This figure helps you remember key elements in the descriptors and the relationship between them:

  • The name attribute in the source and destination elements is used in the mappings element. They help identify a particular destination and of course the source.
  • The rec_id attribute is also a key attribute in the mappings element as it helps identify the record that is used in a mapping relation.
  • The properties attached to source and destination elements are also important for the DTE as they identify the location of files, database connection string or SELECT statements. Remember that these properties can be modified at run-time through DTE notifications.

Source and Destination descriptors

This section is important because it explains the structure of each descriptor so that you'll be able to create your own descriptors. In previous sections we mentioned that critical information about each connector is contained in descriptor files.

There are few general rules that apply to all descriptors:

  • Type attribute in the root element contains the description of record type. The value of this attribute is predefined in the factory settings file.
  • Rec_id attribute in the root element contains the name of the record

Both these elements are mandatory for all descriptors.

CSV Descriptor

<?xml
            version="1.0"?>
<descriptor type="CHAR_SEPARATED_VALUES"
            separation_char="," rec_id=" ">
<column
            name="field1"/>
<column name="field2"/>
<column
            name="field3"/>
</descriptor>

The separation_char attribute indicates which character is going to be used when parsing source records, and also when building destination records from individual fields. A collection of column elements indicates the order of fields and the name of each field.

FWF Descriptor

<?xml
            version="1.0"?>
<descriptor type="FIXED_WIDTH"
            rec_id="POSRecord">
<column name="StoreID" start="0"
            width="8" paddingType="none"/>
<column name="SKUNumber"
            start="8" width="6" paddingType="left" paddingChar="
            "/>
<column name="PurchaseDateTime" start="14" width="10"
            paddingType="none"/>
<column name="ProductDesc" start="24"
            width="30" paddingType="none"/>
<column name="SalePrice"
            start="54" width="10" paddingType="right"
            paddingChar="0"/>
</descriptor>

There are no additional attributes in the root element. A collection of column elements defines each field. For this descriptor, the order is not important because each field is very well defined within the collection. Each column element can have following attributes:

  • Name is mandatory and represents the name of the field.
  • Start is mandatory and represents the start character position for this field. Valid values are greater or equal to zero.
  • Width is mandatory and represents the maximum number of characters the field can contain.
  • PaddingType is optional and contains the type of padding to be applied to the current field. This attribute is used only for destination records. Valid values for this attribute are: right, left, none. If the attribute is not present, the value none is assumed by default.
  • PaddingChar is mandatory if PaddingType is different than none. It represents the character to be used in the padding operation. As for PaddingType, this attribute is used only for destination records.

Xml Descriptor

<?xml version="1.0"?>
<descriptor type="XML" rec_id="XMLRecord">
<record_path>//x/x1</record_path>
<record_tag>y/z</record_tag>
<columns>
<column name="DP_STREAM_NO">a</column>
<column name="CONTRACT">b</column>
<column name="PURCHASE_DATE">c/d</column>
</columns>
</descriptor>

There are no additional attributes in the root element.

The Record_Path element defines a XSL path to the record tag. This element can be empty if the record tag nodes are child elements of the root XML document.

Record_Tag represents a XSL path to the element that contains the list of fields inside the record, plus the name of record element. This element cannot be empty, but can be only the name of the record tag without an XSL path.

The Columns collection contains a list of columns in a record. The order is not important because each element is described as an XSL path. Column elements contain a name attribute representing the name of the field, and the value of column element is the XSL path to the field element.

Database connector

<?xml version="1.0"
            encoding="UTF-8"?>
<descriptor type="DATABASE"
            rec_id="ShoppingCartRec">
<column name="shcCartID"
            dataType="String" length="30"/>
<column name="shcItemID"
            dataType="String" length="30"/>
<column name="shcQuantity"
            dataType="Int32"/>
</descriptor>

The Column elements define a column returned by the SQL statement in a DBSource object, or fields that need to be filled in for a DBDestination object. The order is important only for the source end.

  • Name attribute is the name of the column as defined in the database.
  • DataType attribute specifies the type of field. Valid values for this field are: String, Int32, Int64, Int16, Long, DateTime, etc. In general any data type that can be represented in a database and which has a .NET object defined.

Some additional attributes may appear as they are required by the DataType attribute. For example, String data type requires a Length attribute.

RecordBased descriptor

The structure of this descriptor is a little bit different because of the complexity associated with the connector.

<?xml version="1.0"?>
<descriptor type="RECORD_BASED">
<record_identifier type="AUTO" start_pos="0" length="3" />

<!-- Describes the mapping between record name and record identifier -->
<record_identifier_type rec_id="Rec1" identifier="ID" dest_column_index="1"/>

<!-- for now only FW and CSV record types are supported -->
<record type="CHAR_SEPARATED_VALUES" separation_char="," rec_id="Rec1" import="">
<column name="field1"/>
<column name="field2"/>
</record>
</descriptor>

The Type attribute in the descriptor element identifies the type of connector and should always be RECORD_BASED. This attribute is mandatory and an error is raised if the attribute is not present.

The Record_Identifier element describes the starting position and the number of chars where the record identifier is located. In most cases this element is located at the beginning of record.

The Record_Identifier_Type elements define a map between the record name and the value of the record identifier. As we mentioned before, each record will be identified after a sequence of chars in a specified position in the source file. There must be defined a number of such elements equal with the number of possible values for record identifier field. The Rec_Id attribute defines the name of the record that must be defined later in the file. The Identifier attribute contains the value of the record identifier.

Each record element defines a record that must correspond with a Rec_id attribute in a Record_Identifier_Type element. The structure of this record is identical with that of the descriptors mentioned above. If you want to avoid including the record descriptor in this XML file you can specify in the import attribute a path to the file that contains the descriptor. Import attribute is optional, but if present, takes precedence over column elements, in which case column elements are ignored.

The Sample Application

The application that we'll show you comes to exemplify the use of DTE in a real world scenario. A retail store registers POS sales (Point Of Sale) in a text file with following structure:

  • The file can contain two types of records: sales and customer returned products
  • The record identifier is three characters, starting at position 0. Valid values are: SAL for sale and RET for return.

The record structure for each record type is:

A Sale record has the following structure: the record Identifier is three characters, "SAL" in this example, and then a list of CSV fields containing sale attributes.

SALTransactionID,SKU,Quantity,UnitRetailPrice

A return record has the following structure: the record identifier is three characters, "RET" in this example, and then a list of CSV fields containing return attributes.

RETTransaction,SKU,Quantity

SAL and RET values are record identifiers that tells DTE which record to choose when decoding a line in the source file.

We need to import sales and returns in two different tables in a database. For simplicity, and because not everybody is able to install a database, I changed the requirement with: "We need to separate the file in two different CSV files: sales and returns. The destination files will contain the same number of fields in the corresponding source record. In addition the sale file will contain another field with the record's identifier value".

To accomplish this, we create a file with test data. This file is called POSSales_In_All.txt. The second step is to define the descriptor for the RecordBase connector. The file name is RecordBasedFile1.xml and contains the following elements:

  • Record_identifier is defined as the first three characters in the record (SAL and RET). Notice that for the SAL record we have the dest_column_index attribute defined because we need to copy the identifier in the destination field number four.
  • Two record_identifier_type elements to map each record identifier with a record description. SAL is mapped to SaleSrc record and RET is mapped to ReturnSrc record.
  • Two record definitions one for each destination file: Sales and Returns. They contain the fields as they were defined in the test data file. An additional field RECORD_ID has been added to the SAL record to capture record_id value.

The complete source for RecordBasedFile1.xml is:

<?xml version="1.0"?>
<descriptor type="RECORD_BASED">
<record_identifier type="AUTO" start_pos="0" length="3" />

<!-- Describes the mapping between record name and record identifier -->
<record_identifier_type rec_id="SaleSrc" identifier="SAL" dest_column_index="4"/>
<record_identifier_type rec_id="ReturnSrc" identifier="RET" />

<!-- for now only FW and CSV record types are supported -->
<record type="CHAR_SEPARATED_VALUES" separation_char="," rec_id="SaleSrc">
<column name="TRANSACTION_ID"/>
<column name="SKU_NUMBER"/>
<column name="QUANTITY"/>
<column name="PRICE"/>
<column name="RECORD_ID" />
</record>
<record type="CHAR_SEPARATED_VALUES" separation_char="," rec_id="ReturnSrc">
<column name="TRANSACTION_ID"/>
<column name="SKU_NUMBER"/>
<column name="QUANTITY"/>
</record>
</descriptor>

Next, we must create the description files for each destination. The first destination description file is SalesOnly.xml :

<?xml version="1.0"?>
<descriptor type="CHAR_SEPARATED_VALUES" separation_char="|" rec_id="SaleDst">
<column name="TRANSACTION_ID"/>
<column name="SKU_NUMBER"/>
<column name="QUANTITY"/>
<column name="PRICE"/>
<column name="RECORD_ID" />
</descriptor>

We change the separation char into "|" (pipe), whereas in the source file we have "," (comma). This is just to show that we can define a very flexible scenario. The record contains all the fields from the source file plus another field, RECORD_ID, which will be populated with the record_identifier value, SAL.

The second destination descriptor file is ReturnsOnly.xml :

<?xml version="1.0"?>
<descriptor type="CHAR_SEPARATED_VALUES" separation_char="|" rec_id="ReturnDst">
<column name="TRANSACTION_ID"/>
<column name="SKU_NUMBER"/>
<column name="QUANTITY"/>
</descriptor>

We make the same change in the separation char, and for this record we don't add the additional field. This is not required because we don't need to copy the record identifier value, RET, to this record.

Next we define the transformation descriptor in the file TrxTest.xml :

<?xml version="1.0"?>
<transformation>

Source is defined as RECORD_BASED and we name it "POSAll". The two required properties are:

  • "Data Source" containing the path to the test data file.
  • "Configuration File" which contains the path to the RecordBasedFile1.xml file.

<source mode="RECORD_BASED" name="POSAll">
<property name="Data Source">E:\projects\DTE\misc\test_data\POSSales_In_All.txt</property>
<property name="Configuration File">E:\projects\DTE\misc\test_data\RecordBasedFile1.xml</property>
</source>

The two destinations for Sales and Returns are defined as CHAR_SEPARATED_VALUES and we name them: "SalesOnly" and "ReturnsOnly".

Each of them has the two mandatory properties defined as follows:

  • "Data Source" as SalesOnly_Out.txt and respectively ReturnsOnly_Out.txt
  • "Configuration File" as SalesOnly_Out.xml and respectively ReturnsOnly_Out.xml

<destination mode="CHAR_SEPARATED_VALUES" name="SalesOnly">
<property name="Data Source">E:\projects\DTE\misc\test_data\SalesOnly_Out.txt</property>
<property name="Configuration File">E:\projects\DTE\misc\test_data\SalesOnly.xml</property>
</destination>
<destination mode="CHAR_SEPARETED_VALUES" name="ReturnsOnly">
<property name="Data Source">E:\projects\DTE\misc\test_data\ReturnsOnly_Out.txt</property>
<property name="Configuration File">E:\projects\DTE\misc\test_data\ReturnsOnly.xml</property>
</destination>

We do not define any client object for this test application.

The mappings elements define the transformation of source fields into destination fields, which in this case is symmetrical. Each source field goes to the corresponding destination field:

<mappings source_rec_id="SaleSrc"
            dest_rec_id="SaleDst" dest_name="SalesOnly">
<map
            destination_column_index="0" source_column_index="0"/>
<map
            destination_column_index="1" source_column_index="1"/>
<map
            destination_column_index="2" source_column_index="2"/>
<map
            destination_column_index="3" source_column_index="3"/>
<map
            destination_column_index="4"
            source_column_index="4"/>
</mappings>
<mappings
            source_rec_id="ReturnSrc" dest_rec_id="ReturnDst"
            dest_name="ReturnsOnly">
<map destination_column_index="0"
            source_column_index="0"/>
<map destination_column_index="1"
            source_column_index="1"/>
<map destination_column_index="2"
            source_column_index="2"/>
</mappings>
</transformation>

The last step is to build a C# console application in our solution to test the transformation process. In the solution that comes with this case study, you already have this project created. Open the Test project class TestDTE and you'll see following source code:

using System;
using DTE;

namespace
            TestDTE
{
/// <summary>
/// Summary description for
            TestDTE.
/// </summary>
class TestDTE
{
///
            <summary>
/// The main entry point for the
            application.
/// </summary>
[STAThread]
static void
            Main(string[] args)
{
TestDTE tstApp = new
            TestDTE();
tstApp.Test1();
}

public void
            Test1()
{
DTEApplication appDTE = new
            DTEApplication();

if
            (appDTE.StartTrxProcess("E:\\projects\\DTE\\misc\\test_data\\TrxTest7.xml"))
Console.WriteLine("Operation
            successed.");
else
Console.WriteLine("Operation
            failed.");
}
}
}

This is a very simple console application that uses the DTE namespace. The main function creates a new DTEApplication object and calls the StartTrxProcess method with a parameter containing the path to the transformation descriptor file.

After the application terminates you should be able to see the two files created with the following content:

SalesOnly.txt
10045|256MbRAM|3|67.99|SAL
10046|DELLP4+PRT|1|1867.95|SAL
10047|SCAN16x12|2|78.99|SAL

ReturnsOnly.txt

10045|256MbRAM|1
10047|SCAN16x12|1

Application Setup

To setup the DTE framework you need to download the source code provided with the case study and unpack the zip file to a folder. Open the DTE.sln file which will start Visual Studio .NET and open all projects in this solution.

In order to be able to test the framework you need to modify following files:

DTEApplication.cs in project DTE, line 36:

_factoryReader.Load("E:\\projects\\DTE\\testdata\\DataFactory.config.xml");

Change the path to where you saved the file DataFactory.config.xml. We'll change this in the next version to read this information from the application configuration file.

Change the path to the folder where you unpacked the zip file in the following files:

  • TrxTest.xml
  • RecordBased1.xml
  • SalesOnly.xml
  • ReturnsOnly.xml

To execute the application you must move to the Test project and hit F5 key. This will start the application in debug mode and you'll be able to see if any error messages are displayed during execution.

Case Study Review

Now that you have read through this case study you've seen why Data Transformation Engine represents one of the solutions to a problem that is critical to data processing applications or any other application that needs to easily be able to gather information from diverse sources. This might seem like an old problem with all the standards that we see today in the industry regarding data manipulation and data transfer, like EDI, SOAP or XML, however, there are still many companies and organizations that prefer to stick with current setup and millions of applications still produce data in proprietary formats.

This case study gave you the opportunity to observe how this application was designed and how we made use of design patterns to create reusable code and make room for extensibility. Design patterns are perhaps the most powerful thing not used enough in software development today, and software engineers should perhaps use them more often.

DTE also gave us the opportunity to show you how to make use of new features in .NET Framework, like dynamic loading of assemblies and dynamic invocation of methods on object instances. These are very powerful features previously available only to Java developers. The adoption of metadata and self described classes and assemblies in .NET make it possible for us developers to design better applications and implement a large variety of features not possible before, like the one we've seen in DTE. Imagine how cumbersome it would have been to create a mechanism for object instantiation at run-time knowing only the name of the class and possibly the DLL in which it is defined.

Further Work

The DTE framework presented in this case study is only the first version, and we have in mind a few features that will be added in future versions, such as:

  • Implement the piece of source code that allows the client observer object to determine the type of record in the data source based on raw information. We have designed the interface and the method but we must enable the source code in the DTEApplication class.
  • Build a user interface that allows clients to easily build descriptors and configure DTE settings.
  • Extend the list of connectors with a very interesting one: an Object that can act as source or destination. In this case, the client application will either pass a parameter to the process indicating an instance of the object or instruct the DTE to get/create an instance of the object. This will give DTE the capability to act not only as transformer but also as data source or data consumer.
  • Extend the features for record based connector to allow:
  • Developers to specify a different column where record identifier value goes.

Case Study Information
Author | Catalin Tomescu
Editor | Adam Ryland
Reviewers | Bill Wilson, Matthew MacDonald, David M Williams

If you have any questions or comments about this case study, or suggestions for further pieces, please contact the technical editor. If you wish to talk about the case study with other subscribers, please visit our forum here.

C#Today - © 2002 WroxPress Limited


Cool Tool for XSLT – Part 2

Cool Tool for XSLT – Part 2
http://www.csharptoday.com/content.asp?id=1819
Article Rating: 4.5
Published on February 12, 2002
Abstract
XSLT, or “Extensible Style Language: Transformations”, is a way of transforming XML files from one format to another. XSLT has been in the limelight for a while now, as it has been a popular technology for Web site builders for changing XML data into HTML. XSLT can be a little complicated to understand. In this article, Matthew Reynolds will show you how to build a tool that can help you learn about XSLT and also use XSLT in your applications. In the previous part of this article, we looked at how to build a simple XML editor control and use that control in our application. In this second part, we’ll look in more detail at XSLT and see a few of the basic building blocks that are helpful in day-to-day use.
Article

In this section of our article on XSLT, or rather how to build a cool tool for helping us work with XSLT, we're going to be adapting our control to handle the XML and HTML preview modes and also trying some transformations for the first time.

Using IE

To use Internet Explorer, we have to use COM interop to load and run the Microsoft Web Browser COM ("ActiveX") control. At the time of writing, there is no "managed code" version of IE - i.e. one that runs actually inside of the .NET runtime environment. That's basically because Microsoft hasn't written one yet! A Web browser as sophisticated as IE is represents a considerable investment, so it's not surprising that they expect us to make do with an old school COM component, rather than a nice new .NET component.

In fact, for those of you that don't know this, the Microsoft Web Browser control is IE. The iexplore executable actually acts as a container for this control. When we use this control in our applications, we're actually using Internet Explorer.

The limitation with using IE is that it's geared up to download its own resources from the private network or Internet. It's not particularly easy to give IE a scrap of HTML and ask it to draw it. For this reason, the control is going to create two separate temporary files on the users computer and ask IE to "download" and render those files. We'll have one file for the XML preview, and one for the HTML preview. Technically these files will have the same name, but we'll give one a .xml extension and the other a .html extension as this is the easiest way to give IE instructions on how the two files should be rendered.

What we need to do is create a new User Control called MiniBrowser. Into this control we'll add an instance of the Microsoft Web Browser control. Once we've done that, in the constructor for XmlEditor where we create the TextBox control, we'll also create a MiniBrowser control. We can then alter the ConfigureView method such that the MiniBrowser control is displayed whenever the View or Xml properties are changed.

Create the new User Control now and add a StatusBar and Panel control to it, like this. Set the Panel control's Dock property to Fill.

To use the Microsoft Web Browser control, we have to customize the Toolbox and ask VS .NET to load the control. Right-click on the Toolbox and select Customize Toolbox. Scroll down the list until you find the Microsoft Web Browser control. Check it, and click OK.

The Explorer control will appear somewhere in your Toolbox, most likely right down at the bottom.

Draw an Explorer control onto the panel in the MiniBrowser control. Set its name to axIe, and Dock property to Fill. You won't be able to see the Explorer control once you do this.

We need methods in MiniBrowser that defer to methods on the Explorer control. Specifically, we need to be able to navigate to a given URL, and be able to refresh the page. In addition, we also need to be able to capture the StatusTextChange event off of the Explorer control and update our own status bar text.

Because we're using interop, some of this is a little counter intuitive. First off, when we need to navigate to a URL, we need to call the Navigate2 method of the Explorer control. When we do this, we need to supply a reference to a System.Object that happens to contain the URL that we want to visit. In addition, we also need to supply four blank parameters, using a similar technique. Add this method to MiniBrowser :

public void Navigate(string url)
{
// navigate...
object nullObject = null;
object urlObject = url;
axIe.Navigate2(ref urlObject, ref nullObject,
ref nullObject, ref nullObject, ref nullObject);
}

The RefreshDocument method works in a similar way. However, this time we just supply a single blank parameter:

public void RefreshDocument()
{
object nullObject = null;
axIe.Refresh2(ref nullObject);
}

When we added the Explorer control to the form, VS .NET generated an interop wrapper for us. This wrapper contains new classes that, primarily, allow us to create the control, but also allow us to tap into the events of that control. For example, if we want to create an event handler for the StatusTextChange event, we need to create an instance of AxSHDocVw.DWebBrowserEvents2_StatusTextChangeEventHandler. Add this code the constructor for MiniBrowser :

public MiniBrowser()
{
// This call is
            required by the Windows.Forms Form
            Designer.
InitializeComponent();

// event
            handler...
axIe.StatusTextChange += new
AxSHDocVw.DWebBrowserEvents2_StatusTextChangeEventHandler(IeStatusTextChange);

//
            go to blank...
Navigate("about:blank");
}

The code for IeStatusTextChange looks like this:

protected void IeStatusTextChange(object sender,
AxSHDocVw.DWebBrowserEvents2_StatusTextChangeEvent e)
{
statusbar.Text = e.text;
}

Now that we have the control, we can create an instance of it and display it. To do this, we need private fields in XmlEditor to hold an instance of MiniBrowser and also strings for the temporary filenames. Add the _ie, _xmlPreviewFilename and _htmlPreviewFilename members to XmlEditor :

public class XmlEditor : System.Windows.Forms.UserControl
{
// members...
private TextBox _editor;
private MiniBrowser _ie;
private Control _activeView;
private ViewMode _view = ViewMode.XmlEdit;
private string _xmlPreviewFilename;
private string _htmlPreviewFilename;

In the constructor, we need to create the new MiniBrowser control and use the System.IO.Path class to get the name of a temporary file to use.

public XmlEditor()
{
// This call is
            required by the Windows.Forms Form
            Designer.
InitializeComponent();

// create the editor
            control...
_editor = new TextBox();
_editor.Multiline =
            true;
_editor.WordWrap = false;
_editor.ScrollBars =
            ScrollBars.Both;
_editor.AcceptsReturn =
            true;
_editor.AcceptsTab = true;

// create ie and the temp
            files...
_ie = new MiniBrowser();
_xmlPreviewFilename =
            Path.GetTempFileName();
_htmlPreviewFilename =
            _xmlPreviewFilename + ".html";
_xmlPreviewFilename +=
            ".xml";

// configure the
            view...
ConfigureView();
}

Now we can turn our attention to ConfigureView. When we initially put together this method, we missed out the code that showed the IE control. We first off need to check the View property to see what view we need to show.

// ConfigureView - update the view so that
            we can see the data...
public void ConfigureView()
{
//
            what view do we need to see?
Control requiredView =
            null;
switch(View)
{
case ViewMode.XmlEdit:
requiredView
            = _editor;
break;
case ViewMode.XmlPreview:
requiredView =
            _ie;
break;
case ViewMode.HtmlPreview:
requiredView =
            _ie;
break;

default:
throw new Exception("View mode not
            supported");
}

// do we need to change the
            view?
if(_activeView != requiredView)
{
// clear the
            current view...
panelMain.Controls.Clear();

// show the
            new view...
_activeView = requiredView;
_activeView.Dock =
            DockStyle.Fill;
panelMain.Controls.Add(_activeView);
}

Once we have the view, we need to create the two preview files.

// create the temp
            files...
try
{
FileStream stream = new
            FileStream(_xmlPreviewFilename, FileMode.Create);
StreamWriter
            writer = new
            StreamWriter(stream);
writer.Write(Xml);
writer.Flush();
stream.Close();
stream
            = new FileStream(_htmlPreviewFilename, FileMode.Create);
writer =
            new
            StreamWriter(stream);
writer.Write(Xml);
writer.Flush();
stream.Close();
}
catch(Exception
            ex)
{
MessageBox.Show(ex.GetType().ToString() + ":" +
            ex.Message);
}

Finally, if we're looking at IE, we need to navigate to whichever preview file is appropriate.

// ie?
if(View == ViewMode.XmlPreview)
_ie.Navigate(_xmlPreviewFilename);
if(View == ViewMode.HtmlPreview)
_ie.Navigate(_htmlPreviewFilename);
}

It's bad practice to leave temporary files floating around when an application no longer needs them. What we'll do is adapt the Dispose method of the control such that it will delete the temp files when the object is disposed.

protected override void Dispose( bool
            disposing )
{
if( disposing )
{
if(components !=
            null)
{
components.Dispose();
}
}

// delete temp
            files...
DeleteTempFile(_xmlPreviewFilename);
DeleteTempFile(_htmlPreviewFilename);

base.Dispose(
            disposing );
}

protected void DeleteTempFile(string
            filename)
{
try
{
FileInfo info = new
            FileInfo(filename);
if(info.Exists ==
            true)
info.Delete();
}
catch
{
}
}

You can try this now if you like. Build the project to save the changes and open the Designer for Form1. Change the View property for xmlXml to XmlPreview and you'll see IE.

The View Menu

Being able to change the view mode is really only useful when the application is running. We need to wire up the View toolbar button to allow the user to change the mode that he wants. Ideally we need a drop-down menu available that allows quick access to the appropriate mode.

To do this, open the Designer for XmlEditor and add a new ContextMenu object. Change its Name property to menuView. Add three new items to the list, the names menuViewXmlEdit, menuViewXmlPreview and menuViewHtmlPreview.

Double-click on each of the items and add code to the handlers to set the View property.

private void menuViewXmlEdit_Click(object sender, System.EventArgs e)
{
View = ViewMode.XmlEdit;
}

private void menuViewXmlPreview_Click(object sender, System.EventArgs e)
{
View = ViewMode.XmlPreview;
}

private void menuViewHtmlPreview_Click(object sender, System.EventArgs e)
{
View = ViewMode.HtmlPreview;
}

One more thing needs to be done - we need to attach this ContextMenu control to the buttonView button. Open the Buttons property editor on toolbar and set the DropDownMenu property to menuView and set the Style property to DropDownButton.

Now run the project and you'll be able to change the view using the menu.

Transforming XML

At last we can try and transform some XML using XSLT! The first thing we need is a menu option. If you haven't already done so, add a MainMenu control to Form1 and then add a top-level menu called Transform and a sub-menu item called Transform Document. Set the Name property of this menu item to menuTransformTransform and set the Shortcut property to F5.

The second thing we need is a sample style sheet. Add this to the Lines property of xmlXsl (don't worry about what it does at the moment - we'll look at it later).

<?xml version="1.0"?>
<xsl:stylesheet version="1.0"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform">

<xsl:template match="/Customers">

<ul>
<xsl:for-each select="Customer">
<li>
<xsl:value-of select="Id"/>
</li>
</xsl:for-each>
</ul>

</xsl:template>

</xsl:stylesheet>

The class that we need to use for XSLT transformations is System.Xml.Xsl.XslTransform. The two methods we need are Load (used to load the XSL file) and Transform (used to perform the actual transformation). XslTransform works with XPath documents. XPath is a way of traversing an XML document using a "directory-scheme" style structure. I'll explain this more in a moment once we've done the transformation and we can see what results we actually get back.

An XPath document can be held in the class System.Xml.XPath.XPathDocument. The trick here is to take the value in our Xml property and create an XPathDocument from it. Unlike the XmlValidatingReader class that we saw before, we can't just give XPathDocument a block of text. It either has to load it itself from a network/Internet resource, or it has to get it from a stream. We can use System.IO.MemoryStream to do this. Our next job is to build the CreateMemoryStream method on XmlEditor. This will return a MemoryStream object that we can use as a source of data for the XPathDocument that we need for the transformation.

Creating a populated memory stream requires an extra, non-intuitive step. Because .NET is quite open to the idea of using different encoding standards other than ASCII or UNICODE, we have to use a separate class ( System.Text.ASCIIEncoding) to convert the System.String value to an array of System.Byte objects. Here's the method that should be added to XmlEditor :

public MemoryStream
            CreateMemoryStream()
{
// create the new
            stream...
ASCIIEncoding encoding = new ASCIIEncoding();
return
            new MemoryStream(encoding.GetBytes(Xml));
}

Creating the XPathDocument is simply a matter of getting hold of that stream and passing it to the constructor of the new XPathDocument object, like this:

// CreateXPathDocument - create an XPath
            Document for the data...
public XPathDocument
            CreateXPathDocument()
{
// create the new
            document...
Stream inputStream =
            CreateMemoryStream();
XPathDocument document = new
            XPathDocument(inputStream);
inputStream.Close();

//
            return...
return document;
}

Now that we can get XPathDocument objects back for both the XML and XSL editors, we can look at how do to the transformations. First off, open the Designer for Form1 and add a new Click event handler to the Transform - Transform Document option by double-clicking on it. Add this code:

private void menuTransformTransform_Click(object sender, System.EventArgs e)
{
Transform();
}

The Transform method is responsible for performing the transformation and setting the Xml property of xmlResults. The first thing the method needs to do is to check that the source XML and XSL documents are well formed.

public void Transform()
{
// test the first document?
string message = xmlXml.CheckDocumentWithReport();
if(message != null)
{
MessageBox.Show(
"The XML document is not well formed.\r\n\r\n" +
message, "XSLT");
return;
}
message = xmlXsl.CheckDocumentWithReport();
if(message != null)
{
MessageBox.Show(
"The XSL document is not well formed.\r\n\r\n" +
message, "XSLT");
return;
}

Once that's done, two things can go wrong. One, although the XSL document is well formed, it may not be valid - i.e. XslTransform may not be able to compile it. Two, although the transformation takes place, the results will not be as intended. This is the same deal that we face when writing software everyday - we may either see a compile error, or a run-time error.

The first thing to do is create a new MemoryStream object that the transformation output will be written to. Later, we'll read back the data written into this stream and set the Xml property of XmlResults to this value. We then load the XSL document into the XslTransform object.

// create somewhere to put the
            data...
MemoryStream outputStream = new
            MemoryStream();
try
{
// create the
            transform...
XslTransform transform = new
            XslTransform();

// load...
XPathDocument xslDocument =
xmlXsl.CreateXPathDocument();
transform.Load(xslDocument);

Once we've done that, we try the transformation. We get an XPathDocument back containing the source XML, and then call the Transform method.

// transform...
XPathDocument xmlDocument =
xmlXml.CreateXPathDocument();
transform.Transform(xmlDocument, null, outputStream);

Finally, once we've filled up the stream with the transformation data, we need to make sure all the data that needs to be written has been written by using the Flush method, and then read all the data back out again.

// read the data back and
            display...
outputStream.Flush();
outputStream.Seek(0,
            SeekOrigin.Begin);
StreamReader reader = new
            StreamReader(outputStream);
xmlResults.Xml =
            reader.ReadToEnd();
}

If something goes wrong, we use the static GetReportForException method on XmlEditor and display the information to the user.

catch(Exception ex)
{
// get the
            report...
message = XmlEditor.GetReportForException(
"The
            transformation could not be performed.",
            ex);
MessageBox.Show(message, "XSLT");
}
finally
{
//
            close...
outputStream.Close();
}
}

Run the project now and select the Transform Document option from the menu. You should see a string of XML in the right-hand editor.

Selecting the HTML Preview view shows us what that string of XML would look like if the browser were asked to display it. As it contains HTML tags, it draws it as two items in an unordered list.

"What Happened?"

That looks a little like magic, but what's actually happening?

XSLT can be used to transform XML documents from one format into another. In this case, we've taken an XML document that expresses data and reformatted it into an XML document that in actual fact is an XHTML document. XHTML is an emerging standard that in the long term may replace HTML. It's basically HTML that adheres to XML standards. XML has far more rules than HTML, so when we write XHTML code it tends to be far less ambiguous and therefore far easier to parse than traditional HTML code.

However, all of this is off-topic because virtually all browsers can render XHTML, and for the most part XHTML isn't important to this discussion. The important thing is that we have taken an XML document containing data and transferred it to what you can think of as an "HTML-compatible XML" document. For all intents and purposes, this is HTML.

To understand this, we have to look at the style sheet because this contains all of the instructions. I'll go through it line-by-line.

First off, we have the usual XML declaration that tells us that this is an XML document:

<?xml version="1.0"?>

Secondly, we start a tag defined in the http://www.w3.org/1999/XSL/Transform namespace. This contains the rules for XSL transformations. You can think of it as containing the language used in XSLT:

<xsl:stylesheet version="1.0"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform">

As we work through the document, imagine the document as a folder tree. Each child element is a sub-folder inside the parent element. XPath works on the principle that you have a pointer that points at your location in the tree. When the document is first created, this pointer points before the first data element. In our document, this is the Customers element, which is the top-level element.

The second tag ( xsl:template) we have tells the XSLT compiler that we want to build a template. Usually, whenever you see match or select attributes in XSLT, this is an XPath directive. Here we're saying "match the root-Customers element". The forward-slash is like an absolute path location - i.e. it's at the root of the document:

<xsl:template match="/Customers">

At this point, anything inside the xsl:template will be rendered whenever the compiler comes across the element specified in the match attribute. As there's only one /Customers element, this template will be processed once.

Anything inside the element that isn't an XSLT processing instruction (usually anything that does not begin with xsl:) is copied to the result document. In this case, we're taking a ul start tag over to the results:

<ul>

Next we have a xsl:for-each instruction. This is, as you might expect, a for loop. What we're doing though is telling it to select on Customer. As the XPath pointer is on /Customers, this will be any Customer element inside /Customers. The instructions inside will be executed exactly once per Customer element. Per iteration, the XPath pointer will be moved to the Customer element that's being processed:

<xsl:for-each select="Customer">

Now, a similar deal to before - we copy over an li tag:

<li>

Next, another XSLT instruction. This time, we're using xsl:value-of, which is a way of inserting the value selected into the target document. In this case, we want the value inside the Id element. As the XPath pointer is on Customer at this point, this will be the Id element inside the current Customer element, for example /Customers/Customer/Id :

<xsl:value-of select="Id"/>

After writing the value, we write the end li tag:

</li>

Now we close the xsl:for-each instruction. This is where the loop will either quit, or go back up to the start where the XPath pointer will be positioned at the next Customer element:

</xsl:for-each>

Finally, we close the ul tag (as all the Customer elements would have been processed at this point), close the template, and close the style sheet:

</ul>

</xsl:template>

</xsl:stylesheet>

What Now?

I found that the lynchpin to understanding XSLT was getting to grips with XPath. Once you start imagining the pointer down around the document through the folders and sub-folders, things get a little easier to understand. XSLT is quite dissimilar to traditional programming, which is why a lot of software developers struggle with it. One of the reasons why I built this tool was to make it easy to experiment with XSLT.

Here are some resources that I found useful:

  • http://www.w3schools.com/xsl/xsl_transformation.asp
  • http://www.vbxml.com/xsl/tutorials/intro/default.asp
  • http://hotwired.lycos.com/webmonkey/98/43/index2a.html?tw=authoring

You might also like to check out Michael Kay's "XSLT Programmers Reference". (ISBN 1861005067).

Go experiment! You'll love it!

Summary

In this two-part article we took a look at how to build some cool tools for working with XSLT. In this section, we learned how we can reuse Internet Explorer in our application and get it to render both the XML data in the XmlEditor class and also the HTML data resulting from transformations.

We also saw how to perform a simple transformation and took a look at how XPath works in the context of XSLT.

Article Information
Author | Matthew Reynolds
Technical Editor | Adam Ryland
Project Manager | Helen Cuthill
Reviewers | Vickie Pring, Eddie Chandler

If you have any questions or comments about this article, please contact the technical editor.


Cool Tool for XSLT – Part 1

Cool Tool for XSLT – Part 1
http://www.csharptoday.com/content.asp?id=1818
Article Rating: 3.5
Published on February 11, 2002
Abstract
XSLT, or “Extensible Style Language: Transformations”, is a way of transforming XML files from one format to another. XSLT has been in the limelight for a while now, as it has been a popular technology for Web site builders for changing XML data into HTML. XSLT can be a little complicated to understand. In this article, Matthew Reynolds will show you how to build a tool that can help you learn about XSLT and also use XSLT in your applications. In this first part, we’ll look at how to build a simple XML editor control and use that control in our application. In the second part, we’ll look in more detail at XSLT and see a few of the basic building blocks that are helpful in day-to-day use.
Article

Some of you reading this will be familiar with the concept of XSLT, or "Extensible Style Language: Transformations". For those of you who are not, XSLT is a way of transforming XML files from one format to another. XSLT has two main functions:

  • Changing a XML document used in business exchange from one company's version to another. (For example, renaming elements, moving elements around inside the document, removing elements you don't need, etc).
  • Changing an XML document into a HTML document for display.

XSLT has been in the limelight for a while now, as it has been a popular technology for Web site builders for changing XML data into HTML. Rather than having to write masses of script code to take the database-provided data and turn it into HTML, the data can either be stored as XML files on the database, or some databases (including SQL Server 2000) can be asked to return data as XML. In either case, XSLT can be used to put the data into HTML so that it can be rendered.

XSLT can be a little complicated to understand, mainly because programming in XSLT is not programming in the traditional sense. Most developers work on the principle of writing a method that starts from the top and runs down, making decisions as it goes. XSLT is a bit strange in that you build a file that contains a stylesheet and then ask the XSLT process to apply it to the original XML document and return the new XML document.

Whenever I've done XSLT programming in the past, I've found it a constant process of experimentation - i.e. change a little bit of the stylesheet, try the transformation, change a little more, try again, etc. In this article, I'll be showing you how to build a tool that can help you learn about XSLT and also use XSLT in your applications.

In the first part, we'll look at how to build a simple XML editor control and use that control in our application. In the second part, we'll look in more detail at XSLT and see a few of the basic building blocks that are helpful in day-to-day use.

The "XmlEditor" Control

The first thing we're going to do is to build the XmlEditor control. This control will allow us to edit the XML files. We're not going to see something as sophisticated as the VS .NET XML editor, or the editor in XML Spy (a great tool for working with XML, by the way). In fact, we're going to reuse the TextBox control as the editor control. So that the user can preview the XML or HTML document returned from the transformation, we're also going to host Internet Explorer inside of our control, as this has excellent support for showing XML data as HTML.

Interestingly, IE itself uses XSLT to convert the raw XML data into the HTML view that it uses to present XML data.

To create the control, create a new Visual C# - Windows Application project (we'll roll the control into a Windows Application project rather than a Class Library project to make this demonstration a little easier. If you want to reuse the control in your own projects, simply move the .cs file into a new project. Simple!). Call the new project Transformer.

Laying Out "Form1"

For those of you who can't stand leaving the main form as Form1, change the name of the form now. I typically don't bother for examples like these, as the forms themselves tend to just act as hosts for the controls that we build - i.e. they're not the most important part of the project. I'll leave mine as Form1.

We want to configure the form such that we have the XML file to be transformed in the top left, the XSL in the bottom left and the transformed data on the right hand side. We can do this with a combination of Panel and Splitter controls.

Follow these instructions carefully.

Add a new Panel control to the form. Set its Dock property to Left. Add a splitter control, also making sure that its Dock property is set to Left (these panels are a little hard to see).

Next, add another Panel control to the right-hand side of the form. Set its Dock property to Fill. That gets us the splitter that allows us to change the bias between the left-hand and the right-hand controls.

Now add a new Panel control to the left-hand side. Set its Dock property to Top. Add a Splitter control to the blank area in the bottom-left. The splitter will automatically dock to the left, so change its Dock property to Top. You'll see this:

Finally, add another Panel control to the region in the bottom-left. Set its Dock property to Fill. This gives us the two splitters - one to change the bias between left and right, and another to change the bias between top and bottom on the left only.

Now that we have Form1 configured, we can turn our attention to the XmlEditor control.

Building "XmlEditor"

Add a new User Control to the project. Call it XmlEditor.

XmlEditor will have three view modes - XmlEdit, XmlPreview and HtmlPreview (i've named these modes after the enumeration that we'll use to express them in code.) The XmlEdit mode will use a properly configured TextBox control for its display. XmlPreview and HtmlPreview will use Internet Explorer, contained within the control itself, for its display. When using IE, temporary files will be used to store the data and IE will be asked to display those to the user.

In addition, the editor will also have a toolbar with buttons that allow the user to check whether the XML document is well formed and also to change the view type. A menu will be added to Form1 to allow us to load and save the documents, as well as download the documents from a Web server. (we won't be able to save the documents back to a Web server.)

To the new control, add a ToolBar control and a Panel control. The ToolBar will automatically dock to the top of the control, but you'll need to manually set the Dock property of the Panel to Fill. In addition, set the Name property of the Panel control to panelMain.

The ToolBar will have two buttons: buttonCheck and buttonView. In the download package for this article you'll find some buttons to use (I used a screen capture program to snag these from other programs). To use them, you need to follow these steps:

  • Add an ImageList control to the form. Call it imagelistToolBar. Add the two images to the ImageList by using the Images property.
  • Set the TransparentColor property of the control to Magenta (Bright pink, RGB: FF00FF).
  • Set the ImageList property of the ToolBar to imagelistToolBar.
  • Set the Name property of the ToolBar to toolbar.
  • Add the two buttons. Set the name of the first button to buttonCheck and the name of the second button to buttonView.

You'll get this:

Now, build the project. Add one XmlEditor control to each of the three visible panels on Form1. Set the name of the top-left one to xmlXml, the bottom-left one to xmlXsl and the right-hand one to xmlResults. Set the Dock property of each of them to Fill.

If you run the project, you'll be able to test that the splitter controls do adjust the view properly.

Changing View Modes

To display the editor, we need to implement the View property. This will let us select from the three modes: XmlEdit, XmlPreview and HtmlPreview. In the first instance, we'll only configure XmlEdit.

To support XmlEdit, we need a properly configured TextBox control. We'll build this programmatically, rather than doing it with the editor. First of all, we need private fields in XmlEditor that allow us to hold a reference to this TextBox control and also hold a reference to the currently active control. Plus, we'll need an enumeration that gives the consumer of the control the list of available modes.

Open the code editor for XmlEditor and add these three member variables:

public class XmlEditor : System.Windows.Forms.UserControl
{
// members...
private TextBox _editor;
private Control _activeView;
private ViewMode _view = ViewMode.XmlEdit;

When the control is created, we want to programmatically create the TextBox control. We want the control to have these properties:

  • It must be able to hold multiple lines,
  • It must have vertical and horizontal scroll bars,
  • It must not word wrap,
  • It must accept the Return key and the Tab key,
  • It must use the fixed size, Courier New font (the caller can change this.)

Later we'll also programmatically create the IE control too, but for now just add this code to the constructor:

public XmlEditor()
{
// This call is
            required by the Windows.Forms Form
            Designer.
InitializeComponent();

// create the editor
            control...
_editor = new TextBox();
_editor.Multiline =
            true;
_editor.WordWrap = false;
_editor.ScrollBars =
            ScrollBars.Both;
_editor.AcceptsReturn =
            true;
_editor.AcceptsTab = true;

// configure the
            view...
ConfigureView();
}

ConfigureView will be used whenever the view mode changes - and this most certainly applies when the control is first started. It's responsible for setting up the view so that the data we see reflects the data that the user has chosen.

// ConfigureView - update the view so that
            we can see the data...
public void ConfigureView()
{
//
            what view do we need to see?
Control requiredView =
            null;
switch(View)
{
case ViewMode.XmlEdit:
requiredView
            = _editor;
break;

default:
throw new Exception("View
            mode not supported");
}

// do we need to change the
            view?
if(_activeView != requiredView)
{
// clear the
            current view...
panelMain.Controls.Clear();

// show the
            new view...
_activeView = requiredView;
_activeView.Dock =
            DockStyle.Fill;
panelMain.Controls.Add(_activeView);
}
}

Before we can test this, we need to add the View property. This property needs to be accessible from the Properties window, so we use the Browsable, Category and DesignerSerializationVisibility attributes to configure this.

[Browsable(true), Category("Appearance"),
DesignerSerializationVisibility(DesignerSerializationVisibility.Visible)]
public
            ViewMode View
{
get
{
return
            _view;
}
set
{
_view =
            value;
ConfigureView();
}
}

If you build the project now and open the Designer for Form1, you'll notice that you can indeed see TextBox controls in the editors.

In addition, if you look in the Properties window you'll find the View property. However, do not change this value, as you'll get an exception (we'll fix this later when we add IE to the control).

Editing XML

The next move is to try and edit some XML with the control. If you start the application, you'll notice that you can indeed enter text into the controls. Of course, this text doesn't have to be XML and we're not going to look in this exercise at how to build a sophisticated XML editor like the ones in VS .NET or XML Spy.

What we do need, however, is an Xml property on the control. This will let the caller (and us as we're debugging) enter some XML text into the control.

First off, add this property to XmlEditor. We'll use the Text property of the TextBox control to actually hold the data.

[Browsable(true),
            Category("Appearance"),
DesignerSerializationVisibility(DesignerSerializationVisibility.Visible)]
public
            string Xml
{
get
{
return
            _editor.Text;
}
set
{
_editor.Text =
            value;
ConfigureView();
}
}

Notice how ConfigureView is called when we set the property. We'll need to use this to update the IE view when we get around to putting IE in the control.

After a rebuild, we can set the text in the control from the Properties window. However, this isn't the most useful way to enter XML data, as the edit box is very small and only supports a single line. If we implement a Lines property on the control that returns an array of string objects, VS .NET will give us a larger box to work with. This is simpler than creating our own custom editor for the property.

The Lines property is a little tricky to implement. When we set the Lines property, we need to iterate each line and build a single string, complete with line breaks, and set the Xml property to this new string. When we request the Lines property, we need to break up the string returned in Xml by new-lines (" \n ") codes, but also remove any carriage-return (" \r ") codes. Here's the code:

[Browsable(true),
            Category("Appearance")]
public string[]
            Lines
{
get
{
// blank?
if(Xml == "")
return
            null;

// split up XML by \r codes...
string[] rawLines =
            Xml.Split('\n');
for(int n = 0; n < rawLines.Length;
            n++)
{
// do we need to
            trim?
if(rawLines[n].Substring(rawLines[n].Length - 1, 1) ==
            "\r")
rawLines[n] = rawLines[n].Substring(0, rawLines[n].Length -
            1);
}

// return...
return rawLines;
}
set
{
// anything?
if(value != null)
{
// build up a new
            value for Xml...
StringBuilder builder = new
            StringBuilder();
for(int n = 0; n < value.Length;
            n++)
{
builder.Append(value[n]);
if(n != value.Length -
            1)
builder.Append("\r\n");
}
Xml =
            builder.ToString();
}
else
Xml = "";
}
}

If you build the project, you'll be able to edit the Lines property for the controls. Add this simple XML document to the Lines property of the xmlXml editor:

Checking the XML is Well Formed

Now that we have the document, let's write some code that checks to see if the XML in the editor is well formed. We can use the System.Xml.XmlValidatingReader class to do this for us.

We will be asked to check the code in one of three ways. Firstly, the user can click on the buttonCheck button. If she does this, she'll be expecting to see a message that tells her what is wrong with the XML, if indeed there is anything wrong. Secondly, the developer may want to get a simple Boolean answer to the question, "Is the XML well formed?" Thirdly, the developer may want to ask this question and get an exception back that describes the error in more detail.

We'll add three methods to XmlEditor. CheckDocument will check the document and throw an exception if there is a problem. IsDocumentWellFormed will return True or False, depending on whether it is or isn't. Finally CheckDocumentWithReport will check the document and display a message box telling the user if the document was well formed and if it isn't, why it isn't.

Before you add the code, add namespace import declarations for System.Xml, System.Xml.XPath and System.Xml.Xsl to the top of the code listing.

CheckDocument will simply create an XmlValidatingReader and walk through each node. If it gets to the end without throwing an exception, the document is valid. If it throws an exception, the exception will contain the nature and the location of the problem.

// CheckDocument - check the
            document...
public void
            CheckDocument()
{
XmlValidatingReader reader = null;

//
            walk each node...
try
{
// create a validating
            reader...
reader = new XmlValidatingReader(Xml,
XmlNodeType.Document, null);

//
            walk...
while(true)
{
if(reader.Read() ==
            false)
break;
}
}
catch(Exception ex)
{
//
            rethrow...
throw ex;
}
finally
{
// close the
            reader?
if(reader !=
            null)
reader.Close();
}
}

When we create the XmlValidatingReader, we give it a "document fragment". This happens to be the entire document returned from the Xml property. Once we've done that, we walk each node until we get to the end or something goes wrong. We use the finally clause to close the reader if whatever case occurs.

CheckDocumentWithReport calls this method. If something goes wrong, another method called GetReportForException.

// CheckDocumentWithReport - check the document, and report error...
public string CheckDocumentWithReport()
{
string report = null;
try
{
CheckDocument();
}
catch(Exception ex)
{
// get the report...
report = GetReportForException(
"The document is not well formed.", ex);
}

// return the report...
return report;
}

GetReportForException is a static method, and we'll use this in the next part of the exercise to report on problems when we do the actual XSLT transformations.

// GetReportForException - get the report
            for the exception...
public static string
            GetReportForException(string message, Exception ex)
{
//
            create a builder...
StringBuilder builder = new
            StringBuilder();
builder.Append(message);
builder.Append("\r\n\r\n");

//
            loop
while(ex != null)
{
// do we have an XML
            exception?
if(ex is XmlException)
{
// report the line
            number...
builder.Append("Line:
            ");
builder.Append(((XmlException)ex).LineNumber.ToString());
builder.Append(",
            position:
            ");
builder.Append(((XmlException)ex).LinePosition.ToString());
builder.Append("\r\n");
}
if(ex
            is XsltCompileException)
{
// report the line
            number...
builder.Append("Line:
            ");
builder.Append(((XmlException)ex).LineNumber.ToString());
builder.Append(",
            position:
            ");
builder.Append(((XmlException)ex).LinePosition.ToString());
builder.Append("\r\n");
}
builder.Append(ex.Message);
builder.Append("\r\n\r\n");

//
            next...
ex = ex.InnerException;
}

//
            return...
return builder.ToString();
}

IsDocumentWellFormed is a simple method that relies on exceptions being thrown from CheckDocument if the document is not well formed.

// IsDocumentWellFormed - return
            true/false...
public bool
            IsDocumentWellFormed()
{
try
{
//
            check...
CheckDocument();
}
catch
{
//
            problem...
return false;
}

// ok...
return
            true;
}

Run the project now. If you click the Check button, you should be told that the document is OK. Remove the last </Customers> tag from the document, however, and you should see this:

Validating the XML

Checking that the document is well formed is not the same thing as checking whether or not the document is valid. For this kind of check, a DTD (Document Type Definition) or XSD (XML Schema Definition) file needs to be applied to the document.

We're not going to look at how to do this in this example, but if you do need to check the document against DTD or XSD files, you can use the XmlValidatingReader. To do this, use the Schemas property to add the schemas you want to check against before starting to walk the document nodes.

Menu Options

A typical design pattern for this kind of application is to have a File menu with New, Open, Save and Save As options. To save space, I'm not going to show you how to implement that in this section. However, the download package associated with this article does have this code in place so you can see how this is done.

Next Time

In this section, we took a look at how to build a basic XML Editor that reused the text editing functionality of the TextBox control. We saw how to encapsulate this functionality into a separate control, which we reused three times on a form. We also saw how to build some basic properties into the control and also how we could add functionality to check the well formed-ness of the XML in situ.

Article Information
Author | Matthew Reynolds
Technical Editor | Adam Ryland
Project Manager | Helen Cuthill
Reviewers | Vickie Pring, Eddie Chandler

If you have any questions or comments about this article, please contact the technical editor.


Transforming Formats with XSLT and .NET Object

Transforming Formats with XSLT and .NET Object
http://www.csharptoday.com/content.asp?id=1814
Article Rating: 5
Published on February 5, 2002
Abstract
XSLT is a transformation language to convert from XML to another data format. An XSLT document must be a well-formed XML document. XSLT consists of a set of elements and built-in functions. Although the built-in functions are very powerful, there do not solve all problems. For example, if you want to validate some XML elements against a Database, Active Directory, or calls to COM+ object, you will need to have extensions to XSLT. In this article, Kenny Lim explores a simple sample to transform one input format to HTML data format using XSLT transformation with a .NET object.
Article

XSLT is a powerful transformation language to convert from XML format to other data formats. One prerequisite of using XSLT is XPath. XPath is a language used to interrogate XML document structure. It is normally used to navigate and reference to a particular node or attribute within a document. Most of the time the built-in functionality of XSLT will be sufficient, but there are times that we need some functionality that is not within the XSLT language, or hard to program in XSLT. To tackle this set of problems, we need to extend XSLT to include custom .NET objects.

Why is XSLT different than conventional programming languages?

  • XSLT is template-based language

XSLT is a template-based language. The input XML document will be constantly matched against all the template criteria. It will find the best-matched template and pass those nodes into this template for processing.

  • No modifiable variables

In XSLT, there are no real variables like you would find in languages like C++, C#, or Visual Basic. There are xsl:param and xsl:variable elements, but these two can only be written to once. They cannot be modified after the initial value is set.

  • No deterministic entry-point

In conventional programming, you can have static void Main(). This will be your deterministic entry-point to the program no matter how many times this program is run. In XSLT, the starting point depends on the input coming in. Having a different XML input can result in different starting point (because of different matched template).

System Requirements

  1. Microsoft .NET framework
  2. Microsoft Windows XP Professional or Windows 2000

Extending XSLT

The ability to extend XSLT has been available in an unmanaged environment for a long time. Developers have been extending XSLT with a Jscript or VBScript scripting engine. Many developers have also done this in COM programming or Active Server Pages. All you need is Microsoft XML Parser 3.0 (MSXML3.0).

Here is an example of how to include a script in an XSLT document in an unmanaged environment (eg. ASP).

<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
xmlns:msxsl="urn:schemas-microsoft-com:xslt"
xmlns:mycode="http://kenny-test/"
version="1.0">
<msxsl:script language="JScript" implements-prefix="mycode">
function Foo() {
return "result from my script";
}
</msxsl:script>

<xsl:template match="/">
<xsl:value-of select="mycode:Foo()"/>
</xsl:template>
</xsl:stylesheet>

Listing 1

The sample code in listing 1 is a XSLT document that can be used to transform a XML document. In the sample, I have included my custom function named Foo() and execute it in a XSLT template. Because the template is match="/" , it will always be matched to any input document, and output "result from my script" . This text is returned from the function Foo() .

Listing 1 shows us how to extend XSLT with scripting language like Jscript. What is the different between extending XSLT in an unmanaged world and extending it in Microsoft .NET Framework?

  • Extending and Transforming XSLT in .NET is faster

If you transform XSLT in .NET, a transformation tree is compiled. This means it is faster, as there is no need to parse the XSL or XSL script over and over (the XSL scripting in Listing 1 is parsed whenever it is run).

  • You can add value to XSLT by passing in pure .NET object.

A new feature in .NET Framework allows you to pass a .NET object into XSLT document. This is one of the best new features for XSLT in .NET framework. This means you can extend XSLT without embedding scripts into the XSL document. This means XSLT document will just look like XSLT document (without the confusing scripts in Listing 1).

  • Take advantage of the rich Base Class Libraries of .NET Framework

If you are not familiar with the .NET Framework, it comes with a set of base class Libraries that you as a developer, designer, or architect can include (through containment or inheritance) into your application. There are class libraries that deal with text formatting up to doing raw socket programming.

The Sample Application

In order to explore the topic of "Extending XSLT with .NET object", I choose a simple example that will utilize this technology. This is not a real world example, the focus of this example is just to showcase this technique. My sample application is a typical input, process, and output application. I will start with a XML input format that looks like this:

<root>
<entry>
<ip>207.68.173.254</ip>
<time>0101020000Z</time>
<targeturi>/mydata.asmx</targeturi>
<type>0</type>
</entry>
<entry>
<ip>207.46.197.113</ip>
<time>0101020001Z</time>
<targeturi>/default.ashx</targeturi>
<type>1</type>
</entry>
<entry>
<ip>207.46.197.111</ip>
<time>0101020001Z</time>
<targeturi>/default.ashx</targeturi>
<type>1</type>
</entry>
</root>

Listing 2 (input.xml)

In the input XML document we have repeating nodes of entry . Let's assume that this is data from some tracking software in your web server. In the entry element, we have the IP address, recorded time, request types, and the URI that was requested.

The output required is a HTML document that will display this in table rows in HTML format. These rows will be showing the children of an entry element. In addition to that, we add in the host name for the IP address field through DNS (Domain Name Service) resolution. After the rows are processed, we want to have the total number of occurrences for each type (type element within entry ). The screen output for this sample application should look like Figure 1, and the HTML needed to render this screen output is display in Listing 3.

Two parts of the output require .NET extensions to work. We need DNS resolution facilities to resolve our IP address to a fully qualified host name, and we need the extension object as a storage for certain variables (remember XSLT does not have modifiable variables) to persist those values. I will discuss more on this in the transformation section below.

Figure 1

<html xmlns:Obj="urn:summary">
<body>
<h2>My Sample Report</h2>
<table cellpadding="0" cellspacing="2"
style="border-color:black;border-style:solid;border-width:1px;">
<tr style="background-color:#CCCCCC;">
<td><b>IP Address</b></td>
<td><b>Host Name</b></td>
<td><b>Target URI</b></td>
<td><b>Type</b></td>
<td><b>UTC Time</b></td>
</tr>
<tr>
<td>207.68.173.254</td>
<td>www.msn.com</td>
<td>/mydata.asmx</td>
<td>0</td>
<td>0101020000Z</td>
</tr>
<tr>
<td>207.46.197.113</td>
<td>www.microsoft.akadns.net</td>
<td>/default.ashx</td>
<td>1</td>
<td>0101020001Z</td>
</tr>
<tr>
<td>207.46.197.111</td>
<td>Unresolved</td>
<td>/default.ashx</td>
<td>1</td>
<td>0101020003Z</td>
</tr>
</table>
<div><b>Total:</b>Type 0=1,Type 1=2,</div>
</body>
</html>

Listing 3

Transformation Work

The transformation work is to convert the XML format in Listing 2 (input) to HTML output in Listing 3. To do this work I will be using XSLT in .NET and including .NET objects into XSLT at the same time. This is a very powerful technique to transform any form of data. XSLT processing in .NET framework is much better than unmanaged code transformation because it is compiled. This can yield a lot of performance gain. If you have included Jscript functions in XSLT in a scripting environment, you know it is untidy to have Jscript / VBScript included in the XSLT document. In .NET, you don't have to code inside the XSLT document anymore, you can include a pure .NET object into XSLT. This will result in modular code and will be easier to maintain in the long run.

using System;
using
            System.Collections;
using System.IO;
using
            System.Net;
using System.Text;
using System.Xml;
using
            System.Xml.Xsl;
using System.Xml.XPath;

public class
            transform
{
static void Main(string[]
            args)
{
MemoryStream ms = new
            MemoryStream();
XmlDataDocument oXmlDoc = new
            XmlDataDocument();
oXmlDoc.Load(@"input.xml");

//
            Preparing Transformation and add extension object.
XslTransform x
            = new XslTransform();
Summary h = new
            Summary();
XsltArgumentList a = new
            XsltArgumentList();
a.AddExtensionObject("urn:summary",
            h);

x.Load(@"process.xsl");

// Do the transformation
            and output goes to MemoryStream
x.Transform(oXmlDoc, a,
            ms);
ms.Position = 0;
StreamReader sr = new
            StreamReader(ms);
Console.WriteLine(sr.ReadToEnd());
}
}

public
            class Summary
{
public static Hashtable hTable = new
            Hashtable();
public string Resolve(XPathNodeIterator
            iterator)
{
string sAddress =
            "";
try
{
iterator.MoveNext();
sAddress =
            iterator.Current.Value;
IPAddress myIP =
            IPAddress.Parse(sAddress);
IPHostEntry myHost =
            Dns.GetHostByAddress(myIP);
return(myHost.HostName);
}
catch(Exception e)
{
return
            "Unresolved";
}
}

public string
            Accumulate(XPathNodeIterator
            iterator)
{
iterator.MoveNext();
string sType =
            iterator.Current.Value;
if(hTable.ContainsKey(sType))
{
int
            val = (int)hTable[sType];
val++;
hTable[sType] = val;
}
else
hTable.Add(sType, 1);
return
            sType;
}

public string GetTotal()
{
StringBuilder
            sBuilder=new StringBuilder();
IDictionaryEnumerator oEnumerator =
            hTable.GetEnumerator();
while ( oEnumerator.MoveNext()
            )
{
sBuilder.Append("Type
            ");
sBuilder.Append(oEnumerator.Key);
sBuilder.Append("=");
sBuilder.Append(oEnumerator.Value);
sBuilder.Append(",");
}
return
            sBuilder.ToString();
}
}

Listing 4 (transform.cs)

Transform.cs (Listing 4) is the source code for the transformation. There are two classes in this CS file:

  • Transform class

This is the host application that consists of the entry point Main . All the work is done in Main since this is a sample application.

  • Summary class

This is the class for the object that will be passed during XSLT transformation. This class consists of three methods that will be called from the XPath expression.

Transform Class / Main

The Tranform class consists of just one static method called Main , and this is the entry point to the program. In Main , we first instantiate MemoryStream ms and XmlDataDocument oXmlDoc . We need the MemoryStream to hold the result of the XSLT transformation, and we need XmlDataDocument to load input.xml (input file).

      MemoryStream
            ms = new MemoryStream();
XmlDataDocument oXmlDoc = new
            XmlDataDocument();
oXmlDoc.Load(@"input.xml");

Next, we instantiate the XslTransform object (to hold the XSLT document and do the transformation) and a Summary object to pass into XSLT document.

XslTransform x = new XslTransform();
Summary h = new Summary();

We need XsltArgumentList to add my custom .NET object Summary into the XSLT document. AddExtensionObject will add a new object to the XsltArgumentList and associate it with the namespace URI. In the first argument to this method, I'm specifying an identifier namespace to be used in the XSL document. The second argument is the instance of the Summary object.

XsltArgumentList a = new XsltArgumentList();
a.AddExtensionObject("urn:summary", h);

When x.Load is called, it will load process.xsl (Listing 5) from the disk to memory, compile it, and store it in a compiled style-sheet internally. The real transformation of the XML document will be done with x.Transform . The first argument is the XmlDocument that is loaded with the input data ( input.xml ). The second argument is a XsltArgumentList object. The last argument is basically a stream object where the transformation result will be stored. Next we output the result of the transformation into the console:

      x.Load(@"process.xsl");
x.Transform(oXmlDoc,
            a, ms);
ms.Position = 0;
StreamReader sr = new
            StreamReader(ms);
Console.WriteLine(sr.ReadToEnd());

Summary Class

This Summary class is passed into the XSLT document. This is a .NET class which means it can use any of the Base Class Libraries (BCL) or any third party .NET classes. This is very powerful as you can have a custom object do anything from lookup databases to perform transactions and do .NET Remoting In our example, we are using this class for storage, as an accumulator, and to use the DNS resolution functionality in System.Net.Dns class.

There are three methods in this class. The first method is called Resolve :

      public
            string Resolve(XPathNodeIterator iterator)
{
string sAddress =
            "";
try
{
iterator.MoveNext();
sAddress =
            iterator.Current.Value;
IPAddress myIP =
            IPAddress.Parse(sAddress);
IPHostEntry myHost =
            Dns.GetHostByAddress(myIP);
return(myHost.HostName);
}
catch(Exception e)
{
return
            "Unresolved";
}
}

The purpose of this method is to get the string value from the XPathNodeIterator object. This type is required in my case because I'm referencing XPath as an argument to this method. After getting the IP address in a string, we use the System.Net.Dns class to resolve the IP address field to a DNS host name and return it. If this resolution failed, we return "Unresolved" back to the XSLT processing.

The second method is Accumulate :

   public string
            Accumulate(XPathNodeIterator
            iterator)
{
iterator.MoveNext();
string sType =
            iterator.Current.Value;
if(hTable.ContainsKey(sType))
{
int
            val = (int)hTable[sType];
val++;
hTable[sType] = val;
}
else
hTable.Add(sType, 1);
return sType;
}

This method will get the Type field and check if this Type already exists in the Hashtable object ( Hashtable is an object in the class). If it's does not exist yet, we will add this type to the Hashtable. If it exists, we simple increment the value of this entry by one. The purpose of the method is to serve as storage for the XSLT transformation and as a unique counter for each input type.

The last method is GetTotal :

   public string
            GetTotal()
{
StringBuilder sBuilder=new
            StringBuilder();
IDictionaryEnumerator oEnumerator =
            hTable.GetEnumerator();
while ( oEnumerator.MoveNext()
            )
{
sBuilder.Append("Type
            ");
sBuilder.Append(oEnumerator.Key);
sBuilder.Append("=");
sBuilder.Append(oEnumerator.Value);
sBuilder.Append(",");
}
return
            sBuilder.ToString();
}

This method does not take in any argument, and will query the internal Hashtable object for the number of occurrence of each input type. This will build a string result (total number of occurrence for each input type) and return back to the XSLT processing.

XSLT

Now that we have included the code to transform the input file, we need the XSLT document to understand how the whole process works:

<?xml version="1.0" encoding="UTF-8" ?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:Obj="urn:summary">
<xsl:output method="html" omit-xml-declaration="yes" indent="yes"/>

<xsl:template match="/">
<html>
<body>
<h2>My Sample Report</h2>
<table style="border-style:solid;border-width:1px;">
<xsl:call-template name="Header"/>
<xsl:apply-templates select="/root/entry"/>
</table>
<xsl:call-template name="Summary"/>
</body>
</html>
</xsl:template>

<xsl:template name="Header">
<tr style="background-color:gray;">
<td><b>IP Address</b></td>
<td><b>Host Name</b></td>
<td><b>Target URI</b></td>
<td><b>Type</b></td>
<td><b>UTC Time</b></td>
</tr>
</xsl:template>

<xsl:template name="Summary">
<div>
<b>Total:</b>
<xsl:value-of select="Obj:GetTotal()"/>
</div>
</xsl:template>
<xsl:template match="/root/entry">
<tr>
<td><xsl:value-of select="ip"/></td>
<td><xsl:value-of select="Obj:Resolve(ip)"/></td>
<td><xsl:value-of select="targeturi"/></td>
<td><xsl:value-of select="Obj:Accumulate(type)"/></td>
<td><xsl:value-of select="time"/></td>
</tr>
</xsl:template>

</xsl:stylesheet>

Listing 5 (process.xsl)

The structure of this XSLT is very straightforward, it consist of a main xsl:stylesheet, which is the body for the document. Inside this body, we have four XSLT templates. Two of these templates will be executed when the XPath selection matches, and the other two will be executed when it is called with the xsl:call-template element. The first template will match the root, this means that this is the template to execute first since all XML documents contain "/". This template will further call the 2nd, 3rd and 4th template. (Editor's note: Do superscript for previous sentence before publication).

<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:Obj="urn:summary">

One thing to note is that the xsl:stylesheet element consists of this attribute - (xmlns:Obj="urn:summary"). This is the additional information to map the .NET object (the Summary object) into an identifier (Obj) in this XSLT document. So, whenever we need to call the Summary object we use the Obj: namespace prefix. The xmlns:xsl attribute indicates which version of XSLT this document is.

<xsl:value-of select="Obj:Resolve(ip)"/>

The above xsl-value-of element is calling the Resolve method in the Summary object and passes in the relative XPath of ip (which is equivalent to /root/entry/ip). The result of Resolve method (a string) will be output to the XSLT result tree.

The other parts that will call the Summary object are as follows:

<td><xsl:value-of select="Obj:Accumulate(type)"/></td>

and

<xsl:value-of select="Obj:GetTotal()"/>

Additional Information

You can add a .NET object into XSLT, but what about normal value-type variables like an integer or long? You can add those types into a XSLT transformation using the XsltArgumentList class. Instead of using AddExtensionObject , you need to use the AddParam method. On the XSLT document, you need the xsl:param element to get the parameter from your program.

What happens to <msxml:script>?

Msxml:script is the way to add embedded scripts for custom processing into an XSL document in the unmanaged world. Msxml:script is still supported for backward compatibility with MSXML XSLT Parser 3.0 (unmanaged). There are enhancements to msxml:script - if you transform using managed code, it is now all compiled before transformation. All the functions defined are compiled when the stylesheet is loaded. This means it is faster to transform with script in your .NET code than unmanaged code. One more enhancement is that you can use any Common Language Runtime (CLR) supported language. To do msxml:script in .NET:

  <msxsl:script language="C#"
            implements-prefix="user">
function GetValue(XPathNodeIterator
            iterator)
{
iterator.MoveNext();
return
            iterator.Current.Value;
}
</msxsl:script>

To use the embedded function GetValue , all you need is to use the namespace prefix (which is user: in this example).

<xsl:template match="/">
<xsl:value-of select="user:GetValue(.)"/>
</xsl:template>

In this embedded function, you can call a few .NET namespaces by default. The namespace that are supported by default are:

  • System
  • System.Xml
  • System.Xslt
  • System.Collections
  • System.Text
  • System.Text.RegularExpressions
  • System.Xml.XPath

To include any other namespace, you will need to define your own custom object and add it into the XSLT document.

Conclusion

Having the ability to include any custom .NET object into XSLT makes any form of data transformation possible. We are just scratching the surface of all the great work that can be done with this technology. Other benefits of this technique will be an increase in performance as compared to previous XSLT transformations. We no longer have to script in the XSLT document, the implementation is separated from the XSLT, which is much cleaner.

Article Information
Author | Kenny Lim
Technical Editor | Adam Ryland
Author Agent | Charlotte Smith
Project Manager | Helen Cuthill
Reviewers | Saurabh Nandu, Stephen Trudel

If you have any questions or comments about this article, please contact the technical editor.


Creating an ISAPI FIlter HTTP Packet Analyzer

Creating an ISAPI FIlter HTTP Packet Analyzer
http://www.csharptoday.com/content.asp?id=1719
Article Rating: 4.5
Published on December 14, 2001
Abstract
Part of the job as a software engineer is to design, code and debug HTTP/SOAP application systems. Normally, one would use Windows 2000’s Network Monitoring tool to analyze network packets to and from a machine, but this is a hassle to do every time, because by default this tool captures all network packets (including Ethernet, ATM etc) when all we may want is HTTP and the layer above that. To do this in the Network Monitor, we have to filter to only to view HTTP packets, and even then these raw packets are in hexadecimal, which is hard to read or diagnose. To solve this problem, Kenny Lim shows us how to build a tool to monitor HTTP packets.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

Part of my job as a software engineer is designing, coding and debugging my HTTP/SOAP application systems. Normally, I will use Windows 2000's Network Monitoring tool to analyze network packets to and from my machine. This is a hassle to do every time, because by default this tool will capture all network packets (including Ethernet, ATM etc) and all I want is HTTP and the layer above that. To do this in the Network Monitor, I have to filter to only to view HTTP packets, and another problem is that the raw packet (which is in hexadecimal) is hard to read or diagnose.

To solve my problem, I have designed a small analyzing system to analyze and debug my development process. There are two parts to this program:

  • An Internet Server API (ISAPI) filter written in unmanaged Visual C++ (I call this RawHTTP.dll).
  • A smart client program written in .NET Framework using C# programming language (I call this LogAnalyser.exe).

With this tool, I will have a SYSTEM TRAY icon on my machine, which will blink when a new HTTP packet is captured. Then I click on this icon, a window will be spawned and will list HTTP or SOAP packets in summary showing HTTP headers, date-time, sessions, source IP address and so on (Figure 1). If I want to look at a packet in more detail, I drill down into it by selecting a row (Figure 2).

Figure 1

Figure 2

System Requirements

This program was compiled with Microsoft Visual Studio.NET Beta 2 and run in Windows XP Professional. The C++ ISAPI (RawHTTP) is run on IIS 5.0.

The client program (LogAnalyzer) requires the .NET Framework Beta 2, and is written in C#.

Defining the Problem (Server Side)

This tool listens to all HTTP packets coming into and serving out from IIS. To do this, we have two options:

  • Use an ISAPI Filter (Native Code)
  • Use an ASP.NET HttpModule

After some fact finding, I found out that HttpModule or HttpHandler will not be able to listen to all the extensions and requests that are invalid. It is possible in IIS 6.0 by mapping all extensions to an HttpModule, but this will require all extension to be mapped to ASP.NET. This will break existing ASP or other CGI applications. Knowing this, I decided to write an ISAPI filter for this task.

ISAPI Filter (RawHTTP)

An ISAPI filter is nothing more than a conventional Win32 DLL. A DLL that work as ISAPI filter must export 2 functions - which are as follows:

GetFilterVersion

This function is the entry-point function called by IIS on this ISAPI filter. This function must be preset for the filter to work properly. Normally, this is a very good stage to add an initialization routine.

HttpFilterProc

IIS will call this function whenever the event we are interested happens. IIS will pass certain pointers to data structure to this function.

ISAPI filters are the lowest level in writing web applications, and certainly ISAPI can deliver the highest performance if carefully designed. See figure 3 for a simple illustration of where the ISAPI filter fits within IIS. When a packet comes into IIS, it will go through the ISAPI filters first, from the top of the list to the bottom. Any filter can change or discard the packet so that it won't be seen by any ISAPI filter or extension listed after it.

ISAPI extensions (also known as ISAPI applications in the IIS Manager), will not, unlike the filters, intercept every packet. They will be called only if the client browser explicitly calls it (i.e. http://localhost/isapi/myextension.dll?param=something), or if its mapped extension is being called. For example, any call to .ASP will call the asp.dll to process.

Figure 3

There are a few events that can be registered so that IIS will callback to your routine; IIS events range from controlling IIS logging, performing custom authentication, to reading raw HTTP packets. IIS events also happen in a sequence known as event notification order. For example, SF_NOTIFY_READ_RAW_DATA will happen before SF_NOTIFY_PREPROC_HEADERS. For a complete list of the event notification order for ISAPI filters, check out http://msdn.microsoft.com/library/default.asp?url=/library/en-us/iisref/html/psdk/asp/isgu744y.asp&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq.

For our filter code, we will only need 2 notifications from IIS:

  • SF_NOTIFY_READ_RAW_DATA
  • SF_NOTIFY_SEND_RAW_DATA.

SF_NOTIFY_READ_RAW_DATA is the first event notification that will happen when IIS receives the raw HTTP packet from a client browser or Internet application. SF_NOTIFY_SEND_RAW_DATA is the last event notification from IIS that happens when IIS sends HTTP packets generated by server code.

Overall Flowchart for the ISAPI (RawHTTP)

Start coding the ISAPI Filter

Start Visual Studio .NET and choose Visual C++ Projects | Win32 Project, and enter the name of your project (Figure 4). We will then see Figure 5: choose the Application Settings tab and choose DLL. Click Finish and Visual Studio.NET will generate a blank template.

Figure 4

Figure 5

Next add a DEF file into the project from Solution Explorer and name it RawHttp.DEF. The content of the file should be the following:

LIBRARY RawHttp
EXPORTS
GetFilterVersion
HttpFilterProc

This file defines the function that will be exported from this DLL. These are the two entry points required for it to be an ISAPI filter. In the dialog box above, Visual Studio actually has a wizard to generate the EXPORT function automatically. I'm not using this because it won't work in my case. The wizard will generate the __declspec(dllexport) keyword in front of our function declaration. However, ISAPI functions are declared as __stdcall. The compiler won't let us have both. To work around this, we use a DEF file.

Since we have promised to expose these two functions, let's code it. To begin, add a header (.h) file to the project, with the following content:

extern "C"
{
BOOL WINAPI GetFilterVersion(HTTP_FILTER_VERSION *pVersion);
DWORD WINAPI HttpFilterProc(HTTP_FILTER_CONTEXT *pfc, DWORD dwNotificationType, LPVOID pvNotification);
}

The above code is the function prototypes definition. Now, open RawHTTP.cpp (which has been generated by the VC++ Wizard) to add the following implementation code:

GetFilterVersion

BOOL WINAPI
            GetFilterVersion(HTTP_FILTER_VERSION *pVersion)
{
// Register
            Notifications
pVersion->dwFlags = SF_NOTIFY_ORDER_DEFAULT
            |
SF_NOTIFY_READ_RAW_DATA |
SF_NOTIFY_SEND_RAW_DATA;

//
            Setting Version and Description
pVersion->dwFilterVersion =
            HTTP_FILTER_REVISION;
strcpy(pVersion->lpszFilterDesc,
            "RawHTTP");

// Constructing Logger
pLogger = new
            Logger();

return TRUE;
}

This function will register two of the required IIS notifications, set the ISAPI version and description, and finally instantiate a global instance of Logger (Logger is a generic C++ class). The SF_NOTIFY_ORDER_DEFAULT tells IIS to load this ISAPI filter with the default order, which is Low priority in IIS 5.0. If you want to change the priority so that your filter can be called earlier, you can change this flag to SF_NOTIFY_ORDER_HIGH. When the correct notification is received by IIS, IIS will point to the HttpFilterProc function, which is shown below:

HttpFilterProc

DWORD WINAPI HttpFilterProc(HTTP_FILTER_CONTEXT *pFilterContext, DWORD dwNotificationType, LPVOID pvNotification)
{
DWORD dwReturn = SF_STATUS_REQ_NEXT_NOTIFICATION;

switch(dwNotificationType)
{
case SF_NOTIFY_READ_RAW_DATA:
dwReturn = pLogger->ProcessMessage(pFilterContext,
(HTTP_FILTER_RAW_DATA *)pvNotification, Logger::READ);
break;

case SF_NOTIFY_SEND_RAW_DATA:
dwReturn = pLogger->ProcessMessage(pFilterContext,
(HTTP_FILTER_RAW_DATA *)pvNotification, Logger::WRITE);
break;

default: break;
}
return dwReturn;
}

In this function, we have a switch statement to check if the packet is SF_NOTIFY_SEND_RAW_DATA or SF_NOTIFY_READ_RAW_DATA. It will then pass to the Logger class to perform more work.

Logger Class

Our Logger class will be used by this ISAPI to perform all the work. To add a generic class to the project, in Visual Studio .NET, right click on the project and add a class. Choose Generic C++ Class and Open (Figure 6).

Figure 6

The next dialog wizard will appear (Figure 7), enter Logger into the class name field and click Finish. This will generate the skeleton code of Logger.cpp and Logger.h.

Figure 7

Logger.h

The first section of this header file defines the constants for the class. Hard-coding constant values in the source code is a good example. In a real-world application, it is better to have all the constants loaded up from a configuration file.

#define OFFSET_IDENTIFIER 5
#define OFFSET_START 6
#define BLOCK_SIZE 100
#define GUID_SIZE 36
#define LOCALHOST "127.0.0.1"
#define COMMAND_IDENTIFIER "HGI8H"
#define COMMAND_START_IDENTIFIER '*'
#define COMMAND_STOP_IDENTIFIER 'B'
#define COMMAND_START_PATH "C:\\HTTPRAW\\START.KEY"
#define COMMAND_STOP_PATH "C:\\HTTPRAW\\STOP.KEY"
#define LOG_PATH "C:\\HTTPRAW\\LOG.TXT"

Next we have two structures that will hold valuable information for the class. The ContextStore structure is used to store a session number that relates to a session in IIS. Since IIS can use different threads to service the same request and response, this SessionNo will be the key relating all the conversations for a request-response from the browser/application to IIS and back.

InfoStore is the structure that stores information to be logged. Information such as source IP address, datetime, request direction, and HTTP headers are stored in this structure. In our InfoStore structure, we have a constructor and destructor for the structure. In the constructor, we make sure we have initialized our variable. In the destructor, we will free the memory allocated.

typedef struct ContextStore { DWORD
            SessionNo; } SContextStore;

typedef struct InfoStore
{
const static int c_IPAddressLen = 20;
const static int
            c_DetailTimeLen = 20;
const static int c_MillisecondLen =
            5;
const static int c_SessionNoLen = 20;
const static int
            c_DirectionLen = 10;

InfoStore(){
memset(m_IPAddress,
            NULL, c_IPAddressLen);
memset(m_DetailTime, NULL,
            c_DetailTimeLen);
memset(m_Millisecond, NULL,
            c_MillisecondLen);
memset(m_SessionNo, NULL,
            c_SessionNoLen);
memset(m_Direction, NULL,
            c_DirectionLen);
m_Buffer =
            NULL;
}

~InfoStore()
{
if(m_Buffer)
delete []
            m_Buffer;
}
char m_IPAddress[c_IPAddressLen];
char
            m_DetailTime[c_DetailTimeLen];
char
            m_Millisecond[c_MillisecondLen];
char
            m_SessionNo[c_SessionNoLen];
char
            m_Direction[c_DirectionLen];
char * m_Buffer;
long
            m_BufferLen;
} SInfoStore;

The following is the Logger class definition. The Critical Section (m_Section) is used to control synchronization of multiple threads, to stop them from corrupting the log file. Synchronization is needed because ISAPIs are implicitly multi-thread. FilterAction is a C++ enumeration to be passed to a function and differentiate between READ and WRITE. The member field m_IsOperating stores the state if the logging is turned off or on. The member functions will be explained in more detail.

class Logger
{
public:
Logger();
~Logger();

enum FilterAction { READ, WRITE };
DWORD ProcessMessage(HTTP_FILTER_CONTEXT *pFilterContext,
HTTP_FILTER_RAW_DATA *pFilterRawData, FilterAction f);

private:
BOOL CheckHttpCommand(SInfoStore *pInfo);
char * ReadContent(char *pFileName);
SContextStore * ReadContextStore(HTTP_FILTER_CONTEXT
*pFilterContext, FilterAction f);
BOOL ReadInformation(SInfoStore *pInfo, HTTP_FILTER_CONTEXT
*pFilterContext, FilterAction f);
BOOL WriteLog(SInfoStore *pInfo);
BOOL WriteDetailLog(SInfoStore *pInfo);
DWORD ReadRequestPosition(SInfoStore *pInfo);

CRITICAL_SECTION m_Section;
BOOL m_IsOperating;
};

Logger.cpp

Constructor & Destructor

When the Logger is instantiated, the constructor will be called. In the constructor, we initialize the Critical Section to be used as a synchronization object for all the servicing threads. When the object is done, the destructor will be called. When this happens, it will delete the same Critical Section.

#include
            "stdafx.h"

Logger::Logger()
{
// if you don't want your
            ISAPI to be run by default, uncomment the
// following
            statement.
// m_IsOperating =
            false;
::InitializeCriticalSection(&m_Section);
}

Logger::~Logger()
{
::DeleteCriticalSection(&m_Section);
}

Logger::ProcessMessage

ProcessMessage is the member function that is called from HttpFilterProc. This method will be called regardless of READ or SEND notification. To differentiate the caller context, we use a FilterAction enumeration. Along with this enumeration, this method accepts the standard structure from IIS.

DWORD Logger::ProcessMessage(HTTP_FILTER_CONTEXT *pFilterContext, HTTP_FILTER_RAW_DATA *pFilterRawData, FilterAction f)

{
DWORD dwTemp;
DWORD dwReturn = SF_STATUS_REQ_NEXT_NOTIFICATION;
// Code Continues below .
}

Part of the logic of ProcessMessage is to instantiate a SInfoStore structure (look at logger.h), this will store information about the packet.

Next we attempt to create a block of memory to store the incoming data. We won't define a fixed size array to store this, because we want to make sure we have no buffer overrun problem in this project. Buffer overrun is the number one cause of problems like downtime or hacking. When someone knows that there is a flaw in the buffer management (for example, in using a fixed size array), they can transmit more bytes to the server and ultimately take over the IIS process (inetinfo.exe) by injecting malicious code.

Check if the memory allocation is successful; if not, abort the session. If the allocation is ok, we can copy the packet buffer and source IP address into our structure and then check if it is a command from our smart client, LogAnalyzer (for more information about this routine, look into the CheckHttpCommand member function).

If the packet is not a command, the code will check whether the current state (m_IsOperating) is on or off. If it's on, we can proceed to read the remaining information like datetime and session into our structure (SInfoStore). With all the information populated into the structure, we can then write the information to the log file. After writing to the log file, we can leave the critical section so that other threads can get into the log file and write as well. Finally, we de-allocate the structure (SInfoStore), which will then call its destructor.

SInfoStore *pInfo = new
            SInfoStore();
if(pInfo)
{
pInfo->m_Buffer = new
            char[pFilterRawData->cbInBuffer+1];
if(pInfo->m_Buffer)
{
memcpy(pInfo->m_Buffer,
            pFilterRawData->pvInData,
pFilterRawData->cbInBuffer);
pInfo->m_Buffer[pFilterRawData->cbInData]
            = NULL;
pInfo->m_BufferLen =
            pFilterRawData->cbInData;

::EnterCriticalSection(&m_Section);

dwTemp
            =
            pInfo->c_IPAddressLen;
pFilterContext->GetServerVariable(pFilterContext,
            "REMOTE_ADDR",
pInfo->m_IPAddress,
            &dwTemp);
if(!this->CheckHttpCommand(pInfo)){
if(this->m_IsOperating){
this->ReadInformation(pInfo,
            pFilterContext,
            f);
this->WriteLog(pInfo);
}
}
else
dwReturn =
            SF_STATUS_REQ_FINISHED;

::LeaveCriticalSection(&m_Section);

}
}
delete
            pInfo;
return dwReturn;
}

Logger::CheckHttpCommand

One of the features of our tool is the ability to control the filter from a smart client (LogAnalyzer). For example, we can turn off the packet logging when we're not testing a web application. To achieve this, the ProcessMessage member function will call CheckHttpCommand for every packet that comes in. The algorithm is simple. We first check if the request is coming from localhost. If it's not, return immediately. If it is from localhost, the function will then check if an identifier exists in the HTTP header (COMMAND_IDENTIFIER). If this string exists in the packet, it will go further down to check if it's a START (COMMAND_START_IDENTIFIER) command or STOP command (COMMAND_STOP_IDENTIFIER). Finally it will check if the key is matching correctly. The mechanism used for the key is that the client will generate a GUID at the time of the request (START or STOP) and then write the GUID into a file and send it as part of the request command. This method will compare the GUID from the file and the GUID sent up with the request. If they match, the command is authenticated.

BOOL Logger::CheckHttpCommand(SInfoStore
            *pInfo)
{
BOOL bReturn =
            FALSE;
if(strcmp(pInfo->m_IPAddress,
            LOCALHOST)==0)
{
{
char char *pBlock = new
            char[BLOCK_SIZE+1];
if(pBlock){
if(pBlock)
{
strncpy(pBlock,
            pInfo->m_Buffer, BLOCK_SIZE);
pBlock[BLOCK_SIZE] =
            NULL;
char *p = ::strstr(pBlock,
            COMMAND_IDENTIFIER);
if(p){
char *pKey;
char *pPos =
            p+6;
if(*(p+5) ==
            COMMAND_START_IDENTIFIER){
if(p)
{
char *pKey;
char
            *pPos = p+OFFSET_START;
if(*(p+OFFSET_IDENTIFIER) ==
            COMMAND_START_IDENTIFIER)
{
pKey =
            this->ReadContent(COMMAND_START_PATH);
if(pKey){
if(pKey)
{
if(::strncmp(pKey,
            pPos, GUID_SIZE) == 0){
this->m_IsOperating = TRUE;
bReturn
            = TRUE;
}
delete[] pKey;
}
}
else if(*(p+5) ==
            COMMAND_STOP_IDENTIFIER){
{
this->m_IsOperating =
            TRUE;
bReturn = TRUE;
}
delete[] pKey;
}
}
else
            if(*(p+OFFSET_IDENTIFIER) == COMMAND_STOP_IDENTIFIER)
{
pKey
            pKey pKey = this->ReadContent(COMMAND_STOP_PATH);
if(pKey){
            if(pKey)
{
if(::strncmp(pKey, if(::strncmp(pKey, pPos,
            GUID_SIZE) == 0){
0)
this->m_IsOperating =
            FALSE;
bReturn = TRUE;
}

delete[]
            pKey;
}
}
}
delete[]
            pBlock;
}
}
{
this->m_IsOperating = FALSE;
bReturn
            = TRUE;
}
delete[] pKey;
}
}
}
delete[]
            pBlock;
}
}
return return bReturn;
}

Logger::ReadContent

ReadContent is just a helper function that will return the content of a file. This function is use to fetch the GUID from the file.

char * Logger::ReadContent(char
            *pFileName)
{
FILE *fptr = fopen(pFileName,
            "r");
if(fptr){
char *pKey = new
            char[GUID_SIZE+1];
if(pKey)
{
fgets(pKey, GUID_SIZE+1,
            fptr);
pKey[GUID_SIZE] = NULL;
fclose(fptr);
return
            pKey;
}
}
return NULL;
}

Logger::ReadContextStore

ReadContextStore is invoked by ReadInformation. The purpose of this method is to generate our session number when it is a request (READ). To do so, we have to use AllocMem from ISAPI to allocate memory for our data. When we use AllocMem, IIS will handle the memory management for the allocated memory. It will then stuff into pFilterContext. This way, we can set the context (session number) throughout the same request-response. If it's a reply from the IIS (SEND), we can just grab the information from pFilterContext.

SContextStore *
            Logger::ReadContextStore(HTTP_FILTER_CONTEXT
*pFilterContext,
            FilterAction f)
{
SContextStore *pContext = NULL;
if(f ==
            FilterAction::READ){
if(f == FilterAction::READ)
{
pContext
            = (SContextStore *)pFilterContext->AllocMem(pFilterContext,
(pFilterContext, sizeof(SContextStore),
            0);
pContext->SessionNo =
            ::GetTickCount();
pContext->CountNo =
            1;
pFilterContext->pFilterContext =
            pContext;
}
else
{
pContext = (SContextStore
            *)pFilterContext->pFilterContext;
pContext->CountNo =
            pContext->CountNo+1;
pFilterContext->pFilterContext =
            pContext;
}
return pContext;
}

Logger::ReadInformation

ReadInformation will be called by ProcessMessage to populate the SInfoStore structure:

BOOL Logger::ReadInformation(SInfoStore
            *pInfo, HTTP_FILTER_CONTEXT
*pFilterContext, FilterAction
            f)
{
SYSTEMTIME
            t;
::GetSystemTime(&t);
::sprintf(pInfo->m_DetailTime,
            "%02d:%02d:%02d.%d",
t.wHour, t.wMinute, t.wSecond,
            t.wMilliseconds);
::sprintf(pInfo->m_Millisecond, "%d%c",
            t.wMilliseconds, NULL);
::sprintf(pInfo->m_SessionNo, "%d",
            SYSTEMTIME t;
SContextStore *pContext =
            this->ReadContextStore(pFilterContext,
            f)->SessionNo);
f);
::GetSystemTime(&t);
::sprintf(pInfo->m_DetailTime,
            "%02d:%02d:%02d.%d%d", t.wHour, t.wMinute,
t.wSecond,
            t.wMilliseconds,
            pContext->CountNo);
::sprintf(pInfo->m_Millisecond,
            "%d%d%c", t.wMilliseconds, pContext->CountNo,
            NULL);
::sprintf(pInfo->m_SessionNo, "%d",
            pContext->SessionNo);
if(f if(f ==
            FilterAction::READ)
strcpy(pInfo->m_Direction,
            "Incoming");
else
strcpy(pInfo->m_Direction,
            "Outgoing");
return TRUE;
}
strcpy(pInfo->m_Direction,
            "Incoming");
else
strcpy(pInfo->m_Direction,
            "Outgoing");
return TRUE;
}

Logger::WriteLog

WriteLog is used to append to the log file. The log file is created in shared mode, and we set the file pointer to the end before we begin writing to it. In the body, we use the SInfoStore structure and write XML element strings into the log file. However, this function will just log summary information. The generated XML file will be loaded up by the client program as a data grid. The raw detail packet information is stored in the detail file by WriteDetailFile (below):

BOOL Logger::WriteLog(SInfoStore
            *pInfo)
{
DWORD dwTemp;
HANDLE hFile =
            ::CreateFile(LOG_PATH, GENERIC_WRITE,
FILE_SHARE_READ, NULL,
            OPEN_ALWAYS, FILE_ATTRIBUTE_NORMAL, NULL);
if(hFile !=
            INVALID_HANDLE_VALUE){
::SetFilePointer(hFile, 0, 0,
            FILE_END)

::WriteFile(hFile, "<call>", 6, &dwTemp,
            NULL);

::WriteFile(hFile, "<session>", 9, &dwTemp,
            NULL);
::WriteFile(hFile, pInfo->m_SessionNo,
strlen(pInfo->m_SessionNo), &dwTemp,
            NULL);
::WriteFile(hFile, "</session>", 10, &dwTemp,
            NULL);

::WriteFile(hFile, "<time>", 6, &dwTemp,
            NULL);
::WriteFile(hFile, pInfo->m_DetailTime,
strlen(pInfo->m_DetailTime), &dwTemp,
            NULL);
::WriteFile(hFile, "</time>", 7, &dwTemp,
            NULL);

::WriteFile(hFile, "<ip>", 4, &dwTemp,
            NULL);
::WriteFile(hFile, pInfo->m_IPAddress,
strlen(pInfo->m_IPAddress), &dwTemp,
            NULL);
::WriteFile(hFile, "</ip>", 5, &dwTemp,
            NULL);

::WriteFile(hFile, "<dir>", 5, &dwTemp,
            NULL);
::WriteFile(hFile, pInfo->m_Direction,
strlen(pInfo->m_Direction), &dwTemp,
            NULL);
::WriteFile(hFile, "</dir>", 6, &dwTemp,
            NULL);

DWORD dwPos =
            this->ReadRequestPosition(pInfo);
if(dwPos ==
            0)
{
::WriteFile(hFile, "<request>", 9, &dwTemp,
            NULL);
::WriteFile(hFile, "</request>", 10, &dwTemp,
            NULL);
this->WriteDetailLog(pInfo);
}
else
{
::WriteFile(hFile,
            "<request><![CDATA[", 18, &dwTemp,
            NULL);
::WriteFile(hFile, pInfo->m_Buffer, dwPos, &dwTemp,
            NULL);
::WriteFile(hFile, "]]></request>", 13,
            &dwTemp,
            NULL);
this->WriteDetailLog(pInfo);
}

::WriteFile(hFile,
            "</call>", 7, &dwTemp,
            NULL);

CloseHandle(hFile);
return TRUE;
}
return
            FALSE;
}

BOOL Logger::WriteDetailLog(SInfoStore
            *pInfo)
{
BOOL bReturn = FALSE;
DWORD dwTemp;
char
            *pDetailFile = new char[MAX_PATH];
if(pDetailFile)
            {
memst(pDetailFile, 0, MAX_PATH);
sprintf(pDetailFile,
            "C:\\HTTPRAW\\%s%s%s.txt",
            if(pDetailFile)
{
memset(pDetailFile, 0,
            MAX_PATH);
sprintf(pDetailFile, "C:\\HTTPRAW\\%s%s%s.txt",
            pInfo->m_SessionNo,
pInfo->m_Millisecond,
            pInfo->m_Direction);
HANDLE hDetail = CreateFile(pDetailFile,
            GENERIC_WRITE, FILE_SHARE_READ,
NULL, OPEN_ALWAYS,
            FILE_ATTRIBUTE_NORMAL, NULL);
if(hDetail){
WriteFile(hDetail,
            pInfo->m_Buffer, (DWORD)pInfo->m_BufferLen,
            (DWORD)strlen(pInfo->m_Buffer), &dwTemp,
            NULL);
&dwTemp, NULL);
CloseHandle(hDetail);
bReturn =
            TRUE;
}
delete [] pDetailFile;
}
return
            bReturn;
}
if(hDetail)
{
WriteFile(hDetail,
            pInfo->m_Buffer, (DWORD)pInfo->m_BufferLen, &dwTemp,
            NULL);
CloseHandle(hDetail);
bReturn = TRUE;
}
delete []
            pDetailFile;
}
return bReturn;
}

StdAfx.h

Finally, be sure to add the required header file to the project before compiling. Httpfilt.h is required for ISAPI related functions. String.h is required for string manipulation function such as sprintf.

#pragma once

// Exclude rarely-used stuff from Windows headers
#define WIN32_LEAN_AND_MEAN

// Windows Header Files:
#include <windows.h>
#include <httpfilt.h>
#include <string.h>
#include <stdio.h>
#include "Logger.h"

Compiling & Deploying the DLL

Now all the source code is in place, we can build our DLL. By default, Visual Studio will generate the DLL in DEBUG mode, which is fine, as we may wish to debug our ISAPI. For more information about debugging ISAPI, check out http://support.microsoft.com/default.aspx?scid=kb;EN-US;Q183480&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq.

After building the project, we'll have RawHttp.dll in a debug directory. The next step is to install this DLL in our ISAPI. To do this, open up IIS Manager, right click on the server name, and choose Properties (Figure 8).

In the IIS Master Properties dialog box, choose Edit and then the ISAPI filter tab (Figure 9). Click the Add button and specify the ISAPI name and path to our DLL (Figure 10).

Figure 8

Figure 9

Figure 10

Once we have installed the ISAPI filter, open a command window and run iisreset. This will restart the IIS service and load up the RawHTTP.dll. Once we have done this, create a folder c:\HttpRaw. This is the directory used by this tool.

Now we can run a browser or some other Web Stress utility like WAST to hit our web site. One way that I use to check for memory leaks in my DLL is to use the Performance Monitor, select inetinfo.exe, and choose the Private Bytes counter. By having a stress utility run over the course of a day or two, I can monitor for a memory leak happening.

Windows Form Client (HTTPAnalyzer)

The ISAPI Filter will log summary and detail packet information into files. In this section, we will explore how we are going to utilize those log files. The LogAnalyzer program is a Visual Studio .NET Windows Forms solution consisting of 2 forms. The main form is called FORM1 (Figure 1) and the dialog form is called FORM2 (Figure 2).

FORM1

FORM1 is the main startup window for the client program. This window shows summary HTTP packet information in the log files. To start explaining this window, we should start with the Load event.

Form1_Load Event

The Load event is a good place to initialize resources. In this event, we load icon files into Icon objects. This icon will be used to display the icon on the system tray area:

      private
            void Form1_Load(object sender, System.EventArgs e)
{
//
            loading up icon files to be used in the system tray.
oIconNormal
            = new Icon(Application.StartupPath + "\\icon0.ico");
oIconChange1 = new Icon(Application.StartupPath +
            "\\icon1.ico");
oIconChange2 = new Icon(Application.StartupPath +
            "\\icon2.ico");
oIconDisable = new Icon(Application.StartupPath +
            "\\icon3.ico");

In this section, we instantiate a FileSystemWatcher object. The FileSystemWatcher is used to notify the client program when certain filesystem events like file size changes happen. In our case, we tell it to monitor for log.txt file (the log file that our RawHttp ISAPI writes to) and tell it to notify our callback function (OnChanged) when an event happens.

         //
            registering file system watcher.
oFSWatch = new
            FileSystemWatcher();
oFSWatch.Path =
            sCfgPath;
oFSWatch.NotifyFilter =
            NotifyFilters.LastWrite;
oFSWatch.Filter =
            sCfgFileName;
oFSWatch.Changed += new
            FileSystemEventHandler(OnChanged);
oFSWatch.EnableRaisingEvents =
            true;

In this section, we set up the system tray area. We instantiate the NotifyIcon object, setting up the default icon, text, and add an event handler (OnClick). We also create a context (or right click) menu and add a bunch of menu items. Each menu item has an event handler associated with it:

         //
            setting up NotifyIcon and context menu.
tray = new
            NotifyIcon();
ContextMenu ctx = new ContextMenu();
oMenuStatus
            = new MenuItem("Disable Service", new
            EventHandler(OnStatus));
ctx.MenuItems.Add(oMenuStatus);
ctx.MenuItems.Add(new
            MenuItem("Refresh", new
            EventHandler(OnRefresh)));
ctx.MenuItems.Add(new
            MenuItem("Clear/Reset", new
            EventHandler(OnClear)));
ctx.MenuItems.Add(new
            MenuItem("-"));
ctx.MenuItems.Add(new MenuItem("Exit", new
            EventHandler(OnExit)));
tray.ContextMenu = ctx;
tray.Icon =
            oIconNormal;
tray.Text = "HTTP Analyser, no
            packet";
tray.Click += new EventHandler(OnClick);
tray.Visible
            = true;
oBtnStatus.Text = oMenuStatus.Text;

In this section, we send an HTTP GET request to our HttpRaw ISAPI filter. This is a command packet to tell HttpRaw to start logging now the client analyzer has started. To send the HTTP request, we use the WebRequest class. PrepareStartKey is a method that will both create a start GUID and return it. The catch block is empty because the ISAPI won't return status code 200 for the GET request. Therefore, GetResponse will always throw an exception.

         try
{
WebRequest
            req = WebRequest.Create(sStartURL +
            PrepareStartKey());
req.GetResponse();
}
// the catch block
            is empty because ISAPI will never return any response for
            command.
catch(Exception ex){}

In this section, we are setting some properties for the DataGrid object. At the end of this event, we call Reload.

         //
            setting some properties for
            DataGrid.
oDataGrid.PreferredColumnWidth =
            150;
oDataGrid.ReadOnly = true;
oDataGrid.ParentRowsVisible =
            false;
this.Reload();
}

Reload()

The purpose of Reload is to populate the DataGrid in FORM1. To do this, we use data binding. This method is called at the end of the form1_load event, the Refresh menu item and the Refresh button:

private void Reload()
{
try
{

The first step to check in this method is whether log.txt exists or not. If this file does not exist, there's no point proceeding because there's no log file to use.

if(File.Exists("c:\\httpraw\\log.txt"))
{

When we know log.txt exists, we create a memory stream (256KB) as a buffer to hold the final XML used to populate the DataSet object. All the code does is insert a <root> element before and after those XML elements from log.txt.

               DataSet
            oDoc = new DataSet();
MemoryStream ms = new
            MemoryStream(MEMORYSIZE); // 256KB memory
StreamWriter s = new
            StreamWriter(ms);

StreamReader oStreamXml = new
            StreamReader("c:\\httpraw\\log.txt");
s.Write("<root>");
s.Write(oStreamXml.ReadToEnd());
oStreamXml.Close();
oStreamXml
            = null;
s.Write("</root>");
s.Flush();

To ensure that the memory stream can be read into the DataSet, we have to set the position to 0. This will set the internal pointer of the memory stream to the start position. Next, we reading the DataSet object and bind the DataSet with the DataGrid object. This will display the summary HTTP packet in the DataGrid.

               ms.Position
            = 0;
oStreamXml = new
            StreamReader(ms);
oDoc.ReadXml(oStreamXml);
oStreamXml.Close();
oStreamXml
            = null;
oDataGrid.DataSource =
            oDoc;
oDataGrid.NavigateTo(0,"call");
}
}
catch(Exception
            e)
{
MessageBox.Show("System Error or ISAPI not
            running.");
this.Dispose();
}
}

OnChanged

The OnChanged method will be called when the log.txt is changed. This method updates the text in the window to tell the user that a new packet has arrived. In addition, we animate an icon to get attention from the user.

   private void
            OnChanged(object source, FileSystemEventArgs e)
{
tray.Icon =
            oIconChange1;
tray.Text = "Last HTTP capture : " +
            DateTime.Now.ToLongTimeString();
oStatusBar.Text = "New Data
            Available, click Refresh. " +
            tray.Text;
Thread.Sleep(ANIMATIONDELAY); // 25 ms
tray.Icon =
            oIconChange2;
Thread.Sleep(ANIMATIONDELAY); // 25 ms
tray.Icon
            = oIconNormal;
}

oBtnClear_Click

This event handler will be fired when the user click on the CLEAR button. This event handler will enumerate through all the .txt files and delete them from the c:\httpraw folder. With this functionality, the working folder can be cleaned up when the user decides to do so. After calling Delete() on each FileInfo object, we'll refresh the DataGrid:

   private void
            oBtnClear_Click(object sender, System.EventArgs
            e)
{
try
{
DirectoryInfo di = new
            DirectoryInfo(@"c:\httpraw\");
FileInfo[] fi =
            di.GetFiles("*.txt");
int nLength = fi.Length;
for(int
            i=0;i<nLength;i++)
fi[i].Delete();
oDataGrid.DataSource =
            null;
oDataGrid.Refresh();
}
catch(Exception
            ex)
{
MessageBox.Show("Error : Try again later, some files are
            being used by other process.");
}
}

FORM2

FORM2 is a dialog box that will be created when the user click on a row of summary packet to drill down. FORM2 is just a content viewer for the detail packet information. There are a few tabs in FORM2, each tab showing different encoding for the same data. Here are a few important methods in FORM2.

GetPacketContent

The purpose of this method is to read the content from a detailed log file (written by HttpRaw) and display it in the default tab (ASCII). This method will start to read the file using BinaryReader. Next the binary byte will be converted to a character and appended to StringBuilder. When all the bytes from the BinaryReader are converted, it will show the string in a text box (oTextAscii). Please note that because the TextBox control cannot display the character NULL, this method removes any NULL character in the string. This is a limitation of this method.

     private void
            GetPacketContent(string sFileName)
{
try
{
// opening
            the detailed log file.
FileStream fs = new
            FileStream("c:\\httpraw\\" + sFileName, FileMode.Open,
            FileAccess.Read);
if(fs!=null)
{
BinaryReader bs = new
            BinaryReader(fs);
// only handle first 256kb of
            content.
StringBuilder sBuilder = new
            StringBuilder(READSIZE);
m_InternalBytes =
            bs.ReadBytes(READSIZE);
int nLen = m_InternalBytes.Length;
//
            for loop to ignore character NULL and append to
            StringBuilder.
for(int
            i=0;i<nLen;i++)
if(m_InternalBytes[i] !=
            0)
sBuilder.Append(Convert.ToChar(m_InternalBytes[i]));
//
            Display the String from StringBuilder to the
            TextBox.
oTextAscii.Text =
            sBuilder.ToString();
bs.Close();
fs.Close();
}
else
oTextAscii.Text
            = "No detail packet for this record.";
}
catch(Exception
            e)
{
oTextAscii.Text = "No detail packet for this
            record.";
}
}

SelecedtIndexChanged

This event handler will be called when the user changes from one tab to another. This event handler will convert from ASCII text into various types of format to display in a text box.

   private void
            tabControl1_SelectedIndexChanged(object sender, System.EventArgs
            e)
{
// Hexadecimal Tab
if(tabControl1.SelectedIndex ==
            1)
{
string sTemp;
char [] c =
            oTextAscii.Text.ToCharArray();
StringBuilder sBuilder = new
            StringBuilder();
for(int i=0;i<c.Length;i++)
{
//
            Convert string to hexadecimal (base 16)
sTemp =
            Convert.ToString(Convert.ToInt32(c[i]), 16);
if(sTemp.Length ==
            1)
sBuilder.Append("0");
sBuilder.Append(sTemp);
sBuilder.Append("
            ");
}
oTextHex.Text = sBuilder.ToString();
}
// Decimal
            Tab
else if(tabControl1.SelectedIndex == 2)
{
string
            sTemp;
char [] c =
            oTextAscii.Text.ToCharArray();
StringBuilder sBuilder = new
            StringBuilder();
for(int i=0;i<c.Length;i++)
{
//
            Convert string to decimal (base 10)
sTemp =
            Convert.ToString(Convert.ToInt32(c[i]), 10);
if(sTemp.Length ==
            1)
sBuilder.Append("00");
else if(sTemp.Length ==
            2)
sBuilder.Append("0");
sBuilder.Append(sTemp);
sBuilder.Append("
            ");
}
oTextDecimal.Text = sBuilder.ToString();
}
//
            Base64 Tab
else if(tabControl1.SelectedIndex == 3)
{
//
            Convert string to Base64 string
oTextBase64.Text =
            Convert.ToBase64String(m_InternalBytes);
}
}

Limitations and Future Work

The design of this filter is to be used as a personal HTTP/SOAP packet analyzer. This work is not intended to be installed on any production environment, because it logs a lot of details about all incoming and outgoing packets.

Secondly, we use the file system to store XML documents about the HTTP packets. To improve performance, we could use a named pipe to stream data from the ISAPI filter to the client program without I/O to the disk. Writing to memory is much faster than to disk.

Conclusion

The motivation of this tool is to make the debugging and analysis of HTTP or SOAP packets much easier than it is today. By having this tool, I can spot a problem much earlier at the packet level. In addition, this tool allows me to know which culprit machine is sending those "Code Red" packets to my computer ;-)

If you'd like to discuss a broader range of C#Today topics and issues, or if you have any C# related questions you need answers to, please visit our online resource centres at : http://p2p.wrox.com/list.asp?list=csharptoday_discuss&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq and http://p2p.wrox.com/list.asp?list=c_sharp&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq


Using Remote Access Services in C#

Using Remote Access Services in C#
http://www.csharptoday.com/content.asp?id=1712
Article Rating: 4.5
Published on December 5, 2001
Abstract
The Microsoft .NET Framework does not presently provide any base classes that access the Remote Access Services (RAS) API. RAS API functions can access many features of RAS programmatically, such as creating an application that automatically connects to the Internet or a remote machine. Since the RAS API is designed to be used primarily by C/C++ developers, an extra effort is required to make these API calls work in the .NET Framework using C#. However, only a basic knowledge of C/C++ is required for this article. In this article, Rino Ianotta will demonstrate how to create a RAS .NET component that retrieves RAS phone entries, makes an asynchronous connection to a RAS server, disconnects from the server, and gets the connection status.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

The Microsoft .NET Framework does not presently provide any base classes that access the Remote Access Services (RAS) API. RAS API functions can access many features of RAS programmatically, such as creating an application that automatically connects to the Internet or a remote machine. Since the RAS API is designed to be used primarily by C/C++ developers, an extra effort is required to make these API calls work in the .NET Framework using C#. However, only a basic knowledge of C/C++ is required for this article. In this article, we will describe how to create a managed definition of all used RAS API functions and structures and how to wrap these calls within a managed class. The article will conclude by illustrating the use of the RAS component in a Windows application.

PInvoke

Before diving into how to define the various RAS API functions and structures in managed code, it is important to understand how the .NET framework interoperates with unmanaged code. The code within an unmanaged DLL is not provided within C#, so you must identify the signature of the method to the compiler. This includes information about any return values, and how the parameters are to be marshaled to the DLL. To accomplish this, we need to use the System.Runtime.InteropServices namespace. The most important and notable classes from this namespace are the Marshal and DllImportAttribute classes. The Marshal class contains methods for handling memory operations which we will find useful, while the DllImportAttribute class will be used to help define an unmanaged DLL.

RasAPI Component

By creating a component that wraps the RAS API, we can reuse it in any application that requires this functionality. The following class diagram illustrates the component design:

We will go through the code to build the component, and then illustrate how to access it from an application. The first step would be to create a Visual Studio Class Library Project. Open up Visual Studio and select New Project. You will be prompted with the New Project Dialog, where you should select C# Class Library. Name the project RasManager.

Add a class to your project and name it RasAPI.cs. This file will contain the code that will wrap all the calls to the unmanaged RAS API functions.

RasDialAsync

All RAS information (such as Functions, Structures and Enumeration Types) is located within the RAS Reference of the Platform SDK. After examining the reference, it was discovered that the RasDial function could be used to make an asynchronous connection to a server. The reference also pointed out that this function is located in the Rasapi32.dll file. Within our RasAPI component, the RasDialAsync method will encapsulate and expose this asynchronous connection behaviour. In order to use the RasDial function, we need to declare how to call it within C#. The following shows the RasDial unmanaged function prototype with the managed declaration in C#:

DWORD RasDial(
LPRASDIALEXTENSIONS lpRasDialExtensions, // pointer to function extensions
LPCTST lpszPhoneBook, // pointer to phone-book path and file
LPRASDIALPARAMS lpRasDialParams, // pointer to calling parameters data
DWORD dwNotifierType, // specifies type of RasDial event handler
LPVOID lpvNotifier, // specifies a handler for RasDial events
LPHRASCONN lphRasConn // pointer to variable to receive connection handle
);

public class RasAPI
{

// Ras API Function Declarations

[DllImport("rasapi32.dll")]
private static extern int RasDial(int lpRasDialExtensions, string lpszPhoneBook,
ref RASDIALPARAMS lpRasDialRarams, int dwNotifierType, RasDialCallback lpvNotifier,ref int lphRasConn);

public RasAPI
{
}
}

C# makes use of the DllImport attribute to allow you to describe the DLL function that the application will call to the compiler. Attributes are used to help provide design-time information for a C# type. As shown in the declaration, the DllImport attribute is being used to provide the name of the DLL file. The EntryPoint field could have also been used to provide the name or ordinal of the DLL entry point to be called. When adding this function declaration in the RasAPI class, make sure to include the required static and extern keywords. The extern modifier indicates that the method is implemented externally.

We will not be accessing any of these features so we will ignore this parameter. We can get away with declaring this parameter as an int type and pass in a null. However, we need to include the third parameter, a pointer to a RASDIALPARAMS structure. Since a pointer to the structure is expected, the ref keyword is used in the function declaration. This structure is very important since it contains all the connection parameter information for dialling. This structure is prototyped in unmanaged code as:

typedef struct _RASDIALPARAMS {
DWORD dwSize;
TCHAR szEntryName[RAS_MaxEntryName + 1];
TCHAR szPhoneNumber[RAS_MaxPhoneNumber + 1];
TCHAR szCallbackNumber[RAS_MaxCallbackNumber + 1];
TCHAR szUserName[UNLEN + 1]; // ULEN is User Name Length = 256
TCHAR szPassword[PWLEN + 1]; // PWLEN is Password Length = 256
TCHAR szDomain[DNLEN + 1]; // DNLEN is Domain Length = 15
#if (WINVER >= 0x401)
DWORD dwSubEntry;
ULONG_PTR dwCallbackID;
#endif
} RASDIALPARAMS;

This is defined in managed code as:

[StructLayout(LayoutKind.Sequential)]
public struct RASDIALPARAMS
{
public int dwSize;
[ MarshalAs( UnmanagedType.ByValTStr, SizeConst=257)]
public string szEntryName;
[ MarshalAs( UnmanagedType.ByValTStr, SizeConst=129 )]
public string szPhoneNumber;
[ MarshalAs( UnmanagedType.ByValTStr, SizeConst=129 )]
public string szCallBackNumber;
[ MarshalAs( UnmanagedType.ByValTStr, SizeConst=257 )]
public string szUserName;
[ MarshalAs( UnmanagedType.ByValTStr, SizeConst=257 )]
public string szPassword;
[ MarshalAs( UnmanagedType.ByValTStr, SizeConst=16 )]
public string szDomain;
public int dwSubEntry;
public int dwCallBackID;
}

To define this structure in managed code we need to use the StructLayout attribute. The StructLayout attribute describes the physical layout of the data members of a structure. The LayoutKind value Sequential forces the members to be laid out sequentially in the order they appear. In order to override the default .NET marshaling behavior, the MarshallAs attribute is attached to the fields of the structure. The UnManagedType enumerator is used to identify how parameters or fields should be marshaled to unmanaged code. The ByValTStr member is used to represent fixed length character arrays within a structure, while the SizeConst member defines the size in bytes of the fixed length array.

When operating asynchronously, the RasDial function returns immediately after being called, communicating its progress to a callback function. Therefore, we must include a callback function in our managed code to handle the event notifications. This is achieved by using Delegates. Since Delegates are basically type-safe function pointers, when calling the DLL function, you need to pass a delegate as an argument. The dwNotifierType and lpvNotifier parameters of the RasDial function are important in indicating if RasDial should make an asynchronous or synchronous call. The dwNotifierType parameter identifies if a callback function is going to be used. The lpvNotifier parameter specifies the handle to the callback function. If no callback is required, and thus is a synchronous call, enter null for both parameters. We will use the RasDialFunc1 prototype to receive notifications which requires dwNotifierType to have a value of 1.

VOID WINAPI RasDialFunc1(
HRASCONN hrasconn, // handle to RAS connection
UINT unMsg, // type of event that has occurred
RASCONNSTATE rascs, // connection state about to be entered
DWORD dwError, // error that may have occurred
DWORD dwExtendedError // extended error information for some errors
);

To use the unmanaged RasDialFunc1 callback function we must first declare a delegate to the RasManager namespace.

public delegate void RasDialCallback(int hRasConn, int unMsg, int rascs,
int dwError, int dwExtError);

Note how the delegate matches the RasDialFunc1 signature. All the parameters types, and the return type of the delegate, identically match those of the RasDialFunc1 prototype.

We now need to create the callback function which will get passed to the RasDial function.

private void MyRasDialFunc1(int hRasConn, int unMsg, int rascs, int dwError, int dwExtError)
{
string sState = GetStateString(rascs);

if (dwError !=0)
{
if (RasErrors != null)
RasErrors(dwError, RasError(dwError));
}
}

To pass the callback function to RasDial, a RasDialCallback delegate is declared:

RasDialCallback myCallBack = new RasDialCallback(MyRasDialFunc1);

The declaration of myCallBack creates the function pointer to MyRasDialFunc1. This variable can then be passed into the lpvNotifier parameter. Now, when RasDial needs to make a callback, the code pointed to by MyRasDialFunc1 will get executed. MyRasDialFunc1 handles receiving the progress of the RAS Connection State and RAS Error notifications.

Since we have defined all the necessary requirements to call the RasDial function from managed code, we can implement the RasDialAsync method.

public void RasDialAsnyc()
{
//Declare
            RASDIALPARAMS structure
RASDIALPARAMS rdp = new
            RASDIALPARAMS();
rdp.szEntryName = m_EntryName;
rdp.szUserName = m_UserName;
rdp.szPassword = m_Password;
rdp.szPhoneNumber = m_PhoneNumber;
rdp.szDomain =
            m_Domain;
rdp.dwSize = Marshal.SizeOf(rdp);

//Declare
            delegate used in callback
RasDialCallback myCallBack = new
            RasDialCallback(MyRasDialFunc1);

m_hRasConn =
            0;

//Call RasDial passing in RASDIALPARAMS struct and
            delegate
int rtn = RasAPI.RasDial(0, null, ref rdp, 1,
            myCallBack,
ref m_hRasConn);

//return error if one
            occured
if (rtn != 0)
{
if (RasErrors !=
            null)
RasErrors(rtn, RasError(rtn));
}

}

The m_hRasConn private variable is used to keep track of the connection handle returned from RasDial.

Using Events

The callback function is great in that it receives the connection progress within the component, but we would also like to have the calling application notified of this progress. This allows the calling application to handle the notifications like displaying the connection progress in a ListBox. This is accomplished by using events. Add an event handler and an event declaration to the RasAPI class:

public delegate void DialStateHandler(string state);
public event DialStateHandler DialState;
public delegate void RasErrorsHandler(int ErrVal, string ErrMsg);
public event RasErrorsHandler RasErrors;

Next, in MyRasDialFunc1, make a call to the event. Any application that subscribes to the event will receive the notifications. Additionally, an event publishing RAS Errors is also provided:

private void MyRasDialFunc1(int hRasConn, int unMsg, int rascs, int dwError, int dwExtError)
{
string sState = GetStateString(rascs);

if (DialState != null)
DialState(sState);

if (dwError !=0)
{
if (RasErrors != null)
RasErrors(dwError, RasError(dwError));
}
}

RAS Connection State

The ConnectionState method wraps the unmanaged RasGetConnectStatus function, which retrieves information about the current status of the RAS connection.

DWORD RasGetConnectStatus(
HRASCONN hrasconn, // handle to RAS connection of interest
LPRASCONNSTATUS lprasconnstatus // buffer to receive status data
);

This method is implemented as:

[DllImport("rasapi32.dll")]
private
            static extern int RasGetConnectStatus
(int hrasconn, ref
            RASCONNSTATUS lprasconnstate);

public string
            ConnectionState()
{
RASCONNSTATUS rcs = new
            RASCONNSTATUS();
rcs.dwSize = Marshal.SizeOf(rcs);

int rtn
            = RasGetConnectStatus(m_hRasConn, ref rcs);

if (rtn !=0 )
{
return RasError(rtn);
}
else
{
return
            GetStateString(rcs.rasconnstate);
}

}

The RasGetConnectStatus function requires a pointer to a RASCONSTATUS structure.

The RASCONSTATUS structure contains the RASCONNSTATE enumerator, which indicates the value of the current state of the connection. This value is passed into the GetStateString method, which converts the value into a string value description of the state. This structure is declared in a similar manner to the RASDIALPARAMS structure. The C# declaration of this structure can be found in the RasAPI.cs file of the included sample code.

RAS Error

The RasError function wraps the unmanaged RasGetErrorString function, which retrieves an error message string for a specified RAS error value.

DWORD RasGetErrorString(
UINT uErrorValue, // error to get string for
LPTSTR lpszErrorString, // buffer to hold error string
DWORD cBufSize // size of buffer
);

The unmanaged function defines the lpszErrorString parameter as a buffer to the error string value. This can be defined in managed code as a byte array which can be converted easily to a character array and string.

[DllImport("rasapi32.dll")]
private
            static extern int RasGetErrorString
(int uErrorValue, byte[]
            lpszErrorString, ref int cBufSize);

private string
            RasError(int ErrorValue)
{
int bufsize = 512;
char[] ch =
            new char[bufsize];
byte[] buff = new byte[bufsize];

if
            (ErrorValue > 600)
{
int rtn =
            RasGetErrorString(ErrorValue, buff, ref
            bufsize);
}
else
{
int rtn = FormatMessage( 4096, 0,
            ErrorValue, 0, buff, bufsize, 0);
}

for (int i = 0; i <
            bufsize; i++)
{
ch[i] =
            Convert.ToChar(buff[i]);
}

string err = new
            string(ch);

return err;

}

RasAPI Properties

To provide the ability to get and set connection parameter information, we can add Properties to our component. Properties provide an intuitive way to access and write to the values of private variables. The Properties will map to the members of the RASDIALPARAMS structures. Within the RasDialAsync function, you may have noticed that many of the RASDIALPARAMS structure fields were assigned to private variables, which represent properties. The following properties can be added to the RasAPI class:

      public
            string EntryName
{
get
{
return m_EntryName;
}
set
{
m_EntryName = value;
}
}

public string
            PhoneNumber
{
get
{
return
            m_PhoneNumber;
}
set
{
m_PhoneNumber =
            value;
}
}

public string
            UserName
{
get
{
return
            m_UserName;
}
set
{
m_UserName =
            value;
}
}

public string
            Password
{
get
{
return
            m_Password;
}
set
{
m_Password =
            value;
}
}

public string
            Domain
{
get
{
return m_Domain;
}
set
{
m_Domain = value;
}
}

public ArrayList
            Entries
{
get
{
return m_Entries;
}
}

These properties can be added individually by the calling application, or populated automatically by making a call to the GetEntry method. This method will be described shortly.

Get RAS Entries

In order to make a RAS connection, the user must specify a Phone Entry from a Phone Book. In Windows 2000, for example, a Phone Entry can be created through the Network Connection Wizard. The Phone Entry contains all the calling parameters, such as user name, password, domain, and phone number that were entered into the Connection Wizard. These Phone Entry parameters can be accessed to make a connection to a RAS server.

To get the name of all entries within a Phone Book, we need to call the unmanaged RasEnumEntries function. This function needs to be called twice, the first time to get the size of the Phone Book Entry buffer, and a second time to enumerate all the entries. The lpszPhonebook parameter requires a path to a valid RAS phone book. A null value can be passed in to access the system default phone book.

DWORD RasEnumEntries (
LPCTSTR reserved, // reserved, must be NULL
LPTCSTR lpszPhonebook, // pointer to full path of phone-book file
LPRASENTRYNAME lprasentryname, // buffer to receive phone-book entries
LPDWORD lpcb, // size in bytes of buffer
LPDWORD lpcEntries // number of entries written
);

This function uses a buffer of RASENTRYNAME structures as a parameter to hold the entries of a Phone Book. To represent this buffer in unmanaged code, we can use the IntPtr type. The IntPtr type is used to represent a pointer, in this case a pointer to an array of RASENTRYNAME structures.

[DllImport("rasapi32.dll")]
private
            static extern int RasEnumEntries
(string reserved, string
            PhoneBook, IntPtr lpRasEntries, ref int lpcb, ref int
            lpcEntries);

public void GetRasEntries()
{
RASENTRYNAME
            ren = new RASENTRYNAME();
ren.dwSize =
            Marshal.SizeOf(ren);
IntPtr lpre =
            Marshal.AllocHGlobal(ren.dwSize);
Marshal.StructureToPtr(ren,
            lpre, false);
int lpcb = 0;
int lpcEntries = 0;

int rtn
            = RasEnumEntries( null, null, lpre, ref lpcb, ref
            lpcEntries);

Marshal.FreeHGlobal(lpre);

lpre =
            Marshal.AllocHGlobal(lpcb);
Marshal.StructureToPtr(ren, lpre,
            false);

rtn = RasEnumEntries( null, null, lpre, ref lpcb, ref
            lpcEntries);

RASENTRYNAME managedArray = new
            RASENTRYNAME;
IntPtr buff = lpre;
for (int i = 0 ; i <
            lpcEntries ; i++)
{
managedArray =
            (RASENTRYNAME)Marshal.PtrToStructure( buff,
typeof(RASENTRYNAME)
            );
m_Entries.Add(managedArray.szEntryName);
buff =
            (IntPtr)((int)buff +
            Marshal.SizeOf(managedArray));
}

Marshal.FreeHGlobal(lpre);

}

Since the first call to RASENTRYNAME will give us the size of the RASENTRYNAME buffer, we will then need to allocate memory for the structure. To write a structure to unmanaged memory, we need to use the Marshal.StructureToPtr method. This method requires you to have a memory buffer allocated ahead of time. The Marshal.AllocHGlobal function is used to allocate this buffer. The second call to the function returns the RASENTRYNAME buffer array. The buff IntPtr variable is assigned to point to this buffer array. To get each RASENTRYNAME structure out of the buffer, we need to use the Marshal.PtrToStucture method. The size of each array element is calculated using the Marshal.SizeOf method. For each array element returned, the entry name is added to the Entries collection. Make sure to call Marshal.FreeHGlobal to release memory from the heap.

Get Ras Entry

The RasGetEntryDialParams unmanaged function is used to get the calling parameters of a Phone Entry:

DWORD RasGetEntryDialParams(
LPCTSTR lpszPhonebook, // pointer to the full path of phone-book file
LPRASDIALPARAMS lprasdialparams, // pointer to a RASDIALPARAMS structure
LPBOOL lpfPassword // indicates if user's password was retrieved
);

This function is wrapped in the managed GetEntry method:

[DllImport("rasapi32.dll")]
private
            static extern int RasGetEntryDialParams
(string Phonebook, ref
            RASDIALPARAMS lpRasDialParams, ref int lpfPassword);

public
            void GetEntry()
{
RASDIALPARAMS rdp = new
            RASDIALPARAMS();
rdp.szEntryName = m_EntryName;
rdp.dwSize =
            Marshal.SizeOf(rdp);
int lpfPassword = 0;

int rtn =
            RasGetEntryDialParams( null, ref rdp, ref
            lpfPassword);

m_UserName = rdp.szUserName;
m_Password =
            rdp.szPassword;
m_Domain = rdp.szDomain;
m_PhoneNumber =
            rdp.szPhoneNumber;
}

RasGetEntryParams returns the RASDIALPARAMS structure. The structure members are then mapped to their respective property variables. If the calling application doesn't explicitly assign each property, then call this method before the RasDialAsync method.

Hang Up

A simple call to RasHangUp will terminate an existing RAS connection. This function's only parameter requires that we pass in a valid connection handle. Remember, the private variable m_hRasConn holds this value.

RAS Client Application

Now that we have all the functionality we would like for the component, we will build a release version so it can be accessed by a calling application.

The application we will create will use the newly created RasManager component to connect asynchronously to a Phone Entry. Create a new C# Windows Application, naming it RasClient, and add it to the RasManager solution. Make sure to add a reference to RasManager to the project:

Add a form to the project named frmRas. Place two list boxes on the form, naming them lstEntries and lstStatus respectively. Add three buttons to the form naming then btnConnect, btnHangUp, and btnState respectively. Ensure frmRas is the startup form for the application.

Include the RasManager namespace to allow the declaration of a RasAPI object:

using RasManager;

private RasAPI objRas = new RasAPI();

When the application starts, we would like to get all Phone Entries from the default Phone Book. In the load event for frmRas, call the GetRasEntries method. Loop through each Phone Entry and add them to lstEntries :

private void frmRas_Load(object sender,
            System.EventArgs e)
{

//Get All Ras Entries for Default
            Phone Book
objRas.GetRasEntries();
int c =
            objRas.Entries.Count;
for (int i=0; i<c;
            i++)
{
lstEntries.Items.Add((string)objRas.Entries[i]);
}
}

Add the lstEntries SelectedIndexChanged event to handle the selection of a Phone Entry. When a Phone Entry is selected, the GetEntry method is called to get the connection parameters for that Entry. Now we have enough information to make a connection:

private void lstEntries_SelectedIndexChanged(object sender, System.EventArgs e)
{
objRas.EntryName = lstEntries.Text;
objRas.GetEntry();
btnConnect.Enabled = true;
}

In the btnConnect Click event, make a call to the RasDialAsync method. In order to get the connection notification progress, we must add an event handler. Add the event handle to the frmRas constructor and a private method named OnDialState to handle the event.

objRas.DialState += new
            RasAPI.DialStateHandler(OnDialState);

private void
            OnDialState(string
            sState)
{
lstStatus.Items.Add(sState);
if (sState ==
            "Connected")
btnHangUp.Enabled = true;
}

This method adds the connection state progress to lstStatus. All the connection states will be displayed in lstStatus until the connection is complete, or an error is received.

To disconnect from the connection, call the HangUp method. To check the connection state call the ConnectionState method.

Run the Application

Compile the application and run it. The default form, frmRas, should show up on your screen with a list of Ras Entries. If you have no Ras Entries, add one to your machine. This can be done using the Network Connection Wizard on Windows 2000 operating systems. Select a Phone Entry from the list, and click the Connect button. You will see the connection progress displayed in the List Box. Once successfully connected, click the Hang Up button to terminate the connection. You can also click the Get State button at any time to display the current connection status.

Sample Setup

After unzipping the sample application, open the RasManager.sln solution file in Visual Studio. This solution includes the RasManager and RasClient projects. Ensure that a reference to the RasManger.dll is added to the RasClient project. Compile it, and you're done.

Conclusion

This article explained how to wrap some of the RAS API functions into a reusable component. The component is responsible for getting all the Phone Entries in a Phone Book, and making an asynchronous RAS connection based on a Phone Entry. We then created an application which illustrates how to use the component.


C# Web Component Communication in .Net using Events – Part I

C# Web Component Communication in .Net using Events – Part I
http://www.csharptoday.com/content.asp?id=1711
Article Rating: 4
Published on December 4, 2001
Abstract
Microsoft has cleverly brought server-side, event-driven programming to the web if you so choose to use it. Proper use of these new capabilities can improve the design of individual components and thus, web applications as a whole. In this first article, Stephen Trudel produces a user control that will expose events to be processed on the server by its parent web page, and in doing so, exposes a design programming technique that could be used by many web developers.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

The intent of this first article is to provide a cookbook-like approach for wiring communication between web components. The majority of this communication is likely to occur between a web form and the user controls it contains. It is particularly important to understand the usage of user controls for the purposes of this article, as they will be used to expose our events. Other articles have covered user controls to some extent, so I will not do that here. Instead, I will focus on one very important aspect of user controls that allow them to elegantly communicate information with their parent container or any other .NET web objects for that matter - the event. You could also look at Matthew Reynold's article 'Programmatically Wiring Up Events in C#' (http://www.csharptoday.com/content.asp?id=1575&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq).

Event Usage

The following diagram shows an exaggerated use of user controls to illustrate, not only their usefulness, but also the importance of the communication they share with their parent container. In this diagram we have a web page that is completely rendered using user controls.

Figure 1

Picture, if you will, a list of hyperlinks lined vertically within the Left Menu user control. You could possibly intend to have functionality in your application, such that selecting a hyperlink from the Left Menu user control would force the parent container to dynamically load a new user control in place of the one that now sits in the Main Content cell. This would imply that we have assigned roles to these two particular web components as follows:

  • Parent Web Form: Oversee page layout and navigation based on messages received by user controls for which it loads.
  • User Control: Aside from exposing properties and methods to configure appearance, deliver timely and informative messages to the parent web form.

With these roles in place how would the parent container know when a hyperlink was selected? In addition, how would the parent know which hyperlink was selected? Evidently, some means of communication should be in place. This communication entails notification of an occurrence and any information that can be supplied relating to that occurrence. Facilitating this communication is exactly what events in the .NET Framework are designed for!

It is very likely when building your web applications that you will need to use pages. As an aside, another means for user controls to communicate with one another would be to use a Response.Redirect on the server side, and pass information through a Querystring from code within your user control. The receiving page could then parse the string using the Request.Params collection and take the appropriate action, however, this is not an optimal choice of solutions since its lends itself to tight coupling between components. A mark of a well-designed component is a high level of independence from its container.

Delegates and Events

Before presenting the example to illustrate the use of events, it is first important to say a word about delegates. To fully understand the internals of the following example's code it is necessary to understand delegates. Delegates are objects in the .NET Framework that host one or more objects and their respective functions, to be called in reaction to user intervention, system intervention, or when a segment of time has been reached. Delegates are related to events in that events provide a level of protection to delegates somewhat similar to property procedures introduced in Visual Basic 5, and those now available in C#. On the surface, the internals of how the events, delegates, and other Framework classes involved can initially be confusing to the developer. Understanding the operations the compiler performs behind the scenes helps to alleviate this confusion. For example, looking at the syntax presented in this article for declaring a delegate, it would be difficult for the developer to understand that a delegate is an object, but it is. This information will be presented at a later time in Part II of this article. For those of you who are comfortable with simply just knowing how to use delegates and events, then this article should be sufficient.

The Example

It is conceivable that you could, in fact, have an entire website made up of user controls and just one web page. To illustrate this to some extent, the top part of http://www.wrox.com/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq, for example, could be split horizontally into two user controls (.ascx) inside a parent container (.aspx), as follows:

Figure 2

You will notice the gray boxes on the right, which shows how this section of the page would be split in two. So to demonstrate the use of events, I am simply going to do some simplistic image mapping using User Control 1 (WroxHeader.ascx) in the above figure, to communicate with a Parent Container (Default.aspx). The end result will look like this:

Figure 3

The two gray table cells along the bottom belong to the parent page (Default.aspx). Clicking in the book on the left will produce an event that passes information on that portion of the image. Two table cells are used in this example to exploit multicasting, an instance by which two callbacks to two different functions are made in succession. These functions happen to reside in the same class, but they do not have to. The result of clicking the Professional .NET Framework book on the left will produce the following:

Figure 4

Step by Step

  • Open Visual Studio 7 (Beta 2) and select New Project | Visual C# Projects in the left-hand tree.
  • Select ASP.NET Web Application from the right-hand array of icons.
  • Enter Communication as the project name and place it in http://localhost.
  • Create a new folder in this directory using Windows Explorer as well and name it images. Place the .gif images from the downloadable file in this directory.
  • In the Solution Explorer, right-click Communication | New Folder.
  • Name the new folder UserControls.
  • In the Solution Explorer, right-click the UserControls folder | Add New Item.
  • Select Class from the right pane and name it EventInformation.cs. This file will contain the declaration of delegates and any classes that extend EventArgs.
  • Enter the code below. WroxHeaderEventHandler is a delegate. Make careful note of its signature, as it will be matched to that of the one in the parent web form container (Default.aspx).

Note: Make careful note of the numbered lines (i.e. [1], [2], .) that you see in the code. We will be referring to them later in the article.

using System;
using System.Web.UI.WebControls;

namespace Communication.UserControls.Delegates
{
using EvengArgObjects;
// declaration of the delegate [1]
// behind the scenes the compiler produces an object
// that extends the System.MulitCastDelegate class
public delegate void WroxHeaderEventHandler(object sender, WroxImageEventArgs e);
}

  • In the same file, add the following code. WroxImageEventArgs is a class that extends the EventArgs class to provide information on the image clicked. EventArgs is the base class for event data.

namespace Communication.UserControls.Delegates.EventArgObjects
{
// class used to extend System.EventArgs [2]
public class WroxImageEventArgs : System.EventArgs
{
private string imageTextClicked = null;

public string imageText
{
get {return imageTextClicked;}
set {imageTextClicked = value.Trim();}
}
}
}

  • In the Solution Explorer, right-click the UserControls folder | Add New Item.
  • Select Web user control from the right pane and name it WroxHeader.ascx.
  • Enter the ASP.NET HTML code below. This user control will be used to produce what you see in Figure 2, less the menu (user control 1, but no user control 2). For the purposes of this article, we will mostly use Web Controls as opposed to HTML Controls. I personally find the syntax for Web Controls easier to read, and much more consistent then regular HTML. It also has the advantage of being available to any type of web device.

<%@ Control Language="c#" AutoEventWireup="false" Codebehind="WroxHeader.ascx.cs" Inherits="Communication.UserControls.WroxHeader"%>
<asp:Table Runat="server" CellSpacing="0" CellPadding="0" id="Table1">
<asp:TableRow Runat="server">
<asp:TableCell Runat="Server">
<asp:Table Border="0" CellSpacing="0" CellPadding="0">
<asp:TableRow ID="row1" Runat="server">
<asp:TableCell Runat="server" ColumnSpan="2" HorizontalAlign="Left">
<asp:ImageButton ID="imgNewWroxLogo" Runat="server"
ImageUrl="/Communication/images/newwroxlogo.gif" OnClick="Image_Click" />
</asp:TableCell>
<asp:TableCell Runat="server" ColumnSpan="4" HorizontalAlign="Right">
<asp:ImageButton ID="imgNewWroxHead" Runat="server"
ImageUrl="/Communication/images/newwroxhead.gif" OnClick="Image_Click" />
</asp:TableCell>
</asp:TableRow>
<asp:Table>
<asp:TableRow ID="row2" Runat="server">
<asp:TableCell Runat="server">
<asp:ImageButton ID="imgBanner1" Runat="server"
ImageUrl="/Communication/images/banner1.gif" OnClick="Image_Click" />
</asp:TableCell>
<asp:TableCell Runat="server">
<asp:ImageButton ID="imgBanner2" Runat="server"
ImageUrl="/Communication/images/banner2.gif" OnClick="Image_Click" />
</asp:TableCell>
<asp:TableCell Runat="server">
<asp:ImageButton ID="imgBanner3" Runat="server"
ImageUrl="/Communication/images/banner3.gif" OnClick="Image_Click" />
</asp:TableCell>
<asp:TableCell Runat="server">
<asp:ImageButton ID="imgBanner4" Runat="server"
ImageUrl="/Communication/images/banner4.gif" OnClick="Image_Click" />
</asp:TableCell>
</asp:TableRow>
</asp:Table>
</asp:TableCell>
</asp:TableRow>
</asp:Table>

  • Within the code-behind (WroxHeader.ascx.cs) of this control, enter the following code. Again, make note of the numbered lines. We will be discussing them later in the article.

namespace
            Communication.UserControls
{
using System;
using
            System.Web;
using System.Web.UI;
using
            System.Web.UI.WebControls;
using
            Communication.UserControls.Delegates;
using
            Communication.UserControls.Delegates.EventArgObjects;

public
            abstract class WroxHeader : System.Web.UI.UserControl
{
//
            declare a public event [3]
public event WroxHeaderEventHandler
            wroxHeaderEventHandler;

public void Image_Click(object
            sender, ImageClickEventArgs e)
{
// if event is not wired it
            will be null [4]
if(wroxHeaderEventHandler !=
            null)
{
ImageButton ib = (ImageButton)sender;
// create an
            instance of the EvenArgs object and load it with
// all
            necessary information pertaining to the event
            [5]
WroxImageEventArgs wroxImageEventArgs = new
            WroxImageEventArgs();
wroxImageEventArgs.ImageTextClicked =
            GetImageButtonText(ib.ID, e);

// this is a more controlled
            means of raising an event
// if there is more than one event in
            the InvocationList
// and an error occurs the following events
            will not occur [6]
foreach(WroxHeaderEventHandler h in
            wroxHeaderEventHandler.GetInvocationList())
{
try
{
//
            event is raised here [7]
h(ib, wroxImageEventArgs);
}
catch
            (Exception er)
{
// handle exception
            here
}
}
}
return;
}

  • Additionally, within the code-behind (WroxHeader.ascx.cs), enter the following code. There is nothing special here, we are just associating a given section of the image, by way of its X, Y coordinates returned in the ImageClickEventArgs of the ImageButton control, with a string value.

      private
            string GetImageButtonText(string ID, ImageClickEventArgs
            e)
{
string returnValue = null;

if(ID ==
            "imgNewWroxLogo")
returnValue = "Wrox";

if(ID ==
            "imgNewWroxHead")
{
if((e.Y > 0) && (e.Y <
            35))
returnValue = "wrox.com";
else
returnValue =
            "Programmer to Programmer";
}

if(ID ==
            "imgBanner1")
returnValue = "Sept";

if(ID == "imgBanner2"
            || ID == "imgBanner3")
{
if((e.Y > 0) && (e.Y <
            21))
returnValue = "New this Month";
if((e.Y > 20)
            && (e.Y < 36))
returnValue = "Professional";
if(e.Y
            > 35)
returnValue = (ID == "imgBanner2" ? ".NET Framework" :
            "XML Web Services");
if(e.X > 160)
returnValue = (ID ==
            "imgBanner2" ? "Professional .NET Framework" :
"Professional XML
            Web Services");
}

if(ID == "imgBanner4")
{
if((e.Y
            > 0) && (e.Y < 36))
returnValue =
            ".NET@Wrox.com";
if(e.Y > 35)
returnValue =
            "www.wrox.com/dotnet.asp";
}
return returnValue;
}
} //
            end class
} // end namespace

  • Right-click WebForm1.aspx in the Solution Explorer | Delete.
  • Right-click Communication in the Solution Explorer | Add New Web Form.
  • Name the new web form Default.aspx.
  • Right-click Default.aspx in the Solution Explorer | Set As Start Page.
  • Enter the following HTML. Note: Change the Inherits directive to Inherits="Communication.DefaultPage", since default is a C# keyword. Use ASP.NET HTML for this web form.

<%@ Page language="c#" Codebehind="default.aspx.cs" AutoEventWireup="false" Inherits="Communication.DefaultPage" %>
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" >
<HTML>
<HEAD><TITLE>Trudel Image Mapping Example</TITLE> </HEAD>
<BODY>
<FORM runat="server">
<asp:Table ID="tblLayout" Runat="server" CellSpacing="0" CellPadding="0">
<asp:TableRow ID="row1" Runat="server">
<asp:TableCell ID="cell11" Runat="server"></asp:TableCell>
</asp:TableRow>
<asp:TableRow ID="row2" Runat="server">
<asp:TableCell ID="cellTest1" Runat="server"
Text="The text of the image you select will appear here
first. This is the result of the first event."
Font-Name="Verdana" Font-Size="11pt" BackColor="Gainsboro" BorderStyle="Solid"
BorderWidth="1px" BorderColor="Black" HorizontalAlign="Center" />
</asp:TableRow>
<asp:TableCell ID="cellTest2" Runat="server"
Text="The text of the image you select will appear here
second. This is the result of the second event."
Font-Name="Verdana" Font-Size="11pt" BackColor="Gainsboro" BorderStyle="Solid"
BorderWidth="1px" BorderColor="Black" HorizontalAlign="Center" />
</asp:TableRow>
</asp:Table>
</FORM>
</BODY>
</HTML>

  • Enter the following code in the code behind (Default.aspx.cs) for the default web form. Note: Rename the code behind class to DefaultPage since default is a C# keyword.

using System;
using System.Web;
using System.Web.UI;
using System.Web.UI.WebControls;
using Communication.UserControls;
using Communication.UserControls.Delegates;
using Communication.UserControls.Delegates.EventArgObjects;

namespace Communication
{
public class Default : System.Web.UI.Page
{
protected System.Web.UI.WebControls.TableCell cell11;
protected System.Web.UI.WebControls.TableCell cellTest1;
protected System.Web.UI.WebControls.TableCell cellTest2;

public Default()
{
Page.Init += new System.EventHandler(Page_Init);
}

private void Page_Init(object sender, EventArgs e)
{
InitializeComponent();
}

private void InitializeComponent()
{
this.Load += new System.EventHandler(this.Page_Load);
}

private void Page_Load(object sender, System.EventArgs e)
{
cell11.Controls.Add(LoadHeader());
}

// procedure to be wired!!! [1]
public void SetText1(object sender, WroxImageEventArgs e)
{
cellTest1.Text = "The image mapping text you chose is: <B>{" + e.ImageText + "}</B>";
}

// procedure to be wired!!! [1]
public void SetText2(object sender, WroxImageEventArgs e)
{
cellTest2.Text = "The image mapping text you chose is: <B>{" + e.ImageText + "}</B>";
}

private WroxHeader LoadHeader()
{
// set necessary properties and return user control
WroxHeader wh = (WroxHeader)Page.LoadControl("UserControls/WroxHeader.ascx");
// wire events!!! [2]
wh.wroxHeaderEventHandler += new WroxHeaderEventHandler(this.SetText1);
wh.wroxHeaderEventHandler += new WroxHeaderEventHandler(this.SetText2);
return wh;
}

} // end class
} // end namespace

  • Compile the project Build | Rebuild All and hit Run.
  • Click on different segments of the image, and notice the message change to reflect the text in the closest proximity to where you clicked.

Event Creation Cookbook

Ensuring communication between components is a two-phase process, assuming of course you have your parent container and user control created. Phase 1 of creating an event is a simple 7-step process, and phase 2, wiring the event, is a simple 2-step process.

There are of course other subtleties you have to perform, such as having the right using statements in place, and so on. These are the major steps.

Phase 1: Creating the Event (WroxHeader.ascx)

  • Signature - Declare a delegate to establish the method signature (inside or outside a class). Please note:
  • The conventional signature in C# is (object sender, EventArgs e).
  • sender is typically the instance of the object (this) that is raising the event or an object that initiated the event chain. For example, if a user control has a button that triggers a procedure that in turn raises the event, then that button initiated the event chain, and so is the sender. The button instance could be passed instead of that of the user control.
  • EventArgs is an object containing information pertaining to the event.
  • A delegate could be declared inside a class, but doing so would mean it's only available for that class. If the signature is likely to be used again, it's a good idea to declare the delegate outside a class with a global scope. Declaring the delegate inside a class will force it to be a nested object.

public delegate void WroxHeaderEventHandler(object sender, WroxImageEventArgs e);

  • Extend EventArgs - Extend EventArgs to include any necessary state or functionality. This step is optional. Please note:
  • The naming convention for classes that extend EventArgs is to have each name end in EventArgs. Since we extended EventArgs, we have an object named WroxImageEventArgs.

public class WroxImageEventArgs : System.EventArgs
{
private string imageTextClicked = null;

public string imageText
{
get {return imageTextClicked;}
set {imageTextClicked = value.Trim();}
}
}

  • Public Event Declaration - Declare an event with a public scope inside a class that you would like to raise the event (raising an event is equivalent to making a callback). Clients using this object can pass it delegates.

public event WroxHeaderEventHandler wroxHeaderEventHandler;

  • Check for Null - A delegate that is not wired will be null. Always check your delegates for null.

if(wroxHeaderEventHandler !=
            null)
{
// code here
}

  • Instantiate Extended EventArgs - This step is necessary only if you did the second step. Instantiate an instance of your extended EventArgs object, and load it with information.

WroxImageEventArgs wroxImageEventArgs = new WroxImageEventArgs();
wroxImageEventArgs.ImageTextClicked = GetImageButtonText(ib.ID, e);

  • Use GetInvocationList - Callbacks are more dependable if you are careful. GetInvocationList returns an array of delegate objects that you can use to iterate through. Trap any errors that may occur and continue to raise the remainder of the events.
  • Raise the Event - Finally, raise the event using the object variable name you chose in the third step, along with the parameter information you are passing in the form of an empty or extended EventArgs object. If you do not have any information to return, then pass an empty EventArgs using the read only, static field EventArgs.Empty. That's it.

foreach(WroxHeaderEventHandler h in
            wroxHeaderEventHandler.GetInvocationList())
{
try
{
h(ib,
            wroxImageEventArgs);
// h(ib, EventArgs.Empty);
}
catch
            (Exception er)
{
// handle exception
here
}
}

Phase 2: Wiring the Event (Default.aspx)

  • Parent Procedure - Code the procedure(s) of the event to be called with the same signature as that of the delegate you declared in "Phase 1: Creating the Event". Delegates are type safe so you will not be able to use an alternative signature.

public void SetText1(object sender, WroxImageEventArgs e)
{
cellTest1.Text = "The image mapping text you chose is: <B>{" + e.ImageText + "}</B>";
}

  • Wire-Up - Wire the event declared in the user control to that of the parent.

private WroxHeader LoadHeader()
{
WroxHeader wh = (WroxHeader)Page.LoadControl("UserControls/WroxHeader.ascx");
// wire events
wh.wroxHeaderEventHandler += new WroxHeaderEventHandler(this.SetText1);
wh.wroxHeaderEventHandler += new WroxHeaderEventHandler(this.SetText2);
return wh;
}

Conclusion

It is clear that the usage of events has an important place in the design of .NET classes. In my next article, we will see what is going on behind the curtains on the server-side within the .NET Framework. I hope you have found this article informative and I would like to thank you for reading.


Writing a Messenger Application using C# - Part 2

Writing a Messenger Application using C# - Part 2
http://www.csharptoday.com/content.asp?id=1705
Article Rating: 4
Published on November 23, 2001
Abstract
In his first article, Saravana Kumar developed an assembly MessengerObject.dll, containing of all the functional units of a instant messenging application, and the messenger server, which exposed the public methods of the assembly to the remote clients using .NET remoting. In this article, Saravana develops a Windows Form messenger client application. We also see how to debug the application's SOAP messages.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

In our first article, we developed our assembly MessengerObject.dll, which contained of all the functional units of a messenger application, and the messenger server, which exposed the public methods of the assembly to the remote clients using .NET remoting. In this article, we will develop a messenger client application. This application will allow the end users to communicate with each other instantly via the server. For the user interface, we will create a windows form front end using the capabilities of .NET.

Let us first break down our tasks before proceeding with our client application:

  • Create an instance of the remote object "MessengerObject"
  • Authenticate the user.
  • Build our client with an initial set up (loading all online, offline contacts, and status messages).
  • Send and receive messages with friends.
  • Changing status ("Away", "Be right back", etc).
  • Log out of the server.

Now let us go through each of the above tasks in detail:

Creating an instance of the remote object "MessengerObject"

In Visual Studio .NET, create a new C# Windows Application and call it Client. Before developing our client application, we need to reference the following assemblies inside our application:

  • System.Runtime.Remoting
  • System.Windows.Forms,
  • and MessengerObject.dll (that we created on the first article)

To add the above references in Visual Studio .NET, go to solution explorer, right click on References and Add References. A dialog box will pop up. Select the first two System assemblies from the .NET tab and then click Browse to locate our MessengerObject.dll (present along with the code download).

We need to create an instance of the remote object MessengerObject. Remote objects can easily be created from a client by calling new. The framework contains enough intelligence to realize we are dealing with a remote object and will ensure an instance of the object gets created in the relevant remote application.

The Activator class of the System namespace contains two methods: CreateInstance and GetObject, that can also be used to create an instance of remote objects. The CreateInstance method can be used in place of new to create an object instance, while the GetObject method is normally used to connect to an object at a specified URL.

Since our application is going to communicate via HTTP, and we know our activation URL for our messenger server (http://localhost:1010/Messenger?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq -highlighted in our first article), we will use the GetObject method of the Activator class in our application to create an instance of the remote object.

Below is a code snippet that will create a remote object called objMessengerObject.

using System.Windows.Forms;
using
            System.Runtime.Remoting.Channels.Http;
using
            System.Runtime.Remoting.Channels;
///.
///.
int
            listeningChannel = 0;
HttpChannel channel = new
            HttpChannel(listeningChannel);
ChannelServices.RegisterChannel(channel);
objMessengerObject
            =
            (MessengerObject)Activator.GetObject(typeof(com.Remoting.MessengerObject),Configuration.sRemotingURL);
///.

Even though the object is remotely activated here, we can use this object later on as a local object. The .NET remoting framework will take care of all the hard stuff behind the scenes such as creating a transparent proxy, communication with the original remote proxy on the server, etc. For more information on remoting see http://www.csharptoday.com/content.asp?id=1581&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq.

The second parameter Configuration.sRemotingURL of the GetObject method in the above code snippet is our activation URL, which is stored as a public static member inside our Configuration class. This is shown in the code snippet below:

///.
public class Configuration{
public static String sRemotingURL = "http://localhost:1010/Messenger";
///.

localhost is the name of the machine where our messenger server is running.

Authenticating the User

Users need to log in to the server in order to start communicating with friends. It's important to be sure that we are permitting users who are in our database to log into the server. As soon as we start our client application, the login form pops up. The validateUser method in the server is then called using our remote object objMessengerObject by passing the username and password as arguments.

Figure 1: User Authentication Screen

Below is a code snippet showing the call to the validateUser method:

///..
bool result;
result = objMessengerObject.validateUser(txtUserName.Text,txtPassword.Text);
///..

A boolean value of either true or false is returned from the server after querying the database with the user credentials supplied to the validateUser method. Once the user is successfully authenticated, a client object is created for the user by calling the constructor of the clsMainClient class with appropriate arguments, as shown below:

///.
objClient = new
            clsMainClient(txtUserName,objMessengerObject)
objClient.show();
///.

Since our client application is a windows form application, the clsMainClient class is inherited from the System.Windows.Forms.Form class.

Inside the clsMainClient() constructor, the addUser method of the remote object is called with the user ID and a label box control reference (the details of which are explained later), which allow the server to uniquely identify each online user. The method call is below:

///.
objMessengerObject.addUser (sUserAccount, ref this.lblComPoint);
///.

Building our Client

After successfully authenticating the user and adding the reference to the server, we need to build our main client with initial values (mentioned in the below bulleted list) so that it's ready for communication.

The client application needs to be presented with:

  • Users Online contacts
  • Users Offline contacts, and
  • Menus with status messages like "Away", "Be right back", "On phone", etc.

We perform the above mentioned tasks in the client load event of the clsMainClient class.

Inside the clsMainClient load event we call the method sendOnlineAndOfflineUserList using objMessengerObject by passing the user id as the parameter. This method returns an XML string with all online contacts, offline contacts, and status messages for the user after querying the database. This method call is below:

///.
string sOnlineOfflineXML;
sOnlineOfflineXML = objMessengerObject.sendOnlineAndOfflineUserList (sUserAccount.Trim());
///.

The return XML will look this sample:

<UserList>
<Online><U>julianyap</U></Online>
<Offline>
<U>mario</U>
<U>mike</U>
<U>nigel</U>
<U>helen</U>
</Offline>
<StatusList>
<S id="0">OffLine</S>
<S id="1">Available</S>
<S id="2">Busy</S>
<S id="3">Idle</S>
<S id="4">Be Right Back</S>
<S id="5">On the Phone</S>
<S id="6">Out to Lunch</S>
</StatusList>
</UserList>

Once we have the XML string, which contains online contacts, offline contacts, and status messages from the server, then we can use some of the XML classes in the .NET framework like XmlDocument and XmlNodeList to manipulate this XML string and set up the client. Let us have a look at the clsMainClient_Load event:

private void clsMainClient_Load(object
            sender, System.EventArgs e){
///..
//Fill lBoxOnline list box
            with online contacts
int
            i=0;
objDomeNodeList=objDomDocument.SelectNodes("UserList/Online/U");
foreach(System.Xml.XmlNode
            n in
            objDomeNodeList){
this.lBoxOnline.Items.Add(n.InnerText);

//To
            maintain user status (online or offline) so that we
//can check
            before sending the
            message
aUserIDS.Add(n.InnerText);

//populate context
            menu
this.cMenuSendIMsg.MenuItems.Add(n.InnerText,new
            System.EventHandler(this.cMenuSendIMsg_Click));

//create
            clsFriendsChatWindow for all online users,
//but don't display
            it
frmIW[i] = new
            clsFriendsChatWindow(n.InnerText);
frmIW[i].Name=n.InnerText;
frmIW[i].Text=n.InnerText;
aClientUserIDS.Add(n.InnerText.Trim());
aClientWindowIDS.Add(frmIW[i]);
frmIW[i].Show();
frmIW[i].lblStatus.Text=
            n.InnerText + " status is
            Available";
frmIW[i].picStatusDisplay.Image=image[0];
frmIW[i].Visible=false;
i=i+1;
}
///..
}

The clsMainClient_Load event does the following tasks for us:

  • 1. Picks up users' online friends from the <Online> tag of the XML string and stores them in the array list object aUserIDS. So we can easily check a friend's online status before sending any message to him / her.
  • 2. Adds the online contacts in the <Online> tag to the online contacts list box control (lblBoxOnline), so the user can pick up the friends from the list.
  • 3. Populates the context menu with online contacts, so that the user can easily send messages to a friend by selecting the friend's name from the context menu system tray icon.
  • 4. We need a separate messaging window for each friend, so we create an instance of clsFriendsChatWindow (inherited from the System.Windows.Forms.Form class) for every online friend. We do this by calling the constructor with the friend's id as an argument, adding the reference to the array list object aClientWindowIDS, and making the form invisible by setting up the form's visible property to false. The aClientUserIDS array list is used to support the fetching of the corresponding index of the friend's chat window from the array list aClientWindowIDS with the help of the friend's user id.

The same approach is used for offline friends as well, except we do not add the offline user to the array list aUserIDS and also not to the context menu.

Note: The final step in creating instances of clsFriendsChatWindow and making it invisible for every online and offline contact might not appear to be a feasible solution, as we can create an instance and show them when the user selects a particular friend to chat with. But, problems are encountered in the beta 2 release. When we try to show the clsFriendsChatWindow using the form's show() method inside the lblComPoint_TextChanged event, it hangs the whole application. It works when the event is triggered within the client application locally. But, when we try to capture this event from the server it hangs the whole application. So we take an indirect solution to the problem.

After loading online and offline contacts, we pick up the status messages from the XML string and populate the main menu and context menu. The code snippet, which does this task, is shown below:

///.
//Fill status messages in tray icon context menu and main menu bar
objDomeNodeList=objDomDocument.SelectNodes("UserList/StatusList/S");
foreach(System.Xml.XmlNode n in objDomeNodeList){
this.cMenuMyStatus.MenuItems.Add(n.InnerText ,new System.EventHandler(this.cMenuMyStatus_Click));
this.menuMyStatus.MenuItems.Add(n.InnerText ,new System.EventHandler(this.cMenuMyStatus_Click));
aStatusString.Add(n.InnerText);
}
///.

Figure 2: Context menu with system tray icon

After executing the code in the clsMainClient_Load event the client application will look like this:

Figure 3: Main Client Window

Now our client is set up and the user is ready to communicate with any of their friends listed in the online list box control (lblBoxOnline).

How is communication between server and client(s) is achieved?

So how are we going to communicate between the users? The idea behind this project is to catch an event in the client side that's triggered from the server. Let's see this in more detail.

Inside the constructor clsMainClient() we called the method addUser with two parameters sUserAccount and ref this.lblComPoint. The call to the addUser method stores the reference of the label box control (lblComPoint) passed to it in the server. The server will keep changing the text of this label box control reference in the server with appropriate XML strings for different actions such as when we send or receive messages between users, when we change the status message from the client application, etc. This will change the text of the label box control on the corresponding client application, which in turn triggers the lblComPoint_TextChanged event in the client. The text (XML string) assigned to the Text property of lblComPoint will be loaded into an XmlDocument object and manipulated accordingly within the lblComPoint_TextChanged event.

Send and Receive messages with friends

When a user click on any of their online friends IDs' on the lBoxOnline list box, the corresponding clsFriendsChatWindow object (instant messaging window) associated with that friend will be picked up from the array list aClientWindowIDS (manipulated in the clsMainClient_Load event) and displayed. In order to pick up the corresponding clsFriendsChatWindow object from the array list aClientWindowIDS we need the correct index. This index is obtained from another array list aClientUserIDS, where online friends IDs' are stored using the IndexOf() method of an array list.

The code for lBoxOnline_SelectedIndexChanged is shown below:

private void lBoxOnline_SelectedIndexChanged(object sender,System.Eventargs e){
clsFriendsChatWindow frmIWin
string s;
ListBox objLBox = new ListBox();
objLBox = (ListBox)sender;
s=objLBox.Text;

//pick up the corresponding friend window from
//aClientWindowIDS array list
frmIWin = (clsFriendsChatWindow)aClientWindowIDS[aClientUserIDS.IndexOf(s.Trim())];
frmIWin.RTBMessageDisplay.Text="";
frmIWin.Visible=true;
}

The above code will open an instant messaging window for a particular friend selected from the lBoxOnline list box:

Figure 4: Instant Messaging Window

Now, when the user enters some text in the text box and clicks the Send button, the following tasks are done one by one to deliver the message on the friend's instant messaging window:

1. The static method sendMessage (excerpt shown below) in the clsMainClient class is called, with the friend's ID and the text that needs to be sent to the friend as arguments.

private void btnSend_Click(object sender,
            System.EventArgs e){
if(RTBUserInputMessage.Text!=""){
string
            s,sUserName;
sUserName=this.Name;
s=clsMainClient.sendMessage(sUserName.Trim(),this.RTBUserInputMessage.Text.Trim());
this.RTBUserInputMessage.Text="";
this.btnSend.Enabled=false;
}
}

2. Inside the static method sendMessage, the sUserName (friend ID) is checked to see whether the friend is online by querying aUserIDS array list (because sometimes the friend on the other side might have gone offline when the instance window of them is kept open on the user side). If the friend is online, an XML string (which acts as the message) is formed using the friend's ID from the instant messaging window as the <T> element. Text that needs to be sent to the friend is placed in the <B> element and the user ID in the <F> element. The structure of the XML will look like this:

<M>
<F>julianyap</F>
<T>saravana</T>
<B>julianyap Says: \n Hello there</B>
</M>

The structure is not self-explanatory. One may prefer the actual XML to look more like this:

<Message>
<From>julianyap</From>
<To>saravana</To>
<Body>julianyap Says: \n Hello there</Body>
</Message>

But since the message is transferred regularly, putting extra characters will unnecessarily increase the size of the XML string.

3. After formatting the XML string (message), the getMessage method (excerpt shown below) of our remote object is called by passing this XML string (message) as the argument:

///.
s=objMessengerObject.getMessage(sMessageXML);
///.

This is the main method, which does the job of transferring messages from one client application to another.

4. On the server side, this XML string (message) will be assigned to the corresponding friend's label box control reference Text property. The correct label box reference is selected in the server with the help of the <T> element present in the XML message, which contains the friend's ID.

5. This Text property change, of the label box control reference in the server, will cause the label box Text property on the friend's client application to change, which in turn will trigger the lblComPoint_TextChanged event on the friend's client application.

Now we will look at how the lblComPoint_TextChanged (excerpt shown below) event is handled in the client application. Since the label box text is now an XML string, we can load that string into an XmlDocument object.

private void lblComPoint_TextChanged(object
            sender, System.EventArgs e){
///..
string
            sXML=this.lblComPoint.Text;
oDomDocument.LoadXml(sXML.Trim());
oDomNode
            =
            oDomDocument.SelectSingleNode("M");
///..
///..
sUserTo=oDomNode.SelectSingleNode("T").InnerText;
sUserFrom=oDomNode.SelectSingleNode("F").InnerText;
sBodyText=oDomNode.SelectSingleNode("B").InnerText;
///.
///.
frmIWin=
            (clsFriendsChatWindow)aClientWindowIDS[aClientUserIDS.IndexOf(sUserTo.Trim())];
frmIWin.RTBMessageDisplay.AppendText(sBodyText
            +"\n");
frmIWin.Visible=true;
///..
}

6. The corresponding clsFriendsChatWindow object associated with the friend in the "<F>" element will be picked up from the array list aClientWindowIDS using the index obtained from another array list aClientUserIDS.

Once the clsFriendsChatWindow object is identified (frmIWin in the above code snippet) then the body text from the user present in the <B> of the XML message can be appended to the rich text box RTBMessageDisplay, so the message appears on the friend's instance window.

Changing Status

If the user changes their status (example: "Away ", "Be right back ", etc) in the client application, then the status change needs to be sent to all of their friends who are also online. In the client application the user can change their status in two places, from the context menu, or from the main menu (Figure 2). Both of these selections trigger the same event cMenuMyStatus_Click. Inside the cMenuMyStatus_Click event the sendStatusChangeToAllFriends method (excerpt shown below) is called using our remote object objMessengerObject. On the server, this method will form an XML string as shown below and will be sent to all of the user's online friends' client applications.

<M><S m='1' t='Away'>julianyap</S></M>

///.
objMessengerObject.sendStatusChangeToAllFriends(sUserAccount.Trim(),aStatusString.IndexOf(sUserStatus));
///.

In a similar way to what we have seen while sending and receiving messages, this XML message will be handled by the lblComPoint_TextChanged event.

Logout of the server

When the user logs out of the server, either by closing the application or by clicking on the sign out button in the menu, we need to notify all of the user's online friends about the user's new status (user is going offline) and the resources occupied by the user on the server need to be killed. The removeUser method (excerpt below) of our remote object does this job for us. Although there are number of options for the user to log out of the server, (i.e. the user can log out either by closing the client application, or by clicking Sign out in the context menu or main menu), we'll call the same method in all those places.

///.
objMessengerObject.removeUser(sUserAccount);
///.

Additional information

Dynamic icons:

We need to change icons dynamically inside our application in many places, as different options are changed or chosen (example: system tray, client status bar). Even though it's not necessary, it will give a rich appearance to the client. To do this, we will store the icons in an Icon array so that it will be more efficient than retrieving from a file. The code to achieve this is below:

///.
//icons stored in an icon
            array
icon[0]=new Icon(Configuration.sGraphicsDir +
            "onlinewhite.ico");
icon[1]=new Icon(Configuration.sGraphicsDir +
            "offlinewhite.ico");
icon[2]=new Icon(Configuration.sGraphicsDir
            +
            "awaywhite.ico");
///.

///.
///usage
this.nIcon.Icon=icon[0];
this.statusBarPanel1.Icon=icon[0];
///.

Dynamic Menus:

We created dynamic menus in our application (example: The main menus as well context menus, which are created based upon the values from the database instead of hard coding). Actually there isn't much difference in code between these two menu types. To create a main menu in our application, double-click on the Main Menu icon from the toolbox. Similarly, double-click on the Context Menu icon from the toolbox to create the context menu. We will have a look at code snippets, which dynamically add menu items:

///.
///adds status messages into main and context menus
this.cMenuMyStatus.MenuItems.Add(n.InnerText ,new System.EventHandler(this.cMenuMyStatus_Click));
this.menuMyStatus.MenuItems.Add(n.InnerText ,new System.EventHandler(this.cMenuMyStatus_Click));
///.

System Tray Icon:

In this messenger client application we added a system tray icon to the system tray. Visual Studio .NET eases the job of creating a tray icon. From the toolbox, double click on the NotifyIcon tool and Context Menu tool. On the property page of NotifyIcon, set the contextmenu property to the name of the context menu we added from the toolbox, and select an icon for the Icon property. The system tray icon with context menu is now ready for our application. You can load icons dynamically at runtime as shown below in the code snippet:

///.
icon[0]=new
            Icon(Configuration.sGraphicsDir +
            "onlinewhite.ico");
///.
///.
this.nIcon.Icon=icon[0];
///.

What's happening behind the scenes

We can use the remoting debugging tool developed by Mathew Reynolds in http://www.csharptoday.com/content.asp?id=1525&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq to see what's happening behind the scenes. The screen shot below is taken from his utility when our client tries to authenticate our user.

Figure 5: Remoting debugging tool with SOAP messages

To make our client application work with the debugging tool do the following:

  • 1. Remove the reference "MessengerObject " and reference it back again.
  • 2. Change the sRemotingURL static member inside the configuration class to http://localhost:1009/Messenger?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq instead of http://localhost:1010/Messenger?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq.
  • 3. Run our server "MessengerServer.exe "
  • 4. Run the Debugging tool ("WroxTrace.exe ")
  • 5. Run our client application ("Client.exe ").

All the above-mentioned files are available inside our download for this article. Now you can see the SOAP messages flowing between the client and the server as shown in figure 5. You can use this tool for debugging, to check what's happening in between communication, so that you can improve the efficiency of the system.

Conclusion

In this article we have seen how to develop a messenger client application exploring some of the .NET concepts like remoting, and how to develop a sophisticated windows forms application. If you are more interested in knowing more about instant messaging you can visit http://www.imunified.com/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq it's a forum formed for all instant messaging companies.

If you'd like to discuss a broader range of C#Today topics and issues, or if you have any C# related questions you need answers to, please visit our online resource centres at : http://p2p.wrox.com/list.asp?list=csharptoday_discuss&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq and http://p2p.wrox.com/list.asp?list=c_sharp&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq


Writing a Messenger Application Using C# - Part 1

Writing a Messenger Application Using C# - Part 1
http://www.csharptoday.com/content.asp?id=1703
Article Rating: 4.5
Published on November 22, 2001
Abstract
A messenger application provides users with a simple way to communicate instantly with each other. This article covers an approach to writing a sample realtime chat messenger application, much like the existing ones like MSN, Yahoo, AOL etc using C#. In this first part of this article, Saravana Kumar covers the data model, the core functionality assembly, the server, and how it is configured using .NET remoting to expose it to remote clients. The second part of the article will cover the client applications.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

A messenger application provides users with a simple way to communicate instantly with each other. This article covers an approach to writing a sample realtime chat messenger application, much like the existing ones like MSN, Yahoo, AOL, etc, using C#. In the first part of this article, we will cover the data model, the core functionality assembly, the server, and how it is configured using .NET remoting to expose it to remote clients. The second part of the article will cover the client applications.

We'll call our application 'Wrox Messenger Service.'

System Requirements

  • Visual Studio .NET Beta 2
  • Windows 2000
  • SQL Server (Version 7.0 or greater)

Architectural overview of Wrox Messenger Service:

Figure 1: WMS Architecture

WMS uses a client-server architecture. All communications between clients are routed through the server. The main advantage of this architecture is that complexity is moved from client to the server. This makes it relatively easy to update the functionality of the system and to make the client as thin as possible. The main disadvantage is that communication between clients is not secure, and the server is introduced as another possible point of failure.

WMS clients communicate in XML through HTTP. The XML used for communication is a SOAP message created through the use of a SOAP formatter (explained later).

Figure 1 shows the basic message flow architecture between clients. Now let's cover the core parts of the application:

  • Database model
  • "MessengerObject" Class
  • Messenger Server, and
  • The Client application

Database Model

In order to make it as simple as possible, let us keep the data model straightforward. We use four SQL Server tables, namely:

  • tblUser (Contains user information like name, id, password, etc)
  • tblFriendsList (Contains user contact list - Friends)
  • tblStatusMessages (to store status messages like Away, Offline, Online, etc)
  • tblLoginStatus (Core table - acts as a lookup table for all communication. Updated whenever the user log into the server, log out of the server, and if user changes his/her status)

Figure 2: Data model diagram

Creating the "MessengerObject" Class (MessengerObject.dll assembly)

The MessengerObject class is the heart of our application, and we will spend most of the article explaining it. All the basic functionality required for the clients to communicate is implemented in this class.

We'll first list all the method names that we will implement on our "MessengerObject" class:

  • private string getConnectionString ()
  • private bool validateUser (string sUserID, string sPassword)
  • private bool changeStatusInDB (string sUserID, int iStatus, int iStatusCommand, int iObjRefNo)
  • private bool updateObjRefNoinDB (int iRefNo)
  • private string sendStatusList ()
  • private string userStatusMessage (string sUserID)
  • private void sendMessage (string message)
  • public string getMessage (string sMessage)
  • public void addUser (string sUserID, ref Label objLabel)
  • public void removeUser (string sUserID)
  • public string sendOnlineAndOfflineUserList (string sUserID)
  • public void sendStatusChangeToAllFriends (string sUserID, int iStatus)

The functional architecture of MessengerObject is shown below:

Figure 3: Functional overview of the MessengerObject class

Let us begin coding.

Create a new Visual C# - Class Library project called MessengerObject.

Rename the default class file to MessengerObject.cs and the class name to MessengerObject.

Include the following namespaces inside MessengerObject.cs file:

using System;
using System.Xml;
using System.Data;
using System.Data.OleDb;

Include the following private members inside the class, which are used for database handling:

//To handle database
private DataTable
            objDT;
private DataSet objDS;
private OleDbDataAdapter objDA
            ;
private OleDbConnection objCN;

//To maintain Database
            Connection string
private string sConnectionString;

Using XML classes

We will maintain a configuration XML file, containing configuration properties like the server name, database name, and connection string. This will make future development easier.

Below is an example:

<?xml version="1.0" encoding="utf-8" ?>
<Messenger>
<Database>
<ServerName>localhost</ServerName>
<DatabaseName>Northwind</DatabaseName>
<ConnectionString>Provider=SQLOLEDB;Data source=localhost;
Integrated Security=SSPI;Database=Northwind</ConnectionString>
</Database>
</Messenger>

In order to retrieve the connection string stored in the XML configuration (Config.xml) file, we need to use the XmlDocument and XmlNode classes in the System.Xml namespace. First load the XML file into the document object model using the load method, and use XPath syntax to retrieve the connection string. Store this in global variable sConnectionString. We can use this variable content wherever we need a connection string to connect to the database.

Add a method getConnectionString and include the code below:

Listing 1 getConnectionString method

private string
            getConnectionString(){
XmlDocument objDomDocument;
XmlNode
            objDomNode;
objDomDocument = new XmlDocument();
objDomDocument.Load("Config.xml");
objDomNode =
            objDomDocument.SelectSingleNode("Messenger");
sConnectionString=objDomNode.SelectSingleNode("Database/ConnectionString").InnerText;
return sConnectionString;
}

This stage of a messenger application is to allow the user to sign in to the server. The username and password provided by the user are checked against the database entry in the table tblUser, and a boolean value of either true or false is returned to the client (since it's a sample tutorial application, security aspects are not given importance, but when it comes to a real time environment, the password must be sent to the server using some form of encryption. You can read more about this at http://www.csharptoday.com/content.asp?id=1540&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq).

Add a method called validateUser and include the code below:

Listing 2 the validateUser method

public bool validateUser(string sUserID,
            bstring sPassword){
string sUserAuth;

//Query which will
            return some data if the supplied username and //password exists in
            the database
sUserAuth="select cUserID,cPassword from tbluser
            where cUserID='" +
sUserID.Trim() + "' and cPassword ='" +
            sPassword.Trim() + "'";

objDA = new
            OleDbDataAdapter(sUserAuth,GetConnectionString());
objDA.Fill(objDS,"user");
objDT =
            objDS.Tables["user"];

if (objDT.Rows.Count > 0)
return
            true;
else
return false;
}

Sending online-offline user list and status list to the client

Once the login process is completed, the next step is to send successfully logged in users an online/offline friends list, and their availability status messages from the database. The method sendOnlineAndOfflineUserList is used to perform this task. The friends and their availability form an XML string, which is sent back to the client application.

Add a method called sendOnlineAndOfflineUserList and add the code listed in listing 3.

Listing 3 the sendOnlineAndOfflineUserList method

public string
            sendOnlineAndOfflineUserList(string sUserID){
string
            sOnline,sOffline;
string sReturnXML="<UserList>";
string
            sOnlineXML="<Online>";
string
            sOfflineXML="<Offline>";
string sStatusList;
string
            sTemp=null;

//create xml for status messages - sendStatusList
            method queries the
//database and return a xml string with all
            available status messages
//in the database
sStatusList =
            sendStatusList();

//query to pick up online friends of the
            user
sOnline="select id,cUserID,cFirstName from tbluser where id
            in (select iFriendid from " &
"tblFriendsList f where
            f.id=(select id from tbluser where cUserID='" + sUserID.Trim() +"')
            " &
"and iFriendid in (select id from tblLoginStatus where
            bStatus=1 and iStatusID > 0))";

objDA = new
            OleDbDataAdapter(sOnline,GetConnectionString());
objDA.Fill(objDS,"Online");
objDT
            = objDS.Tables["Online"];

//Create one element for each
            online friend
foreach(DataRow objDR in objDT.Rows){
sTemp +=
            "<U>" + objDR["cUserID"].ToString().Trim() +
            "</U>";
}

sOnlineXML = sOnlineXML + sTemp +
            "</Online>";
sTemp=null;

//query to pick up offline
            friends of the user
sOffline="select id,cUserID,cFirstName from
            tbluser where id in (select iFriendid from tblFriendsList f "
            &
"where f.id=(select id from tbluser where cUserID='" +
            sUserID.Trim() +"') and iFriendid in " &
"(select id from
            tblLoginStatus where bStatus=0 or iStatusID =0))";

objDA =
            new
            OleDbDataAdapter(sOffline,GetConnectionString());
objDA.Fill(objDS,"Offline");
objDT
            = objDS.Tables["Offline"];

//Create one element for each
            offline friend
foreach(DataRow objDR in objDT.Rows){
sTemp +=
            "<U>" + objDR["cUserID"].ToString().Trim() +
            "</U>";
}

sOfflineXML = sOfflineXML + sTemp +
            "</Offline>";

//Combine all bits into one single xml
            string
sReturnXML=sReturnXML + sOnlineXML + sOfflineXML +
            sStatusList +"</UserList>";

return
            sReturnXML;
}

The resultant XML looks like this:

Adding the user to the server

After sending the contact list and status message list back to the client, we need to do some work on the server. The tasks are:

  • Keep track of the user in the server, by adding a reference to uniquely identify the user
  • Send the user's online status to all his/her friends
  • Update the database with the user's Online Status and unique reference index
  • Optionally, you can send a status message to the server console or event log

Add these two static members inside the MessengerObject class, which is going to keep track of the client references in the server.

static ArrayList aUserIDS = new ArrayList();
static ArrayList aUserReference = new ArrayList();

Add a method called addUser and include the following code in Listing 4.

Listing 4 the addUser method

public void addUser(string sUserID, ref
            Label objLabel){
bool userInsertStatus;
try{
aUserIDS.Add(sUserID.Trim());
aUserReference.Add(objLabel);

if(aUserIDS.Count >
            1){sendStatusChangeToAllFriends(sUserID,1); }

userInsertStatus=changeStatusInDB(sUserID,1,1,aUserIDS.IndexOf(sUserID));
sendMessage( sUserID + " -- has Joined") ;
}
catch
            (Exception objExp){Console.WriteLine(objExp.Message);}
}

Uses of the addUser method

I. Keeping track of the user in the server, by adding a reference

This is the first, as well as the most important, step in the application. So, how we are going to keep track of the client reference? We use the simple approach of referencing a label box control in the client application as a client reference, whose text will be changed constantly with XML strings. After client validation, the addUser method in the server will be called from the client by passing a label box control reference as one of the arguments. This reference will be added to an arraylist aUserReference by the addUser method.

The concept is simple; we are trying to throw an event on the client side from the server. In this case it is the label box text change event. When we change the text of the label box reference with suitable XML strings in the server, it automatically throws the label box text change event on the client, which will be handled by the client. The core parts of this application are these objects, which use the ArrayList class:

  • aUserIDS
  • aUserReference

These are collection objects, which can store any type of object inside its collection. In our example we use them to store a collection of strings (user ids) and label box control references.

II. Sending the user's online status to all their friends

Once we add the client reference in the server, we need to send the user's online status to all their friends who are also online. For example:

  • User 'bgates' logs into the server
  • 'bgates' has two friends 'julian' and 'mike'
  • 'julian' is online and 'mike' is offline when 'bgates' logged in
  • So, 'bgates' online status needs to be sent to 'julian'.

This is done within the method sendStatusChangeToAllFriends.

If 'bgates' is the first user to login to the server then there is no need to send a message to anyone, because its obvious nobody is online. So we don't need to call sendStatusChangeToAllFriends. We'll discuss this method a little later, after explaining more uses of the addUser method.

III. Updating the database with a user's online status and unique reference index assigned to the user in the server

After the user has logged into the server, we store the user information in the database. By default, the tblLoginStatus table will have one entry for each user in the tblUser table. The default values are 0 (meaning offline) for the bStatus field, 100 for the iStatusID field, and -1 for the iObjectRefNo field.

Figure 4: Default values

Figure 5: After the first user has logged into the server

Add a method called changeStatusInDB and include the following code:

Listing 5 the changeStatusInDB method

private bool changeStatusInDB(string sUserID,int iStatus,int
iStatusCommand,int iObjRefNo)
{
//iStatusCommand ## 0-delete;1-insert;2-update ##
//calls the stored procedure InsertStatusChange with required
//parameters. Used to insert user status changes in appropriate
//database tables

try
{
objCN = new OleDbConnection(GetConnectionString());
OleDbCommand objCMD = new OleDbCommand("InsertStatusChange", objCN);
objCMD.CommandType = CommandType.StoredProcedure;

OleDbParameter objPARAM1 = objCMD.Parameters.Add("@sUserID", OleDbType.VarChar, 15);
objPARAM1.Value = sUserID;

OleDbParameter objPARAM2 = objCMD.Parameters.Add("@status", OleDbType.Integer);
objPARAM2.Value = iStatus;

OleDbParameter objPARAM3 = objCMD.Parameters.Add("@statusCommand", OleDbType.Integer);
objPARAM3.Value = iStatusCommand;

OleDbParameter objPARAM4 = objCMD.Parameters.Add("@iObjectRefNo", OleDbType.Integer);
objPARAM4.Value = iObjRefNo;

objCN.Open();
OleDbDataReader objREADER = objCMD.ExecuteReader();
objREADER.Close();
objCN.Close();
return true;
}
catch(Exception objExp){
Console.WriteLine(objExp.Message);
return false;
}
}

IV. Sending a message to the Console

Finally (optional), you can send a message to the server console saying the user has logged in.

Add a method called sendMessage and include the following code:

Listing 6 the sendMessage method

private void sendMessage(string message){

Console.WriteLine ("Added Message : {0}", message);

}

Sending status changes to all of a user's friends

When a user logs in or out of the server, we need to inform their friends about the change. The method SendStatusChangeToAllFriends does the job for us. As well as sending online/offline notification, this function also notifies the user's friends of any change in the user's status in the client application. The message is passed to all of the user's friends in the form of XML.

For example, if 'bgates' changes his status to away in the client application, then this method will be called which in turn will send the XML message shown below to all of his friends who are online:

<M><S m='1' t='Away'>bgates</S></M>

Add a method called SendStatusChangeToAllFriends and insert the following code.

Listing 7 the SendStatusChangeToAllFriends method

public void
            sendStatusChangeToAllFriends(string sUserID,int iStatus){
//iStatus -- 1:online; 0:offline
//sMessage -- like away,out
            to lunch...etc

string sObjectRefNos;
string sMessage;

//update the datebase table with new status message the user
            selected
//by calling the stored procedure
            "updateUserStatus"
objCN = new
            OleDbConnection(GetConnectionString());
OleDbCommand objCMD =
            new OleDbCommand("updateUserStatus", objCN);
objCMD.CommandType =
            CommandType.StoredProcedure;

OleDbParameter objPARAM1 =
            objCMD.Parameters.Add("@sUserID",
OleDbType.VarChar, 5);

objPARAM1.Value = sUserID;

OleDbParameter objPARAM2
            = objCMD.Parameters.Add("@iStatusID",
OleDbType.Integer);
objPARAM2.Value = iStatus;

objCN.Open();
OleDbDataReader objREADER =
            objCMD.ExecuteReader();
objREADER.Close();
objCN.Close();

Label oUserTo = new Label();

//Query to get users
            online friends list
sObjectRefNos="select iobjectRefNo,iStatusID
            from tblloginstatus where bstatus <> 0 " &
"and id in
            (select ifriendid from blfriendslist where id= " &
"(select
            id from tbluser where cUserID ='" + sUserID+ "'))";

//get
            users current status message.
sMessage =
            userStatusMessage(sUserID);

objDA = new
            OleDbDataAdapter(sObjectRefNos,GetConnectionString());
            objDA.Fill(objDS,"ObjectRefNos");
objDT =
            objDS.Tables["ObjectRefNos"];

//retrieve user friends who
            are online from the database and change
//their reference label
            control text with the xml string formed here.

foreach(DataRow
            objDR in objDT.Rows){
oUserTo=(Label)aUserReference[(int)objDR["iobjectRefNo"]];
oUserTo.Text = "<M><S m='" + iStatus + "' t='"+
            sMessage +"'>" + sUserID + "</S></M>";
}
}

How does the function work?

We first query the database for the unique reference IDs of all of the user's friends stored in the table tblLoginStatus. Then we picks up the corresponding reference stored in the arraylist aUserReference. Then, we parse the references into an XML string.

This string will throw the text change events in the appropriate clients, which will be handled by the client application.

Add a method called userStatusMessage, which will be used inside the method SendStatusChangeToAllFriends to get the user's current status, and insert the below.

Listing 8 the userStatusMessage method

private string userStatusMessage(string
            sUserID){

string sUserStatusMessage;

//Query which
            will return users current status base upon the user
            id
sUserStatusMessage="select sMessage from tblStatusMessages
            where iStatusID=" &
"(select istatusid from tblloginstatus
            where id = " &
"(select id from tbluser where cuserid='" +
            sUserID + "'))";

objDA = new
OleDbDataAdapter(sUserStatusMessage,GetConnectionString());
objDA.Fill(objDS,"UserStatusMessage");

objDT
            = objDS.Tables["UserStatusMessage"];
DataRow objDR;
objDR =
            objDT.Rows[0];
return objDR["sMessage"].ToString();
}

Removing the user from the server when the user logs out

If the user logs out of the server, then all of the resources occupied by the user need to be removed. The method removeUser is used for this purpose. The steps taken inside this method are:

  • Send status change (offline) to all of the user friends
  • Remove unique reference from the server
  • Update database tblLoginStatus, which stores status id and object reference index number, and
  • Optionally send a message to the server console

The above steps are more or less the same as what we did with the addUser method.

Add a method called removeUser and insert the following code.

Listing 9 the removeUser method

public void removeUser(string
            sUserID){
int iUserPosition;
bool
            userDeleteStatus;
try{
sendStatusChangeToAllFriends(sUserID,0);

iUserPosition=aUserIDS.IndexOf(sUserID);
aUserReference.RemoveAt(iUserPosition);
aUserIDS.RemoveAt(iUserPosition);

userDeleteStatus=changeStatusInDB(sUserID,0,100,iUserPosition);
//In
            collections such as arraylist, the elements that follow the removed
//element move up to occupy the vacated spot, these
            changes
//need to be updated in the Database since we are
            maintaining the object
// ref number in the
            DB
updateObjRefNoinDB(iUserPosition);
sendMessage(sUserID + "
            -- Disconnected ") ;
}
catch(Exception
            ed){
Console.WriteLine(ed) ;
}
}

Add a method called updateObjRefNoinDB and insert the following code.

Listing 10 the updateObjectRefNoinDB method

private bool updateObjRefNoinDB(int
            iRefNo){
objCN = new
            OleDbConnection(GetConnectionString());
OleDbCommand objCMD = new
            OleDbCommand("updateObjRefNos", objCN);
objCMD.CommandType =
            CommandType.StoredProcedure;

OleDbParameter objPARAM1 =
            objCMD.Parameters.Add("@iRefNo",
            OleDbType.Integer);

objPARAM1.Value =
            iRefNo;

objCN.Open();
OleDbDataReader objREADER =
            objCMD.ExecuteReader();
objREADER.Close();
objCN.Close();
return
            true;
}

Note: The User registration part is not implemented here. The SQL script in the download contains scripts to populate the tables with initial values, such as user information, default status, etc.

Next we'll look at how to send messages between users. Once we finish this part the whole framework is complete. It's very simple to write the function that transfers message from one user to another. Add a method called getMessage and insert the following code.

Listing 11 the getMessage method

public string getMessage(string
            sMessage){
Label oUserTo = new Label();
XmlDocument
            oDomDocument = new XmlDocument();
XmlNode oDomNode;

string
            sUserTo;
string sUserFrom;
string sBodyText;
string
            sXML=sMessage;

oDomDocument.LoadXml(sXML.Trim());
oDomNode
            =
            oDomDocument.SelectSingleNode("M");

sUserTo=oDomNode.SelectSingleNode("T").InnerText;
sUserFrom=oDomNode.SelectSingleNode("F").InnerText;
sBodyText=oDomNode.SelectSingleNode("B").InnerText;

//Pick
            up the corresponding label box reference from the arraylist
//mentioned in the to field of the
            sMessage

oUserTo=(Label)aUserReference[aUserIDS.IndexOf(sUserTo.Trim())];

//Replace
            the text of the label box control with suitable xml
//string
            formed here

oUserTo.Text="<M><F>" +
            sUserTo.Trim() + "</F><T>" + sUserFrom.Trim()
+
            "</T><B>" + sUserFrom.Trim() + "
says:\n " +
            sBodyText.Trim() + "</B></M>";

return
            sMessage;
}

This function works in a very similar way to SendStatusChangeToAllFriends. The only difference between these methods is the XML structure. In SendStatusChangeToAllFriends the XML looks like this:

<M><S m='1' t='Away'>bgates</S></M>

But, in the getMessage method, the XML has a "from" element <F/>, a "to" element <T/>, and a "body" element <B/>. For example:

<M>
<F>bgates</F>
<T>saravana</T>
<B>bgates Says: \n Hello there</B>
</M>

This method is called from the client application with the appropriate XML string as the argument, which contains "from", "to", and "body" tags. The content of the body tag is sent to the client mentioned in the "to" field, by picking up the corresponding client reference from the arraylist aUserReference.

A Test Function to prove our concept

Let us create a dummy function as proof of concept (to trigger a client side event from the server).

Add a method called testFunction and insert the following code.

Listing 12 the testFunction method

public void
            testFunction(System.Windows.Forms.Label l){
l.Text="Well
            done...You got your framework to work successfully
            ";
sendMessage("Client call is processed")
}

Compile the MessengerObject class by pressing Ctrl + Alt +B.

This creates an assembly file called MessengerObject.dll.

That's it. The MessengerObject class is complete. Next we will work on the server part to expose this MessengerObject to remote clients.

Creating the Server

Create a new Visual C# - Console Application project and name it MessengerServer.

Rename the default class MessengerServerSettings.cs and delete all existing default code inside the file.

As we discussed earlier about the addUser method, the database table tblLoginStatus needs to be populated with default values for every registered user. The MessengerServerSettings class is used for this purpose.

In the file MessengerServerSettings.cs include the following namespaces:

using System.Data.OleDb;
using
            System.Data;
using System.Xml;
using System;

Create a class called MessengerServerSettings and include the following private member variables:

private DataTable objDT;
private DataSet objDS;
private OleDbDataAdapter objDA ;
private OleDbConnection objCN;
private string sConnectionString;

private XmlDocument objDomDocument;
private XmlNode objDomNode;

Add a method called getConnectionString and include the following code. This method just retrieves and returns the connections string that is stored in the config.xml file.

public string
            getConnectionString()
{
objDT = new DataTable();
objDS =
            new DataSet();
objDomDocument = new
            XmlDocument();
objDomDocument.Load("Config.xml");
objDomNode =
            objDomDocument.SelectSingleNode("Messenger");
sConnectionString=objDomNode.SelectSingleNode("Database/ConnectionString").InnerText;
return
            sConnectionString;
}

Add another method called initializeServer and include the following code. This method just calls a stored procedure in the database, which updates the table tblLoginStatus with default values.

public bool
            initializeServer()
{
try
{
//calls the stored procedure
            ServerStartupProcedures to populate default //values into the
            database tables
objCN = new
            OleDbConnection(GetConnectionString());
OleDbCommand objCMD = new
OleDbCommand("ServerStartupProcedures",
            objCN);

objCMD.CommandType =
            CommandType.StoredProcedure;

objCN.Open();
OleDbDataReader
            objREADER =
            objCMD.ExecuteReader();
objREADER.Close();
objCN.Close();
return
            true;
}
catch(Exception
            objExp){
Console.WriteLine(objExp.Message);
return
            false;
}
}

We don't need anything more inside this class. Let us now start with our MessengerServer class, which will use remoting to expose the MessengerObject assembly to remote clients.

With DCOM, we are used to calling methods on objects running on a server. The programming models are always the same if objects are used on the server or on the client. Without DCOM, we have to deal with ports and sockets, pay attention to the target platforms because of possibly different data representations, and build a custom protocol, where messages are sent to the socket so that we finally call some methods. DCOM handles all these issues for us. The replacement for DCOM is .NET remoting. In contrast to DCOM, .NET remoting can also be used in Internet solutions and is what makes it more powerful and flexible. DCOM is not flexible and efficient enough for use in Internet solutions. It's possible to adapt and extend every part of the architecture with .NET remoting, so it fits for nearly all remoting scenarios (for more information on remoting, see the links at the bottom of the page).

Put simply, a messenger application is just about moving information bits from point A to point B. So, how are we going to establish a connection between point A and point B? That's where remoting comes into the picture.

Figure 6: Message Flow Architecture

Since both the client and the server reside on different machines/processes, there needs to be a mechanism to establish a relationship between the two. Microsoft .NET remoting provides a rich and extensible framework for objects living in different machines/processes to communicate with each other seamlessly. .NET remoting offers a very powerful yet simple programming model and runtime support for making these interactions transparent.

The client interacts with the server with the help of the HTTP protocol. The message sent from the client to the server is a SOAP message transmitted over HTTP. So, we need to convert all the messages that need to be sent to the server in the form of a SOAP message. The SOAP formatter in the .NET remoting framework does the job for us.

All outbound messages are passed through the SOAP formatter (as shown in figure 6), where the message is changed into XML and serialized, and the required SOAP headers are added to the stream.

Next comes the channel. Channels can be thought of as transport mechanisms to pass messages to and from remote objects. All the method calls are passed to the server through the channel. A channel is a type that takes a stream of data, creates a package according to a particular network protocol, and sends the package to another computer / process. NET remoting supports two types of channel:

  • HttpChannel
  • TcpChannel

Since all our communication takes place via HTTP protocol, we use the HTTP Channel in our application.

The HTTP channel transports messages to and from remote objects using the HTTP protocol. The main advantages of an HTTP channel are:

  • Provides communication between sender and receiver using the HTTP protocol as a transport.
  • Provides encoding of payloads in SOAP, an XML encoding standard.
  • Is compliant with SOAP 1.1.

Let us start coding MessengerServer.

Add the following references to the application:

  • MessengerObject.dll
  • System.Runtime.Remoting

      Add a new file, MessengerServer.cs and include the following namespaces:

      using System;
      using System.Runtime.Remoting;
      using System.Runtime.Remoting.Channels.Http;
      using System.Runtime.Remoting.Channels;

      Have a look at the server code below. It's only few lines of code. Most of the complex parts like establishing a port to listen for the client request, processing the client request, converting the message into appropriate network messages, etc, are already done by the .NET remoting framework.

      namespace com.Remoting
{
public class
            MessengerServer {
public static void Main (){

//Initialize
            database settings
MessengerServerSettings objS = new
            MessengerServerSettings();
objS.InitializeServer();

//Register
            channel for remoting
int listeningChannel = 1010;
HttpChannel
            channel = new
            HttpChannel(listeningChannel);
ChannelServices.RegisterChannel
            (channel);
RemotingConfiguration.RegisterWellKnownServiceType(Type.GetType
("com.Remoting.MessengerObject,MessengerObject"),
"Messenger",
            WellKnownObjectMode.SingleCall);

Console.WriteLine
            ("Messenger Server is up and running on port {0}", isteningChannel);
Console.WriteLine ("Press enter to stop the
            server...");
Console.ReadLine ();
}
}
}

      In the first two lines, we set up the database with initial values.

      Then when the server is started, we need to create and register a HTTP channel that will listen for clients connecting on a specific port (in our case we use port 1010). We also need to register the remote object with the remoting framework by calling RegisterWellKnownServiceType. The parameters for this call are as follows:

      • The full type name of the object being registered, in this case com.Remoting.MessengerObject.
      • The assembly name
      • The name of the endpoint where the object will be published. Clients need to know this name in order to connect to the object. Any string will work; for this example, use "Messenger".
      • The object mode, which can be SingleCall or Singleton.

      We use SingleCall object mode while registering our service. This services one and only one request coming in at a time. Single call objects are useful in scenarios where the objects are required to do a finite amount of work, and are usually not required to store state information. Single Call objects can be configured in a load-balanced fashion.

      Although the server process knows everything about each unique Object type, the client knows only that it wants a reference to an object in another application domain, perhaps on another computer. A uniform resource locator (URL) will identify the server. The URLs that represent unique types to the outside world are activation URLs, which ensure that your remote call is made to the proper type.

      In our case, the URL of our server is http://localhost:1010/Messenger.

      Compile the application by pressing Ctrl + F5 which will compile and start an .exe file.

      Proof of concept (Sample client Application)

      In order to check our remoting set up and our concept of throwing client side events from the server, let us build a sample client program to illustrate. It's a simple windows form application with just a label box control and a button as shown below:

      Figure 7: Proof of concept Client application.

      The figures below show the status of the client window, alert message and the server console after clicking the button.

      As soon as the button is clicked, the text of the label box will be changed by the function testfunction in the server. Because of that, a text change event will be fired on the client, which pops up a message box saying text change event triggered. If this happens then everything is fine.

      Figure 9: Alert caused because of label box control text change event

      Figure 10: Message displayed on server console after processing client request

      Look at the code snippet below. Let us call the method testfunction with the label box reference as the argument:

      //Inside button click
            event

com.Remoting.MessengerObject objTest;
HttpChannel
            channel;
this.button1.Enabled=false;

int listeningChannel
            = 0;
channel = new
            HttpChannel(listeningChannel);

ChannelServices.RegisterChannel(channel);
objTest
            = (com.Remoting.MessengerObject) Activator.GetObject
(typeof(com.Remoting.MessengerObject),Configuration.sRemotingURL);

objTest.testfunction(this.label1);

//
            Inside Label box text change event

MessageBox.Show("text
            change event triggered ");

      What we've done, and what's next!

      In this article we've discussed our implementation assembly MessengerObject and the messenger server. Now, our server is ready to accept requests from various remote clients. In the next article we will develop a sophisticated client application to interact with the server. We will also discuss how to debug a remoting application like this, using the tool developed by Matthew Reynolds in his article at http://www.csharptoday.com/content.asp?id=1525&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq).

      • Brought to you by ownSky!


Interoperating Java, VB6, and .NET Web Services

Interoperating Java, VB6, and .NET Web Services
http://www.csharptoday.com/content.asp?id=1738
Article Rating: 3.5
Published on October 25, 2001
Abstract
In this article, Catalin Tomescu illustrates the integration and interoperability of web services. He creates a Tax Calculator web service, which integrates two existing applications: a State Tax Calculator and a Federal Tax Calculator. To further complicate things, the two web services that our tax calculator integrates are built using different technologies – the state tax calculator application is built using Apache SOAP for Java and the federal tax calculator application is built using Visual Basic 6.0 and the MS SOAP 2.0 Toolkit. Finally, we see how to call our Tax Calculator web service from a variety of clients - a C# console application, a VB 6 console application, and an ASP.NET web application.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

There is little doubt that web services are the future of distributed applications. We've seen a tremendous increase in the number of web services available on the Internet. Stock checkers, weather services, book finders, map and address finders - you name it. We don't need to reinvent the wheel; instead we integrate services offered by online providers and our application is almost ready. This article will illustrate how to create a tax calculator web service by integrating two existing applications: a State Tax Calculator and a Federal Tax Calculator. To further complicate things, the two web services that our tax calculator integrates are built using different technologies - the state tax calculator application is built using Apache SOAP for Java, and the federal tax calculator application is built using Visual Basic 6.0 and the MS SOAP Toolkit (STK).

Not long ago, SOAP emerged as a standard for transferring XML data over HTTP, and many companies rewrote their applications using this standard. Most of the new development is focused on writing services and not standalone applications. As we should expect, not all providers are using the same programming language, the same tools, or the same operating system in their development efforts. This creates problems when writing clients for the various different implementations of SOAP. Although most of the various implementations of the SOAP protocol are more or less compliant with the standard, there are certain important differences - for example, the lack of WSDL (unless we use IBM's WebServices ToolKit) in the Java implementation for web services makes interoperability with frameworks like .NET and MS SOAP Toolkit less straightforward than we might expect. In this article, we'll see how to overcome these problems.

Prerequisites

  • MS SOAP Toolkit Gold 2.0
  • Visual Basic 6.0
  • MS XML 3.0
  • MS .NET Framework Beta 2
  • Apache SOAP for Java 2.2
  • Tomcat 4.0.1
  • JDK 1.3.2
  • Xerces 2.0
  • IIS installed on a Windows NT or 2000 platform (Professional edition is fine for our sample).

Reading this first will aid your understanding of this article:

  • The SOAP specification available at http://www.w3c.org/soap?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq.
  • The WSDL specification available at http://www.w3c.org/WSDL?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq.

What we Learn

The article first shows how to build the two underlying web services, a Java State Tax Calculator, and a VB 6.0 Federal Tax Calculator. Then, our .NET Tax Calculator integrates these two web services to provide a single interface for calculating the amount of income tax an individual owes. (In our scenario the income tax is calculated as a sum of federal and state tax). Finally, we will see how to create a couple of different clients to consume the Tax Calculator web service.

Federal Tax Calculator

The Federal Tax Calculator is a web service that computes the amount of federal tax to be deducted, given the annual gross income and marital status of a person. This, of course, is a simplified business logic for federal tax computation.

The application is developed using MS SOAP Toolkit 2.0. The particular thing about this framework is that it assumes that web services are implemented as COM objects. The STK helps us interface the COM object with SOAP requests and responses by providing two types of SOAP listeners:

  • An ASP listener. This listener is implemented as an ASP page that decodes the SOAP request, invokes the COM method on the specified COM object, and packs the response as a SOAP response object.
  • An ISAPI listener. This listener is implemented in the SOAPISAP.dll and performs the same actions as the ASP listener.

The process of enabling a COM object for SOAP communication consists of generating a WSDL file. When we generate the web service .WSDL file from the COM object, we must choose the type of listener we want to use for the particular service. Note that there is no restriction on using both of them for the same web service, as long as the names of the .WSDL files are distinct. The ISAPI Listener is the default one and the one we will use.

Now let's write the COM object which contains the implementation of our FedTaxCalculator. You can write the COM object with whatever tool you want (VC++, VB, Delphi, etc). Here we use VB 6.0.

Here is the content of the FedTaxCalculator.cls file containing the FedTaxCalculator class:

Option Explicit

Public Function ComputeTax(AnnualGrossIncome As Double, MaritalStatus As String) As Double

Dim taxPercent As Single

If AnnualGrossIncome <= 75000 Then
taxPercent = 0.25
ElseIf AnnualGrossIncome <= 150000 Then
taxPercent = 0.3
End If

If MaritalStatus = "M" Then
taxPercent = taxPercent - 0.03
End If

ComputeTax = AnnualGrossIncome * taxPercent
End Function

Create a new VB project, call it FedTaxCalculator, add a class called FedTaxCalculator, and paste the above code into it. Save the project and build the .dll file.

Register the .dll using regsvr32 from the Run command line:

Regsvr32 <path>\FedTaxCalc.dll

Deployment

Using the WSDL Generator utility (which comes with MS SOAP Toolkit), generate the .WSDL file for the FedTaxCalculator service.

When prompted, use a virtual directory called TaxCalculator. If using the ISAPI listener, configure your virtual directory to use SOAPISAP.dll for files with a .wsdl extension (see STK documentation and setup instructions in the download for more details). After we have generated the WSDL file our service is ready to respond to requests.

State Tax Calculator

The State Tax Calculator is a web service that computes the amount of state tax to be deducted, based upon the State we are in. This web service is built using the Apache SOAP for Java Toolkit. Similarly to the MS SOAP Toolkit, we can reuse our existing Java classes and expose them as web services. The Apache implementation of SOAP differs from Microsoft's approach (for both .NET and the SOAP Toolkit) in that the Apache SOAP doesn't rely on .wsdl files to describe a web service. Instead, the Apache SOAP expects us to utilize all the features that the SOAP envelope offers and pass all the information the Apache SOAP needs to make the method invocation:

  • Parameter names
  • Parameter types
  • Parameter order
  • Method call
  • How to serialize the result and how to de-serialize input parameters

Because of this requirement there are couple of problems when trying to connect from a client built with Microsoft toolkits. We'll look at the solution later in this section, but for now let's focus on the code for our StateTaxCalculator service.

Package samples.TaxCalculator;

import
            java.*;

public class StateTaxCalculator {
/**
* This
            function computes the amount of state tax for 2 fictitious states,
            XX and YY.
* The function implements different rules for each of
            the states.
*/
public double ComputeTax(double
            AnnualGrossIncome, String MaritalStatus,
String State, int
            MonthsInState)
{
double taxPercent = 0, tax = 0;

if
            (State.equals("XX")) {
if (AnnualGrossIncome <
            75000)
taxPercent = 0.02;
else
taxPercent =
            0.035;

if (MaritalStatus.equals("M"))
taxPercent -=
            0.005;

tax = AnnualGrossIncome * taxPercent * MonthsInState /
            12;
}
else if (State.equals("YY")) {
if (AnnualGrossIncome
            < 85000)
taxPercent = 0.03;
else
taxPercent =
            0.045;

if (MaritalStatus.equals("M"))
taxPercent -=
            0.003;

if (MonthsInState > 8)
tax = AnnualGrossIncome *
            taxPercent;
else
tax = AnnualGrossIncome * taxPercent *
            MonthsInState / 12;
}

return tax;
}
}

As we can see, the ComputeTax function has four parameters:

  • annual gross income,
  • marital status of the person for which the tax is computed,
  • the state in which the person resides, and
  • the number of months the person has been a resident of this state.

This is a standard Java class and nothing more. To compile the class we would save the file as StateTaxCalculator.java and compile it with the command line:

javac StateTaxCalculator.java

However, this StateTaxCalculator.class is already included in the download in the Samples.TaxCalculator package so that you don't need to reconfigure your Tomcat server (assuming you have already deployed the SOAP samples).

Deployment

We assume you have already downloaded, installed, and configured the Java Runtime Environment 1.3 SDK, Apache SOAP Toolkit 2.2 binaries, the Apache SOAP Quickstart with Xerces 2.0 XML Parser, and Tomcat 4.0. If you haven't, complete setup instructions are included in the download's full_setup_instructions.txt file.

The Apache implementation of SOAP uses a Servlet listener for SOAP requests. In order to make our class available for a SOAP request we must deploy it as a web service using a deployment descriptor. A deployment descriptor is an XML file that contains information about the web service, such as:

  • where the Java class is located,
  • what methods it supports,
  • the encoding used.

Here is the deployment descriptor for the StateTaxCalculator service:

<isd:service StateTaxCalc xmlns:isd="http://xml.apache.org/xml-soap/deployment"
id="urn:StateTaxCalculator">
<isd:provider type="java"
scope="Application"
methods="ComputeTax">
<isd:java class="samples.TaxCalculator.StateTaxCalculator" static="false"/>
</isd:provider>

<isd:faultListener>
org.apache.soap.server.DOMFaultListener
</isd:faultListener>

</isd:service>

Create a file called DeploymentDescriptor.xml that contains the above code. All deployment descriptors for Java web services have the same structure. Only the ID of the service, the Java class where the service resides, and the methods that a service exposes, differ from one service to another.

The deployment descriptor is very important; in fact it is crucial for our web service to work correctly. As mentioned earlier, the Apache SOAP relies on the client to supply complete information about parameters. Notice that in the above deployment descriptor there is no mention about the parameters type, the order of the parameters, or how to serialize / de-serialize each parameter. A Java client will correctly construct a valid SOAP request package, but a Microsoft client (either STK or .NET) fails to do this in the absence of a .WSDL file.

Note: We could use the IBM WSTK at http://www.alphaworks.ibm.com/tech/webservicestoolkit/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq to create .WSDL files for Java classes, but have shown an alternate implementation for completeness of the article.

To overcome this we must add one more section to the deployment descriptor:

<isd:mappings>
<isd:map encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"
xmlns:x="" qname="x:AnnualGrossIncome"
javaType="java.lang.Double"
xml2JavaClassName=
"org.apache.soap.encoding.soapenc.DoubleDeserializer"/>

<isd:map encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"
xmlns:x="" qname="x:MaritalStatus"
javaType="java.lang.String"
xml2JavaClassName=
"org.apache.soap.encoding.soapenc.StringDeserializer"/>

<isd:map encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"
xmlns:x="" qname="x:State"
javaType="java.lang.String"
xml2JavaClassName=
"org.apache.soap.encoding.soapenc.StringDeserializer"/>

<isd:map encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"
xmlns:x="" qname="x:MonthsInState"
javaType="java.lang.Integer"
xml2JavaClassName=
"org.apache.soap.encoding.soapenc.IntDeserializer"/>
</isd:mappings>

Now the deployment descriptor is complete, and we can notify the Tomcat server about the presence of a new web service. Open a command window, change the current folder to the location where you saved StateTaxCalculator.java, and run the deploy.bat file which executes the following command:

java org.apache.soap.server.ServiceManagerClient
http://localhost:8080/soap/servlet/rpcrouter deploy DeploymentDescriptor.xml

To complete the deployment of our StateTaxCalculator, all we have to do is copy the contents of our donwnload's samples\TaxCalculator folder into a corresponding SOAP_HOME\samples\TaxCalculator folder on the server, where SOAP_HOME is the location where the Apache SOAP for Java is installed.

We should now be able to see our deployed service in the Apache SOAP Admin tool. Surf to http://localhost:8080/soap/:

We can view the details of the web service by clicking on its name:

In future, this information will help us to create our own deployment descriptors for any class, by clicking on the Deploy button and following the template shown above.

Surf to the url of the web service and we should see the following:

Tax Calculator

Tax Calculator is the .NET web service that integrates the State and Federal tax calculators to provide a unique interface for tax computation. We have a few choices when writing a web service with the .NET framework:

  • web service using C# or VB.NET
  • Visual C++ web service

The most complex way to write it is using Visual C++. We'll use the simpler and faster way to build a web service, by creating a web service in C#.

Web services written with .NET consist of an .asmx file. We place the .asmx file into a virtual directory where it can be accessed and that's it! The .asmx file is compiled the first time a user accesses it. Thereafter, the file is recompiled only if the server is restarted or the file is changed. This approach offers some flexibility in the development process because we don't have to stop the IIS server for a deployment of the web service.

Let's see the code for the TaxCalculator web service. The TaxCalculator.asmx file contains three classes. The first class is the main class, TaxCalculator, which exposes the ComputeTax function.

The implementation resembles a class written in C#. The only new element is marking the class and the methods for access using the SOAP protocol, using the Webservice and Webmethod attributes.

  • The WebService attribute applied to a class will enable the SOAP code for handling HTTP requests using the SOAP protocol.
  • To mark a method in a class as web enabled, we must specify the WebMethod attribute for that method.

Here is the content of the TaxCalculator.asmx file:

<%@ WebService Language="C#"
            Class="TaxCalculator" %>

using System;
using
            System.Web.Services;
using System.Diagnostics;
using
            System.Xml.Serialization;
using
            System.Web.Services.Protocols;
[WebService(Namespace="http://www.catalin.com/TaxCalculator/")]
public
            class TaxCalculator: WebService {

[
            WebMethod(Description="compute state and fed
            tax")]
[System.Web.Services.Protocols.SoapRpcMethodAttribute(
"http://www.catalin.com/TaxCalculator/ComputeTax",
RequestNamespace="http://www.catalin.com/TaxCalculator/",
ResponseNamespace="http://www.catalin.com/TaxCalculator/")]

public
            double ComputeTax(double AnnualGrossIncome, string MaritalStatus,
string State, int MonthsInState) {

double taxFed = 0,
            taxState = 0;

FedTaxCalculator objFTCalc = new
            FedTaxCalculator();
StateTaxCalculator objSTCalc = new
            StateTaxCalculator();

taxFed =
            objFTCalc.ComputeTax(AnnualGrossIncome, MaritalStatus);
taxState
            = objSTCalc.ComputeTax(AnnualGrossIncome, MaritalStatus,
State,
            MonthsInState);

return (taxState +
            taxFed);
}
}

The TaxCalculator class must access the State and Federal tax calculator services to get this job done. How do we access these services? In order for a .NET client or web service to access another web service, we need to create a proxy class, which has the same interface as the service. Whenever a call is made through this proxy server, the .NET framework forwards the call to the actual service as a SOAP message, and then retrieves the result from the SOAP response message. A proxy class inherits from the SoapHttpClientProtocol class, which will handle all SOAP communication with the server. Again, notice that attributive programming is the key for setting up each method with information about request and response namespaces and SOAP action.

The .NET framework provides a tool called wsdl.exe to generate the proxy for a web service. This executable can generate the proxy class based on the WSDL file corresponding to a web service.

Use the following command line to generate the proxy for the FedTaxCalculator service:

wsdl.exe http://<path to FedTaxCalculator>/FedTaxCalculator.WSDL

The command should generate a file called FedTaxCalculator.cs containing the following content:

[System.Web.Services.WebServiceBindingAttribute(Name="FedTaxCalculatorSoapBinding",
Namespace="http://tempuri.org/wsdl/")]
public class FedTaxCalculator : System.Web.Services.Protocols.SoapHttpClientProtocol {

[System.Diagnostics.DebuggerStepThroughAttribute()]
public FedTaxCalculator() {
this.Url = "http://localhost/TaxCalculator/FedTaxCalculator.WSDL";
}

[System.Diagnostics.DebuggerStepThroughAttribute()]
[System.Web.Services.Protocols.SoapRpcMethodAttribute
("http://tempuri.org/action/FedTaxCalculator.ComputeTax",
RequestNamespace="http://tempuri.org/message/",
ResponseNamespace="http://tempuri.org/message/")]
[return: System.Xml.Serialization.SoapElementAttribute("Result")]
public System.Double ComputeTax(ref System.Double AnnualGrossIncome,
ref string MaritalStatus) {
object[] results = this.Invoke("ComputeTax", new object[] {
AnnualGrossIncome,
MaritalStatus});
AnnualGrossIncome = ((System.Double)(results[1]));
MaritalStatus = ((string)(results[2]));
return ((System.Double)(results[0]));
}

[System.Diagnostics.DebuggerStepThroughAttribute()]
public System.IAsyncResult BeginComputeTax(
System.Double AnnualGrossIncome, string MaritalStatus,
System.AsyncCallback callback, object asyncState) {
return this.BeginInvoke("ComputeTax", new object[] {
AnnualGrossIncome,
MaritalStatus}, callback, asyncState);
}

[System.Diagnostics.DebuggerStepThroughAttribute()]
public System.Double EndComputeTax(System.IAsyncResult asyncResult,
out System.Double AnnualGrossIncome, out string MaritalStatus) {
object[] results = this.EndInvoke(asyncResult);
AnnualGrossIncome = ((System.Double)(results[1]));
MaritalStatus = ((string)(results[2]));
return ((System.Double)(results[0]));
}
}

Paste the content of the FedTaxCalculator.cs into the end of the TaxCalculator.asmx file.

At this point, the TaxCalculator is able to call the Federal Tax Calculator. Let's move next to the State Tax Calculator. The same idea must be applied here too. The only problem is that for a Java web service there is no .wsdl file from which we can automatically generate a proxy. In this situation the solution is to write the proxy class by hand. Since we already have the proxy class for federal tax, we'll modify it to meet our needs.

The major changes that must be made are:

  • The parameter list must be modified accordingly for the StateTaxCalculator service
  • The namespaces used in the proxy must be identical to the ones used by the web service

Here is the proxy class for the StateTaxCalculator service:

[System.Web.Services.WebServiceBindingAttribute(Name="StateTaxCalculatorSoapBinding",
Namespace="http://tempuri.org/wsdl/")]
class StateTaxCalculator : System.Web.Services.Protocols.SoapHttpClientProtocol {

[System.Diagnostics.DebuggerStepThroughAttribute()]
public StateTaxCalculator() {
this.Url = "http://localhost:8080/soap/servlet/rpcrouter";
}

[System.Diagnostics.DebuggerStepThroughAttribute()]
[System.Web.Services.Protocols.SoapRpcMethodAttribute(
"http://localhost:8080/soap/servlet/rpcrouter/ComputeTax",
RequestNamespace="urn:StateTaxCalculator",
ResponseNamespace="urn:StateTaxCalculator")]
[return: System.Xml.Serialization.SoapElementAttribute("Result")]
public System.Double ComputeTax(double AnnualGrossIncome, string MaritalStatus,
string State, int MonthsInState) {

object[] results = this.Invoke("ComputeTax", new object[] {
AnnualGrossIncome,
MaritalStatus,
State,
MonthsInState});

return ((System.Double)(results[0]));
}

[System.Diagnostics.DebuggerStepThroughAttribute()]
public System.IAsyncResult BeginComputeTax(double AnnualGrossIncome,
string MaritalStatus, string State, int MonthsInState,
System.AsyncCallback callback, object asyncState) {

return this.BeginInvoke("ComputeTax", new object[] {
AnnualGrossIncome,
MaritalStatus,
State,
MonthsInState}, callback, asyncState);
}

[System.Diagnostics.DebuggerStepThroughAttribute()]
public System.Double EndComputeTax(System.IAsyncResult asyncResult,
double AnnualGrossIncome, string MaritalStatus,
string State, int MonthsInState) {

object[] results = this.EndInvoke(asyncResult);
return ((System.Double)(results[0]));
}
}

The changes that have been made from the first proxy class are:

  • Because Apache SOAP for Java uses the id of the service as a namespace for SOAP messages, the namespace was replaced with urn:StateTaxCalculator.
  • The list of parameters in the ComputeTax method was changed to: double AnnualGrossIncome, string MaritalStatus, string State, int MonthsInState.

Append this StateTaxCalculator.cs code to the end of TaxCalculator.asmx file and save it. We no longer need the proxy .cs files and now have a web service that is able to call some other services, gather results from these services, and produce a response.

Deployment

Create a virtual directory in the IIS server named TaxCalc and place the TaxCalculator.asmx file into it. To test the service, open a new instance of IE and type in the address http://localhost/TaxCalc/TaxCalculator.asmx?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq. If no errors are encountered we should be able to see a page containing information about the service:

To see the WSDL file open the IE and type in the address http://localhost/TaxCalculator/TaxCalculator.asmx?WSDL&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq.

Accessing the TaxCalculator service

So far we have developed a web service (TaxCalculator) that integrates the functionality of the other two services (StateTaxCalculator and FedTaxCalculator) in order to compute the total amount of income tax payable for a given person. The TaxCalculator service exposes its functionality for client applications.

In this final section we'll see how to access the TaxCalculator service from client applications written in different languages.

A .NET Console Client

Writing a C# client for TaxCalculator is very simple once we generate the proxy class. To generate the proxy class for the C# web service open a command window and execute:

wsdl.exe http://localhost/TaxCalculator/TaxCalculator.asmx?WSDL

The tool will generate a file called TaxCalculator.cs which we can compile and easily use in another C# program, like this:

using System;

/**
* This is a C# client for computing taxes.
*/
public class TaxCalculatorClient {
public static void Main() {

String State = "XX", MaritalStatus = "M";
double AnnualGrossIncome = 76000,
TotalTax = 0;
int MonthsInState = 9;

/**
* Create proxy object to access the tax service.
*/
TaxCalculator objTaxCalc = new TaxCalculator();

/**
* Call the ComputeTax method on each of the services.
*/
TotalTax = objTaxCalc.ComputeTax(AnnualGrossIncome, MaritalStatus,
State, MonthsInState);

/**
* Display the result.
*/
Console.WriteLine("John Doe");
Console.WriteLine("Annual Gross Income: ${0}", AnnualGrossIncome);
Console.WriteLine("Resident of : {0}", State);
Console.WriteLine("Marital status : {0}", MaritalStatus);
Console.WriteLine("No of months in {0} : {1}", State, MonthsInState);
Console.WriteLine("Total tax : ${0}", TotalTax);
}
}

As you can see, the code is very simple and the proxy object is used as a standard C# object. We simply call the method and the framework handles everything, from creating the SOAP envelope message, to retrieving the result from the SOAP response.

Save the C# client into a file called TaxCalculatorClient.cs, then compile the TaxCalculator.cs as a library with:

csc.exe /t:library TaxCalculator.cs

Then compile the client using:

csc.exe /reference:TaxCalculator.dll TaxCalculatorClient.cs

Open a command window, change the current folder to the location where you saved and compiled the C# proxy and client classes, and execute TaxCalculatorClient.exe.

The result should be:

A VB 6.0 MS SOAP Toolkit client

Writing a client for TaxCalculator with the MS SOAP Toolkit is simple when we have the .WSDL file. The client is written in Visual Basic and uses the high level API available with the MS SOAP Toolkit.

The CallTaxCalculator function uses the high level API to create the SOAP request and to parse the server response.

Private m_soapClient As SoapClient

Private Const END_POINT_URL = "http://localhost/TaxCalculator/TaxCalculator.asmx"

Public Sub CallTaxCalculator()

Dim objServer As SoapClient, result As Double
Dim AnnualGrossIncome As Double, MaritalStatus As String, State As String, MonthsInState As Integer

AnnualGrossIncome = 76000
MaritalStatus = "M"
State = "XX"
MonthsInState = 9

On Error GoTo catch_all_err
Set objServer = New SoapClient
Call objServer.mssoapinit(END_POINT_URL + "?WSDL")

result = objServer.ComputeTax(AnnualGrossIncome, MaritalStatus, State, MonthsInState)

' display the results
Debug.Print "Name: John Doe"
Debug.Print "Annual Gross Income: " + CStr(AnnualGrossIncome)
Debug.Print "Marital status: " + MaritalStatus
Debug.Print "State: " + State
Debug.Print "Months in state: " + CStr(MonthsInState)
Debug.Print "Total Tax: " + CStr(result)

cleanup:
On Error Resume Next
Set objServer = Nothing
Exit Sub

catch_all_err:
Debug.Print Err.Description
End Sub

Private Sub main()
Call CallTaxCalculator()
End Sub

Create a new Visual Basic 6.0 project, add a module, and paste the above code. In the VB project add references to MSXML 3.0 and MS SOAP Type Library, set the start point of the project main function, compile and execute. The result should be the same as with the C# client.

An ASP.NET Web Client

Finally, we will see how to create an ASP.NET client using C#. This is even easier than with the above two methods. Once we have opened Visual Studio .NET and created a new C# ASP.NET Web Application we only have to follow 3 steps:

Create the User Interface by dragging five TextBox controls and one Button control from the Toolbox in Visual Studio .NET:

Add a web reference to the web service at http://localhost3/TaxCalc/TaxCalculator.asmx?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq:

Add the click handler code in Webform1.aspx (remember that TextBoxes can only handle string types, so we need to convert AnnualGrossIncome and TotalTax).

private void CalculateTax_Click(object sender, System.EventArgs e)
{
string StringAnnualGrossIncome = TextBox1.Text;
double DoubleAnnualGrossIncome = Convert.ToDouble(StringAnnualGrossIncome);
string StringMonthsInState = TextBox4.Text;
int IntMonthsInState = Convert.ToInt32(StringMonthsInState);

TaxCalculatorClient.localhost3.TaxCalculator ws =
new TaxCalculatorClient.localhost3.TaxCalculator();
double DoubleTotalTax = ws.ComputeTax(DoubleAnnualGrossIncome,
TextBox2.Text, TextBox3.Text, IntMonthsInState);

string StringTotalTax = Convert.ToString(DoubleTotalTax);
TextBox5.Text = StringTotalTax;

}

Run the project by holding down Ctrl + F5 and we should see the following result:

Conclusion

Writing web services and client applications using different languages and toolkits is a necessity today. There will always be issues and problems because of different approaches used by software providers and open source projects. Hopefully this article will help you solve some of the problems when writing C# web services and client applications. You may have further questions - leave your comments in the discussion forum and we will cover them in a future article.

If you'd like to discuss a broader range of C#Today topics and issues, or if you have any C# related questions you need answers to, please visit our online resource centres at : http://p2p.wrox.com/list.asp?list=csharptoday_discuss&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq and http://p2p.wrox.com/list.asp?list=c_sharp&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq


Debugging Remoting Transactions

Debugging Remoting Transactions
http://www.csharptoday.com/content.asp?id=1551
Article Rating: 4
Published on August 20, 2001
Abstract
In this article, Matthew Reynolds builds a small utility that sits between a client and server, allowing you to "peek" into the requests and responses that are being sent between them. It is built so that it works with the SOAP formatter and HTTP channel. This utility has many uses, such as debugging web services, testing security procedures, and monitoring packets, as well as many other things.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

One of my favorite features in .NET is the Remoting functionality. Never before has building a distributed application been this easy or this flexible. Creating objects that live on the server and get called by clients from anywhere on the network or Internet is just a matter of writing a few lines of code. Of course, there are certain implications for security and performance, but it's far easier to work with distributed objects in Remoting than it used to be in Distributed COM.

There is, however, a need to be able to examine the messages that move between client and server in Remoting. For example, you may need to:

  • Ensure that the packets you send are not verbose, i.e. they are well optimized.
  • Ensure that any security and authentication functionality you implement is working as intended, and that the potential for your security scheme to be comprised is mitigated.
  • Learn more about how SOAP envelopes are put together.
  • Need to solve a tricky problem!

In this article, we're going to build a utility that sits between the client and server and allows you to peek into the requests and responses sent between the client and the server.

We're going to build our application so that it works with the SOAP formatter and HTTP channel, and not the TCP channel and binary formatters that are commonly used with Remoting. That's because working with SOAP transferred over HTTP is far easier! Plus, this same utility can be used to debug Web services.

The Test Platform

For this exercise, we're going to assume that we already have client and server applications that communicate with Remoting. Simply put, we have an object on the server supporting a method called CreateCustomer which returns a Customer object. This object has three members: Id , FirstName and LastName . Our client application uses Remoting to call the remote CreateCustomer method and displays the details of the Customer object in a window.

In the package you'll find ServerHost.exe , located inside the ServerHost\bin directory. If you run this, you should see the screen shown below:

Figure 1: running serverhost.exe .

The ServerHost console application contains a reference to another assembly in the download package called Server . This contains a single object called Customers , which in turn supports a single method called GetCustomer . We can access this object through the URL http://localhost:10001/Server.Customers.remote .

Now run client.exe, found in the Client\bin folder that's also in the download package. Change the URL to use port 10001 rather than 10002 , like this:

http://localhost:10001/Server.Customers.remote

Click the Create Customer button and you'll see this on the client.

Figure 2: creating a customer, the client view.

If you flip over to the server, you'll also see this:

Figure 3: creating a customer, the server view.

To create our utility, we're going to listen for connections on a port other than the one that's configured on the server. As the server is configured to listen for connections to port 10001 , we'll set out utility up to listen on 10002 . We'll refer to port 10002 as "client side", because it's on the side of the client. Likewise, port 10001 will be known as "server side" because it's on the side of the server.

When the client connects to the client side port, we'll create a new connection to the server side port. Data coming through the client side will be automatically forwarded to the server side. This is a "tunnel". Likewise, any response from the server side will be passed back to the client side. By writing some smart code in the middle, we can examine the data that gets passed around and do something clever with it.

Creating our Utility

Create a new Visual C# - Windows Application project and call it WroxTrace .

The first thing we need is a Configuration class. This object will have a few public members to define the port that we should listen on and what we should connect to. Using Solution Explorer, create a new class called Configuration and add these members:

   /// <summary>
///
            Summary description for Configuration.
///
            </summary>
public class Configuration
{
//
            members...
public int ClientPort = 10002;
public String
            ServerName = "localhost";
public int ServerPort =
            10001;

public Configuration()
{
//
// TODO: Add
            constructor logic here
//
}
}

To use the configuration, we'll need a member on Form1 . Open the code editor for Form1 and add this public member:

public class Form1 : System.Windows.Forms.Form
{
// members...
public Configuration Configuration = new Configuration();

The next thing we need is an object that will sit around through the life of the application listening for incoming connections. When connections are received from the client side, it will deal with the problem of connecting back to the server side and forwarding the requests and responses. This class, Listener , will spin up a new thread and then open a socket and wait for incoming connections.

Create a new class called Listener and add namespace import declarations to System.Threading , System.Net and System.Net.Sockets .

using System;
using System.Net;
using System.Net.Sockets;
using System.Threading;

Then add these three members to hold a reference back to the owner form, one for a thread and another to keep hold of a TcpListener object.

   public class
            Listener
{
// members...
public Form1 Owner;
private
            Thread _thread;
private TcpListener _listener;

To spin up the thread, all we need is a method that will form the entry point. We'll call ours ThreadEntryPoint , and we'll build this in a moment. For now, add this method:

      //
            SpinUp - start the thread...
public void SpinUp()
{
// do
            we have a thread?
if(_thread == null)
{
// create and start
            the thread...
_thread = new Thread(new
            ThreadStart(ThreadEntryPoint));
_thread.Name =
            "Listener";
_thread.Start();
}
}

Where there's a SpinUp , we need a SpinDown , so add this code:

      //
            SpinDown - stop the thread...
public void SpinDown()
{
//
            do we have a thread?
if(_thread != null)
{
// stop
            it...
if(_listener !=
            null)
_listener.Stop();
_thread.Join();
_thread =
            null;
}
}

ThreadEntryPoint is where things start to get interesting. At this point, we have to create a new socket on port 10002 and listen for incoming connections.

Luckily for us, most of the issues of using sockets for communication have been highly abstracted by the Framework. This means that we can use sockets with the same level of ease that we can access files on disk! General network communication functionality can be found in the System.Net namespace, whereas functionality for dealing with sockets can be found in System.Net.Sockets .

System.Net.Sockets.TcpListener is a class that we can use to wait for incoming connections. All it needs it a port, and we get the client-side port number from the Configuration object.

// ThreadEntryPoint - where the thread starts...
private void ThreadEntryPoint()
{
// listen for connections...
TcpListener listener = new
TcpListener(Owner.Configuration.ClientPort);
listener.Start();

Once that's done, we can call AcceptTcpClient . This will block the thread until such time as a connection actually comes into the client side port. We've put this inside a try.catch because when we want to stop the thread we'll close the _listener object, which will throw an exception. When a connection comes in though, it will return a System.Net.Sockets.TcpClient object. This lets us draw data from the client and send data back to it.

         //
            loop forever...
while(true)
{
// get a
            connection...
TcpClient tcpClient;
try
{
tcpClient =
            _listener.AcceptTcpClient();
}
catch
{
break;
}

We need another class now that represents a connection between client and server. We need to create one of these whenever a connection comes in, and we'll call the class Connection . Add this code:

            //
            create a new connection object
Connection connection = new
            Connection(tcpClient,
            this);
connection.StartProcessing();
}
}

After StartProcessing has been called, the loop will flip back to the top until a time when a new connection comes in and another new Connection object will be created.

Connection, Transaction and TransactionData

That's pretty much everything we need to do with Listener , so we can turn our attention to the other classes that are going to do the cool stuff that this application is all about.

As we know, Connection represents a connection between client and server. Over the connection we'll receive "transactions", and we define a transaction as a pair that comprises both a request from the client and a request from the server. In our case, a transaction will occur whenever the Create Customer button is pressed on our demonstration client application, but in your case a transaction will occur each and every time you call a method on an object that's being used with Remoting. We'll call this class Transaction .

Ultimately, what we want our utility to do is display a list of transactions, and when we select one from the list we'll be able to see the SOAP request that was sent to the server, and the response that was sent back. We'll create a new object called TransactionData that represents either a request or a response.

To make life simpler for the user, we'll need to give each connection and transaction a unique ID. We'll get these IDs from a separate class called IdFactory and basically start at 1 and keep going up - i.e. we'll call the first connection 1 , the second 2 and so on.

Using Solution Explorer, create four new classes: Connection, Transaction , TransactionData and IdFactory .

We'll work with IdFactory first. This needs to be thread-safe as each connection is going to be running in its own thread. In fact, each connection will need two threads: one for data moving between client and server and one for data moving between server and client.

Open the code editor for IdFactory and add a namespace import declaration to System.Threading .

using System.Threading;

Add these two members to the class. The first holds the next ID to be served, and the second creates a way to lock the first member for writing.

public class IdFactory
{
// members...
private static int _nextId = 1;
private static ReaderWriterLock _lock = new ReaderWriterLock();

Then, add this method that will issue a new ID:

      //
            GetNextId - get the next ID in the list...
public static int
            GetNextId()
{
//
            lock...
_lock.AcquireWriterLock(-1);

// get the id and
            increment...
int nextId = _nextId;
_nextId++;

//
            release the lock...
_lock.ReleaseWriterLock();

// return
            the id...
return nextId;
}

Now we can work on Connection . Open the code editor for this class and add these namespace import declarations:

using System;
using System.Net;
using System.Net.Sockets;
using System.Threading;

This class needs quite a few members. We'll add these now. Notice how we use the static GetNextId method to provide a new ID to the connection.

public class Connection
{
// members...
public int Id = IdFactory.GetNextId();

public Listener Listener;
public TcpClient Source;
public TcpClient Destination;
private NetworkStream _sourceStream;
private NetworkStream _destinationStream;
private Transaction _transaction;
private ReaderWriterLock _lockTransaction = new ReaderWriterLock();
private ManualResetEvent _waitTransaction =
new ManualResetEvent(false);

It's going to be useful for us to create a way of identifying whether a certain object is on the client side or the server side of a transaction. Add this enumeration to Connection :

// enum...
public enum Directions { FromClient = 0, FromServer }

Next, delete the existing constructor and replace it with this one:

      public
            Connection(TcpClient source, Listener listener)
{
this.Source
            = source;
this.Listener = listener;
}

Before we go too far, overload ToString so that we can easily get a meaningful name for the object.

// ToString...
public override String ToString()
{
return "Client #" + Id.ToString();
}

As we mentioned before, when we start processing we want to create two threads - one to pass data from the client to the server, and another to pass the data from the server to the client. For this, we're going to use the built-in thread pool functionality of Windows and .NET. Using the thread pool is a really easy way of working with threads. Rather than managing the lifetime of our threads ourselves, we can get Windows to do it for us. We do this through the concept of "queuing" a work item - i.e. we have something that needs to be done so we pass it over to the thread pool and say "create a thread and do this work item when you have a moment".

However, before we can queue the work items we have to open a connection to the server. This is just a matter of creating a new TcpClient object and giving it the correct server name and port number.

      //
            StartProcessing - connect and start transferring data...
public
            void StartProcessing()
{
// connect to the
            server...
Destination = new
TcpClient(Listener.Owner.Configuration.ServerName,
Listener.Owner.Configuration.ServerPort);

To talk to the TcpClients we need NetworkStream objects.

// create streams...
_sourceStream = Source.GetStream();
_destinationStream = Destination.GetStream();

Once we've done that, we can queue our work items. We'll need to create a separate method for both ProcessFromClient and ProcessToServer .

// queue two work items...
ThreadPool.QueueUserWorkItem(new
WaitCallback(ProcessFromClient));
ThreadPool.QueueUserWorkItem(new
WaitCallback(ProcessToServer));
}

At some point in the future, the thread pool will spin up a new thread or use an existing thread and call ProcessFromClient and ProcessToServer . In each case, we have to do the same job, i.e. listen for data on one TcpClient and then immediately pass it down to its opposite number. At the same time, we'll need to pass the data that we get over to the transaction object for processing.

We'll pause our work on Connection for a moment and do some more work with Transaction . Specifically, we need properties for data sent from the client or from the server.

Open the code editor for Transaction and add the members:

public class Transaction
{
// members...
public int Id = IdFactory.GetNextId();
public Connection Connection;
public Listener Listener;
public TransactionData DataFromClient;
public TransactionData DataFromServer;

Then, add this code to the constructor:

      public
            Transaction()
{
DataFromClient = new
            TransactionData();
DataFromClient.Direction =
WroxTrace.Connection.Directions.FromClient;
DataFromServer =
            new TransactionData();
DataFromServer.Direction =
WroxTrace.Connection.Directions.FromServer;
}

Like before, add this new version of ToString :

// ToString...
public override String ToString()
{
return "Transaction #" + Id.ToString();
}

Finally, add this method:

// GetData - get a version of the data...
public TransactionData GetData(Connection.Directions direction)
{
switch(direction)
{
case WroxTrace.Connection.Directions.FromClient:
return DataFromClient;
case WroxTrace.Connection.Directions.FromServer:
return DataFromServer;
default:
return null;
}
}

The next thing we need to do is make sure that TransactionData is capable of storing data that we give it. This is a slightly tricky proposition because we're going to be given data in the form of an array of System.Byte structures, but it's going to be far easier for us to work with System.String structures. We'll need to convert one from the other.

Open the code editor for TransactionData and add this namespace definition.

using System;
using System.Text;

Next, add a member that can hold a System.Text.StringBuilder object. If you've never used them before, these kinds of objects are really useful for efficiently managing a string that grows in size. We'll also need a member called Direction that will record which direction the object is holding data for.

public class TransactionData
{
// members...
public Connection.Directions Direction;
private StringBuilder _rawData = new StringBuilder();

System.Text.ASCIIEncoding can be used to convert an array of Byte structures to the equivalent array of Char structures. Add this code:

// AddData - add data to the raw data...
public void AddData(Byte[] data, int length)
{
// create an encoding object...
ASCIIEncoding encoding = new ASCIIEncoding();

// convert from bytes to chars...
Char[] dataAsChars = encoding.GetChars(data, length);

However, we can't directly add an array of Char structures to our _rawData StringBuilder object, so we'll need to convert again and pass the data to another version of the method.

// finally, convert again and pass to
            another method...
StringBuilder builder = new
            StringBuilder();
builder.Append(dataAsChars);
AddData(builder.ToString());
}

The other version of the method is very straightforward:

public void AddData(String data)
{
_rawData.Append(data);
}

Now that we finally have somewhere to store the data that we collect, we can turn our attention to collecting the data. This is a fairly basic process - read bytes from one stream and write the same bytes down to the other stream.

Open the code editor for Connection . Find the ProcessFromClient method and add this code:

// ProcessFromClient - move data from client to server...
public void ProcessFromClient(object state)
{
try
{
Process(_sourceStream, _destinationStream,
Directions.FromClient);
}
catch
{
}
}

What this is going to do is call a protected method called Process and provides a source stream and a destination stream. The way this will work is that _sourceStream will be the one coming from the client and _destinationStream will be the one going to the server. In our case, each of these will be an entire HTTP message containing a SOAP envelope.

Before we can read data, we need a block of memory that can be used to store whatever data we get. We'll create a buffer that's 2K in size. The size of buffer that I've chosen here is fairly arbitrary. If you experiment with the buffer size, you may well get better performance. The way this code is constructed, if you change the bufferSize variable here to something other than 2048 , the code will continue to work as intended.

// Process - read data and pass it down...
public void Process(NetworkStream source, NetworkStream destination,
Directions direction)
{
// create a buffer...
int bufferSize = 2048;
Byte[] buffer = new Byte[bufferSize];

At this point we have to drop into an endless loop reading data from the source stream until such time as no more data is available. If no data is available, this call will block and the thread will freeze. If the connection is ended or broken, an exception will be thrown and you can see that in ProcessFromClient we have a try.catch ready and waiting for this eventuality.

         //
            loop...
while(true)
{
// read data...
int readBytes =
            source.Read(buffer, 0, bufferSize);
if(readBytes >
            0)
{

If we get some bytes, we need to write that straight away to the destination stream. But we also have to make sure that we have a transaction. This is a challenge of our thread-synchronization skills. Basically, we have two threads that could both try and access the same _transaction member at the same time. Both ProcessFromClient and ProcessFromServer will use this same Process method for data exchange, so Process will be called from two threads at once.

Our first step is to get the current value of _transaction and store it in a local variable. We're going to abstract this value into a property later.

// get the transaction...
Transaction useTransaction = Transaction;

Now the problems come if we don't have a transaction. In fact, we need a rule to define who gets to create the Transaction object. We're going to say that the client does, as it's the client that starts the whole deal anyway. So, if useTransaction is null (and therefore _transaction is null) and we are coming from the client, we create a new Transaction object.

               //
            do we have a transaction...
if(useTransaction == null)
{
//
            if we're the client, create one...
if(direction ==
            Directions.FromClient)
{
// create one...
useTransaction =
            new Transaction();
useTransaction.Connection =
            this;
useTransaction.Listener = Listener;

Once we have the object, we use the Transaction property again to set the value:

// set it...
Transaction = useTransaction;
}

OK, so if we're not FromClient , we must be FromServer , and if we are then we need to wait for the client to create the new object and for the client to tell us about it. We created a ManualResetEvent member called _waitTransaction that does just this. When we create our Transaction property, when we set _transaction to a value (which can only happen when we're FromClient , remember), this object will become signaled and we can retrieve the value.

else
{
// if we're the server,
// we have to block until we have
// one...
_waitTransaction.WaitOne();

// now we should be able get it...
useTransaction = Transaction;
}

The upshot of this is that we give permission to only one side of the problem (the client) to create a Transaction object. The other side (the server) has to wait until this happens.

After we've done all that we can send the data to the destination and give the same data to the relevant TransactionData object.

// write data...
destination.Write(buffer, 0, readBytes);

// get the data...
TransactionData data = useTransaction.GetData(direction);
data.AddData(buffer, readBytes);

Once that's been completed, we can close off the loop.

}
}
}

Once we've sent the entire request to the server, the server will formulate and send its response back to us. We're going to deal with this using the ProcessFromServer method. This will call Process , but will reverse the two streams.

// ProcessFromServer - move data from server to client...
public void ProcessFromServer(object state)
{
try
{
Process(_destinationStream, _sourceStream,
Directions.FromServer);
}
catch
{
}
}

As promised, here is our Transaction property.

      protected
            Transaction Transaction
{
get
{
// get
            it...
_lockTransaction.AcquireReaderLock(-1);
Transaction
            transaction =
            _transaction;
_lockTransaction.ReleaseReaderLock();

//
            return it...
return transaction;
}
set
{
// set
            it...
_lockTransaction.AcquireWriterLock(-1);
_transaction =
            value;

// did we blank it?
if(_transaction !=
            null)
_waitTransaction.Set();
else
_waitTransaction.Reset();

//
            unlock...
_lockTransaction.ReleaseWriterLock();
}
}

The trick here is in the set portion. Basically, whenever we set the property we look to see if it is null. If it is, we reset _waitTransaction , which means that if the server side needs the transaction it will end up blocking until the client side creates it. If it isn't null, we set it, which will release any block the server side may have.

Just so we know we're not painting ourselves into a corner, we're going to run the application in a moment and make sure that it all works - although we won't see anything when the application runs as we haven't got to that part yet. The only thing that remains is to spin up the Listener thread.

Open the code editor for Form1 and add this member:

public class Form1 : System.Windows.Forms.Form
{
// members...
public Configuration Configuration = new Configuration();
public Listener Listener;

Then, flip over to the Designer for Form1 and double-click on the form. This will create a new Load event handler. Add this code:

      private
            void Form1_Load(object sender, System.EventArgs e)
{
// spin
            up...
Listener = new Listener();
Listener.Owner =
            this;
Listener.SpinUp();
}

Run the project and find the Client application that you started earlier. Make sure the URL is set to use port 10002 , like this:

http://localhost:10002/Server.Customers.remote

Click the Create Customer button and the application should work as normal. However, in reality the SOAP requests are being passed through our application and the details stored in Transaction objects.

Figure 4: running the application.

Building the View

OK, so we've built something that works but is generally useless! To make this cool, we need to create a UI.

Close the program if it's still running, and then open the Designer for Form1 . Because the program has active threads, you may find that Visual Studio.NET claims that the program is still running. Select Debug - Stop Debugging from the menu .

Using the Designer, set the Text property of the form to WroxTrace . Using the toolbox, create a new TreeView control and position it in the left third of the form. Set the Name of the new control to treeTransactations and set the Anchor property to Top, Bottom, Left .

Figure 5: the form.

Using Solution Explorer, right click on the WroxTrace project and select Add - Add User Control . Call the new control TransactionDataView .

On the Designer for the new control, draw a TextBox control. Set the Name property of the new control to textRawData and clear the Text property. Set the Multiline property to True and the ScrollBars property to Vertical . Finally, set the Dock property to Fill .

View the code editor for TransactionDataView and add a new member:

public class TransactionDataView : System.Windows.Forms.UserControl
{
// members...
private TransactionData _data;

Next, add this method:

      protected
            void UpdateView()
{
// reset the view...
textRawData.Text =
            "";

// do we have data?
if(Data !=
            null)
textRawData.Text = Data.RawData;
}

Then, add this property:

      public
            TransactionData Data
{
get
{
return
            _data;
}
set
{
_data =
            value;
UpdateView();
}
}

We need to build the project to use the new control, but at the moment we've referenced a property called RawData on TransactionData that doesn't exist. Open the code editor for RawData and add this property:

      public
            String RawData
{
get
{
// return it...
return
            _rawData.ToString();
}
}

Build the project now. This will enable us to use the new control. Open the Designer for Form1 and use the Toolbox to draw on two new TransactionDataView controls. Call the first one viewFromClient and the second viewFromServer .

Figure 6: two views on one form.

Select viewFromClient and set its Anchor property to Top, Left, Right . Set the Anchor property of viewFromServer to Top, Left, Right, Bottom . You should now be able to stretch the form.

Before we go on, add a Timer control to the form. Set its Name property to timerUpdate . Set its Interval property to 3000 (i.e. "3 seconds") and its Enabled property to True.

Our job now is to populate the tree control with a list of the clients and a list of the transactions that we know about. To do this, open the code editor for Form1 and add this method definition:

      //
            RegisterTransaction - tell us about a transaction...
public void
            RegisterTransaction(Transaction transaction)
{
}

We'll call this method directly from Connection whenever we create a new Transaction object. Therefore, different threads can call this object and this causes us a problem. The TreeView control can only be manipulated from the primary application thread. What we'll do is keep a list of the current Transaction objects as a member of Form1 and use a timer to fire an update on the primary thread at a later time. Add these members to Form1 :

public class Form1 : System.Windows.Forms.Form
{
// members...
public Configuration Configuration = new Configuration();
public Listener Listener;
private System.Threading.ReaderWriterLock _lock =
new system.Threading.ReaderWriterLock();
private ArrayList _transactions = new ArrayList();

As we mentioned before, whenever we create a new transaction, we want to register it with Form1 so that it can be displayed in the tree. Open the code editor for Connection and find the Process method. In the code where the new Transaction object is created, add a call to RegisterTransaction .

               //
            do we have a transaction...
if(_transaction == null)
{
//
            create one...
_transaction = new
            Transaction();
_transaction.Connection =
            this;
_transaction.Listener =
            Listener;
Listener.Owner.RegisterTransaction(_transaction);
}

Once again open the code editor for Form1 and add this method. This will add the transaction to a list, which we will periodically walk through and update the display.

      //
            RegisterTransaction - tell us about a transaction...
public void
            RegisterTransaction(Transaction transaction)
{
// lock and
            add...
_lock.AcquireWriterLock(-1);
_transactions.Add(transaction);
_lock.ReleaseWriterLock();
}

Tree Classes

As is often the case with working with some of the controls supplied in the Framework that have a more "fiddly" user interface, we need to build some classes that will help us show the connections and transactions on the tree. To clarify this, something like a text box or a check box have a very basic UI, but something like a ListView, or in this case a TreeView, require a little more work and often involve building new classes.

Using Solution Explorer, create a new class called ConnectionTreeNode . Set the class to inherit from TreeNode , like this:

public class ConnectionTreeNode : System.Windows.Forms.TreeNode

Next, add a member that will hold a reference to a Connection object:

public class ConnectionTreeNode : System.Windows.Forms. TreeNode
{
// members...
public Connection Connection;

Finally, remove the existing constructor and replace it with this one:

      public
            ConnectionTreeNode(Connection connection)
{
// set the
            connection...
this.Connection = connection;

// set the
            text...
this.Text = Connection.ToString();

//
            cross-reference...
connection.Node = this;
}

As you can tell, we're going to make this class and the existing Connection class refer to each other. This will make it easier to show them on the tree. Open the code editor for Connection and add the Node member.

public class Connection
{
// enum...
public enum Directions { FromClient = 0, FromServer }

// members...
public int Id = IdFactory.GetNextId();

public Listener Listener;
public TcpClient Source;
public TcpClient Destination;
private NetworkStream _sourceStream;
private NetworkStream _destinationStream;
private Transaction _transaction;
private ReaderWriterLock _lockTransaction = new ReaderWriterLock();
private ManualResetEvent _waitTransaction =
new ManualResetEvent(false);
public ConnectionTreeNode Node;

Repetitious as it might be, we need to do the same thing for TransactionTreeNode . Create this new class and set its inheritance, add a member and change the constructor. The code for this is:

public class TransactionTreeNode : System.Windows.Forms.TreeNode
{
// members...
public Transaction Transaction;

public TransactionTreeNode(Transaction transaction)
{
this.Transaction = transaction;
this.Text = Transaction.ToString();
transaction.Node = this;
}
}

As before, Transaction will also need a new Node field.

public class Transaction
{
// members...
public int Id = IdFactory.GetNextId();
public Connection Connection;
public Listener Listener;
public TransactionData DataFromClient;
public TransactionData DataFromServer;
public TransactionTreeNode Node;

Open the Designer for Form1 and double-click on timerUpdate . This will create a new Tick event handler. The first thing to do is acquire a reader lock.

      private
            void timerUpdate_Tick(object sender, System.EventArgs e)
{
//
            lock...
_lock.AcquireReaderLock(-1);

Once we've locked, we can iterate the transactions looking for ones that have their Node property set to null. When this happens, we know it's a transaction that's yet to be added to the tree.

// go through each transaction...
foreach(Transaction transaction in _transactions)
{
// do we have a transaction node?
if(transaction.Node == null)
{

If the connection's Node property is also null, we can assume that the connection doesn't appear on the tree either. If it doesn't, we create a new node and add it to the top level node list.

// do we a connection node?
if(transaction.Connection.Node == null)
{
// create this...
ConnectionTreeNode connectionNode =
new ConnectionTreeNode(transaction.Connection);
treeTransactions.Nodes.Add(connectionNode);
}

Then, we can create the new transaction node:

// add the node...
TransactionTreeNode transactionNode =
new TransactionTreeNode(transaction);
transaction.Connection.Node.Nodes.Add(transactionNode);

// expand the connection node...
transaction.Connection.Node.Expand();
}
}

Finally, after we've been through all of the transactions we can release the lock.

// release the lock...
lock.ReleaseReaderLock();
}

Run the project now and start the click. If you click Create Customer , wait a few seconds and the transaction should appear. There will be a delay when you first click the button while Remoting establishes the client/server connection and up to three seconds between the time WroxTrace processes the request pair and displays it on the tree.

Figure 7: the transaction appears.

If you run up another instance of the client and click Create Customer , you'll see another connection.

Figure 8: two transactions.

Showing Transaction Details

That's cool, but the point of the application is to show the details of the transaction! Let's respond to the AfterSelect property of the tree view and update the two TransactionDataView controls. Open the Designer for Form1 and double-click on the TreeView control. Add this code to the new event handler:

private void treeTransactions_AfterSelect(object sender,
System.Windows.Forms.TreeViewEventArgs e)
{
// what kind of node have we selected?
if(e.Node.GetType() == typeof(TransactionTreeNode))
{
// get the transaction...
Transaction transaction =
((TransactionTreeNode)e.Node).Transaction;

// set the data...
viewFromClient.Data = transaction.DataFromClient;
viewFromServer.Data = transaction.DataFromServer;
}
else
{
// clear the dispaly to reduce confusion...
viewFromClient.Data = null;
viewFromServer.Data = null;
}
}

Run the project and make a request from the client. You should see the SOAP messages.

Figure 9: the application running.

Being Intelligent

The problem we have now is that although we can see the SOAP messages being passed, if we make another request we don't get a new item appear in the transaction list. In fact, you'll notice that the scroll bar thumbs get smaller as more data is appended to the existing TransactionData objects.

This happens because we're using version 1.1 of the HTTP protocol and this version can support persistent connections. On the first request, a physical connection is being made through us to the server. Neither the client nor the server are breaking this connection, meaning that the second time a request is made the existing connection is used. As our Connection objects maintain a Transaction object throughout its life, when new data is passed through us, we're appending it to the existing Transaction objects.

A quick and dirty way to get around this problem is to look at the response packets and detect when we see a string </SOAP-ENV:Envelope> appear at the end of data sent back from the server. This will happen in all cases, including ones where the server throws some form of exception.

What we'll do is add a new method to TransactionData that can report if the block of data that we have ends with </SOAP-ENV:Envelope> . Open the code editor for TransactionData and add this code:

        public
            Boolean FoundEnvelopeEnd()
{
// have we ended...
return
            RawData.EndsWith("</SOAP-ENV:Envelope>\r\n");
}

Next, we need to perform that test after we've read bytes from the client, but before we update the transaction. If we're on the client side and we check the data on the server side with FoundEnvelopeEnd , if we return true that transaction is over and we should start a new one.

Open the code editor for Connection and add this new if statement after we read bytes from the stream:

            //
            read data...
int readBytes = source.Read(buffer, 0,
            bufferSize);
if(readBytes > 0)
{
// get the
            transaction...
Transaction useTransaction =
            Transaction;

// if we have one, and we're a client,
// we
            might have ended...
if(useTransaction != null &&
direction == Directions.FromClient)
{
// has the server
            side
            finished?
if(useTransaction.DataFromServer.FoundEnvelopeEnd()
            ==
true) useTransaction = null;
}

As you can see, if we're on the client side and we already have a transaction then we need to look on the server side to see if we've received the end of a SOAP envelope. If we have, we clear useTransaction and the existing code we already have will create a new one for us.

Run the project and the client application and click Create Customers a few times in quick succession. You'll get a new transaction entry in the tree for each click of the button. Selecting each one will report a request/response SOAP envelope pair.

Figure 10: several transactions.

Dropping Connections

From time to time, HTTP 1.1 will drop connections. When this happens depends on the server load, idle time and other factors. If you see more than one client appear in the list, even though you appear to have just one, what's happened is that your connection has been dropped and another has been established. This isn't going to be a problem when you're running the application, but it's worth mentioning.

Spinning Down the Application

You'll have no doubt found that when you close the main application window, the program keeps running - something you can identify by looking for the project state in VS.NET's window caption: if it says " [Run] ", it's still running. We need it to say " [Design] ". That's because the program still has threads processing. We need to create an enforced shutdown, which will involve spinning down the Listener object and killing the threads created by the thread pool to service the clients.

Open the code editor for Form1 and add this code:

protected override void OnClosed(EventArgs e)
{
// stop the listener...
Listener.SpinDown();

// abort the other threads...
_lock.AcquireWriterLock(-1);

We can now walk through the transactions looking for connection objects that haven't been closed. Some of these may already have been closed naturally, but what we'll do is call Close on both the client and the server side just to be sure. We'll use an ArrayList object to keep track of the ones that we have closed and call Abort on each one.

         //
            go through each transaction...
ArrayList closedConnections = new
            ArrayList();
foreach(Transaction transaction in
            _transactions)
{
// have we closed the
            connection?
if(closedConnections.Contains(transaction.Connection)
            == false)
{
// close
            it...
transaction.Connection.Abort();

// add
            it...
closedConnections.Add(transaction.Connection);
}
}

Finally we release the lock and quit the method.

// release the threads...
_lock.ReleaseWriterLock();
}

All that remains now is to implement Abort . Open the code editor for Connection and add this code:

      // Abort
            - forcibly close the sockets...
public void
            Abort()
{
try
{
Source.Close();
}
catch
{
}
try
{
Destination.Close();
}
catch
{
}
}

Run the project now and you will find that if you close the application window, the program will stop properly.

Summary

In this article we've seen how we can use the System.Net and System.Net.Socket namespaces to create a simple proxy server that we can use to capture SOAP messages as they are used with .NET Remoting.


COM+ and C#

COM+ and C#
http://www.csharptoday.com/content.asp?id=1615
Article Rating: 4.5
Published on July 26, 2001
Abstract
In this world of connected network applications, it is quite reasonable that your web applications are expected to scale to meet the needs of the millions of users who will be accessing your web site. COM+ is a core element in the Windows DNA 2000 methodology to create distributed applications that are scalable, extensible and maintainable. In this article, Thiru Thangarathinam helps us understand that COM+ is still an integral part of Microsoft’s strategy to creating n-tiered web applications even in the .NET world. He also shows us how to create a transactional C# object that makes use of COM+ Services in an integrated and logical manner. Finally, we wrap it up by seeing how a transactional C# object can be exposed as a Web service to allow Internet clients to consume it using ubiquitous protocols like XML and SOAP.
Article

Introduction

Component Services in Windows 2000 has a wealth of Enterprise functionality that provides all the plumbing required for creating distributed, web-enabled applications. Although COM+ Services were originally designed for use with COM components, it would be nice if .NET components could also use them because of the services such as object pooling, database connection pooling, sharing resources, role based security and distributed transaction processing monitoring that COM+ runtime provides.

In .NET framework, there is a separate namespace called System.EnterpriseServices that contains all the classes required for providing COM+ services to applications. For an introduction to COM+, please take a look at the following link from TechNet:

http://www.microsoft.com/technet/treeview/default.asp?url=/TechNet/prodtechnol/host/proddocs/hisdoc/appint/comti_introduction_to_com_and_com_.asp

Before we go ahead and create a sample application that makes use of all these classes, let us take a quick tour of some of the important classes present in the System.EnterpriseServices namespace.

ServicedComponent

To be able to create .NET objects that make use of COM+ services, we need to derive our objects from the System.EnterpriseServices.ServicedComponent class. This class provides methods for our objects to interact seamlessly with a COM+ runtime thereby utilizing the COM+ services. It contains the following methods that are directly related to the lifetime of the object:

  • Activate - Allows us to initialize the object when the object is activated. By object activation, we mean the process in which either a new instance of the object is created or an existing object from the pool is used
  • Deactivate - We can use this method to release all the resources that may have been initialized during the Activate method.
  • CanBePooled - Allows us to notify the COM+ whether we want our object to be pooled for reuse.

It also provides the following method that facilitates passing parameters to the object at the time of object construction:

  • Construct

Connection strings to the database is one of the typical parameters that can be passed to the object through the Construct method. In the later part of this article, we will see examples of how to implement these methods in our application.

ContextUtil

Using ContextUtil is the preferred method to access the object context of COM+. The methods present in the ContextUtil class are the primary means through which applications written in managed code can communicate with COM+ runtime. This class can be considered as a replacement for ObjectContext that was used in Pre .Net world in languages such as VB6, C++, ATL, and so on. To get a better understanding of transactions in COM+, please take a look at the following article from MSDN:

Writing MTS style transactions with VB and SQL Server 7.0.

As we already discussed, two important features provided by COM+ are transaction processing monitoring and Object Pooling. COM+ internally maintains two flags. These are a Done bit and a Consistency bit, which are used to vote for the outcome of a transaction and to control the deactivation of objects.

  • SetComplete - This method sets both done and consistency flags to true which indicates that the current transaction can be committed and the object can be deactivated.
  • SetAbort - This method sets the done bit to true and consistency flag to false, which indicates that the current transaction be rolled back, and the object deactivated.
  • EnableCommit - Sets the done bit to false and consistency flag to true.
  • DisableCommit - Sets both done and consistency flags to false.

We can use the above methods to work on the combination of done and consistency bits so that we can indicate COM+ runtime about the state of the transaction and whether the object can be deactivated.

However, there are times where you may want to manipulate the done and consistency bits individually so that you can have finer level of control over the transaction control and the object deactivation processes.

The ContextUtil class exposes the following two properties through which we can deal with the done bit and consistency bit individually.

  • MyTransactionVote - Consists of a property get and set through which we can either get or set the value of the consistency bit.
  • DeactivateOnReturn - Consists of a property get and set, which are used to either get or set the value of the done bit.

AutoCompleteAttribute

Before we go on to look at the AutoCompleteAttribute, let us see what an attribute is and the role of attributes in .NET framework. Attributes are declarative tags that allow the developer to attach additional information to assemblies, classes, methods or properties. This additional information is made available to the applications at runtime through the process known as reflection. We can define reflection as the process, which allows us to examine the types at runtime.

When a method is decorated with the AutoComplete attribute, it indicates that when the method returns successfully, the SetComplete method will be automatically called, and if the method throws an exception, the SetAbort will be called to abort the transaction.

It is also important to note that when we refer to the AutoCompleteAttribute class in our code, we will be simply referring to it only as "AutoComplete ". When the compiler comes across this line, it automatically appends the keyword "Attribute " to it and searches for the class named AutoCompleteAttribute, which it will find. We will see an example of this in action when we create our transactional C# object in the later part of the article.

ObjectPoolingAttribute

This class is used to configure object pooling for a .NET object. Using this class, we can set properties for the maximum pool size, minimum pool size, creation timeout and whether object pooling is enabled for a .NET object.

Registering a .NET object in COM+ Catalog

When you compile a COM component, it creates a type library, which provides COM+ with all the information that is required while installing the component into COM+. But .NET Components, when compiled, emit an assembly (which contains manifest, MSIL, metadata, etc) that is completely different from the type library. So we need to create a type library from the assembly so that the assembly can be installed into COM+ transparently.

There are three ways of registering a .NET component in COM+ in Windows 2000:

  • Using the Regsvcs command line utility. This command line utility takes the name of the assembly as the input, creates a type library by reading the assembly and finally registers the object with COM+.
  • Using the automatic registration mechanism provided by the .NET framework. At runtime, the CLR (Common Language Runtime) automatically creates a type library based on the assembly and COM+ uses the information present in the type library to install the objects.
  • Using the RegistrationHelper class that is part of the system.EnterpriseServices namespace. The InstallAssembly method of the RegistrationHelper class can be used to programmatically register .NET objects with COM+.

In this article, we will see the first two methods of registration.

Implementation

In the first part of the article, we will create a Transactional C# object that leverages the COM+ services of Windows 2000 and then we will go on to create a client application that makes use of the C# object. We will be using ADO.NET to access the database. We will also see how we can take advantage of the automatic registration mechanism provided by CLR to register the objects into COM+.

In the second part of the article, we will create a simple transactional C# 'Hello World' object and then discuss the steps, which we need to follow if we want to expose this object as a web service.

Implementation of a Transactional C# object (Authors)

In this part, we will create a C# class called Authors that contains a single method called AddAuthor which will insert the details of an author into the authors table of the pubs database. Let us take a look at the code of the AddAuthor method. To help avoid redundant typing, we import all the namespaces that we will be using in the application:

using System;
using System.Data;
using Microsoft.ComServices;
using System.Data.SqlClient;
using System.EnterpriseServices;

The ApplicationName attribute indicates the name of the COM+ application that will automatically be created when a client application tries to execute this assembly for the first time:

[assembly: ApplicationName("AuthorsApplication")]
[assembly: Description("ASPToday Demo Authors Application")]

Here, we specify that we want our Authors class to be contained in the AuthorsComponent namespace:

namespace AuthorsComponent
{

In this line of code, we decorate the authors class with the transaction attribute required, which indicates if the caller of this object runs in a transaction, it will be inherited by this object, otherwise it will create a new Transaction for itself and run within the scope of that transaction:

[Transaction(TransactionOption.Required)]

The ObjectPooling attribute allows us to specify the minimum number of objects and the maximum number of objects that exist in the Pool:

[ObjectPooling(MinPoolSize=2,MaxPoolSize=50)]

The ConstructionEnabled attribute allows us to specify a default construction string that will be passed to the object when the client invokes the methods of the object. This construction string is passed to the object during the creation of the object through the Construct method. It is important to note that all the attributes specified here in the code, are automatically set in the component when the component is registered with COM+:

[ConstructionEnabled(Default="server=localhost;uid=sa;pwd=;initial catalog=pubs")]

In this line, we derive the Authors class from the ServicedComponent, which allows the Authors object to make use of the services provided by COM+. Serviced Components are generally hosted in COM+ services as in-process servers (library applications).

public class Authors :
            ServicedComponent
{
public Authors()
{

}

Here, we declare a module level variable to store the connection string.

string mstrConstructString;

As we have already discussed, the Construct method is called during the creation of the object and this method is passed as a string that is specified either programmatically or through the COM+ explorer. In the Construct method, we capture the connection string that is passed, into a module level variable so that it can be used by the AddAuthor method for establishing connection with the database.

public override void Construct(string strConstructString)
{
mstrConstructString = strConstructString;
}

In this line, we enable the AutoComplete attribute for the AddAuthor method. If the method executes successfully and no exceptions are thrown, then the AutoComplete attribute automatically commits all the transactions for us by calling the SetComplete method of ContextUtil. If an exception occurs during the execution of that method, then the SetAbort of ContextUtil is automatically called to undo the transaction.

[AutoComplete]
public bool AddAuthor(string strAuID,
string strAuLName,
string strAuFName,
string strPhone,
string strAddress,
string strCity,
string strState,
string strZip,
int intContract)
{
string strCommandName;
SqlCommand objCommand;
SqlParameter objParam;

Now, we create an instance of the SqlConnection Object, passing the module level variable ConnectionString as an argument.

SqlConnection objConnection = new SqlConnection(mstrConstructString);

We enclose the following statements in a try catch block to catch any errors that may occur during the execution of those statements:

try
{

We assign the name of the stored procedure to be executed to a local variable. This stored procedure is very simple and it inserts a record into the Authors table of Pubs database. The code for the stored procedure is available for download along with the support material of this article.

strCommandName = "spInsertAuthors";

This line of code instantiates an SqlCommand object and two parameters are passed to its constructor. The first one being the name of the stored procedure to be executed and the second one is the SqlConnection object that was created in the previous step:

objCommand = new SqlCommand(strCommandName, objConnection);

By setting the CommandType property of the objCommand object to CommandType.StoredProcedure, we indicate that we want to execute a stored procedure:

objCommand.CommandType = CommandType.StoredProcedure;

The SqlParameter object represents a parameter to an SqlCommand. Here, we create an instance of the SqlParameter passing in the arguments that describe the parameter being passed to the stored procedure.

objParam = objCommand.Parameters.Add(new SqlParameter ("@sAuid", SqlDbType.VarChar, 20));

We set the Direction property to specify that it is an input only parameter. Other values that can be assigned are output only, bi-directional, or a return value parameter.

objParam.Direction = ParameterDirection.Input;
objParam.Value = strAuID;

We follow the same procedure for all the parameters that are passed to the spInsertAuthors stored procedure:

objParam = objCommand.Parameters.Add(new
            SqlParameter ("@sAulname", SqlDbType.VarChar,
            40));
objParam.Direction =
            ParameterDirection.Input;
objParam.Value =
            strAuLName;

objParam = objCommand.Parameters.Add(new
            SqlParameter ("@sAufname", SqlDbType.VarChar,
            20));
objParam.Direction =
            ParameterDirection.Input;
objParam.Value =
            strAuFName;

objParam = objCommand.Parameters.Add(new
            SqlParameter ("@sPhone", SqlDbType.Char, 12));
objParam.Direction
            = ParameterDirection.Input;
objParam.Value =
            strPhone;

objParam = objCommand.Parameters.Add(new
            SqlParameter ("@sAddress", SqlDbType.VarChar,
            40));
objParam.Direction =
            ParameterDirection.Input;
objParam.Value =
            strAddress;

objParam = objCommand.Parameters.Add(new
            SqlParameter ("@sCity", SqlDbType.VarChar,
            20));
objParam.Direction =
            ParameterDirection.Input;
objParam.Value =
            strCity;

objParam = objCommand.Parameters.Add(new
            SqlParameter ("@sState", SqlDbType.Char, 2));
objParam.Direction
            = ParameterDirection.Input;
objParam.Value =
            strState;

objParam = objCommand.Parameters.Add(new
            SqlParameter ("@sZip", SqlDbType.Char, 5));
objParam.Direction =
            ParameterDirection.Input;
objParam.Value =
            strZip;

objParam = objCommand.Parameters.Add(new SqlParameter
            ("@sContract", SqlDbType.Bit));
objParam.Direction =
            ParameterDirection.Input;
objParam.Value =
            intContract;

In this line, we open the ActiveConnection to the Database:

objCommand.ActiveConnection.Open();

ExecuteNonQuery of the SqlCommand object is used to execute a command that does not return any rows against the ActiveConnection:

objCommand.ExecuteNonQuery();

In this line, we return true to indicate that the method is executed successfully. Because of the AutoComplete that we set in the beginning of the method, the transaction will be automatically committed at this point:

return true;
}

If any errors occur during the execution of the above statements, we raise the exception back to the caller:

catch(Exception ex)
{
throw ex;
}

The statements in finally are always executed regardless of whether the statements in the try block are executed successfully or not. Here we inspect the state property of the SqlConnection to check whether the connection is open. If the Connection is open, it is closed.

finally
{
if ( objConnection.State == ConnectionState.Open )
objConnection.Close();
}

Now, let us compile the AuthorsComponent application and create an assembly. The next step is to create a client application, which invokes the AddAuthor method to store the author details to the database.

Implementation of Client Application

First, we select a Windows C# Application from the new project dialog box and we will name the project as AuthorsComponentClient.

Once the project is created, we add reference to the AuthorsComponent by right clicking on the project in the solution explorer, selecting Add reference and then navigating to the path of the AuthorsComponent assembly DLL.

The form that we are going to use for testing our AuthorsComponent looks like this. When the user enters all the details and submits the form, we invoke the AddAuthor method of the Authors class to save the author details to the database.

Let us look at the code that is required for creating an instance of the Author class and invoking its AddAuthor method:

namespace AuthorsComponentClient
{
using System;
using System.Drawing;
using System.Collections;
using System.ComponentModel;
using System.Windows.Forms;
using System.Data;
using AuthorsComponent;

public class AuthorsClient : System.Windows.Forms.Form
{
protected void cmdSaveAuthorsDetails_Click (
object sender, System.EventArgs e)
{
int intContract;

In this line, we convert the text the user enters into the txtContract textbox into an integer and store it in a variable.

intContract = Int16.Parse(txtContract.Text);

Since we have already imported the AuthorsComponent namespace into our application, we need not qualify the Author class with the namespace:

Authors objAuthors = new Authors();

Now, we can call the AddAuthor method of the Author class to save the author details into the database:

objAuthors.AddAuthor(txtAuthorID.Text,
txtAuthorLastName.Text,
txtAuthorFirstName.Text,
txtPhone.Text,
txtAddress.Text,
txtCity.Text,
txtState.Text,
txtZip.Text,
intContract);

MessageBox.Show("Author Details Successfully saved");
}

When we execute the client application, the CLR checks to see whether the assembly is already registered with the COM+ catalogue. If it is already registered, it creates an instance of the object and hands it over to the client. If it is not then it automatically registers the assembly with the COM+.

This kind of registration is called Lazy registration, as it does not require any effort on the part of the developer to register and configure the object to use the services provided by COM+. In the following snapshot, we can see that the name of the assembly name and the assembly description attributes, which we specified in our code, are automatically added to the COM+ application at the time of automatic registration. It is important to remember that the attributes we set at the assembly level correspond to the Application level entries in the in COM+ explorer.

We also specified the transaction attributes and the construction string at the Author class level.

If you right click on the AuthorsComponent Authors class that is present in the components folder of the AuthorsApplication, you can see a window that is similar to the following.

Exposing a Transactional C# object as a web service

In this part of the article, we will create a simple HelloWorld C# object (designed to run in COM+) and then see how to expose this object as a web service so that internet clients can consume this object using internet protocols like XML and SOAP. For an overview of transactions in COM+, please take a look at the following article from Don Box:

Windows 2000 brings significant redefinements to the COM+ Programming model.

Implementation of HelloWorldComponent

First we select a C# class library project from the new project dialog box and we will call this project as HelloWorldComponent.

namespace HelloWorldComponent
{

Here, we import the namespaces that will be required for our application.

using System;
using System.EnterpriseServices;

We want this object to be running as part of the same transaction as that of the calling client, if the calling client is already running in a transaction.

[Transaction(TransactionOption.Supported)]

We derive our HelloWorld class from ServicedComponent to make use of the COM+ services that are encapsulated in ServicedComponent class.

public class HelloWorld : ServicedComponent
{

In this step, we declare the ReturnHelloWorld() method that returns a string back to the caller.

public string ReturnHelloWorld()
{
try
{

Now, we call the SetComplete method of the static ContextUtil class to commit our transactions and as well as to deactivate the object. In the Authors class, we achieved the similar functionality by decorating the AddAuthor method with the AutoComplete attribute, which took care of automatically calling either SetComplete (when the method returns successfully) or SetAbort (when exceptions occur) for us.

   ContextUtil.SetComplete()
            ;
return "Hello World";
}
catch(Exception
            ex)
{

In this line, we call SetAbort to roll back any database changes that we might have done in any of the previous steps.

ContextUtil.SetAbort();
throw ex;
}

Then we will compile the application to create an assembly. This time around, we will manually register the object with COM+ using the regsvcs command line utility.

In the above command, the switch /c indicates that the COM+ application has to be created for the first time and if the application already exists, an exception is thrown. If the assembly is successfully registered with COM+, you should be able to see a new application called HelloWorldApplication that looks something similar to the following, in COM+ Services explorer.

Implementation of HelloWorldDelegate

Once we successfully register the object with COM+, the next step is to expose this object as a web service. To be able to make a method a web callable (web method) one, there are two important requirements that should be met.

  • The method should be decorated with the keyword WebMethod, which indicates that this method is web-callable.
  • The class that contains the method should be derived from WebService class.

In the above example, we have already derived HelloWorld class from ServicedComponent for leveraging COM+ services. Since the CLR does not allow a class to be derived from more than one class, we cannot derive the HelloWorld class from the WebService class. To overcome this constraint, we will create a separate class called HelloWorldDelegate, which simply delegates method calls to the underlying HelloWorld class. Because of its delegating nature, let us call this class as a delegate and we derive this delegate class from the WebService class.

namespace HelloWorldComponentDelegate
{
using System;
using HelloWorldComponent;
using System.Web.Services;

public class HelloWorldDelegate : WebService
{

The attribute [WebMethod] indicates that the HelloWorld method is to be exposed as a web method and when it is deployed, Visual Studio.NET runtime provides all the plumbing required to make this method discoverable to the clients who wish to use this web service.

[WebMethod]public string HelloWorld()
{

In this line of code, we create an instance of the HelloWorld object:

HelloWorld oHelloWorld = new HelloWorld();

Here, we delegate the call to the ReturnHelloWorld method of our HelloWorld object that is hosted in COM+.

return(oHelloWorld.ReturnHelloWorld());
}

Finally, let us build the application into a DLL.

Implementation of HelloWorldDelegateWebService

Now that we have a class that is derived from the web service class and a method that is decorated with the keyword WebMethod, it is very simple to expose this object as a web service. For an introduction to web services, please take a look at the article Web Services with ASP.NET from MSDN.

Let us create a new C# web service project and call it HelloWorldDelegateWebService. We can then rename the default web service (WebService1) to HelloWorldWebService and also add a reference to the HelloWorldDelegate assembly, which we created in the previous step. We will then open the HelloWorldWebService.asmx file and modify its contents to look like the following:

<%@ WebService Language="c#" Class="HelloWorldComponentDelegate.HelloWorldDelegate" %>

That is all we have to do for exposing a precompiled object as a web service. If you compile your web service application and navigate to the HelloWorldWebService.asmx page, you should see a screen similar to the following. This is a test harness, which ASP.NET creates for testing the web services:

Clicking the Invoke button brings up the following screen, which shows the result of the HelloWorld web method's execution.

Prerequisites

To execute the code that is available for download, you need to have the following software installed on your machine:

  • Windows 2000 Professional/Server with Service Pack 1
  • SQL Server 2000
  • Visual Studio.NET Beta2

Conclusion

In this article, we have discussed how to leverage COM+ services from managed environment using the classes supplied by the .NET framework. We have also seen how easy it is to expose a precompiled .NET object as a web service, which allows clients to consume the exposed functionality through Internet protocols like XML and SOAP.

I hope you find this article useful and thank you for reading.


C# and Java Remoting Part 1 - a C# Remoting Primer

C# and Java Remoting Part 1 - a C# Remoting Primer
http://www.csharptoday.com/content.asp?id=1600
Article Rating: 2.5
Published on June 29, 2001
Abstract
In this series of articles, we will look at interoperating between C# and Java on different platforms, and the problems involved with doing so. The bulk of this first article, however, will be a primer on C# Remoting. Ashish Banerjee goes under the covers of the .NET Framework SDK's 'MyHello' remoting example, before showing a method of tracing and debugging SOAP packets - an essential tool to have as remoting around the globe requires us to be able to trace errors should something go missing or wrong.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

In this series of articles, we will look at interoperating between C# and Java on different platforms, and the problems involved with doing so. The bulk of this first article, however, will be a primer on C# Remoting.

An interesting problem for today's developers is in communicating between different servers running different operating systems and different languages. The are a number of options, for example, for interoperation between Java and C#:

  • ActiveX Java bridge
  • Cross platform compilers
  • Remoting with SOAP

The next three sub-sections will explore each briefly.

ActiveX Java Bridge

Both Microsoft and Sun provide their versions of COM and Java interop bridge. Microsoft's Java COM bridge can interpolate Java and Microsoft Languages, making Java appear as COM object. Prior to .NET we could call Java using the Java moniker via the GetObject method of Visual Basic. For example:

DIM OBJ As Object
SET OBJ = GetObject("java:java.util.Date")
print OBJ.toString()

The above code could be executed in VBA and VB and will create the Java date object through the Java moniker and the OBJ will be assigned an object reference to an instance of Java Date object, managed by Microsoft JVM.

Using the Sun's ActiveX Java Bean Bridge allows Java components to be treated as COM objects. The .NET platform however, treats COM components more or less as legacy code. Therefore COM components are executed as unmanaged code and memory is allocated from the standard windows heap space, and not allocated from a heap managed by the Common Language Runtime (CLR). However, it does provide mechanisms for both exposing .NET components to COM objects and vice versa. When a COM client creates an instance of a .NET object, the object is exposed through a COM callable wrapper (CCW) that acts as a proxy for the real managed object residing in the CLR (Common Language Runtime).

We will not discuss this further in any more detail, as it is beyond the scope of the article, but you can read http://www.asptoday.com/content/articles/20000804.asp?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq for more information.

Cross Platform Compilers

The cross platform compilers are an interesting idea, which I have explored in a reasonable detail in my article entitled "Will Java and .NET Framework Co-exist?".

Briefly, when C# and Java cross compilers are available you will be able to interoperate the C# and Java using each other's API. For example you would be able to compile C# program into JVM (Java Virtual Machine) executable or write in Java and compile it to target CLR (Common Language Runtime). However, such cross compilers are still on the drawing boards. If you come across any such cross compilers please let me know.

Remoting with SOAP

Presently, this is the most practical option available to real life programmers and system architects for interoperating Java and C#.

You can download the Java SOAP implementation from xml.apache.org. The source is well written, but is bit heavy. I would like to see a more lean and mean Java SOAP gateway, with a good documentation to go with it.

SOAP is an XML based messaging protocol and uses HTTP and SMTP (Simple Mail Transfer Protocol) as its transport. Though the SOAP specification defines HTTP protocol as its transport, both the Apache Java SOAP gateway and Microsoft .NET SOAP framework implement an SMTP/POP3 transport pair as well.

The SOAP specification is available at http://www.w3.org/TR/SOAP?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq. I strongly recommend you download and have a look at it.

In the next few sections of this article we will see how to set up remoting using SOAP, with the help of Microsoft's MyHello.NET SDK remoting sample, and also see the use of a trace utility which helps us to trace problem messages.

Installing .NET SDK

If you have not already done so, please install the .NET SDK on your system. Though some portions of the SDK requires Windows 2000 server, for all practical purposes of compiling C# and remoting, I have had no problem running it on of Windows 98.

Running the MyHello Sample

Once you have installed the SDK, you will find the samples at the following installation path:

C:\Program Files\Microsoft.NET\FrameworkSDK\Samples\Remoting\

The remoting sample we will focus on is MyHello, which runs in conjunction with the HelloService sample. The remoting samples can be built by running the command:

C:\Program Files\Microsoft.Net\FrameworkSDK\Samples\Remoting > nmake all

You will also find instructions to build and run the sample in readme.htm file in the C:\Program Files\Microsoft.Net\FrameworkSDK\Samples\Remoting\MyHello directory.

To run the sample open two Command Windows, do not forget to run corvars.bat which is available in C:\Program Files\Microsoft.Net\FrameworkSDK\Bin.

  • In one Command Window change the directory to:
    C:\Program Files\Microsoft.Net\FrameworkSDK\Samples\Remoting\MyHello
  • Open another Command Window, and change directory to:
    C:\Program Files\Microsoft.Net\FrameworkSDK\Samples\Remoting\HelloService
  • From the above directory execute the batch file: HelloService.bat
  • After a brief interval you will see a message:
    Press a key and ENTER: G=GC.Collect, 0=ExitThis
  • The above message indicates that the HelloService is ready to serve.
  • From the first Command Window execute the batch file: sample2.bat
  • When this batch file executes, it will prompt you to enter your name, I typed Ash, and then after some time it displays the result:
    Hello.HelloMethod returned: "Hi there Ash"

If you get a similar result, you have successfully executed the remoting sample. Let's analyze what's happening behind the scene.

Code Walking

The batch file, HelloService.bat executes MyHost.exe with the file HelloService.cfg as its parameter.

MyHost is a simple yet generalized program, which can act as a standalone SOAP remoting server. MyHost.cs is the C# source file from which MyHost.exe is derived. The only line of interest to us in MyHost.cs is the one below:

RemotingServices.ConfigureRemoting(args[i+1]);

RemotingServices provides methods for publishing remote objects. This class is functionally equivalent to Java's java.rmi.server.RemoteServer class. However, the designs of the two classes differ significantly.

  • The RemotingServices has a utility design pattern, meaning that all the methods of this class are static and you cannot create an instance of this class.
  • The method ConfigureRemoting takes a file name as parameter, to configure the remoting object services.
  • The argument passed to the above method is the file name HelloService.cfg.

Let us now look inside the content of the configuration file, which is passed as a parameter to the static method ConfigureRemoting of the class RemotingServices.

Only the first three lines of this configuration file are relevant, the rest being comments. These first three lines are reproduced below:

Name#HelloService
WellKnownObject#HelloService.Hello#HelloService#HelloService/Hello.soap#SingleCall
Channel#System.Runtime.Remoting#System.Runtime.Remoting.Channels.HTTP.HTTPChannel#ports=8085

  • The first line describes the name of the application, HelloService.
  • The second line registers the Object name with the remoting service and the last word SingleCall is the keyword that instructs the remoting service to create a new instance every time this object is accessed remotely by a client. For more scalability, use Singleton, whereby every message is dispatched to the same object instance
  • The third line of the configuration file defines the communication channel and it's parameters. That is, the example uses HTTP protocol at port 8085.

Note that the configuration file has a URI (Universal Resource Information) like syntax, similar to that of JDBC (Java Data Base Connectivity) configuration parameters. I would have expected the .NET architects to use XML based syntax.

Now, have a look at the source code of HelloService, contained in the file HelloService.cs. This file defines, within a namespace HelloService, a simple class Hello that extends the class System.MarshalByRefObject. This is the base class for Remoting objects that need marshaling by reference. Java programmers can relate this class to its Java counterpart java.rmi.server.UnicastRemoteObject.

The HelloService.Hello class has only one method, HelloMethod, which concatenates "Hi there" with the input parameter passed to it.

Now, this brings us to the final leg of our code walk through. Change your directory back to C:\Program Files\Microsoft.Net\FrameworkSDK\Samples\Remoting\MyHello and type sample2.bat (which contains the command MyHello -cfg MyHello.DirectHTTP.cfg ).

So, MyHello is being called with a configuration file MyHello.DirectHTTP.cfg. If you casually look into the source code of MyHello class you might not find anything that suggest that you are looking at a remoting sample.

Well this is a double-edged sword! On one side it lets you write remoting programs without bothering about the details of SOAP. The flip side is that while deploying or implementing, someone might accidentally break it. To demonstrate this point, if you comment out lines 2,3,4 of the configuration file MyHello.DirectHTTP.cfg, then you will be executing MyHello sample without remoting! That is, even if MyHost is not running on the other Command Window, your program will still run.

If you have written complex programs that need deployment and configuration and use remoting as well, then I am sure you will immediately realize that this is a potential cause of disaster waiting to happen. It's guaranteed to give you sleepless nights, trying to figure out why the installation on the other side of the globe is misbehaving!

Most of the times it will not be so bad, so let's get back to MyHello.cs code. Look at line 69, which is same as line number 80 of MyHello.cs:

RemotingServices.ConfigureRemoting(args[i+1]);

The same method ConfigureRemoting is to be used by the client code as well. How is this? Well the CLR (Common Language Runtime) provides isolation between Application Domains (AppDomain ). The runtime creates a proxy stub, to handle inter AppDomain method calls. Since this binding is delayed till execution; you can affect its behavior using configuration files similar to the one used by this sample, MyHello.DirectHTTP.cfg, and call the static method RemotingServices.ConfigureRemoting from anywhere in the program, (but first create the instance of an object whose method is being called). In this case it is the HelloService.Hello object.

Incidentally, line 3 of MyHello.DirectHTTP.cfg defines that HelloService is a remoted application and accessed though HTTP protocol, residing at localhost at port 8085. This line is below:

RemoteApplication#HelloService#HTTP://localhost:8085/HelloService

The next section will look into SOAP (Simple Object Access Protocol) packets being exchanged in this sample between the client and the server. Thereafter, I will share with you the tools I use to look under the hood.

How do MyHello and HelloService Talk?

In the sample discussed above, the client and the server exchange request and response packets in SOAP (Simple Object Access Protocol), through HTTP transport.

As discussed, SOAP is an XML based protocol. A SOAP message is contained in an envelope and it has header and a body. The sample exchange the messages reproduced are below in req.txt and res.txt.

Later I will discuss how to trap the messages. The trapping technique will also help you while debugging applications, which spans companies and sometimes continents. In those situations, you cannot use conventional debuggers, neither can you expect your counterparts, working for another company and in different time zone, to co-operate with your team, while they may be themselves struggling to meet their deadlines! I have also found this trapping technique helpful in convincing the team working at the other end of a B2B application that their application is misbehaving and sending out wrong packets.

MS provides a functional SOAP trace utility that is said to be very helpful in debugging. However, I have not used it as of yet, and will demonstrate one I wrote myself.

Analyzing the SOAP Messages

MyHello is the client program, which sends a SOAP request, see req.txt below:

01 POST /HelloService/Hello.soap HTTP/1.1
02 SOAPAction: "http://Schemas.microsoft.com/urt/NSAssem/HelloService.Hello/HelloService#HelloMethod"
03 Content-Type: text/xml
04 Content-Length: 1025
05 Connection: Close
06 Host: localhost
07
08 <SOAP-ENV:Envelope xmlns:xsi="http://www.w3.org/1999/XMLSchema-instance"
xmlns:xsd="http://www.w3.org/1999/XMLSchema"
xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/"
SOAP-ENV:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"
xmlns:a2="http://schemas.microsoft.com/urt/NS/System.Runtime.Remoting"
xmlns:a1="http://schemas.microsoft.com/urt/NS/System"
xmlns:i7="http://Schemas.microsoft.com/urt/NSAssem/HelloService.Hello/HelloService"
09 <SOAP-ENV:Header
10 <__CallContext href="#ref-3" root="1"/
11 <a1:LogicalCallContext id="ref-3"
12 <__RemotingData href="#ref-4"/
13 <__SecurityData href="#ref-5"/
14 </a1:LogicalCallContext
15 <a2:CallContextRemotingData id="ref-4"
16 <_logicalCallID id="ref-6"ca2ac5a1-4c90-11d5-917d-c780bb182871_2</_logicalCallID
17 </a2:CallContextRemotingData
18 <a1:CallContextSecurityData id="ref-5"
19 </a1:CallContextSecurityData
20 </SOAP-ENV:Header
21 <SOAP-ENV:Body
22 <i7:HelloMethod id="ref-1"
23 <name id="ref-8"ash</name
24 </i7:HelloMethod
25 </SOAP-ENV:Body
26 </SOAP-ENV:Envelope

Lines 1 to 6 are HTTP headers. If you happen to be curious about the HTTP protocol or reading through the SOAP specification has not helped your insomnia, then you can download the protocol from http://www.ietf.org/rfc/rfc2616.txt?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq.

The lines 8 to 26 contain the SOAP headers and body, wrapped in the SOAP envelope. The lines 22 to 24, pass the information regarding the method call HelloMethod, with line number 23 containing the parameter value being passed to the server.

The SOAP server is contained within the CLR (Common Language Runtime), which is equivalent to JVM (Java Virtual Machine). This server, activated by MyHost, running on second Command Window, receives the request packet, parses the XML, and calls the requested method of HelloService.Hello class, after instantiating a new instance. The resulting return value is sent back to the client through the response packet listed in res.txt below:

01 HTTP/1.1 200 OK
02 Content-Length: 1058
03 Content-Type: text/xml
04 Connection: close
05
06 <SOAP-ENV:Envelope xmlns:xsi="http://www.w3.org/1999/XMLSchema-instance"
xmlns:xsd="http://www.w3.org/1999/XMLSchema"
xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/"
SOAP-ENV:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"
xmlns:a2="http://schemas.microsoft.com/urt/NS/System.Runtime.Remoting"
xmlns:a1="http://schemas.microsoft.com/urt/NS/System"
xmlns:i7="http://Schemas.microsoft.com/urt/NSAssem/HelloService.Hello/HelloService"
07 <SOAP-ENV:Header
08 <__CallContext href="#ref-3" root="1"/
09 <a1:LogicalCallContext id="ref-3"
10 <__RemotingData href="#ref-4"/
11 <__SecurityData href="#ref-5"/
12 </a1:LogicalCallContext
13 <a2:CallContextRemotingData id="ref-4"
14 <_logicalCallID id="ref-6"ca2ac5a1-4c90-11d5-917d-c780bb182871_2</_logicalCallID
15 </a2:CallContextRemotingData
16 <a1:CallContextSecurityData id="ref-5"
17 </a1:CallContextSecurityData
18 </SOAP-ENV:Header
19 <SOAP-ENV:Body
20 <i7:HelloMethodResponse id="ref-1"
21 <__return id="ref-8"Hi there ash</__return
22 </i7:HelloMethodResponse
23 </SOAP-ENV:Body
24 </SOAP-ENV:Envelope

Again the lines 1 to 4 of res.txt are the HTTP headers. Lines 6 to 24 contain the SOAP envelope headers and body. Line 21 has the return value, which is displayed by the MyHello client. The MyHello client is isolated from all these details by the CLR (Common Language Runtime) proxy stub for the object HelloService.Hello. Behind the scene, the runtime at the client end, receives the request, parses the SOAP message, and de-serializes (or unmarshals) back the string.

Understanding the Trapping Technique

While consulting for a sub contractor, an application we developed was having problems talking to the application running at the trading partner end. I developed these simple Http message trappers to conclusively resolve the debate as to where the problem was laying. As a side benefit, I also found this tool helpful for simulating various error conditions that our programs may encounter from other end.

The idea is to simulate the Http client or the server, depending on which side of the application you plan to debug. If you wish to trap the output of you client side application, use the HttpTrapper.cs, which is the server simulator. To debug the server side of your application you would use the client simulator, HttpTrapClient.cs.

Let me summarize the process. Suppose we have a server application S that calls the client side C. And let us say that under a certain condition C and S refuse to talk. To figure out what is happening, my technique requires a three pass process.

The HttpTrapper is placed in the place of S, and as the client C talks to the server, the trapper will save the client request in a file.

The second pass involves using this captured request file to send the response to the actual server S, using HttpTrapClient. The server, S, will cough up a response, which will be captured in another file. Now we have both the request and responses available in a text file, which we can now scrutinize for errors.

Walking the Code

Now compile the three accompanying C# utilities by executing the from nmake all Command Window. This will produce AddrTest.exe, HttpTrapper.exe and HttpTrapClient.exe, from their respective C# sources with the same name.

AddrTest.cs is a test program, which helps to ensure that TCP/IP and DNS are configured and working properly in my machine:

using System.Net;

public class
            AddrTest
{
public static void
            Main()
{
System.Console.WriteLine(DNS.GetHostName());
}
}

The actual programs we will use for trapping are HttpTrapper.cs and HttpTrapClient.cs.

Let us walk you through the passes.

Pass One: Catching the Request

First, stop the MyHost sample if it is running on any of your Command Windows.

Now run the trapper program in another Command Window using the following command:

HttpTrapper 8085 ERR.TXT

The file err.txt is simple two-line text file that simulates an error condition.

From a Command Window run sample2.bat, as we did in the first section of this article.

The sample2.bat will execute the MyHello sample, give your name when it prompts for it. MyHello will trigger the CLR (Common Language Runtime), which will contact the Trapper, which acts as the server simulator. The trapper HttpTrapper captures the request message in a file called delme0.txt. I renamed this file to create the req.txt. The trapper however, sends back the error message contained in the file err.txt.

At this point, a very interesting event occurs. The CLR (Common Language Runtime) receives back the error response and after displaying an error message box, spills out the stack trace. Analyzing this trace helps us get the brass tacks of remoting, as implemented by the Microsoft engineers. The stack dump occurs during the dereferencing (also called unmarshalling) stage of remote method calls implementation. The Microsoft engineers, I am sure, will plug this hole before a production release, but as of the Beta 1 release (I have not tested this hole on Beta 2), which I am using, it bares it all!

Following the Stack Trace

Let us walk the stack trace:

The ball starts rolling from line 36 of MyHello.cs. This is the place from where the HelloMethod of the HelloService.Hello object is called. From thereon, the CLR proxy takes over, since the configuration file specifies the remote object (see line 3 of file MyHello.DirectHTTP.cfg ). Thus the CLR System.Runtime.Remoting.RealProxy.PrivateInvoke method is called passing frame and type as parameters. This in turn calls System.Runtime.Remoting.RemotingProxy.Invoke passing a request message object which implements IMessage interface.

Next System.Runtime.Remoting.RemotingProxy.InvokeInternal is called, followed by the call to System.Runtime.Remoting.RemotingProxy.CallProcessMessage, which goofs up. The method CallProcessMessage is responsible for actually contacting the server and uses the first parameter, an implement of IMessageSink to talk to the server.

The IMessage interface is an abstract representation of the messages passed between the SOAP client and the Server. The object implementing the IMessage interface can change the way messages are serialized.

The objects implementing the IMessageSink Interface are responsible for transport layer communication. 

Pass Two: Trapping the Response

In the next pass, we will use the HttpTrapClient.cs and pass the trapped request packet to the SOAP server using the sample MyHost, we used in Step 1 section of this article.

Now change the directory in a Command Window to HelloService, and execute HelloService.bat as we did before.

Now from the Command Window we can run the HttpTrapClient program by executing the command below, to send the captured request to the SOAP service and receive back the response.

HttpTrapClient REQ.TXT RES.TXT 127.0.0.1 8085 20

The first two parameters are the request and response files, followed by the server IP address. If you are running the server on a different machine AddrTest.cs will display the IP address for you. The last parameter is the timeout in seconds. The last but one parameter is the server port to use.

Now, we have both the request and response with us, which I used in this article to get a insight into the SOAP communication protocol implementation in Microsoft's CLR (Common Language Runtime).

Pass Three: Putting it Back Together

This pass I have included for completeness. Stop the HelloService, and change directory back to MyHello. Run the batch file sample2.bat again.

From the Command Window run the HttpTrapper again, but this time using res.txt file. Use the command given below:

HttpTrapper 8085 RES.TXT

Change the focus back to the sample2.bat Command Window where MyHello is executing. When prompted type your name and after some delay you will get back the original response!

Conclusion

I hope that I was able to share with you some insights I have gained on Microsoft's SOAP service implementation of the .NET framework.

As for all server side Java programmers, .NET beholds the promise to interoperate Microsoft platforms resident clients with scalable Java server applications.

Now that we have had a good look at C# Remoting, in the next articles in this series we will show how to communicate between C# and the JVM.


Web Site Error Logging and Reporting System

Graphics, Games

Graphics, Games (Sort by Name)

A complete text-printing framework using GDI+
An Introduction to GDI+
An Open Invocation
Beginner's Hangman
Building .NET Reporting Solutions with XML and GDI+
Creating Graphical Reports With Crystal Reports in .NET
Developing A C# Vector Drawing Application - Part 1
Developing A C# Vector Drawing Application - Part 2
Developing A C# Vector Drawing Application - Part 3
Graphical Representation of Data Grids Using C# and GDI+

Graphics and Animation in the MS .NET Framework Using GDI+ - Part 1
Graphics and Animation in the MS .NET Framework Using GDI+ - Part 2
ImageIt - Image processing for Photo Galleries
Irregular Shaped Forms and Other Cool GDI+ Features
Owner Drawn Menus in Visual Studio .NET Using C#
Polymorphism in C# - Creating a Drawing Program
Printing Using C#
Windows Forms Charting - Part 1
Windows Forms Charting - Part 2


A complete text-printing framework using GDI+

A complete text-printing framework using GDI+
http://www.csharptoday.com/content.asp?id=1970
Article Rating: 4
Published on November 4, 2002
Abstract
Applications that are meant to present their users with a sophisticated and rich user interface will probably need to take full advantage of Window’s text formatting capabilities, including numerous fonts, styles, and special effects. Although the System.Drawing namespace does provide useful methods for text printing, a professional application cannot be based on these alone. In this case study, Andreas Papathanasis looks through all the framework-provided services that deal with text, and then creates a base class that all windows controls which need to display ordered text output on their client area will find useful.
Case Study

Applications that are meant to present their users with a sophisticated and rich user interface will probably need to take full advantage of Window's text formatting capabilities, including numerous fonts, styles, and special effects. Although the System.Drawing namespace does provide useful methods for text printing, a professional application cannot be based on these alone. In this case study, we will go through all the framework-provided services that deal with text and then go ahead to create a base class that all windows controls which need to display ordered text output on their client area will find useful.

Introduction

Microsoft's .NET framework provides a wealth of pre-made classes that save developers countless hours of work on operations that used to be both resource and time demanding. The framework class library is carefully divided into tens of namespaces, each one offering useful services through which, with just a few lines of code, a developer can perform complex tasks ranging from displaying images of various formats, to sending e-mails or querying web sites.

The System.Drawing namespace is no exception to the rule. Developers are now able to handle graphics with great ease, creating complex shapes with just a few methods which are both easy to remember and use. The comparison to GDI+'s predecessor is overwhelming, as complicated procedures that had to be designed by the programmer alone in the days of GDI in C++ are now encapsulated in ready-to-use classes and methods. Of course, it's not to say that every conceivable programming task is now easy to accomplish. As computers and their users' needs grow more and more complex, it would be an impossible task for any pre-made class library to do so. The generic methods provided will need to be tailored to specific applications' needs. For example, someone who programs a charting application will firstly create his own classes (based on the .NET framework's classes) that provide exactly the kind of functionality he needs for the specific program he is creating. These classes would use various calls to existing System.Drawing helper classes, and these would help in designing and displaying pie charts, 3D bar chars, etc.

There are cases, however, when these user-designed classes are so generic that they can be used by not just one, but several similar applications. Returning to our charting application example, a complete graphics engine that consists of classes that describe and provide common methods on coordinates, bars, pie charts, etc, could be used in many similar projects. It is in the developer's best interest to design more generic class libraries where applicable, as this allows them to be reused in other projects, thus improving the maintainability and scalability of programs. We will be thoroughly examining a similar situation in this case study. We will design a generic library that can be used by all projects that require a specific kind of functionality.

System Requirements

You should have a PC running Windows 2000 or Windows XP to run the code provided in this case study. A reasonable level of familiarity with GDI+ is assumed (if you wish to read up on the basics of GDI+, please see the two previous C#Today case studies on this topic, shown in the links section of this piece). The reader should also be familiar with C# syntax and concepts, especially classes and inheritance. Visual Studio .NET is a requirement for anyone who wants to follow the step-by-step examples (no service pack is required), although the .NET SDK should be enough for more experienced developers to make the most out of the code provided.

Defining The Problem

If you've tried before to create a custom application that has to display text using GDI+, then there is no doubt that you've stumbled upon the limitations of the provided methods as far as ordered text output is concerned. If you try to print a text string that has a greater length than the form's width, only part of it will be displayed, the part that fits on the window. Print two text sentences too close, and they fall all over each other. Not to mention the word wrapping or automatic scrollbar services for text - there aren't many. Moreover, some conventional coding styles may also lead to problems with the application. A font the developer chooses to use might not exist on every target computer, and the framework will substitute it with a default font, changing not only the intended shape the programmer wanted to give to the letters, but also their size. Not exactly the "features" you would want for a professional application. In general, developers cope with the problem in different ways, most of the time coding specifically for the application they are creating.

As the problem is very generic in nature, it would be much more efficient to deal with it in a generic way -that is, create a class library that contains helpful methods that all applications that need to draw text can benefit from. In the remainder of this case study we are going to create this library, examining in detail the process, from design considerations to using the completed library in common development scenarios.

Case Study Structure

  • Basic concepts - text printing services in GDI+
  • The DrawString() method
  • The System.Drawing.Text namespace
  • Designing our program infrastructure
  • Implementing the LineInfo and SentenceInfo classes
  • Measuring string sizes
  • The TextDisplayControl control
  • Implementing text wrapping
  • Testing the component through a Windows Forms application
  • The TextEditControl control
  • Ideas and guidelines for extending the controls
  • Case study review
  • Limitations and further work

Defining The Solution

During this case study, we are first going to create a complete generic control that will contain all our abstract ideas on ordered text display in lines and columns, eliminating the problems described earlier. This control will extend its functionality to provide simple to use methods that will display text of any possible style. Our implementation of all the necessary text drawing features will be based on the existing GDI+ methods - however, we'll be designing quite complex algorithms to ensure that the control never presents garbled text in any case, and moreover displays scrollbars or wraps the text so that no information is lost because of its dimensions. The trick here will be to simultaneously manage an unrestricted set of displayed strings in different lines and columns, that will not necessarily have the same style. Columns and lines will not have specific lengths or heights, as the fonts used throughout the document will determine these locations accordingly. After that, we'll delve further into how to use that control by looking at two very useful examples. The first one, a Windows forms application, will feature an extended control that will be able to print text very easily, anywhere on its client area. Of course, details like coordinates in pixels will be hidden from the end user, as well as other developers using our control - all they will have to do is provide line and column numbers, as well as the style for the text to be printed (font, color, size, etc). The second application will be another custom control, similar to a textbox: the user will be able to enter text in a word-processing program style, and we will be providing him with the chance to define the font, size, and color of any part of the text that he is writing.

As any solution that applies to a problem of a generic nature should do, our program will be scalable and adaptable to a wide array of applications. Although the document we will be displaying on the control might need to be large, or made up of many different fonts and sizes, the performance hit with large documents will stay low as we'll take special measures to draw only the parts of the document needed each time, conserving system resources. The second text-editing application would seem quite difficult to build to anyone who sees the end product, but as we shall see, it will be quite easy with the implementation of the first control in place. Moreover, towards the end of this case study, we'll examine how it is possible for developers to build on the code presented here.

Basic concepts - Text Printing Services in GDI+

For those who haven't been into the realm of text printing using GDI+ before, it's useful to mention some of the basics. The fundamental concepts are easy to grasp: shapes, text strings, etc, are displayed via an instance of a Graphics object, and more accurately through one of the few overloads of its DrawString() method. If showing text on a windows application is as simple as calling a method, what then is the purpose of this case study? Actually, as we explained above, the pre-made code for displaying text does not check other text strings on the client area of the control, so overlapping will occur without additional code. We will later code our own methods to make sure none of these inconveniences appear when displaying text strings. What we are actually doing is use the control's client area (or part of it) to draw the strings needed -of course, we won't limit the control to just displaying text. We'll talk more about this last issue later on.

The fundamental class when dealing with text is the Font class - it's pretty much to text drawing what the Pen class is to line/shape drawing. Before going into how fonts are implemented in the .NET framework however, we should have a clear representation of them into our minds.

Every font has a name. " Arial ", " Times New Roman ", and " Courier " are some well-known font names that have been used in the Windows operating system for years. This name defines the exact shape of the digits (letters and characters), that is, every font name specifies exactly how every printable character should look on the user's screen or printer -except for its size.

Additionally, every font has a size, which defines the vertical size in points (one point is equal to 1/72 of an inch) of the letters displayed. Fonts also have other attributes that specify the style. These include attributes like bold, italics, underline, and strikethrough. Finally, text displayed on the screen has a specific color.

Some years ago we might have distinguished fonts as true type or bitmaps - that's not really necessary now as fonts are all true type. That means that in their implementations, these fonts specify the digits' shapes using mathematical algorithms so that they can be drawn in any given size. That's why we can exactly specify the size of true type fonts, even using float numbers - although in normal cases, integer numbers will be precise enough for our needs.

Going into the .NET world, the programmer must first distinguish between two classes. There are actually Font and FontFamily classes in the System.Drawing namespace. It might be a bit confusing to learn that the known "fonts" like Arial or Times New Roman are actually font families, in that they represent a group of styles, bound together by their similar shapes for separate characters. Font families specify only these shapes - a Font object, on the other hand, represents a well-defined text style within the font family, with its size and other properties (bold, italic, strikethrough, underline) initialized to specific values. In short, FontFamily is generic, Font is very specific. Ultimately, how the text is actually displayed depends on its Font and drawing Brush. You should already be familiar with brushes, as they are also commonly used when drawing shapes. Although brushes in the text-printing area are almost always used to define a color to display the text with (and that's what we'll do in our application later on), it is also possible to pass some of the more advanced types of brushes (like TextureBrush and LinearGradientBrush) to create some rather peculiar effects when drawing text:

I know - not exactly suitable text styles to include in a serious-looking application! However, with some patience it is possible to create some cool (perhaps even animated) effects with brushes, and there are cases where they could be used -for example, in games. Text effects like the one above are created very easily - let's not forget that in the pre-.NET world, something like that would be much more painful to create.

The Font class has no less than 13 constructors. In all of them, the first parameter is a string, a FontFamily, or another Font object, and sets the font family of the object. If a string is provided, it must exactly match the name of a font family installed on the computer (although casing does not matter), otherwise a default family will be used. Unless we have used a Font as this first parameter, we must also specify the size of the font family we declared as a second, float parameter. From there, it is possible to specify the style of the font, combining elements of the FontStyle enumeration. This enumeration is made up of the members Bold, Italic, Regular, Strikeout and Underline, and has the Flags attribute; therefore, its members can be combined using the bitwise or operator (|). It's rather obvious from the members' names what the effect each one has when drawing the text. Some Font constructors also support a value from the GraphicsUnit enumeration to specify a unit of measurement for the font, like pixels or various parts of an inch. The default is pixels, and this will be fine for our needs later on in this case study. Some other parameters for the more advanced Font constructors have to do with the older GDI framework, and we won't be covering these in this case study.

The DrawString() method

This is the single most important method from the System.Drawing namespace that we will be using later in the case study. It is used to display a string on the output device, using a specified font and brush. There are 6 overloads of this method in total. Their signatures appear below:

public void DrawString(string s, Font font, Brush brush, PointF point);

public void DrawString(string s, Font font, Brush brush, RectangleF layoutRectangle);

public void DrawString(string s, Font font, Brush brush, PointF point, StringFormat format);

public void DrawString(string s, Font font, Brush brush, RectangleF layoutRectangle, StringFormat format);

public void DrawString(string s, Font font, Brush brush, float x, float y);

public void DrawString(string s, Font font, Brush brush, float x, float y, StringFormat format);

In any case, we have to provide the actual string to be displayed, and the Font and Brush objects to draw it with. From there, we have a choice on whether we are going to provide only a starting point (x and y coordinates) from where the drawing will start, or a rectangle inside which the string will be displayed. The second choice (passing a whole rectangle) has numerous advantages in many cases. The DrawString() method can automatically align and wrap text inside a given rectangle (depending on the value of other parameters). If we don't provide a rectangle, we of course can't expect the text to be aligned or wrapped, because the framework won't know where to start the wrapping or alignment.

We can also provide format information by means of an object of type StringFormat, or omit it and leave the default settings. StringFormat objects define many parameters that effect how the string will be displayed. Some properties of interest from the StringFormat class are as follows:

  • Alignment: Defines how the string is aligned when drawn inside a specific rectangle (center, left, or right). Its value is one of the members of the StringAlignment enumeration. This enumeration contains the Near, Far and Center members. In most cases, when the text is drawn from left to right, "near" means left aligned and "far" right aligned, but this is changed on other drawing directions. For example when drawing from right to left (there is a special flag to do that, see the FormatFlags bullet below) "near" will be right alignment, and when drawing from top to bottom, setting the alignment to StringAlignment.Far will mean alignment to the bottom of the rectangle.
  • Trimming: This is how the text is trimmed when drawn in a rectangle (nearest character, nearest word, etc). We'll use this one later in our code, as we'll want lines that are wrapped to be truncated to the nearest word.
  • FormatFlags: Sets various flags that contain string-formatting information. These flags appear in the enumeration StringFormatFlags. This enumeration includes the following members: DirectionRightToLeft (draws text going from right to left), DirectionVertical (draws text vertically rather than horizontally), DisplayFormatControl (displays special control characters with a special mark) and NoWrap (disables automatic wrapping when drawing the string in a rectangle). As with the FontStyle enumeration we examined before, we can combine multiple items when defining the FormatFlags property of a StringFormat object. We'll see some more members of this enumeration in action later in our code, and explain their uses there.

The System.Drawing.Text namespace

For the sake of completion, we shall take a brief look at the functionality the System.Drawing.Text namespace provides as it concerns text printing in the .NET framework, though we won't heavily rely on it in this case study. This namespace provides two classes, which represent font collections. The very useful InstalledFontCollection class, when initialized, automatically collects all the fonts that are available on the computer that is running the program. It therefore exposes a quick and easy way for the programmer to see which fonts he can rely on. We'll examine later how it is used. The other class is named PrivateFontCollection and keeps a list of fonts that were privately deployed with the application. A programmer can choose to include some fonts with his application so that he knows they will always be available. When installed on the target computer along with the application, these fonts do not in any way interfere with already installed fonts (even ones with the same name!) and are available through this private collection, after the programmer adds them to the collection by means of PrivateFontCollection 's AddFontFile() method. This method accepts a string that is the full path to the font file (e.g. "C:\Programs\MyApplication\Fonts\Arial.rtf"). Of course, the setup project used to install the whole application on the target computer would make sure that these .rtf files are installed in the appropriate directories. Neat as it may sound, this is not very commonly used as an everyday practice by developers, who prefer to stick to well-known fonts or even better, check for available fonts on the target computer and pick one that suits them. There are some cases, however, where it can save the day. Imagine, for example, a mathematical application of some kind that would need to display special symbols available in some font family not typically found installed in all computers. The third and final class in the System.Drawing.Text namespace is used as a base from both other classes and provides general functionality as a font collection - appropriately named FontCollection.

Inside this namespace there's also a useful enumeration whose items are used as possible values for the TextRenderingHint property of Graphics objects, and define the quality of text rendering.

Designing our program infrastructure

As with any application, before we actually start coding, it is absolutely vital to think about the guidelines our program will follow. Our first step is to consider in general what the application will do, and then find the data types and the operations that will be performed on them for the program to complete its purpose.

We already outlined the purposes of our program: we want to create a component with enhanced visual representation - specifically, methods to print text of various fonts, neatly organized in lines and columns. So, how should we start? Do we design it from scratch, or use an existing control provided by the framework? The latter choice is obviously the best way - we don't want to reinvent the wheel, since all the necessary functionality exists in the abstract classes. These classes contain general functionality for many kinds of objects - we typically derive new classes from them to add the specific functionality we need. There are some classes in the System.Windows.Forms namespace like these, which could be of much use here. The Control class is the base class for any component with a visual representation that can be displayed on a windows form. Just by inheriting from this class, and without adding a single line of code, we have a control ready to be added on a windows form (or other container control, information on this will follow shortly) with some standard properties ready to be set (like client area width and height or background color). There are also two interesting generic classes that inherit the Control class. ScrollableControl adds the support of horizontal and vertical scrollbars, used if the control's data can't fit into its dimensions, which is useful to us as the text to be displayed on our control will often become many lines or columns wide. The ContainerControl class adds further support for child controls. Forms are a classic example of container controls. Then there is the UserControl class. It extends the ContainerControl class and is widely used for creating new custom controls. We'll see later how to choose from the above three choices (ScrollableControl, ContainerControl and UserControl classes), or when we might need to consider other, more specific classes.

Implementing the LineInfo and SentenceInfo classes

With all of the above in mind, it's obvious that the control needs to keep in memory the text that it is displaying. Since that text is logically divided in lines, and we want to treat lines separately (perhaps deleting them as a whole or inserting new ones), it seems sensible to create an array of some sort as a member to keep information about each line. Since we will want to dynamically add and remove lines as needed, that array should be declared as being of type System.Collections.ArrayList, one of the flexible collections types introduced with the .NET framework.

So, what would this ArrayList contains as members? Our first approach would probably be to imagine that a line only contains the text string to be displayed, so we would just add strings to the line's ArrayList. That's not sufficient, however, as characters can be drawn using countless combinations of fonts and colors. That information must be kept on the line itself. As we shouldn't store this information separately for each character (that would be a huge waste of the system's memory), we can instead logically divide the text string contained in a line into smaller strings, with each one of those strings having a specific style (font and brush). We'll call these smaller strings "sentences" in the rest of this case study, to better distinguish them. That way, each line might contain anything from one to quite a few sentences, depending on whether there are many different text styles on the line. As the user won't, in normal circumstances, change the font used every few characters, we won't waste memory storage as we'll be grouping many characters into a single structure.

We should now go on and create custom classes that will contain all the necessary information on lines and sentences (LineInfo and SentenceInfo, respectively). These classes will be simple, and one might assume that it would be better to display them as simple structs. However, instances of these user-defined types will be contained in the ArrayList, which only contains reference types, and as structs are value types, it's much more efficient to declare the types as classes from the beginning to avoid boxing and unboxing operations. Let's start from the ground up, examining the SentenceInfo class first:

using System;
using
            System.Drawing;

namespace TextControlsLibrary
{
public
            class SentenceInfo
{
internal string strText;
internal Font
            fntTextFont;
internal Brush brTextBrush;

internal int X;
internal int intTextWidth;

public SentenceInfo(string
            text, Font textfont, Brush textbrush, int start)
{
// We don't
            want to create empty sentences. Throw an exception if some
//
            code tries to do that.
if (text == "")
throw new
            ArgumentException("Text can't be null");

this.strText =
            text;
this.X = start;

this.fntTextFont =
            textfont;
this.brTextBrush = textbrush;

}

public
            SentenceInfo(string text, int start)
{
if (text ==
            "")
throw new ArgumentException("Text can't be
            null");

this.strText = text;
this.X = start;

//
            Using default fonts and brushes
Font textfont = new
            Font("Tahoma", 8);
Brush textbrush =
            Brushes.Black;

this.fntTextFont =
            textfont;
this.brTextBrush =
textbrush;
}
}
}

Things are pretty standard here. We start with the usual namespace declarations (we'll be using the Font and Brush classes from System.Drawing). After that, we declare our own namespace to use with our project, and name it TextControlsLibrary. Inside it, we start construction of our first class. SentenceInfo will obviously contain the text of the sentence as a string (in its member variable strText), as well as the Font and Brush objects that this string will be painted with (fntTextFont and brTextBrush). The next member, the integer X, will keep the column number in which this particular sentence starts in the line. One could assume that each sentence starts where the previous one left off, but doing things this way we make the application more flexible, enabling the user to immediately print into whichever column he wants, even inserting text in between other sentences. The last member of the class is the integer intTextWidth. This keeps the width of the sentence in pixels, and will prove extremely useful later, as we shall find out. Note that all member variables are declared internal, as they will need to be set by another class in the assembly (we'll see how that happens later).

Finally, there is a constructor for our SentenceInfo class, which accepts four constructors to initialize the appropriate members. As you can see, we can instantiate a SentenceInfo class without defining font or brush (by using the alternate constructor) - a default font and brush will be used. The intTextWidth property is not initialized here - we'll see how to set its value later.

Let's move on to the LineInfo class now. It is implemented in code starting like this:

using System;
using
            System.Collections;
using System.Drawing;

namespace
            TextControlsLibrary
{
public class LineInfo
{
public int
            intHeightInPixels;
public int intTop;
public ArrayList
            allSentences;

public LineInfo(string text, Font textfont,
            Brush textbrush)
{

allSentences = new
            ArrayList();

if (text != "")
{
SentenceInfo si = new
            SentenceInfo(text, textfont, textbrush,
            0);
allSentences.Add(si);
}

this.intHeightInPixels =
            textfont.Height;
}

public LineInfo(string
            text)
{
allSentences = new ArrayList();

// Using
            default fonts and brushes
Font textfont = new Font("Tahoma",
            8);
Brush textbrush = Brushes.Black;

if (text !=
            "")
{
SentenceInfo si = new SentenceInfo(text, textfont,
            textbrush,
            0);
allSentences.Add(si);
}

this.intHeightInPixels =
            textfont.Height;
}

As LineInfo will contain information for a line, its most important member is the ArrayList allSentences, where objects of type SentenceInfo will be kept. The intTop member will store the distance in pixels of the line from where the document starts. We'll use it later, when we decide where to start the redrawing from when adding new text. There's also the intHeightInPixels member. That will tell us how high, in pixels, the line is, and will always be equal to the height of the largest font used in a sentence contained in the line. The analogy to SentenceInfo 's intTextWidth member is obvious. If you are wondering why we didn't also put a height property for the text into the SentenceInfo class, that's because we are not interested in how high each sentence is, but instead what the highest sentence's (in the whole line) height is. That way, we can calculate where the next lines begin. Even if we have only one very big letter in a line, the line's height will be equal to that letter's height, despite the fact that all other letters are smaller. The following diagram, which represents a text line made up of three sentences, ought to make this concept clearer:

Those three members, the ArrayList, height counter, and distance counter, are really all the data we'll need to store in instances of our SentenceInfo class. In the code above we can also see the constructor of the class. The single most interesting line is the last one, where we get to initialize the intHeightInPixels member. We do that directly by calling the Height property of a Font object - it returns that font's maximum height, that is, the height of its largest character. In contrast to a font's width, its height can be calculated directly through this property. Keep in mind that there is also the `more specialized GetHeight() method, which accepts a Graphics object and returns the height in that device's measuring system. The Height property is sufficient for the default units we will be using, however.

Continuing the implementation of the LineInfo class, we must provide a way for the class to add or insert a sentence. The code that accomplishes this follows and completes the LineInfo class:

// Add a new sentence to the end of this line
public void AppendSentence(string text, int X, Font f, Brush b)
{
SentenceInfo si = new SentenceInfo(text, f, b, X);
AppendSentence(si);
}

public void AppendSentence(SentenceInfo si)
{
allSentences.Add(si);
if (si.fntTextFont.Height > this.intHeightInPixels)
this.intHeightInPixels = si.fntTextFont.Height;
}

// Insert a new sentence in between existing sentences in the
//line, according to the specified index.
public void InsertSentence(string text, int X, Font f, Brush b, int
index, int width)
{
SentenceInfo si = new SentenceInfo(text, f, b, X);
si.intTextWidth = width;
InsertSentence(si, index, width);
}

public void InsertSentence(SentenceInfo si, int index, int width)
{
si.intTextWidth = width;
allSentences.Insert(index, si);
if (si.fntTextFont.Height > this.intHeightInPixels)
this.intHeightInPixels = si.fntTextFont.Height;
}
}
}

Although we declared most members of the SentenceInfo class as internal, so that we can easily update them from inside the project one by one if we have to, we don't want to be inserting sentences by instantiating all their members one by one all the time. That's what the above methods take care of. There are actually two methods, with two overloads each. The AppendSentence() method adds the sentence to the end of the line, and it is used when there are no other sentences to the right of the new one. As it adds the sentence to the end of the list, this method uses the Add() method of the ArrayList. In contrast, the InsertSentence() is used to insert a sentence between existing sentences; therefore it utilizes the Insert() method of the ArrayList class. The Insert() methods we constructed also set the sentence's intTextWidth property, using the appropriate parameter.

So, wrapping things up, let's say that the control will contain a collection of type ArrayList that will contain information on all the lines it contains, storing objects of type LineInfo. Every LineInfo object in turn will contain another ArrayList member (allSentences), which will store objects of type SentenceInfo, so that it can keep track of all the sentences the line has. An object of type SentenceInfo will have as its members a text string (the actual text of the sentence), a Font and Brush that will be used to display the text, an integer (X) which represents the column number where the sentence begins to be displayed on the line, and an integer member to keep its length, in pixels. This all might sound too confusing in theory, but will become much clearer when we examine it in action, in the next section.

Measuring string sizes

Because of the very nature of our program, it is essential that we know exactly how wide and high every string we are displaying is. That's how we will be able to avoid inconveniences like overlapping strings. As far as string heights are concerned, things are very simple. As we already explained, Font objects have a Height property that returns the height, in pixels, of any text drawn with this specified font. The width of strings is, however, an entirely different story. The framework itself provides the MeasureString() method with quite a few overloads. This method returns a SizeF variable containing the measured size (width and height) of a specified string, drawn with the font entered as a parameter. SizeF is similar to Size, only its width and height members are expressed as float numbers, not integers.

So, since the MeasureString() method returns the width and height of the string in float numbers, it must be very precise, right? Not really. In fact, if text fonts are not drawn with an anti-aliasing effect (more on this later), the MeasureString() method returns a larger width than the actual width of the text when drawn on the screen. This small deviation may be acceptable for other kinds of applications, but not for the one we are designing. On our text-edit implementation of the control later, for example, we don't want the users to click on a character to place the cursor there, only to find out that the cursor has been placed two characters earlier. Take my word for it; they won't care much for "It's the framework's fault"-type excuses!

More on the limitations of the MeasureString() method (and why they exist) can be found through the provided links at the end of the case study. In our project, we'll stick to Microsoft's recommendations when it's absolutely essential to measure the exact width of strings, when they are rendered on the screen with the DrawString() method. These recommendations suggest that we should:

  • Use a StringFormat object based on the typographic StringFormat (GenericTypographic), and always pass it to both the MeasureString() and DrawString() methods.
  • Set the Graphics TextRenderingHint property to TextRenderingHintAntiAlias.

Provided we follow the above guidelines when coding, the MeasureString() method will always return the exact size of the string, when it is drawn using the DrawString() method.

The TextDisplayControl control

It's time to start putting our project together. If you are using Visual Studio .NET, you should create a new C# Class Library project. As the standard namespace we'll be creating is named TextControlsLibrary, you'll probably want to name the project with the same name.

Firstly, we add to the project two class files, LineInfo.cs and SentenceInfo.cs, and insert the code for the two classes we examined above. With the definitions of these helper classes in place, we are ready to add the main control - let's name it TextDisplayControl.

From what we've already discussed in the previous section, it's rather obvious that we have some choice on the base class for our control. ScrollableControl is a fine choice, although we could make the whole application more scalable by directly implementing the ContainerControl class (which is lower in the hierarchy - that way we would be able to add extra controls into our own). Other options are also available: by directly inheriting the Form class, we can extend its methods and create extended windows that can print text directly on their client area. These options are interesting, and remember that our selection is not binding at this moment: we can always go back in the code and just change the base class of our control. However, UserControl seems like an excellent choice to go with: it gives us the scrolling features we need, and can be a container for other controls (though we won't add new controls in this case study, it's a good investment now so that we can extend the class in the future - that's the whole point of scalability, and we have to think from now of what features might be needed in the future even if we won't use them immediately). Besides, Visual Studio .NET provides excellent design-time support for user controls, which could save us a lot of time when setting various parameters like visual properties. We can use Visual Studio .NET to create a new control that inherits UserControl. Go to the solution explorer, right click on the project name in the solution explorer and select Add -> User Control. Name the new class file TextDisplayControl.cs - TextDisplayControl will be the name for our control. Our class library project automatically references the System.Windows.Forms and System.Drawing namespaces immediately after we add the user control.

Thinking about the information that will need to be kept in our control, we quickly realize that there should definitely be an ArrayList to keep objects of type LineInfo for each line of text being displayed by the control. We might also want to provide the functionality for some small extra features that applications will find useful. These will be a cursor being displayed at the position of the text entered, and the appearance of a line counter at the start of each line. Both these features can be turned on and off by the user, so we should also add two Boolean properties that take care of this: blShowLineNumbers and blDrawCursor. Another similar feature is line wrapping, which we will examine later. We should however add another Boolean property for the user to indicate whether he wants the text to be wrapped or horizontal scrollbars to be displayed when the string doesn't fit the line.

We can now start examining the first lines of code for the control:

using System;
using System.Collections;
using System.ComponentModel;
using System.Drawing;
using System.Data;
using System.Windows.Forms;

namespace TextControlsLibrary
{
public class TextDisplayControl : System.Windows.Forms.UserControl
{
/// <summary>
/// Required designer variable.
/// </summary>
private System.ComponentModel.Container components = null;

private int intHeightCounter;
private int intWidthCounter;

private const int intDefaultLineHeight = 15;

protected ArrayList allLines;

protected bool _wraptext = false;
protected int intMaxWidth = 0;
protected bool blDrawCursor = false;
protected int intCursorWidth = 1;

public bool blShowLineNumbers = false;

// Font and brush to use when displaying line numbers
protected Font fntLineNumbers = new Font("Tahoma", 8);
protected Brush brLineNumbers = Brushes.Red;

protected Font fntDefaultFont = new Font("Tahoma", 8);
protected Brush brDefaultBrush = Brushes.Red;

protected StringFormat sfTextDisplayFormat =
StringFormat.GenericTypographic;

public bool blWrapText
{
get
{
return _wraptext;
}
set
{
_wraptext = value;
// Zero out the height of all lines, so that they will be correctly
//recalculated
foreach(LineInfo li in allLines)
li.intHeightInPixels = 0;
Invalidate();
}
}

private int _x; // Current column number
private int _y; // Current line number
// Cursor positions, in line/column numbers
public int cX
{
get
{
return _x;
}
set
{
if (value < 0)
throw new ArgumentException("X can't be less than 0");
else
_x = value;
}
}

public int cY
{
get
{
return _y;
}
set
{
if (value < 0)
throw new ArgumentException("Y can't be less than 0");
else
_y = value;
}
}

// Cursor positions, in pixels
protected int cXp;
protected int cYp;

public TextDisplayControl()
{
InitializeComponent();
allLines = new ArrayList();

sfTextDisplayFormat.Trimming = StringTrimming.Word;
sfTextDisplayFormat.FormatFlags =
StringFormatFlags.MeasureTrailingSpaces;
}

Here we declare the first member variables we'll obviously need to start with (there will be more coming afterwards). Ignoring the components member that was added by Visual Studio, there are the intHeightCounter and intWidthCounter integers that are private to the class and will be constantly updated to reflect the horizontal and vertical drawing position (in pixels). We'll explain more about them later. The intDefaultLineHeight constant reflects how many pixels high the empty lines will be (lines that actually contain text will have various heights, depending on the height of the characters entered). The intMaxWidth reflects the length of the longest line, so that we can later set the length of the horizontal scrollbars if it doesn't fit in the control. There's also an ArrayList that will keep all the LineInfo objects for the document as we already explained, and a StringFormat object that will be used to keep information on text display. Finally, we provide default fonts and brushes, in case the user doesn't specify his own, and marked them as public so that they can be set from external applications. In the class constructor, we initialize the variables and also set the StringFormat's FormatFlags variable to MeasureTrailingSpaces. It's necessary to do that if we want the MeasureString() method to take into account any spaces at the end of the string, as it doesn't do that by default.

As far as the cursor is concerned, we have defined in total four variables. The first two, cX and cY, keep the position of the cursor in the column and line numbers respectively. However, it will be useful to know what the actual position of the cursor is each time in screen coordinates (pixels) - that's why the cXp and cYp protected variables exist. We will be using these "exact" coordinates later on, when we will need to find where to draw the cursor's rectangle, as well as to discover the exact location of the screen that needs to be updated.

Two methods contain almost all the meat for our control. The first one is the override of the OnPaint() method, inherited from the Control class, called every time our control or part of it requires redrawing. The other one is the Print() method, which as its name implies, prints text strings anywhere on the client area. It accepts line and column numbers, as well as the text and the accompanying font and brushes used to render it. We will start by examining the Print() method. This method is logically divided into two parts. In the first part, we find where exactly in the specified line the string should be placed, and add or insert it into the allSentences collection of the line. The code for this first part of the method is as follows:

public void Print(string text, int X, int Y,
            Font f, Brush b)
{
// Create new, blank lines if
            necessary
while (allLines.Count <= Y)
{
allLines.Add(new
            LineInfo(""));
}

Graphics dc =
            this.CreateGraphics();
dc.TextRenderingHint =
            System.Drawing.Text.TextRenderingHint.AntiAlias;

LineInfo li
            = (LineInfo)allLines[Y];

int columnCounter = 0;
bool
            overlaps = false;

int sn = 0;

foreach(SentenceInfo si
            in li.allSentences)
{
columnCounter +=
            si.strText.Length;
if (columnCounter > X)
{
overlaps =
            true;
sn =
            li.allSentences.IndexOf(si);

break;
}
}

if
            (overlaps)
{
// Get the sentence that will be seperated by the
            new text
SentenceInfo si =
            (SentenceInfo)li.allSentences[sn];
if
            ((si.fntTextFont.Equals(f))&&(si.brTextBrush.Equals(b)))
{
//
            If the new sentence has its font and brush values same as the
//
            one near it, we merge the two sentences into one...
int a = X -
            si.X;
if (a < 0)
a = 0;

si.strText =
            si.strText.Insert(a, text);

si.intTextWidth =
            (int)dc.MeasureString(si.strText, si.fntTextFont, new PointF(0,0),
sfTextDisplayFormat).Width;
}
else
{
//
            ...otherwise, we split the first sentence and put the new one
//
            in between, effectively creating three sentences.
int relativeCol
            = X - si.X;
if (relativeCol < 0) relativeCol =
            0;

string rest = si.strText.Substring(relativeCol);
si.strText = si.strText.Remove(relativeCol, si.strText.Length -
            relativeCol);
si.intTextWidth =
            (int)dc.MeasureString(si.strText, si.fntTextFont, new
            PointF(0,0),
sfTextDisplayFormat).Width;

SentenceInfo
            newSentence = new SentenceInfo(text, f, b,
            X);

SentenceInfo oldSentenceRest = new SentenceInfo(rest,
            si.fntTextFont,
si.brTextBrush, X +
            text.Length);

li.InsertSentence(newSentence, sn + 1,
            (int)dc.MeasureString(newSentence.strText,
newSentence.fntTextFont,
            new Point(0,0),
            sfTextDisplayFormat).Width);
li.InsertSentence(oldSentenceRest,
            sn + 2,
(int)dc.MeasureString(oldSentenceRest.strText,
            oldSentenceRest.fntTextFont,
new Point(0,0),
            sfTextDisplayFormat).Width);

if (si.strText ==
            "")
{
li.allSentences.Remove(si);
}
}
}
else
{
// Insert blank spaces into the string if
            necessary
while (columnCounter < X)
{
text = " " +
            text;
columnCounter++;
}

bool appendTextToLastSentence
            = false;

if (li.allSentences.Count >
            0)
{
SentenceInfo last =
            (SentenceInfo)li.allSentences[li.allSentences.Count - 1];
if
            (last.fntTextFont.Equals(f) &&
            last.brTextBrush.Equals(b))
appendTextToLastSentence =
            true;
}

// If the new sentence's font and/or brush is
            different than the last
// sentence's, create new sentence,
            otherwise add its text into the last one.
if
            (appendTextToLastSentence)
{
SentenceInfo last =
            (SentenceInfo)li.allSentences[li.allSentences.Count -
            1];
last.strText = last.strText + text;

last.intTextWidth
            = (int)dc.MeasureString(last.strText, last.fntTextFont,
new
            Point(0,0),
            sfTextDisplayFormat).Width;
}
else
{
SentenceInfo news =
            new SentenceInfo(text, f, b,
            columnCounter);

news.intTextWidth =
            (int)dc.MeasureString(text, f, new
            PointF(0,0),
sfTextDisplayFormat).Width;

li.AppendSentence(news);
}
}

As you can see from the code above, before doing anything we have to make sure that the line specified by the Y variable actually exists. If it doesn't (the allLines.Count property is less than or equal to Y), we add blank lines until we have created the one we need. For example, if we start with an empty document and the user wants to print something on the fifth line, we create five blank lines, and then fill the fifth line with whatever text was specified. If we try to print on a line that does not exist, an error will occur because the allLines ArrayList will not contain a member with an index equal to that line's number. Afterwards, we create a Graphics object calling the CreateGraphics() method of our control. We already explained why we should set this object's TextRenderingHint parameter to AntiAlias -because doing things this way means that the MeasureString() method will be accurate.

From that point onward, we have to check if the X parameter specified makes the sentence fall in between existing sentence - if that happens, we create three new sentences (the first part of the existing sentence, the new sentence we are adding, and the second part of the existing sentence) and update them with the line's allSentences collection. If on the other hand no overlapping occurs, we just append the sentence to the end of the previous sentences, adding spaces if necessary until we reach the specified column number. Note that we do go to the trouble of checking if the new text's font and brush equals the respective values of nearby sentences. If they do, we can merge the sentences into just one by just updating the existing sentence's text. That way we avoid creating new SentenceInfo objects unnecessarily, conserving system memory.

The second part of the Print() method, examined below, calculates what area of the control's client surface will need to be redrawn, and calls the Invalidate() method on it so that the redrawing will immediately take place. As you probably know if you've had any experience with GDI+, the Invalidate() method is used to send a message to the controls to redraw themselves, in whole or just a part of them. Mark the following as a golden rule when designing GDI+ applications: use this method with caution and always specify the absolutely essential region where the redrawing needs to take place. It's an unconceivable waste of resources to start drawing everything from the beginning, when all you are trying to do is add a letter of text. Besides, if you are doing any unnecessary redrawing, it will be visible when the application runs, as the old shapes will instantly disappear and the new identical ones will be drawn in the exact same places. That gives the user a "clipping" effect, and is an indication of poor programming habits. Therefore, you should always check how your application handles redrawing if you are experiencing clipping effects when testing the code, and try to find a smaller area to update. This won't always be easy or even possible, and there are cases where the developer just has to put up with the clipping effect in specific cases in his program.

That's what we are doing in the code above. After doing the print, we want to update only the part of the client area that we need to. However, that part isn't very easy to spot, because it might be greater than just the rectangle surrounding the newly inserted text. There might be text to the right of the one being entered, and it will have to move further to the right as a result of the insertion. If we are writing with a new, larger font on the line, we'll be changing the whole line's height and therefore it will need to be redrawn as a whole - and any lines below it will also need to be updated, as they will be drawn in a lower position.

The only sure thing is that no redrawing will have to happen above the area the new text was printed, but we have to check for all other cases (left, right, and below). Take a look in how this is accomplished, in code, in the rest of the Print() method:

bool redrawLine =
            false;

foreach(SentenceInfo si in li.allSentences)
{
//
            Redraw the whole line if there are characters with a font
            height
// smaller than the one we are entering at the left of the
            new sentence.
if (si.fntTextFont.Height < f.Height &&
            si.X < X)
redrawLine = true;

// Update X properties for
            any sentences right of the overwritten one,
// to reflect text
            shifting to the right.
if (si.X > X)
si.X +=
            text.Length;
}

if (!redrawLine &&
            li.allSentences.Count == 1)
{
// Make sure to redraw the line
            if this is the first character entered
SentenceInfo si1 =
            (SentenceInfo)li.allSentences[0];
if (si1.strText.Length ==
            1)
redrawLine = true;
}

// Calculate and redraw the
            clipping (invalid) areas

if (blWrapText)
{
// Redraw
            all the area, from the height of the cursor's position and
            below
Rectangle invalidRectangle = new Rectangle
(0, cYp +
            AutoScrollPosition.Y, Width,
            Height);
Invalidate(invalidRectangle);
}
else
{
if
            (redrawLine)
{
RedrawLine(Y);
if (allLines.Count - 1 >
            Y)
{
LineInfo nextLine = (LineInfo)allLines[Y + 1];
//
            There are more sentences below this one. Redraw them.
Rectangle
            invalidRectangle = new Rectangle
(0, nextLine.intTop, Width,
            Height);
Invalidate(invalidRectangle);
}
}
else
{
Rectangle
            invalidRectangle = new Rectangle
(cXp - AutoScrollPosition.X, cYp
            - AutoScrollPosition.Y,
Width - cXp + AutoScrollPosition.X,
            li.intHeightInPixels);

Invalidate(invalidRectangle);
}
}
}

First, we check if there is any text in the line before the one we are entering that has a smaller height than that of the new font. If there is, we have to redraw the whole line as these smaller letters will be drawn in a lower position (as we shall explain very soon, small fonts are displayed towards the bottom of the line, and since we added text of a bigger height, the line's height has also changed). If we decided that the whole line should be redrawn, we call the RedrawLine() method (explained later) on it, and also check if there are any new lines below the one we are currently in. If there are, we redraw them as well.

If word wrapping is enabled for the control, we count on the cursor's vertical position to update all the parts of the screen from that point downwards.

Finally, let's provide another overload of the Print method that does the same work, but won't accept any font or brush information - it just goes ahead and uses the default ones:

public void Print(string text, int X, int Y)
{
// Call the other Print overload, using default values for font and brush
Print(text, X, Y, fntDefaultFont, brDefaultBrush);
}

We still need to do the code for the RedrawLine() method, used in the second part of the Print() method. As we already said, this method redraws all the characters in one line only. Here's the simple code that implements the method:

public void RedrawLine(int index)
{
LineInfo li = (LineInfo)allLines[index];
Rectangle invalidRectangle =
new Rectangle(0 + AutoScrollPosition.X, li.intTop + AutoScrollPosition.Y,
this.Width, li.intHeightInPixels);
Invalidate(invalidRectangle);
}

Our OnPaint() override is also quite complex. It displays all the sentences contained in our document in their appropriate lines, taking into consideration all the fonts used, automatically adjusting line width and heights. It is also necessary to wrap the text and display line numbers, if the respective control properties (blWrapText and blShowLineNumbers) have been set to true.

Let's go over this method step by step:

protected override void
            OnPaint(PaintEventArgs e)
{
bool cursorDrawn =
            false;
intHeightCounter = 0;

Graphics dc =
            e.Graphics;
dc.TextRenderingHint =
            System.Drawing.Text.TextRenderingHint.AntiAlias;

// Display
            every sentence kept in the each line of the
// document, using
            its appropriate font, size and color
foreach(LineInfo li in
            allLines)
{
// Update the line's intTop property
li.intTop
            = intHeightCounter;

int lheight = li.intHeightInPixels;
if
            (lheight == 0) lheight = intDefaultLineHeight;
int scroll_y =
            (-1)*(AutoScrollPosition.Y);
if ((intHeightCounter + lheight)
            < scroll_y
||(intHeightCounter > scroll_y +
            this.Height))
{
// If this sentence is not in the client area
            of the control that is visible
//(as a result of scrollbar
            positions), we don't draw it.
intHeightCounter +=
            lheight;
continue;
}

intWidthCounter = 0;

int
            lnlength = 0;

// Draw lines number
if
            (blShowLineNumbers)
{
string lnumber = (allLines.IndexOf(li) +
            1).ToString() + ". ";
lnlength = 35;

Rectangle R = new
            Rectangle(intWidthCounter + AutoScrollPosition.X,
intHeightCounter + AutoScrollPosition.Y, 35,
            fntLineNumbers.Height);

dc.DrawString(lnumber,
            fntLineNumbers,
            (allLines.IndexOf(li)==cY)?Brushes.Green:brLineNumbers,
            R);

intWidthCounter += lnlength;
}

int lplus = 0;
int l = 0;

foreach(SentenceInfo si in
            li.allSentences)
{
if (blWrapText)
{
// Code for this
            part is examined in the next section
}
else
{
RectangleF
            rect = new RectangleF(intWidthCounter + AutoScrollPosition.X,
intHeightCounter + AutoScrollPosition.Y + (li.intHeightInPixels
            - si.fntTextFont.Height),
si.intTextWidth + 1,
            si.fntTextFont.Height);

dc.DrawString(si.strText,
            si.fntTextFont, si.brTextBrush, rect,
            sfTextDisplayFormat);

if (!cursorDrawn &&
            blDrawCursor)
{
cursorDrawn = PrintCursor(dc, li, si,
            intWidthCounter, 0);
}
intWidthCounter +=
            si.intTextWidth;

if (li.intHeightInPixels <
            si.fntTextFont.Height)
li.intHeightInPixels =
            si.fntTextFont.Height;
}
}

intHeightCounter +=
            (li.intHeightInPixels==0) ? intDefaultLineHeight :
            li.intHeightInPixels;

if (intMaxWidth < intWidthCounter)
            intMaxWidth = intWidthCounter;
}

this.AutoScrollMinSize =
            new Size((blWrapText) ? 0 : intMaxWidth,
            intHeightCounter);

base.OnPaint(e);
}

We haven't yet added any display code if there is text wrapping in the document. This is because the code needs some special explanation, which is provided in the next section.

The cursorDrawn Boolean variable is initialized to false - we change it after we have drawn the cursor. We start by setting the intHeightCounter to 0 - the variable will be incremented as we start adding text. Remember, intHeightCounter will tell us how high from the top of the screen the next drawing will be, and intWidthCounter contains similar information, only it's for horizontal coordinates. Both variables keep actual pixels, not line or column numbers. Continuing in our method, we obtain the Graphics object from the provided paint event arguments, and as explained before, set its TextRenderingHint property to the AntiAlias member of the TextRenderingHint enumeration, found in System.Drawing.Text. This will give our text a smooth look, as well as make the MeasureString() method accurate. Afterwards, we proceed to our first of two loops that iterate through the entire collection of lines and sentences. For each line, we first set its height equal to the current value of intHeightCounter, and then determine if it fits the area currently viewable in the control. If it does not, we simply update the intHeightCounter with that line's height and move on to the next line, not trying to draw any of its sentences. We would waste resources if we tried to - the user would never see the line anyway, since it is either below or above the area of the control's visible client area.

Note here that controls inherited directly or indirectly from the class ScrollableControl, like the one we are creating here, provide the parameters AutoScrollPosition (the point defining the position of the scrollbars) and AutoScrollMinSize (the size that defines the minimum size for which auto-scroll facilities are provided) that we'll use extensively throughout the rest of the project. We can find the horizontal and vertical coordinates of the scrollbars by looking at the X and Y properties of AutoScrollPosition, and if we need a larger area to be covered by scrollbars (for example, after printing a text string outside of the previous bounds), we have to pass the new size to AutoScrollMinSize.

Continuing with our code (and ignoring the lplus and l variables for now, they will be used for the text-wrap version we examine later but had to be declared there), if the line is indeed placed inside the visible part of the control, we go into a new loop and take each of its sentences and draw it on the screen using the DrawString() method. We do that after calculating the rectangle that includes the area of the drawing. Note that in this rectangle's X and Y starting positions we take into account the positions of the scrollbars, and moreover, in the Y vertical position, we add to the current position in pixels the height of the line minus the height of the current font. By doing that, smaller fonts appear at the bottom of the line when there are larger fonts in the line, not at the top. This is shown in the picture below. We get an effect similar to the one on the right, and avoid the effect shown on the left:

Note that there's a call to a PrintCursor() method if the cursor has not been drawn and the blDrawCursor parameter has been set to true. We'll examine this method in the next section, as it will also apply to cursor drawing in text-wrap mode.

After all this has been done, we update the intWidthCounter and check if the line's intHeightInPixels property also needs to be updated. After going through the process for each sentence, we update the intHeightCounter property, check if the intMaxWidth needs updating, and proceed to the next line. When we're also done with all the lines, we update the positions of the scrollbars using the AutoScrollMinSize property mentioned earlier. If the text wrap option is on, we don't want any horizontal scrollbars and thus set the specified size's width to 0.

In the end, we shouldn't forget to add a call to the base class' OnPaint() method, as the control might have some redrawing of its own to do. It's good practice to do so when overriding the OnPaint() method, although we are not expecting the base class (UserControl) to actually draw something on top of our own output - it is not designed to do so. That will not be the case when we are overriding this method again, in new classes that override this control, or when we have added other controls into our own. There we will have to make sure, in code, that no drawings will happen in the same client area, otherwise the result will probably be something different to what we were intending (in most cases, our text will be covered by the drawings the other controls will perform). We'll mention in the last paragraph of this article a way to reserve client area space for other controls to draw into.

Finally, we make the control class more complete by adding some general methods that will be used later. The code for them is simple. Below you can see the methods that clear the whole document, remove a particular line and insert or append a new line:

// Completely deletes the control's
            text
public virtual void
            Clear()
{
allLines.Clear();
intMaxWidth = 0;
cXp =
            0;
cYp = 0;
cX = 0;
cY = 0;
Invalidate();
}

//
            Completely deletes a line. Shifts up all remaining lines.
public
            void DeleteLine(int lineNumber)
{
try
{
LineInfo li =
            (LineInfo)allLines[lineNumber];
allLines.Remove(li);

Rectangle
            invalidRectangle = new Rectangle(0, li.intTop,
this.Width,
            this.Height -
            li.intTop);
Invalidate(invalidRectangle);
}
catch
            (ArgumentOutOfRangeException)
{
// Ignore the request to
            delete a line if it does not exist
}
}

// Insert a new
            line of text between existing lines, at the specified
            index
public void InsertLine(int index)
{
LineInfo lold =
            (LineInfo)allLines[index - 1];
int top =
            lold.intTop;
allLines.Insert(index, new LineInfo("",
            fntDefaultFont, brDefaultBrush));
Rectangle invalidRectangle =
            new Rectangle(0, top, Width,
            Height);
Invalidate(invalidRectangle);
}

public void
            InsertLine(int index, LineInfo li)
{
LineInfo lold =
            (LineInfo)allLines[index - 1];
int top =
            lold.intTop;
allLines.Insert(index, li);
Rectangle
            invalidRectangle = new Rectangle(0, top, Width,
            Height);
Invalidate(invalidRectangle);
}

public void
            InsertLine(int index, SentenceInfo si)
{
LineInfo lold =
            (LineInfo)allLines[index - 1];
int top = lold.intTop;
LineInfo
            li = new LineInfo(si.strText, si.fntTextFont,
            si.brTextBrush);
allLines.Insert(index, li);
Rectangle
            invalidRectangle = new Rectangle(0, top, Width,
            Height);
Invalidate(invalidRectangle);
}

public void
            AddLine(string text, Font f, Brush b)
{
LineInfo
            newli;

newli = new LineInfo(text, f,
            b);

allLines.Add(newli);
}

Implementing Line Wrapping

We've now reached perhaps the trickiest part of the application. Creating automatic text wrapping for the control we are creating is quite a complex task. GDI+ does provide some useful methods for automatic text wrapping that we will count on, but these are not sufficient because they were designed to deal with text of a specific font. Therefore, we'll do things by hand, going through each sentence in our text (remember, we have defined a "sentence" as a sequence of text that has the same attributes - fonts, colors, etc).

There's an interesting overload of the MeasureString() method that we will find useful. Its signature goes like this:

public SizeF MeasureString (string text, Font font, SizeF layoutArea,
StringFormat stringFormat, out int charactersFitted, out int linesFilled);

The first two parameters are self-explanatory. There's the actual text, passed as a string, and the Font it will be rendered with. The parameter of type SizeF is a size inside which the measurement will take place. If in the passed StringFormat parameter we define that the string should be trimmed to the nearest word (and that's what we'll actually do in our code), then the calculations inside the specified size will take this into account and provide information on the wrapped text through the last two integer output parameters. If it has been quite some time since you last checked on C# syntax, remember that output parameters are like extra return values: their values when the call is made are ignored, and they are filled with new values from the method itself when it is called. Here, the MeasureString() method calculates how the passed string, when drawn with the font and format specified, fits into the size layoutArea. More specifically, charactersFitted is set to the exact number of characters that made it into the layoutArea (if the whole string fitted, then charactersFitted equals the length of the string). linesFilled, on the other hand, tells us how many lines the string was wrapped into as a result of the size's width. Text wrapping is calculated automatically, and depends on the StringFormat parameter's StringTrimming property.

Note, however, that if the size's height is equal to the font's height and its width is too small to fit the string, then the linesFilled will still have a value of 1, because the other wrapped lines would never make it into the specified size. Take a look at the example diagram below to understand how it works. The blue rectangle represents the size passed to the method. Below it we can see the values of the two integer output parameters after the method is called. Notice that strings are wrapped to the nearest word; we accomplish that by passing a StringFormat object with its Trimming property set to Trimming.Word :

In the last case, extra lines of the string are calculated even if they only partially fit into the area available. This happens by default. If you want these lines omitted, use a StringFormat object and supply it with the StringFormatFlags.LineLimit flag.

Usually developers use only one of the charactersFitted and linesFilled parameters, the one that best fits their given problem. We'll be using the charactersFitted parameter.

If every line of text in our control was made up of only one font, it would be easy to implement text wrapping from what we've seen so far. As we've already explained, there are DrawString() overloads that accept Rectangle objects as parameters, and depending on the specified stringFormat may automatically wrap text in various ways. Unfortunately, our control is more complex than that, and supports multiple fonts in every line. These fonts will, in general, have mixed widths and heights. Therefore, we can't specify a rectangle for every given - for example, a sentence may be starting from a point inside the line, but when wrapped, the rest of it must start from point 0 of the next line (we could find a Region object that would precisely include our text, but the DrawString() method doesn't accept Region objects to wrap text with. There are some things the developer just has to do by himself!).

So, we'll do things step-by-step. Let's take the more general case: we have already drawn some sentences on the line, and we want to find out how the next sentence will need to be drawn and wrapped if necessary. Take a look at the following shape (the black rectangle represents the line's boundaries -pay special attention to the line's width as it is fixed, and there's where the wrap will take place later):

We need to place our new sentence just where the last one left off - that width is kept in the intWidthCounter variable. So, we declare a rectangle which contains the remaining physical "line" as shown on the screen (don't confuse that with the document's lines, here we are actually dealing with only one document line that may be split into more screen lines as a result of text wrapping). This rectangle is shown in red in the following illustration:

We will fit the sentence we want to print in this red rectangle. In fact, that's not entirely true: what we will do is fill whatever part of the sentence we want to print, and keep the remaining for later. We'll determine how many characters made it into the rectangle using the appropriate output parameter of the MeasureString() method, which we examined earlier. If the characters that fit the rectangle equal the width of the string we are printing, we are done with the sentence and can therefore move on to the next sentence after updating the intWidthCounter variable. However, if there are still characters left, we continue with the next physical line: by setting intWidthCounter to 0, we have a whole new line to fit the remaining part of the string. We must also update the intHeightCounter variable, to reflect the new height where the next "screen" line will begin. That additional height is, as you might have guessed, equal to the height of the tallest font in the previous "screen" line - it works the same way as normal document lines.

We continue this process of calculating new rectangles and fitting in whatever remains of the original string until we have no more characters remaining until we have printed the whole string in as many screen lines as are needed. By now you should be able to see clearly why we have to go through all of this trouble when there are already methods in the framework that wrap strings to rectangles - it's because the shapes we are wrapping our sentences to are multiple rectangles and so we can't define a single rectangle to pass to the DrawString() or MeasureString() methods. You can't find a single rectangle to fit the blue sentence in the next picture, for example:

We could easily find a rectangle to wrap the whole line (including the first sentence), but sentences in general are made up of different fonts, and the framework-provided methods only accept one font.

The code for text-display when the blWrapText property of the control is set to true follows, and demonstrates the techniques described above:

...
int lplus = 0;
int l = 0;

foreach(SentenceInfo si in li.allSentences)
{
if (blWrapText)
{

int layoutWidth, layoutHeight;
int charsFitted, linesFilled;
string remaining = si.strText;
int charsDisplayed = 0;

do
{
layoutWidth = this.Width - intWidthCounter;
layoutHeight = si.fntTextFont.Height;

if (lplus < si.fntTextFont.Height)
lplus = si.fntTextFont.Height;

int w = (int)dc.MeasureString(remaining, si.fntTextFont,
new SizeF((float)layoutWidth, (float)layoutHeight),
sfTextDisplayFormat, out charsFitted, out linesFilled).Width;

Rectangle rect = new Rectangle(intWidthCounter,
intHeightCounter + AutoScrollPosition.Y + l + lplus - si.fntTextFont.Height,
layoutWidth, layoutHeight);

intWidthCounter += w;

dc.DrawString(remaining, si.fntTextFont, si.brTextBrush, rect, sfTextDisplayFormat);

if (charsFitted < remaining.Length)
{
l += lplus;
lplus = si.fntTextFont.Height;

intWidthCounter = lnlength;

cXp = lnlength;
}

charsDisplayed += charsFitted;

if (!cursorDrawn && blDrawCursor)
{
SentenceInfo nsi = new SentenceInfo
(remaining.Substring(0, charsFitted), si.fntTextFont, si.brTextBrush, si.X +
charsDisplayed);

cursorDrawn = PrintCursor(dc, li, nsi, intWidthCounter, l);
}

remaining = remaining.Substring(charsFitted);

if (li.intHeightInPixels < lplus + l)
li.intHeightInPixels = lplus + l;
}
while (remaining.Length > 0);

}
else
{
...

Of course, nothing will need to change in the Print() method (or anywhere else in our code for that matter) as a result of text wrapping. Column and line numbers are unaffected by wrapping - it's only a matter of display, and not of how lines and columns are stored in the internal data structures of the control class.

Finally, we need to enter the code for the PrintCursor() method (used by our OnPaint() override) whether text wrapping is on or off. This method is implemented in code like this:

public bool PrintCursor(Graphics dc, LineInfo li, SentenceInfo si, int intWidthCounter, int l)
{
if ((cY == allLines.IndexOf(li) && si.X + si.strText.Length >= cX))
{
int cw = intWidthCounter;

// Go through each letter in the sentence, until we reach the cursor position
for (int i = 0; i <= si.strText.Length; i++)
{
if (si.X + i == cX)
{
cw += (int)dc.MeasureString(si.strText.Substring(0, i),
si.fntTextFont, new PointF(0,0), sfTextDisplayFormat).Width;

cXp = cw;
cYp = intHeightCounter + l;
dc.DrawRectangle(Pens.Firebrick, cXp + AutoScrollPosition.X,
cYp + AutoScrollPosition.Y, intCursorWidth, si.fntTextFont.Height);

return true;
}
}
}
return false;
}

This method checks to see if we are actually on the line on which text printing occurs, and if we are, goes through the characters measuring their sizes until we reach the current column position. It returns true if the cursor was actually drawn, false otherwise. As we already noted, we update the cursorDrawn variable with the return value of this method so that it won't get called again if the cursor has already been drawn.

Testing the control through a Windows Forms application

We can now go ahead and create a Windows Forms project to test the control we just created. It's more practical to add this project to the existing solution. To do that, go to the solution explorer window in the Visual Studio .NET IDE, right click on the first line displaying " Solution 'TextControlsLibrary' (1 Project) ", and select Add->New Project. Select a C# Windows Application.

We'll name this new project TextForm. When created, we reference the existing TextControlsLibrary from the new project so that we can use our control there. Again in the solution explorer, locate the TextForm project, right-click on its references tab, and select ' Add Reference. '. As the assembly we'll be referencing is already in the same solution, in the dialog that pops up select the ' Projects ' tab, select the TextControlsLibrary project, click Select and then OK. It would also be wise to add a using statement in the generated code for the control's namespace, so that we won't be qualifying its name every time we reference it:

using TextControlsLibrary;

We can go on and add an object of type TextDisplayControl in the code, and initialize it so that it will properly display on our form. We can find the control already added to the toolbox in the designer of the form created by Visual Studio, under the tab Windows Forms. Dragging it from there to the form is all the work we need to do.

If for any reason you need to skip Visual Studio .NET's toolbox and add the control in the code yourself (for example, to see how it works), the process is as follows:

In the pre-generated code for the Windows Form, add the following line:

namespace TextForm
{
public class Form1 : System.Windows.Forms.Form
{
private TextDisplayControl dc1;

The initialization of the control will take place inside the InitializeComponent() method, automatically generated by Visual Studio .NET. Find this method and inside it, add the following code:

dc1 = new TextControlsLibrary.TextDisplayControl();
dc1.BackColor = System.Drawing.Color.LightSteelBlue;
dc1.cX = 0;
dc1.cY = 0;
dc1.Location = new System.Drawing.Point(8, 8);
dc1.Name = "dc1";
dc1.Size = new System.Drawing.Size(392, 312);

This code firstly creates an instance of our control using the new keyword. Then, it sets various properties like location and size. Of course, we didn't include any definitions for properties like BackColor, Location or Size in the code for our control. These, amongst numerous other properties and methods, are inherited from the base Control class and can be used to define various parameters of the control's visual representation according to system or application settings.

We'll also need to add the control to the form. You can do this by entering the command this.Controls.Add(dc1) in the code initializing the form itself, later in the InitializeComponent() method.

After placing our control on the form (and perhaps giving it a distinct color so that we can better distinguish it - the default is the same as the Form's color), we can go into Visual Studio .NET's IDE and add some more controls to the form so that we can test the control.

As you can see from the image (and the provided project, if you are running it while reading the text) we have added some textboxes, buttons, and numeric selectors so that the user can easily select every attribute of the text he will be placing on the text-display control. There's a textbox where the text to be printed will be entered by the user, and selectors for line and columns numbers. There is also a CheckListBox control under the ' Font Attributes ' label - we have added four members to it, by selecting its Items property on the Properties window in Visual Studio. In the dialog that appears, we enter the strings Bold, Italic, Underline and Strikeout, each on its own line.

ListBox1 is where all the available fonts in the target computer will be available for selection. We had talked about the InstalledFontCollection class before when examining the System.Drawing.Text namespace. It's time to see it in action. In the Form's constructor, in the code automatically generated by Visual Studio .NET, we have added the following code:

InstalledFontCollection allFonts = new
            InstalledFontCollection();
FontFamily[] allFontFamilies =
            allFonts.Families;

foreach(FontFamily f in
            allFontFamilies)
{
listBox1.Items.Add(f.Name);
}

//
            Select the first font in the collection
listBox1.SelectedIndex =
            0;

Firstly, we create a new instance of the class by means of the new keyword. Then, we create a FontFamily array (a collection of font families) to keep all the families in the InstalledFontCollection we have created. Finally, we iterate through all of this collection's items and add their names into the textbox. Later, when creating the font to be displayed to the user, we'll use this name in the constructor of the Font class.

We also need to populate the second listbox with some brushes the user can choose from. The code that accomplishes this when placed in the form's constructor (directly underneath the previous bit of code) is the following:

         //
            Add some pre-defined brushes to the listbox
BrushInfo[] brushes =
            new BrushInfo[6];

brushes[0] = new
            BrushInfo();
brushes[0].ActualBrush =
            Brushes.Black;
brushes[0].Name = "Black";

brushes[1] = new
            BrushInfo();
brushes[1].ActualBrush =
            Brushes.Red;
brushes[1].Name = "Red";

brushes[2] = new
            BrushInfo();
brushes[2].ActualBrush =
            Brushes.Green;
brushes[2].Name = "Green";

brushes[3] = new
            BrushInfo();
brushes[3].ActualBrush =
            Brushes.Blue;
brushes[3].Name = "Blue";

brushes[4] = new
            BrushInfo();
brushes[4].ActualBrush =
            Brushes.Yellow;
brushes[4].Name = "Yellow";

brushes[5] =
            new BrushInfo();
brushes[5].ActualBrush =
            Brushes.Gray;
brushes[5].Name =
            "Gray";

listBox2.Items.AddRange(brushes);

listBox2.SelectedIndex
            = 0;

The BrushInfo class used above is just helping us to keep a track of the brushes we are providing to the user for selection. Its implementation is very simple, and can be placed on a new code file (like BrushInfo.cs), or even at the start of the form file, below the namespace declaration:

namespace TextForm
{
public class
            Form1 : System.Windows.Forms.Form
{
public class
            BrushInfo
{
public string Name;
public Brush
            ActualBrush;

public BrushInfo()
{

}

public
            override string ToString()
{
return
Name;
}
}

We just have two variables, the actual Brush object and a Name to display on the listbox. The override of the ToString() method returns whatever we had set as the brush name, so that we can display it in the listbox later.

After placing the code above and interface elements on the form, we only have one issue left with this Windows Forms application: printing the text that the user enters in the first textbox onto the form, in the specified column and line numbers, using the font family, size, attributes, and brush selected. This will happen when the big ' Print Text ' button is pressed. The code for the button's click event couldn't be simpler - double-click on the button in Visual Studio .NET's IDE and enter the code as follows:

private void button2_Click(object sender,
            System.EventArgs e)
{
if (textBox1.Text.Trim() == "")
            return;
Font f1 = GetSelectedFont();
if (f1 == null)
            return;
dc1.Print(textBox1.Text, (int)columnSelector.Value,
(int)lineSelector.Value, f1,
            GetSelectedBrush());
}

As you probably guessed, all we are doing now is enjoying the fruits of our previous labor by implemented the text display control. With a simple method call to our control, we can draw text with any of these attributes, just by specifying line and column numbers. The GetSelectedFont() and GetSelectedBrush() methods simply return appropriate Font and Brush objects, depending on what the user selected from the form's selection boxes:

Private Font
            GetSelectedFont()
{
try
{

FontStyle fstyle =
            FontStyle.Regular;
if
            (attribs.CheckedItems.Contains("Bold"))
fstyle = fstyle |
            FontStyle.Bold;

if
            (attribs.CheckedItems.Contains("Italic"))
fstyle = fstyle |
            FontStyle.Italic;

if
            (attribs.CheckedItems.Contains("Underline"))
fstyle = fstyle |
            FontStyle.Underline;

if
            (attribs.CheckedItems.Contains("Strikeout"))
fstyle = fstyle |
            FontStyle.Strikeout;

Font f1 = new
            Font(listBox1.SelectedItem.ToString(),
(float)fontSize.Value,
            fstyle);
return f1;
}
catch (ArgumentException)
{
//
            Some font-style combinations are not allowed and throw an
            ArgumentException
MessageBox.Show("This font and attributes
            combination is not supported.\nPlease
try again with a different
            combination.", "Error",
MessageBoxButtons.OK,
            MessageBoxIcon.Error);
return null;
}
}

private
            Brush GetSelectedBrush()
{
BrushInfo bi =
            (BrushInfo)listBox2.SelectedItem;
Brush b1 =
            bi.ActualBrush;
return b1;
}

This is what the form should look like when we run it:

The first listbox has been filled with all available fonts in the system, waiting to be selected. We can also see the brushes we have created, presented in the listbox below. We can start testing the control now, inserting sentences in any line and column, even between existing sentences. The control automatically takes care of any relocations of existing text, so that there is always ordered output.

We've also added some buttons on the form to test some of the other methods of our TextEditControl, like Clear(), DeleteLine() and InsertLine().

Note that we haven't added any code to display the line number or automatically wrap text here, although we could well have done so by changing the blWrapText and blShowLineNumbers properties to true. You can go ahead and directly set these properties for the TextDisplayControl in the form's code and watch the results, although we will be testing both these services extensively in the next control we'll create.

The TextEditControl control

Having already created a control that manages text display, it's time to proceed to our next objective: building a new control where the user can insert text of any style, much like a word processor. We already have much of the functionality ready in the TextDisplayControl class we've already looked at, and that class is where we'll derive our new control from.

It is a very common procedure to inherit classes we have already made when we want to get new, less generic features that are more concentrated on the specific application we are creating. In fact, that's the whole idea of creating generic class libraries that respond to many different kinds of applications. These applications will in turn derive their own classes from the ones provided. In this way, the developer can focus only on the new features he wants his application to have, not design everything from scratch.

That's exactly what we're doing here. We want to create a new control that behaves much like a word processor. This control obviously belongs in the category of applications that need to display text of various styles in their client area. Therefore, it can benefit from the class-control we designed earlier - in fact, we designed that control in such way that all applications which need to display text can benefit from it. But this does not in any way mean that the TextDisplayControl is sufficient for our new, more specific needs. We need to make it more specialized and powerful, and the way to accomplish that is to derive a new class from it, maintaining all its properties and getting the chance to add new ones (or even slightly alter some of the old ones to get the result we want).

So, considering the extra needs for the text-edit control, we quickly find out that things are not that difficult - the hardest part is already behind us. We definitely have to add a handler for the keyboard events, so that the control can respond to the user typing text, and have a way to keep the current line and column where text will be added/deleted. With support for cursors already in the base class, this last option is very easy: we'll just play with the cX and cY properties of TextDisplayControl. It will also be essential for the user to set the current font and brush with which to draw the next character. These requirements are more or less all the extras we need for our new control.

using System;
using
            System.Collections;
using System.Drawing;
using
            System.Windows.Forms;

namespace
            TextControlsLibrary
{
public class TextEditControl :
            TextDisplayControl
{
public Font fntCurrentFont;
public
            Brush fntCurrentBrushfntCurrentBrush;

public
            TextEditControl()
{
InitializeComponent();

this.cX =
            0;
this.cY = 0;

fntCurrentFont =
            fntDefaultFont;
fntCurrentBrushfntCurrentBrush =
            brDefaultBrush;
}

private void
            InitializeComponent()
{
this.KeyPress += new
            System.Windows.Forms.KeyPressEventHandler(this.KeyPressHandler);

this.Resize
            += new System.EventHandler(this.ResizeHandler);
}

In the InitializeComponent() method, called from the control's constructor, we are declaring the methods to be called when events of interest fire up. Of course, the major event in this specific text-editing control will be the KeyPress event. Let's take a look at how it is handled:

private void KeyPressHandler(object sender, System.Windows.Forms.KeyPressEventArgs e)
{

char c = e.KeyChar;

switch ((int)c)
{
case 13:
// Enter Key

// Move to the first column of the next line
InsertLine(cY + 1);
cX = 0;
cY ++;
cXp = (blShowLineNumbers)?35:0;
// This variable keeps the height of the last line, that's where the redrawing will start from
int lastLineHeight = ((LineInfo)allLines[cY - 1]).intTop;
Invalidate(new Rectangle(0, lastLineHeight, this.Width, this.Height - lastLineHeight));
this.AutoScrollPosition =
new Point(0, AutoScrollPosition.Y);
break;

case 27:
// Escape Key

// Move the cursor one position to the left
try
{
cX --;
Rectangle invalidRectangle =
new Rectangle(cXp - fntCurrentFont.Height + AutoScrollPosition.X,
cYp + AutoScrollPosition.Y, fntCurrentFont.Height + 2, fntCurrentFont.Height + 1);

Invalidate(invalidRectangle);
}
catch (ArgumentException)
{
// Don't move the cursor if it is already in the first column
}
break;

case 8:
// Backspace

// Find the current sentence and delete the letter
// at the location of the cursor
if (cX > 0)
{
LineInfo li = (LineInfo)allLines[cY];
foreach(SentenceInfo si in li.allSentences)
{
if ((si.X + si.strText.Length >= cX))
{
int oldwidth = si.intTextWidth;
si.strText = si.strText.Remove(cX - si.X - 1, 1);
if (si.strText == "")
li.allSentences.Remove(si);
else
{
// Update sentence's width
Graphics dc = this.CreateGraphics();
dc.TextRenderingHint = System.Drawing.Text.TextRenderingHint.AntiAlias;

si.intTextWidth = (int)dc.MeasureString(si.strText, si.fntTextFont,
new PointF(0,0), sfTextDisplayFormat).Width;

// Update X property for remaining sentences
for(int i = li.allSentences.IndexOf(si) + 1; i<li.allSentences.Count; i ++)
{
SentenceInfo nsentence = (SentenceInfo)li.allSentences[i];
nsentence.X --;
}
}

cX --;

// Update current font
fntCurrentFont = si.fntTextFont;
fntCurrentBrushfntCurrentBrush = si.brTextBrush;
break;
}
}

RedrawLine(cY);
}
else {
try
{
// Move the cursor to the last position of the previous line
// Delete this line if there is no text left in it

LineInfo l1 = (LineInfo)allLines[cY];

if (l1.allSentences.Count == 0)
DeleteLine(cY);

cY--;
cX = 0;
LineInfo li = (LineInfo)allLines[cY];
if (li.allSentences.Count > 0)
{
SentenceInfo si =
(SentenceInfo)li.allSentences[li.allSentences.Count - 1];
cX = si.X + si.strText.Length;
}
Invalidate(new Rectangle(0, cYp, Width, Height));
}
catch (ArgumentException)
{
// Ignore the instruction to decrement cY by one
// if we already are in the first line
}
break;

default:
// Printable character
Print(c.ToString(), cX, cY, fntCurrentFont, fntCurrentBrush);
cX ++;

// Update scrollbar positions if needed

// Horizontal scrollbar
if (!blWrapText && (cXp - AutoScrollPosition.X > Width))
{
AutoScrollPosition = new
Point(cXp + Width/10 - Width, AutoScrollPosition.Y);
}

// Vertical scrollbar
if (cYp - AutoScrollPosition.Y > Height)
{
AutoScrollPosition = new
Point(AutoScrollPosition.X, cYp - Height/10);
}

break;
}
}

In this method we first get the character pressed, which we find from the provided event arguments. After that, we handle all the possible different cases of keys being pressed.

As well as printable characters, the KeyPress event will also handle the user pressing some special keys, including the Enter key, the backspace key, and the escape key. We handle each of them first in the switch statement - if the key pressed isn't any of these, it is a printed character and control falls to the default case. There, the character that corresponds to the key press is printed at the location of the cursor, and the cursor's horizontal position is incremented by one. After that, we have just added some code to adjust the positions of the scrollbars (for example, we'll need to shift the scrollbars to the right if the letter entered is beyond the horizontal limit of the client area).

As for special keys, we have defined that Escape moves the cursor one position backwards, Enter moves the cursor to the next line, and Backspace deletes the character at the location of the cursor. The code for these handlers is nothing more than a little bit of logical thinking from our side, as all the hard work has already been done while implementing the TextDisplayControl class.

One other event we must pay attention to is the Resize event. If the blWrapText property of the control is set to true, it's essential to redraw the whole document when the control's dimensions change. This way, we give our application the chance to recalculate and redisplay every line again, correctly wrapped to the new dimensions.

private void ResizeHandler(object sender, System.EventArgs e)
{
if (blWrapText)
{
foreach (LineInfo li in allLines)
li.intHeightInPixels = 0;

Invalidate();
}
}

Also note that we set each line's height to 0. We do that so that line heights will be correctly recalculated when the OnPaint() method is called again, since in general, as a result of new wrappings, line's heights may be greater or smaller after the redraw. Why do we have to specifically zero out the height of each line when returning from wrapping mode to normal mode? The next time the OnPaint() method is called, it will go through each line and check each font on the line to see if it is higher than the line's set height. If it is, the line's height is updated. If we hadn't set the line's width to zero, the previous line height would remain from the wrapped version, and no font on the line would be higher than that, since wrapped lines are in general higher than any specific font in the line as a result of text wrapping.

That's all there is to the text editing control. It's a great example on how existing, generic controls (and classes in general) can become more specialized and fit exactly to a single application's needs.

Testing the TextEditControl

The procedure should be familiar from before, when we got to test the first control we had created. Proceeding in exactly the same way, we create a third project and add it to our solution: the TextEdit Windows Forms project. We've already done that in the code download, so if you find the procedure boring you can skip ahead to the fun part and run the complete project, testing the TextEditControl.

The way the selected (from the listboxes) font, size, brush and attributes are handled by the program is exactly the same as the previous test Windows Forms project. Go ahead and write some text into the control, changing the fonts and colors, to see how everything works.

Ideas and guidelines for extending the controls

The second control we've created, the one that can be used for text editing, is on its way to becoming a fully featured word processor. Methods for saving and loading documents from the hard disk are out of the scope of this case study, but their implementation should not be at all difficult, especially with the serialization services provided by the .NET framework - saving and retrieving all the LineInfo and the SentenceInfo classes couldn't be simpler. Besides, the code to load an existing text file is already in the sample Windows Forms application that tests our TextEdit control (included as a code download with this case study). The code to save the file as text only is very similar.

As a bonus, we have added in the second windows form in the code download some extra stuff that demonstrates how we could use existing code to get some interesting features. We made the control a bit more interesting by allowing the user to select the line where he's entering text with a single mouse click. Moreover, if there's no text wrapping, the column number at the position of the click is selected automatically. This is accomplished in the MouseDownHandler() method. If you followed the code discussion in this case study you should have no trouble understanding how it works, it's basically more of the same concepts at work.

There are also some extra menu items. The View -> Statistics menu command will count all the lines and characters in the document and display them to the user. It's very easy to do so - we have all the information needed in the ArrayLists where we keep line and sentence information. Moreover, the File -> Open item will display a file select box where we can search for a text file. When a file is selected, the program displays it into the text-edit control with the selected font and brush. By going through the code for these, relatively simple operations, you can find out how to add similar features.

Never forget that deep below its covers, the component we have created is a control and can be used as such. Numerous properties and methods inherited from the UserControl class can be very useful in many areas, like changing its visual representation (for example, background color) and giving it a different feel. After all, we already used some of these control properties, like Width, Height and AutoScrollPosition in our code. It would be useful to search the functionality these provide before going on to add more features to the control - what you want to add might already be there, or perhaps could be easier to accomplish with existing control properties.

Case Study Review

During this case study we have gone through all major aspects of text drawing in GDI+. We've covered the basics of text not specifically related to .NET, like font families and fonts in the Windows operating system. After that, we examined the classes that the .NET framework provides to represent all these text concepts, like Font and FontFamily, as well as some useful methods. Two of the most important methods used in our code were the DrawString() and MeasureString() methods. We saw how we can use the former to directly display text in the client area of any control, and the latter to measure the size of text when printed. We also went through the shortcomings of the MeasureString() method, and how we could avoid them.

After covering text properties in general, we defined the needs of our program and designed a new text-displaying control by inheriting from existing classes in the framework. The methods and algorithms created ensured that the user can safely print any text (with any style) in the client area of the control by just defining column and line numbers. The application automatically places the text in the appropriate position, taking into consideration all the other strings in the display, so that no overlapping or other problem occurs.

Although we have already covered the major points in the code for the controls and the applications that test them, you might want to go through the whole fully-documented listings provided as a download with this case study. To load the complete project, unzip the provided package into any directory on your hard drive, and from Visual Studio .NET load the file TextPrint.sln from that directory. The project is ready to be built and run.

If you've kept an open mind throughout the discussion about the design phase of our solution, it's quite possible that you might have identified similar problems of a generic nature where a similar approach could prove useful. In this case study, we first created as generic an object as possible, and then used it to create a more specific application - that's what happens in almost any real development scenario. Besides, with the complexity today's applications face, it would be almost impossible to create a program without the help of any external library - if you think about it, even when working with the .NET framework, developers are already utilizing a very complex and high level library that provides automated procedures for hundreds of programming tasks.

Any Limitations or Further Work

We only used anti-aliased fonts in our application. As you have probably seen by now, they appear smoother on the screen, but in small sizes certain fonts can become blurry. Another important issue is that of performance: although it is not visible in normal situations, drawing in anti-aliased mode requires more calculations from the computer and therefore runs more slowly. Besides, the user should have a selection on whether he wants to use anti-aliasing or normal text. Therefore, it would be extremely useful to be able to define a method that will accurately count string sizes (that's the reason we chose to rely on anti-aliased fonts in the first place, as the MeasureString() method does not provide accurate results on strings of other types).

There are some tricks to accomplish this, although some other problems (for example, fitting strings in rectangles) remain and will continue to cause minor problems. One solution that provides an accurate method that returns just the width of the string (we can always find its height from Font's Height property) can be found through the links below, at codeproject.com. This method is based on the more accurate Graphics.MeasureCharacterRanges() method, which returns an array of Region (and not a rectangle or size) objects around specified characters in a string. We end up measuring the width of these regions with the GetBounds() method. The downside of this method is performance and, in most cases, unnecessary use of system resources. As you can see from the following code, we must first provide a rectangle (the rect variable) that will have to be as long and high as the largest string we might need to measure. In normal cases we won't need such a large rectangle, but we must define it a bit large so that we can use the method with large text fonts.

Here is the code for the CountWidth() method:

static public int CountWidth(Graphics dc,
            string text, Font font)
{
if (text.Length == 0) return
            0;

StringFormat format = new StringFormat();
RectangleF
            rect = new RectangleF(0, 0, 100000, 1000);
CharacterRange[]
            ranges = { new CharacterRange(0, text.Length) };
Region[] regions
            = new Region[1];

format.SetMeasurableCharacterRanges
            (ranges);

regions = dc.MeasureCharacterRanges (text, font,
            rect, format);
rect = regions[0].GetBounds(dc);

return
            (int)(rect.Right + 1.0f);
}

Other useful features can be included in the code for the control class before compiling it to a final assembly, ready to be referenced by other projects. The text-edit control might need some client-area space of its own, to display information such as column/line numbers, the status of keys like Caps lock or Num lock, etc. This kind of functionality cannot be added in classes that implement the text-display control. We must go back to the base control source code and add it there. As it is now, the control will draw everywhere on its client area. A good idea might be to reserve space in its top/bottom or right/left edges (or maybe both) and keep the width/heights of these regions as protected variables in the code. In the override of the OnPaint() method in the TextDisplayControl class, we will take into consideration these sizes and ensure in the code that no drawing will take place there. After that, in the constructors of the other classes that inherit TextDisplayControl, we'd set these protected variables to appropriate values, to indicate what regions we want left out of the drawing. Then we'd go on and add another override of the OnPaint() method that would draw any required info into these areas only. Finally, we'd call the TextDisplayControl's own override to take care of text drawing into the rest of the rectangle. Another cool idea would be to display images between our text strings (admire that cute smiley in Windows Messenger?), or even other shapes. With the flexible Graphics class, the framework also makes this a simple task. For example, to accomplish printing images between text, we'd first instantiate a set of Image objects and set them to store the images that can be displayed. A new Print() overload can be constructed for the control, and it will use the Graphics.DrawImage() method to display the specified image in the current cursor location.

Of course, we do not have to limit the control's capabilities to just display or manipulate text. Our objective from the beginning was to give it useful methods to display text in columns and rows. It's very easy to add new, existing controls into our own, since it is inherited from the UserControl class which in turn implements the ContainerControl class. If we are using Visual Studio .NET, this task is really trivial: by double-clicking on the control's file in the solution explorer, we go to its design page. From there it is just a matter of a few clicks and drags to add other controls and set their properties. Think about it: we could have created more self-contained controls in the first place, for example by adding the listboxes and textboxes that our Windows Forms test projects had to our control from the beginning. But it makes more sense to just leave things as they are, and just derive new controls from our own as needed - these will be more specialized and probably have these other controls, like listboxes, inside them.


An Introduction to GDI+

An Introduction to GDI+
http://www.csharptoday.com/content.asp?id=1854
Article Rating: 3.5
Published on August 19, 2002
Abstract
This case study by Thomas Johansen will show you the latest graphics device interface from Microsoft, GDI+, which is part of the .NET framework. We could call the namespace System.Drawing “the GDI+ namespace”, as it is the namespace that contains all of the graphical classes and whatnot that we will be looking at. By the end of this case study you should be familiar with what GDI+ can do for you and your enterprise’s .NET applications, and how to build a quite complex application that can generate different types of charts based on several types of data sources.
Case Study

This case study will show you the latest graphics device interface from Microsoft, GDI+, which is part of the .NET framework. We could call the namespace System.Drawing "the GDI+ namespace", as it is the namespace that contains all of the graphical classes and whatnot that we will be looking at. By the end of the study you should be familiar with what GDI+ can do for you and your enterprise's .NET applications, and how to build a quite complex application that can generate different types of charts based on several types of data sources.

Throughout this case study I will make a total of six applications, varying in complexity. The first four demonstrate foundations and concepts that can be further built upon. The last two will have a bigger sense of usability, as you can use them to generate charts. The final application can only generate a simple line chart based on hard coded values, but included in the source code download there is a bigger and more complex charting application that will allow you to either manually type in the values or connect to a Sql Server. It also supports an additional chart type, pie chart. By the way, after the sixth application I'll explain to you how you can quite easily convert the application into a web service. Remember that there are whole companies that do nothing besides develop report generation software; some of these are Actuate, Crystal, ChartFX and Dundas.

GDI+ is based on GDI, which is also from Microsoft. There is some similarity in syntax and general programming techniques, but the change is quite a big one - many things that used to be quite difficult and tiresome has now become very easy and fun to do.

Those who are familiar with GDI will quickly notice that the biggest change is the new features. These new features are as following: gradient brushes, cardinal splines, independent path objects, the matrix object and transformations, scalable regions, alpha blending, and added support for more image types. Supported types are as follows: bmp, gif, jpeg, exif, png, tiff, icon, wmf and emf.

There is also a change in the programming model. One of the most obvious and clear changes is that in GDI you needed a device context in order to do any type of graphics. The device context holds information about the capabilities and limitations of the graphics device (generally referred to as a DC). In GDI+, the device context has been replaced with a Graphics object. For an example, look at the C++ and C# code below for a comparison.

C++ code

HDC hdc; // a handle to our DC (device
            context)
PAINTSTRUCT ps;
HPEN hPen;
.
hdc =
            BeginPaint(hWnd, &ps);
hPen = CreatePen(PS_SOLID, 3, RGB(255,
            0, 0));
SelectObject(hdc, hPen);
MoveToEx(hdc, 20, 10,
            NULL);
LineTo(hdc, 200, 100);
EndPaint(hWnd,
            &ps);

C# code

Note: In the sample we are going to override the predefined event handler for the OnPaint event. OnPaint is fired when a form or object is painted. By overriding the event we can add our own custom shapes to the object.

class PlainForm : Form
{
protected
            override void OnPaint(PaintEventArgs e)
{
Pen myPen = new
            Pen(Color.Red, 3);
Graphics myGraphics =
            e.Graphics;
myGraphics.DrawLine(myPen, 20, 10, 200,
            100);
}
}

The above blocks of code do exactly the same thing, and that is drawing a line on the users screen. You can see that the change in the programming model has changed quite a bit, and by that I don't mean the differences between C++ and C#, but the way you create a graphics context and pens and so forth.

Things such as pens, brushes, paths and so on are now objects in the .NET framework, which allows for a much simpler way of drawing lines and so on. This is quite easy to spot in the preceding example code.

Assumed skills

All the code in the case study will be written in C# and will at times be quite complex, and by that I mean the logical build up of some if the methods. You should be familiar with structs and event handling, so a good understanding is highly recommended. Also, all the work in this case study has been done in Visual Studio .NET, so a good understanding of this will be advantageous.

Your first GDI+ application

As this will be your first piece of GDI+ code, I will explain every new class, object and so on in detail.

What you will be shown is how to start from scratch and make a very simple GDI+ application.

In this section I will teach you the fundamentals and basics of any GDI+ application.

First let's start of by creating a new C# WinForm project.

Creating a new C# WinForm project

  1. Open Visual Studio.net
  2. Go to File -> New -> Project
  3. In the list in the window that appears after clicking on Project, select C#. Then click the icon that's called Windows Application.
  4. In the Name textbox type in "FirstApp", then press the OK button.

Now your project is set up and ready to be coded in.

Your first glance at GDI+

The basic structure in any GDI+ application is built up by a few methods and events - but of course that is the basic construction of any application! The event directly related to GDI+ is Paint, and our event handler for that event is where we put all of our little goodies.

The events that are used in all GDI+ applications are the paint events. For example let's say we have a button called myButton and we want to draw some special effects on it. What we would need to do first is to subscribe to the Paint event, so that we can tell it what event handler will handle the painting of the button. So it would be something like this:

private Button myButton;
Graphics
            g;
.
myButton.Paint += new
            PaintEventHandler(PaintButton);

private void
            PaintButton(object sender, PaintEventArgs e)
{
g = e.Graphics;
            // Do something fancy after this...
}

The above code is generally the main structure of any GDI+ application (besides the button), and is something you will see a lot more of in the rest of the case study.

Writing your first lines of code

Ok, let's get down to business. Right click Form1.cs in the Solution Explorer, and click in View Code. Once the code has been revealed to you, you should see a bunch of using statements. One of them should be using System.Drawing;. This is one of the GDI+ namespaces we will be using in this case study. It handles all of our primary objects and methods, such as the Draw and Fill methods.

Before we start doing any of the actual coding, let's lay out the form. Start off by adding a Panel. Don't bother resizing or anything like that, just set its docking to center. Also set its background color to white, as this generally looks a lot better as a canvas than a dull grayish color. Name the panel GfxPanel.

The size of the form should be around 260x210. I played around with the sizes so the graphics we are going to draw looked centered. You don't have to use a panel like I've done; you could draw straight onto the form, but then you'd have to redraw the entire form, instead of just the panel control, but I prefer redrawing only the controls that needs to be redrawn. This is important to remember if you have some drawing operations that take a while to complete, as Windows does not prioritize screen paints- it saves the user from having to wait for the entire form to be painted once again.

Now onto the coding part: open the code view for Form1.cs and scroll down to:

Private System.ComponentModel.Container components = null;

Below that line add:

Private Graphics g;

When that has been added, we need to add the event handler for the paint event on our panel control. We are going to do that inside the class constructor, which should be a few lines below. (It's the public Form1() method). Inside the constructor add:

GfxPanel.Paint += new PaintEventHandler(PaintControl);

We haven't created the PaintControl method yet, but type it in anyway. This line tells our program what event handler we want to be invoked when the panel control gets painted. In our case it's a method called PaintControl, and it will draw some simple shapes inside the panel.

Once those lines have been added we are ready to start on the PaintControl method. What's important to remember is that it needs be defined with the structure void (object sender, EventArgs e). In our case, EventArgs will need to be replaced with PaintEventArgs.

Let's add the final code then. First, declare the method called PaintControl, and set its encapsulation to private as only this class needs access to it. Inside it, initialize the Graphics object by setting it to e.Graphics. Set the SmoothingMode property to anti-aliasing as this will make our shapes look less jagged. The System.Drawing.Drawing2D.SmoothingMode enumeration helps you with this.

Then add a call to another method, called DrawGfx. I always structure my GDI+ applications this way, because it separates the different parts of the application, the definition and initialization from the actual drawing. Once you've added the method call we are done with PaintControl for now, so let's get onto the fun drawing part.

As I said just above, a method called DrawGfx will handle the drawing, I won't get into to detail on that just yet, as it contains many things I haven't covered yet. Just type the following code in:

private void DrawGfx()
{
// Sets up
            our pens that will be used for drawing
Pen redPen = new
            Pen(Color.Red, 2.0f);
Pen greenPen = new Pen(Color.Green,
            2.0f);
Pen bluePen = new Pen(Color.Blue, 2.0f);
// Sets up our
            brushes
SolidBrush redBrush = new
            SolidBrush(Color.Red);
SolidBrush greenBrush = new
            SolidBrush(Color.Green);
SolidBrush blueBrush = new
            SolidBrush(Color.Blue);

// This draws our first
            circle
g.DrawEllipse(redPen, 25, 25, 50, 50);

// This
            draws our first square
g.DrawRectangle(greenPen, 100, 25, 50,
            50);

// This draws our first triangle
Point[] triPoints =
            { new Point(200, 25), new Point(225, 75), new Point(175, 75)
            };
g.DrawPolygon(bluePen, triPoints);

// Draws a filled
            circle
g.FillEllipse(redBrush, 25, 100, 50, 50);

// Draws
            a filled square
g.FillRectangle(greenBrush, 100, 100, 50,
            50);

// Draws a filled triangle
Point[] triFillPoints = {
            new Point(200, 100), new Point(225, 150), new Point(175, 150)
            };
g.FillPolygon(blueBrush, triFillPoints);
}

As you can see right away it contains only things that I haven't showed you yet, but they will be covered in the next section of the case study.

All that is left to do now is to add code to dispose of the Graphics object and then build the project. Once the application has started you should have an application that looks like the one in the image below.

Introducing pens, brushes and fills

Pens

A pen is the tool you use if you want to draw something that will only be outlined, not filled with a color or gradient. The Draw methods use a pen to draw the graphics onto the screen, by getting the width of the pen and the color of it.

Creating an instance of a pen is very simple, but just because it's simple to use doesn't mean you should just rush into learning how to use it, as it is probably one of the most important and most used classes in the GDI+ workspace.

There are a few ways of creating an instance of the pen object:

1) public Pen(Color);
2) public Pen(Brush);
3) public Pen(Brush, float);
4) public Pen(Color, float);

The first way creates an instance of the Pen object with the specified color, the second with the specified brush, the third using the specified brush and float, and the fourth using the specified color and float.

The floats are the values that define the width of the pen. Using various colors, width and brushes you can create many cool effects, so don't brush off the pen completely.

Ok, now let's get down and dirty and write a few lines of code that will draw three similar Bezier splines but with different colors and widths:

private void DrawGfx()
{
// Here we
            declare three different pens
Pen smallPen = new Pen(Color.Blue,
            1.0f);
Pen mediumPen = new Pen(Color.Green, 3.0f);
Pen
            largePen = new Pen(Color.Red, 6.0f);

// Ok, lets draw 3
            bezier splines using the different pens..
g.DrawBezier(smallPen,
            10, 40, 90, 5, 210, 90, 260, 30);
g.DrawBezier(mediumPen, 10, 90,
            90, 55, 210, 140, 260, 80);
g.DrawBezier(largePen, 10, 140, 90,
            105, 210, 190, 260, 130);
}

This method is simply the standard method for drawing that is used throughout this case study, so the base code from the first application can be re-used. Just exchange the DrawGfx method in that with the one above.

All that Pen smallPen = new Pen(Color.Blue, 1.0f); does is to create an instance of the Pen object. The pen will draw blue lines that are 1.0 float wide. (As a standard, when no transformations has been applied, 1.0 float is the same as 1 pixel). The same goes for mediumPen and largePen as well, but with different widths and colors. DrawBezier is a method that draws a Bezier spline, and as it is one of the many draw methods it requires a pen in order to draw the spline. After you've written the above lines of code and compiled and run it, you should see something similar to the image below:

Brushes

A brush is the tool you use when you want to fill an area with a color or a gradient, and is used by the Fill methods, as well as one of the draw methods, DrawString. The DrawString method allows us to draw a string, in other words draw some text on the screen using the specified font and the font styles.

Just like the pen, creating an instance of a brush is very simple, although with brushes you have more than one object. There are several brush objects, namely SolidBrush, HatchBrush, LinearGradientBrush, PathGradientBrush and TextureBrush. The complexity of use differs a little: SolidBrush being the easiest, and PathGradient probably being the most difficult to get the full potential out of. As with the pen, the different brushes can be instantiated in several ways, and thus give different results.

SolidBrush

Let's start off by looking at SolidBrush, seeing as this is the least complex of the brushes. It is used when we simply want to fill a shape with a solid color. The SolidBrush object can only be constructed in one way, and the only argument it takes is Color.

SolidBrush myBrush = new SolidBrush(Color.Green);

HatchBrush

The HatchBrush can create some really cool looking pattern effects and be used to simulate "shadows" on 3D looking shapes. There are many different types of hatch styles: in fact there are 56 different styles that can be applied to the HatchBrush! The most used ones are probably the PercentXX styles. They fill the shape with the percentage ration, as in its name. So a Percent50 fills half of the shape with the foreground color and the other half with the background color. The HatchBrush can be constructed in 2 ways, taking a different number of arguments:

HatchBrush myBrush = new
            HatchBrush(HatchStyle, Color);
HatchBrush myBrush = new
            HatchBrush(HatchStyle, Color, Color);

The first one creates a brush with the given style and foreground color. The second constructor is exactly the same, but it also defines a background color to use, instead of using a transparent color.

As I said earlier, the hatch brush is useful when you need to create a 3D effect on an object. Let's say for an example that you're going to draw a cubical shape, and then you would draw the side that has the light on it with a normal solid brush, and the sides with shadow using a hatch brush. This would create the effect of being 3D. Of course you could just use a different color, but using the hatch brush you use the same background color, but different foreground color, perhaps black, and then use the HatchStyle enumeration to create a textured look.

Let's look at some different examples of ellipses filled using a hatch brush:

// We need to define some different
            brushes...
// Left column
HatchBrush hatch1 = new
            HatchBrush(HatchStyle.Percent70, Color.Black,
            Color.Aqua);
HatchBrush hatch2 = new HatchBrush(HatchStyle.Wave,
            Color.Red, Color.Green);
HatchBrush hatch3 = new
            HatchBrush(HatchStyle.DarkUpwardDiagonal, Color.LightSteelBlue,
            Color.Green);
HatchBrush hatch4 = new
            HatchBrush(HatchStyle.LargeConfetti, Color.BlanchedAlmond,
            Color.DarkSlateBlue);
HatchBrush hatch5 = new
            HatchBrush(HatchStyle.DiagonalBrick, Color.MediumPurple,
            Color.DarkSlateBlue);
// Right column
HatchBrush hatch6 = new
            HatchBrush(HatchStyle.OutlinedDiamond, Color.HotPink,
            Color.Red);
HatchBrush hatch7 = new
            HatchBrush(HatchStyle.Shingle, Color.SkyBlue,
            Color.Orange);
HatchBrush hatch8 = new
            HatchBrush(HatchStyle.Trellis, Color.BlanchedAlmond,
            Color.GreenYellow);
HatchBrush hatch9 = new
            HatchBrush(HatchStyle.Wave, Color.Orchid,
            Color.Honeydew);
HatchBrush hatch10 = new
            HatchBrush(HatchStyle.Sphere, Color.Azure, Color.Olive);

//
            Lets draw some ellipses...
// Left
            column
g.FillEllipse(hatch1, 20, 20, 100,
            40);
g.FillEllipse(hatch2, 20, 70, 100,
            40);
g.FillEllipse(hatch3, 20, 120, 100,
            40);
g.FillEllipse(hatch4, 20, 170, 100,
            40);
g.FillEllipse(hatch5, 20, 220, 100, 40);
// Right
            column
g.FillEllipse(hatch6, 140, 20, 100,
            40);
g.FillEllipse(hatch7, 140, 70, 100,
            40);
g.FillEllipse(hatch8, 140, 120, 100,
            40);
g.FillEllipse(hatch9, 140, 170, 100,
            40);
g.FillEllipse(hatch10, 140, 220, 100, 40);

Add the code to the code base and compile. You should end up with something similar to what is in the image below:

LinearGradientBrush

Now things will suddenly become a little more complex, because this brush can be constructed in several ways, and has many new class members; such as Blend, GammaCorrection, InterpolationColors and so on. I will not cover these members now, but instead I'll start of by showing you the different ways of creating an instance of the LinearGradientBrush :

public LinearGradientBrush(Point, Point, Color, Color);
public LinearGradientBrush(PointF, PointF, Color, Color);
public LinearGradientBrush(Rectangle, Color, Color, LinearGradientMode);
public LinearGradientBrush(RectangleF, Color, Color, LinearGradientMode);
public LinearGradientBrush(Rectangle, Color, Color, float);
public LinearGradientBrush(RectangleF, Color, Color, float);
public LinearGradientBrush(Rectangle, Color, Color, float, bool);
public LinearGradientBrush(RectangleF, Color, Color, float, bool);

The first and second constructor creates a brush, which is defined by a starting and ending points and starting and ending colors. Be sure to make a note that a point is defined like this:

Point myPoint = new Point(x, y);

Where x and y are integers. Same goes for PointF, but there x and y are floats instead of integers.

The third and fourth constructors create a brush, which is defined by a Rectangle (either containing integer or float values), a starting and ending color, and LinearGradientMode. Rectangles are defined as following:

Rectangle myRect = new Rectangle(x1, y1, x2, y2);

So basically all a rectangle is is a collection of two points, namely the top left and bottom right corners - all the previous also applies to RectangleF of course.

The LinearGradientMode enumeration object contains four enumerators: BackwardDiagonal, ForwardDiagonal, Horizontal and Vertical :

  • BackwardDiagonal - Specifies a gradient from upper right to lower left.
  • ForwardDiagonal - Specifies a gradient from upper left to lower right.
  • Horizontal - Specifies a gradient from left to right.
  • Vertical - Specifies a gradient from top to bottom.

The fifth and sixth constructors create a brush, which is defined by a Rectangle, start and end color, and a float. The float defines the angle for the gradient fill, thus ranging from 0 to 360 degrees.

The seventh and eighth constructors are the same as the fifth and sixth, but it also has a bool that defines whether or not the angle is affected by a transformation. The angle is measured from the x-axis and counter-clockwise.

Now that you've had a quick walkthrough of the LinearGradientBrush, let's add a few more lines to our gradient brush application:

// We need to define our gradient
            brushes.
Rectangle rect1 = new Rectangle(20, 20, 120,
            120);
LinearGradientBrush lgb1 = new LinearGradientBrush(rect1,
            Color.Red, Color.Green, 45);

// A
            rectangle...
g.FillRectangle(lgb1, 20, 20, 100, 100);

All of the code above is very straightforward - I've used the fifth constructor for this code example, as I felt it would show you most of the brush's capabilities.

PathGradientBrush

The PathGradientBrush is the most versatile and fancy of the brushes, and therefore also the most complex one. The initialization itself is as easy as any of the other brushes, but the definitions of the points it uses to draw a path to fill along can be very time consuming, as you've got to sit and guesstimate where to place all of the points in order to achieve the desired effect. Well enough small talk, let's look at the different ways of constructing a PathGradientBrush :

public PathGradientBrush(GraphicsPath);
public PathGradientBrush(Point[]);
public PathGradientBrush(PointF[]);
public PathGradientBrush(Point[], WrapMode);
public PathGradientBrush(PointF[], WrapMode);

The first constructor creates an instance of PathGradientBrush, taking a GraphicsPath as an argument. A GraphicsPath is basically nothing more than a separate Graphics "layer". It supports most of the methods and properties that the Graphics object does: you can draw ellipses, rectangles, etc, on the layer, and then move it around independently of the Graphics object it is drawn inside. It is something that is very handy when you need to draw a complex collage of items, and need to move them around a lot. So instead of changing the x and y coordinates of all the ellipses and so on, you only change the coordinates of the path. I couldn't do without it when I'm making my little GDI+ games.

The second constructor creates an instance of PathGradientBrush, using an array of points to define the coordinates for the path. For the third constructor, the same applies which takes an array of floating points:

PointF[] myPoints = new PointF[] {
new PointF(21.3f, 10.0f),
new PointF(12.0f, 65.5f),
new PointF( 2.0f, 3.0f)
};

The only change in the fourth and fifth constructor is that it also has an additional argument: a WrapMode enumerator. The WrapMode enumerator contains five enumerators:

  • Clamp: Clamps the texture or gradient to the object boundary.
  • Tile: Tiles the gradient or texture.
  • TileFlipX: Reverses the texture or gradient horizontally and then tiles the texture or gradient.
  • TileFlipY: Reverses the texture or gradient vertically and then tiles the texture or gradient.
  • TileFlipXY: Reverses the texture or gradient vertically and horizontally and then tiles the texture or gradient.

The Clamp enumerator is a really cool feature, as it makes sure the gradient is re-sized to the objects size by clamping it to the boundary of the object.

A good tip when using the PathGradientBrush is to define/draw it the same way as the shape you're going to use it to fill. So let's say you are going to use it on an ellipse, then you should opt for the GraphicsPath constructor and draw an identical ellipse on the path. Even if you have a polygon, you could use a path and draw a polygon on that - just re-use the point array later when you are going to draw the polygon itself. By doing it that way you can make sure that the fill path matches the polygon, and thus gives the best result.

With the PathGradientBrush, just defining styles in the constructor arguments isn't longer enough; we need to start setting property values now. The properties for the PathGradientBrush class are as follows:

  • Blend: Gets or sets a Blend object that specifies positions and factors that define a custom falloff for the gradient.
  • CenterColor: Gets or sets the color at the center of the path gradient.
  • CenterPoint: Gets or sets the center point of the path gradient.
  • FocusScales: Gets or sets the focus point for the gradient falloff.
  • InterpolationColors: Gets or sets a ColorBlend object that defines a multicolor linear gradient.
  • Rectangle: Gets a bounding rectangle for this PathGradientBrush object.
  • SurroundColors: Gets or sets an array of colors that corresponds to the points in the path this PathGradientBrush object fills.
  • Transform: Gets or sets a Matrix object that defines a local transform for this PathGradientBrush object.
  • WrapMode: Gets or sets a WrapMode enumeration that indicates the wrap mode for this PathGradientBrush object.

These are all of the properties for the PathGradientBrush and all of them are very useful, so make sure you remember them and what they do! I won't show you all of them, so you should play around with them yourself and see what cool effects you can get with different mixes. The PathGradientBrush object also has some methods you should know about, so let's quickly go over the useful ones:

  • MultiplyTransform: Overloaded. Multiplies the Matrix object that represents the local geometric transform of this PathGradientBrush by the specified Matrix by prepending the specified Matrix.
  • ResetTransform: Resets the Transform property to identity.
  • RotateTransform: Overloaded. Applies a clockwise rotation of the specified angle to the local geometric transform.
  • ScaleTransform: Overloaded. Scales the local geometric transform by the specified amounts. This method prepends the scaling matrix to the transform.
  • SetBlendTriangularShape: Overloaded. Creates a gradient with a center color and a linear falloff to one surrounding color.
  • SetSigmaBellShape: Overloaded. Creates a gradient falloff between the center color and the first surrounding color based on a bell-shaped curve.
  • TranslateTransform: Overloaded. Applies the specified translation to the local geometric transform. This method prepends the translation to the transform.

Ok, now that you are aware of most of the constructors, properties and methods that are associated with PathGradientBrush, let us start using it. First off let's draw a simple ellipse and fill it with a path gradient:

// Definition of our path gradient filled
            ellipse.
GraphicsPath path1 = new
            GraphicsPath();
path1.AddEllipse(140, 20, 100,
            100);

PathGradientBrush pgb1 = new
            PathGradientBrush(path1);

pgb1.CenterColor =
            Color.Blue;
Color[] colors1 =
            {Color.Aqua};
pgb1.SurroundColors =
            colors1;

g.FillEllipse(pgb1, 140, 20, 100, 100);

Here you can see that I've used a graphics path to add the boundary for the path gradient. I generally only use the graphics path method when I'm using a path gradient, at least when I'm going to fill a shape completely - if I only want a partial fill or I'm going after a special effect, I might turn to just defining the point array myself without using a graphics path.

Note that the size of the ellipse is the same as the path gradient - this is not needed if you define the WrapMode property of the gradient, but by doing it yourself you can be sure that it look just like you want it to.

The next thing on the agenda now is to draw something slightly more complex; a triangle, which will have differently colored corners. We will draw the triangle by using the FillPolygon method and an array of three points:

// PGB triangle
GraphicsPath path2 = new
            GraphicsPath();
Point[] points1 = { new Point(70, 140),
new
            Point(20, 240),
new Point(120,
            240)
};
path2.AddPolygon(points1);

PathGradientBrush
            pgb2 = new PathGradientBrush(path2);

Color[] colors2 = {
            Color.Red, Color.Green, Color.Blue };
pgb2.SurroundColors =
            colors2;

g.FillPolygon(pgb2, points1);

Once again a graphics path was used for the boundary definition for the path gradient. The first point defines the top of the triangle, the second one the lower right corner, and the third the lower left corner.

As you can see, I've got three points that draw the triangle, and I've got three colors in the array of colors called colors2. Each corner of the triangle then gets its own color, and these colors will blend nicely together.

Creating a five-armed star is the next thing we'll look at. When creating something as complex as this, it's a matter of trial and error getting all of the points aligned and setup correctly. A good tip is to start off by creating one side, then when that is done, just mirror it. This can be done in several ways; two of them would be to either add all the reverse points manually by typing them in, or another way could be do a matrix transformation on the half and placing that next to the clone. I opted for the first method, just reversing the points manually:

// PGB star
GraphicsPath path3 = new
            GraphicsPath();
Point[] points2 = {
// TOP MIDDLE
new
            Point(190, 140),
// RIGHT
new Point(205, 170), new Point(240,
            170),
new Point(215, 195), new Point(230, 230),
// BTM
            MIDDLE
new Point(190, 205),
// LEFT
new Point(150, 230),
            new Point(165, 195),
new Point(140, 170), new Point(175,
            170)
};

path3.AddPolygon(points2);
PathGradientBrush
            pgb3 = new PathGradientBrush(path3);

Color[] colors3 =
            {
// TOP MIDDLE
Color.Red,
// RIGHT
Color.Red,
            Color.Red,
Color.Red, Color.Red,
// BTM
            MIDDLE
Color.Red,
// LEFT
Color.Red,
            Color.Red,
Color.Red, Color.Red
};
pgb3.SurroundColors =
            colors3;
pgb3.CenterColor =
            Color.Maroon;

g.FillPolygon(pgb3, points2);

As you will see if you comment out the "left" parts in the code, you'll only get the right side of the star - that's how i started work on it, just doing the right side, and when I felt happy with how that looked I just figured out what the opposite points would be and typed them in there. Au voila, and I suddenly had a complete five armed star that didn't look bad at all.

If you combine the code sections above together, put it into the DrawGfx method and compile, you should end up with something that looks like the image below:

TextureBrush

The TextureBrush can be used to create some really fancy texture effects, as you draw the surfaces using an image. The file needs to be loaded into an Image object.

You don't have to use the entire image, because using a Rectangle or RectangleF object you can specify the bounding area of the image that you want to use as the texture - by doing this, you only need one texture image even if you need several textures. This is because as long as you know the values of the rectangle around the texture in the image you want you can just specify that in the Rectangle. This can be useful when you need to draw several shapes using almost the same textures. For example when only a few colors are different or something, then you can make a function that loops through the different textures. However, if you want to attempt something like that, it's important that all of the textures are of equal sizes.

Let's look at the constructors of the TextureBrush :

public TextureBrush(Image);
public TextureBrush(Image, Rectangle);
public TextureBrush(Image, RectangleF);
public TextureBrush(Image, WrapMode);
public TextureBrush(Image, Rectangle, ImageAttributes);
public TextureBrush(Image, RectangleF, ImageAttributes);
public TextureBrush(Image, WrapMode, Rectangle);
public TextureBrush(Image, WrapMode, RectangleF);

The first constructor creates an instance of the TextureBrush object, taking only one argument, and that is a reference to an Image object. Because it doesn't get a Rectangle it uses the entire image as the texture.

The second and third constructors take an additional argument - a Rectangle reference. The boundaries of the rectangle define the boundaries of the texture based on the referenced image. By supplying the rectangle reference you can choose what part of the image should be used as the texture.

Image myImage =
            Image.FromFile("TextureImage.bmp");
Rectangle myRectangle = new
            Rectangle(0, 0, 64, 64);

TextureBrush myBrush = new
            TextureBrush(myImage, myRectangle);

g.FillEllipse(myBrush, 0,
            0, 128, 256);

The code above draws an ellipse that's 128 pixels wide and 256 pixels high. The ellipse uses a texture brush. The texture is from the image TextureImage.bmp, and the texture is 64 pixels wide and 64 pixels high.

The fourth constructor is defined by an Image object and a WrapMode enumeration object. As you've seen earlier, the WrapMode enumerator defines how the brush is repeated.

The fifth and sixth constructors take three arguments: an Image object, a Rectangle object, and an ImageAttributes enumeration object.

A simple line chart

What the finished application will allow us to do is to draw a very simple line chart. All values are hard coded though, so at this point this line chart application won't be very easy to use, but it will be easy to extend the usability of it, like loading data from a Sql Server database table or similar. First let's take a quick peek at an image of the final application:

As you can see, there is nothing wrong with the chart that is drawn - the only problems are that the chart cannot be saved as a file and the values that the chart is based upon cannot be dynamically added, they have to be hard coded. However, the application will still show you the basics of making a line chart drawing application. How to calculate x and y coordinates and so on.

Starting up a new project

First of all we need to start a new project: create a new windows form project and call it SimpleChart. Let's start off by adding the necessary namespaces, and discuss the structure and functions later.

Needed namespaces

For this application we need only a small assortment of namespaces. The ones we need are:

  • System: Needed for the most basic objects and methods.
  • System.Drawing: The "GDI+ namespace".
  • System.Collections: Contains one object that we will be using to store our data, the ArrayList object. It also contains a variety of other collection classes, such as hash tables, stacks, queues and so on.
  • System.ComponentModel: Needed when developing a windows form; handles the components. It contains licensing classes, base classes, attributes, type converters, descriptors and more.
  • System.Windows.Forms: The windows forms objects, methods and so on.

The namespaces you need to focus on are System.Drawing and System.Collections. System.Drawing is the namespace that contains all of the GDI+ objects needed and so on. The System.Collections namespace is also an important namespace in our application. It contains the ArrayList object and we will use that to store all of our chart data. The data stored in it will be a custom data structure that we will call a DataPoint, and it'll contain two floats and a bool. The floats will store x and y coordinates and the bool will tell us if the data point is valid or not.

Overview of class and method structure

In this application we will have two classes: one is derived from Form, the other is our line chart class, the existing one. Our line chart class will consist of two methods - AddDataPoint and Draw. AddDataPoint will add a DataPoint to our ArrayList. The Draw method will draw a line chart based on the data that is stored in the ArrayList.

Building the application

Let's look at the LineChart class first. Put it into its own .cs file, call it Line.cs or something similar. It's a good practice to separate classes by putting them into separate files - one class in each file.

Now let's start adding some code to the application: we will begin with the variables that are at class level:

public Bitmap b;
public string ChartTitle = "Default Title";
public float OriginX = 0.0f;
public float OriginY = 0.0f;
public float Inset = 50.0f;
public int DivX = 5;
public int DivY = 10;

private ArrayList DataPoints = new ArrayList();
private int ChartWidth;
private int ChartHeight;
private int ScaleX;
private int ScaleY;
private Graphics g;

Some of our variables are public, they need to be so that when we create a new instance of the class we will have access to get and set the values of them. The public ones are the ones that we use when we calculate the size of the chart, get the title string, and so on. ChartTitle doesn't do anything else than supply the title for our chart. If it isn't changed the standard, "Default Title" will be used. OriginX and OriginY tells us where the points should originate from and as a standard has the value 0.0f. Inset defines how far from the edges the chart should be drawn. The standard is 50.0f. DivX and DivY defines how many dividers should be on each axis. The default value is 5 for x-axis and 10 for y-axis.

The ArrayList instance called DataPoints will be used a lot throughout the entire class, if it's empty there won't be a line chart drawn. ChartWidth and ChartHeight are class variables that get defined based on the values that are set when an instance of the class is created. ScaleX and ScaleY defines the scale on the x and y axis: that is how high the numbering on the axes will go. Both variables get set when the class is initialized, just like ChartHeight and ChartWidth. You should be familiar with Graphics object by now.

Now we need to create one last variable, a custom data structure:

struct DataPoint
{
public float
            X;
public float Y;
public bool IsValid;
}

This is the structure we will use to store all of our chart data with. We have to make our own struct because there is no current struct that can handle the data we need to store. A struct is a good way of relating and connecting variables, instead of having disconnected variables that can only relate to one another through similar indexes within an array. The IsValid Boolean is used to mark values as valid or invalid. Invalid values are not drawn in the chart - the only purpose it will have in this sample is to make sure we give the origin point a value.

Well now that we've added the above lines of code we need to add the class Initializer. It will take a few arguments: four integers, width, height, scaleX and scaleY. These arguments set the values of the private variables with the same names, the ones at class level:

public LineChart(int width, int height, int
            scaleX, int scaleY)
{
ChartWidth = width;
ChartHeight =
            height;
ScaleX = scaleX;
ScaleY = scaleY;
b = new
            Bitmap(width, height);
g =
Graphics.FromImage(b);
}

The initialization above is very basic stuff, you ought to make sure you remember the bottom two lines of code though; where we initialize the Bitmap and the Graphics objects. We set the Graphics object to be "derived from" the bitmap. This is unlike how we've done it previously, when we just drew stuff right onto a control. Because we want to store the graphics we have drawn in a file we need to draw on the Bitmap object. That way we can easily save it locally as an image file.

Next we need to create the method that will add data to the array list. It's pretty basic stuff:

public void AddDataPoint(float x, float
            y)
{
DataPoint point;
point.X = x;
point.Y =
            y;
point.IsValid =
            true;

DataPoints.Add(point);
}

All we do in this method is to create an instance of our custom data structure DataPoint and call the instance point. Then we assign values to its parameters based on the data in the argument. We also set the IsValid parameter to true by default. Finally we simply add the DataPoint instance to the DataPoints array list.

We're almost finished now, we just have one more method left, the method that will draw the entire chart and save the bitmap. The method contains a lot of code so I'll divide it into sections, based on what the code in each respective section does. First off we need to define four floating points, these will be used for controlling the x and y coordinates of the origin point and the end point:

public void Draw()
{
float x;
float
            y;
float x0;
float y0;

string tempLabel;
FileStream
            fs = new FileStream(@"C:\TmpChart.png",
            FileMode.OpenOrCreate);

The four float types are used for determining where the previous and current point is located, so that we can draw a line between them. The string called tempLabel is used when we draw the labels on the axes. The FileStream object we use for the file management needed for saving the bitmap to the users:

// We need to define two sets of pens and
            brushes, which we will use when
// drawing our chart...
Pen
            blackPen = new Pen(Color.Black, 1);
SolidBrush blackBrush = new
            SolidBrush(Color.Black);
Pen bluePen = new Pen(Color.Blue,
            1);
SolidBrush blueBrush = new SolidBrush(Color.Blue);

//
            The font that we will use for drawing the text on the axes
Font
            axisFont = new Font("Trebuchet MS",
            10.0f);

g.FillRectangle(new SolidBrush(Color.White), 0.0f,
            0.0f, ChartWidth, ChartHeight);

The first thing we do here is to define the pens and brushes that will be used for drawing the lines and points that builds up the chart itself.

We also define a Font object that we call axisFont, and yes you've guessed it; it will be used when we are drawing the text onto the axes. The last line marks the beginning of the chart - it simply fills the entire drawing area with white, which will be the canvas if you will:

float width = ChartWidth - (2 *
            Inset);
float height = ChartHeight - (2 *
            Inset);

g.DrawRectangle(blackPen, Inset, Inset, width,
            height);

// We need to paint our text items before we do a
            rotation transform...
g.DrawString(ChartTitle, new
            Font("Trebuchet MS", 16.0f), blackBrush, Inset, 10.0f);

The first thing we do is to declare to new floats, width and height. These define the size of the actual chart, not the entire drawing area, which is what ChartWidth and ChartHeight do. After that we draw the black frame around the area where the chart will be contained. The bottom line and the left line act as our x and y axes.

Our chart title is the next thing that is dealt with: the DrawString method takes care of that - the title is defined by the ChartTitle variable.

Drawing labels on the axes is next, and that is exactly what the two for loops do. x in the x-axis loop is calculated based on the margin around the charting area, how many labels have been drawn, and how many dividers we have. y is simply the height of the drawing area. x and y in the y-axis loop are calculated almost exactly the same way, but only the other way around (x being y and y being x in a sense).

// Labels on the X axis
for (int i = 0; i
            < DivX; i++)
{
x = Inset + (i * width) / DivX;
y =
            height + Inset;
tempLabel = ((int) (OriginX + (ScaleX * i /
            DivX))).ToString();
g.DrawString(tempLabel, axisFont, blackBrush,
            (x - 4.0f), (y + 10.0f));
g.DrawLine(blackPen, x, (y + 2.0f), x,
            (y - 2.0f));
}

// Labels on the Y axis
for (int i = 0;
            i < DivY; i++)
{
x = Inset;
y = height + Inset - (i *
            height / DivY);
tempLabel = ((int) (OriginY + (ScaleY * i /
            DivY))).ToString();
g.DrawString(tempLabel, axisFont, blackBrush,
            5.0f, (y - 6.0f));
g.DrawLine(blackPen, (x + 2.0f), y, (x -
            2.0f), y);
}

Well now we need to rotate our drawing area so that we originate from the lower-left instead of the upper-left. This we can accomplish by using a few transformations:

// We need to do a rotation transform in order to use the standard
// orientation for
// line charts - drawing is done from the bottom-left. [Lower-left = (0, 0)]
g.RotateTransform(180.0f);
g.TranslateTransform(0.0f, -ChartHeight);
g.TranslateTransform(-Inset, Inset);
g.ScaleTransform(-1.0f, 1.0f);

As I mentioned above, what this code does is to first rotate the drawing area 180 degrees counter-clockwise. Then the next lines flip everything so that they are located at the correct spots and look like they should.

Now we will look at the code that draws all the points and draws lines between them, in order to form a line chart:

// Ok, now lets draw all of the data
            points;
DataPoint prev = new DataPoint();
prev.IsValid =
            false;

foreach (DataPoint point in DataPoints)
{
if
            (prev.IsValid)
{
x0 = width * (prev.X - OriginX) /
            ScaleX;
y0 = height * (prev.Y - OriginY) / ScaleY;
x = width *
            (point.X - OriginX) / ScaleX;
y = height * (point.Y - OriginY) /
            ScaleY;

g.DrawLine(bluePen, x0, y0, x,
            y);
g.FillEllipse(blueBrush, (x0 - 2.0f), (y0 - 2.0f), 4.0f,
            4.0f);
g.FillEllipse(blueBrush, (x - 2.0f), (y - 2.0f), 4.0f,
            4.0f);
}
prev = point;
}

The thing we do here is to first create a new DataPoint that we call prev. This will keep track of where our previous point was drawn, so that we can draw a line from that point to the current point. We also set the IsValid member to false the first time around; we need to do that in order to add an origin for the first line.

We loop through all the DataPoints in the DataPoints array list using a foreach loop. The way we set the x and y coordinates are really straightforward, no hocus pocus there - the same goes for the line drawing and point drawing. At the end of each loop we set the DataPoint instance prev to be equal to the DataPoint instance called point. We need to do because, as I mentioned earlier, the prev variable stores where the last drawn point was located.

After all the drawing is done, all we need to do is to store the chart in a local file, and in this case in the file called TmpChart.png that is located at the root of C:\

b.Save(fs, System.Drawing.Imaging.ImageFormat.Png);
fs.Close();

Ok, that covers the line chart class. Let's quickly look at what we need to do to the form class.

Basically all we need to do is to create an instance of the charting class, add some data points using the AddDataPoint method, set a few variables like the width of the chart and so on, and finally call the Draw method.

The form class should have a button to call an event that will draw our chart, store it to the disk, and load the image in a PictureBox control using its Image property. An example of this is listed below:

public void DrawChartButton_Click(object
            sender, EventArgs e)
{
LineChart chart = new LineChart(640,
            480, 500, 1000);
chart.ChartTile = "My Line Chart";
chart.DivX
            = 5;
chart.DivY = 10;

chart.AddDataPoint(50,
            100);
chart.AddDataPoint(100, 150);
chart.AddDataPoint(300,
            250);
chart.AddDataPoint(400,
            50);

chart.Draw();

FileStream fs = new
            FileStream(@"C:\TmpChart.png", FileMode.Open);
myPictureBox.Image
            = new Bitmap(fs);
fs.Close();
}

Just add the code above to the application and associate the function with an event. For better usability, add a DataGrid control or some form of database integration into the application and loop through all the records and making sure you add the values into the AddDataPoint method.

(For a more complex charting application and an additional chart class, a pie chart class, check the downloadable source for this case study. Below is a screenshot from the application that is part of the source code download.)

Overview of what you have learned

In this section of the case study you have learned how to build a very simple, yet easily upgradeable charting application. You could attempt to do some of the following:

  • Loading data from a SQL server database table: simply add a DataSet or something similar and fill it with data, then to add the points based on the data, just loop through all of the rows and set the X and Y values based on the data in the columns.
  • You can also load XML data into a DataSet using its member method ReadXml. After the data is loaded you do as explained in the previous bullet point - simply loop through.
  • Other things that you might want to consider, are the ability to change the font style for all the text, dynamic resizing, and so on.

You've gotten the low-down on the logics behind a line chart application; as in how to calculate x and y coordinates, how to draw the line and the logics behind that. We've also looked at something that is needed in any line chart, and that is to transform the drawing origin. You should now have the necessary skills needed to attempt to integrate charting and other GDI+ related fancy stuff into your own .NET applications.

If you are interested in converting the application into a webservice, you should take note of the following: exchange all FileStream objects with MemoryStream objects, as we do not need to write anything out to files, but rather temporarily store the image data. Once we are ready to send the data to the client we need to send it as bytes. That way when the client receives the data he can convert the bytes back into an image, and perhaps store it as a image file. So there you have it, the basics behind converting the application into a chart webservice.

Case study review

Having read this case study, you should feel confident in your newly learned GDI+ skills, and you should feel confident enough to try and add something like charting to your or your company's .NET applications. As you've seen it is not difficult at all.

GDI+ can be used to visualize much more than just charts - you can use it for visualizing data sets, data grids, creating artistic looking buttons, banners for applications, and lots more. The only thing that stands in your way besides your boss is your imagination really. GDI+ can do almost anything: you could write your own 3D simulations too using GDI+.

If you're interested in learning about 3D in GDI+ you should look at some of the other GDI+ case studies on C# Today - a good understanding of matrixes is highly recommended.

Creating new chart classes is not difficult at all, as long as you keep your head cool and sketch things out before you start doing any actual coding. The next step would perhaps be to create a bar chart class.

Hope you've enjoyed reading this as much as I've enjoyed writing it!

Case Study Information
Author | Thomas Johansen
Editor | Adam Ryland
Reviewers | David Williams, Kunal Cheda, Richard Conway

If you have any questions or comments about this case study, or suggestions for further pieces, please contact the technical editor. If you wish to talk about the case study with other subscribers, please visit our forum here.


Building .NET Reporting Solutions with XML and GDI+

Building .NET Reporting Solutions with XML and GDI+
http://www.csharptoday.com/content.asp?id=1796
Article Rating: 4
Published on January 9, 2002
Abstract
Customizing report solutions is far easier today with GDI+ than ever before – and with the increased ease of use of XML, and the elegant way that data sources can be bound to web controls, we can pretty much tap into any source of data with ease. Combine the two and report solutions can be tailored very easily to suit the need of any business. In this article, Richard Conway creates a C# and ASP.NET application that takes an XML data source and, based on user choices, auomatically creates a bar chart, pie graph, or line chart of the user selected data.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

Whilst the number of changes between Visual Studio 6 and Visual Studio .NET are vast, so are the number of new applications of Visual Studio .NET. One area that has always been the domain of a specialist is graphics programming - well, with GDI+ many of the repugnant graphics API calls have been made redundant. This example brings together a whole heap of .NET technologies and allows us to create graphs on the fly from an XML file.

Customizing report solutions is far easier now in .NET. With the increased ease of use of XML, and the elegant way that data sources can be bound to web controls, we can pretty much tap into any source of data with ease. With GDI+ we have the ability to create charts and graphs with precision plotting without much hassle. Combine the two and report solutions can be tailored very easily to suit the need of any business.

This article consists of a core C# dll, an ASP.NET UI and an XML data source as a backing store for the report. There are many ways to retrieve and use the XML. One common way is a use of the System.Xml namespace. A more elegant solution however is to use attributes and a class hierarchy that matches our XML node hierarchy. XML Serialization is one of the truly remarkable features of the CLR. We can start by introducing a tool called xsd.exe, which can be found in the bin directory of the .NET Framework SDK.

It's worthwhile remembering at this point that the use of XML files is not exclusive. I have chosen it since it is the most robust format for retrieving data. I have used a prescribed XML format, which would need to be followed (as you'll see below). I could, for example, instead of using a file, use a dataset and shape the XML to the prescribed format, or I could use a SQL Server stored procedure and (in version 2000) return a resultset shaping the output to something similar to that which will be presented below.

Using Xml

Here is our sample block of XML data, on which we will report:

Report.xml

<REPORT>
<STATISTICS UNITS="%" YEAR="1998">
<VALUE>2.4</VALUE>
<VALUE>0.1</VALUE>
<VALUE>9.6</VALUE>
<VALUE>0.2</VALUE>
<VALUE>2.1</VALUE>
<VALUE>0.1</VALUE>
<VALUE>0.3</VALUE>
<VALUE>0.4</VALUE>
<VALUE>2.0</VALUE>
<VALUE>1.0</VALUE>
<VALUE>1.1</VALUE>
<VALUE>0.9</VALUE>
</STATISTICS>
.
</REPORT>

This is a simple block of XML, which takes 12 values corresponding to months in the year - we will model our custom server control around these monthly values. Run xsd.exe in the command line:

xsd C:\report.xml

This will produce an .xsd file in the C:\ root directory. Further use of the tool will create a set of classes, which can be populated, serialized, and deserialized directly from the XML document. As follows:

xsd /c C:\report.xsd

Try opening this file in Visual Studio .NET. The results are remarkable - what you'll see is effectively a database schema. In effect this is what the XSD file is. It's a schema (or ruleset) for the XML block we've shown above. Visual Studio .NET displays this in a tabulated form, allowing us to immediately see the relationships between discrete parts of XML (in the same way that a database will allow us to see the primary and foreign key constraints of a table).

The XSD file represents the description (schema) of the XML file - it can be modified to allow the XML to have certain constraints in terms of data types and the way that elements are sequenced within the XML document. Unfortunately the code produced through the Beta 2 version of this tool is not always optimal, but we can use the output of C:\report.cs as a starting point for our project (the RC1 version of this tool corrects many of the problems that were present in previous version).

The following code was produced via the XSD tool:

[System.Xml.Serialization.XmlRootAttribute("REPORT", Namespace="", IsNullable=false)]
public class REPORT
{

/// <remarks/>
[System.Xml.Serialization.XmlElementAttribute("STATISTICS")]
public REPORTSTATISTICS[] Items;
}

This is the top-level class, which contains collections of other classes. It represents the start of an object hierarchy, which is consistent with the parent-child relationship of the XML. The attributes that accompany the code are used to serialize and de-serialize the XML from and to these classes. XmlSerialization classes have been written within the .NET Framework to take advantage of attributes and reflection so these classes will undergo introspection at runtime in order to populate these classes. Unfortunately I don't have enough space to delve into the finer points of XML, but Professional C# (http://www.amazon.com/exec/obidos/ASIN/1861004990/ref=nosim/csharptoday-20/107-5604225-4012523?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq) by Wrox Press provides some fantastic examples in Chapter 15.

/// <remarks/>
public class REPORTSTATISTICS
{

/// <remarks/>
[System.Xml.Serialization.XmlElementAttribute("VALUE", IsNullable=true)]
public REPORTSTATISTICSVALUE[] VALUE;

/// <remarks/>
[System.Xml.Serialization.XmlAttributeAttribute()]
public string UNITS;

/// <remarks/>
[System.Xml.Serialization.XmlAttributeAttribute()]
public string YEAR;
}

/// <remarks/>
public class REPORTSTATISTICSVALUE
{

/// <remarks/>
[System.Xml.Serialization.XmlTextAttribute()]
public string Value;
}
}

This marks the hierarchy that we discussed. In effect the object model looks like the XML structure. We start to see how tightly coupled XML is to the .NET Framework.

Reporting on a Wizard

Step 1

Fig 1 - Step 1 of the Report Wizard

Step one of this Report Wizard uses a simple file upload to get the data from a local file on a client PC to the web server. I can browse to the file, and when selected press continue.

//Read the file contents into a stream
StreamReader stReadReportFile = new StreamReader(Request.Files[0].InputStream, Encoding.UTF8);
//insert into the cache - easiest way to store for short amount of time
Session.Add("fileContents", stReadReportFile.ReadToEnd());
//Let's goto the next wizard page
Server.Transfer("ReportType.aspx");

The above code executes when we press the Continue button. It reads in the file from the ASP.NET Request object which each page (and anything in the context of the Web) has access to. It reads the file in as stream and into a StreamReader object. This is then changed into a string and stored as a Session variable. I, and probably many others reading this, had a great fear of using SessionState in previous versions of ASP. This was most notably because business systems are built on web farms, which can contain more than one web server. A Session is held in memory, which would mean that it would need to be "sticky" to be able to use with confidence, or another means of session management would have to be used with a shared database as a backing store. Well, there is no need to fear the use of SessionState anymore, as SQL Server 2000 can be intrinsically set in ASP.NET to store Session information allowing complete scalability. It's beyond the scope of this article but has an obvious entry in web.config, which must be configured properly. The final statement is a transfer to another page. I prefer using these to Response.Redirect() in ASP as they don't involve a client-server interchange (to get the client to download a different page - other than the requested one).

Step 2

If we consider the page_load event on the next page:

EnableViewState = false;
// Put user code to initialize the page here
//Create the reportextract object and later return the collection that we've created
String entireReport = Convert.ToString(Session["fileContents"]);
ReportExtract reGenXml = new ReportExtract(entireReport);
//Populate a list box with all the years so that one can be chosen
foreach(REPORTSTATISTICS rsPresentYear in reGenXml.ReportStatisticsCollection)
{
lstDropYearList.Items.Add(rsPresentYear.YEAR);
}
//Better to replace the session variable with an object now to save further lookup work
Session.Remove("fileContents");
//Re-add it as the object value
Session.Add("fileContents", reGenXml);

So, from the top down, the page first turns off viewstate since we don't want to remember any page state between requests. This would inadvertently add duplicate entries into the DropDownListBox. We convert the report contents to a string from a session variable (as it is cast to object by default) and create a ReportExtract object to create an object model representation of the XML. We'll look at the deserialization code that it produces a little later.

We can enumerate the collection of REPORTSTATISTICS objects stored in the ReportExtract object and add their YEAR property to the DropDownListBox control which will hold all the textual values of the years. Once this is done, we can remove the contents of the Session variable "fileContents " and reinsert the variable but with a different object - this time a ReportExtract object which has a copy of the REPORT object as it's member variable. When this page loads again, the same code will execute so the ReportExtract class must override the ToString() method of the object. This is interesting as it serializes the XML back in exactly the same form it was before we used the XML element-attribute values to populate the fields of our REPORT object hierarchy.

The constructor for the ReportExtract class contains the following:

//Load the xml into a reader again!! and then deserialize contents into REPORT object
XmlSerializer xsReportDetails = new XmlSerializer(typeof(REPORT));
m_ReportObject = (REPORT)xsReportDetails.Deserialize(new StringReader(xml));

This is a very simple means of moving between a series of objects, which contain collections of other objects and XML. An XmlSerializer object is created (which takes a type argument) and an XML string is deserialized to allow a REPORT object to be created and have its initial state defined by the contents of this document. I'm going to skip a great deal of code throughout this project (for brevity) so I'll focus on the things, which are most important. I would encourage everybody to look at the code distributed with the download to understand how this project works in its entirety.

The ToString() method I mentioned before is overridden as follows to allow a string cast from an object to serialize the object REPORT back to XML.

public override string
            ToString()
{
//Create the familiar serialization
            object
XmlSerializer xsReportDetails = new
            XmlSerializer(typeof(REPORT));
//Create a mem stream to
            temporarily hold it
MemoryStream memReportDetails = new
            MemoryStream();
//Serialize the repomemReportDetailsrt from the
            member variable
xsReportDetails.Serialize(memReportDetails,
            m_ReportObject);
//Read in the stream
memReportDetails.Seek(0, SeekOrigin.Begin);
//use overriden
            method
StreamReader srReportDetails = new
            StreamReader(memReportDetails);
//Return all text
return
            srReportDetails.ReadToEnd();
}

This should have a familiar ring to it now. We create the same XML Serialization object and further create a MemoryStream to hold the contents of the Serialized XML. The contents of the member variable referencing the REPORT object are then placed into the MemoryStream.

We have to reset the Stream pointer back to the beginning in order to read the contents (which is done through StreamReader). The Contents are then returned from the ToString() method in the form of a string.

Fig 2 - This show the second page of the wizard

We ask the report user which graphs he/she wants to generate and for which years. I modeled a file reportfile2.xml, which I've included with the download. It reads the years from this file and displays them on screen (in the DropDownListBox). We have the option of generating a Text Table, a Bar Chart, a Pie Chart, a Line Graph and a Multi-Value line graph, which can be used to compare values over all the years listed in the list box (at least those for which we have data). To finalize this screen we'll be asked what to call our report.

REPORTSTATISTICS rsOurReport = null;
string directoryName = String.Empty;
//Here we can actually generate the report
ReportExtract rsColReportVals = (ReportExtract)Session["fileContents"];
foreach(REPORTSTATISTICS rsChosenReport in rsColReportVals.ReportStatisticsCollection)
{
//Lets find a match if(lstDropYearList.SelectedItem.Text.Equals(rsChosenReport.YEAR))
{
//Add the Create the serverreport directory

When the Continue button is clicked the above code is executed. This will enumerate the REPORT object collection which is a member variable of ReportExtract. This will try to find a year match with the selected year (from the list box).

Directory.CreateDirectory(@"C:\inetpub\wwwroot\ReportDirectory\"+txtReportName.Text+@"\"+rsChosenReport.YEAR+"\\");
directoryName = @"C:\inetpub\wwwroot\ReportDirectory\"+txtReportName.Text+@"\"+rsChosenReport.YEAR+"\\";

When this match is found, a directory is created as above with a path of the web directory followed by the report name and the year. The files are stored under this for accessibility.

ReportGenerator rgReportStats = new ReportGenerator(rsOurReport, chkLineGraph.Checked,
chkBarChart.Checked, chkTable.Checked, chkPieChart.Checked, chkMultiLineGraph.Checked,
rsColReportVals.ReportStatisticsCollection, directoryName);

Finally, a report generation object is created which will aggregate all the decisions made so far and create graphing objects, which use GDI+. We will look at these now.

Creating the Graphs

The Bar Graph

The following creates the bar graph by calling the CreateBarGraph() method, which is invoked on creation of a BarGraph object in the ReportGenerator like so:

BarGraph bgReportGraph = new BarGraph(directory, maxValue, rpReportSolution);

The BarGraph inherits from a class called GenericGraph (as do all other graphs in this project). GenericGraph defines a set of common fields and methods which all graphs use, including heights and offsets (as described below).

CreateBarGraph() is described below - it is responsible for creating the bar graph and applying to it styles and spacing to define what kind of graphic emerges.

Bitmap bmp = new Bitmap((int)gWidth+OFFSET_GRAPHWIDTH,
(int)gHeight+OFFSET_GRAPHHEIGHT,
PixelFormat.Format32bppArgb);
Graphics g = Graphics.FromImage(bmp);
g.FillRegion(Brushes.White, new Region(new Rectangle(0, 0,
(int)gWidth+OFFSET_GRAPHWIDTH,
(int)gHeight+OFFSET_GRAPHHEIGHT)));

The above code demonstrates what little effort is required to create a graphic using GDI+. We create a Bitmap graphic with width and height defined by properties that can be set as attributes on the server control tag (this will be described later). Here we've defined that we also want to create a 32 bit graphic. A Graphics object is created using the static FromImage(bmp) method. This gives us a handle to the underlying graphic object - in essence it will allow us to add all types of shapes and color effects. OFFSET_GRAPHWIDTH and HEIGHT represent the spacing needed to add text to describe the graph axes. The FillRegion() method allows us to choose a brush color and define coordinates which can be used to paint a background color on the graphic. In this case we use white just to be consistent with the web page background color. All of the work is done in the following segment of code.

for(float i = 0; i < totalCount;
            i++)
{
if (i == 0) initWidth = (float)((i + 1) / totalCount)
            * gWidth;

At this point we are looping through the values that we've just put into the table. On the first iteration we calculate the width of each bar by working out each bar as a proportion of the total width of the graph.

float x = (float)((((i + 1) / totalCount) * gWidth) -
(float)(1 / totalCount) * gWidth) + OFFSET_GRAPHWIDTH;

Above is the calculation for the x coordinate of the current bar. The x coordinate is read from the left corner of the graphic - here we have to work out what the accumulated width of all the bars is and then add the offset value, which we set earlier (to allow space for x-axis text descriptions).

float height = (float)(((allValues[(int)i] / maxValue) * gHeight));
float y = (float)(gHeight - height) + OFFSET_GRAPHHEIGHT;

The y value coordinate is somewhat more difficult to calculate. It begins from the top of the graphic and we have to subtract the height of the bar from the total height (and then add the offset to provide space to write text at the top of the graph). The height is calculated by taking the value as a proportion of the previously calculated maximum value (this is the value that we set as the scale maximum i.e. 100% of the height of the graph) - from this value we can work out the height of the bar.

   RectangleF rect = new
            RectangleF(x, y, initWidth,
            height);
g.FillRectangle(Brushes.OliveDrab,
            rect);
g.DrawRectangle(new Pen(Color.Black), rect.X, rect.Y,
            rect.Width, rect.Height);
}

This last piece of code above ends the loop by actually drawing a rectangle on the screen in the place of each bar. A Rectangle object is created using the x & y values calculated before. I've chosen to use an OliveDrab brush to color fill the rectangle, but typically you can use any number of different colors defined as Brushes class constants. The outline of the Rectangle is drawn in black to make the bar stand out on the white background.

private void CreateAxis(float width, float height, Graphics g, string heightToolText)
{
g.DrawLine(new Pen(Color.Black, 2), new Point(OFFSET_GRAPHWIDTH, (int)height+OFFSET_GRAPHHEIGHT),
new Point((int)width+OFFSET_GRAPHWIDTH, (int)height+OFFSET_GRAPHHEIGHT));
g.DrawLine(new Pen(Color.Black, 2), new Point(OFFSET_GRAPHWIDTH, OFFSET_GRAPHHEIGHT),
new Point(OFFSET_GRAPHWIDTH, (int)height+OFFSET_GRAPHHEIGHT));
g.DrawString(heightToolText, new Font("tahoma", 10), Brushes.Black, 0, 10);
g.DrawString(maxValue.ToString(), new Font("tahoma", 10), Brushes.Black, 0, OFFSET_GRAPHHEIGHT+10);
g.DrawString((maxValue/2).ToString(), new Font("tahoma", 10), Brushes.Black, 0, (height/2)+OFFSET_GRAPHHEIGHT);
g.DrawString("0", new Font("tahoma", 10), Brushes.Black, 0, height);
}

The CreateAxis method is called within the previous method using the invocation CreateAxis (gWidth, gHeight, g, "%age"). The width, height, Graphic object and the y-axis text are passed to it. From this information we can draw the graph x and y axis lines by using a black pen, with the width and height having the offset values we discussed previously. In addition it adds the text ' %age ' (in this case) to the top of the graph. The DrawLine() method (of the Graphic object) takes a Pen object argument and two points to be able to effectively draw the line. The DrawString() method takes a string value to write to the graphic and x and y coordinate (it also takes a font object which we've set as 10pt Tahoma). The last 3 lines of code add the uppermost value of the graph, the middle value (half the size of the upper most value) and zero, which defines the lowest value. The code could be modified to allow a negative value to be the lowest - negative values are unnecessary, however, if statistics are adjusted accordingly to compensate (it would also require a little more work to create rectangle bars which are drawn and painted under the x axis as opposed to above).

The bmp is then saved as follows:

graphName = "bargraph.jpg";
bmp.Save(directory+graphName, ImageFormat.Jpeg);

Fig 3 - Bar Graph of XML statistics

The Pie Chart

The pie chart is somewhat easier to model than the bar graph. The CreatePieChart() method does most of the work delegating to a series of private methods, which will be described in detail.

Bitmap bmp = new Bitmap((int)gHeight + 300, (int)gHeight, PixelFormat.Format32bppArgb);
Graphics g = Graphics.FromImage(bmp);
g.FillRegion(Brushes.White, new Region(new Rectangle(0, 0, (int)gHeight+300, (int)gHeight)));

The pie chart is constructed initially like the bar graph. We define a greater offset however (of 300) so that the legend can fit onto graphic. The height is used in this case to define the height and the width, as ideally we want a circular rather than elliptical pie chart.

for(float i = 0; i < totalCount;
            i++)
{
PieBrushes allColour = new
            PieBrushes();
FillArc((allValues[(int)i] / totalAddition), g,
            allColour.getBrush((int)i));
}

We won't explain the PieBrushes struct here, but suffice it to say that it contains 12 colors, which present each month so that each arc on the pie chart can be identified distinctly (the code for the struct can be found in the code downloads section of this article). Calling the FillArc method fills the arc, and we pass it as parameters the proportion of the total circle to fill, the Graphic object, and a unique colored brush. FillArc is shown below:

private void FillArc(float section, Graphics
            g, Brush br)
{
g.FillPie(br, 0, 0, gHeight, gHeight,
            lastAngle, section * 360);
lastAngle += section *
            360;
}

This works out a section as a proportion of the whole circle and plots it within the graphic. The lastAngle variable contains the last angle that the previous pie reached from 0 to 360 degrees. This is used to plot the arc.

Finally we draw the circular border in black pen around the series of connected arcs.

g.DrawEllipse(Pens.Black, 0, 0, gHeight, gHeight);

In the same way as you might find through Microsoft Graph (which is most commonly known through charting with the Excel wizards), I decided that it would be better to create a legend for the pie chart instead of writing on each slice of pie. The legend represents a small block of color (5 pixels by 5 pixels) to the month with which the colored arc represents.

private void CreateLegend(Graphics
            g)
{
PieMonths p = new PieMonths();
PieBrushes allColour =
            new PieBrushes();
g.DrawRectangle(Pens.Black, gHeight + 100, 0,
            100, (gHeight / 3)*2);
for(int i = 0; i < totalCount;
            i++)
{
g.FillRectangle(allColour.getBrush(i), gHeight + 120,
            (i * 20) + 8, 5, 5);
g.DrawString(p.getMonths(i), new
            Font("tahoma", 10), Brushes.Black, gHeight + 130, i *
            20);
}
}

A rectangle is drawn first to allow the legend to be encapsulated in a black border. Then we can get all the values from the PieMonths struct, which contains the month names and the values from the PieBrushes struct. We can then use FillRectangle to fill the 5 by 5 colored rectangle and the DrawString method to write the string name of the month next to the colored rectangle.

This is then saved in the same way as the last graphic and a corresponding HTML image object is created (which we will discuss later).

Fig 4 - Result of using Brushes and colors for a Pie Chart

The Line Graphs

public class LineGraphMulti : LineGraphSingle
{
public LineGraphMulti(ArrayList Reports, string directory, float maxValue,
REPORTSTATISTICS rpStatsVal) : base(directory, maxValue, rpStatsVal)
{
//Create a bitmap graphic which will set the height and width
Bitmap bmp = new Bitmap((int)gWidth+OFFSET_GRAPHWIDTH, (int)gHeight+OFFSET_GRAPHHEIGHT,
PixelFormat.Format32bppArgb);
Graphics g = Graphics.FromImage(bmp);
//Fill the region with rectangle white so that we can define a chart area
g.FillRegion(Brushes.White, new Region(new Rectangle(0, 0, (int)gWidth+OFFSET_GRAPHWIDTH,
(int)gHeight+OFFSET_GRAPHHEIGHT)));

graphName = "multilinegraph.jpg";
int i = 0;

Rather than refer to both line graphs (since their code is very similar) I've decided to just refer to the Multi-Value line graph. This graph is created and has the LineGraphSingle class as its base class (which in turn has GenericGraph). As in all other graphs mentioned thus far, we create a Graphics object and use this to generate the graph, whilst using the graphic to create a jpeg from a Bitmap object.

   foreach(REPORTSTATISTICS
            rpSingleUnit in Reports)
{
this.rpStatsVal =
            rpSingleUnit;
LinePens lpPensCollection = new
            LinePens();
CreateLine(g,
            lpPensCollection.getPen(i));
i++;
}
CreateAxis(gWidth,
            gHeight, g, "%age");
bmp.Save(directory+graphName,
            ImageFormat.Jpeg);
}

LinePens is another struct somewhat similar to PieBrushes. With this we are returned a variety of pens to enable us to select a new pen color based on index from 1 to 12. Here we create the graph axis in the same way as before and save the graphic. The only real variation is the CreateLine method, which is shown below:

public new void CreateLine(Graphics g, Pen
            penColor)
{
float x = (float)(((1 / totalCount) * gWidth) -
            (float)(1 / totalCount) * gWidth) + OFFSET_GRAPHWIDTH;
float
            height = (float)(((float.Parse(rpStatsVal.VALUE[0].Value) /
            maxValue) * gHeight));
float y = (float)(gHeight - height) +
            OFFSET_GRAPHHEIGHT;
Point lastPoint = new Point((int)x,
            (int)y);
//Loop through the total number of values
for(float
            i = 1; i < totalCount; i++)
{
x = (float)((((i + 1) /
            totalCount) * gWidth) - (float)(1 / totalCount) * gWidth) +
            OFFSET_GRAPHWIDTH;
height =
            (float)(((float.Parse(rpStatsVal.VALUE[(int)i].Value) / maxValue) *
            gHeight));
y = (float)(gHeight - height) +
            OFFSET_GRAPHHEIGHT;
Point point = new Point((int)x,
            (int)y);
g.DrawLine(penColor, lastPoint, point);
lastPoint =
            point;
}
}

We've come across the calculations already when discussing bar charts, so won't delve into this again. However, in this line graph example we use Point objects to obtain a point either side of the line and then call DrawLine() on the Graphics object with the information. The lastPoint is recorded in a local variable so that it can be used to plot the first line point in the call to DrawLine(). This method is called as many times (via the LineGraphMulti object) as there are REPORTSTATISTICS objects. Further inspection of the code will reveal this.

Fig 5 - Multiple line graph

Now we've looked at the graphs individually it is time to see how this is all tied together. The majority of the "glue" code - as discussed - is in the ReportGenerator object. It is this that feeds our choices from the web and converts them into useful graphs.

maxValue = 0;
for(int j = 0; j <
            ReportList.Count; j++)
{
for(int i = 0; i <
            ((REPORTSTATISTICS)ReportList[j]).VALUE.Length; i++)
{
//Work
            out the maximum value here using Max function
maxValue =
            Math.Max(float.Parse(((REPORTSTATISTICS)ReportList[j]).VALUE[i].Value),
            maxValue);
}
}
LineGraphMulti mlReportGraph = new
            LineGraphMulti(ReportList, directory, maxValue,
            rpReportSolution);

This code works out the maximum value of the all of the lines in a multiline graph. Even if this was a single line graph, bar graph or pie chart. Since the values we are using are proportional to the largest value, and the y-axis on the graph is scalable, we will need to know the maximum value from our sample of values. The Max function in the Math class is a static method. This is a method that is common to virtually all languages. It simply compares two numbers and returns a copy of the larger. With the multi-line graph this must be tested on all the numbers from the REPORTSTATISTICS collection.

html.Append("<image src=\"");
html.Append(mlReportGraph.graphName);
html.Append("\" border=\"0\"><br>");

With every graph that is created and saved as an image, a StringBuilder class is appended too, so that we have an HTML file containing all the images generated, and an HTML table with all the values in.

Fig 6 - the page index.html showing the table

The HTML is then created as:

html.Append("</body></html>");
FileStream
            fwReportFile = new FileStream(directory+"index.html",
            FileMode.Create, FileAccess.Write);
StreamWriter swReportFile =
            new
            StreamWriter(fwReportFile);
swReportFile.Write(html.ToString());
swReportFile.Close();

using a FileStream object and a StreamWriter object to write the string from the StringBuilder object into a file.

Sometimes it may be necessary to generate reports dynamically and stream them to the browser. Rather than saving them in a physical file we could use the above code to create a MemoryStream and stream the image to the browser. This makes for a quick and fast "on-the-fly" customized image.

<%MemoryStream ms = new MemoryStream();
Bitmap.Save(ms, ImageFormat.Jpeg);

Response.ContentType = "image/jpeg";
Response.BinaryWrite(oStream.ToArray());%>

Conclusion

In this example, I have tried to automate a simple task that many of us have needed to do for our employers. Sometimes we have been unable to this due to time constraints and have had to buy in third party products such as ChartFX. However, many of the new features of Visual Studio .NET enable us to fairly easily create custom solutions like the one just described. One way of extending this would be to use the article on Add-Ins (http://www.csharptoday.com/content.asp?id=1619&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq) to create a wizard enabling us to bind to a data source and develop charts based on any data source. A step further may be to introspect an instance of SQL server to create graphical representations of current data, and use OLAP to provide data to create graphs that could represent future trends.

If you'd like to discuss a broader range of C#Today topics and issues, or if you have any C# related questions you need answers to, please visit our online resource centres at : http://p2p.wrox.com/list.asp?list=csharptoday_discuss&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq and http://p2p.wrox.com/list.asp?list=c_sharp&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq


Owner Drawn Menus in Visual Studio .NET Using C#

Owner Drawn Menus in Visual Studio .NET Using C#
http://www.csharptoday.com/content.asp?id=1747
Article Rating: 4
Published on December 28, 2001
Abstract
If you’ve ever attempted to create your own menus in a Windows Application environment, you know how hard it is to “Owner Draw” anything. The idea sounds so straightforward though, doesn’t it? HAll we want to do is subclass the menu and provide our own code that determines the look and feel/functionality! Well, if you knew C++ you could do it with a little pain, black magic and some salt. VB too, if you wanted to mess around with same ugly API’s such as a C++ developer would deal with. One would think that Microsoft would have come up with an easy way to do such a thing. We could pay some money for a third party OCX that would allow you some control over the menus, but that’s no fun if we don’t have the source code. Luckily for us, this all changes with .NET. In this article, Andrew Krowczyk shows us how easy it is to provide a slick enhanced menu system.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

If you've ever attempted to create your own menus in a Windows Application environment, you know how hard it is to "Owner Draw" anything. The idea sounds so straightforward though, doesn't it? Hey I just want to subclass the menu and provide my own code that determines the look and feel/functionality! Well, if you knew C++, you could do it with a little pain, black magic and some salt. VB? Yeah, if you wanted to mess around with ugly API's (such as a C++ developer would deal with). One would think that Microsoft would have come up with an easy way to do such a thing. I suppose that you could pay some money for a third party OCX that would allow you some control over the menus, but that's no fun if you don't have the source code. Luckily for us, this all changes with .NET. In this article we'll see how easy it is to provide a slick enhanced menu system.

In particular, I really like the look of the Menus that are used in the Visual Studio .NET IDE. I've also seen this same type of menu in the latest version of XML Spy and a few other applications. Why Microsoft didn't provide a nice way for us to flip a switch in our Windows Forms code to make our menus look the same I don't know why. I do have to say that if you develop an app and run it on Windows XP, the menus do look a bit different than normal, but what about apps when they are run on Windows 2000?

For sake of clarification, let's take a look at exactly what I'm talking about. Here we see a Visual Studio .NET menu:

By the end of the article, we'll see how to turn our basic Windows Form menu from this default style:

Into this custom style:

One thing you might notice about the second image above is that it doesn't look exactly like the Visual Studio .NET menu as pictured earlier in the article. Unfortunately Owner Drawn Menus are "tacked" on to the existing menu that is provided by the Windows Form class. Because of this, we can't exactly simulate the borderless look and shadow effect of the Visual Studio .NET IDE menus. More than likely, the Microsoft .NET team is using their own basic Menu APIs and such to make the menus look the way they do. Unfortunately they don't expose that functionality to us, so we can only come so close to simulating the IDE's menus. All in all, I think the results are pretty close.

System Requirements

This article assumes that you have some familiarity of C# and the Visual Studio .NET development environment and have produced one or two simple Windows Form applications using these technologies. This code should work equally well on Beta 2 and RC 1.

What are Owner Drawn Menus Anyway?

Well, disregarding the fact that the name gives it away, an Owner Drawn menu is a menu in which the application or "Owner" of the menu instance provides its own GUI drawing/rendering code, in contrast to letting Windows draw the menu. In essence the Owner is simply subclassing the menu and overriding the code used to create the graphical interface part of the menu.

Why would we want to do such a thing? Well, first of all, the menus that are provided in any programming language for Windows don't give you much of anything when it comes to menu look/appearances. So, if you have something special in mind for the look and feel of your menus then Owner Drawing is the only way to go.

What about Windows XP? Windows XP "Windows" drawn menus look different to Windows 2000 menus, but they are still just standard menus, and we really don't have much control over them.

For example, here is what a simple menu looks like on a Windows XP system with the Blue XP Theme, and the same menu on a Windows 2000 machine:

So what is the point of discussing XP vs. Win2K? The point is that the Menu drawing code is very much an Operating System procedure. In XP, the OS draws menus differently to Win2K by default (because of the theme like nature of XP). Our owner drawn menus will work in either Operating System, as we will be taking control of the drawing mechanism.

So now that we've decided that we'd like to take a stab at creating Owner Drawn menus, let's see what we need to do.

How does .NET Help Us?

By now you are all probably thinking that gee, .NET must provide a way to Owner Draw a menu. Well, yes it does. This is a factor of a couple things. First off, .NET gives us some great namespaces with methods that we can use to draw things by ourselves. For instance, we have the System.Drawing and System.Drawing.Text namespaces that give us control of many functions that allow us to actually draw our menus.

We are also allowed to use great Object Oriented Programming (OOP) techniques such as Polymorphism (to override method definitions) and Inheritance. Although these are more a byproduct of the programming languages used, rather than the .NET Framework.

Now that we know that there are a few things in .NET that can help us, let's see how that translates into the code used in our menu namespace DotNetMenu.

DotNetMenuItem Class

There are a few different bits of code in the DotNetMenu namespace that are of interest to us, but since we are focusing on the Owner Drawn implementation part, we'll skip over a few of the other methods included in the class, as they don't really have a huge importance as to the overall understanding of the class. Ok, so let's begin!

When we take a look at the DotNetMenu namespace, we notice that there are 2 classes defined within the code - DotNetMenuItem, and ThemeColors. As we might expect, the DotNetMenuItem is the class that inherits and overrides the MenuItem class provided by the .NET framework. The ThemeColors class is just something that is used to get the correct colors for the menu structure when different themes are used in Windows. This is a somewhat hacked implementation as there isn't a method that allows us to figure out what colors to really use for the Owner Drawn menu attributes based on a particular theme, so in this case we just give a best guess instead of hardcoding 1 color combination for the whole lot of menus. If anyone can think of a better way to get this color scheme information, please do email me with your idea! That being said, we aren't going to focus much on the ThemeColors class, we'll just assume that it's a class that gives us the correct RGB color values for such items as Menu Highlighting and such.

For the sake of brevity, in some cases, we'll leave out some of the cluttering code in the listing displayed within the article. Of course all of the code is included in the sample download application, and organized for clarity!

The DotNetMenuItem class in its basic form, inherits from the MenuItem class which is a member of the System.Windows.Forms namespace:

public class DotNetMenuItem :
            MenuItem
{

// constructors
//
            ---------------------------------------------------
public
            DotNetMenuItem(string name, EventHandler handler, Shortcut shortcut)
            : this(name, handler)
{
this.Shortcut =
            shortcut;
}

public DotNetMenuItem(string name,
            EventHandler handler) : base(name, handler)
{
//set the owner
            draw flag equal to true
//since we are going to owner draw these
            menus
OwnerDraw = true;
}

//default
            constructor
public DotNetMenuItem() : base(){ OwnerDraw =
            true;}

When we are talking about owner drawn menus there are a few events that are very important to notice. These are the OnMeasureItem and DrawItem event handlers.

If we jog our memories related to Windows programming and message based programming you will remember that Windows programs rely on a system of messages being passed back and forth. If you want to handle those messages, you need to write a Windows Procedure that will listen to the messages that Windows is sending, and then you can react and do something when a particular message comes across. Let's say for example, a button click.

In the realm of Owner Drawn items, there are two very important Windows messages that we need to know about. These are WM_MEASUREITEM (http://msdn.microsoft.com/library/default.asp?url=/library/en-us/winui/combobox_85d9.asp&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq) and WM_DRAWITEM (http://msdn.microsoft.com/library/default.asp?url=/library/en-us/wcesdkr4/htm/_wcesdk_win32_wm_drawitem.asp&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq). Let's take a look at these methods.

OnMeasureItem

As you may remember, the WM_MEASUREITEM message is sent to the owner of an owner drawn menu before it is displayed. Basically this message allows Windows to figure out the size of the control/menu before it sends out the WM_DRAWITEM message. As you will see, the code in this method does a few things to determine the final height and with of the MeasureItemEventArgs argument coming in.

Follow along with the comments in the code snippet below to see what is going on in the code.

//override the OnMeasureItem
            method
protected override void OnMeasureItem(MeasureItemEventArgs
            e)
{
base.OnMeasureItem(e);

// measure shortcut text if
            it's there
// this takes into consideration all of the things
            that
// use menu shortcuts that need to be displayed
// in
            the actual menu text
if (Shortcut != Shortcut.None)
{
string text = "";
int key = (int)Shortcut;
int ch =
            key & 0xFF;
if (((int)Keys.Control & key) > 0)
text
            += "Ctrl+";
if (((int)Keys.Shift & key) > 0)
text +=
            "Shift+";
if (((int)Keys.Alt & key) > 0)
text +=
            "Alt+";

if (ch >= (int)Shortcut.F1 && ch <=
            (int)Shortcut.F12)
text += "F" + (ch - (int)Shortcut.F1 +
            1);
else
{
if ( Shortcut == Shortcut.Del)
{
text +=
            "Del";
}
else
{
text +=
            (char)ch;
}
}
shortcuttext = text;
}

//take into
            consideration if we want to actually just show a
//menu seperator
            bar
if (Text == "-")
{
e.ItemHeight = 8;
e.ItemWidth =
            4;
return;
}

//if this is a top level menu, we don't
            show a shortcut
bool topLevel = Parent ==
            Parent.GetMainMenu();
string tempShortcutText =
            shortcuttext;
if ( topLevel )
{
tempShortcutText =
            "";
}
//measure the width of the text
int textwidth =
            (int)(e.Graphics.MeasureString(Text + tempShortcutText,
            SystemInformation.MenuFont).Width);

//measure the height of
            the text (plus some padding)
int extraHeight = 4;
e.ItemHeight
            = SystemInformation.MenuHeight + extraHeight;
//also take
            consideration of top level menu differences
if ( topLevel
            )
e.ItemWidth = textwidth - 5;
else
e.ItemWidth =
            Math.Max(160, textwidth + 50);

// save menu item
            height
itemHeight = e.ItemHeight;
}

Wow, that was interesting huh? Well, now we have the height and width for our MenuItem. The next message that would be sent by Windows would be the WM_DRAWITEM message. Of course we need to handle this message too.

OnDrawItem

The OnDrawItem event handler simply makes some calls to a few other methods that we need to draw the menu in our own way. Please notice that this method makes calls to: DrawBackground, DrawIcon, DrawCheckmark, DrawSeparator, and DrawMenuText. We are going to skip a discussion of the DrawCheckmark and DrawSeparator to try to not bore you with too much code.

//again, override the OnDrawItem of the
            MenuItem class
protected override void
            OnDrawItem(DrawItemEventArgs e)
{
if ( doColorUpdate)
{
DoUpdateMenuColors();
}

base.OnDrawItem(e);

Graphics
            g = e.Graphics;
Rectangle bounds = e.Bounds;
//determine if
            the item is selected
bool selected = (e.State &
            DrawItemState.Selected) > 0;
bool toplevel = (Parent ==
            Parent.GetMainMenu());
bool hasicon = Icon != null;
bool
            enabled = Enabled;

//call the drawbackground method with
            flags
DrawBackground(g, bounds, e.State, toplevel, hasicon,
            enabled);
//if there is an icon, draw it, if there is a checkmark
            draw it too
if (hasicon)
DrawIcon(g, Icon, bounds, selected,
            Enabled, Checked);
else
if (Checked)
DrawCheckmark(g,
            bounds, selected);
//of course we need to handle the separator
            bar
if (Text == "-")
{
DrawSeparator(g, bounds);
}
else
{
//this isn't a separator, so draw the actual
            text!
DrawMenuText(g, bounds, Text, shortcuttext, Enabled,
            toplevel, e.State);
}
}

As you can see, the OnDrawItem as implemented here simply makes calls to our other methods that do the actual drawing in the style ala Visual Studio .NET IDE. Now let's look at the code for some of those methods.

DrawBackground

As you might guess, this takes care of drawing the background of the menu. It's here that we also take care of the case in which the user is selecting the menu.

public void DrawBackground(Graphics g, Rectangle bounds, DrawItemState
state, bool toplevel, bool hasicon, bool enabled)
{
//set the selected variable depending on the state of the item
bool selected = (state & DrawItemState.Selected) > 0;

//if the item is selected, OR the item is in the MouseOver
//state, we want to show the menu item in a highlighted
//look and feel
if (selected || ((state & DrawItemState.HotLight) > 0))
{
if (toplevel && selected)
{ // draw toplevel, selected
bounds.Inflate(-1, 0);
g.FillRectangle(new SolidBrush(stripeColor), bounds);
ControlPaint.DrawBorder3D(g, bounds.Left, bounds.Top, bounds.Width, bounds.Height, Border3DStyle.Flat,
Border3DSide.Top | Border3DSide.Left | Border3DSide.Right);
}
else
{ // draw menuitem, selected OR toplevel, hotlighted
if ( enabled )
{
g.FillRectangle(new SolidBrush(selectionColor), bounds);
g.DrawRectangle(new Pen(borderColor), bounds.X, bounds.Y, bounds.Width - 1, bounds.Height - 1);
}
else
{
// if disabled draw menuitem unselected
g.FillRectangle(new SolidBrush(stripeColor), bounds);
//you may wonder what this whole STRIPE_WIDTH
//thing is. If you remember, the VISUAL STUDIO .NET style
//menu consists of a gray vertical bar and
//a white background for the rest of the menu
//this takes into account that STRIPE of gray
bounds.X += STRIPE_WIDTH;
bounds.Width -= STRIPE_WIDTH;
g.FillRectangle(new SolidBrush(bgColor), bounds);
}

}
}
else
{
if (!toplevel)
{ // draw menuitem, unselected
g.FillRectangle(new SolidBrush(stripeColor), bounds);
bounds.X += STRIPE_WIDTH;
bounds.Width -= STRIPE_WIDTH;
g.FillRectangle(new SolidBrush(bgColor), bounds);
}
else
{
// draw toplevel, unselected menuitem
g.FillRectangle(SystemBrushes.Control, bounds);
}
}
}

DrawIcon

public void DrawIcon(Graphics g, Image icon,
            Rectangle bounds, bool selected, bool enabled, bool
            ischecked)
{
//set the transparency color for the
            icon
Color transparentColor = Color.FromArgb(0, 128,
            128);
Bitmap tempIcon =
            (Bitmap)icon;
tempIcon.MakeTransparent(transparentColor);

int
            iconTop = bounds.Top + (itemHeight - BITMAP_SIZE)/2;
int iconLeft
            = bounds.Left + ( STRIPE_WIDTH - BITMAP_SIZE)/2;
//check if
            enabled, if not we'll draw it as disabled
if (enabled)
{
if (selected)
{
//here we are being a little bit
            sneaky. To simulate
//the slight drop shadow effect when a icon
            is
//moused over, we'll first draw the item as a
            disabled
//image slightly off from where the icon position
            really
//is, and then we'll draw the icon over it.
//this will
            give the effect desired!
ControlPaint.DrawImageDisabled(g, icon,
            iconLeft + 1, iconTop, Color.Black);

g.DrawImage(icon,
            iconLeft, iconTop-1);
}
else
{
//otherwise, we just
            draw the image icon
g.DrawImage(icon, iconLeft + 1,
            iconTop);
}
}
else
{
ControlPaint.DrawImageDisabled(g, icon, iconLeft + 1,
            iconTop, SystemColors.HighlightText);
}
}

Last but not least, we'll talk about the drawing of the menu text for our Owner Drawn menus.

DrawMenuText

public void DrawMenuText(Graphics g,
            Rectangle bounds, string text, string shortcut, bool enabled,
bool toplevel, DrawItemState state )
{
StringFormat
            stringformat = new StringFormat();
stringformat.HotkeyPrefix =
            ((state & DrawItemState.NoAccelerator) > 0) ?
            HotkeyPrefix.Hide : HotkeyPrefix.Show;

//in the case where
            the high contrast mode is on, we need to display
//white text
            instead of black
bool highContrast = false;
bool
            whiteHighContrast = false;
if ( SystemColors.Control.ToArgb() ==
Color.FromArgb(255,0,0,0).ToArgb() ) highContrast = true;
if
            ( SystemColors.Control.ToArgb() ==
Color.FromArgb(255,255,255,255).ToArgb() ) whiteHighContrast =
true;

//if menu is a top level, take out the &
//
            so that the menu text is centered
if ( toplevel )
{
int
            index = text.IndexOf("&");
if ( index != -1 )
{
text =
            text.Remove(index,1);
}
}

//determine the text width
int textwidth = (int)(g.MeasureString(text,
            SystemInformation.MenuFont).Width);
int x = toplevel ?
            bounds.Left + (bounds.Width - textwidth) / 2: bounds.Left + iconSize
            + 10;

int topGap = 4;
if ( toplevel ) topGap = 2;
int y
            = bounds.Top + topGap;

//create a new brush
Brush brush =
            null;

//figure out what color to use for the brush that's
            going to be
//used to draw the menu text
if
            (!enabled)
brush = new SolidBrush(Color.FromArgb(120,
            SystemColors.MenuText));
else if ( highContrast )
brush = new
            SolidBrush(Color.FromArgb(255, SystemColors.MenuText));
else
brush = new SolidBrush(Color.Black);

if (
            whiteHighContrast && ( (state & DrawItemState.HotLight)
            > 0 || ( (state & DrawItemState.Selected)
> 0
            && !toplevel )) )

brush = new
            SolidBrush(Color.FromArgb(255, Color.White));

//using the
            graphics.drawstring method, we pass in the text
//the font we ant
            to use, the brush (for color), the x and
//y coordinates and the
            string format
//this will draw the text for us
g.DrawString(text, SystemInformation.MenuFont, brush, x, y,
            stringformat);

// don't draw the shortcut for top level
            menus
if ( !toplevel )
{
// draw shortcut right
            aligned
stringformat.FormatFlags |=
            StringFormatFlags.DirectionRightToLeft;
g.DrawString(shortcut,
            SystemInformation.MenuFont, brush, bounds.Width - 10 , bounds.Top +
            topGap, stringformat);
}
}

And that's about it. We've now completed our task of creating an Owner Drawn menu. Don't you feel a sense of accomplishment and excitement? Well, maybe that's a little on the flaky side, but I do think Owner Drawn menus are very nice. Now let's take a brief look at the other part of the article, the XML configuration file. Towards the end of the article, we talk about the sample application and such. But if you run the sample code that's included with the article, you will see that the code that we just talked about above will render a menu that looks like this:

XML Config File

The menu implementation provided here uses a XML configuration file to load the menu settings for display. Although this isn't a necessary to use the DotNetMenuItem class, we'll briefly go over the architecture for sake of completeness of the sample application that is bundled with this article.

Let's take a look at a piece of the Menu.xml file that is included in the download:

<?xml version="1.0" encoding="UTF-8"
            ?>
<MainMenu>
<Menu Name="File" Shortcut=""
            Icon="" Checked="False" Enabled="True">
<Menu
            Name="&amp;New" Shortcut="CtrlN" Icon="" Checked="False"
            Enabled="True" />
<Menu Name="&amp;Open"
            Shortcut="CtrlO" Icon="" Checked="False" Enabled="True"
            />
<Menu Name="&amp;Close" Shortcut="CtrlC" Icon=""
            Checked="False" Enabled="True" />
<Menu Name="-"
            Shortcut="" Icon="" Checked="False" Enabled="True" />
<Menu
            Name="A&amp;dd Project" Shortcut="" Icon=""
Checked="False"
            Enabled="True">
<Menu Name="&amp;New Project..."
            Shortcut="" Icon=""
. . .

As you can see, the XML file is fairly straightforward and easily modified. The basic structure is more simply reduced to:

<!--formatted for readability ->
<MainMenu>
<Menu></Menu>
<Menu></Menu>
<Menu>
<Menu></Menu>
</Menu>
</MainMenu>

Where we can nest menus to provide submenus quite easily.

Processing the XML Config File

Displaying the contents of the XML file in the Windows Form application is pretty straightforward. In our main starting point of the Main.cs file, we set the .Menu item of the form to the result of a call to our CreateMainMenu method.

public static void Main(string[]
            args)
{
Form1 mainForm = new Form1();
mainForm.Menu = new
            MainMenu(CreateMainMenu());

Let's now take a quick look at the CreateMainMenu method. First we need to get the path of the XML document. From there we'll create a new XMLDocument object and load the XML file.

public static MenuItem[]
            CreateMainMenu()
{
string menuFile = Application.StartupPath +
            "\\" + "menu.xml";
XmlDocument doc = new
            XmlDocument();
doc.Load(menuFile);

Next we'll grab all of the nodes and start looping through them and calling the ProcessMenuItem on each element. This will take care of setting up the Main menu on the form.

XmlNodeList
            topMenus = doc.DocumentElement.ChildNodes;
MenuItem[] menuItems =
            new MenuItem[topMenus.Count];
int i = 0;
foreach ( XmlElement
            element in topMenus )
{
menuItems[i++] =
            ProcessMenuItem(element);
}

return
            menuItems;
}

ProcessMenuItem Method

>The ProcessMenuItem method takes the XML from the XML document and sets up the Menu as needed. As we noted above, this method is called for each XML Element in the configuration file. The code below is commented to make the reading of the function easier.

public static MenuItem
            ProcessMenuItem(XmlElement xmlelement)
{
//get the name for
            the menu
string menuName =
            xmlelement.Attributes["Name"].InnerXml;

if (
            menuName.IndexOf("&") != -1 )
menuName =
            menuName.Replace("&amp;", "&");

//here we grab the
            other attributes from the XML document
string menuShortcutString
            = xmlelement.Attributes["Shortcut"].InnerXml;
string menuIcon =
            xmlelement.Attributes["Icon"].InnerXml;
string menuChecked =
            xmlelement.Attributes["Checked"].InnerXml;
string menuEnabled =
            xmlelement.Attributes["Enabled"].InnerXml;

Shortcut
            menuShortcut = Shortcut.None;
//determine if there is a shortcut
            presented for this element
if ( menuShortcutString != "" )
{
menuShortcut =
(Shortcut)((Shortcut.F1.GetType()).InvokeMember(menuShortcutString,
            BindingFlags.GetField, null,
Shortcut.F1, new
            object[0]));
}

//create the parent menu as an Owner Drawn
            menu
DotNetMenuItem parentMenu = new DotNetMenuItem(menuName, new
            EventHandler(SelectMenuItem), menuShortcut);

//set up the
            menu icon, checked and enabled portions of the menu
if ( menuIcon
            != "" )
parentMenu.Icon = GetIcon(menuIcon);
if ( menuChecked
            == "True" )
parentMenu.Checked = true;
if ( menuEnabled ==
            "False" )
parentMenu.Enabled = false;

//now take a
            look to see if there are indeed child submenus for
            this
//particular menu item. If there are, then we'll recursively
            call
//the ProcessMenuItem method to generate the submenus
            accordingly!
XmlNodeList nodes = xmlelement.ChildNodes;
if (
            nodes.Count > 0 )
{
MenuItem[] menuItems = new
            MenuItem[nodes.Count];
int i = 0 ;
foreach ( XmlElement node
            in nodes )
{
menuItems[i] = ProcessMenuItem(node);
if (
            menuItems[i] !=
            null)
parentMenu.MenuItems.Add(menuItems[i]);
i++;
}
}
return
            parentMenu;
}

Although it isn't necessary to generate the menus this way, I've found it quite useful to be able to easily define the menu structures in an XML configuration file because of the ease of use and simplicity of the configuration file approach. Because of this, you could easily have more than one menu for your application, and you could switch between them in your application more easily than having a predetermined Menu structure defined via the IDE. It's up to you whether or not you use this approach, but as I tend to like this method.

Looking at the Sample Application

Now that we've covered the basics of the Owner Drawn menu and the XML configuration file, we'll take a look at the sample application in the download. As you can see, I tried to loosely simulate the File menu of the Visual Studio .NET IDE.

One thing of particular interest is the code used to simulate the raised shadow effect of the icons in the menu as the mouse hovers over the menu item (we talked about this briefly in a comment within the code discussion above). As you may or may not have noticed, there is only one bitmap for each of the icons, not 2 that get swapped as you might think. (Like a image rollover in a web type application). Instead we use a tricky .NET method called DrawImageDisabled. It is a member of the System.Windows.Forms.ControlPaint namespace. We call this method, and then call the DrawImage method for the icon. This results in 2 images, with the offset used in the DrawImageDisabled call simulating a dropped shadow type effect.

ControlPaint.DrawImageDisabled(g, icon, iconLeft + 1, iconTop, Color.Black);

As you can see, when the mouse is moved over the menu item, the icon seems "raised" off of the menu with a slightly gray drop shadow effect.

One more point to note. The menu items in the sample application are wired to message boxes that display the menu item chosen when clicked. As you might expect, this is accomplished just like any other menu handling in Visual Studio .NET.

The SelectMenuItem method takes care of this handling as described above.

public static void SelectMenuItem(object sender, EventArgs e)
{
if typeof(DotNetMenuItem).IsInstanceOfType(sender))
{
DotNetMenuItem item = (DotNetMenuItem)sender;
string menuText = item.Text;

MessageBox.Show(menuText);
}
}

Loose Ends

Application Setup

To set up the sample application on your machine, simply unzip the project files into a directory and double click on the .SLN file. The project will open up in Visual Studio .NET. From there, you can compile or run in debug mode as you would with any other C# Windows Form application.

Side note on using the Menu Creator in Visual Studio .NET with NO Config File

There is a bit of added complexity when we use the XML configuration file approach to setting up the Menu structures. The problem comes when hooking in the event handler code for the menu events. An alternative approach that alleviates all of those problems is just to use the Visual Studio .NET IDE to develop your menus as normal, then replace all the occurrences of the System.Windows.Forms.MenuItem declaration in the code with the DotNetMenuItem declaration. This resolves any problems and allows us to use the Visual Studio .NET generated menu handling code with our owner drawn menus.

To use the IDE to create the menu:

  • Go about creating your menu with the IDE tool as usual
  • Once finished designing the menu, open up your code and do a find/replace. Replace all of the occurrences of System.Windows.Forms.MenuItem with YourNamespace.DotNetMenuItem

This should allow you to use this Owner Drawn menu class simply without having to bother with the XML configuration file.

Take a look at the download file folder called "source without xml menu" to view this code.

Conclusion

In this article, we presented a method of simulating the look and feel of the Visual Studio .NET IDE menus for use in our own applications. We learned about Owner Drawn menus in C# and a XML configuration file approach that simplifies the creation and storage of menu structures. Armed with this knowledge, we can add Visual Studio .NET like menus to our applications that will run under Windows 2000 and Windows XP. I haven't yet tried this out on Windows 98 or Windows ME/NT so you are on your own if you are writing apps for those target platforms. Hopefully you'll all see how .NET allows you to somewhat easily (dare I say) Owner Draw your menus. Just another note - you could also leverage this code and move on to owner drawing other controls in Windows. The basic premise is the same, and it shouldn't be that difficult to accomplish once you've gotten the hang of it.

If you'd like to discuss a broader range of C#Today topics and issues, or if you have any C# related questions you need answers to, please visit our online resource centres at : http://p2p.wrox.com/list.asp?list=csharptoday_discuss&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq and http://p2p.wrox.com/list.asp?list=c_sharp&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq


Creating Graphical Reports With Crystal Reports in .NET

Creating Graphical Reports With Crystal Reports in .NET
http://www.csharptoday.com/content.asp?id=1710
Article Rating: 4.5
Published on December 3, 2001
Abstract
Reports are an integral part of many applications. In every organization, executives are very interested in summaries of their business data – especially if it can be presented in graphical form. As much business is on the net these days, being able to display such graphical reports on our web pages is a valuable skill. In this article Farhan Zia investigates some of the power that Microsoft .NET provides to accomplish this goal using Crystal Reports.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

Reports are an integral part of many applications. Business activities like monthly sales, collected receivables, etc., are usually presented in reports. All to the good, then, if the reports can be presented in a graphical format - graphs allow us to take in data more quickly and spot trends more easily. In this article, we will take a simple but fully functional example of a business web application to create the scenario for displaying such reports.

Our application will be a 3-tier model. We will be using ASP.NET with C# code behind pages to accomplish our task, with a Visual Basic .NET middle tier component and SQL Server database.

The Database

Imagine we have a company called "WroxSellers". We need a report on monthly sales of any selected item between two given dates.

First we need to create our database. For the sake of simplicity, we have only three tables. Here are the three tables we need:

All the details about Items are stored in tblItems, and the details of Sales Persons are in tblSalesPersons. Whenever a salesman sells an item, he makes an entry to the system through our site. This activity will be stored in tblSales including the amount and the date.

Run the DBScript.sql code included in the download to create the tables on your own server.

CREATE TABLE [dbo].[tblItem] (
[ItemId] [int] NOT NULL ,
[Description] [varchar] (50) NOT NULL
) ON [PRIMARY]
GO

CREATE TABLE [dbo].[tblSalesPerson] (
[SalesPersonId] [int] NOT NULL ,
[UserName] [varchar] (50) NOT NULL ,
[Password] [varchar] (30) NOT NULL
) ON [PRIMARY]
GO

CREATE TABLE [dbo].[tblSales] (
[SaleId] [int] IDENTITY (1, 1) NOT NULL ,
[SalesPersonId] [int] NOT NULL ,
[ItemId] [int] NOT NULL ,
[SaleDate] [datetime] NOT NULL ,
[Amount] [int] NOT NULL
) ON [PRIMARY]

GO

The code for the key constraints is shown below:

ALTER TABLE tblItem
ADD CONSTRAINT PK_ItemId
PRIMARY KEY (ItemId)
GO

ALTER TABLE tblSalesPerson
ADD CONSTRAINT PK_SalesPersonId
PRIMARY KEY (SalesPersonId)
GO

ALTER TABLE tblSales
ADD CONSTRAINT FK_ItemId
FOREIGN KEY (ItemId) REFERENCES tblItem(ItemId)
GO

ALTER TABLE tblSales
ADD CONSTRAINT FK_SalesPersonId
FOREIGN KEY (SalesPersonId) REFERENCES tblSalesPerson(SalesPersonId)
GO

Then run the script DBData.sql to populate the tables with some sample data.

The Middle Tier Component

Our component is a simple one. We will follow a simple design model, with two objects for every entity. For example, the tblItems table will have two classes, Item and Items. Item will be the record level representation of an item and will store the details of a particular item, while Items will be a table level representation and will contain the manipulation and adding methods.

So, here are our classes, whose objects will be in action.

  • Item and Items
  • SalesPerson and SalesPersons
  • Sale and Sales

As the complete code is included in the attachment, we will just discuss the methods and properties available in these objects.

Item

Item contains two properties:

  • ItemId
  • Description

Items

This object has a single method. This method will return an Item object depending on the nItemId:

Public Function GetAllItems () As Collections.ArrayList

This method returns a collection of all Item objects.

SalesPerson

This object will have these three properties:

  • SalesPersonId
  • Name
  • Password

SalesPersons

This object will have a single method. It returns the SalesPersonId of the sales person and matches the UserName and Password inputted against that in the database. If there is no match, the function returns zero.

Public Function ValidateUser (strUserName as String, strPassword as String) As Integer

Sale

This object has these five properties:

  • SaleId
  • SalesPersonId
  • ItemId
  • SaleDate
  • Amount

Sales

This object has two methods. GetSales() returns a collection of Sales objects, depending on the received parameters.

Public Function GetSales (Optional nSaleId As Integer = 0, Optional nSalesPersonId As Integer = 0,
Optional nItemId As Integer = 0) As Collections.ArrayList

In addition to this method, AddSale() adds a new sale entry to the database.

Public Function AddSale (objSale As Sale)

In the download code you will find one more class called dbConnection. This class is used to store the database connection. It only has one method, GetdbConnection(), that is also shared (static) so that there is no need to instantiate this class. The GetdbConnection returns a connection object with connection details from the registry (we can also store such settings in web.config). You need to alter the registry according to your environment - details are in the class file as remarks.

Now add a new empty web project to the solution. Call it WroxSellers.

We need two pages for our site. The first authenticates our sales staff and allows then to enter records of sales, and the other enables us to select report parameters and view reports based on this data.

Creating Web Pages

So let us look at our first aspx page, index.aspx. This is the page that the sales person will use to login into our system. You will find the code in the download.

This page consists of two text boxes to take a user name and password, and a submit button. If the user is valid we redirect them to the PostEntry.aspx page with their SalesPersonId in the query string.

Here the sales person selects the item they have sold and enters its amount, which is then added in the database. After that we simply return them back to the login page with a thanks message.

Up until now we have just discussed entering the data into our database. Now we will look at the reports of this data.

We create a page called ManagerDefault.aspx where a manager can make custom reports. Here is how it looks:

First the manager selects the Item they want a report on, then they select the date range for which they want to view data. We use ASP.NET Calendar controls to do this:

We also include an option to view the report either in Graph or List form.

This page is simple. The point of interest is the usage of calendar control. Let us first see what additions are made in the ManagerDefault.aspx file after adding the calendar control, then we will come to the code behind.

<asp:calendar id="DtPicker2"
            runat="Server" Height="156px" Width="280px" Visible="False"
            BackColor="White"
SelectorStyle-BackColor="#FFCC66"
            NextPrevStyle-Font-Size="9pt" NextPrevStyle- ForeColor="#FFFFCC"
TodayDayStyle-ForeColor="White"
            TodayDayStyle-BackColor="#FFCC66" DayHeaderStyle-Height="1px"
DayHeaderStyle-BackColor="#FFCC66" BorderWidth="1px"
            Font-Size="9pt" Font-Names="Verdana"
OtherMonthDayStyle-ForeColor="#CC9966"
            TitleStyle-Font-Size="9pt" TitleStyle-Font-Bold="True"
TitleStyle-ForeColor="#FFFFCC" TitleStyle-BackColor="#990000"
            ForeColor="Black"
BorderColor="White"
            SelectedDayStyle-Font-Bold="True"
            SelectedDayStyle-BackColor="#CCCCFF"
NextPrevFormat="FullMonth">

This stuff is for the calendar and as this calendar is displayed by clicking the image, here is the code for the image also in the same file.

<input type="image"
            onclick="Page_ValidationActive=false;" src="datepicker.gif"
            alt="Show Calender"
runat="server" onserverclick="ShowCal1"
            id="ImgCal1" name="ImgCal1">

Here we set the Page_ValidationActive variable to false so that we don't post the form, and perform validation on clicking the image. Secondly, notice onserverclick="ShowCal1". This ShowCal1() method is available in the code behind file ManagerDefault.aspx.cs.

Next we come to the code behind file ManagerDefault.aspx.cs. We will only cover the calendar related functions here. The method below displays the calendar, and is invoked when the calendar image is clicked:

// this method displays the first date calendar
public void ShowCal1(Object sender, System.Web.UI.ImageClickEventArgs e)
{
// Display calendar
DtPicker1.Visible = true;
}

When the user clicks on the calendar to select a date, we move the selected date to its text box and hide the calendar.

// this method get the first date and hide the first calendar
private void DtPicker1_SelectionChanged(object sender, System.EventArgs e)
{
txtStartDate.Text = DtPicker1.SelectedDate.ToShortDateString();
DtPicker1.Visible = false;
}

The second calendar is handled in the same way.

Once the manager has made their selections, they press the ShowReport button. Here is how we deal with it:

// this method redirects the manager to Report page with the selected items
private void bSubmit_ServerClick(object sender, System.EventArgs e)
{
// redirect to the report page
Response.Redirect("ViewReport.aspx?ItemId=" + cboItemType.SelectedItem.Value + "&StartDate=" +
txtStartDate.Text + "&EndDate=" + txtEndDate.Text);

}

We simply redirect the user to the ViewReport.aspx page and send all the inputs as parameters like the ItemId, StartDate and the EndDate.

ViewReport.aspx does all of the work for displaying the report. Let's talk about it now.

The Report

We will create our report using Visual Studio .NET and will save it in a .rpt file. The code above is actually redirecting the user towards the report page that will display the report.

Create a new web page called ViewReport.aspx. From the toolbox drag the Crystal Report Viewer control on the form and name it crViewer. Here is how it looks:

In the properties window you will find some properties like DisplayToolbar, EnableDrillDown, EnableViewState. These properties help us to customize our report; for this example we set them all as false. We also need to write some code to pass the arguments to our report like the ItemId, StartDate and EndDate, which we will do later. Save the form and close it. Before we go any further, we will create the report.

Creating the Report

Right click on the solution name in the project explorer and click Add | Add New Item | CrystalReport. Name it ItemReport. Next we will be asked to select how our report is created. We will use the Report Expert, which is the default option. From the down options we can select the format of your report. Select "Standard" which is again the default. The next screen shows this:

We then have to select our data source. Select OLEDB (ADO) | Microsoft OLE DB Provider for SQL Server | Next. We need to provide the Server name, our UserId and Password, and the database to which we want to connect. After providing the information, click Finish.

Now you need to select the tables to which we will be displaying data. For our example, select "tblSales " only because all the data we need is from this table. Here is the screen.

Click Next to proceed. Now we have to add the fields to be displayed in the report. Select SaleDate and Amount. Click Next we will be offered the option to Group the information on the report using a Group By clause. We do not need this so just click Next again.

Next we have to select our chart type. There are a large number of different charts available. For our example, select Line and the first chart type and click Finish.

This is what you will see.

Right click on the Page Header area and click Insert | Chart. We will see a screen that we have already seen before. Click on the Data tab.

We mention here that we want only one chart in our report. Then we provide our SaleDate field under the On change of column. Note that we mention both the fields that we want to be displayed (SaleDate and Amount). Click on the Text tab and select a suitable Title for your report. Click Ok and we are almost done. We need to set the output parameters our code will query - so let's do that next.

Creating Report Parameters

In the report, open the field explorer by clicking View | Other Windows | Document Outline or by simply pressing Ctrl + Alt + T. Right click on the parameter fields and click New. Here we need to provide the parameter name, and data type that it will receive. For our example create three parameters as follows:

  • Name: ItemId
    Type: Number
  • Name: StartDate
    Type: Date
  • Name: EndDate
    Type: Date

In addition to Number and Date data type, we can also set Boolean, Currency, DateTime, Time, and String data types.

Now we should see these names under the Parameter Fields in the Field Explorer.

Finally, we need to create the query that will use these parameters. Right click anywhere on the report except the Report Header section. Select Report | Edit Selection Formula | Records. We will see a screen like this.

We can write the query in the provided white space (or simply click the fields above to create it). Paste these lines there and click on the tick button to check the syntax.

{tblSales.ItemId}={?ItemId} And
{tblSales.SaleDate} > DateAdd ("d", -1,{?StartDate}) And
{tblSales.SaleDate} < DateAdd ("d",1 , {?EndDate})

This can also be achieved by double clicking on the different visual tools provided. There are three portions above the query area. In the first pane are the table fields that we selected earlier during the report creation. Other fields and tables in the database are also available, if we expand the database server name. In addition to this, all the parameter fields are also present in this pane and are distinguishable by the '?' sign. Double click any one of these fields and they will be copied in the text area. Similarly, in the middle portion, we see all the functions we usually use in our queries. In the last pane, we find all the operators. Arithmetic, conversion, comparison, string etc. Double click on any of them to use them in the query.

As you can see, it is simple. We are just comparing our table fields with the parameter fields. Note that the parameter fields are referred with a question mark (?) sign. Click the Save button to save the query. Save the report and close it. Now we return to the code behind for ViewReport.aspx from where we will perform two tasks:

  • Connect to the report we have just created
  • Pass the arguments to our report

Connecting to the Report

Let us first connect to the report that we have previously created. As we will be using the crystal report object, we need to add a reference to it in our project. Click Project | Add Reference. You will be presented with a dialog box. Under the .NET Tab select two references:

  • CrystalDecisions.CrystalReports.Engine
  • CrystalDecisions.Shared

At the top of the page, we need to import these namespaces. Here is the actual code:

using System;
using
            System.Collections;
using System.ComponentModel;
using
            System.Data;
using System.Drawing;
using System.Web;
using
            System.Web.SessionState;
using System.Web.UI;
using
            System.Web.UI.WebControls;
using
            System.Web.UI.HtmlControls;
using
            CrystalDecisions.Shared;
using
            CrystalDecisions.CrystalReports.Engine;

Here is the code in the Page_Load event. Let us take a look at it and then we will dissect it.

private void Page_Load(object sender, System.EventArgs e)
{
// getting data from Request
nItemId = int.Parse(Request.QueryString.Get("ItemId"));
strStartDate = Request.QueryString.Get("StartDate");
strEndDate = Request.QueryString.Get("EndDate");

// object declration
CrystalDecisions.CrystalReports.Engine.Database crDatabase;
CrystalDecisions.CrystalReports.Engine.Table crTable;

TableLogOnInfo dbConn = new TableLogOnInfo();

// new report document object
ReportDocument oRpt = new ReportDocument();

// loading the ItemReport in report document
oRpt.Load("C:\\\\Inetpub\\\\wwwroot\\\\WroxSellersWeb\\\\ItemReport.rpt");

// getting the database, the table and the LogOnInfo object which holds login onformation
crDatabase = oRpt.Database;

// getting the table in an object array of one item
object[] arrTables = new object[1];
crDatabase.Tables.CopyTo(arrTables, 0);

// assigning the first item of array to crTable by downcasting the object to Table
crTable = (CrystalDecisions.CrystalReports.Engine.Table)arrTables[0];

dbConn = crTable.LogOnInfo;

// setting values
dbConn.ConnectionInfo.DatabaseName = "WroxSellers";
dbConn.ConnectionInfo.ServerName = "Intersoft";
dbConn.ConnectionInfo.UserID = "farhan";
dbConn.ConnectionInfo.Password = "farhan";

// applying login info to the table object
crTable.ApplyLogOnInfo(dbConn);

// defining report source
crViewer.ReportSource = oRpt;

// so uptill now we have created everything
// what remains is to pass parameters to our report, so it
// shows only selected records. so calling a method to set
// those parameters.
setReportParameters();

}

First of all we are getting data from the query string. Remember that we have sent the ItemId, StartDate and EndDate parameters from ManagerDefault.aspx.cs. As the second step, we instantiate Crystal Report's Database, Table and TableLogOnInfo objects. These objects will be used to load the database and table through which we have created our report ItemReport.rpt. Next we create a new Report document object and load our report into it. Notice the line for loading the report:

// loading the ItemReport in report document
oRpt.Load("C:\\\\Inetpub\\\\wwwroot\\\\WroxSellersWeb\\\\ItemReport.rpt");

Please change the path according to your system. Note that we want to return two backslashes, so have escaped both in the code above.

After loading the report we set our Crystal Report's Database object to the report document's database. Then we copy the tables in that database object to an array of objects, from where we select the first table and assign it to the crystal report's table object. Then we set the properties of the LogOnInfo object that will be used for authentication purposes. Again, we need to change it for our system accordingly. Finally we apply this LogOnInfo object to our table object. In the last line we specify the report source of the Crystal Report Object crViewer to our Report Document object:

// defining report source
crViewer.ReportSource = oRpt;

We are connected to our report now. What next?

The answer is: passing parameters to the report. We need to pass the ItemId, StartDate and EndDate to this report so that a report on the selected data can be displayed. If you remember, we have collected these values from a query string. Let us now pass them to our report.

Passing Parameters to a Report

If you take a look at the end of the Page_Load function, we have called a method setReportParameters(). This method will actually add/send the parameters to our report. So let us see this final function:

// this method sets the parameters of the
            report
private void setReportParameters()
{

// all the
            parameter fields will be added to this collection
ParameterFields
            paramFields = new ParameterFields();

// the parameter fields
            to be sent to the report
ParameterField pfItemId = new
            ParameterField();
ParameterField pfStartDate = new
            ParameterField();
ParameterField pfEndDate = new
            ParameterField();

// setting the name of parameter fields
            with wich they will be recieved in
            report
pfItemId.ParameterFieldName =
            "ItemId";

pfStartDate.ParameterFieldName =
            "StartDate";
pfEndDate.ParameterFieldName = "EndDate";

//
            the above declared parameter fields accept values as discrete
            objects
// so declaring discrete
            objects
ParameterDiscreteValue dcItemId = new
            ParameterDiscreteValue();
ParameterDiscreteValue dcStartDate =
            new ParameterDiscreteValue();
ParameterDiscreteValue dcEndDate =
            new ParameterDiscreteValue();

// setting the values of
            discrete objects
dcItemId.Value =
            nItemId;

dcStartDate.Value =
            DateTime.Parse(strStartDate);
dcEndDate.Value =
            DateTime.Parse(strEndDate);

// now adding these discrete
            values to
            parameters
pfItemId.CurrentValues.Add(dcItemId);
pfStartDate.CurrentValues.Add(dcStartDate);
pfEndDate.CurrentValues.Add(dcEndDate);

//
            now adding all these parameter fields to the parameter
            collection
paramFields.Add(pfItemId);
paramFields.Add(pfStartDate);
paramFields.Add(pfEndDate);

//
            finally add the parameter collection to the crystal report
            viewer
crViewer.ParameterFieldInfo =
paramFields;
}

Before dissecting the above code let us study the theory behind it. Our crystal report viewer object crViewer has a property that accepts a collection of ParameterFields type. Through this property, all the parameters are passed to the report. This ParameterFields collection has an Add method, which accepts objects of type ParameterField. So what we do is create three objects of type ParameterField for our three parameters, ItemId, StartDate and EndDate. We then add the values of these parameters to these objects, add them to the collection, and then finally specify this collection as the ParameterFieldInfo property of the crystal report viewer object.

So, from the top where we declare the collection and the three ParameterField objects:

      // all
            the parameter fields will be added to this
            collection
ParameterFields paramFields = new
            ParameterFields();

// the parameter fields to be sent to the
            report
ParameterField pfItemId = new
            ParameterField();
ParameterField pfStartDate = new
            ParameterField();
ParameterField pfEndDate = new
            ParameterField();

Then we name the parameter fields. These are the names with which they are referred in the crystal report. Remember, we used these names there. After this we need to provide values to these parameter fields. But we cannot just set the values of these parameter fields directly - they accept object of type ParameterDiscreteValue. So we will declare three objects of the this type, and after specifying their values, will add them to the parameter fields. This code deals with this task:

// the above declared parameter fields accept values as discrete objects
// so declaring discrete objects
ParameterDiscreteValue dcItemId = new ParameterDiscreteValue();
ParameterDiscreteValue dcStartDate = new ParameterDiscreteValue();
ParameterDiscreteValue dcEndDate = new ParameterDiscreteValue();

// setting the values of discrete objects
dcItemId.Value = nItemId;

dcStartDate.Value = DateTime.Parse(strStartDate);
dcEndDate.Value = DateTime.Parse(strEndDate);

// now adding these discrete values to parameters
pfItemId.CurrentValues.Add(dcItemId);
pfStartDate.CurrentValues.Add(dcStartDate);
pfEndDate.CurrentValues.Add(dcEndDate);

Here we first declare these objects, then set their values and add them to the parameter fields. As you can see, we have called the Add method of the CurrentValues object and passed it the discrete parameters. So this way all the parameter fields are ready to be added to the collection. Here is how this is done:

   // now adding all these
            parameter fields to the parameter
            collection
paramFields.Add(pfItemId);
paramFields.Add(pfStartDate);
paramFields.Add(pfEndDate);

And finally, we pass this collection to the ParameterFieldInfo property of our crystal report viewer object.

// finally add the parameter collection to the crystal report viewer
crViewer.ParameterFieldInfo = paramFields;

We are finished. Build your project and run it. If you have previously added some sale entries in your tables then you can call ManagerDefault.aspx directly to create a report; otherwise first add some records.

You should see the following output in your browser:

We can change the look of the report by changing the ViewReport.aspx page. By making some slight variations in the report, like changing chart type by clicking on the report and selecting ChartExpert, we can obtain some different charts.

Summary

In this article, we first created a simple sales recording web application. Then we created a report file and connected it to our application. We also looked at how to pass arguments to our reports. Of course this is just a start; but hopefully you are now equipped with the basics to go on and create your own reporting applications.

New Wrox Press book on Crystal Reports for Visual Studio .NET
Professional Crystal Reports for Visual Studio .NET provides a comprehensive guide to the functionality included with CR.NET and shows you how to integrate reports into your .NET applications. This book is aimed mainly at readers with some experience of Crystal Reports, but it will also be valuable to programmers who are getting their first taste of Crystal Reports through the Visual Studio .NET development environment.

This book covers the following areas:
  • Crystal Reports .NET overview
  • Report integration for Windows and Web-based applications
  • Creating XML report web services
  • Working with .NET data, formulas, and logic
  • The reports engine
  • Distributing your application


Windows Forms Charting – Part 2

Windows Forms Charting – Part 2
http://www.csharptoday.com/content.asp?id=1680
Article Rating: 4
Published on October 18, 2001
Abstract
In this second part of his GDI+ Windows Forms demonstration, Matthew Reynolds adds some more functionality to the charting control built last time. Specifically, he adds some interactivity to the chart by allowing the user to examine the underlying data values of the bars, and also change these values by grabbing hold of the top of the bar and making the bar either bigger or smaller.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

In this second part of our GDI+ demonstration, we'll add some more functionality to the charting control that we built last time. Specifically, we'll add some interactivity to the chart by allowing the user to examine the underlying data values of the bars and also change the values by grabbing hold of the top of the bar and making the bar either bigger or smaller.

Working with the Mouse

In your Windows Forms experiments so far, it's likely that you've paid little attention to the mouse. If you've exclusively been using the built-in Framework controls, these controls deal with mouse interaction themselves with little need for you to get involved. However, working with the mouse is an important part of Windows Forms development, so in this exercise we're going to take a close look at building rich user interaction for BarChart using the mouse.

The mouse partakes in the event system that we're very used to using at this point. Specifically, .NET will report the mouse interaction in the following ways:

  • MouseDown -the physical mouse button has been pressed.
  • MouseUp - the physical mouse button has been released.
  • MouseMove - the mouse pointer has been moved across the surface of the control.
  • MouseEnter - the mouse pointer has entered the bounds of the control.
  • MouseLeave - the mouse pointer has left the bounds of the control.
  • MouseHover - the mouse pointer was moving, but it stopped and "hovered" over a point on the control for a short while. (This is usually used to display tool tips).
  • MouseWheel - the mouse wheel has been scrolled when the control has focus.

When we are working with the mouse, there are two things we need to consider. Firstly, when the user is using the mouse with your control, we need to provide visual feedback to their actions. We've all seen that when we click a button in Windows, the button appears to move into the screen when the user presses the button and moves out again when the user releases it. This mimics the actual physical action of pressing the mouse button.

Secondly, controls have a varying degree of "interest" as to where the mouse actually is on their surface when they receive a message. For example, a button control doesn't care where the mouse is when a MouseDown event is received - any location on the surface of the control does the same thing. Other controls care where the mouse is. The DataGrid control, for example, needs to look at the position of the mouse and determine which cell, row header or column header has been clicked.

In this exercise, we're going to be looking closely at where the mouse is when we receive messages.

Selecting Bars

When we're interested in which parts of a control have been affected by a mouse action, we need to undertake something called "hit testing". This is a process whereby we look at all of the "things" on the surface of the control and try to pinpoint which one has been affected. For example, as the mouse moves over the surface of the control, we receive MouseMove events. To find out what bar is selected, we have to go through each group, then each bar in the bar group, looking to see if the reported position of the mouse is within the rectangle that defines the bar. In this screenshot, we can determine that the mouse is within the third bar of the third group.

In the first part of this exercise, what we want to do is whenever the user moves the pointer over a bar, we want to highlight it in some way to show the user which one she's working with. This is a three-phase approach.

  • Firstly, we have to provide a way to tell the bar that it is selected and that it should alter its appearance when it is drawn.
  • Secondly, we have to listen for MouseMove events on ChartView and pipe them through to BarChart.
  • Thirdly, when they are received on BarChart, the object has to "hit test" the position of the mouse and adjust the selection accordingly.

We'll do this in order, so firstly open the code editor for Bar and add a new member called State and also a new enumeration called BarState.

public class Bar
{
// members...
public Rectangle Rectangle;
public Color Color = Color.Red;
public Color LightColor = Color.Pink;
public Color DarkColor = Color.DarkRed;
public double Value;
public BarState State = BarState.Unselected;

public enum BarState { Unselected, Selected };

From the enumeration, we can surmise that a bar is going to have two states - selected and unselected.

To change the appearance of the bar depending on State, all we have to do is create a different brush. If the bar is unselected, we paint using the normal color. If it's selected, we use the light color:

      //
            Paint...
public void Paint(Graphics graphics)
{
// create
            the brush...
SolidBrush brush =
            null;
switch(State)
{
case BarState.Unselected:
brush =
            new SolidBrush(Color);
break;

case
            BarState.Selected:
brush = new
            SolidBrush(LightColor);
break;
}

// draw the
            rectangle...
graphics.FillRectangle(brush, Rectangle);

//
            dispose...
brush.Dispose();
}

To route mouse messages through to BarChart, we'll need to put some override-able methods on ChartBase. BarChart can then override the methods it needs to add mouse interaction. Open the code editor for ChartBase and add this method:

public virtual void OnMouseMove(ChartView view, MouseEventArgs e)
{
}

UserControl itself implements a method called OnMouseMove that we can override. In fact, we need to override this method otherwise we won't receive the MouseMove events. Open the code editor for ChartView and add this code:

protected override void OnMouseMove(MouseEventArgs e)
{
// pass it on...
if(Chart != null && Data != null)
Chart.OnMouseMove(this, e);
}

Hit testing is a fairly straightforward process and there are a number of ways to pull it off. The technique we're going to see here involves creating a new method called HitTest that takes the coordinates of the mouse. This method returns a HitTestResult object containing the bar that was under the coordinates and flag indicating which part of the bar it was under. (We'll use this flag later).

Firstly, open the code editor for BarChart. We need a private member to hold the currently selected bar (there can only be one) and also the HitTestResult class.

   public class BarChart :
            ChartBase
{
// members...
public ArrayList Groups = new
            ArrayList();
public int GroupWidth;
public int
            BarWidth;
public double MinValue = 0;
public double
            MaxValue;
private Bar _selectedBar;

//
            HitTestResult...
public class HitTestResult
{
//
            members...
public Bar Bar;
public HitTestType Type =
            HitTestType.None;

// enum...
public enum HitTestType {
            None, BarBody };
}

The HitTest method itself is very straightforward. All we have to do is walk all the groups, and then walk all of the bars in each group. For each bar, we check to see if the supplied x coordinate is inside the rectangle. If it is, we check the y coordinate. If both match, we store the bar and the type of item that was "found" (in this case, it can only be "BarBody ") in the HitTestResult object we're holding in testResult.

      public
            HitTestResult HitTest(int x, int y)
{
HitTestResult testResult
            = new HitTestResult();

// go through the
            groups...
foreach(BarGroup group in Groups)
{
// go through
            the bars...
foreach(Bar bar in group.Bars)
{
//
            match?
if(x >= bar.Rectangle.Left && x <=
            bar.Rectangle.Right)
{
// whole bar?
if(y>=
            bar.Rectangle.Top && y <=
            bar.Rectangle.Bottom)
{
// rest of the
            bar...
testResult.Bar = bar;
testResult.Type =
            HitTestResult.HitTestType.BarBody;
break;
}
}
}

//
            stop scanning?
if(testResult.Bar !=
            null)
break;
}

// return...
return
            testResult;
}

To call HitTest, we have to override the OnMouseMove event. This will be called whenever a MouseMove event is received by the control. When it is called, the first thing we do is call HitTest.

public override void OnMouseMove(ChartView view, MouseEventArgs e)
{
// find the bar...
HitTestResult result = HitTest(e.X, e.Y);

When HitTest returns, we check the bar that is under the coordinates with the selected bar held in _selectedBar. If this is different, we need to do something. Remember, it could be the case that result.Bar is null, in which case no bar has been selected. If this is the case and _selectedBar is not null, we need to remove the highlight from the selected bar.

         if(result.Bar
            != _selectedBar)
{
// do we need to de-select the current
            bar?
if(_selectedBar != null)
{
_selectedBar.State =
            Bar.BarState.Unselected;
view.Invalidate(_selectedBar.Rectangle);
}

If we did have a previously selected bar, we have to mark it as unselected and invalidate it. If a new bar has been selected, we mark it as "selected" and invalidate it too.

            //
            set the new selection...
_selectedBar =
            result.Bar;
if(_selectedBar != null)
{
// set the
            selection...
_selectedBar.State =
            Bar.BarState.Selected;
view.Invalidate(_selectedBar.Rectangle);
}
}

After we've done this, we need to adjust the cursor. If we're floating over a bar we want to display the hand cursor. If we're not floating over a bar, we want to display the default cursor.

         //
            do the cursor either way...
if(result.Bar != null)
{
// set
            the OnMouse...
if(result.Type ==
            HitTestResult.HitTestType.BarBody)
view.Cursor =
            Cursors.Hand;
}
else
view.Cursor =
            Cursors.Default;
}

If we try this now, we should find that the selection highlights as the mouse rolls over it.

Changing Values using the Mouse

In this section, we're going to add some more functionality that will allow us to change the underlying data on the chart by grabbing hold of the top of the chart and moving the bar up and down.

To do this, we need to tweak HitTest so that it can detect when the pointer is near the top of the bar. As a general rule, checking to see if a mouse is absolutely on a given pixel is usually a bad move. We need to give the user an area in which an action is valid. In our case, we're going to create an area that extends two pixels above the top of the bar and two pixels below.

First of all, find the definition for HitTestResult and add a new value to the enumeration called BarSizer.

      //
            HitTestResult...
public class HitTestResult
{
//
            members...
public Bar Bar;
public HitTestType Type =
            HitTestType.None;

// enum...
public enum HitTestType {
            None, BarBody, BarSizer };
}

Next, find the HitTest method. Insert an additional y-coordinate check before the one that looks to see if we're contained within the body of the bar.

      public
            HitTestResult HitTest(int x, int y)
{
HitTestResult testResult
            = new HitTestResult();

// go through the
            groups...
foreach(BarGroup group in Groups)
{
// go through
            the bars...
foreach(Bar bar in group.Bars)
{
//
            match?
if(x >= bar.Rectangle.Left && x <=
            bar.Rectangle.Right)
{
// sizer?
if(y >=
            bar.Rectangle.Top - 2 && y <= bar.Rectangle.Top +
            2)
{
// we have the sizer...
testResult.Bar =
            bar;
testResult.Type =
            HitTestResult.HitTestType.BarSizer;
break;
}

// whole
            bar?
if(y>= bar.Rectangle.Top && y <=
            bar.Rectangle.Bottom)
{
// rest of the
            bar...
testResult.Bar = bar;
testResult.Type =
            HitTestResult.HitTestType.BarBody;
break;
}
}
}

//
            stop scanning?
if(testResult.Bar !=
            null)
break;
}

// return...
return
            testResult;
}

To tell the user that the mouse is in a position where the bar can be dragged, we need to change the mouse pointer from a hand to an up/down arrow. Find the OnMouseMove method and at the bottom where we change the mouse cursor, add an additional statement to check for BarSizer:

// do the cursor either way...
if(result.Bar != null)
{
// set the OnMouse...
if(result.Type == HitTestResult.HitTestType.BarSizer)
view.Cursor = Cursors.SizeNS;
if(result.Type == HitTestResult.HitTestType.BarBody)
view.Cursor = Cursors.Hand;
}
else
view.Cursor = Cursors.Default;

Now if we run the project we'll notice that with the mouse positioned at the top of the bar, the cursor will change to a sizing arrow.

Note: When we move the bars up or down, the data in the DataGrid automatically updates as well.

That's part of the way there. What we need to do now is listen for the MouseDown event and put ourselves into a state where we know the bar is being sized. Once in that state, OnMouseMove has to behave differently - i.e. the bar has to change size - rather than the existing behavior of changing the selection. We also have to listen for MouseUp events to tell us that the user has finished.

Open the code editor for ChartBase and add these methods:

public virtual void OnMouseUp(ChartView view, MouseEventArgs e)
{
}

public virtual void OnMouseDown(ChartView view, MouseEventArgs e)
{
}

Like before, we have to wire these events into ChartView. Open the code editor for ChartView and add this code:

protected override void OnMouseDown(MouseEventArgs e)
{
// pass it on...
if(Chart != null && Data != null)
Chart.OnMouseDown(this, e);
}

protected override void OnMouseUp(MouseEventArgs e)
{
// pass it on...
if(Chart != null && Data != null)
Chart.OnMouseUp(this, e);
}

When the mouse button is pressed, we first need to check to see if the left mouse button has been pressed because the same event is used irrespective of the actual button. If it wasn't the left mouse button that was pressed, we quit the method:

public override void OnMouseDown(ChartView view, MouseEventArgs e)
{
// left button?
if(e.Button != MouseButtons.Left)
return;

If it was pressed, we call HitTest again. This time if BarSizer is reported in the results, we know we're supposed to be dragging. To mark this fact, we set the Capture property of ChartView to true.

         //
            hit test...
HitTestResult result = HitTest(e.X,
            e.Y);
if(result.Type ==
            HitTestResult.HitTestType.BarSizer)
{
// set capture and start
            dragging...
view.Capture = true;
}
}

Capture is a very important part of working with the mouse. Typically, mouse events are sent to the form or control that the mouse is over. However, this doesn't make sense all of the time. If you look at how a scroll bar works, if you click, hold and drag the "thumb", if you move the mouse outside of the bounds of the scroll bar, the scrolling still works. That's because the scroll bar is capturing all mouse input. If it weren't capturing the input, as soon as you moved the mouse outside the bounds of the bar, the scroll bar would stop responding to the mouse movements. This would make the scroll bar very difficult to use, as you'd have to keep the pointer exactly within the bounds of the control all the time.

The other side of capturing mouse events is that you have to be careful to release the capture when you're not using it. Add this method to BarChart:

      public
            override void OnMouseUp(ChartView view, MouseEventArgs e)
{
//
            are we capturing?
if(view.Capture == true)
view.Capture =
            false;
}

Now, if we're receiving MouseMove events and the Capture property of the view is set to true, we need to do something different to our default behavior of changing the selection. Find the OnMouseMove event and add this code to the top. (I've omitted some code for brevity).

public override void OnMouseMove(ChartView view, MouseEventArgs e)
{
// capture?
if(view.Capture == true)
{
OnMouseCaptureMove(view, e);
return;
}

OnMouseCaptureMove is a fairly simple method. First of, we don't care if the pointer is moved outside of the bounds of the DataRectangle described by ChartView, i.e. we can only reduce the bar to 0 and we can only set the bar to whatever MaxValue is set to.

      protected
            void OnMouseCaptureMove(ChartView view, MouseEventArgs e)
{
//
            have we gone outside the range?
Rectangle rect =
            view.DataRectangle;
if(e.Y >= rect.Top && e.Y <=
            rect.Bottom)
{

Setting the value is a little tricky. What we need to do first is look at the y position of the mouse, and from that determine how tall the bar is supposed to be.

// set the new value...
double height = (double)(rect.Bottom - e.Y);

Once we have that, we need to reverse the action of CalculateHeightFromValue and determine a value from a height.

_selectedBar.Value = (height / (double)rect.Height) * MaxValue;

However, when Value has been changed, we need to re-measure the chart.

Measure(view, view.Data);

Finally, we need to invalidate the bar. But, because the bar has changed height, we'll create a new invalidation rectangle that goes right from the bottom of the chart to the top.

// invalidate...
Rectangle invalidRect =
new Rectangle(_selectedBar.Rectangle.Left,
rect.Top, _selectedBar.Rectangle.Width, rect.Height);
view.Invalidate(invalidRect);
}
}

Now try running the project and you should find you can alter the height of the bars using the mouse.

Summary

In this article we took a look at how to extend our Charting control to include mouse interaction. We learned how .NET reports mouse activity back to our application, and how we can use "hit testing" to find out what the user is trying to do. We then illustrated one technique for watching the common mouse "dragging" operation to alter the size of bars in the graph.

If you'd like to discuss a broader range of C#Today topics and issues, or if you have any C# related questions you need answers to, please visit our online resource centres at : http://p2p.wrox.com/list.asp?list=csharptoday_discuss&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq and http://p2p.wrox.com/list.asp?list=c_sharp&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq


Windows Forms Charting – Part 1

Windows Forms Charting – Part 1
http://www.csharptoday.com/content.asp?id=1730
Article Rating: 4.5
Published on October 17, 2001
Abstract
GDI+ is the latest version of Windows’ Graphic Device Interface, which is the part of Windows that abstracts commands like “draw a window”, “draw a line”, etc. into commands that our video driver can understand in order to display images on our computer’s screen. We’ve already seen some of the advanced GDI+ features available to us in Matthew Reynolds' 'Irregular Shaped Forms and Other Cool GDI+ Features' article. In this article Matt looks at some of the more basic features and builds a simple user control that can draw a 3D bar chart. In Part 2, we continue the development of the chart, making it user-interactive.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

In Chapter 11 of Wrox's "Beginning Windows Forms", we look in some detail at the features available in GDI+. GDI+ is the latest version of Windows' Graphic Device Interface, which is the part of Windows that abstracts commands like "draw a window", "draw a line", etc. into commands that your video driver can understand in order to display images on your computer's screen.

We've already seen some of the advanced GDI+ features available to us in Irregular Shaped Forms and Other Cool GDI+ Features. In this article we'll look at some of the more basic features, and build a simple user control that can draw a 3D bar chart.

Charting Basics

Because we all know what a chart looks like, let's dive right into building a user control that can draw a chart.

Firstly, what do we need in order to draw a chart? Well, we need data, and typically that data is stored in a tabular form, like this Excel spreadsheet:

Here we've got some basic data that shows what percentage of the available Widgets that Darren, Brandon and Edward own. In our example, we're going to be a little more ambitious and take chart data that looks like this:

and turn it into a chart that looks a lot like this:

Well, it's not going to look exactly like that because Microsoft Excel generated that chart, but it will look pretty similar!

Our Application

Start a new Visual C# - Windows Application project. Call it Charting.

The first thing we need is some data. It makes sense that rather than building a set of custom classes to store the data, we'll use the Framework's own System.Data data classes, specifically the System.Data.DataSet, System.Data.DataTable, and System.Data.DataRow classes. In fact, as we don't need to hold multiple tables, we'll actually only need to use the DataTable and DataRow classes.

Open the Designer for Form1. Add a new DataGrid control. Set its Name property to datagridChart and its Anchor property to Top, Bottom, Left, Right, so that the control will remain in the same relative position whenever the form is resized.

Next, add a LinkLabel control. Set its Text property to Regenerate Sample Data, its Name property to linkRegenerate, and its Anchor property to Bottom, Left.

Double-click on the LinkLabel control and add a call to a method we'll build in a little while called Regenerate:

private void linkRegenerate_LinkClicked(object sender,
System.Windows.Forms.LinkLabelLinkClickedEventArgs e)
{
Regenerate();
}

Now, flip back to the Designer and double-click on the background of the form. Add the same call to Regenerate to the Form's Load event.

private void Form1_Load(object sender, System.EventArgs e)
{
Regenerate();
}

Now we need to actually build Regenerate. Initially, we need a new DataTable into which we can write the chart data:

      public
            void Regenerate()
{
// create a table...
DataTable table =
            new DataTable();

Once we have that, we need to define the columns in the table. The first column has to be the name - in this case it's going to store the month and year that the data applies to because each row in the table represents a month.

// add a column for the name...
table.Columns.Add("Name", typeof(string));

The remaining four columns will be used to store values for the month.

// now, add columns for the four regions...
table.Columns.Add("North", typeof(double));
table.Columns.Add("South", typeof(double));
table.Columns.Add("East", typeof(double));
table.Columns.Add("West", typeof(double));

Once we've defined the columns, we can add the data. To make this example simpler, we're going to use random numbers, so we'll need a random number generator:

// create a random number generator...
Random random = new Random();

Now we loop through six months. To get the name, we'll create a new DateTime object and call the ToString method with a custom format string that will give us the month name and the year. We then add random values between 10 and 250 for the data. (The values we're using are completely arbitrary - this code should work with pretty much any set of values).

         //
            add six month's worth of data...
for(int n = 1; n <= 6;
            n++)
{
// create the row data...
object[] rowData = new
            Object[5];
rowData[0] = new DateTime(2000, n, 1).ToString("MMMM
            yyyy");
rowData[1] = (double)random.Next(10, 250);
rowData[2]
            = (double)random.Next(10, 250);
rowData[3] =
            (double)random.Next(10, 250);
rowData[4] =
            (double)random.Next(10, 250);

// add the row to the
            table...
table.Rows.Add(rowData);
}

Finally, once we've built up the data, we can go ahead and set the DataSource property of the DataGrid control.

// set the data...
datagridChart.DataSource = table;
}

Run the project and we'll see something like this:

If we click the Regenerate Sample Data link, we get a new set of data.

Drawing the Chart

Now that we have the data we can go ahead and build a user control that's capable of displaying a chart. To keep to our purist object-oriented design rules, we'll build a base ChartBase class that contains basic chart functionality. We'll then derive other chart types from this, although in this exercise we're only going to build one derived type called BarChart.

We'll also need a user control that can be used to draw the control. This user control will be called ChartView and will contain a property that represents the charting class to be used (derived from ChartBase) and a property that holds a DataTable.

We'll kick off by building the user control. Using Solution Explorer, create a new User Control called ChartView. For the most part, we're going to be controlling the drawing of the view, but we do need to make one small change. Set the BackColor property of the control to white.

ChartBase needs to have three methods:

  • Populate - this method will take the chart data and update the chart object's own internal data structures so that the chart can be measured and drawn,
  • Measure - this method takes the chart data and calculates how the chart should be drawn, i.e. how big it is, where the axis should be, where the bars should be, etc.
  • Paint - this method actually draws the chart.

Using Solution Explorer, create a new class called ChartBase. When the code editor appears, add namespace import declarations for System.Data and System.Drawing:

using System;
using System.Data;
using System.Drawing;

Then, add these methods:

public class ChartBase
{
// methods...
public virtual void Populate(ChartView view, DataTable data)
{
}

public virtual void Measure(ChartView view, DataTable data)
{
}

public virtual void Paint(ChartView view, DataTable data, Graphics graphics)
{
}
}

To make this exercise easier to follow, we'll build the stub of BarChart now, but we won't fill in any of the methods until later. Create a new class called BarChart, remember to set it to inherit from ChartBase, and add the three methods. We also need to include namespace import directives to System.Data and System.Drawing, although they're not shown in this code snippet.

public class BarChart : ChartBase
{
public override void Populate(ChartView view, DataTable data)
{
}

public override void Measure(ChartView view, DataTable data)
{
}

public override void Paint(ChartView view, DataTable data, Graphics graphics)
{
}
}

Now that we know what methods ChartBase supports, we can turn our attention back again to ChartView. We need to add members to hold the data and the ChartBase -derived object that will do the drawing for us. Open the code editor for ChartView and add these two members to the class definition:

public class ChartView : System.Windows.Forms.UserControl
{
// members...
private DataTable _data;
private ChartBase _chart;

When either of these properties changes we need to invalidate the view and the data. Invalidating the view is a pretty obvious activity - it means that at some point we need to repaint the control in order to see the changes to the data. Invalidating the data is a little less obvious - it means we need to repopulate the data in the ChartBase -derived object and also re-measure the data.

Add this method to ChartView:

      //
            InvalidateData - invalidate the data...
public void
            InvalidateData()
{
// do we have both data and
            chart?
if(Data != null && Chart != null)
{
//
            populate...
Chart.Populate(this, Data);
Chart.Measure(this,
            Data);
}

// redraw...
Invalidate();
}

Although we haven't built the Populate and Measure methods yet, we will do in a little while.

We now need Chart and Data properties. Any changes to either of these values needs to trigger a call to InvalidateData. Here are the property definitions:

      // Chart
            property...
public ChartBase Chart
{
get
{
return
            _chart;
}
set
{
_chart =
            value;
InvalidateData();
}
}

// Data
            property...
public DataTable Data
{
get
{
return
            _data;
}
set
{
_data =
            value;
InvalidateData();
}
}

When the control is resized, we need to get the chart to re-measure itself so that the chart always fills all of the available space. To do this, we need an event handler that listens for the control's Resize event. Find the constructor for ChartView and add a call to hook in the event handler to a method called OnResize:

      public
            ChartView()
{
// This call is required by the Form
            Designer.
InitializeComponent();

// TODO: Add any
            initialization after the InitForm call
this.Resize += new
            EventHandler(OnResize);
}

Then, add this method that calls the Measure method:

      //
            OnResize - the control's size has been changed...
public void
            OnResize(object sender, EventArgs e)
{
// do we have chart and
            data?
if(Data != null && Chart !=
            null)
Chart.Measure(this, Data);

//
            repaint...
Invalidate();
}

We're almost at a point where we can turn our attention to drawing the bar chart itself, but first we need to provide a way to actually show the ChartView control, and through this, eventually show the chart. To do this, we'll use a floating window separate to the default Form1 form that contains the DataGrid.

Using Solution Explorer, create a new Windows Form. Call it ChartPopup. We need to add a ChartView control to this form, but the view will not be available until you build the project. Select Build - Build from the menu and we should find that the ChartView control has become available:

Draw a ChartView control onto the popup. Set its Name property to chartView and its Dock property to Fill. We should now have this relatively dull looking form:

Next, we'll create Chart and Data properties on ChartPopup that defer over to the same properties on the ChartView. This will let us talk to the properties directly from Form1. Open the code editor for ChartPopup and add this code:

      // Chart
            property...
public ChartBase Chart
{
get
{
return
            chartView.Chart;
}
set
{
chartView.Chart =
            value;
}
}

// Data property...
public DataTable
            Data
{
get
{
return
            chartView.Data;
}
set
{
chartView.Data =
            value;
}
}

Now, when Form1 is created we need to create an instance of ChartPopup and show the window. We also need to maintain a link. Open the code editor for Form1 and add this member to the top of the class definition.

public class Form1 : System.Windows.Forms.Form
{
// members...
private ChartPopup _popup;

To communicate with the popup, we'll create a read-only property called Popup. This property will check to make sure that _popup contains an instance of a ChartPopup object and, if it doesn't, it will create a new one, populate the Chart property and show the window. The wrinkle we have to deal with is that if the user closes the ChartPopup window, we have to clear the _popup member. This means that next time we set the Chart or Data properties on Popup, we'll automatically create a new window if the user has closed it.

Add this property to Form1:

      // Popup
            property...
public ChartPopup Popup
{
get
{
// do we
            have one?
if(_popup == null)
{
// create and set
            Chart...
_popup = new ChartPopup();
_popup.Chart = new
            BarChart();

// wire in closed...
_popup.Closed += new
            EventHandler(OnPopupClosed);

// show
            it...
_popup.Show();
}

// return...
return
            _popup;
}
}

So what we're doing here is, provided _popup is null, we create a new popup and set its Chart property to a new instance of BarChart. This property defers over to the contained ChartView control instance, so effectively ChartView knows that at this point it's supposed to be working with BarChart. An important thing to note there is that we're tying the popup's Closed event into a method called OnPopupClosed. Add this method now:

      //
            OnPopupClosed - the user has closed the popup...
private void
            OnPopupClosed(object sender, EventArgs e)
{
//
            reset...
_popup = null;
}

All this method does is reset the _popup member back to null, so the next time we call the Popup property a new popup window will be created, configured to show BarChart displayed.

Finally, find the Regenerate method again and add a call to set the Data property of Popup right at the end. (I've omitted some code from this listing).

      public
            void Regenerate()
{
// create a table...
DataTable table =
            new DataTable();

// add a column for the
            name...
table.Columns.Add("Name", typeof(string));

// now,
            add columns for the four regions...
.

// set the
            data...
datagridChart.DataSource = table;
Popup.Data =
            table;
}

If you run the project now, the popup will be displayed. (Careful - Form1 may obscure it!) But, nothing will be displayed because we haven't done that bit yet!

Drawing the Chart

Drawing the chart is fairly straightforward. However, BarChart does need to hold the data in its own structures rather than relying on the DataTable and DataRow objects.

If we refer back to the Excel chart above, we notice that values from the four regions for each month are clustered together. We're going to call this a "group", each of these groups matching to exactly one DataRow in the DataTable. Each of the groups contains one or more "bars", each bar matching to exactly one column in the group's DataRow. When the chart is populated, or the view is resized, the graph redraws itself. Each bar needs to hold a rectangle that represents its position on the graph. When we redraw ourselves, all we have to do is iterate through the groups and the bars effectively filling in the rectangle that's stored against each bar. (If we were to add some 3D effects, though, it wouldn't be quite that simple). In addition, each bar will also store its color and its value, i.e. the corresponding column from DataRow.

Using Solution Explorer, create a new class called Bar. Add a namespace import declaration to System.Drawing and then add this code:

public class Bar
{
// members...
public Rectangle Rectangle;
public Color Color = Color.Red;
public Color LightColor = Color.Pink;
public Color DarkColor = Color.DarkRed;
public double Value;

We'll also create a new constructor for the bar that means we have to supply a value whenever we create it:

      public
            Bar(double barValue)
{
Value = barValue;
}

Assuming the bar has been populated, or more specifically, assuming the bar has been given a rectangle that represents its location on the chart, it can draw itself. We'll do this through a method called Paint. Add this method to Bar:

// Paint...
public void Paint(Graphics graphics)
{
// create the brush...
SolidBrush brush = new SolidBrush(Color);

// draw the rectangle...
graphics.FillRectangle(brush, Rectangle);

// dispose...
brush.Dispose();
}

To represent each group, we'll use another class called BarGroup. All this needs to do is hold a list of bars in the group and provide a mechanism through which the Paint method on Bar can be called. Create a new class called BarGroup and add these namespace import declarations:

using System;
using System.Data;
using System.Drawing;
using System.Collections;

Then, add these members:

public class BarGroup
{
// members...
public ArrayList Bars = new ArrayList();
public Rectangle Rectangle;
public DataRow Row;
public string Name;

Like Bar, we'll create a constructor that requires we provide a value. We'll also extract the name from the zeroeth column and go through all of the other columns creating new Bar objects, populating them with the value stored in the column.

      public
            BarGroup(DataRow row)
{
// set the row...
this.Row =
            row;

// get the name from the zeroth column...
this.Name =
            (string)row[0];

// create bars...
int colorIndex =
            0;
for(int n = 1; n < row.Table.Columns.Count; n++)
{
//
            create a bar...
Bar bar = new
            Bar((double)row[n]);
Bars.Add(bar);

As we add each bar, we need to set its color otherwise the graph will look a mess! We'll have a maximum of four colors and go through each one in turn as we add each bar.

            //
            set the color...
switch(colorIndex)
{
case 1:
bar.Color
            = Color.Blue;
bar.LightColor = Color.LightBlue;
bar.DarkColor
            = Color.DarkBlue;
break;
case 2:
bar.Color =
            Color.Green;
bar.LightColor = Color.LightGreen;
bar.DarkColor
            = Color.DarkGreen;
break;
case 3:
bar.Color =
            Color.DarkMagenta;
bar.LightColor =
            Color.Salmon;
bar.DarkColor =
            Color.Purple;
break;
}

//
            next...
colorIndex++;
if(colorIndex == 4)
colorIndex =
            0;
}
}

And then, add this method that loops through each of the bars asking them to paint themselves:

// Paint...
public void Paint(Graphics graphics)
{
// go through each bar...
foreach(Bar bar in Bars)
bar.Paint(graphics);
}

To draw the chart, we have to properly implement the Populate, Measure and Paint methods. Open the code editor for BarChart and find the stub Populate method that we created earlier. All this method has to do is clear the Groups array list and then loop through each of the rows creating new BarGroup objects. Remember, the BarGroup constructor iterates the columns in the row creating new Bar objects. Once the new BarGroup object has been created, we need to loop through each of the bars adjusting the MaxValue member. After we've done this, MaxValue should be set to the highest value contained within any of the bars.

      public
            override void Populate(ChartView view, DataTable data)
{
//
            reset the groups...
Groups.Clear();

// go through each
            row...
foreach(DataRow row in data.Rows)
{
//
            create...
BarGroup group = new
            BarGroup(row);
Groups.Add(group);

// go through each group
            adjusting max value...
foreach(Bar bar in
            group.Bars)
{
if(bar.Value < 0)
bar.Value =
            0;
if(bar.Value > MaxValue)
MaxValue =
            bar.Value;
}
}
}

Populate will be called whenever the Data or Chart properties of the ChartView are set. Measure will be called either when the Data or Chart properties are set, or when the view itself is resized. At this point, we need to go through the groups and through the bars, assigning rectangles to each of the bars.

To draw on the axis and labels, we need to hold some elements of the measurements as members of BarChart. Add these members to the top of BarChart:

public class BarChart : ChartBase
{
// members...
public ArrayList Groups = new ArrayList();
public int GroupWidth;
public int BarWidth;
public double MinValue = 0;
public double MaxValue;

Find the Measure method that we created earlier. If the Groups array list contains no entries, we don't need to measure anything so we can just quit.

      public
            override void Measure(ChartView view, DataTable data)
{
// do
            we have data?
if(Groups.Count == 0)
return;

In a short while, we'll add the DataRectangle property to ChartView. This property returns a rectangle, which describes the bounds of the chart as it should appear on the view. We hold this in a local variable called rect.

// how big are we?
Rectangle rect = view.DataRectangle;

We'll assume that each group contains the same number of bars. This makes sense if we're working with a table of data.

// assume each group has the same number of bars...
int numBarsInGroup = ((BarGroup)Groups[0]).Bars.Count;

Each group needs to take up an equal amount of space, so we'll define GroupWidth to be the width of the rectangle returned by DataRectangle divided by the number of groups. In addition, we'll say that BarWidth is equal to GroupWidth divided the number of bars in each group, plus one. The reason why we say, "plus one" is because we want a gap between each group.

         //
            how big is each group?
GroupWidth = rect.Width /
            Groups.Count;
BarWidth = GroupWidth / (numBarsInGroup +
            1);

Now that we know how wide each bar is, we can start at the left of the rectangle and work to the right. We loop through the group and then through each bar in the group. We use a local variable x to keep track of the left-hand side of the current bar as we work through them.

// go through each group...
int x = rect.Left;
foreach(BarGroup group in Groups)
{

For each group, we store the bounding rectangle of all the bars in the group's Rectangle member:

            //
            store the group's rectangle...
group.Rectangle = new Rectangle(x,
            rect.Top, GroupWidth - BarWidth, rect.Height);
// go through each
            bar...
foreach(Bar bar in group.Bars)
{

The height of the bar is the most important thing here. To find this, we defer to a method called CalculateHeightFromRect, passing in the rectangle of the chart and the value of the bar. Once we have the height, we can set the rectangle and then move x to the left hand side of the next bar.

               //
            how tall?
int height = (int)CalculateHeightFromValue(rect,
            bar.Value);

// set the rect for the bar...
bar.Rectangle =
            new Rectangle(x, rect.Bottom - height, BarWidth, height);

//
            move...
x += BarWidth;
}

Finally, after each group we move x again to create a space between groups.

            //
            space...
x += BarWidth;
}
}

At the moment, although we can populate and measure the graph, we can't paint it. Add this code to BarChart:

public override void Paint(ChartView view, DataTable data, Graphics graphics)
{
// paint each bar...
foreach(BarGroup group in Groups)
group.Paint(graphics);
}

All that remains now is to go back to ChartView in order to implement DataRectangle and get the paint code wired up. Open ChartView and add this property:

// DataRectangle property...
public Rectangle DataRectangle
{
get
{
return new Rectangle(50, 50, Width - 100, Height - 100);
}
}

Then, add this method that paints the chart:

      //
            OnPaint - draw the chart...
public void OnPaint(object sender,
            PaintEventArgs e)
{
// do we have a chart?
if(Chart != null
            && Data != null)
Chart.Paint(this, Data,
            e.Graphics);
}

The OnPaint method needs to be hooked into the Paint event. Add the call to ChartView 's constructor.

      public
            ChartView()
{
// This call is required by the Form
            Designer.
InitializeComponent();

// TODO: Add any
            initialization after the InitForm call
this.Resize += new
            EventHandler(OnResize);
this.Paint += new
            PaintEventHandler(OnPaint);
}

Try running the project now and we should see the basis of a graph:

If we click the Regenerate Sample Data link on Form1, the graph will update itself.

Axis and Labels

Before we leave the discussion for today and pick it up again tomorrow, we'll quickly add axis and labels to the graph.

If the project is running, close it. Open BarChart and find the paint method. After drawing the groups, we need to create pens and brushes that we can use to paint on the axis and labels.

      public
            override void Paint(ChartView view, DataTable data, Graphics
            graphics)
{
// paint each bar...
foreach(BarGroup group in
            Groups)
group.Paint(graphics);

// pens...
Pen pen = new
            Pen(view.ForeColor);
SolidBrush brush = new
            SolidBrush(view.ForeColor);

// rect...
Rectangle dataRect
            = view.DataRectangle;

The axis itself is pretty easy - just draw lines ruling the bottom and left edges of the data rectangle.

// paint on axis...
graphics.DrawLine(pen, dataRect.Right, dataRect.Bottom, dataRect.Left, dataRect.Bottom);
graphics.DrawLine(pen, dataRect.Left, dataRect.Bottom, dataRect.Left, dataRect.Top);

When we come to draw the text, we'll need to create alignments. The x-axis labels need to be centrally aligned, whereas the y-axis labels need to be aligned to the right and vertically aligned to the center.

         //
            text format...
StringFormat xFormat = new
            StringFormat();
xFormat.Alignment =
            StringAlignment.Center;
StringFormat yFormat = new
            StringFormat();
yFormat.Alignment =
            StringAlignment.Far;
yFormat.LineAlignment =
            StringAlignment.Center;

Firstly, we go through the groups rendering the name under each one:

// add the group labels...
foreach(BarGroup group in Groups)
{
// rect...
RectangleF textRect = new RectangleF(group.Rectangle.X,
group.Rectangle.Bottom + 10,
group.Rectangle.Width, 30);
graphics.DrawString(group.Name, view.Font, brush, textRect,
xFormat);
}

Rendering the y-axis labels is a little trickier. What we want to do is have ten "intervals" that we mark up the left-hand side with ticks. What we'll do is look at MaxValue and, assuming that we have ten intervals, the interval should be MaxValue / 10. We then render from the zeroeth interval up to the tenth using the CalculateHeightFromValue method again to determine where it should appear.

         //
            add the y labels...
int numIntervals = 10;
double interval =
            MaxValue / numIntervals;
double intervalValue = 0;
for(int n =
            0; n <= numIntervals; n++)
{
// where should the tick
            be...
int height = CalculateHeightFromValue(dataRect,
            intervalValue);
int y = dataRect.Bottom - height;

// draw
            it...
graphics.DrawLine(pen, dataRect.Left, y, dataRect.Left - 5,
            y);

// label...
RectangleF textRect = new
            RectangleF(dataRect.Left - 60, y - 10, 50,
            20);
graphics.DrawString(intervalValue.ToString("f0"), view.Font,
            brush, textRect, yFormat);

// next...
intervalValue +=
            interval;
}

When we've finished painting, we clean up the pens and the brushes:

         //
            dispose...
pen.Dispose();
brush.Dispose();
}

Now run the project and we should see a labeled chart.

If you look, though, you'll notice that the graph has an odd scale on the intervals. We're more used to seeing scales that end on numbers like 10, 25, 100, etc. What we need to do is tweak MaxValue when we measure so that it always ends on one of these numbers, rather than 236 or some less natural number.

If the project is running, close it. Find the Populate method on BarChart and add this block of code to the end.

      public
            override void Populate(ChartView view, DataTable data)
{
//
            reset the groups...
Groups.Clear();

// go through each
            row...
foreach(DataRow row in data.Rows)
{
//
            create...
BarGroup group = new
            BarGroup(row);
Groups.Add(group);

// go through each group
            adjusting max value...
foreach(Bar bar in
            group.Bars)
{
if(bar.Value < 0)
bar.Value =
            0;
if(bar.Value > MaxValue)
MaxValue =
            bar.Value;
}
}

// adjust max...
int stopOn =
            10;
if(MaxValue % stopOn != 0)
MaxValue += stopOn - (MaxValue
            % stopOn);
}

Now run the project and you'll find that the intervals on the scale are more "natural".

Next Time

In the next installment of this article we'll take a look at how we can add mouse interactivity to the chart so that we can see and change values.


Developing A Vector Based C# Drawing Application - Part 3

Developing A Vector Based C# Drawing Application - Part 3
http://www.csharptoday.com/content.asp?id=1657
Article Rating: 3.5
Published on September 20, 2001
Abstract
In the first part of this series, Budi Kurniawan discussed the specification for a vector based C# drawing application. In Part 2, we looked at the Shape abstract class and the seven classes that inherit it. We also examined how an event handler works in a Windows application and we briefly discussed the System.Collections namespace. In this final part of the project, we will complete the development by discussing the DrawingArea custom control, the Color and Pen classes from the System.Drawing namespace, and object serialization - which is used to persist our drawing to a file, to be restored at a later stage.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

In the first part of this series, we discussed the specification for a vector based C# drawing application. In Part 2, we looked at the Shape abstract class and the seven classes that inherit it. We also examined how an event handler works in a Windows application and we briefly discussed the System.Collections namespace. In this final part of the project, we will complete the development of the project by discussing the DrawingArea custom control, the Color and Pen classes from the System.Drawing namespace, and object serialization. (Object serialization is used to persist our drawing to a file, to be restored at a later stage).

Figure 1: The Final Drawing Application

The ArrayList class

At the end of Part 2, we listed the classes, interfaces and the structure in the System.Collections namespace and gave an explanation of why ArrayList is suitable to be a container that holds the drawn shape objects. In the drawing application, we declare shapesArrayList as an ArrayList inside the DrawForm as well as in the DrawingArea class. We instantiate the ArrayList object in the form so that we can store all the shapes the user draws in the ArrayList, but we also pass a reference to the DrawingArea object so that the DrawingArea can repaint all the shapes when it needs to. With an ArrayList, we can also delete an object when it is no longer needed. When working with an ArrayList, the way we retrieve each object in the collection is also important.

The ArrayList class has the Add method to add an object to the collection and a RemoveAt method to remove an object at a position specified by an index passed to the method. To remove all objects from an ArrayList, we use the Clear method.

While those methods are very straightforward, iterating through an ArrayList is not so simple. We first need to create an IEnumerator object using the GetEnumerator method of the ArrayList class. Then we iterate through the members using the MoveNext method of IEnumerator and get the object member using the Current property.

Creating a DrawingArea custom control

For our drawing application, we need a drawing area: a canvas on which the user can draw shapes. Despite the richness of the .NET Framework Base Class Library, there is no such Canvas control that can be readily added to the form. However, we can create our own Windows Forms Canvas custom control. In the attachments to Part 1 and Part 2 of this series, there was a DrawingArea control that is added to the form. However, the code for that part is not complete and we haven't discussed I yet. Now is the time.

All the controls in the System.Windows.Forms namespace derive from the Control class, either directly or indirectly. For example, Label, GroupBox, PictureBox derive directly from Control; while Button, CheckBox and RadioButton are some of the controls that derive indirectly from Control. The Control class implements very basic functionality required by classes that display a UI to the user. It handles user input through the keyboard and pointing device. The Control class also handles message routing and security. It defines the bounds of a control (its position and size), although it does not paint its body. It also provides a window handle (hWnd).

We can create a custom control by extending the Control class or the UserControl class. By extending this class, our customer control inherits all non-private members of Control. An important aspect of writing our custom control is the user interface. The Control class automatically refreshes its user interface whenever necessary, for example when it is restored from a minimized state and when it gets a focus. As a result, we need to override the Control class's OnPaint method. For instance, if our control displays a circle, that circle must be repainted every now and then. The OnPaint method is invoked when the Paint event occurs.

The OnPaint method receives an argument of type PaintEventArgs containing data related to this event. The properties of PaintEventArgs provide important information. The ClipRectangle read-only property indicates the rectangle in which to paint, and the Graphics property indicates the Graphics object used to paint.

The custom control we write for our drawing application is named DrawingArea. It consists of an object variable of type ArrayList called shapesArrayList. This is a collection where all shape objects are stored. The DrawingArea class is very simple. It only overrides the OnPaint method. Every time the DrawingArea control is repainted, the control must repaint all the shape objects in shapesArrayList. The constructor for the DrawingArea has been modified so we can pass an ArrayList from inside the form.

The code for the DrawingArea class is given in Listing 1.

Listing 1: The DrawingArea Custom Control

using System;
using
            System.Drawing;
using System.Windows.Forms;
using
            System.Collections;
namespace DrawApp
{
public class
            DrawingArea : Control
{
ArrayList shapesArrayList;
public
            DrawingArea(ArrayList shapesArrayList)
{
this.shapesArrayList
            = shapesArrayList;
}

override protected void
            OnPaint(PaintEventArgs e)
{
Graphics graphics =
            e.Graphics;
if (shapesArrayList!=null) {
IEnumerator
            enumerator = shapesArrayList.GetEnumerator();
while
            (enumerator.MoveNext()) {
Shape shape = (Shape)
            enumerator.Current;
if (shape!=null)
{
shape.Draw
            (graphics);
}
}
}
}
}
}

First, the OnPaint method obtains the Graphics object from the passed PaintEventArgs object. Then it creates an IEnumerator object called enumerator by calling the shapesArrayList 's GetEnumerator method. Next, it retrieves each shape object by using the IEnumerator interface's MoveNext method and obtains the shape object through the IEnumerator interface's Current property. Note that the shape object is returned as an object of type System.Object. Therefore, you need to downcast it to a Shape object. Once you get the shape object, you need to call the Draw method to draw the shape on the DrawingArea control.

Creating a Color object

Color, which is part of the System.Drawing namespace, is an important structure used frequently when you draw something. The Color structure provides a large number of static properties that allow you to easily obtain a Color object. For instance, The Red property returns a red color object. To obtain the color object, you write Color.Red.

The Color object is used frequently in our application. For example, we can assign a button's BackColor property with a yellow color, such as:

button1.BackColor = System.Drawing.Color.Yellow;

and another button's BackColor property with a red color, such as:

button2.BackColor = System.Drawing.Color.Red;

However, there are times when we want to use a color that is not provided as a property of the Color structure. It is common to define a color from its RGB component. For this purpose, the Color structure provides the FromArgb method. This method accepts three integers signifying the red, green and blue intensity of the color. For instance, to obtain a Color object with the RGB components of (244, 244, 244), we write Color.FromArgb(244, 244, 244).

If your application is graphic intensive, you probably need more than 24 bit colors. The Color structure supports 32 bit colors as well. One of the FromArgb overloads accepts four arguments: Alpha component, R component, G component and B component.

Creating a Pen object

To draw a shape, we need a pen. For example, the various draw methods in the Graphics class require us to pass a pen with which to draw. Take DrawEllipse as an example. The final Pen class in the System.Drawing namespace is provided to enable .NET programmers to create a Pen object to draw lines and curves. Once instantiated, a Pen draws a line of specified width and style. There are a number of styles we can experiment with. For instance, we can use the DashStyle property to draw several varieties of dashed lines. Also, the line drawn by a Pen object can be filled in a variety of fill styles, including solid colors and textures. The fill style depends on brush or texture that is used as the fill object.

Out of the four Pen class's constructors, the simplest is the one that accepts a Color object as an argument. With this constructor, we can create a Pen with a specified Color and a default width. For instance, to create a red pen called myPen, write the following code:

Pen myPen = new Pen(Color.Red);

If we also want to define the width of the pen, use the other constructor that accepts two arguments: a Color object and a Single. We can pass the width as the second argument.

The more sophisticated constructors require us to pass a Brush object or a Brush object and a Single. The System.Drawing.Brush class is abstract, therefore cannot be instantiated directly to create a Brush object. However, we can use one of its direct descendants: SolidBrush, TextureBrush and LinearGradientBrush. In our application, SolidBrush is used in the Draw methods of the FilledRectangle, BorderlessRectangle, FilledEllipse and BorderlessEllipse classes. The only constructor of the SolidBrush class, requires a Color object to be passed as an argument.

For example, the FillEllipse of the Graphics method, which is used in the BorderlessEllipse class and FilledEllipse class in our application, is written this way:

graphics.FillRectangle(new SolidBrush(backgroundColor),
x1 + penWidth, y1 + penWidth, x2 - x1 - penWidth,
y2 - y1 - penWidth);

Saving our drawings

One of the more important features of a drawing application is the ability to persist the drawing. Our application provides a Save menu item under the File menu that users can click to save what they have drawn into a file. Because each shape is an object of its own, we need a way to save objects and later retrieve them back.

Serialization

The .NET Framework Base Class Library supports object persistence through object serialization. Serialization is the process of converting an object or a graph of objects into a linear sequence of bytes for either storage or transmission to another location. The opposite process is called deserialization. Deserialization is the process of taking in stored information and recreating objects from it.

The System.Runtime.Serialization namespace contains classes that can be used for serializing and deserializing objects. The ISerializable interface provides a way for classes to control their own serialization behavior. Classes in the System.Runtime.Serialization.Formatters namespace control the actual formatting of various data types encapsulated in the serialized objects.

There are two formats supported for object serialization and deserialization in .NET Framework: binary and SOAP. To serialize an object, we need to create an IFormatter object from classes that implement this interface. In the .NET Framework, there are three such classes:

  • System.Runtime.Serialization.Formatter,
  • System.Runtime.Serialization.Formatters.Binary.BinaryFormatter and
  • System.Runtime.Serialization.Formatters.Soap.SoapFormatter.
  • System.Runtime.Serialization.Formatter is an abstract class that provides basic functionality for the common language runtime serialization and provides some helper methods for implementing the IFormatter interface. The Formatter class also manages queuing objects for serialization and generating IDs on a per-object basis. However, the Formatter type is not CLS (Common Language Specification) compliant.
  • The BinaryFormatter class serializes and deserializes an object, or an entire graph of connected objects in binary format. The binary format is very compact and can be parsed quickly.
  • The SoapFormatter class serializes and deserializes an object, or an entire graph of connected objects in SOAP format. Our drawing application uses the latter because the file produced is basically an XML file. As such, this can serve as a good learning tool because the file produced can be viewed with any text editor.

To serialize an object in SOAP format, we need to first create an IFormatter object from a SoapFormatter class.

IFormatter formatter;
formatter = new
            SoapFormatter();

Then, you can use the IFormatter interface's Serialize method to serialize the object. The Serialize method accepts two arguments: a Stream object to which objects are to be serialized into and the object to be serialized. In our case, the object to be serialized is the shapesArrayList inside the DrawForm. The Stream object is one that the user needs to select.

The .NET Framework provides a namespace called System.IO that contains types that allow synchronous and asynchronous reading and writing of data streams and files. The Stream class is one of the members, and this is important in our application because to serialize our shape objects we need a Stream object. The System.IO.Stream class itself is abstract, however it is derived by a number of classes that we can instantiate.

One such class is FileStream. This looks ideal for our application because we intend to persist our shape objects into a file. FileStream is useful for reading and writing files on a file system, as well as other file-related operating system handles (including pipes, standard input, standard output, and so on). Also, FileStream buffers input and output for better performance. The FileStream class can open a file in one of two modes, either synchronously or asynchronously, with significant performance consequences for the synchronous methods (Read and Write) and the asynchronous methods (BeginRead and BeginWrite). Both sets of methods will work in either mode; however, the mode will affect the performance of these methods. FileStream defaults to opening files synchronously, but provides a constructor to open files asynchronously.

It is actually very easy to create a FileStream object. One of the FileStream class's constructors allows you to create a FileStream object by passing a string containing the file path and the file mode. However, hard coding the file path makes the application have to serialize objects to this file only. Fortunately, there is a way to give users the flexibility of choosing their own directory to save the file to. The System.Windows.Forms namespace provides a class called FileDialog. However, this class is abstract therefore cannot be instantiated directly. To create a dialog to save a file, you can use the SaveFileDialog class, which is a class derived from FileDialog. The SaveFileDialog class has a method called ShowDialog that displays a modal dialog to let the user to browse a directory and enter a file name. A SaveFileDialog is shown in Figure 2.

Figure 2: A SaveFileDialog for the user to select a directory and enter a filename to save shape objects into.

If the user enters a file name and clicks the OK button, the ShowDialog method will return a Dialog.OK. We can then invoke its OpenFile method to create a Stream object, which in turn is used as an argument to the Serialize method for our object serialization. The saveDocument method is given in Listing 2.

Listing 2: The saveDocument method

private void saveDocument(object sender ,
            EventArgs e)
{
Stream myStream;
SaveFileDialog
            saveFileDialog = new SaveFileDialog();
saveFileDialog.Filter =
            "All files (*.*)|*";
if
            (saveFileDialog.ShowDialog()==DialogResult.OK)
{
myStream =
            saveFileDialog.OpenFile();
if (myStream!=null)
{
IFormatter formatter;
formatter = (IFormatter) (new
            SoapFormatter());
formatter.Serialize(myStream,
            shapesArrayList);
myStream.Close();
}
}
}

Note that for a class's object to be serializable, the class must be marked [Serializable]. Our application also allows serialized shape objects to be restored into shapesArrayList. This is achieved through the openDocument method. This method is the opposite of the saveDocument method. It first uses an OpenFileDialog object to open a dialog that allows the user to browse the computer's directory system and choose a file. The OpenFileDialog, like SaveFileDialog, also derives from System.Windows.Forms.FileDialog. To show an Open dialog, you use the OpenFileDialog class's OpenFile method. The OpenFileDialog is shown in Figure 3.

Figure 3: An OpenFileDialog.

Like in object serialization, to deserialize objects we need the IFormatter object that has been explained above. Instead of using the Serialize method, however, we use the Deserialize method that opens a Stream object. The Deserialize method returns an object of type System.Object. Since we want the shape objects to be loaded into shapesArrayList, we need to cast the output of Deserialize into an ArrayList.

Listing 3: The openDocument method

private void openDocument(object sender ,
            EventArgs e)
{
Stream myStream;
OpenFileDialog
            openFileDialog = new OpenFileDialog();
// filter the file
            extension that can be selected
openFileDialog.Filter = "Draw
            Application Files" +
" (*.draw)|*.draw|All files
            (*.*)|*.*";
openFileDialog.FilterIndex =
            2;
openFileDialog.RestoreDirectory = true;
if
            (openFileDialog.ShowDialog()==DialogResult.OK)
{
myStream =
            openFileDialog.OpenFile();
if (myStream!=null)
{
IFormatter formatter;
formatter = (IFormatter) (new
            SoapFormatter());
shapesArrayList =
            (ArrayList)
formatter.Deserialize(myStream);
myStream.Close();

Graphics
            graphics = canvas.CreateGraphics();
if (shapesArrayList!=null)
{
IEnumerator enumerator =
shapesArrayList.GetEnumerator();
while
            (enumerator.MoveNext())
{
Shape shape = (Shape)
            enumerator.Current;
if (shape!=null)
{
shape.Draw
            (graphics);
}
}
}
}
}
}

Summary

This is the last article of this series. We have seen the development process of a drawing application in C#. We have also learned techniques such as event handling, custom control creation, and object serialization. There are a number of possible improvements to the application - we could include the facility to resize drawn objects from a previously saved file for example. However such functionality is beyond the scope of this mini-series and is left for further projects. The complete application can be downloaded below.


Developing A C# Vector Drawing Application - Part 2

Developing A C# Vector Drawing Application - Part 2
http://www.csharptoday.com/content.asp?id=1575
Article Rating: 3.5
Published on September 13, 2001
Abstract
This is the second part of a series where we build a vector based drawing application. In Part 1, Budi Kurniawan outlined the specification for the application and discussed some techniques for working with forms with the .NET Framework Base Class Library. In this part, we will continue with our development, first by discussing the various shapes for our drawing application and then writing event handling for some of the controls.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

This is the second part of a series where we build a vector based drawing application. In Part 1, we outlined the specification for the application and discussed some techniques for working with forms with the .NET Framework Base Class Library. These techniques included how to create a form by extending the Form class in the System.Windows.Forms namespace, add controls to a form, group a set of controls, etc. Part 1 can be found here.

In Part 2 of this article, we will continue with our development. First of all, we will discuss the various shapes for our Drawing application and write event handling for some of the controls. The code for this part is also provided as a downloadable attachment.

Shapes

As mentioned in Part 1, we can draw seven different shapes with this application. These shapes are:

  • Line,
  • Rectangle,
  • FilledRectangle,
  • BorderlessRectangle,
  • Ellipse,
  • FilledEllipse, and
  • BorderlessEllipse.

Figure 1 shows the shapes that can be drawn in our application.

Figure 1: Various shapes that can be drawn.

All the seven shapes extend the abstract class Shape. In C# an abstract class is indicated by the keyword abstract. An abstract class is a class that cannot be instantiated directly and must be inherited. Therefore, an abstract class called Shape has the following signature.

public abstract class Shape

The Shape abstract class has the following fields.

  • backgroundColor. A protected Color object that represents the background color of the shape.
  • foregroundColor. A protected Color object that represents the foreground color of the shape.
  • penWidth. A protected integer representing the thickness of the lines that compose the object. It has a value in the range of 1 to 5.
  • x1, y1. Two protected integer values that represent the first coordinate of the shape, i.e. the coordinate that is the start point of the drawing of the shape by the user.
  • x2, y2. Two protected integer values that represent the second coordinate of the shape, i.e. the coordinate that is the end point of the drawing of the shape by the user.

All the fields are declared as protected so that they are accessible from the child classes Shape.

The Shape class has one constructor that accepts two Point objects, an integer, and two Color objects. The first point, p1, is the coordinate in the drawing area where the user clicks the mouse to start drawing. The second point, p2, indicates the end point, i.e. the position of the mouse cursor when the user releases the mouse button.

From these two points, we can derive the two coordinates (x1, y1) and (x2, y2) and store them in the x1, y1, x2, y2 fields. However, as we will see later, drawing a shape, except a line, requires the first coordinate to be always on the left and upper of the second coordinate. If the user drags the mouse upwards or to the left, we will receive a Point p2 whose x or y components are lower than those of Point p1. We therefore need to modify these points, as described in the constructor of the Shape class, given in Listing 1.

The Shape class has only one method: Draw. This is an abstract method that needs to be overridden by the child class that extends Shape. The method accepts a Graphics object where the shape is to be drawn on.

The code for the abstract Shape class is given in Listing 1. Note that in Part 3 we will slightly modify this class to allow it to be serialized.

Listing 1: The Abstract Shape Class

      public
            Shape(Point p1, Point p2, int penWidth,
Color foregroundColor,
            Color backgroundColor)
{
int x1, y1, x2, y2;
x1 =
            p1.X;
y1 = p1.Y;
x2 = p2.X;
y2 = p2.Y;

if (x2 >
            x1)
{
this.x1 = x1;
this.x2 = x2;
}
else
{
this.x1 = x2;
this.x2 = x1;
}
if (y2 > y1)
{
this.y1 = y1;
this.y2 = y2;
}
else
{
this.y1 = y2;
this.y2 = y1;
}
this.penWidth =
            penWidth;
this.foregroundColor =
            foregroundColor;
this.backgroundColor =
            backgroundColor;
}

public abstract void Draw(Graphics
            graphics);

}

Drawing a shape in the classes that extend Shape is trivial. In our application, we have seven classes that are derived from Shape. These classes are Line, Rectangle, FilledRectangle, BorderlessRectangle, Ellipse, FilledEllipse and BorderlessEllipse. Each child class must override the Draw method from the Shape class.

To draw a shape, we use one of the many methods of the Graphics object, passed when the Draw method is called. The Graphics class is a member of the System.Drawing namespace and cannot be extended because it is sealed - we cannot inherit from it. However, some of its methods that can be used to draw shapes are the following.

  • DrawLine. The method draws a line on the Graphics object from the start coordinate (x1, y1) to the end coordinate (x2, y2). It has four overloads. The overload that is used in the Line class is the one that accepts a Pen object and four integers as (x1, y1) and (x2, y2) coordinates.
  • DrawRectangle. This method draws a rectangle on the Graphics object. This method is used in the FilledRectangle and the Borderless classes. It has three overloads; the one used in our applications is the one that accepts a Pen object and four integers.
  • FillRectangle. This method fills the interior of a rectangle. This method is used in the FilledRectangle and the BorderlessRectangle classes.
  • DrawEllipse. This method draws an ellipse specified in a bounding rectangle. This is used in the Ellipse class. The overload that is used is the one that accepts a Pen object and four integers for the bounding rectangle.
  • FillEllipse. This method fills the interior of an ellipse defined by a bounding rectangle.

While most classes needs only one method to draw a shape, the FilledRectangle and the FilledEllipse classes need two. The Draw method in the FilledRectangle class first uses the DrawRectangle method to draw a rectangle and then calls the FillRectangle method to fill its interior. The Draw method in the FilledEllipse class first uses the DrawEllipse method to draw an ellipse and the calls the FillEllipse method to fill its interior.

The code for the seven shape classes are given in Listing 2.

Listing 2: The Classes That Inherits Shape

using System;
using System.Drawing;

namespace DrawApp
{
public abstract class Shape
{
protected int x1, y1, x2, y2;
protected int penWidth;
protected Color foregroundColor;
protected Color backgroundColor;

public Shape(Point p1, Point p2, int penWidth,
Color foregroundColor, Color backgroundColor)
{
int x1, y1, x2, y2;
x1 = p1.X;
y1 = p1.Y;
x2 = p2.X;
y2 = p2.Y;

if (x2 > x1)
{
this.x1 = x1;
this.x2 = x2;
}
else
{
this.x1 = x2;
this.x2 = x1;
}
if (y2 > y1)
{
this.y1 = y1;
this.y2 = y2;
}
else
{
this.y1 = y2;
this.y2 = y1;
}
this.penWidth = penWidth;
this.foregroundColor = foregroundColor;
this.backgroundColor = backgroundColor;
}

public abstract void Draw(Graphics graphics);
}

public class Line: Shape
{
public Line(Point p1, Point p2, int penWidth,
Color foregroundColor, Color backgroundColor) :
base(p1, p2, penWidth, foregroundColor, backgroundColor)
{
this.x1 = p1.X;
this.y1 = p1.Y;
this.x2 = p2.X;
this.y2 = p2.Y;
}
override public void Draw(Graphics graphics)
{
graphics.DrawLine(new Pen(foregroundColor, penWidth), x1, y1, x2, y2);
}
}

public class Rectangle: Shape
{
public Rectangle(Point p1, Point p2, int penWidth,
Color foregroundColor, Color backgroundColor) :
base(p1, p2, penWidth, foregroundColor, backgroundColor)
{
}
override public void Draw(Graphics graphics)
{
graphics.DrawRectangle(new Pen(foregroundColor, penWidth),
x1, y1, x2-x1, y2-y1);
}
}

public class FilledRectangle: Shape
{
public FilledRectangle(Point p1, Point p2, int penWidth,
Color foregroundColor, Color backgroundColor) :
base(p1, p2, penWidth, foregroundColor, backgroundColor)
{
}
override public void Draw(Graphics graphics)
{
graphics.DrawRectangle(new Pen(foregroundColor, penWidth),
x1 + (penWidth / 2), y1 + (penWidth / 2), x2-x1, y2-y1);
graphics.FillRectangle(new SolidBrush(backgroundColor),
x1 + penWidth, y1 + penWidth, x2 -x1 - penWidth, y2 - y1 -
penWidth);
}
}

public class BorderlessRectangle: Shape
{
public BorderlessRectangle(Point p1, Point p2, int penWidth,
Color foregroundColor, Color backgroundColor) :
base(p1, p2, penWidth, foregroundColor, backgroundColor)
{
}
override public void Draw(Graphics graphics)
{
graphics.FillRectangle(new SolidBrush(backgroundColor),
x1, y1, x2 - x1, y2 - y1);
}
}

public class Ellipse: Shape
{
public Ellipse(Point p1, Point p2, int penWidth,
Color foregroundColor, Color backgroundColor) :
base(p1, p2, penWidth, foregroundColor, backgroundColor)
{
}
override public void Draw(Graphics graphics)
{
graphics.DrawEllipse((new Pen(foregroundColor, penWidth)),
x1, y1, x2 - x1, y2 - y1);
}
}

public class FilledEllipse: Shape
{
public FilledEllipse(Point p1, Point p2, int penWidth,
Color foregroundColor, Color backgroundColor) :
base(p1, p2, penWidth, foregroundColor, backgroundColor)
{
}
override public void Draw(Graphics graphics)
{
graphics.DrawEllipse(new Pen(foregroundColor, penWidth),
x1 + (penWidth / 2), y1 + (penWidth / 2), x2 - x1, y2 - y1);

graphics.FillEllipse(new SolidBrush(backgroundColor),
x1 + penWidth, y1 + penWidth,
x2 - x1 - penWidth, y2 - y1 - penWidth);
}
}

public class BorderlessEllipse: Shape
{
public BorderlessEllipse(Point p1, Point p2, int penWidth,
Color foregroundColor, Color backgroundColor) :
base(p1, p2, penWidth, foregroundColor, backgroundColor)
{
}
override public void Draw(Graphics graphics)
{
graphics.FillEllipse(new SolidBrush(foregroundColor), x1, y1,
x2 - x1, y2 - y1);
}
}
}

Event Handling

In C# we can write code that responds to an event easily. As an example, the canvas object variable must respond to the MouseUp and MouseDown events. For this to happen, we need to create a new MouseEventHandler object for each event. To the constructor of the MouseEventHandler, we pass the method that will be called when the event occurs.

canvas.MouseUp += new
            MouseEventHandler(canvas_MouseUp);
canvas.MouseDown += new
            MouseEventHandler(canvas_MouseDown);

Then, create both canvas_MouseUp and canvas_MouseDown event handlers.

private void canvas_MouseDown(object sender, MouseEventArgs e)
{
// code that will be executed when the event occurs
}

private void canvas_MouseUp(object sender, MouseEventArgs e)
{
// code that will be executed when the event occurs
}

MouseUp and MouseDown are only two of the many events for which we can write an event handler. The Form class and all the control classes inherit the events from the Control class. If you look up the .NET Framework reference for the System.Windows.Forms.Control class, you will see that this class has 58 events, including the Disposed event inherited from Component. A few of the more popular ones are: Click, DoubleClick, GotFocus, LostFocus, KeyUp, KeyDown, etc.

The code in our application uses event handling extensively. This is a list of the event handlers used in the code.

When the user clicks a shape button, the event handler of the button calls the ChangeShape method, passing one of the enumeration values in ShapeTypeEnum. The event handler receives two arguments: the sender object and an EventArgs object, which represents the event argument. It then casts the sender object into a Button object and queries the name of the Button, assigning an appropriate value to the class's shapeType variable.

The ChangeShape method is given in Listing 3.

Listing 3: The ChangeShape method

private void changeShape(object sender,
            EventArgs e)
{
Button button = (Button)
            sender;
Console.WriteLine(button.Name);
if
            (button.Name.Equals("lineButton"))
shapeType = (int)
            ShapeTypeEnum.Line;
else
            if(button.Name.Equals("rectangleButton"))
shapeType = (int)
            ShapeTypeEnum.Rectangle;
else
            if(button.Name.Equals("filledRectangleButton"))
shapeType = (int)
            ShapeTypeEnum.FilledRectangle;
else
            if(button.Name.Equals("borderlessRectangleButton"))
shapeType =
            (int) ShapeTypeEnum.BorderlessRectangle;
else
            if(button.Name.Equals("ellipseButton"))
shapeType = (int)
            ShapeTypeEnum.Ellipse;
else
            if(button.Name.Equals("filledEllipseButton"))
shapeType = (int)
            ShapeTypeEnum.FilledEllipse;
else
            if(button.Name.Equals("borderlessEllipseButton"))
shapeType =
            (int) ShapeTypeEnum.BorderlessEllipse;
}

When the user clicks one of the thickness buttons, the event handler of that button assigns an integer that defines a thickness to the penWidth class variable.

When the user clicks one of the color buttons, the event handler of that button calls the ChangeColor method, passing the sender object and a MouseEventArgs object. Among other things, the MouseEventArgs class has the Button property that represents the mouse button. If the left mouse is clicked, we change the foreground color. On the other hand, if the right mouse is clicked, we change the background color. The new color is taken from the button's BackColor property. To obtain the Button object, we cast the sender object into a Button object. The ChangeColor method is given in Listing 4.

Listing 4: The ChangeColor method.

private void changeColor(object sender,
            MouseEventArgs e)
{
if (e.Button ==
            MouseButtons.Left)
{
foregroundColor = ((Button)
            sender).BackColor;
displayButton2.BackColor =
            foregroundColor;
}
else if (e.Button ==
            MouseButtons.Right)
{
backgroundColor = ((Button)
            sender).BackColor;
displayButton1.BackColor =
            backgroundColor;
}
}

When drawing a shape, the user needs to click on the drawing area, drag the mouse and release the mouse at the end point. When the mouse is clicked, the DrawingArea 's MouseDown event is triggered. The event handler then retrieves the X and Y coordinates of the MouseEventArgs passed to the event handler and pass them to the startPoint Point object. The X value represents the distance (in pixels) of the mouse position from the left side of the drawing area. The Y value is the distance (in pixels) of the mouse position from the top of the drawing area.

When the user releases the mouse button when drawing a shape, the DrawingArea 's MouseUp event is triggered. The event handler does a number of things. First, it retrieves the mouse position when the mouse is released and assigned the coordinate to the class's variables x2 and y2. It then assigns the shape object variable with a new object. Note that the shape object variable, which is a Shape, can be assigned to one of the seven child objects whose classes extend Shape. This is polymorphism - which in object-oriented programming is defined as the way an object variable can be assigned an object of another type as long as the other object's class extends the original object's class. In this case, a Rectangle or Line object can be assigned to shape because both Rectangle and Line extend the Shape abstract class. The same applies to FilledRectangle, BorderlessRectangle, Ellipse, FilledEllipse and BorderlessEllipse. The next step is to call Draw to draw the shape.

The MouseDown and MouseUp event handlers are given in Listing 5.

Listing 5: The MouseDown and MouseUp Event Handlers

private void canvas_MouseDown(object sender, MouseEventArgs e)
{
startPoint = new Point(e.X, e.Y);
}

private void canvas_MouseUp(object sender, MouseEventArgs e)
{
Graphics graphics = canvas.CreateGraphics();
Point endPoint = new Point(e.X, e.Y);
Shape shape = null;

switch (shapeType)
{
case (int) ShapeTypeEnum.Line:
shape = new Line(startPoint, endPoint, penWidth,
foregroundColor, backgroundColor);
break;
case (int) ShapeTypeEnum.Rectangle:
shape = new Rectangle(startPoint, endPoint, penWidth,
foregroundColor, backgroundColor);
break;
case (int) ShapeTypeEnum.FilledRectangle:
shape = new FilledRectangle(startPoint, endPoint,
penWidth,foregroundColor, backgroundColor);
break;
case (int) ShapeTypeEnum.BorderlessRectangle:
shape = new BorderlessRectangle(startPoint, endPoint,
penWidth, foregroundColor, backgroundColor);
break;
case (int) ShapeTypeEnum.Ellipse:
shape = new Ellipse(startPoint, endPoint, penWidth,
foregroundColor, backgroundColor);
break;
case (int) ShapeTypeEnum.FilledEllipse:
shape = new FilledEllipse(startPoint, endPoint,
penWidth, foregroundColor, backgroundColor);
break;
case (int) ShapeTypeEnum.BorderlessEllipse:
shape = new BorderlessEllipse(startPoint, endPoint,
penWidth, foregroundColor, backgroundColor);
break;
}
shape.Draw(graphics);
}

.NET Collections

Remember that in the first part of this article series, we reviewed two types of draw applications: vector-based and raster based. Our application falls into the first category. As such, each stroke produces a separate shape, a separate object. In order for us to hold these objects for later use, for examples for storing them in a file, we need a kind of object container. The .NET Framework Class Library provides a namespace called System.Collections that contains classes and interfaces that various collections of objects, such as lists, queues, arrays, hashtables and dictionaries.

The following are classes in the System.Collections namespace - for our shape container, we choose an ArrayList because its size can grow flexibly and we don't need a container that can store name and value pairs, such as a HashTable.

  • ArrayList. Implements the IList interface using an array whose size is dynamically increased as required.
  • BitArray. Manages a compact array of bit values, which are represented as Booleans, where true indicates that the bit is on (1) and false indicates the bit is off (0).
  • CaseInsensitiveComparer. Compares two objects for equivalence, ignoring the case of strings.
  • CaseInsensitiveHashCodeProvider. Supplies a hash code for an object, using a hashing algorithm that ignores the case of strings.
  • CollectionBase. Provides the abstract (MustInherit in Visual Basic) base class for a strongly typed collection.
  • Comparer. Compares two objects for equivalence, where string comparisons are case-sensitive.
  • DictionaryBase. Provides the abstract (MustInherit in Visual Basic) base class for a strongly typed collection of associated keys and values.
  • Hashtable. Represents a collection of associated keys and values that are organized based on the hash code of the key.
  • Queue. Represents a first-in, first-out collection of objects.
  • ReadOnlyCollectionBase. Provides the abstract (MustInherit in Visual Basic) base class for a strongly typed read-only collection.
  • SortedList. Represents a collection of associated keys and values that are sorted by the keys and are accessible by key and by index.
  • Stack. Represents a simple last-in-first-out collection of type Object.

We will discuss ArrayList in the final part of this article.

Summary

In this article, we discussed how a shape can be created by inheriting the abstract class Shape. We also learned how to handle an event and discussed the event handlers we use in the application. Last, we had a look at the System.Collections namespace, an important namespace that provides various collection interfaces and classes. In Part 3, the last part of this series, we will discuss the rest of the application and provide an attachment of the whole application.


Developing A C# Vector Drawing Application - Part 1

Developing A C# Vector Drawing Application - Part 1
http://www.csharptoday.com/content.asp?id=1654
Article Rating: 3.5
Published on September 9, 2001
Abstract
Computer drawing applications are one of the most popular applications. Examples of such applications are Corel Draw and FreeHand - Windows users even have one in their system: Microsoft Paint. Chances are you too have used it once in your life. How about developing one? That has probably been done by a few people and the thought of it could scare beginners. This series of three articles by Budi Kurniawan show us that it is not hard at all with C# and the .NET Framework Base Class Library to build a vector based drawing application. Read on.
Article

There are two types of drawing applications: vector based and raster based. Microsoft Paint is an example of the latter. Your artwork is saved as a bitmap file. When you sketch a line, a circle and a rectangle, those shapes blend into one entity, inseparable from each other. A vector-based draw application, on the other hand, treats each shape as a separate entity that can later be manipulated individually. If you draw an ellipse, for example, you can later change its color or its thickness. If you save your drawing, it is saved as a collection of objects and you can retrieve them later and still be able to manipulate each individual object like the original. This type of drawing application is of course more convenient to work with, and also scales better as each object does not become jagged when resized for viewing or printing.

The application that we will write in this article is a vector-based drawing tool. Even though it is not meant to be a commercial application, it has basic features that serve as a good learning tool. Once you know the basics of drawing in a .NET Framework environment, you can extend it to suit your own needs. Not only will you use the .NET Framework's System.Drawing namespace, you will also learn a few other techniques such as event handling, .NET collections, writing your own Windows control, and object serialization. As usual, we should start a software project by defining the specification.

Because there are several different concepts involved in developing this application, the article has to be split into three parts.

  • This first part will discuss the specification and build the skeleton of the drawing application. The application attached will not do much, except displaying the user interface.
  • Part 2 will add event-handling code to the application in addition to the discussion about the Shape class and all its child classes.
  • Part 3 completes the application by presenting the I/O operations in which we can save the drawing to a file and load it later.

The C# Drawing Application Specification

In addition to being a Vector-based draw application, our C# Drawing application has the following features:

  • The user has seven basic shapes to choose from. These shapes are lines, rectangles, filled rectangles, border-less rectangles, ellipses, filled ellipses, and border-less ellipses. A shape can be chosen by clicking the button for that shape.
  • Each object can be drawn with one of the ten default colors in the palette. If the object has a background color, you can too choose a different background color from the palette. Choosing a foreground color is done by clicking a color button in the palette. Selecting a background color can be done by right-clicking a color button in the palette.
  • Each object can be drawn with one of five kinds of thickness. The user can select a thickness by clicking on one of the five thickness buttons.
  • The application has a non-resizable drawing area, but the drawing area is persistent, in the sense that it will refresh itself when the window is resized or when the application is restored after being minimized. A non-persistent drawing area will lose its drawing when another Windows program is brought in front of it, because the drawing area does not refresh itself. This, of course, is not a preferable solution.
  • The user can draw a shape by first selecting a shape, and then click on the drawing area and drag it to the second coordinate.
  • Each shape is a separate object. There is an undo feature that lets the user cancels the last stroke. Using an undo again will cancel the second last object, and so forth.
  • The user can save the drawing to an XML file, using the object serialization library in the .NET Framework. Later, the drawing can be restored by opening the file.
  • The user has two menus: File and Edit. The File menu has the following menu items: New, Open, Save and Exit. The Edit menu has one menu item: Undo.
  • The New menu item clears the drawing area.
  • The Open menu item displays the standard Open File dialog where the user can browse the file system to find a previously saved drawing file.
  • The Save menu item displays the standard Save As dialog where the user can browse the file system to select a directory for the drawing file to be stored in.
  • The Exit menu item, when selected, closes the application.
  • The Undo menu item cancels the last drawing.

In addition to the items above, each button has an icon that is imported at design time. Figure 1 shows the C# Draw application.

Figure 1: The C# Draw application

As shown above, there are two menus (File and Edit) at the top. The seven shape buttons are grouped together and located at the left-hand corner just below the menus. The five buttons that allow the user to select the thickness of a shape is to be found below the Shape buttons. Next is the palette with ten colors. The two bigger buttons shows the selected foreground and background colors. And, at the center is the drawing area.

Armed with the specification, we are now ready to write our first line of code. However, because .NET Framework is relatively new, please allow me to explain some basic concepts that will help us build our first installment of the application. Other concepts will be explained in the second and third articles in the series.

Creating a Windows Application

Creating a Windows application in C# is easy with the help of the .NET Framework Class Library.

Creating a Form

In any .NET application, you create a form by extending the Form class in the System.Windows.Forms namespace. Note that the namespace has changed from the one in .NET Framework Beta 1. In Beta 1, it used to be called System.WinForms.

The main (and only) form in our application is named DrawForm, and here is the code that creates a new class named DrawForm that inherits System.Windows.Forms.Form:

public class DrawForm : System.Windows.Forms.Form
{
}

Because when working with a form we will also be using classes from the System.Windows.Forms namespace, we can import the namespace to save typing the namespace in front of every class we use. Therefore, the code looks like the following.

using System.Windows.Forms;
public class DrawForm : Form
{
}

That's all it takes to create a window. As a child class of System.Windows.Forms.Form, our DrawForm class inherits all the non-private class members of the Form class, including its properties, methods and events. Writing a .NET Windows application is easiest using Visual Studio.NET, which is currently in its Beta 2 version. If you are using Visual Studio.NET, which you should, it will generate code similar to above.

The Form class itself is the 7th in the hierarchy. It inherits ContainerControl that inherits ScrollableControl, and so on. See Figure 2 below.

Figure 2: The System.Windows.Forms.Form class hierarchy

Once we have a class derived from Form, we can change its properties so that our form will look as we wish. In C#, as in C++ and Java, we use the keyword this to refer to the current class. Therefore, to refer the form from inside the form itself, we use the this keyword as well.

The Form class has the Name property for its name and the Text property can be set with a String that want to appear as the form title. We can set this using Visual Studio (see attached code) or by coding it ourselves:

this.Name = "DrawForm";
this.Text = "Draw
            Form";

As to the size, this can be set either in the Properties window at design time, or programmatically by setting the form's Size property. In the .NET Framework Class Library, a size is an object, like everything else. To be more precise, Size is a type in the System.Drawing namespace. If so desired, we can set the form's Size property by assigning a Size object. Using the new keyword is one of the ways of creating an object. The following line of code creates a Size object of width and height and assigns it to the form's Size property. width and height are measured in pixels.

this.penWidth5.Size = new System.Drawing.Size(width, height);

By default, the top left corner of an object is positioned at coordinate (0, 0) of its container. Since the form has no container, it will be positioned at the top-left corner of the monitor screen. If you change this by modifying the value of the form's Location property, either at design time from the Properties window or programmatically. Doing it programmatically is not difficult. Basically, we need to assign a point for the top left corner. Again, a point is an object. In fact, there is a Point type in the System.Drawing namespace. As an example:

this.Location = new System.Drawing.Point(5, 58);

Adding Controls to the Form

A Windows application usually has controls on the form. The .NET Framework Class Library provides controls that are ready to use, such as buttons, checkboxes, textboxes, labels, . you name it. They are all located in the System.Windows.Forms namespace. If you are new to the .NET Framework, you might want to have a look of their object hierarchies. For instance, the Button control hierarchy is given in Figure 3.

Figure 3: The Button class hierarchy

All controls are indirectly derived from the Control class. Adding a control to a form requires three steps.

1. Dimension a variable for the control.
2. Instantiate the control
3. Add the control to the form

Because C# variables can have different scopes, it does matter where to declare a variable. A variable declared inside a function is only visible from the function. For our application, the controls need to be visible from the whole class, therefore we must declare them in the class level.

Having dimensioned a control, the second step is to instantiate it. Instantiating a control is like instantiating any other object, which means we use the new keyword. Once we have an instance of the control, we can add that object to the form. You can't add a control directly to a form because the Form class has no such a method as Add(Control). However, the Form class inherits the Controls property from the Control class.

In the Form class, we use this property to obtain the S ystem.Windows.Forms.Form.ControlCollection object of the form. The Form.ControlCollection class represents the collection of controls contained within a form. We can use its Add method to add a control to the form and its Remove method to remove a control from the form.

Every shape, color and thickness is represented by a button. For example, the first color is a button named color1. Dimensioning, instantiation and adding it to the form is done with these three lines of code.

private Button color1;
Color1 = new
            System.Windows.Forms.Button();
this.Controls.Add(Color1);

In addition, we can set the control's Size and Location properties, exactly as we set the Form object.

Grouping Controls

Some of the buttons are grouped together, for example to make them aligned and share several common properties. If we move the group, all the controls move with it. The seven shape buttons are grouped together, and so are the ten color buttons and five thickness buttons.

The System.Windows.Forms namespace provides a class called GroupBox in which to group our controls. Grouping our controls requires the following steps.

  • 1. Declaring a GroupBox object variable
  • 2. Instantiating the GroupBox object variable
  • 3. Adding controls to the GroupBox
  • 4. Adding the GroupBox to the form.

The GroupBox then becomes the container of all the controls added to it. Once we add the GroupBox to the form, we don't need to add individual controls directly to the form. In our application, there are three GroupBox objects, known by these names: s hapeGroupBox (for the seven shape buttons), c olorGroupBox (for the 10 colors and two bigger selected foreground and background color), and p enWidthGroupBox (for five thickness buttons). As an example, the p enWidthGroupBox is dimensioned, instantiated by the following code.

private System.Windows.Forms.GroupBox
            penWidthGroupBox;
this.penWidthGroupBox = new
            System.Windows.Forms.GroupBox();

Then we can start adding controls to it: All the five thickness controls are named penWidth1, penWidth2, penWidth3, penWidth4 and penWidth5 respectively. Note that all controls must be instantiated prior to adding them to a GroupBox. We can add a control to a GroupBox by dragging it on to the GroupBox at design time. Alternatively, adding controls to a GroupBox is done using the AddRange method of the Controls object of the GroupBox control. For example, this is the code that adds five penWidth buttons to the PenWidthGroupBox control.

this.penWidthGroupBox.Controls.AddRange(new System.Windows.Forms.Control[] {
this.penWidth1, this.penWidth2, this.penWidth3, this.penWidth4, this.penWidth5});

Creating Menus

Our application needs two menus. Windows Forms supports main menus and context menus.

  • Main menus are displayed on a menu bar that is located immediately below the title bar of a form. The menu bar contains top-level menu items that are used to group related submenu items. For example, by clicking a File top-level menu item, you can display the New, Open, Save and Exit menu items. In Windows Forms, main menus are represented by the MainMenu control.
  • Context menus can be displayed for a specific control or area of your form. They are typically accessed by clicking the right mouse button. In Windows Forms, context menus are represented by the ContextMenu control. At design time, you can drag both MainMenu and ContextMenu controls and start typing in the menu name.

In this application, we will use a MainMenu and a number of MenuItem objects. The MainMenu class has the MenuItems collections, which hold all menu items added to it. Once we have the MainMenu and MenuItem object, we can use the AddRange method of the MainMenu 's MenuItems collection to add a number of menu items.

private System.Windows.Forms.MainMenu
            mainMenu1;
private System.Windows.Forms.MenuItem
            fileMenuItem;
private System.Windows.Forms.MenuItem
            editMenuItem;
this.mainMenu1 = new
            System.Windows.Forms.MainMenu();
this.fileMenuItem = new
            System.Windows.Forms.MenuItem();
this.editMenuItem = new
            System.Windows.Forms.MenuItem();
this.mainMenu1.MenuItems.AddRange(new
System.Windows.Forms.MenuItem[] {
this.fileMenuItem,
            this.editMenuItem});

The Code

The application attached consists of two C# files: DrawForm.cs and DrawingArea.cs. DrawForm.cs contains the DrawForm class, which is the main form of the application. This is given in Listing 1 in the appendix.

DrawingArea.cs contains the custom control called DrawingArea, which is a class derived from the System.Windows.Forms.Control class. This is the drawing area for our application. We shall see more of the code in Part 2 and Part 3 of the series.

Summary

There are two types of drawing applications, raster-based and vector-based. The one developed in this application is vector-based. The first part of this series of articles discusses the specification and the main concepts for building a Windows application, including how to create a form, add controls to the form, group controls and create a main menu and menu items. The attachment that you can download offers the partly finished application with all controls and menus discussed in this article. In Part 2 and 3, we will continue developing our application to its full functions.

Appendix

Listing 1: The code for the DrawForm class

using System;
using
            System.Drawing;
using System.Collections;
using
            System.ComponentModel;
using System.Windows.Forms;
using
            System.Data;

namespace DrawApp
{
///
            <summary>
/// Summary description for DrawForm.
///
            </summary>
public class DrawForm :
            System.Windows.Forms.Form
{
private
            System.Windows.Forms.GroupBox shapeGroupBox;
private
            System.Windows.Forms.Button lineButton;
private
            System.Windows.Forms.Button rectangleButton;
private
            System.Windows.Forms.Button filledRectangleButton;
private
            System.Windows.Forms.Button borderlessRectangleButton;
private
            System.Windows.Forms.Button ellipseButton;
private
            System.Windows.Forms.Button filledEllipseButton;
private
            System.Windows.Forms.Button borderlessEllipseButton;
private
            System.Windows.Forms.GroupBox penWidthGroupBox;
private
            System.Windows.Forms.Button penWidth1;
private
            System.Windows.Forms.Button penWidth2;
private
            System.Windows.Forms.Button penWidth3;
private
            System.Windows.Forms.Button penWidth4;
private
            System.Windows.Forms.Button penWidth5;
private
            System.Windows.Forms.GroupBox colorGroupBox;
private
            System.Windows.Forms.Button color1;
private
            System.Windows.Forms.Button color2;
private
            System.Windows.Forms.Button color3;
private
            System.Windows.Forms.Button color4;
private
            System.Windows.Forms.Button color5;
private
            System.Windows.Forms.Button color6;
private
            System.Windows.Forms.Button color7;
private
            System.Windows.Forms.Button color8;
private
            System.Windows.Forms.Button color9;
private
            System.Windows.Forms.Button color10;
private
            System.Windows.Forms.Button displayButton1;
private
            System.Windows.Forms.Button displayButton2;

// declare the
            canvas
DrawingArea canvas;
private
            System.Windows.Forms.MainMenu mainMenu1;
private
            System.Windows.Forms.MenuItem fileMenuItem;
private
            System.Windows.Forms.MenuItem newMenuItem;
private
            System.Windows.Forms.MenuItem openMenuItem;
private
            System.Windows.Forms.MenuItem saveMenuItem;
private
            System.Windows.Forms.MenuItem exitMenuItem;
private
            System.Windows.Forms.MenuItem editMenuItem;
private
            System.Windows.Forms.MenuItem undoMenuItem;

///
            <summary>
/// Required designer variable.
///
            </summary>
private System.ComponentModel.Container
            components = null;

public DrawForm()
{
//
//
            Required for Windows Form Designer
            support
//
InitializeComponent();

//
// TODO: Add
            any constructor code after InitializeComponent
            call
//
}

/// <summary>
/// Clean up any
            resources being used.
/// </summary>
protected override
            void Dispose( bool disposing )
{
if( disposing )
{
if
            (components != null)
{
components.Dispose();
}
}
base.Dispose( disposing
            );
}

#region Windows Form Designer generated code
///
            <summary>
/// Required method for Designer support - do not
            modify
/// the contents of this method with the code
            editor.
/// </summary>
private void
            InitializeComponent()
{
System.Resources.ResourceManager
            resources = new
            System.Resources.ResourceManager(typeof(DrawForm));
this.penWidth5
            = new System.Windows.Forms.Button();
this.penWidth4 = new
            System.Windows.Forms.Button();
this.penWidth3 = new
            System.Windows.Forms.Button();
this.penWidth2 = new
            System.Windows.Forms.Button();
this.penWidth1 = new
            System.Windows.Forms.Button();
this.color10 = new
            System.Windows.Forms.Button();
this.borderlessRectangleButton =
            new System.Windows.Forms.Button();
this.filledEllipseButton = new
            System.Windows.Forms.Button();
this.lineButton = new
            System.Windows.Forms.Button();
this.shapeGroupBox = new
            System.Windows.Forms.GroupBox();
this.rectangleButton = new
            System.Windows.Forms.Button();
this.filledRectangleButton = new
            System.Windows.Forms.Button();
this.ellipseButton = new
            System.Windows.Forms.Button();
this.borderlessEllipseButton = new
            System.Windows.Forms.Button();
this.undoMenuItem = new
            System.Windows.Forms.MenuItem();
this.mainMenu1 = new
            System.Windows.Forms.MainMenu();
this.fileMenuItem = new
            System.Windows.Forms.MenuItem();
this.newMenuItem = new
            System.Windows.Forms.MenuItem();
this.openMenuItem = new
            System.Windows.Forms.MenuItem();
this.saveMenuItem = new
            System.Windows.Forms.MenuItem();
this.exitMenuItem = new
            System.Windows.Forms.MenuItem();
this.editMenuItem = new
            System.Windows.Forms.MenuItem();
this.penWidthGroupBox = new
            System.Windows.Forms.GroupBox();
this.color7 = new
            System.Windows.Forms.Button();
this.displayButton1 = new
            System.Windows.Forms.Button();
this.displayButton2 = new
            System.Windows.Forms.Button();
this.color6 = new
            System.Windows.Forms.Button();
this.color5 = new
            System.Windows.Forms.Button();
this.color4 = new
            System.Windows.Forms.Button();
this.color3 = new
            System.Windows.Forms.Button();
this.color2 = new
            System.Windows.Forms.Button();
this.color1 = new
            System.Windows.Forms.Button();
this.colorGroupBox = new
            System.Windows.Forms.GroupBox();
this.color8 = new
            System.Windows.Forms.Button();
this.color9 = new
            System.Windows.Forms.Button();
this.canvas = new
            DrawingArea();
this.shapeGroupBox.SuspendLayout();
this.penWidthGroupBox.SuspendLayout();
this.colorGroupBox.SuspendLayout();
this.SuspendLayout();
//
// penWidth5
//
this.penWidth5.Image =
            ((System.Drawing.Bitmap)(resources.GetObject("penWidth5.Image")));
this.penWidth5.Location
            = new System.Drawing.Point(5, 58);
this.penWidth5.Name =
            "penWidth5";
this.penWidth5.Size = new System.Drawing.Size(32,
            12);
this.penWidth5.TabIndex = 3;

//
//
            penWidth4
//
this.penWidth4.Image =
            ((System.Drawing.Bitmap)(resources.GetObject("penWidth4.Image")));
this.penWidth4.Location
            = new System.Drawing.Point(5, 46);
this.penWidth4.Name =
            "penWidth4";
this.penWidth4.Size = new System.Drawing.Size(32,
            12);
this.penWidth4.TabIndex = 3;
//
// penWidth3
//
this.penWidth3.Image =
            ((System.Drawing.Bitmap)(resources.GetObject("penWidth3.Image")));
this.penWidth3.Location
            = new System.Drawing.Point(5, 34);
this.penWidth3.Name =
            "penWidth3";
this.penWidth3.Size = new System.Drawing.Size(32,
            12);
this.penWidth3.TabIndex = 3;
//
// penWidth2
//
this.penWidth2.Image =
            ((System.Drawing.Bitmap)(resources.GetObject("penWidth2.Image")));
this.penWidth2.Location
            = new System.Drawing.Point(5, 22);
this.penWidth2.Name =
            "penWidth2";
this.penWidth2.Size = new System.Drawing.Size(32,
            12);
this.penWidth2.TabIndex = 3;
//
// penWidth1
//
this.penWidth1.Image =
            ((System.Drawing.Bitmap)(resources.GetObject("penWidth1.Image")));
this.penWidth1.Location
            = new System.Drawing.Point(5, 10);
this.penWidth1.Name =
            "penWidth1";
this.penWidth1.Size = new System.Drawing.Size(32,
            12);
this.penWidth1.TabIndex = 3;
//
// color10
//
this.color10.BackColor =
            System.Drawing.Color.FromArgb(((System.Byte)(168)),
            ((System.Byte)(128)), ((System.Byte)(0)));
this.color10.FlatStyle
            = System.Windows.Forms.FlatStyle.Popup;
this.color10.Location =
            new System.Drawing.Point(136, 26);
this.color10.Name =
            "color10";
this.color10.Size = new System.Drawing.Size(16,
            16);
this.color10.TabIndex = 4;
//
//
            borderlessRectangleButton
//
this.borderlessRectangleButton.Image =
            ((System.Drawing.Bitmap)(resources.GetObject("borderlessRectangleButton.Image")));
this.borderlessRectangleButton.Location
            = new System.Drawing.Point(11,
            70);
this.borderlessRectangleButton.Name =
            "borderlessRectangleButton";
this.borderlessRectangleButton.Size
            = new System.Drawing.Size(20,
            20);
this.borderlessRectangleButton.TabIndex = 0;
//
//
            filledEllipseButton
//
this.filledEllipseButton.Image =
            ((System.Drawing.Bitmap)(resources.GetObject("filledEllipseButton.Image")));
this.filledEllipseButton.Location
            = new System.Drawing.Point(11,
            110);
this.filledEllipseButton.Name =
            "filledEllipseButton";
this.filledEllipseButton.Size = new
            System.Drawing.Size(20, 20);
this.filledEllipseButton.TabIndex =
            0;
//
// lineButton
//
this.lineButton.Image =
            ((System.Drawing.Bitmap)(resources.GetObject("lineButton.Image")));
this.lineButton.Location
            = new System.Drawing.Point(11, 10);
this.lineButton.Name =
            "lineButton";
this.lineButton.Size = new System.Drawing.Size(20,
            20);
this.lineButton.TabIndex = 0;
//
//
            shapeGroupBox
//
this.shapeGroupBox.Controls.AddRange(new
            System.Windows.Forms.Control[]
            {
this.lineButton,
this.rectangleButton,
this.filledRectangleButton,
this.borderlessRectangleButton,
this.ellipseButton,
this.filledEllipseButton,
this.borderlessEllipseButton});
this.shapeGroupBox.Name
            = "shapeGroupBox";
this.shapeGroupBox.Size = new
            System.Drawing.Size(42, 155);
this.shapeGroupBox.TabIndex =
            1;
this.shapeGroupBox.TabStop =
            false;
this.shapeGroupBox.Enter += new
            System.EventHandler(this.shapeGroupBox_Enter);
//
//
            rectangleButton
//
this.rectangleButton.Image =
            ((System.Drawing.Bitmap)(resources.GetObject("rectangleButton.Image")));
this.rectangleButton.Location
            = new System.Drawing.Point(11, 30);
this.rectangleButton.Name =
            "rectangleButton";
this.rectangleButton.Size = new
            System.Drawing.Size(20, 20);
this.rectangleButton.TabIndex =
            0;
//
// filledRectangleButton
//
this.filledRectangleButton.Image =
            ((System.Drawing.Bitmap)(resources.GetObject("filledRectangleButton.Image")));
this.filledRectangleButton.Location
            = new System.Drawing.Point(11,
            50);
this.filledRectangleButton.Name =
            "filledRectangleButton";
this.filledRectangleButton.Size = new
            System.Drawing.Size(20, 20);
this.filledRectangleButton.TabIndex
            = 0;
//
// ellipseButton
//
this.ellipseButton.Image =
            ((System.Drawing.Bitmap)(resources.GetObject("ellipseButton.Image")));
this.ellipseButton.Location
            = new System.Drawing.Point(11, 90);
this.ellipseButton.Name =
            "ellipseButton";
this.ellipseButton.Size = new
            System.Drawing.Size(20, 20);
this.ellipseButton.TabIndex =
            0;
//
// borderlessEllipseButton
//
this.borderlessEllipseButton.Image =
            ((System.Drawing.Bitmap)(resources.GetObject("borderlessEllipseButton.Image")));
this.borderlessEllipseButton.Location
            = new System.Drawing.Point(11,
            130);
this.borderlessEllipseButton.Name =
            "borderlessEllipseButton";
this.borderlessEllipseButton.Size =
            new System.Drawing.Size(20,
            20);
this.borderlessEllipseButton.TabIndex = 0;
//
//
            undoMenuItem
//
this.undoMenuItem.Index =
            0;
this.undoMenuItem.Text = "Undo";
//
// mainMenu1
//
this.mainMenu1.MenuItems.AddRange(new
            System.Windows.Forms.MenuItem[]
            {
this.fileMenuItem,
this.editMenuItem});
//
//
            fileMenuItem
//
this.fileMenuItem.Index =
            0;
this.fileMenuItem.MenuItems.AddRange(new
            System.Windows.Forms.MenuItem[]
            {
this.newMenuItem,
this.openMenuItem,
this.saveMenuItem,
this.exitMenuItem});
this.fileMenuItem.Text
            = "File";
//
// newMenuItem
//
this.newMenuItem.Index
            = 0;
this.newMenuItem.Text = "New";
//
//
            openMenuItem
//
this.openMenuItem.Index =
            1;
this.openMenuItem.Text = "Open";
//
//
            saveMenuItem
//
this.saveMenuItem.Index =
            2;
this.saveMenuItem.Text = "Save";
//
//
            exitMenuItem
//
this.exitMenuItem.Index =
            3;
this.exitMenuItem.Text = "Exit";
//
//
            editMenuItem
//
this.editMenuItem.Index =
            1;
this.editMenuItem.MenuItems.AddRange(new
            System.Windows.Forms.MenuItem[]
            {
this.undoMenuItem});
this.editMenuItem.Text = "Edit";
//
// penWidthGroupBox
//
this.penWidthGroupBox.Controls.AddRange(new
            System.Windows.Forms.Control[]
            {
this.penWidth1,
this.penWidth2,
this.penWidth3,
this.penWidth4,
this.penWidth5});
this.penWidthGroupBox.Location
            = new System.Drawing.Point(0, 160);
this.penWidthGroupBox.Name =
            "penWidthGroupBox";
this.penWidthGroupBox.Size = new
            System.Drawing.Size(42, 74);
this.penWidthGroupBox.TabIndex =
            2;
this.penWidthGroupBox.TabStop = false;
//
//
            color7
//
this.color7.BackColor =
            System.Drawing.Color.FromArgb(((System.Byte)(255)),
            ((System.Byte)(128)), ((System.Byte)(0)));
this.color7.FlatStyle
            = System.Windows.Forms.FlatStyle.Popup;
this.color7.Location =
            new System.Drawing.Point(82, 26);
this.color7.Name =
            "color7";
this.color7.Size = new System.Drawing.Size(16,
            16);
this.color7.TabIndex = 4;
//
// displayButton1
//
this.displayButton1.FlatStyle =
            System.Windows.Forms.FlatStyle.Popup;
this.displayButton1.Location
            = new System.Drawing.Point(20, 11);
this.displayButton1.Name =
            "displayButton1";
this.displayButton1.Size = new
            System.Drawing.Size(32, 32);
this.displayButton1.TabIndex =
            4;
//
// displayButton2
//
this.displayButton2.FlatStyle =
            System.Windows.Forms.FlatStyle.Popup;
this.displayButton2.Location
            = new System.Drawing.Point(8, 8);
this.displayButton2.Name =
            "displayButton2";
this.displayButton2.Size = new
            System.Drawing.Size(32, 32);
this.displayButton2.TabIndex =
            4;
//
// color6
//
this.color6.BackColor =
            System.Drawing.Color.Gray;
this.color6.FlatStyle =
            System.Windows.Forms.FlatStyle.Popup;
this.color6.Location = new
            System.Drawing.Point(64, 26);
this.color6.Name =
            "color6";
this.color6.Size = new System.Drawing.Size(16,
            16);
this.color6.TabIndex = 4;
//
// color5
//
this.color5.BackColor =
            System.Drawing.Color.Blue;
this.color5.FlatStyle =
            System.Windows.Forms.FlatStyle.Popup;
this.color5.Location = new
            System.Drawing.Point(136, 8);
this.color5.Name =
            "color5";
this.color5.Size = new System.Drawing.Size(16,
            16);
this.color5.TabIndex = 4;
//
// color4
//
this.color4.BackColor =
            System.Drawing.Color.Lime;
this.color4.FlatStyle =
            System.Windows.Forms.FlatStyle.Popup;
this.color4.Location = new
            System.Drawing.Point(118, 8);
this.color4.Name =
            "color4";
this.color4.Size = new System.Drawing.Size(16,
            16);
this.color4.TabIndex = 4;
//
// color3
//
this.color3.BackColor =
            System.Drawing.Color.Red;
this.color3.FlatStyle =
            System.Windows.Forms.FlatStyle.Popup;
this.color3.Location = new
            System.Drawing.Point(100, 8);
this.color3.Name =
            "color3";
this.color3.Size = new System.Drawing.Size(16,
            16);
this.color3.TabIndex = 4;
//
// color2
//
this.color2.BackColor =
            System.Drawing.Color.Black;
this.color2.FlatStyle =
            System.Windows.Forms.FlatStyle.Popup;
this.color2.Location = new
            System.Drawing.Point(82, 8);
this.color2.Name =
            "color2";
this.color2.Size = new System.Drawing.Size(16,
            16);
this.color2.TabIndex = 4;
//
// color1
//
this.color1.BackColor =
            System.Drawing.Color.White;
this.color1.FlatStyle =
            System.Windows.Forms.FlatStyle.Popup;
this.color1.ForeColor =
            System.Drawing.Color.Black;
this.color1.Location = new
            System.Drawing.Point(64, 8);
this.color1.Name =
            "color1";
this.color1.Size = new System.Drawing.Size(16,
            16);
this.color1.TabIndex = 4;
//
// colorGroupBox
//
this.colorGroupBox.Controls.AddRange(new
            System.Windows.Forms.Control[]
            {
this.displayButton2,
this.displayButton1,
this.color1,
this.color2,
this.color3,
this.color4,
this.color5,
this.color6,
this.color7,
this.color8,
this.color9,
this.color10});
this.colorGroupBox.Location
            = new System.Drawing.Point(0, 300);
this.colorGroupBox.Name =
            "colorGroupBox";
this.colorGroupBox.Size = new
            System.Drawing.Size(160, 46);
this.colorGroupBox.TabIndex =
            3;
this.colorGroupBox.TabStop = false;
//
// color8
//
this.color8.BackColor =
            System.Drawing.Color.Purple;
this.color8.FlatStyle =
            System.Windows.Forms.FlatStyle.Popup;
this.color8.Location = new
            System.Drawing.Point(100, 26);
this.color8.Name =
            "color8";
this.color8.Size = new System.Drawing.Size(16,
            16);
this.color8.TabIndex = 4;
//
// color9
//
this.color9.BackColor =
            System.Drawing.Color.FromArgb(((System.Byte)(64)),
            ((System.Byte)(164)),
            ((System.Byte)(112)));
this.color9.FlatStyle =
            System.Windows.Forms.FlatStyle.Popup;
this.color9.Location = new
            System.Drawing.Point(118, 26);
this.color9.Name =
            "color9";
this.color9.Size = new System.Drawing.Size(16,
            16);
this.color9.TabIndex = 4;
//
// canvas
//
this.canvas.BackColor = Color.White;
this.canvas.Location =
            new System.Drawing.Point(50, 8);
this.canvas.Name =
            "canvas";
this.canvas.Size = new System.Drawing.Size(540,
            290);
this.canvas.TabIndex = 5;
//
// DrawForm
//
this.AutoScaleBaseSize = new System.Drawing.Size(5,
            13);
this.ClientSize = new System.Drawing.Size(592,
            373);
this.Controls.AddRange(new System.Windows.Forms.Control[]
            {
this.canvas,
this.colorGroupBox,
this.penWidthGroupBox,
this.shapeGroupBox,

});
this.Menu
            = this.mainMenu1;
this.Name = "DrawForm";
this.Text = "Draw
            Form";
this.Load += new
            System.EventHandler(this.DrawForm_Load);
this.shapeGroupBox.ResumeLayout(false);
this.penWidthGroupBox.ResumeLayout(false);
this.colorGroupBox.ResumeLayout(false);
this.ResumeLayout(false);

}
#endregion

///
            <summary>
/// The main entry point for the
            application.
/// </summary>
[STAThread]
static void
            Main()
{
Application.Run(new DrawForm());
}

private
            void shapeGroupBox_Enter(object sender, System.EventArgs
            e)
{

}

private void DrawForm_Load(object sender,
            System.EventArgs e)
{
displayButton1.BackColor =
            color1.BackColor;
displayButton2.BackColor = color2.BackColor
            ;
}
}
}

Listing 2: The code for the DrawingArea class

using System;
using
            System.Drawing;
using System.Windows.Forms;
namespace
            DrawApp
{
/// <summary>
/// Summary description for
            Class1.
/// </summary>
public class DrawingArea :
            Control
{
public DrawingArea()
{

//
// TODO: Add
            constructor logic here
//
}
protected void
            onPaint(PaintEventArgs e)
{
}

}
}


Irregular Shaped Forms and Other Cool GDI+ Features

Irregular Shaped Forms and Other Cool GDI+ Features
http://www.csharptoday.com/content.asp?id=1555
Article Rating: 5
Published on August 27, 2001
Abstract
One of the coolest, yet rarely used features in the Windows UI is “shaped forms”. These allow the developer to build an application whose windows are not rectangular, but rather cool and engaging shapes. Previously, these techniques are what separated large game and graphics companies from the rest of the field. Now, with .NET and GDI+, they are “common and easy” tools for every programmer to use. In this article, Matthew Reynolds shows us how we can build controls and forms that have any shape we like.
Article

One of the coolest, yet rarely used features in the Windows UI is "shaped forms". These allow the developer to build an application whose windows are not rectangular, but rather cool and engaging shapes. Here's a screenshot of Sonique, a popular MP3 player.

Using .NET, it's very easy to create forms that have any shape that you like, whether they're regular shapes like rectangles or ellipses or curvaceous, sculptured forms like Sonique. In this article we're going to take a look at how we can build controls and forms that have any shape we like.

The magic we're going to see in this article is done for us by GDI+. GDI, or "Graphics Device Interface" defines a set of guidelines, standards, rules and APIs that Windows uses to talk to the graphics card (or any other graphics device) in order to draw images.

GDI+ is the next generation version of the original GDI and offers a host of new features some of which we're going to examine here.

Shaped Controls

First off, we'll take a look at building a basic control that is circular rather than rectangular. Create a new Visual C# - Windows Application project and call it Shaped Controls.

Using Solution Explorer, right-click on the Shaped Controls project and select Add - Add User Control. Call the new control Scope.

Obviously, by default all controls and forms are created as a rectangular shape. What we need to do is define a new shape and use the control to use that shape rather than the default "rectangle shape" that it gets when it's first created. This is done using a Region, and we can create and manage regions using the System.Drawing.Region object.

Using the Properties window set the BackColor property of the control to Black. Also, set the ForeColor property to some other color. (I'm going to use Green.) You'll see the following, which won't come as much of a surprise.

Now, double-click on the control to open the code editor and add namespace import declaration for System.Drawing.Drawing2D to the top of the code:

using System;
using System.Collections;
using System.ComponentModel;
using System.Drawing;
using System.Data;
using System.Windows.Forms;
using System.Drawing.Drawing2D;

Next, add this method to the code editor.

private void Scope_Resize(object sender, System.EventArgs e)
{

You can define a region in a number of ways, but perhaps the easiest is to use the System.Drawing.Drawing2D.GraphicsPath object. This object lets you build a "map" of how you want the shape to look. What we're trying to do is "color in" a mask of what parts of the original bounding rectangle we actually want to be part of the new shape. In this instance we're adding an ellipse that's the same size as the bounding rectangle of the control.

         //
            create a new graphics path...
GraphicsPath path = new
            GraphicsPath();
path.AddEllipse(0,0, Width, Height);

Once we have the graphics path, we can create a new System.Drawing.Region object. After we have one of those, we can set the Region property on the control to apply the new region.

         //
            create a new region and apply it...
Region region = new
            Region(path);
this.Region = region;
}

We need to tie the Scope_Resize method into the Resize event of the control, so find the InitializeComponent method and add a line to link the Resize event to Scope_Resize.

      private
            void InitializeComponent()
{
//
// Scope
//
this.BackColor = System.Drawing.Color.Black;
this.Name =
            "Scope";
this.Size = new System.Drawing.Size(116,
            108);
this.Load += new
            System.EventHandler(this.Scope_Load);
this.Resize += new
            System.EventHandler(this.Scope_Resize);
}

Before we can use the control we have to build the project, so select Build - Build from the menu.

Open the designer for Form1 and add one Button control and the new Scope control so that they overlap, like this:

One thing to try here is that if you click just outside of the Scope control, but within the bounding rectangle of the control onto the button, the button control becomes selected. Applying a region is not something that just affects the view - it affects the way the entire control works. A circular control will have a circular boundary and mouse actions outside of that boundary filter through to the control underneath in the usual way. Those of you with experience in building shaped controls and forms prior to GDI+ will notice that this is an improvement over the old technique. With GDI, controls always had a rectangle boundary irrespective of their shape.

Making the Scope Do Something

Let's now look at how to make the control do something. Open the Designer for the Scope control and add a new Timer control. Set the Name property of the Timer control to timerScan. Set the Enabled property to True.

Open the code editor for Scope and add these members somewhere near the top of the class declaration:

public class Scope : System.Windows.Forms.UserControl
{
// members...
private static Random _randomNumbers = new Random();
private ArrayList _points = new ArrayList();

Flip back to the Designer and double-click on the Timer control. This will create a new Tick event handler. Add this code:

      private
            void timerScan_Tick(object sender, System.EventArgs e)
{
//
            clear the existing points...
_points.Clear();

// create a
            new set of points...
for(int x = 0; x < Width; x +=
            3)
{
// get a random adjustment...
int adjust =
            _randomNumbers.Next(Height / 4);

// move y up or
            down...
int y = Height / 2;
if(x % 2 == 0)
y -=
            adjust;
else
y += adjust;

// create a new
            point...
_points.Add(new PointF(x, y));
}

// redraw the
            window...
Invalidate();
}

On each tick of the timer, this code will start off at the left hand size of the control and move to the right hand side of the control in steps of three pixels per iteration. Each time it does this, it will get a random adjustment number ("adjust ") that will be up to one-forth of the height of the control. It will then alternate between subtracting this from the midpoint of the control and adding it to the midpoint of the control. This will give something that looks like an oscilloscope trace for some random event.

Once we have the points, we need to draw them, so add this method:

      private
            void Scope_Paint(object sender, PaintEventArgs p)
{
// get the
            graphics object...
Graphics graphics = p.Graphics;

//
            create a pen...
Pen pen = new Pen(this.ForeColor);

// go
            through the points...
float lastX = 0, lastY = 0;
foreach
            (PointF point in _points)
{
// don't worry about the first
            one...
if(lastX >= 0)
{
// draw
            something...
graphics.DrawLine(pen, lastX, lastY, point.X,
            point.Y);
}

// store the current points...
lastX =
            point.X;
lastY = point.Y;
}
}

This method will go through each of the points stored in the _points ArrayList object and join them together. Because we've called Invalidate at the end of the Tick event handler, the control will be repainted on each tick of the timer. (For those of you with some GDI experience, this will make the control flicker and flash like crazy leading to a very ugly control. But, fear not! We'll have this sorted out by the end of the article).

Of course, before Scope_Paint can be called we need to add an event handler. Find the InitializeComponent method and add a handler that will call Scope_Paint. (I've omitted some code here for brevity).

   this.Resize += new
            System.EventHandler(this.Scope_Resize);
this.Load += new
            System.EventHandler(this.Scope_Load);
this.Paint += new
            System.Windows.Forms.PaintEventHandler(this.Scope_Paint);

Build the project and open the Designer for Form1. As the control has been enabled in design mode, the control will actually run when placed onto the designer. (This isn't great practice, but it makes our lives a little easier!)

Anti-Aliasing

OK, so we have a circular control and we also have a spiky line that draws itself ten times a second. But, let's take a look at that line - it's not very smooth. In fact, we can clearly see the pixels that have been used to make up the lines.

Compare these two images:

Look how much smoother the one of the right is. That's because it uses a technique called "anti-aliasing". This is a very common graphics technique for smoothing out lines, shapes and text. The old GDI code didn't support this functionality directly making it very hard to achieve in Windows applications. But, as we're about to see, GDI+ does support anti-aliasing.

In the first example, each green pixel is next to a black pixel. Our eyes are very good at seeing where one color ends and another one begins, which is why the edges look so sharp.

In the second example, each green pixel is "faded" to a black pixel by stepping down through the original bright green color, through green pixels of various shades until we eventually get to black. With the edges "smudged" in this way, our eyes cannot see exactly where the shape ends so it looks smoother.

This image shows a zoomed in portion of the image painted using anti-aliasing - you'll notice that various shades of green and gray have been used to smooth the edged.

This image shows a similar zoomed in portion of the original without anti-aliasing:

Thanks to GDI+, pulling off this anti-aliasing functionality is very easy indeed. Open the code editor for Scope and add these lines at the top of the Scope_Paint method:

      private
            void Scope_Paint(object sender, PaintEventArgs p)
{
// get the
            graphics object...
Graphics graphics = p.Graphics;

//
            smooth...
graphics.SmoothingMode =
            SmoothingMode.AntiAlias;

// create a pen...
Pen pen = new
            Pen(this.ForeColor);

Build the project again and open the Designer for Form1. You should find that the control is now being drawn in an anti-aliased fashion.

One thing to bear in mind with anti-aliasing is that it does take a little more graphics processing power to draw. In some cases you may find that application performance suffers on some computers that you install your application on. In real world applications, you might find it useful to provide the user with a way to switch anti-aliasing off.

Drawing Text

If you've already experimented with drawing text in .NET, you'll know that it's very easy to do. In this section, we'll take a look at some cool things you can do with text.

If you open the Designer for Form1 and select the Scope control, you'll notice that the Text property does not appear. To make it appear, we physically have to create a new property in Scope that overrides the existing Text property. We also have to set a bunch of attributes on this new property so that we can change its value using the Properties window.

Open the code editor for Scope and add this code:

[DesignerSerializationVisibility(DesignerSerializationVisibility.Visible),
Browsable(true)]
public override String Text
{
get
{
return base.Text;
}
set
{
base.Text = value;
}
}

The first of these attributes, DesignerSerializationVisibility turns on the functionality that means that changes to the property are saved into the InitializeComponent method of the container, in this case Form1. The Browsable attribute makes the Text property appear in the properties window. The base UserControl class already contains a Text property, so all we have to do is pass the request onto the base class to handle.

Drawing the text is just a matter of calling Graphics.DrawString. In this case, we give it a rectangle that represents the binding rectangle of the entire control and also provide a StringFormat object that's configured to tell DrawString to paint the text horizontally and vertically aligned around the center of the rectangle.

      private
            void Scope_Paint(object sender, PaintEventArgs p)
{
// get the
            graphics object...
Graphics graphics = p.Graphics;

//
            smooth...
graphics.SmoothingMode =
            SmoothingMode.AntiAlias;

// create a pen...
Pen pen = new
            Pen(this.ForeColor);

// go through the points...
float
            lastX = 0, lastY = 0;
foreach(PointF point in _points)
{
//
            don't worry about the first one...
if(lastX >= 0)
{
//
            draw something...
graphics.DrawLine(pen, lastX, lastY, point.X,
            point.Y);
}

// store the current points...
lastX =
            point.X;
lastY = point.Y;
}

// draw some
            text...
Rectangle rect = new Rectangle(0,0, Width,
            Height);
StringFormat format = new
            StringFormat();
format.Alignment =
            StringAlignment.Center;
format.LineAlignment =
            StringAlignment.Center;
graphics.DrawString (this.Text,
            this.Font, Brushes.White, rect, format);
}

To try this, build the project and open the designer for Form1. Make the form and the control bigger, set the Text property of the Scope control to Yadda, yadda, yadda and also adjust the Font property to make the text bigger too. You should see this:

Double-Buffering

The keen eyed of those amongst you will notice that when you do this, the control will flicker like crazy. That's pretty ugly, so we want to stop that from happening! To stop it, we use a technique called "double buffering".

The reason why you can see the control flickering is that the drawing process takes quite a while to do and you can see it happening. Firstly, the background of the control is erased, which means that once where your brain saw a bunch or color, it's now just seeing black. Then, the lines are being drawn in and while this does happen in a flash, to an extent you can see it. Finally, the text is painted on and again, your brain can tell that the text was there, then it wasn't, then it was, and so on, hence the flickering.

We can easily solve this problem using a technique called "double buffering". This is a technique where the entire image is built in memory and then transferred to the graphics card very quickly. This completely stops the flickering.

GDI+ supports automatic double buffering, but in the VS.NET Beta 2 build I'm using (9254), this didn't appear to work properly. But, what I can do is show you how double buffering can be done. The GDI+ internal code will use a technique very similar to this, but it features optimizations, which means that the built-in functionality is faster - at least it will be once it works consistently!

The trick with double buffering is to create a bitmap in memory and paint on that rather than painting directly to the Graphics object that we're given through PaintEventArgs. In .NET, this is ridiculously easy:

      private
            void Scope_Paint(object sender, PaintEventArgs p)
{
// do some
            painting...
Bitmap bmp = new Bitmap(Width, Height);
Graphics
            graphics = Graphics.FromImage(bmp);

The second line there is very important. By using the static FromImage method of Graphics, we can create a Graphics object that will paint onto to the bitmap rather than directly onto the screen. Once we have that, we can draw as normal - i.e. we don't have to change any of our existing paint code.

We also want to deal with painting the background ourselves, so add this code next:

         //
            paint the background...
Rectangle rect = new Rectangle(0,0,
            Width, Height);
SolidBrush backgroundBrush = new
            SolidBrush(BackColor);
graphics.FillRectangle(backgroundBrush,
            rect);

Leave the rest of the code as is.

After we've finished all of our painting, we have to draw the bitmap on the original Graphics object provided through PaintEventArgs. Again, this isn't particularly difficult:

         //
            blt the bitmap...
p.Graphics.DrawImageUnscaled(bmp,
            0,0);
graphics.Dispose();
bmp.Dispose();
}

This tiny addition of code deals with most of the problem. The other thing we have to solve is we need to tell the control not to bother painting the background as we're now going to do this from within the Paint event. Add this method:

protected override void OnPaintBackground(PaintEventArgs p)
{
}

Now build the project and you should see that the control no longer flickers.

On a Curve

Back when we originally changed the shape of the control to a circle, we used a GraphicsPath object. We mentioned that we are able to use graphics paths to paint a map of how we want the control to look.

GraphicsPaths have a great deal more power than this. In fact, graphics paths can be used to define very complex shapes indeed and, what's more, matrix transformations can be applied to paths to change them. In this section, we'll see how we can use graphics paths to rotate the text on the control around the center point.

To perform the rotation, we first need to know the center point. This is just a matter of halving both the width and the height of the control, like this:

      protected
            override void OnPaint(PaintEventArgs p)
{
// do some
            painting...
Bitmap bmp = new Bitmap(Width, Height);
Graphics
            graphics = Graphics.FromImage(bmp);
graphics.SmoothingMode =
            SmoothingMode.AntiAlias;

// clear the
            background...
Rectangle rect = new Rectangle(0,0, Width,
            Height);
graphics.FillRectangle(new SolidBrush(BackColor),
            rect);

// create a pen...
Pen pen = new
            Pen(this.ForeColor);

// go through the points...
float
            lastX = 0, lastY = 0;
foreach(PointF point in _points)
{
//
            don't worry about the first one...
if(lastX >= 0)
{
//
            draw something...
graphics.DrawLine(pen, lastX, lastY, point.X,
            point.Y);
}

// store the current points...
lastX =
            point.X;
lastY = point.Y;
}

// what's the center
            point?
Point centerPoint = new Point(Size.Width / 2, Size.Height
            / 2);

Once we've done that, we can create a new GraphicsPath object and use the AddString method to draw a string onto the path, just like we used the DrawString method on the Graphics object before. In fact, we'll use the same StringFormat object to horizontally and vertically center the text around the center point.

// create a new graphics path and add the string...
GraphicsPath path = new GraphicsPath();

StringFormat format = new StringFormat();
format.Alignment = StringAlignment.Center;
format.LineAlignment = StringAlignment.Center;
path.AddString(Text, Font.FontFamily, 0, Font.Size, rect, format);

You'll notice that on AddString we haven't passed it the actual Font property of our control. That's because there isn't a version of AddString that takes a Font object. However, to make sure that things work smoothly, we've use the FontFamily (despite it's curious name "FontFamily " refers to the name of the font) and Size properties on the font that we do have to make sure that things match.

Once we've done that, build the project and you'll see this on the Designer form Form1:

OK, so that's pretty similar to what we've already got. Now let's try and apply a matrix transformation to the text.

Open the code editor for Scope again and add this code. (I've omitted some code for brevity, but remember the entire code package for this application can be downloaded using the link at the end of the article).

         //
            create a new graphics path and add the string...
GraphicsPath
            path = new GraphicsPath();

StringFormat format = new
            StringFormat();
format.Alignment =
            StringAlignment.Center;
format.LineAlignment =
            StringAlignment.Center;
path.AddString(Text, Font.FontFamily, 0,
            Font.Size, rect, format);

// transform the path...
Matrix
            rotateMatrix = new Matrix();
rotateMatrix.RotateAt(45,
            centerPoint);
path.Transform(rotateMatrix);

// draw the
            path...
graphics.FillPath(Brushes.White, path);

Build the project and you'll see this:

Matrixes are a common tool used in transforming images. Here we've used it to rotate the graphics path, but they can also be used to scale and shear images. Using matrixes is a slightly complicated issue, so for more information, check out the MSDN documentation (see links below). However, you can see here how we've used the RotateAt method to rotate the path by 45 degrees.

For a bit more fun, open the code editor for make this change:

// what angle do we want?
int angle = (int)((float)DateTime.Now.Millisecond * .36);

// transform the path...
Matrix rotateMatrix = new Matrix();
rotateMatrix.RotateAt(angle, centerPoint);
path.Transform(rotateMatrix);

Now if you build the project you'll notice that the text spins. That's because we've changed the angle used in the transformation to be a product of the current system time by taking the millisecond value and multiplying it by .36 to get a value between 0 and 360.

Gradient Fill

Another feature of GDI+ is the ability to create a gradient fill. This is where rather than using a solid color, we can start off at one color and slowly move through the color spectrum until we get to a different color - as those of you using Windows 2000 can see in the caption bars of your windows. In this section, we'll use a gradient brush to fill in the background of our control.

First of all, we need another property called BackColor2. Open the code editor for Scope and add a private member called _backColor2:

public class Scope : System.Windows.Forms.UserControl
{
// members...
private static Random _randomNumbers = new Random();
private ArrayList _points = new ArrayList();
private Color _backColor2;

Then, add this property:

   [DesignerSerializationVisibility(DesignerSerializationVisibility.Visible),
Browsable(true), Category("Appearance")]
public Color
            BackColor2
{
get
{
return
            _backColor2;
}
set
{
_backColor2 =
            value;
}
}

Open the Designer for Form1 and change the BackColor and BackColor2 to more exciting colors:

Now, open the code editor for Scope. What we need to do is change our existing code that paints the background color and replace it with some new code that creates a LinearGradientBrush that uses both colors.

protected override void OnPaint(PaintEventArgs p)
{
// do some painting...
Bitmap bmp = new Bitmap(Width, Height);
Graphics graphics = Graphics.FromImage(bmp);
graphics.SmoothingMode = SmoothingMode.AntiAlias;

// clear the background...
Rectangle rect = new Rectangle(0,0, Width, Height);
LinearGradientBrush backgroundBrush = new
LinearGradientBrush(rect, BackColor, BackColor2,
LinearGradientMode.Vertical);
graphics.FillRectangle(backgroundBrush, rect);

Now build the project and you should see this:

Alpha Blending

Another feature of GDI+, although this is only available to Windows 2000 and Windows XP users, is alpha channel transparencies. This enables the developer to create a brush or pen that's not completely solid, i.e. it allows some of the color from beneath to bleed through. This can create some very cool effects.

Open the code editor for Scope and add this code before the call to DrawBitmap:

// draw the path...
graphics.FillPath(Brushes.White, path);

// draw a circle...
SolidBrush circleBrush = new SolidBrush(ForeColor);
graphics.FillEllipse(circleBrush, (Width / 2) - 10, (int)((Height * .85) - 10), 20, 20);
graphics.FillEllipse(circleBrush, (Width / 2) - 10, (int)((Height * .15) - 10), 20, 20);

Build the project and you should see this:

You'll notice that both dots appear in green and that (at this angle) the top dot is obscuring the text beneath.

The brush we've created has used the color defined in ForeColor as its base. By default, this color has its alpha transparency set to 255, meaning 100% opaque. If we set its alpha transparency to 127, it will only be 50% opaque. Open the code editor for Scope and change the line that creates the SolidBrush:

// draw a circle...
SolidBrush circleBrush = new SolidBrush(Color.FromArgb(127, ForeColor.R, ForeColor.G, ForeColor.B));
graphics.FillEllipse(circleBrush, (Width / 2) - 10, (int)((Height * .85) - 10), 20, 20);
graphics.FillEllipse(circleBrush, (Width / 2) - 10, (int)((Height * .15) - 10), 20, 20);

Now build the project and you should see this:

The trick here is to use the static FromArgb method to create a new color object. I've used the existing red, green and blue values set in the color returned by ForeColor and used an alpha channel value of 127 to set it to 50% transparency. The lower the alpha channel value, the more transparent the brush will be.

Shaping the Form

We've proven that we can have shaped controls, but what about shaped forms? Well, seeing as Form is derived from Control, it would seem sensible to suggest that we can shape a form by setting its Region property. You'd be right!

First of all, though, open the code editor for Scope and add this code:

protected override void OnMouseDown(MouseEventArgs e)
{
if(e.Button == MouseButtons.Right)
Application.Exit();
}

(This method will make sense in a moment))

Now, open the code editor for Form1 and add a namespace import declaration for System.Drawing.Drawing2D.

using System;
using System.Drawing;
using System.Collections;
using System.ComponentModel;
using System.Windows.Forms;
using System.Data;
using System.Drawing.Drawing2D;

Next, open the Designer for Form1. We need to remove the thick border and caption from the form in order for this to work. Set the FormBorderStyle property to None. Also, resize the form and position the control so that the control is contained exactly within the form boundaries, otherwise when we "cut out" the circle that will make up the form, the cut out will not completely contain the Scope control.

On the Beta 2 version of VS.NET that I'm using, you may find that the Designer keeps making the form bigger than the size you've defined it to be. This isn't a problem - but you may find that when you drag the control around in a little while it will appear to "jump" the first time you move it. I'm pretty confident Microsoft will fix this problem by release.

Double-click on the background of the form. This will create a new Load event handler. Add this code:

      private
            void Form1_Load(object sender, System.EventArgs e)
{
// set
            the path...
GraphicsPath path = new GraphicsPath();

// add
            the ellipse to the path...
path.AddEllipse(scope1.Left,
            scope1.Top, scope1.Width, scope1.Height);

// set the
            region...
this.Region = new Region(path);
}

Run the project and you'll notice that the form now floats on the desktop.

However, you'll also notice that you cannot move the form or minimize it! To close the application, right-click the Scope control and this will call Application.Exit - remember that we tied in the MouseDown event to detect right-clicks and call Application.Exit.

Moving a window around the desktop is usually something managed by Windows. Internally, it's done through something called "non-client hit testing" and accessed in the old school Windows message WM_NCHITTEST.

However, our problem is that any mouse clicking and dragging that happens will be handled by the control itself. What we need to do is roll functionality into the control that allows dragging and moving. We'll do that now.

First of all, we need a single member to support the dragging process. Open the code editor for Scope and add the private _dragAdjustment member.

public class Scope : System.Windows.Forms.UserControl
{
// members...
private static Random _randomNumbers = new Random();
private ArrayList _points = new ArrayList();
private Color _backColor2;
private Point _dragAdjustment;

To support dragging we need to listen for when the mouse is pressed, when it's moved and when it is released. We've already provided a handler for OnMouseDown, but we need to roll in some more code:

      protected
            override void OnMouseDown(MouseEventArgs e)
{
// start
            dragging?
if(e.Button == MouseButtons.Left)
{
// keep the
            drag adjustment...
_dragAdjustment = new Point(e.X,
            e.Y);

// start capturing the mouse...
this.Capture =
            true;
}

// quit?
if(e.Button ==
            MouseButtons.Right)
Application.Exit();
}

The important thing here is to set the Capture property to True. With capture set on in this way, mouse messages will be sent to the control whether or not the mouse is actually over the control. This is useful as if we move the mouse quickly on a slow computer it may go outside of the bounds of the control and the dragging will "stall". We can also use the value of this property as a flag to keep tabs on whether or not we are dragging.

Next, add this method:

protected override void OnMouseMove(MouseEventArgs e)
{
// are we dragging?
if(this.Capture == true)
{

If we're dragging, Capture will be True. We then need to create a new Point object based on the coordinates passed through to the event. These coordinates will be expressed as client coordinates we need to use PointToScreen to turn the client coordinates into screen coordinates.

            //
            create a new point...
Point newPoint = new Point(e.X,
            e.Y);
newPoint = this.PointToScreen(newPoint);

When we started dragging, we stored the position of the initial click in _dragAdjustment. We need to offset the coordinates by the position of the initial click otherwise the control will appear to jump the first time we move it.

// adjust the position...
newPoint.X -= _dragAdjustment.X;
newPoint.Y -= _dragAdjustment.Y;

Finally, we set the Location property of the parent, i.e. the form.

            //
            move the owner...
Parent.Location =
            newPoint;
}
}

The last important step is to listen for the mouse button being released and turn capture off.

protected override void OnMouseUp(MouseEventArgs e)
{
// release capture...
this.Capture = false;
}

Now run the project and you should be able to drag the control around the desktop.

Summary

In this article we saw some cool new GDI+ features including shaped controls, shaped forms, double buffering, anti-aliasing, gradient fills and matrix transformations. Have fun with them in your own projects!


Animation in the MS .NET Framework Using GDI+ Part2

Animation in the MS .NET Framework Using GDI+ Part2
http://www.csharptoday.com/content.asp?id=1620
Article Rating: 4.5
Published on August 3, 2001
Abstract
In the first part of this article we introduced the main and new concepts and techniques of GDI+ within the MS VS.NET Framework, which allow us to extend the standard graphical tools and construct static and dynamic graphics. In this article, Csaba Török turns his attention to the techniques of displaying data and databases, with the stress being on dynamic viewing.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

In the first part of this article we introduced the main and new concepts and techniques of GDI+ within the MS VS.NET Framework. These allow us to extend the standard graphical tools and construct static and dynamic graphics. This article is devoted to the techniques of displaying data and databases, with the stress being on dynamic viewing.

While the first article showed how to handle graphics, here we show how to handle data - read from databases and transformed if necessary - and connect it to the graphical interface. First we will describe the data transformation process. Then we explain data plotting and visualization with dynamic elements. The next section is devoted to our sample applications - an evolving galaxy and a profit graph. In the examples we utilize the ADO.NET and GDI+ methods for dynamic charting numerical data from database tables.

The downloadable code was written in C# with MS Visual Studio .NET Beta 2. Both examples use the BlackBoard user control. The first application includes several charting examples and is implemented to show the main steps in creating dynamic plots for business data. The second application enhances the information appreciation contained in numerical database columns by viewing them dynamically.

Data and Charts

The display of graphical objects in the .NET framework occurs through the graphics device interface GDI+ that acts as a buffer between the applications and output devices. GDI+ is a device-independent graphics output model that processes graphical function calls from a Windows-based application and passes those calls to the appropriate device driver. The driver performs the hardware-specific functions that generate output.

The .NET framework provides several ways for creating a Graphics object, as was shown in the first article. As was shown, we can use the functionality of the GDI+ to paint user controls, graphical shapes, texts and create and use pens, brushes, transformation matrices. A large number of the methods supported by the Graphics object depend on numerical coordinates that are measured by default in pixels. Note we need to define the coordinates for the Windows client area the same as we did in the examples of the first article. Figure 1 shows the coordinate origin we use:

Figure 1. The coordinate origin in Windows

Coordinate systems

Let us briefly cover the basic visual techniques for drawing real data. Consider the two tables in fig. 2. From the point of view of pixel coordinates the left one contains values that are too small whereas the right one both negative and unduly large values. If we want to chart them it is necessary to map them to appropriate coordinates.

Figure 2. Different data for plotting

We composed a windows user control BlackBoard for drawing data charts that enables dynamic viewing large numerical database tables too. The control is aimed to draw the data plot in a chart area within a board area defined in the windows component client area. The relation between these areas is outlined in fig. 3.

Figure 3. The chart, board and component client areas

Before plotting, the data presented by two vectors (columns) of equal length from a data area will be transformed to the chart area, see fig. 4. Observe that the origin of the chart area is put in the left upper corner to ensure correspondence with the windows client area, see fig. 1.

Figure 4. The transformation of a point P to PT

For illustration purposes the transformation formula is given below, however it is not necessary for the understanding of the other parts of the article. The transformation of a point P from the data area to the chart area is given by transformations:

The transformation code is simply:

// BlackBoard.cs
private void
            transformPoint(float xb, float yb, ref int xbT, ref int
            ybT)
{
xbT = (int) (x1 +
            (x2-x1)/(xMax-xMin)*(xb-xMin));
ybT = (int) (y2 -
            (y2-y1)/(yMax-yMin)*(yb-yMin));
}

The method transformPoint is called by methods:

// BlackBoard.cs
private void transformVector(float[] xv, float[] yv, ref int[] xvt, ref int[] yvt)

and:

private void transformMatrix(float[,] xm, float[,] ym, ref int[,] xmt, ref int[,] ymt)

that transform the float vectors xv,yv and matrices xm,ym to integer xvt, yvt and xmt, ymt, respectively.

Static data plots

Due to this transformation approach you can draw vectors measured in any units on a device interface measured (by default) in pixels. We illustrate this by two examples, both on the download Profits.exe. The first one presents the capital letter T and the second one the profit data - see figures 5 and 6.

// Form1.cs
protected void
            dataT()
{
// Capital letter T
const int nn = 8;
float[]
            xv = new float[nn];
float[] yv = new float[nn];

xv[0] =
            0f; yv[0] = 7f;
xv[1] = 6f; yv[1] = 7f;
xv[2] = 6f; yv[2] =
            5f;
xv[3] = 4f; yv[3] = 5f;
xv[4] = 4f; yv[4] = 0f;
xv[5] =
            2f; yv[5] = 0f;
xv[6] = 2f; yv[6] = 5f;
xv[7] = 0f; yv[7] =
            5f;

this.blackBoard1.IniVectors(xv,
            yv);
}

protected void dataProfit()
{
const int nn =
            8;
float[] xv = new float[nn];
float[] yv = new
            float[nn];

xv[0] = 1f; yv[0] = 70000f;
xv[1] = 2f; yv[1] =
            65000f;
xv[2] = 3f; yv[2] = 70200f;
xv[3] = 4f; yv[3] =
            -10000f;
xv[4] = 5f; yv[4] = -50000f;
xv[5] = 6f; yv[5] =
            48000f;
xv[6] = 7f; yv[6] = 102000f;
xv[7] = 8f; yv[7] =
            111000f;
this.blackBoard1.IniVectors(xv, yv);
}

Figure 5. The line of capital T

Figure 6. The line plot of the profit values

The methods DataT and DataProfit contain the data points of the corresponding tables from fig. 2. Notice that to close the capital T letter the last point must equal the first one, so you should add this point:

const int nn = 8;
. . .
xv[8] = 0f; yv[8] = 7f;

unlike the screen coordinates, the corner points of T are defined in a natural way - the second coordinates increases upward and not downward. In methods DataT and DataProfit the data points as vectors are passed to IniVectors to do the transformation:

// BlackBoard.cs
public void
            IniVectors(float[] xv ,float[] yv ) {
int veclen =
            xv.Length;
xvt = new int[veclen];
yvt = new
            int[veclen];
this.transformVector(xv, yv, ref xvt, ref
            yvt);
}

The transformVector returns two integer vectors xvt, yvt that store the first and second pixel coordinates of the original data points. For given non-empty vectors xvt, yvt the static point plot and line plot is accomplished by calling the BlackBoard user control's methods PlotPointsInThread and PlotLinesInThread in the OnPaint method:

// Form1.cs
protected override void
            OnPaint(PaintEventArgs e)
{
if(blackBoard1.Thr !=
            null)
{
blackBoard1.ThrEnd();
}
switch(Example)
{
case 1:
            blackBoard1.PlotLinesInThread();
break;
case 2:
            blackBoard1.PlotPointsLinesInThread();
break;
case 3:
            blackBoard1.wtms =
            500;
blackBoard1.PlotPointsVectDelaytedInThread();
break;
case
            4: blackBoard1.PlotLinesInThread();
break;
case 5:
            blackBoard1.barBasePercent = 50; // between 12 and
            100.
blackBoard1.PlotBarsInThread();
break;
case 6:
            blackBoard1.barBasePercent = 50; // between 12 and
            100.
blackBoard1.wtms =
            500;
blackBoard1.PlotBarsVectDelaytedInThread();
break;
default:
            break;
}
}

protected void radioButton1_CheckedChanged
            (object sender, System.EventArgs e)
{
this.blackBoard1.Example
            = 1;
dataT();
this.Refresh();
}

protected void
            radioButton2_CheckedChanged (object sender, System.EventArgs
            e)
{
. . . // etc.

The values of variable Example are assigned in the CheckedChanged events of the radio-buttons:

// BlackBoard.cs
public void
            PlotPointsInThread() // - it is called from OnPaint
{
waitms =
            wtms;
thr = new Thread(new
            ThreadStart(PlotPoints));
thr.Start();
}

public void
            PlotPoints() // - it is called from ThreadStart
{
Graphics
            g;
g = this.CreateGraphics();

Pen pn = new Pen(Color.Blue,
            4);
int veclen = xvt.Length;
for(int i = 1; i <= veclen;
            ++i)
{
g.DrawRectangle(pn, xvt[i-1], yvt[i-1], 5,
            5);
}
}

public void PlotLinesInThread() // - it is
            called from OnPaint
{
waitms = wtms;
thr = new Thread(new
            ThreadStart(PlotLines));
thr.Start();
}

public void
            PlotLines() // it is called from ThreadStart
{
Graphics
            g;
g = this.CreateGraphics();

Point[] body = new
            Point[xvt.Length];
for(int i = 1;i <= xvt.Length;
            ++i)
{
body[i-1] = new Point(xvt[i-1], yvt[i-1]);
};
Pen
            pn = new Pen(Color.Blue,2);
if(!xMaMi)
{
g.DrawLines(pn,
            body); // does not connect the last and first points by
            default
}
}

The methods PlotPoints and PlotLines use the standard methods DrawRectangle, DrawLines ( for this reason in the transformed vector of the first coordinates xvt at least two elements should be different for a point plot and all the elements of xvt should be different for a line plot). The use of DrawRectangle, Pen and other basic GDI+ issues are explained in the first part of this two-part article series.

Any chart is made up of different elements that help to show and discover the information contained in data. We constructed a method PlotBars for displaying business data with vertical bars, see fig. 7. Vertical bar charts are the best in the following cases:

  • when you have only a few observations (and variables),
  • when no relationship can be discerned by inspecting the bars.

Remember that vertical bars most often are understood as a time sequence, running from left to right, even when no such perspective exists. The closer bars are to each other, the more easily they are understood as an expression of a tendency. You can determine the width of the bars by changing the values of barBasePercent.

// BlackBoard.cs
public void
            PlotBarsInThread() // - it is called from Onpaint
{
waitms =
            wtms;
thr = new Thread(new
            ThreadStart(PlotBars));
thr.Start();
}

public void
            PlotBars() // - it is called from ThreadStart
{
Graphics
            g;
g = this.CreateGraphics();

int veclen =
            xvt.Length;
int barBaseAll = (x2 - x1) / veclen;
int barBase =
            barBaseAll * barBasePercent / 100;
int barStart = (barBaseAll -
            barBase) / 2;
int zeroT = (int) (y2 - (y2-y1) / (yMax - yMin) *
            (0 - yMin));

g.DrawLine(new Pen(Color.Black,1), x1, zeroT-1,
            x2, zeroT-1); // axis x

Pen pn = new Pen(Color.Blue,
            2);
for(int i = 1; i <= veclen; ++i)
{
PlotBar(g,pn, x1
            + barStart + (i-1)*barBaseAll, yvt[i-1], barBase,
            zeroT);
}
}

private void PlotBar(Graphics g, Pen pn,
            int b1, int yb2, int b3, int zero)
{
int jaj = zero -
            yb2;
if( yb2 == zero)
{
yb2 = yb2 - 2;
jaj =
            2;
}
if( yb2 > zero)
{
yb2 = zero-1;
jaj =
            -jaj;
}
g.DrawRectangle(pn,b1,yb2,b3,jaj);
}

Figure 7. The bar plot of the profit values

Take a little time to experiment with setting in the method OnPaint various values for the variables barBasePercent and wtms that is given in milliseconds.

Dynamic business data view

It goes without saying that a data analysis can be made clearer and more comprehensible with the help of a number of tables and charts (plots). Tables and charts are very useful for presenting:

  • a large amount of qualitative data, e.g. types of product you sell or produce, and quantitative data, e.g. annual sales or financial ratios, in a concise and easy to read manner,
  • information which supports your arguments.

You might ask what the point is with dynamic graphics - the answer has a practical application aspect and a psychological one:

  • The combination of dynamic graphics e.g. with pivot tables enables the developers to create powerful visual and dynamical data mining applications, which enable the user to explore, analyze, visualize and discover data: reveal patterns, trends and interesting places in them. The user need not be an expert in either Excel or in database techniques and he can analyze data with such a tools easily and alone. This approach speeds up the information gain process necessary for decision making and eliminates errors and misunderstanding - the data may be viewed and analyzed by managers who can modify their questions in the data exploratory process and so get the most appropriate answer.
  • As the wavelet expert S. Mallat states, concentrating on transients is a strategy for selecting important information from the world by our senses. After several minutes in a restaurant we cease to notice the conversation and movement of neighbors, but a sudden silence reminds us of their presence. Our attention is clearly attracted by transients as opposed to stationary stimuli, which we soon ignore. Here is the psychological justification for the use of dynamic plots.

Thus, dynamic data is not simply moving data. You can use it to enhance the impact of the graphical data view. But you must be carefully. A proper amount and type of dynamic effect may be useful, but too much changing may be confusing. Avoid the distraction.

The implementation of the following simple dynamic bar plot uses threads:

// BlackBoard.cs
public void
            PlotBarsVectDelaytedInThread() // - it is cold from
            OnPaint
{
waitms = wtms;
thr = new Thread(new
            ThreadStart(PlotBarsVectDelayted));
thr.Start();
}

private void PlotBarsVectDelayted() // - it is cold
            from ThreadStart
{
gInThread =
            this.CreateGraphics();
ManualResetEvent wait = new
            ManualResetEvent(false);
wait.Reset();
Pen pn = new
            Pen(Color.Blue,2);

int veclen = xvt.Length;
int barBaseAll
            = (x2 - x1)/veclen;
int barBase = barBaseAll *
            barBasePercent/100;
int barStart = (barBaseAll -
            barBase)/2;
int zeroT = (int) (y2 - (y2-y1) / (yMax-yMin) *
            (0-yMin));

while(true)
{
gInThread.DrawLine(new
            Pen(Color.Black,1), x1, zeroT-1, x2, zeroT-1); // axis x
for(int
            i = 1; i <= veclen; ++i)
{
PlotBar(gInThread,pn, x1 +
            barStart +
            (i-1)*barBaseAll,yvt[i-1],barBase,zeroT);
wait.WaitOne(waitms,
            true);
}
gInThread.Clear(this.BackColor);
wait.WaitOne(waitms,
            true);
}
}

Dynamic database view

Now that we have an insight into the process of creating static and dynamic charts from real data through coordinate transformation, let us see a real C# windows application Galaxy.exe. It uses also the user control BlackBoard.cs for dynamic viewing of matrices that are filled from database tables in the windows form Form1.cs by the methods of the namespaces System.Data, System.Data.SqlClient and System.Data.OleDb.

We already know from the first part that in the most time consuming animations both the shape and the place of the graphical object change. The Galaxy application presents such an animation. It draws sequentially a pair of columns from a database table - the odd columns present the x and the even columns the y coordinates. The pictures in fig. 8 resemble four possible phases of a galaxy evolving in time. The four positions may correspond to four different time moments and as you can see the galaxy object changes its shape. Here the previously computed coordinates of the shapes by a special numerical algorithm are read in from a database table.

Figure 8. Galaxy like shapes

The database

We use a special table with 142 fields/columns and 200 records/rows.

The .NET Framework supports communication with different data stores thanks to ADO.NET which works with data basically in a disconnected way. You can use text files (FileStreams) and databases on a server or out of it, however the most effective way of communication with stored data is provided for servers - SQL server, IIS. We offer two ways of reading the data: from the Access database GalaxyAccDB.mdb (ADO button) and from the GalaxyDB database on MS SQL Server (SQL button), see fig. 9 below. Both databases contain the same table GalaxyTab. The performance is almost alike and the difference between the reading times from these two places is not significant.

ADO connection

The ADO button detects the Access database. The code is here:

// Form1.cs
protected void button3_Click
            (object sender, System.EventArgs e) // ADO
{
ManualResetEvent
            wait = new ManualResetEvent(false); //for
            progressbar
wait.Reset();
this.progressBar1.Visible =
            true;
this.progressBar1.Value =
            0;

++this.progressBar1.Value;
ADOGalaxy(); //the
            progressbar is continued processing in
            ADOGalaxy()

this.progressBar1.Visible =
            false;
wait.Reset();
}

protected void ADOGalaxy() //
            ADO-Access database
{
OleDbConnection galConn =
            null;
OleDbDataAdapter galComm = null;
string strADOCon =
            "Provider=Microsoft.Jet.OLEDB.4.0;Data
            Source=GalaxyAccDB.MDB";
string strADOCom = "SELECT * FROM
            GalaxyTab";
try
{
galConn = new
            OleDbConnection(strADOCon);
galComm = new
            OleDbDataAdapter(strADOCom,galConn);

galDS = new
            DataSet();
galComm.Fill(galDS, "galaxy");
dataGrid1.DataSource
            =
            galDS.Tables["galaxy"].DefaultView;

++this.progressBar1.Value;
this.createMatrixGalaxy();

++this.progressBar1.Value;
this.dataGrid1.Visible
            = true;

button2.Visible = true; // for STOP
this.Text =
            "Galaxy evolving - ADO Access DataBase";
}
catch(Exception
            e)
{
MessageBox.Show("Error while reading Galaxy
            database:\n\r" + e.ToString());
//
            Application.Exit();
}
finally
{
if(galComm !=
            null)
{
galComm.Dispose();
}
if(galConn !=
            null)
{
galConn.Close();
galConn.Dispose();
}
}
}

//
            filling the xm a ym matrices from the database table
public void
            createMatrixGalaxy()
{
DataTable galTab;
galTab =
            galDS.Tables["galaxy"];
int pocet;
int clen =
            galTab.Columns.Count / 2; // = 142/2;
int rlen =
            galTab.Rows.Count; // =200
float[,] xm = new
            float[rlen,clen];
float[,] ym = new
            float[rlen,clen];

string colu;
string colu2;
DataColumn
            myCol;
DataColumn myCol2;

// Fill xm, ym
for(int ic =
            0; ic <= clen -1; ++ic)
{
colu =
            colum("col",2*ic+1);
colu2 = colum("col",2*ic+2);
pocet =
            -1;

foreach(DataRow myRow in
            galTab.Rows){
pocet++;
myCol = galTab.Columns[colu];
myCol2
            = galTab.Columns[colu2];
if(pocet <= rlen-1){xm[pocet,ic]=
            (myRow[myCol].ToString().ToSingle());}
if(pocet <=
            rlen-1){ym[pocet,ic]=
            (myRow[myCol2].ToString().ToSingle());}
}
}
this.blackBoard1.IniMatrix(xm,ym,rlen,clen);
}

private
            string colum(string colSt, int colSZ)
{
string
            colT;
colT=colSt + colSZ.ToString();
if(colSZ<10){colT =
            colSt + "00" + colSZ.ToString();}
else
if(colSZ<100){colT =
            colSt + "0" + colSZ.ToString();}
return(colT);
}

Getting data out of the database may cause problems. To protect the block of the data reading code and keep the error handling localized, we use .NET's standard try and catch exception handling structure in the ADOGalaxy method. The data transfer and view is done by the first five code lines of the try block:

  • the method OleDbConnection addresses the database,
  • OleDbDataAdapter performs the SQL commands,
  • the data is pushed into a DataSet object, however DataReader and DataAdapter could be used too,
  • the data is viewed by the DataGrid, however the view could be done as well as by DataView and ListView.

Once you have read in the data, you can view it. The galaxy data is displayed in a datagrid as seen below. The datagrid is composed of individual cells that form rows and columns. To display the data use the scrollbar located along the right and bottom edges of the datagrid. The odd columns present the first and the even columns the second coordinates of the galaxy shapes.

The process of reading the database and preparing the matrices for plotting is followed by a progressbar. By analogy with plotting the business data, where the first and second coordinates were kept in two vectors, the thecreateMatrixGalaxy method puts them into two matrices: xm stores the odd columns and ym the even columns.

Figure 9. Data reading from SQL and ADO data tables and the datagrid

SQL connection

To use the SQL button, you have to install the database on the SQL server. The main difference in reading the data from the SQL server is in these code lines:

protected void SQLGalaxy() //
            SQL
{
SqlConnection galConn = null;
SqlDataAdapter galComm
            = null;
try
{
galConn = new
            SqlConnection("server=localhost;uid=sa;pwd=;database=GalaxyDB");
galComm
            = new SqlDataAdapter("select * from galaxytab", galConn);
//
            galConn = new
            SqlConnection("server=localhost;uid=sa;pwd=;database=master");
galDS
            = new DataSet();
galComm.Fill(galDS,
            "galaxy");
dataGrid1.DataSource =
            galDS.Tables["galaxy"].DefaultView;

++this.progressBar1.Value;
this.createMatrixGalaxy();
.
            . . // and so on

As you can see the methods SqlConnection and SqlDataAdapter are used instead of OleDbConnection and OleDbDataAdapter.

Dynamic viewing

Our goal is a dynamic visual representation of the information in the galaxy data. The animation is launched with the Galaxy button. The last two code lines in the button2_click event method are the most important ones. The Refresh method refreshes the OnPaint method that on the base of the value of Example executes the animation in the blackboard1 control, instantiated from the BlackBoard user control.

Figure 10. Dynamic viewing of data and speed regulation with a scrollbar

// Form1.cs
public class Form1 :
            System.WinForms.Form
{
/// <summary>
/// Required
            designer variable.
/// </summary>

private
            System.ComponentModel.Container components;

// instantiating
            a BlackBoard object:
private UCBlackBoard.BlackBoard blackBoard1
            = new UCBlackBoard.BlackBoard ();
. . .

// The
            button2_Click event starts the Galaxy plotting:
protected void
            button2_Click (object sender, System.EventArgs
            e)
{
button1.Visible = false;
button2.Visible =
            false;
button3.Visible = false;
button4.Visible =
            true;
Example = 0;
this.Refresh();
}

protected
            override void OnPaint(PaintEventArgs e)
{
if(blackBoard1.Thr
            != null)
{
blackBoard1.ThrEnd();
}
if(Example ==
            0)
{
blackBoard1.wtms =
            50;
blackBoard1.PlotPointsMatrDelaytedInThread();
}
}

// BlackBoard.cs
public void
            PlotPointsMatrDelaytedInThread()
{
thr = new Thread(new
            ThreadStart(PlotPointsMatrDelayted));
thr.Start();
}

private void PlotPointsMatrDelayted() // - it is cold
            from StartThread
{
gInThread =
            this.CreateGraphics();
ManualResetEvent wait = new
            ManualResetEvent(false);
wait.Reset();
Pen pn = new
            Pen(Color.Blue,1);
while(true)
{
for(int j = 0; j <=
            clen-1; ++j)
{
// the first and second coordinates of the
            shapes are contained in matrices xmt, ymt:
for(int i = 0; i <=
            rlen-1; ++i)
{
gInThread.DrawRectangle(pn, xmt[i,j], ymt[i,j],
            2, 2);
}
wait.WaitOne(waitms,
            true);
gInThread.Clear(this.BackColor);
}
}
}

Since the drawing is done in a thread, the speed of the galaxy moving and evolving may be regulated by a scrollbar and the animation stopped in any time.

Conclusion

We have only scratched the surface of the GDI+. There are many more features - antialiasing, clipping, images, metafiles, containers - that were not mentioned. The .NET framework makes it extremely easy to work with graphics and data. We showed the use of dynamic effects in viewing business data. In some circumstances it may be necessary to read data for visualization from databases. We demonstrated with a galaxy example the connection of SQL and ADO databases with graphics.

When developing business applications you can use GDI+ methods or third party tools for displaying data. The downside of using GDI+ methods may be the fact that this approach is time demanding. However due to GDI+ coding your application will be original with possible new data presentation solutions and effects that the end users of the application will surely appreciate.

To recap, this two-part article series on GDI+ presented an overview of the following items:

  • local and global object transformations and transformation matrices
  • shifting, inflating, rotating and reflecting
  • coordinate systems and data transformations
  • static shapes, animation and timing
  • user controls and mouse events
  • static and dynamic data visualization
  • ADO.NET. SQL Server, Access DB
  • datagrid, progressbar
  • user component
  • threading

Web development is a little different to Windows development. While this series showed the techniques for Windows Forms, a further article would be devoted to the (dynamic) visualization in ASP.NET Web Forms.


Graphics and Animation in the MS .NET Framework Using GDI+- Part 1

Graphics and Animation in the MS .NET Framework Using GDI+- Part 1
http://www.csharptoday.com/content.asp?id=1543
Article Rating: 4.5
Published on July 31, 2001
Abstract
In this two part series, Csaba Török gives us a short introduction to the base techniques of GDI+. In this first article, we see how to construct single and composite shapes, how to use transformations and make animations, handle mouse events and combine them with graphics. We also briefly discuss the new concepts of local and global transformations using transformation matrices introduced by the .NET framework.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

This article aims to give you an introduction to the Graphics Device Interface + (GDI+) in MS VS.NET. It describes the new and main concepts of graphics within the MS .NET Framework technology.

The issues covered in this article are:

  • The basic techniques for drawing and painting graphic shapes.
  • A brief description of the role of global and local transformations.
  • Issues involved in the dynamic viewing of static shapes and texts.
  • We see how to move and resize, rotate, reflect and stretch graphic objects - the techniques being illustrated by examples.

While this first article shows the main graphic techniques, the second one will show the static and dynamic visualization of data. You will learn how to work with coordinate systems and connect data and databases with graphics.

This two-part article series shows the key GDI+ techniques for windows forms. In web forms the graphics issues are both similar to the windows ones and different from them. We intend to devote to visualization in ASP.NET Web Forms a further article, however to understand the web graphics better you will find this series useful.

The provided windows application Options.exe described in this article includes eight simple graphic and animation examples. The structure of the application is showed in fig. 1. If you place the mouse in the MS VS.NET on the symbols /**/ and ... at the beginning and end of the code lines,

you will see the comments to the methods and their code, respectively.

Figure 1. The structure of the windows application Options

Painting static shapes

The .NET Framework provides a variety of controls that allow us to view data or pictures and interact with the users of our application. These graphic controls include check boxes and option buttons for yes/no queries, datagrids for viewing database tables with data (strings, numbers, pictures). Sometimes these tools may be insufficient for your needs or you want a special solution for interaction or display. With GDI+ you can add other graphic shapes to your windows form or web form that can act (if you want) as the standard graphic controls. You create these shapes using some of the standard classes and structures of the GDI+ and their methods and properties contained in the namespaces System, Drawing, Drawing2D, Collections, Windows.Forms, Data, Threading, Design.

Colors, shapes, graphics and painting

The four basic steps in painting graphic shapes are:

  • choosing a color
  • determining the shape
  • getting the graphics interface
  • painting - drawing, filling

Let us see the code of our first example that creates and paints some default shapes such as rectangle and circle (see screenshot below). The method Example1 executes three of the graphics steps above (the Graphics object is instantiated elsewhere in the virtual method OnPaint before calling the example methods).

// Form1.cs
protected void
            Example1(Graphics g)
{
Pen pn = new Pen(Color.Blue,
            10);
Brush br = new SolidBrush(Color.AliceBlue);
Rectangle rc
            = new Rectangle(50, 50, 150, 150);

g.FillRectangle(br,
            rc);
g.FillEllipse(Brushes.Red, rc);
g.DrawRectangle(pn,
            rc);
g.DrawEllipse(Pens.Yellow, rc);
pn.Dispose();
br.Dispose();
}

Figure 2. Example 1 - base shapes

Note that the quality of the picture may depend on the order of the painted layers. The frame of the rectangle in fig. 2 was drawn deliberately after filling the circle and it hides four parts of the latter.

Color

There are several ways to set the color for painting - we used two of these different methods for drawing and for filling. The color of drawing can be chosen with classes Pen or Pens and for filling with SolidBrush or Brushes. While the Pen is a class the Brush is an abstract class, so we must use one of the derived classes of Brush, like SolidBrush, TextureBrush or LinearBrush.

Figure 3. IntelliSense in action for the class Pen

The advantage of Pen is in the possibility of setting the width of the lines and curves with the second argument. The Pen class has four constructors and IntelliSense help displays them all, as seen above. You can view the individual signatures using the up and down keys. You can get the same information from the Object Browser that allows you to view the contents of various components that you are interested.

To use the object browser:

  • click on: View - Other Windows - Object Browser (F2 or Ctrl+Alt+J)
  • select in the Objects pane: system.drawing.dll - Pen.

The Members and Description panes show the arguments:

Figure 4. The Object Browser and the class Pen

Shapes and painting

In C# by the Windows convention the upper left corner of the graphics screen is [0, 0]. The x values increment to the right. The y values increment downward. The maximum available coordinates of the windows client's area along the x and y axes may be determined by calling the X, Y properties of the ClientRectangle of the underlying form, as e.g.

this.ClientRectangle.X;

Unlike C and Turbo Pascal, where the rectangle is defined by the coordinates of its two opposite corner points, i.e. by left, top, right, bottom, in C# the rectangle is defined similarly to Visual Basic by its left upper corner point (left, top) and the width and high. As you can see, the circle (Ellipse) uses the same structure as the square (rc Rectangle) and it is not defined by the center and radius. If you want to draw a circle with a center [x, y] and radius r, then you have to use:

Rectangle(x-r, y-r, 2*r, 2*r);
g.FillEllipse(brush, rc);

or

g.DrawEllipse(pen, rc);

The real painting is done in our first example by methods DrawRectangle, DrawEllipse and Fillrectangle, FillEllipse. Below we give some of the default simple shapes and the composite ones and indicate which can be drawn by a line and/or filled as a bounded range.

It follows from this short table that while, for example, the simple curve shape may be drawn only by the method DrawCurve, the composite shape object GraphicsPath may be both drawn and filled by the methods GraphicsPathDraw and GraphicsPathFill, respectively. GraphicsPath is a container of shape, text and image objects. You can add a variety of single objects to a GraphicsPath object gp by methods as gp.AddEllipse or gp.AddString, see examples.

Creating Graphics objects

A Rectangle is an abstract shape structure that exists only in memory. To create the shapes on a device such as a screen or printer we need a graphics device surface - in this case the Windows form's surface, presented by an instantiated object of the Graphics class. This is the missing step from the four required ones. The Graphics class encapsulates a GDI+ drawing surface. A graphics object can be achieved by several ways. Below we briefly describe five approaches, based on:

  • OnPaint
  • Form1_Paint
  • CreateGraphics
  • FromHwnd
  • BitMap

OnPaint

The only argument of the RichControl 's OnPaint virtual method is of type PaintEventArgs and its Graphics property returns a Graphics object.

// Form1.cs
protected override void
            OnPaint (PaintEventArgs e)
{
Graphics g = e.Graphics;
g.Clear(this.BackColor);
if (this.radioButton1.Checked)
            Example1(g);
if (this.radioButton2.Checked) Example2(g);
if
            (this.radioButton3.Checked) Example3(g);
if
            (this.radioButton4.Checked) Example4(g);
if
            (this.radioButton5.Checked) Example5(g);
if
            (this.radioButton6.Checked) Example6(g);
if
            (this.radioButton7.Checked) Example7(g);
if
            (this.radioButton8.Checked)
            Example8(g);
g.Dispose();
}

Notice that OnPaint is a virtual method so it should be overridden. As regards the method Dispose, see the section Dispose and redraw.

Form1_Paint

Similarly to the previous case, the Graphics property in the form's Paint method returns a Graphics object. A possible use of Form1 _ Paint:

private void Form1_Paint (object sender, System.WinForms.PaintEventArgs e)
{
Graphics g = e.Graphics;
. . . // the rest is the same as above
}

CreateGraphics

The WinForms 's CreateGraphics () method also gives us a graphics object:

Graphics g = this.CreateGraphics();

This approach avoids the OnPaint and Paint events and you can use it for painting triggered events like Click or MouseMove. The following illustration code is illustrated below in fig. 5 (the property Contains returns true if the rcMouse rectangle contains the point here). If the mouse is placed on the control"Come here " from the left picture of fig. 5, the control changes its frame and text to "Thank You ".

// Form1.cs
protected void
            Form1_MouseMove (object sender, System.WinForms.MouseEventArgs
            e)
{
Pen pn1 = new Pen(Color.Red, 2);
Point here = new
            Point(e.X,e.Y);
Graphics g = this.CreateGraphics();

if
            (this.radioButton2.Checked)
{
if (rcMouse.Contains(here) &
            bRedraw)
{
bRedraw = false;
pn1 = new Pen(Color.Red,
            2);
string text = "Thank
            you";
Example2b(g,text,pn1);
}
if
            ((!rcMouse.Contains(here)) & (!bRedraw))
{
bRedraw =
            true;
pn1 = new Pen(Color.Blue, 5);
string text = "Come
            here";
Example2b(g,text,pn1);
}
}
pn1.Dispose();
g.Dispose();
}

protected void Example2(Graphics g) {
            // mouse
this.Text = "Example2";
Pen pn1 = new Pen(Color.Blue,
            5);
string text = "Come
            here";
Example2b(g,text,pn1);
pn1.Dispose();
}

protected
            void Example2b(Graphics g, string text, Pen pn1) { //
            mouse
this.Text = "Example2";
Brush br1 = new
            SolidBrush(Color.Red);

g.FillRectangle(Brushes.AliceBlue,
            rcMouse);
g.DrawRectangle(pn1, rcMouse);

GraphicsPath gp1
            = new GraphicsPath();
StringFormat sf = new
            StringFormat();
FontFamily ff = new FontFamily("Times New
            Roman");
gp1.AddString(text, ff, 1, 30f, rcMouse,
            sf);
g.DrawPath(Pens.Red,gp1);
br1.Dispose();
}

Figure 5. Example 2 - handling mouse events for a user control

FromHwnd

The method FromHwnd creates a new graphics object from a handle to a specified object, such as windows forms controls.

This code:

Graphics g1=Graphics.FromHwnd(this.Handle);
Example1(g1);

Graphics g2=Graphics.FromHwnd(this.button1.Handle);
Example1(g2);

Graphics g3=Graphics.FromHwnd(this.label1.Handle);
Example1(g3);

Graphics g4=Graphics.FromHwnd(this.textBox1.Handle);
Example1(g4);

creates the base shapes of four objects on the surface of our Windows Form sample: form, button, label and text-box, respectively.

BitMap

The last approach, which is the main technique for painting and animating on Web Forms in ASP.NET, gets the graphics object from a BitMap object. The schema of this approach is:

   BitMap bm = new
            BitMap(...);
Graphics g = Graphics.FromImage(bm);
. .
            .

The use of this approach in the ASP.NET framework will be covered in an upcoming article.

Dispose and redraw

In addition to the four basic steps of painting, there are two further steps that must be mentioned. Whereas we write mainly managed code in C# and can rely on the Garbage Collector (GC), it is a good programming practice and is highly recommended to dispose of some of the objects we instantiate for painting.

In general the GC will clean up the object, but the problem is when. If a class or structure has a Dispose method then you should call it to release resources: the performance of the application is somewhat better if you clean up the instantiated object by using Dispose.

Sometimes it is necessary to redraw the picture. The methods Invalidate and Refresh, inherited from the Control class, serve this purpose. Invalidate invalidates the control and causes a paint message to be sent. Refresh forces the control to invalidate and immediately repaint itself and any children. We use it in the Click event of the radio buttons:

// Form1.cs
protected void
            radioButton1_Click (object sender, System.EventArgs e)
{
this.Refresh();
}

Global and local transformations

To make graphics and animation simpler, Microsoft introduced in the .NET Framework two types of transformations, global and local. The global transformation applies to every item drawn by a given Graphics object; the local transformation applies only to a specific item to be drawn. The global transformation is related to the whole interface surface so you can execute them by calling the appropriate methods of the Graphics class. The local transformation affects only one shape object and may be used with either a GraphicsPath object or a Rectangle structure.

We present the application of some transformations on five examples - the last three being animations. Let us start with the most simple transformations - Offset and Inflate. Their usage is illustrated by building a pyramid, see fig. 6, coded in example 3:

// Form1.cs
protected void
            Example3(Graphics g) { // pyramid
this.Text = "Example3";
Pen
            pn = new Pen(Color.Blue);
int ten = 10;
Rectangle rc = new
            Rectangle(50, 150, 150, ten);
g.DrawRectangle(pn, rc);
for(int
            i=1;i<=7;i++)
{
rc.Offset(0,-ten+1);
rc.Inflate(-ten,0);
g.DrawRectangle(pn,
            rc);
}
pn.Dispose();
}

The rc.Offset(x, y) shifts the upper left corner of the rectangle rc by x units down and by y units right. The positive and negative arguments in Inflate result in stretching and contracting, respectively. Thus Inflate(-10, 0) means, that the rectangle loses 5 units from both the left and right side, but does not change along the vertical axis as the second parameter is zero.

Figure 6 Example 3 - reflecting and rotating

The Matrix class pays a key role in different local and global transformations - you can read more about the Matrix class and transformations at: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/gdicpp/hh/gdicpp/cpp_aboutgdip05_00c4.asp&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq and about coordinate systems at: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/gdicpp/hh/gdicpp/cpp_aboutgdip05_3yyc.asp&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq.

The static reflection we illustrate in fig. 8 below, which transforms the base triangle five times. After creating the base triangle tri it is added to the GraphicsPath gp:

// To enable transformation for tri we add
            it to a GraphicsPath:
GraphicsPath gp=new
            GraphicsPath();
gp.AddLines(tri);
// We draw the base
            tri:
Pen p1=new
            Pen(Color.Red,2);
g.DrawPath(p1,gp);

The first reflection of the tri with respect to the axis x is achieved by

// Reflection Down with respect to the
            axis x:
Matrix tr=new
            Matrix(1,0,0,-1,0,0);
gp.Transform(tr);
Pen p2=new
            Pen(Color.Magenta,2);
g.DrawPath(p2,gp);

The creation of the upper two triangles is more interesting:

// Form1.cs
protected void
            Example4(Graphics g) // reflection
{
this.Text =
            "Example4";
// Create a triangle shape tri:
int[] x = new
            int[]{0,0,80,0};
int[] y = new int[]{0,-50,0,0};
Point[] tri
            = new Point[x.Length];
for(int i = 0; i < x.Length;
            i++)
{
tri[i]= new Point(x[i],y[i]);
};

// Shift the
            origin and mark it with a blue
            circle:
g.TranslateTransform(150,180);
Rectangle rc = new
            Rectangle(0,0,10,10);
rc.Offset(-5,
            -5);
g.FillEllipse(Brushes.Blue,rc);

// To enable
            transformation for tri we add it to a GraphicsPath:
GraphicsPath
            gp=new GraphicsPath();
gp.AddLines(tri);
// We draw the base
            tri:
Pen p1=new Pen(Color.Red,2);
g.DrawPath(p1,gp);

//
            Reflection Down with respect to the axis x:
Matrix tr=new
            Matrix(1,0,0,-1,0,0);
gp.Transform(tr);
Pen p2=new
            Pen(Color.Magenta,2);
g.DrawPath(p2,gp);

// Reflection
            Left with respect to the axis y:
gp.Transform(tr); // -
            transformation back
tr=new
            Matrix(-1,0,0,1,0,0);
gp.Transform(tr);
g.DrawPath(p2,gp);

//
            Reflection with respect to the origin:
gp.Transform(tr); // -
            transformation back
tr=new
            Matrix(-1,0,0,-1,0,0);
gp.Transform(tr);
g.DrawPath(p2,gp);

//
            Creation of the two upper triangles.
// Rotating by 90 degrees
            and shifting:
gp.Transform(tr); // - transformation
            back
double ang=90.0*System.Math.PI/180.0;
float
            sa=(float)System.Math.Sin(ang);
float
            ca=(float)System.Math.Cos(ang);
tr=new
            Matrix(ca,-sa,sa,ca,0,-70);

// After this transformation we
            could not return back as previously, so make a copy:
GraphicsPath
            gpOld = new GraphicsPath ();
gpOld =
            (GraphicsPath)gp.Clone();
gp.Transform(tr);
g.DrawPath(Pens.Black,gp);

//
            Adding rotation (by a new way) and shift to reflection:
// unlike
            Matrix rotation, Rotate rotates clockwise
// unlike the
            TranslateTransform the positive arguments in Translate shift to the
            right
// and up
tr=new Matrix(-1,0,0,1,0,0);
tr.Rotate(90,
            MatrixOrder.Append);
tr.Translate(70,0); // MS Error: x <->
            y
gpOld.Transform(tr);
g.DrawPath(Pens.Blue,gpOld);

g.ResetTransform();
}

Figure 8. Example 4- reflecting and rotating

Animation

There are two basic types of animation: classic and color effect animation.

  • In classic animation both the pixels and the colors can change.
  • Whereas animation by color effects is achieved by changing only the colors.

      Here we will consider purely the first type of animation. The classic animation in principle consists of five steps:

      • choosing the graphical object (pixel) by defining its x,y coordinates,
      • choosing a color and painting the object with it,
      • overpainting the object with the original color - erasing,
      • shifting our object to a new place,
      • painting the object with the chosen color.

      The first two steps may be considered as the initial steps and the process of animation is done by repeating the last three steps. The most time consuming step is the fourth one. There may be two situations for shifting, depending on the type of (classic) animation:

      • The shape of the object (which is made up by pixels) remains the same but the place of the object changes. In this situation a new location may be defined e.g. by a shift, rotation and reflection transformation.
      • Both the shape and the place of the object change. In the determination of the new shape one can suggest special computational algorithms or use default transformations for dilatation and compression in combination with the earlier mentioned transformations. Usually this case needs the most computation time.

      We will give examples for both situations. The picture in the fig. 9 illustrates the dynamic rotation of static text. It may serve as an example for the first situation, because here the text as a shape does not change, only its location. The second situation is illustrated with the pictures of fig. 10 and fig. 11, respectively. The first one is similar to the previous figure, but here in addition stretching is used.

      Let us see our animation examples. They use a wait object, instantiated from the Threading namespace's ManualResetEvent.

      // Form1.cs
protected void
            Example5(Graphics g) { // rotating Hello
this.Text =
            "Example5";
Brush br1 = new SolidBrush(Color.Red);
Brush br2 =
            new SolidBrush(Color.Blue);

ManualResetEvent wait = new
            ManualResetEvent(false);
wait.Reset( );

int phi =
            36;
int jaj = 0;
for (int i = -phi; i < 360 - phi; i +=
            phi)
{
wait.WaitOne(500,true);

g.TranslateTransform(80,100);
            // left Hello
g.RotateTransform(phi*jaj);
g.DrawString("
            Hello",this.Font,br1,0,-(int)(0.5*this.Font.Height));
g.RotateTransform(-phi*jaj);
g.TranslateTransform(-80,-100);

g.TranslateTransform(220,100);
            // right Hello
g.RotateTransform(phi*jaj+180);
g.DrawString("
            Hello",this.Font,br2,0,-(int)(0.5*this.Font.Height));
g.RotateTransform(-phi*jaj-180);
g.TranslateTransform(-220,-100);

jaj++;
};
wait.Reset(
            );
g.ResetTransform();
br1.Dispose();
br2.Dispose();
}

      Figure 9. Example 5 - animation with rotating

      If you have a close look at the code of example 5 with global transformations, you will notice that it has two weak points. They are the code lines:

      g.TranslateTransform(-80,-100);
      . . .
      g.TranslateTransform(-220,-100);

      These steps are done to return back to the base position. If you ask whether there is a solution without returning to a base position, the answer is yes, but the trick is to introduce two text shapes - one for the left and one for the right Hello from fig. 9 and use local transformations - a technique we will see how to do below.

      The example in fig.10 below shows the combination of local and global transformations. It rotates four objects with global transformations and stretches two GraphicsPath objects with local transformation:

      // Form1.cs
protected void
            Example6(Graphics g) { // rotating and stretching 4 squares with one
            Loop
this.Text = "Example6";
ManualResetEvent wait = new
            ManualResetEvent(false);
int mils = 500;
int phi =
            36;

// White Squares by Rectangles
Brush br1 = new
            SolidBrush(Color.White);
Rectangle rc = new Rectangle(20, 20, 18,
            18);
rc = new Rectangle(40, 0, 18, 18); // 0 => horizantally
            from the origy

g.TranslateTransform(150, 130); // the
            origin
rc.Offset(0, -18/2); // shift up by the half height of the
            rectangle

// A square by GraphicsPath
int[] x;
x = new
            int[]{0, 18, 18, 0, 0};
int[] y = {0, 0, 18, 18, 0};
Point[]
            psqr = new Point[x.Length];
for(int i = 0; i < x.Length;
            i++)
{
psqr[i] = new Point(x[i] + 40, y[i] -
            18/2);
};

GraphicsPath gp1 = new
            GraphicsPath();
GraphicsPath gp2 = new
            GraphicsPath();
GraphicsPath gp3 = new
            GraphicsPath();
gp1.AddLines(psqr); // so far without
            drawing
gp2.AddLines(psqr);
gp3.AddLines(psqr);
// The base
            yellow square
Pen py = new Pen(Color.Yellow,
            10);
g.DrawPath(py, gp1); // 0)Base

// The Blue, Red,
            Magenta path squares
Pen p1 = new Pen(Color.Blue, 2);
Pen p2 =
            new Pen(Color.Red, 2);
Pen p3 = new Pen(Color.Magenta,
            1);

// The transformation for increasing big red
            path-squares
Matrix tr2 = new Matrix(1.05f, 0, 0, 1.05f, 0,
            0);

// The transformation for increasing small magenta
            path-squares
Matrix tr3 = new Matrix(0.1f, 0, 0, 0.1f, 0,
            0);
gp3.Transform(tr3);
// if 0.1*q^10 = 1 => q =
            (1/0.1)^(1/10)
float q = (float)System.Math.Pow(1.0/0.1,
            1.0/10.0);
tr3 = new Matrix(q, 0, 0, q, 0, 0);

for (int i
            = phi; i < 360 + phi; i += phi)
{
wait.WaitOne(mils,
            true);

g.FillRectangle(br1, rc); // The equal white
            rectangle-squares

g.DrawPath(p1, gp1); // The equal blue
            path-squares

gp2.Transform(tr2);
g.DrawPath(p2, gp2); //
            The increasing big red
            path-squares

gp3.Transform(tr3);
g.DrawPath(p3, gp3); //
            The increasing small magenta
            path-squares

g.RotateTransform(phi);
};

gp1.Dispose();
            gp2.Dispose(); gp3.Dispose();
p1.Dispose(); p2.Dispose();
            p3.Dispose();
py.Dispose(); br1.Dispose();
tr2.Dispose();
            tr3.Dispose();

g.ResetTransform();
wait.Reset();
            wait.Close();
}

      The number 0.5 is by default of type double. Since the parameters of Matrix are of float type, we must do type casting in Matrix(1.05f, 0, 0, 1.05f, 0, 0). It can be done in two ways: you can write 0.5f or (float)0.5.

      Figure 10. Example 6 - rotating with stretching

      The stretching technique from the previous example is used once again in the example in fig.11 below to provides a head animation. By setting the values of the variables nn, shrink and milsec in the method plotHead you can change the animation process; however the end size of the head will be always the same.

      // Form1.cs
protected void
            Example7(Graphics g) { // head-fej
this.Text =
            "Example7";
const int nn = 64;
int[] xv = new
            int[nn];
int[] yv = new int[nn];

xv[0]=86;
            yv[0]=8;
xv[1]=96; yv[1]=10;
xv[2]=87; yv[2]=18;
xv[3]=89;
            yv[3]=22;
xv[4]=90; yv[4]=27;
xv[5]=90; yv[5]=29;
xv[6]=90;
            yv[6]=32;
xv[7]=90; yv[7]=36;
xv[8]=87; yv[8]=43;
xv[9]=90;
            yv[9]=46;
xv[10]=93; yv[10]=50;
xv[11]=101;
            yv[11]=52;
xv[12]=104; yv[12]=54;
xv[13]=105;
            yv[13]=56;
xv[14]=104; yv[14]=60;
xv[15]=101;
            yv[15]=61;
xv[16]=98; yv[16]=62;
xv[17]=93;
            yv[17]=62;
xv[18]=93; yv[18]=63;
xv[19]=91;
            yv[19]=65;
xv[20]=92; yv[20]=69;
xv[21]=92;
            yv[21]=71;
xv[22]=92; yv[22]=74;
xv[23]=93;
            yv[23]=76;
xv[24]=93; yv[24]=77;
xv[25]=91;
            yv[25]=78;
xv[26]=80; yv[26]=77;
xv[27]=80;
            yv[27]=77;
xv[28]=80; yv[28]=77;
xv[29]=90;
            yv[29]=78;
xv[30]=93; yv[30]=80;
xv[31]=90;
            yv[31]=83;
xv[32]=90; yv[32]=87;
xv[33]=90;
            yv[33]=88;
xv[34]=87; yv[34]=90;
xv[35]=69;
            yv[35]=88;
xv[36]=65; yv[36]=91;
xv[37]=60;
            yv[37]=93;
xv[38]=63; yv[38]=99;
xv[39]=63;
            yv[39]=102;
xv[40]=63; yv[40]=106;
xv[41]=66;
            yv[41]=111;
xv[42]=63; yv[42]=111;
xv[43]=60;
            yv[43]=112;
xv[44]=48; yv[44]=109;
xv[45]=42;
            yv[45]=109;
xv[46]=36; yv[46]=109;
xv[47]=30;
            yv[47]=112;
xv[48]=29; yv[48]=109;
xv[49]=27;
            yv[49]=107;
xv[50]=33; yv[50]=96;
xv[51]=30;
            yv[51]=91;
xv[52]=27; yv[52]=87;
xv[53]=15;
            yv[53]=90;
xv[54]=11; yv[54]=82;
xv[55]=6;
            yv[55]=75;
xv[56]=0; yv[56]=56;
xv[57]=3;
            yv[57]=43;
xv[58]=6; yv[58]=30;
xv[59]=14;
            yv[59]=10;
xv[60]=27; yv[60]=5;
xv[61]=42;
            yv[61]=0;
xv[62]=75; yv[62]=5;
xv[63]=86; yv[63]=8; // the
            same as xv[0] and yv[0] to close the line

Point[] body = new
            Point[nn];
for(int i = 1;i <= nn; ++i)
{
body[i-1] = new
            Point(xv[i-1],yv[i-1]);
};

plotHead(g,
            body);
g.Dispose();
}

void plotHead(Graphics g, Point[]
            body)
{
int nn = 15; // for loop
float shrink = 0.01f; //
            the beginning shrinkage
int milsec = 50; // for wait

Pen
            pn = new Pen(Color.Blue,1);
Pen p2 = new
            Pen(this.BackColor,1);

g.TranslateTransform(90,80);
GraphicsPath
            gp = new GraphicsPath();
gp.AddLines(body); // AddLines does not
            connect the last and first points by default

Matrix mat = new
            Matrix(shrink,0,0,shrink,0,0);
gp.Transform(mat);
float q =
            (float)System.Math.Pow(1.0/shrink, 1.0/(float)nn);
mat = new
            Matrix(q, 0, 0, q, 0, 0);

ManualResetEvent wait = new
            ManualResetEvent(false);
wait.Reset();

for(int i = 0;i
            <
            nn-1;++i)
{
gp.Transform(mat);

g.DrawPath(pn,gp);
wait.WaitOne(milsec,
            true);
g.DrawPath(p2,gp);
}
g.DrawPath(pn,gp);

pn.Dispose();
            p2.Dispose();
gp.Dispose();
wait.Reset();
            wait.Close();
}

      Figure 11. Example 7 - final frame in head animation

      The last example illustrates the use and definition of the intersection, union and excluding operations in the GDI+ context by the aid of two rectangles - see the magenta and yellow squares in fig. 12.

      Figure 12. Examples 8 - set like operations

      The definition of intersect does not need any explanation (see the blue square), however the result of union is a rectangle, containing totally the two rectangle operands (see the big red square), so it differs from the classic set union drawn in the plane. The white range corresponds to the result of the excluding operation.

      // Form1.cs
protected void
            Example8(Graphics g) { // Region: Intersect, Union, Exclude
            (Xor)
this.Text = "Example8";
Rectangle rec1= new
            Rectangle(50,50,100,100);
Rectangle rec2= new
            Rectangle(100,100,100,100);
Pen p1 = new
            Pen(Color.Magenta,5);
g.DrawRectangle(p1,
            rec1);
g.DrawRectangle(Pens.Yellow, rec2);

Rectangle
            rec3;
rec3=Rectangle.Intersect(rec1,rec2);
g.DrawRectangle(Pens.Blue,
            rec3);

Rectangle
            rec4;
rec4=Rectangle.Union(rec1,rec2);
g.DrawRectangle(Pens.Red,
            rec4);

Region ran;
ran=new
            Region(rec1);
ran.Exclude(rec2);
g.FillRegion(Brushes.Azure,
            ran);

p1.Dispose();
}

      Conclusion

      This article gives a short introduction to the base techniques of GDI+. It shows how to construct single and composite shapes, how to use transformations and make animations, handle mouse events and combine them with graphics. It presents the new concepts introduced by the .NET framework, as local and global transformations or transformation matrices.

      For plotting and charting it is often necessary to read data from databases. In the second article on GDI+ we will demonstrate how to connect to SQL and ADO data with graphics and show the use of dynamic effects in viewing business data.

      • Brought to you by ownSky!


ImageIt - Image processing for Photo Galleries

ImageIt - Image processing for Photo Galleries
http://www.csharptoday.com/content.asp?id=1534
Article Rating: 5
Published on July 25, 2001
Abstract
From photo galleries, to product catalogs, to resource listings, adding images dynamically to a web site is a common occurrence. The uses vary greatly, and so do the users who need to add the images - from web masters and graphic artists, to the people who just purchased their first digital camera. Although the uses are many, how they end up on the web site is usually done in the same tedious manner. In this article, Troy Marchand demonstrates how to create an image processing class that can easily take any uploaded image and transform it to the required dimensions, image format, and even add titles to the image. This article is available as a $12 e-doc from http://www.amazon.com/exec/obidos/ASIN/B00005T5SQ/ref=nosim/wrox-20.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

From photo galleries, to product catalogs, to resource listings, adding images dynamically to a web site is a common occurrence. The uses vary greatly, and so do the users who need to add the images. From web masters and graphic artists, to the people who just purchased their first digital camera.

Although the uses are many, how they end up on the web site is usually done in the same tedious manner:

  • First, find out what the requirements are to add the image. This usually consists of the required image dimensions, possibly a thumbnail image, and image formats (gif, jpg, png, etc).
  • Second, with the information gathered, you open up your image editing application and make the appropriate changes to your image.
  • Third, upload the image to the web server, and usually you will also want to add a title or description.

The steps above may seem trivial to advanced users, however they are still time consuming and complicated enough that they limit the number of users able to accomplish the task. Then there is the 'human factor', even if a user is capable of cropping and resizing images, it is still quite possible for the images to be incorrectly formatted, therefore producing undesired results.

Since these steps are repeated for each image, why not automate the process and let the web server do all of the work. Okay maybe not all of the work, since the images still need to be uploaded.

By placing the image processing on the server side, all of the preparation time, complication, and human error disappear. By reducing the process to just a couple of mouse clicks the number of users willing to upload images greatly increase. Also since the task becomes quick and easy, users will tend to use the feature more often.

Graphics and image processing is one of many areas where MS.Net excels in. Using the CLR's System.Drawing and System.Drawing.Imaging classes, it is quite straightforward to create an image processing class for uploaded images.

The Image processing class

The image processing class that is part of this article can easily take any uploaded image and transform it to the required dimensions, image format, and even add titles to the image. Although this class performs just the basics, it can be easily expanded to include other image processing features that would normally be done by hand.

The class itself is written in C#, but can be used by any .NET language since it is CLS (Cross Language Specification) compliant. The code shown inline as part of the article has been slightly simplified from the code found in the class, all of the extra error handling has been removed. The class is located in the ImageIt.cs file that is included along with this article.

The Image

At the heart of the class is an instance of the System.Drawing.Image class. This class is a container for an image and can load and save images in all common formats such as gif, jpeg, tiff, bmp, png, wmf, emf, exif, and icon.

private Image theImage;

Loading and Saving

The code below shows how an image is loaded and saved within the imaging class.

Private String imageFilename;

public
            void LoadImage(string filename)
{
if(theImage !=
            null)
theImage.Dispose();
theImage =
            Image.FromFile(filename);
imageFilename =
            filename;
}

public void SaveImage()
{
theImage.Save(imageFilename);
}

public void
            SaveImageAs(String
            filename)
{
theImage.Save(filename);
}

public void
            SaveImageAs(String filename,ImageFormat
            format)
{
theImage.Save(filename,format);
}

The Imaging class itself performs the bulk of the work in loading and saving an image. The only real work required when loading an image is to call the images Dispose method to ensure timely resource cleanup, in case the class previously contained an image. Although the garbage collector can free up the resources by itself, when it performs this task it cannot be guaranteed and may lead to low system resources. Also the filename is stored so that it can be referenced during future Save operations.

Save just calls the images own Save function, and saves the image using the original filename that was previously stored.

SaveAs has two overloaded functions. The first requiring just the filename to save the image as, and the second requiring a filename plus an image format (see the .NET class System.Drawing.Imaging.ImageFormat for more information on possible image formats). Two versions are required, since the CLS does not support optional parameters.

Once the image is loaded, .NET's built in drawing capabilities are used within this class to perform the required image processing.

Resizing

Resizing an image is a two-step process within this class. First a new image of the required size is created based off of the original image. Then the original image is disposed of and replaced by the new image.

Original Image
Example of resized image

public void Resize(int width, int
            height)
{
Size s = new Size(width,
            height);
Resize(s);
}

public void Resize(Size
            newSize)
{
Bitmap tempImage = new
            Bitmap(theImage,newSize);
theImage.Dispose();
theImage =
            tempImage;
}

The actual sizing of the image is done during the creation of the new image, since one of the options when creating an image is to base it off of an existing image. Also notice that the original image calls the Dispose method to ensure that the old images resources cleaned up. There are two versions of the Resize function, giving developers a choice of parameter types.

Cropping

Cropping an image is slightly more complicated than resizing, since this ability is not completely built into .NET's drawing capabilities. Cropping an image reduces an image's size by cutting away part of the image; it does not resize any of the original image content.

Original Image
Example of cropped image

public void Crop(int cropWidth,int cropHeight)

{
//only crop if the specified demensions
            are smaller
//than the original image
if(cropWidth >
            theImage.Width)
cropWidth = theImage.Width;
if(cropHeight >
            theImage.Height)
cropHeight = theImage.Height;

//calculate
            the crop rectangle
int xStart = (theImage.Width -
            cropWidth);
if(xStart != 0)
xStart /= 2;
int yStart =
            (theImage.Height - cropHeight);
if(yStart != 0)
yStart /=
            2;

Rectangle cropRect = new Rectangle(
xStart,
            yStart,
cropWidth, cropHeight);

//perform the actual
            crop
Crop(cropRect);
}

There are two versions of the Crop function, each taking different parameters. The first version of the Crop function, shown above, takes a width and height. These dimensions are then centered over the image and a crop rectangle is calculated. This calculated rectangle is then passed to the second version of Crop to perform the actual cropping of the image.

public void Crop(Rectangle cropRect)
{
//create the new image to crop the image into
Bitmap tempImage = new Bitmap( cropRect.Width,
cropRect.Height, theImage.PixelFormat);

Graphics g = Graphics.FromImage(tempImage);

g.DrawImage(theImage,
0,0,
cropRect,
GraphicsUnit.Pixel);

g.Dispose();
theImage.Dispose();
theImage = tempImage;
}

The second version of the Crop function takes a rectangle as a parameter. This allows for full control over the exact area of the image to be cropped.

To perform the actual cropping, a new image is created using the dimensions specified in cropRect, and the pixel format specified in the original image (pixel format refers to pixel, color depth, such as 256 color, 24-bit color, 32-bit color, etc.)

Next a graphics context is created on the new image. A graphics context is required to perform any and all drawing operations within .NET. R efer to .NET's System.Drawing.Graphics class for more information on graphics contexts. Since the graphics context was created from the image, all drawing will be performed directly on the image.

With the graphics context the original image is then drawn onto the new image, well, at least part of it. One of the parameters that the DrawImage function can take is a rectangle that specifies the area to draw. So instead of the whole image being drawn, only the piece within the rectangle co-ordinates is drawn.

Once the new cropped image is created, the graphics context is disposed of, and then the original image is disposed of and replaced by the newly cropped image.

Specialized Formatting

Although the crop and resizing functions give fine-grained control over the manipulation of an image, a specialized hybrid function, FitInto has also been added. This function will perform in a single step all of the cropping and resizing (enlarged or reduced) required to fit an image into a specified image height and width. Unless more sophisticated resizing and cropping is required, this function may be the only one you need to format your images to your desired sizes.

Two versions of this function are also available, a simple one and one with some more advanced features:

Original Image
Example using the cropping option

public void FitInto(int width, int height)

{
FitInto(width,height,ImageItOptions.None,Color.White);
}

public
            void FitInto(int width, int height, ImageItOptions options,
            System.Drawing.Color paddingColor)
{
int newWidth;
int
            newHeight;
float ResizeRatio;

float widthRatio =
            (float)theImage.Width / (float)width;
float heightRatio =
            (float)theImage.Height / (float)height;

//if the FitInto size
            is smaller, then shrink the image
if(width < theImage.Width ||
            height < theImage.Height)
{
//if the width difference is
            greater
if(widthRatio > heightRatio)
if(options ==
            ImageItOptions.Crop)
ResizeRatio =
            heightRatio;
else
ResizeRatio =
            widthRatio;
else
if(options ==
            ImageItOptions.Crop)
ResizeRatio =
            widthRatio;
else
ResizeRatio = heightRatio;
}
//else
            enlarge the image
else
{
//if the width difference is
            greater
if(widthRatio < heightRatio)
if(options ==
            ImageItOptions.Crop)
ResizeRatio =
            heightRatio;
else
ResizeRatio =
            widthRatio;
else
if(options ==
            ImageItOptions.Crop)
ResizeRatio =
            widthRatio;
else
ResizeRatio =
            heightRatio;
}

newWidth = (int)((float)theImage.Width /
            ResizeRatio);
newHeight = (int)((float)theImage.Height /
            ResizeRatio);

if(newHeight > height)
newHeight =
            height;
if(newWidth > width)
newWidth = width;

Size
            s = new Size(newWidth, newHeight);

if(options ==
            ImageItOptions.Crop)
{
//resize then crop
Resize(s);
Crop(width,height);
}
else if(options ==
            ImageItOptions.Pad)
{
//resize then
            pad
Resize(s);

Bitmap tempImage = new Bitmap( width,
            height,
theImage.PixelFormat);
Graphics g =
            Graphics.FromImage(tempImage);
SolidBrush fillBrush = new
            SolidBrush(paddingColor);
g.FillRectangle(fillBrush,0,0,width,height);
g.DrawImage(theImage,
(width
            - s.Width)/2,
(height - s.Height) /2,
s.Width,
s.Height);

g.Dispose();
theImage.Dispose();

theImage =
            tempImage;
}
else
{
//resize
Resize(s);
}
}

The first version only takes a width and height. It then passes these parameters to the second version, with no extra options specified.

The second version does all of the work and has a couple of options that specify how the image formatting is performed.

else
{
//resize
Resize(s);
}
}

public
            enum ImageItOptions

With no options specified (ImageItOptions.None), the image is sized so that it is as large as possible within the given width and height without changing the aspect ratio of the given image. Since the original aspect ratio is maintained, the final dimensions may not match that of the ones specified. For example if the original image were of size 200x100, after calling the FitInto method with 100x100, the output image would be 100x50.

The pad option (ImageItOptions.Pad) preserves the original aspect ratio of the image, while still completely filling the destination rectangle. Since the original image may not fill this entire area, the blank area will be padded with color specified in the paddingColor parameter.

Original Image
Example using padding option, with a gray padding color

With the crop option specified (ImageItOptions.Crop), the image is sized so that its width and height will match the specified width and height while maintaining its aspect ratio. However, in most cases image cropping will be performed to ensure that the new image is within the specified bounds.

ImageIt Usage

The ImageIt class is easy to use on any image file and in many cases only requires five lines of code.

Using DigitalIntersection;
ImageIt
            theImage;
theImage.Load("c:\myImage.jpg");
theImage.FitInto(200,200);
theImage.Save();

The above sample will take the image "myImage.jpg " and adjust it so that it will fit within the given 200x200 area, then save the adjusted image. To do this, first the namespace where the ImageIt class exists is specified. Then an instance of the class is created. Next the image is loaded, sized and finally saved again.

In other cases, where the original image is to be kept and a new thumbnail is to be created, code similar to below can be used.

ImageIt
            theImage;
theImage.Load("c:\myImage.jpg");
theImage.FitInto(150,180);
String
            newFilename = "thumb_" +
            theImage.Filename;
theImage.SaveAs(newfilename);

The example above will create a new image based on "myImage.jpg ", size it to fit within 150x180 and save it as "thumb_myImage.jpg ".

Other Issues

Besides image processing, there are a few other issues that arise when uploading images to a web server. First and most obvious is the upload itself, Second is file naming.

Uploading an image

Before ASP.NET uploading a file required a special upload control located on the server to receive the file and store it. Now, uploading files is no harder then using the well-known Request.Forms collection. Request.Files is the new collection in ASP.NET that holds a list of uploaded files.

There are two pieces required to upload a file, the HTML form for specifying the file to be uploaded and the server side script for processing the file.

<HTML>
<HEAD>
</HEAD>
<BODY>
<FORM id="form1" method="post" runat="server"
enctype="multipart/form-data">
<INPUT name="file1" type="file"> <INPUT type="submit"
value="Submit">
</FORM>
</BODY>
</HTML>

There are two special pieces to the HTML above. The encoding type for the form, enctype. And the input field type, file. The encoding type of "multipart/form-data " is required when a file is part of a form. If this is not specified then the Request.Files collection will not work. The input type of file is required so that the specified file will be uploaded along with the rest of the form data, otherwise only the text in the input box will be uploaded.

Once the file is loaded, its information is retrieved using the Request.Files collection.

System.Web.HttpPostedFile postedFile = Request.Files.Get("file1");
postedFile.SaveAs("c:\uploadedfile");

The code snippet above shows what is required on the server side to save an uploaded file. However there is one problem with the information returned; When requesting the filename, you receive the entire path of the file from the clients machine (for example, "c:\dir\myfile.exe"). So to get only the filename, parsing is required.

Naming uploaded files

As just previously mentioned, there is no direct way to retrieve just the filename of an uploaded file (HttpPostedFile.Filname returns the fully qualified path on the clients machine where the uploaded file came from). To do this parsing of the file path is required. To make it simple to extract, a routine is included with the source that is part of this article that will perform the parsing, ExtractFilename.

Besides retrieving the original filename there is one other major problem with uploading files, and that is file naming conflicts. What happens if two files of the same name are uploaded? Well the last file uploaded will be the winner, and the previous file will be overwritten. To prevent this common problem, uploaded files generally need to be given unique filenames. For uploading images this is not too big of a deal as long as the file extension is kept the same. Just as the filename has to be parsed from the path, so does the file extension. To make it easy to extract this as well, a routine for retrieving the file extension is also included, ExtractExtension.

Conclusion

The ImageIt class can easily turn the manual task of preparing images for use on a web site into a simple automated task. This is achieved by taking advantage of the powerful new graphics and imaging features that are part of the .NET CLR.

Although image processing tasks are generally CPU intensive, the performance of .NET's imaging classes are very impressive. In fact on a test server with a single processor 800Mhz Intel processor the ImageIt class was averaging just over 100 image manipulations per second. The test included loading an image from file, adjusting it using the FitInto routine, and then saving the resulting image.

To help you to get started with a sample ASPX page is included that uploads images and creates thumbnails of the images that have hyperlinks to the originals.

Class Reference

  • Namespace: DigitalIntersection
  • Class name: ImageIt
  • File name: ImageIt.cs

Properties:

  • Width: Type integer
  • Description: returns the width of the current image (read only property)
  • Height: Type integer
  • Description: returns the height of the current image (read only property)
  • Filename: Type String
  • Description: returns filename including the path of the current image (read only property)

Methods:

public void LoadImage(String filename)

Description: | Loads in an image from a file. Accepts the following image files: gif, jpeg, tiff, bmp, png, wmf, emf, exif, and icon
Parameters: | filename - fully qualified path to the image to load
Example: | ImageIt theImage = new ImageIt(); theImage.LoadImage("c:\\myImage.jpg"); theImage.FitInto(150,150); theImage.SaveImage();

public void SaveImage()
public void SaveImageAs(String filename)
public void SaveImageAs(String filename,ImageFormat format)

Description: | Saves the current image to a file. If the filename is not specified then the same filename used to load in the image is used.
Parameters: | filename - fully qualified path to the image to load format - image format to same the file as (gif, jpeg, tiff, bmp, png, wmf, emf, exif, and icon)
Example: | ImageIt theImage = new ImageIt(); theImage.LoadImage("c:\\myImage.jpg"); theImage.Resize(150,150); theImage.SaveImageAs("c:\\smallImage.tiff",ImageFormat.Tiff);

public void SetImage(Image newImage)

Description: | Sets the image to reference the specified existing image. This function is useful for performing image processing operations on an image that has already been loaded into a Image class. This function can be used instead of the Load function to retrieve an image.
Parameters: | NewImage - a reference to an existing image.
Example: | Image anImage; . ImageIt theImage = new ImageIt(); //resize an existing image theImage.SetImage(anImage); theImage.Resize(150,150);

public Graphics GetGraphics()

Description: | This function returns a graphics context for the current image, so that drawing operations can be performed on the image. See System.Drawing.Graphics for more information. Remember to call Dispose(); on the context when finished.
Parameters: | None
Return: | A reference to a graphics context
Example: | ImageIt theImage = new ImageIt(); Graphics g = theImage.GetGraphics(); Pen p = new Pen(Color.Red, 1) g.DrawLine(p,0,0,theImage.Width, theImage.Height);

public Image GetImage()

Description: | Returns a reference to the current image within the ImageIt class.
Parameters: | None
Return: | A reference to the current image
Example: | ImageIt theImage = new ImageIt(); theImage.LoadImage("c:\\myImage.jpg"); theImage.Resize(150,150); Image img = theImage.GetImage(); ImageFormat imgFmt = img.RawFormat;

public void Crop(int cropWidth,int cropHeight)
public void Crop(Rectangle cropRect)

Description: | Crops the current image to the specified dimensions. The first version only specifies the width and height to be cropped, so the cropped area is centered. The second version specifies all four co-ordinates of the cropping area.
Parameters: | CropWidth - width to crop the image to CropHeight - height to crop the image to CropRect - rectangle to crop the image to
Example: | ImageIt theImage = new ImageIt(); theImage.LoadImage("c:\\myImage.jpg"); theImage.Crop(theImage.Width / 2, theImage.Height / 2); theImage.Save();

public void FitInto(int width, int height)
public void FitInto(int width, int height, ImageItOptions options, System.Drawing.Color paddingColor)

Description: | Resizes and/or crops the current image so that it will fit within the given bounds. How the cropping and resizing are performed are based on the functions options. None - the image size is maximized within the given bounds while maintaining its aspect ratio. The resulting image may be smaller than the requested size. Crop - the image is size is maximized within or beyond the given bounds so that it will fill the entire area. Any part of the image outside of the bounds is then cropped. Pad - the image size is maximized within the given bounds while maintaining its aspect ratio. The space where the resulting image does not cover is filled with the specified color.
Parameters: | Width - bounding width to fit the image within Height - bounding height to fit the image within Options - resizing and cropping options Color - color used when padding the resulting image
Example: | ImageIt theImage = new ImageIt(); theImage.LoadImage("c:\\myImage.jpg"); theImage.FitInto(150,150, ImageItOptions.Pad, Color.White); TheImage.SaveAs("c:\\thumb.jpg");

public void Resize(int width, int height)
public void Resize(Size newSize)

Description: | Resizes the current image to the specified size. Aspect ratio is not maintained.
Parameters: | Width - width to resize the image to Height - height to resize the image to NewSize - specifies the width and height to resize the image to
Example: | ImageIt theImage = new ImageIt(); theImage.LoadImage("c:\\myImage.jpg"); theImage.Resize(100,500); TheImage.SaveAs("c:\\stretch.jpg");

Enumerations:

public enum ImageItOptions

Description: | The ImageItOptions enumeration is used by the FitInto functions, to determine how the image is resized and/or cropped. See FitInto for more information.
Values: | None - no cropping or padding is performed when used. Only a resize occurs. Pad - specifies that image padding is to be used after the image is resized. Crop - specifies that the image may be cropped in order to excatly fill the area specified.
  • Namespace: DigitalIntersection
  • Class name: FileUtil
  • File name: FileUtil.cs

Methods:

public String ExtractFilename(String filePath)

Description: | Extracts and returns the filename portion from a fully qualified path.
Parameters: | path - fully qualified path to extract the filename from
Return: | Filename portion of the path

public String ExtractFileExt(String filename)

Description: | Extracts and returns the file extension portion from a filename or a fully qualified path.
Parameters: | filename - filename to extract the file extension from
Return: | File extension

public String GetUniqueName()

Description: | Returns a unique name. The generated name is based off of a GUID therefore making it a web-farm safe way to uniquely name files even if they are being stored on the same storage device.
Parameters: | None
Return: | A unique filename


Printing Using C#

Printing Using C#
http://www.csharptoday.com/content.asp?id=1611
Article Rating: 4
Published on July 23, 2001
Abstract
Those of you who have done any programming with the Win32 API will be familiar with the concept of the Device Context, also known as the “DC”. A Device Context allows a developer to draw things like text, lines, rectangles, curves, etc. on both the screen and the printer, or even in memory in preparation for later drawing to either of these devices. This was a pretty powerful arrangement that allowed a developer who knew how to draw application output on the screen to draw application output on a printer. With .NET, we’re given a variety of graphics classes that eventually allow drawing of certain graphics primitives onto device contexts. In this article, Matthew Reynolds describes how we can build an application that is capable of printing its output on a printer installed on the computer, allow the adjustment of page and printer settings and also provide a print preview.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

Those of you who have done any programming with the Win32 API will be familiar with the concept of the Device Context, also known as the "DC". A Device Context allows a developer to draw things like text, lines, rectangles, curves, etc. on both the screen and the printer, or even in memory in preparation for later drawing to either of these devices. This was a pretty powerful arrangement that allowed a developer who knew how to draw application output on the screen to draw application output on a printer.

With .NET, we're given a variety of graphics classes that eventually allow drawing of certain graphics primitives onto device contexts. This article describes how we can build an application that is capable of printing its output on a printer installed on the computer, allow the adjustment of page and printer settings and also provide a print preview.

Creating a Printing Application

Part of the difficulty of printing from an application is that we lose the concept of using controls to present our program's output to the user. To clarify, with a VB.NET or C# application, if we're building a UI we usually do so by painting controls on the screen and allowing whoever developed them to worry about the whys and wherefores about how that control should look. We don't really care how many individual DrawLine calls it takes to draw a button on the screen.

With printing, you lose this control-oriented abstraction and you have to get back to worry about where to draw lines, selecting fonts, drawing text and so on. This makes life difficult for developers maintaining and enhancing the application. Imagine you have an application that prints simple reports and in this application you have a class called Customer that represents a customer. As part of the report, you want to print all of the customer's details. What the developer wants is a really easy to use framework where she can just say, "Now print the customer's name" or "Print the URL of the customer's home page". She doesn't want to have to worry about where exactly on the page to position the text, or which font to use, just like as a UI developer using controls you don't care about how to find out the default color for the button.

In this article we're going to build a fairly powerful printing framework that can be easily enhanced at a later date. We're going to provide the developer with such elements as print previews, printer settings and we're also going to take over all the hard work of pagination, adding headers and footers and so on.

(For the uninitiated, "pagination" is the process of splitting up a document into pages and laying out each page so that it matches the layout that the user intended.)

Objects, Elements and Primitives

Let's assume that our application has access to objects that describe things, such as Customer, Company, Order and Product. Let's also assume that our reports are going to be fairly simple lists of these object, e.g. "Give me a list of all customers who have ordered from me today." Our report might look like this:

One way to break up this task is to say that we're going to build a page up of different "elements". In the above screenshot, we have three elements:

The elements we have, "Header", "Customer" and "Footer", are all comprised of different "primitives." In this case, we have two kinds of primitives: "text" and "horizontal rule".

So, to build up our application we first of all need an object that describes the print functionality. We'll call this PrintEngine and this object will be responsible for laying out the elements on the page and choosing when new pages need to be added to the report. Each element will be represented by a PrintElement object. This PrintElement object will know how to contain primitives, specifically PrintPrimitiveText and PrintPrimitiveRule objects. It will be these objects that know how to draw themselves on the page and report back to PrintEngine as to their size. (In order for PrintEngine to lay out the elements, it needs to know how large each element is. To determine an elements size, the primitives have to be able to report on their size.)

Now that we know what we're trying to achieve, let's look at building an application capable of printing. We'll build the printing components of this application in such a way that we could extract them and reuse them in another project.

Creating the Project

Create a new Visual C# - Windows Application project now and call it Printing. Add these controls to Form1:

Here are the controls:

  • trackCustomers - this slider bar should have a minimum value of 0 and a maximum value of 128. Set Value to 1.
  • cmdPrintPreview - the print preview button.
  • cmdPageSettings - the page settings button.
  • cmdPrint - the print button.

That's all there is to the form. Let's now look at building the rest of the classes.

"PrintEngine"

The Framework provides a class called PrintDocument that encapsulates everything related to printing. Those of you who have used MFC, don't worry too much about the word "document". There's nothing related to the "document/view" model you might be thinking of. The "document" is simply something that you want to print.

To use this class you have two options: either respond to events fired by the class or inherit from the class and build upon the functionality. In this instance, we're going to inherit from it. Create a new class called PrintEngine and add these using statements:

using System;
using System.Collections;
using System.Drawing;
using System.Drawing.Printing;
using System.Windows.Forms;

Next, derive PrintEngine from PrintDocument:

public class PrintEngine : PrintDocument

This project is a little "chicken and egg", so we'll be jumping between classes as we try and build up the example. Perhaps the best place to start is maintaining a list of the objects that we want to include on the report.

As we build the report, we'll add Customer objects to an ArrayList maintained in PrintEngine. Whenever we want to print the report, or display a preview, we'll use methods to add objects that understand how to print themselves to this ArrayList.

Add this member to PrintEngine:

public class PrintEngine : PrintDocument
{
private ArrayList _printObjects = new ArrayList();

In a moment, we'll build an interface called IPrintable that an object must support if it wants to be printed by PrintEngine. Add this method to PrintEngine that accepts an object supporting this interface and adds it to _printObjects:

// AddPrintObject - add a print object the document...
public void AddPrintObject(IPrintable printObject)
{
_printObjects.Add(printObject);
}

Now, create a new class called IPrintable and add this code:

using System;

namespace
            Printing
{
public interface IPrintable
{
//
            Print
void Print(PrintElement element);
}
}

As you can see, our interface only has a single method: Print. You'll also notice that it takes a parameter of PrintElement type. We haven't built that yet, but it will represent a single instance of an element on a page. Bear with me until we get to it!

To complete this circle, create a new object called Customer. Add these members:

   public class Customer :
            IPrintable
{
// members...
public int Id;
public String
            FirstName;
public String LastName;
public String
            Company;
public String Email;
public String
            Phone;
}

As we've told Customer that it must support IPrintable, we need to implement the method. However, we won't build any print functionality in just yet. Add this method to Customer:

// Print...
public void Print(PrintElement element)
{
}

OK, so we're now at a point where we can create instances of a class called Customer that, because it implements IPrintable, we can use PrintEngine.AddPrintObject to add it to the final document. Let's look now at building up the primitives and then take a look at implementing the elements themselves.

Primitives and Elements

We're only going to implement two kinds of primitives in this exercise: one that prints text, and one that prints a horizontal line. This will let us build up a basic report.

A primitive has to be able to do two things. It must be able to determine how much space it will take up on the page and it must be able to draw itself on whatever graphics context we've been given. This will either be on the printer, or alternatively will be on the screen for a print preview - although we won't actually care which is being used when we're asked to draw ourselves.

Our two primitive classes will both implement IPrintPrimitive. This new interface will define the two methods used to measure and draw the primitives. Create a new class called IPrintPrimitive and add this code:

using System;
using System.Drawing;

namespace Printing
{
public interface IPrintPrimitive
{
// CalculateHeight - work out how tall the primitive is...
float CalculateHeight(PrintEngine engine, Graphics graphics);

// Print - tell the primitive to physically draw itself...
void Draw(PrintEngine engine, float yPos, Graphics graphics, Rectangle elementBounds);
}
}

Let's take a look at PrintPrimitiveRule, the class responsible for drawing a horizontal rule. Create a new class called PrintPrimitiveRule and set the class to implement IPrintPrimitive:

public class PrintPrimitiveRule : IPrintPrimitive

This primitive is always going to be five drawing units high, so add this method that will tell anyone who asks that we're five units high:

// CalculateHeight - work out how tall the primitive is...
public float CalculateHeight(PrintEngine engine, Graphics graphics)
{
// we're always five units tall...
return 5;
}

When we come to draw the primitive, we're going to provide it with details of the position on the page where it should draw itself. elementBounds describes a rectangle that encloses the entire element. yPos describes the current y-coordinate that should be used for drawing. These two in combination tell the primitive where to draw itself, which it can do by using methods on the supplied System.Drawing.Graphics object. Add this method to PrintPrimitiveRule that will draw a line two drawing units down from where the primitive is supposed to start:

// Print - draw the rule...
public void Draw(PrintEngine engine, float yPos, Graphics graphics, Rectangle elementBounds)
{
// draw a line...
Pen pen = new Pen(engine.PrintBrush, 1);
graphics.DrawLine(pen, elementBounds.Left, yPos + 2,
elementBounds.Right, yPos + 2);
}

The PrintPrimitiveText is very similar. Create the new class now and add the same namespace declarations as before:

using System;
using System.Drawing;

Then, tell the class to inherit IPrintPrimitive:

public class PrintPrimitiveText : IPrintPrimitive

The class will also need a String member that contains the text that need to be printed. We'll change the constructor of the class so that we need to supply the text whenever we create one of the objects:

      //
            members...
public String Text;

public
            PrintPrimitiveText(String buf)
{
Text = buf;
}

In a short while we'll add a member to PrintEngine called PrintFont. This will contain a reference to a System.Drawing.Font object that will be used to draw text on the report. We can use the Font object to measure the height of the primitive, although we're going to assume that a single primitive cannot span more than one line:

// CalculateHeight - work out how tall the primitive is...
public float CalculateHeight(PrintEngine engine, Graphics graphics)
{
// return the height...
return engine.PrintFont.GetHeight(graphics);
}

In a similar manner to drawing the line, we can use the Graphics object to draw the text:

// Print - draw the text...
public void Draw(PrintEngine engine, float yPos, Graphics graphics,
Rectangle elementBounds)
{
// draw it...
graphics.DrawString(engine.ReplaceTokens(Text), engine.PrintFont,
engine.PrintBrush, elementBounds.Left, yPos, new StringFormat());
}

The PrintEngine.ReplaceTokens will be used to change the value of special fields that we can place in the text. This will let us create a header that automatically contains the page number, and we could extend this to include things like the current date time, user, computer name and so on. We'll seem them in action later on.

This brings us to the end of building the two primitives, so let's now look at the relationship between PrintElement and classes implementing IPrintPrimitive.

Building "PageElement"

The PrintElement class describes an element that has to be rendered on the report. Simply, it's a list of primitives together with a collection of method that makes life easier for PrintEngine and for the other classes like Customer.

Create a new class called PrintElement and add these using statements:

using System;
using System.Collections;
using System.Drawing;
using System.Drawing.Printing;

We'll need a member for holding a list of primitives, and also a member for holding a reference to an object supporting IPrintable:

public class PrintElement
{
// members...
private ArrayList _printPrimitives = new ArrayList();
private IPrintable _printObject;

public PrintElement(IPrintable printObject)
{
_printObject = printObject;
}

When we want to print the element (i.e. we want to create a list of primitives that we can then ask each of them to draw themselves on the screen or printer) we'll call a method called Print. This will simply call through _printObject to whatever object is underneath it, in this case Customer.

When Customer is asked to print it will call methods like PrintElement.AddText and PrintElement.AddHeader. Let's look at the first of these now:

// AddText - add text to the element...
public void AddText(String buf)
{
// add the text...
AddPrimitive(new PrintPrimitiveText(buf));
}

The AddPrimitive simply adds an object supporting IPrintPrimitive to the _printPrimitives list. We'll define this as a public method so that if anyone wants to build more objects that support IPrintPrimitive, or inherit from objects that already do, they'll be able to extend the framework.

      //
            AddPrimitive - add a primitive to the list...
public void
            AddPrimitive(IPrintPrimitive primitive)
{
// add
            it...
_printPrimitives.Add(primitive);
}

Here are some more methods that let the developer build up the primitives that make up the element:

      //
            AddData - add data to the element...
public void AddData(String
            dataName, String dataValue)
{
// add this data to the
            collection...
AddText(dataName + ": " +
            dataValue);
}

// AddHorizontalRule - add a rule to the
            element...
public void AddHorizontalRule()
{
// add a rule
            object...
AddPrimitive(new PrintPrimitiveRule());
}

//
            AddBlankLine - add a blank line...
public void
            AddBlankLine()
{
// add a blank
            line...
AddText("");
}

// AddHeader - add a
            header...
public void AddHeader(String
            buf)
{
AddText(buf);
AddHorizontalRule();
}

The PrintEngine is going to need to efficiently calculate the height of the element. It will do this through a call to CalculateHeight and this method simply aggregates the results for calling CalculateHeight on each of the primitives.

      public
            float CalculateHeight(PrintEngine engine, Graphics
            graphics)
{
// loop through the print height...
float
            height = 0;
foreach(IPrintPrimitive primitive in
            _printPrimitives)
{
// get the height...
height +=
            primitive.CalculateHeight(engine, graphics);
}

// return
            the height...
return height;
}

Finally, the element is going to need to draw itself. This is just a case of iterating through the primitives and asking each one to draw themselves. As part of this job, the element needs to calculate a rectangle that bounds the element. When we call PrintElement.Draw we'll provide a rectangle that describes the area of the page that can be printed on, saving for the space required for the header and the footer.

As we mentioned, yPos describes the top y-coordinate of where drawing should be done. As we move through each primitive, we move yPos down to the bottom of the last primitive we drew. (We're not going to allow primitives to overlap, but there's no reason why would couldn't add this functionality to your own implementation.) PrintEngine is going to handle the pagination, so we don't need to worry about whether or not we can fit the element onto the page.

// Draw - draw the element on a graphics object...
public void Draw(PrintEngine engine, float yPos, Graphics graphics,
Rectangle pageBounds)
{
// where...
float height = CalculateHeight(engine, graphics);
Rectangle elementBounds = new Rectangle(pageBounds.Left, (int)yPos,
pageBounds.Right - pageBounds.Left, (int)height);

// now, tell the primitives to print themselves...
foreach(IPrintPrimitive primitive in _printPrimitives)
{
// render it...
primitive.Draw(engine, yPos, graphics, elementBounds);

// move to the next line...
yPos += primitive.CalculateHeight(engine, graphics);
}
}

To round of this section, let's see how we can change Customer so that the primitives are created.

Printing Customer Details

Go back to Customer.cs and add this code to Print:

// Print...
public void Print(PrintElement element)
{
// tell the engine to draw a header...
element.AddHeader("Customer");

// now, draw the data...
element.AddData("Customer ID", Id.ToString());
element.AddData("Name", FirstName + " " + LastName);
element.AddData("Company", Company);
element.AddData("E-mail", Email);
element.AddData("Phone", Phone);

// finally, add a blank line...
element.AddBlankLine();
}

As you can see, the work that the developer has to do to get an object to support printing in a report is pretty minimal. She never has to worry about things like pagination, fonts or layout. Instead, she just called methods on PrintElement that add primitives to the page.

Pagination and Printing

Now we're at a point where we have an object called Customer that can describe itself on a report by calling methods on an object called PrintElement. Now all we have to do is arrange the elements on the page, handle pagination, deal with headers and footers, provide print preview functionality and actually send the data to the printer. Thanks to the work that's been done with the Framework, this is perhaps the easiest part of the exercise.

More Members

We're going to need to add more members to the PrintEngine class:

public class PrintEngine : PrintDocument
{
// members...
private ArrayList _printObjects = new ArrayList();
public Font PrintFont = new Font("Arial", 10);
public Brush PrintBrush = Brushes.Black;
public PrintElement Header;
public PrintElement Footer;
private ArrayList _printElements;
private int _printIndex = 0;
private int _pageNum = 0;

The PrintFont and PrintBrush will be used to define the font and brush that will be used to do the printing. In this case, we're saying that we want to use a ten point Arial font, in black.

The Header and Footer describe elements that contain the header and footer for each page. _printElements describes a list of all the elements, except Header and Footer. _printIndex keeps track of the current position in the _printElements list. Finally, _pageNum keeps track of the current page number.

The first thing to do before we try to print is build up the header and footer elements. We'll create default ones in the constructor:

      public
            PrintEngine()
{
// create the header...
Header = new
            PrintElement(null);
Header.AddText("Report");
Header.AddText("Page:
            [pagenum]");
Header.AddHorizontalRule();
Header.AddBlankLine();

//
            create the footer...
Footer = new
            PrintElement(null);
Footer.AddBlankLine();
Footer.AddHorizontalRule();
Footer.AddText("Confidential");
}

The easiest way to develop printing routines is to get the print preview working. This saves a lot of otherwise wasted paper! Luckily for us, the Framework provides a class that containerizes an instance of PrintPreviewControl. Add this method to PrintEngine:

      //
            ShowPreview - show a print preview...
public void
            ShowPreview()
{
// now, show the print
            dialog...
PrintPreviewDialog dialog = new
            PrintPreviewDialog();
dialog.Document = this;

// show the
            dialog...
dialog.ShowDialog();
}

As you can see, PrintPreviewDialog needs to be given a PrintDocument object, so we give it a reference to the PrintEngine object that was used. We then call ShowDialog.

The printing routines have a fairly curious way of working, which I mentioned right at the beginning involved firing events to signal when a page needs to be printed. The first event that gets fired is BeginPrint, but as PrintEngine is inherited from PrintDocument the best way to get at this is to override OnBeginPrint. This is, I'm told, similar to the way that the printing routines in MFC worked.

What we need to do in response to this event is create a new list of PrintElement objects - one for each of the objects we have in our _printObjects list. Add this code to PrintEngine:

      //
            OnBeginPrint - called when printing starts
protected override
            void OnBeginPrint(PrintEventArgs e)
{
//
            reset...
_printElements = new ArrayList();
_pageNum =
            0;
_printIndex = 0;

// go through the objects in the list
            and create print elements for each one...
foreach(IPrintable
            printObject in _printObjects)
{
// create an
            element...
PrintElement element = new
            PrintElement(printObject);
_printElements.Add(element);

//
            tell it to print...
element.Print();
}
}

As we work through the list we create a new PrintElement object for each IPrintable -supporting object that we have. We then ask each element to print itself through the Print method. At this time, the element will call into IPrintable.Print and the underlying class will tall the various AddText, AddHeader, AddHorizontalRule methods to build up the primitives.

Once we've done that, we can turn our attention to the printing mechanism. This is done through OnPrintPage. Add this code:

// OnPrintPage - called when printing needs to be done...
protected override void OnPrintPage(PrintPageEventArgs e)
{
// adjust the page number...
_pageNum++;

The first thing we do is increment the page number. In OnBeginPrint we set this to 0, meaning that on the first page it will be 1. We then want to draw the header element. This will always appear in the top-left hand corner of each page.

The PrintPageEventArgs contains members that tell us everything we need to know about printing. MarginBounds describes a rectangle containing the printable area of the page. Graphics contains a System.Graphics.Drawing object that contains the methods necessary for drawing on the screen or printer. We pass both of these into the header element:

// now, render the header element...
float headerHeight = Header.CalculateHeight(this, e.Graphics);
Header.Draw(this, e.MarginBounds.Top, e.Graphics, e.MarginBounds);

Drawing the footer is a similar deal, except this has to appear at the bottom of the page:

// also, we need to calculate the footer height...
float footerHeight = Footer.CalculateHeight(this, e.Graphics);
Footer.Draw(this, e.MarginBounds.Bottom - footerHeight, e.Graphics, e.MarginBounds);

Once we've done that, we need to calculate the area of the page that's left. This will contain the elements, or rather as many elements as will fit onto the page.

// now we know the header and footer, we can adjust the page bounds...
Rectangle pageBounds = new Rectangle(e.MarginBounds.Left,
(int)(e.MarginBounds.Top + headerHeight), e.MarginBounds.Width,
(int)(e.MarginBounds.Height - footerHeight - headerHeight));
float yPos = pageBounds.Top;

Right at the end there we define and set yPos to be the top of the printable area given over to elements. We'll see how this moves down the page as we draw elements.

The _printIndex keeps track of which element has to be drawn next. In OnBeginPage we set this to 0 meaning that the next element we draw will be the first one in _printElements. To loop through the elements, we need to do this:

         //
            ok, now we need to loop through the elements...
bool morePages =
            false;
int elementsOnPage = 0;
while(_printIndex <
            _printElements.Count)
{
// get the element...
PrintElement
            element = (PrintElement)_printElements[_printIndex];

Once we have the element, we need to find its height. Taking into consideration the current position of yPos, we want to make sure that the element will fit on the page:

            //
            how tall is the primitive?
float height =
            element.CalculateHeight(this, e.Graphics);

// will it fit on
            the page?
if(yPos + height > pageBounds.Bottom)
{
// we
            don't want to do this if we're the first thing on the
            page...
if(elementsOnPage != 0)
{
morePages =
            true;
break;
}
}

If the element will not fit on the page, we quit the loop - and we'll see why in a moment. One important thing to note here is that if this is the first element on the page (i.e. elementsOnPage = 0 ), we never want to leave the loop. If it's the first element and it's the first page, it's too big to fit on any page. Rather than breaking the element up, we'll just render it as it is, meaning that the bottom of the element will draw over the footer.

Providing the element will fit, we can tell the element to draw itself, move yPos down to the position of the next element and adjust _printIndex so that it points to the next element in the list:

            //
            now draw the element...
element.Draw(this, yPos, e.Graphics,
            pageBounds);

// move the ypos...
yPos += height;

//
            next...
_printIndex++;
elementsOnPage++;
}

So what happens if we have more pages? Well, we have to set PrintPageEventArgs.HasMorePages to true. This will result on OnPrintPage being called a second time, in which case we go round the whole thing again:

// do we have more pages?
e.HasMorePages = morePages;
}

Replacing Tokens

When we built our header element we added a line like this:

Header.AddText("Page: [pagenum]");

The [pagenum] represents a token that we want to replace at print time with a value of some kind, in this case the current page number. When we created the PrintPrimitiveText class, we had this calling into PrintEngine.ReplaceTokens before the text was sent to the printer. Add this method to PrintEngine:

      //
            ReplaceTokens - take a string and remove any tokens replacing them
            with values...
public String ReplaceTokens(String buf)
{
//
            replace...
buf = buf.Replace("[pagenum]",
            _pageNum.ToString());

// return...
return
            buf;
}

Although we've just shown how to replace [pagenum] here, the same will work with other tokens that you define.

Calling "ShowPrintPreview"

To call ShowPrintPreview, we actually need something to print. Add this member to Form1:

public class Form1 : System.Windows.Forms.Form
{
// members...
PrintEngine _engine = new PrintEngine();

In response to changing the slider value, we want to add Customer objects to _engine. Add this line to Form1 's constructor:

      public
            Form1()
{
//
// Required for Windows Form Designer
            support
//
InitializeComponent();

// create a default
            print engine...
CreateEngine(1);
}

Now, add this definition for CreateEngine:

      //
            CreateEngine - create a print engine and populate it with
            customers...
public void CreateEngine(int
            numCustomers)
{
// create a new engine...
_engine = new
            PrintEngine();

// loop through the customers...
for(int n
            = 0; n < numCustomers; n++)
{
// create the
            customer...
Customer theCustomer = new
            Customer();
theCustomer.Id = n + 1;
theCustomer.FirstName =
            "Darren";
theCustomer.LastName = "Clarke";
theCustomer.Company
            = "Madras inc.";
theCustomer.Email =
            "darren@pretendcompany.com";
theCustomer.Phone = "602 555
            1234";

// add the customer to the
            list...
_engine.AddPrintObject(theCustomer);
}
}

There's nothing very complicated about this. All we want to do is create Customer objects and add them to the engine.

Next, add this event handler to make calls to CreateEngine whenever the user changes the slider value:

private void trackCustomers_Scroll(object sender, System.EventArgs e)
{
CreateEngine(trackCustomers.Value);
}

Finally, we can wire up cmdPrintPreview_click:

private void cmdPrintPreview_Click(object sender, System.EventArgs e)
{
// tell the print object to display a preview...
_engine.ShowPreview();
}

Now we can try running the application and testing the print preview. Leave the slider where it is and click the button. You should see something like this:

Zooming in, you should see this:

Now, close the print preview and change the slider to about a third of the way along. This will create a bundle of Customer objects and you should see more pages on the preview.

That's the basics of printing illustrated. You can actually print the document if you want by clicking the printer button on the toolbar.

Changing the Page Settings

When printing, you might want to change some of the page settings. You might want to use a different page size or a landscape orientation.

The Framework provides a class that lets you change the layout of the page. The PrintDocument class has a property called DefaultPageSettings. This contains the current settings for the page, and when the object is created this is populated with the default page settings for the default printer.

Add this method to PrintEngine:

      //
            ShowPageSettings - let's us change the page settings...
public
            void ShowPageSettings()
{
PageSetupDialog setup = new
            PageSetupDialog();
PageSettings settings =
            DefaultPageSettings;
setup.PageSettings = settings;

//
            display the dialog and,
if(setup.ShowDialog() ==
            DialogResult.OK)
DefaultPageSettings =
            setup.PageSettings;
}

This method will take the current page settings from the dialog and invite the user to change them. If the user presses OK on the dialog, the changes settings are saved.

Add this event handler to Form1 to finish this off:

private void cmdPageSettings_Click(object sender, System.EventArgs e)
{
// show the page settings...
_engine.ShowPageSettings();
}

If you try this, remember that if you change the slider you'll lose the page settings. (i.e. if you star the application check the print preview, close it down, change the page settings, change the slider and then view the print preview, the page settings will not be saved as a new instance of PrintEngine would have been created.) You should see something like this:

Printing Without the Preview

To print to another printer, you have to use the standard print dialog. We can bring this up in much the same way, so add this method to PrintEngine:

      //
            ShowPrintDialog - display the print dialog...
public void
            ShowPrintDialog()
{
// create and show...
PrintDialog
            dialog = new PrintDialog();
dialog.PrinterSettings =
            PrinterSettings;
dialog.Document =
            this;
if(dialog.ShowDialog() == DialogResult.OK)
{
// save
            the changes...
PrinterSettings =
            dialog.PrinterSettings;

// do the
            printing...
Print();
}
}

Again, if the user clicks OK we save the settings, but this time we also call Print to print directly to the configured printer. Add this event handler to Form1 to finish this off:

      private
            void cmdPrint_Click(object sender, System.EventArgs e)
{
//
            print...
_engine.ShowPrintDialog();
}

Summary

In this exercise, we built a fairly complete example of how to build a C# desktop application with print capabilities. We architected the solution using a flexible combination of elements and primitives meaning that developers wishing to extend the printing functionality down the line would be able to do so easily.


Polymorphism in C# - Creating a Drawing Program

Polymorphism in C# - Creating a Drawing Program
http://www.csharptoday.com/content.asp?id=1606
Article Rating: 3.5
Published on July 19, 2001
Abstract
Polymorphism is a very broad term and has several meanings. However, in the object–oriented programming world, when people talk about polymorphism, they generally mean the concept of one interface with many implementations. To be more specific, the term is used to describe reference variables that may at run–time refer to objects of different classes. In this article Budi Kurniawan shows us what all the buzzwords are, why polymorphism is useful, and how to write a drawing application that illustrates the use of polymorphism.
Article

Polymorphism is a very broad term and has several meanings. However, in the object-oriented programming world, when people talk about polymorphism, they generally mean the concept of one interface with many implementations. To be more specific, the term is used to describe reference variables that may at run-time refer to objects of different classes. In this article you will learn what all the buzzwords are, why polymorphism is useful, and how to write a drawing application that illustrates the use of polymorphism.

Polymorphism is one of the three most important principles in object-oriented programming, the other two being encapsulation and inheritance. For those new to object-oriented programming, polymorphism is unfortunately not an easy concept. Therefore, to truly understand polymorphism, you need to know the different types of polymorphism and which type is normally referred to when this word is mentioned in object-oriented programming discussions or in C# language books.

There are two broad types of polymorphism: universal polymorphism and ad-hoc polymorphism.

  • Universal polymorphism is considered purer than ad-hoc polymorphism and is also called "true polymorphism". It allows a value to have not only many different types, but also an infinite number of them.
  • Ad-hoc polymorphism is less pure than universal polymorphism and can be regarded as "apparent polymorphism". It allows a value to have only a finite set of different types, not an infinite number.

Universal polymorphism is subdivided into parametric polymorphism and inclusion polymorphism; and ad-hoc polymorphism can be subdivided into overloading and coercion. The four types of polymorphism are explained below:

  • parametric polymorphism: this is the purest form of polymorphism, and the one most people mean in an object-oriented context. The same object or function can be used uniformly as a parameter in different type contexts without changes. A function can be thought as parametric polymorphic if it allows the type of its parameters to be determined by an implicit or explicit type parameter, and then performs the same kind of work irrespective of the parameter type.
  • inclusion polymorphism is what we know as subtyping and inheritance. This type of polymorphism allows an object to belong to a number of classes, which are sometimes overlapping. In other words, a class can be included in another class to create an inheritance hierarchy.
  • overloading is what is known as method overloading in object-oriented programming languages. It allows multiple functions to have the same name, yet still perform different functions.
  • coercion allows a parameter to a function to be converted to the type expected by that function to avoid a type error.

For those of you who like to dig deeper, you can start with the following excellent article at http://www.cs.ucl.ac.uk/staff/L.Braine/polymorphism.html

Polymorphism in C#

In object-oriented programming in general, when people talk about polymorphism, it is parametric polymorphism that they mean. In the sense of "one interface, many implementations", polymorphism has various applications. In this article, you will find two. The first application is a simple code that will introduce polymorphism. The second is a Windows drawing application.

Simple Example of Polymorphism

Suppose you are writing a word processing program that has a Print function. In the real world, there are many types of printers from many different manufacturers. How do you write the Print function so that it can print to many different printers? For this case, let's pretend that the current sophisticated operating systems do not yet exist. Without polymorphism, you will have to write the implementation code for each printer. As a new printer arrives and needs to be supported, you have to re-compile your word processing program again to include function that can print to the new printer. What an endless pain!

With polymorphism, however, you can just write an interface called IPrinter with a few methods. Each printer can then implement its own class that implements IPrinter. When a new printer comes, all you need to do is provide a class for the new printer that implements the same interface IPrinter. In fact, this interface acts as a contract between your word processor and any Print class implementation. To prove this concept, consider the code in Listing 1.

Listing 1: A simple polymorphism example in C#

using System;
using
            System.Text;

interface IPrinter {
void Print(String
            s);
}

class CannonPrinter: IPrinter {
public void
            Print(String s) {
Console.WriteLine("Printed by Cannon:" +
            s);
}
}

class EpsomPrinter: IPrinter {
public void
            Print(String s) {
Console.WriteLine("Printed by Epsom:" +
            s);
}
}

public class WordProcessor {
public static
            void Main(String[] args) {
if (args.Length==0)
            {
Console.WriteLine("Usage: WordProcessor <printer
            name>");
return;
}

String message = "Using
            polymorphism in C#";
IPrinter p = null;

if
            (args[0].Equals("Cannon")) {
p = new
            CannonPrinter();
}
else if (args[0].Equals("Epsom")) {
p =
            new EpsomPrinter();
}

if (p != null)
p.Print(message);
}
}

The IPrinter interface in Listing 1 has only one method: print. This method accepts the message to be printed by the printer. The IPrinter interface is the contract between the WordProcessor class (the application) and the printer manufacturers. Any printer manufacturer who wants its printer to be useable from WordProcessor must implement this interface. In the example, two printers, CannonPrinter and EpsomPrinter, implement IPrinter. Cannon writes the CannonPrinter class that implements IPrinter, and Epsom writes the EpsomPrinter class implementing IPrinter. Both classes provide implementations of the Print method. For the sake of simplicity, the Print method simply outputs the message to the console, preceded by the printer name.

To use a printer from WordProcessor, you pass the printer name as an argument to the Main method. If the argument is "Cannon", p will be instantiated with a CannonPrinter object. If the argument is "Epsom", p will be instantiated with an EpsomPrinter object. The variable p is actually a reference of type IPrinter. See how p can be assigned other types of objects? It could be a CannonPrinter or it could be an EpsomPrinter. In essence, it could be any object that implements IPrinter. You have just seen polymorphism in action!

This phenomenon is not illogical at all. Both CannonPrinter and EpsomPrinter implements IPrinter. You can think of implementing an interface kind of extending a class. So, here we have an "is a" relationship between CannonPrinter and EpsomPrinter and IPrinter. In other words, CannonPrinter "is an" IPrinter. EpsomPrinter is also an IPrinter. Because of this, it is logical to "put" either CannonPrinter or EpsomPrinter into p, because both CannonPrinter and EpsomPrinter are IPrinter. This is an example of one interface with many implementations.

The Shape Draw Application

The following example is a simple C# Windows application whose only purpose in life is to demonstrate polymorphism. The application enables you to draw multiple shapes by entering the two coordinates and selecting the type of shape you want. The EXE file of this program is included in the download. To run the source code of this application, you need Visual Studio.NET beta, that you can download or order from http://msdn.microsoft.com/vstudio/nextgen/beta.asp?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq. There are three types of shapes: Line, Rectangle and Ellipse. Every shape is encapsulated in a class and each shape implements the IShape interface.

The code for the interface and the shape classes is given in Listing 2.

Listing 2: The IShape Interface and the three classes that implement it: Line, Rectangle and Ellipse

interface IShape
{
void draw(Graphics
            g);
}

class Line: IShape
{
private int x1, x2, y1,
            y2;
public Line(int x1, int y1, int x2, int y2)
{
this.x1 =
            x1;
this.y1 = y1;
this.x2 = x2;
this.y2 =
            y2;
}

public void draw(System.Drawing.Graphics g)
{
g.DrawLine(new Pen(Color.Brown), x1, y1, x2, y2);
}
}

class Rectangle: IShape
{
private int x1,
            x2, y1, y2;
public Rectangle(int x1, int y1, int x2, int
            y2)
{
this.x1 = x1;
this.y1 = y1;
this.x2 =
            x2;
this.y2 = y2;
}

public void
            draw(System.Drawing.Graphics g)
{
g.DrawRectangle(new
            Pen(Color.Green), x1, y1, (x2 - x1), (y2 -
            y1));
}
}

class Ellipse: IShape
{
private int x1,
            x2, y1, y2;
public Ellipse(int x1, int y1, int x2, int
            y2)
{
this.x1 = x1;
this.y1 = y1;
this.x2 =
            x2;
this.y2 = y2;
}

public void
            draw(System.Drawing.Graphics g)
{
g.DrawEllipse(new
            Pen(Color.Red), x1, y1, (x2 - x1), (y2 - y1));
}
}

The IShape interface has one method: draw. Implementation for this method will be provided by any class that implements IShape. The draw method accepts a System.Drawing.Graphics object on which the shape will be drawn. To draw a shape, two coordinates are required: (x1, y1) and (x2, y2). These coordinates must be available internally in the implementing class and the shape drawn will be determined by these coordinates.

The Line class has one constructor. The constructor accepts four integer values: x1, y1, x2, y2. As you can guess, these values are used as the coordinates for the end points of the line. Implementation of the draw method is straightforward using the DrawLine method of the Graphics object. The DrawLine method overload used in the Line class accepts five arguments: a Pen object, and the four integers x1, y1, x2 and y2. I know you must be wondering how you can get the Graphics and Pen objects for the draw method. However, please bear with me. Now I will focus on the interface and the classes. The Pen object will be explained further in the section to come: "Creating the Graphics and the Pen objects".

The Rectangle and Ellipse classes are also very simple. Both have a constructor that accepts four arguments: x1, y1, x2, y2. These four integers are used as the coordinates. For Rectangle, these two coordinates are used as top left coordinate and right bottom coordinate. For Ellipse, these are the same coordinates for the smallest rectangle that you need to draw to cover the entire ellipse.

The DrawRectangle method of the Graphics class is used to draw a rectangle. Like DrawLine, this method overload accepts five arguments: the Pen object, the x coordinate of the top left corner, the y coordinate of the top left corner, the width, and the height.

In Ellipse, to draw we use the DrawEllipse method of the Graphics object. How it works is very similar to the DrawRectangle method.

Creating the Graphics and the Pen objects

Of course a drawing program needs a canvas to draw on. The System.Drawing namespace provides the Graphics class with a large number of drawing methods readily available once we get a Graphics object. However, to instantiate the Graphics object we need a container. This is where the RichControl class steps in. This class, included in the System.WinForms namespace, is the parent class for all controls. And RichControl is appropriate for our program because of two factors. First, RichControl, like any other control, can be added easily to a Form. Secondly, the RichControl class has a method named CreateGraphics that returns its Graphics object. Using this method, you can then easily get a Graphics object that you can use as your canvas, as follows:

Graphics g = rc.CreateGraphics();

where rc is the RichControl object that is added to the form. Having the Graphics object g, you can then use any of its methods to draw many different shapes, including lines, rectangle and ellipses.

However, one thing is still missing. You need a Pen to draw! Pen is part of the System.Drawing namespace and specifies the width and style of any shape you draw. For example, its DashStyle property specifies several varieties of dashed lines. For this program, however, we will not delve too deeply into this class. We just need to create a Pen object quickly for the DrawLine, DrawRectangle and DrawEllipse methods. To create a Pen, you need to use one of its constructors, one of which accepts one parameter: a Color structure. Fortunately, a Color structure (also part of the System.Drawing namespace) is very easy to use because it has a number of static properties. For instance, to get a black color, you just need to specify the following statement: Color.Black. In the same token, to get a red you specify Color.Red.

To get a red Pen, you just need to use the new keyword with one of the Pen class's constructors:

new Pen(Color.Red)

Drawing the Shapes

The drawing program as seen above has one RichControl control, three buttons to select a shape, four labels and four text boxes. These text boxes are important because you type the coordinates here. When you click one of the buttons, a method named DrawShape will be called, as follows:

protected void button1_Click (object sender, System.EventArgs e)
{
DrawShape("Line", (object) txt_x1.Value, (object) txt_y1.Value,
(object) txt_x2.Value, (object) txt_y2.Value);
}
protected void button2_Click (object sender, System.EventArgs e)
{
DrawShape("Rectangle", (object) txt_x1.Value, (object) txt_y1.Value,
(object) txt_x2.Value, (object) txt_y2.Value);
}
protected void button3_Click (object sender, System.EventArgs e)
{
DrawShape("Ellipse", (object) txt_x1.Value, (object) txt_y1.Value,
(object) txt_x2.Value, (object) txt_y2.Value);
}

The DrawShape method is given as follows:

void DrawShape(String str, object sx1,
            object sy1, object sx2, object sy2)
{
try
{
int x1 =
            Int32.FromString((string) sx1);
int y1 =
            Int32.FromString((string) sy1);
int x2 =
            Int32.FromString((string) sx2);
int y2 =
            Int32.FromString((string) sy2);
Graphics g =
            rc.CreateGraphics();
IShape shape = null;
if
            (str.Equals("Line"))
{
shape = new Line(x1, y1, x2, y2);
}
else if (str.Equals("Rectangle"))
{
shape = new
            Rectangle(x1, y1, x2, y2);
}
else if
            (str.Equals("Ellipse"))
{
shape = new Ellipse(x1, y1, x2,
            y2);
}
shape.draw(g);
}
catch (InvalidCastException
            e)
{
Console.WriteLine(e.ToString());
return;
}
catch
            (ArgumentNullException
            e)
{
Console.WriteLine(e.ToString());
return;
}
}

The DrawShape method first converts the value of the text boxes to integers.

int x1 = Int32.FromString((string) sx1);
int y1 = Int32.FromString((string) sy1);
int x2 = Int32.FromString((string) sx2);
int y2 = Int32.FromString((string) sy2);

Conversion errors will be caught by InvalidCastException. This happens if you try to put non-numeric value in one of the text boxes. Also, if you click the button before entering values to the four text boxes, an ArgumentNullException will be thrown.

The most interesting part of the DrawShape method is given below:

  IShape shape = null;
if
            (str.Equals("Line"))
{
shape = new Line(x1, y1, x2, y2);
}
else if (str.Equals("Rectangle"))
{
shape = new
            Rectangle(x1, y1, x2, y2);
}
else if
            (str.Equals("Ellipse"))
{
shape = new Ellipse(x1, y1, x2,
            y2);
}
shape.draw(g);

This part is where polymorphism happens. shape is a reference variable for IShape, however, depending on the value of the first argument to DrawShape (either "Line", "Rectangle" or "Ellipse"), shape will reference a Line object or a Rectangle object or an Ellipse object. Another instance of polymorphism in action.

What is more amazing is when the draw method is invoked - it knows which shape to be drawn!

Summary

Polymorphism is one of the three main principles of object-oriented programming. This article discussed polymorphism in C# with two examples. The first is a very simple example to get you familiar with polymorphism. The second example is a drawing application that demonstrates polymorphism in action and shows how polymorphism can be useful not only in theory but also in practice.


Web Site Error Logging and Reporting System

Site Design

Site Design (Sort by Name)

An Intelligent Navigation Bar Using C# and XML
Building a Database-Driven Guestbook with C# and ASP.NET
Building an ASP.NET Mailing List
Developing a Universal Data Access Layer leveraging ADO.NET, C# and Factory Design Pattern
Getting Started With Unit Testing
Internationalization With The .NET CultureInfo Class and C#
Introduction to Using Design Patterns - Part 1
Introduction to Using Design Patterns - Part 2
Localization using .NET and C#
Managing Culturally Correct Date and Time Formats in C#

Message Queuing with C# and .NET
Mobile Enabling Existing Applications with the Mobile Internet Toolkit
Programmatically Wiring Up Events in C#
Programming Internet Explorer in C#
Using Legacy COM(+) in .NET - Part 2
Visual Inheritance-An Approach to Superior GUI Design


Programming Internet Explorer in C#

Programming Internet Explorer in C#
http://www.csharptoday.com/content.asp?id=1980
Article Rating: 4.5
Published on November 11, 2002
Abstract
Internet Explorer is a powerful platform that can be used to develop presentation layers for any program that needs to have a rich formatted document interface. The.NET platform makes it possible to access all the power of IE without going too deeply into the details of COM interfaces implementations. However, because the actual version (6.0) is still a pure COM application and the only .NET libraries available for it are simple “interop” wrappers, it can be difficult to write client-side programs that use Internet Explorer, without having some knowledge of particular programming issues. These issues are actually very poorly documented at the moment. To try to fill up this gap, this case study by Nikit Zykov introduces the main starting points that make it possible to effectively access to the power of the IE platform using C#. You can also consider this case study to be about COM to .NET interoperability.
Case Study

Internet Explorer is a powerful platform that can be used to develop presentation layers for any program that needs to have a rich formatted document interface. The.NET platform makes it possible to access all the power of IE without going too deeply into details of COM interfaces implementations. However, because the actual version (6.0) is still a pure COM application and the only .NET libraries available for it are simple "interop" wrappers, it can be difficult to write client-side programs that use Internet Explorer, without having some knowledge of particular programming issues. These issues are actually very poorly documented at the moment. To try to fill up this gap, this case study introduces the main starting points that make it possible to effectively access to the power of the IE platform using C#. You can also consider this case study to be about COM to .NET interoperability.

Introduction

In the last few years, the success of Internet technologies has turned HTML into a widely recognized standard of data presentation. However, its use goes far beyond displaying simple hyperlinked documents. Market demand pushed the companies developing Internet browsers to include into their systems a wide range of formatting, navigation, and programming functionalities. This made it possible for software companies to create complex and fully functional products using HTML browsers as intelligent client presentation platforms.

The best known application paradigm that uses HTML-based technologies at the presentation layer is actually ASP : not Active Server Pages especially, but more generally as Application Service Providing. The idea is to host application software, or to be more precise its data and business layers, on a central Web server, sending the user interface data as a HTML flow to the user's browser, and then exchange data between the client and server via the HTTP protocol. However, there have been some serious problems in developing this model, and as a result it is not yet (at least in Europe) a clear market success.

The first problem is purely marketing: the end-users need some time to get used to this new distribution model, and for the time being, there is not yet enough confidence to it, especially for a non-computing audience. The second reason is technological: the commonly used "page at once" HTTP model makes a really dynamic interface creation difficult, and the fact the programmer has no real control over the browser being used client-side adds extra problems.

Despite these problems (and others that we haven't mentioned), the technologies are constantly in development and we can foresee them becoming overwhelming important in the market in a few years, especially with the probable growth of real-world client-server solutions based on the SOAP exchange protocol. Meanwhile, most programmers still have to work on traditional, more or less "stand-alone", client software.

Naturally, when writing stand-alone programs, we don't want to lose the advantages the HTML presentation model gives us. And fortunately, we don't have to: Internet Explorer (IE), the HTML browser market leader, is not simply a program allowing the user to view some online Web content, but essentially a full-featured programming component that can be integrated into any Windows-based software solution. Here we're going far beyond the traditional "browser-based" models: even if we still have access to the browser's communication model allowing us to manipulate content resources at any location, this part of the functionality remains secondary. The power of Internet Explorer model is the possibility for it to be integrated into any presentation logic, including that of "traditional" Windows-based systems.

There are many advantages to directly programming the Internet Explorer interface. The extended DHTML object model grants the programmer full and real-time access to any presentation element's behavior. This allows us to fully control the entire information flow to and from the user, and to create the richest possible dynamic and intelligent interface for the software. Also, directly accessing IE eliminates the need to have a specific platform for the "server" part of the application, avoiding the need for client-server data exchange and reducing technical support costs. Nevertheless, this programming technique doesn't restrict you to just the world of "stand-alone" applications; you can still use the server-based solutions, simply delegating some of the processing to the client platform, meaning you can still enjoy all of the power of the presentation and communication model Internet Explorer gives you. In fact, this gives your programs a fully scaled presentation layer, concentrating all the presentation logic on one platform, as opposed to the "classic" ASP that delegates only the simple data display to the client, and processes all the presentation management server-side.

Before the .NET platform release, there were basically two main ways to program Internet Explorer. These were:

  • The scripting model
  • Treating IE as a COM component

The scripting model was given by IE itself, while treating IE as a COM component meant programming it directly in C++.

Note : There also was an intermediate way, in that automation-capable languages (those allowing access to IDispatch -type COM interfaces, like VB6 and J++) offered some access to the Internet Explorer object model. However, there were some serious limitations concerning the implementation of advanced interfaces, such as binary behaviors for example, which meant this way was far from perfect, and so we will not explore it in this case study.

Scripting was a good way for those who wanted to easily and quickly write interface code with a minimum knowledge about the platform's functional details. Also, scripting client-side code is generally the best solution when creating "light client" deployment systems when the only coherence between the software users' systems should be the Internet browser version.

At the same time, scripting imposes important limitations on our access to system functionalities, including, in the case of the Internet Explorer, limits on some features of the scripting platform itself. Also, scripting languages, even advanced ones like JavaScript, often take ease of use over rigorousness of the structure. Execution speed is not a strong point of script engines.

C++ programming was a good alternative, because it allowed access to all the platform features and a complete control of the underlying system's functionality. At the same time, this solution required highly qualified developers, and also often meant lots of debugging was necessary.

.NET, and C# in particular, allow us to combine the advantages of both solutions without having to experience their disadvantages. The relative ease of use goes together with a complete access to the operating system and its components. There is of course also the possibility of integrating Internet Explorer features with any .NET software too.

Nevertheless, in releasing .NET, Microsoft has not transformed Internet Explorer into a true .NET system. It's true that since version 5.01 there has been some preparatory work, but the platform access is still a pure COM. The only effort made was to compile a wrapper assembly (named mshtml.dll) that allows us to use the IE COM interfaces from any .NET language.

Looking into this assembly, a programmer can easily understand why the Internet Explorer .NET "new-look programming" has still not really been done. There are countless functions in many hundreds of interface classes, and the classes' functionalities are often redundant. Worse, there is no real hierarchy in these classes: the COM paradigm made an interface immutable - every time there are some new features added to a COM object, a new interface is exported, and generally this interface includes only the new features, without the big "inheritance" effort.

The MSHTML library follows this paradigm, simply giving access to the majority of interfaces exposed by Internet Explorer. A little effort has been done by mechanically combining interfaces relative to any particular underlying COM object into some kind of .NET class, without really trying to organize all that into a more logical manner. In conclusion, the assembly is not really easy to use, especially because there is practically no accessible documentation!

However, programmatically accessing Internet Explorer platform remains a relatively simple task. Most of the effort is essentially organizational. All you need to know is how to access some basic techniques, then to encapsulate this functionality to a consistent class library, and then IE programming using C# or any other .NET language becomes as easy as a traditional DHTML scripting.

Some programmers will ask why they should need to use an Internet Explorer interface when they now have easy access to Windows Forms, which is an extremely powerful presentation platform. In fact, the choice of whether or not to use an IE DHTML presentation for the user interface depends on the type of information the programmer has to show to the final user.

Windows Forms are perfect if you know in advance the exact structure and format of data you have to display. This library is designed to give the programmer full control over every individual object displayed, its characteristics, contents, and position. However, this flexibility has its cost: while it is very easy to work with any number of single objects, the task becomes much more difficult when dealing with complex data flows, as it is the case for any document management system, for example.

This is exactly the situation where Internet Explorer shows its worth. HTML was created because of a need to have a common data format that everyone can read, but it has grown due to a need to have a rich presentation for this data. If you want to give a new skin to the data your client provides in some structured format (which actually often means XML), using Windows Forms, you'll need to write your own parser. This is interesting, but it means very high development costs. With Internet Explorer used as a presentation component, it's straightforward; you just have to add a style sheet. And if you want to add interactivity to this data management, you'll need to access the IE programming interfaces (which is precisely the subject of this case study), thus integrating it de facto to Windows Forms system as one more component, probably the most complex, but also giving powerful rich interactive data presentation.

The best use of the Internet Explorer's "browser" component is integrating it into a more complex system, including other Widows Forms elements, that can offer more power for some particular task. Remember the presentation of rich-formatted documents (especially if this presentation is often changing, don't forget that Internet Explorer understands CSS too) and document-driven data is much easier when using the IE component.

In this case study, we won't be especially considering Internet Explorer as a web browser, i.e. as a tool used to access remote data. Writing server-based and client-independent applications is another situation that has its own advantages and constraints; we'll not analyze it now. Here, we're taking about a means of presentation layer development for applications that are not especially networked, but that do have requirements of high-quality presentation of big volumes of human-readable data. We'll also be considering how this helps reduce the presentation layer development costs.

This case study gives you a starting point; it unveils the techniques needed to effectively access the power of the platform by presenting some basic examples that are easy to understand. It also gives an idea of the way to follow to create an easy-to-use library to access the platform. And for the curious people it also explains how to go further to explore the advanced features of the Internet Explorer.

Requirements

To get real benefit from this case study, the reader must have a good understanding of C# and of object-oriented programming in general. A good knowledge of the DHTML object model would be very helpful too. There is no real need to deeply understand the COM model, but an understanding of its basics would be advantageous.

The examples are meant to be used with the final release of Visual Studio.NET (2002) on machines with Internet Explorer 6 (or more recent) installed.

Definitions and structure

This is not a traditional "one task" case study. Its goal is not to guide the programmer through the steps of developing one particular application, but rather to introduce some essential techniques allowing the use of a particular presentation platform. Each section will talk about its own little bit of code in order to demonstrate the issues being discussed.

The main sections of this case study are:

1 - Internet Explorer as a programming component

This section shows how to integrate an Internet Explorer interface into a C# application created using Visual Studio .NET. It explains how to call its methods and respond to its events.

2 - Accessing DHTML object model

This section explains how, having a document loaded in the IE component, to obtain access to its DHTML structure. We take the page loading procedure from the previous section's example and as an example we extract the HTML code of each page loaded. We also discuss here the interoperability details needed to completely understand the underlying mechanisms.

3 - Introduction to the DHTML document structure

Without going too deeply into the concepts of DHTML, this section presents the notions of "document" and "element" in the DHTML object model. First, loading a blank document to the component, we show that even a blank document has its own DHTML structure. We then programmatically create a text label, a button, and an input zone on the blank page. We obtain interfaces for each of these elements. On the basis of this example, we study some of the basic ideas of encapsulation of the document-related objects into a consistent class structure.

4 - DHTML events treatment

This section discusses the way the Microsoft wrapper treats the DHTML events in the .NET code. It also explains why this method can only by applied to certain cases, and then develops an original method of implementing event handlers. This example also demonstrates some advanced interoperability techniques.

5 - Launching scripts from C#

In this section, we'll show how to call any JavaScript function implemented on the DHTML page's namespace. It also discusses some problems related to parameters and return values, touching on some basic aspects of marshalling data between .NET and COM applications.

6 - Extending the DHTML model

window.external is a scripting interface object that when implemented gives the script author the opportunity to call additional methods defined by the developer of the application hosting the Internet Explorer component. We explore this situation in this section.

1 - Internet Explorer as a programming component

When using the .NET environment, you can consider an Internet Explorer browser window to be a simple Windows Forms component. You can integrate it into a form, call methods and use properties it exports, and attach its events to the handlers you develop. However, all this requires a little preparatory work: don't forget Internet Explorer is not a native .NET application. Fortunately, the new Microsoft platform includes easy to use interoperability functionalities that allow integration to any COM/ActiveX object as if it were a native .NET component.

Let's see now how to effectively do it. Open Visual Studio .NET and create a new C# Windows Application project - this creates an application template with an empty form as a main window. We now have to add Internet Explorer component to this form. To do it, we have first to integrate the Microsoft Web Browser component to the environment as a Windows Forms control.

Go to the Toolbox, right-click it, and choose Customize Toolbox in the context menu.

In the dialog box shown, choose the COM Components tab, and then check the Microsoft Web Browser component. Please note that if your operating system or Internet Explorer version are not English, this component's name will probably have been translated. For example, it is called "Navigateur Web Microsoft" for French systems. After closing the dialog box, you'll find the new Explorer component at the end of the Windows Forms section of your toolbox.

Now drag and drop this component onto your form. This has two effects. First, it automatically creates and adds two interoperability assemblies to the bin/Debug directory of your application (assuming you compiled it in debug mode). These are Interop.SHDocVw.dll and AxInterop.SHDocVw.dll, both referencing the COM shdocvw.dll library that implements the Web Browser control. The first library represents a complete "interop" view of the underlying COM library, giving access to its interfaces and events that allows the programmer to "fine-tune" the component use. The second, used by the Windows Forms integration shell, is an "ActiveX" view of the same library, giving a VB-style access to the Web Browser component implemented as an application class.

The references to these assemblies are added to your projects. This allows you to directly use their classes and interfaces in your code. The second effect of adding the Explorer control to the form is, as with any Windows Forms component, to generate the C# code to effectively add the component to the form at execution time. Look at the Windows Form Designer generated code region in your code to see details of the component integration, and notice it's slightly different compared to ordinary .NET component integration. This difference is transparent for you now, but it can be important if you need to create your code without using Visual Studio.NET.

Normally, the first Web Browser component created will be referenced in your code as the axWebBrowser1 variable of the type AxSHDocVw.AxWebBrowser, which is in fact a wrapper for a Visual Basic-style ActiveX class generated around the IWebBrowser2 interface of the component. This class is our entry point, allowing us to manipulate the IE Web Browser object and to use its events.

The first thing we can try to do is to navigate to a web page of our choice when the program initializes. This functionality can be accessed through the Navigate method of the Web Browser class. The method takes five arguments. We'll only use the first, specifying the URL to navigate to. The four other arguments allow us to specify the navigation details, for example the target frame or POST data.

A small problem is that this method is a direct wrapper to the correspondent C++ specification, so its arguments follow exactly the standard type conversion rules; in particular, when there is a Variant pointer declared in C++, we get a ref object in C#. So, even for passing zero or empty string parameters, we absolutely have to create intermediate (useless) variables. Considering all that, the navigation code (using http://www.csharptoday.com/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq as an example navigation target) will be the following (insert it into the form constructor, after the InitializeComponent() call):

object zero=0;
object nl="";
axWebBrowser1.Navigate("http://www.csharptoday.com",ref zero, ref nl, ref nl, ref nl);

Launch your application, and if you are connected to the Internet, you'll see the C# Today homepage displayed in your navigator component:

Now, adding some basic functionality, we are already able to program the simplest-possible custom Internet browsing program. This program will have an address bar showing the loaded page's URL, a Navigate button allowing the user to go to the address entered in the address bar when it is changed, and a Home button that takes us to a hard-coded homepage.

First go to the form designer and add one text box and two buttons to your form. Label the first button "Go" and the second "Home". Now, by double-clicking the "Home" button, define its activation (mouse click) event; it'll be button2_Click in your code. Move the navigation code actually entered in the Form1 constructor code to this handler. After adding a line that sets a cursor to the "wait" state when the page is loading, our handler will look like this:

object zero=0;
object nl="";
axWebBrowser1.Navigate("http://www.csharptoday.com",ref zero,ref nl,ref nl,ref nl);
this.Cursor=Cursors.WaitCursor;

Navigating to an Internet page in IE is of course an asynchronous process, so we need to handle the event that fires when the page load is complete. This is done by defining a handler to the DocumentComplete event, fired by the Explorer COM object.

Return to the form designer, select the Explorer component, then the Events tab in the Properties window. After that, double-click on the DocumentComplete event. This adds to your code the axWebBrowser1_DocumentComplete handler that is called when the corresponding event fires. If you look at the Windows Forms Designer generated code section, you'll see that DocumentComplete is exposed by the AxWebBrowser class as a C# event and your event handler simply implements the delegate type DWebBrowserEvents2_DocumentCompleteEventHandler. So basically, implementing event handlers for the Explorer object is as easy as for any native .NET object.

Place the following code in the event handler:

textBox1.Text=axWebBrowser1.LocationURL;
this.Cursor=Cursors.Default;

With this code in place, when the page is finished loading, you'll find its complete URL in your "address bar" and the mouse cursor will return to its normal state. Take care however, this works fine for single-frame windows; when you go multi-frame, the DocumentComplete event is fired on every frame load completion, and so if you plan to place some important initialization code here, you'll have to detect which part of the HTML document fires this.

The last thing we have to do is creating an event handler for the "Go" button; the only difference compared to the "Home" handler will be that the URL to navigate to will be taken from our "address bar", and not hard coded:

object zero=0;
object nl="";
axWebBrowser1.Navigate(textBox1.Text,ref zero,ref nl,ref nl,ref nl);
this.Cursor=Cursors.WaitCursor;

That's all; you can now use your program as an Internet browser!

At this point, we get to look at the first bit of interesting encapsulation code: notice that the Navigate method is not really easy to use as is. Boxing parameters makes you waste lines of code, but it is not the only problem in our case: we can't easily emulate the HTTP posting of string data, because even if there is a data conversion (marshaling) between .NET and COM (for instance, a reference to an array in .NET is directly translated to the pointer to SAFEARRAY COM type), the mandatory array data type for the "post data" parameter is not really convenient for everyday use.

The other important issue in the case of this method is the fact that in the most cases we'll really only use one, or a maximum of two, of its five parameters. So, the best idea would be to create a method with a variable number of parameters. C# gives us such a possibility by providing a params keyword. Using it, the Navigate encapsulation should look like this:

public void Navigate(params object[]
            data){
object url= (data.Length>0 && data[0] is
            string) ? data[0] : "about:blank";
object target=
            (data.Length>1 && data[1] is string) ? data[1] :
            "";
object headers= (data.Length>2 && data[2] is
            string) ? data[2] : "";
object
            post="";
if(data.Length>3){
object post=data[3];
if(post
            is
            string){
if(((string)headers).IndexOf("Content-Type:")<0)
headers+="Content-Type:
            application/x-www-form-urlencoded\r\n" ;
string
            tmp=(string)(data[3]) ;
post=new byte[tmp.Length];
for(int
            i=0;i<tmp.Length;i++)
((byte[])post)[i]=(byte)tmp[i];
}
}
object
            flags= data.Length>4 ? data[4] :
            0;
axWebBrowser1.Navigate(url,ref flags,ref target,ref post,ref
            headers);
}

This method changes the order of arguments you pass to the method to put the most frequently used first. The first argument is (as for the original Navigate) the page URL, the second is the target (for example, "_top" for navigating in the topmost frame, just like in the target attribute of the <a href.> HTML tag), the third is a string containing the CrLf-separated HTTP headers sent, the fourth is the post data, and the fifth are the flags modifying the method behavior.

(Headers are the set of technical information the browser sends to the server when requesting a page. It is in general a plain ASCII text containing one parameter per line in the " name: value " form. That can be used, like in the example above, to specify the format of the data posted to the server or, for instance, to specify, for multi-host Web servers, a virtual service URI.)

The post data treatment requires some special work in the function. First, it can be passed as a string, or alternatively, as an array of any integer value type. The array version gives the maximum freedom to the programmer, allowing us to pass any amount of any data to the given address. The string version can be used, for example, to simulate a form data submit. In this case, we need, as it was written before, to first convert the string to the byte array (no, HTTP doesn't always use Unicode) and to add, if it is not already done, a header specifying that we are sending data to a form.

Now, sending two form fields to a dumb ASP script can be done like this:

Navigate("http://www.zykov.com/dumb.asp","","","test=essai&test2=proba");

.and simply requesting a page is just:

Navigate("http://www.wrox.com");

2 - Accessing DHTML object model

Until now, we have had almost nothing to program: Visual Studio automatically did most of the work. The ActiveX interfaces of the Web Browser object give the interoperability system enough information to naturally incorporate this component and to give you all the "visual" programming facilities you have with any .NET component, including IntelliSense, which allows you to easily discover the object's functionalities.

All that gives you is access to manipulate the browser window itself; what is much more interesting is the contents manipulation, i.e. the possibility for us to view and change the structure of the HTML documents loaded into the Internet Explorer browser window. This first requires some supplemental work.

The AxWebBrowser type gives us access to the Document property that represents the object managing the DHTML document that is loaded. This object can be considered as an (extended) equivalent of the document object in the DHTML scripting model. In C#, our problem is that this property is of type object, which doesn't really help us to manipulate it.

If we look into the C++ headers declaring "low-level" the equivalent interface type, we'll see that the Document property is exported as an IDispatch interface pointer, which is effectively represented by an object type after marshalling to .NET. In COM, when we need to use such an object, we have to obtain one or more interfaces that indirectly give you access to the class's methods and properties. The main difference between COM and .NET interface concepts is that in COM the interface is the only possibility to use an object, any other access is (in theory) forbidden. So, if we want to use the object behind the Document property, we first need to obtain interfaces giving access to it.

Fortunately, the relatively complex process of querying a COM interface is hidden in .NET behind a simple type casting procedure: it's enough to explicitly convert the object's type to the interface type you need in order to automatically obtain the pointer giving you access to all the properties and methods declared in the interface. This also has an interesting side effect: it's enough to apply the C# is operator to a .NET reference of the COM object to know if the interface having its type as the other operator's argument is exported.

But, where can we find declarations of the interfaces we need? The ShDocVw libraries we got when integrating the Explorer component into our code don't give us any information on the underlying HTML documents. This is the point when we need to use MSHTML, the library we've briefly spoken about in this case study's introduction. We need to explicitly reference this library before using it.

Go to the Visual Studio Solution Explorer with your project open. Right-click the References folder and choose Add Reference.

In the dialog box that appears, be sure to have the .NET tab selected, and then find the Microsoft.mshtml component in the list. Double-click on it (it must appear in the Selected Components box), and then click OK. Now we simply have to add the using mshtml declaration at the beginning of the C# code file where we need to access it, and we can begin work.

The MSHTML interfaces we can use to access the DHTML object model are actually IHTMLDocument, IHTMLDocument2, IHTMLDocument3, IHTMLDocument4 and IHTMLDocument5. Why are there five interfaces to access the same object? The reason is that in COM, once the interface is declared and uniquely identified, it can't change at all. So, when you add some functionality to your COM class and you want to give the class users access to these new methods and properties, you need to create a new interface that generally declares only new methods. There is no real inheritance mechanism behind all that, so frequently (and that is the case with the Document object) each individual interface contains only the new methods of the object's versions corresponding to the moment the interface was created. In our case, IHTMLDocument is a heritage of "prehistoric" versions of Internet Explorer, with the other four exports being the new functions of versions 4.0, 5.0, 5.5 and 6.0 of the platform respectively.

The MSHTML.NET library that is in fact a wrapper for the corresponding COM library, does a little more by combining all these interfaces into one HTMLDocumentClass.NET class. However, it is preferable to use interface classes directly when creating your Internet Explorer C# applications. The reasons are too complex to be discussed here; we can simply say that you can program easier by creating your own class hierarchies that fit better to your project's style and generally to the .NET class libraries' ideology.

Now, let's try to obtain one of these document interfaces and use it, for instance, to access the loaded page's DHTML code. As we won't be directly working with the form components' behavior from now on, it's better to create a new class and to encapsulate all the DHTML document treatment into it. So, create a new class file in the Solution Explorer, naming it DHTMLDoc (don't forget to add " using mshtml " to the class file's headings!), and add the following "skeleton" code to it:

public class DHTMLDoc{
public
            DHTMLDoc(object doc){
if(!(doc is IHTMLDocument3))throw new
            Exception("This is not a DHTML
            document!");
document=doc;
doc2=(IHTMLDocument2)doc;
doc3=(IHTMLDocument3)doc;
}

object
            document;
IHTMLDocument2 doc2;
IHTMLDocument3
            doc3;
}

This makes you a constructor that, when obtaining the reference of the COM object's wrapper, first confirms that it's really a DHTML document object reference (we can't use the Internet Explorer versions older that 5.01 with the .NET Framework anyway, IHTMLDocument3 is the minimum requirement), and then obtains from it the two most used interfaces: IHTMLDocument2 and IHTMLDocument3. These two are actually declared in the MSHTML library as common .NET interfaces, so you can use them as is and explore them through IntelliSense in the Visual Studio .NET environment.

The last important thing to note before implementing our "HTML code view" is the fact that you can (of course) only use the Document property of the Explorer object when there is an HTML document completely loaded to it. But you already know how to control this: there is the DocumentComplete event in this object that fires exactly when any document is completely loaded. Let's use it.

First add to your axWebBrowser1_DocumentComplete handler the following line:

DHTMLDoc doc=new DHTMLDoc(axWebBrowser1.Document);

This instantiates the document encapsulation class you recently created. We'll now use it to obtain the document's "source code". Add to the DHTMLDoc class the following method:

public override string ToString(){
return doc3!=null ? doc3.documentElement.outerHTML : "";
}

This function overrides the standard ToString function of the Object class, thus we consider here the DHTML document's contents as its string representation from the C# point of view. We use the instance of the IHTMLDocument3 document's interface to obtain its root element (i.e. the container for all the document code), exposed as a documentElement. This element is already declared as an IHTMLElement, that is the default (but not the only) interface it exports, so we can directly use it to get, just like in script DHTML model, the outerHTML property returning the underlying HTML code, as it is parsed by Internet Explorer.

Now, the only thing we have to do is to show the result to the user. Go to the Form designer, add a new text box to your form (named textBox2), and declare it as multi-line, read-only, and using both scroll bars. Now add just one more line to your DocumentComplete event handler:

textBox2.Text=doc.ToString();

When navigating to any page with your new application, you'll now see the page in the navigator window and its HTML code in the text box at the same time:

One little bug fix to end this section: you'll probably have some difficulties effectively resizing the Explorer control you're using. More precisely, resize will be reported as effective (and reflected in the source code), but when your program launches, you'll see the control returning to its initial size. This problem is common to many "VB-style" ActiveX controls: it comes from the fact the component's size is automatically kept in the "OCX state" bag and restored (by default) every time the program is launched. To fix this, after visually modifying the component's size, but before compiling and launching the program, you just have to look into the "Windows Form Generated Code" section in your form code and to find a string containing " OcxState ", and comment it. After that, compile and launch your program, then immediately close it. When that's done, the new size is recorded in the state bag, and you can uncomment the line again.

In conclusion, technically the COM to .NET interoperability is relatively easy to use, but philosophically, it remains a "thing in itself".

3 - Introduction to the DHTML document structure

In the previous section, we introduced the concept of the document. This is the central point of the DHTML object structure. There is one other class of objects, closely related to the document paradigm, a window. It relates to the same underlying object, but looks more at the presentation aspects of the framework than the structural details. These objects often share (especially when seen from the MSHTML point of view, which is more detailed that the script DHTML model) the same functionalities, like script management or frames manipulation for example. In the most cases, you can consider that one document object corresponds exactly to one window object. It seems logical when implementing an encapsulating .NET library, to mix the both functionalities in one generalized class, giving a starting point for manipulating any aspect of the DHTML data flow.

These two object's classes give us our initial framework; now let's see how we can manipulate the inner structure. The primary concept behind HTML means that almost everything you can encounter in a "document" is an element, i.e. object representations of tags and their contents. It is the fact that you can manipulate these objects and change their presentation properties in real time that adds the "D" (dynamic) to the DHTML acronym. Note that the element objects export, just like documents, multiple interfaces. These are actually IHTMLElement, IHTMLElement2, IHTMLElement3 and IHTMLElement4.

Elements are basically organized as a tree, so elements can contain other elements and the document can be considered as a root container. The HTML standard, and especially its Internet Explorer version, forces any document to have some minimum structure. In the program you created in the previous section, if you navigate to about:blank (which is the equivalent of loading an empty HTML page), you'll see there is the following code in the corresponding text box:

<HTML><HEAD></HEAD>
<BODY></BODY></HTML>

This simply means that the document object created from the empty code still gives you access to the visual document presentation contained in the BODY section. This is represented by the body property of the IHTMLDocument2 interface, and its management details that are normally listed in the HEAD section of an HTML file and are accessible through multiple document's and window's properties and collections.

We'll use this by implementing an example of dynamically inserting some HTML into the empty document that is loaded. Change the DocumentComplete handler by inserting (before the string showing the document code, because we want to see the insertion results both in the browser presentation and in the code) a call to the Test method of our doc object, conditioned by the fact the URL of the page loaded is about:blank. The handler's code should now look like this:

private void
            axWebBrowser1_DocumentComplete(object sender,
AxSHDocVw.DWebBrowserEvents2_DocumentCompleteEvent
            e)
{
textBox1.Text=axWebBrowser1.LocationURL;
this.Cursor=Cursors.Default;

DHTMLDoc
            doc=new
            DHTMLDoc(axWebBrowser1.Document);
if(axWebBrowser1.LocationURL=="about:blank")doc.Test();
textBox2.Text=doc.ToString();
}

Yes, you're right; the Test method is not yet implemented in the DHTMLDoc object. In this method, we'll simply fill the innerHTML read-write property of the document's body (which is of course an element object). If you already programmed scripts for Internet Explorer 4+, you probably know this technique:

public void
            Test()
{
doc2.body.innerHTML="<input type=\"text\"
            id=\"inp1\"/><hr/>
<button
            id=\"but1\">Button</button><hr/>
<span
            id=\"text1\">Text zone</span>";
}

Now, once you've compiled and launched your program, and then navigating to the about:blank URL, you'll get the following results:

Note that the HTML code we put into the document's body is a well-formed XML fragment; the result we're seeing in the Browser's document object is not. The Internet Explorer hypertext markup processor converts the code it reads to make it fit to the internal object structure that is actually not true XML, even if some efforts were done in the last version by introducing the DHTML DOM paradigm.

Just as you have done with the body, you can access and manipulate any of the document's elements. To demonstrate it, we'll develop an extension to our program, allowing the user to change the background color of any element.

First, we'll need some preparatory work. We'll have to add some new interface elements (we're talking about Windows Forms here, don't confuse it with DHTML!) to our program to access the DHTML document from their event handlers. So, we first need to grant access to the document object from anywhere in the class implementing our main application's form. For that, just move the doc variable declaration from the DocumentComplete event handler to the Form1 class's body. To be sure we're always using a valid document object, initialize it to null before every Navigate call. Now, go to the form designer and add to the form the following: one combo box (it'll list our document's elements; declare it with a DropDownList type), one label (it'll "ask" for the element's background color), one more text box (we'll specify the color here; we suppose its name is textBox3 in the C# code), and finally one button (we'll use it to apply the changes to the document).

Now we have to prepare three methods to give us the new functionality. The first will list (by specifying tag names and id attributes, if present) all the elements actually present in the DHTML document loaded in our Browser component, returning the result as an object array. The second will extract a specific element's actual background color (the element being identified by its ordinal number in the document's all collection containing every element present). The third will effectively change the background color style property of such an element. Because the first method uses the dynamic array list, we'll also need to add the using System.Collections declaration to the class file heading. The methods' code follows:

public object[] GetAll()
{
ArrayList
            retval=new ArrayList();
IHTMLElementCollection
            alles=doc2.all;
foreach(IHTMLElement elm in
            alles)
retval.Add(elm.tagName+(elm.id==null?"":"
            id="+elm.id));
return retval.ToArray();
}

public string
            GetColor(int en)
{
IHTMLElement
            elm=(IHTMLElement)doc2.all.item(en,0);
if(elm.style==null)return
            null;
return
            (string)elm.style.backgroundColor;
}

public void
            SetColor(int en,string col)
{
IHTMLElement
            elm=(IHTMLElement)doc2.all.item(en,0);
if(elm.style!=null)
elm.style.backgroundColor=col;
}

We have to fill the combo box with the elements' list every time the new document is loaded to the Browser object. To do so, add the following code to your axWebBrowser1_DocumentComplete handler before the line displaying the document code:

object[] elms=doc.GetAll();
foreach(string e in elms)
comboBox1.Items.Add(e);

We just have to code the other two events now. The first, the combo box element selection, will show the selected element's actual background color value. The second, fired when the "Change" button is pressed, will change that color. The handlers code is as follows:

private void comboBox1_SelectedIndexChanged(object sender, System.EventArgs e)
{
string col=doc.GetColor(comboBox1.SelectedIndex);
if(col!=null)textBox3.Text=col;
else textBox3.Text="";
}

private void button3_Click(object sender, System.EventArgs e)
{
try
{
doc.SetColor(comboBox1.SelectedIndex,textBox3.Text);
textBox2.Text=doc.ToString();
}
catch(Exception ex)
{
MessageBox.Show("Color change error:\n"+ex.Message,"Error");
}
}

Notice the second handler's exception handling code. It is needed because the user is free to put any value into the "color" text box, and the DHTML processor does not accept invalid values. Compile and launch your program; test it, taking the previously generated HTML code as an example:

This section's examples show you how to analyze and change everything in any HTML document you load to the Browser component. Don't forget that when you get an object from one of the MSHTML methods or properties which has some particular interface as a declared type (IHTMLElement for IHTMLDocument2.body for example), you can often also obtain the interfaces that were implemented later for the same object. For example, if obj is declared as IHTMLElement, you can almost be sure that (IHTMLElement2)obj and (IHTMLElement3)obj conversions will be valid.

A good idea to optimize the DHTML integration in your projects would be to create a class hierarchy that has two main base classes: document, that combines DHTML document and window features; and element that encapsulates all the methods and properties exported through different interfaces of the underlying COM object. You can then inherit all the classes implementing specific functionality concerning the particular element types (like an input box or an anchor). For complex systems, it can also be interesting to encapsulate the Web Browser object itself in order to give your programmers a complete and consistent class hierarchy.

4 - DHTML events treatment

The features we've studied so far give us full access to DHTML content management, but the interaction with Internet Explorer-side remains completely passive. We still need to use standard Windows Forms components to deal with user input. Nevertheless, the DHTML object model exposes the large event model, allowing us to develop a full-featured user interface just by using Internet Explorer.

Of course, we can manage these events from a C# program. Microsoft developers integrated the full set of .NET events and delegates encapsulating the DHTML events functionality. To give you an example of this encapsulation use, we'll first try to treat the mouse click event for the HTML button from the previous section's example.

If you've already programmed DHTML scripts, you will know that such an event actually uses the onclick handler. In the MSHTML library, a particular event's interface and a full set of corresponding delegates are associated with every particular type of element, like buttons or input boxes. In our case, all the events are managed by the HTMLButtonElementEvents2_Event interface (that we need to explicitly obtain from the element object), and the onclick event is described by the HTMLButtonElementEvents2_onclickEventHandler. Let's put all that together. We'll first implement an easy way to access to a document's element from its id attribute value. For that, we'll implement in our DHTMLDoc class a little indexer method:

public object this[string id]
{
get{
            return doc2.all.item(id,0); }
}

Using this, we'll just need a couple of lines of code to attach the event to the handler function we'll develop in our form class, following the delegate prototype like this:

private bool OnDHTMLClick(IHTMLEventObj e)
{
MessageBox.Show("DHTML button click detected","C# event handler");
return true;
}

The code we have to put at the end of the DocumentComplete event handler follows:

if(doc["but1"] is HTMLButtonElementEvents2_Event)
((HTMLButtonElementEvents2_Event)doc["but1"]).onclick+=new
HTMLButtonElementEvents2_onclickEventHandler(OnDHTMLClick);

The only peculiarity in this two-line code snippet is the fact that we verify if the interface we're interested in is really exported before we actually use it. After compiling and launching the program, you can verify that your C# code is really invoked when you click the HTML button:

It is relatively easy to write this: we really need only two lines of standard C# event code to attach the handler we need. However, it is only easy if we remember the (forty-character) name of the event and that of the corresponding delegate! Also, don't forget that the delegate prototype is not the same for all events! This last point is not easy to understand: the DHTML object treats all its events in a strictly identical manner, there is one event object shared by all the elements through the DHTML window object.

That would not be such a serious problem (the library may not be intuitive, but it can still be usable) if it could be easily implemented, but it is not. If you try to handle the onchange event for our text input box, you'll see that, even after attaching and correctly implementing the HTMLInputTextElementEvents2_onchangeEventHandler delegate for the onchange event of the HTMLInputTextElementEvents2_Event interface there is no reaction from the event handler when the event is fired. A forgotten implementation, probably.

So, if we really want to have access to any DHTML event and do it easily, without having to verify the delegate type at every event attachment, we have to find another way to deal with events. The solution exists, of course: .NET events and delegates are not the native mechanism for dealing with events in the IE COM platform, they are just wrappers. Internet Explorer events handlers are in fact ordinary interface pointers, which you can set to an object of your choice. But what kind of objects is it? The MSHTML C++ documentation says it must be an IDispatch -type COM objects having the event handler as its default method. There is no problem with IDispatch, because any .NET object, when marshalled to COM, implements this basic interface, but what's a "default method"? There is no such notion in C#. In COM, any IDispatch interface method has a numeric identifier, a " dispid " - the default method is simply one that has this identifier set to zero. The .NET platform allows you to force the interoperability engine by setting the dispid of a particular method to any value through the attributes mechanism: so, it is enough to give your method the [DispId(0)] attribute to make it default from the COM point of view.

That means we can create a class that implements the event handler through its "COM-default" method and assign an object of this class to one of the IHTMLElement* interfaces' event handler properties, like onchange or ondblclick. Note these properties are not the .NET-style events; they are interface pointers, declared in .NET, after standard marshalling as an object type.

The last thing we have to do before implementing such an event-handler class is to make it universal. That means that we don't want to create a new class every time we want to handle an event. To do this, we can simply create a delegate and manage a variable of this delegate type; our handlers will simply be attached to this variable. For our first example, we'll use the simplest possible implementation, with nothing dealing with the event information object, our handler will simply respond to the event fired without trying to deepen its contextual knowledge.

Create a new class named DumbDHTMLEventHandler. Put the following code in the class file:

using System;
using
            System.Runtime.InteropServices;

namespace
            IEExample
{
public delegate void
            DumbDHTMLEvent();

public class
            DumbDHTMLEventHandler
{
[DispId(0)]
public void
            Call()
{
Handler();
}

public DumbDHTMLEvent
            Handler;
}
}

Note the use of the System.Runtime.InteropServices namespace. We need it to access the implementation of the DispId attribute. Note also the DumbDHTMLEvent delegate is declared namespace-wide, we'll use it to declare event handlers in our code. Let's implement the onclick event for the input box generated in our example. Place the following code at the end of the DocumentComplete event handler:

if(doc["inp1"] is
            IHTMLInputTextElement)
{
DumbDHTMLEventHandler h=new
            DumbDHTMLEventHandler();
h.Handler+=new
            DumbDHTMLEvent(OnDHTMLText);
((IHTMLInputTextElement)doc["inp1"]).onchange=h;
}

Note we need to obtain the IHTMLInputTextElement interface from our element reference in order to be able to change the onchange property: this event is not common to all the DHTML elements, so we need to deal with it in a particular way; this is a case when using multiple interfaces for one class is completely appropriate.

We can implement the OnDHTMLText handler in our Form1 class like this:

private void OnDHTMLText()
{
MessageBox.Show("DHTML text change detected:
"+(((IHTMLInputTextElement)(doc["inp1"])).value),"C# event handler");
}

After compiling our application, launching it, loading the about:blank page (which launches our DHTML generation code), changing the HTML input box's text, quitting that box (by pushing the Tab key, for example), we can see our event handler work successfully:

This works effectively; however, it would be more interesting to make it a true .NET-style event handler in order to use it in C# with normal event - delegate pairs, without losing the advantages of our implementation simplicity of course. Fortunately, C# allows us to change the manner the event handlers are managed through add and remove modifiers. First, to make our event fully-functional, we once again need to do a little preparatory work.

First, let's create a handler class able to give us access to the DHTML event object interface that we need in order to access to some supplemental information about the event we're managing, like mouse click coordinates or the event source object. As we have already said, the event object is unique for a DHTML document; it is updated every time an event is fired. So, to obtain this object, just like we do in DHTML script code, we can get the event property of the window object attached to the document. We'll extend our DHTMLDoc class to make this operation easy by adding a new Event property to it (note the "@" character preceding a marshalled event property, it's a syntax trick avoiding a keyword rules violation):

public IHTMLEventObj Event
{
get{
            return doc2.parentWindow.@event; }
}

Now we can create a new fully functional DHTMLEventHandler class and a corresponding delegate (don't forget to also declare our use of the mshtml and System.Runtime.InteropServices namespaces):

public delegate void
            DHTMLEvent(IHTMLEventObj e);

public class
            DHTMLEventHandler
{
public DHTMLEventHandler(DHTMLDoc
            doc)
{
document=doc;
}

[DispId(0)]
public void
            Call()
{
Handler(document.Event);
}

public
            DHTMLEvent Handler;
DHTMLDoc document;
}

Now we'll create a new DHTMLElement class taking as a model the DHTMLDoc we developed earlier. This class will allow us to encapsulate the generic DHTML element functionality. The first bit of functionality we'll manage will be event handling. In the code below, we'll implement the onclick event template:

public class DHTMLElement
{
public
            DHTMLElement(DHTMLDoc d,object e)
{
if(!(e is
            IHTMLElement))throw new Exception("This is not a DHTML
            element!");
doc=d;
elm=(IHTMLElement)e;
}

public
            event DHTMLEvent onClick
{
add
{
object
            old=elm.onclick;
DHTMLEventHandler
            h;
if(old.GetType()==typeof(DHTMLEventHandler))h=((DHTMLEventHandler)old);
else
            elm.onclick=h=new
            DHTMLEventHandler(doc);
h.Handler+=value;
}
remove
{
object
            old=elm.onclick;
if(old.GetType()==typeof(DHTMLEventHandler))
((DHTMLEventHandler)old).Handler-=value;
}
}

IHTMLElement
            elm;
DHTMLDoc doc;
}

The code for adding and removing event handlers uses the fact that in C# (and in .NET generally) we can compare object types at runtime. So, if we see the DHTML property used as event handler entry point is already set to an object of our class type, we simply use this object, otherwise we create a new one. You can do copy-and-paste to create other events handlers; the only additional work you'll probably have to do concerns querying for special interfaces for the events that don't concern all the DHTML element types (like onclick).

Once this work is done, you can add an event handler to any DHTML element by using only two lines of code, like with DHTML except that in our case, you are sure it works and you can share one delegate type for all your events. To give you an example, we'll create the following handler in our Form1 class:

private void DHTMLEvtClick(IHTMLEventObj e)
{
MessageBox.Show("Click detected at: "+e.x+"/"+e.y,"C# event handler");
}

To attach this to the event, just add these two lines to your DocumentComplete handler:

DHTMLElement elm=new
            DHTMLElement(doc,doc["text1"]);
elm.onClick+=new
            DHTMLEvent(DHTMLEvtClick);

It works! You can now see the coordinates of your mouse pointer when you click on the text zone you attached your handler to:

5 - Launching scripts from C#

If you're interested in Internet Explorer programming, you've probably already written scripts for this platform. You probably even have a library of script code you use in your everyday work. Of course, it's better to translate these scripts in C#: their extensibility and execution speed will improve. However, this kind of work is longwinded and dull, so you should ask yourself how to directly interact with this script code from C#.

Your first idea may be to extract the script code from the page it's written on and execute it using standard Microsoft scripting engine object. The problem is that in this case you lose access to the page's DHTML object model that is integrated when you execute scripts from Internet Explorer, and you don't even have access to the common variables namespace of the page. So, if you want your code executed correctly, you have to directly access the IE internal script engine, and even more precisely, to the instance of that engine attached to one concrete page, because the variable namespaces in Internet Explorer are attached to every particular document object.

DHTML document interfaces give you direct access to the script engine through the Script property. This one can be accessed through the IHTMLDocument and IHTMLDocument2 interfaces (the second inherits from the first, which is rare in this kind of library). The fact that this property is the only thing exported by IHTMLDocument seems to indicate (it's just a guess) that in the "pre-historic" times (i.e. before IE version 4) all the non-script access to the pre-DHTML object model passed by this property, that is to say that the IE contents was only manipulated as underlying scripting objects.

But what kind of object is that? As is often the case in the MSHTML library, its marshaled type is object and the underlying COM is an IDispatch pointer. When we explore this a bit more precisely, we discover that this "automation" COM object has its list of callable methods modified every time a function is declared in the code it interprets. This means that every "function" (if we use JavaScript terminology) can be seen from outside the script engine as an ordinary COM method of the engine object. Thanks to marshalling, we can call it from .NET too!

How do we call a method of an object without knowing in advance which methods this object exports? Doesn't this mean we are unable to declare it in C#? To get around this, we'll use the interoperability technique called "late binding". For any object, you can call the GetType method to return the type object. In the case of objects marshaled from COM, this object contains all the information you need to access any underlying object's method or property at runtime. This is done by simply giving its literal name.

This information should be sufficient to write a little example. First create a simple HTML file with the following contents:

<html>
<head><script>
function Bonjour(){
alert("Hello world");
}
</script></head>
<body>
Test file
</body>
</html>

Save this file in the same directory as your executable (it should be the bin/Debug subdirectory of your project folder) as test.htm. In our C# main form example, change the code of the handler that navigates to our pre-defined home page (normally button2_click). You'll need to load our test HTML file at this point. To do it, you have to modify your Navigate call like this:

Navigate("file:///"+Application.StartupPath+"\\test.htm");

Now, once again, we have to modify our DocumentComplete handler in order to launch a script call when the test file is loaded (you'll also need to add the using System.Reflection directive to your form code file at this point):

if(axWebBrowser1.LocationURL.EndsWith("test.htm"))
{
object
            script=doc.Script;
script.GetType().InvokeMember("Bonjour",BindingFlags.InvokeMethod,null,script,null);
}

In this code, we first get the script object from our document. To do that, the script property in our DHTMLDoc object can be implemented like this:

public object Script
{
get{ return doc2.Script; }
}

The second line is the script call itself, which uses the InvokeMethod method of the Type class. In our case, it is translated to the COM Invoke method call, which in turn finds the method with the same name as the first parameter that was passed in the underlying script object, and then calls it:

Let's look at the InvokeMember function a little more closely. As we've already seen, its first parameter is the name of the method (or property) we are looking for. The second parameter says what we want to do with the named object found. In our case, we consider it is a method, and we call it. We don't really need the third parameter at the moment, and the fourth is the reference to the object on which we are making the call (don't forget InvokeMember is a function of the Type class, not the object 's). Most interesting to us is the fifth parameter. It allows passing parameters to the function we are calling. Its marshalled type is object[], which is equivalent to VARIANT SAFEARRAY in COM. SAFEARRAY is a COM standard type representing a resizable array that is capable of handling its size and the types of its elements.

So, when we need to pass parameters of some type to a script function, we just have to box the values in the object s and put these objects in an array. The function's return value can be retrieved as an InvokeMember 's return object value.

What about parameter and return types? C# is a strictly-typed language, and JavaScript, for example, has no types at all. More precisely it has one data type, the VARIANT. This type, marshalled as an object to .NET, is a sort of union structure to which we can write almost any data type. The condition is to always specify the type of the data that the variable holds. The mechanism is very similar to .NET boxing, that's why it allows a real equivalence between VARIANT COM variables and .NET object s.

An integer value in script will be translated to .NET as Int32, and a string as a String object. We can clearly show this by calling a simple JavaScript function which we'll add to our test.htm file:

function Sum(a,b){
return a+b;
}

We'll call this function twice to observe its behaviour when treating different variable types:

string show="";
object
            retval=script.GetType().InvokeMember("Sum",BindingFlags.InvokeMethod,null,
script,new
            object[]{ 2, 2 });
show+="Value : "+retval+", type :
            "+retval.GetType().ToString();
retval=script.GetType().InvokeMember("Sum",BindingFlags.InvokeMethod,
null,script,new
            object[]{ "2", "2" });
show+="\r\nValue : "+retval+", type :
            "+retval.GetType().ToString();
MessageBox.Show(show,"C# script
            call result");

The call result is self-explanatory:

That works fine with standard variable types. However, don't try to simply treat a JavaScript array as an object[]. It is not one, because in COM it is not directly encapsulated in a Safearray, which is equivalent to a .NET array. Marshalling arrays and other structured object types is a complex subject; it would take another case study entirely to properly explore it correctly.

The last thing we can do is encapsulate the script call to an easy-to-use function integrated in our simple object model. In C#, such function writing is straightforward; you just need to add the following lines to your DHTMLDoc class (after adding using System.Reflection to the .cs file headings, of course):

public object ScriptCall(string method,params object[] prm)
{
return Script.GetType().InvokeMember(method,BindingFlags.InvokeMethod,null,Script,prm);
}

We can use this function to implement a scripting functionality that we'll probably need: until now, we've always called functions, without trying to execute true code fragments. This last task is very easy, because we can directly access any standard script functions just like those we've written ourselves. Amongst these standard functions there is one called eval that simply executes the code passed as its only string parameter. We simply add one more function to the DHTMLDoc class to do it:

public object ScriptEval(string
            expr)
{
return ScriptCall("eval",expr);
}

That's all! If you now want to implement a more complex scripting model, including variables, complex objects management and more, it could be better to move the scripting functionality from the DHTML document wrapper to a separate encapsulating object, which can be, for instance, returned as a new value of the Script property of the DHTMLDoc class.

6 - Extending the DHTML model

In the previous section, we've seen how to use script code from C#. Now we'll try to go the other way, calling C# code from script. You might need it, for example, to give your script development staff (if they've not yet migrated to .NET) direct access to some system functionalities you implemented.

If you don't want to be subjected to the security constraints and other organizational issues related to the use of components directly integrated to HTML pages, the only "legal" way to extend the DHTML object model your environment uses is to implement the window.external object.

When a DHTML document is loaded in Internet Explorer, it exports a special interface called ICustomDoc. This exposes the SetHandler method, getting a reference to a "container" object as a parameter.

What kind of "container"? It is a COM object communicating with the outer world through the IDocHostUIHandler interface. There are a number of browser's behaviors, like a reaction to a shortcut key press or a contextual menu display that can be redefined through this interface. Before performing such an action, the Internet Explorer engine checks for the existence of an object set by a SetHandler call and by implementing IDocHostUIHandler, then calls the method that corresponds to the behavior in question. One of these customizable processing properties is the reference to the object that implements methods we can put behind the window.external scripting object. The Explorer object asks for this property by calling the GetExternal method of the IDocHostUIHandler interface.

The big problem is that the ICustomDoc and IDocHostUIHandler interfaces are not implemented in the MSHTML .NET library, so there is no way to directly treat them as .NET interfaces. That means we have to look at the interfaces' COM implementations and convert them to C#.

First, we'll try implementing and using the ICustomDoc interface; it seems to be a little simpler than the other one. Searching in the C++ COM headers (MsHtmlHst.h) we find the following definition:

MIDL_INTERFACE("3050f3f0-98b5-11cf-bb82-00aa00bdce0b")
ICustomDoc : public IUnknown
{
public:
virtual HRESULT STDMETHODCALLTYPE SetUIHandler(
/* [in] */ IDocHostUIHandler *pUIHandler) = 0;

};

We'll simply transcribe the same thing in C#, applying interoperability rules. C# attributes let us specify three important things that we can observe when reading a C++ definition. First, the interface we declare is a COM interface, so we need to use the interoperability engine to deal with the classes that implement it. Second, it is an IUnknown -style object, it is not an automation (IDispatch -style) interface. Third, and most important for us, we give the GUID (globally unique identifier) to identify this interface.

Create a new C# class file, name it UIHandler.cs, and declare in its headers the use of the System.Runtime.InteropServices, System.Windows.Forms and mshtml namespaces (we'll also use this class file for the other classes of this section). First make the following interface declaration, which is the C# transcription of the earlier C++ statement:

[ComImport(),
            InterfaceType(ComInterfaceType.InterfaceIsIUnknown),
GuidAttribute("3050f3f0-98b5-11cf-bb82-00aa00bdce0b")]
public
            interface ICustomDoc
{
[PreserveSig]
uint
            SetUIHandler(IDocHostUIHandler pUIHandler);
}

We saw earlier that this interface is exported by the DHTML document object. In .NET, querying for an exported interface of an object simply means converting this object type to this interface type. So, if we want to obtain this interface from our document, we should simply put the following line at the beginning of the DocumentComplete header:

ICustomDoc cust=(ICustomDoc)(axWebBrowser1.Document);

How can we convert the object to a type which has no connection from the point of view of the C# language declarations? As the type is not referenced in the .NET libraries managing Internet Explorer functionalities, there should be a casting error at this point. In reality, there isn't. Don't forget we're working with interoperability libraries. So, our ICustomDoc interface is rightly considered to be related to a COM object (remember, we put the ComImport attribute in its declaration).

That means querying this interface is simply translated to a COM QueryInterface call (asking the underlying COM object for an appropriate interface) for which the only important thing is the interface's GUID. But we already declared this GUID in the C# interface's attributes! So, when we're casting this interface type, the interoperability engine connects through a COM interface identified to the underlying object and maps its methods to the corresponding .NET interface signature.

You can declare and use any COM object accessible via its interfaces in this way. You can even create your own definitions for the DHTML objects you'll use after reading this case study. Sometimes, you'll even need to do it: not all the interfaces in MSHTML are well implemented. Be aware however, the GUID corresponding to every interface is the most important thing, but when describing the interface itself, you also must follow (even if you use attribute modifying marshalling rules) the order of property and method declarations, and you must preserve the general interface signature, i.e. the names of the interface members and at least the size of their arguments and return values types.

We can now describe the IDocHostUIHandler interface which is relatively more complex, but the declaration work in itself is straightforward. We just have to translate the definitions between C++ (the same MsHtmlHst.h header we used to analyze ICustomDoc) and C#. We first declare the enumerations and structures the interface uses in order to preserve the "COM style" (i.e. the enumerations values and secondary type names), when implementing the interface:

public enum HRESULT : uint
{
S_OK = 0,
E_UNEXPECTED = 0x8000ffff,
E_NOTIMPL = 0x80004001,
E_OUTOFMEMORY = 0x8007000e,
E_INVALIDARG = 0x80070057,
E_NOINTERFACE = 0x80004002,
E_POINTER = 0x80004003,
E_HANDLE = 0x80070006,
E_ABORT = 0x80004004,
E_FAIL = 0x80004005,
E_ACCESSDENIED = 0x80074005
}

[ StructLayout( LayoutKind.Sequential )]
public struct DOCHOSTUIINFO
{
uint cbSize;
uint dwFlags;
uint dwDoubleClick;
[MarshalAs(UnmanagedType.BStr)] string pchHostCss;
[MarshalAs(UnmanagedType.BStr)] string pchHostNS;
}

[StructLayout( LayoutKind.Sequential )]
public struct tagMSG
{
IntPtr hwnd;
int lParam;
uint message;
tagPOINT pt;
uint time;
uint wParam;
}

Now, let's declare the interface itself. The only method we'll have to directly deal with now is GetExternal ; however, as it was shown earlier, the interface signature absolutely has to be preserved, so we need to declare all the methods that can be implemented for this interface. Note the use of the PreserveSig attribute for every interface member. This is very important, because it is the only possibility we can use in order to have a direct correspondence between COM and .NET method declarations. Without this attribute, the .NET return value is translated, if it isn't void, to the last argument of the marshaled COM function, this argument being simply referenced as retval -style.

In our case, we need to return a real COM return value, which is of the HRESULT enumeration type and which normally indicates the success of function call. This is assumed to always be successful when dealing with .NET interoperability without specifying PreserveSig.

The complete interface declaration is as follows:

[ComImport(), InterfaceType(ComInterfaceType.InterfaceIsIUnknown),
GuidAttribute("bd3f23c0-d43e-11cf-893b-00aa00bdce1a")]
public interface IDocHostUIHandler
{
[PreserveSig]
HRESULT ShowContextMenu(uint dwID, ref tagPOINT ppt,
[MarshalAs(UnmanagedType.IUnknown)] object pcmdtReserved,
[MarshalAs(UnmanagedType.IDispatch)]object pdispReserved);
[PreserveSig]
HRESULT GetHostInfo(ref DOCHOSTUIINFO pInfo);
[PreserveSig]
HRESULT ShowUI(uint dwID, [MarshalAs(UnmanagedType.Interface)] IntPtr
pActiveObject, [MarshalAs(UnmanagedType.Interface)] IntPtr
pCommandTarget, [MarshalAs(UnmanagedType.Interface)] IntPtr pFrame,
[MarshalAs(UnmanagedType.Interface)] IntPtr pDoc);
[PreserveSig]
HRESULT HideUI();
[PreserveSig]
HRESULT UpdateUI();
[PreserveSig]
HRESULT EnableModeless(int fEnable);
[PreserveSig]
HRESULT OnDocWindowActivate(int fActivate);
[PreserveSig]
HRESULT OnFrameWindowActivate(int fActivate);
[PreserveSig]
HRESULT ResizeBorder(ref tagRECT prcBorder,
[MarshalAs(UnmanagedType.Interface)] IntPtr pUIWindow, int
fRameWindow);
[PreserveSig]
HRESULT TranslateAccelerator(ref tagMSG lpmsg, ref Guid pguidCmdGroup,
uint nCmdID);
[PreserveSig]
HRESULT GetOptionKeyPath(ref string pchKey, uint dw);
[PreserveSig]
HRESULT GetDropTarget([MarshalAs(UnmanagedType.Interface)] IntPtr
pDropTarget, [MarshalAs(UnmanagedType.Interface)] ref IntPtr
ppDropTarget);
[PreserveSig]
HRESULT GetExternal([Out,MarshalAs(UnmanagedType.IDispatch)] out object ppDispatch);
[PreserveSig]
HRESULT TranslateUrl(uint dwTranslate, ref ushort pchURLIn, IntPtr
ppchURLOut);
[PreserveSig]
HRESULT FilterDataObject(IDataObject pDO, ref IDataObject ppDORet);
}

Once the interface is declared, we have to implement it. The only method we really need to implement now is GetExternal. However, if we want to have a class we can effectively instantiate as a result, we have to implement all the interface's methods.

This is exactly the reason for preserving the COM HRESULT return values of the interface's methods. For the methods implementing the behaviors we don't want to change, we have to specify to the caller that these are not customized, by returning the HRESULT.E_NOTIMPL value. Otherwise, Internet Explorer will think that our interface has already done all the work needed and will also block its default behaviors, for example by showing no context menu at all when the user right-clicks the browser window.

If you now look at the GetExternal method declaration, you'll see that the only thing it is intended to do is set its unique argument's value to the pointer to an object (IDispatch, which normally means any .NET object marshaled to COM) implementing the methods we'll access later from script through the window.external reference. Of course, if we have no such pointer, the return value must indicate that the behavior is not implemented. So, if we integrate the pointer, we need to return GetExternal to the class construction logic. Our implementation should look like this:

public class UIHandler : IDocHostUIHandler
{
protected IExternal ext;

public UIHandler(){ ext=null; }

public UIHandler(IExternal e){ ext=e; }

public virtual HRESULT ShowContextMenu(uint dwID, ref tagPOINT ppt,
[MarshalAs(UnmanagedType.IUnknown)] object pcmdtReserved,
[MarshalAs(UnmanagedType.IDispatch)]object pdispReserved)
{
return HRESULT.E_NOTIMPL;
}

public virtual HRESULT GetHostInfo(ref DOCHOSTUIINFO pInfo)
{
return HRESULT.E_NOTIMPL;
}

public virtual HRESULT ShowUI(uint dwID, [MarshalAs(UnmanagedType.Interface)]
IntPtr pActiveObject, [MarshalAs(UnmanagedType.Interface)] IntPtr
pCommandTarget, [MarshalAs(UnmanagedType.Interface)] IntPtr pFrame,
[MarshalAs(UnmanagedType.Interface)] IntPtr pDoc)
{
return HRESULT.E_NOTIMPL;
}

public virtual HRESULT HideUI()
{
return HRESULT.E_NOTIMPL;
}

public virtual HRESULT UpdateUI()
{
return HRESULT.E_NOTIMPL;
}

public virtual HRESULT EnableModeless(int fEnable)
{
return HRESULT.E_NOTIMPL;
}

public virtual HRESULT OnDocWindowActivate(int fActivate)
{
return HRESULT.E_NOTIMPL;
}

public virtual HRESULT OnFrameWindowActivate(int fActivate)
{
return HRESULT.E_NOTIMPL;
}

public virtual HRESULT ResizeBorder(ref tagRECT prcBorder,
[MarshalAs(UnmanagedType.Interface)] IntPtr pUIWindow, int
fRameWindow)
{
return HRESULT.E_NOTIMPL;
}

public virtual HRESULT TranslateAccelerator(ref tagMSG lpmsg, ref Guid
pguidCmdGroup, uint nCmdID)
{
return HRESULT.E_NOTIMPL;
}

public virtual HRESULT GetOptionKeyPath(ref string pchKey, uint dw)
{
return HRESULT.E_NOTIMPL;
}

public virtual HRESULT GetDropTarget([MarshalAs(UnmanagedType.Interface)]
IntPtr pDropTarget, [MarshalAs(UnmanagedType.Interface)] ref IntPtr
ppDropTarget)
{
return HRESULT.E_NOTIMPL;
}

public virtual HRESULT GetExternal([Out,MarshalAs(UnmanagedType.IDispatch)] out object ppDispatch)
{
if(ext==null)
{
ppDispatch=null;
return HRESULT.E_NOTIMPL;
}
ppDispatch=ext;
return HRESULT.S_OK;
}

public virtual HRESULT TranslateUrl(uint dwTranslate, ref ushort pchURLIn, IntPtr ppchURLOut)
{
return HRESULT.E_NOTIMPL;
}
public virtual HRESULT FilterDataObject(IDataObject pDO, ref IDataObject ppDORet)
{
return HRESULT.E_NOTIMPL;
}
}

The last thing we have to do is to implement the object we'll access through window.external and to connect it to the browser by using our new UIHandler object.

The C++ documentation concerning the GetExternal method indicates that the object whose pointer it returns has simply to implement the IDispatch interface to be usable. However, normally, all the C# classes, when marshaled to COM, are IDispatch type. Does it mean we can pass any C# object pointer to the constructor of the UIHandler class to make its methods and properties accessible through window.external ? It seems logical, but it doesn't work. The problem is that if we want an object behaving as a true automation object, it's not enough to implement IDispatch, it's also necessary to implement it well. It means we have to force our object to export an interface with all its methods and properties declared.

There is a trick in C# that allows us to easily do it. It's enough to declare an empty interface, declare it as IDispatch -type, and implement it in any class. This effectively makes all this class's methods exported and accessible to COM. The interface declaration may look like this:

[InterfaceType(ComInterfaceType.InterfaceIsIDispatch)]
public interface IExternal{}

Any class implementing this interface will be considered, at the COM point of view, as a fully-featured automation interface. This means the interoperability engine will be forced to analyze the class's signature and to add, at the COM side, an automatic implementation of everything an automation client needs to access the methods.

We need all this because the IE script engine uses the 'late binding" method; the relation between a concrete property or method name and its implementation's address is established at run time, so we must force the interoperability engine to avoid any shortcuts in the name translations.

We can now declare any class (even our main application's form) as implementing this interface, and then use all its methods from the script. As an example, we'll implement a simple class with one method implementing a message box, allowing the script user to define text for both the box and its header:

public class CExternal : IExternal
{
public void Alert(string text,string head)
{
MessageBox.Show(text,head);
}
}

This class implements our empty interface declared as InterfaceIsIDispatch, so when marshaled to COM, it is seen as a new automation object interface. The function asking at runtime for its member's addresses returns, when it receives the "Alert" string, the entry point allowing it to call the right function.

Note that we have absolute freedom, as there are no methods or properties declared in the interface implemented; we could do everything we want in the implementing class, every method or property we declare will be marshaled and its name will be immediately available to use for any automation client.

To connect the interface handler to the browser, we have to add the following code after the line requesting the ICustomDoc interface we wrote at the beginning of this section:

cust.SetUIHandler(new UIHandler(new CExternal()));

The constructor of the UIHandler class (the class that implements IDocHostUIHandler) receives as an argument an address of a new instance of the class designed to be used in the GetExternal function. The result of all this is passed to the appropriate method of the ICustomDoc interface which declares our new object as a new "container" authorized to modify the browser user interface's behaviour.

Now we'll change our test.htm file to test our implementation. We add the following DHTML code to its body:

<hr/>
<button onclick="window.external.Alert('Extending DHTML','window.external')">
Test window.external</button>

You can now compile your application and run a test by pushing the button on our "home" page:

As you noticed when looking at the IDocHostUIHandler interface, implementing extensions for the DHTML object model is only one of the numerous functionalities you can modify by using it. So, all the work done will serve you in the future as a starting point for writing complex applications using all the power of the Internet Explorer programming platform.

Case study review

As was stated at the beginning of this case study, the subjects covered here are just intended to give you a starting point to begin a serious study of the Internet Explorer browser platform from the C# point of view. The possibilities offered by the platform are endless. The fact that the (D)HTML platform is very popular thanks to the general success of the Web makes this platform very attractive to user interface designers, so the programmers who have to manage application presentations layers should consider this platform as a serious alternative to traditional Windows-based interface development. Deep knowledge of the leading DHTML presentation platform is important for both standalone and online applications programming; the better we know the client interface capabilities, the better the final user's experience and the wider the success of your software.

Programming Internet Explorer is an extremely wide topic; it can't really be covered by a single case study. However, this piece explains the main starting points you need to explore first. Looking at the browser as a .NET component, we observed it's most important functionalities related to internet navigation. Going further in this direction, and studying all the methods and properties exported by the component, we can create a personalized framework implementing almost all the functionality accessible through the standard Internet Explorer interface, and even make it more flexible \ better adapted to your particular tasks.

The true advantage of the platform is the access it gives to the DHTML object model. Many years of development at the front edge of new technologies has made it perfectly adapted to developing any kind of user interface, and especially those concerning document-based systems. Having read this case study you have learned how to access this model; you followed the development of basic classes encapsulating its most important objects in the .NET manner. Now, you can continue its exploration, extending these classes and using them: you'll be surprised how easy is it to write DHTML code in C# once some basic interfaces and properties have been encapsulated. By putting together this case study's information about accessing DHTML, and reading the widely accessible documentation about DHTML scripting, you can combine the ease of the Internet Explorer presentation model with the power of the .NET platform in your applications.

Finally, the example of the DHTML model extension is just the first step into the world of advanced Internet Explorer interfaces. Almost everything in the IE-related objects' behaviour can be personalized. Now, .NET gives access to this power to all the industrial programmers who don't have the time to write and debug low-level C++ code, but who wish to create really powerful interfaces

Further work

If you're courageous enough to explore a library containing hundreds of interfaces, you can open the Object Browser window in a project which references the MSHTML library. Open the microsoft.mshtml/mshtml node and take a look:

Sometimes, by clicking on an interface or on an interface's function you can get a small piece of helpful text on it; however, the true .NET documentation on the subject is actually non-existent! You'll be better off using the corresponding C++ help. All the interfaces and function names are the same.

There are many other interesting domains to explore. It could be the markup interfaces system, giving you direct access to the engine Internet Explorer uses to interpret and display HTML code, or it could be the elements' behaviors, Internet Explorer automation, style sheets, helpers, or web accessories. The possibilities of the platform are endless. Enjoy it!


An Intelligent Navigation Bar Using C# and XML

An Intelligent Navigation Bar Using C# and XML
http://www.csharptoday.com/content.asp?id=1821
Article Rating: 3
Published on February 14, 2002
Abstract
One of the fundamental goals of web design is easy navigation. A dynamic navigation bar (one that changes depending on which page the user is) helps to do exactly that, in addition to lending a polished look to your site. For designers, this involves the time-consuming and laborious process of changing HTML code on each page. In this article, Akshay Luther demonstrate the use of an ASP.NET custom web control (coded in C#) along with XML and XSL to create an intelligent navigation bar that lends itself to easy reuse, highlighting a number of key aspects of the .NET framework along the way.
Article

One of the fundamental goals of web design is easy navigation. A dynamic navigation bar (one that changes depending on which page the user is) helps to do exactly that, in addition to lending a polished look to your site. For designers, this involves the time-consuming and laborious process of changing HTML code on each page.

In this article I demonstrate the use of an ASP.NET custom web control (coded in C#) along with XML and XSL to create an intelligent navigation bar that lends itself to easy reuse, highlighting a number of key aspects of the .NET framework along the way.

Development Setup

The code in this article was developed using Internet Explorer 6, a text editor, and the .NET Framework SDK Version 1.0 (downloadable at http://www.asp.net/download.aspx?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq.) It also works with .NET Beta 2 and release versions of VS.NET

This article assumes familiarity with C#, ASP.NET, XML and XSL.

A Brief Look at .NET Server Controls

ASP.NET allows us to a) truly separate presentation from logic and b) reuse code. This is made possible via server controls, which are a welcome addition to ASP.NET since these features were glaringly absent from traditional ASP. No doubt you are already familiar with the built in server controls: Web Form controls that facilitate event-driven programming, and Rich controls such as the Calendar control, List controls, and Validation controls. Extensive as they are, we are limited to the functionality they provide.

To really leverage the power of the .NET framework, we can incorporate our own functionality into controls by creating user controls and custom controls. They are used just like the built-in server controls but with a different tag prefix, such as <MyShoppingSite:ShoppingCart ... /> as opposed to <asp:button ... />. They are a) easy to deploy and use, b) transparent (support the black-box model) and c) can be quite powerful since they are coded in .NET languages. They are also compiled, which results in greater performance and scalability.

User controls are essentially ASP.NET code fragments contained in .ascx files (in addition to optional .asc.cs code behind files) and are employed to reuse existing ASP.NET code. Custom controls are more powerful than user controls as the developer has complete control over the rendered HTML output and they are entirely self-contained in assemblies. There is more than one type of custom control (depending on which .NET class the control is derived from), however, there are only minor differences between them. I will focus on the creation of a custom web control, which is meant to be used for rendering HTML output. Custom web controls are derived from the class System.Web.UI.WebControls.WebControl.

Design Overview

The following were the design objectives when I created the intelligent navigation bar control.

  • Ease of reuse
  • Ease of customization
  • Flexible deployment (support multiple navigation bars on one site and support multiple sites)

All of these are enabled by the underlying .NET architecture to various degrees. Custom web controls are utilized via XML compliant tags and hence are simple to use. In addition, the other design objectives are met by the use of XML/XSL and the attributes passed to the control.

Having established the basic design objectives, I'll briefly comment on the core function of the control, which is intended to be used identically on each page containing the common navigation bar. After being provided with some attributes (discussed later) by the page that invokes it, it loads a set of navigation links (located in an XML file) to build the navigation bar with. It then determines which of those links is represented by the current page and flags the link as 'active'. Finally the navigation bar is rendered by transforming the altered XML using the corresponding XSL stylesheet.

The following XML file is used in the sample application and demonstrates the structure of XML files to be used with the intelligent navigation bar control:

<links>
<link text="#News"
            url="news.aspx"/>
<link text="#Resources"
            url="resources.aspx"/>
<link text="#Articles"
            url="articles.aspx"/>
<link text="#Codebase"
            url="codebase.aspx"/>
<link text="#Forums"
            url="forums.aspx"/>
</links>

This is what the final XML looks like just before being transformed, if for example the current page is articles.aspx :

<links>
<link text="#News"
            url="news.aspx"/>
<link text="#Resources"
            url="resources.aspx"/>
<link text="#Articles"
            url="articles.aspx" active="true">
<link text="#Codebase"
            url="codebase.aspx"/>
<link text="#Forums"
            url="forums.aspx"/>
</links>

Flexible deployment (i.e. support for multiple navigation bars on one site and support for multiple such sites) is enabled by the two attributes passed to the control. To clarify things, here is a sample usage of the control on an ASP.NET page:

<CSharpToday:Inavbar runat="server" Template="sample"AppRoot="templates" />

An explanation of the attributes follows.

Template

The template specifies which XML and XSL files are to be used. The XML file contains the links and the XSL file is used to render the navigation bar. If for example, the value sample was set (such as in the sample site), the two files used would be sample.xml and sample.xsl.

AppRoot

This specifies the relative path to the directory that contains the relevant XML and XSL files as indicated by the Template attribute. For example, if the value templates was assigned as in the example, the control would look for the XML and XSL files in the templates subdirectory of the current directory. This attribute is optional, and if left out, the control assumes the current directory, which is equivalent to AppRoot=".".

If another site (a set of pages containing a different common navigation bar) were to be set up, both the values would be changed to reflect the location of the new site and the names of the XML and XSL files. If multiple navigation bars are to be set up for a common set of pages, the value of the AppRoot attribute would be the same, but the value of the Template attribute would be different for the second navigation bar.

The Inavbar Control

using System;
using System.Web;
using System.Web.UI;
using System.Web.UI.WebControls;
using System.Xml;
using System.Xml.XPath;
using System.Xml.Xsl;
using System.Text;
using System.IO;

namespace CSharpToday {
public class Inavbar : WebControl {
...

The first bit of code references the appropriate namespaces and begins the definition of the control within the CSharpToday namespace. The important thing to note is that the control class inherits from the System.Web.UI.WebControls.WebControl class.

    ...
protected
            String HtmlOutput;
protected String _Template;
protected
            String _AppRoot;

public String Template {
get { return
            _Template; }
set { _Template = value; }
}

public String
            AppRoot {
get { return _AppRoot; }
set { _AppRoot = value;
            }
}
...

A protected member variable, HtmlOutput, is declared to contain the final HTML output prior to rendering. Two properties are also declared to correspond with the attributes passed from the ASP.NET page that uses the control. One of the wonderful features of custom controls in ASP.NET is that the attributes specified in the control are intelligently mapped to the corresponding properties by ASP.NET at runtime (whether the property is a string, numerical, boolean or enumeration data type).

...
protected override void RenderContents(HtmlTextWriter writer) {
BuildHtmlOutput();
writer.Write(HtmlOutput);
}
...

Next, the RenderContents method of the inherited WebControl class is overridden to suit our purpose, which is to ascertain (via the BuildHtmlOutput method) and then display the appropriate HTML output.

The BuildHtmlOutput method performs the bulk of the work and is discussed next:

    ...
protected
            void BuildHtmlOutput() {
// discover absolute path to xml and xsl
            files
_AppRoot = Page.MapPath(_AppRoot) + "\\";

//
            extract current page name from url
String CurPage;
CurPage =
            (string) Context.Request.Url.AbsoluteUri;
CurPage =
            CurPage.Substring(CurPage.LastIndexOf("/"), CurPage.Length -
CurPage.LastIndexOf("/"));
CurPage =
            CurPage.Remove(0,1);

// retrieve xml file and turn on active
            attribute for current page
XmlDocument Links = new
            XmlDocument();
Links.Load(_AppRoot + _Template +
            ".xml");
XmlNodeList CurLink =
            Links.DocumentElement.SelectNodes("link[@url='"
+ CurPage +
            "']");
if (CurLink.Count > 0) {
XmlAttribute Flag =
            Links.CreateAttribute ("active");
Flag.Value = "true";
foreach
            (XmlNode Match in CurLink)
            {
Match.Attributes.Append(Flag);
}
}

// transform
            xml and store result for output to browser
StringBuilder sb = new
            StringBuilder();
StringWriter sw = new
            StringWriter(sb);
XslTransform ObjXslt = new
            XslTransform();
ObjXslt.Load(_AppRoot + _Template +
            ".xsl");
ObjXslt.Transform(Links, null, sw);
HtmlOutput +=
            sb.ToString();
}
}
};

The code of the BuildHtmlOutput method is fairly straightforward and is performed in four steps.

  • Discover the absolute path to the XML and XSL files
  • Extract the current page name from the URL
  • Retrieve the relevant XML file and turn on the active attribute for the current page
  • Transform the XML using the relevant XSL file and store the result

One point to note is the way the current page's URL is extracted. The Context property of the control provides access to the ASP.NET intrinsic objects such as Application, Session, Request and Response. It is used in the same manner as the Page.Context property. In this instance it is used to discover the absolute URI of the current page.

Compiling the Control

Code compilation is simplified by the creation of a batch file compile.bat :

@echo off
set outdir=..\bin
set outfile=inavbar.dll
set infile=inavbar.cs

rem -= if necessary prefix with path to csc.exe =-
csc /t:library /out:%outdir%\%outfile% %infile%

echo.
echo.
pause

The source files is compiled as a .dll and is stored in wwwroot\bin (the current directory is assumed to be a subdirectory of wwwroot) on compilation.

When a reference to the inavbar control is encountered on an ASP.NET page, the ASP.NET runtime will encounter the control in the bin directory since this is one of the places it searches.

Using the Control

Before a control can be used on an ASP.NET page, it must be registered via the @Register page directive. The following code (used on every page of the sample site) demonstrates how this is done:

<%@ Register TagPrefix="CSharpToday"
            Namespace="CSharpToday"
Assembly="inavbar" %>

Essentially, this line indicates to the compiler that a) all tags with the prefix CSharpToday should be matched with classes of the same name defined within the CSharpToday namespace and b) all such classes are located in the assembly named inavbar.

This is how the control is used in an ASP.NET page:

<CSharpToday:Inavbar runat="server" Template="sample"/>

This bit of code is inserted into each page where the navigation bar is supposed to appear. Note that I have left out the AppRoot attribute since the XML and XSL files are in the same directory.

Application Setup

To set up the application, create a subdirectory in the 'Default Web Site' home directory (on my server, it is c:\inetpub\wwwroot\inavbar). Unzip the accompanying code files into this directory. Modify compile.bat to reflect the location of this directory. Next, create a directory called bin under wwwroot if it does not already exist. Compile the control by running compile.bat and verify that a file called inavbar.dll exists in wwwroot\bin. Finally, navigate to any of the .aspx files to view the sample application at work.

Sample Application

In the sample application, each page is identical except for a bit of text that describes the content of the page:

Note how the current page (News) is highlighted in black.

Conclusion

Let us revisit the design objectives to examine how the intelligent navigation bar control has met them. In order to deploy for another website or navigation bar, the following steps need to be carried out.

  • Change the attributes (Template and AppRoot) to reflect the new navigation bar and insert the control into each page that is to contain the navigation bar.
  • Modify the .XML file to reflect the pages that the navigation bar is to include.
  • Modify the XSL transform (and/or the Cascading Stylesheet) file to reflect the new (design.

As you can see, the control is easy to use and reuse (only the tag attributes need to be modified), easy to customize (only the XSL transform and Cascading Stylesheets need to be modified) and is flexible in deployment since the two attributes of the control make multiple navigation bars possible on each page in addition to allowing multiple such sites.

What is important to note is the activities mentioned in the preceding paragraph can be carried out by web designers. ASP.NET custom controls and indeed ASP.NET in general empower web designers in a way that hasn't been possible before.

The control itself can be extended in many ways. Let us take a step back and analyze it at the most basic level to discover how it may be extended. To use the intelligent navigation bar control, we only have to include bit of code in order to initiate it from each page. We also store data about each page in a datastore (the XML file). It automatically identifies the current page and subsequently which data to work with. It then performs some action on the corresponding data and transforms the data in a useful way. Thus it can be used for any application that requires different content to be displayed on each page in a common format. Simple examples would be HTML titles and page headings. It could even be used as a very simple content management system, where the XML file can be edited by someone without any web design skills.

We have just scratched the surface of what is possible with custom controls. In terms of features, one can build composite controls that consist of other controls. There are also different ways to initialize composite controls on an ASP.NET page. One can also include design-time support for Visual Studio.NET in custom controls such as design view representation, toolbox support, property grid support, IntelliSense support, etc.

Article Information
Author | Akshay Luther
Technical Editor | Adam Ryland
Project Manager | Helen Cuthill
Reviewers | Saurabh Nandu, Neil Piggot

If you have any questions or comments about this article, please contact the technical editor.


Localization using .NET and C#

Localization using .NET and C#
http://www.csharptoday.com/content.asp?id=1723
Article Rating: 4.5
Published on December 21, 2001
Abstract
Developing applications that are targeted towards a wide range of people brings with it several problems: different cultures, languages, currency formats, date formats, number formats, calendars etc. Some questions arise - is it possible to develop a single desktop application or website that can be understood by different groups of people who speak different languages? Is it possible to develop an application in which the user interface varies from language to language with some specific things like currency remaining common for all users (Example: we want all the currency transactions in our application or site using US dollars, but the contents need to be presented in the user’s local language). The answer to these questions is, yes, it is possible, and has been since the introduction of Win32 – but .NET’s built–in classes and utilities make our work simpler and more efficient. In this article, Saravana Kumar looks at Localization using .NET classes.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

Developing applications that are targeted towards a wide range of people brings with it several problems: different cultures, languages, currency formats, date formats, number formats, calendars, etc. Some questions arise:

  • Is it possible to develop a single desktop application or website that can be understood by different groups of people who speak different languages?
  • Is it possible to develop an application in which the user interface varies from language to language with some specific things like currency remaining common for all users (Example: we want all the currency transactions in our application or site using US dollars, but the contents need to be presented in the user's local language).

The answer to these questions is, yes, it is possible, and has been since the introduction of Win32 - but .NET's built-in classes and utilities make our work simpler and more efficient.

The Microsoft Windows International Division defines two areas:

  • Globalization: Coding areas include currency, date/time, number formatting, etc. (For more information refer to the article written by Matthew Reynolds "Globalization using C#" at http://www.csharptoday.com/content.asp?id=1702)
  • Localization: Coding areas include not hard coding strings, no text in graphics, not hard coding error messages, etc.

In this article we are going to look at Localization using .NET classes. Lets us begin with a simple Hello World example. Look at the code below in listing 1. It just prints a string Hello World on the console.

Listing: 1

using System;

public class
            HelloWorld{
public static void Main()
            {
Console.WriteLine("Hello World");
}
}

In the above code snippet the string Hello World is hard coded. The disadvantage of this is if somebody who speaks a different language executes this program, it will still display the same text in English. We need a more intelligent way to present the text, so that the string is not hard coded.

Instead of hard coding the text as shown above we can keep all our text messages in a separate file (called a resource file).

Follow the steps below, which use localization classes to do the same task of the Hello World program discussed above.

Step 1:

Create a text file called HelloWorldStrings.txt, include the following line and save it.

HelloWorld=Hello World

Step 2:

On the command prompt enter the following command (make sure your path variable is set to run command line tools and compilers provided by .NET)

Resgen HelloWorldStrings.txt

Running this command will create a binary resource file (.resources) called HelloWorldStrings.resources

(Resgen is explained in more detail later under the heading "Resgen and AL utility")

Step 3:

Create a C# file called HelloWorldLocalized.cs and include the following code:

Listing: 2

using System;
using
            System.Resources;
using System.Reflection;
using
            System.Globalization;
using System.Threading;

public class
            HelloworldLocalized{
//create a resource manager instance so
            that
//we can get all available resource by specifying
//the
            key
static ResourceManager rm = new
            ResourceManager("HelloWorldStrings",Assembly.GetExecutingAssembly());
//Create
            a new culture info object
CultureInfo ci = new
            CultureInfo(CultureInfo.CurrentUICulture.Name);

public static
            void Main(string[] args) {
HelloworldLocalized objMain= new
            HelloworldLocalized();

if (args[0].Length !=
            0){
objMain.ci = new CultureInfo(args[0]);
//set current
            thread culture to cultureinfo instance we
            created
Thread.CurrentThread.CurrentUICulture=objMain.ci;
}
Console.WriteLine(rm.GetString("HelloWorld"));
}
}

Resourcemanager class

The most important class we used in our HelloWorldLocalized sample is the ResourceManager class. The ResourceManager class is used to get resources for the current culture from resource assemblies. ResourceManagers are expensive to use, so we may want to declare only one, and declare it at the class level. Further, a static (Shared) version of the ResourceManager can be used by many instances of our class, saving even further on memory resources. The below code snippet shows the usage of the ResourceManager class inside our application:

///.
static ResourceManager rm = new ResourceManager("HelloWorldStrings",Assembly.GetExecutingAssembly());
///.

In the above code snippet the first argument of the constructor takes the root name of the resource file and the second argument is the main assembly. We need to identify a particular Assembly to base the ResourceManager on.

The ResourceManager class has functions to retrieve the resources like GetString, GetObject, etc. In our example we used the GetString method to retrieve our HelloWorld string.

Step 4:

Compile our C# source file HelloWorldLocalized.cs using the command line compiler (csc.exe) with the /res option (to embed the resource file) as shown below:

csc /res: HelloWorldStrings.resources HelloWorldLocalized.cs

After executing the above command it will create an executable file called HelloWorldLocalized.exe. Let's view the assembly manifest of HelloWorldLocalized.exe using the Intermediate language disassembler tool (ildasm.exe) as shown below:

ildasm HelloWorldLocalized.exe

It will look something like this:

The lines .mresource public HelloWorldStrings.resources{} tell CLR about the embedded or linked resource to the main assembly.

Functionality wise, the code in listing 1 and listing 2 are identical. Both will print Hello World on the console screen, but the code in listing 2 is more intelligent in that it can print the Hello World message in different languages (if resources are provided) based upon the command line argument passed to it. There is no need to alter or recompile the original source code. If we look at the code in listing 2 we can see the text Hello World is not hard coded, but is taken from a resource file using the ResourceManager class GetString method.

We would expect the output (including the French and German options included in the download) to appear as below:

Figure 1

But, at this point of time our HelloWorldLocalized.exe will work as shown in figure 2. It will print the same message for all languages because we have only created resources for the English language. Before going deeper into the subject by creating further resources, we will see some of the basic topics required for localizability.

Figure 2

Resources and assemblies

When developing an application for localizability we must use 3 types of assemblies. They are:

  • Main Assembly,
  • Feedback Assembly (or fallback assembly)
  • Satellite Assembly

Main Assembly:

When we create our application, it is part of an assembly, which is referred to as the 'main' assembly, since it is the central part of our application. Any code aspects of our application are part of this main assembly. If we change any code in this main assembly, we need to recompile the entire assembly, which can mean a lot of work redistributing our application. In our example above HelloWorldLocalized.exe is the main assembly.

Feedback Assembly:

In our above example we mentioned that for each language (en, fr, de, etc.) it produced the same result Hello World in English. This is because of the feedback assembly. Each application will have only one Feedback assembly, which will be picked up by default if there are no corresponding language specific resources present. Since we didn't create an individual resource file for French and German, when we try to run our application with French or German language it points back to the English language, which is because of our default or Feedback assembly. It's very important to have a feedback assembly inside our application so that at least something is displayed instead of throwing an exception. In our example HelloWorldLocalized.resources is the feedback assembly, this is nothing but the compiled version of our text resource file HelloWorldStrings.txt.

Use: Including a default resource file (Feedback assembly file) simply ensures that at least some reasonable value will be found, without causing an exception to be thrown.

Satellite Assemblies:

Satellite assemblies are used in applications to support language dependent strings. Satellite assemblies hold only resources and no code. They act as an external set of resources to our application. They are also compiled and they have their own assemblies. If we ever need to change information in a satellite assembly, we can do so without having to change the information in our main assembly. Satellite assemblies are compiled as dll's and are referenced by our main assembly. In our example, since we didn't create separate resource files for different languages, we have a single English resource that will act both as satellite as well feedback assembly. So, in our example the satellite assembly is HelloWorldLocalized.resources. The content of both feedback assembly and satellite assembly are same, the main difference between them is in their usage. During runtime, resources will be picked from the satellite assembly; if there is no satellite assembly for a particular language, then the resource will be picked up from the feedback assembly (default for application). If there is no feedback assembly then an exception will be thrown.

Let us now create satellite assemblies for French and German so that our application works as expected in figure 1. We will first work on French. The steps are exactly the same for German except of course for the text in the resource file. Follow the steps below:

Step 1: Create a subfolder called fr

Step 2: Create a text file called HelloWorldStrings.fr.txt and include the following line:

HelloWorld=Bonjour Monde

Step 3: Run the following command at the command prompt from inside the fr directory:

Resgen HelloWorldStrings.fr.txt

This will create a resource file called HelloWorldStrings.fr.resources

Step 4: Run the following command at the command prompt from inside the fr directory:

al /t:lib /culture:fr /embed:HelloWorldStrings.fr.resources /out:HelloWorldLocalized.resources.dll

Our application will now work for French. Notice we did not alter nor recompile our source code. The above command will create a compiled dll resource file, which will be referenced by our main assembly, the ResourceManager does this for us: we do not need any specific code to tell our program how to reference the resource assemblies (all we need to do is to take care of file naming conventions and place them in correct directory - this is explained under "Directory structure and naming convention" later). Any language support can be given to our application by just following the above steps. If we follow the same steps for German with appropriate text message, our application will work for the German language.

Culture and Region

Up to this point we mentioned en, fr, de, etc as different languages, but there are more specific technical terms for this - culture and region. The culture is a set of preferences based on a user's language and cultural conventions. The culture name defines a 2-letter language (en, fr, de, etc) and a 2-letter region code that follow the RFC 1766 hierarchy. You can see all the codes at http://www.dsv.su.se/~jpalme/ietf/language-codes.html?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq.

The region is different from the culture. The region does not represent the user's preferences, but defines the location. Example:

  • fr French (Culture - French)
  • fr-FR French - French (Culture - French , Region - France)
  • fr-CA French - Canadian (Culture - French , Region - Canada)

In our example, a culture of fr-FR and fr-CA will return the same result - Bonjour Monde - but there can be subtle differences between cultures that mean that to really do localization well, we need natives of that culture to perform our translations within the framework provided.

Directory Structure and Naming Convention

The directory structure is very important when dealing with localization in .NET. Our application must follow the same hierarchical directory structure as of a language hierarchy defined by RFC 1766 standard. If we misspell the name of the culture or regions, our application will throw an exception or it will not find the correct culture. The runtime will look for resources using predefined set of rules. So the directory structure should look like this:

In our example, the structure looks like the following:

The resources specific to French must be present inside the fr folder and the resource specific to German must be present inside the de folder. If we have a subculture like French-French, Canadian-French etc it will go as a subfolder inside the French folder fr. Example:

The resource files must be named like this:

  • <myresource>.resources (Example: HelloWorldStrings.resources)
  • <myresource>.xx.resources (Example: HelloWorldStrings.fr.resources)
  • <myresource>.xx-XX.resources (Example: HelloWorldStrings.fr-FR.resources)

The satellite assembly names must be defined in the format:

  • <myproject>.resources.dll (Example: HelloWorldLocalized.resources.dll)

Resgen and AL utility

Resgen is the "Resource File Generator" tool. It converts .txt files and .resx files into .NET common language runtime binary .resources files that can be embedded in a runtime binary executable or compiled into satellite assemblies. So far we used .txt files (HelloWorldStrings.txt) as our resource files, but we can also use a .resx file as a resource file. .resx is an XML formatted resource file. The advantage a .resx file gives us over a .txt file is that we can store object structures inside a .resx file, so long as the object is serializable. The kind of objects we might want to store includes fonts, or actual picture images such as bitmaps, etc.

Using the Resgen utility we can convert files from any one format to any other format. For example we can convert:

  • ".resources" files into ".txt" files or ".resx" files.
  • ".txt" files into ".resx" files or ".resources" files
  • ".resx" files into ".txt" files or ".resources" files

If we do convert this way, we must give Resgen the name of an output file; it will not work if we specify the input file only. In the later part of the article when we develop the same application using visual studio we will be using the .resx format.

Assembly Linker (al.exe) is a stripped down version of the main compiler. The main use of the assembly linker tool is to create assembly manifests, or create a single assembly from multiple module files. Apart from this it's also used to add resources to satellite assemblies as we did in the HelloWorldLocalized example.

Creating resource files programmatically

Until now we kept all our resource files in a text file (HelloWorldStrings.txt) and we converted that into binary form using the Resgen.exe utility provided by the framework. This manual approach will not be feasible at all times. The framework provides us with some base classes by which we can create our resources programmatically. Let us create the same HelloWorldStrings.resources programmatically:

Step 1:

Create a C# file called createResource.cs and include the following code:

using System;
using
            System.Resources;

public class resourceDemo{
public static
            void Main(){
//Create a resource writer class instance by
            specifying
//the .resource file name in the
            constructor
ResourceWriter objRw = new
            ResourceWriter("HelloWorldStringProg.resources");
//add resources
            as name value pair
objRw.AddResource("HelloWorld","Hello
            World");
objRw.Close();

}
}

Step 2:

Compile the createResource.cs file using the following command:

csc createResource.cs

This will result in an executable file called createResource.exe.

Step 3:

Executing the executable file createResource.exe will create a resource file called

HelloWorldStringProg.resources. This binary file is similar to the file HelloWorldStrings.resources, the one we created using the R esgen utility. We can make sure by converting the HelloWorldStringProg.resource s into a text file using Resgen. To do this we need to run the following command:

resgen HelloWorldStringProg.resources HelloWorldStringProg.txt

This will result in a text file (HelloWorldStringProg.txt), which looks similar to our original text resource file HelloWorldStrings.txt.

The above steps show us how to create a resource file programmatically. We used the ResourceWriter class to create a new resource file. We can read a resource file using the ResourceReader class. The below code listing prints all the entries inside the resource file on the console:

using System;
using
            System.Resources;
using System.Collections;

public class
            resourceDemo{
public static void Main(){
//Create a resource
            reader class instance by specifying
//the .resource file name in
            the constructor

ResourceReader objResReader = new
            ResourceReader("HelloWorldStringProg.resources");
//available
            resource can be accessed as a collection
foreach (DictionaryEntry
            objDictEntry in objResReader)
Console.WriteLine(objDictEntry.Key
            + ": " +
            objDictEntry.Value);

objResReader.Close();
}
}

Using Visual Studio .NET

Until now we've used only command-line utilities to create our samples. Even though there are many more steps involved, it's useful to know what's happening behind the scenes. We can create the same application using Visual Studio .NET very easily. Visual Studio .NET hides most of the complexities from the developers, such as its in-built tool to edit resource files. It converts the readable format of the resource file into a binary one. It also allows the insertion of objects such as pictures, fonts, etc. inside the resource file (Note: the object must be serializable). Let's follow the steps to do the same HelloWorldLocalized example in Visual Studio .NET:

Step 1:

Create a new Visual C# Console Application project and call it HelloWorldLocalized as shown in the below figure:

Step 2:

Delete all the default code in Class1.cs and paste the code from listing 2 (in the beginning of the article).

Step 3:

Right-click on the project HelloWorldLocalized in the solution explorer and click Add and then Add new item.

Step 4:

From the available templates select Assembly resource File and name it HelloWorldStrings.resx

This will create a HelloWorldStrings.resx file, which we can view in the solution explorer.

Step 5:

Double-click on the HelloWorldStrings.resx file. By default it will take you to the data view. Enter Hello World for the value field and HelloWorld for the name field as shown in the below figure.

Step 6:

Right-click on the project HelloWorldLocalized in the solution explorer and click Properties from the menu. Set up the command line option to en as shown in the below figure.

Step 7:

The only change we need to make in the source (Class1.cs) is to put the name of the project (HelloWorldLocalized) while instantiating the ResourceManager class. The snippet below shows this:

///.
static ResourceManager rm = new ResourceManager("HelloWorldLocalized.HelloWorldStrings",Assembly.GetExecutingAssembly());
///.

If we compile and run the application it will display Hello World on the console.

So, now we have the application working for the en culture. In order to make it work for another culture, for example fr , repeat the following steps:

  • Step 3,
  • Step 4: Give an appropriate new name, HelloWorldStrings.fr.resx
  • Step 5: Change the value to the corresponding French text. In our case it's Bonjour Monde
  • Step 6: Change the Command Line Argument to fr.

Run the application. Now, it will display Bonjour Monde on the console window. If we want our application to work for German then we can follow the same set of steps.

Winres utility

If we are designing a windows form based application, which needs to be localized and the text need to be altered later, then the .NET framework provides us with a utility called winres.exe (Windows forms resource editor). Winres.exe is a stripped down version of windows form designer. We can directly open the .resource files or .resx files in this editor and we can update corresponding text and, if required, size of the controls. Below is a sample application .resx file opened with winres.exe (Note: it's not a site, it's a Windows forms application resource file that looks like a website):

Summary

In this article we've taken a quick look at how to make Localization work in C#. Most of the core concepts related to Localization are explained in this article. The same approach can be followed in ASP.NET applications as well, which makes developing multilingual websites more easy and scaleable.


Managing Culturally Correct Date and Time Formats in C#

Managing Culturally Correct Date and Time Formats in C#
http://www.csharptoday.com/content.asp?id=1698
Article Rating: 4.5
Published on November 19, 2001
Abstract
In this article, Bill Hall examines elements of the .NET DateTimeFormatInfo class using C# as the vehicle. The goal is to help us write code that behaves correctly in any world region. Why write it twice when we can do it right the first time with little extra effort? We start with something simple that we can use right away, moving on to two examples with dates and calendars, before dissecting a DateSquares example to show how simple changes lead to culturally correct output.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

In this article we examine elements of the rich feature set of the .NET DateTimeFormatInfo class using C# as the vehicle. The goal is to help you write code that behaves correctly in any world region. Why write it twice when we can do it right the first time with little extra effort? We'll start with something simple that we can use right away. This will be done using two console programs. After that, we'll look at two examples that do a few things in ASP.NET with dates and calendars. Along the way, we will dissect the DateSquares example that was presented at the beginning of Chapter 13 of the Wrox publication C# Programming With the Public Beta to show how simple changes lead to culturally correct output.

Console examples using DateTime

Console programs provide a quick way to learn how to deal with some new concepts or programming tasks. Initially, it is not very obvious how DateTimeFormatInfo works with the DateTime structure, the basic container for times and dates in .NET. An instance of DateTime stores a particular instant in time. Using DateTime 's methods and properties, we can then change a particular time instance by adding or subtracting a time interval. It is also possible to parse, compare, convert, and format the contents. DateTime inherits from IFormattable among other interfaces, and that is where DateTimeFormatInfo comes into the picture. DateTime values are formatted using standard or custom patterns stored in the properties of a DateTimeFormatInfo instance.

As it happens, DateTime has a number of built-in formatting methods obtained from its inherited DateTimeFormatInfo. So, without invoking the latter class directly, we can manage international date formats practically without thinking. The format of the output is determined by the thread's implicit CultureInfo, which can be overridden if necessary. Every running thread on any modern operating system has an associated default locale which can be overridden if necessary; in .NET, locale is managed by the CultureInfo class.

By the way, if you are not familiar with using the CultureInfo class, you may find it helpful to read the Internationalization With The .NET CultureInfo Class & C# article. CultureInfo affects the behavior of most members of the System.Globalization namespace, and a basic understanding of this class will aid understanding of this article.

First Console Example

In this first example, we use the facilities of DateTime to provide internationalized formats. Basically, we see how to use the ones most commonly used, namely the short and long date or time formats. Code is also provided to show how to generate all the forms that DateTime knows about for the culture in which it runs, although we really need the help of DateTimeFormatInfo to determine the use of so much information. We also demonstrate how to change the default culture so we can see results from other world regions.

If you want to follow along, create a C# Console project called ConsoleDateTime. In the class file add the code as shown below:

using System;
using System.Globalization;
using System.Threading;

namespace ConsoleDateTime
{
class Class1 {
static void Main(string[] args){
// get the current time
DateTime dt = DateTime.Now;
// display the name
Console.WriteLine("Current Thread Culture : {0}",
Thread.CurrentThread.CurrentCulture.Name);
// display a non-internationalized format
Console.WriteLine("DateTime.ToString: {0}",dt.ToString());
// show culturally correct short and long dates and times
Console.WriteLine("DateTime.ToshortDateString: {0}",
dt.ToShortDateString());
Console.WriteLine("DateTime.ToLongDateString: {0}",
dt.ToLongDateString());
Console.WriteLine("DateTime.ToShortTimeString: {0}",
dt.ToShortTimeString());
Console.WriteLine("DateTime.ToLongTimeString: {0}",
dt.ToLongTimeString());
// enumerate all possible formats
// not all of these are culturally useful
string [] dtfmts = dt.GetDateTimeFormats();
foreach(string s in dtfmts)
Console.WriteLine("{0}", s);
}
}
}

The output from this program can be quite extensive. The exact results depend on the startup thread culture, which is printed out as an RFC 1766 identifier on the first line of output. Read more about RFC 1766 style identifiers at http://www.ietf.org/rfc/rfc1766.txt?number=1766&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq. After this program is run for the first time, change your default culture by selecting a new locale from Control Panel | Regional Options.

Choose a locale reasonably close to our default locale (as we are using the console, we may see only question marks instead of text if the locale we choose is radically different). For example, on my system, I usually develop with the English (United States) locale setting. If I change to French (France), which has a similar console code page, I can get useful output without adding any additional language support.

As you may have guessed, the locale setting made here determines the startup thread's CultureInfo instance. That is why we get different output.

Many, but not all of the forms we will see, are acceptable for normal use. Some have special uses that require an in-depth discussion of DateTimeFormatInfo. Also, in some applications we may have to change the calendar type we use to get to specific forms. DateTimeFormatInfo helps us sort it all out, but in most instances, the default short and long date string methods of DateTime suffice to return useful default behavior in the current thread. We will have a more in-depth look at DateTimeFormatInfo in a future article.

Let's now look at an example that runs through the calendars available in our current locale, and displays a long date in a message box. (Many world regions use different calendars to mark important national events or for religious purposes, and we mention some of them further on). With a message box providing the output, we have a better chance of seeing the results correctly, especially when viewing 'exotic' locales. However, we might have to install additional language support (details are given below). Again, here is the code. We are using a console program but the output is shown in a message box. Note the explicit use of DateTimeFormatInfo to get the appropriate long date pattern and to act as the formatter for DateTime. Observe also that we have added the line 'using System.Windows.Forms; ' along with a corresponding reference to the said .dll in the project.

using System;
using
            System.Globalization;
using System.Windows.Forms;
using
            System.Threading;

namespace
            ConsoleDateTimeCalendars
{
/// <summary>
/// Display
            long dates of the Culture for each supported calendar
///
            <\summary>
class Class1 {
static void Main(string[]
            args){
// get the name of the current culture just in case
//
            the user forgot to enter something on the command line
string
            culturename = CultureInfo.CurrentCulture.Name;
if (args.Length
            > 0)
culturename = args[0];
// Get the current
            date
DateTime dt = DateTime.Now;
try {
// Get a CultureInfo
            object and show the display name
CultureInfo ci =
            CultureInfo.CreateSpecificCulture(culturename);
Console.WriteLine("Specific
            Culture: {0}", ci.DisplayName);
// Get the associated
            DateTimeFormatInfo object
DateTimeFormatInfo dtf =
            ci.DateTimeFormat;
// Enumerate all calendars
Calendar [] cal
            = ci.OptionalCalendars;
// Select in each calendar and
//
            display the long date pattern in a message box
foreach(Calendar c
            in cal) {
dtf.Calendar =
            c;
MessageBox.Show(dt.ToString(dtf.LongDatePattern,
            dtf),
c.ToString());
} catch (Exception e)
            {
Console.WriteLine(e.ToString());
}
}
}
}

Running the code returns a series of message boxes with the name of the calendar in the title and a long date format as the message. For example, here is the output on my machine if I set my locale to Japanese (use the Regional Settings Dialog again), where there are three calendars. The first one that appears is the Japanese Era calendar: the output reads Heisei 13th year, 9th month, 22nd day. Only the year differs; the days and months are the same as the Gregorian calendar. If you are targeting Japan, you may have to support this calendar.

The next result is a Gregorian date using English.

Finally, the last one is a Gregorian calendar in Japanese. The output reads as the 2001st year, 9th month, and 22nd day. (Congratulations! You have just learned three useful Kanji (Chinese) characters).

We can view even more calendars if we set Arabic locales. Most of these have a Hijri lunar calendar and several Gregorian calendars in English, French, and French and/or English translated into Arabic. Other locales with possibly surprising results to those new to internationalization are Chinese (Taiwan), Thailand, Korea, Farsi, and Israel. The latter also has a 13th month. In the next article we'll see how to find such information.

If you don't see these results very well, or cannot find these locales on your system, you may want to install additional language support on your system. It is fairly easy to do. Just use the Regional Settings dialog; check the additional languages you want to use (see the picture of the Regional Settings Dialog above) and have your installation CD handy.

A Note About Managing String Formatting

In the code samples above, you may have noticed the implicit use of the String (not string) class to manage formatting such as this example:

Console.WriteLine("Current Culture: {0}", ci.Name);

One of the biggest mistakes made by programmers is in using string concatenation to build a message. Assembling a sentence by successively adding fragments leads to translation hell, because the result is entirely dependent on the word order and grammar rules of the underlying natural language. Unfortunately, programming languages such as VB, C#, and Java make such bad practices easy, but the result can be very costly when a program or web site has to be translated to several languages; a lot of code has to be rewritten to make the program independent of the underlying user interface. Fortunately, C# (and Java, too) provides a nice mechanism to get around the problem.

Let's see a simple example. Suppose we write code like this to produce the output: "My name is Bill":

string myname = "Bill";
string greeting = "My name is ";
Console.WriteLine(greeting + myname);

In a well-written program, neither of these strings would be in the code but in a message table or resource file. But, even if the strings are stored in resources our problems are not over. Let's translate the desired output into Japanese (we'll use Romanized Japanese to illustrate). One possible translation is "Watashi no namae wa Bill desu", which is something like "As for my name it Bill is". But, if we attempt to build the message by concatenation, we would have to rewrite the code to concatenate three parts together instead of two, because the value of the variable myname has to come before the verb.

string myname = "Bill";
string greeting = "Watashi no namae wa ";
string tobe = " desu"
Console.WriteLine(greeting + myname + tobe);

Such word ordering is typical of Japanese, German to some extent, and many other world languages, but not of English. In other words, we have to change the code to render the message, an unacceptable solution, especially when it is not necessary.

How do we get around the problem? It is actually easy. Change the greeting string to:

string greeting = "Hello, my name is {0}";

Translated into to Japanese it becomes "Watashi no namae wa {0} desu". Now the same code will work for the original and the translated string, namely Console.WriteLine(greeting, myname);

This technique becomes even more useful when multiple replacements are used. You may be reminded of using the printf family of API's in C and C++, FormatMessage in Win32,

CString::FormatMessage in MFC, and the MessageFormat class in Java.

Some Examples from ASP

Since you are interested in C#, you may have read the Wrox publication "C# Programming with the Public Beta", referenced above.

Early in Chapter 13, we are lead through the steps to create a web project. After that the authors ask us to rename the provided default aspx file to DateSquares.aspx and add some C# code to display the date and to create a list of squares of the digits 1 to 10. Here is the code that was added:

<h1>First ASP .NET Page</h1>
Today's date is
<% = DateTime.Now.ToString() %>

<p>
First 10 squares are
<%
int i;
for (i = 1; i <= 10; i++) {
Response.Write("<br> ", + i.ToString() + " squared = " +
(i*i).ToString());
}
%>

Although there are lots of interesting internationalization problems (including the string concatenation combining both HTML and program output, which guarantees that your web site will be essentially untranslatable to another language!), the first item that caught my eye was the code for displaying a date, highlighted above. When I ran the code, I got the expected result: not even the slightest hint of a culturally correct date display. However, it is usually about as easy to do correctly from the beginning, and it certainly saves money and development time over the life of the product, especially when cost is spread over releases into other world regions.

So, let's see what is wrong with the date code above and examine how we can fix it with just a little bit more programming. The problem is the output. Regardless of any system or user locale settings I make on my system, the output of the line of code above always looks like a US style date and time string, i. e. 9/10/2001 7:04:35 PM. The separators between fields are ':' and '/', the time is based on a 12-hour clock, and worse, the date is ambiguous - is it the 10th of September or the 9th of October? If you lived in the US you would conclude the first but most people in Continental Europe would see the second. It could be even more confusing if a two-digit year had been used. For example, 9/10/01 could even be read as the 9th year, 10th month, 1st day of the Heisei era. How quaint you must be saying to yourself, but you may have to handle this issue if you are targeting one of the most important economies in the world - Japan! Finally, too much information is being displayed. Usually a user just needs a date or a time but rarely both together. Let's do two examples to see what can be done.

Revising DateSquares

In the first example, we will rewrite DateSquares.aspx. We replace the date portion with a line that produces a slightly more locale aware display. We also move the text "Today's date is " into a String.Format call to make it more maintainable and translatable, and we use a formatter in the ToString statement. Indirectly, DateTimeFormatInfo comes into play here since CultureInfo has one as a member. Finally, we replace the table of squares by a list of all locales found on the server and show the local name, native name, and the full date and time for each. By the way, when we are mixing so many languages, we will want to set our page encoding (the document property is 'charset') to utf-8 as shown below.

All the code needed to generate the output is in the Page_Load() routine in the code behind module. Note how we have used String.Format to eliminate the string concatenation problem of the original code and to isolate HTML statements.

private void Page_Load(object sender, System.EventArgs e)
{
// Put user code to initialize the page here
DateTime dt = DateTime.Now;
Response.Write("<h1>First ASP Net Page</h1>");
Response.Write(String.Format("Today's date is {0}", dt.ToString()));
CultureInfo [] ci =
CultureInfo.GetCultures(CultureTypes.SpecificCultures);
foreach(CultureInfo c in ci) {
DateTimeFormatInfo dtf = c.DateTimeFormat;
Response.Write(String.Format("<br>{0} {1} {2}",
c.EnglishName,
c.NativeName,
dt.ToString(dtf.FullDateTimePattern, dtf)));
}
}

You may wonder why we have enumerated specific cultures instead of all cultures. A specific culture combines a language with a world region, whereas a parent culture has only a language. But, parent cultures have no associated DateTimeFormatInfo classes. One reason for this is that DateTimeFormatInfo contains the names of the weekdays and the months as well as numerous formats that have both language and regional dependencies. For example, en-US (United States) and en-UK (United Kingdom) have the same parent culture - en - but each has a different first day of the week (Sunday vs. Monday), and the default short date formats have different ordering (15/8/2001 vs. 08/15/2001).

The output from the program is pretty extensive. Included with the article is the actual page generated (datesquares_aspx.htm) and you can open it with your browser to see the output. Depending on the fonts and international support on your local system you may not see everything in its full glory.

Finally observe that the steps to obtain a more culturally correct date display and to reduce the drawbacks of string concatenation do not really require much more code. The only problem with this example is that the output shows what the server can do but disregards client preferences. So, let's look at another example that reads the client language preferences and displays some date, time, and calendar information accordingly.

Another Date Example

In this example, we read the user's language preference in the browser and for each one found, a specific culture is created. For each such culture, all the calendars are enumerated, and for each calendar the full date and time string is displayed. We also show the names of the cultures in two formats (native and UI display) along with the calendar names.

You probably know how to set your language preferences in a browser. In Internet Explorer the steps are Tools | Internet Options | Languages | Language Preference. In the picture below, the ones I have set in my browser have associated RFC 1766 identifiers (en-US, ar-EG, ru, etc.) that can be read and used either as a constructor for a CultureInfo instance or passed to the static method CreateSpecificCulture. We should always do the latter as mentioned above. For example, if we simply use "ru" in the CultureInfo constructor we will end up with a parent culture, which, as noted above, cannot be used for date and time information. However, CreateSpecificCulture will actually create a writeable Russian language CultureInfo object with Russia as the default country. This is what we want.

Below we see the source code that generates the output. Things to note are the isolation of HTML formatting using the String class and the parsing for the RFC 1766 identifiers. As before, all the code is in the code-behind module.

private void Page_Load(object sender, System.EventArgs e)
{
// Put user code to initialize the page here
// get the current time
DateTime dt = DateTime.Now;
CultureInfo ci;
// request all user languages
string [] userLangInfoData = Request.UserLanguages;
if (userLangInfoData != null) {
// split things up into usable pieces
// language information is separated by semicolons
string [][] langdata = new string[userLangInfoData.GetLength(0)][];
// enumerate through all browser languages
for (int i = 0; i < langdata.GetLength(0); i++) {
// split each piece into rfc and quality indicators
langdata[i] = userLangInfoData[i].Split(';');
try {
// create a specific culture
ci = CultureInfo.CreateSpecificCulture(langdata[i][0]);
// get the datetimeformatinfo instance
DateTimeFormatInfo dtf = ci.DateTimeFormat;
// write the native and display names
Response.Write(String.Format("<h2>{0} : {1}</h2>",
ci.DisplayName, ci.NativeName));
// split out the calendars
System.Globalization.Calendar [] cal = ci.OptionalCalendars;
foreach(System.Globalization.Calendar c in cal) {
// write out the names
dtf.Calendar = c;
Response.Write(String.Format(
"<STRONG>{0}</STRONG>", c.ToString()));
// for gregorian subtypes, list them
if (c.GetType().IsInstanceOfType(newGregorianCalendar()))
Response.Write(String.Format(
"<STRONG> ({0})</STRONG><br>",
((GregorianCalendar)c).CalendarType.ToString()));
else
Response.Write("<br/>");
// finally display the full date
Response.Write(String.Format("{0}<br/>",
dt.ToString(dtf.FullDateTimePattern, dtf)));
} catch {
// the browser language is not supported by .NET
// just skip over this one
}
}
}
}
}

Here is how the output looks for the Russian and Egyptian-Arabic cultures.

Summary

As you can see, it is about as easy to provide date and time formats that do the right thing automatically as it is to write code that has no cultural awareness. In most instances, the short and long date formatters from DateTime will suffice for rendering date and time displays in a culturally correct manner. If you need additional granularity, you must turn to explicit use of the DateTimeFormatInfo class. This class is also the repository of much incidental information such as weekday names, month names, abbreviated names, first day of the week, calendar, and an enormous number of specialized formats including those that can be parsed. If there is interest, a future article will provide details of DateTimeFormatInfo (http://www.csharptoday.com/content.asp?id=1801&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq)

If you'd like to discuss a broader range of C#Today topics and issues, or if you have any C# related questions you need answers to, please visit our online resource centres at : http://p2p.wrox.com/list.asp?list=csharptoday_discuss&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq and http://p2p.wrox.com/list.asp?list=c_sharp&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq


Visual Inheritance – An Approach to Superior GUI Design

Visual Inheritance – An Approach to Superior GUI Design
http://www.csharptoday.com/content.asp?id=1696
Article Rating: 4
Published on November 9, 2001
Abstract
The Microsoft .NET Framework offers a plethora of new features. One of these new features is visual inheritance. Visual inheritance is the ability of a windows form to inherit the visual characteristics of another form. This feature makes the task of creating consistency between graphical user interfaces very easy for the developer. In this article, Vikram covers the concepts of good UI design, and demonstrates visual inheritance with a simple windows form.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

The Microsoft .NET Framework offers a plethora of new features. One of these new features is called Visual Inheritance. Visual Inheritance is the ability of a Windows Form to inherit the visual characteristics of another form. This feature makes the task of creating consistency between graphical user interfaces very easy for the developer. The idea of Visual Inheritance promotes the concept of 'UI reuse' where the developer can effectively reuse the UI he has built.

This article is based on Visual Studio .NET Beta2 (Version 7.0.9254), C# and Visual Basic .NET. We will build a simple application to demonstrate visual inheritance and cross language inheritance in Windows Forms.

The Need for Visual Inheritance

Consider any business application - be it a Sales Force Automation system or an accounting package, the one common thing we find is a graphical user interface with which the end user interacts. And one very common requirement is the need to have similar looking interfaces throughout the application. On numerous occasions developers are faced with the task of having to design similar looking interfaces repeatedly.

Let us consider some day to day scenarios that are quite common. Imagine we are designing graphical user interfaces for a rich client application like a data entry system. We would have to go through the tedious process of noting down the settings of quite a few visual elements which are common across multiple screens that you intend to build.

Consider another typical example of a team of multiple developers working on an application. Let us assume that each developer is in charge of a particular portion of functionality. And each portion of functionality needs a graphical user interface. It will take quite an effort to verify and maintain a consistency between the many developers. Also, this concept of redoing the same thing over and over again with a considerable investment of time and effort does not make sense. The solution which fits this situation is visual inheritance.

Now, let us analyze the situations discussed above from the Microsoft .NET perspective and throw some light on how visual inheritance offers a solution to these. In the first situation that we considered, we can create one base form or template form which will contain all the visual elements that are constant across the different user interfaces that are to be built. Then, the rest of the user interfaces will inherit from this base form, which means they will automatically have the features of the base form by default, and extra features we require can be added later. In the second situation, a similar strategy can be adopted. A base form with all the settings can be created and all forms being developed will inherit from this base form.

From the above, you might have noticed that there is some importance given to maintaining some kind of standard or consistency across the many graphical user interfaces that an application has to offer. Let us have a more in-depth look at this aspect.

Designing Graphical User Interfaces

Microsoft Windows is all about graphical user interfaces (GUIs). One of the main reasons for the success of modern operating systems is their user friendliness, which can be attributed to the presence of GUIs. Across Windows, there is a great deal of consistency in the way these GUIs have been built. And it is quite obvious that if the applications that we write are to be successful on the Microsoft Windows Platform, we need to adhere to the standards set by the operating system.

Before designing GUIs, we should be aware of some fundamental principles that need to be followed. Firstly, we need to keep in mind that it is a graphical 'user' interface and the 'user' must be given primary importance when we design it. The users who will use it will likely be non-programmers, and so the interface exposed to such a user should be intuitive and responsive so that it is easy to use. By intuitive, we mean that the user should be able to recognize the interface as a familiar one and should be able to use it spontaneously. This happens when we design our graphical user interfaces in a consistent way.

Consistency

The best way to build a consistent user interface would be to follow the standards set by some of the other successful applications. Let us consider ,for example, Microsoft Word (see the links section at the bottom of the article for a Windows style guide):

If we look at the toolbar, Word uses pictures of objects that are consistent across many applications. The file symbol is a standard for New File, the open folder picture is the standard for the Open command, and the floppy symbol denotes the Save operation. These standards are consistent across Windows for many applications.

We need to understand that in the case of business applications, the main objective of the user is to get their job done, and the easier we make this for the user, the more successful our application will be. It is not a good idea to come up with our own jazzy interfaces, which will confuse the user and will require him to explore the application in order to use it. This rule does have a few exceptions. You might have observed that some multimedia software like MP3 Players, etc, come with a jazzy, novel interface. In such cases not being consistent is justified by the fact that primary importance is again being given to the user. The target audiences of such applications are usually music enthusiasts and these are not people who are trying to get some job done. However, for business applications designing interfaces with consistency is a must. In fact, you are advised not be creative when it comes to designing GUIs. Too much creativity in the area of GUI design will defeat the idea of consistency and create confusion for users since it leads to interfaces whose functionality is unpredictable. A user will assume a floppy disk symbol image in our toolbar will be meant for the Save operation - anything else will likely confuse them.

Consistency in GUI design is a very important factor, which might decide the success or failure of an application. If consistency is not present, the software often becomes difficult to use and the users might need training to learn to use the software. This would be a major hindrance to the software becoming popular. In fact, you might have used the best algorithms possible to get the results, but if the user cannot easily navigate through the interfaces provided, then everything is in vain. Hence maintaining consistency in GUI design must be given top priority.

Another benefit of maintaining consistency in a GUI is that it brings about predictability about the functionality and purpose that the interface is offering to the user. For example, if we have provided a radio button to choose between two things in one interface, then in a subsequent interface we should do the same thing. If something else is used then the consistency is lost and the user will be confused.

Simplicity

Apart from consistency, another fundamental principle of GUI design is simplicity. The idea of software is after all to automate things and to reduce effort and complexity. A truly good design would offer a simple GUI to a complex business process. So, we must aim to keep the GUIs simple and easy to use. Also, we need to always keep in mind that we are not the main user of the software we are creating - somebody else is. So a good strategy would be to ask someone else to use to the interface and then take their feedback. This is a basic but very effective strategy, as the designer is already aware of the requirements and understands his solution to them, but a brand new user may not be sure of either. Further, it is always a good idea to provide some kind of assistance to the user in the form of context sensitive help, which will guide the user in case of them not understanding something. The help can be in the form of a help file or screen tip.

After introducing the idea of Visual Inheritance and some concepts in GUI design, let's now move on to see how these concepts can be applied in practice.

Demonstration of Visual Inheritance

Let's now build a simple application that demonstrates Visual Inheritance in Windows Forms. We will build a Windows Form in C# and then we will visually inherit from this Windows Form in Visual Basic .NET. In .NET, every language compiles source code to an Intermediate language known as Microsoft Intermediate Language (MSIL), which is then interpreted by the Common Language Runtime. Thus, when compiled, source code written in C# and Visual Basic .NET will both generate MSIL, enabling cross language inheritance. Note that the application we build is for demonstration purposes and does not contain any kind of business logic in it.

The C# Template Form

Let us first create a Template Form in C# which we will later inherit in Visual Basic .NET.

  • Start Visual Studio .NET and select Visual C# projects | Windows Application. In solution explorer, right click on the project name and select Properties from the menu. We will see the Property Pages for the project as seen below:

In the screen, click on Output Type and from the dropdown select Class Library. This is needed because, by default, a Windows Application will compile the output to an Executable (.exe) and what we need here is a .NET Assembly DLL, so that other applications can inherit from it.

  • Now let us start building the interface. Drag a PictureBox User Control from the toolbox and set its Image property to the image of the logo that we plan to have. The image used in this sample can be found at \Template\Logo\logo.jpg in the download code.
  • Next drag a Button from the toolbox and call it btnClose and set the Text property of the Button to Close.
  • Add another Button and call it btnSave. Set the Text property to Save.

Once the design is done, the screen should look like this:

Besides this, we need to make a few more changes to this form:

  • Find the place where these controls have been declared, and change the access modifier from private to protected. Alternately you can also set the Modifiers Property in the Properties Grid of the corresponding control to Family from the dropdown (Family is same as protected). The access modifier private is changed to protected because a protected member is accessible to a derived class, while a private member is not accessible to a derived class.
  • Next, click on the Save button to open the btnSave_Click() method and change the access modifier to virtual protected. The implementation of a virtual method means that its functionality can be changed by an overriding method in a derived class. We make btnSave_Click() a virtual method so that we will be able to specify a different implementation for it in the derived class.

The code below shows the changes (Just a portion within the class is shown):

protected System.Windows.Forms.PictureBox pictureBox1;
protected System.Windows.Forms.Button btnClose;
protected System.Windows.Forms.Button btnSave;

protected void btnClose_Click(object sender, System.EventArgs e)
{
this.Dispose();
}

private void frmTemplate_Load(object sender, System.EventArgs e)
{

}

virtual protected void btnSave_Click(object sender, System.EventArgs e)
{

}

  • Once these changes are made, Click on the Build Menu and select Build to create the Template.dll file. We have now completed the C# portion of the sample.

Let us now create the Visual Basic .NET portion of the sample.

The Visual Basic .NET Inheritor

  • Open a new Windows Application in Visual Basic .NET. Click on Form1.vb that has been automatically created and exclude it from the project.
  • Select Solution Explorer | Project | Add | Inherited Form.
  • In the next Window, select Inherited Form and give a name to the form. Then choose Open. The following dialog comes up only if we do not have any installed assemblies from which to inherit:

  • Choose the option Browse, browse to the directory where we have built the Template.dll, and select it.

  • Click on OK and a new form can be seen as shown below:

Now this form is a Visual Basic .NET form but it has visually inherited from another form which was built in C#! Observe that the objects that have been visually inherited from the base class have an arrow symbol on them.

Now let us enhance this in Visual Basic .NET.

  • Firstly, add a few TextBox controls as shown below:

  • Once we are done, double click on the Save Button. We will see that there are some errors since the access modifiers are incorrect. The access modifiers for the method have to be set to protected and the overrides keyword has to be added to denote that we are overriding this method. The overriding method provides a new definition of the method that will be invoked. After making the above mentioned changes the btnSave_Click( ) event handler should look as follows:

Protected Overrides Sub btnSave_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Click

MessageBox.Show("Details Saved.", "Save", MessageBoxButtons.OK, MessageBoxIcon.Information)

End Sub

We have basically made three changes to what already existed:

  • Firstly we changed the access modifier to be set to Protected and also added the keyword overrides to denote that this method will override the method in the base class.
  • Next, we changed the event wiring to denote the Base Class. For this we use the keyword MyBase. MyBase is used to access base class members that are overridden in a derived class. Now, when this application is executed, and the Save Button is clicked, then the code written in the Derived Class will be used.
  • To show this we add a MessageBox, which returns a 'Details Saved' message to the user. (The application however does not actually save anything - it is just an example of using visual inheritance).

Now if we build and run the application we will see the form with the visually inherited features from the C# template as well as the newer ones that were added in Visual Basic .NET.

In cases where we do not want an element of the base class to appear in the derived class, we alter the Visible property of that visual element. Let's now hide the Close Button in our sample.

  • Click on the Close button and open its Properties window.
  • Set the Visible property of the Button to False.
  • Now rebuild the Project and run. The Close button is no longer visible. This way, we can hide the visual elements that are not necessary for us.

With the help of the template form we can create many similar forms with minimal effort preserving consistency and saving developer time.

Conclusion

This article looked at the concepts of GUI design, Visual Inheritance, and their advantages. We should always remember that a good GUI design is one where the user interface behaves as expected by the user. Lastly, we saw the language independence of .NET Framework where the choice of language is really a matter of lifestyle. Always remember that it is not technology that makes software great but the people behind it, the developers, who use the technology . and in the right way!


Introduction to Using Design Patterns Part 2

Introduction to Using Design Patterns Part 2
http://www.csharptoday.com/content.asp?id=1693
Article Rating: 4.5
Published on November 6, 2001
Abstract
In the first part of this article, Kaushal Sanghavi introduced the concept of design patterns, and why they are so useful in software development. In this article, he shows us how to apply some common patterns, and also introduces the concept of bad design, or AntiPatterns.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

In the first part of this article, we introduced the notion of Design Patterns as a best practice for software development. A pattern is a commonly adopted solution to a known problem or set of problems. Patterns are language and platform independent, and present a core solution that needs to be tailored to the specific scenario at hand, and then applied to the problem. Part 1 of this article introduced the different types of design patterns (creational, structural, and behavioural), and we implemented one example of each type of pattern. In this article, we will expand on the first part, and present a few more implementations of commonly used patterns. Note that, often, developers use patterns without consciously realizing they are doing so. The intent of this series of articles is to point out to that patterns are a very common approach to solving design problems; patterns are simple but powerful, and are continuously evolving. In fact, any approach to a common problem that is adopted over and over again, qualifies to be a pattern.

Further, at the end of this article, we will briefly discuss the opposite of patterns, Anti-Patterns. These are approaches commonly adopted by developers (or for that matter, any member of the team) that are destined to fail, and have failed time and again. Anti-patterns are just as important as patterns, and every attempt should be made to adopt patterns, while rooting out anti-patterns.

Note that this article in based on Visual Studio .NET Beta 2 and Windows 2000 (all versions). We use C# as the language of choice for this article, but design patterns are platform and language agnostic, and can be implemented in almost any language. Further, we assume a basic understanding of object oriented analysis and design in this article.

Recap of Design Patterns

A design pattern is a commonly adopted and proven approach to a common problem. Software applications, on one hand, can be entirely different from each other, but on the other hand, almost all applications face a common subset of problems that require a software solution. Design patterns provide an approach to solve a software problem. Patterns make no assumption about other parts of the system, nor does a pattern enforce any requirements; patterns provide a clean encapsulation of a solution. There are three basic kinds of patterns:

  • Creational Patterns: Creational patterns are related to how objects are created. These patterns dictate how objects are created and how an object lifetime is managed. These patterns are extremely useful for building very large scale systems, where close coupling of object lifetimes can get difficult to manage.
  • Structural Patterns: Structural patterns are related to how objects are composed to build larger structures. These patterns allow for the creation of loosely coupled systems, and allow you to build systems comprised of smaller sub-systems.
  • Behavioral Patterns: Behavioral patterns are related to how objects behave and interact with each other. These patterns provide guidelines for object communication and interaction.

For a complete discussion on types of design patterns, please refer to Part 1 of this article. The rest of this article will focus on implementing a few common patterns.

In the following sections, we discuss some implementations of common patterns.

Using a Creational Pattern - the Factory Pattern

The Factory pattern is a creational pattern wherein the creation of an object is controlled by another object, called the Factory. The factory object is responsible for creating an instance of a class that is requested by the caller of the factory. By introducing an additional layer of indirection between the object that needs to create the object, and the actual object itself, these objects are decoupled, and now changing the implementation of the object does not require changes to all the clients that create this object. Factories are often used when direct creation of an object is expensive. For example, in the case of classic COM, creating an object requires resolving its location by referencing the registry. Rather than do this every time an instance of the object is required, a factory can resolve this once, and then use the cached information to create multiple instances of the object. Factories are also used when the actual implementation of an object needs to be hidden from the client object. This is used commonly with interfaces, with the client oblivious of the concrete class that implements the interface. The client requests an interface, and makes calls on the interface; the factory decides the concrete class to instantiate and return to the client. Following this pattern, it is now possible to change the concrete class's implementation or add a new concrete class, and as long as the new class supports the required interfaces, the client does not require any modifications.

Another common use of this pattern is called Parameterized Factories. This is used to create objects, each of which implements a given interface, based on a parameter. A typical use of this pattern is when there is a hierarchy of objects that are semantically equivalent and the only difference in these objects is in their implementation. There are numerous examples of this scenario, for example, consider an application that uses a pipeline of components (like Microsoft Commerce Server). Each component in the pipeline serves a particular function, and the controller of this pipeline does not and should not care about the component's internals. As long as the component can be started, stopped, and executed, the controller should be encapsulated from the pipeline's internals. Another common example is the use of message formatters (used for serialization, messaging and remoting) in the .NET framework. Each formatter can serialize and deserialize a stream of data; the internals of how this is done is immaterial. In each of these scenarios, a parameterized factory should be used for object creation, and then all interaction with the object is based on the implemented interfaces.

The diagram below illustrates this pattern. Objects A, B and C all implement interface I. The factory object is the only object that is aware of the existence of A, B and C, and the client simply requests an object that implements interface I. The factory instantiates the right instance of the object, and returns an interface pointer to the client object, which is used for all subsequent access by the client. The key point here is that the client object is oblivious of objects A, B, and C. It only knows about interface I and the methods supported by this interface.

The most common academic example of this pattern is the Shape example. Consider different classes that correspond to different geometrical shapes, and all these classes implement the IShape interface. The code block below shows the example of a square and circle class.

public interface IShape
{
void
            Draw();
}

public class Square : IShape
{
public void
            Draw()
{
MessageBox.Show("Drawing a square",
            "Square");
}
}

public class Circle :
            IShape
{
public void Draw()
{
MessageBox.Show("Drawing
            a circle", "Circle");
}
}

The above code should be self explanatory. Let us now look at a parameterized factory that is responsible for creating these objects.

public class ShapeFactory
{
private ShapeFactory() {}

public static IShape CreateShape (int iID)
{
switch (iID){
case 0:
return new Square();

case 1:
return new Circle();

default:
return null;
}
}
}

The factory takes in a parameter that identifies the object that needs to be created, and returns an interface pointer to that object. Note that this is a key point; the actual object is returned as an interface pointer, not as the concrete class pointer. By doing this, the implementation of the internal class is always encapsulated from the client, and the client only works with interfaces as shown below.

private void btnDraw_Click(object sender, System.EventArgs e)
{
IShape lShape;
lShape = ShapeFactory.CreateShape(cbShape.SelectedIndex);
lShape.Draw();

}

Note that while this appears to be a very simple concept, this is an extremely powerful method for building loosely coupled systems, and this is one of the most common patterns in the classic COM world. COM objects are never created directly; they are always instantiated by the associated class factories.

The example may seem simple, (and it is), but design patterns are about getting things right at the design stage, not the coding stage. Anyone can code - but good designs are rare.

A Structural Pattern - the Proxy Pattern

The Proxy pattern is an example of a very popular structural pattern that deals with the use of surrogate objects that represent the concrete object. The surrogate object acts as an interception or translation layer between the client object and the actual object. This pattern is heavily employed by COM and the .NET runtime, primarily for allowing calls to remote objects. If the object is not located in the same address space or the same machine as the calling object, direct calls to the remote object are not possible. The proxy pattern allows another object, which is tightly coupled to the remote object, to perform special processing such as marshalling, to allow calls to be made transparently to the remote object. Another common use of this pattern is to optimize a given object. Consider a scenario where the calling object creates an object that is very expensive to create. This could be, for example, due to the fact that the object has to load a lot of data from a database, not all of which will be accessed by the calling object. In this case, the concrete object can use a proxy to load data only on demand, rather than create an instance of the object that loads all the data. Further, a proxy can be used to judiciously create instances of expensive objects, substituting an expensive object with a lightweight object as shown in the following example.

Proxies usually work on the principal of interception. This interception can be transparent to the client or visible to the client object. Calls to the concrete object are intercepted by the proxy, which performs some custom processing before the call can reach the concrete object. Similarly, on the return call, the call is intercepted and reaches the client only via the proxy. This pattern should be used whenever there is additional logic that needs to be added for a component to work. One approach would be to change the client and the concrete object to use this additional logic, but that is not a clean design, since now the client and concrete objects are doing more than they are supposed to. Further, these objects are no longer generic and get tied to a particular implementation. If the requirements or the platform changes, the core objects have to be modified, which causes a rippling effect of change. This should be avoided, via the use of proxy objects that encapsulate the custom processing and shield the client and concrete objects from this overhead. If the requirements change, the proxy object can be easily replaced, or even chained together to form multiple proxies, each of which is responsible for some custom processing.

Consider the case of a simple scientific calculator. For the sake of illustration, let us assume that creating a scientific calculator object is fairly expensive, since it implies loading of various math libraries, while the creation of a simple calculator is very quick. Thus, when a calling object requires a scientific calculator object, rather than creating a scientific calculator object by default, we instead return it a proxy to a calculator object. This proxy starts by creating an instance of a simple calculator, and the proxy creates the scientific calculator only when methods requiring the scientific calculator object are actually called. This is entirely transparent to the calling object, since the proxy exposes all the methods of both the simple calculator and the scientific calculator as shown below.

public class SimpleCalculator
{
public
            int Square(int iNum)
{
return iNum *
            iNum;
}
}

public class
            ScientificCalculator
{
public double Sine(int
            iNum)
{
return Math.Sin(iNum);
}
}

public class
            CalculatorProxy
{
public CalculatorProxy()
{
mSimpleCalc
            = new SimpleCalculator();
mSciCalc = null;
}

public int
            Square(int iNum)
{
return
            mSimpleCalc.Square(iNum);
}

public double Sine(int
            iNum)
{
if (mSciCalc == null)
mSciCalc = new
            ScientificCalculator();

return
            mSciCalc.Sine(iNum);
}

private SimpleCalculator
            mSimpleCalc;
private ScientificCalculator
            mSciCalc;
}

To keep this example simple, the calling object directly creates an instance of the proxy object as shown below. In a more realistic implementation, the proxy object would be entirely transparent to the calling object.

private void btnSquare_Click(object sender, System.EventArgs e)
{
txtResult.Text =
mCalcProxy.Square(Int32.Parse(txtNumber.Text)).ToString();
}

private void btnSine_Click(object sender, System.EventArgs e)
{
txtResult.Text =
mCalcProxy.Sine(Int32.Parse(txtNumber.Text)).ToString();
}

CalculatorProxy mCalcProxy = new CalculatorProxy();

Proxies are very simple objects that should be utilized whenever there needs to be custom processing before a call from a client object is dispatched to the concrete object. The main motivation for not embedding this code in the concrete object itself is to cleanly encapsulate the concrete object. By embedding custom processing into the concrete object, business logic and proxy logic is now combined, thereby making reuse of the concrete object difficult. Further, proxies are often used to support capabilities that the concrete object does not implement, as in the case of the .NET remoting plumbing.

Template Pattern - A Behavioral Pattern

The Template pattern is a common behavioral pattern, wherein an object defines the skeleton of an algorithm or behavior, and the concrete objects provide the implementations for certain sections of the behavior. Templates are common in many programming languages, where they are used to define a class or method that represents an algorithm that is independent of the type of data that the algorithm is acting upon. Templates provide a way to specify a type independent implementation of a behavior; and the parts of the behavior that are type dependent are delegated to the actual type at runtime. Templates are so common that languages like C++ and ATL support the template keyword in the language itself. VB .NET and C# don't directly support templates, but they can be easily implemented via the use of delegates (please refer to the Links section for an article on delegates).

The following diagram shows a template pattern, and while this might be considered similar to a proxy pattern, there is one important distinction. In the case of a template pattern, all objects are aware of the participating objects in this pattern, and work cooperatively to implement the pattern. In fact, the objects utilizing the template object explicitly need to support methods and properties that are required by the template.

Consider the Bubble Sort algorithm that is used to efficiently sort an array of objects. This algorithm is independent of the type of elements in the array; the only support that the algorithm requires from the type is in the form of a Compare function, that takes in two elements of the type, and determines whether one element is greater that the other. This Compare function will be implemented differently by different types, based on their notion of peer comparison and sorting. This bubble sort algorithm is a perfect example of the template pattern.

The following code shows the C# implementation of this templated algorithm. Note that the Sort function operates only on an Object array, since it is independent of the data type of the array elements. Further, this class declares a Compare delegate that is invoked when a comparison of two elements is required. Clients of this class can supply the implementation of this delegate based on their notion of element comparison. The Sort function simply invokes this delegate and uses its return value to determine whether the array elements need to be reshuffled or not.

public class BubbleSortAlgorithm
{
public delegate bool Compare(object iObj1, object iObj2);

public void Sort (ref Array iObjArray)
{
int lLen = iObjArray.GetLength(0);

for (int i = 0; i < lLen; i++)
for (int j = 0; j < lLen - i - 2; j++)
if (mTarget(iObjArray.GetValue(j),
iObjArray.GetValue(j + 1)))
{
object lTemp = iObjArray.GetValue(j);
iObjArray.SetValue
(iObjArray.GetValue(j + 1), j);
iObjArray.SetValue(lTemp, j + 1);
}
}

public void Register(Compare iTarget)
{
mTarget = (Compare)(System.Delegate.Combine(mTarget, iTarget));
}

private Compare mTarget = null;
}

A class that holds an integer array and uses the above BubbleSortAlgorithm class to sort this array is shown below.

public class IntList
{
public IntList
            (Array iArray)
{
mArray = iArray;
}

public Array
            Sort()
{
BubbleSortAlgorithm lAlgo = new
            BubbleSortAlgorithm();

BubbleSortAlgorithm.Compare lCompare =
new BubbleSortAlgorithm.Compare (SortInt);
lAlgo.Register
            (lCompare);
lAlgo.Sort (ref mArray);

return mArray;
}

public bool SortInt(object iInt1, object
            iInt2)
{
return (int)iInt1 >
            (int)iInt2;
}

private Array mArray;
}

Note that this class has a SortInt method that has the same signature as that of the BubbleSortAlgorithm class's delegate. Before invoking the Sort method, the IntList class registers this delegate which will be called to compare two integers. The BubbleSortAlgorithm class calls the SortInt method to compare two integer elements.

Although templates are not supported directly by C#, templates can be easily implemented using delegates as shown in this example. The template pattern is a very powerful pattern that allows developers to write type independent code that can be reused by multiple type instances.

Anti-Patterns

This article has focused on a few commonly adopted design patterns, which are a successful approach to a common problem in software development. Just as one should learn from repeated successes, one should also learn from repeated failures. After the wide adoption of design patterns, a lot of work was done on identifying areas of software development that have failed time and again. These commonly adopted mistakes were dubbed as Anti-Patterns, and similar to patterns, this is a loose collection of solutions which have not worked and thus should be avoided. A complete discussion of anti-patterns is beyond the scope of this article, but the following are a few anti-patterns that are commonly seen in almost every software development project.

  • Copy Paste Programming: Very often programmers, bone weary after long hours of coding, find code which is apparently similar to the code that they need to write. Rather than writing the objects and methods from scratch, they tend to copy some existing code, paste it, rename the objects and methods, and make minor changes to meet their current requirements. This is a very common anti-pattern that leads to the copying of existing bugs from the original code, and also causes some conditions that need special handling to be ignored. This anti-pattern is often referred to as "Editor Inheritance".
  • Scope Creep: All software projects start off as small and manageable. Very few projects seem doomed to slip their deadlines from day one. This slippage seems to occur slowly over a period of time as the project progresses. One of the main reasons for this is scope creep. Once the requirements of a project have been frozen, and schedules built based on these requirements, any attempt to change the requirements or add more scope to the project, should be met with the highest possible level of resistance. However, more often than not, the business or the marketing folks tend to win these battles, and the scope of the project is constantly under revision, throwing off all original delivery schedules. This is an age old anti-pattern that keeps repeating itself on almost every type of project.
  • Design for every scenario: There is a very popular adage in hardware engineering that is called the 80-20 rule. This rule suggests that 80% of the time is spent in 20% of the code, and as a corollary to this adage, there is another adage that says, "Make the common case fast". Software engineers, however, never seem to realize this, and tend to design systems that handle all possible scenarios, no matter how rare they might be. This is a very common anti-pattern, where developers tend to design for every possible scenario, and in the process end up making the common case inefficient. Developers should recognize the common case, make it fast, and then provide for fringe conditions.

Conclusion

Design patterns and anti-patterns are a common denominator in almost all software projects. Developers should identify and adopt patterns, while avoiding the use of anti-patterns, no matter how appealing they look. In this series of articles, we have discussed some common design patterns, and introduced the notion of anti-patterns. In the next articles in this series, we will discuss some of the points brought up from the feedback to these articles - probably UML, Visual Studio .NET implementations of design patterns, and Anti-Patterns.

Happy .NETing.


Introduction to Design Patterns

Introduction to Design Patterns
http://www.csharptoday.com/content.asp?id=1742
Article Rating: 4.5
Published on November 5, 2001
Abstract
Most developers these days are routinely involved in designing and developing large scale enterprise level applications that comprise of a disparate team of individuals, multiple platforms and hundreds of thousands of lines of code. Developing such a system is difficult, and extending and maintaining these systems can pose even bigger challenges. Given this, architects and designers of systems pay careful attention to designing these systems in a clean, elegant and standard manner, and adopt best practices wherever possible. One of these best practices is the concept of a Design Pattern. A design pattern is a pattern in which certain common software design problems can be tackled. While all development projects are different, most face a common set of design issues or tasks. A pattern is a predefined or canned approach (as opposed to canned code) to solving these problems. In this first article in a series, Kaushal Sanghavi discusses how design patterns can help in solving some common problems, discusses a few common design patterns, and implements a few in some sample code.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

Software development has matured considerably over the past couple of decades, and system development is more complex than ever. Most developers are routinely involved in designing and developing large scale enterprise level applications that comprise of a disparate team of individuals, multiple platforms, and hundreds of thousands of lines of code. Developing such a system is not only extremely difficult, but given the attrition rate of the industry, extending and maintaining these systems poses even bigger challenges. Given this, architects and designers of systems pay careful attention to designing these systems in a clean, elegant and standard manner, and adopt best practices wherever possible. One of these best practices is the concept of a Design Pattern. A design pattern is a pattern in which certain very common software design problems can be tackled. While all development projects are different, most face a common set of design issues or tasks. A pattern is a predefined or canned approach to solving these problems. In this article, we will discuss how design patterns can help in solving some common problems, discuss a few common design patterns, and implement a few in our sample code.

Note that this article is based on Visual Studio .NET Beta 2 and Windows 2000. Further, while we use C# to illustrate our patterns, design patterns are platform and language agnostic, and can be implemented in any programming language, using any development tool.

Further, since design patterns are very closely coupled with object oriented programming, we assume a basic understanding of object oriented analysis and design in this article.

Design Patterns

Design patterns in software development have been inspired by similar patterns in the construction of physical entities such as buildings, houses and towns. Christopher Alexander, one of the pioneers in this field, describes a pattern like this:

"Each pattern describes a problem which occurs over and over again in our environment, and then describes the core of the solution to that problem, in such a way that you can use this solution a million times over, without ever doing it the same way twice".

A design pattern is essentially an approach to solve a common problem. Note that a pattern is merely an approach and a high level conceptual solution. A pattern does not assume knowledge of a particular feature in a particular language, platform or tool. Nor does a pattern delve into the very specifics of its implementation. A pattern merely presents the core solution; you need to tailor this approach to meet your specific system and environment. However, this core solution is so powerful and firmly entrenched in the implementation, that a seasoned designer can take one look at a system and immediately point out its use or misuse of design patterns. This holds true for all patterns, no matter how they are implemented or how they are tweaked. The common thread running through all implementations of a given design is so pronounced, it is almost impossible to miss.

Design patterns can be broadly broken down into three different categories.

Creational Patterns

Creational Patterns, as the name suggests, deal with the process of object creation. A creational pattern dictates how objects can be created, who has control over creating the objects, when the object is created, and the relationship of the object that creates other objects with the actual object itself. These types of patterns allow a system to be more flexible and extensible, since there is always a layer of indirection between the actual object and the client using the object. By encapsulating the object creation, you avoid hard coding the object relationship and encapsulate the same. Two of the most popular patterns in this category are the Factory pattern and the Singleton pattern.

  • In the Factory pattern, there is a factory object that is responsible for producing instances of other objects. The client requests a particular type of object from the factory, and the factory decides what object to instantiate. The advantage of this pattern is that the client is oblivious of the actual object created; as long as the object adheres to the client's requirements (this is usually based on supported interfaces), the client never needs to know intricate details of the object. This allows you to change the instantiated object without having to change lines and lines of client code. Intelligence to create the appropriate object is built into the factory, and if the requirements change, all you need to do is change the factory to produce a different object. COM relies heavily on this pattern. The IClassFactory interface is used to instantiate COM objects.
  • In the Singleton pattern, as the name suggests, there is a single instance of the object alive throughout the life of the system. There is a well known way to access this instance, and the accessor method is responsible for ensuring that all clients can access the singleton object. This pattern is very useful when all requests have to be directed to a single object. This pattern is very common in the COM model again; the CoGetClassObject API call returns a reference to a class object, which is a singleton object for that particular class. MFC applications use singleton objects to access the base class objects (like the global CWinApp instance) as well.

Structural patterns

Structural Patterns are related to the structure in which objects are composed. These patterns usually relate to how two or more objects are composed to build larger structures. There are no fixed rules for this composition, and this would depend on the individual system's requirements. These patterns allow us to build large scale systems that comprise of smaller systems, and allow you to build loosely coupled systems. Further, smaller grained objects avoid a massive ripple effect that a change could cause in a very tightly coupled system. The Composite pattern and the Proxy pattern are very common structural patterns.

  • The Composite pattern is a very common pattern where multiple objects are composed to give the appearance of one large object. Again, using our example of COM, COM containment and aggregation are examples of the composite pattern. In this pattern, one object composes other objects by including the composed objects as member variables in its class declaration. It then exposes the services of these composed objects in a unified manner, such that to a client of this composed object, it appears as though all the services provided by the composed object come from one single object. Composition not only allows you to build complex objects based on smaller objects, it also allows you to customize the calls to the composed objects from the outer object.
  • The Proxy pattern is another common pattern that is heavily utilized by DCOM and .NET remoting. In the proxy pattern, one object acts as a surrogate object for one or more actual objects, and forwards calls to these objects. By doing so, the proxy objects presents a level of indirection to the actual object, and this level of indirection could be used to perform many functions before access to the actual object might be allowed. In the DCOM and .NET remoting frameworks, the proxy object is used to marshal data over to a remote process. Proxies can also be used to add a level of access checks that are used to grant or deny access to the actual object, or to group multiple calls to an object into one single call to improve performance.

Behavioral patterns

Behavioral Patterns are related to the way objects interact and behave in conjunction with related objects. These patterns govern the way in which objects interact and communicate with each other. These patterns let us build specialized objects that are used to communicate with the actual objects, thereby encapsulating the actual object's communication internals. By presenting an object that is used to communicate with the actual object, a change to the object only implies changing the related object, and not all the clients who access these objects. The Template pattern and the Iterator pattern are commonly used examples of behavioral patterns.

  • The Template pattern allows the building of a class or algorithm that can operate on multiple classes that support a predefined set of methods. The template class only uses these methods and other primitives to build functionality. The advantage of this pattern is that code that is identical, except the type of object that it operates on, can be written only once, instead of having to be written for every possible type. Templates are natively supported by many languages, and ATL and STL are built entirely on the template pattern.
  • The Iterator pattern allows you to access another object. The actual object would define an iterator object that is aware of the internals of the actual object, and uses this to access and traverse this object. Iterators are primarily used to encapsulate the actual object's internals, by providing another object as an accessor object. Again, the common theme here is that a change to the object's internals only implies a change to the iterator, and not to the end clients. Further, since the traversal is separate from the object itself, if the traversal itself changes, the object does not have to change. ATL, STL, and most of the .NET languages provide for an iterator to access a collection of objects.

Now that we have explained the different kinds of patterns and a basic description of a few common patterns, we will take one pattern from each of the 3 different pattern types, and present an example that demonstrates the pattern.

Singleton Pattern

The Singleton pattern is a Creational pattern that controls the number of instances of an object. The most basic form of this pattern restricts the number of instances to a single instance, but there are variations that can allow a fixed number of instances. For example, in a thread pool, the number of threads that can be created could be fixed to a particular number, and a singleton pattern can be used to limit the creation of threads to this number. In our sample, we will limit the creation to a single object.

public class SingletonObject

{
// prevent direct creation of this
            object
private SingletonObject(){}

public static
            SingletonObject GetSingletonObject()
{
// since multiple
            threads could be calling this, protect
            it
mMutex.WaitOne();

// create a single instance. if
            created, return that instance
if (mObject == null)
mObject =
            new SingletonObject();

mMutex.ReleaseMutex();
return
            mObject;
}

public string SomeMethod()
{
return "Some
            Data";
}

private static SingletonObject mObject =
            null;
private static Mutex mMutex = new Mutex();
}

The above code block shows a very common implementation of the singleton pattern. In this implementation, the singleton class ( SingletonObject class) declares a static method that is responsible for creating the singleton object. We use a static method because we do not yet have an instance of this object, and so we cannot use member methods. Further, the class declares two static member variables, one that represents the singleton object's instance and another that is a Mutex used to protect this object from concurrent access. The key here is that this class declares a private constructor, which prevents clients from creating an instance of this class directly. The only way to obtain a reference to an object of this class is via the static GetSingletonObject method. This method checks to see if the object is already created, and if it is not, it creates an instance of itself and stores it in the static member object mObject. Subsequent calls to this method will return this created instance. Further, since this method can be called by multiple threads at the same time, we use a mutex to guard the object's creation. This way only one instance of the object will be created and its creation will be thread safe.

private void btnSingletonObj1_Click(object sender, System.EventArgs e)
{
mSingletonObj1 = SingletonObject.GetSingletonObject();
}

private void btnSingletonObj2_Click(object sender, System.EventArgs e)
{
mSingletonObj2 = SingletonObject.GetSingletonObject();
}

private void btnEqual_Click(object sender, System.EventArgs e)
{
if (mSingletonObj1.Equals(mSingletonObj2))
txtEqual.Text = "Objects are equal";
else
txtEqual.Text = "Objects are not equal";
}

SingletonObject mSingletonObj1;
SingletonObject mSingletonObj2;

The above code block shows how we can use the GetSingletonObject method to obtain a reference to the singleton object. Further, we can test to see that all references obtained from calling this method, indeed point to the same object.

From the code download example of this pattern:

Note that this is just one implementation of the Singleton pattern. You can come up with your own implementation as long as the semantics of the pattern are preserved.

Composite Pattern

The Composite pattern is a Structural pattern where multiple objects are composed to give the appearance of a single large object. This is one pattern that many developers use commonly, without realizing that it is a design pattern. As we mentioned earlier in the article, COM uses this pattern heavily as Containment and Aggregation. Depending on the object that composes the other objects, calls to the composed objects might be directly delegated or might be passed after the composing object has added some of its own logic. We use a simple calculator object to demonstrate this pattern. There is one object that implements a simple calculator and another that implements a scientific calculator, and one Calculator object that composes these two objects.

public class Calculator
{
public
            Calculator ()
{
mSimpleCalc = new
            SimpleCalc();
mScientificCalc = new
            ScientificCalc();
}

public long Square(int
            iNum)
{
if (iNum > 1000)
return -1;
return
            mSimpleCalc.Square(iNum);
}

public double Sine(int
            iNum)
{
return mScientificCalc.Sine(iNum);
}

private
            SimpleCalc mSimpleCalc;
private ScientificCalc
            mScientificCalc;
}

public class SimpleCalc
{
public
            long Square(int iNum)
{
return iNum *
            iNum;
}
}

public class ScientificCalc
{
public
            double Sine(int iNum)
{
return
            Math.Sin(iNum);
}
}

As you can see from the above code block, this is a very simple pattern. The Calculator class contains the two composed objects as private data members, and it delegates the calls to methods in these objects. In the first method, the Square method, it adds some additional checks before it delegates the call, while for the Sine method, the call is a direct delegate.

private void btnSquare_Click(object sender, System.EventArgs e)
{
Calculator lCalc = new Calculator();
txtResult.Text = lCalc.Square(Int32.Parse(txtNumber.Text)).ToString();
}

private void btnSine_Click(object sender, System.EventArgs e)
{
Calculator lCalc = new Calculator();
txtResult.Text = lCalc.Sine(Int32.Parse(txtNumber.Text)).ToString ();
}

The above code block uses the composed objects as if it were a single object is fairly simple and self explanatory.

From the code download example of this pattern:

Again, as this pattern demonstrates, design patterns are not necessarily complex pieces of design. They are simply well known solutions to common problems.

Iterator Pattern

The Iterator pattern is a Behavioral pattern wherein a separate object is responsible for traversing through an object's data. This pattern is used for objects that hold collections of data, or data that is sequentially ordered. The most common types of these objects are linked list and arrays. The iterator pattern is such a common pattern, that nearly every technology out there like MFC, COM, .NET framework, VB, etc, uses iterators.

In our example, we use a simple object that holds numbers from 1 through N in an array. The iterator class traverses through this linearly. Note that, iterators and the objects that they iterate through are usually very closely coupled, with the iterator knowing the object's implementation details. It is possible to define abstract iterators which are very loosely coupled with the object, but that is a separate pattern, and in this article we will focus only on concrete iterators.

public class NNumbers
{
public
            NNumbers (int iLimit)
{
int[] lArray = new int
            [iLimit];
mArray = lArray;
mLimit = iLimit;

int
            lIndex;
for (lIndex = 0; lIndex < mLimit;
            lIndex++)
mArray[lIndex] = lIndex;
}

public int Item
            (int iIndex)
{
return mArray[iIndex];
}

public int
            Count
{
get{
return mLimit;
}
}

private int[]
            mArray;
private int mLimit;
}

The code block above shows the NNumbers class that holds numbers from 1 to N. This is a fairly simple class that uses an array to hold these numbers, and provides two properties, Count and Item that return the number of numbers held in the array, and the item at the given index. Note that this object merely provides a way to access its data; it really provides no method for traversing through its data sequentially. This is provided by the iterator class as shown below.

public class
            NNumbersIterator

{
public NNumbersIterator(ref NNumbers
            iObject)
{
// initialize to 0, coz it gets incremented before
            use
mCurrent = 0;
mObject = iObject;
}

public int
            MoveNext()
{
mCurrent = mCurrent + 1;
if (mCurrent ==
            mObject.Count) {
mCurrent = mCurrent - 1;

// we use -1 to
            signal an error,
// a real app would raise an error message
return -1;
}
return
            mObject.Item(mCurrent);
}

public int
            MovePrev()
{
mCurrent = mCurrent - 1;
if (mCurrent == 0)
            {
mCurrent = 1;
return -1;
}
return
            mObject.Item(mCurrent);
}

private int mCurrent;
private
            NNumbers mObject;
}

As you can see from the above code block, the iterator class is also fairly simple. Since every iterator works in conjunction with a single data object, the iterator's constructor takes in the object that needs to be iterated. It then initializes the current index into the object, and updates this index as the iterator is moved forward or backward. The iterator uses the object's Count and Item properties to retrieve the data.

The following code block shows how you can use the actual and the iterator object.

private void btnCreate_Click(object sender,
            System.EventArgs e)
{
mNObject = new
            NNumbers(Int32.Parse(txtN.Text));
mIterator = new
            NNumbersIterator(ref mNObject);
}

private void
            btnNext_Click(object sender, System.EventArgs e)
{
txtVal.Text
            = mIterator.MoveNext().ToString ();
}

private void
            btnPrev_Click(object sender, System.EventArgs e)
{
txtVal.Text
            = mIterator.MovePrev().ToString ();
}

private NNumbers
            mNObject;
private NNumbersIterator mIterator;

As you can see from the code above, we create an instance of the NNumbers object, and an instance of the iterator object passing it the NNumbers object. Once the iterator is created, we simply use the MoveNext and MovePrev methods to iterate through the NNumbers object's data.

Attached Sample Code

The following screen shot shows the sample application that demonstrates the code that we have discussed above. All three patterns are used by the same form as shown below:

The attached code consists of the DesignPatterns project which contains the SingletonPattern.cs, CompositePattern.cs, and IteratorPattern.cs class files. The DesignPatterns.cs file contains the main form code.

Conclusion

Design patterns are a very important design tool that aid in the development of large scale and complex systems. Design patterns provide a tried and tested solution framework to common problems faced by many system designers. Rather than attempt to hand craft a new approach and solution to common problems, developers should attempt to utilize a commonly adapted pattern. While design patterns can be intimidating and difficult to grasp initially, they are worth persevering with as a way of bringing best practices to your code design. Further, developers should not limit their adoption of patterns to the few design patterns that have met with wide acceptance and adoption. Any successful approach to a design problem qualifies as a design pattern, capable for reuse in future designs facing similar problems.

Happy .NETing.


Message Queuing with C# and .NET

Message Queuing with C# and .NET
http://www.csharptoday.com/content.asp?id=1665
Article Rating: 4
Published on October 2, 2001
Abstract
Message based communication has been around for a long time now. Entire software, and even hardware architectures are built using this paradigm. Windows itself is built on message passing; everything in Windows is a message. Traditionally, messages have tended to be synchronous and real-time, where the caller and callee work together, hand in hand. However, there is another message based paradigm which works on the queuing up of messages that get processed over time. IBM supports this via its MQSeries solution, and Microsoft via MSMQ. In this article, we look into the support that the .NET framework provides for interacting with MSMQ.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

Message based communication has been around for a long time now. Entire software, and even hardware architectures are built using this paradigm. Windows itself is built on message passing; everything in Windows is a message. Traditionally, messages have tended to be synchronous and real-time, where the caller and callee work together, hand in hand. However, there is another message based paradigm which works on queuing up of messages that get processed over time. IBM supports this via its MQSeries solution, and Microsoft via MSMQ. In this article, we look into the support that the .NET framework provides for interacting with MSMQ.

Message Queuing

Traditional message based communication assumes that the caller and callee are always aware of each other and are tightly coupled. Further, it assumes that the caller and callee are always available to interact. Take the example of an object calling another object to take advantages of its services. In this case, the client object explicitly knows who the server is and how to invoke its services; at times this information is even hardcoded into the client. The client needs this information to invoke the server. This works fine as long as the server is readily available and does not change. If the server happens to be unavailable or unreachable, the client is unable to proceed. Under certain situations, this is unavoidable and acceptable. For example, if the client needs to query the server for some information that it needs to display, and the server is offline, the client cannot proceed. In this situation, it is acceptable for the client to error out and stop processing. But there are situations, where the client should be able to proceed even if the server is offline. Let us consider the case of a client needing to update some data that resides on the server, if the server is offline, the client need not stop. The client should have some way of communicating its request to the server, such that, when the server comes back online, it can process the pending request.

Microsoft Message Queuing (MSMQ) is the technology that allows message passing between two entities that may or may not be offline at the same time. MSMQ introduces a message broker that is guaranteed to be running at all times and provides the interaction between the client and the server. Now the client does not interact directly with the server, instead it passes a message to the message broker which stores it internally till a server can retrieve it. The client and the server now interact with this broker to send and retrieve messages which encapsulate the request details. This allows the client and the server to break their close coupling, and allow for an extensible architecture. Thus it is now possible for the same message to be picked up by multiple servers, each of which could use it to perform a specific function. The client does not know and does not care about these details; it drops a request for service, and carries on with its execution. This message interaction can be bi-directional, wherein the server can send messages to the client as well. This is required if the client needs feedback about the request status or needs an acknowledgement about the message retrieval.

MSMQ Queues

As the name suggests, MSMQ uses queues to store messages. Each queue can store messages from multiple clients, and multiple servers can retrieve these messages. In order for a client and server to communicate, they need to know the queue that will be used to store messages, and the format in which the data will be exchanged. As long as this is known, the client can drop messages in the predefined format to a well known queue, and a server can retrieve these and, since the format is predefined, the messages can be parsed and processed. Under the most common situations, the message format is predefined and the schema is available to both the client and the server, but it is possible to build highly generic message formats that include metadata describing the message contents. Using this, a server need not have prior knowledge of the request format in order to process the request.

MSMQ supports two types of queues that can be used for message passing, Public queues and Private queues. Public queues require a domain controller running the MSMQ server, and can be accessed by clients from anywhere on the LAN. Private queues, on the other hand, can be accessed only from within the local machine. (In this article, all the samples will only deal with Private queues. This is in order to allow all readers to be able to download the samples and use them right off the box, without needing a MSMQ domain installation).

In addition to Public and Private Queues that store user messages, MSMQ also maintains certain system queues that are used internally by MSMQ.

  • Journal Queue: This queue is used to store copies of messages. This can store both source and target messages, and each message can specify whether it should be journalled or not. Note that journal queues have an impact on both performance and storage, since each message now has to be stored twice.
  • Dead Letter Queue: This queue is used to store messages that cannot be delivered or have expired. A message is considered non-deliverable if the target queue cannot be found, or if its quota limitation has been violated, or there is a conflict between the message and the queue type. Messages expire if they do not reach the target queue in a specific amount of time. This timeout can be specified by the client application.
  • Admin Queues: There are different kinds of admin queues that store administrative messages related to responses and notifications.

Now that we know the different kinds of queues, let us consider a simple client that drops a message to the queue, and a server that retrieves it and echoes it to the console.

public void SendMessage
            ()
{
MessageQueue lMQClient = new MessageQueue
            (@".\Private$\TestQ");
lMQClient.Send ("Hello
            world");
}

public void ReadMessage ()
{
MessageQueue
            lMQServer = new MessageQueue (@".\Private$\TestQ");
Message
            lRequest = lMQServer.Receive ();
lRequest.Formatter = new
            XmlMessageFormatter (new
string[]{"System.String"});
Console.WriteLine
            (lRequest.Body);
}

As you can see from the above example, sending and receiving messages is very simple. All you need to do is to create an instance of the MessageQueue and use the Send and Receive methods. The System.Messaging namespace contains all the classes that are used to interact with MSMQ. Let us look at some of the important methods and properties that are supported by the MessageQueue class.

  • Create: This method creates a queue with the specified path name. This queue could be present on the local machine or anywhere on the network. Further, the queue could be created as a transactional queue or as a non-transactional queue.
  • GetPublicQueues: This method returns all the queues on the network that meet specific criteria. These criteria could deal with when the queue was created or modified, or the machine on which this queue resides.
  • Send: This method sends a message to the specified queue.
  • Receive: This method retrieves a message from the queue. This causes the message to be deleted from the queue.
  • Peek: This method returns the first message in the queue. This does not cause the message to be deleted from the queue and will be available in the queue till a receiver retrieves it.
  • GetAllMessages: This method returns all the messages in the specified queue.

Message Formatters

As you can see from the above code, the receiver creates an object of type XmlMessageFormatter to read the message. A receiver needs to do this to specify how the message contents have been serialized and to validate the object schema. If the contents of the message do not match the schema, an exception is thrown. This is useful to ensure that the client and the server interpret the message in a consistent manner. Further, the same type of formatter has to be used to serialize and deserialize the message, if the client and the server use different formatters, an exception is thrown when the message is being deserialized.

The .NET framework currently supports three kinds of formatters; ActiveXMessageFormatter, BinaryMessageFormatter, XmlMessageFormatter. The XmlMessageFormatter is the default formatter used and it serializes the message into XML. This can be verified by viewing the message, as shown in the following screen shot.

The ActiveXMessageFormatter and BinaryMessageFormatter convert the message into a binary format, and are more efficient that the XmlMessageFormatter. However the Xml formatter presents the message in simple Xml which makes it easier to read and troubleshoot. Any class that implements the IMessageFormatter interface can act as a custom formatter.

As you can see from the above screenshot, the XmlMessageFormatter writes the message as XML. There are situations where you do not want to use any of the provided formatters, but would like to write your own custom formatter. Let us see how we can do this by writing a class that implements the IMessageFormatter interface. This is a very simple class that can be used when plain string data needs to be used as the message payload, and you do not want to incur the overhead of creating and parsing XML.

public class CustomStringFormatter : IMessageFormatter
{

public object Clone ()
{
return new CustomStringFormatter ();
}

public bool CanRead (Message iMessage)
{
//perform some error checking to validate the object schema
return true;
}

public void Write (Message iMessage, object iObject)
{
MemoryStream lStream = new MemoryStream ();
string lStringData = (string)iObject;

byte[] lByteArray = System.Text.Encoding.ASCII.GetBytes
(lStringData.ToCharArray());

lStream.Write (lByteArray,0,lByteArray.Length);

iMessage.BodyStream = (MemoryStream )lStream;
return;
}

public object Read (Message iMessage)
{
MemoryStream lStream;
lStream = (MemoryStream)iMessage.BodyStream;

int lStreamLength = (int)lStream.Length;
byte[] lByteArray = new byte[lStreamLength];

lStream.Read (lByteArray, 0, lStreamLength);

string lDataObject = new string
(System.Text.Encoding.ASCII.GetChars (lByteArray));

return lDataObject;
}
}

The above class supports three methods that are present in the IMessageFormatter interface. The Read and the Write methods are the methods called to deserialize and serialize the object data respectively. Let us look at the Write method first. This method gets called by the runtime and is passed the object that needs to be serialized and the message instance that will store this serialized data.

The Message class has a property called BodyStream that stores the serialized contents of the payload. The Write method above simply reads the string data that is passed as its second argument, and since the data can be stored in memory, it uses the MemoryStream class to create a stream of bytes that represent the string data. Once the stream is created, the message stream is set to this stream and now the message contains the serialized form of the payload. In the reverse direction, we carry out the exact reverse steps to deserialize the data. We read the BodyStream property of the message to get the payload stream and convert it into a string that can be passed on to the user.

Using this custom formatter class, the above screenshot shows the "Hello World" message. As you can see, the custom formatter does not use XML, it writes the string data directly. In this case, the body size goes from 51 bytes to 11 bytes, so this should be used when you really don't need the payload in XML.

Message Properties

Every message under MSMQ is an object with properties that determine how the message should be handled and processed. Some of these properties are set by default and suffice for the most common situations, while some should be explicitly set to values appropriate for your application. In this section, we will examine some of the important properties that can be specified on a message object.

  • Acknowledgement: This property is set by the MSMQ runtime and specifies what kind of acknowledgement the message is. This could be a positive ack, which indicates that the message was successfully sent and retrieved, or it could be a negative ack which specifies the error that occurred that stopped the message from being delivered.
  • AcknowledgementType: This property is set by the sender to control whether the receiver wants any kind of acknowledgement about the status of the message. The sender could specify any positive or negative ack to specify the status of the message. This property could be turned off if the sender does not need to know the status of message delivery.
  • Body: This property contains the message contents. For a sender, this is the message content before serialization and for a receiver, this is the message after serialization.
  • Priority: This property specifies the priority of the message. This controls how the messages are retrieved from the queue. Messages with higher priority are processed before the messages with lower priority. This should be used to indicate the urgency of the message to the receiver. A good example of this is the TCP protocol, which sends out of band data that contains control information that needs to be processed as soon as possible.
  • UseDeadLetterQueue: This property specifies whether the message should be stored in the Dead Letter queue on failure. This is useful to find messages that could not be delivered, and can be used by administrators to determine the cause of failure and\or to resubmit the message.
  • UseEncryption: This property specifies whether the messages should be encrypted. By default, messages are sent as raw data and can be intercepted and read. If the message contains sensitive data that should not be compromised, encryption should be turned on for the message.

Messages support quite a few other properties that define the semantics of how the message should be processed. Setting and retrieving these properties for every message can take time and affect the performance of your application. If the sender or receiver only intends to look at a subset of a message's properties, it should communicate this to MSMQ, so that only the specified properties are made available. This conserves resources and improves performance. MSMQ handles this via the MessagePropertyFilter class. Each message queue object has a corresponding MessagePropertyFilter object that specifies what message properties should be retrieved. The MessagePropertyFilter class has a boolean flag for each property that the Message class supports, which should be set or unset.

Transactional Queues

Messages in MSMQ are delivered to the destination queue by the intermediate MSMQ servers. Given the queue and message semantics, it is possible that out of two messages intended for the same destination queue, one might be delivered and the other might fail. This could occur because of routing delays, causing the message to expire, or because the queue is full or quite a few other errors. When messages are sent as a single self contained unit, this is acceptable since the message does not care about other messages in the system. However, it is possible that under certain situations, you want successive messages to be related and want them all to be delivered or none at all. Essentially, you want messages to be a part of a transaction, and have all-or-none semantics. A very common and highly overused example of this is transferring money from one account to another, where one message describes the amount being deducted from one account, and the other message describes the amount being added to the second account.

MSMQ supports this via Transactional Queues. Transactional queues guarantee that all the messages will be delivered together in the order in which they were sent, and that the messages will be delivered only once. Thus if a client sends two messages, M1 and M2, it is guaranteed that the receiver will either see both M1 and M2, or none at all, and further that the receiver will receive M1 before M2. A queue has to be created as transactional in order for it to participate in transactions. This is done programmatically when the queue is created.

Once a transactional queue is created, you can use the BeginTransaction, CommitTransaction and RollbackTransaction methods to send transactional messages.

   Message lRequest1 = new
            Message ("Message 1");
Message lRequest2 = new Message ("Message
            2");
MessageQueue lMQClient = MessageQueue.Create
(@".\Private$\MyTransactionalQ",
            true);

try{
lMQClient.BeginTransaction
            ();
lMQClient.Send (lRequest1);
lMQClient.Send
            (lRequest2);
lMQClient.CommitTransaction ();
}
catch
            {
lMQClient.RollbackTransaction ();
}

Note that transactions are currently supported only on a single queue. As a result, if you want to send messages to multiple queues and maintain transactional semantics, MSMQ does not support this. You would have to use the DTC (Distributed transaction Coordinator). to support this as an external transaction, which would then use the MSMQ services as a resource manager. Note that using transactional queues incurs a hit to performance, and you should use transactional queues only if necessary.

Asynchronous Queues

Under MSMQ, the sender returns immediately, irrespective of when the receiver retrieves the message. However, as far as the receiver is concerned, it has the option of retrieving the messages synchronously or asynchronously. Asynchronous retrieval is useful when the messages being retrieved contain large object or contain multiple properties that must be retrieved. Rather than waiting for the message to be completely retrieved and deserialized, the receiver can specify an asynchronous delegate that will be called when the operation is complete. At this point, the receiver can retrieve the message in its entirety and process it.

public void ReadMessage ()
{
MessageQueue lMQServer = new MessageQueue (@".\Private$\TestQ");
lMQServer.ReceiveCompleted += new
ReceiveCompletedEventHandler(ReceiveCompletedHandler);
lMQServer.BeginReceive();
}

public static void ReceiveCompletedHandler
(Object source, ReceiveAsyncEventArgs asyncResult)
{
MessageQueue lMQServer = (MessageQueue)source;
Message lRequest = lMQServer.EndReceive(asyncResult.AsyncResult);

lRequest.Formatter = new XmlMessageFormatter (new
string[]{"System.String"});
Console.WriteLine(lRequest.Body);
}

Security

MSMQ is designed to be an enterprise level messaging system. As a result, MSMQ needs to provide mechanisms to guarantee secure delivery and retrieval of messages. A thorough discussion of security is beyond the scope of this article, but we will touch upon some of the features that are available to design secure messaging systems. MSMQ provides access control, authentication, encryption and auditing support for messages and queues.

  • Access Control: MSMQ uses standard Windows ACLs (Access Control Lists) to allow or deny access to queues. Administrators can set up security on queues, so that only certain users will have access to specific queues. For example, the following screen shot shows that the user KAUSHAL_HOME2\kaushal only has access to receive messages from the TestQ queue, while members of the Administrators group have full permissions. By correctly specifying the access groups and permissions, administrators can use built in Windows security to grant or deny access to the queue.

  • Authentication: If access control via Windows domain users is not sufficient, MSMQ provides authentication based on certificates and Kerberos. Authentication is usually two-way, where the client and the server both need to verify each other's identity. The client needs to verify the server's identity so that it does not send sensitive messages to a malicious server, and the server needs to validate the client to ensure that only authorized clients can access the queue. MSMQ uses Kerberos, and the sender uses PKI, to prove each entity's identity.
  • Encryption: In addition to the MSMQ server and sender\receiver verifying the other's identity, we need to guard against the data being sniffed. If the data is being sent over a network, it is possible for anyone with a network sniffer to capture all the data that crosses over the wire. This is supported by MSMQ by encrypting the message before it is sent over the network. Encryption is an expensive operation, and this should be turned on, only if required.
  • Auditing: MSMQ provides means to authenticate the users and encrypt the data across the network. However, it is also necessary to provide administrators with an audit trail of all operations on the queue. This is useful to trace all activity that took place on a queue. MSMQ allows you to specify what activities should be audited for specific queues. In the following example, for our TestQ, auditing is set to log all Failed attempts by all users, and to log all Successful operations carried out by a member of the Administrators group.

The Sample Application

I have included a sample application so you can test message queuing for yourself. The buttons are fairly explanatory - enjoy trying it out.

Conclusion

Message based communication provides a very powerful way to build highly available applications. Designed correctly, you can virtually ensure that your system will always be running even if individual components are temporarily unavailable or unreachable. .NET introduces a clean object based framework to communicate with the MSMQ servers that makes developing message passing systems easy to develop and deploy.

Happy .NETing.

If you'd like to discuss a broader range of C#Today topics and issues, or if you have any C# related questions you need answers to, please visit our online resource centres at : http://p2p.wrox.com/list.asp?list=csharptoday_discuss&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq and http://p2p.wrox.com/list.asp?list=c_sharp&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq


Mobile Enabling Existing Applications with the Mobile Internet Toolkit

Mobile Enabling Existing Applications with the Mobile Internet Toolkit
http://www.csharptoday.com/content.asp?id=1568
Article Rating: 4
Published on September 5, 2001
Abstract
With every new technology wave one goes about building applications with new tools and frameworks. Quite a lot of development time and cost are spent on building applications with current technology. However protecting ones investment is paramount. This article is aimed at protecting your existing development investment – by taking your existing applications and moving them forward in to the new wave of the wireless world. Sanjay Shetty shows us how to do this with the aid of managed COM and the Microsoft Mobile Internet Toolkit.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

With every new technology wave one goes about building applications with new tools and frameworks. Quite a lot of development time and cost are spent on building applications with current technology. However protecting ones investment is paramount. This article is aimed at protecting your existing development investment - it does so by taking your existing applications and moves them forward in to the new wave of the wireless world.

Most of us developing in the Windows world are familiar with COM and object oriented development; Many of our applications are built using COM. We'll begin by examining one such application - a banking application. We will extend it using two new frameworks; primarily Microsoft's Web Services model based on the .Net framework beta 2 and the Mobile Internet Toolkit beta 2.

Prerequisites

I assume in this article that you've had some experience with the .Net Framework Beta 2 and the Mobile Internet ToolKit Beta 2. I recommend that you set up the sample code present in source.zip at this stage, by following the instructions present in the setup.doc file. This will enable you to follow the article and understand the process more easily.

Once we've done the above, let's start by understanding what a web service is and then look at the existing application in more detail.

What is a Web Service?

As defined by Microsoft a web service is "A programmable application component accessible via standard Web protocols". In other words, it's a component that can be called remotely, over the Internet, from a client application, using standard web technologies such as XML and SOAP.

It's about making your application available in a manner in which existing or other applications, irrespective of the platform, may consume or use parts of our application.

Web services have been touted as the next evolution of the Internet, where applications are accessed over the web. E.g. Altavista's translation service or Microsoft's Passport authentication and wallet service. So in our applications, most of which require authentication services, why not use the Passport service developed by Microsoft? Another way to put it would be if we have an existing application then one can expose it in a standard manner such that other applications which one builds can use it over the web. Web services are all about the programmable web.

Current scenario:

Let's assume we are part of a bank and make information available to our customers via a web site in addition to ATM, which were present at a fixed location. So we've now provided our customers with the ability to view details about their bank accounts with us online. The figure below shows a standard login screen for such a bank.

A customer could perform a number of functions, like checking their account balance or view a statement online. Figure 2 shows the user after logging in, where the bank offers a choice of viewing the account balance or latest statement.

We've developed this application using COM components and ASP at the front end to serve online clients. The task in front of us is to now futureproof our current application. The first goal is to ensure that clients with WAP enabled mobile phones can view their account balances and their current bank statement. Essentially the business logic is the same but the presentation logic needs to be changed.

We have certain restrictions though:

  • A tight budget
  • An ever-changing device environment. Today the mobile phone, tomorrow the pocket pc, or a smart phone.

The solution is to use the web services model and the Microsoft Mobile Internet ToolKit.

Lets look at our current application in more detail. It represents a basic online web page where a bank allows their customers to view their bank balance and their bank statement online. This page is coded in traditional ASP. It makes a call to a COM component netdb.dll. This component has 3 methods

verifyUser - essentially accepts a user name and password and checks against a database for existence of a customers details.

  • getBalance - retrieves the current balance amount for a user using the account id.
  • getStatement - retrieves the current statement for a user from the transaction table again using the account id. This includes details of money withdrawn from an ATM or a check that has been deposited or cleared.

Our task is to take this current application and future proof it. Since we're building on the Microsoft platform and the future out there is the .NET Framework we're going to do everything on that framework.

Managed code

Our first task is to convert our existing COM component to a managed component on the .NET framework. We're not talking about recoding our component, but making it .NET usable. The process of converting a COM component to a managed creates metadata about the COM component so that the .NET framework can use it - that is all - we do not need to recode our component.

In order to do this, we use a utility available with the framework called the tlbimp.exe. tlbimp.exe is found in the Bin directory of the .NET Framework SDK installation directory. This utility generates the required metadata information by reading a COM type library. The output of this utility is a binary file that contains the metadata of our COM component. The following command line command enables creation of this wrapper:

tlbimp netdb.dll NETDBLib.dll

Tlbimp essentially creates a wrapper for our COM component making it usable on the .NET platform. We've called our wrapper NETDBLib.dll. There will now be a new namespace called NETDBLib with all the methods and classes of the COM DLL.

Once we've created a .NET compatible component, we need to expose this component as a web service. In the .NET framework a web service is essentially an .asmx file, which contains code enabling one to consume an application over the web programmatically using standard protocols such as SOAP (usually XML over HTTP). While using the .NET framework one need not bother about the internals of SOAP messaging and writing SOAP code as the .NET framework hides and does that dirty work for us.

Creating a Web Service

Let's now expose this managed component of ours into a web service via an .asmx file. In the Web Service code(.asmx) we define WebMethod() functions which act as an entry point for our Web Service clients. A WebMethod tag before your method names indicates to the framework that these methods are your web service methods, which you wish to make available to other consumers. In these methods you can invoke your COM component (through the .NET wrapper) and call the appropriate methods. The only things to consider are the return values. There is a limited range for return values in a .NET Web Service. For more information about return value types see http://www.aspng.com/quickstart/aspplus/doc/webservicetypes.aspx?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq.

Let's take a look at the web service code. As I mentioned earlier our COM component has three methods. The web service code will consist of methods with the same signatures. Check the code below - it has three methods; verifyUser, getBalance and getStatement, each accepting the same input and providing the same output as our COM component.

<%@ WebService Language="c#"
            Class="Bankcpp" %>

using System;
using
            System.Collections;
using System.Configuration;
using
            System.ComponentModel;
using System.Data;
using
            System.Diagnostics;
using System.Web;
using
            System.Web.Services;
using
            System.Web.Services.Protocols;
using System.IO;
using
            NETDBLib;

public class Bankcpp :
            System.Web.Services.WebService
{
CDataService ds=new
            CDataService();

[WebMethod]

public int
            verifyUser(string User,string Password)
{

return
            (ds.verifyUser(User,Password));
}

[WebMethod]

public
            string getBalance(int AccId)
{
return
            (ds.getBalance(AccId));
}

[WebMethod]

public string
            getStatement(int AccId)
{
return
            (ds.getStatement(AccId));
}
}

An .asmx web service file begins with <%@ WebService Language="c#" Class="Bankcpp" %> essentially these directives indicate to the framework that this is a web service coded in the language C# you could be using VB too, and the class which one is using is Bankcpp.

We reference our assembly at the top of the code with the using NETDBLib; command (this assembly should be present in a "bin " directory created at the same level in which our web service exists).

We then declare our class in the normal way and indicate that it inherits from the system web service class.

public class Bankcpp : System.Web.Services.WebService

We also create an instance of our class in the COM component and called it "ds"

CDataService ds=new CDataService();

Next, we declare our methods with a signature similar to those present in the COM component. Within each method we make a call to our instance of the managed code.

return (ds.verifyUser(User,Password));

For instance here we call the verifyUser method of the component and pass it the username and password as parameters.

Once we've created the asmx file we've essentially exposed our code and made it SOAP callable.

Open the bankcpp.asmx file that you'll find in the source code provided in your browser window. When the .NET framework finds a .asmx file containing the WebMethod directive it generates code associated so that our web service is exposed in the correct manner making it web callable. If you want to see the associated SOAP code defining this web service, you need to enter http://www.csharptoday.com/HYPERLINK%20view%20the%20SDLhttp://localhost/bankcpp/bankcpp.asmx?wsdl&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq to view the WSDL contract which other consumers need to follow when invoking the application you've just exposed. (Here I'm assuming you've hosted the application on your local server (localhost) following the instructions present in the source code file.

It would be a good idea to save this output as a .wsdl file, as we will be using it later. Save it as BankCpp.wsdl in the same directory as bankcpp.asmx.

Now we can use this functionality that we have exposed in our mobile application.

Review

Let's cover what we have achieved so far. In order to expose existing code as a web service we first made a .NET compatible wrapper for our COM component, then created a .asmx file(web service file), via which we exposed the managed code as a web service using the [WebMethod] directives.

Let's now build a front-end mobile application using the Microsoft Mobile Internet ToolKit and consume this web service we've created.

Consuming a web service

The .NET framework provides a utility called wsdl.exe, which takes a WSDL file as input and generates a proxy which can then be used by your client pages.

/language:VBwsdl /namespace:BankService http://localhost/bankcpp/bankcpp.asmx

This generates a source file called BankCpp.cs and specifies a namespace BankService that will contain all the proxy code.

Using the C# command line compiler, csc.exe, we can compile our C# source file:

Csc.exe /t:library BankCpp.cs

What we've done with the command line compiler is specify 2 things:

  • Create a library (dll) rather than an executable (exe) as indicated by the "library" keyword
  • Secondly specify the source file to compile, BankCpp.cs

The result is BankCpp.dll, a complete proxy to our BankCpp.asmx ASP.NET Web Service that we can now use in .NET applications to communicate via SOAP to our Web Service. Let's use this proxy to build Mobile Internet ToolKit based ASP.NET pages that consumes our Web Service.

Let's go Mobile

Mobile web pages are pretty simple to create. What we essentially need to do here in our code is reference the proxy which will make calls to our web service. If you recollect the way to reference a library in C# is via the "using Namespace " statement. In ASP.NET one uses the similar statement Import Namespace="namespace".

<%@ Page
            Inherits="System.Web.UI.MobileControls.MobilePage" Language="C#"
            %>
<%@ Register TagPrefix="Mobile"
            Namespace="System.Web.UI.MobileControls"
            Assembly="System.Web.Mobile"%>
<%@ Import
            Namespace="BankService"%>

The first two lines of any Mobile Internet ToolKit page are always the same, where one inherits from System.Web.UI.MobileControls.MobilePage and specify the TagPrefix of "Mobile " to indicate all the mobile controls on the page. In addition on Line 2 one specifies the Assembly System.Web.Mobile. When the .NET Framework compiles this page any control which has the tag Mobile:ControlName will be evaluated as a Mobile control belonging to the namespace System.Web.UI.MobileControls.

We then import the Bankcpp namespace that contains our web service.

Look at the code below, which is the login.aspx page:

<%@ Page
            Inherits="System.Web.UI.MobileControls.MobilePage" Language="C#"
            %>
<%@ Register TagPrefix="Mobile"
            Namespace="System.Web.UI.MobileControls"
            Assembly="System.Web.Mobile"%>
<%@ Import
            Namespace="BankService"%>

<script runat="server"
            language="c#">

protected void GoCommand_OnClick(Object
            sender, EventArgs e)
{
int AcctID;
try
{
Bankcpp
            DataService = new Bankcpp();
AcctID =
            DataService.verifyUser(AcctIDEdit.Text,
            PasswordEdit.Text);
}
catch(Exception excep)
{
AcctID =
            -1;
}
if (AcctID > 0 )
{

Session["AcctID"] =
            AcctID;

Response.Redirect("operation.aspx");
}
else
{
ErrorMessage.Text
            =AcctID.ToString() + "Not Logged
            on";
}

}
</script>

<Mobile:Form
            runat="server" >

<Mobile:Label
            runat="server">Account
            ID:</Mobile:Label>
<Mobile:TextBox runat="server"
            id="AcctIDEdit" />
<Mobile:Label runat="server">Enter
            Password:</Mobile:Label>
<Mobile:TextBox runat="server"
            id="PasswordEdit" Password="true" />
<Mobile:Command
            runat="server" OnClick="GoCommand_OnClick" Text="Go!"
            />
<Mobile:Label runat="server" id
            ="ErrorMessage"/>

</Mobile:Form>

The code immediately above this paragraph defines a mobile form, which allows the user to enter the Account ID and password. Refer to the figure below. When the user enters an appropriate user ID and password ( e.g. Account id "np001" and password "n") and then chooses the Options menu on the phone and presses the Go button, it runs the code attached to the OnClick of the same, which is GoCommand_OnClick. This function creates an instance of our proxy BankCpp and invokes the VerifyUser method with the parameters AcctIDEdit that contains the user's account ID and password.

If the account id is valid it allows the user to proceed to the next page, operation.aspx, where the user can then decide whether to check the bank balance or view the account statement. Source.zip contains a file called Account id and pass.txt that lists valid account id's and passwords.

Mobile Capabilities

The Mobile Internet ToolKit is built upon an extensible framework that provides support for a variety of devices, and future support for additional devices. These mobile devices may vary greatly in capabilities, such as screen resolution, color, and input devices available. These differences affect the way Mobile Web Forms pages are rendered.

To work with these differences, the Mobile Internet ToolKit includes an extensible Mobile Capabilities component, built upon the Browser Capabilities feature of ASP.NET. This component identifies the client device, collects together capability information from multiple sources, and makes it available to the Mobile Web Forms page and the mobile controls. This capability information is also available to your application, in case you want to alter your application logic for different devices.

You could also check login.aspx in a plain vanilla browser such as IE. The Mobile Internet ToolKit framework at the back end figures out the kind of device being used to browse the page and generates pages accordingly. The beauty of this is that these same pages can be viewed on a Pocket PC too without a single line of code change.

The figure below shows the same page (login.aspx) when viewed by a PocketPC. Without rewriting code one can now service users with different wireless devices.

There are a number of devices, which the Mobile Internet ToolKit fully supports:

  • Mitsubishi T250
  • Nokia 7110
  • Pocket PC with Microsoft Pocket Internet Explorer version 4.5

Microsoft has performed limited testing for the following devices:

  • Ericsson R380
  • Microsoft Internet Explorer 5.5
  • Microsoft Internet Explorer 6.0
  • Mitsubishi D502i
  • NEC N502i
  • Nokia 6210
  • Palm VIIx
  • Palm V
  • Panasonic P502i
  • RIM Blackberry 950
  • RIM Blackberry 957
  • Samsung 850
  • Siemens S-35i
  • Sprint Touchpoint phone

Microsoft has performed limited testing using the following device emulators:

  • Microsoft Mobile ExplorerT v 2.01 emulator
  • Pocket PC with Microsoft Pocket Internet Explorer version 4.0
  • Phone.com UP.SDK 4.1 emulator with generic skin
  • Phone.com UP.SDK 3.2 for WML emulator with Mitsubishi T250 skin
  • DoCoMo 502 emulator

The Mobile Internet ToolKit may support additional devices however they have not been tested.

Last words

We have seen the simplicity of the .NET framework in mobile enabling existing applications. At the end of this article we might ask - why not just consume our earlier original COM component via our mobile pages? It is true we could do that, but then we're not future proofing the application by building a solution on open protocols like SOAP, XML and HTTP, or enabling our application to be available to others in a programmable way, making way for new revenue streams which were earlier not conceivable. For example, we can now make available functionality within our application to others, like providing merchants with the ability to do a credit check against the bank balance of a user, to verify whether the user has sufficient balance to make payment for a particular service. Essentially once one has exposed functionality as a web service it can be easily integrated along with another service programmatically.

Another thing to notice is that we haven't used the Visual Studio.Net IDE at all. I've done this on purpose to make sure we understand what is happening beneath the hood, and one gets a better understanding of the subject matter.

You could view both the asp pages and the aspx pages associated in the article with standard browsers such as Internet Explorer. However the best way to check this is by downloading a mobile emulator such as the one from http://www.yourwap.com/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq and using the Pocket PC emulator provided along with the Microsoft embedded visual tools.


Internationalization With The .NET CultureInfo Class & C#

Internationalization With The .NET CultureInfo Class & C#
http://www.csharptoday.com/content.asp?id=1635
Article Rating: 4
Published on August 17, 2001
Abstract
CultureInfo is one of about 25 classes, structures, and enumerations that make up the System.Globalization namespace of .NET. Although having only a few methods and properties, CultureInfo is used by many other components from the Globalization namespace to determine how date, time, currency, numbers, and calendars are managed and rendered in order to meet cultural expectations in a particular world region (culture). CultureInfo also affects sorting and text issues. Indeed, in .NET a CultureInfo object plays the role of a Locale Identifier in Win32, the setlocale function on Posix compliant systems, or a Locale class in Java by controlling the international behaviors of other objects or API’s. In this article, Bill Hall shows us the methods and properties provided by CultureInfo along with some examples of practical usage when programming in C#.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

CultureInfo is one of about 25 classes, structures, and enumerations that make up the System.Globalization namespace of .NET. Although having only a few methods and properties, CultureInfo is used by many other components from the Globalization namespace for determining how date, time, currency, numbers, and calendars are managed and rendered in order to meet cultural expectations in a particular world region (culture). CultureInfo also affects sorting and text issues. Indeed, in .NET a CultureInfo object plays the role of a Locale Identifier in Win32, the setlocale function on Posix compliant systems, or a Locale class in Java by controlling the international behaviors of other objects or API's. In this article, we will show you the methods and properties provided by CultureInfo along with some examples of practical usage when programming in C#.

The Importance of Writing Internationalized Code

The classes provided by the System.Globalization namespace are intended to help developers and architects design and write applications and web pages that meet cultural conventions worldwide. Today, almost every piece of production code requires some degree of international support. Indeed Internationalization (usually abbreviated as I18n since 18 letters span the space between the first 'I' and the last 'n') is really an engineering and design task. However, I18n continues to be regarded by most programmers as an esoteric area that can only be managed by those that are literate in several languages and numerous cultures. Nothing could be further from reality. It is not so difficult, with a little bit of knowledge and practice, to write code with global use in mind as part of the normal development cycle. Fixing it later upsets planned release dates, breaks working code, and forces extra QA cycles. Incorrectly regarded as a feature, I18n is found at every level of a program, and some issues can be buried quite deeply. As you will see, it is about as easy to do it correctly as wrongly, but you have to develop an awareness of when a line of code or a routine needs I18n attention, and you must certainly forget some things that you learned in school or from otherwise excellent books and articles on programming!

Example

Very often otherwise well written code can be rewritten to work properly anywhere in the world by minor modifications. Let's look an example from C#. Create a C# Console project - I called mine ShowWeekDays. The first step is to import the System.Globalization Namespace to get access to the needed classes. We will add code that generates and displays a list of weekday names. This is the kind of task you might have to do in a web page or dialog so that the user can select a particular day for some purpose. Unfortunately, so often when programmers face this particular problem, they simply embed an array of weekday names, usually in English but sometimes in their own language, into the code. That might work if the program never crosses the border, but even then you may have local users who prefer to see such information in the language for which they have configured their computer. You could, of course, move the list of names to a Resource file for translation, but it is better to depend on the operating system to provide culturally correct output.

As a first try, let's proceed in a way most programmers solve this problem. Here is some code that will create the desired list in C# using the System.DayOfWeek enumerator. In the Main method add the following code:

DayOfWeek d; //enumerator in System namespace
Console.WriteLine("Generate a list of weekday names as you were taught in school");
// assume the week runs from Sunday to Saturday.
// not true in most places in the world but we have no other information.
for (d = DayOfWeek.Sunday; d <= DayOfWeek.Saturday; d++)
Console.WriteLine(d.ToString());

Run the program to see a list of weekday names in English with the first day of the week as Sunday, last day Saturday.

Next, change the User Locale on your system to French (France) (if not already set). Don't know how to do this? Go to the Control Panel and open the Regional Settings dialog. Choose French (France) from the "Your locale (location) " dropdown list and press "Apply " and/or close the dialog.

Run ShowWeekDays.exe again and you will get the same output as before, even though you as the user have expressed a wish that your computer display date, time, numbers, and the like in a format suitable for France. Remember, this is the kind of setting that your customers make. If you don't meet their expectations they may be so disappointed as to actually buy competing software that does!

To fix the deficiency, comment the code already written and add the following lines:

Console.WriteLine("Generate a list of weekday names your customer can understand");
// get a DateTimeFormat used by the Current Culture (your User locale)
DateTimeFormatInfo dtf = CultureInfo.CurrentCulture.DateTimeFormat;
// generate a list of day names as used in the culture
String [] daysofweek = dtf.DayNames;
// set the first day of the week in this culture
int j = (int)dtf.FirstDayOfWeek;
// find out how many days are there in a week
// it is usually seven but don't assume anything!
int numdays = daysofweek.GetLength(0);
// Now list them in the right order and language
for (int i = 0; i < numdays; i++, j++)
Console.WriteLine(daysofweek[j % numdays]);

As well as the all important:

using System.Globalization;

at the top of the code.

Here is how the output appears. Except for the title bar text and maybe the screen colors, yours should look the same. The complete project is included with this article.

Note that the weekday names are localized to French and the first day of the week is Monday (lundi). In some cultures, the week starts on other days (Saturday is common in the Middle East, for example). You may also think it strange that we were careful about the number of days in a week, and I have to admit that I don't know of any cultures having a different number of weekdays. However, it is best not to make any assumptions. You may also believe that there are just twelve months in the year, but some calendars have 13! For example, lunar calendars retrograde from the solar cycle over a period of about 33 years, so in some cases, an extra month (leap month) is inserted from time to time to synchronize roughly with the solar calendar. The Hebrew Calendar is a good example: its 13th month is Adar. For an interesting reference see http://www.geocities.com/Athens/1584/%20/l%2001?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq. By the way, the .NET System.Globalization.DateTimeFormatInfo makes provisions for 13 months

This example is typical of how a CultureInfo object is used to modify the default behavior of another class to meet user expectations - in this case DateTimeFormatInfo. Another point about the code is that it does the right thing regardless of locale and language without further action on your part. Not even a translator is required! More importantly, by a small change in the algorithm and three more lines of code, we get a result that does the assigned task. However, it is not yet completely internationalized as we will now see.

CultureInfo Methods and Properties

Now that you have seen a use for CultureInfo in a practical situation, let's explore it more deeply. For this purpose we have created a C# form that provides a list of all the CultureInfo objects available on a system and displays some results of the methods and properties. We will discuss the meaning of each displayed element and will show a useful programming technique that can help you store and retrieve results in a language / locale independent way from data that may have to be localized. By the way, Localization is the art of translating software from one language to another. It is often abbreviated as L10n (ten letters between the first and last letter). L10n and I18n together make up the task of Software Globalization (G11n). In simplistic terms, the equation for world-ready software is G11n = I18n + L10n. Keep in mind that L10n requires linguistic skills but responsibility for I18n rests squarely on the programmer's shoulders. If you think I'm trying to make you feel a bit guilty about ignoring I18n issues you are correct! But, it is better (and cheaper!) to do it right once than to do it twice.

Generating a List of Cultures

If you have a list of items from which a user must make a selection you have two problems. One is to provide a friendly name that the user can understand; the second is to associate the name with the data you really want to use after a selection is made. The problem can get very messy if correct data retrieval depends on the friendly name's content and unbeknownst to you it gets translated!

Windows solved this problem by providing the programmer with a way to associate a display string with a hidden 32-bit field called Item Data, which could be used as required by the programmer to tie together presentation strings and action data. C# (and J++'s WFC by the way) enhances this feature by allowing you to simply place an instance of an entire class as an item in a list or combo box. When the user makes a selection, the programmer has access to all the property and methods of the object placed there. Unfortunately, the display string that the user sees is determined by the object's ToString() return value, whose content can be distinctly unhelpful to the ordinary user. In the case of CultureInfo for example, the string is an RFC1766 compliant, language-country pair such as en-GB, ja-JP, etc., which is also one of the two possible data types that can be used in constructing one of these objects (the other is a Win32 locale ID or LCID).

Fortunately, the programmer often has recourse. Provided the class is not sealed, you can derive from it and override ToString. That is what we do here. As seen in the code below, ToString is replaced by a call that produces an output whose content is readily understandable but about which we care nothing except for its presentation value. But, when we pick up the selection from the list of Cultures, we have the associated CultureInfo object that we really need and can use it as required. Furthermore the list is sorted according to the user's current Culture and we did not have to do anything to achieve this very desirable result. Finally, since what the user sees is actually the object's Display Name property, it is also localized according to the platform's language, thereby eliminating any need for translation.

Here is the code for MyCultureInfo.

using System;
using
            System.Globalization;

namespace NewCultureInfo
{
public
            class MyCultureInfo : CultureInfo
{
public
            MyCultureInfo(string name) : base(name)
{
}
public
            MyCultureInfo(string name, bool useoverride) :
base(name,
            useoverride)
{
}
public override string ToString()
{
return base.DisplayName;
}
}
}

If a class is sealed, you can try creating another class that has the base class as a member variable. I once had to use this technique in Java since the java.util.Locale class is final (the equivalent of sealed in C#) and so cannot be extended.

Although CultureInfo actually has four constructors (the other two use a Windows Locale ID (LCID) in stead of an RFC 1766 string) we only needed two for this program. The second parameter, when it is used, allows a CultureInfo class to use User overrides instead of the default data for the Culture. In the sample code, I allowed only the Current Culture to be overridden just to exercise the API. Normally, you want to allow overrides since then the Culture will reflect user customizations such as those made in the Regional Settings dialog. Other programs actually watch for changes here and act on them when they occur. For example, double-click on My Computer and select the ' Details' view. Open a folder and keep it visible in the next step. Run the Regional Settings dialog from the Control Panel, change the short date format on the Date tab to something else, and press Apply. You will see the new date format reflected immediately in your folder list.

Here is the code (located just after the InitializeComponent() call in the sample program) that loads the dropdown list of cultures. You should note that in the real programming world, enumerating all the cultures is not a common I18n task. But, I want you to see just how the derived and base classes are related.

// get all the cultures available and set the selection to the current one
CultureInfo [] ci = CultureInfo.GetCultures(CultureTypes.AllCultures);
// get the current culture since we want to make a good initial selection
// and we will set the current culture to use user overrides
CultureInfo cc = CultureInfo.CurrentCulture;
foreach(CultureInfo c in ci) {
// add MyCultureInfo objects to the dropdown list of cultures
// make them all read only
MyCultureInfo mci = new MyCultureInfo(c.Name, c.Equals(cc));
cbCultures.Items.Add(mci.IsNeutralCulture ? mci :
CultureInfo.ReadOnly(mci));
}
// set the initial selection
cbCultures.SelectedIndex = cbCultures.FindStringExact(cc.DisplayName);

Below we can see a view of the program. You can see the amazing independence of the various locales on a Windows 2000 system. First, the program itself is displaying various methods and properties for Egypt, where the principal language is Arabic. On the other hand, if you look at the Static Properties group box, you can see that the Current Culture (as set by the Regional Settings dialog) is de-LI for German (Liechtenstein). The Current UI Culture is Japanese (ja-JP). Finally, the Installed UI Culture is English, United States (en-US).

Program Operation, Properties, and Methods

Each time the user selects a new Culture, all the controls except the Static Properties group are redrawn. One interesting feature is that for certain Cultures, we changed the Native Name text direction from right to left if the script is bi-directional. Currently .NET supported bi-directional languages include Arabic, Hebrew, Farsi, Urdu, Syriac, and Devehi. Detecting one was done by hard-coding a list of parent culture names that are known (by us) to use bi-directional text and switching the control's text direction when one was chosen. Unfortunately, I have not been able to find a way to determine text direction programmatically. It does not seem to be a part of the System.Globalization namespace.

Except for the list of cultures and the calendar list there are no controls in the program that allow user interaction. However, each control contains a tool tip that either describes the output or gives the CultureInfo member or property name.

The results in the Static Properties group depend on how the user has set up Regional Settings as to the locale and the desired user interface language. By the way, if you are using the English version of Windows 2000 Pro or Server (but not Advanced or Commerce Server), you can add up to 23 additional user interfaces. Windows XP Pro offers as many as 33. You can find out more by going to http://www.microsoft.com/globaldev/FAQs/MultiLang.asp?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq. If you are an MSDN subscriber, look for a MUI (Multiple User Interface) directory on the appropriate platform CD or DVD.

In the Instance Properties group you will see several objects from System.Globalization that you can obtain from an instance of CultureInfo. Below is a brief description of each.

  • Calendar: Represents time in divisions such as week, months, and years
  • CompareInfo: Provides methods for culture-sensitive string comparisons
  • DateTimeFormatInfo: Controls the display of DateTime values according to culture
  • NumberFormatInfo: Determines how numbers and currencies are formatted according to culture
  • TextInfo: Supplies properties and behaviors specific to a writing system

In this program, we have merely displayed the ToString result obtained from the class; discussing each one would require a separate article. But, already we have seen how useful it can be to be able to get a DateTimeFormatInfo class from a particular CultureInfo instance and use it to get culturally correct results, and this is typical usage.

The Calendars list shows all the calendars that the culture can support. The ToString result is not very helpful in describing them since the same output may cover different calendars. However, at least for Gregorian calendars you can see the various types in the Gregorian Calendar Name field. Many cultures have multiple calendars. The Arabic (Egypt) has five Gregorian types and one lunar (Hijri) calendar. The Japanese culture lists three calendars: one Japanese and two Gregorian calendars. One of them provides data such as formats and month and weekday names in Japanese and the other in English for the Common Era. On the other hand, the Japanese calendar reckons years from Eras such as Meiji, TaiShou, Shouwa, and Heisei (the current one - 2002 is Heisei 14). Although such a calendar may seem 'quaint', your product will have to support it if your program is going to be used by Japanese government institutions.

Informative fields are as follows:

  • Display Name: Friendly name of the Culture
  • English Name: An invariant name for the Culture
  • Locale ID: A Win32 Locale ID
  • Name: RFC 1766 compliant identifier string
  • Native Name: A name in the language of the Culture
  • Three Letter ISO Language Name: ISO-639-2 language identifier
  • Two Letter ISO Language Name: ISO-639-1 language identifier
  • Three Letter Windows Language Name: An identifier that goes back to IBM days

The Name field is the same as returned from a call to CultureInfo.ToString() and is an element of the constructor. Conveniently, a string of the same form can be read from a browser using the HTTP_ACCEPT_LANGUAGE server variable provided the user has set language preferences. Thus a web programmer has a chance to set up an appropriate CultureInfo class for the session.

If you have looked carefully at the Name for each locale, you have probably observed that the part to the left of the hyphen is usually the two-letter ISO-639-1 identifier. The part to the right is normally an ISO-3166 country code but there are occasional variations from this pattern as well to handle complexities such as regions speaking one language but writing in two scripts, etc. For example, Azeri can be written with both Cyrillic and Roman letters. You can find others by scrolling through the list of Cultures in the application.

And, you can still find the venerable Windows/IBM three-letter locale identifiers. Did you know that they are present in a Windows .RC file? They are used to control which language blocks are compiled into a program. Never noticed the flags AFX_TARG_XXX (XXX = FRA, DEU, ENU, etc.)? Not surprising, as they have never been documented to my knowledge.

The two check boxes show whether the selected CultureInfo is read-only and whether it responds to user overrides (explained above). In this program we made all cultures on the list as read-only and set the default culture to respond to user overrides. It would take a bit of programming and an interactive form to illustrate these features well. However, the ability to customize a CultureInfo object is interesting and could be very useful if your needs are not covered by the standard issue.

Another feature of CultureInfo is that each culture has a parent locale. For example, the parent culture of fr-FR, French (France) is simply fr, French. A parent locale may lack certain features; for example you cannot ask for an instance of DateTimeFormatInfo or NumberFormatInfo since they are not only language but country specific as well.

Finally, note the existence of an invariant locale. This locale provides a language/locale independent way of managing certain kinds of data that can later be recovered by a formatter and rendered in a locale sensitive way. For example, you could store dates formatted according to the invariant locale in a database. Later, if you need to display them in a selected culture, it is easy to parse the stored data to get the date fields then recast them into the local format.

Conclusion

Learn to recognize when you may need the CultureInfo class in your programs and take advantage of its many capabilities. It is no more difficult to write internationally enabled code than it is to not write it, especially in .NET. System.Globalization is not unique to C#, and for the first time you have a unified set of tools that can be employed in the same way in Visual Basic, C++, and Jscript.

Oh, by the way, the code at the beginning of the article has embedded strings. Did you notice? In a future article, we will cover this issue in more detail. For now, let's just say that embedded strings in code is a serious impediment to localizing a product; they should always be placed into Resource files. Fortunately, C# has good facilities for isolating strings and other components of the user interface. A very useful example is given in the .NET documentation Walkthrough: Localizing Windows Forms. By following the instructions there, I added a resource file to the ShowWeekDays project included with this article. Here is the modified code: Note that the strings are now located in a Resource file called ShowWeekDayRes.resx that is loaded in Class1's constructor. When required, the strings are obtained by a call to ResourceManager.GetString. There's much more to say; hope this will do for now.

using System;
using System.Globalization;
using System.Resources;

namespace ShowWeekDays
{
class Class1 {
// This object will manage storage and recall of strings
private ResourceManager LocRM = null;

public Class1() {
// Load the object with the strings
LocRM = new ResourceManager('MyResources',
typeof(Class1).Module.Assembly);
}
// returns a string from the assembly resource modude
public String GetString(String keystr) {
return LocRM.GetString(keystr);
}
}
// run the code and display the days of the week
static void Main(string[] args) {
Class1 cs = new Class1();
// Example of uninternationalized code
DayOfWeek d; //enumerator in System namespace
Console.WriteLine(cs.GetString(
'msgMethodNotInternationalized')
);
// assume the week runs from Sunday to Saturday.
// not true in most places in the world but we have no other
// information.
for (d = DayOfWeek.Sunday; d <= DayOfWeek.Saturday; d++)
Console.WriteLine(d.ToString());

// example of internationalized code
Console.WriteLine(cs.GetString('msgMethodInternationalized'));
// get a DateTimeFormat used by the Current Culture
DateTimeFormatInfo dtf =
CultureInfo.CurrentCulture.DateTimeFormat;
// generate a list of day names as used in the culture
String [] daysofweek = dtf.DayNames;
// set the first day of the week in this culture
int j = (int)dtf.FirstDayOfWeek;
// find out how many days are there in a week
//(usually seven but don't assume anything)
int numdays = daysofweek.GetLength(0);
// Now list them in the right order and language
for (int i = 0; i < numdays; i++, j++)
Console.WriteLine(daysofweek[j % numdays]);
}
}

Appendix

Here is the code listing for redrawing the form when the selected Culture changes. You will notice a lot of try blocks since not all cultures have associated DateTimeFormatInfo, or NumberFormatInfo objects associated with them.

// respond to change in selected
            culture
private void
cbCultures_SelectedIndexChanged(object
            sender, System.EventArgs e){
// use this to set a text box right
            to left if needed
bool righttoleft = false;
// parent locale
            string
string par = null;
// useful when try blocks get
            caught
string err = '(null)';
// get the current
            culture
MyCultureInfo curci =
            (MyCultureInfo)cbCultures.SelectedItem;
// get a string for the
            parent culture
if (curci.IsNeutralCulture == true)
par =
            curci.Name;
else
par = curci.Parent.Name;
// we might have
            a right to left script such as
// arabic, farsi, hebrew, urdu,
            syriac, divehi
if ((par == 'ar') || (par == 'fa') || (par ==
            'he') || (par == 'ur')
|| (par == 'syr') || (par ==
            'div'))
righttoleft = true;

// fill in the names of the
            classes that CultureInfo can access
// parent cultures don't have
            all the properties of specific cultures
try
            {
textTextInfo.Text = curci.TextInfo.ToString();
} catch
            {
textTextInfo.Text = err;
}

try
            {
textDateTimeFormat.Text = curci.DateTimeFormat.ToString();
}
            catch {
textDateTimeFormat.Text = err;
}

try
            {
textCompareInfo.Text = curci.CompareInfo.ToString();
} catch
            {
textCompareInfo.Text = err;
}

try
            {
textNumberFormat.Text = curci.NumberFormat.ToString();
}
            catch {
textNumberFormat.Text = err;
}

// check the two
            informational check boxes
checkUseUserOverride.Checked =
            curci.UseUserOverride;
checkIsReadOnly.Checked =
            curci.IsReadOnly;

// set some informative
            fields
textEnglishName.Text = curci.EnglishName;
textName.Text
            = curci.Name;

// make this one right to left if applicable
            script
textParentCulture.Text =
            curci.Parent.Name;
textNativeName.RightToLeft = righttoleft ==
            true ?
System.Windows.Forms.RightToLeft.Yes :
System.Windows.Forms.RightToLeft.No;
// more informative
            fields
textNativeName.Text = curci.NativeName;
// show the
            Windows LCID in hex as well
textLCID.Text = String.Format('{0}
            (0x{1})',
curci.LCID.ToString('d'),
            curci.LCID.ToString('X'));
// fill in informational
            fields
textISO3Name.Text =
            curci.ThreeLetterISOLanguageName;
textISO2Name.Text =
            curci.TwoLetterISOLanguageName;
textWin3Name.Text =
            curci.ThreeLetterWindowsLanguageName;
// is this a neutral
            culture?
textIsNeutral.Text =
            (curci.IsNeutralCulture).ToString();

// generate a list of
            calendars
// this action also causes fill of Gregorian calendar
            name text box
cbOptionalCalendars.Items.Clear();
Calendar []
            cal = curci.OptionalCalendars;
foreach(Calendar c in cal)
            {
cbOptionalCalendars.Items.Add(c);
}
cbOptionalCalendars.SelectedIndex = 0;
}


Building an ASP.NET Mailing List

Building an ASP.NET Mailing List
http://www.csharptoday.com/content.asp?id=1634
Article Rating: 4.5
Published on August 14, 2001
Abstract
In his first article, Doug Seven showed how a guestbook could be made in ASP.NET and C#. In this follow-up, he looks at creating a simple mailing list tool. This uses a SQL Server table to store the names and e-mail addresses of the subscribers, and also provides the ability to send an e-mail to the entire list. This article is bundled with Doug's Guestbook article as a £12 e-doc available from http://www.amazon.com/exec/obidos/ASIN/B00005TZ91/ref=nosim/wrox-20.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

In my last article I showed you how to build a guestbook using ASP.NET and C#. In this follow-up article I will show you how to build a mailing list in ASP.NET and C#. The mailing list will have two parts to it. The first part is a form where visitors to your Web site can sign-up for the mailing list. The second part is a Web Form that you can use to send an e-mail, either in text or HTML format, to all of the subscribers on your mailing list.

For the code in this article you will need:

  • .NET Framework Beta 2
  • Windows 2000 with IIS 5.0
  • Internet Explorer 5.5
  • SQL Server 7.0 or 2000
  • A text editor, such as Notepad, TextPad, or Visual Studio.NET Beta 2

Building the Data Structure

The mailing list application uses an SQL Server table to hold the name and e-mail addresses of your subscribers. Visitors to your Web site can sign-up to your mailing list by entering their information in a form, and submitting it. When it is submitted, the name and e-mail address is added to the table using a stored procedure. In this stored procedure you will first check to see if the user is already in the table, and if they are, you will display a message telling them that they are already subscribed, and not re-enter the data. This will prevent duplicate addresses from being added to your mailing list. If their e-mail address is not found in the table, you will add their name and e-mail address to the mailing list table and display a message indicating that they have been subscribed.

All of the data access is abstracted to a data layer class, named MailingListData. The separation of the application layers is illustrated here:

To begin, create a new table in your SQL Server database named "MailingList ".

  1. Open the SQL Server Enterprise Manager
  2. Navigate to your database
  3. Right-click on the "Tables " node and select "New Table "
  4. Create two columns
  5. Email
    varchar 255
    Primary Key
    Allow Nulls = false
  6. Name
    varchar 200
    Allow Nulls = true
  7. Save the table as "MailingList "

In Enterprise Manager Design Mode, the table should look like this:

The email column will be used as the Primary Key. This will be used to ensure that each e-mail address entered is a unique address, helping to prevent duplicate entries.

Configuration Settings

The SqlConnection s in the MailingListData class are constructed using ConfigurationSettings.AppSettings["conString"] in place of their connection string value. This retrieves a key/value pair from the <appSettings> element in your Web.config file. The <appSetting> element is an optional element that you can add to the configuration file that allows you to define application-wide key/value pairs, such as database connection strings, in a centralized location. Since you don't have the conString key/value pair in the configuration file yet, you will have to add it. Add the following element to your configuration file, inside the <configuration> element.

<appSettings>
<add key="conString"
value="server=localhost;database=ASPToday;uid=sa;pwd=;" />

</appSettings>

The Stored Procedure

Before you build the sign-up Web Form, you need to build the JoinMailingList stored procedure. In the SQL Server Enterprise Manager:

  1. Navigate to your database
  2. Right-click on the "Stored Procedures " node and select "New Stored Procedure "
  3. Enter the following stored procedure:

CREATE PROCEDURE [JoinMailingList]
@visitorName varchar (200),
@email varchar (255),
@result bit output
AS
IF (SELECT COUNT(*) FROM Mailinglist WHERE [Email] = @email) > 0
BEGIN
SET @result = 0
END
ELSE
BEGIN
INSERT INTO Mailinglist
(
[Email],
[Name]
)
VALUES
(
@email,
@visitorName
)
SET @result = 1
END
GO

In the JoinMailingList stored procedure you do two things. First you use the SELECT COUNT(*) function to see if there are any records in the table that already have the e-mail address being submitted. If a record exists, you return an output parameter, @result, with the value 0 (failed). If a record does not exist, you use an INSERT command to add the submitted data to the table, and return 1 (succeeded) in the @result output parameter.

The MailingListData Class Structure

The MailingListData class is the center point of the application. This is where data input by the user will be passed for entry into the database, and where data from the database will be retrieved and passed back to the presentation layer.

Create a new file named MailingListData.cs. Begin building this class by adding the following using statements and class declaration:

using System;
using System.Web;
using System.Web.UI;
using System.Web.UI.HtmlControls;
using System.Web.UI.WebControls;
using System.Data;
using System.Data.SqlClient;
using System.Web.Mail;
using System.Configuration;

public class MailingListData
{

}

In this class you will create two methods, SignUp and SendMail. The first will be used to take in the name and e-mail address of the Web site visitor and add it to the database using the JoinMailingList stored procedure.

The SignUp Method

Add the following method inside the class declaration:

public string SignUp(string visitorName, string email)
{

using System;
using System.Web;
using
            System.Web.UI;
using System.Web.UI.HtmlControls;
using
            System.Web.UI.WebControls;
using System.Data;
using
            System.Data.SqlClient;

public class Signup :
            Page
{
protected TextBox Name;
protected TextBox
            Email;
protected Label ResultMsg;

protected void
            Submit_Click(object sender, EventArgs
            e)
{
if(Page.IsValid)
{
SqlConnection con = new
SqlConnection("server=localhost;database=ASPToday;uid=sa;pwd=;");
SqlConnection(ConfigurationSettings.AppSettings["conString"]);
SqlCommand
            cmd = new SqlCommand("JoinMailingList", con);
cmd.CommandType =
            CommandType.StoredProcedure;
            cmd.Parameters.Add(new
SqlParameter("@visitorName",
            SqlDbType.VarChar, 200));
cmd.Parameters.Add(new
            SqlParameter("@email", SqlDbType.VarChar,
            255));
cmd.Parameters.Add(new SqlParameter("@result",
            SqlDbType.Bit));

cmd.Parameters["@visitorName"].Value =
            visitorName;
cmd.Parameters["@email"].Value =
            email;
cmd.Parameters["@result"].Direction =
            ParameterDirection.Output;

try
{
con.Open();
cmd.ExecuteNonQuery();
con.Close();

if(Convert.ToBoolean(cmd.Parameters["@result"].Value))
{
return
            " - You have been subscribed to the mailing
            list.";
}else{
return " - You are already subscribed to the
            mailing list.";
}
}
}
}catch
{
return "There was
            an error processing your sign-up. Please try
            again.";
}
}

In the Signup method you take the name and e-mail address as input arguments, and then create a SqlConnection and a SqlCommand to execute the JoinMailingList stored procedure. You create three SqlParameters for the SqlCommand: @visitorName and @email, which are input parameters, and @result, which is an output parameter. The SqlDbType enumeration for each parameter is set to the data type expected by the stored procedure. The values for the @visitorName and @email parameters are set using the values of the two method arguments.

Once all of the values are set, the SqlConnection is opened, and the SqlCommand is executed using the ExecuteNonQuery method. This method is used when no result set is expected, such as with an INSERT or UPDATE command. Once the SqlCommand has executed, you check the value of the @result output parameter. This parameter returns a bit data type (0 or 1). You use the Convert class to convert the value to a boolean data type (0 = false, 1 = true). If @result returns false, the method returns a message indicating that the visitor is already a subscriber. In the event of an error, the catch handler returns a user-friendly message. If the @result parameter returns true then a message is returned indicating that the visitor has been added to the mailing list, as shown below.

The SendMail Method

Add the following method inside the class declaration:

public void SendMail(HtmlForm _form, string
            fromAddress,
string subject, string message, bool
            isHtml)
{

using System;
using System.Web;
using
            System.Web.UI;
using System.Web.UI.HtmlControls;
using
            System.Web.UI.WebControls;
using System.Data;
using
            System.Data.SqlClient;
using System.Web.Mail;

public class
            SendMail : Page
{
protected TextBox Message;
protected
            TextBox Subject;
protected TextBox From;
protected CheckBox
            Html;
protected HtmlForm myForm;

protected void
            Submit_Click(object sender, EventArgs
            e)
{
if(Page.IsValid)
{
MailMessage mm = new
            MailMessage();
mm.From = From.Text;fromAddress;
mm.Subject =
            Subject.Text;subject;

if(Html.Checked)if(isHtml)
{
mm.BodyFormat
            = MailFormat.Html;
}else{
mm.BodyFormat =
            MailFormat.Text;
}

}

SqlConnection con = new
SqlConnection("server=localhost;database=ASPToday;uid=sa;pwd=;");
SqlConnection(ConfigurationSettings.AppSettings["conString"]);
SqlCommand
            cmd = new SqlCommand("SELECT * FROM MailingList",
            con);

try
{
con.Open();
SqlDataReader reader =
            cmd.ExecuteReader();

while(reader.Read())
{
mm.To =
            (string)reader["email"];
(string)reader["name"];
mm.Body =
            message.Replace("<--Name-->",
            (string)reader["name"]);
SmtpMail.Send(mm);
_form.Controls.Add(new
LiteralControl("Mail sent to: " + mm.To +
            "<br>"));
Response.Flush();
}
//Flush the page output
            to the
            browser
HttpContext.Current.Response.Flush();
}

con.Close();
}
catch
{
//Add
            error handling
}
}

In the SendMail you take in five arguments: _form, which is a reference to the HtmlForm in the calling Web Form; fromAddress, which is the e-mail address that will appear in the form field of the e-mail; subject, which is the value that will appear in the Subject field of the e-mail; message, which is the body of the e-mail you are sending; isHtml, which is a boolean value indicating if the e-mail should be sent as text, or HTML.

In the SendMail method you create an instance of the System.Web.Mail.MailMessage class. This class instance represents the e-mail that gets sent. You set the MailMessage.From and MailMessage.Subject properties to the values the fromAddress and subject arguments passed into the method. If the isHtml argument is true you set the MailMessage.BodyFormat property to the MailFormat.Html enumeration, otherwise you set it to MailFormat.Text.

Once the basic MailMessage is set up, you create a SqlConnection and a SqlCommand to connect to your database and select all of the records from the MailingList table. You use the SqlCommand.ExecuteReader method to return the result set in a SqlDataReader, which creates a data stream from the database. This is the most memory efficient way to return data that is going to be iterated through from beginning to end. A DataReader returns a basic forward-only, read-only stream of data.

Using the SqlDataReader.Read method, you iterate through the records in the result set. For each record, you assign the email column value to the MailMessage.To property.

The last thing to do with the MailMessage before sending it is to set the Body property. This is set using the String.Replace("<--Name-->", (string)reader["name"]) method. This method creates a new string value, replacing any instances of <--Name--> with the value of the name column value from the SqlDataReader. This is how your e-mail gets personalized. Anywhere that the string <--Name--> appears in the message, the name of the subscriber will appear.

To send the MailMessage, you use the System.Web.Mail.SmtpMail.Send method. This method takes the MailMessage object as its argument. For confirmation purposes, you add a new LiteralControl to the HtmlForm referenced in the _form argument. This LiteralControl is just a text string indicating that the e-mail was sent to the specific recipient. You use the HttpContext.Current.Response.Flush method to flush, or render, the output to the client as each MailMessage is sent. Other than providing a confirmation for you, this also enables you to send the e-mail to a large list of subscribers without the Web request timing out. The Response.Flush method maintains the connection between the browser and the server as it flushes the output to the browser after each MailMessage is sent. If you did not do this, and your MailingList table had a few hundred records in it, the Web request may timeout before all of the MailMessage s are sent. Using Response.Flush will prevent the Web request from timing out.

Lets start putting this all together.

Building the Presentation Layer

The signup.aspx Web Form is a fairly simple form. It is two text boxes, one for the visitor's name, one for their e-mail address, and a submit button. The signup.aspx Web Form uses a code behind class file named signup.aspx.cs.

Building the Sign-Up Web Form

First create the Web Form by creating a file named signup.aspx, and add the following code:

<%@ Page Language="C#" INHERITS="Signup" %>
<html>
<body>
<form runat="server">
<table border="0" bordercolor="Tan"
CellPadding="4" CellSpacing="0"
Width="100%">
<tr bgcolor="Maroon">
<td valign="top" colspan="2"
style="font: bold 10pt Verdana; color:white;">
Join the Mailing List <asp:Label runat="server" id="ResultMsg" />
</td>
</tr>
<tr>
<td valign="top"
style="font: bold 9pt Verdana">
Name:
</td>
<td Width="100%" NOWRAP>
<asp:TextBox runat="server" id="Name"
Width="400"
Font-Size="9pt" Font-Name="Verdana"
BorderStyle="Solid" BorderWidth="1" />
<asp:RequiredFieldValidator runat="server"
ControlToValidate="Name"
ErrorMessage="*required"
Font-Size="9pt" Font-Name="Verdana" />
</td>
</tr>
<tr>
<td valign="top"
style="font: bold 9pt Verdana" NOWRAP>
E-Mail:
</td>
<td Width="100%">
<asp:TextBox runat="server" id="Email"
Width="400"
Font-Size="9pt" Font-Name="Verdana"
BorderStyle="Solid" BorderWidth="1" />
<asp:RequiredFieldValidator runat="server"
ControlToValidate="Email"
ErrorMessage="*required"
Font-Size="9pt" Font-Name="Verdana" />
<asp:RegularExpressionValidator
runat="server"
ControlToValidate="Email"
Display="Dynamic"
ValidationExpression="[\w\x2E\x2D]{2,}\x40{1}[\w\x2E\x2D]{2,}\x2E{1}[\w\x2E\x2D]{2,}"
ErrorMessage="Use the format name@server.com"
Font-Size="9pt" Font-Name="Verdana" />
</td>
</tr>
<tr>
<td></td>
<td Width="100%">
<asp:Button runat="server"
Text="Join Mailing List"
Width="400"
Font-Size="9pt" Font-Name="Verdana"
BorderStyle="Solid" BorderWidth="1"
BorderColor="Tan" BackColor="Maroon"
ForeColor="White" Font-Bold="True"
OnClick="Submit_Click" />
</td>
</tr>
</table>
</form>
</body>
</html>

In the signup.aspx Web Form you create a simple table and include the two text boxes, plus a RequiredFieldValidator for each text box. The RequiredFieldValidators are used to ensure that a value is submitted for each text box. You set the ControlToValidate property for each RequiredFieldValidator to the respective TextBox control's ID property. The Email control also has a RegularExpressionValidator associated to it. This uses a regular expression to validate the e-mail address and ensure that it is well formed (i.e. a valid format for an e-mail address).

The Button control's OnClick property indicates that when the button is clicked, the Submit_Click event handler should fire. This event handler is in the code behind class for the Web Form. The code for the signup.aspx.cs file is shown here:

using System;
using System.Web;
using
            System.Web.UI;
using System.Web.UI.HtmlControls;
using
            System.Web.UI.WebControls;

public class Signup :
            Page
{
protected TextBox Name;
protected TextBox
            Email;
protected Label ResultMsg;

protected void
            Submit_Click(object sender, EventArgs
            e)
{
if(Page.IsValid)
{
MailingListData mld = new
            MailingListData();
ResultMsg.Text = mld.SignUp(Name.Text.Trim(),
            Email.Text.Trim());
}
}
}

In the code behind class for the signup.aspx Web Form you map three variables to the server controls in the Web Form. You create one event handler method, Submit_Click. In the Submit_Click method you check to ensure the page is valid, and if it is, you call the MailingListData.SignUp method. When you call the method, you pass the Name.Text and Email.Text values in as arguments. Notice that the Trim() method of the String class is used on each. This is to remove any white space from the beginning and end of the string values in the text boxes. This ensures "clean" values (values with no extraneous white space) are entered into the database. The return value from the method call is set to the Text property of the ResultMsg Label control.

Now when a visitor enters their name and e-mail address and clicks on the button, their name and e-mail address are inserted into the MailingList table, and a message is rendered to the page, as seen here:

The ePony Express

Once you have entries in your MailingList table, you can send bulk mail such as a newsletter to the entire list. This is done with an administrative page named sendmail.aspx. The sendmail.aspx Web Form uses three TextBox controls (subject, from and message), a CheckBox control (HTML format) and a Button control (Send Mail). The mail is sent by creating a System.Web.Mail.MailMessage object, and then sending it to each entry in the MailingList table. The Web Form will look like the page seen here:

Like the signup.aspx Web Form, the sendmail.aspx Web Form uses a code behind class to separate the code and content. The code for the Web Form (sendmail.aspx) is shown here:

<%@ Page Language="C#" INHERITS="SendMail" SRC="SendMail.aspx.cs" %>
<html>
<body style="font: 9pt Verdana">
<form runat="server" id="myForm">
<table border="0" bordercolor="Tan"
CellPadding="4" CellSpacing="0"
Width="100%">
<tr bgcolor="Maroon">
<td valign="top" colspan="2"
style="font: bold 10pt Verdana; color:white;">
Send Mail to the Mailing List
</td>
</tr>
<tr>
<td valign="top"
style="font: bold 9pt Verdana" NOWRAP>
Send Mail As HTML:
</td>
<td Width="100%" style="font: 9pt Verdana">
<asp:CheckBox runat="server" id="Html" />
</td>
</tr>
<tr>
<td valign="top"
style="font: bold 9pt Verdana" NOWRAP>
Subject:
</td>
<td Width="100%" style="font: 9pt Verdana">
<asp:TextBox runat="server" id="Subject"
Width="400"
Font-Size="9pt" Font-Name="Verdana"
BorderStyle="Solid" BorderWidth="1" />
</td>
</tr>
<tr>
<td valign="top"
style="font: bold 9pt Verdana" NOWRAP>
From:
</td>
<td Width="100%" style="font: 9pt Verdana">
<asp:TextBox runat="server" id="From"
Width="400"
Font-Size="9pt" Font-Name="Verdana"
BorderStyle="Solid" BorderWidth="1" />
</td>
</tr>
<tr>
<td valign="top"
style="font: bold 9pt Verdana" NOWRAP>
Message:
</td>
<td Width="100%" style="font: 9pt Verdana">
Put <font color="#CC3300"><b>&lt;--Name--&gt;</b></font>
where you want the name of the recipient in the e-mail.<br>
<asp:TextBox runat="server" id="Message"
Width="400"
TextMode="Multiline" Rows="5"
Font-Size="9pt" Font-Name="Verdana"
BorderStyle="Solid" BorderWidth="1" />
</td>
</tr>
<tr>
<td></td>
<td Width="100%">
<asp:Button runat="server"
Text="Send Mail"
Width="400"
Font-Size="9pt" Font-Name="Verdana"
BorderStyle="Solid" BorderWidth="1"
BorderColor="Tan" BackColor="Maroon"
ForeColor="White" Font-Bold="True"
OnClick="Submit_Click" />
</td>
</tr>
</table>
</form>
</body>
</html>

In the Web Form you simple create a basic HTML table layout. You make three TextBox controls, one for the subject of the e-mail, one for the from-address, and one for the actual message. You also create a CheckBox control named Html. This CheckBox control will be used to allow you to specify that the e-mail should be sent as HTML format. If the CheckBox is not checked, the e-mail will be sent as plain text. The last control is a Button control with its OnClick property set to Submit_Click. Like the signup.aspx Web Form, this property specifies the name of the event handler to fire when the button is clicked.

In the SendMail code behind class you create the Submit_Click event handler. In this event handler you call the MailingListData.SendMail method to send the e-mail to each recipient. The code for the SendMail code behind class is shown here:

using System;
using System.Web;
using
            System.Web.UI;
using System.Web.UI.HtmlControls;
using
            System.Web.UI.WebControls;
using System.Data;
using
            System.Data.SqlClient;
using System.Web.Mail;

public class
            SendMail : Page
{
protected TextBox Message;
protected
            TextBox Subject;
protected TextBox From;
protected CheckBox
            Html;
protected HtmlForm myForm;

protected void
            ClearForm()
{
//Clear the form fields
Subject.Text =
            "";
From.Text = "";
Message.Text = "";
Html.Checked =
            false;
}

protected void Submit_Click(object sender,
            EventArgs e)
{
if(Page.IsValid)
{
MailingListData mld =
            new MailingListData();
mld.SendMail(myForm, From.Text.Trim(),
Subject.Text.Trim(), Message.Text.Trim(),
Html.Checked);
ClearForm();
}
}
}

In the sendmail.aspx.cs code behind class you create variables to map to the server controls on the Web Form, and the HtmlForm element. In the Submit_Click event handler method you check to ensure the page is valid, then call the MailingListData.SendMail method. You pass all of the server control values and a reference to the HtmlForm element into the SendMail method.

Running the Application

Now that all of the class files are written, your application is complete. The only thing left to do is compile the classes into a DLL and put it in the \bin directory of your application. If you are using Visual Studio.NET you can do this by selecting Build from the Build menu. If you are not using Visual Studio.NET then you can use a command line compiler.

Open a command prompt and navigate to the directory that your class files are in using the CD command. The CD command will look something like this:

C:> CD C:\Inetpub\wwwroot\ASPToday\MailingList

Once there, use the csc.exe compile by executing the following command:

csc.exe /target:library /out:ASPToday.dll *.cs /r:System.dll /r:System.Data.dll

This will compile all of the .cs files in the current directory into a DLL named ASPToday.dll. Move this DLL to the \bin directory of your application, and you are ready to go.

Navigate to the sigunp.aspx Web Form and enter your e-mail address. Then, navigate to the sendmail.aspx Web Form. By filling out the sendmail.aspx Web Form as seen here:

Each recipient will receive a text e-mail like this:

If you fill out the sendmail.aspx Web Form, indicating the e-mail should be sent using HTML format, and then put HTML tags in the message TextBox like this:

Each recipient will then receive an e-mail that looks like this:

Summary

In this article I showed you how to build a simple mailing list tool for your Web application. This tool uses a SQL Server table to store the names and e-mail addresses of subscribers to your mailing list. Visitors to your Web site can subscribe to your mailing list by filling out a form and submitting it. Their name and e-mail address are inserted into the database. When you want to send an e-mail you fill out a form, and using the System.Web.Mail namespace, create a MailMessage and send it to each recipient. This is a fairly simple mailing list tool, but has its limitations. For instance, if there are any more than a few hundred subscribers, sending all the mails will take a while and you will also have to leave your browser open while all the messages are sent. For larger mailing lists, this functionality can be converted to a Web Service, and an asynchronous "fire-and-forget" call can be made using a proxy client. The real purpose of this article is to show you the basic structure for building a simple mailing list tool. As your Web application grows, you may need to extend this functionality. This application can be extended to include a number of features, such as a preview Web Form, or a Web Form for sending test e-mails to yourself, etc.


Developing a Universal Data Access Layer leveraging ADO.NET, C# and Factory Design Pattern

Developing a Universal Data Access Layer leveraging ADO.NET, C# and Factory Design Pattern
http://www.csharptoday.com/content.asp?id=1626
Article Rating: 4
Published on August 13, 2001
Abstract
In this article Naveed Zaheer shows us how to build a thin universal data access layer at the top of ADO.NET using Design Patterns and .NET Framework. This component layer, which will be based on ADO.NET, will provide access to OLE DB Managed Providers as well as SQL Server Managed Providers through a single interface. It will be extensible so that in the future we can add access to other Managed providers when they become available.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

Ideally, every software application will access the data from all possible data sources in the same manner. Every application will be able to manipulate data in every possible way. But we do not live in a perfect world. There are limitations on how we can or cannot access certain types of data. In the past few years Microsoft's ActiveX Data Object (ADO) has revolutionized data access mechanism. With the arrival of .NET, comes ADO.NET, which truly is the next generation of the ADO data species. In this article, readers will learn how to build a thin universal data access layer at the top of ADO.NET using Design Patterns and .NET Framework. This component layer, which will be based on ADO.NET, will provide access to OLE DB Managed Providers as well as SQL Server Managed Providers through a single interface. It will be extensible so that in the future we can add access to other Managed providers when they become available.

This article does not require a prior knowledge of Design Patterns. However, familiarity with Object Oriented Design and Analysis techniques is recommended. A brief overview of Design Patterns is covered later in the article. The discussion on n-tier (Presentation Tier, Business Tier, Data Tier) design and architecture is beyond the scope of this article. It assumes that readers have some familiarity with n-tier design.

An Overview of Data Access Strategies

Before we dive into ADO.NET, let's go through a brief overview of history of data access on Microsoft platforms including Windows and DOS. It all started with the arrival of ODBC (Open Database Connectivity). ODBC provided a single API-style interface to access heterogeneous data sources. Components called ODBC drivers provided the abstraction layer between databases and ODBC interface. Theoretically an application could switch to any database without modifying a single line of data access code for which an ODBC driver was available. Because of the API-style interface the use of ODBC was much more rampant in C/C++ applications than in Visual Basic applications. Then came a slew of data access frameworks such as DAO (Data Access Objects) and RDO (Remote Data Objects). DAO basically came with Visual Basic to supply developers with a data access class library for Microsoft's own desktop database named "Access". On the other hand RDO just provided an object-wrapper for ODBC. Although each of these solutions was effective for accessing certain types of data source, none provided a comprehensive solution. The arrival of ADO (Active Data Objects) a few years ago provided a far from elegant, but better solution to common data access problems. ADO is a user-friendly object wrapper around Microsoft's data access technology OLE DB. This object model provides uniform access relational as well as non-relational data. With the many great features it provides, the best one is a single interface to access all type of data sources. Just like ODBC drivers for ODBC, you have to develop OLE DB providers as an abstraction layer between database and OLE DB. From ODBC to ADO one theme was consistent i.e. a single interface to access different types of data sources. Although it was useful to switch between data sources without making many changes to the data access layer, in many cases it took away a chunk from performance. The availability of OLE DB provider for SQL Server alleviated the problem a little bit, but still performance was the issue. Because now it was possible to access SQL Server data by using this native OLE DB provider instead of OLE Db Provider for ODBC.

Next Generation of Data Access - ADO.NET

With the arrival of .NET came ADO.NET. ADO.NET is not here to replace classic ADO, but to offer a different approach to data access. Classic ADO offered disconnected recordsets but still most of the data access was in connected mode. What ADO.NET offers is a disconnected approach. Also, to boost performance it offers a separate, Managed Provider for different types of data sources. The idea is to take advantage of native database capabilities, instead of building a single interface across all backends based upon least common denominator. At the time of writing only three managed providers i.e. MS SQL Server, OLE DB and ODBC are available. Manager Provider for SQL Server goes directly through TDS (Tabular Data Stream), which is SQL Server's native data interface. It is only supported for SQL Server 7.0 or above. This way if you are accessing data from MS SQL Server, you get a big performance boost compared to OLE DB provider. Now we have separate classes to deal with when accessing data for OLE DB and MS SQL Server data sources. Suppose that you are using OLE DB data provider to access data from an Oracle database for your .NET application. In the meantime, an ADO.NET Managed Provider for Oracle native data interface becomes available from Microsoft or a 3rd party. Now to use the new Manager Provider, you may have to alter all your data access code.

Many businesses are running a mixture of different databases. Imagine a company's e-Commerce site for example. It may have a variety of databases that it uses for different purposes. For example, DB2 database, where its entire inventory has stored; Oracle database, where it's entire customer data is stored; and MS SQL Server database where it is planning to migrate in the future. What if we can have a single interface to access all these Managed Providers without sacrificing performance? This interface should be neither too general nor too specific. Whenever possible, this interface should try to expose the native Provider functionality instead of providing a wrapper for it. It should cover the entire common functionality supported by all the Manager Providers. The abstraction layer it provides should not be so thick that it degrades performance.

There are different ways to solve this problem. One way is to have a single class where each method has a switch-case statement to handle different data providers. Sample code for this solution is shown below:

public override void Execute(ProviderType ctProviderType,CommandType ctStatement, string strSQL)
{
switch(ctProviderType)
{
case ProviderType.MSSqlClient:
.
//Do Stuff Here for SqlClient
.
objSqlCommand.ExecuteNonQuery()
break;

case ProviderType.MSOleDb:
.
//Do Stuff Here for OleDb
.
objOleDbCommand.ExecuteNonQuery()
break;

default:
//Do Nothing
break;
}

}

However, this is not an elegant solution, because it will make code unnecessarily complex. Each time a new Managed Provider becomes available you will have to modify that class. After you make a change for one Managed Provider, you will have to regression test the component against all Managed Providers just to make sure you did not break anything.

Overview of Design Patterns

Let's see how Design Patterns can help us solving this problem. However, before we do that let's get a brief look at what Design Patterns are. Design Patterns are the robust solutions of frequently occurring problems, which evolve, because of object-oriented development, over a period of decades. These are not new techniques, which were never used before. You may have been using Design Patterns before without any knowledge of doing so. For example if you have programmed with Visual C++ MFC you must have used Document/View Architecture, which itself is based on a Design Pattern. If you have programmed with ATL (Active Template Library) or STL (Standard Template Library) then you must have used templates. A template is a type of Behavioral Design Pattern. In COM, design of interface IClassFactory is based on Factory Design Pattern. Singleton is a common Design Patten, which is used very often. For a detailed discussion about Design Patterns, please refer to the book mentioned in the Helpful Links section later in this article. Design Patterns are divided into three categories:

  • 1) Creational: Design Patterns in this category deal with object instantiation. Factory and Singleton belong to this category.
  • 2) Structural: Design Patterns in this category deal with object and class composition. Adapter and Facade belong to this category.
  • 3) Behavioral: Design Patterns in this category deal with algorithms and assigning object responsibilities. Mediator and Command belong to this category.

Factory Design Pattern

Sometimes an application or a framework does not know at runtime what kind of objects it has to create at runtime. It may only know about the abstract class for those objects. However, these abstract classes or interfaces cannot be instantiated. In other words, application only knows when to create the object but it does not know the type of the object to create. Factory Design Pattern does solve this problem. Factory Design Pattern Method is a Creational Design Pattern. The purpose of Factory method is to create objects. It helps us design an interface that creates objects of appropriate subclasses at runtime. It provides loose coupling eliminating the need to tie application specific classes together.

Factory Design Pattern is defined as "Define an interface for creating an object, but let the subclasses decide which class to instantiate." The Factory method lets a class defer instantiation to subclasses. The Factory Method lets a class defer instantiation to subclasses in the book Design Patterns.

In simple words, Factory Method lets you decide at runtime which subclasses of an abstract base class will be instantiated.

There are different implementations of Factory Method. One particular implementation that we are interested in is "Parameterized Factory method". In this implementation Factory Method is used to create multiple kinds of objects. It accepts a parameter that identifies which type of object to create. All the objects created implement the same interface or subclasses from the same abstract base class. Consider the following example:

public class Creator
{
public static int gintComicBook = 0;
public static int gintProgrammingBook = 1;
public static int gintFictionBook = 2;
public Creator()
{
}
public Book CreateBook(int iBookType)
{
switch(iBookType)
{
case gintComicBook:
return new ComicBook();

case gintProgrammingBook:
return new ProgrammingBook();

case gintFictionBook:
return new FictionBook();

};
}
}

In the code above function, CreateBook of class Creator takes an integer as a parameter. Based on that parameter it decides what type of object it is going to create. All the three classes i.e. ComicBook, ProgrammingBook, FictionBook, derive from the same base class Book.

Data Access Layer Implementation

Let's see that how we can use the Parameterized Factory Method implementation to create a single interface for all of the Managed Providers using Factory Method. First, we will have to have a Creator class with a function like CreateBook. This function will take a parameter identifying what type of Managed Provider object it has to create. For our implementation, class DataManager is our Creator class. Its function GetDBAccess, which has two overloads, behaves the same way as CreateBook behaves in the above example. Following is the partial code for DataManager class:

public static DBAccess GetDBAccessor(ProviderType ctProviderType,string strConnectionInfo)
{
DBAccess objDBAccess;
switch(ctProviderType)
{
case ProviderType.MSSqlClient:
objDBAccess = new DBAccessSQL(strConnectionInfo);
break;

case ProviderType.MSOleDb:
objDBAccess = new DBAccessOleDb(strConnectionInfo);
break;

default:
objDBAccess = null;
break;
}
return objDBAccess;
}
public static DBAccess GetDBAccessor(ProviderType ctProviderType)
{
return GetDBAccessor(ctProviderType,"");
}

As you can see from the code shown above, both overloads of function GetDBAccess take a parameter of type enumeration ProviderType. Based on the value of that parameter, it is decided from that object which class will be instantiated. Both classes i.e. DBAccessOleDb, DBAccessSQL, derive from the same base class DBAccess. DBAccess is the abstract base class that defines an interface common to all Managed Providers. Following is the partial code shown for the base class DBAccess:

public abstract class DBAccess : IDisposable
{
protected string mstrConnectionInfo;
//public properties
public abstract string ConnectionInfo
{
set;
get;
}
public abstract bool InTransaction
{
get;
}
//public functions
public abstract void BeginTransaction();
public abstract void CommitTransaction();
public abstract void AbortTransaction();
public abstract void Execute(CommandType ctStatement,string strSQL);
public abstract void Execute(CommandType ctStatement,string
strSQL,string strTable,out DataSet objDataSet);
public abstract void Execute(CommandType ctStatement,string
strSQL,out IDataReader objDataReader);
public abstract void SetParameter(string strName, ParameterDirection
pdDirection, ParameterType ptType, int intSize, object objValue);
public abstract void SetParameter(string strName, ParameterDirection
pdDirection, ParameterType ptType, int intSize);
public abstract void SetParameter(string strName, ParameterDirection
pdDirection, ParameterType ptType);
public abstract void ClearParameters();
public abstract void GetParameterValue(string strName, out object
objValue);
public abstract void GetParameterValue(string strName, out string
strValue);
public abstract void GetParameterValue(string strName, out short
shtValue);
public abstract void GetParameterValue(string strName, out int
intValue);
public abstract void GetParameterValue(string strName, out Double
dblValue);
public abstract void GetParameterValue(string strName, out Decimal
decValue);
public abstract void GetParameterValue(string strName, out long
lngValue);
public abstract void GetParameterValue(string strName, out DateTime
dateValue);
public virtual void Dispose()
{
}
}

One interesting thing about this class is that it implements IDisposable interface. This interface only has one method Dispose, for which implementation has to be provided by the class, which implements that interface. As we know, in .NET Framework, Garbage Collection handles the object cleanup. However, it is recommended that if you are dealing with precious system resources, such as database connections, in your class then you should expose a Dispose method. In that method you should properly cleanup those resources. Applications using the objects of your class should call this method to properly and timely free the system resources.

As we know that classes derived from DBAccess encapsulates important system resources such as database connection. It is possible in some cases that an object of type DBAccess is de-referenced but Garbage Collector may take some time to collect. In that case, the database connection that is encapsulated by the subclass of DBAccess will not be freed until the Garbage collection takes place. This is not a desirable behavior and may consume resources on the database sever. That's where use of Dispose method comes into the picture. We can put all the database connection cleanup code in the Dispose method. An application that is using an object of class DBAccess will call Dispose to free up any database resources. Note that classes such as DBAccessSQL, which is derived from DBAccess implements IDisposable inherently because it is implemented in its base class.

As you can see none of the functionality in the abstract base class DBAccess is provider specific. All the properties and functions of base class DBAccess, which are declared abstract, will require implementation in the derived classes. All the provider specific functionality is implemented in the derived classes. It will be a lot clearer when we start looking at the different subclasses of DBAccess class. Let's look at the code shown below for DBAccessSQL:

internal class DBAccessSQL :
            DBAccess
{
private SqlConnection mobjConnection =
            null;
private SqlCommand mobjCommand = null;
private
            SqlDataAdapter mobjDataAdapter = null;
private SqlTransaction
            mobjTransaction = null;
internal DBAccessSQL(string
            strConnectionInfo)
{
mstrConnectionInfo =
            strConnectionInfo;
mobjCommand = new
            SqlCommand();
mobjDataAdapter = new
            SqlDataAdapter();
}
internal
            DBAccessSQL()
{
mstrConnectionInfo = "";
mobjCommand = new
            SqlCommand();
mobjDataAdapter = new
            SqlDataAdapter();
}

Class DBAccessSQL is derived from DBAccess. Access modified for class DBAccess is "public" while access modifier for DBAccessSQL is "internal". Access modifier "internal" makes sure that this class is not visible outside its own namespace. Therefore, when we build DataManager component as a DLL, consumers will not be able to create the instances of class DBAccessSQL directly. They will only be able to create them through the creator class DataManager. That is exactly what we want. In addition, DBAccessSQL declares objects of type SqlConnection, SqlCommand and SqlDataAdapter as data member. Data members of the class mobjCommand and mobjDataAdapters are initialized in the object's constructor. You can set value of data member mstConnectionInfo in different ways. One way is that when you call GetDBAccess method of class DataManager, you can pass connection string as one of the parameters. Another way to set this information is by calling property ConnectioInfo of class DBAccessSQL. Whatever way you set this property, you have to do it before calling functions Execute or BeginTransaction. Otherwise, an SQLException will be thrown by the Managed Provider.

One benefit that we get from building a wrapper around the ADO.NET data access functionality is that we are able to hide complex implementation details from the consumer of the component. For example, you know that to get a DataSet build based on the results returned from a query, we have to use the Command object of DataAdapter class. For a query that returns no results, we can use Command object directly. As you can see from the code listed above, function Execute has four overloads. Each of them returns a different type of object as out parameters. These overloads of function Execute hide the complex details on how different data access objects such as DataReader or DataSet was created.

public override void SetParameter(string strName, ParameterDirection pdDirection,
ParameterType ptType)
{
try
{
SqlParameter objParameter = mobjCommand.Parameters.Add(strName,
GetSqlParameterType(ptType));
objParameter.Direction = pdDirection;
}

//Exception Handling Code Here
}

Code for one of the overloads of function SetParameter is shown above. Overloads of the function SetParameter are used to add parameter to the Parameters collection of Command object for the query or the stored procedure to be executed. As you can see, function SetParameter does not take any of the Provider dependent parameter types as parameter. It takes enumeration ParameterType declared in file DataManager.cs as a parameter for this function. This enumeration abstracts out the differences between the parameters types used by different Managed Providers. It is the responsibility of the derived class of DBAccess to convert it to Provider-Specific parameter type. On thing to note here is that we always add a parameter to the Parameters collection of mobjCommand data member. So, what happens when the overload of function Execute is called that uses data member mobjDataAdapter ? In that case function BindParameters is called by the Execute function, which copies parameters from data member mobjCommand's Parameters collection to the Parameters collection of mobjDataAdapter's SelectCommand property. Listing for function BindParameter is shown below:

private void BindParameters (SqlCommand
            objCommand)
{
if (mobjCommand.Parameters.Count >
            0)
{
foreach(SqlParameter objParameter in
            mobjCommand.Parameters)
{
SqlParameter objNewParameter =
objCommand.Parameters.Add(objParameter.ParameterName,objParameter.SqlDbType,objParameter.Size);
objNewParameter.Value
            = objParameter.Value;
objNewParameter.Direction =
            objParameter.Direction;
}
}
}

Now let's look at the code for some of the more interesting functions of class DBAccessSQL. Following is the code for one of the overloads of function Execute:

public override void Execute(CommandType
            ctStatement, string
            strSQL)
{
try
{
CreateConnection();
SetCommandData(mobjCommand,ctStatement,strSQL);
mobjCommand.ExecuteNonQuery();
}

//Exception
            Handling Code Here
}

This function is pretty simple. Parameters to this function are enumeration CommandType, which may be a stored procedure, name of a table or an SQL query and the statement text itself. This function first calls CreateConnection. Function CreateConnection is responsible for creating and opening the database connection if one does not exists. It does so by checking data member mobjConnection. If data member mobjConnection is not null and its state is Open, this function does not do anything. Otherwise, it creates and opens a new SQLConnection object and assigns it to data member mobjConnection. As said before, it is recommended that as soon as you are done with an instance of class DBAccessSQL, you should call its Dispose method. Dispose method takes care of closing the database connection. The reason why it is done this way is to follow the principle of getting a system resource as late as possible and releasing it as early as possible once it is no longer required. This way no one holds precious system resources for too long and their availability is maximized. A complete example on the sequence of how different methods of this class should be called will be explained in a later section.

Finally, function GetParameterValue that has eight overloads is used to extract the value of parameters returned after executing a stored procedure. Each overload returns a parameter of a specific type as out parameter. Function ClearParameters just clears out all the parameters that are set by calling function SetParameter. If there is no parameter set by calling function SetParameter, this function does not do anything. As mentioned earlier, overloads for function GetParameterValue are used to extract the value of the parameters returned by the stored procedure. As you can see from the code shown below for one of the overloads of this function, it simply gets the parameter value from mobjCommand's Parameters collection.

public override void GetParameterValue(string strName, out object objValue)
{
try
{
objValue = mobjCommand.Parameters[strName].Value;
}

//Exception Handling Code Here
}

Testing

In the code download there is an application called DataManagerTest that we will use to test this component. We will use the pubs database that comes with MS SQL Server as our test database. As you can see from the screenshot below, that each buttons is to test a specific functionality.

First let me show you how we will use this component. The partial code for one of the button onclick event is shown below:

DBAccess objABAccess =
            DataManager.GetDBAccessor(ptType,mstrConnectionInfo);
objABAccess.SetParameter("@JobId",ParameterDirection.Input,ParameterType.dmPtSmallInt,2,14);
objABAccess.SetParameter("@JobDesc",ParameterDirection.Output,ParameterType.dmPtVarChar,50);
objABAccess.Execute(CommandType.StoredProcedure,strSQL);
objABAccess.GetParameterValue("@JobDesc",out
            strJobDesc);
objABAccess.Dispose();

As you can see, we first get a reference of a new objDBAccess component by calling GetDBAccessor method of DataManager with proper ProviderType enumeration value. Then parameters are assigned to that component by calling function SetParameter. The next step is to call Execute method with proper CommandType and SQL text. If the command type is stored procedure and the stored procedure has any output parameters, you can use overloads of function GetParameterValue to extract their values. Finally, once you are done with the DBAccess object then it is highly recommended that you call its Dispose method to cleanup system resources in a timely manner. If you want to reuse the same DBAccess object then do not call its Dispose method. However, make sure to call its function ClearParameters to safely remove all the parameters that were assigned to this object before. You should call ClearParameters before you do anything else with the DBAccess object. Partial code for this DBAccess reuse is shown below:

DBAccess objABAccess = DataManager.GetDBAccessor(ptType,mstrConnectionInfo);
//Code for SetParameters
//Code to Call Execute
//Code to Get Parametrs or Deal with Data Set
//Make Sure you call Clear Paramaetrs before you do anything
objABAccess.ClearParameters()
//Code to Reuse the same object
//Code to SetParameters
//Code to Call Execute
//Code to Get Parametrs or Deal with Data Set
//Once you are done then call Dispose
objABAccess.Dispose();

Tools and Technologies Used

We will be using the following tools and technologies to build this component.

  • . NET Framework Beta 2
  • Visual Studio .NET Beta 2
  • Windows 2000 Server or Professional
  • Internet Information Server 5.0
  • C#
  • SQL Sever 7.0 or 2000

Conclusion

This completes our discussion regarding this article. DBAccessOleDb is the other class that is derived from DBAccess. It works pretty much the same was as DBAccessSQL. There is some functionality, which is only available in Managed Provider for SQL Server. To support that kind of functionality you have two options: One, that you can somehow implement that functionality yourself, the other being to throw a proper exception. Some other features that we may want to implement later are bulk updates using DataSet and DataAdapter.

One other thing to note here is how straightforward Object Oriented development is with .NET Framework. With classic COM, only interface inheritance was possible. So to develop an application based on complex Object Oriented designs you had to improvise and use roundabout techniques. However, with .NET Framework both interface inheritance and implementation inheritance are available. This way any type of Object Oriented implementation is possible using the .NET Framework. You are not forced to convert your OO classes to interfaces to make things work, as you had to do in the case of classic COM. In addition, you can take advantage of different OO techniques without any problem.


Getting Started With Unit Testing

Getting Started With Unit Testing
http://www.csharptoday.com/content.asp?id=1624
Article Rating: 4.5
Published on August 8, 2001
Abstract
In this article, Gerry Shaw walks us through the process required to run a suite of tests on our projects during each build. He uses an open source project as its foundation called NUnit freely available at http://nunit.sourceforge.net
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

Unit testing is the type of testing performed by the developers who write the code. It tests methods, properties, classes and assemblies. It is not the functional testing normally performed by an independent quality assurance department. A lot of people talk about unit testing, but by looking at the source code available in many open source projects the practice is rarely done. This article will give you some practical methods to start writing unit tests quickly and easily.

Where unit testing fits into the development cycle.

A Brief Overview of NUnit

In this article I will be using a testing framework called NUnit to demonstrate unit testing on the .NET platform. The reason for choosing NUnit is that it is a derivative of the popular testing framework used by eXtreme Programming (see links at the end of the article for more information about XP). This framework is available for most languages so that time spent learning the framework can be carried over to other environments. Specifically NUnit is a port of JUnit for .NET by Philip Craig. The current version is 1.10 and has been tested with the .NET SDK Beta 2. The software is available for free and comes with the full source code and documentation.

NUnit works by providing a class framework and test runner application. The class framework allows you to write test cases that inherit from TestCase. You package your TestCase classes into an assembly that is referred to as a TestSuite. The test runner uses your Test Suite assembly and runs all the tests displaying a message for each test case that fails.

Typical Test Case Dependencies

In the following sections we will cover the details of using NUnit for unit testing.

Installing the Software

Before we begin you first need to download NUnit. The package comes with an installer that will copy everything over for you. In the screen shots you will see that I have installed NUnit into c:\dev\NUnit. It is useful to install it into a shorter path than the default Program Files folder because you will need to add the bin subfolder to your path if you want to use the console based tools (highly recommended).

To verify that everything is working:

  • Open a command prompt
  • Change the current directory to the NUnit\bin folder
  • Enter the following command: NUnitConsole NUnit.Tests.AllTests,NUnitTests.dll

You should see something like this:

Successful test run of NUnit using the Console Test Runner.

What you have just done is run the tests that come with NUnit on itself.

There is also a GUI front end that you can use but in order to integrate running the tests with your build you will need to be familiar with the console program. The only tricky thing to remember is the Assembly naming convention that .NET uses. The format is <namespace.classname>,<assembly filename>

To try out the GUI:

  • Double click on the NUnitGUI.exe program in the bin folder.
  • Click the Browse button and select the NUnitTests.dll file.
  • Click the Run button.

You should see something like this:

Successful unit test of NUnit using the GUI Test Runner.

Hello World Test

Now that you have successfully installed and configured the software we will write a very simple class and a test case to test it. The code that we will test will be a trivial library that will return the string "Hello, World!" The test that we will write will ensure that only that string returns. The example may seem trivial but it does perform all the basic requirements for a test framework. There is code that needs to be tested, a test case and an automatic way of building and running the tests with a single command.

First we write the test case to test what we expect the code to do. At first this might seem a bit extreme to have to write 50+ lines of code to test Hello World but as you will see the code is very straightforward.

// HelloWorldTest.cs

Use the same namespace as the class we are testing with a .Test suffix. This isn't a requirement but it does help identify tests quickly.

namespace CSharpToday.UnitTesting.Tests
            {

using System;

Bring in the NUnit framework and the code we are testing.

using NUnit.Framework;
using CSharpToday.UnitTesting;

Each test case must inherit from TestCase and have a name ending with Test in order to have NUnit find it.

public class HelloWorldTest : TestCase {

Our entire test suite will consist of this single test case. Refer to the next example for a more complicated test suite.

public static ITest Suite {
get { return new TestSuite(typeof(HelloWorldTest)); }
}

This is the required constructor for all test cases.

public HelloWorldTest(String name) : base(name) {
}

When you have multiple tests using the same set of objects you can add them to what is called a Fixture. Before each test SetUp is called. After each test TearDown is called.

This may seem like a lot of work but it prevents any side effects and for all but the most complicated tests is fast enough not to notice. If time is starting to be a factor you can create the fixture in the constructor but beware of side effects caused by other test cases.

        HelloWorld
            _hello;

protected override void SetUp() {
_hello = new
            HelloWorld();
}

Now we proceed to test every public and protected member in our class. For most test cases you will just use the various Assert methods that the TestCase class provides. Refer to the Assertion class in the NUnit source code for a list of all the different Assert variations.

Each test must be named Testxxx, return void, and not take any arguments. I use the convention Test_MemberName_TestDetails.

public void Test_Constructor() {
// make sure the object is constructed properly
AssertEquals("Hello, World!", _hello.Message);
}

When possible you should write your tests so that test data can be easily added without copying and pasting. In some cases you may want to store your test data in an external file and read it in the constructor rather than the SetUp method to avoid reading the same values for each test.

public void Test_Message() {
// any non-null string is allowed in message
string[] testdata = new string[] {
String.Empty,
" ",
" Some Padding ",
"G'Day!",
"Hello, World!"
};

foreach (string value in testdata) {
_hello.Message = value;
AssertEquals(value, _hello.Message);
}
}

When testing exceptions execute the code that will cause the exception and explicitly catch the exception you are testing. If the exception is not thrown the Fail will cause an error. If a different exception is thrown NUnit will catch it and report it as an error.

        public
            void Test_Message_ArgumentNullException() {
try
            {
_hello.Message = null;
Fail("ArgumentNullException should
            have been thrown");
} catch (ArgumentNullException)
            {
}
}
}
}

Now we write the code to pass the test case.

// HelloWorld.cs
namespace
            CSharpToday.UnitTesting {

using System;

public class
            HelloWorld {

string _message = "Hello, World!";

public
            string Message {
get { return _message; }
set {
if (value
            == null) {
throw new ArgumentNullException();
}
_message =
            value;
}
}
}
}

In order to fully integrate NUnit into the development cycle you want to run your tests after each build. The following diagram shows the build process.

Here is the makefile to build the component, unit test, and run the test. You will have to change the NUNIT macro to point to the bin folder where NUnit is installed.

# makefile for HelloWorld
CSCFLAGS=/nologo /debug
NUNIT=\dev\NUnit\bin

main: build test

clean:
del HelloWorld.dll
del HelloWorld.pdb
del HelloWorld.Tests.dll
del HelloWorld.Tests.pdb

build: HelloWorld.dll HelloWorld.Tests.dll

test: build
$(NUNIT)\NUnitConsole CSharpToday.UnitTesting.Tests.HelloWorldTest,HelloWorld.Tests.dll

HelloWorld.dll: HelloWorld.cs
csc $(CSCFLAGS) /out:HelloWorld.dll /t:library HelloWorld.cs

HelloWorld.Tests.dll: HelloWorld.dll HelloWorldTest.cs
csc $(CSCFLAGS) /out:HelloWorld.Tests.dll /t:library /r:$(NUNIT)/NUnitCore.dll;HelloWorld.dll HelloWorldTest.cs

The following files are available in the source code download. Using a command prompt you can build the project by typing makefile.

The makefile will first build two libraries, HelloWorld.dll which contains our actual code and HelloWorld.Test.dll which contains the code to test HelloWorld.dll. The makefile then runs the NUnitConsole program on our test case. You can see in the output that 3 tests were run and everything was ok.

Now lets break the code to see what happens. Edit the HelloWorld.cs file and change the default message to "Howdy Y'All! "

namespace CSharpToday.UnitTesting
            {

using System;

public class HelloWorld
            {

string _message = "Howdy Y'All!";

public string
            Message {
get { return _message; }
set {
if (value == null)
            {
throw new ArgumentNullException();
}
_message =
            value;
}
}
}
}

Now when we build the project the code compiles but the test case now fails.

Successful build but failed test case

In most cases you will be able to go to the failed code and correct the problem as in this case. In some cases you will find that the test case needs changing. When this happens make sure the test case is wrong before changing it. It could very well be that the test case is showing a subtle bug.

A Real World Example

The Hello World example will give you a basic introduction to using NUnit but a more complicated example is needed to show you a variety of different testing patterns. The WordCounter example in the .NET SDK will be used as the code to test against. Included in the support materials for this article is a folder called WordCount. This folder contains the program source code with minimal changes to WordCounter, a complete unit test called WordCounter.Tests and a makefile that performs a one step build and test.

The code in WordCounter.Tests is commented and you are encouraged to look through it to see ways of testing a variety of classes. The most important file to look at is the AllTests.cs file as this explains how to create a test suite with more than one test. The other test cases show examples of testing methods that require files as input, checking for thrown exceptions, and a good pattern for making test data easy to add without resorting to massive copy and pasting.

You now have the foundation you need to get started using NUnit for unit testing. The rest of this article will provide some insight into unit testing and some help in writing more complicated tests.

Test First Programming

You might have noticed in the Hello World Test example that we wrote the test case before I wrote the code. This is called Test First Programming. Test first programming is a method of forcing us to understand exactly what the code should and should not do before it is written. By writing tests first you are forced to have a very clear understanding of exactly what is needed by the code to pass the test

The basic process is to:

  • Start with a working code base that is passing 100% of all tests.
  • Write the tests that will test the new functionality that you want to add. If you try to build now you will get compile and link errors. This is correct.
  • Add the new functionality into the main code.
  • Build the code and run the test. If all goes well your build will compile and all the tests will pass. When this happens, and it often does, you will get a feeling that something is not right. This is because you would now normally start testing the code but since you have already done that you can start on the next change.

Test first programming works best when we wish to add functionality to an existing code base. If we are starting a project from scratch or building a prototype, test first programming can really slow us down by writing a lot of test cases that get thrown away or drastically changed. If you do have a clear idea of what the design of the code will be then write the tests, otherwise wait until you know more. Once the design solidifies go back and add all the tests before you add too much functionality otherwise you will never catch up. This is real problem because writing a good unit test does take time.

Best Practices for Writing Tests

Here are some guidelines that I use that you might want to follow as you develop your unit tests. Of course any unit testing is better than none but if you are going to spend time writing tests you might as well write them in a manner that will make then easily extendable and as error free as possible.

Do use a separate test class for each class that you test. Name the class the same but with a Test suffix. The test class should be in a different file so that you can compile a separate standalone library for all your test cases.

// Customer.cs
public class Customer
            {
// implementation
}

// CustomerTest.cs
public class
            CustomerTest : TestCase {
// implementation
}

Do use a separate method for each method you are testing when possible. In many cases you will need more than one test method for each method you are testing. In some cases this will not be practical as you can see in the RealWorld example. I use the convention Test_MemberName_TestDetail. For very simple test cases drop TestDetail and use the Test_MemberName format. For example when testing an Email property of a Customer class I might have the following methods in the test case.

public class CustomerTest : TestCase
            {
public void Test_Email_ValidData {
// test case
            implementation
}

public void Test_Email_InvalidData
            {
// test case implementation
}

public void
            Test_Email_ArgumentNullException {
// test case
            implementation
}
}

Do test with valid data and invalid data.

Do test all assertions. Refer to the HelloWorldTest class for an example of how to do this.

Do concentrate your first tests on the functionality that can be easily tested. Properties and methods that perform calculations are ideal. Simple get, set properties are not normally tested unless they perform error checking. This helps get you a substantial test framework up and running with a minimal time investment. Just keep in mind that the simple things are not likely to be the cause of the problems.

Do not test screen output. The amount of work that is required to test this is usually not worth it. A simple off by one pixel can cause all your test cases to fail. Consider using a test that simply calls the function to ensure that nothing crashes.

Do get positive feedback into your test cycle. Adopt a practice that before a bug fix or new feature can be added you must first add a test case to test the new code. The test case should pass once the bug fix or feature is added.

Do use arrays of data when testing a wide range of input values. In the following example a private Input struct is created to hold the input argument and the expected value. A loop is used to iterate over each input. In the future if you find a certain input cause an error you can now quickly add it to the test case with a single line.

public class BooleanOptionTest : TestCase
            {
private struct Input {

public string argument;
public
            bool expectedValue;

public Input(string argument, bool
            expectedValue) {
this.argument = argument;
this.expectedValue
            = expectedValue;
}
}

public void Test_Parse_ValidData()
            {
ArrayList inputs = new ArrayList();
inputs.Add(new
            Input("/option", true));
inputs.Add(new Input("/option",
            true));
inputs.Add(new Input("/option+",
            true));
inputs.Add(new Input("/option-",
            false));
inputs.Add(new Input(" /option ",
            true));
inputs.Add(new Input(" /option+ ",
            true));
inputs.Add(new Input("\t/option- \t ",
            false));

foreach (Input input in inputs) {
BooleanOption
            option = new BooleanOption("option",
            false);
option.Parse(input.argument);
AssertEquals(input.expectedValue,
            option.ToBoolean());
}
}
}

Conclusion

Unit testing is a powerful tool in developing solid code that will distinguish you among professional developers. By writing unit tests you can feel safe when making wide scale changes to the internal implementation because everything will tested. You will be able quickly ship bug fixes not having to worry if your fix breaks some other part of the project. Using the free open source test framework NUnit your investment time in learning will be well made. NUnit is based on JUnit, the unit testing defacto standard in the Java world and is available for virtually all programming languages.


Building a Database-Driven Guestbook with C# and ASP.NET

Building a Database-Driven Guestbook with C# and ASP.NET
http://www.csharptoday.com/content.asp?id=1546
Article Rating: 3.5
Published on August 7, 2001
Abstract
In this article, Doug Seven focuses on building a guestbook for a Web site. In the next part he will concentrate on building a mailing list as well. Both of these tools will be implemented using some common principals – a functional user interface, database access, and efficient business logic. This article is bundled with Doug's Mailing List article as a £12 e-doc available from http://www.amazon.com/exec/obidos/ASIN/B00005TZ91/ref=nosim/wrox-20.
Article

This is the first of two articles about building useful visitor features for your Web application. This article will focus on building a guestbook for your Web site, and the next article will focus on building a mailing list. Both of these tools can be implemented using some common principals - functional user interface, database access, and efficient business logic.

This article will teach you how, using ASP.NET and C#, you can build a guestbook that displays entries in chronological order in a paging-enabled DataGrid. The guestbook entries will be inserted into a SQL Server database. The DataGrid will enable custom paging, allowing you to retrieve only the number of entries you need to display from the database, making the most efficient use of memory on the server. When you're done, your guestbook will look something like this:

For the code in this article you will need:

  • .NET Framework Beta 2
  • Windows 2000 with IIS 5.0
  • Internet Explorer 5.5
  • SQL Server7.0 or 2000
  • A text editor, such as Notepad, TextPad, or Visual Studio.NET Beta 2

Building the Data Structure

The guestbook will provide a user interface (a form) where visitors to your Web site can enter their message and click a button to submit it. When the button is clicked, the data will be inserted into an SQL Server table using a stored procedure. Each time the page is rendered, including after a post, the data will be retrieved from the table using another stored procedure, and rendered in a DataGrid. All of the data access is abstracted to a data layer class, named GuestbookData. The separation of the application layers is illustrated here:

To begin, you will need to build the table in your SQL Server database. For simplicities sake, name the table "guestbook." The table should have six columns: id, title, visitorName, email, entryDate, and message. The id column is the Primary Key, and should be set up as an Identity column with a Seed value of 1 and an Increment value of 1. In the SQL Server Enterprise Manager, the table will look like this:

Building the Data Access Layer

Since we have started with the backend of the application, the database, we will continue working our way outward. The data access layer (DAL) class, GuestbookData, will expose the two methods we need to make the guestbook work, getGuestbookData and addGuestbookData.

Using a DAL class we abstract the exchange between our application and the database away from the presentation layer, making it more object oriented and the code more reusable. This is good programming practice. The DAL class will be compiled into our application's DLL and be available to any other class, including a Page class (such as a Web Form), at any time.

Configuration Settings

The SqlConnection s in the GuestbookData class are constructed using ConfigurationSettings.AppSettings["conString"] in place of their connection string value. This retrieves a key/value pair from the <appSettings> element in your Web.config file. The <appSetting> element is an optional element that you can add to the configuration file that allows you to define application-wide key/value pairs, such as database connection strings, in a centralized location. Since you don't have the conString key/value pair in the configuration file yet, you will have to add it. Add the following element to your configuration file, inside the <configuration> element.

<appSettings>
<add key="conString"
value="server=localhost;database=guestbook;uid=sa;pwd=;" />
</appSettings>

The Stored Procedures

You need to create the AddMessageToGuestbook stored procedure in your database. To do this, add the following code to your database:

CREATE PROCEDURE [AddMessageToGuesbook]
@title ntext,
@visitorName nvarchar (100),
@email nvarchar (255),
@message ntext
AS
INSERT INTO
guestbook
(
title,
visitorName,
email,
EntryDate,
message
)
VALUES
(
@title,
@visitorName,
@email,
GETDATE(),
@message
)
GO

This stored procedure enters all of the supplied fields into the guestbook table. The only fields not supplied are the ID field, which is an identity or auto increment field (and not used in this stored procedure), and the EntryDate field, which is supplied using the SQL GETDATE() function.

To display the guestbook entries you will use a DataGrid and bind the records from the guestbook table to it. Now, if everything goes well you will have thousands of entries in your guestbook. The last thing you want is to retrieve all of those records every time the page is rendered. Imagine if one hundred people viewed the page all at the same time. If you have 1,000 records in the table, and you retrieve them all for each request, that is 100,000 records you are putting into memory. Now image 1,000 people viewing the guestbook at once, and 10,000 records in the table. To eliminate this memory drain we will use custom paging in the DataGrid, and only retrieve exactly the records you need to display.

To enable the custom-paging scenario you need to create a stored procedure that retrieves only one page of data at a time, as well as a value indicating how many records there are. The page is defined by the DataGrid's PageSize property. For argument's sake, lets say the PageSize property is set to 5. This indicates that five records will be displayed on each DataGrid page. If there are more than five records in the table, page number navigation links will be rendered to the Web page. For the stored procedure you will need to find out how many total records there are, and then retrieve only the records you need to display for the requested page. To do this, add the following stored procedure to the database:

CREATE PROCEDURE [GetGuestbookEntries]
@pagesize int,
@pagenumber int,
@totalrecords int output
AS
DECLARE @startrecord int
SELECT @totalrecords = COUNT(*) FROM Guestbook
SET @startrecord = @totalrecords - (@pagesize * @pagenumber)
SELECT * FROM Guestbook
WHERE
[id] >= @startrecord
AND
[id] <= @startrecord + @pagesize
ORDER BY
entryDate DESC
GO

One point to note is that this stored procedure assumes that all of the records in the database are numbered sequentially beginning at 1, without skipping any numbers, as they are in the sample we are building. If any records are deleted, or numbers are skipped, this stored procedure will not work as expected. If you anticipate this scenario happening, you could consider using a SQL temporary table to retrieve the records and renumber them.

The GetGuestbookEntries stored procedure takes two input parameters and returns a result set and one output parameter. The two input parameters are @pagesize, which is how many records are being displayed per page, and @pagenumber, which is the page number to be displayed. For example, if you are showing five records per page, and you are displaying the 3rd page, then @pagesize is 5 and @pagenumber is 3. The output parameter, @totalrecords, returns the total record count from the table. You will use this number to determine how many pages there are for displaying the navigation links.

In the stored procedure you create another variable, @startrecord, which is used to determine which record number (using the id value) is the first record to be selected. The @startrecord is the total number of records, minus the page size, multiplied by the page number. For example, if there are 100 records, and you are showing five records per page, and you are showing page three, then the @startrecord is 100-(5*3), or 85.

You then select the number of records you are showing on one page (5 in our example), where the id value is greater than or equal to the @startrecord value and less than or equal to the @startrecord plus the page size. In our example, that would be records 85 through 90. These records are then ordered by the entryDate in descending order. This method will return one page of records, ensuring that page one will have the newest records (100-(5*1) = records 95 through 100), and each page will list the entries from newest to oldest.

The GuestbookData Class Structure

Create a new file named GuestbookData.cs. Begin building this class by adding the following code, using statements and class declarations:

using System;
using System.Web;
using System.Data;
using System.Data.SqlClient;
using System.Configuration;

public class GuestbookData
{
}

Inside the class you will expose one public property, VirtualItemCount. This property will be used to enable the paging functionality on the DataGrid. I'll get more into that in a while, but for now add this line of code inside the class declaration:

public int VirtualItemCount;

The addGuestbookData Method

The first method you are going to build is the addGuestbookData. This method will take in the name of the Web site visitor, their e-mail address, the title of their message, and the body of the message as its arguments. These will all be string values. In the method you will connect to the SQL database and execute a stored procedure. Add the following code inside the class declaration:

public void addGuestbookData(string visitorName, string Email, string Title, string Message)
{
SqlConnection con = new SqlConnection(ConfigurationSettings.AppSettings["conString"]);

public class Guestbook : Page
{
protected DataGrid entryGrid;
protected TextBox Name;
protected TextBox Email;
protected TextBox Title;
protected TextBox Message;

protected void Page_Load(object sender, EventArgs e)
{
if(!Page.IsPostBack)
{
BindGrid();
}
}

protected void Submit_Click(object sender, EventArgs e)
{
if(Page.IsValid)
{
SqlCommand cmd = new SqlCommand("AddMessageToGuesbook", con);
cmd.CommandType = CommandType.StoredProcedure;
cmd.Parameters.Add(new SqlParameter("@visitorName", SqlDbType.NVarChar));
cmd.Parameters.Add(new SqlParameter("@email", SqlDbType.NVarChar));
cmd.Parameters.Add(new SqlParameter("@title", SqlDbType.NText));
cmd.Parameters.Add(new SqlParameter("@message", SqlDbType.NText));

cmd.Parameters["@visitorName"].Value = visitorName;
cmd.Parameters["@email"].Value = Email;
cmd.Parameters["@title"].Value = Title;
cmd.Parameters["@message"].Value = Message;

try
{
con.Open();
cmd.ExecuteNonQuery();
con.Close();
}
catch
{
//Add error handling
}
}

The addGuestbookData uses a System.Data.SqlClient.SqlCommand object to execute a stored procedure. The stored procedure, AddMessageToGuestbook, takes four input parameters, @visitorName, @email, @title, and @message. The parameters are created in the command's parameters collection. Each parameter has its value assigned using the arguments passed into the method call.

Once all of the parameters are created and their values are set, you open the connection and use the ExecuteNonQuery method of the SqlCommand to execute the stored procedure. The ExecuteNonQuery method is used when a result set is not expected, such as in an INSERT command.

The getGuestbookData Method

Now we shall move onto BindGrid. We shall build the getGuestbookData method. This method is designed to return the records for one page of data to display in the DataGrid. I will explain how the paging functionality works in a while, but for now let's focus on this method. Add the following code to the Guestbook.cs code behind file, inside the class declaration:

protected void BindGrid()
{
//Make
            sure the form is cleared
Name.Text = Email.Text =
            "";
Title.Text = "";
Message.Text = "";
public DataSet
            getGuestbookData(int PageSize, int PageNumber)
{
//Create all
            the data objects
DataSet myDataSet = new
            DataSet();
SqlConnection con = new
SqlConnection(ConfigurationSettings.AppSettings["conString"]);
SqlConnection("server=localhost;database=ASPToday;uid=sa;pwd=;");
SqlDataAdapter
            cmd = new SqlDataAdapter("GetGuestbookEntries",
            con);
cmd.SelectCommand.CommandType =
            CommandType.StoredProcedure;
cmd.SelectCommand.Parameters.Add(new
SqlParameter("@pagesize",
            SqlDbType.Int));
            cmd.SelectCommand.Parameters.Add(new
SqlParameter("@pagenumber",
            SqlDbType.Int));
            cmd.SelectCommand.Parameters.Add(new
SqlParameter("@totalrecords",
            SqlDbType.Int));

cmd.SelectCommand.Parameters["@pagesize"].Value
            = PageSize;
cmd.SelectCommand.Parameters["@pagenumber"].Value =
            PageNumber;
cmd.SelectCommand.Parameters["@totalrecords"].Direction
            = ParameterDirection.Output;

//Fill the
            DataSet
cmd.Fill(myDataSet, "Messages");

//Set the
            VirtualItemCount to render the correct number of navigation
            links
VirtualItemCount =
            Convert.ToInt32(cmd.SelectCommand.Parameters["@totalrecords"].Value);

entryGrid.DataBind();
}
return
            myDataSet;
}

The getGuestbookData method takes in two arguments, the page size (number of records per displayed page), and the requested page number. The data is retrieved using an SqlDataAdapter, and it is filled into a DataTable in a DataSet. You do this because you need to access the @totalrecords output parameter before the DataGrid is bound. If you were to use a SqlDataReader, then the output parameter would not be accessible until all of the records in the SqlDataReader have been read, which is too late. You need the @totalrecords value to set the DataGrid.VirtualItemCount, which the DataGrid uses in conjunction with the PageSize property to determine how many page navigation links to render.

Executing the SqlDataAdapter.Fill method opens the connection to the database and returns the result set into a new DataTable in the specified DataSet. In the code sample you create a new DataTable named "Messages." Next you set the DataGrid.VirtualItemCountVirtualItemCount property of the GuestbookData class to the value of the @totalrecords parameter. This is the total number of records in the guestbook table. Lastly you return the DataSet to the calling object.

Building the Presentation Layer

With the DAL class built, you can begin building the input interface for the guestbook. The input interface is made up of four TextBox es and a Button. Create a new Web Form named guesbook.aspx, and then add the following code:

<%@ Page Language="C#" INHERITS="Guestbook" %>
<html>
<body>
<form runat="server">
<table border="0" bordercolor="Tan"
cellpadding="4" cellspacing="0"
width="100%">
<tr bgcolor="Maroon">
<td valign="top" colspan="2"
style="font: bold 10pt Verdana; color:white;">
Add your Guesbook Entry:
</td>
</tr>
<tr>
<td valign="top"
style="font: bold 9pt Verdana">
Name:
</td>
<td Width="100%">
<asp:TextBox runat="server" id="Name"
Width="400"
Font-Size="9pt" Font-Name="Verdana"
BorderStyle="Solid" BorderWidth="1" />
<asp:RequiredFieldValidator runat="server"
ControlToValidate="Name"
ErrorMessage="*required"
Font-Size="9pt" Font-Name="Verdana" />
</td>
</tr>
<tr>
<td valign="top"
style="font: bold 9pt Verdana">
E-Mail:
</td>
<td Width="100%">
<asp:TextBox runat="server" id="Email"
Width="400"
Font-Size="9pt" Font-Name="Verdana"
BorderStyle="Solid" BorderWidth="1" />
<asp:RequiredFieldValidator runat="server"
ControlToValidate="Email"
ErrorMessage="*required"
Font-Size="9pt" Font-Name="Verdana" />
</td>
</tr>
<tr>
<td valign="top"
style="font: bold 9pt Verdana">
Title:
</td>
<td Width="100%">
<asp:TextBox runat="server" id="Title"
Width="400"
Font-Size="9pt" Font-Name="Verdana"
BorderStyle="Solid" BorderWidth="1" />
<asp:RequiredFieldValidator runat="server"
ControlToValidate="Title"
ErrorMessage="*required"
Font-Size="9pt" Font-Name="Verdana" />
</td>
</tr>
<tr>
<td valign="top"
style="font: bold 9pt Verdana">
Message:
</td>
<td Width="100%">
<asp:TextBox runat="server" id="Message"
TextMode="Multiline" Rows="4"
Width="400"
Font-Size="9pt" Font-Name="Verdana"
BorderStyle="Solid" BorderWidth="1" />
<asp:RequiredFieldValidator runat="server"
ControlToValidate="Message"
ErrorMessage="*required"
Font-Size="9pt" Font-Name="Verdana" />
</td>
</tr>
<tr>
<td></td>
<td Width="100%">
<asp:Button runat="server"
Text="Click Here to Add Your Entry"
Width="400"
Font-Size="9pt" Font-Name="Verdana"
BorderStyle="Solid" BorderWidth="1"
BorderColor="Tan" BackColor="Maroon"
ForeColor="White" Font-Bold="True"
OnClick="Submit_Click" />
</td>
</tr>
</table>

The preceding code sets up the input portion of the Web Form. This includes four TextBox es for the visitor to enter their name, e-mail address, the message title and the message. The Message TextBox uses the TextMode property and the Row property to create a <textarea> HTML element. Each TextBox control has a RequiredFieldValidator associated with it. The validator control allows you to require that the Web site visitor fills out all the TexBox es. The Button control uses the OnClick property to define the event handler, Submit_Click, that should be fired when the button is clicked.

Just below the input portion of the Web form, you will use a DataGrid to display the guestbook entries. Add the following code:

<asp:DataGrid runat="server" id="entryGrid"
AutogenerateColumns="False"
Width="100%"
BorderColor="Tan" BorderWidth="1"
Gridlines="Horizontal"
Cellpadding="4" CellSpacing="0"
HeaderStyle-Font-Name="Verdana"
HeaderStyle-Font-Size="10pt"
HeaderStyle-Font-Bold="True"
HeaderStyle-BackColor="Maroon"
HeaderStyle-ForeColor="White"
ItemStyle-Font-Name="Verdana"
ItemStyle-Font-Size="9pt"
AlternatingItemStyle-BackColor="Tan"
AllowPaging="True"
PageSize="5"
AllowCustomPaging="True"
OnPageIndexChanged="entryGrid_PageIndexChanged" >
<PagerStyle Mode="NumericPages"
Font-Name="Verdana"
Font-Size="9pt"
Font-Bold="True"
HorizontalAlign="Center"
Position="TopAndBottom" >
</PagerStyle>
<Columns>
<asp:TemplateColumn HeaderText="Guestbook Entires">
<ItemTemplate>
<b><%# DataBinder.Eval(Container.DataItem, "title") %></b>
- <%# DataBinder.Eval(Container.DataItem, "entryDate", "{0:f}") %><br>
Submitted By: <%# DataBinder.Eval(Container.DataItem, "visitorName") %>
- <a href="mailto:<%# DataBinder.Eval(Container.DataItem, "email") %>">
<%# DataBinder.Eval(Container.DataItem, "email") %></a><br>
<%# DataBinder.Eval(Container.DataItem, "message") %>
</ItemTemplate>
</asp:TemplateColumn>
</Columns>
</asp:DataGrid>
</form>
</body>
</html>

A DataGrid, by default, will render a basic HTML table with one table row for each record displayed. You can bind any object that supports the IEnumerable, such as a DataTable or an Array. The DataGrid will automatically generate rows and columns for all of the data in its data source. This is called data binding. It is out of the scope of this article to fully explain data binding. The basic concept is that a data source is provided, such as a DataTable, and bound to the DataGrid, which results in an HTML table representing all of the data.

Your DataGrid uses a number of properties to customize its look and feel, as well as some properties to set up the paging format. The AutogenerateColumns property defines whether the DataGrid will generate the columns of the table based on the data source, or if you will create the columns using the various DataGridColumn objects, such as the BoundColumn, or TemplateColumn. The following style properties are used to customize the layout with things like color, font, etc. The four paging-related properties are used to set up the paging structure of the DataGrid. I'll cover that shortly, but first let's look at how the OnClick event of the Button at the top of the Web form is handled.

The Web Form Code Behind Class

The guestbook.aspx Web Form uses code behind technology to separate the programmatic code and the page layout. Create a file named Guestbook.cs, this will be the code behind the file. I chose this name since it is the same name that Visual Studio.NET would automatically create for the guestbook.aspx Web Form.

The VirtualItemCount property is used in conjunction with the PageSize property to determine how many page navigation links to render. If the VirtualItemCount is 100, and the PageSize is 10, then page navigation links for 10 pages will be rendered (100/10 = 10). Once the VirtualItemCount property is set, you set the Messages table as the DataGrid.DataSource, and call the DataBind method.

Can You Handle Paging?

The last thing you have to do before you can use the guestbook is add an event handler for the DataGrid.PageIndexChanged event. This event is fired whenever one of the page navigation links is clicked. In the PageIndexChanged event handler you set the CurrentPageIndex of the DataGrid to the NewPageIndex property of the DataGridPageChangedEventArgs. Add the following event handler to the Guestbook.cs code behind the file.

protected voidentryGrid_PageIndexChanged(object sender,DataGridPageChangedEventArgs e)
{
entryGrid.CurrentPageIndex = e.NewPageIndex;
BindGrid();
}

In the DataGrid server control you set the OnPageIndexChanged property to "entryGrid_PageIndexChanged". This will call the newly created event handler whenever a page navigation link is clicked. When the event handler fires, the DataGrid.CurrentPageIndex property will be set to the value of the DataGridPageChangedEventArgs.NewPageIndex, which is the page index for the page number that was clicked on. Once the CurrentPageIndex is set, you call the BindGrid method, and the data is retrieved, the DataGrid is bound, and finally the page is rendered to the client.

Add the following code to the code behind file:

using System;
using System.Web;
using
            System.Web.UI;
using System.Web.UI.HtmlControls;
using
            System.Web.UI.WebControls;
using System.Data;

public class
            Guestbook : Page
{
protected DataGrid entryGrid;
protected
            TextBox Name;
protected TextBox Email;
protected TextBox
            Title;
protected TextBox Message;

protected void
            Page_Load(object sender, EventArgs
            e)
{
if(!Page.IsPostBack)
{
BindGrid();
}
}

protected
            void ClearForm()
{
//Make sure the form is
            cleared
Name.Text = "";
Email.Text = "";
Title.Text =
            "";
Message.Text = "";
}
}

The code behind class Guestbook inherits from the System.Page class, which gives it all the functionality of the Page class, such as the Page_Load event. Just after the class decla ration you declare five variables. These variables directly map to objects in the guestbook.aspx file. This maps these objects to the class so you can access them programmatically.

In the Page_Load event handler you check to see if the current request is a postback. If it is not a postback then you call the BindData method, which you will build shortly.

The other method in the preceding code is called ClearForm. This will be used to clear all of the TextBoxes after a user's post is submitted.

Handling the Submit Click

The Submit_Click event handler is fired whenever the Button control in the Web Form is clicked. This is because you mapped the Button.OnClick property to a method named Submit_Click. Add the following code inside the class declaration of the code behind class:

  protected void
            Submit_Click(object sender, EventArgs
            e)
{
if(Page.IsValid)
{
GuestbookData gbDal = new
            GuestbookData();
gbDal.addGuestbookData(Name.Text.Trim(),
Email.Text.Trim(),
Title.Text.Trim(),
Message.Text.Trim());
BindGrid();
}
}

In the Submit_Click event handler you first check to see if the page is valid. This will evaluate true if all of the TextBoxes have something in them. If one or more are empty, the RequiredFieldValidators will return false, and the Page.IsValid property will be false. If the Page is valid then you create an instance of the GuestbookData DAL class you created earlier. You then call the addGuesbookData method, passing in the values from the TextBoxes. After the data has been entered in the database, you call the BindData method. This ensures that the new data is entered into the database before the DataGrid is bound.

Binding Data One Page at a Time

Now we shall look at the BindGrid method. Add the following code to the Guestbook.cs code behind file, inside the class declaration:

  protected void
            BindGrid()
{
ClearForm();
//Instantiate the DAL
            object
GuestbookData gbDal = new GuestbookData();
//Set the
            data source and bind the grid
entryGrid.DataSource =
            gbDal.getGuestbookData(entryGrid.PageSize,
entryGrid.CurrentPageIndex + 1);
entryGrid.VirtualItemCount =
            gbDal.VirtualItemCount;
entryGrid.DataBind();
}

In the BindGrid method you do two things. You call ClearForm to clear all of the TextBoxes so the form starts fresh every time, and you bind the DataGrid to the result of calling the getGuestbookData method.

The PageSize argument of the getGuestbookData method is set from the DataGrid.PageSize value, and the PageNumber argument is set from the DataGrid.CurrentPageIndex + 1. The CurrentPageIndex is a zero-based index; the CurrentPageIndex for page one is 0, the CurrentPageIndex for page two is 1, and so on. To set the PageNumber argument you add 1 to the CurrentPageIndex so page one is 1, page two is 2, and so on.

The DataGrid.DataSource property defines where the DataGrid will find the data to bind. Setting this to the result of calling getGuestbookData will cause that method to fire, and return a DataSet. The DataGrid will default to the first DataTable in the DataSet.Tables collection. In this case, the getGuestbookData method returns a DataSet with only one DataTable in it, so that is what the DataGrid will use as its data source.

The DataGrid.VirtualItemCount property is used in conjunction with the PageSize property to determine how many page navigation links to render. If the VirtualItemCount is 100, and the PageSize is 10, then page navigation links for 10 pages will be rendered (100/10 = 10). When the getGuestbookData method was called, the GuestbookData.VirtualItemCount property was set to the total number of records in the guestbook table. Once the VirtualItemCount property is set, you set the Messages table as the DataGrid.DataSource, and call the DataBind method. The DataGrid is bound to the data source, which is one page of data, and page navigation links are rendered based on the PageSize property and the VirtualItemCount property.

Can You Handle Paging?

The last thing you have to do before you can use the guestbook is add an event handler for the DataGrid.PageIndexChanged event. This event is fired whenever one of the page navigation links is clicked. In the PageIndexChanged event handler you set the CurrentPageIndex of the DataGrid to the NewPageIndex property of the DataGridPageChangedEventArgs. Add the following event handler to the Guestbook.cs code behind file.

protected voidentryGrid_PageIndexChanged(object sender,DataGridPageChangedEventArgs e)
{
entryGrid.CurrentPageIndex = e.NewPageIndex;
BindGrid();
}

In the DataGrid server control you set the OnPageIndexChanged property to "entryGrid_PageIndexChanged". This will call the newly created event handler whenever a page navigation link is clicked. When the event handler fires, the DataGrid.CurrentPageIndex property will be set to the value of the DataGridPageChangedEventArgs.NewPageIndex, which is the page index for the page number that was clicked on. Once the CurrentPageIndex is set, you call the BindGrid method, and the data is retrieved, the DataGrid is bound, and the page is rendered to the client.

Running the Application

Now that all of the class files are written, your application is done. The only thing left to do is compile the classes into a DLL and put it in the \bin directory of your application. If you are using Visual Studio.NET you can do this by selecting Build from the Build menu. If you are not using Visual Studio.NET you can use a command line compiler.

Open a command prompt and navigate to the directory that your class files are in using the CD command. The CD command will look something like this:

C:> CD C:\Inetpub\wwwroot\ASPToday\Guestbook

Once there, use the csc.exe compile by executing the following command:

csc.exe /target:library /out:ASPToday.dll *.cs /r:System.dll /r:System.Data.dll

This will compile all of the .cs files in the current directory into a DLL named ASPToday.dll. Move this DLL to the \bin directory of your application, and now you are ready to go.

Summary

In this article I showed you how to build a functional and efficient guestbook for your Web application. The guestbook uses a DataGrid with custom paging to ensure that only the necessary records are retrieved from the database and displayed in the browser for the visitor. In the following articles I will show you how to build a mailing list and use the System.Web.Mail namespace to send e-mail, such as a newsletter, to a list of recipients in a database. In the third installment, I will show you how to build a message board where visitors to your Web site can post and reply to messages.


Programmatically Wiring Up Events in C#

Programmatically Wiring Up Events in C#
http://www.csharptoday.com/content.asp?id=1601
Article Rating: 3.5
Published on July 17, 2001
Abstract
One of the most powerful features of .NET is the ability to create dynamic forms for Windows Applications. Although possible before, it’s now very easy to achieve in .NET. This enables us to obtain objects at runtime derived from System.Windows.Forms.Control and use them in exactly the same manner as if we’d painted them on with the forms designer at design time. Dynamic controls can be used to adjust your application’s user interface depending on certain environmental information - for example a database administration tool, where a button can be dynamically added to the tool for each table detected within a database. The slightly tricky aspect of dynamic control creation is wiring up event handlers to the controls. In this article, Matthew Reynolds shows us how to dynamically create controls in C# and wire up methods to the control’s events. This article is available as an $8 e-doc at http://www.amazon.com/exec/obidos/ASIN/B00005T5SP/ref=nosim/wrox-20.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

One of the most powerful features of .NET is the ability to create dynamic forms for Windows Applications. Although this was possible before, it's now very easy to achieve in .NET. These enable us to obtain objects at runtime derived from System.Windows.Forms.Control and use them in exactly the same manner as if we'd painted them on with the forms designer at design time.

The slightly tricky aspect of dynamic control creation is wiring up event handlers to the controls. In this article, we'll be looking at how to dynamically create controls in C# and wire up methods to the control's events.

Creating the Project

Create a new Visual C# - Windows Application project now and call it DynamicButtons.

We need to create a basic form layout that looks like this:

You'll need to call the button cmdCreateButtons and the text box should be called txtLog. Make sure that txtLog has its Multiline property set to True and that ScrollBars is set to Vertical.

When cmdCreateButtons is clicked, we want to add six buttons to the form positioned in the blank space to the right of txtLog. When we configure the event handlers for these new buttons, we need to know which button originated the call. To do this, we need each of the buttons to have an additional integer property called ID that we'll number 1 through 6 as we create them.

One feature of .NET that VB developers need to get used to is the idea of inheriting existing classes from the Framework and extending them. This technique is extremely powerful. In our case, what we want to do is create a new class called DynamicButton and inherit it from System.Windows.Forms.Button. This means that our new class will have all the functionality of a normal Button control, but it will also have the other stuff that we need, specifically a new property called DynamicId. As our class is derived from Button, it can do anything that a button control should be able to do, i.e. it responds to user clicks, it can be placed on a form, and so on.

Moreover, a really important aspect of inheritance is that anyone who knows how to use a Button can also use our DynamicButton. In this case, however, they won't necessarily know how to use our extended functionality, but for most part they can use it as a normal button. They can also derive their own classes from DynamicButton and add their own functionality.

Create a new class called DynamicButton and add this code:

namespace DynamicButton
{
using
            System;

public class DynamicButton :
            System.Windows.Forms.Button
{
public int
            DynamicId;

public
            DynamicButton()
{
}
}
}

Literally, that's it! Now we have a new class that behaves exactly like a button, but now has an integer property called DynamicId.

In fact, we do want to add a little more extra functionality to our button. Specifically, we want to supply the ID whenever the button is created, and we also want to set the default text of the button to reflect this ID:

        public
            DynamicButton(int newId)
{
// set the ID...
DynamicId =
            newId;

// set the display...
Text = "Dynamic Button " +
            DynamicId;
}

Now we can move on to creating the buttons.

Creating the Buttons

To create the buttons, we need to add handler code behind the cmdCreateButtons button. This is relatively simple code, but looks a little complicated because we have to manage the layout of the new buttons ourselves, i.e. we have to keep track of the coordinates of the new button as we're adding them.

The first part of the handler sets up the loop:

protected void cmdCreateButtons_Click
            (object sender, System.EventArgs e)
{
// work out where the
            new buttons should be positioned...
int spacing = 8;
int y =
            txtLog.Top;
int width = this.Width - txtLog.Right - (3 *
            spacing);
int height = 25;

// loop and create the new
            buttons...
int n = 0;
for(n = 0; n < 6; n++)

Here, we've said that the height of the button will always be 25 and that y will start at the same point as the top of txtLog. We've also specified that spacing is 8, and this is value that we'll use to space out the buttons.

To create the button, we do this:

// create the new button...
DynamicButton newButton = new DynamicButton(n);

Notice how we pass the value of n through to the constructor as the DynamicId of the button. Next we need to position the button on the form:

newButton.Left = txtLog.Right + spacing;
newButton.Top = y;
newButton.Width = width;
newButton.Height = height;

The complicated bit there is deciding on the width of the button. We want it to be equally spaced between the right edge of txtLog and the right edge of the form itself.

Finally, we add the button to the control array and move y down to the position of the next button:

      // add
            the button to the form...
this.Controls.Add(newButton);

//
            next...
y += (height + spacing);
}
}

If we run the project now, this is what we'll get:

Of course, although the buttons have been created, they won't actually do anything. For this, we need to wire up the event handlers.

Wiring up Event Handlers

In .NET, there's a subtle difference between the ways that the term delegate is used. A "delegate" can be defined using the Delegate keyword and this describes the signature of the method to be used with the delegate. The signature of a method is determined through a combination of the parameters and the return type. So, a method that takes a single Object parameter and returns a String will have a different signature to a method that takes a Hashtable parameter and returns a String.

Where this gets confusing is that the method that's called is also called a delegate. For the remainder of this article, a "delegate" will refer to the definition of a delegate as constructed with the Delegate keyword. A "delegated method" will refer to the method that will actually be called as a result of a delegate invocation.

To wire up an event handler, we have to create a delegated method with this signature:

void DelegatedMethod(Object sender, System.EventArgs e)

What's really important here is that the name of the method that we create is totally arbitrary, i.e. the method can have any name that we wish it to have. So long as we create a method that has no return value, but accepts a System.Object and then a System.EventArgs object, we'll be OK. As you might have guessed, the order of parameters affects the signature of the method. If we reversed the order of the parameters, our method would not have the required signature for our delegated method.

All of the event handlers in the Windows.Forms control set fit this model, meaning that we can create some pretty powerful event handlers. We could, if we wanted, have every event from every control on a form routed through to the same delegated method. In fact, we can call a delegated method on any object we want from any other object. So, a button control on a form object called MyDialog could call a delegated method on a form object called MyDialogOwner.

The sender parameter provides a reference to the control that fired the event. In order to get at our DynamicId property, we have to cast this to a DynamicButton type. The e parameter specifies some event specific arguments. We're not going to worry about this here, but for some events this contains some extra information about what actually called the event.

Here's our click handler:

// DynamicButton_Click...
protected void
            DynamicButton_Click(object sender, System.EventArgs e)
{
//
            cast the sender to a button...
DynamicButton button =
            (DynamicButton)sender;

// msgbox...
AddToLog("Clicked " +
            button.DynamicId);
}

There's nothing at all unusual about this method. Now we have to wire up this method to the Click event of our dynamic buttons. Just before we call Controls.Add, we can add a handler:

      // wire
            up the handler...
newButton.Click += new
            System.EventHandler(this.DynamicButton_Click);

// add the
            button to the form...
this.Controls.Add(newButton);

Now, whenever the button wants to fire the Click event, the Framework goes through the list of delegated methods wired to that event firing each one in turn.

This is an important point - in .NET events can have more than one handler. This means that we could create another delegated method, wire it up using a new System.EventHandler instance and both methods would get called when the event was fired. Of course, the methods are not called simultaneously as this would, in effect, create a multithreaded application, and this is not something we want to approach casually.

We haven't defined what our AddToLog function looks like, so let's so that now:

// AddToLog - updates the log
            view...
private void AddToLog(String buf)
{
// update the
            log control...
txtLog.Text = (buf + "\r\n" +
            txtLog.Text);
}

Now let's try running the application and clicking on the buttons:

Other Handlers

Handlers that respond to other events on DynamicButton can be added in a similar manner. Firstly, we create methods for each one:

// DynamicButton_Enter...
protected void
            DynamicButton_Enter(object sender, System.EventArgs e)
{
//
            cast the sender to a button...
DynamicButton button =
            (DynamicButton)sender;

// msgbox...
AddToLog("Enter " +
            button.DynamicId);
}

//
            DynamicButton_Leave...
protected void DynamicButton_Leave(object
            sender, System.EventArgs e)
{
// cast the sender to a
            button...
DynamicButton button = (DynamicButton)sender;

//
            msgbox...
AddToLog("Left " + button.DynamicId);
}

Then we can add handlers for each one:

// wire up the handler...
newButton.Click += _
new System.EventHandler(this.DynamicButton_Click);
newButton.MouseEnter += _
new System.EventHandler(this.DynamicButton_Enter);
newButton.MouseLeave += _
new System.EventHandler(this.DynamicButton_Leave);

Now if you run the application again, messages will be added to the log as you hover the mouse over the buttons.

Another Example

Now that we've seen an example of how to create dynamic controls, let's take a look at how this technique can be used in your applications.

Dynamic controls can be used to adjust your application's user interface depending on certain environmental information. A classic example of this is adding new options to a toolbar when new "plug ins" or modules are added to the applications directory. For example, installing Adobe Acrobat on my computer automatically adds a button for creating an Acrobat document to my Word toolbar.

Another example could be on a database administration tool. When the tool is connected to the database, a button could be added to the UI representing each table contained within the database. Or, we could create a set of text box controls containing the names of all the XML files contained in a certain directory, and so on.

In this example, we're going to create an application that is able to load an assembly from disk, scan that assembly for types that inherit from System.Windows.Forms.Control and display a button on the form for each type that it finds. Clicking on the button will create an instance of the control and display it on the form.

Creating the Project

Create a new Visual C# - Windows Application project and call it ControlLoader. For these first steps, we won't be painting any controls onto the new form, but you might want to change the Text property to something like " Control Container ".

Add these members to the form:

   public class Form1 :
            System.Windows.Forms.Form
{
// members...
private ArrayList
            _buttons = new ArrayList();
private int _nextY =
            ButtonSpacing;
private Control _containedControl;

//
            constants...
const int ButtonHeight = 25;
const int
            ButtonSpacing = 5;
const int ButtonWidth = 200;

We're going to maintain a list of the buttons that we add in an ArrayList called _buttons. As each button is added to the form, we're going to need to position it at the correct y-coordinate, hence _nextY. We're only going to contain a single control at a time, and this will be stored in _containedControl. Finally, we're going to need some metrics that describe the layout of the buttons and these are described in the three constants at the end.

The buttons themselves will be constructed from a new class derived from System.Windows.Forms.Button. This new class will be called TypeButton and will have an extra property called ControlType that will contain a System.Type object. This Type object will represent the control in the assembly that we load. Create a new class called TypeButton and add a reference to the System.Windows.Forms namespace:

using System;
using System.Windows.Forms;

Next, add this code:

   public class TypeButton :
            System.Windows.Forms.Button
{
public Type
            _controlType;

public TypeButton()
{
}

//
            ControlType - get or set the property...
public Type
            ControlType
{
get
{
return
            _controlType;
}
set
{
_controlType = value;
this.Text
            = _controlType.FullName;
}
}
}

As you can see, when we set the ControlType property, we change the text of the button to be that of the full name of the type.

The only method we need to create on TypeButton is one that creates an instance of the control. Add this method that uses System.Activator to create an instance of the class and cast it to a System.Windows.Forms.Control:

      //
            CreateInstance - create an instance of the type...
public Control
            CreateInstance()
{
// return an instance of the new
            control...
return
            (Control)Activator.CreateInstance(ControlType);
}

Adding the Buttons

For the first test run, we'll add a TypeButton that represents an instance of a System.Windows.Forms.DataGrid control. This will let us test the logic of what we've done without having to go through the extra hassle of loading an assembly and scanning its types.

Let's look at the Form1.AddType method. This method will take a System.Type object and add a new TypeButton to the form. Firstly, we need to create an instance of a TypeButton and set its ControlType property to the new type:

      //
            AddType - add a type button to the layout...
public void
            AddType(Type controlType)
{
// firstly, create a new type
            button...
TypeButton button = new
            TypeButton();
button.ControlType = controlType;

Secondly, we need to add the TypeButton to our ArrayList that contains our list of buttons:

// secondly, add this button to our array...
_buttons.Add(button);

Now we can position the button on the form and add the button to the form's control list:

         //
            now, position the button...
button.Left =
            ButtonSpacing;
button.Width = ButtonWidth;
button.Top =
            _nextY;
button.Height = ButtonHeight;

// adjust the next y
            value...
_nextY += (ButtonHeight + ButtonSpacing);

//
            display the button...
this.Controls.Add(button);

Finally, we need to wire up the button's click event so that we can contain an instance of the control it represents:

         //
            next, wire up an event handler...
button.Click += new
            EventHandler(this.ButtonClick);
}

Of course, we haven't built ButtonClick yet, so let's do that:

// ButtonClick - called whenever a button is clicked...
protected void ButtonClick(object sender, System.EventArgs e)
{
// convert the sender to a type button...
TypeButton button = (TypeButton)sender;

Our first job is to cast sender to a TypeButton. That lets us use CreateInstance to create the control. Before we can do that, though, if we're already containing a control, we need to remove it from the form's control list:

// if we already have a contained control, remove it...
if(_containedControl != null)
Controls.Remove(_containedControl);

Now we can create the control:

// create an instance of the new control...
_containedControl = button.CreateInstance();

Finally, we can position the control on the form:

         //
            position the control on the form...
_containedControl.Left =
            ButtonWidth + (3 * ButtonSpacing);
_containedControl.Top =
            ButtonSpacing;
_containedControl.Width = this.Width -
            _containedControl.Left - (4 *
            ButtonSpacing);
_containedControl.Height = this.Height - (8 *
            ButtonSpacing);
this.Controls.Add(_containedControl);
}

Testing What We Have

Before we can see how far we've got, we need to add a button to the form. We'll use the System.Windows.Forms.DataGrid control. Add this code to Form1:

      private
            void Form1_Load(object sender, System.EventArgs e)
{
// on
            load, add a button type...
DataGrid grid = new
            DataGrid();
AddType(grid.GetType());
}

Now try running the project. If you click the System.Windows.Forms.DataGrid button, you'll see this:

Loading an Assembly

To round of this example, we'll show how we can load up an assembly at run time, scan through the assembly looking for types that derive from System.Windows.Forms.Control and add buttons for each one we find.

To load the assembly, we need a URL. This URL can either point to an assembly on the local computer, the network or somewhere on the Internet. In this example, we're going to use the code base for the current assembly. For this exercise, this will have the effect that it will load up all the types from the assembly that we're currently building. However, the same technique will work irrespective of where the assembly actually comes from.

The first thing to do is get this URL. Add this code to Form1_Load:

      private
            void Form1_Load(object sender, System.EventArgs e)
{
// on
            load, add a button type...
DataGrid grid = new
            DataGrid();
AddType(grid.GetType());

// find the name of
            our assembly...
String filename =
            this.GetType().Module.Assembly.CodeBase;

This line is a roundabout technique for getting hold of the assembly that implements the class of the object that the code is called from. In this case, it gets hold of the URL of the assembly containing Form1.

We need to use try.catch when we load the assembly, as there are a lot of things that can go wrong with this routine. We use a shared method on Assembly to do this:

// try and load the assembly...
try
{
// use load from...
Assembly controlAssembly = Assembly.LoadFrom(filename);

Once we have the assembly we can iterate through the types:

            //
            now get a list of types...
foreach(Type testType in
            controlAssembly.GetTypes())
{

The trick now is to create an instance of the type that we're testing, but cast it to Control. As all this is wrapped in another try.catch if we get to the next line we know that we have a fully qualified control.

// try and create an instance of the control
// and cast it to a control...
try
{
Control testControl = (Control)Activator.CreateInstance(testType);

Here's a wrinkle. System.Windows.Forms.Form is derived from Control as it uses Control 's containerization properties to display the controls drawn onto the form. If we check the TopLevelControl property, this will always be set if the class is derived from form.

                  //
            we need to make sure it's not a "top level"
            control...
if(testControl.TopLevelControl == null)
{
// if
            we got here, it is a
            control...
AddType(testType);
}
}

Finally, we can round off the two exception handlers:

               catch
{
//
            if we got here, we don't care about the
            object!
}
}
}
catch(Exception
            ee)
{
MessageBox.Show("The assembly could not be loaded. " +
            ee.Message);
}
}

Before we test this, we should drop another couple of controls into the project. Create a first class called DemoTextBox and add this inheritance clause:

public class DemoTextBox : System.Windows.Forms.TextBox

Now, create another class, this time called DemoMonthCalendar and add this clause:

public class DemoMonthCalendar : System.Windows.Forms.MonthCalendar

Build and run the project. You'll get something like this:

Summary

In this article, we learned how to create controls on the fly and add them to a form. We created a new class derived from System.Windows.Forms.Button that enabled us to add extra functionality and properties to the button. We also saw how to wire up methods to the new controls that get called whenever the events are fired. Finally, we examined how we could scan through an assembly looking for classes and use the techniques we've learned to build a simple tool that would load and display those classes at the user's command.


Using Legacy COM(+) in .NET – Part 2

Using Legacy COM(+) in .NET – Part 2
http://www.csharptoday.com/content.asp?id=1610
Article Rating: 4
Published on June 20, 2001
Abstract
In part 1 of this article, we discussed how COM and .NET components interact seamlessly using Runtime Callable Wrappers (RCW) and COM Callable Wrappers (CCW). We discussed some of the common scenarios and the limitations of RCWs and CCWs. In addition to the most simple components needing to interact, there are large investments by companies in components that take advantage of some advanced features of COM such as error handling, COM+ services, shared properties and event handling to name a few. In this article, Kaushal Sanghavi explores how components using these advanced features can interoperate with little or no effort at all.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

In part 1 of this article, we discussed how COM and .NET components interact seamlessly using Runtime Callable Wrappers (RCW) and COM Callable Wrappers (CCW). We discussed some of the common scenarios and the limitations of RCWs and CCWs. In addition to the most simple components needing to interact, there are large investments by companies in components that take advantage of some advanced features of COM such as error handling, , COM+ services, shared properties and event handling to name a few. In this article, we will explore how components using these advanced features can interoperate with little or no effort at all.

HRESULTs and Exceptions

The RCW and the CCW are responsible for marshalling interfaces and parameters between .NET and COM. In addition to interfaces, method arguments and return values, the bridge between the COM and .NET worlds also needs to ensure that error handling and reporting is done correctly. COM and .NET work with errors in a very different manner. COM relies heavily on HRESULTs, while .NET ignores HRESULTs and works only with throwing and catching exceptions. There needs to be a translation between these different methods of error handling, and the RCWs and CCWs handle this seamlessly. When a managed component makes a call to an unmanaged component and a failure HRESULT is returned by the unmanaged component, the RCW consumes the HRESULT and throws an equivalent exception. The System.Runtime.InteropServices namespace contains a class called COMException which manages all exceptions thrown by the RCW that do not have an equivalent native exception in the managed space. Most common HRESULT values map to specific exceptions in the managed space; for example an HRESULT value of E_ACCESSDENIED would throw an exception of type UnauthorizedAccessException. However some unmanaged components use custom HRESULT values to communicate application specific errors. If the .NET runtime does not recognize the returned HRESULT, an exception of type COMException is thrown. This exception has a property ErrorCode which is the HRESULT value returned from the unmanaged component. The managed component can read this and take actions specific to the cause of the error.

While HRESULTs can communicate the cause of the error, if a more detailed description of the error is required, unmanaged components can implement the IErrorInfo interface. This contains details of the error such as the description of the error, the GUID of the interface that defined the error, the source of the error and the associated help file to explain this error, if any. If an unmanaged component uses the IErrorInfo interface to return errors, the RCW maps the fields of this interface to equivalent fields in the thrown managed exception.

Similarly, when an unmanaged component calls a managed component, and the managed component throws an exception, the CCW converts this to an equivalent HRESULT.

Some unmanaged components use HRESULTs to communicate successful completion in addition to the standard S_OK value. They define custom HRESULTs with the success bit set, to communicate additional information to the successful execution. Thus, they piggy back details about the method call on the HRESULT value. This is not supported by the RCW. The RCW only passes failure HRESULTs to the managed component; all success HRESULTs are consumed by the RCW.

COM+ Services

COM+ Services under Windows 2000 (a superset of MTS under NT 4.0) provide declarative support for transactions, synchronization, security, queuing and object pooling. A standard COM or COM+ component could be run under COM+ services to take advantage of these features with little to no coding at all. COM+ services are built on the same foundation as COM, and require support from the COM runtime. Further, they require the components to be COM aware, and follow the rules of COM. Thus, for a component to run under COM+, it has to be registered in the Windows registry and should support the COM interfaces.

In spite of this, managed components can run under COM+ services and enjoy all the benefits that COM+ provides like pooling or transaction support. The simplest way would be to create a managed component and use the RegAsm tool to generate the type library and register it in the Windows registry. Once the managed component is registered, you can import this into a COM+ application, and the component can now inherit all the benefits of running under a COM+ service. As you can see, in this case, the component itself is unaware of whether it is run under a COM+ application or whether it is running under vanilla COM. There is nothing inherent in the component that prevents it from running under either context.

Another way for a managed component to run under COM+ is to derive from the ServicedComponent class in the System.EnterpriseServices namespace. This ensures that this component will always run under a COM+ application. Even if the component is not explicitly added to a COM+ application, the first time an instance of this component is invoked, the .NET runtime checks to see if the component is registered with COM+, and if not, the .NET runtime registers the component. The serviced component can now be called from both managed and unmanaged components. This is known as Lazy Registration.

A component deriving from the ServicedComponent class can now use attributes that define the support it needs from the COM+ services. For example, a component can declaratively set its transaction characteristics to commit once a method returns without any exceptions. The System.EnterpriseServices defines attributes that can be used by serviced components to set COM+ properties. Let us explain a few of the important attributes.

  • ApplicationActivation: This is set on an assembly and specifies whether the component should be run in the caller's process or out of process. This is used during lazy registration.
  • ApplicationName: This is set on an assembly and specifies the name of the COM+ application. This is used during lazy registration.
  • Transaction: This is set on a component and specifies the type of transaction an instance of this component will run under. The options available are the same as those exposed by the COM+ MMC.
  • AutoComplete: This is set on a method and specifies whether the transaction should be committed automatically when the method returns without throwing an exception.
  • Description: This can be set on an assembly or a component and contains a description of the COM+ application or component.
  • ObjectPooling: This is set on a component and specifies whether an instance of this component will be pooled. Note that MTS does not support object pooling, so this attribute applies only to COM+.

The following code sample shows how these attributes can be set. We have not tied this to a database, so the transactions cannot commit or abort in reality. This sample only demonstrates the use of some of the attributes that we have discussed above.

using System.EnterpriseServices;

[assembly: ApplicationName("Sample Bank for AspToday")]
[assembly: ApplicationActivation(ActivationOption.Library)]
[assembly: Description("Simulate a sample bank")]

namespace SampleServicedComponent
{
using System;
using System.EnterpriseServices;

[Description ("This stores the bank information")]
[Transaction(TransactionOption.Required)]
[ObjectPooling (true, 1, 10)]
public class Bank : ServicedComponent
{
public Bank ()
{
mSimulateError = false;
}

public bool SimulateError
{
get{
return mSimulateError;
}
set{
mSimulateError = value;
}
}

[AutoComplete(true)]
public void Move (Account iAcct1, Account iAcct2, int iAmount)
{
iAcct1.Balance += iAmount;

if (mSimulateError)
throw new ApplicationException ("simulated error
to abort transaction");

iAcct2.Balance -= iAmount;

return;
}

private bool mSimulateError;
}

[Transaction(TransactionOption.Required)]
[Description ("This stores the account information")]
public class Account : ServicedComponent
{
public Account ()
{
mBalance = 0;
}
public int Balance
{
get{
// read mBalance from a DB
return mBalance;
}
set{
mBalance = value;
// write this to a DB
}
}

private int mBalance;
}
}

If a serviced component is accessed by a managed component, the .NET runtime will load the assembly and register it with COM+ services. This is possible since the component derives from ServicedComponent and the metadata contains the information that the runtime can access to do this. However, if this managed serviced component is to be accessed by an unmanaged component, the COM runtime knows nothing about serviced components and thus it cannot locate the assembly and register it automatically. For unmanaged components, you need to explicitly register the managed component with COM+ services. Not only do you need to register it, but you also need to create the COM+ application and import the component into the application. The .NET framework provides a tool just to do this. The tool is called RegSvcs and you can run it on your managed component to register it, add it to COM+, and configure the services added to your class. For our sample component above, you would run it as follows:

regsvcs SampleServicedComponent

Shared Property Manager

COM+ used a resource dispenser called the Shared Property Manager (SPM) to share state within a server process. The SPM maintains groups of properties that can be shared across different objects. Note that these objects have to be running in the same server process if they are to share state. Objects running in separate server processes cannot use the SPM. Further, the SPM takes care of protecting the properties against concurrent access by providing locks on the data. The .NET framework also provides an implementation of the SPM for serviced components that is identical to the one provided by COM+. Creating and accessing these properties and property groups is identical to the way it was done under COM+.

bool lExists;
PropertyLockMode lLockMode = PropertyLockMode.SetGet;
PropertyReleaseMode lReleaseMode = PropertyReleaseMode.Standard;
SharedPropertyGroupManager lSPM = new SharedPropertyGroupManager();

SharedPropertyGroup lGroup = lSPM.CreatePropertyGroup
("MyGroupName", ref lLockMode, ref lReleaseMode, out lExists);

lGroup.CreateProperty ("MyPropName", out lExists);
SharedProperty lProperty = lGroup.Property ("MyPropName");
lProperty.Value = 100;

The above code shows how you can instantiate a Shared Property Manager and use it to create groups and properties. Once these groups and properties are created, you can use these to share and pass data between all objects in the same server process as this object.

Delegates and Events

COM introduced the concept of event sources and event sinks which were hooked up using Connection Points. A connection point was a way in which a server could call the client under specific conditions. Essentially, connection points implemented callbacks where multiple clients could subscribe to the callback. A client could register a callback with the server, and the server would invoke the callback when it needed to alert or update the client with some information. VB 6.0 encapsulated the connection point implementation into Events that can be fired and trapped. A class can declare events that a caller of a class can write handlers for. Once the class fires the event, the handler for this event is invoked. Events can take any number of parameters which could contain the data the server wants to pass to the client.

Public Event SampleEvent()

Public Sub PerformWork()

'Do stuff
RaiseEvent SampleEvent
'Do stuff

End Sub

This is what a sample class that can raise an event looks like. It declares a public event and a public method that raises the event. Now let us look at how a client can catch this event:

Private WithEvents mEventObject As MyEventObject

Public Sub InvokeEventObject()

Set mEventObject = New MyEventObject
mEventObject.PerformWork

End Sub

Private Sub mEventObject_SampleEvent()

Debug.Print "Event handler: Event Fired"
End Sub

The client declares a member variable and uses the WithEvents keyword, and adds a method called mEventObject_SampleEvent. This is the method that will be invoked when the object fires the event. Since events can be used with any object, it is possible that an unmanaged component fires an event when a managed component calls it. In this case, the managed component should be able to handle the fired event. However, events are implemented as connection points, which is a COM interface that unmanaged components implement. Both the source and the sink need to implement this interface in order to fire and trap events. VB does this for you internally, but managed components need some extra plumbing in order to handle this. Let us look at how VB handles event classes and then we can look into how a managed object can trap unmanaged events.

If you look at the DLL that these classes belong to using OleView, you can see the type library as in the above screen shot. As you can see, the class MyEventObject contains a dispinterface __MyEventObject. This is marked as the default source interface for the MyEventObject class. The [source] attribute in the type library signifies that this interface supports connection points, also known as events. In the managed world, delegates are equivalent to events. The managed world supports events as well, but internally they are implemented as events.

If we run TlbImp on this unmanaged component, we will get a managed assembly that we can reference from our C# project. Opening this assembly using ILDasm will show us how the unmanaged types were converted to managed types.

As you can see from the above screenshot, the MyEventObject object defines an event and two methods to add and remove the event. The __MyEventObject_SampleEventEventHandler class is the delegate class that will be used to handle the unmanaged event. Note that TlbImp will generate one class for every event that is defined in the unmanaged component. The delegate is named as "UnmanagedInterfaceName_EventNameEventHandler". In our example, the interface is __MyEventObject, and the event name is SampleEvent, thus the delegate class is __MyEventObject_SampleEventEventHandler. We can now use the Add and Remove methods to add delegate handlers in our managed code. These are the methods that will be called when the unmanaged component raises the event.

static void MyManagedHandler ()
{
//Do
            stuff
return;
}

public void
            InvokeUnmanagedEventObject()
{
MyEventObject lObject = new
            MyEventObject();

lObject.SampleEvent +=
new
            __MyEventObject_SampleEventEventHandler
            (MyManagedHandler);

lObject.PerformWork ();
}

The above code shows how the managed component can add a delegate for the unmanaged event. Note that the delegates are completely type safe and a compiler error will be generated if the signature of the managed handler is not identical to the signature of the unmanaged event. Also note that although the class supports methods like AddEventSampleEvent and RemoveEventSampleEvent, you cannot directly call these functions. These are reserved methods that will give a compiler error if you try to call them directly. You should use the += and -= operators to add and remove handlers.

// the following two methods will give compiler errors
lObject.AddEventSampleEvent
(__MyEventObject_SampleEventEventHandler (MyManagedHandler));

lObject.RemoveEventSampleEvent
(__MyEventObject_SampleEventEventHandler (MyManagedHandler));

// use the following syntax to add and remove handlers

lObject.SampleEvent +=
new __MyEventObject_SampleEventEventHandler (MyManagedHandler);

lObject.SampleEvent -=
new __MyEventObject_SampleEventEventHandler (MyManagedHandler2);

Once the delegates are defined correctly, your managed component should be able to handle all unmanaged events transparently.

Download Code

Included is some sample code to download to test interoperation for yourself:

Conclusion

As we discussed in this article series, classic COM and .NET components can easily interact, and the .NET framework provides tools that make this integration easy. For the foreseeable future, COM and .NET components will have to coexist and interact peacefully. This interaction will subside over time, as companies start porting or rewriting most of their existing applications over to .NET. Developers who develop COM objects should do so with an eye towards .NET and the future, and design their components such that they can easily be ported to and integrated with .NET components.

Happy .NETing.


Web Site Error Logging and Reporting System

Mobile

Mobile (Sort by Name)

An Introduction to Mobile Programming in C#


An Introduction to Mobile Programming in C#

An Introduction to Mobile Programming in C#
http://www.csharptoday.com/content.asp?id=1578
Article Rating: 2.5
Published on September 25, 2001
Abstract
With the expansion of the web away from traditional browsers, current technology has brought the internet into mobile devices. Microsoft has constantly improved its support to such devices, and the introduction of the .NET framework has taken this support a step further. With .NET we can program for a mobile device as a normal web page and the Common Language Runtime and .NET framework will take care of rendering the data in the appropriate format, i.e. as HTML for browsers, as WML for mobile phones, etc. In this article Niranjan Kumar introduces us to programming for Mobile devices using C# and ASP.NET pages. This is illustrated with a Stock List application that accesses a SQL Server database and lists available stocks on either a Mobile device or HTML browser.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

With the expansion of the web away from traditional browsers, current technology has brought the internet into mobile devices. In this scenario, the internet can be accessed from devices like cell phones, PDA devices, etc. Microsoft has constantly improved its support for such devices, and the introduction of the .NET framework has taken this support a step further. Programming to such devices is made easy with a set of base classes and tools. With .NET you can program for a mobile device as a normal web page and the Common Language Runtime and the .NET framework will take care of rendering the data in appropriate format, i.e. as HTML for browsers, as WML for mobile phones, etc.

This article is an introduction to programming in .NET for Mobile phones using C# and ASP.NET pages. This is illustrated with a Stock List application that accesses a SQL Server database and lists available stocks on a Mobile device. Users can select a stock and view its details.

Prerequisites

  • .NET Mobile Web SDK which is available as a free download from Microsoft's msdn site (http://msdn.microsoft.com/downloads/default.asp?URL=/code/sample.asp?url=/MSDN-FILES/027/001/516/msdncompositedoc.xml&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq).
  • To run ASP.NET pages you need Microsoft Windows 2000 SP 1 installed with .NET framework and IIS 5.0 running.
  • A knowledge of WML and WAP is preferable.
  • To execute our Mobile application, we will be using a mobile simulator. I have used Openwave's UP. Simulator 4.1 that is available as a free download. It can be downloaded from http://130.75.66.2/software/Netz/WAP/default.asp?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq.

Designing our Stock Application

Database Design

We will use SQL server as our database. We will deal with two tables: Stock_Master that maintains all stock information like stock ID and company name, and the Stock_Detail table which stores stock details and contains fields stock ID, stock value and stock date. The Stock_ID field in the Stock_Detail table is the child key to the Stock_Master table.

Table: Stock_Master

Table: Stock_Detail

Front End Design

We will have two forms, the first of which will display all stock symbols in a list (see below):

On selecting a stock symbol the second form is shown with stock details as seen below:

Mobile Forms

We will design our forms in a text editor. Once the .NET Mobile SDK is installed on our Web server machine, a special class called System.Web.IU.MobileControls is added under the System namespace. This class lets us design a form for a Mobile device. A mobile server control helps developers to build mobile web applications for cell phones. These controls along with Mobile Web forms can generate WML version 1.1 and HTML 3.2 according to the client device on which it is rendered.

In order to make our form a Mobile Form, we will derive our ASP.NET page from the System.Web.IU.MobileControls.MobilePage namespace. We will also attach a prefix for our mobile server controls with a name "Mobile". The Language= attribute specifies the language used to code the page. In our case it is set to C#. See StockList.aspx for the complete source code. The following lines of code illustrates this:

<%@ Page
            Inherits="System.Web.UI.MobileControls.MobilePage" Language="C#"
            %>
<%@ Register TagPrefix="Mobile"
            Namespace="System.Web.UI.MobileControls"
            Assembly="System.Web.Mobile" %>

Since we will be accessing a SQL Server database for stock details, we need to include the System.Data and System.Data.SqlClient namespaces. The following lines of code illustrates this:

<%@ Import Namespace="System.Data" %>
<%@ Import Namespace="System.Data.SqlClient" %>

Next we write code to design our Stock List Form. This form basically lists all stocks from the Stock_Master table. We will display all stock name in a Mobile:List control. This control represents a list of items in a mobile device and it is similar to a ListBox control in Windows Forms. Items are filled in the Page_Load Event of this Form. The following code shows the code that designs our StockListForm:

<Mobile:Form runat="server"
            id="StockListForm">
<Mobile:Label id ="m_label0"
            runat="server" text="Select your stock"/>
<Mobile:List
            runat="server" id="m_lsStocks" OnItemCommand= "OnItemCommandStocks"
DataValueField="StockID" DataTextField="StockName">
</Mobile:List>
<Mobile:Label id ="m_label1"
            runat="server" />
</Mobile:Form>

Each Mobile Form is represented by a <Mobile:Form> tag. Here the word Mobile is the same as that given in the TagPrefix value. The second line in this code creates a label control that displays a standard caption "Select your stock ". Next our List control is created, where m_lsStocks is the id given to this control. All mobile controls are accessed using their id property. This is similar to the Name property of a Windows Forms control.

When an item is selected from the <Mobile:List> control, the OnItemCommand event is fired and we associate a server side function OnItemCommandStocks(). Every List item is associated with two properties, Text and Value. Every List Item is visually represented by Text property, and programmatically represented by Value property.. The DataValueField property represents the data that is associated with the Value property, whereas DataTextField represents data associated with the Text property. </Mobile:List> tag ends the declaration of the List control.

Now lets get on to filling this List control with stocks from the Stock_Master table. This is done in the Page_Load event handler. This event handler function takes two arguments, Object and EventArgs. Object represents the object that initiated the event and EventArgs represents the arguments to this event. Inside this function, we connect to a SQL Server database using the ADO.NET classes SqlConnection and SqlCommand. The following code fragment illustrates this:

/* Create connection object
            */
l_dbConnection = new SqlConnection();

/* Form
            ConnectString */
l_dbConnection.ConnectionString =
            "server=localhost;user id=sa;database=Axis;";

/* Open DB
            connection */
l_dbConnection.Open();

In the above code, the SqlConnection object is created, then its ConnectionString property is set, then the Open() method is called, which connects to database. If there are any errors while opening database connection, system raises an exception that is captured and an error message is displayed. After successfully connecting to the database, a SqlCommand object is created. This object contains our query statement to retrieve all stock data and takes SqlConnection as a parameter to its overloaded constructor.

/* Create SqlCommand object
            */
l_dbCommand = new SqlCommand("Select Stock_ID,Stock_Name From
            Stock_Master", l_dbConnection);

The command is executed using the ExecuteReader() method of SqlCommand and it returns the SqlDataReader object. This class reads data sequentially. Its Read() function advances the cursor to the next row and returns false when it encounters an end of record marker. GetString() takes a zero based column index and returns the data in string format. To retrieve data other than a string, the corresponding function should be used, or the InvalidCastException exception will be thrown. To get a list of data retrieval functions for SqlDataReader see MSDN.NET that ships along with Visual Studio.NET.

ArrayList l_arrStocks = new
            ArrayList();

l_drStocks = l_dbCommand.ExecuteReader();
for
            ( ; l_drStocks.Read() ; ){
l_strStockID =
            l_drStocks.GetString(0);
l_strStockName =
            l_drStocks.GetString(1);
l_arrStocks.Add( new
            StockMaster(l_strStockID
            ,l_strStockName));
}
l_drStocks.Close();

After retrieving data from SqlDataReader, we store it in a temporary array l_arrStocks derived from the ArrayList object. I have created a helper class StockMaster that represents the Stock_Master table. Every row returned from database, is stored as an array element in l_arrStocks object. When the for loop is terminated SqlDataReader is closed with Close(). Finally this object is bound to our <Mobile:List> control using two simple statements. The following code illustrates this:

/* Add to <Mobile:List> control */
m_lsStocks.DataSource = l_arrStocks;
m_lsStocks.DataBind();

DataSource is the property that binds our array to the <Mobile:List> control. The DataBind() method will actually populate the control with our array object l_arrStocks. Now we have completed the code for displaying all stocks in our List control. The database connection is closed in the finally block.

The next task is to display details of a stock when the user selects a particular stock item. We will display the stock details in another form called StockDetailsForm. I have created 3 Label controls and a TextView control to display stock detail data. See the following code that designs StockDetailsForm:

<Mobile:Form runat="server"
            id="StockDetailsForm" >
<Mobile:Label runat="server"
            id="m_labelHeading" Text="Stock Details">
            </Mobile:Label>
<Mobile:Label runat="server"
            id="m_labelStockID"></Mobile:Label>
<Mobile:Label
            runat="server"
            id="m_labelStockName"></Mobile:Label>
<Mobile:TextView
            runat="server"
            id="m_tvStockDetail"></Mobile:TextView>
</Mobile:Form>

When an item is selected on the list, the method OnItemCommandStocks() will be called. Recall the OnItemCommand event that is associated with OnItemCommandStocks() function in our <Mobile:List> control creation statement. We have to write code in the OnItemCommand event to display selected stock data. The first argument to this event handler function OnItemCommandStocks() is the object that initiated the event - in our case the List control. The second argument is the ListCommandEventArgs object. This object has a member called ListItem that represents the exact item on which the event has occurred. The Value and Text properties will return the Stock ID and Stock Name of the selected item respectively. So we simply get these values and assign it to a local string variables.

/* Get Selected List item Value and Text
            */
string l_strStockID = e.ListItem.Value;
string
            l_strStockName = e.ListItem.Text;

m_labelStockID.Text =
            "Ticker ID :" + l_strStockID;
m_labelStockName.Text = "Company :"
            + l_strStockName;

The last two statements in the above code fragment will display our selection in a label control. Then we connect to our SQL Server database using SqlConnection. Now we have to fetch stock details according to user's selection. We generate a SQL statement where an ORDER BY clause is used to show the latest data at the top. This SQL query is passed as input parameter to the SqlCommand constructor. The following code illustrates this:

/* Form SQL statement */
string
            l_strSQLString = "Select
            Stock_Value,convert(datetime,Stock_Date,101) From Stock_Detail Where
            ";

l_strSQLString += "Stock_ID = '" + l_strStockID + "' Order
            By Stock_Date DESC" ;

l_dbCommand = new
            SqlCommand(l_strSQLString,l_dbConnection);

The SQL statement is executed using the ExecuteReader() method of the SqlCommand class. Data is read using Read() of the SqlDataReader class. The retrieved data is stored in a string variable and displayed on the form using a TextView control.

for ( ; l_drStocks.Read() ;
            ){
l_strStockValue =
            l_drStocks.GetDecimal(0).ToString();
l_strStockDate
            =l_drStocks.GetDateTime(1).ToString();
l_strTemp +=
            l_strStockValue + "," + l_strStockDate;
l_strTemp +=
            "\n";
}
/* Set the output to <Mobile:TextView> object
            */
m_tvStockDetail.Text = l_strTemp;

TextView is a special mobile control that is capable of displaying long text, wraps it automatically and handles scrolling of data by itself. Finally our new form will be activated using the ActiveForm property.

/* Show the 2nd form */
ActiveForm = StockDetailsForm;

Now we have completed the stock list application. Enter your URL in the simulator to see our application in action.

Conclusion

In this article we learnt about Microsoft .NET support for mobile programming. We learnt how to write a simple stock application using C# and ASP, and access the same using a mobile device simulator. The page can also be viewed with IE 5.5+.


Web Site Error Logging and Reporting System

E-Commerce

E-Commerce (Sort by Name)

An In-Depth Look at the DateTimeFormatInfo Class
An In-Depth Look at the NumberFormatInfo Class
Creating an Order Tracking Application using .NET Framework
Creating Graphical Reports With Crystal Reports in .NET
Introduction of UDDI (Universal Description Discovery Integration)-A Boon to Web Services


An In-Depth Look at the NumberFormatInfo Class

An In-Depth Look at the NumberFormatInfo Class
http://www.csharptoday.com/content.asp?id=1754
Article Rating: not yet rated
Published on March 4, 2002
Abstract
NumberFormatInfo (NFI) is the class in System.Globalization that provides culture-sensitive formatting for currency, numbers, and percentages. Next to confusing short dates (is 11-08-90 the 8th of November or the 11th of August?), numeric values displayed without regard to user expectations can cause annoyance and misinterpretation. What does 2,103 really mean? Is it two thousand one hundred and three, or two and one hundred and three thousandths? In financial or medical software the answer can be crucial. In this article, Bill Hall explores the methods and properties of NFI to discover how it can help us to create displays that meet user requirements for a given world region.
Article

NumberFormatInfo (NFI) is the class in System.Globalization that provides culture-sensitive formatting for currency, numbers, and percentages. Next to confusing short dates (is 11-08-90 the 8th of November or the 11th of August?), numeric values displayed without regard to user expectations can cause annoyance and misinterpretation. What does 2,103 really mean? Is it two thousand one hundred and three, or two and one hundred and three thousandths? In financial or medical software the answer can be crucial. In this article, we will explore the methods and properties of NFI to discover how it can help us to create displays that meet user requirements for a given world region. Assisting us is a form written in C# that shows most of the properties and methods of the class. The program is interactive; you can change specific properties, including separators, grouping, signs, and precision, and view the results in any .NET-supported culture.

The implementation of NFI has its roots in two Win32 functions, GetNumberFormat and GetCurrencyFormat. Although useful and versatile, these two API's are not particularly convenient to apply, especially when you want to change a parameter from the standard locale offering. For example, users like to see an integer with many digits with group separators, as its value is less likely to be interpreted incorrectly. The number 210398398750 is hard to read, but 210,398,398,750 (or if you prefer 201.398.398.750) is much easier to sort out visually. In Win32, a straightforward application of GetNumberFormat groups the whole part correctly according to the locale setting, but always tacks on a fractional component as well. To eliminate the fractional part, the programmer has to fill out six fields in a NUMBERFMT structure by repeated calls to GetLocaleInfo using LCTYPES that correspond to the NUMBERFMT fields, and then set the fractional digits to zero. Using NFI, a simple property setting, NumberDecimalDigits = 0, does the trick.

An instance of NFI is a property of a specific culture instance of System.Globalization.CultureInfo. A specific culture is determined by both language and region. A parent culture, which references language but not region, contains only a null value for its NFI. The reason is that the features of NFI are dependent on a particular world region as well as language. For example, here are two rather different number formats from two countries bordering on each other and sharing a common language:

French (Switzerland) | 123'456'789'876'543'210.99
French (France) | 123 456 789 876 543 210,99

Before reading this article you may find it helpful to review our previous one about .NET's CultureInfo class in Internationalization With The .NET CultureInfo Class & C#. You may also want to look at An In-Depth Look at the DateTimeFormatInfo Class, as the two classes have similar uses.

Simple Number Formatting

It is easy to control the format of a number. Just use the variable's ToString method with one of the eight formatting letters given in Table 1. Not all data types will work with all formatting strings; for example, you cannot use the hex formatter with a decimal type. If in doubt, use a try-catch block.

The typical code is val.ToString("<formatstring>"); where val is a numeric data type such as double, integer, decimal, float, etc. Unfortunately, not all of the possible format strings are completely documented in the .NET 1.0 Help. For example, the percentage formatters "p" and "P" are missing. Also, there is no explanation of why lower and upper case forms are provided, although apparently the reason is to case certain letters in the output (see, for example, the scientific and hexadecimal results) in Table 1.

Type | Format string | Output (lower case) | Output (upper case)
Currency | C, c | $123,456,789.99 | $123,456,789.99
Integer | D, d | 123456789 | 123456789
Scientific | E, e | 1.234568e+008 | 1.234568E+008
Fixed point | F, f | 123456789.99 | 123456789.99
General | G, g | 123456789.987654 | 123456789.987654
Number | N, n | 123,456,789.99 | 123,456,789.99
Round Trip | R, r | 123456789.98765433 | 123456789.98765433
Percent | P, p | 98.77 % | 98.77 %
Hexadecimal | X, x | 75bcd15 | 75BCD15

Table 1: Formatting Characters for Numbers

Of these, only Currency, Number, and Percentage are controlled by NFI. For example, in the table above, the thread culture just happened to be en-US when the results were recorded. However, we can create a new culture, say Swiss (French), and then augment the formatting by referencing the CultureInfo's implicit NFI as follows:

CultureInfo ci = CultureInfo.CreateSpecificCulture(de-CH);
String result = Val.ToSTring("<character>", ci);

In this case, three of the outputs above now appear as shown in Table 2; the rest remain the same:

Currency | SFr. 123'456'789.99 | SFr. 123'456'789.99
Number | 123'456'789.99 | 123'456'789.99
Percent | 98.77% | 98.77%

Table 2: Swiss-French Number Formats

So, most of the time, you don't have to explicitly reference an NFI instance if the current thread's CultureInfo matches the user's requirements since CultureInfo itself contains an instance of NFI. However, if you want to obtain information unique to NFI or modify the output, then you will have to obtain one explicitly. This is very easy to do with the line of code NumberFormatInfo nfi = ci.NumberFormat; where ci is a specific (not parent) CultureInfo object. Now let's take a look at the sample application that accompanies this article to see what is provided.

NumberFormatInfo Details

Figure 1 shows the demo program with the culture set to Russian (Russia). The form itself is divided into three sections, reflecting the partition of properties and methods for Currency, Number, and Percentage. Within each section the properties themselves are shown. Also displayed are the resulting formats for number, currency, and percentage. At the bottom are some additional pieces of information about how to correctly render "not a number" and negative and positive infinity in the selected culture.

You should be aware that although the formats tend to be somewhat similar for currency, number, and percent, each should be regarded as a separate formatting task. Often, programmers tend to think of currency formatting as a special kind of number, but that is not the case. You cannot just drop in a currency symbol in front of a number expression and expect to have a culturally correct result. Rounding rules can be very different between currencies and numbers. Numbers can have fractional parts but the currency may have none (Japanese Yen, Indonesian Rupees). Placement of the currency symbol in relationship to the number is another issue -front or back - spaced or not - from the actual value. The problem becomes even more acute when working with negative values. Not all cultures use a minus sign, but if present, positioning becomes even more complicated when considering relative order of symbol, sign, and value. If you run through the listed cultures you can find sixteen ways of arranging debit currency and five ways for showing negative numbers. Even countries using the Euro have their own individual styles: sometimes the Euro symbol is in front (Italy, Austria) and sometimes behind the value (Portugal, Greece, Luxembourg, Germany, Spain, Finland, France).

Another interesting problem is how to separate whole part grouping of numbers. You may have noticed the row of nine NumericUpDown controls in the currency and number subareas (we could have also done the same for percentage as well). Many Indian cultures actually separate whole parts not simply by three's but twos and threes such as this example from Hindi -1,23,45,67,89,87,65,43,210.99. In NFI, you can actually create up to 9 possible group sizes and in any order. In the demo program you can experiment with the various possibilities. This feature is probably overkill, both in the class implementation in .NET as well as in the demo program, but it was probably just as easy to implement the general case as it was to meet the specific requirements of the Indian locales. It does, however, make writing the demo program harder!

Figure 1: NumberFormatInfo Demo

By the way, you may notice that for some cultures, the text may not look quite right, especially for the fields at the bottom of the form. In this case, you might want to add additional language support. You can do this using the Regional Settings dialog and your Windows installation CD. Just click on the additional language settings you need and then insert the CD when requested.

NumberFormatInfo Properties

The main properties that you can change are described next. Note they are different properties for each number, currency, and percent and are not shared. For example, you will find separate properties for the number of decimal digits used in formatting currency, number, and percentage, namely CurrencyDecimalDigits, NumberDecimalDigits, and PercentDecimalDigits, respectively. For simplicity in the description below, we have grouped them together by category and used a descriptive name for each. Not all are present for each number type.

Decimal Separator | The string separating whole and fractional parts
Group Separator | The string separating groups of whole part digits
Decimal Digits | An integer representing the number of fractional digits (0-99)
Currency Symbol | The string representing the currency symbol
Positive Sign | A string that denotes that a number is positive
Negative Sign | A string that denotes that a number is negative
Percent Symbol | A string that denotes a percentage value
Per Mille Symbol | A string that denotes a per mille (thousandth) value
Positive Pattern | A digit representing positioning of symbol, value, and spacing. Ranges are (0-3) for currencies and (0-2) for percentages. For example, the possible orderings are for credit currency are $n, n$, $ n, and n $, where $ represents the monetary string.
Negative Pattern | A digit representing possible positioning of symbol, sign, value, and spacing. Ranges are (0-15) for currencies, (0-4) for numbers, and (0-2) for percentages. For example, negative percentages could have the forms -98.77 %, -98.77%, or -%98.77.
Group Sizes | An array of 9 values, each ranging from 1 to 9, that describes how many elements to group for each whole part segment. The last element can be zero to indicate repetition of the next to last value.

Changing the Default Formats

NumberFormatInfo gives you a set of useful defaults formats. By using the properties of a suitable NFI you can modify the output. There may be many reasons for doing this. Let's give an example. Many European countries now use the Euro, but historical data might have to be rendered using a pre-Euro format. Here is an example from Portugal using the Euro.

Adjusting the decimal separator to ($) and the currency symbol to Escudos provides the older format:

We have already mentioned the example of providing a readable format to an integer just by adjusting the number of decimal digits. In this example, we have adjusted the number of decimal digits in a number format to none.

If we want to go wild with group sizes, try this one. Just for fun, we also changed the negative pattern so that the negation sign is at the back and spaced away from the value. We could have also changed the separators and signs. These examples should give you a good idea of what NFI can be made to do for you.

By the way, after you have completely messed us the settings in the demo you can reset them to the culture's defaults by using the Reset button!

Code Samples

There is absolutely nothing difficult about working with NFI. As for the demo form itself, the only complicated part is managing the Group Sizes display, and that is not particularly interesting, just tedious. The details of generating a specific CultureInfo list have already been discussed in An In-Depth Look at the DateTimeFormatInfo Class.

Here are some details about the test values used in the program and the code for rendering the currency, number, and monetary displays. Notice that we alternated use of upper and lower case formatting characters, but found no differences in results for the currently supported cultures. We also used the NFI explicitly in the ToString methods, i. E. curci.NumberFormat, but the code works the same if just curci is used. Here, curci is just a writeable copy of the currently selected specific culture.

private CultureInfo curci;
private
            decimal d = 123456789876543210.987654321M;
private decimal n =
            -123456789876543210.987654321M;
private double p =
            1234.987654321;
private double pn =
            -1234.987654321;

private void ShowCurrency()
            {
txtCurrency.Text = d.ToString("c",
            curci.NumberFormat);
txtNegativeCurrency.Text = n.ToString("C",
            curci.NumberFormat);
}

private void ShowNumber()
            {
txtNumber.Text = d.ToString("n",
            curci.NumberFormat);
txtNegativeNumber.Text = n.ToString("N",
            curci.NumberFormat);
}

private void ShowPercent()
            {
txtPercent.Text = p.ToString("p",
            curci.NumberFormat);
txtNegativePercent.Text = pn.ToString("P",
            curci.NumberFormat);
}

The rest of the code manages the events that occur when the user alters the selected culture or modifies the fields; most is as easy as setting or reading a property.

Conclusion

Respect your clients and customers preferences by making good use of the formatting (NumberFormatInfo, DateTimeFormatInfo) classes of CultureInfo. These classes will give you culturally correct results for your thread of execution almost without intervention or modification on your part. Use these classes right from the beginning of project development, and you'll be able to get to market without the wasteful and expensive step of an internationalization rewrite.

If you would like to read some background on the issues of internationalizing a product take a look at the two articles located here: http://www.mlmassociates.cc/thoughts_on_internationalization.htm?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq. For more information you should consult http://www.microsoft.com/globaldev?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq.

Article Information
Author | Bill Hall
Technical Editor | Adam Ryland
Project Manager | Helen Cuthill
Reviewers | Andrew Krowczyk, Saurabh Nandu

If you have any questions or comments about this article, please contact the technical editor.


Introduction of UDDI (Universal Description Discovery Integration) – A Boon to Web Services

Introduction of UDDI (Universal Description Discovery Integration) – A Boon to Web Services
http://www.csharptoday.com/content.asp?id=1803
Article Rating: 4.5
Published on January 21, 2002
Abstract
UDDI (Universal Description Discovery Integration) provides an XML-based specification for businesses to describe themselves and expose the Web Services provided by them. It helps business organizations discover new business partners, and opens new avenues for global transaction. If the emerging world of Web Services is to flourish, companies need an efficient and automated way to find other businesses on the Internet and determine if their systems and applications are able to work together via the Web. The UDDI uses a Public Registry data store where the organizations can register and expose the information regarding the Web Services they offer.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

This article gives a basic introduction of UDDI and explains how the interaction with the UDDI Registry can be achieved using the .NET UDDI SDK.

What's UDDI?

UDDI (Universal Description Discovery Integration) provides an XML-based specification for businesses to describe themselves and expose the Web Services provided by them. It helps business organizations in discovering new business partners, and opens new avenues for global transaction. If the emerging world of Web Services is to flourish, companies need an efficient and automated way to find other businesses on the Internet and determine if their systems and applications are able to work together via the Web. The UDDI uses a Public Registry data store where the organizations can register and expose the information regarding the Web Services they offer. In short, the UDDI helps business organizations to achieve the following:

  • Discover each other.
  • Define the mode of Interaction through Internet.
  • Sharing all the information through a global registry.

Why UDDI?

At first glance, discovering a Web Service would seem very simple. An organization having a known business partner with a known service leaves nothing to discover since all the information is readily available. However, new requirements may arise in the future, which would make the organization search for Web Services offering advanced features. Now the organization wants to know the type of services used by various organizations across the world, and identify the best service catering to their needs. What's the solution to this?

To solve this problem, a group of 280 companies joined hands and started to come up with an innovative idea of placing all the organizations exposing Web Services in a common place, where they can be easily accessed and the Information can be shared sufficiently. Thus UDDI was born!

The website http://www.uddi.org/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq provides a browser-based interface to register and describe business services and look into other registered business organization for further transactions. The goal here is to provide a clear picture of the companies that expose the Web Services, making them easily identifiable. In addition to these browser-based capabilities, the UDDI can be directly integrated with the existing business applications using SOAP (Simple Object Access Protocol) and XML technologies. For example, a programmer well versed in XML, SOAP, HTTP can interact with the UDDI Registry directly from a program.

UDDI - A Layer above XML and Soap

XML and SOAP have gone a long way in sorting out the Integration and Interoperability problems among services. The SOAP (Simple Access Object Protocol) built on XML defines a simple way to package information across boundaries. The systems corresponding to two different organizations are made compatible with each other using XML and SOAP. The UDDI acts as an extra layer above XML and SOAP wherein the systems are not only made compatible, but are also provided with an increased capacity to interact with each other and know each others capabilities. The UDDI enriches the usage of Web Services and acts as a stimulus for more and more organizations to go for it.

How the Information is Stored?

The core component of the UDDI is the UDDI Business registry. The UDDI uses an XML file to describe the business entity and its Web Services. The information required for identification of the Web Service is stored in different sub structures of the XML, which corresponds to a particular business entity. In simple words, when an organization registers itself with UDDI, it is provided with an XML schema to store all the necessary information. Let's imagine we are starting a new business organization. What is the basic information necessary to advertise the organization in the Web?

  • Name & other Information (Categorization, Key Features, etc)
  • Information regarding the Services offered by the organization (including the references and address of the Service)

The XML Schema is basically composed of the information mentioned above.

UDDI Specification

A number of companies, including Microsoft, IBM, Sun, Oracle, Compaq, Hewlett Packard, Intel, SAP, and over three hundred other companies (see http://www.uddi.org/community.html?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq for the complete list), came together to develop a specification based on open standards and non-proprietary technologies to solve the challenges discussed earlier. The UDDI specifications are nothing but a combination of XML Schema and UDDI Programmer API as shown below:

XML SCHEMA

The core information model used by UDDI registries is defined in the XML Schema. XML was chosen since it is platform neutral, and hierarchical information can be stored easily. The XML Schema also supports rich data types, and can be easily used to describe and validate information based on information model represented in schemas.

The information stored in the schema is classified as:

  • Business Information.
  • Service & Binding Information.
  • Service Specification Information.

Business Information

The topmost level of the schema corresponds to "Business Entity", which contains a small set of facts about the business, like the name and some key identifiers. This part of the schema consumes the Yellow Pages of the organization. Searches corresponding to the names, product categories, and geographical locations can be performed with respect to this part of the Schema.

The sample business information schema is shown below:

<element name = "businessEntity">
<type content = "elementOnly">
<group order = "seq">
<element ref = "discoveryURLs" minOccurs = "0" maxOccurs = "1"/>
<element ref = "name"/>
<element ref = "description" minOccurs = "0" maxOccurs = "*"/>
<element ref = "contacts" minOccurs = "0" maxOccurs = "1"/>
<element ref = "businessServices" minOccurs = "0" maxOccurs = "1"/>
<element ref = "identifierBag" minOccurs = "0" maxOccurs = "1"/>
<element ref = "categoryBag" minOccurs = "0" maxOccurs = "1"/>
</group>
<attribute name = "businessKey" minOccurs = "1" type = "string"/>
<attribute name = "operator" type = "string"/>
<attribute name = "authorizedName" type = "string"/>
</type>
</element>

Service and Binding Information

The Service information contains two layers:

  • Business Service
  • Binding Template (Service Specification)

The technical and business description Green Pages are made available in these structures, and they basically contain information regarding Web Services and their paths. The business service gives information regarding the various types and categories of Web Services available, and the binding template provides the links (address) to contact the Web Services. They contain information on how an application program can invoke the Web Service.Along with the links to the particular Web Service, additional information is required regarding the nature of data accepted by the Web Service, the protocols required, compatibility, etc. More information regarding the behavior of the Web Service and the programming interface used by the Web Service is always useful. This sort of information is obtained through pointers known as TMODEL references. Each binding template is linked to a TMODEL Reference, which contains information about the specifications of the Web Service to be invoked. The sample service information schema and the binding template schema, along with the Tmodel details, are shown below:

<element name = "businessService">
<type content = "elementOnly">
<group order = "seq">
<element ref = "name"/>
<element ref = "description" minOccurs = "0" maxOccurs = "*"/>
<element ref = "bindingTemplates"/>
<element ref = "categoryBag" minOccurs = "0" maxOccurs = "1"/>
</group>
<attribute name = "serviceKey" minOccurs = "1" type = "string"/>
<attribute name = "businessKey" type = "string"/>
</type>
</element>

<element name = "bindingTemplate">
<type content = "elementOnly">
<group order = "seq">
<element ref = "description" minOccurs = "0" maxOccurs = "*"/>
<group order = "choice">
<element ref = "accessPoint" minOccurs = "0" maxOccurs = "1"/>
<element ref = "hostingRedirector" minOccurs = "0" maxOccurs = "1"/>
</group>
<element ref = "tModelInstanceDetails"/>
</group>
<attribute name = "bindingKey" minOccurs = "1" type = "string"/>
<attribute name = "serviceKey" type = "string"/>
</type>
</element>

Programmer API

The UDDI API is classified into:

  • Inquiry API
  • Publication API

Inquiry API

The inquiry API is used to quickly locate the registration data corresponding to the Web Services, based on the Search Criteria. It provide two types of functions, find_xx and get_xx.

The find_xx API function is used to get a broad overview of registration data based on the combination of search criteria. For example, the find_business function can be used to locate summary information about a complete business registration. Similar functions are available for all other data structures of the XML, like find_service, find_binding and find_tModel. All the functions support searching by category, identity, specifications used, registered URLs, and business name. If the keys for the specific structure are known, then the complete set of information can be obtained by using the get_xx API function. For example, the get_businessDetail function can be used to get an entire business registration using one or more specific business entity identifiers (business Key values). The data returned includes all registered business service and binding template information. Similar functions are available for all other data structures of the XML, like get_servicedetail, get_bindingdetail and get_tModeldetail.

Error Handling

The inquiry API is also adequately equipped with error trapping mechanisms in case the Web Service invocation failures. Let's go through the steps followed by the programmer in invoking the Web Service, and how things get handled in case of errors during invocation. The programmer uses the web interface or API calls to retrieve the binding template corresponding to the business entity. The programmer stores the binding template for later use and retrieves the needed information for invoking the Web Services from the binding template (tmodel key). Once the specifications are known, the programmer uses the cached binding template information to invoke the corresponding Web Service at runtime. In ideal conditions, assuming that the interface considerations given in the tmodel are implemented correctly, the service would be invoked successfully. But what if a failure occurs during invocation?

Lets go through the steps of error handling:

  • Obtain the binding template information for the required Web Service through program (API call) or web Interface.
  • Cache the binding template obtained from the UDDI Registry and use it to invoke the Web Service
  • If an error occurs during invocation, retrieve the binding template information again, using the API calls (get_bindingtemplate).
  • Compare the binding template retrieved with the cached binding template. Repeat the invocation again with the new binding template.
  • If retry succeeds, replace the cached binding template with the latest binding template.
  • This mechanism of cache, retrieve again, retry & replace improves the performance considerably. This technique is called retry on failure, and prevents the retrieval of fresh copies of binding template each time a Web Service is invoked.

Publication API

The publication API is used to save and delete the registration data corresponding to the web service stored in the UDDI Registry. It contains four Save_xx functions and four Delete_xx functions, one each for the four UDDI data structures [ BusinessEntity, BusinessService, BindingTemplate and TModel ]. For example, the save_business function is used to modify an existing registration or create a new business registration.

Microsoft .NET UDDI SDK

The Microsoft UDDI SDK has been released, and has reduced the burden of interacting with the UDDI Registry. A person with very little knowledge in both XML and SOAP can use the UDDI SDK with consummate ease, and derive excellent benefits from it. A programmer just needs to be familiar with languages compatible to COM like Visual Basic, VBScript, Jscript, etc. The tougher job of managing XML / SOAP messages is taken care of by the UDDI SDK:

Let's now go through the various stages of interaction with the UDDI. They are as follows:

  1. Registering the UDDI Account.
  2. Publishing the Business and TModel Information in the UDDI Account.
  3. Searching / Discovering the Information published.
  4. Removing the Information Published.

Registration of UDDI Account

At the moment, there are three sites available, provided by Microsoft, IBM and Ariba, where the UDDI business registry account could be registered. The registrations will be replicated across all the three sites on a regular basis, thus resulting in a complete set of registered entries available to all. All these sites support a common set of APIs that will ensure the integrity and availability of the information provided. What we mean by registration here is nothing but the reservation of space in the Web for the registry entry. The main UDDI Registration sites are available at:

Microsoft: http://uddi.microsoft.com/register.aspx?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq

IBM: https://www-3.ibm.com/services/uddi/protect/home.jsp?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq

Let's now register an UDDI account through the Microsoft UDDI Test Registration Site.

Go to http://test.uddi.microsoft.com/register.aspx?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq and this will guide you through the simple three step UDDI registration as shown below:

As seen above, the UDDI Registration through Microsoft requires a Microsoft Passport account, which can be obtained at http://www.passport.com/Consumer/default.asp%20/t%20_blank?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq. After following these three steps, the UDDI account will be registered. After Registration, it's time to publish the information. The information can be published in the UDDI through the page http://uddi.microsoft.com/administer.aspx?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq.

The information can be published by clicking the corresponding links. For example, the business information can be provided by clicking the Add a New Business link, as shown below:

Since we are dealing with .NET SDK, let's find out how this information can be added to the UDDI Registry programmatically using UDDI .NET SDK. First we need to download the UDDI .NET SDK from the Microsoft Site http://msdn.microsoft.com/uddi?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq. The UDDI .NET SDK requires SQL Server 2000 or MSDE (Microsoft Data Engine) as a prerequisite for installation.

Publication in UDDI Registry

Once the UDDI Account is registered, the Information can be added in the UDDI Registry using the objects exposed by Uddi.Sdk.dll, which comes as part as .NET UDDI SDK Installation.

Before proceeding further, let's decide on what to publish in the UDDI Registry. Since we are focusing on the interaction with the UDDI registry, let's skip the Web Service creation part and concentrate more on the data publication in UDDI Registry. To get basic information regarding Web Services and how they can be created, check out the following link: http://www.webservices.org/article.php?sid=374&mode=thread&order=0&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq.

Let's now consider the following imaginary scenario: "A company XYZ has created a Web Service, which gives updated information on the weather all over the world. Let's assume the Web Service is available at the following link http://www.XYZCompany.com/Weather.asmx, and the company wants to publish the information in the UDDI Registry". We are ready to code now!

Open the C# Project and add a reference to Microsoft.Uddi.Sdk.dll (it can be found in \Program Files\Microsoft UDDI SDK\VS7 after installation), then add the following namespaces in the code. The object corresponding to these namespaces will be later used to transfer business and Tmodel details to the UDDI Registry:

using Microsoft.Uddi; using Microsoft.Uddi.Binding; using Microsoft.Uddi.Business; using Microsoft.Uddi.Service;
using Microsoft.Uddi.ServiceType;

The next step is to provide the appropriate publication URL and establish the passport authentication details. The information validated before access is provided to the UDDI Registry:

Publish.Url = " https://test.uddi.microsoft.com/publish"; Publish.User = "yourname@passport.com"; Publish.Password = "******";

The next step is to expose Tmodel Information of the Web Service. Let's assume the WSDL file for our Web Service is available at http://www.XYZCompany.com/Weather.wsdl. The SaveTModel object is used to store the Tmodel information, which will later be passed to the UDDI Registry to get a unique registration key.

SaveTModel tModel = new SaveTModel();
            tModel.TModels.Add(); tModel.TModels[0].Name = "Weather
            Report";
tModel.TModels[0].Descriptions.Add("en","Provides
            Weather Information");
tModel.TModels[0].OverviewDoc.OverviewURL
            = "http://www.XYZCompany.com/Weather.wsdl";

The TModel categorization is provided as follows. The categorization is based on various taxonomies, which can be found at http://www.uddi.org/taxonomies/Core_Taxonomy_OverviewDoc.html?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq.

// Categorisation of TModel
tModel.TModels[0].CategoryBag.Add ( "uddi-org:types", "wsdlSpec", "uuid:c1acf26d-9672-4404-9d70-39b756e62ab4" );

Finally, the information is sent to the UDDI test registry, and the UDDI registry key is returned, which can be used for further inquires.

string stModelKey = "";
try {
            TModelDetail tmDetail = tModel.Send(); stModelKey =
            tmDetail.TModels[0].TModelKey; }
catch (UddiException ue) {
            Console.WriteLine ( ue.Message ); return; }

On execution of the above code, the WSDL gets registered to UDDI and a TModelKey is generated for the particular WSDL, thus making it a unique registration. This key can be used for any further references to the UDDI with respect to the WSDL file. Additional information can be added to the existing UDDI registry based on this key. In our case, since we have registered our first Web Dervice to UDDI, we need to make a note of the TModelKey, which represents the WSDL Interface, which would later help us to add additional binding information.

Let's now provide the necessary business information using the SaveBusiness class, as shown below. The company name and description are provided, which is followed by the name of the business service (Web Service) along with a short description:

//Create Business
SaveBusiness sBusiness = new SaveBusiness ();
sBusiness.BusinessEntities.Add();
// Add Business Name and Description
sBusiness.BusinessEntities[0].Name = "XYZCompany";
sBusiness.BusinessEntities[0].Descriptions.Add("en","Software Company");

//Create BusinessService
sBusiness.BusinessEntities[0].BusinessServices.Add();
sBusiness.BusinessEntities[0].BusinessServices[0].Name = "Weather Report";
sBusiness.BusinessEntities[0].BusinessServices[0].Descriptions.
Add("en","Get the Weather Details for any City");

The service specification information is provided specifying the access point (URL) of the Web Service. Finally, the TModel registration key is provided for authentication:

//Create BindingTemplate
sBusiness.BusinessEntities[0].BusinessServices[0].BindingTemplates.Add();
sBusiness.BusinessEntities[0].BusinessServices[0].BindingTemplates[0].
Descriptions.Add("en","Get in Touch with the Weather all across the World");
sBusiness.BusinessEntities[0].BusinessServices[0].BindingTemplates[0].
AccessPoint.Text = "http://www.xyzcompany.com/Weather.asmx?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq";
sBusiness.BusinessEntities[0].BusinessServices[0].BindingTemplates[0].
AccessPoint.URLType = Microsoft.Uddi.Api.URLTypeEnum.Http;

//Create tModelInstanceInfo
sBusiness.BusinessEntities[0].BusinessServices[0].BindingTemplates[0].
TModelInstanceDetail.TModelInstanceInfos.Add();
sBusiness.BusinessEntities[0].BusinessServices[0].BindingTemplates[0].
TModelInstanceDetail.TModelInstanceInfos[0].Descriptions.
Add("en","Weather Report");

//Use tModelKey string from above
sBusiness.BusinessEntities[0].BusinessServices[0].BindingTemplates[0].
TModelInstanceDetail.TModelInstanceInfos[0].TModelKey = stModelKey;

All the details attached to the Business object are finally sent to the UDDI test registry for publication, using the business detail object as shown below:

//Send to UDDI
BusinessDetail bDetail = sBusiness.Send();

The BusinessDetail object (XML Schema) returned after publication looks like this:

<?xml version="1.0"?>
<save_business xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema"
generic="1.0" xmlns="urn:uddi-org:api">
<authInfo>Agc2DxCNGVkw1WXfz55*LdBE7sLm*wD6Lfs6*OhDxv2CjS24!wTA9N6</authInfo>
<businessEntity businessKey="">
<name>XYZCompany</name>
<description xml:lang="en">Software Company</description>
<businessServices>
<businessService serviceKey="" businessKey="">
<name>Weather Report</name>
<description xml:lang="en">Get the Weather Details for any City</description>
<bindingTemplates>
<bindingTemplate serviceKey="" bindingKey="">
<description xml:lang="en">Get in Touch with the Weather all across the World</description>
<accessPoint URLType="http">http://www.XYZCompany.com/Weather.asmx</accessPoint>
<tModelInstanceDetails>
<tModelInstanceInfo tModelKey="uuid:3953023e-001c-4433-9a6e-32288223ce9f">
<description xml:lang="en">Provides Weather Information</description>
<instanceDetails>
<overviewDoc />
</instanceDetails>
</tModelInstanceInfo>
</tModelInstanceDetails>
</bindingTemplate>
</bindingTemplates>
</businessService>
</businessServices>
</businessEntity>
</save_business>

The corresponding entries made in the UDDI Registry can be verified by accessing the test registry administration page https://test.uddi.microsoft.com/administer.aspx?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq.

The Information we had sent to the UDDI registry has been published, as you can see above. The correctness of the entries passed to the UDDI registry is not verified during the publication. It's the responsibility of the programmer to provide valid Web Service information to be stored in the Registry.

Note: The publications made in UDDI test registry are bounded by some restrictions (for each registry entry made). The complete information regarding this can be obtained at:

https://test.uddi.microsoft.com/policies/default.aspx#Publishing?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq.

Searching the UDDI Registry (Discovery)

After publishing the Web Service it's now time for other organizations to discover it. Let's find out how the published Web Service information can be discovered using .NET UDDI SDK.

Open a new C# project and add the following namespaces:

using Microsoft.Uddi;
using
            Microsoft.Uddi.Binding;
using Microsoft.Uddi.Business;
using
            Microsoft.Uddi.Service;
using Microsoft.Uddi.ServiceType;
using Microsoft.Uddi.Api;

Provide the necessary inquiry URL and use the FindBusiness object to get the business list. In the code below, let's search for the business name ("XYZCompany") that we had published earlier, and check whether it is retrieved:

Inquire.Url = "https://test.uddi.microsoft.com/publish";
FindBusiness fBusiness = new FindBusiness();
fBusiness.Name = "XYZCompany";
BusinessList bInfos = fBusiness.Send();

When the above code is executed, the following XML is passed to the UDDI registry.

<?xml version="1.0"?>
<find_business xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema"
generic="1.0" xmlns="urn:uddi-org:api">
<name>XYZCompany</name>
</find_business>

The business object is returned, which allows us to find the company (name, description) and business service (name, key) details.

foreach( BusinessInfo bInfo in
            bInfos.BusinessInfos)
{
Console.WriteLine("Business Name : " +
            bInfo.Name);
foreach( Description desc in bInfo.Descriptions
            )
{
Console.WriteLine("Business Description : " +
            desc.Text);
foreach( ServiceInfo sInfo in bInfo.ServiceInfos
            )
{
Console.WriteLine("Service Name : " +
            sInfo.Name);
Console.WriteLine("Servive Key : " +
            sInfo.ServiceKey);
}
}
}

We are at the halfway stage now. The service key obtained earlier can be used to retrieve the service specification and Tmodel details. The GetServiceDetail object is used for this purpose:

GetServiceDetail getsdetail = new
            GetServiceDetail();
getsdetail.ServiceKeys.Add(sInfo.ServiceKey);
ServiceDetail
            sdetail = getsdetail.Send();

When the above code is executed, the following XML is passed to the UDDI Registry, along with the service key:

<get_serviceDetail xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:xsd="http://www.w3.org/2001/XMLSchema" generic="1.0" xmlns="urn:uddi-org:api">
<serviceKey>83a06449-b38c-4e4f-8fc7-8dd621f93beb</serviceKey>
</get_serviceDetail>

The ServiceDetail object is returned, which allows us to obtain the binding template and TModelInstance information:

foreach( BusinessService service in
            sdetail.BusinessServices )
{
foreach( Description sDesc in
            service.Descriptions )
{
Console.WriteLine("Service
            Description : " + sDesc.Text);
foreach( BindingTemplate bTemplate
            in service.BindingTemplates )
{
foreach( Description bDesc in
            bTemplate.Descriptions )
{
Console.WriteLine("Binding Template
            Description : " + bDesc.Text);
Console.WriteLine("Access Point :
            " + bTemplate.AccessPoint.Text);
foreach( TModelInstanceInfo
            tmiInfo in bTemplate.TModelInstanceDetail.TModelInstanceInfos
            )
{
Console.WriteLine("TModel Description : " +
            tmiInfo.TModelKey);
}
}
}
}
}

We have now found how the business details can be easily retrieved from the UDDI Registry using the .NET UDDI SDK objects, following a simple drill down approach. This technique can be further extended for any combinations of search criteria, for example: the complete schema Info can be retrieved based on the information on any part of the schema.

Removing Entries from UDDI Registry

Let's now find out how the entries made earlier can be deleted from the UDDI Registry through code. First provide the necessary authentication:

Publish.Url = " https://test.uddi.microsoft.com/publish";
Publish.User = "yourname@passport.com"; Publish.Password =
            "******";

Retrieve the complete details of all the entries available in the UDDI account using the GetRegisteredinfo object:

// Call get_registeredInfo to get all entries for this UDDI account
GetRegisteredInfo getRegInfo = new GetRegisteredInfo();
RegisteredInfo RegInfo = getRegInfo.Send();

Use the DeleteBusiness object to delete the available business entries in the UDDI Account:

DeleteBusiness delBusiness = new
            DeleteBusiness();
foreach( BusinessInfo bInfo in
            RegInfo.BusinessInfos )
{
delBusiness.BusinessKeys.Add(
            bInfo.BusinessKey );
}
DispositionReport dispRep =
            delBusiness.Send();

Similarly, the Tmodel entries can be deleted using the DeleteTModel object:

DeleteTModel delTModel = new
            DeleteTModel();
foreach( TModelInfo tInfo in
            RegInfo.TModelInfos)
{
delTModel.TModelKeys.Add(
            tInfo.TModelKey );
}
DispositionReport dispRep =
            delTModel.Send();

After executing the above code, all the entries added to the UDDI account will be deleted. Unlike the business entries, the TModel entries are not physically removed from the UDDI account. The TModel entries, once created, are permanently available. On deletion, the TModel key is made invisible when a search operation is performed, and can still be retrieved using the GetRegisteredInfo object.

Next Version

The second version of the UDDI specification has been released with the following enhancements:

  • Description of complex organizations with their hierarchical structure. (department, sub department, etc)
  • Flexibility to describe the business in multiple languages - internationalization.
  • Flexible search option with more query parameters.
  • Additional categorization and identification schemes during registration.

UDDI Business Registry v2 beta implementations can be found at the following sites:

Microsoft: https://uddi.rte.microsoft.com/register.aspx?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq

IBM: https://www.ibm.com/services/uddi/v2beta/protect/registry.html?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq

SAP: http://udditest.sap.com/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq

Hewlett Packard Company: http://uddi.hp.com/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq

Conclusion

With the advent of more advanced features, the UDDI is gaining large-scale popularity by enabling companies to quickly identify each other, and continue their transactions. UDDI is the right place to be if developers start thinking about developing applications as services.

Article Information
Technical Editor | Adam Ryland
Author | Vaidyaraman S
Author Agent | Charlotte Smith
Project Manager | Helen Cuthill
Reviewers | Saurabh Nandu

If you have any questions or comments about this article, please contact the technical editor.


Creating an Order Tracking Application using .NET Framework

Creating an Order Tracking Application using .NET Framework
http://www.csharptoday.com/content.asp?id=1598
Article Rating: 4
Published on June 28, 2001
Abstract
With an ever–increasing amount of business being done on the web, it becomes a challenge for organizations to determine how to best take advantage of the enormous new business opportunities that exist on the Web. The Microsoft .NET framework provides an ideal platform for the Internet and exposes a unified and simplified programming model for developers to create rich Next Generation web applications. It also provides all the underlying plumbing that is required for implementing asynchronous but still reliable solutions for the internet thereby improving developers' productivity. In this article, Thiru Thanganathiram demonstrates how some of the new features of .NET framework can be used to create an asynchronous and reliable Order Tracking mechanism for a real time E–commerce website.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

With an ever-increasing amount of business being done on the web, it becomes a challenge for organizations to determine how to best take advantage of the enormous new business opportunities that exist on the Web. The Microsoft .NET framework provides an ideal platform for the Internet and exposes a unified and simplified programming model for developers to create rich Next Generation web applications. It also provides all the underlying plumbing that is required for implementing asynchronous but still reliable solutions for the internet thereby improving developers' productivity. In this article, we will see how some of the new features of .NET framework can be used to create an asynchronous and reliable Order Tracking mechanism for a real time E-commerce website.

Business Scenario

In this article, we will implement an asynchronous Order Tracking mechanism for a fictitious Web site. The simple web site used for the demonstration consists of 3 web pages.

  • The first page shows the list of books available in the site.
  • When the user clicks a book, the user is taken to the page where the user enters all the details required for submitting the order.
  • Once the user submits the details, the confirmation page is shown.

It is assumed that the user, who places the orders through the site, already has an account number assigned to him at the time of the registration.

If we were to handle the order submission in a synchronous way, the user would have to wait till the order information is stored in the database because the control will not be returned to the application till the changes are committed. This might result in increase in the response time for the resultant page to be drawn on the user's browser.

Since we are going to handle the order submission in asynchronous fashion, the control will be returned to the application as soon as the order details are captured in an xml file. After that, a separate set of operations will be used to carry out the tasks such as saving the details to the database and sending email to the user. This kind of handling not only helps in increasing the performance but also provides the users with the best possible browsing experience.

How the Asynchronous Order Workflow works

When the user places the order in the site, a number of independently running operations are used to complete the order request.

  • The user navigates to the ASP.NET web page (Web form) and places the order by entering all the details
  • The submitted request is processed by an ASP.NET web page that converts the Order information into an .xml file and stores it in a specified directory. Then it gives the message to the user that the Order Information has been submitted.
  • This specified directory is already set up for monitoring for creation of new files by the FileSystemWatcher component, which is placed in a Windows Service Application that always runs as a service.
  • When an .xml file is created in that directory, an event is raised to the Windows Service application that passes the path of the file to a C# Transacted object that runs in Component Services in Windows 2000.
  • C# object takes the path of the .xml file, parses the file and stores the details in a table called Orders in the SQL Server database and it also sends an Email to the user with the order number indicating that the order has been submitted.

Before we start looking at the code, let me briefly introduce some of the new .Net features such as Windows Service, FileSystemWatcher and .NET Framework XML API that we will be using in this application.

Windows Services

Prior to Visual Studio.Net, if you wanted to write a windows service application you either had to use the template provided by ATL, or embed custom NT service controls in VB to achieve the same functionality. But with Visual Studio.Net, you can easily create an application that has the capability to run as a Service. Visual Studio.Net is supplied with a new project template called Windows Service that provides all the plumbing required for creating the application that can run as a Service. When you create a Visual Studio.Net project as a Service, you can write code to respond to the actions that should happen when the service is started, paused, resumed and stopped.

Once you create the service, it has to be installed using the command line InstallUtil.exe utility which is used to install the service in the local computer. Once we install the service, we can start, stop, pause and resume it using Service Control Manager.

FileSystemWatcher Component

One of the exciting new features of the .NET framework is the ability to create applications that can interact with file system and raise events at scheduled intervals based on the activities in the file System. The FileSystemWatcher component that is contained in the System.IO namespace exposes methods and properties through which we can watch for changes to directories and files and take specific actions when these changes occur.

The FileSystemWatcher component can be created in two different ways:

  • You can drag an instance of the FileSystemWatcher component from the Components tab of the Toolbox to the designer.
  • You can create an instance of the FileSystemWatcher component programmatically after setting reference to the System.IO assembly.

More information on FileSystemWatcher component can be found in the following MSDN link.
http://msdn.microsoft.com/library/dotnet/cpguide/cpconconfiguringfilesystemwatchercomponentinstances.htm?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq

.NET Framework XML API

The .NET Framework provides a set of classes for developers to create applications based on xml. These xml classes are contained in the System.XML namespace. The methods and properties of the classes present in the System.XML namespace are similar to its predecessor, the MSXML parser, which it is designed to eventually replace.

All the .Net framework xml classes are derived from two abstract classes called XmlReader and XmlWriter:

  • XmlReader provides a fast, forward-only, read-only cursor for reading an xml document and
  • XmlWriter provides a mechanism for producing xml document streams that conformstreams.

The output produced by these classes conforms to the W3C recommendations. In this article, we will be using XmlTextWriter and XmlTextReader for writing into and reading from an xml file.

For more information on Xml classes in .NET Framework, please take a look at the following article from MSDN:
http://msdn.microsoft.com/library/periodic/period01/xml0101.htm?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq.

Implementation of the System

We will split the whole application into three different parts:

  • User Interface Tier (ASP.NET Web Forms) - Created using Visual C#.
  • Windows Service Application - Created using Visual Basic.Net.
  • C# Transacted Class that runs in Component Services in Windows 2000.

For this application we use a table called Orders to store the details of the order.

Implementation of User Interface Tier (ASP.NET Web Forms)

The Web Application consists of 3 different web forms. To create a Web Application in Visual C#, select File->New Project and then select Visual C# Projects from the Project Types and select Web Application from the List of templates.

Once you create the project, right click on the project name in Solution Explorer to add reference to the System.Xml assembly that is required for making use of the .Net Framework Xml classes. We use the xml classes for creating an .xml file on the fly when the user places an order in the site.

The first page BookList.aspx shows the list of books available in the site for purchasing.

The code for BookList.aspx is available for download and it is simple except it uses one of the new server side controls called <asp;hyperlink runat=server> control to navigate to another page when the user clicks on the hyperlink of the name of the book. QueryString parameters are also passed along the way to the GetOrders.aspx.

For example, the <asp:hyperlink> control for the book name Professional ASP.NET looks like this:

<asp:hyperlink navigateurl="GetOrders.aspx?ISBN=1861004885&Name=Professional ASP.NET"
id="lnkProfASP" runat="Server">Professional ASP.NET
</asp:hyperlink>

The navigateurl property identifies the path of the url to be navigated to when the user clicks on the hyperlink.

When the user clicks on a particular book, the user is taken to the web form GetOrders.aspx where he can place an order for that book by entering the account number, (which is assigned to every user who purchases books through this site), Email address and all the necessary details.

The screenshot of the GetOrders.aspx page looks similar to this:

Lets dissect the code of GetOrders.cs which is the code-behind file for the GetOrders.aspx page:

using System;
using System.Collections;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Web;
using System.Web.SessionState;
using System.Web.UI;
using System.Web.UI.WebControls;
using System.Web.UI.HtmlControls;

In this line, we import the System.Xml assembly into our application.

using System.Xml;

In the Page_Load event, we capture the QueryString parameters from the Params collection that are passed from the BookList.aspx and assign them to appropriate controls.

protected void Page_Load(object sender, EventArgs e)
{
//The following line returns true when the browser hits the page for the first time.
if (!IsPostBack)
{

lblBookTitle.Text = Request.Params["Name"];
lblISBN.Text = Request.Params["ISBN"];

}

}

Whenever the user enters all the details and clicks the Place Order button, the following lines of code are executed in the server side.

public void btnPlaceOrder_Click (object sender, System.EventArgs e)
{

XmlTextWriter objWriter = null;
string strDocument;

Here, we enclose all the executable statements in a try..catch block to handle any exceptions that may occur during the execution of those statements.

try
{

This line of code initializes a new instance of the Guid class and obtains the new Guid (Globally Unique Identifier) by invoking the ToString method of that class. Every time a customer places an order, an uniquely generated Guid is used to name the xml file.

strDocument = System.Guid.NewGuid().ToString() + ".xml";

We create the .xml file on the C:\Orders directory with the name of the file being the value contained in the variable strDocument.

objWriter = new XmlTextWriter("c:\\Orders\\" + strDocument, null);

To format the XML Document, we assign the Indented value to the Formatting property of the XmlTextWriter. When this option is set, child elements are indented using Indentation and IndentChar properties. IndentChar property decides which character to use for Indenting and Indentation property determines how many IndentChars to write for each level in the hierarchy.

Here we use the default values for Indentation and IndentChar, which are 2 and space respectively.

objWriter.Formatting = Formatting.Indented;

We use the StartDocument to write out the XML declaration with the version and the standalone attributes. The version attribute will be set to "1.0" and the standalone will have the value "no" as we pass false as an argument to the WriteStartDocument method.

objWriter.WriteStartDocument(false);

In this line, we write a Comment to indicate that the Order Tracking Application has created this file.

objWriter.WriteComment("This file represents order placed by the user");

Here we write out the specified start tag with the name of the element as order.

objWriter.WriteStartElement("order", null);

We write all the following elements as Child Elements of the order element and these elements are all of type string.

objWriter.WriteElementString("firstname", txtFirstName.Text);
objWriter.WriteElementString("lastname", txtLastName.Text);
objWriter.WriteElementString("email", txtEMail.Text);
objWriter.WriteElementString("accountnumber", txtAccountNumber.Text);
objWriter.WriteElementString("isbn",lblISBN.Text);

Then we close the order element by using the WriteEndElement method.

objWriter.WriteEndElement();
//Write the XML to file and close the objWriter
objWriter.Flush();
objWriter.Close();

The xml file that was created dynamically looks like this:

After we write the file, we transfer the control to the ShowResult.aspx that is used to show the final confirmation page.

Server.Transfer("ShowResult.aspx?BookTitle=" + lblBookTitle.Text + "&ISBN=" + lblISBN.Text);

}
catch (Exception ex)
{

We throw any errors that may occur during the execution of the above statements.

throw ex;
}

The statements in finally block are always executed regardless of whether the statements in the try block are executed successfully or not.

      finally
{

if (objWriter !=
            null)
objWriter.Close();
}
}

Once the user enters the details and submits the order, the user is redirected to the final confirmation page that looks something similar to this.

Implementation of Windows Service Application

To create a Windows Service application in Visual Basic.NET, select File->New Project and then select Visual Basic Projects from the Project Types and select Windows Service from the List of templates.

Once you create a new project, go to the properties of the Default Service project called WindowsFileSystemService, open up the default service (Service1) that is created and rename it to FileSystemService using its properties window. You can also set properties like CanStop and CanShutdown through the Properties window. These properties can also be set programmatically and these settings determine what methods can be called on your service. For example, when the CanStop property is set to true, the OnStop method can be called and the code that is associated with that method will be executed.

Let us briefly explore some of the theory behind the Windows Service applications.

The two main classes involved in Windows Service creation are:

  • System.ServiceProcess.ServiceBase We can override methods of this class when creating a service and define the code to determine how the service functions . The ServiceBase class exposes the following methods and we can override these methods are overridden to determine what happens when we change the state of the service (like starting, stopping, etc) in the Services Control Manager.

We can write code in the following events depending on the requirements.

  • OnStart - Invoked when the service is started.
  • OnPause - To indicate what should happen when the service is paused.
  • OnStop - To indicate the behavior of the service when the service stops running.
  • OnContinue - To decide the behavior that should happen when the service resumes normal functioning after being paused for a while.
  • OnShutDown - To indicate what should happen just prior to system shutdown.

The Run method of the ServiceBase class is the main entry point for the service. When we create a service using the Windows Service template in Visual Studio.Net, it automatically inserts a line in the Sub Main method to run the service. Let's quickly look at the code of this method:

Shared Sub Main()

Here, an Array of type ServiceBase is declared:

Dim ServicesToRun() As System.ServiceProcess.ServiceBase

All the Services contained by the application can be added and run together. Since we are creating only one service, we add only the UserService1 to the application.

ServicesToRun = New System.ServiceProcess.ServiceBase() {New UserService1()}

Finally, the Run method of ServiceBase is invoked passing in the array of Services created in the previous step as an argument.

System.ServiceProcess.ServiceBase.Run(ServicesToRun)
End Sub

  • System.ServiceProcess.ServiceProcessInstaller and System.ServiceProcess.ServiceInstaller These classes are used to install and uninstall the service. ServiceProcessInstaller is used when the Windows Service project contains more than one service. To add these classes to your Windows Service right click on the Design Editor of your service and select Add Installer from the Context menu and Visual Studio.Net automatically adds these two classes to the project and allows you to install or uninstall the Service by using InstallUtil.exe utility.

After we set all these options, we have to add a reference (By right clicking on the project name in the Solution Explorer) to the C# Class Library Application which we will be creating in the later part of this article.

We add this line of code to import our C# Class Library namespace into the application:

Imports WinFileSystemServiceComponent

Public Class UserService1
Inherits System.ServiceProcess.ServiceBase

The WithEvents keyword specifies that the variable mobjFileSystemWatcher that will be used to handle the events raised by the FileSystemWatcher component.

Private WithEvents mobjFileSystemWatcher As FileSystemWatcher

All the variables declared with the keyword WithEvents should be declared at the module level. As soon as you declare a variable with the keyword WithEvents, the variable mobjFileSystemWatcher appears in the left-hand drop down of the Code Editor with the right hand side drop down having all the event procedures of FileSystemWatcher class. From the right hand side drop down, you can select either the Created, Deleted, Renamed or Changed events and write code to handle these actions.

Here we create an instance of the Orders class and call the AddOrder method passing in the path the file that is created as an argument.

Public Sub mobjFileSystemWatcher_Created(ByVal sender As Object,
ByVal e As System.IO.FileSystemEventArgs)
Handles mobjFileSystemWatcher.Created

Dim objOrders As New Orders()
objOrders.AddOrder(e.FullPath)
End Sub

Code in the OnStart method is executed when the Service is started from the Service Control Manager.

Here we set all the properties of the FileSystemWatcher object so that it starts monitoring the Orders directory for creation of new .xml files.

Protected Overrides Sub OnStart(ByVal args() As String)

mobjFileSystemWatcher = New FileSystemWatcher()

We set the path to indicate that we want to monitor the Orders directory for new .xml files.

mobjFileSystemWatcher.Path = "C:\Orders\"
mobjFileSystemWatcher.Filter = "*.xml"

WatcherTarget enumeration is used to indicate that we want to watch for a file, directory, or both files and directories. But only one member in the enumeration can be used at a time.

Here, we set the value to File since we want to monitor the directory for the creation of new Files.

mobjFileSystemWatcher.Target = WatcherTarget.File

As we are interested in watching only for files not directories or subdirectories, we set the IncludeSubdirectories property to false.

mobjFileSystemWatcher.IncludeSubdirectories = False

The members of the ChangedFilters enumeration can be combined to watch for more than one type of change at a time. For example, you can watch for any changes in the size of a file or folder, and for any changes in the LastWrite time. This raises an event whenever there is a change in the file or folder size, or a change in the LastWrite time of the file or folder.

mobjFileSystemWatcher.ChangedFilter = ChangedFilters.LastWrite

Finally, we enable the component to begin watching for the creation of new files.

mobjFileSystemWatcher.Enabled = True
End Sub

Now that we have created the Windows Service application, it has to be installed using InstallUtil.exe utility. We need to navigate to the directory path that contains the WindowsFileSystemService.exe file and then execute the following command from the command prompt.

InstallUtil WindowsFileSystemService.exe

To uninstall the existing service, we add the switch /u to the above command.

InstallUtil /u WindowsFileSystemService.exe

Once we install the service, go to Start ->Control Panel -> Administrative Tools -> Services and then select the FileSystemService to start, stop or pause or resume the service.

Implementation of C# Transacted Class that runs in Component Services

To create a Visual C# Class library, select Visual C# from the Project Types in the left hand side and select Class Library from the list of project templates from the right side.

Once we create the project, we set reference to the following assemblies: System.IO, System.Xml, System.Data and System.Web.

The code for the C# Transacted class is available for download along with the support material for this article and it goes like this:

using System;

We import the Microsoft.ComServices namespace to leverage COM+ Services like Object Pooling and COM+ Transactions. The .NET Framework provides a set of new classes that make COM+ Programming a whole lot easier. It also makes XCOPY Deployment possible, i.e.; if you want your component to run in a Transaction, you simply add a Transaction attribute to the top of the class, compile the component and copy the compiled component into the Bin Directory of the Client application.

When the object is activated for the first time from a .NET Application (Managed Code), it will automatically be registered with COM+ for you. However, if you want to create a new instance of the class from UnManaged Code, then the registration in the Component Services has to be done manually.

using Microsoft.ComServices;

The System.Data namespace contains all the ADO.NET classes required for accessing the backend database. For an introduction to ADO.NET, please refer to the article ADO.NET for the ADO Programmer.

using System.Data;
using System.Data.SQL;

The System.Web.Util namespace contains an SmtpMail class that is used for describing properties of an Email message and also for sending the Email.

using System.Web.Util;

The System.Xml is the overall namespace for all the XML Classes that provide universally accepted standards based support for XML Processing.

using System.Xml;

Next we specify the COM+ Application's name as it appears in the COM+ catalog:

[assembly: ApplicationName("OrderTracking")]

We define the namespace for this application as WinFileSystemServiceComponent

namespace WinFileSystemServiceComponent

The Transaction attribute is used to set the type of automatic transaction requested by the component. TransactionEnum can take any one of these values: Ignored,

None, Supported, Required and RequiresNew. For more information on Transaction Attributes in COM+, Please refer to Windows 2000 Brings Significant Refinements to the COM(+) Programming Model by Don Box.

We set the Transaction Attribute to Required which indicates that if the caller of this object runs in a Transaction, it will be inherited by this object, otherwise it will create a new Transaction for itself and run within the scope of that transaction.

[Transaction(TransactionOption.Required)]

Since the Orders class derives from the ServicedComponent class, it is automatically serviced by Microsoft Windows 2000 Component Services (COM+ Services). Serviced Components are generally hosted in COM+ Services as an In-Process Server (i.e.; as a Library Application).

public class Orders : ServicedComponent
{

The AutoComplete attribute indicates that the transaction governing the object will automatically call SetComplet() should the method call return successfully. If the method call throws an Exception, the transaction will be aborted.

[AutoComplete]

The AddOrder method takes the path of the XML File to be processed as an argument.

public String AddOrder(string strPath)

{

XmlTextReader objReader;
XmlDocument objDocument;
XmlNode objOrderNode;
SQLParameter objParam ;
SQLCommand objCommand;
SQLParameter objParam;
MailMessage objMailMessage;

Now, we create an instance of the SQLConnection Object passing the ConnectionString as an argument.

SQLConnection objConnection =
new SQLConnection("server=localhost;uid=sa;pwd=;Initial Catalog=Home");

We enclose the following statements in a try..catch block to catch any errors that may occur during the execution of those statements.

      try
{
objReader
            = new XmlTextReader(strPath);
objDocument = new
            XmlDocument();
objDocument.Load(objReader);

The DocumentElement property of the XmlDocument is used to get reference to the root XmlElement of the Document.

objOrderNode = objDocument.DocumentElement;

Get the values of the First name, Last name, Email, Account Number and ISBN from the Child Nodes of the Order node.

strFirstName = objOrderNode.ChildNodes.Item(0).InnerText;
strLastName = objOrderNode.ChildNodes.Item(1).InnerText;
strEMail = objOrderNode.ChildNodes.Item(2).InnerText;

The FromString method of Int32 returns a 32 bit signed integer that is equivalent to the number specified in the String.

intAccountNumber = Int32.FromString(objOrderNode.ChildNodes.Item(3).InnerText);

strISBN = objOrderNode.ChildNodes.Item(4).InnerText;

We assign the name of the Stored procedure to be executed to a local variable. This stored procedure inserts a record into the Orders table.

strCommandName = "spInsertOrder";

This line instantiates a SQLCommand object passing the Name of the Stored procedure to be executed and the SQLConnection object that was created in the previous step.

SQLCommand objCommand = new SQLCommand(strCommandName, objConnection);

By setting the CommandType property of the objCommand object, we indicate that we want to execute a Stored Procedure.

objCommand.CommandType = CommandType.StoredProcedure;

The SQLParameter object represents a parameter to a SQLCommand. Here, we create an instance of SQLParameter passing in the arguments that describe the parameter being passed to the stored procedure.

objParam = objCommand.Parameters.Add(new SQLParameter("@ISBN", SQLDataType.VarChar, 50));

We set this Direction property to specify that it is an input only parameter. Other values that can be assigned are output only, bidirectional, or a return value parameter.

objParam.Direction = ParameterDirection.Input;
objParam.Value = strISBN;

We follow the same procedure for all the parameters except for the final one. This is used as an Output parameter to return the newly added Order Number from the Stored Procedure.

objParam = objCommand.Parameters.Add(new SQLParameter("@OrderNumber", SQLDataType.Int));

objParam.Direction = ParameterDirection.Output;

Open the ActiveConnection to the Database.

objCommand.ActiveConnection.Open();

ExecuteNonQuery of the SQLCommand object is used to execute a command against an ActiveConnection, and does not return any rows against the ActiveConnection.

objCommand.ExecuteNonQuery();

Once the Stored Procedure is executed, we retrieve the return value of the Stored Procedure from the Parameters collection of the SQLCommand object.

intOrderNumber = Int32.FromString((objCommand.Parameters["@OrderNumber"].Value.ToString()));

Here, we create an instance of the MailMessage object that is part of the System.Web.Util namespace to send Emails.

objMailMessage = new MailMessage();

Now, we set the properties of the MailMessage object.

objMailMessage.To = strEMail;
objMailMessage.From = "Orders@OrderTracking.com";
objMailMessage.Subject = "Order Details";
objMailMessage.BodyFormat = MailFormat.Html;
objMailMessage.Body = "<html><body>Your Order has been Recieved </b> Your Order Number is :"
+ intOrderNumber.ToString() + " </body></html>";

Finally, we send the mail the using the Send method of the SmtpMail using the values set in the MailMessage object.

SmtpMail.Send(objMailMessage);

}

If any errors occur during the execution of the above statements, we raise the exception back to the caller.

catch (Exception ex)
{
throw ex;
}

The statements in finally are always executed regardless of whether the statements in the try block are executed successfully or not. Here we inspect the state property of the SQLConnection to check whether the connection is open. If the Connection is open, it is closed.

finally
{

if ( objConnection.State
            == DBObjectState.Open
            )
objConnection.Close();
}

}

When this C# Transacted class is instantiated by the Windows Service application for the first time, it is automatically registered with Component Services Explorer.

To see how the COM+ auto-registration mechanism worked, open up the Component Services from Start->Settings->Control Panel->Administrative Tools. If we open up the COM+ Applications folder, we see the presence of the OrderTracking application. This was created in response to the Assembly attribute we specified in the previous step. You can also verify the Transaction attribute of the Orders class, which will have the value set to Required due to the TransactionOption enum value we set programmatically.

For more information on creating .NET components for Component Services in Windows 2000, please take a look at the following MSDN article. Microsoft .NET Framework Component Services, Part 1

Putting it all together

Now that we have put together the different parts of the application, let us test the application by navigating to the BookList.aspx file that is present in the OrderTracking directory. When you click on particular book, you will be taken to the GetOrders.aspx page where you enter all the order details. When you submit the order, an .xml file will be created on the fly in the C:\Orders directory and you will get the confirmation message that the order has been submitted. This dynamically generated file will be captured by the Windows Service application which in turn will invoke the method of the C# Transacted class to process the order.

Conclusion

In this article, we have seen the following:

  • How to create a .NET Application that can run as a Windows Service using Visual Studio.Net.
  • How to use the rich .NET XML classes available in System.XML namespace to write to an .xml file and read from an .xml file.
  • How to use the FileSystemWatcher component to monitor a directory or file for changes.

Although the application we have created was simple in functionality, it should provide a solid foundation for understanding how .NET platform can be leveraged to build reliable, flexible and scalable 24x7 internet applications in an asynchronous fashion.


Web Site Error Logging and Reporting System

Other Tech

Other Tech (Sort by Name)

.NET Application Domains
.NET Code Generation with Visual Studio .NET Enterprise Architect
.NET Remoting Contexts
.NET Remoting Interception
5.Visual J# .NET - a Technical and Political Introduction
A .NET Data Provider
A C# Web Spider written as a Web Service
A complete text-printing framework using GDI+
A POP3 Client and C#
Adding Scripting to C# Windows Forms Applications

An in-depth look at Remoting
An In-Depth Look at the DateTimeFormatInfo Class
An In-Depth Look at the NumberFormatInfo Class
An Introduction to GDI+
An Open Invocation
Automated Generation of Table Columns, Class Properties, and Form Field Mapping with C# Reflection
Build a Better Design Tool with Visio Automation
Building a Forms Generator using Code DOM
Building an Interface to Legacy Data Using BizTalk Framework 2.0
C# and Java Remoting Part 1 - a C# Remoting Primer

C# to UML Reverse Engineering
COM Enabling Your .NET Components
COM+ and .NET Security Interoperation
COM+ Transactions in Managed Environment
Creating a .NET Windows Installer - Part 1
Creating a .NET Windows Installer - Part 2
Creating International-Ready Applications
Custom Controls and Design-Time Support - Part 1
Custom Controls and Design-Time Support - Part 2
Data Transformation Engine

Design Patterns in C# The Decorator Pattern
Dynamic Assemblies in .NET framework
Exploring the System.IO namespace
Extending Application Functionality
Generating AIM Buddy List import file with Active Directory and C#
Generating Design Patterns Using CodeDOM
Graphical Representation of Data Grids Using C# and GDI+
Improved Service Control Manager (SCM) - Part 1
 Improved Service Control Manager (SCM) - Part 2
Improved Service Control Manager (SCM) - Part 3

Interoperating Java, VB6, and .NET Web Services
Introducing .NET Regular Expressions with C#
Introducing Smart Device Extensions and the .NET Compact Framework
Introduction to Managed C++
Java stream Vs C# Streams
Mail Tracking and Analysis System
Manipulating Binary File Data
Migrating a Windows DNA Application to .NET Architecture
Migrating from C++ to C#
Monitoring IIS Systems

Multi-threading In C#
Network Programming in C#
Object-Relational Mapping with Reflection and Design Patterns
Programming Internet Explorer in C#
Programming the GXA
Protecting Your Code-Reflection, Obfuscation and Encryption - Part 1
Protecting Your Code-Reflection, Obfuscation and Encryption - Part 2
Runtime Reverse Engineering with System.Diagnostics
String and File handling base classes of the .Net framework
Synchronized Collections An Alternative Implementation for Many-to-Many Relationships

Syntax highlighting with regular expressions - Part 1
Syntax highlighting with regular expressions - Part 2
The .NET Property Grid Control
timeTrack An ASP.NET Contract Management System
Windows Management using the .NET Framework
Word Encryption Add-In
Writing Serviced Components in C#
XML Documentation Using C# Compilers


Build a Better Design Tool with Visio Automation

Build a Better Design Tool with Visio Automation
http://www.csharptoday.com/content.asp?id=2051
Article Rating: not yet rated
Published on December 16, 2002
Abstract
In this case study, Tony Loton shows how to write a simple VBA macro or a fully-blown C# / .NET application to extract meaningful information from design diagrams in Microsoft Visio. If you’re a little frustrated by the limited functionality of Visio as compared with some other design tools, why not extend the functionality yourself and thereby build a better design tool? The focus is on UML diagrams, but is not limited to that notation. If you’re working with other diagram types ?like Web Site diagrams or Enterprise Application diagrams ?you may well find something in here for you.
Case Study

In this case study, Tony Loton shows how to write a simple VBA macro or a fully-blown C# / .NET application to extract meaningful information from design diagrams in Microsoft Visio. If you're a little frustrated by the limited functionality of Visio as compared with some other design tools, why not extend the functionality yourself and thereby build a better design tool?

The focus will be on UML diagrams, but not limited to that notation. If you're working with other diagram types - like Web Site diagrams or Enterprise Application diagrams - you may well find something in here for you.

System Requirements

The ideal setup is the Visual Studio .NET / Visio for Enterprise Architects combination.

Alternatively you will get some value out of this case study if you have only Visio Professional 2002. You'll be able to interrogate the Visio object model from within a VBA macro, but you won't be able to write a fully-blown C# / .NET application in the style of the sample application provided.

Defining the Problem

I've been looking for some time into the prospect of using Visio as a tool for .NET application design, as an alternative to shelling out yet more cash on a separate tool like Rational XDE. I'm compelled to do this mainly because Visio for Enterprise Architects - with UML support and Visual Studio .Net integration - comes bundled with my MSDN universal subscription, so from my point of view it's free! Because of a degree of backward compatibility with Visio Professional 2002, there is a route into exchanging design details with those other professionals who do not have an MSDN subscription.

The results of some of my Visio / .NET investigations have been published as previous C#Today case studies, which you might find to be interesting background reading:

".NET Code Generation with Visual Studio .NET Enterprise Architect" (http://www.csharptoday.com/content.asp?id=1849&WROXEMPTOKEN=969263Zzcb3IJSgp1b342dVQN2) shows how you can generate Visual Studio .NET source code from a UML model in Visio.

"C# to UML Reverse Engineering" (http://www.csharptoday.com/comments.asp?id=1837&WROXEMPTOKEN=969263Zzcb3IJSgp1b342dVQN2) introduces the idea of using reflection to inspect a compiled .NET assembly, with a view to importing the assembly class definitions into a UML modeling tool such as Rational Rose. Combining those ideas with the information in this case study raises the prospect of importing the same information into Visio.

Taking a step back from the low level details of enhancing the modeling environment, I'll also mention that I've just finished work on my contribution to the new Wrox book "Professional UML Modeling with VS.NET" (http://www.wrox.com/books/1861007957.htm?WROXEMPTOKEN=969263Zzcb3IJSgp1b342dVQN2), which shows - from first principles - how to work with UML diagrams and other design diagrams in Visio for Enterprise Architects, plus details on how to generate and reverse engineer code thanks to the Visual Studio .NET integration.

Having done all that work, it's clear to me that Microsoft have made a good start on the road to creating a credible design tool based on Visio. It has a lot going for it, including:

  • Support for a wide range of UML diagrams and other design notations, not limited to UML.
  • Integration with Visual Studio.NET, for code generation and source code reverse engineering.
  • A familiar diagramming environment for existing Visio users, and a degree of backward compatibility.
  • No additional cost if you're an MSDN universal subscriber.

However, it's not quite there yet in comparison with other design tools such as Rational XDE (for .NET languages), Rational Rose (for Java and other languages), or Together ControlCenter.

The most significant difference between generic diagramming tools (in which I include Visio) and dedicated design tools, like the ones just listed, is that the latter have an underlying meta-model for the notation being used. This means that the tool actually understands the model being constructed, rather than regarding it simply as set of connected shapes. What's more, you usually have access to that meta-model via a built-in scripting language or a set of COM interfaces so you can customize and extend the tool functionality to suit your needs.

A meta-model is a model that describes another model at a higher level of abstraction. Your UML design model or entity-relation model may show a relationship between a "customer" and an "account", but in the meta-model this boils down to a relationship between an entity (labeled as "customer") and another entity (labeled as "account").

I've hinted that the functionality provided by Visio is a little limited as compared with some of the other contenders. That wouldn't be such a problem if we had access to an underlying meta-model, and a scripting language or set of COM interfaces to drive it, because we could devise our own set of extensions to bring it up to scratch.

The good news is that Visio does have a meta-model. It's a generic meta-model for a set of connected shapes, but the properties of those shapes do contain just enough information for us to infer the meta-model for UML or whatever other notation we're using. There is a scripting language (Visual Basic for Applications), and for more advanced applications there is a set of COM interfaces that are accessible from the .NET languages.

The problem then, in a nutshell, is how to gain access to the Visio object model as a mechanism for extending the basic functionality; in other words, how to build a better design tool with Visio automation.

Article Structure

Having defined the problem above, the remainder of this case study will consist of:

  • Defining the Solution in terms of how we can gain access to the Visio object model via a VBA macro, or from a C# / .NET application using automation.
  • Presenting The Sample Application which extracts information from Visio UML sequence diagrams, and which presents that information in the form of inter-object (actually inter-class) dependencies.
  • Discussing some of the Limitations of this approach and the possibilities for Further Work using this approach as a starting point.

Defining the Solution

The solution to our problem lies in the Visio Object Model, the VBA macro language, and the COM interfaces that become accessible to us through automation. Let's take each one in turn.

Visio Object Model

The following figure shows - in UML, of course - the Visio Object Model. Not the complete model, but the portion that we'll be concerned with in this case study. You can find a picture of the complete model (not in UML notation) in the Developer's Reference section of the Visio Help.

So as not to necessitate a UML revision class in the case study - you'll find one in the book - I have represented all relationships as composition (meaning containment, and shown with a closed diamond) except for the association (non-containment relationship) between Application and Document. I've used the stereotype <<type>> in the figure above to indicate fundamental types from the type library (e.g. Page) so as to distinguish them from collections (e.g. Pages).

In a nutshell, the Application (i.e. Visio itself) holds a collection of Documents, with the current document in particular being accessible via the global variable ThisDocument. Each Document holds a collection of Pages, with each Page having Name and Text properties (plus many other attributes not shown). Each Page holds a collection of Shapes (which you see displayed on the pages) with each Shape also having Name and Text properties. Each Shape holds a collection of Connects (i.e. links to other shapes) with each Connect having a ToSheet property that points us towards another Shape.

You can see that this model represents the meta-model for Visio's internal generic representation of pages containing connected shapes. At first glance it tells us nothing about the meta-model for a UML diagram, a Web Site diagram, an Enterprise Application diagram, or any other kind of design diagram. But if we dig a little deeper we find that much of the information we need is encoded into the Name, Text, and Connects properties of each shape.

I'll demonstrate with a simple VBA macro that you can create within Visio itself: the Professional 2002 version of the Enterprise Architects version.

Visio VBA Macro

To create a new macro in Visio you choose the Tools / Macros / Macros menu option. When the following dialog appears, type the name of your new macro and press the Create button.

Don't do that just yet. I've provided a couple of sample Visio files with the macro already included, and I'll tell you about those later. First let's take a look at the macro code.

Sub LOTONtechInformationExtractor()

Here we define two collections in which we can record the shape keys and their meaningful names as we encounter shapes on a page.

Dim shapeKeys As New Collection
Dim shapeNames As New Collection

Here we define variables to correspond with the Visio object model types such as Page and Shape, and their respective collections.

Dim visioPages As Pages
Dim visioPage As Page
Dim visioShapes As Shapes
Dim visioShape As Shape
Dim visioConnects As Connects

Now we get hold of the Pages collection of ThisDocument (the current document), and we step through the pages one-by-one. For each Page we print its name.

Set visioPages = ThisDocument.Pages

For Each visioPage In visioPages

Debug.Print Tab(0); visioPage.Name

From a Page we get hold of the Shapes collection. We step through the shapes, recording the unique shape key (encoded into the Name property) and meaningful name (encoded into the Text property) for each one. Note that for each entry in the shapeKeys collection there will always be an entry in the shapeNames collection.

Set visioShapes = visioPage.Shapes

' Record the user-defined name of each shape
For Each visioShape In visioShapes
shapeKeys.Add (visioShape.Name)
shapeNames.Add (visioShape.Text)
Next

To avoid confusion later, I'll clarify the distinction between shapes keys on a page (the .Name properties) and the shape meaningful names (the .Text properties). The following table shows how these two properties relate on a typical diagram.

Page | Shape.Name (the key) | Shape.Text
Bank Technical Infrastructure | Mainframe | Live Mainframe
 | Mainframe.18 | Backup Mainframe
 | Server | LONDON Branch Server
 | Server.60 | Manchester Branch Server

Having recorded all of the shape keys and names on this page we then step through the shapes collection again. The two passes are necessary so that we can make sense of any connections that we encounter.

For Each visioShape In visioShapes

For each shape that we encounter, we print out its Text property (the meaningful name) and - in brackets - the Name property (the shape key).

Debug.Print
Debug.Print Tab(4); visioShape.Text; " ("; visioShape.Name; ") "

Next we get hold of the collection of Connects for the shape, and we step through each one.

Set visioConnects = visioShape.Connects

For Each visioConnect In visioConnects

Dim shapeKey As String
Dim shapeName As String

The ToSheet property of the Connect points us towards another Shape (the one that is connected) and we can read that second shape's Name attribute (its unique key).

shapeKey = visioConnect.ToSheet.Name

Now we use the connected shape's key to look up its meaningful name from our collections.

' Lookup the user-defined name of the shape
For i = 1 To shapeKeys.Count
If shapeKeys(i) = shapeKey Then
shapeName = shapeNames(i)
End If
Next

That might seem like overkill, but I've taken this approach specifically to highlight the distinction between the Shape.Name attributes (as Visio shape keys within a page) and the Shape.Text attributes (as meaningful names in the context of whatever diagram we're dealing with). You could streamline the macro, perhaps by discarding the shapeKeys and shapeNames collections completely, and instead reading the visioConnect.ToSheet.Text property directly.

Finally we print out the meaningful name of the connected shape:

Debug.Print Tab(8); "- connects -> "; shapeName

Next

Next

Next
End Sub

Sample Diagrams and Results

I've provided two sample Visio files called WebSite.vsd and EnterpriseApplication.vsd respectively, each of which has this macro built in. Open the first of these files, and click the Enable Macros button to see the following web site diagram:

Now select the Tools / Macros / Macros menu option and choose the LOTONtechInformationExtractor macro from Module1 as shown below, and click the Step Into button.

The Visual Basic Editor will launch and you will be able to press the blue arrow icon - shown below the Debug menu option in the following figure - to execute the macro. Initially this icon may well be labeled with the pop-up caption "Continue" (because we've already stepped into the macro) and thereafter labeled "Run".

The macro results will appear in the Visual Basic Editor Immediate window (as shown above). On subsequent runs the results will be appended, so you might want to clear the Immediate window contents each time by clicking in the window, pressing CTRL-A (to select all) and then pressing the DEL key.

The results are.

LOTONtech Web Site

LOTONtech Home (Main page)

Books ( Page)

Articles ( Page.13)

Software ( Page.17)

Consulting ( Page.21)

Order Form ( Page.25)

(Dynamic connector)
- connects -> LOTONtech Home
- connects -> Books

(Dynamic connector.30)
- connects -> LOTONtech Home
- connects -> Articles

(Dynamic connector.31)
- connects -> LOTONtech Home
- connects -> Software

(Dynamic connector.32)
- connects -> LOTONtech Home
- connects -> Consulting

(Dynamic connector.33)
- connects -> Software
- connects -> Order Form

What is interesting about those results is that the shapes representing the web pages themselves (Books, Articles, Software, etc.) are not actually connected to anything. Rather, there are additional shapes representing the connections themselves, which are connected at either end to our web pages. Only these Dynamic connector.XX shapes have a populated Connects collection property.

To show that the same approach may be taken with other diagram types, I can refer to the second file, EnterpriseApplication.vsd, which contains the following diagram:

For that sample diagram, the LOTONtechInformationExtractor macro will produce the following results:

Bank Technical Infrastructure

Live Mainframe (Mainframe)

Backup Mainframe (Mainframe.18)

(Communication Link)
- connects -> Live Mainframe
- connects -> Backup Mainframe

LONDON Branch Server (Server)

Manchester Branch Server (Server.60)

Edinburgh Branch Server (Server.84)

(Communication Link.108)
- connects -> Live Mainframe
- connects -> LONDON Branch Server

(Communication Link.109)
- connects -> Live Mainframe
- connects -> Manchester Branch Server

(Communication Link.110)
- connects -> Live Mainframe
- connects -> Edinburgh Branch Server

Sales Laptops (Laptops)

Manchester Cashiers' Workstations (Workstations)

LONDON Cashiers' Workstations (Workstations.281)

Edinburgh Cashiers' Workstations (Workstations.351)

(Communication Link.421)
- connects -> Edinburgh Branch Server
- connects -> LONDON Branch Server

(Communication Link.422)
- connects -> LONDON Branch Server

(Communication Link.423)
- connects -> Manchester Cashiers' Workstations
- connects -> Manchester Branch Server

(Communication Link.424)
- connects -> Edinburgh Cashiers' Workstations
- connects -> Edinburgh Branch Server

Did you notice what was different about those results as compared with the previous set? This time around, the connection shapes (named Communication Link.XX) are intermingled with the shapes representing equipment nodes. That's simply because I drew the nodes and links in that order, whereas in the previous example, I drew all of the nodes before drawing in the links. Try it yourself by drawing items on diagrams in various orders.

To summarize, in the first part or our solution I've given some insight into the Visio object model, and in particular the role of the Text property in providing the information required to infer the particular notation meta-model: the Web Site meta-model, the Enterprise Application meta-model, or - as you'll see later - the UML meta-model.

My use of a VBA macro has made it easier to set the scene without requiring you to set up a C# Project for automation in Visual Studio .NET. For many kinds of ad-hoc extensions to the Visio functionality this may be your preferred approach, but for more advanced applications - including manipulation of the Visio XML file format that I'll mention at the end - you'll need to take the plunge into C# or one of the other .NET languages.

Setting up a C# Project for Visio Automation

The second part of our solution will involve accessing the same Visio object model from within a .NET application, specifically a C# application. A sample application will be coded and demonstrated in the next section, but first we need to do some set-up work.

Assuming you have a C# project, either existing or a new one that you've created, you can enable it for Visio automation by adding a new entry to the project References. Right-click References and choose Add Reference as shown here:

On the next dialog, select the COM tab and choose a Visio 2002 Type Library entry as shown below:

Note: I chose the latest Service Release 1 type library, but you can choose an earlier one. You might also need to change the corresponding reference in the VS.NET solution that is provided for download.

At this point, the project references will include new stdole (not discussed further) and Visio entries. The properties of the Visio entry show that this refers to an assembly file (Interop.Visio.dll) within the ./obj subdirectory of your project.

This referenced assembly will be our route into the Visio object model. One way to investigate that object model, to find out what interfaces and classes are available, is to use the Visual Studio .NET ildasm utility.

  • Open a Visual Studio .NET Command Window by choosing All Programs / Microsoft Visual Studio .NET / Visual Studio .Net Tools / Visual Studio .NET Command Prompt from the Windows Start menu.
  • Change directory to the one containing the referenced DLL and type ildasm Interop.Visio.dll to give:

This display shows the types that comprise the Visio object model, plus the methods and properties of those types, so we can use it as our quick-reference manual. You will see that for each type there is an interface (such as Shape) and a class (such as ShapeClass). We'll be working with instances of the classes, but casting them to the interface types.

As an alternative to using ildasm, you could instead explore the Visio object model by choosing View / Object Browser from Visio's Visual Basic Editor as shown next. Actually that's my preference, mainly because I like to prototype ideas in a VBA macro before committing them to a C# project, and also because it provides some additional help text for each type.

All that remains in terms of setting up a C# project for Visio automation is to make sure that each source file includes the statement:

using Visio;

The Sample Application

What I have in mind for a sample application is a program that will interrogate a set of UML Sequence Diagrams drawn in Visio in order to determine which classes in the UML model use which other classes, thus are dependent on them. You might do this as part of an impact analysis exercise, to answer questions like "If I change this class, what other classes might be affected?" Or, you might do this in order to validate the dependencies and associations shown separately in the Static Structure diagrams.

So that you know exactly what we're aiming for, I'll tell you how to run the application - and show you the results - before looking into the code.

Sample Application Results

Load up the supplied solution VisioUMLAutomation.sln into Visual Studio .NET, and choose the Debug / Start menu option. Enter the full path of the supplied UMLScenarios.vsd file and click the Find Dependencies button.

Visio will launch automatically, the sample application will do its stuff, and the end result will be as follows:

NOTE: Your full path to the sample file .vsd will be different from mine shown here, and to run this again, you will first need to close the instance of Visio that has been launched on the desktop.

To save you the effort of trying to read the results from the screenshot I'll now reproduce those results in table form:

Object | Depends on | In scenarios
WithdrawCashPage | Account | Withdraw Cash
DepositCashPage | AccountManager | Deposit Cash
BankUser | DepositCashPage | Deposit Cash
BankUser | WithdrawCashPage | Withdraw Cash
DepositCashPage | Account | Deposit Cash
BankUser | MainMenuPage | Withdraw Cash, Deposit Cash
WithdrawCashPage | AccountManager | Withdraw Cash
MainMenuPage | WithdrawCashPage | Withdraw Cash
MainMenuPage | DepositCashPage | Deposit Cash

Now compare those results with the two sequence diagrams in the sample file that correspond with the two scenarios Withdraw Cash and Deposit Cash. In particular, notice how the BankUser makes use of the MainMenuPage in both scenarios, as documented in the table above.

Withdraw Cash Scenario (Sequence Diagram)

Deposit Cash Scenario (Sequence Diagram)

That gives you a good idea of what this application does, which should help with your understanding of the code that follows.

Sample Application GUI Code - VisioAutomationForm

The application code comprises just two classes, the VisioAutomationForm and the VisioAutomationDependencyFinder. The first class represents our GUI, the second implements the engine in which the real work is done.

So as not to get distracted by GUI code that is not essential to the plot, I'll not list the code for this class in full. You can refer to it in the sample solution if you wish.

The only method that is relevant to the task at hand is the one that processes the "Find Dependencies" button click. Here is the code for that method:

private void runButton_Click(object sender, System.EventArgs e)
{

Create an instance of our VisioAutomationDependencyFinder class and call its getDependencies() method, passing in the name of the Visio file specified by the user.

VisioAutomationDependencyFinder dependencyFinder=new
VisioAutomationDependencyFinder();

Hashtable dependencies
=dependencyFinder.getDependencies(fileNameBox.Text);

A hashtable is returned, containing keys representing the object dependencies (example key "BankUser,MainMenuPage") and with values representing the scenarios in which that dependency occurs (example value "WithdrawCash, DepositCash"). Here we loop through the keys:

foreach (String key in dependencies.Keys)
{

We separate out the comma-separated sender (dependant) class name and the receiver (dependee) class name.

int commaPos=key.IndexOf(",");
String messageSender=key.Substring(0,commaPos);
String messageReceiver
=key.Substring(commaPos+1,key.Length-commaPos-1);

Now we get the value for that key (the list of scenarios for that dependency) from the hashtable.

String scenarioList=(String) dependencies[key];

At this point we have three values to insert into the three columns of the ListView component of our form. You can find out more about the .NET ListView class by looking into the Visual Studio .Net help at ms-help://MS.VSCC/MS.MSDNVS/cpref/html/frlrfSystemWindowsFormsListViewClassTopic.htm.

        ListViewItem
            listItem = new
            ListViewItem(messageSender,0);
listItem.SubItems.Add(messageReceiver);
listItem.SubItems.Add(scenarioList);

dependencyListView.Items.Add(listItem);
}
}

As you can see, the real work is done inside the VisioAutomationDependencyFinder class, which we'll now investigate.

Sample Application Component Code - VisioAutomationDependencyFinder

The first thing to notice about this class is that it does not begin with a using Visio; statement, despite my earlier advice. That's because I wanted to show explicitly where classes from the Visio object model were being used, by prefixing them with the word Visio, as in Visio.Shape.

using System;
using System.Collections;

namespace VisioAutomationDemo
{

public class VisioAutomationDependencyFinder
{

The following member variable will hold the dependencies that we find.

        private
            Hashtable dependencies;

public
            VisioAutomationDependencyFinder()
{
}

The getDependencies() Method

The following getDependencies() method is the one that is invoked from the VisioAutomationForm.

public Hashtable getDependencies(String fileName)
{

First we reset the contents of the dependencies hashtable.

dependencies=new Hashtable();

Next we launch a Visio application instance by creating an instance of the Visio.ApplicationClass, which we cast to the interface Visio.Application. We then instruct the Visio instance to open with the document of the user-supplied filename.

Visio.Application visioApp=new Visio.ApplicationClass();
Visio.Document visioDoc=visioApp.Documents.Open(fileName);

We get hold of the document's Pages collection and step through the individual pages one-by-one:

Visio.Pages visioPages=visioDoc.Pages;

foreach (Visio.Page visioPage in visioPages)
{

From each page we get hold of the page's Shapes collection and step through the shapes one-by-one. If we encounter a shape with the name "Watermark Title" and the text "Sequence", we conclude that this page holds a UML Sequence Diagram because all sequence diagrams in Visio have such a watermark title like this:

We call the parseSequenceDiagram() method to parse it:

                Visio.Shapes
            visioShapes=visioPage.Shapes;

foreach (Visio.Shape visioShape
            in visioShapes)
{
if (visioShape.Name.Equals("Watermark
            Title")
&&
            (visioShape.Text.Equals("Sequence")))
parseSequenceDiagram(visioPage);
}
}

To save passing the dependencies hashtable around in parameters and return values, we have taken advantage of the fact that it is held in a member variable, and is therefore accessible to the parseSequenceDiagram() method. Whatever that method places in the hashtable, we return it here.

return dependencies;
}

The parseSequenceDiagram() method

This method takes in a Visio.Page that we have concluded holds a sequence diagram. We step through the shapes on that page.

private void parseSequenceDiagram(Visio.Page visioPage)
{
foreach (Visio.Shape visioShape in visioPage.Shapes)
{

If the name of the shape begins with the word "Message" - e.g. "Message.19" - we take it as a message link between two object lifelines, like this:

if (visioShape.Name.StartsWith("Message"))
{

This "Message" shape will have two entries in its Connects collection; to the sender and receiver objects respectively. We'll use a connectionCount to tell us which of the two connections we're dealing with, and by default, we'll set the sender and receiver objects to be named as "UNKNOWN".

int connectionCount=0;
String sender="UNKNOWN";
String receiver="UNKNOWN";

Now we'll step through the Connects collection in which we expect two entries; the code will be able to cope with the unlikely event that we encounter 0, 1, or more than 2 entries:

foreach (Visio.Connect visioConnect
in visioShape.Connects)
{

Get the meaningful name - in UML meta-model terms - of the connected shape (.ToSheet) by reading its .Text property directly. We're not using a lookup table for shape names and keys as we did for illustration in the earlier macro.

String realObjectName=visioConnect.ToSheet.Text;

The connected shape will represent an object lifeline, named in the form instanceName:ClassName, where the instanceName may be blank. In any case, we're interested in dependencies between classes rather than dependencies between individual instances, so we strip out just the ClassName for subsequent attention.

int colonPos=realObjectName.IndexOf(":");

if (colonPos>=0)
realObjectName=realObjectName.Substring
(colonPos+1,realObjectName.Length-colonPos-1);

Let's increment the connectionCount so that it reads one (if this is the connected sender object) or two (if we've been round the loop once and this is the connected receiver object). Record this as the sender object if connectionCount is 1.

connectionCount++;
if (connectionCount==1) sender=realObjectName;

Record it as the receiver object if connectionCount is 2. In this case, we have a complete dependency relationship which we pass through to the addDependency() method.

                        else
            if
            (connectionCount==2)
{
receiver=realObjectName;
addDependency(sender,receiver,visioPage.Name);
}
else
            break;
}
}
}
}

The addDependency() method

This method records a dependency between a sender and receiver class in the dependencies member variable.

private void addDependency(String sender, String receiver
, String scenarioName)
{

For each key in the dependencies hashtable we encode a unique sender,receiver relationship.

String key=sender+","+receiver;

The value associated with that key will be a comma-separated list of scenarios in which that dependency is present. So here we update the scenarioList value - or create it - as appropriate.

            String
            scenarioList=(String) dependencies[key];
if
            (scenarioList==null)
{
dependencies.Add(key,scenarioName);
}
else
            if
            (scenarioList.IndexOf(scenarioName)<0)
{
scenarioList=scenarioList+",
            "+scenarioName;
dependencies.Remove(key);
dependencies.Add(key,scenarioList);
}
}

}
}

Referring back to our sample results, the dependencies hashtable will contain these entries:

KEY | VALUE
WithdrawCashPage,Account | Withdraw Cash
DepositCashPage,AccountManager | Deposit Cash
BankUser,DepositCashPage | Deposit Cash
BankUser,WithdrawCashPage | Withdraw Cash
DepositCashPage,Account | Deposit Cash
BankUser,MainMenuPage | Withdraw Cash, Deposit Cash
WithdrawCashPage,AccountManager | Withdraw Cash
MainMenuPage,WithdrawCashPage | Withdraw Cash
MainMenuPage,DepositCashPage | Deposit Cash

That concludes the code for the sample application. You might want to relate that insider information to the sample results presented earlier and to the Visio file itself.

We've achieved something genuinely useful, I think, or at least sown the seeds of something genuinely useful. But it's not perfect and we'll explore some of the reasons why not in the next section.

Limitations and Further Work

I say it's not perfect, but unless - or until - Microsoft provide an API based around a true UML meta-model, it's pretty much all we have to work from. Some of the specific issues that provide an opportunity for further investigation are:

Modeling elements must appear on diagrams

This technique provides access to shapes that appear on pages, which means that we only have access to modeling elements (UML or otherwise) that actually appear on diagrams. As far as I know, there is no easy way to gain access to the underlying model, for example as displayed in the Visio UML Model Explorer window.

In fact, that seems to be a more general limitation of the Visio UML solution. In the book "Professional UML with Visual Studio .NET" I discuss the fact that in Visio an association between two classes exists only if that association is drawn on at least one diagram.

Working with other diagram types

In the second half of this case study we've concentrated on extracting information from UML sequence diagrams, but the technique is not limited to that one diagram type. Just as we can extract meaningful information from non-UML diagrams like Web Site diagrams and Enterprise Application diagrams, so we can also extract meaningful information from some of the other UML diagram types. As an example, open the UseCases.vsd file provided and run the original LOTONtechInformationExtractor macro to give the following output in the Visual Basic Editor Immediate window:

CashierUseCases

Use Case (Watermark Title)

Cashier (Actor)

DepositFunds (Use Case)

(Communicates)
- connects -> Cashier
- connects -> DepositFunds

WithdrawFunds (Use Case.9)

(Communicates.10)
- connects -> Cashier
- connects -> WithdrawFunds

SalesUseCases

Use Case (Watermark Title)

Salesman (Actor)

SellProduct (Use Case)

(Communicates)
- connects -> Salesman
- connects -> SellProduct

Writing to diagrams

We have concentrated in this case study on reading information from diagrams in Visio. However, to enhance the Visio UML experience in the ways that I would like, it would be necessary to be able to write to diagrams ; that is, to update them.

I've not looked too far into writing diagrams with automation at this stage, but I have put together a simple macro that modifies an existing UML Activity Diagram in Visio. The diagram can be found in the supplied file WalkDontWalk.vsd and it looks like this:

Also in that file you'll find the WalkDontWalk macro, containing the following code. In a nutshell this macro toggles the global variable called walk so that each time it is run it either:

  • Changes the final activity from "WALK" to "DON'T WALK"
  • Or changes the final activity from "DON'T WALK" to "WALK".

Sub WalkDontWalk()

Dim activitiesPage As Page
Dim currentShape As Shape

Set activitiesPage = ThisDocument.Pages("SampleActivities")

Here we toggle the walk global variable:

walk = Not walk

For Each currentShape In activitiesPage.Shapes

If the walk variable is set to True and the activity diagram currently displays "WALK", then we update the shape text to "DON'T WALK".

If walk And currentShape.Text = "WALK"
Then currentShape.Text = "DON'T WALK"

If the walk variable is set to False and the activity diagram currently displays "DON'T WALK", then we update the shape text to "WALK".

If Not walk And currentShape.Text = "DON'T WALK"
Then currentShape.Text = "WALK"

Next

End Sub

The first time you run the macro, the sample activity diagram will be changed to:

You can run it again to restore the original diagram:

What, no Static Structure Diagrams?

When most of you think of UML, you probably think first of the Class (also known as Static Structure) diagrams. Yet I've ignored them in my prior discussion in favor of Use Case diagrams, Sequence diagrams, and Activity diagrams.

The fact is that gaining access to the information contained within static structure diagrams appears - at first sight - to not be as easy using automation. Consider the following diagram that you'll find in the file UMLStaticStructure.vsd.

Don't worry too much if you can't read the detail in that diagram. The problem will be obvious when you see the results of running the original LOTONtechInformationExtractor macro against it:

Static Structure-1

(Class)

(Class.12)

1 (Binary Association)
- connects ->
- connects ->

(Class.26)

1 (Binary Association.35)
- connects ->
- connects ->

1 (Binary Association.40)
- connects ->
- connects ->

1 (Binary Association.45)
- connects ->
- connects ->

(Class.50)

1..5 (Binary Association.59)
- connects ->
- connects ->

(Class.67)

* (Composition)
- connects ->
- connects ->

Static Structure (Watermark Title)

The Shapes' Name, Text, and Connects properties - or any other attributes for that matter - simply do not contain the information necessary to determine the class definitions. There are forces at work here which we do not yet understand, or so it would seem.

Yet with just a few changes to that macro we can produce results like this:

Static Structure-1
=======================

(Class)
-----------------------
-DeliveryNumber : string
-PackageSize : string (Attributes)
-----------------------
«type?BR>Delivery (Name)

(Class.12)
-----------------------
-Name : string (Attributes)
-----------------------
«type?BR>Customer (Name)

1 (Binary Association)
-----------------------
-delivery (end1_name)
-----------------------
+customer (end2_name)

(Class.26)
-----------------------
-Line1 : string
-Line2 : string
-Line3 : string
-Line4 : string
-PostalCode : string (Attributes)
-----------------------
«type?BR>Address (Name)

1 (Binary Association.35)
-----------------------
-receivedDelivery (end1_name)
-----------------------
+toAddress (end2_name)

1 (Binary Association.40)
-----------------------
-collectedDelivery (end1_name)
-----------------------
+fromAddress (end2_name)

1 (Binary Association.45)
-----------------------
-occupier (end1_name)
-----------------------
+invoiceAddress (end2_name)

(Class.50)
-----------------------
-Status : string
-DateTime : ulong
-Initials : string
-Annotation : string (Attributes)
-----------------------
«type?BR>StatusHistory (Name)

1..5 (Binary Association.59)
-----------------------
-delivery (end1_name)
-----------------------
+status (end2_name)
-----------------------
1..5 (end2_mp)

(Class.67)
-----------------------
DeliveryDataSet (Name)

* (Composition)
-----------------------
-End5 (end1_name)
-----------------------
-delivery (end2_name)

Static Structure (Watermark Title)

When working with static structure diagrams, what you need to understand is that each shape representing a single class is actually made up of several shapes grouped together: one to hold the class name, one to hold the attributes, one to hold the operations, and so on.

To gain access to those group shapes - let's call them inner shapes - you need to go down one more level and look into each shape's Shapes collection property. Here's the crucial piece of code that shows you how to do just that, and you can find the complete code for the new macro in the UMLStaticStructure.vsd file.

Debug.Print visioShape.Text; " ("; visioShape.Name; ") "

For Each visioInnerShape In visioShape.Shapes
If Len(visioInnerShape.Text) > 1 Then
Debug.Print "-----------------------"
Debug.Print visioInnerShape.Text; " ("; visioInnerShape.Name; ") "
End If
Next

Alternative XML-based Approach

In versions 2002 onwards, Visio provides an alternative file format based on XML, which allows any Visio drawing file (e.g. UMLStaticStructure.vsd) to be saved out - and subsequently loaded again - from a Visio XML drawing file (e.g. UMLStaticStructure.vdx).

Suppose we save the drawing in XML format and then open the file in a text editor like WordPad. By scrolling to the right points in that file, we can begin to see things that look quite familiar:

<Shape ID='30' NameU='Attributes' Type='Shape' MasterShape='11'>
.
<Text><cp IX='0'/>-Line1 : string
-Line2 : string
-Line3 : string
-Line4 : string
-PostalCode : string</Text>
</Shape>

In case you haven't worked it out, I'll tell you that those entries correspond with the attributes of the Address class on our static structure diagram. Take a look.

As you might expect, the XML file format must contain all of the information necessary to reconstitute a Visio UML model completely. In other words, we have access to the complete UML meta-model as far as it is implemented in Visio.

I cannot yet tell you how to interpret the XML file format, but I do know that we could parse that XML from within a C# application, perhaps even wrap it up in an XML-based DataSet. As you've seen in this case study, we can also drive the Visio application itself from a C# application. So, in a nutshell, my idea is to write a C# program that:

  • Loads a UML model file into Visio and saves it out automatically in XML format.
  • Parses the XML representation, extracts the relevant information, and maybe even updates it.
  • Loads the XML back into Visio and saves it out in the original Visio format.

That alternative approach deserves a whole new discussion, perhaps a future case study once I've figured it all out.

As a final thought, it would be much easier to interpret, and update, the UML meta-model if the XML representation was in industry-standard XMI format rather than in the generic Visio drawing format. There's some light at the end of that tunnel because Microsoft has provided a free-to-download XMI export utility at http://www.microsoft.com/downloads/release.asp?releaseid=35645&WROXEMPTOKEN=969263Zzcb3IJSgp1b342dVQN2. That's great for getting information out, perhaps for loading into another tool, but as far as I know there is no equivalent utility for loading the XMI representation back into Visio.

Conclusion

In this case study I have aimed to show:

  • How you can gain access to, and extract meaningful information from, the Visio object model via a VBA macro within Visio itself. This technique is applicable to a wide range of diagram types including - but not limited to - the Web Site, Enterprise Application, and UML Use Case diagrams used specifically in the case study. For many applications the VBA macro approach may be sufficient.
  • How to set up a C# project in Visual Studio so as to make the Visio object model accessible via automation, thereby hinting at how automation could be used to drive any Windows desktop application from a .NET project. Microsoft Word and Excel are the usual candidates for an automation demonstration, so I decided to present something a little different here.
  • How to extract information from UML diagrams specifically, by taking advantage of the information encoded in each Shape's Name, Text, and Connects properties, thus providing a route into the UML meta-model.
  • How to extract additional information from Static Structure diagrams by digging one level deeper in the shapes hierarchy.
  • An alternative approach based on the Visio XML file format as a representation of the UML meta-model.

On the subject of UML I think that Visio - with its Visual Studio .NET integration - has potential as a UML modeling tool, but as it stands it falls short of the capabilities provided by other such tools. Maybe Microsoft will provide the necessary enhancements as time goes on. In the mean time, I've tasked myself with using the techniques from this case study, plus other techniques that I have developed and documented elsewhere, to enhance the Visio UML experience; in other words to build a better design tool.

So as to start moving in the right direction I'd like to collect as many ideas as possible. What features do you think are missing in the Visio + Visual Studio .NET offering as compared with the other modeling tools? You can let me know via this case study's discussion page or by posting a message at http://www.lotontech.com/contact.html?WROXEMPTOKEN=969263Zzcb3IJSgp1b342dVQN2.


Creating International-Ready Applications

Creating International-Ready Applications
http://www.csharptoday.com/content.asp?id=1996
Article Rating: 4.5
Published on November 25, 2002
Abstract
It is essential to build nimble software that can quickly adapt to new challenges. One of these challenges is in building applications that can respond to an increasingly global economy. While modern development languages have greatly eased the task of creating software for any locale, the challenge is still in creating software that can simultaneously support multiple locales. The .NET platform assists with many of these aspects, but there are still many design aspects to consider.In this case study, Damon Allison and Ben Hyrman discuss the design of international applications, and introduce you to the .NET Framework’s built-in support for internationalization.
Case Study

It is essential to build nimble software that can quickly adapt to new challenges. One of these challenges is in building applications that can respond to an increasingly global economy. While modern development languages have greatly eased the task of creating software for any locale, the challenge is still in creating software that can simultaneously support multiple locales. The .NET platform assists with many of these aspects, but there are still many design aspects to consider. In this case study, we will discuss designing of international applications, and introduce you to the .NET Framework's built-in support for internationalization.

Introduction

Why Internationalization?

Companies today must quickly adapt to change. The national brand of today is the international corporation of tomorrow. Businesses need software that can support this growth. Trade agreements open new commerce opportunities, and provide a new consumer base. It is a given that nimble companies will grow and rigid companies will fail. The same can be said of software.

The simple fact is that the world marketplace is becoming unified; this is especially true with regards to software, as computers are ingratiated worldwide. The case for creating internationalized software is a simple one. An application developer should be concerned with creating software that reaches as wide a base of customers as possible, and this increasingly means looking to other countries as well. An IT developer has to be ready to respond to company needs, which increasingly involves expansion into foreign markets and reusing core systems wherever possible.

There are many issues related to adapting software to new markets. Some of these issues include:

  • UI design considerations. Color meanings differ by culture, text widths can vary greatly, screen flow is not always left to right, and input fields may need to be larger. For example, red is often used to indicate danger or warning in the US. In fact, you may often see red used to denote data entry errors. However, red means happiness in China. The same consideration must be given to logos, icons, and images in general. An icon that seems harmless in the US, such as a thumbs-up gesture on a dialog box, is considered offensive and obscene in Japan.
  • Translation. This is one of the most time-intensive parts of localization. Every piece of text that a user will see must be changed, including menus, labels, and errors. Even elements that may not spring to mind, such as wording embedded in a graphic, must be translated.
  • Formatting differences. Everything from currency, to date and time, to weights and measures must be formatted for each locale.
  • Database design. Planning a database layout can be a challenge. Planning for a layout that can support multiple locales can seem almost impossible, at first.
  • Code segmentation and versioning. Code should be partitioned to separate the internationalized core from the localized components. Testing and versioning can become a large issue if not properly addressed.

In this case study, we will begin by discussing the concepts behind internationalization. We will point out where design mistakes are often made. We will also provide best practices and insight into planning and designing to handle internationalization. We will then focus on .NET's abilities to ease internationalization, and how to design applications to take advantage of the Common Language Runtime (CLR)'s built-in multi-cultural support for localization.

For better understanding of internationalization, it is important to discuss common terminology. Let's turn our attention to the concepts and terminology used in creating international apps.

Key Concepts in Internationalization

Globalization: This is the entire scope of effort defined here, including creating an internationalized core, localized extensions, and locale-specific translations.

Internationalization: Internationalization is the act of generalizing code so that it can support multiple locales with a minimal amount of redesign and redevelopment. Internationalized code is independent of the language, culture, and character set. Internationalization does not include translation, cultural design considerations for a specific locale, or any work related to localizing an application.

Internationalization is abbreviated as I18N. It may seem cryptic and mystical at first, but this abbreviation likely came about out of laziness, or ingenuity. If you will notice, there are eighteen letters between the first 'I' and the last 'n' in internationalization.

Localization: Localization is the exercise of customizing the application to meet specific requirements for deployment to a locale. Localization generally covers areas such as translation, formatting, and user interface redesign. Localization is often referred to as L10N.

Typically, a localization effort will include the help of a subject matter expert for the given locale. In other words, if you're US-based, developing for France, and the only thing you know about the culture is that they have an accent, then you should engage an expert to assist with UI design, locale formatting, and translation.

In addition, this effort will employ the use of graphics artists to localize images, translators to tackle all of the text in the application, and programmers will be responsible for any code changes to support a specific locale.

Locale: A locale defines a user's environment, including information specific to country, language, and code set. For example, date formatting, numeric formatting, currency type, time zone, and language are part of what make up a user's locale. This becomes crucial in determining both how to store and display data. In the US, we might state that an appointment is on 07/04/2002; while in Germany we would display the date as 04.07.2002.

Translation: Translation is the linguistic component of localization. Translation involves the conversion of thoughts, ideas, and concepts from one language to another. This may be the single largest effort of localization because it applies to all text a user will see, including user interfaces, help files, and documentation.

One thing that we should point out here is that translation is still largely a manual process. While there are many automatic translation services available, ranging from free to not-so-free, these are still not quite to the level of being able to be fully automated features of any application.

System Requirements

This case study assumes a basic understanding of C# and ASP.NET. In order to run the code you will need the ASP.NET V1.0 Framework installed and access to a SQL-aware database. The SQL make table scripts are included. They may need to be slightly modified to work with your database. SQL Server 2000 was used for this piece.

Defining The Problem

Design Challenge

The challenge with internationalizing applications is determining an effective design. A good design allows the application to add locales without the need to change database, code, or application logic. In the ideal scenario, adding additional locales to the application would simply be a matter of creating locale specific data, putting the data into the proper data stores, and testing the application. For example, someone would translate the user interface components such as text, dialog boxes, and graphics and give the translated data to the developer. They would add the data and the application would be off and running in a new locale. While this seems far-fetched and difficult, it isn't. If the application is designed with internationalization in mind (the application is "internationalized"), it is possible to rapidly speed up rolling out changes to an application to support new languages, currencies, and locales. Of course, testing must still happen, but many parts of development are made much easier.

The application created in this case study will pay particular attention to the design considerations and choices developers need to consider in order to create a well designed internationalized application. As a developer, you know the challenges of changing applications mid-lifecycle and how much more difficult it can be than creating an application from scratch. Internationalization is a far-reaching 'change' and needs to be considered early to avoid extensive recoding.

While designing for internationalization may seem blatantly obvious, many subtleties are often overlooked in the design stage. For example, if images have embedded text, they must be modified for each culture.

System Upgrades

There is a point mentioned in the last section that deserves to be said again: a more difficult challenge internationalization poses is determining how to implement locales into a current, non-internationalized application. It is important to understand how much work is involved in upgrading systems. Not only does the international design need to be developed, the current logic must be preserved and the implementation must be transparent to the user. Imagine an application designed to work with ASCII data that now must support Unicode. Serious changes could be needed, which could have easily been avoided if Unicode had been considered from the start.

Pre-.NET Landscape

Prior to .NET, internationalization was not always easy to implement in Microsoft systems, especially with Web-based applications. While VB and C++ applications had some support for resource files and could use API calls to retrieve locale information, there was little inherent support for internationalization that didn't have to be customized and tweaked. The landscape was even worse for ASP developers. This led to many different solutions of varying levels of sophistication and reliability.

When attempting to create international-ready ASP applications, programmers were often forced to buy third party products or implement custom database-driven applications to return localized information. Some crafty developers created XML file based solutions with a small API to retrieve localized data from the file. However, these solutions were all one-offs, there was no single means for internationalization, and these solutions took time to build. Plus, in the end, parsing an XML document or issuing SQL calls brings a significant overhead. What was needed was a single API, built into the runtime, which would take much of the burden off of the developer.

Microsoft's choice to implement the .NET framework with internationalization features further eases the development cycle. The .NET framework incorporates many of the common features found in internationalized applications (resource lookup and formatting in particular) directly into the framework.

Our Case Study

Our fictitious client, ComputerReviews, is a computer and technology retailer based in the US. ComputerReviews is developing a website to accept product reviews online. The owner of ComputerReviews would like to start by offering a German and English interface to their site. Consideration must be given for additional languages to be implemented in the future with little to no recoding.

The possibilities for an international website are infinite. Perhaps a site that you have developed would be a candidate for a multicultural website. Perhaps your business would have a competitive advantage if you could offer a Spanish or French translation. Our case study goal is to present a detailed look at how to design an internationalized application using the technologies available with the .NET framework.

Case Study Structure

In this case study, we will be creating an internationalized website from scratch to support English and German languages using the .NET framework. Since perhaps the most important part of internationalizing an application occurs in the design phase, we will begin by discussing design considerations. Once we have an application design, we will implement the database, component layers, and user interface. The outline is as follows:

  • Planning for Internationalization
  • Code Structure
  • Best Practices
  • Designing for Internationalization
  • Database Design and Structure
  • Component Design and Development
  • UI Design and Localization
  • Satellite Assemblies versus Resource Files
  • Next Steps
  • Related Information and considerations for improvements

Planning for Internationalization

Planning and designing a system for internationalization will take more time and effort than designing an application for a single locale. However, the effort spent at the start will reap enormous benefits the first time the application is expanded to support a second locale, or even a second language in the same locale.

While it may not seem too obvious to a normal web user, there is a huge amount of text and images that are used when building a website. All the labels, links, menus, images, colors, error messages and text must be translated and reviewed from a translator. Anything that is presented to the user is subject to be different across locales, and incorporating all the data into your application requires significant initial investment.

Code Structure

An internationalized application should be designed to maximize reuse across locales. On the surface this may sound obvious, but in practice it is not always followed. Code that will span multiple locales, such as business logic, should be placed in the application core - separated from locale specific data. Services that are specific to a locale, such as tax calculation, should be exposed to the core through a façade if needed, and placed into the localized extensions.

Quite simply, a façade is a means to hide a specific implementation from other components. For example, a tax component is generally specific to each locale. However, it wouldn't be wise to directly tie the application core to each specific tax implementation. A façade can provide a common interface between the application core and the various tax components. Graphically, this would like this:

In this example, we've created a core to the application that houses the bulk of the application logic. The localized extension should contain only information specific to a given locale. Over time, the application should be refactored to move functionality duplicated across locales into the core. We've also shown a service whose concrete implementation is specific to a locale. A tax engine is a good example of a utility that is location specific. We've hidden the implementation to keep from tying the core to any single locale. Of course, this is very easy to do on paper, but will require quite a bit of thought for any real application.

User interface design is a bit more difficult, since unlike the component layer, the user interface is drastically affected by culture. For example, as the culture changes, so does the need for larger or smaller sized labels and text controls. A 20 character string in English can translate into a 60 character string in another language. One option to deal with the interface differences across cultures is to create a separate UI for each locale. While we have total control of the interface, we lose the ability to create and manage a single code base. Plus, maintenance and support across locales becomes difficult as we have the same user interface logic repeated many times over.

Another option is to make everything from colors, to layout, to labels and messages dynamic in order to support all locales. The advantage that there is only one code base to maintain. As you will see, the .NET Framework has made this option much easier and more attractive. Even with the improvements, however, internationalization requires careful thought and design.

A possible trade-off between the two options, especially taking into account your first target locales, is to build a UI for logical groups of locales. For example, you may want to create one UI for Latin-based languages and another for each graphical-based language such as Bengali and Kanji.

Again, the goal is to push the bulk of the application into the core and have lightweight extensions per locale, as described in the diagram below. This will greatly ease development for future locales.

One more note, concerning testing. It should be obvious, but changes to the core must be tested across all locales, while changes to the extension will generally only require testing between that locale and the core. This is a trade-off to the recommended approach of maximizing the core in that the frequency of changes, and therefore testing with all locales, is increased. However, the benefits in reuse and leveraging a single code base greatly outweigh this drawback. Again, a solid design up front will further assist with minimizing the need for changes later on.

Internationalization Best Practices

There are many facets of internationalization that are very straightforward. For example, many Americans do not use the metric system for measurement, but you would be hard-pressed to not support metric measurements in Europe. When designing an application, it becomes important to identify and account for these differences. While not an exhaustive list, here are a few examples of how cultures differ and how to design for them in your application.

  • Currency
  • Do not store numeric separators with the currency value
  • Do not store currency symbols with the currency value
  • Ensure the field size is larger enough to support multiple locales
  • Ensure it is possible to identify currency locale, especially if it must be converted for multiple locales
  • Language
  • Store strings in Unicode
  • Store literals, labels, and messages in resource files or a database; never hard-code strings
  • Always use the Framework for sorting and comparing strings
  • Store user input in the original language entered

Measure

If storing multiple units of measure, such as both English Imperial and Metric:

  • Store the measure as well as unit of measure (i.e., store both "50" and "lbs", not just "50")

Or

  • Store only the measure but ensure that storage in a single format is understood across the application. In the case of measures, this is generally the preferred approach, as converting measures is a fairly static operation.

Measures are a bit unique in that conversions from one unit to another are constant, the formulas are well known, and the overhead for conversion is small. The key in design is to provide a consistent approach.

  • Date and Time
  • Always store date in a 24-hour format (HH:MM:SS)
  • Store time as Greenwich Mean Time (GMT) + offset. For example, US Central Standard Time is GMT-6
  • Be ready to handle daylight savings time. If not handled correctly, this can pose problems, especially where timestamps are used to ensure proper ordering
  • Store date in ISO standard format, YYYY-MM-DD; see ISO 8601 for details
  • Graphics
  • Graphics should not contain text. For static graphics, the text would need to be re-edited for every locale. For dynamically created graphics, the text may not fit properly for every language.
  • Graphics should be easily replaceable. Colors may need to be changed for graphics in other cultures. Icons may need to be replaced to offer a clear meaning in other cultures. These graphics should be replaceable with minimal impact to the application.

Database Design and Structure

Go ahead and open a database you've designed in the past. Is it well designed? Sure, although you probably see some things now that you'd change. Now, can it handle multiple locales, or even multiple languages? Would you know what records belong to which locale? Regardless if your database is designed for internationalization or not, there are certain principles that all databases can use to improve their support for internationalization. Let's first take a look at a typical, one language database design and consider the impact of changing the structure to support internationalization.

Typical Database Design

In a typical database design, the data is normalized into logical relationships in a manner that minimizes data redundancy. However, thought is rarely given to supporting multiple locales. Take a look at our first design for the product catalog and reviews.

We have two tables joined with a foreign key. The 'Products' table represents a list of products for sale and the 'Reviews' table represents a user review for a particular product. While this design certainly meets our needs, it presents some problems when we discuss expansions into even our second locale. We'll likely want to carry the same product in all locales. However, our current design dictates that we must duplicate product information for each language if we want to display the product description in the user's language.

Let's examine the 'Products' table. How would you add a German record in the database? How would you know the record is German when retrieving for display? At the very least, we need to specify a locale to retrieve this information. Assume for arguments sake that we added a field and called it locale. We would have the ability to determine which products are German, but we would run into another problem. The ProductID for German is different from the ProductID in English. We would need to change the primary key to be a combination of ProductID and Locale.

Even in our simple two table example, designing for internationalization is beginning to require quite a bit of database change. Notice how drastic the changes are becoming and think of how those changes will impact deployed code. All the code that relied on ProductID to be the primary key would now need to be edited. Stored procedures would need to be edited. At the object layer, the design would need to make room for locale. With a more complicated database, we could only imagine the change necessary.

Internationalized Database Design

We've expanded the database design to take into account multiple locales. At first the design may seem to be much more complex. However, the benefits should become evident rather quickly. We've further segmented the design to group data common to all locales and languages apart from the data that must change for each language or locale.

So why did we make such changes? There is some information in the Products table that is culture neutral that we can separate into its own table. The new 'Products' table contains data that does not vary over cultures. For our example, Manufacturer (Say IBM) does not translate any differently in German than it does in English. The same holds true for model number.

The design that you choose for your database may be different than ours. Since we don't show any information from the Products table to the user (we use ProductDetails), we can abstract this information into its own table. Products is used internally by our company and does not need to be taken into consideration when internationalizing. We do need it, however, for our internal uses.

If this data would be presented to the user, you would want to consider locale. For example, Chinese may require a different model number and manufacturer. Design the system as it meets your needs.

No matter how you design the system, there is one principal that should be considered: data that is common to all locales and languages should be placed in base tables. Data that must be customized or translated for each locale or language should be separated into extensions of the base tables. Candidates for the base table may not always be obvious. For example, measurement and weight would likely be best placed in the base table. The conversions for these units are simple, and a single value will be much easier to manage than values spread across all entries in the extension. In our example, Products is the base table, ProductDetails is the extension.

NOTE: We have made this database available as a SQL script in the source download. The source for this application can be found by clicking the appropriate tab on the toolbar at the top of the screen.

Component Design and Coding

Components are where the application's logic and rules are applied. A well-designed system will have its logic separated from the user interface into a set of components. In traditional COM, DLLs might be written in C++ or Visual Basic to provide the logic needed for the application to run. In .NET, assemblies have replaced COM objects and are a much more versatile unit of deployment than an unmanaged DLL.

Have you ever thought about how the design of a component changes with the introduction of multi-language sites? For example, can your application support Unicode or ASCII? What languages are the error messages written in? These design considerations that were taken for granted with one language now become important.

Dealing with Interfaces

How much of your component design, in particular the interface design, is impacted by internationalization? If designed properly, not much! The interfaces that are exposed by the components will need to expose parameters or functionality for setting the locale information used for retrieving data, but the majority of the interface will remain similar to your current designs.

The component layer is often more complicated than what we are presenting in this small application (we will see the code shortly). But no matter how complicated, we can generalize topics that impact internationalization at the component layer.

Watch out for all data points and avoid hard coding strings and values that will be presented to the user. Internal data and data structures can hold and use strings internally written in one language. For example, we should not be concerned with using foreign keys in one common language for access into tables or using English constants to represent database names and system values. Since these values are not returned to the user, they are language neutral. However, the text for the accompanying error message cannot be hard coded as it is specific to each locale and must be returned to the user. .

Let's take a look at the component code used to retrieve information from our internationalized database above. Pay particular attention to the use of Locale in the functions. Here is the code in its entirety. You don't need to completely understand the code in its entirety as we will discuss in more detail shortly.

/***********************************************************************
* ProductRetrieval.Retrieve
* Purpose: Provides data access (middle tier)
* layer to retreve/save data to/from the
* data store. This abstraction allows us to
* separate data access from UI.
*
* Improvements:
* Passed in culture is not verified.
* Perhaps create a table with supported cultures.
* This example uses inline sql for brevity.
* A more robust approach would be to call Stored
* Procedures maintained within the database.
* CommandBehavior.CloseConnection - closes connection when data reader is closed.
***********************************************************************/
//csc /t:library /r:System.Data.dll /r:System.Xml.dll ProductRetrieval.cs

using System;
using System.Data;
using System.Data.SqlClient;
namespace ProductRetrieval
{
/// <summary>Provides data retrieval procedures.</summary>
public class Retrieve
{
private string CONN;
private string locale;
public Retrieve(string LocaleCode)
{
//NOTE: Change conn string for your environment.
//Possible improvement: use a configuration file to avoid hardcoding
CONN = "Data Source=localhost;Initial Catalog=Products;Integrated Security=sspi";
locale = LocaleCode;

}
/// <summary>
/// Returns the product list in the specified locale code.
/// The ProductDetails table in the product database must have a
/// product record for each product and each locale.
/// </summary>
/// <returns>A SqlDataReader of all avaliable products for a given locale</returns>
public SqlDataReader GetProducts()
{
SqlConnection cn = new SqlConnection(CONN);
SqlCommand cm = new SqlCommand("Select * from ProductDetails where LocaleCode = '" +
locale + "'", cn);
cn.Open();
SqlDataReader dr = cm.ExecuteReader(CommandBehavior.CloseConnection);
return dr;
}

/// <summary>
/// Returns specific product information on a given product/locale combination
/// </summary>
/// <param name="ProductID">Product ID to retrieve</param>
/// <returns>SqlDataReader with 1 record containing product information</returns>
public SqlDataReader GetProductDetails(int ProductID)
{
SqlConnection cn = new SqlConnection(CONN);
SqlCommand cm = new SqlCommand("Select * from ProductDetails Where LocaleCode = '" +
locale + "' and ProductID = " + ProductID, cn);
cn.Open();
SqlDataReader dr = cm.ExecuteReader(CommandBehavior.CloseConnection);
return dr;
}
/// <summary>
/// Returns review information for a product. The user
/// will only see the reviews specified in their locale.
/// </summary>
/// <param name="ProductID">ProductID to retrieve reviews from</param>
/// <returns>SqlDataReader containing the product reviews. 0->n rows</returns>
public SqlDataReader GetReviews(int ProductID)
{
SqlConnection cn = new SqlConnection(CONN);
SqlCommand cm = new SqlCommand("Select * from Reviews Where LocaleCode = '" +
locale + "' and ProductID = " + ProductID + " order by ReviewID DESC", cn);
cn.Open();
SqlDataReader dr = cm.ExecuteReader(CommandBehavior.CloseConnection);
return dr;
}

/// <summary>
/// Add a review to a specific product.
/// Possible improvements include validating data before inserting into database.
/// This procedure could insert a blank record.
/// </summary>
/// <param name="ProductID">ProductID to add review</param>
/// <param name="Name">Reviewer Name</param>
/// <param name="Rating">Reviewer Rating</param>
/// <param name="Title">Title of reviewer's comments.</param>
/// <param name="Review">Review text supplied by reviewer.</param>
public void AddReview(int ProductID, string Name, int Rating,
string Title, string Review)
{
SqlConnection cn = new SqlConnection(CONN);
SqlCommand cm = new SqlCommand("INSERT INTO Reviews (ProductID, ReviewerName, LocaleCode, " +
"DatePosted, Rating, Title, Review) VALUES (@ProductID, @ReviewerName, @LocaleCode, " +
"@DatePosted, @Rating, @Title, " +
"@Review)", cn);
cm.Parameters.Add("@ProductID", ProductID);
cm.Parameters.Add("@ReviewerName", Name);
cm.Parameters.Add("@LocaleCode", locale);
cm.Parameters.Add("@DatePosted", DateTime.Now);
cm.Parameters.Add("@Rating", Rating);
cm.Parameters.Add("@Title", Title);
cm.Parameters.Add("@Review", Review);
cn.Open();
cm.ExecuteNonQuery();
cn.Close();
}
}
}

The code here is very simple. It performs no logic. It simply retrieves and saves data into the database. What is important in terms of internationalization is the inclusion of LocaleCode as a parameter to the constructor. The locale parameter could have been passed into each function, but since an object is created every time it is needed, we are assuming the locale will be the same from the time the object is created until it is destroyed. Let's examine the important function:

public Retrieve(string LocaleCode)
{
//NOTE: Change conn string for your environment.
//Possible improvement: use a configuration file to avoid hardcoding
CONN = "Data Source=localhost;Initial Catalog=Products;Integrated Security=sspi";
locale = LocaleCode;
}

In this constructor, locale is passed into the constructor and used throughout the other functions for storing the locale into the database. The other functions simply issue a SQL statement to the database and retrieve a SqlDataReader or write a record into the database. This code will allow us to retrieve from the database and save product reviews. These four functions are the extent of our component layer. To compile this code, save the file as ProductRetrieval.cs and use the following statement at a command prompt:

> csc /t:library /r:System.Data.dll /r:System.Xml.dll ProductRetrieval.cs

csc.exe is, of course, the command-line compiler for C#

NOTE: You will want to edit the connection string to pertain to your environment. You may also want to create a test project and test the four functions out. They are quite self-explanatory and can be tested very easily.

At this point, it becomes quite obvious that we are not designing a true enterprise application, but rather an example to show internationalization. There are many improvements that can be made to this code for security, scalability, and performance. Some of the more obvious improvements would be moving the database connection string into a configuration file, using stored procedures to execute the SQL, and proper error handling. We will show how to retrieve locale specific strings when we discuss UI, just understand that locale specific error messages should be considered.

This example illustrates including locale at the component level and also how the middle tier is largely language agnostic. Adding locale at the constructor level or at the function level enables us to vary query results dependent on language. If this component were to handle logic, the locale should be largely transparent. For example, if we are adding dates or subtracting dates, the base class libraries should be used to handle the calculation.

The base class libraries are designed with culture in mind. For example, if you pass in a currency, it will be returned in a culture specific format. Since this functionality is built into the base class libraries, it becomes important to understand the formatting options presented in the base class libraries and avoid recreating similar functionality. When we discuss UI, you will see examples of the base class libraries in action.

UI Design and Coding

So far we have the database designed and the component layer designed without performing any translation. This is not uncommon since the bulk of the work in internationalization surrounds the presentation layer. Internationalization is largely about presentation. The application logic and storage is relatively similar across all cultures.

Very briefly, let's brainstorm what needs to happen at the user interface level to support multiple languages.

First, strings, captions, images, colors, and images need to be retrieved from a data store. We want the ability to have one .aspx page to use across cultures to avoid repetition. In order to achieve this goal, we need to store locale specific strings in some form of data store. Much of this data is not dynamic in nature; it is just different across cultures. If it were dynamic, it would change frequently and is better implemented in a database. The UI elements such as captions, copyrights, and slogans are not generally very dynamic in nature.

Second, the data retrieval must be very fast. Since potentially thousands of strings and culture specific data will be retrieved with each request, we need to make sure we do not hinder performance drastically. Storing perhaps a thousand resource strings into a database can hinder performance, drastically if not properly cached.

Third, we want the process of interacting with the culture specific data to involve as little effort as possible. We do not want to spend time and write large amounts of code to retrieve and display data. For example, we do not want to code if blocks or switch statements around the different cultures. It is obvious this code will not scale if additional cultures are added.

if(culture != "en-US")
myString = "Willkommen";
else
myString = "Welcome";

Lastly, we need the ability to plug in another culture without rewriting any code. We want to be able to insert the strings into our data store and the locale will be ready to go! With .NET, there is an entire namespace devoted to using resource information: System.Resources.

Resource Files

.NET continues the same concept of a resource file to provide resources to an application. At a high level, a resource file is a set of culture specific data that is compiled and read by the System.Resources.ResourceManager class. String data is entered in a resource file and saved with an appropriate filename to signify what culture the resources belong to. A resource file is created for every language and the user interface designer is free from needing to store the data into the database or into the interface directly. The resource file is similar to a configuration file, where parameters are retrieved and used when needed.

The vast majority (if not all) applications use textual resources. A resource is simply a piece of data that is used by the application when executing. The data itself is not code and cannot be executed; it is simply read into the application at runtime.

The simplest form of a resource file is a .txt file containing name value pairs. Let's take a look at an example of a resource file:

;example resource file
;comment lines start with ;

;General strings
Welcome=Welcome to my website:

;Product
ProductName=Take a look at our product list!

Resource files have a special file naming requirement. Each file is named with the following format:

<basename>.<culture>.txt

Basename is the name used when referencing the application. The culture is the name of the locale these resources are saved as. To create a set of 'strings' resources, the following names would be used for the American English (en-US) and German (de-DE) locales:

strings.en-US.txt
strings.de-DE.txt
strings.txt

As an aside, a resource for England would be en-GB Australia would be en-AU.

When a user enters the site with an en-US locale, the en-US resource file is used. Likewise, when the user enters with a de-DE locale, the de-DE resource file is used. The file strings.txt is called a fallback resource file. When a user enters the application with an unrecognized locale (not en-US or de-DE in our example), the fallback resource file is used for resources. Fallback resources are important to implement, especially on the web, where you are not certain what locales will be used. If a user enters the site with an unrecognized locale and the site does not implement fallback resources, an error will be thrown. Resources should not be a source of application errors.

Once we have the resource files built for our application, we will compile them with the resgen.exe tool. The command to compile a resource file is:

> resgen strings.en-US.txt

The resgen.exe tool will create a file called strings.en-US.resources. All resource files, once compiled, are given the .resources extension. The ResourceManager class (discussed shortly) is programmed to use the .resources extension when looking for resource files. If you deviate from the expected filename, the resources will not be recognized. The resource files are compiled into a more compact format

Resource files must be created prior to creating the user interface. At this point, it becomes difficult to recognize the importance of the resource files. If we created the user interface at this point, we would not be able to internationalize the application. Now that they are created, we can begin to construct the user interface. Let's look at the English and German resource files used in our case study. The basename of our resource files is ui.

English version :

;ui.en-US.txt

;Banner User Control
BannerHeader=Product Display
;Product.aspx
Welcome=Welcome to the product reviews website. This page is designed to provide product information and user reviews.
Click on one of the following products for more information.
ProductTableHeader=User reviews and comments
CultureSelect=Select a culture
;ProductInfo
WriteReview=Want to write a review? Now's your chance!
ReviewTableHeader=Product reviews
ReviewerName=Your Name:
ReviewerRating=Rating (0 to 5):
ReviewerTitle=Title:
ReviewerReview=Enter your review:
Submit=Submit
ReturnLink=Return to products page

German Version :

;ui.de-DE.txt
;Banner User Control
BannerHeader=Produkt-Anzeige
;Product.aspx
Welcome=Willkommen zum Produkt wiederholt website. Diese Seite wird entworfen, um Produktinformation und Benutzerberichte
zur Verfügung zu stellen. Klicken Sie an eins der folgenden Produkte zu mehr Information.
ProductTableHeader=Produkte
CultureSelect=Wählen Sie eine Kultur vor
;ProductInfo
WriteReview=Möchten einen Bericht schreiben? Jetzt Ihre Wahrscheinlichkeit!
ReviewTableHeader=Produkt-Berichte
ReviewerName=Ihr Name:
ReviewerRating=Veranschlagen (0 bis 5):
ReviewerTitle=Titel:
ReviewerReview=Tragen Sie Ihren Bericht ein:
Submit=Reichen Sie ein
ReturnLink=Gehen Sie zur Produktseite zurück

It is interesting to note how fast the resource file can fill up. Every color, every string, every data point that is culture specific comes from the resource file. It becomes critical to keep a detailed illustration of the site and the resource fields used from page to page in order to reduce confusion. Maintenance is also critical. If you remove controls or UI from the application without deleting from the resource file, you will have unnecessary data in your resource file. Over time, this will make managing and maintaining the resource files slightly more difficult. So, as a best practice, always revisit the resource file when making changes.

Using Resource Files From ASP.NET

Once the resource files are compiled, they can be used by the .aspx application. Our application has two pages. The first page is used to present a list of products to the user. For the purposes of demonstration, we allow the user to select which locale they want displayed. In a production application, you would retrieve the locale information directly from the browser using the Request.UserLanguages property. In the HTTP stream, language information is sent to the web server and can be read via the Request object. The page's name is products.aspx(and can be found in the support material for this case study) and has the following presentation (shown with German locale):

The user is presented with a list of products, and can select one for more information. Even in this small example, notice how many data points are needed. If this site had any serious UI, the amount of resources will be quite large. If we wanted to improve on this design, we would also have made the languages German and English select box items resource items also. We left these hard coded as English strings.

The second page contains the product details and a list of reviews submitted for each product. The second page is productInfo.aspx :

Here, resources are needed for the banner and the product reviews ("Produkt-Berichte") labels. The rest of the text is retrieved from the database via the component layer. The entire code for these two pages is available via download. We want to focus on how to retrieve the resources from the recently compiled .resources files.

Importing Namespaces

There are a few key namespaces that should be imported into the application. Here are the namespaces relevant to internationalization:

using System.Threading;
using System.Globalization;
using ProductRetrieval;
//System.Resources imports namespace for ResourceManager
using System.Resources;

System.Threading is used to get access to the current thread. The current thread holds the locale information the runtime uses for the request. For numbers, dates, currencies, and whenever the culture is needed, the runtime uses the current threads culture by default. System.Globalization contains the CultureInfo object. CultureInfo represents the actual culture used by the thread. It specifies the formatting rules, calendaring rules, and culture-specific data for the culture. ProductRetrieval is the namespace for our component. System.Resources is the namespace used to do the actual resource lookup.

We have a lot of new material here. Let's discuss each namespace a bit further by looking at the code that makes internationalization a reality in our application. Let's look at the code in productInfo.aspx that is responsible for working with these namespaces.

private void Page_Load(object sender, System.EventArgs e)
{
//Set culture
Thread.CurrentThread.CurrentCulture = CultureInfo.CreateSpecificCulture(Session["selectedLocale"].ToString());
Thread.CurrentThread.CurrentUICulture = Thread.CurrentThread.CurrentCulture;

//Retrieve product information to populate controls
Retrieve r = new Retrieve(Session["selectedLocale"].ToString());

//Set product ID
if(!IsPostBack)
Session["prodID"] = Request.QueryString["prodID"];
else
{
//Record payment
r.AddReview(Int32.Parse(Session["prodID"].ToString()), NameText.Text, Int32.Parse(RatingText.Text),
TitleText.Text, ReviewText.Text);

}
//Retrieve product details into UI labels
SqlDataReader dr = r.GetProductDetails(Int32.Parse(Session["prodID"].ToString()));
dr.Read();
ProductName.Text = dr["ProductName"].ToString();
Description.Text = dr["Description"].ToString();
EditorComments.Text = dr["EditorComments"].ToString();
Price.Text = Double.Parse(dr["Price"].ToString()).ToString("c", Thread.CurrentThread.CurrentCulture);

//Bind reviews
SqlDataReader drReviews = r.GetReviews(Int32.Parse(Session["prodID"].ToString()));
ReviewList.DataSource = drReviews;
ReviewList.DataBind();

//Set international labels using resources in app's main directory)

ResourceManager rm =
ResourceManager.CreateFileBasedResourceManager
("ui", Server.MapPath(".")
+ System.IO.Path.DirectorySeparatorChar, null);
WriteReview.Text = rm.GetString("WriteReview");
ReviewTableHeader.Text = rm.GetString("ReviewTableHeader");
ReviewerName.Text = rm.GetString("ReviewerName");
ReviewerRating.Text = rm.GetString("ReviewerRating");
ReviewerTitle.Text = rm.GetString("ReviewerTitle");
ReviewerReview.Text = rm.GetString("ReviewerReview");
ReturnLink.Text = rm.GetString("ReturnLink");
Submit.Text = rm.GetString("Submit");
}

Let's examine the code in more detail.

//Set culture
Thread.CurrentThread.CurrentCulture = CultureInfo.CreateSpecificCulture(Session["selectedLocale"].ToString());
Thread.CurrentThread.CurrentUICulture = Thread.CurrentThread.CurrentCulture;

Since each request is stateless, each time the user enters our site, we need to set the current thread locale. In products.aspx, the user selected a locale. We saved that locale string (en-US or de-DE) into session. The CurrentThread object's CurrentCulture and CurrentUICulture are set to the proper locale. The CurrentCulture determines the culture settings for formatting, date, and number presentation. The CurrentUICulture represents the current user interface culture. Setting them both ensures that the base class libraries will format to the specific culture when appropriate.

//Retrieve product information to populate controls
Retrieve r = new Retrieve(Session["selectedLocale"].ToString());

Here we create an instance of our component and prepare for data retrieval.

ResourceManager rm =
ResourceManager.CreateFileBasedResourceManager ("ui", Server.MapPath(".")
+ System.IO.Path.DirectorySeparatorChar, null);
WriteReview.Text = rm.GetString("WriteReview");
ReviewTableHeader.Text = rm.GetString("ReviewTableHeader");
ReviewerName.Text = rm.GetString("ReviewerName");
ReviewerRating.Text = rm.GetString("ReviewerRating");
ReviewerTitle.Text = rm.GetString("ReviewerTitle");
ReviewerReview.Text = rm.GetString("ReviewerReview");
ReturnLink.Text = rm.GetString("ReturnLink");
Submit.Text = rm.GetString("Submit");

Here, we create an instance of the ResourceManager object. ResourceManager is responsible for retrieving the appropriate set of resources from the .resources files. It uses the current locale and retrieves the resources from the file matching the current locale. (Remember the specific naming conventions used for resource files? The ResourceManager needs the files to have the proper naming conventions.) The .resources in this application are maintained in the root directory where the .aspx pages reside.

The ResourceManager is very easy to use. The process of determining what file to access, how to parse the file, and what strings to retrieve is handled by this class. All you need to do is set the locale and the ResourceManager takes care of making sure the correct resource file is used. As you can imagine, keeping track of what resources are stored in the files is much more time consuming than creating the code to create the ResourceManager.

Limitations of resource files

Resource files are nice because they allow us a quick way to retrieve culture specific data fast. However they are not the most ideal case in the ASP.NET internationalized scenario. Two useful features introduced in ASP.NET are xcopy deployment and assembly versioning.

Resource files are just that, files. When a file is in use, the operating system locks the file. If you are in the process of deploying changes to resource files and the file is locked, the xcopy command will error. Assembly versioning allows us to host multiple versions of applications side by side. Applications can be configured to use only a particular version of an assembly. When new versions are created, they can be linked to updated assemblies. Both copies of the application can be executed simultaneously without conflict. The benefits of assembly versioning are very well documented. Unfortunately resource files cannot be versioned alone. In order to be versioned, they need to be compiled into assemblies. Assemblies containing only resources are called satellite assemblies in .NET.

Satellite Assemblies

Satellite assemblies contain no code, only resources. Satellite assemblies are the preferred way to deploy resource files. You retain the benefits of resource files and obtain the benefits of assemblies, such as versioning. In order to create satellite assemblies, there is an extra compilation step to convert .resources files into .dll assemblies.

The assembly linker tool, al.exe, is used to convert .resources into assemblies. The following batch script was created with this application to create the resource files from the .txt files and create the .dll files from the .resources. The batch file is called deploy.bat and contains the following console commands:

resgen ui.en-US.txt
resgen ui.de-DE.txt
resgen ui.txt
al /t:library /embed:ui.resources /out:bin/ui.dll
al /t:library /embed:ui.en-US.resources /culture:en-US /template:bin/ui.dll /out:bin/en-US/ui.resources.dll
al /t:library /embed:ui.de-DE.resources /culture:de-DE /template:bin/ui.dll /out:bin/de-DE/ui.resources.dll

The resgen lines we saw previously simply create the .resources files. Once the resources are created, the al.exe tool is responsible for making the assemblies and copying them into the appropriate directory. al.exe generates the .dll manifest from the resource files and embeds the resources into the assembly. The /embed switch copies the contents of the resources file into the assembly. The /culture switch tells the assembly loader what culture to associate the final assembly with. The /template switch tells al.exe to inherit the metadata from the template assembly. In this case, the template is the fallback resource.

The /out switch is rather interesting. Notice that each .dll file is named the same (ui.resources.dll). In general, the assemblies used as satellite assemblies must be the same name and put into a directory that matches their culture. Just like resources had required filenames, so do assemblies. With resource files, they must be named according to their culture and installed into the same directory. Assemblies must be named the same and installed into a directory according to their culture. To use locally in the application (as opposed to globally, which we'll touch on shortly), we need to install the assemblies under the ASP.NET application's \bin directory.

Here is what the \bin directory looks like when the assemblies are deployed. At the root of the \bin directory, ui.dll, the fallback resource, is installed. The fallback resource is culture-independent and does not need a unique directory. The en-US and de-DE cultures both contain a ui.resources.dll as well. When the ResourceManager class needs to use resources, it first checks for the existence of the appropriate culture's assembly. If the assembly cannot be located, the fallback ui.resources.dll must be used. Note that the assembly filename must end in .resouces.dll. The standard convention for assembly filenames is:

<basename>.resources.dll

Here is an image of the \bin folder's contents. The i18l.dll assembly is the assembly created for the ASP.NET project. ProductRetrieval.dll is the component we developed earlier.

Using Satellite Assemblies

When we retrieved resources from resource files, we used ResourceManager.CreateFileBasedResourceManager. CreateFileBasedResourceManager tells the resource manager to search for resources in the specific directory (in our case the application's root, ".") instead of using the assembly manifest. When using satellite assemblies, the ResourceManager is passed an instance of our base (fallback) resource assembly. The ResouceManager is using that assembly as its base for locating the culture specific resources. Here is the Page_Load function from products.aspx. The assembly ui.resources is passed to the ResourceManager constructor along with the base name of the resource files. With those two parameters, the ResourceManager has enough information to retrieve resources for any culture implemented.

private void Page_Load(object sender, System.EventArgs e)
{
//Set locale
Session["selectedLocale"] = Locale.SelectedItem.Value;
Thread.CurrentThread.CurrentCulture = CultureInfo.CreateSpecificCulture(Session["selectedLocale"].ToString());
Thread.CurrentThread.CurrentUICulture = Thread.CurrentThread.CurrentCulture;

//Bind reviews
Retrieve r = new Retrieve(Session["selectedLocale"].ToString());
ProductList.DataSource = r.GetProducts();
ProductList.DataBind();

//Load resources
System.Reflection.Assembly a = System.Reflection.Assembly.Load("ui.resources");
ResourceManager rm = new ResourceManager("ui", a);

CultureSelect.Text = rm.GetString("CultureSelect");
ProductTableHeader.Text = rm.GetString("ProductTableHeader");
Welcome.Text = rm.GetString("Welcome");
}

Global Assemblies

Satellite assemblies can be made global to the machine by compiling the assembly (using al.exe) with a strong name. The sn.exe utility is used to create a unique keyfile that can be applied to the assembly. Once compiled, use the gacutil.exe utility to install the files into the GAC. The makes the resource files global to all applications running on the current machine. It is usually sufficient to copy directories into the bin directory; however the GAC gives assemblies additional advantages, such as versioning.

The Sample Application

Since the application contains a database, middle tier, and UI, running the sample application takes a little bit of preparation and configuration. IIS, .NET, Visual Studio (optionally), and SQL Server are needed. Complete the following steps to install the code:

1. Download the application source

2. Extract the .zip file to any location. We suggest a subdirectory under the IIS WWWRoot directory.

3. Create a virtual directory named i18l in IIS pointing to the i18l directory under your root.

4. Launch the i18l.sln file within Visual Studio.

Points of interest will be:

Products.aspx - User interface for the products page.

Products.aspx.cs - Resource retrieval for products.aspx

ProductInfo.aspx - Product details page

ProductInfo.aspx - Resource retrieval and UI for an individual products page with reviews

ProductInfo.aspx.cs - Resource Retrieval and Page_Load for individual products page

Deploy.bat - build script for resource files

Ui.txt - Fallback resource file

Ui.en-US.txt - en-US resource file

Ui.de-DE.txt - de-DE resource file

ProductsSQL2000.bak - database backup file. You should restore this as the 'Products' database in your instance of SQL 2000.

Case Study Review

Internationalization is an important and often underestimated design topic. Since internationalization affects the heart of an application, it cannot be easily added to an application without initial design planning. If designed properly, the effort of adding another language to an existing application can be drastically reduced.

At the database level, internationalized applications determine what fields of data are dependent on culture and which are not. Data dependent on culture is separated from culture neutral data. In the case study example, the product description and editors comments were deemed culture specific and separated out from the Products table. This allowed us to create a product detail record for each culture.

The component layer of the application is largely culture neutral. The case study component was made aware of the culture at the constructor level. The rest of the interface remained similar to a non-internationalized application. Because of the stateless nature of the web, each request from a user can potentially come over a different thread. Each request must set the culture, which means we want to generally avoid storing state at the component level.

The user interface is where we begin to see internationalization come to life. Resource files and satellite assemblies provide access to resource information and the ResourceManager handles all the work of retrieving the appropriate resources for the application to use.

With the introduction of .NET, many of the common tasks have been absorbed into the Common Language Runtime and the Base Class Libraries. For example, assembly resolution, resource file location and retrieval, data formatting, and culture are natively built into the framework. Knowing how to use the tools to create, compile, and distribute resources represent the extent of the learning curve. The vast majority of the time spent in internationalizing code involves determining resource names and the actual translation involved with the user interface's data points.

Limitations or Future Work

The resource file presented in this case study was all string, name=value pair, based. .NET also offers another form of resource file called a .resx resource (.resx is the file extension). Resx resources are similar to those presented in this case study, however they are capable of holding more complex data. For example, the .resx file, which is xml based and human readable, can contain binary data or serialized objects. The default resource creator in Visual Studio.NET creates resx files by default. For more information on .resx files, refer to the .NET SDK.

The System.Resources namespace provides the developer programmatic access to create resource files. The System.Resources.ResourceWriter class can be used to dynamically create resource files. This class would come in handy if you want to create an application the language translators can use to create resource files. It would also come in handy if you have a large system and would like to automate resource file creation.

The application presented did not take security, performance, or proper logical code design into consideration. Many improvements can be made to this code. For example, the resource manager class could be created once and stored in session if desired, stored procedures could be used instead of inline SQL, and the default culture could be determined by the browser, not requiring the user to select the culture manually.

This case study was presented to a technical audience and thus was focused around .NET's ability to assist the programmer with internationalization. Another aspect of internationalization that is more cultural in nature is the process of translation. For example, the color red is a sign of rage in the United States, but could be pleasing to foreign cultures. Slogans or text could be misinterpreted and become derogatory or offensive to another culture. The slogans and text in this application were converted to German by Babelfish provided by AltaVista. Babelfish can be found at http://babelfish.altavista.com/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq.

If you want to learn more about Internationalization, why not take a look at the new C# Text Manipulation Handbook from Wrox?

The .NET Framework has a variety of string manipulation features, and incorrect or ill-considered use of these methods could drastically increase your memory and processor time consumption. This book teaches you how to make use of regular expressions to match and replace text, and how to manipulate text using the various string matching, manipulation, and replacement classes. Issues such as Localization and data conversion will also be investigated.

For interviews with authors and editors, sample chapters, and news on the other C# handbooks, please see our Handbooks page.


Network Programming in C#

Network Programming in C#
http://www.csharptoday.com/content.asp?id=1994
Article Rating: 4.5
Published on November 18, 2002
Abstract
This case study by Irfan Rizvi focuses on network programming in C# using sockets. Sockets provide a medium of communication between two applications, whether those applications reside on the same computer, two different computers connected to each other on local network, or two computers connected to the internet.
Case Study

This case study focuses on network programming in C# using sockets. Sockets provide a medium of communication between two applications, whether those applications reside on the same computer, two different computers connected to each other on local network, or two computers connected to the internet.

A Brief Introduction To Networks

There are two main protocols that are used in network related applications; both have their own advantages and disadvantages. TCP (Transmission Control Protocol) is a highly reliable connection-oriented protocol, but sends acknowledgements for each request and response and therefore is more time consuming. TCP is the most widely used protocol because of its reliability; it is the standard protocol of the Internet.

The other protocol is UDP (User Datagram Protocol), which is a connectionless protocol - this means no acknowledgements are sent after request or responses, and hence it is faster then TCP but less reliable. UDP is used in applications in which time is the critical factor.

If you are new to sockets, you might be interesting to know that almost all servers, browsers, and almost everything in a TCP/IP network environment are using socket technology in the background. The theory behind TCP/IP network communication is that every machine on a network has a unique address (its IP address). You might be wondering that if each computer has but a single address, how can they differentiate which data is for which application when multiple applications are running on a machine? For example, if you have two browser windows open, each requesting information from a different website, then how does the computer know which browser to send what data to? The answer to this is addressing, which is done on two levels.

The first level is NSAP (Network Service Access Point), usually called IP addressing, which identifies the machine. When the data reaches the correct machine, we need to find the correct SAP (Service Access Point). This is the second level, which is level addressing. Service Access Points are also called port addresses, and these uniquely identify each application. Sockets are made up of a unique combination of an IP address and a port address, allowing the correct port address on the correct machine to be found.

The main aim of this case study is to demonstrate network programming in C# using sockets and TCP. .NET provides base classes for TCP server and clients in its System.Net namespace, and there are several classes available to deal with various requirements of network applications.

Introduction

In this case study we will be using a simple client/server application to demonstrate how socket programming works. This application uses TCP for the communication.

In this application, the server lets multiple clients to connect to it. Once the client is connected, the server can view all of its information like name, IP address, user name, the port at which it is connected, and the protocol it is using.

The server can also request the list of processes running on the client, and can view information related to each of these processes such as process name, start time, etc. Similarly, the server can also request the list of services that are available on the client, and information relating to these. These services are the same ones that you can manually access via Control Panel.

The server can perform several operations on the clients such as pausing, resuming or stopping a service. Through the Stop Client operation, the server can also disconnect the client - that is, the client will no longer be connected to the network.

By reading this case study, you will gain valuable knowledge regarding sockets, TCP, and the knowledge required to start programming client server applications.

Networks In More Detail: Theory and Technology

In this section we will take a detailed look at what networks are, how they work, and how the .NET architecture supports network programming.

OSI Reference Model

The OSI (Open Systems Interconnections) model is based on the proposal developed by the International Standards Organization (ISO) as a first step towards an international standardization of the protocols used. This model is called the Open System Interconnections Reference Model because it deals with connecting open systems, that is, systems that are open for communication with other systems.

The OSI Model has seven layers which are described briefly as follows:

Physical Layer: This layer handles the transmission of bits over the network. It provides the physical connection for the transmission of data among the network devices. The physical layer is responsible for making sure that data is read the same way on the destination device as it is sent from the source device. For example, the physical layer specifications include the amount of voltage on a cable, how a signal changes to signify a 1 or a 0 being sent, and in what order a signal is sent.

Data Link Layer: The main purpose of this layer is to provide a reliable method of transmitting data across the physical media. This layer breaks the input data into frames, transmits the frames sequentially, and processes the frames when they reach are received. It adds a header and trailer to the frames it creates - this allows the destination device to see when a frame begins or ends.

Network Layer: This layer is responsible for routing information from one network device to another; the network layer decides what path data will take if the destination device is located on another network. It accepts messages from the source, converts them into packets, and make sure that the packets are directed towards the destination. It is also responsible for deciding on the best route the packet should take through the network

Transport Layer: The transport layer is the fourth layer of the OSI model. It provides a transport service between the session layer and the network layer. This service takes information from the session layer and splits it up into packets if necessary, then passes this information to the network layer and checks to make sure the information arrived at the destination device successfully. The transport layer adds a header that includes error detection (to detect corruption of data), sequencing (as the message is split into several packets, this allows it to be re-assembled correctly), and flow control messages (which are used to make sure the destination is receiving information correctly).

Session Layer: This layer lets users establish a connection (called a "Session") between devices. Once the connection has been established, the session layer can manage the dialog. To establish a session, a user must provide the remote address they want to connect to. These addresses are not like the network addresses; they are intended for the user, and are easier to remember.

Presentation Layer: The presentation layer establishes the format in which data is exchanged. This layer is responsible for character set or numeric translations needed. It is also responsible for data compression to reduce the amount of data transmitted, as well as encryption.

Application Layer: The top layer of the OSI model is the application layer. This layer is the interface between the user's applications and the network. It allows the application the user sees to transfer files, send emails, and to do anything else it needs to on the network.

TCP/IP Protocol Suite

The Transmission Control Protocol/Internet Protocol (TCP/IP) suite, also known as the Internet Protocols, is a collection of industry-standard protocols. The TCP/IP suite is made up of many protocols, not just TCP and IP, and has a broad feature set due to its large number of open-standard protocols. Over the years, the individual components have evolved to handle almost any need a network user may have.

The original designs for TCP/IP (Transmission Control Protocol/ Internet Protocol) were stated long before the OSI model was developed; instead of the OSI's seven layers, TCP/IP has just four layers. The four layers can be loosely matched to the OSI model as follows:

Network Access Layer: This layer corresponds to the physical and data link layers of the OSI model.

Internet Layer: This layer roughly matches up with the network layer of the OSI model. Both of these layers are responsible for moving data to other devices on the network.

Host-to-Host Layer: This one is similar to the transport layer of the OSI model. The job of both of these layers is to communicate between peers on the network. As a result, almost all devices on a TCP/IP network are considered host, whether they are workstations, servers, or network-attached printers.

Process/Application Layer: This fourth layer does the same job as the top three layers of the OSI model, which is to provide network services.

TCP (Transmission Control Protocol)

In client-server applications, the server will provide some service such as processing database queries or sending out current stock prices. The client uses the service provided by the server, in these cases to either display database query results to the user or to make stock purchase recommendations to an investor. The communication that occurs between the client and the server must be reliable. That is, no data can be dropped, and it must arrive at the client side in the same order in which the server sent it.

TCP provides a reliable, point-to-point communication channel that client-server applications on the Internet use to communicate with each other. TCP was specifically designed to provide a reliable end-to-end byte stream over an unreliable internetwork. An internetwork differs from a single network because different parts may have widely different topologies, bandwidths, delays, packet sizes, and other parameters. TCP was designed to dynamically adapt to properties and to be robust in the face of many kinds of failures.

To communicate over TCP, a client program and a server program establish a connection to one another. Each program binds a socket to its end of the connection. To communicate, the client and the server each reads from and writes to the socket bound to the connection. The client and server send an IP packet to each other; this packet contains the TCP header that includes flow control, sequencing, and error checking (as touched upon earlier).

Each client is given the same port number of the server so that all the data being sent reaches the correct virtual connection. A port is similar to a mailbox. When data is delivered to a computer via the network, it must be sent to a process on the computer. A computer may have multiple processes running on it. Each service that expects data from the network will register itself onto a port number. For example, an internet web server uses port 80, while a mail server uses port 25. Inside the header of the network packet is a port number for which the data is intended, similar to a postal address for a letter.

Sequence Numbers: TCP uses a 32-bit sequence number that counts bytes in the data stream. Each TCP packet contains the starting sequence number of the data in that packet, and the sequence number (called the acknowledgment number) of the last byte received from the remote peer. Forward and reverse sequence numbers are completely independent, and each TCP peer must track both its own sequence numbering and the numbering being used by the remote peer.

Window Size and Buffering: Each endpoint of a TCP connection will have a buffer for storing data that is transmitted over the network, waiting until the application is ready to read the data. This lets network transfers take place while applications are busy with other processing, which improves overall performance. To avoid buffer overflow, TCP sets a Window Size field in each packet it transmits. This field contains the amount of data that may be transmitted into the buffer. If this number falls to zero, the remote TCP can send no more data. It must wait until buffer space becomes available and it receives a packet announcing a non-zero window size.

What is an IP Address?

In order for two applications to make a connection, they need to be able to identify each other. Computers on networks have a unique identifier called an IP address which is represented in dot-notation, an example would be 10.20.120.127. No two computers on the same network can have the same IP address. As we discussed earlier, the IP address consists of a network address and a host address. The network address represents the network to which this machine belongs, while the host address is the address of the machine itself. IP addresses can either be dynamic or static. There are some IP addresses that are used for special proposes, such as 127.0.0.1, which is known as the loop back address, etc.

What are Ports?

IP addresses are network addresses which identify a particular machine on a network. However, there are many applications running on a single machine, many of them communicating over the network, so each of these applications need an address to uniquely identify them. The solution to this problem is ports. Ports consist of an integer address which ranges from 1 to 65536, and each application on a machine has a port address. Servers usually have standard port addresses which are used to handle clients. For example, all web servers have a default port address of 80.

What are Sockets?

A socket is a unique combination of an IP address and a port number, identifying a destination that data can be sent to.

Normally, a server runs on a specific computer and has a socket that is bound to a specific port number. The server just waits, listening to the socket for a client to make a connection request. The client knows the hostname or the address of the machine on which the server is running, and the port number to which the server is connected. To make a connection request, the client tries to rendezvous with the server on the server's machine and port. If everything goes well, the server accepts the connection. Upon acceptance, the server gets a new socket, bound to a different port. It needs a new socket (and consequently a different port number) so that it can continue to listen to the original socket for connection requests while tending to the needs of the connected client.

On the client side, if the connection is accepted, a socket is successfully created and the client can use the socket to communicate with the server. Note that the socket on the client side is not bound to the port number used to rendezvous with the server. Rather, the client is assigned a port number local to the machine on which the client is running. The client and server can now communicate by writing to, or reading from, their sockets.

Socket Programming in C#

Network programming in Windows is possible using socket communication. A socket is like a handle to a file. You can use socket programming to have two applications communicate with each other. The applications are typically on different computers, but they can be on the same computer, or even the same process or thread of the same application.

For the two applications to talk to each using sockets, either on the same or different computers, one application is generally a server that keeps listening to the incoming requests and the other application acts as a client and makes the connection to the server application. The server application can either accept or reject the connection. If the server accepts the connection, a dialog can begin between the client and the server. Once the client is done with whatever it needs to do, it can close the connection with the server. Connections are expensive in the sense that servers allow finite connections to occur. This is because there are 65536 available ports on a machine, and another reason is that usually servers create a thread for each client, and as the number of client grows, large number of threads are running simultaneously, which will consume the server's resources.

During the time the client has an active connection, it can send data to the server and/or receive data. The complexity begins here. When either side (client or server) sends data, the other side is supposed to read the data. But how will the other side know when data has arrived? There are two options - either the application needs to poll for the data at regular intervals, or there needs to be some sort of mechanism that would enable applications to get notifications when new data arrives. Well, Windows is an event driven system, so the notification system is the obvious best choice.

The System.Net namespace

The .NET Framework provides a wide range of excellent features that developers can take advantage of while building networking applications. The System.Net classes are similar to Microsoft's WinInet API (used for internet protocol programming) in that they allow applications to get and send data using Internet protocols. The .NET classes are quite different in design and function however, the main difference being that the .NET classes are designed with an emphasis on standing up to a high-stress server environment and present an object based API model, while the WinInet was a functional API.

The System.Net namespace provides a simple programming interface for many of the protocols used on networks today. The WebRequest and WebResponse classes form the basis of what are called pluggable protocols, an implementation of network services that enables you to develop applications that use Internet resources without worrying about the specific details of the individual protocols.

The System.Net namespace contains a number of classes that can be used to communicate with resources on the Internet.

  • Socket programming can be done through System.Net.Sockets.Socket class.
  • Web resources can be downloaded by using the System.Net.WebRequest class.
  • Use System.Net.IPAddress to represent an IP address. You can use System.Net.Dns to resolve the IP address to a name ( System.Net.IPHostEntry ) or vice versa.
  • System.Net.WebProxy objects can be used to pass requests through a proxy server.

Since our case study only focuses on the System.Net.Sockets.Socket class, we will discuss only this class here.

System.Net.Sockets.Socket

The Socket class creates a managed version of an Internet transport service. Once the Socket is created, the Socket is bound to a specific endpoint through the Bind method, and the connection to that endpoint is established through the connect method. Data is sent to the Socket using the Send or SendTo methods, and data is read from the Socket using the Receive or RecieveFrom methods. After you are done with the Socket, the ShutDown method is used to disable the Socket, and the Close method to close the Socket.

The Socket class is used by the Microsoft .NET Framework to provide Internet connections to TcpClient, UdpClient, WebRequest, and the derived classes. We will only be discussing TcpListener and TcpClient here.

TcpListener: The TcpListener class builds upon the Socket class to provide TCP services at a higher level of abstraction. You can use a TcpListener to listen for connections from TCP clients. Application level protocols such as FTP and HTTP are built on the TcpListener class. You can create an instance of the TcpListener class by providing one of the following constructor argument lists: an IPEndPoint containing your local IP address and port number, your IP address and port number, or just your port number (in this case, the default network interface will be used).

TcpClient: The TcpClient class builds upon the Socket class to provide TCP services at a higher level of abstraction. TcpClient provides simple methods for connecting, sending, and receiving data over a network. You can connect with a remote host using one of the following two ways:

  • Create an instance of the TcpClient class, using either no parameters or using a local IPEndPoint, and then calling one of the three available TcpClient Connect methods.
  • Create an instance of the TcpClient class using the host name and port number of the device to which you intend to connect. This constructor will automatically establish a connection.

To send and receive data, a GetStream method is used that obtain a NetworkStream that can send and receive data on the underlying connected Socket. After utilizing the write and read methods available through the NetworkStream, the Close method is used to release all resources associated with the TcpClient.

I/O Stream

Streams in C# usually involve reading and writing bytes to and from the console, the filesystem, or the network. In C#, the stream paradigm is used more generally whenever a program needs to move or operate on a group of bytes.

To read and write to and from the console, C# provides a single class called System.IO.Stream. Instead of having two objects, one for reading and one for writing, a C# Stream object needs to be tested for its capabilities via the CanRead and the CanWrite properties. This class has methods to operate on one byte at a time, along with methods that operate on an array of bytes.

NetworkStream: This implements the standard .NET Framework stream mechanism to send and receive data through network sockets. NetworkStream supports both synchronous and asynchronous access to the network data stream. NetworkStream does not support random access to the network data stream.

Applying Our Knowledge

We have now taken a detailed look at the theory behind networks, and you should now have a good knowledge of why and how they are used. We will spend the rest of this case study using this theoretical knowledge to practically implement our client\server application, showing how all its features work as we go.

A Note About Loopback

Please note before we begin that the application that we will be creating in this case study is network related, and so it is better that client and server should run on separate machines - but this is not absolutely necessary. You can run both server and client on the same machine using the Loopback address. This address is a special IP number (127.0.0.1) that is designated for the software loopback interface of a machine. The loopback interface has no hardware associated with it, and it is not physically connected to a network.

An Overview Of The Final Application

The application we will be creating looks like the following:

From this screen (on the server), the administrator will be able to view all the clients that are connected to the server (in the left hand box). Selecting a client will cause the relevant details (such as IP Address, Username, the communication protocol being used, etc) to be shown in the top right hand collection of boxes. In the bottom right hand box, the administrator can view either the list of processes or services currently being run on that client. By using the buttons in the toolbar at the top, the administrator can manipulate these, for example pausing, resuming or stopping a process.

Now that we have seen what the final application will look like, and seen what it will actually do, let's get down to the hard work of putting together the code for this project. By closely following the following code and explanation, you will gain a good understanding of how sockets work in C#, what namespaces are important, and some of the issues involved, such as dealing with byte streams.

Component Design and Coding

Server Description

The above diagram represents the class architecture of the server. The main class is Server, which is the start point of this application and handles the User Interface; it also starts a thread of serverListener class which will run in the background and handle the incoming client connection requests. Whenever a new client arrives, this thread will make an object of ClientInfo class with the information about that client, and pass this object to Server where it will be added to its collection. The serverListenerArgs class acts as a wrapper class for transporting ClientInfo instances from the serverListener class to the Server class. The ProcessObject class and ServiceObject class are representatives of a Process and a Service.

Now we are going to discuss each class individually:

Class Description: Server

The Server class actually handles the User Interface via events. Most of the code is generated by Visual Studio itself - we will not describe this generated code, as it is out of the scope of this case study.

Attributes

Name | Type | Access Modifier | Description
alClientInformation | ArrayList | Private | Container for all clients
slListener | serverListener | Private | Listener object
GUI object | System.Windows.Form.* | Private | All other form objects

Methods

private void
            InitializeComponent()
{
this.slListener = new
            serverListener(1024);
this.slListener.OnClientSignal += new
            serverListener.ClientSignal(this.ClientConnectionHandler);
}

Although this method is Visual Studio generated, some of it is worth mentioning:

this.slListener = new serverListener(1024);

This statement starts the server's listener thread by making an instance of the serverListener class, while

this.slListener.OnClientSignal += new serverListener.ClientSignal(this.ClientConnectionHandler);

registers the ClientSignal event with the ClientConnectionHandler method.

private void ClientConnectionHandler (object sender , serverListenerArgs args)
{
ClientInfo ciClient = args.ClientInfo;
alClientInformation.Add (ciClient);

lstClients.Items.Add(ciClient.MachineName);
}

This is the event handler of a new client connection. Whenever a new client request arrives, the listener throws a notification about this request, with all the information about that client in an object of serverListenerArgs which is inherited from EventArgs class. One thing worth mentioning here in the architecture of event driven programming is that all the event handlers in .NET receive two and only two arguments; the first one is of object type which describes the sender from whom the notification is coming, and the second is the one through which all the information about that particular event is sent. This second argument must be an object of EventArgs class or any class inherited from EventArgs. We will describe the serverListenerArgs class later.

Getting back on track, this method or event handler will be initiated whenever a client arrives. It receives the information about that client (via the parameters) and adds it to its ClientInformation container. The client's machine name is added to the list box to show which client has been connected.

private void lstClients_SelectedIndexChanged(object sender, System.EventArgs args)
{
ClientInfo ciClient = (ClientInfo) alClientInformation [lstClients.SelectedIndex];

tbMachineName.Text = ciClient.MachineName;
tbUserName.Text = ciClient.UserName;
tbIPAddress.Text = ciClient.StrIPAddress;
tbPort.Text = ciClient.PortNumber.ToString() ;
tbProtocol.Text = ciClient.SConnectedSocket.ProtocolType.ToString();

if(rdbProcesses.Checked)
{
lstServices.Items.Clear();
IEnumerator en = ciClient.ProcessList.GetEnumerator();
while (en.MoveNext())
{
ProcessObject processObject = (ProcessObject) en.Current;
lstServices.Items.Add(processObject.StrProcessName);
}
}
else if (rdbServices.Checked)
{
lstServices.Items.Clear();
IEnumerator en = ciClient.ServiceList.GetEnumerator();

while (en.MoveNext())
{
ServiceObject serviceObject = (ServiceObject) en.Current;
lstServices.Items.Add (serviceObject.DisplayName);
}
}
}

This is the lstClients event handler. Whenever the server admin selects a client's machine name, this event handler is executed and it displays that machine's information.

private void lstServices_SelectedIndexChanged (object sender, System.EventArgs args)
{
ClientInfo ciClient = (ClientInfo) alClientInformation[lstClients.SelectedIndex];

if (rdbProcesses.Checked)
{

ProcessObject processObject = (ProcessObject) ciClient.ProcessList[lstServices.SelectedIndex];
tbServiceName.Text = processObject.StrProcessName;
tbServiceType.Text = processObject.ProcessStartTime;
}
else
{
ServiceObject serviceObject = (ServiceObject) ciClient.ServiceList [lstServices.SelectedIndex];
tbServiceName.Text = serviceObject.DisplayName;
tbServiceType.Text = serviceObject.ServiceType;
tbServiceStatus.Text = serviceObject.Status;
}
}

This event handler executes when the user selects a particular process or service from the list box. It get the relevant information and displays it on screen.

private void
            rdbProcesses_CheckedChanged(object sender, System.EventArgs
            e)
{
if (rdbProcesses.Checked)
{
lbllist.Text = "List
            of Process";
lblname.Text = "Process Name";
lbltype.Text =
            "Process Type";

tbServiceStatus.Visible =
            false;
lblstatus.Visible = false;

if
            (lstClients.SelectedIndex >= 0)
{
ClientInfo ciClient =
            (ClientInfo) alClientInformation
            [lstClients.SelectedIndex];

lstServices.Items.Clear();
IEnumerator
            en = ciClient.ProcessList.GetEnumerator();

while
            (en.MoveNext())
{
ProcessObject processObject =
            (ProcessObject)
            en.Current;
lstServices.Items.Add(processObject.StrProcessName);
}
}
}
}

This event handler switches the on-screen display from service mode into process mode. So, if the user selects the "process" radio button, information about the highlighted machine's process are then shown on the screen.

private void
            rdbServices_CheckedChanged(object sender, System.EventArgs
            e)
{
if (rdbServices.Checked)
{
lbllist.Text = "List of
            Service";
lblname.Text = "Service Name";
lbltype.Text =
            "Service Type";
tbServiceStatus.Visible =
            true;
lblstatus.Visible = true;
if (lstClients.SelectedIndex
            >= 0)
{
ClientInfo ciClient = (ClientInfo)
            alClientInformation[lstClients.SelectedIndex];

lstServices.Items.Clear();
IEnumerator
            en = ciClient.ServiceList.GetEnumerator();
while
            (en.MoveNext())
{
ServiceObject serviceObject =
            (ServiceObject)
            en.Current;
lstServices.Items.Add(serviceObject.DisplayName);
}
}
}
}

This event handler is the opposite of rdbProcesses_CheckedChanged, in that it switches the view from process mode into service mode, as oppose to switching from service to process.

private void tlbClicked(object sender,
            System.Windows.Forms.ToolBarButtonClickEventArgs args)
{
if
            (args.Button.Equals(this.tlbOptions.Buttons[0]))
{
MessageBox.Show("stop
            client");
}
else if
            (args.Button.Equals(this.tlbOptions.Buttons[2]))
{
MessageBox.Show("Pause
            Service");
}
else if
            (args.Button.Equals(this.tlbOptions.Buttons[3]))
{
MessageBox.Show("Resume
            Service");
}
else if
            (args.Button.Equals(this.tlbOptions.Buttons[4]))
{
if
            (selectedClient != null && selectedService !=
            null)
{
bool result = selectedClient.stopService
            selectedService.DisplayName);

if
            (result)
{
selectedService.Status="Stopped";
lstServices_SelectedIndexChanged(sender,
            e);
}
else
MessageBox.Show("Operation Failed","Error
            Message");
}
else
{
MessageBox.Show("No Service
            selected","Alert");
}
}
else if
            (e.Button.Equals(this.tlbOptions.Buttons[6]))
{
MessageBox.Show("Stop
            Process");
}
}

The above method is the event handler for the toolbar. We are only going to discuss one operation here ( stopService ), as this will be enough to give you a good understanding of what is happening.

if (selectedClient != null && selectedService != null)
{
bool result = selectedClient.stopService(selectedService.DisplayName);
if (result)
{
selectedService.Status="Stopped";
lstServices_SelectedIndexChanged(sender, e);
}
else
MessageBox.Show("Operation Failed","Error Message");
}
else
{
MessageBox.Show("No Service selected","Alert");
}
}

Whenever the stopService button is pressed, the code above is executed. It calls the stopService method of the selected client and passes the selected service's name to it. What this stopService method actually does will be discussed in more detail later, for now we are just assuming that if the operation is performed successfully then true is returned, else false is returned.

Class Description: ServerListener

Attributes

Name | Type | Access Modifier | Description
ClientSignal | delegate | Public | Delegate for the new client connection event
OnClientSignal | <<Event>> ClientSignal | Public | Represents the event shown when a new client arrives
Server_Socket | TcpListener | Private | Listens for connections from TCP network clients
iPort | Int | Private | Specifies the port where the Listener is listening
bRunning | Bool | Private | Its used for starting and stopping the Listener

Methods

public serverListener(int
            prt)
{
try
{
iPort=prt;
Server_Socket=new
            TcpListener(iPort);
Server_Socket.Start();

Thread thr=new
            Thread(new
            ThreadStart(server_listen));
thr.Start();
}
catch(Exception
            e)
{
throw new Exception ("Error Occurred in
            Listener");
}

}

This initiates the Listener object and starts it through

Server_Socket.Start();

The difference between initializing and starting the TcpListener's object is that when initializing, the port is initialized/opened, but no operation can be performed at that moment. Actually the TcpListener's object starts working when its start method is called. The start method acts as an entry point for its object. This method also starts a Listener thread which continuously listens for any new clients while working in the background.

public void
            server_listen()
{
while(bRunning)
{
try
{
Socket
            sock=Server_Socket.AcceptSocket ();
ClientInfo clt=new
            ClientInfo(sock);
OnClientSignal(this , new
            serverListenerArgs(clt));
}
catch(Exception e)
{
throw
            new Exception("Error Occurred while listening for clients ");
}
}
}

This method accepts requests from the clients which arise as objects of the socket class.

Socket sock=Server_Socket.AcceptSocket();

In the statement above, the thread goes into a wait mode. This means that this statement will not get completely executed until a client socket arrives. When a new client does arrive, its socket is received into a local variable named sock, and then an object of ClientInfo class is instantiated by passing this socket object. After that, an event is thrown to notify the server class that a new client has arrived.

OnClientSignal(this , new serverListenerArgs(clt));

You may be wondering why we created a serverListenerArgs object while we had a ClientInfo object that we could have passed? The answer to this question is that when throwing an event, the first parameter represents the object which is throwing this event (in this case that would be the serverListener class), and the second parameter must be an object of EventArgs (or any class inherited from the EventArgs ). So to achieve our requirement we created another class called serverListenerArgs and inherited it from EventArgs. We'll explain the serverListenerArgs class later.

public void
            close()
{
bRunning=false;
Server_Socket.Stop();
}

This method is simply used to close the Listener thread.

Class Description: ClientInfo

Attributes

Name | Type | Access Modifier | Description
sConnectedSocket | Socket | Private | This object creates a managed version of an Internet transport service
ClientThread | Thread | Private | it is used for each of the client that arrives
strMachineName | string | Private | The name of the machine of each client
strIPAddress | string | Private | The IP address of each client
iPortNumber | int | Private | Its used for starting and stopping the Listener
strUserName | String | Private | The username of each client
alServiceList | ArrayList | Private |
alProcessList | ArrayList | Private |
bRunning | Bool | Private | Its used for starting and stopping the Listener

Methods

public ClientInfo(Socket
            sock)
{
try
{
sConnectedSocket=sock;

string
            temp = sock.RemoteEndPoint.ToString();
StrIPAddress =
            temp.Substring(0,temp.IndexOf(':'));
iPortNumber =
            Convert.ToInt32(temp.Substring(temp.IndexOf(':')+1));

getRemoteMachineName();
getRemoteUserName();

alServiceList = new ArrayList();
alProcessList = new
            ArrayList();

getProcessList();
getServiceList();

}
catch(Exception
            e)
{
MessageBox.Show(e.Message,"Information");
}
}

This constructor gets all the information from the remote client which this object is handling. For example

getRemoteMachineName();

This method gets the machine name from the client. This constructor also initiates a thread to listen to all the requests that this particular client sends. This means that each and every client which is connected to the server has its own thread which is monitoring its activities. This multithreaded approach to handle clients is used so that we can handle all the clients concurrently.

public void
            run_Thread()
{
try
{
Byte[] bReceive = new
            Byte[1024];

while(bRunning)
{
int i =
            sConnectedSocket.Receive( bReceive);
string
            sBuffer=Encoding.ASCII.GetString(bReceive);
}
}
catch(Exception
            e)
{
MessageBox.Show(e.Message);
}

finally
{
sConnectedSocket.Close();
}
}

This is the entry point for a particular client's thread, and it continuously listens for data coming from the client side.

int i = sConnectedSocket.Receive( bReceive);

Notice that every socket has the ability to send and receive data, to or from the machine which it is representing.

The socket class provides methods to achieve this. The Receive method used in the code above waits until the data arrives and then moves the data into a byte array. There are several overloads present for the Receive method. The integer value that it is returning is the number of bytes that it has received.

string sBuffer=Encoding.ASCII.GetString(bReceive);

We use the above statement to convert the byte stream into a string.

public void
            getRemoteMachineName()
{
string strMachineName = "<command
            type=\"Machine_Name\">";
strMachineName +=
            "</command>$";

byte[] bBuffer = new
            byte[1024];
bBuffer =
            Encoding.ASCII.GetBytes(strMachineName);
sConnectedSocket.Send(bBuffer);

sConnectedSocket.Receive
            (bBuffer);
string s =
            Encoding.ASCII.GetString(bBuffer);

while(sConnectedSocket.Available
            >0 )
{
sConnectedSocket.Receive (bBuffer);
s +=
            Encoding.ASCII.GetString(bBuffer);
}

string
            strResultXML = s.Substring(0,s.IndexOf('$'));

XmlDocument
            doc = new
            XmlDocument();
doc.LoadXml(strResultXML);

XmlNode node
            =doc.FirstChild;
if (node.Attributes.GetNamedItem("type").Value
            == "Machine_Name")
{
XmlNode childNode =
            node.FirstChild;
this.strMachineName =
            childNode.Attributes.GetNamedItem("name").Value;
}
else
throw new Exception("Wrong Packet Recieved");

}

This is the first method where actual client server communication is happening. There are a few things you will need to understand before we go into more detail; in socket communication, stream of bytes are passed because the TCP protocol is unaware of the type of data that is passed. To TCP, any type of data, whether it is an object or any predefined type (such as int, string, etc), is simply a collection of bytes. This is the reason why all the send and receive methods in socket programming uses byte arrays as a buffer to store or retrieve data. Therefore we have used string objects to send and receive, as they can easily be converted into byte arrays. The best medium to produce self-describing data in string format is XML.

In recent years, XML has become the standard way to describe data. Although we could have just passed simple strings and still have achieved all the required results, using XML provides the following benefits:

  • Easy interpretation of data using the System.xml classes.
  • Data is self describing, which means that anyone can easily understand what a particular string is representing. Without the use of XML, only the author of the code would understand the particular data.

Now I that we have explained why XML will be used in our application, let's go back to the getRemoteMachineName method.

string strMachineName = "<command type=\"Machine_Name\">";
strMachineName += "</command>$";

This method is preparing a command, in the form of an XML string, to get a machine name. The character $ acts as an end-of-data identifier. Here the string which we have sent acts as a command for the client, ordering it to reply with its machine name.

bBuffer = Encoding.ASCII.GetBytes(strMachineName);
sConnectedSocket.Send(bBuffer);

These statements convert the XML string into byte form. The send method of the socket object sends the data to the application which this socket is representing.

sConnectedSocket.Receive (bBuffer);
string s = Encoding.ASCII.GetString(bBuffer);

The logic here is that whenever the server sends a command to a client, it expects a response. The response is either the data that was requested, or a notification that the operation requested was performed or not. In this case, the server has sent a command for the machine name - it is now waiting for the client to respond to the request and send back its machine name. How the client responds to this command will be explained in the client code description. The receive method receives the clients response.

while(sConnectedSocket.Available >0 )
{
sConnectedSocket.Receive (bBuffer);
s += Encoding.ASCII.GetString(bBuffer);
}

The limitation with the receive method is that it doesn't receive all the data in one go; instead, it obtains it in chunks. Usually the receive method receives 40 bytes at a time. To overcome this problem, we continue receiving this data until it has all arrived. The Available property of the socket object tells us the amount of data that has yet to be received, in other words, the number of bytes available on the stream.

string strResultXML = s.Substring(0,s.IndexOf('$'));

The line above removes the end-of-string identifier from the string received.

XmlDocument doc = new
            XmlDocument();
doc.LoadXml(strResultXML);

Now we come to the response interpretation part. The string received from the client is also in XML form because of the reasons described earlier. The lines above make an XMLDocument object and load the client's response into it. As we know the format of the response, we can easily parse it:

XmlNode node =doc.FirstChild;
if (node.Attributes.GetNamedItem("type").Value == "Machine_Name")
{
XmlNode childNode = node.FirstChild;
this.strMachineName = childNode.Attributes.GetNamedItem("name").Value;
}
else
throw new Exception("Wrong Packet Recieved");

In the above code, we traverse to the first node and check the type. If it is valid, then it moves to its child node and gets its value, which will be the name of the machine. Otherwise, an exception is thrown.

The description of the getRemoteMachineName method explains in detail how we have achieved client server communication. There are several other ways of doing what we have done, but this approach is much more error-safe and close to standards.

public void
            getRemoteUserName()
{
string strUserName = "<command
            type=\"User_Name\">";
strUserName +=
            "</command>$";

byte[] bBuffer = new
            byte[1024];
bBuffer =
            Encoding.ASCII.GetBytes(strUserName);
sConnectedSocket.Send(bBuffer);

sConnectedSocket.Receive (bBuffer);
string s =
            Encoding.ASCII.GetString(bBuffer);

while(sConnectedSocket.Available
            >0 )
{
sConnectedSocket.Receive (bBuffer);
s +=
            Encoding.ASCII.GetString(bBuffer);
}

string strResultXML =
            s.Substring(0,s.IndexOf('$'));

XmlDocument doc = new
            XmlDocument();
doc.LoadXml(strResultXML);

XmlNode node
            =doc.FirstChild;
if (node.Attributes.GetNamedItem("type").Value
            == "User_Name")
{
XmlNode childNode =
            node.FirstChild;
this.strUserName =
            childNode.Attributes.GetNamedItem("name").Value;
}
else
throw new Exception("Wrong Packet Recieved");
}

The working of this getRemoteUserName method is very similar to that of getRemoteMachineName, so we will not go into its details. This method gets the name of the user who is logged into the client's machine.

public void getProcessList()
{
string
            strProcesslist = "<command
            type=\"Process_List\">";
strProcessList +=
            "</command>$";

byte[] bBuffer = new
            byte[3000];
bBuffer =
            Encoding.ASCII.GetBytes(strProcessList);
sConnectedSocket.Send(bBuffer);

sConnectedSocket.Receive (bBuffer);
string s =
            Encoding.ASCII.GetString(bBuffer);

while(sConnectedSocket.Available
            >0 )
{
sConnectedSocket.Receive (bBuffer);
s +=
            Encoding.ASCII.GetString(bBuffer);
}

string strResultXML =
            s.Substring(0,s.IndexOf('$'));

XmlDocument doc = new
            XmlDocument();
doc.LoadXml(strResultXML);

XmlNode node
            =doc.FirstChild;
if (node.Attributes.GetNamedItem("type").Value
            == "Process_List")
{
XmlNodeList nodeList =
            node.ChildNodes;
IEnumerator en =
            nodeList.GetEnumerator();

while
            (en.MoveNext())
{

XmlNode childNode = (XmlNode)
            en.Current;
string pname=
            childNode.Attributes.GetNamedItem("name").Value;
string ptime =
            childNode.Attributes.GetNamedItem("time").Value;
ProcessObject
            processObject = new ProcessObject(pname,ptime);
strProcessList.Add(processObject);
}
}
else
throw
            new Exception("Wrong Packet Recieved");
}

This getProcessList method obtains the list of processes of the particular client. The command and receive part of this method is the same as in the previous methods, but it differs in the way the response is interpreted. In the other two methods discussed above, the result was a single string such as machine name or username, but in this case the response is a list of processes with several properties. We have defined a class named ProcessObject to represent a process which will be discussed later.

XmlNodeList nodeList = node.ChildNodes;

The above statement gets a list of all the child nodes of the resulting parent node. Each child node describes a single process.

      IEnumerator
            en = nodeList.GetEnumerator();
while
            (en.MoveNext())
{
XmlNode childNode = (XmlNode) en.Current;
string pname=
            childNode.Attributes.GetNamedItem("name").Value;

string ptime
            = childNode.Attributes.GetNamedItem("time").Value;
ProcessObject
            processObject = new ProcessObject(pname,ptime);
strProcessList.Add(processObject);
}

We traverse through each child node and obtain two values. The first one is the name of process and the second one is the time at which the process was started. Once we get both these values we create an object of ProcessObject :

new ProcessObject(pname,ptime);

We then add this newly created abject to the container strProcessList.

public void getServiceList()
{
string
            strServiceList = "<command
            type=\"Service_List\">";
strServiceList +=
            "</command>$";

byte[] bBuffer = new
            byte[3000];
bBuffer =
            Encoding.ASCII.GetBytes(strServiceList);
sConnectedSocket.Send(bBuffer);

sConnectedSocket.Receive (bBuffer);
string s =
            Encoding.ASCII.GetString(bBuffer);

while(sConnectedSocket.Available
            >0 )
{
sConnectedSocket.Receive (bBuffer);
s +=
            Encoding.ASCII.GetString(bBuffer);
}

string strResultXML =
            s.Substring(0,s.IndexOf('$'));

XmlDocument doc = new
            XmlDocument();
doc.LoadXml(strResultXML);

XmlNode node
            =doc.FirstChild;
if (node.Attributes.GetNamedItem("type").Value
            == "Service_List")
{
XmlNodeList nodeList =
            node.ChildNodes;
IEnumerator en =
            nodeList.GetEnumerator();

while
            (en.MoveNext())
{
XmlNode childNode = (XmlNode) en.Current;
string sname=
            childNode.Attributes.GetNamedItem("name").Value;
string stype =
            childNode.Attributes.GetNamedItem("type").Value;
string sstatus =
            childNode.Attributes.GetNamedItem("status").Value;
ServiceObject
            serviceObject = new ServiceObject(sname,stype,sstatus);
strServiceList.Add(serviceObject);
}
}
else
throw
            new Exception("Wrong Packet Recieved");
}

This method is similar to getProcessList; it receives the list of all the windows services running on the client machine. Each service is represented by a ServiceObject class which will be discussed later.

public bool stopService(string
            servicename)
{
string strServiceString = "<command
            type=\"Stop_Service\">";
strServiceString += "<param
            name=\""+servicename+"\"></param>";
strServiceString +=
            "</command>$";

byte[] bBuffer = new
            byte[1024];
bBuffer =
            Encoding.ASCII.GetBytes(strServiceString);
sConnectedSocket.Send(bBuffer);

sConnectedSocket.Receive
            (bBuffer);
string s =
            Encoding.ASCII.GetString(bBuffer);

while(sConnectedSocket.Available
            >0 )
{
sConnectedSocket.Receive (bBuffer);
s +=
            Encoding.ASCII.GetString(bBuffer);
}

string strResultXML =
            s.Substring(0,s.IndexOf('$'));
XmlDocument doc = new
            XmlDocument();
doc.LoadXml(strResultXML);

XmlNode node
            =doc.FirstChild;
if (node.Attributes.GetNamedItem("type").Value
            == "Stop_Service")
{
XmlNode childNode =
            node.FirstChild;
string a =
            childNode.Attributes.GetNamedItem("result").Value;
if (a ==
            "Success")
return true;
else
return
            false;
}
else
{
throw new Exception("Wrong Packet
            Recieved");
return false;
}
}

This method is different from the other methods in the way that it is performed due to the request of a user. Whenever the user needs to stop any service that is running on its machine, it sends the request along with the service name to this method. The sending and receiving part of this method are similar to the previous ones.

XmlNode node =doc.FirstChild;
if (node.Attributes.GetNamedItem("type").Value == "Stop_Service")
{
XmlNode childNode = node.FirstChild;
string a = childNode.Attributes.GetNamedItem("result").Value;
if (a == "Success")
return true;
else
return false;
}
else
{
throw new Exception("Wrong Packet Recieved");
return false;
}

Since this method is not requesting any information, but is asking the client to perform an operation, the response will not be any information but just an acknowledgement which will tell the server whether the requested operation was successfully performed or not.

Class Description: serverListenerArgs

This class is inherited from the EventArgs class. The reason behind using this class is that in event driven programming, when an event is throw,n the parameters passed to the event handler must be through an object of EventArgs class or any of its derived classes. Hence the object of this class is instantiated whenever a listener thread receives a new client and it throws an OnClientSignal event. This class has only one attribute, a ClientInfo object. In short, serverListenerArgs provides a transportation medium for ClientInfo objects to be sent from listener to server class.

Attributes

Name | Type | Access Modifier | Description
ClientInfo | ClientInfo | Private | The client information object which will be passed to the server class.

Class Description: ProcessObject

This class just acts as a user defined type for representing processes. An object of the process class itself has many attributes and methods, but of two of them are of use to us. As we have discussed above, data transferring in Socket programming is restricted in that only byte streams can be sent, so we cannot send objects. What we have done is fetched the information from the Process class object (such as process name and process start time) and sent it through the stream. Since this information wasn't complete enough to make a new Process instance, we have made our own class which is in a sense representative of the process class.

Attributes

Name | Type | Access Modifier | Description
StrStrProcessName | String | Private | Specifies the name of that particular process
StrProcessStartTime | String | Private | Specifies the time the process started

Class Description: ServiceObject

Like the ProcessObject class, ServiceObject is a user defined type to represent Services.

Attributes

Name | Type | Access Modifier | Description
StrDisplayName | String | Private | Specifies the name of that particular service
strServiceType | String | Private | Specifies the type of the service.
strStatus | String | Private | Status of service (Stopped, Started, Paused, etc.)

Client Description

Class Description: netClient

Attributes

Name | Type | Access Modifier | Description
ClientSocket | TcpClient | Private | The main object that connects the client to the server
iServerPort | Int | Private | Specifies the port of the server where the client is to be connected
sServerIP | String | Private | Specifies the IP Address of the server
sMachineName | String | Private | The client's machine name
sUserName | String | Private | The name of the user who has logged onto the client's machine
NetworkStream | NetworkStream | Private | Provides the underlying stream of data for the network access.
Services | ServiceController [] | Private | List of ServiceController instances, each representing a single service available on the clients machine
Processes | Prosess[] | Private | List of processes running on the client's machine
bRunning | Bool | Private | It is used for starting and stopping the client

Methods

public netclient(string ipaddr, int prt)
{
iServerPort=prt;
sServerIP=ipaddr;

Services = new ServiceController[50];
refreshServiceList();

Processes = new Process[20];
refreshProcessList();

sMachineName = SystemInformation.ComputerName;
sUserName = SystemInformation.SUserName;

ClientSocket=new TcpClient();
ClientSocket.Connect(sServerIP,iServerPort);
Console.WriteLine("Client Connected to Server");

ClientSocket.ReceiveBufferSize = 3000;
ClientSocket.SendBufferSize = 3000;
NetworkStream = ClientSocket.GetStream();

if(NetworkStream.CanWrite && NetworkStream.CanRead)
{
try
{
while (bRunning)
{
Byte[] recBytes = new Byte[1024];
int a= NetworkStream.Read(recBytes,0,1024);
string sBuffer = Encoding.ASCII.GetString(recBytes);
sBuffer = sBuffer.Substring(0,sBuffer.IndexOf('$'));

XmlDocument doc = new XmlDocument();
doc.LoadXml(sBuffer);

XmlNode node = doc.FirstChild;
XmlNode childNode;

string sCommandType = node.Attributes.GetNamedItem("type").Value;

switch (sCommandType)
{
case "User_Name":
sendUserName();
break;
case "Machine_Name":

sendMachineName();
break;
case "Service_List":
SendServiceList();
break;
case "Process_List":

SendProcessList();
break;
case "Restart_Client":
restartClient();
break;
case "Shutdown_Client":
shutdownClient();
break;
case "Pause_Service":
childNode = node.FirstChild;

pauseService(childNode.Attributes.GetNamedItem("name").Value);
break;
case "Stop_Service":
childNode = node.FirstChild;

StopService(childNode.Attributes.GetNamedItem("name").Value);
break;
case "Resume_Service":
childNode = node.FirstChild;
resumeService(childNode.Attributes.GetNamedItem("name").Value);
break;
}
}
}
catch(Exception e)
{
Console.WriteLine(e.Message);
}
}
Console.WriteLine("Press any key to continue..");
Console.Read();
}

The netclient method comprises the main functionality of the client. It gathers all the client information, receives all the servers commands, perform actions in accordance to those commands, etc. Now we'll discuss this method in a modular fashion.

iServerPort=prt;
sServerIP=ipaddr;

Services = new ServiceController[50];
refreshServiceList();

Processes = new Process[20];
refreshProcessList();

sMachineName = SystemInformation.ComputerName;
sUserName = SystemInformation.SUserName;

The first thing this method does is to gather all the information about itself and the servers where it has to be connected. The functionality of refreshServiceList and refreshProcessList will be discussed later. SystemInformation is a class from the System.Windows.Forms namespace; it gives all the low level information about the computer. A detailed description of this class is unnecessary with respect to our case study. We have used this class to get the machine and user name.

       ClientSocket=new
            TcpClient();
ClientSocket.Connect(sServerIP,iServerPort);

The above lines are the main part of any client. The first line creates an instance of TcpClient's class. The connect method of this class has several overloads, but the one we have used takes two parameters. The first one is the IP Address of the server, and the second one is the port at which the server is listening for clients. These two parameters must be known before any connection request is sent. The connect method prepares the socket object of the client with information such as IP Address, port number at which the client application is running, the protocol type which is being used for the transmission, and some other information. It makes a packet of them, and sends it through the network to the server whose IP was specified in the parameters, and the application running on that server whose port number was specified.

ClientSocket.ReceiveBufferSize = 3000;
ClientSocket.SendBufferSize = 3000;

Once the client is connected to the server, a few things are done before actually sending or receiving data. The code above sets the size of the send and receive buffer of the client. Every TcpClient instance has two buffers associated with it, one where all the data arrives and the other from where all the data is sent. Different computers have different send and receive rates, and because of this, buffers are used. Buffer sizes are application dependent; large sized buffers are used for the applications in which the amount of data is large.

NetworkStream = ClientSocket.GetStream();

The GetStream method gets an instance of the NetworkStream class. Each TcpClient has its own stream, through which it can send and receive data. By providing separate streams for each TcpClient instance, developers have enforced a level of security, so that if two or more clients are running on same machine each of them get their own data, although all the data is coming from the same medium.

Byte[] recBytes = new Byte[1024];
int a= NetworkStream.Read(recBytes,0,1024);
string sBuffer = Encoding.ASCII.GetString(recBytes);
sBuffer = sBuffer.Substring(0,sBuffer.IndexOf('$'));

As the NetworkStream class is inherited from the Stream class, like all other NetworkStream instances it has the ability to read and write. The above code is where the client listens for commands from the server. The Read method used here waits until data arrives from serve. It has several overloads, but the one we have used has three parameters: the first one is the buffer in which data will be received, the second one is the offset telling it from which position in the byte array it has to start writing data, and the third is the number of bytes to be read. The last two statements of the code perform the same function as in the server i.e. of converting byte array intro string and removing the end-of-string character.

XmlDocument doc = new XmlDocument();
doc.LoadXml(sBuffer);

XmlNode node = doc.FirstChild;
XmlNode childNode;

string sCommandType = node.Attributes.GetNamedItem("type").Value

Once the data has arrived it has to be interpreted - again the same method is applied by parsing the XML string, and getting the type of the command. The last statement gets the type of the command sent.

switch (sCommandType)
{
case "User_Name":
sendUserName();
break;

case "Machine_Name":
sendMachineName();
break;

//Rest of the code
}

Once we have received the type of command, we take the appropriate action. On each type of command we are calling a method to perform the requested action and send a response back to the server.

public void sendMachineName()
{
string
            sMachineNameXML;
sMachineNameXML = "<result
            type=\"Machine_Name\">";
sMachineNameXML += "<params
            name=\"" + sMachineName + "\" />";
sMachineNameXML +=
            "</result>$";

byte[] bBuffer = new
            byte[1024];
bBuffer = Encoding.ASCII.GetBytes(sMachineNameXML);
NetworkStream.Write(bBuffer,0,machineNameXML.Length);
}

The above sendMachineName method is called whenever a request for the Client's machine name arrives. It prepares an XML string with the requested machine name in parameters, and sends this response back to the server. We are just going to take a brief look at the write method. Like the read method, write is also inherited from the stream class, and it also has several overloads available. The one we have used has three parameters: the first one is the reference to the buffer from which data has to be sent, the second one is the offset in the buffer, and the third is the number of bytes to be sent.

public void sendUserName()
{
string
            sUserNameXML ;
sUserNameXML = "<result
            type=\"User_Name\">";
sUserNameXML += "<params name=\"" +
            sUserName + "\" />";
sUserNameXML +=
            "</result>$";
byte[] bBuffer = new byte[1024];
bBuffer =
            Encoding.ASCII.GetBytes(sUserNameXML);
NetworkStream.Write(bBuffer,0,sUserNameXML.Length);
}

This sendUserName method is used to retrieve the user name from the client. Whenever a request for the Client's user name arrives, an XML string is generated that include the requested user name in parameters, and sends this response back to the server.

public void sendProcessList()
{
string
            sProcessXML = "";
sProcessXML += "<result
            type=\"Process_List\">";
IEnumerator en =
            Processes.GetEnumerator();
while (en.MoveNext())
{
Process
            process = (Process) en.Current;
sProcessXML += "<params
            name=\""+process.StrProcessName+"\"
            time=\""+process.StartTime.ToString()+"\"
            />";
}
sProcessXML += "</result>$";
byte[]
            bBuffer = new byte[3000];
bBuffer =
            Encoding.ASCII.GetBytes(sProcessXML);
NetworkStream.Write(bBuffer,0,sProcessXML.Length);

}

The sendProcessList method sends the list of the processes running on the client to server. This method differs to the other methods in the way the response is generated. Since in this method we have to generate a list of processes with their attributes, the XML generating code is a little different. Once a complete response is generated, it is sent back to the server.

public void SendServiceList()
{
string
            sServiceXML = "";
sServiceXML += "<result
            type=\"Service_List\">";
IEnumerator en =
            Services.GetEnumerator();
while
            (en.MoveNext())
{
ServiceController service =
            (ServiceController) en.Current;
sServiceXML += "<params
            name=\""+service.DisplayName+"\"
type=\""+service.ServiceType.ToString()+"\"
status=\""+service.Status.ToString()+"\"
            />";
}
sServiceXML += "</result>$";
byte[] bBuffer
            = new byte[3000];
bBuffer = Encoding.ASCII.GetBytes(sServiceXML);
NetworkStream.Write(bBuffer,0,sServiceXML.Length);
}

The sendServiceList method is more or less like the sendProcessList, the only difference is that instead of the process list, the requested service list of that particular client is sent to the server.

 public bool stopService(string
            serviceName)
{
try
{
IEnumerator en =
            Services.GetEnumerator();
while
            (en.MoveNext())
{
ServiceController service =
            (ServiceController) en.Current;
if (service.DisplayName ==
            serviceName)
service.Stop();
}
refreshServiceList();
string strResultXML;
strResultXML = "<result
            type=\"Stop_Service\">";
strResultXML += "<params
            result=\"Success\"/>";
strResultXML +=
            "</result>$";
byte[] bBuffer = new byte[3000];
bBuffer =
            Encoding.ASCII.GetBytes(strResultXML);
NetworkStream.Write(bBuffer,0,strResultXML.Length);
return
            true;
}
catch(Exception e)
{
string strResultXML =
            "<result type=\"Stop_Service\">";
strResultXML +=
            "<params result=\"Failure\"/>";
strResultXML +=
            "</result>$";
byte[] bBuffer = new byte[3000];
bBuffer =
            Encoding.ASCII.GetBytes(strResultXML);
NetworkStream.Write(bBuffer,0,strResultXML.Length);
return
            false;
}
}

The above method is used to stop a particular service of the client. The name of the service is received in the parameters of the stopService method. The ServiceController class is used here to stop the service, this class is used to connect to and control the existing windows services. Every instance of the service controller class has a Stop method, which stops the service. If the service cannot be stopped, it throws an InvalidOperationException, so in our case, if the service is stopped successfully, we prepare a result message with success as its parameter, else a failure response is sent back to the server.

Designing The User Interface

Our main goal in this application was to demonstrate network programming in C# using sockets, therefore the Graphical User Interface of this application was not the main focus. The server side comprises of a UI, as it is going to perform all the operations, while the client side does not have any UI.

Now we will be discussing each screen, as the operations are performed.

Main Screen

Above is the main screen when the server is executed. This screen contains information on each client that has been logged on. This is the only screen, so all the operations will be performed here. You can find this form in the code download that accompanies this case study. Let's discuss each section.

Name | Type | Description
lbClientsConnected | List Box | This list box contains all the clients that have been connected to the server.
tbComputerName | Text Box | When we select a client from the client connected list, the name of that particular client is displayed here
tbIPAddress | Text Box | When we select a client from the client connected list, the IP Address of the client is displayed here
tbUserName | Text Box | The name of the user of that client is displayed here
tbServiceAccessPoint | Text Box | The port of the server at which the client is connected
tbCommunicationProtocol | Text Box | The protocol that client is using e.g TCP, UDP
lbListofProcesses | List Box | It contain the name of the list of processes that are running on the clients computer
lbListofServices | List Box | It contain the name of the list of services that are available on the clients computer
tbStrProcessName | Text Box | When the process from the List of Process is selected, the name of that process goes here
tbProcessStartTime | Text Box | When the process from the List of Process is selected, the time the process started goes here

Whenever the client is connected to the server, its name is displayed in the Clients Connected list. The server administrator can then observe all the information regarding that client on the screen. The Process List shows all the processes that are running on the client's side, and similarly, the Service List shows all the services available on the client side.

The server can stop the client by pressing the Stop Client button; this button will disconnect the client from the server. The server can also perform operations on the services available from the client, that is, it can pause, resume, or stop the service. This can be done by selecting a service from the Service List and then pressing the respective button. For example, the service can be stopped by selecting it from the Server List and then pressing the Stop Service button. Selecting the process from the process List and then pressing the Stop Process button can only kill the process.

Now well show how a connection is established, and how a service is killed.

The above screen shows the client connected to the server.

The above screen demonstrates what happens when the client is selected. As you can see, all the information described earlier is displayed in the respective boxes. In this screen, the Process list is selected, which means that all the processes running on that particular client are shown.

The above screen is similar to the previous one, the only difference is that the Service List is selected here, which means that all the services available on the client are shown.

In this screen shot, one of the operations is demonstrated. By selecting any service from this list and then pressing any button at the top of the screen, the respective operation is performed. As can be seen, the ASP. NET State Service 's status is set to be stopped (highlighted).

Conclusion

While using .NET, I realized a need to demonstrate the networking concepts in C#. One can certainly read lots about these concepts, but what I wanted was a real application that shows the basic concepts of networking, like client server connection for example, in C#. For this reason, I made up a simple client server application using Sockets and TCP protocol. This application uses the System.Net namespace that includes base classes for TCP server and clients provided by .NET.

The case study illustrates some basic knowledge about .NET technologies, as well as some other technologies such as threads, XML, etc. As the key objective of the case study was to demonstrate Socket programming, a whole section is devoted to the Sockets and TCP concepts. It shows the networking concepts and describes in detail the OSI model, mapping of TCP over OSI model, Sockets, Socket programming, the System.Net namespace, etc. After going through all the technologies, the case study describes how Sockets can be used in C# by creating a small application.

The application is a simple client/server application in which multiple clients connect to a single server through sockets. This application uses TCP for the communication; it is a connection oriented protocol which is highly reliable as it sends acknowledgements for each request and response. The application allows the server to view all the client's information such as its name, its IP Address, the protocol it is using, the port it is connected to, the user's name, etc. The server can also view the processes and services running on the client side. It has the ability to make changes on the client's machine, such as to make the client stop, make the service stop, pause, or resume, or make the process stop.


Introducing Smart Device Extensions and the .NET Compact Framework

Introducing Smart Device Extensions and the .NET Compact Framework
http://www.csharptoday.com/content.asp?id=1884
Article Rating: 3.5
Published on September 16, 2002
Abstract
The Smart Device Extensions for Visual Studio .NET provides an environment for the development of .NET applications targeted at Pocket PC, Windows CE .NET, and other smart devices. In this case study, Tony Loton provides advice on installing and configuring the Smart Device Extensions Beta 1 and discusses the relationship between these Visual Studio .NET extensions and the .NET Compact Framework APIs. He offers a simple – but interesting – first application that may be deployed to a real device or to the SDE emulator and introduces some ideas for novel speech-based applications ideally suited to the smart device platform.
Case Study

The Smart Device Extensions for Visual Studio .NET provides an environment for the development of .NET applications targeted at Pocket PC, Windows CE .NET, and other smart devices. In this case study, Tony Loton provides advice on installing and configuring the Smart Device Extensions Beta 1 and discusses the relationship between these Visual Studio .NET extensions and the .NET Compact Framework APIs. He offers a simple - but interesting - first application that may be deployed to a real device or to the SDE emulator and introduces some ideas for novel speech-based applications ideally suited to the smart device platform.

Introduction

In mid-2002, Microsoft released the Smart Device Extensions for Visual Studio .NET Beta 1, incorporating the .NET Compact Framework. "Great!" I hear you say, but what are they?

  • The Smart Device Extensions for Visual Studio .NET enables developers to develop, debug, and deploy applications for devices running the .NET Compact Framework. In a nutshell, it adds smart-device-specific features to the Visual Studio .NET IDE.
  • The .NET Compact Framework is a subset of the .NET Framework that is designed to run on a smart device, providing support for managed code and XML web services. So it's a cut-down set of .NET classes biased in favor of the client-side features.

In this introductory case study we'll discuss:

  • Installation and configuration of the Smart Device Extensions Beta 1. It's a logical starting point, and you can take advantage of the fact that I've already overcome many of the difficulties that you may face.
  • How to test the installation and how to resolve deployment problems.
  • The implementation of a first example application that accesses a native device function in order to play sampled sounds.
  • My ideas for future speech-based applications targeted at the Pocket PC platform.

Before moving on I'll just state up-front how developer-friendly the Smart Device Extensions environment is in comparison with the Symbian SDK that I have experimented with earlier. For me, it's now the environment of choice for smart device application development. Keep that positive statement in your mind as motivation while you work through the installation and configuration tips presented later.

What is a Smart Device?

The term smart device refers to Personal Digital Assistants and handheld computers, like the Psion Series 5 or the Compaq iPaq. Within the last year or so, these kinds of devices have become even smarter with the incorporation of an integral mobile phone. I'm thinking here of the Nokia 9210, which I used to own, and the O2 XDA of which I am now the proud owner.

These devices fall roughly into two camps, the Symbian camp (comprising Psion, Nokia, and some other mobile phone manufacturers) and the Microsoft camp. For the purposes of this piece, you can disregard the Psion Series 5 and Nokia 9210 completely as they're Symbian devices. You can rule in any of those devices that come labeled as Pocket PC, Pocket PC 2002, Windows CE, and Pocket PC Phone Edition.

The main thing to understand about smart devices - in contrast to desktop / laptop PCs - is that they are constrained devices: constrained in terms of memory (typically 32Mb-64Mb for RAM and permanent storage), screen size (typically 320 x 240 pixels), and bandwidth (usually connected via a mobile phone, integral or external, at 9600 bps GSM or 56 kbps GPRS). These constraints are important factors to take into account when designing applications for smart devices.

Oh, and it's because of device constraints that I decided to switch my development effort from the Nokia 9210 / Symbian SDK combination to the Pocket PC / VS.NET SDE combination. The Symbian SDK that I used was the Java variant, which suffered from the unfortunate side effect that by the time the Java Virtual Machine had been loaded into the Nokia's limited memory there was no memory left to run my applications! So far so good with the .NET Compact Framework though, which seems to load onto a device with plenty of room to spare.

What's in the .NET Compact Framework?

The compact framework shares with the full .NET framework a common language runtime, assemblies and file formats (but not binary compatibility), multi language support (but currently only Visual C# and Visual Basic .NET), a common type system, and a threading model. More information can be found at ms-help://MS.VSCC/MS.MSDNVS/ms.embeddedtoolssdk/evtuv/html/etconsimilaritieswithnetframework.htm.

It does not share support for application domains, ASP .NET, COM interoperability and callback functions (but you can use platform invoke as described later), asynchronous delegates, application profiling, binary serialization, classes from the System.Data.OleDb namespace, XML schema validation and XPath queries. More information can be found at ms-help://MS.VSCC/MS.MSDNVS/ms.embeddedtoolssdk/evtuv/html/etcondifferenceswithnetframework.htm.

On the whole the compact framework is constrained in order to cope with the low memory and low bandwidth characteristics of smart devices. As I understand it, the XML schema validation is not supported due to memory and processor speed limitations. The features that have been retained from the full .NET framework are those that help to position smart devices as a client platform, thus data access will typically be via a server-side web service rather than via direct communication with the database.

The .NET Compact Framework class library supports a subset of the .NET Framework class library. It contains additional classes exclusively designed for the .NET Compact Framework which are limited (in the Beta 1 at least) to Infrared Data Association (IrDA) classes and SQL Server CE Classes. The former may be found in the System.Net namespace, and the latter in the System.Data.SqlServerCe namespace. You can find more information about the exclusive classes in help topic ms-help://MS.VSCC/MS.MSDNVS/ms.embeddedtoolssdk/evtuv/html/etconClassesExclusiveToNETCompactFramework.htm.

System Requirements

For the remainder of this case study you will need the Visual Studio .NET IDE plus the Smart Device Extensions for Visual Studio .NET Beta 1. The latter can be found included in the July 2002 MSDN CD distribution and - at the time of writing - is available by registering for the beta program at URL http://msdn.microsoft.com/vstudio/device/sdebeta.asp?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq.

Optionally you will need a real device, Pocket PC or Windows CE .NET, on which to deploy your applications. That's not mandatory because the Smart Device Extensions (from now on SDE) includes a very effective device emulator, so in theory you could write your first smart device applications without a smart device. Everything in this case study has been tested using the emulator and a real device, the device being an O2 XDA (http://www.o2.co.uk/business/productsservices/xda?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq) which generically is a Pocket PC Phone Edition.

Defining the Problem

For this case study, our initial problem statement will be:

"How to install and configure the Visual Studio .NET Smart Device Extensions, thereby allowing us to write a first smart device application using the .NET Compact Framework."

The problem will subsequently expand to incorporate:

"How do we access native device functions that are otherwise not (yet) encapsulated with .NET Compact Framework classes?"

Case Study Structure

In the Defining the Solution section I will offer advice on:

  • Installing the Smart Device Extensions and .NET Compact Framework
  • Testing the Installation
  • Resolving Deployment Problems
  • Closing the Emulator and Redeployment

In The Sample Application section I will discuss:

  • Component Design for the application, incorporating platform Native API Access
  • Component Coding of a class to play sample sounds through the device speaker
  • UI Design and UI Code for a simple form that responds to button clicks
  • Application Set-up
  • Debugging
  • Limitations and Further Work for this application, including speech synthesis and problems of playing back of .wav files

I review the case study and present some ideas for future applications in my Conclusion at the end.

Defining the Solution

There will be two aspects to my solution:

First we will look at a set of tips on installing, configuring, and testing the installation of the Smart Device Extensions. Here you will benefit from my own experiences of trying, failing, and finally succeeding in getting the whole thing up-and-running.

Second, a fun sample application that shows in particular how you can step outside the limitations of the current .NET Compact Framework by calling native device functions directly.

Installing the Smart Device Extensions and .NET Compact Framework

A single installation executable is provided with the SDE distribution, along with a README.HTML file that helps you resolve any installation and configuration problems.

I predict that your installation will either go very smoothly, or very badly. I've been in both situations, so I am able to distill my experiences - good and bad - into a list of tips that should help you to avoid the potential pitfalls. It looks more frightening than it is, and once it's all up-and-running you can trust me that it's quite effortless in use. The ordering of this list is important!

  • If you have a spare PC that is not crucial to your operations, install there first. My first installation attempts ground my PC to a halt consistently at the same point, and from then on each reboot prompted for the installation of a new hardware device - some kind of Virtual PC Application as far as I can remember - which also caused the PC to hang. Admittedly I had an odd combination of the .NET command line SDK plus Visual Studio .NET, plus a not very convincing ActiveSync installation (see later), but nothing that was not resolved by reinstalling the entire machine (including Windows XP) from scratch. Sounds a bit drastic, I know.
  • Install the Microsoft TCP Loopback adapter. This is vital if your PC is not networked and you want to deploy to the emulator as well as a real device. Instructions for installing this adapter are included in the README.HTML file, and it's not necessary if your PC is on a network. even a dial-up network!
  • Disable the Internet Connection Firewall. The README.HTML file suggests that deployment problems may occur if the Internet Connection Firewall is enabled. I can confirm that it's true, and that's probably a good reason to run on a disconnected PC using the loopback adapter.
  • Install Microsoft ActiveSync. This should be on the CD that came with your device, or you can download it from http://www.microsoft.com/mobile/pocketpc/downloads/activesync35.asp?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq. If you have a real device, make sure you can connect to it now using ActiveSync.
  • Install the Smart Device Extensions and .NET Compact Framework. This will modify your Visual Studio .NET installation.

Testing the Installation

You can check that the installation has been successful by launching the New Project dialog. You will see a new kind of project called a Smart Device Application, as shown below, but we won't create one just yet.

You will also see that the help content includes a new category called Smart Device Extensions, with various new topics.

To test the installation fully we will build, deploy, and run the Calculator application shown in the previous figure. If you click the Calculator Sample document you will find instructions on how to locate the relevant Visual Studio solution. I located the Pocket PC / C# version of the solution in directory C:\Program Files\Microsoft Visual Studio .NET\CompactFrameworkSDK\v1.0.3300\Samples\PocketPC\VC#\Calculator.

Once you've located a version of the solution - in Visual Basic if you prefer, or for Windows CE .NET - you can choose the Debug / Start without Debugging menu option to compile, deploy, and run it. A dialog will appear asking you whether to deploy to your real device (if you have one) or to the emulator:

Let's assume you'll choose the second option, Pocket PC Emulator, as you might prefer during the development phase. At this point you should see some reassuring messages in the Visual Studio .NET output window like this:

------ Build started: Project: Calculator, Configuration: Debug Pocket PC ------

Preparing resources...
Updating references...
Performing main compilation...

Build complete -- 0 errors, 0 warnings
Building satellite assemblies...
Visual Studio is ready to deploy Calculator

------ Deploy started: Project: Calculator, Configuration: Debug Pocket PC ------

Deploying to Pocket PC Emulator using Emulation
Connected to Pocket PC Emulator (Pocket PC) running on X86.
Copying files from 'C:\Program Files\Microsoft Visual Studio .NET\CompactFrameworkSDK\v1.0.3300\
Samples\PocketPC\VC#\Calculator\bin\Debug' to '\Windows'
Copying Calculator.exe
Copying netcf.cjk.ppc3.x86.cab
Launching device installation of 'netcf.cjk.ppc3.x86.cab'. Please check device screen
for further instructions...................

---------------------- Done ----------------------

Build: 1 succeeded, 0 failed, 0 skipped
Deploy: 1 succeeded, 0 failed, 0 skipped

Launching the Application

The emulator will be launched in a separate window and you will see the progress of the deployment on the device itself:

Finally you will be rewarded with the Calculator form:

Did it all work out for you? If so, you can skip the next section.

Resolving Deployment Problems

My real device is an O2 XDA, which generically is a Pocket PC Phone Edition. Deployment to that device works pretty consistently as long as it's connected. However, deployment to the emulator can be a more hit-and-miss affair and in my experience it's usually a networking problem.

Here are some tips:

  • If you previously had a network connection, connect to a network once again. even if that means establishing a dial-up connection to the Internet. Try the deployment while connected.
  • If you've never had a network connection, make sure the Microsoft TCP Lookback adapter is enabled.
  • In all cases, disable the Internet Connection Firewall.

If my tips don't help then you're on your own and it's time to search the MSDN news groups. I found an interesting posting along the lines of.

".and deployment failed until I disabled the Microsoft TCP Loopback adapter."

Strange when you consider what I've already said, but probably true in some situations, so best to try it both ways.

Closing the Emulator and Redeployment

After deploying and testing an application on the emulator it will be tempting to close down the emulator by clicking the red X in the top-right corner of the emulator window. In that case you will see the following dialog that invites you to Save Emulator State or Turn off Emulator :

Note that if you're repeatedly redeploying an application that you are developing, you do not strictly need to close down the emulator and restart it each time. I have often closed down just the application-under-test by clicking the OK button on the application form. Subsequently redeploying the application with the emulator still running seems to work fine, and it's much quicker.

The Sample Application

Having got the Smart Device Extensions environment installed and configured, we'll now create a sample application that will give you a feel for development using the .NET Compact Framework, as well as demonstrating how to break out of the limited .NET Compact Framework functionality by invoking native device functions directly.

My original idea was to create a BeatBox application that would allow rhythms to be tapped out on the Pocket PC screen, by coupling up sampled (.wav) sound effects to on-screen buttons. It's probably not a good idea to tap frantically on the screen, so I've decided to change the name from BeatBox to SoundBox. We'll still have some drum sounds, but also a range of fun sound effects like screams and gun shots. Think of it as an executive stress toy, or as something to annoy your friends and family.

Component Design

There is a serious side to this application, which stems from the fact that we cannot play sounds on the device via the .NET Compact Framework classes themselves. So to achieve this feat we'll need to access a capability of the underlying device platform.

Native API Access

The Smart Device Extensions documentation (section ms-help://MS.VSCC/MS.MSDNVS/ms.embeddedtoolssdk/evtuv/html/etconnativeapiaccess.htm in Visual Studio .NET) states:

The .NET Compact Framework provides limited support for interoperating with native APIs. It supports accessing functions in native DLLs, but does not support creating COM objects in managed code (COM interop) or interoperating with ActiveX controls.

It goes on to say:

You can, however, call an unmanaged DLL by using platform invoke (PInvoke) with . the DLLImport attribute in Visual C#. PInvoke allows managed code to call unmanaged code and accesses APIs through standard DLL entry points.

That's how we'll solve the problem of playing sounds on the device. We'll create a class to hold a DLL function, thereby exposing an entry point to a COM object. Our class, called Sound, will be defined in the next section.

I won't go into much more detail about accessing functions in native DLLs because for the purposes of this case study it was just a means to an end. I just wanted a way to play sampled sounds in my demonstration application, with a view ultimately to realizing my aim of developing a rudimentary Pocket PC speech synthesizer (mentioned later).

In any case this technique is not limited to smart device development, so the hardcore platform SDK programmers amongst you can read more about the technicalities in the .NET Framework Developer's Guide "Consuming Unmanaged DLL Functions" at ms-help://MS.VSCC/MS.MSDNVS/cpguide/html/cpconconsumingunmanageddllfunctions.htm.

Component Coding (for class "Sound")

The Smart Device Extensions documentation (section ms-help://MS.VSCC/MS.MSDNVS/ms.embeddedtoolssdk/evtuv/html/etconPlayingSoundsUsingPInvoke.htm in Visual Studio .NET help) provides some example code (but not a complete sample application) under the heading "Playing Sounds Using PInvoke". I took that example as a starting point and rationalized the two sample classes - Helpers and Sound - to the one class that follows:

using System;

We must import the following System.Runtime.InteropServices namespace:

using System.Runtime.InteropServices;

namespace SoundBox
{

public class Sound
{
public Sound()
{
}

The original example shows how we can define a set of flags to control how the sound will be played. We will combine these flags using a bitwise OR.

[Flags]
public enum SoundFlags : int
{
SND_SYNC = 0x0000,
SND_ASYNC = 0x0001,
SND_NODEFAULT = 0x0002,
SND_MEMORY = 0x0004,
SND_LOOP = 0x0008,
SND_NOSTOP = 0x0010,
SND_NOWAIT = 0x00002000,
SND_ALIAS = 0x00010000,
SND_ALIAS_ID = 0x00110000,
SND_FILENAME = 0x00020000,
SND_RESOURCE = 0x00040004
}

Here is the DllImport attribute that specifies the DLL containing our target native function:

[DllImport("coredll")]

and this is our .NET wrapper for that function:

public static extern bool PlaySound( string szSound, IntPtr hMod, SoundFlags flags );

Finally we'll allow other classes to play a sound by making a method call of the form Sound.play("mySound.wav") :

public static void play(string fileName)
{
PlaySound( fileName, IntPtr.Zero, SoundFlags.SND_FILENAME | SoundFlags.SND_ASYNC );
}

}

}

Notice how we've combined the SoundFlags according to how the sound should be played. In a nutshell, we want to play the sound directly from a .wav file (SoundFlags.SND_FILENAME) and we want the playback to be asynchronous so that the application remains responsive (SoundFlags.SND_ASYNC). Feel free to experiment with other combinations of flags.

Now we need a user interface.

User Interface Design

When creating a new Smart Device Application project a dialog appears to prompt for the kind of application you want. This was to be a Windows Application.

You get a default form, sized to the Pocket PC (or other device) screen, onto which you can drag components from the Device Controls toolbar shown next. No real difference here compared with designing forms for any .NET application, apart from the limited form size and the limited set of device controls.

For my application I dragged Button controls onto the form so that the resulting application would look like this:

That's all there is to the UI design, and now we need to connect those buttons to the sound effects they represent.

User Interface Code

As with any other forms based .NET application, Visual Studio .NET does the hard work of coupling up components to their event handlers. Just double-click a button in the design view and you'll be transported into an event handling method for the button click. Here's my implementation for the screamButton_Click() method that plays the scream.wav sampled sound (located relative to the application deployment location) via our earlier Sound class:

private void screamButton_Click(object sender, System.EventArgs e)
{
Sound.play("scream.wav");
}

Each button of the final application has a _Click() method almost identical to that one. I know that hard coding sounds to buttons is not necessarily the best way to do it, but it will suffice for our first application.

Application Setup

I've included a complete solution so that you don't have to build it yourself from scratch. It includes some sample sound effects that I've recorded. Just load the SoundBox solution into Visual Studio .NET and in the Solution Explorer you'll see something resembling the screen shot below. Besides the two source files - Sound.cs and SoundForm.cs - you can see a collection of .wav files which I had included originally by right-clicking the SoundBox project and choosing Add / Add Existing.

To build, deploy, and run the application choose the Debug / Start Without Debugging menu option. You can specify whether to deploy to a real device (if you have one) or to the emulator. I'm sure you need no further instruction on how to use this application.

It's interesting enough to run it in the emulator, but not half as much fun as walking around with a real Pocket PC directing screams and gunshots at unsuspecting friends!

Debugging

I've just invited you to try out the sample application by choosing the Debug / Start Without Debugging menu option. Previously I suggested that you test the SDE installation by loading up the Calculator application and running it in the same way.

No doubt you'll now be wondering what happens if you run a Smart Device application using the Debug / Start menu option instead. Answer: it runs the application in full debugging mode, which doesn't sound that remarkable until you discover that it should work equally well whether you choose to deploy the application to the emulator or to a real device. In my experience it does work equally well either way.

To try it out I set a breakpoint on the following line of code:

Sound.play("tom3.wav");

I then deployed the application and pressed the various buttons. Upon pressing the other buttons, the appropriate sound was played without delay, but when the button marked Tom 3 was pressed, no sound ensued. The application was temporarily suspended on the device and control transferred to the breakpoint within the Visual Studio .NET IDE, and there it stayed until I stepped over the line of code.

Limitations and Further Work

In testing the SoundBox application I've discovered an interesting anomaly relating to the playback of .wav files. I've also thought up a really interesting idea for where to take this next, by developing a low-memory Pocket PC speech synthesizer and text-to-speech engine as the basis of a new set of killer applications.

Playback of .wav files

I've noticed that when deployed on a real device - my Pocket PC - the sampled sounds are played in response to button presses regardless of the origin of the .wav files. When deployed to the emulator for the first time however, some .wav files played OK and some (for example, my speech samples) resulted in no sound whatsoever. No error message, just silence.

My limited investigations have shown that this depends on the application I used to record the samples, or more particularly on the exact format in which the samples were saved. Should it be PCM 8.000 kHz, 8 bit, Mono or CCIT A-law 22.050 kHz, 8 bit, Stereo or one of the other myriad choices? Ultimately I would guess that it depends on the sound CODECs installed on your machine, but the success of playback in the emulator seems unrelated to the success of playback in the Windows Media Player, and unrelated to the success on the device itself.

Conclusion? If my samples don't work in your emulator then try substituting some of your own. And if they don't play on your real device it's best to check first that you have program sounds enabled via this dialog:

Yes many is the time that I have turned off program sounds in a crowded office, only to be frustrated later upon discovering that the SoundBox application is not working properly.

Speech Synthesis

This is where it gets really interesting for me.

In a previous article for Java World, "Talking Java" at http://www.javaworld.com/javaworld/jw-08-2001/jw-0817-javatalk.html?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq I described the implementation of a speech synthesizer that worked by concatenating individual speech samples - called allophones - to form whole spoken words. That was really a vehicle for showing off the Java 2 Sound APIs, but it led me on to implementing a Java 1.1 version that would work even in applets and even on small Java-enabled devices such as my Nokia 9210 phone / PDA. With limited bandwidth (in the case of a speech applet) or limited memory (in the case of a Java-enabled device) there was no possibility of basing the speech synthesis on samples of whole words or sentences, so concatenation - in various combinations - of about 60 shorter allophone samples was the only viable solution.

Having proven the technique with Java, I'd now like to try the same thing on the Pocket PC. I've made a start by including a Speak button in this case study's sample application. The method for the Speak button's click event is as follows:

private void speakButton_Click(object
            sender, System.EventArgs e)
{
Sound.play("s.wav");
Sound.play("m.wav");
Sound.play("ar.wav");
Sound.play("tt.wav");
Sound.play("10ms.wav");
Sound.play("dd.wav");
Sound.play("e.wav");
Sound.play("v.wav");
Sound.play("ii.wav");
Sound.play("s.wav");
}

It shouldn't be too difficult to figure out that when you press the button the phrase " Smart Device " is spoken, with a 10ms silence between the two words. But not until you change one of the lines of code within the Sound class to read:

PlaySound( fileName, IntPtr.Zero, SoundFlags.SND_FILENAME | SoundFlags.SND_SYNC);

Rather than the original:

PlaySound( fileName, IntPtr.Zero, SoundFlags.SND_FILENAME | SoundFlags.SND_ASYNC);

The difference is that we want the speech samples to be played out synchronously, one-by-one in strict sequence, with each new sound (allophone) not interrupting the previous one. So, this time we use the SoundFlags.SND_SYNC flag. Conversely, for the sound effects and drum samples we wanted a much more responsive application that allows a button to be pressed, and a new sound played, before the old sound have finished. So previously we used the SoundFlags.SND_ASYNC flag.

To really appreciate the suitability of the allophone technique for implementing a speech capability on a constrained device I invite you to play a game. See how many new words and phrases you can make by rearranging the allophones of the phrase "Smart Device". To get you started how about "Dart", "Site", and even "Smart Mice" ?

Now try to figure out how many kilobytes would be required to store all of your new phrases in entirety as separate .wav files. I bet it's considerably more than the 63 kb combined size of the allophone .wav files.

Text-to-Speech Conversion

It's all well and good that we have a way to assemble spoken sentences from a limited set of allophones, but it's a bit cumbersome to figure out the correct allophone sequences in advance. In my conclusion, I'll discuss some novel applications for smart devices that would take information from various sources - the device calendar, incoming email messages, web pages - and speak that information to the user. Since we won't know in advance what the text will be, some form of automatic text-to-speech conversion will be required.

I can't yet provide a complete solution for that, but I can offer a starting point that you can take further if you wish. The challenge here is to take some ordinary text:

"SMART DEVICE"

And convert that text into a sequence of allophones (below) that may be replayed via our Sound.play() method:

"s|m|ar|tt|10ms|dd|e|v|ii|s|"

Simply replacing the individual letters - or groups of letters - of the input text with allophone names in the output text is a technique that has worked surprisingly well for me in the past. So, on a letter-by-letter basis, we would do some replacements like this:

text = Regex.Replace(text,
            @"\s","10ms|");

text = Regex.Replace(text,
            @"A","a|");
text = Regex.Replace(text, @"C","k|");
text =
            Regex.Replace(text, @"D","dd|");
text = Regex.Replace(text,
            @"E","e|");
text = Regex.Replace(text, @"I","i|");
text =
            Regex.Replace(text, @"M","m|");
text = Regex.Replace(text,
            @"R","r|");
text = Regex.Replace(text, @"S","s|");
text =
            Regex.Replace(text, @"T","tt|");
text = Regex.Replace(text,
            @"V","v|");

I'm using simple regular expressions there to perform the replacements. So, you'll be pleased to hear that the .NET Compact Framework supports regular expressions providing you import the System.Text.RegularExpressions namespace and include a reference to the System.Text.RegularExpressions.dll in your project. As I'm not including a tutorial on regular expressions here you'll also be pleased to hear that have I provided a separate tutorial on that subject at C#Today, "Introducing .NET Regular Expressions with C#" at http://www.csharptoday.com/content/articles/20020201.asp?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq.

With just those letter-to-allophone replacements the result would be:

"s|m|a|r|tt|10ms|dd|e|v|i|k|e"

This might not look too bad, apart for the "k" allophone. However, by common convention the "a" allophone (file a.wav) in a complete set would be sounded as in "cat", and the "i" allophone (file i.wav) would be sounded as in "pig", which is not at all what we want. To fix that problem we need two additional replacement rules that take precedence over the individual letter replacements. These new rules are:

text = Regex.Replace(text,
            @"ICE","ii|s|");
text = Regex.Replace(text,
            @"AR","ar|");

As I said, I've used this simple technique to remarkable effect in a previous project. Of course a complete text-to-speech conversion requires many more replacement rules, including some for whole words, and the ordering of those rules is vital. To get you started I've built the starter set of rules into a TextToSpeech class included in the sample project. Here is the code listing for that class:

namespace SoundBox
{

public class
            TextToSpeech
{

public static void speak( string
            text)
{
string allophones=textToSpeech(text);

Regex
            allophoneRegex=new Regex(@"(?<allophone>\w+)\|");
Match
            allophoneMatch=allophoneRegex.Match(allophones);

for ( ;
            allophoneMatch.Success; allophoneMatch =
            allophoneMatch.NextMatch())
{
Sound.play(allophoneMatch.Groups["allophone"]+".wav");
}
}

public
            static string textToSpeech(string text)
{
// Compound
            replacements

text = Regex.Replace(text,
            @"ICE","ii|s|");
text = Regex.Replace(text,
            @"AR","ar|");

// Single letter replacements
text =
            Regex.Replace(text, @"\s","10ms|");

text =
            Regex.Replace(text, @"A","a|");
text = Regex.Replace(text,
            @"C","k|");
text = Regex.Replace(text, @"D","dd|");
text =
            Regex.Replace(text, @"E","e|");
text = Regex.Replace(text,
            @"I","i|");
text = Regex.Replace(text, @"M","m|");
text =
            Regex.Replace(text, @"R","r|");
text = Regex.Replace(text,
            @"S","s|");
text = Regex.Replace(text, @"T","tt|");
text =
            Regex.Replace(text, @"V","v|");

return
            text;
}
}
}

To try out that class you just need to modify the SoundForm.speakButton_Click() method to read:

private void speakButton_Click(object sender, System.EventArgs e)
{
TextToSpeech.speak("SMART DEVICE");
}

If that idea turns you on, don't forget to check out the separate regular expressions tutorial that I've mentioned.

.NET Compact Framework Wish List

I'm really interested in writing smart device applications that can take advantage of the integrated phone capabilities on my device. Though you can issue HTTP requests and collect the responses using classes of the System.Net namespace, there appears to be no obvious way of sending and receiving SMS messages, sending and receiving email messages, or simply placing telephone calls using the .NET Compact Framework classes.

From what I've read, it should be possible to achieve those things using the Pocket PC 2002 SDK - if I can motivate myself to write some C++ code. Alternatively, I suspect that the platform invoke technique introduced in this case study could be used, providing I can identify the appropriate DLL and the right function within it. In that respect, the POOM application (discussed in the next section) might be a good starting point.

But what I'd really like to see is a .NET equivalent of - for example - the JavaMail API that I've used previously.

In the spirit of the non-device-specific nature of the .NET framework these omissions may appear to be quite reasonable. After all, not all devices have mobile communication capabilities. If that's what you're thinking, my challenge is for you to find a smart device that doesn't have phone capabilities in a year's time.

Conclusion

In this introductory case study I wanted to focus on the Smart Device Extensions installation and configuration issues that I had encountered, the resolutions of which will be immediately useful to you.

To test the installation, and as a vehicle for discussing some of the possible deployment problems, I introduced you to the Calculator sample that comes bundled with the SDE distribution. The other bundled applications are worth looking at if you want to learn more about various aspects of the .NET Compact Framework.

Bundled Sample Applications

All of the bundled sample applications can be found in directory C:\Program Files\Microsoft Visual Studio .NET\CompactFrameworkSDK\v1.0.3300\Samples. At that location you will find them subdivided into directories for Pocket PC and Windows CE platforms, and further subdivided for Visual Basic and Visual C# samples. The full path to the Bubble sample for Pocket PC written in C# is therefore: C:\Program Files\Microsoft Visual Studio .NET\CompactFrameworkSDK\v1.0.3300\Samples\PocketPC\VC#\Scribbler.

You can load up any one of the samples into Visual Studio .NET by double-clicking the solution (.sln) file, and then choose Debug / Start Without Debugging to run it.

Bubble - This application demonstrates some drawing and animation features that might be a good starting point for those of you looking to write games for the platform.

Scribbler - Demonstrates a very rudimentary paint box-type application that demonstrates how to capture mouse clicks and drags.

WaitCursor - Shows how to change the state of the on-screen cursor.

StockQuote - Shows how the device can act as a client to a remote web service, which could prove to be the most common model for smart device applications. This example requires a lot more set-up than the others in order to simulate a web service provider, and it requires IIS to be running on your computer.

SoundBox Sample Application

My own sample application introduced the idea of accessing a native device function from .NET, and I only scratched the surface in terms of the technicalities and in terms of the uses for that technique. In my search for other examples, I found an interesting sample application on the GotDotNet web site at http://www.gotdotnet.com/team/netcf/samples.aspx?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq. This sample called POOM (Pocket Outlook Object Mode) " .lets you view all of the contacts on your Pocket PC, add new ones, and delete existing ones. " From a technical perspective it " .consists of a managed application built on top of the .NET Compact Framework and a native DLL written in C++ using Embedded Visual C++ 3.0 " So rather than accessing a function within an existing device DLL - as we have done in this case study - the POOM sample accesses a function in a custom-written DLL which in turn interrogates one of the device on-board applications.

Why do I find that POOM example so interesting? Well, imagine that we extracted information from the on-board calendar application and combined that with my speech synthesis idea so that up-coming appointments are spoken to the user like this:

"Tony, you have an appointment with Mr. Smith and 10 o'clock"

Now imagine that we gain access to the device InBox so that incoming email (or SMS) messages are spoken like this:

"Tony, you have a new SMS message from phone number 0123456. The message reads."

Towards the end of this case study I mentioned some messaging and telephone features that would make the .NET Compact Framework truly useful, providing for applications that make full use of the integrated phone features of many of the new devices. I'm surprised that this functionality is not yet included, given the enormous potential of messaging applications; but not too surprised because I suffered exactly the same frustrations previously in trying to develop Java messaging / phone applications for my Nokia 9210 PDA-and-phone using the Symbian SDK. Once again, no access to SMS, email, or the integrated phone.

But we do have a web access capability thanks to the System.Net classes, so finally imagine an application that extracts selected information from particular web sites and speaks that information to you. As I'm driving along with the headset plugged into my Pocket PC Phone Edition I need never take my eyes off the road or my hands off the wheel in order to discover that:

"The latest stock price for XYX industries is $100. Maybe you should sell out now!"

I'll be doing a great deal of thinking about that application in the coming months, and so might you. You could start by reading my previous article "Working with Web Data in C#" (http://www.csharptoday.com/content/articles/20020128.asp?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq) which describes how to extract data - such as stock quotes -- from live web pages. Then on the speech synthesis side you'll need a complete set of allophones and a more complete text-to-speech engine. I'm working on it.


Monitoring IIS Systems

Monitoring IIS Systems
http://www.csharptoday.com/content.asp?id=1864
Article Rating: 4
Published on September 4, 2002
Abstract
In this case study, Paul Wilton looks at monitoring the IIS services. We create two programs. The first will be a Windows Service that does the actual monitoring of the IIS services. As a Windows Service can't have a user interface, a second program will be created which allows the IIS Monitoring service to be managed and important parameters set. While creating these programs we'll also be looking at how we can manage IIS programmatically using .Net. We'll see how we can find out what sites have been defined on IIS, what the properties of these sites are, and whether they are running or not. We'll see how we can restart an individual site or, as a last resort, we'll be seeing how a full re-start of the IIS services can be performed.
Case Study

I run a Windows 2000 web and e-mail server, and I was finding that when I least expected it things stopped working. Important e-mails I was expecting never arrived and only later did I spot that the SMTP service of IIS was not functioning. It was for this reason I created a .Net based IIS services monitor. At regular intervals this will check to see if important IIS services such as HTTP, FTP and SMTP services are still up and running. If they are not then I wanted them re-started and a note made of the failure.

In this case study we'll be creating two programs. The first will be the Windows Service that does the actual monitoring of the IIS services. As a Windows Service can't have a user interface, a second program will be created which allows the IIS Monitoring service to be managed and important parameters set. Of course this is not the only IIS monitoring program available, many commercial programs are available, for example Microsoft Operations Manager. The advantage with the one in this case study is its free, simple, and easily modifiable by someone with C# knowledge.

While creating these programs we'll also be looking at how we can manage IIS programmatically using .Net. We'll see how we can find out what sites have been defined on IIS, what the properties of these sites are, and whether they are running or not. We'll see how we can restart an individual site or, as a last resort, we'll be seeing how a full re-start of the IIS services can be performed, useful for those occasions when IIS is having such a bad day that nothing but a total restart will get it running again. If we wanted we could take things further and force a restart of the entire computer, but that's not something we will do in this example.

Requirements

We'll be assuming in this case study that you already have a good knowledge of C# and also of the Visual Studio.Net development environment. Obviously for the program to work you'll need a system with IIS installed, and Windows XP.

Defining The Problem

IIS Services can sometimes fail without warning, when you least expect it. Web and e-mail services are vital to companies and even a few hours downtime can result in significant financial losses or at the very least inconvenience. However it's not always possible, especially in smaller companies, to be constantly checking to see if the IIS Services are performing correctly. What is required is a program that checks at regular intervals to see if the IIS Services are functioning. By functioning, we mean not just running but able to be connected to. There are some occasions when a service is "running" but attempts to connect to it fail. The program needs to replicate the normal connection method, for example the request of a web page or start of an SMTP session.

It's not good security practice to leave a server unattended and logged onto. We therefore need our program to run in the background and to check IIS Services regardless of whether someone is logged in to the server or not.

It would be nice to minimize disruption to other IIS Services. So for example, if it were just one web site that has failed, it'd be preferable to just attempt to restart that site, not the whole web service, or even the whole IIS Service. However, if an attempt to start a single site fails, then a full re-start of IIS is the best solution as it cures most IIS issues.

Regular failure of services indicates a problem somewhere, one that needs to be investigated by the system administrator. We must therefore make a note of each failure. It might also be nice to send the system admin an SMS message or e-mail to alert him of problems, although for reasons of space we don't include this functionality in the program (though it would be very easy to add).

What we don't want our program to do is to restart sites that we have deliberately stopped. For example, for security reasons we may have stopped the administration web site - we don't want to over ride this. We therefore need to allow the system administrator to specify which sites should be monitored.

So to summarize what we want our program to do:

  • Check that IIS Services are running and able to accept the normal requests
  • To always run in the background regardless of whether a user is logged into the server
  • If an individual site has failed then make a note and start the failed site
  • If when the program next checks a site that had been started and the site is still not responding then we need to do a full IIS re-start. This unloads the whole IIS process and is very effective at getting services running again. A note should be made of the failure and total re-start.
  • To not check all sites but only those specified by the system administrator.

Defining The Solution

Our solution will consist of two parts. First we will create the IIS Service Monitor management console. This will be a standard Windows program. It will allow us to set various parameters relating to the monitor service including:

  • Monitoring Interval
  • Which HTTP, FTP and SMTP sites to monitor

For this case study, the saved settings will be stored in the Windows registry, though an obvious alternative would be a config file. The console will also allow us to stop and start the service.

Then we will create the IIS Services Monitoring service itself. When the service is started, it reads the settings defined by the console and then in intervals defined in the console it checks various IIS services by making a connection to them. Any settings stored in the registry won't be picked up until the service is re-started.

Building The IIS Service Monitor Console

First we need to open VS.Net and create a new Windows Application project:

Now within VS.Net let's layout the controls on the default form:

As the screen shot above shows, the form is fairly simple. It contains three checked list boxes, one each for the HTTP, FTP, and SMTP services. When the program is run, these list boxes will be populated with the details of all the sites running on the local server, as shown below:

The check box within the list box allows the user to select which sites they want to be monitored. In the screen shot above, I have decided not to monitor the admin website or the default ftp website.

The form also contains a text box in which we can enter the monitoring interval (i.e. how often to check the services) and three buttons: one to start the service, one to stop it, and one that saves any changes made to the service settings.

Shown in the table below are the essential property settings for the form and its controls:

Control Type | Properties
Form | Name : frmManagementConsole Text : IIS Service Monitor Management Console
Text Box | Name : txtMonitoringInterval Text : 240
Button | Name : cmdStartMonitoring Text : Start Monitoring
Button | Name : cmdStopMonitoring Text : Stop Monitoring
CheckedListBox | Name : lstServicesToMonitorHTTP
CheckedListBox | Name : lstServicesToMonitorFTP
CheckedListBox | Name : lstServicesToMonitorSMTP
Button | Name : cmdSaveSettings Text : Save Settings

Ok we have the form laid out, let's start adding some code.

References

Our project makes use of a number of references that are not included in a default Windows Application project. From the screenshot below, you can see that the project needs two references added, one to System.DirectoryServices and one to System.ServiceProcess. We do this by right clicking References, clicking the Add button, and then selecting the two references that need to be added.

The DirectoryServices reference allows us to access Windows 2000's Active Directory, and from that, the properties of IIS. The ServiceProcess reference will allow us to start and stop Windows services, such as the IISServiceMonitor service that we'll be creating later in the case study.

Our project will be using various namespaces not specified by default, so to save us typing long names let's add some using keywords at the top of the project just below the existing default ones:

using System.DirectoryServices;
using System.ServiceProcess;
using System.Text.RegularExpressions;
using Microsoft.Win32;

The Form Load

When the form is loaded, we want to populate the list boxes with the available data for each of the three IIS we monitor: web, ftp and smtp. We also want to retrieve from the registry details of which sites were selected to be monitored, and also the monitoring interval set by the user. We therefore need to add code to the Form's Load event that does this. The code is quite extensive so let's look at it bit by bit.

Starting from the top, we first retrieve the details of the monitoring interval and then use this to set the value in the txtMonitoringInterval text box:

private void frmManagementConsole_Load(object sender, System.EventArgs e)
{

RegistryKey regKey =
Registry.LocalMachine.OpenSubKey("System\\ServicesMonitor", true);

int monitoringInterval = (int)regKey.GetValue("MonitoringInterval", 60000);
monitoringInterval = monitoringInterval / 1000;
this.txtMonitoringInterval.Text = monitoringInterval.ToString();

All the settings we'll be saving will be saved, as above, under the LocalMachine\System\ServicesMonitor registry key - a key that will be accessible to the monitoring service even when no user is logged on. The monitoring interval is stored in milliseconds as this is the value required by the timer that'll be used in the IISServicesMonitor service. If no registry key exists, i.e. if this is the first time the program has been run, then 60,000 is used as the default value, i.e. 60 seconds.

When we display the interval to the user via the text box, it's in seconds, not milliseconds, and so we need to divide the value retrieved from the registry by 1000.

The next task is to obtain the previously selected sites, if any, to be monitored. Again this information is retrieved from the registry and comes in the form of a delimited string that contains the name of each site to be monitored. We'll be using the information shortly to add check marks in the list boxes for http, ftp and smtp monitoring:

String HTTPValuesToCheck = regKey.GetValue("HTTPSitesToCheck").ToString();
String FTPValuesToCheck = regKey.GetValue("FTPSitesToCheck").ToString();
String SMTPValuesToCheck = regKey.GetValue("SMTPSitesToCheck").ToString();
Hashtable sHTTPSitesToCheck = getSitesToBeChecked(HTTPValuesToCheck, Server);
Hashtable FTPSitesToCheck = getSitesToBeChecked(FTPValuesToCheck, Server);
Hashtable SMTPSitesToCheck = getSitesToBeChecked(SMTPValuesToCheck, Server);

The function getSitesToBeChecked(), which we'll create shortly, takes the delimited string obtained from the registry and splits it to obtain the individual site names and details to be checked. It returns the results in a Hashtable.

Now let's obtain a list of all the sites available for the http, ftp and smtp services and add an entry to the listboxes on our form:

m_HTTPSites = GetIISSites(Server, "W3SVC");
foreach (Hashtable Site in m_HTTPSites.Values)
{
lstServicesToMonitorHTTP.Items.Add(Site["SiteName"] + " - " +
Site["ServerComment"]);
}

// Get the details of all the FTP sites and display in a listbox
m_FTPSites = GetIISSites(Server, "MSFTPSVC");

foreach (Hashtable Site in m_FTPSites.Values)
{
lstServicesToMonitorFTP.Items.Add(Site["SiteName"] + " - " +
Site["ServerComment"]);
}

// Get the details of all the SMTP sites and display in a listbox
m_SMTPSites = GetIISSites(Server, "SMTPSVC");
foreach (Hashtable Site in m_SMTPSites.Values)
{
lstServicesToMonitorSMTP.Items.Add(Site["SiteName"] + " - " +
Site["ServerComment"]);
}

We set the global variables m_HTTPSites, m_FTPSites and m_SMTPSites to the Hashtable returned by function GetIISSites() (which we will create later). This function accesses the active directory and obtains a list of all the sites for a particular service, creates a Hashtable object for each site, and then adds it to the Hashtable object in the global variables. The site Hashtable contains the details of the site: for example, its site name, its friendly name, port number, ip address, and so on. Then a foreach loop loops through the global variables, and for each Hashtable object containing site details, it adds an entry to the form's list boxes.

We have already found out which sites were selected to be monitored last time the console was used, and we've also populated the list boxes with details of every site. Now we need to go through each item in the list boxes and see if they were sites previously marked to be monitored. If they are, then we tick the check box for that item:

// Retrieve existing selected sites and
            update listboxes
Regex getSiteNameRegex = new Regex("\\d+",
            RegexOptions.Compiled);
Int32 iListItemCounter;
String
            sSiteName;
for (iListItemCounter = 0;
iListItemCounter <
            this.lstServicesToMonitorHTTP.Items.Count;
            iListItemCounter++)
{
sSiteName =
this.lstServicesToMonitorHTTP.Items[iListItemCounter].ToString();

if
            (
            sHTTPSitesToCheck.Contains(getSiteNameRegex.Match(sSiteName).Value))
{
this.lstServicesToMonitorHTTP.SetItemChecked(iListItemCounter,
            true);
}
}

for ( iListItemCounter = 0;
iListItemCounter < this.lstServicesToMonitorFTP.Items.Count;
            iListItemCounter++)
{
sSiteName =
this.lstServicesToMonitorFTP.Items[iListItemCounter].ToString();
if
            (FTPSitesToCheck.Contains(getSiteNameRegex.Match(sSiteName).Value))
{

this.lstServicesToMonitorFTP.SetItemChecked(iListItemCounter,
            true);
}
}

for (iListItemCounter = 0;
iListItemCounter <
this.lstServicesToMonitorSMTP.Items.Count;
            iListItemCounter++)
{
sSiteName =
this.lstServicesToMonitorSMTP.Items[iListItemCounter].ToString();

if
            (SMTPSitesToCheck.Contains(getSiteNameRegex.Match(sSiteName).Value))
{
this.lstServicesToMonitorSMTP.SetItemChecked(iListItemCounter,
            true);
}
}

By default, the start service button is enabled and the stop disabled. This is fine if the service is not running, but if it is we want to make sure that the service start/stop buttons are set so that if the service is already running it can be stopped but not started again:

ServiceController serviceMonitor = new
ServiceController("IISServiceMonitor");
if
            (serviceMonitor.Status ==
            ServiceControllerStatus.Running)
{
cmdStartMonitoring.Enabled
            = false;
cmdStopMonitoring.Enabled = true;
}

We have enlisted the help of the ServiceController class and created a new object, passing the name of our service to the constructor. When we create the IIS Service Monitor it's important that we keep the same name we have used here.

We then use the Running property that returns true or false to see if the service is currently running. If it is, we disable the start button and enable the stop button.

In the load event we used three global variables that haven't yet been declared, so lets rectify that and add the following to the top of the class definition:

const String Server = "localhost";
const String TestPage = "test.htm";
Hashtable m_HTTPSites;
Hashtable m_FTPSites;
Hashtable m_SMTPSites;

You'll also notice that two constants have been declared: Server and TestPage. These will be used in the functions we're about to create.

Lets start by creating the function getSitesToBeChecked() that we used in the Load function to get a list of the sites that need to be monitored. Remember this information was obtained from the registry as a delimited string, and is passed to the function along with the ip address or name of the default server:

private Hashtable getSitesToBeChecked(String sSitesString,
String sDefualtServer )
{

sSiteString will contain the sites to be checked. A tab delimits each site. Within the site information is 5 more bits of information: site name, server comment, server, port, and header.

A colon delimits each of these bits of information. Site name is the name of the site, but it's not the friendly name we know our sites by, for example "Default", "Administration", etc. It's actually the name by which IIS knows our sites and is a number. Server comment is the nice friendly name we use for each site. Server is the name or ip address of the server. Port is the port via which the site is accessed. For a website, this is normally port 80, for ftp its port 20 or 21, and smtp is normally port 25. However we can set up sites to operate on different ports.

At the top of the function we declare four variables:

Regex getDataValuesRegex = new
Regex("(?<sitename>[^:]*):(?<Servercomment>[^:]*)") +
":(?<Server>[^:]*):(?<port>[^:]*):(?<header>[^:]*)",
RegexOptions.Compiled);
Match siteMatch;
Hashtable SitesToCheck = new Hashtable();
Hashtable SiteProperties;

The first is a Regex, this is our regular expression that will split the five bits of individual information contained about a single site. We have defined named regular expression groups for each bit of information, this will make it easier to access via code the information. So for example:

(?<sitename>[^:]*)

Specifies a group called sitename and says that the group contains any number of characters from zero upwards that are not a colon. As soon as a colon is found no more characters are matched for the group. This is then followed by:

:(?<Servercomment>[^:]*)

This specifies that following the sitename group must be a colon. Then we define another group of characters called Servercomment. Again, this is described as containing zero or more characters so long as they are not a semi-colon. The regular expression continues in the same way, of a colon followed by a group, and so on. It makes extracting the individual bits of site information very easy. Also note the regular expression option Compiled is set: this will cause the regular expression to be compiled into something more machine friendly and speeds up matching.

The next variable defined is siteMatch. When we come to split the site information string, this will be used to contain each match for a site. We'll then use it to extract the specific site data.

The variable SitesToCheck will contain a hashtable object. Each item in the hashtable will be another hashtable object, stored in the variable SiteProperties, containing information about a specific site that must be monitored.

Now all we need to do is split the sSitesString containing each site delimited by a tab, and then loop through the array returned. In each iteration we create a new hashtable object and store the site name, server comment, server, port, and header information for the site:

String[] sSites = sSitesString.Split('\t');

foreach( String sSite in sSites )
{
siteMatch = getDataValuesRegex.Match(sSite);
SiteProperties = new Hashtable();
SiteProperties.Add("SiteName", siteMatch.Groups["sitename"].Value);
SiteProperties.Add("ServerComment",
siteMatch.Groups["Servercomment"].Value);
SiteProperties.Add("Port", siteMatch.Groups["port"].Value);
SiteProperties.Add("Header", siteMatch.Groups["header"].Value);
if (siteMatch.Groups["Server"].Value == "")
{
SiteProperties.Add("Server", sDefualtServer);
}
else
{
SiteProperties.Add("Server", siteMatch.Groups["Server"].Value);
}
SitesToCheck.Add(SiteProperties["SiteName"], SiteProperties);
}

Note how we use the siteMatch, which is a Match object, to obtain the characters matched by each of the groups in our regular expression. We didn't need to name our groups, and could just have easily used numbers to refer to them, but using names makes the code that little bit more readable.

Finally we return the hashtable object containing all the details of sites currently specified to be monitored:

return SitesToCheck;

Our next function is GetIISSites(). This function returns all the sites for a particular service. Examples of services include web, ftp (file transfer), smtp (simple mail transfer protocol), pop (post office protocol) and so on. The table below lists them:

Service Name | Description
w3svc | Web Services
msftp | File Transfer Protocol service
smtp | Simple Mail Transfer Protocol (e-mail send/receive)
pop | Post office Protocol (for picking up e-mails off a server)

We don't monitor the pop service, so the function's parameter sService will contain either w3svc, msftp, or smtp. The sServer parameter in this case study will always be "localhost", but it could be an ip address or the name of the server:

private Hashtable GetIISSites(String
            sServer, String sService)
{
return IISSites;
}

In this function we hook up to the IIS metabase directory on the local machine, and then use this to extract the information we need. We'll be using the System.DirectoryServices namespace that makes accessing directories very easy. First we need to get a path to the directory tree containing the information about the service we are interested in, and get a reference to a DirectoryEntry object that will represent that node in the directory hierarchy:

String IISPath = "IIS://" + Server + "/" +
            sService;
System.DirectoryServices.DirectoryEntry
            IIS;
Hashtable siteProperties;
String sSiteName;
IIS = new
            DirectoryEntry(IISPath);
Hashtable IISSites = new
            Hashtable();
Regex isANumber = new Regex("\\d+");

IIS contains the node for a particular IIS service, and each site is represented by a child DirectoryEntry object of that IIS service node. What we now need to do is walk through each child (i.e. site) and get its details and add them to a hashtable:

foreach
            (System.DirectoryServices.DirectoryEntry Site in
            IIS.Children)
{
if ( isANumber.IsMatch(Site.Name)
            )
{
sSiteName = Site.Name;
siteProperties = new
            Hashtable();
siteProperties.Add("SiteName",
            sSiteName);
siteProperties.Add("ServerComment",
Site.Properties["ServerComment"].Value);
siteProperties.Add("ServerBindings",
Site.Properties["ServerBindings"].Value);
IISSites.Add(sSiteName,
            siteProperties);
}
}

Not all the child objects are necessarily sites, some are simply there for the operating system. How do we tell which are which? Well all the sites with a number for a name are our web/ftp/smtp virtual sites. That's why we use a regular expression to check to see if the Site.Name property contains a numerical value. Remember it's Site.ServerComment that contains the friendly name we are used to seeing in the IIS management console. If we have found one of our virtual sites, we access its properties and create a new hashtable object into which we store the properties such as Site Name, Server Comment, and Server Bindings. Note the ServerBindings contains things like port number, as each site can potentially have a different port number to distinguish it from other sites, and also a header, again to distinguish it from other sites on the same server. The hashtable object created is then added to the hashtable object referenced in IISSites.

Once we have looped through each site we then return the hashtable object references IISSites:

return IISSites;
}

That completes all the code that executes when the form is loaded so let's turn to the code for each of the three buttons on the form, starting with the cmdStartMonitoring button's click event:

private void cmdStartMonitoring_Click(object sender, System.EventArgs e)
{
ServiceController serviceMonitor = new
ServiceController("IISServiceMonitor");
if (serviceMonitor.Status != ServiceControllerStatus.Running)
{
serviceMonitor.Start();
}

cmdStartMonitoring.Enabled = false;
cmdStopMonitoring.Enabled = true;
}

If this code looks familiar, it's because it's similar to code we had in the form's load event which checked to see if our IISServiceMonitor service was running, and enable/disabled the start/stop buttons accordingly. It uses the ServiceController class to check if the IISServiceMonitor service is running, and if it's not, then start it running and then enable/disable the start/stop buttons accordingly.

The click event for the cmdStopMonitoring button is, as you'd expect, simply the reverse of the code above:

private void cmdStopMonitoring_Click(object sender, System.EventArgs e)
{
ServiceController serviceMonitor = new
ServiceController("IISServiceMonitor");
if (serviceMonitor.Status != ServiceControllerStatus.Stopped)
{
serviceMonitor.Stop();
}

cmdStartMonitoring.Enabled = true;
cmdStopMonitoring.Enabled = false;
}

The final bit of code we need to create for the console is that for the cmdSaveSettings button's click event. Essentially it's the reverse of what we did in the load event. Instead of loading the saved settings from the registry, we'll be saving the settings (i.e. monitoring interval and which sites to monitor) to the registry.

Before we change the settings we need to first stop the service, if it is actually running:

private void cmdSaveSettings_Click(object sender, System.EventArgs e)
{
ServiceController serviceMonitor = new
ServiceController("IISServiceMonitor");
Boolean doRestartAfterChanges = false;
if (serviceMonitor.Status != ServiceControllerStatus.Stopped)
{
serviceMonitor.Stop();
doRestartAfterChanges = true;
}

As we'll see later, it's when the IISServiceMonitor service is started that the initial reading of the registry and settings takes place, so to make any of our changes update a currently running IISServiceMonitor service we need to re-start it.

Now we need to create a RegistryKey object and set it to point to where we store the settings for the IISServicesMontior, i.e. in the LocalMachine\\System\\ServicesMonitor part of the registry:

RegistryKey regKey;
regKey =
            Registry.LocalMachine.CreateSubKey("System\\ServicesMonitor");
regKey.SetValue("MonitoringInterval",
Int32.Parse(txtMonitoringInterval.Text)
            * 1000);

Regex getSiteNameRegex = new Regex("\\d+",
            RegexOptions.Compiled);
Int32 itemIndex;
String
            sSiteName;
Hashtable sSiteParameters;

We need to save the details of which http, ftp and smtp sites need to be monitored. Let's start with the http. We'll do this by going through each item in the appropriate check list box, seeing if it has been checked by the user. If it has been checked, then we get the site's details from the hashtable object stored in global variable m_HTTPSites that we populated with a list of all the available http sites when the form was loaded. To get a reference to the correct hashtable object within the m_HTTPSites hashtable we need the site name, which we added to the front of the description shown in the list box. Having got the hashtable object for that site, we then add the site properties to the string we'll be saving in the registry. A tab character delimits each site and each bit of information within the site details is delimited by a colon:

// Create the string that stores HTTP sites to check
String sHTTPSitesToCheck = "";
for (itemIndex = 0; itemIndex < lstServicesToMonitorHTTP.CheckedItems.Count; itemIndex++)
{
sSiteName =
getSiteNameRegex.Match(lstServicesToMonitorHTTP.CheckedItems[itemIndex].ToString()).Value;
sSiteParameters = (Hashtable)m_HTTPSites[sSiteName];
sHTTPSitesToCheck = sHTTPSitesToCheck + sSiteName + ":" +
sSiteParameters["ServerComment"] + ":" +
sSiteParameters["ServerBindings"] + "\t";
}

if (sHTTPSitesToCheck != "")
{
sHTTPSitesToCheck = sHTTPSitesToCheck.Remove(
sHTTPSitesToCheck.Length - 1, 1);
}

The code to create the string that'll be stored in the registry is identical for the ftp and smtp services, it's just the global variable and list box that changes:

// Create the string that stores FTP sites to check
String sFTPSitesToCheck = "";

for (itemIndex = 0; itemIndex < lstServicesToMonitorFTP.CheckedItems.Count; itemIndex++)
{
sSiteName =
getSiteNameRegex.Match(lstServicesToMonitorFTP.CheckedItems[itemIndex].ToString()).Value;
sSiteParameters = (Hashtable)m_FTPSites[sSiteName];
sFTPSitesToCheck = sFTPSitesToCheck + sSiteName + ":" +
sSiteParameters["ServerComment"] + ":" +
sSiteParameters["ServerBindings"] + "\t";
}

if (sFTPSitesToCheck != "")
{
sFTPSitesToCheck = sFTPSitesToCheck.Remove(sFTPSitesToCheck.Length - 1, 1);
}

// Create the string that stores SMTP sites to check
String sSMTPSitesToCheck = "";
for (itemIndex = 0; itemIndex < lstServicesToMonitorSMTP.CheckedItems.Count; itemIndex++)
{
sSiteName =
getSiteNameRegex.Match(lstServicesToMonitorSMTP.CheckedItems[itemIndex].ToString()).Value;
sSiteParameters = (Hashtable)m_SMTPSites[sSiteName];
sSMTPSitesToCheck = sSMTPSitesToCheck + sSiteName + ":" +
sSiteParameters["ServerComment"] + ":" +
sSiteParameters["ServerBindings"] + "\t";
}

if (sSMTPSitesToCheck != "")
{
sSMTPSitesToCheck = sSMTPSitesToCheck.Remove(
sSMTPSitesToCheck.Length - 1, 1);
}

Having got out the information we need to store, let's update the registry:

regKey.SetValue("HTTPSitesToCheck", sHTTPSitesToCheck);
regKey.SetValue("FTPSitesToCheck", sFTPSitesToCheck);
regKey.SetValue("SMTPSitesToCheck", sSMTPSitesToCheck);

Finally we need to restart the IISServicesMonitor service, assuming if was running when we started the save. We first refresh, just in case somehow it's been restarted already. Then, if doRestartAfterChanges is true (i.e. the service was running before the save button was clicked and it's not running now) then we start it. When it restarts, the new settings will be read in by the service:

serviceMonitor.Refresh();

if
            (doRestartAfterChanges && serviceMonitor.Status !=
            ServiceControllerStatus.Running)
{
serviceMonitor.Start();
}
MessageBox.Show("Settings
            Updated", "Success");
}
}

That completes the coding for the IIS Service Monitor Console. Our next task is to create the service itself.

The IISServiceMonitor

First we need to open VS.Net and create a new C# Windows Service project with the name IISServiceMonitor, as shown below:

Most of the code will be going inside a new class called clsIISServiceAccess. Let's create that class now by right clicking the project name in the solution explorer and selecting "Add new item", and then selecting a blank class as shown below:

Give the class the name clsIISServiceAccess.cs.

At the top of the class let's add the private variables that will be used throughout the class:

public class clsIISServiceAccess
{
#region private varables
String m_sServer; // contains the name or ip address of the local server
String m_sHTTPTestPage; // holds the location of the test page used to
// check that http services are working
Int32 m_iMonitoringInterval; // the interval, in milliseconds, that the 3
// services are monitored
Hashtable m_FTPSitesToCheck; // contains a hashtable object for each ftp
// site to be checked
Hashtable m_SMTPSitesToCheck; // contains a hashtable object for each smtp
// site to be checked
Hashtable m_HTTPSitesToCheck; // contains a hashtable object for each ftp
// site to be checked
System.Timers.Timer m_HTTPTimer; // a timer object - used to call code to
// check that the http service is still running ok
System.Timers.Timer m_FTPTimer; // a timer object - used to call code to
// check that the ftp service is still
System.Timers.Timer m_SMTPTimer; // a timer object - used to call code to
// check that the smtp servic still running
#endregion

Each of the private class variables above is set/get using public properties, as shown in the code below:

#region Public Properties
public String Server
{
get
{
return m_sServer;
}

set
{
m_sServer = value;
}
}

public String HTTPTestPage
{
get
{
return m_sHTTPTestPage;
}

set
{
m_sHTTPTestPage = value;
}
}

public Int32 MonitoringInterval
{
get
{
return m_iMonitoringInterval;
}

set
{
m_iMonitoringInterval = value;
}
}

public Hashtable HTTPSitesToCheck
{
get
{
return m_HTTPSitesToCheck;
}

set
{
m_HTTPSitesToCheck = value;
}
}

public Hashtable FTPSitesToCheck
{
get
{
return m_FTPSitesToCheck;
}

set
{
m_FTPSitesToCheck = value;
}
}

public Hashtable SMTPSitesToCheck
{
get
{
return m_SMTPSitesToCheck;
}

set
{
m_SMTPSitesToCheck = value;
}
}

#endregion

Let's turn to the public methods for this class, of which there are six. However they break down into two types - methods that start the monitoring of a particular IIS service, and methods that stop monitoring. We will start by looking at the one that starts the monitoring of HTTP sites:

public void
            StartMonitoringHTTP()
{
try
{
if (m_HTTPSitesToCheck !=
            null)
{
m_HTTPTimer = new
            System.Timers.Timer();
m_HTTPTimer.Elapsed+=new
ElapsedEventHandler(checkHTTPRunning);
m_HTTPTimer.Interval =
            m_iMonitoringInterval;
m_HTTPTimer.Enabled =
            true;
}
}
catch (Exception
            exp)
{
EventLog.WriteEntry("StartMonitoringHTTP()",
exp.Message, EventLogEntryType.Error);
}
}

As there is no user interface, it's vitally important any exceptions are caught using a try.catch statement and the exception details written to the NT event log.

The if statement checks to see there are actually any sites to check. If there are, then we create a new timer object and store its reference in class variable m_HTTPTimer. We then add our checkHTTPRunning() method, which we create later, to be called by the timer.

Looking at the reverse, i.e. stopping monitoring:

public void
            StopMonitoringHTTP()
{
try
{
if (m_HTTPTimer !=
            null)
{
m_HTTPTimer.Enabled = false;
m_HTTPTimer =
            null;
}
}
catch (Exception
            exp)
{
EventLog.WriteEntry("StopMonitoringHTTP()",
            exp.Message,
EventLogEntryType.Error);
}
}

If the reference in m_HTTPTimer is not null, then we disable the timer to stop it, and then allow it to be garbage collected by setting m_HTTPTimer to null, which removes any references to the timer object.

The start and stop methods for ftp and smtp are identical to the http ones, it is just the global variable and method called that changes:

public void
            StartMonitoringFTP()
{
try
{
if (m_FTPSitesToCheck !=
            null)
{
m_FTPTimer = new
            System.Timers.Timer();
m_FTPTimer.Elapsed+=new
ElapsedEventHandler(checkFTPRunning);
m_FTPTimer.Interval =
            m_iMonitoringInterval;
m_FTPTimer.Enabled =
            true;
}
}
catch (Exception
            exp)
{
EventLog.WriteEntry("StartMonitoringFTP()",
            exp.Message,
EventLogEntryType.Error);
}
}

public
            void StopMonitoringFTP()
{
try
{
if (m_FTPTimer !=
            null)
{
m_FTPTimer.Enabled = false;
m_FTPTimer =
            null;
}
}
catch (Exception
            exp)
{
EventLog.WriteEntry("StopMonitoringFTP()",
exp.Message,
EventLogEntryType.Error);
}
}

public
            void StartMonitoringSMTP()
{
try
{
if
            (m_SMTPSitesToCheck != null)
{
m_SMTPTimer = new
            System.Timers.Timer();
m_SMTPTimer.Elapsed+=new
ElapsedEventHandler(checkSMTPRunning);
m_SMTPTimer.Interval =
            m_iMonitoringInterval;
m_SMTPTimer.Enabled =
            true;
}
}
catch (Exception
            exp)
{
EventLog.WriteEntry("StartMonitoringSMTP()",
exp.Message, EventLogEntryType.Error);
}
}

public
            void StopMonitoringSMTP()
{
try
{
if (m_SMTPTimer !=
            null)
{
m_SMTPTimer.Enabled = false;
m_SMTPTimer =
            null;
}
}
catch (Exception
            exp)
{
EventLog.WriteEntry("StopMonitoringSMTP()",
exp.Message, EventLogEntryType.Error);
}
}

That's all the public methods for the class completed, so now let's turn to the private methods. Let's start by looking at the checkHTTPRunning() method.

To test if each http site is working we'll be retrieving a test page called "testpage.htm", which you'll need to create and add to the root directory of each site to be monitored. Our method takes two arguments, we use neither, they are there because this method is actually called by the timer.

As with the other methods, the main code is within a try.catch clause, just in case things go wrong. After declaring two variables we then have the start of the foreach clause that'll loop through each site in the m_HTTPSitesToCheck class variable:

private void checkHTTPRunning(Object source, ElapsedEventArgs e)
{
try
{
String urlToTest;
Boolean bServiceRunning;
foreach (Hashtable HTTPSite in m_HTTPSitesToCheck.Values )
{

Now within the foreach loop we begin building up the url to the test page. First we need to check if it's port 443, the port normally used for secure communications. If it is we need to start with https rather than http:

if (HTTPSite["Port"].ToString() ==
            "443")
{
urlToTest = "https://" +
            HTTPSite["Server"].ToString();
}
else
{
urlToTest =
            "http://" + HTTPSite["Server"].ToString();
}

Although web services normally use port 80 or 8080, it is possible that a site will use a different port, as a way to distinguish it from other sites on the same server. We therefore need to check if a port number has been defined, and if so add it to our url:

if (HTTPSite["Port"].ToString() !=
            "")
{
urlToTest = urlToTest + ":" +
            HTTPSite["Port"].ToString();
}

Finally we check to see if the url needs a header. Again, some sites distinguish themselves using a unique header that we'll need to add to the url. At the end of the url we add the name of the test page:

if (HTTPSite["Header"].ToString() !=
            "")
{
urlToTest = urlToTest + ":" +
            HTTPSite["Header"].ToString();
}
urlToTest = urlToTest + "/" +
            m_sHTTPTestPage;

Now that we have our url, let's use it to attempt to open a stream and read the test page:

try
{
WebClient myClient = new
            WebClient();
System.IO.Stream response =
            myClient.OpenRead(urlToTest);
response.Close();
response =
            null;
}
catch (WebException
            httpException)
{
bServiceRunning = false;
}

We're not actually interested in getting the page's data, we simply want to attempt to read it to see if we can. If we can't, then it suggests a problem with the web service, either that it's not running or perhaps we've not put the test page in the right place.

Finally we come to the end of the foreach and call the method CheckRestartSuccess(). This method, which we create next, will check to see if the service was running, or if a previous attempt to restart the service was successful or not. It'll make a note of when a site was down, of attempted restarts, and of their success:

CheckRestartSuccess(bServiceRunning, "W3",
            HTTPSite);
}
}
catch (Exception
            exp)
{
EventLog.WriteEntry("ServerMonitor.checkHTTPRunning()",
exp.Message, EventLogEntryType.Error);
}

}

Let's look at the CheckRestartSuccess() method used in the method we just discussed, as well as by the other service checking methods. It takes three arguments: whether the site is running, whether it's a w3, ftp or smtp service, and finally a hashtable object containing the site details.

We start at the top of the method by seeing if the service is not running. If it's not running then clearly we are going to have to attempt to restart the site, or if an attempt was made to restart the site and clearly it has failed then we need to re-start the whole of IIS:

private void CheckRestartSuccess(Boolean bServiceRunning, String sService, Hashtable Site)
{
if (!bServiceRunning)
{
if ((Site.ContainsKey("FullRestartAttempted")))
{
NotifyOfServiceStatus(sService + " - " + Site["ServerComment"],
sService + " site " +
Site["ServerComment"] + " was down. " +
"ServerMonitor's attempt to restart " +
"IIS has failed.", EventLogEntryType.Error);
Site.Remove("FullRestartAttempted");
}
else if ((Site.ContainsKey("SiteRestartAttempted")))
{
Site.Remove("SiteRestartAttempted");
NotifyOfServiceStatus(sService + " - " + Site["ServerComment"],
"Attempt to start " + sService +
" site " + Site["ServerComment"] +
" was unsuccessful. " +
"Attempting to restart all of IIS.",
EventLogEntryType.Warning);
RestartServices();
Site.Add("FullRestartAttempted", false);
}
else
{
NotifyOfServiceStatus(sService = " - " + Site["ServerComment"],
sService + " site " + Site["ServerComment"] +
" was down. " +
"ServerMonitor will attempt to re-start the site",
EventLogEntryType.Warning);
RestartService(Site["Server"].ToString(),
sService + "SVC", Site["SiteName"].ToString());
Site.Add("SiteRestartAttempted", false);
}

In the second if statement we check to see if the hashtable object referenced in the variable Site contains the key "FullRestartAttempted". When a restart is attempted, a key is added to that site's hashtable object so that we can tell if the restart is successful next time the site is checked. If the site is not running now and the key "FullRestartAttempted" exists, it means a full restart of IIS was done and despite this the service is still not back up and running. In this case, we would have no option but to stop checking that site and make a note of this in the event log.

If "FullRestartAttempted" was not found, then we check to see if a restart of that site was attempted. When a site is initially found to be down, we first attempt to restart just the site, not the whole of IIS. So if the site is still down and the key "SiteRestartAttempted" has been found, it means the site restart failed and we need to attempt a full re-start of IIS.

Finally in the else statement we deal with the case where this is an initial failure and no restart of any type has been attempted. We therefore just attempt to restart the site and add a key to the site's hashtable to make a note of this.

In the remainder of the method we deal with the situation where the site is now running, but we want to check to see if last time it was checked it was down and had to be restarted. The first else if deals with the case where a full restart has been attempted and this has been successful. We make a note of this success using the NotifyOfServiceStatus() method we create shortly, and then remove the "FullRestartAttempted" key from the site's hashtable object. The second else if deals with the case where a restart of just the site was attempted and this has been successful - again we make a note of this and remove the key:

else if ((Site.ContainsKey("FullRestartAttempted")))
{
NotifyOfServiceStatus(sService + " - " + Site["ServerComment"],
"full restart of IIS was successful",
EventLogEntryType.Information);
Site.Remove("FullRestartAttempted");
}
else if ((Site.ContainsKey("SiteRestartAttempted")))
{
NotifyOfServiceStatus(sService + " - " + Site["ServerComment"],
sService + " site " + Site["ServerComment"] +
" has been restarted successfully.",
EventLogEntryType.Information);
Site.Remove("SiteRestartAttempted");
}
}

In the CheckRestartSuccess() method we used a number of methods we have yet to create, including the NotifyOfServiceStatus(), RestartService() and RestartServices() methods. Let's start with NotifyOfServiceStatus():

private void NotifyOfServiceStatus(String sService,
String sMessage,
EventLogEntryType warningLevel )
{
EventLog.WriteEntry(sService, sMessage, warningLevel);
}

You can see that this is a very simple method that simply writes the message to the event log. In fact, we could dispense with the method altogether, but it does have a purpose. Let's say we want to do more than add an entry to the event log, perhaps we want to send an e-mail or text message to the system administrator warning them of the problems. In this case this method is the ideal place to add the code to do that.

Next, let's create the RestartService() method which restarts just one site of a particular service:

private void RestartService(String sServer , String sServiceName ,
String sSiteName)
{
String IISPath = "IIS://" + sServer +
"/" + sServiceName + "/" + sSiteName;
System.DirectoryServices.DirectoryEntry IIS;
IIS = new DirectoryEntry(IISPath);
Int32 iSiteServiceStatus;
iSiteServiceStatus =
Int32.Parse(IIS.Properties["ServerState"].Value.ToString());
if (iSiteServiceStatus == 5 || iSiteServiceStatus == 6)
{
IIS.Invoke("Continue");
}
else if (iSiteServiceStatus == 3 || iSiteServiceStatus == 4)
{
IIS.Invoke("Start");
}
}

We're using similar code to access the site as we used in the console when we accessed a site and obtained it's properties. The difference here is that we'll be invoking methods rather than just accessing properties. At the top of the method we create the path to the DirectoryEntry object in the IIS metabase that we're interested in, i.e. the site that needs starting. Then we get the site's current status. It could be the site has been paused rather than actually stopped, in which case we need to invoke the continue() method. If it's stopped then we need to invoke the start() method. A service status of 5 or 6 means the site is paused or being paused. A status of 3 or 4 means the site is stopped or being stopped.

We've seen how to restart a site, now let's look at how we restart the whole of IIS. Our method RestartServices() does the full restart of all of IIS's services. The method first stops the monitoring timers, as it can sometimes take a while for IIS to restart. We don't want to start monitoring again until the restart is complete. Then we call FullIISRestart() where the actual restart is done. Finally all the timers are started going again:

private void RestartServices()
{
//
            stop Server checks;
Hashtable TimersEnabled = new
            Hashtable();
TimersEnabled.Add( "HTTPTimer",
            m_HTTPTimer.Enabled);
m_HTTPTimer.Enabled =
            false;
TimersEnabled.Add("FTPTimer",
            m_FTPTimer.Enabled);
m_FTPTimer.Enabled =
            false;
TimersEnabled.Add("SMTPTimer",
            m_SMTPTimer.Enabled);
m_SMTPTimer.Enabled =
            false;

FullIISRestart();

// start Server
            checks;
m_HTTPTimer.Enabled =
            (bool)TimersEnabled["HTTPTimer"];
m_FTPTimer.Enabled =
            (bool)TimersEnabled["FTPTimer"];
m_SMTPTimer.Enabled =
            (bool)TimersEnabled["SMTPTimer"];
}

Turning to the FullIISRestart() method, this restarts IIS by stopping the IISAdmin service, then starting it again. Stopping it will automatically cause all the services within IIS to be stopped, as they are all dependant on this service being running. It's a very effective way of clearing problems with IIS, in fact if a full restart doesn't solve the problem then it's most likely that a system admin is needed. We use the ServiceController class as our means of controlling the IISAdmin service. Then if the service is not stopped, then we stop it.

We give IIS time to shut down by using Thread.Sleep() - this puts our code to "sleep" but still gives other running programs and services the chance to run. On waking, we need to start the services again. Unfortunately we can't just start IISAdmin service and expect the other services to restart, we need to restart each service individually. However, IISAdmin must be running before any other IIS services can be started. We don't know which services are running on IIS, as this varies from server to server depending on installation, so instead we loop through each service dependant on IISAdmin and start them. The DependentServices property gives us a collection of those services, and we can easily loop through and start them:

private void FullIISRestart()
{
//
            stop IIS;
ServiceController iisAdminController = new
ServiceController("IISADMIN");
if (iisAdminController.Status
            !=
            ServiceControllerStatus.Stopped)
{
iisAdminController.Stop();
}

//
            give IIS time to shut down;
Thread.Sleep(30000);

// start
            iis;
if (iisAdminController.Status !=
            ServiceControllerStatus.Running)
{
iisAdminController.Start();
}

//
            Give IISAdmin a chance to re-start;
Thread.Sleep(10000);

foreach ( ServiceController dependantService in
iisAdminController.DependentServices)
{
if
            (dependantService.Status !=
            ServiceControllerStatus.Running)
{
dependantService.Start();
}
}
//
            give the other services a chance to restart before monitoring
            them;
Thread.Sleep(60000);
}

We've now got all the code to check that the web service is running, we need to check the ftp and smtp services. We'll be checking the services are running by creating a new session, either ftp or smtp. If we can connect to the server, start a session, and get a response from a session we started then we know that the service is running ok.

Let's start with the code that monitors the ftp service. This method gets called by the timer we started to check the ftp service. The code loops through each site, represented by a hashtable object in the m_FTPSitesToCheck hashtable. It calls the GetServerStatusCode() method and it's this method that attempts to start a session, in this case ftp session, with the server and returns the response code which will indicate if the session was successful or not. A code of 220 means a session started correctly, anything else suggests a failure. If there was a failure, then we need to restart the site, and possibly IIS, and so we call the CheckRestartSuccess() method that we created earlier to do just that:

private void checkFTPRunning(Object source, ElapsedEventArgs e)
{
String sResponseCode;
String sResponseMessage;
Boolean bServiceRunning;
try
{
foreach (Hashtable Site in m_FTPSitesToCheck.Values)
{
sResponseMessage =
GetServerStatusCode(Site["Server"].ToString(),
Int32.Parse(Site["Port"].ToString()), 10);
sResponseCode = sResponseMessage.Substring(0, 3);

if (sResponseCode != "220")
{
bServiceRunning = false;
}
else
{
bServiceRunning = true;
}

CheckRestartSuccess(bServiceRunning, "MSFTP", Site);
}
}
catch (Exception ftpException)
{

EventLog.WriteEntry("ServerMonitor.checkFTPRunning()",
ftpException.Message,
EventLogEntryType.Error);
}
}

The code that checks if the smtp service is running is almost identical, except for which global variables are used, and that the name of the service is smtp rather than msftp:

private void checkSMTPRunning(Object source, ElapsedEventArgs e)
{
String sResponseCode;
String sResponseMessage;
Boolean bServiceRunning;
try
{
foreach (Hashtable Site in m_SMTPSitesToCheck.Values)
{
sResponseMessage =
GetServerStatusCode(Site["Server"].ToString(),
Int32.Parse(Site["Port"].ToString()), 10);
sResponseCode = sResponseMessage.Substring(0,3);
if (sResponseCode != "220")
{
bServiceRunning = false;
}
else
{
bServiceRunning = true;
}
CheckRestartSuccess(bServiceRunning, "SMTP", Site);
}
}
catch (Exception smtpException)
{
EventLog.WriteEntry("ServerMonitor.checkSMTPRunning()",
smtpException.Message,
EventLogEntryType.Error);
}
}

The final method we need to add to the class is GetServerStatusCode(). We saw this used above in the ftp and smtp checking methods to see if a particular site is running. We'll be using the System.Net.Sockets namespace to make the network connections necessary for ftp and smtp communications. Starting with the function header:

private String GetServerStatusCode(String sServer, Int32 iPort, Int32 iTimeOut)
{
try
{

The function takes three parameters, sServer, which is the ip or name of the server to connect to, iPort, which is the port number (usually 25 for smtp and 20 or 21 for ftp), and iTimeOut, which is how long the code should wait for a response from the server.

The protocol in both ftp and smtp is tcp ip, and so we declare the variable tcpClient and set it to reference a new TcpClient object. This class provides the tcp connectivity, reading the information requires the NetworkSystem class:

System.Net.Sockets.TcpClient tcpClient = new
System.Net.Sockets.TcpClient();
System.Net.Sockets.NetworkStream
            networkStream;
Double Start;
Double Tmr;
byte[]
            bytes;
byte[] sendBytes = new byte[6];
String
            sResponseCode;

Next we connect to the server and open a data stream ready to read the response back from the server to the incoming request:

tcpClient.Connect(sServer,
            iPort);
networkStream = tcpClient.GetStream();
tcpClient = new
            System.Net.Sockets.TcpClient();

Now we wait for the server's response. We keep looping the while data is not available via the network stream. If we exceed the time limit then we return from the function with the value "TIMEOUT":

Start =
            DateTime.Now.TimeOfDay.TotalSeconds;

bytes = new
            byte[tcpClient.ReceiveBufferSize];

while
            (!networkStream.DataAvailable)
{
Tmr =
            DateTime.Now.TimeOfDay.TotalSeconds - Start;
if (Tmr >
            iTimeOut)
{
sResponseCode = "TIMEOUT!";
return
            sResponseCode;
}
}

If we have got this far then it means some response has been returned by the server and as it's the start of a new session, that response will be a code and message telling us if the server has successfully accepted our new session request. What we need to do now is get the response, assuming data is available:

if (networkStream.DataAvailable)
{
//
            Returns the data received from the host to the
            console.;
networkStream.Read(bytes, 0,
            tcpClient.ReceiveBufferSize);
sResponseCode =
            Encoding.ASCII.GetString(bytes);

The response comes as a byte array which we need to convert to a string using the Encoding.ASCII.GetString() method.

We don't want to leave the session hanging, so let's terminate it cleanly rather than letting it time out, as this helps free server resources:

sendBytes[0] = (byte)'Q';
sendBytes[1] =
            (byte)'U';
sendBytes[2] = (byte)'I';
sendBytes[3] =
            (byte)'T';
sendBytes[4] = 13;
sendBytes[5] =
            10;
networkStream.Write(sendBytes, 0, 6);
}

Closing a session simply involves sending "QUIT" followed by a line feed and carriage return.

Finally, if no data was available to read, we set sResponseCode to "TIMEOUT". At the end of the method we close the tcp session:

else
{
sResponseCode =
            "TIMEOUT!";
}

if (tcpClient !=
            null)
{
tcpClient.Close();
}

tcpClient =
            null;
return sResponseCode;
}
catch (Exception
            ex)
{
return ex.Message;
}
}

That completes the code for the class. We now need to add code to the OnStart and OnStop events of our Service. These events fire when the Service is first started and when it's ended. In OnStart we'll read in the settings from the registry that were placed there by the console:

protected override void OnStart(string[] args)
{
try
{
RegistryKey regKey;
regKey = Registry.LocalMachine.OpenSubKey("System\\ServicesMonitor", true);
String sServer = regKey.GetValue("Server", "localhost").ToString().Trim();
String sTestPage = regKey.GetValue("TestPage",
"test.htm").ToString().Trim();
String HTTPValuesToCheck = regKey.GetValue("HTTPSitesToCheck",
"").ToString().Trim();
String FTPValuesToCheck = regKey.GetValue("FTPSitesToCheck",
"").ToString().Trim();
String SMTPValuesToCheck = regKey.GetValue("SMTPSitesToCheck",
"").ToString().Trim();
Int32 iMonitoringInterval = (Int32)regKey.GetValue("MonitoringInterval",
300000);

Then we create a new object based on the clsIISServiceAccess we created above, and set the properties to the values just obtained from the registry:

m_IISServiceAccess = new clsIISServiceAccess();
m_IISServiceAccess.MonitoringInterval = iMonitoringInterval;
m_IISServiceAccess.Server = sServer;
m_IISServiceAccess.HTTPTestPage = sTestPage;

In the remaining code we check to see if any sites for each of the three services, web, ftp and smtp, need checking. We use getSites() to split the string obtained from the registry and return a hashtable object containing all the sites needing monitoring:

if (HTTPValuesToCheck != "")

{
m_IISServiceAccess.HTTPSitesToCheck = getSites(HTTPValuesToCheck, sServer);
m_IISServiceAccess.StartMonitoringHTTP();
}

if (FTPValuesToCheck != "")

{
m_IISServiceAccess.FTPSitesToCheck = getSites(FTPValuesToCheck, sServer);
m_IISServiceAccess.StartMonitoringFTP();
}

if (SMTPValuesToCheck != "")

{
m_IISServiceAccess.SMTPSitesToCheck = getSites(SMTPValuesToCheck, sServer);
m_IISServiceAccess.StartMonitoringSMTP();
}
}
catch (Exception exp)
{
EventLog.WriteEntry("OnStart()", exp.Message, EventLogEntryType.Error);
}

}

The method getSites() is identical in every way to getIISSites() in the console program and so is simply repeated here:

private Hashtable getSites(String sSitesString, String sDefualtServer )
{
Regex getDataValuesRegex = new Regex("(?<sitename>
[^:]*):(?<Servercomment>[^:]*):(?<Server>[^:]*):(?<port>
[^:]*):(?<header>[^:]*)",
RegexOptions.Compiled);
Match siteMatch;
Hashtable SitesToCheck = new Hashtable();
Hashtable SiteProperties;
String[] sSites = sSitesString.Split('\t');

foreach( String sSite in sSites )
{
siteMatch = getDataValuesRegex.Match(sSite);
SiteProperties = new Hashtable();
SiteProperties.Add("SiteName",
siteMatch.Groups["sitename"].Value);
SiteProperties.Add("ServerComment",
siteMatch.Groups["Servercomment"].Value);
SiteProperties.Add("Port", siteMatch.Groups["port"].Value);
SiteProperties.Add("Header",
siteMatch.Groups["header"].Value);
if (siteMatch.Groups["Server"].Value == "")
{
SiteProperties.Add("Server", sDefualtServer);
}
else
{
SiteProperties.Add("Server",
siteMatch.Groups["Server"].Value);
}
SitesToCheck.Add(SiteProperties["SiteName"],
SiteProperties);
}

return SitesToCheck;
}
}

Next comes the OnStop event, which fires when the service is stopped. We simply call the IISServiceAccess class's StopMonitoring methods for each of the services and then set the m_IISServiceAccess to null to free its reference:

protected override void OnStop()
{
//
            TODO: Add code here to perform any tear-down necessary
to stop
            your
            service.
try
{
m_IISServiceAccess.StopMonitoringFTP();
m_IISServiceAccess.StopMonitoringHTTP();
m_IISServiceAccess.StopMonitoringSMTP();
m_IISServiceAccess
            = null;
}
catch (Exception
            exp)
{
EventLog.WriteEntry("OnStart()", exp.Message,
EventLogEntryType.Error);
}
}

We have been using a global variable, m_ IISServiceAccess, which we need to define at the top of the class:

private clsIISServiceAccess m_IISServiceAccess;

That completes all of the code for this case study. However before we run the service we need to set the application up.

Application Setup

The management console needs no set up - simply run it. However there are a few steps to undertake before we can use the monitoring service.

First in the service project within VS.Net we need to add an installer to our project. From the design view click the Add Installer link, which is found in the properties box. See below:

We need to make sure that the service name is IISServiceMonitor, and also make sure it's run under the localsystem account.

We now need to let Windows know about our service. First we need to compile a release version of our monitor. Then we need to use the command line program InstallUtil.exe to register the service by running this program and passing it the name and path of our IISServiceMonitor.exe, as shown below:

Our service is now ready to use, and we can change its settings using the console we created.

Conclusion

This case study has demonstrated a lot of useful techniques for manipulating IIS, techniques that can also be extended to general Active Directory manipulation too. We've seen how we can access the properties and methods of IIS.

I've had the monitor service installed on a live server for four months with no problems, so what we have here is a reliable basic IIS monitor. However it is basic, and there are plenty of additions that could be made:

  • Full restart of the machine if a restart of IIS failed
  • Defining a specific webpage for each site
  • Sending an e-mail to the administrator should a site fail
  • Sending an SMS message to an administrator
  • Sending an administrative alert
  • Keeping an easily accessible log of problems and providing access to it from the console
  • Monitoring of the POP (Post Office Protocol Service)
  • Defining a sites level of importance. For example, we may only want to restart IIS if an important site has gone down.

The code has been designed to be easily extendable, so adding the above can be done without seriously breaking the code. If we want very sophisticated monitor then we could purchase a commercial product like Microsoft Operations Manager.


Protecting Your Code - Reflection, Obfuscation and Encryption - Part 2

Protecting Your Code - Reflection, Obfuscation and Encryption - Part 2
http://www.csharptoday.com/content.asp?id=1855
Article Rating: 4.5
Published on August 26, 2002
Abstract
The first part of this case study by Roger Rowland examined how reflection and CodeDOM could be used together to reverse engineer a compiled .NET assembly into C# source code. This second part looks at the implications of this, and explores ways of attacking the problem. We’ll look in detail at code obfuscators – how they work, what are their strengths and weaknesses, we’ll discuss alternative methods of securing sensitive code, and finally develop an example application demonstrating an alternative technique for making life more difficult for the casual hacker.
Case Study

The first part of this case study (here) examined how reflection and CodeDOM could be used together to reverse engineer a compiled .NET assembly into C# source code. This second part looks at the implications of this, and explores ways of attacking the problem. We'll look in detail at code obfuscators - how they work, what are their strengths and weaknesses, we'll discuss alternative methods of securing sensitive code, and finally develop an example application demonstrating an alternative technique for making life more difficult for the casual hacker.

Introduction

Before the advent of .NET, Win32 executables were hard to reverse engineer. You had to have a certain level of skill and knowledge and be very familiar with assembly code and operating system internals. It wasn't a game that everyone could play. Over the years, a variety of techniques have been invented, and third-party tools developed, to make it even harder for the hackers and crackers. In essence there already exists some sort of equilibrium where the good guys and bad guys use quite sophisticated techniques to outwit each other, while the rest of the world just gets on with the job and takes very little notice.

With .NET, the situation has changed somewhat. Now, you don't need a great deal of knowledge and expertise to get inside someone else's code. We proved that in the first part of this case study. Now anyone with a copy of ILDasm, let alone any of the commercial .NET decompilers, can look over your assemblies and reverse engineer your designs and source code. Suddenly, your software licensing code, copy protection mechanisms, and proprietary business logic are much more open to the common gaze - whether it's legal to do so or not. Anyone can peruse the details of your software for whatever reason they like - be it an attempt to debug their own interface with your application or to search for security flaws or loopholes. This must be enough to make many people pause for thought.

In this case study we're first going to take a look at the different options available for protecting your code. Then we'll look in more detail at obfuscators. These are tools designed to make the information retrievable from an assembly more difficult to interpret. We'll see in principle how they manipulate the metadata in a .NET assembly, and look at the benefits and any side effects the use of such tools may bring. We'll go on to explore encryption by taking a look at what's available as standard within the .NET framework class library (FCL) and finally develop an alternative technique to try and push those goalposts a little further out of reach.

System Requirements

This system requires Visual Studio .NET V1.0 or later running on Windows 2000 or Windows XP. The case study assumes that the reader has a good understanding of C# and some familiarity with encryption terminology.

Case Study Structure

After a very brief recap on the ground we covered in part one of this case study, we'll look at the options available for protecting compiled code from decompilers - primarily by obfuscation. Then we'll finish up by demonstrating an alternative technique using just pure C# and the FCL to add one more weapon to the arsenal.

Recapitulating on Reflection

In part one of this case study we developed a simple .NET decompiler, which took a compiled assembly and reverse engineered this back into understandable C# source code using just the System.Reflection namespace together with the System.CodeDOM code-generating classes. We showed how this was possible due to the comprehensive nature of the metadata stored in every .NET assembly and that it did not take a great deal of time or ingenuity to do so.

The real dilemma is that this situation arises as a natural consequence of the techniques Microsoft used to bring about some of the major advantages of .NET. In order to tackle things like the long-standing "DLL hell" problem, binary compatibility issues with COM, and to provide language interoperability in an object-oriented environment, it almost became inevitable that we would end up with completely self-describing components. Which is exactly what we've got. The JIT (Just In Time) compiler, which forms part of the .NET framework, uses the metadata held in .NET executables to achieve early-binding without having to worry about the exact order of methods in a vtable or the memory footprint of an object. This allows applications to adapt at run time when updated versions of components are installed and is nicely complemented by the side-by-side versioning capability of the CLR. I have to say that, in my opinion, it is an entirely logical solution and I really like it. It's certainly not going to go away.

So, is this something we're stuck with? Do we have to accept that it will now be easier than ever before for our applications to be reverse engineered? After all, this is not a new problem - Java developers have always been faced with this issue and the world still turns. Is it just that it's different to what we've been used to? Should we be using .NET just for server-side development? If so, why is there such a wealth of support in .NET for deployment and end-user security? How can we migrate our existing applications to the .NET platform and make sure we retain control of our intellectual property?

Well, the answer is that there are a number of options. The word of the day here is compromise. It's impossible to prevent reverse engineering of compiled code - all we can do is make it harder. As long as we accept that, there's a lot that can be done to maintain the equilibrium. That's what this second part of the case study is all about

Protecting Intellectual Property

The following sections will describe some of the current options available for protecting the intellectual property contained in compiled .NET assemblies. This is not intended to be a review or comparison of particular products or techniques. The purpose of this section is to give you as much detail as I can on the technical issues involved in protecting your code in order to help you decide on the best policy for your individual requirements. We will cover the following areas:

  • Use of unmanaged code
  • Native code generation
  • Code obfuscation
  • Other solutions

Use of Unmanaged Code

This is perhaps the most obvious solution. At one end of the spectrum, we forget the .NET environment and develop all of our applications as unmanaged code using C++. At the other end, we identify a couple of important subroutines and implement them in an unmanaged DLL, using P/Invoke to call into these from our managed code as required. Alternatively, we could use the managed C++ compiler to produce an assembly containing both managed and unmanaged code.

At first sight this might seem like a nice low-impact solution. We probably already have the necessary C++ (or whatever) skills to write and maintain the unmanaged portions, and it might seem like a gentle way of dipping a toe in the .NET waters by redeveloping non-critical parts of a system first and proceeding in an incremental manner. Of course, we need to be happy that some portions of our system will still be easy to decompile and so not all applications will be suitable. However, this approach really just postpones the issue and there are other disadvantages. There are no real killers but you nevertheless need to be aware of them.

The ability to call unmanaged code from .NET requires a highly trusted security status, which may mean your application has to have more permissions than it really needs. As .NET does not provide for restricted delegation of authority, this interface into unmanaged code presents an ideal point of attack for viruses and trojans to use your application for their own undesirable purposes. At the point where a trusted assembly calls into unmanaged code, the unmanaged code inherits the same permissions as the caller and, similarly, any executable invoked in turn by this unmanaged code also inherits the original permissions. It would be quite easy for someone to insert a call into your unmanaged dll, which would execute any program of their choice with a highly-trusted status. It is not possible in the .NET environment to restrict this delegation of authority to particular modules and this is a potential security weakness.

Using unmanaged code inevitably ties you down to a particular implementation platform. This is not a problem just yet because there's no established cross-platform support for .NET. However, if and when this happens you immediately lose out on a big potential benefit.

If you're using P/Invoke to access the unmanaged code in your application, there is a performance hit. How much of a problem this causes will depend on how often you call into the unmanaged code and how much data needs to be marshalled across the interoperation boundary.

Maybe an obvious point, but one that shouldn't be missed, is that you will need to retain the necessary skills to support and maintain the unmanaged portions of code and also ensure you have a suitable development environment to do so. I'm not suggesting for a moment that C# will replace C++ overnight (or even very soon) but I am reminded of the very large daily rates on offer for "legacy" Cobol programmers in the run up to Y2K - maybe an extreme example but thinking ahead is not as common as 20/20 hindsight.

Ok, and of course unmanaged code can also be reverse engineered - a counter often espoused on the newsgroups and, in my opinion, not particularly helpful. This is of course exactly where we were before .NET and we're just trying hard not to move too far in the wrong direction. I don't think that's unreasonable.

In essence, use of unmanaged code to protect intellectual property is a viable option for some people in some situations, but it's not for everyone.

Native Code Generation

If you frequent the .NET newsgroups, you'll see that some people have asked whether the use of the ngen.exe utility, which is part of the .NET Framework SDK, might be employed to prevent people decompiling your assemblies. The short answer to this is really no. The ngen utility is specifically designed to allow you to pre-JIT an assembly on a user's computer at the time the application is installed. It is not designed to protect your assembly from reverse engineering.

What ngen does is to compile your assembly to native code during the installation process and to store this in the Native Image Cache, which is part of the Global Assembly Cache (GAC). This is an entirely worthwhile and reasonable exercise and will reduce application load and run times on the target system. The crux of the matter is that it has to run at install time - you can't just copy and paste a ngen image out of the GAC. In any case, the native image won't run without the CLR, and the CLR still needs access to the original assembly, complete with metadata, for type resolution, reflection and version checking.

Just attempting to use ngen to produce a native image before distribution to the end user is fraught with problems. Not only do you need to do this on a machine with the same processor and operating system but you also need to consider a host of other dependencies. You need to know the exact identities of any other assemblies you bind to, which security policy the code should run under, and what version of the CLR is on the end user's machine. Not a simple task! If you can identify all of these, then any subsequent change on the user's machine that affects dependent assemblies may result in the CLR disregarding your native image and reverting to JIT - for which it needs the IL in the original assembly.

Some documentation suggests that pre-JITted assemblies are not compiled with the same optimisations possible with a true JIT on the target platform. I don't have any hard evidence about this, but it's certainly true that the operating system and processor make a big difference. For example, Thread Local Storage (TLS) is optimised for a particular OS, and some processor instructions (e.g. CMOV) can be used on a Pentium 4 but not a Pentium 3. These differences definitely exist but are not comprehensively documented and are of course subject to change. Still keen?

The documentation for ngen states "If Ngen.exe encounters any methods in an assembly that it cannot generate, it excludes them from the native image. When the runtime executes this assembly, it will revert to JIT compilation for the methods that were not included in the native image." So there's not even any guarantee that you can completely pre-compile your assembly anyway. And, assuming it has pre-compiled, the ngen image will contain data structures defined and used in the CLR which could change in later versions - you may find that a .NET framework service pack renders your distributed image invalid. The CLR won't care, it will again revert to JIT and carry on regardless - assuming, that is, that you have also made the original assembly available, which somewhat defeats the object.

There are so many things which make this option a non-starter that you need to be really sure of what you're doing and the likely effects before you attempt to do this. If you want my advice - don't go there. Maybe look at a third-party obfuscator instead.

Acknowledgements

In preparing this part of the case study, I would like to extend my thanks to the following individuals and their respective organisations for their valuable time and information. Without exception, they have been more than happy to discuss technical details of their products and so made the preparation of this case study both satisfying and informative.

Brent Rector (http://www.wiseowl.com/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq) - Demeanor for .NET, Enterprise Edition

Daniel Jebaraj (http://www.syncfusion.com/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq) - Essential Obfuscator

Dr. Huihong Luo (http://www.remotesoft.com/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq) - Remotesoft Obfuscator and .NET Protector

Gabriel Torok (http://www.preemptive.com/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq) - Dotfuscator

Obfuscation

Already there are a number of commercial .NET obfuscators. I've looked at four of the most well-known in researching this case study and have tried to present a composite view of the technology on offer. There has obviously been a great deal of thought and effort behind the development of these products, with some useful innovations coming to light. I don't doubt that they have all generated a high level of interest from the .NET community and will continue to do so as the .NET movement spreads.

In this section, which is quite detailed, we'll look first at the principles underlying obfuscation before going into more depth as we see exactly how these products go about their job. This section will conclude with a discussion of the strengths and weaknesses of obfuscation in principle.

What is Obfuscation?

The Oxford English Dictionary defines obfuscate like so.

"obfuscate - 1. obscure or confuse (a mind, topic, etc.). 2. stupefy, bewilder"

In the context that we're using the word, this is exactly what we mean. Obfuscation is not about preventing someone from reverse engineering your code, it's all about making the information they get out much more difficult to understand.

The primary method employed by obfuscators is to rename symbols in the metadata. By symbols, I mean the names of your classes, methods, member fields, etc. This has absolutely no effect on the functionality of your code - in these terms it still runs as it always did, the CLR still sees it as a valid .NET assembly and it can still (in theory) be decompiled just as easily. It's just that - to a human reader - it's much more difficult to make any sense out of it when all of the identifiers are meaningless.

Most of the commercial obfuscators try to throw away as much information as possible without compromising the function of your code in any way. This is known as a lossy transformation and the extent to which this can be done might surprise you. Also, the effect that the loss of these vital clues has on the ability of someone to understand your code should not be underestimated. Mind you, it's not just humans who need to be fooled - it's not so easy to confuse a decompiler by using duplicate names and apparent absence of context. Some of these more psychological obfuscatory techniques can be mitigated during decompilation so there's a real battle here to produce something that is less comprehensible both to humans and machines and yet that still has to run as it always did. A tough challenge.

So let's see how it's done.

Obfuscation in Practice

Some .NET obfuscators - certainly early ones - adopted a technique that has become known as round-tripping. This involves dumping an assembly to an IL source file (using ILDasm), making changes to this source file, and finally recompiling with ILasm. It's fair to say that this technique has some major drawbacks and is not used by any of the commercial obfuscators I looked at. The biggest problem with this is the inability to handle mixed images - those that contain both managed and unmanaged code. This includes almost every assembly developed with the Visual C++ .NET compiler.

The problem is that ILDasm just ignores any code that is not IL - if you disassemble a mixed image .NET assembly using ILDasm, any native code functions will be left out of the dumped source file. This is really a problem for applications developed with the managed C++ compiler. In this environment, it is possible (and likely) that there will be both managed and unmanaged code in the same assembly, so any attempt to decompile, obfuscate, and recompile with ILasm will result in an incomplete assembly.

Mainly for this reason, all of the .NET obfuscators examined here do not employ round-tripping but operate directly on the metadata in a .NET assembly, which, in my opinion, is the right thing to do.

In addition to favouring direct obfuscation of .NET binaries, it seems that most commercial obfuscators are moving feature-wise in the same direction. There are differences in terminology and undoubtedly implementation, but there are a number of common features we can examine. These are:

  • Symbol Obfuscation
  • Metadata Obfuscation
  • Control Flow Obfuscation
  • String Encryption

I'll describe each of these in turn and then cover a few of the additional features on offer.

Symbol Obfuscation

Symbol obfuscation affects the names of the types and members defined in an assembly and, in some cases, types referenced in other assemblies. To understand this better, we need to see what the internal structure of a .NET assembly is like. We'll do this in general terms, not being too pedantic about the details. If you are interested in looking deeper then consult the Word document Partition II Metadata.doc, which is installed along with the framework SDK.

Figure 1 shows, in schematic form, the major parts of a compiled assembly (an exe or a dll). At the head of the file are conventional PE (Portable Executable) headers. These identify the file type to the operating system and allow it to be loaded for execution. Apart from the mechanism used to ensure that this executable is handled by the CLR, this is essentially no different to any executable binary and is of little interest to us for the purposes of obfuscation.

Figure 1 - Inside an Assembly

Following the PE headers is the CLI header (Common Language Infrastructure). The CLI header contains all of the runtime-specific data (e.g. CLR version, strong name signature) and includes pointers to the metadata and the method definition of the assembly's entry point. This is followed by the metadata and IL code itself - the section labelled CLI Data in Figure 1 - this is the section we're really interested in, so we'll look a little closer inside.

The metadata (as I'll generically refer to this section of the assembly) contains a number of tables and some heaps. The tables are of three types - there is a set of tables that describe the types and members defined in the assembly, a set of tables that list the types and members referenced by the assembly, and a set of manifest tables that basically contains the names of the files that are part of the assembly. The latter primarily concerns multi-file assemblies, where an assembly can be logically composed of a number of physical modules.

The heaps include the string heap and the blob heap. These heaps are used to store various information and are referenced by pointers in the metadata tables. For example, the names of all methods, fields, properties, etc. are held in the string heap, and the values of compile-time constants, assembly hash value, public keys, etc. are held in the blob heap.

Figure 2 shows this schematically.

Figure 2 - Inside the Metadata

The items we're most interested in for performing symbol obfuscation are the string heap and the definition tables. Some obfuscators allow a number of interdependent assemblies to be obfuscated as a logical unit, in which case some of the information in the reference tables is also changed. We'll just look at the more simple case here, as the principle is the same.

Out of the many definition tables held in the metadata, three of the most relevant for performing symbol obfuscation are the following:

Table Name | Description
TypeDef | Contains one row for each type defined in the assembly. Indicates such things as the accessibility (public, private, etc.) and the base type, and contains pointers to the methods, fields, etc. defined in the relevant metadata table. The type's name and its namespace are indicated by pointers into the string heap.
FieldDef | Contains one row for every field defined in the assembly. Indicates the field type and accessibility and holds the field name as a pointer into the string heap.
MethodDef | Contains one row for each method defined in the assembly. A pointer to the string heap provides the name and a pointer to the blob heap provides the signature. In addition to the accessibility and other flags, there will be a pointer to the IL code for the method and pointers to parameters in the ParamDef table

You can see how each table is organised to keep the size of the assembly to a minimum. The use of the string heap plays a big part in this as each unique string is held only once. If, in our source code, we used the same identifier for a method, field, parameter, property, event, etc. then the compiler just puts one string in the heap and ensures that all relevant rows in the definition tables contain a pointer to it. This technique extends to the reference tables too - if we call a method on a type in a referenced assembly that happens to have the same name as a member in our own assembly, then we still only hold the name in one entry on the string heap.

You might possibly begin to see how the process of symbol obfuscation is assisted by this structure. Especially when you consider that the actual names used in the source code only need to remain visible for certain purposes. After all, the compiled IL code has already been processed to ensure that any types, methods, fields, etc. that it uses are resolved into pointers to the correct row in the relevant metadata table. Once it's been compiled (to the IL stage), the original names are largely redundant. We only need to be careful about situations where the name still matters. These include the following:

Any public types, methods, fields, properties, events, etc. defined in our assembly. There's not much point in exposing a public type and then changing its name!

Specific places in code where types are referenced by name - for example by using the reflection classes to create an instance of a type (effectively, late binding, or dynamic object creation).

Serialization, and therefore remoting. By default, the process of serializing to and from XML uses the names of members (determined at run time) as element names. If, at run time, your applications have obfuscated versions of the original names then this will not work correctly.

Apart from the above cases, the majority of private members defined in an assembly can be safely renamed without affecting the operation of the code in any way. The way that an obfuscator will do this is by manipulating values in the string heap and changing pointers in the definition tables to make the resulting picture appear nothing like your original source code.

Most obfuscators start by replacing meaningful names with single-character identifiers and re-use these same names as much as possible while still keeping the assembly logically the same as far as the CLR is concerned. This means that method names may be heavily overloaded - as long as no two methods have the same signature, they may as well share the same name - even if this makes no sense as far as the application design goes. That's the whole point. It's not just the original names that are lost, but the picture of which methods are genuinely overloaded. This all helps to confuse matters if the assembly is decompiled.

This obfuscation can be performed on a type definition or any of its members - not just the methods. As long as the resulting assembly is still valid to the CLR it doesn't make any difference. For example, consider the following class definition:

Figure 3 - Before Symbol Obfuscation

Figure 3 shows a picture of the content of the definition tables and string heap, which might have resulted from building this C# class. The class and its members are entered by the C# compiler into the relevant definition tables in the assembly's metadata. The string heap contains all of the member identifiers, and the rows in the definition tables reference these using pointers.

Figure 4 shows how the picture may appear after obfuscation.

Figure 4 - After Symbol Obfuscation

You can see that there's been no serious change made to the metadata - all of the original definitions are untouched apart from the pointers to the string heap. The string heap itself has been trimmed and this is one reason why assemblies physically shrink following obfuscation.

While still making sense to the CLR, any attempt to use the information now contained in the metadata to decompile the class would result in the following (invalid) C# source code:

private class a
{
string a;
int
            a;
a a;
a[] a() {.}
}

This is obviously much less understandable to any human reader. However, note that the structure of the types defined in the assembly is still visible and class hierarchies can still be derived - although some obfuscators do attempt to randomise things by placing related classes into different namespaces and vice versa (we saw in part one of this case study that the namespace used when declaring a type is held in the metadata but is not required by the runtime, so there is no adverse effect on the function of an assembly obfuscated in this way).

Notice that we have done nothing that affects the IL code which implements each method body, so the application will function exactly as it did before obfuscation.

Metadata Obfuscation

There is some information held in metadata definition tables that is not really necessary at runtime. In particular, this is information about properties and events. In the first part of this case study, we discovered exactly how these constructs are implemented in a compiled assembly so this should not be too surprising (unless of course you didn't read the first part!).

The get and set accessors of a property are implemented just like any other method, and any use of properties in the remainder of your assembly's code is resolved into the appropriate method call at compile time (that's at the C# compiler stage, not the JIT compiler). Similarly, any code that registers or unregisters interest in an event is resolved into the corresponding add, remove or raise method call and the remaining work is taken care of by the compiler-generated MulticastDelegate.

This means that there is no need to treat properties or events as special cases at runtime. However, the metadata includes a number of tables that serve to identify the properties and events defined in the assembly and to associate these with their respective MethodDefs. This makes it easy - via reflection - to identify which methods handle which properties, and this information could be useful to a hacker (it was certainly useful for our decompiler in part one). Remember, the presence of these tables does not directly affect runtime behaviour.

Figure 5 shows how a number of metadata tables are used to associate events with a type. The EventMap table allows a number of events in the Event table to be linked with their defining type. For each such event, the MethodSemantics table identifies the relevant entries in the MethodDef table for those methods that implement the event. This rather convoluted linking allows the exact type of method (add handler, remove handler, raise event) to be identified as well as the method itself. The situation is similar for properties:

Figure 5 - Event Information in Metadata

So, while this information is very useful for the Intellisense prompting of the development environment, and certainly useful for a decompiler, it's completely disregarded by the CLR at runtime. More than one of the commercial obfuscators available remove this redundant information completely - thus permanently losing the relationships between these methods. As the method names will also be subjected to symbol obfuscation, there's no foolproof way of rebuilding these relationships during decompilation. It all muddies the waters further.

One of the more controversial obfuscation techniques coming into play does much more than we've seen so far. I gather that this is not something that everyone will be comfortable with, and I expect it will be implemented as an optional obfuscation technique. This is because it affects the IL code itself and potentially the performance and/or functionality of the application. We'll discuss this next.

Control Flow Obfuscation

As the name suggests, control flow obfuscation involves making it more difficult to follow the flow of control through your code. At the time of writing, it's not so easy to get a good idea of the techniques being employed as a number of the commercial obfuscator vendors are still in the process of developing this feature. What I can do is to describe in general terms some of the known techniques, but bear in mind that this is a developing area.

If you've ever tried to work your way through a bad developer's spaghetti-code, you will appreciate how much confusion can be generated by adding a little spaghetti into a previously well-structured application. The major difficulty here is obviously being able to do this without affecting the logical functioning of the application. The techniques I've identified as possible candidates for use in an obfuscator are as follows:

  • Inlining
  • Outlining
  • Code synthesis

Inlining - is exactly what you expect. This involves replacing a method call with the contents of the called method. This is something quite often performed anyway during JIT compilation. It's quite likely that the bodies of simple get and set accessor methods will be inlined by the JIT compiler to overcome the performance overhead associated with the use of properties. Apart from possibly bloating code by inlining a large method in multiple places, there's no real danger of producing unwanted side-effects by inlining IL code to produce larger, more monolithic methods. This effectively removes structure from the code and hinders comprehension.

Outlining - is the opposite of inlining. This means removing a section of code from a routine into a new method and inserting a method call in its place. This technique is more normally associated with profile-based optimisers, where less-frequently-accessed blocks of code (often error handlers) are outlined from the body of a routine to speed execution through the most likely path. When used in an obfuscatory fashion, outlining creates superfluous structure in decompiled code, so adding to the confusion. Like inlining, outlining is pretty safe functionality-wise, although as an obfuscator would probably not perform a profiling exercise, it's equally likely that outlining could hinder as well as help performance.

Code synthesis - is where code is genuinely changed in the assembly and is, in my opinion, the most difficult to assess in terms of unwanted side effects. It's true from a theoretical view that there exist code constructs that are logically the same yet can be expressed using different expressions. But, as IL code doesn't retain these original higher-level constructs, it's difficult to see how this can be used to confuse a decompiler. Consider the following class:

class Class1
{
private void
            forloop()
{
for (int i = 0; i < 10;
            i++)
Console.WriteLine(i);
}

private void
            whileloop()
{
int i = 0;
while (i <
            10)
{
Console.WriteLine(i);
i++;
}
}

These two methods - forloop() and whileloop() - are functionally identical. If you compile this class with the C# compiler and examine the generated IL using ILDasm, you will see that they produce exactly the same output, byte for byte. See Figure 6 for an example:

Figure 6 - IL Code "for" loop

So how does a decompiler decide whether this should be reverse engineered to a for loop or a while loop?

The answer is that it doesn't. If you decompile the above assembly using Anakrino, both methods will appear as while loops. If you decompile using Remotesoft's Salamander, both will appear as for loops. So what exactly is my point?

My point is that, although IL is a much higher-level representation of the original source code than native x86 assembler, it still appears to be at too low a level for an obfuscator to reliably identify patterns in the code that might be misinterpreted by a decompiler. Don't misunderstand me here - I'm not saying that it can't be done, I'm just having trouble reconciling the need for a provably equivalent code change with a result that is in some way obfuscated. Nobody will want to gamble that their code might work incorrectly following obfuscation, so it has to be provable in theory that this can't happen. Given that this is indeed true, then the obfuscated code must still stand a fair chance of being abstracted correctly by a decompiler. Whether, in a situation like that above, we get a for loop or a while loop is irrelevant - someone can still understand it. If, on the other hand, we end up with something that does fool the decompiler, then the chances are that it will also fool (or mislead) the JIT compiler and we may end up with a less than optimal native code version. It has been suggested to me that JIT compilers are optimised for the IL code patterns generated by code compilers such as C# and VB.NET and this doesn't seem an unreasonable assumption.

I think I will keep a keen eye on developments in this area and would welcome any feedback which might help resolve the issue. In the meantime, let's change tack slightly and look at another widely used obfuscation technique.

String Encryption

You shouldn't be surprised to hear that the metadata holds any string literals used in your code. This is no different in principle to an x86 executable, and it's already quite easy to look for string references with a Win32 disassembler to identify a particular section of code for deeper study. If someone can identify the exact method that displays the string "Invalid license code - please try again", then it probably doesn't matter too much that the name of the method has been changed by an obfuscator.

Most of the obfuscators I looked at offer the ability to encrypt any such literals in the assembly. It's almost ironic that the accessible nature of the .NET assembly, which has been working against us all along, now makes it relatively easy to identify all of the string literals, find the methods that use them, and insert a call to a procedure that will decrypt the string at runtime. We can even add the classes and methods to do this right into the existing file without breaking anything.

Yes, this has a slight performance hit (we're talking compromise remember) but it seems crazy to go to the trouble of obfuscating all of the symbols in your assembly if you intend to leave a bunch of signposts around that make navigating to the correct method a complete doddle. Ok, suggestive clues in string literals won't be available in all cases, but it's another item of information than can be hidden, so why not hide it?

That's all that can be said about the most common features of all serious commercial .NET obfuscators. The final section in this topic will cover a few of the innovations introduced by various vendors to make their product more useful or attractive. If you'd like to obtain further details of any of these obfuscators, follow the links at the end of this case study.

Other Features

Before I start this section, I have to state that I have no financial interest in any of the products mentioned (I should be so lucky). I have attempted to remain objective and give a composite view of the technology on offer as this is the most important factor in deciding what's best for your situation. There are other factors to consider but this is not the place to discuss them.

The following features may be provided by one, more, or all of the listed products and I've deliberately not made mention of who offers what. Some features may still be on the wish-list. While I have my own personal view on the most useful features, I earn my living by developing software and writing technical articles, not by making sales pitches so I'll leave this to your own judgement!

In no particular order, the features described in this section are as follows:

  • Multiple Assembly Obfuscation
  • Incremental Obfuscation
  • Overload Induction
  • Selective Exclusion
  • Extensibility
  • Assembly Size Reduction
  • ILDASM Protection
  • Code Optimisation
  • Data Obfuscation

Multiple Assembly Obfuscation - covers the situation where you might have a number of interdependent assemblies, which together may be considered an application unit. Although each individual assembly may define public types, if these are intended to be used just by other assemblies in the same application, then they may undergo symbol obfuscation within the application unit. This means that not only the metadata definition tables are obfuscated, but also the reference tables.

Incremental Obfuscation - this assists multiple assembly obfuscation where a development team may be working on a number of dependent assemblies independently. Incremental obfuscation allows mapping tables to be maintained containing the pre- and post- obfuscated names of the members in an assembly so that a dependent assembly may be subsequently obfuscated and yet still maintain referential integrity without the need to rebuild and re-obfuscate the complete application. This is particularly effective when distributing updated components.

Overload Induction - this takes method overloading to a new level. This patented algorithm is already used in the Java environment and has now been migrated to .NET. Using this technique, method identifiers are maximally overloaded after an exhaustive scope analysis. This makes it much more difficult for a human reader to determine the true target of any method call. Generally, this technique results in a larger number of methods sharing a smaller number of names.

Selective Exclusion - this is an essential feature. In order to allow assemblies to expose a public interface, allow reflection, or undergo serialization, there needs to be some method whereby individual types and/or members can be excluded from the obfuscation process. Some obfuscators allow specification of XML configuration files to assist this exclusion, others allow custom attributes to be defined in the source code. Many products also automatically recognise circumstances where obfuscation should not be applied and either handle this intelligently or flag a warning.

Extensibility - only one of the obfuscators I looked at offers this facility. This allows the development of plug-ins to the obfuscator, which can define customised rules controlling symbol obfuscation.

Assembly Size Reduction - is an inevitable result of symbol obfuscation. If shorter and fewer strings are used in the string heap then the size of the file will be smaller. Removing other redundant metadata (properties and events) also helps keep the size down. This probably provides more of a benefit for deployment rather than performance, especially if components are delivered over the internet - either as ActiveX components, trickle-fed applications, or as a downloadable installation. Generally, any decrease in size is a good thing.

ILDASM Protection - a feature in some products allows the obfuscator to emit some invalid IL code into the assembly. Although this doesn't affect the existing code, it has the side-effect of crashing ILDasm when used to examine it. This is more likely due to a bit of poor coding in ILDasm than any true innovation on the part of the obfuscator but every little helps. Unfortunately this technique is not so effective with commercial decompilers and can't be used in all types of assembly.

Code Optimisation - you might wonder how your code could be optimised even further - having spent so long writing and testing it and then having it compiled not once but twice, by the C# or VB.NET compiler and then the JIT. Apparently though there is still room for improvement. This is another area where I'm still a little sceptical and it only features in the "coming soon" category of some products. From what I understand, it is possible to identify redundant methods and unreachable code and to restructure data to reduce size and improve performance. Another area to keep an eye on.

Data Obfuscation - this is as yet a non-existent feature, but it has been proposed as a potential technique by at least one of the obfuscator vendors. This technique involves obfuscating not the names within an assembly but the way in which data is represented. For example, converting certain local variables into global variables or changing multi-dimensional arrays into single-dimension arrays. I can see some value in this so it's yet another area to keep a watch over.

Strengths and Weaknesses

We've seen in principle what can be achieved by obfuscation and we already banged the point home earlier that there is no way to prevent reverse engineering at some level. So what are the benefits we can gain by using an obfuscator?

Well, first off, it's easy to do. You don't need to change the way you write software and your developers don't need any additional training. Maybe changing the way you design your systems could maximise the degree of obfuscation possible but in general terms you don't need to think too hard.

Fitting an obfuscator into your development environment is pretty easy. Some products offer just a command-line tool, which can be invoked in a post-build step, while others offer a product that integrates with the Visual Studio .NET IDE and allows you see the effect of the obfuscation while you work. Some provide both. Also, team-based development is assisted by features like incremental obfuscation.

If you've invested a great deal of time and money in developing your software and you want to keep a competitive advantage then you'll probably agree that a commercial obfuscator is very good value for money. Compared to the normal level of expenditure on software development tools, enterprise level obfuscation is still cheap.

There's no effect on deployment. You don't need to distribute any runtime code or libraries as everything happens within the assembly itself. You can distribute it just like any other assembly and it may even be a little smaller.

It definitely works. Try one yourself and you'll see that the results you get back from a decompiler are more difficult to understand. If your application is large or complex beforehand, this additional confusion can thwart many human hackers. Almost certainly a decompiler will not be able to produce valid compilable code following obfuscation. Don't forget though that it's not just humans that need to be confused - decompilers can't be baffled by psychological tricks.

In short, an obfuscator will deter casual hackers and it will buy you time if someone is really determined to reverse engineer your code. How many people are deterred and how much time you can buy are dependent on a couple of weaknesses in this technique. Not weaknesses in the products, but in the principles.

The first problem is that you often can't obfuscate everything. If you expose a public interface or use serialization or remoting, then these parts of your code won't work if they are obfuscated. Also, inheritance constraints may mean that some identifiers - mostly virtual methods - cannot be renamed (for example, you cannot rename an overridden ToString() method).

The second problem is that the IL code is still available in the obfuscated assembly. All that's happened is that it is more difficult to examine this in the correct context. If the part of your assembly you need to protect is a license code validation, for example, this is probably an important issue. All that a hacker needs to do is to find the method(s) that handle this validation and then examine the IL code with ILDasm or a commercial decompiler. They won't need to understand the whole application, they just need to pinpoint a relatively small area of interest and, once they've done so, obfuscation can only help so much.

This last point brings us nicely to a slightly different product, which has a new way of tackling this particular problem.

Other Solutions

One recently-released product that deserves a mention is Remotesoft's .NET Protector. This is not an obfuscator - it doesn't change any of the metadata in the assembly. It is primarily concerned with protecting IL method bodies from decompilation.

As it doesn't tamper with the metadata, it has none of the drawbacks mentioned earlier. It works fine with reflection, serialization and remoting, and the public members of the assembly need no special treatment. Also, it requires no additional security privileges, is compatible with both pure .NET images and mixed images, and performs as well as a pre-JITted assembly. The IL ordinarily contained in the assembly is replaced by what the vendor describes as pseudo-native code, which is neither IL nor native and which is encrypted for good measure. The only apparent limitation is that, at present, support is limited to the Windows platform and the one downside is that the pseudo-native code is typically larger than the original IL.

So far, this sounds pretty good. The Windows platform constraint is bit of a non-problem at the moment as this will probably be the platform of choice for an awful lot of serious .NET development in the foreseeable future. The ability to protect decompilation of the IL while exposing all of the original metadata is the complete converse of the way an obfuscator works and yet it seems perfectly sensible. Where an obfuscator will try to make it difficult for you to find your way around a decompiled assembly so that you hopefully never get to the sections of code you're interested in, the .NET Protector effectively allows you full visibility of the structure of the types in an assembly, confident in the fact that this is much as you'll ever get.

At runtime, the pseudo-native code is decrypted and transformed into true native format and then is executed by the CLR as normal. According to the vendor, the performance is the same as a pre-JITted image although native-related invocations, such as P/Invoke and COM Interop should be faster. In addition to protecting IL code, the .NET Protector also allows string encryption, resource encryption and the option of runtime password protection.

At the time of writing, I don't have any technical details of how the product works. What I do know is that the pseudo-native code is pretty much just native code and that the encryption stage is not crucial to keeping the IL protected. This would suggest that there must be at least a partial compilation of IL during the protection process and either the vendors have created their own IL compiler or they utilise existing facilities available in the .NET runtime. Certainly the claim that performance matches a pre-JITted image supports the latter, but I really have no hard facts, just guesses. The .NET framework includes JIT compilers for all CPU configurations so the limitation of Windows as the development and deployment platform may suggest that these are used at some point in the process. Again, I don't know. I haven't been able to confirm whether or not the JIT compiler is used at all at runtime but, if not, then the decrypted pseudo-native code must be completely valid as far as the CLR is concerned as the protected .NET assembly is not only valid according to the Microsoft .NET specification but is also fully verifiable. It's also not clear whether there is a runtime component to .NET Protector, although I can certainly envisage situations where this would be helpful.

This is an intriguing solution to a difficult problem and I think that this product, if it really does all it claims, would be a more secure method of protecting an assembly from decompilation. I shall certainly be monitoring this area with great interest.

If you're now completely bewildered by the options available for protecting your code, we're going to make it worse by describing yet another technique. This idea is a product of my own devious mind and uses just native C# and the FCL - I've not seen it used elsewhere so this may be of interest in its own right or it may generate ideas for alternative solutions. Here goes ...

The Sample Application

Our goal is to make it as difficult as we can for someone to inspect our assemblies by accessing the metadata. The way they do this may be by using reflection (as we discovered in part one of this case study), by using a commercial decompiler or simply by inspecting the assembly with ILDasm. In all cases, we need to hide this information in some way that makes it less accessible to the more casual hacker.

I'll say again that, in tackling this problem, all we're really doing is reaching a compromise. There's no way we can absolutely prevent someone reverse engineering our assembly if they're really determined to do so. The only way to be 100 percent protected is to never release your compiled assemblies into the public domain (i.e. keep everything server-side) but obviously this isn't realistic for every case. So, given that some bad guys have a copy of your compiled code then, given enough time and expertise, they will be able to figure out exactly what it does and how it does it. This has always been true and always will be. If your code can't be reverse engineered, then it also won't run on a computer.

The compromise is based on the importance we place on keeping our code secure and the amount of time and trouble we are prepared to invest to make this happen. Earlier we discussed obfuscators and alternative techniques such as using unmanaged code for implementing particular parts of a system. This is fine, but it still means that we have to accept the associated overheads. Obviously, if we're using unmanaged code, we can't take such great advantage of the numerous benefits the .NET development environment provides to a developer. We have to retain skills in these areas and accept any consequential costs. There will always be some sort of balance to be made.

What I want to try to achieve is, at least for some cases, a better compromise and I believe this may be possible by the judicious use of encryption.

.NET Cryptography

Within the FCL resides the System.Security.Cryptography namespace. This allows programmatic access to a number of cryptographic services through a relatively simple interface. It is certainly easier to use than the underlying Crypto API, which was first introduced by Microsoft in 1996. Unlike this API, the programming model exposed by the System.Security.Cryptography classes is stream-based. For most cryptographic transformations (encryption, decryption, hashing), the CryptoStream class, which derives from System.IO.Stream, is used. This takes care of all necessary buffering and makes it very easy to stream data from other sources (file streams, memory streams, network, etc.) through an encryptor or decryptor to achieve the desired result. We'll see this in action shortly.

The System.Security.Cryptography namespace supports a number of common algorithms covering three basic cryptographic techniques:

  • Symmetric cryptography
  • Asymmetric cryptography
  • Hashing

Symmetric cryptography is sometimes called secret key encryption. This is where the encryption algorithm requires the same key for both encryption and decryption. The advantage of this is that these algorithms can be made very fast and efficient. However, the primary weakness is the problem of key exchange. The mechanism for ensuring that both the sender and receiver of an encrypted message have the same key and that it remains secure poses a number of logistical problems, particularly where the sender and receiver never physically meet. This was the prime motivation for the development of asymmetric algorithms.

Asymmetric cryptography is also known as public key encryption. In this technique, the key used to decrypt a message (the private key) is different from the key used to encrypt the message (the public key). The public and private keys are mathematically related, but it is computationally infeasible to derive the private key from the public key. In use, the sender of a message would first use the intended recipient's public key to encrypt the content and the recipient would use his matching (but secret) private key to decrypt the message. As the roles of the public and private keys may also be reversed, a variation of this technique is used to produce a digital signature. If a sender encrypts a message using his private key, then any and all recipients should be able to decrypt it using the sender's public key. As the sender is the only person in possession of the private key, he is the only person who could encrypt the message in this way and a successful decryption therefore implies that the message did indeed originate from him. What's more, he can't deny it (this is known as non-repudiation). Thus, an item of data encrypted with a private key is called a digital signature. A common practice is to use a message digest as the item of data to be encrypted - read on.

Hashing doesn't per se involve any encryption of the message. It is a technique whereby the message content is used to produce a fixed-length digital fingerprint, also known as a message digest. This has the property that the same process applied to the same message will produce the same digest, so that if any small change is made to the message content, the message digest will also change. Furthermore, it is highly unlikely that any two different messages would produce the same digest. As described above, when producing a digital signature, the original content of the message is first used to produce a message digest. This is then encrypted using the originator's private key and transmitted along with the message. The message itself need not be encrypted, but the accompanying digital signature, which may be decrypted using the originator's public key and compared with a recomputed message digest, ensures that the message really was from the originator and that it had not been tampered with. Algorithms employed to produce a digest are known as hash functions.

The .NET framework class library contains support for the common symmetric (DES, Triple-DES, RC2, Rijndael), asymmetric (RSA, DSA) and hash algorithms (MD5, SHA1, SHA256, SHA384, SHA512) in the System.Security.Cryptography namespace. In addition, the namespace System.Security.Cryptography.XML contains a full implementation of the W3C standard for digitally signing XML. The majority of the .NET cryptography algorithms are implemented as managed wrappers over the Crypto API although there are a handful that are implemented directly in managed code. The former tend to have class names ending with "CryptoServiceProvider" (e.g. RSACryptoServiceProvider, DESCryptoServiceProvider) while the latter have class names ending with "Managed" (e.g. RijndaelManaged, SHA1Managed).

Outline Design

What I intend to do is really rather simple. The idea is that the public and private aspects of the application you wish to protect are developed either as a number of interdependent assemblies or as separate modules within an assembly. This is quite possibly a natural design choice in many applications anyway and is in no way counterintuitive. This type of design fits well with a number of standard design patterns, for example the Adapter, Bridge, Façade and Proxy structural patterns, which all serve to promote a weak coupling between a subsystem and its clients. We can take advantage of these situations to physically remove the private implementation of a public interface into a separate assembly. This makes it easier to secure.

The use of modules, allowing the development of multi-file assemblies, requires command line compilation. There's currently no support in Visual Studio .NET for this type of development so, for this reason, the example I'll show will use a Windows Forms .exe as the public part of the application and a .NET class library (a DLL containing metadata and IL) as the private part. In fact, we'll use the application we developed in the first part of this case study as our target. This was the reflection-based decompiler, DeComp. We're going to try and protect our decompiler from decompilation!

The original design resulted in a single .NET assembly containing the main classes we developed to undertake the decompilation. These were Generate, a class derived from System.Windows.Forms.Form to which we added a number of private methods used during decompilation, and ILReader the class we used to decompile the IL code for each method body. There were also a small number of additional classes associated with and supporting ILReader. This placed everything in the one assembly and so made everything visible through the same metadata.

What we will do is to remove the private methods we added to Generate into a new class, called Decompiler, and place this new class along with ILReader and its supporting classes into a separate class library (a .NET dll). The modified Generate class, which will still descend from System.Windows.Forms.Form will still handle the UI interface but will invoke an instance of Decompiler to do the dirty work. We'll go through this is more detail later but for now you just need to appreciate the reason for splitting the application this way (apart from producing a better design of course!).

The end effect is that the code we want to protect should be physically contained in its own file (be it a class library or a .NET module), which we can distribute in an encrypted form. We can then take advantage of the way the CLR attempts to find and load these files at runtime to decrypt the assembly on the fly and ensure that the decrypted version only exists in memory while the host application is running. This immediately raises the stakes and requires a lot more work on the part of a would-be hacker to identify, isolate, decrypt and inspect the protected code. Like I said before, this is in no way foolproof (I'll discuss the limitations later) but it does take a lot of players out of the game and doesn't place too much overhead on the part of the developer to implement. We also don't lose any of the benefits of the .NET development environment nor do we need to perform any obfuscation (although we still could).

The CLR really helps us here because it allows us to trap missing modules and/or assemblies programmatically at run time and to write our own code to supply the missing item as the return value of an event handler. This is not something we could do before .NET. If a Win32 executable indicated a dependence on a particular library (dll) and this library could not be found by the operating system when the application was loaded, then we were faced with an unrecoverable error. The only way around this was to use LoadLibrary() and GetProcAddress() to provide dynamic linking at run time. Not only is this more messy to code but it also hides the true dependencies of an executable and vastly complicates version control.

Before we go into the details of reworking the design of our decompiler, we need to develop a utility to allow us to encrypt a compiled assembly. We'll do that now.

Encrypting an Assembly

For the situation outlined above, the use of a symmetric encryption algorithm is the obvious choice. This does have the drawback that, to be really secure, we need to keep the key secret. As we will be decrypting the assembly on an end user's desktop at runtime, there's really no way to achieve this, especially as we'll be performing this part of the operation from a straightforward unencrypted, unobfuscated assembly, which can be freely examined by ILDasm.

But never forget that we aren't aiming for the unattainable 100% protection - we're only making it a little harder. The way to keep this as safe as possible is to be more inventive with the way we get hold of the secret key at runtime. At one end of the spectrum, we might have a string constant nicely visible in the metadata with a name like secretkey, while at the other we might supply the key over a secure internet channel at runtime and make sure we authenticate the user before doing so. We could even supply the encrypted assembly over the internet at runtime and so make it even more inconvenient to decrypt. Whatever happens, it will always be possible for someone with the right knowledge to obtain the key and so decrypt our assembly, so all I'll do here is not make it so obvious.

When the CLR decides that it can't find an assembly, it fires an event and passes in a string argument containing the full name of the missing assembly. This includes the name, version number and culture and more or less uniquely identifies the assembly. For convenience then, and to avoid hardcoding some form of password in the metadata, we'll use this full assembly name as the basis for our encryption key.

We could just use the string itself (or part of it) as a key, but this would be naïve and is not something that would ever be done in a real world situation. What usually happens is that a character-based password is used to derive the actual encryption key, which is of a pseudo-random nature. We can do this using the PasswordDeriveBytes class, which basically takes an incoming character-based password and transforms it into a strong key.

The algorithm is deterministic, so although the output is pseudo-random, it will produce the same result when presented with the same input. This is essential if we ever want to decrypt the data using the original password! There are a number of ways to use this class but the way we'll use it here is to pass in our variable length character-based password in the constructor, then use the GetBytes() method to retrieve a pseudo-random byte array of a fixed length. So, assuming we have the full assembly name in the string fullname, we can get a 192 bit key like this:

PasswordDeriveBytes db = new PasswordDeriveBytes(fullname, null);
byte[] namekey= db.GetBytes(24);

Note that 24 bytes equals 192 bits (24 x 8) - I'm sure you get the idea.

Let's put this into practice. From Visual Studio .NET, create a new Windows Form project called EncryptAssembly. Rename the Form1 class to MainForm, add a MainMenu, an OpenFileDialog, and a SaveFileDialog. Name the dialogs dlgopen and dlgsave respectively. I prefer to build a Windows Forms application just for the convenience of using these dialogs to enter the input and output filenames, but we could just as easily build a console application and accept the filenames as arguments on the command line. Maybe I'm getting too lazy.

So this is just a simple GUI to wrap our encryption method. I've added a label with some descriptive text and also added Open and Exit options to the main menu. The result is shown in Figure 7.

Add a click event handler for the File/Open menu option and present the open and save file dialogs in turn:

private void menuFileOpen_Click(object
            sender, System.EventArgs e)
{
// prompt for file to
            encrypt
if (dlgopen.ShowDialog() == DialogResult.OK)
{
//
            prompt for file to save result
if (dlgsave.ShowDialog() ==
            DialogResult.OK)
{
Encrypt(dlgopen.FileName,
            dlgsave.FileName);
MessageBox.Show("Encryption
            completed");
}
}
}

Figure 7 - Encrypt Assembly UI

Also add a handler for the File/Exit option to close the application:

private void menuFileExit_Click(object sender, System.EventArgs e)
{
// exit application
this.Close();
}

Now we can write the Encrypt() method, which does all of the work. We'll need to import a few namespaces:

using System.IO;
using System.Security;
using System.Security.Cryptography;
using System.Text;
using System.Reflection;

Then start the Encrypt() method as follows:

private void Encrypt(string fromfile, string tofile)
{
// attempt to encrypt given assembly
string fullname;
ICryptoTransform desencrypt;

The ICryptoTransform object will be used to hold an interface to one of the .NET cryptography symmetric encryption classes. In this example we'll use the Triple-DES algorithm, but as the above interface defines the basic operation of any cryptographic transformation, we could plug in a different algorithm pretty easily.

Next we want to load the assembly we're going to encrypt. We do this for two reasons - first, to make sure it really is an assembly, and second, to get the full name, which we need to generate the key:

// check that input file is an
            assembly
try
{
// load assembly and retrieve full
            name
Assembly assembly = Assembly.LoadFrom(fromfile);
fullname
            = assembly.FullName;
}
catch(Exception ex)
{
// failed
            to load assembly
throw new Exception("Unable to load assembly - "
            +
fromfile + "\n" + ex.ToString());
}

We can use PasswordDeriveBytes to get the strong key and also a 64 bit initialisation vector (IV):

// create a 192 bit key and 64 bit iv using the full assembly name
PasswordDeriveBytes db = new PasswordDeriveBytes(fullname, null);
byte[] namekey= db.GetBytes(24);
byte[] nameiv= db.GetBytes(8);

The initialisation vector is a random number used as a starting point in the encryption process. If we used the same key to encrypt the same data, but with a different initialisation vector, we would get two different results. Effectively, it adds a little more randomness into the process.

One of the reasons for using an IV is when encrypting a known file type (e.g. a Microsoft Word document or a PE file) where the first few bytes of the unencrypted file might be in some way predictable. In this situation, knowing what the first few bytes are likely to be gives an attacker a foothold when trying to decipher the data. By using an IV, these bytes are encrypted in a much less predictable way and make it considerably more difficult for the encryption key to be discovered. It's not really relevant to our scenario because it would be much easier for the key to be obtained by reverse engineering the decryption process at runtime than by undertaking a conventional cryptographic attack. I've used it here for good order and it's useful to know for other situations.

The next thing we do is to select which encryption algorithm we want to use. This is very simple using the classes provided by the System.Security.Cryptography namespace, all of which provide "safe" defaults for any uninitialised members. There's no danger that you'll inadvertently produce a weak encryption by using these classes - they will generate a random encryption key and initialisation vector for you if you fail to supply them. However, if you also fail to save these generated values, you may never be able to decrypt your own data!

This is how we create an encryptor for the Triple-DES algorithm:

try
{
// create a tripleDES encryptor
TripleDESCryptoServiceProvider DES3 =
new TripleDESCryptoServiceProvider();
DES3.Key = namekey;
DES3.IV = nameiv;
desencrypt = DES3.CreateEncryptor();
}
catch(Exception ex)
{
throw new Exception("Unable to create encryptor - " +
ex.ToString());
}

I've used a try/catch block mainly because different algorithms accept different sized keys. If we decided to change the algorithm and forget to adjust the key length, the above code will trap and report the error. The valid key sizes for each cryptography algorithm are not terribly well explained in the .NET framework documentation so the best place to go for this information is to examine the LegalKeySizes property of the encryption class itself.

This property is an array of KeySizes, each giving the minimum and maximum key sizes (in bits) together with what is known as a skip size. This is a fixed increment between valid key sizes. For example, the RC2 algorithm implemented by the RC2CryptoServiceProvider class has a minimum key size of 40, a maximum key size of 128, and a skip size of 8. This means that the valid key sizes are 40, 48, 56, 64, . 120, 128 bits.

Getting back to business, we've now got all we need to encrypt the assembly. All we do is to stream it from the input file into a byte array in memory, then stream it back from memory to the output file through the encryptor:

// encrypt fromfile into tofile
try
{
// read input file into byte array
FileStream fsInput = new FileStream(fromfile, FileMode.Open,
FileAccess.Read);
byte[] bytearrayinput = new byte[fsInput.Length - 1];
fsInput.Read(bytearrayinput, 0, bytearrayinput.Length);
fsInput.Close();

// write encrypted version
FileStream fsEncrypted =
new FileStream(tofile, FileMode.Create, FileAccess.Write);
CryptoStream cryptostream =
new CryptoStream(fsEncrypted, desencrypt, CryptoStreamMode.Write);
cryptostream.Write(bytearrayinput, 0, bytearrayinput.Length);
cryptostream.Close();
fsEncrypted.Close();
}
catch(Exception ex)
{
// catch error during file handling
throw new Exception("Error encrypting file - " +
ex.ToString());
}
}

You see how we use a CryptoStream to do this? We first create a normal FileStream giving access to our output file, then we supply this FileStream together with the ICryptoTransform encryptor to the CryptoStream constructor. As far as our code is concerned, this works just like a normal stream so we just write our byte array out to it and then tidy up. Behind the scenes, the CryptoStream uses the algorithm we supplied to encrypt the data and then uses the FileStream we supplied to write the encrypted version out. It's difficult to see how this could be any simpler. You'll see later how the reverse process is just as simple, we can use any of the methods available for streams and just plug in an encryptor or a decryptor and pump our data in or out as we please. The rest gets handled automatically and we hardly need to think about it.

Using an Encrypted Assembly

Now we'll rework the design of the DeComp decompiler according to the earlier outline design. We'll create a new solution for this and call it CSDecomp (for C# Decompiler). The solution will contain a Windows Forms application, also called CSDecomp, and a C# class library called CSDLibrary (see Figure 8).

I don't intend to regurgitate the details of DeComp, which we covered in great detail in part one of this case study. If you don't have access to the download material from part one, and want to build the new version as we go, then you'll need to get hold of it now. Otherwise, the complete source for this version of the decompiler is available with the download material for this case study.

Figure 8 - C# Decompiler

Add a reference to CSDLibrary from CSDecomp and add the following using statements to the latter:

using System.Reflection;
using System.IO;
using System.Security;
using System.Security.Cryptography;
using CSDLibrary;

As before, in CSDecomp, rename the Form1 class to Generate and save the project. Then, in CSDLibrary, rename the default Class1 to Decompiler and add the ILReader classes in the existing Reader.cs source file.

Remember from the outline design that what we need to do is to move anything we want to protect into a separate assembly. Figure 9 shows the situation we had with DeComp, where everything was accessible in one assembly, while Figure 10 shows the picture we're aiming for now:

Figure 9 - DeComp original design

Now, copy the declarations for the following member fields from DeComp into the Decompiler class in CSDLibrary :

private CodeCompileUnit codeunit;
private ILReader reader;

Next, add another private member field that we'll use to hold the decompiled source:

private string decompiledsource;

Then copy in the following private methods from DeComp :

private CodeAttributeDeclarationCollection getCustomAttributes(object[] attrs)
private bool isMethodDefined(MethodInfo meth, Type type)
private MemberAttributes getMemberAttributes(MethodInfo info)
private MemberAttributes getMemberAttributes(ConstructorInfo info)
private MemberAttributes getMemberAttributes(FieldInfo info)
private CodeParameterDeclarationExpressionCollection
getParameterDeclarations(ParameterInfo[] infoarray)
private void addType(Type aType, CodeTypeDeclaration owner)
private void addConstructor(ConstructorInfo info, CodeTypeDeclaration owner)
private void addEvent(EventInfo info, CodeTypeDeclaration owner)
private void addField(FieldInfo info, CodeTypeDeclaration owner)
private void addMethod(MethodInfo info, CodeTypeDeclaration owner)
private string getArgumentList(ParameterInfo[] parms)
private void addProperty(PropertyInfo info, CodeTypeDeclaration owner)
private CodeNamespace findNamespace(string name)
private CodeCommentStatement[] GetILForMethod(MethodBody methodBody)

Figure 10 - CSDecomp amended design

That gets the bulk of the code into the new class library. Add the following imports for the namespaces we're using:

using System.CodeDom;
using System.CodeDom.Compiler;
using System.IO;
using System.Text;
using Microsoft.CSharp;
using System.Reflection;
using System.Reflection.Emit;
using System.Reflection.ILReader;

Then add a read-only property to allow the string of decompiled source code to be retrieved:

public string Source
{
// read-only
            property for decompiled source
get
{
return
            decompiledsource;
}
}

To finish off the Decompiler class, we'll provide a constructor that accepts a fully qualified filename to the assembly which needs to be decompiled. The constructor will then undertake the decompilation at once.

This is perhaps not the way we would normally design a constructor. Ordinarily, we would want the constructor to be as short and fast as possible, limiting its activities to simple member initialisation. So we would probably have defined a parameterless constructor and then provided a second public method to perform the actual decompilation. I've deliberately designed the constructor this way so that the absolute minimum amount of information is held in the referencing assembly.

If we're trying to keep as much information as possible away from the public gaze, we can slightly compromise the ideal design to this end (note the word slightly !). This way, all that is visible from the referencing assembly is the presence of a class called Decompiler, which has a property called Source. Nothing else is visible.

So the constructor is as follows:

public Decompiler(string
            filename)
{
// create compile unit and load assembly for
            reflection
codeunit = new CodeCompileUnit();
Assembly
            mainAssembly = Assembly.LoadFrom(filename);

// load for
            ILReader
reader = new ILReader(mainAssembly.GetModules()[0], new
            AssemblyLoader());

// reflect over all types in the
            assembly
Type[] allTypes =
            mainAssembly.GetTypes();
foreach(Type t in allTypes)
{
//
            add top-level types to codedom compile unit
if (t.MemberType ==
            MemberTypes.TypeInfo)
addType(t, null);
}

// create
            code generator
CSharpCodeProvider cs = new
            CSharpCodeProvider();
ICodeGenerator csGen =
            cs.CreateGenerator();
CodeGeneratorOptions options = new
            CodeGeneratorOptions();
options.BracingStyle = "C";

//
            generate code and save result
StringWriter sw = new
            StringWriter();
csGen.GenerateCodeFromCompileUnit(codeunit, sw,
            options);
decompiledsource = sw.ToString();
}

The above code is basically similar to the code we previously had in the File/Open menu handler in the DeComp project.

To finish off the CSDLibrary assembly, we just need to add the AssemblyLoader class into the CSDLibrary namespace. This class was also explained in part one of the case study so I won't repeat that here:

class AssemblyLoader : IAssemblyLoader
{
public Assembly Load(string assemblyName)
{
return Assembly.Load(assemblyName);
}

public Assembly LoadFrom(string fileName)
{
return Assembly.LoadFrom(fileName);
}
}

Now, we can code the public part of the application in the Windows Forms project CSDecomp. This will have the same UI as the original DeComp, basically a menu, a file open dialog, and a text box to display the result.

By using the Decompiler class from CSDLibrary, the File/Open menu handler is much simpler:

private void menuFileOpen_Click(object
            sender, System.EventArgs e)
{
// open file and decompile
if
            (dlgopen.ShowDialog() == DialogResult.OK)
{
textBox1.Text =
            "";
Decompiler decomp = new
            Decompiler(dlgopen.FileName);
textBox1.Text =
            decomp.Source;
}
}

This is all we need just now to be able to build and test the project, which - take it from me - works just fine. Again, I won't repeat the testing we performed in part one.

This is where we can add the code needed to locate and decrypt the encrypted version of CSDLibrary.dll that we'll distribute with the finished application. Apart from the additional code that I'm going to show you in a moment, we don't need to change the way we develop the application at all. We can continue to make changes both to CSDecomp and CSDLibrary and the only additional step needed will be to use EncryptAssembly on the release version of CSDLibrary.dll before we distribute the application.

So at this point, run EncryptAssembly, select CSDLibrary.dll as the input file, and save the encrypted version in a file named CSDLibrary.enc. You can then delete the original dll to make sure we're really decrypting the hidden version at runtime. Before this, we need to make some more changes.

The first change we'll make is to add an event handler to catch the situation where the CLR cannot find the referenced assembly. We want to do this before we attempt to reference any of the types in the missing assembly, so a good place to do this is in the Main() method:

[STAThread]
static void Main()
{
// trap unresolved assembly references
AppDomain appdom
            = AppDomain.CurrentDomain;
appdom.AssemblyResolve += new
            ResolveEventHandler(MyResolveHandler);

// show main
            form
Application.Run(new Generate());
}

If, as shown, we're trapping a missing assembly, we register a handler for the AssemblyResolve event in the current AppDomain (as shown). For a multi-file assembly, we could handle an encrypted module in a similar manner by trapping the ModuleResolve event of the assembly.

The event handler itself is another static method, which starts as follows:

static Assembly MyResolveHandler(Object sender, ResolveEventArgs e)
{
// handle missing assembly
string path = Path.GetDirectoryName(Application.ExecutablePath);
string filename = path + @"\CSDLibrary.enc";

Here, we build the fully qualified filename for the encrypted version of the missing assembly. It's worth noting that, as we get the full name of the missing assembly in the ResolveEventArgs parameter, we could have used this to derive the filename of the encrypted version rather than hardcoding the name as above. Indeed, if we had multiple encrypted assemblies, this would be an ideal way to do it as the same routine could handle any missing assembly without change.

I've kept things as simple as possible for our single encrypted assembly but there's plenty of scope for including additional complications at this point if you wish. One option might be to hold encrypted assemblies as embedded resources rather than as separate physical files. This would introduce one more hurdle to be overcome for someone trying to access the encrypted code.

Now that we've got the path to the encrypted file and the full name of the missing assembly, we can reconstruct the encryption key and initialisation vector that we used during encryption:

// recreate 192 bit key and 64 bit iv using the full assembly name
PasswordDeriveBytes db = new PasswordDeriveBytes(e.Name, null);
byte[] namekey= db.GetBytes(24);
byte[] nameiv= db.GetBytes(8);

In a similar manner to the encryption process, we create a Triple-DES decryptor using these values:

// create a
            decryptor
TripleDESCryptoServiceProvider DES3 = new
            TripleDESCryptoServiceProvider();
DES3.Key = namekey;
DES3.IV
            = nameiv;
ICryptoTransform desdecrypt =
            DES3.CreateDecryptor();

Decrypting the assembly is then a simple matter of creating a CryptoStream and reading the decrypted data from the file straight into memory in a byte array:

// decrypt file into a byte array
FileStream fsInput =
new FileStream(filename, FileMode.Open, FileAccess.Read);
CryptoStream cryptostream =
new CryptoStream(fsInput, desdecrypt, CryptoStreamMode.Read);
byte[] bytearrayinput = new byte[fsInput.Length - 1];
cryptostream.Read(bytearrayinput, 0, bytearrayinput.Length);
cryptostream.Close();

Like we did during encryption, we define an underlying FileStream to get the source data, combine this with our decryption algorithm to get a CryptoStream, then just read from the CryptoStream to retrieve and decrypt the data in one. So we end up with a decrypted version of CSDLibrary.dll in a memory array.

The static Load() method of the Assembly class conveniently has an overload that accepts a byte array as input, so we can recreate the missing library as an Assembly object and return it as the result of the event handler:

// load assembly from byte array
Assembly assembly = Assembly.Load(bytearrayinput);
return assembly;
}

And that's it. The CLR now has the assembly in memory, just as if it had loaded it itself, the additional working memory we needed for decryption will be reclaimed by the garbage collector, and all we've added by way of an overhead is the one-off performance hit while we performed the actual decryption:

Figure 11 - ILDasm references to hidden assembly

The private assembly never exists in a decrypted format except in memory and so the best anyone could do with a decompiler is to retrieve details of the public part of the application referenced in CSDecomp.exe. This doesn't give much away at all, as the output from ILDasm shows in Figure 11.

To get any more than this, it would be necessary to disassemble the MyResolveHandler() method and write a similar method in a new application to perform the decryption using this same technique. This is still well within the capability of a determined hacker but takes more than a quick browse with a decompiler or ILDasm and is quite easy to complicate further.

I already mentioned the possibility of alternative methods of key exchange - maybe the key could be supplied at runtime by user input, internet access, etc. Perhaps the decrypted assembly could also be made less accessible - embedded as a resource in the executable, or again supplied at runtime over an internet connection. There are a number of possibilities.

We could even take the trouble to use a commercial obfuscator in addition to the above and so place even more obstacles in the way. This is the same sort of game that's already played between software developers and hackers (for the want of a better word - I mean anyone who is intent on reverse engineering your code for whatever reason).

Application Setup

If you want to try this out for yourself, download the support material for this case study from the C# Today web site and follow the instructions in the ReadMe.txt file included. The download contains the CSDecomp project and the CSDLibrary class library described above as well as the EncryptAssembly source code.

Limitations

We discussed the limitations of obfuscation and other techniques earlier. This section covers the limitations of using an encrypted assembly as described above.

The main problem is that the code that locates and decrypts the "hidden" assembly is held in a conventional .NET executable and so is easy to locate and decompile. It would be possible to use an obfuscator to make this more difficult, but it would probably be better to implement this method in unmanaged code. The ability to decompile the decryption routine in the above example means that it becomes clear exactly where the encrypted assembly is located and what key was used to encrypt it.

A stronger method would be to ensure that the key could not be determined just by examining the static code. If the key was instead received via a network or internet connection, or even provided by a call into unmanaged code, then this would necessitate some dynamic tracing of the process at runtime in order for the key to be found. While still possible, this would require a higher level of expertise and possibly some additional software.

Case Study Review

In part one of this case study, we showed how vulnerable an unprotected .NET assembly is to reverse engineering and decompilation. The intention of this part of the case study was to examine a range of options that might be used to mitigate the effects of this exposure. We looked at the use of unmanaged code, native code generation and commercial obfuscators, then went on to develop an alternative technique using encryption.

Somewhere in this range of options - either singly or in combination - there is probably a solution for anyone concerned with protecting their investment in intellectual property. We already know (from the number of times I've repeated it!) that it is not possible to absolutely prevent reverse engineering as long as you distribute executable code. But the amount of information contained in the metadata of a .NET assembly, together with the high-level nature of IL code, has caused a lot of concern and the absence of any inbuilt method in the CLR to encrypt or otherwise hide the details of a method's implementation has been viewed as a major shortcoming of .NET.

Personally, I'm not yet sure of the real importance of this problem. I'm convinced that a lot of the adverse reaction has been a knee-jerk response to the sudden change brought about by these accessible, self-describing components and, in many cases, there's probably little real danger of someone losing out financially due to reverse engineering attacks. In the Win32 world we're just not used to our source code being publicly exposed and never have been. It's perhaps a bit of a shock to have the curtains pulled away to reveal your every coding secret in its naked glory. This is quite a contrast to traditionally open environments like the various flavours of Unix, yet these are still commercially successful.

Looking at the reality, there are not many categories of application that are genuinely sensitive to this exposure. Taking a different tack, we can look around at any number of open source developments that mirror popular commercial products and wonder how much of this freely available code implements techniques that are considered trade secrets by the originator. Does this exposure have an effect? Are there just normal market forces coming into play, where commercial advantage is really achieved by bringing out new features rather than copying someone else's old ones? Why would anyone really take the trouble to reverse engineer your product?

Well, when writing this case study, I really wanted to get inside some of the commercial obfuscators. I wanted to know in detail how they worked, how they might be beaten by a decompiler. But did I launch into a major reverse engineering exercise? No - I just sent e-mails to the companies concerned and asked them to tell me how their products work. I looked at their web sites, downloaded FAQ's and whitepapers. I didn't need to look at their source code - they were all happy enough to explain it to me!

This is perhaps a valid point. Consider encryption, where one might think it's crucial to keep the details of encryption and key generation algorithms out of the clutches of the bad guys. If you do think that, then you're 100% wrong. The very best cryptographic techniques are completely open and subject to scrutiny from experts worldwide - they have to be if they are to be recognised for what they are. If someone offers you "new" encryption software with a "secret" algorithm, avoid it. For these products, using a well-known and trusted algorithm, providing technical documentation explaining their implementation, and making the source code available are all signs that a vendor is confident about its product's security.

Ok, so I'm tending to talk about the ends of the spectrum and most people will fall somewhere in the middle. Yes, there are portions of code that need to be kept private, and this is not because they use weak algorithms or are poorly designed. You should now have a much better idea of the options available for keeping private things private, but my last and most important point is this - before you spend a lot of time and effort preventing someone reverse engineering your systems, take the time to really consider exactly how much harm they can do anyway.

It might save you a few headaches.

Sleep well - and don't have nightmares ..

Case Study Information
Author | Roger Rowland
Editor | Adam Ryland
Reviewers | Benny Mathew, Sean Schade, Dinar Dalvi

If you have any questions or comments about this case study, or suggestions for further pieces, please contact the technical editor. If you wish to talk about the case study with other subscribers, please visit our forum here.


Protecting Your Code – Reflection, Obfuscation and Encryption – Part 1

Protecting Your Code – Reflection, Obfuscation and Encryption – Part 1
http://www.csharptoday.com/content.asp?id=1852
Article Rating: 5
Published on August 5, 2002
Abstract
The System.Reflection namespace provides the means whereby the public and private members of a type’s implementation may be accessed from the metadata held within every .NET assembly. There is no way to prevent this, and along with the accessible nature of intermediate language (IL) code, this has been the cause of much concern when considering the protection of intellectual property. In the first of two parts, this case study by Roger Rowland examines just how far reflection alone can be used to reverse engineer a compiled .NET assembly into C# source code. In the second part of this series, we will examine exactly what can be done to protect sensitive code from prying eyes.
Case Study

The System.Reflection namespace provides the means whereby the public and private members of a type’s implementation may be accessed from the metadata held within every .NET assembly. There is no way to prevent this and, along with the accessible nature of intermediate language (IL) code, this has been the cause of much concern when considering the protection of intellectual property. In the first of two parts, this case study examines just how far reflection alone can be used to reverse engineer a compiled .NET assembly into C# source code. In the second part of this series, we will examine exactly what can be done to protect sensitive code from prying eyes.

Introduction

Reflection is a major aspect of .NET. It allows dynamic discovery of the types contained in an assembly along with the fields, properties and methods defined therein. It underlies many of the advantages of the .NET framework– late binding to objects in dynamically loaded assemblies, type browsers and Intellisense prompting during development, serialization and hence – indirectly – remoting. It is reflection that facilitates component reuse in a much more integrated fashion than COM ever did. Certainly the reflection capability built into .NET brings many advantages.

The downside of this is that the facilities available through the System.Reflection namespace provide a much easier way than ever before for other people to examine the structure of your applications by querying the metadata that is contained in the manifest held within every .NET assembly. It’s not just the public types and members which could come under such scrutiny – private, internal and protected fields, properties and methods can all be examined through reflection, and the Common Language Runtime (CLR) provides no mechanism for prohibiting this.

If you are familiar with ILDasm.exe – the Microsoft Intermediate Language (IL) disassembler – you will already be aware how much information is available. You may not be aware, however, exactly how easy it can be to render this information into a format much more digestible to a developer.

In this case study we’ll develop a simple decompiler, which uses Reflection and CodeDOM together to allow a .NET assembly to be reverse engineered into C# source code. As a final twist, we’ll fill in the decompiled method bodies with disassembled IL code as comment lines. This is about all we need to do to produce a C# source file that, if not directly compilable, would be easily comprehensible to any moderately experienced developer. All of this using standard .NET tools, pure C# code, and starting with nothing more than a .NET assembly binary. Just like the ones you distribute with your applications.

Worried? You should be.

.NET Decompilers

There are already a number of .NET decompilers on the market (for example, the Salamander decompiler at http://www.remotesoft.com/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq). Often, these are marketed alongside obfuscators, which make the task of understanding decompiled code much harder. We’ll take a closer look at obfuscators in part two of this case study, as there’s a lot to discuss. These commercial decompilers typically operate directly at the IL level, which makes them conceptually different from the approach we’ll be taking but, while the results can be that much more impressive, you’ll see enough to realise that it doesn’t need a great deal of time, effort, expertise or expense for a hacker to get inside your code.

Background Reading

Previously on C# Today, the use of reflection for reverse engineering has been touched on in C# to UML Reverse Engineering. Also, using CodeDOM to generate source code has been covered in Building a Forms Generator Using Code DOM and Generating Design Patterns Using CodeDOM. If you haven’t read these articles then it may be an idea to do so now as this will help your understanding of what’s to follow.

System Requirements

This system requires Visual Studio .NET V1.0 or later running on Windows 2000 or Windows XP. The case study assumes that the reader has a good understanding of C# and some familiarity with CodeDOM.

Defining The Problem

The problem here is not so much one of design as one of discovery. The System.Reflection namespace contains many classes that allow the types contained in an existing assembly to be inspected. The System.CodeDOM namespace likewise contains many classes that allow types to be defined in sufficient detail to automatically generate C# or VB.NET source code describing these types.

The problem here is that neither of these namespaces is particularly well documented.

The System.Reflection namespace (and the more specialised System.Reflection.Emit namespace) contain some types and some type members specifically for use by compiler developers, which would not normally be used by application developers. The framework documentation doesn’t flag any distinctions between these types and this makes it somewhat confusing in places. The documentation for System.CodeDOM is also difficult to digest and can be confusing to an application developer due to the different nomenclature used. Where a developer might think of protected or internal modifiers for a type member, the CodeDOM documentation uses the CLR attributes Family and Assembly respectively. Believe me, it’s not hard to get lost in the mass of CodeDOM classes, properties, methods, collections and enumerations.

The following table shows some of the differences in nomenclature between CLR, C# and VB.NET terms:

CLR Term | C# Term | VB.NET Term
Private | Private | Private
Family | Protected | Protected
FamilyAndAssembly | (not supported) | (not supported)
Assembly | Internal | Friend
FamilyOrAssembly | Protected internal | Protected friend
Public | Public | Public
Abstract | Abstract | MustInherit / MustOverride
Final | Sealed | NotInheritable / NotOverridable
Static | Static | Shared
InitOnly | Readonly | ReadOnly
Virtual | Virtual | Overridable
NewSlot | New | Shadows
ReuseSlot | override | Overrides

Case Study Structure

The sample application we’ll develop will have a simple user interface, just enough to allow us to select an assembly to decompile and to display the resulting source code. I’ll briefly mention the UI design and then go into the detail where we get to grips with the Reflection and CodeDOM classes. Then we’ll dive a little deeper into the metadata to see how we can extract the IL code for each method body, after which we’ll put our decompiler through its paces before discussing what we’ve learnt and looking ahead to see what can be done about it!

  • The Sample Application
  • UI Design
  • Coding
  • Loading an Assembly
  • Types and Nested Types
  • Events
  • Fields
  • Methods
  • Constructors
  • Properties
  • Parameter Declarations
  • Custom Attributes
  • Methods – Body Snatching!
  • Testing the Decompiler
  • Application Setup
  • Limitations
  • Case Study Review

The Sample Application

The sample application will be a Windows Forms application named DeComp. From Visual Studio .NET, create a new C# Windows Forms project and rename the Form1 class to Generate.

UI Design

Using the Forms Designer, resize the Generate form to 584 x 413, add a MainMenu and an OpenFileDialog, naming the latter dlgopen. Next add a TextBox and set the Dock property to Full, the Font to Courier New 9pt, set MultiLine and ReadOnly to True, WordWrap to False and ScrollBars to Both. The resulting design should look something like Figure 1 below.

Figure 1 - UI Design

On the MainMenu, add a File menu item containing the options Open and Exit. Add Click event handlers for these two items and complete the click handler for the exit option as follows:

private void menuFileExit_Click(object sender, System.EventArgs e)
{
// exit
this.Close();
}

None of this should have caused you any problems, so we’ll get straight to the details now and start coding the decompiler.

Coding

The first thing we’ll do is to import some of the namespaces we’ll be using,

using System.CodeDom;
using System.CodeDom.Compiler;
using System.IO;
using System.Text;
using Microsoft.CSharp;
using System.Reflection;
using System.Reflection.Emit;

and we’ll also declare a private field to hold the CodeDOM code unit to contain all of our generated declarations:

private CodeCompileUnit codeunit;

The following sections describe the remainder of the coding, which is not challenging, but there is a lot of it. In general, we’ll cover the following sequence of steps, most of which involve converting information obtained from reflection into equivalent CodeDOM declarations.

  • Loading an assembly and examining the types defined therein.
  • Declaring types (classes, interfaces, structs and enums).
  • Declaring events.
  • Member fields (private, public, static, const, initialised)
  • Method declarations.
  • Constructors (private, public, instance, static)
  • Properties and type indexers
  • Parameter lists
  • Custom attributes
  • Decompiling IL method bodies

We’ll start with the way we load an assembly from disk and examine the types defined within.

Loading an Assembly

The main processing will be initiated when the user selects File/Open from the menu. So, in the Click event handler for the Open option on the menu, display the OpenFileDialog to allow the user to select a .NET assembly (either an .exe or a .dll) for decompilation.

private void menuFileOpen_Click(object sender, System.EventArgs e)
{
// open an existing .net assembly
if (dlgOpen.ShowDialog() == DialogResult.OK)
{

Having acquired a filename, we can load the assembly by using the static method Assembly.LoadFrom(). This makes sure we load the assembly from the file selected by the user. If we used the Load() method, then the same algorithm used by the CLR to implicitly load an assembly would be used. This means, that the assembly may instead be loaded from the global assembly cache (GAC) or from the application path of the decompiler, and so we may not get exactly what we’re aiming for.

We can reinitialise the CodeCompileUnit at the same time we load a new assembly, so the menuFileOpen_Click() method continues as follows:

// create compile unit and load assembly for reflection
codeunit = new CodeCompileUnit();
Assembly mainAssembly = Assembly.LoadFrom(dlgOpen.FileName);

Having loaded the assembly, we reflect over all the types contained therein and add these to our CodeDOM definitions.

// reflect over all types in the
            assembly
Type[] allTypes =
            mainAssembly.GetTypes();
foreach(Type t in allTypes)
{
//
            add top-level types to codedom compile unit
if (t.MemberType ==
            MemberTypes.TypeInfo)
addType (t, null);
}

You’ll see that the bulk of the processing is carried out by the addType() method, which we’ll cover in a moment. For now, we can complete the above method by assuming we’ve reflected over the complete assembly and added all of our CodeDOM definitions to the codeunit.

All that remains then is to generate the decompiled C# source code and display it in the TextBox on the main form:

      //
            create code generator
CSharpCodeProvider cs = new
            CSharpCodeProvider();
ICodeGenerator csGen =
            cs.CreateGenerator();
CodeGeneratorOptions options = new
            CodeGeneratorOptions();
options.BracingStyle = "C";

//
            generate code and show result
StringWriter sw = new
            StringWriter();
csGen.GenerateCodeFromCompileUnit(codeunit, sw,
            options);
textBox1.Text = sw.ToString();
}
}

This is pretty straightforward, although we really need some exception handling in here as it’s quite possible for the code generator to choke on an invalid declaration if we’ve not been careful. The above code has been amended for clarity - the download material for this article contains a more robust version.

Types and Nested Types

Now for the addType() method. The purpose of this method is to add a CodeDOM type declaration for a type we’ve retrieved from the Assembly using reflection.

As it’s possible for types to be nested, this method may be called recursively to add a type declaration to an existing CodeDOM type. For this reason, the method takes an optional argument specifying the owning CodeDOM type declaration. If this is null (as in the above code), then the new type is added directly to the CodeDOM namespace. Like this:

private void addType(Type aType,
            CodeTypeDeclaration owner)
{
// build a new CodeDOM type
            declaration
CodeTypeDeclaration newtype = new
            CodeTypeDeclaration(aType.Name);
if (owner == null)
{
// if
            owner is null, add type definition to its namespace
CodeNamespace
            ns =
            findNamespace(aType.Namespace);
ns.Types.Add(newtype);
}
else
{
//
            otherwise, add a nested
            type
owner.Members.Add(newtype);
}

Namespaces actually mean absolutely nothing to the CLR, they exist purely as an aid to the developer to organise his/her code more effectively and to allow the compiler to resolve type names coded in a shorthand syntax.

However, it’s very convenient that the namespace is held in the metadata describing each type so we can create a CodeDOM namespace with the correct name when we decompile the assembly. This brings added clarity to our decompiled code and so makes it easier for us to interpret the developer’s original design.

As every type we come across in the assembly will have a Namespace property, we need to make sure that we don’t go creating a new CodeDOM namespace declaration for each type we find that was defined in it. We need a helper method to check if we’ve already created that namespace and to pass back a reference if so.

That’s the purpose of the findNamespace() method invoked above. It’s defined as follows:

private CodeNamespace findNamespace (string
            name)
{
// return CodeDOM namespace with given name
int
            index = -1;

// find namespace if already declared
for (int
            i = 0; i < codeunit.Namespaces.Count; i++)
{
// check by
            name
if (codeunit.Namespaces[i].Name == name)
{
// set
            index and exit loop
index = i;
break;
}
}

// add
            namespace to unit or use existing
if (index == -1)
{
index
            = codeunit.Namespaces.Add(new CodeNamespace(name));
}

//
            return namespace
return
            codeunit.Namespaces[index];
}

This method looks through all of the namespaces already defined within the CodeCompileUnit, attempting to find a match by name. If a match is not found, the namespace is added and, in either case, a reference to the correct CodeDOM namespace is returned.

Now, let's return to the addType() method. By now we’ve created a CodeDOM type declaration for the type, we’ve added it to the correct CodeDOM namespace, and the namespace has been added to the CodeDOM CodeCompileUnit referenced by the private member field codeunit. Everything is in place for us to generate source code once we’ve finished defining the type.

We can continue to fill in the details of our new type by adding some basic information available from the metadata:

// fill in basic details of this type
newtype.TypeAttributes = aType.Attributes;
newtype.IsClass = aType.IsClass;
newtype.IsEnum = aType.IsEnum;
newtype.IsInterface = aType.IsInterface;

These properties identify whether our type is a class, enumeration, interface, etc. and also determine the accessibility (public, private, protected, etc.). The latter is achieved by setting the CodeDOM TypeAttributes directly from the Attributes obtained by reflection. According to the documentation, this is the only compatible mapping – you’ll see later that we need to do a little more work to map other attributes between CodeDOM and reflection enumerations.

Now we’ve reached this point, we need to be a little more careful and maybe need a deeper understanding of how the CLR view differs from the familiar C# perspective. First off, let’s look a little more closely at how an enum is defined in an assembly.

Enumerated Types

If you create an enum in a C# project, then examine the compiled assembly using ILDasm, you’ll see what I mean. The following C# declaration,

public enum NormalByteEnum : byte
{
ByteEnum1 = 1,
ByteEnum2 = 2
}

looks like this from ILDasm (see Figure 2) –

Figure 2 - Enum in ILDasm

You can see that an enum is really defined as a class descending from System.Enum and containing static fields for each entry in the enumeration (initialised to their respective values). Furthermore, the actual type of the enumeration (we defined it as a byte in our example above) is recorded against a field named value__, which has the attributes SpecialName and RTSpecialName.

We need this information to define the enum correctly using CodeDOM because we only have one class – CodeTypeDeclaration – to handle classes, enumerations, structs and interfaces. In this case, the important thing to do is to define the correct base class – not System.Enum as you might expect, the enumeration is indicated instead by the IsEnum property, by the type of the value__ field.

We can do this like so:

   // handle enum and
            struct
if (newtype.IsEnum)
{
// for enums, get type of
            field
FieldInfo finf =
            aType.GetField("value__");
newtype.BaseTypes.Add(finf.FieldType);
}

Now we can continue and set the base type for any non-enum types:

   else
{
// flag
            struct if required
newtype.IsStruct =
            aType.IsValueType;

// only add a non-default base type
if
            (aType.BaseType != null)
{
if
            (!aType.BaseType.Equals(typeof(object))
            &&
!aType.BaseType.Equals(typeof(System.ValueType)))
{
newtype.BaseTypes.Add(aType.BaseType);
}
}
}

Notice we also set a flag here to differentiate classes and structs and we only bother to add a base type if its not one of the defaults (System.Object for a class or System.ValueType for a struct). This is not strictly necessary but it does make our generated code look more like the original if we leave these as implied derivations.

Remember that in C#, a class can only have at most one base class, so while that takes care of enums and structs, we still need to handle interfaces. If we’re dealing with a class or an interface then this could also inherit from one or more other interfaces, and we need to know this to produce the correct CodeDOM declaration. We can find out what interfaces a type inherits from via the Type.GetInterfaces() method, but in doing so there’s another small hurdle to overcome.

The problem is that the GetInterfaces() method will return all interfaces inherited by the type we’re examining and any inherited from its base class, and so on right up the class hierarchy! Whereas we only want to know about any interfaces for which our type actually implements methods, that is, just the interfaces it directly inherits.

We can find this out in a roundabout way by using another method defined on the Type class, GetInterfaceMap(). The interface map for a type contains an array of target methods for a particular interface, which represent the actual methods that implement the interface at this point in the class hierarchy. By checking all of the target methods for all of the interfaces inherited by the type we’re examining, we can find out if any of those methods are declared in that type, and so work out which interfaces are directly inherited.

If you followed that, you’ll follow this next piece of code. If you didn’t follow it, then perhaps looking at the code may make it easier. What we do here is first make sure we’re dealing with a class or an interface – we’re not interested in anything else. Then we get an array of the interfaces inherited by that type. Next, for each of these interfaces, we get an array of the actual methods that implement that interface. After this, for each method, we check the DeclaringType to see if it matches the type we’re examining. If any method does, then we add the interface to the collection of CodeDOM base types for the type declaration before moving on and checking the next interface. And, yes, I wish it were more simple too, but it’s not. Here’s the code:

   // add implemented
            interfaces
if (newtype.IsClass || newtype.IsInterface)
{
//
            add any inherited interfaces
Type[] ifaces =
            aType.GetInterfaces();
foreach(Type iface in ifaces)
{
//
            add interface if this type implements it
            directly
InterfaceMapping imap =
            aType.GetInterfaceMap(iface);
MethodInfo[] meths =
            imap.TargetMethods;
foreach(MethodInfo meth in meths)
{
if
            (meth.DeclaringType.Equals(aType))
{
newtype.BaseTypes.Add(iface);
break;
}
}
}
}

The code actually takes up much less space than the preceding explanation and is probably easier to understand!

Almost the last thing we do to complete our declaration of the reflected type is to copy over any custom attributes. Now almost anything can have custom attributes, so I’ve employed another helper method to do this and I’ll detail this later. For now, we can just add a couple more lines to the addType() method.

// add custom attributes
object[] attrs = aType.GetCustomAttributes(false);
newtype.CustomAttributes.AddRange(getCustomAttributes(attrs));

So we’ve now defined the basic details for the reflected type in CodeDOM terms. We’ve got the name, accessibility and any custom attributes, and we’ve made sure we’ve got the correct base classes and that we’ve taken care of anything particular to classes, structs, enums and interfaces. Now we can dive inside this type and look to see what members it contains.

Still using the Type class to get these details from the assembly, we can retrieve an array of all members declared within this type.

// get members for this type
MemberInfo[] allMembers = aType.GetMembers(BindingFlags.DeclaredOnly |
BindingFlags.NonPublic | BindingFlags.Public | BindingFlags.Instance |
BindingFlags.Static);

Note the use of the BindingFlags argument. This allows us to specify exactly what sort of members we’re looking for. The above combination will retrieve all public and private, static and instance members. Using BindingFlags.DeclaredOnly allows us to filter out any inherited members so that we only get details of members actually declared by the type rather than those inherited. It needs to be said that if there were some similar mechanism for retrieving interfaces this would have saved us a lot of trouble earlier!

We can then process the array of MemberInfo objects and invoke helper methods to process each one according to their member type. For most of these, we recast the MemberInfo to something more specific to allow us access to the details we need. Note here, that if we come across a nested type, we call addType() recursively to include a complete type declaration within the current CodeDOM type declaration. The following switch statement identifies the member type and passes control to the relevant helper method:

// process members according to member type
foreach(MemberInfo mem in allMembers)
{
switch (mem.MemberType)
{
case MemberTypes.NestedType:
case MemberTypes.TypeInfo:
addType((Type)mem, newtype);
break;
case MemberTypes.Event:
addEvent((EventInfo)mem, newtype);
break;
case MemberTypes.Field:
addField((FieldInfo)mem, newtype);
break;
case MemberTypes.Method:
addMethod((MethodInfo)mem, newtype);
break;
case MemberTypes.Constructor:
addConstructor((ConstructorInfo)mem, newtype);
break;
case MemberTypes.Property:
addProperty((PropertyInfo)mem, newtype);
break;
default:
MessageBox.Show("Unknown type - " +
mem.MemberType.ToString());
break;
}
}
}

That completes the addType() method. Now we’ll examine each of the helper methods in turn.

Events

Events are probably the most straightforward class members to decompile. Just to recap, the addEvent() method is invoked with two parameters – the first is an EventInfo object from the System.Reflection namespace, which contains details of an event in the type we’re currently decompiling. The second parameter is a reference to the CodeDOM type declaration into which we need to add a matching declaration for this event.

The CodeDOM class that declares an event is CodeMemberEvent. Apart from adding any custom attributes (using the same helper we invoked earlier), about all we need to find out is the name of the event and the type of the delegate needed to trap the event. Both of these can be found directly from the EventInfo object in the Name and EventHandlerType properties respectively, so the addEvent() method is really simple:

private void addEvent(EventInfo info,
            CodeTypeDeclaration owner)
{
// add event to given
            type
CodeMemberEvent newevent = new
            CodeMemberEvent();
newevent.Name = info.Name;
newevent.Type =
            new CodeTypeReference(info.EventHandlerType);

object[]
            cusattr =
            info.GetCustomAttributes(false);
newevent.CustomAttributes.AddRange(getCustomAttributes(cusattr));

//
            add event to owner
owner.Members.Add(newevent);
}

We’ve not heard the last of events though, so you may have a few surprises later!

Fields

Fields are not as simple as events, because we have to take special care with enumerations and indeed any field defined with an initial value. By this I really mean constants. Normal instance fields which are declared in C# with an initial value are not described in this way in the metadata. Instead, the compiler inserts the relevant initialisation code into the body of the constructor so we cannot regenerate the exact same syntax when we decompile. However, we don’t end up losing this information, as we’ll see later.

We also need to make sure we don’t output a field declaration for the special value__ field we discovered hiding in an enumeration. Fortunately, we can find this (and any as yet undiscovered special fields) by examining the FieldInfo.Attributes property to see if the SpecialName flag is set.

The addField() method starts like this:

private void addField(FieldInfo info, CodeTypeDeclaration owner)
{
// add field details to given type

// don't add special fields
if ((info.Attributes & FieldAttributes.SpecialName) != 0)
return;

Then we create a new CodeDOM definition for the field and set the basic attributes:

// create CodeDOM field definition
CodeMemberField newField = new CodeMemberField(info.FieldType, info.Name);
newField.Attributes = getMemberAttributes(info);

I’ve created an overloaded method called getMemberAttributes(), which I’ll use to map the basic attributes between reflected classes (EventInfo, FieldInfo, PropertyInfo, etc.) and the corresponding CodeDOM classes (CodeMemberEvent, CodeMemberField, CodeMemberProperty, etc.). Unlike the simple mapping we had between Attributes and TypeAttributes at the class level, these enumerations – while basically conveying the same information – do not necessarily have the same internal values. We have to test each attribute separately and decide whether or not to set the corresponding bit flag in the equivalent CodeDOM enumeration.

This is the getMemberAttributes() method that handles fields:

private MemberAttributes
            getMemberAttributes(FieldInfo info)
{
// convert Reflection
            FieldInfo attributes to CodeDOM
MemberAttributes attrDom =
            0;

// map attributes between types
if (info.IsAssembly)
            attrDom |= MemberAttributes.Assembly;
if (info.IsFamily) attrDom
            |= MemberAttributes.Family;
if (info.IsFamilyOrAssembly) attrDom
            |= MemberAttributes.FamilyOrAssembly;
if (info.IsPrivate) attrDom
            |= MemberAttributes.Private;
if (info.IsPublic) attrDom |=
            MemberAttributes.Public;
if (info.IsLiteral)
attrDom |=
            MemberAttributes.Const;
else if (info.IsStatic)
attrDom |=
            MemberAttributes.Static;

// return CodeDOM
            enumeration
return attrDom;
}

This is something to be careful with, and at first it gave me all sorts of problems. You’ll note that I have coded this method so that a constant field will never be also marked as static (even though it is in the metadata – take a look at one with ILDasm if you don’t believe me). The reason for this is that, in CodeDOM terms, declaring a static const field doesn’t make any more sense than it does to the C# compiler. Whereas the C# compiler will throw out an error, CodeDOM just refuses to output either the static or the const keywords if you define both. This is not documented anywhere and although – knowing the answer – it makes some sort of sense, it does mean that mapping these attributes is not as simple as finding the matching meaning and flipping the bit.

Also here we find out that there is apparently no way to declare a readonly field using CodeDOM. We know from reflection if we’ve got such a field as the InitOnly flag is set in the Attributes property of the FieldInfo object, but there’s no corresponding setting in the CodeDOM MemberAttributes enumeration. This is annoying but not critical to our understanding of the decompiled code.

Having mapped the attributes, we now need to handle the special case where a field is marked as Literal in the reflection class. This will happen for constants and enumerations, both of which need to have an initial value specified. I’ll show you the code first and then talk through it:

// handle field initialisation
if (info.IsLiteral)
{
// get initial value
object val = info.GetValue(null);

// make sure we use a valid primitive type
switch(Convert.GetTypeCode(val))
{
case TypeCode.Empty:
break;
case TypeCode.Double:
case TypeCode.Single:
val = System.Convert.ChangeType(val, typeof(System.Double));
break;
case TypeCode.String:
val = System.Convert.ChangeType(val, typeof(System.String));
break;
default:
val = System.Convert.ChangeType(val, typeof(System.Int64));
break;
}

// create initial expression
newField.InitExpression = new CodePrimitiveExpression(val);
}

There’s no problem identifying literals, nor in obtaining the initial value. This is all held safely in the metadata and the reflection classes allow us to query this information with ease.

The problem here is really with CodeDOM.

Whenever we use the CodePrimitiveExpression class to create an expression containing a primitive value, the CSharpCodeProvider actually expects this to be not just a primitive type, but a CLS compliant data type. If it’s not, it will throw an exception when we try to generate code. This means for instance that we cannot use any unsigned numeric types to create a CodePrimitiveExpression even though these are valid C# primitive data types.

This is not too much of a problem to overcome, but it’s really something that shouldn’t be necessary. The switch statement above examines the TypeCode of the initial value (which must have been a valid primitive type or the assembly wouldn’t have compiled) and recasts the object to a compatible CLS compliant type. This avoids any problems with the code provider, but it does feel like a bit of a kludge. After all, we’re only adding a literal value into the source code, we’re not leaving anything around that implies a data type.

Having initialised any literals, all that remains is to define any custom attributes then add the field to the owning CodeDOM type declaration. As before, we use the getCustomAttributes() helper to do this and this will be explained fully later.

   // add custom
            attributes
object[] cusattr =
            info.GetCustomAttributes(false);
newField.CustomAttributes.AddRange(getCustomAttributes(cusattr));

//
            add field to owning
            type
owner.Members.Add(newField);
}

Methods

Methods can be reflected quite nicely. The main thing we can’t currently do from the System.Reflection namespace is to access the IL code in the method body. However, there is an alternative way to do this that I’ll talk about later.

One other thing we need to be careful of with methods is that we don’t extract any methods generated by the compiler – similar to the value__ field in enumerations. One example of when this happens is when a class uses properties – the get and set accessors are actually compiled as separate methods in the assembly and we can see this by using ILDasm as before.

Create a C# class library and define a simple property:

public class Test
{
private int
            prop;

public int Prop
{
get { return prop; }
set {
            prop = value; }
}
}

Compile the project and then examine it using ILDasm. You’ll see that the compiler has generated two member methods by prefixing the property name with get_ and set_ respectively (see Figure 3).

Figure 3 - Accessor Methods

As with the value__ field in an enumeration, these methods have the SpecialName flag set in their Attribute property, so we can check this during decompilation and just ignore these methods. We could of course just carry on and decompile them, but we’ll be dealing with properties as a separate issue so we don’t actually gain anything in terms of nicely decompiled code.

The addMethod() method therefore begins as follows:

private void addMethod(MethodInfo info, CodeTypeDeclaration owner)
{
// add a member method

// don't add special methods
if ((info.Attributes & MethodAttributes.SpecialName) != 0)
return;

// create CodeDOM method declaration
CodeMemberMethod newmeth = new CodeMemberMethod();
newmeth.Name = info.Name;
newmeth.Attributes = getMemberAttributes(info);

As with fields, we use an overload of getMemberAttributes() to map the reflection attributes to the corresponding CodeDOM versions. Again, we have to introduce some extra logic into this mapping process to make sure we don’t end up with any invalid combinations of flags in CodeDOM.

The problem this time is with virtual methods. If a member is not flagged as Virtual in the reflection class MethodInfo, then it must be flagged as Final in the corresponding CodeDOM declaration. If a member is flagged as Virtual, then we need to establish if this is an override of an inherited method, a new virtual method that hides an inherited method, or a genuine virtual method in the declaring class.

We start like so:

private MemberAttributes
            getMemberAttributes(MethodInfo info)
{
// convert Reflection
            MethodInfo attributes to CodeDOM
MemberAttributes attrDom =
            0;

// map attributes between types
if (info.IsAssembly)
            attrDom |= MemberAttributes.Assembly;
if (info.IsFamily) attrDom
            |= MemberAttributes.Family;
if (info.IsFamilyOrAssembly) attrDom
            |= MemberAttributes.FamilyOrAssembly;
if (info.IsPrivate) attrDom
            |= MemberAttributes.Private;
if (info.IsPublic) attrDom |=
            MemberAttributes.Public;
if (info.IsStatic) attrDom |=
            MemberAttributes.Static;
if (info.IsAbstract) attrDom |=
            MemberAttributes.Abstract;

This gets details of the accessibility of the method. Then we can examine the Virtual flag and take appropriate action:

   // check virtual
            methods
if (info.IsVirtual)
{
// see if virtual method was
            defined in a base class
if (isMethodDefined(info,
            info.ReflectedType.BaseType))
{
// if so, check for new or
            override
if ((info.Attributes & MethodAttributes.NewSlot) !=
            0)
attrDom |= MemberAttributes.New;
else
attrDom |=
            MemberAttributes.Override;
}
}
else
{
// if not
            virtual, make sure final is set
attrDom |=
            MemberAttributes.Final;
}

// return CodeDOM
            enumeration
return attrDom;
}

The reflected attributes we need to examine to distinguish between an override and a new method are NewSlot and ReuseSlot. These flags indicate whether the method takes a new slot in the vtable or reuses an existing slot (the default action). As these are mutually exclusive, we need only test one of them to see if this is an override or not. However, we only need to make this distinction when the method is declared in a base class, hence the initial test in the code above. This is more messy than I’d like, but we can’t tell the difference between a new virtual method (that hides a base class method) and a genuinely new virtual method in the derived class just by examining the reflected attributes! We’re forced to examine the base class to see if such a method is also defined there. This is done by the isMethodDefined() helper, which takes the MethodInfo of the method to match and the Type of the class to match it in. It uses Type.GetMethod() to search for a method with the same name and the same number, order and type of parameters. Like this:

private bool isMethodDefined(MethodInfo meth, Type type)
{
// return true if a matching method is defined on type
if (type == null) return false;

// create array of parameter types
ParameterInfo[] parms = meth.GetParameters();
Type[] parmtypes = new Type[parms.Length];
for(int i = 0; i < parms.Length; i++)
parmtypes[i] = parms[i].ParameterType;

// set binding flags
BindingFlags bindingattr =
BindingFlags.ExactBinding | BindingFlags.NonPublic |
BindingFlags.Public | BindingFlags.Instance;

// attempt to find method with same name and parms
MethodInfo match = type.GetMethod(meth.Name, bindingattr,
null, parmtypes, null);

// return result
return (match != null);
}

Right, we’ve now got the method’s attributes converted correctly so we can carry on with the addMethod() helper and add any custom attributes.

object[] cusattr = info.GetCustomAttributes(false);
newmeth.CustomAttributes.AddRange(getCustomAttributes(cusattr));

Then we can find the parameter list for this method and build matching CodeDOM declarations. As we’ll also need to do this for other members, we’ll use another helper method – getParameterDeclarations() – which I’ll describe later.

newmeth.Parameters.AddRange(getParameterDeclarations(info.GetParameters()));

The last thing we need to do is to set the return type of the method and then add it to the owning CodeDOM type declaration:

   newmeth.ReturnType = new
            CodeTypeReference(info.ReturnType);

// add method to owning
            type
owner.Members.Add(newmeth);
}

We’ll come back to the addMethod() helper later when we deal with the method body. This is also something we need to do for constructors and properties so we’ll cover the basics of these first.

Constructors

As far as the CLR is concerned, constructors come in two flavours. There’s the normal instance constructor, which is flagged in the metadata with the SpecialName attribute and given the name .ctor. Also, there may optionally be a type constructor (also known as a static constructor, class constructor or type initialiser). This also has the SpecialName attribute, but this time with a name of .cctor.

These constructors have different CodeDOM declarations. An instance constructor is declared using CodeConstructor and a type constructor is declared using CodeTypeConstructor, both of which descend from CodeMemberMethod. We can check the constructor flavour up front because, besides having the special name .cctor, a type constructor is also flagged as static.

private void addConstructor(ConstructorInfo
            info, CodeTypeDeclaration owner)
{
// add a constructor to the
            given type
CodeMemberMethod newcons;

// check for a class
            constructor
if (info.IsStatic)
newcons = new
            CodeTypeConstructor();
else
newcons = new
            CodeConstructor();

Then we can set the name, attributes and parameters:

   // set name, attributes
            and parameters
newcons.Name = info.Name;
newcons.Attributes =
            getMemberAttributes(info);
object[] cusattr =
            info.GetCustomAttributes(false);
newcons.CustomAttributes.AddRange(getCustomAttributes(cusattr));
newcons.Parameters.AddRange(getParameterDeclarations(info.GetParameters()));

//
            add constructor definition to
            owner
owner.Members.Add(newcons);
}

A type constructor never actually has any parameters, but the above coding is clean and simple and there’s no real harm in performance terms by attempting to process an empty collection.

We’re using the still-to-be-defined methods getCustomAttributes() and getParameterDeclarations() here and also another overload of getMemberAttributes().

private MemberAttributes
            getMemberAttributes(ConstructorInfo info)
{
// convert
            Reflection ConstructorInfo attributes to CodeDOM
MemberAttributes
            attrDom = 0;

// map attributes between types
if
            (info.IsAssembly) attrDom |= MemberAttributes.Assembly;
if
            (info.IsFamily) attrDom |= MemberAttributes.Family;
if
            (info.IsFamilyOrAssembly) attrDom |=
            MemberAttributes.FamilyOrAssembly;
if (info.IsPrivate) attrDom |=
            MemberAttributes.Private;
if (info.IsPublic) attrDom |=
            MemberAttributes.Public;
if (info.IsStatic) attrDom |=
            MemberAttributes.Static;

// return CodeDOM
            enumeration
return attrDom;
}

Ideally, these attributes need to be mapped slightly differently into CodeDOM to avoid unexpected side effects, although we could have just use the same mapping as we did for a normal method. There are no conflicting attributes between constructors and methods, but the additional code we have for handling virtual methods clouds the issue a little and I believe it’s best to keep these mappings separate.

Properties

We touched on properties earlier, where we saw how the accessor methods are held in the assembly as specially-named members. There’s another member type that’s held in an assembly as a property – a type indexer.

Unfortunately, although we can see these indexers as properties using reflection, we cannot reconstruct the C# source code using the CodeDOM property class, CodeMemberProperty. This just doesn’t have the ability to represent an indexer and there is no specialised CodeDOM class that does. So what can we do?

Well, we can cheat.

CodeDOM doesn’t actually validate the source code it generates, and we can make it happily spit out any invalid member names we like. This helps us here because, if we remind ourselves what an ordinary property looks like:

public int NormalProperty
{
get { //
            TODO }
set { // TODO }
}

and then what a type indexer looks like:

public bool this[SomeParameters]
{
get { // TODO }
set { // TODO }
}

Then we can just build a property named in the above format (i.e. " this[parameter list] ") and supply this to CodeDOM in the normal way to reconstruct the desired code!

So let’s start building a property from the reflected PropertyInfo. The first thing we do is to get the accessibility information (public, private, etc.), which, for a property, is actually held against the SpecialName accessor methods (either one will do). We can get hold of these easily enough like this:

private void addProperty(PropertyInfo info, CodeTypeDeclaration owner)
{
// add a property to the given type
CodeMemberProperty newprop = new CodeMemberProperty();

// get accessibility from get/set methods
MethodInfo[] methinfo = info.GetAccessors(true);
if (methinfo.Length > 0)
newprop.Attributes = getMemberAttributes(methinfo[0]);

By using the GetAccessors() method, we don’t care whether this is a read only or write only property, we will always get at least one of these.

Next, we add any custom attributes using our ubiquitous helper method getCustomAttributes().

// add custom attributes
object[] cusattr = info.GetCustomAttributes(false);
newprop.CustomAttributes.AddRange(getCustomAttributes(cusattr));

Then we can set more basic information about the property – its name, type and whether it is read only or write only. All of this can be copied straight from the PropertyInfo object we’ve obtained using reflection.

   // set basic
            details
newprop.Name = info.Name;
newprop.Type = new
            CodeTypeReference(info.PropertyType);
newprop.HasGet =
            info.CanRead;
newprop.HasSet = info.CanWrite;

Then we check to see if this is a type indexer or not. In C#, a type indexer is normally given the name Item. We could test this name against the PropertyInfo but, as it is possible for a developer to have forced the C# compiler to use a different name (by applying the System.Runtime.CompilerServices.IndexerName custom attribute), there’s a much simpler way. For an indexer, the get accessor method will have a parameter list. If it doesn’t have one, then we can safely assume that we have a genuine property. We can get the parameter list for the get accessor like this:

// check for a type indexer
MethodInfo getmeth = info.GetGetMethod(true);
ParameterInfo[] parms = getmeth.GetParameters();

Then we can check for a type indexer and, if necessary, assign our carefully constructed name:

   // if get accessor has
            parameters, this is an indexer
if (parms.Length >
            0)
{
// derive new name based on parameters
newprop.Name =
            "this[" + getArgumentList(parms) + "]";
}
// add property to
            owning type
owner.Members.Add(newprop);
}

The getArgumentList() method just loops through the parameters and builds a comma delimited string in conventional C# format.

private string
            getArgumentList(ParameterInfo[] parms)
{
// build string with
            any parameter details
StringBuilder sb = new
            StringBuilder();
bool comma = false;
foreach(ParameterInfo
            parm in parms)
{
// add comma if not first in list
if
            (comma) sb.Append(", ");

// add parameter type and
            name
sb.Append(parm.ParameterType.ToString());
sb.Append("
            ");
sb.Append(parm.Name);
comma = true;
}

// return
            result
return sb.ToString();
}

This takes care of both properties and type indexers. We’ll just clear up a couple of loose ends before looking at extracting details of method bodies, so here goes with parameters and custom attributes…

Parameter Declarations

Parameters may be declared on methods or constructors. When we dealt with member types, we invoked a helper method – getParameterDeclarations(), which accepted a ParameterInfo array (from reflection) and returned a CodeParameterDeclarationExpressionCollection for CodeDOM (that must be one of the longest class names in the FCL!). This method is defined as follows:

private CodeParameterDeclarationExpressionCollection getParameterDeclarations(ParameterInfo[] infoarray)
{
// convert Reflection ParameterInfo to CodeDOM
CodeParameterDeclarationExpressionCollection parms =
new CodeParameterDeclarationExpressionCollection();
foreach(ParameterInfo info in infoarray)
{
// create a new CodeDOM parameter
CodeParameterDeclarationExpression parm =
new CodeParameterDeclarationExpression();

// copy declaration from reflected info
parm.Name = info.Name;
parm.CustomAttributes.AddRange(getCustomAttributes(info.GetCustomAttributes(false)));
parm.Type = new CodeTypeReference(info.ParameterType);
if (info.IsIn)
{
// check for ref parameter
if (info.ParameterType.IsByRef)
parm.Direction = FieldDirection.Ref;
else
parm.Direction = FieldDirection.In;
}
else if (info.IsOut) parm.Direction = FieldDirection.Out;

// add parameter to collection
parms.Add(parm);
}

// return CodeDOM collection
return parms;
}

There’s not a great deal of explanation needed here. Each parameter in an argument list must have a type and a name and may optionally have any number of custom attributes and/or a field direction qualifier. All of this we can get from the ParameterInfo so we just need to create a CodeParameterDeclarationExpression for each parameter, copying the various properties and then add it to the collection.

Here we again use the getCustomAttributes() method to map custom attributes between the reflection and CodeDOM classes, so we’ll cover that next.

Custom Attributes

Custom attributes cause a lot of problems. Two of these are pretty big ones.

First, not everything you might think of as a custom attribute is actually treated as such by the C# compiler or the CLR. There are a number of common attributes that get special treatment during compilation and are actually encoded in the metadata as bit flags rather than as serialised attribute classes. This means that they are not visible through the normal reflection mechanism using the GetCustomAttributes() method. Some of these (but not all) are visible through additional properties held in MemberInfo, FieldInfo or MethodInfo objects, but they’re still not easy to get at. The sort of attributes I’m talking about here are things like [Serializable], [DllImport], [StructLayout], etc. so they are quite commonly used. They are referred to in the documentation as pseudo-custom attributes.

The second problem is that, even for genuine custom attributes, it’s not easy to reconstruct the correct source code to produce the attribute based on the information in the metadata. What happens when you define a custom attribute (not a pseudo one) in an application is that, at compile time, an instance of the attribute class is constructed using the parameters you supply in the source – these may be a combination of constructor arguments and/or public fields/properties. Then the compiler serialises the initialised attribute instance straight out into the metadata. When the assembly is subsequently loaded, these custom attributes are deserialised and so can be accessed and have their properties tested to provide all the functionality for which they are intended. This is great but it doesn’t help us reverse engineer the source code! We don’t actually want an initialised attribute, we want to know what arguments to supply to the attribute constructor to produce the same effect.

There are so many pitfalls here that it’s difficult to know what (if anything) to do. For a large number of cases, it may not be relevant that an assembly employs custom attributes if all we want to do is examine the general structure of the decompiled assembly. For some of the pseudo-custom attributes, we could check the IsSerializable, IsExplicitLayout, IsSequentialLayout, properties of a reflected type and then attempt to reconstruct some of these more well-known ones by adding CodeAttributeDeclaration objects to our CodeDOM model.

I want to keep this decompiler as clean and uncluttered as possible while still demonstrating how reflection can be used as a reverse engineering tool. The line I’m going to take here is to overlook the pseudo-custom attributes but to make you aware that we could glean some of this information from the metadata with a little more effort. It would make our code messier and it may be better to use ILDasm (with the /ADV command line switch) to dump this information. I’m also going to pass over the much more difficult problem of reconstructing a correct parameter list in the call to a custom attribute constructor and just try and give a little bit of an edge by popping out the names of any custom attributes the designers of an assembly have used and to show where they appear. This is exactly what the getCustomAttributes() helper method does:

private CodeAttributeDeclarationCollection
            getCustomAttributes(object[] attrs)
{
// convert Reflection
            custom attributes to CodeDOM
CodeAttributeDeclarationCollection
            attrsDom =
new CodeAttributeDeclarationCollection();

//
            process all incoming attributes
foreach(object attr in
            attrs)
{
// create a CodeDOM
            version
CodeAttributeDeclaration attrDom = new
            CodeAttributeDeclaration();
attrDom.Name =
            attr.ToString();

// add it to the
            collection
attrsDom.Add(attrDom);
}

// return CodeDOM
            collection
return attrsDom;
}

The reflected attributes are presented as an array of objects (although they should all descend from System.Attribute, the reflection classes are able to return any non-CLS compliant attributes too), which is processed in a simple manner to retrieve the names and to produce a matching CodeDOM collection for the relevant declaration.

I’m going to leave that one alone now as we’ve got something much more rewarding to get stuck into.

Methods – Body Snatching!

If, when you installed the .NET Framework SDK, you didn’t have a browse around the various folders created, then you may not be aware what a wealth of information can be found.

Under the Framework SDK root folder is <FrameworkSDK>\Tool Developers Guide\docs, which contains a number of MS Word documents describing in great detail the structure, layout and content of assembly manifests, the IL instruction code set, the content and meaning of all manner of metadata tables and, in the document Partition II Metadata.doc, you will find (in Chapter 24) a complete description of the layout of a Portable Executable (PE) file (that’s a DLL or an EXE), giving you everything you need to know to navigate to the manifest and locate the pointers to raw IL code bytes for the body of every method defined in the assembly.

If you take a look at that document now and feel a little daunted, you’re in good company – so did I! But, nevertheless, I realised that I could find the information I needed programmatically by loading the DLL or EXE under examination into an array of raw bytes (rather than by using the Assembly loader we used earlier) and walk my way through this data to find what I needed.

Now, there are also documents in the same folder, which describe an arguably easier way to get at the metadata, and one might suggest that we could have used these when developing our decompiler. But these functions, which provide an unmanaged API, are intended for use by compilers and loaders and provide a very fast but low-level COM interface to both inspect and emit structures in metadata. With its heavy use of pointers and unmanaged data types, the API is not amenable to use from a native C# application – it’s intended to be used from C++. As the primary purpose of this case study is to demonstrate how easy it is to decompile an assembly into a digestible form without resorting to low level tools, I made a conscious effort to disregard the unmanaged API in favour of a more laborious parsing of the metadata.

Having resolved to adopt this approach, I first took time out to surf the Internet and see if I could find any information to save me a little time. Maybe someone had already tried this and identified a way to accumulate offsets so that the relevant information could be pinpointed more quickly. There wasn’t much point in treading through individual bytes of raw metadata if there was a way to skip over large chunks. Although hopeful of finding something useful, I wasn’t expecting to be quite so fortunate in my quest.

From the web site of an obviously talented and capable developer by the name of Lutz Roeder (see http://www.aisto.com/roeder/dotnet?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq), I found the following.

Managed Reflection IL Reader for .NET

"A class library that extends the existing System.Reflection model with functions for accessing the IL bytecode by reading information out of the corresponding Module file."

Yes, Lutz – who currently works for Microsoft’s Developer Platform Division – had produced a C# class library to do exactly what I needed and, what’s more, the C# source code for this library was available for download from the above web site. After a brief correspondence by e-mail, Lutz has kindly agreed to allow me to use this class as part of my sample application for this case study. This is on the understanding that any bugs or technical problems with the class should be fired in my direction. As I have had to make some changes to the original code in order to integrate it with the decompiler, I think this is only fair!

The library consists of a C# source file, Reader.cs, which contains the class ILReader defined in a namespace System.Reflection.ILReader. I don’t intend to describe this class in great detail, really just to show you how it can be integrated into our decompiler to provide the icing on the cake. We can already produce a comprehensive decompilation of the structure of the types defined in an assembly using just the features built into the FCL itself, but the chance to pop in the IL code for each method body in the correct place was just too good to pass up! This information could in any case be obtained from ILDasm, just not in such a convenient manner.

After adding the Reader.cs source file to the DeComp project, we can access the ILReader using only a few additional lines of code. First, add a using statement:

using System.Reflection.ILReader;

Then add a private member field to the Generate class to hold an instance of the reader:

private ILReader reader;

The Reader.cs library defines an interface to be used for loading an assembly. Following the example from the IL reader download, we can add a class to the DeComp project that implements this interface as follows:

class AssemblyLoader : IAssemblyLoader
{
public Assembly Load(string assemblyName)
{
return Assembly.Load(assemblyName);
}

public Assembly LoadFrom(string fileName)
{
return Assembly.LoadFrom(fileName);
}
}

Then we need to amend the existing File / Open menu handler to create and initialise an ILReader just after we load the assembly as shown below:

private void menuFileOpen_Click(object sender, System.EventArgs e)
{
// open an existing csharp source file
if (dlgOpen.ShowDialog() == DialogResult.OK)
{
// create compile unit and load assembly for reflection
codeunit = new CodeCompileUnit();
Assembly mainAssembly = Assembly.LoadFrom(dlgOpen.FileName);

// load for ILReader
reader = new ILReader(mainAssembly.GetModules()[0],
new AssemblyLoader());

This will load a copy of the first module in the assembly (that will be the one containing the manifest) into the ILReader class. If you browse the source code for this class and look through the Microsoft documentation describing the layout of a PE file, you will see in essence what is happening here and how much work it saves us.

ILReader exposes a public method – GetMethodBody() – that returns a MethodBody for a given method. MethodBody implements an IEnumerable interface, which allows us to use the foreach syntax to access the IL instructions in the method body. So once we have instantiated an ILReader, we just need to add a couple of lines to the addMethod() helper to retrieve the method body and add it to the CodeDOM method definition as comments. I’ve used another helper method, GetILForMethod(), which does just that.

// add IL as comment block
MethodBody methodBody = reader.GetMethodBody(info);
if (methodBody != null)
newmeth.Statements.AddRange(GetILForMethod(methodBody));

We can do likewise to retrieve the body of instance and type constructors by adding the following lines to our addConstructor() method:

// add IL as comment block
MethodBody methodBody = reader.GetMethodBody(info);
if (methodBody != null)
newcons.Statements.AddRange(GetILForMethod(methodBody));

This leaves just properties (and therefore type indexers). These are slightly different as we potentially have two method bodies to retrieve – one for the get accessor and one for the set accessor.

The PropertyInfo we get from reflection allows us to access the accessor methods as MethodInfo objects by way of the GetGetMethod() and GetSetMethod() methods respectively. We don’t need to worry about finding these SpecialName methods ourselves and we also don’t care whether we have a genuine property or a type indexer. All we need to do is to check for a read-only or write-only property so that we don’t needlessly attempt to access a non-existent method. Add the following code to the addProperty() method:

   // add IL for get as
            comment block
if (info.CanRead)
{
MethodInfo getinfo =
            info.GetGetMethod(true);
MethodBody methodBody =
            reader.GetMethodBody(getinfo);
if (methodBody !=
            null)
newprop.GetStatements.AddRange(GetILForMethod(methodBody));
}

//
            add IL for set as comment block
if
            (info.CanWrite)
{
MethodInfo setinfo =
            info.GetSetMethod(true);
MethodBody methodBody =
            reader.GetMethodBody(setinfo);
if (methodBody !=
            null)
newprop.SetStatements.AddRange(GetILForMethod(methodBody));
}

To finally complete the decompiler, we can now define the method GetILForMethod(), which returns the decoded IL method body as an array of CodeCommentStatement objects. I have adapted this routine from Lutz Roeder’s example code and in general it works like this:

  • IL instructions are written as text lines to a StringWriter.
  • The StringWriter is converted to an array of strings.
  • The string array is used to build a CodeCommentStatement array.

We start off by declaring a StringWriter and adding some basic details about the method:

private CodeCommentStatement[]
            GetILForMethod(MethodBody methodBody)
{
// return method body
            as decoded IL in CodeDOM comments
StringWriter sw = new
            StringWriter();

Sw.NewLine = "\n";
sw.WriteLine(".maxstack
            " + methodBody.MaxStack);

// list local variables
Type[]
            locals = methodBody.Locals;
if ((locals != null) &&
            (locals.Length > 0))
{
sw.Write(".locals (");
for (int i
            = 0; i < locals.Length;
            i++)
{
sw.Write(locals[i].ToString() + " V_" + i);
if (i !=
            (locals.Length - 1)) sw.Write(",
            ");
}
sw.WriteLine(")");
}

In a similar manner to ILDasm, a list of local variables is produced. Unlike ILDasm, however, ILReader always assigns new variable names. In debug builds, the original variable names can be obtained from the metadata but it is not a straightforward task to retrieve these. In short, ILDasm does it but we don't.

Next, details of exception handlers are listed. These are held in one or more separate data sections following the method body in the assembly and are extracted by ILReader into an array of ExceptionHandler objects. The details of the IL code lines covered by each exception handler are listed at the top of the method body following the list of local variables:

// exception handlers
ExceptionHandler[]
            exceptions = methodBody.Exceptions;
if (exceptions !=
            null)
{
foreach (ExceptionHandler handler in
            exceptions)
{
sw.Write(".try L_" +
            handler.Try.Offset.ToString("x4"));
sw.Write(" to L_" +
(handler.Try.Offset + handler.Try.Length).ToString("x4") + "
            ");

if (handler is Catch)
{
Catch catchHandler =
            (Catch) handler;
sw.Write("catch " +
            catchHandler.Type.ToString());
}

if (handler is
            Filter)
{
Filter filterHandler = (Filter)
            handler;
sw.Write("filter L_" +
filterHandler.Expression.ToString("x4"));
}

if
            (handler is Finally) sw.Write("finally");
if (handler is Fault)
            sw.Write("fault");

sw.Write(" L_" +
            handler.Handler.Offset.ToString("x4"));
sw.WriteLine(" to L_" +
(handler.Handler.Offset +
            handler.Handler.Length).ToString("x4"));
}
}

The IL instructions are written next. These are formatted according to the instruction type and OpCode using a switch statement:

// instructions
foreach (Instruction instruction in methodBody)
{
OpCode opCode = instruction.OpCode;
Object operand = instruction.Operand;
Byte[] operandData = instruction.OperandData;

sw.Write("L_" + instruction.Offset.ToString("x4") + ": ");
sw.Write(opCode.Name + " ");

if (operand != null)
{
switch (opCode.OperandType)
{
case OperandType.InlineNone:
break;

case OperandType.ShortInlineBrTarget:
case OperandType.InlineBrTarget:
int target = (int) operand;
sw.Write("L_" + target.ToString("x4"));
break;

case OperandType.ShortInlineI:
case OperandType.InlineI:
case OperandType.InlineI8:
case OperandType.ShortInlineR:
case OperandType.InlineR:
sw.Write(operand.ToString());
break;

case OperandType.InlineString:
sw.Write("\"" + operand.ToString() + "\"");
break;

case OperandType.ShortInlineVar:
case OperandType.InlineVar:
if (operand is int)
sw.Write("V_" + operand.ToString());
else if (operand is ParameterInfo)
{
ParameterInfo parameterInfo =
(ParameterInfo) operand;
sw.Write((parameterInfo.Name != null) ?
parameterInfo.Name : ("A_" + parameterInfo.Position));
}
break;

case OperandType.InlineSwitch:
sw.Write("(");
int[] targets = (int[]) operand;
for (int i = 0; i < targets.Length; i++)
{
if (i != 0) sw.Write(", ");
sw.Write("L_" + targets[i].ToString("x4"));
}
sw.Write(")");
break;

case OperandType.InlineSig:
case OperandType.InlineMethod:
case OperandType.InlineField:
case OperandType.InlineType:
case OperandType.InlineTok:
if (operand is Type)
sw.Write((Type) operand);
else if (operand is MemberInfo)
{
MemberInfo memberInfo = (MemberInfo) operand;
if (memberInfo.DeclaringType != null)
{
sw.Write(memberInfo.DeclaringType);
sw.Write("::");
}
sw.Write(memberInfo);
}
else
throw new Exception();
break;
}
}
else
{
if (operandData != null)
{
sw.Write("null // " + instruction.OpCode.OperandType + " ");
foreach (Byte b in operandData)
sw.Write(b.ToString("X2") + " ");
}
}

sw.WriteLine();
}

When we reach this point, we have the decoded IL method body in string format (in the StringWriter), with individual lines delimited by newline characters.

To convert these to an array of strings, we can split on the newline delimiter:

// convert to string
string ilcode = sw.ToString();

// split lines into string array
string[] lines = ilcode.Split(new char[] {'\n'});

Then we create an array of CodeCommentStatement, which will be returned as the result of the method:

// create an array of comment statements
CodeCommentStatement[] states = new CodeCommentStatement[lines.Length];

Finally, we build a CodeCommentStatement from each string, add it to the above array and return the result:

   // add strings as comment
            lines
for (int i = 0; i < lines.Length; i++)
{
states[i]
            = new CodeCommentStatement(lines[i]);
}

// return
            array
return states;
}

At last, this completes the reflection-based decompiler. We can give it a test…

Testing the Decompiler

With the download material for this article, I have included a project – TestDecomp – that contains all of the following code. This project is a C# class library in which I have coded all of the structures we need to test the various aspects of the decompiler. In this section, I will go through each test, showing the original source code together with the decompiled version. This will not show the TestDecomp source as if I have compiled each version, and it will be in no particular order. I’ll just extract relevant elements from the complete decompilation in order to illustrate particular items of interest. It’s easier to highlight different features this way than by presenting you with two three-page source files!

The first thing to check is that we can decompile the general structure of an assembly in terms of its constituent classes. The source code to do this includes conventional classes, an abstract class and a nested class. The original definitions (edited for clarity) are as follows:

namespace TestDecomp
{
public class
            ClassWithStaticConstructor { }

public abstract class
            AbstractClass { }

public class DescendsFromAbstractClass :
            AbstractClass { }

public class TestClass :
            DescendsFromAbstractClass
{
private class PrivateNestedClass {
            }

public class PublicNestedClass { }
}
}

When the compiled class library is decompiled from the resulting DLL, we get the following source code back (again, edited for clarity) – see Figure 4 for a screenshot:

namespace TestDecomp
{
public class
            ClassWithStaticConstructor { }

public abstract class
            AbstractClass { }

public class DescendsFromAbstractClass :
            TestDecomp.AbstractClass {
            }

[System.Reflection.DefaultMemberAttribute()]
public
            class TestClass :
            TestDecomp.DescendsFromAbstractClass
{
private class
            PrivateNestedClass { }

public class PublicNestedClass {
            }
}
}

Figure 4 - Testing the Decompiler

You can see we got everything we expected, even in the same order as originally defined (though this is not guaranteed), but with one additional line. The DefaultMemberAttribute has been automatically applied to the TestClass type. This is because (edited out of the above example) I have defined a type indexer in this class. Discussing why the compiler does this is a little out of scope right here but it’s the first of several indications that, in terms of source code, our decompiler actually appears to get back more information than went in! What we’re exposing here is code that the compiler has added on our behalf to implement our source code in an assembly suitable for the CLR. We touched on this when we discovered how the set and get accessor methods are compiled from a property definition and we’ll see more in a while. Just staying with properties and type indexers for a moment, these are the original definitions within TestClass :

public int publicproperty
{
get {
            return privateintfield; }
set { privateintfield = value;
            }
}

public bool this[int bitnumber]
{
get { return
            (bitnumber == 1); }
set { if (value) privateintfield = 1;
            }
}

And this is the decompiled equivalent. First the property:

public int
            publicproperty
{
get
{
// .maxstack 1
// L_0000:
            ldarg.0
// L_0001: ldfld TestDecomp.TestClass::Int32
            privateintfield
// L_0006: ret
}
set
{
// .maxstack
            2
// L_0000: ldarg.0
// L_0001: ldarg.1
// L_0002: stfld
            TestDecomp.TestClass::Int32 privateintfield
// L_0007: ret
}
}

Then the type indexer:

public bool this[System.Int32
            bitnumber]
{
get
{
// .maxstack 2
// L_0000: ldarg.1
// L_0001: ldc.i4.1
// L_0002: ceq
// L_0004: ret
}
set
{
// .maxstack 2
// L_0000: ldarg.2
//
            L_0001: brfalse.s L_000a
// L_0003: ldarg.0
// L_0004:
            ldc.i4.1
// L_0005: stfld TestDecomp.TestClass::Int32
            privateintfield
// L_000a: ret
}
}

You see in both cases how it would not take a great deal of effort to understand the operation of these (albeit simple) accessor methods.

Let’s make sure we handle class declarations and constructors correctly. The ClassWithStaticConstructor is exactly that. It’s defined in TestDecomp like this:

public class ClassWithStaticConstructor
{
public static string staticstring;

static ClassWithStaticConstructor()
{
staticstring = "test";
}

And it decompiles like this:

public class
            ClassWithStaticConstructor
{
public static string
            staticstring;

static ClassWithStaticConstructor()
{
//
            .maxstack 1
// L_0000: ldstr "test"
// L_0005: stsfld
            TestDecomp.ClassWithStaticConstructor::System.String
            staticstring
// L_000a: ret
}

public
            ClassWithStaticConstructor()
{
// .maxstack 1
// L_0000:
            ldarg.0
// L_0001: call System.Object::Void .ctor()
//
            L_0006: ret
}

We can see here that we have decompiled a static constructor and an instance constructor in the same class. Looking at the original source, we see we didn’t actually define an instance constructor – this again has been generated automatically by the C# compiler, and you can probably deduce from the constructor body that, as we’d expect, it just calls the base class constructor (i.e. from System.Object).

Moving on, we can add some enumerations to this class. We’ll define one that uses bit flags and one that doesn’t. The original definitions are:

[Flags]
public enum BitflagUintEnum : uint
{
UintEnum1 = 0x00000001,
UintEnum2 = 0x00000002,
UintEnum4 = 0x00000004,
UintEnum8 = 0x00000008
}

public enum NormalByteEnum : byte
{
ByteEnum1 = 1,
ByteEnum2 = 2
}

Which decompiles almost verbatim to the following:

[System.FlagsAttribute()]
public enum BitflagUintEnum : System.UInt32
{
UintEnum1 = 1,
UintEnum2 = 2,
UintEnum4 = 4,
UintEnum8 = 8,
}

public enum NormalByteEnum : System.Byte
{
ByteEnum1 = 1,
ByteEnum2 = 2,
}

All that’s happened here is that the type names are now fully-qualified genuine names rather than the C# aliases uint and byte. The initial values have also been correctly regenerated.

We’ll make sure we can handle struct’s as well as classes and enums. So, still within the same class, we’ll add the following original definitions:

private struct PrivateStruct
{
public
            int nSize;
public int nType;

public PrivateStruct(int
            size, int type)
{
nSize = size;
nType =
            type;
}
}

Which we find will decompile successfully as follows:

private struct PrivateStruct
{
public
            int nSize;
public int nType;

public PrivateStruct(int
            size, int type)
{
// .maxstack 2
// L_0000: ldarg.0
//
            L_0001: ldarg.1
// L_0002: stfld
            TestDecomp.ClassWithStaticConstructor+PrivateStruct::Int32
            nSize
// L_0007: ldarg.0
// L_0008: ldarg.2
// L_0009:
            stfld TestDecomp.ClassWithStaticConstructor+PrivateStruct::Int32
            nType
// L_000e: ret
}
}

Here again we can see the benefit of providing a decompiled IL body right in the generated source code.

Moving back briefly to the class level, we need to check that the abstract/virtual/override/new scenarios all work correctly. So in the TestDecomp project, we’ll define an abstract class and a class that is derived from it, with two method overrides.

I’ve added a nonsensical statement in the overridden method bodies just so we can check what action the compiler will take.

The original code is:

public abstract class
            AbstractClass
{
public AbstractClass()
{
}

public
            abstract void abstractmethod();
public abstract void
            anotherabstractmethod();
}

public class
            DescendsFromAbstractClass : AbstractClass
{
public override
            void abstractmethod()
{
int x = 0;
}

public override
            void anotherabstractmethod()
{
int x =
0;
}
}

And the decompiled version:

public abstract class
            AbstractClass
{
public AbstractClass()
{
// .maxstack
            1
// L_0000: ldarg.0
// L_0001: call System.Object::Void
            .ctor()
// L_0006: ret
}

public abstract void
            anotherabstractmethod();
public abstract void
            abstractmethod();
}

public class DescendsFromAbstractClass
            : TestDecomp.AbstractClass
{
public
            DescendsFromAbstractClass()
{
// .maxstack 1
// L_0000:
            ldarg.0
// L_0001: call TestDecomp.AbstractClass::Void
            .ctor()
// L_0006: ret
}

public override void
            anotherabstractmethod()
{
// .maxstack 0
// L_0000: ret
}

public override void abstractmethod()
{
//
            .maxstack 0
// L_0000: ret
}
}

Notice here how the compiler has created a default constructor for DescendsFromAbstractClass, this time calling the correct base class constructor. The compiler has also optimised out our nonsensical method bodies! This is because I have decompiled the Release version of TestDecomp. Decompiling the Debug version retrieves the non-optimised nonsensical methods just as we wrote them. For example:

    public override void
            anotherabstractmethod()
{
// .maxstack 1
// .locals
            (System.Int32 V_0)
// L_0000: ldc.i4.0
// L_0001: stloc.0
// L_0002: ret
}

This is a case where the decompiler may sometimes give us less than we expected! This is a point to bear in mind when testing the decompiler although it can also give some useful insights into your own code when you can examine what the compiler considers important during its optimisation phase.

Moving on to the TestClass definition, we can check that we are able to decompile member fields correctly by declaring all manner of combinations of access modifiers. Again, the original is:

public class TestClass : DescendsFromAbstractClass
{
public int[] publicintarray;
private const string privateconststring = "pcstring";
public int publicintfield;
public static float publicstaticfloatfield;
public const float publicconstfloatfield = 10.5f;
public readonly int publicreadonlyint = 1;
protected int protectedintfield;
protected static int protectedstaticintfield;
protected const int protectedconstintfield = 1;
private int privateintfield;
private static int privatestaticintfield;
private const int privateconstintfield = 1;
internal int internalintfield;
internal static int internalstaticintfield;
internal const int internalconstintfield = 1;
protected internal int protectedinternalintfield;
protected internal static int protectedinternalstaticintfield;
protected internal const int protectedinternalconstintfield = 1;

Followed by the decompiled version:

[System.Reflection.DefaultMemberAttribute()]
public class TestClass : TestDecomp.DescendsFromAbstractClass
{
public int[] publicintarray;
public int publicintfield;
public int publicreadonlyint;
protected int protectedintfield;
private int privateintfield;
internal int internalintfield;
protected internal int protectedinternalintfield;
public static System.Single publicstaticfloatfield;
protected static int protectedstaticintfield;
private static int privatestaticintfield;
internal static int internalstaticintfield;
protected internal static int protectedinternalstaticintfield;
private const string privateconststring = "pcstring";
public const System.Single publicconstfloatfield = 10.5;
protected const int protectedconstintfield = 1;
private const int privateconstintfield = 1;
internal const int internalconstintfield = 1;
protected internal const int protectedinternalconstintfield = 1;

Looking through and comparing these definitions you’ll see that the only failure has been with the readonly field publicreadonlyint. As discussed earlier, although we know from the reflected attributes that the field is readonly, there is no way declare such a field using CodeDOM. This appears not to be the only problem though – it looks like we’ve lost the initial value too! Well, yes and no, all becomes clear when we consider the constructor for this class. First, the original:

public TestClass(int
            intfield)
{
publicintfield = intfield;
}

Then the decompiled equivalent:

public TestClass(int intfield)
{
//
            .maxstack 2
// L_0000: ldarg.0
// L_0001: ldc.i4.1
//
            L_0002: stfld TestDecomp.TestClass::Int32 publicreadonlyint
//
            L_0007: ldarg.0
// L_0008: call
            TestDecomp.DescendsFromAbstractClass::Void .ctor()
// L_000d:
            ldarg.0
// L_000e: ldarg.1
// L_000f: stfld
            TestDecomp.TestClass::Int32 publicintfield
// L_0014: ret
}

Notice the first three lines in the method body? As I mentioned earlier, this is what the C# compiler does with initialised instance fields. All the initialisation code is moved to the front of the instance constructor. As we saw before, there will always be a constructor, even if we don’t define one, and the compiler always initialises any non-constant fields in this way.

Earlier, we started checking that we could correctly reconstruct overridden methods from abstract base classes. As TestClass sits at the bottom of the previous hierarchy, we can also check that we can detect hidden virtual methods. In TestClass, we’ll define a smattering of methods (bodies omitted):

public override void abstractmethod() {
            }
public virtual new void anotherabstractmethod() { }
public
            void publicvoidmethod(int intparm) { }
public static void
            publicstaticmethod(int intparm) { }
public virtual void
            publicvirtualmethod(int intparm) { }
public int
            publicintmethod(ref int refintparm) { }

Then we can check each of these in the decompiled version:

public override void abstractmethod() {
            }
public new virtual void anotherabstractmethod() { }
public
            void publicvoidmethod(int intparm) { }
public static void
            publicstaticmethod(int intparm) { }
public virtual void
            publicvirtualmethod(int intparm) { }
public int
            publicintmethod(System.Int32& refintparm) { }

Which seems to work fine. We’re handling the virtual, new and override methods correctly and recreating the correct method parameters and types. We’ve even detected the ref parameter, though CodeDOM has rendered the source in a slightly different format.

In TestClass, we had defined two nested classes. For the first of these – PrivateNestedClass – we can include some exception handling in the constructor to make sure we decompile that in an understandable manner. So the original source is like this:

private class
            PrivateNestedClass
{
private string privatestringfield =
            "test";
private float privatefloatfield;

public
            PrivateNestedClass(string s, float f)
{
float localfloat =
            1.5f;
try
{
privatestringfield = s;
}
catch(Exception
            ex)
{
throw new
            ArgumentException("Error");
}
finally
{
privatefloatfield
            = f * localfloat;
}
}
}

I’ve added a couple of member fields to make sure these get output at the right level in the nested class and also introduced some different data types and multiple parameters.

The decompiled version of this class appears as follows:

private class
            PrivateNestedClass
{
private string
            privatestringfield;
private System.Single
            privatefloatfield;

public PrivateNestedClass(string s,
            System.Single f)
{
// .maxstack 3
// .locals (System.Single
            V_0)
// .try L_0017 to L_0020 catch System.Exception L_0020 to
            L_002c
// .try L_0017 to L_002e finally L_002e to L_0038
//
            L_0000: ldarg.0
// L_0001: ldstr "test"
// L_0006: stfld
            TestDecomp.TestClass+PrivateNestedClass::System.String
            privatestringfield
// L_000b: ldarg.0
// L_000c: call
            System.Object::Void .ctor()
// L_0011: ldc.r4 1.5
// L_0016:
            stloc.0
// L_0017: ldarg.0
// L_0018: ldarg.1
// L_0019:
            stfld TestDecomp.TestClass+PrivateNestedClass::System.String
            privatestringfield
// L_001e: leave.s L_002c
// L_0020: pop
// L_0021: ldstr "Error"
// L_0026: newobj
            System.ArgumentException::Void .ctor(System.String)
// L_002b:
            throw
// L_002c: leave.s L_0038
// L_002e: ldarg.0
//
            L_002f: ldarg.2
// L_0030: ldloc.0
// L_0031: mul
//
            L_0032: stfld TestDecomp.TestClass+PrivateNestedClass::Single
            privatefloatfield
// L_0037: endfinally
// L_0038: ret
}
}

Without reiterating too many details, you’ll see that this is a fair decompilation of the original code. Notice how the line numbers covered by the try and catch blocks are listed at the start of the decompiled method body.

The last thing to check is events. For this, we’ll use the other nested class – PublicNestedClass – and just define a delegate and an event:

public class PublicNestedClass
{
public delegate void publicdelegate(object sender, EventArgs e);
public event publicdelegate publicevent;
}

This should be pretty straightforward, the event member type was one of the first (and easiest) we came across. However, when we come to decompile this class, as you’ve probably already guessed, we get much more than we bargained for:

public class PublicNestedClass
{
private TestDecomp.TestClass.PublicNestedClass.publicdelegate publicevent;

public PublicNestedClass()
{
// .maxstack 1
// L_0000: ldarg.0
// L_0001: call System.Object::Void .ctor()
// L_0006: ret
}

private event TestDecomp.TestClass.PublicNestedClass.publicdelegate publicevent;

public sealed class publicdelegate : System.MulticastDelegate
{

public publicdelegate(object @object, System.IntPtr method)
{
}

public virtual void EndInvoke(System.IAsyncResult result)
{
}

public virtual System.IAsyncResult BeginInvoke(object sender,
System.EventArgs e, System.AsyncCallback callback, object @object)
{
}

public virtual void Invoke(object sender, System.EventArgs e)
{
}
}
}

If you’ve read any of the .NET framework documentation covering events and delegates then perhaps this will not be so much of a surprise. In any case, let’s pick it apart and see what the compiler actually did when it compiled our original two lines of code.

First, ignore the automatically generated constructor. We’ve covered that already and it has nothing to do with the event or delegate. However, the following line,

public event publicdelegate publicevent;

actually caused the compiler to generate three things. Two of these we don’t see in the decompiled code, they are methods with the SpecialName attribute set (we deliberately ignored these, remember?). They are named add_publicevent() and remove_publicevent() and are the methods used to allow objects to register and unregister interest in the event.

You can see these methods using ILDasm as before (see Figure 5):

Figure 5 - Event Methods

The third thing generated by the event declaration is a private field named after the delegate – shown by this line in the decompiled source:

private event TestDecomp.TestClass.PublicNestedClass.publicdelegate publicevent;

You’ll see that this field has a data type of publicdelegate, so what exactly is this type? Well you can see from the decompiled code and from Figure 5 that publicdelegate is a complete nested class definition. The class is derived from System.MulticastDelegate (as are all delegate types) and has four methods defined. It’s marked as public because we defined our delegate as public. Things become clearer when you realise now why a delegate is the only C# entity that can be declared at global scope – this is because it’s actually a class, so we’re not really breaking any rules.

The four methods – the constructor, Invoke, BeginInvoke and EndInvoke – handle the mechanics of invoking call-back methods in objects who have registered an interest in the corresponding event. This is too deep a subject to go into here, the real point being the appreciation that decompiling a .NET assembly could end up telling you as much about the way the C# compiler and the CLR go about their business as it does about the application code itself.

You may also notice that there are no method bodies in the delegate class. This is because they are marked as RunTime code and so cannot be accessed in the same way as our other method bodies. If you examine the ILReader class, you will see that it specifically ignores certain types of method, RunTime being just one of them. The others are Native and OPTIL, representing native code (e.g. x86 instructions) and parts of the .NET framework infrastructure respectively.

We could have produced a cleaner decompilation of the delegate declaration – now we know how it’s implemented – by using the parameter list from the delegate’s Invoke() method to create a CodeDOM CodeMemberDelegate and so recreate the original source code. However, we would still need to filter out both the nested delegate class and the private event field – neither of which are marked in any special way in the metadata, thus making our task a little more complicated. Overall I think we have a pretty good decompiler that has the side effect of opening our eyes a little to what’s going on under the hood.

Application Setup

If you want to try this out for yourself, download the support material for this case study from the C#Today web site and follow the instructions in the ReadMe.txt file included.

The download contains the DeComp project and the TestDecomp class library used for testing.

Limitations

The decompiler suffers from some limitations. These fall into three categories: limitations of reflection, limitations of CodeDOM, and technical limitations.

Limitations of reflection include the inability to get directly to the IL code for a method. We needed to go to some lengths for a workaround, although this was still possible without needing to drop down into low level tools. Also, and perhaps not relevant for C# assemblies, there is currently no way using reflection that we can retrieve default values for optional parameters. A relatively minor problem.

Limitations of CodeDOM are probably a little more serious. For instance, we cannot define a readonly field using CodeDOM. Neither can we define a type indexer. For the latter, although we came up with a workaround, the effect of this is that we limited ourselves to C# as the decompilation language. If it were not for our custom treatment of type indexers, we could have offered a choice of C# or VB.NET code as the output from the decompiler. To do so now would require more workarounds and so a less optimal solution.

What I’ve termed technical problems are probably the most difficult to overcome. To reproduce native C# source code in as near a match as possible to the original is hard. Here, we have to fight the compiler itself to reverse engineer events and delegates from its automatically generated classes, fields and methods. We also have a big job on our hands to rebuild the correct constructor arguments for custom attributes. When it comes to compiler optimisation we have a slightly subtler problem. If the objective is to reverse engineer the exact source code used during compilation then we’re almost certainly lost – in an assembly of any size, the compiler will have removed redundant code, altered parameter lists, inlined code, and a whole host of other things that really mean that information has been thrown away during compilation. This is not information needed to make the assembly operate as designed, just information that may have provided additional clues as to the original nature of the source. This may or may not be a problem depending on one’s motives for decompiling in the first place.

Case Study Review

This case study set out to reverse engineer compiled .NET assemblies into C# source code. We tried to keep things as simple as possible and to keep the solution within the bounds of the C# language and the framework class library (FCL). The objective was not to demonstrate how clever we can be, but rather how little we need to know in order to glean a great deal of information about an assembly – much more than we ever could with pre-.NET executables.

While we did have to dig a little deeper than intended to produce decompiled IL method bodies, I think we succeeded in these aims. If what you’ve discovered has given you cause to think about the safety of your own intellectual property, then I have achieved what I set out to do.

The second part of this case study will look at ways of attacking this problem. We’ll look in detail at code obfuscators– how they work, what are their strengths and weaknesses, we’ll discuss alternative methods of securing sensitive code, and finally develop an example application demonstrating at least one technique for protecting your valuable code from prying eyes.

Case Study Information
Author | Roger Rowland
Editor | Adam Ryland
Reviewers | Kaushal Sanghavi, Owen Cutajar, Sean Schade

If you have any questions or comments about this case study, or suggestions for further pieces, please contact the technical editor. If you wish to talk about the case study with other subscribers, please visit our forum here.


An in-depth look at Remoting

An in-depth look at Remoting
http://www.csharptoday.com/content.asp?id=1851
Article Rating: 4.5
Published on July 29, 2002
Abstract
.NET Remoting allows us to communicate with applications that are in different application domains, irrespective of whether they are on the same machine, on different machines on the same network, or even on a totally different network half way around the world. Remoting allows us to develop distributed applications which can spread all over the globe and yet act as one single application. It is even possible to have parts of the application running on totally different architectures such as UNIX, provided .NET is installed on it. The scope of such a system can be mind-boggling. In this case study, Benny Mathew takes an in-depth look at the topic of Remoting.
Case Study

.NET Remoting is an object oriented, extensible framework integrated into the CLR (Common Language Runtime), which will enable an application to access other external objects. It is a distributed component technology such as DCOM or Java's Remote Method Invocation (RMI), but .NET is extendable, so for example it is possible to create our own custom channels which are used to communicate with the remote machine over a network. .NET can be configured to run on different protocols like TCP/IP, HTTP, SMTP, or can be even extended to run on our own custom built protocol. .NET remoting is highly configurable using an external XML based configuration file, as we will see in this case study.

.NET Remoting allows us to communicate with applications that are in different application domains, irrespective of whether they are on the same machine, on different machines on the same network, or even on a totally different network half way around the world. Remoting allows us to develop distributed applications which can spread all over the globe and yet act as one single application. It is even possible to have parts of the application running on totally different architectures such as UNIX, provided .NET is installed on it. The scope of such a system can be mind-boggling.

Application Domain Vs Process

.NET has introduced a new concept called 'Application Domain'. Previously each application would run in its own private virtual memory, so that if an application crashed, it would bring down only that application, and the other applications could continue to work without getting affected. It also provides security by not allowing an application to access memory outside its process boundary. If an application tries to access a memory location outside of its process boundary, the operating system will immediately terminate the application and throw up an error message.

Since the Intermediate Language that is generated by the .NET compilers is executed in a managed environment, which has a lot of security built-in, it will not allow the code to access resources which do not belong to it. If the code misbehaves, it will be terminated by the CLR. Hence it is not required to run a .NET application in a separate process, and moreover running an application in a different process has a lot of overheads. So Microsoft came up with the Application Domain idea. Now there can be multiple .NET applications running in the same process but in different application domains.

Communication between applications in different application domains is done using .NET remoting.

System Requirements

To try out the remoting examples in this article, the minimum system requirements are:

Development System:

  • A computer (preferably two computers) with any one of the following
  • Visual C# .NET Standard
  • Visual Studio .NET Professional
  • Visual Studio .NET Enterprise Developer
  • Visual Studio .NET Enterprise Architect
  • TCP/IP

Take a look at the minimum system requirement for installing Visual C# .NET provided by Microsoft at the following URL: http://msdn.microsoft.com/vcsharp/productinfo/sysreq.asp?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq

Deployment System

  • A computer with .NET Framework and CLR installed (preferably two computers)
  • TCP/IP

You can have a look at the following URL for information on installing the common language runtime and .NET Framework components that are necessary to run .NET Framework applications: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnnetdep/html/redistdeploy.asp&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq

Note: In all my examples I have used a single machine to run the samples. If you want to try it out on different machines, make sure you replace the string ' localhost ' with the name of your remote machine. Run the server on one machine and the client on the other.

Fundamentals of .NET Remoting

Basically, .NET Remoting consists of 3 main components:

  • The Server - hosts the remote object server-side.
  • The Remote Object - is the object whose methods we call remotely from the client-side.
  • The Client - remotely makes use of the functionality provided by the remote object.

When we create or reference a remote object at the client side, a proxy object (not the real remote object) gets created at the client side. This proxy object will intercept all the method calls you make on the remote object and it will be passed to the actual object on the server side.

The proxy object looks like the real object to the client application, with the same public methods, properties and return values. The proxy first converts \ encodes the messages into a format suitable for transmission using a formatter, and sends it to the server through a channel over the available network. At the server side, these messages are again run through a formatter, which will decode the messages and invoke the remote object method. This is taken care of internally by the remoting framework.

Namespaces

Most of the classes and interfaces used for remoting can be found under the namespace System.Runtime.Remoting. These classes can be found in a dll file called System.Runtime.Remoting.dll.

An In-depth look

Before we take a more in-depth look, let us write a small, simple, console based remoting application to aid our understanding. In this sample application, the client sends the server a string, and the server then prints it and sends back an acknowledgement. We will try to make the code in this example clean and simple without any exception handling, namespaces, etc. so that it is easier to follow the code.

The Remote Object (RemObj.dll)

We will first create the remote object:

  • Start Visual Studio .NET and click on File->New->Project.
  • Choose a C# Class Librar y type project.
  • Name the project as RemObj.
  • Remove the default namespace by right clicking on the project name RemObj in the solution explorer window, and clicking on properties.
  • Delete the code that appears by default and add the code shown below.

using System;

public class ClsRemObj : MarshalByRefObject
{
public ClsRemObj()
{
Console.WriteLine("Constructor called...");
}

public string PrintString(string str)
{
System.Console.WriteLine(str);
return "Successfully sent a string of length " + str.Length +
" characters to the server.";
}
}

  • Build the project by clicking Build-> Rebuild Solution
  • If everything went well, the compiler will generate a DLL file called RemObj.dll , which can be located under the bin/Debug folder which is under your application folder.

The Server (server.exe)

Next, we will create the Server, which will host the remote object that we created above:

  • Start Visual Studio .NET and click on File->New->Project.
  • Choose a C# Console Application type project.
  • Name the project as Server.
  • Remove the default namespace by right clicking on the project name Server in the solution explorer window, and clicking on properties
  • Add a reference to System.Runtime.Remoting by clicking on Project->Add Reference
  • Delete the code that appears by default and add the code shown below:

using System;
using
            System.Runtime.Remoting;
using
            System.Runtime.Remoting.Channels;
using
            System.Runtime.Remoting.Channels.Tcp;

public class
            ClsServer
{
public static void Main()
{
TcpChannel chan
            = new
            TcpChannel(8888);
ChannelServices.RegisterChannel(chan);
RemotingConfiguration.RegisterWellKnownServiceType(
Type.GetType("ClsRemObj,RemObj"),
"MyRemObj",WellKnownObjectMode.SingleCall);

System.Console.WriteLine("Press
            ENTER<- to
            quit");
System.Console.ReadLine();
}
}

  • Build the project by clicking Build-> Rebuild Solution
  • If everything went well, the compiler will generate an EXE file called Server.exe, which can be located under the bin/Debug folder.

The Client (Client.exe)

Next, we will create the Client, which will access the Server and make use of the remote object:

  • Start Visual Studio .NET and click on File->New->Project.
  • Choose a C# Console Application type project.
  • Name the project as Client.
  • Remove the default namespace by right clicking on the project name Client in the solution explorer window, and click on properties
  • Delete the code that appears by default and add the code shown below:

using System;
using System.Runtime.Remoting;
using System.Runtime.Remoting.Channels;
using System.Runtime.Remoting.Channels.Tcp;

public class ClsClient
{
public static void Main()
{
TcpChannel chan = new TcpChannel();
ChannelServices.RegisterChannel(chan);
ClsRemObj remObj = (ClsRemObj) Activator.GetObject(typeof(ClsRemObj),
"tcp://localhost:8888/MyRemObj");

Console.WriteLine(remObj.PrintString("Hi There!"));
Console.WriteLine(remObj.PrintString("Helloooo!"));
Console.WriteLine(remObj.PrintString("Benny here..."));
}
}

  • Add a reference to System.Runtime.Remoting by clicking on Project->Add Reference
  • Add a reference to our remote object that we created earlier by clicking on Project->Add Reference, and then clicking on the Projects TAB. If the RemObj.dll is not visible, click on the browse button and go to the directory where the RemObj.dll is stored and add it.
  • Build the project by clicking Build-> Rebuild Solution
  • If everything went well, the compiler will generate a EXE file called Client.exe
  • Run the Sample Application
  • To run the sample application:
    • Make two folders, one called the 'server' and the other called the 'client'
    • Copy Server.exe and RemObj.dll to the 'server'folder
    • Copy Client.exe and RemObj.dll to the 'client'folder
    • Open two command windows - run Server.exe in one command window and Client.exe in another.
    • Make sure that the server is running before running the client.

Note: The remote object (RemObj.dll) should be available in both the folders, since the client also needs to load the remote object class at runtime.

When you run the Client.exe, you must be able to see the following string printed on the console window of the server. This string was sent to the server from the client.

Server Console Output (SingleCall)

Press ENTER<- to quit
Constructor called...
Constructor called...
Hi There!
Constructor called...
Helloooo!
Constructor called...
Benny here...

The Client window should display the following string sent back by the server as an acknowledgement:

Client Console Output

Successfully sent a string of length 9 characters to the server.
Successfully sent a string of length 9 characters to the server.
Successfully sent a string of length 13 characters to the server.

Working of the sample application

We will now look at the working of each of the components in our example above.

The Remote Object

The remote object has to be inherited from System.MarshalByRefObject ; only those objects derived from MarshalByRefObject can be called remotely by the clients. The Remote Object Class shown above has only one method. All it does is receive a string, print it to the console, and return a string back to the client as an acknowledgement.

The Server

The role of the server is that of a container, it is responsible for hosting the remote objects. As you can see from the above code, we first create a TCP channel (I chose TCP channel for no particular reason, you could choose HTTP channel as well), which will listen on port 8888 for requests from the clients. Next we register this channel with the .NET Remoting framework, then we register the remote object so that it is available to all the clients by calling the RegisterWellKnownServiceType method (you can register the remote object as other types as well, which we will discuss later in this article) of the .NET Remoting framework.

We pass three parameters to the RegisterWellKnownServiceType method:

  • The type of the remote object to be registered

Type.GetType("ClsRemObj,RemObj")

Where 'ClsRemObj' is the name of the class. You also need to prefix the namespace if there is one.

RemObj is the name of the assembly.

  • The URI or the endpoint by which the object will be identified. In our sample application, we will name it as MyRemObj, although y ou can choose any name.
  • WellKnownObjectMode. We can specify any of the two modes, 'SingleCall' or 'Singleton'. In our example we have specified SingleCall, which means a new instance of the remote object is created every time we access the object.

If you notice the Server Console Output shown above, you can see that the constructor of the remote object is called at every method call, which means the remote object is created afresh every time one of the methods of the remote object are called. The remote object get's marked for garbage collection as soon as the method completes.

In Singleton mode, the remote object is created only once, and all the clients use the same object, every time. This is useful for sharing information between clients. Change the WellKnownObjectMode in the server program from WellKnownObjectMode.SingleCall to WellKnownObjectMode.Singleton and run the application after recompiling - you will see the following output on the Server Console Window:

Press ENTER<- to quit
Constructor called...
Hi There!
Helloooo!
Benny here...

Notice in the above output that the constructor is called only once even when the client has called the method three times, which means that the same remote object instance is used for every call.

The Client

The client creates a channel, and then registers it with the remoting framework using the RegisterChannel method of the ChannelServices class. This channel is used to communicate with the server.

Next, it gets a reference (proxy) to the remote object by using the GetObject method of the Activator class. You must understand that at this point, only the proxy is created and nothing happens at the server side. The connection to the server happens and the remote object is created at the server side only when we call a method. However, it is different for client-activated remote objects in that a remote objects is created at the server side as soon as you reference the remote object.

Here we pass two parameters to the GetObject method:

  1. The type of the remote object. typeof(ClsRemObj)
  2. The full URL where the remote object can be found, which includes the protocol, machine name, port number and the endpoint name. tcp://localhost:8888/MyRemObj

Once we have the reference to the remote object, we can use it just like a normal, local object.

In the client code above, notice that we do not specify a port number when we create the channel. This means that the channel is used to connect (not for listening) to the remote port specified by the URL. A client need not specify a specific port number since it is not publishing a service that needs to be accessed through a known port address, whereas as server needs to provide its services through a specific port number so that the clients know where to connect.

Using an interface instead of the actual remote object

Note that in the above example during compilation and at runtime, the client needs a reference to the remote object. This means that the remote object must be developed first and the DLL must be ready and available before developing the client.

Now, think of a scenario where different teams that are geographically apart develop parts of the application, and you cannot wait for the remote object to be implemented before starting development on the client. In such cases, we should first create an interface class of the remote object. The interface will have methods declared with the same signature as the methods that will be present in the actual remote object. Package the interface in a Class Library (DLL) and distribute it to all the development teams that need the remote object for development. This approach not only allows you to develop the client without first developing the remote object, it has other advantages like:

  • The implementation details are abstract / hidden from the client and can be changed / maintained at a single location.
  • Separation of Client code and Server code.
  • The code for the remote object need not be made available to all clients.
  • Since .NET assemblies can be easily disassembled, an interface approach will also protect the intellectual property (codes).

Our sample application will now look like this:

Interface (IRemObj.dll)

using System;

public interface
            IRemObj
{
string PrintString(string str);
}

The remote object needs to implement this interface:

The Remote Object (RemObj.dll)

using System;

public class ClsRemObj : MarshalByRefObject, IRemObj
{
public string PrintString(string str)
{
System.Console.WriteLine(str);
return "Successfully sent a string of length " + str.Length +
" characters to the server.";
}
}

No changes are required for the server.

The Server (Server.exe)

using System;
using System.Runtime.Remoting;
using System.Runtime.Remoting.Channels;
using System.Runtime.Remoting.Channels.Tcp;

public class ClsServer
{
public static void Main()
{
TcpChannel chan = new TcpChannel(8888);
ChannelServices.RegisterChannel(chan);
RemotingConfiguration.RegisterWellKnownServiceType (
Type.GetType("ClsRemObj,RemObj"),"MyRemObj",
WellKnownObjectMode.SingleCall);
System.Console.WriteLine("Press ENTER<- to quit");
System.Console.ReadLine();
}
}

The client will use the interface IRemObj instead of RemObj.

The Client (Client.exe)

using System;
using System.Runtime.Remoting;
using System.Runtime.Remoting.Channels;
using System.Runtime.Remoting.Channels.Tcp;

public class ClsClient
{
public static void Main()
{
TcpChannel chan = new TcpChannel();
ChannelServices.RegisterChannel(chan);
IRemObj remObj = (IRemObj) Activator.GetObject(typeof(IRemObj),
"tcp://localhost:8888/MyRemObj");
Console.WriteLine(remObj.PrintString("Hi, This is Benny here!"));
}
}

Note: You should try to freeze the design of the interface before making the interface available to the client application developers, and avoid any changes to be made to the interface once it is published. Frequent changes to the interface will mean you will need to distribute the changed interface every time, to all the developers using it.

Using SoapSuds.exe

What if the server and the remote objects have already been developed and packaged into one assembly? Does this mean that we will have to include the whole server application at every client for reference? Or will we have to redesign the server application? Thankfully, Microsoft has included a tool called SoapSuds.exe along with the .NET framework SDK, which will solve this problem. All you have to do is run the SoapSuds tool and pass the URI of the running server as the parameter. The SoapSuds tool will extract information from the server and generate metadata as source or as a DLL file, which can be used instead of an interface.

Eg: soapsuds -url:http://<Machine Name>:8888/ClsRemObj -oa:RemObj.dll

This will create an assembly called RemObj.dll, which can be used to reference the remote object. Unfortunately, this tool is useful only if you are using a HTTP channel.

Configuration Files

In our sample applications above, we have hard coded the configuration information, such as channel type, port number, activation type, URL, etc, into the program itself. This is ok for small applications, but huge enterprise level applications will generally need a lot of tweaking over the course of their lifetime, for example changing a port address due to clashes with other applications. For this reason, Microsoft has added another way of specifying the configuration information, externally using an XML file. The benefit of this is that you can change the configuration without recompiling the code. Using external configuration files, the system administrator can change the following:

  • Change the activation type to Client-Activated / Server-Activated
  • Change the mode of Server-Activated objects like SingleCall / Singleton
  • Change the Remote object itself and plug-in a new one
  • Change the location of the server / remote object by changing the URL / URI
  • Change the Channel type to TCP / HTTP etc.
  • Change the port number

This is like akin to changing virtually the whole architecture of the application without recompiling the code. Amazing isn't it!

It is possible to have both machine-level configuration as well as application-level configuration. To configure at the machine level, you need to tweak a file called machine.config that contains all the machine level settings. It can be found at:

%SystemRoot%\Microsoft.NET\Framework\<version x.x.x>\CONFIG

All the pre-defined channels like ' tcp ', ' http ', ' tcp client ', etc. are defined in machine.config. We use the ref attribute in the application configuration files to reference this channel id.

Eg: <channel ref="tcp" port="8888" />

The application-level configuration files are optional, and if they exist, should be located at the root directory of the application.

The convention followed in naming the application level configuration file is ApplicationName.exe.config. So, you append ' .config ' to the executable file name of the application.

Eg: MyClient.exe.config

The configuration file for our sample Server and Client will look like this:

Server.exe.config

<configuration>
<system.runtime.remoting>
<application name="Server">

<service>
<wellknown
mode="SingleCall"
type="ClsRemObj,RemObj"
objectUri="MyRemObj"/>
</service>

<channels>
<channel ref="tcp server" port="8888" />
</channels>

</application>
</system.runtime.remoting>
</configuration>

Client.exe.config

<configuration>
<system.runtime.remoting>

<application name="Client">

<client url="tcp://localhost:8888/Server">
<wellknown
type="ClsRemObj, RemObj"
url ="tcp://localhost:8888/Server/MyRemObj"
/>
</client>

<channels>
<channel ref="tcp client" />
</channels>

</application>
</system.runtime.remoting>
</configuration>

Now the entire configuration code in the client and server program can be replaced with just one line. The remote object class RemObj.dll will remain as it is, with no change.

RemObj.dll

using System;

public class ClsRemObj : MarshalByRefObject
{
public ClsRemObj()
{
Console.WriteLine("Constructor called...");
}

public string PrintString(string str)
{
System.Console.WriteLine(str);
return "Successfully sent a string of length " + str.Length +
" characters to the server.";
}
}

Server.exe

using System;
using
            System.Runtime.Remoting;

public class
            ClsServer
{
public static void
            Main()
{
RemotingConfiguration.Configure("Server.exe.config");
System.Console.WriteLine("Press
            ENTER<- to
            quit");
System.Console.ReadLine();
}
}

Client.exe

using System;
using
            System.Runtime.Remoting;

public class
            ClsClient
{
public static void
            Main()
{
RemotingConfiguration.Configure("Client.exe.config");
ClsRemObj
            remObj = new
            ClsRemObj();

Console.WriteLine(remObj.PrintString("Hi
            There!"));
Console.WriteLine(remObj.PrintString("Helloooo!"));
Console.WriteLine(remObj.PrintString("Benny
            here..."));
}
}

If the configuration files are properly configured, we can simply use the new keyword to instantiate the remote object at the client side just as you would instantiate a local object. The remoting framework will take care of the rest.

Client Activated Remote Objects

Sometimes, creating a lot of remote objects at the server side and waiting for clients to call their methods is not very efficient. What if the clients do not call the remote objects at all? In such situations, it is better to let the client ask the server to activate a remote object when it needs it. Client-activated remote objects are instantiated by the client. This type of activation is also used to hold the state information of the client that activated the remote object. In a client-activated method, a different instance of the remote object is created for each client; and every method call from that client will be received by the same instance of the remote object. They are stateful objects, and as a result, it is possible to maintain state information about the client between method calls.

An ideal application for client activated remote objects would be a shopping cart, where the remote object maintains information about the individual customer and the items the customer has added to the cart across method calls.

To register the remote object to be used in client-activated mode, we have to use the RemotingConfiguration.RegisterActivatedServiceType() method of the remoting framework, instead of the RemotingConfiguration.RegisterWellKnownServiceType().

If you are using the configuration file, you will have to use the <activated> tag instead of the <wellknown> tag. It is not necessary to specify the URI of the remote object. It is also possible to have our own overloaded constructors in our client activated remote object.

Client Activated Remote Object Example

Let us write another sample application, this one will demonstrate client-activated remoting.

The Server (Server.exe)

using System;
using System.Runtime.Remoting;
using System.Runtime.Remoting.Channels;
using System.Runtime.Remoting.Channels.Http;

class Server
{
public static void Main(string[] args)
{
ChannelServices.RegisterChannel(new HttpChannel(8888));
RemotingConfiguration.ApplicationName = "MyApp";
RemotingConfiguration.RegisterActivatedServiceType(typeof(RemObj));

Console.WriteLine("Hit ENTER to quit!");
Console.ReadLine();
}
}

The Client (client.exe)

using System;
using System.Runtime.Remoting;
using System.Runtime.Remoting.Activation;
using System.Runtime.Remoting.Channels;
using System.Runtime.Remoting.Channels.Http;

public class Client
{
public static void Main()
{
ChannelServices.RegisterChannel(new HttpChannel());
object[] atrb = {new UrlAttribute("http://localhost:8888/MyApp")};

//Call the default constructor
ObjectHandle handle = Activator.CreateInstance("RemObj", "RemObj", atrb);
RemObj ro1 = (RemObj) handle.Unwrap();

//Call a Non default constructor by passing 1 parameter-- a string
object[] pmtr1 = new object[1];
pmtr1[0] = "Benny";
RemObj ro2 = (RemObj) Activator.CreateInstance(typeof(RemObj),
pmtr1, atrb);

//Call another overloaded Non default constructor by passing 2
//parameters-- an int and a string
object[] pmtr2 = new object[2];
pmtr2[0] = 222;
pmtr2[1] = "Mathew";
RemObj ro3 = (RemObj) Activator.CreateInstance(typeof(RemObj),
pmtr2, atrb);

ro1.PrintMessage("Call 1 to RemoteObject ro1");
ro2.PrintMessage("Call 1 to RemoteObject ro2");
ro3.PrintMessage("Call 1 to RemoteObject ro3");

ro1.PrintMessage("Call 2 to RemoteObject ro1");
ro2.PrintMessage("Call 2 to RemoteObject ro2");
ro3.PrintMessage("Call 2 to RemoteObject ro3");

}
}

In the client code above, notice that we call two different overloaded versions of Activator.CreateInstance. One which calls the default constructor with no parameters, and another which calls the non-default constructors. The default constructor returns the ObjectHandle, whereas the non-default constructors return an Object. For the non-default constructors, the parameter arguments are passed through an object array, so that any number of arguments can be passed.

The client creates three instances of the remote object, every time invoking different overloaded constructors. The three remote object instances are independent and have their own state. This can be seen by the output of the application. Any changes to the internal state of one instance will not change the others.

The Remote Object (RemObj.dll)

using System;

public class RemObj : MarshalByRefObject
{
private int myInt;
private string myString;

public RemObj()
{
Console.WriteLine("Default constructor called...");
Console.WriteLine();
}

public RemObj(string msg)
{
Console.WriteLine(
"Non default constructor called... by passing the string '{0}'", msg);

myString = msg;
Console.WriteLine();
}

public RemObj(int val, string msg)
{
Console.WriteLine(
"Non default constructor called... by passing an int {0} and a string '{1}'",
val, msg);

myInt = val;
myString = msg;
Console.WriteLine();
}

public void PrintMessage(string message)
{
Console.WriteLine("Message: '{0}'",message);
Console.WriteLine("MyString: '{0}'",myString);
Console.WriteLine("MyInt: {0}",myInt);
Console.WriteLine();
}
}

In the remote object code note that there are total of three constructors defined, two of which are overloaded.

  1. A default constructor, which takes no arguments
  2. A non-default constructor, which takes a string arguments
  3. A non-default constructor, which takes two arguments, a string and an int

Output

Hit ENTER to quit!
Default constructor called...

Non default constructor called... by passing the string 'Benny'

Non default constructor called... by passing an int 222 and a string 'Mathew'

Message: 'Call 1 to RemoteObject ro1'
MyString: ''
MyInt: 0

Message: 'Call 1 to RemoteObject ro2'
MyString: 'Benny'
MyInt: 0

Message: 'Call 1 to RemoteObject ro3'
MyString: 'Mathew'
MyInt: 222

Message: 'Call 2 to RemoteObject ro1'
MyString: ''
MyInt: 0

Message: 'Call 2 to RemoteObject ro2'
MyString: 'Benny'
MyInt: 0

Message: 'Call 2 to RemoteObject ro3'
MyString: 'Mathew'
MyInt: 222

In the output, notice that each instance of the remote object (ro1, ro2, ro3) maintain their own state, (MyString and MyInt). Also notice that the constructors are called only once for each instance of the remote object.

The most common ways of creating and accessing a remote object are:

  • Using the new keyword (Can be used for both server-activated as well as client-activated remote objects)
  • Using the GetObject method of the Activator class (Can be used only for server-activated remote objects)
  • Using the CreateInstance method of the Activator class (Can be used only for client-activated remote objects)
  • Using the Connect() method of the RemotingServices class (Can be used only for server-activated remote objects)

Formatters

When the client calls a method of the remote object, the proxy takes that message and passes it through a formatter which will encode the message into a suitable form (serialized) so that it can be sent out through a channel to the server over a network.

There are three different kinds of formatters available: SOAP, Binary, and Custom, where it's possible to create our own formatter. The SOAP formatter converts the messages into XML, whereas the binary formatter encodes messages into a binary stream. Binary encoding is generally preferred where performance is critical, and where the scope of the application is limited to small networks like the office intranet. SOAP/XML encoding is preferred where compatibility and interoperability is required with other remoting systems that have been developed on disparate architectures. This is also the preferred encoding technique for communication through firewalls, since some firewall configurations will not allow binary streams. By default, TCP channels uses the Binary formatter and the HTTP channel uses the SOAP formatter, but they can be changed.

At the server side, the message that is received is again passed through a formatter that will de-serialize the message and it will be sent to the remote object. If the remote object returns any message back to the client, it has to again go through the same process in the reverse direction.

Sinks and Interceptions

.NET Remoting was built from the ground up, with security in mind, and as a result, it is possible to monitor and intercept messages at every stage of the remoting process. The messages that are serialized and sent across the network to the server can be intercepted at different stages using sinks. Sinks are connected to sink chains. There are different kind of sinks like formatter sinks, channel sinks and transport sinks. These "hooks" are useful to monitor the messages at different stages of transportation. For example it is possible to provide our own sink, which will add headers to the messages that are sent out of the application domain. Sinks can also be used to log messages, add more security, redirect messages, etc.

Lifetime & Leasing

We know that in the SingleCall activation method, the remote object is created afresh with every method call to the remote object, and once this method has finished executing, the remote object is marked for garbage collection. This is different than in the Singleton and Client activated methods, where the remote object needs to remain activated for a long period of time, as long as the client requires it.

But how will the server know when the remote object is no longer required? You might want to code the remote object to remain alive until the client tells it that it's no longer needed. But what if the client suddenly crashes, and was not able to let the server know that it no longer needs the remote object? Will the remote object remain alive forever? The technique used by Microsoft DCOM technology is that the client keeps pinging the server and notifies it of its existence, but this is not very efficient and does not scale well for the Internet environment.

The technique used by the .NET remoting framework is to assign a predefined lifespan to every remote object. If no methods were called on the remote object before the time allotted expires, the remote object will be marked for garbage collection. If, during the initial lifespan, a method is invoked on the remote object, the lifetime of the remote object gets extended by another predefined value. Every method call will extend the lifetime by the predefined value, which can be set by the developer.

Note: Even if multiple methods get invoked, the lifetime of the remote object gets extended by only one predefined unit of time. The next unit is added only when a method is invoked while it is on the last unit of time.

Note: With singleton, it is ensured that there will be only one object in memory at a time, but there is still the possibility that the object may be destroyed and a new object will be created. To keep that same remote object for an infinite period we have to override the initializeLifetimeService method in our remote object and return null.

Sponsors

In the previous method, what if a client calls the remote object's method a bit too late, say just after the lifetime has expired? A remoting exception will occur. To avoid this situation we can use another way of extending the lifetime of the remote object (in addition to the method mentioned above) by using something known as sponsors. Sponsors are used when we are not sure about the frequency of method calls. Sponsor objects are ideally created on the client side and registered with the remote object. When the remote object is about to expire, the lease manager tries to contact any available sponsors and if it finds one, asks the sponsor if it wants to extend the remote objects lifespan.

Marshaling

The parameter types of remote method calls can also include classes, i.e. objects that we define, not just basic data types. The passing of parameter objects and return values between the client and the server is known as marshaling. The process by which the marshaled data is converted back to an object is known as Unmarshaling.

The parameters can be marshaled to the remote object in one of two ways:

  • Marshal by Value (MBV)
  • Marshal by reference (MBR)

The remoting framework will decide whether the object has to be marshaled by value or marshaled by reference, depending on the type of object.

Marshal by Value

When you pass an object by value to the remote method, this object is first serialized into a stream and then transported to the server, where it is unmarshalled and passed to the remote object method. To marshal an object by value as a parameter, the object has to be marked with a [serializable] attribute, or the class should implement the Iserializable interface.

Marshal by Reference

When you pass an object by reference to the client, a proxy is created at the client side, and all the calls made on it is passed to the actual object at the server.

To pass an object by reference, the object should be extended from MarshalByRefObject.

The framework will internally use a static method RemotingServices.Marshal() to marshal this object. The RemotingServices.Marshal() method will return an object of type ObjRef, which is serializable and this is sent to the remote location where it is used to create a proxy using the RemotingServices.Unmarshal() method.

Let us now write a small application to demonstrate MBV and MBR.

In this sample application, the remote object passes two objects to the client: one is an MBR object and the other is an MBV object. The client executes a method on each of these objects. These methods write out a string to the console.

Note that the method of the object that is passed to the client by value writes the string at the client console window, and the method of the object that is passed to the client by reference writes the string at the server console window. This clearly shows that the MBV object is available and executing at the client, whereas the MBR object is executing at the server through a proxy which is at the client side.

An Example

Remote Object (RemObj.dll)

using System;

public class ClsRemObj
            : MarshalByRefObject
{
public ClsRemObj()
{
Console.WriteLine("ClsRemObj Constructor
            called...");
}

public MyMBVClass
            GetMyMBVClass()
{
return new MyMBVClass();
}

public
            MyMBRClass GetMyMBRClass()
{
return new
            MyMBRClass();
}
}

[Serializable]
public class
            MyMBVClass
{
public
            MyMBVClass()
{
Console.WriteLine("MyMBVClass Constructor
            called...");
}

public void
            MyMBVMethod()
{
Console.WriteLine("I am now inside
            MyMBVMethod");
}
}

public class MyMBRClass :
            MarshalByRefObject
{
public
            MyMBRClass()
{
Console.WriteLine("MyMBRClass Constructor
            called...");
}

public void
            MyMBRMethod()
{
Console.WriteLine("I am now inside
            MyMBRMethod");
}
}

Server (Server.exe)

using System;
using System.Runtime.Remoting;
using System.Runtime.Remoting.Channels;
using System.Runtime.Remoting.Channels.Tcp;

public class ClsServer
{
public static void Main()
{
TcpChannel chan = new TcpChannel(8888);
ChannelServices.RegisterChannel(chan);
RemotingConfiguration.ApplicationName = "MyMarshalingApp";
RemotingConfiguration.RegisterActivatedServiceType(typeof(ClsRemObj));

System.Console.WriteLine("Press ENTER<- to quit");
System.Console.ReadLine();
}
}

Client (Client.exe)

using System;
using
            System.Runtime.Remoting;
using
            System.Runtime.Remoting.Activation;
using
            System.Runtime.Remoting.Channels;
using
            System.Runtime.Remoting.Channels.Tcp;

public class
            ClsClient
{
public static void Main()
{
TcpChannel chan
            = new
            TcpChannel();
ChannelServices.RegisterChannel(chan);
object[]
            atrb = {new
            UrlAttribute(
"tcp://localhost:8888/MyMarshalingApp")};

ObjectHandle
            handle = Activator.CreateInstance(
"RemObj", "ClsRemObj",
            atrb);
ClsRemObj remObj = (ClsRemObj)
            handle.Unwrap();

MyMBVClass objMBV =
            remObj.GetMyMBVClass();
MyMBRClass objMBR =
            remObj.GetMyMBRClass();

objMBV.MyMBVMethod();
objMBR.MyMBRMethod();
}
}

Server Console Window Output

ClsRemObj Constructor called...
MyMBVClass Constructor called...
MyMBRClass Constructor called...
I am now inside MyMBRMethod

Client Console Window Output

I am now inside MyMBVMethod

Care has to be taken while deciding if a particular object has to be marshaled by value or reference. If the object is small and will be called often, it is better to pass the object by value, so that the client has an independent copy of the object and it can call its methods any number of times without making a call over the network.

However, there are some situations where the object has to be passed by reference. For instance, if the object passed to the client has to make use of resources on the server like accessing the database, or the file system, etc. then we have no choice but to pass it by reference.

On the other hand, if the object to be passed to the client is huge and takes up a lot of bandwidth, if passed by value, then it is better to marshal it by reference.

Tracking Services

The objects that are marshaled across the client and the server can be tracked by using the TrackingServices class. This is very simple to do, all that needs to be done is to write a class that will implement the ITrackingHandler interface, and register this object with the TrackingServices class.

The ITrackingHandler interface defines three methods:

  • MarshaledObject - gets called when an object is marshaled
  • UnmarshaledObject - gets called when an object is unmarshaled
  • DisconnectedObject - gets called when an object is disconnected

When the MBV or MBR objects are passed to and from the remote object, the marshalling and unmarshalling of object occurs. When this occurs, the UnmarshaledObject and the MarshaledObject methods defined in the class that implements the ITrackingHandler interface gets triggered.

Let us extend the previous example to demonstrate tracking services. First we shall write the Tracking Class that implements the ITrackingHandler interface as a separate library (dll).

MyTrackingService.dll

using System;
using System.Runtime.Remoting;
using System.Runtime.Remoting.Services;

public class ClsMyTrackingService: ITrackingHandler
{
public void MarshaledObject(object obj, ObjRef or)
{
Console.WriteLine(
"I am now inside ClsMyTrackingService.MarshaledObject()");
}
public void UnmarshaledObject(object obj, ObjRef or)
{
Console.WriteLine(
"I am now inside ClsMyTrackingService.UnmarshaledObject()");
}

public void DisconnectedObject(object obj)
{
Console.WriteLine(
"I am now inside ClsMyTrackingService.DisconnectedObject()");
}
}

Add the following code to the Client and the Server to track marshaling and unmarshaling at both sides.

TrackingServices.RegisterTrackingHandler(new ClsMyTrackingService());

Note you have to import the System.Runtime.Remoting.Services namespace into the client and the server source code to use the RegisterTrackingHandler method of the TrackingServices class.

Server Console Window Output

ClsRemObj Constructor called...
I am now inside ClsMyTrackingService.MarshaledObject()
I am now inside ClsMyTrackingService.MarshaledObject()
MyMBVClass Constructor called...
MyMBRClass Constructor called...
I am now inside ClsMyTrackingService.MarshaledObject()
I am now inside MyMBRMethod

Client Console Window Output

I am now inside ClsMyTrackingService.UnmarshaledObject()
I am now inside MyMBVMethod

Asynchronous Remoting, Callbacks and Delegates

So far we have only talked about synchronous remoting, that is, the client calls a method on the remote object hosted on a server, which can be on a different machine, and waits for the method to finish before it can continue executing the rest of the code. The call on the remote object's method can sometimes take a long time to complete, especially if the call has to go over a network. In some situations, this delay may not be acceptable. In such situations we can make use of the asynchronous method calling facility provided by the .NET framework.

When you call a method asynchronously, the calling thread will spawn a new thread which will do the calling and waiting for the method to complete. The original thread will continue its normal execution path. You can call methods asynchronously on both local objects as well as remote objects.

There are three ways you can find out if an asynchronous method call has finished:

  1. You can call the remote method and forget about it and continue doing other things. When the remote method completes, it will automatically callback and notify the client.
  2. After calling the remote object's method, you can keep polling to see if the remote method has finished executing.
  3. Use the WaitOne() method of the AsyncWaitHandle of the IAsyncResult interface, which is a blocking call and will wait until the remote method completes.

Before we go into the details of how asynchronous method calls work, we will have to thoroughly understand another mechanism provided by .NET known as delegates.

Delegates

Delegates are exactly what their name implies. They are objects that delegate the calls made on them to the actual methods that are registered with them. A small code example will clearly illustrate their use.

using System;

class
            DelegateTest
{
public delegate void
            MyDelegate();

public static void Main()
{
MyDelegate
            d;

d = new MyDelegate(Method1);
Console.WriteLine("Calling
            delegate...");
d();

d += new
            MyDelegate(Method2);
Console.WriteLine("Calling
            delegate...");
d();

d -= new
            MyDelegate(Method1);
Console.WriteLine("Calling
            delegate...");
d();
}

public static void
            Method1()
{
Console.WriteLine("Method1
            Called");
}

public static void
            Method2()
{
Console.WriteLine("Method2 Called");
}
}

In the above code you see that we have declared a delegate of type MyDelegate using the delegate keyword. Next, we create an instance of MyDelegate, d, by passing to the constructor the name of the method to which it has to delegate the call.

Now when you call the delegate d(), it passes the call to Method1, and you see the following message printed on the console window.

Calling delegate...
Method1 Called

It is possible to do a lot of neat things with delegates, like registering multiple methods on the same delegate, but remember that the signature (parameter types and return value type) of all the methods to be registered must match the signature of the delegate.

Next, the program adds and registers another method to the same delegate d, so that one call to the delegate will call both the methods.

Now when you call the delegate d, you see that both the methods have been called:

Calling delegate...
Method1 Called
Method2 Called

It is also possible to selectively remove and un-register methods from a delegate as shown in the code.

After we remove the call to the first method and call the delegate, you see that only Method2 is getting called.

Calling delegate...
Method2 Called

Asynchronous Method Calling

Let us write a small application to demonstrate how asynchronous method calling works, and then we'll further extend this concept to use with remote objects.

Asnchronous Example

using System;
using
            System.Threading;
public class AsyncExample
{
private
            delegate void MyDelegate();
static void Main(string[]
            args)
{
Console.WriteLine("Main Thread
            Started...");

AsyncExample AnObj = new
            AsyncExample();
MyDelegate md = new
            MyDelegate(AnObj.MyMethod);
IAsyncResult result =
            md.BeginInvoke(null,
            null);

Thread.Sleep(2000);
Console.WriteLine("Calling from
            Main 1");
Thread.Sleep(2000);
Console.WriteLine("Calling from
            Main 2");
Thread.Sleep(2000);
Console.WriteLine("Calling from
            Main 3");
Thread.Sleep(2000);
Console.WriteLine("Calling from
            Main 4");
Thread.Sleep(2000);
Console.WriteLine("Calling from
            Main 5");
Thread.Sleep(2000);
Console.WriteLine("Calling from
            Main 6");

result.AsyncWaitHandle.WaitOne();
if
            (result.IsCompleted)
            md.EndInvoke(result);
Console.WriteLine("Main Thread
            Ending...");
}

public void
            MyMethod()
{
Console.WriteLine("MyMethod Thread
            Starting...");
Thread.Sleep(3000);
Console.WriteLine("Calling
            from MyMethod
            1");
Thread.Sleep(3000);
Console.WriteLine("Calling from
            MyMethod 2");
Thread.Sleep(3000);
Console.WriteLine("Calling
            from MyMethod
            3");
Thread.Sleep(3000);
Console.WriteLine("Calling from
            MyMethod 4");
Thread.Sleep(3000);
Console.WriteLine("Calling
            from MyMethod
            5");
Thread.Sleep(3000);
Console.WriteLine("Calling from
            MyMethod 6");
Console.WriteLine("MyMethod Thread
            Ending...");
}
}

Output

Main Thread Started...
MyMethod Thread Starting...
Calling from Main 1
Calling from MyMethod 1
Calling from Main 2
Calling from Main 3
Calling from MyMethod 2
Calling from Main 4
Calling from MyMethod 3
Calling from Main 5
Calling from Main 6
Calling from MyMethod 4
Calling from MyMethod 5
Calling from MyMethod 6
MyMethod Thread Ending...
Main Thread Ending...

In the above code, to call the method MyMethod asynchronously, we do not call it directly, but create an instance of MyDelegate object by passing the name of the method MyMethod, and then we have to call the BeginInvoke method of our MyDelegate object md.

Note: The main method is a dedicated thread that doesn't require thread.start. It is started when the application is executed. When the main thread dies so does all the worker threads, as in MyMethod.

You can see from the output that both the Main method and the MyMethod are running concurrently.

At compile time, the compiler adds three methods, Invoke(), BeginInvoke() and EndInvoke() to the delegate object.

The Invoke() method is used to call the delegate synchronously, while the BeginInvoke() method is used to call the delegate asynchronously. The EndInvoke() method is called to retrieve the return value.

When we call the BeginInvoke() method it returns an IAsyncResult interface, which we can use to determine the status of the asynchronous method call. In the example above, we use the WaitOne() method of the AsyncWaitHandle property of the IAsyncResult interface, which is a blocking call, and will wait until the method call completes.

Apart from the signature of the delegate, we need to pass two more parameters to the BeginInvoke() method (in the above example we passed null for both of them). One of them is an AsyncCallback delegate object that will call back a method once the asynchronous call completes. This callback method can be used to do whatever needs to be done once the call completes, instead of waiting and polling for the method to complete.

The method that will be called after the completion of the asynchronous call needs to have the same signature as that of the AsyncCallback. The AsyncCallback delegate has just one parameter, which is an object of type IAsyncResult.

The EndInvoke() method is called on the delegate object to retrieve the return value of the asynchronous method call.

Note: EndInvoke() method has nothing to do with ending the asynchronous method call; it is used only to fetch the return value of the asynchronous method call once the call completes.

Asynchronous Remoting

Let us use the concepts of delegates, asynchronous method calling and callbacks that we just learned by extending our first example.

The Server Object (Server.exe)

using System;
using
            System.Runtime.Remoting;
using
            System.Runtime.Remoting.Channels;
using
            System.Runtime.Remoting.Channels.Tcp;

public class
            ClsServer
{
public static void Main()
{
TcpChannel chan
            = new
            TcpChannel(8888);
ChannelServices.RegisterChannel(chan);

RemotingConfiguration.RegisterWellKnownServiceType(
Type.GetType("ClsRemObj,RemObj"),"MyRemObj",
WellKnownObjectMode.Singleton);

System.Console.WriteLine("Press
            ENTER<- to
            quit");
System.Console.ReadLine();
}
}

The Remote Object (RemObj.dll)

using System;

public class ClsRemObj : MarshalByRefObject
{
public string PrintString(string str)
{
System.Console.WriteLine(str);

return "Successfully sent a string of length " + str.Length +
" characters to the server.";
}
}

Nothing needs to be changed on the remote object and the server, which hosts the remote object.

using System;
using
            System.Threading;
using System.Runtime.Remoting;
using
            System.Runtime.Remoting.Channels;
using
            System.Runtime.Remoting.Channels.Tcp;

public class
            ClsClient
{
private delegate string PrintStringDelegate(string
            str);
private static PrintStringDelegate psd;

public
            static void Main()
{
TcpChannel chan = new
            TcpChannel();
ChannelServices.RegisterChannel(chan);
ClsRemObj
            remObj = (ClsRemObj)
            Activator.GetObject(typeof(ClsRemObj),
"tcp://localhost:8888/MyRemObj");

psd
            = new PrintStringDelegate(remObj.PrintString);
AsyncCallback acb
            = new AsyncCallback(PrintStringCallback);

psd.BeginInvoke("Hi
            There!", acb, null);
psd.BeginInvoke("Helloooo!", acb,
            null);
psd.BeginInvoke("Benny here...", acb,
            null);

Thread.Sleep(5000);
}

public static void
            PrintStringCallback(IAsyncResult res)
{
if
            (res.IsCompleted)
{
string RetValFromRemObj =
            psd.EndInvoke(res);
Console.WriteLine(RetValFromRemObj);
}
}
}

In the client code, first we declare a delegate PrintStringDelegate with the same signature as the remote object's method PrintString() that we need to call asynchronously.

Next, we will create an object of this delegate type and pass the remote object's method PrintString() to the constructor.

We will also create a delegate object acb of type AsyncCallback and pass the method name PrintStringCallback (the method which needs to be called back once the call finishes) to the constructor.

Now all you have to do is call the BeginInvoke() method of the PrintStringDelagate psd, with the required parameters. One of the parameters is the delegate object acb of type AsyncCallback, which will be used by the remoting framework to call back the method PrintStringCallback once the remote call completes.

Server Console Ouput

Press ENTER<- to quit
Hi There!
Helloooo!
Benny here...

The Client window should display the following string sent back by the server as an acknowledgement:

Client Console Output

Successfully sent a string of length 9 characters to the server.
Successfully sent a string of length 9 characters to the server.
Successfully sent a string of length 13 characters to the server.

Call Context

We know that server-activated remote objects cannot retain information about the clients across method calls. We will have to pass all the required information as parameters to the method every time. Microsoft has made things a little easier if you have to pass certain information every time to a method - you can bundle that information into a class and register it with the CallContext (System.Runtime.Remoting.Messaging.CallContext) and it will take care of passing it to the remote object every time you call the methods on it.

An Example

Let me illustrate the point by giving you a simple example. Imagine that we are building a distributed payroll application, and we have to pass the basic employee information like name, employee code, designation and department to every method of ProcessPayroll (remote object). In such situations, it is better and more elegant to register the basic employee information once with the CallContext, so that it will automatically pass this information to the remote object's ApplyLeave method every time this method is called.

In the example below we will see that the client calls the ApplyLeave method with just the number of days as the parameter, but the remote object at the server side retrieves other basic information about the employee and prints them on the console window along with the leave amount.

We will create a class in a new class library, which will hold all the information that has to be passed every time. This class library has to be available both at the server side and the client side, since the remote object as well as the client references this at runtime.

EmpInfo (EmpInfo.dll)

using System;
using
            System.Runtime.Remoting.Messaging;

[Serializable]
public
            class EmpInfo : ILogicalThreadAffinative
{
public
            EmpInfo()
{
}

private string name;
public string
            Name
{
get
{
return name;
}

set
{
name =
            value;
}
}

private string empCode;
public string
            EmpCode
{
get
{
return
            empCode;
}

set
{
empCode =
            value;
}
}

private string designation;
public string
            Designation
{
get
{
return
            designation;
}

set
{
designation =
            value;
}
}

private string department;
public string
            Department
{
get
{
return
            department;
}

set
{
department =
            value;
}
}
}

There are two requirements for defining this class:

  • This class has to implement the ILogicalThreadAffinative interface
  • It has to be marked with the [Serializable] attribute so that it can be marshaled across application domains.

The ILogicalThreadAffinative interface does not define any methods. It is a markup interface.

The client code looks like this:

The Client (Client.exe)

using System;
using
            System.Runtime.Remoting;
using
            System.Runtime.Remoting.Messaging;
using
            System.Runtime.Remoting.Channels;
using
            System.Runtime.Remoting.Channels.Tcp;

public class
            ClsClient
{
public static void Main()
{
TcpChannel chan
            = new
            TcpChannel();
ChannelServices.RegisterChannel(chan);
ClsRemObj
            remObj = (ClsRemObj)
            Activator.GetObject(typeof(ClsRemObj),
"tcp://localhost:8888/MyRemObj");

EmpInfo
            e = new EmpInfo();
e.Name = "Benny Mathew";
e.EmpCode =
            "bkm345";
e.Designation = "Programmer";
e.Department =
            "Software Development";
CallContext.SetData("EmployeeInfo",
            e);

remObj.ApplyLeave(5);
}
}

Notice that in the highlighted code above, we create an EmpInfo object and store all the information that has to be passed every time to the remote object and register it with the CallContext under the name Employeeinfo. The remote object at the server side to retrieve the data from the CallContext will use this name.

The Remote Object (RemObj.dll)

using System;
using System.Runtime.Remoting.Messaging;

public class ClsRemObj : MarshalByRefObject
{
public void ApplyLeave(int leave)
{
EmpInfo e = (EmpInfo)CallContext.GetData("EmployeeInfo");
if (e != null)
{
Console.WriteLine("Name: {0}",e.Name);
Console.WriteLine("Employee Code: {0}",e.EmpCode);
Console.WriteLine("Designation: {0}",e.Designation);
Console.WriteLine("Department: {0}",e.Department);
}
Console.WriteLine("Leave Applied: {0}",leave);
}
}

The Server (Server.exe)

using System;
using System.Runtime.Remoting;
using System.Runtime.Remoting.Channels;
using System.Runtime.Remoting.Channels.Tcp;

public class ClsServer
{
public static void Main()
{
TcpChannel chan = new TcpChannel(8888);
ChannelServices.RegisterChannel(chan);
RemotingConfiguration.RegisterWellKnownServiceType(
Type.GetType("ClsRemObj,RemObj"),"MyRemObj",WellKnownObjectMode.SingleCall);

System.Console.WriteLine("Press ENTER<- to quit");
System.Console.ReadLine();
}
}

The output of the program will look like this:

Name: Benny Mathew
Employee Code: bkm345
Designation: Programmer
Department: Software Development
Leave Applied: 5

Summary

.NET Remoting is much faster when compared to SOAP/XML based Web Services. It is also much easier to develop distributed applications when compared to Java's RMI, since most of the complexities are taken care of by the underlying remoting framework.

Case Study Information
Author | Benny Mathew
Editor | Adam Ryland
Reviewers | Peter Ehli, David Williams, Kunal Cheda

If you have any questions or comments about this case study, or suggestions for further pieces, please contact the technical editor. If you wish to talk about the case study with other subscribers, please visit our forum here.


Mail Tracking and Analysis System

Mail Tracking and Analysis System
http://www.csharptoday.com/content.asp?id=1845
Article Rating: 4.5
Published on June 10, 2002
Abstract
One of the easiest, fastest and most reliable ways of communication is through e-mail. Many people carry out business with the help of e-mail, as e-mail is one of the primary means of communication in eBusiness. In this case study, Vinod Kumar looks at a Mail Client software solution that uses XML Webservices to store mails which have been sent or received in a centralized database for future analysis. A mail analyzing solution is also discussed.
Case Study

The easiest, fastest and most reliable method of communication is via e-mail. Many people carry out business with the help of e-mail. This kind of business communication can be classified into two main categories:

  • Inter-office communication, which is the communication within an organization between various departments. In this situation, the organization uses its own mail server for communication, or can use an external one.
  • Communication with external clients, where there is little control over how the user sends\deals with mails.

In the latter scenario, the tracking of the mails sent and received is tedious and difficult for to the following reasons:

  • Each client may use a different mail server for the purpose of sending and receiving mails; it would be difficult or highly impossible to keep track of the mails.
  • Suppose if each of them have their own mail account information, it would be difficult to get all the information and perform analysis.

To solve this kind of problems, we develop mail client software, called WebMail, which stores the mailing information sent and received in a centralized database. This stored information is used for mail analysis, which is discussed in the later part of this case study.

System Requirement

Before continuing, please take note of the following system requirements for the purposes of this discussion, as well as for working with the sample code accompanying the case study:

  • Windows 2000 or XP with IIS
  • . Net framework
  • Visual Studio .NET RTM
  • SQL Server 7.0 or 2000

We can use any external mail server that supports SMTP and POP for sending and receiving the mails using this tool. This case study requires some familiarity with the Microsoft .Net Framework, C#, and XML. Familiarity with SMTP and POP protocols is not necessary, but would be helpful. For a good background on these topics, please refer to:

POP3 standards

RFC 1939

ftp://ftp.isi.edu/in-notes/rfc1939.txt?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq

RFC1730

ftp://ftp.isi.edu/in-notes/rfc1730.txt?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq

A POP3 Client and C#

http://www.csharptoday.com/content.asp?id=1681&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq

TcpClientClass

http://msdn.microsoft.com/library/en-us/cpref/html/frlrfSystemNetSocketsTcpClientClassTopic.asp?frame=true&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq

NetworkStreamClass

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/cpref/html/frlrfSystemNetSocketsNetworkStreamClassTopic.asp&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq

EncodingClass

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/cpref/html/frlrfsystemtextencodingclasstopic.asp&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq

The WebMail System

Now let’s see how the WebMail works. This software consists of two different applications, namely a windows (WebMailClient) application and a Web Service (WebMailWS). The Windows application is used to send and receive the mails using any mail server that supports SMTP and POP protocol. The WebMailClient application also helps to update the mail sent and received information to a remote database using the WebMailWS Web service after storing the information in the client place in an XML format. Since a remote database can be situated anywhere in the world, we use WebServer for storing the Mail Information in a heterogeneous network. Once the information is stored, it can be used for mail analysis. Since we are using XML to store a copy of the mail information in the client location before storing it in the remote location, a client can get access to the mail information without a connection to the remote database.

Each time a mail is sent using this software, a unique identity number is generated and is appended to the subject (the WebMailClient tool hides the Unique ID from the user) along with the reference MailID (if any) and department code. This unique number is used for further references to the mail. The ClientWebMail application receives and stores only the mails that have this unique ID.

Database Design and Structure

As we see from the architecture of the system, two data stores are used to store and provide information for our application. An XML data source is used to store the information locally in the client, whereas an SQL Server database is used to store the information in the remote location.

Mail Tracking and Analysis System’s Database design

The database structure consists of five tables. Here is a list of tables that are added to the database.

The AccessInfo table sets the Access privilege for a user (through EMailID) to access others email information. This stores the EmailID, which is mapped to the AccessID.

The Department table stores the Department code, along with the name.

The AddressBook table stores the EmailID and password, along with the department code and AccessID.

The Send Mail table stores sent Mail information like Mail ID, DeptCode, Reference ID, Sender Email ID, sent date, subject, and message.

The ReceivedMailInfo table stores the mail receivers information along with the mail’s received date and Mail Read status.

The diagram below shows the table relationship:

Business Processes

The scope of this case study is to show how to manage the mails sent and received between a organization, which includes storing the mail information in a centralized database. The case study is split into three parts for better understanding.

In the first part, we build the Mail component, which helps us send and receive mails. The SendMail class sends mails using SMTP standard, and the POP3Client Class helps to receive mails, which supports the POP protocol.

In the second part, we discuss the web service (WebMailWS) that is used to perform login authentication and update the remote database with the sent and received mail’s information.

In the third part we build a client WinForm application (WebMailClient), which act as an interface to send and receive mails and also helps in analyzing the mails.

Component Design

The Mail component solution has four components. One is used to send mail, and the other three are used to receive mail.

The SendMail Class

Let’s examine the SendMail class that is used for sending mails. This class contains a set of private variables that are used to hold mail information. Some of these variables are exposed in the form of properties, since it allows us to exercise a greater degree of control over the values that are assigned to the variables. The other variables like m_Date and m_MessageID are used to set the sent date of the mail and the unique mail Message ID. m_SMTP is set to the TCPClient class, which is present in the System.Net.Sockets namespace. The TcpClient is used to establish a connection with the remote host. Application-level protocols such as FTP and HTTP are built on this class, which in turn is built upon the Socket class to provide TCP services at a higher level of abstraction.

The diagram below shows all the members in SendMail class:

public class
            SendMail
{
protected string m_MessageID=null;
protected
            string m_SMTPServer=null;
protected string
            m_From=null;
protected string m_FromName=null;
protected
            string m_To=null;
protected string m_Subject=null;
protected
            string m_Body=null;
protected string m_Date=
            DateTime.Today.ToString ();
protected TcpClient m_SMTP = null;
public SendMail()
{
}
//property to get and set From
            EmailID
public string From
{
get
{
return
            m_From;
}
set
{
m_From = value;
}
}
//property
            to get and set From Name
public string
            FromName
{
get
{
return
            m_FromName;
}
set
{
m_FromName =
            value;

}
}
//property to get and set To
            EmailID's
public string To
{
get
{
return
            m_To;
}
set
{
m_To = value;

}
}
//property
            to get and set Subject of the Mail
public string
            Subject
{
get
{
return
            m_Subject;
}
set
{
m_Subject =
            value;

}
}
//property to get and set Body of the
            Mail
public string Body
{
get
{
return
            m_Body;
}
set
{
m_Body =
            value;

}
}
//property to get and set SMTP
            ServerName
public string SMTPServer
{
get
{
return
            m_SMTPServer;
}
set
{
m_SMTPServer =
            value;

}
}

The Send method is used to send mails to the remote mail server. This method establishes a connection to the remote server using the TCPClient class, which returns a NetworkStream object that is used to read and write data to the remote host. This method uses two other methods, namely IsValidReply and SendMessage, to read and write data to the remote server. The IsValidReply method is used to read and validate the data as commands sent from the remote server. SendMessage sends the message as a command to remote server:

public string Send()
{
try
{
m_SMTP= new TcpClient(m_SMTPServer,25);
NetworkStream strm =
            m_SMTP.GetStream();
string reply = '';
// Server must replay
            220 - Server OK
reply = IsValidReply(strm);
if(reply !=
            'OK')
{
return reply;
}
// Send Hello to
            server
SendMessage('EHLO ' + m_SMTPServer +
            '\r\n',strm);
reply = IsValidReply(strm);
if(reply !=
            'OK')
{
return reply;
}
// Send From Mail ID to the
            server
SendMessage('MAIL FROM: <' + m_From +
            '>\r\n',strm);
reply = IsValidReply(strm);
if(reply !=
            'OK')
{
return reply;
}
// sending TO mailID to
            server
//Regular expression is used to split the
//to emailid
            which are seperated by comma delimiter
Regex RgxMail=new
            Regex(',');
string[] strMailSplit =
            RgxMail.Split(m_To);
foreach(string t in strMailSplit)
{
SendMessage('RCPT TO: <' + t + '>\r\n',strm);
reply =
            IsValidReply(strm);
if(reply != 'OK')
{
return
            reply;
}
}
SendMessage('DATA \r\n',strm);
reply =
            IsValidReply(strm);
if(reply != 'OK')
{
return reply;
}

The code below is used to send the message body data to the remote server, and then disconnects. This message body is built using a function buildMessageBody.

// Sending Message Body
byte[] bMime
            =
            Encoding.Default.GetBytes(buildMessageBody());
strm.Write(bMime,0,bMime.Length);
//
            Notify End of Data
SendMessage('\r\n.\r\n',strm);
reply =
            IsValidReply(strm);
if(reply != 'OK')
{
return reply;
}
// disconnecting from the
            Server
SendMessage('QUIT\r\n',strm);
return
            'OK';
}
catch(Exception x)
{
return 'ERR ' +
            x.Message;
}
}

Note: '\r\n” should be appended to each command sent. This is because SMTP specification indicates that a CRLF pair terminates all commands. SendMessage has two arguments, Data (of type string) and Stream (of type NetworkStream). Stream is a reference type variable, and Data is a value type variable. The Data argument receives SMTP commands and returns values such as 250, 220, and 354 and so on (signifying if a particular command sent to the server is carried out correctly or has failed). Before writing to the Network stream, the string has to be converted to byte format. To achieve this conversion, we use the Encoding.ASCII property of the Encoding class (System.Text Namespace) and the GetBytes function, which returns bytes out of characters.

private void SendMessage(string data,NetworkStream stream)
{
byte[] byteData = Encoding.ASCII.GetBytes(data.ToCharArray());
stream.Write(byteData,0, byteData.Length);
}

The IsValidReply method is used to read and validate the data as commands sent from the remote server. This method uses stream reader to read the data. This data, which is in byte format, is converted to a string type using the Encoding.ASCII property of the Encoding class and the GetString function. A waiting mechanism is deployed, which waits for the remote server to reply within a specific time frame before throwing an error code as a return value.

private string
            IsValidReply(NetworkStream strm)
{
string serverReply =
            'ERR';
int counter = 0;
while(true)
{
//checking for
            dada avaliability
if(strm.DataAvailable)
{
//setting byte
            size
byte[] buffer = new byte[1024];
int bytes =
            strm.Read(buffer,0,1024);
serverReply =
            Encoding.ASCII.GetString(buffer,0,bytes);
serverReply.Trim();
serverReply
            =
            serverReply.Substring(0,serverReply.Length-2);
break;
}
//waiting
            for the server replay
Thread.Sleep(300);
counter++;
if(counter > 50)
{
break;
}
}
// 250 -
            Command was Sucessful, 354 - Start inputing Data, 220 - Server is
            ready
if(serverReply.Substring(0,3) == '250' ||
            serverReply.Substring(0,3) == '354' || serverReply.Substring(0,3) ==
            '220')
{
return 'OK';
}
else
{
return
            serverReply;
}
}

The buildMessageBody() method is used to build the mail information before sending it using the SendMessage method. This method uses the BuildBody and SplitTo functions to build the mail message:

private string buildMessageBody()
{
m_MessageID = Guid.NewGuid().ToString();
string
            MailBuilder = '';
MailBuilder += 'From: \'' + m_FromName + '\'
            <' + m_From + '>\n';
MailBuilder += 'To: ' + SplitTo() +
            '\n';
MailBuilder += 'Subject: ' + m_Subject +
            '\n';
MailBuilder += 'Date: ' + m_Date + '\n';
MailBuilder +=
            'MIME-Version: 1.0\n';
MailBuilder += 'Content-Type:
            multipart/mixed;\n';
MailBuilder +=
            '\tboundary=\'----=_NextPart_' + m_MessageID +
            '\'\n';
MailBuilder += BuildBody();
return
            MailBuilder;
}

The BuildBody function is used to build the body of the message, and the SplitTo function is used to split and format the receiver’s e-mail in POP standard, for sending data using the Regular Expression RegEx class (in the System.Text.RegularExpression namespace). A comma delimiter separates the EMailID.

private string BuildBody()
{
string retVal=””;
retVal += '------=_NextPart_' + m_MessageID
            + '\n';
retVal += '\tcharset=\'iso-8859-1\'\n';
retVal +=
            'Content-Transfer-Encoding: 8bit\n\n';
retVal += 'STARTBODY\n\n';
retVal += m_Body + '\n';
retVal += 'ENDBODY\n\n';
retVal
            += '\n';
return retVal;
}
private string SplitTo()
{
string retVal = '';
Regex Regex=new Regex(',');
string[]
            strSplitParts = Regex.Split(m_To);
foreach(string t in
            strSplitParts)
{
retVal += '<' + t + '>';
if(i <
            strSplitParts.Length)
{
retVal +=
            ',\n\t';
}
else
{
retVal += '\n';
}
}
return
            retVal;
}

Now let’s look in more detail at the three classes that are used for receiving mails from the mail server.

POP3Client Class

The POP3Client class is used to send and receive POP3 commands to/from the POP3 email server. Using the POP3Client class, the user can connect to the POP3 server, read messages, and so on.

The diagram below show all the members in the POP3Client class:

This class has a set of private variables that are used to hold POP server information. Let’s look at some of these variables in the form of properties. A variable to hold the POP3MessagesInfo class instance is also declared.

public class Pop3Client
{
private
            TcpClient m_pPop3_Client = null;
private string m_Pop3Server =
            '';
private string m_UserName = '';
private string m_Password
            = '';
private bool m_LoggedIn = false;
private
            Pop3MessagesInfo m_MessagesInfo = null;
public string Pop3Server
{
get{ return m_Pop3Server; }
}
public string
            UserName
{
get{ return m_UserName; }
}
public string
            Password
{
get{ return m_Password; }
}

The LogIn method is used to validate the user with his username and password after establishing the connection to the remote server using the TCPClient class by the POP3Client method. The LogIn Method uses two functions, IsValidReply and SendMessage, to read and write the data to the remote server.

public bool LogIn(string userName,string
            password)
{
////POP Command USER - Sending USER Name for Mail
            Server
///Authentication
byte[] reply = SendMessage ('USER '
            + userName + '\r\n');
// Check if userName is
            ok.
if(IsValidReply(reply)){
////POP Command PASS - Sending
            Password for Mail Server
///Authentication
reply =
            SendMessage ('PASS ' + password + '\r\n');
// Check if password
            is ok.
if(IsValidReply (reply)){
m_LoggedIn =
            true;
m_UserName = userName;
m_Password = password;
return
            true;
}
}
return false;
}

SendMessage has two arguments, Data (of type string) and Terminator. This method is used to send the data, and waits until it gets the reply from the server for the data sent:

private byte[] SendMessage (string
            data,string terminator)
{
byte[] retVal =
            null;
try
{
NetworkStream strm =
            m_pPop3_Client.GetStream();
if(data.Length > 0){
byte[]
            byte_data =
            Encoding.ASCII.GetBytes(data.ToCharArray());
strm.Write(byte_data,0,byte_data.Length);
}
MemoryStream memStrm = new MemoryStream();
// Create
            loop
int counter = 0; // For command time out.
while(true){
if(strm.DataAvailable){
byte[] receivedData =
            new byte[m_pPop3_Client.ReceiveBufferSize];
int size =
            strm.Read(receivedData,0,m_pPop3_Client.ReceiveBufferSize);
memStrm.Seek(0,
            SeekOrigin.End);
memStrm.Write(receivedData,0,size);
// Create
            buffer for Compare eg. '\r\n.\r\n'.
byte[] bufComp = new
            byte[10];
// Read last 10 symbols from result
            stream
memStrm.Position = memStrm.Length -
            7;
memStrm.Read(bufComp,0,10);
string strCompare =
            Encoding.ASCII.GetString(bufComp);
// Check if message ends with
            terminator
if(strCompare.IndexOf(terminator) > -1){
break;
}
counter = 0;
}
else
{
System.Threading.Thread.Sleep(100);
}
if(counter
            > 100){
// ('time
            out');
break;
}
counter++;
}
retVal =
            memStrm.ToArray();
memStrm.Close();
}
catch(Exception
            x)
{
//('erro:' + x.Message);
}
return
            retVal;
}

The IsValidReply method is similar to the IsValidReply method in the SendMail class. This method uses stream reader to read the data, and this data, which is in the byte format, is converted to a string type using the Encoding.ASCII class and the GetString function. After converting the data to string, the data is validated.

private bool IsValidReply(byte[] reply)
{
if(reply != null){
string str = Encoding.ASCII.GetString
            (reply);
str =
            str.TrimStart();
if(str.StartsWith('+OK')){
return
            true;
}
}
return false;
}

The getMessagesList method is a public method. It is used to get the list of mails from the remote mail server. This method invokes the POP3MessageInfo class, which in turn gets the list of mails as an arraylist.

public Pop3MessagesInfo getMessagesList
            ()
{
// If not loggedIn, throw
            exception.
if(!m_LoggedIn){
throw new Exception('You must
            login before using this system !');
}
//POP Command LIST -
            Will list the mail info
byte[] reply = SendMessage
            ('LIST\r\n','.\r\n');
m_MessagesInfo = new
            Pop3MessagesInfo(reply);
return m_MessagesInfo;
}

Pop3MessagesInfo Class

The POP3MessageInfo class is used to list the number of mails for a user in the mail server, along with the size of each mail in an ArrayList. This class gets all the messages data in byte format and converts the byte data into a string and then splits the messages using the Split function:

private ArrayList m_MsgList = null;
public Pop3MessagesInfo(byte[] replay)
{
m_MsgList = new
            ArrayList();
string str =
            System.Text.Encoding.ASCII.GetString(replay);
string[] lines =
            str.Split(new Char[]{'\n'});
string[] msgs =
            null;
if(lines.Length > 3){
msgs = new
            string[lines.Length-3];
for(int
            i=1;i<lines.Length-2;i++){
msgs[i-1] = lines[i];
}
foreach(string s in msgs){
//
            System.Windows.Forms.MessageBox.Show(s);
string[] msgInf =
            s.Split(new char[]{' '});
if(msgInf.Length > 1){
string
            msgNr = msgInf[0];
int size =
            Convert.ToInt32(msgInf[1]);
m_MsgList.Add(new
            Pop3Message(msgNr,size));
}
else{
string msgNr =
            msgInf[0];
m_MsgList.Add(new
            Pop3Message(msgNr,-1));
}
}
}
}

The GetMessage() method is used to retrieve the mail message from the server after receiving the mail number as an argument. This method sends the POP command 'RETR' along with the mail number, using the SendDataWaitReplay function to get the message.

public byte[] getMessage(string
            messageNo)
{
// If not loggedIn, throw
            exception.
if(!m_LoggedIn){
throw new Exception('LogIn before
            using this function !');
}
//POP Command RETR - Will Retrive
            the the mails
byte[] replay = SendDataWaitRelpay('RETR ' +
            messageNo + '\r\n','.\r\n');
// Check that message is
            complete
if(EndsWith(replay,'.\r\n')){
return
            replay;
}
else{
return null;
}
}

The LogOff Method sends the POP command “QUIT” to logout from the Mail Server, but only after checking the logged in status.

public void LogOff()
{
// If not
            loggedIn, throw exception.
if(!m_LoggedIn){
throw new
            Exception('You must logged in to use this function
            ');
}
////POP Command QUIT - Will Disconnect from the
            Server.
SendDataWaitRelpay('QUIT
            \r\n');
m_pPop3_Client.Close();
}

WebMailWS - WebService

Now we are in the second part of our case study. Here we build a WebService that is used for storing the mail sent and received information. This webservice is also used to provide other information like MailDetails, Address Book, and Department List.

The WebMail Web service exposes seven methods to Internet clients:

  • IsUserValid
  • SendMailInfo
  • ReceiveUpdateInfo
  • getMailDetails
  • getAddressBook
  • getDeptDetails
  • getUserNames

Let's see each of these methods in detail.

We declare a class WebMailService that extends the WebService class from the System.Web.Services namespace. This namespace consists of the classes that enable us to build and use the Web Service.

public class WebMailService : System.Web.Services.WebService
{
private string m_UserName=null;
private string m_Password =null;
private string connStr=ConfigurationSettings.AppSettings['connStr'];

The WebService class is the base class of the WebmailService class. There can be any number of public or private classes. However, if a class is to expose some or all of its methods as part of the web service, the class and the method should be declared public.

The Web Service Attribute [WebService()] is used to specify the name of the web service and is also used in describing and providing help pages for the WebService.

IsUserValid WebMethod

IsUserValid is one of the public methods in the class WebMailService. This method validates a remote client by taking the UserName and Password as arguments. This validation allows the remote client to access various other services in the WebMailService class. Here we use the SqlDataReader object to access the database, since SqlDataReader provides a means of reading a forward-only stream of rows from a database in a much faster way. In this web service method, SqlDataReader is built using the SqlCommand object by executing the SQL statement using the ExecuteReader method. The user authentication fails if the reader returns a null value.

public bool isUserValid (string
            UserName,string Password)
{
try
{
string
            strSQL='';
m_UserName
            =UserName;
m_Password=Password;
SqlConnection sqlConn=new
            SqlConnection(connStr);
strSQL+=' Select emailid,password from
            Addressbook ';
strSQL+= ' where emailid='' + m_UserName + '' and
            ';
strSQL += ' password='' + m_Password + ''';
SqlCommand
            sqlComm = new
            SqlCommand(strSQL,sqlConn);
sqlComm.Connection.Open();
SqlDataReader
            dbReader =sqlComm.ExecuteReader(CommandBehavior.CloseConnection
            );
if (dbReader.Read())
{
//valid user
return
            true;
}
else
{
// authentication failed
return
            false;
}
}
catch
{
return false;
}
}

The SendMailInfo web method is used to store the mail send information of the remote client in the database. This web method takes MailID, Receivers Email ID, Mail Body, Mail Subject, etc as arguments and updates the database once the user has been validated. This method uses SQLConnection, SQLCommand and SQLDataReader objects for database update.

public string SendMailInfo(string
            UserName,string Password,string MailId,
string RefId,string
            ReceiverEMails,string DeptCode, string subject,string
            Message)
{
try
{
string SQL=null;
StringBuilder
            strSQL= new StringBuilder ();
m_UserName =UserName;
m_Password
            =Password;
if(IsUserValid(m_UserName,m_Password))
{
strSQL.Append (
            'Insert into SendMail values ('' + MailId + ''');
strSQL.Append
            (' ,'' + RefId + '',' + DeptCode + ','' + UserName +
            ''');
strSQL.Append (' ,'' + DateTime.Today + '','' + subject +
            '','' + Message + '')');
SqlConnection sqlConn=new
            SqlConnection(connStr);
SqlCommand sqlComm = new
            SqlCommand();
sqlComm.CommandText=strSQL.ToString();
sqlComm.Connection
            = sqlConn;
sqlComm.Connection.Open
            ();
sqlComm.ExecuteNonQuery();
sqlComm.Connection.Close();
Regex
            RgxEmail=new Regex(',');
string[] RecEmails =
            RgxEmail.Split(ReceiverEMails);
SqlCommand sqlCommUpdate= new
            SqlCommand();
foreach(string RecEmail in
            RecEmails)
{
SQL='';
SQL='insert into ReceiveMail values
            ('' + MailId + '','' + RecEmail + '','N','' + DateTime.Today +
            '','N')';
sqlCommUpdate.CommandText=SQL;
sqlCommUpdate.Connection
            =
            sqlConn;
sqlCommUpdate.Connection.Open();
sqlCommUpdate.ExecuteNonQuery();
sqlCommUpdate.Connection.Close();
}
return
            'Mail Send Successfully' ;
}
else
{
return 'User Not
            Valid';
}
}
catch(Exception e)
{
return
            e.ToString();
}
}

ReceivedDBUpdate WebMethod

The ReceivedDBUpdate WebMethod is used to set the flag “Y” in the Received column of the ReceiveMail table. This WebMethod gets the UserName, Password (as type string) and Received Mail information of a remote client as a dataset. The UserName and Password are used for the remote client authentication, and the dataset, which holds the information about the mail received by the remote user, is used to update the Mails Received status for all the MailID's in the Dataset. This method is called when a remote client receives a new mail or a set of new mails. These mails are the ones sent by any of the other clients using the WebMailClient software.

public string ReceivedDBUpdate(string
            UserName,string Password,DataSet MailUpdate)
{
try
{
string strSQL= null;
m_UserName =UserName;
m_Password
            =Password;
SqlConnection sqlConn=new SqlConnection(connStr);
if(isUserValid(m_UserName,m_Password))
{
SqlCommand
            sqlCommUpdate= new SqlCommand();
DataSet dsMailUpdate= new
            DataSet();
dsMailUpdate = MailUpdate;
//gets no of rows in the
            dataset
for (int i=0; i < dsMailUpdate.Tables[0].Rows.Count;
            i++)
{
strSQL='';
strSQL +='Update ReceiveMail set
            Received='Y',MailRead='Y',';
strSQL += ' RecDate='' +
            DateTime.Today + ''';
strSQL += ' where REMail='' + UserName + ''
            and ';
//get the mail id from the daatset
strSQL += '
            MailID='' +
            dsMailUpdate.Tables[0].Rows[i].ItemArray.GetValue(0).ToString() +
            ''';
sqlCommUpdate.CommandText=strSQL.ToString();
sqlCommUpdate.Connection
            =
            sqlConn;
sqlCommUpdate.Connection.Open();
sqlCommUpdate.ExecuteNonQuery();
sqlCommUpdate.Connection.Close();
}
}
return
            'Updated';
}
catch(Exception e)
{
return
            e.ToString();
}
}

MailDetails WebMethod

The MailDetails WebMethod takes the UserName and Password as arguments and returns mail information as a dataset. This method calls the function MailInfo for returning the dataset. This method not only gets the mail information of logged-in users, but also mail information for the other users that the logged-in user has permission to view.

public DataSet MailDetails(string
            UserName,string Password)
{
DataSet ds=new
            DataSet('SendMail');
m_UserName =UserName;
m_Password
            =Password;
if(IsUserValid
            (m_UserName,m_Password))
{
return MailInfo();
}
else
{
return ds;
}
}

SendMailInfo Method

The SendMailInfo function is used to get all the information from the SendMail table along with its corresponding information from the ReceiveMail table. The code starts out by filling a dataset with the SendMail table. The SQL query populates only the user’s mail information, which can be accessed by the logged-in user. The dataset is again filled with the ReceiveMail table. The relationship between these two tables in the dataset is created using the DataRelation object. This relationship enables us to see what links information from one table to another. The essential elements of a DataRelation are the name of the relationship, the two tables being related, and the related columns in each table. Here we relate the MailID columns of the two tables and keep the relationship name as Receiver Details.

public DataSet SendMailInfo()
{
DataSet ds;
StringBuilder strSQL = new
            StringBuilder();
SqlConnection myConnection = new
            SqlConnection(connStr);
// Create a
            SqlDataAdapter.
SqlDataAdapter myAdapter = new
            SqlDataAdapter();
myAdapter.TableMappings.Add('Table',
            'SendMail');
myConnection.Open();
strSQL.Append ('select *
            from sendmail where sender in');
strSQL.Append (' (select emailid
            from accessinfo where accessid=');
strSQL.Append (' (select
            accessid from addressbook where ');
strSQL.Append (' emailid=''+
            m_UserName + ''))');
SqlCommand myCommand = new
            SqlCommand(strSQL.ToString(),myConnection);
myCommand.CommandType
            = CommandType.Text;
myAdapter.SelectCommand = myCommand;
ds
            = new DataSet('Mail');
myAdapter.Fill(ds);
// Create a second
            Adapter and Command.
SqlDataAdapter adpReceive = new
            SqlDataAdapter();
adpReceive.TableMappings.Add('Table',
            'ReceiveMail');
strSQL.Remove(0,strSQL.Length);
strSQL.Append
            ('select * from ReceiveMail where REmail in');
strSQL.Append ('
            (select emailid from accessinfo where accessid=');
strSQL.Append
            (' (select accessid from addressbook where ');
strSQL.Append ('
            emailid=''+ m_UserName + ''))');
SqlCommand cmdReceive = new
            SqlCommand(strSQL.ToString(),myConnection);
adpReceive.SelectCommand
            = cmdReceive;
adpReceive.Fill(ds);
myConnection.Close();
// Create DataRelation.
System.Data.DataRelation
            dr;
System.Data.DataColumn dc1;
System.Data.DataColumn dc2;
dc1 = ds.Tables['SendMail'].Columns['MailID'];
dc2 =
            ds.Tables['ReceiveMail'].Columns['MailID'];
//setting the
            relation
dr = new System.Data.DataRelation('Receiver Details',
            dc1, dc2);
// Add the relation to the
            DataSet
ds.Relations.Add(dr);
return ds;
}

As well as these four methods, we also have another three web methods, which are all basically used to get general information like DepartmentList, AddressBook and UserName.

WebMethod Name | Purpose
getAddressBook | This web method is used to get all the email ids from the address book.
getDeptDetails | This web method is used to get the available department name along with the department code.
getUserName | This web method is used to get the EmailID along with the department code and department name.

Below is the screen shot of the web page rendered by the .NET Framework:

WebMailClient

Now we move to the last part of this case study. Here we build a user interface, which helps to send and receive e-mails. This interface is also used to keep track of all the mails sent using this software.

WebMailClient has a single WinForm. This does four different activities. For each of these activities we have a separate panel control. Before going further, let’s take a look at the four different activities:

  • Login to the System
  • Sending Mails
  • Receiving Mails
  • Mail Analysis

The ClientWebMail class contains a set of private variables that are used to hold information throughout the application.

public class ClientWebMail :
            System.Windows.Forms.Form
{
private string m_UserName =
            '';
private string m_Password = '';
private string m_DeptCode
            = '';
private bool m_LoggedIn = false;
private string
            XMLFilePath= Application.StartupPath + @'\';
// to store
            Addressbook info
DataSet dsAddress=new DataSet();
// to store
            departmentlist info
DataSet dsDept= new DataSet();
// to store
            username with DeptCode
DataSet dsUserName= new DataSet();
//
            to store MailInfo for analysis
DataSet dsMailInfo= new
            DataSet();
// creating a instant of the
            webservice
WebMailService WebMailWS= new WebMailService
            ();

Login to the System

The remote web server has to authenticate the user before he can access the system.

A panelControl panelDefault is used as a container to get the UserName and Password for authentication.

cmdLogin_Click Event

The cmdLogin_Click event is triggered when the user clicks the cmdLogin button after entering the UserName and Password. This event calls the IsUserValid WebMethod for authenticating the user. Once the user is authenticated, the event sets the variables m_UserName and m_Password to the username and password respectively, and then checks for the existences of the AddressBook XML file of that user. The AddressBook XML file is the one that stores the EmailID of the users using this Software. If the XML file does not exist, then a new one is created and will store the AddressBook data using the WebMethod getAddressBook. The data in the XML file is used to populate the EmailID’s in the listbox lstAddress. After populating the lstAddress, the cmdLogin_Click event populates the dataGrid with the mail information from the XML file that stores the downloaded mail information. If the XML file does not exist, then it will create a new one using the WebMethod MailDetails, which returns all the mail information from the remote database. We can even synchronize the XML file with updated information from the remote database. The cmdLogin_Click event also populates the cboDept with department codes using another WebMethod, getDeptDetails. The getUserName WebMethod is called to get the Department Code. Since most of the information is gathered from the remote WebService, we need not connect to the remote WebService regularly.

private void cmdLogin_Click(object
            sender, System.EventArgs e)
{
// checking for the textbox -
            empty
if (txtLoginID.Text==''
            ||txtLoginPassword.Text=='')
{
MessageBox.Show ('Enter
            UserName and
            Password');
}
else
{
statusBar.Panels[1].Text
            ='Connecting to WebService for User Validation';
//calling the
            remote webmethod
if
            (WebMailWS.IsUserValid(txtLoginID.Text.Trim(),txtLoginPassword.Text.Trim()))
{
statusBar.Panels[1].Text = 'Login
            Succeded';
statusBar.Panels[0].Text = txtLoginID.Text.Trim() + '
            is Logged In.';
// assigning the value
m_LoggedIn
            =true;
m_UserName=txtLoginID.Text.Trim();
m_Password=txtLoginPassword.Text.Trim();
string
            fileXML= XMLFilePath + txtLoginID.Text.Trim() + '_AddressBook.xml';
statusBar.Panels[1].Text = 'Checking for Address Book XML
            file..';
// checking for addressbook XML exist
FileInfo fi =
            new FileInfo (fileXML);
if
            (!fi.Exists)
{
statusBar.Panels[1].Text = 'File Does Not
            exist';
statusBar.Panels[1].Text = 'Getting the AddressBook
            information from WebService';
// If file does not exist
            Addressbook info
// got using the Webservice
DataSet DsAddXML=
            WebMailWS.getAddressBook ();
XmlDataDocument XD=new
            XmlDataDocument(DsAddXML);
statusBar.Panels[1].Text = 'Creating
            the XML File..';
// saving the addressbook XML
            file
XD.Save(fileXML);
}
statusBar.Panels[1].Text =
            'Popualting the Address Book';
statusBar.Panels[1].Text
            ='Reading the Address Book XML File';
dsAddress.ReadXml(fileXML);
statusBar.Panels[1].Text ='Populating the Address Book In
            ListBox';
lstAddressBook.DataSource
            =dsAddress;
lstAddressBook.DisplayMember='EMails.EMailID';
lstAddressBook.ValueMember
            ='EMails.EMailID';
fileXML= XMLFilePath + txtLoginID.Text.Trim()
            + '_MailInfo.xml';
FileInfo fiWS = new FileInfo (fileXML);
if
            (!fiWS.Exists)
{
statusBar.Panels[1].Text = 'Connecting to the
            WebService..';
WebMailWS.Credentials
            =System.Net.CredentialCache.DefaultCredentials;
statusBar.Panels[1].Text = 'It may take some time to download
            the
            MailInfo..';
dsMailInfo.Merge(WebMailWS.MailDetails(m_UserName,m_Password));
statusBar.Panels[1].Text = 'Getting the Mail
            Information..';
dsMailInfo.WriteXml(fileXML,XmlWriteMode.WriteSchema
            );
statusBar.Panels[1].Text = 'Populating the Information in the
            Grid';
//populating the mail info in
            DataGrid
dataGrid1.SetDataBinding(dsMailInfo,'SendMail');
}
else
{
statusBar.Panels[1].Text
            = 'Getting the Mail Information from Local
            Database';
dsMailInfo.ReadXml(fileXML,XmlReadMode.ReadSchema
            );
dataGrid1.SetDataBinding(dsMailInfo,'SendMail');
}
if
            (cboDept.Items.Count==0)
{
dsDept=WebMailWS.getDeptDetails();
dsUserName=WebMailWS.getUserNames
            ();
DataViewManager dvmDept = new
            DataViewManager();
dvmDept.DataSet =
            dsDept;
dvmDept.DataViewSettings[0].Sort =
            'Deptcode';
dvmDept.DataViewSettings[0].RowFilter='Distinct
            Deptcode';
//populating the DeptCode in combo
            using
//GetDeptDetails WebMethod
cboDept.DataSource
            =dvmDept;
cboDept.DisplayMember='Department.DeptName';
cboDept.ValueMember
            ='Department.DeptCode';
DataViewManager dvmDeptCode = new
            DataViewManager();
dvmDeptCode.DataSet =dsUserName;
dvmDept.DataViewSettings[0].RowFilter= 'EmailID='' + m_UserName
            + ''';
// getting the DeptCode of the loggedin
            user.
m_DeptCode =
            dvmDept.DataViewSettings[0].Table.Rows[0].ItemArray.GetValue(0).ToString
            ();
}
statusBar.Panels[1].Text ='';
panelDefault.Visible
            =false;
panelAnalysis.Visible =true;
Form.ActiveForm.Text
            ='WebMailClient -
            Analysis';
}
else
{
statusBar.Panels[1].Text = 'User
            Validation Fails Enter Correct UserName and
            Password';
}
}
}

Sending Mails

The next activity that is performed using the ClientWebMail class is sending mails. The PanelControl panelWrite is used as a container for other controls to get the mail information.

cmdSendMail_Click Event

The cmdSendMail_Click event is triggered when the user clicks the cmdSendMail button after entering the mail information. An instance of the SendMail class is created for sending the mail. After sending the mail, the mail information is stored in the remote database using the WebMethod SendMailInfo. Finally, the mail information is also stored in the XML file using the XmlDataDocument.

private void cmdSendMail_Click(object
            sender, System.EventArgs e)
{
try
{
XmlDataDocument
            XD=new XmlDataDocument();
string XMLBuilder = '';
if
            (txtUserName.Text==''
            ||txtHost.Text=='')
{MessageBox.Show('Information as to be
            Filled');}
else
{
//sending mail to remote
            server
statusBar.Panels[1].Text = 'Preparing to Send
            Mail..';
SendMail SendMail=new
            SendMail();
SendMail.SMTPServer=txtHost.Text.Trim();
SendMail.FromName
            =txtUserName.Text.Trim();
SendMail.From=txtUserName.Text.Trim() +
            '@' + txtHost.Text.Trim() ;
SendMail.To
            =lstAddressBook.Text.Trim()
            ;
SendMail.Subject=txtMailID.Text.Trim() + '#' + m_DeptCode + '#'
            + txtRefID.Text.Trim() + '#'
+
            txtSubject.Text.Trim();
SendMail.Body=txtBody.Text.Trim();
statusBar.Panels[1].Text
            = 'Sending Mail..';
SendMail.Send();
statusBar.Panels[1].Text
            = 'Mail has been Sent';
statusBar.Panels[1].Text = 'Preparing to
            Update the Mail info in WebService';
//updating the mail
            information n remote
            db
WM.SendMailInfo(m_UserName,m_Password,txtMailID.Text.Trim(),txtRefID.Text.Trim(),
lstAddressBook.Text.Trim(),m_DeptCode,txtSubject.Text.Trim
            (),txtBody.Text.Trim());
statusBar.Panels[1].Text = 'Updated the
            WebServie..';
string fileXML= XMLFilePath +
            txtLoginID.Text.Trim() + '_SendMail.xml';
statusBar.Panels[1].Text = 'Checking for Address Book XML
            file..';
// checking for addressbook XML exist
FileInfo fi =
            new FileInfo (fileXML);
if (!fi.Exists)
{
XD.LoadXml('');
XD.Save(fileXML);
}
statusBar.Panels[1].Text =
            'Writing the information in Local XML file.';
//creating the XML
            Structure
XMLBuilder += '';
XMLBuilder += ''
            + txtMailID.Text.Trim() + '';
XMLBuilder += '' +
            txtRefID.Text.Trim() + '';
XMLBuilder += '' +
            m_DeptCode + '';
XMLBuilder += '';
Regex
            Regex=new Regex(',');
string[] parts =
            Regex.Split(lstAddressBook.Text.Trim());
foreach(string t in
            parts)
{
XMLBuilder += '' + t +
            '';
}
XMLBuilder += '';
XMLBuilder
            += '' + txtSubject.Text.Trim() +
            '';
XMLBuilder += '' + txtBody.Text.Trim() +
            '';
XMLBuilder += '';
XmlNode currNode;
//Writing to the XML file
XmlDataDocument XDWriteMail = new
            XmlDataDocument ();
XDWriteMail.Load(fileXML);
XmlDocumentFragment docFrag =
            XDWriteMail.CreateDocumentFragment();
docFrag.InnerXml=
            XMLBuilder;
currNode =
            XDWriteMail.DocumentElement;
currNode.InsertBefore(docFrag,
            currNode.FirstChild );
//save the output to a
            file
statusBar.Panels[1].Text = 'Saving the XML
            File..';
XDWriteMail.Save(fileXML);
txtMailID.Text ='MID' +
            Guid.NewGuid().ToString();
statusBar.Panels[1].Text = 'New mail
            Id is generated you can send new mail
            now..';
}
}
catch(Exception
            ex)
{
statusBar.Panels[1].Text
            =ex.ToString();
}
}

cmdUpdateAddress_Click Event

The panel control panelWrite contains a command button cmdUpdateAddress. On clicking this, the cmdUpdateAddress_Click event gets triggered. This event helps to update the EMailID listed in the listbox lstAddress (lstAddress gets populated once the user is authenticated), with the latest one available in the remote database using the getAddressBook web method. This event also gives an option to update the AddressBook XML file with the latest Email Address from the remote database.

private void
            cmdUpdateAddress_Click(object sender, System.EventArgs e)
{
//calling the WebMethod
dsAddress=
            WM.GetAddressBook();
string message = 'Do you want to update the
            Client AdressBook?';
string caption = 'Update Client AddressBook
            ';
MessageBoxButtons buttons =
            MessageBoxButtons.YesNo;
DialogResult result;
result =
            MessageBox.Show(this, message, caption,
            buttons,
MessageBoxIcon.Question,
            MessageBoxDefaultButton.Button1,
MessageBoxOptions.RightAlign);
if(result
            == DialogResult.Yes)
{
//saving the latest adddressbook in XML
            file
string fileXML= XMLFilePath + txtLoginID.Text.Trim() +
            '_AddressBook.xml';
XmlDataDocument XD=new
            XmlDataDocument(dsAddress);
XD.Save(fileXML);
}
//populating the lstAddressBook with latest
            AddressBook
lstAddressBook.BindingContext['dsAddress'].SuspendBinding();
lstAddressBook.DataSource
            =dsAddress;
lstAddressBook.DisplayMember='EMails.EMailID';
lstAddressBook.ValueMember
            ='EMails.EMailID';
lstAddressBook.Refresh();
}

Receive Mail

The next activity that can be done using the ClientWebMail class is to check for new mail. The panel control panelChecMail is used as a container for showing the mail received.

menuCheckMail_Click Event

The menuCheckMail_Click event helps to get the e-mail from the remote mail server, update the received information in the remote database, and also updates the local XML file with the received mail information.

The code below creates an instant of the POP3Client class and builds a DataTable dtMailBox to store the received mail information.

private void menuCheckMail_Click(object
            sender, System.EventArgs e)
{
if
            (m_LoggedIn)
{
Pop3Client clnt = new
            Pop3Client(m_UserName,110);
string MailID=null;
string
            RefID=null;
string Date=null;
string From=null;
string
            Dept=null;
string Subject=null;
string Body=null;
string
            fileXML= XMLFilePath + txtLoginID.Text.Trim() + '_MailReceived.xml';
Regex Regex =new Regex('#');
string[]
            subjectArray;
DataSet dsOffLineMailinfo =new
            DataSet('MailInfo');
DataSet dsMailinfo =new
            DataSet('MailInfo');
DataTable dtMailBox=new
            DataTable('MailBox');
DataRow
            newRow;
dtMailBox.Columns.Add('MailID');
dtMailBox.Columns.Add('RefID');
dtMailBox.Columns.Add('DeptCode');
dtMailBox.Columns.Add('Sender');
dtMailBox.Columns.Add('MailSize');
dtMailBox.Columns.Add('Date');
dtMailBox.Columns.Add('Subject');
dtMailBox.Columns.Add('Body');

Next we get all the mail messages from the mail server using the getMessageList and getMessage methods of the POP3Client class. The received messages are split into MailID, RefID, Subject, Body, etc, by the getString function and are stored in the DataTable dtMailBox.

if(clnt.LogIn(m_UserName ,m_Password))
{
// Get messages list from server
Pop3MessagesInfo
            msgsInfo = clnt.GetMessagesList();
// Loop all messages and show
            them
foreach(Pop3Message m in
            msgsInfo.MessageInfo)
{
byte[] mime =
            clnt.GetMessage(m.MessageNr);
string
            str=System.Text.Encoding.ASCII.GetString(mime);
From =getString
            (str,'From:','To:');
Subject=getString
            (str,'Subject:','SendDate:');
subjectArray =
            Regex.Split(Subject);
if (subjectArray.GetUpperBound(0)>=
            3)
{
MailID= (string)subjectArray.GetValue(0);
Dept=
            (string)subjectArray.GetValue(1);
RefID=
            (string)subjectArray.GetValue(2);
Subject =
            (string)subjectArray.GetValue(3);
}
else
{
continue;
}
Date=getString
            (str,'SendDate:','MIME-Version:');
Body=getString
            (str,'STARTBODY','ENDBODY');
if
            (isMailReceived(MailID.Trim()))
{
newRow=
            dtMailBox.NewRow();
newRow['MailID']=MailID.Trim();
newRow['RefID']=RefID.Trim();
newRow['DeptCode']=Dept.Trim();
newRow['Sender']=From.Trim();
newRow['MailSize']
            =
            m.Size.ToString();
newRow['Date']=Date.Trim();
newRow['Subject']=Subject.Trim();
newRow['Body']=Body.Trim();
dtMailBox.Rows.Add(newRow);
}
}
clnt.LogOff();
int
            intMessageCount=dtMailBox.Rows.Count ;

Next the newly-received mail stored in the datatable dtMailBox are used to populate the listview lstReceiveMail control. To find the new mails we compare the MailId that is gotten from the received mails with the MailID's already stored in the XML file using the function IsMailReceived. The remote database is also updated with the new mail information using the WebMethod ReceivedDBUpdate.

for (int i=0; i <
            dtMailBox.Rows.Count; i++)
{
lstReceiveMail.Items.Add(dtMailBox.Rows[i].ItemArray.GetValue(0).ToString(),0);
lstReceiveMail.Items[i].BackColor=System.Drawing.Color.LightGray
            ;
lstReceiveMail.Items[i].SubItems.Add(dtMailBox.Rows[i].ItemArray.GetValue(1).ToString());
lstReceiveMail.Items[i].SubItems.Add(dtMailBox.Rows[i].ItemArray.GetValue(2).ToString());
lstReceiveMail.Items[i].SubItems.Add(dtMailBox.Rows[i].ItemArray.GetValue(3).ToString());
lstReceiveMail.Items[i].SubItems.Add(dtMailBox.Rows[i].ItemArray.GetValue(4).ToString());
lstReceiveMail.Items[i].SubItems.Add(dtMailBox.Rows[i].ItemArray.GetValue(5).ToString());
lstReceiveMail.Items[i].SubItems.Add(dtMailBox.Rows[i].ItemArray.GetValue(6).ToString());
}
DataSet
            dsWS = new
            DataSet();
dsWS.Tables.Add(dtMailBox);
WebMailWS.ReceivedDBUpdate(m_UserName
            ,m_Password,dsWS)
public bool IsMailReceived(string MailID )
{
string fileXML= XMLFilePath + txtLoginID.Text.Trim() +
            '_MailReceived.xml';
FileInfo fi=new FileInfo(fileXML);
if
            (fi.Exists)
{
XmlDocument doc = new
            XmlDocument();
doc.Load(fileXML);
XmlNodeList elemList =
            doc.GetElementsByTagName('MailID');
for (int i=0; i <
            elemList.Count; i++)
{
if
            (MailID.Equals(elemList[i].InnerXml.Trim()))
{
return
            false;
}
}
return true;
}
else
{
return
            true;
}
}

Now we again populate the Listview with the mail information stored previously in the XML file and then update the new mail information in the XML file.

FileInfo fi=new FileInfo(fileXML);
if(fi.Exists)
{
dsOffLineMailinfo.ReadXml(fileXML);
}
if (dsOffLineMailinfo.Tables.Count > 0 )
{
for (int
            j=0; j < dsOffLineMailinfo.Tables[0].Rows.Count ;
            j++)
{
lstReceiveMail.Items.Add(dsOffLineMailinfo.Tables[0].Rows[j].ItemArray.GetValue(0).ToString(),0);
lstReceiveMail.Items[j].SubItems.Add(dsOffLineMailinfo.Tables[0].Rows[j].ItemArray.GetValue(1).ToString());
lstReceiveMail.Items[j].SubItems.Add(dsOffLineMailinfo.Tables[0].Rows[j].ItemArray.GetValue(2).ToString());
lstReceiveMail.Items[j].SubItems.Add(dsOffLineMailinfo.Tables[0].Rows[j].ItemArray.GetValue(3).ToString());
lstReceiveMail.Items[j].SubItems.Add(dsOffLineMailinfo.Tables[0].Rows[j].ItemArray.GetValue(4).ToString());
lstReceiveMail.Items[j].SubItems.Add(dsOffLineMailinfo.Tables[0].Rows[j].ItemArray.GetValue(5).ToString());
lstReceiveMail.Items[j].SubItems.Add(dsOffLineMailinfo.Tables[0].Rows[j].ItemArray.GetValue(6).ToString());
}
}
dsOffLineMailinfo.Merge(dtMailBox);
dsOffLineMailinfo.WriteXml(fileXML);

Mail Analysis

The final activity that is done using the ClientWebMail class is analyzing the mail received. The panel control panelAnalysis is used as a container for showing the mail information that is populated in a dataGrid by the time the user has logged in to the system. The mail information taken from the remote database using the web method GetMailDetails is stored in a dataset dsMailInfo. A search criterion is provided to refine the mail information with respect to department code, Username, and by date using the DataView searchDV object.

private void cmdSearch_Click(object
            sender, System.EventArgs e)
{
DataView searchDV = new
            DataView();
if
            (cboDept.Items.Count!=0)
{
searchDV=dsMailInfo.Tables['SendMail'].DefaultView
            ;
searchDV.RowFilter ='DeptCode = '' + cboDept.SelectedValue +
            ''';
if(!cboUName.Text.Equals (''))
{
searchDV.RowFilter
            ='Sender = '' + cboUName.SelectedValue +
            ''';
}
searchDV.RowFilter ='SendDate >= '' + dtpFrom.Value
            + '' and SendDate <='' + dtpTo.Value + ''';
dataGrid1.SetDataBinding( searchDV ,'');
}
}

Scope for Improving the WebMail

We have now developed a WebMail mail client. Even though most features have been provided, we can still improve both the design and functionality of the WebMail. We can:

  • Provide an option to delete the mails both in the Mail server as well in the remote database.
  • Provide an option to send an attachment with the mail could be provided.
  • Since the mail information can be stored in the local disk, more analyzing options could be provided.
  • We could provide an option to get part of the mail information from the remote server, thus avoided bulk download of mail information from remote server.
  • We could improve the interface of the WebMailClient by adding HTML support.
  • We could increase the security of the XML file stored in the local disk by encrypting the file.

Summary

In this case study we have tried to develop a real time application, which helps to track and analyze mail. We also have developed a send and receive mail component that can be easily integrated with the other application.

As we have seen, this case study has been divided into three parts for better understanding; the first part explains the process involved to build the send and receive mail component, the second part explains the process involved for login authentication and for updating the mail information in a remote database using a WebService, while the third part explains the interface used to send and receive mails and to do mail analysis.

Case Study Information
Author | Vinod Kumar
Editor | Adam Ryland
Reviewers | Andrew Krowczyk, Saurabh Nandu, Shefali Kulkarni

If you have any questions or comments about this case study, or suggestions for further pieces, please contact the technical editor. If you wish to talk about the case study with other subscribers, please visit our forum here.


Extending Application Functionality

Extending Application Functionality
http://www.csharptoday.com/content.asp?id=1843
Article Rating: 3.5
Published on June 3, 2002
Abstract
Most of today’s commercial software applications support some way through which the application can be extended and customized. This allows external software vendors and developers to add functionality that is not part of the core product itself, or customize the application to form part of a bigger solution. Some companies even develop an entire solution by extending and customizing a combination of various off the shelf products. An example of this may be where a corporate customer develops an in-house solution by extending and customizing Microsoft Office. In this case study, Jerrie Pelser will take a brief look at the various techniques available to application developers to open their applications up for customization by external developers. We will then develop a basic add-in framework and develop an application which views the metadata of a .NET assembly using this framework.
Case Study

Most of today's commercial software applications support some way through which the application can be extended and customized. This allows external software vendors and developers to add functionality that is not part of the core product itself, or customize the application to form part of a bigger solution. Some companies even develop an entire solution by extending and customizing a combination of various off the shelf products. An example of this may be where a corporate customer develops an in-house solution by extending and customizing Microsoft Office.

In this case study, we will take a brief look at the various techniques available to application developers to open their applications up for customization by external developers. We will then develop a basic add-in framework and develop an application which views the metadata of a .NET assembly using this framework.

Before we continue please note the following software requirements for this case study as well as the accompanying source code:

  • Microsoft .NET Framework Version 1.0
  • Microsoft Visual Studio.NET

The various customisation mechanisms

There are various methods through which software developers can make their application more extensible:

  • Add-ins
  • Scripting
  • Exposing application functionality through an object model.
  • Exposing application functionality through exported DLL functions.

Add-ins

Add-ins allow software developers to distribute application functionality over various modules which can be added and removed from the main application. This can be done without having to recompile the entire application. The advantages of add-ins include:

  • Software developers can provide advanced application features as separate modules. This allows users of the application to pay for only the modules that they are really interested in using. Once these modules are installed on the user's machine, the extra features will typically surface in the main application via added menus or toolbar buttons.
  • Software developers can ship upgrades to their end users without having to redistribute the entire application. Only the add-in to which the changes were made need to be sent out to users.
  • If a user of the application has a need for some specific functionality, an add-in can be developed, managed and distributed to that user independently of the main application.

Add-ins are available in most modern commercial software products, including Microsoft Visual Studio .NET. Prior to .Net, applications typically took advantage of COM by implementing a specific interface through which the main application would communicate with the add-in. As stated earlier, once an add-in is loaded into the application, it will make itself typically available by appending extra menu items or toolbar buttons to the main application user interface. Other add-in technologies, such as Microsoft Management Console (MMC) snap-ins, will add extra nodes to the scope pane on the left-hand side of the main MMC user interface.

Scripting

Scripting allows users of our application to enhance its functionality by writing small pieces of software code called scripts that manipulate our application in some way, or change its standard behaviour. The most common example of scripting is found in Microsoft Office. The various applications in Microsoft Office allow end users to record macros, or develop them from scratch using VBA.

These macros are simply small pieces of software code which will perform certain tasks when they are executed. These macros are written using VBA and will manipulate the main application by performing actions on some of the objects which are exposed by the application.

Exposing the Object Model

Contemporary applications expose their functionality to other applications. In the COM world this is done through COM Automation. Automation was originally developed as a way for applications to expose their functionality to other applications, including scripting languages. The goal here is to provide a way for external client applications to access properties and call methods of the automation server. In most cases this is the method through which the previous two methods (namely Add-ins and Scripting) manipulate the main application.

Exporting DLL functions

One of the classic ways of exposing application functionality to external applications is to export DLL functions. An application developer would typically export a function from a DLL either by its name or by its ordinal value so it can be used in external applications. When an application developer wanted to use these exported functions, he would import them so they would be available to be called from his application. When importing a function, the developer would typically need to declare the name of the function, the data types of parameters it expects, as well as the name of the DLL in which it is defined. The syntax used to export and import functions from a DLL differ between programming languages and are beyond the scope of this document. For more information on how this is done under C++, please refer to the section of Importing and Exporting DLLs under the Visual C++ documentation which accompanies Visual Studio .NET.

Developing an extensible application with add-ins

During the remainder of this case study we will develop a sample application which can be extended by the use of add-ins. The sample application which we will be developing will be a .NET assembly browser. This application will allow us to view all the data types and resources which are defined in a .NET assembly.

Reflection in .NET

Before we continue discussing the .NET assembly browser in more detail, let us first have a quick look at assemblies, data types and reflection in the .NET Framework.

.NET Assemblies

Assembly Overview

A .NET assembly is a physical file which consist of one ore more portable executable (PE) files generated by a .NET compiler. An assembly consists of a manifest, one or more modules, and optionally some resources. For more information on assemblies please refer to "Simplifying Deployment and Solving DLL Hell with the .NET Framework" (http://msdn.microsoft.com/library/en-us/dndotnet/html/dplywithnet.asp?frame=true&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq).

Assemblies can be either static or dynamic. A static assembly is stored on disk in PE files. Dynamic assemblies are created in memory and executed directly without saving to disk. Dynamic assemblies are created using the classes in the System.Reflection.Emit namespace, but further discussion about this is beyond the scope of this article. For more information on dynamic assemblies please refer to the articles entitled "Dynamic assemblies in .NET framework" (http://www.csharptoday.com/content.asp?id=1725&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq) and "Emitting Dynamic Assemblies on the fly using Reflection.Emit" (http://www.csharptoday.com/content.asp?id=1784&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq).

The Assembly Manifest

The assembly manifest is the part of the assembly which makes it self-describing. The following information gets stored in the assembly manifest:

  • Assembly name. This is a text string which specifies the assembly's name.
  • Version Information. The version number consists of four parts, namely a major and minor version number, as well as a revision and build number. The Common Language Runtime (CLR) uses this information to enforce the version policy.
  • Strong name information. Contains the public key from the publisher of the assembly, which is used to give an assembly a strong name.
  • List of files. A hash of each file contained in the assembly and a file name.
  • Type information. This is the list of all the types in the assembly with a mapping to the module which contains the type. This information in used by the Reflection API to discover type information.
  • Referenced assemblies. This is a list of all external assemblies which are statically referenced by an assembly.
  • Culture Information. Defines the culture or language information for a satellite assembly. A satellite assembly contains culture or language specific resources.

Assembly Packaging

An assembly can be packaged as either a single-file assembly or as a multi-file assembly. A single-file assembly contains only one associated file. In this case the manifest is incorporated into the PE file to form a single-file assembly. A multi-file assembly contains more than one associated file. In this case the manifest can exist either as a standalone entity, or be incorporated into one of the PE files in the assembly. The diagram below illustrates this:

.NET Types

In .NET all types can be classified in two basic categories, namely value types and reference types.

Value Types

A variable of a value type contains the actual data. Every time you declare a variable of a value type, you have allocated the number of bytes associated with that type on the stack. Whenever you pass a variable that is a value type, the variable value is passed and not a reference to the underlying object.

Reference Types

Reference types are stored on the heap, and may only be accessed via a reference to its position in memory. Samples of value types are classes, interfaces, delegates, and arrays. A variable to a reference type will always contain a reference to a value of that type, or a null reference.

Everything is an object

All types in .NET are ultimately derived from System.Object, even value types. This ensures that all types have a minimum set of capabilities. Whenever you treat a value type such as an int or double as an object it goes through a process on boxing and unboxing.

Boxing is the process of converting a value type to a reference type (in other words to an object on the garbage collected heap). Boxing creates an object instance and copies the value into the new object.

Unboxing is the process of converting a reference type to a value type. All of this may seem confusing, but what is important to us is that all types in .NET ultimately inherit from System.Object and therefore has some base functionality.

Querying Metadata with Reflection

The metadata of a .NET assembly can be interrogated using the classes contained in the System.Reflection namespace. These classes enable you to logically traverse assembly and type information. If we look at the System.Reflection namespace you will notice the System.Reflection.Assembly class. This class contains information about the assembly such as the list of modules that make up the assembly, the assembly's version information, assembly dependencies, and other resources that make up the assembly.

Beneath the Assembly class we will find the System.Reflection.Module class. A module represents a single PE file consisting of one or more types. The System.Type class represents a single type declaration: a class type, interface type, array type or value type. Looking at the System.Type class we notice that it consists of various members each of which is represented by a type of System.Reflection.MemberInfo. A member can basically be a constructor, field, property, method, event or other nested types. These are all represented more specifically by classes inheriting from System.Reflection.MemberInfo, namely the ConstructorInfo, FieldInfo, PropertyInfo, MethodInfo, EventInfo, and Type classes. For constructors and methods there is one more level to the metadata. Constructors and methods may have parameters which are represented by the System.Reflection.ParameterInfo class.

The complete metadata hierarchy is displayed in the diagram below.

For more information on reflection in .NET please refer to ".NET Metadata and Custom Attributes in C#" (http://www.csharptoday.com/content.asp?id=1576&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq)

The Assembly Browser

Overview of the Assembly Browser

In the remaining part of this case study we are going to develop an application that allows us to view all the metadata for any given assembly. This Assembly Browser will be architected in such a way that it will allow other people to extend the functionality of the browser by writing a separate add-in which implements extended functionality. The Assembly Browser will consist of three main parts:

  • The Assembly Browser application. This is the main application which will be executed by the end users. It also serves as a host application for the add-ins which we will develop.
  • The extensibility module. This module is where we will define the interfaces and classes which our host application and the various add-ins will use to communicate with one another.
  • The various add-ins. For the purpose of this article we are going to develop three different add-ins:
  • A Module browser which will allow us to view the different modules which are defined in the assembly, as well as the different types which are defined in each module.
  • A Resource browser which will allow us to view information about the various resources which are included in the assembly.
  • A References browser which will allow us to view information about the other assemblies which are referenced by the specific assembly.

Below is a screenshot of what the final Assembly Browser looks like:

The user interface of the Assembly Browser is divided into the following sections:

The Explorer Pane.

The explorer pane contains the explorer tree view. The nodes that make up this tree are added by the various add-ins.

The Property Pane.

Once a node is selected in the explorer pane, the property pane contains information about the specific node which was selected. The property pane contains property sheets which displays information on the node which is selected.

The add-in framework

The add-in framework is at the heart of our Assembly Browser. It is here that we specify the interfaces that add-in developers will need to implement in order to develop an add-in for our application.

There are three ways in which an add-in could extend the Assembly Browser user interface:

  • The first way would be to register a tree node to automatically be added as a child node to another type of node.
  • The second way would be to register a callback procedure which will execute when a certain type of node is expanded.
  • The third and final way is to develop custom property sheets which will display in the property pane when a node is selected in the explorer pane.

Let us continue by looking at the various classes and interfaces which make up the add-in framework. The complete source code of all these can be found in the Extensibility project.

IAddin interface

The IAddin is implemented by add-in modules. This interface allows us to notify an add-in of events such as when an add-in gets loaded or unloaded.

Below is the definition of the IAddin interface:

public interface IAddin
{
void
            OnUnload();
void OnLoad();
}

The OnLoad() method gets executed whenever the host application loads the add-in. This allows an add-in to register the nodes and property sheets which will extend the Assembly Browser. The OnUnload() method gets executed whenever the host application unloads an add-in from memory. This will allow the add-in to perform any cleanup procedures that may be required.

ExplorerNode

The ExplorerNode class acts as the base class for all nodes which will appear in the Explorer tree view. This class has functionality which allows us to handle the expanding of nodes, as well as display popup menus.

Let's have a more detailed look at the various parts which makes up the ExplorerNode class:

protected bool HasChildren
{
get {
            return hasChildren; }
set
{
if (hasChildren !=
            value)
{
hasChildren = value;
if
            (hasChildren)
{
AddDummyNode();
}
else
{
Nodes.Clear();
}
}
}
}

The HasChildren property dictates whether a node has child nodes. Since child nodes are loaded on demand we need to specify upfront whether a node has any. If it does, we need to add a "dummy" node so that a + sign appears next to the node. Once the node is expanded the first time, the "dummy" node is removed and the actual child nodes are added to the node.

private void
            AddDummyNode()
{
Nodes.Clear();
Nodes.Add(new
            AssemblyBrowser.Extensibility.DummyExplorerNode());
}

The AddDummyNode method appends the actual dummy or placeholder node to the tree view. The first thing we do is to delete all existing child nodes from the node. Once that is done we add a new instance of the DummyExplorerNode class. This is a special class which inherits from ExplorerNode and is used only for the "dummy" node.

private bool LoadedOnce
{
get { return loadedOnce; }
}

Since child nodes are loaded on demand, we only want to call the procedure which loads child nodes once. The LoadedOnce property is used to indicate whether the nodes child nodes have already been loaded.

public void LoadNodes()
{
if
            (!LoadedOnce)
{
TreeView.Cursor =
            System.Windows.Forms.Cursors.WaitCursor;
TreeView.BeginUpdate();
try
{
RemoveDummyNode();
PerformLoadNodes();
}
finally
{
TreeView.EndUpdate();
TreeView.Cursor
            = System.Windows.Forms.Cursors.Default;
}
loadedOnce =
            true;
}
}

The LoadNodes method is called by the host application whenever the end user expands a tree node. The first task is to check whether the child nodes for this node have already been loaded. If it has been loaded we do nothing. If it has not been loaded, the first thing we do is remove our "dummy" node. Remember that this node acts as a sort of placeholder node until the actual child nodes for the node have been loaded. Next we call the PerformLoadNodes method. This is a virtual method which must be overridden by inheriting nodes to perform the actual loading of the child nodes.

While loading the child nodes we change the screen cursor to the busy cursor to give the end user an indication that something is happening. We also call the BeginUpdate method to disable redrawing of the tree view while we are loading the child nodes. This also improves the performance when adding items to the tree view. Once we have finished loading all the child nodes, we call the EndUpdate method to tell the tree view that it can resume painting.

The final thing to do is to set the private loadedOnce field. We only allow inherited classes read-only access to this field via the LoadedOnce property.

private void
            RemoveDummyNode()
{
foreach(System.Windows.Forms.TreeNode node
            in Nodes)
{
if (node is
            AssemblyBrowser.Extensibility.DummyExplorerNode)
Nodes.Remove(node);
}
}

The RemoveDummyNode method iterates through the list of child nodes, and removes all instances of the DummyExplorerNode.

private void
            Reset()
{
Nodes.Clear();

loadedOnce = false;

if
            (HasChildren)
AddDummyNode();
}

The Reset method is used to initialise an ExplorerNode to its default state. During the Reset method all the child nodes are deleted and the loadedOnce flag is cleared. If the node has child nodes, a dummy child node is added as well.

public void
            Refresh()
{
TreeView.BeginUpdate();
try
{
bool
            wasExpanded =
            IsExpanded;
this.Collapse();

Reset();

if
            (wasExpanded)
this.Expand();
}
finally
{
TreeView.EndUpdate();
}
}

The Refresh method is used to reload all child nodes. To refresh a node we call the Reset method which will restore the node to its initial state. An internal flag is maintained called wasExpanded, which is used to remember if the node was in an expanded state when the Refresh method was called. After the node is reset to its initial state, we re-expand the node if it was expanded to start off with. You will also note that one of the first things we do is to collapse the node. If we do not explicitly collapse the node, the IsExpanded property of the tree node does not get set to false, and the node does not expand correctly if we try to re-expand it. Once again all of this code is enclosed by calls to the BeginUpdate and EndUpdate methods of the tree view to prevent it from repainting while child nodes are deleted and added during the Refresh method.

protected bool HasDeleteMenu
{
get {
            return hasDeleteMenu; }
set { hasDeleteMenu = value;
            }
}

protected bool HasRefreshMenu
{
get { return
            hasRefreshMenu; }
set { hasRefreshMenu = value;
}
}

The HasDelete and HasRefresh properties are used to indicate if the context menu of the node must contain Delete and Refresh menu items. When the context menu for the node gets created, these two menu items may automatically get added based on the values of the HasRefreshMenu and HasDeleteMenu properties.

public void ShowMenu(System.Windows.Forms.Control control,
System.Drawing.Point pos)
{
System.Windows.Forms.ContextMenu contextMenu = CreateMenu();

if (contextMenu == null)
contextMenu = new System.Windows.Forms.ContextMenu();

if (HasDeleteMenu || HasRefreshMenu)
{
if (contextMenu.MenuItems.Count > 0)
{
contextMenu.MenuItems.Add("-");
}

if (HasDeleteMenu)
{
System.Windows.Forms.MenuItem deleteMenuItem = new
System.Windows.Forms.MenuItem(
"Delete",
new System.EventHandler(DeleteMenuClick));

contextMenu.MenuItems.Add(deleteMenuItem);
}

if (HasRefreshMenu)
{
System.Windows.Forms.MenuItem refreshMenuItem = new
System.Windows.Forms.MenuItem(
"Refresh",
new System.EventHandler(RefreshMenuClick));

contextMenu.MenuItems.Add(refreshMenuItem);
}
}

if (contextMenu.MenuItems.Count > 0)
contextMenu.Show(control, pos);
}

The ShowMenu method is called by the host application whenever the user right-clicks on a tree node. The application makes a call to the virtual CreateMenu method. Inherited nodes can override this method to create their own context specific menu items. Now check if the inherited nodes actually returned a context menu. If not, we create one. We check the HasDeleteMenu and HasRefreshMenu properties as discussed above, and create the Delete and Refresh menu items as appropriate. Additionally we check if other menu items already exist before we create the Delete and Refresh menu item. If there are other menu items we create a separator menu item.

You will notice that the OnClick event handlers of the Delete and Refresh menu items point to the DeleteMenuClick and RefreshMenuClick methods respectively. These will be discussed below. Lastly we check if any menu items were added to the context menu after all of this, and if so, display the context menu.

private void DeleteMenuClick(object sender,
            System.EventArgs e)
{
if
            (PerformDelete())
this.Remove();
}

protected virtual
            bool PerformDelete()
{
return false;
}

The DeleteMenuClick method is set as the OnClick event handler of the Delete menu item of the context menu. We call the virtual PerformDelete method that allows inherited nodes to perform their own deletion of the data that they represent. If the PerformDelete method returns true, we delete the actual tree node. By default the PerformDelete method returns false.

private void RefreshMenuClick(object sender,
            System.EventArgs e)
{
PerformRefresh();
}

protected
            virtual void
            PerformRefresh()
{
this.Refresh();
}

The RefreshMenuClick method is set as the OnClick event handler of the Refresh menu item of the context menu. It calls the virtual PerformRefresh method, which allows inherited nodes to perform their own processing when we refresh it. By default, the PerformRefresh method calls the Refresh method of the node.

public virtual System.Drawing.Icon
            Image
{
get { return null; }
}

public virtual
            System.Drawing.Icon SelectedImage
{
get { return null;
            }
}

The Image and SelectedImage properties are used to get the image to display next to a node. By default, these return a null value, but inherited nodes can override them to specify their images.

The IPropertySheet Interface

The IPropertySheet interface needs to be implemented by all property sheets.

public interface IPropertySheet
{
string Caption
{
get;
}

AssemblyBrowser.Extensibility.ExplorerNode ExplorerNode
{
get;
set;
}

void LoadData();
}

The Caption property must return the caption for a property sheet. This caption will be displayed as the text for the tab page on which the property sheet is displayed. The ExplorerNode property will be used to specify the node whose data the property sheet must display. Finally the LoadData method will be called to notify the property sheet that it must display its data.

The BaseAssemblyNode Class

The BaseAssemblyNode acts as the base class from which all explorer nodes in our assembly browser will inherit. It handles the displaying of nodes that were registered to add themselves automatically to the tree view. It also handles the execution of the callback methods which were registered by add-ins. We could have actually combined the ExplorerNode and BaseAssemblyNode functionality into one class for the purpose of this case study. Putting most of the functionality in the ExplorerNode class allows us to better re-use it in the future for other programs.

public System.Reflection.Assembly
            Assembly
{
get { return assembly; }
set
{
assembly
            = value;
InitializeNode();
}
}

The Assembly property allows us to keep track of a reference to the currently open assembly. Once a value is assigned to the Assembly property we call the virtual InitializeNode method to execute some initialization routines.

protected virtual void
            InitializeNode()
{
if (this.TreeView != null &&
            this.TreeView.ImageList != null)
{
int nodeIndex = -1;
int
            selectedNodeIndex = -1;

System.Drawing.Icon nodeImage =
            this.Image;
if (nodeImage !=
            null)
{
this.TreeView.ImageList.Images.Add(nodeImage);
nodeIndex
            = this.TreeView.ImageList.Images.Count - 1;
}
this.ImageIndex
            = nodeIndex;

System.Drawing.Icon nodeSelectedImage =
            this.SelectedImage;
if (nodeSelectedImage !=
            null)
{
this.TreeView.ImageList.Images.Add(nodeSelectedImage);
selectedNodeIndex
            = this.TreeView.ImageList.Images.Count - 1;
}
if
            (selectedNodeIndex >= 0)
this.SelectedImageIndex =
            selectedNodeIndex;
else
this.SelectedImageIndex =
            nodeIndex;
}
}

The InitializeNode method allows inherited nodes to perform their own initialization. By default, the InitializeNode method gets the images for the selected and unselected state of the node from the Image and SelectedImage properties. It adds these images to the ImageList component of the tree view, and sets the ImageIndex and SelectedImageIndex properties of the tree node to the indexes of the newly added images in the image list.

Please note: This routine is not optimized for efficient use of resources! As you can see it will just keep on adding images to the image list indefinitely. A future improvement might include a way to get the images for a node based on the node type, and not for each and every instance of a node.

protected System.Drawing.Icon GetImageFromResource(
System.Reflection.Assembly resourceAssembly, string resourceName)
{
System.IO.Stream iconStream = resourceAssembly.GetManifestResourceStream(
resourceName);

if (iconStream != null)
{
System.Drawing.Icon nodeIcon = new System.Drawing.Icon(iconStream);
return nodeIcon;
}
return null;
}

GetImageFromResource is a helper method to load the specified manifest resource from the assembly which is passed in as a parameter. The GetManifestResourceStream method of the assembly gives us the ability to return a data stream from which the icon is created.

protected override void PerformLoadNodes()
{
System.Type[] autoChildNodeTypes =
NodeRegistrationUtil.GetNodeTypes(this);
if (autoChildNodeTypes != null)
{
foreach(System.Type autoChildNodeType in autoChildNodeTypes)
{
if (autoChildNodeType.IsSubclassOf(
typeof(AssemblyBrowser.Extensibility.BaseAssemblyNode)))
{
AssemblyBrowser.Extensibility.BaseAssemblyNode autoNode =
(AssemblyBrowser.Extensibility.BaseAssemblyNode)
System.Activator.CreateInstance(autoChildNodeType);
this.Nodes.Add(autoNode);

autoNode.Assembly = this.Assembly;
}
}
}

AssemblyBrowser.Extensibility.NodeLoadEventHandler[] nodeLoadEventHandlers
= NodeRegistrationUtil.GetNodeLoadEventHandlers(this);
if (nodeLoadEventHandlers != null)
{
foreach(AssemblyBrowser.Extensibility.NodeLoadEventHandler
eventHandler in nodeLoadEventHandlers)
{
AssemblyBrowser.Extensibility.NodeLoadEventArgs eventArgs =
new AssemblyBrowser.Extensibility.NodeLoadEventArgs(this);
eventHandler(this, eventArgs);
}
}
}

The application overrides the base PerformLoadNodes method to allow for loading of registered nodes and the execution of node load event handlers.

A call to the GetNodeTypes method of the NodeRegistrationUtil class returns an array of System.Type for all the node types which we registered to load themselves under this node. We now iterate through the returned array and ensure that each of the types is a subclass of BaseAssemblyNode and call the CreateInstance method of the Activator class. This class is part of the Reflection API and the CreateInstance method is used to create an instance of a type. Once the new node is created, assign its Assembly property.

Now call the GetNodeLoadEventHandlers method of the NoderegistrationUtil class which will return an array of event handlers to execute when this node expands. Iterate through the returned array and execute each of the event handlers.

The AssemblyNode Class

Figure 1: An Assembly Node

The AssemblyNode class represents a specific assembly. This class does not really do much by itself, but represents the assembly which is currently loaded. Typically, all of our add-ins will register themselves to attach to this node.

protected override void
            InitializeNode()
{
base.InitializeNode();

HasChildren =
            true;
this.Text = Assembly.GetName().Name;
}

In the InitializeNode method of the AssemblyNode class, call the base InitializeNode method. We also set the HasChildren property to true, to specify that this node will have children. Finally set the Text property of the node to the name of the assembly.

public override System.Drawing.Icon
            Image
{
get
{
return
            GetImageFromResource(this.GetType().Assembly,
"AssemblyBrowser.Extensibility.Images.Assembly.ico");
}
}

public
            override System.Drawing.Icon SelectedImage
{
get
{
return GetImageFromResource(this.GetType().Assembly,
"AssemblyBrowser.Extensibility.Images.Assembly_Selected.ico");
}
}

Override the Image and SelectedImage properties to specify the icons display when the node is in the unselected and selected state.

The PropertyGridSheet Class

Figure 2: A property grid

The PropertyGridSheet class acts as the base class of all property sheets that display information about an object in a PropertyGrid. This class is basically a form which contains a PropertyGrid component and implements the IPropertySheet interface. For more information on the PropertyGrid control please refer to "The .NET Property Grid Control" (http://www.csharptoday.com/content.asp?id=1646&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq).

Besides the implementation of the IPropertySheet interface, it introduces a virtual method called GetPropertyGridObject. Derived classes or subclasses must override this method to supply the object that must be displayed in the property grid.

Developing the host application

Before we go into much more detail on each of the add-ins, let us first take a look the host application. The source code for the host application can be found in the AssemblyBrowser project.

Loading the add-ins

When the host application first starts up, the add-ins are loaded. The list of add-ins to load and their locations are specified in the application's configuration file. The structure of the configuration file is discussed later in the document under the section "The Application Configuration File".

private void LoadAddins()
{
try
{
System.Collections.IDictionary addinLocations =
(System.Collections.IDictionary)
System.Configuration.ConfigurationSettings.GetConfig("AddIns");

if (addinLocations != null)
{
foreach(System.Collections.DictionaryEntry addinLocation in
addinLocations)
{
LoadAddin((string)addinLocation.Value);
}
}
}
catch(Exception ex)
{
throw new System.ApplicationException(
"An error occurred while loading the addins", ex);
}
}

To get the list of add-ins from the application configuration file, we use the GetConfig method of the System.Configuration.ConfigurationSettings class. This will return a reference to an IDictionary which contains the settings for the configuration section which we specified. Now iterate through the list of items in the returned IDictionary, and call LoadAddin for each of the values.

private void LoadAddin(string location)
{
System.Reflection.Assembly addinAssembly =
System.Reflection.Assembly.LoadFrom(location);

if (addinAssembly != null)
{
System.Type[] addinTypes = addinAssembly.GetTypes();
foreach(System.Type addinType in addinTypes)
{
System.Type interfaceType =
addinType.GetInterface(
typeof(AssemblyBrowser.Extensibility.IAddin).FullName);
if (interfaceType != null)
{
AssemblyBrowser.Extensibility.IAddin addinInstance =
(AssemblyBrowser.Extensibility.IAddin)
System.Activator.CreateInstance(addinType);

addinInstance.OnLoad();

addins.Add(addinInstance);
}
}
}
}

The LoadAddin method loads the assembly for the add-in at the specified location. It then iterates through all the types in the assembly, and inspects each type to determine if it implements the IAddin interface. For each type that implements the IAddin interface, it creates a new instance of the type using the CreateInstance method of the Activator class. Finally it calls the OnLoad method of the add-in, and adds it to the list of add-ins which are loaded.

Opening an assembly

Figure 3: The Open menu item

public void OpenAssembly(string filename)
{
CloseAssembly();

currentAssembly = System.Reflection.Assembly.LoadFrom(filename);

AssemblyBrowser.Extensibility.AssemblyNode assemblyNode =
new AssemblyBrowser.Extensibility.AssemblyNode();
tvwBrowser.Nodes.Add(assemblyNode);
assemblyNode.Assembly = currentAssembly;

tvwBrowser.SelectedNode = assemblyNode;
}

The OpenAssembly method is used to open a new assembly. This code will be executed when the user selects the File | New menu item from the Assembly Browser user interface. When this option is selected, the application checks if it an assembly is open and if so it closes it. The selected assembly is then loaded by calling the static LoadFrom method of the System.Reflection.Assembly class. The LoadFrom method takes the name or path of the assembly to open as a parameter.

A new AssemblyNode is created and added to the explorer tree view and given focus.

Closing an assembly

Figure 4: The Close menu item

public void CloseAssembly()
{
if
            (currentAssembly !=
            null)
{
ClearPropertySheets();

tvwBrowser.Nodes.Clear();

imageList.Images.Clear();

currentAssembly
            = null;
}
}

The CloseAssembly method is executed when the user selects the File | Close menu item from the assembly user interface. This method clears (or deletes) all the property sheets and clears all the nodes from the tree view. Furthermore, it will clear all images from the image list. Finally we set the reference to the currently opened assembly to null.

Expanding a tree node

private void tvwBrowser_BeforeExpand(object sender,
System.Windows.Forms.TreeViewCancelEventArgs e)
{
if (e.Node is AssemblyBrowser.Extensibility.BaseAssemblyNode)
((AssemblyBrowser.Extensibility.BaseAssemblyNode)
e.Node).LoadNodes();
}

When the user expands a tree node, check to see if the node inherits from BaseAssemblyNode or one if its descendants. If it does, cast the node to a BaseAssemblyNode and call the LoadNodes method.

Displaying the property sheets for a node

private void tvwBrowser_AfterSelect(object sender,
System.Windows.Forms.TreeViewEventArgs e)
{
if (e.Node is AssemblyBrowser.Extensibility.BaseAssemblyNode)
LoadPropertySheets(
(AssemblyBrowser.Extensibility.BaseAssemblyNode)e.Node);
}

When the user selects a node in the explorer tree view, check to see if the node inherits from BaseAssemblyNode. If it does, call the LoadPropertySheets method and pass the node as a parameter.

public void LoadPropertySheets(
AssemblyBrowser.Extensibility.BaseAssemblyNode assemblyNode)
{
System.Windows.Forms.TabPage tabPage;

ClearPropertySheets();

System.Type[] propertySheetTypes =
AssemblyBrowser.Extensibility.NodeRegistrationUtil.
GetNodePropertySheets(assemblyNode);

if (propertySheetTypes != null)
{
foreach(System.Type propertySheetType in propertySheetTypes)
{
if (propertySheetType.GetInterface(
typeof(AssemblyBrowser.Extensibility.IPropertySheet).
FullName) != null)
{
AssemblyBrowser.Extensibility.IPropertySheet propertySheet =
(AssemblyBrowser.Extensibility.IPropertySheet)
System.Activator.CreateInstance(propertySheetType);

tabPage = new System.Windows.Forms.TabPage(propertySheet.Caption);
tabPropertySheets.TabPages.Add(tabPage);

if (propertySheet is Form)
{
((Form)propertySheet).FormBorderStyle = FormBorderStyle.None;
((Form)propertySheet).TopLevel = false;
((Form)propertySheet).Show();
}

tabPage.Controls.Add((Control)propertySheet);
((Control)propertySheet).Dock = DockStyle.Fill;

propertySheet.ExplorerNode = assemblyNode;
propertySheet.LoadData();
}
}
}

tabPropertySheets.Visible = (tabPropertySheets.TabPages.Count > 0);
}

The LoadPropertySheets method gets the list of property sheets for the node by calling the GetNodePropertySheets method of the NodeRegistrationUtil class. This will return an array of System.Type for the types of the property sheets to display for the node.

Iterate through the list of property sheet types to ensure that they implement the IPropertySheet interface. If it does, create a new instance of the property sheet. Then create a new tab page for the tab control and set the Text of the tab page to the Caption of the property sheet.

If the property sheet is a form (which it will be most of the time), set the BorderStyle to None, and set the TopLevel property to false. We need to set TopLevel to false, else we will not be able to add the property sheet to the controls collection of the tab page. Finally we call the Show method so the form can display itself.

The property sheet is then appended to the list of controls which is contained by the tab page, and its Dock property is set to DockStyle.Fill to fill the entire client area of the tab page. Assign the ExplorerNode property of the property sheet and call its LoadData method to allow it to display the data for the node.

Finally check if the selected node does indeed contain any property sheets and if it doesn't, simply hide the tab control by setting its Visible property to false.

Displaying the context menu of a node

private void tvwBrowser_MouseDown(object
            sender,
System.Windows.Forms.MouseEventArgs e)
{
if
            (e.Button == MouseButtons.Right)
{
// Get the node at the
            specified point
TreeNode node = tvwBrowser.GetNodeAt(e.X,
            e.Y);

if (node != null)
{
// Select the
            node
tvwBrowser.SelectedNode = node;

// Display the
            context menu
if (node is
            AssemblyBrowser.Extensibility.BaseAssemblyNode)
{
((AssemblyBrowser.Extensibility.BaseAssemblyNode)node).
ShowMenu(tvwBrowser,
            new System.Drawing.Point(e.X, e.Y));
}
}
}
}

When the user clicks on a node in the explorer tree view, check to see if it was the right mouse button which was clicked. If this is true, we get the tree node at the specified X and Y coordinates, and select the node. Next, check to see if the node inherits from BaseAssemblyNode. If it does, call the ShowMenu method of the node and pass the tree view and coordinates as parameters.

The application configuration file

As mentioned above, the list of add-ins are specified in the application configuration file. The configuration file is located in the same directory as the host application and has the same name, but with a .config extension added to the filename. In our case, this file is called AssemblyBrowser.exe.config. An example of this file can be found with the accompanying source code and executable application. We specify the list of add-ins under the Addins section of the configuration file. You can have a look at the files which accompany this case study to look at the contents of the configuration file, and set the locations of the add-ins to the proper locations as per your own directory structure on your computer.

For more information on application configuration files, please refer to the .NET Framework documentation.

The Modules Add-in

The Modules add-in displays a list of all the modules which are contained in an assembly, and all the types which are defined in each module. All the source code for the Modules add-in can be found in the Modules assembly.

ModuleNode

Figure 5: A module node

The ModuleNode class represents a module which is contained in the assembly.

public System.Reflection.Module
            Module
{
get { return module; }
set
{
module =
            value;
this.Text = module.Name;
}
}

The Module property contains the reference to the module which is represented by this node. When a value is assigned to the Module property, we set the Text property of the node to the name of the module.

protected override void
            InitializeNode()
{
base.InitializeNode();

HasChildren =
            true;
HasRefreshMenu = true;
}

In the InitializeNode method of the Module class we indicate that the node has children and that is has a Refresh menu item.

protected override void
            PerformLoadNodes()
{
base.PerformLoadNodes();

System.Type[]
            moduleTypes = module.GetTypes();
foreach(System.Type moduleType
            in moduleTypes)
{
TypeNode typeNode =
            TypeNodeBuilder.CreateTypeNode(moduleType);
if (typeNode !=
            null)
{
this.Nodes.Add(typeNode);
typeNode.Assembly =
            this.Assembly;
}
}
}

The PerformLoadNodes method of the Module class loads all the types which are contained in the module. These types can either be a class, interface, struct, enum, or delegate. To get the list of types contained in the module, we call the GetTypes method of the module field which contains a reference to an instance of the System.Reflection.Module class. This will return an array of System.Type objects, which contains a list of the types in the module. We iterate through the list of types to add them to the ModuleNode.

Each of the types are represented by a node which inherits from TypeNode. In order to create the correct TypeNode, we call the CreateTypeNode method of the TypeNodeBuilder class. We then append the TypeNode we just created to list child nodes for this node, and assign its Assembly property.

The TypeNodeBuilder Class

The TypeNodeBuilder class is just a helper class which we use to create the correct descendant class of the TypeNode class depending on the actual type. This class only has one static method, called CreateTypeNode, which return the correct TypeNode based on the type which is passed to it.

public static TypeNode
            CreateTypeNode(System.Type type)
{
if
            (type.IsClass)
{
if (type.BaseType ==
            typeof(System.MulticastDelegate))
{
DelegateTypeNode
            delegateTypeNode = new
            DelegateTypeNode();
delegateTypeNode.MemberInfo =
            type;

return
            delegateTypeNode;
}
else
{
ClassTypeNode classTypeNode =
            new ClassTypeNode();
classTypeNode.MemberInfo =
            type;

return classTypeNode;
}
}
else if
            (type.IsEnum)
{
EnumTypeNode enumTypeNode = new
            EnumTypeNode();
enumTypeNode.MemberInfo = type;

return
            enumTypeNode;
}
else if
            (type.IsInterface)
{
InterfaceTypeNode interfaceTypeNode = new
            InterfaceTypeNode();
interfaceTypeNode.MemberInfo =
            type;

return interfaceTypeNode;
}
else if
            (type.IsValueType)
{
StructureTypeNode structTypeNode = new
            StructureTypeNode();
structTypeNode.MemberInfo =
            type;

return
            structTypeNode;
}
else
{
UnknownTypeNode unknownTypeNode
            = new UnknownTypeNode();
unknownTypeNode.MemberInfo =
            type;

return unknownTypeNode;
}
}

To create the correct TypeNode we interrogate the attributes of the type which was passed in. If it is a class (the IsClass property of the type is true) and it inherits from MultiCastDelegate, we can safely assume that this is a delegate and create a DelegateTypeNode. Otherwise, if it is a class, we create a ClassTypeNode.

If the IsEnum property of the type is true, we create an EnumTypeNode. If the IsInterface property of the type is set to true, we create an InterfaceTypeNode. If the IsValueType property of the type is true, we can assume that this is a struct, as all other value types are accounted for already, so we create a StructTypeNode. If none of the above cases were true, it is a type which we do not cater for, so we create a special type of node called an UnknownTypeNode to indicate to the user that it is an unknown type.

The MemberInfoNode Class

The MemberInfoNode class represents a System.Reflection.MemberInfo class. All the tree nodes which represents a type node inherit from MemberInfoNode. Below is the class hierarchy of all the type nodes along with the reflection class that it maps to.

Based on the hierarchy above, lets have a quick look at the source code for each of these classes.

The TypeNode Class

The TypeNode class acts as a base class for all the classes which represents a descendant of System.Type. This includes classes, interfaces, structs, delegates and enums.

public class TypeNode : MemberInfoNode
{
public System.Type Type
{
get { return MemberInfo as System.Type; }
}
}

The TypeNode class has only one property called Type. This returns a reference to the System.Type that is represented by the class.

The ContainerTypeNode Class

The ContainerTypeNode class acts as the base class for all classes which contains other members, such a fields, methods, or properties. The ContainerTypeNode overrides the PerformLoadNodes method which implements the ability to load members. Classes which inherit from ContainerTypeNode are ClassTypeNode, InterfaceTypeNode, and StructureTypeNode.

protected override void PerformLoadNodes()
{
base.PerformLoadNodes();

System.Reflection.MemberInfo[] fields = Type.GetMembers(
System.Reflection.BindingFlags.Instance |
System.Reflection.BindingFlags.DeclaredOnly |
System.Reflection.BindingFlags.NonPublic |
System.Reflection.BindingFlags.Public |
System.Reflection.BindingFlags.Static
);

foreach(System.Reflection.MemberInfo memberInfo in fields)
{
if (memberInfo is System.Reflection.FieldInfo)
{
System.Reflection.FieldInfo fieldInfo =
(System.Reflection.FieldInfo)memberInfo;

FieldNode fieldNode = new FieldNode();
this.Nodes.Add(fieldNode);

fieldNode.Assembly = this.Assembly;
fieldNode.MemberInfo = fieldInfo;
}
else if (memberInfo is System.Reflection.PropertyInfo)
{
System.Reflection.PropertyInfo propertyInfo =
(System.Reflection.PropertyInfo)memberInfo;

PropertyNode propertyNode = new PropertyNode();
this.Nodes.Add(propertyNode);

propertyNode.Assembly = this.Assembly;
propertyNode.MemberInfo = propertyInfo;
}
else if (memberInfo is System.Reflection.MethodBase)
{
System.Reflection.MethodBase methodInfo =
(System.Reflection.MethodBase)memberInfo;

if (!methodInfo.IsSpecialName || methodInfo.IsConstructor)
{
MethodNode methodNode = new MethodNode();
this.Nodes.Add(methodNode);

methodNode.Assembly = this.Assembly;
methodNode.MemberInfo = methodInfo;
}
}
}
}

The PerformLoadNodes method gets a list of all the members for the underlying type. We do this by calling the GetMembers method of System.Type, which will return an array of MemberInfo objects for the specified type. Once a list of all the members is returned, iterate through it and create its appropriate node accordingly. For fields we create a FieldNode, for properties we create a PropertyNode, and for methods we create a MethodNode. What is important is that we exclude methods where the IsSpecialName property of the MemberInfo object is true. An example of the special methods that we exclude is the get and set methods of properties.

The ClassTypeNode, InterfaceTypeNode and StructureTypeNode Classes

As can be seen in the diagram above the ClassTypeNode, InterfaceTypeNode and StructureTypeNode all inherit from ContainerTypeNode and therefore have the ability to contain fields, methods and properties. The only property which we override in these classes is the Image property, which returns the correct icon for the node.

The remaining nodes in the Modules add-in

None of the rest of the nodes in the modules add-in adds much more functionality. They all only override the Image property which returns the correct icon for the node.

The ModulePropertySheet Class

The ModulePropertySheet class is inherited from PropertyGridSheet, and overrides the GetPropertyGridObject method.

protected override object
            GetPropertyGridObject()
{
if (ExplorerNode is
            ModuleNode)
{
return
            ((ModuleNode)ExplorerNode).Module;
}

return
            null;
}

Since the ModulePropertySheet class is defined as the property sheet class of the ModuleNode, the GetPropertyGridObject method which the ModulePropertySheet class overrides checks to see if the object referenced by the ExplorerNode property inherits from ModuleNode. If it does, it returns the Module property of the ModuleNode object.

MemberInfoPropertySheet

The MemberInfoPropertySheet class is inherited from PropertyGridSheet, and overrides the GetPropertyGridObject method.

protected override object
            GetPropertyGridObject()
{
if (ExplorerNode is
            MemberInfoNode)
{
return
            ((MemberInfoNode)ExplorerNode).MemberInfo;
}

return
            null;
}

The GetPropertyGridObject method which the MemberInfoPropertySheet class overrides checks to see if the object referenced by the ExplorerNode property inherits from MemberInfoNode. If it does, it returns the MemberInfo property of the MemberInfoNode object.

MethodParametersPropertySheet

The MethodParametersPropertySheet class displays the parameters of a method in a grid. The property sheet contains a DataGrid component in which the name and type of all of the parameters of the method are displayed.

void AssemblyBrowser.Extensibility.IPropertySheet.LoadData()
{
if (explorerNode is MethodNode)
{
System.Collections.ArrayList parameterList =
new System.Collections.ArrayList();

System.Reflection.MethodBase methodInfo =
((MethodNode)explorerNode).MethodInfo;
System.Reflection.ParameterInfo[] parameters =
methodInfo.GetParameters();

foreach(System.Reflection.ParameterInfo parameter in parameters)
{
parameterList.Add(new MethodParameter(
parameter.Name,
parameter.ParameterType.ToString()));
}

dataGrid.DataSource = parameterList;
}
}

The LoadData method first checks to see if the ExplorerNode inherits from MethodNode. If this is true, retrieve the parameters of the method by calling the GetParameters method of the MemberNode's MethodInfo property. This returns an array of ParameterInfo objects. Next we iterate through the array of parameters. For each parameter we create an instance of the internal MethodParameter class and add it to the internal array list which contains the parameters. Finally we set the DataSource property of the DataGrid to point to the array list containing the MethodParameters objects. This will display all the parameters in the grid.

The Modules add-in class

The ModulesAddin class implements the IAddin interface. This class is responsible for registering the nodes and property sheets. In the IAddin.OnLoad method of the ModulesAddin class, we register a node load event handler for the AssemblyNode class called LoadModuleNodes. This means that when the user expands a tree node that is an instance of the AssemblyNode class, the event handler which we specified is executed.

public void LoadModuleNodes(object sender, NodeLoadEventArgs e)
{
if (e.ParentNode is AssemblyBrowser.Extensibility.AssemblyNode)
{
AssemblyBrowser.Extensibility.AssemblyNode assemblyNode =
(AssemblyBrowser.Extensibility.AssemblyNode)e.ParentNode;
System.Reflection.Module[] assemblyModules =
assemblyNode.Assembly.GetModules(false);

foreach(System.Reflection.Module assemblyModule in assemblyModules)
{
ModuleNode moduleNode = new ModuleNode();
assemblyNode.Nodes.Add(moduleNode);

moduleNode.Assembly = assemblyNode.Assembly;
moduleNode.Module = assemblyModule;
}
}
}

The LoadModuleNodes method gets a list of all the modules in the assembly. It does this by executing the GetModules method of the Assembly class. This returns an array containing Module objects representing the modules in the assembly. Next we iterate through the array of modules and create ModuleNode objects for each of them.

The rest of the ModulesAddin class implements the remaining methods and properties which are specified by the IAddin interface.

The References Add-in

The References add-in displays a list of all the assemblies which are referenced by the assembly. All the source code for the References add-in can be found in the References project.

The ReferencesNode Class

Figure 6: A References node

The ReferencesNode is a container node below which all the referenced assemblies are displayed.

protected override void
            InitializeNode()
{
base.InitializeNode();

HasChildren =
            true;
Text = "References";
}

The InitializeNode method of the ReferencesNode class specifies that the node will have child nodes and it sets the Text property of the node to display "References".

protected override void
            PerformLoadNodes()
{
base.PerformLoadNodes();

System.Reflection.AssemblyName[]
            assemblyNames =
Assembly.GetReferencedAssemblies();
foreach(System.Reflection.AssemblyName
            assemblyName in assemblyNames)
{
ReferencedAssemblyNode
            referencedAssemblyNode = new
ReferencedAssemblyNode();
Nodes.Add(referencedAssemblyNode);

referencedAssemblyNode.Assembly
            = Assembly;
referencedAssemblyNode.AssemblyName =
            assemblyName;
}
}

The PerformLoadNodes method retrieves a list of all the assemblies which are referenced by the assembly. It does this by calling the GetReferencedAssemblies method of the Assembly class. This returns an array of AssemblyName objects, containing the information about each referenced assembly. Next we iterate through the array of AssemblyName objects, and create a ReferencedAssemblyNode class for each of them.

The ReferencedAssemblyNode Class

Figure 7: A referenced assembly node

The ReferencedAssemblyNode class represents an AssemblyName object for an assembly which is referenced by the assembly.

public System.Reflection.AssemblyName
            AssemblyName
{
get { return assemblyName; }
set
{
assemblyName = value;
Text = assemblyName.Name;
}
}

The AssemblyName property allows us to specify the AssemblyName object which is referenced by the ReferencedAssemblyNode. When we assign the AssemblyName property, we set the Text property of the node to the Name property of the AssemblyName object.

The ReferencePropertySheet class

The ReferencePropertySheet class is inherited from PropertyGridSheet, and overrides the GetPropertyGridObject method.

protected override object
            GetPropertyGridObject()
{
if (ExplorerNode is
            ReferencedAssemblyNode)
{
return
            ((ReferencedAssemblyNode)ExplorerNode).AssemblyName;
}

return
            null;
}

The GetPropertyGridObject method which the ReferencePropertySheet class overrides checks to see if the object referenced by the ExplorerNode property inherits from ReferencedAssemblyNode. If it does, it returns the AssemblyName property of the ReferencedAssemblyNode object.

The ReferencesAddin Class

The ReferencesAddin class implements the IAddin interface. This class is responsible for registering the nodes and property sheets.

void IAddin.OnLoad()
{
NodeRegistrationUtil.RegisterNodeType(
typeof(ReferencesNode),
typeof(AssemblyBrowser.Extensibility.AssemblyNode));

NodeRegistrationUtil.RegisterNodePropertySheet(
typeof(ReferencedAssemblyNode),
typeof(ReferencePropertySheet));
}

The IAddin.OnLoad method of the ReferencesAddin class registers the ReferencesNode to automatically be displayed below the AssemblyNode class when it is expanded. It also registers the ReferencePropertySheet class as a property sheet for the ReferencedAssemblyNode class.

The Resources Add-in

The Resources add-in displays a list of all the resources contained by the assembly. All the source code for the Resources add-in can be found in the Resources assembly.

The ResourcesNode Class

Figure 8: A resources node

The ResourcesNode acts as a container node below which all the resources contained in the assembly are displayed.

protected override void
            InitializeNode()
{
base.InitializeNode();

HasChildren =
            true;
HasRefreshMenu = true;
Text =
            "Resources";
}

The InitializeNode method specifies that the node has child nodes and the context menu contains a Refresh menu item. It also sets the Text property of the node to display "Resources".

protected override void
            PerformLoadNodes()
{
base.PerformLoadNodes();

string[]
            resourceNames =
            Assembly.GetManifestResourceNames();
foreach(string resourceName
            in resourceNames)
{
ResourceNode resourceNode = new
            ResourceNode();
Nodes.Add(resourceNode);

resourceNode.Assembly
            = Assembly;
resourceNode.ResourceName =
            resourceName;
}
}

The PerformLoadNodes method loads all the resources contained in the assembly. To get the list of resources we call the GetManifestResourceNames method of the assembly. This returns an array of strings with the names of all of the resources. We iterate through the array of names and create a ResourceNode for each resource.

The ResourceNode Class

Figure 9: A resource node

The ResourceNode class represents a resource which is contained in the assembly.

public System.Reflection.ManifestResourceInfo ResourceInfo
{
get
{
if (Assembly != null)
return Assembly.GetManifestResourceInfo(ResourceName);

return null;
}
}

The ResourceInfo property returns a ManifestResourceInfo object which contains manifest resource topology information. We get this information by calling the GetManifestResourceInfo method of the assembly.

protected override System.Windows.Forms.ContextMenu CreateMenu()
{
System.Windows.Forms.ContextMenu contextMenu =
new System.Windows.Forms.ContextMenu();

// Add the 'Export' menu item
contextMenu.MenuItems.Add("Export...");

return contextMenu;
}

The CreateMenu method is overridden in order to illustrate how easy it is to add a menu item to the context menu of a node. To do this we create a new ContextMenu object. Then we add a menu item to the context menu which displays the text "Export...".

The ResourcesAddin Class

The ResourcesAddin class implements the IAddin interface. This class is responsible for registering the nodes and property sheets.

void IAddin.OnLoad()
{
NodeRegistrationUtil.RegisterNodeType(
typeof(ResourcesNode),
typeof(AssemblyBrowser.Extensibility.AssemblyNode));

NodeRegistrationUtil.RegisterNodePropertySheet(
typeof(ResourceNode),
typeof(ResourcePropertySheet));
}

The IAddin.OnLoad method of the ResourcesAddin class registers the ResourcesNode to automatically be displayed below the AssemblyNode class when it is expanded. It also registers the ResourcePropertySheet class as a property sheet for the ResourceNode class.

Case Study Review

In this case study we covered the following:

  • Background information about the different customization methods which are available to developers if they want to make their applications extensible.
  • Background information about the format and structure of .NET assemblies and how we can use reflection to query the metadata of an assembly.
  • The development of an add-in framework which can be implemented in applications and will enable applications to be extended through the utilization of add-ins.
  • Putting together all of the information above, we developed an Assembly Browser application which allows users to view the metadata of any .NET assembly.

During the discussion of the development of the add-in framework and the Assembly Browser, I touched on all of the most important aspects of the source code. It is important for developers to understand the methods which they can use to open up their applications for customization by external developers. I would urge you however to explore the source code which accompanies this article in detail, so that you can gain a better understanding of the underlying technologies and hopefully adapt it so you can implement it in your own projects.

Case Study Information
Author | Jerrie Pelser
Technical Editor | Adam Ryland
Project Manager | Helen Cuthill
Reviewers | Dinar Dalvi, John Hodgson, Shefali Kulkarni

If you have any questions or comments about this case study, or suggestions for further pieces, please contact the technical editor. If you wish to talk about the case study with other subscribers, please visit our forum here.


Runtime Reverse Engineering with System.Diagnostics

Runtime Reverse Engineering with System.Diagnostics
http://www.csharptoday.com/content.asp?id=1775
Article Rating: not yet rated
Published on May 27, 2002
Abstract
In this case study, Tony Loton looks at how you can trace the execution of a .NET application to find out which methods are invoked, on which objects, by which objects, and in what order; so you'll discover how your application really behaves at runtime. You'll learn how to turn tracing on and off, and how to configure the tracing behavior outside of the application. As the logical conclusion to this work we'll build a UML sequence diagram from an application trace, which could be used for comparison against an original dynamic model design or simply as an aesthetic form of presentation.
Case Study

In this case study we'll look at how you can trace the execution of a .NET application to find out which methods are invoked, on which objects, by which objects, and in what order; so you'll discover how your application really behaves at runtime.

You'll learn how to turn tracing on and off, and how to configure the tracing behavior outside of the application. Though we'll be using C# as the implementation language, our use of the System.Diagnostics classes should be equally applicable to any .NET language.

As the logical conclusion to this work we'll build a UML sequence diagram from an application trace, which could be used for comparison against an original dynamic model design or simply as an aesthetic form of presentation.

Introduction

In a previous case study, "C# to UML Reverse Engineering" (http://www.csharptoday.com/content.asp?id=1742&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq), I discussed techniques for reverse engineering a C# source file or any .NET assembly into a UML class diagram. I ended that case study by saying this.

"As implemented in many modeling tools, the term reverse engineering can be taken in a limited sense to mean static reverse engineering, and that's what we've dealt with in this case study. Start with a non-executing binary or source code file and end up with a static UML class diagram.

UML is not limited to static modeling and the UML notation supports dynamic modeling of the execution behavior of applications. I'm thinking here of sequence diagrams and collaboration diagrams, which you normally draw up-front to describe how you expect the application to behave. Wouldn't it be good if you could reverse engineer those diagrams too, to show how the application really does behave when you run it?"

In the discussion that follows we'll look at doing exactly that, and if you're not a UML fan don't tune out just yet. While a UML sequence diagram is the logical conclusion to this work, I'm sure you'll pick up something useful along the way in terms application profiling in general. no UML required.

Judging by the responses to a Java article that I wrote on the same subject some time ago, this topic should be interesting to a wide audience, so here goes.

System Requirements

You'll need the .NET Framework SDK that's available for free from the Microsoft web site at http://msdn.microsoft.com/netframework/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq, or the Visual Studio .NET that's not. Either way, you'll need to be able to launch a command window that's aware of the .NET environment, which VS.NET users can do by choosing the Microsoft Visual Studio .NET / Visual Studio .NET Tools / Visual Studio .NET Command Prompt program from the Windows Start menu.

If you'd like to take advantage of the Rational Rose script that I'll offer for rendering the results in UML, you'll need a copy of Rational Rose '98 or later. You could try picking up a trial copy from http://www.rational.com/tryit/rose/index.jsp?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq.

Defining The Problem

The problem I've set myself is that of capturing information from a running application so that we can trace which operations have been called, on which objects, in what order. That is, to come up with a trace like this:

which when (optionally) rendered as UML would be a sequence diagram like this:

Regardless of the representation used, what this information tells us is how the application really behaves at run time, so that you can compare that behavior with your original dynamic model. You did produce one at design time, didn't you? Alternatively it could prove to be a very handy technique for debugging.

Case Study Structure

  • In the "Defining the Solution" section we'll define the solution as comprising some instrumentation code, a compilation flag, and an application configuration file. Plus an optional Rational Rose script to build the UML sequence diagram.
  • In the "Component Design and Coding" section we'll step through the code of a program to insert instrumentation automatically into any application, and we'll look at the code for the Rose script.
  • In the "Sample Application" section I'll offer a simple Banking program, which we'll trace and for which I'll present the trace results.

Defining The Solution

It's not the first time I've looked at this kind of problem. The first time resulted in an article for Java Developer's Journal (March 2001) entitled "Using the Java Platform Debugger Architecture" and the second time resulted in chapter 10 "Servlet Debugging Techniques" of the Wrox Press "Professional Java Servlets 2.3" book (http://www.wrox.com/ACON11.asp?WROXEMPTOKEN=141193ZxsCGPbA0ntBAQn4Emvg&ISBN=186100561X). In the first case, the technology employed was the Java Platform Debugger Architecture (JPDA), and in the second case it was Servlet Filters and Event Listeners.

For the .NET solution we'll use two classes from the .NET Framework System.Diagnostics namespace: System.Diagnostics.Trace and System.Diagnostics.StackTrace. We'll instrument an application by inserting a few lines of code at the start of each method:

// ** START INSTRUMENTATION **

Get a stack trace that tells us where we are in the running application:

StackTrace stackTrace=new StackTrace();

The top stack frame will tell us which object / method is executing, and the next stack frame will tell us which (other) object invoked the method:

StackFrame topFrame=stackTrace.GetFrame(0);
StackFrame nextFrame=stackTrace.GetFrame(1);

Assume no calling object unless there really is a second stack frame, in which case we'll record the class (declaring type) of the calling object:

String __callingObject=":";
if (nextFrame!=null) __callee=":"+nextFrame.GetMethod().DeclaringType;

Now write out the calling object, the method that was called (i.e. the method into which this code has been inserted), and the class of the current object:

System.Diagnostics.Trace.Write(__callee);
System.Diagnostics.Trace.Write("|"+topFrame.GetMethod());
System.Diagnostics.Trace.WriteLine("|:"+topFrame.GetMethod().DeclaringType);

//
            ** END INSTRUMENTATION **

The result will be output as follows, with pipe-separated entries for the calling class (none on the first line), the method invoked, and the receiving class. Don't worry about the colon ":" characters just yet, which are there to allow for a future enhancement in which we would specify each object as instanceID:ClassName rather than just :ClassName.

:|Void Main(System.String[])|:Banking.Bank
:Banking.CurrentAccount|Void .ctor()|:Banking.Account
:Banking.Bank|Void .ctor()|:Banking.CurrentAccount
:Banking.CurrentAccount|Void .ctor()|:Banking.Account
:Banking.Bank|Void .ctor()|:Banking.CurrentAccount
:Banking.Bank|Double directdebit(Double)|:Banking.CurrentAccount
:Banking.SavingsAccount|Void .ctor()|:Banking.Account
:Banking.Bank|Void .ctor()|:Banking.SavingsAccount
:Banking.Bank|Double applyInterest(Double)|:Banking.SavingsAccount

But we won't get that at all unless we compile the instrumented application with the /d:TRACE option with this command:

> csc /d:TRACE Banking.cs

.and have a supporting application configuration file (named Banking.exe.config) with the following content:

<configuration>
<system.diagnostics>
<trace autoflush="true" indentsize="4">
<listeners>
<add name="myListener"
type="System.Diagnostics.TextWriterTraceListener"
initializeData=".\myListener.log" />
<remove type="System.Diagnostics.DefaultTraceListener"/>
</listeners>
</trace>
</system.diagnostics>
</configuration>

In that file we're telling the .NET Runtime to add a new listener for trace events, of type System.Diagnostics.TextWriterTraceListener, with the output destination specified as the file myListener.log. This listener will replace the default listener, which we've asked to be removed above.

So that's the basis of our solution comprising some instrumentation code, compilation with the d:/TRACE flag, and a supporting configuration file. Before moving on to the coding and demonstration I'll tell you a little more about each of those and about the optional Rational Rose scripting.

Instrumentation with System.Diagnostics classes

The "Introduction to Instrumentation and Tracing" within the "Visual Basic and Visual C# Concepts" section of the .NET documentation defines instrumentation as the ability for an application to incorporate:

  • Code tracing
  • Debugging
  • Peformance counters
  • Event logs

Those kinds of instrumentation are supported by the classes of the System.Diagnostics namespace. We'll be concentrating on code tracing and therefore the System.Diagnostics.Trace class. That said, the techniques discussed here could be extended to debugging, performance counters, and event logs ; also thanks to the Trace class, or by incorporating the Debug class from the same namespace.

The Trace and Debug classes have various static methods that allow you to write out messages to the output window, a text log file, or some other destination. Some of these methods are conditional, such as Trace.Assert(), and some are unconditional such as Trace.Write(). The code listed in the previous section uses the latter.

Trace output is not written out directly, but is intercepted by one or more trace listeners. You can add trace listeners into the Listeners collection, to supplement the DefaultTraceListener or to replace it completely. In code you could replace the default listener with new listener like this:

Trace.Listeners.Clear();
Trace.Listeners.Add(new TextWriterTraceListener(Console.Out));

In our solution we're not doing that because it would couple the tracing behavior too closely to the application code. Instead we're configuring the trace listener(s) via an application configuration file.

Application Configuration Files

Configuration files are XML files that can be used to change settings without recompiling applications. For an executable-hosted application (which is what my sample Banking application is) the configuration file is located in the same directory as the application. It has the same name plus a .config extension.

Banking.exe is the application.

Banking.exe.config is the corresponding configuration file.

As background information I'll tell you that configuration files for ASP.NET hosted applications follow a different scheme. In that case there will be a set of configuration files located at various levels in the URL path of the application. For an Internet Explorer application, the relevant configuration file will be specified by the <link> tag. You can read more about the various kinds of configuration files, for the various kinds of applications, at ms-help://MS.VSCC/MS.MSDNVS/cpguide/html/cpconapplicationconfigurationfiles.htm.

Because it is a standard XML file, an application configuration file has a schema to which the contents must conform. For Debug and Trace Settings, the schema elements (from ms-help://MS.VSCC/MS.MSDNVS/cpgenref/html/gngrftracedebugsettingsschema.htm in the .NET help documentation) are:

Element | Description
<add> element for the <listeners> element | Adds a listener to the Listeners collection.
<add> element for the <switches> element | Specifies the level where a trace switch is set.
<assert> | Specifies whether to display a message box when you call the Debug.Assert method; also specifies the name of the file to write messages to.
<listeners> | Specifies a listener that collects, stores, and routes messages.
<remove> | Removes a listener from the Listeners collection.
<swithces> | Contains trace switches and the level where the trace switches are set.
<system.diagnostics> | Specifies trace listeners that collect, store, and route messages and the level where a trace switch is set.
<trace> | Contains listeners that collect, store, and route tracing messages.

In our solution we're making use of the <system.diagnostics>, <trace>, <listeners>, <add> and <remove> elements. We're not using <switches> in our solution, so for completeness I'll just make you aware of what a trace switch is.

The System.Diagnostics.TraceSwitch class provides a multilevel switch to control tracing and debug output without recompiling your code, and you can use the trace levels to filter out messages based on their importance. Switches may be added and removed in the application configuration file, as with listeners.

In summary, the purpose of an application configuration file is to allow certain behaviors - including tracing - to be changed without requiring the application to be recompiled.

Compiling with the /d:TRACE flag

To enable tracing in a C# application you can either add a preprocessor directive #define TRACE to the top of the source file or add the d:/TRACE flag to the compiler command line. In this case study we're doing the latter, and you might like to try compiling with and without that flag to see the effect.

If you're working with Visual Basic you can enable tracing in a similar way by adding the /d:TRACE=True flag to the compiler command line.

If you preference is to compile from Visual Studio .NET itself, rather than from the command line, it's worth keeping in mind that by default the IDE compilation has tracing enabled. So unless you change it, you're released application could well be pumping out tracing information that you don't want it to. You can change this behavior via the Project Properties dialog, in the Configuration Properties / Build / Code Generation entry.

Rational Rose Scripting

Rational Rose has been one of the most widely used Object Modeling CASE tools in recent years, and as far back as I can remember it has had a scripting language that provides programmatic access to the Rose application and the model(s) housed within it.

The scripting language is called Summit BasicScript, and if you're familiar with early versions of Visual Basic (many visitors to this site, I suppose) you'll be quite comfortable with it. Access to the UML model (actually the meta-model) is provided by the Rose Extensibility Interface (REI) classes that represent the model elements - classes, methods, and so on - as well as the UML diagrams.

We'll make use of those classes via the scripting language, but the same set of classes is available for manipulation from your favorite Microsoft language via the OLE/COM objects of Rose Automation. If that's your preference, it shouldn't be too difficult to convert the script to a VB program, or for that matter even a C# program.

You can find more information about the scripting language and the extensibility classes in the Rose Entensibility Interface / Summit BasicScript Reference and Rose Extensibility Interface / Rose Extensibility Reference Rational Rose help topics.

Component Design and Coding

As described above, the key to our solution will be a few lines of instrumentation code inserted at the start of each method in the original program. Of course, you don't want to go through your program pasting that code into each and every method do you? So we'll automate it with a C# program.

We'll then implement a Rational Rose script to import the trace results and to build a UML sequence diagram reflecting the sequence of interactions between objects - i.e. the scenario - of the running program. That part's not compulsory, but quite interesting.

Automating Insertion of the Instrumentation Code

The following code of the InsertInstrumentation class will be used to read in any C# program and write it out again with instrumentation code included.

using System;

We'll make use of System.IO so as to read the original source file, and System.Text.RegularExpressions to help with parsing of that file. If you've read other articles that I've written you'll have been expecting regular expressions to appear somewhere, and - if not - you can get up to speed with them by reading my article "Introducing .NET Regular Expressions with C#" at http://www.csharptoday.com/content.asp?id=1786&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq.

using System.IO;
using System.Text.RegularExpressions;

Our class will be called InsertInstrumentation, housed within the reverse namespace:

namespace reverse
{
public class InsertInstrumentation
{

In the main method we open the original source file and call the instrument() method to insert the appropriate instrumentation:

        public
            static void Main(string[] args)
{
String
            inputString="";
String inputLine;

StreamReader sr =
            File.OpenText(args[0]);

while
            ((inputLine=sr.ReadLine())!=null)
inputString=inputString+"\n"+inputLine;

sr.Close();

instrument(inputString);
}

The instrument() method

The instrument() method is where we read the text of the input file, then write it out again with the instrumentation code inserted into each method:

private static void instrument(String text)
{

Our output file will contain modified source code with instrumentation. That output will only compile if we start by importing the System.Diagnostics namespace classes:

System.Console.WriteLine("using System.Diagnostics;");

As we go, we'll be keeping track of the previous match of the regular expression:

Match previousMatch=null;

Here we set up the regular expression itself. In a nutshell it matches each method declaration, which it reads into a named group (think of it as a variable) called "<method>", and any preceding code which is read into a named group "<priorcode>".

string regex=@"(?s)(?<priorcode>.*?)"
+@"(?<method>(public|private)"
+@"\s+[\w\s]*\w+\s*\(.*?\)\s*\{)";
Regex methodRegex=new Regex(regex);

That regular expression is not perfect because it handles only those methods that are declared explicitly as public or private. Once you become familiar with regular expressions - perhaps via my earlier article - you may wish to revisit this to account for protected and internal methods, plus those with no explicit access modifier.

Now we'll match the regular expression against the input text (the original source code) and step through the matching portions:

            Match
            methodMatch = methodRegex.Match(text);

for ( ;
            methodMatch.Success;
methodMatch =
            methodMatch.NextMatch())
{

For each match, we'll pick out the priorcode and the method declaration, and we'll write out the priorcode:

string priorcode=""+methodMatch.Groups["priorcode"];
string method=""+methodMatch.Groups["method"];
System.Console.WriteLine(priorcode);

Next we write out the method declaration along with the instrumentation code, via a separate method named writeInstrumentation(). We record this match as previousMatch and go round again:

writeInstrumentation(method);
previousMatch=methodMatch;
}

Because of how the regular expression works, we'll be left with some code at the end that we've not yet written out. Here we tidy up by writing the remaining code:

            if
            (previousMatch!=null && previousMatch.Success)
{
int
            remainingCodeStart=previousMatch.Index
+previousMatch.Length;
int
            remainingCodeLength=text.Length
-remainingCodeStart;
System.Console.WriteLine(text.Substring(remainingCodeStart
,remainingCodeLength));
}
}

You will have noticed in the above code that - upon discovering each match of the regular expression - we call a writeInstrumentation() method to actually write out the instrumentation code.

The writeInstrumentation() method

Here is that method, which takes in the method signature text as its sole parameter:

private static void writeInstrumentation(string method)
{

We write out the method signature itself as part of the modified source code, along with a comment marking the start of the instrumentation:

System.Console.WriteLine(method);
System.Console.WriteLine("\n// ** START INSTRUMENTATION **");

Now we write the instrumentation code that was explained previously in the Defining the Solution section:

System.Console.WriteLine(
"StackTrace stackTrace=new StackTrace();");
System.Console.WriteLine(
"StackFrame topFrame=stackTrace.GetFrame(0);");
System.Console.WriteLine(
"StackFrame nextFrame=stackTrace.GetFrame(1);");

System.Console.WriteLine("String __callingObject=\":\";");
System.Console.WriteLine(
"if (nextFrame!=null) "
+"__callingObject=\":\"+nextFrame.GetMethod().DeclaringType;");

System.Console.WriteLine(
"System.Diagnostics.Trace.Write(__callingObject);");
System.Console.WriteLine( "System.Diagnostics.Trace.Write(\"|\"+topFrame.GetMethod());");
System.Console.WriteLine(
"System.Diagnostics.Trace.WriteLine(\"|:\"+topFrame.GetMethod().DeclaringType);");

And we mark the end of the instrumentation with a comment:

            System.Console.WriteLine("\n//
            ** END INSTRUMENTATION **");
}

}
}

Remember that the result will be the original source file unchanged, save for the inclusion of instrumentation code at the start of every method (including constructors) like this:

public double applyInterest(double
            rate)
{

// ** START INSTRUMENTATION **
StackTrace
            stackTrace=new StackTrace();
StackFrame
            topFrame=stackTrace.GetFrame(0);
StackFrame
            nextFrame=stackTrace.GetFrame(1);
String
            __callingObject=":";
if (nextFrame!=null)
            __callingObject=":"+nextFrame.GetMethod().DeclaringType;
System.Diagnostics.Trace.Write(__callingObject);
System.Diagnostics.Trace.Write("|"+topFrame.GetMethod());
System.Diagnostics.Trace.WriteLine("|:"+topFrame.GetMethod().DeclaringType);
//
            ** END INSTRUMENTATION **

Balance=Balance*rate;
return
            Balance;
}
}

When we compile and run the instrumented application, as I'll show later, we'll get the logged output shown previously in Defining the Solution. That application trace may be further processed or displayed in a number of ways, one of which is as a UML sequence diagram in Rational Rose.

The Rational Rose Script (importScenario.ebs)

If you're using Rational Rose you'll no doubt find this script very interesting, and - if not - you may skip this discussion. But if you're working with another UML modeling tool you might just want to look through it anyway as it will give you some ideas about what you could do via the scripting language of your own tool.

In the Main subroutine we ask for an input file (the log file produced by the instrumentation) via a file dialog. We then pass the file name into the ImportScenario subroutine:

Sub Main
FileName$ = OpenFileName$ ("Import", "All files:*.*")
If FileName$ <> "" Then ImportScenario FileName$
End Sub

The ImportScenario subroutine follows. I've used the term scenario because, in UML terminology, a scenario is what we'll be presenting as a sequence diagram. You can think of a scenario as a usage pattern for application under investigation:

Sub ImportScenario (FileName As String)

Here we get hold of the current Rational Rose model, so that we can add to it:

Dim MyModel As Model
Set MyModel = RoseApp.CurrentModel

We'll be creating a new scenario (sequence) diagram with the same name as the log file containing the application trace, e.g. myListener.log or - more meaningfully - Banking.log, but only if that scenario does not already exist:

scenarioID = MyModel.RootCategory.ScenarioDiagrams.FindFirst (FileName)
Dim sDiagram As ScenarioDiagram
If scenarioID <> 0 Then
Set sDiagram = MyModel.RootCategory.ScenarioDiagrams.GetAt (scenarioID)
Else
Set sDiagram = MyModel.RootCategory.AddScenarioDiagram (FileName,1)
End If

Our sequence diagram will comprise object instances that exchange messages, with each instance having a unique ID. These variables will be used to keep track of the instances.

Dim instanceIDs() As String
ReDim instanceIDs(1)
instanceIDs(1)=""
Dim theInstances As New ObjectInstanceCollection

Note that our output log file supports the notion of sender and receiver objects of the form instanceID:ClassName, but in this case study we're actually not recording unique instance IDs for each object. The log file contents will always refer to objects as :ClassName and the resulting sequence diagram will be as shown earlier, with anonymous instances as preferred by many object modelers. If we improve the instrumentation in future to include instance IDs, this script will be ready.

Each message in our sequence diagram will have an ordinal sequence number, so here we start with number 1:

seq%=1

We open the viewport window to report the progress of the script and we open the input file for read-only access:

Viewport.Open
Open FileName$ For Input Access Read As #1

Now we step through the file, reading each line as we go:

While Not EOF (1)
Line Input #1, inString$

Each entry in the file will contain three pipe-separate fields: the sender (caller), the message (method), and the receiver (callee):

caller$=Item$(inString$,1,1,"|")
callee$=Item$(inString$,3,3,"|")
method$=Item$(inString$,2,2,"|")

The caller and callee objects could be identified by an instance ID and a class name, separated by a colon. Remember that for this case study, the instance IDs will actually be blank, but the script will take them if they're there:

callerID$=Item$(caller$,1,1,":")
callerClass$=Item$(caller$,2,2,":")
calleeID$=Item$(callee$,1,1,":")
calleeClass$=Item$(callee$,2,2,":")

For simplicity we'll put the two classes into the root category (root package) of the model, if they don't already exist there. A more advanced script could build a category (package) hierarchy reflecting the namespaces of the classes:

Dim thisCat As Category
Set thisCat=MyModel.RootCategory

If thisCat.GetAllClasses().FindFirst(callerClass$) = 0 Then
Set thisClass=thisCat.AddClass (callerClass$)
End If

If thisCat.GetAllClasses().FindFirst(calleeClass$) = 0 Then
Set thisClass=thisCat.AddClass (calleeClass$)
End If

Each object instance will be uniquely identified by a combination of its ID (not present in our log file) and its class, plus we'll use two variables to hold the caller and callee instances respectively.

callerInstanceID$=callerID$+":"+callerClass$
calleeInstanceID$=calleeID$+":"+calleeClass$
Dim callerInstance As ObjectInstance
Dim calleeInstance As ObjectInstance

We'll look to see if the caller instance has been recorded in the instanceIDs list and is therefore already on the diagram. We don't want to add a new object timeline every time this instance is the sender of a message:

foundCallerPos%=0
size%=UBound(instanceIDs)
For pos%=1 To size%
If instanceIDs(pos%)=callerInstanceID$ Then
foundCallerPos%=pos%
End If
Next pos%

If the caller instance is not already on the diagram, we add it to the diagram and also to the list of known instances:

If foundCallerPos% = 0 Then
Set callerInstance=sDiagram.addInstance(callerID$,callerClass$)
ReDim Preserve instanceIDs(size%+1)
instanceIDs(size%+1)=callerInstanceID$
foundCallerPos%=size%+1
theInstances.Add callerInstance
End If

Same again for the callee instance. Add it to the diagram if it's not already there:

foundCalleePos%=0
size%=UBound(instanceIDs)
For pos%=1 To size%
If instanceIDs(pos%)=calleeInstanceID$ Then
foundCalleePos%=pos%
End If
Next pos%
If foundCalleePos% = 0 Then
Set calleeInstance=sDiagram.addInstance(calleeID$,calleeClass$)
ReDim Preserve instanceIDs(size%+1)
instanceIDs(size%+1)=calleeInstanceID$
foundCalleePos%=size%+1
theInstances.Add calleeInstance
End If

Make sure we have a handle on the sequence diagram representations of the caller - and callee - instances, then add a message (the method signature) from one to the other:

Set callerInstance=theInstances.GetAt(foundCallerPos%-1)
Set calleeInstance=theInstances.GetAt(foundCalleePos%-1)

Dim theMessage As Message
Set theMessage=sDiagram.CreateMessage(method$,callerInstance, calleeInstance, seq%)

Give the user some feedback in the viewport window, increase the sequence number for next time around, and complete the loop:

Print callerClass$,"---",method$,"-->",calleeClass$
seq%=seq%+1
Wend

Finally, close the input file nicely and end the subroutine:

Close #1
End Sub

Sample Application

In this section we'll take a sample application, instrument it, and view the trace results as a simple list. Then we'll reverse engineer a sequence diagram representation using the Rose script.

You'll find everything you need in the download, which contains the following files. Some of them will be obvious immediately and the purpose of the others will become clear as you read on.

Download Contents

File Banking.cs contains the source code for the sample application (see below).

File Banking.exe is a compiled version of the sample application.

File Banking.save is a spare copy of the Banking.cs source code, just in case.

File importScenario.ebs is the Rational Rose script that builds the UML sequence diagram.

File InsertInstrumentation.cs is the source code of the program to add instrumentation.

File InsertInstrumentation.exe is the compiled version.

File instrumented\Banking.cs will be the instrumented version of the sample application.

File instrumented\Banking.exe is the compiled version.

File instrumented\Banking.exe.config is the application configuration file for the sample application.

File instrumented\myListener.log is where the trace results will be written.

File build.bat contains the command csc /d:TRACE Banking.cs to compile the instrumented sample application.

File sampletraces\BankingTrace.mdl represents the sample trace as a Rational Rose model.

File sampletraces\BankingTrace.xls represents the sample trace as an Excel speadsheet.

File sampletrace\BankingTrace.xml represents the sample trace as an XML file, for Internet Explorer.

Banking Application Code

If you've read my previous (Static) Reverse Engineering case study at http://www.csharptoday.com/content.asp?id=1742&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq you'll be familiar with my sample Banking program. I've changed it slightly for this case study and the latest version is provided in file Banking.cs of the code download.

I wont give a blow-by-blow account of every single line of code because that's not important, but here is the complete code with a few useful pointers.

using System;

namespace Banking
{

The Account class will be a base class for various types of account, such as CurrentAccount and SavingsAccount. We won't be concerned with the member variables and properties but the constructor will be instrumented:

    public class
            Account
{
private static int
            _nextAccountNumber;

private int _accountNumber;
private
            double _balance;

public
            Account()
{
_accountNumber=_nextAccountNumber;
_nextAccountNumber++;
}

public
            int AccountNumber
{
get { return _accountNumber;
            }
}

public double Balance
{
get { return _balance;
            }
set { _balance=value; }
}
}

The CurrentAccount class extends Account. It will have its constructor and its directdebit() method instrumented:

    public class
            CurrentAccount : Account
{
public
            CurrentAccount()
{
}

public double directdebit(double
            amount)
{
Balance=Balance-amount;
return
            Balance;
}
}

The SavingsAccount class extends Account. It will have its constructor and its applyInterest() method instrumented:

    public class
            SavingsAccount : Account
{
public
            SavingsAccount()
{
}

public double applyInterest(double
            rate)
{
Balance=Balance*rate;
return
            Balance;
}
}

The Bank class has a Main() method to drive the other classes through a typical sequence of events, i.e. a scenario:

public class Bank
{
public static void Main(string[] args)
{
// Current Account #0
CurrentAccount currentAccount=new CurrentAccount();
currentAccount.Balance=100;
System.Console.WriteLine("Current Account "
+currentAccount.AccountNumber
+" has balance "+currentAccount.Balance);

// Current Account #1
currentAccount=new CurrentAccount();
currentAccount.Balance=100;
currentAccount.directdebit(50);
System.Console.WriteLine("Current Account "
+currentAccount.AccountNumber
+" has balance "+currentAccount.Balance);

// Savings Account #1
SavingsAccount savingsAccount=new SavingsAccount();
savingsAccount.Balance=1000;
savingsAccount.applyInterest(1.1);
System.Console.WriteLine("Savings Account "
+savingsAccount.AccountNumber
+" has balance "+savingsAccount.Balance);
}
}
}

Just to clarify, the sequence of events is:

  • Create a CurrentAccount.
  • Create another CurrentAccount and call its directdebit() method.
  • Create a SavingsAccount and call its applyInterest() method.

Instrumenting the application

If you don't have an InsertInstrumentation.exe file, create one by compiling the InsertInstrumentation.cs C# program:

> csc InsertInstrumentation

Now run the InsertInstrumentation program against the Banking.cs source file, taking care to divert the output into the \instrumented directory.

> InsertInstrumentation Banking.cs > instrumented\Banking.cs

For convenience, if you accidentally overwrite the original Banking.cs file with the instrumented version (yes, I've done that!) you should be able to recover by renaming file Banking.save to be Banking.cs.

You might like to take a look at the instrumented version of the application source code before moving on.

Compiling the application

Compile the application in the \instrumented directory with tracing enabled, like this:

> cd instrumented
> csc /d:TRACE Banking.cs

At this point it's worth noting that the \instrumented directory contains a file called Banking.exe.config, which was not generated by the compilation but written in advance by me. You can use my file as the starting point of a configuration file of your own applications by simply renaming it to YourApplication.exe.config. I explained the contents of the file earlier, but as a reminder here it is again:

<configuration>
<system.diagnostics>
<trace autoflush="true" indentsize="4">
<listeners>
<add name="myListener"
type="System.Diagnostics.TextWriterTraceListener"
initializeData=".\myListener.log" />
<remove type="System.Diagnostics.DefaultTraceListener"/>
</listeners>
</trace>
</system.diagnostics>
</configuration>

Note that this application configuration file specified the file myListener.log to take the output of the TextWriterTraceListener.

Running the application

Now we can run the instrumented application and see its output:

> Banking
Current Account 0 has balance 100
Current Account 1 has balance 50
Savings Account 2 has balance 1100

For this case study we're not interested in the output from the program itself, but in the contents of the myListener.log file which are:

:|Void Main(System.String[])|:Banking.Bank
:Banking.CurrentAccount|Void .ctor()|:Banking.Account
:Banking.Bank|Void .ctor()|:Banking.CurrentAccount
:Banking.CurrentAccount|Void .ctor()|:Banking.Account
:Banking.Bank|Void .ctor()|:Banking.CurrentAccount
:Banking.Bank|Double directdebit(Double)|:Banking.CurrentAccount
:Banking.SavingsAccount|Void .ctor()|:Banking.Account
:Banking.Bank|Void .ctor()|:Banking.SavingsAccount
:Banking.Bank|Double applyInterest(Double)|:Banking.SavingsAccount

That shows a complete trace of the constructors and other methods that have been invoked during the execution of the application. Each line shows the calling object (blank on the first line), the message that was sent (or method invoked), and the receiving object.

It's not the most eye catching representation, so to make it clearer I simply opened the log file in Microsoft Excel and applied a little formatting to the columns to give:

In the Case Study Review section I'll offer some more thoughts on presenting and analysing the application trace.

Building the UML Sequence Diagram

If you have access to Rational Rose you can use my script to build a real UML sequence diagram for the scenario represented by the application trace.

First, launch Rose and use the Tools / Open Script menu option to open the importScenario script; or simply double-click the script file to launch Rose with the script already loaded. You can then run the script by pressing the green "Start" button.

Choose the file that contains the logged trace results, i.e. myListener.log:

Then watch the messages that appear in the Viewport window:

Finally you will discover that the Rose model browser contains a set of classes, and a new sequence diagram representing the application trace, as shown here:

That completes the sample application, but did you notice anything odd about the sequence of events? According to that trace it looks as though the Banking.CurrentAccount calls its superclass Banking.Account constructor before it has had its own constructor invoked. This pattern is repeated for every subclass / super class construction and there's not much we can do about it.

To understand what's going on here you have to imagine that the subclass constructor as an implicit first line of code that invokes the constructor of the superclass, and which --in effect - causes the superclass instrumentation to be executed before its own instrumentation. This following diagram might help:

Apart from that, the sequence of events is ordered as you would expect for the sample application.

Case Study Review

In this case study we've investigated a technique for instrumenting a C# application, which allows us to capture the runtime - i.e. dynamic - behavior of the application. The resulting application trace may be useful as a teaching aid (to see demonstrate objects really interact), and for debugging or performance monitoring (if we include method timings in the instrumentation code).

Ultimately it allows us to reverse engineer a UML sequence diagram to complement the reverse engineered class diagrams demonstrated in my previous case study http://www.csharptoday.com/content.asp?id=1742&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq. In a typical project you may well draw sequence diagrams for your applications up-front to describe the intended behavior, and now we have a way of validating those diagrams against the actual behavior during testing.

For the instrumentation we used mainly the System.Diagnostics.Trace class, and the System.Diagnotics.StackTrace class to discover the calling (caller) and called (callee) objects. We automated the insertion of instrumentation code into every method by implementing the InsertInstrumentation program. Through that program I demonstrated a practical use for regular expressions.

We looked at how tracing could be enabled and disabled with the /d:TRACE compiler flag and how the tracing behavior could be configured outside of the target application in an application configuration file.

With the help of a Rational Rose script I showed how the end results really could be rendered in UML, and finally I used the sample sequence diagram to highlight an interesting feature about the order of class construction.

During the review phase for this case study, one of the reviewers made a very interesting point that this approach should be a method of last resort. Surely it would be better to insert trace statements as you build an application rather than as an afterthought? My own opinion is that it would be less flexible, i.e. hard-coded, and possibly inconsistent across the application, but it's certainly something you should be thinking about as part of the design.

Notwithstanding the recommendations I've given in the next section, the important messages to take away from this case study are:

  • You can capture information about the behavior of your application as it runs, i.e. you can capture an application trace, using the classes of the System.Diagnostics namespace and the techniques described here. But there may be other ways, mentioned in "Non-invasive Tracing" below.
  • The information you capture may be useful for debugging purposes, for measuring performance (if we included counters in the instrumentation), or simply as a learning tool. Remember the order-of-construction effect that was highlighted in the example?
  • Ultimately you can reverse engineer the runtime (dynamic) behavior of an application and present it as UML; to complement any static reverse engineeringyou may do, or to validate the application designer's original dynamic model.

Remember that although we've used C# in this case study, in .NET all languages are equal. So it shouldn't be too difficult to adapt this approach to work with Visual Basic, C++ or even J# applications.

Limitations and Further Work

In this case study we've made a good start on the problem of reverse engineering the behavior of an application as it runs. In principle it works, and there are a few improvements that I can think of that you might like to work on.

Intermediate File Format

I kept the intermediate file format quite simple, mainly because my Rose scripting skills are not up to parsing a more complex structure such as XML. But I do think that XML would be a sensible choice for the intermediate results, so its something you might like to think about. I envisage an XML file like this:

<trace>
<interaction seq="1">
<sender>:</sender>
<message>Void Main(System.String[])</message>
<receiver>:Banking.Bank</receiver>
</interaction>

<interaction seq="2">
<sender>:Banking.CurrentAccount</sender>
<message>Void .ctor()</message>
<receiver>:Banking.Account</receiver>
</interaction>
</trace>

The rendering in Internet Explorer would be:

Processing the Results

We've processed the trace results by loading them into an Excel spreadsheet and by constructing a UML sequence diagram in Rational Rose. By writing out the trace as XML (above) you could open them up to a wider range of analysis or presentation tools, or maybe write a dedicated trace viewer yourself.

Remember that we've decoupled the trace handling from the application itself through the use of an application configuration file. In that file we specified a TextWriterTraceListener for writing the results to a log file, which replaced the default trace listener that would write the results as console output.

You can write your own TraceListener classes and add them in the same way, thus opening up a whole new set of possibilities. What about a trace listener that sends out each trace event as a SOAP message to a remote real-time debugging or profiling tool?

Alternative Instrumentation

As implemented here we've made use of the System.Diagnostics.StackTrace class to obtain information on the calling class, the called class, and the method that was invoked, and we've written that information out via the System.Diagnostics.Trace.Write() method. In a typical method, the instrumentation code looks like this:

public double applyInterest(double
            rate)
{

// ** START INSTRUMENTATION **
StackTrace
            stackTrace=new StackTrace();
StackFrame
            topFrame=stackTrace.GetFrame(0);
StackFrame
            nextFrame=stackTrace.GetFrame(1);
String
            __callingObject=":";
if (nextFrame!=null)
            __callingObject=":"+nextFrame.GetMethod().DeclaringType;
System.Diagnostics.Trace.Write(__callingObject);
System.Diagnostics.Trace.Write("|"+topFrame.GetMethod());
System.Diagnostics.Trace.WriteLine("|:"+topFrame.GetMethod().DeclaringType);
//
            ** END INSTRUMENTATION **

In fact, the instrumentation code looks exactly like that, regardless of the method it's in.

Now think about an alternative instrumentation that is method-specific and which therefore sends out some method related information. like the parameter values. I'm thinking about a line of code like this inserted into the method above.

System.Diagnostics.Trace.WriteLine(
"|applyInterest( rate="+rate+" )");
);

That information would be quite useful and you might like to think about inserting such lines of code via the InsertInstrumentation program. I think the contents of the regular expression's <method> group will tell you everything you need to know about the parameter names.

Finally in terms of improved instrumentation, you could think about how to identify the actual object instance on which the method has been invoked; not just the class. Remember that the Rational Rose script can cope with individually-identifiable instances when building the sequence diagram.

Non-Invasive Tracing

The approach I've described decouples the tracing behavior from the application code thanks to the application configuration file. The tracing can be enabled or disabled at the flick of a compiler switch, and a lot of typing is saved by the automated insertion of the instrumentation code.

All that said, this approach is essentially invasive; which means we must modify the original application code. In contrast, the runtime reverse engineering solutions that I devised using Java were non-invasive, using the Java Platform Debugger Architecture (JPDA) for applications in general and using sevlet filters and event listeners for web applications specifically.

Non-invasive techniques for tracing of .NET applications are worth investigating. You could try driving the Machine Debug Manager using DCOM, or maybe take advantage of Windows Management Instrumentation (WMI). I'm no expert on the latter, but it looks promising and you can read Richard Conway's article on that technology at http://www.csharptoday.com/content.asp?id=1807&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq. Either way, my past experiences tell me that by using these non-invasive techniques it could be much more difficult to capture the method-specific information that I described above.

Case Study Information
Author | Tony Loton
Technical Editor | Adam Ryland
Project Manager | Helen Cuthill
Reviewers | Andrew Krowczyk, Saurabh Nandu, John Hodgson, Adam Ryland

If you have any questions or comments about this case study, or suggestions for further pieces, please contact the technical editor. If you wish to talk about the case study with other subscribers, please visit our forum here.


timeTrack : An ASP.NET Contract Management System

timeTrack : An ASP.NET Contract Management System
http://www.csharptoday.com/content.asp?id=1840
Article Rating: 4
Published on May 13, 2002
Abstract
ASP.NET has brought with it a huge leap forward in the way that web applications are designed and developed. The following case study by Brian Boyce takes you from business requirements to completed application, focusing on the various technologies and techniques that the .NET Framework provides to simplify implementation and to dramatically reduce the development lifecycle.
Case Study

ASP.NET has brought with it a huge leap forward in the way that web applications are designed and developed. The following case study by Brian Boyce takes you from business requirements to completed application, focusing on the various technologies and techniques that the .NET Framework provides to simplify implementation and to dramatically reduce the development lifecycle.

Introduction

The case study revolves around a simple and prevalent problem in many organizations; accurate measurement and charging of resources. The application presented in this case study is a rewrite of a portion of a larger resource management system. In order to distinguish the two, the application presented in this case study has been called timeTrack Demo. The material presented in the case study relates exclusively to the demo version, with the exception of the Defining the Solution section which contains a little background on the applications history. The application aims to solve the following problems:

  • How much is my internal development costing me?
  • Which contracts are most profitable?
  • Ability to support all service oriented charging (accountability).

This case study will walk through the development of an ASP.NET application in C# that satisfies these requirements. The case study presents a simple application architecture that can be re-used for rapid development and prototyping of departmental and simple line-of-business applications. The technologies discussed in the development of the application include:

  • Web Forms
  • Writing User Controls
  • Custom Forms based Security
  • Implementing role based security
  • Creation of a site Navigation Control
  • ASP.NET Validation Controls and regular expressions
  • The use of HttpContext

These will be presented at the appropriate juncture in the development process.

System Requirements

The article assumes a basic familiarity with C#, ASP.NET and Visual Studio. In order to run the timeTrack Demo application you will need to have the ASP.NET V1.0 Framework installed, IIS 5 or later, and either SQL Server 2000 or MSDE 2000.

Defining the Problem

The client is a fictional software house, ABC Consulting, who provides business consultancy services, off-the-shelf products and consultancy lead development of bespoke software for a number of clients throughout the world.

As their business has grown and the number of contracts increased, they are finding it increasingly difficult to track the amount of time consultants are working on each contract. In addition, they believe that while many of their consultants are logging a single day, they are in fact working through lunch breaks and often into the evening. All of this additional time is going unrecorded and is therefore not being charged to their clients resulting in lost revenue for the company. In addition, the company is unable to monitor (and reward) its highest revenue generating consultants.

The defining requirements are as follows:

  • ABC Consultants must be able to log their time remotely
  • The system must be simple to use and require minimal input from each user
  • The application must enable the logging of time against contract for profitability tracking, enabling ABC to charge for days that are currently slipping through as part days. A combination of lost half hours across 40 consultants could amount to significant lost revenue.
  • The application will also enable ABC as a company to monitor the cost of different internal development and project work; which in turn will help them focus on truly profitable work streams.

The specific functional requirements will be explained as I describe the data model underpinning timeTrack.

Defining the Solution

As mentioned earlier, the timeTrack Demo application presented here is a simplified subset of a larger corporate intranet application. The original application was written in C# using the .NET Framework Preview Release and upgraded through Beta 1, Beta 2, and Version 1. The choice to implement the timeTrack Demo application in ASP.NET followed on from this. I now want to discuss some of the alternative implementation tools considered and what lead to me choosing to implement the application with ASP.NET.

The nature of the application was ideally suited to a web application; as such, the initial focus was on UltraDev and ASP. However, shortly after working with the ASP+ Preview (later called ASP.NET) I saw the enormous potential this technology had for shortening the development cycle of web and windows applications. The key features that drove the decision to use ASP.NET on this particular project were:

  • The separation of function from form (code behind, user controls and custom controls).
  • A clean and consistent object oriented platform.
  • The Framework classes which provided an enormous functional resource, making the grunt work an integration exercise rather than a large development effort.

The separation of function from form was a key factor in the decision; not only did it allow the developer to focus on the business functionality through the support of the framework classes but through the use of web controls the developer did not need to understand the nuances of cross browser HTML and scripting (though this does become a distinct advantage in more complex projects). The separation of UI also allowed the development to be split between graphic designers and developers in a way that had been almost impossible in ASP. The more development I did with ASP.NET, the more I realized that these were the most obvious advantages, but by no means the most compelling.

Database Design

The design implemented can be broken into two distinct areas:

  • Security
  • Contract Management

Diagram 1 shows the table structure.

Diagram 1: timeTrack Database Diagram

Database structure

Security

Both windows and forms mode authentication are well suited to this application, and I had initially decided on windows authentication. However, as I began to work through the application design, it became clear that forms based authentication better suited timeTrack for the following reasons:

  • The application is suited to small software and consultancy companies, many of whom will not have a dedicated web server (often subscribing to a shared hosting package). They are therefore not able to manage NT users and groups as easily.
  • The obvious extension of this application is to enable clients to submit requests for resources and view invoices on-line. In general, use the system as a means of simplifying and reducing the overall cost of communicating with your clients.

This case study presents the implementation using a custom forms based security model. The model presented allows us to store user and role information within the database. The four key roles are as follows (a user can be a member of 0 or more roles):

Role | Description
Administrator | The Administrator is responsible for maintenance of users and role membership.
ContractAdmin | The ContractAdmin is more of a business role, and involves maintaining user profiles, contract, client, and country definitions.
Consultant | Consultants are all those users who are able to record their time against a Contract.
Analyst | The Analyst has access to reports and historical analysis but is not able to alter the definition of the business entities nor record time against a Contract.

The Users table is used to store information about each of the registered application users. The Resources table is used to store information about resources that can be charged to a contract. The decision to separate the Users information from the Resources table was taken for three main reasons. Firstly, it separates the security from the functional entities, making it easier to re-use both the security model and contract data model in other web applications. Secondly, and more importantly, there may be a need for users who cannot log time against a contract, such as analysts and administrators (or in the future clients and customers) logging onto the system in order to generate invoices, requests or reports. The final reason is that although Resources in the current business model are people, in the future a requirement may arise to track the use of other resources and bill for their usage, e.g. Conference facilities, etc. These reasons also help explain why there is no reference between the Users and Resources tables, i.e. Resources are not necessarily people (e.g. Conference Facility), and Users are not necessarily Resources (e.g. Administrator).

The Roles for each user are defined in the Roles table and the UserRoles table is used to represent a many-to-many relationship that exists between Users and Roles. The only notable setting here is the definition of delete referential integrity on the relationship between the Users and UserRoles tables and the Roles and UserRoles tables. This setting ensures that any deletion of records from the primary key table in the relationship results in the deletion of the associated records in the foreign key table. See figure 1, which shows the Roles table relationship property window.

Figure 1: Configuring Delete Referential Integrity

The stored procedures for these tables are fairly straightforward; those pertinent to particular core functionality will be discussed in the component design section.

Defining a Contract

The Country, Category and Client tables are simple informational tables used to provide additional information about these entities. The primary purpose for the normalization of these tables is to ensure that analysis of ChargeableTime information can be done by Country, Category and Client in a consistent and reliable way. As such, their use and structure is self-explanatory. The code has been included in each table in order to simplify the creation of dimension tables for the creation of OLAP Cubes, if required.

The Country table is used to store information about the country a contract relates to.

The Category table can be used by ABC Consulting to categorize their contracts in any way they like. This table was included to provide a further user-definable level of grouping to assist their analysis.

The Client table is used to store information about the client the contract relates to.

The interesting tables are Contract, Resource and ChargeableTime. The Contract table, as you might expect, contains a single record for each Contract. Each of the fields in this table, its purpose and use are defined in table 1 below.

Table 1: Defining a Contract

Field | Use
Code | User-definable account code, which can be used to uniquely identify a Contract in dimension tables.
Description | Contract title or description for reports
CategoryID | Foreign key reference to the Category table.
PurchaseOrderNum | The purchase order number associated with a Contract. This is included to allow for direct interfacing with an accounts package or automatic generation of client invoices.
CountryID | Foreign key reference to the Country table.
Period | The period is used to identify how the contract is charged. The acceptable values are: H - Hourly D - Daily FP - Fixed Price The hourly and daily setting for this field are used in conjunction with StandardRate to determine how much to charge the client Per Hour (Period of 'H') or Per Day (Period of 'D').
IsExternal | Is this an internal or an external contract. This allows ABC Consulting to track the cost of internal work such as research, development of off-the-shelf products, etc.
FixedPrice | The fixed price as stipulated in the contract. Note time is still recorded against the contract to enable ABC to track how profitable and how accurate their estimates are for Fixed Price work.
StandardRate | Is the monetary rate per period to charge the client.
OutOfHoursMultiplier | This multiplier can be used to reflect any special rates negotiated in the contract for overtime or unsociable hours etc. A value of 1.5 results in the client being charged time-and-a-half for any out of hours work undertaken by the ABC Resources.
WeekendMultiplier | Is similar to OutOfHoursMultiplier and can be used to reflect any special weekend rates applicable to the contract.
ContractDay | This is used to reflect the length of a work day in hours as negotiated in the contract. This allows you to accurately charge for different contracts where the number of hours in a work day is different. ABC can use this to offer discounted rates for 7.5 hour days etc.
InitialDays | The number of agreed days in the contract. This field can be used to track man-days bought by a client, and allows the project managers to track the burn rate of contract days and renegotiate or extend the contract with the client before the total number of days is exhausted. See the discussion of highlighting DataRows in ContractAdmin.ascx user control.
StartDate | The start date of the term of the contract.
EndDate | The end date of the term of the contract.
ClientID | Foreign key reference to the client details relating to this contract.
IsDeleted | Is used to flag a Contract as inactive. It controls whether or not the Contract is available in the drop down list box presented to Consultants when they log their time.

The stored procedures that surround the Contract table perform the usual SelectAll, SelectSingle, Insert, Delete and Update. The SelectAll stored procedure called GetContracts returns not only the details of the Contract definition, but also the number of Days remaining on the Contract (based on the information in ChargeableTime and the initial days field). The provided implementation of this calculation applies the multiplier to the hours, e.g. 19:00 - 20:00 on a Saturday is calculated as 1 * weekend mult. * out of hours mult. This field could alternatively have been implemented by a series of triggers on the ChargeableTime table, however the calculation of the information only when it is requested rather than on each entry to the ChargeableTime is far more efficient.

The DeleteContractItem stored procedure does not delete the record from the table (though I have included the necessary SQL to do this in the stored proc as a comment). This feature was included to ensure that the Contract definition would not be deleted before the final invoice had been sent to the client. It is also important to persist historical Contract information for fixed price and internal contracts in order to monitor the profitability of these contracts and adjust future quotes according to the proven profitability of previous similar contracts.

Modeling a Resource

The Resource table models the Consultant's standard working day. The detail involved in this table is a direct result of the business requirement that specifies minimal user interaction in order to log time. To meet this requirement, the timeTrack system stores a profile of each individual resource's working pattern. This enables the consultant to record a typical day with a single click (see figure 2).

Figure 2: The Time Log Screen

By choosing 'Add a Day (based on profile)', timeTrack will take the user to the data entry screen with all of the fields already filled in according to their individual profile settings. This helps the organization capture the additional half hours, etc, that some employees work in a typical day.

Don't Forget the Data

The ChargeableTime table stores all of the information logged by each of the Resources / Users. The majority of the information stored within this table is self-explanatory. The description fields can be used to provide additional details about the work that was undertaken by the resource in the specified period. This enables ABC Consulting to provide an audit trail of all charged time to any clients who request it.

The IsLoadedInCube bit field is worthy of some additional discussion. Although the implementation of an OLAP Cube against the timeTrack data is out of the scope of this Case Study, I have included some of the control fields in the DB design to enable me to highlight some of the considerations and solutions to the management of generating dependable Analysis Services feeds. For those not familiar, Analysis Services is the OLAP engine that comes with SQL Server 2000 (it was called OLAP Services in SQL 7). This bit field is set to a default of 0. This field is only ever explicitly set in the UpdateContractData stored procedure, which sets the field to 0 to ensure that any fact table (the central table in a star or snowflake schema used for population of an OLAP cube; in simpler terms the table with the raw line-item or transactional data rather than a table describing business entities such as country, site, or resource) construction is aware of the change to the possibly historical data item. The field is only set to 1 during the fact table construction to indicate that the line item data has been included in the fact table information. Another useful means of using this type of control flag is to effectively lock records from update after an invoice has been generated, to ensure consistency of historical reporting.

A critical requirement of all charging related and time tracking systems is to ensure that a single resource has not recorded the same time period against multiple tasks. In timeTrack, the procedures AddContractData and UpdateContractData check before adding a record that the Resource has not already allocated this time period using the SQL ,shown below:

IF EXISTS (
SELECT StartTime, EndTime
FROM ChargeableTime
WHERE ResourceID = @ResourceID
AND [Date] = @Date
AND ((@StartTime > StartTime and @StartTime < EndTime)
OR (@StartTime < StartTime and @EndTime > StartTime)
OR (@StartTime < EndTime and @EndTime > EndTime)
OR (@StartTime = StartTime and @EndTime > StartTime)
))
BEGIN
RAISERROR ('The times specified overlap other already allocated times!', 16, 1)
END

The RAISERROR function allows me to pass an error back through to the appropriate User Control for display to the user.

Component Design

The high level component design for the demo version of timeTrack has been simplified to enable me to focus on the techniques used to implement each component.

The timeTrack application uses a SQL Server database with all table access controlled through stored procedures rather than in-line SQL: this improves performance, standardizes access, and simplifies server side security implications. It is worth noting that the use of stored procedures, and indeed the System.Data.SqlClient namespace, tie the application to SQL Server (see resources for article on problems with stored procedures). The alternative is to use a database independent DAL. I have seen applications which use an XML file (certainly better than placing the SQL in-line throughout the application) to store the SQL, and use either the System.Data.SqlClient or System.Data.OleDb namspaces to execute the SQL depending on a configuration setting. This is a more versatile solution, but does suffer from the additional hit on query plan determination at run-time. I chose to use SQL Server for the timeTrack case study because the .NET Framework installs the MSDE, and so all readers will be able to run the accompanying code. One point I would like to stress is that any business logic should not be implemented in the SQL stored procedures without very good reason. The database design has already been discussed above.

The data access layer contains a series of database access wrapper classes. The majority of the classes are stateless, acting only as efficient .NET wrappers on the stored procedure calls. As such, the methods are static.

The final logical application layer comprises the user control components, the security configuration, navigation control, and overall site presentation. The user controls are used to modularize the application functionality in order to simplify the final site implementation, and are discussed in detail in the Component Coding section.

Component Coding

DAL : The Data Access Layer

The data access layer provides a series of .NET objects which can be used to interact with the underlying database. The main DAL classes in timeTrack are ContractDB and ProfileDB. Both of these classes exist only to execute stored procedure calls and return the results. Since this task requires no state knowledge, the methods are implemented as static methods, enabling them to be called without first instantiating an instance of ContractDB or ProfileDB. This is very common in DAL's. The classes provide the following set of methods for each controllable table in the database:

GetAll - Returns all records from the table

GetSingle - Returns a single record from the table

Add / Insert - Adds a new record to the table, returning the identity values of the record inserted.

Update - Updates a specific record in the table with some new values

Delete - Deletes a specific record from the table

Due to the repetitive nature of DAL coding, there are several tools available which will generate a skeleton DAL wrapper class for a table. Some will even generate the SQL stored procedures (see resources at the end of this article for a link to ".Net Enterprise Development in C#". This book contains the Wrox Enterprise Objects tool). The SQL Server information schema views are very useful for providing a complete list of all parameters and their properties for any databases stored procedure. See listing 1 for some sample SQL.

Listing 1 : Interrogating the SQL Information Schema Views

SELECT SPECIFIC_NAME, PARAMETER_MODE, PARAMETER_NAME, DATA_TYPE, CHARACTER_MAXIMUM_LENGTH
FROM INFORMATION_SCHEMA.PARAMETERS
WHERE SPECIFIC_NAME NOT LIKE 'dt%'
ORDER BY SPECIFIC_NAME, ORDINAL_POSITION

As ever, when dealing with databases it is important to ensure that connections are properly closed. There has been a great deal of discussion on this topic (see http://www.asplists.com/asplists/aspngdata.asp?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq). The connection used to populate a DataSet can and should be closed immediately after the Fill. The behaviour of the SqlDataReader is a little more complex. In general, code like the following from AdminDB.GetUsers() will close the connection only after the SqlDataReader itself is explicitly closed.

// Open the database connection and execute the command
myConnection.Open();
SqlDataReader dr = myCommand.ExecuteReader(CommandBehavior.CloseConnection);

If you are binding the SqlDataReader, the close method is handled for you. However, when you are instead iterating through the data reader you must explicitly call the SqlDataReader.Close() method to ensure that the underlying connection object is closed.

Security and the Login Control

The application security settings are controlled by the <authentication> and <authorization> tags in the web.config file. Authentication is the process of associating and validating a users identity, whereas authorization is used to control the access granted to anonymous and authenticated users to specific resources.

ASP.NET provides four standard authentication modes: Windows, Forms, Passport and None. All of these are easily configurable through the authentication tag in the web.config file. Windows is most commonly used in an Intranet environment, though it can be used with IIS Basic, Digest and Integrated security configurations. Specifying windows authentication is as simple as adding the following tag to your web.config file.

<authentication mode="Windows" />

The authentication sub tag <forms> is used to configure custom forms-based authentication. This tag allows you to configure the name of the cookie used (name attribute). This name should be unique to your application in order to prevent conflicts with other applications residing on the same IIS server. It also allows you to configure the encryption and validation options for the cookie (see .NET help on the <forms> element for full details of the encryption and validation options), the default is to use both cookie encryption and validation and is the recommended setting. The timeTrack application's authentication tag is:

      <authentication
            mode="Forms">
<forms name="ContractCS"
            loginUrl="Login.aspx" />
</authentication>

The loginUrl attribute allows you to specify a page that all unauthorized users will be directed to. This simple mechanism enables you to ensure that no-one can access any ASP.NET controlled resource in your application without first being authorized. The controlled resources I refer to are those file types that are associated, within IIS, with the ASP.NET ISAPI filter (e.g. .aspx, ascx, .config, etc). While it is possible to add other file types to this association, thereby bringing them under the control of ASP.NET authentication, this approach is not recommended because of the performance overhead. A better solution to this problem, though only applicable to Windows mode authentication, is to use ACL's to protect the non-ASP.NET files and allow impersonation (by specifying <identity impersonate="true"> in web.config). Impersonation causes the ASP.NET code to execute in the context of the users security profile. So, any user who requests default.aspx will be redirected to Login.aspx unless they are authorized (in the case of timeTrack this requires them to have been authenticated). Once the user signs in by providing a valid username and password they are then redirected to the page they originally requested.

The authorization tag can be used to determine whether a specified identity should be granted access to a specific resource. The URLAuthorization module, which controls URL authorization, implements allow and deny assertions on resource access, allowing you to specify the access to specific parts of the URI namespace at the role or identity (HttpContext.User.Identity) level (a set of roles or identities can be specified by use of a comma-separated list). The following authorization tag demonstrates this:

<authorization>
<allow roles="ABCConsulting\Consultants, ABCConsulting\Management">
<allow users="ABCConsulting\DGoodwin">
<deny users="*">
</authorization>

The above authorization tag allows members of the roles Consultants and Management to access the site, as well as a specific user DGoodwin. All identities (indicated by "*") are denied, including anonymous users which could have been explicitly denied using <deny users="?">. The default ASP.NET configuration contains an <allow users="*"> element, authorizing all users. You can also specify the HTTP verbs (GET, POST, or HEAD) allowed by each user by entering tags such as <allow verb="POST" roles="ABCConsulting\Consultants">.

Figure 3: The timeTrack Login Control

The login Control (Login.ascx, see figure 3) works by asking the user to enter an e-mail address and password. These values are then checked against the Users table, and if the values match a users definition in the Users table, the code creates a cookie for the user using:

// Use security system to set the UserID within a client-side Cookie
FormsAuthentication.SetAuthCookie(email.Text, RememberCheckbox.Checked);

Given that the login control is only available on the login page, it may seem strange to have chosen to implement it as a control at all. This is, to some degree, personal preference; I find later restructuring of code and interface to be simpler when dealing with controls (custom or user).

The FormsAuthentication class provides static helper utilities for the manipulation of authentication tickets. The SetAuthCookie retrieves the encrypted values from the cookie as an HttpCookie instance. If the user asked to persist the cookie by checking the RememberCheckbox checkbox, the SetAuthCookie creates an authentication ticket, encrypts it, and stores it within the cookie on the client machine. Listing 2 shows the code in the timeTrack Demo used to control the persistence and subsequent retrieval of the cookie authentication ticket.

Listing 2: Cookie Creation in Global.asax

protected void Application_AuthenticateRequest(Object sender, EventArgs e)
{
if (Request.IsAuthenticated == true)
{
String[] roles;
// Create the roles cookie if it doesn't exist yet for this session.
if ((Request.Cookies["ContractCSroles"] == null) ||
(Request.Cookies["ContractCSroles"].Value == ""))
{
// Get roles from UserRoles table, and add to cookie
UsersDB user = new UsersDB();
roles = user.GetRoles(User.Identity.Name);

// Create a string to persist the roles
String roleStr = "";
foreach (String role in roles)
{
roleStr += role;
roleStr += ";";
}

// Create a cookie authentication ticket.
FormsAuthenticationTicket ticket = new FormsAuthenticationTicket(
1, // version
Context.User.Identity.Name, // user name
DateTime.Now, // issue time
DateTime.Now.AddHours(1), // expires every hour
false, // don't persist cookie
roleStr // roles
);

// Encrypt the ticket
String cookieStr = FormsAuthentication.Encrypt(ticket);

// Send the cookie to the client
Response.Cookies["ContractCSroles"].Value = cookieStr;
Response.Cookies["ContractCSroles"].Path = "/";
Response.Cookies["ContractCSroles"].Expires =
DateTime.Now.AddMinutes(1);
}
else
{
// Get roles from roles cookie
FormsAuthenticationTicket ticket =
FormsAuthentication.Decrypt(Context.Request.
Cookies["ContractCSroles"].Value);

// convert the string representation of the role data
// into a string array
ArrayList userRoles = new ArrayList();
foreach (String role in ticket.UserData.Split( new char[] {';'} ))
{
userRoles.Add(role);
}

roles = (String[]) userRoles.ToArray(typeof(String));
}

// Add our own custom principal to the request containing the
// roles in the auth ticket
Context.User = new GenericPrincipal(Context.User.Identity, roles);
}
}

This code instantiates a GenericPrincipal object which implements IPrincipal and exposes an IIdentity interface through the identity property, enabling us to check role membership (IsInRole), Name (Name), and Authentication (.IsAuthenticated).

Site Navigation

One of the main reasons for implementing a custom role based authentication model was to enable me to vary the accessibility and user interface presentation based on a user's security settings. I decided to implement the navigation control as a user control (SiteBanner.ascx, see figure 4) in order to encapsulate as much of the logic as possible. I considered several approaches to this control and hit quite a few problems in its implementation, which I will now discuss.

Figure 4: The timeTrack SiteBanner Control

The first and most obvious question I faced was whether or not to create a custom control or a user control. I decided to use a user control mainly because of the fact that the control is a composite control and had no special design time, rendering, or deployment requirements (custom controls are the better option for creating controls that will be distributed to be used in multiple applications). As an aside, I also believe that the development of a truly flexible and re-usable navigation control is a suitable subject for a whole case study and would only cloud the core subject matter of this case study (see resources for links on Custom Control Creation).

The most obvious and simplest solution would have been to dynamically create a single row table of hyperlinks, based on the users roles. However, I wanted to be able to maintain the state navigation control, displaying the selected tab differently from the unselected. The first problem I hit in the creation of this control was my inability to trap the SelectedIndexChanged event of the BaseDataList. Since this event is only triggered when the selected index changes between server postbacks, I could not use this event (due to the nature of the navigation control, when a new item was selected a new page was requested). Eventually, I realized I could use a LinkButton and persist the SelectedItemIndex by means of the dlNavBar_ItemCommand event. The ItemCommand event allows you to pass a command string back to your server side code using the CommandName property. The CommandName property can be used to pass a string that indicates the type of command to be performed. An example from the timeTrack AdminRoles.ascx user control is shown in Listing 3 below:

Listing 3: Use of the CommandName property to pass the action type back to server side code

<asp:DataList id="rolesList" DataKeyField="RoleID" runat="server">
<ItemTemplate>
<asp:ImageButton ImageUrl="~/images/edit.gif" CommandName="edit" AlternateText="Edit this item"
runat="server" ID="Imagebutton1" NAME="Imagebutton1" />
<asp:ImageButton ImageUrl="~/images/delete.gif" CommandName="delete" AlternateText="Delete this item"
runat="server" ID="Imagebutton2" NAME="Imagebutton2" />

From the AdminRoles.ascx.cs rolesList_ItemCommand event handler:

if (e.CommandName == "edit")
{
//
            Set editable list item index if "edit" button clicked next to the
            item
rolesList.EditItemIndex = e.Item.ItemIndex;

//
            Repopulate the datalist control
BindData();
}
else if
            (e.CommandName == "apply")
{
// Apply changes
String
            _roleName = ((TextBox)
            e.Item.FindControl("roleName")).Text;
..

This allows you to add several ImageButton and LinkButton controls to a control item's template and identify the action to take based on the CommandName string property. Each control can specify an optional CommandArgument string which can be used to pass parameters or additional information to supplement the command action.

Now I was able to at least set the Selected index of the horizontal DataList control. This still didn't solve the problem of persisting the index setting across requests. While viewstate and ASP.NET handles the persistence across postbacks it does not persist the control settings across requests for different pages. Initially I thought of creating a protected int variable on each page called NavId to hold the Selected index value. I then added a NavId parameter to the hyperlink stored in each tabs NavBar control, as follows:

authorizedLinks.Add(new NavBar("Contract Admin", "~/ContractAdmin.aspx?NavId=1", "Contract Admin"));

This approach was flawed in that the index of the Tab in the navigation control changes depending on the user's role membership. For example, user A, a Consultant, would sign on and see links for Home (index 0) and Log Time (index 1), while and User B, a ContractAdmin and Consultant, would sign on and see Home (0), Contract Admin (1) and Log Time (2). This one aspect of the problem could have been resolved by passing the name of the Tab and iterating through the navigation list (or performing a direct read on a hashtable) to find the matching entry before setting the SelectedIndex property. This solution is more efficient than enabling sessionState for the sole purpose of storing the navigation index, but the timeTrack application is likely to be used not as a sole web application but rather as part of a larger company intranet/internet application. I am also averse to putting unnecessary information in the URL (especially since in the above scenario User A sending User B a hyperlink could generate an incorrect rendering of the navigation control), preferring instead to handle storage state information in session, application, or cache objects. The code accompanying this case study uses the HttpSessionState class through Context.Session to store the currently selected tab index.

The HttpSessionState object allows you to persist information relating to a particular Session for the entire duration of their visit to the site. It works by assigning a unique session ID for a user on their first visit to the site, the ID is then associated with the user until the session times out. This is configurable at both page level through the Timeout property

Session.Timeout = 3 // in minutes

Or through the web.config file:

    <sessionState
mode="InProc"
stateConnectionString="tcpip=127.0.0.1:42424"
sqlConnectionString="data
            source=127.0.0.1;user id=sa;password="
cookieless="false"
timeout="20"
/>

The sessionState tag above shows the default settings for sessionState. This configuration will cause ASP.NET to track the session id using cookies on the client machine. For the purposes of our demo this is adequate, although timeout could be dropped dramatically since the NavId is reset on each click on the Navigation control. The available modes for session state are:

sessionState mode | Description
Off | SessionState is not enabled
InProc | SessionState is stored locally
StateServer | SessionState is stored on a remote server.
SQLServer | SessionState is stored in a SQL Server (note this is only available in SQL Server 7 and above)

In-Process session state is the fastest, and ideal for small amounts of non-critical, volatile information. The drawback of in-process sessionsState, however, is that an IIS restart or ASP.NET process crash results in the loss of the session information by virtue of the fact that the session state is managed within the IIS process. The advantages of a separate out-of-process session management is that the session state information is retained throughout an IIS restart or process crash, and that the sessionState is not inextricably linked to any one machine or server. While this loss is not critical in an application like timeTrack, in an e-commerce site where shopping cart information is held in sessionState, this kind of data loss has a direct impact on revenue and customer confidence.

In order to provide an alternative out-of-process solution to state management, ASP.NET provides SQL Server and State Server modes. StateServer mode allows you to start a separate state server on a remote machine using net start aspnet_state, and enter a stateConnectionString value in web.config. To configure SessionState to store its information in SQL Server, you must run a special SQL script (InstallSqlState.sql) against your DB. Fortunately MS have provided this script with the .NET Framework, it is located in the <%WINPATH%>\Microsoft.NET\Framework\<version>\ folder by default. This creates a new ASPState database and some stored procedures to manage state. Once the script has run successfully and the new database has been created, all you need to do to configure SQL Server StateManagement is alter the web.config, specifying mode=SQLServer and providing a connection string in the sqlConnectionString attribute:

<configuration>
<system.web>
<sessionState mode="SQLServer" sqlConnectionString="data source=127.0.0.1;user id=sa;password=" />
<system.web>
</configuration>

The decision to use in-process or out-of-process state management depends, to a large degree, on the information being persisted. In-process state management also offers a performance benefit over out-of-process management.

By using session state in the SiteBanner.ascx, I am able to pass the selected item index across to the next page, allowing it to render the navigation control correctly. The removal of the NavId parameter from the query string also means that a hyperlink to timeTrack used by different operators will not result in an incorrectly rendered selected tab setting.

Defining the Contracts, Assigning the Resource

While the navigation and login controls were the most deeply entrenched in ASP.NET configuration, their function is structural rather than functional in the sense that they do not help solve the business problem at hand. What follows is a simple model for quickly putting together prototypes and departmental line of business systems that are easy to maintain.

The reason for demonstrating this approach is to show how you can build valuable customer ASP.NET applications with a relatively narrow field of knowledge with regards to ASP.NET. I have already shown how easily the security can be configured for your application, all that remains is to provide a clean usable user interface to the function. In my opinion, this is the beauty of .NET: it allows you to focus on the application specific function by providing libraries of standard functions and simple reliable configuration mechanisms.

Each of the core business entities Contract, Client, Country, Category and Resource are handled in essentially the same way. Once the requisite informational structure has been defined in the database table I write several standard stored procedures that perform the basic expected interactions: Select All (returning a DataSet or SqlDataReader), Select Single instance (returning a DataSet, DataRow or SqlDataReader), Insert (returning the identity value of the item just added), Update and Delete (both having void return type). The next stage is to provide a .NET wrapper to the stored procedure call. I chose to implement each method by establishing a connection to the database, using the web.config file to store the connection string:

<configuration>
<appSettings>
<add key="ConnectionString" value="server=localhost;uid=sa;pwd=;database=ContractCS" />
.
</appSettings>
.
</configuration>

and

ConfigurationSettings.AppSettings["connectionString"]

to retrieve the value.

The stored procedure call is built up by adding to the parameters collection of the command or adapter object. In general, for this case study I have used DataSet return types as I do when prototyping since this allows greater flexibility in the manipulation of the front end user interface, allowing me to present to the client a number of different front end interface concepts based on the same backend DAL components. In actual fact, more recently I have replaced this habit firstly with a SqlDataReader to SqlDataSet conversion function in prototypes, with the full strongly typed entity and entity collection wrappers being used for second stage development (see resources for some resources on the construction of the entity wrappers).

Next I create a composite user control to display the records in the table and provide a means of editing, adding, and deleting new items. A typical examples is the ContractProfile.ascx (see figure 5)

Figure 5: The timeTrack User Profiles Control

The ContractProfile control makes use of the ASP.NET DataGrid's built-in paging functionality. This allows you to configure the DataGrid to display a subset of the underlying records and provide the user with pager controls to move through the information. ASP.NET provides standard and custom paging options. The standard paging option is very simple to set-up and only requires you to set the AllowPaging property of the DataGrid to true (the default page size is 10) and set the PagerStyle objects mode property in the PagerStyle template.

<ASP:DataGrid id="dgProfile" AllowPaging="true" . runat="server">
...
<PagerStyle Mode=NextPrev></PagerStyle>
</ASP:DataGrid>

The two modes for the PagerStyle object are NextPrev, which defaults to a display of '< >', or NumericPages which displays a row of page numbers. DataGrid provides a vast number of style based properties for configuring the appearance of the PagerStyle object. One important point to note about the standard ASP.NET paging mechanism is that it retrieves all records on each page request and should not, therefore, be used with a very large numbers of records. If you do have to deal with paging through large numbers of records, then the back-end approach is very similar to how you would do it in traditional ASP, by creating a stored procedure that creates a temporary table with an identity column and selecting a specified page based on input parameters of @pagesize, @CurrentMinID and @currentMaxID (the minimum and maximum identity values from the page currently being displayed). With this information you can use

SELECT TOP @pagesize * FROM <##table> WHERE ID > @currentMaxID

To return the Next page of records, and

SELECT TOP @pagesize * FROM <##table> WHERE ID < @currentMinID

To return the Previous page of records. Your stored procedure would also contain bounds checking.

As you can see from figure 5, the user can page through the available resources, edit any resource by clicking on the arrow to the left of the resource, and add new resource profile definitions. The add, update and delete is handled by an EditContractProfile.aspx page. Each of the core functional .ascx controls has an associated web form that handles the edits. I make use of this fact in the user control by using the ItemTemplate to define a hyperlink that links to a URL to edit the current record as follows:

<asp:HyperLink ImageUrl="~/images/edit.gif" NavigateUrl='<%# "EditProfile.aspx?ItemID="
+ DataBinder.Eval(Container.DataItem,"cres_id") %>'

The edit pages contain role based security checks in the Page_Load to ensure that a user cannot gain unauthorized edit access by editing the URL. The code to implement this simply checks the current users HttpContext IsInRole member:

// Verify that the user has access to the
            current page
if
            (Context.User.IsInRole("ContractAdmin".ToString()) ==
            false)
{
//
            Response.Redirect("~/AccessDenied.aspx");
Response.Redirect(Configuration.ApplicationPath
            + @"\AccessDenied.aspx");
}

The above snippet should contain a slight cautionary note on the use of '~'. In C#, the '~' operator can be used as a shortcut to specifying the Request.ApplicationPath. In virtual directories this works fine, however, if the application is at the root domain level it can result in the generation of invalid URL's. It should be noted that HttpRequest.ApplicationPath also returns '/' at the root domain level. I have not been able to find an elegant solution to this problem. In an attempt to enable you to run the application at either domain or virtual directory level, I created a small helper class (Configuration.cs) with a single static property ApplicationPath, which returns String.Empty if the HttpRequest.ApplicationPath returns '/'. The code accompanying this article does not use this method throughout; rather it is demonstrated in SiteBanner.ascx control. The simple reason for this is that I could not find an elegant way of replacing every ~ within the application while retaining full VS.NET designer support, short of specifying every link URL within the code.

The ContractAdmin.ascx also uses the ItemCreated event of the DataGrid to color code specific values to give the user visual cues to the state of the Contracts. The code that performs this uses FindControl to locate the particular control in the templated data bound grid and change its forecolor. See listing 3.

Listing 3: Providing Visual Cues through Color

if
            (((Label)e.Item.FindControl("lblActiveField")).Text ==
            Boolean.FalseString)
{
e.Item.ForeColor =
            System.Drawing.Color.DarkGray;
}
else
{
decimal
            DaysRemaining =
            System.Decimal.Parse(((Label)e.Item.FindControl("lblDaysRemaining")).Text);
if
            (DaysRemaining < 0)
{
((Label)e.Item.FindControl("lblDaysRemaining")).ForeColor =
            System.Drawing.Color.Red;
}
else
{
if (DaysRemaining
            > 0 && DaysRemaining <
            10)
{
((Label)e.Item.FindControl("lblDaysRemaining")).ForeColor
            = System.Drawing.Color.Blue;
}

The FindControl method is enormously powerful and avoids the maintenance headaches caused by hardcoding indexes in control collections to refer to objects, such as:

((Label)e.Item.Cells[8].Controls[1]).Text

which works only as long as another column is not added to the data grid columns template.

The Client, Country and Category controls have fragment caching defined at the control level since the data in these controls is not subject to frequent change. The Contract and Contract data controls are transactional by nature or require accurate up to the minute provision of figures. Fragment caching can be achieved by using the OutputCache page directive:

<%@ OutputCache Duration="5" VaryByParam="none" %>

VaryByParam allows you to cache a different version of the control based on a parameter value, VaryByParam="*", which will cache a different page fragment whenever any parameter changes. Another very useful attribute for caching controls is VaryByControl, which allows you to vary the output cache based on a semi-colon separated list of user control properties. Caching is one of my favourite features of ASP.NET, and I cannot stress its importance in the development of heavy-use web applications enough; my advice is to make caching the next thing you learn about in ASP.NET if you don't already know it!

Figure 7: Component Development Model

Figure 7 shows the simple model used in the timeTrack application to wrap all interaction with the database by means of a Data Access Layer which calls the SQL stored procedures. The biggest advantage gained from this simple model of development is that the wrapper around each business entity forms a distinct module that is reliable, consistent, and easy to maintain. In small systems such as the timeTrack demonstrated here, there is increasing pressure on developers to be able to develop, deploy and maintain these applications quickly and consistently. Often when change requests are raised, they are in reaction to a business process change with immediate impact, and as such require very rapid resolution and implementation. A simple model as outlined above minimizes the impact on the application of table alterations and user interface changes. I am not suggesting that this approach be taken for large scale enterprise systems, but do believe that this approach is very valuable in being able to provide the business with solutions quickly and cheaply. The secondary advantage is that the similarity among the controls allows them to be cloned very quickly, enabling you to add a whole new business entity with full user interaction to the system in a matter of minutes or hours rather than days.

UI Design

The importance of user interface design can never be underestimated. Through the development of countless systems for clients I have come to realize that an attractive and usable user interface is critical right from pre-sales presentations through to implementation. Often the user interface will bear the brunt of more scrutiny and form the basis of more opinions than the underlying functionality: sad but true.

The use of Edit pages rather than using the DataGrids EditItemTemplate provides the user with a fuller and more user friendly data entry screen. Although the EditItemTemplate can contain quite an extensive set of controls for the update of the record, it is difficult to provide a clean and consistent visual appearance across a whole site. The edit page also provides me with sufficient white space to provide detailed error text for my validation controls, and in my opinion leads to a better user experience. I believe each has its role, and use EditItemTemplate in my applications where appropriate.

One of the main business requirements for this application was that the recording of information requires minimal effort and time on the part of the consultant. In order to achieve this I considered various options. The idea of storing individual work profiles for each resource allowed the user to record a typical day with a single click on the Add a Day (based on Profile) link. Initially, the application simply created a new instance for the current day without taking the user to the EditContractData page. However, as I was testing the application, I realized that on days where the user has left 15 minutes early or stayed a little later but otherwise worked a typical day, the convenience of clicking on a link to fill in all of the default properties would be appreciated.

The ContractData control also displays a couple of links that allow the user to vary the period of time shown in the DataGrid. This allows them to perform checks on the information they've entered and also to look back and see what they were working on at any given date.

The focus of the application from a Contract Admin user's perspective is on the number of days remaining in a contract. For this reason, I implemented colour highlighting of key values in the DataItem to show contracts that were running low on agreed dates, etc.

The user interface should be as clean and consistent as possible. The use of a standard approach to all our controls ensured a consistent user experience throughout the application usage, and a single cascading stylesheet (ContractCS.css) was used throughout to provide a consistent visual format. When using VS.NET, it is often tempting to set style attributes to specific fonts or colours within the control properties. This leads to fragmented and often inconsistent settings. Cascading Stylesheets provide a way to separate content from style by allowing for the definition of named collections of style attributes. These can be associated with HTML tags as follows:

A.CommandButton:link {
text-decoration:
            underline;
color: #FF702B;
}

A.CommandButton:visited
            {
text-decoration: underline;
color: #FF702B;
}

The use of a stylesheet in this way enables you to easily alter the appearance of the entire site in a consistent manner, which can be a very powerful tool in facilitating customer specific branding of ASP.NET and other web applications.

UI Code

Due to the critical nature of Roles within the application, any attempt to delete a role prompts the user for confirmation. This is achieved by creating invisible Yes/No link buttons in the DataList template. I could have used JavaScript on the delete button (see Listing 4) to create a pop-up confirmation dialog box, but felt that the link buttons were more in keeping with the look and feel of the application.

Listing 4 : Implementing a JavaScript Confirmation Dialog Box

ImageButton2.Attributes.Add("onClick","javascript: return confirm('Are you sure?');");

Another valuable addition to the developers kitbag introduced in ASP.NET is the validation controls. These controls allow you to provide the user with meaningful feedback on incorrect data entry values and help ensure that any input information is valid and correctly formed. Within timeTrack demo I have used these where appropriate to improve the user experience. The EditContractData user control, for example, contains several RequiredFieldValidator controls (to ensure that data is entered) and several RegularExpressionValidator controls (to ensure that the time supplied by the user is a valid time). To add a validation control to your web form or user control simply drag the appropriate validation control onto the form and set the properties as outlined in the table below:

Property | Description
ControlToValidate | The name of the control you wish to validate
Text | The text you want to appear in the validation control if the validation fails
ErrorMessage | The text you want to appear in any ValidationSummary control on the page if validation fails
Display | None - the control will never appear in-line. A message will only be output to the validation summary on failure. Static - Space is allocated to the control Dynamic - Space is only allocated to the control when validation fails.

Application Setup

The following steps describe the process of installing the timeTrackDemo application:

  1. Restore the supplied ContractCS database to your SQL Server 2000 or MSDE database.
  2. In IIS create a new Virtual Directory and extract the files from the timeTrackCode.zip file into the folder used for this virtual directory.
  3. Edit the .webinfo file in the folder you have just created so that it points to your virtual directory. This will enable VS.NET to connect to the appropriate virtual directory.

    <appSettings>
    <add key="ConnectionString" value="server=localhost;uid=sa;pwd=;database=ContractCS" />
    <!-- Mail Server MUST be changed to your IIS_configured Server name -->
    <add key="MailServer" value="mymail.myserver.com" />
    </appSettings>

  4. Edit the web.config file, changing the connection string to suit your database server name and password. The code supplied will connect to localhost as 'sa' with blank password. If you have set-up SMTP mail on your IIS Server then edit the MailServer value (don't worry if you don't, this is only required to enable the mail me my password feature and will not affect the overall running of the application in any other way).

You should now be able to login as admin@timetrack.com with password 'admin'. The admin@timetrack.com user is a member of every role and has full access to the application.

The Sample Application

The following screen shots show sample usage scenarios for different roles.

All Users

Users login to the application. Use admin@timeTrack.com and password admin for full access to the demo.

Figure 7: Logging In

Consultant

A Consultant can then see what time they logged in the last x weeks.

Figure 8: Users personal time sheet summary

from here the Consultant can log time against a contract using either Add New Task or Add Profile Day, or edit an existing entry.

Figure 9: Recording Time

ContractAdmin

A ContractAdmin is able to define Contracts, monitor their status and create or edit user profiles. At the moment, the ContractAdmin is not able to restrict the contracts a user can log time against, so a user can log time against any contract (that has not been deleted).

Figure(s) 10: Contract Definition Screens

Analyst

The timeTrackDemo provides a sample query interface to show members of the Analyst role all time logged for any specified day.

Figure 11: Sample Enquiry Function

Administrator

The Administrator is responsible for setting up users and maintaining roles.

Figure 12: Account Admin and User Management

Any Limitations or Further Work

The timeTrack Demo is a Case Study implementation, and while it contains the framework and base functionality, there are a few enhancements and extensions that could be made. The following is a list of the ones that occurred to me

  • The application does not allow you to record a time period that spans midnight. This is in part intentional, to ensure that all weekend and out of hours work is correctly accounted for and charged.
  • On the EditContractData screen, you could give ContractAdmins the ability to log time for any user, by means of a drop down list box. All other users would still be presented with a read only textbox.
  • The case study does not include reports. I have however discussed some of the issues surrounding the creation of OLAP cubes from the contract data.
  • A Mobile interface would allow Consultants to login and record time from their PDA or phone.
  • The Role based security is too tightly coupled to the Navigation control. Extending the application to use permissions within roles would improve the flexibility of the model.
  • The User and Profile tables are not directly linked. I had considered using triggers to automatically create a default Profile when a New User was registered. However, I felt that the assignment of default contracts was the responsibility of a delivery manager or project supervisor rather than a security or network support function, and so left the two distinct.

Hopefully, the ideas presented above and the timeTrack Demo application have given you a better understanding of how an ASP.NET application hangs together. If you do develop the application in any significant way please drop me an e-mail as I'd be very interested to see how you extended it.

Conclusion

In this case study I have shown how to go from a set of business requirements to a full-featured application by following a simple user control based development model. Throughout the discussion I have highlighted various features of ASP.NET, including custom forms validation, user control creation, fragment caching, the validation controls and state management.

The purpose of this article was to provide an end to end example of an ASP.NET application for those new to ASP.NET and to introduce some of the powerful new features .NET brings to web application development. Please feel free to e-mail me with any comments, queries or suggestions.

Case Study Information
Author | Brian Boyce
Technical Editor | Adam Ryland
Project Manager | Helen Cuthill
Reviewers | David Williams, Daniel Cazzulino, Slavomir Furman

If you have any questions or comments about this case study, or suggestions for further pieces, please contact the technical editor. If you wish to talk about the case study with other subscribers, please visit our forum here.


Building an Interface to Legacy Data Using BizTalk Framework 2.0

Building an Interface to Legacy Data Using BizTalk Framework 2.0
http://www.csharptoday.com/content.asp?id=1766
Article Rating: not yet rated
Published on April 15, 2002
Abstract
To remain competitive, businesses must increase the business process efficiency through enterprise application integration (EAI) and by providing customers with self-service account maintenance through web services. Yet, monolithic legacy applications with proprietary access interfaces are poorly suited for adaptation to these roles. The BizTalk Framework solves these problems by providing a common language protocol for communication and data transfer between diverse business applications. This case study by Kevin Farnham documents the creation of a BizTalk schema and a BizTalk processor that provide a standards-based interface to a proprietary legacy application and data store at an imaginary electric utility called ‘Central Power and Light’.
Case Study

To remain competitive, businesses must increase the business process efficiency through enterprise application integration (EAI) and by providing customers with self-service account maintenance through web services. Yet, monolithic legacy applications with proprietary access interfaces are poorly suited for adaptation to these roles. The BizTalk Framework solves these problems by providing a common language protocol for communication and data transfer between diverse business applications. This case study documents the creation of a BizTalk schema and a BizTalk processor that provide a standards-based interface to a proprietary legacy application and data store at an imaginary electric utility called 'Central Power and Light'.

Introduction

Large corporations have become accustomed to successive waves of technological advances that require adaptation of existing systems to new purposes. The bolder companies will be quicker to embrace the new technologies, seeking an edge that will enable them to leap ahead of the competition in efficiency and/or customer service. More conservative companies will delay costly changes in infrastructure until the new technological paradigm appears sure to last, in an effort to minimize the chance of devoting precious resources to a technology that fails to meet its promise. In the end, those technologies that change everything, that truly become the new generation of computing, communications, engineering - whatever - must be adopted by all major participants in an industry. Since the technologies succeed only if they provide increased productivity at reduced long-term cost, any company that fails to implement the technology soon finds itself at a competitive disadvantage that can put it out of business.

Enterprise application integration (EAI) and web services are new computing technologies that have great potential for increasing corporate productivity and efficiency today. Most large companies are studying the possibilities. Several standards organizations and industry consortia have been formed in efforts to channel the new technologies in directions the participants mutually agree on, in order to avoid the very inefficient situation where multiple similar but incompatible protocols compete. The BizTalk Framework represents one such attempt at standardization in the realm of EAI.

The problem facing companies that seek to automate and streamline business processes using EAI is that legacy applications designed and developed decades before the advent of the internet are not well suited for the type of integration that would enable automated business-to-business (B2B) transactions with partners, or business-to-consumer interaction with customers via self-service web-based access to company applications and data stores. Indeed, even internal application-to-application integration will often be very difficult, because the internal software applications were developed by different teams at different times in response to different "current" critical corporate needs and requirements. Few of the developers who worked on the legacy systems are still employees at the company, and all that is really known about the legacy applications is that they are huge and their processing can be accessed via a few proprietary interfaces that are well-known only to a few members of the Information Systems department.

The BizTalk Framework and BizTalk Server are key components of Microsoft's emerging business process and application integration initiative. Information Week terms this integration "the next generation of computing" in "A New Wave Nears: Microsoft's Web Services" (February 11, 2002; http://www.informationweek.com/story/IWK20020208S0037?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq). While BizTalk Server is a Microsoft product, the BizTalk Framework is an open protocol for Business-to-Business (B2B) communication and Business Process Integration that has been contributed to and embraced by a large number of companies and standards organizations.

The BizTalk Framework provides a model and technology designed to bridge the gap between legacy systems with proprietary message and data interfaces and modern standards-based interfaces that open the legacy applications for business process integration within the enterprise and with partner systems and customers. This case study demonstrates how the BizTalk Framework can be applied to an exemplary simulated "legacy" system to accomplish this objective. The case study shows how to design and publish a BizTalk schema, introduces Microsoft BizTalk Server, and also presents a custom application that performs basic data mapping and transformation functions similar to those performed by BizTalk Mapper, but tailored specifically to the presented example legacy system. BizTalk Mapper is the component in BizTalk Server that allows mapping data fields in one document format with equivalent fields in another document format. Once the data mapping has been defined, BizTalk Mapper will automatically convert documents between the two formats, thus enabling communication between the two systems even though their native input and output data formats differ.

The case study will be of interest to software developers charged with the task of integrating legacy data stores and applications for Business-to-Business automated transaction processing, internal business application integration, or creation of web services providing customers with self-service access to data stored in legacy formats. Examples include electronic marketplaces (B2B) and customer self-service account management and viewing of monthly statements. From the point of view of technology, the case study will be of interest to developers interested in business process integration, XML, SOAP, BizTalk, .NET, C#, and Windows Forms.

Article Structure

The case study will begin with an overview of the history and current state of business process integration, the BizTalk Framework 2.0, and relevant software development technologies to be utilized in the case study. The case study will then present an example "legacy" data store, and illustrate the development of server-side and client software that provides access to the data store and documents using BizTalk Framework messaging.

System Requirements

Software Technologies and Environment:

  • Visual Studio .NET Beta 2 (Version 7.0.9254 was used to develop the software)
  • Windows ME, Windows NT 4.0, or Windows 2000 (required by .NET class FileSystemWatcher)

Software Development Knowledge and Experience:

  • Basic knowledge of XML and SOAP
  • familiarity with the BizTalk Initiative and BizTalk Framework
  • introductory experience developing applications using C#, Windows Forms, and .NET

Technology Aspects of the Problem

Electronic Data Interchange (EDI) was designed at a time when processing power and network bandwidth were much less capable than is the case today, and memory was much more expensive. One of the design objectives of EDI was to pack a large amount of information into a small packet of data bytes. As a result, EDI messages are not easy to read and decipher-they are cryptic to anyone who is not well-versed in EDI abbreviations, notations, and message formats. This increases the difficulty of implementing standard EDI interfaces.

In the decade or more since EDI implementation began at many large companies, infrastructure technology - processor speed, network bandwidth, memory - has advanced to the point where the speed advantage of small EDI messages over more verbose XML with the same content is virtually nil. The shorter messages will produce no noticeable effect in most B2B scenarios. Implementation of classical EDI remains difficult and maintenance costly, but there is virtually no performance gain over XML, which is much easier to implement, much more flexible, and readily adaptable to new processes and modifications and enhancements of existing processes.

As a result, traditional EDI is being replaced by XML-based standards. For example, RosettaNet (http://www.rosettanet.org/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq) is implementing future standards using XML, and is documenting strategies to assist companies who have a significant investment in the traditional RosettaNet EDI standards in upgrading to the new XML-based platform.

The dot.com boom brought with it a proliferation of new proposed specifications for eCommerce and business-to-business integration and document interchange. While the number of competing specifications was large, there was general agreement on one point: that XML (eXtensible Markup Language) is the most appropriate vehicle from transporting the documents and information. All of the important new B2B/eCommerce standards adopt XML as their interface and transport language. While the utility of standards decreases when there are multiple standards serving essentially the same purpose, the fact that the standards all implement XML makes translation of documents between the standards much more straightforward than would be the case if different underlying transport languages were implemented.

Important B2B and EAI standards at present include:

  • RosettaNet (http://http://www.rosettanet.org/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq): industry consortium, features Partner Interface Processes (PIPs); now embracing XML
  • OFX (http://www.ofx.net/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq): Open Financial Exchange, unified specification for exchange of electronic financial data over the internet.
  • ebXML (http://www.ebxml.org/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq): United Nations sponsored, specifications for "Electronic Business using XML".
  • BizTalk Framework (http://www.biztalk.org/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq): Microsoft-led consortium; business-to-business document exchange

Important schema repositories include:

  • RosettaNet (http://www.rosettanet.org/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq): repository for Partner Interface Processes (PIPs)
  • xml.org (http://www.xml.org/xml/registry.jsp?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq): repository for XML schemas and DTDs
  • BizTalk.org (http://www.biztalk.org/library/library.asp?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq): repository for BizTalk XML "objects" (schema)

Commercial software applications for implementing EDI and Business Process Integration include:

  • webMethods (http://www.webmethods.com/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq)
  • BEA eCollaborate (http://www.bea.com/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq)
  • Microsoft BizTalk Server (http://www.microsoft.com/biztalk?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq)

The BizTalk Framework 2.0

The BizTalk Framework is designed to facilitate communication and interoperability between diverse organizations and applications. The http://www.biztalk.org/?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq web site provides information on the BizTalk Framework, a repository for BizTalk schema, with capability for browsing and searching for schema and registering interest in particular schema, and message boards where visitors can ask questions and discuss the BizTalk technology. As of March 2002, several hundred schemas were registered at BizTalk.org.

The BizTalk Framework is organized around the following central concepts:

  • Application: a self-contained business system that creates, consumes, or stores business information.
  • Adapter: a component that performs any functions necessary to enable an application to exchange Business Documents with a BizTalk Framework Compliant Server.
  • Business Document: an XML document containing business data that conforms to a published Schema.
  • Schema: a published document that defines the structure of a class of Business Document.
  • BizTalk Framework Compliant (BFC) Server: a server that performs Business Document exchange with applications and BizTalk Document exchange with other BFC Servers, in accordance with Biztalk Framework specifications.
  • BizTalk Document: a SOAP 1.1 message containing one or more Business Documents that is transmitted by a BFC Server within BizTalk Messages.
  • BizTalk Message: wire-level message units containing BizTalk Documents and associated data (attachments), transmitted between two BFC servers.

The figure below illustrates the relationship between various components related to document exchange between two applications:

The application is the appropriate starting point for discussing BizTalk, because if there was no need to exchange information between two applications, there would be no reason to implement BizTalk! In BizTalk parlance, an "Application" is defined as "the line-of-business system where the business data or logic are stored and executed." In other words, an application is the pre-existing system within the enterprise which performs a specific set of business functions and manages a specific set of business information. The application could be a legacy system built in COBOL running on mainframes, an enterprise software package such as SAP or Seibel, a commercial document management package, custom software built in MFC, etc. Any software that consumes, produces, or stores business data is an "application." If there is a business benefit in exchanging information between two applications that do not inherently share a common interface, then the BizTalk Framework can be used to structure and automate the business data exchange.

An adapter is a component that is linked to a particular type of application. The adapter's role is to perform any operations required to permit the application to consume and emit Business Documents and communicate with the BizTalk Framework Compliant (BFC) server. For outgoing documents, an adapter must translate information and data structures from the application's internal format into the format defined by the business document schema. For incoming documents, the adapter must translate information from the format defined by the business document schema into the application's internal format and data structures. As of March 2002, more than 300 adapters have been developed for use with Microsoft BizTalk Server by commercial software vendors. The available adapters include Application Adapters for commercial business software packages, and Technology Adapters. The technology adapters provide interfaces with a great many legacy and modern software component technologies, database structures, file formats, document formats, terminal protocols, internet communication protocols, etc. The full list of current adapters compatible with BizTalk Server is available at: http://www.microsoft.com/biztalk/evaluation/adapters/adapterslist.asp?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq. The commercial availability of these adapters and BizTalk Framework Compliant Servers makes it possible in many cases for companies to integrate their existing applications with other internal or external applications using the BizTalk Framework, without costly and time-consuming custom software development.

A business document is an XML representation of the information that is to be exchanged between the applications. There is no required format or content for business documents with the BizTalk Framework. The only necessity is that the documents consist of well-formed XML. Business documents are contained and transported within the Body section of BizTalk documents. There is no requirement for a BizTalk Framework Compliant Server to examine the contents of the enclosed business document and validate that the document conforms to its published schema. This role can be performed by an adapter or the receiving application. While the business document is transported by the BizTalk Framework, it is not actually a part of the Framework. This separation of transport architecture and transported object guarantees that the BizTalk Framework will remain adaptable to changes and extensions in business document payloads in the future.

A schema (not shown in the above figure) is a published document that defines the structure of a particular class of business document. The schema is a detailed description of the valid elements and attributes for the business document, and the allowed placement and parent-child structure of the elements within the document. The schema includes information describing the required data type for each element, whether each child element is required or optional, whether the child elements must appear in a specific order, whether a given child element can appear multiple times beneath a single parent, etc. If an incoming business document does not conform to its schema, the document is invalid and should be rejected by adapter, BFC Server, or whatever software agent is performing the conformance validation. If an incoming document is invalid, no action should be taken based on the document. There is no requirement for an application to notify the sender of a nonconforming document that the document has been rejected. The currently supported schema types are XDR Schemas (http://www.itg.ed.ac.uk/~ht/XMLData-Reduced.htm), W3C XML (XSD) Schemas (http://www.w3.org/TR/xmlschema-1), and BizTalk Server Specifications. BizTalk schemas may be published in the BizTalk Schemas Library at BizTalk.org (http://www.biztalk.org/xxxxx?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq) or at another site that is publicly accessible.

A BizTalk Framework Compliant (BFC) Server is defined as "the set of services providing the message-processing functionality defined in the BizTalk Framework specifications" in the BizTalk Framework 2.0 specification. This intentionally vague description points out the fact that while Microsoft is the biggest participant in the BizTalk initiative, the BizTalk Framework is a business process standard that does not depend on Microsoft's BizTalk Server product. Certainly, BizTalk Server is at present the most complete implementation of a BizTalk Framework Compliant Server, but other BFC Servers could be developed by other vendors. There is no requirement that a specific technology be utilized in a BFC Server, and the necessary inputs, outputs, and processing mechanisms are all based on open industry standards (SOAP, XML, MIME, S/MIME, SLL, etc.). In this sense, the BizTalk Framework is similar to J2EE There, dozens of vendors have applied varying technologies to develop J2EE-compliant application servers. The internals of the servers (J2EE application servers or BFC Servers) are irrelevant, so long as the functionality embodied in the specification is incorporated into the server.

A BizTalk Document is a SOAP 1.1 message that consists of a header element containing BizTalk message handling information and a body element containing one or more business documents. The BizTalk document header element contains at minimum an <endpoints> element and a <properties> element. The <endpoints> element specifies the source and destination of the BizTalk document, that is the "to" address and "from" address. The <properties> element contains identification and time information: a URI reference that uniquely identifies this BizTalk document, the time the document was sent, the time when the document expires, and the topic URI reference. The <expiresAt> element illustrates an important feature of BizTalk and the BizTalk Framework: any BizTalk document that is received after the expiration time must not be processed or acknowledged by the receiving server. An expired document is treated as null and void. This is one of several mechanisms applied by the BizTalk Framework to minimize potential problems that could occur under conditions of network outages, lost or duplicated messages, etc. BizTalk document headers may also include a <services> element, which contains message receipt and commitment instructions; a <manifest> element containing document catalog information; and a <process> element that correlates this BizTalk document with a specific type of business process and instance. Validating and processing the header portion of BizTalk documents is a primary function of a BizTalk Framework Compliant Server.

A BizTalk Message is defined in the specification (page 8) as "the unit of wire-level interchange between BFC Servers". Each BizTalk message contains a primary BizTalk document and possibly related attachment files. The attachments may include additional BizTalk documents that are related to the primary BizTalk document in the message. The exact format of a BizTalk message is dependent on the transport being used to send the message (SMTP, HTTP, etc.), often resulting in the actual BizTalk message being wrapped in or preceded by data structures that are specific to the transport protocol.

The BizTalk Framework 2.0 specification includes detailed descriptions of processing and protocols governing reliable delivery, attachments, and security. While these aspects of the BizTalk Framework have not been addressed in the sample program in this case study, it is critical for them to be properly implemented in any automated business-to-business document interchange that is put into production.

BizTalk Framework 2.0 does not address every aspect of business process integration. For example, there is no reference to processing of transactions (with commit and roll-back) in the specification. The framework "does not deal directly with legal issues, agreements regarding arbitration, and recovery from catastrophic failures, nor does it specify specific business processes such as those for purchasing or securities trading." (BizTalk Framework 2.0: Document and Message Specification, page 4).

The BizTalk Framework 2.0 specification summarizes the framework as providing "a set of basic mechanisms required for most business-to-business electronic exchanges" (page 4). The framework is not intended to be a static standard; rather, it is a growing and evolving entity, designed for future enhancement to leverage new XML and other internet and message-related standards as they gain acceptance by the business and technology communities.

Microsoft BizTalk Server

Microsoft BizTalk Server is not a part of the BizTalk Framework, but it is a BizTalk Framework Compliant (BFC) Server. Indeed, it is, and may remain the only BFC Server that complies with the full BizTalk Framework specification. The purpose of BizTalk Server is to support Enterprise Application Integration using the BizTalk Framework. BizTalk Server can be integrated with Microsoft Application Center for scalability. Over 300 adapters have been developed by Microsoft and many other software companies, providing BizTalk Server with a capability for delivering documents between an enormous number of combinations of enterprise software packages and legacy and modern software and hardware technologies.

BizTalk Server includes many sub-components, including:

  • BizTalk Editor: defines and edits schema for any structured document type
  • BizTalk Orchestration Designer: defines and builds complex distributed business processes
  • BizTalk Mapper: maps elements from one document format to the corresponding elements on another document type
  • BizTalk Messaging Manager: defines trading partnership agreements
  • BizTalk Server Groups: horizontal scaling for clusters dedicated to the same type of BizTalk processing

BizTalk Server also includes direct integration capabilities for UN/EDIFACT and ANSI X12 EDI, HIPAA, RosettaNet Implementation Framework 1.1, and Chemical Industry Data Exchange (CIDX). Security functionality includes support for Secure/Multipurpose Internet Mail (S/MIME) and Public Key encryption Infrastructure (PKI).

It can be assumed that most companies that decide to use the BizTalk Framework for enterprise application integration will purchase BizTalk Server. However, using BizTalk Server is not the only way to apply the BizTalk Framework. Custom software that interfaces with applications and converts data into BizTalk documents can be built to provide an outward facing interface into applications for which no adapter exists. The sample BizTalk Processor application in this case study illustrates how some portions of the framework can be implemented in relation to a specific legacy application.

Defining the Problem: Technology Aspects

The need for business process integration and customer self-service is clear. Competition within a specific industry sector will require all companies to integrate their processes and provide better and more efficient customer service once the leading companies in the sector bring the first core Business Process Integration implementations on-line. As is the case with all technologies that transform business processes, once the long-term gains in efficiency and reduced cost become illustrated in practice by the pioneers, the entire industry follows suit, resulting in a full-scale transformation in the industry's way of doing business.

Once the decision to integrate business processes has been made at the executive level, the Information Systems and/or Development teams are assigned with the task of planning and implementing the required systems and technology. In this case study, this task has been presented at a mythical electric utility called Central Power and Light. The company runs a software application called LegacyApp when it requires historical billing information for individual customer accounts. LegacyApp was used to provide information on usage patterns to enable the company to better plan seasonal power resources. The data output by LegacyApp also contained the details presented on customer bills, and so feeds from LegacyApp were sometimes used as a reference by the company's customer service department.

Now management has decided that it would be ideal if the LegacyApp information could be sent on request in an automated manner to other applications within the company and possibly even to a self-service web site where customers could view their own historical billing information. The latter would be a convenience to customers, who would no longer have to wait to speak with a customer service representative or send a written request for the previous bill details. It would also enable the company to increase efficiency, by maintaining customer service personnel only for non-routine customer inquiries and requests.

Defining the Technology Solution

Unfortunately, the data structure output by LegacyApp, a flat file with no labels, is totally unsuitable for these purposes. Changing LegacyApp itself is not an option: the application is very old, it is highly interwoven with a host of other business critical legacy software components, and there is no one on the current development staff who has ever looked at the LegacyApp source code. Touching LegacyApp - even recompiling it - seems like an enormously risky option. If that's even possible: does anyone actually know where to find the version of LegacyApp source code that matches what's been running for the past N years? The Development Manager quickly decides against all strategies that call for changing LegacyApp.

The only alternative is some type of interface that would meet the objectives of providing a modern interface into LegacyApp. But how could such an interface be made? In an actual business scenario, LegacyApp may run on a mainframe server that is isolated from the Internet for security reasons. Security policies might forbid direct network contact between the LegacyApp machine and a machine that faces the Internet. In addition, direct invocation of the mainframe-resident LegacyApp may not be possible from a Windows 2000 server. The only interface possible could be via file exchange.

Let's assume all of the above is the case. The senior members of the development team consider the extensive problems that need to be overcome and evaluate the limited selection of available alternatives. They need something that will provide very flexible and generic application interface capabilities. After much deliberation, they decide that .NET and the BizTalk Framework appear to be cost-effective technologies that will meet the near term objectives for the project and provide extensibility for future enhancement and adaptation. Management goes along with this idea, but requires a proof of concept before proceeding with the full scale project, which eventually could include clusters of BFC Servers sending requests and receiving responses from LegacyApp.

The software presented in this case study is an early prototype the development team engineers for its first feasibility demonstration to management.

Legacy Application Data Access

A common method for data input and output in legacy applications is through the use of "flat files." Flat files are text files with a specified format that is typically unique to each specific application. The BizTalk Mapper component of Microsoft's BizTalk Server provides extensive capabilities for mapping fields in formatted flat files with elements and attributes in XML business documents.

The LegacyApp example application presented in this case study provides access to historical electricity billing records. The application reads an input command file that specifies the requesting user and the account data requested. If the user's permissions allow access to the requested information, the account details are written to an output flat file. If an error condition occurs, then the error message is written to the output file.

LegacyApp Command File Structure

The LegacyApp input command file consists of a two-line header followed by a single line for each bill data request. The fields are space-delimited on each line. The file is structured as follows:

Line | Field Number | Field Type | Field Definition
1 | 1 | String | user name for logging into the legacy application
1 | 2 | String | password for logging into the legacy application
2 | 1 | Integer | nBill: number of bills requested in the following lines
3+ | 1 | Integer | 5-digit account number for requested bill
3+ | 2 | Integer | 4-digit year of period for requested bill
3+ | 3 | Integer | 1- or 2-digit numeric month identifying period for requested bill

The total number of lines in a valid LegacyApp command file is two, plus the number of requested bills. The following is an example file:

britney britneyp
12
55555 2001 1
55555 2001 2
55555 2001 3
55555 2001 4
55555 2001 5
55555 2001 6
55555 2001 7
55555 2001 8
55555 2001 9
55555 2001 10
55555 2001 11
55555 2001 12

In order to request billing data from the legacy application, the Biztalk application must extract the business document from the incoming BizTalk document, validate the business document with respect to the published Biztalk schema, and translate the billing requests into a command file in this format.

LegacyApp Data Delivery File Structure

The LegacyApp output data delivery file consists of a set of space-delimited lines, each containing the detailed billing data for a monthly billing period. The legacy data delivery file is structured as follows:

Line | Field Number | Field Type | Field Definition
1+ | 1 | integer | 5-digit account number for requested bill
1+ | 2 | integer | 4-digit year of period for requested bill
1+ | 3 | integer | 1- or 2-digit numeric month identifying period for requested bill
1+ | 4 | float | kilowatt-hours of electricity used in the monthly billing period
1+ | 5 | float | generation services charge
1+ | 6 | float | customer service charge
1+ | 7 | float | transmission charge
1+ | 8 | float | distribution charge
1+ | 9 | float | systems benefits charge
1+ | 10 | float | load management charge
1+ | 11 | float | conservation charge
1+ | 12 | float | renewable energy investments charge
1+ | 13 | float | fuel adjustment charge
1+ | 14 | float | total charge: total amount billed for the month

The total number of lines in a valid LegacyApp data delivery file will match the number of requested bills. The following is an example data delivery file that corresponds to the command file listed above:

55555 2001 1 1067.794434 51.720001 11.320000 6.406767 32.033835 4.805075 3.203384 1.601692 0.800846 3.155333 115.046933
55555 2001 2 958.180054 51.720001 11.320000 5.749081 28.745403 4.311810 2.874540 1.437270 0.718635 2.831422 109.708163
55555 2001 3 655.962830 51.720001 11.320000 3.935777 19.678885 2.951833 1.967889 0.983944 0.491972 1.938370 94.988672
55555 2001 4 463.082153 51.720001 11.320000 2.778493 13.892465 2.083870 1.389246 0.694623 0.347312 1.368408 85.594417
55555 2001 5 572.241333 51.720001 11.320000 3.433448 17.167241 2.575086 1.716724 0.858362 0.429181 1.690973 90.911017
55555 2001 6 874.381653 51.720001 11.320000 5.246290 26.231448 3.934717 2.623145 1.311572 0.655786 2.583798 105.626757
55555 2001 7 1067.640503 51.720001 11.320000 6.405843 32.029214 4.804382 3.202921 1.601461 0.800730 3.154878 115.039430
55555 2001 8 958.936707 51.720001 11.320000 5.753620 28.768101 4.315215 2.876810 1.438405 0.719203 2.833658 109.745013
55555 2001 9 656.874207 51.720001 11.320000 3.941245 19.706226 2.955934 1.970623 0.985311 0.492656 1.941063 95.033058
55555 2001 10 463.237701 51.720001 11.320000 2.779426 13.897131 2.084570 1.389713 0.694857 0.347428 1.368867 85.601994
55555 2001 11 571.485718 51.720001 11.320000 3.428914 17.144572 2.571686 1.714457 0.857229 0.428614 1.688740 90.874213
55555 2001 12 873.469727 51.720001 11.320000 5.240818 26.204092 3.930614 2.620409 1.310205 0.655102 2.581103 105.582344

The BizTalk application must read the billing data from this file structure, transform it into a business document formatted in accordance with the published BizTalk schema, and apply the appropriate SOAP wrapper to generate a valid BizTalk document to be delivered to the requesting entity.

Component Design

The LegacyApp command file structure defines the necessary input data elements for accessing monthly billing data from the Central Power bill repository, while the LegacyApp data delivery file structure identifies the data elements that are available for publication or delivery to a requesting entity. However, the formats of the files are proprietary. They do not conform to any standards-based Enterprise Application Integration format.

Altering the legacy application itself will be out of the question for a system that has been in place for decades and is interwoven with perhaps hundreds of other proprietary applications that are critical for the daily operations of the company. For example, a legacy application that manages records of electricity charges will be intimately coupled with other legacy applications that manage meter readings, customer information, categories of charges, accounts receivable, etc. The legacy applications are probably tightly coupled in a way that will cause several critical systems to fail if a single system fails. Therefore, the only acceptable way to enable standards-based access to the legacy data is to define and publish an interface that conforms to an EAI standard, and purchase or develop a software component that translates between the published interface and the proprietary legacy interface.

Designing and Publishing BizTalk Interfaces

If a company wishes to enable open standards-based access to portions of its legacy data store using the BizTalk Framework, it must first design and publish an interface. BizTalk.org is the central repository for such interfaces. For the Central Power LegacyApp application, two interface structures are required: an incoming request structure, which will contain the information required to generate the LegacyApp command file; and an outbound response structure, which will contain the billing details for the requested accounts and billing periods.

The most important factor in designing a BizTalk interface is the structure of the business document. The business document is an XML document whose structure is published in great detail at Biztalk.org. Entities that may wish to interact with an application will use the published business document structure as an indicator of whether or not the application can meet their needs. In addition, since an interface is being published, it is critical to design the interface such that future changes will most likely be minimal. For an organization (internal or external) that has developed software conforming to a published interface, changes to the interface are costly and also entail risk of downtime for mission-critical systems.

CentralPowerRequest BizTalk Schema

The data required to construct a command file for the Central Power LegacyApp is conveniently encapsulated in an XML structure exemplified by the following XML file (CPRequest.xml):

<?xml version="1.0" ?>
<biztalk_1 xmlns="urn:schemas-biztalk-org:biztalk:biztalk_1">
<body>
<CentralPowerRequest xmlns="urn:schemas.biztalk.org:Central_Power.com/1wfknb9j.xml">
<requestID>123</requestID>
<user>admin</user>
<password>adminp</password>
<billDetailRequests>
<billDetailRequest>
<account>55555</account>
<year>2001</year>
<month>1</month>
</billDetailRequest>
</billDetailRequests>
</CentralPowerRequest>
</body>
</biztalk_1>

CPRequest.xml represents the business document portion of a BizTalk document, which is the portion that is published in the BizTalk.org repository library. The portion of the file designed by the publishing entity lies within the <body> element. Examining this section of CPRequest.xml, you see a structure composed in XML that is comparable with the LegacyApp command file structure presented above. The user name and password are followed by a groups of account/year/month data. In the command flat file, these parameter sets appear as space-delimited lines of text. In the CPRequest.xml file, they appear as child elements of <billDetailRequest> elements. The nBill number of bills requested data element does not appear in the XML document. Since the number of child elements is an easily extractable property, there is no need to include it in the public <CentralPowerRequest> interface.

The other notable difference between the XML <CentralPowerRequest> and the LegacyApp command file is the <requestID> element. This is inserted into the interface to provide a capability for identifying individual requests which may be identical with respect to all other elements, and also to enable maintenance of an historical database of received requests. This record-keeping would not be performed by the LegacyApp itself, since it has no capability for receiving and processing the requestID. Rather, the BizTalk interface software would have to perform whatever logging or database record-keeping was required.

While an example XML business document that represents a BizTalk interface is published at BizTalk.org, the primary reference interface definition is published as an XML Schema. The schema definition is termed an "object" at BizTalk.org (refer to http://www.biztalk.org/Help/help.asp?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq). At BizTalk.org, organizations make "contributions" consisting of BizTalk "objects" (schema), keywords, detailed object descriptions, and sample XML files that represent instances of the objects. BizTalk.org considers the name of the object to be of critical importance. The section "How do I publish my objects" states:

"Naming a schema requires special consideration. In most cases, the recipient of an XML document uses the name of the schema's top element as a signal for what is required. Therefore, using a well-structured name now will save a log of confusion later when there are many schemas to manage."

BizTalk.org allows up to 64 characters for the Object Name field, inviting very descriptive names that include both the originating organization and terms that accurately categorize the object's function.

In summary, publishing a BizTalk interface at BizTalk.org requires the originating organization to provide the following for each BizTalk object:

  • Object (XDR schema, W3C XML (XSD) schema, or BizTalk Server specification);
  • Object name (up to 64 characters)
  • Summary (brief description, up to 256 characters)
  • File name (identifies the object file on the BizTalk.org server)
  • Keywords (used by BizTalk.org search engine; up to 256 characters)
  • Object location (can be at BizTalk.org, or elsewhere)
  • Object sample location (location of the sample XML file; can be at BizTalk.org or elsewhere)
  • Documentation (detailed technical document written as an HTML document, presenting field definitions, error handling, contact information, etc.)

The CentralPowerRequest XDR schema looks like this:

<?xml version="1.0" ?>
<Schema name="CentralPowerRequest.xsd" xmlns="urn:schemas-microsoft-com:xml-data"
xmlns:dt="urn:schemas-microsoft-com:datatypes">
<ElementType name="CentralPowerRequest" content="eltOnly" model="closed" order="seq">
<element type="requestID"/>
<element type="user"/>
<element type="password"/>
<element type="billDetailRequests"/>
</ElementType>
<ElementType name="requestID" content="textOnly" dt:type="string"/>
<ElementType name="user" content="textOnly" dt:type="string"/>
<ElementType name="password" content="textOnly" dt:type="string"/>
<ElementType name="billDetailRequests" content="eltOnly" model="closed" order="many">
<element type="billDetailRequest" minOccurs="0"/>
</ElementType>
<ElementType name="billDetailRequest" content="eltOnly" model="closed" order="seq">
<element type="account"/>
<element type="year"/>
<element type="month"/>
</ElementType>
<ElementType name="account" content="textOnly" dt:type="string"/>
<ElementType name="year" content="textOnly" dt:type="string"/>
<ElementType name="month" content="textOnly" dt:type="string"/>
</Schema>

The XDR schema is essentially a map defining the required format for an XML file. Browsing the CentralPowerRequest schema and comparing it with the example CentralPowerRequest business document presented above, you can see the correlation in structure between the schema and the business document. Each <ElementName> element in the schema identifies an element in the business document. The attributes of <ElementName> in the schema define the name of the element, the types of child node it can have, whether the nodes are required, whether they must occur in a definite sequence, etc. "eltOnly" means that the element can only have child elements as its content (that is, the element does not consist of, for example, a text string). "textOnly" means that the element consists only of a text string and cannot have child elements. The <element> child elements beneath the schema <ElementName> identify the possible child elements for each parent element in the business document. A "closed" model means all child elements are required to be present. order="seq" means the child elements in the business document must appear in the exact order listed in the schema. Full details of XDR schema are available in the XDR specification at the address listed above.

Before the full package is submitted to BizTalk.org, it is wise to validate the schema and example business document files. BizTalk.org provides a tool to perform this validation in the Help pages (http://www.biztalk.org/Help/help.asp?WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq) under the heading "How do I validate my objects?". Executing the download exe file (XDR_Schema_Validator.exe) produces the file BizTalkSchemaValidator.exe. This program is used by BizTalk.org to validate submitted objects and sample files before they are published on the BizTalk.org site.

BizTalkSchemaValidator.exe has a simple GUI which prompts the user to specify an XDR schema and an XML file that is intended to conform to the schema. First, the schema itself is examined for consistency with the XDR schema standard. If the schema is valid, the XML document is examined and validated for conformance to the schema. Any errors are reported in the text box at the bottom of the GUI. Running BizTalkSchemaValidator.exe against the CPRequestSchema.xml and CPRequest.xml files produces the following result:

Both the schema and sample file pass all tests and would be published were they submitted to BizTalk.org with appropriate accompanying documentation.

CentralPowerResponse BizTalk Schema

The CentralPowerResponse BizTalk interface is designed similarly to the CentralPowerRequest interface. This is reasonable since the corresponding LegacyApp command and data delivery flat files are similar in structure, both consisting primarily of sets of lines containing a fixed number of space-delineated fields. The following is an example CentralPowerRequest business document:

<?xml version="1.0" ?>
<biztalk_1 xmlns="urn:schemas-biztalk-org:biztalk:biztalk_1">
<body>
<CentralPowerResponse xmlns="urn:schemas.biztalk.org:Central_Power.com/1wfknb9k.xml">
<requestID>123</requestID>
<requestStatus>Success</requestStatus>
<user>admin</user>
<monthlyBills>
<monthlyBill>
<account>55555</account>
<year>2001</year>
<month>1</month>
<kWh>1067.794434</kWh>
<generationServicesCharge>51.720001</generationServicesCharge>
<customerServiceCharge>11.320000</customerServiceCharge>
<transmissionCharge>6.406767</transmissionCharge>
<distributionCharge>32.033835</distributionCharge>
<systemsBenefitsCharge>4.805075</systemsBenefitsCharge>
<loadManagementCharge>3.203384</loadManagementCharge>
<conservationCharge>1.601692</conservationCharge>
<renewableEnergyInvestmentCharge>0.800846</renewableEnergyInvestmentCharge>
<fuelAdjustmentCharge>3.155333</fuelAdjustmentCharge>
<totalCharges>115.046933</totalCharges>
</monthlyBill>
</monthlyBills>
</CentralPowerResponse>
</body>
</biztalk_1>

The primary difference in content between the XML document and the LegacyApp data delivery flat file is that the XML business document includes status and identification information that is not present in the legacy document. Since multiple entities could make very similar or identical requests for billing detail data, it is important to supplement the raw billing data output by the legacy application with information that will uniquely identify the content of the document, including the request that originated the invocation of the legacy application. The element <requestStatus> is included in the response document to provide a location for sending problem and error messages back to the requesting entity, in the event that either the BizTalk processor component or the legacy application throws a fatal error that has a meaningful message text.

The XDR schema corresponding to the response business document is as follows:

<?xml version="1.0" ?>
<Schema name="CentralPowerResponse.xsd" xmlns="urn:schemas-microsoft-com:xml-data"
xmlns:dt="urn:schemas-microsoft-com:datatypes">
<ElementType name="CentralPowerResponse" content="eltOnly" model="closed" order="seq">
<element type="requestID"/>
<element type="requestStatus"/>
<element type="user"/>
<element type="monthlyBills"/>
</ElementType>
<ElementType name="requestID" content="textOnly" dt:type="string"/>
<ElementType name="requestStatus" content="textOnly" dt:type="string"/>
<ElementType name="user" content="textOnly" dt:type="string"/>
<ElementType name="monthlyBills" content="eltOnly" model="closed" order="many">
<element type="monthlyBill" minOccurs="0"/>
</ElementType>
<ElementType name="monthlyBill" content="eltOnly" model="closed" order="seq">
<element type="account"/>
<element type="year"/>
<element type="month"/>
<element type="kWh"/>
<element type="generationServicesCharge"/>
<element type="customerServiceCharge"/>
<element type="transmissionCharge"/>
<element type="distributionCharge"/>
<element type="systemsBenefitsCharge"/>
<element type="loadManagementCharge"/>
<element type="conservationCharge"/>
<element type="renewableEnergyInvestmentCharge"/>
<element type="fuelAdjustmentCharge"/>
<element type="totalCharges"/>
</ElementType>
<ElementType name="account" content="textOnly" dt:type="string"/>
<ElementType name="year" content="textOnly" dt:type="string"/>
<ElementType name="month" content="textOnly" dt:type="string"/>
<ElementType name="kWh" content="textOnly" dt:type="string"/>
<ElementType name="generationServicesCharge" content="textOnly" dt:type="string"/>
<ElementType name="customerServiceCharge" content="textOnly" dt:type="string"/>
<ElementType name="transmissionCharge" content="textOnly" dt:type="string"/>
<ElementType name="distributionCharge" content="textOnly" dt:type="string"/>
<ElementType name="systemsBenefitsCharge" content="textOnly" dt:type="string"/>
<ElementType name="loadManagementCharge" content="textOnly" dt:type="string"/>
<ElementType name="conservationCharge" content="textOnly" dt:type="string"/>
<ElementType name="renewableEnergyInvestmentCharge" content="textOnly" dt:type="string"/>
<ElementType name="fuelAdjustmentCharge" content="textOnly" dt:type="string"/>
<ElementType name="totalCharges" content="textOnly" dt:type="string"/>
</Schema>

Running BizTalkSchemaValidator.exe against CPResponseSchema.xml and CPResponse.xml verifies that the schema is valid and that the sample business document conforms to the schema.

Since the CentralPowerResponse schema is different from the CentralPowerRequest schema, they would need to be submitted to BizTalk.org as unique objects; a full set of documentation would be required for each.

BizTalk documents containing business documents conforming to the Central Power schemas could be processed by any BizTalk Framework compliant (BFC) server, for example, Microsoft's BizTalk Server. While the BizTalk Processor application developed in this case study is by no means a fully compliant BFC server, its processing stages illustrate key functions that are necessary to implement the BizTalk Framework.

UI Design

The BTProc Processing Stage section of the user interface illustrates the processing steps performed by the prototype software. The sequential processing steps are as follows:

  • Scan the ./incoming directory for a new incoming BizTalk document
  • Validate that the structure of the BizTalk document is correct
  • Process the extracted business document
  • Generate the LegacyApp command file
  • Scan the LegacyApp output directory for the LegacyApp data delivery result file
  • Process the LegacyApp data delivery file
  • Generate the response BizTalk document

As can be seen, the prototype software performs a subset of the duties of a BFC Server and also fulfills the role of an adapter for the LegacyApp.

BizTalk Processor Design

The BizTalk Processor's core functionality is in mapping BizTalk documents with the LegacyApp input and output "documents". To accomplish this, it must interpret received BizTalk documents, translate the received content into a LegacyApp command file, then read the LegacyApp response data delivery file, and translate its content into a response BizTalk document. This is essentially the functionality that is automated by the Mapper component of BizTalk Server.

BizTalk Document Input

The BizTalk Processor application (BTProc) processes incoming documents one at a time. The processing of a document is launched when the user clicks the Start Processing button in the sample application, to provide the reader with a way to evaluate each processing step for each individual document that is processed by the application. In a production environment, the BizTalk Processor would most likely be structured to run as a Windows service that watches the incoming BizTalk documents directory and immediately processes each BizTalk document once it has determined to have been stabilized.

The BTProc example software supports only receipt of BizTalk documents in a directory. Many additional protocols for document exchange are available, including Simple Mail Transfer Protocol (SMTP), HTTP, Electronic Data Interchange (EDI), and Value-Added Network Service (VANS). Full scale BizTalk Framework compliant servers, such as Microsoft BizTalk Server, support these varieties of BizTalk document exchange.

Component Coding

The code that processes each incoming BizTalk document is invoked when the Start button is clicked. In a production system, the BTProc software would most likely run as a service, with no human interaction required except for monitoring status and certain error conditions. But since the purpose of the prototype BTProc is to demonstrate BizTalk concepts, the processing for each incoming BizTalk document is linked to a press of the Start button. When the user clicks Start, the button click event invokes the function btnStart_Click(). This function consists of an initialization section followed by a section of code for each Processing Stage label on the user interface. When each processing stage begins, the radio button for that stage is turned on, the radio button for the previous stage is turned off, and a message indicating that the new processing stage has initiated is displayed in the Status/Result text box to the right of the radio button. As the processing proceeds within a processing stage, the Status/Result text box is updated, to show progress and any error conditions that occur. When the processing stage is complete, a final status message is displayed, and processing moves to the next stage.

The Verbose check box is provided to allow viewing the BizTalk document processing in a step-by-step manner. When Verbose is checked, the processing pauses each time a new Status/Result entry is written and at various other points during a processing stage when display of data being processed is useful for tracking the processing of a specific input document.

Processing Stage 1: Scanning for incoming documents

The System.IO.Directory class is used to search the ./ incoming directory for new input BizTalk documents. The Directory class is an easy-to-use, versatile class that provides basic directory status and management capabilities, including directory creation, deletion, file lists, creation and write times, etc. In BTProc, the method Directory.GetFiles() returns a string array containing all of the files found in the ./incoming directory. The array Length() method returns a count of the number of elements in the fileNames[] list returned by GetFiles(). If no files were found, a message is displayed and processing exits. If one or more files exist in the directory, the file returned in the first fileNames[] position is selected for processing in the next stage.

//stage 1: scan for incoming
            files
radioScan.Checked=true;
textBoxScan.Text = "Searching
            for incoming files";
textBoxScan.Refresh();

int
            nFiles=0;
String[]
            fileNames;
fileNames=Directory.GetFiles(".\\incoming");
nFiles=fileNames.Length;
if
            (nFiles == 0)
{
textBoxScan.Text = "No incoming files
            found...";
if
            (checkBoxVerbose.Checked)
{
MessageBox.Show("No incoming files
            found in directory .\\incoming");
MessageBox.Show("Processing
            Stage 1 failed.");
}
radioScan.Checked =
            false;
btnStart.Enabled = true;
btnClear.Enabled =
            true;
return;
}

incomingFile =
            fileNames[0];
textBoxScan.Text = "found " +
            incomingFile;
textBoxScan.Refresh();
if
            (checkBoxVerbose.Checked)
{
MessageBox.Show(textBoxScan.Text);
MessageBox.Show("Processing
            Stage 1 is complete.");
}

Again, in a production environment, where the system was running as an asynchronous service, the Stage 1 processing would actually consist of a continuous polling of the ./incoming directory. Any time a new file was found, it would be automatically be accessed and processed through the subsequent stages. Directory polling of this type is utilized in Processing Stage 5, "Searching for LegacyApp output file," described below.

Processing Stage 2: Validating incoming BizTalk document

In a fully compliant BizTalk Framework server, this processing stage would require enormous amounts of code. Validating received BizTalk documents is one of the primary functions of a BFC Server. The contents of the BizTalk document header contain all of the information required for transaction-based processing of the type that would be performed by trading partners. Indeed, much of the BizTalk Framework 2.0 specification is devoted to the required contents and processing of each "BizTag" element contained in the BizTalk document header.

In a demonstration prototype, the validation is necessarily pared down quite a lot. In addition, the focus of the BTProc prototype is on creating a standards-based interface into the LegacyApp data store. For this reason, greater attention was given to the data mapping functions that would exist in a BFC Server and adapter configuration, with less focus on the types of analysis and validation that would occur between two BFC Servers. In other words, in BTProc the focus is more on the business document stored in the BizTalk document body, and less on the BizTalk document header.

That said, basic validation of the input BizTalk document is performed in processing stage 2. The System.Xml XmlDocument() and XmlNodeReader() classes are used to contain and read the BizTalk document. The XmlDocument() class represents an XML document using the W3C Document Object Model (DOM). The DOM model views the XML document as a tree structure. The XmlDocument() class is very broad, and provides properties and methods enabling analysis and navigation of the XML document with respect to child nodes, lists of attributes, node depth, counts of number of attributes and child elements, etc. The XmlNodeReader() class provides capability for reading an XML DOM tree. The class also implements the ReadInnerXml() and ReadOuterXml() methods, which provide a method for reading an entire XML subtree into a string.

//stage 2: validate incoming BizTalk
            document

radioScan.Checked=false;
radioInDocValid.Checked=true;
textBoxInDocValid.Text=incomingFile;
textBoxInDocValid.Refresh();
XmlDocument
            doc = new XmlDocument();
try
{
//load the incoming
            file
doc.Load(incomingFile);

//create a new XmlNodeReader
            to read the XML
//from the doc XmlDocument
            container
XmlNodeReader nr = new
            XmlNodeReader(doc);

//validate that the document is a BizTalk
            Document
//by verifying the existence of the SOAP
            envelope,
//SOAP header, and SOAP body elements
nr.Read();
if (nr.NodeType !=
            XmlNodeType.XmlDeclaration)
{
textBoxInDocValid.Text =
            "XmlDeclaration error";
if
            (checkBoxVerbose.Checked)
{
MessageBox.Show(textBoxInDocValid.Text);
MessageBox.Show("Processing
            Stage 2 failed.");
}
btnStart.Enabled =
            true;
btnClear.Enabled =
            true;
return;
}

nr.Read();
if (!((nr.NodeType ==
            XmlNodeType.Element)
            &&
(nr.Name.CompareTo("SOAP-ENV:Envelope") ==
            0)))
{
textBoxInDocValid.Text = "SOAP-ENV:Envelope
            error";
if
            (checkBoxVerbose.Checked)
{
MessageBox.Show(textBoxInDocValid.Text);
MessageBox.Show("Processing
            Stage 2 failed.");
}
btnStart.Enabled =
            true;
btnClear.Enabled =
            true;
return;
}

nr.Read();
if ((nr.NodeType ==
            XmlNodeType.Element)
            &&
(nr.Name.CompareTo("SOAP-ENV:Header") ==
            0))
{
soapHeader=nr.ReadOuterXml();
}
else
{
textBoxInDocValid.Text
            = "SOAP-ENV:Header error";
if
            (checkBoxVerbose.Checked)
{
MessageBox.Show(textBoxInDocValid.Text);
MessageBox.Show("Processing
            Stage 2 failed.");
}
btnStart.Enabled =
            true;
btnClear.Enabled = true;
return;
}

if
            ((nr.NodeType == XmlNodeType.Element)
            &&
(nr.Name.CompareTo("SOAP-ENV:Body") ==
            0))
{
textBoxInDocValid.Text = "SOAP envelope is
            valid";
textBoxInDocValid.Refresh();
}
else
{
textBoxInDocValid.Text
            = "SOAP-ENV:Body error";
if
            (checkBoxVerbose.Checked)
{
MessageBox.Show(textBoxInDocValid.Text);
MessageBox.Show("Processing
            Stage 2 failed.");
}
btnStart.Enabled =
            true;
btnClear.Enabled = true;
return;
}

if
            (checkBoxVerbose.Checked)
{
MessageBox.Show(textBoxInDocValid.Text);
MessageBox.Show("Processing
            Stage 2 is complete.");
}

The Stage 2 code applies the XmlDocument.Load() method to load the input BizTalk document into memory. All of the input document processing is performed within a try block so that any I/O error is trapped and properly handled. The try block actually spans processing stages 2 and 3, which both read the incoming document.

The XmlNodeReader() reads the nodes of the document and validates that the document contains in sequence:

an XmlDeclaration node (for example, <?xml version='1.0'?>);

a <SOAP-ENV:Envelope> element node;

a <SOAP-ENV:Header> element node;

a <SOAP-ENV:Body> element node.

The XmlNodeReader.ReadOuterXml() method is executed to read the <SOAP-ENV:Header> node and its entire subtree into the soapHeader string, which is saved for later use. The string returned by ReadOuterXml() includes the element that is the starting point of the read, in this case <SOAP-ENV:Header>. This is the portion of the BizTalk document that would be thoroughly analyzed in a fully-compliant BizTalk Server. For the purposes of the BTProc prototype, we simply validate that the SOAP header structure exists.

If any errors occur, or if one of the required BizTalk document structures is missing or does not appear in the expected place, an error message is displayed and no further processing is performed on the input BizTalk document. No attempt is made to send a response back to the sending application. Since the BizTalk document as it was received is not valid, any information it might contain regarding the sender is considered unreliable.

After updates of the status text box, the Refresh() method is called. Strictly speaking, this should not be necessary. However, on my particular system (IBM NetVista running Windows 2000 Professional), working with Visual Studio .NET 7.0.9274 (Beta), setting the Text property often did not result in visible text. There is nothing in the .NET documentation that suggests we would need to call the Refresh() method to make text visible, but calling Refresh() was necessary in my particular environment. If Refresh() was not called, the text boxes remained blank after the Text property was set.

Processing Stage 3: Processing extracted business document

The Processing Stage 3 code reads and analyzes the contents of the <SOAP-ENV:Body> element. The XmlNodeReader() created in stage 2 is used to continue reading the input file, still within the try block created in stage 2. The stage 3 code reads each node of the business document and validates that its structure matches the requirements specified in the CentralPowerRequest schema. If any variation from the required structure is found, an error message is displayed and processing of the business document ceases.

However, while no further processing can be performed on the business document, the incoming BizTalk document structure was valid. So, in this case (unlike in Stage 2) a response BizTalk document can be generated to notify the sending application of the error condition. The fatalError flag is set, an error message is stored in the request status string reqStatus, and the processing jumps to line label Stage7, where the response BizTalk document is generated.

//stage 3: process business
            document

radioInDocValid.Checked=false;
radioProcInDoc.Checked=true;
textBoxProcInDoc.Text="evaluating
            business
            doc";
textBoxProcInDoc.Refresh();

nr.Read();
if
            ((nr.NodeType == XmlNodeType.Element)
            &&
(nr.Name.CompareTo("CentralPowerRequest") ==
            0))
{
textBoxProcInDoc.Text = "processing
            CentralPowerRequest";
textBoxProcInDoc.Refresh();
}
else
{
textBoxProcInDoc.Text
            = "business document type
            error";
textBoxProcInDoc.Refresh();
if
            (checkBoxVerbose.Checked)
{
MessageBox.Show(textBoxProcInDoc.Text);
MessageBox.Show("Processing
            Stage 3 failed.");
}
fatalError = true;
reqStatus = "Error:
            " + textBoxProcInDoc.Text;
goto
            Stage7;
}

do
{
nr.Read();
} while (nr.NodeType !=
            XmlNodeType.Element);
if (nr.Name.CompareTo("requestID") !=
            0)
{
textBoxProcInDoc.Text = "requestID element
            error";
textBoxProcInDoc.Refresh();
if
            (checkBoxVerbose.Checked)
{
MessageBox.Show(textBoxProcInDoc.Text);
MessageBox.Show("Processing
            Stage 3 failed.");
}
fatalError = true;
reqStatus = "Error:
            " + textBoxProcInDoc.Text;
goto Stage7;
}

reqID =
            nr.ReadInnerXml();
user = nr.ReadInnerXml();
passwd =
            nr.ReadInnerXml();

nBill=0;
nr.Read();
do
{
nr.Read();
accounts[nBill]
            = nr.ReadInnerXml();
years[nBill] =
            nr.ReadInnerXml();
months[nBill] =
            nr.ReadInnerXml();
nBill++;
nr.Read();
} while
            (nr.Name.CompareTo("billDetailRequest") ==
            0);

textBoxProcInDoc.Text = "found " + nBill + " bill
            requests";
textBoxProcInDoc.Refresh();

//close the
            XmlNodeReader
nr.Close();

if
            (checkBoxVerbose.Checked)
{
MessageBox.Show(textBoxProcInDoc.Text);
MessageBox.Show("Processing
            Stage 3 is complete.");
}
}
catch (Exception
            err)
{
MessageBox.Show("attempt to process file " +
            incomingFile +
" failed: " + err.Message);
btnClear.Enabled =
            true;
btnStart.Enabled = true;
return;
}

The XmlNodeReader.ReadInnerXml() method is used to quickly extract the values of the business document elements. ReadInnerXml() returns a string that does not include the element that is the starting point of the read. That is, it returns only the interior of the element. For example, executing ReadInnerXml() on <account>55555</account> will return a string with value '55555'.

In a production system, it would be advisable to validate each element tag as it is read. For brevity, the prototype BTProc simply reads the expected number of elements using ReadInnerXml() and assumes that the element names were correct. An alternative approach would be to use the XmlValidatingReader() reader class to process the business document. The XmlValidatingReader() class allows specification of an XML schema for validation of the read XML. If the XML is not valid, an XmlException is thrown. Or, a ValidationEventHandler can be defined to perform the appropriate action when an invalid XML node is encountered.

The Stage 3 code stores the request account, year, and month information in a set of arrays, and counts the entries as they are read, in preparation for writing the LegacyApp command file in Stage 4.

Processing Stage 4: Generating LegacyApp command file

The Stage 4 code, which writes the LegacyApp command file, is very straightforward. The File.CreateText() method is called to create the command file in the directory listed in the LegacyApp inDir text box. The file is assigned to StreamWriter sw, and the flat text command file is written. The file creation and writing are performed within a try block, so that any I/O error is trapped.

As with Stage 3, if an error occurs it is still possible to send a reply BizTalk document back to the sender. If an error occurs, processing of the input business document ceases, but the software branches to line Stage7 to generate the output BizTalk document.

//stage 4: generate command file for Legacy
            application
radioProcInDoc.Checked=false;
radioGenCmd.Checked=true;

String
            cmdFile = textBoxLegacyInDir.Text +
            "\\cmd.fil";
textBoxGenCmd.Text="generating " +
            cmdFile;
textBoxGenCmd.Refresh();
try
{
String
            tmpStr;
StreamWriter sw =
            File.CreateText(cmdFile);
sw.WriteLine(user + " " +
            passwd);
sw.WriteLine(nBill);
for (int i=0; i<nBill;
            i++)
{
sw.WriteLine(accounts[i]+" "+years[i]+"
            "+months[i]);
}
sw.Close();
textBoxGenCmd.Text = "completed
            " + cmdFile;
textBoxGenCmd.Refresh();
}
catch
{
textBoxGenCmd.Text="error writing " +
            cmdFile;
textBoxGenCmd.Refresh();
if
            (checkBoxVerbose.Checked)
{
MessageBox.Show(textBoxGenCmd.Text);
MessageBox.Show("Processing
            Stage 4 failed.");
}
fatalError = true;
reqStatus = "Error:
            " + textBoxGenCmd.Text;
radioGenCmd.Checked = false;
goto
            Stage7;
}

LegacyApp Operations

After Processing Stage 4 ends, LegacyApp performs whatever its normal operations are, given a command file. The mechanisms by which LegacyApp receives the command file and produces a data delivery file in the output directory are irrelevant from the point of view of the BizTalk Processor. LegacyApp is simply a 'black box', a mechanism of unknown internal construction, but which has the known characteristic of producing an output data delivery file of a specified format within a given amount of time after it is fed a properly structured input command file.

Processing Stage 5: Searching for LegacyApp output file

The Stage 5 code waits for the LegacyApp to output its data delivery file into the directory specified in the LegacyApp outDir text box. The System.IO.FileSystemWatcher.NET component is used to watch the LegacyApp output directory for the creation of, or a change in, a file named Out_cmd.fil. The FileSystemWatcher class provides powerful and flexible capability for monitoring files and directories on the local computer or on remote servers. The class can be used to trigger events when directory or file changes occur, or the class can wait for a change to occur.

The FileSystemWatcher can observe when a file or a directory is changed, created, deleted, or renamed. However, care must be taken in using the class, as can be seen in the Processing Stage 5 code. The FileSystemWatcher fsw is initialized and configured to watch the file Out_cmd.fil in the LegacyApp output directory (legOutPath). The assumption is that LegacyApp will always output a result file within 15 seconds. So, the WaitForChanged() method is called using the overloaded parameter list that includes a time limit (measured in milliseconds). The curious thing about this call is the first parameter, which is set to WatcherChangeTypes.Created | WatcherChangeTypes.Changed. Why isn't it enough to just watch for the creation of the output file? Because if a file named Out_cmd.fil already exists in the legOutPath directory when the WaitForChanged() is initiated, and LegacyApp subsequently overwrites the existing file, the Created condition is not set in the WaitForChanged(). Because the directory's contents have not changed, the FileSystemWatcher considers there to have been no new file created.

To account for this situation, WatcherChangeTypes.Changed is OR'd into the first parameter of the WaitForChanged() call. The Changed condition is thrown when a file is overwritten, even if the file is overwritten by an identical version of itself. What is changed by an overwrite is the file's last write time, which is one of the properties the FileSystemWatcher uses in evaluating the condition of files and directories.

The FileSystemWatcher.WaitForChanged() method waits for up to 15 seconds for Out_cmd.fil to be created or changed. If LegacyApp does not write a new output file into the directory within that time, the WaitForChanged() times out. An error message is generated and processing jumps to line Stage7, where the output BizTalk document is generated.

//stage 5: search for Legacy application
            output
            file

radioGenCmd.Checked=false;
radioLegacyOut.Checked=true;

String
            legOutFile = legOutPath + "\\Out_cmd.fil";

int waitSec =
            0;
int lastSec = 0;
textBoxLegacyOut.Text="wait for LegacyApp
            output file";
textBoxLegacyOut.Refresh();

//create and
            enable FileSystemWatcher; search for a created
            file
System.IO.FileSystemWatcher fsw = new
            System.IO.FileSystemWatcher();
fsw.Path=legOutPath;
fsw.Filter="Out_cmd.fil";

//wait
            up to 15 seconds for the output file to
            arrive
WaitForChangedResult fswResult =
            fsw.WaitForChanged(
WatcherChangeTypes.Created |
            WatcherChangeTypes.Changed,15000);

if
            (fswResult.TimedOut)
{
textBoxLegacyOut.Text = "LegacyApp
            output file not received";
textBoxLegacyOut.Refresh();
if
            (checkBoxVerbose.Checked)
{
MessageBox.Show(textBoxLegacyOut.Text);
MessageBox.Show("Processing
            Stage 5 failed.");
}
fatalError = true;
reqStatus = "Error:
            " + textBoxLegacyOut.Text;
radioLegacyOut.Checked =
            false;
goto Stage7;
}

textBoxLegacyOut.Text = "Found "
            + legOutFile;
textBoxLegacyOut.Refresh();
if
            (checkBoxVerbose.Checked)
{
MessageBox.Show(textBoxLegacyOut.Text);
MessageBox.Show("Processing Stage 5 is
            complete.");
}

Processing Stage 6: Processing LegacyApp output file

The Stage 6 code reads the file output by LegacyApp. The first action performed by the Stage 6 code is Thread.Sleep(5000). The purpose of this line is to allow the LegacyApp to complete its writing of its output file before BTProc tries to open and read the file. Immediately after a file is detected in the LegacyApp output directory in Stage 5, the Stage 6 processing begins. Without a delay, BTProc will attempt to open the LegacyApp file before it is completely written. While a more sophisticated approach would be employed to assess file stability in a production system, in a low volume prototype a simple fixed delay is adequate.

The LegacyApp output data delivery file has two possible formats. If an error occurred during the LegacyApp processing, the file consists of an error message that starts with the word "Problem". If no error occurred, the file consists of rows of space-delimited values representing account information for a specific month.

The StreamReader() class is used to read the output file one line at a time. The first line is read and tested using the String.StartsWith() method to see if the line begins with the word "Problem". The assumption is that "Problem" is not a valid account number and will only appear in the file if an error occurred during the LegacyApp processing of its input command file. If "Problem" is found, the message is copied into reqStatus and processing jumps to Stage7, where the message will be output as part of the outgoing business document.

//stage 6: process LegacyApp output
            file

radioLegacyOut.Checked=false;
radioProcLegOut.Checked=true;

textBoxProcLegOut.Text
            = "waiting for " + legOutFile + " to
            stabilize";
textBoxProcLegOut.Refresh();
Thread.Sleep(5000);
textBoxProcLegOut.Text = "processing " +
            legOutFile;
textBoxProcLegOut.Refresh();
if
            (checkBoxVerbose.Checked)
{
MessageBox.Show(textBoxProcLegOut.Text);
}

String[]
            legOutLines = new String[1000];

String problemMsg;
int
            nOut = 0;
bool legacyProblem = false;
try
{
StreamReader
            sr = new
            StreamReader(legOutFile);
legOutLines[0]=sr.ReadLine();
if
            (legOutLines[0].StartsWith("Problem"))
{
legacyProblem=true;
problemMsg
            = legOutLines[0];
textBoxProcLegOut.Text = legOutFile + ": " +
            problemMsg;
textBoxProcLegOut.Refresh();
if
            (checkBoxVerbose.Checked)
{
MessageBox.Show(textBoxProcLegOut.Text);
MessageBox.Show("Processing
            Stage 6 failed.");
}
fatalError = true;
reqStatus = "Error:
            " + textBoxProcLegOut.Text;
goto Stage7;
}
else
{
int
            pos0=0;
int pos1=0;
while (nOut <
            nBill)
{
pos1=legOutLines[nOut].IndexOf("
            ");
accounts[nOut]=legOutLines[nOut].Substring(0,pos1-1);
pos0=pos1+1;
pos1=legOutLines[nOut].IndexOf("
            ",pos0);
years[nOut]=legOutLines[nOut].Substring(pos0,pos1-pos0);

pos0=pos1+1;
pos1=legOutLines[nOut].IndexOf("
            ",pos0);
months[nOut]=legOutLines[nOut].Substring(pos0,pos1-pos0);

pos0=pos1+1;
pos1=legOutLines[nOut].IndexOf("
            ",pos0);
kWhs[nOut]=legOutLines[nOut].Substring(pos0,pos1-pos0);

pos0=pos1+1;
pos1=legOutLines[nOut].IndexOf("
            ",pos0);
genChgs[nOut]=legOutLines[nOut].Substring(pos0,pos1-pos0);

pos0=pos1+1;
pos1=legOutLines[nOut].IndexOf("
            ",pos0);
custChgs[nOut]=legOutLines[nOut].Substring(pos0,pos1-pos0);

pos0=pos1+1;
pos1=legOutLines[nOut].IndexOf("
            ",pos0);
transChgs[nOut]=legOutLines[nOut].Substring(pos0,pos1-pos0);

pos0=pos1+1;
pos1=legOutLines[nOut].IndexOf("
            ",pos0);
distChgs[nOut]=legOutLines[nOut].Substring(pos0,pos1-pos0);

pos0=pos1+1;
pos1=legOutLines[nOut].IndexOf("
            ",pos0);
systChgs[nOut]=legOutLines[nOut].Substring(pos0,pos1-pos0);

pos0=pos1+1;
pos1=legOutLines[nOut].IndexOf("
            ",pos0);
loadChgs[nOut]=legOutLines[nOut].Substring(pos0,pos1-pos0);

pos0=pos1+1;
pos1=legOutLines[nOut].IndexOf("
            ",pos0);
consChgs[nOut]=legOutLines[nOut].Substring(pos0,pos1-pos0);

pos0=pos1+1;
pos1=legOutLines[nOut].IndexOf("
            ",pos0);
renewChgs[nOut]=legOutLines[nOut].Substring(pos0,pos1-pos0);

pos0=pos1+1;
pos1=legOutLines[nOut].IndexOf("
            ",pos0);
fuelChgs[nOut]=legOutLines[nOut].Substring(pos0,pos1-pos0);

pos0=pos1+1;
totalChgs[nOut]=legOutLines[nOut].Substring(pos0);
nOut++;
if
            (nOut <
            nBill)
{
legOutLines[nOut]=sr.ReadLine();
}
}
textBoxProcLegOut.Text
            = legOutFile + ": read " + nOut + "
            lines";
textBoxProcLegOut.Refresh();
}
sr.Close();
File.Delete(legOutFile);
}
catch
            (Exception err)
{
MessageBox.Show("attempt to process file " +
            legOutFile + " failed: " + err.Message);
textBoxProcLegOut.Text =
            " error processing " +
            legOutFile;
textBoxProcLegOut.Refresh();
if
            (checkBoxVerbose.Checked)
{
MessageBox.Show(textBoxProcLegOut.Text);
MessageBox.Show("Processing
            Stage 6 failed.");
}
fatalError = true;
reqStatus = "Error:
            " + textBoxProcLegOut.Text;
}
if
            (checkBoxVerbose.Checked)
{
MessageBox.Show(textBoxProcLegOut.Text);
MessageBox.Show("Processing
            Stage 6 is complete.");
}

To parse each line of the LegacyApp data delivery file, the String.Substring() and String.IndexOf() methods are applied to extract the unlabelled space-delimited fields. After each line is parsed, the next line is read, until a number of lines matching the number of requested account records has been processed.

All of the reading and parsing is performed within a try block. If an error occurs, a message is stored in reqStatus and the processing falls through to Processing Stage 7.

Processing Stage 7: Generating response BizTalk document

The Stage 7 code writes the response BizTalk document. The output BizTalk document will contain a CentralPowerResponse business document that either contains the set of data requested in the incoming CentralPowerRequest document or specifies an error message identifying the reason the request could not be fulfilled.

The most important class used to generate the outbound BizTalk document is XmlTextWriter(). This provides fast forward-only generation of XML streams, making it ideal for writing XML files. Employing the XmlTextWriter() class simplifies the process of generating XML files because of the many specialized methods contained in the class. The Stage 7 code illustrates the use of several key methods and properties in the XmlTextWriter() class:

Stage7: //stage 7: generate response BizTalk
            document

radioProcLegOut.Checked=false;
radioGenResp.Checked=true;

int
            fileStart = incomingFile.LastIndexOf("\\");
String respFile =
            ".\\outbound" +
            incomingFile.Substring(fileStart);
textBoxGenResp.Text =
            "generating " + respFile;
textBoxGenResp.Refresh();
if
            (checkBoxVerbose.Checked)
{
MessageBox.Show(textBoxGenResp.Text);
}

try
{
String
            tmpstr;
XmlTextWriter xw = new XmlTextWriter(respFile,
            null);
xw.Formatting = Formatting.Indented;
xw.Indentation =
            2;

xw.WriteStartDocument();

tmpstr="<SOAP-ENV:Envelope
            ";
tmpstr+=
"xmlns:SOAP-ENV=\"http://schemas.xmlsoap.org/soap/envelope/\"
            ";
tmpstr+="xmlns:xsi=\"http://www.w3.org/1999/XMLSchema-instance\">";
xw.WriteRaw(tmpstr);

xw.WriteRaw(soapHeader);

xw.WriteStartElement("SOAP-ENV:Body");

tmpstr="<CentralPowerResponse
            xmlns=\"urn:schemas.biztalk.org:";
tmpstr+="Central_Power.com/1wfknb9j.xml\">";
xw.WriteRaw(tmpstr);

xw.WriteStartElement("requestID");
xw.WriteString(reqID);
xw.WriteEndElement();

xw.WriteStartElement("requestStatus");
xw.WriteString(reqStatus);
xw.WriteEndElement();

xw.WriteStartElement("user");
xw.WriteString(user);
xw.WriteEndElement();

xw.WriteStartElement("monthlyBills");

int
            j;
for (int i=0; i<nBill;
            i++)
{
xw.WriteStartElement("monthlyBill");

xw.WriteStartElement("account");
xw.WriteString(accounts[i]);
xw.WriteEndElement();

xw.WriteStartElement("year");
xw.WriteString(years[i]);
xw.WriteEndElement();

xw.WriteStartElement("month");
xw.WriteString(months[i]);
xw.WriteEndElement();

xw.WriteStartElement("kWh");
xw.WriteString(kWhs[i]);
xw.WriteEndElement();

xw.WriteStartElement("generationServicesCharge");
xw.WriteString(genChgs[i]);
xw.WriteEndElement();

xw.WriteStartElement("customerServiceCharge");
xw.WriteString(custChgs[i]);
xw.WriteEndElement();

xw.WriteStartElement("transmissionCharge");
xw.WriteString(transChgs[i]);
xw.WriteEndElement();

xw.WriteStartElement("distributionCharge");
xw.WriteString(distChgs[i]);
xw.WriteEndElement();

xw.WriteStartElement("systemsBenefitsCharge");
xw.WriteString(systChgs[i]);
xw.WriteEndElement();

xw.WriteStartElement("loadManagementCharge");
xw.WriteString(loadChgs[i]);
xw.WriteEndElement();

xw.WriteStartElement("conservationCharge");
xw.WriteString(consChgs[i]);
xw.WriteEndElement();

xw.WriteStartElement("renewableEnergyInvestmentCharge");
xw.WriteString(renewChgs[i]);
xw.WriteEndElement();

xw.WriteStartElement("fuelAdjustmentCharge");
xw.WriteString(fuelChgs[i]);
xw.WriteEndElement();

xw.WriteStartElement("totalCharges");
xw.WriteString(totalChgs[i]);
xw.WriteEndElement();

xw.WriteEndElement();
}

xw.WriteEndElement();
            //monthlyBills
xw.WriteRaw("</CentralPowerResponse>");
xw.WriteEndElement();
            //SOAP-ENV:Body

xw.WriteRaw("</SOAP-ENV:Envelope>");

xw.Flush();
xw.Close();
textBoxGenResp.Text
            = "completed " + respFile;
textBoxGenResp.Refresh();
}
catch (Exception err)
{
MessageBox.Show("error
            writing " + respFile + ": " +
            err.Message);
textBoxGenResp.Text="error writing " +
            respFile;
if
            (checkBoxVerbose.Checked)
{
MessageBox.Show("Processing Stage
            7 failed.");
}
}

if
            (checkBoxVerbose.Checked)
{
MessageBox.Show(textBoxGenResp.Text);
MessageBox.Show("Processing
            Stage 7 is complete.");
}

btnStart.Enabled =
            true;
btnClear.Enabled = true;

return;

All of the file operations are performed within a try block so that any I/O errors during the write are trapped. After the XmlTextWriter() is opened, the Formatting and Indentation properties are set to specify that the output XML file should be written with indentation of two spaces per depth level. This capability for producing formatted XML output is not easily duplicated if an XML file is written using a generic StreamWriter() class. The XmlTextWriter() class maintains knowledge of the current level of depth in the XML tree structure, and automatically outputs the correct number of spaces as each new element is written.

The first section of XML written is the XML declaration element, which is written by the XmlTextWriter.WriteStartElement() method. This element is followed by the SOAP related elements that begin the BizTalk document. The XmlTextWriter() class is very flexible. There are many ways that the same XML output can be written. Nodes written using the WriteStartElement() and WriteEndElement() methods can be intertwined with nodes written using the WriteRaw() method. However, whenever WriteRaw() is used, it is the responsibility of the programmer to ensure that the resultant XML output is well-formed.

The XmlTextWriter() maintains knowledge of which start elements have been written by WriteStartElement() and will correctly close a sequence of open elements in successive calls to WriteEndElement(). But any start element that was written using WriteRaw() must be closed by the programmer using another WriteRaw() applied at the appropriate position within an element closing sequence. The Stage 7 code illustrates this point. The <SOAP-ENV:Body> and <monthlyBills> start elements are written using WriteStartElement(). The <CentralPowerResponse> element, which is a child of <SOAP-ENV:Body> and a parent of <monthlyBills>, is written using WriteRaw(). Although the closing elements for these three start elements are not written until after the loop that writes the <monthlyBill> element subtree for each returned set of billing data, the XmlTextWriter() maintains the knowledge that two start elements (<SOAP-ENV:Body> and <monthlyBills>) have not yet been closed. It has no knowledge that an element was opened using WriteRaw(). If three consecutive xw.WriteEndElement() statements are placed in the code, an error will be generated at runtime. The point is that all start elements generated using the XmlTextWriter.WriteRaw() method must be closed using WriteRaw() as well. That is, the XmlTextWriter() class demands consistency in the methods selected for generating start and end elements of individual nodes.

Much of the Stage 7 code is writing elements in a simple sequence of WriteStartElement(), WriteString(), and WriteEndElement(). After the closing elements for the SOAP body and SOAP envelope are written, the XmlTextWriter.Flush() method is called to ensure that all characters have been written to disk, and the Close() method is called, closing the file and the XmlTextWriter().

The Start Processing and Clear Results buttons are re-enabled, and the program returns to interactive mode.

Sample Application

Installing the Software

The BizTalk Processor prototype software and LegacyApp utilize several directories for their operation. If the directory structure is not correct, then the applications will fail.

The recommended directory structure for installing and running the programs is as follows. Create an AppDir folder to contain the BizTalkProcessor.exe and LegacyApp.exe executables. Beneath the AppDir folder, create folders named "incoming" and "outbound" (the folders for BizTalk documents) and "LegacyIn" and "LegacyOut" (the read and write folders for LegacyApp).

LegacyApp Setup and Testing

The first exercise in demonstrating the BizTalk interface is to ensure that LegacyApp is correctly installed and is running as expected.

To test that LegacyApp can be run, do the following:

  • open a Command Prompt window
  • change directory to the installation AppDir
  • execute 'dir'
  • execute 'LegacyApp'

The command prompt window should look something like the following:

This screen shows that LegacyApp.exe ran and exited because no parameters were included in the execution command. LegacyApp requires specification of an input directory (to receive the LegacyApp command file) and an output directory (where the program writes its output data delivery file). Optionally, an additional parameter may be specified. If a third parameter is specified, then LegacyApp runs in "verbose" mode, displaying information in the command window as it processes each command file. If a third parameter is not entered on the command line, LegacyApp runs in "quiet" mode.

Now enter the following at the command prompt: LegacyApp LegacyIn LegacyOut 1

This launches LegacyApp in verbose mode, using folders LegacyIn and LegacyOut for its command and data delivery files, respectively. If everything is working fine, LegacyApp will begin searching the LegacyIn folder for input command files, updating the command prompt window if no input files are found.

Now test LegacyApp's capability for processing an input command file by copying the sample LegIn1.txt file into AppDir/LegacyIn. The file should be picked up and processed by LegacyApp, resulting in a command prompt window similar to the following:

In the LegacyOut folder, the file Out_LegIn1.txt should be present. Opening the file with Notepad should produce a file that matches the Out_LegIn1.txt file included with the case study:

To test LegacyApp's error processing, copy the file LegErr1.txt into the LegacyIn folder. This will result in LegacyApp exiting due to an invalid account number error. File Out_LegErr1.txt will be created in the LegacyOut folder, containing a single line:

Problem: cannot find account [55ADD] in request #0

Delete the erroneous LegErr1.txt file from the LegacyIn folder, and relaunch LegacyApp with the command used previously. If you would like to experiment further with LegacyApp by making your own command files, use the following list of user names, passwords, and accounts for the years 1981-2001. The admin user has permission to access all accounts. Other users may access only their own account.

  • admin, adminp, 00000
  • nsync, nsyncp, 11111
  • alanis, alanisp, 22222
  • robThomas, robThomasp, 33333
  • drDre, drDrep, 44444
  • britney, britneyp, 55555
  • backstreet, backstreetp, 66666
  • jewel, jewelp, 77777
  • pink, pinkp, 88888
  • gwen, gwenp, 99999

BizTalk Processor Setup

If the AppDir directory structure outlined above was implemented for installing the software, no additional setup is required for the BizTalk Processor application. Simply execute the BizTalkProcessor.exe file. The default LegacyApp input and output directories are legacyIn and legacyOut. If the suggested AppDir directory structure was not implemented, these entries will have to be changed accordingly. The BizTalkProcessor paths for incoming and outbound BizTalk documents are hard-coded to .\incoming and .\outbound, so these folders must be present beneath the folder where BizTalkProcessor.exe resides.

To test the BizTalk Processor, make sure LegacyApp is running, copy the CPRequestSOAP1.xml file into the .\incoming directory beneath the BizTalkProcessor.exe AppDir, check Verbose (if explanatory pauses are desired over the course of the processing), and click the Start Processing button. You should see the following:

As BTProc steps through each Processing Stage, the results are displayed in the Status/Result text boxes. If Verbose is checked, message boxes are also created to pause the processing at varying points.

After Processing Stage 3 has completed, the BTProc window looks like this:

When OK is clicked, the command file for LegacyApp will be created and written to the LegacyApp input directory (Processing Stage 4). BTProc will immediately jump into Processing Stage 5 to await the arrival of the LegacyApp output file.

Shortly thereafter, the LegacyApp command prompt window should look something like this:

And, in the LegacyOut directory, you should see file Out_cmd.fil with the following contents:

When the BizTalk Processor has completed its processing, it should look like this:

And the file CPRequestSOAP1.xml should be in the AppDir/outbound folder. The <CentralPowerResponse> business document portion of the file is shown below:

To view what happens to a request for account information for which the requester does not have access privileges, clear the AppDir/incoming and AppDir/LegacyOut folders, copy CPRequestSOAPErr1.xml into the folder, press the Clear Results button, and click Start Processing. Part way through Processing Stage 6, the following should appear:

The LegacyApp will have stopped due to the error. After all BTProc processing stages are complete, the outbound\CPRequestSOAPErr1.xml file will have been created with a <requestStatus> of:

Error: legacyOut\Out_cmd.fil: Problem: access to account [55555] is invalid for user [nsync] in request #0

Limitations and Further Work

Clearly, a server that is fully compliant with the BizTalk Framework cannot be built in hundreds or even a few thousand lines of code. The BizTalk Processor software would need to be extended in many ways before it could be put into production. Additional error protection, proper validation and processing of BizTalk header elements, and solid synchronization of document management with respect to incoming documents and document transfers with the LegacyApp would have to be implemented. Nonetheless, the BizTalk Processor demonstrates that the BizTalk Framework can provide legacy applications with a standards-based open interface for enterprise application integration.

Conclusion

The BizTalk Framework is a modern specification that provides flexible and robust capability for linking applications that do not share a common interface, for business-to-business data exchange and other forms of EAI. Microsoft .NET provides the tools necessary to build applications that can implement the basic components of the BizTalk Framework.

Case Study Information
Author | Kevin Farnham
Technical Editor | Adam Ryland
Project Manager | Helen Cuthill
Reviewers | Daniel Cazzulino, David Williams, John Timney, Adam Ryland

If you have any questions or comments about this case study, or suggestions for further pieces, please contact the technical editor. If you wish to talk about the case study with other subscribers, please visit our forum here.


A .NET Data Provider

A .NET Data Provider
http://www.csharptoday.com/content.asp?id=1760
Article Rating: not yet rated
Published on March 25, 2002
Abstract
In this case study, Todd Clemeston covers what a .NET data provider is and examines the reasons why you might consider writing one. Todd also summarizes the guidelines that a custom provider needs to follow in order to “play nicely” in the .NET framework. Once the basics are covered, Todd will walk you through the implementation of a basic data provider that supports querying of the Windows Management Instrumentation (WMI) API. To round out the piece, Todd will demonstrate the use of the custom data provider by covering a simple.NET Windows application that displays the results of WMI queries in a ListView.
Case Study

In this case study, Todd Clemetson covers what a .NET data provider is and examines the reasons why you might consider writing one. Todd also summarizes the guidelines that a custom provider needs to follow in order to "play nicely" in the .NET framework. Once the basics are covered, Todd will walk you through the implementation of a basic data provider that supports querying of the Windows Management Instrumentation (WMI) API. To round out the case study, Todd will demonstrate the use of the custom data provider by covering a simple.NET Windows application that displays the results of WMI queries in a ListView.

Before continuing please take note of the following environmental requirements for the purposes of this discussion, as well as for working with the sample code accompanying the case study:

  • .NET RTM
  • Visual Studio .NET RTM (or a text editor)

What is a .NET Data Provider?

A .NET data provider is a set of specialized classes that implement standard ADO.NET interfaces and serve as a bridge between a data source and .NET. Two data providers are included with the .NET framework: SQL Server and OLE DB. The SQL Server data provider connects to SQL Server Versions 7.0 and 2000, and the OLE DB data provider connects to OLE DB data sources. Additionally, Microsoft has made an ODBC data provider available that you may download. As its name implies, this data provider connects to ODBC based data sources. If you wish to download this provider, there is a link at the end of this case study.

Why would you ever need to write your own data provider within .NET? With the addition of the ODBC data provider mentioned above and the various XML and File IO classes that .NET provides, it is uncommon to find yourself in a situation where writing a custom data provider is the only solution. One major reason you might be forced into writing a custom data provider is if the .NET framework does not supply a data provider for a particular data source. In this case, the underlying data source technology may be old or so new that one hasn't been provided (maybe you are the designer of the data source!) A more interesting reason for writing your own data provider might be that you want all data access to go through a set of standard interfaces (i.e. ADO.NET interfaces). Writing a .NET Data Provider to encapsulate a data source allows your data access code to work with standard interfaces and not be the least bit concerned that the underlying data source is Joe's 3rd party data store and not SQL Server or Oracle. Along the same lines, having a standard set of interfaces also plays nicely for those .NET GUI components that integrate with ADO.NET providers (the DataGrid for example). Finally, creating a .NET Data Provider for a data source also helps you integrate with 3rd party products that use ADO.NET such as Crystal Reports for .NET, etc.

Guidelines for a .NET Data Provider

There really is no comprehensive specification on creating data providers within .NET. The help that ships with the RTM (see "Getting Started with a .NET Data Provider Implementation within VS.NET") defines the ADO.NET interfaces, offers suggestions for overloaded constructors, and generally gets you off to a good start. It also provides source code that implements a basic provider that may help you get the gist of what each method and property is intended to do. I have found one other article on the internet that describes how to write a custom .NET data provider. You may find the link to it at the end of this case study. Additionally, Wrox has published a book entitled "Professional ADO.NET" that does a good job covering .NET data provider construction. My recommendation is to take the "shotgun" approach and read as much material on the subject as possible. Study the help files that ship within .NET, visit the links at the end of this case study, and of course read this case study!

In the next few sections, I will attempt to provide some general guidelines to follow when writing .NET data providers. The intent of these sections is not to regurgitate the help that ships with .NET. Instead these sections highlight some guidelines that are of the most use. When we get down to the details of implementing a custom provider, I will go into specifics that you probably won't glean from the .NET help.

Namespaces

Let's start off with a relatively simple guideline. Actually, this should really be a requirement: data providers should use a unique namespace. In other words, don't put your custom .NET data provider in another data provider's namespace. For reference, the ADO.NET namespaces are listed below:

Namespace | Description
System.Data | Defines common interfaces for ADO.NET
System. Data. Common | Classes that are common across the .NET data providers
System. Data. SqlClient | Classes specific to the SQL Server data provider
System. Data. OleDb | Classes specific to the OLE DB data provider

Additionally, namespaces should follow the CompanyName.TechnologyName namespace documented in the .NET help. The WMI data provider we will write in this case study will use the CsharpToday.DataProviders.WmiData namespace.

Minimal Interfaces

The minimum set of interfaces that a .NET data provider should implement are listed below. We will cover each one of these interfaces in detail when we get into the source code that accompanies this case study.

Interface Name | Description
IDbConnection | A connection to a data source.
IDbCommand | A command or query used against a data source.
IDataReader | Represents forward - only/read-only access to data from the underlying data source.
IDbDataAdapter | Provides the link between a data source and a .NET Dataset.

Interfaces play an important role in flexibility. Writing code against interfaces gives programmers the ability to swap out chunks of code with little or no ill-effects to the source code using those interfaces. The power and flexibility of interfaces is what drives the requirement that .NET data providers implement standard ADO.NET interfaces. This requirement affords developers using your provider the choice of writing code against the standard ADO.NET interfaces listed above, or against the concrete classes that a particular data provider offers (SqlConnection, SqlCommand, etc in the case of the SQL Server Data Provider). In cases where the data access layer within a framework must support a number of current and future data source technologies, it makes sense for the layer to work with the interfaces supplied by ADO. NET. Other projects may not require this much flexibility. The nature of the application may dictate a single database platform, and the programmer may be safe to code against concrete classes. In summary, the requirement that all data providers implement a standard set of interfaces is important in that it leaves the decision of flexibility up to the developer using your provider.

Interface "No-ops"

You are not forced to implement every property and method that an ADO.NET interface specifies. For instance, if the implementation of your data provider will not support the concept of a Transaction, it is acceptable to throw a NotSupportedException. It is up to the author of the data provider to implement methods that make sense and to take appropriate action when methods and properties don't apply. The bottom line: use common sense ! As you will see in subsequent sections, the data provider that we will examine in this case study throws exceptions for areas that don't apply to the WMI API. When we encounter these areas, I will provide you with an explanation of what you would have to do to implement the specific functionality.

Compiling

We will close the guideline section with a relatively simple one. Each Data Provider should exist in its own assembly. Additionally, in order to implement the required interfaces, each assembly should reference the System.Data.dll. If you use VS.NET, simply using the Class library wizard helps you adhere to this guideline as it references this assembly.

Creating a Custom Provider

So far we have covered the basics of .NET data providers. You now know what type of data providers are included in the .NET framework, you understand the two major reasons for writing a custom data provider, and you had a quick overview of the major guidelines to keep in mind when implementing a data provider. With the basics out of the way, we are almost ready to take a look at a sample data provider. Before doing so, let's examine the process of writing a provider from a very high level by breaking the process into the following steps:

  1. Select Interfaces and determine applicability of Functionality - Aside from the standard interfaces we covered earlier, there a few other interfaces you might wish to implement (e.g. IDbDataParameter). Examine your requirements and determine the interfaces you need to implement. Within each interface, highlight methods applicable to your provider and note methods or properties that do not apply.
  2. Determine a namespace - Determine what namespace your data provider will occupy.
  3. Create the Assembly - This is somewhat of a no-brainer. Simply use the VS.NET IDE to create a class library. By default the wizard includes a reference to System.Data.dll.
  4. Implement the interfaces - In this step, you write the necessary code to implement the interfaces you selected in step 2. A few words of wisdom follow. If you are the type of programmer who compiles often and can't stand compiler errors as you code, consider providing "thin" implementations of all the interfaces before you start any hardcore coding. This suppresses compiler errors, for instance, when implementing the IDbCommand class which makes reference to the Connection class. Also, after providing a "thin" implementation of each interface, it generally makes sense to implement the interfaces in the following order: IDbConnection, IDbCommand, IDataReader, IDbDataAdapter, however it really doesn't make a difference which order you implement them in.
  5. Test - Had to throw this one in here. Be sure to test your data provider extensively. I recommend installing NUnit which is a freely available unit-testing framework from SourceForge. I provide a link to NUnit at the end of the case study.

The remainder of this case study focuses on the creation of a custom data provider that wraps a piece of the WMI API for querying and retrieving WMI results. Once we cover implementing the provider, we will look at a sample Windows application that uses the WMI provider to query a server. I kept a few things in mind when choosing a data source technology for this case study's custom data provider: I wanted to make sure I didn't use an example that another author has used, and I wanted to stick with a data source technology that was relatively simple to understand and didn't take away the focus of the case study. The WMI API fits both of these requirements. One of the disadvantages of choosing WMI as a data source is that it may make some aspects of creating an ADO.NET data provider seem simpler than they really are. I will speak more on this issue when we implement the IDataReader interface. One last note: the WMI provider we will create requires a reference to the Microsoft WMI Scripting V1.1 Library. If you don't have this COM object on your machine, you may download it from Microsoft's web site. A link to downloading the WMI SDK is provided at the end of this case study.

What is WMI?

In short, WMI is a management architecture that lets you remotely manage computer systems. Some of the things you can do with WMI that pertain to the WMI provider we will examine include listing all of the processes running on a remote machine or listing all the NTFS partitions on a server that have a specific percentage of free disk space. You may also perform operations such as starting/stopping services or rebooting machines. There is much more to the WMI API than I would care to go into at this point. If you would like more details on WMI, follow the WMI link at the bottom of this case study. The case study describes WMI and its architecture. Alternatively, you can read the case study "WMI for .NET" on C#Today (http://www.csharptoday.com/content.asp?id=1807&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq). Now that you have a basic understanding of WMI, let's examine some fictitious criteria that our WMI provider must meet:

  • Connect - The WMI data provider must be able to connect to a server that supports WMI. We will create a class named WmiConnection that manages this connection. This class will implement the ADO.NET IDbConnection interface.
  • Query - The custom provider must also support querying a server for WMI data. We will create a class named WmiCommand that encapsulates a WMI Query. This class will implement the ADO.NET IDbCommand interface.
  • Moving through Results - We also need a mechanism for calling programs to iterate through the result set that is returned from WMI. We will create a WmiDataReader class to handle this functionality. This class will implement the ADO.NET IDataReader interface.
  • Must support the .NET Dataset - The provider must support populating a dataset with the results of a WMI query. The WmiDataAdapter class will handle this requirement. This class will implement the ADO.NET IDataAdapter interface.
  • Starting/Stopping a Service - For WMI queries against Services running on a WIN32 server (i.e. SELECT * FROM WIN32_SERVICE ) we want users of the provider to be able to change the State of the service and effectively stop or start a service. The WmiDataAdapter class will handle this piece of functionality through the standard Update method exposed within the IDataAdapter interface.

Now that we have defined the functionality for the WMI provider, let's dive into some code. The first interface we will implement is the IDbConnection interface.

WmiConnection

As mentioned earlier, the IDbConnection interface defines a connection to a data source. We will implement this interface through a class called WmiConnection. The WmiConnection class manages the connection to a server that supports WMI. The details of the IDbConnection interface are depicted below. As we implement each method and property, I will provide a description and walk you through the accompanying source code.

In the guidelines section at the beginning of this case study, I mentioned that it is safe to throw exceptions when interface methods are not applicable. We will see this guideline in action within the WmiConnection class because it doesn't have the concept of a timeout, a database, or transactions. As outlined earlier, we will handle these situations by throwing a NotSupportedException. In cases where properties are not applicable, we will perform some "magic" by returning a consistent value and implementing an empty Set method.

Let's take a look at a partial class definition for WmiConnection:

public class WmiConnection : IDbConnection
{
/// <summary>
/// Represents the state of the WmiConnection.
/// Possible values are: Open, Closed, Broken,
/// Connecting, Executing and Fetching.
/// </summary>
private ConnectionState _connectionState;
/// <summary>
/// The connection string used to connect to
/// the server. Although I named this
/// connection string, in our case it is the name of the server
/// that we will be connecting to in order
/// to issue WMI queries.
/// </summary>
private string _connectionString;
/// <summary>
/// Used to connect to a server that can
/// process WMI requests.
/// </summary>
private WbemScripting.SWbemLocator _locator;
/// <summary>
/// WMI object that represents an authenticated
/// connection to a namespace. This object
/// is exposed as an internal property for
/// Reader and Adapter classes to use for
/// the purpose of executing WMI queries.
/// </summary>
private WbemScripting.ISWbemServices _services;
.
.
.

}

Note that the definition of WmiConnection includes implementing the IDbConnection interface. For a description of each member variable used within the Connection class, examine the comments that precede each variable declaration. Let's examine the properties that the WmiConnection class contains.

Properties

This section documents the properties that make up the WmiConnection class. Properties that have IDbConnection in parenthesis indicate that the IDbConnection interface dictates the specific implementation.

State (IDbConnection) - The State property describes the current state of the connection through the ConnectionState enumeration. The different states in this enumeration are listed below:

Value | Description
Broken | The underlying connection is broken and may only occur once the connection is open.
Closed | The underlying connection to the data source is closed.
Connecting | The connection to the data source is in progress.
Executing | The connection to the data source is in the process of executing a command.
Fetching | The connection is in the process of retrieving data from the data source.
Open | The underlying connection to the data source is open.

Note that this property is read-only as it only implements the get function. The underlying _connectionState variable is set internally by the connection object only. We certainly don't want code outside of the connection class modifying the state of the connection.

public ConnectionState State
{
get
{
return _connectionState;
}
}

ConnectionString (IDbConnection) - The ConnectionString property defines how to
            connect to a particular data source. If you have written code to
            access a RDBMS such as SQL server or Oracle, you most likely have
            had to specify a connection string in order to connection to the
            database server. I chose to keep the connection string simple for
            the WMI data provider. It is simply used to specify the name of the
            server to connect to. Note that we could extend this property to
            support a semi-colon separated list of parameters which also
            includes the user name and password. In this case we would have then
            had to parse the connection string when a calling program invoked
            the connection's Open method. Note
            that this property is exposed as a read/write property.

public string
            ConnectionString
{
get
{
return
            _connectionString;
}
set
{
_connectionString =
            value;
}
}

ConnectionTimeout (IDbConnection) - The ConnectionTimeout property represents the
            amount of time that expires before a connection attempt fails and an
            error is generated. Although the ConnectionTimeout property does not apply to
            the WmiConnection connection class, we
            still must provide a body for it. Here we are just returning 0 when
            the value is requested. The set implementation below has an empty
            body and does not maintain any state.

public int
            ConnectionTimeout
{
get
{
return 0;
}
set
{
}
}

Database (IDbConnection) -
            The Database property is the name of the database being referenced
            by the connection. The IDbConnection
            interface defines this property as read-only. This property is
            applicable to relational databases, but is not applicable to the WMI
            provider. The implementation will simply return an empty string.

public string
            Database
{
get
{
return "";
}
}

WmiServices - The WmiServices property is not part of the
            IDbConnection interface, but needs to
            be exposed by the WmiConnection class
            so that other WMI data provider classes (namely the WmiCommand and WmiDataAdapter) can leverage it to query for
            WMI data. The WmiServices property is declared as the ISWbemServices interface. This interface
            exposes a method called ExecQuery that
            takes a WMI query and returns a result set. We will cover this
            method and the result set in more detail later on. Note that this
            property is marked as internal within the scope of the WmiConnection class. The internal access
            modifier makes a member publicly available to all code within an
            assembly, but hides it from classes outside of the assembly. We
            don't want the connection class making the underlying WMI API
            available.

internal ISWbemServices
            WmiServices
{
get
{
return
            _services;
}
}

Methods

The following methods define the behavior of the WmiConnection class. In the case of the
            properties above, methods marked with the IDbConnection interface in parenthesis imply
            that the method is a requirement of the IDbConnection interface.

CreateCommand (IDbConnection) - The CreateCommand method is essentially a
            factory method that creates the appropriate Command object associated with a provider
            connection class. A factory method is a creational design pattern
            that is responsible for creating and returning objects that derive
            from a common abstract type. The WMI provider's CreateCommand method creates and returns
            objects that implement the IDbCommand
            interface. Even more important is the fact that the CreateCommand
            method returns an instance of a WmiCommand object, the concrete command
            object that the WMI provider knows how to work with. We will dive
            into the details of the WmiCommand
            class in the next major section. Here is the CreateCommand implementation for the WMI
            provider:

public IDbCommand
            CreateCommand()
{
return new WmiCommand();
}

Open (IDbConnection) - The
            IDbConnection Open method is responsible for opening up a
            connection to a data source given the ConnectionString property of the Connection object. The WmiConnection. Open () method below connects to a server
            that will respond to WMI requests. I kept the call to the WMI
            locator object method ConnectServer
            simple and only pass in the name of server by passing the entire
            connection string. The implementation for the Open method is shown below:

public void Open()
{
if(this.State ==
            ConnectionState.Open)
{
return;
}
if(ConnectionString !=
            "")
{
_services =
_locator.ConnectServer(ConnectionString,
null, null, null, null, null,
0, null);
this._connectionState =
            ConnectionState.Open;
}
else
{
throw new
            Exception("Invalid Connection String.");
}
}

This method first checks the state of the connection before
            opening it. If the calling program inadvertently tried opening a
            connection that is already open, we don't want to run through the
            process of re-connecting. Depending upon your implementation, you
            may wish to throw an exception, refresh the connection by simply
            closing and opening the connection, or do nothing at all since it is
            already open. Once the Open method
            verifies that the connection string is valid, it connects to the
            server and sets the state of the connection to open. Here is the
            signature to the ConnectServer
            method:

ConnectServer(Server, Namespace, User,
            Password, Locale, Authority, SecurityFlags,
            NamedValueSet)

Argument | Description
Server | Specifies the name of the computer to connect to.
Namespace | Specifies the namespace with which you log on.
User | The user name to use when connecting to the server.
Password | The password to use when connecting to the machine.
Locale | Specifies the locale from which information should be retrieved.
Authority | Specifies the authentication to be used. Should only be used with connections to remote WMI servers.
SecurityFlags | Allows you to pass flag values to ConnectServer. Passing 0 tells the ConnectServer method to return only after a connection to the server has been made.

Close (IDbConnection) - The
            IDbConnection.Close method must
            release resources that are associated with the connection. The WMI
            provider's simple implementation sets the services and locator
            objects to null in addition to setting the state of the connection
            to Closed.

public void Close()
{
_services =
            null;
_locator = null;
this._connectionState =
            ConnectionState.Closed;
}

Constructors -At a minimum, the Connection class should implement a default
            constructor in addition to an overloaded one that takes a string
            that represents the connection string of the connection. Both
            constructors should also be sure to initialize the ConnectionState to Closed. Here is the implementations for
            WmiConnection 's two constructors:

public WmiConnection
            ()
{
_connectionState =
            ConnectionState.Closed;
}

public WmiConnection(string
            strConnectionString)
{
_connectionState =
            ConnectionState.Closed;
this.ConnectionString =
            strConnectionString;
}

Unsupported Methods - The WMI provider will not be
            supporting transactions or the concept of changing a database with
            them. Because of this, it is important to throw NotSupportException's for these methods to
            let programs calling the provider know this is the case. I will,
            however, cover the intent of these methods below.

The BeginTransaction method is
            responsible for creating and returning a data provider specific
            transaction object for starting a database transaction. The
            transaction object that gets returned from the BeginTransaction method must support the
            IDbTransaction interface shown
            below:

If you need to support Transactions in your provider, you would
            also need to provide a provider specific implementation of the IDbTransaction interface. The Connection property maintains a reference to
            the data provider specific connection object. The Isolation level is
            an enumeration that describes how locking behavior will occur within
            the transaction. For the different types of isolation levels, look
            up IsolationLevel in the .NET help files. As you probably guessed,
            the Commit method commits the
            transaction and the Rollback method
            rolls the transaction back. If you need to have support in your
            custom provider, you will also need to provide an implementation for
            the Transaction property of your command object.

The other method that the WmiConnection class will not support is the
            ChangeDatabase method; WMI does not
            have the concept of a database. The purpose of the ChangeDatabase method is to change the
            database for a connection object that already has an open
            connection. Although this method really doesn't apply, we could
            extend the WMI provider and equate the ChangeDatabase method to changing the server
            name that we connect to. Under this scenario, the provider would
            have to actually disconnect from the existing server and
            re-establish its connection to the new server.

WmiCommand

Now that we have a method for connecting to a server, we will
            move onto the next requirement which calls for querying a server
            using the WMI Query language. We will implement the WMI query
            functionality by implementing a class called WmiCommand. This class implements the IDbCommand interface. A Command in ADO.NET
            represents a request to a data source. The SqlClient data provider
            in .NET, for instance, has a SqlCommand class that encapsulates the
            concept of a SQL command. The WmiCommand object will encapsulate a WMI SQL
            command. The IDbCommand interface is
            shown in the picture below:

Here is a partial class definition of the WmiCommand that shows the member variables
            of the class. The comments within the class definition describe what
            each member does.

public class WmiCommand :
            IDbCommand
{
/// <summary>
/// The WMI SQL used to
            obtain data.
/// </summary>
private string
            _wmiCommand;
/// <summary>
/// This represents how the
            results
/// of a command will be applied to a DataRow
///
            when the Update method of the DataAdapter
/// is invoked.
///
            </summary>
private UpdateRowSource _updRowSource;
///
            <summary>
/// Represents a connection to a server
///
            that serves up WMI data.
/// </summary>
private
            WmiConnection _connection;
.
.
.
}

Properties

The following is a list of the properties that the WmiCommand class contains.

CommandText (IDbCommand) -
            The CommandText property represents
            the actual query string that gets executed against the data source.
            In the case of the SQL Server and OLE DB data providers, this
            property represents the SQL to be executed against a particular
            RDBMS. The CommandText property for
            the WMI provider is the actual WMI SQL query that will be executed
            against the server.

public string
            CommandText
{
get
{
return
            _wmiCommand;
}
set
{
_wmiCommand =
            value;
}
}

CommandTimeout (IDbCommand)
            - Represents the amount of time to wait before stopping the
            execution of a command. We are not going to support the concept of a
            timeout, so we just simply return 0 to the caller and provide an
            empty set function.

public int
            CommandTimeout
{
get
{
return
            0;
}
set
{
}
}

CommandType (IDbCommand) -
            The CommandType property represents
            the type of command to be executed through the CommandType enumeration. The CommandType enumeration defines the
            following possible types: StoredProcedure, TableDirect, and Text. The data providers in the .NET
            framework support all three of these enumerations. Setting this
            property to StoredProcedure specifies
            that the CommandText property is the
            name of a stored procedure. Setting this property to TableDirect specifies that the CommandText property is the name of a table
            in a database. Setting the property to Text specifies that the CommandText is a query (e.g. SQL or WQL in
            the case of the WMI provider). Because we are only supporting the
            Text option, the implementation below always returns text and only
            supports the setting of text.

public CommandType
            CommandType
{
get
{
return
            CommandType.Text;
}
set
{
if(value !=
            CommandType.Text)
{
throw new NotSupportedException("This
            provider only supports text commands.");
}
}
}

Connection (IDbCommand) -
            The Connection property is simply the
            connection that the Command object is
            associated with. Its implementation is below:

public IDbConnection
            Connection
{
get
{
return
            _connection;
}
set
{
_connection =
            (WmiConnection)value;
}
}

Parameters - The Parameters represents a collection of
            IDataParameter objects used to specify
            the parameters of a SQL statement or stored procedure. The WMI
            custom provider will not support parameters, so it just simply
            returns null in its implementation. If you need to implement
            parameters, you must create a ParameterCollection class that implements
            the IDataParameterCollection. You will
            also need to create your own DataParameter class implementing the IDataParameter interface.

public IDataParameterCollection
            Parameters
{
get
{
return null;
}
}

Transaction (IDbCommand) -
            The Transaction property represents the transaction which the
            command will execute under. The WmiCommand does not support transactions, so
            we simply return null when asked for a transaction and throw an
            exception when the caller attempts to specify a transaction.

public IDbTransaction
            Transaction
{
get
{
return
            null;
}
set
{
throw
new NotSupportedException("The
            WmiCommand object doesn't " +
"support
            transactions.");
}
}

UpdateRowSource (IDbCommand)
            - Lets a calling program specify the method with which values
            returned from a data source are applied to a row within an ADO.NET
            Dataset. This property comes into play within the Update method of the DataAdapter. The WMI
            provider doesn't support return (or output) values in its Adapter's
            Update method, so the enumeration is
            set to None in
            the constructor and may not be changed. If a calling program
            attempts to change this to any other value, the provider will throw
            an exception.

For the curious, the following table describes the different
            values of the UpdateRowSource
            enumeration, and how they affect the behavior of a command used with
            a DataAdapter:

Name | Description
Both | The output parameters and the first row returned get mapped to the changed row in the Dataset.
FirstReturnedRecord | Data in the first returned row is mapped to the changed row in the DataSet.
None | Returned parameters or rows are ignored.
Output Parameters | Output parameters are mapped to the changed row in the DataSet.

public UpdateRowSource
            UpdatedRowSource
{
get
{
return
            _updRowSource;
}
set
{
if(value !=
            UpdateRowSource.None)
{
throw
new
            NotSupportedException("The WmiCommand " +
"object only supports
            None as an " +
"UpdateRowSource
            type.");
}
}
}

WmiCommand Methods

There are really only two methods of importance for the WmiCommand class. The rest of the methods
            don't apply to the WMI provider, although I will go over them so you
            understand what each means.

ExecuteReader (IDbCommand) -
            ExecuteReader is a factory method that
            returns a DataReader associated with
            the Command class. Remember than an
            IDataReader provides forward-only
            access to data from a data source. The ExecuteReader implementation is shown
            below:

public IDataReader
            ExecuteReader()
{
if(this.Connection == null)
{
throw
            new Exception("Cannot execute reader. The WmiConnection "
            +
"object is not valid.");
}
if(this.Connection.State !=
            ConnectionState.Open)
{
throw new Exception("Cannot execute
            reader. The connection " +
"to the server is
            invalid.");
}
// create a reader
WmiDataReader reader =

new WmiDataReader(this._connection);
// the object set that
            is returned as a result of the
// query.
ISWbemObjectSet
            ObjSet = null;
// execute the query as WMI Query Language

ObjSet =
            _connection.WmiServices.ExecQuery(_queryCommand,
"WQL", 0x10,
            null);
// set the readers data to the object set
reader.Data =
            ObjSet;
// return the reader to the caller
return
            reader;

}

The above code is not overly complex. After verifying that the
            connection is valid, the method creates a WmiDataReader object, executes the WMI
            Query, and sets the WmiDataReader.Data
            property to the result set of the WMI query. We will cover
            the details of the DataReader
            implementation in the next major section, but essentially the DataReader will use the Data property to iterate through the result
            set.

The WMI ExecQuery method
            highlighted in the code above takes the following four
arguments:

  1. WMIQuery - The first specifies the WMI query to execute. The
              syntax of this query follows WQL (WMI SQL), which is a subset of
              the ANSI standard SQL language with WMI-specific extensions.

  2. QueryLanguage - The second argument is the query language to
              use, and must always be "WQL" if anything is specified.

  3. QueryBehavior - The third argument specifies the behavior of
              the query and determines if this call returns immediately. We have
              specified WbemFlagReturnImmediately
              (0x10) for this option.

  4. NamedValueSet - The final argument is NamedValueSet. We have left this as null,
              but you can pass a SwbemNamedValueSet object to give
              contextual information that can be used by a provider servicing
              the request. If a provider supports this argument, its
              documentation will have specified the values, data types, allowed
              values, and semantics.

I mentioned earlier that using WMI as the data source made some
            aspects of authoring a data provider seem simpler than they normally
            are. This is indeed the case for the ExecuteReader() method. When implementing
            the ExecuteReader method, it is common
            to have to write somewhat complex parsing logic to break a query
            apart. In the implementation above, we simply forward the query to
            the WMI COM object.

ExecuteScalar (IDbCommand) -
            The IDbCommand. ExecuteScalar method is meant to be an
            efficient means to access data that is in the form of a single value
            within a single row. A good example of a situation where this method
            is helpful is in the case where a record count from a SQL select
            statement is required. The result of a SELECT
            COUNT(*) FROM Table1 SQL query returns a single column and
            single value. The ExecuteScalar method
            is meant to provide quick access to these types of queries. WMI does
            not support a count type query, but for purposes of this case study,
            I have provided an implementation that invokes the ExecuteReader method above, and returns the
            first column of the first row. Admittedly, this is inefficient, but
            it illustrates the intent of the method.

public object
            ExecuteScalar()
{
if(this.Connection == null)
{
throw
            new Exception("Cannot execute reader. The WmiConnection "
            +
"object is not valid.");
}
if(this.Connection.State !=
            ConnectionState.Open)
{
throw new Exception("Cannot execute
            reader. The connection " +
"to the server is
            invalid.");
}
// execute the reader
IDataReader dr =
            this.ExecuteReader();
dr.Read();
// return the first column
            for the first
// row in the resultset.
return
            dr[0];
}

The following methods throws NotSupportExceptions because WMI doesn't
            support the particular functionality:

  • Cancel - You would provide an implementation for this method
              if you wanted to give a calling program the ability to cancel the
              execution of a command.

  • CreateParameter - If your data provider supports Data
              parameters, you would override this method to create a data
              provider specific instance of an IDataParameter object.

  • ExecuteNonQuery - You would provide an implementation of this
              method if your data provider supported adding, updating, and/or
              deleting of data from your data source. The SQL and OLE DB data
              providers, for instance, use this method to execute INSERT and
              UPDATE SQL statements.

  • Prepare - This method only applies when the CommandType property is set to StoredProcedure. If your data provider
              supports stored procedures, you would write the necessary code to
              compile the command for your data source.

WmiDataReader

The purpose of an ADO.NET DataReader is to provide efficient
            forward-only/read-only access to the results of a command. The DataReader is also responsible for mapping
            the data types of the source data to the Common Language Runtime
            (CLR) data types. The WMI provider's implementation of the
            DataReader is accomplished via the WmiDataReader class. The WmiDataReader class provides access to the
            result set of a WMI query command. Recall that the WmiCommand. ExecuteReader() method executes a WMI Query,
            creates a WmiDataReader, and sets the
            internal Data property of the WmiDataReader to the result set of the
            query. The WmiDataReader implements
            interface methods allowing a calling program to iterate through the
            WMI result set. Note that the WmiDataReader class implements the IDataReader interface. This interface in
            turn implements the IDataRecord
            interface. The IDataReader interface
            is shown below. It consists of a few properties and methods.

The IDataRecord interface (which
            the IDataReader interface implements)
            is comprised of a very large number of "Getter" functions, aimed at
            retrieving individual column values from the current record in the
            result set. A partial interface definition is shown below to give
            you an idea of what this interface looks like:

Here is a partial class definition for the WmiDataReader class, with descriptions for
            each member variable:

public class WmiDataReader:
            IDataReader
{
/// <summary>
/// The
            connection associated with the reader
///
            </summary>
private WmiConnection _connection;
///
            <summary>
/// Holds the number of rows retrieved from the
            last query.
/// </summary>
private int _numRows;
///
            <summary>
/// The resultset of the WMI query.
///
            </summary>
ISWbemObjectSet _data = null;
///
            <summary>
/// The enumerator of the object set.
///
            </summary>
IEnumerator _wmiEnumerator;
///
            <summary>
/// used to map the names of the columns to
            indexes. WMI supports
/// looking up by names, but we need an
            efficient way of looking
/// up column values by index. This
            array list holds this lookup.
/// </summary>
ArrayList
            _indexLookup = new ArrayList();
/// <summary>
///
            reference to the current record in the WMI resultset.
///
            </summary>
ISWbemObject _currentObject = null;
///
            <summary>
/// reference to the property set (i.e collection
            of columns)
/// </summary>
ISWbemPropertySet
            _currentPropertySet = null;
.
.
.
}

Properties

Let's first take a look at the properties of the WmiDataReader class. They are listed
            below.

Data - You got a sneak preview of this property when we explored
            the WmiCommand.ExecuteReader() method.
            This property is marked internal so that the WmiCommand object can set its value after
            executing the WMI query. Recall that the Data property is of the
            type ISWbemObjectSet. You can think of
            the ISWbemObjectSet interface as a
            result set. This object exposes an enumerator that WmiDataReader will utilize to move forward
            through the result set. The implementation of the Data property in the next code block sets
            the private _ data member variable to
            the value passed in. It then sets the _ wmiEnumerator member variable to the ISWbemObjectSet's enumerator followed by
            setting the _ numRows member variable
            to the number of rows in the result set. This private variable will
            be accessed via the RecordsAffected
            property of the DataReader.

internal ISWbemObjectSet
            Data
{
set
{
_data = value;
// we need to keep track
            of the enumerator
// for the object set.
this._wmiEnumerator =
            _data.GetEnumerator();
// set the number of rows to the
            count.
this._numRows = _data.Count;
// build the index lookup
            table.
this.BuildIndexLookup();
}
}

Accessing actual column values within ISWbemObjectSet is accomplished by
            specifying a column name. The IDataReader
            interface, however, must provide access to these values by
            name as well as and by an integer-based index. To support looking up
            values through an index, the WMI provider creates an ArrayList named _ indexLookup that contains the ordered names
            of the columns in the WMI result set. The BuildIndexLookup method that gets invoked
            below builds this list. Methods within the WmiDataReader class will leverage this ArrayList by passing in an index to get a
            column name, and using the column name to get at the data. Here is
            the code for the BuildIndexLookup()
            method:

private void
            BuildIndexLookup()
{
ISWbemObject WbemObj =
            null;
ISWbemPropertySet WbemPropSet = null;

WbemObj =
            this.GetFirstObjectInResultSet();
WbemPropSet =
            WbemObj.Properties_;
// grab the enumerator for the
            properties
IEnumerator e = WbemPropSet.GetEnumerator();
//
            Advance the enumerator's cursor until we reach
// the last
            item
while (e.MoveNext())
{
// grab the current
            property
ISWbemProperty ob = (ISWbemProperty)e.Current;
// add
            the property name to the
            index
_indexLookup.Add(ob.Name);
}

}

The code above takes the first result, grabs the property
            collection, and adds an entry to an array that simply stores the
            name of the property.

RecordsAffected (IDataReader) - Returns the number of rows
            affected by the Command. The number of
            rows may be the result of a SELECT, UPDATE, DELETE, or INSERT query.
            With regards to the WMI provider, this property will be the number
            of records in the result set as represented by the _numRows member variable.

public int
            RecordsAffected
{
get
{
return
            _numRows;
}
set
{
}
}

FieldCount (IDataReader) -
            Returns the number of fields that are in the result set. The WMI
            provider implementation of this property simply forwards the call to
            the WMI PropertySet object's count, as
            shown below:

public int
            FieldCount
{
get
{
return
            _currentPropertySet.Count;
}
}

Depth (IDataReader) - This
            is the nesting depth for the current row. This property applies to
            data providers where queries return multidimensional or hierarchical
            results. The WmiDataReader does not
            support this concept, so the implementation of the Depth property returns 0, and contains no
            logic in the "setter" function.

public int Depth
{
get
{
return
            0;
}
set
{
}
}

IsClosed (IDataReader) -
            Returns a Boolean specifying if the DataReader is closed. The WMI provider in
            this example does not support the concept of being closed so it
            always returns false.

public bool
            IsClosed
{
get
{
return
            false;
}
set
{
}
}

Methods

Constructors - The WmiDataReader contains two constructors.
            Both initialize the _ numRows member
            variable to 0. The overloaded constructor takes a WmiConnection as a parameter and sets the
            internal connection property. Note that both constructors are marked
            internal. Users must not be able to construct readers. Only the
            Command object must create and return
            a reader. Here are the constructor definitions:

internal WmiDataReader()
{
_numRows =
            0;
}

internal WmiDataReader(WmiConnection
            connection)
{
_connection = connection;
_numRows =
            0;
}

Read (IDataReader) - The
            Read method is responsible for moving
            the DataReader to the next record in
            the result set. It returns false once it reaches the end of the
            result set, and true otherwise. The implementation below advances
            the enumerator of the result set. If it's not at the end of the
            results, it sets the _ currentObject
            and _ currentPropertySet variables. If
            it has reached the end, it sets the objects to null.

Here is the implementation:

public bool Read()
{
// move to the
            next result
bool bRet = _wmiEnumerator.MoveNext();
if(bRet ==
            true)
{
// make sure the current object points
// to the
            current item on the enumerator
_currentObject = (ISWbemObject)
            _wmiEnumerator.Current;
// also set the propertyset member
            variable
_currentPropertySet = _currentObject.Properties_;
//
            return the result of the movenext
            operation
}
else
{
_currentObject =
            null;
_currentPropertySet = null;
}
return
            bRet;
}

GetValue (IDataReader) - The
            GetValue method is part of the IDataRecord interface definition. It takes
            an integer as a parameter, and returns the value of a column for the
            given record as a .NET object. The WMI provider's implementation
            verifies that the result set contains data, verifies that the index
            is valid, looks up the column name based on the index, and invokes a
            custom method called GetCurrentValueByName().

public Object GetValue(int
            i)
{
if(_numRows == 0)
{
throw new
            IndexOutOfRangeException("Must first call read");
}
if(i <
            0 || i > this._currentPropertySet.Count)
{
throw new
            IndexOutOfRangeException("No such column");
}
string
            strFieldName =
(string)_indexLookup[i];

return
            this.GetCurrentValueByName(strFieldName);
}

The GetCurrentValueByName method
            simply returns the column value given the name passed to it. Its
            definition is shown below:

private string
            GetCurrentValueByName(string strName)
{
return
            _currentPropertySet.Item(strName,
            0).get_Value().ToString();
}

GetXXX Functions (IDataRecord) -Users of a DataReader may also obtain strongly typed
            values from the result set by invoking one of the many GetXXX methods that make up the IDataRecord interface. Recall that the WmiDataReader class implements the IDataReader interface, which also implements
            the IDataRecord interface. Listing the
            implementations of all of the IDataRecord methods would be a bit mundane,
            but I have provided two implementations of the GetXXX methods below. In general the GetXXX methods invoke the GetValue method we implemented earlier and
            cast the return value to the appropriate type.

public string GetString(int
            i)
{
return this.GetValue(i).ToString();
}

public
            bool GetBoolean(int i)
{
return
            (bool)this.GetValue(i);
}

this[parameter] (IDataReader) -The IDataReader interface also defines two
            separate indexers which allows the implementing class to be treated
            as if it were a collection. There are two implementations of the
            indexer. The first provides access to column values by name; the
            second by an integer index. Implementations for the WMI provider of
            these two indexers follow:

public object this[string
            name]
{
get
{
return
            this.GetCurrentValueByName(name);
}
}

Here is the integer based implementation of the indexer:

public object this[int
            i]
{
get
{
return
this.GetValue(i);
}
}

GetFieldType (IDataReader) -
            The GetFieldType method returns the
            CLR type of the data item at the specified index within the current
            record. WMI uses Common Information Model (CIM) data types to
            represent its data. For more information on CIM, navigate to the CIM
            link found at the end of this case study. It is up to the WmiDataReader to translate CIM to CLR data
            types. After grabbing the property at the specific index, the method
            invokes the ConvertCIMToCLR method
            whose implementation follows:

public Type GetFieldType(int
            i)
{
ISWbemProperty p =
            _currentPropertySet.Item(_indexLookup[i].ToString(),0);

return
            ConvertCIMToCLR(p.CIMType);
}

To simplify converting types, the WmiDataReader simply converts all data types
            to the CLR string data type as shown below. If we wanted to
            elaborate on this further, we could create a switch statement that
            provided a mapping for each CIM type.

private Type ConvertCIMToCLR(WbemCimtypeEnum
            en)
{
return typeof(string);
}

GetName (IDataReader) - The
            GetName method returns the name of the
            field within the result set given the index passed in. Its
            implementation is shown below. Note that it must use the _ indexLookup ArrayList because the WMI PropertySet object looks up column names by
            name. Remember that all the _ indexLookup array does is allow for
            efficient lookup of column names by index.

public string GetName(int i)
{
// grab
            the object given the index passed in
ISWbemProperty ob =

_currentPropertySet.Item(_indexLookup[i].ToString(),0);
//
            return the property name
return ob.Name;
}

GetValues (IDataReader) -
            The GetValues method populates an
            array of objects passed into the method with the data from the
            current record, where each column value represents an object. Note
            that the caller of this method must allocate enough space in the
            object array to satisfy the number of columns. The WMI provider's
            implementation for this method is shown below:

public int GetValues(object[]
            values)
{
// make sure we have a valid
// number of
            properties
if(_currentPropertySet.Count >
            0)
{
if(values.Length < this.FieldCount)
{
throw new
            Exception("You must allocate the " +
"object array with enough
            room to " +
"hold the number of fields in the " +
"row
            set.");
}

int iIndex = 0;
// iterate through the list
            of properties
foreach(ISWbemProperty p in
            _currentPropertySet)
{
// set the value objects to
// the
            value
values[iIndex] = p.get_Value();
}
// return the
            number of values
return values.Length;
}
else
{
// if
            we go here, there weren't any
// properties
values =
            null;
return 0;
}
}

GetOrdinal (IDataReader) -
            The GetOrdinal method takes a name of
            a column and returns the index at which the column name is located.
            The WmiDataReader implementation uses
            the IndexOf method of the string class to provide this functionality
            as shown below:

public int GetOrdinal(string
            name)
{
return _indexLookup.IndexOf(name);
}

Close (IDataReader) - The
            WMI implementation does not have a concept of closing a DataReader. The SQL Server and OLE DB
            providers within .NET only allow one DataReader to be open for each connection,
            and the DataReader must be explicitly
            closed after the programmer is done.

GetSchemaTable (IDataReader)
            - GetSchemaTable returns a DataTable populated with metadata about the
            columns of the reader. This method is useful in that users may
            generically execute and display the results of a query by populating
            a grid full of columns not known at design time. We will create a
            Windows application that utilizes this method to populate the
            headings of a ListView control with column names. The GetSchemaTable implementation below creates
            a table called WMIMetaData, defines a
            ColumnName and DataType column within the table, and
            creates a row for each column in the result set.

public DataTable GetSchemaTable()
{
//
            create a datatable
DataTable dt = new
            DataTable("WMIMetaData");
// add the schema
            columns
dt.Columns.Add("ColumnName",
            typeof(string));
dt.Columns.Add("DataType", typeof(Type));
//
            iterate through the list of properties
foreach(ISWbemProperty p
            in _currentPropertySet)
{
// add a new row
DataRow dr =
            dt.NewRow();
// set the name of the colum and
// the data type
            properties
dr["ColumnName"] = p.Name;
dr["DataType"] =
            this.ConvertCIMToCLR(p.CIMType);
// add the row to the
            table
dt.Rows.Add(dr);
}
return dt;
}

GetDataTypeName (IDataReader) - This method returns the name
            of the data type for the specified field index. The implementation
            below looks up the property object based upon the index, converts
            the data type to a CLR type, and returns the string representation
            of the type.

GetDataTypeName(int i)
{
// grab the
            object given the index passed in
ISWbemProperty ob =
_currentPropertySet.Item(_indexLookup[i].ToString(),0);
//
            determine the CLR type.
Type t =
            ConvertCIMToCLR(ob.CIMType);
// return the name of the
            type
return t.ToString();
}

NextResult (IDataReader) -
            NextResult is applicable to those data
            providers that support multiple result sets. The SQL Client data
            provider supports multiple result sets when SQL statements are
            separated by a semi-colon. Invoking this method tells the DataReader to point to the next result set.
            The WMI provider will not support this concept, so this method
            throws a NotSupportedException.

GetData (IDataReader) - The
            GetData method is used to expose
            nested tables and other hierarchical data. Both data providers
            within .NET do not support this method. We are not implementing this
            method either.

WmiDataAdapter

A .NET DataAdapter provides the
            link between a data source and a .NET Dataset. I assume you have a
            basic understanding of what a .NET Dataset is. If you need further
            details, please see the following article on CSharpToday: http://www.csharptoday.com/content.asp?id=1568&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq.
            The SQL Server and OLE DB implementations of the DataAdapter within the .NET framework
            support populating a Dataset with data
            from a data source in addition to updating a data source given
            changes to the Dataset. Updating of
            the data source is accomplished via the DataAdapter's Update method. This is a relatively easy
            concept to grasp in regards to a database. A typical sequence of
            events might be:

  • Populate a dataset using an adapter

  • Make modifications to a data set by modifying, adding, or
              deleting rows, and

  • invoke the DataAdaper's Update
              method to synchronize the changes back to the data source.

However, WMI's WQL only supports read-only access to the WMI
            data, so we are a limited in what we can do in this regard. However,
            the WMI provider will support the concept of Updating WMI data by
            adding logic to the Adapter's update
            method that inspects the State of a service (e.g., Stopped or
            Started) and stops or starts the service.

The WmiDataAdapter class implements
            the IDbDataAdapter interface. This
            interface derives from the IDataAdapter interface to create the
            following interface definition:

Properties

SelectCommand (IDataAdapter)
            - This is the command that the DataAdapter uses to populate the Dataset. The WMI provider's implementation
            represents this property as an instance of a WmiCommand object:

public WmiCommand
            Command
{
get
{
return
            _command;
}
set
{
_command =
value;
}
}

UpdateCommand, DeleteCommand and InsertCommand (IDataAdapter) - The WMI provider only
            supports the concept of the SelectCommand so we have empty
            implementations of these properties to satisfy the interface. This
            interface is typically geared toward a RDBMS type data provider. I
            could have chosen to just implement the IDataAdapter interface which would remove
            these four commands, but I wanted to illustrate the various commands
            associated with an adapter.

TableMappings (IDataAdapter)
            - This property specifies how a table in the data source maps to a
            DataTable within a DataSet. This allows a program to
            essentially hide the underlying physical table names and exposes
            names within a DataSet that are more "user friendly" to a user or
            calling program. It allows for a physical table named COMP_SALES to
            be represented in a DataSet as Company Sales. Additionally, each
            TableMapping instance contains a
            reference to a collection of DataColumnMapping objects that offer a
            similar service. With a DataColumnMapping, a physical column named
            COMPANY_ID could be represented in a DataSet as Company ID. If you
            need to support TableMappings in a
            provider, you must examine them when you populate a DataSet and when
            callers issue the Update method. We
            will cover these methods later.

MissingMappingAction (IDataAdapter) - This property is an
            enumeration that defines the action taken when a specified table or
            column mapping (e.g. DataTableMapping,
            DataColumnMapping) is missing from the source table or source
            column. Recall our discussion in the previous section and assume
            that a calling program attempts to populate a DataSet. In setting up
            the DataAdapter, the calling program has specified mappings one of
            which maps the COMP_SALES physical table to the Company Sales table
            in a DataSet. If the COMP_SALES table doesn't exist in the database,
            the data provider needs to know what action to take. The MissingMappingAction property specifies this
            action through the MissingMappingAction enumerator which
            provides for the following actions:

  • Error - Throwing an exception

  • Ignore - Ignoring the source tables and columns that failed to
              map. This results in some source data not making it in the Dataset

  • Passthrough - Creating the DataTable and DataColumn that failed to map.

Since the WMI provider does not support mappings, the MissingMappingAction always returns Passthrough (which creates the tables and
            columns in the dataset). The set operation throws an exception for
            any other type but Passthrough.

public MissingMappingAction
            MissingMappingAction
{
get
{
return
            MissingMappingAction.Passthrough;
}
set
{
}
}

MissingSchemaAction (IDataAdapter)
            - While the MissingMappingAction
            specifies the action to be taken when an expected table or column is
            missing from a physical data source, the MissingSchemaAction property specifies the
            action to be taken when a DataTable or
            DataColumn is missing from the schema
            within a Dataset. Possible actions
            include:

  • Add - Adding the necessary columns to the schema

  • AddWithKey - Adding the columns with primary key information

  • Error - Throwing an exception

  • Ignore Ignoring the extra columns

Our implementation only supports the Add member of the enumeration.

public MissingSchemaAction
            MissingSchemaAction
{
get
{
return
            MissingSchemaAction.Add;
}
set
{
}
}

Methods

Constructor - Aside from the default constructor, there
            are three additional constructors that make up the adapter. Their
            signatures and descriptions are below:

// sets the Command property of the
            adapter
public WmiDataAdapter(WmiCommand comm)
// sets the
            Command property of the adapter and also sets the WmiCommand's //
            Connection property.
public WmiDataAdapter(WmiCommand comm,
            WmiConnection conn)
// creates a new WmiCommand and WmiConnection
            objects.
public WmiDataAdapter(string strCommand, string
            strConnection)

Fill (IDbDataAdapter) - This
            method Inserts or Updates rows in the Dataset to match those within the data
            source. The WmiDataAdapter.Fill method
            is shown in the next code block. Its first job is to make sure the
            command object is valid. Note that this method also keeps track of
            the ConnectionState. If the connection
            is closed, it must open the connection and set a flag (bMustCloseConnection) so that the connection
            will be left in its initial state once the Fill operation is complete. It then proceeds
            to fill the schema in the Dataset and
            grabs a reference to the table based upon the table name. It then
            creates a WmiDataReader and iterates
            through it adding a new row to the table for each row in the reader.
            Note the call to AcceptChanges at the
            end of the Fill method. The call to
            AcceptChanges is vital in order to
            keep the state of each row within the DataSet in a consistent state.
            Internally, the DataSet keeps state information for columns within
            each row. It does this through the RowState property associated with each Row.
            The RowState property returns the
            DataRowState which specifies if a row has been Added, Deleted, Detached, Modified, or
            Unchanged. As the fill method adds records to the DataSet,
            each row's RowState is set to the
            Added state. The AcceptChanges method highlighted in the code
            below accepts these inserted rows and changes all of the row's
            RowState to Unchanged. Why is this important? If we removed
            this line and a program was to make modifications to rows inside the
            DataSet, the state will remain marked as Added. Now assume a program
            wants to invoke the Adapter's Update
            method to update the data source with their modifications. The Update method would query the DataSet to
            find out the state of its Data and erroneously determine that a
            bunch of rows have been inserted (because the RowState is set to Added). If the WMI provider supported adding
            rows in this scenario, it would end up duplicating all of the rows
            that the Fill method added. The AcceptChanges puts all rows in a "pristine"
            state by setting the RowState to Unchanged. The importance of this method
            will become clearer when we cover the Update method. Here is the source code to
            the Fill method:

public int Fill(DataSet ds)
{
// make
            sure the command is good
if(_command == null ||
            _command.CommandText.Trim() == "")
{
throw new
            Exception("Cannot fill without receive command");
}
// if the
            connection is closed, we must open it.
// We also need to make
            sure we close it before
// leaving this method.
bool
            bMustCloseConnection = false;
if(_command.Connection.State !=
            ConnectionState.Open)
{
_command.Connection.Open();
bMustCloseConnection
            = true;
}
// fill the schema for this
            query
this.FillSchema(ds, SchemaType.Mapped);
// create the
            table
DataTable dt = ds.Tables[this.CommandTableName];
//
            create a data reader
WmiDataReader dr =
(WmiDataReader)_command.ExecuteReader();

// iterate
            through the reader
while(dr.Read())
{
// create a new
            row
DataRow newRow = dt.NewRow();
// set all of the
            fields
for(int i = 0; i < dr.FieldCount;
            i++)
{
newRow[i] = dr[i];
}
// add the row to the
            table
dt.Rows.Add(newRow);
}
// accept all of the
            changes
ds.AcceptChanges();
// if we had to open up the
            connection to
// perform this operation then we must
// close
            it.
if(bMustCloseConnection)
{
_command.Connection.Close();
}
//
            return the number of rows affected
return
            dr.RecordsAffected;
}

FillSchema (IDbDataAdapter)
            -This method is responsible for creating a table in the DataSet and configuring the schema to match
            the source table based on the SchemaType passed in. The SchemaType may specify Mapped or Source. Mapped directs the adapter to use
            table mappings when creating the schema. WmiDataAdapter doesn't inspect the SchemaType, and implements the SchemaType.Source method that uses the
            source data to specifying the schema. The result is that the table
            and columns names inside the Dataset
            will map directly to the table and column names in the data source.

In the implementation below, the FillSchema method creates a table that maps
            to the source table (removing it if it already exists). It then
            iterates through the set of properties adding a column for each
            column in the result set.

public DataTable[] FillSchema(DataSet ds,
            SchemaType schemaType)
{
// need the connection for this
            adapter
WmiConnection conn =
            (WmiConnection)_command.Connection;
// we need to object
            set
ISWbemObjectSet ObjSet = null;
ISWbemObject WbemObj =
            null;
ISWbemPropertySet WbemPropSet = null;
// unfortunately
            we need to execute the query
// to get the metadata information
            necessary
// to fillin the schema
ObjSet = conn
            .WmiServices.ExecQuery(_command.CommandText,
"WQL", 0x10,
            null);
// create 1 table (for this version we will
// only
            support single select statements
DataTable []dt = new
            DataTable[1];
// new and create the table with the table
            name
string strTableName = this.CommandTableName;
// if the
            dataset contains a definition for this
// table name, remove
            it.
if(ds.Tables.Contains(strTableName))
{
ds.Tables.Remove(strTableName);
}

dt[0]
            = new DataTable(strTableName);
// grab the enumerator for the
            object set
IEnumerator obEnum = ObjSet.GetEnumerator();
//
            grab the current object
obEnum.MoveNext();
WbemObj =
            (ISWbemObject)obEnum.Current;
// grab the
            properties
WbemPropSet = WbemObj.Properties_;
// grab the
            property enumerator
IEnumerator ePropSet =
            WbemPropSet.GetEnumerator();
// Advance the enumerator's cursor
            until we reach
// the last item
while (ePropSet.MoveNext())
{
DataColumn dc = new DataColumn();
// Cast the Current
            item to the proper type
ISWbemProperty ob =
            (ISWbemProperty)ePropSet.Current;
// set the column
            name
dc.ColumnName = ob.Name;
dc.DataType =
            ConvertCIMToCLR(ob.CIMType);
// add this column to the
            table
dt[0].Columns.Add(dc);
}

// add this table to the
            dataset
ds.Tables.Add(dt[0]);
// return the table
return
            dt;

}

Update (IDbDataAdapter) -The
            Update method is responsible for
            updating a physical data source with changes made to the DataSet
            that was populated by the adapter. The WMI implementation of this
            method will meet the final requirement of the data provider by
            supporting stopping and starting of Window's services. The sequence
            of events might unfold as follows:

  1. A calling program uses the WMI adapter to fill the Dataset
              with the following WMI query: SELECT * FROM WIN32_SERVICE

  2. The adapter obliges this request and creates a DataTable
              called WIN32_SERVICE within the DataSet and populates it with a
              list of Windows services

  3. The calling program uses the DataSet select command and
              obtains the row containing the World Wide Web Publishing Service
              (IIS) and sets the State field from "Running" to "Stopped."

  4. The calling program invokes the Adapter's Update method.

  5. Finally the Adapter examines the changed State column for the
              World Wide Web Publishing service and uses the .NET
              ServiceController class to stop the service.

Now that you have an understanding of what the WMI provider's
            update method will do, let's look at its implementation:

public int Update(DataSet
            dataSet)
{
// as a special case to demonstrate
// updating
            functionality, we
// are going to demonstrate what
//
            happens when someone
// updates the state of a
            service
if(this.CommandTableName.ToUpper()
==
            "WIN32_SERVICE")
{
// this call gets a list of rows that
// were modified. These rows
// contain current values
            (after changes)
DataRow[] modifiedRows =
dataSet.Tables[CommandTableName].
Select("","",DataViewRowState.ModifiedCurrent);

//
            iterate through each modified row
foreach(DataRow dr in
            modifiedRows)
{
string previousServiceState =
dr["State",
            DataRowVersion.Original].ToString();

// make sure there was a
            change in the states. Someone
// could have made modifications to
            Data column values
// besides the State column. This line makes
            sure
// that the State column for the service row
// has
            indeed changed.
if(dr["State"].ToString() !=
            previousServiceState)
{
// create the Service controller
// associated with the
// service and system
            name
ServiceController sc =
new
            ServiceController(dr["Caption"].ToString(),
dr["SystemName"].ToString());

// this logic checks the
            modified rows
// state and either stops
// or starts the
            service.
if(dr["State"].ToString() ==
            "Stopped")
{
sc.Stop();
}
else if(dr["State"].ToString()
            == "Running")
{
sc.Start();
}
}
}
// return the
            number of modified rows
return
            modifiedRows.Length;
}
else
{
return
            0;
}
}

Since the provider only keys in on updates for WIN32_SERVICE
            objects, the method first verifies that the WMI Table name is
            WIN32_SERVICE . It then proceeds to grab all of the modified rows
            into a DataRow collection called modifiedRows by utilizing the
            DataTable's Select method and
            specifying the ModifiedCurrent DataViewRowState. The DataViewRowState ModifiedCurrent value informs the Select method to only return a list of
            current records that have been modified. Once the method has a list
            of rows that were modified, it iterates through each row, grabbing
            the previous value of the State column (remember this column
            indicates whether the service is running or not). If the calling
            program changed the value, the Update
            method then sets up the ServiceController object the correct
            information and starts or stops the service.

Using The Custom WMI Provider

Now that we have covered all of the properties and methods for
            the four interfaces, let's examine a quick application that tests it
            out. Included with the source code of the data provider is a Windows
            based application that allows you to type in a WMI query and
            populate a ListView control with the results. The screenshot of the
            application below shows the results from a "SELECT * FROM
            WIN32_PROCESS" query on my local machine.

The Server TextBox in the screen
            shot above allows a user to specify a server to execute queries
            against. The WmiQuery TextBox takes
            the specific WMI SQL query. Once the user has completed these two
            TextBoxes they may click the Execute
            button to execute the WMI query. The Stop/Start a Service button pertains to
            WIN32_SERVICE WMI queries. Once a WIN32_SERVICE query has been
            executed (i.e. SELECT * FROM WIN32_SERVICE), the user may select a
            service from the results grid and change the service's status from
            Running to Stopped and vice versa. The Clear button clears the Results
ListView.

There are quite a few WMI queries to choose from. Here are a few
            more for you to play around with:

QUERY | Description
SELECT * FROM Win32_service | Obtains a list of services running on the target machine.
SELECT * FROM Win32_LogicalMemoryConfiguration | Obtains one record listing details of the logical memory configuration from the target machine.
SELECT * from win32_LogicalProgramGroup | Retrieves a listing of the logical program groups on the target machine.
SELECT * from win32_operatingsystem | Obtains information about the operating system of the target machine.

You may also replace the * from the SELECT queries above with
            specific column names if you wish.

When you click on the Execute
            button, the button_click event is
            triggered, as shown in the next code block:

private void btnExecutDS_Click(object
            sender, System.EventArgs e)
{
// make sure the UI is completed
            correctly
if(tbWmiQuery.Text.Trim() ==
            "")
{
MessageBox.Show("Query is
            invalid.");
return;
}
if(tbServer.Text.Trim() ==
            "")
{
MessageBox.Show("Server must be
            specified.");
return;
}
try
{
this.ClearListView();
//
            create a WMI Connection with the
// connection
            string
WmiConnection conn = new
            WmiConnection();
conn.ConnectionString =
            tbServer.Text;
conn.Open();
// make sure the connection is
            open
if(conn.State !=
            ConnectionState.Open)
{
MessageBox.Show("Could not connect to
            {0}",
tbServer.Text);
return;
}
// create a new command
            with the WmiQuery
WmiCommand comm = new
            WmiCommand();
comm.Connection = conn;
comm.CommandText =
            tbWmiQuery.Text;
// Execute the command. We will be working
            with
// a reader
WmiDataAdapter wda = new
            WmiDataAdapter();
wda.Command = comm;
DataSet ds = new
            DataSet();

wda.Fill(ds);

this.CreateResultColumns(ds.Tables[0]);
this.PopulateListView(ds.Tables[0]);

}
catch(Exception
            ex)
{
MessageBox.Show(ex.Message);
}

}

After verifying the UI controls the code, it clears the ListView
            control, creates a connection to the server, and creates a WmiCommand and WmiDataAdapter object. It then populates a
            Dataset using the Adapter. Once the
            Dataset is populated, the method uses
            the DataSet to populate the ListView
            with columns and rows.

The CreateResultColumns method
            shown below creates a ListView column for each DataColumn in the DataTable:

private void CreateResultColumns(DataTable
            dt)
{
// the schema returned us the name and
// type of
            each of the columns.
// this routine creates ListView
            Columns
// for each column returned
foreach(DataColumn dc in
            dt.Columns)
{
lvResults.Columns.Add(dc.ColumnName,
100,
HorizontalAlignment.Left);
}
}

The PopulateListView method
            iterates through each row in the table and adds each column within
            the row to the result set.

private void PopulateListView(DataTable
            dt)
{
foreach(DataRow dr in dt.Rows)
{
ListViewItem lvi
            = new ListViewItem();
for(int i = 0; i < dt.Columns.Count;
            i++)
{
// itearate through each of the fields
// in the
            result set to continue
// updating the ListViewItem
if(i ==
            0)
{
lvi.Text =
            dr[i].ToString();
}
else
{
lvi.SubItems.Add(
            dr[i].ToString());
}
// add the results to the
//
            ListView
}
lvResults.Items.Add(lvi);
}

}

Additionally, the program shown in the screen shot above
            exercises the DataAdapter's update which allows you to start or stop
            a Windows service. The Windows application code that exercises this
            functionality is shown below and assumes the DataSet contains the
            results of a WIN32_SERVICE result set. After issuing a WIN32_SERVICE
            query, the user may select a service and click the Start/Stop a Service button. The following
            code gets executed when the user clicks the Stop/Start a Service button:

string strServiceName =
            this.GetCaptionItemText();

if(strServiceName ==
            "")
{
MessageBox.Show("Operation failed. Was your last "
            +
"query against the WIN32_SERVICE object? " +
"You must
            execute this query and then " +
"click the Start/Stop Service
            button.");
return;

}

if(_wmiDataSet != null
            && _wmiDataSet.Tables.Count > 0)
{
// look for the
            service with the correct caption

DataRow []dr =
            _wmiDataSet.Tables[0].
Select("Caption='" + strServiceName +
            "'",
"", DataViewRowState.CurrentRows);

// make sure we
            found a row
if(dr != null && dr.Length == 1)
{
//
            editing of the row begins
dr[0].BeginEdit();
// set the state

if(dr[0]["State"].ToString() ==
            "Running")
{
dr[0]["State"] =
            "Stopped";
}
else
{
dr[0]["State"] =
            "Running";
}
// editing of the row
            ends
dr[0].EndEdit();
// let the adapter handle the stopping
            or starting
// of the
            service
_wmiDataAdapter.Update(_wmiDataSet);

}
else
{
MessageBox.Show("Operation
            failed. Was your last " +
"query against the WIN32_SERVICE
            object? " +
"You must execute this query and then " +
"click
            the Start/Stop Service
            button.");

}
}
else
{
MessageBox.Show("Operation
            failed. Was your last " +
"query against the WIN32_SERVICE
            object? " +
"You must execute this query and then " +
"click
            the Start/Stop Service button.");
}

The code above first retrieves the value of the caption column
            within the ListView by invoking the GetCaptionItemText method. We will examine
            the code for this method next. The value of the caption column
            represents the name of the Service that the user selected. After
            retrieving the Service name, the code then uses the DataTable.Select method to locate the
            correct row in the DataSet. Once it has a valid row, the code then
            edits the row, toggles the State of the service and invokes the
            adapter's Update method. The Adapter
            takes over and executes our Update
            code which in turn starts or stops the service.

Here is the code for the GetCaptionItemText method:

private string
            GetCaptionItemText()
{
if(lvResults.SelectedItems.Count >
            0)
{
foreach(ColumnHeader ch in
            lvResults.Columns)
{
if(ch.Text == "Caption")
{
return
            lvResults.SelectedItems[0].
SubItems[ch.Index].Text;
}
}
}
return "";
}

Case Study Review

This case study covered quite a bit of material. We examined what
            a .NET data provider is, and explored the major reasons you might
            create one. We also touched on some of the guidelines to keep in
            mind when writing a provider. After a quick introduction to WMI, we
            proceeded to cover the implementation of the four major interfaces
            for ADO.NET providers. We completed the case study by walking
            through a simple Windows application that used the custom WMI data
            provider to retrieve and Update WMI data.

Case Study Information
Author | Todd Clemeston
Technical Editor | Adam Ryland
Project Manager | Helen Cuthill
Reviewers | Andrew Krowczyk, Saurabh Nandu, Slavomir Furman, Adam Ryland

If you have any questions or comments about this case study,
            or suggestions for further pieces, please contact the technical
            editor. If you wish to talk about the case study with other
            subscribers, please visit our forum here.

     Brought to you by ownSky!
 C#Today - © 2002 Wrox Press
Limited


Syntax highlighting with regular expressions – Part 2

Syntax highlighting with regular expressions – Part 2
http://www.csharptoday.com/content.asp?id=1830
Article Rating: 3
Published on March 8, 2002
Abstract
In the first part of this article, we developed a small application to make it simple to write, test and store regular expressions in .NET. In order to achieve our goal to be able to use syntax highlighting in our own applications, we need a way to assign formatting information to the regular expressions and save the data to a file that can be used in arbitrary applications. In this part, Alex Lehmberg develops a Windows Forms application that makes it easy to load a regular expression from a database and assign fonts and colors to the capturing groups in the regular expression.
Article

In the first part of the article (http://www.csharptoday.com/content.asp?id=1730&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq), we developed a small application to make it simple to write, test and store regular expressions in .NET (for more information on Regular Expressions, please see http://www.csharptoday.com/content.asp?id=1786&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq). In order to achieve our goal to be able to use syntax highlighting in our own applications, we need a way to assign formatting information to the regular expressions and save the data to a file that can be used in arbitrary applications. In this part, we will develop a Windows Forms application that makes it easy to load a regular expression from a database and assign fonts and colors to the capturing groups in the regular expression. When all the information is added, the entire configuration consisting of one or more regular expressions with formatting information can be saved to a file using binary serialization. Alternatively, the current configuration can be tested against an arbitrary file using the same application. By including the class library developed here, it should be easy to use syntax highlighting in any application using the RichTextBox control.

Introduction

Regular Expressions are ideal when we want to find patterns in text for later usage. By using regular expressions it is possible to take a piece of text and find all constructs of a particular type, which is very useful when developing application using syntax highlighting. Most applications of that kind use hard coded logic that matches and formats the different parts. In an XML-editor, it would be the elements, attributes, processing instructions, comments and so on. In this article we want to develop a more flexible approach that enables the use of an external file to hold all information about which patterns to match and how to format them. To achieve our goal, we will develop a small class library containing the code that has to be included in all applications using syntax highlighting and a Windows Form application to compose and test the configuration files. The configuration files will be stored and retrieved using binary serialization - one of the two serialization formats available in the .NET Framework.

System Requirements

No in depth knowledge is required, although prior experience with Windows Forms would come in handy. The code was developed with the final version of Visual Studio .NET. This article uses the SQL Server database developed in part 1, and an SQL Script is enclosed in the download.

Defining The Problem

Most people would, with some practice, be able to write some fairly powerful regular expressions, but the code needed to take the captured groups from the expression and apply the right colors and fonts can be quite tiresome to write. If there is only one expression with few capturing groups, it is entirely possible, but most applications would need multiple expressions, some of them with many capturing groups. The format information would in many cases be coded directly into the application making it difficult to customize.

To avoid this we want to be able to visually define the relationship between a regular expression, the capturing groups, and the font color and font type used to markup the captured group. When this is done, the information can be saved in a binary file, and subsequently used in other applications that use a RichTextBox control to display text.

The solution we are going to make consists of two projects. One for the WinForm application that we use to define format information, and one for the class library that any client application must use. To illustrate the application I have written a couple of regular expressions designed to deal with XML-files. One expression matches and stores the ordinary elements and any attributes, while the other matches XML comments. A series of other expressions would be required in order to match all content in an XML file, but the ones enclosed should illustrate the point nicely.

Defining The Solution

Database Design

The database used in the article was described fully in the first article. To recap quickly, the database consist of a single table with four columns used to store the regular expression, a description, a sample text or a sample file.

Component Design

Our class library contains four classes named FormatData, RegexInfo, FormatInfo and TextFormatter. The FormatData is the basic class containing information about the format of a single capturing group in a regular expression.

[Serializable]
public class FormatData : IComparable
{
private int groupNum;
private Color color;
private Font font;
...
}

The groupNum member corresponds to a capturing Group in a regular expression, while the font and color members contain the chosen color and font to apply to the group. Notice the Serializable attribute before the class declaration. This attribute is mandatory if we wish to serialize objects of this type. Another important point is that FormatData inherits the IComparable interface which makes it possible to sort Arrays and ArrayLists containing FormatData objects.

The actual implementation of the interface method is seen below:

public int CompareTo(object fd)
{
FormatData second = (FormatData)fd;
if(groupNum == second.groupNum)
return 0;
else if(groupNum < second.groupNum)
return -1;
else
return 1;
}

By using the CompareTo method it is possible to sort an array of FormatData objects based on the groupnumber in the regular expression. The FormatData class also implements properties for all the private members which makes it easy to set or get their values.

The second class RegexInfo encapsulates all the information about a single regular expression.

[Serializable]
public class RegexInfo
{
private ArrayList formatAL;
private Regex regex;
..
}

Just like the FormatData class, the RegexInfo class uses the Serializable attribute to indicate that objects of this type can be serialized. The class has two private members, the first one being an ArrayList to hold FormatData for every capturing group in the regular expression, and the second one being the actual expression to match. Most of the methods in the class are self explanatory as they just make it possible to add, retrieve and remove FormatData objects form the ArrayList. The only method needing some explaining is FindDataForGroup :

public int FindDataForGroup(int
            index)
{
int i = 0;
FormatData fd = null;
while(i <
            formatAL.Count)
{
fd =
            (FormatData)formatAL[i];
if(fd.GroupNumber ==
            index)
break;
i++;
}
if((fd != null) &&
            (fd.GroupNumber == index))
return i;
else
return
            -1;
}

The method searches the ArrayList for a FormatData object with the right groupNum and returns the index in the list.

The class FormatInfo acts as the top level class that holds information about several regular expressions. It contains an ArrayList that holds RegexInfo objects for each regular expression to test and format. It also stores the default color and font to apply to the text in the RichTextBox control.

[Serializable]
public class FormatInfo
{
private Color background;
private Font standard;
private ArrayList info;
..
}

The class contains properties to set and get the default color and font, and methods to add and retrieve RegexInfo objects from the ArrayList. So far the classes mentioned all hold information relating to regular expressions, or the color and font to apply to the different capturing groups. The last class, called TextFormatter, is the one doing the actual matching and formatting of a regular expression. The constructor takes a FormatInfo object as a parameter and stores the object in a private member variable. The only other method in the class, called Format, is the heart of the application doing the matching and formatting:

public void Format(RichTextBox textBox)
{
textBox.SuspendLayout();
textBox.ForeColor = info.BackGroundColor;
textBox.Font = info.BackGroundFont;

ArrayList regs = info.GetList();

The method uses the RichTextBox control to hold the formatted text. It starts by calling the SuspendLayout method on the RichTextBox. In doing so, the text is not refreshed while the formatting takes place, only after the method ResumeLayout is called. The default color and font is then applied to the entire text, and the ArrayList containing all the RegexInfo objects is retrieved.

while(i < regs.Count)
{
regexinfo =
            (RegexInfo)regs[i];
regex = regexinfo.GetRegex;
m =
            regex.Match(textBox.Text);
ArrayList temp =
            regexinfo.GetAllFormatData();
if(m.Success)
{
FormatData[]
            fd = new FormatData[m.Groups.Count];
int iter;
for(iter =
            0;iter < m.Groups.Count;iter++)
fd[iter] = null;
iter =
            0;
while(iter < temp.Count)
{
FormatData current =
            (FormatData)temp[iter];
fd[current.GroupNumber] =
            current;
iter++;
}
..
}

The outermost loop iterates over all the RegexInfo objects in the ArrayList. First the current regular expression object is retrieved and matched against the text in the RichTextBox control. If a match is found, a temporary array of FormatData objects is created and filled with the FormatData of the current RegexInfo object. The reason why we want to use a temporary array where each index corresponds to the group numbers in the regular expression is performance. By not having to search through all the FormatData in order to find the right one, the overhead is minimized - if we could be certain that every group number contained FormatData we could have avoided the array.

while(m.Success)
{
GroupCollection
            groups = m.Groups;
int num = 0;
while(num < groups.Count)
{
CaptureCollection captures = groups[num].Captures;
for(int tester=0; tester < captures.Count;
            tester++)
{
if(fd[num] !=
            null)
{
textBox.Select(captures[tester].Index,
            captures[tester].Length);
textBox.SelectionColor =
            fd[num].FontColor;
if(fd[num].FontType !=
            null)
textBox.SelectionFont = fd[num].FontType;
}
}
num++;
}
m = m.NextMatch();
}

The inner loop iterates through all the matches for a given regular expression. For every match, the code iterates over all the capturing groups and retrieves a CaptureCollection for every group. The for loop iterates through the CaptureCollection and checks if FormatData is defined and formats the captured text.

UI Design

The UI of the Winforms app is divided into two parts by using tabbed pages. The first part lets you edit the format information and decide which regular expressions from the database you want to include. It also lets you save the finished format information to a file or load an existing file. Most of the stuff is not that interesting. We use the ColorDialog and FontDialog to allow the user to pick the color and font that is to be used as default in every capturing group. When we add a regular expression from the database, it is displayed in the ListBox on the left. When an expression is clicked, the capturing groups are displayed in the ListBox on the right. If the user wants to assign format information to a capturing group, the group is clicked and the format information added by using the font and color dialogs.

The ability to save and load format information to and from files is the most important feature, because it makes it easy to make applications that use the format information defined in this application.

private void SaveData(string
            filename)
{
BinaryFormatter formatter = new
            BinaryFormatter();
try
{
Stream stream = new
            FileStream(filename, FileMode.Create, FileAccess.Write,
            FileShare.None);
formatter.Serialize(stream,
            fi);
stream.Close();
UpdateListBox();
}
catch(Exception
            e)
{
MessageBox.Show(e.ToString());
}
}

When the SaveData method is called, it creates a new BinaryFormatter object and opens a file for writing. When the Serialize method is called, the FormatInfo object is serialized to the file. By using binary serialization we conserve space and get the ability to serialize entire object graphs.

The code to get a saved file is quite similar, as seen below:

private void LoadData(string
            filename)
{
BinaryFormatter formatter = new
            BinaryFormatter();
try
{
Stream stream = new
            FileStream(tbFileName.Text, FileMode.Open, FileAccess.Read,
            FileShare.Read);
fi = (FormatInfo)
            formatter.Deserialize(stream);
stream.Close();
UpdateListBox();

}
catch(Exception
            e)
{
MessageBox.Show(e.ToString());
}
}

The code opens a file and the BinaryFormatter calls the Deserialize method. We use an explicit cast to create a FormatInfo object from the serialized data.

The second tab page is used to test the format information currently active. It consists of a RichTextBox control and an OpenFileDialog. When a file has been opened to test against, the Test button invokes the TextFormatter object with the RichTextBox as parameter.

private void btntest_Click(object sender,
            System.EventArgs e)
{
TextFormatter tf = new
            TextFormatter(fi);
tf.Format(rtbTest);
}

The Sample Application

To illustrate the application, I have written a couple of regular expressions and stored them in the database. The most important expression is:

(<)\s*(/*)\s*([^\n\r\t\f>/ ]*)\s*(?:\s*([^/=?>]*)\s*(=)\s*(")\s*([^\n\r\t\f"]*)\s*("))*\s*(/*)\s*(>)

It may look quite overwhelming, but it is actually quite simple. It matches XML elements and captures the different parts in capturing groups. The expression is not fully tested but works in all the cases I have tried. The advantage of having so many capturing groups is that it makes it easy to customize the look in any way imaginable.

In the screenshot you can see that capture group number 4 is set to blue and bold italic. How does that look if you try it on a simple XML file? Let's see:

You can see that the attribute names are colored blue and written in bold italic. The rest of the XML file is also formatted using different colors and fonts. On small XML files the speed is satisfactory, but as soon as the files get big it takes a long time to format the text.

Application Setup

All the relevant code is included in the download. I have also included a SQL Script which makes it easy to create the database. The database is the same as the one used in part 1 of the article. The connection string is placed in a separate XML file to make it easy to change.

Any Limitations or Further Work

As mentioned before, the code is all right if the XML files are small in size. If the files are big, it may be necessary to make changes to the application to make it more responsive. One solution would be to run the code in a separate thread, thus enabling the user to interact with the application while the formatter is working in the background. Otherwise, try experimenting with different regular expressions to get a feel of the speed.

Conclusion

The process of using regular expressions to format code in a RichTextBox is made easy by developing an application that lets developers write and test their own expressions and apply the desired formatting options to the different capturing groups. By using binary serialization, the settings are easily saved and restored in custom applications thus providing an easy way to deliver syntax highlighting in applications.

Article Information
Author | Alex Lehmberg
Technical Editor | Adam Ryland
Project Manager | Helen Cuthill
Reviewers | Shefali Kulkarni, Sean Schade

If you have any questions or comments about this article, please contact the technical editor.


Syntax highlighting with regular expressions – Part 1

Syntax highlighting with regular expressions – Part 1
http://www.csharptoday.com/content.asp?id=1756
Article Rating: 4.5
Published on March 7, 2002
Abstract
In many development tools, syntax highlighting (the process of marking up constructs and keywords with different colours and fonts) is commonplace. However, it is not very easy to write programs that format text in this way without ending up with a lot of code to parse the text and apply the correct formatting. In .NET, the support for regular expressions is quite extensive, and they make finding patterns in text very easy. In the first article in this two-part series, Alex Lehmberg explains how to use the different regular expressions classes to write the necessary regular expressions, and writes an application that makes it easy to test and store the expressions.
Article

In many development tools, syntax highlighting (the process of marking up constructs and keywords with different colours and fonts) is commonplace. However, it is not very easy to write programs that format text in this way without ending up with a lot of code to parse the text and apply the correct formatting. It can be done, but it is very hard to reuse the code in other applications where different rules apply. In .NET, the support for regular expressions is quite extensive, and they make finding patterns in text very easy. In the first article in this two-part series, I want to explain how to use the different regular expressions classes to write the necessary regular expressions. Furthermore I will write an application that makes it easy to test and store the expressions.

In the second article, I will show how to write a tool to apply different colours and fonts to the result from the regular expressions. The tool will store all information regarding regular expressions, colours and fonts in a file ready to be used in various applications.

Introduction

Most development tools offer some kind of syntax highlighting which helps developers get a better overview of the code they are working on. It is not all that hard to write a program that can take some form of code and display it using syntax highlighting, but in most cases it can be very time consuming. The worst aspect of writing code to do syntax highlighting is that it is very seldom that the code can be used for different formats. If you write a piece of code that can markup XML-files with the right colors and fonts, it is unlikely that you'll be able to reuse the code to markup CSS-files (Cascading Style Sheets).

Instead of writing essentially the same code every time, it is much easier to use the regular expressions classes in the .NET framework. In this article I'll start out by giving a brief introduction to the classes dealing with regular expressions, and following that we will build a test bed for developing regular expressions.

The application is meant to make it much easier to test different regular expressions against some text, as well making it easy to store and retrieve the expressions in a database. The database will come in handy in part 2 of the article.

System Requirements

The application was written using the final release of Visual Studio .NET, and SQL Server 2000 was used to create the database. No specific knowledge is required to understand the article, but some experience with Windows Forms is recommended. Even though the article is dealing with regular expressions it is not necessary to have any previous knowledge. In fact, the application should make it very easy for complete beginners to experiment with writing various regular expressions.

Defining Problem

Searching for specific strings in text is quite an easy thing to program if you know the exact string you are looking for. However, it is far more common that we want to find strings based on certain pattern, instead of precise words. An example could be an XML file, where we want to find all elements contained in the file. We know that an element starts with "<" and ends with ">" but we don't know the name of the element or how many attributes it contains. In an even more complex query we could search for the name of the element or for the name of the attributes. Doing so with normal code would be very tedious, and not all that effective and that is why regular expressions are a very welcome addition to the .NET Framework. Until now, regular expressions were mostly used with JavaScript and VBScript, but in .NET all languages have access to the same classes.

Defining Solution

All classes to do with regular expressions are contained in the System.Text.RegularExpressions namespace, and there are seven classes that you'll be using frequently:

  • Regex
  • MatchCollection
  • Match
  • CaptureCollection
  • Capture
  • GroupCollection
  • Group

The Regex is the class doing the actual work, and it is possible to initialize the class with a string representing the regular expression pattern you want to search for. The class contains several useful methods, such as GetGroupNames that returns an array of capturing group names. When a Regex object is constructed, any matches are obtained by calling the Match method, passing the string to search for through as a parameter.

The return value from Regex.Match() is a Match class and it is possible to determine if a match was found by checking the Success property. It is worth noting that calling the Match method on a Regex object always returns a Match class regardless of whether an actual match was found or not. The Match class has a property called groups that return a GroupCollection representing all the capturing groups in the regular expression. If the expression doesn't contain groups, or if all you want to do is to get the result from the entire expression, it is possible to retrieve the index and length of the match directly from the Match class.

The GroupCollection taken from Match contains all the groups in the expression. The individual Group members of the collection have a collection of captures. The Capture class represents the basic information about the individual string matched by the groups of the regular expression. The Capture class has only three properties: Index (the position in the original string), Length (the length of the capture), and Value (returns the string that was captured).

When calling the Match method on Regex only the first match is returned, and the class only looks for additional matches when the NextMatch method is called. If you want all matches found in one go, then use the Matches method from Regex which returns a MatchCollection containing all matches in the text.

By using the classes mentioned here, it is quite easy to write a small application that allows you to enter a regular expression and a sample text and display any matches in terms of groups and captures. For a more complete introduction please visit the links at the bottom. We also want to be able to save the regular expressions, and the sample text in a database for later use.

Database Design

The database is quite simple, and only contains one table with four columns: expression, description, filename, and sampletext. The stored procedures used to insert, update, delete, and retrieve data is generated automatically by Visual Studio.

The expression column contains the regular expression that you want to store, and the description column makes it possible to store a description for every expression. Both expression and description are required in every row. The last two columns, filename and sampletext, are related and only one of them is stored in every row. If the user enters a sample text to test an expression against, the text is stored in the sampletext field. If the sampletext already exist in a file, the user can browse for it and when selected, the content of the file is read into the textbox. When stored, the database only stores the name and path of the file and not the contents of the textbox.

UI Design

The user interface is divided into two parts using tabbed controls. The first part deals with database access and makes it easy to update, delete, or add new expressions to the database. The second and most interesting part deals with the construction and testing of regular expressions.

The first thing that you need to do is to create a new C# windows application using the wizard shown below:

When the project initializes, you see a blank form ready to accept the controls needed to crate the user interface. To avoid unnecessary clutter we divide the UI into two parts using a tabbed control to the form. Just add the TabControl by dragging it from the toolbox. In order for the tabcontrol to fill the entire form, set the Dock property to "fill". The next step is to add the TabPages. Find the Property named tabPages in the property inspector and click to the right of "Collection". This opens up the TabPage Collection Editor shown below:

Add to TabPages and adjust the Text-property to Data and Match respectively. We are now ready to start building the UI. Let's start by building the Data page. First we need to create a database connection, a DataAdapter, and a dataset to hold out data. Open the toolbox and find the data controls and drag an oleDbdataAdapter to the form. This opens the wizard that guides you through the configuration of the database connection and the SQL commands. Fist you need to choose the database to connect to, which is called Regex. Next select all the tables to use. When a database connection is established and the dataAdapter configured, create a new dataset by choosing the Generate Dataset in the Data menu. Name the new dataset dsRegex, and create it. Next drag a DataGrid control onto the form and adjust the DataSource property to dsRegex1, and DataMember to expressions. We are now ready to add some buttons to the form.

Drag a button to the form and name it btnLoad, and set the Text property to "Load Database". Double clik on the button to open up the code window, and type in the following code:

private void btnLoad_Click(object sender, System.EventArgs e)
{
dsRegex1.Clear();
oleDbDataAdapter1.Fill(dsRegex1);
}

The code clears the dataset and loads the data from the database to the dataset. The next button we want to add makes it possible to create a new expression. Drag another button onto the form, and name it btnNew. Set the Text property to "New Expression". Add the following code to the button:

private void btnNew_Click(object sender,
            System.EventArgs e)
{
currentIndex =
            -1;
tabControl1.SelectedTab = tabPage2;
tbExpression.Text =
            "";
tbDescription.Text = "";
tbFileName.Text =
            "";
rtbSample.Text = "";
}

The code sets the current index in the DataTable to -1 to indicate that the expression does not exist in the database. All TextBoxes are cleared and the visible TabPage is set to the Match TabPage.

The next button is used to edit an existing row in the DataTable. Drag another button onto the form, and name it btnEdit. Set the Text property to "Edit Row". Add the following code to the button:

private void btnEdit_Click(object sender, System.EventArgs e)
{
DataRow dr = dsRegex1.Tables["expressions"].Rows[dataGrid1.CurrentRowIndex];
tabControl1.SelectedTab = tabPage2;
currentIndex = dataGrid1.CurrentRowIndex;
tbExpression.Text = dr["expression"].ToString();
tbDescription.Text = dr["descriptions"].ToString();
tbFileName.Text = dr["filename"].ToString();
rtbSample.Text = dr["sampletext"].ToString();
}

The code retrieves the current row and fills all the TextBoxes in the other TabPage with the information in the DataTable.

The button with the text "Delete Row" deletes the current row from the database and updates the database. Drag another button onto the form, and name it btnDelete. Set the Text property to "Delete Row". Add the following code to the button:

private void btnDelete_Click(object sender, System.EventArgs e)
{
dsRegex1.Tables["expressions"].Rows[dataGrid1.CurrentRowIndex].Delete();
btnUpdate.PerformClick();
}

The last button on the page updates the database with all the changes made to the DataTable. Drag another button onto the form, and name it btnUpdate. Set the Text property to "Update Database". Add the following code to the button:

private void btnUpdate_Click(object sender, System.EventArgs e)
{
oleDbDataAdapter1.Update(dsRegex1);
MessageBox.Show("Database Updated");
}

On the second TabPage we have the ListBoxes that makes it easy to write and test our regular expressions. The topmost control is a RichTextBox which makes it easy to format the text. We also have an OpenFileDialog that makes it easy to pick a sample file on the computer. If the path and name of the sample text file is stored in the database, it is easily opened by clicking the "Open File" button.

When the user enters an expression in the textbox and presses the button Next Match , the event handler checks to see if the expression is new. If that is the case, the expression and the sample text are trimmed (which means removing leading and trailing whitespace characters) and a new Formatter object is created. If the expression is not new (meaning that it is not the first time the Next Match button has been pressed for the current expression) the Next method on the Formatter object is called:

private void btnNextMatch_Click(object
            sender, System.EventArgs e)
{
if(changed)
{
string temp
            = tbExpression.Text;
string trimmed = temp.Trim();
temp =
            rtbSample.Text;
string trimmed1 = temp.Trim();
RegexFormatter
            = new Formatter(rtbSample,trimmed1, trimmed,
lbGroups,
            lbCaptures, Color.Black, Color.Blue);
changed =
            false;
}
else
{
RegexFormatter.Next();
}
}

When the user has entered a new regular expression, it is saved to the database by clicking the Save To Database button:

private void btnSave_Click(object sender,
            System.EventArgs e)
{
if(currentIndex < 0)
{
DataRow
            newRow =
            dsRegex1.Tables["expressions"].NewRow();
newRow["expression"] =
            tbExpression.Text;
newRow["descriptions"] = tbDescription.Text;
if(tbFileName.Text != "")
newRow["filename"] =
            tbFileName.Text;
else
newRow["sampletext"] = rtbSample.Text;
dsRegex1.Tables["expressions"].Rows.Add(newRow);
tabControl1.SelectedTab = tabPage1;
btnUpdate.PerformClick();
}
else
{
DataRow dr =
            dsRegex1.Tables["expressions"].Rows[currentIndex];
dr["expression"]
            = tbExpression.Text;
dr["descriptions"] = tbDescription.Text;
if(tbFileName.Text != "")
dr["filename"] = tbFileName.Text;
else
dr["sampletext"] = rtbSample.Text;
tabControl1.SelectedTab = tabPage1;
btnUpdate.PerformClick();
}
}

The code first checks to see if we are editing an existing expression or a making a new one. If it is a new expression, a new row is added to the DataTable and the database is updated. Otherwise the existing row is modified and saved to the database.

Component Design

To keep the code in the user interface methods simple, I have chosen to abstract all the code dealing with formatting and regular expressions out into classes named Formatter and Matcher . The Formatter class is the link between the user interface and the regular expression classes. The constructor is called with the RichTextBox containing the text to search, the trimmed content of the RichTextbox, and the TextBox containing the regular expression to use. Besides that, the two ListBoxes used to display the groups and the captures are supplied together with the color to apply as background in the sample text and the color to highlight the found expression.

public Formatter(RichTextBox samplebox,
            string sample, string
expression, ListBox groups, ListBox
            captures, Color background, Color select)
{
match = new
            Matcher(expression, sample);
group = groups;
capture =
            captures;
rtbSample = samplebox;
highlight =
            select;

groups.Items.Clear();

string[] names =
            match.GroupNames();
foreach(string s in
            names)
groups.Items.Add(s);

rtbSample.SelectAll();
rtbSample.SelectionColor =
            background;
this.Next();
}

The constructor creates a Matcher object and displays all the capturing groups in the expression in the lbGroups ListBox. After that, it sets the default color of the text and calls the Next method, which is called every time the Next Match button is pressed:

public void
            Next()
{
match.NextMatch();
if(match.Success)
{
capture.Items.Clear();
capture.Items.AddRange(match.CaptureVals(0));
rtbSample.Select(match.GetCaptureIndex(),
match.GetCaptureLength());
rtbSample.SelectionColor
            = highlight;
}
}

The method starts out by calling the NextMatch method of the Matcher class. If a new match is found, the listbox containing the captures is cleared and the captures for the entire regular expression are displayed in the listbox. At the same time, the capture is located in the sample text and highlighted.

If the regular expression contains more than one capturing group, the user can choose to get the captures of another group displayed by clicking on the group in the ListBox. When doing so, the SelectGroup method is called:

public void SelectGroup(int
            groupnum)
{
capture.Items.Clear();
capture.Items.AddRange(match.CaptureVals(groupnum));
}

The parameter groupnum is the index in the ListBox listing the groups, and all the captured substrings for the given group are returned by calling the CaptureVals method of the Matcher class. The return values are displayed in the ListBox.

All the code dealing with the various regular expression classes is placed in the Matcher class that acts as a wrapper around the appropriate methods. The constructor takes the expression and the test string and creates an instance of the Regex class and the Match class by calling the Match method.

public Matcher(string regex, string
            text)
{
expression = new Regex(regex);
MatchNumber =
            0;
match = expression.Match(text);
}

The method NextMatch calls the NextMatch method of the Match class unless it is the first time it is called.

public void
            NextMatch()
{
if(MatchNumber != 0)
match =
            match.NextMatch();

MatchNumber++;
}

Most of the other methods just encapsulate the method calls to the underlying classes with the exception of CaptureVals . CaptureVals is used when the captured substrings from a capturing group needs to be displayed. The method iterates through a CaptureCollection and retrieves the captured string.

public string[] CaptureVals(int
            groupNum)
{
string[] captures = new
            string[match.Groups[groupNum].Captures.Count];
int i =
            0;

foreach(Capture c in
            match.Groups[groupNum].Captures)
{
captures[i] = (c.Value);
i++;
}
return captures;
}

The Sample Application

From the above screenshot it is easy to see how the sample application works. In the topmost textbox we have written some text to test with the regular expression written in the Expression textbox. If you look carefully at the expression which is: "(?<proto>\w+)://[^/]+?(?<port>:\d+)?/", you can see that it contains two groups named proto and port. The expression scans HREF's for the protocol and port number and saves them in the two groups.

In the Groups listbox there are three items (one is hidden on the screenshot) with the names: 0, proto and port. The reason why there are three instead of two is that the first group is always the entire regular expression. The captures of the first group (the entire regular expression) are displayed in the Captures listbox.

Application Setup

The application consists of a single Windows Form application which uses a database created with SQL Server 2000. A script file that can be used to recreate the database in Query Analyzer is in the code download for this article. When the database is setup it is probably necessary to change the connection string. That can be easily done using the Visual Studio .NET environment.

Any Limitations or Further Work

The application is not equipped with any error handling code, but should be fairly stable if used correctly. The sample application is only a small step towards the ultimate goal, which is to make an application that can be used to format arbitrary code using regular expressions.

Conclusion

We have developed a small application that makes it easy to write, test, and store regular expressions. We are however a long way away from accomplishing our goal which is to build an application that can take a regular expression and combine it with formatting information such as color and font and save the info in an external file which can be used in any Windows Forms application. That is our goal for the last part of this article, coming tomorrow.

Article Information
Author | Alex Lehmberg
Technical Editor | Adam Ryland
Project Manager | Helen Cuthill
Reviewers | Saurabh Nandu, Sean Schade

If you have any questions or comments about this article, please contact the technical editor.


Custom Controls and Design-Time Support – Part 2

Custom Controls and Design-Time Support – Part 2
http://www.csharptoday.com/content.asp?id=1755
Article Rating: 5
Published on March 6, 2002
Abstract
In .NET development, creating your own control is as easy as defining a custom class. However, making sure that your control behaves properly in the design time environment requires a little bit of extra wizardry. In this article, Matthew MacDonald considers the sample DirectoryBrowser control we looked at in the previous article, and shows how we can equip it for the design-time environment with a custom control designer and a UITypeEditor.
Article

In .NET development, creating your own control is as easy as defining a custom class. However, making sure that your control behaves properly in the design time environment requires a little bit of extra wizardry. In this article, we'll consider the sample DirectoryTree control we looked at in the last article (http://www.csharptoday.com/content.asp?id=1803&WROXEMPTOKEN=630431Z8rJR4vesk2Zd5MUhDgq), and show how you can equip it for the design-time environment with a custom control designer and a UITypeEditor.

The DirectoryTree

In the first article in this series, we considered a DirectoryTree control that automatically displays a hierarchical list of directories in the drive indicated by the Drive property. This control introduces several advantages over performing the same sort of work manually:

  • Though the DirectoryTree is based on a TreeView control, it isolates the client programmer from TreeView-specific details like nodes. It even raises a higher lever DirectorySelected event.
  • Directory entries are added automatically as needed when nodes are expanded, ensuring that the control performs equal well for small drives as for drives that have tens of thousands of subdirectories.
  • All the details are completely encapsulated. All the client programmer needs to worry about is setting the Drive property and handling the DirectorySelected event.

Of course, this control had its own quirks as well. Notably, if you set the Drive property at design time, you ended up with two copies of the root directory at run time-one created dynamically by the control, and one from the nodes that Visual Studio .NET has automatically serialized. Last article, we demonstrated a simple workaround, but a more elegant solution is possible with a custom control designers.

Before continuing any further, we need to import three additional namespaces, the bottom three shown below. These give us access to key pieces of .NET design-time support.

using System;
using System.IO;
using System.Windows.Forms;
using System.ComponentModel;
using System.ComponentModel.Design;
using System.Windows.Forms.Design;
using System.Drawing.Design;

Control Designers

One problem with traditional ActiveX control development is that the control's design-time behavior must be coded alongside the control's runtime behavior. In the world of .NET, this problem is neatly sidestepped by a new feature: control designers. Control designers are exclusively concerned with providing the design-time behavior for a control.

The .NET framework provides a basic control designer with the ControlDesigner class in the System.Windows.Forms.Design namespace. It also provides derived classes that add support for child control containment and scrolling. Controls can use the default designer (as we've done so far with the DirectoryTree control), or they can derive their own custom designers.

So why would you create your own designer?

  • To add special designer tools, like context menu options.
  • To remove inappropriate events or properties from view (or add design-time only events or properties).
  • To add support for controls that contain other controls (like the toolbar) or controls with special needs (like menus).

Filtering Control Class Members

From time to time, you might want to hide a control event or property from the design-time view of the developer. For example, every control provides a Text property, because every control inherits from the base System.Windows.Forms.Control class. However, not all controls can effectively use the Text property. For example, the TreeView control doesn't actually use its Text property - you can set it in code, but it won't modify the user interface. For this reason, the Text property is not displayed in the Properties window.

If you are defining a property, you can use the Browsable attribute to keep it from appearing in the Properties window. However, consider the TreeView control, which provides a Nodes collection. The DirectoryTree inherits the Nodes property, and allows it to be modified at design-time, even though the list of nodes is built automatically at runtime based on the Drive property. This conflict means that there are really two ways the developer can alter the Nodes collection-indirectly through the Driver property, or directly through the Nodes collection, which is sure to lead to a great deal of confusion.

Unfortunately, because the TreeView.Nodes property is not overridable, you can't use the Browsable attribute. However, you can create a custom designer that ensures it won't appear at design-time. To use filtering with the DirectoryTree, create a custom designer class that derives from ControlDesigner :

public class DirectoryTreeDesigner : ControlDesigner
{

protected override void PostFilterProperties(
System.Collections.IDictionary properties)
{
properties.Remove("Nodes");
}

}

This designer overrides the PostFilterProperties() method, and removes the Nodes property from the properties collection.

The next step is to link the custom designer to the DirectoryTree control. To do this, use the Designer attribute, and specify the appropriate designer type.

[DefaultEvent("DirectorySelected"),
DefaultProperty("Drive"),
Designer(typeof(DirectoryTreeDesigner))]
public class DirectoryTree : TreeView
{

Note that the Nodes property is still accessible in code. This allows clients to perform other useful tasks (like enumerating through the collection of nodes) at their discretion. However, it can't be designed, and it won't be serialized.

Designer Verbs

You can also use a custom designer to add to the context menu that's displayed when a programmer right-clicks on your control in the design environment. This menu contains some standard options provided by Visual Studio .NET, but it can also contain your own commands (technically known as verbs).

To add your own verbs, you need to override the Verbs property, create a new DesignerVerbCollection, and add the appropriate DesignerVerb object entries. Your control designer handles the verb, generally by updating the associated control.

The following example retrieves a list of all the drives on the current computer, and adds a context menu entry for each one. The user can click on the appropriate entry to set the Drive property of the control.

public class DirectoryTreeDesigner : ControlDesigner
{

protected override void PostFilterProperties(
System.Collections.IDictionary properties)
{
properties.Remove("Nodes");
}

DesignerVerbCollection verbs = new DesignerVerbCollection();

public DirectoryTreeDesigner()
{
// Configure the designer verb collection.
string[] drives = System.IO.Directory.GetLogicalDrives();

foreach (string drive in drives)
{
verbs.Add(new DesignerVerb("Set Drive " + drive,
new EventHandler(OnVerb)));
}
}

public override DesignerVerbCollection Verbs
{
get
{ return verbs; }
}

protected void OnVerb(object sender, EventArgs e)
{
// Retrieve the selected drive.
char driveLetter = ((DesignerVerb)sender).Text[10];

// Adjust the associated control.
((DirectoryTree)this.Control).Drive = driveLetter;
}

}

Generally you won't use your designer verbs to provide settings for a property. A more interesting technique is to provide higher-level configuration operations that adjust several properties at once. One example of this is found in the ASP.NET Calendar control, which allows the user to choose a theme from a list of preset choices. When a theme is selected, several properties are modified in conjunction.

The implementation for this design is refreshing easy. Just add a Windows form to your project, and display it when the appropriate designer verb is selected. Here's another simple example using the DirectoryTree. This time, only a single verb is available, which then displays a window that allows the user to choose a drive. When a drive is chosen, a public form-level variable is set, and then retrieved by the designer, which applies the change. This approach is more manageable than the previous design, and doesn't clutter the context menu with drive letters:

public class DirectoryTreeDesigner : ControlDesigner
{

protected override void PostFilterProperties(
System.Collections.IDictionary properties)

{
properties.Remove("Nodes");
}

DesignerVerbCollection verbs = new DesignerVerbCollection();

public DirectoryTreeDesigner()
{
// Configure the designer verb collection.
verbs.Add(new DesignerVerb("Set Drive",
new EventHandler(OnVerb)));
}

public override DesignerVerbCollection Verbs
{
get
{ return verbs; }
}
protected void OnVerb(object sender, EventArgs e)
{
// Show the form.
SelectDrive frm = new SelectDrive();
frm.DriveSelection = ((DirectoryTree)this.Control).Drive;
frm.ShowDialog();

// Adjust the associated control.
((DirectoryTree)this.Control).Drive = frm.DriveSelection;
}

}

The SelectDrive form is quite simple:

public class SelectDrive : System.Windows.Forms.Form
{
private System.Windows.Forms.Button cmdClose;
private System.Windows.Forms.ListBox lstDrives;

private void cmdClose_Click(object sender, System.EventArgs e)
{
this.Close();
}

public char DriveSelection;

private void listDrives_SelectedIndexChanged(object sender,
System.EventArgs e)
{
DriveSelection = lstDrives.Text[0];
}

private void SelectDrive_Load(object sender, System.EventArgs e)
{
string[] drives = System.IO.Directory.GetLogicalDrives();
lstDrives.DataSource = drives;

// Select the current drive.
lstDrives.SelectedIndex =
lstDrives.FindString(DriveSelection.ToString());

// Attach the event handler.
// This step is performed after the selected index is set,
// to prevent it from being overwritten as the list is built.
this.lstDrives.SelectedIndexChanged += new
System.EventHandler(this.listBox1_SelectedIndexChanged);

}

}

One quirk remains in the designer that we've built. When the DirectoryTree.Drive property is modified by the designer, the Properties window is not updated until the control is deselected and then reselected. To correct this defect, you need to explicitly notify the IDE that a change has been made.

The rewritten OnVerb() method handles this:

// Notify the IDE that the Drive property has changed.
PropertyDescriptorCollection properties;
properties = TypeDescriptor.GetProperties(typeof(DirectoryTree));
PropertyDescriptor changedProperty = properties.Find("Drive", false);
this.RaiseComponentChanged(changedProperty, "", frm.DriveSelection);

When you add a form to a control project in this way, the client will be able to see the form class, and even create and display instances of it. If this isn't the behavior you want, you will need to nest your form class inside your control class and make it private or protected. Unfortunately, if you do this, you have to forego Visual Studio .NET's design-time support, and manually copy the form code into the class.

UITypeEditors

The Properties window does a solid job of providing support for all common data types. Basic data types like strings and numbers are all seamlessly supported. If you create a property that uses a custom enumeration, the Properties window automatically provides a drop-down list with the values from the enumeration.

In addition to this basic level of support, you may have noticed that some richer object types have additional design-time support. For example, if you create a Property that has a Font or Color data type, a special color picker or font selection dialog will be used in the Properties window. Similar magic happens if you create a Collection property. This support is provided through .NET UITypeEditor classes, which are special components that provide the design-time user interface that allows the programmer to set complex properties.

Prebuilt UITypeEditors

The base UITypeEditor class is found in the System.Drawing.Design namespace. You can inherit from this class to create your own custom UITypeEditors, or you can use one of the derived classes that are provided with the .NET framework. These classes are listed below (UITypeEditors that are only usable with specific web controls have been omitted-see MSDN for more information about those).

Class | Description
System.ComponentModel.Design.ArrayEditor | Edits an array by allowing the programmer to enter a list of strings. Used automatically for supported arrays.
System.Drawing.Design.FontEditor | Allows the programmer to select and configure a font. User automatically for font properties.
System.Drawing.Design.ImageEditor | Allows the programmer to create an Image object by selecting a bitmap from an open file dialog. Used automatically for image types.
System.Web.UI.Design.WebControls.RegexTypeEditor | Allows the programmer to choose a regular expression from a list of common choices. This UITypeEditor works with string properties.
System.Windows.Forms.Design.AnchorEditor | Allows the Anchor property to be set at design time.
System.Windows.Forms.Design.FileNameEditor | Allows a fully qualified filename to be set by choosing a file from an open file dialog box. This UITypeEditor works with string properties.

Depending on the data type of your property, .NET may use the corresponding UITypeEditor automatically (for example, with a Font). On the other hand, some UITypeEditors do not have dedicated types. An example is the RegExTypeEditor, which allows the programmer to choose a common regular expression for a control property. The regular expression is stored as an ordinary string, so it needs to be explicitly associated with the appropriate UITypeEditor.

Custom UITypeEditors

You can also develop your own custom UITypeEditor classes to allow special settings to be configured. For example, consider the TreeView control. Its Nodes property is a collection, but it doesn't use the standard collection editor (which only allows strings to be entered). Instead, it uses its own specialized UITypeEditor.

To create a custom type editor, you must first create a class that derives from System.Drawing.Design.UITypeEditor. You can then override the four methods shown below:

Class | Description
EditValue() | Invoked when the property is edited. Generally, this is where you would create a special dialog box for property editing.
GetEditStyle() | Specifies whether the type editor is a DropDown (provides a list of specially drawn choices), Modal (provides a dialog box for property selection), or None (no editing supported).
GetPaintValueSupported() | Use this to return True if you are providing a PaintValue() implementation.
PaintValue() | Invoked to paint a graphical thumbnail that represents the value in the property grid.

The next example uses the EditValue() method with the DirectoryTree control. It allows editing of the Drive property by presenting the dialog box we developed earlier:

public class DriveEditor : UITypeEditor
{
public override System.Drawing.Design.UITypeEditorEditStyle
GetEditStyle(System.ComponentModel.ITypeDescriptorContext context)
{
// We will use a window for property editing.
return UITypeEditorEditStyle.Modal;
}

public override object EditValue(
System.ComponentModel.ITypeDescriptorContext context,
System.IServiceProvider provider, object value)
{
SelectDrive frm = new SelectDrive();

// Set current drive in window.
frm.DriveSelection = (char)value;
frm.ShowDialog();

// Return the new value.
return frm.DriveSelection;
}

public override bool GetPaintValueSupported(
System.ComponentModel.ITypeDescriptorContext context)
{
// No special thumbnail will be shown for the grid.
return false;
}

}

The type editor is attached to the appropriate property using an attribute:

[Category("Appearance"),
Description("A letter representing the drive the DirectoryTree will use."),
Editor(typeof(DriveEditor), typeof(UITypeEditor))]
public char Drive
{
get
{
return _drive;
}
set
{
_drive = value;
{
RefreshDisplay();
}
}
}

One benefit of this design is that you can reuse this UITypeEditor with any drive property, in any control. It is specific to the property data type, not the control. An alternative approach is to use a DirectoryInfo object to represent the drive instead of an underlying char. Because the property editing is now handled by the UITypeEditor, there's no need to choose a basic type that can be edited with the default design-time support built into the property grid.

Summary

In this article, we considered two of the most important ingredients for creating design-time support for custom controls: control designers and UITypeEditors. The most exciting part about these classes is that they use 100% standard .NET code. That means you can write user interfaces to support your controls in the design-time environment with the same controls and windows form classes you would use in an ordinary .NET application. Similarly, you can weave UITypeEditors that read settings from an XML file, control designers that incorporate rich windows controls in a multi-step wizard, or even a license provider that connects to a web service!

Article Information
Author | Matthew MacDonald
Technical Editor | Adam Ryland
Project Manager | Helen Cuthill
Reviewers | Andrew Krowczyk, Saurabh Nandu

If you have any questions or comments about this article, please contact the technical editor.


Custom Controls and Design-Time Support – Part 1

Custom Controls and Design-Time Support – Part 1
http://www.csharptoday.com/content.asp?id=1829
Article Rating: 4.5
Published on March 5, 2002
Abstract
Creating custom controls is a key technique in .NET development. A well-written custom control can improve encapsulation, simplify a programming model, and bundle up an important set of functionality into one attractive package. However, though custom controls perform wonders at runtime, many act oddly in design mode, don’t provide any developer support, and appear with only a generic icon in the toolbox. This article by Matthew MacDonald is the first of a two part series (the next part is tomorrow) where he looks at how to create a custom Windows Form control that behaves properly in Visual Studio .NET.
Article

Creating custom controls is a key technique in .NET development. A well-written custom control can improve encapsulation, simplify a programming model, and bundle up an important set of functionality into one attractive package - and if you are a control vendor, you can even sell it! However, though custom controls perform wonders at runtime, many act oddly in design mode, don't provide any developer support, and appear with only a generic icon in the toolbox. This article is the first of a two part series (the next part is tomorrow) where we look at how to create a custom Windows Form control that behaves properly in Visual Studio .NET.

Inherited Controls in C#

.NET supports three basic types of custom controls for Windows Forms applications:

  • User controls inherit from the System.Windows.Forms.UserControl class, and follow a model of composition. User controls represent the simplest type of control, and generally combine more than one existing .NET control in a logical unit (like a group of text boxes for entering address information).
  • Inherited controls derive from another .NET control that provides the basic functionality. The .NET class library is filled with examples of inherited controls. For example, LinkLabel derives from Label and CheckedListBox derives from ListBox.
  • Owner-drawn controls derive from the base System.Windows.Forms.Control class, and provide their own drawing logic. With an owner-drawn control, the programmer needs to manage every aspect of the control's behavior, including details like mouse clicks.

In this article, we'll consider an example of an inherited control. Inherited controls are an ideal way to take functionality from the .NET base classes and extend it. Inherited controls can closely resemble their parent class, or they can be dramatically different, with a unique properties, methods, and events.

Control Projects

Custom controls are nothing more than custom classes that derive from a .NET base control class. That means you can add them to any .cs code file, and even create them alongside your application code, in an ordinary Windows Forms project. However, the easiest way to develop custom controls is in a separate dedicated project. This approach makes it possible to reuse the control as a compiled assembly DLL. It also gives you an opportunity to add and test the required design-time support. If you add an inherited control to an ordinary project, you won't be able to add the control to the Toolbox or test its design-time support.

The easiest way to begin coding a custom control is to create a Class Library Project (which is recommended for most custom control projects) or a Windows Control Project (which is generally used for user controls). Both of these selections work equally well-the only difference is in the default references that are set, and the namespaces that are initially imported into your project. The important detail is that you are creating a library project, which creates a DLL assembly, not a stand-alone executable. You can create a class library project that contains several files and multiple controls, or you can create a separate project and assembly for each custom control you make.

When you begin your control project, you will probably find that you need to add a few assembly references and import some namespaces:

  • Typically, you'll need to add references to the System.Windows.Form.dll, System.Drawing.dll, and System.Design.dll assemblies.
  • Useful namespaces include System.Windows.Forms, System.ComponentModel, and System.Drawing. Importing namespaces isn't required-it's just a handy convenience that helps trim down long lines of code.

To build your project at any time, right-click on it in the Solution Explorer and choose Build. The DLL file will be placed in the bin\debug or bin\release subdirectory of your project directory, and can be shared with any other project that needs to use the control.

The DirectoryTree Control

The DirectoryTree control inherits from the standard TreeView, and adds the features needed to display a hierarchical view of directories. .NET does not include any type of native directory control, so this TreeView is genuinely useful.

The DirectoryTree provides the following functionality:

  • Adds a Drive property that allows the programmer to specify what drive will be shown.
  • Refreshes its display automatically, but "just in time" as branches are expanded. This ensures optimum performance, particularly on large hard drives.
  • Raises a higher-level DirectorySelected event that provides information about the directory a user has clicked on. This allows you to use the DirectoryTree on its own, or create a basic Windows Explorer type of application by updating a ListView with file items every time the DirectorySelected event occurs.

The basic structure for the TreeView control is shown below. A DirectoryTree namespace includes both the control and the custom event arguments object used for the DirectorySelected event:

using System;
using System.IO;
using
            System.Windows.Forms;
using
            System.ComponentModel;

namespace DirectoryTree
{
public
            class DirectoryTree : TreeView
{
// Code
            omitted.
}

public class DirectorySelectedEventArgs :
            EventArgs
{
// Code omitted.
}
}

The basic public interface of the DirectoryTree class includes the Drive property (which allows a single Char to represent the drive letter), and the DirectorySelected event:

public delegate void DirectorySelectedEventHandler(object sender,
DirectorySelectedEventArgs e);
public event DirectorySelectedEventHandler DirectorySelected;

char _drive;

public char Drive
{
get
{
return _drive;
}
set
{
_drive = value;
RefreshDisplay();
}
}

The DirectoryTree is populated in the RefreshDisplay method. Note that the DirectoryTree fills itself by reading sub-directories as the appropriate branches are expanded. That means that the control operates very quickly, even if the drive has tens of thousands of subdirectories. Only the expanded directory levels are actually shown. The collapsed branches all have a dummy node inserted:

// This is public so a Refresh can be
            triggered manually.
public void RefreshDisplay()
{
// Erase
            the existing tree.
this.Nodes.Clear();

// Set the first
            node.
TreeNode rootNode = new TreeNode(_drive +
            ":\\");
this.Nodes.Add(rootNode);

// Fill the first level
            and expand
            it.
Fill(rootNode);
this.Nodes[0].Expand();
}

void
            Fill(TreeNode dirNode)
{
DirectoryInfo dir = new
            DirectoryInfo(dirNode.FullPath);

// An exception could be
            thrown in this code if you don't
// have sufficient security
            exceptions for a file or directory.
// You can catch and then
            ignore this exception.

foreach (DirectoryInfo dirItem in
            dir.GetDirectories())
{
// Add node for the
            directory.
TreeNode newNode = new
            TreeNode(dirItem.Name);
dirNode.Nodes.Add(newNode);
newNode.Nodes.Add("*");
}
}

Every time a directory branch is expanded, the inherited control checks if a dummy node is present and, if it is, the dummy node is removed and the directories are read from the disk. Note that the base TreeView events are handled by overriding the corresponding "On" methods, which is the recommended approach. These methods must always call the base implementation of the overridden method to ensure the control works as expected, and raises the event to all listeners.

protected override void OnBeforeExpand(TreeViewCancelEventArgs e)
{
base.OnBeforeExpand(e);

// If a dummy node is found, remove it and read the real directory
// list.
if (e.Node.Nodes[0].Text == "*")
{
e.Node.Nodes.Clear();
Fill(e.Node);
}
}

The Directory Selected event is raised when a node is clicked. Once again, the event is handled by method overriding:

protected override void OnAfterSelect(TreeViewEventArgs e)
{
base.OnAfterSelect(e);
// Raise the DirectorySelected event.
if (DirectorySelected != null)
{
DirectorySelected(this,
new DirectorySelectedEventArgs(e.Node.FullPath));
}
}

The DirectorySelected EventArgs class is quite straightforward, with a single DirectoryName property and a basic constructor:

public class DirectorySelectedEventArgs : EventArgs
{
public string DirectoryName;

public DirectorySelectedEventArgs(string directoryName)
{
this.DirectoryName = directoryName;
}

}

Throughout the rest of this article, and tomorrow's installment, we'll look at the DirectoryTree control, and consider how to improve its design-time support, bringing it from a useful control class to a redistributable control component.

Creating a Client

To use your control in another project, you need to add a reference to the compiled assembly. When you add a reference, Visual Studio .NET stores the location of the file. Every time you rebuild your file, it copies the latest version from that directory into the bin directory, along with the executable for your test program. That ensures that you are always testing against the most recent build. Because .NET recognizes classes using assembly metadata, you never need to worry about registering or registering your control.

To add a reference to a control, right-click on the Toolbox, and choose Customize. Then, select the .NET Framework Components tab, and click the Browse button. Once you select the appropriate assembly, all the controls will be added to the list, and checkmarked automatically (see Figure 8-4).

When you click OK, your control will be added to the bottom of the Toolbox, alongside its .NET counterparts (alternatively, you could add a custom Toolbox tab for your custom controls to improve organization). Because no custom icon is configured for the DirectoryTree control, it will appear with the default gear icon. You can then create instances of this control by drawing on the design surface.

When you actually deploy an application that uses a custom control, all you need to do is ensure that the required control DLL is in the same directory as the application executable. When you copy these files to another computer, you do not need to worry about registering them or performing any additional steps. This is the infamous XCOPY deployment that is heavily hyped with .NET.

Quirky Design-Time Behavior

When you first begin using the DirectoryTree control, you may notice some quirky design-time behavior. At first, it seems straightforward enough-once you set the Drive property, the corresponding directory tree will appear. You can even expand nodes and browse the directory structure at design time.

However, when you start your program, a second set of identical directory nodes will appear. The problem is that the nodes you create at design time are automatically serialized to the form's designer code. At runtime, the control is recreated, the directory nodes are rebuilt when the drive property is set, and then the serialized nodes are added.

There are several ways you can resolve this problem. First of all, you could change the order of the form designer lines so that the Drive property is configured after the serialized nodes are added (setting the Drive property automatically clears the current list of nodes). Alternatively, you could create a custom designer, as we will in the next article. The simplest approach, however, is just to configure the DirectoryTree control so that it doesn't provide its directory node display at design-time. You can implement this simple fix by explicitly checking what mode the control is in before refreshing the display:

public char Drive
{
get
{
return
            _drive;
}
set
{
_drive = value;
if
            (!this.DesignMode)
{
RefreshDisplay();
}
}
}

Adding a Toolbox Bitmap

Adding a toolbox icon is refreshing easy. All you need to do is add a bitmap to your project, and ensure it has the same file name as your custom control class. This bitmap must meet a few basic criteria:

  • It must be 16 pixels by 16 pixels. Otherwise, Visual Studio .NET will attempt to scale it at the results will be ugly.
  • It must use only 16 colors.

Set the Build Action for the file to Embedded Resource, and recompile the control project. When you add the control to a client project, the embedded bitmap will appear in the toolbox.

Attributes

The next step in improving the DirectoryTree control is adding designer attributes. These attributes instruct the IDE how to treat various parts of your control. To specify this type of information about a custom control in another programming language, you would either need to create a separate file in a proprietary format, or use some sort of visual tool. With attributes, the information that describes your control can be easily created and edited alongside your code, but it is still cleanly separated from the logic that generates the user interface.

You may have noticed that the DirectoryTree.Drive property appears in the design window grouped under the generic "Misc" category, without any additional information. You can improve on this situation using attributes. The example below adds a Description, Category, and DefaultValue attribute to the Value property. Note that when you use more than one attribute, they are all enclosed between square brackets, and separated with commas. The underscore character is used to spread the attributes over several lines for better readability.

[Category("Appearance"),
Description("A letter representing the drive the DirectoryTree will use.")]
public char Drive

All of these attributes are found in the System.ComponentModel namespace, along with many more that allow you to configure various aspects of your control's design-time behavior. The table below lists some of the most useful attributes you can use to configure properties.

Attribute | Description
Browsable(True|False) | If False, indicates that a property should not be shown in the Properties window. However, the property will still be accessible through code.
Category("") | Sets the category under which the property will appear in the Properties window. If a category with this name doesn't exist, it will be created.
DefaultValue("") | Sets the initial value that will be used for this property when the control is created.
Description("") | Specifies the text description that will be displayed for this property in the Object Browser of Properties window.
DesignOnly(True|False) | When set to True, this property will only be available at design time. This is typically used with special properties that configure how a control behaves at design time (like a SuppressUI property), and don't correspond to a "real" piece of information about the control.
ImmutableObject(True|False) | When set to True on an object property, this attribute ensures that the sub-properties of this object will be displayed as read-only. For example, if you apply this to a property that uses a Point object, the X and Y sub-property will be read-only.
Localizable(True|False) | When set to True, the design-time value for this property will be stored in a resource file, instead of in the generated code. This makes it easy to swap the value later by introducing a new resource file. When the user configures properties that don't use this attribute, the appropriate code will be inserted in the hidden designer region of the form, unless it requires a special data type (like an image) that must be stored in a resource file.
NotifyParentProperty(True|False) | Set this to True to indicate that a parent property should receive notification about changes to the property's value (and update its display accordingly). For example, the Size property has two nested properties: Height and Width. These nested properties should be marked with this attribute.
ParenthesizePropertyName(True|False) | When True, indicates that the property should be displayed with brackets around it in the Properties window (like the Name property).
ReadOnly(True|False) | When True, this property is read-only in the Properties window at design time.
RefreshProperties() | You use this attribute with a value from the RefreshProperties enumeration. It specifies whether the rest of the Properties window must be updated when this property is changed (for example, if one property procedure could change another property).

A few attributes can be applied to your custom control class declaration, rather than a specific property. These include two attributes that set the default event and property. Here's how we could use these attributes with the DirectoryTree :

[DefaultEvent("DirectorySelected"),
DefaultProperty("Drive")]
public class DirectoryTree : TreeView

Attribute | Description
DefaultEvent | When the application programmer double-clicks on your control, Visual Studio .NET will automatically add an event handler for the default event.
DefaultProperty | The DefaultProperty is the property that is highlighted in the Properties window by default, the first time the control is selected.

Summary

In this article, we developed the DirectoryTree control and showed how to add the basic design-time support by adding a toolbox icon, checking the control run mode, and applying attributes. In the next article, we'll tackle designers and UITypeEditors to add a richer level of design-time interface. These tools provide another way for developers to configure their custom controls at design-time.

If you want to continue to work with the DirectoryTree code, you could add a number of enhancements. For example, you might provide a SelectedDirectory property that returns the full path of the currently selected node, and allows you to change it programmatically. You could also add a custom ListView control to provide a synchronized file list, simply by responding the DirectorySelected event.

Article Information
Author | Matthew MacDonald
Technical Editor | Adam Ryland
Project Manager | Helen Cuthill
Reviewers | Andrew Krowczyk, Saurabh Nandu

If you have any questions or comments about this article, please contact the technical editor.


An In-Depth Look at the NumberFormatInfo Class

An In-Depth Look at the NumberFormatInfo Class
http://www.csharptoday.com/content.asp?id=1754
Article Rating: not yet rated
Published on March 4, 2002
Abstract
NumberFormatInfo (NFI) is the class in System.Globalization that provides culture-sensitive formatting for currency, numbers, and percentages. Next to confusing short dates (is 11-08-90 the 8th of November or the 11th of August?), numeric values displayed without regard to user expectations can cause annoyance and misinterpretation. What does 2,103 really mean? Is it two thousand one hundred and three, or two and one hundred and three thousandths? In financial or medical software the answer can be crucial. In this article, Bill Hall explores the methods and properties of NFI to discover how it can help us to create displays that meet user requirements for a given world region.
Article

NumberFormatInfo (NFI) is the class in System.Globalization that provides culture-sensitive formatting for currency, numbers, and percentages. Next to confusing short dates (is 11-08-90 the 8th of November or the 11th of August?), numeric values displayed without regard to user expectations can cause annoyance and misinterpretation. What does 2,103 really mean? Is it two thousand one hundred and three, or two and one hundred and three thousandths? In financial or medical software the answer can be crucial. In this article, we will explore the methods and properties of NFI to discover how it can help us to create displays that meet user requirements for a given world region. Assisting us is a form written in C# that shows most of the properties and methods of the class. The program is interactive; you can change specific properties, including separators, grouping, signs, and precision, and view the results in any .NET-supported culture.

The implementation of NFI has its roots in two Win32 functions, GetNumberFormat and GetCurrencyFormat. Although useful and versatile, these two API's are not particularly convenient to apply, especially when you want to change a parameter from the standard locale offering. For example, users like to see an integer with many digits with group separators, as its value is less likely to be interpreted incorrectly. The number 210398398750 is hard to read, but 210,398,398,750 (or if you prefer 201.398.398.750) is much easier to sort out visually. In Win32, a straightforward application of GetNumberFormat groups the whole part correctly according to the locale setting, but always tacks on a fractional component as well. To eliminate the fractional part, the programmer has to fill out six fields in a NUMBERFMT structure by repeated calls to GetLocaleInfo using LCTYPES that correspond to the NUMBERFMT fields, and then set the fractional digits to zero. Using NFI, a simple property setting, NumberDecimalDigits = 0, does the trick.

An instance of NFI is a property of a specific culture instance of System.Globalization.CultureInfo. A specific culture is determined by both language and region. A parent culture, which references language but not region, contains only a null value for its NFI. The reason is that the features of NFI are dependent on a particular world region as well as language. For example, here are two rather different number formats from two countries bordering on each other and sharing a common language:

French (Switzerland) | 123'456'789'876'543'210.99
French (France) | 123 456 789 876 543 210,99

Before reading this article you may find it helpful to review our previous one about .NET's CultureInfo class in Internationalization With The .NET CultureInfo Class & C#. You may also want to look at An In-Depth Look at the DateTimeFormatInfo Class, as the two classes have similar uses.

Simple Number Formatting

It is easy to control the format of a number. Just use the variable's ToString method with one of the eight formatting letters given in Table 1. Not all data types will work with all formatting strings; for example, you cannot use the hex formatter with a decimal type. If in doubt, use a try-catch block.

The typical code is val.ToString("<formatstring>"); where val is a numeric data type such as double, integer, decimal, float, etc. Unfortunately, not all of the possible format strings are completely documented in the .NET 1.0 Help. For example, the percentage formatters "p" and "P" are missing. Also, there is no explanation of why lower and upper case forms are provided, although apparently the reason is to case certain letters in the output (see, for example, the scientific and hexadecimal results) in Table 1.

Type | Format string | Output (lower case) | Output (upper case)
Currency | C, c | $123,456,789.99 | $123,456,789.99
Integer | D, d | 123456789 | 123456789
Scientific | E, e | 1.234568e+008 | 1.234568E+008
Fixed point | F, f | 123456789.99 | 123456789.99
General | G, g | 123456789.987654 | 123456789.987654
Number | N, n | 123,456,789.99 | 123,456,789.99
Round Trip | R, r | 123456789.98765433 | 123456789.98765433
Percent | P, p | 98.77 % | 98.77 %
Hexadecimal | X, x | 75bcd15 | 75BCD15

Table 1: Formatting Characters for Numbers

Of these, only Currency, Number, and Percentage are controlled by NFI. For example, in the table above, the thread culture just happened to be en-US when the results were recorded. However, we can create a new culture, say Swiss (French), and then augment the formatting by referencing the CultureInfo's implicit NFI as follows:

CultureInfo ci = CultureInfo.CreateSpecificCulture(de-CH);
String result = Val.ToSTring("<character>", ci);

In this case, three of the outputs above now appear as shown in Table 2; the rest remain the same:

Currency | SFr. 123'456'789.99 | SFr. 123'456'789.99
Number | 123'456'789.99 | 123'456'789.99
Percent | 98.77% | 98.77%

Table 2: Swiss-French Number Formats

So, most of the time, you don't have to explicitly reference an NFI instance if the current thread's CultureInfo matches the user's requirements since CultureInfo itself contains an instance of NFI. However, if you want to obtain information unique to NFI or modify the output, then you will have to obtain one explicitly. This is very easy to do with the line of code NumberFormatInfo nfi = ci.NumberFormat; where ci is a specific (not parent) CultureInfo object. Now let's take a look at the sample application that accompanies this article to see what is provided.

NumberFormatInfo Details

Figure 1 shows the demo program with the culture set to Russian (Russia). The form itself is divided into three sections, reflecting the partition of properties and methods for Currency, Number, and Percentage. Within each section the properties themselves are shown. Also displayed are the resulting formats for number, currency, and percentage. At the bottom are some additional pieces of information about how to correctly render "not a number" and negative and positive infinity in the selected culture.

You should be aware that although the formats tend to be somewhat similar for currency, number, and percent, each should be regarded as a separate formatting task. Often, programmers tend to think of currency formatting as a special kind of number, but that is not the case. You cannot just drop in a currency symbol in front of a number expression and expect to have a culturally correct result. Rounding rules can be very different between currencies and numbers. Numbers can have fractional parts but the currency may have none (Japanese Yen, Indonesian Rupees). Placement of the currency symbol in relationship to the number is another issue -front or back - spaced or not - from the actual value. The problem becomes even more acute when working with negative values. Not all cultures use a minus sign, but if present, positioning becomes even more complicated when considering relative order of symbol, sign, and value. If you run through the listed cultures you can find sixteen ways of arranging debit currency and five ways for showing negative numbers. Even countries using the Euro have their own individual styles: sometimes the Euro symbol is in front (Italy, Austria) and sometimes behind the value (Portugal, Greece, Luxembourg, Germany, Spain, Finland, France).

Another interesting problem is how to separate whole part grouping of numbers. You may have noticed the row of nine NumericUpDown controls in the currency and number subareas (we could have also done the same for percentage as well). Many Indian cultures actually separate whole parts not simply by three's but twos and threes such as this example from Hindi -1,23,45,67,89,87,65,43,210.99. In NFI, you can actually create up to 9 possible group sizes and in any order. In the demo program you can experiment with the various possibilities. This feature is probably overkill, both in the class implementation in .NET as well as in the demo program, but it was probably just as easy to implement the general case as it was to meet the specific requirements of the Indian locales. It does, however, make writing the demo program harder!

Figure 1: NumberFormatInfo Demo

By the way, you may notice that for some cultures, the text may not look quite right, especially for the fields at the bottom of the form. In this case, you might want to add additional language support. You can do this using the Regional Settings dialog and your Windows installation CD. Just click on the additional language settings you need and then insert the CD when requested.

NumberFormatInfo Properties

The main properties that you can change are described next. Note they are different properties for each number, currency, and percent and are not shared. For example, you will find separate properties for the number of decimal digits used in formatting currency, number, and percentage, namely CurrencyDecimalDigits, NumberDecimalDigits, and PercentDecimalDigits, respectively. For simplicity in the description below, we have grouped them together by category and used a descriptive name for each. Not all are present for each number type.

Decimal Separator | The string separating whole and fractional parts
Group Separator | The string separating groups of whole part digits
Decimal Digits | An integer representing the number of fractional digits (0-99)
Currency Symbol | The string representing the currency symbol
Positive Sign | A string that denotes that a number is positive
Negative Sign | A string that denotes that a number is negative
Percent Symbol | A string that denotes a percentage value
Per Mille Symbol | A string that denotes a per mille (thousandth) value
Positive Pattern | A digit representing positioning of symbol, value, and spacing. Ranges are (0-3) for currencies and (0-2) for percentages. For example, the possible orderings are for credit currency are $n, n$, $ n, and n $, where $ represents the monetary string.
Negative Pattern | A digit representing possible positioning of symbol, sign, value, and spacing. Ranges are (0-15) for currencies, (0-4) for numbers, and (0-2) for percentages. For example, negative percentages could have the forms -98.77 %, -98.77%, or -%98.77.
Group Sizes | An array of 9 values, each ranging from 1 to 9, that describes how many elements to group for each whole part segment. The last element can be zero to indicate repetition of the next to last value.

Changing the Default Formats

NumberFormatInfo gives you a set of useful defaults formats. By using the properties of a suitable NFI you can modify the output. There may be many reasons for doing this. Let's give an example. Many European countries now use the Euro, but historical data might have to be rendered using a pre-Euro format. Here is an example from Portugal using the Euro.

Adjusting the decimal separator to ($) and the currency symbol to Escudos provides the older format:

We have already mentioned the example of providing a readable format to an integer just by adjusting the number of decimal digits. In this example, we have adjusted the number of decimal digits in a number format to none.

If we want to go wild with group sizes, try this one. Just for fun, we also changed the negative pattern so that the negation sign is at the back and spaced away from the value. We could have also changed the separators and signs. These examples should give you a good idea of what NFI can be made to do for you.

By the way, after you have completely messed us the settings in the demo you can reset them to the culture's defaults by using the Reset button!

Code Samples

There is absolutely nothing difficult about working with NFI. As for the demo form itself, the only complicated part is managing the Group Sizes display, and that is not particularly interesting, just tedious. The details of generating a specific CultureInfo list have already been discussed in An In-Depth Look at the DateTimeFormatInfo Class.

Here are some details about the test values used in the program and the code for rendering the currency, number, and monetary displays. Notice that we alternated use of upper and lower case formatting characters, but found no differences in results for the currently supported cultures. We also used the NFI explicitly in the ToString methods, i. E. curci.NumberFormat, but the code works the same if just curci is used. Here, curci is just a writeable copy of the currently selected specific culture.

private CultureInfo curci;
private
            decimal d = 123456789876543210.987654321M;
private decimal n =
            -123456789876543210.987654321M;
private double p =
            1234.987654321;
private double pn =
            -1234.987654321;

private void ShowCurrency()
            {
txtCurrency.Text = d.ToString("c",
            curci.NumberFormat);
txtNegativeCurrency.Text = n.ToString("C",
            curci.NumberFormat);
}

private void ShowNumber()
            {
txtNumber.Text = d.ToString("n",
            curci.NumberFormat);
txtNegativeNumber.Text = n.ToString("N",
            curci.NumberFormat);
}

private void ShowPercent()
            {
txtPercent.Text = p.ToString("p",
            curci.NumberFormat);
txtNegativePercent.Text = pn.ToString("P",
            curci.NumberFormat);
}

The rest of the code manages the events that occur when the user alters the selected culture or modifies the fields; most is as easy as setting or reading a property.

Conclusion

Respect your clients and customers preferences by making good use of the formatting (NumberFormatInfo, DateTimeFormatInfo) classes of CultureInfo. These classes will give you culturally correct results for your thread of execution almost without intervention or modification on your part. Use these classes right from the beginning of project development, and you'll be able to get to market without the wasteful and expensive step of an internationalization rewrite.

If you would like to read some background on the issues of internationalizing a product take a look at the two articles located here: http://www.mlmassociates.cc/thoughts_on_internationalization.htm?WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0. For more information you should consult http://www.microsoft.com/globaldev?WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0.

Article Information
Author | Bill Hall
Technical Editor | Adam Ryland
Project Manager | Helen Cuthill
Reviewers | Andrew Krowczyk, Saurabh Nandu

If you have any questions or comments about this article, please contact the technical editor.


An In-Depth Look at the DateTimeFormatInfo Class

An In-Depth Look at the DateTimeFormatInfo Class
http://www.csharptoday.com/content.asp?id=1827
Article Rating: 4
Published on March 1, 2002
Abstract
The DateTimeFormatInfo (DTFI) class is one of the most important members of the .NET System.Globalization namespace. Although acting mainly as the IFormattable interface for the DateTime structure, DTFI is also informational. You can, for example, find out the names of the days of the week and months of the year in native form, an amazing number of alternate date and time patterns that are acceptable in a particular culture, and which week is the first one of a new year. In addition, DTFI provides the main access to the more than half-dozen calendars supported by .NET. In this article, Bill Hall will show you how easy it is to use the facilities of DTFI, and will gain experience in writing code that functions correctly independently of language and world region.
Article

The DateTimeFormatInfo (DTFI) class is one of the most important members of the .NET System.Globalization namespace. Although acting mainly as the IFormattable interface for the DateTime structure, DTFI is also informational. You can, for example, find out the names of the days of the week and months of the year in native form, default time and date separators, ante- and post-meridiem strings for time expressions, an amazing number of alternate date and time patterns that are acceptable in a particular culture, and which week is the first one of a new year. In addition, DTFI provides the main access to the more than half-dozen calendars supported by .NET. After you read this, you will see how easy it is to use the facilities of DTFI, and will gain experience in writing code that functions correctly independently of language and world region.

System.Globalization.CultureInfo affects nearly every class in System.Globalization, and DTFI is no exception. In fact, an instance of CultureInfo normally contains a DTFI object (except in a special case, explained below). So, you may find it helpful to review our previous article about cultures in .NET in Internationalization With The .NET CultureInfo Class & C#. For an introductory article to DTFI, see Managing Culturally Correct Date and Time Formats in C#, and you may also want to glance through it for some basic information on DTFI usage in both client and web programming. Most of the material here is based on a single C# form that we wrote to illustrate and exercise the many methods and properties of DTFI. You will find it helpful to download the program and run it yourself as you read through the article.

Obtaining a DateTimeFormatInfo Instance

Most of the time, DTFI does its job without you being aware of it. For example, if we create an instance of DateTime and then display its contents with the ToLongDateString or ToShortDateString methods then we will obtain a reasonably well-formatted result that meets user expectations for the CultureInfo of the current thread. But, you have to work directly with a DTFI for some problems. For example, suppose you want to provide the user with a list of the days of the week in his or her language and highlight the start day. Using DTFI we can carry out this task very easily. Notice also that you don't have to research this information yourself.

// use the current thread culture
CultureInfo ci = CultureInfo.CurrentCulture;

// get a DateTimeFormatInfo from CultureInfo
DateTimeFormatInfo dtf = ci.DateTimeFormat;

// clear the listbox
lstDayNames.Items.Clear();

// get weekday list, full and abbreviated
string [] daynames = dtf.DayNames;
string [] abbrevdaynames = dtf.AbbreviatedDayNames;

// probably seven days but don't assume anything!
int len = daynames.GetLength(0);
// format the result using a string formatter (makes translation easier!)
for (int i = 0; i < len; i++)
lstDayNames.Items.Add(String.Format("{0} ({1})", daynames[i], abbrevdaynames[i]));

// choose the first day of the week in this locale and select it
int fd = lstDayNames.SelectedIndex = (int)dtf.FirstDayOfWeek;

You can write very similar code in ASP.NET, however, you will want to reflect the user's preferred culture not that of the server's thread culture. One way would be to use the HTTP_ACCEPT_LANGUAGE parameter. However, you want to be careful to create a specific culture. For example, suppose the user's browser language preference is Russian. In both IE and Netscape, the language tag transmitted to the server is "ru". If you instantiate a CultureInfo using new CultureInfo("ru"), you'll be out of luck getting to a DTFI since the CultureInfo object is a parent culture, and parent cultures have only a null value for their DTFI.

By contrast, a specific culture must be instantiated by a parameter that contains both language and regional content. Examples are en-UK for English (United Kingdom), ar-JO (Arabic, Jordan), fr-FR for French (France), etc. Why is a specific culture necessary? It is because a DTFI contains information that can be both region and language dependent. For example, the usual UK short date form of day/month/year differs from its US month/day/year counterpart, despite a (more or less!) common language. Another example can be found between the two German speaking regions Austria (de-AT) and Germany (de-DE), where the name of the first month of the year differs in spelling: 'Januar' in Germany, but 'Jänner' in Austria. In this example, both regions even share a common frontier, but an Austrian customer will be annoyed if your software mixes them up- and vice versa.

The implication is that if you create a parent culture - one that references language only - then the instance's DateTimeFormatInfo property is null. A similar comment holds for another formatter class, NumberFormatInfo. So, if you expect to be able to use culture-specific dates and times (or numbers, currencies, and percentages), use CultureInfo.CreateSpecificCulture to obtain a specific CultureInfo instance. Even if you instantiate it with a language only (i. e. "de", "en", "ar"), you will get a specific culture with a default region. Thus the solution to the problem above is solved by using the call CultureInfo.CreateSpecificCulture("ru"). Once you have a specific culture, you can use CultureInfo.DateTimeFormat to obtain a usable DTFI.

By the way, web programmers tend to disregard the browser language tag and depend more on a previously set cookie for such information. Even if set, it may not indicate a user's preferences. However, many web sites, including http://www.microsoft.com/?WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0, seem to use it as an initial preference. For testing purposes, I often modify my browser language setting and later forget what I have done. More than once, I've found myself looking at Microsoft's Chinese or Indian web site wondering how I got there.

Basic Features of DateTimeFormatInfo

An instance of DTFI contains lots of useful information. Figure 1 (below) will give you an idea of its scope. In this case, the selected culture is English (United States), mainly to provide easy readability for most programmers. However if you have the demo program at hand, try viewing some of the other 100+ cultures available. You may not be able to read everything, but it will help you gain awareness and appreciation of how many different ways there are for handling date, time, and calendar related issues.

At the top of the form is a list of specific cultures obtained basically by calling the static method:

CultureInfo [] ci = CultureInfo.GetCultures(CultureTypes.SpecificCultures);

Internally, the enumeration is obtained indirectly. List and dropdown controls allow you to insert a reference to a class as an item in the list. If you do this, then the display string is determined by the ToString method of the object. Since the return from ToString is not always very user friendly, I derived a class from CultureInfo whose ToString method returns the CultureInfo.DisplayName property. The derived class is the one actually inserted into the culture dropdown list. Using this technique, the friendly name matches the current .NET runtime language. When I have to use the selected culture, I have all its methods and properties immediately available. Even better, the list is sorted correctly according to the current thread culture and I don't have to maintain any kind of internal data structure; the control is storing the data and doing all the work for me.

Figure 1: DateTimeFormatInfo Demo Program

Here is the code for the derived class.

using System;
using
            System.Globalization;

namespace WinDateTimeFormat {
public
            class MyCLInfo : CultureInfo {
public MyCLInfo(string name) :
            base(name) {}
public MyCLInfo(string name, bool useoverride)
: base(name, useoverride) {
}
public override string
            ToString() {
return base.DisplayName
}
}
}

The second controlling list in the demo program is the list of calendars of the culture. The .NET Framework supports eight calendars, although a detailed discussion of the Calendar class is beyond the scope of this article. However, the particular calendar in use greatly affects the content of a DTFI. In the demo program, you can view any combination of culture and available calendars. You may see more than one Gregorian calendar for a given culture. Even the en-US culture has two; one is US English and the other is Localized. Just below the calendar list you can see the Gregorian Calendar type name.

Here is a complete list of all supported calendars along with some long date displays. All displays represent the same instant of time. You can clearly see the impact a change of calendar might have on a string representation. By the way, a Calendar class provides a way for managing time increments and decrements. The action of adding a day or a month can produce quite different results from one calendar to another, and can be very difficult to do correctly, especially for lunar calendars. Are any of these alternate calendars useful in the real world? You bet, if you are doing business with a customer whose preferred calendar is one of them!

Gregorian Calendar | Sunday, November 11, 2001
Hebrew Calendar | 25 ???? 5762
Hijri Calendar | 25 ?????, 1422
Japanese Calendar | ?? 13?11?11?
Julian Calendar | Monday, October 29, 2001
Korean Calendar | ?? 4334? 11? 11? ???
Taiwan Calendar | 90?11?11?
Thai Buddhist Calendar | 11 ????????? 2544

By the way, you may need to add additional language support for you Windows 2000 or XP system to see some of these forms. To do this, get your installation CD and go to the Regional Settings dialog. Check the language groups you want, and insert the CD when requested.

Let's look again at Figure 1 to see a number of useful items. We have already discussed the native names of the weekdays and the first day of the week. But, you can also read a list of month and abbreviated month names, strings for the equivalent of 'AM' and 'PM' in other locales, default field separators for date and time expressions, calendar era names, and a value that may be new to you: the first week of the year. In fact, System.Globalization actually has an enumerator for this: the CalendarWeekRule:

FirstDay | The first week of the year starts on the first day of the year and ends before the following designated first day of the week. The value is 0.
FirstFourDayWeek | The first week of the year is the first week with four or more days before the designated first day of the week. The value is 2.
FirstFullWeek | The first week of the year begins on the first occurrence of the designated first day of the week on or after the first day of the year. The value is 1

Accounting and other rules for a given world region can be affected by such rules, and some years may have more or less than 52 weeks, particularly when the FirstFourDayWeek rule is in effect. Examples of locations using this rule include Denmark, The Netherlands, Estonia, Faro Islands, Finland, Switzerland, Germany, Iceland, Italy, Latvia, Norway, Poland, and Sweden. The only FirstFullWeek country seems to be Pakistan. Most countries use the FirstDay option.

Date and Time Options

The main feature of DTFI is the plethora of formats for date and time in an amazing number of combinations. For the U. S. alone you can find 129 formats. Why so many? The main reason is to be able to provide your customer with a complete list of culturally acceptable formats. You don't have to do the research - it has been done for you. In addition, there are a few specialized forms such as RFC 1123 and ISO 8601. All together, there are eleven categories with one aggregating all the rest. A glance at Figure 1 shows the available patterns on the left as dropdown lists, and the current selection on the right as formatted by DTFI according to the current culture and calendar. All values are based on DateTime.Now, which changes each time you select a new culture or calendar.

How do you enumerate each pattern category? The basic DTFI method is GetAllDateTimePatterns using a character parameter, or none at all if all formats are required. Early .NET documentation was very confusing on which characters to use; it took some experimentation to find out what really works. Fortunately, the help has improved with the 1.0 release. Below is a code fragment from the demo application that enumerates all the forms for the full date and time patterns (this group is selected by the characters 'f' and 'F') and inserts the results into a combo box. The routine also selects the default pattern if DTFI has a corresponding property for that category. Also note that the text direction is set to RightToLeft for bi-directional writing systems such as Arabic, Farsi, Divehi, Hebrew, Urdu, and Syriac. Actually, this step is not entirely necessary since .NET will manage right-to-left behavior correctly (including reorienting the controls) if the appropriate thread culture has been set. You can see this in the demo program by setting your user locale in the Regional Settings dialog to one the Arabic locales. A similar technique works when serving a web page from .NET; in the code-behind, set the thread locale accordingly in Page_Load().

// full date time
// clear the combo
            box
cbFullDateTimePattern.Items.Clear();

// enumerate the
            full date time patterns and load them
foreach(string pattern in
            dtf.GetAllDateTimePatterns('F'))
cbFullDateTimePattern.Items.Add(pattern);

//
            enumerate the full date, short time patterns and load
            them
foreach(string pattern in
            dtf.GetAllDateTimePatterns('f'))
cbFullDateTimePattern.Items.Add(pattern);

//
            there is a default FullDateTimePattern method.
// find it in the
            list and select it
cbFullDateTimePattern.SelectedIndex =
cbFullDateTimePattern.FindStringExact(dtf.FullDateTimePattern);

//
            set the text direction
txtFullDateTime.RightToLeft =
            textdir;

There are differences, by the way, if the formatting character is lower case or upper case. Not all are explained very well. In the code fragment above, 'f' enumerates all full dates with short times (no seconds) and 'F' provides full dates with long times. A similar comment holds for 'g' and 'G' when applied to general dates and times. Others are explicitly separated, such as 'd' for short date, and 'D' for long date. Others, such as year/month ('y', 'Y') and month/day ('m', 'M') seem to generate the same patters for both upper and lower case, and I have not been able to infer from examples whether there are significant differences.

DateTimeFormatInfo Patterns

As you have probably guessed, you can pass a date-time pattern obtained from DTFI directly to the appropriate overloaded ToString method of DateTime. You can even create your own patterns, although if you are trying to get a culturally correct result, this may not be advisable. Here is how a selected full date time pattern is used to create the corresponding output in the demo program. The code here is in response to a user changing the combo box selection containing the full date time format strings. A similar refresh takes place if the user changes the calendar.

// full date time
// dt is a DateTime instance created from DateTime.Now
// cbFullDateTimePattern is the combo box showing full date time patterns
// txtFullDateTime is a text box holding the formatted result
// curci is the currently selected CultureInfo instance

protected void
cbFullDateTimePattern_TextChanged(object sender, System.EventArgs e) {
try {
txtFullDateTime.Text = dt.ToString(cbFullDateTimePattern.Text,
curci.DateTimeFormat);
} catch {
}
}

A complete discussion of pattern types is documented in the .NET help. Look in the Index for DateTimeFormatInfo - about DateTimeFormatInfo class. However, just by looking at the demo program you will certainly figure most of them out. For example the full date pattern

dddd, MMMM dd, yyyy h:mm:ss tt

is interpreted as follows: use the full name of the weekday (dddd), the full name of the month (MMMM), a two digit number with leading zero for the day (dd), a four-digit year (yyyy), an hour with no leading zero (h), a two-digit minute with leading zero (mm), a two-digit seconds display with leading zero (ss), and a meridiem string such as AM or PM (tt).

Did you look carefully at Figure 1? In the long date pattern there is some English text, 'Today it is '. In fact, in the demo program all the combo boxes containing date patterns are writeable so that you can insert your own text. Be sure to enclose your text in single quotes, however. You can add text as needed when rendering displays and pass the resulting combination of text and date/time pattern to the ToString method. However, it is strongly recommended that you use a formatter to make the result localizable to other languages. Fragments or concatenated strings can be nearly impossible to translate because of differing word order, syntax, and grammar constraints. For example, you should format this string like this before passing it on to the DateTime.ToString method:

// cbFullDateTimePattern is a combo box
            holding a list of date time patterns
string
            strMyDate;
strMyDate = String.Format("Today it is {0}",
            cbFullDateTimePattern.Text);

Default Formats in DateTimeFormatInfo

Having all those alternate formats can be useful, but usually you are happy just to have one. Here is a table of date and time default properties, patterns, and examples sorted by the left column. I've chosen Czech to illustrate. The property names are self-documenting (if you read English!)

As a concluding comment, although the month name for February in Czech is únor, it is not used quite right here in most of the displays. The form shown is in the nominative case (brush off your grammar book - it is the dictionary form) but it should appear as "února" (the genitive or 'of' case) in most of the examples. Case endings in languages often take the place of prepositions. In this case you translate "11. února" as the "11th of February". The same problem exists in .NET with related languages such as Russian, Ukrainian, Polish, Slovak, and others. Declensional languages can add a new layer of complication in both internationalization and localization of programs and web sites and it is always best if such details can be handled by the runtime system. Interestingly, Microsoft does it right in Win32 in GetDateFormat, although it took them several years to get around to the task. I'm told that this oversight in .NET will be fixed 'soon'.

Conclusion

.NET provides a powerful and versatile way to manage date and time formats and to provide essential information about months of the year, days of the week, first day of the week and first week of the year, alternate calendars and eras, and a large selection of alternate date and time patterns that meet local cultural expectations. Of course, the ideas for the class have roots in Win32, and if you would like to obtain a useful utility that runs there, you can download it from http://www.mlmassociates.cc/dl-win32.htm?WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0. Look for the nLocale application. You can also find quite a lot of information on the web site about internationalization on Win32 and .NET systems as well as links to some other sites dealing with the subject. You can also find a long list of references to articles and books there at http://www.mlmassociates.cc/biblio.htm?WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0. Finally, don't overlook the Microsoft site http://www.microsoft.com/globaldev?WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0, which provides an often-overlooked resource for international development.

Article Information
Author | Bill Hall
Technical Editor | Adam Ryland
Project Manager | Helen Cuthill
Reviewers | Andrew Krowczyk, Saurabh Nandu

If you have any questions or comments about this article, please contact the technical editor.


Dynamic Assemblies in .NET framework

Dynamic Assemblies in .NET framework
http://www.csharptoday.com/content.asp?id=1751
Article Rating: 4
Published on February 27, 2002
Abstract
One of the compelling features provided by the .NET framework is the usage of assemblies in simplifying deployment problems and solving DLL hell. Assemblies are considered as the fundamental building blocks of an application created using .NET Framework. Assemblies can be classified into two categories: Static Assemblies and Dynamic Assemblies. In this article we will see what Dynamic Assemblies are, their applications, and how they can be created and consumed using the .NET Framework.
Article

One of the compelling features provided by the .NET framework is the usage of assemblies in simplifying deployment problems and solving DLL hell. Assemblies are considered as the fundamental building blocks of an application created using .NET Framework. Assemblies can be classified into two categories: Static Assemblies and Dynamic Assemblies. In this article we will see what Dynamic Assemblies are, their applications, and how they can be created and consumed using the .NET Framework.

Introduction

Before we proceed with the details of the dynamic assemblies, let us take some time to illustrate the core concepts of .NET framework such as Assemblies and Reflection that are crucial to understanding the importance of dynamic assemblies.

Assemblies and Reflection

An assembly represents a logical collection of one or more exe or dlls that contain an application's code and resources. Assemblies are considered as the primary building blocks of a .NET Framework application, and are the units of deployment for a .NET application. Assemblies are much more than deployment. The following list serves to explain the importance of assemblies in the .NET Framework:

  • Versioning in .NET is done at the assembly level. It is also the unit at which side-by-side execution is supported.
  • The code access security system uses assemblies at the core of its permissions model.
  • The most important element of the assembly is the manifest, which is a metadata description of the code and resources inside the assembly. Since the manifest contains all the information about the assembly, it is responsible for the self-describing nature of the assembly.

For more information on assemblies, please look at the following article from MSDN. Simplifying Deployment and Solving DLL hell with the .NET Framework.

In simple words, Reflection can be defined as the ability to examine type information and then invoke its methods at runtime. The type information that we can examine includes fields and methods present in the class, attributes defined for the methods and classes, interfaces implemented by the class and much more. The importance of reflection can be gauged from the fact that development tools like Visual Studio.NET depend on reflection to provide the users with the rich IDE that makes developing NET applications such an easy experience. For example, the Intellisense feature is made possible due to the reflection capabilities supported by .Net framework. Also tools like debuggers and object browsers use reflection internally to accomplish their functionality.

What are Dynamic Assemblies?

When we create an assembly, we typically include .NET Framework types such as interfaces and classes, as well as resources for the assembly (like resource files, bitmap files, JPEG files, and so on), and then compile it to create an output file. These files are stored on disk in the form of Portable Executable (PE) files. They are called Portable Executable files because they are portable across all Microsoft 32-bit operating systems. For example, the same PE-format executable can be executed on any version of Windows 95, 98, Me, NT, and 2000. Therefore, all PE files are executable, but not all executable files are portable. Since we statically compile the desired items into the assembly, this kind of assembly is called Static assembly. For example, if you use Visual Studio.NET and build or compile a typical Visual Studio.NET project, you will end up making a S tatic Assembly that can either be an exe or a dll depending on the type of project you have chosen.

As the name suggests, dynamic assemblies means creating code on the fly. This means we create code dynamically at runtime and then execute it directly from the memory without saving it to disk. However it is possible to save dynamic assemblies into disk before executing them. This process of emitting code dynamically (and creating dynamic assembly) is called Reflection Emit . It is mainly accomplished through a set of managed types defined in the System.Reflection.Emit namespace that not only enables us to emit metadata and MSIL code at runtime but also optionally generate a Portable Executable (PE) file on disk. The System.Reflection.Emit namespace exposes a wealth of functionality that is mainly useful for creating dynamic assemblies. The following list briefly explains some of the functionalities provided by System.Reflection and System.Reflection.Emit namespaces:

  • Allows us to define an assembly at runtime and then execute it
  • Allows us to define an assembly at runtime and save it to disk
  • Allows us to define modules in assemblies and then execute it
  • Allows us to define types dynamically, create instance of these types, and invoke the type's methods
  • Allows us to define symbolic information for types, which can then be used by tools such as debuggers and code profilers.

Dynamic assemblies are created using the methods defined in the System.AppDomain class. Later in this article we will look at the means through which we can create dynamic assemblies. There are three modes in which Dynamic assemblies can be created. These modes determine the lifetime and the transient nature of an assembly. They are defined as follows:

  • Run - When this is used as an access mode, the dynamic assembly generated using the AssemblyBuilder object is transient (which means it exists only for the duration of the execution of the program and does not survive system shutdown).
  • Save - Using this mode, we can create a dynamic assembly that can be persisted. However we cannot execute the saved assembly until we load the saved assembly file from the disk into the memory.
  • RunAndSave - This is similar to Save mode except that it allows for the flexibility to execute an assembly even before saving it to the disk.

Implementation

We will start by creating a new Visual C# Windows Application named DynamicAppl as shown in the following screenshot:

We will split the example into two parts. In the first part of this example, we will create an assembly dynamically, add a module to it, create a class called DynamicClass (that has a constructor and a method named DynamicMethod ), and then invoke the Assembly.

In the second part of this example, we will see how to save the dynamic assembly onto the disk. We will then invoke this saved assembly's methods and test its functionality. Finally we will see how the saved assembly looks like by dissecting it with ILDASM utility.

Let us start by considering the first part of the example.

Here we import all the required namespaces:

using System;
using System.Drawing;
using System.Collections;
using System.ComponentModel;
using System.Windows.Forms;
using System.Threading;

To take advantage of the reflection emitting services provided by the .NET Framework, we import the following two namespaces:

using System.Reflection;
using System.Reflection.Emit;

To the form, we add four command buttons named btnCreateDynamic , btnInvokeAssembly , btnInvokeSaved , and btnSaveDynamic . After adding the command buttons, the form should look like the this:

The AddItem method is a utility method that adds an item to the listbox named lstMessages . It is implemented as follows:

private void AddItem(string
            sMessage)
{
lstMessages.Items.Add(sMessage);
}

In the CreateType() function, we create an assembly dynamically (without saving it to the disk) and then return the type that was created dynamically.

protected Type CreateType()
{
AppDomain curAppDomain = Thread.GetDomain();
AddItem("Found current application domain");
// Create a simple name for the Assembly.
AssemblyName assemblyName = new AssemblyName();
assemblyName.Name = "CSharpTodayAssembly";
// Create the dynamic oAssembly.
AssemblyBuilder assemblyBuilder = curAppDomain.
DefineDynamicAssembly(assemblyName, AssemblyBuilderAccess.Run);

AddItem("Assembly creation succeeded");
//Create a dynamic module named DynamicModule in the assemblyBuilder.
ModuleBuilder moduleBuilder = assemblyBuilder.
DefineDynamicModule("DynamicModule");
AddItem("Module creation succeeded");
// Define a public class named "DynamicClass" in the assemblyBuilder.
TypeBuilder dynamicClassObject = moduleBuilder.DefineType("DynamicClass", TypeAttributes.Public);

// Define a private String field named "Message" in the type.
FieldBuilder fieldBuilder = dynamicClassObject.DefineField("Message", Type.GetType("System.String"),
FieldAttributes.Private);

// Create the constructorBuilder.
Type[] constructorArgs = new Type[1];
constructorArgs[0] = Type.GetType("System.String");
ConstructorBuilder constructorBuilder = dynamicClassObject.DefineConstructor(MethodAttributes.Public,
CallingConventions.Standard, constructorArgs);

//Generate IL for the method. The constructorBuilder calls its superclass
// constructorBuilder. The constructorBuilder stores its argument in the private field.
ILGenerator constructorILGenerator = constructorBuilder.GetILGenerator();
constructorILGenerator.Emit(OpCodes.Ldarg_0);
Type objectClass = Type.GetType("System.Object");

ConstructorInfo constructorBuilderInfo = objectClass.GetConstructor(new Type[0]);
constructorILGenerator.Emit(OpCodes.Call, constructorBuilderInfo);
constructorILGenerator.Emit(OpCodes.Ldarg_0);
constructorILGenerator.Emit(OpCodes.Ldarg_1);
constructorILGenerator.Emit(OpCodes.Stfld, fieldBuilder);
constructorILGenerator.Emit(OpCodes.Ret);

MethodBuilder methodBuilder = dynamicClassObject.DefineMethod("DynamicMethod", MethodAttributes.Public,
Type.GetType("System.String"), null);

// Generate IL for DynamicMethod.
ILGenerator methodIL = methodBuilder.GetILGenerator();
methodIL.Emit(OpCodes.Ldarg_0);
methodIL.Emit(OpCodes.Ldfld, fieldBuilder);
methodIL.Emit(OpCodes.Ret);

// Bake the DynamicClass.
return dynamicClassObject.CreateType();
}

Let us walk through the above code.

We start by instantiating an AppDomain object from the current domain. Every application that is created using the .NET Framework runs in an AppDomain that provides an isolated environment for the application to execute:

AppDomain curAppDomain = Thread.GetDomain();
AddItem("Found current application domain");

Then we instantiate an AssemblyName object by using the new keyword. Once we create a new instance of an AssemblyName object, we then set the Name property to an appropriate value. This assembly name is used by the assembly cache manager to retrieve information about the assembly.

// Create a simple name for the Assembly.
AssemblyName assemblyName = new AssemblyName();
assemblyName.Name = "CSharpTodayAssembly";

Once we create the AssemblyName object, the next step is to create the new assembly by calling the AppDomain.DefineDynamicAssembly . We pass the Assembly name and the mode in which the assembly will be accessed to this method as arguments. In this case, since we only want to create a transient assembly (that is not saved to the disk), we pass the enum constant AssemblyBuilderAccess.Run .

// Create the dynamic oAssembly.
AssemblyBuilder assemblyBuilder = curAppDomain.
DefineDynamicAssembly(assemblyName, AssemblyBuilderAccess.Run);

AddItem("Assembly creation succeeded");

The DefineDynamicAssembly method returns an AssemblyBuilder object, which we use for constructing the different elements of the assembly. Now that we have defined the assembly, let us create a dynamic module and add it to the assembly. We begin by calling the Assembly.DefineDynamicModule method to create a ModuleBuilder object:

//Create a dynamic module named DynamicModule in the assemblyBuilder.
ModuleBuilder moduleBuilder = assemblyBuilder.
DefineDynamicModule("DynamicModule");
AddItem("Module creation succeeded");

We then call the DefineType method of the ModuleBuilder object to create a TypeBuilder object, passing to it the name of the type and the attributes used to define it. Since we want to create a type called DynamicClass , we pass DynamicClass as the first argument. We want the scope of the DynamicClass class to be Public, so we pass TypeAttributes.Public as the second argument.

// Define a public class named "DynamicClass" in the assemblyBuilder.
TypeBuilder dynamicClassObject = moduleBuilder.DefineType("DynamicClass", TypeAttributes.Public);

Now we will add a private field of type string called Message to the DynamicClass type by invoking the DefineField method of the DynamicClass type, which we created in the previous step:

// Define a private String field named "Message" in the type.
FieldBuilder fieldBuilder = dynamicClassObject.DefineField("Message", Type.GetType("System.String"),
FieldAttributes.Private);

In this step, we will add a constructor that takes a string as an argument to our DynamicClass , and set the visibility of the constructor to public.

// Create the constructorBuilder.
Type[] constructorArgs = new Type[1];
constructorArgs[0] = Type.GetType("System.String");
ConstructorBuilder constructorBuilder = dynamicClassObject.DefineConstructor(MethodAttributes.Public,
CallingConventions.Standard, constructorArgs);

Now that we have defined the constructor, let us add code to it. We first get reference to the ILGenerator by invoking the GetILGenerator method of the ConstructorBuilder object. Once we get reference to the ILGenerator object, we can directly emit MSIL code into the constructor method using the Emit method of the ILGenerator . This emitted MSIL code is placed into the MSIL stream, which is then given as an input to the JIT Compiler:

//Generate IL for the method. The constructorBuilder calls its superclass
// constructorBuilder. The constructorBuilder stores its argument in the private field.
ILGenerator constructorILGenerator = constructorBuilder.GetILGenerator();
constructorILGenerator.Emit(OpCodes.Ldarg_0);
Type objectClass = Type.GetType("System.Object");

In the following lines written using MSIL, we indicate that the value passed to the constructor should be assigned to the private string field that we defined in the previous step:

ConstructorInfo constructorBuilderInfo = objectClass.GetConstructor(new Type[0]);
constructorILGenerator.Emit(OpCodes.Call, constructorBuilderInfo);

By using Ldarg_x (where x specifies the number of the argument to be loaded), we can load the arguments onto the stack. For example, the following line of code is used to load the argument number 0 into the stack that is required for invoking the constructor of the base class System .object:

constructorILGenerator.Emit(OpCodes.Ldarg_0);

Here, we load the argument 1 onto the stack to enable it to be used in the assignment operation in the next statement:

constructorILGenerator.Emit(OpCodes.Ldarg_1);

The following statement stores the value that is passed to the constructor into the variable represented by the FieldBuilder object:

constructorILGenerator.Emit(OpCodes.Stfld, fieldBuilder);

Using OpCodes.Ret , we can signal the end of the constructor and return control to the caller:

constructorILGenerator.Emit(OpCodes.Ret);

In this step, we create a method named DynamicMethod by invoking the DefineMethod of the DynamicClass object. We also indicate that the scope of this method is public and the return value of the method is string. The null in the fourth parameter indicates that DynamicMethod does not take any parameters:

MethodBuilder methodBuilder = dynamicClassObject.DefineMethod("DynamicMethod",
MethodAttributes.Public, Type.GetType("System.String"), null);

In the following lines written using MSIL, we specify that the DynamicMethod method return the value stored in the private string field:

// Generate IL for DynamicMethod.
ILGenerator methodIL = methodBuilder.GetILGenerator();
methodIL.Emit(OpCodes.Ldarg_0);
methodIL.Emit(OpCodes.Ldfld, fieldBuilder);
methodIL.Emit(OpCodes.Ret);

Finally, we create the Type by invoking the CreateType method of the TypeBuilder object:

// Bake the DynamicClass and return it to the caller.
return dynamicClassObject.CreateType();
}

Finally, we have a dynamically created class named DynamicClass , which consists of a private string field, a constructor, and a public method named DynamicMethod . Now, let us test the dynamically constructed DynamicClass by creating an instance of it and then invoking the DynamicMethod .

To do this, all we have to do is to invoke our CreateType method, get a DynamicClass type, create an instance of it using the CreateInstance method of Activator class, and then call the InvokeMember method of the instance:

private void btnInvokeAssembly_Click(object
            sender, System.EventArgs e)
{
Type type =
            CreateType();
AddItem("Found DynamicClass");
Object[]
            constructorArgs = new Object[1];
String sMessage = " This
            message is from the DynamicClass object ";
constructorArgs[0] =
            sMessage;
Object dynamicClassObject =
            Activator.CreateInstance(type,constructorArgs);
AddItem("Instantiated
            DynamicClass");
//Invoke the "DynamicMethod" method of the
            "DynamicClass" class.
AddItem("Invoking
            DynamicMethod");
Object obj = type.InvokeMember("DynamicMethod",
            BindingFlags.InvokeMethod, null, dynamicClassObject,
new
            Object[0]);
String sReturnMessage = obj.ToString();
AddItem("DynamicMethod returned: " + sReturnMessage);
AddItem("...");
}

In this part of the article, we will see how to create an assembly and then save it to disk in the form of a PE file. The implementation of the CreateAndSaveType method is not much different than the CreateType method except for the difference that here we additionally provide the plumbing that enables us to save the dynamically generated assembly:

private void CreateAndSaveType()
{
AppDomain curAppDomain = Thread.GetDomain();
AddItem("Found current application domain");
// Create a simple name for the Assembly.
AssemblyName assemblyName = new AssemblyName();
assemblyName.Name = "CSharpTodayAssembly";
// Create the dynamic oAssembly.
AssemblyBuilder assemblyBuilder = curAppDomain.
DefineDynamicAssembly(assemblyName, AssemblyBuilderAccess.Save);

AddItem("Assembly creation succeeded");
//Create a dynamic module named DynamicModule in the assemblyBuilder.
ModuleBuilder moduleBuilder = assemblyBuilder. DefineDynamicModule("DynamicModule",
"DynamicModule.dll");
AddItem("Module creation succeeded");
// Define a public class named "DynamicClass" in the assemblyBuilder.
TypeBuilder DynamicClassObject = moduleBuilder.DefineType("DynamicClass", TypeAttributes.Public);

// Define a private String field named "Message" in the type.
FieldBuilder fieldBuilder = DynamicClassObject.DefineField("Message", Type.GetType("System.String"),
FieldAttributes.Private);

// Create the constructorBuilder.
Type[] constructorArgs = new Type[1];
constructorArgs[0] = Type.GetType("System.String");
ConstructorBuilder constructorBuilder = DynamicClassObject.DefineConstructor(MethodAttributes.Public,
CallingConventions.Standard, constructorArgs);

//Generate IL for the method. The constructorBuilder calls its superclass
// constructorBuilder. The constructorBuilder stores its argument in the private field.
ILGenerator constructorILGenerator = constructorBuilder.GetILGenerator();
constructorILGenerator.Emit(OpCodes.Ldarg_0);
Type oObjectClass = Type.GetType("System.Object");

ConstructorInfo constructorBuilderInfo = oObjectClass.GetConstructor(new Type[0]);
constructorILGenerator.Emit(OpCodes.Call, constructorBuilderInfo);
constructorILGenerator.Emit(OpCodes.Ldarg_0);
constructorILGenerator.Emit(OpCodes.Ldarg_1);
constructorILGenerator.Emit(OpCodes.Stfld, fieldBuilder);
constructorILGenerator.Emit(OpCodes.Ret);

MethodBuilder methodBuilder = DynamicClassObject.DefineMethod("DynamicMethod", MethodAttributes.Public,
Type.GetType("System.String"), null);

// Generate IL for DynamicMethod.
ILGenerator methodIL = methodBuilder.GetILGenerator();
methodIL.Emit(OpCodes.Ldarg_0);
methodIL.Emit(OpCodes.Ldfld, fieldBuilder);
methodIL.Emit(OpCodes.Ret);

// Bake the class DynamicClass.
DynamicClassObject.CreateType();

// Save the assemblyBuilder.
assemblyBuilder.Save("CSharpTodayAssembly.dll");
AddItem("Assembly Saved");
AddItem("....");
}

In the above method, we will consider only the lines of code that are differnet from the CreateType method.

At the time of creating the dynamic assembly, we pass the enum constant AssemblyBuilder.Save to the DefineDynamicAssembly method to ensure that the assembly can be saved to the disk:

// Create the dynamic Assembly.
AssemblyBuilder assemblyBuilder = curAppDomain.
DefineDynamicAssembly(assemblyName, AssemblyBuilderAccess.Save);

Since we want to persist the dynamically created module to the disk, we need to specify the name of the file under which this module can be stored:

//Create a dynamic module named DynamicModule in the assemblyBuilder.
ModuleBuilder moduleBuilder = assemblyBuilder.
DefineDynamicModule("DynamicModule","DynamicModule.dll");

Here we create a Type object for the class by i nvoking the CreateType method:

// Bake the class DynamicClass.
DynamicClassObject.CreateType();

In this step, we save the generated assembly to the disk using the Save method of AssemblyBuilder object.

// Save the assemblyBuilder.
assemblyBuilder.Save("CSharpTodayAssembly.dll");

Now that we have created a persisted assembly, let us dissect the assembly by using the ILDASM utility. If you view the saved assembly using the utility, it looks like this:

As you can see from the above screenshot, the dynamically generated class consists of a private field named Message , a Constructor , and a method named DynamicMethod .

When you double-click on the method DynamicMethod , you get a screen that is similar to the following:

The above screenshot shows the IL generated for the DynamicMethod . Now that we have seen the steps involved in creating a persistable assembly, let us take a step forward and understand the procedures to be followed for invoking this saved assembly.

We write the assembly invocation code in the Click event of the button named bntInvokeSaved :

private void bntInvokeSaved_Click(object
            sender, System.EventArgs e)
{
//Find the "DynamicClass"
            class
Assembly assembly =
            Assembly.LoadFrom("CSharpTodayAssembly.dll");
Type[]
            arrTypes;
arrTypes = assembly.GetTypes();
foreach(Type type in
            arrTypes)
{
if (type.Name ==
            "DynamicClass")
{
AddItem("Found DynamicClass");
Object[]
            constructorArgs = new Object[1];
String sMessage = " This
            message is from the DynamicClass object ";
constructorArgs[0] =
            sMessage;
Object dynamicClassObject =
            Activator.CreateInstance(type,constructorArgs);
AddItem("Instantiated
            DynamicClass");
//Invoke the "DynamicMethod" method of the
            "DynamicClass" class.
AddItem("Invoking
            DynamicMethod");
Object obj = type.InvokeMember("DynamicMethod",
            BindingFlags.InvokeMethod, null, dynamicClassObject, new
            Object[0]);
String sReturnMessage = obj.ToString();
AddItem("DynamicMethod returned: " + sReturnMessage);
}
}
AddItem("...");
}

In this step, we instantiate the assembly object by invoking the static method LoadFrom of AssemblyName class, passing to it the name of the dll that contains the assembly:

Assembly assembly = Assembly.LoadFrom("CSharpTodayAssembly.dll");
Type[] arrTypes;

Now we get reference to all the types in the assembly in an array by calling the GetTypes method of the assembly object. We then loop through elements in the array and check the name of each type object in the array, and if the name is equal to DynamicClass , we identify that this is the class we created dynamically in the previous step:

arrTypes = assembly.GetTypes();
foreach(Type type in arrTypes)
{
if (type.Name == "DynamicClass")
{
AddItem("Found DynamicClass");
Object[] constructorArgs = new Object[1];

In this step, we create an instance of the DynamicClass , passing to its constructor the string value contained in the variable sMessage :

String sMessage = " This message is from the
DynamicClass object ";
constructorArgs[0] = sMessage;
Object dynamicClassObject = Activator.CreateInstance(type,constructorArgs);

AddItem("Instantiated DynamicClass");
//Invoke the "DynamicMethod" method of the "DynamicClass" class.
AddItem("Invoking DynamicMethod");

Now we invoke our DynamicMethod by calling the InvokeMember method passing to it all the required parameters. The value returned by the DynamicMethod is stored in the variable obj and then displayed in the list box:

Object obj =
            type.InvokeMember("DynamicMethod", BindingFlags.InvokeMethod, null,
dynamicClassObject, new Object[0]);
String sReturnMessage =
            obj.ToString();
AddItem("DynamicMethod returned: " +
            sReturnMessage);
}
}
AddItem("...");

If you run the above application, you get an output that is similar to the following:

The above screenshot is captured after clicking the right-side command buttons one by one from top to bottom.

When do we use Dynamic Assemblies

Now that we have discussed enough of the implementation of dynamic assemblies, let us try to understand the real time situations where dynamic assemblies can be really useful:

  • Useful in situations where the exact structure of an assembly is not known at design-time. For example, if your application wants to work with multiple unknown servers and needs to generate assemblies dynamically based on the input parameters, then you need to use dynamic assemblies. This kind of arrangement provides a lot of flexibility.
  • Can also be useful in situations when we try to execute script in web browsers.
  • If you are writing your own compiler, you may need to emit metadata and MSIL at runtime based on the supplied input. Dynamic assemblies are very useful in those situations as well.

Prerequisites

To execute the code that is available for download, you need to have the following software installed on your machine:

  • Windows 2000 Professional/Server
  • Visual Studio.NET

Summary

In this article, we have seen the creation of Dynamic Assemblies in the .Net framework and also understood how easy it is to leverage them to create flexible and powerful applications.

Article Information
Author | Thiru Thangarathinam
Technical Editor | Adam Ryland
Project Manager | Helen Cuthill
Reviewers | Saurabh Nandu, David Schultz

If you have any questions or comments about this article, please contact the technical editor.


COM+ Transactions in Managed Environment

COM+ Transactions in Managed Environment
http://www.csharptoday.com/content.asp?id=1826
Article Rating: 4.5
Published on February 26, 2002
Abstract
Transactions are the fundamental unit of processing in applications, and just about any operation in an application can be considered as a transaction. With distributed applications becoming pervasive, it is necessary to have the tools and techniques to create, destroy, and monitor transactions that might be distributed across multiple servers. In addition, having these tools and techniques available within a unifying architecture such as the .NET Framework makes it easy for us to develop distributed applications. In this article, Thiru explains the Transactional support provided by COM+ through a set of managed classes that form an integral part of the .NET Framework.
Article

Transactions are the fundamental unit of processing in applications, and just about any operation in an application can be considered as a transaction. With distributed applications becoming pervasive, it is necessary to have the tools and techniques to create, destroy, and monitor transactions that might be distributed across multiple servers. In addition, having these tools and techniques available within a unifying architecture such as the .NET Framework makes it easy for us to develop distributed applications. In this article, Thiru explains the Transactional support provided by COM+ through a set of managed classes that form an integral part of the .NET Framework.

Before we go on to take a look at the transactional support provided by COM+, let us briefly define the following terms: Transactions, and Distributed Transactions.

A Transaction can be considered as a unit of work. Transaction processing is easy as long as the transaction involves only a single database. Unfortunately, most companies don't store their data in a single database. In many cases, the information will be spread out over many databases. For example, an online retailer might have a database of customer account information that is stored in a SQL Server database at the corporate office, but order fulfillment information may be stored in a Oracle database at the warehouse of a partner company. When a customer places an order with the online retailer, the customer's account should be debited in the SQL Server database at the corporate office, and an order record should be written to the partner company's Oracle database. So, in this case, the updates need to be performed on both of these databases as a single transaction. If either operation fails, then each resource manager will have to roll back its part of the transaction. This type of transaction is called a distributed transaction. Considering the number of entities that may be involved in a distributed transaction, it may get really complex and hence require a lot of code to be written to support it. After seeing the amount of time the developers spent on writing this plumbing code, Microsoft has decided to build the distributed transaction capabilities into COM+ in the form of MSDTC (Microsoft Distributed Transaction Coordinator). As the name suggests, it provides for most of the plumbing code that is required for automating distributed transactions. In this article, we will see how we can take advantage of the built-in transaction capabilities provided by COM+, when creating managed applications in the .NET Framework.

For more information on introduction to transactions, please take a look at the following article from MSDN. Transaction Properties.

System.EnterpriseServices namespace

Component Services in Windows 2000 provides a wealth of enterprise functionality that provides all the plumbing required for creating distributed, transaction based web applications. This enterprise functionality is available to the developers even in the managed world. However in .NET, they have been encapsulated in the form of classes that are not only easy to use but also very flexible. All these classes are contained in the namespace System.EnterpriseServices namespace.

If you have done any MTS/COM+ programming using Visual Studio tools, you will agree that for a COM component to be able to use the services provided by COM+, it needs to have a reference to the ObjectContext interface that encapsulates most of the core functionality of COM+. In .NET framework, we have a class named ContextUtil that acts as a wrapper around the ObjectContext class and it is contained in the System.EnterpriseServices namespace. The methods present in the ContextUtil class are the primary means through which applications written in managed code can communicate with COM+ runtime.

Two of the most important features provided by COM+ are Transaction Processing Monitoring (TPM) and Just In Time (JIT) Activation. COM+ internally maintains two flags namely the Done bit and the Consistency bit, which are used not only to vote for the outcome of a transaction but also to control the deactivation of objects. We can use the following methods to work on the combination of done and consistency bits so that we can indicate to the COM+ runtime about the state of the transaction and whether the object can be deactivated:

  • SetComplete: This method sets both done and consistency flags to true which indicates that the current transaction can be committed and the object deactivated.
  • SetAbort: This method sets the done bit to true and consistency flag to false, which indicates that the current transaction has to be rolled back, and the object deactivated.
  • EnableCommit: Sets the done bit to false and consistency flag to true.
  • DisableCommit: Sets both done and consistency flags to false.

The methods seen above allow us to control both the transactional and the activation behavior of an object. However, there are times where you may want to individually manipulate the done bit and consistency bit so that you can have a finer level of control over the transaction and the object deactivation processes. The ContextUtil class exposes the following two properties through which we can deal with the done bit and consistency bits individually.

  • MyTransactionVote: It has got properties called get and set through which we can either get or set the value of the consistency bit.
  • DeactivateOnReturn: It also has got properties get and set which are used to either get or set the value of the done bit.

There are two different ways by which we can programmatically specify whether we want to commit or rollback the transaction.

  • Using the utility methods (explained above) supplied by ContextUtil class
  • Using the AutoComplete attribute class

We will see both of these methods in action in the later part of this article.

Implementation

We will split this article into three parts:

  • In the first part, we will create managed C# transactional objects (using Visual C# Class library project) that are designed to run in COM+. We will also create a Windows C# client application to learn how transactional C# objects can be accessed.
  • In the second part, we will create a web service client that will access our transactional objects. While creating this web service client, we will also demonstrate how a transaction initiated by a web service can be used to control the transactional behavior of the objects running in COM+.
  • In the third part, we will create an ASP.NET web form client that invokes the transactional C# objects. Along the way, we will also see for ourselves how the transaction attribute of the Page directive can be used to control transactions.

We start by creating two tables named Emp and Dept in the Sql Server database. We will then programmatically add records to these two tables by using Transactional C# objects named Dept and Emp that are hosted in Component Services explorer.

Implementation of Class Library

Once the project is created, we will change the name of the default class from Class1 to Dept.

In the AssemblyInfo.cs file, we add the entries shown below that are specific to the COM+ services that we want to take advantage of. In the following lines, we specify the name of the COM+ application, the type of COM+ application (in-process or out-of process with the client), and the key file that is used to give our assembly the strong name. It is the strong name of the assembly that allows us to ensure that the assembly is globally unique.

[assembly: ApplicationName("COMServerApp")]
[assembly: ApplicationActivation(ActivationOption.Library)]
[assembly: AssemblyKeyFile("COMServer.key")]

Apart from the above attributes, the System.EnterpriseServices namespace also contains the following attributes that can be used to specify the characteristics of the COM+ application.

Attribute \ Usage

ApplicationName \ Allows us to define the name of the application that the components of the assembly will be installed into.

ApplicationID \ Indicates the GUID that uniquely identifies the COM+ application

ApplicationActivation \ Allows us to specify whether the application is a Server application (that runs in a separate process) or Library application (that runs in the caller's process).

AssemblyKeyFile \ Allows us to specify the key file that is required giving a strong name for the assembly.

Description \ Specifies a description for the application.

The application attributes that we described above are used at the assembly level, and these settings will be used by the CLR to install the objects into the Component Services Explorer. Now that we have seen the attributes required to describe the assembly, let us turn our attention to the process of creating the managed classes that perform the desired operations. We will start by creating a class named Dept .

Implementation of Dept class

The Dept class encapsulates the functionalities that are related to the Dept such as adding a new dept and updating an existing dept and so on. For this article, we will consider only the addition of a new dept. The code for the Dept class is as follows:

[ConstructionEnabled(Default="server=localhost;uid=sa;pwd=;database=ASPToday")]
[Transaction(TransactionOption.Required)]
[JustInTimeActivation]

public
            class Dept : ServicedComponent
{

private string
            constructString = null;

public Dept()
{
//
// TODO:
            Add constructor logic here
//
}

protected override void
            Construct(string constructString)
{
this.constructString =
            constructString;
}

public int AddDept(string deptName,
            string location)
{
try
{
int deptNo;
//Create the
            SQLConnection object passing to it the connection
            string
SqlConnection sqlConn = new
            SqlConnection(constructString);
sqlConn.Open();

//Create
            and set the SqlCommand object
SqlCommand sqlCommand = new
            SqlCommand("AddDept",sqlConn);
sqlCommand.CommandType =
            CommandType.StoredProcedure;

//Add the DeptName
            parameter
SqlParameter paramDeptName = new
            SqlParameter("@DeptName",SqlDbType.VarChar,50);
paramDeptName.Value
            = deptName;
paramDeptName.Direction =
            ParameterDirection.Input;
sqlCommand.Parameters.Add(paramDeptName);

//Add
            the Location parameter
SqlParameter paramLocation = new
            SqlParameter("@Location",SqlDbType.VarChar,50);
paramLocation.Value
            = location;
paramLocation.Direction =
            ParameterDirection.Input;
sqlCommand.Parameters.Add(paramLocation);

//Add
            the DeptNo parameter as the Output parameter
SqlParameter
            paramDeptNo = new
            SqlParameter("@DeptNo",SqlDbType.Int,4);
paramDeptNo.Direction =
            ParameterDirection.Output;
sqlCommand.Parameters.Add(paramDeptNo);

sqlCommand.ExecuteNonQuery();

deptNo
            =
            (int)sqlCommand.Parameters["@DeptNo"].Value;

ContextUtil.SetComplete();
return
            deptNo;

}
catch(Exception
            ex)
{
ContextUtil.SetAbort();
throw
            ex;
}
}

}

Let us try to understand the above lines of code by considering them line-by-line.

Using the ConstructionEnabled attribute at the class level, we can specify a default connection string that will be passed to the object when the client invokes a method of the object. This is made possible because of the Construct method that automatically gets called and supplied with the construction string when a method of the object is invoked.

[ConstructionEnabled(Default="server=localhost;uid=sa;pwd=;database=ASPToday")]

In the code shown below, we decorate the Dept class with the Transaction attribute to specify the transaction which the Dept object will be part of. Due to the transaction attribute Required, our Dept class will run within the scope of the caller's transaction, if the caller of the object is already running in a transaction. Otherwise, the Dept object will create a new transaction for itself and run within the scope of that transaction.

[Transaction(TransactionOption.Required)]

The possible values that can be specified for the TransactionOption enum are:

  • Supported - Indicates that this object will share the transaction of the client, if the client has one.
  • NotSupported - Object is created in a context without any governing transaction.
  • Required - Shares the transaction with the client if the client has one, otherwise it creates a new one for itself.
  • RequiresNew - Creates the object in a new transaction context regardless of the caller's current transaction context state.
  • Disabled - Completely ignores any transaction in the current context.

By specifying the JustInTimeActivation attribute, we ensure that our object is active only when a method of the object is invoked. This ensures that the actual creation of the object takes place only when a method of the object is invoked. As soon as the method has finished its execution, COM+ deactivates the object, thereby releasing all the expensive resources such as connection to the database and so on. Please note that JustInTimeActivation is automatically enabled for managed classes that are configured to use automatic transactions.

[JustInTimeActivation]

In the following line, we derive the Dept class from the ServicedComponent , which allows the Dept object to make use of the services provided by COM+. Serviced Components are generally hosted in COM+ services as in-process servers (in other words, library applications).

public class Dept : ServicedComponent

As we already discussed, the Construct method is called during the creation of the object, and this method is passed a string that is specified either programmatically or through the Component Services Explorer. In the Construct method, we capture the connection string into a module level variable so that it can be used by the AddDept method for establishing connection with the database.

protected override void Construct(string constructString)
{
this.constructString = constructString;
}

In the AddDept method, we execute a stored procedure named AddDept and return the newly created DeptID to the caller.

The code for the stored procedure is as follows:

CREATE Procedure AddDept
(
@DeptName varchar(50),
@Location varchar(50),
@DeptNo int out
)

as
set nocount on
insert into Dept
(DeptName,Location)
values
(@DeptName,@Location)

select @DeptNo = @@identity
GO

As you can see from the above, the AddDept stored procedure simply inserts a record into the Dept table and then returns the newly created identity value to the caller:

public int AddDept(string deptName, string location)
{
try
{
int deptNo;
//Create the SQLConnection object passing to it the connection string

Here we create the SqlConnection object by passing to it the connection string that is used to connect to the database.

SqlConnection sqlConn = new SqlConnection(constructString);

Once we have created an instance of the SqlConnection object, we can then open the connection by invoking the Open method.

sqlConn.Open();

In this step, we create an instance of the SqlCommand object passing to it the name of the stored procedure to be executed and the SqlConnection object as arguments.

//Create and set the SqlCommand object
SqlCommand sqlCommand = new SqlCommand("AddDept",sqlConn);

We then set the CommandType to CommandType.StoredProcedure to indicate that we want to execute a stored procedure.

sqlCommand.CommandType = CommandType.StoredProcedure;

Now that we have created the SqlCommand object, we can add the parameters to the stored procedure by using the following lines of code.

      //Add
            the DeptName parameter
SqlParameter paramDeptName = new
            SqlParameter("@DeptName",SqlDbType.VarChar,50);
paramDeptName.Value
            = deptName;
paramDeptName.Direction =
            ParameterDirection.Input;
sqlCommand.Parameters.Add(paramDeptName);

//Add
            the Location parameter
SqlParameter paramLocation = new
            SqlParameter("@Location",SqlDbType.VarChar,50);
paramLocation.Value
            = location;
paramLocation.Direction =
            ParameterDirection.Input;
sqlCommand.Parameters.Add(paramLocation);

Since the DeptNo is specified as an output parameter in the stored procedure, we set the Direction property to ParameterDirection.Output .

//Add the DeptNo parameter as the Output parameter
SqlParameter paramDeptNo = new SqlParameter("@DeptNo",SqlDbType.Int,4);
paramDeptNo.Direction = ParameterDirection.Output;
sqlCommand.Parameters.Add(paramDeptNo);

Now that we have added all the parameters, we execute the stored procedure by invoking the ExecuteNonQuery method.

sqlCommand.ExecuteNonQuery();

After the stored procedure is executed, we can retrieve the output parameter returned from the stored procedure from the Parameters collection of the SqlCommand object.

deptNo = (int)sqlCommand.Parameters["@DeptNo"].Value;

Now that we have executed the stored procedure successfully, we can commit the transaction. This is accomplished by the SetComplete method of the ContextUtil class. The SetComplete method not only commits the transaction, but also deactivates the object so that the resources held by the object can be released.

ContextUtil.SetComplete();
return deptNo;
}

If any errors occur during the execution of above statements, the control will be transferred to the catch block, where we rollback all of the previously performed database operations by invoking the SetAbort method of ContextUtil class.

   catch(Exception
            ex)
{
ContextUtil.SetAbort();
throw
            ex;
}
}

Implementation of Emp Class

Similar to the Dept class, the Emp class is used to perform operations related to an employee. For this article, we will only consider the addition of a new employee using the Emp class. The code required for implementing the Emp class looks like the following (since most of the code is similar to the Dept class, we will only explain the lines of code that are unfamiliar to us):

[ConstructionEnabled(Default="server=localhost;uid=sa;pwd=;database=ASPToday")]
[Transaction(TransactionOption.Required)]
[JustInTimeActivation]
public class Emp :
            ServicedComponent
{
private string constructString =
            null;

public Emp()
{
//
// TODO: Add constructor
            logic here
//
}

protected override void
            Construct(string constructString)
{
this.constructString =
            constructString;
}

In the AddDept method, we controlled the outcome of the transaction by means of ContextUtil.SetComplete (if everything goes fine) and ContextUtil.SetAbort (if any exception occurs during the execution) methods. We can achieve a similar result by using an attribute named AutoComplete . Instead of us calling the SetComplete and SetAbort methods, we can delegate this responsibility to the AutoComplete attribute, which automatically calls either SetComplete (if no exceptions occur) or SetAbort (if an exception is thrown) depending on the result of the execution. For the AddEmp method, we will use the AutoComplete method.

   [AutoComplete]
public
            int AddEmp(string empName,string empAddress,int
            deptNo)
{
try
{
int empNo;
SqlConnection sqlConn =
            new SqlConnection(constructString);
//Open the Connection to the
            database
sqlConn.Open();

//Create and set the SqlCommand
            object options
SqlCommand sqlComm = new
            SqlCommand("AddEmp",sqlConn);
sqlComm.CommandType =
            CommandType.StoredProcedure;

//Set the EmpName
            parameter
SqlParameter paramName = new
            SqlParameter("@EmpName",SqlDbType.VarChar,50);
paramName.Direction
            = ParameterDirection.Input;
paramName.Value =
            empName;
sqlComm.Parameters.Add(paramName);

//Set the
            Address parameter
SqlParameter paramAddress = new
            SqlParameter("@Address",SqlDbType.VarChar,250);
paramAddress.Direction
            = ParameterDirection.Input;
paramAddress.Value =
            empAddress;
sqlComm.Parameters.Add(paramAddress);

//Set
            the DeptNo parameter
SqlParameter paramDeptNo = new
            SqlParameter("@DeptNo",SqlDbType.Int,4);
paramDeptNo.Value =
            deptNo;
paramDeptNo.Direction =
            ParameterDirection.Input;
sqlComm.Parameters.Add(paramDeptNo);

//Set
            the EmpNo parameter as the Stored Procedure return
            value
SqlParameter paramEmpNo = new
            SqlParameter("@EmpNo",SqlDbType.Int,4);
paramEmpNo.Direction =
            ParameterDirection.ReturnValue;
sqlComm.Parameters.Add(paramEmpNo);

//Execute
            the sql stored procedure
sqlComm.ExecuteNonQuery();
empNo =
            (int)sqlComm.Parameters["@EmpNo"].Value;
return
            empNo;
}
catch(Exception e)
{
throw
            e;
}
}

Implementation of WinForms client application

Now that we have finished coding up the transactional objects, let us test them by writing a simple C# Windows application. For this we will create a new project named COMServerClient as shown below.

Once the project is created, we will add a reference to our COMServer class library by selecting Project -> Add Reference . We also change the name of the default form to frmDeptClient . After adding the required controls, the frmDeptClient form looks like the following.

Now we will import the COMServer namespace by using the following line of code. This allows us to reference the classes in the COMServer namespace without having to qualify them with the name of the namespace.

using COMServer;

In the Click event of the Save command button, we execute the following statements.

private void cmdSave_Click(object sender, System.EventArgs e)
{
int deptNo;
Dept dept = new Dept();
deptNo = dept.AddDept(txtDeptName.Text,txtLocation.Text);
MessageBox.Show("The Dept has been successfully added, Dept No is : " + deptNo.ToString());
}

The first time you execute the above WinForms application, the CLR automatically registers the COMServer assembly with Component Services Explorer. Alternatively, we can also do this by using the Command line utility called Regsvcs.exe . Please take a look at the following link from MSDN for more information on using the Regsvcs utility. .NET Services Installation Tool.

To test the Emp class, we create a new windows form named frmEmpClient.cs that looks something like the following. The code for this windows application is available for download along with the supporting material for this article.

When the user clicks the Save command button, we execute the code shown below:

private void cmdSave_Click(object sender,
            System.EventArgs e)
{
int empNo;
Emp emp = new
            Emp();
empNo = emp.AddEmp(txtEmpName.Text,txtAddress.Text,
            Convert.ToInt32(txtDeptNo.Text));
MessageBox.Show("The Employee
            has been added successfully: Emp Number is : " +
            empNo);
}

In the above windows forms application's code, since we executed only a single stored procedure through an object, there was no way for us to tell if the object was running part of a transaction.

Implementation of client applications

To demonstrate the transactional behavior of the objects, we will create client applications that have the capability to initiate a transaction, and then include our Emp and Dept objects as part of that transaction. This allows us to demonstrate the usefulness of transactional objects and then show how these transactional objects can all be part of a single transaction and hence the operations performed by them will be either committed in its entirety or completely roll backed. For this, we will create two types of client applications.

  • ASP.NET Web Service Application
  • ASP.NET Web Application

We will start by creating an ASP.NET Web Service client application for our Dept and Emp objects.

Implementation of ASP.NET Web Service Client Application

To start with, we will create a new ASP.NET web service named COMTransWebService as shown in the following screenshot:

To learn more about creating web services in Visual Studio.NET, please take a look at Creating Web Services using Visual Studio.NET.

Once the project is created, we will then rename the default web service to TransWebService . To the code-behind file of the web service ( TransWebService.cs ), we will add a new method called SaveDeptEmployees . As the name of the method suggests, this method will invoke the methods of our transactional C# objects (Dept and Emp) to perform the following: Add a new Dept and then add employees to that Dept. By making these operations part of a single transaction initiated by the web service, we can ensure that either the Dept and Employee details are added or nothing is added. The code for the SaveDeptEmployees looks like this.

[WebMethod(false,TransactionOption.Required)]
public bool SaveDeptEmployees(string deptName,string deptLocation,
string empName,string empAddres)
{

try
{
int deptNo;
Dept deptDetails = new Dept();
Emp empDetails = new Emp();
//Add the Dept details to the database
deptNo = deptDetails.AddDept(deptName,deptLocation);

//Add the Employee details to the database
int empNo = empDetails.AddEmp(empName,empAddres,deptNo);
return true;
}
catch(Exception e)
{
throw e;
}
}

In the above lines of code, the attribute WebMethod indicates that the SaveDeptEmployees method is to be exposed as web service and when it is deployed, Visual Studio.NET runtime provides all the plumbing required to make this method discoverable to the clients who wish to use this web service. We also specify that we want our web service to run in a transaction by using the enum constant TransactionOption.Required . We can specify any one of the following values for the transaction attribute.

  • Disabled
  • NotSupported
  • Supported
  • Required
  • RequiresNew

To learn more about using transactions in web services, Please take a look at the article Transactions in Web Service.

[WebMethod(false,TransactionOption.Required)]
public bool SaveDeptEmployees(string deptName,string deptLocation,
string empName,string empAddres)
{

try
{
int deptNo;
Dept deptDetails = new Dept();
Emp empDetails = new Emp();

Now that we have created instances of the Dept and Emp objects, we can now call the AddDept and AddEmp methods to add the dept and employee details to the database.

      //Add
            the Dept details to the database
deptNo =
            deptDetails.AddDept(deptName,deptLocation);
//Add the Employee
            details to the database
int empNo =
            empDetails.AddEmp(empName,empAddres,deptNo);
return
            true;
}
catch(Exception e)
{
throw e;
}
}

Let us test the above code by navigating to the path http://localhost/COMTransWebService/TransWebService.asmx from the browser. As a result, you should see a screen that is something like the following.

Using the above test harness (generated by the ASP.NET), we can test our web service by supplying the required parameters. When we invoke the web service, we get the following screen:

Now if you go back to database, you will find that both the dept and employee details have been added successfully. Let us make a slight modification to our code to see if the Dept and Emp objects are running part of the single transaction initiated by the web service.

In the SaveDeptEmployees method, we will modify the following line of code:

int empNo = empDetails.AddEmp(empName,empAddres,deptNo);

In the above line, we will replace the deptNo variable with the value -100 (assuming that there is no dept with the DeptNo -100 in the Dept table). After modification, it looks like the following.

int empNo = empDetails.AddEmp(empName,empAddres,-100);

Now if you execute the web service, you will get the following exception message saying that there is a Foreign Key Violation. And if you check the database, you will find that no values have been added to either the Dept table or the Emp table. This clearly shows that both the objects are running part of the same transaction initiated by the web service.

Implementation of ASP.NET Web Application

When we created the ASP.NET Web Service client application, we saw how to initiate a transaction from the web service by making use of the Transaction attribute. In the same way, we can create ASP.NET web forms that can initiate a transaction and then have the ability to include other objects in the same transaction. To accomplish this, we need to add an attribute to the Page Directive named transaction . By using this attribute, we can indicate to the ASP.NET runtime whether the page needs to be run in a transaction. The possible values that can be used with the transaction attribute are: Disabled, NotSupported, Supported, Required, and RequiresNew.

To illustrate the usage of the transactions in ASP.NET web forms, let us consider the following code snippet.

In the Page directive, we set the transaction attribute to Required specifying that this ASP.NET page runs in a transaction.

<%@ Page transaction="Required"
            language="c#"
            Codebehind="COMServerClient.aspx.cs"
AutoEventWireup="false"
            Inherits="COMServerASPClient.COMServerClient" %>

In the Click event of the command button, we execute the following lines of code:

private void btnSubmit_Click(object sender, System.EventArgs e)
{
try
{
int deptNo;
Dept deptDetails = new Dept();
Emp empDetails = new Emp();
//Add the Dept details to the database
deptNo = deptDetails.AddDept(txtDeptName.Text,txtLocation.Text);

//Add the Employee details to the database
int empNo = empDetails.AddEmp(txtEmployeeName.Text,txtAddress.Text,deptNo);
lblResult.Text = "Transaction completed succesfully";
}
catch(Exception ex)
{
lblResult.Text = "Transaction failed";
}
}

We get the following as the result of the execution of the above code:

Conclusion

In this article, we have seen how to use the classes supplied by the System.EnterpriseServices namespace to create managed applications that can take advantage of the transactional support provided by COM+ services. We have also seen how easy it is to leverage those transactional objects from ASP.NET and the .NET Framework.

Article Information
Author | Thiru Thangarathinam
Technical Editor | Adam Ryland
Project Manager | Helen Cuthill
Reviewers | Stephen Trudel, Neil Piggot

If you have any questions or comments about this article, please contact the technical editor.


Improved Service Control Manager (SCM), Part 3

Improved Service Control Manager (SCM), Part 3
http://www.csharptoday.com/content.asp?id=1825
Article Rating: not yet rated
Published on February 22, 2002
Abstract
This article presents an alternative implementation of the Windows Service Control Manager (SCM) found under the Control Panel’s Administrative Tools folder. This alternate SCM is implemented using .NET, specifically Windows Forms. This .NET variation of the SCM is not meant to reinvent the wheel, but instead is designed to provide functionality the stock windows SCM does not provide, in particular to give a mechanism was needed to turn off and then turn back on suites of related services. This allows us to free up resources (memory) and remove contentions for certain items (such as the FTP, SMTP and HTTP ports). Part 3 covers using .NET’s interoperability support in order to access Windows SCM functions. This final article installment discusses how to create a class that wraps legacy functions contained in DLL’s.
Article

Thus far (in the first of the three installments - http://www.csharptoday.com/content.asp?id=1798&WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0) we have introduced why we went to all the effort to implement an improved service control manager. This first installment also discussed creating Windows services using Visual Studio .NET, installing services using the infrastructure provided by the .NET Framework, and administering services using classes exposed by the .NET Framework. Given the complexity of the improved SCM application, a usage tutorial was also presented. Furthermore, the first installment in this series reviewed how the application's implementation relies on elements of the System.Collections namespace and uses XML serialization in order to save and restore the state of services and their descendents.

Also discussed in our series of installments was the Windows Forms elements associated with our improved SCM application. This topic was discussed in the second installment of this article (see http://www.csharptoday.com/content.asp?id=1723&WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0). The Windows Forms elements addressed included a discussion of the TreeView and Splitter controls. Also presented was an example of how to use a Form as a dialog and how to retrieve the results of this dialog.

Each technology related to the improved service controller manager has been reviewed at this point, save for one. The lone area left to cover is using .NET's interoperability support in order to access Windows SCM functions. This final article installment discusses how to create a class that wraps legacy functions contained in DLL's. This wrapper class declares the legacy functions as methods that are implemented externally (hence the use of C#'s extern keyword). Custom attributes are used to map between .NET methods and legacy functions. The System.Runtime.InteropServices namespace is responsible for exposing this interoperability support.

With respect to complexity, there are certain caveats when dealing with the Windows service control manager API. A key issue to overcome as part of this exercise is how to access legacy Windows structures using .NET. This is basically a mapping between a legacy structure and .NET class. This mapping is supported by elements of the System.Runtime.InteropServices namespace.

This wrapper class demonstrated is the SWPService class. This class is implemented in the SWPService.cs source file. The SWPService class can be used as a template with respect to logically organized and access legacy Windows functions.

Win32 Interoperability and Windows SCM Functions

Clearly when developing in .NET it is not necessary for a developer to delve into the dark recesses of the Windows API. This delving leads to extra effort with respect to interfacing between .NET and legacy Windows functions. The System.ServiceProcess namespace's ServiceController class allegedly provides enough functionality to manage services. To quote MSDN with respect to the ServiceController class's, "You can then use the class [ ServiceController ] to start, stop, and otherwise manipulate the service." For the MSDN text in full, see:

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/cpref/html/frlrfSystemServiceProcessServiceControllerClassTopic.asp&WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0

Notice how the previous quote makes no mention of changing how a service starts (manual, automatic or disabled), the name of the service, the display name of the service or how a service handles errors. These properties of a service are set at installation using the properties of the ServiceInstaller class. One of these properties, StartType, is of type ServiceStartMode and can be set to: Automatic, Disabled or Manual. To quote MSDN with respect to the StartType property, "You cannot change ServiceInstaller property values after installation. To change the StartType, you either have to uninstall and reinstall your service, or manually change the setting using the Service Control Manager." For the MSDN text in full, see: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/cpref/html/frlrfSystemServiceProcessServiceInstallerClassStartTypeTopic.asp

Uninstalling and reinstalling seems a bit of overkill especially given that the installers for various services might not be readily available. So our alternate SCM has no mechanism within the .NET Framework that can be used to alter how a service starts. For this reason we need to regress and interface with Windows SCM functions using elements of the System.Runtime.InteropServices namespace.

In order to retrieve information such as service start type, the following sequence of legacy windows functions should be executed:

  • OpenSCManager - this function returns a handle to the service control manager.
  • OpenService - this function takes a SCM handle and the name of a service and returns a handle to a specific service.
  • QueryServiceConfig - takes as a parameter a service-specific handle, and retrieves a Windows structure, QUERY_SERVICE_CONFIG, that contains service-specific information such as service type, how a service starts, the name of the service, the display name of the service, and how a service handles errors.
  • CloseServiceHandle - use this function to close the service-specific handle.
  • CloseServiceHandle - use this function to close the handle to the SCM.

The sequence used to set service-specific information is identical to the previous sequence, save that the ChangeServiceConfig function is called to set service-specific information. This is as opposed to calling QueryServiceConfig to query service-specific information.

CloseServiceHandle: Simple Interoperability with the Windows API

The SWPService.cs source file contains the SWPService class. This class exposes the previously reviewed Windows functions used to administer services. At the top of the SWPService.cs source file is the following using statement which exposes the classes used by .NET in order to interoperate with legacy functions such as those exposed by the Windows API:

using System.Runtime.InteropServices;

Once method exposed by the SWPService class is as follows:

[DllImport("Advapi32.dll")]
private static extern bool CloseServiceHandle(IntPtr hSCObject);

The lone parameter to the CloseServiceHandle method is a IntPtr structure. This structure is found in the System namespace. The IntPtr structure is used to specify a pointer or a handle in a platform-specific manner. In the case of this method, the IntPtr structure corresponds to a handle to be closed (a handle to the SCM or a handle to a specific service). In order to understand the platform-specific nature of the IntPtr structure, consider Win32 versus Win64. Under Win32, a pointer/handle represented by an IntPtr structure would be 32-bits, while under Win64 a pointer/handle represented by an IntPtr structure would be 64-bits.

The CloseServiceHandle method previously demonstrated is specified as extern. This means that it is implemented externally (as in implemented by the Windows). The DllImport attribute specifies where the CloseServiceHandle method's implementation is found, namely in the Advapi32.dll (the CloseServiceHandle function is within this DLL). This knowledge did not require a crystal ball. Simply looking up the CloseServiceHandle method in MSDN reveals that this method is found in the Advapi32.dll system DLL.

OpenSCManager: Intermediate Interoperability with the Windows API

As examples go, the CloseServiceHandle method was selected because it is extremely simple A slightly more complicated function to interoperate with is OpenSCManager. This function retrieves a handle to the SCM which is subsequently closed by CloseServiceHandle. The OpenSCManager method of the SWPService class is used for interoperation with the OpenSCManager function. This method is implemented as follows:

[DllImport("Advapi32.dll",
CharSet=CharSet.Unicode,
SetLastError=true)]
private static extern IntPtr OpenSCManager(string machineName,
string databaseName,
SCMAccess desiredAccess);

As before, the OpenSCManager method is specified as extern because it is implemented externally. The DllImport attribute indicates that the OpenSCManager function is contained in the Advapi32.dll system DLL. The DllImport attribute actually corresponds to the DllImportAttribute class found in the System.Runtime.InteropServices namespace. Understanding how to use the DllImport attribute is a matter of understanding the DllImportAttribute class. For example, the constructor for the DllImportAttribute class is defined as follows:

public DllImportAttribute(string dllName);

It should be clear based on this constructor how the Advapi32.dll is specified as the DLL in which OpenSCManager is contained (the DLL's name is the parameter to the constructor). Two fields of this DllImportAttribute class are also demonstrated in the prototype for OpenSCManager : SetLastError and CharSet. The SetLastError field is defined as follows:

public bool SetLastError;

When the value of DllImportAttribute 's SetLastError is set to true, the function called (OpenSCManager) sets the value of "last error" using the Windows SetLastError function. Since the SetLastError property in the previous code snippet is set to true, the alternate SCM application can call (in the event of an error) the Marshal class's GetLastWin32Error in order to retrieve the value of last error. The Marshal class provides a veritable arsenal of methods and fields that aid .NET applications when interoperating with legacy functions or COM objects. Unsurprisingly, the Marshal class is found within the System.Runtime.InteropServices namespace.

The other field of the DllImportAttribute class set in our previous code snippet was CharSet (set as part of the DllImport attribute). The prototype for the CharSet field is as follows:

public CharSet CharSet;

The CharSet field is a CharSet enumeration. The CharSet enumeration is a member of the System.Runtime.InteropServices namespace and the values for this enumeration include:

  • Ansi -- marshal strings as one-byte ANSI characters
  • Unicode -- marshal strings as two-byte Unicode characters
  • Auto -- marshal strings based on the underlying operating system. For Windows 98/ME, marshaling is performed using ANSI characters. For Windows NT/2000/XP marshaling is performed as the more readily internationalized Unicode characters.

Recall that in the previous code snippet, CharSet=CharSet.Unicode. This means that Unicode is used to marshal strings passed to the legacy function. Given that Windows services only run on NT/2000/XP and these operating systems are natively Unicode, setting CharSet to CharSet.Unicode was (to quote Spock) logical. The default value of the CharSet field is CharSet.Ansi. Legally it is possible to pass ANSI strings to the likes of OpenSCManager and other functions exposed by Windows NT/2000/XP. The reason for this is that these operating systems implement most functions in Unicode but provide an ANSI implementation. The ANSI implementation incurs additional overhead in that (under the covers) it must convert ANSI strings to/from Unicode. Yes the reason CharSet=CharSet.Unicode is specified is to improve performance given that services only run on operating systems that are natively Unicode.

In actuality, Windows exposes two version of the OpenSCManager function: OpenSCManagerW, which is the Unicode implementation (as denoted by the 'W' suffix), and OpenSCManagerA which is the ANSI implementation (as denoted by the 'A' suffix). The 'W' and 'A' suffixes represent the mechanism used by every Windows function to distinguish between Unicode and ANSI versions. When coding in C++, macros (#define 's) typically hid these suffixes from the developer. When coding with .NET and using the DllImportAttribute class, the 'W' version of a function (e.g. OpenSCManagerW) is called if the CharSet field is set to CharSet.Unicode, and the 'A' version of a function (e.g. OpenSCManagerA) is called if the CharSet field is set to CharSet.Ansi. Setting CharSet to CharSet.Auto would call the 'W' or 'A' version of a function based on the underlying operating system.

Our example did not fully exercise all the fields exposed by the DllImport attribute via its implementation, the DllImportAttribute class. The fields not presented thus far include:

  • CallingConvention -- lets the calling convention of the function be specified. This enumeration can be set to values such as Cdecl, StdCall and Winapi. The default value of this field is StdCall. Windows function use the StdCall calling convention by default and hence this field was not set in our example.
  • EntryPoint -- specifies the actual name (type, string) of the function name mapped to .NET method. For our example, this field was not specified because DllImportAttribute mapped the OpenSCManager method name to the OpenSCManagerW function.
  • PreserveSig - by default this field is set to true which indicates that the signature of the method (the form the prototype takes) corresponds to the signature of the underlying function. Setting this value to false causes the causes the function signature to be altered in order to provide consistent error handling (see MSDN for more information).
  • ExactSpelling - if this Boolean field is set to true (i.e. use the exact spelling of the function name) then no 'A' or 'W' suffix is added to the function name as specified by the value of the CharSet field. The default value for this field is false which means to handle the 'A' and 'W' prefixes under the covers, and hence it was not set as part of our example.

The documentation as of the MSDN released with the first version Visual Studio .NET is not correct with respect to how it describes the ExactSpelling field. The write up specified in this article is correct and has been tested programmatically to be accurate. The write up in the article has been verified against the version of MSDN initially released with Visual Studio .NET.

Thus far we have explained how OpenSCManager is called, but not what it does. Recall that the prototype took the following form:

IntPtr OpenSCManager(string machineName,
string databaseName,
SCMAccess desiredAccess);

The return value for the OpenSCManager function is of type IntPtr which corresponds to a handle to the service control manager. The machineName parameter corresponds to the machine name on which the service resides. A value of null indicates that a service is to be opened on the local host. The databaseName parameter corresponds to a specific SCM database. Setting this value to null causes the database associated with currently active services to be opened (which is the behavior desired by the alternate SCM application).

The desiredAccess parameter corresponds to the type of access the SCM affords the handle returned. The permissible values for this parameter are specified in the legacy header file, WinSvc.h, and are documented in MSDN. This header file can be found within the Visual Studio .NET installation under:

\Program Files\Microsoft Visual Studio .NET\Vc7\PlatformSDK\Include

An example of a macro found in the WinSvc.h header file is SC_MANAGER_ENUMERATE_SERVICE. The SC_MANAGER_ENUMERATE_SERVICE macro indicates the handle can enumerate over the services managed by the SCM. Within the C# source file, SWPService.cs, the SCMAccess enumeration contains the values that can be permissibly passed to the desiredAccess parameter. Each value in this enumeration corresponds to a value in the WinSvc.h header file. The SCMAccess enumeration is defined as follows:

// Taken from WinSvc.h
enum SCMAccess :int
{
SC_MANAGER_CONNECT = 0x0001,
SC_MANAGER_CREATE_SERVICE = 0x0002,
SC_MANAGER_ENUMERATE_SERVICE = 0x0004,
SC_MANAGER_LOCK = 0x0008,
SC_MANAGER_QUERY_LOCK_STATUS = 0x0010,
SC_MANAGER_MODIFY_BOOT_CONFIG = 0x0020
}

So after much explanation and a bit of pontification, the SWPService class's OpenSCManager method can be called as follows, where the handle returned is simply able to connect to the SCM database (as specified by SCMAccess.SC_MANAGER_CONNECT):

IntPtr scmHandle =
OpenSCManager(null, null, SCMAccess.SC_MANAGER_CONNECT);

OpenService Function

Within SWPService.cs, the OpenService method provides a conduit through which the OpenService Windows function is accessed. This method is defined as follows:

[DllImport("Advapi32.dll",
CharSet=CharSet.Unicode,
SetLastError=true)]
private static extern IntPtr OpenService(IntPtr hSCManager,
string serviceName,
ServiceAccess desiredAccess);

Based on our discussion of the OpenSCManager method it should be clear what each element of the DllImport attributes represent. The " Advapi32.dll " string specifies the DLL containing the OpenService function, the flavor of the function accessed is Unicode since CharSet=CharSet.Unicode, and the OpenService function sets last error if it encounters an error since SetLastError=true. We also know that since it is the Unicode version of the function, the actually Windows function accessed is OpenServiceW (the function name with a 'W' suffix).

The OpenService returns a handle (type IntPtr) to a specific service contained in the SCM database. This method takes as its first parameter a handle (type IntPtr) previously opened by a call to OpenSCManager. The serviceName parameter represents the name of the service to which a handle will be opened. The ServiceAccess desiredAccess parameter specifies the permissible forms of access to the service. The ServiceAccess type is actually an enumeration declared as part of the SWPService class. This enumeration is defined as follows:

// Taken from WinSvc.h
enum ServiceAccess :int
{
SERVICE_QUERY_CONFIG = 0x0001,
SERVICE_CHANGE_CONFIG = 0x0002,
SERVICE_QUERY_STATUS = 0x0004,
SERVICE_ENUMERATE_DEPENDENTS = 0x0008,
SERVICE_START = 0x0010,
SERVICE_STOP = 0x0020,
SERVICE_PAUSE_CONTINUE = 0x0040,
SERVICE_INTERROGATE = 0x0080,
SERVICE_USER_DEFINED_CONTROL = 0x0100
}

As the lone comment in the previous code snippet states, the values for each element within the ServiceAccess enumeration were determined based on macros (C++ #define 's) found in the WinSvc.h header file, and of course said macros are discussed in detail in MSDN under the documentation for the OpenService function. An example of call the OpenService method including specifying values for the ServiceAccess parameter is as follows:

IntPtr serviceHandle = OpenService(scmHandle,
serviceName,
ServiceAccess.SERVICE_QUERY_CONFIG);

The type of access, ServiceAccess.SERVICE_QUERY_CONFIG, is the ability to query information with regards to the service. Such information is queried using a function such as QueryServiceConfig. This function exposed via the SWPService class's QueryServiceConfig method.

To retrieve information using the SWPService class's ChangeServiceConfig method (which maps to the ChangeServiceConfig function), the ServiceAccess.SERVICE_CHANGE_CONFIG style of ServiceAccess should be specified when calling OpenService :

serviceHandle = OpenService(scmHandle,
serviceName,
ServiceAccess.SERVICE_CHANGE_CONFIG);

QueryServiceConfig: Structures and Windows Interoperability

In order to query the configuration of a service, the QueryServiceConfig function is called using the SWPService 's class's QueryServiceConfig method. The prototype for this method is as follows:

[DllImport("Advapi32.dll",
CharSet=CharSet.Unicode,
SetLastError=true)]
// pServiceConfig is actually a QUERY_SERVICE_CONFIG structure
unsafe private static extern bool QueryServiceConfig(IntPtr hService,
void *pServiceConfig,
int lBufSize,
int *pBytesNeeded);

As has been the modus operandi thus far, the QueryServiceInterface is marked as extern and uses the DllImport attribute to specify the location of the external implementation. Notice also in the previous prototype that the QueryServiceConfig method is marked as unsafe. The unsafe keyword is used because of the parameters containing pointers which are passed to QueryServiceConfig. For those new to the concept of unsafe, recall that garbage collection coalesces by reference objects. When a "by reference" object is coalesced, its underlying pointer changes. This is why .NET uses reference objects and not pointers to objects. This is also why the keyword unsafe is used when calling methods that take parameters.

When QueryServiceConfig is called, the first parameter is a service handle (IntPtr hService) previously opened. The data retrieved based on this query is returned in a buffer created by the client calling this method (void *pServiceConfig). The size of said buffer is specified int lBufSize and the size the buffer should be is returned by the int *pBytesNeeded parameter.

This method is a classic example of the "Which came first, the chicken or the egg?" quandary. The buffer specified must be large enough to hold the data returned, but how do you know the size of the data returned until you call the method? The remedy for this quandary (aside from fried chicken with an omelette on the side) is to call the QueryServiceConfig method, specifying a buffer set to null and a buffer size set to zero. The QueryServiceConfig method will return false, indicating an error. When the value of last error is examined, a last error value of ERROR_INSUFFICIENT_BUFFER (a value of 122 from winerror.h) will be detected. This value indicates that that buffer was too small. The size required of the buffer is returned via the pBytesNeeded parameter.

The scheme sounds fairly straightforward, but the value of ERROR_INSUFFICIENT_BUFFER must be specified in the managed world of the SWPService class. This is achieved with the WinError enumeration which is declared as follows:

// Taken from winerror.h
public enum WinError :int
{
NO_ERROR = 0,
ERROR_INSUFFICIENT_BUFFER = 122
}

The WinError enumeration demonstrated above also contains the NO_ERROR enumeration value. With respect to the "last error" set by a Windows function, a value of 0 is considered to be the success case (the "no error" case).

It would be possible using the DllImport attribute to specify a method that accesses Windows' GetLastError function (the function that retrieves the value of each thread's "last error"). The Marshal class exposes a static method, GetLastError. This method is already mapped to call Window's GetLastError function.

The basic idea is to execute the code as follows, but this still means in the managed world that a buffer has to be allocated, mapped to a structure, and then freed:

bool result;
int lastError;
int size = 0;

// serviceHandle of type IntPtr already opened by OpenService
result = QueryServiceConfig(serviceHandle, null, 0, &size);
if (!result)
{
lastError = Marshal.GetLastWin32Error();
if ((int)WinError.ERROR_INSUFFICIENT_BUFFER == lastError)
{
// 1) allocate buffer of "size" bytes in size
// 2) Call QueryServiceConfig with correctly sized buffer
// 3) Marshal buffer to appropriate structure,
// QUERY_SERVICE_CONFIG
// 4) Release buffer
}

else
{
return lastError;
}
}

The steps missing in the previous code snippet can be handled as follows:

  • Call the Marshal class's AllocCoTaskMem method to allocate the buffer. Since the buffer is a pointer type it is represented by the IntPtr class. Under the covers, AllocCoTaskMem uses the legacy Windows function CoTaskMemAlloc, and is called as follows:

    IntPtr buffer = Marshal.AllocCoTaskMem(size);

  • Call QueryServiceConfig specifying the buffer and the correct size for the buffer:

    result = QueryServiceConfig(serviceHandle,
    buffer.ToPointer(),
    size,
    &size);

  • Call the Marshal class's Marshal.PtrToStructure method to marshal an unmanaged buffer into a managed class or structure. In the case of our example, the QUERY_SERVICE_CONFIG class was specified as an inner class of SWPService. The QUERY_SERVICE_CONFIG class maps directly to the legacy Windows QUERY_SERVICE_CONFIG structure. The code performing this unmanaged/managed mapping task is as follows:

    QUERY_SERVICE_CONFIG serviceConfig = new QUERY_SERVICE_CONFIG();

    Marshal.PtrToStructure(buffer, serviceConfig);

  • Now that the results of the QueryServiceConfig method are safely in a managed class, the unmanaged buffer can be freed using the Marshal class's FreeCoTaskMem method. Under the covers, FreeCoTaskMem uses the legacy Windows function, CoTaskMemFree, and is called as follows:

    Marshal.FreeCoTaskMem(buffer);

Those interested in seeing the legacy Windows incarnation of the QUERY_SERVICE_CONFIG structure should see the Winsvc.h header file. Within the SWPService class the QUERY_SERVICE_CONFIG inner-class is defined as follows (which directly parallels the declaration of this structure within Winsvc.h):

// Found in Winsvc.h
[StructLayout(LayoutKind.Sequential, CharSet=CharSet.Unicode)]
public class QUERY_SERVICE_CONFIG
{
public ServiceType serviceType = // DWORD in Winsvc.h
ServiceType.SERVICE_NO_CHANGE;
public StartType startType = // DWORD in Winsvc.h
StartType.SERVICE_NO_CHANGE;
public ErrorControlType errorControlType = // DWORD in Winsvc.h
ErrorControlType.SERVICE_NO_CHANGE;
public string lpBinaryPathName = null;
public string lpLoadOrderGroup = null;
public int dwTagId = 0; // DWORD in Winsvc.h
public string lpDependencies = null;
public string lpServiceStartName = null;
public string lpDisplayName = null;

public System.ServiceProcess.ServiceStartMode StartMode
{
get
{
if (((startType & StartType.SERVICE_AUTO_START) != 0) ||
((startType & StartType.SERVICE_BOOT_START) != 0) ||
((startType & StartType.SERVICE_SYSTEM_START) != 0))
{
return System.ServiceProcess.ServiceStartMode.Automatic;
}
if ((startType & StartType.SERVICE_DEMAND_START) != 0)
{
return System.ServiceProcess.ServiceStartMode.Manual;
}

// Do not need if because it is the only case remaining
// if ((startType & StartType.SERVICE_DISABLED) != 0)
// {
return System.ServiceProcess.ServiceStartMode.Disabled;
// }
}
}
}

Each string data member in the QUERY_SERVICE_CONFIG class is initialized to null. The data members which specify an enumeration are all set to -1 (a value named SERVICE_NO_CHANGE). These choices of initial values are pertinent to the ChangeServiceConfig function reviewed later in this section. Each of the aforementioned initial values indicates that settings for a service should not be changed should the data member be passed as a parameter to ChangeServiceConfig.

The previous class is associated with the StructLayout attribute which corresponds to the StructLayoutAttribute class found in the System.Runtime.InteropServices namespace. The StructLayout attribute is used to explicitly dictate the layout of the class of structure associated with the attribute. Under most circumstances, the Common Language Runtime (CLR) dictates class layout, but explicit layout is required when interacting with the Marshal class's Marshal.PtrToStructure method.

The ServiceType, StartType and ErrorControlType types specified in the QUERY_SERVICE_CONFIG class are actually all enumerations. As has been the policy, these enumerations correspond to macros defined in a legacy header file, WinNT.h. The ServiceType, StartType, and ErrorControlType enumerations are defined as follows:

// From WinNT.h
public enum StartType
{
SERVICE_BOOT_START = 0x00000000,
SERVICE_SYSTEM_START = 0x00000001,
SERVICE_AUTO_START = 0x00000002,
SERVICE_DEMAND_START = 0x00000003,
SERVICE_DISABLED = 0x00000004,
SERVICE_NO_CHANGE = -1
}

// From WinNT.h
public enum ErrorControlType
{
SERVICE_ERROR_IGNORE = 0x00000000,
SERVICE_ERROR_NORMAL = 0x00000001,
SERVICE_ERROR_SEVERE = 0x00000002,
SERVICE_ERROR_CRITICAL = 0x00000003,
SERVICE_NO_CHANGE = -1
}

// From WinNT.h
public enum ServiceType
{
SERVICE_KERNEL_DRIVER = 0x00000001,
SERVICE_FILE_SYSTEM_DRIVER = 0x00000002,
SERVICE_ADAPTER = 0x00000004,
SERVICE_RECOGNIZER_DRIVER = 0x00000008,
SERVICE_WIN32_OWN_PROCESS = 0x00000010,
SERVICE_WIN32_SHARE_PROCESS = 0x00000020,
SERVICE_INTERACTIVE_PROCESS = 0x00000100,
SERVICE_NO_CHANGE = -1
}

The StartType enumeration contains the various approaches a service can take to starting (automatically, disabled, manually on demand, etc.). The ServiceType enumeration specifies the type of service. Recall that services include hardware drivers, file system drivers, and Windows processes. The ErrorControlType enumeration specifies how a service responds when encountering an error when starting a service at system startup. See MSDN for an overview of the intricate minutiae associated with this member of the QUERY_SERVICE_CONFIG class.

The string type data members of the QUERY_SERVICE_CONFIG class include lpBinaryPath, which specifies the path where the service's executable resides. The lpLoadOrderGroup specifies in which load order group or service resides (the groups that dictate when a service is loaded). The lpDependencies data member specifies on which services the present service depends. The lpServiceStartName data member contains the service name, and the lpDisplayName data member contains the name displayed in the SCM for the service.

The QUERY_SERVICE_CONFIG class does contain a single property:

public System.ServiceProcess.ServiceStartMode StartMode

The previous property converts between the QUERY_SERVICE_CONFIG class's startMode data member (of type SWPService.StartMode, which reflects how Windows specifies start mode) and the ServiceStartMode enumeration exposed by the System.ServiceProcess namespace (which reflects how .NET specifies start mode). This conversion correlates the Windows style StartType.SERVICE_DEMAND_START start mode with ServiceStartMode.Manual. In a similar mapping, ServiceStartMode.Disabled is equivalent to StartType.SERVICE_DISABLED. The .NET start mode of ServiceStartMode.Automatic is actually equivalent to the following Windows-style start modes: StartType.SERVICE_AUTO_START, StartType.SERVICE_BOOT_START, and StartType.SERVICE_SYSTEM_START.

Clearly the StartMode property serves as a bridge between the Windows API arena exposed by the SWPService class and the .NET world exposed by the System.ServiceProcess namespace.

The SWPService class exposes the SWPQueryServiceConfig method. This method implements all of the steps required to retrieve the QUERY_SERVICE_CONFIG information associated with a specific service (call OpenSCManager to open a handle to the SCM, open a handle to the service using OpenService, retrieve the information using QueryServiceConfig, etc.). The SWPQueryServiceConfig method:

  • Takes an input parameter string serviceName corresponding to the service whose configuration information is to be retrieved.
  • Takes an output parameter, QUERY_SERVICE_CONFIG serviceConfig, which contains the configuration information associated with the specified service.
  • Returns zero on success, or the value returned by Marshal.GetLastWin32Error(); in the event the method encounters an error.

The SWPQueryServiceConfig method is its entirety is as follows, and given the previous discussion, this method should be quite readable:

unsafe public static int
SWPQueryServiceConfig(string serviceName,
out QUERY_SERVICE_CONFIG serviceConfig)
{
serviceConfig = null;
IntPtr scmHandle = new IntPtr(0);
IntPtr serviceHandle = new IntPtr(0);
IntPtr buffer = new IntPtr(0);
bool result;

try
{
scmHandle =
OpenSCManager(null,
null,
SCMAccess.SC_MANAGER_CONNECT);
if (scmHandle.ToInt32() == 0)
{
return Marshal.GetLastWin32Error();
}

serviceHandle = OpenService(scmHandle,
serviceName,
ServiceAccess.SERVICE_QUERY_CONFIG);
if (serviceHandle.ToInt32() == 0)
{
return Marshal.GetLastWin32Error();
}

int lastError;
int size = 0;

result = QueryServiceConfig(serviceHandle,
null,
0,
&size);
if (!result)
{
lastError = Marshal.GetLastWin32Error();
if ((int)WinError.ERROR_INSUFFICIENT_BUFFER == lastError)
{
buffer = Marshal.AllocCoTaskMem(size);

if (null == buffer.ToPointer())
{
return Marshal.GetLastWin32Error();
}

result = QueryServiceConfig(serviceHandle,
buffer.ToPointer(),
size,
&size);
if (!result)
{
return Marshal.GetLastWin32Error();
}

serviceConfig = new QUERY_SERVICE_CONFIG();
Marshal.PtrToStructure(buffer,
serviceConfig);
}

else
{
return lastError;
}
}
}

finally
{
if (0 != serviceHandle.ToInt32())
{
CloseServiceHandle(serviceHandle);
}

if (0 != scmHandle.ToInt32())
{
CloseServiceHandle(scmHandle);
}

if (null != buffer.ToPointer())
{
Marshal.FreeCoTaskMem(buffer);
}
}

return 0;
}

ChangeServiceConfig: Advanced Interoperability with the Windows API

The SWPService class exposes the ChangeServiceConfig method. To the surprise of no one reading this article, the ChangeServiceConfig method corresponds to the ChangeServiceConfig Windows function found in Advapi32.dll. This not-so-mysterious behavior is clearly based on the prototype for the ChangeServiceConfig method found in the SWPService.cs source file:

[DllImport("Advapi32.dll", CharSet=CharSet.Unicode, SetLastError=true)]
unsafe private static extern bool
ChangeServiceConfig(IntPtr hService,
ServiceType dwServiceType,
StartType dwStartType,
ErrorControlType dwErrorControl,
string lpBinaryPathName,
string lpLoadOrderGroup,
int *lpdwTagId,
string lpDependencies,
string lpServiceStartName,
string lpPassword,
string lpDisplayName);

The reason for the previous method being marked as extern is "Same as it ever was" (to quote the Talking Heads). The reason the ChangeServiceConfig method is unsafe is due to the int *lpdwTagId pointer parameter. The return value of this method is a Boolean (true for success and false for error under which circumstances the value of last error is set). The first parameter is a handle to a service opened using the previously discussed OpenService method.

The remaining ten parameters allow the configuration of a service to be changed. Cleary when changing a service configuration it might not be desirable to change all ten parameters. For this reason, each parameter can be passed a value that will change a service's configuration, or each parameter can be passed a specific value indicating that the configuration setting is not to be changed. For the parameters of the enumeration types ServiceType, StartType, and ErrorControlType, the way to turn off setting this configuration values is to specify the following (found in the winsvc.h header file):

#define SERVICE_NO_CHANGE 0xffffffff

Since we are using C#, and not C++ which supports #define -ing specific macro values, we were required to include the value of SERVICE_NO_CHANGE in the previously discussed enumerations. The ServiceType, StartType, and ErrorControlType enumerations each include the ability to not set a configuration value by specifying SERVICE_NO_CHANGE :

    public enum
            StartType
{
// enumeration values same as previous shown
SERVICE_NO_CHANGE = -1 // 0xFFFFFFFF
}

public enum
            ErrorControlType
{
// enumeration values same as previous
            shown
SERVICE_NO_CHANGE = -1 // 0xFFFFFFFF
}

public
            enum ServiceType
{
// enumeration values same as previous
            shown
SERVICE_NO_CHANGE = -1 // 0xFFFFFFFF
}

The reason -1 was used instead of 0xFFFFFFFF is because the later is viewed as an unsigned integer. Since C# supports only signed integer types, we use -1 which in signed terms is the equivalent of 0xFFFFFFFF for unsigned, four-byte integer values.

Each parameter passed to ChangeServiceConfig of type string can also be specified as "do not modify this configuration setting". This is achieved by passing null instead of a string instance. Similarly a null can be passed for the int *lpdwTagId parameter, indicating that the configuration setting for this parameter is not to be retrieved/modified. The lpdwTagId parameter presents a documentation quandary. According to the documentation, this value is an out only parameter but the documentation also says to specify a value of null for this parameter if it is not to be set. How can it be set if it is an out-only parameter? Rather then worry about this inconsistency in the documentation, the code simply passes in null for this parameter because it needs not be retrieved or set by our improved service control manger application.

SWPChangeServiceConfig - a Method for Changing Service Status

Now that we know how to set a service's configuration using ChangeServiceConfig, we should probably use this method to actually set something. For our alternate service manager application the SWPService class exposes the SWPChangeServiceConfig method in order to provide an ability to change the configuration of a service.

unsafe public static int
SWPChangeServiceConfig(string serviceName,
QUERY_SERVICE_CONFIG serviceConfig,
string password)

Just a friendly reminder as to why we had to go through all this trouble -- .NET does not provide this functionality, and our alternative service control manager requires this functionality. The basic idea of the SWPChangeServiceConfig method is to specify a service by name (string serviceName) and a service configuration. The service configuration is contained in a QUERY_SERVICE_CONFIG instance and password. If the strings of the QUERY_SERVICE_CONFIG instance are set to null or are empty (""), then these configuration settings are not modified by SWPChangeServiceConfig. The enumerations within the QUERY_SERVICE_CONFIG instance can be set to SERVICE_NO_CHANGE indicating that these configuration settings are not to be modified. If an error is returned, the value of "last error" will be returned, or a zero (enumeration value WinError.NO_ERROR) indicating success. Yes, in the previous prototype the SWPChangeServiceConfig method was marked as unsafe. This is because SWPChangeServiceConfig calls ChangeServiceConfig which is declared as unsafe.

The SWPChangeServiceConfig method contains a variety of data members that will be passed as pointers to a Windows function. Each data member of type string is passed as a pointer, and the dwTagID data member is passed as a pointer. These pointers can potentially be null. For this reason, each of these data members is assigned to a variable of type IntPtr. If the string is null or empty ("") then the IntPtr is set to null, otherwise the value of the IntPtr is assigned a pointer value. The mechanism used to covert between a string value and a pointer is the Marshal.StringToCoTaskMemUni method. Actually, the StringToCoTaskMemUni method creates a block of unmanaged memory using the COM task allocator, and then copies the contents of the string to this region of unmanaged memory. Once this unmanaged memory is not needed, the Marshal.FreeCoTaskMem method should be called to release said memory. The NullablePointerFromString method has been provided in order to automate the process of converting the string to unmanaged memory blocks. The NullablePointerFromString method is implemented as follows:

    unsafe private
            static IntPtr NullablePointerFromString(string str)
{
if (null
            == str)
{
return new IntPtr(null);
}

if (str.Length
            == 0)
{
return new
            IntPtr(null);
}

else
{
return new IntPtr((void
            *)Marshal.StringToCoTaskMemUni(str));
}
}

The dwTagId data member is similarly assigned to a IntPtr so it can be treated as a pointer. If this data member is zero, then it is not being updated so the IntPtr is assigned a value of null

The basic layout for SWPChangeServiceConfig has already been discussed:

  • Assign the data members of the QUERY_SERVICE_CONFIG class to IntPtr variables so that they can be passed to native Windows calls as pointers.
  • OpenSCManager - retrieve a handle to the service control manager
  • OpenService - use the SCM handle to retrieve a service-specific handle
  • ChangeServiceConfig - specify new configuration settings for the service
  • Free the strings assigned using Marshal.StringToCoTaskMemUni by calling the Marshal.FreeCoTaskMem method.
  • CloseServiceHandle - close the service-specific handle
  • CloseServiceHandle - close the handle to the service control manager

The code associated with the SWPChangeServiceConfig method is as follows:

unsafe public static int
SWPChangeServiceConfig(string serviceName,
QUERY_SERVICE_CONFIG serviceConfig,
string password)
{
IntPtr scmHandle = new IntPtr(0);
IntPtr serviceHandle = new IntPtr(0);
IntPtr pPassword =
NullablePointerFromString(password);
IntPtr lpDisplayName =
NullablePointerFromString(serviceConfig.lpDisplayName);
IntPtr lpBinaryPathName =
NullablePointerFromString(serviceConfig.lpBinaryPathName);
IntPtr lpLoadOrderGroup =
NullablePointerFromString(serviceConfig.lpLoadOrderGroup);
IntPtr lpDependencies =
NullablePointerFromString(serviceConfig.lpDependencies);
IntPtr lpserviceStartName =
NullablePointerFromString(serviceConfig.lpServiceStartName);
IntPtr tag =
NullablePointerFromInt(serviceConfig.dwTagId);

bool result;

try
{
scmHandle =
OpenSCManager(null,
null,
SCMAccess.SC_MANAGER_CONNECT);
if (scmHandle.ToInt32() == 0)
{
return Marshal.GetLastWin32Error();
}

serviceHandle =
OpenService(scmHandle,
serviceName,
ServiceAccess.SERVICE_CHANGE_CONFIG);
if (serviceHandle.ToInt32() == 0)
{
return Marshal.GetLastWin32Error();
}

int dwTagId = serviceConfig.dwTagId;

result = ChangeServiceConfig(serviceHandle,
serviceConfig.serviceType,
serviceConfig.startType,
serviceConfig.errorControlType,
(char *)lpBinaryPathName.ToPointer(),
(char *)lpLoadOrderGroup.ToPointer(),
(int *)tag.ToInt32(),
(char *)lpDependencies.ToPointer(),
(char *)lpserviceStartName.ToPointer(),
(char *)pPassword.ToPointer(),
(char *)lpDisplayName.ToPointer());
if (!result)
{
return Marshal.GetLastWin32Error();
}
}

finally
{
if (null != pPassword.ToPointer())
{
Marshal.FreeCoTaskMem(pPassword);
}

if (null != lpDisplayName.ToPointer())
{
Marshal.FreeCoTaskMem(lpDisplayName);
}

if (null != lpBinaryPathName.ToPointer())
{
Marshal.FreeCoTaskMem(lpBinaryPathName);
}

if (null != lpLoadOrderGroup.ToPointer())
{
Marshal.FreeCoTaskMem(lpLoadOrderGroup);
}

if (null != lpDependencies.ToPointer())
{
Marshal.FreeCoTaskMem(lpDependencies);
}

if (null != lpserviceStartName.ToPointer())
{
Marshal.FreeCoTaskMem(lpserviceStartName);
}

if (0 != serviceHandle.ToInt32())
{
CloseServiceHandle(serviceHandle);
}

if (0 != scmHandle.ToInt32())
{
CloseServiceHandle(scmHandle);
}
}

return 0;
}

Conclusion

The improved service control manager presented in this series of installments represented a fairly complicated .NET Windows Forms application. The technical elements of .NET exploited by this application were quite diverse: collections, XML serialization, using custom attributes, service development, service management, interfacing with legacy Windows functions, and understanding the legacy Windows service control API. The Windows Forms-specific features discussed were also varied in nature, ranging from how to develop a dialog to the usage of the Panel and GroupBox controls. The improved SCM used a dozen different Windows Forms Controls.

Indirectly this article did present a particular coding style philosophy. Within the .NET documentation Microsoft has ceased using Hungarian notation. For those not familiar with this style of coding, variables (amongst other constructs) are prefixed by a set of characters indicating the data type. There are a variety of standards with respect to what prefix maps to what type, but the basics include Booleans prefixed by "b", pointers by "p", and integers by "i". Within this article, class names and certain methods were prefixed by SWP (an abbreviation for my corporation's name). No reader has memorized every class and method with .NET. The SWP prefix is designed to let a reader know that these classes and methods are not part of .NET. Another aid to the programs readability was supplied because each data member used by a class was prefixed by an underscore character ('_').

So in addition to review potpourri of .NET features, this article is also advocating a coding style. It is an exercise left to the reader to determine if this more limited form of Hungarian notation is worth the tidbit of extra typing required.

Readers seeking to extend this application should consider some of the following enhancements:

  • Make loading/persisting of XML files part of a popup menu accessed by right-mouse-clicking the service. At present the service must be selected and then serialized/de-serialized using the File menu.
  • Descriptive icons can be added to the TreeView which displays the services. For example different icons could be displayed depending on whether or a not a service was running, disabled, or paused.
  • Implement better error handling such as using Windows Forms ErrorProvider, allowing multiple error messages to be displayed in the error TextBox and providing a way to clear the error TextBox by possibly selecting the TextBox directly. One way might be to right mouse click and clear the TextBox using a menu item.
  • Add a filter option so that services which are dependents are not visible at the root-level of the TreeView control. This would insure that the TreeView contained no duplicates.
  • After a service and its dependents are administered, the entire TreeView control is refreshed. An optimization could be performed that only updates the specific service that was updated, and the descendents of the updated service. The same optimization can be performed when a service and its descendents are restored from a file containing serialized XML.
  • Separate the service management classes into a separate assembly so they can be used by other applications.

Article Information
Author | Jan Narkiewicz
Technical Editor | Adam Ryland
Project Manager | Helen Cuthill
Reviewers | Andrew Krowczyk, Saurabh Nandu

If you have any questions or comments about this article, please contact the technical editor.


Improved Service Control Manager (SCM), Part 2

Improved Service Control Manager (SCM), Part 2
http://www.csharptoday.com/content.asp?id=1749
Article Rating: 3
Published on February 21, 2002
Abstract
This article presents an alternative implementation of the Windows Service Control Manager (SCM) found under the Control Panel’s Administrative Tools folder. This alternate SCM is implemented using .NET, specifically Windows Forms. This .NET variation of the SCM is not meant to reinvent the wheel, but instead is designed to provide functionality the stock windows SCM does not provide, in particular to give a mechanism was needed to turn off and then turn back on suites of related services. This allows us to free up resources (memory) and remove contentions for certain items (such as the FTP, SMTP and HTTP ports). Part 2 presents the Windows Forms implemented GUI.
Article

The first part in this series (http://www.csharptoday.com/content.asp?id=1798&WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0) presented a rationalization as to why another service control manager was needed. Basically, this new and improved service control manager allows services and their dependents to be managed as a unit. This simplifies service administration and allows an administrator to free up resource associated with an application composed of a suite of services. To demonstrate what is meant by resources, consider the following snippet from Task Manager from a laptop used to develop both SQL Server and Oracle:

Clearly the previous screenshot shows a lot of resources (129 MB) that could be cleaned up if the current day's task is not Oracle development. Given the service-oriented subject matter, the first part of this article discussed creating and installing services under .NET. Also presented were discussions of some other useful areas exploited in the application name, the classes found in the System.Collections namespace, and performing XML serialization using the features exposed by the System.Xml.Serialization namespace.

This section of our improved service control manager (part 2) presents the Windows Forms implemented GUI. The elements presented in the first installment of this article included basics of service administration, the collections used by this application, and the methods used to serialize/de-serialize XML. The Form loaded by this application contains a TreeView control that displays services and their dependent. The MainMenu of the application is used to specify when the service state is serialized as XML and de-serialized from XML. The MainMenu also allows statistical information to be cleared, the services displayed to be reset, and the services viewed to be limited by applying filters. A separate dialog is implemented in order to handle the administration of services and their dependents.

What is not covered in this second installment is the low-level manipulation of services using the Windows API. This material is left for the third installment of this series.

Windows Forms

The basic layout for the Form used to implement the improved service control manager is easily seen using Visual Studio .NET's Windows Forms designer:

The previous screenshot demonstrates the primary Windows Forms controls that compose the application. Each of these controls was dragged from Visual Studio .NET's Toolbox while in design mode and placed on the form. The primary types of controls utilized include:

  • Treeview - the TreeView is best known as "that control in the left panel of Windows Explorer." The TreeView lives up to its name in that it readily displays a hierarchy (tree) of information such as shows a service at the root level (Level 0) and each dependent service at the next level of the tree (Level 1). The services dependent on the services in Level 1 would be displayed in Level 2 of the tree, and so on.
  • TextBox -to no-one's surprise, this control displays text. For example, a TextBox could display statistics for all services managed by an application (upper-right corner of previous screenshot). Another use of a TextBox could be to display the errors encountered while serializing and de-serializing services to/from a file (lower-right corner of previous screenshot).
  • Splitter - the Splitter control is useful in that it allows docked controls to be resized. Additionally the term splitter was featured quite often in the Monty Python's movie, "Life of Brian." Keeping to the subject of Windows Forms, a Splitter can be (for example) placed between two text boxes. When the Splitter is dragged up, the top text box gets smaller and the bottom text box gets correspondingly larger. Conversely, when the Splitter is dragged down, the top text box gets larger and the bottom text box gets correspondingly smaller.
  • Panel - the two TextBox controls and Splitter control on the right side of the form actually reside on a Panel control. The Panel control is used to contain multiple controls (e.g. two TextBox's and a Splitter) and act on them as a unit. The Panel on the right side of the form is used to "split" the form. The left-side of the form is the TreeView and the right-side is the Panel. The Splitter (vertical line) running down the middle of the form can be used to resize the TreeView or the Panel.
  • MainMenu - provides a menu for the form which could be used to specify file related actions such as serializing the data contained on the form to a file or de-serializing the data contained on the form from a file.
  • OpenFileDialog - supports traversing directories, reading a list of files from a directory, and selecting a file or files from this directory. The selected file (or files) can then be opened. This dialog is particularly handy when performing a task such a de-serializing XML data from a file.
  • SaveFileDialog -- supports traversing directories and specifying a file name to be contained within said directory. The specified file can then be saved. This dialog is particularly handy when performing a task such a serializing XML data to a file.

MainMenu Control

The basic layout of the improved SCM's main menu and menu items is as follows:

Note: the previous menu's were drawn using the version of Visio released with Visual Studio .NET's Architect Edition. Drawing GUI's is just one of the many new perks in Visio.

Specifying a method that is triggered when a MenuItem is clicked is trivial. Simply double-click the menu item within Visual Studio's Windows Forms Designer and a method is created. This method is associated with the Click event (such as the following method, menuItemFilePersist_Click, associated with the File menu's Persist menu item):

this.menuItemFilePersist.Click +=
new System.EventHandler(this.menuItemFilePersist_Click);

The breakdown of the each menu item for the improved service control manager is as follows:

  • File, Load - causes the state of the currently selected service and its dependents to be reset based on a configuration that was previously persisted. This involves de-serializing objects previously persisted as XML. The method which handles this task is menuItemFileLoad_Click.
  • File, Persist - causes the state of a service and its dependent services to be serialized to a file using XML as the format. The method which handles this is menuItemFilePersist_Click.
  • File, Exit - causes the application to exit by calling the Application class's Exit method as follows:

private void menuItemFileExit_Click(object sender, System.EventArgs e)
{
Application.Exit();
}

  • View, Refresh - causes the tree of services displayed in the TreeView to be refreshed. This is accomplished by having this MenuItem 's Click event associated with the menuItemViewRefresh_Click method. The menuItemViewRefresh_Click method then calls SWPRefreshTree to refresh the hierarchy of services.
  • View, Clear Last Error - causes the TextBox in which errors are displayed to be reset thus indicating that there are no errors presently. This is accomplished using the menuItemClearLastError_Click method which is implemented as follows:

private const string _strNoErrors = "No errors";

private void menuItemClearLastError_Click(object sender,
System.EventArgs e)
{
textBoxErrors.Text = _strNoErrors;
}

The third MenuItem in the View menu is actually a separator. A separator menu item can be inserted by right-mouse-clicking on a menu item and then by selecting the Insert Separator menu item as follows:

All menu items underneath this separator are filters. If the menu item is checked (the Checked property of the MenuItem instance set to true) then services of the corresponding ServiceType are not displayed in the TreeView control. For example if the Adaptor menu time is checked, then the filter associated with services of type ServiceType.Adaptor is set to true. The services displayed are refreshed using the SWPRefreshTree method. If the Adaptor menu item is clicked on while checked, it becomes un-checked. The filter associated with services of type ServiceType.Adaptor is set to false. The services displayed are refreshed using the SWPRefreshTree method. When the services are refreshed, this time it is possible to display services of ServiceType.Adaptor. The code for handling when the Adaptor menu time is clicked is as follows:

    private void
            menuItemAdaptor_Click(object sender, System.EventArgs e)
{
if
            ((bool)_serviceTypeFilters[ServiceType.Adapter])
{_serviceTypeFilters
[ServiceType.Adapter]
            = false;
((MenuItem)sender).Checked =
            false;
}
else
{
_serviceTypeFilters[ServiceType.Adapter]
            = true;
((MenuItem)sender).Checked =
            true;
}
SWPRefreshTree();
}

In the previous code, the _ serviceTypeFilters data member is a Hashtable that correlates a Boolean with each ServiceType enumeration value. The implementation in the previous mode is repeated for each menu item below the separator in the View menu. The prototype for each method associated with a View menu's filter is as follows:

private void menuItemFileSystemDriver_Click(object sender,
System.EventArgs e)

private void menuItemInteractiveProcess_Click(object sender,
System.EventArgs e)

private void menuItemKernelDriver_Click(object sender,
System.EventArgs e)

private void menuItemRecognizerDriver_Click(object sender,
System.EventArgs e)

private void menuItemWin32OwnProcess_Click(object sender,
System.EventArgs e)

private void menuItemWin32ShareProcess_Click(object sender,
System.EventArgs e)

As it turns out, there are too many filters provided in the View menu. The services displayed in the TreeView control are retrieved using the ServiceController class's GetServices method. To quote MSDN with regards to the GetServices method, "Retrieves all the services on the local computer, except for the device driver services." This means that services of the types Adaptor, FileSystemDriver, KernelDriver, and RecognizerDriver are never displayed. As part of the menu setup, MenuItem 's are disabled if no services exist for that particular type of service. The point is that there is no reason to filter service type unless there is actually a service of that type running. For this precise reason, the View menu is displayed as follows:

The menu items for services of the types Adaptor, FileSystemDriver, KernelDriver, and RecognizerDriver are grayed out using the MenuItem 's Enabled property. By setting Enabled to false, the menu time is disabled (grayed-out). Setting the Enabled property to true for each of the aforementioned service types causes the menu item corresponding to the service type to be enabled.

The SWPCheckServiceTypeFilterDisable method is used to enable and disable filter-related menu times. This method takes a ServiceType and Boolean parameter. If the Boolean parameter is set to true, then the MenuItem corresponding to the service type is enabled. If the Boolean parameter is set to false, then the MenuItem is disabled. The SWPCheckServiceTypeFilterDisable method is implemented as follows:

void SWPCheckServiceTypeFilterDisable(ServiceType serviceType,
bool bEnabled)
{
switch(serviceType)
{
case ServiceType.Adapter:
this.menuItemAdaptor.Enabled = bEnabled;
break;

case ServiceType.FileSystemDriver:
this.menuItemFileSystemDriver.Enabled = bEnabled;
break;

case ServiceType.InteractiveProcess:
this.menuItemInteractiveProcess.Enabled = bEnabled;
break;

case ServiceType.KernelDriver:
this.menuItemKernelDriver.Enabled = bEnabled;
break;

case ServiceType.RecognizerDriver:
this.menuItemRecognizerDriver.Enabled = bEnabled;
break;

case ServiceType.Win32OwnProcess:
this.menuItemWin32OwnProcess.Enabled = bEnabled;
break;

case ServiceType.Win32ShareProcess:
this.menuItemWin32ShareProcess.Enabled = bEnabled;
break;
}
}

TreeView Control

The left-side of the improved service control manager contains a TreeView control. The following screenshot shows this TreeView control (the left-side of the form) displaying services created for demonstration purposes with this article (all services starting with prefix, SWP):

The TreeView control in the previous screenshot contains every copy of every service at the root level of the tree. Services with dependents contain leaves. Dependent services with their own dependent services also contain leaves (and so on down the hierarchy). A service which is a dependent service will appear at root level of the TreeView and also as a dependent service. For example SWPQLLeaf01 is a service at the root level of the tree, and is also contained as a leaf of the SWPQLServerRoot service.

Each TreeView control contains a Nodes property of type TreeNodeCollection. The elements of the TreeNodeCollection are of type TreeNode. Each TreeNode contains the following properties instrumental to the creation of the tree:

  • Text - this property, of type string, is the text displayed for a given node within a TreeView hierarchy. In the previous screenshot an example of the Text property is System Event Notification (SENS, Running).
  • Tag - this property, of type Object, contains the data associated with a TreeNode. For the improved service control manager application, the Tag property contains enough information to uniquely identify the service.
  • Nodes - this property, of type TreeNodeCollection, contains the children of a TreeNode. Each child can in turn contain Nodes and their children can contain Nodes, etc.

The basic layout of the TreeView control and its children (TreeNodes) is as follows:

The SWPRefreshTree method of the SWPSCMClient.cs source file initiates the population of the TreeView instance that contains the services. Before the TreeView is populated, the total service counter and the per-service type counters are set to zero:

_serviceCount = 0;
SWPSetupServiceTypesHash(out _serviceTypeCounts, 0);

The per-service type counters are based on the number of services of type InteractiveProcess, Win32OwnProcess, etc. Once these counters are set to zero, the TreeView 's SuspendLayout method is called. This method is inherited from the TreeView 's Control base-class. The SuspendLayout method turns off the updating of the TreeView control while the control is being updated. The purpose of this is to reduce flicker while a control is being modified.

Once the TreeView 's layout has been suspended, then all existing nodes are removed (treeViewServices.Nodes.Clear();) and new nodes (containing per-service data) are added using the form's SWPLoadServices method. Once the hierarchy of services has been added to the TreeView control, the TreeView 's ResumeLayout method is called in order to layout the freshly updated TreeView control. The SWPRefreshTree method which performs all of the aforementioned tasks is implemented as follows (with a bit removed for brevity):

    void
            SWPRefreshTree()
{
// Set service count and serviceType counts
            to 0
_serviceCount = 0;
SWPSetupServiceTypesHash(out
            _serviceTypeCounts,
            0);
treeViewServices.SuspendLayout();
treeViewServices.Nodes.Clear();
SWPLoadServices(ServiceController.GetServices(),
treeViewServices.Nodes);
treeViewServices.ResumeLayout(true);
//
            setup service statistics so textBoxServiceStats.Lines can be
            set
// Code not provided for reasons of
            brevity
textBoxServiceStats.Lines = outputLines;
}

The SWPLoadServices method uses recursion in order to populate the TreeView. Back in university, while taking "Data Structures 101", we were all taught that recursion is slower from a performance standpoint than iteration. This might be true, but at most there are dozens of services (maybe even hundreds worst-case) and the number of levels in the hierarchy might at most be five or six. This corresponds to five or six levels of recursion. Performance is not prohibitive since this is not (for example) a recursive quick sort operating on millions of entries. And to the Teaching Assistant who deducted five points from my sorting assignment in Data Structures 101: I write for Wrox Press and you live in your parent's basement.

The parameters passed to the SWPLoadServices methods are an array containing all services at a particular level of the tree (ServiceController[] services) and the TreeNode collection (type, TreeNodeCollection) to which this service information will be added. The first time SWPLoadServices is called, the list of all services is specified (as returned by the ServiceController class's GetServices method) as the first parameter. The second parameter is the root-level TreeNode collection which corresponds to the nodes property of the TreeView instance, treeViewServices. Recall that SWPLoadServices was called from the SWPRefreshTree method as follows:

SWPLoadServices(ServiceController.GetServices(),
treeViewServices.Nodes);

Before delving further, there is a special case of services that needs to be addressed: duplicate services. Under Windows 2000 Advance server, there are certain services that had multiple instances running. Since our application saves the state of a service and its dependents to a file and then restores it, duplicates pose a unique problem. What if ServiceX was serialized to a file and an attempt was made to de-serialize the state of ServiceY from the file. Such a mistake could take place if ServiceX and ServiceY had the same name, but represented different services. For this reason, the return value of the SWPLoadServices method is a Boolean which returns true if the service being traversed contains a duplicate. If the service being traversed contains a duplicate, then the parent service also cannot be serialized or de-serialized because one of its children is a duplicate and is hence ambiguous with respect to identity. It should be noted that this is an exceptional case.

In order to handle duplicates, the rather simplistic SWPServiceDupFinder class was implemented (source file, swpservicedupfinder.cs) as follows:

internal class SWPServiceDupFinder
{
internal bool isDuplicate = false;
internal ServiceController service = null;

internal SWPServiceDupFinder(ServiceController service,
bool isDuplicate)
{
this.service = service;
this.isDuplicate = isDuplicate;
}
}

The SWPServiceDupFinder class contains a data member corresponding to the service (ServiceController service) and a data member indicating if the service is a duplicate (meaning a service with the same name exists). The bool isDuplicate data member is used to specify if a service contains a duplicate. The SWPServiceDupFinder class, its constructor, and its data members are all declared as internal with respect to the level of protection. The reason for this is that the class is quite trivial and is designed to just add a true/false data member to a service instance of type ServiceController.

The SWPLoadServices method performs the following basic steps:

  • Creates an instance of a SortedList in which services are stored. This is a sorted list so that the services displayed are in sorted order and also so that duplicates can be detected. The code that which creates the SortedList is as follows:

SortedList orderServices =
new SortedList(new CaseInsensitiveComparer());

  • Iterate over each service in the collection (a non-sorted list) so services can be placed in a sorted collection:

foreach (ServiceController service in services)

  • Add each entry in the service list to the sorted list, oderServices. Entries are of type SWPServiceDupFinder, a class which was already discussed in detail. The strExtra string and isDuplicated Boolean variables will be discussed later, but basically these variables are used to handle the case of duplicate service names:

orderServices.Add(service.DisplayName + strExtra,
new SWPServiceDupFinder(service, isDuplicate));

  • Iterate over each service in the sorted collection so a sorted list of services can be added to the tree node collection:

foreach (DictionaryEntry entry in orderServices)

  • Add each service to the tree node collection:

TreeNode treeNode =
new TreeNode(SWPServiceDisplay(serviceDupFinder.service));

treeNode.Tag = serviceDupFinder;
nodes.Add(treeNode);

  • For each service in the sorted list, add its dependent services to the Nodes property of the TreeNode just added to represent the service:

if (SWPLoadServices(serviceDupFinder.service.DependentServices,
treeNode.Nodes))

Three variables are declared to help manage duplicate service entries:

    string strExtra =
            "";
int tick = 0;
bool isDuplicate = false;

Each time a duplicate name is detected, the value of the tick variable is incremented. When the duplicate is placed in the SortedList, the name of the service is changed by appending the value of strExtra to the service's name. The strExtra variable is just the value of tick converted to a string using the ToStringMethod. When the first duplicate service is detected a value of "1" (tick=1, strExtra="1") is appended to the service's name. If the service added to the sorted list is not a duplicate then strExtra is set to an empty string (strExtra="") and appended to the service's name. Recall that services are added to the orderServices SortedList as follows:

orderServices.Add(service.DisplayName + strExtra,
new SWPServiceDupFinder(service, isDuplicate);

When the second duplicate is detected, even if the service differs from the original duplicate service name, a "2" (tick=2, strExtra="2") is appended to the service's name. The isDuplicate data member is set (true if the service is a duplicate) and this value is passed as the second parameter of the SWPServiceDupFinder class's constructor (indicating the service is a duplicate). Do not fret that the strExtra will get displayed in the TreeView. This key value of the orderServices SortedList is for sorting purposes only (to insure no duplicate entries) and this key value is never used for display purposes.

A duplicate service is detected using orderServices.Contains. When a duplicate is detected, the service already contained in the orderServices collection is retrieved and marked as being a duplicate service:

SWPServiceDupFinder dupFinder =
(SWPServiceDupFinder)orderServices[service.DisplayName];

dupFinder.isDuplicate = true;

While iterating over the services, certain services are skipped based on the service's ServiceType. Remember that the serviceTypeFilters Hashtable contains a list of ServiceType enumeration values that are filtered (not displayed in the improved service control manager). Filtering within the for loop that iterates services is simply implemented using the SWPIsFiltered method, which looks up a particular ServiceType in the serviceTypeFilters HashTable and determines "is this service type to be filtered?":

if (SWPIsFiltered(service.ServiceType))
{
continue; // we are filtered
}

Now that the coding highlights have been reviewed, the SWPLoadServices method in its entirety is as follows:

private bool SWPLoadServices(ServiceController[] services,
TreeNodeCollection nodes,
Hashtable serviceTypeExists,
bool isRootLoadServices)
{
bool duplicateFound = false;

if (services.Length == 0)
{
return duplicateFound;
}

SortedList orderServices =
new SortedList(new CaseInsensitiveComparer());
string strExtra = "";
int tick = 0;
bool isDuplicate = false;

foreach (ServiceController service in services)
{
// make sure we can handle the case of a new
// service type without error
if (SWPIsFiltered(service.ServiceType, serviceTypeExists))
{
continue; // we are filtered
}

if (isRootLoadServices)
{
SWPUpdateServiceTypeCounts(service.ServiceType);
}

if (orderServices.Contains(service.DisplayName))
{
SWPServiceDupFinder dupFinder = (SWPServiceDupFinder)
orderServices[service.DisplayName];

duplicateFound = true;
dupFinder._isDuplicate = true;
tick++;
strExtra = tick.ToString();
isDuplicate = true;
}

else
{
strExtra = "";
isDuplicate = false;
}

orderServices.Add(service.DisplayName + strExtra,
new SWPServiceDupFinder(service,
isDuplicate));
}

foreach (DictionaryEntry entry in orderServices)
{
SWPServiceDupFinder serviceDupFinder =
(SWPServiceDupFinder)entry.Value;
TreeNode treeNode =
new TreeNode(SWPServiceDisplay(serviceDupFinder._service));

treeNode.Tag = serviceDupFinder;
nodes.Add(treeNode);
if (SWPLoadServices(serviceDupFinder._service.DependentServices,
treeNode.Nodes,
serviceTypeExists,
false))
{
duplicateFound = true;
}
}

return duplicateFound;
}

Another element of the TreeView class which is critical to the implementation of the improved service control manager is the SelectedNode property (of type TreeNode). This property corresponds to the element in the tree that is currently selected. The SelectedNode property can be accessed as follows:

TreeNode selectedNode = treeViewServices.SelectedNode;

The SelectedNode property is used when serialization is initiated. The SelectedNode property corresponds to the service whose settings are to be serialized by the menuItemFilePersist_Click method.

TextBox Control

Statistics for all services managed (upper-right TextBox) and errors (lower-right TextBox) are displayed on the form using instances of TextBox controls. For both these TextBox instances the following properties are set:

textBoxInstance.ScrollBars = System.Windows.Forms.ScrollBars.Vertical;
textBoxInstance.WordWrap = true;
textBoxInstance.MultiLine = true;

The MultiLine property is set to true because the information can take up multiple lines. The WordWrap property is set to true so that there will be no need for horizontal scrollbars. The ScrollBars property is set to ScrollBars.Vertical in case the information displayed cannot be completely displayed in the TextBox. The vertical scrollbars are crucial because there is a Splitter control between these TextBox 's. The Splitter control allows one TextBox to be made larger at the expense of the other TextBox, which is made smaller. The properties of the TextBox pertaining to the Splitter are discussed in detail in the "Splitter Control" section of this article.

The TextBox in the upper-right of the dialog is specified by the textBoxServiceStats instance. The individual statistics for all services are displayed on separate lines of this TextBox. These statistics are specified using an array of strings which is assigned to the TextBox 's Lines property. The SWPRefreshTree method of the SWPSCMClient.cs source file is used to populate this TextBox. The first step in populating this TextBox is to allocate an array (string [] outputLines) with room for each string of information displayed, and to set a counter (int i) corresponding to the current line being set:

string [] outputLines = new string [serviceTypeCounts.Count + 4];
int i = 0;

The number of elements in the outputLines array corresponds to one-per-service type, (serviceTypeCounts.Count) plus four more (+ 4) used to contain some overall statistics. Once the array to hold the information is created (string [] outputLines) then the serviceTypeCounts hash needs to be traversed and displayed. Amongst the service statistics displayed is a count of each type of service (InteractiveProcess, Win32OwnProcess, etc.). This per-service-type count is maintained in the serviceTypeCounts hash. Traversing and displaying the information contained in this hash is achieved as follows:

foreach (DictionaryEntry entry in serviceTypeCounts)
{
outputLines[i] = String.Format("{0}: {1}",
(ServiceType)entry.Key,
(int)entry.Value);
i++;
}

Notice in the previous code snippet that the String class's Format method is used to display the ServiceType and the number of entries for said service type. Old school C and C++ developers recognize String.Format as how .NET implements the legacy C-runtime function, sprintf. Actually, String.Format is an improvement on old, reliable sprintf.

Once the counts of each service type are displayed, an overall service count is added to the outputLines array. Once this information is added then the outputLines array is assigned to the TextBox 's Lines property:

    outputLines[i] =
            "--------------------------------";
i++;
outputLines[i] =
            String.Format("Service Count: {0}",
            serviceCount);
i++;
outputLines[i] = " (service type total
            can";
i++;
outputLines[i] = " exceed service
            count)";
textBoxServiceStats.Lines = outputLines;

Serialization errors are displayed using the textBoxErrors TextBox instance. The methods that handle de-serialization, menuItemFileLoad_Click, and serialization, menuItemFilePersist_Click, each contain a try / catch block. If an error occurs (an exception is thrown) during serialization or de-serialization then the error is caught in the catch block and the Text property of the textBoxErrors TextBox is set to string representation of this error. The basic implementation of the menuItemFileLoad_Click and menuItemFilePersist_Click methods are as follows:

    void
            menuItemFileLoad_Click(object sender, System.EventArgs
            e)
{
try
{
// De-serialize file
            here
}

catch(Exception ex)
{
textBoxErrors.Text =
            ex.ToString();
}
}

void
            menuItemFilePersist_Click(object sender, System.EventArgs
            e)
{
try
{
// Serialize file
            here
}

catch(Exception ex)
{
textBoxErrors.Text =
            ex.ToString();
}
}

Panel Control

The two TextBox controls and the Splitter were added to the Panel solely using the Windows Forms Designer. The Panel was created by dragging it from Visual Studio .NET's ToolBox and placing it on the form. The two TextBox controls and the Splitter were added to the Panel also by dragging the appropriate icon from the ToolBox and placing each control on the Panel. The purpose of the Panel is to allow the Panel and all the controls that reside on it to be resized as an atomic unit.

One useful trick when dealing with Panel controls and GroupBox controls (similar to the Panel controls) is in how to select the control. Normally it is simple to select (highlight) a control using Windows Forms Designer. Since a Panel or a GroupBox can be covered by child controls it is difficult to select in this manner. The trick to selecting a Panel or GroupBox is to use the Properties windows and select the appropriate control from the drop-down list:

The other major perk of the Panel control (not used in this application) is this control's Enabled property. When the Enabled property is set to true, all child controls on the Panel are enabled. When this property is set to false then all child controls are disabled en-masse.

Splitter Control

The Splitter control allows docked controls to be resized. The Splitter control, for example, allows the applications TreeView control (left-side of form) and Panel control (right-side of form) to be resized. The steps used to associate the Splitter with the TreeView and Panel controls were as follows:

  • Drag a TreeView from the ToolBox to the form and place it on the left-side of the form.
  • Drag a Panel from the ToolBox to the form and place it on the right-side of the form.
  • Drag a Splitter from the ToolBox and place it on the TreeView control.
  • Set the Dock property of the TreeView control to DockStyle.Left. The following screenshot demonstrates how to accomplish this using Visual Studio .NET's Property window:

  • Set the Dock property of the Panel control to DockStyle.Fill. The following screenshot demonstrates how to accomplish this using Visual Studio .NET's Property window:

When the application containing the form is run, the Splitter can be pulled right to make the TreeView control larger and Panel smaller (Dock set to Fill). Dragging the Splitter left makes the TreeView smaller and Panel larger (Dock set to Fill).

A Splitter is also used between the TextBox in which service statistics are displayed and the TextBox in which errors are displayed. These TextBox controls reside on the Panel and are resized by dragging the Splitter up or down.

OpenFileDialog and CloseFileDialog Controls

Dialogs are associated with a form by dragging the corresponding icon (e.g. OpenFileDialog icon) from the Toolbox to the form contained in the Windows Forms designer. This action will create a data member in the form associated with the dialog and will cause an icon corresponding to the dialog to be displayed in Windows Forms Designer below the actual form itself. The previous screenshot of the form associated with the SWPSCMHelper application contains an icon for both an OpenFileDialog and a CloseFileDialog control residing below the form itself.

The OpenFileDialog and a CloseFileDialog controls share a common inheritance ancestry:

System.Object
System.MarshalByRefObject
System.ComponentModel.Component
System.Windows.Forms.CommonDialog
System.Windows.Forms.FileDialog
System.Windows.Forms.OpenFileDialog
System.Windows.Forms.SaveFileDialog

The CommonDialog base class exposes the ShowDialog method. This method displays the dialog such as in the following excerpt from the SWPSCMClient.cs file's menuItemFilePersist_Click method which displays an instance of the SaveFileDialog :

DialogResult result = saveFileDialogSerialize.ShowDialog();

The prototype for the ShowDialog method is as follows:

Overloads Public Function ShowDialog() As DialogResult ' VB.NET
public DialogResult ShowDialog(); // C#

The return value of ShowDialog corresponds to which button on the dialog was ultimately selected. An enumeration value of Abort indicates the "Abort" button was selected, while the Cancel enumeration indicates the "Cancel" button was selected. Other permissible values for this enumeration include: Ignore, No, None, OK, Retry, and Yes.

The DialogResult enumeration is easily understood when examining an instance of the OpenFileDialog such as the following (displayed on a system running Windows XP):

The OpenFileDialog contains a Open and a Cancel button which correspond to specific DialogResult enumeration values (OK and Cancel respectively). The code inside the SWPSCMClient.cs file's menuItemFilePersist_Click method recognizes when the dialog is canceled and hence the services should not be serialized:

DialogResult result =
openFileDialogSerialize.ShowDialog();

if ((result == DialogResult.Abort) ||
(result == DialogResult.Cancel))
{
return ; // noop case
}

// Serialize services and descendants here.

The previous code simply returns if the "Cancel" button is selected. Cancel is also returned if the dialog is closed using the X button in the upper right corner of the dialog or if the dialog is close using the ESC key.

The screenshot of the OpenFileDialog control contained no file name. If the SaveFileDialog for this application had been previously used to serialize an XML file, then the FileName property of the SaveFileDialog would be considered the "last file serialized." The FileName property (last file serialized) of the SaveFileDialog could then be used to populate the file name initially displayed in the OpenFileDialog dialog. The file name initially displayed in the OpenFileDialog dialog is set using this class's FileName property. The code inside the SWPSCMClient.cs file's menuItemFilePersist_Click method which sets the initial file name displayed in the OpenFileDialog control is as follows:

if ((openFileDialogDeserialize.FileName == null) ||
(openFileDialogDeserialize.FileName == ""))
{
openFileDialogDeserialize.FileName =
saveFileDialogSerialize.FileName;
}

Server Admin Form

When a node in the TreeView control (corresponding to a service) is double clicked, an event is fired which causes the SWPServerAdmin dialog to be displayed. This double click association is setup using the Properties windows corresponding to the TreeView control and double clicking in the drop-down box next to the DoubleClick text. This creates a method associated with having the TreeView control handle the double click. This association is shown below:

Knowing which service to administer (based on the double click taking place) is a matter of determining the currently selected node within the TreeView control. The node doubled clicked within the tree corresponds to the currently selected node. Ascertaining the currently selected node is achieved by calling the TreeView class's SelectedNode property as is demonstrated in the following code snippet:

TreeView serviceTreeView = (TreeView)sender;
TreeNode clickedNode = serviceTreeView.SelectedNode;
SWPServiceDupFinder serviceDupFinder =
(SWPServiceDupFinder)clickedNode.Tag;

As shown in the previous code snippet the X and Y properties of the MouseEventArgs instance can be passed to the TreeView 's GetNodeAt method in order to retrieve the node clicked. Hopefully readers recognize the X and Y properties as corresponding to Cartesian coordinates on the screen indicating where the mouse click was plunked down.

One the service clicked has been determined, the service administration GUI can be displayed:

SWPServiceAdmin serviceAdmin =
new SWPServiceAdmin(serviceDupFinder._service);

if (DialogResult.OK == serviceAdmin.ShowDialog(this))
{
// reset current node
}

Once the dialog has been displayed, and hence the service has been administered, the TreeView control is refreshed. Once refreshed the currently selected node is restored so that the TreeView is approximately in the same state as before the administrative dialog was displayed:

string whatToSelect = serviceDupFinder._service.ServiceName;
SWPRefreshTree();

// resync the tree to current position
clickedNode = null;
foreach (TreeNode node in serviceTreeView.Nodes)
{
serviceDupFinder = (SWPServiceDupFinder)node.Tag;
if (0 == String.Compare(serviceDupFinder._service.ServiceName,
whatToSelect))
{
clickedNode = node;
break;
}
}

serviceTreeView.SelectedNode = clickedNode;

The treeViewServices_DoubleClick method in its entirety is as follows:

private void treeViewServices_DoubleClick(object sender,
System.EventArgs e)
{
try
{
TreeView serviceTreeView = (TreeView)sender;
TreeNode clickedNode =
serviceTreeView.GetNodeAt(_savedMouseEventArgs.X,
_savedMouseEventArgs.Y);
SWPServiceDupFinder serviceDupFinder =
(SWPServiceDupFinder)clickedNode.Tag;
SWPServiceAdmin serviceAdmin =
new SWPServiceAdmin(serviceDupFinder._service);

if (DialogResult.OK == serviceAdmin.ShowDialog(this))
{
string whatToSelect =
serviceDupFinder._service.ServiceName;
SWPRefreshTree();

// resync the tree to current position
clickedNode = null;
foreach (TreeNode node in serviceTreeView.Nodes)
{
serviceDupFinder =
(SWPServiceDupFinder)node.Tag;
if (0 == String.Compare(
serviceDupFinder._service.ServiceName,
whatToSelect))
{
clickedNode = node;
break;
}
}

serviceTreeView.SelectedNode = clickedNode;
}
}

catch(Exception ex)
{
SWPRefreshTree();
textBoxErrors.Text = ex.ToString();
return ;
}

textBoxErrors.Text = _strNoErrors;
}
}

The service administrative dialog displayed by serviceAdmin.ShowDialog(this) is found in the SWPServiceAdmin.cs source file and is as follows:

The items displayed in the previous dialog are all set up in the SWPServiceAdmin class's constructor. The region displaying W3SVC is a Label control containing the ServiceName property of the service being administered (the ServiceController instance). While the region displaying World Wide Web Publishing is also a Label control containing the DisplayName property of the service being administered (the ServiceController instance).

The dialog contains two GroupBox controls entitled, Status and Start Type. These GroupBox controls each contain three RadioButton controls. By placing each triplet of RadioButton controls in a GroupBox insures that only one of the three will be selected at a given time. In the dialog, one of each RadionButton triplet is marked in boldface. This boldface indicates the current status or start type of the service. The SWPMakeRadioButtonChecked method is used to tweak the font of the RadioButton that corresponds to the current service state:

private static void
SWPMakeRadioButtonChecked(RadioButton radioButton)
{
radioButton.Checked = true;
radioButton.Font = new System.Drawing.Font("Microsoft Sans Serif",
8.25F,
FontStyle.Bold,
GraphicsUnit.Point,
((System.Byte)(0)));
}

The previous code snippet contains in boldface the specific parameter that in turn displays the CheckBox in boldface.

Underneath the two GroupBox controls is a CheckBox control whose Text property is set to "Also apply settings to dependent services". When this CheckBox's Checked property is set to true, administration settings are reflected for the service and all services dependent on the service.

The two Button controls at the bottom of the dial are appropriately labeled: Ok and Cancel. When the Cancel button is clicked, the buttonServiceAdminCancel_Click method is called:

private void buttonServiceAdminCancel_Click(object sender,
System.EventArgs e)
{
this.DialogResult = DialogResult.Cancel;
this.Close();
}

The buttonServiceAdminCancel_Click method sets the dialog's result. This is achieved by setting the DialogResult property to DialogResult.Cancel indicating that the Cancel button was selected. This method also closes the dialog (causes the dialog not to be displayed). The tail end of the buttonServiceAdminOK_Click method also sets the DialogResult property, but under this circumstance sets this property to DialogResult.Ok. The buttonServiceAdminOK_Click method also closes the dialog. The beginning part of this method correlates each RadioButton to the appropriate service status or start mode. For example if the radioButtonStarted Checked property is true then the ServiceStartMode is set to ServiceControllerStatus.Running. Once the start mode and service status values have been determined, the service can be administered by calling SWPUpdatedStartMode and SWPUpdatedStatus :

SWPUpdatedStartMode(_service,
startMode,
checkBoxAlsoDependent.Checked);
_service.Refresh();
SWPUpdatedStatus(_service,
status,
checkBoxAlsoDependent.Checked);

The buttonServiceAdminOK_Click method in its entirety is implemented as follows:

private void buttonServiceAdminOK_Click(object sender,
System.EventArgs e)
{
ServiceStartMode startMode;
ServiceControllerStatus status;

if (radioButtonStarted.Checked)
{
status = ServiceControllerStatus.Running;
}

else if (radioButtonStopped.Checked)
{
status = ServiceControllerStatus.Stopped;
}

else // if (radioButtonPaused.Checked)
{
status = ServiceControllerStatus.Paused;
}

if (radioButtonAutomatic.Checked)
{
startMode = ServiceStartMode.Automatic;
}

else if (radioButtonDisabled.Checked)
{
startMode = ServiceStartMode.Disabled;
}

else // if (radioButtonManual.Checked)
{
startMode = ServiceStartMode.Manual;
}

SWPUpdatedStartMode(_service,
startMode,
checkBoxAlsoDependent.Checked);
_service.Refresh();
SWPUpdatedStatus(_service,
status,
checkBoxAlsoDependent.Checked);
this.DialogResult = DialogResult.OK;
this.Close();
}

Updating Service Status

The SWPUpdatedStatus method is used to update a service's execution status. This method takes as parameters the service to be updated (ServiceController service), the new start mode to be specified (ServiceControllerStatus status), and a Boolean (bool handleDependents) indicating if the start mode should also be reflected in the dependents of said service.

public static void SWPUpdatedStatus(ServiceController service,
ServiceControllerStatus status,
bool handleDependents)

Updating the start mode of dependents is simply a matter of iterating over the service's dependents and calling the SWPUpdatedStartMode method is a recursive fashion. This updating only occurs if the handleDependents parameter is set to true :

    if
            (handleDependents)
{
foreach (ServiceController dependent in
            service.DependentServices)
{
SWPUpdatedStatus(dependent,
            status, handleDependents);
}
}

Once the service status of the dependents has been updated (or not), the service instance is refreshed so it is cognoscente of the changes made to its dependents:

service.Refresh();

At this stage, the service's state can actually be changed. A switch statement is used to choose the service state to which a service will be changed. Before making this change, care is taken in order to insure the service's state can be changed accordingly. In order to understand this, consider the case of changing the service's state to ServiceControllerStatus.Paused :

    if
            (service.CanPauseAndContinue)
{
if (service.Status ==
            ServiceControllerStatus.Stopped)
{
service.Start();
}

if
            (service.Status ==
            ServiceControllerStatus.Running)
{
service.Pause();
}
}

The previous snippet of code demonstrates the subtleties of pausing a service. If the service is stopped, then the state of the service is first changed to running (started). Once the service is running, then the state can be changed to paused, provided the service supports being paused. If the service is already running, then the state can be changed to paused, again provided the service can be paused.

The SWPUpdatedStatus method in its entirety is as follows:

public static void SWPUpdatedStatus(ServiceController service,
ServiceControllerStatus status,
bool handleDependents)

{
if (handleDependents)
{
foreach (ServiceController dependent in
service.DependentServices)
{
SWPUpdatedStatus(dependent,
status,
handleDependents);
}
}

service.Refresh();
switch(status)
{
case ServiceControllerStatus.Paused:
if (service.CanPauseAndContinue)
{
if (service.Status == ServiceControllerStatus.Stopped)
{
service.Start();
}

if (service.Status == ServiceControllerStatus.Running)
{
service.Pause();
}
}

break;

case ServiceControllerStatus.Stopped:
if (service.CanStop)
{
if ((service.Status == ServiceControllerStatus.Running)
||
(service.Status == ServiceControllerStatus.Paused))
{
service.Stop();
}
}

break;

case ServiceControllerStatus.Running:
if (service.Status == ServiceControllerStatus.Stopped)
{
service.Start();
}

else if (service.Status == ServiceControllerStatus.Paused)
{
service.Continue();
}

break;
}
}

Updating Service Start Mode

The SWPUpdatedStartMode method is used to update a services start mode. This method takes as parameters the service to be updated (ServiceController service), the new start mode to be specified (ServiceStartMode startMode), and a Boolean (bool handleDependents) indicating if the start mode should also be reflected in the dependents of said service.

Updating the start mode of dependents is simply a matter of iterating over the service's dependents and calling the SWPUpdatedStartMode method is a recursive fashion. The actual start mode is changed using elements of the SWPService class (the class that interfaces with the legacy, Windows service management functions). The SWPService class contains the QUERY_SERVICE_CONFIG nested class. The StartMode property of this nest class can be set and the instance of type QUERY_SERVICE_CONFIG can then be passed to SWPService.SWPChangeServiceConfig in order to update the start mode. The code associated with the SWPUpdatedStartMode method is as follows:

public static void SWPUpdatedStartMode(ServiceController service,
ServiceStartMode startMode,
bool handleDependents)
{
// Handle dependents first
if (handleDependents)
{
foreach (ServiceController dependent in
service.DependentServices)
{
SWPUpdatedStartMode(dependent,
startMode,
handleDependents);
}
}

SWPService.QUERY_SERVICE_CONFIG serviceConfig =
new SWPService.QUERY_SERVICE_CONFIG();

serviceConfig.StartMode = startMode;
int error = SWPService.SWPChangeServiceConfig(service.ServiceName,
serviceConfig,
null); // do not change password
if (error != (int)SWPService.WinError.NO_ERROR)
{
throw new Win32Exception(error);
}
}

This final method reviewed uses numerous elements of the SWPService class. This class is ultimately presented in detail in the third installment of this article.

Conclusion

The GUI presented clearly takes advantage of large number of Windows Forms related features. The administrative dialog portion of the GUI demonstrated how to implement a dialog, using RadioButton 's on a GroupBox control, and how to return results from a dialog. The main Form associated with the improved service control manager discussed the effervescent nuances of the TreeView, Splitter, and MainMenu controls.

While presenting the SWPUpdatedStartMode method, the missing link in this series of articles was revealed - namely that service start status cannot be modified using .NET classes. This leads elegantly into our third installment where the SWPService class is presented in detail. This class interfaces with the Windows service management API and therefore allows the start status to be both queried and modified.

Article Information
Author | Jan Narkiewicz
Technical Editor | Adam Ryland
Project Manager | Helen Cuthill
Reviewers | Andrew Krowczyk, Saurabh Nandu

If you have any questions or comments about this article, please contact the technical editor.


Improved Service Control Manager (SCM), Part 1

Improved Service Control Manager (SCM), Part 1
http://www.csharptoday.com/content.asp?id=1824
Article Rating: 4
Published on February 20, 2002
Abstract
This article presents an alternative implementation of the Windows Service Control Manager (SCM) found under the Control Panel’s Administrative Tools folder. This alternate SCM is implemented using .NET, specifically Windows Forms. This .NET variation of the SCM is not meant to reinvent the wheel, but instead is designed to provide functionality the stock windows SCM does not provide, in particular to give a mechanism was needed to turn off and then turn back on suites of related services. This allows us to free up resources (memory) and remove contentions for certain items (such as the FTP, SMTP and HTTP ports). Part 1 discusses: Service creation and installation using .NET, Service administration, Using .NET collections, and XML Serialization.
Article

This article presents an alternative implementation of the Windows Service Control Manager (SCM) found under the Control Panel's Administrative Tools folder. This alternate SCM is implemented using .NET, specifically Windows Forms. This .NET variation of the SCM is not meant to reinvent the wheel, but instead is designed to provide functionality the stock windows SCM does not provide. This functionality includes the ability to save the present state of a service (status started/stopped and start behavior as manual/automatic/disabled) and all services that depend on it. This service state is persisted to a file using XML serialization. This service-state file can be de-serialized and used to restore the state of a service and all services that depend on it. The alternate SCM also has the ability to manage services and dependent services as a unit (e.g. set all services and dependent services to stopped, set all services to start manually, etc.). The reason for desiring this functionality is presented later on in the article but the gist of it is that a mechanism was needed to turn off and then turn back on suites of related services. The reason was to free up resources (memory) and remove contentions for certain items (such as the FTP, SMTP and HTTP ports).

This article is broken down into multiple parts. Today's segment discusses:

  • Service creation and installation using .NET
  • Service administration
  • Using .NET collections
  • XML Serialization

The next segment of this article presents the Windows Forms application used to administer services. Broken down within this portion of the article will be the controls used, dialogs developed, and generally how the GUI interfaces with services and the XML serialization used in the application. The final installment of this article discusses .NET interoperability with legacy Windows functions. The category of Windows functions interoperated with support service management in a manner not readily exposed by .NET classes.

The technology used in developing and discussing an alternate-SCM is quite diverse and includes:

  • Service creation and installation - in order to demonstrate a SCM, a variety of services had to be developed and subsequently managed (the System.ServiceProcess namespace's ServiceInstaller, ServiceProcessInstaller, and ServiceBase classes). These demonstration-purpose services are specifically designed to utilize dependencies within services as inter-service dependencies fall within the management tasks of the alternative SCM implemented.
  • Service management - the administration of Windows services is discussed specifically as it pertains to the System.ServiceProcess namespace's ServiceController class.
  • Win32 Interoperability - the ServiceController class does not provide the functionality of Windows API associated with service management. For this reason .NET and Win32 interoperability is discussed within the context of service administration (elements of the System.Runtime.InteropServices namespace). This discussion of interoperability includes a discussion of how to retrieve structures from legacy Windows functions.
  • Unsafe Code - elements of the Windows service control manager API require C# to run as unsafe using the unsafe keyword. This lack of safety pertains to how buffers are retrieved from Windows functions and then converted to managed structures and classes.
  • Collections founder under System.Collections - classes within the oft-used System.Collections namespace will be presented based on how they are used by the alternative SCM (DictionaryEntry, SortedList, CaseInsensitiveComparer, and Hashtable classes).
  • XML Serialization - XML serialization will be used to persist and load the state of various services and their dependent services. In support of this the System.Xml.Serialization namespace will be utilized including the XmlSerializer class.
  • TreeView control - the TreeView control is used to display Windows services in hierarchal-form based on the dependencies of each service. This style of display is not provided by the standard Windows SCM.
  • Splitter control - the Splitter control is used within the alternate SCM to provide resizing of the controls on the form.
  • OpenFileDialog, SaveFileDialog - these dialogs are used to access files (OpenFileDialog class) containing objects that were serialized using XML and to specify the files used during XML serialization (SaveFileDialog class).
  • Developing custom dialogs will all be presented by showing how a service administration can be encapsulated on a Form displayed as a dialog.

Readers who are content with the standard Windows SCM should certainly find some useful .NET technical nuggets given the plethora of diverse technology to be presented. Readers who (to quote the band, Depeche Mode) "just can't get enough" of the previous technologies should take a look at other works the author of this article has contributed to:

  • Chapter 19 of "Professional VB.NET," Wrox Press (ISBN 1861004974): this chapter discusses .NET and XML. XML serialization is discussed including the use of attributes to fine tune the serialization process.

http://www.wrox.com/Books/Book_Details.asp?sub_section=1&isbn=1861004974&subject=%2ENET&subject_id=63&WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0

  • Chapters 8, 9, 10 (with Benny Johanson) of "Professional Windows Forms," Wrox Press (ISBN 1861005547): these chapters discuss the vast majority of the Windows Forms controls including those presented in this article: TreeView, Splitter, OpenFileDialog, CloseFileDialog, TextBox, MainMenu, Form, Button, RadioButton, Button, Panel, GroupBox and MenuItem

http://www.wrox.com/Books/Book_Details.asp?sub_section=1&isbn=1861005547&subject=%2ENET&subject_id=63&WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0

All methods and classes created as part of this article are prefixed by the letters SWP. Those who have read my articles recognize this as an abbreviation for my corporation's name (Software Pronto, Inc.). The reason for the prefix is so that each class or method created is not confused with the zillions classes and methods that make up the .NET Framework.

Why an Improved SCM and its Features

For every job I work as a consultant, I need to install the software on my laptop that is required for each position worked at. For example, at present the machine has four relational databases (SQL Server, MSDE the lightweight version of SQL Server, DB2, and Oracle), two web servers (Internet Information Server and Apache) and a dozen other enterprise applications. All of the aforementioned individual applications run as one or more services. Quite often it is necessary to shutdown these services to save system resources (e.g. memory) or to remove a system conflict (multiple web servers listening on socket port 80). In order to start/stop each application, an application-specific administrative tool must be launched. Those of use with Oracle know that it takes up 100 MB of RAM and adds significantly to the boot time of the machine. This is a bit of a bummer when your current focus is not Oracle development.

Then it hit me that there was a unified administrative tool for each of the applications, Windows Service Control Manager. By using the SCM, I administer my applications and set up my machine to be configured for a particular job. With this alternate form of administration come issues that are best understood with a bit of service knowledge.

First of all, each service is installed with the service control manager. As part of this install process, the services is configured to launch in one of the following ways:

  • Automatic - the services starts when the operating system starts.
  • Manual - the service much be manually launched by an administrator using the SCM or programmatically.
  • Disable - service is set not to start even if the service's services are required.

Given the various ways to start a service it was often necessary to set services that run manually or mark them as disabled to ensure resources were not being used. This situation was further complicated because one service might be dependent on another service. For example, if an attempt is made to stop the IIS Admin service using the SCM, the following dialog is displayed:

As it turns out, the Word Wide Web Publishing, Simple Main Transfer Protocol (SMTP), and FTP Publishing services all depend on the IIS Admin service. So if IIS Admin was disabled, then each service dependent on it would also have to be disabled. If IIS Admin was set to manual, then the dependent services would also have to be set to manual. If the dependent services were left as start mode, "automatic", then they would start a boot-up and cause the service on which they depend to start even though this service was set to manual. When restoring services back to their original run state, there are now four services (IIS Admin, WWW Publishing, FTP, and SMTP) that have to be set back to their original start state (automatic, disabled, or manual).

Making this even more complicated is that the start type of a service is not necessarily the present run state. A service can be started, stopped, or paused. The latter case is where a service is still running but not active. Consider the case were a service is disabled (to save resources). The original service could have been configured to start automatically, but presently paused. When the service is restored it has to be set back to its original start type and run state. Each dependent service must similarly be restored to its original state.

It would be vastly simpler if an improved SCM were around that would behave as follows:

  1. Save the start type and status of each service and dependent service to a file.
  2. Change the start type and status of entire service dependencies trees (e.g. make a service start manually and apply same behavior to all dependent services, etc.).
  3. Restore the start type and status of a tree of services from a file.

And with this need was born the SWPSCMHelper Windows Forms application.

Tutorial - Using the Improved Servce Control Manager

Before going into the gruesome details as to how this improved SCM is implemented, a quick tutorial is in order. Basically services are displayed in hierarchical form using a TreeView control (the control on the left side of the following screenshot):

In the previous screenshot, the File menu was deliberately selected because this menu contains menu items that pertain to managing the currently highlighted service, COM+ Event System, and its dependent services (System Event Notification). The specific menu items that help manage services are:

  • Persist -- save the state of a service and its dependents to a file
  • Load -- restore the state of a service and its dependents from a file

Step 1 in using our improved SCM tool is to save the state of a currently running service. So for our selected service, we select the Persist menu item. This causes a dialog to be displayed (SaveFileDialog) that allows a file name to be entered. Once the file name is chosen, then the state of the selected service is persisted.

Step 2 in using our improved SCM tool is to change the state of this service. Several reasons for doing this have already been discussed. With respect to the COM+ Event System service, this service could be disabled in order to determine what error conditions are raise when using COM+ loosely coupled events in an environment where the system administrator disabled the requisite service. To change the state of a service, we double click the service thus displaying the following dialog:

Using the previously displayed dialog, the COM+ Event System service can be disabled by:

  • Changed the Status RadioButton from Started to Stopped thus indicating that the state of the service should be stopped rather than running.
  • Change the Start Type RadioButton from Manual to Disabled thus indicating that the service cannot be started until it is re-enabled.
  • Set the Also apply settings to dependent services CheckBox to checked thus indicating that this change of status should apply to the current service and its dependents.
  • Click on the OK Button thus causing the new administrative settings for the service to be applied. The service and its dependents will now be displayed as Stopped and Disabled within the TreeView control of the application's main window.

Step 3 in using our improved SCM is to take advantage our new service state. Under the scenario discussed, code could be run at this time to determine the behavior of COM+ loosely coupled events when this features required service is not available.

Step 4 in using our improved SCM is to restore the state of our service. This is accomplished by using the File menu of the application's main window. The specific menu item selected is the Load menu item. This causes a dialog to be displayed (OpenFileDialog) which allows a file to be selected. Using this dialog the file originally persisted can be selected. This causes the state of the service to be restored. This mean that in the TreeView of the application's main window that that services related to COM+ event handling will now be displayed at Running with a start type of Manual.

Creating and Managing Services in .NET

Although this article is a bit preoccupied with services, the main thrust is service administration rather than service development. Still a set of services are developed in order to demonstrate how they are administered. Since the topic at hand is services, it should be recognized that this code will not run on the consumer operating systems that support the .NET Framework (Windows 98 and Windows ME). These services and our improved service control manager application will run on all other desktop operating systems that support .NET:

The services created for administrative purposes are contained in the SWPServiceNightmare solution and are as follows:

  • SWPQLServerLeaf01 -- depends on SWPQLServerRoot
  • SWPQLServerLeaf02 -- depends on SWPQLServerRoot
  • SWPQLServerRoot
  • SWPracleLeaf01 -- depends on SWPracleRoot
  • SWPracleLeaf02 -- depends on SWPracleRoot
  • SWPracleRoot

Numerous articles have already appeared on C# Today discussing the elements used in creating a service project, setting the properties associated with a service, and associating an installer with a service. An excellent example of article which illustrates these development tasks in entitled, "Practical .NET - a Browser History Monitor" and can be found at - http://www.csharptoday.com/?WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0.

Included with the source code for the SWPSCMHelper application are two batch files: setitup.bat and cleanitup.bat. These files install and uninstall the sample services respectively using the .NET Framework's InstallUtil utility. The InstallUtil works in conjunction with the classes exposed by the System.Configuration.Install namespace. Each sample service developed contains a project installer class not surprisingly derived from the System.Configuration.Install.Install class:

public class ProjectInstaller : System.Configuration.Install.Installer

The contents of setitup.bat batch file is as follows:

InstallUtil SWPQLServerRoot/bin/debug/SWPQLServerRoot.exe
InstallUtil SWPQLServerLeaf01/bin/debug/SWPQLServerLeaf01.exe
InstallUtil SWPQLServerLeaf02/bin/debug/SWPQLServerLeaf02.exe
InstallUtil SWPracleRoot/bin/debug/SWPracleRoot.exe
InstallUtil SWPracleLeaf01/bin/debug/SWPracleLeaf01.exe
InstallUtil SWPracleLeaf02/bin/debug/SWPracleLeaf02.exe

The contents of the cleanitup.bat batch file looks just like setitup.bat save that each InstallUtil command contains the /u command-line option (the uninstall option).

Collections

For those of us enamored with .NET there is a special place in our heart for the System.Collections namespace. Need to store data in a hash, list, queue or stack? You've got it! A prime example of the System.Collections namespace in action is the SortedList class. Want a list of key/value pairs in ordered form (a.k.a. sorted)? The SortedList is a prime candidate! This section provides a quick introduction to the collections used in this article.

The improved service control manger displays services alphabetically. Services are retrieved in array form using the ServiceController class GetServices method (return value, ServiceController []). The aforementioned method does not necessarily return the services in sorted order. The basic idea is to traverse the array of services returned by GetServices and place them in a SortList where the key value is the service name and the value stored is the ServiceController class corresponding to each service.

The constructor of the SortedList class dictates how the objects in the list are sorted. This constructor is overloaded, thus providing a variety of ways to configure sorting order. One way to control the ordering is to use the SortedList constructor which takes the following form:

public SortedList(IComparer comparer);

The idea is to specify a class which implements the IComparer interface. This interface allows two objects to be compared and is declared as follows:

    interface
            IComparer
{
int Compare(object x, object y);
}

The System.Collections namespace provides the CaseInsensitiveComparer class which implements the IComparer interface. The CaseInsensitiveComparer class compares the keys specified when adding elements to the sorted list by treating said keys as strings compared in a case-insensitive manner. The SortedList used in our SCM application is thus created as follows:

SortedList _orderServices = new SortedList(new CaseInsensitiveComparer());

Elements are administered within the sorted list in rather self-documenting fashion using methods such as:

public virtual void Add(object key, object value);

public virtual void Remove(object key);

public virtual bool Contains(object key);

The one issue with the SortedList class and our application occurs when there are multiple services with the same name. A sorted list does not permit duplicate keys. For this reason, duplicate service names had to be appended with a value that made them unique. This topic will be discussed later in the article but still not permitting duplicates (else the System.ArgumentException exception is raised) is something to be aware of with respect to the SortedList class.

The other collection used from the System.Collections namespace is the Hashtable. Like the SortedList, the Hashtable is a key/value collection. A Hashtable 's collection is stored based on the hash of the key and subsequently the values contained in the Hashtable are not guaranteed to be ordered. Like the SortedList class, the Hashtable exposes methods such as Add, Contains, and Remove. The specific hash table instances used by the improved SCM application are declared as follows:

private Hashtable _serviceTypeFilters = null;
private Hashtable _serviceTypeCounts = null;

Both of these hashes store a value associated with a key from the System.ServiceProcess namespace's ServiceType enumeration (InteractiveProcess, KernelDriver, etc.). For the serviceTypeFilters Hashtable the value is of type Boolean. If the value is true where the key is of type ServiceType then the specific ServiceType is not displayed (service type is filtered) by the improved SCM. If the value is false then that ServiceType is displayed (service type is not filtered). The SWPIsFilter method for example uses the serviceTypeFilters Hashtable in conjunction with a service ServiceType (parameter, ServiceType serviceType) type to determine if a particular ServiceType is to be filtered:

if ((ServiceType.Adapter & serviceType) != 0)
{
serviceTypeExists[ServiceType.Adapter] = true;
if ((bool)_serviceTypeFilters[ServiceType.Adapter])
{
return true;
}
}

There is one line in the previous code snippet that does not apply to filtering:

serviceTypeExists[ServiceType.Adapter] = true;

The SWPIsFilter method takes a second parameter of type Hashtable (Hashtable serviceTypeExists). This hash contains true if at least one service of the corresponding service ServiceType exists. If a ServiceType is not found, the menu item of the View menu (reviewed in the next segment of this article) corresponding to the ServiceType is disabled. For example, notice in the following screenshot that the Adaptor, File System Driver, Kernel Driver, and Recognizer Driver menu items are disabled.

The idea is that if the ServiceType does not exist then there is no menu item to allow the service to be filtered, where filtering means a service is not displayed by the improved SCM application. The SWPIsFilter method in its entirety is as follows:

private bool SWPIsFiltered(ServiceType serviceType,
Hashtable serviceTypeExists)
{
// there are no else's with the if
if ((ServiceType.Adapter & serviceType) != 0)
{
serviceTypeExists[ServiceType.Adapter] = true;
if ((bool)_serviceTypeFilters[ServiceType.Adapter])
{
return true;
}
}

if ((ServiceType.FileSystemDriver & serviceType) != 0)
{
serviceTypeExists[ServiceType.FileSystemDriver] = true;
if ((bool)_serviceTypeFilters[ServiceType.FileSystemDriver])
{
return true;
}
}

if ((ServiceType.InteractiveProcess & serviceType) != 0)
{
serviceTypeExists[ServiceType.InteractiveProcess] = true;
if ((bool)_serviceTypeFilters[ServiceType.InteractiveProcess])
{
return true;
}
}

if ((ServiceType.KernelDriver & serviceType) != 0)
{
serviceTypeExists[ServiceType.KernelDriver] = true;
if ((bool)_serviceTypeFilters[ServiceType.KernelDriver])
{
return true;
}
}

if ((ServiceType.RecognizerDriver & serviceType) != 0)
{
serviceTypeExists[ServiceType.RecognizerDriver] = true;
if ((bool)_serviceTypeFilters[ServiceType.RecognizerDriver])
{
return true;
}
}

if ((ServiceType.Win32OwnProcess & serviceType) != 0)
{
serviceTypeExists[ServiceType.Win32OwnProcess] = true;
if ((bool)_serviceTypeFilters[ServiceType.Win32OwnProcess])
{
return true;
}
}

if ((ServiceType.Win32ShareProcess & serviceType) != 0)
{
serviceTypeExists[ServiceType.Win32ShareProcess] = true;
if ((bool)_serviceTypeFilters[ServiceType.Win32ShareProcess])
{
return true;
}
}

return false;
}

The serviceTypeCounts Hashtable contains a count of each ServiceType. Each time a service of a particular type is encountered, the value of its hash table entry is incremented. These counts are later displayed in a TextBox that is part of the SCM application (the textBoxServiceStats TextBox).

The SWPUpdateServiceTypeCounts method of the SWPSCMClient.cs source file demonstrates the serviceTypeCounts Hashtable in action. The implementation of this method is fairly self-explanatory. The ServerType is passed as the SWPUpdateServiceTypeCounts method's sole parameter. A series of if statements are used to determine if a ServiceType count value should be incremented:

private void SWPUpdateServiceTypeCounts(ServiceType serviceType)
{
// increment the total service count too
_serviceCount++;
// there are no else's with the if
if ((ServiceType.Adapter & serviceType) != 0)
{
_serviceTypeCounts[ServiceType.Adapter] =
1 + (int)_serviceTypeCounts[ServiceType.Adapter];
}

if ((ServiceType.FileSystemDriver & serviceType) != 0)
{
_serviceTypeCounts[ServiceType.FileSystemDriver] =
1 + (int)_serviceTypeCounts[ServiceType.FileSystemDriver];
}

if ((ServiceType.InteractiveProcess & serviceType) != 0)
{
_serviceTypeCounts[ServiceType.InteractiveProcess] =
1 + (int)_serviceTypeCounts[ServiceType.InteractiveProcess];
}

if ((ServiceType.KernelDriver & serviceType) != 0)
{
_serviceTypeCounts[ServiceType.KernelDriver] =
1 + (int)_serviceTypeCounts[ServiceType.KernelDriver];
}

if ((ServiceType.RecognizerDriver & serviceType) != 0)
{
_serviceTypeCounts[ServiceType.RecognizerDriver] =
1 + (int)_serviceTypeCounts[ServiceType.RecognizerDriver];
}

if ((ServiceType.Win32OwnProcess & serviceType) != 0)
{
_serviceTypeCounts[ServiceType.Win32OwnProcess] =
1 + (int)_serviceTypeCounts[ServiceType.Win32OwnProcess];
}

if ((ServiceType.Win32ShareProcess & serviceType) != 0)
{
_serviceTypeCounts[ServiceType.Win32ShareProcess] =
1 + (int)_serviceTypeCounts[ServiceType.Win32ShareProcess];
}
}

The reason a series of if statements were used and not if followed by else if is because an enumeration variable can contain multiple values. For example, it is possible for the serviceType parameter to be set to both ServiceType.Win32OwnProcess and ServiceType.InteractiveProcess. If an enumeration could only contain a single value, then a switch construct would have been used instead of the more verbose if statements.

XML Serialization

The menuItemFileLoad_Click and menuItemFilePersist_Click are used to load (de-serialize) and persist (serialize) the state of a service and its dependents from/to a file. These rather popular methods are associated with the Load and Persist menu items of the MainMenu 's File menu. The SWPSerializeService class is the entity actually serialized to XML. This class contains the following data members:

  • string _serviceName - specifies the name of the service serialized or de-serialized.
  • ServiceControllerStatus _status - specifies the ServiceControllerStatus (Paused, Running, Stopped, etc.) of the service serialized or de-serialized.
  • ServiceStartMode _startMode - specifies the ServiceStartMode (Automatic, Disabled or Manual) of the service serialized or de-serialized.
  • SWPSerializeService [] _dependentServices - specifies the list of services that depend on the service being serialized or de-serialized. This list of dependent services is also serialized or de-serialized using XML Serialization. Any dependent services contained in this array of dependent services are also serialized or de-serialized. This behavior holds for all levels within the service dependence hierarchy.

Each of the previous data members is declared as having access mode, public. The reason for these data members being specified as public is that serialization and de-serialization involves only public properties and fields. The previously discussed data members are actually declared as follows:

    [XmlElement("ServiceName")]
public string _serviceName;
[XmlElement("Status")]
public
            ServiceControllerStatus _status;
[XmlElement("StartMode")]
public ServiceStartMode
            _startMode;
[XmlElement("DependentServices")]
public
            SWPSerializeService [] _dependentServices;

Each of the data members is associated with the XmlElement custom attribute. This attribute is found (class, XmlElementAttribute) in the namespace which supports XML serialization, the System.Xml.Serialization namespace. This custom attributes indicates that the data member will be serialized as an XML element and said element will be named as specified in the attribute's constructor. For example, the _status data member will be serialized as an XML element named, Status. The reason for this is that the name, Status, was passed to the constructor of the XmlElement custom attribute: [XmlElement("Status")]. The reason for changing the name is so that the XML elements generated do not start with an underscore and to capitalize each element. The leading underscore in front of each data member is a coding style which indicates that the particular entity is a data member of the class. The coding style left in place would produce atypically named XML elements.

If the XmlElement custom attribute had not been specified, the data member still would have been serialized as XML elements. The difference is the element names would literally be the same as the data member names.

When an object is de-serialized, it utilizes a parameter-less constructor for the class previously serialized. For this reason the default constructor (the constructor with no parameters) for the SWPSerializeService class is specified:

    public
            SWPSerializeService()
{
}

Before being serialized, a SWPSerializeService instance is created using the constructor with the following prototype:

public SWPSerializeService(ServiceController service)

The implementation of the previous constructor assigns the service parameter's name (ServiceName property) to the _serviceName data member of the SWPSerializeService instance:

_serviceName = service.ServiceName;

The _status data member is set in the SWPSerializeService constructor using what technically could be an ambiguous value. For example, consider the case where the ServiceController service parameter exposed a Status property set to ServiceControllerStatus.ContinuePending. The service being serialized is not paused (ServiceControllerStatus.Paused), but then again is not technically running (ServiceControllerStatus.Running). Rather than serialize a transitioning status, the next logical status will be serialized. For the case of a service that is ContinuePending, the value set for _status will be Running (the next logical service status). The code in the SWPSerializeService constructor that sets the _status data member accordingly is as follows:

switch(service.Status)
{
case ServiceControllerStatus.ContinuePending:
case ServiceControllerStatus.StartPending:
case ServiceControllerStatus.Running:
_status = ServiceControllerStatus.Running;
break;

case ServiceControllerStatus.Paused:
case ServiceControllerStatus.PausePending:
_status = ServiceControllerStatus.Paused;
break;

case ServiceControllerStatus.Stopped:
case ServiceControllerStatus.StopPending:
_status = ServiceControllerStatus.Stopped;
break;

default:
throw new Exception(
String.Format("Unknown ServiceControllerStatus: {0}",
service.Status));
}

The _startMode data member of the SWPSerializeService class cannot be retrieved from the ServiceController service parameter. This is because the ServiceController class does not have knowledge of what mode is used to start a service (Automatic, Disabled, or Manual). This knowledge under .NET is found only in the ServiceInstaller class used during service installation. For this reason, the SWPSerializeService class implemented the ServiceStartMode method. This method in turn calls the SWPService 's class's SWPQueryServiceConfig method. The SWPQueryServiceConfig method returns an instance of the QUERY_SERVICE_CONFIG class. The QUERY_SERVICE_CONFIG class exposes a property that returns the ServiceStartMode so that the SWPSerializeService class's _startMode can be initialized. The code used to setup the _startMode data member is as follows:

public static ServiceStartMode
SWPGetServiceStartMode(string serviceName)
{
SWPService.QUERY_SERVICE_CONFIG serviceConfig = null;
int result;

result = SWPService.SWPQueryServiceConfig(serviceName,
out serviceConfig);
if (0 != result)
{
throw new Win32Exception(result);
}

return serviceConfig.StartMode;
}

The SWPService class is used to access service management functions exposed by the Windows API. The implementation of this class, its QUERY_SERVICE_CONFIG sub-class and its SWPQueryServiceConfig method are discussed in the third installment of this article.

The SWPSerializeService constructor also populates an array of dependent services for the service to be serialized. The code performing this task simply iterates over dependent services found in the ServiceController service parameter's DependentServices property. This code is as follows:

_dependentServices =
new SWPSerializeService[service.DependentServices.Length];
for (int i = 0; i < _dependentServices.Length; i++)
{
_dependentServices[i] =
new SWPSerializeService(service.DependentServices[i]);
}

Performing Serialization

The SWPSerializeService class is found in the SWPSerializeService.cs source file and presented in full in Appendix A of this portion of the article.

The portion of the menuItemFilePersist_Click method that performs the actual serialization is implemented as follows:

  • Instance of SaveFileDialog is displayed using ShowDialog method so that file name to serialize to can be specified.

DialogResult result =
saveFileDialogSerialize.ShowDialog();

  • Create an instance of the XmlSerializer class that is associated with the type to be serialized (the SWPSerializeService class):

XmlSerializer serializer =
new XmlSerializer(typeof(SWPSerializeService));

  • Open the file to serialize to as a an instance of type TextWriter :

TextWriter writer = new StreamWriter(saveFileDialogSerialize.FileName);

  • Perform the actually serialization to XML by calling the XmlSerializer class's Serialize method, specifying the file to be serialized to and the class instance of type, SWPSerializeService, to be serialized:

serializer.Serialize(writer, serializeService);

The code in the menuItemFilePersist_Click method that performs serialization is as follows:

DialogResult result =
saveFileDialogSerialize.ShowDialog();

if ((result == DialogResult.Abort) ||
(result == DialogResult.Cancel))
{
return ; // noop case
}

// Serialize services and descendants here.
XmlSerializer serializer =
new XmlSerializer(typeof(SWPSerializeService));
using(TextWriter writer =
new StreamWriter(saveFileDialogSerialize.FileName))
{
serializer.Serialize(writer, serializeService);
}

One construct to note in the previous code snippet is the use of the C# using statement. All objects created in between the parentheses of the using statement are required to implement the IDisposal interface. When the end bracket of the using statement is reached, the Dispose method of the IDisposable interface is called, thus cleaning up the resource. The TextWriter class implements the IDisposable interface. When the end bracket using statement is reached the file associated with the TextWriter is closed since the object is cleaned up (disposed of). The using statement is equivalent to using a try with a finally. The difference is that the using statement is less verbose to use than a try with a finally.

Three of the sample services developed for this article are related as follows where the service SWPQLServerRoot has two dependent services (SWPQLServerLeaf02 and SWPQLServerLeaf01):

ServiceName=SWPQLServerRoot, Status=Running,
            StartMode=Automatic
DependentServices:
ServiceName=SWPQLServerLeaf02,
            Status=Running,
            StartMode=Automatic
ServiceName=SWPQLServerLeaf02,
            Status=Stopped, StartMode=Manual

When these three services are serialized, the XML generated takes the following form:

<?xml version="1.0" encoding="utf-8"?>
<SWPSerializeService xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<ServiceName>SWPQLServerRoot</ServiceName>
<Status>Running</Status>
<StartMode>Automatic</StartMode>
<DependentServices>
<ServiceName>SWPQLServerLeaf02</ServiceName>
<Status>Running</Status>
<StartMode>Automatic</StartMode>
</DependentServices>
<DependentServices>
<ServiceName>SWPQLServerLeaf01</ServiceName>
<Status>Stopped</Status>
<StartMode>Manual</StartMode>
</DependentServices>
</SWPSerializeService>

From the previous XML document, a perk of XML should clearly be demonstrated, namely that the document can be edited not only programmatically but also by a human. That's right, a human using NotePad (or any editor) could easily change a Status to Paused or a StartMode to Disabled. With features like that, XML might just catch on and become a standard (note to reader: the previous sentence was an example of the author being "sarcastic").

Performing De-Serialization

The menuItemFileLoad_Click method performs de-serialization as follows:

  • Displays the OpenFileDialog control so that the file containing the objects to be de-serialized can be specified.
  • Create an instance of the XmlSerializer class that is associated with the type to be de-serialized (the SWPSerializeService class).
  • Create a FileStream (FileStream reader) that can be used to read the file containing the objects serialized as XML.
  • Call the XmlSerilizer 's instances Deserialize method (serializer.Deserialize) to read the objects from the FileStream.
  • Call the SWPUpdatedServiceAndDescendents method so that the service state just de-serialized can be reflected in the actual services installed under Windows.

The body of the menuItemFileLoad_Click method is as follows:

if ((openFileDialogDeserialize.FileName == null) ||
(openFileDialogDeserialize.FileName == ""))
{
openFileDialogDeserialize.FileName =
saveFileDialogSerialize.FileName;
}

DialogResult result =
openFileDialogDeserialize.ShowDialog();

if ((result == DialogResult.Abort) ||
(result == DialogResult.Cancel))
{
return ; // noop case
}

SWPSerializeService serializeService;
XmlSerializer serializer =
new XmlSerializer(typeof(SWPSerializeService));
using(FileStream reader =
new FileStream(openFileDialogDeserialize.FileName,
FileMode.Open))
{
serializeService = (SWPSerializeService)
serializer.Deserialize(reader);
}

SWPUpdatedServiceAndDescendents(serializeService);

Administering Services using De-Serialized Service State

The SWPUpdatedServiceAndDescendents method takes the de-serialized state and uses it to update the currently installed services. This task is performed by finding the service to be administered. The services are stored as nodes (TreeNode instances) contained in a TreeView control. The basic idea is to iterate through the nodes in the TreeView until a match is found (using String.Compare):

    foreach (TreeNode
            candidate in treeViewServices.Nodes)
{
serviceDupFinder =
            (SWPServiceDupFinder)candidate.Tag;
if (0 ==
            String.Compare(serviceDupFinder._service.ServiceName,
serializeService._serviceName))
{
found
            = true;
break;
}
}

Update the start mode and status of the current service using static methods from the SWPServiceAdmin class. The method which updates the start mode is SWPUpdatedStartMode and the method that updated service status is SWPUpdatedStatus. The final parameter to both of these methods is set to true. This indicates that both the present service and its descendents are to be updated. The code handling this task is as follows:

SWPServiceAdmin.SWPUpdatedStartMode(serviceDupFinder._service,
serializeService._startMode,
true);
// insure service knows start mode of its desendents
serviceDupFinder._service.Refresh();
SWPServiceAdmin.SWPUpdatedStatus(serviceDupFinder._service,
serializeService._status,
true);

The service's Refresh method is called between updating the start mode and updating the status. This is to ensure that the service is current with respect to the state of its dependents. The SWPUpdatedServiceAndDescendents method in its entirety is as follows:

private void
SWPUpdatedServiceAndDescendents(SWPSerializeService serializeService)
{
SWPServiceDupFinder serviceDupFinder = null;
bool found = false;

foreach (TreeNode candidate in treeViewServices.Nodes)
{
serviceDupFinder = (SWPServiceDupFinder)candidate.Tag;
if (0 == String.Compare(serviceDupFinder._service.ServiceName,
serializeService._serviceName))
{
found = true;
break;
}
}

if (found)
{
// update present service's start mode first because when
// the status of a child might depend on present service's
// start mode.
SWPServiceAdmin.SWPUpdatedStartMode(serviceDupFinder._service,
serializeService._startMode,
true);
// insure service knows start mode of its desendents
serviceDupFinder._service.Refresh();
SWPServiceAdmin.SWPUpdatedStatus(serviceDupFinder._service,
serializeService._status,
true);
}
}

The SWPServiceAdmin class's SWPUpdatedStartMode and SWPUpdatedStatus methods are discussed in the next installment of this article. These methods do a bit more than set a property associated with a service (just a bit more!).

Conclusion

This segment of the article presented a quick overview of services, the System.Collections namespace and XML serialization. The .NET elements that glue together the service administration, collections and XML serialization are Windows Forms. The improved service control manager's GUI is implemented with Windows Forms. The GUI presented in the next installment of this article allows objects to be serialized to XML, objects to be de-serialized, and services to be administered. The final installment will cover the intricacies of administering services by having .NET interoperate with Windows service management functions.

Appendix A -- SWPSerializeService

The SWPSerializeService class from the SWPSerializeService.cs source file is implemented as follows in its entirety:

using System;
using System.ServiceProcess;
using System.Xml.Serialization;
using System.ComponentModel;

namespace SWPSCMHelper
{
public class SWPSerializeService
{
[XmlElement("ServiceName")]
public string _serviceName;
[XmlElement("Status")]
public ServiceControllerStatus _status;
[XmlElement("StartMode")]
public ServiceStartMode _startMode;
[XmlElement("DependentServices")]
public SWPSerializeService [] _dependentServices;

public SWPSerializeService()
{
}

// make this public/static so it can be used by other classes
public static ServiceStartMode
SWPGetServiceStartMode(string serviceName)
{
SWPService.QUERY_SERVICE_CONFIG serviceConfig = null;
int result;

result = SWPService.SWPQueryServiceConfig(serviceName,
out serviceConfig);
if (0 != result)
{
throw new Win32Exception(result); }

return serviceConfig.StartMode;
}

public SWPSerializeService(ServiceController service)
{
_serviceName = service.ServiceName;
_startMode = SWPGetServiceStartMode(_serviceName);
switch(service.Status)
{
case ServiceControllerStatus.ContinuePending:
case ServiceControllerStatus.StartPending:
case ServiceControllerStatus.Running:
_status = ServiceControllerStatus.Running;
break;

case ServiceControllerStatus.Paused:
case ServiceControllerStatus.PausePending:
_status = ServiceControllerStatus.Paused;
break;

case ServiceControllerStatus.Stopped:
case ServiceControllerStatus.StopPending:
_status = ServiceControllerStatus.Stopped;
break;

default:
throw new Exception(
String.Format(
"Unknown ServiceControllerStatus: {0}",
service.Status));
}

_dependentServices =
new SWPSerializeService[service.DependentServices.Length];
for (int i = 0; i < _dependentServices.Length; i++)
{
_dependentServices[i] =
new SWPSerializeService(service.DependentServices[i]);
}
}

public bool IsSameNode(ServiceController service)
{
if (_serviceName != service.ServiceName)
{
return false;
}

for (int i = 0; i < _dependentServices.Length; i++)
{
if (!_dependentServices[i].IsSameNode(
service.DependentServices[i]))
{
return false;
}
}

return true;
}
}
}

Article Information
Author | Jan Narkiewicz
Technical Editor | Adam Ryland
Project Manager | Helen Cuthill
Reviewers | Andrew Krowczyk, Saurabh Nandu

If you have any questions or comments about this article, please contact the technical editor.


Windows Management using the .NET Framework

Windows Management using the .NET Framework
http://www.csharptoday.com/content.asp?id=1816
Article Rating: 5
Published on February 7, 2002
Abstract
Systems Management has been simplified since the implementation of the Windows Management Instrumentation (WMI) as a system component in Windows 2000. With the .NET Framework was provided a set of classes allowing a quicker and easier way to access the WMI object model. Inside the System.Management namespace, we will find a full range of interesting classes giving programmatic access to WMI functionalities like query-based information retrieval and event notification. In this article, Mathieu Clerte shows how to collect and modify system information, and how to grab system events, using some of the most useful classes of the System.Management namespace.
Article

Systems Management has been simplified since the implementation of Windows Management Instrumentation (WMI) as a system component in Windows 2000. With the .NET Framework was provided a set of classes allowing a quicker and easier way to access the WMI object model. Inside the System.Management namespace, we will find a full range of interesting classes giving programmatic access to WMI functionalities like query-based information retrieval and event notification. In this article, we will learn how to collect and modify system information, and how to grab system events, using some of the most useful classes of the System.Management namespace.

Requirements

To run the attached code examples, you will need the following things:

  • The .NET Framework. The attached code was written using the 1.0.2914 core version.
  • The .NET Framework help (useful for some of the included links to work)
  • Visual Studio .NET. Even if this product is not required to run the samples, it may help to understand them. If you do possess another C# editor, you can use it as well.
  • In order to access to most of the facilities of WMI, you may need local administrator rights. To access information on other computers in your network, you need also administrative rights on these machines.
  • For Windows 9x or NT, you need the Windows Management Instrumentation Service to be installed on your system. You can find it for download on the Microsoft website.

Finally, in order to understand this article, the reader must have sufficient knowledge of C#, .NET classes and Windows forms.

About WMI

"The Windows Management Instrumentation (WMI) technology is an implementation of the Web-Based Enterprise Management (WBEM) that extends the Common Information (CIM), an industry standard to represent systems applications, devices, networks, SNMP devices and other resources in an enterprise environment." (From Microsoft Windows Management Instrumentation Scripting, Microsoft Co., 1999)

WMI consists of a unified COM programming interface model giving programming interfaces to all the necessary information on your local or remote systems. It provides key features to enable administrators to solve several daily management tasks:

  • A query capability to retrieve information, based upon WMI Query Language (WQL), which is very similar to the standard SQL query language for database queries.
  • Remote administration, using an object-oriented interface.
  • A powerful event notification, with a particular set of classes which allows request notification of very specific system events.

Using WMI, you can control the following aspects of your enterprise environment:

  • Security and user management,
  • Network management,
  • Printer management,
  • Active directory maintenance,
  • Monitor Performance Data,
  • Access SNMP devices,
  • System Health monitoring (through hardware probes like fan probes)

As the WMI Architecture has already been a subject for many previous articles (see references at the end), we will not detail it here.

The WMI Framework has a hierarchical classes structure. Among those classes, we will focus only on the Windows related classes, or Win32 classes, whose objects can be divided in five categories:

  • Hardware related classes,
  • Operating System related classes,
  • Software related classes,
  • WMI Service Management related classes,
  • Formatted and raw performance data related classes.

WMI and the .NET Framework

Before the .NET Framework, WMI services were only accessible over a COM programming interface. With the .NET Framework were released a set of managed classes that interop with the WMI Service. These classes are found within the System.Management namespace:

Some classes are used to initialise the path to WMI resource:

  • The ManagementPath class, which provides a wrapper for building paths to WMI objects.
  • The ManagementScope class, which defines the scope for management operations.

Some classes provides direct access to WMI objects:

  • The ManagementObject class, which represents a WMI data management object.
  • The ManagementObjectSearcher class, which retrieves a collection of WMI management objects, based on a specified query.

Some class are designed to execute Queries on WMI Classes:

  • The ManagementQuery class describes an arbitrary management query, from which we derive the ObjectQuery (queries on an object) and EventQuery (query on an event) classes.

Classes used to watch for WMI events:

  • The EventQuery class derived from the ManagementQuery class
  • The ManagementEventWatcher class defines a subscriber to WMI event notifications.

This can be very confusing the first time, but let us explain how to use those classes for:

  • Retrieving system information with WQL queries
  • Changing system state with WMI methods
  • Listening to system events

WQL Queries

First of all, we will see how can we retrieve local or remote system information such as:

  • Logical Disks ( Win32_LogicalDisk ) and disk partitions ( Win32_SystemPartitions )
  • Windows services and their state ( Win32_Service )
  • Running processes ( Win32_Process )
  • Environment variables ( Win32_Environment )
  • Computer system ( Win32_ComputerSystem ) and operating system ( Win32_OperatingSystem ) information.
  • Shares ( Win32_Share ) and connections to them ( Win32_ServerConnection )
  • User and Group Accounts ( Win32_Account )
  • Performance counters ( Win32_PerfFormattedData_* )

Let us build a simple Windows form C# application, consisting of:

  • a TextBox to receive the machine name
  • a TextBox to receive the query statement
  • an "Execute Query" Button
  • a RichTextBox to receive query results

Figure 1. A simple Query application

In order to execute a query through WMI, we need to follow three steps:

  • As for querying a repository, we need to define the path to it and to open a connection,
  • We must then define the query, specifying its WQL sentence.
  • We will execute the query, and retrieve its results in an object
  • We will browse the query results and output it.

To be able to use the set of classes located in the System.Management namespace, we need to add its reference to our Visual Studio project references. Moreover, we should declare it at the beginning of our code with:

using System.Management;

The path to the local or remote WMI repository must specify the machine name and the root namespace name. WMI classes are grouped in a collection of logical units called namespaces. The root namespace of the WMI classes defined in the CIM schema is root\cimv2. In this namespace, we will find the necessary classes which provides Windows system information.

The ManagementPath object defines the path to the WMI repository:

ManagementPath p = new
            ManagementPath();
p.NamespacePath = "\root\cimv2";
p.Server=
            "machine_name";

To build the path to the WMI repository, the machine name is preceded by two slashes and followed by the path of the namespace, just like a standard windows file path. So, we can also write:

ManagementPath p = new ManagementPath( "\\machine_name\root\cimv2" );

As you can see, the ManagementPath object doesn't specify any authentication information used to connect to the repository. However, the ManagementScope does. The ManagementScope takes a ManagementPath (or string to path) and a ConnectionOptions object as parameters. The ConnectionOptions will allow us to specify more details on the connection.

For example, the following code will set a Username and Password to the connection:

ConnectionOptions opt = new ConnectionOptions();opt.Username = "MyUserName";
opt.Password = "MyPassword";ManagementScope = new ManagementScope(p , opt);

If no ConnectionOptions is specified, the scope uses the current logged-in user's Windows Integrated Authentication by default for this operation. Be careful to check that the user (you) has sufficient permissions, that is local/remote administrative rights in order to log onto the WMI repository.

Now that we defined the connection, let's connect to the scope. The Connect() method will do that for us:

s.Connect();

If you don't have administrative rights on the local or remote machine, an error should be reported at this time.

We can check if the connection succeeded with the IsConnected property of the scope object, which returns a Boolean.

Now that we are connected to the WMI data source, we have to build and send our query. For that, the System.Management namespace provides the ObjectQuery class. As we said before, WQL has a very similar language structure to standard SQL. For example, to query all launched processes, we simply select all instances from the Win32_Process class, that is:

SELECT * FROM Win32_Process

Each row reported in the result of this query corresponds to a Win32_Process management object.

The ObjectQuery class from the System.Management namespace represents a WQL query. In order to build a new WQL query, we need to instantiate an ObjectQuery object and set its Querystring property to the WQL sentence:

ObjectQuery q = new
            ObjectQuery();
q.QueryString = "SELECT * FROM
            Win32_Process";

However, an overridden version of the constructor allows us to include the WQL sentence as a parameter:

ObjectQuery q = new ObjectQuery( "SELECT * FROM Win32_Process" );

Unfortunately, the ObjectQuery exists only to keep details about the query, so we need another class to execute that query: ManagementObjectSearcher . This class is responsible for executing the query, and for returning its result as a browsable object collection, through one of its Get() methods. Furthermore, this object collection is an ManagementObject collection, defined by the ManagementObjectCollection object. In our example, we simply pass the scope and the query as parameters for the ManagementObjectSearcher constructor and then call its Get() method, putting its result in a new ManagementObjectCollection through which we will iterate later.

ManagementObjectSearcher os = new ManagementObjectSearcher(s, q);
ManagementObjectCollection c = os.Get();

We can then iterate through the ManagementObjectCollection in order to list all objects - or rows - resulting from the query:

foreach (ManagementObject o in c) { . }

Every ManagementObject has a set of properties, depending on the parent WMI class. This property list can be browsed through the Properties collection. Each property is then defined as a PropertyData object, from which we will output the Name, Type, and Value:

foreach (PropertyData pd in o.Properties)
            {
output += "\t" + pd.Name + " ( " + pd.Type.ToString() + " )
            =";
if (pd.Value != null)
output += pd.Value.ToString() +
            "\n";
}

Well, for now, the code is over. Below is a piece of the possible output for the select query mentioned above:

--------------
ClassPath:\\TOTARA\root\cimv2:Win32_Process
ClassName:Win32_Process
ExecutablePath ( String ) =E:\WINNT\system32\winlogon.exe
Name ( String ) =WINLOGON.EXE
PageFileUsage ( UInt32 ) =5914624
ParentProcessId ( UInt32 ) =176
Priority ( UInt32 ) =13
ProcessId ( UInt32 ) =228
ThreadCount ( UInt32 ) =17
..
--------------
ClassPath:\\TOTARA\root\cimv2:Win32_Process
ClassName:Win32_Process
ExecutablePath ( String ) =E:\WINNT\system32\services.exe
Name ( String ) =SERVICES.EXE
PageFileUsage ( UInt32 ) =3002368
ParentProcessId ( UInt32 ) =228
Priority ( UInt32 ) =9
ProcessId ( UInt32 ) =256
ThreadCount ( UInt32 ) =28
...

I selected only a few properties because it is interesting here to compare them with their value shown by the Task Manager. Now change the WMI class in the WQL query from Win32_Process to Win32_Services . You will notice all the useful information WMI reports us about installed windows services on the system. You can try other WMI classes like Win32_Share , Win32_OperatingSystem , and Win32_ComputerSystem , and grab the information you need.

Modifying repository information

Now that we know how to retrieve information from the WMI repository, we may look forward and learn how to change it. Considering a ManagementObject , WMI allows us to modify its writable properties and to execute its methods.

Because the ManagementObject defines a generic WMI object, we cannot access its properties or methods directly ( myobject.Itsproperty or myobject.Itsmethod() ), but only through particular interfaces:

  • Two generic methods called SetPropertyValue() and Put() to change a property's value
  • A generic method caller for invoking the object's methods: InvokeMethod()

Changing object properties

All ManagementObject 's properties are read-only; fortunately, some generic method have been implemented to work as an interface for changing those properties values:

myManagementObject.SetPropertyValue( string propertyname, object propertyvalue);

which sets the value of the named property, and

myManagementObject.Put();

which commits the changes to the object (notice that this method can also be called asynchronously).

The following example shows how to modify the Operating System boot delay to 5 seconds:

ManagementObjectSearcher os = new
            ManagementObjectSearcher(@"\\localhost\root\cimv2",
"SELECT *
            FROM Win32_ComputerSystem");

foreach( ManagementObject o in
            os.Get() )
{
o.SetPropertyValue("SystemStartupDelay", 5);
o.Put();
}

If we forget to call the Put() method, all changes will be cancelled, setting properties' data to their initial values.

Environment variables can be modified like this, as is shown in Figure 2 below.

Figure 2. Setting Boot Delay or Environment Variables using WMI

Calling WMI Methods

In some circumstances, we need to execute some of the WMI 's objects methods. Some "object state" related properties, like StartupMode for a Win32_Process object, can only be modified through a method, in this case the ChangeStartMode() method. To illustrate this feature, we will see how to start or stop a windows service, change its startup mode, and terminate or launch a process. Figure 3 below shows a sample services manager very close to the original Services control panel.

Figure 3. A simple services manager

We use a simple select query on the Win32_Service class to list in a ListView object all the currently installed services by name (their DisplayName property), running state (their State property), and startup mode (their StartMode property) . Let's look at the code:

private void RefreshServices()
{
ManagementObjectSearcher os = new ManagementObjectSearcher(
            @"\\localhost\root\cimv2", "SELECT Name,
DisplayName, State,
            StartMode FROM Win32_Service" );

// Clears the
            ListView
lvServices.Items.Clear();

ListViewItem
            lvi;
string[] items = new
            string[3];

foreach(ManagementObject o in os.Get() )
{
// Adds an item in the ListView for each
            ManagementObject
items[0] =
            o["DisplayName"].ToString();
items[1] =
            o["StartMode"].ToString();
items[2] =
            o["State"].ToString();

lvi = new ListViewItem( items );
//
            We can use the Tag property to push any important data
lvi.Tag =
            o["Name"].ToString();
lvServices.Items.Add(
            lvi);
}

}

As you can see, it's a simple query to the Win32_Service class. We call an overloaded constructor for the ManagementObjectSearcher which takes for input parameters the path to the WMI resource and the WQL query, both as strings. We use the ListViewItem's Tag property to push the ManagementObject name so that we can recall it later.

To start, stop, or change a service start mode, we will respectively call the following methods from the Win32_Service class:

  • StartService()
  • StopService()
  • ChangeStartMode(string mode)

As the first two methods don't take any input parameter, we will use a very similar syntax. First, we need to retrieve the correct ManagementObject corresponding to the service we selected in the ListView. We can declare a more specific ManagementPath that contains a filter leading to the Win32_Service object; then we build a ManagementObject upon it like this:

ManagementObject mo = new ManagementObject( @"\\localhost\root\cimv2:Win32_Service.Name='"
+ serviceName + "'" );

After that, we execute the method:

// Juste replace "StartService" by "StopService" or "PauseService"
mo.InvokeMethod("StartService", null);

InvokeMethod() takes for parameters the called-method's name as a string, and its parameters in an object array. In our case, neither StartService nor StopService require parameters, which explains the "null".

However, the ChangeStartMode() function requires an input parameter: the new start mode as a string. It can be "Automatic", "Manual", or "Disabled". Assuming the variable startMode is correctly filled with one of these values, there are two different ways (and more.) to pass it to the method:

string startMode = "Manual"; // for example

Using the InvokeMethod() constructor, taking an object[] as second argument:

//Calling Method 1: an object array for
            input parameters
object[] args = { startMode }
            ;
mo.InvokeMethod("ChangeStartMode", args);

There is also another way to reach the same results. From our ManagementObject , we can use the GetMethodParameters(string methodname) function to retrieve a ManagementBaseObject which contains as many properties as there are parameters for the method we want to call. Then, we set each parameter's value using the SetPropertyValue() method we studied previously. Finally, we can use another version of InvokeMethod() 's constructor, which requires that the input parameters for the called-method ( ChangeStartMode ) should be enclosed in a ManagementBaseObject . It is a little tricky:

// OR Calling Method 2: using a ManagementBaseObject for input parameters
ManagementBaseObject inParameters = mo.GetMethodParameters("ChangeStartMode");
inParameters.SetPropertyValue("StartMode", startMode);
mo.InvokeMethod("ChangeStartMode", inParameters, null);

Anyway, both techniques work, and you can verify in the Service Control Panel that the values have been changed.

Intercepting System Events

Now it is time to jump to the last part of our WMI exploration: Monitoring system events using WMI. The fact is that many WMI classes generate events on request when a creation, modification, or deletion occurs on an instance of the class. Therefore, events from many sources can be intercepted:

  • The Win32_Process class will generate an event for each process that is started or killed,
  • The Win32_NTLogEvent class will throw an event when a modification occurs in the EventLog,
  • The Win32_Service class will generate an event when a service starts or stops,
  • The Win32_DisplayConfiguration will generate an event when the screen resolution changes
  • The Win32_LogicalDisk will throw an event when a disk is inserted

Some specific .NET classes from the System.Diagnostics namespace (for process and event log monitoring) or from the System.ServiceProcess namespace (windows services control) can also be used to report events, but this article focuses only on WMI, which gives us a wider range of action on events while we wait for new .NET classes in the replacement. An event query is required to monitor a WMI event. Very similar to a standard WQL query, it respects the following syntax:

SELECT * FROM EventClass [WHERE property = value]
[GROUP] WITHIN interval [BY property_list]
[HAVING NumberOfEvents operator constant]

The EventClass parameter is the name of a WMI system event classes. We will generally use one of the three following system classes:

  • __InstanceCreationEvent
  • __InstanceModificationEvent
  • __InstanceDeletionEvent

Those classes group events related to instance creation, modification and deletion respectively in any Win32 class.

In other words, when an instance changes in a Win32 class, a new instance is created in the corresponding system class. For example, when a windows service is stopped, the "started" property of the Win32_Service object changes from true to false and an event is created in the __InstanceModificationEvent system class, which refers to the Win32_Service object.

Below are some examples of different WQL queries:

SELECT * FROM __InstanceModificationEvent WITHIN 20 WHERE TargetInstance ISA
'Win32_DisplayConfiguration' AND TargetInstance.DisplayFrequency > 70

Every 20 seconds, this query will monitor any changes applied to the frequency of the current display when it is above 70 KHz.

SELECT * FROM __InstanceDeletionEvent WITHIN 10 WHERE TargetInstance ISA 'Win32_Process'

This one will monitor for any killed process every 10 seconds:

SELECT * FROM __InstanceCreationEvent WHERE TargetInstance ISA
'Win32_NTLogEvent' GROUP WITHIN 600 BY TargetInstance.SourceName HAVING NumberOfEvents > 25

This last query will monitor all new entries in the Event Log every 600 seconds, but only if there is at least 25 entries from the same source. These will be grouped by their SourceName property.

According to the .NET documentation, the EventQuery objects represent a WQL event query:

EventQuery myQuery = new EventQuery( "SELECT * FROM __InstanceModificationEvent WITHIN 5
WHERE TargetInstance ISA 'Win32_Process' " );

We can subscribe to this event, using the ManagementEventWatcher object:

public ManagementEventWatcher(
ManagementScope scope,
EventQuery query,
EventWatcherOptions options
);

In its most detailed form, the ManagementEventWatcher constructor requires three parameters: the management scope, the query, and eventually some options like a 'timeout'.

A ManagementEventWatcher has to be started - using its Start() method - to begin event reporting and stopped - using the Stop() method - before exiting application. The ManagementEventWatcher is capable of both synchronous or asynchronous event reporting:

  • For synchronous reporting, simply use the WaitForNextEvent() method. This method returns a ManagementBaseObject which can be used in the application.
  • For asynchronous reporting, use the EventArrived event of the ManagementEventWatcher object. On each event received, the EventArrived event is fired, and when bound to a local delegate, can fire up a customized method. The ManagementBaseObject object related to the fired event can be retrieved through one the delegate's input parameters, a EventArrivedEventArgs object.

As the asynchronous technique is by far the most interesting, the following example will focus on it. Now let us apply this method to our EventQuery . We need to instantiate a new ManagementEventWatcher :

ManagementScope localScope = new
            ManagementScope(@"\\localhost\root\cimv2");
localScope.Connect();
ManagementEventWatcher
            w = new ManagementEventWatcher(localScope, myQuery);

Before starting the watcher, we need to link its EventArrived event to an event handler, which is in fact a local method used as delegate:

w.EventArrived += new EventArrivedEventHandler(MyEventArrivedEventHandler);

We also need to declare the customized event handler:

private void
            MyEventArrivedEventHandler(object sender, EventArrivedEventArgs e)
{
// we'll see the code further in this
            article
}

Then we can start the watcher using the Start() method:

w.Start();

If the WQL sentence is correct, and the WMI class - here the Win32_Process - does exist, no error will be reported and the event listening will start. On each event received, the MyEventArrivedEventHandler method is called. By the way, we should use the benefits of this user-defined handler to get deeper details from the fired event. The EventArrivedEventArgs input parameter hides a property called NewEvent , which refers to the instance created in the WMI system class, as a ManagementBaseObject .

ManagementBaseObject scInstance = (ManagementBaseObject) e.NewEvent;

This NewEvent object has two properties, both defined as a ManagementBaseObject :

  • "TargetInstance", which refers to the current WMI Win32 class instance from which the event has been fired
  • "PreviousInstance", which in some case refers to the previous state of the TargetInstance .

The instance which we look for is the current one, so we focus on the TargetInstance property. As the Properties collection of a ManagementBaseObject object doesn't have an indexer, we need to use an alternate method, called GetPropertyValue() , to read the TargetInstance property:

object targetinstance = scInstance.GetPropertyValue("TargetInstance");

As the GetPropertyValue() method returns a generic object, we need to cast it to the type of the TargetInstance property, that is a ManagementBaseObject :

ManagementBaseObject win32Instance = (ManagementBaseObject) targetinstance;

Now that we have retrieved the WMI Win32 class instance 'source' of the event, we can browse through its properties and get more detailed information. In our example, we can write to the output its "Caption" property, a very common property to most of the Win32_* classes:

Console.Write ( win32Instance.GetPropertyValue("Caption") ) ;

A lot of WMI classes support event reporting: Win32_DisplayConfiguration , Win32_LogicalDisk , Win32_Process , Win32_CDROMDrive , Win32_Environment , Win32_Service , but the use of each one depends of your application's needs.

The following WQL query will display all the processes that are currently running:

SELECT * FROM __InstanceModificationEvent WITHIN 1 WHERE TargetInstance ISA 'Win32_Process'

Figure 4 above shows an output result of the query:

Figure 4. A simple WMI Event Listener

Last minute discovery...

I will not detail all the hierarchical structure of WMI classes, but you may look on Microsoft's MSDN Library website at http://msdn.microsoft.com/library/en-us/wmisdk/r_wmicls_6r6t.asp?frame=true&WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0

For those using Visual Studio.NET, I won't finish without inviting you to download the new add-on "Managing Windows Management Instrumentation (WMI) Data and Events with Server Explorer" (http://msdn.microsoft.com/library/default.asp?url=/library/en-us/WMISE/wmioriManagingWindowsManagementInstrumentationWMIDataEventsWithServerExplorer.asp&WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0). As you will see, this very useful add-on adds two nodes to the Server Explorer in Visual Studio .NET:

  • Management Classes, which lists all WMI class instances by categories, giving for each instance its properties through the Properties window with real-time read and write facilities,
  • Management Events, which allows creation of new event queries via the "Add Event Query..." item of the associated contextual menu.

Each Management class instance or event query can be drag and dropped onto an open form. For event queries, a new ManagementEventWatcher object is created. For WMI class instances, each class is related to an associated code file which is added automatically in your project. For example, if you drag and drop an instance of the Processes class, a Win32_Process.cs file is added to your project, and a new [yournamespace].ROOT.CIMV2.Process object is created in your code.

Conclusion

In this article, we explained how to use the .NET classes as programming interfaces with the Windows Management Instrumentation service. We learned how to use different classes from the System.Management namespace in order to retrieve system information using WQL queries, changing system state with WMI methods, and listening to system events via WMI. WMI allows a huge range of user-system interaction, but unfortunately is still used below its capacity by developers. A lot of specific information can be brought by the WMI Win32 classes, and the difficulties might be more in finding the right class than in using it. Even if some native .NET classes can be used to retrieve information from the system, we should not forget the power offered by WMI through the System.Management namespace. I hope this article will open some doors for people who write from system management tools to desktop applications.

Article Information
Author | Mathieu Clerte
Technical Editor | Adam Ryland
Project Manager | Helen Cuthill
Reviewers | Saurabh Nandu, Ramanam P

If you have any questions or comments about this article, please contact the technical editor.


Introducing .NET Regular Expressions with C#

Introducing .NET Regular Expressions with C#
http://www.csharptoday.com/content.asp?id=1812
Article Rating: 4.5
Published on February 1, 2002
Abstract
If you’re building any kind of application that involves looking for patterns in text, picking out segments of text according to certain criteria, or transforming the text itself, you can save a lot of time and energy by becoming familiar with regular expressions. In this article, Tony Loton demonstrates the Regular Expression language itself, and also introduces the main classes of the .NET System.Text.RegularExpressions namespace that allow you to harness the power of regular expressions from within your C#, Visual Basic or C++ programs.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

If you're building any kind of application that involves looking for patterns in text, picking out segments of text according to certain criteria, or transforming the text itself, you can save a lot of time and energy by becoming familiar with regular expressions.

In this article, I'll demonstrate the Regular Expression language itself, which is not unique to the .NET framework. That discussion may well be of interest if you're thinking of using the RE engine of the JDK 1.4, or if you're working with one of the older languages - like AWK or Perl - that support regular expressions. Then I'll introduce the main classes of the .NET System.Text.RegularExpressions namespace that allow you to harness the power of regular expressions from within your C#, Visual Basic or C++ programs.

Introduction

Consider the following problem: how many lines of code do you think you would need in order to transform all the $ (dollar) amounts in the following sentence from the form "$10" to the form "10 US dollars"?

Each item is priced at $10 but you can purchase ten for only $80 which
is much cheaper than 10 * $10 = $100.

One solution would be to loop through the string using strtok (for C) or StringTokenizer (for Java), read ahead from each $ symbol that you found up to the next word boundary, and meanwhile build a second version of the string using concatenation.

Or, you could use the .NET Regular Expression engine and this single line of code

String newText = Regex.Replace(sourceText, @"\$(?<amount>\d*)"
,"${amount} US dollars");

Which would give:

Each item is priced at 10 US dollars but you can purchase ten for only 80 US dollars which
is much cheaper than 10 * 10 US dollars = 100 US dollars.

Other practical uses include parsing (of HTML / XML content, or even natural language), feature extraction (to pick out names and addresses from documents), and even the humble search-and-replace operation of a text editor.

On the subject of HTML and XML, in a previous article entitled "Working with Web Data in C#" (http://www.csharptoday.com/content.asp?id=1808&WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0) I hinted at a practical use for regular expressions to extract data from web pages. In the following screenshot, the SELECT and MATCHES clauses both contain regular expressions.

The text " .html#0.table#1.tr#0.td#1.table#1.tr#1.td#0.table#0.tr#\d+.td#0$ " is a regular expression that matches certain HTML elements - in this case book titles from the Wrox web site - according to their positions on the page.

The text " \w*.NET\w* " is a regular expression that picks out only those titles that mention .NET.

Later you'll see that the RE language elements used in those examples are:

\d (as in .tr#\d+ above) - a character class that matches any decimal digit.

+ (as in .tr#\d+ above) - a quantifier that matches one or more of the preceding character.

$ (at the end of the SELECT string) - an atomic zero-width assertion that ensures a match up to the end of the string.

\w (as in \w*.NET\w*) - a character class that matches any word character.

* (as in \w*.NET\w*) - a quantifier that matches zero or more of the preceding character.

Regular Expressions Language Elements

We'll now look a little deeper at the elements that comprise the regular expressions language, which the .NET documentation splits up into these categories:

  • Character Classes and Character Escapes
  • Substitutions
  • Atomic Zero Width Assertions
  • Quantifiers
  • Grouping Constructs, Backreference Constructs and Alternation Constructs

The aim is not to provide an exhaustive account of each language element, but to demonstrate the use of each kind of element via a simple concrete example. For consistency, I'll make use of a single test sentence throughout, which I introduced earlier as:

Each item is priced at $10 but you can purchase ten for only $80 which
is much cheaper than 10 * $10 = $100.

After explaining the regular expression syntax, I'll show you the C# code that I used to drive the examples.

Character Escapes

The characters . $ ^ { [ ( | ) * + ? \ have special meanings as operators in regular expressions. You will see later how $ is an Atomic Zero-Width Assertion and how * is a Quantifier. This leaves us with a problem if we want to use a regular expression to discover, for example, the occurrences of the literal $ character in a given text. We solve the problem by prepending a backslash to the front of any operator that we wish to match literally.

Consider our test sentence:

Each item is priced at $10 but you can purchase ten for only $80 which
is much cheaper than 10 * $10 = $100.

A regular expression "\$" (ignore the quotes, and notice the backslash) will match four times, corresponding with the four occurrences of the $ sign, whereas a regular expression of "$" (no backslash) will match only once as the special end-of-string assertion. More about end-of-string later.

Similarly, the regular expression "\*" will match once, whereas "*" will trigger an exception in C# / .NET with the following description (because the "*" has been treated as a quantifier rather then a literal character).

An unhandled exception of type 'System.ArgumentException' occurred in system.dll. Additional information: Parsing "*" - Quantifier {x,y} following nothing.

So, any operator character will be treated literally when escaped, that is preceded by a backslash.

For a complete list of Character Escapes, look in the "Character Escapes" section of the ".NET Framework General Reference".

Character Classes

We may wish to treat certain groups of characters as being of the same type, or class, of characters, and thus form a regular expression that matches any character of a particular class. For example, we might wish to treat characters 0 to 9 as belonging to the "decimal digit" class.

The general syntax for character classes uses the square brackets [ and ] to show that certain characters belong to the same class. For decimal digits we could therefore use the regular expression "[0123456789]" which would match 11 times in our test sentence for each of the 11 decimal digits.

For convenience we could shorten this to the range "[0-9]" (or a more limited range of say "[0-1]" for binary numbers) and we could reverse the sense in each case - that is, find non-digits - by adding the "^" character to give [^0123456789] or [^0-9].

You might be interested to know that we could instead take advantage of the special escape sequences "\d" (for decimal digits) and "\D" (for non-digits).

For a complete list of Character Classes, look in the "Character Classes" section of the ".NET Framework General Reference".

Atomic Zero-Width Assertions

Atomic Zero-Width Assertions cause a match to succeed or fail depending on the current position in the string. We've already met one of these, the $ character that matches the end-of-line or end-of-string position providing it is not preceded by a backslash. Do you remember the following result?

Technically that result was true, because there is exactly one end-of-string position in our sample text.

Now suppose we wanted to count up the number of words in the input text. We can use the assertion \b to ensure that a regular expression matches at a word boundary, like this:

We're saying that each collection of one or more characters A-z - i.e. [A-z]+ - that has a word boundary \b before, and after, counts as a word. Note that there are only 17 such words (comprising only letters) in the input text.

Note also the use of the + symbol to specify "one or more", which will lead us neatly onto quantifiers in the next section.

For a complete list of Atomic Zero-Width Assertions, look in the "Atomic Zero-Width Assertions" section of the ".NET Framework General Reference".

Quantifiers

Quantifiers act like multiplicities in database or object modeling, allowing you to specify "one or more", "zero or more", "exactly 2", and so on.

To find any dollar amounts in our input text that are multiples of $100 dollars (including $200, and up to $900) we could use the regular expression \$\d0{2}\b that matches a literal $ sign followed by a single digit, followed by exactly 2 zeros and a word boundary.

So we've matched $100, but not £10 or $80, and the word boundary is a necessary inclusion to avoid matching values of the form £1001

For a complete list of Quantifiers, look in the "Quantifiers" section of the ".NET Framework General Reference".

Grouping Constructs and Substitutions

Suppose we wanted to reformat the dollar amounts in our test sentence to replace the prefixed symbol "$" with postfixed text "US dollars". The .NET API allows us to do that with a piece of code like this:

String newText = Regex.Replace(sourceText, @"\$(?<amount>\d*)"
,"${amount} US dollars");

That single line of code transforms our test sentence as follows.

Each item is priced at $10 but you can purchase ten for only $80 which
is much cheaper than 10 * $10 = $100.

The sentence now becomes:

Each item is priced at 10 US dollars but you can purchase ten for only 80 US dollars which
is much cheaper than 10 * 10 US dollars = 100 US dollars.

The regular expression we're searching for is "\$(?<amount>\d*)". In words it means "match a literal dollar sign followed by a group of characters (named "amount") comprising zero or more decimal digits". Think of amount as a variable that will contain the decimal amount (without the $ sign) for each match.

We could write some .NET code to step through the matches one-by-one, and for each one we could pick out the value of the amount grouping. Or, as we have done in the code above, we could simply specify a replacement string to substitute the matched text with our new format. That simple substitution should be quite self-explanatory.

As an aside, you might be wondering why the regular expression string was preceded by the @ symbol in our method call. That's how we tell the C# compiler not to interpret backslashes as its own escape characters, and to preserve them to be interpreted by the Regex engine. This is not necessary if you're using Visual Basic.

For a complete list of Grouping Constructs and Substitutions, look in the "Grouping Constructs" and "Substitutions" sections of the ".NET Framework General Reference".

Backreference Constructs

The grouping constructs mentioned above allow us to mark out sections of text with variable names, so that we can refer to them in our code or for use in substitutions as we've seen. We can also make use of groups by referring back to them within the same regular expression.

A classic example is to find instances of double letters in words with a regular expression like (?<letter>[A-z])\k<letter>. When run against our test sentence we get no matches because there are no words with double letters.

But, if we run it against the substituted version (having $ symbols replaced by "US dollars" text, see above) we get this result corresponding with the four occurrences of the word "dollars".

In our regular expression we defined a group named "letter" comprising any character A-z, and we referred back to it with the backreference \k<letter> as the next matching character. The \k backreference succeeds if the subsequent character is the same as that matched by the prior grouping construct of the same name, in this case "<letter>".

For your information, we could have used a simpler version ([A-z])\1 instead, which backreferences the 1st unnamed group.

For a more practical purpose, backreferences could be used to pick out content from between matching tags in a HTML or XML document using a regular expression like this one.

"<(?<tag>\w+)>(?<content>(.|\n)*?)</\k<tag>>"

For a complete list of Backreference Constructs, look in the "Backreference Constructs" section of the ".NET Framework General Reference".

Alternation Constructs

To understand the regular expression that I've just shown you, you really need to understand the meaning of the pipe "|" symbol. It's an alternation construct meaning "OR".

As a simpler example, consider the number of times the number 10 appears in our test sentence. I count four, if we include the spelled-out word "ten" in the calculation.

That regular expression looks for occurrences of the text "ten" or "10" followed by a word boundary.

For a complete list of Alternation Constructs, look in the "Alternation Constructs" section of the ".NET Framework General Reference".

.NET Regular Expression Classes

Once you understand how to build regular expressions, you'll need a mechanism for driving them from you're .NET program, so we'll look briefly at these .NET classes included in the System.Text.RegularExpressions namespace:

  • System.Text.RegularExpressions.Regex
  • System.Text.RegularExpressions.Match
  • System.Text.RegularExpressions.Group
  • System.Text.RegularExpressions.RegexOptions

Regex class encapsulates a regular expression string and a set of options. You can look for occurrences (returned as matches) of the regular expression pattern within an input text using instance or static methods of the class.

RegexOptions provide enumeration values that may be combined to affect the matching operations, for example to run in case sensitive or case insensitive mode.

Match class instances represent occurrences of the regular expression pattern within the input text and provide access to text segments via group names or numbers (demonstrated in the Review and Further Work section later).

Group class instances represent text segments from the input text that were mapped to group name or numbers in the regular expression. Each match may have several groups associated with it as demonstrated in the Review and Further work section later.

The first sample application, which I'll list next and which was used to drive the previous examples, makes use of the following classes from that namespace:

Sample Application #1 - RegexCounter

To demonstrate the various regular expressions language elements, I used a simple C# / .NET program that took a regular expression as input, and which produced as output a message box like this:

You might like to try out the examples for yourself and experiment with some of your own, for which you'll need the following program listing:

using System;
using System.Text.RegularExpressions;
using System.Windows.Forms;

namespace RegularExpressions
{

public class RegexCounter
{

public static void Main(String[] args)
{
String sourceText ="Each item is priced at $10 but you can purchase ten for only $80 which
is much cheaper than 10 * $10 = $100.";

RegexCounter.countRegex(sourceText, @"\$\d0{2}\b",true);
}

That Main() method should be easy enough to understand. All the interesting work is done in the next method:

public static void countRegex(String sourceText, String regexText
, bool ignoreCase)
{

First of all we can set some RegexOptions, for example to make the matches case-sensitive or case-insensitive, before creating the Regex instance:

      RegexOptions
            options = 0;
if (ignoreCase) options =
            RegexOptions.IgnoreCase;

Regex countRegex = new
            Regex(regexText,options);

Next we create a Match instance to match occurrences of the regular expression in our source text:

Match countMatch = countRegex.Match(sourceText);

We step through each successful match, incrementing the count as we go:

int count=0;
for ( ; countMatch.Success; countMatch
= countMatch.NextMatch()) count++;

Finally we display the result.

MessageBox.Show("The regular expression '"+regexText
+"' appears "+count+" times.");

}
}
}

That code may be found in the RegexCounter.cs source file provided, and you can drive it using the RegexTester.cs file also provided.

You can find out more about the classes used by looking in the"System.Text.RegularExpressions Namespace" section of the ".NET Framework Class Library".

Sample Application #2 - TagParser

The previous sample application that counts up the number of occurrences of a given regular expression was sufficient for the basic demonstration. In any practical application we'd want to know a little more about each matching instance of the regular expression, and in particular we might want to gain access to the text segments enclosed within named groups.

Remember this regular expression that I showed earlier?

"<(?<tag>\w+)>(?<content>(.|\n)*?)</\k<tag>>"

It's used by the next class to pick out matching tag pairs from an input string and to print out the tags with appropriate indents to show the levels of embedding. A method call as follows:

RegexSamples.showTags("<HTML><HEAD><TITLE>Sample HTML
Table</TITLE></HEAD><BODY><TABLE><TR><TD>row0,col0</TD><TD>row0,col1</TD></TR>
<TR><TD>row1,col0</TD><TD>row1,col1</TD></TR></TABLE></BODY>/</HTML>","");

Will result in console output of:

<HTML>
<HEAD>
<TITLE>
</TITLE>
</HEAD>
<BODY>
<TABLE>
<TR>
<TD>
</TD>
<TD>
</TD>
</TR>
<TR>
<TD>
</TD>
<TD>
</TD>
</TR>
</TABLE>
</BODY>
</HTML>

Hopefully that provides a more realistic example of something you might do with regular expressions, and the code is as follows:

using System;
using System.Text.RegularExpressions;

namespace RegularExpressions
{
public class TagParser
{
public static void showTags(String text, String indent)
{

There we defined a TagParser class within a RegularExpressions namespace, having a single showTags(.) method. Next we create an instance of our regular expression:

Regex tagRegex =
new Regex(@"<(?<tag>\w+)>(?<content>(.|\n)*?)</\k<tag>>");

We're looking for:

  • A group (called <tag>) of one or more (+) word characters (\w) within angled brackets < and > , followed by
  • A group (called <content>) of zero or more (*?) any character or newline (.|\n), followed by
  • A / character and the backreferenced (\k) content of the <tag> group within angles brackets < and >.

In a nutshell, we want to find matching <anytag> and </anytag> combinations with enclosed content. If you're wondering about the question mark (?) character that follows the asterisk (*) in our regular expression, I'll explain that shortly.

Now we match the regular expression against our input text and loop through the matches:

      Match
            tagMatch = tagRegex.Match(text);

for ( ; tagMatch.Success;
            tagMatch = tagMatch.NextMatch())
{

In the following line, take note of the tagMatch.Groups["tag"] method invocation to extract the text that matches our regular expression's <tag> group:

System.Console.WriteLine(indent+"<"+tagMatch.Groups["tag"]+">");

Because the content enclosed between two matching tags may itself contain other pairs of matching tags, we're now doing some clever recursion by re-entering the showTags(.) method with the text of the <content> group.

showTags(""+tagMatch.Groups["content"],indent+" ");

And when we wind back out of the recursion we print the closing tag at this level.

System.Console.WriteLine(indent+"</"+tagMatch.Groups["tag"]+">");
}

Now close off the method, class, and namespace:

}
}
}

That recursive approach is not the most performant solution, and for large input files the call stack will grow and grow, but it's the neatest way to demonstrate the principle. The code may be found in the TagParser.cs source file provided, and you can drive is using the RegexTester.cs file also provided.

Lazy Quantifiers and the Unexplained Question Mark

That second example illustrates the importance of lazy quantifiers, like *?, that match the minimum number of repetitions.

Assuming TR as a value matched by the <tag> group in our regular expression, the <content> group would first match the text shown in bold below because it is enclosed within <TR> </TR> combination.

<TR><TD>row0,col0</TD><TD>row0,col1</TD></TR><TR><TD>row1,col0</TD><TD>row1,col1</TD></TR>

And the next match would be.

<TR><TD>row0,col0</TD><TD>row0,col1</TD></TR><TR><TD>row1,col0</TD><TD>row1,col1</TD></TR>

That's exactly what is needed to give our desired end result, but if we omitted the question mark to leave (.|\n)* in the regular expression, the first match for the <content> group would be the whole of:

<TR><TD>row0,col0</TD><TD>row0,col1</TD></TR><TR><TD>row1,col0</TD><TD>row1,col1</TD></TR>

Technically that is correct because it is content enclosed within matching <TR> and </TR> tags, but not at all what we wanted, and the final output in that case would be as follows. Look carefully at how the <TR> and <TD> tags are indented.

<HTML>
<HEAD>
<TITLE>
</TITLE>
</HEAD>
<BODY>
<TABLE>
<TR>
<TD>
<TD>
</TD>
</TD>
</TR>
</TABLE>
</BODY>
</HTML>

That was a practical demonstration of the subtle difference between a lazy quantifier (*?) and a greedy quantifier (*). The former matches the shortest text segment that meets the criteria, and the latter matches the longest text segment that matches the criteria.

Conclusion

Over the years, many programming and scripting languages -- like AWK, Perl and Python - have included Regular Expression as part of their fabric.

Many other languages and runtime environments - like the Java SDK1.4 and C#/VB/C++/.NET - have been furnished with a regular expressions capability, and with good reason. Regular Expressions provide a very powerful mechanism for parsing, transforming, and otherwise manipulating text with little coding effort.

For those who are new to the regular expressions, I've tried to demystify them a little via simple, concrete examples of the various RE language elements. For those familiar with regular expressions, but unfamiliar with the .NET supporting classes, I've introduced the main classes from the System.Text.RegularExpressions namespace.

If this article has whetted your appetite you will find plenty of documentation and some more examples in the official .NET documentation, and you might like to revisit my previous "Working with Web Data in C#" (http://www.csharptoday.com/content.asp?id=1782&WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0) article to try out a few more regular expression in a real-life scenario.

Article Information
Author | Tony Loton
Technical Editor | Adam Ryland
Author Agent | Charlotte Smith
Project Manager | Helen Cuthill
Reviewers | John Boyd Nolan, Phil Sidari

If you have any questions or comments about this article, please contact the technical editor.


Generating AIM Buddy List import file with Active Directory and C#

Generating AIM Buddy List import file with Active Directory and C#
http://www.csharptoday.com/content.asp?id=1809
Article Rating: 4.5
Published on January 29, 2002
Abstract
Active Directory Service is a very important new feature in Windows 2000. It is a centralized location for storing, managing, and locating information and resources in an enterprise. It allows a single network logon and a single point of administration for the network users and managers. This means that the administrators can just create an account in one place, then the user will be able to access all the difference resources with the same account. It also serves as an organized naming service for looking up resource locations. Its services range from locating a printer that’s on the same floor as the user, to locating a database server for an application that’s on the other side of the planet. In this article, Tin Lam looks as Active Directory Service, and uses it to create an application that maintains a list of screen names for AOL's Instant Messenger application.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

Active Directory Service is a very important new feature in Windows 2000. It is a centralized location for storing, managing, and locating information and resources in an enterprise. It allows a single network logon and a single point of administration for the network users and managers. This means that the administrators can just create an account in one place, then the user will be able to access all the difference resources with the same account. It also serves as an organized naming service for looking up resource locations. Its services range from locating a printer that's on the same floor as the user, to locating a database server for an application that's on the other side of the planet.

Where did the Microsoft Active Directory come from?

Why do we need Directory Services?

About two decades ago, nobody had implemented any directory services yet. A lot of organizations already had some resourceful network systems, but working in those organizations as a user might have meant having to memorize a dozen different usernames and passwords. Working as a networking manager might have meant to follow a huge checklist for all the accounts creation and termination for each employee. This is not to mention the problems if the organization wanted to deploy any new services. It's troublesome for the users and a nightmare for the administrators. Clearly, companies back then demanded a solution to solve these problems.

Directory Services in general

The idea of "directory" in computing terms is exactly the same as the "directory" we use in our everyday life, like a phone book. A directory service is a set of information that is organized in a hierarchical fashion. It's usually optimized for rapid read access. In 1986, VINES had created the first commercially available Network Services, known as StreetTalk. It integrated all the network services and centralized all the resource management in its VINES NOS, such as user authentications, authorizations, files and printers access, emails, etc. The VINES networks rapidly became popular. Other companies such as Novell and Microsoft started implementing their own directory services some years later. However, they all implemented the directory services with different formats and APIs. Clearly the companies not only needed directory services, but also a standard.

LDAP, Lightweight Directory Application Protocols

In 1988, a standard directory format had been created and called the X.500. However, it was too complex for the market to adopt. Then, in 1994, the LDAP standard was created, which is a greatly simpilfied version of X.500. LDAP is a protocol that runs on top of TCP/IP. It later became the RFC 1487. A LDAP API has also been developed, which became the RFC 1823. It sets us free to build directory-enabled applications with the same set of APIs, and have them run and integrate with other directories from different vendors.

Microsoft Active Directory Services

Microsoft took the idea of LDAP a step further and integrated it in most of its own products; the result is the Microsoft Active Directory Services. A few examples are the Windows 2000 server itself, the DNS server, and the Exchange Server. An active directory server can be configured to replicate its entries to all other active directory servers. An administrator can therefore manage user accounts from any server (Single Point of Administration) and abandoning the need of PDC and BDC. The Microsoft Exchange Server now shares the same user database as the Windows 2000 server itself, eliminating the need to create two different accounts for the same user. The Microsoft DNS Server can now store entries in an Active Directory for faster replications and dynamic update. More information can be found at: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnactdir/html/msdn_actdirintro.asp&WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0.

Programming the Microsoft Active Directory?

ADSI, Active Directory Service Interfaces

ADSI was shipped by Microsoft in February 1997. It's a higher level API than the LDAP API in C. It's a lot easier to program while retaining the full support from LDAP API. Although the ADSI is a higher-level API than LDAP, the ADSI routing layer can efficiently handle the calls from the application to the service providers resulting in almost no performance degradation. ADSI comes with 4 service providers: WinNT, NDS, NetWare bindery and LDAP. Other service providers such as the Exchange connectors, IIS, etc, can be downloaded separately. To the service providers, ADSI provides a set of open interfaces for them to implement. To the developers, ADSI provides a set of COM interfaces to the directory objects, encapsulating the underlying different directory service formats, and provide a unified API to access them. Therefore, developers can build directory-enabled applications with ADSI in any COM aware language. In windows, Microsoft also provided two GUI utilities called the ADSI Editor and the ADSI Viewer for editing and viewing the data easily. They can be found in the Supporting Tools folder in the Windows 2000 CD.

The System.DirectoryServices namespace

The System.DirectoryServices namespace defines fifteen classes and five enumerations in the .NET Framework Beta 2. It provides access to the Active Directory for all the .NET languages. It works with any service providers that implement the ADSI. The system-supplied service providers are: IIS, LDAP, NDS and WinNT. Note that the classes in this namespace make use of ADSI.

The program

AIM, or the AOL Instant Messenger, has become more and more popular in the past few years. Many organizations have adopted it as an alternate way to exchange messages within the organization other than email. However, as the organizations grow, maintaining the most updated lists of screen names of the employees from all the departments and groups can be a time consuming task. Wouldn't it be nice to manage the screen names at the same place as the users' accounts? In this article, we are making use of a little application that retrieves all the user groups from an active directory domain, and gets all the users for each group. Then for each member, it examines the values in the description field. If it starts with "AIM: ", then we will parse out the rest of the string. It then prints a string that can be redirected to a file. We will be importing the file to the AIM software ourselves. The imported screen names in AIM will be ordered by their user group in the active directory domain.

The program was developed and tested with Microsoft .NET framework beta 2.0 and AOL Instant Messenger 3.0, and also with the Microsoft Windows 2000 Advanced Server, with Active Directory installed and configured. We will start by looking at ActiveDirectoryHelper.cs, which contains methods to get a list of groups and users.

ActiveDirectoryHelper.cs

/// <summary>
/// This class
            contains methods to be called by the GetScreenName
            executable.
/// </summary>

using System;
using
            System.DirectoryServices;

public class ActiveDirectoryHelper
            {

private String _adServer;
private String _dn;
private
            String _username;
private String _password;

private
            DirectoryEntry rootEntry;
private DirectorySearcher
            search;

/// <summary>
/// Constructor of the
            class.
/// It initializes the class with the server connection
            information.
/// </summary>
/// <param
            name="pAdServer">the server name</param>
/// <param
            name="pDN">the path to an entry in the directory to start
            with</param>
/// <param name="pUsername">the username
            to be authorized with</param>
/// <param
            name="pPassword">the password of the
            username</param>
public ActiveDirectoryHelper(String
            pAdServer, String pDN, String pUsername, String pPassword)
            {

_adServer = pAdServer;
_dn = pDN;
_username =
            pUsername;
_password = pPassword;

rootEntry = new
            DirectoryEntry();
// setting the server and directory
            location
rootEntry.Path = "LDAP://" + _adServer + "/" +
            _dn;
// setting the username and password for authenication and
            authorization
rootEntry.Username =
            _username;
rootEntry.Password = _password;
}

///
            <summary>
/// It searches the directory with the connection
            configuration
/// initialized from the constructor, for all the
            user groups.
/// </summary>
/// <returns>a
            SearchResultCollection containing all the
            groups</returns>
public SearchResultCollection
            GetAllUserGroups() {

search = new
            DirectorySearcher();
search.SearchRoot = rootEntry;
// for the
            entries in the DN that we use in this article,
// if the
            objectClass is not user,
// then the entry is a group of some
            sort
search.Filter = "(!(objectClass=user))";
// searching the
            entire tree under the DN
search.SearchScope =
            SearchScope.Subtree;
// only loading the property member and
            name
search.PropertiesToLoad.Add("member");
search.PropertiesToLoad.Add("name");
return
            search.FindAll();
}

/// <summary>
/// Connecting
            to the server with the DN specified in userDN,
/// only retieving
            a single user entry here.
/// </summary>
/// <param
            name="userDN">a string that represents a DN of an
            user</param>
/// <returns>a DirectoryEntry object
            retrieved from the DN specified in the
            parameter</returns>
public DirectoryEntry GetUser(String
            userDN) {

DirectoryEntry userEntry = new
            DirectoryEntry();
userEntry.Path = "LDAP://" + _adServer + "/" +
            userDN;
userEntry.Username = _username;
userEntry.Password =
            _password;
return userEntry;

}
}

public ActiveDirectoryHelper(String pAdServer, String pDN, String pUsername, String pPassword) is the constructor of the class. It first sets the path specifying the protocol, the server, and the DN, the Distinguished Name (we will be covering DN soon) of the entry in the directory we will be performing actions on. The username and password to access the directory can be overridden by setting the Username and Password property.

public SearchResultCollection GetAllUserGroups() will perform a search on the entry we had specified in the constructor. SearchRoot is to set as the root of the search. Filter is to narrow the criteria of the search. It's in LDAP syntax, which is defined in RFC 2254. All conditions must be enclosed in parentheses. SearchScope indicates the scope that we would like to perform the search on. Here, we would like to search all the entries in the entire tree under the SearchRoot, so we assign it a value of SearchScope.SubTree. It can be one of these: SearchScope.SubTree, SearchScope.Base (only the properties of the SearchRoot entry) or SearchScope.OneLevel (only one level in the tree under the SearchRoot entry). PropertiesToLoad is used to specify which properties we would like to retrieve. We are only retrieving the member and name of the entries, which are the groups. member holds a collection of DN of its members, while name holds the name of the group. Finally FindAll() is the method we can call to execute the search. It will find all the entries in the directory that match the criteria. It returns an object of type SearchResultCollection.

public DirectoryEntry GetUser(String userDN) will return us an object of type DirectoryEntry. We will pass it a DN. The DN is a string representing the path of the Users Object in the directory, which has the common name (CN) as Users, in the active directory domain named go-ms.net. On my machine, my active directory domain go-ms.net is a valid Internet domain, which has two components GO-MS and NET. It's represented by the Domain Components (DC) DC=GO-MS, DC=NET. If it's not a valid Internet domain, I would have used DC=GO-MS, DC=local instead. For the DN like CN=tommy chan, CN=Users, DC=GO-MS, DC=NET, it represents the object with the CN being "tommy chan" contained in an object with the CN being "User" in the go-ms.net domain. The full specification and the syntax of DN is defined in RFC 2253.

GetScreenName.cs

ActiveDirectoryHelper.cs is only a helper class. It provides methods to retrieve a user and groups from the Active Directory. The logic to deal with the AIM happens in GetScreenName.cs, which is compiled to be an executable.

/// <summary>
/// This class is
            compiled to be an executable.
/// It calls the methods in the
            ActiveDirectoryHelper class,
/// to retrieve and to generate a
            list of screen names for AIM import.
///
            </summary>

using System;
using
            System.DirectoryServices;

public class GetScreenName
            {

/// <summary>
/// The entry point of the
            class.
/// It's used to generate the list for the import.
///
            </summary>
public static void Main(String[] args)
            {

if ( args.Length < 4 )
throw new Exception("Not
            enough arguments");

ResultPropertyValueCollection
            members;
SearchResultCollection groups;
DirectoryEntry
            userEntry;
ActiveDirectoryHelper ad;
String ScreenName,
            ScreenNames, aimGroupList;

// initializing and setting the
            infos required to locate the server and the directory
ad = new
            ActiveDirectoryHelper(args[0], args[1], args[2], args[3]);
//
            getting all the user groups from the server
groups =
            ad.GetAllUserGroups();

aimGroupList = "";
foreach
            (SearchResult group in groups) {

ScreenNames = "";
//
            getting all the members DN in the group
// a member DN would look
            like: "CN=tommy chan, CN=Users, DC=GO-MS, DC=NET"
members =
            group.Properties["member"];
if ( members != null )
            {

foreach (String member in members) {
// getting a
            DirectoryEntry object of this member
userEntry =
            ad.GetUser(member);
PropertyValueCollection descriptions =
            userEntry.Properties["description"];
if ( descriptions != null )
            {
foreach (String description in descriptions) {
// if the
            description field holds an AIM screen name
// then appends it to
            ScreenNames
ScreenName = GetSN(description);
if ( ScreenName
            != "" ) {
ScreenNames += " " + ScreenName + "\r\n"
            ;
}
}
}
}
}
// if there's at least one screen name
            found from a user in this group,
// then add the group to the AIM
            import list
if ( ScreenNames != "" )
aimGroupList += "\r\n \""
            + group.Properties["name"][0] + "\""
+ " {\r\n" + ScreenNames + "
            }";
}
// print the AIM import list
Console.Write(
            FormatImportString(aimGroupList) );
}

///
            <summary>
/// Parse the AIM screen name from the
            parameter.
/// If the string starts with "AIM: ",
/// then
            assume it is followed by an AIM screen name
///
            </summary>
/// <param name="pDescription">the string
            to be parsed</param>
/// <returns>the SN or an empty
            string</returns>
private static String GetSN(String
            pDescription) {
if ( pDescription.IndexOf("AIM: ") == 0 )
            {
pDescription = "\"" + pDescription.Replace("AIM: ", "") +
            "\"";
} else {
pDescription = "";
}
return
            pDescription;
}

/// <summary>
/// Wrapping around
            the string that the AIM software expects from the import
            files
/// </summary>
/// <param
            name="groupList">the string that contains the group
            list</param>
/// <returns>a formated group list
            string</returns>
private static String
            FormatImportString(String groupList) {
return "Config {\r\n
            version 1\r\n}\r\nBuddy {\r\n list {" + groupList + "\r\n
            }\r\n}";
}
}

Main() first checks to see if enough arguments are passed, then passes them to construct a ActiveDirectoryHelper object followed by a call to GetAllUserGroups() to get a collection of groups. Then for each group, it finds out its members by looking at its member attribute. The values in the member attribute is a DN, which might be something like: CN=tommy chan, CN=Users, DC=go-ms, DC=net. Then we pass the DN to GetUser() in ActiveDirectoryHelper to get a user object of type DirectoryEntry. Then we get its description attribute the same way we did to get the member attribute from a group. For each value in the description attribute, we pass it to the GetSN(). This method returns the screen name if the value holds one, an empty string otherwise. Note that although the description field only allows one value in the Active Directory Users and Computers console, it's possible to have it store multiple values via some APIs or the ASDI Editor. If there's at least one screen name found from a user in the group, then it's added to the list for the AIM import. When all of the groups are processed, we call the FormatImportString() to wrap around the string with the text that the AIM software expects from an import file.

run.bat

@echo off
GetScreenName netism "CN=Users, DC=GO-MS, DC=NET" GO-MS\administrator c:\winnt

This is a DOS batch file. netism is the server name, CN=Users, DC=GO-MS, DC=NET is the DN. GO-MS\administrator and c:\winnt is the login to access the active directory.

The steps

The following steps are to show how to (1) create the groups, (2) create the users, (3) assign the users to groups, (4) run the program, and (5) importing the list to AIM. Here, I am assuming you have (1) the Active Directory configured, (2) .NET Framework beta 2 or higher installed, and (3) the AIM software 3.0 or higher installed and signed on.

1. Open the Active Directory Users and Computers console, usually located in Start | Programs | Administrative Tools. Then from the left panel, do a right click on Users, select New, and click on Group.

2. Type in a name for the group, for example "sales". The same name will be automatically filled in the Group name(pre-Windows 2000) field. Change the Group scope and Group type by clicking on different radio buttons base on your need, or leave them as they are, then click OK.

3. If everything went well, you will see the new group is created and listed in the right panel. In this case, the sales group.

4. Now create a user by doing a right click on Users in the left panel again, then select New, but click on User instead of Group this time.

5. Type in the First name and the Last name, the Full name will be filled automatically. Then type in the User logon name. Then click Next.

6. Now type in the password, and confirm it. Check the checkboxes where applicable. Then click Next.

7. Click Finish to confirm and to create the user.

8. If everything goes well, a new user will be created and listed in the right panel. In this case, it's Tommy Chan.

9. Now double click on the user tommy chan to edit his properties. You will see the description field. Type in AIM: tommy chan.

10. Do a right click on the user Tommy Chan, and click Add members to a group.

11. A Select Group windows is opened. Locate the group, in this case, sales. Select it, and then click OK.

12. Repeat step 1-11 until all the necessary groups and users are created and assigned.

13. Now after you have compiled both the ActiveDirectoryHelper.cs and GetScreenName.cs you can just run the run.bat. You can compile the files by running mk_ad.bat and mk_getsn.bat from the code download. If you just type run, the output is printed on the screen. You can use the " > " character to tell the command console to redirect the output to another resource (it's a file in our case). The d:\NetworkShare is a network share folder on my network with the UNC path as \\netism\NetworkShare. After the second time we run it with the output redirected to a file, a file is created in this folder.

14. Now from any machines on the network with access to the shared folder, sign on to the AIM.

15. Click on File (in some later versions, such as 4.7, it is My AIM instead), then Load Buddy List.

16. Navigate or type in the UNC path to the share folder, you should see the file.

17. Uncheck the groups that you DO NOT want to be imported to the AIM from the file, and then click Load. All groups are checked by default.

18. Now you should see all the groups and screen names you entered in the description fields for the users in the List Setup tab.

Conclusion

As you can see, the users and group management can now be done programmatically. Through the Active Directory Services and the ADSI, Microsoft has exposed almost all the network users and system resource management to the programming world. It's possible to write a program, or even just a script, to automate all the user account creations across different services. A network administrator can create almost all the accounts programmatically. Instead of just creating a user account "Eddie" for the domain logon, you can also create a virtual site in the IIS, and a DNS entry to map to the site http://eddie.mycompany.com/?WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0, then an email eddie@mycompany.com, then maybe also a FTP account and a Telnet account etc . You no longer need to open different services in different servers to create the individual accounts one by one. More information on ADSI programming can be found at: http://www.microsoft.com/windows2000/techinfo/howitworks/activedirectory/adsilinks.asp?WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0.

Article Information
Author | Tin Lam
Technical Editor | Adam Ryland
Author Agent | Charlotte Smith
Project Manager | Helen Cuthill
Reviewers | Susan Connery, Kaushal Sanghavi

If you have any questions or comments about this article, please contact the technical editor.


Adding Scripting to C# Windows Forms Applications

Adding Scripting to C# Windows Forms Applications
http://www.csharptoday.com/content.asp?id=1801
Article Rating: 4
Published on January 16, 2002
Abstract
The main reason we might add scripting to a .NET application, is to make our applications customizable, or give the user of our application a chance to enhance its functionality by making it programmable. Our application becomes more flexible, and we do not get hung up on developing for ‘every case’ scenarios – we can spend more development time making our ‘common case’ scenarios better for the average user. In the past we could use Visual Basic for Applications (VBA) or scripting like JScript or VBScript to customize our applications - in this article, Norbert Ruessman details the scripting methods curently available with .NET to extend a sample charting application.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

Why add Scripting to an application?

The main reason we might add non-.NET scripting to a .NET application is to make our applications customizable, or give the user of our application a chance to enhance its functionality by making it programmable. Our application becomes more flexible, and we do not get hung up on developing for 'every case' scenarios - we can spend more development time making our 'common case' scenarios better for the average user.

To understand the power of scripting lets have a look at Microsoft Excel. Most users can use it as it is. They enter data into cells, or load them from files, and then do their calculations and charts. Excel even supports different file formats, so many people can load data files created by other programs.

But maybe a user has lots of data in formats not supported by Excel. These may be files in ASCII format, but in a structure not supported by Excel, or data from a database, or maybe even data the user wants to read from an RS232 or a SAP system. The user cannot import this data directly. But Excel has VBA (Visual Basic for Applications), and with some Visual Basic programming, the user can read the data into an Excel spreadsheet.

With VBA it is also possible to enhance the user interface by defining forms for data entry, and to transfer data to other programs using Automation. And if Excel does not provide a formula for some special calculations, that formula can be defined using VBA as well.

This article is about scripting, and the example above is about VBA. How is scripting related to VBA? In the days before .NET, a programmer could choose between two technologies: license VBA from Microsoft, or use scripting engines.

Both solved the problem, but there are differences. To license VBA from Microsoft was not possible for small companies or even a single programmer, because it is quite expensive, and not very easy to implement. To use the script engines was very cheap (it comes for free), and it can use different languages, like VBScript (similar to Visual Basic) and JScript (Microsoft's implementation of JavaScript) - and other companies supplied languages like Perl, Python and Haskell.

The advantage of VBA was the IDE. It included an editor with IntelliSense, object browser, form editor and a very powerful debugger. All this was missing with scripting engines, only a simple debugger was provided by Microsoft, again for free.

In the .NET framework it is the same. VBA will be replaced by VSA (Visual Studio for Applications), which again must be licensed from Microsoft. (Download it from http://msdn.microsoft.com/downloads/default.asp?url=/downloads/sample.asp?url=/MSDN-FILES/027/001/669/msdncompositedoc.xml&WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0) It will include an IDE similar to VBA, with editor and debugger etc. Scripting comes for free, but without any IDE.

Currently only Visual Basic .NET is supported as a scripting engine, but in the future Microsoft will make it possible to use other languages as well (C#), other companies will probably also provide engines for Perl, etc. However, you should start learning scripting today: once there are other languages available, we only have to change a few lines in our code to support these.

Features of scripting

A script is a piece of code the user of your application has to write in a scripting language your application supports, if possible using a small IDE provided by your application. Once compiled, the user can call the functions defined in this code. If desired, you can make your application objects (form, controls on you form, or data objects) accessible to the script. The script can then access the properties and methods of your objects, or you can even enable the script to handle events fired by your objects.

How scripting works

In principle, making scripting work is very easy. We must create a script engine and give it a reference to a class implemented in our application that implements the interface IVsaSite. This is required in order to enable the script engine to communicate with an application. If we want the script to access methods and properties of objects implemented our application, we must add references to these objects in the script. Then we add the script code to the engine and are ready to compile it. If there are no errors, any method of the script can be invoked, or the script just sits there idle to wait for events of your application.

System requirements

This sample will work on builds of Visual Studio .NET later than Beta 2. Scripting also works with Visual Studio.NET Beta 2, but there is a bug in the VSA classes.

Sample: small charting application

We will develop a sample application to show the power of scripting. Imagine we have a charting application. Our software gives the user a chance to enter data manually, or load it from files in common formats. But we know that if we add the functionality ourselves supporting all kinds data formats, or data sources, then we will not have much time to improve our core charting software. So we decide to add scripting, so that our customers can enhance the application by themselves.

Because its beyond the scope of this article to develop a charting application, the sample application only has a DataGrid control with 4 columns, and a Panel control where the data of these columns is drawn. The chart is very simple; maximum and minimum values from the grid are calculated, and curves are drawn inside the panel using these values for scaling. No axis is displayed.

The user can enter data manually, save it to disk as an XML file, and reload it. The code is provided in the download accompanying this article, in the STEP1 directory. The following picture shows the charting application in action:

The first column of the grid is filled with data; the others are partially empty in this sample. The first column is always drawn using a blue pen. The other columns are always drawn with a red pen.

For details of this application have a look at the source provided; our concern here is to show how to incorporate scripting into an application. The form has a private member:

private DataSet m_ds;

The DataSet holds the data. It is created and connected to the grid in the form's constructor:

m_ds = new DataSet();
DataTable dt =
            m_ds.Tables.Add("Data");
dt.Columns.Add("Value1",typeof(double));
dt.Columns.Add("Value2",typeof(double));
dt.Columns.Add("Value3",typeof(double));
dt.Columns.Add("Value4",typeof(double));
dataGrid1.DataSource
            = dt;

In form1.cs the paint event for the panel control is handled (function PaintPanel()). The maximum and minimum values of the data are calculated for scaling. The actual drawing is done in function DrawOneCurve(), which is called from PaintPanel().

The panel is not redrawn automatically when the user changes data in the DataGrid control. He has to press the button labeled DrawIt. The event handler for this button just calls the panel's Refresh() method.

Saving and loading the data is very straightforward. Because we use a DataSet object to hold the data, we can simply call the DataSet 's WriteXML() and ReadXML() methods to save and load the data. These methods need a filename, which we get by using a SaveFileDialog and an OpenFileDialog object. For the complete code have a look at the sources provided in the download.

This is our sample application. It can draw a simple chart, using the data the user entered manually, and the data can be stored to disk and reloaded again. Now lets take this application and add scripting to it.

Adding Scripting to the sample application

The following sample can be found in the step2 directory in the download.

To use scripting we need to create a script engine. The script engine should be able to access our objects (it is not necessary to enable access to all application objects, but this may limit the user's ability to enhance the application).

In scripting we do not add a reference to our objects to the scripting engine, but just tell the scripting engine the type and the name of the object. When the engine runs a script and needs our application's objects, it will ask our application for a reference to an object. For this callback we have to implement a class that supports the IVsaSite interface. The engine will also compile the code we give it.

We will create two classes: ScriptHost to encapsulate the script engine, and VsaSite to implement the engine's callback interface.

Implementation of IVsaSite Interface

Class VsaSite implements the callback interface IVsaSite. Later on in our sample application we will create an instance of this class, and reference it in the script engine. The script engine will then use this reference to communicate with our sample application.

We only need 2 functions of IVsaSite:

  • GetEventSourceInstance and
  • OnCompileError.

GetEventSourceInstance is called whenever the script engine wants to access one of our application objects, and we must return a reference to our object. OnCompileError is called during compilation of script code whenever there is a compiler error.

The other functions of IVsaSite are left empty, because they are only needed for debugging. Here we only add scripting to our sample application, without support for debugging scripts. With scripting it is possible to write a debugger, but this is beyond the scope for this article (and there is little information available so far).

IVsaSite has 5 methods. The implementation of class VsaSite is fairly simple:

public class VsaSite : Microsoft.Vsa.IVsaSite
{
private Hashtable m_EventInstances;

public VsaSite()
{
m_EventInstances = new Hashtable();
}

// NOT a member of IVSaSite
// we call it when we add an application object to script engine
// then it is easier for us to to retrieve the
// object in "GetEventSourceInstance"
// objects are added to a hash table
public void AddEvent(string Name, string Type , object Instance)
{
try
{
m_EventInstances.Add(Name, Instance);
}
catch (Exception ex)
{
MessageBox.Show("Couldn't add event instance to the hashtable\r\n" +ex.ToString() + "\r\n" );
}
}

// this function is called by script engine only if
// debugging is supported.
// we only through an exception here. This will never be thrown,
// because we do not implement support for debugger
public void GetCompiledState(out byte[] pe,out byte[] debugInfo)
{
throw new Microsoft.Vsa.VsaException(Microsoft.Vsa.VsaError.CompiledStateNotFound);
}

// this function is called by engine whenever it needs a
// reference to one of our application objects
// we get it from the hash table (see AddEvent)
public object GetEventSourceInstance(string itemName,string eventSourceName)
{
try
{
Object eventInstan = m_EventInstances[eventSourceName];
return eventInstan;
}
catch
{
throw new Exception ("cannot find eventSource in VsaSite::GetEventSourceInstance :" + eventSourceName);
}
}

// following 2 functions only needed when supporting debuggers,
// therefore left empty
public object GetGlobalInstance(string name)
{
return null;
}

public void Notify(string notify,object info)
{
}

// called by script engine whenever there is a compile error
public bool OnCompilerError(IVsaError error)
{
MessageBox.Show (error.Description + "\nLine : " + error.Line.ToString());
return true;
}

}

The functions GetGlobalInstance() and Notify() are left empty. They are needed only if you want to add support for debugging scripts. GetCompiledState() is only needed for adding debugging as well.

OnCompileError() is called by the script engine whenever there is a compiler error. The implementation here is very simple, just showing a message box with a description and a line number.

The last function required by IVsaSite is GetEventSourceInstance(). As mentioned before, we want the script engine to access objects in our application. We will only add a name and a type for each object to the script engine. Whenever the engine needs a reference to a real object, it will call this function. Our implementation gets this reference from a hash table. This hash table is filled with the function AddEvent(). AddEvent() is not part of IVsaSite, but we add it to our class to make it easier for us.

And that is it for IVsaSite. Had we added a debugger, however, we would have coded much more.

Implementation of ScriptHost

This class encapsulates the script engine, and also takes care of creating a VsaSite object required by the script engine. For the complete code, have a look at the sources provided for download; here we will only discuss the highlights. First let's have a look at the CreateEngine() method:

private void CreateEngine()
{
switch
            (m_Language)
{
case "VB":
m_Engine = new
            Microsoft.VisualBasic.Vsa.VsaEngine();
break;
default:
throw
            new Exception("Unknown Engine");
}
}

This function is only called from the constructor of the class ScriptHost. The only reason we have this function is that in the future we will have more languages that support scripting, and having this function makes it easier to enhance our application to support them.

ScriptHost has a property ScriptCode, which is used to set and retrieve the code:

public String ScriptCode
{
get
{
return
            m_Script.SourceText;
}
set
{
m_Script.SourceText =
            value;
}
}

Now lets have a look at the Compile() method:

public void Compile()
{
if
            (m_Engine.IsRunning == true)
m_Engine.Reset();
try
{
if
            (m_Engine.Compile() == true)
this.Run();
}
catch
            (Microsoft.Vsa.VsaException e)
{
throw new Exception ("error
            compile");
}
}

A script code can only be compiled if the engine is not running. After compiling, the engine holds the code in memory in executable form. Before anything can be done with a script it must be put in a running state, which is done with method Run() of ScriptHost.

Only after calling this function can the compiled script code receive events from our application objects, and only then can our application invoke methods implemented in script code.

public void
            Run()
{
try
{
m_Engine.Compile();
m_Engine.Run();
}
catch
            (Microsoft.Vsa.VsaException e)
{
throw new Exception("Unable
            to run the engine");
}
catch (Exception
            ex)
{
MessageBox.Show(ex.ToString());
}
}

To ensure that the code is compiled, we compile it again (this will take time, but that may be negligible compared to the time taken to run the scripts).

Once the script is running, it can remain idle and wait for events fired by objects of the hosting application. Or, methods in the script can be invoked. For invoking methods, ScriptHost has the member Invoke():

public Object Invoke(String ModuleName,
            String MethodName, Object [] Arguments )
{
string FullName
            ;
Type ModType ;
MethodInfo method ;
if (m_Engine.IsRunning
            == false)
{
throw new Exception("Engine isn't
            running!");
}

FullName = m_Engine.RootNamespace + "." +
            ModuleName;
try
{
ModType =
            m_Engine.Assembly.GetType(FullName, true, true);
}
catch
            (Microsoft.Vsa.VsaException e)
{
throw new Exception("Unable
            to get assembly named " + FullName);
}
method =
            ModType.GetMethod(MethodName);

if (method == null)
throw
            new Exception("method not found");
else
{
Object ooo =
            null;
return method.Invoke(ooo, Arguments);
}
}

Invoke() gets as parameters the ModuleName and the MethodName to be invoked. The third parameter is an array of objects. These are the method's parameters. The return value is of type object again.

The method SetupImports() is just a helper function. This enables our application to generate statements like imports system (Visual Basic .NET syntax for using system) automatically, so the user does not have to enter them in script code.

Finally, let's have a look at ScriptHost's constructor:

public ScriptHost(string Language,string Moniker,string Namespace)
{
try
{
m_Language = Language;
this.CreateEngine();
m_Engine.RootMoniker = Moniker;
m_Site = new VsaSite();
m_Engine.Site = m_Site;
m_Engine.InitNew();
m_Engine.RootNamespace = Namespace;
}
catch (Microsoft.Vsa.VsaException e)
{
}
m_Items = m_Engine.Items;
m_Script = (IVsaCodeItem) m_Items.CreateItem("scriptcode", Microsoft.Vsa.VsaItemType.Code,
Microsoft.Vsa.VsaItemFlag.Module);
this.AddReference("System", "System.dll");
this.SetupImports("System");
}

After creating the engine, we must give it a unique name. We also create an instance of class VsaSite, which implements the IVsaSite interface, and provide it to the engine to enable it to call back to our application. We also specify a namespace for this engine.

Then we create a code item (m_Script). Note that this implementation of ScriptHost only supports one code item; in general it is possible to add several code items to an engine. An example would be a MDI application with one instance of a script engine, but for every MDI child window opened some code is added to a script engine.

Finally we add a reference to system.dll, and the call to SetupImports adds a line imports system to the script code, which will probably always be needed.

Creating a simple IDE

Visual Studio for Application (VSA) will have a powerful IDE, similar to the IDE for macros in Visual Studio .NET. In our application, we will have to make one on our own. We design a simple form and name it IDE:

This form has a multiline TextBox control and 4 buttons. The TextBox control is used for defining the script code.

The code for the Compile button compiles the code entered in the text box. The script engine keeps the compiled code in memory.

private void btnCompile_Click(object sender, System.EventArgs e)
{
m_Host.ScriptCode = textBoxScript.Text;
try
{
m_Host.Compile();
}
catch (Exception ex)
{
MessageBox.Show(this,"error in compile\r\n" + ex.ToString());
}
}

With button Run Main the main function in code can be invoked:

private void btnRunMain_Click(object sender, System.EventArgs e)
{
try
{
m_Host.Invoke("scriptcode", "main",null);
}
catch (Exception ex)
{
MessageBox.Show(this,"error Invoke Main\r\n" + ex.ToString());
}
}

The remaining buttons save and load script code from disc.

private void btnSaveAs_Click(object sender,
            System.EventArgs e)
{
saveFileDialog1.InitialDirectory =
            Application.ExecutablePath;
saveFileDialog1.DefaultExt =
            ".Macro";
saveFileDialog1.Filter = "Macro Files
            (*.Macro)|*.Macro|All Files|*.*";
DialogResult ret =
            saveFileDialog1.ShowDialog(this);
if (ret ==
            DialogResult.OK)
{
StreamWriter sw = File.CreateText(
            saveFileDialog1.FileName);
sw.WriteLine(textBoxScript.Text);
sw.Close();
}
}

private
            void btnLoad_Click(object sender, System.EventArgs
            e)
{
openFileDialog1.InitialDirectory =
            Application.ExecutablePath;
openFileDialog1.DefaultExt =
            ".Macro";
openFileDialog1.Filter = "Macro Files
            (*.Macro)|*.Macro|All Files|*.*";
DialogResult ret =
            openFileDialog1.ShowDialog(this);
if (ret ==
            DialogResult.OK)
{
StreamReader sr =
            File.OpenText(openFileDialog1.FileName);
String input;
input =
            sr.ReadToEnd();
sr.Close();
textBoxScript.Clear();
textBoxScript.Text
            = input;
}
}

This is our very simple IDE. Now that we have classes ScriptHost and VsaSite we can add some code to our sample charting application and make it scriptable.

Make the charting application scriptable

First, let's add two buttons to the charting application form:

We add the code for the new button IDE:

private void btnIDE_Click(object sender,
            System.EventArgs e)
{
m_Host = new ScriptHost
            ("VB","WROXscript://macros/","scriptcode");
m_Host.AddReference("System.Data",
            "System.Data.dll");
m_Host.AddObject("ButtonPaint",
            "System.Windows.Forms.Button", button1, "",
            true);
m_Host.AddObject("Table", "System.Data.DataTable",
            m_ds.Tables["Data"], "", true);
m_Host.AddCode("Sub main" +
            "\r\n" + "\r\n" + "End Sub");
IDE ide = new
            IDE(m_Host);
ide.Show();
}

The first line creates an instance of our ScriptHost class. Then we add a reference to System.Data, because we want the script to be able to fill our DataSet object as this is the source for the chart. Then we add two of our application's objects to the script engine: The DrawIt button, and the DataTable of the DataSet. Note that we provide a name, and the namespace where these are defined. Finally we add the code for an empty Main method (to make life easier for the user of our application), before showing the IDE.

Let's start the application now and have a look at the IDE:

As can be seen, there is some code there already. The interesting part is the Automatically generated code. There we find the objects we added to the script engine.

Press the Compile button and then the button Run Main. Nothing happens, because the Main method is empty.

For a first test add the line:

imports system.windows.forms

before the first line of the script and in Sub Main add the line:

MessageBox.Show ("Hello there")

Click on Compile and Run Main again, and a message box is shown with the text "Hello there ". We have just run our first script.

But this is boring. Let's do something with our application. Close the IDE and open it again to get the initial code. Then add:

imports System.Data

just below the first line. Fill Sub Main with this code:

dim i as double
Table.clear()
for
            i=0.1 to 10 step 0.1
Dim newRow As DataRow =
            Table.NewRow()
newRow(0) = math.sin(i) /
            i
Table.Rows.Add(newRow)
next I

The Table in our DataSet object is cleared, and in a for .. next loop the Table is filled with values.

Click on Compile and Run Main. If there are no compile errors, take a look at our drawing application. Note that the DataGrid control shows the data we calculated. Press the DrawIt button, and we get the following picture.

It's not very user-friendly that the user has to press the button DrawIt after the data is calculated. But remember that we added this button to the script using the name ButtonPaint. Because we can access this button from script, add the line:

ButtonPaint.PerformClick()

after the for . next loop. Hit Compile and Run Main again, and you will see that the chart is repainted automatically.

Now we have a working script, let's save it using the SaveAs. button in the IDE. enter SAMPLE.Macro as the filename.

We added a button run macro to our charting application before. Let's fill it with code:

private void btnRunMacro_Click(object
            sender, System.EventArgs e)
{
openFileDialog1.InitialDirectory
            = Application.ExecutablePath;
openFileDialog1.DefaultExt =
            ".Macro";
openFileDialog1.Filter = "Macro Files
            (*.Macro)|*.Macro|All Files|*.*";
DialogResult ret =
            openFileDialog1.ShowDialog(this);
if (ret ==
            DialogResult.OK)
{
StreamReader sr =
            File.OpenText(openFileDialog1.FileName);
String input;
input =
            sr.ReadToEnd();
sr.Close();
m_Host = new ScriptHost
            ("VB","WROXscript://macros/","scriptcode");
m_Host.AddReference("System.Data",
            "System.Data.dll");
m_Host.AddObject("ButtonPaint",
            "System.Windows.Forms.Button", button1, "",
            true);
m_Host.AddObject("Table", "System.Data.DataTable",
            m_ds.Tables["Data"], "", true);
m_Host.ScriptCode =
            input;
try
{
m_Host.Compile();
m_Host.Invoke("scriptcode",
            "main",null);
}
catch (Exception
            ex)
{
MessageBox.Show(this,"error in compile\r\n" +
            ex.ToString());
}
}
}

We use an OpenFileDialog to get a filename. We read the file using a StreamReader object and create a new ScriptHost. We add a reference to our objects, just the same as when we start the IDE. Finally we add the script code we read from the file, compile it and invoke the Main method.

Here is another sample script. It shows what is now possible with our application. See comments in code for explanation.

imports System
imports System.Data
Public Module scriptcode

#Region "Automatically generated code, do not modify"
'Automatically generated code, do not modify
'Event Sources Begin
<System.ContextStaticAttribute()> Public WithEvents ButtonPaint As System.Windows.Forms.Button
<System.ContextStaticAttribute()> Public WithEvents Table As System.Data.DataTable
'Event Sources End
'End of automatically generated code
#End Region

Sub main
dim i as double
Table.clear() ' clear data table
' make sure that table has at least 4 coumns
if Table.columns.count < 4 then
for i= table.columns.count+1 to 4
table.columns.add ' add a column to table
next i
end if
' variable i starts at 0.1 and is incremented by 0.3
' in this loop up to 100
for i=0.1 to 100 step 0.3
Dim newRow As DataRow = Table.NewRow() ' create a new row for table
' calculate values for 4 columns
newRow(0) = math.sin(i) / math.exp(i / 50)
newRow(1) = 1/math.exp(i / 50)
newRow(2) = 0 ' a constant
newRow(3) = -1/math.exp(i / 50)
Table.Rows.Add(newRow) ' add newly created row to table
next i

ButtonPaint.PerformClick() ' repaint graph
End Sub
End Module

Run it, and we get this result:

Note that we did not make any changes to our original charting application, we only added scripting to it.

Handling events in script

So far we have only invoked a method defined in script code. But it is also possible to handle events of objects implemented in our application. Open the IDE and add:

imports system.data

at the beginning and a sub:

Sub ButtonPaint_MouseMove(ByVal sender As Object, ByVal e As System.Windows.Forms.MouseEventArgs) _
Handles ButtonPaint.MouseMove
Dim newRow As DataRow = Table.NewRow()
newRow(0) =e.x
Table.Rows.Add(newRow)
ButtonPaint.PerformClick()
End Sub

ButtonPaint_MouseMove is automatically called whenever the mouse moves over the DrawIt button. This sub always adds a new row to our DataTable, and fills the first field with the x-coordinate of the mouse. Finally it redraws the chart.

Run this macro, and just move the mouse over the DrawIt button. You will see a graphical display of horizontal mouse movement.

Summary

These sample scripts used mathematical functions and mouse events to generate some data. But an experienced Visual Basic .NET programmer would be able to get data from any data source, for instance a database or an RS232 or as SAP system. We have made our application customizable - our customers can enhance the application themselves, if they want, and we can concentrate on making our charting application better.

It is very important to note that we, the C# programmers, only added scripting to our application. The sample scripts did not require any changes to C# sources, therefore these features could have been added to our application by the user.

This article only shows some basic features of scripting. We added only one module to the engine. It is possible to add several modules, and classes as well. It is also possible to find out which functions are defined in a module, including parameters and their types. And it is probably possible to add debugging as well, but I can't find any information available so far. This article also does not explain everything in detail. It shows how to implement a basic IVsaSite interface, how to compile and run code, and how to invoke script methods. But the source code is available for download, and the best way to understand scripting is running the application, and stepping through it using the debugger.

Currently only Visual Basic.NET can be used as script language, but in the future other script languages will be available, either from Microsoft or other vendors.

I have some experience with adding script engines to applications in the days before .NET, when it was much more complicated than it is now. It also was nearly impossible to switch from script engines to VBA, because the concept for integrating them was very different. In .NET the information available so far says that, if we add support for scripting, like in this example, it will be very easy to include VSA, with its powerful IDE and debugger.

A final note: the sample scripts are very simple. But they are in Visual Basic .NET, and with script you can do everything you can do with this language. Enjoy trying it out.

If you'd like to discuss a broader range of C#Today topics and issues, or if you have any C# related questions you need answers to, please visit our online resource centres at : http://p2p.wrox.com/list.asp?list=csharptoday_discuss&WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0 and http://p2p.wrox.com/list.asp?list=c_sharp&WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0


Writing Serviced Components in C#

Writing Serviced Components in C#
http://www.csharptoday.com/content.asp?id=1799
Article Rating: 4
Published on January 14, 2002
Abstract
As developers begin to embrace the .NET platform and begin to develop enterprise level applications in C#, there is a continued need for enterprise services currently provided by COM+ such as Transaction Support, Object Pooling, and Shared Property Management. A serviced component is the mechanism that enables context sharing between COM+ and .NET Framework classes. Utilizing the System.EnterpriseServices namespace and COM+ interoperability, you can develop components that co-exist in harmony with existing COM+ components, as well as develop new .NET-only applications that utilize COM+ services. In this article, Adam Cartwright explains how to write serviced components in C# and how to take advantage of the enterprise services you have come to depend on in the COM+ world.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

Now that we are beginning to build applications using the .NET framework, we are entering a stage where managed and un-managed code must co-exist and work together. Furthermore, some of the core services we as developers have come to rely on do not yet exist in a "managed" environment, thus we must learn to bridge the gap between managed and un-managed code in order to build enterprise level applications.

Today we use tools such as Visual Basic 6.0 to build components, and install those components in Windows 2000 Component Services (aka COM+ Services) or Windows NT Server 4.0's Microsoft Transaction Server (MTS) in order to utilize core services such as transactions, just in time activation, and object pooling. With .NET we will begin to develop components using Visual Studio .NET, and install them in Windows 2000 Component Services to gain access to those services.

Here we will discuss the advantages in using Visual Studio .NET and the .NET Framework, how to build a serviced component using C#, and how to install that component in COM+ services for use by both .NET and COM clients.

System Requirements

This article assumes that you have a general understanding of COM+ Services and the C# programming language. All code and samples provided here were built on Windows XP.

Overview of Serviced Components

What is a Serviced Component?

A serviced component is a .NET class that is hosted in a COM+ application in order to take advantage of various COM+ services such as automatic transactions and object pooling. In order for a class to be allowed to be hosted by COM+, it must be specially configured in the code of the class itself. The first and most basic configuration is that the class must be derived directly or indirectly from the System.EnterpriseServices.ServicedComponent class. This class provides the core functionality to communicate with COM+ services. Additional configuration information is provided below.

From the perspective of a .NET class, a serviced component is no different than any other .NET class. Likewise, COM components view serviced components as just another COM component. I'll explain how this happens below.

COM+ Services

COM+, introduced with Windows 2000, is a set of DLLs, COM interfaces, methods, and tools created to provide special development and administrative services for enterprise level applications and the components of those applications. COM+ is one of the primary technologies used in developing enterprise level applications for the .NET platform using Visual Studio .NET.

It is important to understand the true nature of COM+. Because of its name, it is easy to mistakenly think that COM+ is dependent on the COM component architecture. This is not true, since the component model used is irrelevant. COM+, which is the evolution of Microsoft Transaction Server (MTS), is a services architecture, and provides complimentary services for .NET components and COM components.

The following diagram describes the evolution of COM+:

COM+ is automatically installed and configured as part of the Windows 2000 and Windows XP operating systems. Under Windows 2000 and Windows XP, COM+ exists as unmanaged code, whereas with the introduction of Windows .NET Server, COM+ is a hybrid of managed and unmanaged code in order to more tightly integrate with the .NET framework. Future versions of COM+ will be migrated completely into 100% managed code, providing complete managed services.

Using the Component Services administrative tool, you can configure component and application behavior, including transaction participation, security configuration, etc. Other COM+ services include connection pooling, thread pooling, process isolation, and message queuing.

In addition to providing enterprise services needed for today's enterprise class applications, COM+ Services also makes it possible to create components with less code, which equates to less code to create, test, debug, and maintain. This translates into quicker, simpler, and more effective application development.

.NET and COM+ Interoperability

Components written in C# and hosted in a COM+ application are available to COM clients, just as if the components were created using VB 6. Although there are some special considerations when building .NET components that you plan to consume from COM clients (such as type handling), the application is fairly straight forward. I will discuss in the next section some considerations for COM compatibility; however a full discussion of COM interoperability is out of the scope of this article.

Any considerations for COM interoperability do not apply if the consumer of your component is also written in managed code. In other words, although COM+ is used to provide enterprise services to your components, full fidelity continues to exist between managed objects. This means that you can write type specific code and still utilize COM+ enterprise services, so long as the client is also written in .NET.

Considerations for Building Serviced Components

There are some specific considerations you should keep in mind when building serviced components in C#.

COM+ Compatibility

Consider who will use your C# component before you build it. By identifying if COM-based components or .NET components will be consuming your component before you begin building it, you can make certain architectural decisions that will save you time and effort in the future.

The .NET framework extends many COM programming features, and the common language runtime provides simplified access to and from COM and .NET components across the interoperation boundary. However, COM clients must adhere to the restrictions of the COM programming model. These restrictions can affect how you will use the extensive features available in the .NET framework.

As a simple guideline, consider the following if you expect COM clients to consume your component:

  • Methods, properties, fields, events, and types must be public
  • Types must have a default constructor.
  • Do not use parameterized constructors
  • Avoid using static methods
  • Supply GUIDs for types that require them
  • Expect inheritance differences
  • Include COM Interop attributes as necessary

Versioning

Because serviced components are written in managed code, they have the same versioning rules and restrictions of other managed code. This includes full support for compatible and incompatible versions. The version field of an assembly includes Major Version, Minor Version, Revision, and Build numbers. Because COM+ versioning uses CLSIDs, a CLSID must be generated in COM+ for your assembly. This is accomplished during registration, which is described later.

Platforms

Whereas COM+ support was available on Windows NT 4 in the form of MTS, at the time of this writing support for serviced components written in .NET is available only on Windows 2000, Windows XP, and Windows .NET Server platforms.

Note that Windows XP includes COM+ version 1.5, which includes several enhancements and new features beyond COM+ 1.0 as included in Windows 2000.

COM+ Services Available to .NET

Following is a summary of the COM+ services accessible through .NET. All of these services are accessed through the System.EnterpriseServices namespace, which is described later in this article.

Service | Description
Automatic Transaction Processing | Automatic Transaction Processing is one of the most widely used services. It allows you to configure a class at design time to participate in a transaction at runtime.
Bring Your Own Transaction (BYOT) | Allows components to use a preexisting Microsoft Distributed Transaction Coordinator (DTC) or Transaction Internet Protocol (TIP) transaction.
COM Transaction Integrator (COMTI) | Allows components to participate in IBM CICS and IMS system transactions.
Compensating Resource Managers (CRMs) | A CRM is a COM+ service that allows you to include non-transactional objects in a DTC transaction.
Just-In-Time-Activation (JIT) | JIT activation allows you to create an inactive, context-only object which remains inactive until the runtime creates the full object.
Loosely Coupled Events | Provides support for late-bound events and method calls.
Object Construction | Allows you to specify initialization and construction information externally.
Object Pooling | Allows pooling pre-created objects for immediate reuse.
Private Components | Allows creation of private (sometimes called helper) components accessible to your application, but not publicly accessible.
Queued Components | Provides support for invoking and executing components asynchronously.
Role-Based Security | Provides mechanism for applying security permissions to objects and methods based on security roles.
SOAP Services | Allows taking an existing component in COM+ and exposing it as a SOAP Service.
Synchronization | Handles concurrency and synchronization support.
XA Interoperability | Enables you to access databases that support the X/Open transaction processing model.

System.EnterpriseServices Namespace

The System.EnterpriseServices namespace provides the infrastructure for developing enterprise level applications that participate in COM+ services. All serviced components are derived directly or indirectly from the System.EnterpriseServices.ServicedComponent namespace class.

When building serviced components, most interaction with the System.EnterpriseServices namespace will be in the form of setting attributes related to COM+ services. While you can directly call and access the various System.EnterpriseServices classes, it is not necessary to do so in order to implement the simplest level of COM+ services support.

Building a Serviced Component

Now that we have established what a serviced component is made of, let's begin building a serviced component of our own in C#. Using Visual Studio .NET, we'll create a new class library project in C# and start the basic functionality for our component, a bank component with an account class, with one method to transfer funds.

using System;

namespace
            BankComponent
{
public class Account
{
public bool
            TransferFunds( int accountNum, double amount)
{
// Add logic
            to update the database
return false;
}
}
}

In order to make it a serviced component, we must add two lines to reference the System.EnterpriseServices namespace and derive our Account class from the ServicedComponent class. In the Visual Studio .NET IDE, you must first add a project reference to the namespace by selecting "Add Reference" from the Project menu, and selecting the System.EnterpriseServices namespace.

using System;
using System.EnterpriseServices;

namespace BankComponent
{
public class Account : ServicedComponent
{
public bool TransferFunds( int accountNum, double amount)
{
// Add logic to update the database
return false;
}
}
}

Adding the ": ServicedComponent" after the name of our class specifies that this class is derived directly from the ServicedComponent class.

Service Related Attributes

Adding the above statements may make our component a serviced component, but we need to set the appropriate service related attributes in order to take advantage of the COM+ services we're targeting. Some service-related attributes carry an un-configured default value. Knowing these can eliminate the need to include some of them in our code should their defaults meet our needs. Others must be explicitly set depending on the COM+ service we are writing for. The .NET Framework documentation outlines each of the service related attributes and their default values.

For our component, we're going to configure it to require a transaction by adding the "[Transaction(TransactionOption.Required)]" attribute immediately before our class declaration, and specifying the "[AutoComplete]" attribute immediately before our TransferFunds method.

using System;
using System.EnterpriseServices;

namespace BankComponent
{
//Set our transaction required attribute
[Transaction(TransactionOption.Required)]
public class Account : ServicedComponent
{
//Specify transactions to autocomplete
[AutoComplete]
public bool TransferFunds( int accountNum, double amount)
{
// Add logic to update the database
return false;
}
}
}

If you're familiar with using transactions in COM+, you know that you commit a transaction by calling the SetComplete keyword, and you abort a transaction by calling the SetAbort keyword. In .NET we are able to use the "[AutoComplete]" attribute, which means that we do not have to worry about calling SetComplete or SetAbort in our code. The runtime automatically calls SetComplete for us if no exception is generated, otherwise it will automatically call SetAbort . Of course, should we want more granular control we can call SetComplete and SetAbort manually.

Deploying Your Component

There are a few other service-related attributes that we must set in our code as part of deploying it to COM+ and testing it. The following must be accomplished:

  • The component must have a strong name
  • The component must be registered in the Windows registry
  • Type library definitions must be registered and installed into a specific COM+ application

COM+ Application Identity

You may be used to the ease of drag-and-drop deployment with COM+ components. One of the great features of COM+ is that you can take a .dll file and, after creating a COM+ application for it, drag and drop it right into your COM+ application, then set any specific properties on the application and component, and you're ready to go.

Deployment with .NET created components is just as simple, if not more so, given that most of the details of configuration can be added with a few service related attributes. The COM+ application identity is one of these values. You should specify the COM+ application that this component belongs too. During deployment, the application can be created automatically for you, eliminating the step of manually creating the COM+ application yourself.

To specify the COM+ application identity, we specify the application name prior to declaring our namespace, as follows:

using System;
using System.EnterpriseServices;

//Specify the COM+ application name
[assembly: ApplicationName("BankComponent")]

namespace BankComponent
{
//Set our transaction required attribute
[Transaction(TransactionOption.Required)]
public class Account : ServicedComponent
{
//Specify transactions to autocomplete
[AutoComplete]
public bool TransferFunds( int accountNum, double amount)
{
// Add logic to update the database
return false;
}
}
}

Additionally, we want to specify that our COM+ application is a Server application, and not a Library application. We're doing this to simplify our component, as we want it to run in a dedicated server process, and not in the callers (client) process. To do this, we add the following service related attribute:

using System;
using System.EnterpriseServices;

//Specify the COM+ application name
[assembly: ApplicationName("BankComponent")]

//Specify Server Application
[assembly: ApplicationActivation(ActivationOption.Server)]

namespace BankComponent
{
//Set our transaction required attribute
[Transaction(TransactionOption.Required)]
public class Account : ServicedComponent
{
//Specify transactions to autocomplete
[AutoComplete]
public bool TransferFunds( int accountNum, double amount)
{
// Add logic to update the database
return false;
}
}
}

Creating a Strong Name

Creating a strong name for our component is necessary in order to create a CLSID to register it in the COM+ catalog. It is best practice to always create strong names for all of your components. A Strong Name is essentially the unique name for your component. Whereas under COM you would generate a GUID to uniquely identify your component, .NET relies on Strong Names to provide uniqueness among assembly files. A Strong Name consists of an assembly's identity (simple text name, version number, culture information, etc.), strengthened by a public key and digital signature generated over the assembly. Assemblies with the same strong name are expected to be identical.

Creating a strong name for your component requires that you sign it with a public/private key pair. Creating a key pair is quite simple using the Strong Name tool ( sn.exe ) that comes with the .NET framework and Visual Studio .NET. Follow these steps to create a strong name for the BankComponent that we have created.

  1. Open a Visual Studio .NET command prompt by selecting it from the Visual Studio .NET tools menu on the Start Menu / Visual Studio .NET menu.
  2. Navigate to the folder where you want to create the strong name key pair file. To keep things clean, I prefer to create the file in the same directory as the binary of the component I am creating, so in this case I will navigate to the obj\debug folder in my BankComponent project.
  3. Type "sn.exe -k <filename>", where <filename> is the name of the key pair file that you wish to create (key pair files usually have a .snk extension). For my file, I will type "sn.exe -k sgKey.snk". This will create a key pair file in that directory called sgKey.snk .

Once we have created a strong name key pair file, we must associate it with our BankComponent assembly so that during the build process it is signed and given a strong name. To do this, we are going to update one line of code in the AssemblyInfo.cs file. This file is automatically created for each project when working with Visual Studio .NET. Update the AssemblyKeyFile attribute in the bottom portion of the AssemlyInfo.cs file as shown.

[assembly: AssemblyDelaySign(false)]
[assembly: AssemblyKeyFile("sgKey.snk")]
[assembly: AssemblyKeyName("")]

Alternately, if you are not using Visual Studio .NET, or if you are not using the AssemblyInfo.cs file, you can specify the AssemblyKeyFile attribute directly in your code as shown below (note the inclusion of two additional namespaces required for compilation; these namespaces are typically included in the AssemblyInfo.cs file).

using System;
using System.Reflection;
using System.Runtime.CompilerServices;
using System.EnterpriseServices;

//Specify the COM+ application name
[assembly: ApplicationName("BankComponent")]

//Specify Server Application
[assembly: ApplicationActivation(ActivationOption.Server)]

//Supply a strong-named assembly
[assembly: AssemblyKeyFile("sgKey.snk")]

namespace BankComponent
{
//Set our transaction required attribute
[Transaction(TransactionOption.Required)]
public class Account : ServicedComponent
{
//Specify transactions to autocomplete
[AutoComplete]
public bool TransferFunds( int accountNum, double amount)
{
// Add logic to update the database
return false;
}
}
}

You are now ready to build your component by selecting the Build Solution option from the Build menu.

COM+ Registration

Now that we have created our BankComponent , given it a strong name, and built the .dll file, it is time to deploy it and register it with COM+. We will both register our component in the Windows registry and add it to our COM+ application in one single step using the .NET Framework Services Installation Tool ( Regsvcs.exe ).

Like the Strong Name tool, Regsvcs.exe is also called from the Visual Studio .NET command prompt. This is done on the server where your component will run (not on the client or development machine unless it is the same machine that will serve your component). When you run Regsvcs.exe , the following steps take place:

  1. The Assembly (your component) is loaded.
  2. The Assembly is registered in the Windows registry.
  3. A type library is created for your component.
  4. The type library is loaded and registered in the Windows registry.
  5. The type library is installed into the specified COM+ application (if the application does not already exist, a new application is created).
  6. The COM+ application is configured (transaction required, synchronization disabled, etc. depending on service related attributes set in your component).

Note: You can also access these registration features programmatically with the System.EnterpriseServices.RegistrationHelper class by creating an instance of the RegistrationHelper class and using the InstallAssembly method.

To register, install, and deploy our BankComponent , run the Regsvcs.exe utility from the Visual Studio .NET command prompt.

Notice the WARNING in the output. This is telling us that because we did not include a class interface with our component, any unmanaged late bound calls cannot take advantage of AutoComplete methods. This is expected and in our case is acceptable given the fact that we will always use early binding from any unmanaged clients.

Configuring Your Component

Now our component is registered, installed, and deployed in the BankComponent application in COM+. You can open the component services administration tool and view your new COM+ application.

Configuring your component is already complete since we added all of the necessary confirmation parameters in the code using service related attributes. We can, however, modify any configuration parameters, such as identity or activation timeout, just as we would with any other COM+ application using the administration tool.

Because our component is registered in the Windows registry, it is available for early-binding in any COM language. You can test the new component using Visual Basic 6.0 by simply adding a reference to the BankComponent in the VB references screen.

Summary

Enterprise services provided by COM+ are a vital part of any enterprise application. You can see from the above example that creating serviced components in C# can be easier and cleaner than creating them with non .NET languages. By utilizing the System.EnterpriseServices namespace, you can easily access all of the COM+ services you have become dependant on for building Enterprise level applications.

Enterprise services is the first step in an evolution to managed services as the COM+ libraries are migrated to 100% managed code. Enterprise services provide all the power of COM+ services to managed applications.


Design Patterns in C#: The Decorator Pattern

Design Patterns in C#: The Decorator Pattern
http://www.csharptoday.com/content.asp?id=1797
Article Rating: 4.5
Published on January 11, 2002
Abstract
Designing well-structured, reusable object-oriented software is difficult, even for veteran developers. However, a developer who is familiar with design patterns can apply them to design problems without having to reinvent solutions. In this article, Michael G. Emmons discusses how to use the Decorator pattern to extend the functionality of classes without creating unnecessarily complex class hierarchies.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

Introduction: The Exploding Class Hierarchy

Object-oriented programming languages such as C# provide an important mechanism for extending the functionality of classes, called subclassing. However, there are times when subclassing your way to a solution leads to an unnecessarily complex hierarchy, and unmaintainable code. For example, suppose you are developing software that monitors and manages bank accounts. As you design the system, you determine that one of the primary classes will be the Account class. The Account class will be responsible for all account transactions: deposits, withdrawals, balance requests, etc. You design this class, congratulate yourself on being an excellent object-oriented developer, and continue in the design process.

You are well into the design of the system and you begin running across new types of accounts that must be supported. These new accounts include OverdraftProtection, MinimumBalance, AdvantageMember, and FeeBased. You think that this is a textbook example of when to use subclassing, so with a twinkle in your eye you begin furiously subclassing the Account class to extend functionality to these new account types. But wait! AdvantageMember accounts can also be OverdraftProtection accounts or MinimumBalance accounts. Worse yet, AdvantageMember accounts can be both an OverdraftProtection account and MinimumBalance account. In fact, each account type can be any combination of the other account types. The twinkle in your eye slowly dies as you realize that there are an exponential number of subclasses for each new account type. You, my friend, have an exploding class hierarchy.

Your system is not doomed to failure, however. In fact, the problem of how to extend the responsibilities of a class without subclassing has been encountered so many times in software systems that the solution to it has its own pattern, called the Decorator pattern. In the remainder of this article, we will discuss the Decorator pattern in detail, develop an example solution in C# to the exploding class hierarchy problem from above, and discuss the benefits, liabilities, and implementation issues associated with this pattern.

The Decorator Pattern to the Rescue

The purpose of a decorator, also know as a wrapper, is to extend the responsibilities of an object without polluting the class hierarchy with one-off classes, that is, classes in which there is only one major difference from the parent class. Moreover, it allows the client to dynamically and transparently attach the new responsibilities to the objects at run-time, allowing them to incur resource expenses on a pay-as-you-go basis.

An OMT (Object Modelling Technique) diagram of the decorator pattern can be seen in figure 1. The decorator usually has at least four participants. The first participant is the Component, which defines the interface, or base functionality, for the objects which need to be decorated. The next participant is the ConcreteComponent, which defines the object to which additional responsibilities can be attached. Then there is the Decorator, which provides the lens through which the component will be referenced. Finally, we have the ConcreteDecorators which adds responsibilities to the Component, usually by performing additional operations before and after forwarding the request to the referenced component. This can be seen in the diagram. In it, the Operation() method is the default behavior for the ConcreteComponent. ConcreteDecoratorA and ConcreteDecoratorB define additional responsibilities with the AddedBehavior() method which would be called somewhere within these classes' Operation() method.

Figure 1: The Decorator design pattern OMT diagram.

Notice that each decorator will have a reference to a component. This can be either another decorator or a concrete component. Although it may not be apparent at first, this has a particularly important consequence that makes the decorator so useful: decorators can be chained together to aggregate the new responsibilities of a concrete component. This is exactly what we need in our Account class example from above: a way to allow the AdvantageMember class to take on the added responsibilities of the MinimumBalance and/or OverdraftProtection classes dynamically without having to create a subclass for every combination of responsibilities available.

Figure 2 shows an object diagram which illustrates this point. In this diagram, we see that an aMinimumBalanceDecorator object has a reference to an aOverdraftProtectionDecorator object, which in turn has a reference to an aAccount object. The result of this particular chain is that, in addition to all the base behaviors that a plain Account object has, this decorated Account can also charge a fee if the account drops below a minimum balance, and Transfer() money from another account if overdrawn. We can chain as many decorators together as we like, but the chain will always have the same pattern of concrete decorator objects chained together with a concrete component object at the end of the chain: ConcreteDecorator1 -> ConcreteDecorator2 -> ConcreteDecorator(n) -> ConcreteComponent. An important consideration to keep in mind when chaining decorators together is the order in which they are chained. In some cases it may not matter, but in this example it could be very important. Should you apply a minimum balance fee before overdraft protection is applied, or after? These are business rules that need to be established when designing your decorators. We will see a consequence of decorating an account in this order when we test the example we develop below.

Figure 2: Decorators can be chained together to add responsibilities dynamically.

Decorating the Account Classes

Now that we have a better understanding of how we may solve the problem of our exploding class hierarchy, let's refactor the Account example using the decorator pattern. This new solution will have an AbstractAccount class from which we will create an Account and an AccountDecorator subclass. The AccountDecorator class will then have a concrete decorator for each of the account types available: OverdraftProtectionDecorator, MinimumBalanceDecorator, AdvantageMemberDecorator and FeeBasedDecorator. The class diagram for this new design can be seen in Figure 3.

Figure 3: The Bank Account example refactored to use a Decorator design pattern.

Notice that each of the decorator classes inherits a property named Component. This property will hold a reference to either another decorator class (if you are chaining decorators together) or an Account class.

Now that we have a good idea about our class design, let's go ahead and write the C# code. For the sake of brevity, I did not code the AdvantageMemberDecorator or the FeeBasedDecorator, but it should be clear from the developed classes that these will be no different.

using System;

namespace
            BankAccount
{
/// <summary>
/// An abstract class
            used to define the interface for the concrete
/// components and
            the decorators
/// </summary>
public abstract class
            AbstractAccount
{
public abstract decimal Balance();
public
            abstract decimal Deposit(decimal amount);
public abstract bool
            Withdraw(decimal amount);
}

/// <summary>
/// An
            abstract class used to define the interface for the concrete
///
            components and the decorators
/// </summary>
public
            class Account : AbstractAccount
{
protected decimal
            balance;
protected string accountNumber;

///
            <summary>
/// A read-only property which returns the
            account number
/// </summary>
public string
            AccountNumber
{
get
{
return
            accountNumber;
}
}

/// <summary>
/// The
            default constructor. Each account must be associated
/// with
            both an account number and a beginning balance.
///
            </summary>
public Account(string accountNumber, decimal
            amount)
{
this.accountNumber = accountNumber;
this.balance
            = amount;
}

/// <summary>
/// Returns the account
            balance
/// </summary>
public override decimal
            Balance()
{
return balance;
}

///
            <summary>
/// Deposit money into the account.
///
            </summary>
/// <returns>The new balance after the
            deposit.</returns>
public override decimal Deposit(decimal
            amount)
{
this.balance += amount;

return
            this.balance;
}

/// <summary>
/// Withdraw money
            from the account.
/// </summary>
///
            <returns>
/// True if the withdrawal is successful, false
            if the amount
/// being withdrawn exceeds the balance.
///
            </returns>
public override bool Withdraw(decimal
            amount)
{
bool bSuccess = false;

if (this.balance >=
            amount)
{
this.balance -= amount;
bSuccess =
            true;
}

return bSuccess;
}
}

///
            <summary>
/// A class used to define the decorators
///
            components and the decorators
/// </summary>
public
            class AccountDecorator : AbstractAccount
{
protected
            AbstractAccount component;

/// <summary>
/// Accepts
            a reference to a component. This reference can be
/// either a
            decorator or an account.
/// </summary>
public
            AccountDecorator(AbstractAccount component)
{
this.component =
            component;
}

/// <summary>
/// The base decorator
            simply passes all messages through to
/// the referenced
            component.
/// </summary>
public override decimal
            Balance()
{
return component.Balance();
}

public
            override decimal Deposit(decimal amount)
{
return
            component.Deposit(amount);
}

public override bool
            Withdraw(decimal amount)
{
return
            this.component.Withdraw(amount);
}

}

public class
            OverdraftProtectionDecorator : AccountDecorator
{
protected
            string overdraftAccountNumber;

/// <summary>
///
            Returns the account number used for overdraft transactions.
///
            </summary>
public string
            OverdraftAccountNumber
{
get
{
return
            overdraftAccountNumber;
}
}

public
            OverdraftProtectionDecorator(AbstractAccount component, string
            overdraftAccountNumber) :
            base(component)
{
this.overdraftAccountNumber =
            overdraftAccountNumber;
}

/// <summary>
/// For
            this example the OverdraftProtectionDecorator won't
/// worry
            about whether there is enough money in an overdraft
/// account
            to cover the withdrawal.
/// </summary>
///
            <returns>
/// Will always return true since any overdrawn
            amount will
/// be transfered from an overdraft account.
///
            </returns>
public override bool Withdraw(decimal
            amount)
{
if (!component.Withdraw(amount))
{
decimal
            amountToTransfer = (amount -
            component.Balance());
Transfer(amountToTransfer);

// The
            funds available should now be
            sufficient
component.Withdraw(amount);
}

return
            true;
}

public void Transfer(decimal amount)
{
//
            Code would go here to transfer amount
// from overdraft account.
            For demo
// purposes we will simply deposit the
// amount
            required into the
            account
component.Deposit(amount);
}
}

public class
            MinimumBalanceDecorator : AccountDecorator
{

protected
            decimal minimumBalance;
protected decimal
            belowMinimumFee;

public decimal
            MinimumBalance
{
get
{
return
            this.minimumBalance;
}
}

public decimal
            BelowMinimumFee
{
get
{
return
            this.belowMinimumFee;
}
}

public
            MinimumBalanceDecorator(AbstractAccount component,
decimal
            minimumBalance, decimal belowMinimumFee) :
            base(component)
{
this.minimumBalance =
            minimumBalance;
this.belowMinimumFee =
            belowMinimumFee;
}

/// <summary>
/// If the
            balance goes below the required minimumBalance for
/// this
            account the account will be charged a belowMinimumFee
/// after
            each withdrawal.
/// </summary>
public override bool
            Withdraw(decimal amount)
{

bool bSuccess =
            component.Withdraw(amount);

if (bSuccess)
{
if
            (component.Balance() <=
            this.minimumBalance)
component.Withdraw(this.belowMinimumFee);
}

return
            bSuccess;
}
}
}

To test the decorator pattern and to verify that in fact decorators can be chained together to create dynamically assigned class behavior, I've created a simple console application that uses the concrete component and decorators:

using System;
using BankAccount;

namespace BankAccountConsole
{
class Class1
{
static void Main(string[] args)
{
Account ac1 = new Account("234234", (decimal)2000.00);
OverdraftProtectionDecorator opd = new OverdraftProtectionDecorator(ac1, "8382747");
MinimumBalanceDecorator mbd = new MinimumBalanceDecorator(opd, (decimal)1900.00, (decimal)5.00);

Console.WriteLine("Balance: ${0}. Minimum balance: ${1}.
Fee: ${2}",mbd.Balance(), mbd.MinimumBalance, mbd.BelowMinimumFee);
Console.WriteLine("Withdrawing $100. (a fee should be charged)");
mbd.Withdraw((decimal)100.00);
Console.WriteLine("New balance: {0}", mbd.Balance());
Console.WriteLine();

Console.WriteLine("Now withdrawing everything in the account
+ $10. (Overdraft should occur)");
bool bSuccess = mbd.Withdraw(mbd.Balance() + (decimal)10.00);
Console.WriteLine("Was the withdrawal successful? {0}", bSuccess);
Console.WriteLine("New balance: ${0}", mbd.Balance());

Console.Read();
}
}
}

What we are creating is a bank account with both a minimum balance and overdraft protection. This is the same example as was shown in the object diagram in figure 2. The output can be seen in figure 4:

Figure 4: Output from an Account object decorated with minimum balance and overdraft protection behaviours.

Notice that once that account balance dropped to $1900, which was the minimum balance required for that account, the $5 fee was charged by the MinimumBalanceDecorator. Also notice that even if we attempt to take out more money than is available in the account, the OverdraftProtectionDecorator will take care of transferring the required amount from the overdraft account. Finally, note a third interaction that was alluded to previously, but may not be immediately apparent: when the account is overdrawn, after the OverdraftProtectionDecorator has transferred the required amount into the account, the MinimumBalanceDecorator will notice that the balance is below the minimum and attempt to withdraw the minimum balance fee. But, since the balance is currently $0, the OverdraftProtectionDecorator kicks in once again to transfer the required fee from the overdraft account. The order in which decorators are chained together are often significant, and business rules for decorator interaction and precedence should be developed in detail before coding the classes. An unclear definition of how decorators should interact can result in unexpected and undesired behaviour.

You should now have a good understanding about the usefulness of the decorator pattern, when to use it, and how to implement it. Now let's take a look at some of the issues associated with using the decorator pattern.

Decorator Issues

As with any design pattern, there are both benefits and liabilities associated with the decorator pattern. It is much more flexible than static inheritance, and provides an alternative to inheritance in cases where your hierarchy is very complex. It allows the client to incur resource expenses on a pay-as-you-go basis. In our example, we could dynamically create a MinimumBalance account, an OverdraftProtection account, or a combination of both as needed. If a monolithic Account class was created to handle these different iterations instead, every instance of the class would be burdened with resource requirements that they would not necessarily be using. The decorator pattern makes it easy for the developer to add unforeseen responsibilities to classes by creating new decorators.

On the other hand, when you use the decorator pattern, you end up with lots of lightweight objects. This can result in some performance degradations when instantiating a chained list of decorators and when communicating with an object with many decorations. Also, classes built using decorators can be confusing to understand and maintain by someone who is not familiar with the pattern's design.

Conclusion

I hope the twinkle in your eye has returned as you realize the real effectiveness of identifying and using design patterns in your development efforts. The next time your find yourself with an exploding class hierarchy, try applying the decorator pattern to the design.

If you'd like to discuss a broader range of C#Today topics and issues, or if you have any C# related questions you need answers to, please visit our online resource centres at : http://p2p.wrox.com/list.asp?list=csharptoday_discuss&WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0 and http://p2p.wrox.com/list.asp?list=c_sharp&WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0


Synchronized Collections: An Alternative Implementation for Many-to-Many Relationships

Synchronized Collections: An Alternative Implementation for Many-to-Many Relationships
http://www.csharptoday.com/content.asp?id=1726
Article Rating: 3.5
Published on December 31, 2001
Abstract
In this article, Jeff Benson reviews the basics of implementing relationships in C#, examining the pros and cons of using association classes to implement many-to-many relationships, and presents a mechanism called SynchedCollection as an alternative solution. This article demonstrates that .NET’s sophisticated collection class framework, in conjunction with powerful support for inheritance, allow object relationships to be managed in novel ways.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

Implementing many-to-many relationships in an object model can be challenging. Because most modern object-oriented languages don't include native mechanisms for implementing many-to-many relationships, a typical implementation creates a dedicated association class for each many-to-many relationship in an object model. That approach, while standard, has its drawbacks, and doesn't address the problem of constraint management: how do we prevent the parties to a relationship from having inconsistent views of it?

It's important to include constraint management code at both ends of the relationship to maintain consistency. When the membership set on one side changes, the other side needs to reflect that change as well. One traditional approach peppers the involved business objects with relationship management code that might more appropriately belong elsewhere. We'll review that approach in this article, then we'll introduce an alternative mechanism for automatically maintaining these constraints; when used appropriately, this technique could make life easier for developers.

The System.Collections Namespace

Developers coming from a VB6 background, where the only viable choices for collections and set management were arrays, native collections, and Dictionary objects, will appreciate the breadth and depth of the collection support built into .NET. The .NET framework ships with an extensive Collections namespace, including a number of different collection types like stacks, queues, hashtables, etc.

In my examples, I've chosen to use ArrayList C# code to manipulate the ArrayList class. It's a simple collection class that accepts any kind of object (including strings and primitive types). It behaves more like an old-fashioned VB collection rather than a Dictionary object in that elements are not named as they are added to the collection, and therefore cannot be retrieved by name. (The concepts presented in this article can be extended to derive from other collection types as well; that is left as an exercise for the reader).

Modeling Associations: A Review

Object modelers are well aware that there are two primary kinds of associations to deal with: one-to-many, and many-to-many.

One-to-many relationships occur frequently in business modeling, and are fairly simple to implement. The following class diagram illustrates a common example. A Customer has zero, one, or several Orders, each of which has one or more OrderItem objects.

Many-to-many relationships are not uncommon. A simple example is shown in the model below: a Promotion involves any number of Products, and a particular Product can belong to zero, one, or more than one Promotion at a time.

The asterisk (*) in the diagrams is standard UML notation, and can be interpreted as "zero, one, or more." The notation "1" implies that there should always be one, and only one, object present at the other end of the association.

Implementing One-to-Many Relationships

As a starting point and a refresher, let's survey some traditional techniques used in implementing simple, one-to-many associations. Clearly, it is desirable to allow client code to traverse the children of a parent object, allowing it to add, remove, and iterate through them.

A simplistic implementation might simply implement a one-to-many relationship by adding a collection object to the parent class as a public member. This allows clients to use standard dot-notation to drill down into the child collection and to add and remove child elements.

public class Person
{
public ArrayList Addresses = new ArrayList();
}

In addition, some designs implement and maintain backpointers (object references that allows client code to traverse from a child object back to its parent through standard dot-notation). In other words, you can move both from the many side to the one side, and vice versa. This level of navigability implies that an association is bi-directional.

Even more robust implementations often guard the collection by making it private, then adding public child management methods. This allows backpointers to be automatically maintained at the time that children are added and removed to parents.

   public class
            Customer
{
// Add a child order
public void AddOrder(Order
            newOrder)
{
m_Orders.Add(newOrder);
newOrder.Customer =
            this;
}

// Remove a child order
public void
            RemoveOrder(Order
            newOrder)
{
m_Orders.Remove(newOrder);
newOrder.Customer =
            null;
}

// Return a read-only iterator representing child
            orders
public ICollection GetOrders()
{
return
            m_Orders;
}

private ArrayList m_Orders = new
            ArrayList();
}

Another useful feature is the ability to associate objects in a relationship simply by setting the parent onto the child. In other words, it's possible to allow an Order to be added to a Customer without explicitly calling an Add method to manipulate a collection.

Order NewOrder = new
            Order();
NewOrder.Customer = OurCustomer;

This is implemented by special code within the setter method for the backpointer to the parent. Behind the scenes, the appropriate collection management code is called to ensure that the new Order is added to the Customer's Orders collection (if the Order object thought it belonged to the Customer, but the Customer had no record of the Order in its Orders collection, the system would be in an inconsistent state and bad things would surely result).

Designs like this are robust, and result in a convenient, fully navigable object model that is convenient and intuitive for the developers writing client code against it. However, we can already see that the coding requirements in the object model are a little intensive. Developers must remember to include special code in both the parent and child classes for each relationship they wish to implement. Forgetting this in even one place could lead to erroneous application behavior.

Many-to-Many Relationships

Introduction

Many-to-many relationships present additional challenges. The same requirements as one-to-many relationships apply (the need for guaranteeing consistency, the desirability of a flexible API for the client), but as we will see, additional complications apply.

Because most modern object-oriented programming languages don't include native mechanisms for implementing many-to-many relationships, a standard technique often seen in class modeling is to "simplify" the association by adding an association class.

Association Classes

An association class turns one many-to-many relationship into two one-to-many relationships, making it easier to implement in code.

Unfortunately, client code must understand the association class and work with it directly. In other words, we've bent the logical model of the business domain in response to the limits placed on us by our implementation tools. To associate class instances together in a many-to-many relationship implemented with this approach, the client code looks a lot like this:

Product myProduct = new Product("Beef
            Ravioli");

// Add a Promotion
ProductPromotion ppro = new
            ProductPromotion();
Promotionppro.Promotion = new
            Promotion("Pasta Clearance
            Sale");
myProduct.Promotions.Add(ppro);

When is it appropriate to use a standard association class? When the association itself has important pieces of information pertaining to it, data that doesn't appropriately belong on either of the two classes being linked. An example is an EmploymentHistory association between a Company and an Employee. Each instance has a starting date, an ending date, and the title held by the employee during his or her time with the company. It would be difficult and awkward to store this information in either the Company or the Employee classes, so in this case, using a standard association class is acceptable and recommended.

Sometimes, however, association classes are fairly dumb and contain no data or methods of their own. The only significance of a particular instance of an association class is to link a specific instance of class A and another specific instance of class B. In essence, it's a sophisticated workaround that pollutes the namespace with trivial classes and can make a system less understandable.

This workaround can be avoided with a mechanism that requires only one additional class, regardless of how many relationships exist in a system.

Introducing the SynchedCollection

General

When developing an application with multiple one-to-many relationships like the above, writing and testing constraint logic to manage the consistency of all these relationships is tedious and can be error-prone. The Customer and Order classes are peppered with relationship management code, which has less to do with the business domain and more to do with implementation issues.

Let's encapsulate the complexity of maintaining consistency into its own generic, reusable class. Our design goals will be to present a simple and flexible interface to the client, and to reduce coding requirements for large systems with many constraints.

The Client Perspective

We'd like to insulate client code from having to be aware of the way we're implementing relationships. It would be desirable if the relationships looked like standard collections - the clients shouldn't have to implement special coding techniques, like the ones required when using association classes.

Because working with a many-to-many relationship now looks much more similar to working with a one-to-many relationship, this approach actually eases the burden on developers working with your object model. The old approach required the client code to instantiate an association class and hook it up to the business objects on each end of the association. Why not simplify the interface to allow client code to associate a Company and an Employee with one line of code?

Person NewPerson = new Person();
Address
            NewAddress = new
            Address();

NewPerson.Addresses.Add(NewAddress);

Reducing Coding Requirements

Unless you're using code generation technology, ensuring that all of the "constraint management" code has been implemented correctly can be painstaking. Backpointers and collection contents must be maintained. As we have seen, implementing even one discrete relationship requires scattering code throughout the involved classes. The SynchedCollection component allows much of this code to be centralized and reused.

Implementation

SynchedCollection Component

Each collection needs to know what its "connected" collection is. SynchedCollection instances exist in conceptual pairs, and each one maintains the name of its opposite. Each also maintains a reference to its business object owner. Whenever a child element is added to (or removed from) an instance of SynchedCollection, it locates its counterpart collection on the child and communicates with it to ensure that its owner is added to, or removed from, its corresponding collection appropriately. This keeps the relationship properly balanced.

Confused yet? Let's look at an actual runtime object diagram to see what this pattern might look like in a running system.

In addition to the above requirements, the SynchedCollection must supplement the standard Add and Remove operations to add constraint management functionality and to generally look like a standard .NET collection.

Let's take a look at the implementation of the SynchedCollection class.

To begin with, let's inherit from an existing collection class-the ArrayList. We'd like our component to look like any of the standard .NET collections. We'll simply add some additional members and constructors to the inherited implementation.

using System;
using System.Collections;

// This is a collection class that helps manage many-to-many
relationships without the need for an association class
[Serializable]
public class SynchedCollection: ArrayList
{
// Constructor that requires parameters (no half-baked object allowed)
public SynchedCollection(object Owner, string LinkedCollectionName)
{
m_Owner = Owner;
m_LinkedCollectionName = LinkedCollectionName;
}

// Override to maintain synchronization
public override int Add(object value)
{
// Obtain the linked collection on the child
IList LinkedCollection = GetLinkedCollection(value);
int returnValue = base.Add(value);

if(LinkedCollection.IndexOf(m_Owner) == -1)
LinkedCollection.Add(m_Owner);

return returnValue;
}

// Override to maintain synchronization
public override void Remove(object value)
{
// Obtain the linked collection on the child
IList LinkedCollection = GetLinkedCollection(value);

base.Remove(value);

if(LinkedCollection.IndexOf(m_Owner) > -1)
LinkedCollection.Remove(m_Owner);
}

// Override to maintain synchronization
public override void Clear()
{
// Make sure that a call to Clear keeps everything synchronized
while(Count > 0)
Remove(this[0]);
}

// Returns the collection that is synched with this one
protected IList GetLinkedCollection(object value)
{
return (IList)(value.GetType().GetField(m_LinkedCollectionName).GetValue(value));
}

// The object that owns this collection
private object m_Owner;

// Name of corresponding collection on opposite side of relationship
private string m_LinkedCollectionName;
}

First, notice that the Add and Remove methods have been overridden to provide extra functionality. For example, the Add method takes the incoming child argument, identifies the proper collection on that child, and passes its owner business object into that collection's Add method. The Remove method behaves similarly. These methods use the services of the GetLinkedCollection() method, described below. An easy improvement would be to raise custom events when children are added and removed. See the article "Adapting .NET GUI Controls to a Business Tier" (http://www.asptoday.com/content/articles/20011105.asp) for an example.

Second, also notice that the implementation of GetLinkedCollection() uses special methods on the System.Type object to identify the corresponding collection on the child object. By representing its property name as a string, we can avoid hard-coding property names in the code. Although this is a form of reflection (inspecting and accessing metadata about object members at runtime), a thorough discussion of reflection is beyond the scope of this article.

Finally, take a look at the special constructor we've defined. We can see that the SynchedCollection requires a couple of crucial data values in order to function properly: its backpointer to the business object that owns it, and the name of its conceptual collection counterpart. By declaring a constructor that accepts both of these values (and not including a default, parameterless constructor), we can require that the client must provide them at object instantiation time. This eliminates the possibility of "half-baked" SynchedCollection instances that have only been partially initialized.

Client Configuration

How does all of this information get set up? How do the business objects declare and configure instances of SynchedCollections ? Thanks to constructors, it's not too difficult. Take a look at this sample code from the Product and Promotion classes.

   public class
            Product
{
public string ProductName;
public string
            ProductID;

public SynchedCollection Promotions;

public
            Product()
{
// Initialize this end of the
            association
Promotions = new SynchedCollection(this,
            "Products");
}
}

public class Promotion
{
public
            string Name;
public DateTime EffectiveDate;
public DateTime
            ExpirationDate;

public SynchedCollection
            Products;

public Promotion()
{
// Initialize this end
            of the association
Products = new SynchedCollection(this,
            "Promotions");
}
}

The SynchedCollection is declared normally along with all other class members. However, the actual instantiation (the call to "new") must be placed in the constructor because a this reference is required. Where does the name "Products" come from? From our knowledge of the structure of the type of child object. We know that only Product objects will be added to this particular SynchedCollection, and each of those has a Promotions collection (containing only objects like themselves). This is the conceptually linked collection that we keep mentioning. Misspelling this collection name can result in runtime errors when the collection can't be found on the Promotion type, so be thorough!

The following client code is from the simple test harness included in the downloadable sample project. The assertions (which pass with flying colors) demonstrate the results of the constraint management logic which the SynchedCollection manages behind the scenes:

Product p = new Product();
p.ProductName = "Beef Ravioli";

Promotion r = new Promotion();
r.Name = "Pasta Clearance Sale";

// Associate the product and promotion together
p.Promotions.Add(r);

// Verify the add operation worked at OTHER end of the association
// (we never explicitly added anything to the r.Products collection!)
Debug.Assert(r.Products.Count == 1);
Debug.Assert(r.Products[0] == p);

Console.WriteLine("Elements were associated successfully");

// Now, remove the association
r.Products.Remove(p);

// Verify the remove operation worked at OTHER end of the association
Debug.Assert(p.Promotions.Count == 0);

Console.WriteLine("Elements were disassociated successfully");

Console.WriteLine("The test was successful");
Console.ReadLine();

Drawbacks and Tradeoffs

When using a SynchedCollection instead of custom child management methods on the involved classes, an element of type safety is lost. An AddOrder method on Customer, for example, can be written to take an explicit Order parameter - an attempt to pass in anything else will result in a failure to compile. Because the SynchedCollection 's Add method accepts a generic object, developers must be wary of collection contents with unexpected object types. This drawback is inherent when directly using any standard .NET collection. (One way to mitigate this risk would be to extend a built-in collection class and allow the user to specify its allowable type, using the System.Type object. The overridden implementation of the Add method could then throw a runtime error if the client code attempts to add any other object to the collection. We won't demonstrate this here, since that's secondary to the purpose of the article).

The SynchedCollection requires a bit more configuration than a plain ArrayList, although this work can be confined to a one-line constructor.

In addition, if you change the name of an involved class, you must remember to change the name of the LinkedCollectionName property as well, or bizarre runtime idiosyncrasies will occur.

Finally, if you're developing a system that will require future maintenance (and, hopefully, you are), prudence dictates more conscientious documentation around declarations and usages of the SynchedCollection than in a more plain implementation.

Conclusion

In this article, we've reviewed the basics of implementing relationships in C#. We've examined the pros and cons of using association classes to implement many-to-many relationships, and we've presented a mechanism called a SynchedCollection as an alternative solution.

The SynchedCollection presented in this article is only a simple beginning. Other modifications can be applied as needed. For example, by deriving from a base class like Hashtable (where collection elements are named as they are added, and can be retrieved by these names), the SynchedCollection could exhibit this naming behavior as well. If additional constraints must be managed (such as prevention of duplicate children), this functionality can be added directly to a SynchedCollection or one of its subclasses.

This article demonstrated that .NET's sophisticated collection class framework, in conjunction with powerful support for inheritance, allow object relationships to be managed in novel ways.


COM Enabling Your .NET Components

COM Enabling Your .NET Components
http://www.csharptoday.com/content.asp?id=1794
Article Rating: 4.5
Published on December 27, 2001
Abstract
In this article, John Vanko describes Microsoft’s vision for interoperability between .NET managed code and COM unmanaged code. The article examines issues of .NET to COM interoperability, how it is accomplished, and extending basic interoperability in the context of migration from COM to .NET. Concepts are given as well as examples. The first section discusses generation of Request Call Wrappers using .NET tools. In situations where the toolset is not enough, a more complex demonstration shows how to define COM classes and interfaces from scratch within a .NET component. The final section shows how to create custom .NET proxies in order to abstract the RCW in order to take advantage of advanced .NET features while still interoperating with a COM object.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

Though the .NET platform offers tremendous advantages to us as Microsoft developers, we are not expected to convert all of our existing code to this next generation platform overnight. We would, however, like to take advantage of the .NET platform as soon as possible. We will cover this problem in this article.

Since the term interoperability is the focus of this article, let's define it further. As far as .NET and COM are concerned there are three major types of interoperability: .NET talking to COM, COM talking to .NET, and .NET talking to unmanaged API functions. In this article, we will examine the first of these areas, .NET to COM interoperability. From an architectural point of view, we use .NET as our base and write new modules on the .NET platform, with COM objects serving the requests of our .NET objects. This can provide us with a smooth migration path from one platform to the other.

This article assumes a basic familiarity with the classic COM model. First, we will observe the use of a standard approach to generating Request Call Wrappers (RCWs). Next, we will take a more in-depth look at what goes on under the hood as we learn to create a RCW manually. Finally, we will take a look at how we can extend the COM model with .NET by creating Custom Managed Wrappers (CMWs).

The code for this article was written using Visual Studio .NET beta 2 service pack 2, containing version 7.00.9254 of the C# compiler and version 1.0.2914 of the CLR.

Introducing .NET to COM Interoperability

In order to integrate our .NET components with COM, it is first necessary to have a basic understanding of the COM model. In this model, any object that wants to communicate with another object is required to expose at least one interface, which holds definition of public methods, properties, fields, and events supported by the object. In essence the interface is a contract that a COM server agrees to support for its clients. These interfaces are described with the Microsoft Interface Definition Language, MIDL (http://msdn.microsoft.com/library/default.asp?url=/library/en-us/rpc/ov-files_6x9h.asp&WROXEMPTOKEN=993439ZKcSILm3p6eWxHUKIBZn). Objects can have more than one interface, which can be grouped into a type library. This type library can either be embedded in the same file as our code, as in the case of a .dll or .ocx file, or it can be put into a stand-alone file with a .tlb extension. The type library information is provided so that COM clients can bind to the types and methods of our component at compile time. In addition, each unmanaged COM object and its interfaces must have unique registered GUIDs. As for error handling, COM methods usually return with an HRESULT, which can be inspected against a constant in order to determine the type of error or success.

In contrast to COM objects, .NET managed objects can communicate directly with other managed objects without the need for a formal COM style interface, eliminating the need for type libraries. Instead, managed components carry descriptive metadata to self describe themselves and their APIs. Members described by this metadata may support other levels of scope besides public. In addition, managed components simplify programming greatly by not requiring registration of a unique GUID for each component, and help to standardize and structure our code by throwing and catching exceptions rather than returning and examining HRESULTs.

The CLR can communicate with unmanaged COM objects on behalf of its managed objects; however, it when it communicates with the unmanaged object in any way, it must do so through the interfaces exposed by the COM object. Therefore, we need to create a managed proxy that describes the interface so that the CLR can create COM coclasses, call methods, and marshal data between its environment and the unmanaged environment. This type of assembly is called a Request Call Wrapper (RCW) and acts as a proxy to an unmanaged COM object, replacing the COM based interface.

In this figure, we see how our managed clients use the RCW as a proxy to the unmanaged environment. The RCW contains metadata, which is used to describe the members of the COM Server to objects running in the managed environment. The CLR then uses the information from the RCW to communicate with the unmanaged object. It is important to note that the metadata need not describe the interfaces IUnknown or IDispatch as is necessary in COM. This is because the CLR will call the IUnknown interface directly on the COM object in order to manage the object life cycle. It is also important to note that one RCW instance will be generated for each COM coclass that is instantiated.

This article focuses on early binding to COM using the RCW. The RCW allows us to bind to the COM object at compile time. This gives us the advantage of type safety as well as enhanced performance. In contrast, we could use the Type class to dynamically discover methods at run time. In this case the CLR would query the IDispatch interface on the COM object.

Ways of creating an RCW assembly from a type library:

  • Use the tblimp.exe utility
  • Use the TypeLibConverter Class
  • Create a custom RCW

As we will see in the next section, the type library utility tlbimp.exe is the standard tool for converting a type library to an RCW assembly. Since it is a command line utility, it offers us great flexibility in building a RCW. It also gives us the ability to embed the calls to tlbimp as lines in a batch (.bat) file.

Classes in the System.Runtime.InteropServices namespace are used to assist the CLR in communicating with COM. One of these classes is the TypeLibConverter class, which translates COM type libraries to .NET metadata and vice versa. TypeLibConverter will give us exactly the same results as tlbimp.exe, with the added advantage of being able to use it as an object within an application and being able to convert type libraries that have been loaded into memory rather than just from files. Keep in mind that the performance should be significantly decreased if you are planning to convert type libraries on the fly at runtime instead of creating a RCW just once.

As we will see later, if the automated methods of producing metadata do not work, you can always create a custom RCW. While using tlbimp and TypeLibConverter are the preferred methods of producing this metadata, learning detailed aspects of this conversion process allows us to tweak the RCW in specialized circumstances as well as giving a better understanding of how the RCW describes and marshals data.

Remote Call Wrappers: The Standard Path

Using the RCW approach is considered a good practice and will be our main method for interfacing with COM coclasses. Even if there are problems with the standard method, it is preferable to modify the COM component or interface to make them ".NET friendly" and regenerate the RCW rather than to create a custom RCW or do custom marshaling. This section demonstrates an example using a standard set of simple steps that one would normally use to generate a RCW and bind to the COM object at compile time.

In order to generate a RCW, we only need to use the tlbimp.exe command line utility. We can convert type library information contained in a .dll, .ocx, or .tlb file into an assembly containing metadata with the following:

tlbimp C:\WINNT\system32\shdocvw.dll /namespace:RCW.Explorer /out:bin\ExplorerRCW.dll

This command generates an RCW called ExplorerRCW.dll from the COM based "Shell Object and Control Library"shdocvw.dll, and is used to access functionality of Internet Explorer. The RCW assembly is placed in our application folder since it is a simple private assembly. Also note that you can define the namespace the RCW will use; in this case RCW.Explorer. If the namespace is not defined, a default namespace will be created with the name given to the RCW itself.

We can look at the metadata inside the RCW by using the ildasm.exe (Intermediate Language Disassembler) utility.

ildasm ExplorerRCW.dll

The command line argument above will display a graphical tree view structure of the interfaces we've just imported. Note the name of the assembly (ExplorerRCW.dll) and the namespace (RCW.Explorer) at the top:

Now let's examine how a RCW can be accessed using client code. In order to successfully access COM coclasses, we need to instantiate the class defined by the RCW and choose an interface to use with it. If we open the RCW.Explorer branch in ildasm we will see the classes, interfaces and enumerations that we've imported. In our example, we will be using the IWebBrowser2 interface (shown below) to access the InternetExplorer class.

Now let's take a look at the client code for our example, located in 1RCW\EndClient.cs of the download package:

using System;

namespace Client
{
using RCW.Explorer; //our RCW namespace

class Driver
{
//parameter defaults
static string pURL =
            "www.csharptoday.com";
static int pWidth = 640, pHeight = 480;

public static void Main(string[] args)
{
//call
            format: Driver [URL] [width] [height]
try
{
if
            (args.Length>0 && args[0] != null) pURL = args[0];
if
            (args.Length>1 && args[1] != null) pWidth =
            Int32.Parse(args[1]);
if (args.Length>2 && args[2] !=
            null) pHeight = Int32.Parse(args[2]);

//create an
            InternetExplorer object and cast it to IWebBrowser2
IWebBrowser2
            oBrowser = (IWebBrowser2)(new InternetExplorer());

//set
            properties
oBrowser.AddressBar = false;
oBrowser.MenuBar =
            false;
oBrowser.StatusBar = false;
oBrowser.ToolBar =
            0;
oBrowser.Height = pHeight;
oBrowser.Width =
            pWidth;

//navigate to the URL
object oNull =
            null;
oBrowser.Navigate(pURL,ref oNull,ref oNull,ref oNull,ref
            oNull);
oBrowser.Visible = true;

//display the name of the
            browser (read-only)
Console.WriteLine("Browser name is
            {0}",oBrowser.Name);

//display the handle of the browser
            window
Console.WriteLine("Browser handle is
            {0}",oBrowser.HWND);

//uncomment this if you want to see a
            COMException being
            thrown!
//oBrowser.GoForward();
}
catch(System.Exception e)
{
//show the type of the exception raised and do a stack
            trace for debugging
Console.WriteLine("Error type:
            {0}",(e.GetBaseException()).GetType());
Console.WriteLine("StackTrace:
            \n" + e.StackTrace);
}
}
}
}

In this example, the Driver class takes three optional arguments: URL, width, and height. It creates the COM coclass, maps it to an interface, and uses it to open a browser and navigate to a web page. Notice that we are using the RCW.Explorer namespace, which is the namespace we gave to our RCW.

We can create the COM object exactly as if it were a managed one, since we are instantiating it through the RCW. We need to cast the object to the interface we want to use to access it with. In this case we are casting the InternetExplorer object to the IWebBrowser2 interface, as in the following snippet:

IWebBrowser2 oBrowser = (IWebBrowser2)(new InternetExplorer());

Finally, the example sets properties such as .Width and .Height. It calls the .Navigate method to navigate to a web page and then displays the browser window by setting .Visible to true.

Note that we are catching any Exceptions inside a try/catch block. By default, a HRESULT generated by COM is mapped to a COMException by the CLR. The COMException class has the following inheritance structure:

Object
Exception
SystemException
ExternalException
COMException

Also notice, the tlbimp utility will also create by default .NET style event handlers for each of the events, for instance, DWebBrowserEvents2_NavigateComplete2EventHandler, which is fired upon successful navigation to a web page and can be used as a delegate when added to an event sink.

We need to compile our client with a reference to the RCW (using the /R switch) in order to bind our code to it.

csc /R:bin\ExplorerRCW.dll /out:bin\EndClient.exe EndClient.cs

This will generate a managed application called EndClient.exe that we can call from the command line to open browser objects.

Developers familiar with ASP.NET can also use RCWs directly from web scripts. If we created a RCW for a component appropriate for use by a web server, such as a file upload component, we could place a RCW named FileUploadRCW.dll in the \bin folder of our virtual directory. In ASP.NET, this folder has the responsibility of holding managed components for use by the web scripts. To access the RCW component from a script, just add the name of the RCW as a namespace in an import directive, as follows:

<%@ Import Namespace="FileUploadRCW" %>

It is as easy as that. In your web scripts now you can instantiate an Uploader object within the component referenced by FileUploadRCW normally, as in the following code:

Uploader oUl = new Uploader();

If you want to use Visual Studio to create a RCW, the steps to take are even quicker. First, open your project and go to Project | References and then to the COM tab.

Next, select the COM object you wish to bind to and click OK. The tool presents you with a dialog:

Click OK, and Visual Studio creates a RCW automatically, placing it in the bin\debug folder and giving it a name with the prefix Interop and a version number. In this case, the filename given is Interop.SHDocVw_1_1.dll.

Notice that Visual Studio automatically sets our main thread to use the Single Threaded Apartment (STA) threading model in Form1.cs by using the STAThread attribute:

[STAThread]
static void Main()

This is one way of setting the default threading model for your main thread. This is important because if the threading models of your .NET code and the COM component conflict, an additional proxy will be generated in between them, affecting performance. Keep in mind that legacy VB components (VB6 and earlier) can only use STA (Single Threaded Apartment) threading (i.e. they cannot make use of the MTA "Multi Threaded Apartment" model). If you need some background information about the STA and MTA threading models, see http://msdn.microsoft.com/library/default.asp?url=/library/en-us/com/aptnthrd_8po3.asp&WROXEMPTOKEN=993439ZKcSILm3p6eWxHUKIBZn.

At this point our project is already bound to the COM object so we can use the same code as in the command line example to access it. The IDE really makes our lives easier. Of course, in these examples we also assume that everything is going fine, without any of the unexpected complications that we see all too commonly in the work environment.

Custom RCWs: Rolling your own

Using tlbimp is not always possible. Sometimes the utility is not able to convert the type library information, or an older component may not even have a type library. We might also decide not to accept the default type conversion or marshaling done by tlbimp, and even if one never has to create a custom RCW, it's a good idea to get exposure to this method in order to deepen your understanding of how things work between managed and unmanaged code, or for when we run into a situation where we have to convert a class now and it's just not functioning correctly. With an exposure to writing these classes and interfaces manually, we gain insight into MSIL, metadata, and how the tlbimp is doing its job. In this section, we will look at a simple example that manually creates a RCW.

Manually defining COM objects and their interfaces in our C# code requires the use of attributes. These attributes will give a special significance to our objects that defines them as being used for COM. One of the benefits of this method is that it is easier to compile everything into one assembly, instead of having the RCW and our client classes defined in separate assemblies. This will reduce the number of files required in a distribution package, and eliminates the need to remember which RCW assembly to bind to your client assembly. On the downside, it is much more complex to define an RCW by hand, and requires a greater knowledge of COM interfaces as well as .NET attributes.

The following example shows how a custom RCW can be defined for a VB6 object with the public methods Add(), Div(), Mult(), and the AddStr() method that concatenates strings. The code for this example is located in 2CRCW\EndClient.cs of the download package:

using System;
using System.Runtime.InteropServices; //this namespace contains attributes

//this namespace will hold our custom RCW definitions
namespace CalculatorsCRCW
{
//define the _CMath interface
//VB uses InterfaceIsIDispatch for InterfaceType
//C++ may use InterfaceIsDual to designate IDispatch/Iunknown
//GUIDs taken from Ole View of Calculators.dll
[Guid("3A77CDAD-657A-424D-A6AA-6B5D26DDCBFC"),
InterfaceType(ComInterfaceType.InterfaceIsIDispatch)]
interface _CMath {

//These members use blittable types
int Add( [In] int p1, [In] int p2, [In, Out] ref int p3);

int Div([In] int p1, [In] int p2);

int Mult([In] int p1, [In] int p2);

//This member uses non-blittable types
[return: MarshalAs(UnmanagedType.Interface)]
object AddStr([In, MarshalAs(UnmanagedType.BStr)] string str1,
[In, MarshalAs(UnmanagedType.BStr)] string str2);

}

//define the CMath coclass
//VB needs to specify AutoDispatch for ClassInterface Attribute
[ComImport, Guid("D097DE33-E75A-445B-AE01-609399DCFC15"),
ClassInterface(ClassInterfaceType.AutoDispatch)]
class CMath {}

//define _CStringGroup interface
[Guid("D569E598-2FB7-45AA-B3E1-AEFE82124C76"),
InterfaceType(ComInterfaceType.InterfaceIsIDispatch)]
interface _CStringGroup {

[return: MarshalAs(UnmanagedType.BStr)]
string GetConcat();

[return: MarshalAs(UnmanagedType.BStr)]
string GetString1();

[return: MarshalAs(UnmanagedType.BStr)]
string GetString2();
}

//define the CStringGroup coclass
[ComImport, Guid("EE8B3578-31AF-4C78-9EE7-718E666D9419"),
ClassInterface(ClassInterfaceType.AutoDispatch)]
class CStringGroup {}

}
//The definition for our Driver client class goes here
}

Let's break down the above example. In order to define interfaces to COM we need to use the attribute classes from the System.Runtime.InteropServices namespace mentioned earlier. Basically, we define a managed interface type, and a managed class type. However, we mark each of them with attributes defining them as being from a COM type library and a COM coclass. Notice the following snippet:

[Guid("3A77CDAD-657A-424D-A6AA-6B5D26DDCBFC"), InterfaceType(ComInterfaceType.InterfaceIsIDispatch)]
interface _CMath {

The Guid attribute is required to define a COM style interface. This can be obtained using Ole Viewer, or by generating a RCW and taking a look at the extra information for the interface under the .class extra information for the class or interface.

The InterfaceType attribute follows and is also required to compile our example. Also, since this is an automatically generated VB6 component, we need to use the setting ComInterfaceType.InterfaceIsIDispatch since the interface does not include IUnknown. If this were a component generated by a language like C++ using the MSIL compiler, we could instead use InterfaceIsDual, to specify that the GUID refers to both the IDispatch and IUnknown interfaces. It is important to note that this type of interface cannot extend other interfaces, i.e., it must be a base interface type.

[ComImport, Guid("D097DE33-E75A-445B-AE01-609399DCFC15"), ClassInterface(ClassInterfaceType.AutoDispatch)]
class CMath {}

Above is the class definition that points to our COM coclass. The ComImport attribute is necessary to mark this as a coclass. As with the interface definition, we also must specify the Guid attribute. Since this is a VB6 object, we must also specify the ClassInterface attribute with a setting of ClassInterfaceType.AutoDispatch. This specifies that only an IDispatch interface should be generated and exposed to COM. The class definition itself must be empty since it is a coclass, and cannot inherit any other type except for the base object class, which it inherits by default. Also note that an empty constructor is provided by default, and that parameterized constructors cannot be used.

//These members use blittable types
int Add([In] int p1, [In] int p2, [In, Out] ref int p3);

int Div([In] int p1, [In] int p2);

int Mult([In] int p1, [In] int p2);

//This member uses non-blittable types
[return: MarshalAs(UnmanagedType.Interface)]
object AddStr([In, MarshalAs(UnmanagedType.BStr)] string str1,
[In, MarshalAs(UnmanagedType.BStr)] string str2);

In the code above, the Add() method returns an int and takes 3 parameters. The first two are defined as input parameters, using the [In] attribute, passed by value. The last parameter takes a reference to an int and is described as being both input and output using the [In, Out] attribute. Using the [In, Out] attribute followed by the ref modifier tells the CLR that the parameter is to be passed by reference.

It is important to note that these methods should be listed in the same order as in the interface, or a compiler error may occur.

The functions Add(), Div() and Mult() above all use what is known as blittable types. This means that they use types whose data representation is the same in both managed and unmanaged environments, in a one to one fashion. In contrast, non-blittable types are types that do not have a direct one to one memory mapping, such as strings and dates. For instance, you could have a string composed of ANSI characters in an unmanaged environment that needs to be converted to a Unicode string to be used in a managed environment. For each non-blittable conversion that the interop marshaler performs, it incurs a performance penalty, so we try to keep the use of non-blittable types to a minimum. This may involve rethinking your COM interfaces and making them more .NET friendly (so long as these interface changes don't require more work than rewriting the component!).

The MarshalAs attribute is used to specify how your .NET type will be marshaled to COM or vice versa. Usually this attribute is used to specify how a non-blittable type is mapped to an unmanaged type. In the snippet above, we see that the AddStr() method takes 2 arguments which are marshaled as BSTR strings and returns an object type. We see that the return type of this object is marshaled as UnmanagedType.Interface, so we must also define the actual CStringGroup class that is returned and its interface _CStringGroup in our code, as seen above.

Now let's take a take a look at the Driver client class which we use to call our manually defined RCW, located in 2CRCW\EndClient.cs of the download package:

//The client class to test our custom RCW
public class Driver {

public static void Main()
{
try
{
//create an instance of
            Math
CalculatorsCRCW.CMath cmath = new
            CalculatorsCRCW.CMath();

//cast our cmath object to the
            _CMath interface
CalculatorsCRCW._CMath icmath =
            (CalculatorsCRCW._CMath) cmath;

//call the Add method - the
            result is returned by reference in resAdd
int resAdd =
            0;
icmath.Add(10,5, ref resAdd);

//call the Div and Mult
            methods and return the results
int resDiv =
            icmath.Div(10,5);

int resMult =
            icmath.Mult(10,5);

//cast the returned type to the
            _CStringGroup interface
CalculatorsCRCW._CStringGroup sg =
            (CalculatorsCRCW._CStringGroup)icmath.AddStr("string1","string2");

//display
            the results
Console.WriteLine("Add result is {0}", resAdd); //15
Console.WriteLine("Div result is {0}", resDiv); //2
Console.WriteLine("Mult result is {0}", resMult); //50
Console.WriteLine("String result is {0}", sg.GetConcat());
            //string1string2
}
catch (Exception e)
{
Console.WriteLine(e.StackTrace);
}
}
}

Note that this class creates an instance of the CMath class and casts it to the _CMath interface just as it would for any RCW. Methods are also called in a normal fashion. Remember that the AddStr method returns the CStringGroup, which requires special marshaling defined in our code.

Now we compile this example into an RCW using the following command:

Csc CustCalc.cs

Let's take a look at how the attributes in our AddStr() method have been converted. Click on the node for the method, as shown below:

Notice in the figure above that we have the CMath and CStringGroup classes as well as the _CMath and _CStringGroup interfaces included in the CalcRCW namespace. The Driver class is also included in this assembly, but in the default namespace.

The following view shows the metadata generated from the AddStr() method defined in our class. We can find this information by double clicking on the AddStr() method. Notice that it resembles some of the attribute definitions we provided for the AddStr() method in our custom RCW.

.method public hidebysig newslot virtual abstract
instance object
marshal( interface)
AddStr([in] string marshal( bstr) str1,
[in] string marshal( bstr) str2) cil managed
{
} // end of method _CMath::AddStr

When we run the EndClient.exe from the command line we get the following:

Custom Managed Wrappers: Wrapping the RCW

As we have seen in the past, COM based coclasses have certain limitations. In the strict sense of inheritance, these classes cannot inherit from or be inherited by other COM classes. Class based inheritance is non-existent; only interface based inheritance is available in COM. Furthermore, these classes cannot use parameterized constructors or use static methods inside our .NET code. Why not take advantage of the full features that .NET has to offer? To accomplish this, we can create a Custom Managed Wrapper (CMW), which can act as a proxy or façade to our RCW in order to produce objects that are composed of both .NET and COM code.

Some of the things CMWs are good for include:

  • maintaining well known APIs intact while extending code,
  • reducing chatty interfaces,
  • storing state data in .NET components,
  • moving a DCOM boundary,
  • and translating complex data structures.

In approaching the development of CMWs, we can use standard software development concepts. Inheritance, containment, and aggregation are three ways we may use to extend COM code with .NET code. On an abstract level, let's take a look at each of these and then follow up with brief examples of inheritance and containment.

Inheritance

In the diagram above, the Car and Plane classes (both managed .NET classes) inherit from the class MotorVehicle, a COM class. Car overrides the Move() and Halt() methods defined in MotorVehicle, and adds a new Turn() method, accepting the MotorVehicle implementation of StartEngine(). Plane merely overrides Move() and Halt().

This example is expressed in the code below, located in 3CMW\Inheritance\EndClient.cs of the download package:

using System;
using BaseClassesRCW;
            //BaseClassesRCW.dll

namespace
            DerivedClasses
{
//inherit from the RCW (COM) class
//(default interface is used)
public class Car : MotorVehicle
            {

private int Doors = 4;

//notice we are overriding
            every method from MotorVehicle
public override string Move()
            {
return "Car Move()";
}

public override string Halt()
            {
return "Car Halt()";
}

//forward this method to the
            base class
public override string StartEngine() {
return
            base.StartEngine();
}

//Car extends MotorVehicle
            functionality by adding Turn()
public string Turn(int pDegrees)
            {
return "Turn() " + pDegrees + " Degrees";
}

//constructors
public Car() {}

public
            Car(int pDoors) {
Doors = pDoors;
}
}

public class
            Driver {
public static void Main(string[] args) {
//create a
            new Car and call its methods
Car c = new
            Car();

Console.WriteLine("Move(): {0}", c.Move()); //Car
            Move()
Console.WriteLine("Halt(): {0}", c.Halt()); //Car
            Halt()
Console.WriteLine("StartEngine(): {0}", c.StartEngine());
//MotorVehicle StartEngine()
Console.WriteLine("Turn(): {0}",
            c.Turn(90)); //Turn() 90 degrees
}
}
}

The way that this type of inheritance works is that if we override one method from the coclass, they all must be overridden, so we use the override keyword on each method as shown here in Move():

    public override
            string Move() {
return "Car Move()";
}

We could have also used the new keyword, however, this specifies a semantic break with the original methods and is not our intent. In cases where we want to keep the functionality of the original method, we can use the base keyword, as shown in the StartEngine() method:

    public override
            string StartEngine() {
return base.StartEngine();
}

Here is the result of the running the code in the download:

Containment

Our next example demonstrates the concept of containment. This concept is also known as composition in UML, and describes a case where one object keeps another as one of its private members. It then creates and controls access to this class. Aggregation is similar to containment, with subtle differences; the main difference being that the contained class is made visible to outside classes and they are able to reference it.

The diagram above shows a conceptual composition of class NetData containing AdoDataclass. NetData represents an object in our CMW, which accesses AdoDataclass in the COM component's RCW. This offers many advantages to us, including being able to convert complex data types returned from ADO. For instance, AdoDataclass contains the method GetRecordset() that returns an ADO Recordset object. In order to call this method from .NET, our client has to provide references to ADO in order to use the Recordset. Instead, we can create a CMW with a new method called GetArray(), which can convert the recordset to a string array, or even to a .NET DataTable, and return it to the client. This allows .NET clients to use ADO.NET instead of ADO and loosely couples our .NET data object from the COM data object, making migration easier.

The code below shows the definition of our CMW, which contains the NetData class. Notice that the code is using the COM.Dataclasses namespace, which is included in the RCW and contains the AdoDataclass class as a private member. Each time a NetData is instantiated, its constructor also creates a new instance of the AdoDataclass, oDat. When the GetArray() method is called, it in turn calls the GetRecordset() method from oDat and proceeds to populate one of the fields in each record into a string array which it returns to the client. This source for the following example is located in 3CMW\Containment\NetDataClasses.cs of the download package:

using System;
using COM.Dataclasses;
            //DataclassesRCW.dll
using System.Data;

namespace
            Net.Dataclasses
{
public class NetData
{
//the
            coclass member is hidden from outside access
private AdoDataclass
            oDat = null;

public NetData()
{
//tie creation of
            AdoDataclass to the creation of NetData
oDat = new
            AdoDataclass();
}

public string[] GetArray(string sConn,
            string sSql, int iFld)
{
try
{
//call GetRS and return
            a recordset
MSADO25._Recordset rs =
            (MSADO25._Recordset)oDat.GetRecordset(sConn, sSql);

//get the
            number of results
string[] results = new
            string[rs.RecordCount];

//loop through the ADO recordset,
            populating our string array with each result
int ix=0, ctr=0;;
while (rs.EOF != true) {

System.Collections.IEnumerator
            flds = rs.Fields.GetEnumerator();

ix = 0;
while
            (flds.MoveNext()) {
MSADO25.Field current_cell =
            (MSADO25.Field)flds.Current;
if (ix==iFld) results[ctr] =
            (string)current_cell.Value;
ix++;
}

ctr++;
rs.MoveNext();
}

return
            results; //return the string array to the
            caller
}
catch(System.Exception e)
            {
Console.WriteLine(e.StackTrace);
return
            null;
}
}

}
}

Note that NetData can also provide a GetDataset() method that translates the ADO Recordset to an ADO.NET DataTable or even a DataSet before we return it. This method can provide an excellent transition between migrating from COM to .NET.

Here is our code for the end client accessing this CMW, located in 3CMW\Containment\EndClient.cs of the download package:

using System;
using Net.Dataclasses;
            //NetDataclasses.dll

namespace EndClient
{
class
            Driver
{
//change the path in the following line to the
            location where you copied ImageList.mdb
static private string
            conn =
"Driver={Microsoft Access Driver (*.mdb)};
            DBQ=D:\\net2com\\3CMW\\Containment\\ImageList.mdb";

public
            static void Main(string[] args)
{
try
{
//create the
            CMW
NetData oNd = new NetData();

string[] results =
            oNd.GetArray(conn, "select * from OpenedImages", 0);

foreach
            (string s in results) {
Console.WriteLine("Field value={0}",s);
}

}
catch(System.Exception e)
{
Console.WriteLine(e.StackTrace);
}
}
}
}

In the above code, we use the Net.Dataclasses namespace, which is the namespace for our CMW. Our client creates an instance of NetData and calls its GetArray() method with parameters for the connection string, SQL string to execute, and the field number to return from the results. This populates an array of strings containing the field values and displays them on the console, as seen below:

Conclusion

The figure below shows a high level view of how the use of these CMW components can ease our migration from COM to .NET. In the data tier, the containment CMW acts as a new, separate component that will completely take over the responsibilities of the COM object in time. In the business tier, the inheritance CMW continues to expose well known API of the COM object, gradually adding improved functionality to a COM object.

We have covered a lot of ground in this article. Starting with the RCW, we have seen how to generate a RCW using command line tools as well as with Visual Studio. We have taken a deeper look at RCWs by learning to create our own custom RCWs that were then translated into MSIL and metadata. We have also taken a look at some methods of extending COM functionality using managed code. In conclusion, we have been exposed to some of the important features that we will need to know for integrating our current and future systems with the .NET platform.

C#Today - ©2002 Wrox Press Limited


Word Encryption Add-In

Word Encryption Add-In
http://www.csharptoday.com/content.asp?id=1724
Article Rating: 4.5
Published on December 24, 2001
Abstract
In the first two articles of this series we discussed how Visual Studio .NET can be used to build add–ins that can be hosted by the Visual Studio .NET IDE. These .NET add–ins were built using the Add–in Wizard that is a part of Visual Studio .NET; the wizard generated all the boiler plate code for writing the add–in. One major limitation of this Add–in Wizard, however, is that it only supports add–ins for the Visual Studio .NET environment. Add–ins are also hosted by the various Microsoft Office products, but the .NET add–in wizard does not currently support these applications as a host. In this article, Kaushal Sanghavi discusses how we can develop COM based add–ins for Microsoft Office, and how we can use .NET components in these add–ins via COM Interop. We will build an encryption add–in for Microsoft Word that uses the symmetric DES encryption algorithm to encrypt a part of a Word document based on a user defined password.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

In the first two articles of this series (http://www.csharptoday.com/content.asp?id=1619&WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0 and http://www.csharptoday.com/content.asp?id=1551&WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0), we discussed how Visual Studio .NET can be used to build add-ins that can be hosted by the Visual Studio .NET IDE. These .NET add-ins were built using the Add-in Wizard that is a part of Visual Studio .NET; the wizard generated all the boiler plate code for writing the add-in. One major limitation of this Add-in Wizard, however, is that it only supports add-ins for the Visual Studio .NET environment. Add-ins are also hosted by the various Microsoft Office products, but the .NET Add-in wizard does not currently support these applications as a host. In this article, we will discuss how we can develop COM based add-ins for Microsoft Office, and how we can use .NET components in these add-ins via COM Interop (the runtime capability that allows calls between managed and unmanaged components). This article will focus on building an encryption add-in for Microsoft Word that uses the 56 bit symmetric DES encryption algorithm to encrypt a part of a Word document based on a user defined password. The encrypted data is stored as part of the Word document, and when the user wants to decrypt it, she will have to enter the same password that was used for encryption.

Requirements

Note that this article is based on Visual Studio .NET Beta 2, Visual Basic 6.0, Office XP and Windows 2000. While we have not tested this on earlier versions of Office, readers should be able to use this by changing the references to the Office XP (Office 10) components to the version on their machine.

Microsoft Office Add-ins

Microsoft Office add-ins are COM components that implement the IDTExtensibility2 interface. This interface contains methods that are called when an add-in is loaded or unloaded. Further, this interface contains the methods that can be used to add custom user interface elements for the add-in, such as custom menus or toolbars. This interface contains the following methods:

  • OnAddInsUpdate: This method is called by the application hosting the add-in when the list of currently installed add-ins changes. This happens when a new add-in is installed, or an existing add-in is unloaded. This method is fired for all the active add-ins in the host application, and can be used by an add-in to gain information about other add-ins that are active.
  • OnBeginShutdown: This method is called by the application hosting the add-in, when the application is being shut down. This method is only called for active add-ins and can be used to perform clean up, or to save the state of the add-in.
  • OnConnection: This method is called by the application hosting the add-in, when the add-in is being loaded. If the add-in is marked to load on startup, this method will be called during application startup, and if the add-in is loaded by the user after the host application has started, this method will be called when the user loads the add-in. Since this method is called once when the add-in is first being loaded, this is the perfect place for the add-in to draw custom user interface elements, such as adding menus or toolbars.
  • OnDisconnection: This method is called by the application hosting the add-in, when the add-in is being unloaded. The OnDisconnection method is generally used to perform cleanup functions or to save the state of the add-in.
  • OnStartupComplete: This method is called by the application hosting the add-in, when the hosting application has finished its startup procedures. Note that this method is called only when the add-in is marked to load on application startup.

Any traditional COM component that implements the IDTExtensibility2 interface qualifies as an add-in. Microsoft Office uses specific keys in the registry to load an installed add-in. Any component that needs to be loaded as a Word add-in needs to add an entry to the registry tree as shown below. The key name is the ProgID of the COM component that implements the IDTExtensibility2 interface. The FriendlyName is a description of the add-in, and the LoadBehavior specifies how the add-in should be loaded.

For further details on add-in basics and fundamentals, please refer to the earlier articles in this series. The rest of this article will focus on building an encryption add-in for Microsoft Word.

VB 6.0 Add-in Component

The Word encryption add-in consists of two components; a VB component that interfaces with Word and provides a custom toolbar for encryption and decryption, and a C# component that implements the actual encryption algorithm. Since, currently it is difficult to directly build a .NET add-in for Microsoft Word, we follow this two component approach. Future versions of Office will support directly writing .NET components that can be loaded by Office. As you can see from the diagram below, Word loads the VB COM component that uses Interop to create and invoke the .NET component.

Let us now focus on the VB component. The main purpose of the VB component is to create the Encryption toolbar as shown in the following screenshot. This toolbar supports two actions, Encrypt and Decrypt.

The VB component is a standard ActiveX DLL project (note that VB provides an Add-in project option as well, but for this add-in, a simple ActiveX DLL suffices). This component implements the IDTExtensibility2 interface, but the only method that does any processing is the OnConnection method. The rest of the methods in this interface are simply stubs with no processing, as shown in the following code block.

Private Sub IDTExtensibility2_OnAddInsUpdate(custom() As Variant)
End Sub

Private Sub IDTExtensibility2_OnBeginShutdown(custom() As Variant)
End Sub

Private Sub IDTExtensibility2_OnDisconnection
(ByVal RemoveMode As AddInDesignerObjects.ext_DisconnectMode,
custom() As Variant)
End Sub

Private Sub IDTExtensibility2_OnStartupComplete(custom() As Variant)
End Sub

Before we take a look at the OnConnection method, let us look at the Project References for this VB component. The Microsoft Add-In Designer component is used for referring to the IDTExtensibility2 interface and the Office 10 and Word 10 Object Libraries are used for referring to the Office object models. If the reader is using a different version of Office (like Office 97 or Office 2000), she will have to change these references to point to the correct libraries.

Let us now take a look at the OnConnection method for this component, which creates the custom encryption menu.

Dim mWord As Word.Application
Dim mCB As Office.CommandBar
Dim mCBs As Office.CommandBars
Dim WithEvents mEncryptButton As CommandBarButton
Dim WithEvents mDecryptButton As CommandBarButton

Private Sub IDTExtensibility2_OnConnection(ByVal Application As Object,
ByVal ConnectMode As AddInDesignerObjects.ext_ConnectMode,
ByVal AddInInst As Object, custom() As Variant)

Set mWord = Application
Set mCBs = mWord.CommandBars
Set mCB = mCBs.Add(Name:="Encryption", Temporary:=True)

Set mEncryptButton = mCB.Controls.Add(msoControlButton)
mEncryptButton.Caption = "Encrypt"
mEncryptButton.Style = msoButtonCaption

Set mDecryptButton = mCB.Controls.Add(msoControlButton)
mDecryptButton.Caption = "Decrypt"
mDecryptButton.Style = msoButtonCaption

mCB.Visible = True
mEncryptButton.Visible = True
End Sub

When an add-in is connected, the Word environment passes the add-in a reference to its Application object. This object is the root of the application object model and can be used to access all the elements in the application. The OnConnection method stores this in a member variable, and then uses this to access the CommndBars collection to create a new CommandBar. This represents the encryption toolbar, and then we add two control buttons to this toolbar, that represent the Encrypt and Decrypt buttons that the user can click to encrypt or decrypt the data. As you can see, this is fairly straightforward; we simply call the methods exposed by the Word Application object.

Let us now look at the code behind the Encrypt button:

Private Sub mEncryptButton_Click(ByVal Ctrl As Office.CommandBarButton, CancelDefault As Boolean)

Dim lSelection As Word.Selection
Set lSelection = mWord.Selection

Dim lEncryptedStr As String
Dim lDESObject As Object

Set lDESObject = CreateObject("DESObjectNS.DESObject")
lEncryptedStr = lDESObject.Encrypt(lSelection.Text)

Call lSelection.Delete

lSelection.InsertAfter ("{START_ENCRYPT__" & lEncryptedStr & "__END_ENCRYPT}")
End Sub

For a section of text to be encrypted, the user needs to select the text, and then click on the Encrypt button. The Word Application object exposes a Selection object that can be used to access the text that the user has selected. We read this selected text, and create an instance of the DESObjectNS.DESObject object. This is our C# component that implements the encryption algorithm. This object exposes an Encrypt method that takes in a string, and returns the encrypted string.

This encrypted string will usually contain non alphanumeric characters. For the decryption to work successfully, the exact encryption string has to be passed to the C# object. In order for the user to be able to exactly identify the encrypted string, we surround the encrypted string with the "{START_ENCRYPT__ " and "__END_ENCRYPT} " tags. During decryption, the user should select all the data between these two tags, and pass it to the decryption method. The following screenshot shows the encrypted string representing the string "abcd". Note that this representation is a function of the password that we used; if you use a different password, the encrypted string will be different from the one shown below.

We use the Delete and InsertAfter methods of the Selection object to delete the original text, and replace it with the encrypted text:

The Decrypt function is very similar to the Encrypt function as shown below, with the only difference being that we call the Decrypt method on the DESObject in this case.

Private Sub mDecryptButton_Click(ByVal Ctrl As Office.CommandBarButton, CancelDefault As Boolean)

Dim lSelection As Word.Selection
Set lSelection = mWord.Selection

Dim lDecryptedStr As String
Dim lDESObject As Object

Set lDESObject = CreateObject("DESObjectNS.DESObject")
lDecryptedStr = lDESObject.Decrypt(lSelection.Text)

Call lSelection.Delete
lSelection.InsertAfter (lDecryptedStr)

End Sub

As you can see, the VB component is fairly simple. We use it as a conduit to get to the C# component which provides the real functionality, as discussed in the next section.

C# Encryption Component

The C# encryption component uses the DES algorithm to implement a symmetric encryption based on a used inputted password. A detailed discussion of cryptography is beyond the scope of this article, (although the reader may wish to read Richard Conway's article on the subject at http://www.csharptoday.com/content.asp?id=1614&WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0) but a symmetric algorithm is one that uses the same secret key for both encryption and decryption. An asymmetric algorithm on the other hand, uses a key pair, one for encryption and the other for decryption. The following diagram shows a symmetric algorithm.

In our sample, we will use a user input password as the secret key. Note that this same password will be required to decrypt the data to restore it to its original state. If the user forgets this password, the data will be lost. We have no way to restore the data, if the password used for encryption is not available.

The C# component consists of a form that is used to accept the user password, and a class that will use this password as a key to encrypt the data. Since DES requires an 8 byte key, we require the user to enter an 8 character password. The password form is shown below; this checks whether the password is exactly 8 characters in length, and whether the passwords match.

Let us now look at the code that is used for encrypting a string of data based on the user password:

public string EncryptData (string iPwd,
            string iData)
{
byte[] lIV = ConvertStringToByteArray
            (iPwd);
byte[] lKey = ConvertStringToByteArray (iPwd);
string
            lOrigData = iData;
byte[] lByteData = null;
byte[] lEncrData =
            null;
ICryptoTransform lCT;

try{
DES lDES = new
            DESCryptoServiceProvider();
int lBlockSize = lDES.BlockSize /
            8;
int lPad = lBlockSize - lOrigData.Length % lBlockSize;
if
            (lPad > 0 && lPad != lBlockSize)
lOrigData =
            lOrigData.PadRight (lOrigData.Length + lPad,
            mPadChar);

lByteData = ConvertStringToByteArray
            (lOrigData);
lEncrData = new byte
            [lByteData.Length];

lDES.Key = lKey;
lDES.IV = lIV;
lCT
            = lDES.CreateEncryptor ();

for (int lLen = lByteData.Length,
            lCurIndex = 0;
lCurIndex < lLen; lCurIndex +=
            lBlockSize)
lCT.TransformBlock (lByteData, lCurIndex, lBlockSize,
            lEncrData, lCurIndex);

}
catch (Exception
            lE){
MessageBox.Show (lE.Message,
            "EncryptData");
}

string lEncrStr =
            ConvertByteArrayToString (lEncrData);
return
            lEncrStr;
}

The DES algorithm is based on a secret key and an initialization vector that is used to encrypt the data. This key and vector is a byte array of 8 bytes that is generated from the password that the user enters. The System.Security.Cryptography provides classes that provide different encryption and hashing algorithm implementations. The DESCryptoServiceProvider class provides an implementation of the DES algorithm, and we use that to obtain a new instance of the DES object.

Most encryption algorithms don't work on the entire stream of data. Instead, they accept small chunks of data and work on the chunks one at a time. This chunk of data is known as a block, and each algorithm accepts data only in a block size. The DES object exposes a BlockSize property that returns the block size for this implementation in bits. We convert it to bytes by dividing it by 8, and if the input data is not an exact multiple of the block size, we pad it with a pad character to get it to an integral block size multiple. In this sample, we use a space character as the pad character, and during decryption, we remove the padding in order to return the data in its original format. Since encryption algorithms work only on byte arrays, we convert all our string data to a byte array, and then call the DES properties to specify the secret key and the initialization vector.

We then call the CreateEncryptor method to return an instance of an object that supports the ICryptoTransform interface. This is the interface that is responsible for transforming the data, and this is done by calling the TransformBlock method on this interface. Note that we call the TransformBlock in a repeated loop, passing only one block of data at a time. Since the data is padded to be an exact multiple of the block size, we can safely call this with one full block at a time till the entire data is transformed. Once the entire data is transformed, we convert the byte array into a string and return it to the VB 6.0 component. As we mentioned in the earlier section, this string will contain non alphanumeric data.

The DecryptData method is almost identical to the EncryptData method. The only difference is that we don't need to pad the data since the encrypted string is guaranteed to be a block size multiple, and we call the CreateDecryptor method on the DES object to obtain an instance of the object that will decrypt the data. Again, we repeatedly call the TransformBlock method until the entire data is decrypted. We convert this to a string, and trim it to get rid of any pad characters, before finally returning it to the VB 6.0 component.

public string DecryptData (string iPwd,
            string iData)
{
byte[] lIV = ConvertStringToByteArray
            (iPwd);
byte[] lKey = ConvertStringToByteArray (iPwd);
string
            lOrigData = iData;
byte[] lEncrData = null;
byte[] lDecrData =
            null;
ICryptoTransform lCT;

try{
DES lDES = new
            DESCryptoServiceProvider();
int lBlockSize = lDES.BlockSize /
            8;

lEncrData = ConvertStringToByteArray
            (lOrigData);
lDecrData = new byte
            [lEncrData.Length];

lDES.Key = lKey;
lDES.IV = lIV;
lCT
            = lDES.CreateDecryptor ();

for (int lLen = lEncrData.Length,
            lCurIndex = 0; lCurIndex < lLen; lCurIndex +=
            lBlockSize)
lCT.TransformBlock (lEncrData, lCurIndex, lBlockSize,
            lDecrData, lCurIndex);
}
catch (Exception
            lE)
{
MessageBox.Show (lE.Message,
            "DecryptData");
}

string lDecrStr =
            ConvertByteArrayToString (lDecrData);
return Trim
            (lDecrStr);
}

Note that we do not store the user password that is used to encrypt the data anywhere in the data itself. Thus, if the user enters a wrong password to decrypt the string, no error will be thrown, since we cannot check to determine an incorrect password. However, since the algorithm uses this password to decrypt the data, the decrypted data will be entirely different from the encrypted data. This is a fundamental principle of all symmetric algorithms, the secret key and only the secret key can be used for decrypting the data; no other key will generate the original data.

Attached Sample Code

The attached sample code consists of 2 projects; EncryptAddin which is the VB 6.0 component that is the Word add-in, and DESObject which is the C# encryption component. Please follow the steps below to install this add-in.

  • Merge the contents of Install_EncryptAddin.reg into your registry. This will create an entry for the add-in that is recognized by Microsoft Word.
  • Register EncryptAddin.dll.
  • Compile DESObject.
  • Copy DESObject.dll to the folder where Microsoft Word is installed. If you have followed the default setup for Office, this will be under C:\Program Files\Microsoft Office\Office10. This is a very important step; if this component is not found in the same location as the Word executable, the add-in will not load.
  • Register the C# component by running the following command. This is very important for the Interop layer to treat a managed component as a COM component.

regasm DESObject.dll /tlb:DESObject.tlb

We again stress that the password used for encryption has to match the password used for decryption. Without this, the encrypted data will be lost, and it will be impossible to retrieve the original data. Further, the user should pass the exact same string returned by the Encrypt method to the Decrypt method. Please use the "{START_ENCRYPT__ " and "__END_ENCRYPT} " tags to help you identify the encrypted string. During decryption, we do not delete these tags; the user will have to do this manually.

Conclusion

Writing add-ins that extend an application's capability is a very simple process. All Office applications, current versions of Visual Studio, and future versions of Visual Studio .NET support this functionality. Further, as we demonstrated in this sample, it is very easy to build an add-in that uses a mix of tradition COM components, and .NET components. This article concludes the series on writing Add-Ins in C#.

Happy .NETing.

If you'd like to discuss a broader range of C#Today topics and issues, or if you have any C# related questions you need answers to, please visit our online resource centres at : http://p2p.wrox.com/list.asp?list=csharptoday_discuss&WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0 and http://p2p.wrox.com/list.asp?list=c_sharp&WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0


Exploring the System.IO namespace

Exploring the System.IO namespace
http://www.csharptoday.com/content.asp?id=1721
Article Rating: 4
Published on December 19, 2001
Abstract
An application often may need to import or export data. This information can be anywhere; present in a file, on a disk, somewhere on the network, in variables in memory, or in another program. The System.IO namespace contains a wide assortment of classes, interfaces and structures used to read and write data that can be in a wide range of formats. These classes provide a broad range of functionalities by exposing a simple unified programming model that is easy and flexible to use. In this article, Thiru Thangarathinam explores the important and frequently required IO related classes by considering examples that will aid our understanding of how easy it is to perform I/O operations in the .NET Framework.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

Often an application may need to import or export data. This information can be anywhere; present in a file, on a disk, somewhere on the network, in variables in memory, or in another program. The System.IO namespace contains a wide assortment of classes, interfaces and structures used to read and write data that can be in a wide range of formats. These classes provide a broad range of functionalities by exposing a simple unified programming model that is easy and flexible to use. In this article, we will explore the important and frequently required IO related classes. We will consider some examples that will show how easy it is to perform I/O operations in the .NET Framework.

Introduction

Using the classes contained in the System.IO namespace, we can perform reading or writing of data from any kind of backing store. In simple words, a backing store can be defined as a storage medium, such as a disk or memory or a socket on the network.

Before we start looking at the actual classes, let us categorize the important ones in the System.IO namespace, based on the kind of data they deal with and the type of operations they perform:

  • Classes that perform file system operations such as creating, copying, moving, deleting, and opening of files. We can further classify these classes into two categories: the first of set classes that fall under this category are: File and Directory. These classes expose static methods meaning that we can invoke the methods of these classes without having to create an instance of them. The second group of classes includes all the classes that derive from FileSystemInfo. Some examples are the FileInfo and DirectoryInfo classes, which are similar to File and Directory classes except that they expose instance methods, which means that we need to create an instance of them before we can start using them.
  • Classes that read and write primitive data types such as bool, int, decimal, single, and string so on. e.g. BinaryReader and BinaryWriter.
  • Classes that deal with character data. TextReader and TextWriter classes encapsulate this functionality.
  • Classes that are designed to handle byte input and output. The root base class that exposes properties and methods to deal with byte input and output is the Stream class.
  • The FileSystemWatcher class that allows us to monitor a file or directory for changes and take appropriate actions based on the kind of changes that take place.

Let us summarize our explanation of the base classes by considering the following diagram, which shows some of the important classes in the System.IO namespace. The diagram also provides an idea of the capabilities of the different classes by showing the inter-dependencies between them. For example, since StreamReader derives from TextReader, it is used to deal with a sequential series of characters (since the TextReader class declares methods for handling a sequential series of characters).

As we already mentioned, the File and Directory classes supply static methods for carrying out such tasks as creating, copying, deleting, and opening of files, and so on. Using the methods and properties of the Path class, we can process the names of files and directories from different platforms in a unified and consistent manner. The above diagram also indicates that the TextReader class acts as the base class for classes such as StreamReader and StringReader. However it does not define any methods for opening or connecting to an underlying data source - those are provided by the derived classes like StreamReader and StringReader. Similar to the TextReader, we also have TextWriter, which is the base class that contains properties and methods to read or write character data.

We can also see from the above diagram that the Stream class is the base class on top of which such classes as BufferedStream, FileStream, and MemoryStream are built. Since the Stream class represents a generic and extensible view of a sequence of bytes, it provides for easy extensibility by allowing the derived classes (like FileStream, BufferedStream, MemoryStream, etc) to represent data in any of the backing stores. The Stream class is mainly used to perform any of the following functions:

  • Reading of data
  • Writing of data
  • To move the current pointer position to the desired location. This process is called Seeking. If you were to create an application that often performs random reading, you would definitely want to consider the Stream class, as it has the built-in support for performing random access of data.

Even though the Stream class is designed to provide the above-listed functionalities, the capabilities of a stream object will depend on the backing store. For example, if we use NetworkStream to read data from a socket, we cannot change the pointer position (that represents the position in a data stream) to the desired location, as we cannot push data back into a socket once it has been read. The following properties help determine the capabilities of a stream object with respect to its backing store.

  • CanRead - Indicates whether the data can be read from a stream. Once we verify that the return value of this method is true, we can then use the Read or ReadByte methods to read either a specified block of bytes or a single byte from the stream.
  • CanWrite - Allows us to determine if data can be written to a stream.
  • CanSeek - Indicates if a stream provides for random access of data. If this returns true, we can then use the Seek method to set the pointer to a position that is relative to any of the following positions: from the start of the stream, end of the stream, or the current position of the stream.

For more information on the classes in the System.IO namespace, please take a look at the following link from MSDN. http://msdn.microsoft.com/library/default.asp?url=/library/en-us/cpref/html/frlrfsystemiohierarchy.asp&WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0

I/O Classes

To understand how to carry out the reading and writing of data using the I/O classes, we will create a new ASP.NET web application in Visual C# using the File -> New Project dialog box. We will name the project IOExamples as shown in the following diagram:

We will kick off our demonstration of the I/O classes by considering the classes that are related to performing file system operations.

File System Operations

As discussed before, the classes that are used to browse around the file system and perform operations like moving, copying, and deleting files, can be broadly classified into two categories.

  • Classes that expose static or shared methods, e.g. File and Directory. If you want to only do operations on a folder or a file then using the File or Directory class is more efficient as it saves us the overhead of instantiating a .NET class.
  • Classes that expose instance methods. e.g. FileInfo and DirectoryInfo. These classes can be used to perform the same set of functions as the File and Directory classes except that we need to create an instance of them to be able to do so. These classes will be very efficient if you are performing multiple operations using the same object, because they will read in the authentication and other information for the appropriate file system object at the time of the construction of the object. Once this information is read and stored as part of the object, it need not be read every time a method call is made.

To demonstrate the implementation of the file system related classes, let us consider the following example:

When the user selects the file using the Browse dialog box, and clicks Get Attributes, we execute the code shown below.

We start by getting the name of the selected file from the form element named fileOpener, and putting it into a local variable named fileName. Then we create an instance of the FileInfo object, passing to it the full path of the selected file. We then invoke its properties and methods to display the attributes of the selected file.

private void
            bntGetAttributes_Click(object sender, System.EventArgs
            e)
{
//Clear the
            listbox
lstAttributes.Items.Clear();
//Get the name of the
            selected file
string fileName = fileOpener.Value;
//Set the
            label to display the selected file
lblMessage.Text = "Selected
            file is :" + fileName;
FileInfo fileInfo = new
            FileInfo(fileName);
lstAttributes.Items.Add("The file attributes
            are : " );
lstAttributes.Items.Add(" Name = " +
            fileInfo.Name);
lstAttributes.Items.Add(" Creation Time = " +
            fileInfo.CreationTime);
lstAttributes.Items.Add(" Last Access
            Time = " +
fileInfo.LastAccessTime.ToString());
lstAttributes.Items.Add("
            Directory Name = " +
            fileInfo.DirectoryName);
lstAttributes.Items.Add(" Length = " +
            fileInfo.Length.ToString());
lstAttributes.Items.Add("...........");

We then get the name of the parent directory of the selected file, and create an instance of the DirectoryInfo object using the directory name as the argument. Once we create an instance of the DirectoryInfo object, we can then display the attributes of the directory.

string dirName =
            fileInfo.DirectoryName;
DirectoryInfo dirInfo = new
            DirectoryInfo(dirName);
lstAttributes.Items.Add("The Directory
            attributes are : " );
lstAttributes.Items.Add(" Name = " +
            dirInfo.Name);
lstAttributes.Items.Add(" Root Name = " +
            dirInfo.Root.Name);
lstAttributes.Items.Add("...........");
lstAttributes.Items.Add("All
            the files contained
in the directory are : " );

We also display all of the child files contained in the directory by looping through the array of FileInfo objects returned by the GetFiles method.

foreach(FileInfo fileInfoChild in
            dirInfo.GetFiles())
{
lstAttributes.Items.Add(fileInfoChild.Name);
}

lstAttributes.Items.Add("...........");
lstAttributes.Items.Add("All
            the subdirectories
contained in the directory are : "
            );

Finally we display all the directories contained in the directory by enumerating through the DirectoryInfo 's array returned by the GetDirectories method.

foreach(DirectoryInfo dirInfoChild in
            dirInfo.GetDirectories())
{
lstAttributes.Items.Add(dirInfoChild.Name);
}
}

This method is used to copy over the selected file from its current directory to the application path.

private void bntCopyFile_Click(object sender, System.EventArgs e)
{
//Clear the listbox
lstAttributes.Items.Clear();
//Get the full path of the selected file
string sourcePath = fileOpener.Value;
//Set the label to display the selected file
lblMessage.Text = "Selected file is :" + sourcePath;
//Get the file name alone from the string
string fileName = Path.GetFileName(sourcePath);
string destPath =
Server.MapPath(Request.ApplicationPath) + "\\" + fileName ;
//Copy the file to the current application path
File.Copy(sourcePath,destPath,true);
lstAttributes.Items.Add(" The file " + fileName +
" has been copied from " );
lstAttributes.Items.Add(sourcePath);
lstAttributes.Items.Add(" to " + destPath);
}

We use this method to move the selected file from its current location to the application path.

private void btnMoveFile_Click(object sender, System.EventArgs e)
{
//Clear the listbox
lstAttributes.Items.Clear();
//Get the full path of the selected file
string sourcePath = fileOpener.Value;
//Set the label to display the selected file
lblMessage.Text = "Selected file is :" + sourcePath;
string fileName = Path.GetFileName(sourcePath);
string destPath = Server.MapPath(Request.ApplicationPath) +
"\\" + fileName ;
//Move the file to the current application path
File.Move(sourcePath,destPath);
lstAttributes.Items.Add(" The file " + fileName +
" has been moved from " );
lstAttributes.Items.Add(sourcePath);
lstAttributes.Items.Add(" to " + destPath);
}

In this method, we get the selected file and delete it from its current location.

private void btnDeleteFile_Click(object sender, System.EventArgs e)
{
//Clear the listbox
lstAttributes.Items.Clear();
//Get the full path of the selected file
string sourcePath = fileOpener.Value;
//Set the label to display the selected file
lblMessage.Text = "Selected file is :" + sourcePath;
string fileName = Path.GetFileName(sourcePath);
//Delete the file
File.Delete(sourcePath);
lstAttributes.Items.Add(" The file " + fileName +
" has been deleted from " );
lstAttributes.Items.Add(sourcePath);
}

Path

Since the Path class contains only static methods, its methods can be accessed without having to create an instance of it. The methods exposed by the Path class are mainly used to manipulate path names in a platform-independent manner. Using the Path class is a whole lot easier than trying to fiddle about with path separation characters manually, especially because the Path class encapsulates the different formats for path names on different operating systems.

The following example illustrates the different important properties and methods of the Path class. The web form we are going to use for the demonstration looks like this:

Once you select the file name using the Browse file open dialog box and click Get Path attributes, we execute the following lines of code:

private void bntGetAttributes_Click(object
            sender, System.EventArgs e)
{
//Clear the
            listbox
lstAttributes.Items.Clear();
string fileName =
            fileOpener.Value;
//Set the label to display the file
            name
lblMessage.Text = "The file you have selected is " +
            fileName;
//Display the properties in the
            listbox
lstAttributes.Items.Add(" File name = : " +
            Path.GetFileName(fileName));
lstAttributes.Items.Add(" Directory
            Name = " +
Path.GetDirectoryName(fileName));
lstAttributes.Items.Add("
            Extension = " +
            Path.GetExtension(fileName));
lstAttributes.Items.Add(" File name
            without extension = " +
Path.GetFileNameWithoutExtension(fileName));
lstAttributes.Items.Add("
            Root Path = " +
Path.GetPathRoot(fileName));lstAttributes.Items.Add(" Has
            Extension = " +
Path.HasExtension(fileName).ToString());
lstAttributes.Items.Add("
            Path Separator = " +
            Path.PathSeparator);
lstAttributes.Items.Add("Directory Separator
            = " +
Path.DirectorySeparatorChar);
lstAttributes.Items.Add("Alt
            Directory Separator = " +
Path.AltDirectorySeparatorChar);
lstAttributes.Items.Add("Temporary
            folder name = " + Path.GetTempPath());
}

In the code presented above, we get the name of the file that the user has selected by using the following line:

string fileName = fileOpener.Value;

Once we get the file name, we pass this as an argument to the static members of the Path class and show the results in the list box.

BinaryReader and BinaryWriter

When we work with streams of binary data, we often need to read and write primitive types. The System.IO namespace provides BinaryReader and BinaryWriter classes to accomplish this. Using the BinaryWriter class, we can not only write data that belongs to any of the primitive types, but also specify the type of encoding to be used for writing.

Similar to the BinaryWriter class, the BinaryReader exposes properties and methods to read primitive data types as binary values. Some of the important methods that provide this functionality include: ReadBoolean, ReadByte, ReadChar, ReadDecimal, ReadDouble, ReadInt16, ReadInt32, ReadInt64, ReadString, ReadSingle, and so on. From the names of the methods, we can easily figure out the kind of data each method is designed to handle.

Let us consider the following example for the illustration of writing of data using the BinaryWriter class. In this example, we write primitive types such as a string, an int, a single, and a decimal to the file.

private void Page_Load(object sender,
            System.EventArgs e)
{
// Put user code to initialize the page
            here
string fileName =
            Server.MapPath("binaryfile.txt");
//Create the FileStream object
            with the proper settings
FileStream stream = new
FileStream(fileName,FileMode.Append,FileAccess.Write);
//Associate
            the filestream with the BinaryReader
BinaryWriter writer = new
            BinaryWriter(stream);
writer.Write("This is a string
            value");
writer.Write(1233);
writer.Write(1.8f);
writer.Write(90.1m);
stream.Flush();
stream.Close();
}

To read the file created by the above example, we will use the BinaryReader class. Here we invoke the different forms of Read method such as ReadString, ReadInt16, ReadSingle, and ReadDecimal to read primitive types from the file. We then display the read contents in the list box named lstContents:

private void bntReadData_Click(object
            sender, System.EventArgs e)
{
string fileName =
            Server.MapPath("binaryfile.txt");
//Create the FileStream object
            with the proper settings
FileStream stream = new
FileStream(fileName,FileMode.Open,FileAccess.Read);
//Associate
            the filestream with the BinaryReader
BinaryReader reader = new
            BinaryReader(stream);
//Read the primitive types from the
            file
lstContents.Items.Add(reader.ReadString());
lstContents.Items.Add(reader.ReadInt16().ToString());
lstContents.Items.Add(reader.ReadSingle().ToString());
lstContents.Items.Add(reader.ReadDecimal().ToString());
stream.Flush();
stream.Close();
}

StreamReader and StreamWriter

As mentioned before, these classes are used to read and write data that is in character format. They are mainly useful in cases when we are trying to read from or write data to a text file. When constructing a StreamReader, we need to provide values for the following options.

  • Encoding method: in this parameter, we need to specify what to do about the different encoding methods. There are two ways by which we can do this. We can either let the StreamReader examine the byte order marks in the file to determine the encoding method, or we can simply tell the StreamReader to assume that the file uses a specified encoding method.
  • Reference to another stream: instead of supplying a file name to be read from, we can supply a reference to another stream. This option is very important as it demonstrates the advantage of basing our model for reading and writing data around the concept of streams. This is useful in situations where you have another stream that contains the actual data, and you want to process the data contained in that stream in your application. We can easily carry this out by passing the output of that stream to a StreamReader constructor and start using it.

For example, the following line of code shows how to construct a simple StreamReader object. We pass in the file name to be read and the encoding to be assumed as arguments, to the constructor of the StreamReader object.

StreamReader reader = new StreamReader(@"c:\test.txt",Encoding.UTF8);

In the later stages of this article, when we look at the FileStream class, we will see how we can hook a StreamReader object up to a FileStream object to read data from a text file.

The following list shows some of the important methods of the StreamReader class:

  • Read - Allows us to read the next character or a set of next characters from the input stream.
  • ReadBlock - Allows us to read a block of characters into the buffer.
  • ReadToEnd - As the name suggests, this provides the ability to read character data from the current stream until the end.
  • ReadLine - Permits us to read a single line of characters from the current stream.
  • Peek - It is similar to the Read method except for the difference that it does not advance the current character position.

The StreamWriter class works basically the same as the StreamReader class, except that it is used to write to a text file. The StreamWriter class exposes the following methods to accomplish this: Write and WriteLine.

The following simple example shows how to read and write to a text file using the StreamReader and StreamWriter classes.

First off, we create a StreamWriter object and associate it with a file named StreamWriterFile.txt. We then write few lines of text using the WriteLine method. Finally we flush the contents and close the StreamWriter object.

private void Page_Load(object sender,
            System.EventArgs e)
{
//Create the StreamWriter object with
            the proper settings
StreamWriter writer = new
StreamWriter(Server.MapPath("StreamWriterFile.txt"),true);
writer.WriteLine("Programming
            Languages
            :");
writer.WriteLine("ASP.NET");
writer.WriteLine("Visual
            C#");
writer.WriteLine("Visual
            Basic.NET");
writer.WriteLine("JScript.NET");
writer.WriteLine("ADO.NET");
writer.Flush();
//Close
            the StreamWriter
writer.Close();

In the following block of code, we read the lines of text written by the above code and write them to the client browser:

//Create the StreamReader object
StreamReader reader = new
StreamReader(Server.MapPath("StreamWriterFile.txt"));
Response.Write(reader.ReadLine()
            + "<br>");
Response.Write(reader.ReadLine()+
            "<br>");
Response.Write(reader.ReadLine() +
            "<br>");
Response.Write(reader.ReadLine()+
            "<br>");
Response.Write(reader.ReadLine() +
            "<br>");
Response.Write(reader.ReadLine()+
            "<br>");
//Close the
            reader
reader.Close();
}

Executing the above code results in the following output:

MemoryStream

As we already saw in the diagram, this class derives from the Stream base class and it is primarily used to read and write bytes of data from memory. The amount of memory buffer allocated for storing the data is determined by the value we pass to one of the many overloaded constructors of this class. The default size is 256 bytes.

The Capacity property allows us to determine the amount of data the memory stream can hold, after which it will have to do reallocation of its buffer. Since it is a read/write property, we can also set this property to shrink the size of the memory stream to the desired size. To determine the amount of data that is currently held in the memory stream, we can use the read-only Length property.

Let us consider a simple example to see the MemoryStream class in action:

private void Page_Load(object sender,
            System.EventArgs e)
{
// Put user code to initialize the page
            here
MemoryStream memStream = new MemoryStream();
StreamWriter
            writer = new StreamWriter(memStream);
writer.WriteLine("This line
            is written to the memory stream");
writer.Flush();
Byte[]
            arrayByte;
//convert the stream into a byte array
arrayByte =
            memStream.ToArray();
Response.Write ("Capacity = " +
            memStream.Capacity.ToString() + "<br>");
Response.Write
            ("Lengh = " + memStream.Length.ToString() +
            "<br>");
Response.Write ("CanRead = " +
            memStream.CanRead.ToString()+ "<br>");
Response.Write
            ("CanSeek = " + memStream.CanSeek.ToString()+
            "<br>");
Response.Write ("CanWrite = " +
            memStream.CanWrite.ToString()+
            "<br>");
writer.Close();
}

In the above code, once we create an instance of the MemoryStream class, we associate that with the StreamWriter class by passing it to the constructor of the StreamWriter class. Then we write a line of text using the WriteLine method of the StreamWriter class.

To convert the memory stream into a byte array, we use the ToArray method. Once we have the data in the form of byte array, it can then be easily sent across the wire to another application which might want to process the data.

arrayByte = memStream.ToArray();

In the following statements, we determine the capabilities of the MemoryStream class by inspecting its properties:

Response.Write ("CanRead = " + memStream.CanRead.ToString()+ "<br>");
Response.Write ("CanSeek = " + memStream.CanSeek.ToString()+ "<br>");
Response.Write ("CanWrite = " + memStream.CanWrite.ToString()+ "<br>");

If you execute the above code, the generated output looks like the following:

StringReader and StringWriter

The StringReader class derives from the TextReader class, and it uses a string as the input stream. At the time of instantiating the StringReader class, we can pass the string to be read as an argument to its constructor. Like StringReader, the StringWriter class derives from the TextWriter class. The StringWriter class uses a StringBuilder object internally to build and manage the output stream.

The following snippet of code shows how to use the StringWriter class to manipulate the string output stream. Once we are done writing the contents into the underlying string, we can then retrieve its contents using any one of the following ways:

  • By calling the ToString method
  • By invoking the GetStringBuilder method - this method returns the actual StringBuilder object that is used to construct the underlying stream.

private void Page_Load(object sender, System.EventArgs e)
{
// Put user code to initialize the page here
StringWriter writer = new StringWriter();
writer.WriteLine("This is the first line");
writer.WriteLine("This is the second line");
writer.WriteLine("This is the third line");
//Get the Contents by invoking through the ToString method
//txtContents.Text = writer.ToString();
//Can also get the Contents through the underlying StringBuilder object
txtContents.Text = writer.GetStringBuilder().ToString();
writer.Flush();
writer.Close();
}

The output produced by the above code looks like the following:

FileStream

The FileStream class is intended for reading and writing binary data to any binary file. However if you wish, you could also use this to read and write to any file. One of the cool features of the FileStream class is its ability to support random access of files. This makes the FileStream class the most preferred class when it comes to reading data from a file. This is especially true when designing an application that often needs to perform random accessing of files. The random accessing of files is made possible through the use of the Seek method. The FileStream class supports both synchronous and asynchronous read and write operations, with the default being the synchronous mode.

In order to construct a FileStream object, we need to provide any of the following four pieces of information:

  • Path - This is used to specify the path of the file we are trying to access.
  • FileMode - This parameter allows us to specify the mode in which we want to open the file. For example, if we want to open a file for appending, we can pass the enum constant FileMode.Append. The other valid values for this argument include: FileMode.Create, FileMode.CreateNew, FileMode.Open, FileMode.OpenOrCreate, and FileMode.Truncate.
  • FileAccess - Allows us to detail the kind of access we require on the file. Valid values for this argument include: FileAccess.Read, FileAccess.Write, and FileAccess.ReadWrite.
  • FileShare - Allows us to control the kind of access others will have when you are working on the file. It can have any one of the following values: FileShare.None, FileShare.Read, FileShare.ReadWrite, and FileShare.Write.

For detailed information on all the properties and methods of the FileStream class, please take a look at the following link: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/cpref/html/frlrfSystemIODirectoryInfoClassTopic.asp&WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0.

To help realize the implementation and usage of the FileStream class, we will implement an error handling solution for our application by using the FileStream class. Whenever any exceptions occur in the system, we will capture them in the Application_Error event and process them by logging them into a text file. We will use the FileStream class to perform the logging of exception information into a text file.

The following lines of code are required to implement this:

protected void Application_Error(Object
            sender, EventArgs e)
{
string fileName =
            Server.MapPath(@"\IOExamples\ErrorLog\")+
System.Guid.NewGuid().ToString() + ".txt";
FileStream
            fileStream = new
            FileStream(fileName,FileMode.Append,
FileAccess.Write,FileShare.ReadWrite);
StreamWriter
            writer = new
            StreamWriter(fileStream);
writer.Write(Server.GetLastError().ToString());
writer.Flush();
writer.Close();
}

We start our implemenation by constructing the full path of the file:

string fileName = Server.MapPath(@"\IOExamples\ErrorLog\")+
System.Guid.NewGuid().ToString() + ".txt";

In the above line, we use the NewGuid method of the System.Guid class to create a new unique guid. This means that every time an exception occurs, a new file will be created in the ErrorLog directory. We will discuss the need for this when we discuss the FileSystemWatcher in the latter part of this article.

We then create an instance of the FileStream class and specify that we want to open the file for appending.

FileStream fileStream = new FileStream(fileName,FileMode.Append,
FileAccess.Write,FileShare.ReadWrite);

Once we create an instance of the FileStream class, we can associate it with a StreamWriter by passing it to the constructor:

StreamWriter writer = new StreamWriter(fileStream);

Here, we write the exception information using the Write method of the StreamWriter object.

writer.Write(Server.GetLastError().ToString());

Finally, we flush and close the writer:

writer.Flush();
writer.Close();

FileSystemWatcher

One of the new exciting features of the .NET framework is the ability to create applications that can interact with the file system and raise events at scheduled intervals, based on the activities in the File System. The FileSystemWatcher component can be considered as a good alternative to asynchronous mechanisms like MSMQ for creating flexible solutions. If you have ever done any work in MSMQ, you might have created an application that receives the events from the message queue whenever a new message arrives in the queue. Creating and maintaining an application of this kind, sometimes, becomes a daunting task due to the complexities involved. However in .NET, we can use the FileSystemWatcher component in conjunction with the Windows service applications to accomplish the same set of functionalities in a flexible and easier manner.

The FileSystemWatcher component that is contained in the System.IO namespace exposes methods and properties through which we can watch for changes to directories and files, and take specific actions when these changes occur. Using the FileSystemWatcher component, we can watch for files on a local machine, a network drive or a remote machine.

We can create the FileSystemWatcher component using any one of the following ways:

  • We can drag an instance of the FileSystemWatcher component from the Components tab of the Toolbox and drop it in the designer.
  • We can create an instance of the FileSystemWatcher component programmatically after importing the System.IO namespace.

Before we go on to look at an example of FileSystemWatcher in action, let us consider the important properties of it:

  • EnableRaisingEvents: Allows us to determine whether the component is enabled for monitoring.
  • Filter: Allows us to get or set the filter string that determines the files to be monitored.
  • Path: Allows us to specify the path of the directory that is to be watched for changes.
  • IncludeSubDirectories: Allows us to indicate whether the subdirectories within the specified path should be monitored.
  • NotifyFilter: Accepts an enum constant as an argument that indicates the different types of changes to watch for.
  • InternalBufferSize: Allows us to retrieve or set the value of the size of the internal buffer that is used to store the file system changes momentarily before they are passed to the applications that are interested in receiving notifications. The default size of the buffer is 8K.

As we already discussed, the NotifyFilter enumeration allows us to specify the changes that we want to monitor for in a file or folder. It can have any one of the following values, which determine the changes to watch for: Attributes, CreationTime, DirectoryTime, FileName, LastAccess, LastWrite, Security, Size.

Now that we have understood the important properties whose values need to be set for us to be able to start receiving notifications, let us go ahead and understand the different notifications (or events) the FileSystemWatcher can raise in response to the above properties:

  • Changed: It is raised when changes occur in the file or directory that is being monitored. These changes include changes in the size, system attributes, last write time, last access time, or security permission changes of the file or directory.
  • Created: Raised when a file or directory is created in the path set by the Path property.
  • Deleted: Raised when a file or directory is deleted in the path set by the Path property.
  • Renamed: Provides us with the notification when a file or directory is renamed in the path indicated by the Path property.
  • Error: Allows us to determine the exception condition that is caused by the internal buffer overflow.

The following link from MSDN gives detailed description of the methods, properties and events exposed by the FileSystemWatcher class - http://msdn.microsoft.com/library/default.asp?url=/library/en-us/cpref/html/frlrfsystemiofilesystemwatcherclasstopic.asp&WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0.

To understand the FileSystemWatcher component, we will extend our error handling solution by adding monitoring capabilities to our application. In our explanation of the FileStream class, we added code in the Application_Error event to handle any exceptions that occur in our application. We processed the exceptions by logging them into a text file in the folder ErrorLog. Now let us say, for example, we also need to send an Email to the administrator whenever exceptions occur. For this scenario, let us see how the FileSystemWatcher component can be used to help accomplish the desired functionality.

To understand the flow of the system, let us consider the following diagram:

In the above steps, the following sequence of operations are executed:

  • In the Application_Start event, set up the FileSystemWatcher component to monitor the ErrorLog directory for creation of new files. While setting up the FileSystemWatcher component, we also specify the method to be called (or notified) whenever a new file is created in the ErrorLog directory. In our application, we will use a method called OnCreated for this purpose.
  • In the Application_Error event, as explained already in the explanation of the FileStream class, we have code to process the exceptions by creating a new file in the directory ErrorLog.

Now that we have set up the above-mentioned procedures, let us see what happens at runtime. Whenever an exception occurs, the control is transferred to the Application_Error event, where the exception is logged in to a newly created file in the ErrorLog directory.

For example, if we deliberately generate a System.DivideByZeroException by dividing a number by zero in the code, our application handles the exception by generating a log. As soon as a new file is created in the ErrorLog directory, the FileSystemWatcher picks it up and raises an event (or notification) to our application by invoking the OnCreated method (which we have already specified at the time of setting up the file monitoring in the Application_Start event). In the OnCreated method, we read the contents of the file and send an Email to the administrator informing him of the error.

Let us look at the code required to implement this.

The meat of this example is the Application_Start event as that is where we set up the file monitoring on the ErrorLog directory:

protected void Application_Start(Object sender, EventArgs e)
{
FileSystemWatcher watcher = new FileSystemWatcher();
//Set the path to specify the directory to monitored
watcher.Path = Server.MapPath(@"\IOExamples\ErrorLog\");
//Monitor for files that end with extension .txt
watcher.Filter = "*.txt";
//Indicate not to monitor the subdirectories
watcher.IncludeSubdirectories = false;
//Enable the component to begin watching for changes
watcher.EnableRaisingEvents = true;
watcher.Created += new FileSystemEventHandler(OnCreated);
}

We start by creating an instance of the FileSystemWatcher component.

FileSystemWatcher watcher = new FileSystemWatcher();

Once we create the FileSystemWatcher component, we then set the Path property to the physical path returned by the Server.MapPath method:

//Set the path to specify the directory to be monitored
watcher.Path = Server.MapPath(@"\IOExamples\ErrorLog\");

Then we specify that we want to monitor only for text files by using the following line of code:

//Monitor for files that end with extension .txt
watcher.Filter = "*.txt";

We also specify that we do not want to monitor the subdirectories by setting the IncludeSubDirectories to false:

//Indicate not to monitor the subdirectories
watcher.IncludeSubdirectories = false;

Here we start the monitoring process by setting the EnableRaisingEvents to true:

//Enable the component to begin watching for changes
watcher.EnableRaisingEvents = true;

We then hook the Created event up to the OnCreated method, which gets called whenever a new file is created:

watcher.Created += new FileSystemEventHandler(OnCreated);

As mentioned already, the OnCreated method is called whenever a new file is created. In this method, we read the contents of the newly created file and send an Email to the administrator informing him that the exception that has occurred:

protected void OnCreated(object
            source,FileSystemEventArgs e)
{
String line;
FileStream
            fileStream = new
            FileStream(e.FullPath,FileMode.Open,
FileAccess.Read);
StreamReader
            reader = new StreamReader(fileStream);
StringBuilder builder =
            new StringBuilder();
//Read the contents of the file and append
            it to the StringBuilder
while(reader.Peek() != -1)
{
line =
            reader.ReadLine();
builder.Append(line);
}

//Set the
            Message properties
MailMessage message = new
            MailMessage();
message.From =
            "IOExamples@admin.com";
message.To =
            "webadmin@admin.com";
message.Subject = "An error has occured in
            your site";
message.Body = builder.ToString();
//Send EMail
SmtpMail.Send(message);
}

We start by creating an instance of the FileSteam object, passing in appropriate arguments. The FullPath property of the FileSystemEventArgs object returns the full path of the newly created file. To the constructor of the FileStream class, we pass this as an argument to the Path parameter. We also specify that we want to open the file for reading by using the enum constants FileMode.Open and FileAccess.Read:

FileStream fileStream = new FileStream(e.FullPath,
FileMode.Open,FileAccess.Read);

We then loop through the contents of the file by using the combination of the Peek and ReadLine methods and append it to the string builder object:

   while(reader.Peek() !=
            -1)
{
line =
            reader.ReadLine();
builder.Append(line);
}

In the following lines of code, we set the properties of the MailMessage object to the required values:

//Set the Message
            properties
MailMessage message = new
            MailMessage();
message.From =
            "IOExamples@admin.com";
message.To =
            "webadmin@admin.com";
message.Subject = "An error has occured in
            your site";
message.Body = builder.ToString();

Finally, we send the mail using the static method Send of the SmtpMail class.

//Send EMail
SmtpMail.Send(message);

Conclusion

In this article, we have seen the following:

  • All the important classes contained in the System.IO namespace, and how they are related to each other in terms of their functionality.
  • How to use the File, Directory, FileInfo, and DirectoryInfo classes to perform file system operations.
  • How to use the Path class to process path names in a platform-independent manner.
  • How to read and write to different kinds of backing stores using classes like FileStream and MemoryStream.
  • How to take advantage of the capabilities of the FileSystemWatcher component to create asynchronous applications.

We have also seen how easy it is to create flexible and powerful applications that perform IO operations using the IO related classes supplied by the .NET Framework class library.


A POP3 Client and C#

A POP3 Client and C#
http://www.csharptoday.com/content.asp?id=1707
Article Rating: 4
Published on November 28, 2001
Abstract
In this article, Levent Camlibel looks at how to create a POP3 client that recieves e-mail from a POP3 server, showing how the .NET platform allows this project to be created in a very short space of time. Over the course of the article, he looks at POP3 standards, .NET network programming, and custom POP3 objects.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

POP3 is the most recent version of an industry wide standard (RFC 1939 can be found at ftp://ftp.isi.edu/in-notes/rfc1939.txt?WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0) for email clients to connect to servers. The POP3 specification states that POP3 is not intended to provide extensive manipulation operations for mail on the server; normally, mail is simply downloaded and then deleted. A more advanced (and complex) protocol, IMAP4, is discussed at RFC1730 (RFC1730 can be found at ftp://ftp.isi.edu/in-notes/rfc1730.txt?WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0). However, it is estimated that over 90% of all email clients use POP technology for non-web enabled email. I always wondered why Microsoft released a SMTP client, but have never released a POP3 client. Both new and old versions of ASP, ASP.NET, VS, and VS.NET have SMTP objects to send emails from web and windows applications. I decided to work on a POP3 project that receives emails from POP3 servers. The .NET platform allowed me to complete this project in a very short time. In this article, we will look at POP3 standards, .NET network programming, and custom POP3 objects that make it possible to receive emails.

Defining Problem

This article starts with an introduction to POP3. In this section we will cover the basics of POP3, look at its required and optional commands, and also discuss the difference between POP3 and IMAP4. After that, we will look at the .NET remoting section. We will not cover everything in .NET networking, but rather focus on the remoting objects used in this article. Next, we will focus on the components for the POP3 client application, which are Pop3MailHeader, Pop3MailBody, Pop3MailMessage, and Pop3Client objects. After introducing these components, we will see the component coding in detail. Finally, we will look at the installation and running of the sample application.

Introduction to POP3

Well, let's start with the basics first. POP3 stands for Post Office Protocol version 3. POP3 is the most recent version of an industry wide standard for email clients to connect to servers. In other words, it is a TCP/IP application protocol (RFC 1939) that allows a user to receive mails. A POP3 user is provided with an account on a POP3 server that holds the user's mailbox. A client on the user's machine that understands the POP3 protocol can converse with the POP3 server, and receive mails for the user. It is estimated that over 90% of all email clients use POP technology for non-web enabled email. You only need a few things to connect to a POP3 email server. These include a valid username and password, and the address of the POP server. To start with, you need to make sure that you connect to the correct port. Usually, the standard port for POP communications is port 110. In some cases, your Local Area Network or your ISP might use a different port, then you have to make sure that you set it correctly when connecting.

POP3 is the older of the two methods (IMAP4 vs. POP3) for email servers and clients to deal with email messages. When your POP3 client contacts your POP3 server, the first order of business is to download all new messages from the server to your local hard drive. Copies of these messages may or may not be retained on the server, depending on your client configuration. When you read and delete your messages in a POP3 email client, you are actually working with message files on your hard drive.

Using POP3 has some implications:

  • POP3 has some privacy issues. POP3 leaves a copy of your messages on your computer, and so anyone who has access to your computer has the opportunity to read the messages on your computer.
  • If you read your email on more than one computer, POP3 becomes more complicated to live with. If you try to read it from another computer, you are out of luck, which is why (or one reason why, at least) Internet based email, such as Microsoft's hotmail, has become so popular.

To minimize these problems, most POP3 email clients offer a number of different synchronization configurations that involve leaving copies of your mail on the server after the messages are downloaded to your computer.

IMAP4 is a newer protocol for handling email. While your IMAP4 client handles the interface, the messages remain on the server while you work with them. No messages are copied to your hard drive, unless you explicitly copy them. This means that when you close your IMAP4 email client, there is nothing on the hard drive of the local computer for anyone else to see. Unless they have your Computing ID and password, they cannot read your messages. There is no synchronization between the email server and the local email client, so you do not see the POP3 synchronization effects mentioned above. Moreover, no matter what computer you use to access your email, you will see the same list of messages, because the list comes from the email server. POP3 has some advantages for some users in certain cases. If you always read mail from one computer, no one else has access to that computer, and you have reasons to limit your connection time to the network, POP3 can transfer all your emails to your hard drive so that you can disconnect and work offline with your email. IMAP4 mailers also allow you to work offline, but require that you manually choose which messages to download to work on later. POP3 and IMAP4 deal with the receiving of e-mail, and are not to be confused with the Simple Mail Transfer Protocol (SMTP), a protocol for transferring e-mail across the Internet. You send email with SMTP and a mail handler receives it on your recipient's behalf. Then the email is read using POP3 or IMAP4 protocols.

POP3 Commands

The server host initially starts the POP3 service by listening on TCP port 110, which is the default port. When a client host wants to make use of the service, it establishes a TCP connection with the server host. When the connection is established, the POP3 server sends a greeting. The client and POP3 server then exchange commands and responses (respectively) until the connection is closed or aborted.

Commands in the POP3 consist of a case-insensitive keyword, possibly followed by one or more arguments. All commands are terminated by a CRLF pair. Keywords and arguments are each separated by a single SPACE character. Keywords are three or four characters long. Each argument may be up to 40 characters long. Responses may be up to 512 characters long, including the terminating CRLF. There are currently two status indicators: positive ("+OK") and negative ("-ERR"). According to POP3 specification, servers must send the "+OK" and "-ERR" in upper case. Keywords and arguments consist of printable ASCII characters.

Required Commands

  • USER name: This is the username sent to the POP3 server.
  • PASS string: This is the password.
  • QUIT: This command terminates/ends the current session.
  • DELE msg: This command deletes a mail message from the server.
  • RSET: This undoes any changes made within the current session. If any messages have been marked as deleted by the POP3 server, they are unmarked. The POP3 server then replies with a positive response.
  • STAT: This returns the number of messages on the server.
  • RETR msg: This retrieves the content of a message.
  • LIST [msg] If an argument was given, the POP3 server issues a positive response with a line containing information for that message.
  • NOOP: The POP3 server does nothing, it merely replies with a positive response.

Optional Commands

  • TOP msg n: the POP3 server sends the headers of the message, the blank line separating the headers from the body, and then the number of lines of the indicated message's body.
  • UIDL [msg]: If an argument was given, the POP3 server issues a positive response with a line containing information for that message. This line is called a "unique-id listing" for that message.
  • APOP name digest: A string identifying a mailbox and a MD5 digest string (both required)

The MD5 algorithm takes a message of arbitrary length as its input, and produces a 128-bit "fingerprint", or "message digest", of the input. It is conjectured that it is computationally infeasible to produce two messages with the same message digest. The MD5 algorithm is intended for digital signature applications, where a large file must be "compressed" in a secure manner before being encrypted with a private (secret) key under a public-key cryptosystem such as RSA.

POP3 Session States

POP3 sessions consist of three states:

  • Authorization state
  • Transaction state
  • Update state

We may consider the Authorization state as the greeting part, the Transaction state as the body part, and the Update state as the closing part.

The Authorization state

Once the TCP connection has been opened by a POP3 client, the POP3 server issues a one-line greeting. The POP3 session is now in the Authorization state. The client must now identify and authenticate itself to the POP3 server. The USER and PASS command combination and the APOP command should be implemented in this state. You send the command "USER" and your username by using the following code. The WriteToNetStream, checkForError, and ReadFromNsetStream functions are created solely for this particular article.

WriteToNetStream (ref netStream, "USER " + this.userName);

The Response of POP3 server will be in returnMsg string:

returnMsg=ReadFromNetStream(ref netStream);

The example response might be:

"+OK Password required for Leventc." (Leventc is my username.)

checkForError (returnMsg);

If the username is not in the POP3 server, the response will be "-ERR". The checkForError() function checks the response, and if it cannot find the "+OK" response, then it throws an Pop3Exception, which is created solely for the article. After this step, we send the password for the "Leventc" username. The POP3 command for sending password is "PASS ******", which is the reserved word PASS, plus the password itself.

WriteToNetStream(ref netStream, "PASS " + this.password);

If the password is correct, the example response might be:

"+OK Leventc has 1 message(s) (1374 octects)."

If the password is not correct, the response will be "-ERR", so then the application throws a Pop3Exception.

returnMsg=ReadFromNetStream(ref
            netStream);
checkForError(returnMsg);

The Transaction state

Once the client has successfully identified itself to the POP3 server, the POP3 session is now in the Transaction state. The client may now issue any of the fo llowing POP3 commands repeatedly: STAT, LIST, RETR, DELE, NOOP, TOP, UIDL, and RSET. After each command, the POP3 server issues a response. Let me give you just one example for the transaction state. The following is taken from the STAT function in the Pop3Client class. First, we have to check if the session is in the transaction stage. If it is not, it throws a new Pop3Exception, otherwise we send the STAT command to receive statistics about emails. The sample response might be

"+OK 1 1374". The response can be divided into two parts based on the space between them. The first value, "1", indicates that there is only one message waiting on the POP3 server, and the second value, "1374", indicates the size of the message.

if(this.pop3Status!="TRANSACTION")
{
throw
            new Pop3Exception("Pop3 state is not in TRANSACTION
            state");
}

WriteToNetStream(ref netStream,
            "STAT");

returnMsg=ReadFromNetStream(ref
            netStream);
checkForError(returnMsg);

///split the
            information to total number of messages and total size
string[]
            TotalStat= returnMsg.Split(new char[] {'
            '});

this.count=Int32.Parse(TotalStat[1]);
this.totalSize=Int32.Parse(TotalStat[2]);

The Update State

When the client issues the QUIT command from the Transaction state, the POP3 session enters the Update state. When the user calls the Disconnect() function, the POP3 session enters the Update state. To disconnect from POP3 server, the QUIT command is used. The sample response of POP3 server might be "+OK pop server at mail singing off." After that, we have to close the NetworkStream and the TcpClient object in this function.

public void
            Disconnect()
{
this.pop3Status="UPDATE";

WriteToNetStream(ref
            netStream, "QUIT");

returnMsg=ReadFromNetStream(ref
            netStream);
checkForError(returnMsg);

///if there is not
            problem, close the
            socket
try
{
netStream.Close();
tcpClient.Close();
}
catch
{
throw
            new Pop3Exception("There is a problem with socket, Failure in
            Disconnect");
}
}

Introduction to .NET Networking

The TCP/IP protocol suite components are the bricks and mortar of all networked applications, and is the most widely used protocol among the private and public networks. The .Net classes provide a simple yet robust solution for writing networked applications in managed code. The .Net TCP/IP classes as a base or component (even by inheritance, aggregation or composition), provide the necessary infrastructure for the applications to use, since they are network-centric.

Microsoft suggests that when writing socket-level protocols, we should use TcpClient or UdpClient whenever possible, instead of writing directly to a socket. These two client classes encapsulate the creation of TCP and UDP sockets, without requiring you to handle the details of the connection.

Let us now look at the Socket class. It creates a managed version of an Internet transport service. Once the socket is created, it is bound to a specific end point with the Bind method. The connection to the end point is established with the Connect method, and then data is sent to the socket using the Send or SendTo methods, and data is read from the socket using the Receive and ReceiveFrom methods. After use, the Shutdown method is used to disable the socket, and the Close method closes the socket. The Socket class is used by the .NET framework to provide Internet connections to the TCPClient, UDPClient, and WebRequest and descendent classes.

TcpClient Object

The TcpClient class, which is in the System.Net.Sockets namespace, builds upon the Socket class to provide TCP services at a higher level of abstraction. TcpClient is used to create a client connection to a remote host. Application-level protocols such as FTP and HTTP are built on the TcpClient class. The following code illustrates that the connection to the POP3 server has to be in the stage of Authorization. If it is not, it throws a Pop3Exception. If everything goes well, it instantiates a TcpClient object, and tries to connect to the POP3 server by using the Connect function. The Connect function in this case requires a Host Address or IP number and port number. We get the values of this.pop3Port from the email client application. We should consider the fact that there might be some problem with the email server or name or port number. Therefore, it is good practice to put the connect function in to try-catch block.

if(this.pop3Status!="AUTHORIZATION")
{
throw
            new Pop3Exception("Pop3 State is not in
            AUTHORIZATION");
}
tcpClient = new
            TcpClient();
this.userName=userName;
this.password=password;
try
{
tcpClient.Connect(this.pop3Server,this.pop3Port);
}
catch
{
throw
            new Pop3Exception("Cannot connect to target host " +this.pop3Server
            +" and port "+
this.pop3Port.ToString());
}

///get the
            response of pop3 mail server
netStream =
            tcpClient.GetStream();

The TcpClient.GetStream() function returns a NetworkStream object that is used to read and write data to a remote host.

NetworkStream Object

The NetworkStream implements the standard .NET Framework stream mechanism to send and receive data through network sockets. NetworkStream supports both synchronous and asynchronous access to the network data stream. It is in the System.Net.Sockets namespace. I have used two functions, WriteToNetStream and ReadFromNetStream, to read and write to a remote host. WriteToNetStream has two arguments, NetStream of type NetworkStream, and Command of type string. NetStream is a reference type variable, and Command is a value type variable. The Command argument receives POP3 commands such as RETR, LIST, DELE, TOP, and so on. I append commands with "\r\n" because the POP3 specification indicates that all commands are terminated by a CRLF pair. After that, the string has to be converted to byte type. To do this conversion, we can use Encoding.ASCII (in the System.Text namespace) class and the GetBytes function, which returns bytes out of characters. After that, we are now ready to write the command to the network stream.

private void WriteToNetStream(ref NetworkStream NetStream, string Command)
{
string stringToSend = Command + "\r\n";
Byte[] arrayToSend = Encoding.ASCII.GetBytes(stringToSend.ToCharArray());
///write to network stream
NetStream.Write(arrayToSend, 0, arrayToSend.Length);
}

So far, we have seen how to write data to NetworkStream. It is time to look at the code that reads responses from the POP3 server from NetworkStream. The argument (NetStream) of ReadFromNetStream is again a reference type variable. I have used the StreamReader object to read from NetworkStream. Since there might be unusual responses from the POP3 server, we will check the response to check whether it is null, or if its length is zero. If either is true, we will read the NetworkStream again until we get the proper response. The response might be multi-line or single line. "if (sr.Peek()!=-1)" helps to capture the response. If the response is multi-line, it continues reading from NetworkStream.

private String ReadFromNetStream(ref
            NetworkStream NetStream)
{
StringBuilder strReceived= new
            StringBuilder();
StreamReader sr= new
            StreamReader(NetStream);
String strLine =
            sr.ReadLine();

while(strLine==null ||
            strLine.Length==0)
{
strLine =
            sr.ReadLine();
}

strReceived.Append(strLine);

if(sr.Peek()!=-1)
{
while
            ((strLine=sr.ReadLine())!=null)
{
strReceived.Append(strLine);
}
}
return
            strReceived.ToString();
}

StreamReader Object

The StreamReader object implements a TextReader that reads characters from a byte stream in a particular encoding. StreamReader is designed for character input in a particular encoding, whereas the Stream class is designed for byte input and output. We can use StreamReader for reading lines of information from a standard text file. StreamReader defaults to UTF-8 encoding unless specified otherwise, instead of defaulting to the ANSI code page for the current system. UTF-8 handles Unicode characters correctly and provides consistent results on localized versions of the operating system. It is in the System.IO namespace.

The following code is taken from the Retrieve function in the Pop3Client class. First we have to use the POP3 WRITE command (RETR [msg]) to NetworkStream. After writing to NetworkStream, the POP3 server will respond to the command. StreamReader, therefore, gets the response.

WriteToNetStream(ref netStream, "RETR " +
            MessageNumber);
StreamReader sr = new
            StreamReader(netStream);

What does the first while loop do? It reads from NetworkStream until it gets the proper response from the POP3 server. The second while loop provides the POP3 specification, stating that after the initial +OK, the POP3 server sends the headers of the message, and the blank line separating the headers from the body. Therefore, if(inputLine.Trim().Length ==0) captures the first blank line. If there is a blank line in the header of the email, this would create a bug! However, we can assume that the POP3 servers will not make this kind of mistake, so there should be no problems.

inputLine=sr.ReadLine();

while(inputLine==null
            ||
            inputLine.Length==0)
{
inputLine=sr.ReadLine();
}
headerList.Add(inputLine);
while
            ((inputLine=sr.ReadLine())!=null)
{
if(inputLine.Trim().Length
            ==0)
break;
headerList.Add(inputLine);
}

The following code continues to read the NetworkStream from the point after the header of the email. Instead of the while loop, we could have used the StreamReader object's ReadToEnd() function to read to end of the NetworkStream, but for some reason, this function halts the application. It might be a bug! Anyway, the POP3 specification states that responses to certain commands are multi-line. In these cases, which are clearly indicated below, after sending the first line of the response and a CRLF, any additional lines are sent, each terminated by a CRLF pair. When all the lines of the response have been sent, a final line is sent, consisting of a termination octet (decimal code 046, ".") and a CRLF pair. If any line of the multi-line response begins with the termination octet, the line is "byte-stuffed", by pre-pending the termination octet to that line of the response. Hence a multi-line response is terminated with the five octets "CRLF.CRLF". When examining a multi-line response, the client checks to see if the line begins with the termination octet. If so, and if octets other than CRLF follow, the first octet of the line (the termination octet) is stripped away. While we are here, let me give you an idea of what an octet is. It is a byte of eight binary digits, usually operated upon as an entity. In short, an octet means a byte. To catch the CRLF.CRLF, I have used two strings, one is prevString to hold the previous readLine, the other is readLine.

The statement if (prevString.Trim().Length ==0 && inputLine.Trim() == ".") captures the CRLF. In the case of CRLF, the length of this line will be zero, and the inputLine variable will be the "." character. After that, we need to check whether there is more characters after the period character by using if(sr.Peek()==-1). If there is no character, that means we are at the end of the message.

inputLine=null;
string
            prevString="";
while ((inputLine=sr.ReadLine())!=null)
{
if
            (prevString.Trim().Length ==0 && inputLine.Trim() ==
            ".")
{
if(sr.Peek()==-1)
break;
}
prevString=inputLine;
bodyMessage.Append(inputLine+"\r\n");
}

Component Design

The project has two sub components:

  • POP3 Mail Message component: which is designed for holding and parsing message headers and body text.
  • POP3 Client Access Component: which is designed for sending and receiving POP3 commands from/to the POP3 email server.

Let's start with the POP3MailMessage component. I designed the mail message with three objects:

  • Pop3MailMessage Object
  • Pop3MailHeader Object
  • Pop3MailBody Object

The Pop3MailMessage object holds the Pop3MailHeader and Pop3MailBody objects. As you can see, Pop3MailHeader parses the header of the mail, and Pop3MailBody holds the message part of the mail. The Pop3MailBody class currently does not support parsing emails with attachments. The detailed component design is as follows:

  • The header variable in the Pop3MailMessage object is declared as a Pop3MailHeader object, so it links Pop3MailMessage and Pop3MailHeader objects.
  • The body variable in the Pop3MailMessage object is declared as Pop3MailBody, so it links Pop3MailMessage and Pop3MailBody objects.

In short, the Pop3MailMessage object is holding message header and text information in Pop3MailHeader and Pop3MailBody, respectively.

The logic of the application is in the Pop3Client class. User can connect to the POP3 server, read messages, retrieve statistics, and so on. We invoke Pop3MailMessage in the Retrieve function in the Pop3Client object. This function adds headers to the Pop3MailHeader object, and message text to the Pop3MailBody body. After parsing them, I add Pop3MailHeader and Pop3MailBody to the Pop3MailMessage object. The Retrieve function returns this Pop3MailMessage object.

Component Coding

The Pop3MailMessage object, Pop3MailHeader object, and Pop3MailBody object are instantiated in the retrieve Function in the Pop3Client class as follows:

Pop3MailMessage pop3MailMessage= new
            Pop3MailMessage();
Pop3MailHeader pop3Header;
Pop3MailBody
            pop3Body= new Pop3MailBody();

After receiving the header of the message, we add this message header to the Pop3MailHeader object as:

pop3Header= new Pop3MailHeader(headerList);

When you do this, the Pop3MailHeader constructor calls the Parse function automatically. The Parse function parses the Arraylist and assigns information to local variables. After parsing the header, we assign this Pop3Header object to a Pop3MailMessage object as follows:

pop3MailMessage.Header=pop3Header;

It is now time to extract the body of the message. After that, we put this message body into the Pop3MailBody object using:

pop3Body.Text=bodyMessage.ToString();

Then, we add this Pop3Mailbody object to a Pop3MailMessage object using:

pop3MailMessage.Body=pop3Body;

After parsing the header and message body, we now can return the Pop3MailMessage object that has the Pop3MailHeader object and Pop3MailBody object with the assigned values.

public Pop3MailMessage Retrive(int
            MessageNumber)
{
if(this.pop3Status!="TRANSACTION")
{
throw
            new Pop3Exception("Pop3 state is not in TRANSACTION
            state");
}

ArrayList headerList =new
            ArrayList();
StringBuilder bodyMessage= new
            StringBuilder();
Pop3MailMessage pop3MailMessage= new
            Pop3MailMessage();
Pop3MailHeader pop3Header;
Pop3MailBody
            pop3Body= new Pop3MailBody();

String
            inputLine;

if(this.count==0)
{
throw new
            Pop3Exception("There is no email in your
            account");
}

WriteToNetStream(ref netStream, "RETR " +
            MessageNumber);
StreamReader sr = new
            StreamReader(netStream);

while
            ((inputLine=sr.ReadLine())!=null)
{
if(inputLine.Trim().Length
            ==0)
break;
headerList.Add(inputLine);
}

pop3Header=
            new
            Pop3MailHeader(headerList);

pop3MailMessage.Header=pop3Header;

inputLine=null;
string
            prevString="";
while ((inputLine=sr.ReadLine())!=null)
{
if
            (prevString.Trim().Length ==0 && inputLine.Trim() ==
            ".")
{
if(sr.Peek()==-1)
break;
}
prevString=inputLine;
bodyMessage.Append(inputLine+"\r\n");
}

pop3Body.Text=bodyMessage.ToString();

pop3MailMessage.Body=pop3Body;

return
            pop3MailMessage;
}

The Sample POP3 Client Application

To demonstrate how you can use these components, I coded a sample email client application that receives email messages, without deleting them, from POP3 server. The purpose of the article is to show how a programmer can use .NET networking and RFC 1939 to implement email client capabilities. Therefore, the functionality of the application is not as much as you might expect.

Before using the application, you have to configure the application with POP3 server parameters. As you will see, you have to enter:

  • The POP3 Server host name or IP address
  • The POP3 Server Port number
  • Your UserName
  • Your password

After you entered this information, click on the Check Email button. Let's see what is going on behind this button. You can see this code in the btnEmail_Click event in the emailclient.cs.

The first thing to do is to instantiate the Pop3Client by providing the necessary information we covered earlier. After instantiating the Pop3Client object, you can retrieve the emails from the POP3 server. As we have already seen, the retrieve function returns an ArrayList object that holds all the email messages. The MessageCount variable holds the total number of emails received. If it is zero, alert the user by using MessageBox.Show("There is no Message in the Mail Server");

After receiving emails from POP3 server, we now can iterate the ArrayList object to extract Pop3MailMessage objects out of it. During this extraction, we add the information of "FROM" from the header to listEmail. After that, we can now disconnect the POP3 session by calling the function Disconnect().

private void btnEmail_Click(object sender,
            System.EventArgs e)
{
Pop3Client pop3 =new
            Pop3Client(txtServer.Text,Int32.Parse(txtPort.Text));
pop3.Pop3Connect(txtUser.Text,txtPass.Text);
Pop3MailMessage
            popmsg;

statusBar1.Panels[1].Text="Receiving....";
Emails=pop3.Retrive();
statusBar1.Panels[1].Text="Received....";
int
            MessageCount=pop3.Count;

statusBar1.Panels[0].Text="Message
            Count:"+
            MessageCount.ToString();

if(MessageCount==0)
{
MessageBox.Show("There
            is no Message in the Mail Server");
return;
}
for(int
            i=0;i<MessageCount;i++)
{
popmsg=((Pop3MailMessage)Emails[i]);
listEmail.Items.Add(popmsg.Header.From);

}

if(listEmail.Items.Count!=-1)
listEmail.SetSelected(0,
            true);

pop3.Disconnect();
listEmail.Focus();
}

Whenever you browse the listEmail, you can see the message body in the textbox object. How does this work? The following code will give you some idea:

private void
            listEmail_SelectedIndexChanged(object sender, System.EventArgs
            e)
{
lblSubject.Text="Subject:";
lblDate.Text="Date:";
lblCc.Text="Cc:";
lblTo.Text="To:";

int
            i=((ListBox)sender).SelectedIndex;

lblSubject.Text+="
            "+((Pop3MailMessage)Emails[i]).Header.Subject;
lblDate.Text+="
            "+((Pop3MailMessage)Emails[i]).Header.ReceivedDate;
lblCc.Text+="
            "+((Pop3MailMessage)Emails[i]).Header.Cc;
lblTo.Text+="
            "+((Pop3MailMessage)Emails[i]).Header.To;
txtMessage.Text=((Pop3MailMessage)Emails[i]).Body.Text;
}

As you see in the code, whenever you browse the listEmail object, SelectedIndexChanged is called. We can capture the SelectedIndex of the listEmail with this event, then we can use this index to retrieve information from the ArrayList object, such as:

  • Subject
  • Date
  • Cc
  • To
  • Message

Application Setup

For this project, I have used two Visual C# projects, one is for the email client, the other is for the POP3 components. There are two folders:

  • Pop3Client folder (in which there is Email Client project)
  • Pop3ClientLib folder (in which there is POP3 components project)

To run the application, you need to run VS.NET, and open the solution file named Pop3Client.sln in the Pop3Client folder.

Any Limitations or Further Work

There are some limitations in the code: the application does not support parsing emails with attachments, and does not support APOP connection. These features could be added with further work.

Conclusion

We have seen in this article how we can create a POP3 client application. During this process, we have used POP3 commands and .NET networking objects in order to connect to POP3 server, receive email messages, and parse the received messages into mail headers and mail body text. This will not be enough to show how it works; therefore I have coded a sample email application that receives email by using Pop3Client and other components. During this process, I have not coded any procedures for deleting received message from the POP3 server, because the emails are not saved to computer. As you can see, it is not difficult to write code for POP3 standards. Even though it is easy, I do not understand why Microsoft has not released one so far. This has been a very good practice and experience for me. Thanks to C#Today.


.NET Application Domains

.NET Application Domains
http://www.csharptoday.com/content.asp?id=1704
Article Rating: 4
Published on November 27, 2001
Abstract
One of the most important features of any multi-tasking and multi-user system is process or application isolation. When multiple applications are active on a single system, it is crucial that these applications are protected and isolated from one another, to avoid them causing any damage to each other. The unit of isolation under Windows has traditionally been at the process level. This has changed slightly under the .NET framework. The Common Language Runtime (CLR) provides a slightly finer grained layer of isolation known as Application Domains or App domains. In this article, we will explore the .NET runtime’s support for application domains, and build a sample application that explicitly creates an application domain, and loads and executes an assembly in it.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

One of the most important features of any multi-tasking and multi-user system is process or application isolation. When multiple applications are active on a single system, it is crucial that these applications are protected and isolated from one another. This prevents one application from causing harm to another, either accidentally or maliciously. While applications need to be isolated from each other, applications also need the ability to communicate with each other. The base operating system is responsible for allowing this process isolation, as well as interprocess communication. The unit of isolation under Windows has traditionally been at the process level. Individual processes are isolated from each other and cannot access each other's memory directly. This has changed slightly under the .NET framework. The Common Language Runtime (CLR) provides a slightly finer grained layer of isolation known as Application Domains or App domains. An application domain is the entity that is protected from all other application domains by the runtime plumbing. In this article, we will explore the .NET runtime's support for application domains, and build a sample application that explicitly creates an application domain, and loads and executes an assembly in it.

Note that this article is based on Visual Studio .NET Beta 2 and Windows 2000.

Processes and Application Domains

Every application under Windows runs inside its own process space. A process space is essentially a contiguous chunk of memory, known as an address space. An address space holds all the code and data that makes up an application. Every process is allocated its own address space, and a memory address within a process is relative to the address space in which the process is loaded. As a result, any access to a memory location is always mapped to an address within the same process. Since all processes share the same virtual address space, it is impossible to directly access memory from one address space into another address space. This has been one of the fundamental principles of memory management and process isolation under Windows. This is a great model when it comes to protecting processes from buggy or malicious code in a different process, but this model is fairly expensive when it comes to interprocess communication, and the ability to run a large number of processes on a single machine. Since the processes are totally isolated, interprocess communication has to rely on the operating system's support for marshalling the data into the other process. Further, this can also lead to a large number of context switches, where the operating system scheduler grants CPU time to another process. A context switch is a fairly expensive operation, since in addition to saving the process's context, it could also mean swapping the process out to virtual memory (to the page file stored on disk). This issue is further compounded when a single machine has a large number of active processes (common for a server serving a large number of web sites or web services). Under this scenario, the CPU is often reduced to swapping processes in and out of memory continuously, a phenomenon known as thrashing. This leads to almost 100% CPU utilization, with very little time available to any process for actually processing requests, causing a massive degradation in response time and throughput.

Process isolation, as described above, cannot be avoided in a system that runs unmanaged code. Unmanaged code cannot be checked before execution to determine type safety or its memory access patterns. Unmanaged code could be written to arbitrarily access any block of memory using pointers, and if processes were not isolated, this could lead to total chaos. This problem, however, does not exist under managed code. Managed code can never access memory directly, and managed code is guaranteed to be type safe. Managed code always runs under the context of the common language runtime; and thus managed code, as the name suggests, is always managed and controlled by an external entity. Since the traditional issues that require high process isolation do not exist under the CLR, there isn't sufficient motivation for incurring the overhead of process isolation with managed code. In order to optimize application isolation, without having to resort to process isolation, the .NET runtime introduces the concept of an Application Domain. An application domain is a logical process within a physical process. Application domains provide all the benefits of process isolation, without incurring any of the overheads associated with process isolation.

A single physical process can host multiple application domains, as shown in the following diagram:

Application domains consist of one of more assemblies, and each app domain is isolated from the other. The following are some of the features of app domains.

  • Isolation: Application domains are completely isolated from each other. This is guaranteed due to the type safety and verification of managed code. Since memory cannot be directly accessed, there is no way to bypass this verification process. Isolation of application domains ensures that faults in one application do not affect other applications.
  • Proxies: Communication between different application domains is possible via the use of proxies. This creation of proxies is transparent to the developer, and the necessary plumbing is provided by the runtime. Objects that are passed by value are directly copied over to the remote app domain, and objects passed by reference are accessed via proxies. Note that the same remoting infrastructure is used for cross app domain calls, irrespective of whether the app domains are in the same process, different processes, or even different machines.
  • Independent Control: An app domain can be controlled independently of other app domains hosted in the same physical process. App domains can be loaded and unloaded independently, and configuration settings are applied at the app domain level. Note that an individual assembly cannot be unloaded, the entire app domain has to be unloaded.

Application domains, in short, are logical processes within a physical process. Inter-app domain communication, in spite of marshalling and proxies, is much more efficient that interprocess communication, and furthermore, app domains do not compound the context switching and process swapping issue.

App domains, Assemblies and Hosts

As we discussed in the earlier section, an application domain contains one or more assemblies. These assemblies are loaded by the application domain, either implicitly (when a component in an assembly is referenced from a loaded assembly), or explicitly when the System.AppDomain class is used to load assemblies. Note that an assembly can be loaded in multiple application domains running within a single process. By default, each app domain will load the assembly's code as shown in the diagram in the earlier section. Thus, it is possible for an assembly to be loaded multiple times in different app domains. This duplication leads to higher memory consumption for the process, since the code is loaded multiple times in different locations. In order to avoid this duplication, the .NET runtime introduces the concept of domain neutral assemblies. These are assemblies that are not loaded into a specific application domain, and can be shared across all domains within a process. In case of domain neutral assemblies, only code is shared across app domains - any data that is associated with a particular domain is specific to that domain. Win32 developers will note the similarity of domain neutral assemblies with dynamic link libraries that share code across different Win32 processes. The process hosting the application domains makes a determination of whether assemblies should be loaded as domain specific or domain neutral. Domain neutral assemblies can introduce a slight performance impact since it results in a cross domain call.

A process that loads the .NET runtime into a process and creates one or more app domains to run managed code, is known as a Runtime Host. A runtime host is responsible for loading the .NET runtime into a process, then loading the assemblies required to run an application into an application domain, and finally executing the main assembly in the domain. The runtime host is also responsible for setting configuration properties, security settings, and policies for all application domains hosted within that process. A runtime host is essentially a fully functional instance of the plumbing required to run managed code. Note that the runtime host itself is not required to handle any of the .NET runtime's capabilities; this is all delegated to the runtime, which itself runs within the host process. Currently, there are three runtime hosts that are available from Microsoft:

  • ASP .NET: This handles requests for all Web applications and Web services. This is the process that makes the single most effective use of application domains. Every Web application hosted on a Web server is run in its own separate application domain to provide application isolation. In the absence of app domains, every application would have been run in its own separate process, leading to a large number of processes on a server hosting many web applications and services.
  • Internet Explorer: This manages the execution of downloaded managed controls.
  • Windows Shell: This is the traditional method of launching executable applications. Any managed executable can be launched from the command line or Windows Explorer, and the Windows shell acts as a host for these applications.

It is possible to write custom hosts that can run managed code in addition to the three hosts mentioned above. Please refer to the Links section for details on how to write a custom host.

System.AppDomain Class

Now that we have covered the details of application domains and assemblies in detail, let us now focus on writing some code that will allow us to manipulate app domains. The System namespace contains a class called AppDomain that contains methods that support the creation and manipulation of application domains. This class supports methods that allow for the creation of a new application domain, and to load and execute assemblies into that domain. Further, this class supports properties and methods that can be used to configure the application domain, such as the path where assemblies are located and security policies.

Let us start with a simple example that lists all the assemblies that have been loaded into a specific application domain. We will build a simple Windows Forms application as shown in the following screen shot:

The following code block shows how we can use the AppDomain class to get information about the loaded assemblies:

lstList.Items.Clear();
string
            lAssemblyName;

AppDomain lAppDomain =
            AppDomain.CurrentDomain;
Assembly[] lAssemblies =
            lAppDomain.GetAssemblies();

foreach (Assembly lAssembly in
            lAssemblies){
lAssemblyName =
            lAssembly.FullName;
lstList.Items.Add(lAssemblyName);
}

The System.AppDomain class contains a shared property CurrentDomain that returns a pointer to the application domain in which the current assembly is executing. Once we have an instance of this object, we can invoke the GetAssemblies method to get a collection of all the assemblies that have been loaded. Next, we iterate through this collection to obtain the full name of each loaded assembly. The following screen shot shows the assemblies that are loaded for a bare bones Windows Forms application:

The mscorlib assembly represents the .NET runtime, the AssemblyList assembly is the name of our sample application, and the other System assemblies are the assemblies required for a Windows Form application. The Accessibility assembly is also loaded by default by the runtime. Note that the runtime loads assemblies only on demand. Thus, if the code contains a reference to a class in an assembly, the assembly is loaded only when this code is executed. To demonstrate this, we have a button called System.XML, which creates an instance of the XMLDocument class from the System.XML namespace (and assembly), as shown below:

private void btnXML_Click(object sender,
            System.EventArgs e)
{
System.Xml.XmlDocument
            lXMLDoc;
lXMLDoc = new System.Xml.XmlDocument
();
}

Note that in spite of this reference to a class in the System.XML assembly, this assembly is not loaded at startup. If you click this button to execute this code, and then click on Get List again to retrieve a list of loaded assemblies, you will see the System.XML assembly loaded, as shown in the following screen shot:

Now that we have covered a simple usage of the System.AppDomain class, let us see how we can use this class to create a new application domain, and to load and execute an assembly in it. This is a fairly simple process; all you need to do is create an application domain with a new name, and then call the ExecuteAssembly method on the newly created app domain, passing in the name and location of the assembly that you want to execute. Note that for an assembly to be executed, it needs to expose a well defined entry point. Executables have an entry point, which is usually the Main method of the startup class; class libraries on the other hand, don't have entry points by default. The entry point of an assembly is marked in the assembly's metadata with the .entrypoint keyword. Assemblies that do not declare a well defined entry point cannot be executed directly in an app domain.

The following screenshot shows this for the Main method of the frmAssemblyList class, as seen from the LILDasm tool:

The following code block shows how an application domain can be created and unloaded.

private void btnExecute_Click(object sender,
            System.EventArgs e)
{
string lUniqueName =
            Guid.NewGuid().ToString ();
try{
mAppDomain =
            AppDomain.CreateDomain(lUniqueName);
mAppDomain.ExecuteAssembly(txtAssembly.Text);
}
catch
            (AppDomainUnloadedException lException){}
}

private void
            btnUnload_Click(object sender, System.EventArgs e)
{
if
            (mAppDomain != null)
            {
AppDomain.Unload(mAppDomain);
mAppDomain =
            null;
}
}

We use the Guid.NewGuid method to generate a unique name for the newly created application domain, and then call the ExecuteAssembly method, passing in the name of the assembly that we want to execute. When we want to terminate this app domain, we use the AppDomain.Unload method, passing in the app domain object returned by the CreateDomain call. Any attempt to access the app domain after it has been unloaded results in an AppDomainUnloadedException.

This has been a very simple example to demonstrate the use of app domains, but this is a very powerful concept. A question that comes to mind after reading this is - why would one want to create app domains explicitly when the runtime host should be handling this? One of the biggest uses of explicit app domain creation is to isolate questionable code. If you are developing a managed application using downloaded code, or a shareware or open source component, and cannot trust it completely, running it in its own application domain provides an excellent way to isolate the rest of the applications from this code. Further, you can specify custom security policies on this app domain to ensure that this third party code runs in an extremely strict and controlled app domain, where the chances of it causing damage to the rest of the system are minimal. Explicit app domains can also be used when the developer wants fine grained control of when assemblies are loaded and unloaded for performance reasons. Note that individual assemblies cannot be unloaded, the entire app domain has to be unloaded.

Attached Sample Code

The attached sample code consists of two Windows forms applications: AssemblyList and AppDomainCreation. We have discussed the Assemblylist application earlier in this article. The following screen shot shows the AppDomain Creation application. This takes in an assembly name and attempts to execute it in a separate app domain. Clicking on the Unload button results in the application domain being unloaded, which can be seen by the launched application terminating.

Conclusion

Application domains provide an excellent alternative to process level application isolation. Application domains offer all the benefits of process isolation, but are much more efficient than processes. Further, as seen from the example in this article, creating an application domain and executing code in it is trivial. Developers should take advantage of this feature when high isolation of a component or assembly is desired.

Happy .NETing.


Automated Generation of Table Columns, Class Properties, and Form Field Mapping with C# Reflection

Automated Generation of Table Columns, Class Properties, and Form Field Mapping with C# Reflection
http://www.csharptoday.com/content.asp?id=1702
Article Rating: 4
Published on November 21, 2001
Abstract
In this article, Tin Lam shows how to use reflection to help us with complex mappings. Reflection allows us to examine assemblies and types at runtime, and find out everything about that type. We will also see how to dynamically extract the type information, and then carry out automated mappings between database table columns, HTML form fields and class properties.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

Have you ever needed to create a web application that consisted of some twenty, thirty, or even more database fields? Surely most of us have at one time or another. Some of us might have built hundreds of them already. When building such applications in ASP and VB, we often need to take the values from the form fields, and assign them to the class properties manually, one by one, for updating or insertion to the database. Then, when we want to select the data from the database, we need to manually take each column from the recordset and assign them to the class properties, again one by one, perhaps in a loop, to build a collection or array of objects of some sort. However, using an amazing feature in the C# language known as reflection, our life will be much easier. We can build some tools to simplify and automate all these mappings. In the ASP.NET pages, regardless of how many fields you have, with just one line of code you can initiate an object with all its properties initialized from a HTML form. With another line of code, you can get an ArrayList of objects with all the properties of each object loaded from a DataReader based on a select query.

In this article, we are using the words "type" and "class" interchangeably, but they are pretty much referring to the same thing.

The old annoying ASP and VB approach

With ASP and Visual Basic, we really don't have any other choice but to map out each field line-by-line. Some code would be used to load the form data to an object, then insert the object to a corresponding database table:

Set m = Server.CreateObject("myCom_thisAppl.member")
m.firstName = Request.Form("firstName")
m.lastName = Request.Form("lastName")
...
m.DBInsert ()

Then, in a Visual Basic class, some code would be used to build a collection of member objects:

Rs.Open "select * from member", Conn, 0, 1
Set members = new Collection
Do While Not Rs.EOF
Set m = new myCom_thisAppl.member
m.firstName = rs("firstName")
m.lastName = rs("lastName")
...
members.Add m
rs.MoveNext
Loop

In some enterprise applications, the "..." might represent another thirty lines of mappings. There is nothing wrong with this! There are millions of applications already built in this way, and they all work just fine, but wouldn't you rather save some of that time and energy spent typing all those boring lines, and instead use it to deal with the much more interesting and important business logic?

C# Reflection

Reflection allows us to examine any assemblies or types at runtime. It lets us find out everything about a type. It allows us to dynamically extract the type information and examine it, looking at such things as property names, as well as dynamic object creation and method invocations. We are then able to carry out some automated mappings between the database table columns, HTML form fields, and the class properties.

In this article, we are making use of a "turtle" application that's driven by five files:

  • turtle.aspx: contains a form and some layout logic.
  • turtle.aspx.cs: is a code-behind file behind the turtle.aspx. It contains a method that triggers a server-side onClick event of a button to do a database insert, and also a property to get an ArrayList of turtle.
  • turtle.dll: is a single file assembly that contains the characteristic of a turtle (properties of a class).
  • db.dll: is also a single file assembly. It contains methods to put turtles into the database, as well as pulling some back out (database access). It also reads through a DataReader object to construct an ArrayList of objects (which is turning out to be some turtles!).
  • util.dll: is also a single file assembly. It contains two static methods to initiate, and initialize an object with the request parameters.

All the source code can be found in the downloadable support material, as well as some batch files to compile them. Detailed setup instructions can be found in readme.txt. All the sample codes were developed and tested on the .NET Framework beta 2.0.

A database table, and a stored procedure

The following code shows the database table for the turtles, and a stored procedure which adds turtles to the database.

create table turtle (
turtleID int identity(1, 1),
turtleName varchar(20),
worth float,
birthday datetime,
isCute bit,
dateCreated datetime default getdate()
)
go

create proc addTurtle
@turtleName varchar(20),
@worth float,
@birthday datetime,
@isCute bit
as
insert turtle (
turtleName, worth, birthday, isCute)
values (
@turtleName, @worth, @birthday, @isCute)

select @@identity as turtleID
go

Turtle.cs

The following is a listing of the turtle.cs file:

using System;

public class turtle
            {

private int _turtleID = 0;
private string _turtleName =
            "";
private double _worth = 0;
private DateTime _birthday =
            new DateTime();
private bool _isCute = false;

public int
            turtleID {
get { return _turtleID; }
set { _turtleID = value;
            }
}

public string turtleName {
get { return
            _turtleName; }
set { _turtleName = value; }
}

public
            double worth {
get { return _worth; }
set { _worth = value;
            }
}

public DateTime birthday {
get { return _birthday;
            }
set { _birthday = value; }
}

public bool isCute
            {
get { return _isCute; }
set { _isCute = value;
            }
}

public bool isSmartLooking {
// if worth more than
            100 bucks, then it must be smart looking!
get { return ( _worth
            > 100 ); }
}

public turtle() {}

public bool
            doDBInsert() {
try {
String sqlQuery = "addTurtle " + "'"
+ db.QF(_turtleName) + "', "
+ _worth + ", "
+ "'" +
            _birthday + "', "
+ ( isCute ? 1 : 0 );
_turtleID = new
            db().getNewID(sqlQuery);
return true;
} catch {
return
            false;
}
}
}

isSmartLooking is a read only property. Its value is dependant on another property (worth). Also, it will not be retrieved from, or stored to a database. doDBInsert() is a method that will pass the fields to the db.getNewID() method, which will call the stored procedure to insert the data. Then it will assign the returned new ID to _turtleID. The method returns true if insertion is successful and a new ID is returned, false otherwise.

turtle.aspx

The following is a listing of the turtle.aspx file:

<%@ Page Language = 'C#' AutoEventWireUp = 'false' Inherits = 'wrox.turtleHelper' Src = 'turtle.aspx.cs' %>
<html>

<head>
<title>turtle</title>
</head>

<body>

<table border='1'>

<tr>
<td><b>id</b></td><td><b>name</b></td><td><b>birthday</b></td>
<td><b>worth</b></td><td><b>cute?</b></td><td><b>smart?</b></td>
</tr>

<% if ( turtles != null ) {
foreach (turtle t in turtles) { %>
<tr>
<td><%= t.turtleID %></td>
<td><%= t.turtleName %></td>
<td><%= t.birthday.ToShortDateString() %></td>
<td><%= t.worth %></td>
<td><%= ( t.isCute ? 'yah' : 'nah' ) %></td>
<td><%= ( t.isSmartLooking ? 'yes' : 'no' ) %></td>
</tr>
<% }
} %>

<tr> <td colspan='6'><hr></td></tr>
<form runat='server'>
<tr>
<td> </td>
<td><input type='text' size='10' maxlength='10' name='turtleName'></td>
<td><input type='text' size='10' maxlength='10' name='birthday'></td>
<td><input type='text' size='10' maxlength='10' name='worth'></td>
<td><select name='isCute'><option value='yes'>yes</option><option value='no'>no</option></select></td>
<td> </td>
</tr>
<tr>
<td colspan='4'><asp:Label runat='server' id='msg'/></td>
<td><asp:Button runat='server' id='addTurtle' text='add!'/></td>
<td> </td>
</tr>
</form>

</table>

</body></html>

This page inherits the wrox.turtleHelper class from the code-behind file turtle.aspx.cs. turtles is an ArrayList property of wrox.turtleHelper. This page contains only the layout logic of the list and the form. Its code-behind file does all the other work. In this article, we are only displaying the objects properties in a HTML table. Obviously in ASP.NET, there are so many List Controls with styles and templates you can use to bind the data to, they will do a much better listing job for you. However, if you also need to carry out some processing on each object, then you can't totally rely on the List Controls.

turtle.aspx.cs

The following is a listing of the code-behind turtle.aspx.cs file:

namespace wrox {

using
            System;
using System.Drawing;
using
            System.Collections;
using System.Web.UI;
using
            System.Web.UI.WebControls;

public class turtleHelper : Page
            {

protected Button addTurtle;
protected Label
            msg;

protected ArrayList turtles {
get {
return new
            db().getArrayListOf("select * from turtle",
"turtle",
            "turtle");
}
}

public turtleHelper() {
Page.Init +=
            new System.EventHandler(Page_Init);
}

protected void
            Page_Load(object sender, EventArgs e) { }

protected void
            Page_Init(object sender, EventArgs e)
            {
InitializeComponent();
}

protected void
            addTurtle_Click(object sender, EventArgs e) {

turtle t =
            (turtle)util.getObject(Request, "turtle",
"turtle",
            "post");
if ( t.doDBInsert() ) {
msg.Text = "added. new id =
            <b>" + t.turtleID + "</b>.";
msg.ForeColor =
            Color.Blue;
} else {
msg.Text = "error
            adding";
msg.ForeColor = Color.Red;
}
}

private void
            InitializeComponent() {

msg.Text = "enter then
            add!";
this.addTurtle.Click += new
System.EventHandler(this.addTurtle_Click);
this.Load += new
            System.EventHandler(this.Page_Load);
}
}
}

As you can see, the code to get data from the form fields, and to select/insert data, is minimal due to the "help" from util and the db, because the util.getObject() and db.getArrayList() are making all the magic happen for us. The label control msg is used in the ASP.NET page to display a message, and the button control addTurtle acts like a submit button. Once it's clicked, it triggers the addTurtle_Click() method.

The util.getObject() method takes four arguments: a reference to the requested object for the current page request, the assembly name, the class name, then a string indicating the page request method (post, get, etc). It returns the object of the type specified in the second and third parameters. Now that the page has a reference to the object with the data from the form, it can simply call its doDBInsert() method to insert the data, and expect a new ID for the new turtle.

The db.getArrayList() method takes just two arguments: an SQL string, and either a select query or a stored procedure that returns some records. This method returns an ArrayList of objects with their property values reflecting the fields in the recordset defined by the SQL string. Now all of a sudden, the page has an ArrayList to work with, using just one line of code!

Now let's see how db makes the magic happens (listing from db.cs):

using System;
using
            System.Security;
using System.Data.SqlClient;
using
            System.Collections;
using System.Reflection;

public class
            db {

private SqlConnection dbConn;
private SqlCommand
            dbComm;
private SqlDataReader dr;
private string connString =
            'DATABASE=wrox;SERVER=localhost;UID=wrox;PWD=wrox';

public
            static string QF(string s) {
return s.Replace(''',
            '''');
}

public int getNewID(string sqlQuery) {
int
            newID;
dbConn = new SqlConnection(connString);
dbComm = new
            SqlCommand(sqlQuery, dbConn);
dbConn.Open();
newID =
            Int32.Parse( dbComm.ExecuteScalar().ToString()
            );
dbConn.Close();
return newID;
}

public ArrayList
            getArrayListOf(string sqlQuery, string assemblyName, string
            className) {

Type t;
int fieldCount, i;
object
            currentObject;
ArrayList listOfWhatever = new
            ArrayList();

try {

dbConn = new
            SqlConnection(connString);
dbConn.Open();
dbComm = new
            SqlCommand(sqlQuery, dbConn);
dr =
            dbComm.ExecuteReader();
fieldCount = dr.FieldCount;
t =
            Activator.CreateInstance(assemblyName,
            className).Unwrap().GetType();

while ( dr.Read() )
            {
currentObject = Activator.CreateInstance(assemblyName,
            className).Unwrap();
for ( i = 0; i < fieldCount; i ++ )
            {
if ( !dr.IsDBNull(i) ) {
try {
t.InvokeMember(
            dr.GetName(i), BindingFlags.SetProperty,
null, currentObject, new
            object [] { dr.GetValue(i) } );
} catch (MissingFieldException
            mfe) {
} catch (MissingMethodException mme) {
} catch
            (SecurityException se) {
} catch (Exception e) {
// other
            exceptions
}
} else {
// let this one pass, not interested
            in null values
}
}
listOfWhatever.Add( currentObject
            );
}
dr.Close();
dbConn.Close();
return
            listOfWhatever;
} catch(Exception e) {
return
            null;
}
}

t = Activator.CreateInstance(assemblyName, className).Unwrap().GetType(), Activator.CreateInstance() is the heart of everything in this article. This static method is overloaded with eight different signatures. We are using this one: public static ObjectHandle CreateInstance(string assemblyName, string typeName). First the method creates an ObjectHandle object that wraps the object of the type specified by the parameters. From this returned ObjectHandle object, it calls the Unwrap() method to unwrap the wrapped object in the ObjectHandle to get the object we are interested in (the turtle!). Finally, we call GetType() to get a Type object, which lets us know pretty much everything about the turtle, such as the property names, types, etc.

currentObject = Activator.CreateInstance(assemblyName, className).Unwrap() creates an object for each row we process. Note that we are just creating the turtle objects, not the type objects of the turtle.

t.InvokeMember(dr.GetName(i), BindingFlags.SetProperty, null, currentObject, new object [] { dr.GetValue(i) } ) is the method that is making all of the mapping magic happen. It's overloaded with three different signatures, and we are using this one: public object InvokeMember(string name, BindingFlags invokeAttr, Binder binder, object target, object[] args). The first parameter is the name of the property we want to assign the values to, and we get the names from the DataReader's column by calling the GetName() method. Of course, we need the names of the DataReader's columns and the class properties to be identical for this method to make the magic happen! The second parameter indicates that we are trying to set a value to a property. The third parameter defines a binder for binding a set of properties or overloaded methods, but since we not making use of it, so we can just pass null to it. The fourth parameter is a reference to the instance of the type, which we are assigning values to its properties for. The fifth parameter is an array of objects to fill the argument list of the method/property we are invoking. GetValue() will implicitly box the values (bit, for example) from the record fields and return some objects (System.Boolean, for example). When methods are invoked, the returned object will be implicitly unboxed and mapped to Value in the set block. The InvokeMember() method actually returns an object, which the invoked method returns too. But since we only setting the properties, we just let them go.

You might be wondering what happens to the data field dateCreated, as we don't have a matching property for it. The InvokeMember() method is in a try-catch block with multiple catches. If the matching property does not exist, or it's read-only, then it will throw an exception, most likely MissingFieldException, MissingMethodException, SecurityException, or Exception for all other errors. It will then be routed to one of the catch blocks. Our catch blocks are not doing anything, so it just moves on to the next column. Alternatively, you can use the GetProperty() method to try to get a PropertyInfo object, so you can then examine the property you are dealing with via the PropertyInfo object. If you get a null value, that means that the property does not exist. We will be making use of the PropertyInfo type in util.getObject().

Here is what the util type with the getObject method looks like (from util.cs):

using System;
using System.Web;
using
            System.Reflection;

public class util {

public static
            object getObject(HttpRequest req, string assemblyName, string
            className, string loadFrom) {

PropertyInfo[]
            properties;
TypeCode currentType;
object
            currentObject;
object currentValue;
Type t;

try { //
            see if the class can be created, if so, then get its Type
            infos
currentObject = Activator.CreateInstance(assemblyName,
            className).Unwrap();
t = currentObject.GetType();
} catch
            {
return null;
}

loadFrom =
            loadFrom.ToLower();
properties =
            t.GetProperties();
foreach(PropertyInfo proc in properties)
// taking a property at a time ...
if ( proc.CanWrite ) {
// if it's writable,
try {
// then match a value from the
            QS/form
currentType =
            Type.GetTypeCode(proc.PropertyType);
currentValue = getParam(req,
            proc.Name, currentType, loadFrom);
if ( currentValue != null )
// if there's a value returned,
proc.SetValue(currentObject,
            currentValue, null);
// then set the value to the property
}
            catch {
// shouldn't reach here ...
}
}

return
            currentObject;
}

public static object getParam(HttpRequest
            req, string paramName, TypeCode paramType,
string loadFrom)
            {

try {

string paramValue;

if ( loadFrom ==
            'post' )
paramValue = req.Form.Get(paramName);
else if (
            loadFrom == 'get' )
paramValue =
            req.QueryString.Get(paramName);
else
return null; // the
            request method is not supported/implemented

if ( paramType ==
            TypeCode.Byte ) return Byte.Parse( paramValue );
if ( paramType
            == TypeCode.Char ) return Char.Parse( paramValue );
if (
            paramType == TypeCode.Decimal ) return Decimal.Parse( paramValue
            );
if ( paramType == TypeCode.Double ) return Double.Parse(
            paramValue );
if ( paramType == TypeCode.Int16 ) return
            Int16.Parse( paramValue );
if ( paramType == TypeCode.Int32 )
            return Int32.Parse( paramValue );
if ( paramType ==
            TypeCode.Int64 ) return Int64.Parse( paramValue );
if ( paramType
            == TypeCode.SByte ) return SByte.Parse( paramValue );
if (
            paramType == TypeCode.Single ) return Single.Parse( paramValue
            );
if ( paramType == TypeCode.UInt16 ) return UInt16.Parse(
            paramValue );
if ( paramType == TypeCode.UInt32 ) return
            UInt32.Parse( paramValue );
if ( paramType == TypeCode.UInt64 )
            return UInt64.Parse( paramValue );
if ( paramType ==
            TypeCode.DateTime ) return DateTime.Parse( paramValue );
if (
            paramType == TypeCode.String ) return paramValue;
// already a
            string, so no need to parse.
if ( paramType == TypeCode.Boolean )
            {
switch( paramValue.ToLower() ) {
case '+':
case
            '1':
case 'ok':
case 'right':
case 'on':
case
            'true':
case 't':
case 'yes':
case 'y':
return
            true;
default:
return false;
}
}
return null;
//
            if paramType is not any of the .NET CLT, ex: an object/array
}
            catch {
return null;
// paramType might be a CLT, but there's
            an error parsing the value.
}
// ex: paramValue = '' or
            characters for numeric types or null
}
}

We are using the Activator.CreateInstance() and the GetType() the same way we did in db.getArrayListOf(). However, the fields-mapping mechanism is quite different. In db.getArrayListOf() we looped through the data columns collection, and for each column we tried to get a matching property from the type, which might or might not exist, and might or might not be read only. In util.getObject(), we are looping through the properties array of the type we specified in the parameters, examining it, making sure it's not read only, and then trying to get a matching field from the request parameters. When we get an array of properties from the turtle type, isSmartLooking will be in the array too, but its CanWrite property will tell us that it's read only, so we won't bother to assign it any values. Since each PropertyInfo object knows its own status (for example, it knows its name, the fact that it is a property, etc), we can use its SetValue() method to assign the objects some values. This way, we don't have to use the much wider scope and more generic InvokeMember() method in the Type class. This SetValue() method is overloaded with two signatures, and we are using: public virtual void SetValue(object obj, object value, object[] index). The first parameter is the object you are setting the property for, and the second is the value you are setting to the property. The third is some optional index values for indexed properties. It's not applicable to our sample application, so we just send null to it.

Screen shots

We have just added the turtle John and Benny, who's neither smart nor cute. And we are about to add Tin, who's cute, and worth enough to be considered smart.

We have just added Tin, and its new ID is 3, as displayed by the blue text at the bottom.

Just added another turtle, not cute, but smart.

Issues to consider

Fields and properties naming in different tiers

As mentioned earlier, the magic happens on identical names in the table columns, class properties and the form fields. Although the class properties will stay pretty much the same, you can alias the recordset columns to accommodate the property names. With the form fields to class property mapping, sometimes you might want to create some additional hidden fields, matching the names to the properties whose values get assigned by some client-side scripting languages (most likely JavaScript) before the form gets submitted. For example, you might have three separate input boxes for a date entry, but just a single column/property of type DateTime.

Performance tradeoff

Although it's easy to be lazy and let the reflection do the boring mappings, it's a lot more work for the server. For the mappings that occur in util.getObject(), it's probably fine, because most of the time we just need to map a few objects from the form for updating/insertion/deletion, which would take just a few extra milliseconds. The db.getArrayListOf(), on the other hand, is another story. It loops through the DataReader, then it has to make a new object for each record, loop through the columns, box the values in the fields to an object, then dynamically invoke the method and un-box the object! That's a considerable amount of work. I have done some performance testing to see what the trade off is, and the result is that with the same DataReader object, it's about 30% slower on average with the mappings done by reflection, than it is with the manual line-by-line mappings. On my PIII-800 with 256 RAM, it took about 4 seconds on average to map 10,000 records line by line, and about 6 seconds with reflection. The recordset consists of twelve columns, consisting of the following values: 2 bits, 2 smallints, 2 ints, 2 doubles, 2 datetimes and 2 varchars. For web applications, normally you won't process this many records at once anyway. Say you are dealing with one or two hundred records, the tradeoff won't even be noticeable to the users.

Using A Typed DataSet

If 30% sounds like an unaffordable tradeoff to you, and you would still like to automate the mapping somehow, consider a typed DataSet. Typed DataSets are classes derived from the DataSet class, and so they inherit all the properties and methods found in a DataSet. Given an XML Schema Definition language (XSD) file, which describes how the mappings should be done, a tool in the .NET Framework SDK will generate all the mapping code for you. You will then be able to modify this and add your own code. It is also capable of generating code to build a class hierarchy that reflects the table relationships. With a typed DataSet, you have to know XSD to describe precisely how each mapping should be done. Any changes to the mappings might well mean you have to go back to the XSD, and regenerate the class files, then perhaps put in your own code again, and recompile it, which is not as flexible as the reflection approach.

Enhancements

You can also put the util.getObject() method in a code-behind file or a generic ASP.NET page, which other pages will be diverting from, so that you don't have to pass in the reference of the Request object to it. The mapping between the class properties and the HTML fields only happens in web applications anyway, so other windows applications won't find it very helpful. You can even build it as a custom control and register it in the page directives. With some changes to the util class itself, you can do something like this:

<% object obj; %>
<util:useType
assembly = "turtle" class = "turtle"
propertyFromPost = "turtleName, worth, birthday, isCute"
propertyFromGet = "turtleID"
/>
<%= ((turtle)obj).doDBUpdate() ? "updated" : "update failed" %>

Here, you initiate an object, then get the turtleID from the QueryString and the rest from the Post headers. Then you do a database update, and display a quick message.

Conclusion

In this article we have learnt how to use reflection to help us with complex mappings. Reflection allows us to examine assemblies and types at runtime, and find out everything about that type. We have seen how to dynamically extract the type information, and then carry out automated mappings between database table columns, HTML form fields and class properties.


COM+ and .NET Security Interoperation

COM+ and .NET Security Interoperation
http://www.csharptoday.com/content.asp?id=1699
Article Rating: 4.5
Published on November 16, 2001
Abstract
The most common form of security interoperation is the use of COM+ security in .NET components. A lot of companies have COM+ components and applications that have been developed with no knowledge of .NET and managed code. Furthermore, a lot of these components and applications are in production, which makes changing or updating these components difficult. Thus, newer .NET components have to be built keeping security interoperation with COM+ components in mind. In this article, the final part of the week long strand of security articles, Kaushal Sanghavi explores how managed and unmanaged components can use the same security settings to interoperate. He focusses on writing .NET components that can work with COM+ security.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

Although there are good reasons to start building all our applications in .NET, many companies still have a large investment in COM and COM+. Realizing this, Microsoft has made every effort possible to allow traditional COM+ unmanaged applications to seamlessly and transparently integrate with .NET components. This interoperability is made possible by thin managed wrappers called Runtime Callable Wrappers (RCW) and COM Callable Wrappers (CCW). These features have been covered in a previous article at http://www.csharptoday.com/content.asp?id=1514&WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0. While interop between components is very important, another important feature to keep in mind is the interoperation of security between COM+ and .NET applications. While both COM+ and .NET allow role based and declarative security schemes, the underlying implementations are totally different, and the interoperation is not transparent as in the case of RCWs and CCWs.

The commonest form of security interoperation would be the use of COM+ security in .NET components. A lot of companies have COM+ components and applications that were developed with no knowledge of .NET and managed code. Further, a lot of these components and applications are in production, which makes changing or updating these components difficult. Thus, the newer .NET components have to be built with COM+ security interoperation in mind. In this article, we will explore how managed and unmanaged components can use the same security settings to interoperate. We will focus on writing .NET components that can work with COM+ security.

Note that this article is based on Visual Basic 6.0, Visual Studio .NET Beta 2 and Windows 2000 (all versions).

COM+ Security

Before we discuss how COM+ security can be applied to .NET components, we need to briefly recap the main elements of COM+ security. (For more COM+ security information, follow the links at the bottom of the article). COM+ provides two main security checks for access to a COM+ application (by application, we mean both the COM+ library application and the server application) and the components housed within the application.

The first check is applied at the COM+ application level as shown in the following screen shot. This controls whether or not security in general is enabled for components within the application, and whether the access checks are performed only at the process level, or at the process and component level. The authentication and impersonation levels are also controlled at the application level.

If security checking is enabled, the second check checks role based security at every access to the component. For example, in the following screen shot, only users who belong to the Managers role will be allowed to access the CSharpBank.Account component.

These security settings can be made by an administrator via the Component Services administration tool, or programmatically with the COM+ admin objects.

Installing a .NET Component as a COM+ Application

Managed components that run as COM+ applications, or want to take advantage of COM+ services, are called Serviced Components. These serviced components have to inherit from the ServicedComponent class in the System.EnterpriseServices namespace.

Using System.EnterpriseServices;

This namespace contains classes and attributes representing various COM+ services. In this article, we will focus on the security related capabilities of the System.EnterpriseServices namespace.

The implementation of COM+ security in .NET can be split into two broad areas - configuring the security settings, and impersonating users and roles defined in COM+ to call traditional COM+ applications. In this section, we configure security settings on a COM+ application.

The System.EnterpriseServices namespace defines the following attribute classes, corresponding to the security settings in COM+:

  • ApplicationAccessControlAttribute: this controls the application level settings - such as whether security should be enabled, the level of security to be applied, and the authentication and impersonation levels. This is an assembly level attribute, and is applied to all components in the assembly.
  • ComponentAccessControlAttribute: this controls the component level access check. This attribute has a single boolean property representing whether access checks are applied or not at the component level. This is a class level attribute and is applied to a single component.
  • SecurityRoleAttribute: this controls the different roles that should be associated with this COM+ application. This is applied at the assembly level and controls all roles within the COM+ application. This class supports two properties, one for the description of the role, and another specifying whether all users belong to this role.

The following code block shows a sample class declaration using these attributes:

[assembly: ApplicationName("ManagedServicedApplication")]
[assembly: ApplicationActivation(ActivationOption.Server)]
[assembly: AssemblyKeyFile("..\\..\\ManagedServicedComponent.snk")]

[assembly: ApplicationAccessControl (
AccessChecksLevel = AccessChecksLevelOption.ApplicationComponent,
Authentication = AuthenticationOption.Packet,
ImpersonationLevel = ImpersonationLevelOption.Impersonate
)]

// specify assembly level attributes for roles
[assembly: SecurityRole("WroxReaders", SetEveryoneAccess = true)]
[assembly: SecurityRole("WroxAdministrators", SetEveryoneAccess = false)]

namespace ManagedServicedComponent
{
// specify component level access checks
[ComponentAccessControl (true)]
public class ManagedServicedComponent : ServicedComponent
{
}
}

The regsvcs utility can be used to import this managed component into a new COM+ application as shown below. Note that regsvcs requires the assembly to have a strong name. A strong name uses a key pair to uniquely identify the component. A strong name key pair can be generated from the command line using the sn utility as shown below:

sn -k ManagedServicedComponent.snk
regsvcs ManagedServicedComponent.dll

This creates a COM+ application as shown in the following screen shots:

The first security check:

The second security check:

By carrying out these steps, a managed component behaves in an identical manner to a traditional COM+ application. The COM+ runtime enforces the security settings that are specified by the managed component using the above attributes.

Accessing COM+ Roles from .NET Components

The second element of COM+ security is role based security. Both COM+ and .NET support role based security, where only users belonging to a specific role can access the component. Unfortunately, the implementation of role based security in COM+ is totally different to that in .NET.

This difference stems from the fact that COM+ runs on Windows 2000 (and later versions), and thus uses the underlying Windows security model for its implementation of role based security. The .NET runtime on the other hand, is a common runtime that runs on many different platforms (like Windows 9x, ME and Windows CE) and is completely independent of the underlying operating system's security support. As a result of this disparity, interoperation between COM+ and .NET roles cannot be done transparently, and when designing an application, one needs to decide which security scheme to use. While it is possible to use a mix of COM+ and .NET roles, it is advisable to pick a single common security scheme. (For further details on .NET role based security, please refer to http://www.csharptoday.com/content.asp?id=1671&WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0). If we use purely COM+ roles with serviced .NET components, we can leverage existing applications, while the advantage of using solely .NET roles is that we obtain platform independence.

By deriving a managed class from the ServicedComponent class, the .NET runtime ensures that the COM+ security context is available to the managed object. The System.EnterpriseServices namespace contains a class called ContextUtil that represents the COM+ object context. This class supports methods for checking whether security is enabled, and for checking whether the accessing user belongs to a specific role. The following code shows how COM+ roles can be used to grant or deny access to specific users:

public string AdminMethod ()
{
string lReturnVal = "Access to AdminMethod GRANTED!";
if (ContextUtil.IsSecurityEnabled)
if (!ContextUtil.IsCallerInRole ("WroxAdministrators"))
lReturnVal = "Access to AdminMethod DENIED!";
return lReturnVal;
}

In addition to the object context, the System.EnterpriseServices namespace also contains a class, SecurityCallContext, which represents the call context of the current call. The call context represents all the callers (different assemblies can be running under different identities) in the current call stack. The following code block shows how the SecurityCallContext class can be used to obtain the user name of the original caller in the chain. Since identities can change in a call chain (i.e. assembly A runs under a user identity, and calls assembly B that is configured to run under a different user identity), the SecurityCallContext also offers a property called DirectCaller to get the identity of the direct caller in the chain. While these account names can be used in code directly, hard coding of individual users is not recommended; developers should use roles instead.

public string GetAccessingUserName ()
{
string lReturnVal = "Security needs to be enabled before calling this method.";

if (ContextUtil.IsSecurityEnabled){
SecurityCallContext lCallContext =
SecurityCallContext.CurrentCall;
lReturnVal = lCallContext.OriginalCaller.AccountName;
}
return lReturnVal;
}

These methods can be called from a VB 6.0 client, using COM interop, as shown below:

Private Sub btnAdmin_Click()

Dim lSC As New ManagedServicedComponent.ManagedServicedComponent
MsgBox lSC.AdminMethod(), , "Security Interop"
Set lSC = Nothing
End Sub

Private Sub btnReader_Click()

Dim lSC As New ManagedServicedComponent.ManagedServicedComponent
MsgBox lSC.ReaderMethod(), , "Security Interop"
Set lSC = Nothing
End Sub

Private Sub btnUser_Click()

Dim lSC As New ManagedServicedComponent.ManagedServicedComponent
txtUserName.Text = lSC.GetAccessingUserName()
Set lSC = Nothing
End Sub

Calling COM+ components using Impersonation

By default, .NET role based security is based on principals and identities that are associated with every thread in an appdomain (for a further explanation of role based security, see this article from earlier in the week). When Windows based users are in place, the roles that a user belongs to is determined by the user's Windows group, like Administrators, Power Users, etc. COM+ roles, on the other hand, are user defined and members of any given role are user accounts on a Windows domain. Thus it is possible that, if a .NET scheme is not based on Windows accounts, it will not be able to access any COM+ applications that have role based security enabled.

There is a way around this however. If we impersonate the Windows users that belong to the COM+ roles, we will be allowed access to the component. Since impersonation is tied very closely to the specific operating system's implementation of security, the CLR does not provide a direct way of impersonating other users. However, by using COM interop, we can call the Win32 API LogonUser and pass it the credentials of the user that we need to impersonate. If the impersonation succeeds, we are returned a user token, which can then be used with the WindowsIdentity class in the System.Security.Principal namespace to obtain a Windows identity for the impersonated user. We can now make calls on the COM+ application based on the impersonated user's identity.

The following code block shows how we can use the LogonUser and WindowsIdentity classes to impersonate users:

private const int LOGON32_LOGON_NETWORK = 3;
private const int LOGON32_PROVIDER_DEFAULT = 0;

[DllImport("ADVAPI32")]
private unsafe static extern int LogonUser (
String lpszUsername,
String lpszDomain,
String lpszPassword,
int dwLogonType,
int dwLogonProvider,
out IntPtr phToken
);

private void btnImpersonate_Click(object sender, System.EventArgs e)
{
IntPtr lToken;
int lRetVal = 0;

lRetVal = LogonUser (txtUserName.Text, txtDomain.Text, txtPwd.Text,
LOGON32_LOGON_NETWORK, LOGON32_PROVIDER_DEFAULT, out lToken);

if (lRetVal == 0){
//handle login error here
return;
}

WindowsIdentity lImpIdentity = new WindowsIdentity(lToken);
WindowsImpersonationContext lImpContext = lImpIdentity.Impersonate ();

//call unmanaged COM+ components here
//UnmanagedComponent lUnmanagedComponent = new UnmanagedComponent();
//lUnmanagedComponent.UnmanagedAdminMethod ();

lImpContext.Undo ();
}

We pass in the user name, password and domain of the user we want to impersonate, and the LogonUser returns a token that is used to create a new WindowsIdentity object. We then call the Impersonate method on this object to attach the new token to the current thread. Once the impersonate calls succeeds, we can call the unmanaged COM+ components that will authorize us based on the impersonation. Once we have finished the unmanaged execution, we call Undo on the impersonation context object, reverting our identity back to the original state.

In order for the LogonUser call to work, we need a special privilege known as SE_TCB_NAME. This corresponds to the "Act as part of the operating system" privilege that must be available to us. This can be checked by viewing the local security policy settings (found under Control Panel | Administrative Tools | Local Security Policy | Local Policies | User Rights Assignment) as shown below. If this privilege is not available to the user, the LogonUser call fails and impersonation cannot be accomplished.

Important note to the readers! The SE_TCB_NAME privilege is a very important permission that allows a user to carry out certain low level methods - it would allow a hacker to have uncontrolled access to the system. This privilege is turned off by default (even for the Administrator account), and most domain policies of companies do not allow users to turn this on. Even on our test machines, we could get this to work only by removing the machine from the domain. As a result, readers are strongly discouraged from turning this on. The code associated with this is fairly simple, and we recommend that readers understand the notion of impersonation, and use this only when all other options are been exhausted. Developers should either install .NET components as COM+ applications and then use the MMC to configure roles (thereby eliminating the use of .NET roles completely), or port the COM+ applications over to .NET and use .NET roles (thereby eliminating the use of COM+ roles completely).

Attached Sample Code

The attached sample code contains 3 projects, ManagedClient, ManagedServicedComponent, and VB60UnmanagedClient. The ManagedServicedComponent and VB60UnmanagedClient projects deal with using COM+ security and roles from managed and unmanaged objects, and the ManagedClient project deals with impersonation.

The following screen shot shows a simple unmanaged client calling methods on a managed serviced component. Depending on the configured roles, access to the Admin and Reader methods may or may not be allowed.

Please follow the steps below to setup the sample projects.

  • Compile the ManagedServicedComponent C# project to generate the assembly DLL.
  • Run the registration tool on the managed assembly as:
  • regsvcs ManagedServicedComponent.dll
  • In the Component Services tool, right click on ManagedServicedComponent.ManagedServicedComponent and select Properties. In the Security tab, enable the checkboxes corresponding to the roles WroxAdministrators and WroxReaders. This will allow access to the component from users belonging to both these roles.

Conclusion

Role based security is a very powerful and extensible way to control access to parts of the system based on user roles. While both COM+ and .NET support role based security, the underlying implementations for these two are totally different. In cases where existing COM+ components need to be leveraged, COM+ and .NET components can interoperate easily as we have shown in this article. When designing applications, developers should determine which scheme offers the best option for their individual needs.

Happy .NETing.

If you'd like to discuss a broader range of C#Today topics and issues, or if you have any C# related questions you need answers to, please visit our online resource centres at : http://p2p.wrox.com/list.asp?list=csharptoday_discuss&WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0 and http://p2p.wrox.com/list.asp?list=c_sharp&WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0


Creating a .NET Windows Installer – Part 2

Creating a .NET Windows Installer – Part 2
http://www.csharptoday.com/content.asp?id=1741
Article Rating: 4
Published on November 2, 2001
Abstract
In the first part of this series, Matthew MacDonald described how to create a simple setup program with Visual Studio .NET that could copy .NET runtime files, configure the registry, retrieve user information, and apply simple serial key validation. In this second article, he considers some more advanced techniques, and examine how they might be used in a database-based incident logging application. Some of the techniques covered include registering a custom file type, using launch conditions and using a custom add-in application to perform additional setup tasks.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

The first part of this series described how to create a simple setup program with Visual Studio .NET that could copy .NET runtime files, configure the registry, retrieve user information, and apply simple serial key validation. In this second article, we consider some more advanced techniques, and examine how they might be used in a database-based incident logging application.

The techniques you'll see in this article include:

  • Using variables and making setup actions conditional.
  • Registering a custom file type.
  • Using a custom add-in application to perform additional setup tasks (in this case, database connection configuration or database creation).
  • Using launch conditions.

For some background information about Windows Installer (MSI) technology and the setup designers provided in Visual Studio .NET, refer to the first article in this series (http://www.csharptoday.com/content.asp?id=1665&WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0), or see the white papers at http://www.installshield.com/?WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0.

The Incident Logger Application

In this article we create an installer for the C# program shown below:

This simple, one-window application is used to track customer bug reports and observed problems in the field. It provides a series of controls for incident information, and it commits a record with the combined information when the user clicks the Report button.

private void cmdReport_Click(object sender, System.EventArgs e)
{
if (txtDescription.Text == "" | cboIncident.Text == "" |
cboReporter.Text == "" | cboProduct.Text == "")
{
MessageBox.Show ("Fill out all the required fields.",
"Missing Information", MessageBoxButtons.OK,
MessageBoxIcon.Exclamation);
}
else
{
this.Cursor = Cursors.WaitCursor;

try
{
DataSet ds = GetDataSetSchema();
DataTable dt = ds.Tables["Incidents"];
DataRow dr = dt.NewRow();

// In the database, these index numbers correspond to
// descriptive rows in other tables.
// In this simple example, we hard-code the information
// instead of using table relations.
dr["IncidentType"] = cboIncident.SelectedIndex;
dr["ReportedBy"] = cboReporter.SelectedIndex;
dr["Product"] = cboProduct.SelectedIndex;
dr["Description"] = txtDescription.Text;
dr["DateReported"] = DateTime.Now;

dt.Rows.Add(dr);
SaveData(ds);

lblStatus.Text = "Record Inserted Successfully";
this.Cursor = Cursors.Default;
}
catch (Exception err)
{
this.Cursor = Cursors.Default;
MessageBox.Show(err.Message, "Data Access Error",
MessageBoxButtons.OK, MessageBoxIcon.Error);
}
}
}

This code uses the standard ADO.NET DataSet object to record the newly entered information. Two special methods, GetDataSetSchema and SaveData are used to handle the transfer of information between the DataSet and the underlying data source. This detail is important, because the Incident Logger application can work in one of two modes. The mode is determined based on registry setting that are exposed through a custom property procedure in the form class.

enum Mode
{
Local = 0,
Network = 1
}

private Mode ConnectionMode
{
get
{
RegistryKey rk;
rk = Registry.LocalMachine.OpenSubKey(
"Software\\ProseTech\\IncidentLogger");
if (rk == null)
{
return Mode.Local;
}
else
{
return (Mode) rk.GetValue("Mode");
}
}
}

Network Mode

In network mode, the application connects directly to a centralized database, using the connection string retrieved from the registry. Database access is performed using the ADO.NET classes in the .NET framework. Network mode would typically be used by on-site employees logging information gained from customer calls.

Local Mode

The Incident Logger also provides a local mode option that is ideal for field engineers and support workers at customer sites. In local mode, the application retrieves an incident.xsd file that defines the schema for the database. (This XSD would have been created based on the network database, probably using the DataSet.WriteXmlSchema method). This schema is used to create the appropriate DataSet object, and the data-filling code works the exact same as it does in network mode. The difference is that this information is written out to a simple XML file instead of a relational data source. A unique filename is generated using the current date and time. This information could later be processed and added to the centralized database with a simple utility.

The Data Access Code

The Incident Logger examines the registry, determines the mode, and takes the appropriate actions in the GetDataSetSchema and SaveData methods.

private DataSet
            GetDataSetSchema()
{
// Check the registry for the operating
            mode.
if (ConnectionMode == Mode.Network)
{
SqlConnection
            con = new SqlConnection(ConnectionString);
con.Open();
string
            connect = "SELECT * FROM Incidents"
SqlDataAdapter adapter = new
            SqlDataAdapter(conect, con);
DataSet ds = new
            DataSet();
adapter.FillSchema(ds, SchemaType.Mapped,
            "Incidents");
con.Close();
return
            ds;
}
else
{
DataSet ds = new
            DataSet();
ds.ReadXmlSchema(Application.StartupPath +
            "\\incident.xsd");
return ds;
}
}

private void
            SaveData(DataSet ds)
{
// Check the registry for the operating
            mode.
if (ConnectionMode == Mode.Network)
{
SqlConnection
            con = new SqlConnection(ConnectionString);
con.Open();
string
            connect = "SELECT * FROM Incidents"
SqlDataAdapter adapter = new
            SqlDataAdapter(connect, con);
SqlCommandBuilder comBuilder = new
            SqlCommandBuilder(adapter);
adapter =
            comBuilder.DataAdapter;
adapter.Update(ds,
            "Incidents");
con.Close();
}
else
{
string filename =
            DateTime.Now.ToString("yyMMdd-HHmmss");
ds.WriteXml(Application.StartupPath
            + "\\" + filename + ".ilog");
}
}

The installer must take these two different modes into consideration, as each mode has its own distinct setup requirements.

The Database

The database is hosted by SQL Server, and uses a single table (Incidents), which is shown below.

The full source code for this application is included with the download for this article.

The Basic Setup Project

To start, we add a basic setup project to the solution that contains the Incident Logger. Various properties can be set for the setup project, including the name of the product, the company that developed it, and the title of the setup application.

The first task is to use the Visual Studio .NET's File System designer to add the project output for the Incident Logger to the Application directory. The incident.xsd database schema file is also required.

We must also explicitly exclude the .NET installer files in order to create a smaller setup. This setup will only run on users that already have the .NET framework. (As we saw in the previous article, we don't need to take this step. However, because the .NET framework is still in late beta, it's best to install .NET on a computer using the distributable SDK available from http://www.gotdotnet.com/?WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0). Two Incident Logger shortcuts are added: one to the Programs Incident Logger folder in the Start menu, and one to the desktop.

Setup Conditions

Setup conditions provide a simple, generic way to customize a standard setup. Most setup actions, including registry settings, file copy operations, and custom actions, can be made conditional.

A setup condition is a simple variable expression that can evaluate to true or false. For example, the setup expression PhysicalMemory => 64 evaluates to true as long as the computer has at least 64 megabytes installed.

Built-In Variables

A conditional expression can use one of the built-in variables that every setup project provides. (In the first part of this article, we used these built-in variables to create a registry setting that recorded the current user's name). Some of the most important built-in variables are listed below.

Variable Name | Description
Version9X, VersionNT, ServicePackLevel, WindowsBuild, and MsiNetAssemblySupport | Retrieves information about the operating system version. MsiNetAssemblySupport is set to the version of the fusion.dll file, provided the current computer has the .NET framework installed.
SystemLanguageID | The language identifier for the operating system.
PhysicalMemory | The number of megabytes of installed (non-virtual) RAM memory.
ComputerName, LogonUser, and AdminUser | Identity information about the current user and the computer. The AdminUser property is only set if the currently logged in user is an administrator.
COMPANY, and USERNAME | Values retrieved from the user information window.
Author, Manufacturer, ProductName, etc. | Retrieves the setup property information you have set.

Choice Windows

The built-in variables only provide a few specific pieces of information, and are of limited use for the Incident Logger. In this case, a user selection variable, which retrieves a piece of information from the setup user, is more useful. To create such a variable, you need to add a choice window to the setup using the User Interface designer.

Each choice window is based on a single type of control. The different versions of each window (A, B, and C) are identical, but each window can only be used once in a setup project. (Clearly Windows Installer files do not use .NET class-based technology!). This limited range of options is designed to encourage consistency and prevent long installations with dozens of user setting options. Wherever possible, you should default to a reasonable setting, and not require a decision at installation time, when the user typically knows very little about the application.

The Incident Logger uses a two-button Radio Button window with the settings shown below. Other choice window allow you to set variables based on checkbox selections or text entries.

Property Name | Value
BannerText | "Connection Mode"
BodyText | "Choose the type of data access that Incident Logger will use on your computer."
ButtonProperty | CONNECTIONMODE
Button1Label | "Local mode (used for offsite reporting)"
Button2Label | "Network mode (used for on-site reporting with network access to the central database)"
Button1Value | 0
Button2Value | 1
DefaultValue | 0

This window creates a setup variable called CONNECTIONMODE. Depending on which radio button the user selects, a different numeric value will be set for the CONNECTIONMODE variable. (The values chosen in this example represent the Mode enumeration in the Incident Logger program). Choice window variables are always uppercase.

You can use this setup variable at other points in your setup. Remember, if you want to make a setup operation conditional, it must occur after you prompt the user for the corresponding information.

Making Setup Operations Conditional

To use this variable, you can assign it to a Condition. Conditions are provided as properties to various setup elements and tasks, including:

  • Folders and files
  • Registry entries
  • Custom actions

User interface dialogs, shortcuts, and file type registration cannot be made conditional.

For example, the following conditional expression is applied to the incident.xsd schema file, so that it is only copied if the user has selected local mode. (In reality, you would probably install this file no matter what, in case the client computer needs to change between modes of operation).

The Incident Logger program also stores the CONNECTIONMODE value in the registry, so it can be retrieved by the application. Its possible numbers correspond to values in the CONNECTIONMODE enumeration.

Custom Actions

Custom actions allow you to run a utility at the end of an installation to handle additional configuration that may not be possible from inside the Windows Installer setup. This could include configuring a database, adding a user account, or setting up an application-specific configuration file.

In the Incident Logger setup, a custom action is used to run a .NET program that configures the database connection string (or even create the database, if needed).

Using the Custom Action designer, you can add an action to any installation phase. Right-click the appropriate phase, and choose Add Custom Action. Once again, you must choose the program from the files included in your setup project (as there is no guarantee that any other files will be available).

The DBConfig custom action has a Condition property of CONNECTIONMODE=1 so that it is only activated for database (network mode) installation. If needed, additional information (like setup variables) can be passed in the CustomActionData property.

Custom actions are useful because MSI installers provide relatively few options for customization (for example, you can't code specific UI and special file operations in a setup project). By using a custom action, you have access to all the features of standard .NET programs. A more sophisticated configuration utility would use a wizard to walk through all the different required components of a connection string, and create it dynamically.

File Registration

Our Incident Logger also registers its file type (.ilog) so that double clicking on a log file automatically opens the application. To register a file extension, you use the File Types designer, which allows you to associate your application with a file extension. To add a file type association, right-click in the window and choose Add File Type. You will need to specify the following pieces of information in the Property Window:

  • The Name of this type of document. This is used for descriptive purposes only.
  • The associated file Extension. File extensions are not restricted to three characters.
  • The program to launch for this extension (Command). You must choose the application from one of the files provided in your setup project.
  • An Icon, which must also be part of your setup project.
  • A brief (two or three word) Description that identifies the format (e.g.SampleApp Profile).

After you have added a File Type, you need to add at least one action to make use of it. Actions appear in the right-click context menu, and one action can be set to be the default, in which case it appears in bold in the setup window. The default action is invoked when the user double-clicks on the file. Typically, the Open action is used for the default action.

By convention, the document filename is passed to your program, and it is your program's responsibility to check these command-line arguments and take the appropriate action (in this case, opening the file). In the Properties Window, the "%1" symbol in the Arguments property indicates that the action passes the file name to you.

You can also add other actions. For example, you could add a command named "Insert In DB" that would automatically copy a local-mode log file into the database. The log files could be emailed to an administrator who would use this feature. To add this feature, alter the Arguments property to some other format that your program can recognize. For example, you could set it to insert "%1" and specifically check in your program for the insert command-line parameter.

Remember, you can check for command-line arguments by accepting an array of strings in your Main method. (.NET automatically separates the command-line string into separate strings based on spaces).

static void Main(string[] args) {
//
            Process command-line arguments.
foreach (string arg in
            args)
{
if (arg == "insert")
{
// Set a class property
            or variable here.
}
}

// Run app.
}

Launch Conditions

One other useful addition is a launch condition. A launch condition is a special type of condition that is evaluated when the setup is first executed. It allows you to specify ingredients a setup must have. If these conditions are not met, the setup will be automatically cancelled, and the error message you define will be displayed. Because launch conditions occur at the beginning of a setup before any dialogs have been displayed, they cannot use user variables.

A launch condition has two parts.

  • First, there is a search operation that looks for a registry, file, or an installed component that you identify. A Boolean true or false is set using the corresponding property. You can search a specific location (path or key), or you can search the entire registry or file system. A setup variable is set based on the results.
  • Second, there is a Launch condition that either allows the setup to continue, or stops it with an error based on the variable. You just set the Condition property to the appropriate variable name, and specify the error Message.

For example, the Incident Logger uses a registry search that attempts to determine is a pre-.NET version of the application is installed. If it is, the setup is halted. (Note that the setup program automatically detects if you have already used the current setup, and prompts you to repair or remove the program).

This launch condition has two components:

  • A registry search for SOFTWARE\ProseTech\IncidentLogger\Version, which sets the variable FOUNDPREVIOUS to True if successful.
  • A launch condition that allows the setup to continue only if FOUNDPREVIOUS = False.

Custom actions always occur at the end of a setup. If a custom action fails, or the application throws an exception, then the installation will be rolled back.

Conclusion

In this article we considered some of the advanced aspects of Visual Studio.NET setup projects, like variables, conditions, and custom actions. These features combine to create an extensible framework that can take you beyond the simple setup designers. Additional information about the Windows Installer technology can be found in the Windows documentation (for example, online at http://www.microsoft.com/windows2000/en/professional/help/sag_WinInstall_Using_topnode.htm?WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0) and in the MSDN help installed with the beta release (ms-help://MS.NETFrameworkSDK/cpguidenf/html/cpcondeployingcommonlanguageruntimeapplicationusingwindowsinstaller.htm and ms-help://MS.VSCC/MS.MSDNVS/vsintro7/html/vbconWhatYouNeedToKnowAboutMicrosoftWindowsInstaller.htm).

If you'd like to discuss a broader range of C#Today topics and issues, or if you have any C# related questions you need answers to, please visit our online resource centres at : http://p2p.wrox.com/list.asp?list=csharptoday_discuss&WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0 and http://p2p.wrox.com/list.asp?list=c_sharp&WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0


Creating a .NET Windows Installer – Part 1

Creating a .NET Windows Installer – Part 1
http://www.csharptoday.com/content.asp?id=1691
Article Rating: 4.5
Published on November 1, 2001
Abstract
If you’re developing an application that you want to distribute to multiple users (or sell as package) you need a convenient, automated approach. Visual Studio .NET provides this with its native support for Windows Installer (MSI) projects. These projects can be used to make setup programs that selectively copy files, create shortcuts and registry settings, and incorporate user input. In this article, Matthew MacDonald considers how to use VS.NET to build a simple setup program for a .NET desktop application.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

There's been a great deal of discussion about .NET's new "zero-touch" or XCopy deployment features. However, if you're developing an application that you want to distribute to multiple users (or sell as package), you still need a more convenient, automated approach. Visual Studio .NET provides this solution with its native support for Windows Installer (MSI) projects. These projects can be used to make setup programs that selectively copy files, create shortcuts and registry settings, and incorporate user input. In this article, we'll consider how to use Visual Studio .NET to build a simple setup program for a .NET desktop application.

Introducing the Windows Installer Service

Microsoft's Windows Installer (MSI) technology isn't specific to .NET. In fact, it's included with all new versions of the Windows operating system (including Windows 2000, ME, and XP) and is available as a separate download for Windows 98 and Windows NT 4. Some of its hallmarks include:

  • Support for a system-maintained installation database. Windows uses this database to uninstall software without damaging other applications.
  • Support for self-repair, which allows modified or deleted files to be restored automatically from a Windows Installer setup. This allows your application to recover if a curious user makes some reckless changes. All you need to do is right-click on the original setup file and choose Repair from the context menu.
  • Support for transactional installation. If a user cancels a setup, or if a setup fails halfway through, any changes that have been made are rolled back, leaving the computer in the exact same state that it was in before installation began.

Windows Installer files always have the extension .msi, which is registered to the Windows Installer service (typically C:\WINNT\System32\msiexec.exe). Though they look like ordinary installation programs, MSI files do not contain a traditional stand-alone setup program (.exe file). Instead, they rely on the installation service built into the Windows operating system. This service supplies a few pre-defined dialogs and a limited amount of preset configuration.

It's important to note that Windows Installer projects are not really.NET projects. They aren't coded in a .NET language, and they don't use the CLR. However, Visual Studio .NET can help you create Windows Installer projects based on .NET programs with ease, automatically taking care of assemblies and dependencies. Instead of writing installation scripts, you configure the options provided in the Visual Studio .NET designers and property windows.

The Sample Project

In this article we will create an installer for the .NET project shown below.

This simple application allows the user to choose a font from a drop-down list. The font is drawn onto the form surface in a Paint event handler. The form code (aside from the Windows designer region) is shown below:

public class main :
            System.Windows.Forms.Form
{
private void main_Load(object
            sender, EventArgs e)
{
InstalledFontCollection fonts = new
            InstalledFontCollection();
foreach (FontFamily family in
            fonts.Families)
{
lstFonts.Items.Add(family.Name);
}
}

private
            void main_Paint(object sender, PaintEventArgs e)
{
if
            (lstFonts.Text != "")
{
try
{
e.Graphics.DrawString(lstFonts.Text, new
            Font(lstFonts.Text,
50), Brushes.Black, 10,
            50);
statusBar.Panels[0].Text = "";
}
catch (Exception
            err)
{
statusBar.Panels[0].Text =
            err.Message;
}
}
}

private void
            lstFonts_SelectedIndexChanged(object sender, EventArgs e)
{
if
            (lstFonts.Text != "") this.Invalidate();
}
}

Creating the Windows Installer Project

You can create a Windows Installer project on its own, or you can add it to an existing solution. The latter solution is often better, because it guarantees that the source project and the setup are always conveniently linked together in the same solution. It's also the approach we use with the FontViewer application.

To start, open the existing project that requires the setup program. Right-click on the solution item in the Solution Explorer window, and choose Add New Project. Choose Setup Project from the Setup and Deployment Projects group.

Once you type in the name of the project, click OK.

You also need to configure the build options for your solution, otherwise every time you run the program from the IDE, both the source and the setup projects will be built. As setup programs typically take several minutes to build, this can be a very time consuming (and annoying) process.

To change this behavior: right-click on the solution item in the Solution Explorer, and then choose Properties. Choose Configuration Properties and clear the Build checkmark for the setup project.

Now the setup file will not be created automatically. To build it manually, right-click on the setup project and choose Build.

The Setup Designers

When you create a setup project, Visual Studio .NET provides you with several different designers, each of which can be used to configure one aspect of your setup program. To see all the designers that you can use, right click on the solution item in the Solution Explorer, and select View.

  • The File System designer allows you to copy files to the destination computer. You can use a pre-defined directory, or one of several "recognized" directory options.
  • The Registry designer allows you to add new keys and values to the destination computer's registry.
  • The File Types designer allows you to register a document extension with your application (so that double-clicking the document launches the program automatically). These settings could be set through the Registry designer, but it would involve more work.
  • The User Interface designer allows you to tweak the default user interface (which is provided automatically) or add your own custom dialogs.
  • The Custom Actions designer allows you to link to another program, which will launch at the end of the setup to perform extra configuration.
  • The Launch Conditions designer allows you to specify special conditions that must be met in order for a setup to run.

You can also set some basic options like Author, Manufacturer, ManufacturerURL, Title, ProductName, and Version from the project properties window. Most of these are descriptive settings used in the setup wizard or other Windows dialogs, like the Support Info window (which can be launched from the Add/Remove Programs window). You can also set the icon that represents your program in the list of currently installed applications (AddRemoveProgramsIcon).

Adding Files, Project Outputs, and Shortcuts

The first step in creating a setup is specifying what files you want to be copied to the target computer, using the File System designer. You can specify that files should be added to a system-recognized folder like the System folder, Fonts folder, the GAC (Global Assembly Cache), or even the user's My Documents folder. You can also add files to a hard-coded path (using the Custom Folder option) or to the directory the user selects for the application (the Application Folder). The full list of supported folder options is shown below:

The File System window is divided into two panes: a list of folders (on the left), and a list of files for the currently selected folder (on the right). You can add a new folder by right-clicking on the folder list, and choosing Add Special Folder.

Support Files

You can add the files your application uses (like bitmaps, custom data files, and so on) manually. Just right-click on the file list, choose Add File, and browse to the correct file (you can also create any sub-directories you need by choosing Add Folder.) The files you select will be copied to the corresponding location on the target computer.

Project Outputs

You should add the main .NET assembly for your application as a project output. This ensures that the file will be automatically regenerated every time you build the setup. It also ensures that the required dependent assemblies will be included. To add the project output for the FontView application, right-click on the Application folder and choose Add Project Output.

This window allows outputs from any project in the current solution. Once you choose the correct project, you can select the type of output.

Primary output adds the assembly built by the referenced project, and any dependent assemblies. This is the most common option. (You can also set the Configuration option to Release .NET to ensure that the setup program always uses the release version of the assembly, regardless of the current IDE settings).

Localized resources uses the satellite assemblies for the project, which contain locale-specific resources.

Source files includes all the text source files for the referenced project. Because these files do not include the actual compiled assembly, dependent assemblies will not be included automatically.

Any dependent assemblies are automatically referenced. In this case, the dependent assembly mscorlib.cfgmscorlib.dll (a core part of the .NET framework) is included.

When you add a .NET assembly, the .NET redistributable files are automatically added to the setup project, allowing you to install your application on computers that don't have the .NET runtime installed. It also balloons the size of your setup project (from about 100K to over 15Mb). If you are creating a setup for clients that already have the .NET framework installed, you should exclude these components to save space. This is the approach taken in the FontViewer example.

Right-click on the dotnetfxredist_x86_enu.msm merge module in the Solution Explorer, and choose Exclude.

Shortcuts

You can also add shortcuts. Typically, you will create these in the Desktop folder, the StartUp folder, the user's Favorites folder, or (most commonly) an application-specific sub-directory in the Programs menu.

To create a shortcut for the FontViewer application, you should follow these steps:

  1. Add the shortcut icon to the setup project.
  2. Create a folder for the application in the Programs menu (in this case, we call it FontViewer).
  3. Right-click on the new folder and select Create New Shortcut. A special window will appear prompting you to choose one of the files in the setup as the shortcut target.
  4. Once you choose the target, the shortcut will be added. You can rename it, and configure additional options using the Properties Window.

At this point, you have a fully functional simple setup. You can build it to a single .msi file, and use it to install the FontViewer application. A very simple interface will be used that provides a generic "welcome" window, prompts for a destination, shows a progress bar as it copies files and shortcuts, and ends with a generic message. You'll see how to modify some of these elements in the next section.

Customizing the User Interface

With the User Interface designer, you can see the windows that the user will see. Windows are listed in the order they will be presented, and sub-grouped by setup stage. You will see two versions of your setup listed, one for the normal user install, and one for the administrative install. You can access the administrative install by using the /a parameter. Usually, the administrative install is used if you need to provide a network setup.

To actually see what a window looks like, you need to actually run the setup, or refer to the MSDN help, which describes the properties and shows the appearance for each one. You cannot debug a setup project like you would a traditional application.

You can customize the user interface of your setup program in a number of ways:

  • Modify the options for a window. Each window provides a different set of options. To see what options are available, select the window, and examine the Properties Window.
  • Rearrange the order of windows by right-clicking on one and choosing Move Up or Move Down.
  • Remove a window from the setup by right-clicking on it and selecting Delete.
  • Add a new window pre-defined to the setup by right-clicking on a setup state and selecting Add Dialog.

Window Types

You can add one of several pre-defined windows to the setup wizard. Each window can also only be added once, and then it disappears from the list of choices. Each window is designed for a single purpose, and provides very few configurable options. Almost all windows provide a BannerBitmap property that allows you to specify a graphic file that will be shown in the white banner area at the top of the window (the default graphic is a computer). Some of the options for other window types are shown below.

Window Type | Description/Properties
Splash | Used at the beginning of an install to display a company or product graphic. Provides a boolean Sunken property and a picture through the SplashBitmap property.
Welcome | An introductory window. Can display text information from the CopyrightWarning and WelcomeText properties.
License Agreement | Used to provide a license information window that requires the user to select " I agree " before continuing. The license information is drawn from the .rtf file specified by the LicenseFile property, and displayed in a window that can be Sunken.
Customer Information | Provides a window that requires a customer name. You can also require additional information by using the boolean ShowOrganization and ShowSerialNumber properties. You can perform simple validation of the serial number using the SerialNumberTemplate, which is described in a section below.
Progress | Displays a progress bar, provided the ShowProgressBar setting is True.
Register User | Allows a user to register through a separate program. It provides a " Register Now " button that launches an executable or navigates to a web site. You specify the application through the Executable and Arguments properties.
Read Me | Displays information from the .rtf file specified by the LicenseFile property, in a window that can be Sunken.

The User Interface designer may come as a disappointment to most developers, because it allows relatively limited customization. You can add and rearrange pre-built window types, but you cannot design an entirely new window or add in a custom block of code. In this case, Microsoft places a greater premium on consistency and enforced simplicity than on flexibility.

The Serial Number Template

You can use the serial number template (the SerialNumberTemplate property in the Customer Information window) to prevent a user from installing your application without a valid serial number. The template also determines how the serial number is formatted. However, you cannot directly validate serial numbers. Instead, you must use the template-based validation routines built into the Windows Installer product, which may not be adequate for all applications.

To start, the SerialNumberTemplate is bracketed by "greater than" and "less than" signs (< and >) so the characters do not appear directly in the text box. You can also use the dash character to separate the serial number into more than one text box.

In the template itself, you can use the following characters:

  • # requires a digit, but does not perform additional validation.
  • ? requires an alphanumeric character, but does not perform additional validation.
  • ^ requires an uppercase character or a digit, but does not perform additional validation.
  • % requires a digit that will be used by the built-in validation algorithm.
  • ? requires an alphanumeric character that will be used by the built-in validation algorithm.

Any other character is treated as a literal (required) constant. The validation algorithm (for % and ?) is built into the Windows Installer service. It adds up all the digits, and divides them by 7. If the remainder is 0 validation succeeds, otherwise it fails.

The FontViewer setup uses the template <%%D-##-##>, which uses three textboxes. The last four characters must be digits, the third character must be D, and the sum of the first two digits must be divisible by 7. (A valid code is 77D-00-00).

Registry Settings

The last feature we will add to the FontView setup is an automatic registry entry. The Registry designer is modeled after the familiar regedit.exe program. You can browse to a node in the registry, and add keys or values by right-clicking and choosing the relevant option. These settings will be committed to the registry of the target computer. Note that you cannot remove existing entries, which prevents you from making potentially damaging changes.

For our registry setting, we use special variables recognized by the installer. These variables are identified by square brackets. At runtime, the setup swaps the appropriate values for the [Manufacturer] and [Product] (which are set in the Properties Window for the setup project) and the [CUSTOMERNAME] (which the user specifies in the registration window) to create the registry setting HKEY_LOCAL_MACHINE\ProseTech\FontViewer\Customer. Setup variables are described in more detail in the second part of this series.

To incorporate this change, the FontViewer application is modified slightly. It now displays the user name of the user who installed it in the title bar.

Here's the required code in the FontViewer application:

private void main_Load(object sender, EventArgs e)
{

// (Original code omitted for space.)

RegistryKey rk;
rk = Registry.LocalMachine.OpenSubKey(
"Software\\ProseTech\\FontViewer");
if (rk != null) this.Text += " - " + rk.GetValue("Customer");
}

Summary

In this article, you've seen how .NET incorporates Windows Installer packages. Additional information about the Windows Installer technology can be found in the Windows documentation (for example, online at http://www.microsoft.com/windows2000/en/professional/help/sag_WinInstall_Using_topnode.htm?WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0) and in the MSDN help installed with the beta version (ms-help://MS.NETFrameworkSDK/cpguidenf/html/cpcondeployingcommonlanguageruntimeapplicationusingwindowsinstaller.htm and ms-help://MS.VSCC/MS.MSDNVS/vsintro7/html/vbconWhatYouNeedToKnowAboutMicrosoftWindowsInstaller.htm).

This article explained the MSI fundamentals. In the second part of this series, we'll look at some advanced options including setup conditions, custom actions, and file type registration.


Visual J# .NET - a Technical and Political Introduction

Visual J# .NET - a Technical and Political Introduction
http://www.csharptoday.com/content.asp?id=1688
Article Rating: 4
Published on October 26, 2001
Abstract
Delevlopers have long held that C# is Microsoft's attempt at a Java 'clone'. However, C# has many features, such as operator overloading and structs, which derive from C++ rather than from Java. Now Microsoft have just released the Beta 1 of J#, and this really is an attempt to lure Java developers into the .NET world. Adrian Young and Julian Skinner discuss the potentials and pitfalls of this new language.
Article

When C# was first introduced, many immediately noticed the similarity with Java in both structure and syntax. Many a time at many a conference, the speaker would write some Java code, and then change it to C# by modifying just a few characters and replacing an import statement with the using equivalent, before compiling and running the application. However, as you delve deeper into C# and discover its linguistic power and flexibility, you will no doubt realise that it owes far more to C/C++ than it does to Java.

This article introduces a technology that is more similar to Java than C# can ever be - in fact it is syntactically almost identical to Java - Visual J# .NET. We will compare J# to C# and Java, and demonstrate porting existing Java code into J#. Finally we discuss some of the more contentious issues surrounding this new language. Our main conclusions are:

  • J# beta 1 is based on a very old implementation of Java - Java 1.1.4.
  • RMI, JNI, and applets are not supported.
  • J# compiles to IL, NOT Java bytecode.
  • There is potential confusion in ensuring the right types - .NET, or Java-simulant - are used.
  • Java code compiled with the J# compiler is neater than Java bytecode ported to IL using the jbimp.exe import tool.
  • J# is not so useful as a migration tool for Java developers, but it shows promise as a learning tool.

What is J#?

Visual J# .NET is Microsoft's latest addition to the family of languages supported by the .NET Framework. It is available as a download at http://download.microsoft.com/download/VisualJSharpNet/Install/1.0/NT45XP/EN-US/VJSharpSetup.exe?WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0, but will not be shipped with the earliest versions of the .NET Framework or Visual Studio .NET. J# is a compiler and set of runtime libraries that provides a syntax and functionality Java developers should find familiar. But instead of compiling to Java bytecode, the compiler compiles to Intermediate Language. The libraries also are not the complete set of libraries you will find in a Java runtime - and certainly not in a modern implementation of the Java 2 platform. We gain access to the WFC (Windows Foundation Classes) which are the adaptations of standard Java packages (based on JDK level 1.1.4), and various com.ms.* packages. Visual J# .NET also includes a tool which converts the bytecode produced by compiling standard Java code (.class files) into IL, thereby enabling us to include a subset of standard Java functionality in our applications.

What Visual J# .NET applications cannot do is run on a JVM - it is a purely .NET Framework technology. Neither can we use any version of the JDK newer than JDK 1.1.4, which means we cannot use any of the more up-to-date functionality offered by Java, such as Swing. We cannot create or deploy applets on the browser, and there is no support for the Java Native Interface (JNI) and Remote Method Invocation (RMI).

Here is a summary of some of the things we can do with Visual J# .NET:

  • Create console applications
  • Create Windows Forms applications
  • Create Web Forms applications
  • Develop ASP.NET Web Services

Being fully integrated with the Visual Studio .NET IDE, the Visual J# .NET installation offers several AppWizards to help us develop these applications and more.

Installation

The beta 1 of Visual J# .NET comes as a free download from:

http://download.microsoft.com/download/VisualJSharpNet/Install/1.0/NT45XP/EN-US/VJSharpSetup.exe?WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0.

Double-click on the EXE once it is downloaded, and Visual J# is installed at this path (if you don't specify another):

C:/Program Files/Microsoft Visual J# .NET

This folder has several subfolders containing all that we need to build Visual J# .NET applications. The more interesting subfolders are:

  • Framework/bin - contains the J# compiler (jc.exe) and the bytecode to IL converter (jbimp.exe) as well as other tools.
  • Framework/sdk - contains the class libraries, WFC and the com.ms.* binaries.
  • Help - holds all the help files which are automatically integrated with the Visual Studio .NET Help facility, though you have to set the "Filtered by" option to "no filter" to see them.

The Visual J# .NET Compiler

As its name suggests, Visual J# .NET is meant to be run from within Visual Studio .NET and subsequent code examples will use this IDE. However, if we wish, we can create J# applications from the command line using jc.exe, whose path is automatically added to the system path on installation.

There are three types of binary that we can create - a normal EXE, a Windows EXE, and a library (DLL); the switches are the same as for C# compilation:

  • /target:exe or /t:exe - to create a normal EXE file (the default)
  • /target:winexe or /t:winexe - to create a Windows EXE
  • /target:library or /t:library - to create a DLL file

Note that there is no /t:module option or a corresponding /addmodule:MyModule.dll switch.

References to external libraries are made as follows:

/ reference:MyLibrary.dll or /r:MyLibrary.dll

You can specify your output file as follows:

/out:MyFile.exe

You can also switch debugging on and off, set the warning level, and carry out recursive compilation. However you cannot create XML documentation, include a resource in your binary file, or carry out optimized or incremental compilations.

Here is an example command line compilation:

C:\J#>jc /t:exe /r:MyLibrary.dll /debug+ /warn:0 /out:MyEXE.exe Test.jsl

Here we are creating a normal EXE and referencing a DLL, with debugging turned on and warnings turned off and specifying a name for the output file. Test.jsl is the source file.

Note that the extension for J# files is .jsl.

For more information on the Visual J# .NET command line options, see the Visual J# .NET documentation.

Comparing C# and J#

Having briefly covered the J# compiler, we turn our attention to creating Visual J# .NET applications using Visual Studio .NET. The aim of this section is to compare and contrast the syntax of both C# and J#, by creating two functionally equivalent applications, one in C# and the other in Visual J# .NET, and pointing out the differences between the code.

The example application is nothing special. All it does is link up to SQL Server's Northwind database and display the first and last names of each entry in the Employees table in a simple Windows Form:

The data itself is accessed by a SqlDataReader object, and the Next button reads in the next item of data from the table. The Refresh button recreates the data reader object and redisplays the first record.

The code for the C# version of the form looks like this:

using System;
using
            System.Drawing;
using System.Collections;
using
            System.ComponentModel;
using System.Windows.Forms;
using
            System.Data;
using System.Data.SqlClient;

namespace
            CSDataReader
{
/// <summary>
/// Summary description
            for CSDataReader.
/// </summary>
public class
            CSDataReader : System.Windows.Forms.Form
{
private
            System.Windows.Forms.Label label1;
private
            System.Windows.Forms.Label label2;
private
            System.Windows.Forms.TextBox firstName;
private
            System.Windows.Forms.TextBox lastName;
private
            System.Windows.Forms.Button buttonNext;
private
            System.Windows.Forms.Button buttonRefresh;

///
            <summary>
/// Required designer variable.
///
            </summary>
private System.ComponentModel.Container
            components = null;

// SQL objects
private SqlConnection cn
            = null;
private SqlCommand cmd = null;
private SqlDataReader
            dr = null;

public CSDataReader()
{
//
// Required
            for Windows Form Designer
            support
//
InitializeComponent();

GetData();
//
//
            TODO: Add any constructor code after InitializeComponent
            call
//
}

/// <summary>
/// Clean up any
            resources being used.
/// </summary>
protected override
            void Dispose( bool disposing )
{
// Clean up SQL
            objects
dr.Close();
cn.Close();

if( disposing
            )
{
if (components != null)
{
components.Dispose();
}
}
base.Dispose( disposing
            );
}

#region Windows Form Designer generated code
///
            <summary>
/// Required method for Designer support - do not
            modify
/// the contents of this method with the code
            editor.
/// </summary>
private void
            InitializeComponent()
{
this.label1 = new
            System.Windows.Forms.Label();
this.label2 = new
            System.Windows.Forms.Label();
this.buttonNext = new
            System.Windows.Forms.Button();
this.lastName = new
            System.Windows.Forms.TextBox();
this.firstName = new
            System.Windows.Forms.TextBox();
this.buttonRefresh = new
            System.Windows.Forms.Button();
this.SuspendLayout();
//
//
            label1
//
this.label1.Location = new System.Drawing.Point(8,
            24);
this.label1.Name = "label1";
this.label1.Size = new
            System.Drawing.Size(64, 23);
this.label1.TabIndex =
            3;
this.label1.Text = "First Name:";
this.label1.TextAlign =
            System.Drawing.ContentAlignment.MiddleLeft;
//
//
            label2
//
this.label2.Location = new System.Drawing.Point(8,
            64);
this.label2.Name = "label2";
this.label2.Size = new
            System.Drawing.Size(64, 16);
this.label2.TabIndex =
            4;
this.label2.Text = "Last Name:";
this.label2.TextAlign =
            System.Drawing.ContentAlignment.MiddleLeft;
//
//
            buttonNext
//
this.buttonNext.Location = new
            System.Drawing.Point(192, 112);
this.buttonNext.Name =
            "buttonNext";
this.buttonNext.TabIndex =
            2;
this.buttonNext.Text = "Next";
this.buttonNext.Click += new
            System.EventHandler(this.buttonNext_Click);
//
//
            lastName
//
this.lastName.Location = new
            System.Drawing.Point(80, 64);
this.lastName.Name =
            "lastName";
this.lastName.Size = new System.Drawing.Size(192,
            20);
this.lastName.TabIndex = 1;
this.lastName.Text =
            "";
//
// firstName
//
this.firstName.Location = new
            System.Drawing.Point(80, 24);
this.firstName.Name =
            "firstName";
this.firstName.Size = new System.Drawing.Size(192,
            20);
this.firstName.TabIndex = 0;
this.firstName.Text =
            "";
//
// buttonRefresh
//
this.buttonRefresh.Location
            = new System.Drawing.Point(104, 112);
this.buttonRefresh.Name =
            "buttonRefresh";
this.buttonRefresh.TabIndex =
            5;
this.buttonRefresh.Text =
            "Refresh";
this.buttonRefresh.Click += new
            System.EventHandler(this.buttonRefresh_Click);
//
//
            CSDataReader
//
this.AutoScaleBaseSize = new
            System.Drawing.Size(5, 13);
this.ClientSize = new
            System.Drawing.Size(292, 165);
this.Controls.AddRange(new
            System.Windows.Forms.Control[]
            {
this.buttonRefresh,
this.label2,
this.label1,
this.buttonNext,
this.lastName,
this.firstName});
this.Name
            = "CSDataReader";
this.Text =
            "CSDataReader";
this.ResumeLayout(false);

}
#endregion

///
            <summary>
/// The main entry point for the
            application.
/// </summary>
[STAThread]
static void
            Main()
{
Application.Run(new
            CSDataReader());
}

private void buttonNext_Click(object
            sender, System.EventArgs e)
{
if (dr.Read())
{
// While
            there are entries to read, display data in text
            boxes
firstName.Text =
            dr["FirstName"].ToString();
lastName.Text =
            dr["LastName"].ToString();
}
}

private void
            GetData()
{
// Open connection
cn = new SqlConnection("data
            source=JULIANS2;initial catalog=Northwind;persist security
            info=False;user i" +
"d=sa;workstation id=MACBETH;packet
            size=4096");
cn.Open();

// Create SQL command
cmd =
            cn.CreateCommand();
cmd.CommandText = "SELECT LastName, FirstName
            FROM Employees";

// Create data reader
dr =
            cmd.ExecuteReader();

// Put details of first entry into the
            text boxes
DisplayFirstRecord();
}

private void
            DisplayFirstRecord()
{
dr.Read();
firstName.Text =
            dr["FirstName"].ToString();
lastName.Text =
            dr["LastName"].ToString();
}

private void
            buttonRefresh_Click(object sender, System.EventArgs e)
{
//
            Close original data reader
dr.Close();

// Recreate data
            reader adn display first record
dr =
            cmd.ExecuteReader();
DisplayFirstRecord();
}
}
}

Let's move onto the J# version. In Visual Studio .NET, select New | Projects | Visual J# Projects | Windows Application.

Proceed to fill out the form as you would for other projects. Dragging and dropping controls onto the form and setting their properties is exactly the same for J# as it is for C#. However the code that we get is quite different. Here it is in full:

package JSDataReader;

import
            System.Drawing.*;
import System.Collections.*;
import
            System.ComponentModel.*;
import System.Windows.Forms.*;
import
            System.Data.*;
import System.Data.SqlClient.*;

// Summary
            description for JSDataReader.
public class JSDataReader extends
            System.Windows.Forms.Form
{
private System.Windows.Forms.Label
            label1;
private System.Windows.Forms.Label label2;
private
            System.Windows.Forms.Button button1;
private
            System.Windows.Forms.TextBox firstName;
private
            System.Windows.Forms.TextBox lastName;
private
            System.Windows.Forms.Button buttonRefresh;
// Required designer
            variable.
private System.ComponentModel.Container components =
            null;

// SQL objects
private SqlConnection cn =
            null;
private SqlCommand cmd = null;
private SqlDataReader dr
            = null;

public JSDataReader()
{
//
// Required for
            Windows Form Designer
            support
//
InitializeComponent();

getData();
//
//
            TODO: Add any constructor code after InitializeComponent
            call
//
}

// Clean up any resources being
            used.
protected void Dispose(boolean disposing)
{
// Clean
            up SQL objects
dr.Close();
cn.Close();

if
            (disposing)
{
if (components !=
            null)
{
components.Dispose();
}
}
super.Dispose(disposing);
}

#region
            Windows Form Designer generated code
// Required method for
            Designer support - do not modify
// the contents of this method
            with the code editor.
private void
            InitializeComponent()
{
this.button1 = new
            System.Windows.Forms.Button();
this.label2 = new
            System.Windows.Forms.Label();
this.lastName = new
            System.Windows.Forms.TextBox();
this.label1 = new
            System.Windows.Forms.Label();
this.firstName = new
            System.Windows.Forms.TextBox();
this.buttonRefresh = new
            System.Windows.Forms.Button();
this.SuspendLayout();
//
//
            buttonNext
//
this.button1.set_Location(new
            System.Drawing.Point(((int)200),
            ((int)112)));
this.button1.set_Name("button1");
this.button1.set_TabIndex(((int)2));
this.button1.set_Text("Next");
this.button1.add_Click(
            new System.EventHandler(this.buttonNext_Click) );
//
//
            label2
//
this.label2.set_Location(new
            System.Drawing.Point(((int)8),
            ((int)64)));
this.label2.set_Name("label2");
this.label2.set_Size(new
            System.Drawing.Size(((int)64),
            ((int)23)));
this.label2.set_TabIndex(((int)4));
this.label2.set_Text("Last
            Name:");
this.label2.set_TextAlign(System.Drawing.ContentAlignment.MiddleLeft);
//
// lastName
//
this.lastName.set_Location(new
            System.Drawing.Point(((int)80),
            ((int)64)));
this.lastName.set_Name("lastName");
this.lastName.set_Size(new
            System.Drawing.Size(((int)192),
            ((int)20)));
this.lastName.set_TabIndex(((int)1));
this.lastName.set_Text("");
//
// label1
//
this.label1.set_Location(new
            System.Drawing.Point(((int)8),
            ((int)24)));
this.label1.set_Name("label1");
this.label1.set_Size(new
            System.Drawing.Size(((int)72),
            ((int)24)));
this.label1.set_TabIndex(((int)3));
this.label1.set_Text("First
            Name:");
this.label1.set_TextAlign(System.Drawing.ContentAlignment.MiddleLeft);
//
// firstName
//
this.firstName.set_Location(new
            System.Drawing.Point(((int)80),
            ((int)24)));
this.firstName.set_Name("firstName");
this.firstName.set_Size(new
            System.Drawing.Size(((int)192),
            ((int)20)));
this.firstName.set_TabIndex(((int)0));
this.firstName.set_Text("");
//
// buttonRefresh
//
this.buttonRefresh.set_Location(new
            System.Drawing.Point(((int)104),
            ((int)112)));
this.buttonRefresh.set_Name("buttonRefresh");
this.buttonRefresh.set_TabIndex(((int)5));
this.buttonRefresh.set_Text("Refresh");
this.buttonRefresh.add_Click(
            new System.EventHandler(this.buttonRefresh_Click) );
//
//
            JSDataReader
//
this.set_AutoScaleBaseSize(new
            System.Drawing.Size(((int)5),
            ((int)13)));
this.set_ClientSize(new
            System.Drawing.Size(((int)292),
            ((int)165)));
this.get_Controls().AddRange(new
            System.Windows.Forms.Control[]
            {this.buttonRefresh,
this.label2,
this.label1,
this.button1,
this.lastName,
this.firstName});
this.set_Name("JSDataReader");
this.set_Text("JSDataReader");
this.ResumeLayout(false);

}
#endregion

//
            The main entry point for the application.
/** @attribute
            System.STAThreadAttribute() */
public static void main(String[]
            args)
{
Application.Run(new
            JSDataReader());
}

private void buttonNext_Click
            (System.Object sender, System.EventArgs e)
{
if
            (dr.Read())
{
// While there are entries to read, display data
            in text
            boxes
firstName.set_Text(dr.get_Item("FirstName").ToString());
lastName.set_Text(dr.get_Item("LastName").ToString());
}
}

private
            void getData()
{
// Open connection
cn = new
            SqlConnection("data source=JULIANS2;initial
            catalog=Northwind;persist security info=False;user i"
            +
"d=sa;workstation id=MACBETH;packet
            size=4096");
cn.Open();

// Create SQL command
cmd =
            cn.CreateCommand();
cmd.set_CommandText("SELECT
            FirstName,LastName FROM Employees");

// Create data reader
dr = cmd.ExecuteReader();

// Put details of first entry
            into the text boxes
displayFirstRecord();
}

private
            void
            displayFirstRecord()
{
dr.Read();
firstName.set_Text(dr.get_Item("FirstName").ToString());
lastName.set_Text(dr.get_Item("LastName").ToString());
}

private
            void buttonRefresh_Click (System.Object sender, System.EventArgs
            e)
{
// Close original data reader
dr.Close();

//
            Recreate data reader adn display first record
dr =
            cmd.ExecuteReader();
displayFirstRecord();
}
}

And this is what the form will look like.

Note that it is identical to the C# form, though we have taken the trouble to rename it appropriately.

If we flip between the above code listing and the previous one, we see that they are substantially different. The reason we chose a Windows Form application (with its long code listings), is that in a console application, the differences between C# and J# are purely syntactic - that is, the only real differences between them would be the keyword variations and the capitalization (or otherwise) of function calls and object types. For a Windows Form application, however, the differences go well beyond syntax. There are certain features of C# syntax and structure that are not replicated in J#. For J# to maintain its syntactical likeness to Java, anything that Java itself doesn't have, J# doesn't have either.

First, the very top line of the J# source file:

package JSDataReader;

This defines the code as belonging to the JSDataReader package. In C#, the same kind of effect is achieved by wrapping the code in a namespace block:

namespace CSDataReader
{
...
}

Note that the J# package does not need curly brackets as everything in the file is automatically contained in the package, including the import directives. In C# code, using directives can lie either inside or outside the namespace block.

The next area of note is inside InitializeComponent(). As you will know, when we create a Windows Form control, we assign values to its properties. In C#, this is done by simple assignment:

//
// firstName
//
this.firstName.Location = new System.Drawing.Point(80,
            24);
this.firstName.Name = "firstName";
this.firstName.Size =
            new System.Drawing.Size(192, 20);
this.firstName.TabIndex =
            0;
this.firstName.Text = "";

However this does not work in J#, as J# does not support the property syntax familiar to C# programmers. Visual Studio .NET knows this, an creates the appropriate J# syntax as follows:

//
// firstName
//
this.firstName.set_Location(new System.Drawing.Point(((int)80),
            ((int)24)));
this.firstName.set_Name("firstName");
this.firstName.set_Size(new
            System.Drawing.Size(((int)192),
            ((int)20)));
this.firstName.set_TabIndex(((int)0));
this.firstName.set_Text("");

As we can see, this syntax is more explicit than C#'s. Instead of simple value assignment, we have to use explicit function calls. However, Visual Studio .NET has done far more work than it really needed to. The integer parameters passed to the Point and Size constructors and the value of TabIndex do not have to be explicitly cast to int or wrapped in a gratuitous second set of brackets. The code can just as easily be written like this:

//
// firstName
//
this.firstName.set_Location(new System.Drawing.Point(80
            ,24));
this.firstName.set_Name("firstName");
this.firstName.set_Size(new
            System.Drawing.Size(192,
            20));
this.firstName.set_TabIndex(0);
this.firstName.set_Text("");

However we cannot get around the need to call the set_ functions here as this is the only way to set property values in J#. Indeed, when we set these properties using code, the Visual Studio .NET Intellisense for Visual J# includes all the get_ and set_ accessor functions for each and every property in its list of class members:

While we can't use a simplified syntax for defining properties, we can achieve the same effect by defining a private variable and manually implementing public accessor functions to get and set the value of that variable.

Next we come to the Click events for the buttons. The familiar C# syntax for the Next button looks like this:

this.buttonNext.Click += new System.EventHandler(this.buttonNext_Click);

The corresponding J# code looks like this:

this.button1.add_Click( new System.EventHandler(this.buttonNext_Click) );

Once again we have a function call instead of the simplified += syntax. We use add_ functions to subscribe to events and corresponding remove_ functions to unsubscribe from events. Each and every add_ and remove_ function appears in the class member list when the Intellisense window pops up.

So J# has no simplified syntax for getting and setting properties values or for subscribing and unsubscribing to events. It therefore comes as no surprise to know that it does not support indexer syntax either. In C#, we get the data field from the data reader using the syntax shown in bold:

firstName.Text = dr["FirstName"].ToString();

In J#, the equivalent syntax is also shown in bold, to distinguish it from the surrounding code:

firstName.set_Text(dr.get_Item("FirstName").ToString());

The C# indexer syntax has been replaced by an explicit function call - get_Item(). If we look at the .NET documentation for the SqlDataReader class, we find that it has a property called Item, which cannot be accessed using C#, as the predefined indexer is used to get and set the data for the current column instead. For J#, which has no indexer syntax, the Item property is now accessible using a get_ function just like all other properties.

So we've examined the three main areas where the C# and J# code differs, but there is one more area worthy of note which is largely unrelated to what we have already discussed. When creating Windows Form applications using Visual Studio .NET, the entry point function has an attribute applied to it:

// The main entry point for the
            application.
[STAThread]
static void Main()
{
Application.Run(new CSDataReader());
}

The equivalent code for the J# file looks like this:

// The main entry point for the
            application.
/** @attribute System.STAThreadAttribute()
            */
public static void main(String[] args)
{
Application.Run(new JSDataReader());
}

So what's happening here? In the case of J#, the attribute looks like it has been commented out. In fact it is a comment, and this arises because Java does not have an equivalent syntax for attributes and, to keep the Visual J# .NET syntax in parallel with that of Java, all attributes have to be in a comment format. However this is a special type of comment which the J# compiler will act upon rather than ignore. It starts with /** (you have to have two consecutive asterisks otherwise it will be treated as a normal comment). The @attribute directive tells the compiler that we are dealing with an attribute and what follows is the name of the attribute that is to be processed. In C# we omit the parentheses where the attribute takes no parameters, but we do not do this in J#.

.NET Features not Supported by Visual J# .NET

Unlike C#, which is closely tied into the .NET Framework, there are several key areas of .NET functionality which are not supported by Visual J# .NET for the simple reason that they are not features of Java or of Visual J++, the precursor to Visual J# .NET:

  • Visual J# .NET has no special syntax for properties, indexers and events, as already described.
  • It has no special syntax for operator overloading.
  • It does not support implicit or explicit user-defined type conversions.
  • We cannot define properties, events, structures (value types) or delegates.

Also, the .NET and Java native type systems are not wholly compatible with each other - which will become more apparent when we look at standard Java in more detail in the next section. For example, Java has no equivalent to any of the C# unsigned integer types or the decimal data type. So if we have a .NET class which exposes a function which takes, say, a uint as a parameter and we want to call this function from J# code, then we have to create an instance of the .NET System.UInt32 data type and pass that in. There is also no equivalent for the C# checked construct, so if we do need to convert between types and we want to safeguard against possible data overflows or invalid values, then we have to use other means, such as an if statement:

if (var1 >= 0)
{
System.UInt32 var2 = (System.UInt32)var1;
}

J# and Java

Being able to write .NET code using Java syntax is in itself very nice, but the real power of J# lies in the fact that J# is shipped with a library of .NET classes which map exactly to the standard Java libraries of the JDK 1.1.4. What this means is that we can convert Java code to run on the .NET platform simply by recompiling it using the J# compiler (so long, of course, that the code doesn't require any features implemented in later versions of the JDK - the latest version currently available is 1.4 Beta 2). Later on in this article, we'll look at a simple example of recompiling a Java app with the J# compiler.

Java Classes in J#

As we've implied, this interoperability with Java is due to an assembly (bjlib.dll), which contains .NET implementations of all the classes in the JDK 1.1.4. This leads to a certain ambiguity, as the JDK classes often overlap in both name and functionality with the .NET classes. To take a simple example, both .NET and the JDK provide a String class (System.String and java.lang.String respectively). This means that we have two implementations of strings available to us in J#, and these implementations are partially, but not wholly, compatible. Java-type strings can be converted implicitly into .NET strings, so this will compile:

java.lang.String s1 = "A java.lang
            string";
System.String s2 = s1;

Notice that string literals are of the type java.lang.String, not System.String.

We can also pass java.lang strings directly into methods where a System.String is expected:

java.lang.String s1 = "A java.lang
            string";
System.Console.WriteLine(s1);

But the reverse conversions (from the .NET types to the Java types) are not allowed, so this produces an error:

// This won't compile
System.String s1 = "A .NET string";
java.lang.String s2 = s1;

Even explicit casts of this sort are not permitted, so there are potential pitfalls if we try to mix-and-match the System and Java -type classes. In the case of String we can easily overcome this simply by concatenating the System.String with an empty string:

// This compiles OK
System.String s1 = "A
            .NET string";
java.lang.String s2 = s1 + "";

We initially assumed this was due to an overload of the + operator, but intriguingly, this technique will work even if we concatenate only System.Strings, with no java.lang.String in sight! For example, this will compile:

// This compiles OK - but how?
System.String s1 = "A .NET string";
System.String s2 = "Another .NET string";
java.lang.String s3 = s1 + s2;

However, in some other cases (such as java.util.Date and System.DateTime), there's no compatibility at all, so if you need to convert one to the other, you will have to parse it manually and create a new object of the target type. For this reason, it's probably a good idea to be very careful if you're using both the .NET System classes and the simulated Java classes - there's no guarantee that there's going to be an easy way to convert between the different types, and methods won't accept both types. (As an aside, the use of the standard .NET classes means that we can't use the CodeDOM classes to create Java code from a compiled .NET assembly and recompile it with the Java compiler).

Before we leave this subject, there are a couple of other warnings that are worth raising. Firstly, the java.lang.* classes are imported into J# projects by default, so if we also import the System namespace, there is ambiguity between the .NET types and the simulated Java types; for example, String could refer either to System.String or to java.lang.String. In this case, the compiler makes the choice for us and uses java.lang.String unless a variable is specifically declared as a System.String. However, other cases of ambiguity may result in a compile time error, and at the very least we're likely to end up with confusing and hard-to-maintain code unless we clearly indicate which type we're referring to.

The second point to note is that, like all .NET types, the Java-simulant types derive from System.Object. But all Java classes derive from java.lang.Object, which has similar methods. All Java methods are camel-cased, whereas .NET methods are Pascal-cased, so all the Java-type classes have (for example) a toString() method inherited from java.lang.Object, and a ToString() method inherited from System.Object(). The first of these returns a java.lang.String, and the second a System.String. This doesn't cause a problem in case-sensitive languages such as J# and C#, although it would cause problems if you attempted to use these classes from a case-insensitive language such as Visual Basic .NET. However, it can be confusing that the two methods exist side-by-side. Although the ToString() method (capital T) doesn't show up in Visual Studio .NET's Intellisense, it can be called behind the scenes when, for example, we pass a Java type into a method which takes a System.Object parameter, such as the Console.WriteLine() method. For example, suppose we use Console.WriteLine() to display a Java Date:

Date today = new
            Date();
System.Console.WriteLine(today);

Here, the ToString() method is called implicitly, and we get the following output:

java.util.Date

However, if we instead use Java's System.out.println() method, it will be the toString() method (lower-case t) which is called instead, and we get the (probably more useful) output:

Sun Oct 21 17:21:03 GMT+01:00 2001

This difference highlights the fact that these methods are not identical, and it's important to use the right one.

Porting Java Code

Now that we've seen briefly most of what J# can do, let's have a quick look at converting a sample Java application to run on the .NET Framework. We'll base this application on the .NET application we looked at earlier in the article, to highlight the different approaches that it's possible to take with J#. As before, we'll keep it very simple, but just complex enough to demonstrate a couple of interesting features: awt frames, database access, and events.

Here's the Java code, which successfully compiles and runs using the JDK 1.1.x. This code uses an ODBC Data Source Name (DSN) named Northwind to access the SQL Server Northwind database, and simply displays the FirstName and LastName fields from the Employees table. As before, we provide a button to move to the next record, and another to move back to the start of the data:

// DatabaseTest.java
import
            java.awt.*;
import java.awt.event.*;
import
            java.sql.*;

// Definition for the DatabaseTest
            frame
public class DatabaseTest extends Frame implements
            ActionListener
{
Button b1;
Button b2;
TextField
            t1;
TextField t2;
Label l1;
Label l2;

ResultSet
            rs;
Connection cn;
Statement stmt;

// Constructor for
            the class
public DatabaseTest()
{
// Set the title for the
            frame
setTitle("Sample Java App");

// Define the controls
            for the frame
b1 = new Button("Next");
b1.setFont(new
            Font("Ariel", Font.BOLD, 12));
b1.addActionListener(this); // Add
            event handler for the b1 button

b2 = new
            Button("Refresh");
b2.setFont(new Font("Ariel", Font.BOLD,
            12));
b2.addActionListener(this); // Add event handler for the b2
            button

l1 = new Label("First name:");
l1.setFont(new
            Font("Ariel", Font.BOLD, 12));

l2 = new Label("Last
            name:");
l2.setFont(new Font("Ariel", Font.BOLD, 12));

t1
            = new TextField(10);
t1.setFont(new Font("Ariel", Font.BOLD,
            12));

t2 = new TextField(10);
t2.setFont(new Font("Ariel",
            Font.BOLD, 12));

// Define the layout for the controls on the
            frame
Insets i = new Insets(0,0,0,0);

GridBagLayout gbl =
            new GridBagLayout();
setLayout(gbl);

GridBagConstraints
            gbc = new GridBagConstraints();

gbc.gridx = 0;
gbc.gridy =
            0;
gbc.gridwidth = 1;
gbc.gridheight = 1;
gbc.weightx =
            0.0;
gbc.weighty = 0.0;
gbc.anchor =
            GridBagConstraints.WEST;
gbc.fill =
            GridBagConstraints.NONE;
gbc.insets = i;
gbc.ipadx =
            0;
gbc.ipady =
            0;

gbl.setConstraints(l1,gbc);

gbc.gridx =
            1;
gbc.anchor =
            GridBagConstraints.EAST;
gbl.setConstraints(t1,gbc);

gbc.gridx
            = 0;
gbc.gridy = 1;
gbc.anchor =
            GridBagConstraints.WEST;
gbl.setConstraints(l2,gbc);

gbc.gridx
            = 1;
gbc.anchor =
            GridBagConstraints.EAST;
gbl.setConstraints(t2,gbc);

gbc.gridy
            = 2;
gbc.anchor =
            GridBagConstraints.SOUTHEAST;
gbl.setConstraints(b1,gbc);

gbc.gridx
            = 0;
gbc.anchor =
            GridBagConstraints.SOUTHWEST;
gbl.setConstraints(b2,gbc);

//
            Add the controls to the
            frame
add(l1);
add(t1);
add(l2);
add(t2);
add(b1);
add(b2);

//
            Add the event handler to close the window
addWindowListener(new
            WinClosing());

// Set the size and position of the
            frame
setBounds(100, 100, 300, 200);

// Make the frame
            visible
setVisible(true);

try
{
// Use the Sun
            JDBC-ODBC
            bridge
Class.forName("sun.jdbc.odbc.JdbcOdbcDriver");

//
            Use the Northwind data source name
String dsn =
            "jdbc:odbc:Northwind";

// Open the connection
cn =
            DriverManager.getConnection(dsn, "sa", "");

// Open the
            ResultSet
stmt = cn.createStatement();
rs =
            stmt.executeQuery("SELECT FirstName, LastName FROM
            Employees");

// Move to the first record in the
            ResultSet
rs.next();

// Set the text in the
            textboxes
t1.setText(rs.getString("FirstName"));
t2.setText(rs.getString("LastName"));
}

//
            Handle exceptions
catch (ClassNotFoundException
            cnfe)
{
System.err.println(cnfe);
}

catch
            (SQLException
            sqle)
{
System.err.println(sqle);
}
}

// Event
            handler for the buttons
public void actionPerformed(ActionEvent
            ae)
{
if
            (ae.getActionCommand().equals("Next"))
{
try
{
// If we
            can move to the next record, do so and update the textboxes
if
            (rs.next())
{
t1.setText(rs.getString("FirstName"));
t2.setText(rs.getString("LastName"));
}
}
catch
            (SQLException
            sqle)
{
System.err.println(sqle);
}
}

// If the
            Refresh button was pressed, close and re-open the ResultSet
if
            (ae.getActionCommand().equals("Refresh"))
{
try
{
rs.close();
rs
            = stmt.executeQuery("SELECT FirstName, LastName FROM
            Employees");
rs.next();
t1.setText(rs.getString("FirstName"));
t2.setText(rs.getString("LastName"));
}
catch
            (SQLException
            sqle)
{
System.err.println(sqle);
}
}
}

// The
            program entry point - just instantiate our DatabaseTest
            object
public static void main(String
            args[])
{
DatabaseTest d = new
            DatabaseTest();
}
}

// WindowClosing event
            handler
class WinClosing extends WindowAdapter
{
public
            void windowClosing(WindowEvent
            we)
{
System.exit(0);
}
}

This can be compiled under the JDK 1.1.x using the command:

javac DatabaseTest.java

And run using:

java DatabaseTest

The running application looks like this:

The full value of J# sinks in when you realize that this application can be recompiled to run on the .NET Framework with just one (minor) change. In the original Java code, we used Sun's JDBC-ODBC driver:

Class.forName("sun.jdbc.odbc.JdbcOdbcDriver");

Perhaps not surprisingly, this class isn't available to us in J#, so we have to use Microsoft's JDBC-ODBC driver instead. This is provided in the bjlib.dll assembly, and is called com.ms.jdbc.odbc.JdbcOdbcDriver. So, all we have to do is change this one line:

Class.forName("com.ms.jdbc.odbc.JdbcOdbcDriver");

And we're ready to recompile! Once you've made this change, save the source file as DatabaseTest.jsl and recompile using the J# compiler:

jc DatabaseTest.jsl

This produces a .NET assembly named DatabaseTest.exe, which looks and runs virtually identically to the Java version:

The only discernible difference is that we now have the Windows icon in place of the Java coffee cup!

The most striking thing about this code is that we're programming using a model that is in some respects essentially different from the standard .NET model. This is particularly noticeable with respect to events: Java handles events by creating a new event object (such as an ActionEvent) whenever an event is raised. We register our program to receive notification of these events using the addActionListener() method (passing in this as the parameter, because we want the event to be handled in the DatabaseTest class; our class needs to implement the ActionListener interface to do this):

b1.addActionListener(this); // Add event handler for the b1 button
b2.addActionListener(this); // Add event handler for the b2 button

When the event is raised, the actionPerformed() method is called, and the ActionEvent object is passed into this as a parameter. We add our event handler code to this method. Java supports a number of these event classes; as well as the ActionEvent, the code above also uses the WindowEvent, to shut down the application when the window is closed.

This syntax is in some ways similar to the code we saw in the previous example using the normal .NET events, since we are calling a specific method to add the event handler:

this.buttonRefresh.add_Click(new System.EventHandler(this.buttonRefresh_Click));

However, using the Java syntax, we don't actually add an event handler, but an event listener - in the case of the button click events, this is the frame itself. We don't need to specify the method to use as the event handler, because that's fixed by the Java language. In fact, the same method is used to handle multiple events (in our example, the actionPerformed() method handles the click events for both of our buttons). In other words, yet again we've got two different ways of doing basically the same thing!

Potential Pitfalls

So far, this is very impressive - much easier, in fact, than converting a VB6 application to Visual Basic .NET. The big question is "Will this work for any Java application?" The answer to that, sadly, is no. The main drawback is that J# is based on a very old version of the JDK, and crucially one prior to the major changes of the JDK 1.2 (so called "Java 2"). This means, for example, that we can't recompile any Swing applications using J# (unless we could get hold of and recompile the Java source code, of course).

As well as entirely new class libraries provided in JDK 1.2 and later versions, we also need to watch out for less obvious changes that will still prevent the J# compiler converting the code to IL. For example, in our example above, we use the GridBagConstraints class to position the controls on the frame. We instantiate this using the parameterless constructor, and then set its properties. In later versions of the JDK, it's possible to pass the property values directly into the constructor; however, the J# compiler (like earlier versions of the Java compiler) won't recognize this, and will refuse to compile.

The second major pitfall is that J# can't read compiled Java .class files. This, of course, is no surprise, since J# compiles to IL rather than to bytecode. J# does, however, ship with a utility called jbimp.exe which can convert bytecode .class files to IL assemblies, and which we'll look at in the next section, but initial tests suggest this is less successful than recompiling Java source code using the J# compiler. In any case, jbimp will of course only convert .class files which don't rely on features from later versions of the JDK.

The remaining incompatibilities between Java and J# are surprisingly few. RMI, JNI, and applet development are not supported, and, as we've seen, some of the packages shipped with the JDK 1.1.x aren't supported (some of the com.ms.* packages which shipped with Visual J++ 6 are not supported, and also the sun.* packages (although the classes in these shouldn't be instantiated directly anyway, except as in the code above, when we're really just using the class name to identify the driver)). There are also a few unsupported classes and methods, such as the allowThreadSuspension() method of the ThreadGroup class. More generally, any features supported by Java but not by the .NET Framework (such as certain encoding mechanisms) are also not supported by J#. Class loading differs significantly because J# can load only .NET assemblies dynamically, not Java bytecode. For this reason, Microsoft recommends that we don't use use custom class loaders.

Converting Java Bytecode

As mentioned above, J# ships with a utility named jbimp.exe, which allows us to convert compiled Java .class files from bytecode directly into IL assemblies. This utility isn't perfect yet, and Microsoft recommends recompiling with the J# compiler whenever we have access to the Java source, rather than using jbimp to convert the bytecode. In our tests, jbimp failed to convert Java windows applications which use awt - no valid assembly was created. As an interesting exercise, we compiled a simple Java console application to bytecode, and then converted this to IL using jbimp. We also compiled the same app using the J# compiler, so we could compare the IL produced by these two methods. The Java code we used was:

// DBTest.java
import java.sql.*;

public class DBTest
{
public static void main(String[] args)
{
try
{
// Use the MS JDBC-ODBC bridge, because we will only be
// running this sample as a .NET assembly
Class.forName("com.ms.jdbc.odbc.JdbcOdbcDriver");
String dsn = "jdbc:odbc:Northwind";
Connection cn = DriverManager.getConnection(dsn, "sa", "");
Statement stmt = cn.createStatement();
ResultSet rs = stmt.executeQuery(
"SELECT FirstName, LastName FROM Employees");
while (rs.next())
{
System.out.println(rs.getString("FirstName") + " " +
rs.getString("LastName"));
}
}
catch (ClassNotFoundException cnfe)
{
System.err.println(cnfe);
}

catch (SQLException sqle)
{
System.err.println(sqle);
}
}
}

Compile this using the Java command-line compiler:

javac DBTest.java

This generates a .class file, which we can now run jbimp against. The options for jbimp are similar to those for the J# and C# compilers; the main ones being: /t:exe for an executable, /t:library for a DLL, and /r:filename to reference other .NET assemblies. It also allows us to sign the generated assembly with a specified keyfile, and to create stub references for any missing class references (this allows us to convert the bytecode to IL, but an error will be raised if the assembly is executed and the missing class is called).

In our case, the command for jbimp will be:

jbimp /t:exe DBTest.class

This generates an executable assembly, which we can run just like any other .NET assembly.

We can generate the readable IL code for this file using ildasm:

ildasm /out:DBTest_jbimp.il DBTest.exe

We can recompile the same Java source file (without changes) directly into IL using the J# compiler:

jc DBTest.java

Finally, we can output the IL from this assembly, again using ildasm:

ildasm /out:DBTest_jsharp.il DBTest.exe

Space won't allow us to show the whole of the IL code, but it's striking how similar the two files are. However, the J# compiler does in places seem to produce neater IL code. For example, this is the start of the code generated by the System.out.println() call in the while loop. First, the J# version:

ldsfld class [BJLIB]java.io.PrintStream [BJLIB]java.lang.System::'out'
newobj instance void [BJLIB]java.lang.StringBuffer::.ctor()
newobj instance void [BJLIB]java.lang.StringBuffer::.ctor()

The converted bytecode equivalent only instantiates one StringBuffer object, but loads a couple of null pointers (which are almost immediately popped off the stack again without being used), and an extra local variable (V_6):

ldsfld class [BJLIB]java.io.PrintStream [BJLIB]java.lang.System::'out'
ldnull
ldnull
newobj instance void [BJLIB]java.lang.StringBuffer::.ctor()
stloc.s V_6
pop
pop
ldloc.s V_6

This cleaner IL code further stresses the advantages of using the J# compiler, rather than converting bytecode to IL. Finally, for the sake of comparison, here are the equivalent lines in Java bytecode:

getstatic #11 <Field java.io.PrintStream out>
new #12 <Class java.lang.StringBuffer>
dup
invokespecial #13 <Method java.lang.StringBuffer()>

Note that the IL generated by jbimp will vary according to the version of the JDK originally used to compile the Java source code into bytecode. In this case, we used the latest version at the time of writing (JDK 1.4 Beta 2).

Why J#?

We've seen most of what J# can do, but no discussion of J# would be complete without some consideration of the "political" angle. After all, if we are building a new application from scratch, J# doesn't really give us anything new compared to C#. Moreover, there's likely to be some performance cost because of all the wrapper classes for the Java-simulant types such as java.lang.String. So why did Microsoft (after the somewhat unsuccessful history of J++) feel that a .NET implementation of Java was worth spending a significant amount of time on (not just a new compiler, but also the .NET implementations of all the JDK 1.1.4 classes)? There are four possible areas where J# can do things that C# can't:

  • J# as a migration tool. As the example of porting Java code to the .NET platform earlier in the article showed, J# is a powerful tool for migrating Java and J++ applications to the .NET Framework.
  • Cross-platform applications. Because Java is cross-platform, applications can be written in J# which will compile on non-Windows platforms, so long as no .NET-specific features are used.
  • J# as a learning tool. J# allows developers who are already familiar with Java to start programming straight away on the .NET Framework, without having to learn all the .NET class libraries in one go.
  • J# as a marketing tool. It's impossible to conclude that J# has nothing at all to do with Microsoft's perennial cold war with Sun.

Let's look at each of these in a bit more detail.

J# as a Migration Tool

It's quite likely that the official motivation Microsoft will give for J# is for the conversion of Visual J++ applications to run on the .NET Framework, and in fact much of the documentation supplied with J# is concerned with migrating J++ apps to J#. And if this is the sole purpose of J#, it must be said that it does a very fine job. However, the number of existing (pure) Java applications in relation to the number of J++ apps surely suggests that Microsoft has its sights on a bigger goal. And again, J# is very impressive when it comes to migrating Java applications written on the JDK 1.1.4 or earlier. However, it's impossible to avoid feeling that the real target is current Java developers, and in this case J# falls down fairly heavily as a migration tool, due to its lack of support for JDK 1.2 and later features.

J# for Cross-Platform Applications

While it doesn't seem to feature anywhere in the official documentation, one useful side effect of J# could be the ability to write programs for the .NET platform on Windows that can then be exported to run on the JVM on other platforms. There are a couple of caveats here, of course - we would need to forsake all the .NET classes that don't map to classes in the JDK, and nor can we use any of the modern, Java 2 features in our application. However, until such time as the proposed non-Windows implementations of the Common Language Infrastructure (CLI) are released, J# could provide a useful tool for developers who want to take advantage of .NET on Windows, but who also want to be able to port their code to other platforms.

J# as a Learning Tool

In some ways, J# puts Java developers in a singularly privileged position (and it's hard to believe that this isn't by design). Developers coming to .NET from other languages have two big hurdles to jump: they need to learn a new language (even if C++ users decide to stick with Managed C++ rather than C#, there are still the Managed Extensions to master, and Visual Basic .NET is very different from VB6), and they need to find their way around the vast new class libraries. While Java developers coming to .NET will need to make these transitions too in the long run, they also have a familiar set of APIs and an identical language which they can use while they're getting the grasp of the bigger picture of .NET.

J# as a Marketing Tool

The final point is closely related to this. Surely it can't be coincidence that everything possible has been done to ease the path for Java developers - the group of developers that Microsoft most keenly wants to attract - to migrate to the .NET Framework? It does raise an interesting question, though, that Microsoft has felt it necessary to do this. After all, C# itself was widely regarded (at least in the Java community) as a Java clone, designed specifically to attract Java developers to .NET. The popularity of C# among developers from many different backgrounds (C++, VB, and ASP) tends to suggest it had wider design aims, but the necessity of releasing J# does seem to imply that C# hasn't had the uptake in the Java community that Microsoft had hoped.

Conclusion

In this article, we've taken just a high-level overview of J#, the latest .NET language to emerge. We've seen how we can use J# to build "traditional" .NET applications, such as Windows Forms, using the classes familiar to us from C#. However, C# has a number of features which J# doesn't, and given the overall similarity of the languages, it seems unlikely that many developers will build entire .NET applications from scratch in J#. The real innovation behind J# (one that distinguishes it from other .NET languages so far) is that J# is shipped with a whole library of classes which simulate the JDK classes. This allows the experienced Java or J++ developer to get a handle on .NET without first having to master the vast library of .NET classes. This does lead to some confusion, as there are frequently two sets of classes with similar functionality, and these aren't entirely compatible. However, it also provides a very powerful mechanism for porting at least older Java and J++ code onto the .NET Framework. While we can't really see J# in any way supplanting C#, it does look to be a very useful tool that will enormously ease the transition to .NET for Java and J++ developers.

If you'd like to discuss a broader range of C#Today topics and issues, or if you have any C# related questions you need answers to, please visit our online resource centres at : http://p2p.wrox.com/list.asp?list=csharptoday_discuss&WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0 and http://p2p.wrox.com/list.asp?list=c_sharp&WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0


Interoperating Java, VB6, and .NET Web Services

Interoperating Java, VB6, and .NET Web Services
http://www.csharptoday.com/content.asp?id=1738
Article Rating: 3.5
Published on October 25, 2001
Abstract
In this article, Catalin Tomescu illustrates the integration and interoperability of web services. He creates a Tax Calculator web service, which integrates two existing applications: a State Tax Calculator and a Federal Tax Calculator. To further complicate things, the two web services that our tax calculator integrates are built using different technologies – the state tax calculator application is built using Apache SOAP for Java and the federal tax calculator application is built using Visual Basic 6.0 and the MS SOAP 2.0 Toolkit. Finally, we see how to call our Tax Calculator web service from a variety of clients - a C# console application, a VB 6 console application, and an ASP.NET web application.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

There is little doubt that web services are the future of distributed applications. We've seen a tremendous increase in the number of web services available on the Internet. Stock checkers, weather services, book finders, map and address finders - you name it. We don't need to reinvent the wheel; instead we integrate services offered by online providers and our application is almost ready. This article will illustrate how to create a tax calculator web service by integrating two existing applications: a State Tax Calculator and a Federal Tax Calculator. To further complicate things, the two web services that our tax calculator integrates are built using different technologies - the state tax calculator application is built using Apache SOAP for Java, and the federal tax calculator application is built using Visual Basic 6.0 and the MS SOAP Toolkit (STK).

Not long ago, SOAP emerged as a standard for transferring XML data over HTTP, and many companies rewrote their applications using this standard. Most of the new development is focused on writing services and not standalone applications. As we should expect, not all providers are using the same programming language, the same tools, or the same operating system in their development efforts. This creates problems when writing clients for the various different implementations of SOAP. Although most of the various implementations of the SOAP protocol are more or less compliant with the standard, there are certain important differences - for example, the lack of WSDL (unless we use IBM's WebServices ToolKit) in the Java implementation for web services makes interoperability with frameworks like .NET and MS SOAP Toolkit less straightforward than we might expect. In this article, we'll see how to overcome these problems.

Prerequisites

  • MS SOAP Toolkit Gold 2.0
  • Visual Basic 6.0
  • MS XML 3.0
  • MS .NET Framework Beta 2
  • Apache SOAP for Java 2.2
  • Tomcat 4.0.1
  • JDK 1.3.2
  • Xerces 2.0
  • IIS installed on a Windows NT or 2000 platform (Professional edition is fine for our sample).

Reading this first will aid your understanding of this article:

  • The SOAP specification available at http://www.w3c.org/soap?WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0.
  • The WSDL specification available at http://www.w3c.org/WSDL?WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0.

What we Learn

The article first shows how to build the two underlying web services, a Java State Tax Calculator, and a VB 6.0 Federal Tax Calculator. Then, our .NET Tax Calculator integrates these two web services to provide a single interface for calculating the amount of income tax an individual owes. (In our scenario the income tax is calculated as a sum of federal and state tax). Finally, we will see how to create a couple of different clients to consume the Tax Calculator web service.

Federal Tax Calculator

The Federal Tax Calculator is a web service that computes the amount of federal tax to be deducted, given the annual gross income and marital status of a person. This, of course, is a simplified business logic for federal tax computation.

The application is developed using MS SOAP Toolkit 2.0. The particular thing about this framework is that it assumes that web services are implemented as COM objects. The STK helps us interface the COM object with SOAP requests and responses by providing two types of SOAP listeners:

  • An ASP listener. This listener is implemented as an ASP page that decodes the SOAP request, invokes the COM method on the specified COM object, and packs the response as a SOAP response object.
  • An ISAPI listener. This listener is implemented in the SOAPISAP.dll and performs the same actions as the ASP listener.

The process of enabling a COM object for SOAP communication consists of generating a WSDL file. When we generate the web service .WSDL file from the COM object, we must choose the type of listener we want to use for the particular service. Note that there is no restriction on using both of them for the same web service, as long as the names of the .WSDL files are distinct. The ISAPI Listener is the default one and the one we will use.

Now let's write the COM object which contains the implementation of our FedTaxCalculator. You can write the COM object with whatever tool you want (VC++, VB, Delphi, etc). Here we use VB 6.0.

Here is the content of the FedTaxCalculator.cls file containing the FedTaxCalculator class:

Option Explicit

Public Function ComputeTax(AnnualGrossIncome As Double, MaritalStatus As String) As Double

Dim taxPercent As Single

If AnnualGrossIncome <= 75000 Then
taxPercent = 0.25
ElseIf AnnualGrossIncome <= 150000 Then
taxPercent = 0.3
End If

If MaritalStatus = "M" Then
taxPercent = taxPercent - 0.03
End If

ComputeTax = AnnualGrossIncome * taxPercent
End Function

Create a new VB project, call it FedTaxCalculator, add a class called FedTaxCalculator, and paste the above code into it. Save the project and build the .dll file.

Register the .dll using regsvr32 from the Run command line:

Regsvr32 <path>\FedTaxCalc.dll

Deployment

Using the WSDL Generator utility (which comes with MS SOAP Toolkit), generate the .WSDL file for the FedTaxCalculator service.

When prompted, use a virtual directory called TaxCalculator. If using the ISAPI listener, configure your virtual directory to use SOAPISAP.dll for files with a .wsdl extension (see STK documentation and setup instructions in the download for more details). After we have generated the WSDL file our service is ready to respond to requests.

State Tax Calculator

The State Tax Calculator is a web service that computes the amount of state tax to be deducted, based upon the State we are in. This web service is built using the Apache SOAP for Java Toolkit. Similarly to the MS SOAP Toolkit, we can reuse our existing Java classes and expose them as web services. The Apache implementation of SOAP differs from Microsoft's approach (for both .NET and the SOAP Toolkit) in that the Apache SOAP doesn't rely on .wsdl files to describe a web service. Instead, the Apache SOAP expects us to utilize all the features that the SOAP envelope offers and pass all the information the Apache SOAP needs to make the method invocation:

  • Parameter names
  • Parameter types
  • Parameter order
  • Method call
  • How to serialize the result and how to de-serialize input parameters

Because of this requirement there are couple of problems when trying to connect from a client built with Microsoft toolkits. We'll look at the solution later in this section, but for now let's focus on the code for our StateTaxCalculator service.

Package samples.TaxCalculator;

import
            java.*;

public class StateTaxCalculator {
/**
* This
            function computes the amount of state tax for 2 fictitious states,
            XX and YY.
* The function implements different rules for each of
            the states.
*/
public double ComputeTax(double
            AnnualGrossIncome, String MaritalStatus,
String State, int
            MonthsInState)
{
double taxPercent = 0, tax = 0;

if
            (State.equals("XX")) {
if (AnnualGrossIncome <
            75000)
taxPercent = 0.02;
else
taxPercent =
            0.035;

if (MaritalStatus.equals("M"))
taxPercent -=
            0.005;

tax = AnnualGrossIncome * taxPercent * MonthsInState /
            12;
}
else if (State.equals("YY")) {
if (AnnualGrossIncome
            < 85000)
taxPercent = 0.03;
else
taxPercent =
            0.045;

if (MaritalStatus.equals("M"))
taxPercent -=
            0.003;

if (MonthsInState > 8)
tax = AnnualGrossIncome *
            taxPercent;
else
tax = AnnualGrossIncome * taxPercent *
            MonthsInState / 12;
}

return tax;
}
}

As we can see, the ComputeTax function has four parameters:

  • annual gross income,
  • marital status of the person for which the tax is computed,
  • the state in which the person resides, and
  • the number of months the person has been a resident of this state.

This is a standard Java class and nothing more. To compile the class we would save the file as StateTaxCalculator.java and compile it with the command line:

javac StateTaxCalculator.java

However, this StateTaxCalculator.class is already included in the download in the Samples.TaxCalculator package so that you don't need to reconfigure your Tomcat server (assuming you have already deployed the SOAP samples).

Deployment

We assume you have already downloaded, installed, and configured the Java Runtime Environment 1.3 SDK, Apache SOAP Toolkit 2.2 binaries, the Apache SOAP Quickstart with Xerces 2.0 XML Parser, and Tomcat 4.0. If you haven't, complete setup instructions are included in the download's full_setup_instructions.txt file.

The Apache implementation of SOAP uses a Servlet listener for SOAP requests. In order to make our class available for a SOAP request we must deploy it as a web service using a deployment descriptor. A deployment descriptor is an XML file that contains information about the web service, such as:

  • where the Java class is located,
  • what methods it supports,
  • the encoding used.

Here is the deployment descriptor for the StateTaxCalculator service:

<isd:service StateTaxCalc xmlns:isd="http://xml.apache.org/xml-soap/deployment"
id="urn:StateTaxCalculator">
<isd:provider type="java"
scope="Application"
methods="ComputeTax">
<isd:java class="samples.TaxCalculator.StateTaxCalculator" static="false"/>
</isd:provider>

<isd:faultListener>
org.apache.soap.server.DOMFaultListener
</isd:faultListener>

</isd:service>

Create a file called DeploymentDescriptor.xml that contains the above code. All deployment descriptors for Java web services have the same structure. Only the ID of the service, the Java class where the service resides, and the methods that a service exposes, differ from one service to another.

The deployment descriptor is very important; in fact it is crucial for our web service to work correctly. As mentioned earlier, the Apache SOAP relies on the client to supply complete information about parameters. Notice that in the above deployment descriptor there is no mention about the parameters type, the order of the parameters, or how to serialize / de-serialize each parameter. A Java client will correctly construct a valid SOAP request package, but a Microsoft client (either STK or .NET) fails to do this in the absence of a .WSDL file.

Note: We could use the IBM WSTK at http://www.alphaworks.ibm.com/tech/webservicestoolkit/?WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0 to create .WSDL files for Java classes, but have shown an alternate implementation for completeness of the article.

To overcome this we must add one more section to the deployment descriptor:

<isd:mappings>
<isd:map encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"
xmlns:x="" qname="x:AnnualGrossIncome"
javaType="java.lang.Double"
xml2JavaClassName=
"org.apache.soap.encoding.soapenc.DoubleDeserializer"/>

<isd:map encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"
xmlns:x="" qname="x:MaritalStatus"
javaType="java.lang.String"
xml2JavaClassName=
"org.apache.soap.encoding.soapenc.StringDeserializer"/>

<isd:map encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"
xmlns:x="" qname="x:State"
javaType="java.lang.String"
xml2JavaClassName=
"org.apache.soap.encoding.soapenc.StringDeserializer"/>

<isd:map encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"
xmlns:x="" qname="x:MonthsInState"
javaType="java.lang.Integer"
xml2JavaClassName=
"org.apache.soap.encoding.soapenc.IntDeserializer"/>
</isd:mappings>

Now the deployment descriptor is complete, and we can notify the Tomcat server about the presence of a new web service. Open a command window, change the current folder to the location where you saved StateTaxCalculator.java, and run the deploy.bat file which executes the following command:

java org.apache.soap.server.ServiceManagerClient
http://localhost:8080/soap/servlet/rpcrouter deploy DeploymentDescriptor.xml

To complete the deployment of our StateTaxCalculator, all we have to do is copy the contents of our donwnload's samples\TaxCalculator folder into a corresponding SOAP_HOME\samples\TaxCalculator folder on the server, where SOAP_HOME is the location where the Apache SOAP for Java is installed.

We should now be able to see our deployed service in the Apache SOAP Admin tool. Surf to http://localhost:8080/soap/:

We can view the details of the web service by clicking on its name:

In future, this information will help us to create our own deployment descriptors for any class, by clicking on the Deploy button and following the template shown above.

Surf to the url of the web service and we should see the following:

Tax Calculator

Tax Calculator is the .NET web service that integrates the State and Federal tax calculators to provide a unique interface for tax computation. We have a few choices when writing a web service with the .NET framework:

  • web service using C# or VB.NET
  • Visual C++ web service

The most complex way to write it is using Visual C++. We'll use the simpler and faster way to build a web service, by creating a web service in C#.

Web services written with .NET consist of an .asmx file. We place the .asmx file into a virtual directory where it can be accessed and that's it! The .asmx file is compiled the first time a user accesses it. Thereafter, the file is recompiled only if the server is restarted or the file is changed. This approach offers some flexibility in the development process because we don't have to stop the IIS server for a deployment of the web service.

Let's see the code for the TaxCalculator web service. The TaxCalculator.asmx file contains three classes. The first class is the main class, TaxCalculator, which exposes the ComputeTax function.

The implementation resembles a class written in C#. The only new element is marking the class and the methods for access using the SOAP protocol, using the Webservice and Webmethod attributes.

  • The WebService attribute applied to a class will enable the SOAP code for handling HTTP requests using the SOAP protocol.
  • To mark a method in a class as web enabled, we must specify the WebMethod attribute for that method.

Here is the content of the TaxCalculator.asmx file:

<%@ WebService Language="C#"
            Class="TaxCalculator" %>

using System;
using
            System.Web.Services;
using System.Diagnostics;
using
            System.Xml.Serialization;
using
            System.Web.Services.Protocols;
[WebService(Namespace="http://www.catalin.com/TaxCalculator/")]
public
            class TaxCalculator: WebService {

[
            WebMethod(Description="compute state and fed
            tax")]
[System.Web.Services.Protocols.SoapRpcMethodAttribute(
"http://www.catalin.com/TaxCalculator/ComputeTax",
RequestNamespace="http://www.catalin.com/TaxCalculator/",
ResponseNamespace="http://www.catalin.com/TaxCalculator/")]

public
            double ComputeTax(double AnnualGrossIncome, string MaritalStatus,
string State, int MonthsInState) {

double taxFed = 0,
            taxState = 0;

FedTaxCalculator objFTCalc = new
            FedTaxCalculator();
StateTaxCalculator objSTCalc = new
            StateTaxCalculator();

taxFed =
            objFTCalc.ComputeTax(AnnualGrossIncome, MaritalStatus);
taxState
            = objSTCalc.ComputeTax(AnnualGrossIncome, MaritalStatus,
State,
            MonthsInState);

return (taxState +
            taxFed);
}
}

The TaxCalculator class must access the State and Federal tax calculator services to get this job done. How do we access these services? In order for a .NET client or web service to access another web service, we need to create a proxy class, which has the same interface as the service. Whenever a call is made through this proxy server, the .NET framework forwards the call to the actual service as a SOAP message, and then retrieves the result from the SOAP response message. A proxy class inherits from the SoapHttpClientProtocol class, which will handle all SOAP communication with the server. Again, notice that attributive programming is the key for setting up each method with information about request and response namespaces and SOAP action.

The .NET framework provides a tool called wsdl.exe to generate the proxy for a web service. This executable can generate the proxy class based on the WSDL file corresponding to a web service.

Use the following command line to generate the proxy for the FedTaxCalculator service:

wsdl.exe http://<path to FedTaxCalculator>/FedTaxCalculator.WSDL

The command should generate a file called FedTaxCalculator.cs containing the following content:

[System.Web.Services.WebServiceBindingAttribute(Name="FedTaxCalculatorSoapBinding",
Namespace="http://tempuri.org/wsdl/")]
public class FedTaxCalculator : System.Web.Services.Protocols.SoapHttpClientProtocol {

[System.Diagnostics.DebuggerStepThroughAttribute()]
public FedTaxCalculator() {
this.Url = "http://localhost/TaxCalculator/FedTaxCalculator.WSDL";
}

[System.Diagnostics.DebuggerStepThroughAttribute()]
[System.Web.Services.Protocols.SoapRpcMethodAttribute
("http://tempuri.org/action/FedTaxCalculator.ComputeTax",
RequestNamespace="http://tempuri.org/message/",
ResponseNamespace="http://tempuri.org/message/")]
[return: System.Xml.Serialization.SoapElementAttribute("Result")]
public System.Double ComputeTax(ref System.Double AnnualGrossIncome,
ref string MaritalStatus) {
object[] results = this.Invoke("ComputeTax", new object[] {
AnnualGrossIncome,
MaritalStatus});
AnnualGrossIncome = ((System.Double)(results[1]));
MaritalStatus = ((string)(results[2]));
return ((System.Double)(results[0]));
}

[System.Diagnostics.DebuggerStepThroughAttribute()]
public System.IAsyncResult BeginComputeTax(
System.Double AnnualGrossIncome, string MaritalStatus,
System.AsyncCallback callback, object asyncState) {
return this.BeginInvoke("ComputeTax", new object[] {
AnnualGrossIncome,
MaritalStatus}, callback, asyncState);
}

[System.Diagnostics.DebuggerStepThroughAttribute()]
public System.Double EndComputeTax(System.IAsyncResult asyncResult,
out System.Double AnnualGrossIncome, out string MaritalStatus) {
object[] results = this.EndInvoke(asyncResult);
AnnualGrossIncome = ((System.Double)(results[1]));
MaritalStatus = ((string)(results[2]));
return ((System.Double)(results[0]));
}
}

Paste the content of the FedTaxCalculator.cs into the end of the TaxCalculator.asmx file.

At this point, the TaxCalculator is able to call the Federal Tax Calculator. Let's move next to the State Tax Calculator. The same idea must be applied here too. The only problem is that for a Java web service there is no .wsdl file from which we can automatically generate a proxy. In this situation the solution is to write the proxy class by hand. Since we already have the proxy class for federal tax, we'll modify it to meet our needs.

The major changes that must be made are:

  • The parameter list must be modified accordingly for the StateTaxCalculator service
  • The namespaces used in the proxy must be identical to the ones used by the web service

Here is the proxy class for the StateTaxCalculator service:

[System.Web.Services.WebServiceBindingAttribute(Name="StateTaxCalculatorSoapBinding",
Namespace="http://tempuri.org/wsdl/")]
class StateTaxCalculator : System.Web.Services.Protocols.SoapHttpClientProtocol {

[System.Diagnostics.DebuggerStepThroughAttribute()]
public StateTaxCalculator() {
this.Url = "http://localhost:8080/soap/servlet/rpcrouter";
}

[System.Diagnostics.DebuggerStepThroughAttribute()]
[System.Web.Services.Protocols.SoapRpcMethodAttribute(
"http://localhost:8080/soap/servlet/rpcrouter/ComputeTax",
RequestNamespace="urn:StateTaxCalculator",
ResponseNamespace="urn:StateTaxCalculator")]
[return: System.Xml.Serialization.SoapElementAttribute("Result")]
public System.Double ComputeTax(double AnnualGrossIncome, string MaritalStatus,
string State, int MonthsInState) {

object[] results = this.Invoke("ComputeTax", new object[] {
AnnualGrossIncome,
MaritalStatus,
State,
MonthsInState});

return ((System.Double)(results[0]));
}

[System.Diagnostics.DebuggerStepThroughAttribute()]
public System.IAsyncResult BeginComputeTax(double AnnualGrossIncome,
string MaritalStatus, string State, int MonthsInState,
System.AsyncCallback callback, object asyncState) {

return this.BeginInvoke("ComputeTax", new object[] {
AnnualGrossIncome,
MaritalStatus,
State,
MonthsInState}, callback, asyncState);
}

[System.Diagnostics.DebuggerStepThroughAttribute()]
public System.Double EndComputeTax(System.IAsyncResult asyncResult,
double AnnualGrossIncome, string MaritalStatus,
string State, int MonthsInState) {

object[] results = this.EndInvoke(asyncResult);
return ((System.Double)(results[0]));
}
}

The changes that have been made from the first proxy class are:

  • Because Apache SOAP for Java uses the id of the service as a namespace for SOAP messages, the namespace was replaced with urn:StateTaxCalculator.
  • The list of parameters in the ComputeTax method was changed to: double AnnualGrossIncome, string MaritalStatus, string State, int MonthsInState.

Append this StateTaxCalculator.cs code to the end of TaxCalculator.asmx file and save it. We no longer need the proxy .cs files and now have a web service that is able to call some other services, gather results from these services, and produce a response.

Deployment

Create a virtual directory in the IIS server named TaxCalc and place the TaxCalculator.asmx file into it. To test the service, open a new instance of IE and type in the address http://localhost/TaxCalc/TaxCalculator.asmx?WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0. If no errors are encountered we should be able to see a page containing information about the service:

To see the WSDL file open the IE and type in the address http://localhost/TaxCalculator/TaxCalculator.asmx?WSDL&WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0.

Accessing the TaxCalculator service

So far we have developed a web service (TaxCalculator) that integrates the functionality of the other two services (StateTaxCalculator and FedTaxCalculator) in order to compute the total amount of income tax payable for a given person. The TaxCalculator service exposes its functionality for client applications.

In this final section we'll see how to access the TaxCalculator service from client applications written in different languages.

A .NET Console Client

Writing a C# client for TaxCalculator is very simple once we generate the proxy class. To generate the proxy class for the C# web service open a command window and execute:

wsdl.exe http://localhost/TaxCalculator/TaxCalculator.asmx?WSDL

The tool will generate a file called TaxCalculator.cs which we can compile and easily use in another C# program, like this:

using System;

/**
* This is a C# client for computing taxes.
*/
public class TaxCalculatorClient {
public static void Main() {

String State = "XX", MaritalStatus = "M";
double AnnualGrossIncome = 76000,
TotalTax = 0;
int MonthsInState = 9;

/**
* Create proxy object to access the tax service.
*/
TaxCalculator objTaxCalc = new TaxCalculator();

/**
* Call the ComputeTax method on each of the services.
*/
TotalTax = objTaxCalc.ComputeTax(AnnualGrossIncome, MaritalStatus,
State, MonthsInState);

/**
* Display the result.
*/
Console.WriteLine("John Doe");
Console.WriteLine("Annual Gross Income: ${0}", AnnualGrossIncome);
Console.WriteLine("Resident of : {0}", State);
Console.WriteLine("Marital status : {0}", MaritalStatus);
Console.WriteLine("No of months in {0} : {1}", State, MonthsInState);
Console.WriteLine("Total tax : ${0}", TotalTax);
}
}

As you can see, the code is very simple and the proxy object is used as a standard C# object. We simply call the method and the framework handles everything, from creating the SOAP envelope message, to retrieving the result from the SOAP response.

Save the C# client into a file called TaxCalculatorClient.cs, then compile the TaxCalculator.cs as a library with:

csc.exe /t:library TaxCalculator.cs

Then compile the client using:

csc.exe /reference:TaxCalculator.dll TaxCalculatorClient.cs

Open a command window, change the current folder to the location where you saved and compiled the C# proxy and client classes, and execute TaxCalculatorClient.exe.

The result should be:

A VB 6.0 MS SOAP Toolkit client

Writing a client for TaxCalculator with the MS SOAP Toolkit is simple when we have the .WSDL file. The client is written in Visual Basic and uses the high level API available with the MS SOAP Toolkit.

The CallTaxCalculator function uses the high level API to create the SOAP request and to parse the server response.

Private m_soapClient As SoapClient

Private Const END_POINT_URL = "http://localhost/TaxCalculator/TaxCalculator.asmx"

Public Sub CallTaxCalculator()

Dim objServer As SoapClient, result As Double
Dim AnnualGrossIncome As Double, MaritalStatus As String, State As String, MonthsInState As Integer

AnnualGrossIncome = 76000
MaritalStatus = "M"
State = "XX"
MonthsInState = 9

On Error GoTo catch_all_err
Set objServer = New SoapClient
Call objServer.mssoapinit(END_POINT_URL + "?WSDL")

result = objServer.ComputeTax(AnnualGrossIncome, MaritalStatus, State, MonthsInState)

' display the results
Debug.Print "Name: John Doe"
Debug.Print "Annual Gross Income: " + CStr(AnnualGrossIncome)
Debug.Print "Marital status: " + MaritalStatus
Debug.Print "State: " + State
Debug.Print "Months in state: " + CStr(MonthsInState)
Debug.Print "Total Tax: " + CStr(result)

cleanup:
On Error Resume Next
Set objServer = Nothing
Exit Sub

catch_all_err:
Debug.Print Err.Description
End Sub

Private Sub main()
Call CallTaxCalculator()
End Sub

Create a new Visual Basic 6.0 project, add a module, and paste the above code. In the VB project add references to MSXML 3.0 and MS SOAP Type Library, set the start point of the project main function, compile and execute. The result should be the same as with the C# client.

An ASP.NET Web Client

Finally, we will see how to create an ASP.NET client using C#. This is even easier than with the above two methods. Once we have opened Visual Studio .NET and created a new C# ASP.NET Web Application we only have to follow 3 steps:

Create the User Interface by dragging five TextBox controls and one Button control from the Toolbox in Visual Studio .NET:

Add a web reference to the web service at http://localhost3/TaxCalc/TaxCalculator.asmx?WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0:

Add the click handler code in Webform1.aspx (remember that TextBoxes can only handle string types, so we need to convert AnnualGrossIncome and TotalTax).

private void CalculateTax_Click(object sender, System.EventArgs e)
{
string StringAnnualGrossIncome = TextBox1.Text;
double DoubleAnnualGrossIncome = Convert.ToDouble(StringAnnualGrossIncome);
string StringMonthsInState = TextBox4.Text;
int IntMonthsInState = Convert.ToInt32(StringMonthsInState);

TaxCalculatorClient.localhost3.TaxCalculator ws =
new TaxCalculatorClient.localhost3.TaxCalculator();
double DoubleTotalTax = ws.ComputeTax(DoubleAnnualGrossIncome,
TextBox2.Text, TextBox3.Text, IntMonthsInState);

string StringTotalTax = Convert.ToString(DoubleTotalTax);
TextBox5.Text = StringTotalTax;

}

Run the project by holding down Ctrl + F5 and we should see the following result:

Conclusion

Writing web services and client applications using different languages and toolkits is a necessity today. There will always be issues and problems because of different approaches used by software providers and open source projects. Hopefully this article will help you solve some of the problems when writing C# web services and client applications. You may have further questions - leave your comments in the discussion forum and we will cover them in a future article.

If you'd like to discuss a broader range of C#Today topics and issues, or if you have any C# related questions you need answers to, please visit our online resource centres at : http://p2p.wrox.com/list.asp?list=csharptoday_discuss&WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0 and http://p2p.wrox.com/list.asp?list=c_sharp&WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0


.NET Remoting Contexts

.NET Remoting Contexts
http://www.csharptoday.com/content.asp?id=1586
Article Rating: 4.5
Published on October 9, 2001
Abstract
In a distributed architecture, disparate components co-operate to perform work, and in such an architecture there needs to be a streamlined infrastructure for providing cross object communication. .NET framework introduces a new one, called .NET Remoting. .NET remoting is the glue that links together components (aka assemblies) in different .NET application domains. In addition to the marshalling of calls across these app domains, the remoting infrastructure also maintains a context for each assembly, and each method call that crosses over assembly boundaries. In this article, Kaushal Sanghavi looks at what contexts are, how they work, and how they can be used to pass data between different objects.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

Distributed application development is becoming very common now. Developers are building component based distributed system architectures routinely, and few have the luxury of building straight monolith applications. In a distributed architecture, disparate components co-operate to perform work, and in such an architecture there needs to be a streamlined infrastructure for providing cross-object communication. Currently, there are a few different protocols like Corba and DCOM for distributed computing, and the .NET framework introduces another, called .NET Remoting. .NET remoting is the glue that links together components (aka assemblies) in different .NET application domains. In addition to the marshalling of calls across these app domains, the remoting infrastructure also maintains a context for each assembly, and each method call that crosses over assembly boundaries. A context is information about an object, such as its method calls or security permissions, which can flow across app domains. Contexts are not a new concept under .NET - they have been around since the days of MTS (Microsoft Transaction Server) and COM+. In this article, we will look at how contexts work, and how they can be used to pass data between different objects.

Note that this article is based on Visual Studio .NET Beta 2 and Windows 2000.

MTS and COM+ Contexts

Contexts were first introduced under MTS via the popular ObjectContext object. A context is state information that is maintained implicitly by the different runtimes, and is associated with an MTS, COM+, or .NET object. Every object is associated with exactly one context between activation and deactivation, and one context can hold multiple associated objects. The state maintained by the context object contains information about the object's execution environment (such as the identity of the object's creator), the threading restrictions it is executing under (known as Activity under COM+), any necessary security credentials that the object is executing under, and for a transactional component, the state of the current transaction. This information is examined by the runtime to determine how various events in the object's lifetime should be handled. These could be events like granting or denying the object access to a protected resource, or automatic completion or abortion of its transaction. In addition to these properties, the context can be extended to carry any information that is relevant to a given runtime environment. For example, VB components executing under IIS and ASP often use the execution context to retrieve the intrinsic ASP objects such as the Request and Response objects. In short, a context can be used to maintain any information about any object. The association between an object and its context is implicitly maintained by the runtime infrastructure, and the context can be transparently accessed within our application.

When an object is created under COM+, a context is associated with this object. This context could either be newly created, or the creating object's context. This determination is made depending on the properties of the different objects: if the objects are compatible (similar transactional and synchronization semantics) then the same context is used, but in case of an incompatibility a new context is created, and the newly created object lives under the new context. In the case of a new context being created, certain context properties (such as the transaction ID) from the creator's context flow into the new context. Once a new context is created, method calls that are made across context boundaries are intercepted by the runtime, and are redirected to lightweight proxies that ensure that the context semantics are preserved across calls. Further, the context objects are valid only within their own contexts - any attempt to access a foreign context object can lead to disaster.

Consider the above diagram, where there are two contexts. Objects A and B lie within the same context, but Object C lies in a separate context. Since A and B share a context, A can pass a reference to its object context to B, but B should never pass its object context to C.

Remoting Contexts

Let us now look at contexts within .NET remoting. Remoting contexts are almost identical to COM+ contexts. A remoting context is a boundary that contains objects that share common runtime properties. Under the CLR, a runtime property is represented as a context attribute - this is where remoting contexts differ slightly from COM+ contexts. Context attributes are completely extensible and can be custom attributes defined by the user. Certain attributes like synchronization and thread affinity are predefined context attributes, but any .NET attribute that derives from the ContextAttribute class in the System.Runtime.Remoting.Contexts namespace, qualifies as a context attribute.

When an object is created, the CLR takes a look at all the context attributes that have been declared on the creator object's class and the new object's class, and if the contexts are compatible then the new object is created in the creator's context. If they are not compatible, a new context is created, and all calls from one context into another go through a context proxy. Again, this is similar to the context interception under COM+. There is another similarity to COM+ contexts in that one object is bound to a single context, and a single context can have multiple objects residing in it.

Objects that are bound to a context via the context attributes are called as context bound objects. In order for an object to be classified as a context based object, the object needs to derive from the ContextBoundObject class in the System namespace. A unique identifier identifies every context, and we can obtain a reference to the current context by using the Context property of the System.Threading.Thread class.

Context Creation

Now that we have covered the basics of what a context is all about, let us look at some code. The very first thing that we will look at is how contexts are created.

The following code block shows three classes, Foo, Baz and Creator. The Creator class will create an instance of the Foo and Baz classes. Each of these classes derives from the ContextBoundObject class and defines one context attribute, the SynchronizationAttribute, from the System.Runtime.Remoting.Contexts namespace. Thus each of these classes qualifies to be a context bound class.

[Synchronization (SynchronizationAttribute.NOT_SUPPORTED)]
public class Foo : ContextBoundObject
{
public Foo()
{
Console.WriteLine ("Foo Context: " +
Thread.CurrentContext.ContextID.ToString());
}
}

[Synchronization(SynchronizationAttribute.NOT_SUPPORTED)]
public class Baz : ContextBoundObject
{
public Baz()
{
Console.WriteLine ("Baz Context: " +
Thread.CurrentContext.ContextID.ToString());
}
}

[Synchronization(SynchronizationAttribute.NOT_SUPPORTED)]
public class Creator
{
public static int Main(String[] args)
{
Console.WriteLine ("Creator Context: " +
Thread.CurrentContext.ContextID.ToString());

Foo foo = new Foo();
Baz baz = new Baz();

return 0;
}
}

In our first sample, we specify the same context attribute on all three classes. Since the context attributes are the same, instances of the Foo and the Baz class should be created in the same context as that of the Creator object. We use the Thread.CurrentContext to obtain the current context, and then use the ContextID property to output the context that we are currently in.

As you can see from the following screen shot, all three objects are created in the same context, since all their context attributes are compatible.

Now let us change the context attributes of the Foo and Baz classes such that they differ from that of the Creator class as shown below. Note that the REQUIRES_NEW attribute is specified when a new context is explicitly required.

[Synchronization (SynchronizationAttribute.REQUIRES_NEW)]
public class Foo : ContextBoundObject
{
public Foo()
{
Console.WriteLine ("Foo Context: " +
Thread.CurrentContext.ContextID.ToString());
}
}

[Synchronization(SynchronizationAttribute.REQUIRES_NEW)]
public class Baz : ContextBoundObject
{
public Baz()
{
Console.WriteLine ("Baz Context: " +
Thread.CurrentContext.ContextID.ToString());
}
}

[Synchronization(SynchronizationAttribute.NOT_SUPPORTED)]
public class Creator
{
public static int Main(String[] args)
{
Console.WriteLine ("Creator Context: " +
Thread.CurrentContext.ContextID.ToString());

Foo foo = new Foo();
Baz baz = new Baz();

return 0;
}
}

As you can see from the following screenshot, since the context attributes differ from each of the created classes, both the Foo and Baz objects are created in a new context.

Passing Data using Contexts

Since contexts are shared amongst compatible objects, contexts can be used to store data that can be passed between objects that lie in the same context. Note that since the context automatically flows between the creator of an object and the new object, this can be a fairly efficient way to pass data when the creator is aware that the object will be created in its own context. The context supports name value pairs that can be attached to the context, and are made available to all objects within the same context. Win32 developers should note that this implementation of context data is very similar to TLS (Thread Local Storage) that is available in Win32.

The Context class supports the following methods that can be used to pass data between objects in the same context.

  • AllocateNamedDataSlot: This allocates a slot of memory in the context, and associates it with the given name. This method returns an object of type LocalDataStoreSlot. Named data slots are public, and can be manipulated by all objects in the context.
  • GetNamedDataSlot: This looks up the current context for a slot with the given name: if a matching slot is found, the slot is returned; if a matching slot is not found, a new slot is allocated with the given name.
  • FreeNamedDataSlot: This releases an allocated slot of memory.
  • SetData: Once a data slot has been allocated, this method is used to store data in this slot.
  • GetData: Once a data slot has been retrieved, this method is used to retrieve data from the given slot.

Let us now look at some code that uses these methods to pass data between objects using their contexts.

[Synchronization (SynchronizationAttribute.NOT_SUPPORTED)]
public class Foo : ContextBoundObject
{
public void SomeMethod ()
{
Console.WriteLine ("Foo Context: " +
Thread.CurrentContext.ContextID.ToString());

LocalDataStoreSlot lSlot2 =
Context.GetNamedDataSlot ("MyContextData");

string lStr = (string)Context.GetData (lSlot2);
if (lStr == null)
lStr = "null";

Console.WriteLine ("Foo Context Data: " + lStr);
}
}

[Synchronization(SynchronizationAttribute.NOT_SUPPORTED)]
public class Creator
{
public static int Main(String[] args)
{
Console.WriteLine ("Creator Context: " +
Thread.CurrentContext.ContextID.ToString());

LocalDataStoreSlot lSlot =
Context.AllocateNamedDataSlot ("MyContextData");

Context.SetData (lSlot, "some data string");
string lStr = (string)Context.GetData (lSlot);
Console.WriteLine ("Creator Context Data: " + lStr);

Foo foo = new Foo();
foo.SomeMethod ();

return 0;
}
}

The above code block shows the Creator and Foo classes, both of which are context bound objects and have the same context attributes. The Creator class starts off by allocating a data slot named MyContextData. It then uses the SetData method to store some data into this slot. To ensure that the data has been stored, it uses the GetData method to retrieve the data. Next, it creates an instance of type Foo, which is in the same context, and the new object tries to retrieve the data from the same named slot. Since both objects are in the same context, Foo will be able to retrieve the data as can be seen from the following screenshot.

However, if these two objects were in a different context, this data passing would not work. By changing the attribute of the Foo class so that its context attributes are incompatible with that of the Creator class, sharing data would not be possible.

Change the attribute of the Foo class as shown below.

[Synchronization (SynchronizationAttribute.REQUIRES_NEW)]
public class Foo : ContextBoundObject

Once this is changed, the context changes, and the data is no longer available - as shown in the screenshot below.

Call Contexts

In the above sections we have discussed how the remoting infrastructure and the CLR create contexts for objects, and how objects reside in a context based on their context attributes. Furthermore, we have seen how objects that reside in the same context can use the context to pass data between them. The only limitation of this approach has been that the context is created at the object level, and is valid only amongst objects that reside in the same context.

In addition to the object context, the remoting infrastructure and the CLR maintain another kind of context object that is not associated with an object instance, but with the chain of method calls that are made between objects. This context is called the Call Context, and is maintained between all the objects involved in the call chain, irrespective of their object contexts. This call context is valid for the duration of the call, and flows from the first calling object in the chain to the last called object, and back down to the first object.

As you can see from the above diagram, objects A, B and C reside in different contexts, and thus they cannot share data using the object context. However, there exists a call context that includes all three objects as shown by the shaded box above, and this call context has a span across all three object contexts. Note that the call context exists even across assemblies and app domains, as long as the object itself is remotable. (A remotable object needs to derive from MarshalByRefObject. Remotable objects are objects that can either be accessed outside their application domain or context using a proxy, or they can be copied, and these copies can be passed outside their application domain or context). This call context supports GetData and SetData methods, which can be used to store name value pairs just like the object context. Let us now look at some code that uses three objects in three different object contexts, and uses the call context to share data.

[Synchronization (SynchronizationAttribute.NOT_SUPPORTED)]
public class Foo : ContextBoundObject
{
public void SomeMethod ()
{
Console.WriteLine ("Foo Context: " +
Thread.CurrentContext.ContextID.ToString());

string lStr = (string)CallContext.GetData ("data name");
if (lStr == null)
lStr = "null";
Console.WriteLine ("Foo CallContext Data: " + lStr);
}
}

[Synchronization (SynchronizationAttribute.REQUIRED)]
public class CalleeObject : ContextBoundObject
{
public void SomeMethod ()
{
Console.WriteLine ("Callee Context: " +
Thread.CurrentContext.ContextID.ToString());

string lStr = (string)CallContext.GetData ("data name");
if (lStr == null)
lStr = "null";
Console.WriteLine ("Callee CallContext Data: " + lStr);

Foo foo = new Foo ();
foo.SomeMethod ();
}
}

[Synchronization(SynchronizationAttribute.SUPPORTED)]
public class CallerObject : ContextBoundObject
{
public static int Main(String[] args)
{
Console.WriteLine ("Caller Context: " +
Thread.CurrentContext.ContextID.ToString());

CallContext.SetData ("data name", "some data string");
string lStr = (string)CallContext.GetData ("data name");
Console.WriteLine ("Caller CallContext Data: " + lStr);

CalleeObject lObj = new CalleeObject();
lObj.SomeMethod ();

return 0;
}
}

As you can see from the above code block, CallerObject calls CalleeObject, which in turn calls the Foo object. Each of these objects has a different value for the context attribute, and they use the CallContext.GetData and CallContext.SetData methods to store and retrieve data. The following screenshot shows that the three objects do have separate object contexts, but can retrieve the same data in the call context.

Attached Sample Code

The attached sample code contains three C# console projects. The first project, ContextCreation, deals with the creation of object contexts based on the context attributes of each object, as explained in the "Context Creation" section of this article. The second project, ContextData, deals with passing data between objects in the same context, using the object context as explained in the "Passing Data using Contexts" section. Finally, the CallContext project deals with creating a call context between objects that reside in different object contexts, as explained in the previous section.

Readers are encouraged to change the values of the context attribute to see how that affects the output. Note that changing the context attribute to SynchronizationAttribute.REQUIRES_NEW will always create a new context. Thus, even if both the caller and callee have this attribute set, they will reside in different contexts in spite of having the same context attribute value. Furthermore, two context attributes can have different values but can be compatible, and thus the objects can reside in the same context.

Conclusion

Object contexts and call contexts are heavily used by the remoting infrastructure in .NET. This article is an introduction and high level discussion of contexts, since application developers would rarely need to dig deep into the remoting layer and mess around with these context objects. However, it is very important to understand the internals of the system that you are developing on, and also be aware of its capabilities, so that you can take advantage of its features in your design.

Happy .NETing.

If you'd like to discuss a broader range of C#Today topics and issues, or if you have any C# related questions you need answers to, please visit our online resource centres at : http://p2p.wrox.com/list.asp?list=csharptoday_discuss&WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0 and http://p2p.wrox.com/list.asp?list=c_sharp&WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0


The .NET Property Grid Control

The .NET Property Grid Control
http://www.csharptoday.com/content.asp?id=1672
Article Rating: 3.5
Published on October 8, 2001
Abstract
In this article, Morgan Skinner will explore the facilities available in the .NET Property Grid control, a new addition to the arsenal of controls available within the .NET environment. This control will be familiar to anyone who has used Visual Basic in its earlier incarnations, as it’s basically a new version of the VB property grid. The control allows you to display and modify properties of an object (or set of objects), in a manner that is fairly intuitive to the user. This article will begin with a simple example of using the control to display properties of an object, and will build through successive examples showing how to extend the user interface available with the control.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

This article will explore the facilities available in the .NET Property Grid control, a new addition to the arsenal of controls available within the .NET environment. This control will be familiar to anyone who has used Visual Basic in its earlier incarnations, as it's basically a new version of the VB property grid. The control allows you to display and modify properties of an object (or set of objects), in a manner that is fairly intuitive to the user.

This article will begin with a simple example of using the control to display properties of an object, and will build through successive examples showing how to extend the user interface available with the control.

The downloadable zip file that comes with this article includes seven example programs from this text. Each is provided as a Visual Studio project, and has been compiled with Beta2 of the .NET framework.

Property Grid 101

The first thing we need to know about the property grid control is that it's not included in the toolbox in VS.NET by default - we need to add it there ourselves. To do this, open Visual Studio and create a new Windows Forms application. Then, display the toolbox, popup the right mouse button menu, and choose Customize Toolbox. Click on the '.NET Framework Components' tab, and scroll down to the PropertyGrid entry as shown in the following screenshot.

Select the checkbox and hit OK. You'll then see the PropertyGrid as an entry in the toolbox. Next, add the property grid to the default form generated when we created the project, and dock it or size it as you wish.

Next up, we need to put something into the Property grid control, so for this example we'll pass it an instance of the form on which the control is placed. The control will then display all of the modifiable properties of the form, as shown in the next image. This may seem a little peculiar at first - assigning the PropertyGrid controls parent form to the control, but it saves me creating an object just for the first demonstration.

The code to assign an object to the control couldn't be much simpler!

private void Form1_Load(object sender, System.EventArgs e)
{
// Hook the property grid up to an object - in this instance
// we'll hook it to the form it is displayed upon...
this.propertyGrid1.SelectedObject = this;
}

This has the effect of displaying all properties of the form. What's more is that we can alter the properties and change the appearance on the fly - for instance changing the background color or the window title. One of the cooler properties to change is the Opacity - set it to something like 50% and see what happens! The image below shows some of the properties available at runtime on the form - we can, for example, change the Text as shown below, and the window caption will be updated.

The 'PropertyGrid101' example, which is included in the download ZIP file, displays a window like the above for you to test. Now that we've seen a forms properties being displayed, it's time to show how this is done with a class of our own.

Displaying custom classes

In the following examples we'll work through how to display a simple .NET class, and add features to the class to support a better user interface. We are using a 'shape' class as the base for this example, as it's fairly well understood. Here's the first definition of the shape class - the downloaded code includes full source comments, but these are omitted from the article for clarity.

// Excerpt from file Shape.cs in examples
            Shape1 to Shape5
using System;
using
            System.Drawing;

public class Shape
{
public
            Shape()
{
}

public Point TopLeft
{
get { return
            m_topLeft ; }
set { m_topLeft = value ; }
}

public
            Point BottomRight
{
get { return m_bottomRight ; }
set {
            m_bottomRight = value ; }
}

public Color
            FillColor
{
get { return m_fillColor ; }
set { m_fillColor
            = value ; }
}

private Point m_topLeft = new Point ( 0 , 0
            ) ;
private Point m_bottomRight = new Point ( 100 , 100 )
            ;
private Color m_fillColor = Color.Aquamarine ;
}

If we create an instance of this class and assign it to the property grid control, the following UI is displayed. The source code for this example is available in the Shape1 subdirectory. As we can see, the public properties of the object (BottomRight, FillColor and TopLeft) are shown, and provide the familiar GUI elements, such as the dropdown color selector.

Even without too much code, the UI already looks pretty good. To make it more user-friendly however, we have to delve into custom attributes. The first step is to provide better descriptive information for each of the properties shown on the control.

Descriptions and Categories

A number of attributes are used to affect the display of a given property within the property grid control. The simplest are the DescriptionAttribute and CategoryAttribute. These classes both reside within the System.ComponentModel namespace, so add a using clause to reference this namespace at the top of your .cs file, and then add these attributes as appropriate.

Using these attributes is not only useful when writing classes for use with the property grid control - if you are developing Custom Controls then these (and other) attributes will mark your control out from the crowd.

The Description attribute is used to supply the text that is displayed in the bottom part of the control. This can be any information you desire. The Category attribute is used to group similar properties, just as in the development environment. Several default categories are defined - such as Appearance, Action, Format, etc, and you're free to think up your own at will. Any property that is not categorized will be displayed under the 'Misc' category.

This code snippet shows how to define the category and description attributes for one of the properties on the control. The example source code available in the Shape2 directory contains the complete project for this step.

[Description("The bottom right corner of the
            shape")]
[Category("Appearance")]
public Point
            BottomRight
{
get { return m_bottomRight ; }
set {
            m_bottomRight = value ; }
}

This results in the following UI.

With many more properties, it makes much more sense to add in categories. There are 13 predefined categories in .NET - take a look at the CategoryAttribute documentation available in MSDN for complete information on these options.

Merging Properties

The property grid can accept more than one object at a time, so can be used to edit an array of objects. When we are modifying several objects, it may not make sense to show all attributes - so for instance, we might want to define that the 'Text' property of a control only shows up in the property grid if the user is editing a single control, and be hidden otherwise.

To do this we need to add another attribute - 'MergableProperty'. The code below shows how to define this attribute to inhibit an attribute from display when multiple objects are shown.

[MergableProperty(false)]
public Point TopLeft
{
get { return m_topLeft ; }
set { m_topLeft = value ; }
}

The default is to allow all properties to be merged together, so we need to add this attribute for any that should not show up in a multiple selection of objects. To add multiple objects to the property grid, simply set the SelectedObjects property to an array of objects as shown in the following code.

Shape s1 = new Shape ( ) ;
Shape s2 = new
            Shape ( ) ;
Shape s3 = new Shape ( )
            ;

this.propertyGrid1.SelectedObjects = new Object[] { s1 , s2
            , s3 } ;

This results in the following UI being displayed - both the TopLeft and BottomRight properties are defined as non-mergeable.

Hiding Properties

If we want to inhibit the display of a given property within the PropertyGrid, just add the Browsable (false) attribute as shown in the following snippet of code.

[Browsable(false)]
public int CantSeeMe
{
get { return m_invisible ; }
set { m_invisible = value ; ]
}

Type Converters

In the previous examples, we've seen that when a point is added to a property grid, it is expanded into its composite parts - in this case the X and Y coordinates. Not only that, but if we type in something like '100,100' into the text box above these properties, we'll see that both the X and Y properties change. This voodoo magic is performed by using another set of classes and attributes, which we'll see in this section.

The main method of getting hold of information about a given type in .NET is called 'Reflection'. It's a topic and a half in its own right, and in my opinion the absolute best thing about .NET. For more information on Reflection, check out the MSDN documentation on the System.Reflection namespace. We could use Reflection ourselves to elicit the available properties on a given class, however Microsoft has already done the work for us by providing classes such as TypeDescriptor (which in all fairness uses reflection internally, but saves you the bother).

To support good design time behavior, a class needs to be associated with a 'TypeConverter'. A TypeConverter can be used to convert an object to another type (i.e. from string to point), and is also is used to 'describe' that object based on the properties it supports.

To see what a type converter can be used for, and how to get one, let's first take an inside look at the Point class to see what attribute(s) may be defined. We'll do this by decompiling the assembly that contains the Point class, and see what information we can find out.

One tool available to decompile .NET assemblies is ildasm, the IL Disassembler. This .exe is installed with .NET, and is usually available in the framework SDK's bin directory (on my PC this is 'C:\Program Files\Microsoft.NET\FrameworkSDK\Bin').

My personal favorite is a tool called .NET Reflector from Lutz Roeder. You can download it from http://www.aisto.com/roeder/dotnet?WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0, and you'll also find some extensions to it on the Microsoft GotDotNet site, http://www.gotdotnet.com/?WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0. If you haven't already downloaded a copy, do it now (well, as soon as you have finished this article!). This tool is a brilliant example of the use of reflection to peer inside an assembly and see all classes, methods, properties, interfaces, etc.

The image below shows the .NET Reflector displaying the Point class, which is located within the System.Drawing namespace.

Note that there are two attributes displayed (the items in Green), and if we hover over an attribute we see its definition. As we mentioned earlier you can use ildasm to do this, but the display in Reflector is far superior. If you don't see the attributes as above, select Customize from the View menu, and check 'Show Custom Attributes'.

So, the Point class has a 'TypeConverter' attribute applied, which points to the 'PointConverter' class, which is displayed immediately below the Point class in this screenshot. Expand out this class, and we see the methods below.

Not surprisingly this class derives from TypeConverter. There are some interesting methods on this class, a couple of which we'll discuss in more detail.

Converting between types

Say we have an instance of a class that we would like to convert to another representation - possibly a string representation. Normally we would utilize the ToString() method, however this is only a one-way conversion, and typically ToString() does not output all the properties of the object, it just displays a simple representation. If we want to convert a string back into a Point then you'd be on your own.

To address these limitations, we need to use the type converter methods. Here's a snippet of code showing how to convert from a point to a string and back again.

// Create a point instance.
Point p = new
            Point ( 100 , 200 ) ;

// Get the type converter for a
            Point
TypeConverter tc = TypeDescriptor.GetConverter ( p.GetType
            ( ) ) ;

// Use that class to convert it to a string
string
            pointAsString = tc.ConvertToString ( p ) ;

// Now convert
            back to a point.
p = (Point) tc.ConvertFromString ( pointAsString
            ) ;

Just in case you were wondering, the string representation of a Point is a string containing the two coordinates, separated by a comma, as in '100,200'. When we call the ConvertFromString method, this will parse the string, validate that the contents are correct, and return a point created from the string representation.

If for some reason we pass an invalid syntax to ConvertFromString, we will be rewarded with an exception. A small example program is included in the PointConverter directory, which permits us to convert from a point to a string and back. If we attempt to type in an invalid point, we'll see an exception like the following.

Here we have attempted to type in a point with coordinates '100,1f'. The PropertyGrid control itself is making use of the StringToPoint conversion, as the edit box next to Location on the screenshot is created by the PropertyGrid itself. If you want to see the hoops that Microsoft have gone through to perform the conversion, it's instructive to look at the call stack for an exception raised on an invalid value, such as that shown above. These conversions take note of culture information (unless of course we call ConvertFromInvariantString).

Getting Properties

So, we've seen the property grid displaying properties, but how does it know which properties to display? This is again the job of the TypeDescriptor class.

TypeDescriptor has a method GetProperties (with several overloads), which is used to return a collection of properties - well almost! What is actually returned is a collection of PropertyDescriptors. A PropertyDescriptor describes the property in terms of the actual value of the property, the state (such as IsReadOnly), and many other facets, including the retrieval of the type converter as shown above.

So, let's see an example of retrieving properties from our shape object programmatically - which is basically the same method that the PropertyGrid control uses to display its UI. The following example creates a shape instance, then loops through the properties and displays the list of name, value pairs on screen. We'll also use the type converter for each property to actually display the value.

Shape s = new Shape ( ) ;
StringBuilder
            sb = new StringBuilder ( "Properties.\r\n" )
            ;

PropertyDescriptorCollection props =
TypeDescriptor.GetProperties ( shape ) ;

foreach (
            PropertyDescriptor pd in props )
{
object val = pd.GetValue (
            ) ;

sb . AppendFormat ( "{0,-20}{1}\r\n" ,
pd.DisplayName
            ,
pd.Converter.ConvertToString ( val )
            );
}

Console.WriteLine ( sb . ToString ( ) ) ;

A property might have sub-properties, so we create a simple recursive function which can be called with an object, a property descriptor collection, and a couple of other parameters, which will dump all the properties that it finds. The image below shows the entire list of properties that were found when calling this function with a shape instance.

As you can see, there are a number of properties shown that are not displayed within the property grid - the reason is that they are marked with attributes which the grid (and any other UI) can use to determine whether to show the properties or not.

Providing a GUI

The previous two examples have shown how the PropertyGrid produces the list of items that can be edited, by calling into the TypeDescriptor class to get the properties, then displaying these in a list. However, I've not described how a particular UI component can be generated, such as the ellipses (.) you see when editing a filename property, or the drop down that you get when you edit a color.

We can get an editor for a particular property by obtaining the property descriptor, and simply calling the GetEditor() method.

PropertyDescriptor pd = {some property
            descriptor }

UITypeEditor gui = pd.GetEditor ( typeof (
            UITypeEditor ) ) ;

All editors in .NET are subclasses of UITypeEditor (defined in System.Drawing.Design). This class implements some standard methods which should be overridden, such as GetEditStyle, which is used to determine what type of user interface is used to edit the property (modal dialog, drop down, or none). We can also alter the display of a property by overriding GetPaintValueSupported and providing code in the PaintValue function.

How does .NET work out what editor to use for a property you might ask? It should come as no surprise by now that it uses another attribute. In the following example, we extend the shape class to add on a gradient fill property. This class will include the top left and bottom right colors, which will be displayed within the property grid. Just for good measure, we'll override the painting logic to display something pretty in the property grid.

The image above shows a custom 'fill' object, expanded to show its sub-properties. When you change one of these properties, the UI will update to display a gradient fill derived from those colors. This really is a nifty control, as it allows us to expand a type, change the appearance (i.e. the gradient fill from yellow through red), and as seen before, provide either a dropdown (as in the editor for Color) or a file type editor with ellipses.

We'll see here how to alter one facet of the display - the custom rendered gradient fill shown in the screenshot. First we need to define the new GradientFill class.

[TypeConverter(typeof(ExpandableObjectConverter))]
[Editor(typeof(GradientFillEditor),typeof(UITypeEditor))]
public
            class GradientFill
{
public Color TopLeft
{
get { return
            m_topLeft ; }
set { m_topLeft = value ; }
}

public
            Color BottomRight
{
get { return m_bottomRight ; }
set {
            m_bottomRight = value ; }
}

private Color m_topLeft =
            Color.Yellow ;
private Color m_bottomRight = Color.Red
            ;
}

This class includes public properties for TopLeft and BottomRight - they could have beeen attributed with [Description], etc, if required. The interesting thing is the attributes applied to the class - TypeConverter and Editor.

The TypeConverter has been used throughout the previous sections, as that's what the TypeDescriptor class uses to get information about a type. In this case, the ExpandableObjectConverter is a class derived from TypeConverter, which returns a list of all public properties not marked with Browsable (False). Basically, it returns 'true' to the GetPropertiesSupported method, and an array of property descriptors to the GetProperties methods.

It's probably a good idea to derive from ExpandableObjectConverter for most custom type converters, unless of course you want to roll all the methods yourself.

The other attribute, Editor ( ), defines what object will be used to provide the GUI. In this case, we want our class to do some drawing, so we define the editor to be GradientFillEditor, the code for which is shown below.

public class GradientFillEditor : UITypeEditor
{
public override bool GetPaintValueSupported ( ITypeDescriptorContext ctx )
{
// Return true to get to add in an image...
return true ;
}

public override void PaintValue ( PaintValueEventArgs e )
{
// Draw an image in the rectangle passed
GradientFill gf = e.Value as GradientFill ;

// Create the brush
LinearGradientBrush brush =
new LinearGradientBrush ( e.Bounds , gf.TopLeft, gf.BottomRight ,
LinearGradientMode.ForwardDiagonal ) ;

// Paint the rectangle
e.Graphics.FillRectangle ( brush , e.Bounds ) ;
}
}

As you can see, the code for this is fairly simple. we override the GetPaintValueSupported function to return true, which indicates to .NET that we want to draw an image next to the value of the property. The PaintValue function converts the passed argument to a GradientFill instance, then we utilize the LinearGradientBrush class to create a brush which will be used to fill a rectangle with color.

For more functional user interfaces, it is likely that we would create a UITypeEditor subclass that extends GetEditStyle and EditValue. Similarly, we would want to define a TypeConverter that actually performed some conversion.

Conclusion

This article has presented some of the capabilities of the PropertyGrid control, and we have seen how to utilize the TypeDescriptor to find out various pieces of information about an object. There are a large number of classes that already utilize this functionality in .NET - for a few other editors you need to reference System.Design.dll and take a look at things like the FileNameEditor, FolderNameEditor, AnchorEditor (used to determine which sides to anchor a control to); and the ImageEditor and FontEditor classes in System.Drawing.Design.dll.

If you'd like to discuss a broader range of C#Today topics and issues, or if you have any C# related questions you need answers to, please visit our online resource centres at : http://p2p.wrox.com/list.asp?list=csharptoday_discuss&WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0 and http://p2p.wrox.com/list.asp?list=c_sharp&WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0


String and File handling base classes of the .Net framework

String and File handling base classes of the .Net framework
http://www.csharptoday.com/content.asp?id=1585
Article Rating: 1.5
Published on October 3, 2001
Abstract
In this article, Hersh Bhasin shows how to create a simple Beta 1 to Beta 2 conversion utility, and uses this utility as a backdrop to examine the main focus of this article, the file and string classes of the .Net Framework. Even if you do not have any code to convert over to Beta 2, you should find that this utility serves as a practical example of using these classes.
Article

I wrote a book on ASP .NET which was based on Beta 1. When Beta 2 was released, I began the painful task of modifying my code examples to be compliant with Beta 2. After doing a "find and replace" on the first two chapters, I started pulling my hair in frustration. "This is going to be a never ending task", I said to myself. At that time, the programmer in me came to the rescue, and the result was this Beta 1 to Beta 2 conversion utility which I wrote as a C# console application.

Taking this utility as a backdrop, we will explore the file and string classes of the .NET framework. So even if you do not have any code to convert over to Beta 2, you should find that this utility serves as a practical example of using these classes.

This utility uses the File and String handling classes in C# to parse an input file and do a "find and replace", so that Beta 1 code is converted to Beta 2. The user enters the file name at the console prompt and the utility creates a Beta 2 compliant version of the file, saving the original file with a different name. There are also some complex replacements that require user input. This tool will mark such keywords with a series of asterisks so that the user can manually modify these keywords.

This utility has been designed to work only with ASP.NET code, and is built around my own peculiar scripting idiosyncrasies. You might want to fine-tune it to your own scripting habits. Note that your base file will also be modified - although it will automatically make a backup of your file, you should make copies of your source files in a separate folder.

Code Overview

The utility is made up of four methods. These are:

  • Main: This is the main entry point of the utility. This method is mandatory since a C# program can only be executed through its Main method.
  • ReadFromFile: This method is called from the Main method and is passed the source file name. This method then reads the source file line by line and passes each line on to the method ChangeIt.
  • ChangeIt: This method makes use of a number of methods of the string class to find Beta 1 code and replace it with Beta 2 equivalents. After making the replacements, it passes the line over to the AppendFile method.
  • AppendFile: The AppendFile method writes the replaced line to a new file.
  • Finally, when the ReadFromFile method has finished reading all the lines in the file, it renames the original file by attaching a prefix of "old" to it. The "new" file (the one with the Beta 2 code) is given the name of the original source file, so that in effect we have a file with the Beta 2 compliant code and a backup file containing the original Beta 1 code.

We will now discuss the string and file handling .NET classes in context of this utility.

String Handling in C#

The base .NET class containing string manipulation functionality is System.String. This class has a number of methods and properties that we can use for string manipulation. In the tool developed in this article, we will use the following public instance methods and properties of the string class:

Methods

  • IndexOf
  • Replace
  • ToUpper
  • Trim

Property

  • Length

The IndexOf method locates the first occurrence of a given substring or character in the string, and returns the index of this occurrence. The following code snippet taken from the Converter Tool tries to find a match for the word "ADODataSetCommand" within the string stored in the variable "s". In this case, a match will be found and an integer representing the position of the first occurrence is returned. Since the if statement evaluates to true, the word "ADODataSetCommand" will be replaced by the word "OleDbDataAdapter" by the Replace method in the next line.

String s = "Dim myCommand As
            ADODataSetCommand"
if (
            s.IndexOf("ADODataSetCommand")>0)
{
s=s.Replace("ADODataSetCommand","OleDbDataAdapter");
}

That's basically all we are doing in the body of the method called "ChangeIt". Certain keywords in Beta 2 are different from those used in Beta 1. These differences are indentified in Table 1 in the Appendix. We parse the input file using the ReadFromFile method, discussed later in this article. Each line read is passed on to the ChangeIt method. In the body of this method are a number of conditional if statements, each of which checks for a specific keyword, and if it finds it, replaces it with the Beta 2 equivalent.

The Replace method of the String class simply replaces occurrences of a given character or substring with another character or substring.

The Trim method of the String class removes the leading and trailing white spaces. The ToUpper method converts a string to upper case, and the length property of the string returns an integer representing the length of the string. These three are used together to trap occurrences of the string <property name="Columns"> as follows:

if ( s.ToUpper().IndexOf("COLUMNS")>0
            &&
s.ToUpper().IndexOf("NAME")>0
            &&
s.ToUpper().IndexOf("PROPERTY")>0
            &&
s.Trim().Length
            <30)
{
s=s.Replace(s,"<Columns>");
}

Since in the original code file we could have mixed and matched cases (for example we could have written the word Property as either Property or property), we use the ToUpper method to convert the string to uppercase to make the comparison. We then check to see if the line contains all three words i.e. "COLUMNS", "NAME", and "PROPERTY". Just to be on the safe side, I check the length of the string, since the string <property name="Columns"> will appear as one complete line of code in my original script file and its length will be less than 30 letters. The same logic is repeated for subsequent tags. I first convert the string to upper case, then find a match using the IndexOf method, and if a match is found, replace the keyword with the Beta 2 equivalent.

If you look at Table 1 and examine the Beta 2 changes for the DataGrid, you will note that the property tag has been has been made redundant. In Beta 1 we wrote this tag as follows:

<property name="Columns"> </property>

In Beta 2 this is now:

<Columns> </Columns>

Similarly, the TableItemStyle attribute is now redundant. This attribute was used with the following DataGrid tags:

  • <HeaderStyle>
  • <ItemStyle>
  • <AlternatingItemstyle>

For example in Beta 1 we wrote the following code:

<property name="HeaderStyle">
<asp:TableItemStyle Width="150px"/>
</property>

In Beta 2 this is:

<HeaderStyle Width="150px"></HeaderStyle>

Similarly for the ItemStyle tag:

<property name="ItemStyle">
<asp:TableItemStyle ForeColor="DarkSlateBlue"/>
</property>

Is now:

<ItemStyle ForeColor="DarkSlateBlue">
</ItemStyle>

The <Columns> tag is the topmost tag and the <HeaderStyle>, <ItemStyle> and the <AlternatingItemstyle> are nested within the <Columns> tag.

Replacing the leading tag (<property name="HeaderStyle"> with <HeaderStyle>, and <property name="ItemStyle"> with <ItemStyle>) is simple enough. The challenge is to provide the closing tag. For example the following tags have the same closing tag (</property>):

<property name="HeaderStyle"></property>
<property name="ItemStyle"></property>

What do we replace the closing tag with? The answer is - with the appropriate tag, which is </HeaderStyle> and </ItemStyle> in the above two cases.

The way we identify this is as follows:

  • We declare a private variable called ClosingTag. As soon as we change an opening tag, we store that tag name here. For example, to change the HeaderStyle, we do the following:

if (
            s.ToUpper().IndexOf("PROPERTY
            NAME=\"HEADERSTYLE\"")>0)
{
s=s.ToUpper().Replace(
            "PROPERTY NAME=\"HEADERSTYLE\"","HeaderStyle");
ClosingTag =
            "HeaderStyle";
}

  • Also, somewhere at the end of the function, we have the following code which sets the closing tag as follows:

//Closing Tag for columns
if ( s.ToUpper().IndexOf("/PROPERTY")>0 )
{
s=s.ToUpper().Replace("PROPERTY",ClosingTag);
ClosingTag = "Columns"; //Closing Tag

}

  • Note that as soon as we set the ending tag, we reset the ClosingTag variable to Columns. The logic here is that we know that the parent tag is <Columns> and that all other tags (like <HeaderStyle>, <ItemStyle> etc, fall within these two tags. For tags other than Columns, the opening and closing tags will follow each other (as I am reading the file line by line, going from top to bottom), so at the first occurrence of the keyword Property, we replace the opening tag. The next occurrence of the keyword Property then has to be the closing tag. We reset the variable ClosingTag to Columns each time, as we know that when we are done creating closing tags for the sub tags, the final closing tag will relate to the Columns tag.

  • Also note that wherever we use double quotes within a string, they are preceeded with a backslash as follows:

s.ToUpper().IndexOf("TEMPLATE NAME=\"ITEMTEMPLATE\"")>0

Here, we are looking for a string TEMPLATE NAME = "ITEMTEMPLATE". In C#, string literals are enclosed in double quotes (if enclosed in single quotes, the compiler will take the value as a char and throw an error). We also need to include double quotes as part of the string. This is done by preceding the double quotes with a backslash.

File Operations with C#

The Converter Utility accepts the name of the file to be parsed from the user in the Main method as follows:

static void Main(string[]
            args)
{
System.Console.WriteLine("Please enter filename:
            ");
string s=System.Console.ReadLine();
Filename =
            s.ToString();
ReadFromFile(s.ToString());
}

After obtaining the file name, the Main method passes it on to the ReadFromFile method, which has the following code:

static void ReadFromFile(string
            filename)
{
StreamReader SR;
string S,
            s2;
SR=File.OpenText(filename);
S=SR.ReadLine();
while(S!=null)
{
s2
            =
            ChangeIt(S);
AppendFile(s2);
S=SR.ReadLine();
}
SR.Close();

/*Rename
            original File to Old*/
FileInfo MyFile = new
            FileInfo(Filename);
MyFile.MoveTo("Old"+Filename);

/*Rename
            modified file to have the original name*/
FileInfo MyFile2 = new
            FileInfo("New"+Filename);
MyFile2.MoveTo(Filename);
}

This method makes use of the StreamReader and FileInfo classes. The StreamReader and the StreamWriter classes are optimized to read and write to text files. Their methods automatically detect convenient points to stop reading text, based on the contents of the stream. Reading and writing to text files is done by the StreamReader.ReadLine() and StreamReader.WriteLine() methods. The StreamReader.ReadLine() method can automatically detect carriage returns and will stop reading at that point. Similarly, the StreamWriter.WriteLine() method will automatically append a carriage line-feed combination at the end of the lines that it writes out.

There are two classes available that are responsible for file manipulation activities like moving, copying and deleting files. These are:

  • File
  • FileInfo

Similarly there are two classes that deal with directory (folder) manipulation:

  • Directory
  • DirectoryInfo

In the function ReadFromFile described above, we use both the File class and the FileInfo class. For example, to open a file, we use the OpenText method of the File class. This method returns a StreamReader, which is passed on to the variable SR:

SR=File.OpenText(filename);

Once we have parsed the file, we rename it by prefixing the word Old to the filename. We also rename the New file to the original file name. This is done by making use of the MoveTo method of the FileInfo class, as follows:

/*Rename original File to
            Old*/
FileInfo MyFile = new
            FileInfo(Filename);
MyFile.MoveTo("Old"+Filename);

Note that when we call the OpenText method of the File class, we don't have to instantiate the File class. When calling the MoveTo method of the FileInfo class, we have to create a new instance of the FileInfo class by using the new keyword. This is where the difference lies between these two classes. The file (and directory) contains only static methods and are never instantiated. FileInfo (and DirectoryInfo) implement roughly the same methods as the File and Directory classes, except for the fact that they are stateful and their members are not static.

In general, we use the File class if we just want to do a single operation on a file. This would be efficient, as it avoids the overhead of instantiating a .NET class. However if we need to do multiple operations with the same object, its better to create an instance of the FileInfo class. This is because the authentication and other required information will be read in only once, no matter how many methods we call against each object instance.

A detailed discussion on static and instance methods is beyond the scope of this article. However one way of understanding the difference between these two is that a method declared as static belongs to the whole class, whereas an instance method belongs to a object which inherits from that class. A static method can be called without instantiating an object of that class while instance methods needs an object of that class to be instantiated.

When we call a static method, we put the name of the class before the . operator. For example, to call the OpenText method of the File class, we would call the method like this:

File.OpenText(filename)

However when we call an instance method, we have to specify the name of the instance. For example, to call the MoveTo method of the FileInfo class, you would do the following:

MyFile.MoveTo("Old"+Filename);

Where MyFile is the name of the instance of the FileInfo class.

Earlier we saw that both the classes (File and FileInfo) have similar methods. Had we wanted to open the file using the FileInfo class instead of the File class, we would have done the following:

FileInfo myFile = new FileInfo(FileName);
StreamReader sw = MyFile.OpenText();

Going back to the discussion on the method ReadFile, the filename passed into this function will be opened and each line read sequentially. Each line will be passed into the ChangeIt method, which will search for Beta 1 keywords and replace them with Beta 2 equivalents. It will return the modified line back to the ReadFile method, from where it is redirected to the AppendFile method.

The AppendFile method simply writes out the modified line to another text file. The name of this text file is the original file name with the prefix new. This is the script of this method:

static void AppendFile(String
            s)
{
if (s.Trim() != "" )
{
StreamWriter
            SW;
SW=File.AppendText("New"+Filename);
SW.WriteLine(s);
SW.Close();
}
}

After the modified line is written out to the "new" file, the control returns to the ReadFromFile method. This process continues until there are no more lines left to read. At this point, the StreamWriter is closed, the original file is renamed by having a prefix of "old" added, and the "new" file is renamed by having the prefix of new removed.

Enhancements

The change-log for the Beta 1 to Beta 2 code can be incorporated within an XML file or a database. I am grateful to my friend Shefali for suggesting this and designing a table structure for the change-log as below:

ID (Primary Key) | Entity | Beta 1 | Beta 2 | RC1 | RC2 | CommercialRelease
1 | XXX | B1B1 | B2B2 | RC11 | RC22 | CR1

Having the change-log in a database will allow conversion between not only Beta 1 and Beta 2, but also to any more pre-release or release candidates (RC versions), and finally to the commercial release. Using this approach one can move code from any of the prior releases to any of the succeeding releases. This will also help version control systems keep a tight rein on programs that are being moved from one release to another.

Conclusion

This utility serves as a backdrop to an exposition on the String and File handling base classes of the .NET framework. The String class, though a very powerful class, is not one that is very efficient. This is because the string is an immutable data type; once the String class is initiated, it can never change, and operations like concatenation and string replacements actually create new strings. This means that each time we make a string replacement, we have two string objects taking up space in memory (this memory space is called the Heap, and it is cleared periodically by the Garbage Collector). Clearly this is not efficient if we have a large number of string operations to do.

I would like to direct your attention to the System.Text.StringBuilder class that seeks to overcome the memory-gobbling nature of the String class. Due to length constraints of this article, I have not been able to discuss the StringBuilder class. However, this class is very similar to the string class. The string operations that we can do with this class are limited to appending, removing or replacing text from strings. However this class is memory efficient compared to the string class.

Appendix: Table of Changes

Table 1 gives a detailed list of changes between Beta 1 and Beta 2 for ASP .NET applications. This utility seeks to fix all the changes listed in this table.

Change # | Beta 1 Code | Beta 2 Code
ADO .NET
1 | ADODataSetCommand | OleDbDataAdapter
2 | FillDataSet | Fill
3 | ADO | Oledb
4 | MyCommand.ActiveConnection.Open() | MyConnection.Open()
5 | MyCommand.ActiveConnection.Close() | MyConnection.Close()

Example

 Sub Page_Load(Source As Object, E _ As EventArgs)
Dim myConnection As ADOConnection
Dim myCommand As ADODataSetCommand
Dim ds As New DataSet
Dim ConnStr As String
Dim SQL As String

ConnStr = "Provider=SQLOLEDB; Data _ Source=(local);"

ConnStr = ConnStr + _
"Initial Catalog=ASPNET; User ID=sa"

myConnection = _
New ADOConnection(ConnStr)

SQL = "select * from Masters"
myCommand = New ADODataSetCommand _
(SQL, myConnection)
myCommand.FillDataSet(ds, "Masters")

DataGrid1.DataSource= _
ds.Tables("Masters").DefaultView
DataGrid1.DataBind()
End Sub 		Sub Page_Load(Source As Object, E _ As EventArgs)
Dim myConnection As OleDbConnection
Dim myCommand As OleDbDataAdapter
Dim ds As New DataSet
Dim ConnStr As String
Dim SQL As String

ConnStr = "Provider=SQLOLEDB; _
Data Source=(local);"

ConnStr = ConnStr + _
"Initial Catalog=ASPNET; User ID=sa"

myConnection = _
New OleDbConnection(ConnStr)

SQL = "select * from Masters"
myCommand = New _ OleDbDataAdapter(SQL, myConnection)
myCommand.Fill(ds, "Masters")

DataGrid1.DataSource= _
ds.Tables("Masters").DefaultView
DataGrid1.DataBind()
End Sub

Example (VB.NET)

 Sub Page_Load(Source As Object, E _ As EventArgs)
Dim myConnection As ADOConnection
Dim myCommand As ADODataSetCommand
Dim ds As New DataSet
Dim ConnStr As String
Dim SQL As String

ConnStr = "Provider=SQLOLEDB; Data _ Source=(local);"

ConnStr = ConnStr + _
"Initial Catalog=ASPNET; User ID=sa"

myConnection = _
New ADOConnection(ConnStr)

SQL = "select * from Masters"
myCommand = New ADODataSetCommand _
(SQL, myConnection)
myCommand.FillDataSet(ds, "Masters")

DataGrid1.DataSource= _
ds.Tables("Masters").DefaultView
DataGrid1.DataBind()
End Sub 		Sub Page_Load(Source As Object, E _ As EventArgs)
Dim myConnection As OleDbConnection
Dim myCommand As OleDbDataAdapter
Dim ds As New DataSet
Dim ConnStr As String
Dim SQL As String

ConnStr = "Provider=SQLOLEDB; _
Data Source=(local);"

ConnStr = ConnStr + _
"Initial Catalog=ASPNET; User ID=sa"

myConnection = _
New OleDbConnection(ConnStr)

SQL = "select * from Masters"
myCommand = New _ OleDbDataAdapter(SQL, myConnection)
myCommand.Fill(ds, "Masters")

DataGrid1.DataSource= _
ds.Tables("Masters").DefaultView
DataGrid1.DataBind()
End Sub

Custom Controls
1 | State | ViewState

Example (VB.NET)
Public Property Mode as string
Get
Return State("ls_mode")
End Get
Set
State("ls_mode") = value
End Set
End Property 		Public Property Mode as string
Get
Return _
Cstr(ViewState("ls_mode"))
End Get
Set
ViewState("ls_mode") = value
End Set
End Property
2 | Assembly name can be omitted | Assembly( dll) name required when registering a control

Example

 <%@ Register TagPrefix="Hersh" Namespace="Generic" %> <%@ Register TagPrefix="Hersh" Namespace="Generic" Assembly="GenEditAdd" %>
Where GenEditAdd is tde name of a custom control tdat you want to use on your page.
3 | Writer | Output

Example (VB.NET)

 Protected Overrides Sub Render _
(Output As HtmlTextWriter)
writer.Write _
( "<H1>Hello World</H1>" )
End Sub 		Protected Overrides Sub _
Render(Output As HtmlTextWriter)
Output.Write _
("<H1>Hello World</H1>")
End Sub

Data Grid
1 | <Propery> tag is now redudant | Now use specific tag

Example

 <property name="Columns"> </property> <Columns> </Columns>
2 | TableItemStyle now redundant | Attributes moved to main tag

Example

 <property name="HeaderStyle">
<asp:TableItemStyle
Widtd="150px"/>
</property> 		<HeaderStyle Widtd="150px"> </HeaderStyle>

Example

 <property name="ItemStyle">
<asp:TableItemStyle
ForeColor="DarkSlateBlue"/>
</property>> 		<ItemStyle ForeColor="DarkSlateBlue">
</ItemStyle>

Example

 <property name="AlternatingItemstyle">
<asp:TableItemStyle
BackColor="Beige"/>
</property> 		<AlternatingItemstyle BackColor="Beige"> </AlternatingItemstyle>
3 | SortField | SortExpression

Example

 <asp:TemplateColumn HeaderText="Name" SortField="code_display"> <asp:TemplateColumn HeaderText="Name" SortExpression="code_display">

DataList
1 | <Template name> tag now redundent | Use specific Tag

Example

 <template name="HeaderTemplate"> Name </template> <HeaderTemplate>Name</HeaderTemplate>

Example

 <template name="ItemTemplate">
<asp:LinkButton id="button1" runat="server" Text="Edit"
CommandName="edit" />
<%# Container.DataItem("code_display") %>
</template> 		<ItemTemplate>
<asp:LinkButton id="button1" runat="server" Text="Edit" CommandName="edit" />
<%# Container.DataItem("code_display")%>
</ItemTemplate>

Example

 <template name="EditItemTemplate"></template> <EditItemTemplate> </EditItemTemplate>

Web Service
1 | <WebMetdod()> tag appears after tde word "Function" | <WebMetdod()> tag appears as tde first word in tde function declaration

Example (VB.NET)

 Public Function <WebMetdod()> TestFunction (vInput as Boolean) As String 'some code End Function <WebMetdod()> Public Function TestFunction (vInput as Boolean) As String 'some code End Function
2 | SDL | Is now WSDL
3 tde command to make tde proxy was: Is now:

Example (VB.NET)

 webserviceutil /command:proxy /language:vb /patd:SQLService.SDL wsdl.exe /l:VB /n:SqlService /out:SqlService.vb SqlService.wsdl
4 | Namespace System.Xml.Serialization.dll | Is now: System.Xml.dll

Custom Validation
1 Function parameters changed: Is now:

Example (VB.NET)

Sub ServerValidate (sender As Object, value As String) As Boolean
response.write(value)
if value = "10" tden
ServerValidate = true
else
ServerValidate = false
end if End Function 		Sub ServerValidate (sender As Object, value As ServerValidateEventArgs)
Dim num As Int32 =
Int32.Parse(value.Value)
response.write(num)
if num= "10" tden
value.IsValid = True
else
value.IsValid = False
end if End Sub

If you'd like to discuss a broader range of C#Today topics and issues, or if you have any C# related questions you need answers to, please visit our online resource centres at : http://p2p.wrox.com/list.asp?list=csharptoday_discuss&WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0 and http://p2p.wrox.com/list.asp?list=c_sharp&WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0


.NET Remoting Interception

.NET Remoting Interception
http://www.csharptoday.com/content.asp?id=1664
Article Rating: 3.5
Published on October 1, 2001
Abstract
With the advent of .NET, Microsoft realized the need for a more flexible approach for building distributed applications. .NET Remoting is the technology that allows remote components to interface and integrate. However, in order for this transparent integration, .NET remoting needs to provide some plumbing wherein it can resolve object references and direct method calls to the specific application hosting the required component, on the specific machine. This concept is based on the principle of Interception. In this article, Kaushal Sangahvi discusses the interception mechanism that is used by remoting, to allow for location transparency of objects.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

Distributed applications are becoming the de facto standard for building enterprise level systems. Applications usually comprise of components that run on different physical machines, and even on different platforms. The last few years have seen a number of technologies introduced to aid in this distributed development, such as DCOM or Corba\IIOP, all with some limitations. With the advent of .NET, Microsoft realized the need for a more flexible approach for building distributed applications, and to allow components from different parts of the system to communicate. .NET Remoting is the technology that allows remote components to seamlessly and transparently interface and integrate, by allowing the components to be transferred across AppDomains. However, in order for this transparent integration, .NET remoting needs to provide some plumbing wherein it can resolve object references and direct method calls to the specific application hosting the required component, on the specific machine. This was a concept first introduced by Microsoft Transaction Server (MTS), and is based on the principle of Interception. Interception implies that method calls to objects are intercepted by the runtime, and then dispatched appropriately to the remote object. In this article, we will discuss the interception mechanism that is used by remoting, to allow for location transparency of objects.

Note that this article is based on Visual Studio .NET Beta 2 and Windows 2000.

.NET Remoting

Before we start discussing the remoting interception mechanism, let us quickly recap the basics of .NET remoting. There have been quite a few articles on this on the http://www.csharptoday.com/?WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0 site, please refer to the links section for a detailed explanation of remoting.

.NET introduces the notion of Application Domains, or AppDomains, which is similar to a logical address space that hosts certain components. An application domain is a runtime representation of a logical process within a physical process. Components within a single AppDomain can communicate using traditional local procedure calls and method calls. Components that lie in different appdomains, however, need to use remoting in order to communicate. Components in different appdomains are considered to be remote objects; this is true even if the appdomains are within a single physical process, or within different processes on the same machine.

In order to facilitate this communication, .NET remoting introduces the concepts of channels and formatters. Using a combination of a channel and a formatter, an object in one AppDomain can communicate with another in a remote AppDomain. A channel represents the medium over which the communication happens, and a formatter represents how the messages are encoded to flow over the channel. Currently, .NET supports three channels: HTTP, TCP and SMTP. It also supports two formatters: a binary formatter and a SOAP formatter. Depending on the physical architecture, developers can choose the appropriate combination of channels and formatters. For example, to obtain optimum performance, use a TCP channel with a binary formatter, but if the goal is flexibility and interoperability, then use a HTTP channel with a SOAP formatter.

Remoting Interception

After the very brief recap of remoting, let us move on to talk about how the remoting mechanism uses interception and proxies to allow cross-AppDomain calls.

In order for the remote objects to communicate, the object references, method parameters, and return values have to be marshaled from the source AppDomain to the target AppDomain. Similar to DCOM, .NET remoting uses proxies to handle this. A proxy is just another managed component that the runtime produces that intercepts the call from the calling object to the remote object. Since neither the calling object nor the remote object are aware of this remoting mechanism, calls have to be intercepted on both ends. There are two different kinds of proxies that handle this interception: the proxy on the calling side is called a transparent proxy, and the proxy on the remote object's side is called the real proxy. The following diagram illustrates this process.

The calling object and the remote object are totally oblivious of this interception and redirection. In fact, the only thing we need to do to enable our object to be remoted is to derive from the MarshalByRefObject class from the System namespace. Further, if we are passing instances of local objects as method parameters to the remote object, or the method returns an object reference as a return value, these objects have to cross over to the target AppDomain. In order for objects to move across AppDomains, the objects need to be serialized and marshaled. Thus all local objects that have to cross over AppDomains either need to implement the ISerializable interface, or need to specify the Serializable attribute on the class declaration. If we want to use the default serialization mechanism, use the Serializable attribute, and if we want to provide our own serialization, we should implement the ISerializable interface. If our class derives from MarshalByRefObject, explicit serialization support is not necessary.

The MarshalByRefObject class is used mainly for providing lifetime services for the remote object, and the ISerializable interface supports methods that are used for serializing and deserializing the marshaled object.

Proxies and Messages

As we discussed in the above section, remoting uses proxies at both ends to allow remote objects to communicate. When an object creates a reference to a remote object, the remoting runtime detects this and it creates an instance of the transparent proxy for the calling object, and an instance of the real proxy for the remote object. The transparent proxy is an instance of the TransparentProxy class, which is an internal class and cannot be inherited or extended. The transparent proxy looks up the real remote object and creates a stub for every method that is implemented by the remote object. Once this is created, the remoting runtime hands off a reference to the transparent proxy to the calling object, instead of a reference to the remote object. Thus, the remoting mechanism has to intercept the call only once to create the transparent proxy: once the proxy is created, the calling object makes calls on the proxy object and not on the remote object. In fact, the calling object never gets a reference to the remote object; the creation call is intercepted, and a reference to the proxy is returned instead of a reference to the remote object. From this point on, the transparent proxy object starts dispatching off all the calls that it receives from the calling object to the remote object's real proxy, which in turn dispatches the call to the actual remote object. The exact opposite happens on the return trip for the return value.

The transparent proxy and the real proxy use messages to communicate. A message is a managed object that implements the IMessage interface. This message object contains all the information that is necessary to represent a unit of communication. The .NET remoting interception mechanism uses the following units of communication to represent method calls to the remote object.

  • IMethodMessage: This interface derives from IMessage and represents the method being called on the remote object. This message object contains data about the method name, method signature, the type and URI of the target object, and the call context for the object.
  • IMethodCallMessage: This interface derives from IMessage and represents the parameters to the method being called on the remote object. This message object only represents the information that flows from the calling object to the remote object, which is essentially all the input arguments to the method call.
  • IMethodReturnMessage: This interface derives from IMessage and represents the return trip from the target object to the calling object. This message object represents all the information that flows in the opposite direction, such as the output arguments, the return value of the method, and any exception information associated with the execution of the method.

Note that in reality, the messages need not flow directly between the transparent and real proxies. Just as the proxies intercept the messages between the actual objects, there could be another object that intercepts the messages that flow between the proxies. Messages flow to a message sink, and any object can register itself as a message sink. The transparent proxy and real proxy are also message sinks that flow messages. Thus, it is possible that there exists an entire message chain between the communicating objects. Each sink within this chain can manipulate the message that flows to the next sink in the chain. System tools and debugging utilities that need to monitor the data flowing between objects usually take advantage of this interception mechanism offered by .NET.

Extending Proxies

The transparent proxy that is created in the calling object's AppDomain is an internal class that is neither exposed by the .NET framework, nor can be extended. The real proxy in the remote object's AppDomain, on the other hand, is exposed via the RealProxy class and can be inherited by developers to extend and customize the default behavior of the proxy class. This can be done for multiple reasons depending on the need of our specific application, but one of most obvious reasons for doing this might be to add software component load balancing on our application server. By having a custom real proxy for our server object, we can intercept all calls to the server object, and then redirect them to an appropriate server from a server farm based on the relative utilization of each server. The RealProxy class exposes a method named Invoke, which invokes the method on the target object. By deriving a class from the RealProxy class, we can override this method to perform any custom processing necessary before we invoke the actual method on the target object. In addition to the Invoke method, the RealProxy class supports the following two important methods that can be used to obtain a reference to the transparent proxy and to query the type of the remote object behind the real proxy.

  • GetProxiedType: This method returns the type of the remote object that is represented by this instance of the real proxy.
  • GetTransparentProxy: Real proxies and transparent proxies always operate in pairs. For every transparent proxy on the calling object, there exists a real proxy on the target object. This method returns a reference to the transparent proxy corresponding to this instance of the real proxy. This can be used to call methods on the remote object as we explain in the next section.

Sample Custom RealProxy

In the attached sample, we will demonstrate a very simple application that uses a custom proxy that is derived from the RealProxy class. Our custom proxy will simply log the contents of the message that is flowing across the proxy and output some trace data. This is useful to really understand the whole interception mechanism. We will use our standard "add two numbers" application: again, a very simple application, but the purpose here is to demonstrate how a custom proxy can be implemented and how interception works.

Let us start by taking a look at our simple addition object. Since this object needs to be capable of being remoted, we inherit from the MarshalByObject class. Everything else in this class is like any other ordinary class.

public class SimpleAdd : MarshalByRefObject
{
public int Add (int iOp1, int iOp2)
{
return iOp1 + iOp2;
}
}

Now let us take a look at the implementation of our custom proxy. The CustomProxy class derives from the base class RealProxy, and we override the Invoke method of the base class. The Invoke method is the method called by the remoting runtime to invoke the target object's method. When the CustomProxy object is created, we need to know the type of the target object that this proxy will be pointing to. Once we obtain this type, we need to create an instance of this type so that we can delegate all calls to the target object. We do this in the CustomProxy 's constructor, where we store the object type, and use the Activator object to create an instance of the object based upon its type.

Once the object instance is created, in the Invoke method we simply obtain the method name and the method arguments from the IMessage object passed to this proxy (also known as a message sink), and use the InvokeMember method call from the System.Type class to call a method by name. (This type of invocation is very similar to a late bound method invocation by name that is used by scripting languages while accessing COM objects using the IDispatch interface). The InvokeMember method returns the return value from the method invocation, and we use this to create an instance of the ReturnMessage class which is an implementation of the IMessage interface for sending the return value back to the calling object.

public class CustomProxy :
            RealProxy
{
public CustomProxy(Type iType) :
            base(iType)
{
mType = iType;
mInstance =
            Activator.CreateInstance (iType);
}

public override
            IMessage Invoke(IMessage iMsg)
{
IMethodMessage lMethod = iMsg
            as IMethodMessage;
Object lRet = mType.InvokeMember
(lMethod.MethodName, BindingFlags.InvokeMethod,
null,
            mInstance, lMethod.Args);

Object[] lOutArgs =
            null;
ReturnMessage lRetMsg = new ReturnMessage
(lRet,
            lOutArgs, 0, null,(IMethodCallMessage)iMsg);
return
            lRetMsg;
}

private Object mInstance = null;
private
            Type mType = null;
}

As we can see from the above code block, it is very simple to create a custom proxy that can intercept calls to the remote object. For the sake of clarity, we have omitted some of the logging code in this article: the attached sample code uses the IMessage methods to log information about the IMessage object as shown in the following screen shot.

private void btnAdd_Click(object sender, System.EventArgs e)
{
CustomProxy lCustomProxy = new CustomProxy (typeof(SimpleAdd));
SimpleAdd lAddObj = (SimpleAdd)lCustomProxy.GetTransparentProxy ();

txtResult.Text = lAddObj.Add
(Int32.Parse (txtOp1.Text), Int32.Parse (txtOp2.Text)).ToString ();
}

The above code block shows how we can use an instance of the CustomProxy object to obtain a transparent proxy to the remote object. Once we have obtained the transparent proxy, we can make method calls on the object as though the object were a locally created object being referenced directly. As the name Transparent Proxy suggests, the calling object does not know that the call is being made on a proxy and not on the actual remote object.

Conclusion

The .NET framework and runtime make heavy use of the underlying remoting technologies to provide an extensible architecture and framework for distributed development. While similar frameworks have existed in the pre-.NET days, .NET remoting provides an extremely flexible framework that can be extended based on our application's needs. Remoting in turn, makes heavy use of interception to provide the illusion of seamless object references and method calls. Further, as we discussed in the article, this interception mechanism itself is extensible, with multiple interceptors possible. Most application developers will never need to dig into the remoting or the interception mechanism, but understanding the underlying mechanisms will greatly aid the developer into building a better and more performant distributed system.

Happy .NETing.


A C# Web Spider written as a Web Service

A C# Web Spider written as a Web Service
http://www.csharptoday.com/content.asp?id=1584
Article Rating: 4
Published on September 28, 2001
Abstract
A Web Spider is a program that is used to navigate sites, follow links and collect various information. It is sometimes called a “Web Robot”. In this article. ST Tan & Aldy Karna discuss the subject, and show how a Web Spider can be written in C#.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

A Web Spider is a program that is used to navigate sites, follow links, and collect various information. It is sometimes called a "Web Robot". The Web Robots FAQ (http://www.robotstxt.org/wc/faq.html?WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0) contains the definition: "A robot is a program that automatically traverses the Web's hypertext structure by retrieving a document, and recursively retrieving all documents that are referenced." An overview of available robots can be found here - http://www.robotstxt.org/wc/active/html/?WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0. There we find robots with names like Spider, Cruiser, Digger, Crawler, Octopus, Walker, Agent, or just Bot. Some of them are commercial.

Spiders can be extended to do indexing for search engines. Other bots can help us in purchasing, e.g. when we want to buy a particular item, it will search for it in different places and display a comparison of prices. One bot named Junglee does exactly that for books, and was quickly acquired by Amazon.com. Bots could also be harmful in many ways, which we would rather not go into here.

Programming wise, there are Spiders written in Java, C++, Perl, and ASP (a good ASP example is http://www.asp101.com/articles/chris/spider/default.asp?WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0). We are not aware of any Spiders written in C#, and that was one of the motivations behind writing this article.

It turns out that writing the program in C# is not difficult, because many of the tools required are already provided in the .NET Framework. This is particularly apparent if we compare the program below with the ASP program mentioned in the previous paragraph, which makes use of a control called InetCtls. InetCtls is a control provided by Microsoft to access the Internet from within ASP. However it is not a standard control that one commonly finds on hosted servers. In contrast, the program below does not use any controls. We chose to write the program below using regular expressions, DataSet objects, and Web Services, hoping that the readers will find the techniques interesting.

Navigating web sites is similar to directory browsing, and by nature they are recursive. However, parsing html pages is much harder because a html page is pretty much unstructured. Here we use regular expressions to process the page and collect links as well as to filter links that may not be interesting to follow, such as images or cascading style sheets. In addition, we must avoid getting into endless loops, and have a criteria when to stop. In the program below, we stop when we encounter external links. An alternative is to use a stopping criteria based on the search depth.

The program is first written as a console application and then converted to a Web Service which will return a DataSet as result. This in-memory DataSet is also used to store results during navigation. These result consists of lines, each of them describing a link address, its parent link, and various information about whether the link is dead, external, etc. DataSets are newly introduced with .NET. They have the same structure as a database, except that they are located in memory, perhaps similar to the abandoned idea of IMDB (in memory database) in COM+. Finally, a hash table is used to keep track of links already visited.

The program could be extended to do indexing if desired, by building a database of words found and their links. This would allow one to build one's own customized search engine using keywords of interest.

Getting web pages as strings

Microsoft .NET Framework SDK already provides the necessary classes to fetch a web page and convert the result into a string. To get a page whose URL is given, we use the WebRequest and WebResponse classes from System.Net. The WebRequest syntax shown is compatible with .NET Beta 2; the earlier Beta 1 uses a WebRequestFactory.Create expression. The result is a StreamReader that can then be converted into a string using StringBuilder:

WebRequest req =
            WebRequest.Create(addr);
WebResponse resp =
            req.GetResponse();
StreamReader sr = new
            StreamReader(resp.GetResponseStream(),
            Encoding.ASCII);
StringBuilder pageBuffer = new
            StringBuilder();
String line;
while ((line = sr.ReadLine()) !=
            null {
pageBuffer.Append(line);
}
sr.Close();

The above is part of the code in the method LoadPage() of the class WebPage. It is pretty straightforward; a StreamReader has a ReadLine() method, which is used in a loop to append to a StringBuilder.

Parsing a web page using regular expressions

After a page is fetched, we have a html page in the form of a string. There are many possible ways to parse it. In the future all pages could be in XML or XHTML, and parsing would be much easier using the XML DOM object model, but at present the majority of web pages are still in html. A brute force string search is also possible, but we chose to use regular expressions to collect all the links in it:

r = new Regex("href\\s*=\\s*(?:'(?<1>[^']*)'|\"(?<1>[^\"]*)\"|(?<1>[^>\\s]+))",
RegexOptions.IgnoreCase|RegexOptions.Compiled);
for (m = r.Match(page); m.Success; m = m.NextMatch()){
link = m.Groups[1].ToString().ToLower();
link=FilterLinks(link);
if (link.Length!=0) bagOfLinks.Add(link);
}

Although regular expressions look rather cryptic (except perhaps to seasoned Perl programmers), they have the advantage of being powerful and efficient. Microsoft .NET regular expressions are compatible to Perl regular expressions, and in addition have an object-oriented outer wrapping. The above classes, properties, and methods ( Match, Groups, Success, and NextMatch) are used together to process the results of the matching.

The regular expression object has the options Compiled (the regular expression will be compiled as assembly for faster execution at the expense of slower startup) and case insensitive.

The \s will match any white space, and \s* any number of white spaces, including none. The symbol backslash \ is also used as an escape character within a string, \" means double quotation, \\s is white space. (?: ) is a non-capturing group, i.e. it matches, but does not store the results in Group. (?<name> ) on the other hand is a capturing group, which can be named or numbered. In the above example we use the number <1> and m.Groups[1] to extract to results.

The symbol | is used for alternation, we see it in operation once in the above expression, as the URL can be single quoted, double quoted, or not quoted at all. The square bracket pair [ ] is the set construct, e.g. [abc] is the set consisting of the characters a, b, and c. On the other hand, [^abc] is the complement set, i.e. the set that does not contain a, b, or c.

We hope that this explanation helps to show what the above expression is trying to match: href followed by white space, followed by =, followed by the URL delimited by double or single quotes, or unquoted.

The above code is used in the method LoadPage(), in the class WebPage. The result of the matching is stored in an ArrayList bagOfLinks called " bagOfLinks".

Unfortunately the links obtained still include unwanted links, such as href within javascript, gifs, jpgs, mpeg, js, and css. Of course this is not always so, and in some cases we might like to follow the images instead. For example, "Web Bugs" (http://www.eff.org/Privacy/Marketing/web_bug.html?WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0) are often very small 1-by-1 pixel images hidden within a page to monitor visitors without their being aware of them. The image is made as small as possible to escape detection, and has a link to the site of the bug owner. This link is not just a static link, but carries information within a query string passed to the bug owner.

Some well-known sites have been found to contain "Web Bugs". A program to search for "Web Bugs" could be developed along similar lines as in this program. Here we assume that we want to exclude links to images. To get rid of the unwanted links, the method FilterLinks() is used. This filter also operates using regular expressions. During this process, we would also like to distinguish between internal and external links, because external links will not be followed further, other than verifying whether the external link is alive (available) or not.

Recursive navigation and the overall structure

UML Class Diagram (simplified)

The above class diagram shows the major classes used, omitting minor ones. The class Spider has the main entry point. It accepts the URL given by the user as one of its parameters, and passes it to the Explore object by writing it to the instance variable addr. The class Explore contains Run() which calls itself recursively.

public void Run(){
ArrayList
            pageLinks = new ArrayList();
// bag to accumulate found URLs in
WebPage ObjWPage=new
            WebPage(addr);
ObjWPage.LoadPage();
if (ObjWPage.addr ==
            "error") { return; }
if (addr.IndexOf(Parameter.ParentLink)!=-1)
            {
ObjWPage.CollectLinks();
pageLinks =
            ObjWPage.bagOfLinks;
Results.WriteLog(addr,"","has","has
            "+pageLinks.Count+" links.");
}
else { return; }
try
            {
// Now process all found URLs
if
            (!SharedData.alreadyVisited(Parameter.MainLink))
SharedData.markAsVisited(Parameter.MainLink);
if
            (!SharedData.alreadyVisited(ObjWPage.addr))
SharedData.markAsVisited(ObjWPage.addr);
foreach (String wpage in pageLinks) {
if
            (!SharedData.alreadyVisited(wpage)) {
SharedData.markAsVisited(wpage);
Explore ObjExpl = new
            Explore();
ObjExpl.addr=wpage;
ObjExpl.Run();
}
}
}
catch(Exception
            e) {
Results.WriteLog("run","explore
            error","[err]",e.ToString());
}
}

WebPage is the class that ties together the fields and methods related to a Web page. WebPage has the responsibility to load a page, collect and filter links. The Parameter class is basically a singleton, which it contains static data used by several classes. SharedData is used to keep track of pages already visited, otherwise the algorithm may never stop because the same links could appear in several pages. It uses a hash table data structure. Finally, Results contains the DataSet, and within it the DataTable where the final results of the navigation are stored. This DataSet is returned by the Web Service to the caller. We have chosen to use a DataSet here, since it is structurally equivalent to a database, and it is one of the data types allowed to be returned by a Web Service. Furthermore, if we want to store the results in a persistent storage, the DataSet could be easily converted into a conventional database. In the above code, WriteLog is a method of Results to write to the DataTable.

Using a DataSet to store results

DataSets and DataTables are new ideas introduced with ADO.NET. It allows representation of a relational database in memory, including Tables, Relations and Constraints. A DataSet could be a snapshot of data from a database, or it could be independent disconnected data. A DataAdapter can be used for conversion to and from persistent databases. DataSets can be returned by Web Services.

The DataSet in our case is very simple, it has only one Table, which can be seen from the XML returned when the Web Service is called, which shows the structure of the Table:

<xsd:element name="ParentTable">
<xsd:complexType>
<xsd:sequence>
<xsd:element name="Link" msdata:ReadOnly="true" type="xsd:string"/>
<xsd:element name="Available" type="xsd:string" minOccurs="0" />
<xsd:element name="ParentLink" type="xsd:string" minOccurs="0" />
<xsd:element name="State" type="xsd:string" minOccurs="0" />
<xsd:element name="Remarks" type="xsd:string" minOccurs="0" />
</xsd:sequence>
</xsd:complexType>
</xsd:element>

The fields defined in the above record are:

  • "Link": URL of the link
  • "Available": "a" for available, and "na" for not available (dead link)
  • "ParentLink": URL of page where the link is found,
  • "State": internal, external, excluded, or error
  • "Remarks": the number of links in the page.

"Link" is used as a primary key to find records.

An example of the partial output data is as follows:

<ParentTable diffgr:id="ParentTable1" msdata:rowOrder="0">
<Link>http://localhost/c/delta/webexplorer/satunet_com.htm</Link>
<Available>[a]</Available>
<ParentLink />
<State>has</State>
<Remarks> has 112 links.</Remarks>
</ParentTable>
<ParentTable diffgr:id="ParentTable2" msdata:rowOrder="1">
<Link>satunet_com_files/global.css</Link>
<Available>[na]</Available>
<ParentLink>http://localhost/c/webexplorer/satunet_com.htm</ParentLink>
<State>[exc]</State>
<Remarks />
</ParentTable>

The method WriteLog in the class Results shows how the DataRow class is used:

public static void WriteLog(String
            addr,String Parentaddr,String Stateaddr,String Remarks) {

ObjPrimary[0]=addr;
ResultRow=ResultTbl.Rows.Find(ObjPrimary);
if
            (ResultRow!=null) {
if (Stateaddr=="[err]" &&
ResultRow["Remarks"].ToString().IndexOf(Remarks)!=-1) {
return;
} else {
if (Stateaddr=="[na]" ||
            Stateaddr=="[a]")
ResultRow["Available"]=Stateaddr;
else
            {
if (Stateaddr=="[exc]")
            ResultRow["Available"]="[na]";
ResultRow["State"]=Stateaddr;
}
ResultRow["Remarks"]=ResultRow["Remarks"]+"
            "+Remarks;
ResultTbl.AcceptChanges();
}
} else
            {
ResultRow=ResultTbl.NewRow();
if (Stateaddr=="[na]" ||
            Stateaddr=="[a]")
ResultRow["Available"]=Stateaddr;
else
            {
if
            (Stateaddr=="[exc]")
ResultRow["Available"]="[na]";
ResultRow["State"]=Stateaddr;
}
ResultRow["Link"]=addr;
ResultRow["ParentLink"]=Parentaddr;
ResultRow["Remarks"]=Remarks;
ResultTbl.Rows.Add(ResultRow);
ResultTbl.AcceptChanges();
}
}

The code is very similar in writing to traditional databases. First we use the method Find based on the primary key to see if a record exists, if not a new record is created, otherwise it is updated. To finalize the changes, we use the method AcceptChanges.

The Console application version

We will first describe the console application version. It is identical to the Web Service version, except that it uses a text file as output instead of a DataSet. We found that this version is worth doing because it is faster, and does not time-out easily compared to the Web Service version. The output text file is named " SpiderLog.txt ", it is written in the same directory as the program, and it will overwrite any file of the same name without warning. This text file has one record in each line, and each field is tab delimited. If you compare the text file to the XML produced by the Web Service against the same URL, the text file will in general have more lines because the same URL may appear in several lines. In the DataSet version the same URL will appear in one record only.

The spider.exe in action

The console application version takes two parameters: the URL address, and the option to observe robot Meta tags or not. The enclosed download material contains the console application source code and compiled exe file. This exe file can be run from the DOS prompt window on any computer where the .NET Runtime is installed.

Conversion into a WebService

A Web Service is a programmable entity that provides a particular element of functionality, such as application logic, and is accessible to any number of potentially disparate systems through the use of Internet standards, such as XML and HTTP. In .NET, a Web Service is written as a text file with the extension .asmx. This is similar to .aspx files, except for the additional WebService declaration in its header, and a WebMethod attribute for the Main method. The .asmx can then be called directly on the web. Compilation occurs automatically, and the WSDL (Web Service Declaration Language) description of the Web Service is taken care of by the .NET Framework.

<%@ WebService Language="C#"
            class="Spider" %>
using System;
using System.Net;
using
            System.Web.Services;
using System.Collections;
using
            System.IO;
using System.Text;
using
            System.Text.RegularExpressions;
using System.Data;
using
            System.Timers;

[WebService(Namespace="Interdata")]

public
            class Spider : WebService
{
[WebMethod] public DataSet
            Main(String Link, String Robot)
{............etc

The above snippet shows all the class libraries used by the program:

Class Library | Classes used
System.Net | WebRequest and WebResponse
System.Web.Services | WebService
System.Collections | HashTable, ArrayList
System.IO | FileStream, StreamReader
System.Text | StringBuilder
System.Text.RegularExpressions | Regex, Match
System.Timers | Timer
System.Data | DataSet, DataTable, DataRow, DataColumn

When running the Web Service, we encountered problems with time-outs. You will need to adjust the time-out parameter for your server if you want to try it for a lengthy crawl. To make it easier, we decided to add the time-out as an additional input parameter to the Web Service, which the user can set him/herself. The Web Service has three parameters: the root URL, Robot observance option, and the timeout in milliseconds. If the robot observance option is "y," the Meta tags will be observed, and if it is "n", they will be disregarded. The following screen capture shows this:

Screenshot of the Web Service Method

The Web Service Spider.asmx can be called using http://servername/spider.asmx?WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0 and then click on the method, or call it directly: http://servername/spider.asmx?op=Main&WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0. Alternatively, you may want to create a client proxy class using the Web Service Description Language Tool (Wsdl.exe) of the NET Framework. After the proxy stub is compiled, it can be called by the client, and the proxy will in turn make the remote call to the Web Service.

Robot exclusion

Robots are not always welcome to web pages (for one thing they could mess up the web statistics, not to mention a Denial of Service attack), and it is generally accepted that robot exclusion rules are to be followed, even though they are not legally binding. These rules can be seen in the article "A standard for Robot exclusion" (http://www.robotstxt.org/wc/norobots.html?WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0). It works by creating a text file " robots.txt " in the server where the webpage is located. For example, a robots.txt containing:

User-agent: *
Disallow: /

will not allow robot navigation.

A second method of robot exclusion is by using html Meta tags (http://www.robotstxt.org/wc/meta-user.html?WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0) embedded within the web page itself. This has the advantage of circumventing write access restrictions to the robots.txt file. For example, the following html contains a robot exclusion Meta tag.

<html>
<head>
<meta name="robots" content="noindex,nofollow">
<meta name="description" content="This page ....">
<title>...</title>
</head>
<body>

Our program uses the second method only, it will ignore robots.txt files. For this reason, we would not recommend using the program in the real world. We used it for intranets and for our own sites to find dead links. In the last case, we need to ignore robot exclusion Meta tags. Because of this, the second parameter of the Web Service is a flag for whether robot Meta tags are to be observed or not. If you enter "y", and the Meta tag says "NOINDEX" or "NOFOLLOW" then the page will not be investigated.

Concluding remarks

We have found the program quite useful as web masters for finding dead links. To do this, we import the XML into Microsoft Excel and sort on the field "Status". If we use the console application version, the text file Spiderlog.txt can similarly be imported into Excel with the same results. It is also useful to build a site index.

Future enhancements and modifications to the programs may include use of threading for better efficiency, building an index for later searching, or finding "web bugs". An option to save the DataSet to a persistent storage may also come handy. The enclosed supporting material includes the asmx file, the cs file for the console version and its compiled exe file and a readme.txt file containing instructions on how to use them.


Manipulating Binary File Data

Manipulating Binary File Data
http://www.csharptoday.com/content.asp?id=1659
Article Rating: 3
Published on September 24, 2001
Abstract
In one of Matthew Reynolds earlier articles, he showed us how we can save and load document data to a file. This data was stored in a binary format that mimics the way your computer stores the data in memory. This can be easily achieved in .NET, providing that we understand how data is stored within the file. What we haven’t seen is how to manipulate data that is in a specific format, but where that format is not designed by us or stored in a binary format. In this article, Matthew shows us how we can use various classes in the Framework to read binary data directly from a very popular file format: MP3.
Article

In my Serialization article published earlier this month, we saw how we could save and load document data to a file. This data was stored in a binary format that mimicked the way your computer itself stores the data in memory.

We've seen how in .NET this can easily be achieved - providing that we understand how data is stored within the file. We've also seen through myriad other articles how easy it is to manipulate data intended for information exchange through XML. What we haven't seen is how to manipulate data that is in a specific format, but where that format is not designed by us or stored in a binary format. In this article I'll show you how you can use various classes in the Framework to read binary data directly from a very popular file format: MP3.

In an imperfect world, these skills will be essential. You can't guarantee that you're going to be given data to process that's stored in a safe, easy format that fits neatly into the .NET world. You might have to get your hands dirty!

Most CD rippers will embed details about the track when the MP3 file is extracted from the CD. In this screenshot, you can see Winamp's ID3 tag editor for a Portishead track.

In the wonderful world of MP3 tags, there are two general formats: ID3v1 and ID3v2. In this article we're going to concentrate on ID3v1 because it's both the most prevalent and the easiest to work with. ID3v2 tags offer the possibility of carrying more information, as you can see from the screenshot above.

If you're in the relatively unusual position of not having any MP3 files on your computer, try downloading Easy CD-DA Extractor from http://www.poikosoft.com/cdda/ and rip some tracks from one of your favorite CDs. Remember to check on the Enable ID3 tag option when you extract the files.

The ID3v1 file follows a fairly basic binary format. 128 bytes in from the end of the file, the word TAG will appear. (If this isn't present, you can assume the file does not contain an ID3v1 tag.) After this, you have:

  • Song title (30 characters)
  • Artist (30 characters)
  • Album (30 characters)
  • Year (4 characters)
  • Comment (30 characters)
  • Genre (1 byte)

Here's what the data looks like in Visual Studio's hex editor:

To open the file in Visual Studio.NET's hex editor, rename the file and put a .bin extension on the end. If you're not using Visual Studio, try Hex Workshop or UltraEdit, both of which you can download from download.com. When you try and open the file Explorer, it will prompt you for an application. Select Microsoft Visual Studio.NET 7.0 and the file will open. Remember to rename the file back again afterwards.

On the screenshot, you can see that I've highlighted the TAG . Immediately following this is the name of the track ("Numb"). Then, we pad out the remainder of the 30 characters with spaces (Hex code 20 on the hex view). Next comes the artist name, then the album name, the year, a blank comment field, and finally the genre. The genre is given as 0x0D , which in decimal is 13 . This represents "Pop". In all there are 148 genre codes, but I won't be reproducing them here!

Building our Extractor

Create a new Visual C# - Windows Application project. Call it MP3 Extractor .

When the Designer appears, add a ListView control and a Button control. Change the Name property of the ListView control to listviewData and change the name of the Button control to buttonOpen .

Add two new columns to the ListView control by editing the Columns property. Both should have their Width set to 200 .

Finally, draw a new OpenFileDialog control onto form. Change its Name property to dialogOpenFile .

Adding Data to the ListView

You've probably guessed that we're going to write MP3 data to the ListView control. To do this, we're going to create a new class called Mp3DataListViewItem that derives from System.Windows.Forms.ListViewItem . This will make it easy for us to add a name/value pair to the ListView in a single move.

Create a new class called Mp3DataListViewItem . Set it to inherit from ListViewItem :

public class Mp3DataListViewItem : System.Windows.Forms.ListViewItem
{

Then, add this constructor:

      public
            Mp3DataListViewItem(String dataName, String dataValue)
{
//
            set it up...
this.Text =
            dataName;
this.SubItems.Add(dataValue);
}

What this constructor will do is update the value for the first column through the Text value. It will then add a new sub-item to the list, which represents the second column.

Namespaces

Before we start coding in earnest, we need to include the System.IO and System.Text namespaces in our project. Add this code to the top of the code listing:

using System;
using
            System.Drawing;
using System.Collections;
using
            System.ComponentModel;
using System.Windows.Forms;
using
            System.Data;
using System.IO;
using System.Text;

Reading Data

Reading text data from a file is remarkably easy, although you do have to go through a few strange hoops. Microsoft has been extremely keen to adopt an open and flexible architecture for handling strings in .NET. This may well be down to a motivation to ensure that .NET will port to other platforms that handle strings differently to the way that Windows and most other desktop/server systems do. The upshot of this is that you can't just read bytes from a file and say, "OK, turn that lot into a string". Instead, you have to read the bytes, use a separate class derived from System.Text.Encoding to turn them into an array of System.Char objects, and then somehow mangle the array into a string.

On the Designer, double-click on the Open Button control. The first thing we need to do is open the file, so add this code:

      private
            void buttonOpen_Click(object sender, System.EventArgs e)
{
//
            open the file...
dialogOpenFile.Filter = "MP3 Files
            (*.mp3)|*.mp3|All Files (*.*)|*.*||";
DialogResult result =
            dialogOpenFile.ShowDialog();
if(result ==
            DialogResult.Cancel)
return;

Once we've told it that we want to load a filename, we need to clear the list. We also add a new Mp3DataListViewItem object to the ListView control to represent the filename.

// clear the information...
listviewData.Items.Clear();
listviewData.Items.Add(new Mp3DataListViewItem("Filename",
dialogOpenFile.FileName));

We can then open the file in the normal way:

// open the file...
FileStream stream = new FileStream(dialogOpenFile.FileName, FileMode.Open);

The ID3v1 tag appears 128-bytes from the end of the file. We can use the Seek method to move into position, like this:

// move 128 bytes in from the end...
stream.Seek(0 - 128, SeekOrigin.End);

Once there, we need to read three bytes. If there is a tag, this byte array will contain the string equivalent of TAG .

// read three bytes...
Byte[] tag = new Byte[3];
stream.Read(tag, 0, 3);

To convert a byte array into a string, we're going to use a function called ConvertBytes . We'll build this in a little while:

// convert the data...
String tagName = ConvertBytes(tag);

If tagName reads TAG , then we know we have an ID3v1 tag. If so, we will use another method called ReadTagData that will read data from the stream, convert it into a string, and add it to the ListView. Because this method has to add the pair to the ListView, we pass the name through to this method.

// do we have a tag?
if(tagName == "TAG")
{
// get the data...
ReadTagData(stream, "Title", 30);
ReadTagData(stream, "Artist", 30);
ReadTagData(stream, "Album", 30);
ReadTagData(stream, "Year", 4);
ReadTagData(stream, "Comment", 30);
}
else
MessageBox.Show(this, "The file does not contain an MP3 tag.",
"MP3 Extractor", MessageBoxButtons.OK, MessageBoxIcon.Exclamation);

Finally, we can close the file:

// close the file...
stream.Close();
}

ConvertBytes is the interesting method here. Let's build this now.

// ConvertBytes - take a byte array and turn it into data...
protected String ConvertBytes(Byte[] data)
{

The first thing we need is a class derived from System.Text.Encoding that is capable of transforming an array of bytes into an array of System.Char objects. As the data is stored in ASCII format, we'll use System.Text.ASCIIEncoding .

// use encoding...
ASCIIEncoding encoding = new ASCIIEncoding();
Char[] dataChars = encoding.GetChars(data);

Once we have an array of System.Char objects, we need to transform it into System.String . One way of doing this is to use a System.Text.StringBuilder object. This has an Append method that takes an array of System.Char objects and adds them to the string.

// use a builder to turn this into a string...
StringBuilder builder = new StringBuilder();
builder.Append(dataChars);

You can extract the string built up using a StringBuilder , by using to ToString method.

// get a string value back...
String dataString = builder.ToString();

However, as this string is padded with spaces, we need to use Trim to remove them.

// trim it...
dataString = dataString.Trim();

Finally, we return the decoded and trimmed string:

// return it...
return dataString;
}

ReadTagData is the method that extracts data from the stream, converts it, and adds it to the ListView. Firstly, we create a new byte array:

// ReadTagData - read data from the stream...
protected void ReadTagData(Stream stream, String name, int bytesToRead)
{
// create an array...
Byte[] data = new Byte[bytesToRead];

We then read the data from the stream:

// read the data...
stream.Read(data, 0, bytesToRead);

Next we convert the data using ConvertBytes :

// convert it...
String dataString = ConvertBytes(data);

Finally, we can add the data to the ListView by creating a new Mp3DataListViewItem object.

// add it...
listviewData.Items.Add(new Mp3DataListViewItem(name, dataString));
}

If you run the project and open a file that you know contains an ID3v1 tag, you'll see something like this:

Summary

In this article we took a closer look at the more exoteric features of System.IO.Stream to read bytes directly from a stream. We then converted those bytes into String objects using StringBuilder and ASCIIEncoding .


XML Documentation Using C# Compilers

XML Documentation Using C# Compilers
http://www.csharptoday.com/content.asp?id=1652
Article Rating: 4.5
Published on September 11, 2001
Abstract
In this article, Salim Naim takes a look at XML documentation using C# compilers, a useful solution to the often tedious process of documenting code. Documentation is very important, particularly when a developer has to work with another developer's code. The advent of XML documentation is a big help in this area.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

Getting down to the often tedious process of documentation, is for most programmers, something we feel we can't be bothered with. The result of this lethargy has produced, on the whole, rather shoddy documentation. Despite this, we can appreciate the value of sound documentation, because the success of our code depends on it in many instances, especially when a developer has to work with another developer's code. Fortunately, with the advent of C# and XML Documentation support the documentation process has never been easier. This solution allows for documentation to be integrated with the development environment.

For years programmers have had to manually manage code documentation with external tools like word processors. Due to the lack of integration between the developing environment, programming language, and the documenting application, the manual process has been time-consuming and inefficient. C# allows for total integration of all three areas, which facilitates writing documentation.

XML documentation is a concept introduced in C#, which provides compiler support for source code documentation. The output documentation is presented in an XML document.

XML documentation is based on standard schemas, which is advantageous, as it can be rendered with a XSLT style sheet that the audience is accustomed to, or one which is based on a standard format that a company or group complies with.

XML Documentation

To generate XML Documentation, the compiler must be executed with the /doc:{file name} switch from the command line. From the Development Environment (IDE) of Visual Studio .NET (VS.NET), this option can be set in the Configuration Properties of the project properties.

Figure 1

Once the XML documentation file name has been set as the compiler option, you simply type in three forward slashes beside the source code and add the documentation within the documenting tag.

///<summary>
///<remarks>This is the overview of the documenting code</remarks>
///</summary>

The C# compiler determines the basic structure of the XML document. The document structure is not hierarchical in the traditional sense that classes contain collections of properties, methods, events, etc. The XML document is a flat document, based on a root element called <doc> that contains the documentation of an assembly. The element <doc> contains a child collection of <member> (member) elements. A member element describes the intended source code entity. Since the XML document is not hierarchical and all code entities are generated as member elements, the compiler therefore generates name attributes for each <member> element and prefixes the value of the attributes with a single character identifier based on the type.

Figure: 2

Note: a full listing of the prefix characters with their explanations are available in the .NET Framework SDK documentation. (The above figure is from the .NET Framework SDK documentation.)

The following sample is the output XML documentation of a class that contains a single default constructor.

  • The first <member> element in the figure above is the overview documentation for a class, therefore the generated name attribute is prefixed with the character T.
  • The second <member> element is the documentation for the default constructor of the class, and its identifier is prefixed with the character M. The #ctor specifies that this member is not a regular class member function, but is rather the constructor of the class.

The C# compiler provides sixteen standard tags for documenting C# code, seven of which have their syntax verified upon compilation. The verifications the compiler executes are:

  • Tag name and attribute verification based on the standard tags schema
  • Cross-checking between function declaration and Member documentation
  • Accuracy and adherence of the standard XML tags and custom tags

Note: All XML Documentation errors encountered during the verification stage of the compilation are flagged as compiler warnings, or marked as errors in the output document if their content is not well-formed.

As a programmer, you aren't limited to using only the sixteen standard tags that C# provides, you can extend the elements within the <member> element with any custom well-formed XML tag.

Note: Even though you have the flexibility of extending the documenting tags, it is unadvisable to customize, since custom stylesheets (XSLT) would then be required to handle the additional elements, defeating the purpose of generic documentation formatters.

XML Documentation Standard Tags

The standard tags that the C# compiler provides can be broken down into four main categories (the following classifications are not Microsoft's identifications of the tags, these classifications are used only within this article to aid the reader):

  • <member> element child tags
  • Data Formatters
  • Document referencing
  • Utility tags

<member> element child tags

Member element child node tags are tags that are produced within the member element. These tags provide the basic method of identifying and representing the member element. One way of thinking about these tags is that the member tag is comparable to the <HTML></HTML> tag, and the child nodes are the different tags that can be used within this element.

The following is a listing of the member element standard child tags:

  • <summary>: This tag is a brief overview of the member.
  • <remarks>: This tag is used when you would like to make a general comment to the reader.
  • <param>: This tag provides parameter name information about the member, and is verified by the compiler. For multi parameter members, it's important to note that the elements are not represented hierarchically within a collection, but are repeated accordingly based on the left to right declaration order of the method.
  • <returns>: This tag is used to return information on a member and is not a verified tag by the compiler. Therefore a void method can still have a <returns> tag associated with its member element; however the VS.NET doesn't provide the tag through Intellisense.
  • <permission>: This tag is permission information about a member. The compiler verifies this tag.
  • <example>: This tag can be used to provide the reader with a code example using the member. The tag is always used with the <code> tag. (The <code> tag will be discussed in the 'Data Formatters' section of this tutorial).
  • <exception>: This tag is a reference to the Error type information that will be thrown when errors occur within the member.
  • <value>: This tag is used to describe a member element that documents a class property.

The following is a snippet from the documentation of a standard method, which shows the use of the tags described above.

Data Formatters

The data formatting standard tags are tags that are used within the member element child tags, and provide additional information about the text for the formatting style sheets.

Note: the sample stylesheet provided with the article extends these tags and provides additional formatting support for XHTML.

The following is a listing of the standard data formatting tags:

  • <c> and <code>: These tags are formatting tags that indicate that the text used within the element is code. When formatting stylesheets, these tags render the present information accordingly. The <c> tag is a single line identification tag and <code> is a multi-line tag.
  • <paramref>: This tag is a data formatting tag that provides a method for identifying a reference to a <param> tag within the text. This tag is verified by the compiler and will generate a warning if its name attribute doesn't exist.
  • <list>: This tag is best compared to the <ul> and <li> tags of HTML. They provide 'list formatting' identification.

The following is a snippet of the standard Data Formatter tags:

Document referencing

The document referencing tags that support XML documentation are comparable to the <a> tag anchor behavior of HTML, and allow cross referencing between member elements. The compiler verifies the cref attribute of these tags upon compilation.

  • <see> and <seealso>

Note: The cref attribute is verified at compilation; however, the compiler doesn't reference the cref value if the referencing method is not XML documented.

The following is a snippet of the document referencing tags:

Utility tags

The last category that standard XML documentation tags fall under is utility tags. In Beta 2, only one tag belongs to this classification, the <include> tag. This tag allows external XML documents to be included as part of the documentation. The compiler will load the file attributes file, and use the path attributes XPath query to locate the appropriate documentation.

Note: The <include> tag in my opinion is not recommended because it breaks the integration which should exist between the IDE and the document. The purpose of this tag is to simplify the code and to eliminate clutter in the workspace. If a cluttered workspace becomes an issue for you, then the VS.NET feature can resolve this problem as the pre-processor directive #region enables you to create collapsible regions that hide underlying text.

The Sample

The best way to learn XML documentation, and to have an understanding of the XML document that the compiler produces, is via an example. For demonstration purpose we will implement and document a class named ShoppingCart. This class will contain constructors, Properties, Methods, Events and code regions using VS.NET pre-processor directives.

Note: The sample accompanying this article contains a generic formatter XSLT stylesheet that can render most assemblies in XML documentation. This stylesheet offers great scope for innovating and creating your own custom formatter. To demonstrate how you can extend the functionality of XML documentation standard tags, this style sheet extends the <remarks> tag, enabling this tag to contain text as well as any valid XHTML.

The ShoppingCart Class

The ShoppingCart class used in this article is a class containing the following:

Constructors

Two constructors are implemented in the ShoppingCart class, one is a default constructor, the other instantiates the class based on a string argument.

Documenting constructors is an important topic, since the C# compiler identifies the Method and Constructors with the M character Identifier (refer to Figure 2). It is interesting to differentiate between a regular method and a constructor, and this is solved with the compiler adding #ctor after the class name. E.g. M:ShoppingCart.#ctor, indicates that this member element is the default constructor for the ShoppingCart class.

Properties

The ShoppingCart class implements a single property that illustrates handling of P character identified member elements.

Methods

Three methods are implemented with the ShoppingCart class, and their method illustrates the use of parameters <param>, code examples <example>, and returns <returns>.

Events

Events are also implemented in the ShoppingCart class.

Before devouring the sample and implementing an XML documentation style sheet, it's important to understand how the stylesheet will identify <member> elements within the <doc> document.

XML documentation is not hierarchal as we mentioned earlier, and the C# compiler generates character identifiers as a prefix to the qualified path of the member. Each prefixed character identifier represents a code entity (refer to Figure 2). E.g. <member name="P:ShoppingCart.Cookie"> is the property Cookie member in the ShoppingCart class.

To identify a member and categorize it correctly with its respective code entity, the string manipulation routines of XSLT starts-with() and substring-after() are used along with XPath queries to navigate within the XML Documentation document. E.g. The following XPath Query locates all class or T identified member elements "members/member[starts-with(./@name,\'T'>./@name,"T:")] "

The following is a snippet from the ShoppingCart class in XMLDoc.cs.

// A delegate type for event
            notifications
/// <summary>
///
            GenericEventHandler
/// </summary>
public delegate void
            GenericEventHandler(object sender, EventArgs e);

#region
            ShoppingCart overview
/// <summary>
/// This class is a
            sample Shopping cart class, for an online store.
///
            </summary>
/// <example>
/// using
            XMLDoc.ShoppingCart;
/// ..
/// ShoppingCart shoppingCart =
            new ShoppingCart();
/// //create the cart with a cart id based on
            a cookie
/// ShoppingCart shoppingCart = new
            ShoppingCart("cookie:0001");
///</example>
///<remarks>
///<font
            color="red"><b>This is a dummy class to demostrate working
            is XML
///Documentation</b></font>
///</remarks>
#endregion

class
            ShoppingCart
{
private string cookie;

//event handlers
public event GenericEventHandler Insert,Delete;

  • The above code starts by declaring a GenericEventHandler delegate for the events implemented within the ShoppingCart class, a simple summary is provided for this type.
  • The remaining code is the declaration of the ShoppingCart class; a summary of the class is provided along with an example and remarks comments for documentation. The comments associated with this declaration are encapsulated in a region "ShoppingCart Overview"

The following is a snippet from formatter.xslt:

<xsl:for-each select="$classMethods">
<TR bgcolor="#f5f5f5">
<TD>
<UL>
<LI>
<U><FONT color="blue">
<xsl:value-of select="substring-
after(@name,concat($cEvents,$classname,'.'))" />
</FONT></U>
</LI>
<xsl:call-template name="OutputDetails">
<xsl:with-param name="member">
<xsl:value-of select="." />
</xsl:with-param>
</xsl:call-template>
</UL>
</TD>
</TR>
</xsl:for-each>

The above transformation renders the class declaration Details with a template called OutputDetails.

Working with the sample

To work with the sample provided, Beta 2 of Visual Studio .NET and the framework must be installed. The sample class ShoppingCart can be compiled via VS.NET, and the following line must be added to the produced XML document.

<?xml version="1.0" ?>
<?xml-stylesheet type="text/xsl" href="formatter.xslt"?>

<doc>
....

The processing instruction presented above applies the formatter.xslt stylesheet, the formatter accompanying the sample. Once this instruction has been added to the XML file, the transformation can be viewed in Internet Explorer (IE 6.0 is the recommended browser for this style sheet).

Summing It Up

Whilst documentation tools have been available for current programming languages for a long time, XML documentation is sure to be appreciated, especially amongst C# programmers. C# and XML documentation bridges the gap between the development environment and the documenting tool which presents the information in the most suitable and exchangeable format, XML.


Introduction to Managed C++

Introduction to Managed C++
http://www.csharptoday.com/content.asp?id=1619
Article Rating: 4.5
Published on August 1, 2001
Abstract
C++ is a language which divides programmers into two camps – those who love it and those who hate it. It is, of course, one of the most powerful languages available after bare bones C and assembly, but is also very complex and offers many opportunities for developers to shoot themselves in the foot. This is ironic because C++’s best features are considered by most to be its worst. One of the biggest issues with C++ has been memory and resource management, where the developer is in complete control. While this is a boon for some, it’s a bane for most. Realizing this, Microsoft knew that it needed to make life simple for C++ developers, especially with the advent of its .NET technologies. The solution was an enhancement and addition to ANSI C++, called Managed Extensions to C++ (also known as MEC or as Managed C++). This brings all the support of the .NET framework, ideology and methodology to the C++ language. In this article, Kaushal Sanghavi explores some of these new capabilities that C++ now supports.
Article

Introduction

C++ is a language which divides programmers into two camps - those who love it and those who hate it. It is, of course, one of the most powerful languages available after bare bones C and assembly, but is also very complex and offers many opportunities for developers to shoot themselves in the foot. This is ironic because C++'s best features are considered by most to be its worst. One of the biggest issues with C++ has been memory and resource management, where the developer is in complete control. While this is a boon for some, it's a bane for most. Realizing this, Microsoft knew that it needed to make life simple for C++ developers, especially with the advent of its .NET technologies. Since C++ is an ANSI standard and is not Microsoft proprietary, it could make a change to the language itself. The solution to this was an enhancement and addition to ANSI C++, called Managed Extensions to C++ (also known as MEC or as Managed C++). This brings all the support of the .NET framework, ideology and methodology to the C++ language. In this article, we will explore some of these new capabilities that C++ now supports; however as the title suggests, this article is an introduction to this new language and we will refrain from discussions about the advanced features and uses of Managed C++.

Note that this article assumes a basic knowledge of C or C++ and the .NET framework, and focuses entirely on Microsoft's compiler for Managed C++ that ships with Visual Studio .NET and the .NET Framework. It is highly possible that other vendors might announce support for Managed C++ in the future and most of the concepts should remain the same.

Managed C++

Some of the difficulties associated with C++ are:

  • The lack of automatic memory management support. Developers are forced to allocate and deallocate memory (and other resources), and this can be extremely difficult. There are very few C++ developers who have never deleted memory that is already in use, or have never accessed memory that is already deleted, or have never forgotten to deallocate free memory. Memory leaks and dangling pointers are the source of most of the issues in C++ development.
  • Another issue has been the use of C++ code from non-C++ languages. Before .NET, the only way a VB object could use the services of a C++ object would be via dispatch interfaces in COM, or via exporting public non-class functions.
  • Native C++ did not support a consistent form of error handling. While there was support for throwing and trapping exceptions, there were enough loopholes to allow developers to shoot themselves.
  • Further, with the advent of .NET, none of the .NET languages allowed developers to mix native and .NET code. While Microsoft supports interoperability with classic COM, you could not write code that was partly unmanaged and partly managed.

Managed C++ attempts to solve all these problems and a few more. In the following sections, we will focus on these features and drill down on Managed C++'s support for them. However, in the traditional norm of introducing new languages, let us start with a simple Hello World program written in Managed C++. (Note that for a Managed C++ application to execute, the .NET framework needs to be installed. Further, since we are working with a Beta release, the differences between a traditional C++ compiler and a managed C++ compiler are not well known yet, and comparisons may even be unfair).

VS .NET natively supports Visual C++ projects. These range from classic COM projects using ATL, to MFC based projects, to the standard .NET projects. For our sample, we chose the Managed C++ Application from the New Project dialog box (from the File->New->Project menu).

Selecting this project, you get a simple application that writes out Hello World to the console using the System.Console class from the .NET class library. The screen shot below shows the code and the files that are generated by VS.NET. (For the sake of simplicity, we have deleted some of the files that are required for supporting precompiled C++ headers.)

As you can see from the above screenshot, the code looks like some very standard C++ code with a mix of some .NET code. The first thing that stands out for a C++ developer is the #using and the using namespace syntax.

  • #using: This keyword tells the compiler to load the metadata for the referenced managed assembly. This is necessary so that you can refer to the types in the referenced assembly. In this example, mscorlib.dll is the binary that contains all the basic .NET types.
  • using namespace: This is similar to .NET code, where all types are contained within a namespace. The using namespace is shorthand for referring to the types in a namespace without having to use the namespace's name. Thus since we have the using namespace System directive, we can refer to the Console class directly, as opposed to using System.Console.

Managed and Unmanaged Code

Managed C++ is the only language that allows developers to write managed and unmanaged code within the same executable (or assembly in .NET parlance). VB .NET allows only managed code; it does allow interoperating with classic COM code, but that is via Runtime Callable Wrappers. C# allows managed code and interoperability with classic COM via RCWs, and it also allows unsafe code that operates on pointers. However, it is impossible to write native (unmanaged) code in VB .NET or C#. This can be one of the biggest motivations for using Managed C++. The Managed C++ compiler can recognize and compile both managed and unmanaged code, with a simple pragma in your code. By adding the following two lines in your source files, you can write both managed and unmanaged code.

#pragma managed
#pragma unmanaged

This tells the compiler that the classes contained within this source file will use a mix of managed and unmanaged code. Further, when you use one of the managed C++ projects in VS.NET, you can turn support on and off for managed code via the Project Properties as well as shown below.

One of the biggest issues with mixing managed and unmanaged code is memory management. Unmanaged code relies on the developer, while managed code relies on the .NET Garbage Collector and totally absolves the developer from this responsibility. Since the compiler cannot guess whether a class is managed or not, you have to explicitly specify whether instances of this class will be garbage collected or explicitly handled by the developer. Managed C++ introduces two new keywords, __gc and __nogc for this purpose. By specifying the __gc keyword on a class, the class becomes a managed class and you don't need to free up instances of this class explicitly. The object will be allocated on the managed heap and will be automatically released by the garbage collector when all references to the object are released. You create the object using the new keyword, but you don't need to use the delete keyword to release the object. However, if you do use the delete keyword, the object will be destroyed immediately. Since the .NET garbage collector runs non-deterministically, you may want to do this for objects that consume a large amount of memory or other resources.

__gc class
            GCClass
{
public:
GCClass(void);
~GCClass(void);

int
            SomeMethod();
};

GCClass::GCClass(void)
{
//
            constuctor
}

GCClass::~GCClass(void)
{
//
            destructor
}

int GCClass::SomeMethod ()
{
return
            1;
}

The code above shows a managed class. Note that the class declaration and definition is identical to that of a normal C++ class, the only difference being the __gc keyword. Creating and invoking a method on this class is shown below. Note that while you can call delete on lGCObj, you are not required to.

GCClass* lGCObj = new GCClass
            ();
lGCObj->SomeMethod ();
// delete lGCObj;

All managed classes derive from the System.Object class. As you can see from the code above, the GCClass does not derive from this class. This is automatically done for you when you specify the __gc keyword on the class. You can see this by looking at the methods that are displayed by the intellisense feature in VS.NET. The displayed methods are those that are inherited from the System.Object class.

If you want to declare an unmanaged class, you can use the __nogc keyword. However, this is the default, and if you don't specify any keywords on a class, the class is treated as an unmanaged class. Thus the following two lines of code are identical.

class UnmanagedClass { . }
__nogc class UnmanagedClass { . }

Since Managed C++ allows the writing of both managed and unmanaged code, it is possible to write code that passes parameters from a managed method into an unmanaged method. While this works fine for native value types, passing pointers becomes an issue. Consider the example of passing a managed pointer into an unmanaged method. Since the .NET garbage collector knows nothing about unmanaged code and vice versa, the garbage collector can collect (free) this managed object if there are no other outstanding references on it. Since the garbage collector does not know about the unmanaged code, it does not and cannot track object references from unmanaged code. This can cause the unmanaged code to blow up if the garbage collector decides to collect the managed object before the unmanaged code has completed its use of this object. In order to avoid this situation from happening, Managed C++ introduces a keyword called __pin. This pins a managed object to a fixed location in memory and prevents the garbage collector from destroying this object. When the pinning pointer goes out of scope, the object is unpinned.

__gc class GCClass
{
public:
int mData;
};

__nogc class UnmanagedClass
{
public:
int SomeMethod (int* pData);

};

GCClass* lGCObj = new GCClass ();
GCClass __pin *lPinnedObj = lGCObj;
UnmanagedClass* lObj = new UnmanagedClass ();

lObj->SomeMethod (&lGCObj->mData); // This gives a compiler error

lObj->SomeMethod (&lPinnedObj->mData); // right way to pass pointers

The above code block shows how the __pin keyword can be used. The UnmanagedClass declares a method that takes in a pointer to an integer. Before passing this pointer to this method, the managed object should be pinned to ensure that the object will be alive for the duration of the unmanaged method. The compiler checks for this and if the pointer is not pinned, it generates the following error.

error C2664: 'UnmanagedClass::SomeMethod' : cannot convert parameter 1 from 'int __gc *' to 'int *'

Properties

The .NET framework introduces the notion of properties. In addition to methods and data members, classes can also support properties, and there is a specific syntax for invoking a property. In addition, C# and VB.NET support explicit keywords for declaring properties. Managed C++ introduces the __property keyword to indicate that a method is a property, and can be accessed using the Get/Set property syntax. Further, like other .NET languages, Managed C++ allows read only and write only properties as well.

__gc class GCClass
{
public:
__property void set_MECProp (int iValue);
__property int get_MECProp ();

};

The above code block shows how you can use the __property keyword to declare a property with get and set methods. The Get method has to have a get_ prefix and the Set method requires a set_ prefix. Once this property is declared, you can access this as follows:

CClass* lGCObj = new GCClass ();

lGCObj->MECProp = 12;
int lValue = lGCObj->MECProp;

As you can see, invoking a property is identical to accessing a public data member. In fact, this is exactly what the Managed C++ compiler does under the covers. Every time the Managed C++ compiler encounters a __property keyword, it creates a data member for the class with the property name. The following screen shot proves this by looking at the intellisense feature in VS.NET.

This can further be proven by looking at the assembly using the ILDasm tool as shown below:

As you can see from the above screen shot, GCClass contains two public methods get_MECProp and set_MECProp and a public data member MECProp. Clients of this class can call either the method or the property syntax to get or set the property.

Exception Handling

Managed C++ supports both Win32 Structured Exception Handling (SEH) and the native try\catch exception handling. The try\catch construct also supports the __finally keyword, which denotes a block of code that will always be executed irrespective of any exceptions. If no exception occurs, control reaches the __ finally block at the end of the try block, and if there is an exception, control reaches the __finally block at the end of the catch block. Further, Managed C++ supports all the .NET exceptions, i.e. all the classes that derive from the System.Exception class.

#using <mscorlib.dll>
using
            namespace System;

__gc class GCClass
{
public:
void
            ExceptionMethod ()
{
throw new Exception ("some
            exception");
}
};

int main(void)
{
GCClass*
            lGCObj = new GCClass
            ();

try{
lGCObj->ExceptionMethod();
}
catch
            (Exception* e){}
__finally {
// do cleanup
            here;
}

return 0;
}

The above code block demonstrates the use of .NET exception handling in Managed C++. As you can see, this is identical to native C++ exceptions; the only difference being that we use the #using <mscorlib.dll> directive to gain access to the .NET class hierarchy. The using namespace System allows us to refer to the Exception class without having to use the System.Exception syntax. The __finally clause should handle resource cleanup. Since this clause is guaranteed to execute, this is the ideal location to put clean up code.

Interfaces

Native C++ did not support interfaces. A workaround that has been employed by C++ developers to counter this has been via the use of abstract base classes and pure virtual methods. The .NET framework introduces a much cleaner way to handle interfaces, and Managed C++ supports this via the __interface keyword. By marking a class with the __interface keyword, all methods in this interface are now public and pure virtual, and as a result do not require any implementation. Classes that derive from this interface will have to supply the implementation for all the methods in the interface. Interfaces support methods and properties, but do not support data members. The following code block demonstrates the use of interfaces in Managed C++.

__gc __interface IInterface
{
void ExceptionMethod ();
};

__gc class GCClass : public IInterface
{
public:
void ExceptionMethod ()
{
throw new Exception ("some exception");
}

};

While using interfaces in Managed C++, you have to adhere to the following rules:

  • Managed classes can inherit managed interfaces. Managed classes cannot inherit unmanaged interfaces.
  • Unmanaged classes can inherit unmanaged interfaces. Unmanaged classes cannot inherit managed interfaces.
  • Interfaces have to be inherited using public inheritance.
  • Interfaces cannot be derived from other classes except System.Object. Interfaces can inherit other interfaces.
  • Managed interfaces can only inherit managed interfaces; unmanaged interfaces can only inherit unmanaged interfaces.

Attached Sample Code

The attached sample code is a very simple application that demonstrates a class library written in Managed C++ and used by a C# client. The Managed C++ class library exposes a single method and a read\write property, and uses .NET exceptions and the System.String class to pass data. The MECCalculator is the Managed C++ project with a single class, and the MECClient project is the C# client that references the MECCalculator assembly. The following screenshot displays the MECClient. The Get Title and Set Title buttons demonstrate the use of properties in Managed C++ and the Divide button demonstrates cross language interoperability.

Conclusion

C++ is an extremely powerful language, but this very power has caused a lot of pain and grief to many programmers. Managed C++ brings all the advantages and benefits of the .NET framework to native C++. However, Managed C++ also brings the slight performance degradation of the .NET framework. While Managed C++ offers a lot to the common C++ developer, developers who want pedal to the metal performance from their application and cannot tolerate any inefficiencies, should keep this in mind. In this article, we have merely scratched the surface of Managed C++; please refer to the links below for further details and advanced discussions on this topic.

Happy .NETing.


C# and Java Remoting Part 1 - a C# Remoting Primer

C# and Java Remoting Part 1 - a C# Remoting Primer
http://www.csharptoday.com/content.asp?id=1600
Article Rating: 2.5
Published on June 29, 2001
Abstract
In this series of articles, we will look at interoperating between C# and Java on different platforms, and the problems involved with doing so. The bulk of this first article, however, will be a primer on C# Remoting. Ashish Banerjee goes under the covers of the .NET Framework SDK's 'MyHello' remoting example, before showing a method of tracing and debugging SOAP packets - an essential tool to have as remoting around the globe requires us to be able to trace errors should something go missing or wrong.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

In this series of articles, we will look at interoperating between C# and Java on different platforms, and the problems involved with doing so. The bulk of this first article, however, will be a primer on C# Remoting.

An interesting problem for today's developers is in communicating between different servers running different operating systems and different languages. The are a number of options, for example, for interoperation between Java and C#:

  • ActiveX Java bridge
  • Cross platform compilers
  • Remoting with SOAP

The next three sub-sections will explore each briefly.

ActiveX Java Bridge

Both Microsoft and Sun provide their versions of COM and Java interop bridge. Microsoft's Java COM bridge can interpolate Java and Microsoft Languages, making Java appear as COM object. Prior to .NET we could call Java using the Java moniker via the GetObject method of Visual Basic. For example:

DIM OBJ As Object
SET OBJ = GetObject("java:java.util.Date")
print OBJ.toString()

The above code could be executed in VBA and VB and will create the Java date object through the Java moniker and the OBJ will be assigned an object reference to an instance of Java Date object, managed by Microsoft JVM.

Using the Sun's ActiveX Java Bean Bridge allows Java components to be treated as COM objects. The .NET platform however, treats COM components more or less as legacy code. Therefore COM components are executed as unmanaged code and memory is allocated from the standard windows heap space, and not allocated from a heap managed by the Common Language Runtime (CLR). However, it does provide mechanisms for both exposing .NET components to COM objects and vice versa. When a COM client creates an instance of a .NET object, the object is exposed through a COM callable wrapper (CCW) that acts as a proxy for the real managed object residing in the CLR (Common Language Runtime).

We will not discuss this further in any more detail, as it is beyond the scope of the article, but you can read http://www.asptoday.com/content/articles/20000804.asp?WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0 for more information.

Cross Platform Compilers

The cross platform compilers are an interesting idea, which I have explored in a reasonable detail in my article entitled "Will Java and .NET Framework Co-exist?".

Briefly, when C# and Java cross compilers are available you will be able to interoperate the C# and Java using each other's API. For example you would be able to compile C# program into JVM (Java Virtual Machine) executable or write in Java and compile it to target CLR (Common Language Runtime). However, such cross compilers are still on the drawing boards. If you come across any such cross compilers please let me know.

Remoting with SOAP

Presently, this is the most practical option available to real life programmers and system architects for interoperating Java and C#.

You can download the Java SOAP implementation from xml.apache.org. The source is well written, but is bit heavy. I would like to see a more lean and mean Java SOAP gateway, with a good documentation to go with it.

SOAP is an XML based messaging protocol and uses HTTP and SMTP (Simple Mail Transfer Protocol) as its transport. Though the SOAP specification defines HTTP protocol as its transport, both the Apache Java SOAP gateway and Microsoft .NET SOAP framework implement an SMTP/POP3 transport pair as well.

The SOAP specification is available at http://www.w3.org/TR/SOAP?WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0. I strongly recommend you download and have a look at it.

In the next few sections of this article we will see how to set up remoting using SOAP, with the help of Microsoft's MyHello.NET SDK remoting sample, and also see the use of a trace utility which helps us to trace problem messages.

Installing .NET SDK

If you have not already done so, please install the .NET SDK on your system. Though some portions of the SDK requires Windows 2000 server, for all practical purposes of compiling C# and remoting, I have had no problem running it on of Windows 98.

Running the MyHello Sample

Once you have installed the SDK, you will find the samples at the following installation path:

C:\Program Files\Microsoft.NET\FrameworkSDK\Samples\Remoting\

The remoting sample we will focus on is MyHello, which runs in conjunction with the HelloService sample. The remoting samples can be built by running the command:

C:\Program Files\Microsoft.Net\FrameworkSDK\Samples\Remoting > nmake all

You will also find instructions to build and run the sample in readme.htm file in the C:\Program Files\Microsoft.Net\FrameworkSDK\Samples\Remoting\MyHello directory.

To run the sample open two Command Windows, do not forget to run corvars.bat which is available in C:\Program Files\Microsoft.Net\FrameworkSDK\Bin.

  • In one Command Window change the directory to:
    C:\Program Files\Microsoft.Net\FrameworkSDK\Samples\Remoting\MyHello
  • Open another Command Window, and change directory to:
    C:\Program Files\Microsoft.Net\FrameworkSDK\Samples\Remoting\HelloService
  • From the above directory execute the batch file: HelloService.bat
  • After a brief interval you will see a message:
    Press a key and ENTER: G=GC.Collect, 0=ExitThis
  • The above message indicates that the HelloService is ready to serve.
  • From the first Command Window execute the batch file: sample2.bat
  • When this batch file executes, it will prompt you to enter your name, I typed Ash, and then after some time it displays the result:
    Hello.HelloMethod returned: "Hi there Ash"

If you get a similar result, you have successfully executed the remoting sample. Let's analyze what's happening behind the scene.

Code Walking

The batch file, HelloService.bat executes MyHost.exe with the file HelloService.cfg as its parameter.

MyHost is a simple yet generalized program, which can act as a standalone SOAP remoting server. MyHost.cs is the C# source file from which MyHost.exe is derived. The only line of interest to us in MyHost.cs is the one below:

RemotingServices.ConfigureRemoting(args[i+1]);

RemotingServices provides methods for publishing remote objects. This class is functionally equivalent to Java's java.rmi.server.RemoteServer class. However, the designs of the two classes differ significantly.

  • The RemotingServices has a utility design pattern, meaning that all the methods of this class are static and you cannot create an instance of this class.
  • The method ConfigureRemoting takes a file name as parameter, to configure the remoting object services.
  • The argument passed to the above method is the file name HelloService.cfg.

Let us now look inside the content of the configuration file, which is passed as a parameter to the static method ConfigureRemoting of the class RemotingServices.

Only the first three lines of this configuration file are relevant, the rest being comments. These first three lines are reproduced below:

Name#HelloService
WellKnownObject#HelloService.Hello#HelloService#HelloService/Hello.soap#SingleCall
Channel#System.Runtime.Remoting#System.Runtime.Remoting.Channels.HTTP.HTTPChannel#ports=8085

  • The first line describes the name of the application, HelloService.
  • The second line registers the Object name with the remoting service and the last word SingleCall is the keyword that instructs the remoting service to create a new instance every time this object is accessed remotely by a client. For more scalability, use Singleton, whereby every message is dispatched to the same object instance
  • The third line of the configuration file defines the communication channel and it's parameters. That is, the example uses HTTP protocol at port 8085.

Note that the configuration file has a URI (Universal Resource Information) like syntax, similar to that of JDBC (Java Data Base Connectivity) configuration parameters. I would have expected the .NET architects to use XML based syntax.

Now, have a look at the source code of HelloService, contained in the file HelloService.cs. This file defines, within a namespace HelloService, a simple class Hello that extends the class System.MarshalByRefObject. This is the base class for Remoting objects that need marshaling by reference. Java programmers can relate this class to its Java counterpart java.rmi.server.UnicastRemoteObject.

The HelloService.Hello class has only one method, HelloMethod, which concatenates "Hi there" with the input parameter passed to it.

Now, this brings us to the final leg of our code walk through. Change your directory back to C:\Program Files\Microsoft.Net\FrameworkSDK\Samples\Remoting\MyHello and type sample2.bat (which contains the command MyHello -cfg MyHello.DirectHTTP.cfg ).

So, MyHello is being called with a configuration file MyHello.DirectHTTP.cfg. If you casually look into the source code of MyHello class you might not find anything that suggest that you are looking at a remoting sample.

Well this is a double-edged sword! On one side it lets you write remoting programs without bothering about the details of SOAP. The flip side is that while deploying or implementing, someone might accidentally break it. To demonstrate this point, if you comment out lines 2,3,4 of the configuration file MyHello.DirectHTTP.cfg, then you will be executing MyHello sample without remoting! That is, even if MyHost is not running on the other Command Window, your program will still run.

If you have written complex programs that need deployment and configuration and use remoting as well, then I am sure you will immediately realize that this is a potential cause of disaster waiting to happen. It's guaranteed to give you sleepless nights, trying to figure out why the installation on the other side of the globe is misbehaving!

Most of the times it will not be so bad, so let's get back to MyHello.cs code. Look at line 69, which is same as line number 80 of MyHello.cs:

RemotingServices.ConfigureRemoting(args[i+1]);

The same method ConfigureRemoting is to be used by the client code as well. How is this? Well the CLR (Common Language Runtime) provides isolation between Application Domains (AppDomain ). The runtime creates a proxy stub, to handle inter AppDomain method calls. Since this binding is delayed till execution; you can affect its behavior using configuration files similar to the one used by this sample, MyHello.DirectHTTP.cfg, and call the static method RemotingServices.ConfigureRemoting from anywhere in the program, (but first create the instance of an object whose method is being called). In this case it is the HelloService.Hello object.

Incidentally, line 3 of MyHello.DirectHTTP.cfg defines that HelloService is a remoted application and accessed though HTTP protocol, residing at localhost at port 8085. This line is below:

RemoteApplication#HelloService#HTTP://localhost:8085/HelloService

The next section will look into SOAP (Simple Object Access Protocol) packets being exchanged in this sample between the client and the server. Thereafter, I will share with you the tools I use to look under the hood.

How do MyHello and HelloService Talk?

In the sample discussed above, the client and the server exchange request and response packets in SOAP (Simple Object Access Protocol), through HTTP transport.

As discussed, SOAP is an XML based protocol. A SOAP message is contained in an envelope and it has header and a body. The sample exchange the messages reproduced are below in req.txt and res.txt.

Later I will discuss how to trap the messages. The trapping technique will also help you while debugging applications, which spans companies and sometimes continents. In those situations, you cannot use conventional debuggers, neither can you expect your counterparts, working for another company and in different time zone, to co-operate with your team, while they may be themselves struggling to meet their deadlines! I have also found this trapping technique helpful in convincing the team working at the other end of a B2B application that their application is misbehaving and sending out wrong packets.

MS provides a functional SOAP trace utility that is said to be very helpful in debugging. However, I have not used it as of yet, and will demonstrate one I wrote myself.

Analyzing the SOAP Messages

MyHello is the client program, which sends a SOAP request, see req.txt below:

01 POST /HelloService/Hello.soap HTTP/1.1
02 SOAPAction: "http://Schemas.microsoft.com/urt/NSAssem/HelloService.Hello/HelloService#HelloMethod"
03 Content-Type: text/xml
04 Content-Length: 1025
05 Connection: Close
06 Host: localhost
07
08 <SOAP-ENV:Envelope xmlns:xsi="http://www.w3.org/1999/XMLSchema-instance"
xmlns:xsd="http://www.w3.org/1999/XMLSchema"
xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/"
SOAP-ENV:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"
xmlns:a2="http://schemas.microsoft.com/urt/NS/System.Runtime.Remoting"
xmlns:a1="http://schemas.microsoft.com/urt/NS/System"
xmlns:i7="http://Schemas.microsoft.com/urt/NSAssem/HelloService.Hello/HelloService"
09 <SOAP-ENV:Header
10 <__CallContext href="#ref-3" root="1"/
11 <a1:LogicalCallContext id="ref-3"
12 <__RemotingData href="#ref-4"/
13 <__SecurityData href="#ref-5"/
14 </a1:LogicalCallContext
15 <a2:CallContextRemotingData id="ref-4"
16 <_logicalCallID id="ref-6"ca2ac5a1-4c90-11d5-917d-c780bb182871_2</_logicalCallID
17 </a2:CallContextRemotingData
18 <a1:CallContextSecurityData id="ref-5"
19 </a1:CallContextSecurityData
20 </SOAP-ENV:Header
21 <SOAP-ENV:Body
22 <i7:HelloMethod id="ref-1"
23 <name id="ref-8"ash</name
24 </i7:HelloMethod
25 </SOAP-ENV:Body
26 </SOAP-ENV:Envelope

Lines 1 to 6 are HTTP headers. If you happen to be curious about the HTTP protocol or reading through the SOAP specification has not helped your insomnia, then you can download the protocol from http://www.ietf.org/rfc/rfc2616.txt?WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0.

The lines 8 to 26 contain the SOAP headers and body, wrapped in the SOAP envelope. The lines 22 to 24, pass the information regarding the method call HelloMethod, with line number 23 containing the parameter value being passed to the server.

The SOAP server is contained within the CLR (Common Language Runtime), which is equivalent to JVM (Java Virtual Machine). This server, activated by MyHost, running on second Command Window, receives the request packet, parses the XML, and calls the requested method of HelloService.Hello class, after instantiating a new instance. The resulting return value is sent back to the client through the response packet listed in res.txt below:

01 HTTP/1.1 200 OK
02 Content-Length: 1058
03 Content-Type: text/xml
04 Connection: close
05
06 <SOAP-ENV:Envelope xmlns:xsi="http://www.w3.org/1999/XMLSchema-instance"
xmlns:xsd="http://www.w3.org/1999/XMLSchema"
xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/"
SOAP-ENV:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"
xmlns:a2="http://schemas.microsoft.com/urt/NS/System.Runtime.Remoting"
xmlns:a1="http://schemas.microsoft.com/urt/NS/System"
xmlns:i7="http://Schemas.microsoft.com/urt/NSAssem/HelloService.Hello/HelloService"
07 <SOAP-ENV:Header
08 <__CallContext href="#ref-3" root="1"/
09 <a1:LogicalCallContext id="ref-3"
10 <__RemotingData href="#ref-4"/
11 <__SecurityData href="#ref-5"/
12 </a1:LogicalCallContext
13 <a2:CallContextRemotingData id="ref-4"
14 <_logicalCallID id="ref-6"ca2ac5a1-4c90-11d5-917d-c780bb182871_2</_logicalCallID
15 </a2:CallContextRemotingData
16 <a1:CallContextSecurityData id="ref-5"
17 </a1:CallContextSecurityData
18 </SOAP-ENV:Header
19 <SOAP-ENV:Body
20 <i7:HelloMethodResponse id="ref-1"
21 <__return id="ref-8"Hi there ash</__return
22 </i7:HelloMethodResponse
23 </SOAP-ENV:Body
24 </SOAP-ENV:Envelope

Again the lines 1 to 4 of res.txt are the HTTP headers. Lines 6 to 24 contain the SOAP envelope headers and body. Line 21 has the return value, which is displayed by the MyHello client. The MyHello client is isolated from all these details by the CLR (Common Language Runtime) proxy stub for the object HelloService.Hello. Behind the scene, the runtime at the client end, receives the request, parses the SOAP message, and de-serializes (or unmarshals) back the string.

Understanding the Trapping Technique

While consulting for a sub contractor, an application we developed was having problems talking to the application running at the trading partner end. I developed these simple Http message trappers to conclusively resolve the debate as to where the problem was laying. As a side benefit, I also found this tool helpful for simulating various error conditions that our programs may encounter from other end.

The idea is to simulate the Http client or the server, depending on which side of the application you plan to debug. If you wish to trap the output of you client side application, use the HttpTrapper.cs, which is the server simulator. To debug the server side of your application you would use the client simulator, HttpTrapClient.cs.

Let me summarize the process. Suppose we have a server application S that calls the client side C. And let us say that under a certain condition C and S refuse to talk. To figure out what is happening, my technique requires a three pass process.

The HttpTrapper is placed in the place of S, and as the client C talks to the server, the trapper will save the client request in a file.

The second pass involves using this captured request file to send the response to the actual server S, using HttpTrapClient. The server, S, will cough up a response, which will be captured in another file. Now we have both the request and responses available in a text file, which we can now scrutinize for errors.

Walking the Code

Now compile the three accompanying C# utilities by executing the from nmake all Command Window. This will produce AddrTest.exe, HttpTrapper.exe and HttpTrapClient.exe, from their respective C# sources with the same name.

AddrTest.cs is a test program, which helps to ensure that TCP/IP and DNS are configured and working properly in my machine:

using System.Net;

public class
            AddrTest
{
public static void
            Main()
{
System.Console.WriteLine(DNS.GetHostName());
}
}

The actual programs we will use for trapping are HttpTrapper.cs and HttpTrapClient.cs.

Let us walk you through the passes.

Pass One: Catching the Request

First, stop the MyHost sample if it is running on any of your Command Windows.

Now run the trapper program in another Command Window using the following command:

HttpTrapper 8085 ERR.TXT

The file err.txt is simple two-line text file that simulates an error condition.

From a Command Window run sample2.bat, as we did in the first section of this article.

The sample2.bat will execute the MyHello sample, give your name when it prompts for it. MyHello will trigger the CLR (Common Language Runtime), which will contact the Trapper, which acts as the server simulator. The trapper HttpTrapper captures the request message in a file called delme0.txt. I renamed this file to create the req.txt. The trapper however, sends back the error message contained in the file err.txt.

At this point, a very interesting event occurs. The CLR (Common Language Runtime) receives back the error response and after displaying an error message box, spills out the stack trace. Analyzing this trace helps us get the brass tacks of remoting, as implemented by the Microsoft engineers. The stack dump occurs during the dereferencing (also called unmarshalling) stage of remote method calls implementation. The Microsoft engineers, I am sure, will plug this hole before a production release, but as of the Beta 1 release (I have not tested this hole on Beta 2), which I am using, it bares it all!

Following the Stack Trace

Let us walk the stack trace:

The ball starts rolling from line 36 of MyHello.cs. This is the place from where the HelloMethod of the HelloService.Hello object is called. From thereon, the CLR proxy takes over, since the configuration file specifies the remote object (see line 3 of file MyHello.DirectHTTP.cfg ). Thus the CLR System.Runtime.Remoting.RealProxy.PrivateInvoke method is called passing frame and type as parameters. This in turn calls System.Runtime.Remoting.RemotingProxy.Invoke passing a request message object which implements IMessage interface.

Next System.Runtime.Remoting.RemotingProxy.InvokeInternal is called, followed by the call to System.Runtime.Remoting.RemotingProxy.CallProcessMessage, which goofs up. The method CallProcessMessage is responsible for actually contacting the server and uses the first parameter, an implement of IMessageSink to talk to the server.

The IMessage interface is an abstract representation of the messages passed between the SOAP client and the Server. The object implementing the IMessage interface can change the way messages are serialized.

The objects implementing the IMessageSink Interface are responsible for transport layer communication. 

Pass Two: Trapping the Response

In the next pass, we will use the HttpTrapClient.cs and pass the trapped request packet to the SOAP server using the sample MyHost, we used in Step 1 section of this article.

Now change the directory in a Command Window to HelloService, and execute HelloService.bat as we did before.

Now from the Command Window we can run the HttpTrapClient program by executing the command below, to send the captured request to the SOAP service and receive back the response.

HttpTrapClient REQ.TXT RES.TXT 127.0.0.1 8085 20

The first two parameters are the request and response files, followed by the server IP address. If you are running the server on a different machine AddrTest.cs will display the IP address for you. The last parameter is the timeout in seconds. The last but one parameter is the server port to use.

Now, we have both the request and response with us, which I used in this article to get a insight into the SOAP communication protocol implementation in Microsoft's CLR (Common Language Runtime).

Pass Three: Putting it Back Together

This pass I have included for completeness. Stop the HelloService, and change directory back to MyHello. Run the batch file sample2.bat again.

From the Command Window run the HttpTrapper again, but this time using res.txt file. Use the command given below:

HttpTrapper 8085 RES.TXT

Change the focus back to the sample2.bat Command Window where MyHello is executing. When prompted type your name and after some delay you will get back the original response!

Conclusion

I hope that I was able to share with you some insights I have gained on Microsoft's SOAP service implementation of the .NET framework.

As for all server side Java programmers, .NET beholds the promise to interoperate Microsoft platforms resident clients with scalable Java server applications.

Now that we have had a good look at C# Remoting, in the next articles in this series we will show how to communicate between C# and the JVM.


Java stream Vs C# Streams

Java stream Vs C# Streams
http://www.csharptoday.com/content.asp?id=1539
Article Rating: 3
Published on June 19, 2001
Abstract
Most applications deal with reading data in from a source, doing something with it, and outputting it in some new form. The source can be many things, user input, local files, another application, a hardware device, etc. Data Input and output streams are the foundation of working with data in the .NET APIs and understanding them is an important part of programming for this new environment. This article, by Mark Tutt, is geared towards Java programmers considering migrating to .NET, however C# developers unfamiliar with the .NET stream classes should also find the information useful. Mark attempts to review the basic Java stream classes and illustrate the analogous .NET classes, demonstrating how one would write the equivalent code using C#.
Article

Editor's Note: This article's code has been updated to work with the final release of the .Net framework.

Most applications deal with reading data in from a source, doing something with it, and outputting it in a new form. The source can be many things, user input, local files, another application, a hardware device, etc. Data input and output streams are the foundation of working with data in the .NET APIs and understanding them is an important part of programming for this new environment.

This article is geared towards Java programmers considering migrating to C# and .NET, however C# developers unfamiliar with the .NET stream classes should also find the information useful. I will attempt to review the basic Java stream classes and illustrate the analogous .NET classes, demonstrating how one would write the equivalent code using C#.

Stream Basics

When one first examines the comparison between the Java stream classes and those found in the .NET Beta 1 SDK, there is a momentary panic, since there appear to be far fewer classes in .NET.

Figure 1 Java Stream Classes

Figure 2 .NET Stream Classes

However, appearances are deceiving. The .NET stream classes are based on a single abstract class System.IO.Stream. .NET classes inheriting from this class can optionally read or write data. The JDK stream classes are derived from one of four base classes, java.io.InputStream, java.io.OutputStream, java.io.Reader or java.io.Writer. The reason for 4 base classes is because before JDK 1.1, the input and output stream classes only supported 8-bit byte streams, and had no facilities for working with 16-bit Unicode character streams. The .NET development team had the advantage of learning from this, and thankfully avoided this somewhat confusing situation.

Since most of the JDK classes follow the separate Reader/Writer model, you will see them implemented as separate classes for reading and writing data. An example of this, starting at the top of the list, would be the java.ioBufferedInputStream and java.io.BufferedOutputStream, or their character equivalents, java.io.BufferedReader and java.io.BufferedWriter. The corresponding .NET class is simply System.IO.BufferedStream, which offers both reading and writing methods, and makes it much cleaner for the developer.

The constructors for the classes for both API's are virtually identical. The first Java constructor syntax creates a buffered stream with a default buffer size of 512 bytes, whereas the .NET default buffer size is 4096 bytes.

[Java]

BufferedInputStream( InputStream in );
BufferedInputStream( InputStream in, int size );

BufferedOutputStream( OutputStream in );
BufferedOutputStream( OutputStream in, int size );

BufferedReader( Reader in );
BufferedReader( Reader in,int size );

BufferedWriter( Writer in );
BufferedWriter( Writer in,int size );

[C#]

public BufferedStream( Stream stream );
public BufferedStream( Stream stream, int bufferSize );

Buffered stream classes read and write blocks of data from their source stream into a buffer, reducing the number of calls to low level functions to access the underlying data source. An example of this would be reading and writing records from a disk file. Unless the records were fairly large, reading the records individually might require an excessive number of disk access operations, which could degrade performance, while the same argument could be made for writing records to the file. Utilizing the buffered stream classes allows the developer to determine an optimal 'chunk' of data to be buffered, or allow the system to use a default setting.

File Streams

Since I'm using reading and writing files as our first example, we should discuss the classes that relate to this before putting them together for an example.

The file stream classes in both APIs are somewhat less similar. For this class, the .NET API offers a plethora of options, which allow the developer to assert a fine grained control, and tune the application for the intended use.

[Java - Binary Files]

FileInputStream( String name );
FileInputStream( File file );
FileInputStream( FileDescriptor fdObj );

FileOutputStream( String name );
FileOutputStream( String name, boolean append );
FileOutputStream( File file );
FileOutputStream( FileDescriptor fdObj );

[Java - Character Files]

FileReader( String name );
FileReader( File file );
FileReader( FileDescriptor fdObj );

FileWriter( String name );
FileWriter( String name, boolean append );
FileWriter( File file );
FileWriter( FileDescriptor fdObj );

[Java - Files which require random access, i.e. a mixed Read/Write usage]

RandomAccessFile( String name, String mode );
RandomAccessFile( File file, String mode );

[C#]

FileStream( int handle, FileAccess access );
FileStream( string name, FileMode mode );
FileStream( int handle, FileAccess access, bool ownsHandle );
FileStream( string name, FileMode mode, FileAccess access );
FileStream( string name, FileMode mode, FileAccess access, FileShare share);
FileStream( int handle, FileAccess access, bool ownsHandle, bool useAsync );
FileStream( string name, FileMode mode, FileAccess access, FileShare share, int bufferSize );
FileStream( int handle, FileAccess access, bool ownsHandle, bool useAsync, int bufferSize );
FileStream( string name, FileMode mode, FileAccess access, FileShare share, int bufferSize, bool useAsync );

Starting with the arguments from the top, FileAccess is an enumeration of the file access options available: Read, ReadWrite and Write. FileMode is an enumeration of the file opening options: Create, CreateNew, Append, etc. File share is yet another enumeration, this one of the sharing options for sharing files between processes. The useAsync parameter signals the system to use overlapped (asynchronous) IO if available, and the buffersize parameter sets the streams internal buffers.

For full details of these options, check the .NET SDK documentation. For our purposes, we are going to use the constructor form: FileStream(string name, FileMode mode, FileAccess access);

Now, it is possible to read and write data using just the classes we have covered thus far, however, it's not very convenient. These low level classes provide basic methods for reading and writing single bytes or arrays of bytes (characters for the Java classes based on java.io.Reader and java.io.Writer), but that's not how we normally work with application data these days. We tend to think of our basic data types, i.e. ints, floats and strings, rather than the individual bytes they are made up of.

Fortunately, the developers of both APIs understood this, and provide classes that are in line with the needs of client programmers. Still, the knowledge of the underlying classes is important if you need to implement your own stream classes.

Binary Streams

For the example we are going to use, we are going to read and write binary data, so these classes allow you to read primitive types from their underlying streams:

[Java]

DataInputStream( InputStream in );
DataOutputStream( OutputStream out);

[C#]

BinaryReader( Stream input );
BinaryWriter( Stream output );
BinaryReader( Stream input, Encoding encoding );
BinaryWriter( Stream output, Encoding encoding );

The Encoding parameter allows the developer to set the character encoding for the stream. There are several options here, found in the System.Text namespace. Java streams do not include these options as they are written in a platform independent manner. You may need to investigate the encoding options if you are writing applications that will need to be localized, however going into this subject is outside the scope of this article. Richard Grimes describes encoding briefly in his article "Streams and .NET" at http://www.codeguru.com/cs_data/streaming.html?WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0, however I have not found a great deal of other information available online.

Putting what we've covered this far together to assemble streams for reading and writing files takes the following forms:

[Java - read]

FileInputStream in =
            new FileInputStream("test.data");
BufferedInputStream bin = new
            BufferedInputStream(in, 4096);
DataInputStream din = new
            DataInputStream(bin);

[Java - Write]

FileOutputStream
            out = new FileOutputStream("test.data");
BufferedOutputStream
            bout = new BufferedOutputStream(out, 4096);
DataOutputStream dout
            = new DataOutputStream(bout);

[C#]

FileStream fStream
            = new FileStream("test.data",
            System.IO.FileMode.OpenOrCreate,FileAccess.ReadWrite);
BufferedStream
            bStream = new BufferedStream(fStream);

BinaryReader rStream =
            new BinaryReader(bStream);

BinaryWriter wStream = new
            BinaryWriter(bStream);

If you wanted to arbitrarily read and write the file, in Java you would have to take a somewhat different approach. To fill this need, you would use the java.io.RandomAccessFile class:

RandomAccessFile rFile = new RandomAccessFile("test.data", "rw");

However, while it can work with the stream classes, this class provides methods for reading and writing primitive data types, as well as the positioning information, i.e. seek(long pos), skipBytes(int n), and getFilePointer() (returns the current offset in the file.) Since this class provides all of this functionality, a stream class is not commonly used with it. If you do, you end up giving up some of the functionality this way, for instance the DataInputStream and DataOutputStream classes do not offer the full range of movement options, only the skip(long bytes) method, which requires the developer to do a little math. By comparison, the BinaryReader and BinaryWriter classes offer the seek(int offset, SeekOrigin) method, which I find to be a little more intuitive. (SeekOrigin is another enumeration in the System.IO namespace, containing three members Begin, Current and End.) Please note that for the BinaryReader class, you have to access the underlying stream with BinaryReader.BaseStream() to access the base streams seek method. (I wonder if anyone can tell me why a seek method didn't make it in the BinaryReader class itself?)

Ok, now we've covered what we needed to know for our first example. We're going to create several instances of a Customer class, populate each one with some data, and write them out to a file. Then we'll read the file, and display to the screen what we have read.

[Java]

//First the customer
            class.

public class Customer
{
public int
            idNumber;
public String Name;

public Customer(int number,
            String FullName)
{
idNumber = number;
Name =
            FullName;
}
}

//And the relevant functions from our
            demonstration program.

static void SaveCustomers( Vector
            CustList ) {

try {
// Create our output
            stream
FileOutputStream out = new FileOutputStream("test.data");
BufferedOutputStream bout = new BufferedOutputStream(out,
            4096);
DataOutputStream dout = new
            DataOutputStream(bout);

//Cycle through the Vector and write
            out the customer objects
for (int i = 0; i < CustList.size();
            i++) {
Customer tCustomer =
            (Customer)CustList.elementAt(i);
// Write the idNumber as an
            integer
dout.writeInt(tCustomer.idNumber);
// Write the name
            as a UTF encoded
            string
dout.writeUTF(tCustomer.Name);
}

dout.close();
}
            catch (IOException e) {}
}

static void LoadCustomers(
            Vector CustList) {

try {
// Create our input
            stream
FileInputStream in = new FileInputStream("test.data");
BufferedInputStream bin = new BufferedInputStream(in,
            4096);
DataInputStream din = new DataInputStream(bin);
//
            Simple use of a boolean to tell when we should quit
            reading
boolean more = true;

while (more == true) {
try
            {
// Read the ID as an integer
int id = din.readInt();
//
            Read the name as a UTF encoded string
String name =
            din.readUTF();
// Create a new customer from the read data
//
            And add it to the passed Vector
CustList.addElement(new Customer(
            id, name ));
} catch( EOFException e) {
more =
            false;
}}
}

din.close();
} catch (IOException e)
            {}
}

[C#]

namespace Binary
{
using
            System;

public class Customer
{
public int
            idNumber;
public string Name;

public Customer(int number,
            string FullName)
{
idNumber = number;
Name =
            FullName;
}
}
}

And the relevant functions from our demonstration program.

   public static void
            SaveCustomers( ArrayList CustList ) {
// Create our output
            stream
FileStream fStream = new
            FileStream("test.data",
System.IO.FileMode.OpenOrCreate,FileAccess.Write);

BufferedStream
            bStream = new BufferedStream(fStream);
BinaryWriter wStream = new
            BinaryWriter(bStream);

// Cycle through the ArrayList and
            write out the customer objects
for (int i = 0; i <
            CustList.Count; i++) {
Customer tCustomer =
            (Customer)CustList[i];
// BinaryWriter's Write method is
            overloaded, so you
// don't have to specify the data
            type
wStream.Write(tCustomer.idNumber);
wStream.Write(tCustomer.Name);
}

wStream.Close();
}

public
            static void LoadCustomers( ArrayList CustList) {

// Create
            our input stream
FileStream fStream = new
            FileStream("test.data",
System.IO.FileMode.Open,FileAccess.Read);

BufferedStream
            bStream = new BufferedStream(fStream);
BinaryReader rStream = new
            BinaryReader(bStream);
bool more = true;

while (more ==
            true) {
try {
// Read the ID as a 32 bit integer
int id =
            rStream.ReadInt32();
// Read the name as a string
string name
            = rStream.ReadString();
// Create a new Customer object and add
            it to the
// passed ArrayList
CustList.Add( new Customer( id,
            name ));
}
catch( EndOfStreamException e) {}
more =
            false;
}
}

rStream.Close();
}

Object Serialization

Before I get the email flood pointing out that there is an easier way to do this using the serialization features of the APIs, I'll demonstrate that next.

In order to persist an object in Java using the API functionality, the object class must implement the java.io.Serializable interface. To do the same thing with an object in C#, the object class must implement the System.Runtime.Serialization.ISerializable interface. Additionally, the .NET serialization mechanism uses a formatter object, for our example we're going to use the BinaryFormatter found in the System.Runtime.Serialization.Formatters.Binary namespace.

Both APIs provide means of controlling serialization of class variables. In Java, a class implementing the Serializable interface declares variables that should not be serialized as transient variables, where the .NET API uses the metadata tag [NonSerialized] to determine that a variable should not be saved.

We need to use different Java stream classes to support object serialization.

ObjectInputStream(InputStream in)
ObjectOutputStream(OutputStream out)

You can serialize .NET objects to its various stream types using different formatters. While we're going to use the BinaryFormatter and BinaryReader/Writer classes here, for example if our stream was based on a HTTP socket, we could use the SoapFormatter from the System.Runtime.Serialization.Formatters.Soap namespace. You could do this with this example simply by replacing the BinaryFormatter with a SoapFormatter.

So here's our example code, rewritten to use object serialization:

[Java]

public class Customer
            implements Serializable
{
public int idNumber;
public
            String Name;

public Customer(int number, String
            FullName)
{
idNumber = number;
Name =
            FullName;
}
}

static void SaveCustomers(Vector
            CustList) {

try {
FileOutputStream out = new
            FileOutputStream("test.data");
BufferedOutputStream bout = new
            BufferedOutputStream(out, 4096);
ObjectOutputStream dout = new
            ObjectOutputStream(bout);

for (int i = 0; i <
            CustList.size(); i++) {
Customer tCustomer =
            (Customer)CustList.elementAt(i);
dout.writeObject(tCustomer);
}
dout.close();
}
            catch (IOException e) {}
}

static void LoadCustomers(
            Vector CustList) {

try {
FileInputStream in = new
            FileInputStream("test.data");
BufferedInputStream bin = new
            BufferedInputStream(in, 4096);
ObjectInputStream din = new
            ObjectInputStream(bin);
boolean more = true;

while (more
            == true) {
try {
Customer tCustomer =
            (Customer)din.readObject();
CustList.addElement(tCustomer);
}
            catch( ClassNotFoundException e) {
more =
            false;
}}
}

din.close();
} catch (IOException e)
            {}
}

[C#]

namespace Binary
{
using
            System;

[Serializable]
public class
            Customer
{
public int idNumber;
public string
            Name;

public Customer(int number, string
            FullName)
{
idNumber = number;
Name =
            FullName;
}
}
} public static void SaveCustomers( ArrayList
            CustList ) {
FileStream fStream = new FileStream("test.data",
System.IO.FileMode.OpenOrCreate,FileAccess.Write);
BufferedStream
            bStream = new BufferedStream(fStream);
BinaryFormatter bFormatter
            = new BinaryFormatter();

for (int i = 0; i <
            CustList.Count; i++) {
Customer tCustomer =
            (Customer)CustList[i];
bFormatter.Serialize(bStream,
            tCustomer);
}
bStream.Close();
}

public static void
            LoadCustomers( ArrayList CustList) {
FileStream fStream = new
            FileStream("test.data",
System.IO.FileMode.Open,FileAccess.Read);

BufferedStream
            bStream = new BufferedStream(fStream);
BinaryFormatter bFormatter
            = new BinaryFormatter();
bool more = true;
while (more ==
            true) {
try
            {
CustList.Add((Customer)bFormatter.Deserialize(bStream));
}
catch(
            SerializationException e) {
more =
            false;
}
}
bStream.Close();
}

Text Streams

Reading and writing text streams is fairly similar. We've already covered some of the required classes in Java, however here are the new ones we're going to use:

PrintWriter(OutputStream out);
PrintWriter(OutputStream out, boolean autoFlush);
PrintWriter(Writer out);
PrintWriter(Writer out, boolean autoFlush);

The autoFlush parameter determines if the stream buffer should be flushed when one of the println methods is called. You could of course use the FileWriter class directly, or any of the other character stream classes, but PrintWriter offers a friendlier interface for our example. After all, software development is all about making choices, isn't it?

LineNumberReader(Reader in, int sz);
LineNumberReader(Reader in);

The sz parameter determines the size of the buffer used. The LineNumberReader class extends BufferedReader, so it is not necessary to use a buffered stream class with it.

[C#]

StreamWriter( string path );
StreamWriter( Stream stream );
StreamWriter( string path,
            bool append );
StreamWriter( Stream stream, Encoding encoding );
StreamWriter( Stream stream, Encoding encoding, int bufferSize
            );
StreamWriter( string path, bool append, Encoding encoding );
StreamWriter( string path, bool append, Encoding encoding, int
            bufferSize );

StreamReader( string path );
StreamReader(
            Stream stream );
StreamReader( string path, Encoding encoding );
StreamReader( Stream stream, Encoding encoding );

The .NET API gives you a shortcut option here. You can construct a StreamReader or StreamWriter for character files directly from the path and filename, without having to first construct a stream object, as you do in Java when using the PrintWriter class we chose for this example. The other parameters should be familiar from the previous examples.

The following trivial examples demonstrate reading and writing lines of test:

[Java]

FileOutputStream out = new
            FileOutputStream("test.txt");
BufferedOutputStream bout = new
            BufferedOutputStream(out, 4096);
PrintWriter pWriter = new
            PrintWriter(bout);
pWriter.println("some
            string");

FileInputStream out = new
            FileInputStream("test.txt");
LineNumberReader lReader = new
            LineNumberReader(bout, 4096);
String s =
            lReader.readLine();

[C#]

StreamWriter sWriter = new
            StreamWriter("text.txt");
SWriter.WriteLine("some
            string");

StreamReader sReader = new
            StreamReader("text.txt");
String s =
            sReader.ReadLine();

So far we have covered reading and writing both binary and text data using the stream classes. I'll cover one more area before wrapping up.

Memory Streams

Many of today's applications are frequently at least network aware, and as such need to read and write data from TCP/IP sockets. Fortunately, since you can use the stream classes that we've already covered with your socket code, it's not a lot different, at least for the basics. The following functions serialize and de-serialize objects into a byte array using memory streams. (Socket classes in both API's deal with writing byte arrays.)

[Java]

static byte[] serialize(Object
            s)
{
ByteArrayOutputStream bo = new
            ByteArrayOutputStream();

try {
ObjectOutputStream os = new
            ObjectOutputStream(bo);

os.writeObject(s);

return
            bo.toByteArray();
} catch (Exception e)
            {
System.out.println(e.getMessage());
}
}

Object
            deserialize(byte[] data) {
try {
ByteArrayInputStream bi = new
            ByteArrayInputStream(data);
ObjectInputStream is = new
            ObjectInputStream(bi);

return is.readObject();
} catch
            (Exception e) {
System.out.println(e.getMessage());
}
}

[C#]

public static byte[] serialize(object
            s)
{
try {
MemoryStream ms = new
            MemoryStream();
BinaryFormatter b = new
            BinaryFormatter();
b.Serialize(ms,s);

return
            ms.ToArray();
} catch (Exception e) {
throw
            Trace.error(Trace.SERIALIZATION_FAILURE,
            e.Message);
}
}

public object deserialize(byte[] data)
{
try {
MemoryStream ms = new
            MemoryStream(data);
BinaryFormatter b = new
            BinaryFormatter();

return b.Deserialize(ms);
} catch
            (Exception e) {
throw Trace.error(Trace.SERIALIZATION_FAILURE,
            e.Message);
}
}

As you can see, the Java ByteArrayInputStream and ByteArrayOutputStream are directly compatible with the .NET MemoryStream class in this example.

The .NET API offers another class for working with socket data that is perhaps a little easier to work with, however, to my knowledge there is no corresponding class in the Java API. This is the NetworkStream class. Here is a brief example of its use:

[Writing data]
// Attach to socket 5008
            on the local machine
TCPClient myClient = new
            TCPClient("localhost",5008);
// Create a BinaryWriter for the
            NetworkStream class
// returned from the TCPClient.GetStream()
            method
BinaryWriter bWriter = new
            BinaryWriter(myClient.GetStream());

[Reading data]
//
            Listen on socket 5008
TCPListener myListener = new
            TCPListener(5008);
// TCPListener.Accept() returns a Socket
            object
Socket mySocket = myListener.Accept();
// Create a
            network stream for the connected socket
NetworkStream nStream =
            new NetworkStream(socket);
// Attach a BinaryReader object to the
            stream
BinaryReader bReader = new
            BinaryReader(nStream);

Using the NetworkStream classes allows the developer to skip the implicit conversion of byte arrays used by the socket.

Additional Subjects

This article did not address all of the stream classes in both API's, but should have given you enough information to build on if you run into usage of one of the classes I did not cover. There is one possibility that you may encounter that I will mention though. There are several stream classes in Java that currently do not have a correlation in the .NET API. However, it is fairly straightforward to implement similar classes in C# with a little research.

For instance, the java.util.zip.* classes implement support (including stream classes) for reading and writing files in the common Zip file format. Implementing something like this sounds like a monumental task, unless you know that these classes in Java are little more than a wrapper around a modified version of the zlib library, and in fact only expose a portion of it's functionality. zlib is a freely available data-compression library for use on virtually any computer hardware and operating system that is also legally unencumbered; that is, not covered by any patents. Since you can make DLL calls from C# code, writing a wrapper and letting the DLL do all the real work makes it a bit easier, after all Sun thought so. A Win32 DLL version of zlib is available from:

http://www.winimage.com/zLibDll/.http://www.winimage.com/zLibDll?WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0

Both binary versions and source code are available.

If there is enough interest, perhaps I can do another article on extending the .NET Stream classes to demonstrate this.

If you're not interested in rolling your own classes, XCEED Software is currently offering a beta version of their XCEED Zip Compression Library v5.0, which offers this functionality for .NET developers. You can find more information at http://www.xceedsoft.com/dotnet?WROXEMPTOKEN=666149ZBcgfK16PKYMu0LWKdv0, the currently available version requires .NET Beta 2.

Conclusion

A zip file containing sample applications written in Java and C# accompanies the article. While the Java code was written using Visual Cafe and includes the project files for that environment, it can be compiled and executed using just the JDK. The following screen capture shows the output from both the C# and Java programs created.

Hopefully this article has given you enough information to see that porting code using the Java API's stream classes to C# and .NET is not difficult, and that there are a number of similarities between the two APIs. There are a lot of Java programmers out there who, with an introduction to the .NET APIs could quickly become proficient C# programmers. Likewise, there is a large body of Java code available from various books and online sources that address a wide variety of programming needs, and understanding how to translate that into C# just might come in handy!


An In-Depth Look at the DateTimeFormatInfo Class

An In-Depth Look at the DateTimeFormatInfo Class
http://www.csharptoday.com/content.asp?id=1827
Article Rating: 4
Published on March 1, 2002
Abstract
The DateTimeFormatInfo (DTFI) class is one of the most important members of the .NET System.Globalization namespace. Although acting mainly as the IFormattable interface for the DateTime structure, DTFI is also informational. You can, for example, find out the names of the days of the week and months of the year in native form, an amazing number of alternate date and time patterns that are acceptable in a particular culture, and which week is the first one of a new year. In addition, DTFI provides the main access to the more than half-dozen calendars supported by .NET. In this article, Bill Hall will show you how easy it is to use the facilities of DTFI, and will gain experience in writing code that functions correctly independently of language and world region.
Article

The DateTimeFormatInfo (DTFI) class is one of the most important members of the .NET System.Globalization namespace. Although acting mainly as the IFormattable interface for the DateTime structure, DTFI is also informational. You can, for example, find out the names of the days of the week and months of the year in native form, default time and date separators, ante- and post-meridiem strings for time expressions, an amazing number of alternate date and time patterns that are acceptable in a particular culture, and which week is the first one of a new year. In addition, DTFI provides the main access to the more than half-dozen calendars supported by .NET. After you read this, you will see how easy it is to use the facilities of DTFI, and will gain experience in writing code that functions correctly independently of language and world region.

System.Globalization.CultureInfo affects nearly every class in System.Globalization, and DTFI is no exception. In fact, an instance of CultureInfo normally contains a DTFI object (except in a special case, explained below). So, you may find it helpful to review our previous article about cultures in .NET in Internationalization With The .NET CultureInfo Class & C#. For an introductory article to DTFI, see Managing Culturally Correct Date and Time Formats in C#, and you may also want to glance through it for some basic information on DTFI usage in both client and web programming. Most of the material here is based on a single C# form that we wrote to illustrate and exercise the many methods and properties of DTFI. You will find it helpful to download the program and run it yourself as you read through the article.

Obtaining a DateTimeFormatInfo Instance

Most of the time, DTFI does its job without you being aware of it. For example, if we create an instance of DateTime and then display its contents with the ToLongDateString or ToShortDateString methods then we will obtain a reasonably well-formatted result that meets user expectations for the CultureInfo of the current thread. But, you have to work directly with a DTFI for some problems. For example, suppose you want to provide the user with a list of the days of the week in his or her language and highlight the start day. Using DTFI we can carry out this task very easily. Notice also that you don't have to research this information yourself.

// use the current thread culture
CultureInfo ci = CultureInfo.CurrentCulture;

// get a DateTimeFormatInfo from CultureInfo
DateTimeFormatInfo dtf = ci.DateTimeFormat;

// clear the listbox
lstDayNames.Items.Clear();

// get weekday list, full and abbreviated
string [] daynames = dtf.DayNames;
string [] abbrevdaynames = dtf.AbbreviatedDayNames;

// probably seven days but don't assume anything!
int len = daynames.GetLength(0);
// format the result using a string formatter (makes translation easier!)
for (int i = 0; i < len; i++)
lstDayNames.Items.Add(String.Format("{0} ({1})", daynames[i], abbrevdaynames[i]));

// choose the first day of the week in this locale and select it
int fd = lstDayNames.SelectedIndex = (int)dtf.FirstDayOfWeek;

You can write very similar code in ASP.NET, however, you will want to reflect the user's preferred culture not that of the server's thread culture. One way would be to use the HTTP_ACCEPT_LANGUAGE parameter. However, you want to be careful to create a specific culture. For example, suppose the user's browser language preference is Russian. In both IE and Netscape, the language tag transmitted to the server is "ru". If you instantiate a CultureInfo using new CultureInfo("ru"), you'll be out of luck getting to a DTFI since the CultureInfo object is a parent culture, and parent cultures have only a null value for their DTFI.

By contrast, a specific culture must be instantiated by a parameter that contains both language and regional content. Examples are en-UK for English (United Kingdom), ar-JO (Arabic, Jordan), fr-FR for French (France), etc. Why is a specific culture necessary? It is because a DTFI contains information that can be both region and language dependent. For example, the usual UK short date form of day/month/year differs from its US month/day/year counterpart, despite a (more or less!) common language. Another example can be found between the two German speaking regions Austria (de-AT) and Germany (de-DE), where the name of the first month of the year differs in spelling: 'Januar' in Germany, but 'Jänner' in Austria. In this example, both regions even share a common frontier, but an Austrian customer will be annoyed if your software mixes them up- and vice versa.

The implication is that if you create a parent culture - one that references language only - then the instance's DateTimeFormatInfo property is null. A similar comment holds for another formatter class, NumberFormatInfo. So, if you expect to be able to use culture-specific dates and times (or numbers, currencies, and percentages), use CultureInfo.CreateSpecificCulture to obtain a specific CultureInfo instance. Even if you instantiate it with a language only (i. e. "de", "en", "ar"), you will get a specific culture with a default region. Thus the solution to the problem above is solved by using the call CultureInfo.CreateSpecificCulture("ru"). Once you have a specific culture, you can use CultureInfo.DateTimeFormat to obtain a usable DTFI.

By the way, web programmers tend to disregard the browser language tag and depend more on a previously set cookie for such information. Even if set, it may not indicate a user's preferences. However, many web sites, including http://www.microsoft.com/?WROXEMPTOKEN=620568Zhiox25QOrrsnTPfm53p, seem to use it as an initial preference. For testing purposes, I often modify my browser language setting and later forget what I have done. More than once, I've found myself looking at Microsoft's Chinese or Indian web site wondering how I got there.

Basic Features of DateTimeFormatInfo

An instance of DTFI contains lots of useful information. Figure 1 (below) will give you an idea of its scope. In this case, the selected culture is English (United States), mainly to provide easy readability for most programmers. However if you have the demo program at hand, try viewing some of the other 100+ cultures available. You may not be able to read everything, but it will help you gain awareness and appreciation of how many different ways there are for handling date, time, and calendar related issues.

At the top of the form is a list of specific cultures obtained basically by calling the static method:

CultureInfo [] ci = CultureInfo.GetCultures(CultureTypes.SpecificCultures);

Internally, the enumeration is obtained indirectly. List and dropdown controls allow you to insert a reference to a class as an item in the list. If you do this, then the display string is determined by the ToString method of the object. Since the return from ToString is not always very user friendly, I derived a class from CultureInfo whose ToString method returns the CultureInfo.DisplayName property. The derived class is the one actually inserted into the culture dropdown list. Using this technique, the friendly name matches the current .NET runtime language. When I have to use the selected culture, I have all its methods and properties immediately available. Even better, the list is sorted correctly according to the current thread culture and I don't have to maintain any kind of internal data structure; the control is storing the data and doing all the work for me.

Figure 1: DateTimeFormatInfo Demo Program

Here is the code for the derived class.

using System;
using
            System.Globalization;

namespace WinDateTimeFormat {
public
            class MyCLInfo : CultureInfo {
public MyCLInfo(string name) :
            base(name) {}
public MyCLInfo(string name, bool useoverride)
: base(name, useoverride) {
}
public override string
            ToString() {
return base.DisplayName
}
}
}

The second controlling list in the demo program is the list of calendars of the culture. The .NET Framework supports eight calendars, although a detailed discussion of the Calendar class is beyond the scope of this article. However, the particular calendar in use greatly affects the content of a DTFI. In the demo program, you can view any combination of culture and available calendars. You may see more than one Gregorian calendar for a given culture. Even the en-US culture has two; one is US English and the other is Localized. Just below the calendar list you can see the Gregorian Calendar type name.

Here is a complete list of all supported calendars along with some long date displays. All displays represent the same instant of time. You can clearly see the impact a change of calendar might have on a string representation. By the way, a Calendar class provides a way for managing time increments and decrements. The action of adding a day or a month can produce quite different results from one calendar to another, and can be very difficult to do correctly, especially for lunar calendars. Are any of these alternate calendars useful in the real world? You bet, if you are doing business with a customer whose preferred calendar is one of them!

Gregorian Calendar | Sunday, November 11, 2001
Hebrew Calendar | 25 ???? 5762
Hijri Calendar | 25 ?????, 1422
Japanese Calendar | ?? 13?11?11?
Julian Calendar | Monday, October 29, 2001
Korean Calendar | ?? 4334? 11? 11? ???
Taiwan Calendar | 90?11?11?
Thai Buddhist Calendar | 11 ????????? 2544

By the way, you may need to add additional language support for you Windows 2000 or XP system to see some of these forms. To do this, get your installation CD and go to the Regional Settings dialog. Check the language groups you want, and insert the CD when requested.

Let's look again at Figure 1 to see a number of useful items. We have already discussed the native names of the weekdays and the first day of the week. But, you can also read a list of month and abbreviated month names, strings for the equivalent of 'AM' and 'PM' in other locales, default field separators for date and time expressions, calendar era names, and a value that may be new to you: the first week of the year. In fact, System.Globalization actually has an enumerator for this: the CalendarWeekRule:

FirstDay | The first week of the year starts on the first day of the year and ends before the following designated first day of the week. The value is 0.
FirstFourDayWeek | The first week of the year is the first week with four or more days before the designated first day of the week. The value is 2.
FirstFullWeek | The first week of the year begins on the first occurrence of the designated first day of the week on or after the first day of the year. The value is 1

Accounting and other rules for a given world region can be affected by such rules, and some years may have more or less than 52 weeks, particularly when the FirstFourDayWeek rule is in effect. Examples of locations using this rule include Denmark, The Netherlands, Estonia, Faro Islands, Finland, Switzerland, Germany, Iceland, Italy, Latvia, Norway, Poland, and Sweden. The only FirstFullWeek country seems to be Pakistan. Most countries use the FirstDay option.

Date and Time Options

The main feature of DTFI is the plethora of formats for date and time in an amazing number of combinations. For the U. S. alone you can find 129 formats. Why so many? The main reason is to be able to provide your customer with a complete list of culturally acceptable formats. You don't have to do the research - it has been done for you. In addition, there are a few specialized forms such as RFC 1123 and ISO 8601. All together, there are eleven categories with one aggregating all the rest. A glance at Figure 1 shows the available patterns on the left as dropdown lists, and the current selection on the right as formatted by DTFI according to the current culture and calendar. All values are based on DateTime.Now, which changes each time you select a new culture or calendar.

How do you enumerate each pattern category? The basic DTFI method is GetAllDateTimePatterns using a character parameter, or none at all if all formats are required. Early .NET documentation was very confusing on which characters to use; it took some experimentation to find out what really works. Fortunately, the help has improved with the 1.0 release. Below is a code fragment from the demo application that enumerates all the forms for the full date and time patterns (this group is selected by the characters 'f' and 'F') and inserts the results into a combo box. The routine also selects the default pattern if DTFI has a corresponding property for that category. Also note that the text direction is set to RightToLeft for bi-directional writing systems such as Arabic, Farsi, Divehi, Hebrew, Urdu, and Syriac. Actually, this step is not entirely necessary since .NET will manage right-to-left behavior correctly (including reorienting the controls) if the appropriate thread culture has been set. You can see this in the demo program by setting your user locale in the Regional Settings dialog to one the Arabic locales. A similar technique works when serving a web page from .NET; in the code-behind, set the thread locale accordingly in Page_Load().

// full date time
// clear the combo
            box
cbFullDateTimePattern.Items.Clear();

// enumerate the
            full date time patterns and load them
foreach(string pattern in
            dtf.GetAllDateTimePatterns('F'))
cbFullDateTimePattern.Items.Add(pattern);

//
            enumerate the full date, short time patterns and load
            them
foreach(string pattern in
            dtf.GetAllDateTimePatterns('f'))
cbFullDateTimePattern.Items.Add(pattern);

//
            there is a default FullDateTimePattern method.
// find it in the
            list and select it
cbFullDateTimePattern.SelectedIndex =
cbFullDateTimePattern.FindStringExact(dtf.FullDateTimePattern);

//
            set the text direction
txtFullDateTime.RightToLeft =
            textdir;

There are differences, by the way, if the formatting character is lower case or upper case. Not all are explained very well. In the code fragment above, 'f' enumerates all full dates with short times (no seconds) and 'F' provides full dates with long times. A similar comment holds for 'g' and 'G' when applied to general dates and times. Others are explicitly separated, such as 'd' for short date, and 'D' for long date. Others, such as year/month ('y', 'Y') and month/day ('m', 'M') seem to generate the same patters for both upper and lower case, and I have not been able to infer from examples whether there are significant differences.

DateTimeFormatInfo Patterns

As you have probably guessed, you can pass a date-time pattern obtained from DTFI directly to the appropriate overloaded ToString method of DateTime. You can even create your own patterns, although if you are trying to get a culturally correct result, this may not be advisable. Here is how a selected full date time pattern is used to create the corresponding output in the demo program. The code here is in response to a user changing the combo box selection containing the full date time format strings. A similar refresh takes place if the user changes the calendar.

// full date time
// dt is a DateTime instance created from DateTime.Now
// cbFullDateTimePattern is the combo box showing full date time patterns
// txtFullDateTime is a text box holding the formatted result
// curci is the currently selected CultureInfo instance

protected void
cbFullDateTimePattern_TextChanged(object sender, System.EventArgs e) {
try {
txtFullDateTime.Text = dt.ToString(cbFullDateTimePattern.Text,
curci.DateTimeFormat);
} catch {
}
}

A complete discussion of pattern types is documented in the .NET help. Look in the Index for DateTimeFormatInfo - about DateTimeFormatInfo class. However, just by looking at the demo program you will certainly figure most of them out. For example the full date pattern

dddd, MMMM dd, yyyy h:mm:ss tt

is interpreted as follows: use the full name of the weekday (dddd), the full name of the month (MMMM), a two digit number with leading zero for the day (dd), a four-digit year (yyyy), an hour with no leading zero (h), a two-digit minute with leading zero (mm), a two-digit seconds display with leading zero (ss), and a meridiem string such as AM or PM (tt).

Did you look carefully at Figure 1? In the long date pattern there is some English text, 'Today it is '. In fact, in the demo program all the combo boxes containing date patterns are writeable so that you can insert your own text. Be sure to enclose your text in single quotes, however. You can add text as needed when rendering displays and pass the resulting combination of text and date/time pattern to the ToString method. However, it is strongly recommended that you use a formatter to make the result localizable to other languages. Fragments or concatenated strings can be nearly impossible to translate because of differing word order, syntax, and grammar constraints. For example, you should format this string like this before passing it on to the DateTime.ToString method:

// cbFullDateTimePattern is a combo box
            holding a list of date time patterns
string
            strMyDate;
strMyDate = String.Format("Today it is {0}",
            cbFullDateTimePattern.Text);

Default Formats in DateTimeFormatInfo

Having all those alternate formats can be useful, but usually you are happy just to have one. Here is a table of date and time default properties, patterns, and examples sorted by the left column. I've chosen Czech to illustrate. The property names are self-documenting (if you read English!)

As a concluding comment, although the month name for February in Czech is únor, it is not used quite right here in most of the displays. The form shown is in the nominative case (brush off your grammar book - it is the dictionary form) but it should appear as "února" (the genitive or 'of' case) in most of the examples. Case endings in languages often take the place of prepositions. In this case you translate "11. února" as the "11th of February". The same problem exists in .NET with related languages such as Russian, Ukrainian, Polish, Slovak, and others. Declensional languages can add a new layer of complication in both internationalization and localization of programs and web sites and it is always best if such details can be handled by the runtime system. Interestingly, Microsoft does it right in Win32 in GetDateFormat, although it took them several years to get around to the task. I'm told that this oversight in .NET will be fixed 'soon'.

Conclusion

.NET provides a powerful and versatile way to manage date and time formats and to provide essential information about months of the year, days of the week, first day of the week and first week of the year, alternate calendars and eras, and a large selection of alternate date and time patterns that meet local cultural expectations. Of course, the ideas for the class have roots in Win32, and if you would like to obtain a useful utility that runs there, you can download it from http://www.mlmassociates.cc/dl-win32.htm?WROXEMPTOKEN=620568Zhiox25QOrrsnTPfm53p. Look for the nLocale application. You can also find quite a lot of information on the web site about internationalization on Win32 and .NET systems as well as links to some other sites dealing with the subject. You can also find a long list of references to articles and books there at http://www.mlmassociates.cc/biblio.htm?WROXEMPTOKEN=620568Zhiox25QOrrsnTPfm53p. Finally, don't overlook the Microsoft site http://www.microsoft.com/globaldev?WROXEMPTOKEN=620568Zhiox25QOrrsnTPfm53p, which provides an often-overlooked resource for international development.

Article Information
Author | Bill Hall
Technical Editor | Adam Ryland
Project Manager | Helen Cuthill
Reviewers | Andrew Krowczyk, Saurabh Nandu

If you have any questions or comments about this article, please contact the technical editor.


C#Today Search

Wrox Press
192 Articles and 38 Case Studies
in the Solutions Library 		Log off

HOME | SUBSCRIBE | MY PAGE | SITE MAP
Advanced Search
By Category:
 | Application Development
 | C# Language
 | Web Services
 | XML
 | Performance
 | Security
 | Data Access / ADO.NET
 | SOAP / Data Transfer
 | Graphics / Games
 | Site Design
 | Mobile
 | E-Commerce
 | Other Tech
Authors
Reviewers
Recent Content
Reference Articles
Tip Archive
Free Trial
C#Today News
Feedback
Content Suggestions
Write For Us
About C#Today
Related Sites:
www.wrox.com
ASPToday
Wroxbase
P2P
The link you followed is no longer valid, why not try and search for the content instead.
Full Text Search
Search: Sort by: Type:
between and
including: Articles Case Studies Reference Articles Tips Help

C#Today is brought to you by Wrox Press. Please see our terms and conditions and privacy policy.
Report problems to webmaster@csharptoday.com. Contact editor@csharptoday.com for affiliate opportunities.
Copyright © 2002 Wrox Press Limited. All Rights Reserved.